Draft: Montreal NETCONF Interim Meeting Minutes

[Andy Bierman <ietf@andybierman.com>]

Hi,

I would like the people who were at the meeting to check these
minutes for correctness.

thanks,
Andy

=========================================================================

[DRAFT]
Network Configuration WG (netconf)
Montreal Interim Meeting Minutes
July 14 and 15, 2006
=====================

Chairs:  

Simon Leinen <simon@switch.ch>
Andy Bierman <ietf@andybierman.com>

Notes: Andy Bierman, Juergen Shoenwaelder
Minutes: Andy Bierman

-----------------------------------------------

Agenda:

- Interim Meeting Location
 IETF Headquarter Hotel: Delta Centre-Ville
 777 University Street
 Montreal, Quebec, Canada, H3C 3Z7
 Hotel WEB site:
 http://www.deltahotels.com/hotels/hotels.php?hotelId=35

- Interim Meeting Times
 Friday, July 14, 2006 (1:30 PM - 6:00 PM)
 Saturday, July 15, 2006 (9:00 AM - 6:00 PM)

- Internet Drafts:

 - NETCONF Event Notifications
   draft-ietf-netconf-notification-02.txt

 - A SYSLOG Capability for NETCONF
   draft-shafer-netconf-syslog-00.txt

- Functional Requirements
 - determine WG consensus on need and priority of Juergen's list
   http://www.eecs.iu-bremen.de/wiki/index.php/Netconf_notifications

- Architecture

 - role of notifications within netconf
 - relationship to syslog and snmp notifications
 - scaling issues
 - security issues (session vs. signed msg, etc.)
 - standard RPC methods vs. new RPC methods
 - agent resource vs. NMS resource issues

- Use within Sessions

 - endless RPC
 - mixed-mode (rpc operations and notifications)
 - single vs. multiple subscriptions within a single session


- Transport Specific Support

 - use of channels (ssh, beep, what about soap?)
 - callhome support for agent initiated sessions

- Agent Notification Configuration and Control Mechanism

 - config-data vs. subscription-data
 - RPCs used to configure and control agent notification generation

- Notification Event Classes

 - WG consensus on specific list
 - Guidelines for selecting classifications

- Notification Filtering

 - Configuration mechanism(s)
 - Agent processing model

- Notification Header Content

 - Syntax and semantics of data included in all notifications

- Standard Notification Content

 - WG consensus on 0 or more fully-specified standard notifications
 - Full syntax and semantics of each standard notification

- Read-only (monitoring) data for agent notification activity

- Access Control Issues

- Data Model and XSD Issues

 - Data model design
 - XSD correctness
 - Need for 'min-access' and 'max-access'

- Documentation Issues

 - Terminology
 - Inclusion of non-normative text
 - Document section order
 - Document clarity
 - Choice of examples
 - Example correctness (need to validate against the XSD)

---------------------------------------------------------------

Minutes:

1) Functional Requirements

The interim meeting began with a continuation of the requirements
discussion began in the WG meeting.  The group tried to understand
and prioritize each requirement, as captured by Juergen on th
wiki page set up for this purpose.

The following is a list of requirements (Rn), optional comments (Cn)
on the issues discussed, and the resolutions (Sn) reached by the group.

R1) Scope of work

The solution should focus on notification support for configuration.

C1)

A basic disagreement within the WG is caused by the assumption
of different implied application infrastructures, which would
use a notification system in very different ways.  

The group interpreted this requirement to mean that configuration
specific notification content should be defined for some
configuration related tasks.

S1) Accepted

The solution is required to identify configuration changes on
a device (i.e., NETCONF agent).

R2) Hierarchical Data

The solution should support structured hierarchical data.

C2)

This requirement is superseded by the content-independent
requirement.

S2) Rejected

This requirement is not meaningful, since we have XML encoded
data already, and we do not focus on the content of a notification.

R3) Configuration Fragments

The solution should be able to carry configuration fragments.

S3) Rejected

This requirement is not meaningful since we do not focus on the
content of a notification.

R4) Message Size Limit

The solution should support a reasonable message size limit (syslog
and SNMP are rather constrained in terms of message sizes)

S4) Rejected (N/A)

The NETCONF protocol does not impose message size constraints.

R5) Reliable Notification Delivery

The solution should provide reliable delivery of notifications.

C5)

The WG discussed this issue at length on the mailing list.
The result of that discussion was that the underlying
transport protocol that NETCONF uses provides this feature.
Application level ACKs for notifications have been rejected
by the WG already.  

S5) Rejected

The WG consensus on this issue has not changed.
Application-level ACKs will not be supported.

R6) Pre-configured Notification Destinations

The solution should support pre-configured notification destinations.

S6) Deferred

The group determined this is a nice to have feature if there is
a solution (done somewhere else)- ISMS is investigating
whether there is a solution for SSH (seems to be easy over BEEP).
This is just a notification specific instantiation of the call home
feature (see below)

R7) Agent Initiated Connections

The solution should support agent initiated connections.

C7)

This is the Callhome feature.
After much discussion, it was determined this feature
should not be part of this notification work because
it is not notification-specific.

S7) Rejected

This is a transport issue and as such must be handled outside
the notification work.

R8) Subscription Mechanism

The solution should provide a subscription mechanism.

C8)

The actual meaning of this requirement was discussed at length.
There was agreement (at the start) that a special RPC call
to initiate notification delivery was reasonable, but
also significant differences on the structure of that
RPC method.

S8) Accepted

The term 'subscription' refers to the delivery of notifications
(if any to send), and is bound to the lifetime of a session, not
a more permanent subscription?

An agent does not send notifications before asked to do so
by the manager, i.e., the manager initiates the flow of
notifications.

R9) Multiple Subscriptions

The solution should support multiple subscriptions.

C9)

The term was clarified to mean multiple subscriptions
per session, since multiple sessions is already supported.

S9) Rejected

Multiple subscriptions over the same session are not required.

R10) Filtering Mechanism

The solution should provide a filtering mechanism.

C10)

There are many different interpretations of this feature,
and the group determined that notification filtering
means the manager somehow selects one of possibly
several notification content output formats, and
also somehow selects a subset of all possible
notifications, based on some portion of the
notification data content.

S10) Accepted

There continues to be WG consensus that this feature
is needed.

R11) Notification Names

The solution should support notification names.

S11) Rejected

Notification names belong into the data content.

S12) Notification Timestamps

The solution should support notification timestamps.

C12)

The group discussed the need for timestamps in the
notification header.  The following diagram shows
the 3 different timestamps that were considered:

  +----------+       +-----------+       +--------------+
  |          |       |           |       |              |
  |  Mgr App | <---- |  Agent Tx | <---- | Agent Detect |
  |          |       |           |       |              |
  +----------+       +-----------+       +--------------+
       TS-3               TS-2                TS-1


R12) Rejected

Notification timestamps (TS-1) belong into the data content.
A (possibly additional) timestamp (TS-2) to indicate when
the agent transmitted the notification is not needed.

R13) Notification Classes

The solution should support notification classes.

C13)

This has been a contentious topic all along due to
the overlapping classifications and the low probability
of interoperability of complex NE devices, based
solely on the text in the draft.

S13) Rejected

Notification classes belong into the data content.

R14) Notification Info

The solution should support notification info.

C14)

It was never really clear to everybody what this
feature really meant.  It was clear that this is
just part of the data model specification.

S14) Rejected

Notification info belong into the data content.

R15) Notification Content Specification

The solution should provide the ability to specify the content
of notifications to ensure predictability.

S15) Rejected

Notifications should be formally defined, to be addressed by data
modeling language efforts.

R16) Transport and Session Independent Notification Content

The solution should send sufficient information in a notification
so that it can be analyzed independent of the transport mechanism,
or the session-specific context.

C16)

This item deals with the identifiers used in the notification
header to identify its classification in various ways.
This feature was obsoleted by other related decisions.

S16) Accepted

Data content fully describes a notification; protocol information
should not be not needed to understand a notification

R17) Reference to Prior Configuration Change RPC Invocations

The solution should allow notifications to refer to prior
configuration change RPCs

S17) Rejected

This feature belongs into the data model content.

R18) Do not bind Subscriptions to Connections

The solution should not bind subscriptions to a connection.

C18)

It was unclear how this would ever work in NETCONF,
since NETCONF is session-based.

S18) Rejected

Sounds a bit too complex - unclear what the real use case is.

R19)Fate Sharing Notification Channels

The channels for configuration change notifications should share
fate with a session that includes a configuration channel.

S19) Rejected

Sessions are independent

R20) Notification Replay

The solution should support replay of locally logged notifications.

C20)

A manager needs to be able to distinguish between retrieval of
logged notifications and replay of 'fresh' notifications

This feature was explored in more detail later in the meeting,
but the basic decision did not change.

S20) Accepted (as capability)

Could be an optional feature.

R21) Message Chunking

The solution should support message chunking capability in
cases channels carry mixed RPCs.

S21) Rejected

Netconf does not interleave messages, no chunking/fragmentation
needed.

R22) Notification Transmitter Scaling

The solution should scale to 30.000-100.000 nodes which may
emit notifications.

C22)

R23 was removed because it is considered a duplicate of R22.

S22) Rejected

The netconf transport has already been decided.
It is a transport and implementation specific matter
how many concurrent sessions a single node can handle.

1.2) Requirements Evaluation Summary

R1) Scope of work is configuration         : Accepted
R2) Hierarchical Data                      : Rejected
R3) Configuration Fragments                : Rejected
R4) Message Size Limit                     : Rejected
R5) Reliable Notification Delivery         : Rejected
R6) Preconfigured Notif. Destinations      : Deferred
R7) Agent Initiated Connections            : Rejected
R8) Subscription Mechanism                 : Accepted
R9) Multiple Subscriptions                 : Rejected
R10) Filtering Mechanism                   : Accepted
R11) Notification Names                    : Rejected
S12) Notification Timestamps               : Rejected
R13) Notification Classes                  : Rejected
R14) Notification Info                     : Rejected
R15) Notification Content Specification    : Rejected
R16) Independent Notification Content      : Accepted
R17) Reference to Prior Config RPCs        : Rejected
R18) Do not bind Subscriptions to Conn.    : Rejected
R19) Fate Sharing Notification Channels    : Rejected
R20) Notification Replay                   : Accepted (as capability)
R21) Message Chunking                      : Rejected
R22) Notification Transmitter Scaling      : Rejected

2) Notification Delivery Control

The group agreed that the real 'standards' goal is to provide
for the delivery of standard NETCONF notifications without
the need for any proprietary configuration to be used at all.
There are different ways of approaching this problem,
which required a lot of discussion to resolve.

2.1) Streams vs. Subscriptions

The group discussed the details of the 'streams' design.

The essential difference between the architectures
in two drafts under consideration is that in the 'Streams'
approach, the device has a set of potentially many distinct
notification paths and data model encodings, and in the 'Subscription'
approach, there is one conceptual notification path with
a monolithic data model encompassing all possible data encodings
and notification content data models.

There are 2 distinct uses for multiple notification streams:

1) Different data encodings
   The same notification information may be encoded in multiple
   formats to accommodate legacy and new applications, such as
   syslog/RAW, syslog/COOKED, syslog/SD-Params, snmp/XML, etc.

2) Different event sources and/or event processing SW
   Not all events may be reported in all streams for various
   reasons.  For example, console syslog messages and
   SNMP notifications rarely coincide, event for event.
   
The application of session-specific parameters to connect to
a stream or subscription are basically the same.  There were
no real objections to the mechanisms proposed in the 'Streams'
approach.

After some clarifications, the following model was
agreed upon as the correct approach for the WG:

============================
 (Within 1 NETCONF Agent)

 Part 1:

 Stream X:
 +--------+      +--------------+      +---------+
 | Event  |      |    Event     |      |  Data   |
 | Source | ...  | Processor A  | ---> | Model A | ---> X
 +--------+      +--------------+      +---------+

 Stream Y:
 +--------+      +--------------+      +---------+
 | Event  |      |    Event     |      |  Data   |
 | Source | ...  | Processor B  | ---> | Model B | ---> Y
 +--------+      +--------------+      +---------+

 Stream Z:
 +--------+      +--------------+      +---------+
 | Event  |      |    Event     |      |  Data   |
 | Source | ...  | Processor C  | ---> | Model C | ---> Z
 +--------+      +--------------+      +---------+

 Part 2:

  +------------------------------------------------------+
  |  Session N                                           |
  | +---------+      +--------+     +----------+         |
  | | Stream  |      | Filter |     | Notif.   |     To  |
  | | Select  | ---> | Set    | --> | Delivery | --> Mgr |
  | | (X,Y,Z) |      | Config |     | Capabil. |         |
  | +---------+      +--------+     +----------+         |
  |                                                      |
  +------------------------------------------------------+

===============

2.2) Group Consensus Points

a) The solution will support multiple event streams.  The
  exact capabilities and characteristics of a stream are
  determined by the agent, and advertised to the manager.
  The manager can select 1 of N streams, and optionally
  apply notification suppression filters to that stream.

b) There will be a special named stream to carry "netconf"
  notifications, which is mandatory to implement.

c) There might be other special named streams to carry "snmp"
  or "syslog" notifications.

d) Filters are applied before a notification is shipped over a
  session. Filters are established as part of the stream selection
  (and they share fate with the underlying session).

e) An empty filter matches all notifications.

f) The configuration of streams is out of scope for the current work
  in this working group (part 1 above).  

g) The configuration of stream selection and notification
  suppression filters in in scope (part 2 above).

h) Notification suppression filtering is data-model and
  stream dependent.

i) It was noted that just because the WG can define a read-only
  data model which provides a 'snapshot' of the standard
  Stream or Filter parameters, the actual parameters used to
  configure these mechanisms may be different, and may not
  even map 1:1 to a given standard parameter.  Therefore,
  the configuration of device-specific streams are not in scope.


3) Notification Use Within a Session

3.1) Long-Lived RPC vs. Mixed-Mode RPC

It is assumed that a new RPC method will be created for
the manager to use to cause the agent to start the delivery
of notifications.  All parameters needed to configure
the session-specific delivery of notifications should
be passed as parameters (by reference or by value).
Additional features such as filtering and notification replay
are also supported via this new RPC method.

Once this new RPC method is invoked, there are three potential ways
to ship notifications:

A) One big (potentially never ending) <rpc-reply> to the initial
  request that sends multiple notifications and then eventually
  sends the closing <rpc-reply> end-tag to complete the RPC
  invocation.

B) This new RPC method is followed by an <ok> reply,
  followed by a sequence of notifications, each
  one carrying a single notification message (but no
  other RPCs interleaved).  It is unclear whether
  <rpc> requests received by the agent would be queued
  until the notification delivery is terminated,
  simply ignored, or caused an <rpc-error> to be returned.

C) Same as b) but interleaved requests are allowed, and supported
  by the agent.   This has been called 'mixed-mode'.  The agent
  will buffer and properly interleave <notification> and
  <rpc-reply> elements as needed, on each session using
  this operational mode.  This mode could be optional, and
  identified with an additional capability.

[Ed. - XML declarations omitted from examples.]

Example A:                  
S: <rpc-reply>
S:   <data>
S:     <notification>...
S:     </notification>
S:     <notification>...
S:     </notification>
S:   </data>
S: </rpc-reply>
S: ]]>]]>
                           
Example B:
S: <rpc-reply>
S:   <ok/>
S: </rpc-reply>
S: ]]>]]>
S: <notification>
S:   ...
S: </notification>
S: ]]>]]>

Example C - Mixed Mode:
S: <rpc-reply>
S:  <ok/>
S: </rpc-reply>
S: ]]>]]>
S: <notification>
S:   ...
S: </notification>
S: ]]>]]>
S: <rpc-reply>
S:   ...
S: </rpc-reply>
S: ]]>]]>
S: <notification>
S:   ...
S: </notification>
S: ]]>]]>
S: <notification>
S:   ...
S: </notification>
S: ]]>]]>

3.2) Consensus Points

a) The group decided to use "Approach B".

b) An agent is not required to process RPC requests until the
  notification stream is done.

c) A capability may be defined to announce that a server is capable to
  process RPCs while a notification stream is active on a session.

d) Every implementation should be correct  :-)

e) The use cases for modifying the subscription filtering
  parameters frequently were not widely accepted, although
  a great deal of time was spent trying to convince the group
  that the <modify-subscription> feature is important.

f) The WG needs to investigate the use of SSH channels
  for separation of operations and notifications.

  [ed. - does this imply multiple NETCONF sessions over
  a single SSH connection? What exactly does this mean?]

4) Notification Replay

The group discussed a replay feature, which is not the same
as a notification logging MIB.  Instead of using the <get>
operation to retrieve stored notifications, the agent sends
them as <notification> PDUs instead.

There are several agent implementation requirements related
to support for this feature:

4.1) Notification Storage

The actual number of stored notifications available for retrieval
at any given time is an agent implementation specific matter.
Control parameters for this aspect of the feature are outside
the scope of the current work.

4.2) Retrieval Mechanisms

There was strong agreement that the CLI operation "tail -f logfile"
is an important use case for the behavior of this feature.

Translation: Retrieval of all matching notifications since an
absolute or relative point in time, optionally followed by
any new notifications since the request was started.
In addition, a manager may request that all future notifications
that match the selection criteria are delivered, so the
request for stored and 'live' notifications can be mixed.

4.3) Live Notification Buffering

If an agent supports notification replay, then it must also
support the ability to defer delivery of new notifications
while replayed notifications are currently being delivered
on a particular session, thereby maintaining temporal order
of the notification stream.

4.4) Issues

a) How does a manager tell the difference between these 2
  'empty response' conditions?
   1) no notifications buffered for the requested time period
   2) no notifications occurred during the requested time period

b) Should there be a mode where stored and live notifications
  are mixed together, and temporal order is not preserved?

c) How can this feature be realized with the fewest number
  of options and permutations, while still maintaining
  enough implementation flexibility?

5) Notification Data Models

5.1) State and Statistics

A standard data model will be developed to provide some
important information to managers:

a) Available Streams

The streams supported by the agent are stored by the agent
for retrieval.

b) State Data

It is possible some notification management state data will
be standardized to help applications debug and interact
properly with the agent and other applications.

5.2) <special-get> vs. <get>

The group discussed the use of the standard <get> operation vs.
collections of specific (standard and proprietary) <get-xxx>
operations.  The biggest concern is that an ad-hoc collection
of RPC calls is not as interoperable as one RPC method
used with a common data framework.  The issue is whether
an agent must support the <get> or <get-config> operations
for each data set returned in a 'special get' operation.

One point raised was that a specialized operation might have the
feature that it can pull things together from different subtrees
and namespaces, e.g. <get-vlan-info> with parameter
42 might retrieve all information related to vlan 42.  Another use
for a 'special get' RPC is to simplify access (e.g., iteration
over data-model-specific 'chunks' such as rows in a table) to
data which would otherwise be difficult to access efficiently
using the standard <get> RPC.

The group agreed that the data expected for notifications did
not warrant the use of special get RPCs, do a standard data
model (for use with <get>) will be defined instead.

5.3) XML Data Hierarchy

There was some discussion of the need for a standard data
framework to give standard data a 'root', or more precisely,
a unique absolute Xpath expression that describes the
path from a conceptual XML root to any given element
in any namespace.

There are many different ways to organize XML data,
but the target namespace and the nested element names
are the two 'independent variables' that can be used
to place XML data somewhere in a conceptual tree.

The <config> or <data> element (used in many NETCONF
operations) is a placeholder for the conceptual root
of the XML data.  All Xpath expressions and other filters
that operate on NETCONF data use this point in the PDU
as the root of the filter expression.

It was noted that the NETCONF WG does not have to define
a data organization framework for the entire data modeling
universe, but rather just a small 'bootstrap' model
that focused on the NETCONF protocol itself.

The manner in which this small set of NETCONF standard data
should be organized is still an open issue.  There were some
on-the-fly proposals made, but they provided more questions
than answers:

 Q1: What is the granularity of a namespace?

 Q2: How does the granularity affect filter expressions?

 Q3: How should the element hierarchy be defined?

  a) /ietf/netconf/notifications (in 1 netconf-data namespace)
  b) /ietf:netconf/nc-data:notifications (in 2 namespaces)
  c) ???

The WG considers the data organization problem to be a high
priority work item.  An initial solution is required.
It is possible that this initial data model will later be
discarded in favor of a more comprehensive and widely
deployed data model framework (if and when that happens).
This is similar to how MIB-1 was later replaced by MIB-2
in the SNMP protocol.  It is critical to define even a
small set of standard data in order to begin interoperability
testing and deployment of the NETCONF protocol.


5.4) Filtering

The use of POSIX regular expressions is planned for this
feature.  There are still a lot of details that need to
be written down before an interoperable filtering
mechanism (i.e., same NETCONF behavior across agents)
is fully defined.

It was noted that the function library of XPATH 2.0 supports
regular expressions in the function library. XPATH 1.0 does
not have something like that.

The authors will propose some text in the next revision of
the Notifications I-D.

6) Document Integration

The authors of both documents under consideration will rewrite
the WG notification document, which will continue to be
the deliverable for this work item.

a) All the non-normative appendices will be removed.
  Data model specific issues need to be addressed with
  a separate WG effort.

b) The 'Streams' approach and other recent design decisions will
  be implemented in the next revision of the WG draft, and the
  appropriate text will be moved or adapted from the 'Syslog
  Capability for Netconf' draft.  New text will be written as
  needed.

c) A new version will be ready soon, hopefully in September.

7) Summary

The two drafts under consideration will be merged, and possibly
design approaches (as decided by the WG) will be attempted by
all the authors, in the next version of the 'Netconf Event
Notifications' draft.

Some specific feature mechanisms are data-dependent, and
a minimal set of data modeling issues must be resolved
by the WG in order for this notification work to be satisfactorily
completed.


---------------------------------------------------------------

Appendix A)  Attendee List  (Email addresses removed to reduce spam)

Andy Bierman
Simon Leinen
Juergen Schoenwaelder   
David Harrington
Dan Romascanu
Opiuier Festor
Sharon Chisholm
Rodu State
Hector Travino
David Perkins
Bert Wijnen
William Chow
Yoshifumi Atarashi
Hideki Okita
Vincent Cridlig
Martin Bjorklund
Balazs Lengyel
James Balestriere
Phil Shafer
Rob Enns
Brian Trammell
     

--
to unsubscribe send a message to netconf-request@ops.ietf.org with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/netconf/>