Re: hierarchical naming

[Juergen Schoenwaelder <schoenw@ibr.cs.tu-bs.de>]

>>>>> Andy Bierman writes:

>> (b) All existing SNMP applications and tools (except primitive
>> snmpwalk tools) are based on the notion of conceptual tables and a
>> "flat" OID space. They won't work with SMI-DS nested naming. To
>> make them work, you have to "unfold tables" (as you say in section
>> 8.3 in your ID).

Andy> I don't think this is a significant factor.  One could claim
Andy> many years ago that no existing applications understand
Andy> Counter64 (or even Unsigned32) so these data types cannot be
Andy> added. Applications have to change to use brand new
Andy> features. IMO, the advantages of hierarchical naming outweigh
Andy> the disadvantages.

What do you think are the advantages? How does hierarchical naming in
the OIDs reduce the costs of implementing agents with current protocol
operations?

>> (c) Since SMI-DS assumes RFC 1905 protocol operations, you can't
>> get/set "complex" objects. You still ship lists of varbinds of the
>> leaf attributes - the same list of attributes you would be shipping
>> with SMIng or SMIv2. Only the OIDs differ (usually making the
>> messages a bit longer, but that is not relevant here).

Andy> The same is true for SMIv2, SMIng, or SMI-DS. Macros for
Andy> not-accessible constructs cannot be transported.  The purpose of
Andy> the SMI is to express management information in the most
Andy> efficient and interoperable manner possible.  I don't see what
Andy> ProtoOps have to do with the evaluation of different SMI
Andy> approaches.

The structural aspects are almost the same in SMIng and SMI-DS. (We
have some differences in the details but I am sure there is room to
work them out). The hierarchical naming is of course the major
difference. From the language point of view, it is easy to adapt SMIng
to do hierarchical naming as well.

My point is that hierarchical names IMHO give real benefits if I can
use them to access (nested) objects rather then simple variables that
represent aspects of a "virtual" object. It makes a difference whether
I can create a complex object (preferably in one and only one way) or
whether I have a bunch of variables that I can ship in arbitrary
orders spread across multiple transactions in order to create the
complex nested object. Without protocol operations that take advantage
of the fact that hierarchical naming provides names for "objects"
rather than just "variables", I just fail to see the big win.

Note: I am not at all negative about your work and nested naming. I am
just looking to understand where the gain is when we use hierarchical
names with our existing SNMP.

>> (d) So the question boils down to: What do you gain by having
>> hierarchical names in the OIDs (under the assumption of RFC 1905
>> protocol operations)? I did not yet find a good convincing answer
>> to this question. Note that a code generator which understands
>> SMIng does know the data organization of the nested structures
>> before they were flattened out and thus it can generate code to
>> reconstruct the nested data structure from the varbind list without
>> requiring the nesting information to be present in the OID. I did
>> not find a way to do the same thing dynamically at run-time since
>> you generally do not know where the OID nesting ends and the
>> instance identifier starts.

Andy> I don't think it's reasonable to assume the SNMP engine/toolkit
Andy> will efficiently rearrange nested sub-aggregates, to allow for
Andy> the same type of reusability that could be achieved if instances
Andy> of the same complex data type were named in a consistent
Andy> (hierarchical) manner. The engine may not know where the
Andy> instance OIDCs start, but the reusable code to validate a
Andy> specific data type does.

My assumption is that a reasonable SNMP toolkit converts received
varbind lists into implementation language data structures derived
from MIB modules before any callbacks are invoked. Right now, all such
a tool can generate (without additional hints) are simple structures.
With SMIng and SMI-DS, we can define in the MIB module more complex
data types and thus a smart toolkit will take advantage of this and
generate code to convert received varbind lists to more complex
implementation language data structures. And since SMIng and SMI-DS
allow to reuse (complex) data types, chances are good that smart
toolkits can produce more compact implementation code. With current
SNMP operations, all this happens at compile-time and at run-time, all
you need are unique OIDs - I think it does not really matter much
whether whether they contain some information about the nesting or
not.

Perhaps I am just missing something. Do you have an algorithm in mind
which you can outline in pseudo-code which shows me how a command
responder (which current SNMP protocol operations) can take advantage
of having hierarchical OIDs when it processes a varbind list?

>> (e) The SMI-DS ID says in section 5.5 that hierarchical naming can
>> help to reduce implementation complexity for agent and application
>> developers for SNMP Set operations. The only benefit I can see is
>> that you can reduce a few RowStatus objects by the expense of
>> having larger set operations. I do not see how the hierarchical
>> naming scheme affects the complexity of having to support
>> arbitrariluy ordered and arbitrary complex set operations and the
>> issues with createAndWait and notInService states.

Andy> I think the ease-of-use factor is important. 

Ease of use for whom? The MIB author / reader?

Andy> The current SMIng drafts require manual and explicit unfolding,
Andy> flattening, and mapping of the natural data structure into an
Andy> SMIv2-style row.

Yes.

Andy> I don't see how SW will efficiently take advantage of the
Andy> hierarchical nature of the data structures once this is
Andy> done.

As I said before: Programs that generate manager or agent stubs now
know what the complex structures were that the MIB author had in mind
and can do something useful with this information. And old programs
that do not understand SMIng will continue to function since the OID
assignement basically defines the conversion to SMIv2. Furthermore, it
is possible to map existing SMIv2 MIBs into SMIng and at the same time
providing more information so that you can take advantage of this.
Frank and I have been very conversative because we believe that
forward and backward transitions are very important give the existing
number of SMIv2 MIBs the community has defined in the last several
years.

Andy> This important relationship should be self-describing
Andy> (like XML). I think new applications that wish to take advantage
Andy> of new SMI features will be more hindered than helped by
Andy> flattening complex data objects.  Both SMIng and SMI-DS agent
Andy> code will easily co-exist with SMIv2 agent code, because the
Andy> <oidBase> structure is identical, and SNMP engines dispatch by a
Andy> longest <oidBase> match of the registered instrumentation
Andy> handlers.

With the current SNMP and all the many applications and toolkits that
assume a flat OID naming system based on rectangular tables, your
agent will have to provide flat OIDs in addition to hierarchical OIDs
for a long long time. Without protocol operations that take advantage
of the fact that we now can address complex objects, I just fail to
see the savings.

>> With my current understanding (which might miss important pieces -
>> if so, please correct me), weighting (b) against (d) and (e), I
>> think that the current SMIng approach provides a much better
>> transition path since we can continue to use all existing tools but
>> we can improve them to be much smarter in efficiently handling
>> complex (reused) data structures.
>> 
>> Sure, if you break (a) by defining new protocol operations which do
>> not anymore operate on a list of lexicographic variables but on
>> structured objects, things will be much different. But we do not
>> have such a protocol nor do we have proposals underway in EOS nor
>> does the charter tell me that we should work under the assumption
>> that we have such a protocol.

Andy> I guess it's a judgement call as to which type of new code is
Andy> easier to write, and which will have a smaller footprint.  We
Andy> seem to agree on the value of hierarchical data structures, but
Andy> not how they should be mapped to OIDs. This is significantly
Andy> further along than we were at the last IETF.

As I said before: I am not totally negative or against hierarchical
names. I just want to make it clear that I believe this step has
serious implications on the existing applications/tools that assume a
flat tabular naming style and that it only plays off its full
potential if there is support for "complex" objects rather than simple
typed variables in the protocol.

<soap>
  If the IETF (not this WG) would be going down the path to manipulate
  objects rather than lists of typed variables, then you can ask
  whether this is still SNMP. Sure, today's Ethernet works quite 
  different from the good all yellow cable and it is still called
  Ethernet. But Ethernet had a good branding while SNMP ...]
</soap>

<not-so-soap>
  The cool thing about SMIng is that it allows to map to protocols
  which manipulate objects rather than lists of typed variables by
  doing another protocol mapping. ;-)
</not-so-soap>

<much-more-soap>
  Perhaps it is time to redo COPS-PR by creating COPS-NM - a protocol 
  which ships real nested objects instead of just flat table entries.
</much-more-soap>

/js

-- 
Juergen Schoenwaelder    <http://www.informatik.uni-osnabrueck.de/schoenw/>