Re: comments on draft-bierman-sming-ds-02.txt

[Frank Strauss <strauss@ibr.cs.tu-bs.de>]

Hi!

[Probably the most controversial issue on SMI-DS is concerned with the
 naming scheme, which is discussed in a separate thread. However, here
 are some further comments/questions on some of the other issues we
 have raised.]


>> 6. We are not sure whether the goal of 100% forward and backward
>> compatibility has been reached or to what extend. The SMI-DS
>> translation to SMIv2 is not really defined in Appendix A and thus
>> it is hard to judge how much is in fact translatable (and what the
>> compatibility implication on the wire will be).

Andy> I agree this appendix isn't finished and should have more detail.
Andy> Keep in mind that the SMI is a description of object semantics.
Andy> It does not specify on-the-wire encoding. 

I agree that this is true for all core SMI (and thus SMIng) definitions.

But the SNMP naming issues that we are discussing are somewhat
inbetween these core SMI definitions and how to transport instances of
them on the wire.


Andy> SMIv2 semantics can be transparently converted to SMI-DS and
Andy> then transparently converted back to SMIv2, with no on-the-wire
Andy> changes in either direction.  Just as with SMIv2 -> SMIv1, there
Andy> are new features for which the conversion will not be
Andy> transparent.  Converting UNIONs and STRUCTs to SMIv2 is simply a
Andy> matter of adding lots of verbose text to every affected
Andy> DESCRIPTION clause, where the phrase "associated instance" is
Andy> used quite frequently.

I think, the SMI-DS/SMIv2 comparison is different from the SMIv2/SMIv1
comparison.  There is just one issue with SMIv2 which cannot be
handled by applications that support only SMIv1: Counter64 objects;
all other SMIv2 features compared to SMIv1 are transparent to
SMIv1-only applications. And even this `minor' compatibility issue was
so important for the past 9 years, that many standard MIBs support
Counter32 representations besides the new Counter64 objects.

SMI-DS is different: it brings significant new features that represent
the core benefits of SMI-DS that I would expect to be used in almost
all new SMI-DS MIBs, so that they are not compatible with SMIv2, i.e.,
they cannot be translated to SMIv2 because the instance naming is not
(generally) compatible.


>> 7. It is unclear whether compatibility on the wire between engines
>> that use SMIv2 style definitions and engines that use SMIng style
>> definitions (and naming) is still a design objective.

Andy> Please explain what on-the-wire problems exist

Existing SMIv2-aware and MIB-aware applications and those applications
that parse MIBs during runtime cannot work with SMI-DS peers. This is
caused due to the fact that what is shipped over the wire is different
in SMIv2 and SMI-DS.

An example: snmptable (from the NET-SNMP package) parses an SMIv2 MIB
during runtime and retrieves and shows the objects of a table/array in
a tabular form. If I would have written an SMIng MIB containing two
related classes mapped to two SNMP tables I could translate it to
SMIv2, so that snmptable could retrieve these tables from an agent
that implements this SMIng MIB. If the MIB would have been written
using SMI-DS, I would probably make use of the powerful data structure
containment, but I could not translate the SMI-DS MIB to SMIv2 so that
it could be used with snmptable. At least this is my understanding.


>> 9. We believe that a syntax which looks close to SMIv2 but means
>> sometimes something different is worse than a syntax which looks
>> different to SMIv2 and means something different. People who do not
>> understand the differences will merge SMIv2/SMI-DS in arbitrary
>> ways. This has happened with SMIv1/SMIv2.

Andy> Please provide examples where this is the case in SMI-DS.
Andy> It was certainly my intent to maintain semantics and syntax
Andy> as much as possible.  I think it would be more confusing to
Andy> MIB readers and writers to transition to SMIng syntax, in
Andy> which absolutely every macro and clause in changed.

I think such an explicit syntax change is in fact a benefit! I'm quite
sure that in case of SMIng MIB writers would first think and read the
specification (description and/or grammar), before they write
something down, that they are not yet familiar with. I'm not a fortune
teller, but in SMI-DS, I could imagine that people would write modules...

  - which start with ``MY-MIB DEFINITIONS ::= BEGIN'',
  - which include OBJECT IDENTIFIER assignments outside a NODES container,
  - which contain TEXTUAL-CONVENTIONs, OBJECT-TYPEs, etc.
  - which import base types from SNMPv2-SMI.
  - ...

just as there have been many people in the past who wrote SMIv2
modules containing TRAP-TYPEs, modules without a MODULE-IDENTITY
clause, etc.

Of course, a strict MIB checker/compiler would help a lot, so that
at least less people would publish illegal MIBs.


>> 13. We believe that an array should be similar to an ASN.1 SEQUENCE
>> OF, that is the array feature should work with an arbitrary
>> type. The SMI-DS ARRAY mixes containment and multi-valuedness in
>> one construct.  Programming languages such as C and even ASN.1 do
>> not do this.  Arrays do not yet exist in SMIng (only in the form of
>> tables in the protocol mappings).

Andy> Programming languages such as C allow both types of
Andy> constructs. [...]

No. In C an array of a struct type (thus not of a base type) always
requires a `struct', either explicitly defined or implicitly.


>> 14. We expect that not only base types, but also derived types
>> should be allowed in index clauses (just constructed types are not
>> allowed).

Andy> This is rather problematic and a big change from SMIv2.  How do
Andy> you INDEX an ARRAY with a UNION, or a complex STRUCT with nested
Andy> STRUCTs and UNIONs? This capability is not that important,
Andy> otherwise it would have shown up in the requirements document.

I think we are in agreement. We just talked about derived types (thus
single-valued typedefs based on a single base types), not constructed
types!


>> 16. We believe that SMIng achieves a much clearer and thus simpler
>> separation of potentially reusable type definitions and variable
>> declarations.

Andy> Can you provide an example of what you mean here? This is
Andy> a broad generalization.

In SMIng there is a strict separation of 

  - `class' and `typedef' construct, which represent the type and
    data structure definitons, and
  - protocol mappings all contained in a `snmp {...}' clause (or
    others for other protocols), and just referencing the class
    definitions but never defining new ones.


>> 21. We agree with the idea to hold compliance information in the
>> agent.  However, we do not understand the value in introducing
>> "pseudo OIDs". Why are conformance definitions not being kept in a
>> properly indexed conformance table? Conformance data is just data
>> and there should be no need for a special mechanism for this
>> purpose. The pseudo OIDs may give funny results when you do
>> getnext walks. (Also, the restriction that you can report
>> conformance to only a single compliance definition might hurt if
>> conformance statements are revised and you need to report multiple
>> compliance statement OIDs for continued interoperation.)

Andy> It would be very burdensome to maintain a MIB for the conformance
Andy> information, for every MIB object that ever gets defined.

In my opion, the burden of doing index allocation in such a separate
conformance table is less than the burden of mixing `usual' MIB data
and conformance information in the lexicographic MIB tree.

Andy> Since the conformance data is always the same for every
Andy> data object, it is easier to align the naming.  I suppose
Andy> it could be done either way.  A given implementation of a
Andy> given object instance can only comply with one MODULE-COMPLIANCE, 
Andy> so there is no need to allow for multiple compliance statement
Andy> OIDs.

No. E.g., ifDescr complies with ifCompliance3 *and* ifCompliance and
ifCompliance2 (via ifGeneralInformationGroup and ifGeneralGroup).


>> 24. Example 5.7.3:
>> 
>> - Would the example still work if an interface would first be
>> defined as a constructed type which is than assigned to a
>> variable or would this cause incompatible OIDs?

Andy> I actually struggled with this issue for awhile, and tried to
Andy> come up with a construct called UNWRAPPED STRUCT, but the
Andy> power of automatic hierarchical naming falls apart because
Andy> you can't arbitrarily strip the wrapper.  Defining the
Andy> ifTable as a STRUCT TYPEDEF would add an extra layer of
Andy> naming.  I think it's important that at least one
Andy> method for transparent conversion be provided. It isn't
Andy> a requirement that every possible way to define a given 
Andy> data structure be transparently compatible with SMIv2.

I have no suggestion how to solve this, but it's just my feeling that
it's confusing to have two ways of defining a complex type with equal
semantics from the data model viewpoint but resulting in different
OID naming schemes.


>> - We do not understand why there is a need for a new SIZE clause.

Andy> This is a machine-readable indication of the number of instances
Andy> that are expected/allowed for that ARRAY INDEX.  I suppose this
Andy> should usually only be relevant in VAR declarations, not
Andy> TYPEDEFs.  The DSMON example of a fixed sized array of 64
Andy> DSCP-to-Counter Aggregation groups is an example where it's
Andy> useful in a TYPEDEF, but this is the exception.

I think it's a too unusual case not justifying special language
support.  Even in this example it would be sufficient to let the MIB
compiler recognize from the SYNTAX Integer32 (0..63) clause that
there's a maximum of 64 array elements.


>> - We think that the introduction of INDEX clauses in parallel to
>> the SCALAR clause complicates things. This leads to several
>> special cases. How is the ordering of index components defined?
>> See for example the rules for MAX-ACCESS in INDEX clauses and
>> the optionality of subid assignments. We think it would be
>> simpler to keep the old notion of a separate index definition.

Andy> The order is specified in the document to be the order they
Andy> appear in the MIB, just like with SMIv2 INDEX clauses.
Andy> I think explicit INDEX clauses are more readable than
Andy> hunting around various MIBs looking for the OBJECT-TYPE
Andy> macros that match the descriptor identified in an SMIv2
Andy> INDEX clause.

This means you propose redundant (and potentially inconsistent)
information in case of `EXTERN INDEX' clauses. I would prefer to avoid
this redundancy. If you want to make the MIB more readable for humans
with requring to read external MIB definitions, the necessary
information can be provided in the table's/array's DESCRIPTION clause.
For MIB compilers it's no problem to refer the imported definition
(which should be done anyway).

>> - SPARSE-CONDITIONS can also go into normal description clauses
>> since it is just free form text. How do we decide that some text
>> is so important that it justifies its own clause?

Andy> This existence relationship between tables is used so often
Andy> that it is worth formalizing. 

SPARSE-CONDITIONS is no formal information, it's just free form text.


>> - Regarding SPARSE-EXPRESSION, what is the intended usage?  Is it
>> just documentation clarity? Is the intention to support
>> automatic test tools? Do these machine readable expressions
>> significantly simplify/improve code generators? And what after
>> all does the expression language look like?

Andy> This is intended to be a machine readable version of the
Andy> SPARSE-CONDITION. The syntax is TBD, but I would like
Andy> to use a subset of the PolicyScript syntax, found in the PM MIB.
Andy> If a simple expression is possible, then tools could take
Andy> advantage of it. If a complex expression is needed, then this
Andy> clause would be omitted, rather than making the expression
Andy> syntax too complicated.

I understand your description of the SPARSE-CONDITION, but I still
don't see the benefit (see the questions (except the last) in our
paragraph above). Do the benefits justify the measures of such a
specific `language within a language'.


>> 26. Augments Example:
>> 
>> - We guess that the SIZE clauses in the definition of inetAddrType
>> and inetAddr are just cut & paste errors.

Andy> no. this refers to the number of allowed instances of the INDEX
Andy> component, not the number of octets in an individual instance.

So, supplying this SIZE information more than once is redundant?  Or
what would it mean to supply different SIZE clauses for ifIndex,
inetAddrType, and inetAddr? Maybe, you would prefer to move the SIZE
clause from the INDEX clause to the ARRAY clause?


>> - We are still undecided what the added value of arrays is. It
>> looks like SMI DS arrays are pretty close to tables except that
>> they can conceptually be embedded due to the hierarchical naming
>> scheme.

Andy> The ability to embed any of the complex data types in a
Andy> manner consistent with modern programming languages is
Andy> important.

Agree. But that's no different from SMIng where a class may contain
attributes of another class type.


>> 27. The VAR STRUCT format is unclear. It sometimes contains other
>> SCALAR/STRUCT/UNION/ARRAY clauses (see ipXStats example) and
>> sometimes it just contains flat SYNTAX etc. clauses (see myAddress
>> example).  From reading the grammar, it looks like both are
>> allowed? What is the difference between both formats in case I
>> just have a single STRUCT, except that you safe a few lines in
>> some cases?

Andy> Data definitions are dictated by the problem being addressed
Andy> by the MIB author. This is no different than a C programmer
Andy> defining data types for a program. Sometimes they only
Andy> contain base types, and sometimes they contain some complex
Andy> types. SMI-DS provides the capability, and it's up to the
Andy> MIB designer to define appropriate data types for the
Andy> problem being addressed.

Ok. You explained why an VAR STRUCT clause may contain
SCALAR/STRUCT/UNION/ARRAY clauses. I think we understood this and we
agree. But we do not yet understand why sometimes a VAR STRUCT clause
may contain a flat SYNTAX clause?


 -frank