Re: comments on draft-bierman-sming-ds-02.txt

[Juergen Schoenwaelder <schoenw@ibr.cs.tu-bs.de>]

>>>>> Andy Bierman writes:

Andy> thanks for reviewing 31 of the 83 pages. That's a good start, I
Andy> guess.  I will try to respond to the technical questions
Andy> raised. I think terminology can be finalized after the technical
Andy> details are resolved.

We have actually read all pages. However, we prefer to stick to the
fundamental technical contributions at this point in time rather than
discussing the syntax specification or your view on how SMI-DS
achieves the objectives stated in RFC 3216.

I also believe that trying to converge on a common terminology makes
discussions simpler and more efficient. It took some time for us to
figure out what your terms mean to us (and the same might be true the
other way round). And I guess this is even more true for people who
just want to follow the WG process.

>> 6. We are not sure whether the goal of 100% forward and backward
>> compatibility has been reached or to what extend. The SMI-DS
>> translation to SMIv2 is not really defined in Appendix A and thus
>> it is hard to judge how much is in fact translatable (and what the
>> compatibility implication on the wire will be).

Andy> I agree this appendix isn't finished and should have more
Andy> detail.  Keep in mind that the SMI is a description of object
Andy> semantics.  It does not specify on-the-wire encoding. 

I agree - on the wire encoding is the wrong phrase. What we meant is
the assumptions that existing applications and tools have how OIDs are
stuctured.

Andy> SMIv2 semantics can be transparently converted to SMI-DS and
Andy> then transparently converted back to SMIv2, with no on-the-wire
Andy> changes in either direction.  Just as with SMIv2 -> SMIv1, there
Andy> are new features for which the conversion will not be
Andy> transparent.  Converting UNIONs and STRUCTs to SMIv2 is simply a
Andy> matter of adding lots of verbose text to every affected
Andy> DESCRIPTION clause, where the phrase "associated instance" is
Andy> used quite frequently.

You do not say what will be mapped and what will be lost. Lets try an
example: How does a struct which contains another struct, a union and
an array look like once translated to SMIv2?

>> 7. It is unclear whether compatibility on the wire between engines
>> that use SMIv2 style definitions and engines that use SMIng style
>> definitions (and naming) is still a design objective.

Andy> Please explain what on-the-wire problems exist

See above. We meant compatibility of how existing applications and
tools expect OIDs to be formed.

>> 8. The proposed new naming touches both the language and the
>> protocol, at least in the sense that no existing implementation we
>> are aware of will be able to handle hierarchically named variables
>> in a meaningful way. We would expect that 100% forward/backward
>> compatibility implies that I can use existing tools to look at data
>> implemented by an agent whose implementation is based on an SMI-DS
>> MIB module.

Andy> The new hierarchical naming is identical to SMIv2 for all the
Andy> constructs currently available in SMIv2.  For generic
Andy> applications which have no knowledge of the objects they are
Andy> manipulating (e.g. mibwalk), all the protoOps work exactly as
Andy> before, and they still work. For applications that need to
Andy> understand the objects they use, the semantics of the object and
Andy> the structure of the instance portion of the OID need to be
Andy> known.  This is no different than the current situation with
Andy> SMIv2.

Does this mean a struct which contains another struct, a union and an
array will just not exist in the SMIv2 mapping? I do not think you
mean this since it conflicts with the stated requirement in SMI-DS:

   - Maintain 100% forward and backward translation compatibility with
     SMIv2. [...]

>> 9. We believe that a syntax which looks close to SMIv2 but means
>> sometimes something different is worse than a syntax which looks
>> different to SMIv2 and means something different. People who do not
>> understand the differences will merge SMIv2/SMI-DS in arbitrary
>> ways. This has happened with SMIv1/SMIv2.

Andy> Please provide examples where this is the case in SMI-DS.  It
Andy> was certainly my intent to maintain semantics and syntax as much
Andy> as possible.  I think it would be more confusing to MIB readers
Andy> and writers to transition to SMIng syntax, in which absolutely
Andy> every macro and clause in changed.

Frank provided examples. SMIng as well as SMI-DS are trying to make a
big step forward on how we describe management data. Trying to give
the impression that this did not happen has more potential to mislead
users about what they are doing. You obviously noticed while working
on SMI-DS some some syntactic constructs show be harmonized and so you
changed some SMIv2 constructs which I am sure others will regard as
unnecessary change. I guess we can debate this endlessly.

>> 13. We believe that an array should be similar to an ASN.1 SEQUENCE
>> OF, that is the array feature should work with an arbitrary
>> type. The SMI-DS ARRAY mixes containment and multi-valuedness in
>> one construct.  Programming languages such as C and even ASN.1 do
>> not do this.  Arrays do not yet exist in SMIng (only in the form of
>> tables in the protocol mappings).

Andy> Programming languages such as C allow both types of constructs.

No.

>> 14. We expect that not only base types, but also derived types
>> should be allowed in index clauses (just constructed types are not
>> allowed).

Andy> This is rather problematic and a big change from SMIv2.  How do
Andy> you INDEX an ARRAY with a UNION, or a complex STRUCT with nested
Andy> STRUCTs and UNIONs? This capability is not that important,
Andy> otherwise it would have shown up in the requirements document.

See, we do not use the same terminology and thus we do not understand
each other...

>> 17. The relationships to SMIv2 are interesting. You change
>> NOTIFICATION-TYPE to NOTIFICATION for language consistency reasons,
>> which we have sympathy for. I am sure someone else will see this as
>> a gratitious change. You also claim that lots of other constructs
>> do not change as well, including MODULE-COMPLIANCE.  Even if this
>> is true for the syntax (which I am not sure of), I am sure that
>> semantics will have to change.

Andy> This is not a gratuitous change.  The syntax structure of SMIv2
Andy> is "<descriptor> <construct-type> <construct>".  The syntax
Andy> structure of SMI-DS is "<construct-type> <descriptor>
Andy> <construct>" which is much more consistent with programing
Andy> languages.

It is nice to see how it varies over time what you consider a
gratuitous change. The syntax of SMIng has gone through a process
of intense discussions and we believe that there are almost no
gratuitous changes. It is nice to see that you value a consistent
structure and even something which "is much more consistent with
programing languages." (I could have written that myself. ;-)

>> 20. It is unclear how much savings the new naming scheme provides
>> on agents. With SMI-DS, you probably have fewer RowStatus objects
>> since you do not need them for "contained" multi-valued
>> attributes. You also can improve code stubs since the compiler now
>> knows that one table is really just used to hold multi-valued
>> attributes of some other object. We fail however to understand why
>> the hierarchical naming simplifies things such as (a) reducing the
>> degrees of ordering freedom we have in SNMP or (b) reducing the
>> need to have createAndWait/notInService states in the first place.

Andy> Removing the multiple interacting r/c tables will help reduce
Andy> agent complexity because the object ancestry is fully specified
Andy> in every object instance. Using one top level RowStatus (or none
Andy> at all with new EOS support) simplifies agent code a great deal.

You can do one RowStatus today - but it only works if you can ship all
the stuff which depends on the RowStatus in a single set request. I do
not see how SMI-DS naming changes that at all.

Andy> MIB design dictates whether createAndWait or notInService are
Andy> needed. SMI-DS does change that.  

How? I did not find anything specific on this issue in SMI-DS. Where
do I have to look at? BTW, the DiffServ MIB also does not require
createAndWait or notInService and is written in SMIv2.

Andy> It is important to maintain the hierarchical structure of
Andy> complex data types so generic functions can be written to
Andy> process them, no matter where they occur for a given usage. 

We agree on this. And SMIng supports that since the generic functions
deals with the data type - not necessarily with the name.

Andy> The SMIng proposal will require that the SNMP engine have
Andy> knowledge of every usage of every such construct, and rearrange
Andy> the OIDs manually to achieve what SMI-DS provides automatically.

I am not sure I agree or disagree. It probably depends on which agent
toolkit you are used to. You only do it manually if you do not use
compilers to do all the low-level OID smashing - which I prefer.

Andy> Forcing humans to manipulate the OID ordering in protocol
Andy> mapping clauses is not workable.

As you said before, there are a few MIB authors but much more MIB
readers. Explicit mapping to SMIv2 optimizes for the readers - they
can easily lookup what they get. Using an algorithmic mapping
optimizes for the authors for the expense of the users (that now have
to apply the algorithm somehow).

Andy> It means that every time a new object is added to an embedded
Andy> construct, every MIB that uses that construct has to be updated
Andy> to assign the new embedded node to a different OID.  This is not
Andy> consistent with the use of TEXTUAL-CONVENTIONs today.

You can't update textual conventions in this way so I fail to see your
point. But despite this, I think that manual propagation of changes is
a feature. I doubt that a system where you can affect hundreds of MIBs
just by adding something to a reused struct is desirable in practice.
In analogy to C, if you change a struct, you better recompile all
modules that use this struct or you are lost since variables now have
a different memory layout. Our "variables" are SMIng protocol mappings
and it is thus not surprising that you have to update them. The good
news is that old implementations will continue to interoperate with
new implementations - they will just not see the additions.

>> 21. We agree with the idea to hold compliance information in the
>> agent.  However, we do not understand the value in introducing
>> "pseudo OIDs". Why are conformance definitions not being kept in a
>> properly indexed conformance table? Conformance data is just data
>> and there should be no need for a special mechanism for this
>> purpose. The pseudo OIDs may give funny results when you do getnext
>> walks. (Also, the restriction that you can report conformance to
>> only a single compliance definition might hurt if conformance
>> statements are revised and you need to report multiple compliance
>> statement OIDs for continued interoperation.)

Andy> It would be very burdensome to maintain a MIB for the
Andy> conformance information, for every MIB object that ever gets
Andy> defined.  Since the conformance data is always the same for
Andy> every data object, it is easier to align the naming.  I suppose
Andy> it could be done either way.  A given implementation of a given
Andy> object instance can only comply with one MODULE-COMPLIANCE, so
Andy> there is no need to allow for multiple compliance statement
Andy> OIDs.

We were not talking about a MIB for every MIB object. We were talking
about a single conformance MIB which works for all object types ever
defined. Take your SMI-DS fragment and turn it into SMIv2 tables and
index that tables with the OIDs of object-type macros.

See Frank's response wrt. multiple compliance statements.

>> - We do not understand why there is a need for a new SIZE clause.

Andy> This is a machine-readable indication of the number of instances
Andy> that are expected/allowed for that ARRAY INDEX.  

Expected or allowed? This is not the same. The examples usually just
repeat the range restriction - which is kind of useless if the meaning
is "allowed" and somewhat impractical if the meaning is "expected" and
the value is 2^31-1.

>> 25. Data structure augmentations:
>> 
>> - What is the expected value of introducing SPARSE-CONDITION and
>> SPARSE-EXPRESSION?

Andy> To provide human and machine readable definitions of the
Andy> existence relationship between the augmented and augmenting
Andy> table.  SPARSE-AUGMENTS is a concept that has been around for
Andy> years, and buried in DESCRIPTION clauses.

Is free form text in a SPARSE-CONDITION clause any different from free
form text in a DESCRIPTION clause?

>> - SPARSE-CONDITIONS can also go into normal description clauses
>> since it is just free form text. How do we decide that some text is
>> so important that it justifies its own clause?

Andy> This existence relationship between tables is used so often that
Andy> it is worth formalizing.

SMIng formalizes the relationship by way of "extends" and "expands"
(proposals for better keywords are welcome). SPARSE-CONDITIONS just
add add another informal documentation clause (I use formal here in
the sense that machines can read and understand it).

>> - Regarding SPARSE-EXPRESSION, what is the intended usage?  Is it
>> just documentation clarity? Is the intention to support automatic
>> test tools? Do these machine readable expressions significantly
>> simplify/improve code generators? And what after all does the
>> expression language look like?

Andy> This is intended to be a machine readable version of the
Andy> SPARSE-CONDITION. The syntax is TBD, but I would like to use a
Andy> subset of the PolicyScript syntax, found in the PM MIB.  If a
Andy> simple expression is possible, then tools could take advantage
Andy> of it. If a complex expression is needed, then this clause would
Andy> be omitted, rather than making the expression syntax too
Andy> complicated.

Not sure what a simple expression is. Not sure what a program would
be able to do with it. I can't say what the value is or is not.

>> 26. Augments Example:
>> 
>> - We guess that the SIZE clauses in the definition of inetAddrType
>> and inetAddr are just cut & paste errors.

Andy> no. this refers to the number of allowed instances of the INDEX
Andy> component, not the number of octets in an individual instance.

So the SIZE belongs to the whole array and not to the INDEX components.

>> 27. The VAR STRUCT format is unclear. It sometimes contains other
>> SCALAR/STRUCT/UNION/ARRAY clauses (see ipXStats example) and
>> sometimes it just contains flat SYNTAX etc. clauses (see myAddress
>> example).  From reading the grammar, it looks like both are
>> allowed? What is the difference between both formats in case I just
>> have a single STRUCT, except that you safe a few lines in some
>> cases?

Andy> Data definitions are dictated by the problem being addressed by
Andy> the MIB author. This is no different than a C programmer
Andy> defining data types for a program. Sometimes they only contain
Andy> base types, and sometimes they contain some complex
Andy> types. SMI-DS provides the capability, and it's up to the MIB
Andy> designer to define appropriate data types for the problem being
Andy> addressed.

Are you saying: "There is no difference except that you save a few
lines in some cases"??

/js

-- 
Juergen Schoenwaelder    <http://www.informatik.uni-osnabrueck.de/schoenw/>