Transactional semantics and it's effect on the SMI

[Steve Waldbusser <waldbusser@nextbeacon.com>]

The purpose of this message is to bring up another requirement for our
new SMI and to discuss some of the alternatives for meeting this
requirement.


The current SMI does not allow for specification of what columns must,
may, or must not be included in a create transaction. It was felt that
doing so would unnecessarily limit flexibility and that
implementations should accept any combination of columns. While this
was a nice goal in theory, it worked poorly in the real world.
Implementations had a choice:

1. Allow any combination of columns - the method routine was more
   complicated because it had to keep track of what columns were set and

   which were empty. This also caused unworkable interactions with the
   device because most systems didn't otherwise know how to deal with
   partial rows (i.e. the routing code doesn't know what to do with
   partial rows).

2. Choose a certain set of columns as mandatory, others as optional,
   and others as unsupported. The problem with this is that there is
   little chance of interoperability. If a manager assumes a certain
   set of columns as mandatory and that set is different than the
   agent's assumption, only a lengthy trial and error process can
   discover the right combination. Even if possible, it's unclear if
   the user's desires would be fulfilled by the resulting transaction
   (was it really OK to just silently omit the ipRouteMetric3 column
   because the agent didn't support it?). I like to call this last
   issue 'backpressure' and it's a thorny issue - for example, what if
   the user isn't present at a console to answer the question
   "ipRouteMetric3 not supported. OK to skip?". Applications would
   need to be completely rewritten to propagate this backpressure
   "upstream" to the point of data entry (in other words, the user
   needs to provide instructions as to what to do if a column is
   unsupported).

To make creates easier to implement and to provide a solid foundation
for interoperability, we must address this issue. This requirement
raises protocol issues and it also raises SMI issues.

The following are some observations and discussion about the solution:

Sometimes an agent requires a varbind be part of a transaction
(mandatory).
Sometimes an agent can accept it but it isn't required (optional).
Variables that aren't mandatory or optional will fail (must-not).
I don't think you can list the must-not variables because it is the
universe of variables minus the mandatory and optional vars. So the
suggestion is to list mandatory and optional variables.

The SMI must document these mandatory/optional lists as the
definitions of create transactions and directives to agents of minimum
requirements and to managers of what they can depend on.
The syntax might look like the following clauses added to an
ipAddrEntry object:
    CREATE-TRANSACTION { ipAdEntAddr, ipAdEntIfIndex, ipAdEntNetMask }
    CREATE-OPTIONAL { ipAdEntBcastAddr, ipAdEntReasmMaxSize }
where the CREATE-TRANSACTION clause defines the basic transaction
with the mandatory list and the CREATE-OPTIONAL lists the additional
optional columns, any of which may also be included.

Sometimes when a manager sends a variable it requires that it be
processed and wants the agent to fail if it isn't supported.
Sometimes when a manager sends a variable it wants the agent to
process it IF it is supported, but allow the transaction to succeed
even if it isn't supported. This introduces an orthogonal
mandatory/optional designation that describes the desire of the
manager.

Thus the protocol should also have the capability of describing
whether included varbinds are mandatory or optional. A mandatory column
that isn't supported by the agent would cause the transaction to fail,
while an optional column that isn't supported by the agent would be
ignored.  It would be possible for a manager to take mandatory
columns (as per the MIB) and designate them optional in the
transaction which would make the transaction more resilient to
non-conformant implementations. It would also be possible for a
manager to designate optional columns (as per the MIB) as mandatory
in the transaction in which case the manager should be aware that the
transaction might fail on conformant implementations. In either case,
the transaction will represent the desire of the user.

Another possible protocol extension would be to retrieve a list of
which columns the agent actually implements as mandatory or
optional. This feature seems like a "nice to have" rather than a
"must".

With regard to order of variables in the PDU, I think we should go for
a strict order to reduce complexity of the agent. It is trivial for
the manager to know the order and to ensure that they are sent in a
particular order. So we should adopt the rule of "be conservative in
what you send" and say that the varbinds must be in a particular
order.

If we use syntax like CREATE-TRANSACTION { ...} CREATE-OPTIONAL { ...}
then we can say that the order is the order in the CREATE-TRANSACTION
list followed by the CREATE-OPTIONAL list. Another option would be to
use lexi-order (An interesting fact is with SMI-DS hierarchical
naming, when we say lexi-order we mean lexi-order of the underlying
objects (without the instance) because the instance would change the
order).

There's another dimension to this. What about other transactions in
the same PDU. Let's say that ABCD are the names of columns in one
table and RTSU are columns in another table.

So the simplest transaction includes ABC and D for a particular
instance: {A.6, B.6, C.6, D.6}.

Should we allow other rows in the table in the same PDU? {A.6, B.6,
C.6, D.6, A.7, B.7, C.7, D.7} ?

What about transactions on other tables? {A.6, B.6, C.6, D.6, R.7,
S.7, T.7, U.7} ?

I think we should say that transactions must be complete
(i.e. includes all mandatory columns) and in order, but multiple
transactions can be in a packet whether they are to other rows in the
same table or in different tables.


I realize that half of this message touches on protocol issues but
this is an example of where we need to start developing the SMI and
the protocol in sync.

Thoughts?
Steve


P.S. Here's a proposed definition of the syntax to be added to the SMING
spec:

Mapping of the CREATE-TRANSACTION clause

   For each create transaction to be defined, a CREATE-TRANSACTION
   clause may be present if the object corresponds to a conceptual row.
   The CREATE-TRANSACTION clause defines the list of mandatory objects
   to be included in a row creation transaction and the order in which
   those objects must be included in the transaction. The objects can
   be objects from the same or another conceptual table that share the
   same indexes.

Mapping of the CREATE-OPTIONAL clause

   For each create transaction to be defined, a CREATE-OPTIONAL clause
   may be present if the object corresponds to a conceptual row.  The
   CREATE-OPTIONAL clause must immediately the CREATE-TRANSACTION
   clause to which it adds optional objects.  The CREATE-OPTIONAL
   clause defines the list of optional objects that may be included in
   a row creation transaction and the order in which those objects
   must be included in the transaction. The objects can be objects
   from the same or another conceptual table that share the same
   indexes.

    CreatePart ::= "CREATE-TRANSACTION"  "{" Objects "}"
                |  "CREATE-OPTIONAL"     "{" Objects "}"
                | empty

    Objects ::=   Object
                | Objects "," Object

    Object ::=  value(ObjectName)