[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

some observations on OID compression / suppression


I have written a program which can read OIDs in dotted notation as
input and produce OID encodings in hex as output. (The program can
also read hex encodings as input and produce OIDs in dotted notation
as output. This allows me to verify that the encoding/decoding did not
change the OIDs.) The program right now supports the following
encodings / compression algorithms:

BER   OID no     compression  <RFC 1905>
ODC   OID delta  compression  <draft-irtf-nmrg-snmp-compression-01.txt>
OPC   OID prefix compression  <draft-ietf-eos-oidcompression-00.txt>

I have not yet implemented OID suppression as defined in
<draft-ietf-eos-snmp-rowops-00.txt> yet since I do not fully
understand how the row operations work, e.g. what is the row
operations equivalent to a GetBulk?


Observations:

(1) The EOS WG has the option to specify:

    (a) OID compression only
    (b) OID suppression only
    (c) OID compression and suppression

    The current EOS documents suggest to do (c). However, the OID
    suppression technique described in <draft-ietf-eos-snmp-rowops-00.txt>
    is not particularly efficient. In fact, it seems that just encoding
    the rows as normal varbind elements and using ODC compression leads
    to more compact encodings.

(2) The efficiency of OPC largely depends on how you retrieve data. If
    you walk tables row by row, then OPC compression will always
    encode the complete instance identifier. In cases where you have
    tables with complex instance identifiers, one can observe
    relatively poor compression ratios. If you walk tables column by
    column, then OPC can do a better job and compress instance
    identifier prefixes as well. (Of course, this is by design since
    the authors of OPC expect that an OID suppression mechanism takes
    care of the row by row case.)

(3) The efficiency of ODC also depends to some degree on how you
    retrieve data, although it seems in general to be more stable. In
    fact, ODC usually performs best if tables are read row by row
    since the delta is just a single column identifier substitution in
    this case. If you walk tables column by column, then ODC still can
    compress common sub-identifiers in the instance identifier
    portion.

(4) OPC performs better than ODC in those cases where the OIDs only
    differ in the last sub-identifier, such as ifInOctets.1 and
    ifInOctets.2. This once again shows OPCs tendency to operate
    better on data which is retrieved in column by column order.

(5) Implementation complexity: It is true that ODC is slightly more
    complex to implement compared to OPC. However, a straight-forward
    implementation still fits on a single piece of paper and I do not
    think the code is in any way complex (but I am of course biased).
    I will include the C code in the next ID so that people can look
    at it.

(6) Runtime costs: The test implementation uses a non-optimized BER
    sub-identifier encoding function from a forward BER encoder. With
    this sub-identifier encoding function, it turns out that BER is
    the slowest (surprise). OPC and ODC are both faster since they cut
    the number of sub-identifiers that must be encoded or decoded. OPC
    and ODC did not show any significant differences. (I used a 200
    MHz and 900 MHz Intel under Linux for the tests and I used gcc
    (egcs-2.91.66) -O2 for compilation - not sure how all this
    influences the numbers).

    The sub-identifier encoding function could be optimized.  But even
    then, I do still expect that the overhead of the compression
    functions is almost irrelevant. (And you can optimize the
    compression functions as well...)

(7) I believe that more experiments are needed to fully understand
    whether we need compression or suppression or both. My experience
    tells me that intuition often fools in this area.


The following concrete tests has been made:

(1) Example #1 in the ODC document:

    ber  84 bytes, opc  48 bytes: (43%)
    ber  84 bytes, odc  33 bytes: (61%)

(2) Example #2 in the ODC document:

    ber 114 bytes, opc  69 bytes: (40%)
    ber 114 bytes, odc  67 bytes: (42%)

(3) Example #3 in the ODC document:

    ber  61 bytes, opc  42 bytes: (32%)
    ber  61 bytes, odc  32 bytes: (48%)

(4) Example #1 in the OPC document:

    ber  40 bytes, opc  25 bytes: (38%)
    ber  40 bytes, odc  22 bytes: (45%)

(5) Example #2 in the OPC document:

    ber  88 bytes, opc  47 bytes: (47%)
    ber  88 bytes, odc  49 bytes: (45%)

(6) Encoding example in the OPC document:

    ber  83 bytes, opc  44 bytes: (47%)
    ber  83 bytes, odc  40 bytes: (52%)

(7) Encoding of the OIDs for ifInOctets, ifOutOctets, ifOperStatus for
    ifIndex [1-8]:

    ber 264 bytes, opc 117 bytes: (56%)
    ber 264 bytes, odc 110 bytes: (59%)

    These are actually 8 row operations. Note that the current row
    operations document proposed an OID suppression technique which
    seems to produce less efficient encodings than the compression
    techniques OPC and ODC.

(8) Retrieving the vacmViewTreeFamilyMask and the vacmViewTreeFamilyType
    for a number of non-trivial VACM view tree families row by row. We
    then computed the compression that is possible:

    ber 4026 bytes, opc 3225 bytes: (20%)
    ber 4026 bytes, odc  830 bytes: (80%)

    As you can see, OPC fails to achieve a reasonable reduction in
    this case.

(9) Retrieving the vacmViewTreeFamilyMask and the vacmViewTreeFamilyType
    for a number of non-trivial VACM view tree families column by column.
    We then computed the compression that is possible:

    ber 4026 bytes, opc 1659 bytes: (59%)
    ber 4026 bytes, odc 1121 bytes: (73%)

    As you can see, OPC now performs much better, but you still loose
    quite a bit against ODC due to the complex nature of the OIDs.

The VACM configuration has been obtained from the public Jasmin test
agent running on osborne.ibr.cs.tu-bs.de:161.

The input for the first 7 tests as well as the BER, ODC, OPC hex
encodings are available so that you can check and report any errors:

<http://www.ibr.cs.tu-bs.de/~schoenw/eos/misc>		input
<http://www.ibr.cs.tu-bs.de/~schoenw/eos/misc.ber>	ber output
<http://www.ibr.cs.tu-bs.de/~schoenw/eos/misc.odc>	odc output
<http://www.ibr.cs.tu-bs.de/~schoenw/eos/misc.opc>	opc output

Enjoy.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<schoenw@ibr.cs.tu-bs.de>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>