#SNMP Design: Compiler Design Proposal for CrossRoad

You may notice the current official release of #SNMP is still UnicornHorn Refresh (1.1) which is available since August. Now TwinTower (1.5) RC2 is out, and the final release will be published weeks later. So it is time to define detailed baselines for both TwinTower and CrossRoad (2.0) that match the situation.

In this post, I already stated that no new features (even the performance tuning) will be added to TwinTower except bug fixes. So the following features will definitely be delayed to CrossRoad,

1. MIB compiler prototype.
2. MIB browser performance improvement for 1000+ MIB documents.
3. SNMP v2c missing pieces.

So in this post I am going to provide some ideas about the MIB compiler design.

You already notice I start to call the MIB handler in #SNMP “a compiler”. This is not yet a true compiler, but the structure is similar. I have divided the whole process into two phases, parsing and assembling. The current structure is not yet completed, so the following problems remain,

• The parser does not generate enough information for the assembler. For example, when the assembler receives an IEntity object, only its parent name, name, and value are available, so a recursive handler must be there to assemble the object into the tree.
• The assembler cannot use an efficient way to assemble the entities. Take a look at ObjectTree.Refresh, and you will see how complex it is. This is a direct result of the first item.
• The parser cannot report progress, errors, and warnings.
• It is even worse if there are complex OID definitions defined in the MIB documents. The assembler consumes more time to locate their position in the tree, and assemble them. In some cases place holders must be generated by the assembler to make sure the node can be attached correctly. So the new design contains a few new ideas to simplify the entire process.

Parser (Front-End)

In order to make the assembler easier, the parser must generate as much information as possible. For example, if the complete OID of an object can be generated by the parser, then the assembler can use a much more efficient way to assemble the object by calling ObjectTree.Find(uint[] numerical).

However, this means the object location must be determined by the parser instead of the assembler. So during compilation, the parser must retrieve the tree to know where to place the new nodes, and create place holders if necessary.

Because the parser can be separated from the browser completely in the future (if the assembler in browser only reads in module files), the time consumes by the parser will no longer affect the browser performance. This separation can significantly simplify the browser.

Module File Format

It is possible to store those objects into text files just like a real compiler. For example, define a .module file format to contain the objects,

1. The file name is the same as the module name.
2. Every line is an object.
3. The object is stored in a comma separated way. ;;;(;)

Then whenever a MIB document is imported, such module files are generated by the parser. It is much easier for the assembler to read in these module files and build up the tree from scratch.

Assembler (Back-End)

Now the assembler simply read in module files, and make sure the nodes are generated correctly. This “thin” assembler can work much faster.

I may check in latest research work into the repository periodically, but surely the final work is going to be included in CrossRoad.