PYSMI(1) SNMP SMI compiler PYSMI(1)

pysmi - SNMP SMI compiler

The PySMI library and tools are designed to parse, verify and transform SNMP SMI MIB modules from their original ASN.1 form into JSON or pysnmp representation.

PySMI library is highly modular. The top-level component is called compiler and it acts as main user-facing object. Most of other components are plugged into the compiler object prior to its use.

Normally, user asks compiler to perform certain transformation of named MIB module. Compiler will:

  • Search its data sources for given MIB module (identified by name) noting their last modification times.
  • Search compiler-managed repositories of already converted MIB modules for modules that are more recent than corresponding source MIB module.
  • If freshly transformed MIB module is found, processing stops here.
  • Otherwise compiler passes ASN.1 MIB module content to the lexer component.
  • Lexer returns a sequence of tokenized ASN.1 MIB contents. Compiler then passes that sequence of tokens to the parser component.
  • Parser runs LR algorithm on tokenized MIB thus transforming MIB contents into Abstract Syntax Tree (AST) and also noting what other MIB modules are referred to from the MIB being parsed.
  • In case of parser failure, what is usually an indication of broken ASN.1 MIB syntax, compiler may attempt to fetch pre-transformed MIB contents from configured source. This process is called borrowing in PySMI.
  • In case of successful parser completion, compiler will pass produced AST to code generator component.
  • Code generator walks its input AST and performs actual data transformation.
  • The above steps may be repeated for each of the MIB modules referred to as parser figures out. Once no more unresolved dependencies remain, compiler will call its writer component to store all transformed MIB modules.

The location of ASN.1 MIB modules and flavor of their syntax, as well as desired transformation format, is determined by respective components chosen and configured to compiler.

The mibdump tool

The mibdump.py tool is a command-line frontend to the PySMI library. This tool can be used for automatic downloading and transforming SNMP MIB modules into various formats.

$ mibdump.py --help
Synopsis:
  SNMP SMI/MIB files conversion tool
Documentation:
  http://snmplabs.com/pysmi
Usage: mibdump.py [--help]
      [--version]
      [--quiet]
      [--debug=<all|borrower|codegen|compiler|grammar|lexer|parser|reader|searcher|writer>]
      [--mib-source=<URI>]
      [--mib-searcher=<PATH|PACKAGE>]
      [--mib-stub=<MIB-NAME>]
      [--mib-borrower=<PATH>]
      [--destination-format=<FORMAT>]
      [--destination-directory=<DIRECTORY>]
      [--cache-directory=<DIRECTORY>]
      [--disable-fuzzy-source]
      [--no-dependencies]
      [--no-python-compile]
      [--python-optimization-level]
      [--ignore-errors]
      [--build-index]
      [--rebuild]
      [--dry-run]
      [--no-mib-writes]
      [--generate-mib-texts]
      [--keep-texts-layout]
      <MIB-NAME> [MIB-NAME [...]]]
Where:
    URI      - file, zip, http, https, ftp, sftp schemes are supported.
               Use @mib@ placeholder token in URI to refer directly to
               the required MIB module when source does not support
               directory listing (e.g. HTTP).
    FORMAT   - pysnmp, json, null

When JSON destination format is requested, for each MIB module mibdump.py will produce a JSON document containing all MIB objects. For example, IF-MIB module in JSON form would look like:

{
   "ifMIB": {
       "name": "ifMIB",
       "oid": "1.3.6.1.2.1.31",
       "class": "moduleidentity",
       "revisions": [
         "2007-02-15 00:00",
         "1996-02-28 21:55",
         "1993-11-08 21:55"
       ]
     },
   ...
   "ifTestTable": {
     "name": "ifTestTable",
     "oid": "1.3.6.1.2.1.31.1.3",
     "class": "objecttype",
     "maxaccess": "not-accessible"
   },
   "ifTestEntry": {
     "name": "ifTestEntry",
     "oid": "1.3.6.1.2.1.31.1.3.1",
     "class": "objecttype",
     "maxaccess": "not-accessible",
     "augmention": {
       "name": "ifTestEntry",
       "module": "IF-MIB",
       "object": "ifEntry"
     }
   },
   "ifTestId": {
     "name": "ifTestId",
     "oid": "1.3.6.1.2.1.31.1.3.1.1",
     "class": "objecttype",
     "syntax": {
       "type": "TestAndIncr",
       "class": "type"
     },
     "maxaccess": "read-write"
   },
   ...
}

In general, JSON MIB captures all aspects of original (ASN.1) MIB contents and layout. The snippet above is just an example, here is the complete IF-MIB.json file.

Specifying MIB source

The --mib-source option can be given multiple times. Each instance of --mib-source must specify a URL where ASN.1 MIB modules should be looked up and downloaded from. At this moment three MIB sourcing methods are supported:

  • Local files. This could be a top-level directory where MIB files are located. Subdirectories will be automatically traversed as well. Example: file:///usr/share/snmp
  • ZIP archives containing MIB files. Subdirectories and embedded ZIP archives will be automatically traversed. Example: zip://mymibs.zip
  • HTTP/HTTPS. A fully specified URL where MIB module name is specified by a @mib@ placeholder. When specific MIB is looked up, PySMI will replace that placeholder with MIB module name it is looking for. Example: http://mibs.snmplabs.com/asn1/@mib@
  • SFTP/FTP. A fully specified URL including FTP username and password. MIB module name is specified by a @mib@ placeholder. When specific MIB is looked up, PySMI will replace that placeholder with MIB module name it is looking for. Example: http://mibs.snmplabs.com/asn1/@mib@

When trying to fetch a MIB module, the mibdump.py tool will try each of configured --mib-source transports in order of specification till first successful hit.

By default mibdump.py will search:

Once another --mib-source option is given, those defaults will not be used and should be manually given to mibdump.py if needed.

There is no single convention on how MIB module files should be named. By default mibdump.py will try a handful of guesses when trying to find a file containing specific MIB module. It will try upper and lower cases, a file named after MIB module, try adding different extensions to a file (.mib, .my etc), try adding/cutting the '-MIB' part of the file name. If nothing matches, mibdump.py will consider that probed --mib-source does not contain MIB module it is looking for.

There is a small chance, though, that fuzzy natching may result in getting a wrong MIB. If that happens, you can disable the above fuzzyness by giving mibdump.py the --disable-fuzzy-source flag.

It well may happen that many MIB modules refer to a common single MIB module. In that case mibdump.py may transform it many times unless you tell mibdump.py where to search for already transformed MIBs. That place could of course be a directory where mibdump.py writes its transforms into and/or some other local locations.

The --mib-searcher option specifies either local directory or importable Python package (applicable to pysnmp transformation) containing transformed MIB modules. Multiple --mib-searcher options could be given, mibdump.py will use each of them in order of specification till first hit.

If no transformed MIB module is found, mibdump.py will go on running its full transformation cycle.

By default mibdump.py will use:

  • --mib-searcher=$HOME/.pysnmp/mibs
  • --mib-searcher=pysnmp_mibs

Once another --mib-searcher option is given, those defaults will not be used and should be manually given to mibdump.py if needed.

Some MIBs may not be automatically transformed into another form and therefore must be explicitly excluded from processing. Such MIBs are normally manually implemented for each target MIB format. Examples include MIBs containing base SMI types or ASN.1 MACRO definitions (SNMPv2-SMI, SNMPV2-TC), initially compiled but later manually modified MIBs and others.

Default list of blacklisted MIBs for pysnmp transformation target is: RFC-1212, RFC-1215, RFC1065-SMI, RFC1155-SMI, RFC1158-MIB, RFC1213-MIB, SNMP-FRAMEWORK-MIB, SNMP-TARGET-MIB, SNMPv2-CONF, SNMPv2-SMI, SNMPv2-TC, SNMPv2-TM, TRANSPORT-ADDRESS-MIB.

If you need to modify this list use the --mib-stub option.

Curiously enough, some MIBs coming from quite prominent vendors appear syntactically incorrect. That leads to MIB compilers fail on such MIBs. While many MIB compiler implementations (PySMI included) introduce workarounds and grammar relaxations allowing slightly broken MIBs to compile, however severely broken MIBs can't be reliably compiled.

As another workaround PySMI offers the borrow feature. It allows PySMI to fetch already transformed MIBs even if corresponding ASN.1 MIB can't be found or parsed.

Default source of pre-compiled MIBs for pysnmp target is:

If you wish to modify this default list use one or more --mib-borrower options.

PySMI design allows many transformation formats to be supported in form of specialized code generation components. At the moment PySMI can produce MIBs in form of pysnmp classes and JSON documents.

JSON document schema is chosen to preserve as much of MIB information as possible. There's no established JSON schema known to the authors.

Setting destination directory

By default mibdump.py writes pysnmp MIBs into:

  • $HOME/.pysnmp/mibs (on UNIX)
  • @HOME@PySNMP ConfigurationMIBs (on Windows)

and JSON files in current working directory.

Use --destination-directory option to change default output directory.

By default PySMI will avoid creating new transformations if fresh enough versions already exist. By using --rebuild option you could trick PySMI doing requested transformation for all given MIB modules.

Ignoring transformation errors

By default PySMI will stop on first fatal error occurred during transformations of a series of MIBs. If you wish PySMI to ignore fatal errors and therefore skipping failed MIB, use the --ignore-errors option.

Keep in mind that skipping transformation of MIBs that are imported by other MIBs might make dependant MIBs inconsistent for use.

Most MIBs rely on other MIBs for their operations. This is indicated by the IMPORT statement in ASN.1 language. PySMI attempts to transform all MIBs IMPORT'ed by MIB being transformed. That is done in recursive manner.

By using --no-dependencies flag you can tell PySMI not to transform any MIBs other than those explicitly requested to be transformed.

Keep in mind that skipping dependencies may make the whole set of transformed MIBs inconsistent.

Most MIBs are very verbose. They contain many human-oriented descriptions and clarifications written in plain English. Those texts may be useful for MIB browser applications (to display those texts to human operator) but might not make any sense in other applications.

To save space and CPU time, PySMI does not by default include those texts into transformed MIBs. However this can be reverted by adding --generate-mib-texts option.

When MIB texts are generated, whitespaces and new lines are stripped by default. Sometimes that breaks down ASCII art should it occur in MIB texts. To preserve original text formatting, --keep-texts-layout option may be used.

If --build-index option is given, depending on the destination format chosen, the mibdump.py tool may create new (or update existing) document containing MIB information in a form that is convenient for querying cornerstone properties of MIB files.

For example, building JSON index for IP-MIB.json, TCP-MIB.json and UDP-MIB.json MIB modules would emit something like this:

{
   "compliance": {
      "1.3.6.1.2.1.48.2.1.1": [
        "IP-MIB"
      ],
      "1.3.6.1.2.1.49.2.1.1": [
        "TCP-MIB"
      ],
      "1.3.6.1.2.1.50.2.1.1": [
        "UDP-MIB"
      ]
   },
   "identity": {
       "1.3.6.1.2.1.48": [
         "IP-MIB"
       ],
       "1.3.6.1.2.1.49": [
         "TCP-MIB"
       ],
       "1.3.6.1.2.1.50": [
         "UDP-MIB"
       ]
   },
   "oids": {
       "1.3.6.1.2.1.4": [
         "IP-MIB"
       ],
       "1.3.6.1.2.1.5": [
         "IP-MIB"
       ],
       "1.3.6.1.2.1.6": [
         "TCP-MIB"
       ],
       "1.3.6.1.2.1.7": [
         "UDP-MIB"
       ],
       "1.3.6.1.2.1.49": [
         "TCP-MIB"
       ],
       "1.3.6.1.2.1.50": [
         "UDP-MIB"
       ]
   }
}

With this example, compliance and identity keys point to MODULE-COMPLIANCE and MODULE-IDENTITY MIB objects, oids list top-level OIDs branches defined in MIB modules. Full index build over thousands of MIBs could be seen here.

Minor speedups

There are a few options that may improve PySMI performance.

The --cache-directory option may be used to point to a temporary writable directory where PySMI parser (e.g. Ply) would store its lookup tables.

By default PySMI performing transformation into pysnmp format will also pre-compile Python source into interpreter bytecode. That takes some time and space. If you wish not to cache Python bytecode or to do that later, use the --no-python-compile option.

The mibcopy tool

The mibcopy.py tool attempts to normalize the file name of the MIB file.

It turned out that sometimes vendors name their MIBs in any possible way, not necessarily after the canonical MIB name. This causes problems to the MIB consumers as they may not be able to locate the MIB they need on the file system.

The way how mibcopy.py works is that it tries to read the MIB from the given file (or all files from a given directory or archive), parse MIB's canonical name from the contents of the file. Based on that, the tool tries to rename MIB file into the name which is the same as canonical MIB name. If mibcopy.py encounters the same named file already present on the file system, it reads it up to see its revision date. Then the tool compares the revision dates of the colliding MIB files and either overrides the offending file or drops the file being copied as outdated.

The ultimate goal is to end up with the latest versions of the MIB files all named after their canonical names.

$ mibcopy.py --help
Synopsis:
  SNMP SMI/MIB files copying tool. When given MIB file(s) or
  directory(ies) on input and a destination directory, the tool
  parses MIBs to figure out their canonical MIB module name and
  the latest revision date, then copies MIB module on input
  into the destination directory under its MIB module name
  *if* there is no such file already or its revision date is
  older.
Documentation:
  http://snmplabs.com/pysmi
Usage: mibcopy.py [--help]
      [--version]
      [--verbose]
      [--quiet]
      [--debug=<all|borrower|codegen|compiler|grammar|lexer|
                parser|reader|searcher|writer>]
      [--mib-source=<URI>]
      [--cache-directory=<DIRECTORY>]
      [--ignore-errors]
      [--dry-run]
      <SOURCE [SOURCE...]> <DESTINATION>
Where:
    URI      - file, zip, http, https, ftp, sftp schemes are
               supported. Use @mib@ placeholder token in URI to
               refer directly to the required MIB module when
               source does not support directory listing
               (e.g. HTTP).

Specifying MIB source

The --mib-source option can be given multiple times. Each instance of --mib-source must specify a URL where ASN.1 MIB modules should be looked up and downloaded from. At this moment three MIB sourcing methods are supported:

  • Local files. This could be a top-level directory where MIB files are located. Subdirectories will be automatically traversed as well. Example: file:///usr/share/snmp
  • ZIP archives containing MIB files. Subdirectories and embedded ZIP archives will be automatically traversed. Example: zip://mymibs.zip
  • HTTP/HTTPS. A fully specified URL where MIB module name is specified by a @mib@ placeholder. When specific MIB is looked up, PySMI will replace that placeholder with MIB module name it is looking for. Example: http://mibs.snmplabs.com/asn1/@mib@
  • SFTP/FTP. A fully specified URL including FTP username and password. MIB module name is specified by a @mib@ placeholder. When specific MIB is looked up, PySMI will replace that placeholder with MIB module name it is looking for. Example: http://mibs.snmplabs.com/asn1/@mib@

When trying to fetch a MIB module, the mibcopy.py tool will try each of configured --mib-source transports in order of specification till first successful hit.

By default mibcopy.py will search:

Once another --mib-source option is given, those defaults will not be used and should be manually given to mibcopy.py if needed.

Setting destination directory

The mibcopy.py writes MIBs into the <DESTINATION> directory.

Ignoring transformation errors

By default PySMI will stop on first fatal error occurred during transformations of a series of MIBs. If you wish PySMI to ignore fatal errors and therefore skipping failed MIB, use the --ignore-errors option.

Keep in mind that skipping transformation of MIBs that are imported by other MIBs might make dependant MIBs inconsistent for use.

Minor speedups

The --cache-directory option may be used to point to a temporary writable directory where PySMI parser (e.g. Ply) would store its lookup tables. That should improve PySMI performance a tad bit.

The MibCompiler object is the top-most interface to PySMI library features. It holds together the otherwise isolated pieces of the compiler infrastructure and manages the workflow of ASN.1 MIB transformation.

This example showcases some of its features:

from pysmi.reader import HttpReader
from pysmi.searcher import StubSearcher
from pysmi.writer import CallbackWriter
from pysmi.parser import SmiStarParser
from pysmi.codegen import JsonCodeGen
from pysmi.compiler import MibCompiler
inputMibs = ['IF-MIB', 'IP-MIB']
httpSources = [('mibs.snmplabs.com', 80, '/asn1/@mib@')]
# store compiled MIBs by calling this function
def store_mibs(mibName, jsonDoc, cbCtx):
    print('# MIB module %s' % mibName)
    print(jsonDoc)
mibCompiler = MibCompiler(
    SmiStarParser(), JsonCodeGen(), CallbackWriter(store_mibs)
)
# pull ASN.1 MIBs over HTTP
mibCompiler.addSources(*[HttpReader(*x) for x in httpSources])
# never recompile MIBs with ASN.1 MACROs
mibCompiler.addSearchers(StubSearcher(*JsonCodeGen.baseMibs))
status = mibCompiler.compile(*inputMibs)
print(status)

MibStatus class instance is used by MibCompiler.compiler() to indicate the outcome of MIB transformation operation.

PySMI offers a handful of distinct transport mechanisms for fetching MIBs by name from specific locations. In all cases MIB module name to file name match may not be exact -- some name fuzzying can be performed to mitigate possible changes to MIB file name.

FileReader class instance looks up MIB files in given directories on the host running PySMI.

ZipReader class instance looks up MIB files in local ZIP archive. ZIP subdirectories and embedded ZIP archives would be traversed.

HttpReader class instance tries to download MIB files using configured URL.

FtpReader class instance tries to download MIB files from configured FTP server.

CallbackReader class instance tries to fetch MIB files by calling user object.

There are cases when MIB transformation may or must not be performed. Such cases include:

  • foundation MIBs containing manually implemented pieces or ASN.1 MACRO's
  • obsolete MIBs fully reimplemented within modern MIBs
  • already transformed MIBs

MibCompiler expects user to supply a searcher object that would allow or skip MIB transformation for particular name based on whatever reason it is aware of.

In general, searcher logic is specific to target format. At the time being, only pysnmp code generation backend requires such filtering.

Transformed MIBs that were saved in form of Python files can be checked with PyFileSearcher class instances.

Some MIBs, most frequently the base ones, can be stored at a Python package. There existence can be checked with the PyPackageSearcher class instances.

Foundation or obsolete MIBs that should never be transformed can be blindly excluded by listing their names at the StubSearcher class instance.

MIBs may be written in one of the two major SMI language versions (v1 and v2). Some MIBs may contain typical errors.

PySMI offers a way to customize the parser to consume either of the major SMI grammars as well as to recover from well-known errors in MIB files.

SNMP MIBs are written in two kinds of special language - SMIv1 and SMIv2. The first SMI version is obsolete, most MIBs by now are written in SMIv2 grammar. There are also efforts aimed at improving SMIv2, but those MIBs are in great minority at the time of this writing.

PySMI is designed to handle both SMIv1 and SMIv2. The way it is done is that SMIv2 is considered the most basic and complete, whereas SMIv1 is a specialization of SMIv2 syntax.

For a user to acquire SMIv2 parser the parserFactory function should be called with the SMI dialect object.

The parser object should be passed to the MibCompiler object.

NOTE:

Please, note that parserFactory function returns a class, not class instance. Make sure to instantiate it when passing to MibCompiler class constructor.

PySMI offers a pre-built collection of parser grammar relaxation options to simplify its use:

  • pysmi.parser.dialect.smiV2 - canonical SMIv2 grammar
  • pysmi.parser.dialect.smiV1 - canonical SMIv1 grammar
  • pysmi.parser.dialect.smiV1Relaxed - relaxed SMIv1 grammar allowing some deviations

The grammar object should be passed to the parserFactory function.

from pysmi.parser.dialect import smiV1
from pysmi.parser.smi import parserFactory
SmiV1Parser = parserFactory(**smiV1)

Apparently, many production MIBs were shipped in syntactically broken condition. PySMI attempts to work around such issues by allowing some extra SMI grammar relaxations. You can enable all those relaxations at once to maximize the number of MIBs, found in the wild, successfully compiled.

from pysmi.parser.dialect import smiV1Relaxed
from pysmi.parser.smi import parserFactory
RelaxedSmiV1Parser = parserFactory(**smiV1Relaxed)

Once ASN.1 MIB is parsed up, AST is passed to a code generator which turns AST into desired representation of the MIB.

Some MIBs in circulation appear broken beyond automatic repair. To handle such cases PySMI introduces the MIB borrowing functionality. When MibCompiler gives up compiling a MIB, it can try to go out and take a copy of already transformed MIB to complete the request successfully.

Successfully transformed MIB modules' contents will be passed to writer object given to MibCompiler on instantiation.

CallbackWriter class instance passes the contents of the compiled MIB files to a user object.

The following examples focus on various feature of the PySMI library.

Look up specific ASN.1 MIBs at configured Web and FTP sites, compile them into JSON documents and print them out to stdout.

Try to support both SMIv1 and SMIv2 flavors of SMI as well as popular deviations from official syntax found in the wild.

from pysmi.reader import FileReader, HttpReader
from pysmi.searcher import StubSearcher
from pysmi.writer import CallbackWriter
from pysmi.parser import SmiStarParser
from pysmi.codegen import JsonCodeGen
from pysmi.compiler import MibCompiler
# from pysmi import debug
# debug.setLogger(debug.Debug('reader', 'compiler'))
inputMibs = ['IF-MIB', 'IP-MIB']
srcDirectories = ['/usr/share/snmp/mibs']
httpSources = [
    ('mibs.snmplabs.com', 80, '/asn1/@mib@')
]
def printOut(mibName, jsonDoc, cbCtx):
    print('\n\n# MIB module %s' % mibName)
    print(jsonDoc)
# Initialize compiler infrastructure
mibCompiler = MibCompiler(
    SmiStarParser(), JsonCodeGen(), CallbackWriter(printOut)
)
# search for source MIBs here
mibCompiler.addSources(*[FileReader(x) for x in srcDirectories])
# search for source MIBs at Web sites
mibCompiler.addSources(*[HttpReader(*x) for x in httpSources])
# never recompile MIBs with MACROs
mibCompiler.addSearchers(StubSearcher(*JsonCodeGen.baseMibs))
# run recursive MIB compilation
results = mibCompiler.compile(*inputMibs)
print('\n# Results: %s' % ', '.join(['%s:%s' % (x, results[x]) for x in results]))

Download script.

Look up specific ASN.1 MIBs at configured Web and FTP sites, compile them into pysnmp form and save Python modules as plain-text files in a local directory.

Try to support both SMIv1 and SMIv2 flavors of SMI as well as popular deviations from official syntax found in the wild.

In this example we disable automatic dependency checking on MIB compilation using noDeps flag.

Also, we do not check if target file already exists thus MIB compilation occurs on every invocation.

from pysmi.reader import HttpReader
from pysmi.reader import FtpReader
from pysmi.searcher import StubSearcher
from pysmi.writer import PyFileWriter
from pysmi.parser import SmiStarParser
from pysmi.codegen import PySnmpCodeGen
from pysmi.compiler import MibCompiler
inputMibs = ['IF-MIB', 'IP-MIB']
httpSources = [
    ('mibs.snmplabs.com', 80, '/asn1/@mib@')
]
ftpSources = [
    ('ftp.cisco.com', '/pub/mibs/v2/@mib@')
]
dstDirectory = '.pysnmp-mibs'
# Initialize compiler infrastructure
mibCompiler = MibCompiler(
    SmiStarParser(), PySnmpCodeGen(), PyFileWriter(dstDirectory)
)
# search for source MIBs at Web and FTP sites
mibCompiler.addSources(*[HttpReader(*x) for x in httpSources])
mibCompiler.addSources(*[FtpReader(*x) for x in ftpSources])
# never recompile MIBs with MACROs
mibCompiler.addSearchers(StubSearcher(*PySnmpCodeGen.baseMibs))
# run non-recursive MIB compilation
results = mibCompiler.compile(*inputMibs, **dict(noDeps=True))
print('Results: %s' % ', '.join(['%s:%s' % (x, results[x]) for x in results]))

Download script.

Look up specific ASN.1 MIBs at configured local directories, compile them into pysnmp form if not done yet and save Python modules as plain-text files in a local directory.

Try to support both SMIv1 and SMIv2 flavors of SMI as well as popular deviations from official syntax found in the wild.

When figuring out if compilation is needed, check all known places where pysnmp MIBs could possibly be found.

You can force MIB re-compilation by passing rebuild flag to MIB compiler (see below).

Default invocation of MIB compiler does not generate [potentially large] comments and texts found in MIBs. If you need them in pysnmp MIB modules, just pass genTexts flag to MIB compiler.

from pysmi.reader import FileReader
from pysmi.searcher import PyFileSearcher, PyPackageSearcher, StubSearcher
from pysmi.writer import PyFileWriter
from pysmi.parser import SmiStarParser
from pysmi.codegen import PySnmpCodeGen
from pysmi.compiler import MibCompiler
inputMibs = ['IF-MIB', 'IP-MIB']
srcDirectories = ['/usr/share/snmp/mibs']
dstDirectory = '.pysnmp-mibs'
# Initialize compiler infrastructure
mibCompiler = MibCompiler(SmiStarParser(),
                          PySnmpCodeGen(),
                          PyFileWriter(dstDirectory))
# search for source MIBs here
mibCompiler.addSources(*[FileReader(x) for x in srcDirectories])
# check compiled MIBs in our own productions
mibCompiler.addSearchers(PyFileSearcher(dstDirectory))
# ...and at default PySNMP MIBs packages
mibCompiler.addSearchers(*[PyPackageSearcher(x) for x in PySnmpCodeGen.defaultMibPackages])
# never recompile MIBs with MACROs
mibCompiler.addSearchers(StubSearcher(*PySnmpCodeGen.baseMibs))
# run [possibly recursive] MIB compilation
results = mibCompiler.compile(*inputMibs)  #, rebuild=True, genTexts=True)
print('Results: %s' % ', '.join(['%s:%s' % (x, results[x]) for x in results]))

Download script.

Invoke user callback function to provide MIB text, compile given text string into pysnmp MIB form and pass results to another user callback function for storing.

Here we expect to deal only with SMIv2-valid MIBs.

We use noDeps flag to prevent MIB compiler from attemping to compile IMPORT'ed MIBs as well.

import sys
from pysmi.reader import CallbackReader
from pysmi.searcher import StubSearcher
from pysmi.writer import CallbackWriter
from pysmi.parser import SmiV2Parser
from pysmi.codegen import PySnmpCodeGen
from pysmi.compiler import MibCompiler
inputMibs = ['IF-MIB', 'IP-MIB']
srcDir = '/usr/share/snmp/mibs/'  # we will read MIBs from here
# Initialize compiler infrastructure
mibCompiler = MibCompiler(
    SmiV2Parser(),
    PySnmpCodeGen(),
    # out own callback function stores results in its own way
    CallbackWriter(lambda m, d, c: sys.stdout.write(d))
)
# our own callback function serves as a MIB source here
mibCompiler.addSources(
  CallbackReader(lambda m, c: open(srcDir+m+'.txt').read())
)
# never recompile MIBs with MACROs
mibCompiler.addSearchers(StubSearcher(*PySnmpCodeGen.baseMibs))
# run non-recursive MIB compilation
results = mibCompiler.compile(*inputMibs, **dict(noDeps=True))
print('Results: %s' % ', '.join(['%s:%s' % (x, results[x]) for x in results]))

Download script.

Look up specific ASN.1 MIBs at configured Web/FTP sites. If no required MIB is found or its compilation fails for some reason, attempt to download precompiled version of failed MIB and store it locally as if we had compiled it.

from pysmi.reader import HttpReader
from pysmi.searcher import PyFileSearcher
from pysmi.searcher import StubSearcher
from pysmi.borrower import PyFileBorrower
from pysmi.writer import PyFileWriter
from pysmi.parser import SmiStarParser
from pysmi.codegen import PySnmpCodeGen
from pysmi.compiler import MibCompiler
# from pysmi import debug
# debug.setLogger(debug.Debug('borrower', 'reader', 'searcher'))
inputMibs = ['BORROWED-MIB']
httpSources = [ 
    ('mibs.snmplabs.com', 80, '/asn1/@mib@')
]
httpBorrowers = [
    ('mibs.snmplabs.com', 80, '/pysnmp/notexts/@mib@')
]
dstDirectory = '.pysnmp-mibs'
# Initialize compiler infrastructure
mibCompiler = MibCompiler(
    SmiStarParser(), PySnmpCodeGen(), PyFileWriter(dstDirectory)
)
# search for source MIBs at Web sites
mibCompiler.addSources(*[HttpReader(*x) for x in httpSources])
# never recompile MIBs with MACROs
mibCompiler.addSearchers(StubSearcher(*PySnmpCodeGen.baseMibs))
# check compiled/borrowed MIBs in our own productions
mibCompiler.addSearchers(PyFileSearcher(dstDirectory))
# search for compiled MIBs at Web sites if source is not available or broken
mibCompiler.addBorrowers(*[PyFileBorrower(HttpReader(*x)).setOptions(genTexts=False) for x in httpBorrowers])
# run non-recursive MIB compilation
results = mibCompiler.compile(*inputMibs)
print('Results: %s' % ', '.join(['%s:%s' % (x, results[x]) for x in results]))

Download script.

Try to borrow precompiled pysnmp MIB file(s) from a web-site.

In this example no attempt is made to find and compile ASN.1 MIB source.

Fetched pysnmp MIB(s) are stored in a local directory.

from pysmi.reader import HttpReader
from pysmi.searcher import PyFileSearcher
from pysmi.borrower import PyFileBorrower
from pysmi.writer import PyFileWriter
from pysmi.parser import NullParser
from pysmi.codegen import NullCodeGen
from pysmi.compiler import MibCompiler
inputMibs = ['BORROWED-MIB']
httpBorrowers = [
    ('mibs.snmplabs.com', 80, '/pysnmp/notexts/@mib@')
]
dstDirectory = '.pysnmp-mibs'
# Initialize compiler infrastructure
mibCompiler = MibCompiler(
    NullParser(), NullCodeGen(), PyFileWriter(dstDirectory)
)
# check compiled/borrowed MIBs in our own productions
mibCompiler.addSearchers(PyFileSearcher(dstDirectory))
# search for precompiled MIBs at Web sites
mibCompiler.addBorrowers(
    *[PyFileBorrower(HttpReader(*x)) for x in httpBorrowers]
)
# run MIB compilation
results = mibCompiler.compile(*inputMibs)
print('Results: %s' % ', '.join(['%s:%s' % (x, results[x]) for x in results]))

Download script.

In case of any troubles or confusion, try enabling PySMI debugging and watch the output:

from pysmi import debug
debug.setLogger(debug.Debug('all'))

Project source code is hosted at GitHub. Everyone is welcome to fork and contribute back!

We maintain the detailed log of changes to our software.

The best way to obtain SNMP SMI library is by running pip:

$ virtualenv venv
$ source venv/bin/activate
$ pip install pysmi

Alternatively, you can download the latest release from GitHub or PyPI.

  • Added implied key to JSON SNMP table index structure
  • Rebased MIB importing code onto importlib because imp is long deprecated
  • Fixed Py file borrower to become functional

  • Added mibcopy.py documentation
  • Copyright notice bumped up to year 2019

  • Bumped upper Python version to 3.7 and enabled pip cache
  • Exit code indication of the command-line tools aligned with sysexits.h to report more useful termination status

  • Fixed pysnmp lower version in test-requirements.txt
  • Fixed compiler crash when building comments at a platform which has broken users/groups databases

  • The mibcopy tool implemented to copy MIB modules from files with potentially messed up names into a directory under canonical MIB names picking up the latest MIB revision along the way.
  • ZIP archive reader implemented to pull ASN.1 MIB files from .zip archives pretty much in the same way as from plain directories
  • HTTP/S proxy support added (through respecting http_proxy environment variable) by switching from httplib to urllib2 internally
  • Copyright notice bumped up to year 2018
  • Project site in the docs changes from SourceForge to snmplabs.com
  • PRODUCT-RELEASE generation added to the JSON code generator
  • Added special handling of BITS-like DEFVAL syntax for Integers that occurs in buggy MIBs
  • Fixed missing REVISIONS generations in MODULE-IDENTITY

  • Library documentation refactored and updated
  • Fixed malformed Python code being produced by pysnmp code generator

  • Added MIB status, product release and revision description set calls at pysnmp code generator
  • Changed REVISION field format in JSON representation - it is now a list of dicts each with revision timestamp and description text
  • MIB REFERENCE fields are only exported if --with-mib-text is on
  • Sphinx documentation theme changed to Alabaster
  • Multiple fixes to pysnmp codegen not to produce function calls with more than 255 parameters

  • Fix to SMI lexer to treat tokens starting from a digit as belonging to a lower-cased class. This fixes sub-OID parsing bug (specifically, 802dot3(10006))
  • Fix to the mibdump.py local MIB path automatic injection in front of existing --mib-sources

  • INET-ADDRESS-MIB configured as pre-built at pysnmp codegen
  • JSON codegen produces "nodetype" element for OBJECT-TYPE
  • Fix to mibdump.py --destination-directory option
  • Fix to pysnmp and JSON code generators to properly refer to MIB module defining particular MIB object

  • The @mib@ magic in reader's URL template made optional. If it is not present, MIB module name is just appended to URL template
  • Send User-Agent containing pysmi and Python versions as well as platform name.
  • Fixed missing STATUS/DISPLAY-HINT/REFERENCE/etc fields generation at pysnmp backend when running in the non-full-text mode
  • Fixed broken ordereddict dependency on Python 2.6-

  • Generate REFERENCE and STATUS fields at various SMI objects
  • Generate DESCRIPTION field followed REVISION field at MODULE-IDENTITY objects
  • Generate PRODUCT-RELEASE field at AGENT-CAPABILITIES objects
  • Generated Python source aligned with PEP8
  • MIB texts cleaned up by default, --keep-texts-layout preserves original formatting
  • Fix to the ordereddict conditional dependency
  • Missing test module recovered
  • Failing tests fixed

  • JSON code generating backend implemented
  • Experimental JSON OID->MIB indices generation implemented
  • Package structure flattened for easier use
  • Minor refactoring to the test suite
  • Source code statically analyzed, hardened and PEP8-ized
  • Files closed explicitly to mute ResourceWarnings
  • Fixed to Python 2.4 (and aged ply) compatibility
  • Added a workaround to avoid generating pysnmp TextualConvention classes inheriting from TextualConvention (when MIB defines a TEXTUAL-CONVENTION based on another TEXTUAL-CONVENTION as SYNTAX)
  • Author's e-mail changed, copyright extended to year 2017

  • Crash on existing .py file handling fixed.
  • Fix to __doc__ use in setup.py to make -O0 installation mode working.
  • Fix to PyPackageSearcher not to fail on broken Python packages.
  • Source code pep8'ed
  • Copyright added to source files.

  • Several typos fixed, source code linted again.
  • Some dead code cleaned up.

  • Wheel distribution format now supported.
  • Handle the case of MIB symbols conflict with Python reserved words.
  • Handle binary DEFVAL initializer for INTEGER's.
  • Generate LAST-UPDATED at pysnmp code generator.

  • Fix to MRO compliance for mixin classes generation at pysnmp backend
  • Fix to repeated imports in generated code at pysnmp backend
  • Fix to mibdump tool to properly handle the --generate-mib-texts option.
  • Fix to Python compile() - optimize flag is valid only past Python 3.1
  • Fix to SMIv1 INDEX clause code generation for pysnmp backend.
  • Tighten file creation security at pysmi.writer.pyfile

  • Two-pass compiler design allows for much accurate code generation.
  • Sphinx-based documentation first introduced

First public release, not fully operational yet

The SNMP SMI library software is distributed under 2-clause BSD License.

Copyright (c) 2015-2019 Ilya Etingof <etingof@gmail.com> All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

The PySMI project maintains a collection of publicly available ASN.1 MIB files collected on the Internet. You are welcome to use this MIBs archive however we can't guarantee any degree of consistency or reliability when it comes to these MIB modules.

The mibdump.py tool as well as many other utilities based on PySMI are programmed to use this MIB repository for automatic download and dependency resolution.

You can always reconfigure PySMI to use some other remote MIB repository instead or in addition to this one.

In case of questions or troubles using SNMP SMI library, please open up an issue at GitHub or ask at Stack Overflow .

Ilya Etingof <etingof@gmail.com>

2015-2024, Ilya Etingof <etingof@gmail.com>

April 10, 2024 0.1