PYSMI(1) SNMP SMI compiler PYSMI(1) NAME pysmi - SNMP SMI compiler The PySMI library and tools are designed to parse, verify and transform SNMP SMI MIB modules from their original ASN.1 form into JSON or pysnmp representation. DOCUMENTATION PySMI documentation PySMI library is highly modular. The top-level component is called compiler and it acts as main user-facing object. Most of other components are plugged into the compiler object prior to its use. Normally, user asks compiler to perform certain transformation of named MIB module. Compiler will: o Search its data sources for given MIB module (identified by name) noting their last modification times. o Search compiler-managed repositories of already converted MIB modules for modules that are more recent than corresponding source MIB module. o If freshly transformed MIB module is found, processing stops here. o Otherwise compiler passes ASN.1 MIB module content to the lexer component. o Lexer returns a sequence of tokenized ASN.1 MIB contents. Compiler then passes that sequence of tokens to the parser component. o Parser runs LR algorithm on tokenized MIB thus transforming MIB contents into Abstract Syntax Tree (AST) and also noting what other MIB modules are referred to from the MIB being parsed. o In case of parser failure, what is usually an indication of broken ASN.1 MIB syntax, compiler may attempt to fetch pre-transformed MIB contents from configured source. This process is called borrowing in PySMI. o In case of successful parser completion, compiler will pass produced AST to code generator component. o Code generator walks its input AST and performs actual data transformation. o The above steps may be repeated for each of the MIB modules referred to as parser figures out. Once no more unresolved dependencies remain, compiler will call its writer component to store all transformed MIB modules. The location of ASN.1 MIB modules and flavor of their syntax, as well as desired transformation format, is determined by respective components chosen and configured to compiler. The mibdump tool The mibdump.py tool is a command-line frontend to the PySMI library. This tool can be used for automatic downloading and transforming SNMP MIB modules into various formats. $ mibdump.py --help Synopsis: SNMP SMI/MIB files conversion tool Documentation: http://snmplabs.com/pysmi Usage: mibdump.py [--help] [--version] [--quiet] [--debug=] [--mib-source=] [--mib-searcher=] [--mib-stub=] [--mib-borrower=] [--destination-format=] [--destination-directory=] [--cache-directory=] [--disable-fuzzy-source] [--no-dependencies] [--no-python-compile] [--python-optimization-level] [--ignore-errors] [--build-index] [--rebuild] [--dry-run] [--no-mib-writes] [--generate-mib-texts] [--keep-texts-layout] [MIB-NAME [...]]] Where: URI - file, zip, http, https, ftp, sftp schemes are supported. Use @mib@ placeholder token in URI to refer directly to the required MIB module when source does not support directory listing (e.g. HTTP). FORMAT - pysnmp, json, null When JSON destination format is requested, for each MIB module mibdump.py will produce a JSON document containing all MIB objects. For example, IF-MIB module in JSON form would look like: { "ifMIB": { "name": "ifMIB", "oid": "1.3.6.1.2.1.31", "class": "moduleidentity", "revisions": [ "2007-02-15 00:00", "1996-02-28 21:55", "1993-11-08 21:55" ] }, ... "ifTestTable": { "name": "ifTestTable", "oid": "1.3.6.1.2.1.31.1.3", "class": "objecttype", "maxaccess": "not-accessible" }, "ifTestEntry": { "name": "ifTestEntry", "oid": "1.3.6.1.2.1.31.1.3.1", "class": "objecttype", "maxaccess": "not-accessible", "augmention": { "name": "ifTestEntry", "module": "IF-MIB", "object": "ifEntry" } }, "ifTestId": { "name": "ifTestId", "oid": "1.3.6.1.2.1.31.1.3.1.1", "class": "objecttype", "syntax": { "type": "TestAndIncr", "class": "type" }, "maxaccess": "read-write" }, ... } In general, JSON MIB captures all aspects of original (ASN.1) MIB contents and layout. The snippet above is just an example, here is the complete IF-MIB.json file. Specifying MIB source The --mib-source option can be given multiple times. Each instance of --mib-source must specify a URL where ASN.1 MIB modules should be looked up and downloaded from. At this moment three MIB sourcing methods are supported: o Local files. This could be a top-level directory where MIB files are located. Subdirectories will be automatically traversed as well. Example: file:///usr/share/snmp o ZIP archives containing MIB files. Subdirectories and embedded ZIP archives will be automatically traversed. Example: zip://mymibs.zip o HTTP/HTTPS. A fully specified URL where MIB module name is specified by a @mib@ placeholder. When specific MIB is looked up, PySMI will replace that placeholder with MIB module name it is looking for. Example: http://mibs.snmplabs.com/asn1/@mib@ o SFTP/FTP. A fully specified URL including FTP username and password. MIB module name is specified by a @mib@ placeholder. When specific MIB is looked up, PySMI will replace that placeholder with MIB module name it is looking for. Example: http://mibs.snmplabs.com/asn1/@mib@ When trying to fetch a MIB module, the mibdump.py tool will try each of configured --mib-source transports in order of specification till first successful hit. By default mibdump.py will search: o file:///usr/share/snmp o http://mibs.snmplabs.com/asn1/@mib@ Once another --mib-source option is given, those defaults will not be used and should be manually given to mibdump.py if needed. Fuzzying MIB module names There is no single convention on how MIB module files should be named. By default mibdump.py will try a handful of guesses when trying to find a file containing specific MIB module. It will try upper and lower cases, a file named after MIB module, try adding different extensions to a file (.mib, .my etc), try adding/cutting the '-MIB' part of the file name. If nothing matches, mibdump.py will consider that probed --mib-source does not contain MIB module it is looking for. There is a small chance, though, that fuzzy natching may result in getting a wrong MIB. If that happens, you can disable the above fuzzyness by giving mibdump.py the --disable-fuzzy-source flag. Avoiding excessive transformation It well may happen that many MIB modules refer to a common single MIB module. In that case mibdump.py may transform it many times unless you tell mibdump.py where to search for already transformed MIBs. That place could of course be a directory where mibdump.py writes its transforms into and/or some other local locations. The --mib-searcher option specifies either local directory or importable Python package (applicable to pysnmp transformation) containing transformed MIB modules. Multiple --mib-searcher options could be given, mibdump.py will use each of them in order of specification till first hit. If no transformed MIB module is found, mibdump.py will go on running its full transformation cycle. By default mibdump.py will use: o --mib-searcher=$HOME/.pysnmp/mibs o --mib-searcher=pysnmp_mibs Once another --mib-searcher option is given, those defaults will not be used and should be manually given to mibdump.py if needed. Blacklisting MIBs Some MIBs may not be automatically transformed into another form and therefore must be explicitly excluded from processing. Such MIBs are normally manually implemented for each target MIB format. Examples include MIBs containing base SMI types or ASN.1 MACRO definitions (SNMPv2-SMI, SNMPV2-TC), initially compiled but later manually modified MIBs and others. Default list of blacklisted MIBs for pysnmp transformation target is: RFC-1212, RFC-1215, RFC1065-SMI, RFC1155-SMI, RFC1158-MIB, RFC1213-MIB, SNMP-FRAMEWORK-MIB, SNMP-TARGET-MIB, SNMPv2-CONF, SNMPv2-SMI, SNMPv2-TC, SNMPv2-TM, TRANSPORT-ADDRESS-MIB. If you need to modify this list use the --mib-stub option. Dealing with broken MIBs Curiously enough, some MIBs coming from quite prominent vendors appear syntactically incorrect. That leads to MIB compilers fail on such MIBs. While many MIB compiler implementations (PySMI included) introduce workarounds and grammar relaxations allowing slightly broken MIBs to compile, however severely broken MIBs can't be reliably compiled. As another workaround PySMI offers the borrow feature. It allows PySMI to fetch already transformed MIBs even if corresponding ASN.1 MIB can't be found or parsed. Default source of pre-compiled MIBs for pysnmp target is: o http://mibs.snmplabs.com/pysnmp/fulltexts/@mib@ o http://mibs.snmplabs.com/pysnmp/notexts/@mib@ If you wish to modify this default list use one or more --mib-borrower options. Choosing target transformation PySMI design allows many transformation formats to be supported in form of specialized code generation components. At the moment PySMI can produce MIBs in form of pysnmp classes and JSON documents. JSON document schema is chosen to preserve as much of MIB information as possible. There's no established JSON schema known to the authors. Setting destination directory By default mibdump.py writes pysnmp MIBs into: o $HOME/.pysnmp/mibs (on UNIX) o @HOME@PySNMP ConfigurationMIBs (on Windows) and JSON files in current working directory. Use --destination-directory option to change default output directory. Performing unconditional transformation By default PySMI will avoid creating new transformations if fresh enough versions already exist. By using --rebuild option you could trick PySMI doing requested transformation for all given MIB modules. Ignoring transformation errors By default PySMI will stop on first fatal error occurred during transformations of a series of MIBs. If you wish PySMI to ignore fatal errors and therefore skipping failed MIB, use the --ignore-errors option. Keep in mind that skipping transformation of MIBs that are imported by other MIBs might make dependant MIBs inconsistent for use. Skipping dependencies Most MIBs rely on other MIBs for their operations. This is indicated by the IMPORT statement in ASN.1 language. PySMI attempts to transform all MIBs IMPORT'ed by MIB being transformed. That is done in recursive manner. By using --no-dependencies flag you can tell PySMI not to transform any MIBs other than those explicitly requested to be transformed. Keep in mind that skipping dependencies may make the whole set of transformed MIBs inconsistent. Generating MIB texts Most MIBs are very verbose. They contain many human-oriented descriptions and clarifications written in plain English. Those texts may be useful for MIB browser applications (to display those texts to human operator) but might not make any sense in other applications. To save space and CPU time, PySMI does not by default include those texts into transformed MIBs. However this can be reverted by adding --generate-mib-texts option. When MIB texts are generated, whitespaces and new lines are stripped by default. Sometimes that breaks down ASCII art should it occur in MIB texts. To preserve original text formatting, --keep-texts-layout option may be used. Building MIB indices If --build-index option is given, depending on the destination format chosen, the mibdump.py tool may create new (or update existing) document containing MIB information in a form that is convenient for querying cornerstone properties of MIB files. For example, building JSON index for IP-MIB.json, TCP-MIB.json and UDP-MIB.json MIB modules would emit something like this: { "compliance": { "1.3.6.1.2.1.48.2.1.1": [ "IP-MIB" ], "1.3.6.1.2.1.49.2.1.1": [ "TCP-MIB" ], "1.3.6.1.2.1.50.2.1.1": [ "UDP-MIB" ] }, "identity": { "1.3.6.1.2.1.48": [ "IP-MIB" ], "1.3.6.1.2.1.49": [ "TCP-MIB" ], "1.3.6.1.2.1.50": [ "UDP-MIB" ] }, "oids": { "1.3.6.1.2.1.4": [ "IP-MIB" ], "1.3.6.1.2.1.5": [ "IP-MIB" ], "1.3.6.1.2.1.6": [ "TCP-MIB" ], "1.3.6.1.2.1.7": [ "UDP-MIB" ], "1.3.6.1.2.1.49": [ "TCP-MIB" ], "1.3.6.1.2.1.50": [ "UDP-MIB" ] } } With this example, compliance and identity keys point to MODULE-COMPLIANCE and MODULE-IDENTITY MIB objects, oids list top-level OIDs branches defined in MIB modules. Full index build over thousands of MIBs could be seen here. Minor speedups There are a few options that may improve PySMI performance. The --cache-directory option may be used to point to a temporary writable directory where PySMI parser (e.g. Ply) would store its lookup tables. By default PySMI performing transformation into pysnmp format will also pre-compile Python source into interpreter bytecode. That takes some time and space. If you wish not to cache Python bytecode or to do that later, use the --no-python-compile option. The mibcopy tool The mibcopy.py tool attempts to normalize the file name of the MIB file. It turned out that sometimes vendors name their MIBs in any possible way, not necessarily after the canonical MIB name. This causes problems to the MIB consumers as they may not be able to locate the MIB they need on the file system. The way how mibcopy.py works is that it tries to read the MIB from the given file (or all files from a given directory or archive), parse MIB's canonical name from the contents of the file. Based on that, the tool tries to rename MIB file into the name which is the same as canonical MIB name. If mibcopy.py encounters the same named file already present on the file system, it reads it up to see its revision date. Then the tool compares the revision dates of the colliding MIB files and either overrides the offending file or drops the file being copied as outdated. The ultimate goal is to end up with the latest versions of the MIB files all named after their canonical names. $ mibcopy.py --help Synopsis: SNMP SMI/MIB files copying tool. When given MIB file(s) or directory(ies) on input and a destination directory, the tool parses MIBs to figure out their canonical MIB module name and the latest revision date, then copies MIB module on input into the destination directory under its MIB module name *if* there is no such file already or its revision date is older. Documentation: http://snmplabs.com/pysmi Usage: mibcopy.py [--help] [--version] [--verbose] [--quiet] [--debug=] [--mib-source=] [--cache-directory=] [--ignore-errors] [--dry-run] Where: URI - file, zip, http, https, ftp, sftp schemes are supported. Use @mib@ placeholder token in URI to refer directly to the required MIB module when source does not support directory listing (e.g. HTTP). Specifying MIB source The --mib-source option can be given multiple times. Each instance of --mib-source must specify a URL where ASN.1 MIB modules should be looked up and downloaded from. At this moment three MIB sourcing methods are supported: o Local files. This could be a top-level directory where MIB files are located. Subdirectories will be automatically traversed as well. Example: file:///usr/share/snmp o ZIP archives containing MIB files. Subdirectories and embedded ZIP archives will be automatically traversed. Example: zip://mymibs.zip o HTTP/HTTPS. A fully specified URL where MIB module name is specified by a @mib@ placeholder. When specific MIB is looked up, PySMI will replace that placeholder with MIB module name it is looking for. Example: http://mibs.snmplabs.com/asn1/@mib@ o SFTP/FTP. A fully specified URL including FTP username and password. MIB module name is specified by a @mib@ placeholder. When specific MIB is looked up, PySMI will replace that placeholder with MIB module name it is looking for. Example: http://mibs.snmplabs.com/asn1/@mib@ When trying to fetch a MIB module, the mibcopy.py tool will try each of configured --mib-source transports in order of specification till first successful hit. By default mibcopy.py will search: o file:///usr/share/snmp o http://mibs.snmplabs.com/asn1/@mib@ Once another --mib-source option is given, those defaults will not be used and should be manually given to mibcopy.py if needed. Setting destination directory The mibcopy.py writes MIBs into the directory. Ignoring transformation errors By default PySMI will stop on first fatal error occurred during transformations of a series of MIBs. If you wish PySMI to ignore fatal errors and therefore skipping failed MIB, use the --ignore-errors option. Keep in mind that skipping transformation of MIBs that are imported by other MIBs might make dependant MIBs inconsistent for use. Minor speedups The --cache-directory option may be used to point to a temporary writable directory where PySMI parser (e.g. Ply) would store its lookup tables. That should improve PySMI performance a tad bit. PySMI library The MibCompiler object is the top-most interface to PySMI library features. It holds together the otherwise isolated pieces of the compiler infrastructure and manages the workflow of ASN.1 MIB transformation. This example showcases some of its features: from pysmi.reader import HttpReader from pysmi.searcher import StubSearcher from pysmi.writer import CallbackWriter from pysmi.parser import SmiStarParser from pysmi.codegen import JsonCodeGen from pysmi.compiler import MibCompiler inputMibs = ['IF-MIB', 'IP-MIB'] httpSources = [('mibs.snmplabs.com', 80, '/asn1/@mib@')] # store compiled MIBs by calling this function def store_mibs(mibName, jsonDoc, cbCtx): print('# MIB module %s' % mibName) print(jsonDoc) mibCompiler = MibCompiler( SmiStarParser(), JsonCodeGen(), CallbackWriter(store_mibs) ) # pull ASN.1 MIBs over HTTP mibCompiler.addSources(*[HttpReader(*x) for x in httpSources]) # never recompile MIBs with ASN.1 MACROs mibCompiler.addSearchers(StubSearcher(*JsonCodeGen.baseMibs)) status = mibCompiler.compile(*inputMibs) print(status) MIB compiler Compilation status MibStatus class instance is used by MibCompiler.compiler() to indicate the outcome of MIB transformation operation. MIB sources PySMI offers a handful of distinct transport mechanisms for fetching MIBs by name from specific locations. In all cases MIB module name to file name match may not be exact -- some name fuzzying can be performed to mitigate possible changes to MIB file name. Local file reader FileReader class instance looks up MIB files in given directories on the host running PySMI. ZIP archive reader ZipReader class instance looks up MIB files in local ZIP archive. ZIP subdirectories and embedded ZIP archives would be traversed. HTTP reader HttpReader class instance tries to download MIB files using configured URL. FTP reader FtpReader class instance tries to download MIB files from configured FTP server. Callback reader CallbackReader class instance tries to fetch MIB files by calling user object. Conditional compilation There are cases when MIB transformation may or must not be performed. Such cases include: o foundation MIBs containing manually implemented pieces or ASN.1 MACRO's o obsolete MIBs fully reimplemented within modern MIBs o already transformed MIBs MibCompiler expects user to supply a searcher object that would allow or skip MIB transformation for particular name based on whatever reason it is aware of. In general, searcher logic is specific to target format. At the time being, only pysnmp code generation backend requires such filtering. Python files searcher Transformed MIBs that were saved in form of Python files can be checked with PyFileSearcher class instances. Search Python packages Some MIBs, most frequently the base ones, can be stored at a Python package. There existence can be checked with the PyPackageSearcher class instances. Unconditionally ignore MIBs Foundation or obsolete MIBs that should never be transformed can be blindly excluded by listing their names at the StubSearcher class instance. Parser configuration MIBs may be written in one of the two major SMI language versions (v1 and v2). Some MIBs may contain typical errors. PySMI offers a way to customize the parser to consume either of the major SMI grammars as well as to recover from well-known errors in MIB files. SMI parser SNMP MIBs are written in two kinds of special language - SMIv1 and SMIv2. The first SMI version is obsolete, most MIBs by now are written in SMIv2 grammar. There are also efforts aimed at improving SMIv2, but those MIBs are in great minority at the time of this writing. PySMI is designed to handle both SMIv1 and SMIv2. The way it is done is that SMIv2 is considered the most basic and complete, whereas SMIv1 is a specialization of SMIv2 syntax. For a user to acquire SMIv2 parser the parserFactory function should be called with the SMI dialect object. The parser object should be passed to the MibCompiler object. NOTE: Please, note that parserFactory function returns a class, not class instance. Make sure to instantiate it when passing to MibCompiler class constructor. SMI language dialects PySMI offers a pre-built collection of parser grammar relaxation options to simplify its use: o pysmi.parser.dialect.smiV2 - canonical SMIv2 grammar o pysmi.parser.dialect.smiV1 - canonical SMIv1 grammar o pysmi.parser.dialect.smiV1Relaxed - relaxed SMIv1 grammar allowing some deviations The grammar object should be passed to the parserFactory function. from pysmi.parser.dialect import smiV1 from pysmi.parser.smi import parserFactory SmiV1Parser = parserFactory(**smiV1) Apparently, many production MIBs were shipped in syntactically broken condition. PySMI attempts to work around such issues by allowing some extra SMI grammar relaxations. You can enable all those relaxations at once to maximize the number of MIBs, found in the wild, successfully compiled. from pysmi.parser.dialect import smiV1Relaxed from pysmi.parser.smi import parserFactory RelaxedSmiV1Parser = parserFactory(**smiV1Relaxed) Code generators Once ASN.1 MIB is parsed up, AST is passed to a code generator which turns AST into desired representation of the MIB. JSON document generator PySNMP MIB generator Code generation stub Borrow pre-compiled MIBs Some MIBs in circulation appear broken beyond automatic repair. To handle such cases PySMI introduces the MIB borrowing functionality. When MibCompiler gives up compiling a MIB, it can try to go out and take a copy of already transformed MIB to complete the request successfully. Any file borrower Python file borrower Write compiled MIBs Successfully transformed MIB modules' contents will be passed to writer object given to MibCompiler on instantiation. File writer Python file writer Callback writer CallbackWriter class instance passes the contents of the compiled MIB files to a user object. Examples The following examples focus on various feature of the PySMI library. Compile MIBs into JSON Look up specific ASN.1 MIBs at configured Web and FTP sites, compile them into JSON documents and print them out to stdout. Try to support both SMIv1 and SMIv2 flavors of SMI as well as popular deviations from official syntax found in the wild. from pysmi.reader import FileReader, HttpReader from pysmi.searcher import StubSearcher from pysmi.writer import CallbackWriter from pysmi.parser import SmiStarParser from pysmi.codegen import JsonCodeGen from pysmi.compiler import MibCompiler # from pysmi import debug # debug.setLogger(debug.Debug('reader', 'compiler')) inputMibs = ['IF-MIB', 'IP-MIB'] srcDirectories = ['/usr/share/snmp/mibs'] httpSources = [ ('mibs.snmplabs.com', 80, '/asn1/@mib@') ] def printOut(mibName, jsonDoc, cbCtx): print('\n\n# MIB module %s' % mibName) print(jsonDoc) # Initialize compiler infrastructure mibCompiler = MibCompiler( SmiStarParser(), JsonCodeGen(), CallbackWriter(printOut) ) # search for source MIBs here mibCompiler.addSources(*[FileReader(x) for x in srcDirectories]) # search for source MIBs at Web sites mibCompiler.addSources(*[HttpReader(*x) for x in httpSources]) # never recompile MIBs with MACROs mibCompiler.addSearchers(StubSearcher(*JsonCodeGen.baseMibs)) # run recursive MIB compilation results = mibCompiler.compile(*inputMibs) print('\n# Results: %s' % ', '.join(['%s:%s' % (x, results[x]) for x in results])) Download script. Compile MIBs from web Look up specific ASN.1 MIBs at configured Web and FTP sites, compile them into pysnmp form and save Python modules as plain-text files in a local directory. Try to support both SMIv1 and SMIv2 flavors of SMI as well as popular deviations from official syntax found in the wild. In this example we disable automatic dependency checking on MIB compilation using noDeps flag. Also, we do not check if target file already exists thus MIB compilation occurs on every invocation. from pysmi.reader import HttpReader from pysmi.reader import FtpReader from pysmi.searcher import StubSearcher from pysmi.writer import PyFileWriter from pysmi.parser import SmiStarParser from pysmi.codegen import PySnmpCodeGen from pysmi.compiler import MibCompiler inputMibs = ['IF-MIB', 'IP-MIB'] httpSources = [ ('mibs.snmplabs.com', 80, '/asn1/@mib@') ] ftpSources = [ ('ftp.cisco.com', '/pub/mibs/v2/@mib@') ] dstDirectory = '.pysnmp-mibs' # Initialize compiler infrastructure mibCompiler = MibCompiler( SmiStarParser(), PySnmpCodeGen(), PyFileWriter(dstDirectory) ) # search for source MIBs at Web and FTP sites mibCompiler.addSources(*[HttpReader(*x) for x in httpSources]) mibCompiler.addSources(*[FtpReader(*x) for x in ftpSources]) # never recompile MIBs with MACROs mibCompiler.addSearchers(StubSearcher(*PySnmpCodeGen.baseMibs)) # run non-recursive MIB compilation results = mibCompiler.compile(*inputMibs, **dict(noDeps=True)) print('Results: %s' % ', '.join(['%s:%s' % (x, results[x]) for x in results])) Download script. Compile SMIv1/v2 MIBs Look up specific ASN.1 MIBs at configured local directories, compile them into pysnmp form if not done yet and save Python modules as plain-text files in a local directory. Try to support both SMIv1 and SMIv2 flavors of SMI as well as popular deviations from official syntax found in the wild. When figuring out if compilation is needed, check all known places where pysnmp MIBs could possibly be found. You can force MIB re-compilation by passing rebuild flag to MIB compiler (see below). Default invocation of MIB compiler does not generate [potentially large] comments and texts found in MIBs. If you need them in pysnmp MIB modules, just pass genTexts flag to MIB compiler. from pysmi.reader import FileReader from pysmi.searcher import PyFileSearcher, PyPackageSearcher, StubSearcher from pysmi.writer import PyFileWriter from pysmi.parser import SmiStarParser from pysmi.codegen import PySnmpCodeGen from pysmi.compiler import MibCompiler inputMibs = ['IF-MIB', 'IP-MIB'] srcDirectories = ['/usr/share/snmp/mibs'] dstDirectory = '.pysnmp-mibs' # Initialize compiler infrastructure mibCompiler = MibCompiler(SmiStarParser(), PySnmpCodeGen(), PyFileWriter(dstDirectory)) # search for source MIBs here mibCompiler.addSources(*[FileReader(x) for x in srcDirectories]) # check compiled MIBs in our own productions mibCompiler.addSearchers(PyFileSearcher(dstDirectory)) # ...and at default PySNMP MIBs packages mibCompiler.addSearchers(*[PyPackageSearcher(x) for x in PySnmpCodeGen.defaultMibPackages]) # never recompile MIBs with MACROs mibCompiler.addSearchers(StubSearcher(*PySnmpCodeGen.baseMibs)) # run [possibly recursive] MIB compilation results = mibCompiler.compile(*inputMibs) #, rebuild=True, genTexts=True) print('Results: %s' % ', '.join(['%s:%s' % (x, results[x]) for x in results])) Download script. Compile SMIv2 MIBs Invoke user callback function to provide MIB text, compile given text string into pysnmp MIB form and pass results to another user callback function for storing. Here we expect to deal only with SMIv2-valid MIBs. We use noDeps flag to prevent MIB compiler from attemping to compile IMPORT'ed MIBs as well. import sys from pysmi.reader import CallbackReader from pysmi.searcher import StubSearcher from pysmi.writer import CallbackWriter from pysmi.parser import SmiV2Parser from pysmi.codegen import PySnmpCodeGen from pysmi.compiler import MibCompiler inputMibs = ['IF-MIB', 'IP-MIB'] srcDir = '/usr/share/snmp/mibs/' # we will read MIBs from here # Initialize compiler infrastructure mibCompiler = MibCompiler( SmiV2Parser(), PySnmpCodeGen(), # out own callback function stores results in its own way CallbackWriter(lambda m, d, c: sys.stdout.write(d)) ) # our own callback function serves as a MIB source here mibCompiler.addSources( CallbackReader(lambda m, c: open(srcDir+m+'.txt').read()) ) # never recompile MIBs with MACROs mibCompiler.addSearchers(StubSearcher(*PySnmpCodeGen.baseMibs)) # run non-recursive MIB compilation results = mibCompiler.compile(*inputMibs, **dict(noDeps=True)) print('Results: %s' % ', '.join(['%s:%s' % (x, results[x]) for x in results])) Download script. Borrow pysnmp MIBs on failure Look up specific ASN.1 MIBs at configured Web/FTP sites. If no required MIB is found or its compilation fails for some reason, attempt to download precompiled version of failed MIB and store it locally as if we had compiled it. from pysmi.reader import HttpReader from pysmi.searcher import PyFileSearcher from pysmi.searcher import StubSearcher from pysmi.borrower import PyFileBorrower from pysmi.writer import PyFileWriter from pysmi.parser import SmiStarParser from pysmi.codegen import PySnmpCodeGen from pysmi.compiler import MibCompiler # from pysmi import debug # debug.setLogger(debug.Debug('borrower', 'reader', 'searcher')) inputMibs = ['BORROWED-MIB'] httpSources = [ ('mibs.snmplabs.com', 80, '/asn1/@mib@') ] httpBorrowers = [ ('mibs.snmplabs.com', 80, '/pysnmp/notexts/@mib@') ] dstDirectory = '.pysnmp-mibs' # Initialize compiler infrastructure mibCompiler = MibCompiler( SmiStarParser(), PySnmpCodeGen(), PyFileWriter(dstDirectory) ) # search for source MIBs at Web sites mibCompiler.addSources(*[HttpReader(*x) for x in httpSources]) # never recompile MIBs with MACROs mibCompiler.addSearchers(StubSearcher(*PySnmpCodeGen.baseMibs)) # check compiled/borrowed MIBs in our own productions mibCompiler.addSearchers(PyFileSearcher(dstDirectory)) # search for compiled MIBs at Web sites if source is not available or broken mibCompiler.addBorrowers(*[PyFileBorrower(HttpReader(*x)).setOptions(genTexts=False) for x in httpBorrowers]) # run non-recursive MIB compilation results = mibCompiler.compile(*inputMibs) print('Results: %s' % ', '.join(['%s:%s' % (x, results[x]) for x in results])) Download script. Always borrow pysnmp MIBs Try to borrow precompiled pysnmp MIB file(s) from a web-site. In this example no attempt is made to find and compile ASN.1 MIB source. Fetched pysnmp MIB(s) are stored in a local directory. from pysmi.reader import HttpReader from pysmi.searcher import PyFileSearcher from pysmi.borrower import PyFileBorrower from pysmi.writer import PyFileWriter from pysmi.parser import NullParser from pysmi.codegen import NullCodeGen from pysmi.compiler import MibCompiler inputMibs = ['BORROWED-MIB'] httpBorrowers = [ ('mibs.snmplabs.com', 80, '/pysnmp/notexts/@mib@') ] dstDirectory = '.pysnmp-mibs' # Initialize compiler infrastructure mibCompiler = MibCompiler( NullParser(), NullCodeGen(), PyFileWriter(dstDirectory) ) # check compiled/borrowed MIBs in our own productions mibCompiler.addSearchers(PyFileSearcher(dstDirectory)) # search for precompiled MIBs at Web sites mibCompiler.addBorrowers( *[PyFileBorrower(HttpReader(*x)) for x in httpBorrowers] ) # run MIB compilation results = mibCompiler.compile(*inputMibs) print('Results: %s' % ', '.join(['%s:%s' % (x, results[x]) for x in results])) Download script. In case of any troubles or confusion, try enabling PySMI debugging and watch the output: from pysmi import debug debug.setLogger(debug.Debug('all')) SOURCE CODE & CHANGELOG Project source code is hosted at GitHub. Everyone is welcome to fork and contribute back! We maintain the detailed log of changes to our software. DOWNLOAD & INSTALL Download & Install The best way to obtain SNMP SMI library is by running pip: $ virtualenv venv $ source venv/bin/activate $ pip install pysmi Alternatively, you can download the latest release from GitHub or PyPI. CHANGES Changelog Revision 0.3.4, XX-01-2019 o Added implied key to JSON SNMP table index structure o Rebased MIB importing code onto importlib because imp is long deprecated o Fixed Py file borrower to become functional Revision 0.3.3, 29-12-2018 o Added mibcopy.py documentation o Copyright notice bumped up to year 2019 Revision 0.3.2, 22-10-2018 o Bumped upper Python version to 3.7 and enabled pip cache o Exit code indication of the command-line tools aligned with sysexits.h to report more useful termination status Revision 0.3.1, 10-06-2018 o Fixed pysnmp lower version in test-requirements.txt o Fixed compiler crash when building comments at a platform which has broken users/groups databases Revision 0.3.0, 29-04-2018 o The mibcopy tool implemented to copy MIB modules from files with potentially messed up names into a directory under canonical MIB names picking up the latest MIB revision along the way. o ZIP archive reader implemented to pull ASN.1 MIB files from .zip archives pretty much in the same way as from plain directories o HTTP/S proxy support added (through respecting http_proxy environment variable) by switching from httplib to urllib2 internally o Copyright notice bumped up to year 2018 o Project site in the docs changes from SourceForge to snmplabs.com o PRODUCT-RELEASE generation added to the JSON code generator o Added special handling of BITS-like DEFVAL syntax for Integers that occurs in buggy MIBs o Fixed missing REVISIONS generations in MODULE-IDENTITY Revision 0.2.2, 13-11-2017 o Library documentation refactored and updated o Fixed malformed Python code being produced by pysnmp code generator Revision 0.2.1, 11-11-2017 o Added MIB status, product release and revision description set calls at pysnmp code generator o Changed REVISION field format in JSON representation - it is now a list of dicts each with revision timestamp and description text o MIB REFERENCE fields are only exported if --with-mib-text is on o Sphinx documentation theme changed to Alabaster o Multiple fixes to pysnmp codegen not to produce function calls with more than 255 parameters Revision 0.1.4, 14-10-2017 o Fix to SMI lexer to treat tokens starting from a digit as belonging to a lower-cased class. This fixes sub-OID parsing bug (specifically, 802dot3(10006)) o Fix to the mibdump.py local MIB path automatic injection in front of existing --mib-sources Revision 0.1.3, 19-05-2017 o INET-ADDRESS-MIB configured as pre-built at pysnmp codegen o JSON codegen produces "nodetype" element for OBJECT-TYPE o Fix to mibdump.py --destination-directory option o Fix to pysnmp and JSON code generators to properly refer to MIB module defining particular MIB object Revision 0.1.2, 12-04-2017 o The @mib@ magic in reader's URL template made optional. If it is not present, MIB module name is just appended to URL template o Send User-Agent containing pysmi and Python versions as well as platform name. o Fixed missing STATUS/DISPLAY-HINT/REFERENCE/etc fields generation at pysnmp backend when running in the non-full-text mode o Fixed broken ordereddict dependency on Python 2.6- Revision 0.1.1, 30-03-2017 o Generate REFERENCE and STATUS fields at various SMI objects o Generate DESCRIPTION field followed REVISION field at MODULE-IDENTITY objects o Generate PRODUCT-RELEASE field at AGENT-CAPABILITIES objects o Generated Python source aligned with PEP8 o MIB texts cleaned up by default, --keep-texts-layout preserves original formatting o Fix to the ordereddict conditional dependency o Missing test module recovered o Failing tests fixed Revision 0.1.0, 25-03-2017 o JSON code generating backend implemented o Experimental JSON OID->MIB indices generation implemented o Package structure flattened for easier use o Minor refactoring to the test suite o Source code statically analyzed, hardened and PEP8-ized o Files closed explicitly to mute ResourceWarnings o Fixed to Python 2.4 (and aged ply) compatibility o Added a workaround to avoid generating pysnmp TextualConvention classes inheriting from TextualConvention (when MIB defines a TEXTUAL-CONVENTION based on another TEXTUAL-CONVENTION as SYNTAX) o Author's e-mail changed, copyright extended to year 2017 Revision 0.0.7, 12-02-2016 o Crash on existing .py file handling fixed. o Fix to __doc__ use in setup.py to make -O0 installation mode working. o Fix to PyPackageSearcher not to fail on broken Python packages. o Source code pep8'ed o Copyright added to source files. Revision 0.0.6, 01-10-2015 o Several typos fixed, source code linted again. o Some dead code cleaned up. Revision 0.0.5, 28-09-2015 o Wheel distribution format now supported. o Handle the case of MIB symbols conflict with Python reserved words. o Handle binary DEFVAL initializer for INTEGER's. o Generate LAST-UPDATED at pysnmp code generator. Revision 0.0.4, 01-07-2015 o Fix to MRO compliance for mixin classes generation at pysnmp backend o Fix to repeated imports in generated code at pysnmp backend o Fix to mibdump tool to properly handle the --generate-mib-texts option. o Fix to Python compile() - optimize flag is valid only past Python 3.1 o Fix to SMIv1 INDEX clause code generation for pysnmp backend. o Tighten file creation security at pysmi.writer.pyfile Revision 0.0.3, 28-06-2015 o Two-pass compiler design allows for much accurate code generation. o Sphinx-based documentation first introduced Revision 0.0.0, 11-04-2015 o First public release, not fully operational yet LICENSE The SNMP SMI library software is distributed under 2-clause BSD License. License Copyright (c) 2015-2019 Ilya Etingof All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: o Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. o Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. MIB FILES ARCHIVE The PySMI project maintains a collection of publicly available ASN.1 MIB files collected on the Internet. You are welcome to use this MIBs archive however we can't guarantee any degree of consistency or reliability when it comes to these MIB modules. The mibdump.py tool as well as many other utilities based on PySMI are programmed to use this MIB repository for automatic download and dependency resolution. You can always reconfigure PySMI to use some other remote MIB repository instead or in addition to this one. CONTACT In case of questions or troubles using SNMP SMI library, please open up an issue at GitHub or ask at Stack Overflow . AUTHOR Ilya Etingof COPYRIGHT 2015-2024, Ilya Etingof 0.1 April 10, 2024 PYSMI(1)