SG3_UTILS_JSON(8) SG3_UTILS SG3_UTILS_JSON(8)

sg3_utils_json - JSON output for some sg3_utils utilities

sg_* --json[=JO] [--js-file=JFN] [OTHER_OPTIONS] [DEVICE]

sg3_utils is a package of utilities that send SCSI commands to the given DEVICE via a SCSI pass through interface provided by the host operating system. Some utilities, mainly those decoding structured data returned by SCSI commands (e.g. sg_vpd) can optionally provide JSON output, rather than simple, plain text output. The default remains plain text output.

JavaScript Object Notation (JSON) is an open standard file format that can be used for data exchange between programs including across a network. See https://en.wikipedia.org/wiki/JSON . JSON comes in many flavours and this one uses the json-builder C implementation found at https://github.com/json-parser/json-builder which implements four simple JSON data types: string, integer, boolean and null. Its other data types are JSON object and JSON array.

This project uses the 'snake_case' convention for JSON object names: all in lower case letters or numerals with individual words joined with a single underscore (e.g. "starting_lba"). There should be no leading or trailing underscore characters. The json-builder library uses the SPDX-License-Identifier: BSD-2-Clause which is the same license as the bulk of the utilities in the sg3_utils package.

The json-builder library is relatively lightweight (700 lines of C code) and is 'hidden' fully within the sg3_utils library so that its function interface and data types are not available (directly) to the utilities in the sg3_utils package. That is why the json-builder interface (a file named sg_json_builder.h) is in the lib directory and not in the include directory. As presented on github, json-builder shares some header files with its companion json-parser. The author has modified the json-builder header to include what is needed from the json-parser header so that only the builder and not the parser are built. The parser could be added later, but currently there seems to be no need for it.

The user interface to JSON functionality in the sg3_utils package is heavily based on what has been done by Christian Franke and others in smartctl, a utility in the smartmontools package for getting S.M.A.R.T. information from disks (and other storage devices).

This manpage discusses the --json[=JO] and --js-file=JFN command line options. Notice that the argument to --json is itself optional. In its shorter form the --json option may either be -j or -J (lower case preferred if not already in use). The shorter form may also take an argument but an "=" must precede the JO argument with no spaces either side of the "=".

Some care has been taken with quotes in this manpage. Double quotes are used around JSON object names, single quotes are used for all other purposes.

The SG3_UTILS_JSON_OPTS environment variable allows the user to override the default values of the JO settings. Those settings can again be overridden by the command line --json[=JO] option. If the string associated with SG3_UTILS_JSON_OPTS cannot be parsed this error message is sent to stderr: 'error parsing SG3_UTILS_JSON_OPTS environment variable, ignore'.

Since the argument to --json[=JO] is optional, in the shorter form there can be no space(s) between the option and its argument.

JO is a string of zero or more characters whose order is not significant apart from the negation characters ('-' is preferred). The negation character must appear immediately before the (boolean) feature it is toggling.
In the short form the option letter may be other than -j if that letter has already been used (-J is preferred next). For example the sg_ses utility uses -j for its 'join' operation. Also since the argument to the short form option is itself optional, there can be no spaces between the short form option, the "=", and JO, if it is given. Some short form examples: '-jl' requests JSON output (with no argument option) and whatever the '-l' short option means in the invoked utility; '-j=h' requests JSON output with some integers additional rendered as JSON strings of hex.
output is in JSON format and it is sent to a file named JFN. If that file exists then it is truncated. By default, the JSON output is sent to stdout.
When this option is given, the --json[=JO] option is implied and need not be given. The --json[=JO] option may still be needed to set the JO parameter to non-default values.
The short option may be other than -J if that short option was already in use.

Each JO string is made up of zero or more of the following JSON control characters.

0
If pretty printing JSON output, tab to 2 spaces.
2
If pretty printing JSON output, tab to 2 spaces.
4
If pretty printing JSON output, tab to 4 spaces.
This is the default tab setting for pretty printing JSON.
8
If pretty printing JSON output, tab to 8 spaces.
-
negation character. Toggles the (boolean) sense of the following control character.
this is a boolean control character for "exit status". If active an "exit status" field is placed at the end of the JSON output. The integer value of this field is the Unix exit status value that is return to the operating system when this utility exits. The value of 0 is a good exit (i.e. no errors detected).
This boolean control character is default on (true).
this is a boolean control character for 'hexadecimal'. Many values associated with SCSI are best (or at least historically) viewed in hexadecimal while JSON output prefers decimal integers (assumed to have a maximum size of 64 bits, signed). The maximum size of most SCSI fields is 64 bit _unsigned_ . Also some SCSI fields are masks which are best viewed in hex. When this control character is active most (non-trivial) fields that have an integer value instead receive a a sub-object containing at least a "i" field with the integer value and a "hex" field with the corresponding hex value in a JSON string. That hex string has no hex decorations (i.e. no leading '0x' nor trailing 'h').
This boolean control character is default off (false).
this is a boolean control character for finer control of non-pretty printed JSON output. If the 'p' control character is set on (true) then this option has no effect.
If the 'p' control character is set off (false) and this control character is set off (false) then the single line JSON output contains some spaces for readability. If the 'p' control character is set off (false) and this control character is set on (true) then the JSON single line JSON output is 'packed' removing all unnecessary spaces.
This boolean control character is default off (false).
this is a boolean control character to control whether lead-in fields are output. Lead-in fields are at the start of the JSON output and include "json_format_version" and "utility_invoked" sub-objects. The "utility_invoked" sub-object includes "name", "version_date" string fields and an JSON array named "argv" with an entry for each command line argument. If the o control character is also active, then if available, the non_JSON output (i.e. the original, 'plain text' form) is placed in an array called "plain_text_output" with one element per line of 'normal' output.
This boolean control character is default on (true).
this is a boolean control character for "name_extra". It is used to provide additional information about the name it is a sub-object of. The most common usage is to spell out an abbreviated name (e.g. a T10 name like 'SKSV' to 'Sense Key Specific Valid'). Another use is to note that a T10 field is obsolete and in which T10 standard it first became obsolete. Also if the named field's value is a physical quantity where the unit is unclear (e.g. a timeout) then "name_extra" can state that (e.g. 'unit: millisecond'). Only some fields have associated "name_extra" data.
This boolean control character is default off (false).
this is a boolean control character to control whether normal (i.e. non-JSON) lines of output are placed in a JSON array (one element per line of normal output) within the utility_invoked subject (see control character l). The name of the array is "plain_text_output". This control character is active even if the lead-in fields control character (l) is negated.
This boolean control character is default off (false).
this boolean control character controls whether the JSON output is 'pretty printed' or sent in a relatively compact stream suitable for more efficient transmission over a communications channel.
The pretty printed form of output has one JSON name with its associated integer, string or boolean value per line; and one array element per line. JSON objects and arrays that have an associated JSON object as their value, have their name on a separate line. These lines are indented with the current tab setting to indicate the level of nesting. Basically the pretty printed form is for human consumption.
There are two forms of non-pretty printed output, see the 'packed' control character ['k'].
This boolean control character is default on (true).
this boolean control character controls whether T10 field values that have a defined meaning are broken out with an added JSON sub-object usually named "meaning". When active the field name has a sub-object that contains at least an "i" field with the integer value of the field and a JSON string object, usually named "meaning", with a string that corresponds to the T10 defined meaning of the value in the "i" field.
This boolean control character is default on (true).
this is an integer control character that controls the amount of debug output. It can be given multiple times to increase the level of JSON debug verbosity in the output.
Note that this verbose control character is JSON specific while the --verbose option (short form: fI-v often repeated: fI-vvv) that most utilities support is more general.
This integer control character is set to 0 by default.

The default remains the same for all utilities that support the --json option, namely the decoded information is sent to stdout in plain text form. Errors are reported to stderr and may cause the early termination of a utility (e.g. command line option syntax error).

When the --json option is given and no errors are detected, then only JSON is normally sent to stdout. As the SCSI response is parsed, a JSON representation is built as a tree in memory. After all other actions (perhaps apart from the final exit status report) that JSON tree is 'dumped' to stdout. This means if there is any non-JSON output sent to stdout that it will appear _before_ the JSON output.

If the 'o' control character is in the JO argument to the --json option, then the former plain text output is placed in a JSON array named "plain_text_output" within a JSON object named "utility_invoked".
Each line of the former 'plain text' output is placed in its own element of the JSON array.

A JSON tree is built in memory as the utility parses the data returned from the SCSI device (e.g. sg_vpd parsing a VPD page returned from a SCSI INQUIRY command). SCSI 'list's become JSON named arrays (e.g. in the Device Identification VPD page there is a 'Designation descriptor list' that becomes a JSON array named "designation_descriptor_list").

At the completion of the utility that JSON tree is 'measured' taking into account the form of output (i.e. pretty-printed, single line or packed single line). For the pretty-printed JSON output, the size of each indentation in spaces is also given (i.e. the tab width). The JSON is then output to a single C string, then sent to stdout. If a NULL character (ASCII zero and C string terminator) somehow finds its way into a field that should (according to the spec) be space padded, then the JSON output may appear truncated.

Note that this JSON processing means that if a utility is aborted for whatever reason then no JSON output will appear. With the normal, plain text output processing, some output may appear before the utility aborts in such bad situations.

In general, the JSON generated by this package outputs 1 bit SCSI fields as the integer value 0 (for false) and 1 (for true). This follows the SCSI convention which predates the common use of boolean. Also SCSI reserved fields are output as the integer value 0. Extensions to SCSI commands and associated data descriptors typically use parts of commands or data descriptors that were previously reserved.

As stated above, the default output is in plain text form using 7 bit ASCII. The --json[=JO] option is designed to be an alternative to that plain text form. There are other alternative output formats such as the response output as a hexadecimal sequence of bytes or in 'raw' binary output; both of those take precedence over the --json[=JO] option. Other specialized output format (e.g. 'sg_inq --export') will usually take precedence over JSON output.

When the --raw option is used together with the --inhex=FN option only the data input to the utility is interpreted as binary. So the output format defaults to plain text form and thus can be changed to JSON if the --json[=JO] option is also used.

There is typically only one form of JSON output so options like --brief and --quiet are ignored in the JSON output. In some cases (i.e 'sg_inq --descriptors') the JSON output is expanded.

No attempts have been made to translate errors into JSON form, apart from the final "exit_status" JSON object where a value of 0 means 'no errors'. Exit status values indicating a problem range from 1 to 255.

The sg_decode_sense utility will parse SCSI sense data into JSON form if requested. So if another utility is failing with a sense data report (most often seen when the --verbose option is used). That sense data (in hex bytes) could be cut-and-pasted onto the command line following 'sg_decode_sense -j ' which should then render that sense data in JSON.

Otherwise, when a error is detected while JSON output is selected, the error message is sent to stderr in plain text form. Typically once an error is detected the utility will exit, first dumping the JSON in-memory tree as discussed above and a non-zero exit status will be set. The JSON output will be well formed but missing any fields or list elements following the point that the error was detected.

The summary is that when JSON output is selected and an error occurs each utility will process the error the same way as it would if JSON output had not been selected. In most cases error messages, in plain text form, are sent to stderr.

Written by Douglas Gilbert. Some utilities have been contributed, see the CREDITS file and individual source files (in the 'src' directory).

Report bugs to <dgilbert at interlog dot com>.

Copyright © 2022-2023 Douglas Gilbert
This software is distributed under the GPL version 2 or the BSD-2-Clause license. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

sg3_utils(sg3_utils), smartctl(smartmontools)

August 2023 sg3_utils-1.48