Data Collection

Whenever a command is executed with instrumentation enabled, a v1 Snippet File is created in the project build tree with data specific to that command. These files remain until after Indexing occurs.

CMake sets the RULE_LAUNCH_COMPILE, RULE_LAUNCH_LINK and RULE_LAUNCH_CUSTOM global properties to use the ctest --instrument launcher in order to capture details of each compile, link and custom command respectively. If the project has been configured with CTestUseLaunchers, ctest --instrument will also include the behavior usually performed by ctest --launch.

Indexing

Indexing is the process of collating generated instrumentation data. Indexing occurs at specific intervals called hooks, such as after every build. These hooks are configured as part of the v1 Query Files. Whenever a hook is triggered, an index file is generated containing a list of snippet files newer than the previous indexing.

Indexing and can also be performed by manually invoking ctest --collect-instrumentation <build>.

Callbacks

As part of the v1 Query Files, users can provide a list of callbacks intended to handle data collected by this feature.

Whenever Indexing occurs, each provided callback is executed, passing the path to the generated index file as an argument.

These callbacks, defined either at the user-level or project-level should read the instrumentation data and perform any desired handling of it. The index file and its listed snippets are automatically deleted by CMake once all callbacks have completed. Note that a callback should never move or delete these data files manually as they may be needed by other callbacks.

Enabling Instrumentation at the Project-Level

Project code can contain instrumentation queries with the cmake_instrumentation() command.

In addition, query files can be placed manually under <build>/.cmake/instrumentation/<version>/query/ at the top of a build tree. This version of CMake supports only one version schema, API v1.

Enabling Instrumentation at the User-Level

Instrumentation can be configured at the user-level by placing query files in the CMAKE_CONFIG_DIR under <config_dir>/instrumentation/<version>/query/.

Enabling Instrumentation for CDash Submissions

You can enable instrumentation when using CTest in Dashboard Client mode by setting the CTEST_USE_INSTRUMENTATION environment variable to the current UUID for the CMAKE_EXPERIMENTAL_INSTRUMENTATION feature. Doing so automatically enables the dynamicSystemInformation query.

The following table shows how each type of instrumented command gets mapped to a corresponding type of CTest XML file.

By default the command line reported to CDash is truncated at the first space. You can instead choose to report the full command line (including arguments) by setting CTEST_USE_VERBOSE_INSTRUMENTATION to 1.

v1 Query Files

Any file with the .json extension under the instrumentation/v1/query/ directory is recognized as a query for instrumentation data.

These files must contain a JSON object with the following keys. The version key is required, but all other fields are optional.

The callbacks listed will be invoked during the specified hooks at a minimum. When there are multiple query files, the callbacks, hooks and queries between them will be merged. Therefore, if any query file includes any hooks, every callback across all query files will be executed at every hook across all query files. Additionally, if any query file includes any optional queries, the optional query data will be present in all data files.

Example:

{
  "version": 1,
  "callbacks": [
    "/usr/bin/python callback.py",
    "/usr/bin/cmake -P callback.cmake arg",
  ],
  "hooks": [
    "postCMakeBuild",
    "postInstall"
  ],
  "queries": [
    "staticSystemInformation",
    "dynamicSystemInformation"
  ]
}

In this example, after every cmake --build or cmake --install invocation, an index file index-<timestamp>.json will be generated in <build>/.cmake/instrumentation/v1/data containing a list of data snippet files created since the previous indexing. The commands /usr/bin/python callback.py index-<timestamp>.json and /usr/bin/cmake -P callback.cmake arg index-<timestamp>.json will be executed in that order. The index file will contain the staticSystemInformation data and each snippet file listed in the index will contain the dynamicSystemInformation data. Once both callbacks have completed, the index file and all snippet files listed by it will be deleted from the project build tree.

v1 Snippet File

Snippet files are generated for every compile, link and custom command invoked as part of the CMake build or install step and contain instrumentation data about the command executed. Additionally, snippet files are created for the following:

These files remain in the build tree until after Indexing occurs and any user-specified Callbacks are executed.

Snippet files have a filename with the syntax <role>-<hash>-<timestamp>.json and contain the following data:

Example:

{
  "version": 1,
  "command" : "\"/usr/bin/c++\" \"-MD\" \"-MT\" \"CMakeFiles/main.dir/main.cxx.o\" \"-MF\" \"CMakeFiles/main.dir/main.cxx.o.d\" \"-o\" \"CMakeFiles/main.dir/main.cxx.o\" \"-c\" \"<src>/main.cxx\"",
  "role" : "compile",
  "return" : 1,
  "target": "main",
  "language" : "C++",
  "outputs" : [ "CMakeFiles/main.dir/main.cxx.o" ],
  "outputSizes" : [ 0 ],
  "source" : "<src>/main.cxx",
  "config" : "Debug",
  "dynamicSystemInformation" :
  {
    "afterCPULoadAverage" : 2.3500000000000001,
    "afterHostMemoryUsed" : 6635680.0
    "beforeCPULoadAverage" : 2.3500000000000001,
    "beforeHostMemoryUsed" : 6635832.0
  },
  "timeStart" : 1737053448177,
  "duration" : 31
}

v1 Index File

Index files contain a list of v1 Snippet File. It serves as an entry point for navigating the instrumentation data. They are generated whenever Indexing occurs and deleted after any user-specified Callbacks are executed.

Example:

{
  "version": 1,
  "hook": "manual",
  "buildDir": "<build>",
  "dataDir": "<build>/.cmake/instrumentation/v1/data",
  "snippets": [
    "configure-<hash>-<timestamp>.json",
    "generate-<hash>-<timestamp>.json",
    "compile-<hash>-<timestamp>.json",
    "compile-<hash>-<timestamp>.json",
    "link-<hash>-<timestamp>.json",
    "install-<hash>-<timestamp>.json",
    "ctest-<hash>-<timestamp>.json",
    "test-<hash>-<timestamp>.json",
    "test-<hash>-<timestamp>.json",
  ]
}