.\" Automatically generated by Pandoc 3.1.8
.\"
.TH "" "" "" "" ""
.SS NAME
sysdig - the definitive system and process troubleshooting tool
.SS SYNOPSIS
\f[B]sysdig\f[R] [\f[I]option\f[R]]...
[\f[I]filter\f[R]]
.SS DESCRIPTION
\f[B]Note: if you are interested in an easier to use interface for the
sysdig functionality, use the csysdig command line utility.\f[R]
.PP
sysdig is a tool for system troubleshooting, analysis and exploration.
It can be used to capture, filter and decode system calls and other OS
events.
sysdig can be both used to inspect live systems, or to generate trace
files that can be analyzed at a later stage.
.PP
sysdig includes a powerful filtering language, has customizable output,
and can be extended through Lua scripts, called chisels.
.PP
\f[B]Output format\f[R]
.PP
By default, sysdig prints the information for each captured event on a
single line, with the following format:
.PP
\f[CR]*%evt.num %evt.time %evt.cpu %proc.name (%thread.tid) %evt.dir %evt.type %evt.info\f[R]
.PP
where:
.IP \[bu] 2
evt.num is the incremental event number
.IP \[bu] 2
evt.time is the event timestamp
.IP \[bu] 2
evt.cpu is the CPU number where the event was captured
.IP \[bu] 2
proc.name is the name of the process that generated the event
.IP \[bu] 2
thread.tid id the TID that generated the event, which corresponds to the
PID for single thread processes
.IP \[bu] 2
evt.dir is the event direction, > for enter events and < for exit events
.IP \[bu] 2
evt.type is the name of the event, e.g.
\[aq]open\[aq] or \[aq]read\[aq]
.IP \[bu] 2
evt.args is the list of event arguments.
.PP
The output format can be customized with the -p switch, using any of the
fields listed by \[aq]sysdig -l\[aq].
.PP
Using -pc or -pcontainer, the default format will be changed to a
container-friendly one:
.PP
\f[CR]*%evt.num %evt.time %evt.cpu %container.name (%container.id) %proc.name (%thread.tid:%thread.vtid) %evt.dir %evt.type %evt.info\f[R]
.PP
\f[B]Trace Files\f[R]
.PP
A trace file can be created using the -w switch:
.RS
.PP
$ sysdig -w trace.scap
.RE
.PP
The -s switch can be used to specify how many bytes of each data buffer
should be saved to disk.
And filters can be used to save only certain events to disk:
.RS
.PP
$ sysdig -s 2000 -w trace.scap proc.name=cat
.RE
.PP
Trace files can be read this using the -r switch:
.RS
.PP
$ sysdig -r trace.scap
.RE
.PP
\f[B]Filtering\f[R]
.PP
sysdig filters are specified at the end of the command line.
The simplest filter is a basic field-value check:
.RS
.PP
$ sysdig proc.name=cat
.RE
.PP
The list of available fields can be obtained with \[aq]sysdig -l\[aq].
Filter expressions can use one of these comparison operators:
\f[I]=\f[R], \f[I]!=\f[R], \f[I]<\f[R], \f[I]<=\f[R], \f[I]>\f[R],
\f[I]>=\f[R], \f[I]contains\f[R], \f[I]icontains\f[R], \f[I]in\f[R] and
\f[I]exists\f[R].
e.g.
.RS
.PP
$ sysdig fd.name contains /etc $ sysdig \[dq]evt.type in (
\[aq]select\[aq], \[aq]poll\[aq] )\[dq] $ sysdig proc.name exists
.RE
.PP
Multiple checks can be combined through brackets and the following
boolean operators: \f[I]and\f[R], \f[I]or\f[R], \f[I]not\f[R].
e.g.
.RS
.PP
$ sysdig \[dq]not (fd.name contains /proc or fd.name contains /dev)\[dq]
.RE
.PP
\f[B]Chisels\f[R]
.PP
sysdig\[aq]s chisels are little scripts that analyze the sysdig event
stream to perform useful actions.
To get the list of available chisels, type
.RS
.PP
$ sysdig -cl
.RE
.PP
To get details about a specific chisel, type
.RS
.PP
$ sysdig -i spy_ip
.RE
.PP
To run one of the chisels, you use the -c flag, e.g.
.RS
.PP
$ sysdig -c topfiles_bytes
.RE
.PP
If a chisel needs arguments, you specify them after the chisel name:
.RS
.PP
$ sysdig -c spy_ip 192.168.1.157
.RE
.PP
If a chisel has more than one argument, specify them after the chisel
name, enclosed in quotes:
.RS
.PP
$ sysdig -c chisel_name \[dq]arg1 arg2 arg3\[dq]
.RE
.PP
Chisels can be combined with filters:
.RS
.PP
$ sysdig -c topfiles_bytes \[dq]not fd.name contains /dev\[dq]
.RE
.SS OPTIONS
\f[B]-A\f[R], \f[B]--print-ascii\f[R] Only print the text portion of
data buffers, and echo end-of-lines.
This is useful to only display human-readable data.
.PP
\f[B]-b\f[R], \f[B]--print-base64\f[R] Print data buffers in base64.
This is useful for encoding binary data that needs to be used over media
designed to handle textual data (i.e., terminal or json).
.PP
\f[B]-c\f[R] \f[I]chiselname\f[R] \f[I]chiselargs\f[R],
\f[B]--chisel\f[R]=\f[I]chiselname\f[R] \f[I]chiselargs\f[R] run the
specified chisel.
If the chisel require arguments, they must be specified in the command
line after the name.
.PP
\f[B]-C\f[R] \f[I]filesize\f[R] Break a capture into separate files, and
limit the size of each file based on the specified number of megabytes.
The units of \f[I]filesize\f[R] are millions of bytes (10\[ha]6, not
2\[ha]20).
Use in conjunction with \f[B]-W\f[R] to enable automatic file rotation.
Otherwise, new files will continue to be created until the capture is
manually stopped.
.PP
Files will have the name specified by \f[B]-w\f[R] with a counter added
starting at 0.
.PP
\f[B]-cl\f[R], \f[B]--list-chisels\f[R] lists the available chisels.
Sysdig looks for chisels in the following directories: ./chisels,
\[ti]/.chisels and /usr/share/sysdig/chisels.
.PP
\f[B]-d\f[R], \f[B]--displayflt\f[R] Make the given filter a display
one.
Setting this option causes the events to be filtered after being parsed
by the state system.
Events are normally filtered before being analyzed, which is more
efficient, but can cause state (e.g.
FD names) to be lost.
.PP
\f[B]-D\f[R], \f[B]--debug\f[R] Capture events about sysdig itself,
display internal events in addition to system events, and print
additional logging on standard error.
.PP
\f[B]-E\f[R], \f[B]--exclude-users\f[R] Don\[aq]t create the user/group
tables by querying the OS when sysdig starts.
This also means that no user or group info will be written to the
tracefile by the \f[B]-w\f[R] flag.
The user/group tables are necessary to use filter fields like user.name
or group.name.
However, creating them can increase sysdig\[aq]s startup time.
Moreover, they contain information that could be privacy sensitive.
.PP
\f[B]-e\f[R] \f[I]numevents\f[R] Break a capture into separate files,
and limit the size of each file based on the specified number of events.
Use in conjunction with \f[B]-W\f[R] to enable automatic file rotation.
Otherwise, new files will continue to be created until the capture is
manually stopped.
.PP
Files will have the name specified by \f[B]-w\f[R] with a counter added
starting at 0.
.PP
\f[B]-F\f[R], \f[B]--fatfile\f[R] Enable fatfile mode.
When writing in fatfile mode, the output file will contain events that
will be invisible when reading the file, but that are necessary to fully
reconstruct the state.
Fatfile mode is useful when saving events to disk with an aggressive
filter.
The filter could drop events that would cause the state to be updated
(e.g.
clone() or open()).
With fatfile mode, those events are still saved to file, but
\[aq]hidden\[aq] so that they won\[aq]t appear when reading the file.
Be aware that using this flag might generate substantially bigger traces
files.
.PP
\f[B]--filter-proclist\f[R] apply the filter to the process table.
A full dump of /proc is typically included in any trace file to make
sure all the state required to decode events is in the file.
This could cause the file to contain unwanted or sensitive information.
Using this flag causes the command line filter to be applied to the
/proc dump as well.
.PP
\f[B]-G\f[R] \f[I]numseconds\f[R] Break a capture into separate files,
and limit the size of each file based on the specified number of
seconds.
Use in conjunction with \f[B]-W\f[R] to enable automatic file rotation.
Otherwise, new files will continue to be created until the capture is
manually stopped.
.PP
Files will have the name specified by \f[B]-w\f[R] which should include
a time format as defined by strftime(3).
If no time format is specified, a counter will be used.
.PP
\f[B]-h\f[R], \f[B]--help\f[R] Print this page
.PP
\f[B]-H\f[R] \f[I]pluginname\f[R][:\f[I]initconfig\f[R]],
\f[B]--plugin\f[R] \f[I]pluginname\f[R][:\f[I]initconfig\f[R]] Registers
a plugin, using the passed init config if present.
A path can also be used as pluginname.
The format of initconf is controlled by the plugin, refer to each
plugin\[aq]s documentation to learn about it.
.PP
\f[B]-I\f[R] \f[I]pluginname\f[R][:\f[I]openparams\f[R]],
\f[B]--input\f[R] \f[I]pluginname\f[R][:\f[I]openparams\f[R]] Capture
events using the plugin with name pluginname, passing to the plugin the
openparams string as parameters.
The format of inputargs is controller by the plugin, refer to each
plugin\[aq]s documentation to learn about it.
The event sources available for capture vary depending on which plugins
have been installed.
You can list the plugins that have been loaded by using the -Il flag.
.PP
\f[B]-Il\f[R], \f[B]--list-inputs\f[R] List the loaded plugins.
Sysdig looks for plugins in the following directories: ./plugins,
\[ti]/.plugins, /usr/share/sysdig/plugins.
.PP
\f[B]--plugin-config-file\f[R] Load the plugin configuration from a
Falco-compatible yaml config file.
Mixing this option with \[aq]-H\[aq] or \[aq]-I\[aq] is unsupported.
See the plugin section in for
additional informations.
.PP
\f[B]-i \f[BI]chiselname\f[B]\f[R], \f[B]--chisel-info=\f[R]_chiselname_
Get a longer description and the arguments associated with a chisel
found in the -cl option list.
.PP
\f[B]-j\f[R], \f[B]--json\f[R] Emit output as json, data buffer encoding
will depend from the print format selected.
.PP
\f[B]-k\f[R], \f[B]--k8s-api\f[R] Enable Kubernetes support by
connecting to the API server specified as argument.
E.g.
\[dq]\[dq].
The API server can also be specified via the environment variable
SYSDIG_K8S_API.
.PP
\f[B]-K\f[R] \f[I]btfile |
certfile:keyfile[#password][:cacertfile]\f[R],
\f[B]--k8s-api-cert=\f[R]_btfile |
certfile:keyfile[#password][:cacertfile]_ Use the provided files names
to authenticate user and (optionally) verify the K8S API server
identity.
Each entry must specify full (absolute, or relative to the current
directory) path to the respective file.
Private key password is optional (needed only if key is password
protected).
CA certificate is optional.
For all files, only PEM file format is supported.
Specifying CA certificate only is obsoleted - when single entry is
provided for this option, it will be interpreted as the name of a file
containing bearer token.
Note that the format of this command-line option prohibits use of files
whose names contain \[aq]:\[aq] or \[aq]#\[aq] characters in the file
name.
Option can also be provided via the environment variable
SYSDIG_K8S_API_CERT.
.PP
\f[B]-L\f[R], \f[B]--list-events\f[R] List the events that the engine
supports
.PP
\f[B]-l\f[R], \f[B]--list\f[R] List the fields that can be used for
filtering and output formatting.
Use -lv to get additional information for each field.
.PP
\f[B]--list-markdown\f[R] Like -l, but produces markdown output
.PP
\f[B]-m\f[R] \f[I]url[,marathon-url]\f[R],
\f[B]--mesos-api=\f[R]_url[,marathon-url]_ Enable Mesos support by
connecting to the API server specified as argument (e.g.
).
Mesos url is required.
Marathon url is optional, defaulting to auto-follow - if Marathon API
server is not provided, sysdig will attempt to retrieve (and
subsequently follow, if it migrates) the location of Marathon API server
from the Mesos master.
Note that, with auto-follow, sysdig will likely receive a cluster
internal IP address for Marathon API server, so running sysdig with
Marathon auto-follow from a node that is not part of Mesos cluster may
not work.
Additionally, running sysdig with Mesos support on a node that has no
containers managed by Mesos is of limited use because, although cluster
metadata will be collected, there will be no Mesos/Marathon filtering
capability.
The API servers can also be specified via the environment variable
SYSDIG_MESOS_API.
.PP
\f[B]-M\f[R] \f[I]num_seconds\f[R] Stop collecting after reaching
.PP
\f[B]-n\f[R] \f[I]num\f[R], \f[B]--numevents\f[R]=\f[I]num\f[R]
.PD 0
.P
.PD
Stop capturing after \f[I]num\f[R] events
.PP
\f[B]--page-faults\f[R] Capture user/kernel major/minor page faults
.PP
\f[B]-P\f[R], \f[B]--progress\f[R]
.PD 0
.P
.PD
Print progress on stderr while processing trace files.
.PP
\f[B]-p\f[R] \f[I]outputformat\f[R],
\f[B]--print\f[R]=\f[I]outputformat\f[R]
.PD 0
.P
.PD
Specify the format to be used when printing the events.
With -pc or -pcontainer will use a container-friendly format.
With -pk or -pkubernetes will use a kubernetes-friendly format.
With -pm or -pmesos will use a mesos-friendly format.
Specifying \f[B]-pp\f[R] on the command line will cause sysdig to print
the default command line format and exit.
.PP
\f[B]-q\f[R], \f[B]--quiet\f[R]
.PD 0
.P
.PD
Don\[aq]t print events on the screen.
Useful when dumping to disk.
.PP
\f[B]-r\f[R] \f[I]readfile\f[R], \f[B]--read\f[R]=\f[I]readfile\f[R]
.PD 0
.P
.PD
Read the events from \f[I]readfile\f[R].
.PP
\f[B]-R\f[R], \f[B]--resolve-ports\f[R] Resolve port numbers to names.
.PP
\f[B]-S\f[R], \f[B]--summary\f[R]
.PD 0
.P
.PD
print the event summary (i.e.
the list of the top events) when the capture ends.
.PP
\f[B]-s\f[R] \f[I]len\f[R], \f[B]--snaplen\f[R]=\f[I]len\f[R]
.PD 0
.P
.PD
Capture the first \f[I]len\f[R] bytes of each I/O buffer.
By default, the first 80 bytes are captured.
Use this option with caution, it can generate huge trace files.
.PP
\f[B]-t\f[R] \f[I]timetype\f[R], \f[B]--timetype\f[R]=\f[I]timetype\f[R]
.PD 0
.P
.PD
Change the way event time is displayed.
Accepted values are \f[B]h\f[R] for human-readable string, \f[B]a\f[R]
for absolute timestamp from epoch, \f[B]r\f[R] for relative time from
the first displayed event, \f[B]d\f[R] for delta between event enter and
exit, and \f[B]D\f[R] for delta from the previous event.
.PP
\f[B]-T\f[R], \f[B]--force-tracers-capture\f[R]
.PD 0
.P
.PD
Tell the driver to make sure full buffers are captured from /dev/null,
to make sure that tracers are completely captured.
Note that sysdig will enable extended /dev/null capture by itself after
detecting that tracers are written there, but that could result in the
truncation of some tracers at the beginning of the capture.
This option allows preventing that.
.PP
\f[B]--unbuffered\f[R]
.PD 0
.P
.PD
Turn off output buffering.
This causes every single line emitted by sysdig to be flushed, which
generates higher CPU usage but is useful when piping sysdig\[aq]s output
into another process or into a script.
.PP
\f[B]-v\f[R], \f[B]--verbose\f[R]
.PD 0
.P
.PD
Verbose output.
This flag will cause the full content of text and binary buffers to be
printed on screen, instead of being truncated to 40 characters.
Note that data buffers length is still limited by the snaplen (refer to
the -s flag documentation) -v will also make sysdig print some summary
information at the end of the capture.
.PP
\f[B]--version\f[R]
.PD 0
.P
.PD
Print version number.
.PP
\f[B]-w\f[R] \f[I]writefile\f[R], \f[B]--write\f[R]=\f[I]writefile\f[R]
.PD 0
.P
.PD
Write the captured events to \f[I]writefile\f[R].
.PP
\f[B]-W\f[R] \f[I]num\f[R]
.PD 0
.P
.PD
Turn on file rotation for continuous capture, and limit the number of
files created to the specified number.
Once the cap is reached, older files will be overwritten (ring buffer).
Use in conjunction with the \f[B]-C\f[R] / \f[B]-G\f[R] / \f[B]-e\f[R]
options to limit the size of each file based on number of megabytes,
seconds, and/or events (respectively).
.PP
\f[B]-x\f[R], \f[B]--print-hex\f[R]
.PD 0
.P
.PD
Print data buffers in hex.
.PP
\f[B]-X\f[R], \f[B]--print-hex-ascii\f[R]
.PD 0
.P
.PD
Print data buffers in hex and ASCII.
.PP
\f[B]-z\f[R], \f[B]--compress\f[R]
.PD 0
.P
.PD
Used with \f[B]-w\f[R], enables compression for tracefiles.
.SS EXAMPLES
Capture all the events from the live system and print them to screen
.RS
.PP
$ sysdig
.RE
.PP
Capture all the events from the live system and save them to disk
.RS
.PP
$ sysdig -w dumpfile.scap
.RE
.PP
Capture all the events in the latest 24 hours and save them to disk
organized in files containing 1 hour of system activity each
.RS
.PP
$ sysdig -G 3600 -W 24 -w dumpfile.scap
.RE
.PP
Read events from a file and print them to screen
.RS
.PP
$ sysdig -r dumpfile.scap
.RE
.PP
Prepare a sanitized version of a system capture
.RS
.PP
$ sysdig -r dumpfile.scap \[aq]not evt.buffer contains foo\[aq] -w
cleandump.scap
.RE
.PP
Print all the open system calls invoked by cat
.RS
.PP
$ sysdig proc.name=cat and evt.type=open
.RE
.PP
Print the name of the files opened by cat
.RS
.PP
$ sysdig -p\[dq]%evt.arg.name\[dq] proc.name=cat and evt.type=open
.RE
.PP
List the available chisels
.RS
.PP
$ sysdig -cl
.RE
.PP
Use the spy_ip chisel to look at the data exchanged with 192.168.1.157:
.RS
.PP
$ sysdig -c spy_ip 192.168.1.157
.RE
.SS FILES
\f[I]/usr/share/sysdig/chisels\f[R]
.PD 0
.P
.PD
The global chisels directory.
.PP
\f[I]\[ti]/.chisels\f[R]
.PD 0
.P
.PD
The personal chisels directory.
.SS BUGS
.IP \[bu] 2
sysdig and its chisels are designed to be used with LuaJIT in Lua 5.1
mode.
While it is possible to use sysdig with LuaJIT in Lua 5.2 mode or
regular Lua, some chisels may not work as expected.
.SS AUTHOR
Draios Inc.
aka sysdig
.SS SEE ALSO
\f[B]csysdig\f[R](8), \f[B]strace\f[R](8), \f[B]tcpdump\f[R](8),
\f[B]lsof\f[R](8)