'\" t
.\" Title: perf-sched
.\" Author: [FIXME: author] [see http://www.docbook.org/tdg5/en/html/author]
.\" Generator: DocBook XSL Stylesheets vsnapshot
.\" Date: 2024-10-14
.\" Manual: perf Manual
.\" Source: perf
.\" Language: English
.\"
.TH "PERF\-SCHED" "1" "2024\-10\-14" "perf" "perf Manual"
.\" -----------------------------------------------------------------
.\" * Define some portability stuff
.\" -----------------------------------------------------------------
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.\" http://bugs.debian.org/507673
.\" http://lists.gnu.org/archive/html/groff/2009-02/msg00013.html
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.ie \n(.g .ds Aq \(aq
.el .ds Aq '
.\" -----------------------------------------------------------------
.\" * set default formatting
.\" -----------------------------------------------------------------
.\" disable hyphenation
.nh
.\" disable justification (adjust text to left margin only)
.ad l
.\" -----------------------------------------------------------------
.\" * MAIN CONTENT STARTS HERE *
.\" -----------------------------------------------------------------
.SH "NAME"
perf-sched \- Tool to trace/measure scheduler properties (latencies)
.SH "SYNOPSIS"
.sp
.nf
\fIperf sched\fR {record|latency|map|replay|script|timehist}
.fi
.SH "DESCRIPTION"
.sp
There are several variants of \fIperf sched\fR:
.sp
.if n \{\
.RS 4
.\}
.nf
\*(Aqperf sched record \*(Aq to record the scheduling events
of an arbitrary workload\&.
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
\*(Aqperf sched latency\*(Aq to report the per task scheduling latencies
and other scheduling properties of the workload\&.
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
Example usage:
perf sched record \-\- sleep 1
perf sched latency
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
Task | Runtime ms | Count | Avg delay ms | Max delay ms | Max delay start | Max delay end |
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
perf:(2) | 2\&.804 ms | 66 | avg: 0\&.524 ms | max: 1\&.069 ms | max start: 254752\&.314960 s | max end: 254752\&.316029 s
NetworkManager:1343 | 0\&.372 ms | 13 | avg: 0\&.008 ms | max: 0\&.013 ms | max start: 254751\&.551153 s | max end: 254751\&.551166 s
kworker/1:2\-xfs:4649 | 0\&.012 ms | 1 | avg: 0\&.008 ms | max: 0\&.008 ms | max start: 254751\&.519807 s | max end: 254751\&.519815 s
kworker/3:1\-xfs:388 | 0\&.011 ms | 1 | avg: 0\&.006 ms | max: 0\&.006 ms | max start: 254751\&.519809 s | max end: 254751\&.519815 s
sleep:147736 | 0\&.938 ms | 3 | avg: 0\&.006 ms | max: 0\&.007 ms | max start: 254751\&.313817 s | max end: 254751\&.313824 s
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
It shows Runtime(time that a task spent actually running on the CPU),
Count(number of times a delay was calculated) and delay(time that a
task was ready to run but was kept waiting)\&.
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
Tasks with the same command name are merged and the merge count is
given within (), However if \-p option is used, pid is mentioned\&.
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
\*(Aqperf sched script\*(Aq to see a detailed trace of the workload that
was recorded (aliased to \*(Aqperf script\*(Aq for now)\&.
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
\*(Aqperf sched replay\*(Aq to simulate the workload that was recorded
via perf sched record\&. (this is done by starting up mockup threads
that mimic the workload based on the events in the trace\&. These
threads can then replay the timings (CPU runtime and sleep patterns)
of the workload as it occurred when it was recorded \- and can repeat
it a number of times, measuring its performance\&.)
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
\*(Aqperf sched map\*(Aq to print a textual context\-switching outline of
workload captured via perf sched record\&. Columns stand for
individual CPUs, and the two\-letter shortcuts stand for tasks that
are running on a CPU\&. A \*(Aq*\*(Aq denotes the CPU that had the event, and
a dot signals an idle CPU\&.
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
\*(Aqperf sched timehist\*(Aq provides an analysis of scheduling events\&.
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
Example usage:
perf sched record \-\- sleep 1
perf sched timehist
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
By default it shows the individual schedule events, including the wait
time (time between sched\-out and next sched\-in events for the task), the
task scheduling delay (time between runnable and actually running) and
run time for the task:
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
time cpu task name wait time sch delay run time
[tid/pid] (msec) (msec) (msec)
\-\-\-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-
79371\&.874569 [0011] gcc[31949] 0\&.014 0\&.000 1\&.148
79371\&.874591 [0010] gcc[31951] 0\&.000 0\&.000 0\&.024
79371\&.874603 [0010] migration/10[59] 3\&.350 0\&.004 0\&.011
79371\&.874604 [0011] 1\&.148 0\&.000 0\&.035
79371\&.874723 [0005] 0\&.016 0\&.000 1\&.383
79371\&.874746 [0005] gcc[31949] 0\&.153 0\&.078 0\&.022
\&.\&.\&.
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
Times are in msec\&.usec\&.
.fi
.if n \{\
.RE
.\}
.SH "OPTIONS"
.PP
\-i, \-\-input=
.RS 4
Input file name\&. (default: perf\&.data unless stdin is a fifo)
.RE
.PP
\-v, \-\-verbose
.RS 4
Be more verbose\&. (show symbol address, etc)
.RE
.PP
\-D, \-\-dump\-raw\-trace=
.RS 4
Display verbose dump of the sched data\&.
.RE
.PP
\-f, \-\-force
.RS 4
Don\(cqt complain, do it\&.
.RE
.SH "OPTIONS FOR \FIPERF SCHED LATENCY\FR"
.PP
\-C, \-\-CPU
.RS 4
CPU to profile on\&.
.RE
.PP
\-p, \-\-pids
.RS 4
latency stats per pid instead of per command name\&.
.RE
.PP
\-s, \-\-sort
.RS 4
sort by key(s): runtime, switch, avg, max by default it\(cqs sorted by "avg ,max ,switch ,runtime"\&.
.RE
.SH "OPTIONS FOR \FIPERF SCHED MAP\FR"
.PP
\-\-compact
.RS 4
Show only CPUs with activity\&. Helps visualizing on high core count systems\&.
.RE
.PP
\-\-cpus
.RS 4
Show just entries with activities for the given CPUs\&.
.RE
.PP
\-\-color\-cpus
.RS 4
Highlight the given cpus\&.
.RE
.PP
\-\-color\-pids
.RS 4
Highlight the given pids\&.
.RE
.PP
\-\-task\-name
.RS 4
Map output only for the given task name(s)\&. Separate the task names with a comma (without whitespace)\&. The sched\-out time is printed and is represented by
\fI*\-\fR
for the given task name(s)\&. (\fI\-\fR
indicates other tasks while
\fI\&.\fR
is idle)\&.
.RE
.PP
\-\-fuzzy\-name
.RS 4
Given task name(s) can be partially matched (fuzzy matching)\&.
.RE
.SH "OPTIONS FOR \FIPERF SCHED TIMEHIST\FR"
.PP
\-k, \-\-vmlinux=
.RS 4
vmlinux pathname
.RE
.PP
\-\-kallsyms=
.RS 4
kallsyms pathname
.RE
.PP
\-g, \-\-call\-graph
.RS 4
Display call chains if present (default on)\&.
.RE
.PP
\-\-max\-stack
.RS 4
Maximum number of functions to display in backtrace, default 5\&.
.RE
.PP
\-C=, \-\-cpu=
.RS 4
Only show events for the given CPU(s) (comma separated list)\&.
.RE
.PP
\-p=, \-\-pid=
.RS 4
Only show events for given process ID (comma separated list)\&.
.RE
.PP
\-t=, \-\-tid=
.RS 4
Only show events for given thread ID (comma separated list)\&.
.RE
.PP
\-s, \-\-summary
.RS 4
Show only a summary of scheduling by thread with min, max, and average run times (in sec) and relative stddev\&.
.RE
.PP
\-S, \-\-with\-summary
.RS 4
Show all scheduling events followed by a summary by thread with min, max, and average run times (in sec) and relative stddev\&.
.RE
.PP
\-\-symfs=
.RS 4
Look for files with symbols relative to this directory\&.
.RE
.PP
\-V, \-\-cpu\-visual
.RS 4
Show visual aid for sched switches by CPU:
\fIi\fR
marks idle time,
\fIs\fR
are scheduler events\&.
.RE
.PP
\-w, \-\-wakeups
.RS 4
Show wakeup events\&.
.RE
.PP
\-M, \-\-migrations
.RS 4
Show migration events\&.
.RE
.PP
\-n, \-\-next
.RS 4
Show next task\&.
.RE
.PP
\-I, \-\-idle\-hist
.RS 4
Show idle\-related events only\&.
.RE
.PP
\-\-time
.RS 4
Only analyze samples within given time window: ,\&. Times have the format seconds\&.microseconds\&. If start is not given (i\&.e\&., time string is
\fI,x\&.y\fR) then analysis starts at the beginning of the file\&. If stop time is not given (i\&.e, time string is
\fIx\&.y,\fR) then analysis goes to end of file\&.
.RE
.PP
\-\-state
.RS 4
Show task state when it switched out\&.
.RE
.PP
\-\-show\-prio
.RS 4
Show task priority\&.
.RE
.PP
\-\-prio
.RS 4
Only show events for given task priority(ies)\&. Multiple priorities can be provided as a comma\-separated list with no spaces: 0,120\&. Ranges of priorities are specified with \-: 120\-129\&. A combination of both can also be provided: 0,120\-129\&.
.RE
.PP
\-P, \-\-pre\-migrations
.RS 4
Show pre\-migration wait time\&. pre\-migration wait time is the time spent by a task waiting on a runqueue but not getting the chance to run there and is migrated to a different runqueue where it is finally run\&. This time between sched_wakeup and migrate_task is the pre\-migration wait time\&.
.RE
.SH "OPTIONS FOR \FIPERF SCHED REPLAY\FR"
.PP
\-r, \-\-repeat
.RS 4
repeat the workload n times (0: infinite)\&. Default is 10\&.
.RE
.SH "SEE ALSO"
.sp
\fBperf-record\fR(1)