'\" t
.\"     Title: libtracefs
.\"    Author: [see the "AUTHOR" section]
.\" Generator: DocBook XSL Stylesheets vsnapshot <http://docbook.sf.net/>
.\"      Date: 07/25/2024
.\"    Manual: libtracefs Manual
.\"    Source: libtracefs 1.8.1
.\"  Language: English
.\"
.TH "LIBTRACEFS" "3" "07/25/2024" "libtracefs 1\&.8\&.1" "libtracefs Manual"
.\" -----------------------------------------------------------------
.\" * Define some portability stuff
.\" -----------------------------------------------------------------
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.\" http://bugs.debian.org/507673
.\" http://lists.gnu.org/archive/html/groff/2009-02/msg00013.html
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\" -----------------------------------------------------------------
.\" * set default formatting
.\" -----------------------------------------------------------------
.\" disable hyphenation
.nh
.\" disable justification (adjust text to left margin only)
.ad l
.\" -----------------------------------------------------------------
.\" * MAIN CONTENT STARTS HERE *
.\" -----------------------------------------------------------------
.SH "NAME"
tracefs_sql \- Create a synthetic event via an SQL statement
.SH "SYNOPSIS"
.sp
.nf
\fB#include <tracefs\&.h>\fR

struct tracefs_synth *\fBtracefs_sql\fR(struct tep_handle *\fItep\fR, const char *\fIname\fR,
                                        const char *\fIsql_buffer\fR, char **\fIerr\fR);
.fi
.SH "DESCRIPTION"
.sp
Synthetic events are dynamically created events that attach two existing events together via one or more matching fields between the two events\&. It can be used to find the latency between the events, or to simply pass fields of the first event on to the second event to display as one event\&.
.sp
The Linux kernel interface to create synthetic events is complex, and there needs to be a better way to create synthetic events that is easy and can be understood via existing technology\&.
.sp
If you think of each event as a table, where the fields are the column of the table and each instance of the event as a row, you can understand how SQL can be used to attach two events together and form another event (table)\&. Utilizing the SQL \fBSELECT\fR \fBFROM\fR \fBJOIN\fR \fBON\fR [ \fBWHERE\fR ] syntax, a synthetic event can easily be created from two different events\&.
.sp
For simple SQL queries to make a histogram instead of a synthetic event, see HISTOGRAMS below\&.
.sp
\fBtracefs_sql\fR() takes in a \fItep\fR handler (See \fItep_local_events\fR(3)) that is used to verify the events within the \fIsql_buffer\fR expression\&. The \fIname\fR is the name of the synthetic event to create\&. If \fIerr\fR points to an address of a string, it will be filled with a detailed message on any type of parsing error, including fields that do not belong to an event, or if the events or fields are not properly compared\&.
.sp
The example program below is a fully functional parser where it will create a synthetic event from a SQL syntax passed in via the command line or a file\&.
.sp
The SQL format is as follows:
.sp
\fBSELECT\fR <fields> \fBFROM\fR <start\-event> \fBJOIN\fR <end\-event> \fBON\fR <matching\-fields> \fBWHERE\fR <filter>
.sp
Note, although the examples show the SQL commands in uppercase, they are not required to be so\&. That is, you can use "SELECT" or "select" or "sElEct"\&.
.sp
For example:
.sp
.if n \{\
.RS 4
.\}
.nf
SELECT syscalls\&.sys_enter_read\&.fd, syscalls\&.sys_exit_read\&.ret FROM syscalls\&.sys_enter_read
   JOIN syscalls\&.sys_exit_read
   ON syscalls\&.sys_enter_read\&.common_pid = syscalls\&.sys_exit_write\&.common_pid
.fi
.if n \{\
.RE
.\}
.sp
Will create a synthetic event that with the fields:
.sp
.if n \{\
.RS 4
.\}
.nf
u64 fd; s64 ret;
.fi
.if n \{\
.RE
.\}
.sp
Because the function takes a \fItep\fR handle, and usually all event names are unique, you can leave off the system (group) name of the event, and \fBtracefs_sql\fR() will discover the system for you\&.
.sp
That is, the above statement would work with:
.sp
.if n \{\
.RS 4
.\}
.nf
SELECT sys_enter_read\&.fd, sys_exit_read\&.ret FROM sys_enter_read JOIN sys_exit_read
   ON sys_enter_read\&.common_pid = sys_exit_write\&.common_pid
.fi
.if n \{\
.RE
.\}
.sp
The \fBAS\fR keyword can be used to name the fields as well as to give an alias to the events, such that the above can be simplified even more as:
.sp
.if n \{\
.RS 4
.\}
.nf
SELECT start\&.fd, end\&.ret FROM sys_enter_read AS start JOIN sys_exit_read AS end ON start\&.common_pid = end\&.common_pid
.fi
.if n \{\
.RE
.\}
.sp
The above aliases \fIsys_enter_read\fR as \fBstart\fR and \fIsys_exit_read\fR as \fBend\fR and uses those aliases to reference the event throughout the statement\&.
.sp
Using the \fBAS\fR keyword in the selection portion of the SQL statement will define what those fields will be called in the synthetic event\&.
.sp
.if n \{\
.RS 4
.\}
.nf
SELECT start\&.fd AS filed, end\&.ret AS return FROM sys_enter_read AS start JOIN sys_exit_read AS end
   ON start\&.common_pid = end\&.common_pid
.fi
.if n \{\
.RE
.\}
.sp
The above labels the \fIfd\fR of \fIstart\fR as \fBfiled\fR and the \fIret\fR of \fIend\fR as \fBreturn\fR where the synthetic event that is created will now have the fields:
.sp
.if n \{\
.RS 4
.\}
.nf
u64 filed; s64 return;
.fi
.if n \{\
.RE
.\}
.sp
The fields can also be calculated with results passed to the synthetic event:
.sp
.if n \{\
.RS 4
.\}
.nf
select start\&.truesize, end\&.len, (start\&.truesize \- end\&.len) as diff from napi_gro_receive_entry as start
   JOIN netif_receive_skb as end ON start\&.skbaddr = end\&.skbaddr
.fi
.if n \{\
.RE
.\}
.sp
Which would show the \fBtruesize\fR of the \fInapi_gro_receive_entry\fR event, the actual \fIlen\fR of the content, shown by the \fInetif_receive_skb\fR, and the delta between the two and expressed by the field \fBdiff\fR\&.
.sp
The code also supports recording the timestamps at either event, and performing calculations on them\&. For wakeup latency, you have:
.sp
.if n \{\
.RS 4
.\}
.nf
select start\&.pid, (end\&.TIMESTAMP_USECS \- start\&.TIMESTAMP_USECS) as lat from sched_waking as start
   JOIN sched_switch as end ON start\&.pid = end\&.next_pid
.fi
.if n \{\
.RE
.\}
.sp
The above will create a synthetic event that records the \fIpid\fR of the task being woken up, and the time difference between the \fIsched_waking\fR event and the \fIsched_switch\fR event\&. The \fBTIMESTAMP_USECS\fR will truncate the time down to microseconds as the timestamp usually recorded in the tracing buffer has nanosecond resolution\&. If you do not want that truncation, use \fBTIMESTAMP\fR instead of \fBTIMESTAMP_USECS\fR\&.
.sp
Because it is so common to have:
.sp
.if n \{\
.RS 4
.\}
.nf
   (end\&.TIMESTAMP_USECS \- start\&.TIMESTAMP_USECS)
.fi
.if n \{\
.RE
.\}
.sp
The above can be represented with \fBTIMESTAMP_DELTA_USECS\fR or if nanoseconds are OK, you can use \fBTIMESTAMP_DELTA\fR\&. That is, the previous select can also be represented by:
.sp
.if n \{\
.RS 4
.\}
.nf
select start\&.pid, TIMESTAMP_DELTA_USECS as lat from sched_waking as start JOIN sched_switch as end ON start\&.pid = end\&.next_pid
.fi
.if n \{\
.RE
.\}
.sp
Finally, the \fBWHERE\fR clause can be added, that will let you add filters on either or both events\&.
.sp
.if n \{\
.RS 4
.\}
.nf
select start\&.pid, (end\&.TIMESTAMP_USECS \- start\&.TIMESTAMP_USECS) as lat from sched_waking as start
   JOIN sched_switch as end ON start\&.pid = end\&.next_pid
   WHERE start\&.prio < 100 && (!(end\&.prev_pid < 1 || end\&.prev_prio > 100) || end\&.prev_pid == 0)
.fi
.if n \{\
.RE
.\}
.sp
\fBNOTE\fR
.sp
Although both events can be used together in the \fBWHERE\fR clause, they must not be mixed outside the top most "&&" statements\&. You can not OR (||) the events together, where a filter of one event is OR\(cqd to a filter of the other event\&. This does not make sense, as the synthetic event requires both events to take place to be recorded\&. If one is filtered out, then the synthetic event does not execute\&.
.sp
.if n \{\
.RS 4
.\}
.nf
select start\&.pid, (end\&.TIMESTAMP_USECS \- start\&.TIMESTAMP_USECS) as lat from sched_waking as start
   JOIN sched_switch as end ON start\&.pid = end\&.next_pid
   WHERE start\&.prio < 100 && end\&.prev_prio < 100
.fi
.if n \{\
.RE
.\}
.sp
The above is valid\&.
.sp
Where as the below is not\&.
.sp
.if n \{\
.RS 4
.\}
.nf
select start\&.pid, (end\&.TIMESTAMP_USECS \- start\&.TIMESTAMP_USECS) as lat from sched_waking as start
   JOIN sched_switch as end ON start\&.pid = end\&.next_pid
   WHERE start\&.prio < 100 || end\&.prev_prio < 100
.fi
.if n \{\
.RE
.\}
.sp
If the kernel supports it, you can pass around a stacktrace between events\&.
.sp
.if n \{\
.RS 4
.\}
.nf
select start\&.prev_pid as pid, (end\&.TIMESTAMP_USECS \- start\&.TIMESTAMP_USECS) as delta, start\&.STACKTRACE as stack
   FROM sched_switch as start JOIN sched_switch as end ON start\&.prev_pid = end\&.next_pid
   WHERE start\&.prev_state == 2
.fi
.if n \{\
.RE
.\}
.sp
The above will record a stacktrace when a task is in the UNINTERRUPTIBLE (blocked) state, and trigger the synthetic event when it is scheduled back in, recording the time delta that it was blocked for\&. It will record the stacktrace of where it was when it scheduled out along with the delta\&.
.SH "KEYWORDS AS EVENT FIELDS"
.sp
In some cases, an event may have a keyword\&. For example, regcache_drop_region has "from" as a field and the following will not work
.sp
.if n \{\
.RS 4
.\}
.nf
  select from from regcache_drop_region
.fi
.if n \{\
.RE
.\}
.sp
In such cases, add a backslash to the conflicting field, and this will tell the parser that the "from" is a field and not a keyword:
.sp
.if n \{\
.RS 4
.\}
.nf
  select \efrom from regcache_drop_region
.fi
.if n \{\
.RE
.\}
.SH "HISTOGRAMS"
.sp
Simple SQL statements without the \fBJOIN\fR \fBON\fR may also be used, which will create a histogram instead\&. When doing this, the struct tracefs_hist descriptor can be retrieved from the returned synthetic event descriptor via the \fBtracefs_synth_get_start_hist\fR(3)\&.
.sp
In order to utilize the histogram types (see xxx) the CAST command of SQL can be used\&.
.sp
That is:
.sp
.if n \{\
.RS 4
.\}
.nf
  select CAST(common_pid AS comm), CAST(id AS syscall) FROM sys_enter
.fi
.if n \{\
.RE
.\}
.sp
Which produces:
.sp
.if n \{\
.RS 4
.\}
.nf
 # echo \*(Aqhist:keys=common_pid\&.execname,id\&.syscall\*(Aq > events/raw_syscalls/sys_enter/trigger

 # cat events/raw_syscalls/sys_enter/hist

{ common_pid: bash            [     18248], id: sys_setpgid                   [109] } hitcount:          1
{ common_pid: sendmail        [      1812], id: sys_read                      [  0] } hitcount:          1
{ common_pid: bash            [     18247], id: sys_getpid                    [ 39] } hitcount:          1
{ common_pid: bash            [     18247], id: sys_dup2                      [ 33] } hitcount:          1
{ common_pid: gmain           [     13684], id: sys_inotify_add_watch         [254] } hitcount:          1
{ common_pid: cat             [     18247], id: sys_access                    [ 21] } hitcount:          1
{ common_pid: bash            [     18248], id: sys_getpid                    [ 39] } hitcount:          1
{ common_pid: cat             [     18247], id: sys_fadvise64                 [221] } hitcount:          1
{ common_pid: sendmail        [      1812], id: sys_openat                    [257] } hitcount:          1
{ common_pid: less            [     18248], id: sys_munmap                    [ 11] } hitcount:          1
{ common_pid: sendmail        [      1812], id: sys_close                     [  3] } hitcount:          1
{ common_pid: gmain           [      1534], id: sys_poll                      [  7] } hitcount:          1
{ common_pid: bash            [     18247], id: sys_execve                    [ 59] } hitcount:          1
.fi
.if n \{\
.RE
.\}
.sp
Note, string fields may not be cast\&.
.sp
The possible types to cast to are:
.sp
\fBHEX\fR \- convert the value to use hex and not decimal
.sp
\fBSYM\fR \- convert a pointer to symbolic (kallsyms values)
.sp
\fBSYM\-OFFSET\fR \- convert a pointer to symbolic and include the offset\&.
.sp
\fBSYSCALL\fR \- convert the number to the mapped system call name
.sp
\fBEXECNAME\fR or \fBCOMM\fR \- can only be used with the common_pid field\&. Will show the task name of the process\&.
.sp
\fBLOG\fR or \fBLOG2\fR \- bucket the key values in a log 2 values (1, 2, 3\-4, 5\-8, 9\-16, 17\-32, \&...)
.sp
The above fields are not case sensitive, and "LOG2" works as good as "log"\&.
.sp
A special CAST to \fICOUNTER\fR or \fICOUNTER\fR will make the field a value and not a key\&. For example:
.sp
.if n \{\
.RS 4
.\}
.nf
  SELECT common_pid, CAST(bytes_req AS _COUNTER_) FROM kmalloc
.fi
.if n \{\
.RE
.\}
.sp
Which will create
.sp
.if n \{\
.RS 4
.\}
.nf
  echo \*(Aqhist:keys=common_pid:vals=bytes_req\*(Aq > events/kmem/kmalloc/trigger

  cat events/kmem/kmalloc/hist

{ common_pid:       1812 } hitcount:          1  bytes_req:         32
{ common_pid:       9111 } hitcount:          2  bytes_req:        272
{ common_pid:       1768 } hitcount:          3  bytes_req:       1112
{ common_pid:          0 } hitcount:          4  bytes_req:        512
{ common_pid:      18297 } hitcount:         11  bytes_req:       2004
.fi
.if n \{\
.RE
.\}
.SH "RETURN VALUE"
.sp
Returns 0 on success and \-1 on failure\&. On failure, if \fIerr\fR is defined, it will be allocated to hold a detailed description of what went wrong if it the error was caused by a parsing error, or that an event, field does not exist or is not compatible with what it was combined with\&.
.SH "CREATE A TOOL"
.sp
The below example is a functional program that can be used to parse SQL commands into synthetic events\&.
.sp
.if n \{\
.RS 4
.\}
.nf
   man tracefs_sql | sed \-ne \*(Aq/^EXAMPLE/,/FILES/ { /EXAMPLE/d ; /FILES/d ; p}\*(Aq > sqlhist\&.c
   gcc \-o sqlhist sqlhist\&.c `pkg\-config \-\-cflags \-\-libs libtracefs`
.fi
.if n \{\
.RE
.\}
.sp
Then you can run the above examples:
.sp
.if n \{\
.RS 4
.\}
.nf
  sudo \&./sqlhist \*(Aqselect start\&.pid, (end\&.TIMESTAMP_USECS \- start\&.TIMESTAMP_USECS) as lat from sched_waking as start
                  JOIN sched_switch as end ON start\&.pid = end\&.next_pid
                  WHERE start\&.prio < 100 || end\&.prev_prio < 100\*(Aq
.fi
.if n \{\
.RE
.\}
.SH "EXAMPLE"
.sp
.if n \{\
.RS 4
.\}
.nf
#include <stdio\&.h>
#include <stdlib\&.h>
#include <stdarg\&.h>
#include <string\&.h>
#include <errno\&.h>
#include <unistd\&.h>
#include <tracefs\&.h>

static void usage(char **argv)
{
        fprintf(stderr, "usage: %s [\-ed][\-n name][\-s][\-S fields][\-m var][\-c var][\-T][\-t dir][\-f file | sql\-command\-line]\en"
                "  \-n name \- name of synthetic event \*(AqAnonymous\*(Aq if left off\en"
                "  \-t dir \- use dir instead of /sys/kernel/tracing\en"
                "  \-e \- execute the commands to create the synthetic event\en"
                "  \-m \- trigger the action when var is a new max\&.\en"
                "  \-c \- trigger the action when var changes\&.\en"
                "  \-s \- used with \-m or \-c to do a snapshot of the tracing buffer\en"
                "  \-S \- used with \-m or \-c to save fields of the end event (comma deliminated)\en"
                "  \-T \- used with \-m or \-c to do both a snapshot and a trace\en"
                "  \-f file \- read sql lines from file otherwise from the command line\en"
                "            if file is \*(Aq\-\*(Aq then read from standard input\&.\en",
                argv[0]);
        exit(\-1);
}

enum action {
        ACTION_DEFAULT          = 0,
        ACTION_SNAPSHOT         = (1 << 0),
        ACTION_TRACE            = (1 << 1),
        ACTION_SAVE             = (1 << 2),
        ACTION_MAX              = (1 << 3),
        ACTION_CHANGE           = (1 << 4),
};

#define ACTIONS ((ACTION_MAX \- 1))

static int do_sql(const char *instance_name,
                  const char *buffer, const char *name, const char *var,
                  const char *trace_dir, bool execute, int action,
                  char **save_fields)
{
        struct tracefs_synth *synth;
        struct tep_handle *tep;
        struct trace_seq seq;
        enum tracefs_synth_handler handler;
        char *err;
        int ret;

        if ((action & ACTIONS) && !var) {
                fprintf(stderr, "Error: \-s, \-S and \-T not supported without \-m or \-c");
                exit(\-1);
        }

        if (!name)
                name = "Anonymous";

        trace_seq_init(&seq);
        tep = tracefs_local_events(trace_dir);
        if (!tep) {
                if (!trace_dir)
                        trace_dir = "tracefs directory";
                perror(trace_dir);
                exit(\-1);
        }

        synth = tracefs_sql(tep, name, buffer, &err);
        if (!synth) {
                perror("Failed creating synthetic event!");
                if (err)
                        fprintf(stderr, "%s", err);
                free(err);
                exit(\-1);
        }

        if (tracefs_synth_complete(synth)) {
                if (var) {
                        if (action & ACTION_MAX)
                                handler = TRACEFS_SYNTH_HANDLE_MAX;
                        else
                                handler = TRACEFS_SYNTH_HANDLE_CHANGE;

                        if (action & ACTION_SAVE) {
                                ret = tracefs_synth_save(synth, handler, var, save_fields);
                                if (ret < 0) {
                                        err = "adding save";
                                        goto failed_action;
                                }
                        }
                        if (action & ACTION_TRACE) {
                                /*
                                 * By doing the trace before snapshot, it will be included
                                 * in the snapshot\&.
                                 */
                                ret = tracefs_synth_trace(synth, handler, var);
                                if (ret < 0) {
                                        err = "adding trace";
                                        goto failed_action;
                                }
                        }
                        if (action & ACTION_SNAPSHOT) {
                                ret = tracefs_synth_snapshot(synth, handler, var);
                                if (ret < 0) {
                                        err = "adding snapshot";
 failed_action:
                                        perror(err);
                                        if (errno == ENODEV)
                                                fprintf(stderr, "ERROR: \*(Aq%s\*(Aq is not a variable\en",
                                                        var);
                                        exit(\-1);
                                }
                        }
                }
                tracefs_synth_echo_cmd(&seq, synth);
                if (execute) {
                        ret = tracefs_synth_create(synth);
                        if (ret < 0) {
                                fprintf(stderr, "%s\en", tracefs_error_last(NULL));
                                exit(\-1);
                        }
                }
        } else {
                struct tracefs_instance *instance = NULL;
                struct tracefs_hist *hist;

                hist = tracefs_synth_get_start_hist(synth);
                if (!hist) {
                        perror("get_start_hist");
                        exit(\-1);
                }
                if (instance_name) {
                        if (execute)
                                instance = tracefs_instance_create(instance_name);
                        else
                                instance = tracefs_instance_alloc(trace_dir,
                                                                  instance_name);
                        if (!instance) {
                                perror("Failed to create instance");
                                exit(\-1);
                        }
                }
                tracefs_hist_echo_cmd(&seq, instance, hist, 0);
                if (execute) {
                        ret = tracefs_hist_start(instance, hist);
                        if (ret < 0) {
                                fprintf(stderr, "%s\en", tracefs_error_last(instance));
                                exit(\-1);
                        }
                }
        }

        tracefs_synth_free(synth);

        trace_seq_do_printf(&seq);
        trace_seq_destroy(&seq);
        return 0;
}

int main (int argc, char **argv)
{
        char *trace_dir = NULL;
        char *buffer = NULL;
        char buf[BUFSIZ];
        int buffer_size = 0;
        const char *file = NULL;
        const char *instance = NULL;
        bool execute = false;
        char **save_fields = NULL;
        const char *name;
        const char *var;
        int action = 0;
        char *tok;
        FILE *fp;
        size_t r;
        int c;
        int i;

        for (;;) {
                c = getopt(argc, argv, "ht:f:en:m:c:sS:TB:");
                if (c == \-1)
                        break;

                switch(c) {
                case \*(Aqh\*(Aq:
                        usage(argv);
                case \*(Aqt\*(Aq:
                        trace_dir = optarg;
                        break;
                case \*(Aqf\*(Aq:
                        file = optarg;
                        break;
                case \*(Aqe\*(Aq:
                        execute = true;
                        break;
                case \*(Aqm\*(Aq:
                        action |= ACTION_MAX;
                        var = optarg;
                        break;
                case \*(Aqc\*(Aq:
                        action |= ACTION_CHANGE;
                        var = optarg;
                        break;
                case \*(Aqs\*(Aq:
                        action |= ACTION_SNAPSHOT;
                        break;
                case \*(AqS\*(Aq:
                        action |= ACTION_SAVE;
                        tok = strtok(optarg, ",");
                        while (tok) {
                                save_fields = tracefs_list_add(save_fields, tok);
                                tok = strtok(NULL, ",");
                        }
                        if (!save_fields) {
                                perror(optarg);
                                exit(\-1);
                        }
                        break;
                case \*(AqT\*(Aq:
                        action |= ACTION_TRACE | ACTION_SNAPSHOT;
                        break;
                case \*(AqB\*(Aq:
                        instance = optarg;
                        break;
                case \*(Aqn\*(Aq:
                        name = optarg;
                        break;
                }
        }

        if ((action & (ACTION_MAX|ACTION_CHANGE)) == (ACTION_MAX|ACTION_CHANGE)) {
                fprintf(stderr, "Can not use both \-m and \-c together\en");
                exit(\-1);
        }
        if (file) {
                if (!strcmp(file, "\-"))
                        fp = stdin;
                else
                        fp = fopen(file, "r");
                if (!fp) {
                        perror(file);
                        exit(\-1);
                }
                while ((r = fread(buf, 1, BUFSIZ, fp)) > 0) {
                        buffer = realloc(buffer, buffer_size + r + 1);
                        strncpy(buffer + buffer_size, buf, r);
                        buffer_size += r;
                }
                fclose(fp);
                if (buffer_size)
                        buffer[buffer_size] = \*(Aq\e0\*(Aq;
        } else if (argc == optind) {
                usage(argv);
        } else {
                for (i = optind; i < argc; i++) {
                        r = strlen(argv[i]);
                        buffer = realloc(buffer, buffer_size + r + 2);
                        if (i != optind)
                                buffer[buffer_size++] = \*(Aq \*(Aq;
                        strcpy(buffer + buffer_size, argv[i]);
                        buffer_size += r;
                }
        }

        do_sql(instance, buffer, name, var, trace_dir, execute, action, save_fields);
        free(buffer);

        return 0;
}
.fi
.if n \{\
.RE
.\}
.SH "FILES"
.sp
.if n \{\
.RS 4
.\}
.nf
\fBtracefs\&.h\fR
        Header file to include in order to have access to the library APIs\&.
\fB\-ltracefs\fR
        Linker switch to add when building a program that uses the library\&.
.fi
.if n \{\
.RE
.\}
.SH "SEE ALSO"
.sp
\fBsqlhist\fR(1), \fBlibtracefs\fR(3), \fBlibtraceevent\fR(3), \fBtrace\-cmd\fR(1), \fBtracefs_synth_init\fR(3), \fBtracefs_synth_add_match_field\fR(3), \fBtracefs_synth_add_compare_field\fR(3), \fBtracefs_synth_add_start_field\fR(3), \fBtracefs_synth_add_end_field\fR(3), \fBtracefs_synth_append_start_filter\fR(3), \fBtracefs_synth_append_end_filter\fR(3), \fBtracefs_synth_create\fR(3), \fBtracefs_synth_destroy\fR(3), \fBtracefs_synth_free\fR(3), \fBtracefs_synth_echo_cmd\fR(3), \fBtracefs_hist_alloc\fR(3), \fBtracefs_hist_alloc_2d\fR(3), \fBtracefs_hist_alloc_nd\fR(3), \fBtracefs_hist_free\fR(3), \fBtracefs_hist_add_key\fR(3), \fBtracefs_hist_add_value\fR(3), \fBtracefs_hist_add_name\fR(3), \fBtracefs_hist_start\fR(3), \fBtracefs_hist_destory\fR(3), \fBtracefs_hist_add_sort_key\fR(3), \fBtracefs_hist_sort_key_direction\fR(3)
.SH "AUTHOR"
.sp
.if n \{\
.RS 4
.\}
.nf
\fBSteven Rostedt\fR <\m[blue]\fBrostedt@goodmis\&.org\fR\m[]\&\s-2\u[1]\d\s+2>
\fBTzvetomir Stoyanov\fR <\m[blue]\fBtz\&.stoyanov@gmail\&.com\fR\m[]\&\s-2\u[2]\d\s+2>
\fBsameeruddin shaik\fR <\m[blue]\fBsameeruddin\&.shaik8@gmail\&.com\fR\m[]\&\s-2\u[3]\d\s+2>
.fi
.if n \{\
.RE
.\}
.SH "REPORTING BUGS"
.sp
Report bugs to <\m[blue]\fBlinux\-trace\-devel@vger\&.kernel\&.org\fR\m[]\&\s-2\u[4]\d\s+2>
.SH "LICENSE"
.sp
libtracefs is Free Software licensed under the GNU LGPL 2\&.1
.SH "RESOURCES"
.sp
\m[blue]\fBhttps://git\&.kernel\&.org/pub/scm/libs/libtrace/libtracefs\&.git/\fR\m[]
.SH "COPYING"
.sp
Copyright (C) 2020 VMware, Inc\&. Free use of this software is granted under the terms of the GNU Public License (GPL)\&.
.SH "NOTES"
.IP " 1." 4
rostedt@goodmis.org
.RS 4
\%mailto:rostedt@goodmis.org
.RE
.IP " 2." 4
tz.stoyanov@gmail.com
.RS 4
\%mailto:tz.stoyanov@gmail.com
.RE
.IP " 3." 4
sameeruddin.shaik8@gmail.com
.RS 4
\%mailto:sameeruddin.shaik8@gmail.com
.RE
.IP " 4." 4
linux-trace-devel@vger.kernel.org
.RS 4
\%mailto:linux-trace-devel@vger.kernel.org
.RE