BORG-PATTERNS(1)               borg backup tool               BORG-PATTERNS(1)

NAME
       borg-patterns - Details regarding patterns

DESCRIPTION
       The path/filenames used as input for the pattern matching start from
       the currently active recursion root. You usually give the recursion
       root(s) when invoking borg and these can be either relative or absolute
       paths.

       Starting with Borg 1.2, paths that are matched against patterns always
       appear relative. If you give /absolute/ as root, the paths going into
       the matcher will start with absolute/.  If you give ../../relative as
       root, the paths will be normalized as relative/.

       A directory exclusion pattern can end either with or without a slash
       ('/').  If it ends with a slash, such as some/path/, the directory will
       be included but not its content. If it does not end with a slash, such
       as some/path, both the directory and content will be excluded.

       Borg supports different pattern styles. To define a non-default style
       for a specific pattern, prefix it with two characters followed by a
       colon ':' (i.e. fm:path/*, sh:path/**).

       Fnmatch, selector fm:
              This is the default style for --exclude and --exclude-from.
              These patterns use a variant of shell pattern syntax, with '*'
              matching any number of characters, '?' matching any single
              character, '[...]' matching any single character specified,
              including ranges, and '[!...]' matching any character not
              specified. For the purpose of these patterns, the path separator
              (backslash for Windows and '/' on other systems) is not treated
              specially. Wrap meta-characters in brackets for a literal match
              (i.e. [?] to match the literal character ?). For a path to match
              a pattern, the full path must match, or it must match from the
              start of the full path to just before a path separator. Except
              for the root path, paths will never end in the path separator
              when matching is attempted.  Thus, if a given pattern ends in a
              path separator, a '*' is appended before matching is attempted.
              A leading path separator is always removed.

       Shell-style patterns, selector sh:
              This is the default style for --pattern and --patterns-from.
              Like fnmatch patterns these are similar to shell patterns. The
              difference is that the pattern may include **/ for matching zero
              or more directory levels, * for matching zero or more arbitrary
              characters with the exception of any path separator. A leading
              path separator is always removed.

       Regular expressions, selector re:
              Regular expressions similar to those found in Perl are
              supported. Unlike shell patterns regular expressions are not
              required to match the full path and any substring match is
              sufficient. It is strongly recommended to anchor patterns to the
              start ('^'), to the end ('$') or both. Path separators
              (backslash for Windows and '/' on other systems) in paths are
              always normalized to a forward slash ('/') before applying a
              pattern. The regular expression syntax is described in the
              Python documentation for the re module.

       Path prefix, selector pp:
              This pattern style is useful to match whole sub-directories. The
              pattern pp:root/somedir matches root/somedir and everything
              therein. A leading path separator is always removed.

       Path full-match, selector pf:
              This pattern style is (only) useful to match full paths.  This
              is kind of a pseudo pattern as it can not have any variable or
              unspecified parts - the full path must be given.
              pf:root/file.ext matches root/file.ext only. A leading path
              separator is always removed.

              Implementation note: this is implemented via very time-efficient
              O(1) hashtable lookups (this means you can have huge amounts of
              such patterns without impacting performance much).  Due to that,
              this kind of pattern does not respect any context or order.  If
              you use such a pattern to include a file, it will always be
              included (if the directory recursion encounters it).  Other
              include/exclude patterns that would normally match will be
              ignored.  Same logic applies for exclude.

       NOTE:
          re:, sh: and fm: patterns are all implemented on top of the Python
          SRE engine. It is very easy to formulate patterns for each of these
          types which requires an inordinate amount of time to match paths. If
          untrusted users are able to supply patterns, ensure they cannot
          supply re: patterns.  Further, ensure that sh: and fm: patterns only
          contain a handful of wildcards at most.

       Exclusions can be passed via the command line option --exclude. When
       used from within a shell, the patterns should be quoted to protect them
       from expansion.

       The --exclude-from option permits loading exclusion patterns from a
       text file with one pattern per line. Lines empty or starting with the
       number sign ('#') after removing whitespace on both ends are ignored.
       The optional style selector prefix is also supported for patterns
       loaded from a file. Due to whitespace removal, paths with whitespace at
       the beginning or end can only be excluded using regular expressions.

       To test your exclusion patterns without performing an actual backup you
       can run borg create --list --dry-run ....

       Examples:

          # Exclude '/home/user/file.o' but not '/home/user/file.odt':
          $ borg create -e '*.o' backup /

          # Exclude '/home/user/junk' and '/home/user/subdir/junk' but
          # not '/home/user/importantjunk' or '/etc/junk':
          $ borg create -e 'home/*/junk' backup /

          # Exclude the contents of '/home/user/cache' but not the directory itself:
          $ borg create -e home/user/cache/ backup /

          # The file '/home/user/cache/important' is *not* backed up:
          $ borg create -e home/user/cache/ backup / /home/user/cache/important

          # The contents of directories in '/home' are not backed up when their name
          # ends in '.tmp'
          $ borg create --exclude 're:^home/[^/]+\.tmp/' backup /

          # Load exclusions from file
          $ cat >exclude.txt <<EOF
          # Comment line
          home/*/junk
          *.tmp
          fm:aa:something/*
          re:^home/[^/]+\.tmp/
          sh:home/*/.thumbnails
          # Example with spaces, no need to escape as it is processed by borg
          some file with spaces.txt
          EOF
          $ borg create --exclude-from exclude.txt backup /

       A more general and easier to use way to define filename matching
       patterns exists with the --pattern and --patterns-from options. Using
       these, you may specify the backup roots, default pattern styles and
       patterns for inclusion and exclusion.

       Root path prefix R
              A recursion root path starts with the prefix R, followed by a
              path (a plain path, not a file pattern). Use this prefix to have
              the root paths in the patterns file rather than as command line
              arguments.

       Pattern style prefix P
              To change the default pattern style, use the P prefix, followed
              by the pattern style abbreviation (fm, pf, pp, re, sh).  All
              patterns following this line will use this style until another
              style is specified.

       Exclude pattern prefix -
              Use the prefix -, followed by a pattern, to define an exclusion.
              This has the same effect as the --exclude option.

       Exclude no-recurse pattern prefix !
              Use the prefix !, followed by a pattern, to define an exclusion
              that does not recurse into subdirectories. This saves time, but
              prevents include patterns to match any files in subdirectories.

       Include pattern prefix +
              Use the prefix +, followed by a pattern, to define inclusions.
              This is useful to include paths that are covered in an exclude
              pattern and would otherwise not be backed up.

       NOTE:
          Via --pattern or --patterns-from you can define BOTH inclusion and
          exclusion of files using pattern prefixes + and -. With --exclude
          and --exclude-from ONLY excludes are defined.

       The first matching pattern is used, so if an include pattern matches
       before an exclude pattern, the file is backed up. Note that a
       no-recurse exclude stops examination of subdirectories so that
       potential includes will not match - use normal excludes for such use
       cases.

       Example:

          # Define the recursion root
          R /
          # Exclude all iso files in any directory
          - **/*.iso
          # Explicitly include all inside etc and root
          + etc/**
          + root/**
          # Exclude a specific directory under each user's home directories
          - home/*/.cache
          # Explicitly include everything in /home
          + home/**
          # Explicitly exclude some directories without recursing into them
          ! re:^(dev|proc|run|sys|tmp)
          # Exclude all other files and directories
          # that are not specifically included earlier.
          - **

       NOTE:
          It's possible that a sub-directory/file is matched while parent
          directories are not.  In that case, parent directories are not
          backed up thus their user, group, permission, etc. can not be
          restored.

       Note that the default pattern style for --pattern and --patterns-from
       is shell style (sh:), so those patterns behave similar to rsync
       include/exclude patterns. The pattern style can be set via the P
       prefix.

       Patterns (--pattern) and excludes (--exclude) from the command line are
       considered first (in the order of appearance). Then patterns from
       --patterns-from are added. Exclusion patterns from --exclude-from files
       are appended last.

       Examples:

          # backup pics, but not the ones from 2018, except the good ones:
          # note: using = is essential to avoid cmdline argument parsing issues.
          borg create --pattern=+pics/2018/good --pattern=-pics/2018 repo::arch pics

          # use a file with patterns:
          borg create --patterns-from patterns.lst repo::arch

       The patterns.lst file could look like that:

          # "sh:" pattern style is the default, so the following line is not needed:
          P sh
          R /
          # can be rebuild
          - home/*/.cache
          # they're downloads for a reason
          - home/*/Downloads
          # susan is a nice person
          # include susans home
          + home/susan
          # also back up this exact file
          + pf:home/bobby/specialfile.txt
          # don't backup the other home directories
          - home/*
          # don't even look in /proc
          ! proc

       You can specify recursion roots either on the command line or in a
       patternfile:

          # these two commands do the same thing
          borg create --exclude home/bobby/junk repo::arch /home/bobby /home/susan
          borg create --patterns-from patternfile.lst repo::arch

       The patternfile:

          # note that excludes use fm: by default and patternfiles use sh: by default.
          # therefore, we need to specify fm: to have the same exact behavior.
          P fm
          R /home/bobby
          R /home/susan

          - home/bobby/junk

       This allows you to share the same patterns between multiple
       repositories without needing to specify them on the command line.

AUTHOR
       The Borg Collective

                                  2024-03-29                  BORG-PATTERNS(1)