crio(8) System Manager's Manual crio(8)

crio - OCI-based implementation of Kubernetes Container Runtime Interface

crio

[--absent-mount-sources-to-reject]=[value]
[--add-inheritable-capabilities]
[--additional-devices]=[value]
[--allowed-devices]=[value]
[--apparmor-profile]=[value]
[--big-files-temporary-dir]=[value]
[--bind-mount-prefix]=[value]
[--blockio-config-file]=[value]
[--blockio-reload]
[--cdi-spec-dirs]=[value]
[--cgroup-manager]=[value]
[--clean-shutdown-file]=[value]
[--cni-config-dir]=[value]
[--cni-default-network]=[value]
[--cni-plugin-dir]=[value]
[--config-dir|-d]=[value]
[--config|-c]=[value]
[--conmon-cgroup]=[value]
[--conmon-env]=[value]
[--conmon]=[value]
[--container-attach-socket-dir]=[value]
[--container-exits-dir]=[value]
[--ctr-stop-timeout]=[value]
[--decryption-keys-path]=[value]
[--default-capabilities]=[value]
[--default-env]=[value]
[--default-mounts-file]=[value]
[--default-runtime]=[value]
[--default-sysctls]=[value]
[--default-transport]=[value]
[--default-ulimits]=[value]
[--device-ownership-from-security-context]
[--disable-hostport-mapping]
[--drop-infra-ctr]
[--enable-criu-support]
[--enable-metrics]
[--enable-nri]
[--enable-pod-events]
[--enable-profile-unix-socket]
[--enable-tracing]
[--gid-mappings]=[value]
[--global-auth-file]=[value]
[--grpc-max-recv-msg-size]=[value]
[--grpc-max-send-msg-size]=[value]
[--help|-h]
[--hooks-dir]=[value]
[--hostnetwork-disable-selinux]
[--image-volumes]=[value]
[--imagestore]=[value]
[--infra-ctr-cpuset]=[value]
[--insecure-registry]=[value]
[--internal-repair]
[--internal-wipe]
[--irqbalance-config-file]=[value]
[--irqbalance-config-restore-file]=[value]
[--listen]=[value]
[--log-dir]=[value]
[--log-filter]=[value]
[--log-format]=[value]
[--log-journald]
[--log-level|-l]=[value]
[--log-size-max]=[value]
[--log]=[value]
[--metrics-cert]=[value]
[--metrics-collectors]=[value]
[--metrics-host]=[value]
[--metrics-key]=[value]
[--metrics-port]=[value]
[--metrics-socket]=[value]
[--minimum-mappable-gid]=[value]
[--minimum-mappable-uid]=[value]
[--namespaces-dir]=[value]
[--no-pivot]
[--nri-disable-connections]=[value]
[--nri-listen]=[value]
[--nri-plugin-config-dir]=[value]
[--nri-plugin-dir]=[value]
[--nri-plugin-registration-timeout]=[value]
[--nri-plugin-request-timeout]=[value]
[--pause-command]=[value]
[--pause-image-auth-file]=[value]
[--pause-image]=[value]
[--pids-limit]=[value]
[--pinned-images]=[value]
[--pinns-path]=[value]
[--profile-cpu]=[value]
[--profile-mem]=[value]
[--profile-port]=[value]
[--profile]
[--rdt-config-file]=[value]
[--read-only]
[--registry]=[value]
[--root|-r]=[value]
[--runroot]=[value]
[--runtimes]=[value]
[--seccomp-profile]=[value]
[--seccomp-use-default-when-empty]
[--selinux]
[--separate-pull-cgroup]=[value]
[--shared-cpuset]=[value]
[--signature-policy-dir]=[value]
[--signature-policy]=[value]
[--stats-collection-period]=[value]
[--storage-driver|-s]=[value]
[--storage-opt]=[value]
[--stream-address]=[value]
[--stream-enable-tls]
[--stream-idle-timeout]=[value]
[--stream-port]=[value]
[--stream-tls-ca]=[value]
[--stream-tls-cert]=[value]
[--stream-tls-key]=[value]
[--tracing-endpoint]=[value]
[--tracing-sampling-rate-per-million]=[value]
[--uid-mappings]=[value]
[--version-file-persist]=[value]
[--version-file]=[value]
[--version|-v]

OCI-based implementation of Kubernetes Container Runtime Interface Daemon

crio is meant to provide an integration path between OCI conformant runtimes and the kubelet. Specifically, it implements the Kubelet Container Runtime Interface (CRI) using OCI conformant runtimes. The scope of crio is tied to the scope of the CRI.

1.
Support multiple image formats including the existing Docker and OCI image formats.
2.
Support for multiple means to download images including trust & image verification.
3.
Container image management (managing image layers, overlay filesystems, etc).
4.
Container process lifecycle management.
5.
Monitoring and logging required to satisfy the CRI.
6.
Resource isolation as required by the CRI.

Usage:

crio [GLOBAL OPTIONS] command [COMMAND OPTIONS] [ARGUMENTS...]

--absent-mount-sources-to-reject="": A list of paths that, when absent from the host, will cause a container creation to fail (as opposed to the current behavior of creating a directory).

--add-inheritable-capabilities: Add capabilities to the inheritable set, as well as the default group of permitted, bounding and effective.

--additional-devices="": Devices to add to the containers.

--allowed-devices="": Devices a user is allowed to specify with the "io.kubernetes.cri-o.Devices" allowed annotation. (default: "/dev/fuse")

--apparmor-profile="": Name of the apparmor profile to be used as the runtime's default. This only takes effect if the user does not specify a profile via the Kubernetes Pod's metadata annotation. (default: "crio-default")

--big-files-temporary-dir="": Path to the temporary directory to use for storing big files, used to store image blobs and data streams related to containers image management.

--bind-mount-prefix="": A prefix to use for the source of the bind mounts. This option would be useful if you were running CRI-O in a container. And had '/' mounted on '/host' in your container. Then if you ran CRI-O with the '--bind-mount-prefix=/host' option, CRI-O would add /host to any bind mounts it is handed over CRI. If Kubernetes asked to have '/var/lib/foobar' bind mounted into the container, then CRI-O would bind mount '/host/var/lib/foobar'. Since CRI-O itself is running in a container with '/' or the host mounted on '/host', the container would end up with '/var/lib/foobar' from the host mounted in the container rather then '/var/lib/foobar' from the CRI-O container.

--blockio-config-file="": Path to the blockio class configuration file for configuring the cgroup blockio controller.

--blockio-reload: Reload blockio-config-file and rescan blockio devices in the system before applying blockio parameters.

--cdi-spec-dirs="": Directories to scan for CDI Spec files. (default: "/etc/cdi", "/run/cdi")

--cgroup-manager="": cgroup manager (cgroupfs or systemd). (default: "systemd")

--clean-shutdown-file="": Location for CRI-O to lay down the clean shutdown file. It indicates whether we've had time to sync changes to disk before shutting down. If not found, crio wipe will clear the storage directory. (default: "/var/lib/crio/clean.shutdown")

--cni-config-dir="": CNI configuration files directory. (default: "/etc/cni/net.d/")

--cni-default-network="": Name of the default CNI network to select. If not set or "", then CRI-O will pick-up the first one found in --cni-config-dir.

--cni-plugin-dir="": CNI plugin binaries directory.

--config, -c="": Path to configuration file (default: "/etc/crio/crio.conf")

--config-dir, -d="": Path to the configuration drop-in directory.
This directory will be recursively iterated and each file gets applied
to the configuration in their processing order. This means that a
configuration file named '00-default' has a lower priority than a file
named '01-my-overwrite'.
The global config file, provided via '--config,-c' or per default in
/etc/crio/crio.conf, always has a lower priority than the files in the directory specified
by '--config-dir,-d'.
Besides that, provided command line parameters have a higher priority
than any configuration file. (default: "/etc/crio/crio.conf.d")

--conmon="": Path to the conmon binary, used for monitoring the OCI runtime. Will be searched for using $PATH if empty. This option is deprecated, and will be removed in the future.

--conmon-cgroup="": cgroup to be used for conmon process. This option is deprecated and will be removed in the future.

--conmon-env="": Environment variable list for the conmon process, used for passing necessary environment variables to conmon or the runtime. This option is deprecated and will be removed in the future.

--container-attach-socket-dir="": Path to directory for container attach sockets. (default: "/run/crio")

--container-exits-dir="": Path to directory in which container exit files are written to by conmon. (default: "/run/crio/exits")

--ctr-stop-timeout="": The minimal amount of time in seconds to wait before issuing a timeout regarding the proper termination of the container. The lowest possible value is 30s, whereas lower values are not considered by CRI-O. (default: 30)

--decryption-keys-path="": Path to load keys for image decryption. (default: "/etc/crio/keys/")

--default-capabilities="": Capabilities to add to the containers. (default: "CHOWN", "DAC_OVERRIDE", "FSETID", "FOWNER", "SETGID", "SETUID", "SETPCAP", "NET_BIND_SERVICE", "KILL")

--default-env="": Additional environment variables to set for all containers.

--default-mounts-file="": Path to default mounts file.

--default-runtime="": Default OCI runtime from the runtimes config. (default: "runc")

--default-sysctls="": Sysctls to add to the containers.

--default-transport="": A prefix to prepend to image names that cannot be pulled as-is. (default: "docker://")

--default-ulimits="": Ulimits to apply to containers by default (name=soft:hard).

--device-ownership-from-security-context: Set devices' uid/gid ownership from runAsUser/runAsGroup.

--disable-hostport-mapping: If true, CRI-O would disable the hostport mapping.

--drop-infra-ctr: Determines whether pods are created without an infra container, when the pod is not using a pod level PID namespace.

--enable-criu-support: Enable CRIU integration, requires that the criu binary is available in $PATH.

--enable-metrics: Enable metrics endpoint for the server on localhost:9090.

--enable-nri: Enable NRI (Node Resource Interface) support. (default: false)

--enable-pod-events: If true, CRI-O starts sending the container events to the kubelet

--enable-profile-unix-socket: Enable pprof profiler on crio unix domain socket.

--enable-tracing: Enable OpenTelemetry trace data exporting.

--gid-mappings="": Specify the GID mappings to use for the user namespace. This option is deprecated, and will be replaced with Kubernetes user namespace (KEP-127) support in the future.

--global-auth-file="": Path to a file like /var/lib/kubelet/config.json holding credentials necessary for pulling images from secure registries.

--grpc-max-recv-msg-size="": Maximum grpc receive message size in bytes. (default: 83886080)

--grpc-max-send-msg-size="": Maximum grpc receive message size. (default: 83886080)

--help, -h: show help

--hooks-dir="": Set the OCI hooks directory path (may be set multiple times)
If one of the directories does not exist, then CRI-O will automatically
skip them.
Each '*.json' file in the path configures a hook for CRI-O
containers. For more details on the syntax of the JSON files and
the semantics of hook injection, see 'oci-hooks(5)'. CRI-O
currently support both the 1.0.0 and 0.1.0 hook schemas, although
the 0.1.0 schema is deprecated.
This option may be set multiple times; paths from later options
have higher precedence ('oci-hooks(5)' discusses directory
precedence).
For the annotation conditions, CRI-O uses the Kubernetes
annotations, which are a subset of the annotations passed to the
OCI runtime. For example, 'io.kubernetes.cri-o.Volumes' is part of
the OCI runtime configuration annotations, but it is not part of
the Kubernetes annotations being matched for hooks.
For the bind-mount conditions, only mounts explicitly requested by
Kubernetes configuration are considered. Bind mounts that CRI-O
inserts by default (e.g. '/dev/shm') are not considered. (default: "/usr/share/containers/oci/hooks.d")

--hostnetwork-disable-selinux: Determines whether SELinux should be disabled within a pod when it is running in the host network namespace.

--image-volumes="": Image volume handling ('mkdir', 'bind', or 'ignore')
1. mkdir: A directory is created inside the container root filesystem for
the volumes.
2. bind: A directory is created inside container state directory and bind
mounted into the container for the volumes. 3. ignore: All volumes are just ignored and no action is taken. (default: "mkdir")

--imagestore="": Store newly pulled images in the specified path, rather than the path provided by --root.

--infra-ctr-cpuset="": CPU set to run infra containers, if not specified CRI-O will use all online CPUs to run infra containers.

--insecure-registry="": Enable insecure registry communication, i.e., enable un-encrypted and/or untrusted communication.
1. List of insecure registries can contain an element with CIDR notation to
specify a whole subnet.
2. Insecure registries accept HTTP or accept HTTPS with certificates from
unknown CAs.
3. Enabling '--insecure-registry' is useful when running a local registry.
However, because its use creates security vulnerabilities, it should ONLY
be enabled for testing purposes. For increased security, users should add
their CA to their system's list of trusted CAs instead of using
'--insecure-registry'.

--internal-repair: If true, CRI-O will check if the container and image storage was corrupted after a sudden restart, and attempt to repair the storage if it was.

--internal-wipe: Whether CRI-O should wipe containers after a reboot and images after an upgrade when the server starts. If set to false, one must run 'crio wipe' to wipe the containers and images in these situations. This option is deprecated, and will be removed in the future.

--irqbalance-config-file="": The irqbalance service config file which is used by CRI-O. (default: "/etc/sysconfig/irqbalance")

--irqbalance-config-restore-file="": Determines if CRI-O should attempt to restore the irqbalance config at startup with the mask in this file. Use the 'disable' value to disable the restore flow entirely. (default: "/etc/sysconfig/orig_irq_banned_cpus")

--listen="": Path to the CRI-O socket. (default: "/run/crio/crio.sock")

--log="": Set the log file path where internal debug information is written.

--log-dir="": Default log directory where all logs will go unless directly specified by the kubelet. (default: "/var/log/crio/pods")

--log-filter="": Filter the log messages by the provided regular expression. For example 'request.*' filters all gRPC requests.

--log-format="": Set the format used by logs: 'text' or 'json'. (default: "text")

--log-journald: Log to systemd journal (journald) in addition to kubernetes log file.

--log-level, -l="": Log messages above specified level: trace, debug, info, warn, error, fatal or panic. (default: "info")

--log-size-max="": Maximum log size in bytes for a container. If it is positive, it must be >= 8192 to match/exceed conmon read buffer. This option is deprecated. The Kubelet flag '--container-log-max-size' should be used instead. (default: -1)

--metrics-cert="": Certificate for the secure metrics endpoint.

--metrics-collectors="": Enabled metrics collectors. (default: "operations", "operations_latency_microseconds_total", "operations_latency_microseconds", "operations_errors", "image_pulls_by_digest", "image_pulls_by_name", "image_pulls_by_name_skipped", "image_pulls_failures", "image_pulls_successes", "image_pulls_layer_size", "image_layer_reuse", "containers_events_dropped_total", "containers_oom_total", "containers_oom", "processes_defunct", "operations_total", "operations_latency_seconds", "operations_latency_seconds_total", "operations_errors_total", "image_pulls_bytes_total", "image_pulls_skipped_bytes_total", "image_pulls_failure_total", "image_pulls_success_total", "image_layer_reuse_total", "containers_oom_count_total", "containers_seccomp_notifier_count_total", "resources_stalled_at_stage")

--metrics-host="": Host for the metrics endpoint. (default: "" for all interfaces)

--metrics-key="": Certificate key for the secure metrics endpoint.

--metrics-port="": Port for the metrics endpoint. (default: 9090)

--metrics-socket="": Socket for the metrics endpoint.

--minimum-mappable-gid="": Specify the lowest host GID which can be specified in mappings for a pod that will be run as a UID other than 0. This option is deprecated, and will be replaced with Kubernetes user namespace support (KEP-127) in the future. (default: -1)

--minimum-mappable-uid="": Specify the lowest host UID which can be specified in mappings for a pod that will be run as a UID other than 0. This option is deprecated, and will be replaced with Kubernetes user namespace support (KEP-127) in the future. (default: -1)

--namespaces-dir="": The directory where the state of the managed namespaces gets tracked. Only used when manage-ns-lifecycle is true. (default: "/run")

--no-pivot: If true, the runtime will not use 'pivot_root', but instead use 'MS_MOVE'.

--nri-disable-connections="": Disable connections from externally started NRI plugins. (default: false)

--nri-listen="": Socket to listen on for externally started NRI plugins to connect to. (default: "/run/nri/nri.sock")

--nri-plugin-config-dir="": Directory to scan for configuration of pre-installed NRI plugins. (default: "/etc/nri/conf.d")

--nri-plugin-dir="": Directory to scan for pre-installed NRI plugins to start automatically. (default: "/opt/nri/plugins")

--nri-plugin-registration-timeout="": Timeout for a plugin to register itself with NRI. (default: 5s)

--nri-plugin-request-timeout="": Timeout for a plugin to handle an NRI request. (default: 2s)

--pause-command="": Path to the pause executable in the pause image. (default: "/pause")

--pause-image="": Image which contains the pause executable. (default: "registry.k8s.io/pause:3.9")

--pause-image-auth-file="": Path to a config file containing credentials for --pause-image.

--pids-limit="": Maximum number of processes allowed in a container. This option is deprecated. The Kubelet flag '--pod-pids-limit' should be used instead. (default: -1)

--pinned-images="": A list of images that will be excluded from the kubelet's garbage collection.

--pinns-path="": The path to find the pinns binary, which is needed to manage namespace lifecycle. Will be searched for in $PATH if empty.

--profile: Enable pprof remote profiler on localhost:6060.

--profile-cpu="": Write a pprof CPU profile to the provided path.

--profile-mem="": Write a pprof memory profile to the provided path.

--profile-port="": Port for the pprof profiler. (default: 6060)

--rdt-config-file="": Path to the RDT configuration file for configuring the resctrl pseudo-filesystem.

--read-only: Setup all unprivileged containers to run as read-only. Automatically mounts the containers' tmpfs on '/run', '/tmp' and '/var/tmp'.

--registry="": Registry to be prepended when pulling unqualified images. Can be specified multiple times.

--root, -r="": The CRI-O root directory. (default: "/var/lib/containers/storage")

--runroot="": The CRI-O state directory. (default: "/run/containers/storage")

--runtimes="": OCI runtimes, format is 'runtime_name:runtime_path:runtime_root:runtime_type:privileged_without_host_devices:runtime_config_path'.

--seccomp-profile="": Path to the seccomp.json profile to be used as the runtime's default. If not specified, then the internal default seccomp profile will be used.

--seccomp-use-default-when-empty: Use the default seccomp profile when an empty one is specified. This option is currently deprecated, and will be replaced by the SeccompDefault FeatureGate in Kubernetes.

--selinux: Enable selinux support. This option is deprecated, and be interpreted from whether SELinux is enabled on the host in the future.

--separate-pull-cgroup="": [EXPERIMENTAL] Pull in new cgroup.

--shared-cpuset="": CPUs set that will be used for guaranteed containers that want access to shared cpus

--signature-policy="": Path to signature policy JSON file.

--signature-policy-dir="": Path to the root directory for namespaced signature policies. Must be an absolute path. (default: "/etc/crio/policies")

--stats-collection-period="": The number of seconds between collecting pod and container stats. If set to 0, the stats are collected on-demand instead. (default: 0)

--storage-driver, -s="": OCI storage driver.

--storage-opt="": OCI storage driver option.

--stream-address="": Bind address for streaming socket. (default: "127.0.0.1")

--stream-enable-tls: Enable encrypted TLS transport of the stream server.

--stream-idle-timeout="": Length of time until open streams terminate due to lack of activity.

--stream-port="": Bind port for streaming socket. If the port is set to '0', then CRI-O will allocate a random free port number. (default: "0")

--stream-tls-ca="": Path to the x509 CA(s) file used to verify and authenticate client communication with the encrypted stream. This file can change and CRI-O will automatically pick up the changes within 5 minutes.

--stream-tls-cert="": Path to the x509 certificate file used to serve the encrypted stream. This file can change and CRI-O will automatically pick up the changes within 5 minutes.

--stream-tls-key="": Path to the key file used to serve the encrypted stream. This file can change and CRI-O will automatically pick up the changes within 5 minutes.

--tracing-endpoint="": Address on which the gRPC tracing collector will listen. (default: "0.0.0.0:4317")

--tracing-sampling-rate-per-million="": Number of samples to collect per million OpenTelemetry spans. Set to 1000000 to always sample. (default: 0)

--uid-mappings="": Specify the UID mappings to use for the user namespace. This option is deprecated, and will be replaced with Kubernetes user namespace support (KEP-127) in the future.

--version, -v: print the version

--version-file="": Location for CRI-O to lay down the temporary version file. It is used to check if crio wipe should wipe containers, which should always happen on a node reboot. (default: "/run/crio/version")

--version-file-persist="": Location for CRI-O to lay down the persistent version file. It is used to check if crio wipe should wipe images, which should only happen when CRI-O has been upgraded. (default: "/run/crio/version")

Generate bash, fish or zsh completions.

Generate the man page documentation.

Generate the markdown documentation.

--help, -h: show help

help, h

Shows a list of commands or help for one command

Outputs a commented version of the configuration file that could be used by CRI-O. This allows you to save you current configuration setup and then load it later with --config. Global options will modify the output.

--default: Output the default configuration (without taking into account any configuration options).

--migrate-defaults, -m="": Migrate the default config from a specified version.

The migrate-defaults command has been deprecated and will be removed in the future.
To run a config migration, just select the input config via the global
```
crio -c /etc/crio/crio.conf.d/00-default.conf config -m 1.17
```
The migration will print converted configuration options to stderr and will
output the resulting configuration to stdout.
Please note that the migration will overwrite any fields that have changed
defaults between versions. To save a custom configuration change, it should
be in a drop-in configuration file instead.
Possible values: "1.17" (default: "1.17")

display detailed version information

--json, -j: print JSON instead of text

--verbose, -v: print verbose information (for example all golang dependencies)

wipe CRI-O's container and image storage

--force, -f: force wipe by skipping the version check

Display status information

--socket, -s="": absolute path to the unix socket (default: "/run/crio/crio.sock")

Show the configuration of CRI-O as a TOML string.

Display detailed information about the provided container ID.

--id, -i="": the container ID

Retrieve generic information about CRI-O, such as the cgroup and storage driver.

help, h

Shows a list of commands or help for one command

crio.conf (/etc/crio/crio.conf)
cri-o configuration file for all of the available command-line options for
the crio(8) program, but in a TOML format that can be more easily modified
and versioned.

policy.json (/etc/containers/policy.json)
Signature verification policy files are used to specify policy, e.g. trusted
keys, applicable when deciding whether to accept an image, or individual
signatures of that image, as valid.

registries.conf (/etc/containers/registries.conf)
Registry configuration file specifies registries which are consulted when
completing image names that do not include a registry or domain portion.

storage.conf (/etc/containers/storage.conf)
Storage configuration file specifies all of the available container storage
options for tools using shared container storage.

All command-line options may also be specified as environment variables. The options detailed in this section, however, can only be set via environment variables.

KUBENSMNT: Path to a bind-mounted mount namespace that CRI-O should join before launching any containers. If the path does not exist, or does not point to a mount namespace bindmount, CRI-O will run in its parent's mount namespace and log a warning that the requested namespace was not joined.

crio.conf(5), crio.conf.d(5), oci-hooks(5), policy.json(5), registries.conf(5), storage.conf(5)