fi_mrail(7)                    Libfabric v1.21.0                   fi_mrail(7)

NAME
       fi_mrail - The Multi-Rail Utility Provider

OVERVIEW
       The mrail provider (ofi_mrail) is an utility provider that layers over
       an underlying provider to enable the use of multiple network ports
       (rails).  This increases the total available bandwidth of an underlying
       provider.  The current status of mrail provider is experimental - not
       all libfabric features are supported and performance is not guaranteed.

REQUIREMENTS
   Requirements for underlying provider
       mrail provider requires the underlying provider to support the
       following capabilities / modes:

       o Buffered receive (FI_BUFFERED_RECV)

       o FI_SOURCE

       o FI_AV_TABLE

   Requirements for applications
       Applications need to: * Support FI_MR_RAW MR mode bit to make use of
       FI_RMA capability.  * Set FI_OFI_MRAIL_ADDR env variable (see RUNTIME
       PARAMETERS section below).

SUPPORTED FEATURES
       Endpoint types
              The provider supports only FI_EP_RDM.

       Endpoint capabilities
              The following data transfer interface is supported: FI_MSG,
              FI_TAGGED, FI_RMA.

       # LIMITATIONS
              Limitations of the underlying provider may show up as that of
              mrail provider.  mrail provider doesn't allow pass-through of
              any mode bits to the underlying provider.

   Unsupported features
       The following are the major libfabric features that are not supported.
       Any other feature not listed in "Supported features" can be assumed as
       unsupported.

       o FI_ATOMIC

       o Scalable endpoints

       o Shared contexts

       o FABRIC_DIRECT

       o Multicast

       o Triggered operations

FUNCTIONALITY OVERVIEW
       For messages (FI_MSG, FI_TAGGED), the provider uses different policies
       to send messages over one or more rails based on message size (See
       FI_OFI_MRIAL_CONFIG in the RUNTIME PARAMETERS section).  Ordering is
       guaranteed through the use of sequence numbers.

       For RMA, the data is striped equally across all rails.

RUNTIME PARAMETERS
       The ofi_mrail provider checks for the following environment variables.

       FI_OFI_MRAIL_ADDR
              Comma delimited list of individual rail addresses.  Each address
              can be an address in FI_ADDR_STR format, a host name, an IP
              address, or a netdev interface name.

       FI_OFI_MRAIL_ADDR_STRC
              Deprecated.  Replaced by FI_OFI_MRAIL_ADDR.

       FI_OFI_MRAIL_CONFIG
              Comma separated list of <max_size>:<policy> pairs, sorted in
              ascending order of <max_size>.  Each pair indicated the rail
              sharing policy to be used for messages up to the size <max_size>
              and not covered by all previous pairs.  The value of <policy>
              can be fixed (a fixed rail is used), round-robin (one rail per
              message, selected in round-robin fashion), or striping (striping
              across all the rails).  The default configuration is
              16384:fixed,ULONG_MAX:striping.  The value ULONG_MAX can be
              input as -1.

SEE ALSO
       fabric(7), fi_provider(7), fi_getinfo(3)

AUTHORS
       OpenFabrics.

Libfabric Programmer's Manual     2022-12-09                       fi_mrail(7)