NAME¶

ibv_reg_mr, ibv_reg_mr_iova, ibv_reg_dmabuf_mr, ibv_dereg_mr - register or deregister a memory region (MR)

SYNOPSIS¶

#include <infiniband/verbs.h>

struct ibv_mr *ibv_reg_mr(struct ibv_pd *pd, void *addr,
                          size_t length, int access);

struct ibv_mr *ibv_reg_mr_iova(struct ibv_pd *pd, void *addr,
                               size_t length, uint64_t hca_va,
                               int access);

struct ibv_mr *ibv_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset,
                                 size_t length, uint64_t iova,
                                 int fd, int access);

int ibv_dereg_mr(struct ibv_mr *mr);

DESCRIPTION¶

ibv_reg_mr() registers a memory region (MR) associated with the protection domain pd. The MR's starting address is addr and its size is length. The argument access describes the desired memory protection attributes; it is either 0 or the bitwise OR of one or more of the following flags:

IBV_ACCESS_LOCAL_WRITE Enable Local Write Access

IBV_ACCESS_REMOTE_WRITE Enable Remote Write Access

IBV_ACCESS_REMOTE_READ Enable Remote Read Access

IBV_ACCESS_REMOTE_ATOMIC Enable Remote Atomic Operation Access (if supported)

IBV_ACCESS_FLUSH_GLOBAL Enable Remote Flush Operation with global visibility placement type (if supported)

IBV_ACCESS_FLUSH_PERSISTENT Enable Remote Flush Operation with persistence placement type (if supported)

IBV_ACCESS_MW_BIND Enable Memory Window Binding

IBV_ACCESS_ZERO_BASED Use byte offset from beginning of MR to access this MR, instead of a pointer address

IBV_ACCESS_ON_DEMAND Create an on-demand paging MR

IBV_ACCESS_HUGETLB Huge pages are guaranteed to be used for this MR, applicable with IBV_ACCESS_ON_DEMAND in explicit mode only

IBV_ACCESS_RELAXED_ORDERING Allow system to reorder accesses to the MR to improve performance

If IBV_ACCESS_REMOTE_WRITE or IBV_ACCESS_REMOTE_ATOMIC is set, then IBV_ACCESS_LOCAL_WRITE must be set too.

Local read access is always enabled for the MR.

To create an implicit ODP MR, IBV_ACCESS_ON_DEMAND should be set, addr should be 0 and length should be SIZE_MAX.

If IBV_ACCESS_HUGETLB is set, then application awares that for this MR all pages are huge and must promise it will never do anything to break huge pages.

ibv_reg_mr_iova() ibv_reg_mr_iova is the same as the normal reg_mr, except that the user is allowed to specify the virtual base address of the MR when accessed through a lkey or rkey. The offset in the memory region is computed as 'addr + (iova - hca_va)'. Specifying 0 for hca_va has the same effect as IBV_ACCESS_ZERO_BASED.

ibv_reg_dmabuf_mr() registers a dma-buf based memory region (MR) associated with the protection domain pd. The MR starts at offset of the dma-buf and its size is length. The dma-buf is identified by the file descriptor fd. The argument iova specifies the virtual base address of the MR when accessed through a lkey or rkey. It must have the same page offset as offset. The argument access describes the desired memory protection attributes; it is similar to the ibv_reg_mr case except that only the following flags are supported: IBV_ACCESS_LOCAL_WRITE, IBV_ACCESS_REMOTE_WRITE, IBV_ACCESS_REMOTE_READ, IBV_ACCESS_REMOTE_ATOMIC, IBV_ACCESS_RELAXED_ORDERING.

ibv_dereg_mr() deregisters the MR mr.

RETURN VALUE¶

ibv_reg_mr() / ibv_reg_mr_iova() / ibv_reg_dmabuf_mr() returns a pointer to the registered MR, or NULL if the request fails. The local key (L_Key) field lkey is used as the lkey field of struct ibv_sge when posting buffers with ibv_post_* verbs, and the the remote key (R_Key) field rkey is used by remote processes to perform Atomic and RDMA operations. The remote process places this rkey as the rkey field of struct ibv_send_wr passed to the ibv_post_send function.

ibv_dereg_mr() returns 0 on success, or the value of errno on failure (which indicates the failure reason).

NOTES¶

ibv_dereg_mr() fails if any memory window is still bound to this MR.

SEE ALSO¶

ibv_alloc_pd(3), ibv_post_send(3), ibv_post_recv(3), ibv_post_srq_recv(3)

AUTHORS¶

Dotan Barak <dotanba@gmail.com>