Tribblix: manual page: dat_lmr_sync_rdma

DAT_LMR_SYNC_RDMA_WRITE(3DAT) Direct Access Transport Library Functions

NAME

dat_lmr_sync_rdma_write - synchronize local memory with RDMA write on
non-coherent memory

SYNOPSIS

cc [ flag... ] file... -ldat [ library... ]
#include <dat/udat.h>

DAT_RETURN
dat_lmr_sync_rdma_write (
IN DAT_IA_HANDLE ia_handle,
IN const DAT_LMR_TRIPLET *local_segments,
IN DAT_VLEN num_segments
)

PARAMETERS

ia_handle
A handle for an open instance of the IA.

local_segments
An array of buffer segments.

num_segments
The number of segments in the local_segments
argument.

DESCRIPTION

The dat_lmr_sync_rdma_write() function makes effects of an incoming
RDMA Write operation visible to the Consumer. This operation
guarantees consistency by locally invalidating the non-coherent cache
whose buffer has been populated by remote peer RDMA write operations.

The dat_lmr_sync_rdma_write() function is needed if and only if the
Provider attribute specifies that this operation is needed after an
incoming RDMA Write operation. The Consumer must call
dat_lmr_sync_rdma_write() before reading data from a memory range in
this region that was the target of an incoming RDMA Write operation.
The dat_lmr_sync_rdma_write() function must be called after the RDMA
Write operation completes, and the memory range that was modified by
the RDMA Write must be supplied by the caller in the local_segments
array. After this call returns, the Consumer may safely see the
modified contents of the memory range. It is permissible to batch
synchronizations of multiple RDMA Write operations in a single call
by passing a local_segments array that includes all modified memory
ranges. The local_segments entries need not contain the same LMR and
need not be in the same Protection Zone.

The Consumer must also use dat_lmr_sync_rdma_write() when performing
local writes to a memory range that was or will be the target of
incoming RDMA writes. After performing the local write, the Consumer
must call dat_lmr_sync_rdma_write() before the RDMA Write is
initiated. Conversely, after an RDMA Write completes, the Consumer
must call dat_lmr_sync_rdma_write() before performing a local write
to the same range.

If the Provider attribute specifies that this operation is needed and
the Consumer attempts to read from a memory range in an LMR without
properly synchronizing using dat_lmr_sync_rdma_write(), the returned
contents are undefined. If the Consumer attempts to write to a memory
range without properly synchronizing, the contents of the memory
range become undefined.

RETURN VALUES

DAT_SUCCESS
The operation was successful.

DAT_INVALID_HANDLE
The DAT handle is invalid.

DAT_INVALID_PARAMETER
One of the parameters is invalid. For
example, the address range for a local
segment fell outside the boundaries of the
corresponding Local Memory Region or the LMR
handle was invalid.

USAGE

Determining when an RDMA Write completes and determining which memory
range was modified is the Consumer's responsibility. One possibility
is for the RDMA Write initiator to post a Send DTO message after each
RDMA Write that identifies the range in the body of the Send. The
Consumer at the target of the RDMA Write can receive the message and
know when and how to call dat_lmr_sync_rdma_write().

This call ensures that the Provider receives a coherent view of the
buffer contents after a subsequent remote RDMA Write operation. After
the call completes, the Consumer can be assured that all platform-
specific buffer and cache updates have been performed, and that the
LMR range has consistency with the Provider hardware. Any subsequent
read by the Consumer can void this consistency. The Provider is not
required to detect such access.

The action performed on the cache before the RDMA Write depends on
the cache type:

o I/O noncoherent cache will be flushed.

o CPU noncoherent cache will be invalidated.

ATTRIBUTES