An RDMA buffer represents a piece of pinned memory which can be used to transfer data over RDMA. More...
#include <irdma_context.h>
Public Member Functions | |
| virtual Uint64 | get_id () const =0 |
| Returns the ID of this buffer. More... | |
| virtual Sint32 | get_gpu_id () const =0 |
| Indicates whether the RDMA buffer is in main memory or on a GPU. More... | |
| virtual const Uint8 * | get_data () const =0 |
| Returns a const pointer to the data in this buffer. More... | |
| virtual Uint8 * | get_data ()=0 |
| Returns a mutable pointer to the data in this buffer. More... | |
| virtual Size | get_size () const =0 |
| Returns the size of the data in this buffer buffer. More... | |
| virtual const IRDMA_buffer * | duplicate (Size offset, Size size) const =0 |
| Duplicates an RDMA buffer (const). More... | |
| virtual IRDMA_buffer * | duplicate (Size offset, Size size)=0 |
| Duplicates an RDMA buffer (mutable). More... | |
| virtual bool | is_gpudirect ()=0 |
Returns true if this is a GPU puffer and GPUDirect registration was successful. More... | |
 Public Member Functions inherited from mi::base::IInterface | |
| virtual Uint32 | retain () const =0 |
| Increments the reference count. More... | |
| virtual Uint32 | release () const =0 |
| Decrements the reference count. More... | |
| virtual const IInterface * | get_interface (const Uuid &interface_id) const =0 |
| Acquires a const interface from another. More... | |
| template<class T> | |
| const T * | get_interface () const |
| Acquires a const interface from another. More... | |
| virtual IInterface * | get_interface (const Uuid &interface_id)=0 |
| Acquires a mutable interface from another. More... | |
| template<class T> | |
| T * | get_interface () |
| Acquires a mutable interface from another. More... | |
| virtual Uuid | get_iid () const =0 |
| Returns the interface ID of the most derived interface. More... | |
Additional Inherited Members | |
| Public Types inherited from mi::base::Interface_declare< 0x9b7b00e6, ... > | |
| typedef Interface_declare< id1, ... > | Self |
| Own type. More... | |
| typedef Uuid_t< id1, ... > | IID |
| Declares the interface ID (IID) of this interface. More... | |
| Public Types inherited from mi::base::IInterface | |
| typedef Uuid_t<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0> | IID |
| Declares the interface ID (IID) of this interface. More... | |
| Static Public Member Functions inherited from mi::base::Interface_declare< 0x9b7b00e6, ... > | |
| static bool | compare_iid (const Uuid &iid) |
Compares the interface ID iid against the interface ID of this interface and of its ancestors. More... | |
| Static Public Member Functions inherited from mi::base::IInterface | |
| static bool | compare_iid (const Uuid &iid) |
Compares the interface ID iid against the interface ID of this interface. More... | |
An RDMA buffer represents a piece of pinned memory which can be used to transfer data over RDMA.
On Linux, an RDMA buffer can be allocated on CPU (i.e., host memory) or on GPU (i.e., device memory). When an RDMA buffer is allocated on GPU, the device ID must be specified (because a GPU can have multiple devices). When GPUDirect RDMA is installed, a GPU-based RDMA buffer can be used to transmit or receive data directly over RDMA, i.e., the CPU is not involved and the transferred data does not touch the CPU memory. This feature (GPUDirect RDMA) is currently available only on Linux. Further, the following requirements must be satisfied.
Hardware requirements for GPUDirect RDMA:
Software requirements for GPUDirect RDMA:
To check whether GPUDirect RDMA is running on your Linux system, execute "service nv_peer_mem status" or "lsmod | grep nv_peer_mem" (this depends on the flavor of your Linux system).
In case, GPUDirect RDMA is not available, DiCE offers a fallback solution by allocating an RDMA buffer on CPU and copies the data between CPU-based and GPU-based RDMA buffers in a transparent manner.
|
pure virtual |
Duplicates an RDMA buffer (const).
The new RDMA buffer will still use the memory provided by the original RDMA buffer but will point to a sub part of the original buffer. This is useful if a continuous piece of memory is needed for generating the data, but the result should be sent in different fragmented job results, e.g., to different hosts.
The source RDMA buffer will be retained by this buffer, thus keeping the memory pinned and unusable for other operations as long as the returned buffer exists.
| offset | The offset in the original RDMA buffer at which the data of the returned RDMA buffer starts. |
| size | The size of the data in the returned RDMA buffer. The data in the returned RDMA buffer ends at offset plus size (excluding) in the original RDMA buffer. |
NULL if offset and size result in a memory region not completely in the original buffer.
|
pure virtual |
Duplicates an RDMA buffer (mutable).
The new RDMA buffer will still use the memory provided by the original RDMA buffer but will point to a sub part of the original buffer. This is useful if a continuous piece of memory is needed for generating the data, but the result should be sent in different fragmented job results, e.g., to different hosts.
The source RDMA buffer will be retained by this buffer, thus keeping the memory pinned and unusable for other operations as long as the returned buffer exists.
| offset | The offset in the original RDMA buffer at which the data of the returned RDMA buffer starts. |
| size | The size of the data in the returned RDMA buffer. The data in the returned RDMA buffer ends at offset plus size (excluding) in the original RDMA buffer. |
NULL if offset and size result in a memory region not completely in the original buffer.
|
pure virtual |
Returns a const pointer to the data in this buffer.
|
pure virtual |
Returns a mutable pointer to the data in this buffer.
|
pure virtual |
Indicates whether the RDMA buffer is in main memory or on a GPU.
|
pure virtual |
Returns the ID of this buffer.
To be used to send to the other host.
|
pure virtual |
Returns the size of the data in this buffer buffer.
|
pure virtual |
Returns true if this is a GPU puffer and GPUDirect registration was successful.