cupy.cuda.nccl.NcclCommunicator#
- class cupy.cuda.nccl.NcclCommunicator(int ndev, tuple commId, int rank)#
Initialize an NCCL communicator for one device controlled by one process.
- Parameters:
ndev (int) – Total number of GPUs to be used.
commId (tuple) – The unique ID returned by
get_unique_id()
.rank (int) – The rank of the GPU managed by the current process.
- Returns:
An
NcclCommunicator
instance.- Return type:
Note
This method is for creating an NCCL communicator in a multi-process environment, typically managed by MPI or
multiprocessing
. For controlling multiple devices by one process, useinitAll()
instead.See also
Methods
- abort(self)#
- allGather(self, intptr_t sendbuf, intptr_t recvbuf, size_t count, int datatype, intptr_t stream)#
- allReduce(self, intptr_t sendbuf, intptr_t recvbuf, size_t count, int datatype, int op, intptr_t stream)#
- bcast(self, intptr_t buff, size_t count, int datatype, int root, intptr_t stream)#
- broadcast(self, intptr_t sendbuff, intptr_t recvbuff, size_t count, int datatype, int root, intptr_t stream)#
- check_async_error(self)#
- destroy(self)#
- device_id(self)#
- static initAll(devices)#
Initialize NCCL communicators for multiple devices in a single process.
- Parameters:
devices (int or list of int) – The number of GPUs or a list of GPUs to be used. For the former case, the first
devices
GPUs will be used.- Returns:
A list of
NcclCommunicator
instances.- Return type:
Note
This method is for creating a group of NCCL communicators, each controlling one device, in a single process like this:
from cupy.cuda import nccl # Use 3 GPUs: #0, #2, and #3 comms = nccl.NcclCommunicator.initAll([0, 2, 3]) assert len(comms) == 3
In a multi-process setup, use the default initializer instead.
See also
- rank_id(self)#
- recv(self, intptr_t recvbuf, size_t count, int datatype, int peer, intptr_t stream)#
- reduce(self, intptr_t sendbuf, intptr_t recvbuf, size_t count, int datatype, int op, int root, intptr_t stream)#
- reduceScatter(self, intptr_t sendbuf, intptr_t recvbuf, size_t recvcount, int datatype, int op, intptr_t stream)#
- send(self, intptr_t sendbuf, size_t count, int datatype, int peer, intptr_t stream)#
- size(self)#
- __eq__(value, /)#
Return self==value.
- __ne__(value, /)#
Return self!=value.
- __lt__(value, /)#
Return self<value.
- __le__(value, /)#
Return self<=value.
- __gt__(value, /)#
Return self>value.
- __ge__(value, /)#
Return self>=value.
Attributes
- comm#