cupy.cuda.nccl.groupEnd#
- cupy.cuda.nccl.groupEnd()#
End a group of NCCL calls. Must be paired with
groupStart()
.Note
This method is useful when the
NcclCommunicator
instances are created viainitAll()
. A typical usage pattern is like this:comms = cupy.cuda.nccl.NcclCommunicator.initAll(n, dev_list) # ... do some preparation work cupy.cuda.nccl.groupStart() for rank, comm in enumerate(comms): # ... make some collective calls ... cupy.cuda.nccl.groupEnd()
Other use cases include fusing several NCCL calls into one, and point-to-point communications using
send()
andrecv()
(with NCCL 2.7+).See also