cupy.cuda.nccl.groupStart#

cupy.cuda.nccl.groupStart()#

Start a group of NCCL calls. Must be paired with groupEnd().

Note

This method is useful when the NcclCommunicator instances are created via initAll(). A typical usage pattern is like this:

comms = cupy.cuda.nccl.NcclCommunicator.initAll(n, dev_list)
# ... do some preparation work
cupy.cuda.nccl.groupStart()
for rank, comm in enumerate(comms):
    # ... make some collective calls ...
cupy.cuda.nccl.groupEnd()

Other use cases include fusing several NCCL calls into one, and point-to-point communications using send() and recv() (with NCCL 2.7+).