Start a group of NCCL calls. Must be paired with
This method is useful when the
NcclCommunicatorinstances are created via
initAll(). A typical usage pattern is like this:
comms = cupy.cuda.nccl.NcclCommunicator.initAll(n, dev_list) # ... do some preparation work cupy.cuda.nccl.groupStart() for rank, comm in enumerate(comms): # ... make some collective calls ... cupy.cuda.nccl.groupEnd()
Other use cases include fusing several NCCL calls into one, and point-to-point communications using
recv()(with NCCL 2.7+).