cupy.cuda.MemoryHook#

class cupy.cuda.MemoryHook[source]#

Base class of hooks for Memory allocations.

MemoryHook is an callback object. Registered memory hooks are invoked before and after memory is allocated from GPU device, and memory is retrieved from memory pool, and memory is released to memory pool.

Memory hooks that derive MemoryHook are required to implement six methods: alloc_preprocess(), alloc_postprocess(), malloc_preprocess(), malloc_postprocess(), free_preprocess(), and free_postprocess(), By default, these methods do nothing.

Specifically, alloc_preprocess() (resp. alloc_postprocess()) of all memory hooks registered are called before (resp. after) memory is allocated from GPU device.

Likewise, malloc_preprocess() (resp. malloc_postprocess()) of all memory hooks registered are called before (resp. after) memory is retrieved from memory pool.

Below is a pseudo code to descirbe how malloc and hooks work. Please note that alloc_preprocess() and alloc_postprocess() are not invoked if a cached free chunk is found:

def malloc(size):
    Call malloc_preprocess of all memory hooks
    Try to find a cached free chunk from memory pool
    if chunk is not found:
        Call alloc_preprocess for all memory hooks
        Invoke actual memory allocation to get a new chunk
        Call alloc_postprocess for all memory hooks
    Call malloc_postprocess for all memory hooks

Moreover, free_preprocess() (resp. free_postprocess()) of all memory hooks registered are called before (resp. after) memory is released to memory pool.

Below is a pseudo code to descirbe how free and hooks work:

def free(ptr):
    Call free_preprocess of all memory hooks
    Push a memory chunk of a given pointer back to memory pool
    Call free_postprocess for all memory hooks

To register a memory hook, use with statement. Memory hooks are registered to all method calls within with statement and are unregistered at the end of with statement.

Note

CuPy stores the dictionary of registered function hooks as a thread local object. So, memory hooks registered can be different depending on threads.

Methods

__enter__(self)#
__exit__(self, *_)#
alloc_postprocess(self, **kwargs)#

Callback function invoked after allocating memory from GPU device.

Keyword Arguments:

  • device_id (int) – CUDA device ID

  • mem_size (int) – Rounded memory bytesize allocated

  • mem_ptr (int) – Obtained memory pointer. 0 if an error occurred in allocation.

alloc_preprocess(self, **kwargs)#

Callback function invoked before allocating memory from GPU device.

Keyword Arguments:

  • device_id (int) – CUDA device ID

  • mem_size (int) – Rounded memory bytesize to be allocated

free_postprocess(self, **kwargs)#

Callback function invoked after releasing memory to memory pool.

Keyword Arguments:

  • device_id (int) – CUDA device ID

  • mem_size (int) – Memory bytesize

  • mem_ptr (int) – Memory pointer to free

  • pmem_id (int) – Pooled memory object ID.

free_preprocess(self, **kwargs)#

Callback function invoked before releasing memory to memory pool.

Keyword Arguments:

  • device_id (int) – CUDA device ID

  • mem_size (int) – Memory bytesize

  • mem_ptr (int) – Memory pointer to free

  • pmem_id (int) – Pooled memory object ID.

malloc_postprocess(self, **kwargs)#

Callback function invoked after retrieving memory from memory pool.

Keyword Arguments:

  • device_id (int) – CUDA device ID

  • size (int) – Requested memory bytesize to allocate

  • mem_size (int) – Rounded memory bytesize allocated

  • mem_ptr (int) – Obtained memory pointer. 0 if an error occurred in malloc.

  • pmem_id (int) – Pooled memory object ID. 0 if an error occurred in malloc.

malloc_preprocess(self, **kwargs)#

Callback function invoked before retrieving memory from memory pool.

Keyword Arguments:

  • device_id (int) – CUDA device ID

  • size (int) – Requested memory bytesize to allocate

  • mem_size (int) – Rounded memory bytesize to be allocated

__eq__(value, /)#

Return self==value.

__ne__(value, /)#

Return self!=value.

__lt__(value, /)#

Return self<value.

__le__(value, /)#

Return self<=value.

__gt__(value, /)#

Return self>value.

__ge__(value, /)#

Return self>=value.

Attributes

name = 'MemoryHook'#