cubed.measure_reserved_mem#
- cubed.measure_reserved_mem(executor, work_dir=None, **kwargs)#
Measures the reserved memory use for a given executor runtime.
This is the memory used by the Python process for running a task, excluding any memory used for data for the computation. It can vary by operating system, Python version, executor runtime, and installed package versions.
It can be used as a guide to set
reserved_mem
when creating aSpec
object.This implementation works by running a trivial computation on a tiny amount of data and measuring the peak memory use.
- Parameters:
executor (cubed.runtime.types.Executor) – The executor to use to run the computation. It must be an executor that reports peak memory, such as Lithops or Modal.
work_dir (str or None, optional) – The directory path (specified as an fsspec URL) used for storing intermediate data. This is required when using a cloud runtime.
kwargs – Keyword arguments to pass to the
compute
function.
- Returns:
The memory used by the runtime in bytes.
- Return type:
int