Cubed: Scalable out-of-core array processing in Python#
Cubed is a Python library for scalable out-of-core multi-dimensional array processing with bounded memory.
Familiar API
Cubed provides NumPy and Xarray APIs for processing your multi-dimensional array data
Dask replacement
Cubed is a drop-in replacement for Dask’s Array API
Predictable memory usage
Cubed will tell you if your computation would run out of memory before running it
Reliable
Cubed is designed to be robust to failures and will reliably complete a computation
Run locally
Cubed can process hundreds of GB of array data on your laptop using all available cores
Scale in the cloud
Cubed is horizontally scalable and stateless, and can scale to multi-TB datasets in the cloud