Package: mattress
Python bindings for tatamiOverviewThemattresspackage implements Python bindings to thetatamiC++ library for matrix representations.
Downstream packages can usemattressto develop C++ extensions that are interoperable with many different matrix classes, e.g., dense, sparse, delayed or file-backed.mattressis inspired by thebeachmatBioconductor package, which does the same thing for R packages.Instructionsmattressis published toPyPI, so installation is simple:pipinstallmattressmattressis intended for Python package developers writing C++ extensions that operate on matrices.Addmattress.includes()to theinclude_dirs=of yourExtension()definition insetup.py.
This will give you access to the varioustatamiheaders to compile your C++ code.Add#include "Mattress.h"to your C++ source files.
This defines aMattressclass where theptrmember is a pointer to atatamimatrix.
Python-visible C++ functions should expect to take aMattress*or equivalent address (e.g.,uintptr_t),
after which theptrshould be extracted for use intatami-compatible functions.Callmattress.tatamize()on Python matrix objects within each of your functions that calltatamiC++ code.
This will wrap the Python matrix in atatami-compatible C++ representation for use in the C++ code.
The pointer to the C++ instance can be accessed through theptrproperty of the returned object,
which can then be passed to C++ code as anuintptr_tto aMattressinstance.So, for example, we can writectypesbindings like:#include"Mattress.h"extern"C"{intdo_something_interesting(constvoid*mat){returnreinterpret_cast<constMattress*>(mat)->ptr->nrow();}}Which we can subsequently call like:importmattressimportctypesasctlib=ct.CDLL("compiled.so")lib.do_something_interesting.restype=ct.c_intlib.do_something_interesting.argtypes=[ct.c_void_p]defdo_something_interesting(x):mat=mattress.tatamize(x)returndo_something_interesting(x.ptr)Of course, any FFI that accepts a pointer address can be used here.Supported matricesDensenumpymatrices of varying numeric type:importnumpyasnpfrommattressimporttatamizex=np.random.rand(1000,100)tatamat=tatamize(x)ix=(x*100).astype(np.uint16)tatamat2=tatamize(ix)Compressed sparse matrices fromscipywith varying index/data types:fromscipyimportsparseasspfrommattressimporttatamizexc=sp.random(100,20,format="csc")tatamat=tatamize(xc)xr=sp.random(100,20,format="csc",dtype=np.uint8)tatamat2=tatamize(xr)Delayed arrays from thedelayedarraypackage:fromdelayedarrayimportDelayedArrayfromscipyimportsparseasspfrommattressimporttatamizeimportnumpyxd=DelayedArray(sp.random(100,20,format="csc"))xd=numpy.log1p(xd*5)tatada=tatamize(xd)To be added:File-backed matrices from theFileBackedArraypackage, including HDF5 and TileDB.Arbitrary Python matrices?Utility methodsTheTatamiNumericPointerinstance returned bytatamize()provides a few Python-visible methods for querying the C++ matrix.tatamat.nrow()//numberofrowstatamat.column(1)//contentsofcolumn1tatamat.sparse()//whetherthematrixissparse.It also has a few methods for computing common statistics:tatamat.row_sums()tatamat.column_variances(num_threads=2)grouping=[i%3foriinrange(tatamat.ncol())]tatamat.row_medians_by_group(grouping)tatamat.row_nan_counts()tatamat.column_ranges()These are mostly intended for non-intensive work or testing/debugging.
It is expected that any serious computation should be performed by iterating over the matrix in C++.Operating on an existing pointerIf we already have aTatamiNumericPointer, we can easily apply additional operations by wrapping it in the relevantdelayedarraylayers and callingtatamize()afterwards.
For example, if we want to add a scalar, we might do:fromdelayedarrayimportDelayedArrayfrommattressimporttatamizeimportnumpyx=numpy.random.rand(1000,10)tatamat=tatamize(x)wrapped=DelayedArray(tatamat)+1tatamat2=tatamize(wrapped)This avoids relying onxand is more efficient as it re-uses theTatamiNumericPointergenerated fromx.Developer NotesBuild the shared object file:pythonsetup.pybuild_ext--inplaceFor quick testing, we usually do:pytestFor more complex testing, we do:pythonsetup.pybuild_ext--inplace&&toxTo rebuild thectypesbindings withcpptypes:cpptypessrc/mattress/lib--pysrc/mattress/_cpphelpers.py--cppsrc/mattress/lib/bindings.cpp--dll_core