Package: dexp
dexp| light-sheet Dataset EXploration and Processingdexpis anapari,CuPy,Zarr, andDASKbased library for managing, processing and visualizing light-sheet microscopy datasets. It consists in light-sheet specialised image processing functions (equalisation, denoising, dehazing, registration, fusion, stabilization, deskewing, deconvolution), visualization functions (napari-based viewing, 2D/3D rendering, video compositing and resizing, mp4 generation), as well as dataset management functions (copy, crop, concatenation, tiff conversion). Almost all functions are GPU accelerated viaCuPybut also have anumpy-based fallback option for testing on small datasets. In addition to a functional API, DEXP offers a command line interface that makes it very easy for non-coders to pipeline large processing jobs all the way from a large multi-terabyte raw dataset to fully processed and rendered video in MP4 format.How to installdexpPrerequisites:dexpworks on OSX and Windows, but it is recomended to use the latest version of Ubuntu.
We recommend a machine with NVIDIA graphics card with at least 12G.First, make sure to have aworking python installation.
Second, make sure to have a compatible and functionalCUDA installationOnce these prerequisites are satified, you can installdexp.Installation:dexpcan simply be installed with:pip install dexpTo installsdexpwith GPU support, the optional features and colored console output:conda install cupy
pip install dexp[optional,colored]For OSX users:Before installating dexp, you will first need to install cairo:brew install cairoQuick conda environment setup and installation:There are multiple options when using a conda environment:You can create our suggested DEXP (and some additional packages) environment by, it requires GPU:conda env create --name dexp --file env-linux-gpu.yamlOr create your own conda environment from scratch with the following commands delete any existing dexp environment, recreate it, installdexpwith CUDA support andnapari:conda deactivate
conda env remove --name dexp
conda create -y --name dexp python=3.9
conda activate dexp
conda install cupy
pip install dexp[optional,colored]
pip install napari[all]If you are having problems with the cuda/cuda-toolkit the best is to use conda to install the correct version of the cudatoolkit:conda install -c conda-forge cudatoolkit==11.2.2You can checkherefor the best matching version.Notes:You might get some error messages recommending you install missing libraries such as CUDNN, CuTensor, nccl, etc... These messages often come with instructions on what to do.Adjust your driver version (here 11.2) to your card(s) and drivers.Windows users should callconda install -c conda-forge pyopenclbefore running the second to last step.Leveraging extra CUDA libraries for faster processing:If you want youdexpCUDA-based processing to be even faster, you can install additional libraries such as CUDNN and CUTENSOR
with the following command:conda install -y -c conda-forge cudnn cutensor ncclordexp-install cudalibs 11.2Change the CUDA version accordingly.dexpZarr dataset structureThe zarr datasets injested and written bydexpare organized as below:/(root)└──channel1(group)├──channel1(array)├──channel1_projection_0(optional)├──channel1_projection_1(optional)└──channel1_projection_2(optional)└──channel2(group)├──channel2(array)├──channel2_projection_0(optional)├──channel2_projection_1(optional)└──channel2_projection_2(optional)└──morechannels...Channels (zarr group) could be of a particular emission color (e.g. DAPI, GFP, etc), or/and of a particular imaging views
(e.g. view1 and view2 in a dual view acquisition).
Under each channel group, there could be multiple zarr array. The array that has the same name as the group name is typically
a n-dimentional stack (e.g. time-z-y-x). The projections of that nd array are optional (useful for quick exploration of the
nd stack). When writting output datasetsdexpautomatically generates these projections. Future versions ofdexpmight
add more such convenience arrays, high in the list is of course downscaled version sof the stacks for faster visualisation and
browsing...Note: Our goal is to eventually transition to a ome-zarr and/or ngff storage by defaut for both reading and writting.
For reading we have also support for specific dataset produced by our light-sheet microscopes, seeherefor supported microscopes and formats. This is currently limited but contributions are very welcome!DaXiDEXP supports processing datasets acquired on theDaXi microscope(paper).
You can test processing of DaXi data using anexample datasetVersionsThe list of released versions can be foundhere. The version format is: YYYY.MM.DD.M where YYYY is the year, MM the month, dd the day, and M is the number of elapsed minutes of the day. Git tags are automatically set to link pipy versions to github tagged versions so that the corresponding code can be inspected on github, probably the most important feature. This is a very simple and semantically clear versionning scheme that accomodates for a rapid rate of updates.How to usedexp?In depth documentation can be foundherefor both command line  commands and for the python API.Contributors:Jordao Bragantini, Ahmet Can Solak, Bin Yang, and Loic A Royer