Package: locpix
locpixlocpixis a Python library for analysing point cloud data from SMLM.This project is under active developmentThis is a short ReadMe just containing a QuickStart guide.
For more comprehensive documentation please seehttps://oubino.github.io/locpix/locpixincludes the following functionality in order they are used in a normal workflow:Preprocess: Initialises project and converts .csv files representing SMLM data (point cloud) into .parquet files,necessary for this softwareAnnotate: Generating histograms from the SMLM data and manually annotating these histograms to extract relevant localisations & labelling with seeds for watershed algorithmSegmentation:Classic segmentation: Use classic method to segment histograms to extract relevant localisationsCellpose segmentation: Use Cellpose method to segment histograms to extract relevant localisationsIlastik segmentation: Use Ilastik method to segment histograms to extract relevant localisationsUNET segmentation: Use UNET to segment histograms to extract relevant localisationsMembrane performance: Performance metrics calculation based on the localisations (not the histograms!)Project StructureWe assume your input SMLM data are .csv files.This input data must first be preprocessed into a user chosen project directory, using thePreprocessscript.
We strongly suggest this project directory is located outside the locpix folder.The input and output of all further scripts will remain located inside the project directory, the input data folder
will not be accessed again!Usage configurationEach script needs a configuration file (.yaml file), which should be
specified using the -c flag.Each configuration used will be saved in the project directory.The templates for the configuration files can be found in thetemplates folder.For instructions for the parameters in the configuration files seehttps://oubino.github.io/locpix/user_guide/templates/index.htmlQuickstartInstallationPrerequisitesYou will need anaconda or miniconda or mamba.
We recommendmambaInstallCreate an environment and install via pypiconda create -n locpix-env python==3.10
conda activate locpix-env
pip install locpixPreprocessingPreprocessThis script preprocesses the input .csv data for later use ANDmust be run first.This script will take in .csv files, and convert them to .parquet files,
while also wrangling the data into our data format.To run the script -i -c and -o flags should be specifiedpreprocess -i path/to/input/data -c path/to/config/file -o path/to/project/directoryOptionally we can add:-s flag to check the correct files are selected before preprocessing them-p flag if our files are .parquet files not .csv files.AnnotateThis script allows for manual segmentation of the localisations and adding markers to the images as seeds for the watershed algorithm.To run the script -i and -c flags should be specifiedannotate -i path/to/project/directory -c path/to/config/fileOptionally we can add:-m flag to check the metadata of the project before running-r flag to relabel histograms, will assume some labels are already presentImage segmentationClassic segmentationPerform classic segmentation on our localisation dataset.To run the script -i and -c flags should be specifiedclassic -i path/to/project/directory -c path/to/config/fileOptionally we can add:-m flag to check the metadata of the project before runningCellpose segmentationNeed to activate extra requirements - these are big and not included in initial install.Note that if you have a GPU this will speed this up.Note we modified Cellpose to fit in with our analysis, therefore you need to install our forked repository - note below will clone the Cellpose repository to wherever you are locatedIf you have a GPUpip install torch torchvision --extra-index-url https://download.pytorch.org/whl/cu117
git clone https://github.com/oubino/cellpose
cd cellpose
pip install .If you don’t have a GPUpip install pytorch
git clone https://github.com/oubino/cellpose
cd cellpose
pip install .To evaluate Cellpose model on the localisation dataset without any retraining on your dataset run the script with -i and -c flags specifiedcellpose_eval -i path/to/project/directory -c path/to/config/fileOptionally we can add:-m flag to check the metadata of the project before running-o flag to specify folder in project dir to save output (defaults to cellpose_no_train)-u flag to specify a user model to load inTo retrain first then evaluate we insteadPrepare data for trainingtrain_prep -i path/to/project/directory -c path/to/config/fileOptionally we can add:-m flag to check the metadata of the project before runningTrain cellposecellpose_train -i path/to/project/directory -ct path/to/config/train_file -ce path/to/config/eval_fileOptionally we can add:-m flag to check the metadata of the project before runningUNET segmentationNeed to activate extra requirements - these are big and not included in initial install.Note that if you have a GPU this will speed this up.Note this is only needed if haven’t done for cellpose aboveIf you have a GPUpip install torch torchvision --extra-index-url https://download.pytorch.org/whl/cu117If you don’t have a GPUpip install pytorchTo train UNETunet -i path/to/project/directory -c path/to/config/fileOptionally we can add:-m flag to check the metadata of the project before runningIlastik segmentationNeed to prepare the data for Ilastik segmentationilastik_prep -i path/to/project/directory -c path/to/config/fileOptionally we can add:-m flag to check the metadata of the project before runningThen run the data through the Ilastik GUI, which needs to be installed fromIlastikand to run it
please seehttps://oubino.github.io/locpix/user_guide/usage.html#id7Then convert the output of the Ilastik GUI back into our formatilastik_output -i path/to/project/directoryOptionally we can add:-m flag to check the metadata of the project before runningMembrane performanceTo evaluate membrane performance for a particular method, run below, where method name needs to match where the segmentation files aremembrane_performance_method -i path/to/project/directory -c path/to/config/file -o method_nameOptionally we can add:-m flag to check the metadata of the project before runningTo evaluate performance of  membrane segmentation from classic, cellpose and ilastikmembrane_performance -i path/to/project/directory -c path/to/config/fileOptionally we can add:-m flag to check the metadata of the project before runningTo aggregate the performance over the folds for methods classic, cellpose without training, cellpose with training and ilastikagg_metrics -i path/to/project/directoryLicensesPackageLicensealabaster 0.7.12UNKNOWNapp-model 0.1.1BSD 3-Clause Licenseappdirs 1.4.4MITarrow 1.2.3Apache 2.0astroid 2.12.13LGPL-2.1-or-laterasttokens 2.2.0Apache 2.0attrs 22.1.0MITBabel 2.11.0BSDbackcall 0.2.0UNKNOWNbackports.functools-lru-cache 1.6.4UNKNOWNbeautifulsoup4 4.11.1MITbinaryornot 0.4.4BSDblack 22.12.0MITbuild 0.9.0MITcachey 0.2.1BSDcellpose 2.1.2.dev26+g731fe4eBSDcertifi 2022.9.24MPL-2.0cfgv 3.3.1MITchardet 5.1.0LGPLcharset-normalizer 2.1.1MITclick 8.1.3BSD-3-Clausecloudpickle 2.2.0BSD 3-Clause Licensecolorama 0.4.6UNKNOWNcomm 0.1.3BSD 3-Clause Licensecommonmark 0.9.1BSD-3-Clausecontourpy 1.0.6BSD-3-Clausecookiecutter 2.1.1BSDcoverage 6.5.0Apache 2.0cycler 0.11.0BSDdask 2022.11.1BSDdebugpy 1.6.4MITdecorator 5.1.1new BSD Licensedistlib 0.3.6Python licensedocstr-coverage 2.2.0MITdocstring-parser 0.15MITdocutils 0.17.1public domain, Python, 2-Clause BSD, GPL 3 (see COPYING.txt)entrypoints 0.4UNKNOWNexceptiongroup 1.0.4UNKNOWNexecuting 1.2.0MITfastremap 1.13.3LGPLv3filelock 3.9.0UNKNOWNflake8 6.0.0MITfonttools 4.38.0MITfreetype-py 2.3.0UNKNOWNfsspec 2022.11.0BSDHeapDict 1.0.1BSDhsluv 5.0.3MITidentify 2.5.17MITidna 3.4UNKNOWNimagecodecs 2022.9.26BSDimageio 2.22.4BSD-2-Clauseimageio-ffmpeg 0.4.7BSD-2-Clauseimagesize 1.4.1MITimportlib-metadata 6.6.0UNKNOWNin-n-out 0.1.6BSD 3-Clause Licenseiniconfig 1.1.1MIT Licenseipykernel 6.17.1UNKNOWNipython 8.13.2BSD-3-Clauseipython-genutils 0.2.0BSDjedi 0.18.2MITJinja2 3.1.2BSD-3-Clausejinja2-time 0.2.0MITjoblib 1.2.0BSDjsonschema 4.17.3MITjupyter-client 7.4.7UNKNOWNjupyter-core 5.1.0UNKNOWNkiwisolver 1.4.4UNKNOWNlazy-object-proxy 1.8.0BSD-2-Clauseline-profiler 4.0.2BSDllvmlite 0.39.1BSDlocket 1.0.0BSD-2-Clauselocpix 0.0.12.dev70+ga7833b4.d20230120UNKNOWNmagicgui 0.6.1MIT licenseMarkupSafe 2.1.1BSD-3-Clausematplotlib 3.6.2PSFmatplotlib-inline 0.1.6BSD 3-Clausemccabe 0.7.0Expat licensemypy-extensions 0.4.3MIT Licensenapari 0.4.17BSD 3-Clausenapari-console 0.0.6BSD 3-Clausenapari-locpix 0.0.3MITnapari-plugin-engine 0.2.0MITnapari-svg 0.1.6BSD-3natsort 8.2.0MITnest-asyncio 1.5.6BSDnetworkx 2.8.8UNKNOWNnodeenv 1.7.0BSDnpe2 0.6.1BSD-3-Clausenumba 0.56.4BSDnumpy 1.23.5BSDnumpydoc 1.5.0BSDopencv-python-headless 4.6.0.66MITpackaging 21.3BSD-2-Clause or Apache-2.0pandas 1.5.2BSD-3-Clauseparso 0.8.3MITpartd 1.3.0BSDpathspec 0.10.2MPL 2.0pep517 0.13.0UNKNOWNpickleshare 0.7.5MITPillow 9.3.0HPNDPint 0.20.1BSDpip 23.1.2MITplatformdirs 2.5.4UNKNOWNpluggy 1.0.0MITpolars 0.15.1MITpre-commit 3.0.3MITprettytable 3.8.0BSD (3 clause)prompt-toolkit 3.0.33UNKNOWNpsutil 5.9.4BSD-3-Clausepsygnal 0.6.1BSD 3-Clause Licensepure-eval 0.2.2MITpyarrow 10.0.1Apache License, Version 2.0pycodestyle 2.10.0Expat licensepydantic 1.10.2MITpydata-sphinx-theme 0.12.0BSD 3-Clause Licensepyflakes 3.0.1MITPygments 2.13.0BSD LicensePyOpenGL 3.1.6BSDpyparsing 3.0.9UNKNOWNPyQt5 5.15.7GPL v3PyQt5-Qt5 5.15.2LGPL v3PyQt5-sip 12.11.0SIPpyrsistent 0.19.2MITpytest 7.2.0MITpytest-cov 4.0.0MITpython-dateutil 2.8.2Dual Licensepython-dotenv 0.21.0BSD-3-Clausepython-slugify 7.0.0MITpytomlpp 1.0.11UNKNOWNpytz 2022.6MITPyWavelets 1.4.1MITpywin32 304PSFPyYAML 6.0MITpyzmq 24.0.1LGPL+BSDqtconsole 5.4.0BSDQtPy 2.3.0MITrequests 2.28.1Apache 2.0rich 12.6.0MITscikit-image 0.19.3Modified BSDscikit-learn 1.1.3new BSDscipy 1.9.3UNKNOWNseaborn 0.12.2UNKNOWNsetuptools 67.7.2UNKNOWNsetuptools-scm 7.0.5MITsix 1.16.0MITsnowballstemmer 2.2.0BSD-3-Clausesoupsieve 2.3.2.post1UNKNOWNSphinx 4.5.0BSDsphinx-autoapi 2.0.0MITsphinxcontrib-applehelp 1.0.2BSDsphinxcontrib-devhelp 1.0.2BSDsphinxcontrib-htmlhelp 2.0.0BSDsphinxcontrib-jsmath 1.0.1BSDsphinxcontrib-qthelp 1.0.3BSDsphinxcontrib-serializinghtml 1.1.5BSDstack-data 0.6.2MITsuperqt 0.4.1BSD 3-Clause Licensetext-unidecode 1.3Artistic Licensethreadpoolctl 3.1.0BSD-3-Clausetifffile 2022.10.10BSDtomli 2.0.1UNKNOWNtoolz 0.12.0BSDtorch 1.13.0+cu117BSD-3torchsummary 1.5.1UNKNOWNtorchvision 0.14.0+cu117BSDtornado 6.2http://www.apache.org/licenses/LICENSE-2.0tqdm 4.64.1MPLv2.0, MIT Licencestraitlets 5.6.0UNKNOWNtyper 0.7.0UNKNOWNtyping-extensions 4.4.0UNKNOWNUnidecode 1.3.6GPLurllib3 1.26.13MITvirtualenv 20.17.1MITvispy 0.11.0(new) BSDwcwidth 0.2.5MITwheel 0.40.0MIT Licensewrapt 1.14.1BSDzipp 3.15.0UNKNOWN