Package: actk
actkAutomated Cell ToolkitA pipeline to process field-of-view (FOV) microscopy images and generate data and
render-ready products for the cells in each field. Of note, the data produced by this
pipeline is used for theCell Feature Explorer.FeaturesAll steps and functionality in this package can be run as single steps or all together
by using the command line.In general, all commands for this package will follow the format:actk {step} {command}stepis the name of the step, such as "StandardizeFOVArray" or "SingleCellFeatures"commandis what you want that step to do, such as "run" or "push"Each step will check that the dataset provided contains the required fields prior to
processing. For details and definitions on each field, see ourdataset fields documentation.An example dataset can be seenhere.PipelineTo run the entire pipeline from start to finish you can simply run:actkallrun--dataset{pathtodataset}Step specific parameters can additionally be passed by simply appending them.
For example: the stepSingleCellFeatureshas a parameter forcell_ceiling_adjustmentand this can be set on both the individual step run level and
also for the entire pipeline with:actkallrun--dataset{pathtodataset}--cell_ceiling_adjustment{integer}See thesteps module in our documentationfor a full list of parameters for each stepPipeline ConfigA configuration file can be provided to the underlyingdatasteplibrary that manages
the data storage and upload of the steps in this workflow.The config file should simply be calledworkflow_config.jsonand be available from
whichever directory you runactkfrom. If this config is not found in the current
working directory, defaults are selected by thedatasteppackage.Here is an example of our production config:{"quilt_storage_bucket":"s3://allencell","project_local_staging_dir":"/allen/aics/modeling/jacksonb/results/actk"}You can even additionally attach step-specific configuration in this file by using the
name of the step like so:{"quilt_storage_bucket":"s3://example_config_7","project_local_staging_dir":"example/config/7","example":{"step_local_staging_dir":"example/step/local/staging/"}}AICS Distributed ComputingFor members of the AICS team, to run in distributed mode across the SLURM cluster add
the--distributedflag to the pipeline call.To set distributed cluster and worker parameters you can additionally add the flags:--n_workers {int}(i.e.--n_workers 100)--worker_cpu {int}(i.e.--worker_cpu 2)--worker_mem {str}(i.e.--worker_mem 100GB)Individual Stepsactk standardizefovarray run --dataset {path to dataset}, Generate standardized,
ordered, and normalized FOV images as OME-Tiffs.actk singlecellfeatures run --dataset {path to dataset}, Generate a features JSON
file for each cell in the dataset.actk singlecellimages run --dataset {path to dataset}, Generate bounded 3D images
and 2D projections for each cell in the dataset.actk diagnosticsheets run --dataset {path to dataset}, Generate diagnostic sheets
for single cell images. Useful for quality control.InstallationInstall Requires:The python package,numpy, must be installed prior to the
installation of this package:pip install numpyStable Release:pip install actkDevelopment Head:pip install git+https://github.com/AllenCellModeling/actk.gitDocumentationFor full package documentation please visitallencellmodeling.github.io/actk.Published DataFor a large-scale example of what this library is capable of, please see the data
produced by this pipeline after running our largest cell dataset through it. The data
from the Allen Institute for Cell Science created from this pipeline can be foundhere.This package contains the source microscopy images, segmentation files, pre-processed
single cell images and features, and diagnostic sheets.Our source images are of endogenously-tagged hiPSC, grown for 4 days on Matrigel-coated
96-well, glass bottom imaging plates. Each field of view (FOV) includes 4 channels (BF,
EGFP, DNA, Cell membrane) collected either interwoven with one camera (workflow
Pipeline 4.0 - 4.2) or simultaneously with two cameras (Workflow Pipeline 4.4). You can
use the file metadata of each image to target the specific channel you are interested
in. FOVs were either selected randomly (mode A), enriched for mitotic events (mode B)
or sampling 3 different areas of a colony (edge, ridge, center) using a photo
protective cocktail (mode C). The images cataloged in this dataset come in several
flavors:Field of view (FOV) images with channels* :BrightfieldEGFPDNACell MembraneSegmentation files with channels:Nucleus SegmentationNucleus ContourMembrane SegmentationMembrane Contour* Some FOV images contain seven channels rather than four. The extra three channels
are "dummy" channels added during acquisition that can be ignored.The full details of the Allen Institute cell workflow are available on our websitehere.The full details of the Allen Institute microscopy workflow are available on our
websitehere.The following is provided for each cell:Cell IdCell Index (from within the FOV's segmentation)Metadata (Cell line, Labeled protein name, segmented region index, gene, etc.)3D cell and nuclear segmentation, and, DNA, membrane, and structure channels2D max projects for dimension pairs (XY, ZX, and ZY) of the above 3D imagesA whole bunch of features for each cellFor the 3D single cell images the channel ordering is:Segmented DNASegmented MembraneDNA (Hoechst)Membrane (CellMask)Labeled Structure (GFP)Transmitted LightTo interact with this dataset please see theQuilt Documentation.DevelopmentSeeCONTRIBUTING.mdfor information related to developing the code.For more details on how this pipeline is constructed please seecookiecutter-stepworkflowanddatastep.To add new steps to this pipeline, runmake_new_stepand follow the instructions inCONTRIBUTING.mdDeveloper InstallationThe following two commands will install the package with dev dependencies in editable
mode and download all resources required for testing.pipinstall-e.[dev]pythonscripts/download_test_data.pyAICS Developer InstructionsIf you want to run this pipeline with the Pipeline Integrated Cell dataset
(pipeline 4.*) run the following commands:pipinstall-e.[all]pythonscripts/download_aics_dataset.pyOptions for this script are available and can be viewed with:python scripts/download_aics_dataset.py --helpAcknowledgmentsA previous iteration of this pipeline was created and managed byGregory Johnsonfor work withPyTorch Integrated Cell.This version of this pipeline is more generalized and while still used for the
Integrated Cell model, can be used to pre-process a variety of microscopy image
datasets.The previous version of this pipeline produced thepipeline_integrated_single_cell dataset.Free software: Allen Institute Software License