Package: atlas-densities
OverviewThis project contains the tools to create theBBP Cell Atlas, using datasets from theAllen Institute for Brain Science (AIBS)and collected literature density values.
The AIBS datasets files were obtained via experiments performed on P56 wild-type mouse brains.
The outcome of this project is a list of volumetric files that provides cell type density estimates
for each voxel of the mouse brain volume. The BBP Cell Atlas is the first model required to
reconstruct BBP circuits of the mouse brain.The tools implementation is based on the methods ofEroe et al. (2018),Rodarie et al. (2022),
andRoussel et al. (2022).The source code was originally written by Csaba Eroe, Dimitri Rodarie, Hugo Dictus, Lu Huanxiang,
Wojciech Wajerowicz, Jonathan Lurie, and Yann Roussel.The tools allow to:combine AIBS annotation files to reinstate missing mouse brain regionscombine several AIBS gene marker datasets, to be used as hints for the spatial distribution of glial cellscompute cell densities for several cell types including neurons and glial cells in the whole mouse brainTools can be used through a command line interface.After installation, you can display the available command lines with the followingbashcommand:atlas-densities--helpInstallationgitclonehttps://github.com/BlueBrain/atlas-densitiescdatlas-densitiespipinstall-e.cgal-pybindThis project depends on the BBP python projectcgal-pybind.
The python projectcgal-pybindneeds to be installed prior to the above instructions.ExamplesNote: Depending on the size and resolution of the atlas, it can happen that some of the examples may not run with the 25um atlas from the Allen Institute due to memory requirements.  For instance, 8GB of memory is often not enough for the “Combine ISH datasets for glia cells”, depending on the datasets being combined.Reference atlasesMost the pipeline steps rely on the following AIBS reference datasets (seeRodarie et al. (2022)for more
details on the different versions of these datasets):A Nissl volumeAn annotation volumeA brain region hierarchy fileThe AIBS Nissl reference volume is derived from high resolution image stacks of Nissl-stained
coronal brain slices. These images are realigned and arranged to form a 3D brain volume. Nissl
stains each cell somas. The intensity of each voxel of the Nissl volume correlates with the number
of cells in this voxel. This Nissl volume has a corresponding annotation volume which maps each
voxel of the Nissl volume to a brain region. The AIBS organize brain regions in a hierarchical tree,
with leaves representing the finest parcellation.These datasets are publicly available on the AIBS website.Download reference atlasesMakedatadirectory, and download needed annotation volumes, Nissl volume and hierarchy file:mkdir-pdata/ccfv2data/ccfv3# hierarchy file:curl-odata/1.jsonhttp://api.brain-map.org/api/v2/structure_graph_download/1.json# also available in atlas_densities/app/data/1.json# CCFv2 annotation volumes:curl-odata/ccfv2/annotation_25.nrrdhttp://download.alleninstitute.org/informatics-archive/current-release/mouse_ccf/annotation/mouse_2011/annotation_25.nrrdcurl-odata/ccfv2/annotation_fibers_25.nrrdhttp://download.alleninstitute.org/informatics-archive/current-release/mouse_ccf/annotation/mouse_2011/annotationFiber_25.nrrd# CCFv3 annotation volume:curl-odata/ccfv3/annotation_25.nrrdhttp://download.alleninstitute.org/informatics-archive/current-release/mouse_ccf/annotation/ccf_2017/annotation_25.nrrd# AIBS Nissl volume:curl-odata/ccfv2/ara_nissl_25.nrrdhttp://download.alleninstitute.org/informatics-archive/current-release/mouse_ccf/ara_nissl/ara_nissl_25.nrrdPreparation of the annotation volumeDepending on the chosen version for the reference volumes pairs, some additional preparation
are needed.The ccfv2 annotations are split into two volumes. One describes solely the fibers and
ventricular related regions while the other contains all other brain regions.
If you choose the ccfv2 version to estimate cell densities, then you should combine these two datasets:# Here we override the ccfv2 annotation_25.nrrd fileatlas-densitiescombinationcombine-ccfv2-annotations\--brain-annotation-ccfv2=data/ccfv2/annotation_25.nrrd\--fiber-annotation-ccfv2=data/ccfv2/annotation_fibers_25.nrrd\--output-path=data/ccfv2/annotation_25.nrrdThe ccfv3 annotation volume is more smooth but lacks a few brain parcellations that were present in
the ccfv2 counterpart. If you choose the ccfv3 version to estimate cell densities, you can extend
the ccfv3 annotation atlas with annotations of ccfv2.# Here we override the ccfv3 annotation_25.nrrd fileatlas-densitiescombinationcombine-v2-v3-annotations\--hierarchy-path=data/1.json\--brain-annotation-ccfv2=data/ccfv2/annotation_25.nrrd\--fiber-annotation-ccfv2=data/ccfv2/annotation_fibers_25.nrrd\--brain-annotation-ccfv3=data/ccfv3/annotation_25.nrrd\--output-path=data/ccfv3/annotation_25.nrrdIn the following examples, we will use the ccfv2 reference atlases.In situhybridization datasetsDownload, realign and interpolate AIBS ISH datasetsThe BBP Cell Atlas pipeline leveragesin situhybridization (ISH) volumetric datasets from the AIBS
to estimate cell type densities in all regions of the mouse brain. These ISH datasets have to be
realigned to the Nissl reference volume.TheDeepAtlastoolkit allows you to download AIBS ISH image stacks, align these images to the Nissl
volume, and interpolate between them to create volumetric ISH datasets for each gene of your interest.Combine ISH datasets for glia cellsGlial cells express many genetic markers but none is expressed by every glial cells.
We combine herein situhybridization (ISH) volumetric datasets for glia markers and compute the
glia subtypes global scaling factors, as described inEroe et al. (2018).
The list of required genes to combine for the generic glia marker can be found atcombine_markers_ccfv2_config.yaml.atlas-densitiescombinationcombine-markers\--hierarchy-path=data/1.json\--annotation-path=data/ccfv2/annotation_25.nrrd\--config=atlas_densities/app/data/markers/combine_markers_ccfv2_config.yamlISH datasets for inhibitory/excitatory neuronsInEroe et al. (2018)(i.e., BBP Cell Atlas version 1), the excitatory neurons are distinguished
from the inhibitory neurons using the Nrn1 and GAD67 (or GAD1) genetic marker.InRodarie et al. (2022)(i.e., BBP Cell Atlas version 2), the authors used parvalbumin (Pvalb),
somatostatin (SST), vasoactive intestinal peptide (VIP) and gabaergic (GAD1) markers (see alsofit_average_densities_ccfv2_config.yaml).Cell density estimationCompute and save the overall mouse brain cell density based on Nissl stained AIBS data.# make output directorymkdir-pdata/ccfv2/density_volumes/atlas-densitiescell-densitiescell-density\--hierarchy-path=data/1.json\--annotation-path=data/ccfv2/annotation_25.nrrd\--nissl-path=data/ccfv2/ara_nissl_25.nrrd\--output-path=data/ccfv2/density_volumes/cell_density.nrrdDistinguish neurons from glia cellsCompute and save the glial cell and neuron densities, based on overall cell densities from previous
section and the combined ISH glia datasets (seeIn situhybridization datasets section).
The filesglia.nrrd,oligodendrocyte.nrrd,microglia.nrrd,astrocyte.nrrdandglia_proportions.jsoncome from the output of the ISH dataset combination step.atlas-densitiescell-densitiesglia-cell-densities\--hierarchy-path=data/1.json\--annotation-path=data/ccfv2/annotation_25.nrrd\--cell-density-path=data/ccfv2/density_volumes/cell_density.nrrd\--glia-density-path=data/ccfv2/relative_marker_volumes/glia.nrrd\--astrocyte-density-path=data/ccfv2/relative_marker_volumes/astrocyte.nrrd\--microglia-density-path=data/ccfv2/relative_marker_volumes/microglia.nrrd\--oligodendrocyte-density-path=data/ccfv2/relative_marker_volumes/oligodendrocyte.nrrd\--glia-proportions-path=data/ccfv2/relative_marker_volumes/glia_proportions.json\--output-dir=data/ccfv2/density_volumesExtract literature neuron type densities estimatesInRodarie et al. (2022), the authors collected density estimates from the literature for
inhibitory neurons. Some estimates are in a format that can not be directly used by the pipeline
(e.g., counts instead of densities). This part of the pipeline integrates the literature values into
csv files, that will be used later on for the fitting.Format literature review filesWe compile here the cell density estimates related to measurements ofKim et al. (2017)density
file (mmc3.xlsx) andRodarie et al. (2022)literature
review file (gaba_papers.xlsx) into a single CSV file.
Regions known to be purely excitatory or inhibitory (in terms of neuron composition) are also listed
in a separate CSV file.# make output directorymkdir-pdata/ccfv2/measurementsatlas-densitiescell-densitiescompile-measurements\--measurements-output-path=data/ccfv2/measurements/measurements.csv\--homogenous-regions-output-path=data/ccfv2/measurements/homogeneous_regions.csvConvert literature measurements into average densitiesCompute and save average cell densities based on literature measurements and Cell Atlas data (e.g.,
region volumes).WARNING:
Different versions of the annotation atlas or the hierarchy file might have different sets brain
regions (seeRodarie et al. (2022)for more details). The region names used by the literature
measurements might therefore have no match in these datasets.
Regions from the measurements that are not in the hierarchy or do not appear in the annotations will
be ignored. A warning message will display these regions, allowing us to review them.atlas-densitiescell-densitiesmeasurements-to-average-densities\--hierarchy-path=data/1.json\--annotation-path=data/ccfv2/annotation_25.nrrd\--cell-density-path=data/ccfv2/density_volumes/cell_density.nrrd\--neuron-density-path=data/ccfv2/density_volumes/neuron_density.nrrd\--measurements-path=data/ccfv2/measurements/measurements.csv\--output-path=data/ccfv2/measurements/lit_densities.csvFit transfer functions from mean region intensity to neuron densityWe fit here transfer functions that describe the relation between mean ISH expression in regions of
the mouse brain and literature regional density estimates (seeRodarie et al. (2022)for more
details). This step leverages AIBS ISH marker datasets (in their expression form, see alsofit_average_densities_ccfv2_config.yaml) and the previously computed
literature density values.
These transfer functions are used to obtain first estimates of neuron densities in regions not
covered by literature.
The result of the following command is a list of first density estimates for each neuron type and
for each region of the annotation volume.# make output foldermkdir-pdata/ccfv2/first_estimatesatlas-densitiescell-densitiesfit-average-densities\--hierarchy-path=data/1.json\--annotation-path=data/ccfv2/annotation_25.nrrd\--neuron-density-path=data/ccfv2/density_volumes/neuron_density.nrrd\--average-densities-path=data/ccfv2/measurements/lit_densities.csv\--homogenous-regions-path=data/ccfv2/measurements/homogeneous_regions.csv\--gene-config-path=atlas_densities/app/data/markers/fit_average_densities_ccfv2_config.yaml\--fitted-densities-output-path=data/ccfv2/first_estimates/first_estimates.csv\--fitting-maps-output-path=data/ccfv2/first_estimates/fitting.jsonCompute inhibitory/excitatory neuron densitiesThe neuron subtypes are here distinguished from each other using either the pipeline fromEroe et al. (2018)(BBP Cell Atlas version 1) orRodarie et al. (2022)(BBP Cell Atlas version
2).BBP Cell Atlas version 1Estimate excitatory and inhibitory neuron densities from the Nrn1 and GAD1 (or GAD67) AIBS ISH markers
and whole brain estimates fromKim et al. (2017)(located atmmc1.xlsx).atlas-densitiescell-densitiesinhibitory-and-excitatory-neuron-densities\--hierarchy-path=data/1.json\--annotation-path=data/ccfv2/annotation_25.nrrd\--gad1-path=data/ccfv2/marker_volumes/gad1.nrrd\--nrn1-path=data/ccfv2/marker_volumes/nrn1.nrrd\--neuron-density-path=data/ccfv2/density_volumes/neuron_density.nrrd\--inhibitory-neuron-counts-path=atlas_densities/app/data/measurements/mmc1.xlsx\--output-dir=data/ccfv2/density_volumes/BBP Cell Atlas version 2Estimate GAD67, Pvalb, SST, and VIP neuron densities from the literature and the
transfer functions computed previously (first density estimates).atlas-densitiescell-densitiesinhibitory-neuron-densities\--hierarchy-path=data/1.json\--annotation-path=data/ccfv2/annotation_25.nrrd\--neuron-density-path=data/ccfv2/density_volumes/neuron_density.nrrd\--average-densities-path=data/ccfv2/first_estimates/first_estimates.csv\--output-dir=data/ccfv2/density_volumes/Compute ME-types densities from a probability mapMorphological and Electrical type densities of inhibitory neurons in the isocortex can be estimated
usingRoussel et al. (2022)’s pipeline. This pipeline produces a mapping from inhibitory neuron
molecular types (here PV, SST, VIP and GAD67) to ME-types defined inMarkram et al. (2015).The following command creates neuron density nrrd files for the me-types listed in a probability
mapping csv file (see alsomtypes_probability_map_config.yaml).atlas-densitiesmtype-densitiescreate-from-probability-map\--hierarchy-path=data/1.json\--annotation-path=data/ccfv2/annotation_25.nrrd\--probability-mapdata/probability_map01.csv\--probability-mapdata/probability_map02.csv\--synapse-classINH\--markergad67data/molecular_densities/gad67.nrrd\--markerpvdata/molecular_densities/pv.nrrd\--markersstdata/molecular_densities/sst.nrrd\--markervipdata/molecular_densities/vip.nrrd\--markerapprox_lamp5data/molecular_densities/approx_lamp5.nrrd\--n-jobs8\--output-dir=data/ccfv2/me-types/The molecular density of approx_lamp5 was calculated from the other molecular densities as\begin{equation*}
approx_lamp5 = gad67 - vip - sst - pv
\end{equation*}which approximates the molecular density of lamp5.This can be calculated via command line via:atlas-densitiescombinationmanipulate\--clip\--base-nrrddata/molecular_densities/gad67.nrrd\--subtractdata/molecular_densities/vip.nrrd\--subtractdata/molecular_densities/pv.nrrd\--subtractdata/molecular_densities/sst.nrrd\--output-pathapprox_lamp5.nrrdThe command outputs the density files in the output-dir and a metadata json file:{"SLAC":{"bNAC":"data/ccfv2/test_regions_me-types/SLAC|bNAC_densities.nrrd","cAC":"data/ccfv2/test_regions_me-types/SLAC|cAC_densities.nrrd","cNAC":"data/ccfv2/test_regions_me-types/SLAC|cNAC_densities.nrrd"},"NGC-SA":{"cNAC":"data/ccfv2/test_regions_me-types/NGC-SA|cNAC_densities.nrrd",// ...},// ...}Subdivide excitatory files into pyramidal subtypesThis should run after the inhibitory/excitatory computation above. To run:atlas-densitiescell-densitiesexcitatory-split\--annotation-path=data/ccfv2/annotation_25.nrrd\--hierarchy-path=data/1.json\--neuron-density=data/ccfv2/density_volumes/neuron_density.nrrd\--inhibitory-density=data/ccfv2/density_volumes/gad67+_density.nrrd\--output-dir=data/ccfv2/excitatory_splitInstructions for developersRun the following commands before submitting your code for review:cdatlas-densitiesisort-l100--profileblackatlas_densitiestestssetup.pyblack-l100atlas_densitiestestssetup.pyThese formatting operations will help you pass the linting checktestenv:lintdefined intox.ini.CitationWhen you use this software, we kindly ask you to cite the following DOI:AcknowledgementsThe development of this software was supported by funding to the Blue Brain Project, a research
center of the École polytechnique fédérale de Lausanne (EPFL), from the Swiss government’s ETH Board
of the Swiss Federal Institutes of Technology.For license and authors, see LICENSE.txt and AUTHORS.txt respectively.Copyright © 2022 Blue Brain Project/EPFL