DOI: 10.5061/dryad.sxksn038s
title: Berkeley Single-Cell Computational Microscopy (BSCCM) dataset

description: Computational microscopy, in which hardware and algorithms of an imaging
 system are jointly designed, shows promise for making imaging systems that
 cost less, perform more robustly, and collect new types of information.
 Often, the performance of computational imaging systems, especially those
 that incorporate machine learning, is sample-dependent. Thus, standardized
 datasets are an essential tool for comparing the performance of different
 approaches. Here, we introduce the Berkeley Single Cell Computational
 Microscopy (BSCCM) dataset, which contains over 400,000 images of
 individual white blood cells. The dataset contains images captured with
 multiple illumination patterns on an LED array microscope and fluorescent
 measurements of the abundance of surface proteins that mark different cell
 types. We hope this dataset will provide a valuable resource for the
 development and testing of new algorithms in computational microscopy and
 computer vision with practical biomedical applications.
description: # Berkeley Single-Cell Computational Microscopy (BSCCM) Dataset
 [https://doi.org/10.5061/dryad.sxksn038s](https://doi.org/10.5061/dryad.sxksn038s) This dataset contains the raw data for the Berkeley Single Cell Computational Microscopy Dataset. The data is compressed and chunked to facilitate downloading. The easiest way to download and use it is through the `bsccm` python package. The code for this package can be found at [https://github.com/Waller-Lab/BSCCM/blob/main/Getting_started.ipynb](https://github.com/Waller-Lab/BSCCM/blob/main/Getting_started.ipynb) and is archived with a DOI at [https://zenodo.org/doi/10.5281/zenodo.10392182](https://zenodo.org/doi/10.5281/zenodo.10392182) ## Loading data& The [Getting Started jupyter notebook](https://github.com/Waller-Lab/BSCCM/blob/main/Getting_started.ipynb) shows the full documentation for how to use this dataset, including installation, downloading, image/metadata querying, and more. Here we reproduce the first few steps of the notebook: * First, install the `bsccm` python package with `pip install bsccm` * Then download the data: ``` from bsccm import download_dataset dataset_path = download_dataset('/path/to/download', tiny=True) print('Downloaded dataset to ' + dataset_path) ``` * Then open an image ``` dataset = BSCCM (dataset_path) valid_indices = dataset.get_indices() an_index = valid_indices[0] image = dataset.read_image(an_index, channel='DPC_Left') ``` ## Data organization After downloading and decompressing the data as described in the Getting_Started notebook, the dataset will be organized as follows. This section describes how the data is organized: which files contain which data, what metadata is available, etc. We note that a full understanding of these details is not necessary for using the dataset, as the Python package we provide abstracts away many of these implementation details. ## File structure and organization All image data are stored in Zarr datasets using Blosc/zstd compression. Tabular metadata (i.e., per-cell metadata) are stored in .csv files. Global metadata, which contains information that is not specific to individual cells, but rather pertains to the whole dataset is stored in text files in Javascript Object Notation (JSON) format. Each top-level BSCCM (regular, coherent, tiny, or coherent-tiny) contains (up to) 5 items: * **BSCCM_images.zarr**: A Zarr dataset containing all the images of cells * **BSCCM_backgrounds.zarr**: A Zarr dataset containing the background intensity over the full field of view, for each LED array illumination pattern. * **BSCCM_global_metadata.json**: A text file containing metadata about the full dataset (pixel size, wavelength, channel names, etc.) in JSON format * **BSCCM_index.csv**: A comma-separated value (CSV) file containing metadata specific to each cell in the dataset * **BSCCM_surface_markers.csv**: A comma-separated value (CSV) file containing information about the surface protein marker levels of each cell, along with many measurements derived from the fluorescence images and intermediate values used in computing these levels ### BSCCM\_images.zarr Zarr datasets contain a hierarchy of directories. For the BSCCM_images.zarr file, this has the following structure: ``` +-- antibodies_CD16 | +-- batch_0 | | +-- slide_replicate_0 | | +-- dpc | | +-- cell_0 | | +-- cell_1 | | ... | | +-- fluor | | ... | | +-- led_array | | ... | | +-- histology | ... +-- antibodies_CD45 ``` The outermost directory contains "antibodies_" followed by the name of the antibody used to stain the cells, or "unstained"/"all" for the no-antibody and all-antibody conditions, respectively. ### BSCCM\_backgrounds.zarr This file contains the background images for each channel across the full field of view (2056x2056 pixels). The top-level directory contains the channel name. The structure is as shown below. ``` +-- Brightfield | +-- 5\_percentile | +-- 10\_percentile | +-- 20\_percentile | +-- 40\_percentile | +-- 50\_percentile +-- DF\_50 ``` ### BSCCM\_global\_metadata.json This file contains metadata specific to the full dataset, including names of channels, collection settings like exposure, and useful information for calibration like wavelength, objective NA, etc. It is a text file with JSON structure. ``` { "led_array": { "image_shape": [128, 128], "channel_names": ["Brightfield", ... "LED119"], "channel_indices": {"Brightfield": 0, ... "LED119": 200}, "exposure_ms": {"Brightfield": 8, ... "LED119": 200}, "camera": { "offset": 30, "gain_db": 4, "quantum_efficiency": 0.68 }, "wavelength_nm": 515, "pixel_size_um": 0.166, "objective": {"NA": 0.5, "magnification": 20}}, "fluorescence": { "image_shape": ... ... ``` ### BSCCM\_index.csv This contains per-cell metadata in a single, large CSV file with one row per each cell. It has the following columns: ``` "global_index": An integer uniquely identifying the cell "position_in_fov_y_pix"/"position_in_fov_x_pix":Location of the cell center within the image field of view "detection_radius": The radius reported by the blob finding algorithm that initially located the cell, which gives a rough estimate of its size "has_matched_histology_cell": Whether or not the cell has a matching cell in histology contrast "fov_center_x"/"fov_center_y"/"fov_center_z": the microscope stage coordinates of the field of view from which the cell was drawn "batch": the index of the cell isolation experiment the cells were drawn from (either 0 or 1) "antibodies": the name or the single antibody used to stain the cells, or 'all' or 'unstained' if every antibody or no antibodies were used "imaging_date": the date the slide of cells was imaged "data_path": the path to the image data with the BSCCM\_images.zarr file "slide_replicate": the index of the slide replicate within the same antibody/batch conditions (either 0 or 1) ``` ### BSCCM\_surface\_markers.csv This contains per-cell calculations about fluorescence surface marker levels. It is entirely derived from the fluorescence imaging data. It contains metadata in a single, large CSV file with one row per each cell. ``` "global_index": An integer uniquely identifying the cell // raw measurements derived from fluorescence images "Fluor_426-446_total_raw" //Raw foreground fluorescence in 426-446nm channel "Fluor_500-550_total_raw" //Raw background fluorescence in 500-550nm channel ... "Fluor_426-446_background" //Raw background fluorescence in 426-446nm channel ... (Many more intermediate calculations) ... // protein levels estimates from unmixing procedure "CD45_single_antibody_model_unmixed" // CD45 protein levels using 2 spectrum unmixing model "CD123_single_antibody_model_unmixed" ... "CD123/HLA-DR/CD14_full_model_unmixed" // Combined CD123/HLA-DR/CD14 protein levels using 4 spectrum unmixng model ... ``` # Versions 6 May 2025: fixed bug in BSCCM-tiny metadata