Microscopy Image Browser: A Platform for Segmentation and Analysis of Multidimensional Datasets

Understanding the structure-function relationship of cells and organelles in their natural context requires multidimensional imaging. As techniques for multimodal 3-D imaging have become more accessible, effective processing, visualization, and analysis of large datasets are posing a bottleneck for the workflow. Here, we present a new software package for high-performance segmentation and image processing of multidimensional datasets that improves and facilitates the full utilization and quantitative analysis of acquired data, which is freely available from a dedicated website. The open-source environment enables modification and insertion of new plug-ins to customize the program for specific needs. We provide practical examples of program features used for processing, segmentation and analysis of light and electron microscopy datasets, and detailed tutorials to enable users to rapidly and thoroughly learn how to use the program.

section_content: Imaging and image analysis are among the key methods in biosciences nowadays. The knowledge of complex 3-D structures of cells and cell organelles in their natural context is important for understanding the structure-function relationship. Moreover, statistical quantification of 3-D objects based on 2-D image information cannot be reliably made; therefore, segmentation, analysis, visualization, and comparison of whole 3-D volumetric datasets are required. Recently evolved 3-D/5-D light microscopy (LM) and electron microscopy (EM) techniques have enabled a new insight into the morphology of tissues, cells, and cell organelles that had not been conceivable before [1, 2]. As the amount of collected data is exponentially increasing, the effectiveness of processing raw data into analyzed results has key importance. There are a number of both commercial (e. g., Amira and Imaris) and freeware image processing packages (e. g., ImageJ [3], Fiji [4], BioImageXD [5], IMOD [6], Ilastik [7], and 3D Slicer [8] ) available. However, the performance and usability of image segmentation tools in most packages are still suboptimal, resulting in laborious and time-consuming workflows. Furthermore, with the wide range of software and applications used for data collection and image analysis, there is a clear need for open-access programs that can be adjusted according to the needs of a specific project as well as for cross communication between existing programs. 

Here, we present a new open-source software, Microscopy Image Browser (MIB) [9], that was designed for, but not limited to, easy and effective segmentation of multidimensional datasets, improving and facilitating the full utilization and quantitation of acquired data. MIB has a user-friendly graphical interface and is available for all common computer operating systems, either together with MATLAB (Windows, Linux, and Mac OS) or as a standalone package for Windows and Mac OS. At present, MIB has been utilized in more than ten different scientific projects, ranging from studies at the cellular level to those dealing with whole organisms; examples include projects on the endoplasmic reticulum (ER) and cytoskeletal filaments in cultured cells [10, 11], the organ of Corti in mouse inner ear [12, 13], the development of the sieve element in Arabidopsis thaliana root [14, 15], and the characterization of cryptomonad Rhinomonas nottbecki n. sp. [16]. Although MIB was originally designed for the processing of relatively large EM datasets, it can be used for analysis of LM and any other microscopy datasets. Here, we provide several examples highlighting the various features of the program, and online tutorials have been made to provide detailed instructions on how to use them [17].
section_name: Introduction

section_content: The output files from different microscopes and programs are routinely stored in proprietary formats, and access to the collected images and corresponding metadata after acquisition often requires customized software from the manufacturer. MIB overcomes this problem by offering reading capabilities of up to 100 microscopy image and video formats powered by custommade MATLAB and Bio-Formats [18] readers (Fig 1A ; see MIB home page for the full list [9] ). MIB was designed as an image browser to allow fast access to individual image datasets for viewing and assembling into 3-D and 4-D stacks (X:Y:Color:Z or X:Y:Color:Time). Up to eight datasets can be simultaneously opened and synchronized, facilitating the comparison and analysis of data from different experiments. The processed images can be exported using most frequently used output formats (Fig 1A ). 

Quantitative analysis and efficient segmentation of large or otherwise challenging datasets can be facilitated by preprocessing of images (Fig 1A ): MIB can resize, flip, rotate, transpose, or crop the datasets as well as adjust brightness, contrast, and gamma settings. In all cases, the voxel (volume pixel) dimensions and the bounding box (exact coordinates of a data block in 3-D space) of the dataset are automatically controlled and adjusted with respect to the performed action. MIB also offers more advanced tools; for example, it has a tool to normalize intensities within the dataset. As intensity levels can easily fluctuate between different images, it is essential to normalize intensities prior to further processing. In many cases, normalization is done based on analysis of intensities of complete slices, while MIB offers additional options for normalization based on preselected areas, which allows normalization even for nonuniform images. Prefiltering of images, especially using anisotropic diffusion filters [19, 20], facilitates the segmentation process because of simplification and noise reduction of the images (see MIB homepage for full list of available filters [9] ). During a modelling workflow, the filters can be applied to certain arbitrarily shaped (rectangular, ellipsoidal, or polygonal) regions of interest. The alignment feature in MIB can be used to stitch together individual 3-D/4-D datasets while preserving and adjusting the voxel size and bounding box coordinates, correspondingly.
section_name: MIB Recognizes a Large Number of Imaging Formats and Offers a Variety of Image Processing Tools

section_content: Biological specimens vary greatly in regard to their shape, size, distribution, and intensity, thereby requiring multiple image segmentation procedures. The image segmentation in MIB combines manual and automatic approaches that may be further polished by filtering the segmented objects using quantification analysis (Fig 1B ). This analysis can be easily performed on the shape or intensity properties of the resulting 2-D and 3-D objects. The main manual segmentation tools comprise of brush (normal or against superpixels [21] for faster selection), membrane click tracker (for tracing membrane profiles), magic wand and region growing, spot, and 3-D ball (S1 Video). When using these tools, the researcher interactively follows the modelling process slice by slice. To speed up the manual segmentation, MIB has shape and line interpolation algorithms that fill the gaps between objects drawn on two separated slices. The program also has several semiautomated routines, such as black and white thresholding, that can be applied to the entire dataset or to local selected (masked) area (S1 Video). Selection of such local areas is simple and usually done with the brush tool and the shape interpolation. This approach is especially useful for 3-D EM datasets, as the contrast differences are small and global thresholding (even with quantification filtering) is usually ineffective. The Frangi tubular filter [22] allows the detection of elongated, tubular structures in both 2-D and 3-D space, which is often useful for the segmentation of membranes or vessels. The best suitable segmentation tool depends on the specimen, the imaging method, and the object of interest, and has to be found empirically. As an example, the brush tool was used for segmentation of individual Golgi cisternae ( ).
section_name: Multiple Manual and Semiautomatic Tools Are Gathered Together to Enable Segmentation of Various Specimens and Objects of Interest

section_content: Recent advances in both LM and EM have made it possible to acquire large volumetric datasets in a relatively easy manner. Those datasets may vary in size from moderately large volumes of up to 1,000 x 1,000 x 1,000 voxels to huge datasets of about three orders of magnitude higher (e. g., [23, 24] ). Manual segmentation of such datasets would be tedious, slow, and inefficient, and, as a result, a significant amount of high-quality data may stay unprocessed. Latest developments in image segmentation aim to minimize manual work by implementing automatic approaches of machine learning [25]. A promising example of the machine learning methods for image segmentation is the use of classifiers [7, 23, 26, 27]. Classification starts by manual labelling of representative areas belonging to objects of interest and background. Next, by using various quantitative features (e. g., intensities, texture, and morphology) of the labelled area, the classifier can extend the local segmentation to cover the whole dataset. In some cases, the results can be improved when classification is combined with other methods, for example, the use of supervoxels with watershed [28] or graph-cut-based algorithms [29]. Usually, the use of supervoxels facilitates processing without significant degradation of segmentation results. 

MIB currently includes two classifiers. The first one (called membrane detection) is based on a protocol developed by Kaynig et al. [27] for membrane detection from EM images. However, it is also suitable for segmentation of organelles from EM (S2 Video) and LM datasets (Fig 3A ). We have used it to detect ER from time-lapse LM videos when, because of the gradient of the background intensities and variety of shapes (sheets and tubules), the standard thresholding methods are insufficient. The second classifier (called supervoxel classification) was designed to be fast and is based on classification of superpixels (2-D images) or supervoxels (3-D volumes) generated by the simple linear iterative clustering (SLIC) algorithm [21]. This classification uses only basic image intensity properties (e. g., minimal, maximal, average intensity, and cuts through the histogram) of each supervoxel and its neighbours. It is powerful in detection of objects that have distinct intensity features, for example, fluorescently labelled cells or organelles. 

A graph-based semiautomatic segmentation in MIB starts by clustering of voxels to supervoxels [21] that are assigned to vertices of a graph, where the edges connecting vertices are defined by the difference between average intensities of the corresponding supervoxels. Once the user interactively labels the areas belonging to the target and background, MIB uses the maxflox/min-cut algorithm [30] to perform the segmentation. For objects that have distinct boundaries, the watershed transformation [31] is extremely efficient. The marker-controlled watershed segmentation can be applied to detect both 2-D and 3-D objects. In practice, labelling the object(s) and background on a single slice allows segmentation of membrane-enclosed organelles in 3-D, as demonstrated for segmentation of the nucleus (Fig 3B and S2 Video) [14]. Objects that do not have clear boundaries and therefore would not qualify for segmentation using watershed can be preprocessed with the gradient filter to generate the required boundaries. Watershed can also be used to separate merged 2-D and 3-D objects even in anisotropic datasets [32], as exemplified by the separation of lipid droplets (LDs) (Fig 3C ). For processing of large volumes, the chop tool can be used to divide the initial volume into smaller datasets for parallel segmentation on multiple workstations. Import of the chopped images then automatically assembles the segmented blocks together. 

In automated segmentation, an essential part of the workflow is the estimation of accuracy of the applied method. As MIB was designed for segmentation of various objects coming from multiple imaging modalities, finding suitable parameters for accuracy metrics would be challenging, and the current version does not offer any. Therefore, MIB is most suitable for the processing of moderately large datasets, where the workflow can still be interactive and a researcher approves each step iteratively. [8]. In other cases, the models may be saved in the TIF format for free packages such as BioImageXD [5], Drishti [36], or Vaa3D (Fig 4 ) [37].
section_name: Segmentation of Large Datasets Is Challenging and Requires a Special Set of Tools

section_content: For full utilization of the acquired data, the visualization should be complemented with numerical analysis. MIB provides several tools for numerical analysis of shapes (e. g., size, eccentricity, perimeter, and orientation) and intensities (e. g., minimum, maximum, and average) of 2-D and 3-D objects. These calculated parameters can either be used for filtering of objects as a part of the segmentation protocol or be exported to MATLAB and Microsoft Excel A-D ) and commercial (E) software packages. (A) Trypanosoma brucei was chemically fixed and imaged with electron tomography (ET). Each Golgi cisternae (four shades of blue) was manually segmented using the brush tool, while ER and ER-derived vesicles (yellow) were segmented using a combination of the brush tool and shape interpolation. The resulting 3-D model was rendered directly in MIB. (B) A Huh-7 cell was high-pressure frozen and freeze-substituted, and a portion of the cell was subjected to ET [11]. ER (yellow) was segmented using the brush tool with shape interpolation and microtubules (magenta) using the line tracker tool. The resulting model was exported in the IMOD-compatible format and rendered in IMOD [6]. (C) A Huh-7 cell transiently expressing ssHRP-KDEL was cytochemically stained (dark precipitate) and imaged with a serial block-face scanning electron microscope (SB-EM) [11]. The ER network (yellow) was segmented semiautomatically using global black-and-white thresholding and further polished using quantification filtering [11]. The model was exported in the nearly raw raster data (NRRD) format and rendered in 3D Slicer [8]. (D) Mouse cochlea was perilymphatically fixed, and the sensory epithelium of the medial part of the cochlear duct was imaged with SB-EM [13]. Different cell types of the organ of Corti (inner hairs in green, outer hair cells in yellow, external rod in vermilion, internal rod in sapphire blue, and phalangeal part of the Deiters' cells in greyish blue) were segmented using local thresholding combined with shape interpolation and model rendered using 3D Slicer. (E) Stepwise segmentation workflow is needed to generate a 3-D model of a complex structure. A metaphase Huh-7 cell transiently expressing ssHRP-KDEL was cytochemically stained (dark precipitate in ER lumen) and imaged with SB-EM [10]. The modelling workflow for ER consists of 13 steps (S1 Table ). For the visualization, the model was exported in the AmiraMesh format and rendered in Amira.
section_name: 3-D Visualization and Quantification of Images and Models Are the Final Steps in Imaging

section_content: MIB is written with MATLAB, which is available for all common computer operating systems (Windows, Linux, and Mac OS). As a high-level scientific programming language, MATLAB's program code is easy to understand, and, thus, its use for implementing image-processing routines is fast and cost efficient, not only for initial development but also for any enhancements of MIB with additional features, which can be achieved because of MIB's open-source code. and background (red), which were then used to train the classifier and segment ER throughout the time-lapse video (right image). (B) The semiautomatic watershed segmentation was used for segmentation of nucleus in the sieve element of A. thaliana root imaged with SB-EM [14]. Assigning of just two labels (green for nucleus and vermilion for background) was sufficient to segment the complete nucleus in 3-D (light blue, image on the right). (C) The separation of the fused objects using the watershed segmentation. The human U251MG astrocytoma cells were loaded with oleic acid producing a large amount of lipid droplets (LDs) (left image) that tend to form clusters. LDs were segmented using marker-controlled watershed; however, because of close proximity, most of the LDs appear merged (the second image). The object separation mode of the watershed tool was used to separate individual LDs for quantitative analysis (third and fourth images). The 3-D models were rendered with 3D Slicer [8]. Scale bars: 2 μm. doi:10. 1371/journal. pbio. 1002340. g003 MATLAB has a large community of users who are developing and sharing algorithms using MATLAB Central File Exchange [38], an online library of over 20,000 user-contributed MATLAB files and toolboxes. Many of these submissions are dedicated to image processing and may easily be integrated into MIB, for example, as plug-ins. To support the development of plug-ins, MIB includes a description of its application programming interface that includes tutorials. MIB automatically detects all available plug-ins and initializes them during program start-up. For those researchers who are not familiar with MATLAB or who do not have a MATLAB license, we provide MIB as a stand-alone package (64-bit Windows and Mac OS) that can be freely downloaded and used on standard computers. 

MATLAB itself is mostly a collection of functions that may be difficult to use by novices. To improve the usability, MIB includes an intuitive graphical user interface with standard components such as a menu, toolbar, and panels in the main window (Fig 5A ). The toolbar and panels provide fast access to the most essential features described earlier, while the menu is used to access less frequently used actions. Some of the panels can be changed to adapt to the specific needs of a scientific project. MIB is designed to take the challenges of color-blind researchers into account, as we have selected a default color palette in which each color appears as a distinctive shade for color-blind users [39]. Alternatively, the default color palette can easily be replaced by one of the other color-blind-friendly ready assembled palettes [40] or each color can be individually selected. Each image-processing manipulation is logged, and the logs are stored with the data for future reference. To overcome accidental errors or to test most suitable tools and parameters during image processing, MIB has a flexible undo system that can erase recent changes made to the data or model. The number of undo steps for both 2-D and 3-D actions can be individually customized, thereby allowing optimized memory usage. MIB has a large variety of key shortcuts that improve segmentation performance and minimize unnecessary mouse movements. All details about the program usage are well documented and available from online tutorials on the MIB website (Fig 5B ) [17] or from a built-in help system accessed through the Help menu or via dedicated buttons in each panel and auxiliary windows. 

One of our aims during MIB development was to create a freely distributed open-source platform that would also allow a smooth cross talk with other image-processing programs. Depending on the program, the integration was accomplished with either direct data exchange or by data format compatibilities. The open microscopy environment (OMERO) image server [41] can be directly accessed to download data, whereas the direct data exchange [33] with Imaris and Fiji [4] allows a large collection of their plug-ins to be used without generating space-consuming intermediate files. In cases where the direct link is not possible, the compatibility is achieved through use of common file formats. We anticipate that MIB will become a useful tool not only for biological researchers but also for mathematicians as a platform for implementation of new methods for image segmentation developed in MATLAB. We will also keep on upgrading and extending MIB with each new imaging project.
section_name: MIB Is a Freely Distributed Open-Source Program with a User-Friendly Graphical User Interface

section_content: S1 Methods. A description of cell lines, constructs, specimen preparation, and imaging conditions. (DOCX) S1 Table. Segmentation of a densely packed mitotic cell requires utilization of various segmentation tools provided by MIB (Fig 2E ). First, the data was filtered using anisotropic diffusion filter (adapted from http://www. peterkovesi. com/matlabfns/) to eliminate the local noise while preserving the edges of the objects (step 1). As the ER network is extensive and makes contacts with nearly all other organelles, the removal of ER facilitates segmentation of all the other organelles later. Also in this example, the ER has a high contrast because of the luminal cytochemical staining making its segmentation feasible. In step 2, the morphological bottomhat filter was used to temporally remove all large objects (such as mitochondria, chromosomes, and LDs). The Frangi vessel enhancement filter (adapted from http://www. mathworks. com/ matlabcentral/fileexchange/24409-hessian-based-frangi-vesselness-filter) was then applied sequentially in the XY and ZX planes to segment ER tubules and sheet remnants (step 3: generation of the ER model). Next, the areas belonging to the ER in the anisotropically filtered image (from step 1) were replaced by the background color to generate degraded image without the ER (step 4). The resulting image was thresholded to select the dark singular objects such as LDs, peroxisomes, and lysosomes (step 5). To eliminate small objects, e. g., vesicles, the segmented areas were further smoothed using erosion followed by dilation in 3-D (step 6: generation of models for lysosomes, peroxisomes, and LDs). These areas were then replaced by the background color, similarly as in step 4 (step 7). As equatorially aligned chromosomes have contrast quite close to the mitochondria, global thresholding could not be used to discriminate them. Therefore, by using the brush tool and the shape interpolation, the central area of the cell was masked and thresholded to segment the chromosomes (step 8). The chromosomes were smoothed similarly as in step 6 (step 9: generation of the chromosome model). The chromosomal areas were replaced by the background color, and additional anisotropic diffusion filtering was applied (step 10). Segmentation of mitochondria was done by using the morphological image opening and thresholding (step 11), followed by smoothing and filtration of small objects (step 12: generation of the mitochondria model). The final model was assembled by combining all individual models and then visualized (step 13). (DOCX) S2 Table. List of all third-party tools and functions, including URL links.
section_name: Supporting Information

section_name: Acknowledgments

section_content: We would like to acknowledge the User Community of MATLAB-Central and the authors whose code was used during MIB development (S2 Table ). We thank all beta testers for critical comments and suggestions. We also thank Mervi Lindman and Antti Salminen ( University of Helsinki, Institute of Biotechnology, Electron Microscopy unit ) for excellent technical assistance with ET and SB-EM specimen preparation, Cris de Graffenried and Kathy Schmidt ( Max F. Perutz Laboratories, Vienna ) for providing the fixed T. brucei specimen, and Yuki Ohsaki and Elina Ikonen ( Faculty of Medicine, University of Helsinki ) for providing lipid-loaded U251MG cells.
section_name: Acknowledgments

section_content: This work was supported by Biocenter Finland, Biological Imaging Network (EJ), Academy of Finland (projects 131650 and 1287975; EJ) and the Integrative Life Science Doctoral Program of the University of Helsinki (MJ and DK). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.