# Experimenters' guide to colocalization studies: finding a way through indicators and quantifiers, in practice

Fabrice P. Cordelieres, Susanne Bolte

► To cite this version:

Fabrice P. Cordelieres, Susanne Bolte. Experimenters' guide to colocalization studies: finding a way through indicators and quantifiers, in practice. Jennifer C. Waters; Torsten Wittman. Quantitative Imaging in Cell Biology, 123, pp.395-408, 2014, Methods in Cell Biology, 978-0-12-420138-5. 10.1016/B978-0-12-420138-5.00021-5. hal-01536678

HAL Id: hal-01536678

https://hal.science/hal-01536678

Submitted on 22 Apr 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Experimenters’ guide
to colocalization studies: finding a
way through indicators and
quantifiers, in practice

Fabrice P. Cordelières*, Susanne Bolte†

*Bordeaux Imaging Center, UMS 3420 CNRS—Université Bordeaux Segalen—US4 INSERM,
Pôle d'imagerie photonique, Institut François Magendie, Bordeaux Cedex, France

†Sorbonne Universités—UPMC Univ Paris 06, Institut de Biologie Paris-Seine—CNRS FR 3631,
Cellular Imaging Facility, Paris Cedex, France

**Abstract**

Multicolor fluorescence microscopy helps to define the local interplay of subcellular compo-
nents in cell biological experiments. The analysis of spatial coincidence of two or more
markers is a first step in investigating the potential interactions of molecular actors. Coloca-
lization studies rely on image preprocessing and further analysis; however, they are limited by
optical resolution. Once those limitations are taken into account, characterization might be
performed. In this review, we discuss two types of parameters that are aimed at evaluating
colocalization, which are indicators and quantifiers. Indicators evaluate signal coincidence
over a predefined scale, while quantifiers provide an absolute measurement. As the image is
both a collection of intensities and a collection of objects, both approaches are applicable.
Most of the available image processing software include various colocalization options; how-
ever, guidance for the choice of the appropriate method is rarely proposed. In this review, we
provide the reader with a basic description of the available colocalization approaches, propos-
ing a guideline for their use, either alone or in combination.

INTRODUCTION

In cell biology, colocalization studies are performed to infer the coincidence of two
or more signals within the volume of a sample. This event might occur in two ways:
Proteins of interest are locally present on a structure; their concentrations are locally
linked. The diagnosis of coincidence is based on a representation of the biological
sample by a set of images. The latter is formed through a microscope and its
attached numerization device (detector). Since the resolution of an optical system
is diffraction-limited, colocalization studies may also be impaired by this limited op-
tical resolution. Thus, optical resolution is a referential that should always be clearly
stated. In cell biology, the conclusion of colocalization analysis is usually formulated
as “two markers are at the same location.” But the real conclusion of a colocalization
analysis should rather be stated as “knowing the current resolution, it cannot be ex-
cluded that the two markers are at the same location.” The term “colocalization” is
used abusively, as only a diagnosis of close vicinity can be stated with certitude.

The image formation process being the key point for setting the colocalization
referential, care should be taken in both, sample preparation and image acquisition.
The former implies preserving the spatial distribution of the components to analyze
and using appropriate markers, usually fluorescent, that are devoid of cross talk and
bleedthrough (for review, refer to Bolte & Cordelières, 2006). Sample mounting is
also a critical parameter. When dealing with 3D samples, the use of setting mounting
media may alter the thickness of cells and impair further analysis. As resolution
depends on both the composite refractive index (RI) of the crossed media and the
angular properties of the objective (numerical aperture, NA), oil and objectives
should be chosen with care. For instance, when working on subcellular structures,
the use of high-NA objectives is recommended, with immersion media that match
mounting medium RI. Finally, the imaging process is also crucial. As stated by
the Shannon–Nyquist–Whittaker theorem (Shannon, 1998), an element of resolution
should always be sampled at least twice, which means in practice that a subdiffrac-
tion object should be represented by 3 × 3(×3) pixels (voxels).

Several strategies might be used to unravel colocalization occurrence in a set of
images. One may want to have a global diagnosis, taking the image as a whole or
increasing the level of granularity and rather work on objects. In the former case,
two types of colocalization evaluation might be performed, extracting either indica-
tors or quantifiers. When working on objects, only quantifiers have yet been pro-
posed as colocalization evaluators. Two choices appear when working on a set of
images: the experimenter might analyze intensities, expecting some links between their distributions, or investigate signal coincidence. There is a whole set of parameters one has to choose from. In this review, we present the most commonly used colocalization approaches, giving insight in their domains of application, while guiding the user in the choices and combinations that are offered.

# 21.1 AN OVERVIEW OF COLOCALIZATION APPROACHES

## 21.1.1 TWO TYPES OF NUMERICAL VALUES TO EXTRACT: COLOCALIZATION INDICATORS AND COLOCALIZATION QUANTIFIERS

Colocalization analysis should always start by a close visual inspection of the image couple and a simple channel overlay might be the starting point. However, the experimenter should stay critical as the typical yellowish colocalization pattern might be achieved through an exaggerated image processing. This first step might be a final step in conditions where evident colocalization is present in the sample. However, care should be taken about cross talk and bleedthrough.

In the following lines, colocalization evaluation will be used as a generic term encompassing the use of both colocalization indicators and colocalization quantifiers. *Colocalization indicators* are numerical values that evaluate a degree of signal coincidence over a predefined scale, but are not suitable to calculate a precise amount of overlap. They are suitable for relative comparisons, without being usable for direct quantitative studies. *Colocalization quantifiers* provide the experimenter with an absolute value, quantifying the overlap of signals by using physical descriptors such as area or volume or by measuring distances between defined coordinates within the structures.

## 21.1.2 TWO WAYS TO WORK ON COLOCALIZATION EVALUATION: TAKING THE IMAGE AS A WHOLE AND SPLITTING IT INTO OBJECTS

### 21.1.2.1 Working on image intensities

#### 21.1.2.1.1 Legacy colocalization indicators and visualization methods
How should one evaluate the dependency between signals acquired for colocalization evaluation? Most basically, a linear relationship among the intensities of both channels can be assumed. Manders, Stap, Brakenhoff, van Driel, and Aten (1992) transposed a classical visualization of flow cytometry data by a scatter plot to con-focal images: The intensity of a given pixel in the green image is used as the x-co-ordinate of the scatter plot and the intensity of the corresponding pixel in the red image as the y-coordinate. The composed figure takes the shape of a dot cloud, its form unraveling the colocalization state. In case of unambiguous colocalization, the shape might be approximated as a line centered over the cloud. The scatter plot might also display two separated populations of dots, close to each axis. In this case, two conclusions might be drawn: If these two clouds have a line shape, they might result from either cross excitation or cross detection of signals. Alternatively, less
structured clouds hint for nonlinked signals. Finally, the scatter plot might also display additional dot arrangements: combinations of the former, multiple linear dependencies, and/or single/multiple nonlinear dependencies. Those arrangements increase the difficulty of the interpretation process. Under such circumstances, one should remember that the scatter plot representation omits spatial information. Therefore, it is crucial to go back to the overlay image, trying to identify regions where each single dependency (linear/nonlinear) occurs and redo the analysis on identified regions of interest (ROI).

To characterize the linear dependency of two signals, Manders et al. introduced the use of the Pearson correlation coefficient (PC) (Pearson, 1901), as a colocalization indicator in 1992 (Manders et al., 1992) (see Table 21.1). The PC characterizes the linear relationship between two variables, providing a value of 1 in case of complete positive correlation (colocalization), -1 for negative correlation (exclusion), and zero when no correlation is found. Although this scale seems comfortable, the extent of each extreme value remains to be determined: which limit should be put between total positive correlation (PC = 1) and no correlation (PC = 0)? Midrange values will remain hard to interpret, when taken alone. Therefore, a scatter plot should always accompany PC. As previously shown (Bolte & Cordelières, 2006), a midrange value might correspond to either no correlation or correlation corrupted by noise. Depending on the shape of the dot clouds, the experimenter will be able to infer the former or the latter. In cases where several experimental conditions are to be compared, PC might be used to show a difference of colocalization. However, the PC remains an indicator and may not be used to express the amount of colocalization within a sample.

<table><caption>**Table 21.1** Colocalization Evaluators</caption><thead><tr><th>Method</th><th>Formula</th></tr></thead><tbody><tr><td>Pearson coefficient (indicator)</td><td>PC = <sup>&Sigma;<sub>i</sub> (A<sub>i</sub> - a) &times; (B<sub>i</sub> - b)</sup>&frasl;<sub>&radic;(&Sigma;<sub>i</sub>(A<sub>i</sub> - a)<sup>2</sup> &times; &Sigma;<sub>i</sub>(B<sub>i</sub> - b)<sup>2</sup>)</sub></td></tr><tr><td>Overlap coefficient (indicator)</td><td>Overlap = <sup>&Sigma;<sub>i</sub>A<sub>i</sub> &times; B<sub>i</sub></sup>&frasl;<sub>&radic;(&Sigma;<sub>i</sub>(A<sub>i</sub>)<sup>2</sup> &times; &Sigma;<sub>i</sub>(B<sub>i</sub>)<sup>2</sup>)</sub></td></tr><tr><td>k<sub>1</sub> and k<sub>2</sub> coefficients (indicators)</td><td>k<sub>1</sub> = <sup>&Sigma;<sub>i</sub>A<sub>i</sub> &times; B<sub>i</sub></sup>&frasl;<sub>&radic;(&Sigma;<sub>i</sub>A<sub>i</sub><sup>2</sup>)</sub>, k<sub>2</sub> = <sup>&Sigma;<sub>i</sub>A<sub>i</sub> &times; B<sub>i</sub></sup>&frasl;<sub>&radic;(&Sigma;<sub>i</sub>B<sub>i</sub><sup>2</sup>)</sub></td></tr><tr><td>M<sub>1</sub> and M<sub>2</sub> coefficients (quantifiers)</td><td>M<sub>1</sub> = <sup>&Sigma;<sub>i</sub>A<sub>i</sub>,coloc</sup>&frasl;<sub>&Sigma;<sub>i</sub>A<sub>i</sub></sub>, M<sub>2</sub> = <sup>&Sigma;<sub>i</sub>B<sub>i</sub>,coloc</sup>&frasl;<sub>&Sigma;<sub>i</sub>B<sub>i</sub></sub></td></tr><tr><td>M<sub>1</sub> and M<sub>2</sub> thresholded coefficients (quantifiers)</td><td>tM<sub>1</sub> = <sup>&Sigma;<sub>i</sub>A<sub>i</sub>,thr<sub>A</sub></sup>&frasl;<sub>&Sigma;<sub>i</sub>A<sub>i</sub></sub>, tM<sub>2</sub> = <sup>&Sigma;<sub>i</sub>B<sub>i</sub>,thr<sub>B</sub></sup>&frasl;<sub>&Sigma;<sub>i</sub>B<sub>i</sub></sub></td></tr></tbody></table>

A<sub>i</sub> and B<sub>i</sub>: intensities of the pixel i on images A and B, respectively.

a and b: average intensities of images A and B, respectively.

A<sub>max</sub> and B<sub>max</sub>: maximum intensities of images A and B, respectively.

A<sub>i, coloc</sub>, B<sub>i, coloc</sub>: A<sub>i, coloc</sub> takes the value A<sub>i</sub> if B<sub>i</sub> > 0 and B<sub>i, coloc</sub> takes the value B<sub>i</sub> if A<sub>i</sub> > 0.

A<sub>i, thr<sub>A</sub></sub>, B<sub>i, thr<sub>B</sub></sub>: A<sub>i, thr<sub>A</sub></sub> takes the value A<sub>i</sub> if B<sub>i</sub> > thr<sub>B</sub> and B<sub>i, thr<sub>B</sub></sub> takes the value B<sub>i</sub> if A<sub>i</sub> > thr<sub>A</sub>.
Attempts have been made to ease the interpretation of PC. The overlap coefficient (see Table 21.1) has been designed by taking the mean intensity values out of the equation (Manders, Verbeek, & Ate, 1993). This results in a numerical value in the 0–1 range, where 0 corresponds to negative correlation, 0.5 to no correlation, and 1 to full correlation. This shift brings confusion in the interpretation. The experimenter may abusively refer to this [0, 1] as a percentage of colocalization, which, of course, is not true.

Looking more closely to the overlap coefficient, one might distinguish two parts within the expression: one encountering for channel A, the other for channel B. Subsequently, two colocalization indicators were built, $k_1$ and $k_2$ (Manders et al., 1993, see Table 21.1). When a perfect correlation is present in the image couple, $k_1$ tends to a value close to the normalized stoichiometry (value in the 0–1 range) depending on the slope of the average line in the scatter plot, and $k_2$ to $1/k_1$. A noise- and/or background-corrupted channel will result in an increase in the denominator and therefore the k coefficient will decrease (Cordelières & Bolte, 2008). The evaluation of the distortion between the k coefficient and its expected value when considering plain correlation might be a trail to pursue, helping to infer colocalization diagnosis in the case of noisy signals.

Correlation-based methods require a certain degree of dependency between signal intensity. Colocalization might be a more subtle phenomenon: two proteins might appear in a spatially correlated manner, not implying specific stoichiometries. To assess the superimposition of distribution patterns, Manders et al. (1993) introduced $M_1$ and $M_2$ coefficients, nowadays named after the author (see Table 21.1). Calculating $M_1$ implies calculating the ratio of intensities, taking summed intensities of channel A that find a non-null counterpart in channel A, and dividing it by the total intensity of pixels in A. $M_2$ is obtained by calculating the other way round. $M_1$ and $M_2$ express the percentage of fluorescence having a counterpart in the other channel. Manders coefficients are therefore to be considered as colocalization quantifiers. Those coefficients were originally defined for use with confocal images, based on photomultiplier detectors, for which a detection offset has to be set. Technical innovations on detectors and imaging modalities have opened a wide range of applications where the minimum intensity on an image might not be null. As a consequence, the zero value is not always the appropriate value to distinguish non-pertinent from pertinent signal. Two new parameters have been derived from the Manders coefficient, the thresholded Manders coefficients ($tM_1$ and $tM_2$; see Table 21.1), belonging to the class of colocalization quantifiers. Thresholded Manders coefficients consider intensity values above a user-defined value.

### 21.1.2.1.2 Legacy colocalization indicators and visualization methods, revisited

Interpreting the scatter plot might seem easy when dealing with a well-defined colocalization phenomenon. The experimenter might find easily the contribution of background/noise as a shapeless cloud surrounding the scatter plot origin and cross talk/bleedthrough as a rather linearly shaped dot cloud located near the axes. It is however difficult to eliminate those unwanted contributions.
Simple thresholding of the images is one option when using Manders coefficients however, it is user-dependent. Costes et al. (2004) proposed an automated thresholding method. It consists in iteratively lowering the threshold, starting at highest intensity values for both channels, and calculating PC only taking into account values below these thresholds. When a null or negative value of PC is found, that is when the uncorrelated population has been delineated (background/noise), and the process is stopped. Pertinent coefficients are then calculated from the intensities lying above both thresholds. As background, cross talk, and bleedthrough are hardly present as a well-defined line parallel to the axis, this procedure will only remove them partially.

Gavrilovic and Wählby (2009) proposed an alternative to the scatter plot, in the form of a spectral angle representation. This histogram carries angles taken within the 0° to 90° range on the x-axis. Each couple of intensities ($A_i, B_i$) contributes to the histogram through the angle atan($B_i/A_i$), assuming that channel A's intensities are plotted on the x-axis of the scatter plot, and a magnitude depending on the distance of the corresponding point to the origin. Corrections to the spectral angle histogram are made to account for the discrete nature of image intensities (please refer to the original paper for more details). From this representation, three populations of pixels might be extracted: two corresponding to cross talk/bleedthrough and the third to colocalization, from which legacy indicators and quantifiers might be calculated. This method has the advantage of defining populations in the scatter plot based on angles, rather than intensity thresholds. It is also a method of choice when several stoichiometries link channel intensities as multiple threshold angles might be extracted out of the spectral angle histogram.

An additional issue relies on the interpretation of indicator/quantifier values, especially when dealing with PC coefficient. Although this process is rather simple when comparing several experimental conditions, care should be taken if only one situation is to be evaluated. Two descriptive methods might be used to help interpreting data: Van Steensel's method and Costes' randomization.

Van Steensel et al. (1996) proposed the use of a cross correlation function in colocalization studies. It consists in calculating PC while operating a pixel shift to one of the two channels. As a consequence, colocalization contribution should disappear when the pixel shift is applied: the higher the pixel shift, the lower the PC. Plotting the PC as a function of the pixel shift results in a bell-like curve when colocalization occurs, a rather flat line when no colocalization is present, and a pit in case of signal exclusion. Van Steensel's method is a graphical way to assess colocalization for low PC values (close to zero). It also provides a mean to quantify and further correct for chromatic aberration, since in such case, the maximum of the bell-shaped curve will not fit the zero pixel shift.

Costes' randomization (Costes et al., 2004) is a process based on the comparison of the PC of original image couples and the PC between a randomized image of channel A and the original image of channel B. Randomization of the image of channel A is done by shuffling pixel positions. Repeating the process allows collecting a distribution of PCs that encounters for colocalization events due to hazard. Comparing
the original PC value, obtained from nonrandomized channels, to this distribution
helps to position the current situation to a random colocalization event.

Finally, in spite of those methods, some experimental situations are simply not
compatible with the use of PC. PC requires the two signals to be linearly linked,
which may not always be the case. When a monotonic relation links intensities, either
increasing or decreasing, an additional *colocalization indicator* might be used:
Spearman’s coefficient (SC) (French, Mills, Swarup, Bennett, & Pridmore, 2008;
Spearman, 1904). Rather than working on raw intensities, a classification is first
made. To exemplify, let’s take a group of intensities: {0, 1, 2, 4, 8, and 16}. Grouping
intensities into classes will convert the former array of values to the following: {0, 1,
2, 3, 4, and 5}. While the original data were unevenly distributed, the transformed
data are well ordered, all classes being evenly spaced. This process results in a lin-
earization of the nonlinear dataset. The PC calculated accordingly is called SC and its
value is in the [-1, 1] range, while its interpretation is similar to the PC.

21.1.2.1.3 Which strategy to adopt?

As we have seen, when dealing with pixel intensity-based colocalization methods, a
plethora of tools exist. While most of the experimenters would usually only pick one
tool, several of them can be combined to describe the signal interplay.

First, as a preliminary step, colocalization study should always start by overlay-
ing channels. The resultant image gives a first insight on signal coincidence. It also
helps defining ROI, to which analysis might be restricted later on. Chromatic
aberration-free images being a prerequisite to proper colocalization evaluation, over-
lay is also a tool to detect such phenomenon. Van Steensel’s cross correlation ap-
proach helps quantifying it, as its extremum’s position (maximum in case of
colocalization and minimum in case of exclusion) generally corresponds to chro-
matic shift.

Once images have been corrected for chromatic aberration, a scatter plot should
be built. Its analysis will be useful in defining the contribution of cross talk/bleed-
through and background/noise to pertinent signal. Two strategies are to be applied:
The first consists in removing both contributions by applying thresholds to channels
A and B. This step might be achieved either manually or using automated Costes’
threshold. Another approach is to perform spectral angle representation and use
the histogram segmentation method proposed by Gavrilovic and Wählby (2009).

Signals being now exempt of parasite contributions, the colocalization evaluation
process can be performed. Two kinds of tests might be performed: retrieval of colo-
calization indicators and quantifiers. The former are to be used to prove a monotonic
relationship between intensities of both channels and the later to quantify the amount
of overlapping signals.

Being the oldest used indicator, the PC is usually the first to be calculated. This is
only appropriate when a linear relationship links intensities of both channels. Alter-
natively, in case the connection is not linear but still monotonic, the use of SC should
be preferred. In both cases, care should be taken with their interpretation. While low,
null, and high values permit straightforward conclusions (exclusion, no correlation,
and colocalization, respectively), midrange values are ambiguous. Going back to the scatter plot helps, as the spread of cloud distribution is an indication of noise-corrupted signal. This can easily be assessed by calculating the two components of the overlap coefficient: *k*₁ and *k*₂. Both values should drift from the ideal 1/a and a values, being the slope of the regression line describing best the dot cloud. Alternatively, one may identify more than one dot cloud on the scatter plot, which would lead to similar issues on PC/SP, *k*₁ and *k*₂. This situation might be circumvented by isolating individual dot populations and performing the former tests on individual groups of intensities.

Once those observations and partitioning have been performed, the experimenter may want to investigate colocalization indicator relevance in the current experimental situation. When comparing several conditions, PC/SP for each might be compared using regular statistical tests (see the recent review by McDonald & Dunn, 2013). When only one experimental situation is to be diagnosed for colocalization, the task might seem harder to pursue. Using Costes' randomization is then the best available choice, as a reference dataset is generated from the actual content of both input images, one of them being shuffled. When performing this approach, care should be taken for two parameters: the size of the ROI, relative to the extent of pertinent signal, and the number of randomizations to be performed. Performing the analysis on an ROI where the ratio between the number of pixels belonging to the objects is low, as compared to the number of nonobject pixels, will lead to a distribution of PC where values are expected to be centered around 0 and the full width at half maximum (FWHM) is rather low. In this situation, the PC calculated on the original couple of images is likely to be well outside this distribution. To the opposite, an ROI devoted to nonobject pixels will give a larger FWHM, intensities within objects likely being correlated. Under those circumstances, the original PC will fall into the distribution, and erroneous conclusion of "random" colocalization might be drawn. Therefore, it is advisable to consider an ROI where approximately 50% of the pixels belong to structures, when performing such an approach. The number of randomization rounds is also of main concern. A low number of randomization will end up in a sharp bell-shaped curve, centered on the most probable PC encountered in random colocalization situations, omitting the less occurring events, located on both extreme sides of the distribution. In their original paper, Costes et al. (2004) used 200 rounds of randomization of their tests. The final output of Costes' method is a *P*-value (to be differentiated from the statistical test output, *p*-value). It corresponds to the area under the distribution of PC from randomized images, starting from its minimum, until the intercept with the original PC value. Colocalization is considered true when the *P*-value is above 95%, meaning that for 200 rounds of randomization, the original PC value or higher is found in less than 10 cases.

Among legacy colocalization quantifiers lay the Manders coefficients. *M*₁ and *M*₂ should rather be used to estimate the amount of colocalization between two channels, meaning when proper quantitative values have to be extracted. The first step here is to correct for potential chromatic aberration using Van Steensel's method. To get rid of cross talk/bleedthrough and noise, either manual thresholding or Costes' method
might be applied, together with spectral angle histogram representation, as previ-
ously stated. Alternatively, image processing might be applied to isolate the pertinent
objects from the image. Denoising, wavelet transformation, texture-based analysis,
etc., might be performed to segment the image. Once information-containing regions
have been extracted, the analysis is to be performed on the original image. Binary
masks may be generated from the processed images and applied on the original im-
ages. Manders et al. had not proposed these kinds of processing steps in their original
paper but rather applied a simple threshold before calculating tM₁ and tM₂ (M₁ and
M₂ in their thresholded form). Once clean out has been performed on both images,
both coefficients are calculated on remaining pixel intensities. While Manders coef-
ficients give a direct readout of the amount of signal engaged in the colocalization
process, this method requests a precocious determination of the thresholds, partition-
ing the intensities belonging to objects and the ones lying in the background. When
comparing several sets of conditions, this process should be first applied to the im-
ages where colocalization is supposed to be the most present, assuming all sets of
images have been acquired under the same conditions. Once determined, those pa-
rameters should be left untouched and applied to all remaining images within the
dataset.

21.1.2.2 Working on objects

Manders coefficients are colocalization quantifiers calculated using only pixels be-
longing to objects. While this approach is object-based, the calculations are made
using a different level of granularity, pixel-wise. Those quantifiers are therefore
global, and more advanced diagnostic might be performed using a lower level of
granularity, object-wise.

Two strategies might be used when dealing with problematic colocalization: con-
sidering the image couple as a container for intensity couples, which may or may not
be spatially related, and, alternatively, as a container for a population of object cou-
ples. The latter opens the possibility to evaluate the interplay between signals locally.
The methods that will be applied not only are restricted to colocalization evaluation
but can also be used to investigate parameters such as proximity, apposition, and
overlap in an object-to-object way.

21.1.2.2.1 Grouping pixels into objects: Image segmentation

Images are composed of discrete elements, resulting from the sampling of the orig-
inal scene, namely, the biological sample, seen through a detector, and by the use of
fluorescent molecules. Each pixel carries an intensity that corresponds to the local
concentration of the probes. Fluorescence emission and its digitalization are
noise-creating processes, and the experimenter has to deal with those parasite con-
tributions to the signal. Generating an image through a microscope is also not a faith-
ful process. The view of a biological sample presented by an image is dependent on
the instrument response function that takes the form of a point-spread function (PSF).
The image of a subresolution object will therefore be a 3D hourglass shape, whose
minimum width is the actual resolution of the optical system. Performing a proper
isolation of the objects out of the image will imply taking into account all those con-
tributions to the image formation process.

Two types of noise impair the image, Poisson and Gaussian noise, resulting from
both the stochastic nature of the fluorescence process and the electronics used to col-
lect photons and to digitize the signal. A good object delineation might be hard to
achieve, when dealing with low signal-to-noise ratio (SNR) images, for instance,
in video-microscopy experiments. Several denoising tools have been released that
use both local and temporal estimations of noise to enhance the original signal
(Boulanger et al., 2010; Luisier, Vonesch, Blu, & Unser, 2010). Additionally, image
restoration algorithms may be applied to account for and revert the instrument func-
tion response. Deconvolution is a process aimed at bringing the signal spread out
back to its origin, using the PSF. It will enhance the SNR and ease the object delin-
eation process (for review, see Sibarita, 2005). Recent work by Paul, Cardinale, and
Sbalzarini (2013) proposed to do both deconvolution and image segmentation con-
comitantly, a strategy that may also be applied to images before object-based
colocalization.

Once image enhancement has been performed, easy object delineation steps
might be achieved. Wavelet transformation, among other methods (for a comparative
review, see Ruusuvuori et al., 2010), can be used to isolate objects of specific shape
and size.

The final steps of object isolation are achieved through thresholding and connex-
ity analysis of the resultant image. Pixels are therefore separated into two popula-
tions, based on their intensities, namely, the nonobject and the object pixels.
Then, classification of spatially juxtaposed pixels is done to determine the objects.

Those preprocessing steps impair the original signal and thus have to be per-
formed with care. However, they are crucial for faithful object delineation. Knowing
the nature of analysis to be performed on object geometry or intensity, the resulting
images will have to be used raw or as masks, through which original intensities will
be observed, respectively.

21.1.2.2.2 Colocalization quantifiers based on object overlaps

Connexity analysis is a means to combine pixels carrying relevant intensities (i.e., which lie above the threshold) into objects and to retrieve pertinent information from them subsequently. This information may contain the measurement of the area in two dimensions, volume in three dimensions, perimeter, and centers. Centers can be considered either geometric, only taking into account positions of pixels belonging to objects, or intensity-based (center of mass or barycenter). Using those descriptors, new *colocalization quantifiers* might be built.

Objects larger than the optical resolution will always appear as covering a surface larger than the expected 3 × 3 pixel area, when optimal sampling has been performed. A straightforward analysis is to measure the overlap surface existing between the two channels for each object. In a way, this approach is similar to the Manders coefficient method but differs from it as it relies on physical overlap rather than on intensity overlap. A Manders coefficient can also be calculated, object per object, determining the proportion of signal involved in colocalization.
The proposed approaches rely on local analysis as well as global analysis. They
allow investigating region per region coincidence of signals or, more generally,
physical overlap.

However, when dealing with size heterogeneity of structures between two chan-
nels, physical or intensity-wise overlaps are hard to interpret. For instance, taking
images presenting large objects in one channel and resolution-limited objects in
the other, the measures will be low in the former and close to 100% in the latter.
There are two ways to proceed: The first would consist in setting a threshold of over-
lap above which structures are considered as colocalized. Two ratios are then calcu-
lated, expressing the percentage of objects in channel A that overlap, physically or
intensity-wise, with an object from channel B, and vice versa. In this matter, the
thresholding step relies only on user input, which might be impaired by a priori
knowledge of the expected results. A second option consists in plotting a histogram
of the degrees of overlap, to help tuning the limit above which two objects are con-
sidered as colocalized.

Alternatively, resolution-limited objects from channel A might be reduced to
their centers. In homogeneously labeled structures, both centers of mass and geomet-
ric centers should fall on the same pixel. Colocalization quantifiers are then
calculated by counting the number of centers from channel A that fall onto the
surface/volume of objects from channel B. This approach was proposed by
Lachmanovich et al. (2003). The reduction of an object to its center is only allowed
in cases where its dimensions are close to the optical resolution. This is also true for
the barycenter, when intensities are evenly distributed within the object. Therefore,
using one or the other parameter will depend on both the area of the object and the
intensity distribution within the object.

21.1.2.2.3 Colocalization quantifiers based on object distances

Previously described methods were based on the search for an overlap between rel-
evant object descriptors. However, due to the combination of the limit of resolution
and the sampling performed by the imaging process, two point-shaped structures will
appear on blocks of 3 × 3 pixels. In this situation, colocalizing objects may fall onto
pixels as distant as 3 pixels. This imprecision of localization may result in underes-
timation of colocalization when only looking for a true, precise overlap. The alter-
native might be to work rather on distances than on coincidences (Cordelières &
Bolte, 2008; Lachmanovich et al., 2003).

Knowing all parameters that can be extracted out of objects, several choices are
offered to quantify colocalization. Distances might be calculated between the enve-
lopes of objects from both channels, considering that colocalization occurs when two
surfaces/volumes are distant from less than the optical resolution. The same rule may
be applied to distances between centers of objects from both channels. Direct quan-
tifications are achieved by counting the number of colocalization events and dividing
it by the overall number of objects for one channel.

More subtle characterization may be performed: A histogram of all distances can
be plotted. The analysis of this representation may help in distinguishing several
populations. More than a binary analysis, distance histograms may help to reveal
colocalization (distances below the optical resolution), apposition (distances higher but close to optical resolution), or non-colocalization events. When considering an experimental situation where colocalization varies upon the addition of a drug, and/or as a function of time, a comparison of the histogram helps directly to visualize the evolution of the three aforementioned populations.

#### 21.1.2.2.4 Which strategy to adopt?

There are a plethora of parameters that can be extracted out of objects, after the segmentation process, and this might impede the choice of the appropriate colocalization method. It has to be made depending on three factors: the size of the structures, their shapes, and their relative distribution between channels A and B.

When the objects have a size close to the optical resolution, the easiest way to work is to use their centers. The small number of pixels makes the use of both types of centers (of mass and geometric) equivalent. Objects larger than the optical resolution have to be used as a whole: Their surface, if working in 2D, or volume, if working in 3D, has to be used. Colocalization methods will then depend on the parameters extracted from both channels. In the center-center case, evaluation has to be done on distances. In the center-area/volume case, the experimenter should rather investigate coincidence between the latter and the former. Finally, in the area/volume-area/volume, measure of the overlap may be performed. This measure might be performed either geometry-based or intensity-based. The choice should be performed in view of the phenomenon to study and the signal distribution within objects. When the intensity is homogeneous within the structure, both methods will end up with similar results. Combining measures of the overlap of surfaces/volumes and the coincidence of signal might be used to infer the nonhomogeneous distribution of the markers within the structures.

Additional analysis is accessible from those parameters. For instance, when two vesicular markers are to be analyzed, one being the content and the other an outer labeling, apposition of the two signals may be evaluated in a two-step process. First, using the center-center approach, the experimenter will measure distances close to the optical resolution. As a second step, the overlap measurements between areas/volumes will display a value close to zero. This is a typically expected result when working on round structures, surrounded by a donut-shaped signal. When the structures are rather elongated than round-shaped, for instance, when working on pre-/postsynaptic markers, apposition might be revealed by considering the minimum distances between envelopes of objects from both channels.

## CONCLUSION

Colocalization studies are usually performed using the most widely available methods, namely, by calculating the PC or Manders coefficient. However, these are not generic methods, and both have domains of application that should be respected. Alternative methods should be investigated, as they either are more
appropriate or might give insights to the legacy indicator and quantifier interpreta-
tion. All the tools that were made available were designed to answer a specific bi-
ological problem. A more general approach should therefore consist in first verifying
that published tools are applicable to the experimenter’s problem and alternatively
trying to design appropriate indicators/quantifiers relevant to the field of
investigation.

The development of superresolution microscopy techniques is now breaking
down the resolution limit. Previously published colocalization studies might soon
have to be revisited in light of the improved performances of optical systems. The
output of structured illumination and STED microscopies is still images, where leg-
acy methods might be applied. However, pointillist techniques generate not only im-
ages but also localization information that is characterized by uncertainty of
measure, making the data uneven resolution-wise. Using generic methods is the most
straightforward strategy, which should be applied to images reconstructed from lo-
calization data. Raw data have to be analyzed as distance maps, taking into account
the localization imprecision. In this matter, up to now, no real effort has been made in
trying to find good indicators/quantifiers. This definitely opens new fields of inves-
tigation for colocalization specialist.

REFERENCES

Bolte, S., & Cordelières, F. P. (2006). A guided tour into subcellular colocalization analysis in light microscopy. *Journal of Microscopy*, 224(3), 213-232.

Boulanger, J., Kervrann, C., Bouthemy, P., Elbau, P., Sibarita, J.-B., & Salamero, J. (2010). Patch-based nonlocal functional for denoising fluorescence microscopy image sequences. *IEEE Transactions on Medical Imaging*, 29(2), 442–454.

Cordelières, F. P., & Bolte, S. (2008). JACoP v2.0 : Improving the user experience with co-localization studies. In *ImageJ User & Developer Conference* (pp. 174–181).

Costes, S. V., Daelemans, D., Cho, E. H., Dobbin, Z., Pavlakis, G., & Lockett, S. (2004). Automatic and quantitative measurement of protein–protein colocalization in live cells. *Biophysical Journal*, 86(6), 3993–4003.

French, A. P., Mills, S., Swarup, R., Bennett, M. J., & Pridmore, T. P. (2008). Colocalization of fluorescent markers in confocal microscope images of plant cells. *Nature Protocols*, 3(4), 619–628.

Gavrilovic, M., & Wählby, C. (2009). Quantification of colocalization and cross-talk based on spectral angles. *Journal of Microscopy*, 234(3), 311–324.

Lachmanovich, E., Shvartsman, D. E., Malka, Y., Botvin, C., Henis, Y. I., & Weiss, A. M. (2003). Co-localization analysis of complex formation among membrane proteins by computerized fluorescence microscopy: Application to immunofluorescence co-patching studies. *Journal of Microscopy*, 212(Pt. 2), 122–131.

Luisier, F., Vonesch, C., Blu, T., & Unser, M. (2010). Fast interscale wavelet denoising of Poisson-corrupted images. *Signal Processing*, 90(2), 415–427.

Manders, E. M., Stap, J., Brakenhoff, G. J., van Driel, R., & Aten, J. A. (1992). Dynamics of three-dimensional replication patterns during the S-phase, analysed by double labelling of DNA and confocal microscopy. *Journal of Cell Science*, 103, 857–862.
Manders, E. M. M., Verbeek, F. J., & Ate, J. A. (1993). Measurement of co-localisation of objects in dual-colour confocal images. *Journal of Microscopy*, **169**(3), 375-382.

McDonald, J. H., & Dunn, K. W. (2013). Statistical tests for measures of colocalization in biological microscopy. *Journal of Microscopy*, **252**(3), 295-302.

Paul, G., Cardinale, J., & Sbalzarini, I. F. (2013). Coupling image restoration and segmentation: A generalized linear model/Bregman perspective. *International Journal of Computer Vision*, **104**(1), 69-93.

Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. *Philosophical Magazine Series 6*, **2**(11), 559-572.

Ruusuvuori, P., Aijö, T., Chowdhury, S., Garmendia-Torres, C., Selinummi, J., Birbaumer, M., et al. (2010). Evaluation of methods for detection of fluorescence labeled subcellular objects in microscope images. *BMC Bioinformatics*, **11**, 248.

Shannon, C. E. (1998). Communication in the presence of noise. *Proceedings of the IEEE*, **86**(2), 447-457.

Sibarita, J. (2005). Deconvolution microscopy. *Advances in Biochemical Engineering/Bio-technology*, **95**, 201-243.

Spearman, C. (1904). The proof and measurement of association between two things. *The American Journal of Psychology*, **15**(1), 72-101.

Van Steensel, B., van Binnendijk, E., Hornsby, C., van der Voort, H., Krozowski, Z., de Kloet, E., et al. (1996). Partial colocalization of glucocorticoid and mineralocorticoid receptors in discrete compartments in nuclei of rat hippocampus neurons. *Journal of Cell Science*, **792**, 787-792.