Original: CP2Image: generating high-quality single-cell images using CellProfiler representations
Revised: CP2Image: Generating high-quality single-cell images using CellProfiler representations

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Single-cell high-throughput microscopy images contain key biological informationunderlying normal and pathological cellular processes.
Revised: Single-cell high-throughput microscopy images contain key biological information underlying normal and pathological cellular processes.

Original: Image-based analysis andprofiling are powerful and promising for extracting this information but are madedifficult due to substantial complexity and heterogeneity in cellular phenotype.
Revised: Image-based analysis and profiling are powerful and promising for extracting this information but are made difficult due to substantial complexity and heterogeneity in cellular phenotype.

Original: Hand-crafted methods and machine learning models are popular ways to extract cellimage information.
Revised: Hand-crafted methods and machine learning models are popular ways to extract cell image information.

Original: Representations extracted via machine learning models, whichoften exhibit good reconstruction performance, lack biological interpretability.
Revised: Representations extracted via machine learning models, which often exhibit good reconstruction performance, lack biological interpretability.

Original: Hand-crafted representations, on the contrary, have clear biological meaningsand thus are interpretable.
Revised: Hand-crafted representations, on the contrary, have clear biological meanings and thus are interpretable.

Original: Whether these hand-crafted representations can alsogenerate realistic images become an interesting question.
Revised: Whether these hand-crafted representations can also generate realistic images is not clear.

Original: In this paper, we proposea CellProfiler to image (CP2Image) model that can directly generate realistic cellimages from CellProfiler representations.
Revised: In this paper, we propose a CellProfiler to image (CP2Image) model that can directly generate realistic cell images from CellProfiler representations.

Original: We also demonstrate most biologicalinformation encoded in the CP representations are well-preserved in the generatingprocess.
Revised: We also demonstrate most biological information encoded in the CellProfiler representations is well-preserved in the generating process.

Original: This is the first time hand-crafted representations be shown to havegenerative ability and provide researchers  an intuitive way for their further analysis.
Revised: This is the first time hand-crafted representations be shown to have generative ability and provide researchers with an intuitive way for their further analysis.

Original:  Introduction
Revised:  Introduction

Original: With the advance in high-throughput microscopy, researchers can efficiently acquire a large numberof cell images under a variety of conditions [20].
Revised: With the advance in high-throughput microscopy, researchers can efficiently acquire a large number of cell images under a variety of conditions [20].

Original: These images contain rich biological informationon cell lineage, biomolecular pathway activation, and morphological cell characteristics, encouragingdiscoveries focused on visual manifestations of cellular state [13].
Revised: These images contain rich biological information on cell lineage, biomolecular pathway activation, and morphological cell characteristics, encouraging discoveries focused on visual manifestations of cellular state [13].

Original: Several approaches exist to either implicitly or explicitly capture or characterize the morphologyof cells [8, 9, 18, 19].
Revised: Several approaches exist to either implicitly or explicitly capture or characterize the morphology of cells [8, 9, 18, 19].

Original: Traditionally, cell morphology has been quantified and investigated throughthe use of hand-crafted features such as CellProfiler (CP) [3] or EBImage; however, machinelearning approaches have recently shown that it is possible to learn useful representations of cellmorphology [4] for image generation, phenotype characterization, and downstream prediction tasks[7].
Revised: Traditionally, cell morphology has been quantified and investigated through the use of hand-crafted features such as CellProfiler (CP) [3] or EBImage; however, machine learning approaches have recently shown that it is possible to learn useful representations of cell morphology [4] for image generation, phenotype characterization, and downstream prediction tasks [7].

Original: Learned representation often results in impressive performance when generating images of cells[4, 7] or discriminating, for example, the mechanism of action; however, the learned representations(i.e. , internal/latent states in a neural network model) are often difficult to understand from a biologicalperspective as they lack clear linkage with the known biological phenomenon.
Revised: Learned representation often results in impressive performance when generating images of cells [4, 7] or discriminating, for example, the mechanism of action (MoA); however, the learned representations (i.e., internal/latent states in a neural network model) are often difficult to understand from a biological perspective as they lack clear linkage with the known biological phenomenon.

Original: The use of machinelearning for learning representation of cells is therefore partly hindered by the lack of interpretabilitywhen it comes to the model’s internal, quantitative representation of the cell morphology.
Revised: The use of machine learning for learning representation of cells is therefore partly hindered by the lack of interpretability when it comes to the model’s internal, quantitative representation of the cell morphology.

Original: On theother hand, many of the machine learning models are so-called generative models with the advantagethat they allow scientists an intuitive visual understanding of the representation thus improving theinterpretability of the learned model for non-experts.
Revised: On the other hand, many of the machine learning models are so-called generative models with the advantage that they allow scientists an intuitive visual understanding of the representation, thus improving the interpretability of the learned model for non-experts.

Original: In contrast to learned representations, hand-crafted representations like CP have much higher inherentinterpretability due to the explicit link to, for example, geometric properties of the cell.
Revised: Prior work has shown that CP representations contain discriminative information about drug effects and functions, such as the MoA.

Original: However, such extracted feature representations lack the ability to generate high-quality images potentiallyimproving interpretability even further. In this work, we investigate whether CP representations cansupport the generation ofrealistic high-quality single-cell images from the existing CP feature vector.
Revised: Hence if we know or make changes to CP representations associated with a specific drug, it would be valuable to visualize the expected effects on cells, thereby facilitating a better understanding of the counterfactuals, out-of-distribution samples, and the overall causal structure between drugs and effects on cells. Furthermore, in rare cases, researchers only have access to CP representations without the corresponding images, thus leaving no opportunity for visual comparison across experiments.

Original: We propose a new model (CP2Image) that uses CPvector representations to generate realistic singlecell images.
Revised: In this work, we propose a new model (CP2Image) that uses CP representations to generate photorealistic images of cells.

Original: Our CP2Imagemodel consists of a convolutional neural network, called a generator,which takes CP representations(i.e. a feature vector) as input, and outputs single-cell images.
Revised: CP2Image consists of a convolutional neural network-based generator, which takes CP representations as input, and outputs cell images.

Original: Thebasic neural network is augmented with a discriminator network similar to the VAEplus model[7], which greatly enhances the image quality.
Revised: The model leverages a discriminator network, similar to the VAEplus model [7], to enhance the image quality.

Original: We show that our model generates high-quality cellimages from the CP representations.
Revised: We show that our model generates high-quality cell images from the CP representations.

Original: Measured by the Fréchet inception distance (FID) score, ourmodel outperforms the  variational autoencoder VAE [6], which is a benchmark method to generaterealistic images in representation learning.
Revised: Measured by the Fréchet inception distance (FID) score, our model outperforms the state-of-the-art variational autoencoder VAE [6].

Original: We also demonstrate how changing the values of input CP representations, e.g. nuclei-related dimensions, results ina corresponding phenotypic changein the generated images.
Revised: We also demonstrate how changing the values of input CP representations, such as nuclei-related dimensions, results in corresponding effects in the generated images.

Original: We believe this is the first successful attempt at generating photo-realisticsingle-cell images solely from CP features.
Revised: We believe this is the first successful attempt at generating photo-realistic single-cell images solely from CP representations.

Original:  The CP2Image model
Revised:  The CP2Image model

Original: To investigate whether interpretable CP representations can be used to generate cell images, wepropose a convolutional neural network model shown in Figure 1.
Revised: To investigate whether interpretable CP representations can be used to generate cell images, we propose a convolutional neural network model shown in Figure 1.

Original: The input is  CP representation z ∈53 R p , where p is the dimension of a  feature vector.
Revised: The input is a CP representation z ∈ R p , where p is the dimension of a CP feature vector.

Original: The output is the generated image X gen ∈ R d × d × 3 ,where d is the image size.
Revised: The output is the generated image X gen ∈ R d × d × 3 , where d is the image size.

Original: The model consists of four blocks, and each block contains a convolutionallayer, a leaky ReLu layer, and an upsampling layer.
Revised: The model consists of four blocks, and each block contains a convolutional layer, a leaky ReLu layer, and an upsampling layer.

Original: Batch normalisation layer is used throughout themodel.
Revised: Batch normalisation layers are used throughout the model.

Original: To enhance the generating performance, we add a  discriminator as the second component.
Revised: To enhance the generative performance, we add a GAN-style discriminator as the second component.

Original: The discriminator consists of five blocks, and each block contains a convolutional layer, a leaky layerand a pooling layer.
Revised: The discriminator consists of five blocks, and each block contains a convolutional layer, a leaky layer and a pooling layer.

Original: Batch normalisation layer is used throughout the network.
Revised: Batch normalisation layers is used throughout the network.

Original: The loss functionis calculated used the mean squared error (MSE) loss of generated image X gen and correspondingoriginal image
Revised: The loss function is calculated using the mean squared error (MSE) loss of generated image X gen and corresponding

Original:  X orig ∈ R d × d × 3 .
Revised: original image X orig ∈ R d × d × 3 .

Original: We first fed the original images and generated images to the discriminator.
Revised: We first fed the original images and generated images to the discriminator.

Original: For each layer i , weextracted the hidden representation, and calculated the MSE loss between original and generatedimages, which is denotedit as L Hi .
Revised: For each layer i , we extracted the hidden representation for that layer and calculated the MSE loss between original and generated images at that layer, which is denoted as L Hi .

Original: We added up the layers, define L H =
Revised: We added up the layers, defined as L H =

Original: (cid:80)
Revised: (cid:80)

Original: i γ i
Revised: i γ i

Original: L Hi .
Revised: L Hi .

Original: This lossencourages generated images  similar to  original images.
Revised: This loss encourages the generated images to be similar to the original images at all stages of the process.

Original: Then we fed another real image batch X real ∈ R d × d × 3 into the discriminator D , where X real is independent from X orig .
Revised: Then we fed another real image batch X real ∈ R d × d × 3 into the discriminator D , where X real is independent from X orig .

Original: For real images and generated images, we calculated the cross entropy loss which is used to train the discriminator to classify the real and generated images.
Revised: For real images and generated images, we calculated the cross entropy loss which is used to train the discriminator to classify the real and generated images.

Original: [Equation]
Revised: [Equation]

Original: We alternate the training between the generator and discriminator during every iteration.
Revised: We alternate the training between the generator and discriminator during every iteration.

Original: Whentraining the generator, we optimize the objective L MSE + L H .
Revised: When training the generator, we optimize the objective L MSE + L H .

Original: When training the discriminator, we optimize the objective L Dis .
Revised: When training the discriminator, we optimize the objective L Dis .

Original: To stabilize the discriminator training, we added spectralnormalization [11] in the discriminator network.
Revised: To stabilize the discriminator training, we added spectral normalization [11] in the discriminator network.

Original:  Experiment and results 
Revised:  Experiment and results 3.1 Experiment

Original: [Figure] 3.1 Experiment

Original: We use the benchmark BBBC021 dataset, which has been used for capturing cell responses to drugs inprevious research[7, 10, 16, 12, 1, 5, 2, 15, 14].
Revised: We use the benchmark BBBC021 dataset, which has been used for capturing cell responses to drugs in previous research [7, 10, 16, 12, 1, 5, 2, 15, 14].

Original: The dataset is comprised of images of cells that werefluorescently stained against markers for DNA, beta-tubulin and F-actin.
Revised: The dataset is comprised of images of cells that were fluorescently stained against markers for DNA, beta-tubulin and F-actin.

Original: A subset of the capturedimages are labelled as the distinct mechanism of action (MOA) to represent phenotypic cell effectsunder drug treatments[17, 10].
Revised: A subset of the captured images is labelled as the distinct MoA to represent phenotypic cell effects under drug treatments [17, 10].

Original: Figure 2 shows the overview of the experiment.
Revised: Figure 2 shows the overview of the experiment.

Original: During the experiment, CP representations areobtained by feeding original BBBC021 cell images into CellProfiler software [16].
Revised: During the experiment, CP representations are obtained by feeding original BBBC021 cell images into CellProfiler software [16].

Original: Every dimensionin the CP representations describes an aspect of biological phenotypic information, called  feature.
Revised: Every dimension in the CP representations describes an aspect of biological phenotypic information called a feature.

Original: Thelocation of the nucleus centre in CP representations is used as the centre of single-cell images duringsegmentation.
Revised: The location of the nucleus centre in CP representations is used as the centre of single-cell images during segmentation.

Original: To be comparable with other work [5, 7, 14], we only keep the MOA annotated subsetof 480k single-cell patches and their corresponding CP representations.
Revised: To be comparable with other work [5, 7, 14], we only keep the MoA annotated subset of 480k single-cell patches and their corresponding CP representations.

Original: Then we feed the single-cellimages to the CP2Image model to generate images and evaluate the generating performance bycomparing with VAE and VAEplus models.
Revised: Then we feed the single-cell images to the CP2Image model to generate images and evaluate the generating performance by comparing them with VAE and VAEplus models.

Original: To investigate which feature has been retained well inthe generating process, we measure CP on generated images and original images, then calculate thepairwise correlation between these two measurements for different features.
Revised: To investigate which feature has been retained well in the generating process, we measure CP on generated images and original images, then calculate the pairwise correlation between these two measurements for different features.

Original: 3.2 Results
Revised: 3.2 Results

Original: CP representations generate realistic cell images via CP2Image model: To explore the generativeperformance of CP2Image model, we generate images via CP2Image and compare them with imagesgenerated from VAE and VAEplus.
Revised: CP representations generate realistic cell images via CP2Image model: To explore the generative performance of CP2Image model, we generate images via CP2Image and compare them with images generated from VAE and VAEplus.

Original: Figure 3 shows four generated images from different models.
Revised: Figure 3 shows four generated images from different models.

Original: To quantify the generating performance, we evaluate the images by FID score, with smaller valuesindicating greater similarity between generated images and real images.
Revised: To quantify the generating performance, we evaluate the images by FID score, with smaller values indicating greater similarity between generated images and real images.

Original: Table 1 illustrates that
Revised: Table 3.2 illustrates that

Original: CP2Image model can generate less blurry cell images than VAE model, thus CP2Image has superiorgenerative ability compared to the basic VAE.
Revised: CP2Image model can generate less blurry cell images than VAE model, thus CP2Image has superior generative ability compared to the basic VAE.

Original: Even though CP2Image has a higher FID score than
Revised: Even though CP2Image has a higher FID score than

Original: VAEplus, it should be noted that CP2Image only uses CP representation to produce images, whereas
Revised: VAEplus, it should be noted that CP2Image only uses CP representation to produce images, whereas

Original: VAE and VAEplus both generate reconstructed images using original images.
Revised: VAE and VAEplus both generate reconstructed images using original images.

Original: Figure 3 also indicatesthat generated images via the CP2Image model have similar nuclei compared to the original images;however, they could not reconstruct the shape of whole cells very well, where the boundaries aresometimes blurry or ofan irregular shape.[Table]
Revised: Figure 3 also indicates that generated images via the CP2Image model have similar nuclei compared to the original images; however, they could not reconstruct the shape of whole cells very well, where the boundaries are sometimes blurry or of irregular shape.

Original: Correlating features from generated images with original images: To further explore if thegenerated images maintain biological phenotypic information, and which feature  has been retained inthe generating process, we calculate the pairwise correlation of CP measurement between originaland generated images.
Revised: Correlating features from generated images with original images: To further explore if the generated images maintain biological phenotypic information, and which feature column in CP representations has been retained in the generating process, we calculate the pairwise correlation of CP measurement between the original and generated images.

Original: Figure 4(a) shows the correlation coefficient value, and we can see thatmore than 30% of feature dimensions have a correlation coefficient larger  than 0.5.
Revised: Figure 4(a) shows the correlation coefficient value, and we can see that more than 30% of feature dimensions have a correlation coefficient larger [Table] than 0.5.

Original: These highlycorrelated features demonstrate a large amount of morphological information has been well-preservedduring generating process.
Revised: These highly correlated features demonstrate a large amount of morphological information has been well-preserved during generating process.

Original: The 20 features with the largest value are about the shape and intensityof the nucleus, while those features with the smallest values are almost features which describe thedistribution of intensity across cells.
Revised: The 20 features with the largest value are about the shape and intensity of the nucleus, while those features with the smallest values are almost features which describe the distribution of intensity over the single-cell image.

Original: Figure 4(b) shows the 4 dimensions with the highest correlationcoefficient values.
Revised: Figure 4(b) shows the 4 dimensions with the highest correlation coefficient values.

Original: Figure 4(c) demonstrates when we manipulate a single well-preserved interpretabledimension, the generated images show corresponding variation.
Revised: Figure 4(c) demonstrates when we manipulate a single well-preserved interpretable dimension, the generated images show corresponding variation.

Original: For example, nuclear area featuremeasures the number of pixels in nuclei.
Revised: For example, nuclear area feature measures the number of pixels in nuclei.

Original: When we increase its value and fix all other featuredimensions, from top to bottom, the size of the nucleus gradually increases.
Revised: When we increase its value and fix all other feature dimensions, from top to bottom, the size of the nucleus gradually increases.

Original: The nuclear orientationfeature measures the angle between the horizontal axis and major axis of the nuclei ellipse.
Revised: The nuclear orientation feature measures the angle between the horizontal axis and major axis of the nuclei ellipse.

Original: Increasingits value allows us to observe a clockwise orientation variation in the generated images.
Revised: Increasing its value allows us to observe a clockwise orientation variation in the generated images.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we present the CP2Image model to generate high-quality, single-cell images from CPrepresentations.
Revised: In this work we present the CP2Image model to generate high-quality, single-cell images from CP representations.

Original: We also show most biological information is well-preserved in the generated imagesand it is the first time that hand-crafted representations are shown to have the generative ability.
Revised: We also show most biological information is well-preserved in the generated images and it is the first time that hand-crafted representations are shown to have the generative ability.

Original: Ourwork has potential practical usage.
Revised: Our work has potential practical usage.

Original: For example, emerging image-based drug screens often reportonly conventional descriptive computer vision features rather than actual images.
Revised: For example, emerging image-based drug screens often report only conventional descriptive computer vision features rather than actual images.

Original: Having the abilityto turn these features into photo-realistic images gives scientists an intuitive way to compare theirown images with large-scale reference datasets.
Revised: Having the ability to turn these features into photo-realistic images gives scientists an intuitive way to compare their own images with large-scale reference datasets.

Original: Our work shows promising prospects in this field.
Revised: Our work shows promising prospects in this field