MorphoDiff: Cellular Morphology Painting with Diffusion Models
Authors: Zeinab Navidi
Jun Ma
Esteban Miglietta
Le Liu
Anne E Carpenter
Beth A Cimini
Benjamin Haibe-Kains
BO WANG
Venue: ICLR 2025 Spotlight (2025)
Keywords: Generative Modelling
Latent Diffusion Model
Cell Painting
Morphology
Drug Response Prediction
Cellular Phenotype
Machine Learning
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We present MorphoDiff as the first generative framework capable of producing guided, high-resolution predictions of cellular morphology that generalizes across both chemical and genetic interventions.

Abstract: Understanding cellular responses to external stimuli is critical for parsing biological mechanisms and advancing therapeutic development. High-content image-based assays provide a cost-effective approach to examine cellular phenotypes induced by diverse interventions, which offers valuable insights into biological processes and cellular states. We introduce MorphoDiff, a generative pipeline to predict high-resolution cell morphological responses under different conditions based on perturbation encoding. To the best of our knowledge, MorphoDiff is the first framework capable of producing guided, high-resolution predictions of cell morphology that generalize across both chemical and genetic interventions. The model integrates perturbation embeddings as guiding signals within a 2D latent diffusion model. The comprehensive computational, biological, and visual validations across three open-source Cell Painting datasets show that MorphoDiff can generate high-fidelity images and produce meaningful biology signals under various interventions. We envision the model will facilitate efficient in silico exploration of perturbational landscapes towards more effective drug discovery studies.

Published as a conference paper at ICLR 2025
MORPHODIFF: CELLULAR MORPHOLOGY PAINTING WITH DIFFUSION MODELS
Zeinab Navidi Jun Ma Esteban A. Miglietta', Le Liu', Anne E. Carpenter', Beth A. Cimini", Benjamin Haibe-Kains?5500 3,5,6,7-f, Bo Wang 1,2.3,
Department of Computer Science, University of Toronto, Toronto, ON, Canada
2. Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada Vector Institute, Toronto, ON, Canada
4. Imaging Platform, Broad Institute of Harvard and MIT, Cambridge, MA, USA Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
6. Structural Genomics Consortium, Toronto, ON, Canada
7. Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada
8. Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada
9. AI Hub, University Health Network, Toronto, ON, Canada Corresponding authors, Supervised this work equally
ben jamin n. haibe. ka dainseutoronto.com ca, bowang@restoriissiittee...
ABSTRACT
Understanding cellular responses to external stimuli is critical for parsing biolog- ical mechanisms and advancing therapeutic development. High-content image- based assays provide a cost-effective approach to examine cellular phenotypes induced by diverse interventions, which offers valuable insights into biological processes and cellular states. We introduce MorphoDiff, a generative pipeline to predict high-resolution cell morphological responses under different conditions based on perturbation encoding. To the best of our knowledge, MorphoDiff is the first framework capable of producing guided, high-resolution predictions of cell morphology that generalize across both chemical and genetic interventions. The model integrates perturbation embeddings guiding signals within 2D latent diffusion model. The comprehensive computational, biological, and visual vali- dations across three open-source Cell Painting datasets show that MorphoDiff can generate high-fidelity images and produce meaningful biology signals under var- ious interventions. We envision the model will facilitate efficient silico explo- ration of perturbational landscapes towards more effective drug discovery studies.
INTRODUCTION
Recent advancements in generative artificial intelligence (AI) have propelled significant progress across various domains, including computer vision, natural language processing, and healthcare (Rombach et al. 2022 OpenAl 2022) Meskó & Topol 2023). Notable advancements in Al method- ologies, coupled with the availability of vast datasets, have yielded foundation models with exten- sive applications and capabilities. Their practical application holds great promise for empowering practitioners in more effective drug development, thereby streamlining this process scale while conserving expert and financial resources (Moor et al. 2023).
Modeling cellular responses to external interventions a key focus computational biology; it aims to uncover biological insights that can inform the design of more effective therapies, which are, in essence, interventions aiming a particular cellular response (Celik al., (024). While there have been significant efforts to model cellular dynamics using AI algorithms, much of the prior work has concentrated on transcritoomi-level changes (Roohani et al. 2023 Lotfollahi et al. 2023 2019 Cui et al. 2024 Hetzel et al. 2022 However, advances in high-throughhut screening technologies now enable the exploration of rich phenotypic readouts, such as those generated by high-content microscopy imaging, which provide critical insights into cellular activity and accelerate drug target identification and mode-of ofttciion studies under diverse conditions (Seal et al.) (024). Among these technologies, the Cell Painting assay high-content microscopy imaging platform- has emerged

---

Published as a conference paper at ICLR 2025
as powerful, cost-effective approach for cellular phenotype screening, playing a pivotal role in un-
derstanding the morphological characteristics of cells under various perturbations (Bray et al. 2016).
Image analysis software, such as CellProfiler, has been widely adopted to extract detailed features
from microscopy images (Carpenter et al. 2006 CCow et al. 2022 Moshkov et al. 2024 These
features have provided valuable insights into compound polypharmacology (Chow al. 2022).
mechanisms of action (Tian et al.] 2023] WWy et al.) 2022; Wong et al. 2023; Dee et al. 2024). and
target genes associated with specific perturbations (Way et al. 2022 Despite these advances,
significant challenge in virtual screening remains: the size of existing screening libraries represents only a small fraction of the vast chemical space, which is estimated to contain over 10 drug-like like molecules (Lu et al., 2024 Lipinski et al.) 2012 Reymond, 2015). This limitation is particularly crit- ical when seeking to identify the most effective treatments for specific cellular conditions. Machine learning (MI.) and generative models might overcome this barrier by enabling response estimation across a much larger perturbational space, thus improving treatment efficacy while substantially
reducing cost.
There is growing interest in leveraging advanced deep learning methods to directly learn cellular patterns from high-content images, rather than relying solely on engineered features extracted using traditional image analysis software. A notable example is a recent study that developed a strategy for learning representations of treatment effects from high-throughput imaging using a causal frame- work (Moshkov et al.] 2024). By employing weakly supervised learning, the model captured both confounding factors and phenotypic features in the learned representations, providing a comprehen- changes. Another significant contribution is retrieval system based on multi-modal contrastive learning, which maps molecular perturbation and their corresponding Cell Painting image features into a unified embedding space Fänchee-Fernandee et al. (023). Al- though the model lacks generative capabilities, it proves to be valuable tool for retrieving perturba- tions with morphological effects most similar to those of the input query. Two relevant studies used generative models to transfer cell style between conditions low-resolution, single cell cropped image patches (Palma et al.) 2023 Bourou et al. 2023). Although these methods offer insights into cellular transitions, their focus on isolated cells overlooks inter-cellular effects in a wide well area as screened in original microscopy images and limits practical applicability. The most relevant method is a pipeline that employs conditional flow models for cellular phenotype estimation using perturbation information that was only tested on molecular interventions (Yang et al.) 2021).
sive view of greatment-inducee
To the best of our knowledge, this work represents one of the first efforts for estimating cell re- sponses a high-resolution and high-content image scale. By leveraging advanced generative mod- els in computer vision, as exemplified by |Rombach et al. (2022). and incorporating state-of-the-an (SOTA) perturbation encoding modules, including the single-cell foundational model proposed by Cui et al. (2024). we introduce MorphoDiff, a novel diffusion- based generative pipeline for high- resolution cellular morphology prediction, generalizable to both genetic and chemical perturbations. We benchmarked MorphoDiff on three microscopy imaging datasets, evaluating image fidelity and biological properties of cells in in-distribution and unseen perturbations, demonstrating promising performance overall. This work showcases the potential of advanced generative models to predict high-resolution cellular phenotypic responses across a broad range of perturbations, and we hope it paves the way for future research in this direction.
METHOD
Diffusion models have emerged as a powerful approach for image generation, overcoming chal- lenges such as training instability and mode collapse common in GAN-based architectures. In this work, we developed MirphoDiff, a novel pipeline capable of generating perturbed cellular mor- phology in high-resolution and high-content microscopy images with the integration of perturba- tion embedding as a guiding signal within the generative model. Our pipeline contains two key components: an image-based denoising diffusion probabilistic model (DDPM) and a perturbation projection module. The schematic representation of MirphoDiff is illustrated in Figure I
2.1 LATENT DIFFUSION MODEL FOR IMAGE GENERATION
The Stable Diffusion (SD) framework is built upon a latent diffusion model (LDM) pre-trained on a vast corpus of natural images, enabling the generation of realistic images based on input text prompts
2

---

Published as a conference paper at ICLR 2025
(Rombach et al. 2022). We adapted SD pipeline for task, and transformed available Cell Paint- ing channels (elaborated in the 3 section) into three RGB channels for modelling. For our specific task, fine-tuned the pre-tramed DDPM model in the SD framework on Cell Painting images projected into latent space. We opted to employ the existing SI) VAE trained on natural images due to its demonstrated efficacy in reconstructing Cell Painting images (sample images provided in Appendix Figure(5). More specifically, given a high resolution 512 x 512 RGB image X, the image encoder E' transforms X into latent representation 20 E(X) consisting of four channels of 64 64 feature maps. Subsequently, the decoder D reconstructs the generated image in latent space as the final outcome. The DDPM model learns sample distribution the latent space, and perturbs the latent image representation by introducing noise during the forward diffusion process.
High resolution
Cell Painting image
Latent Space
Perturbation Identifier
1
E
Forward Diffusion Process
Generative Process
Perturbation Encoder
Sample from N(O, /)
Generated Cell
Painting image
Denoising U-Net
D
D
Diffusion Condition
Z'o
Z'i
Z'rr
2'
Figure 1: MorphoDiff overview: Workflow diagram illustrating the architecture of the MorphoD- iff pipeline, comprising two primary modules: a DDPM module and perturbation encoder. The diffusion algorithm serves the central component of the workflow, performing conditional image generation. The perturbation projection module can be adapted depending on perturbation type.
In particular, the LDM training consists of two steps: forward diffusion process and generative (or sampling) process. The forward process is conducted on the latent variable z, generating a noisy perturbed embedding at various time points t from 0 to T. This process is done by adding Gaussian noise at each step, which is defined by:
N /1 1,1--,,,111,
(1)
where B B1..... Br is the pre-defined variance schedule. It has been shown that the distribution
of noisy samples converges to a standard Gaussian distribution approaches T (Ho et al.) 20200)
Assuming that the noise values follow Gaussian distribution, can be sampled based - at any desired time step t closed form using the reparameterization trick:
0, /1 a,4,1
(2)
where a 1 3e and (0, By using the chain rule and recalling that merging multiple Gaussian distributions remains Gaussian, we can derive direct formula for 2 from 2o as follows
v v1 arC,
where a. II' o,, and represents sample noise drawn from a standard Gaussian distribution.
The forward process generates the noisy perturbed image embedding 2₁ at time step trained to estimate the noise level from the noisy embedding 2 (Ronneberger al. 2015). In this process, the projected perturbation embedding p is integrated as a condition along with the z and time variable f, and are fed into the U-Net to predict the noise. The conditional LDM algo- rithm in MiffhoDiff leverages the information from to guide the conditional denoising step and perturbation-specife phenotype estimation.
U-Net
The MorphoDiff pipeline optimizes the U-Net model prediction by minimizing the Mean Squared Error (MSE) between the ground truth noise and the predicted noise. The loss enables the network
3

---

Published as a conference paper at ICLR 2025
to learn and accurately predict the added noise for each perturbed sample at time point t, facilitating the denoising process. The objective of the diffusion model can be summarized as:
loss MSF(ff (p, vo+20 + v1 - Cig, c),
(4)
where fo denotes the U-Net model. During the sampling process, MorphoDiff generates images based on the input perturbation. To initiate the generation process, a random noise vector is then sampled from a standard Gaussian distribution, denoted as z4. The pipeline iteratively calculates z/.
for ranging from T to 0 by:
1 Vat
az V1 -1 -
1
2
4
+ BBe,
(5)
where 2/-1 represents a less noisy embedding of the generated perturbed image at time step t t- 1. It is obtained using the current embedding zi the prediction fo z/. t) made by the U-Net model, and random noise sample The coefficients a, and Br control the contributions of the embedding and the noise, respectively. Through this iterative process, MorphoDiff generates a contextually relevant embedding that captures the desired cellular phenotype.
2.2 PERTURBATION PROJECTION MODULE
The original SD pipeline employed the CLIP model as the text encoder (Radford et al.
2021 Rom-
bach et al., 2022). To generate images exhibiting cellular morphological changes under specific treatments, we replaced CLIP with the perturbation embeddings obtained from projection algo- rithms depending on the type of perturbations. Our pipeline provides the flexibility of integrating different projection modules and can handle both genetic and chemical interventions.
For genetic perturbations, we used the SOTA single-cell foundation model ScGPT, which has been trained on 33 million normal human cells and employs stacked transformer layers to generate cell and gene embedding simultaneously (Cui et al. 2024). ScGPT has demonstrated strong ability to implicitly encode gene relationships in gene embeddings through generative modeling of gene ex- pression (mRNA profiles). This property is particularly relevant for our task as it facilitates learning meaningful patterns of cellular morphology linked to the encoded gene representation containing transcriptomic signals, thereby enhancing generalizability to unseen perturbations.
For chemical compounds, employed the molecular encoder software RDKit, well-established tool that converts standard molecular representations in SMILES format into numerical embedding [Landrum] 2023). It therefore captures structural similarity of the chemical compounds. The pro- jected latent variables are then fed into the MorphoDiff pipeline as a guiding signal. Further details on the perturbation incorporation steps (Appendix Note A.3.I). training protocol (Appendix Note and experiment information (Appendix Table 4 are provided in the Appendix section.
A.3.2).
3
EXPERIMENTS
3.1
DATASET
Three publicly available Cell Painting datasets were used for modelling and validation, representing a diverse range of perturbations and cell types, described as follows.
RxRx1 dataset contains 1108 perturbations across four cell types (Sypetkowski et al. 2023). Each sample comprises six-channel fluorescent microscopy images capturing key cellular structures. Our analysis concentrated on the largest group within the dataset (HUVEC cell line) with samples drawn from multiple experimental batches. Each small interfering RNA perturbation in the dataset targets a gene, resulting in significant mRNA knockdown and corresponding changes in protein expression (Sypetkowski et al. 2023). We conducted two sets of experiments: one using all HUVEC images (All Batches), and the other constrained to a single batch (Single Batch) to minimize batch effects. The authors' provided code was used to convert the six-channel images into RGB format |
BBBC021 dataset comprises 13200 images of MCF7 breast cancer cells, stained for DNA, F-actin, and B- B-tubulin, and imaged using three-channe fluorescent microscopy, which were directly mapped
Link to RxRx1 conversion code
4

---

Published as a conference paper at ICLR 2025
to RGB format for modelling (Caie et al. 2010). The MCF7 cells were treated with 113 small molecules, each administered at eight different concentrations. We conducted two sets of experi- ments: using all compounds for which SMILES MMldeddbased projections bedding was generated, and another included 14 compounds obtained from a list of six mechanisms of actions (MOA) reported having distinct phenotypes (Caie et al. 2010).
available and the em-
Rohban et al. dataset contains U2OS cell images with 323 over-expressed genes (Rohban et al,, 2017). Based on expert consultation, three of the five imaging channels-RNA, Mitochondria, and DNA- were selected for modeling and validation, prioritizing channels essential for image segmentation and feature extraction that are also biologically informative and interpretable (Stirling et al. [202]] Carpenter et al. (2006). Two gene subsets were analyzed, including: (1) five genes from pathways reported to affect cellular morphology, and (2) a list of 12 genes obtained from clustering performed in |Rohban et al. (2017) based on gene morphological features. Detailed description of perturbations for each dataset and their download links are provided in Appendix Note AA.33
Pre-processing: All images with larger than 512 x 512 pixels were resized to 512 x 512 resolutions to ensure consistent dimensions, with the largest possible area covered in each image. Normal- ization and scaling followed best practices in image pre-processing (Rombach et al. 2022). explanation pre-processing are provided in Appendix Note A.3.4
More
3.2 COMPARISON
For benchmarking, set of comparative analysis was conducted against the fine-tuned unconditional SD as the baseline (with fixed prompt encoding), assessing the impact of integrating perturbation data guiding the generative process to mimic cellular phenotype consistent with real signals. To the best of our knowledge, only one existing work offers conditional generative capabilities a high-resolution setting comparable to approach (Yang et al. (02T). However, due technical challenges, adapting their model for benchmarking on our datasets was not feasible. To provide an assessment of MorphoDiff's performance against the SOTA, we benchmarked against an exiting method that generates cellular phenotypes in individual cell cropped patches (Palma et al. 2023).
For experimental validation, 500 images were generated per perturbation. Real (ground truth) im- ages were augmented to match this number where necessary, using random flipping and rotation, to ensure consistency in evaluation metrics. The two-sample t-test was conducted for statistical anal- ysis, with a p-value threshold of < 0.05 considered as statistically significant. Further evaluation details are provided in Appendix Note A.3.5
3.3 RESULTS AND DISCUSSION
In this section, we present the results of MorphoDiff in predicting perturbation-speeific cellular mor- phology, including image fidelity metrics, visual assessment, and biologically interpretable features.
3.3.1 MORPHODIFF IMPROVES
CEEEIIRRAATIN-PPECIIEE DISTRIBUTIONAL DISTANCE
To assess MorphoDiff's effectiveness improve similarity of pixel distributions in generated im- ages to the associated real images for each perturbation, we employed well-established metrics for validating the quality of images created by generative models, including Fréchet Inception Distance (FID) and Kernel Inception Distance (KID) (Heusel et al. 2017: Sutherland et al. (2018). FID sum- marizes the distance between the Inception feature vectors for real and generated images in the same domain, while KID measures the dissimilarity between two probability distributions using indepen- dently drawn samples. We computed these distance metries between generated images for each perturbation and their corresponding real image groups
Table presents the average of calculated FID and KID distances across all perturbations. Lower FID and KID values indicate better alignment between generated and real distributions, with statisti- cally significant differences between MorphoDiff and the unconditional Stable Diffusion highlighted in bold. Our experiments showed that MorphoDiff consistently outperformed the baseline across all experiments, by effectively embedding perturbation -ppecffc patterns for both genetic and chemical interventions. In the RxRx1 (All Batches) and BBBC021 (All Compounds) experiments, distance metrics improved compared to smaller, single-batch experiments, suggesting that larger training sets enhance model performance. Additionally, we observed that MorphoDiff predictions improved
5

---

Published as a conference paper at ICLR 2025
the average ranking of images to the matched real perturbed cohort based on the distance metries. Further details are provided in Appendix Table
Table I: Average FID (x and KID metrics across all perturbations in each experiment, assess- ing distributional similarity between generated and real images from the corresponding perturbation conditions (lower is better). Bold values highlight statistically significant differences with p-value < 0.05, * indicating p-value <
0.01 and ** indicating p-value < 0.001.
Dataset
Experiment
Method
FIDJ
KIDI
RxRx
All Batches
MorphoDiff
0.78
0.05 x
RxRx1
All Batches
Stable Diffusion
1.15
0.11
RxRx1
Single Batch
MorphoDiff
1.14
0.12
RxRx1
Single Batch
Stable Diffusion
1.45
0.16
BBBC021
All Compounds
MorphoDiff
1.99
0.21
BBBC021
All Compounds
Stable Diffusion
3.84
0.47
BBBC021
14 Compounds
MorphoDiff
2.26'
0.30
BBBC021
14 Compounds
Stable Diffusion
3.22
0.42
Rohban et al.
Genes
MorphoDiff
2.51*
0.33*
Rohban et al.
5 Genes
Stable Diffusion
3.26
0.45
Rohban et al.
12 Genes
MorphoDiff
2.77'
0.38
Rohban et al.
12 Genes
Stable Diffusion
3.17
0.45
We further benchmarked MorphoDiff-generatte images the single-cell cropped patch scale against another existing method by Palma et al. (2023), with results summarized in Table|2land details of the analysis provided in Appendix Note A.3.5 Statistical testing demonstrated that MorphoDiff outper- forms the second-bes method across all compounds except Cytochalasin B. For this compound, our investigation revealed that in larger (broader field of view) images, MorphoDiff' achieved superior FID (1.9 vs. 2.9 (x 10))) and KID (0.23 vs. 0.37) metrics compared to Stable Diffusion, aligning with our qualitative assessment (sample images provided in Appendix Figure (6) We propose that generating and validating images with larger fields of view facilitates capturing cellular density and intercellular relationships. This approach has the potential to provide holistic insights into cellular interactions, phenotypic shifts induced by various perturbations, and cellular diversity within well.
Table 2: Average FID (x (x10-2) and KID metrics assessing distributional similarity between gen-
erated and real images of cell cropped patches (lower is better). Bold values highlight statistically significant differences of the best model with the second best model with p-value < 0.05, * indicat- ing p-value < 0.01 and ** indicating p-value 0.001.
AZ138
AZ258
Taxol
Cytochalasin B
Vincristine
Model
FID
KID|
FID.
KID|
FID.
KID,
FIDI.
KID.
FID.
KID,
Stable Diffusion
1.40
0.14
0.94
0.08
1.50
0.15
0.83*
0.07*
1.83
0.20
IMPA
0.98
0.07
1.18
0.12
1.28
0.11
1.23
0.11
1.05
0.07
MorphoDiff
*.
0.06**
0.76*
0.05*
1.09
0.10
1.19
0.11
0.86**
0.06**
3.3.2 MORPHODIEF CAPTURES CERTURBATIIO-SPECIIIE CELL MORPHOLOGY SIGNALS
Among the three datasets, BBBCO21 stands out due to its strong, visually detectable phenotypic changes induced by small molecule treatments, as well as for including several compounds with annotated Mechanisms of Action (MOA) (Caic et al. 2010). This dataset is particularly valuable for modeling and validating the biological interpretability of generated images, motivating us to exam- ine MorphoDiff's ability to capture the biological signals associated with different conditions. Our visual assessment reveals that MorphoDiff effectively captures perturbation induced cellular pat- terns for distinct compounds, closely mimicking the morphology of treated cells in real images. In contrast, the baseline SD model primarily learns general cellular structures. Figure 2 presents sam- ple images from six sample compounds representing the different MOA groups in the BBBC021 experiment (14 compounds). Our investigation further revealed that SypphoDiff-geneeaatt images successfully captured relatively rare cell cycle events, with sample images provided in Appendix

---

Published as a conference paper at ICLR 2025
Figure Overall, the qualitative comparison between real images and those generated by Mor- pho highlights MorphoDiff's promising alignment with real phenotypes.
Additional analysis was performed by training and validating models images obtained by crop- ping 512 x 512 pixel patches from the original BBBC021 images to assess the impact of focusing on smaller well area. Further description of pre- proceeseingg and sample images for all experiments provided in Appendix Note A.3.4 Appendix Figure 8 and with image distance validation results of the cropped version of experiments provided in Appendix Table
Compound
Compound: 2P12
Compound: Dytochalean
Compound: Epothdoone
Compaund:: Nocodlanda
Compound:
Real
MorphoDif
Stable Diffusion
Figure 2: MorphoDiff captures perturbation-speeifee cell morphology signals: Sample images from the BBBC021 (14 Compounds) experiment demonstrating perturbation induced cellular mor- phology for compounds, each representing one of the annotated Mechanism of Actions (MOA) present in this experiment. The top row shows real images: the middle row displays MorphoDiff- generated images: and the bottom row presents images generated by unconditional SD model.
3.3.3 MORPHODIFF LEARNS BIOLOGICALLY INTERPRETABLE FEATURES
To validate the model's capacity generate images with biologically meaningful representation, we expanded our benchmarking efforts using CellProfiler, a widely-used image analysis software in biomedical research to extract morphological cellular features from images across all conditions
Carpenter et al., 2006, Stirling et al., 2021). We quantified a comprehensive set of measurements related cellular morphology across the different channels for the Cell, Nucleus, and Cytoplasm compartments. All features were pre- processed and standardized following best practices before analysis (Serrano et al. 2023). This approach allowed for a systematic comparison of CellProfiler- extracted Teatures between different cell groups, providing robust evaluation of the biological accuracy of generated images. Further description of CellProfiler feature extraction and processing are provided in the Appendix Note|A.3.6)
In our initial validation, we applied Principal Component Analysis (PCA) the pre- processed Cell- Profiler features derived from MorphooDiff-generaate images, as shown in Figure 3h (3D visualiza- tions are provided the supplementary material). Sample distribution the space indicated that biological features extracted from MorphoDiff-generaa images under different compound treat- ments clustered cohesively in feature space. Additionally, when annotated by MOA labels visual- ized in Figure 3. many compound groups with similar MOAs formed distinct clusters, reflecting their distinct biological effects. The MOA annotations are provided in Appendix Table
We further applied K-Means clustering to CellProfiler features representing Cell, Nuclei. and Cyto- plasm characteristics, using five different seeds to ensure robust evaluation. The clustering perfor- mance based perturbation (compound) and MOA labels in real samples highlighted the inherent

---

Published as a conference paper at ICLR 2025
b
Perturaation
Mechanism Action
A2241
Aurara cirase inhibitors
AZ258
Actin disruptors
Cytochalisin D
Inhib
Labrunculin
A2138
PC2
Nocedazale
PC2
Microtubure stabliizers
Cytocholesin
PP
Taxol
Demecoicine
Epothilona
Uocetase!
PC1
RC
Cell Features
Nuclei Features
Cytoplasm Features
10
Real MorphoO
Real
Morch:Drrt
NorphoDif
NM1
NMI
Nerturbatiion
Perturbation
Epothilone B
AZ138
Docetaxel
Vincristine
DMSO
DIRGO
Marphobart
Cell Features
OREO Noighooft
DASO Mest Morshiodt
DREGO Real
DRIGO
RAck
Nuclel Features
Merphoont
MoratsOft
Cytoplasm Features
HorphsDum
NarphoDer
Norphaber
PC 2
Figure Biologically interpretable feature analysis. PCA visualization of CellProfiler fea- tures extracted from sraphoDifffgeeeratt images, annotated by (a) compound labels and (b) Mechanism of Action (MOA) labels.
(c) Normalized Mutual Information (NMI) metrics obtained from clustering CellProfiler features of real and MorphoDiff generated images (considering per- turbation and MOA labels) using the K-Means algorithm with different seeds, comparing their performance across three feature subsets representing Cell, Nuclei. and Cytoplasm properties. (d) Perturbation-specite PCA visualization of CellProfiler features comparing DMSO, real, and MirphoDiff- -eneertteee images for four example compounds for different feature subsets.
8

---

Published as a conference paper at ICLR 2025
complexity of feature space, with overlapping sample distributions making clustering challenging. However, comparing the K-Means clustering results on the marized in 3p showed that the model maintained this complexity while improving the distinction between different compounds and MOAs, in most cases, which resulted in enhanced clustering per- formance. Overall, the visualizations and clustering analysis provide strong evidence that features extracted from MorphoDiff-generatee terns, underscoring the model's ability generate biologically differentiated sample groups. PCA and clustering analysis for the cropped images are provided in Appendix Figure T0.
features sum-
images capture perturbation-speciffe and MOA-specific pat-
We examined the distributional shift of CellProfiler features in MorphoDiff generated images com- pared to those extracted from real images from cells treated with the the same compound versus DMSO (control). Figure B illustrates sample distribution across three groups of CellProfiler fea- tures for four representative compounds. Notably, the generated samples' distribution consistently demonstrated shift towards the feature distribution of real treated samples, diverging from DMSÓ controls. This trend provides strong evidence that MorphoDiff' successfully learns biologically meaningful features that closely align with the target phenotypic space. Additional visualizations for these compounds, encompassing morphological features representing Area Shape, Texture, and a set of features that describe the shape of cells using a basis of Zernike polynomials (Zemikke) are presented in Appendix Figure 11 Yang et al., 2021). Furthermore, additional visualizations for all six feature subsets across a broader range of compounds and cropped version of images are provided in Appendix Figure 12 and reinforeing the robustness of our findings.
Cell
Nucial Features
Cytoplasm Features
42341
0,t9da 6in
A2335
42339
Oytoca
Bal
Epsndow
Dacebux
Vinblastino
Bryostatin
Forskolin
AG-1478
Podophyllotoxin
Real
MorphoDiff
Figure 4: MorphoDiff' improves biological feature correlation and generalizes to unseen per- turbations. (a) Spearman correlation coefficient 1) of CellProfiler features extracted from images generated by the generative models compared to the same features from real perturbed images across Cell, Nuclei, and Cytoplasm compartments. Upper bound correlation between random split real image features are provided. (b) Sample images of the top 5 unseen compounds, comparing cellular phenotypes in real vs. MorphoDifff-eneee geffratgee cohorts.
To assess the models' ability to replicate cellular morphology, we calculated Spearman correlation coefficients between the averaged CellProfiler features of real and generated images for each com- pound. Given that feature correlations can vary across compounds due to factors like potency and morphological divergence from DMSO-treated controls, focused on the improvement in positive

---

Published as a conference paper at ICLR 2025
correlation achieved by MorphoDiff compared to the SD model. Figure 4a displays the correla- tion coefficients for Cell, Nuclei, and Cytoplasm features, which align with the range reported in previous work (Yang et al., 202Ib. The significant improvement observed in MorphoDiff over the baseline suggests it can generate biologically relevant, compound-specifie cellular morphologies.
In addition to these major results, noticed MorphoDiff images replicated certain biological nuances present in real images. We examined Pearson correlation coefficients between averaged CellProfiler features of images for pairs of 14 compounds in the BBBC021 experiment. Notably, the extreme cases of compound correlations observed in real images were faithfully reproduced in seiphoDiff-geeerttee
features. Specifically, AZ841 and AZ258 exhibited the highest correlation (0.96 in real vs 0.95 in MorphoDiff generated images), while AZ841 and Cytochalasin D demon- strated the lowest correlation (-0.029 in real vs. -0.45 Morpholiff generated images). The distributional similarity of A7841 and AZ258 can also be observed in Figure 3h. Preservation of relative phenotypic relationships among compounds underscores MorphoDiff's capacity to capture nuanced biological interactions. Moreover, comparing standard-processed and cropped images in the BBBC021 experiments showed slight improvement in computational metrics and sample dis- tributions for cropped images overall (see Appendix Table possibly due to enhanced compound- specific signals. While cropping may boost model performance, further research needed to assess its impact ML learning and its practicality.
3.3.4 MORPHODIFF GENERALIZABIN TO UNSEEN DRUGS
We investigated MorphoDiff's capability to generalize to held-out compounds, an important yet challenging task drug response prediction (Saha et al. 2024).. Using Pearson correlation of com- pounds' structural embeddings generated by RDKit, evaluated MorphoDiff's performance on the top 15 held-out compounds most correlated with in-distribution drugs (see Appendix T which allowed us to assess its generalization to unseen perturbations. Computational validation using FID and KID metrics revealed significant improvements in CrrphoDiff-generatte treated samples, with performance metrics summarized in Table 3 for both standard and cropped experiments. These results underscore MorphoDiff's capability in generalization as a diffusion- based model, which could be further enhanced with larger, more diverse training sets and advanced computational resources.
vs. SD-generated
Table 3: FID (x10-2) and KID) metrics of real and generated images for unseen compounds. Bold values indicate p-value < 0.05, * indicates p-value < 0.01, and ** indicates p-value 0.001
Experiment FIDI (Standard) FID4 (Cropped) KIDJ (Standard) KIDJ (Cropped)
MorphoDiff
1.83
1.57
0.2*
0.17 0.23
Stable Diffusion
2.56
2.03
0.30
Figure 4 shows images for the top five held-out drugs most correlated with in-distribution com- pounds. While most generated images closely resembled real samples based on molecular structure embeddings, Forskolin exhibited distinct phenotype. Our analysis revealed these images were more similar to Cytochalasin D, Forskolin's 's most correlated in in-distribution compound, suggesting that MorphoDiff generalizes to new compounds by leveraging perturbation embedding similarities. This highlights the impact of perturbation encoding accuracy on performance. While the current setting of MorphoDiff pipeline offers flcxibility to integrate various tools, joint learning of the gen- erative pipeline and perturbation encoders could enhance the model's generalizability.
4 CONCLUSION
We have introduced MofphoDiff, a new diffusion model-based generative framework for predicting cellular morphological responses to genetic and chemical perturbations in high-resolution images. Our experiments have demonstrated promising performance with respect to fidelity metrics and bi- ological interpretability of the generated images. We expect that MorphoDiff can serve as strong foundation for advancing cellular response prediction tools. Future work could incorporate addi- tional covariates, such as drug concentration and cell type, by leveraging scaled imaging datasets and further investment this area. Additionally, exploring combinatorial therapies and their syner- gistic effects presents exciting direction toward advancing precision medicine.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by Natural Sciences and Engineering Research Council of Canada Discov- ery Grant [RGPIN-2021-02688) (to B.H.K.), and funding from the NIH P41 GM135019 (to A.E.C. and B. B.A.C.).
We would like to thank Drs. Paula Llanos and Suganya Sivagurunathan from the Cimini Lab, as well as Dr. Ian Smith, for their valuable guidance and expertise in biological image analysis and data interpretation.
REFERENCES
Anis Bourou, Thomas Boyer, Kévin Daupin, Véronique Dubreuil, Aurélie De Thonel, Valérie Mezger, and Auguste Genovesio. Phendiff: Revealing invisible phenotypes with conditional diffusion models. arXiv preprint arXiv:2312.08290, 2023.
Mark-Anthony Bray, Shantanu Singh, Han Han, Chadwick T Davis, Blake Borgeson, Cathy Hart- land, Maria Kost-Alimova, Sigrun M Gustafsdottir, Christopher C Gibson, and Anne Carpenter. Cell painting, high-content image-based assay for morphological profiling using multiplexed fluorescent dyes. Nature Protocols, 1199::755711774 2016.
Peter Caie, Rebecca Walls, Alexandra Ingleston-Orme, Sandeep Daya, Tom Houslay, Rob Eagle, Mark E Roberts, and Neil O Carragher. High-content phegh-oontenphhentttppi profiling of drug response signatures across distinct cancer cells. Molecular Cancer Therapeutics, 166):1913-192 2010.
Anne E Carpenter, Thouis R Jones, Michael R Lamprecht, Colin Clarke, In Han Kang, Ola Friman, David A Guertin, Joo Han Chang, Robert A Lindquist, Jason Moffat, al. Cellprofiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biology, 7:1-11, 2006.
Safiye Celik, Jan- Chni-hhistian Hütter, Sandra Melo Carlos, Nathan H Lazar, Rahul Mohan, Conor Till- inghast, Tommaso Biancalani, Marta Fay, Berton A Earnshaw, and Imran S Haque. Building, benchmarking. and exploring perturbative maps of transcriptional and morphological data. PLOS Computational Biology. 20110): 2024.
Yuen Ler Chow, Shantanu Singh, Anne E Carpenter, and Gregory P Way. Predicting drug polyphar- macology from cell morphology readouts using variational autoencoder latent space arithmetic. PLoS Computational Biology, 18(2))e 1009888, 2022.
Haotian Cui, Chloe Wang, Hassaan Maan, Kuan Pang, Fengning Luo, Nan Duan, and Bo Wang. segpt: toward building a foundation model for single-cell multi-omics using generative ai. Nature Methods, pp. 1-11, 2024.
William Dee, Ines Sequeira, Anna Lobley, and Gregory Slabaugh. Cell-vision fusion: A swin transformer- based approach for predicting kinase inhibitor mechanism of action from cell painting data. Iscience, 27(8), 2024.
Leon Hetzel, Simon Bochm, Niki Kilbertus, Stephan Günnemann, Fabian Theis, et al. Predicting cellular responses to novel drug perturbations at single-cell resolution. Advances in Neural Information Processing Systems, 35:26711-26721 2022.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in Neural Information Processing Systems, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
Greg Landrum. Open-source cheminformatics software, 2023. URL httph::t///// www. rak org/]
Christopher A Lipinski, Franco Lombardo, Beryl W Dominy, and Paul J Feeney. Experimental and computational approaches to estimate solubility and permeability in drug discovery and develop- ment settings. Advanced Drug Delivery Reviews, 64:4-17, 2012.

---

Published as a conference paper at ICLR 2025
Mohammad Lotfollahi, F Alexander Wolf, and Fabian J Theis. segen predicts single-cell perturba- tion responses. Nature Methods, 16(8):715-27) 721,, 2019.
Mohammad Lotfollahi, Anna Klimovskaia Susmelj, Carlo De Donno, Leon Hetzel, Yuge Ji, Igna- cio L Ibarra, Sanjay R Srivatsan, Mohsen Naghipourfar, Riza M Daza, Beth Martin, et al. Predict- ing cellular responses to complex perturbations in tigh-throughput screens. Molecular Systems Biology, pp. e11517, 2023.
Stephen Zhewen Lu, Ziqing Lu. Ehsan Hajiramezanali, Tommaso Biancalani, Yoshua Bengio, Gabriele Scalia, and Michal Koziarski. Cell morphology-gun ggggdded small molecule generation with gflownets. In ICMI. 2024 Workshop Structured Probabilistic Inference ({&) Generative Mod- eling, 2024.
Claire McQuin, Allen Goodman, Vasiliy Chernyshev, Lee Kamentsky, Beth A Cimini, Kyle W Karhohs, Minh Doan, Liya Ding, Susanne M Rafelski, Derek Thirstrup, et al. Cellprofiler 3.0: Next-generation image processing for biology. PLoS Biology, 16(7):e2055970, 2018.
Bertalan Meskó and Eric J Topol. The imperative for regulatory oversight of large language models (or generative ai) in healthcare. NPJ Digital Medicine, 6(1):120, 2023.
Michael Moor, Oishi Banerjee. Zahra Shakeri Hossein Abad, Harlan M Krumholz, Jure Leskovec, Eric J Topol, and Pranav Rajpurkar. Foundation models for generalist medical artificial intelli- gence. Nature, 616(7956):259 265, 2023.
Nikita Moshkov, Michael Bornholdt, Santiago Benoit, Matthew Smith, Claire McQuin, Allen Good- man, Rebecca A Senft, Yu Han, Mehrtash Babadi, Peter Horvath, et al. Learning representations for image-based profiling of perturbations. Nature Communications,, 15(1):1594, 2024.
OpenAl. Chatgpt.
com/ cht 2022. Accessed: February 2024.
Alessandro Palma, Fabian J Theis, and Mohammad Lotfollahi. Predicting cell morphological sponses to perturbations using generative modeling. bioRxiv, pp. 2023-07, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference Machine Learning, pp. 8748-8763. PMLR, 2021.
Jean-Louis Reymond. The chemical space project. Accounts of Chemical Research, 48(3):722-730, 2015.
MH Rohban, S Singh, X Wu, JB Berthet, MA Bray, Y Shrestha, X Varelas, JS Bochm, and AE Car- penter. Systematic morphological profiling of human gene and allele function via cell painting. elife 6. Santos, R., Ursu, O., Gaulton, A. (l.170)7). A comprehensive map of molecular drug targets. Nat. Rev. Drug Discovery, 16(1):19-3) 19119,442220 2017.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Con- ference Computer Vision and Pattern Recognition, pp. 10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical image computing and csssuter-assisted intervention- MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings, part III 18. pp. 234-241. Springer, 2015.
Yusuf Roohani, Kexin Huang, and Jure Leskovec. Predicting transcriptional outcomes of novel multigene perturbations with gears. Nature Biotechnology, pp. 1-9, 2023.
Udit Surya Saha, Michele Vendruscolo, Anne E Carpenter, Shantanu Singh, Andreas Bender, and Srijit Seal. disttribbttiitto validation for bioactivity prediction in drug discovery: Lessons from materials science. In JCML'24 24 Workshop for Life and Material Science: From Theory to Industry Applications, 2024.
12

---

Published as a conference paper at ICLR 2025
Ana Fanchez-Ferrandee Elisabeth Rumetshofer, Sepp Hochreiter, and Günter Klambauer. Cloome: contrastive learning unlocks bioimaging databases for queries with chemical structures. Nature Communicationss 14(1):7339, 2023.
Srijit Seal, Maria-Anna Trapotsi, Ola Spjuth, Shantanu Singh, Jordi Carveras-Puigvert, Nigel Greene, Andreas Bender, and Anne E Carpenter. A decade in a systematic review: The evolution and impact of cell painting. ArXiv, 2024.
Erik Serrano, Srinivas Niranj Chandrasekaran, Dave Bunten, Kenneth Brewer, Jenna Tomkinson, Roshan Kern, Michael Bornholdt, Stephen Fleming, Ruifan Pei, John Arevalo, et al. Reproducible image-based profiling with pycytominer. ArXiv, 2023.
David R Stirling, Madison Swain-Bowden, Alice M Lucas, Anne Carpenter, Beth A Cimini, and Allen Goodman. Cellprofiler 4: improvements in speed, utility and usability. BMC Bioinformat- ics, 22:1-11, 2021.
JD Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans. In International Conference for Learning Representations, pp. 1-36, 2018.
Maciej Sypetkowski, Morteza Rezanejad, Saber Saberian, Oren Kraus, John Urbanik, James Tay- lor, Ben Mabey, Mason Victors, Jason Yosinski, Alborz Rezazadeh Sereshkeh, et al. Rxrxl: A dataset for evaluating experimental batch correction methods. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 4284 4293, 2023.
Antti Tarvainen and Harri Valpola. Mean teachers better role models: Weight-averaged sistency targets improve semi-supervised deep learning results. Advances in Neural Information Processing Systems, 30, 2017.
Guangyan Tian, Philip Harrison, Akshai Sreenivasan, Jordi Carreras- Puigvert, and Ola Spjuth. Combining molecular and cell painting image data for mechanism of action prediction. Artificial Intelligence in the Life Sciences, 3:100060, 2023.
Gregory Way, Ted Natoli, Adeniyi Adeboye, Lev Litichevskiy, Andrew Yang, Xiaodong Lu, Juan C Caicedo, Beth A Cimini, Kyle Karhohs, David J Logan, al. Morphology and gene expression profiling provide complementary information for mapping cell state. Cell Systems, 13 (11):911-923, 2022.
Daniel R Wong, David J Logan, Santosh Hariharan, Robert Stanton, Djork-Arn Clevert, and An- drew Kiruluta. Deep representation learning determines drug mechanism of action from cell painting images. Digital Discovery, 2(5):1354-1366 1367, 2023.
Karren Yang. Samuel Goldman, Wengong Jin, Alex X Lu, Regina Barzilay, Tommi Jaakkola, and
Caroline Uhler. Mol2image: improved conditional flow models for molecule to image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 6688 06986 2021.
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1
FIGURES
Original
Reconstructed
Figure 5: Sample images with their reconstructed version generated by the Stable Diffusion Varia- tional Autoencoder module used for image encoding and decoding in the MorphoDiff pipeline.
Real
MorphoDiff
Stable Diffusion
Figure 6: Random sample images generated by MorphoDiff, Stable Diffusion and from real cohorts for the Cytochalasin compound.
14

---

Published as a conference paper at ICLR 2025
Cytochalasin B
Colchicine
AZ841
AZ258
Real
MorphoDiff- generated
Figure 7: Random sample images generated by MorphoDiff and from real cohorts, demonstrating the generated images' capturing infrequent phenotypes associated with cell cycle alterations, such as bi-nucleated cells (in Cytochalasin B) and cells in metaphase stage (Colchicine, AZ841, and AZ258).
15

---

Published as a conference paper at ICLR 2025
MorphoDiff Generated Images
Real Images
RaiRxt All Batches
FioRkt Single Balch
BBBCO21 All Compounds
BBBC021 14 Compounds
Rohbban et al. Genes
Rohban al. 12 Genes
Figure 8: Random sample images generated by MorphoDiff and from real cohorts in different datasets.
16

---

Published as a conference paper at ICLR 2025
Compound A2258 MOA Autore linase im biors
Compoand: MDA: Eperidial
Compound: Cytochalesin
Compound: Epclh MOA: Mic stabilcous
Compound: Nocodaaske MOA: Marotttede dosabiloors
Compound A2138 MOAI Eghinnarars
MOA: Adin dianplors
Resl
MorphoDiil
Stable Diflusion
Figure 9: Sample images from the BBBCO21 (14 Compounds) experiment with cropped pre- processing including smaller well area, demonstrating perturbation-inducee cellular morphology for 6 compounds: AZ258, PP 2, Cytochalasin D, Epothilone B, Nocodazole, and AZ138, each rep- resenting one of the annotated Mechanism of Action (MOA) groups present in this dataset. The top row shows real (ground truth) images: the middle row displays images generated by the MorphoDiff pipeline: and the bottom row presents images generated by unconditional Stable Diffusion model.
Perturbation
Mechanism of Action
A2844
inhibitors
AZ258
Actin disruptors
Cytochelasin
Eg5 inhibitors
Latrunculin B
Microtubule destabilizors
A2138
Epithelial
PC2
Vincristine
Microtabule stabilizers
Nocodazole
Cytochalasin
Demecolcine Epothilone
Colchicine
Docetaxel
PC 1
Cell Features
Nuclei Features
Cytoplasm Features
MarphoDIfT
0.4
62-
Neturbation
MDA
Perturbation
Narturbation
MDA
Figure 10: PCA visualization of CellProfiler features extracted from MorphoDiff-generatee images cropped image processing experiment of the BBBC021 experiment with 14 compounds, anno- tated by (a) compound labels and (b) Mechanism of Action (MOA) labels. (c) Normalized Mutual Information (NMI) metrics obtained from clustering CellProfiler features of real and MorphoDiff generated images using the K-Means algorithm with 5 different seeds, comparing their performance across three feature subsets representing Cell, Nuclei, and Cytoplasm properties.
17

---

Published as a conference paper at ICLR 2025
Epothilone
AZ138
Docetaxel
Vincristine
Real Mepfco
Beat MerphcDki
Marphole
MarghuDe
Areashape Features
DPSC
Pell HarfeDD
Wall
Texture Features
Roal
Real
Harpholit
MorphoDit
NorphoDer
Zernike Features
PC2
Figure 11: Perturbation-speeifie PCA visualization of CellProfiler features of the BBBC021 experi- images for
with 14 compounds comparing DMSO, real perturbed, and MorphoDiff-genennee four example compounds: Epothilone B, AZ138, Docetaxel, and Vincristine considering different subsets of CellProfiler features including Area Shape, Texture, and Zernike relevant properties.
18

---

Published as a conference paper at ICLR 2025
AZ258
AZ841
fest
Not
Call Festures
Feol
Nuclei Features
Kasl
Cytoplasm Features
Areashape Features
DI50
MarphoO
Texture Features
Marpho0
Zemika Features
PC
PC
Figure 12: Perturbation-speccfie PCA visualization of CellProfiler features of the BBBC021 experi- ment with 14 compounds comparing DMSO, real perturbed, and MorphoDiif-generatel images for two compounds: AZ258, AZ841, considering different subsets of CellProtiler features including Cell, Nuclei, Cytoplasm, Area Shape, Texture, and Zernike relevant properties.
19

---

Published as a conference paper at ICLR 2025
Epothilone B
AZ138
Docetaxel
Vincristine
DeSO Mosthele
Dedo Marphuta
Dedo Marpaabee
DMEO
He Merphete
Cell Features
DAND Ncchhedat
TRNO Marphablt
DANO Micrnatitt
Nuclei Features
DINO Merpicoot
Cytoplasm Features
No shobint
Morp sobit
AreaShape Features
Moraviia
Texture Features
Zeriike Features
Poud
Fisl
PC2
PC
Figure 13: Perturiation-specifie PCA visualization of CellProfiler features of cropped processed images in the BBBC021 experiment with 14 compounds, comparing DMSO, real perturbed, and MorphoDiff-ge SrpphoDDff-ggeertteed images for four example compounds: Epothilone B, AZ138, Docetaxel, and Vineristine, considering different subsets of CellProfiler features including Cell, Nuclei, Cytoplasm, Area Shape, Texture, and Zernike relevant properties.
20

---

Published as a conference paper at ICLR 2025
A.2 TABLES
Table 4: Table of experiments' details from different datasets, including dataset name, experiment information, model type, number of trained steps, number of total training images, number of bal- anced images per perturbation included in training.
Trained steps
Training
Per perturbation
Dataset
Experiment
Model
images
balanced images
RxRxI
All Batches
MorphoDiff
[130K-133K]
59K
50
RxRx1
All Batches
Stable Diffusion
130K-133KK
59K
50
RxRx1
Single Batch
MorphoDiff
113K-118K]
2.46K
50
RxRx1
Single Batch
Stable Diffusion
[11K--188K]
2.46K
50
BBBCO21
All Compounds
MorphoDiff
[10K--142K]
98K
1K
BBBC021
All Compounds
Stable Diffusion
[140K-142K]
98K
1K
BBBC021
Compounds
MorphoDiff
[16K-120K]
14K
1K
BBBC021
14 Compounds
Stable Diffusion
[16K-120K]
14K
1K
Rohban et al.
5 Genes
MorphoDiff
[130K]
IK
200
Rohban al.
5 Genes
Stable Diffusion
[130K]
200
Rohban et al.
12 Genes
MorphoDiff
[6]K-63K]
6K
500
Rohban et al.
12 Genes
Stable Diffusion
[61K-63K]
6K
500
Table 5: Table showing the average rank of matched compounds based on FID and KID metric com- parisons between generated images and all real perturbed image cohorts. Ranks were normalized by the total number of compounds in each experiment, with MorphoDiff consistently achieving lower average rank compared to the unconditional Stable Diffusion model. A lower rank (highlighted in bold) indicates better performance, with the ideal case (rank 1st) representing the scenario where the generated images for a specific perturbation have the smallest distance metric with the matched real cohort.
Dataset
Experiment
Method
FID.
KID
RxRx1
All Batches
MorphoDiff
0.13
0.12
RxRx1
All Batches
Stable Diffusion
0.51
0.51
RxRxI
Single Batch
MorphoDiff
0.12
0.12
RxRx1
Single Batch
Stable Diffusion
0.51
0.51
BBBC021
All Compounds
MorphoDiff
0,41
0.42
BBBC021
All Compounds
Stable Diffusion
0.50
0.50
BBBC021
14 Compounds
MorphoDiff
0.42
0.43
BBBC021
14 Compounds
Stable Diffusion
0.53
0.53
Rohban al.
Genes
MorphoDiff
0.56
0.56
Rohban et al.
Genes
Stable Diffusion
0.6
0.6
Rohban al.
12 Genes
MorphoDiff
0.51
0.5
Rohban et al.
12 Genes
Stable Diffusion
0.54
0.54
21

---

Published as a conference paper at ICLR 2025
Table 6: Average of FID (x10-3) and KID metrics across all perturbations in the BBBC021 ex- periments for two different pre-pri processess approaches (standard and cropped around smaller well area), assessing distributional similarity between generated and real images from the corresponding perturbation conditions (lower is better). Bold values highlight statistically significant differences with p-value 0.05, * indicates p-value 0.01, and ** indicates p-value 0.001.
Dataset
Experiment
Method
FID
KID,
BBBC021 All Compounds (Standard)
MorphoDiff
1.99
0.21
BBBC021 All Compounds (Standard)
Stable Diffusion
3.84
0.47
BBBCO21 All Compounds (Cropped)
MorphoDiff
1.55
0.14
BBBC021 All Compounds (Cropped)
Stable Diffusion
2.80
0.29
BBBC021 Compounds (Standard)
MorphoDiff
2.26"
0.30
BBBC021
14 Compounds (Standard)
Stable Diffusion
3.22
0,42
BBBC021
14 Compounds (Cropped)
MorphoDiff
1.48
0.17*
BBBC021 14 Compounds (Cropped)
Stable Diffusion
2.42
0.30
Table 7: List of compounds and their annotated mechanism of actions (MOA) in the BBBC021
experiment (14 compounds)
Compound
Mechanism Of Action (MOA)
AZ1188
Eg5 inhibitors
A7841
Aurora kinase inhibitors
AZ258
Aurora kinase inhibitors
Cytochalasin
Actin disruptors
Cytochalasin
Actin disruptors
Latrunculin
Actin disruptors
PP 2
Epithelial
Demecoleine
Microtubule destabilizers
Nocodazole
Microtubule destabilizers
Colchicine
Microtubule destabilizers
Vincristine
Microtubule destabilizers
Epothilone B
Microtubule stabilizers
Taxol
Microtubule stabilizers
Docetaxel
Microtubule stabilizers
Table List of top most correlated held-ou out compounds to the in-distribution compounds (Based on the molecular encoding obtained from RDKit tool) in the BBBC021 experiment with 14 compounds included training.
In-distribution compound Held-out compound Pearson correlation
Vincristine
Vinblastine
0.92
PP
AG-1478
0.78
Docetaxel
Bryostatin
0.70
Colchicine
Podophyllotoxin
0.70
Taxol
Bryostatin
0.66
Cytochalasin D
Forskolin
0.61
Cytochalasin D
Rapamycin
0.61
Demecolcine
Emetine
0.57
PP
AZ701
0.53
Cytochalasin B Cytochalasin B Docetaxel Nocodazole
Simvastatin
0.52
Mevinolin-lovasttie
0.50
Nystatin
0.49
Acyclovir
0.48
PP 2
Bohemine
0.47
PP 2
Roscovitine
0.46
Cytochalasin B
Taurocholate
0.46
22

---

Published as a conference paper at ICLR 2025
A.3 NOTES
A.3.1 PERTURBATION ENCODING
MorphoDiff offers the flexibility incorporate different perturbation encoding modules, tailored to the type and properties of the perturbation.
For genetic perturbations, gene IDs from the original dataset were mapped to genes available in the scGPT model, pretrained on 33 million normal human cell The 512-dimensional gene embed- dings generated by this model were incorporated into the MorphoDiff pipeline as model conditions. For the RxRx1 dataset, siRNA perturbation IDs were mapped to their associated gene IDs using the Thermo Fisher Scientific website In the dataset by Rohban et al., gene IDs were directly provided as the perturbations associated with each treated image.
For compound perturbations in the BBBC021 dataset, SMILES representations of each compound were provided as metadata by the Broad Institute websita RDKit software was applied to the SMILES representations to generate numerical encodings representing chemical structures, with each feature normalized by its mean and standard deviation.
The code for generating chemical and genetic perturbation embeddings is available in the project's GitHub.
A.3.2 TECHNICAL DETAILS
Publicly released parameters of Stable Diffusion v1.4 along with the accompanying training script from Huggingface were used as the base mode Perturbation embeddings generated by the pro- jection module were padded match the shape of Stable Diffusion's prompt embedding. During training, we applied the Exponential Moving Average (EMA) technique to enhance model learn- ing and stability (Tarvainen & Valpola 2017). EMA is employed to reduce noise during training and improve the model's 's generalization performance. For all experiments, we used the most recent parameters available at the time of evaluation, and training was conducted based on the available computing and time resources, ensuring that the number of training steps across models for each experiment was comparable to facilitate a fair evaluation of models. To address class imbalance, we applied image augmentation techniques such as rotation and flipping the training samples from each perturbation cohort, enhancing sample diversity. The code for data pre- pre-processing model train- ing, evaluation, and result reproduction, along with the model weights available on the project's GitHub page. All training and evaluation processes were conducted on NVIDIA T4 and NVIDIA A40 GPUs. Most hyper-parameters remained consistent across all models and were aligned with the original Stable Diffusion settings, unless stated. A batch size of 32 and a learning rate of 1 x 10-5 were used for all training runs.
A.3.3 DATASETS
RxRx1 dataset developed by Recursion (Sypetkowski et al. 2023). contains 1108 perturbation classes across four distinct cell types. Each sample comprises six-channel fluorescent mieroscopy images capturing key cellular structures, including the nucleus, endoplasmic reticulum, actin cy- toskeleton, nucleolus, mitochondria, and Golgi apparatus. Our analysis concentrated on the Human Umbilical Vein Endothelial Cells (HUVEC), the largest group within the dataset, comprising ap- proximately 59,000 samples drawn from multiple experimental batches. We included only images with gene annotations available on the Thermo Fisher Scientific websita7 corresponding to the original siRNA perturbations. The RxRx authors identified pronounced batch effects across exper- imental runs, presenting challenge for ML model training. To investigate this, we conducted two sets of experiments: one using all HUVEC images (RxRx1 All Batches), and the other constrained to single batch (RxRxI Single Batch) to minimize batch effects. Given the large number of
https://www.lt hutps//wwww.herrmoiiier...cc.o
Link to BBBC021 metadata
Training Script
23

---

Published as a conference paper at ICLR 2025
siRNA perturbations in RxRxI, random subset of 50 siRNAs was selected for validation in each experiment, balancing the constraints of time and computational resources. The authors' provided code was used to convert the six-channel images into RGB format for input into the MorphoDiff and Stable Diffusion pipeline The images were directly downloaded from Recursion's website [
We further extended our analysis to the BBBC021 dataset comprising 13200 images of MCF7 breast cancer cells, stained for DNA, F-actin, and B-tubulin, and imaged using three-channel flu- orescent microscopy (Caie al., 2010). For consistency, these channels were directly mapped to RGB format for modelling and validation. The dataset profiles MCF7 cells treated with 113 small molecules, each administered at eight different concentrations over a 24-hour period. We conducted two sets of experiments: one using all compounds for which MaIILES-basee projections were avail- able and the embedding was generated by the projection module (98 in total), and another included 14 compounds with 6 MOAs provided on the dataset website (Caie et al. 2010). These MOAs included Eg5 inhibitors, Aurora kinase inhibitors, Actin disruptors, Epithelial modulators, and Mi- crotubule destabilizers, and Microtubule stabilizers (Caie et al.) 2010). Images from this dataset were directly downloaded from Broad Bioimage Benchmark Collection "ebsite"
The Rohban et al. dataset contains Cell Painting images of U2OS cells with 323 over-expressed genes (Rohban et al., 2017). Based on expert consultation, we selected three of the five imag- Mitochondria, and DNA-fon for modeling and validation, prioritizing channels
ing channels-RRM essential for image segmentation and feature extraction that are also biologically informative and interpretable (Stirling et al.] [2021] Carpenter et al. 2006). The choice of channels be flexible depending on dataset properties. Two gene subsets were used from this dataset for modeling. In the first experiment, we used set of five genes (gene list: RACI, KRAS, CDC42, RHOA, PAKI) reported in the original study as being involved in pathways known to affect cellular morphology (Rohban al. 2017). For the second experiment, we focused on gene clusters identified in (Rohban et 2017) based on morphological profiling. We selected genes from clusters with at least three genes, having a Gene Ontology (GO) term with p-value below 0.01, and where at least half of the genes in the cluster were associated with the GO terms (gene list: XBP1, MAPK14, 14, RAC1, AKTI, AKT3, RHOA, PRKACA, SMAD4, RPS6KBI, 1, KRAS, BRAF, RAFI). These genes were sourced from Supplementary File IF in (Rohban et al. 2017). Images for this dataset were directly downloaded form the Cell Painting Gallery GitHub page"
A.3.4 PRE-PROCESSING
Standard Pre-processing: For all experiments, images larger than 512 512 pixels were resized to this dimension, preserving as much of the original image area possible. Pre-processing followed Stable Diffusion best practices, scaling pixel values the (0, 1) range and normalizing with a mean and standard deviation of 0.5 (Rombach et al. 2022).
Cropped Pre-processing: To evaluate the impact of focusing on smaller well areas with enhanced cellular texture, five 512 x 512 pixel patches were cropped from the four corners and center of the BBBC021 images. This approach augmented the data, and the processed images were different from the standard processing by magnifying cellular details. Cropped images were used for mod- cling and validation, with comparisons made against the standard pre-process preeppocessing results. The same normalization process as in standard pre-processing was applied prior to further analysis.
Cell Painting Specific Pre-processing: For the RxRx1 and BBBC021 datasets, self-standardizzttio was applied as recommended by (Sypetkowski et al.) 2023), where each image channel was stan- dardized separately. In the experiments from Rohban el al., illumination correction arrays provided by the authors were used to adjust for brightness differences as part of the pre-processing pipeline (Rohban et al. 2017).
Conversion Code
https:/wwwwrxxx.i//ssea
24

---

Published as a conference paper at ICLR 2025
A.3.5 VALIDATION
MorphoDiff's performance was validated with respect to computational distance metries, visual assessment, and CellProfiler feature analysis to interpret the biological relevance of the generated images. We also benchmarked its performance against the unconditional Stable Diffusion frame- work as the baseline by fixing the imput prompt condition, and fine-tuning the SD framework on Cell Painting images, similar to what was used to train MorphoDiff.
For benchamrking cell cropped patches, we used the IMPA checkpoint trained on the BBBC021 dataset (six conditions: DMSO and five compounds Our models, trained adapted for image quality evaluation. Using a pre-trained generalist cell segmentation mode we segmented generated and real images to obtain cell masks. Consistent 96 x 96 pixel patches centered around single cells were cropped for fair evaluation against IMPA images, and compound-specine FID and KID scores were calculated. Following the Comparison section in the main manuscript and to provide robust evaluation, 500 cell patches were randomly selected for ground truth and the same amount were generated for each model (Stable Diffusion, IMPA, and MorphoDiff) across ten random seeds, with image quality metrics and significance tests reported for each compound.
compounds, were
In the CellProfiler analysis, image-level features extracted from real images were compared to those extracted from the generated images. Notably, the number of real images per condition was often smaller than the number of generated images. The two-sample t-test was used for statistical analysis, with p-vauues of less than 0.05 were considered statistically significant unless otherwise stated.
A.3.6 CELLPROFIER ANALYSIS
We used the CellProfiler tool for quantitative phenotype measurement across thousands of images, as is widely regarded as the state-of-the-art image analysis software in the literature. Due to the large volume of images, we employed Distributed CellProfiler (McQuin al., 2018), which facilitates running a Dockerized version of CellProfiler on Amazon Web Services (AWS) by leveraging AWS's storage and computing infrastructure. All CellProfiler pipelines used for analysis were implemented and customized by experts for each experiment.
comprehensive set of cellular morphology measurements was extracted across different channels for the Cell, Nucleus, and Cytoplasm compartments. The mean, median, and standard deviation of extracted features were summarized for each image. Image features were pre-processed follow- ing best practices, including the removal of features containing NaNs, elimination of outliers, and removal of highly correlated features. All features were normalized prior to analysis using the "stan- dardized" method from pycytominer Python package [4] (Serrano al. 2023 We analyzed different subsets of CellProfiler features (1088 in total) by focusing on those included "Cell" (300 features), "Nuclei" (394 features), "Cytoplasm" (318 features). "AreaShape" (267 features). "Texture" (346 feature) and "Zernike" (225 features) in their feature names, each correspond to aspects of cell mor- phological properties. The CellProfiler pipeline, feature processing and analysis scripts provided on the project's GitHub page.
"lauss//ithubb.cm////elllbbbbbbp http:///tthhh..oommcccCChhaahah
25


Reviews:
[{"review_id": "w3IYBdKPxi", "reviewer": "Reviewer_Pgmp", "summary": "This paper presents a novel generative model framework called MorphpDiff, which produces high-resolution cell morphology images in response to a drug or genetic perturbations. The authors leverage the use of latent diffusion models combined with perturbation-specific embeddings to simulate cellular response.  This model provides an alternative to traditional lab assays/screens by generating silicon drug and genetic responses in the cell. MorphoDiff is validated using three publicly available datasets (RxRx1, BBBC021, and Rohban et al. l Cell painting dataset)  ensuring that they have cell shape and state variability and perturbations. These models are evaluated with common image fidelity and metrics including Frechet Inception Distance (FID) and kernel Inception Distance (KID) showing improvements over the standard stable diffusion. Most importantly, the authors were able to capture cell morphology changes using CelProfiler for PCA and other morphology-based metrics.", "questions": "- Could the authors clarify the failure modes of MorphoDiff and the range of validity? How does the model perform across a broad range of drugs and genetic perturbations? Is there a category of drugs where the performance drops?\n\n- What kind of data augmentations were made and how do these impact the generalization across different drugs and genetic perturbations? I suggest ablations to improve the model's robustness. Additionally, could the authors consider conducting ablation studies to identify which augmentations make the model more robust?\n\n- What are the other factors were included in the selection of perturbation conditions? Did you add prior information such as concentrations of chemicals?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "MorphoDiff method shows novel methods combining latent diffusion models with perturbation embedding to predict cellular morphologies. The model generalizes across different cell types, drugs, and genetic alterations better than the standard stable diffusion model. It performs better than the standard diffusion model showing that the perturbation embedding helps the model learn and has the dual capability with gene knock-in/out and small molecule perturbations. \nThe authors use public and diverse datasets to validate their model and have clear reporting on how these were used. The model seems to generalize across different perturbations. The authors use generative metrics as well as biological meaningful metrics to evaluate the performance of their model. The models are tested using tools like CellProfiler to get interpretable and comparable results to the ground truth biological data.\n\nThis method scales and enables larger high-content screens for drugs. It provides a tool for simulating the effects of drugs or other genetic perturbations.", "weaknesses": "MorphoDiff fails to generalize to compounds that are out-of-distribution, and seems like it performs better at the lower frequency than the high-frequency features. How accurate is this model and how consistent is it at capturing other cellular structures across the cell?\nIn many of these stages, cell cycle plays a big role. Here, the paper lacks exploring how their model handles more complex events in the fields of view such as cell division.\nThis method relies on RGB images, it would be interesting to do an ablation study and see the performance with fewer channels.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5ruzWekPDo", "reviewer": "Reviewer_EJPX", "summary": "The paper addresses the challenge of virtual phenotypic screening for drug discovery.  The authors train a guided 2D latent diffusion model on microscopy images (with Cell Painting markers) to generate images of cells, as they would appear after treatment with a genetic or chemical perturbation.  In order to guide the model, they use embeddings of the corresponding perturbation: RDKit for compounds and scGPT for genes.\n\nThe authors fine-tune the publicly available Stable Diffusion v1.4 on three different Cell Painting datasets: Rxrx1 (siRNA), BBC021 (compound) and Rohban (over-expression), and compare the impact of the Perturbation Identifier Module for guiding the image generation process.  In addition to standard image quality metrics (FID and KID), the authors also use the popular Cell Profiler framework to extract engineered features from both real and generated images, and compare their ability to cluster perturbations by perturbation ID (ie, replicate) and mechanism of action (MOA).  The authors also evaluate correlation of Cell Profiler features extracted from real and generated images, demonstrating the impact of guiding the image generation process with perturbation embeddings.", "questions": "Suggestions (summarized from above):\n1. Train the MorphoDiff model on larger CellPainting datasets, to improve the learned relationship between perturbation and phenotype\n2. Evaluate model performance in the harder case of more subtle phenotypes\n3. Conduct additional evaluations on datasets with a larger number of MOA clusters, leveraging the evaluation framework from: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1012463\n4. Explore the possibility of updating the Perturbation Identifier Module with learned perturbation embeddings, which would provide a more useful output in the scenario of virtual phenotypic screening.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper is clearly described, including a thorough description of the model, datasets and training procedure.  It also includes links to code for data conversion, the pre-trained Stable Diffusion model, and author's fine-tuning script, which taken together provide a high level of reproducibility.\n\nOne of the main contributions of the paper is to apply state-of-the-art guided diffusion models to the task of virtual phenotypic screening.  Other papers have similarly applied generative models (GANs and flow-based models); however, this is the first application of diffusion models (that I'm aware of) to this task.  The generated images look high quality, despite fine-tuning on a relatively small training set of Cell Painting images.\n\nIn addition, the authors evaluate their method on 3 independent open-source datasets: Rxrx1, BBBC021 and Rohban.", "weaknesses": "Major concerns:\n\nMy primary concern with the paper is regarding benchmarking of the MorphoDiff model against other methods.  In particular, the only comparison that is included throughout the paper is a baseline of Stable Diffusion (without guidance by the Perturbation Identifier module).  While this comparison is positive in all cases, in my view it is equivalent to an ablation study, demonstrating that the model is leveraging the information in the perturbation embeddings.  It doesn't demonstrate if the model accuracy is sufficiently high to be useful in virtual phenotypic screening, or compare the model performance against other published models such as Mol2Image, IMPA, or GRAPE (https://arxiv.org/pdf/2406.07763).  I suggest including additional discussion and comparisons of other published methods, and expanding the analysis of MOA inference to a larger set of MOA classes, as it's a core evaluation metric for phenotypic screening.\n\nIn addition, while the method is nicely implemented and clearly explained in the paper, I find it does not have high novelty compared to other methods (Mol2Image, IMPA, GRAPE) for generating images of cellular morphology under perturbation, and therefore a rigorous comparison of MorphoDiff's performance relative to the existing state-of-the-art would be very important to demonstrate a significant contribution from the paper.\n\nAdditional concerns:\n\nThe MorphoDiff model was fine-tuned on very small datasets, each on the order of 1000 perturbations.  Much larger Cell Painting datasets have been released in recent years, such as JUMP (120,000 perturbations) and Rxrx3 (20,000 perturbations).  Demonstrating the capability of MorphoDiff on these datasets would make it more impactful related to the current state-of-the-art for virtual phenotypic screening, such as MoCoP (https://www.biorxiv.org/content/10.1101/2023.05.01.538999v3.full.pdf).  I suggest the authors discuss the potential challenges of applying diffusion models to large Cell Painting datasets, and how these could be overcome to advance the field of generative virtual phenotypic screening.\n\nThe images generated based on perturbations seen during training look quite good.  However, in each case, they are shown for very strong drug treatments, such as cytoskeletal destabilizers.  It would be informative to demonstrate if MorphoDiff can capture more subtle phenotypes that are typically present in phenotypic screens. \n\nIn the conclusion, the authors suggest that future work could establish standardized evaluation frameworks for this emerging field. \n I recommend to check out the following paper that provides a framework and numerous metrics for evaluating perturbative maps: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1012463\n\nIn Figure 3b, very few MOA clusters were considered.  I suggest leveraging datasets with a larger number of available MOAs / gene clusters.  One such option would be to show the 1108 siRNA perturbations in Rxrx1 (used in the paper); however, due to the large off-target effects of siRNA that may not be very meaningful.  In which case, JUMP would provide another option.\n\nIn Figure 3c, comparing NMI of morphoDiff on Perturbation ID doesn't seem meaningful.  I may have misunderstood this point, but it seems that the variation in Cell Profiler features extracted from morphoDiff with a constant Perturbation ID should reflect only the noise in the generation process.  Whereas the real data will have large technical effects between replicates.  As such, I would expect a generative model to be more self-consistent than the raw data (which is known to be very noisy at the replicate level).  I suggest focusing evaluation on the MOA clustering (ideally on an expanded MOA set), and perhaps investigating the effect of diversity/noise in the generation process on the accuracy of MOA clustering.\n\nIn Figure 4a, the correlation coefficient between CellProfiler features extracted from MorphoDiff vs features extracted from real images for the same perturbation is close to zero.  I suggest including a separate measure of the correlation of CellProfiler features between replicates (real images) of the same perturbation, to provide a rough upper-limit for how strongly we could expect the model to correlate with real data.  If the model correlation is similar to the replicate correlation, then it can be argued more meaningfully that the model generated images are \"interchangeable\" with real data.\n\nFinally, the authors observe that the similarity of images generated by MorphoDiff relies on the similarity of the output from the perturbation embedding module.  In other words, the model does not learn an improved perturbation embedding from the data, as for example Mol2Image and GRAPE do. As such, it's not clear to me that this approach would be informative in a virtual screening setting, where the goal is to learn a predictive embedding of unseen perturbations that improves generalization beyond eg, the performance of RDKit.  I suggest additional discussion on the implications of this limitation for virtual screening, and how they might be addressed through modifications to the Perturbation Embedding Module.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bv2PMn9chi", "reviewer": "Reviewer_RSvH", "summary": "The paper introduces MorphoDiff, a novel generative model designed to predict high-resolution cellular morphologies in response to different chemical and genetic perturbations. Using a latent diffusion model and a perturbation projection module, MorphoDiff generates images that mimic cellular phenotypes across multiple datasets. The model is validated through common generative modelling metrics, visual assessments, and biologically interpretable feature analysis, showing its ability to capture perturbation-specific and mechanism-of-action (MOA)-specific morphology. MorphoDiff aims to support drug discovery by enabling in silico exploration of large perturbational landscapes.", "questions": "1. **Clarity on Clustering Technique:**\n\nThe term “KNN clustering” is ambiguous. It’s unclear if the authors mean k-means clustering or a different method. Given the context of CellProfiler feature clustering, clarifying this point would improve readability and understanding of the methodology.\n\n2. **Assessing Combination Therapies:**\n\nCombination therapies, where multiple drugs or genetic modifications are applied simultaneously, are increasingly important in personalised medicine and complex disease treatment strategies. Assessing cellular responses to these combinations is crucial, as interactions between therapies can lead to synergistic effects, enhanced efficacy, or reduced side effects that might not be observed with single agents alone. Given MorphoDiff's promising results with single perturbations, could the model be adapted or extended to predict morphology for combination therapies? Specifically, what modifications would be necessary to capture the interactions between therapies accurately, and are there unique challenges in validating these predictions with current datasets?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. **Proposed Impact:**\n\nThe MorphoDiff framework’s potential to generate biologically meaningful cellular images based on perturbations could greatly impact drug discovery and cellular profiling. This innovation might streamline in silico analysis of large drug libraries and provide a baseline for future generative work in this domain.\n\n2. **Writing and Accessibility:**\n\nThe paper is well-organised and clearly written, allowing readers from diverse backgrounds to follow the technical details of the diffusion model, perturbation embedding, and validation steps.\n\n3. **Thorough Experimental Validation:**\n\nThe authors provide comprehensive computational, visual, and biological validations across multiple datasets, which demonstrates the versatility and robustness of MorphoDiff. The quantitative metrics, qualitative results, and the consistent comparative performance against Stable Diffusion enhance the reliability of their claims.\n\n4. **Biological Insight:**\n\nBy clustering CellProfiler features, the authors demonstrate that the generated images capture perturbation-specific cellular morphology that aligns with real biological signals. \n\n5. **Generality Across Chemical and Genetic Perturbations:**\nMorphoDiff’s capacity to handle diverse perturbation types — both genetic and chemical — indicates potential applications across a wide range of experimental conditions, making it a versatile tool in cellular morphology prediction.", "weaknesses": "1. **Statistical Significance Lacking Clarity:**\n\nThe term \"statistically significant\" is used frequently (e.g., in results regarding FID and KID metrics) without detailing which statistical tests were applied or providing p-values. The methodology section would benefit from specifying these statistical tests to support the significance claims convincingly. Please could the authors specify which statistical tests were used for comparing FID and KID metrics between MorphoDiff and the baseline, and to provide p-values where applicable.\n\n2. **Missing Citations for Established Findings:**\n\n Statements that reference existing biological knowledge, such as \"MorphoDiff images revealed biological evidence that corroborates established findings,\" should include appropriate citations. This would anchor these statements in established literature. Please provide specific examples of relevant literature or established findings that support the claims about biological evidence.\n\n3. **Limited comparison to other methods:**\n\n The authors compare MorphoDiff to Stable Diffusion, however, overlook other methods like PhenDiff. Although PhenDiff was proposed for cropped images, this can easily be extended. Could the authors extend their comparison to include PhenDiff? If not, could the authors discuss why this was left out of comparison?\n\n4. **Minor:**\n\n- There are occasional instances of incorrect citation formatting, which might distract readers or reduce clarity. For example, “...generative models in computer vision Rombach et al. (2022) and integrating state-of-the-art…”. Rombach et al. (2022) should appear in parentheses here. There are numerous other examples of this. Furthermore, citations like “Saha et al.” are missing the publication year, which should be updated to 2024​. \n- The sentence “comparing standard-processed and cropped images in the BBBC021 experiments showed a slight improvement…”, should point to the results. I assume this is referring to Appendix Table 5.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rcPixa9mLH", "reviewer": "Reviewer_qrds", "summary": "The authors propose MorphoDiff, a perturbation-based conditional latent diffusion model designed to generate high-resolution images of drug-treated cell morphologies. The model incorporates both chemical and genetic perturbations as conditioning inputs, enabling it to generalise across diverse interventions. Evaluations indicate that MorphoDiff exceeds the performance of a conventional Stable Diffusion model in replicating biologically realistic cellular responses, with strong results in both visual fidelity and biological interpretability.", "questions": "## Sensitivity to Diffusion Model Selection: \nHow sensitive is MorphoDiff to the choice of latent diffusion model? The authors use Stable Diffusion as a baseline, but further exploration into alternative pre-trained diffusion models could provide valuable insights.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "## Clear Structure and Motivating Background: \nThe paper is well-organised, with each section contributing logically to the overall narrative. The background provides a thorough review of core concepts in conditional diffusion modeling, highlighting the advantages of diffusion models for image generation tasks in biological contexts. \n\n## Innovative Use of scGPT for Perturbation Conditioning: \nThis integration of a large, pre-trained transformer model to embed meaningful genetic perturbation features enables MorphoDiff to generalise across complex perturbations with improved accuracy, potentially opening new avenues for generative applications in cellular biology. \n\n## Comprehensive Biological Validation Beyond Standard Generative Metrics: \nThe authors go beyond typical generative metrics such as FID by evaluating MorphoDiff’s performance on biological realism through CellProfiler, a tool widely used in biological research. This thorough approach highlights the biological validity of generated samples by extracting quantitative morphological features, offering a layer of interpretability and trustworthiness that is essential for practical applications in drug discovery. The use of CellProfiler also strengthens the model’s credibility by demonstrating the relevance of the generated images in a real biological context, not just in terms of visual fidelity.", "weaknesses": "## Limited Range of Comparative Baselines: \nWhile MorphoDiff is evaluated in terms of biological interpretability and generative quality, the paper lacks a sufficient range of comparative baselines, particularly given the advancements in conditional latent diffusion models. The authors mention MO2Image, yet they do not compare their approach to several other conditional diffusion models that could have served as relevant baselines. For instance, competing methods like PhenDiff [1] offer similar approaches in generating drug-treated cellular images using conditional diffusion. Expanding the range of baselines would solidify MorphoDiff’s competitive advantage and contextualise its performance relative to other SOTA models. \n\n## Clarification on Novelty Claims: \nThe claim that MorphoDiff is the first to handle both chemical and genetic perturbations in a conditional latent diffusion model is somewhat overstated, as methods like PhenDiff [1] have previously addressed similar challenges. Citing these comparable methods (or similar) and clearly articulating MorphoDiff’s unique contributions (e.g., specific model adaptations, or validation approach) would improve the transparency of the novelty claim. \n\n## Insufficient Detail in Key Methodological Components: \nCertain methodological sectionsc lack sufficient depth, making it difficult to reproduce the model. The perturbation projection module, which appears to be a crucial element of MorphoDiff’s design, is only briefly discussed. Given that this module enables MorphoDiff to conditionally generate diverse morphological responses, more information on how it was adapted for this task would enhance clarity and allow for reproducibility. Furthermore, the absence of code or a repository also hinders reproducibility.\n\n## Overlooked Metrics for Sample Variability: \nThe paper does not address generated sample variability or uniqueness within the output set for each condition. Evaluating generated sample uniqueness is particularly important in high-fidelity generation, as latent diffusion models can sometimes suffer from mode collapse, resulting in overly similar or even identical samples across iterations. Implementing additional metrics such as Structural Similarity Index (SSIM) could quantify the variability of generated samples within each condition, helping to ensure MorphoDiff generates a diverse range of morphologies. These metrics would also provide insight into the model's robustness and its ability to capture subtle differences in morphological responses across varying treatments.\n\n\n[1] https://arxiv.org/pdf/2312.08290", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)