Title: How Molecules Impact Cells: Unlocking Contrastive PhenoMolecular Retrieval
Venue: NeurIPS 2024
Accepted: yes
Keywords: Multi-Modality

Contrastive Learning

CLIP

Cell Morphology

Molecules

Molecular Retrieval

Zero-Shot Learning

Cell-Painting
Abstract: Predicting molecular impact on cellular function is a core challenge in therapeutic design. Phenomic experiments, designed to capture cellular morphology, utilize microscopy based techniques and demonstrate a high throughput solution for uncovering molecular impact on the cell. In this work, we learn a joint latent space between molecular structures and microscopy phenomic experiments, aligning paired samples with contrastive learning. Specifically, we study the problem of Contrastive PhenoMolecular Retrieval, which consists of zero-shot molecular structure identification conditioned on phenomic experiments. We assess challenges in multi-modal learning of phenomics and molecular modalities such as experimental batch effect, inactive molecule perturbations, and encoding perturbation concentration. We demonstrate improved multi-modal learner retrieval through (1) a uni-modal pre-trained phenomics model, (2) a novel inter sample similarity aware loss, and (3) models conditioned on a representation of molecular concentration. Following this recipe, we propose MolPhenix, a molecular phenomics model. MolPhenix leverages a pre-trained phenomics model to demonstrate significant performance gains across perturbation concentrations, molecular scaffolds, and activity thresholds. In particular, we demonstrate an 8.1 times improvement in zero shot molecular retrieval of active molecules over the previous state-of-the-art, reaching 77.33% in top-1% accuracy. These results open the door for machine learning to be applied in virtual phenomics screening, which can significantly benefit drug discovery applications.

How Molecules Impact Cells: Unlocking Contrastive PhenoMolecular Retrieval

Abstract

Predicting molecular impact on cellular function is a core challenge in therapeutic design. Phenomic experiments, designed to capture cellular morphology, utilize microscopy based techniques and demonstrate a high throughput solution for uncovering molecular impact on the cell. In this work, we learn a joint latent space between molecular structures and microscopy phenomic experiments, aligning paired samples with contrastive learning. Specifically, we study the problem of Contrastive PhenoMolecular Retrieval , which consists of zero-shot molecular structure identification conditioned on phenomic experiments. We assess challenges in multi-modal learning of phenomics and molecular modalities such as experimental batch effect, inactive molecule perturbations, and encoding perturbation concentration. We demonstrate improved multi-modal learner retrieval through (1) a uni-modal pre-trained phenomics model, (2) a novel inter sample similarity aware loss, and (3) models conditioned on a representation of molecular concentration. Following this recipe, we propose MolPhenix , a molecular phenomics model. MolPhenix leverages a pre-trained phenomics model to demonstrate significant performance gains across perturbation concentrations, molecular scaffolds, and activity thresholds. In particular, we demonstrate an 8.1 × improvement in zero shot molecular retrieval of active molecules over the previous state-of-the-art, reaching 77.33% in top-1% accuracy. These results open the door for machine learning to be applied in virtual phenomics screening, which can significantly benefit drug discovery applications.

1 Introduction

Quantifying cellular responses elicited by genetic and molecular perturbations represents a core challenge in medicinal research [4, 57]. Out of an approximate 10 60 druglike molecule designs, a small number are able to alter cellular properties to reverse the course of diseases [5, 27]. In recent years, microscopy-based cell morphology screening techniques, demonstrated potential for quantitative understanding of a molecule's biological effects. Experimental techniques such as cell-painting are used to capture cellular morphology, which correspond to physical and structural properties of the cell [6, 7]. Cells treated with molecular perturbations can change morphology, which is captured by staining and high throughput microscopy techniques. Perturbations with similar cellular impact induce analogous morphological changes, allowing to capture underlying biological

effects in phenomic experiments. Identifying such perturbations with similar morphological changes can aid in discovery of novel therapeutic drug candidates [50, 29, 22].

Determining molecular impact on the cell can be formulated as a multi-modal learning problem, allowing us to build on a rich family of methods [43, 62, 53]. Similar to text-image models, paired data is collected from phenomic experiments along with molecules used to perturb the cells. Contrastive objectives have been used as an effective approach in aligning paired samples from different modalities [43, 32]. A model that has learned a cross-modal joint latent space must be able to retrieve a molecular perturbant conditioned on the phenomic experiment. We identify this problem as contrastive phenomolecular retrieval (see Figure 2). Addressing this problem can allow for identification of molecular impact on cellular function, however, this comes with its own set of challenges. [18, 2, 65].

(1) Firstly, multi-modal paired phenomics molecular data suffers from lower overall dataset sizes and is subject to batch effects. Challenges with uniform processing and prohibitive costs associated with acquisition of paired data, leads to an order of magnitude fewer data points compared to text-image datasets [49, 11]. Furthermore, data is subject to random batch effects that capture non-biologically meaningful variation [33, 55]. (2) Paired phenomic-molecular data contains inactive perturbations that do not have a biological effect or do not perturb cellular morphology. It is difficult to infer a priori whether a molecule has a cellular effect, leading to the collection of paired molecular structures with unperturbed cells. These data-points are challenging to filter out without an effective phenomic embedding, as morphological effects are rarely discernible. These samples can be interpreted as misannotated, under the assumption of all collected pairs having biologically meaningful interactions. (3) Finally, a complete solution for capturing molecular effects on cells must capture molecular concentration. The same molecule can have drastically different effects along its dose response curve, thus making concentration an essential component for learning molecular impact.

In this work, we explore the problem of contrastive phenomolecular retrieval by addressing the above challenges circumvented in prior works. Our key contributions are as follows:

We demonstrate significantly higher phenomolecular retrieval rates by utilizing a pretrained unimodal phenomic encoder. Thus alleviating the data availability challenge, reducing the impact of batch effects, and identifying molecular activity levels.

We propose a novel soft-weighted sigmoid locked loss ( S2L ) that addresses the effects of inactive molecules. This is done by leveraging distances computed in the phenomic embedding space to learn inter-sample similarities.

We explore explicit and implicit methods to encode molecular concentration, assessing the model's ability to perform retrieval in an inter-concentration setting and generalize to unseen concentrations.

Explicit

Concentration

MAE

Phenomics

Experiments

Molecular

Structures

GNN

S2L

Guideline 3:

Implicit concentration

encoding with

S2L

Guideline 1:

Uni-modal pretrained

Phenomics and

Molecular models

Guideline 2:

Undersampling

inactive molecules

Vision

Encoder

Molecular

Encoder

Guideline 2:

Inter-sample

similarities

S2L

Guideline 3:

Explicit concentration

encoding

MolPhenix

+

Phenom1

MolGPS

Following these principles, we build MolPhenix , a multi-modal molecular phenomics model addressing contrastive phenomolecular retrieval (Figure 1). MolPhenix demonstrates large and consistent improvements in the presence of batch effects, generalizing across different concentrations, molecules, and activity thresholds. Additionally, MolPhenix outperforms baseline methods in zero-shot setting, achieving 77.33% top-1% retrieval accuracies on active molecules, which corresponds to a 8.1 × improvement over the previous state-of-the-art (SOTA) [48].

2 Related Work

Uni-modality Pretraining: Self-supervised methods have demonstrated success across a variety of domains such as computer vision, natural language processing and molecular representations [3, 44, 61]. In vision, contrastive methods have been used to minimize distance in the model's latent space of two views of the same sample [12, 51, 19, 21]. Reconstruction objectives have also permeated computer vision, such as masked autoencoders (MAE). MAEs typically utilize vision transformers to partition the image into learnable tokens and reconstruct masked patches [20, 17, 10, 14]. These methods have been extended to microscopy experimental data designed to capture cell morphology [60, 28]. Phenom1 utilizes a masked autoencoder with a ViT-L/8+ architecture and a custom Fourier domain reconstruction loss, yielding informative representations of phenomic experiments [28, 13]. From a representational perspective, Graph Neural Networks (GNN) have been used to predict molecular properties by reasoning over graph structures. A combination of reconstruction and supervised objectives have led to models generalizing to a diverse range of prediction tasks [36, 66, 56, 47]. Our work leverages uni-modal foundation models, which are used to generate embeddings of phenomic images and molecular graphs.

Multi-Modal Objectives: Multi-modal models combine samples from two or more domains, to learn rich representations and demonstrate flexible ways to predict sample properties [43, 1, 23]. Contrastive methods minimize distances between paired samples, traditionally in text-image domains. However, training these models is computationally expensive, requiring large datasets. Multiple contributions have allowed for a reduction in compute and data budgets by an order of magnitude. In LiT , the authors demonstrate that utilizing uni-modal pretrained models for one or both modalities matches zero-shot performance with an order of magnitude fewer paired examples seen [63]. Zhai et al. (2023) demonstrate that by replacing the softmax operation over cosine similarities with an element wise sigmoid loss, allows contrastive learners to improve performance under label noise regime [62]. By using a uni-modal pre-trained modal to calculate similarities between samples from one of the modalities, Srinivasa et al. (2023) have demonstrated improved performance on zero-shot evaluation [53]. In our work, we build along these directions in molecular phenomic multi-modal training.

Molecular-Phenomic Contrastive Learning: Prior works in contrastive phenomic retrieval have utilized the InfoNCE objective as a pre-training technique to construct uni-modal representations [38, 64]. Nguyen et al. (2023) propose a multi-modal objective trained on hand-engineered visual features and a GNN molecular encoder. The work demonstrates improved molecular property prediction with no image encoder pre-training [37, 54]. Recent methods have attempted to improve retrieval by using the InfoLOOB objective [41]. Specifically, CLOOME utilizes the InfoLOOB loss with hopfield networks for zero-shot retrieval on unseen data samples [45, 48]. InfoCORE aims to mitigate batch effects in multimodal molecular representations, improving retrieval capabilities and property prediction by adaptively reweighting samples to minimize confounding from non-biological associations [59]. Our work is parallel to the above directions, demonstrating a significant increase in molecular-phenomic retrieval by building on algorithmic improvements from the multi-modality literature.

3 Methodology

In this section, we explain key challenges facing phenomolecular retrieval and provide guidelines that are key methodological improvements behind the success of MolPhenix 1.

Preliminaries: Our setting studies the problem of learning multi-modal representations of molecules and phenomic experiments of treated cells [48]. The aim of this work is to learn a joint latent space which maps phenomic experiments of treated cells and the corresponding molecular perturbations into the same latent space. We consider a set of lab experiments E defined as the tuple ( X , M , C , Ψ ). Each experiment ϵ ∈ E consists of data samples x i ∈ X (such as images) and perturbations m i ∈ M

(such as molecules) which are obtained at a specific dosage concentration c i ∈ C , while ψ ∈ Ψ denotes molecular activity threshold.

Figure 2 describes the problem of contrastive phenomolecular retrieval, where for a single image x i , the challenge consists of identifying the matching perturbation, m i , and concentration, c i , used to induce morphological effects. This can be accomplished in a zero-shot way by generating embeddings for ( m 1 , c 1 ) , ... ( m j , c j ) and x i using functions f θ m ( m , c ) , f θ x ( x ) which map samples into R d . Then, by defining a similarity metric between generated embeddings z x i and z m i , f sim , we can rank ( m 1 , c 1 ) ... ( m j , c j ) based on computed similarities. An effective solution to the contrastive phenomolecular retrieval problem would learn f θ m ( m , c ) and f θ x ( x ) that results in consistently high retrieval rates of ( m i , c i ) used to perturb x i .

In practice, the image embeddings are generated using a phenomics microscopy foundation MAE model [28, 20]. We use phenomic embeddings to marginalize batch effects, infer inter-sample

Contrastive

PhenoMolecular

Retrieval

similarities, and undersample inactive molecules. Activity is determined using consistency of replicate measurements for a given perturbation. For each sample, a p value cutoff ψ ∈ Ψ is used to quantify molecular activity. Only molecules below the p value cutoff ψ are considered active.

Prior methods in multi-modal contrastive learning utilize the InfoNCE loss, and variants thereof [38] to maximize the joint likelihood of x i and m i . Given a set of N × N random samples ( x 1 , m 1 , c 1 ) , · · · , ( x N , m N , c N ) containing N positive samples at k th index and ( N -1) × N negative samples, optimizing Equation 1 maximizes the likelihood of positive pairs while minimizing the likelihood of negative pairs:

Where z x , z m correspond to phenomics and molecular embeddings respectively, τ is softmax temperature, and ⟨·⟩ corresponds to cosine similarity.

Challenge 1: Phenomic Pretraining and Generalization

We find that using a phenomics foundation model to embed microscopy images allows for mitigation of batch effects, reduces the required number of paired data points, and improves generalization in the process. While CLIP , a hallmark model in the field of text-image multi-modality, was trained on 400 million curated paired data points, there is an order of magnitude fewer paired molecular-phenomic molecule samples [43]. Cost and systematic pre-processing of data make large scale data generation efforts challenging, and resulting data is affected by experimental batch effects. Batch effects induce noise in the latent space as a result of random perturbations in the experimental process, while biologically meaningful variation remains unchanged [39, 52]. Limited dataset sizes and batch effects make it challenging for contrastive learners to capture molecular features affecting cell morphology, yielding low retrieval rates [48].

Weaddress data availability and generalization challenges by utilizing representations from a large unimodal pre-trained phenomic model , θ Ph, trained to capture representations of cellular morphology. θ Ph is pretrained on microscopy images using a Fourier modified MAE objective, utilizing the ViT-L/8 architecture with methodology similar to Kraus et al. (2024) [20, 14, 28]. For simplicity in future sections, we refer to this model as Phenom1 . This pretrained model allows a drastic

reduction in the required number of paired multi-modal samples [63]. In addition, using phenomic representations alleviates the challenge of batch effects by averaging samples, z x , generated with the same perturbation m i over multiple lab experiments ϵ i . Averaging model representations 1 N Σ 1 i ∈ N z x i allows marginalizing batch effect induced by individual experiments.

Guideline 1 Utilizing pre-trained uni-modal encoder, θ Ph, can be used to reduce the number of paired data-points compared to training θ without prior optimization. In addition, averaging phenomic embeddings z x from matched perturbations can alleviate batch effects.

To reason over molecular structures, we make use of features learned from GNNs trained on molecular property prediction [34]. We utilize a pretrained MPNN foundational model up to the order of 1B parameters for extracting molecular representations following a similar procedure to Sypetkowski et al. (2024) [56]. We refer to this model as MolGPS .

Challenge 2: Inactive Molecular Perturbations

The phenomics-molecular data collection process can result in pairing of molecular structures with unperturbed cells in cases where the molecule has no effect on cell morphology (Figure 3)

Since the morphological effects observed in cell x i is conditioned on the perturbation, in the absence of a molecular effect P ( x i | x 0 i , c i , m i ) ∼ P ( x i | x 0 i ) . In these samples, phenomic data will be independent, from paired molecular data, which results in misannotation under the assumption of data-pairs having an

underlying biological relationship. We demonstrate how utilizing Phenom1 to undersample inactive molecules and learn continuous similarities between samples can alleviate this challenge.

To undersample inactive molecules , we extract the embeddings from Phenom1 and calculate the relative activity of each perturbation ( m i , c i ) ∈ ( M , C ) . This is done using the rank of cosine similarities between technical replicates produced for a molecular perturbation against a null distribution. The null distribution is established by calculating cosine similarities from random pairs of Phenom1 embeddings generated with perturbation ( m j , c j ) , ( m k , c k ) . Hence, we can compute a p-value and filter out samples likely to belong to the null distribution with an arbitrary threshold ψ .

In addition, by utilizing an inter-sample aware S2L training objective , the model can learn similarities between inactive molecules. S2L is grounded in previous work which demonstrates improved robustness to label noise (SigLip) and learnable inter-sample associations (CWCL) [62, 53]. Continuous Weighted Contrastive Loss (CWCL) provides better multi-modal alignment using a uni-modal pretrained model to suggest sample distances, relaxing the negative equidistant assumption present in InfoNCE [53]:

CWCL weights logits with a continuous measure of similarity w X , resulting in better alignment of embeddings z x i and z m j across modalities. In equation 2, w X is computed using a within modality similarity function such as w X i,j = ⟨ z x i , z x j ⟩ / 2 + 0 . 5 . Note, the above formula is used only for mapping samples from modality M to X for which a pre-trained model θ Phis available.

Another work, SigLIP, demonstrates robustness to label noise and reduces computational requirements during contrastive training [62]. It does so by avoiding computation of a softmax over the entire set of in-batch samples, instead relying on element-wise sigmoid operation:

Algorithm 1 S2L loss pseudo-implementation.

In equation 3, α and b are learned, calibrating the model confidence conditioned on the ratio of positive to negative pairs. y i,j is set to 1 if i = j and -1 otherwise.

Inspired by prior works, we introduce S2L for molecular representation learning, which leverages inter-sample similarities and robustness to label noise to mitigate weak or inactive perturbations.

In the equation above, z x i and z m j correspond to latent representations of images and molecules, respectively. α and b correspond to learnable temperature and bias parameters for the calibrated sigmoid function. w X ij is an inter-sample similarity function computed from images using the pretrained model θ Ph. To compute w X i,j , we use the arctangent of L2 distance instead of cosine similarity, as was the case for Equation 2 (more details in Appendix D.3). Intuitively, S2L can be thought of as shifting from a multi-class classification to a soft multi-label problem. In our problem setting, the labels are continuous and determined by sample similarity in the phenomics space.

Guideline 2 When training a molecular-phenomic model, mitigating the effect of inactive molecules in training data distribution can be carried out by undersampling inactive molecules and using an inter-sample similarity aware, S2L loss (equation 4).

Challenge 3: Variable Concentrations

Perturbation effect on a cell is determined by both molecular structure and corresponding concentration [58]. A model capturing molecular impact on cell morphology must be able to generalize across different doses, since variable concentrations can correspond to different data distributions.

We note that providing concentrations c i as input to the model would benefit performance, as this would indicate the magnitude of molecular impact. However, we find that simply concatenating concentrations does not result in effective training due to its compressed dynamic range. To that end, we add concentration information in two separate ways: implicit and explicit formulations.

We add implicit concentration as molecular perturbation classes by using the S2L loss (Equation 4) to treat perturbation m i with concentrations c i and c j as distinct classes. This pushes samples apart in the latent space proportionally to similarities between phenomic experiments.

We add explicit concentration c i by passing it to the molecular encoder. We explore different formulation for dosage concentrations, f ′ ( c i ) , where f ′ maps c i → R . Encoded representations f ′ ( c i )

are concatenated at the initial layer of the model. We find simple functional encodings f ′ (such as one-hot and logarithm ) to work well in practice.

Guideline 3 When training a molecular-phenomic model, conditioning on an (implicit and explicit) representation of concentration f ′ ( c i ) aids in capturing molecular impacts on cell morphology and improves generalization to previously unseen molecules and concentrations.

4 Experimental Setup

In this section, we describe evaluation datasets used, and descriptions of the underlying data modalities. To assess phenomolecular retrieval, we use 1% recall metric unless stated otherwise, as it allows direct comparison between datasets with different number of samples. Additional implementation and evaluation details can be found in Appendix D.

Datasets: Our training dataset consists of fluorescent microscopy images paired with molecular structures and concentrations, which are used as perturbants. We assess models' phenomolecular retrieval capabilities on three datasets of escalating generalization complexity. First dataset, consisting of unseen microscopy images and molecules present in the training dataset. Second, a dataset consisting of previously unseen phenomics experiments and molecules split by the corresponding molecular scaffold. Finally, we evaluate on an open source dataset with a different data generating distribution [16]. In the case of the latter two datasets, the model is required to perform zero-shot classification, as it has no access to those molecules in the training data. This requires the model to reason over molecular graphs to identify structures inducing corresponding cellular morphology changes. Using methodology described in guideline 2 we report retrieval results for all molecules as well as on an active subset. Finally, all datasets are comprised of molecular structures at multiple concentrations ( . 01 , . 1 , 1 . 0 , 10 , etc.) Additional details regarding the datasets can be found in Appendix C.

Modality Representations: In our evaluations, we consider different representations for molecular perturbations and phenomic experiments and quantitatively evaluate their impact.

Images : Image encoders utilize 6-channel fluorescent microscopy images of cells representing phenomic experiments. Images are 2048 × 2048 pixels, capturing cellular morphology changes post molecular perturbation. We downscale each image to 256 × 256 using block mean downsampling.

Phenom1 : We characterize phenomic experiments by embedding high resolution microscopy images in the latent space of a phenomics model θ Ph as described in guideline 1.

Fingerprints : Molecular fingerprints utilize RDKIT [31], MACCS [30] and MORGAN3 [46] bit coding, which represent binary presence of molecular substructures. Additional information such as atomic identity, atomic radius and torsional angles are included in the fingerprint representations.

MolGPS : We generate molecular representations from a large pretrained GNN. Specifically, we obtain molecular embeddings from a 1B parameter MPNN [34].

Table 1: Impact of pre-trained Phenom1 and MolGPS on CLOOME and MolPhenix for a matched number of seen samples (Top), where we observe an 8.1 × improvement of MolPhenix over the CLOOME baseline for active unseen molecules. SOTA results trained with a higher number of steps by utilizing the best hyperparameters (Bottom *). We note that MolPhenix's main components such as S2L and embedding averaging relies on having a pre-trained uni-modal phenomics model.

5 Results and Discussion

To evaluate the effectiveness of Guidelines 1, 2, and 3 we carry out evaluation in two different settings: (1) cumulative concentrations, and (2) held-out concentrations, testing the models' ability to generalize to new molecular doses. Finally, we perform comprehensive ablations testing model performance with varying data, model, and optimization parameters. The comprehensive set of results can be found in Tables 10, 11, 12, and 13.

5.1 Evaluation on cumulative concentrations:

We demonstrate improvements in phenomolecular recall due to usage of a phenomics pre-trained foundation model, identify that MolPhenix set of design choices results in higher final performance, and more data efficient learning. Figure 4 demonstrates recall accuracy on all molecules and an active subset for CLOOMEand MolPhenix models, as a function of training samples seen.

We observe a large performance gap between models trained on Phenom1 embeddings as opposed to images, emphasizing the utility of using a pre-trained encoder for microscopy images (Table 1). We note that provision of Phenom1 (CLOOME-Phenom1 Vs CLOOMEImages) significantly improves both active and all molecule retrieval by 5 . 69 × and, 4 . 75 × respectively (Table 1).

Furthermore, we identify that while all molecules retrieval stagnates throughout training, the performance on an active subset keeps improving, underscoring the importance of identification of the active subset. Finally, we com-

10 6

10 7

Number of Samples Seen

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Top 1% Retrieval

- -   All Compounds

Active Compounds

Molecular Retrieval

Molphenix-Phenom1

CLOOME-Phenom1

CLOOME-Images

pare CLOOME and MolPhenix trained using Phenom1 embeddings and find there is a consistent

retrieval performance gap, throughout training, with a 1.26 × final improvement (Figure 4, Table 1). Compared to CLOOME [48] trained directly on images, MolPhenix achieves an average improvement of 8 . 78 × on active molecules on the unseen dataset. These results verify the effectiveness of Guideline 1 in accelerating training, and the importance of Guidelines 2 and 3 in recall improvements over CLOOME.

We evaluate the impact of different loss objectives on the proposed MolPhenix training framework. Table 2 presents top-1% retrieval accuracy across different contrastive losses utilized to train molecular-phenomics encoders on cumulative concentrations. Compared to prior methods, the proposed S2L loss demonstrates improved retrieval rates in cumulative concentration setting. Label noise and inter-sample similarity aware losses such as CWCL and SigLip also demonstrate improved performance. The effectiveness of S2L can be attributed to smoothed inter-sample similarities and implicit concentration information.

Finally, in Table 3, we observe recall improvements when considering both molecular structures and concentration. We note the importance of the addition of implicit concentration, further confirming the importance of considering molecular effects at different concentrations as different classes. Explicitly encoding molecular concentration with one-hot , logarithm and sigmoid yields improved recall performance, where one-hot performs the best in a cumulative concentration setting. These findings verify the efficacy of implicit and explicit concentration encoding outlined in Guideline 3.

Results are averaged across experiments for each dropped concentration, and across three seeds. Recall is reported for active molecules, while the results for all molecules can be found in Table 13.

5.2 Evaluation on held-out concentrations:

Next, we evaluate recall on held-out concentrations to obtain a measure of generalization performance. This evaluation allows us to capture the utility of our models for prediction of unseen concentrations, hence resembling in-silico testing. We omit concentrations from the training set and evaluate recall at the excluded data, where we observe a drop in retrieval performance for unseen concentrations. Similar to cumulative concentration results, we find that using S2L improves recall over other losses and outperforms CLOOME by up to 126% (Table 4). While one-hot encoding exhibits significant improvements in cumulative concentrations, its expressivity on unseen concentrations is limited (Table 5) and sigmoid encoding provides a sufficient representation of concentration.

5.3 Ablation Studies

We assess the importance of our design decisions by conducting an ablation study over our proposed guidelines. Figure 5 presents the variation of top-1% recall accuracy across key components such as cutoff p value, fingerprint type, and embedding averaging. We observe that employing a lower cutoff p value yields improved generalization for unseen dataset, while employing a higher cutoff appears to be optimal for unseen images + unseen molecules. For molecular structure representations, we find that using embeddings from the large pretrained MPNN graph based model (e.g., MolGPS) surpasses traditional fingerprints. Finally, utilization of embedding averaging demonstrates improved recall.

256

512

1024

0.4

0.6

0.8

1.0

Recall@1

Unseen Images

Unseen Images +

Unseen Molecules

Unseen Dataset

10 7

Number of Parameters (log scale)

0.4

0.6

0.8

1.0

128

512

2048

8192

0.4

0.6

0.8

1.0

0.01

0.1

1.0

Cut-Off p-value (log scale)

0.4

0.6

0.8

1.0

MACCS

RDKit

Morgan

Multi-FPs

MolGPS

Fingerprint

0.4

0.6

0.8

1.0

W/O Embedding-Avg

With Embedding-Avg

0.5

0.6

0.7

0.8

0.9

1.0

6 Conclusion

In this work, we investigate the problem of contrastive phenomolecular retrieval by constructing a joint multi-modal embedding of phenomic experiments and molecular structures. We identify a set of challenges afflicting molecular-phenomic training and proposed a set of guidelines for improving retrieval and generalization. Empirically, we observed that contrastive learners demonstrate higher retrieval rates when using representations from a high-capacity uni-modal pretrained model. Use of inter-sample similarities with a label noise resistant loss such as S2L allows us to tackle the challenge of inactive molecules. Finally, adding implicit and explicit concentrations allows models to generalize to previously unseen concentrations. MolPhenix demonstrates an 8.1 × improvement in zero shot retrieval of active molecules over the previous state-of-the-art, reaching 77.33% in top-1% accuracy. In addition, we conduct a preliminary investigation on MolPhenix's ability to uncover biologically meaningful properties (activity prediction, zero-shot biological perturbation matching, and molecular property prediction in Appendix E.1, E.2, and E.3, respectively.). We expect a wide range of applications for MolPhenix, particularly in drug discovery. While there's a remote chance of misuse for developing chemical weapons, such harm is unlikely, with our primary focus remaining on healthcare improvement.

Limitations and Future Work: While our study covers challenges in phenomolecular recall, we leave three research directions for future work. (1) Future investigations could consider studying additional modalities such as text, genetic perturbations and chemical multi-compound interventions. (2) While we propose and evaluate our guidelines on previously conducted phenomic experiments, we note that a rigorous evaluation would evaluate model predictions in a wet-lab setting. (3) In addition, our work makes the assumption that the initial unperturbed cell state x 0 i can be marginalized by utilizing a single cell line with an unperturbed genetic background. Future works can relax this assumption, aiming to capture innate intercellular variation.

Acknowledgments and Disclosure of Funding

We thank the broader Valence Labs team and Recursion Pharmaceuticals for support on the project. We thank Berton Earnshaw, Jason Hartford, Emmanuel Bengio, Oren Kraus, and Emmanuel Noutahi for providing valuable feedback on the manuscript.

References

J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, R. Ring, E. Rutherford, S. Cabi, T. Han, Z. Gong, S. Samangooei, M. Monteiro, J. Menick, S. Borgeaud, A. Brock, A. Nematzadeh, S. Sharifzadeh, M. Binkowski, R. Barreira, O. Vinyals, A. Zisserman, and K. Simonyan. Flamingo: a visual language model for few-shot learning, 2022.

S. Albelwi. Survey on self-supervised learning: auxiliary pretext tasks and contrastive learning methods in imaging. Entropy , 24(4):551, 2022.

R. Balestriero, M. Ibrahim, V. Sobal, A. Morcos, S. Shekhar, T. Goldstein, F. Bordes, A. Bardes, G. Mialon, Y. Tian, A. Schwarzschild, A. G. Wilson, J. Geiping, Q. Garrido, P. Fernandez, A. Bar, H. Pirsiavash, Y. LeCun, and M. Goldblum. A cookbook of self-supervised learning, 2023.

C. Bock, P. Datlinger, F. Chardon, M. A. Coelho, M. B. Dong, K. A. Lawson, T. Lu, L. Maroc, T. M. Norman, B. Song, G. Stanley, S. Chen, M. Garnett, W. Li, J. Moffat, L. S. Qi, R. S. Shapiro, J. Shendure, J. S. Weissman, and X. Zhuang. High-content crispr screening. Nature Reviews Methods Primers , 2(1), Feb. 2022.

R. S. Bohacek, C. McMartin, and W. C. Guida. The art and practice of structure-based drug design: A molecular modeling perspective. Medicinal Research Reviews , 16(1):3-50, Jan. 1996.

M. Boutros, F. Heigwer, and C. Laufer. Microscopy-based high-content screening. Cell , 163(6):1314-1325, 2015.

M.-A. Bray, S. Singh, H. Han, C. T. Davis, B. Borgeson, C. Hartland, M. Kost-Alimova, S. M. Gustafsdottir, C. C. Gibson, and A. E. Carpenter. Cell painting, a high-content imagebased assay for morphological profiling using multiplexed fluorescent dyes. Nature protocols , 11(9):1757-1774, 2016.

M.-A. Bray, S. Singh, H. Han, C. T. Davis, B. Borgeson, C. Hartland, M. Kost-Alimova, S. M. Gustafsdottir, C. C. Gibson, and A. E. Carpenter. Cell painting, a high-content imagebased assay for morphological profiling using multiplexed fluorescent dyes. Nature protocols , 11(9):1757-1774, 2016.

J. C. Caicedo, S. Cooper, F. Heigwer, S. Warchal, P. Qiu, C. Molnar, A. S. Vasilevich, J. D. Barry, H. S. Bansal, O. Kraus, et al. Data-analysis strategies for image-based cell profiling. Nature methods , 14(9):849-863, 2017.

S. Cao, P. Xu, and D. A. Clifton. How to understand masked autoencoders. arXiv preprint arXiv:2202.03670 , 2022.

S. N. Chandrasekaran, J. Ackerman, E. Alix, D. M. Ando, J. Arevalo, M. Bennion, N. Boisseau, A. Borowa, J. D. Boyd, L. Brino, P. J. Byrne, H. Ceulemans, C. Ch'ng, B. A. Cimini, D.-A. Clevert, N. Deflaux, J. G. Doench, T. Dorval, R. Doyonnas, V. Dragone, O. Engkvist, P. W. Faloon, B. Fritchman, F. Fuchs, S. Garg, T. J. Gilbert, D. Glazer, D. Gnutt, A. Goodale, J. Grignard, J. Guenther, Y. Han, Z. Hanifehlou, S. Hariharan, D. Hernandez, S. R. Horman, G. Hormel, M. Huntley, I. Icke, M. Iida, C. B. Jacob, S. Jaensch, J. Khetan, M. Kost-Alimova, T. Krawiec, D. Kuhn, C.-H. Lardeau, A. Lembke, F. Lin, K. D. Little, K. R. Lofstrom, S. Lotfi, D. J. Logan, Y. Luo, F. Madoux, P. A. Marin Zapata, B. A. Marion, G. Martin, N. J. McCarthy, L. Mervin, L. Miller, H. Mohamed, T. Monteverde, E. Mouchet, B. Nicke, A. Ogier, A.-L. Ong, M. Osterland, M. Otrocka, P. J. Peeters, J. Pilling, S. Prechtl, C. Qian, K. Rataj, D. E. Root, S. K. Sakata, S. Scrace, H. Shimizu, D. Simon, P. Sommer, C. Spruiell, I. Sumia, S. E. Swalley, H. Terauchi, A. Thibaudeau, A. Unruh, J. Van de Waeter, M. Van Dyck, C. van Staden, M. Warchoł, E. Weisbart, A. Weiss, N. Wiest-Daessle, G. Williams, S. Yu, B. Zapiec, M. ˙ Zyła, S. Singh, and A. E. Carpenter. Jump cell painting dataset: morphological impact of 136,000 chemical and genetic perturbations. bioRxiv , 2023.

T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and G. E. Hinton. Big self-supervised models are strong semi-supervised learners. Advances in neural information processing systems , 33:22243-22255, 2020.

M. Dehghani, J. Djolonga, B. Mustafa, P. Padlewski, J. Heek, J. Gilmer, A. P. Steiner, M. Caron, R. Geirhos, I. Alabdulmohsin, et al. Scaling vision transformers to 22 billion parameters. In International Conference on Machine Learning , pages 7480-7512. PMLR, 2023.

A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

K. Dunn, A. Aotaki-Keen, F. Putkey, and L. Hjelmeland. Arpe-19, a human retinal pigment epithelial cell line with differentiated properties. Experimental eye research , 62(2):155-170, 1996.

M. M. Fay, O. Kraus, M. Victors, L. Arumugam, K. Vuggumudi, J. Urbanik, K. Hansen, S. Celik, N. Cernek, G. Jagannathan, et al. Rxrx3: Phenomics map of biology. Biorxiv , pages 2023-02, 2023.

C. Feichtenhofer, Y. Li, K. He, et al. Masked autoencoders as spatiotemporal learners. Advances in neural information processing systems , 35:35946-35958, 2022.

A. Fürst, E. Rumetshofer, J. Lehner, V. T. Tran, F. Tang, H. Ramsauer, D. Kreil, M. Kopp, G. Klambauer, A. Bitto, et al. Cloob: Modern hopfield networks with infoloob outperform clip. Advances in neural information processing systems , 35:20450-20468, 2022.

J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. H. Richemond, E. Buchatskaya, C. Doersch, B. A. Pires, Z. D. Guo, M. G. Azar, B. Piot, K. Kavukcuoglu, R. Munos, and M. Valko. Bootstrap your own latent: A new approach to self-supervised learning, 2020.

K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16000-16009, 2022.

K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning, 2020.

M. Hofmarcher, E. Rumetshofer, D.-A. Clevert, S. Hochreiter, and G. Klambauer. Accurate prediction of biological assays with high-throughput microscopy images and convolutional networks. Journal of chemical information and modeling , 59(3):1163-1171, 2019.

S. Huang, L. Dong, W. Wang, Y. Hao, S. Singhal, S. Ma, T. Lv, L. Cui, O. K. Mohammed, B. Patra, Q. Liu, K. Aggarwal, Z. Chi, J. Bjorck, V. Chaudhary, S. Som, X. Song, and F. Wei. Language is not all you need: Aligning perception with language models, 2023.

W. E. Johnson, C. Li, and A. Rabinovic. Adjusting batch effects in microarray expression data using empirical bayes methods. Biostatistics , 8(1):118-127, 2007.

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 , 2020.

P. Kemmeren, K. Sameith, L. A. Van De Pasch, J. J. Benschop, T. L. Lenstra, T. Margaritis, E. O'Duibhir, E. Apweiler, S. van Wageningen, C. W. Ko, et al. Large-scale genetic perturbations reveal regulatory networks and an abundance of gene-specific repressors. Cell , 157(3):740-752, 2014.

C. Knox, M. Wilson, C. M. Klinger, M. Franklin, E. Oler, A. Wilson, A. Pon, J. Cox, N. E. L. Chin, S. A. Strawbridge, M. Garcia-Patino, R. Kruger, A. Sivakumaran, S. Sanford, R. Doshi, N. Khetarpal, O. Fatokun, D. Doucet, A. Zubkowski, D. Y. Rayat, H. Jackson, K. Harford, A. Anjum, M. Zakir, F. Wang, S. Tian, B. Lee, J. Liigand, H. Peters, R. Q. R. Wang, T. Nguyen, D. So, M. Sharp, R. da Silva, C. Gabriel, J. Scantlebury, M. Jasinski, D. Ackerman, T. Jewison, T. Sajed, V. Gautam, and D. S. Wishart. Drugbank 6.0: the drugbank knowledgebase for 2024. Nucleic Acids Research , 52(D1):D1265-D1275, Nov. 2023.

O. Kraus, K. Kenyon-Dean, S. Saberian, M. Fallah, P. McLean, J. Leung, V. Sharma, A. Khan, J. Balakrishnan, S. Celik, et al. Masked autoencoders for microscopy are scalable learners of cellular biology. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11757-11768, 2024.

O. Z. Kraus, B. T. Grys, J. Ba, Y. Chong, B. J. Frey, C. Boone, and B. J. Andrews. Automated analysis of high-content microscopy data with deep learning. Molecular systems biology , 13(4):924, 2017.

H. Kuwahara and X. Gao. Analysis of the effects of related fingerprints on molecular similarity using an eigenvalue entropy approach. Journal of Cheminformatics , 13:1-12, 2021.

G. Landrum et al. Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling. Greg Landrum , 8(31.10):5281, 2013.

F. Lanusse, L. Parker, S. Golkar, M. Cranmer, A. Bietti, M. Eickenberg, G. Krawezik, M. McCabe, R. Ohana, M. Pettee, B. R.-S. Blancard, T. Tesileanu, K. Cho, and S. Ho. Astroclip: Cross-modal pre-training for astronomical foundation models, 2023.

J. T. Leek, R. B. Scharpf, H. C. Bravo, D. Simcha, B. Langmead, W. E. Johnson, D. Geman, K. Baggerly, and R. A. Irizarry. Tackling the widespread and critical impact of batch effects in high-throughput data. Nature Reviews Genetics , 11(10):733-739, 2010.

D. Masters, J. Dean, K. Klaser, Z. Li, S. Maddrell-Mander, A. Sanders, H. Helal, D. Beker, A. Fitzgibbon, S. Huang, et al. Gps++: Reviving the art of message passing for molecular property prediction. arXiv preprint arXiv:2302.02947 , 2023.

D. Mendez, A. Gaulton, A. P. Bento, J. Chambers, M. De Veij, E. Félix, M. P. Magariños, J. F. Mosquera, P. Mutowo, M. Nowotka, et al. Chembl: towards direct deposition of bioassay data. Nucleic acids research , 47(D1):D930-D940, 2019.

O. Méndez-Lucio, C. Nicolaou, and B. Earnshaw. Mole: a molecular foundation model for drug discovery, 2022.

C. Q. Nguyen, D. Pertusi, and K. M. Branson. Molecule-morphology contrastive pretraining for transferable molecular representation. bioRxiv , pages 2023-05, 2023.

A. v. d. Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 , 2018.

H. S. Parker and J. T. Leek. The practical effect of batch on genomic prediction. Statistical applications in genetics and molecular biology , 11(3), 2012.

Z. Pincus and J. Theriot. Comparison of quantitative methods for cell-shape analysis. Journal of microscopy , 227(2):140-156, 2007.

B. Poole, S. Ozair, A. Van Den Oord, A. Alemi, and G. Tucker. On variational bounds of mutual information. In International Conference on Machine Learning , pages 5171-5180. PMLR, 2019.

D. L. Purich. Enzyme kinetics: catalysis and control: a reference of theory and best-practice methods . Elsevier, 2010.

A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748-8763. PMLR, 2021.

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models are unsupervised multitask learners. 2019.

H. Ramsauer, B. Schäfl, J. Lehner, P. Seidl, M. Widrich, T. Adler, L. Gruber, M. Holzleitner, M. Pavlovi´ c, G. K. Sandve, et al. Hopfield networks is all you need. arXiv preprint arXiv:2008.02217 , 2020.

D. Rogers and M. Hahn. Extended-connectivity fingerprints. Journal of chemical information and modeling , 50(5):742-754, 2010.

Y. Rong, Y. Bian, T. Xu, W. Xie, Y. Wei, W. Huang, and J. Huang. Self-supervised graph transformer on large-scale molecular data, 2020.

A. Sanchez-Fernandez, E. Rumetshofer, S. Hochreiter, and G. Klambauer. Cloome: contrastive learning unlocks bioimaging databases for queries with chemical structures. Nature , 2023.

C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems , 35:25278-25294, 2022.

J. Simm, G. Klambauer, A. Arany, M. Steijaert, J. K. Wegner, E. Gustin, V. Chupakhin, Y. T. Chong, J. Vialard, P. Buijnsters, et al. Repurposing high-throughput image assays enables biological activity prediction for drug discovery. Cell chemical biology , 25(5):611-618, 2018.

K. Sohn. Improved deep metric learning with multi-class n-pair loss objective. In Proceedings of the 30th International Conference on Neural Information Processing Systems , NIPS'16, page 1857-1865, Red Hook, NY, USA, 2016. Curran Associates Inc.

C. Soneson, S. Gerster, and M. Delorenzi. Batch effect confounding leads to strong bias in performance estimates obtained by cross-validation. PloS one , 9(6):e100335, 2014.

R. S. Srinivasa, J. Cho, C. Yang, Y. M. Saidutta, C.-H. Lee, Y. Shen, and H. Jin. Cwcl: Crossmodal transfer with continuously weighted contrastive loss. Advances in Neural Information Processing Systems , 36, 2023.

D. R. Stirling, M. J. Swain-Bowden, A. M. Lucas, A. E. Carpenter, B. A. Cimini, and A. Goodman. Cellprofiler 4: improvements in speed, utility and usability. BMC bioinformatics , 22:1-11, 2021.

M. Sypetkowski, M. Rezanejad, S. Saberian, O. Kraus, J. Urbanik, J. Taylor, B. Mabey, M. Victors, J. Yosinski, A. R. Sereshkeh, et al. Rxrx1: A dataset for evaluating experimental batch correction methods. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4284-4293, 2023.

M. Sypetkowski, F. Wenkel, , F. Poursafaei, N. Dickson, K. Suri, P. Fradkin, and D. Beaini. On the scalability of foundational models for molecular graphs. arxiv , 2024.

F. Vincent, A. Nueda, J. Lee, M. Schenone, M. Prunotto, and M. Mercola. Phenotypic drug discovery: recent successes, lessons learned and new directions. Nature Reviews Drug Discovery , 21(12):899-914, 2022.

R. M. Walmsley and N. Billinton. How accurate is in vitro prediction of carcinogenicity? British Journal of Pharmacology , 162(6):1250-1258, Feb. 2011.

C. Wang, S. Gupta, C. Uhler, and T. Jaakkola. Removing biases from molecular representations via information maximization, 2023.

R. Xie, K. Pang, G. D. Bader, and B. Wang. Maester: Masked autoencoder guided segmentation at pixel resolution for accurate, self-supervised subcellular structure recognition. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, June 2023.

S. Zaidi, M. Schaarschmidt, J. Martens, H. Kim, Y. W. Teh, A. Sanchez-Gonzalez, P. Battaglia, R. Pascanu, and J. Godwin. Pre-training via denoising for molecular property prediction, 2022.

X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer. Sigmoid loss for language image pretraining. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 11975-11986, 2023.

X. Zhai, X. Wang, B. Mustafa, A. Steiner, D. Keysers, A. Kolesnikov, and L. Beyer. Lit: Zero-shot transfer with locked-image text tuning, 2022.

S. Zheng, J. Rao, J. Zhang, L. Zhou, J. Xie, E. Cohen, W. Lu, C. Li, and Y. Yang. Cross-modal graph contrastive learning with cellular images. Advanced Science , 11(32), June 2024.

Y. Zhong, H. Tang, J. Chen, J. Peng, and Y.-X. Wang. Is self-supervised learning more robust than supervised learning? arXiv preprint arXiv:2206.05259 , 2022.

G. Zhou, Z. Gao, Q. Ding, H. Zheng, H. Xu, Z. Wei, L. Zhang, and G. Ke. Uni-mol: A universal 3d molecular representation learning framework. In The Eleventh International Conference on Learning Representations , 2023.

7 NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: In the abstract, we claim that we build a multi-modal molecular-phenomics model and demonstrate improvements over prior works. This is done by taking using a uni-modal pre-trained phenomics model, tackling inactive molecules by undersampling and learning inter-sample similarities. In addition, we take into account concentration in our model training. We demonstrate comprehensive results supporting these claims.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: In the conclusion, we have a limitations subsection discussing future research directions and assumptions in our work.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: Our work does not contain proofs or theorems.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Our work documents our design decisions in detail and has comprehensive details about the underlying dataset. We document all our hyperparameter choices and model architectural decisions. Our evaluation is performed on a publicly accessible dataset RXRX3, allowing for benchmarking of other methods. To reproduce the pre-trained phenomics model, we base our architecture on the work from [28], for which they have also provided access to a snakker model, namely Phenom-Beta via a web platform hosted on the BioNeMo platform https://www.rxrx.ai/phenom . To reproduce the pre-trained molecular model, we based our architecture on [56], for which the authors provide all the code and data needed to reproduce it. We further note that the molecular model can be replaced by simple molecular fingerprints with only a slight drop in performance.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: As part of the submission, we are unable to provide code to reproduce model training due to use of its proprietary nature. The training dataset is also an asset of a private institution, meaning that we are unable to be made publicly accessible. The unseen dataset RXRX3 is, however, open source and can be used by the community to evaluate public phenomics models.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide details regarding our hyperparameter choices in the Appendix C. In addition we document the use of scaffold splitting for Unseen Molecules & Images dataset. Unseen Dataset RXRX3 is publicly accessible.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Our reported results are averaged over 3 random seeds used to initialize the model and dictating stochasticity during model training. We report most standard deviations in the main text, and the remaining ones are all present in the Appendix E.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide details on compute time for each experiment in Appendix D.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research described does not violate the NeurIPS Code of Ethics. Our experiments do not include human subjects, we follow fair use of data, privacy, and do not release model weights for mitigating impact measures.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Our work discusses the potential in which MolPhenix can have positive societal impact and we touch on the extenralities in our concluding statements.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: In our work we do not release model weights or the underlying code.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Assetts used are referenced and licenses checked or otherwise not released publicly.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing not human subject research.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing not human subject research.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

A Glossary

Cell morphology: The examination and characterization of cellular structure, including shape, size, and organization, which can provide insights into cellular function and health [40].

Cell line (ARPE-19): A specific immortalized human retinal pigment epithelial cell line, widely utilized in ophthalmic research due to its differentiated properties and stability [15].

Molecular concentration: The quantitative measure of a specific molecule's abundance within a defined volume, typically expressed in molar units [42].

Cell staining: A laboratory technique involving the application of dyes or fluorescent markers to enhance the visibility and differentiation of cellular components under microscopic examination [9].

Molecular perturbations: Induced alterations in cellular molecular systems, often used to study cellular responses and regulatory networks [26].

Inactive molecule perturbations: Cellular system alterations caused by molecules lacking significant biological activity.

Batch effects: Systematic non-biological variation between groups of samples in an experiment, resulting from technical or experimental factors rather than true biological differences. This phenomenon is commonly observed in high-throughput molecular biology experiments, such as microarray studies, mass spectrometry, and single-cell RNA sequencing [24].

Molecular fingerprints: Distinctive patterns of molecular features that characterize specific cellular states or responses, often used for comparative analyses and classification [8].

B Assumption of the Initial Cell State

There is an important distinction between phenomics - molecule and text - image contrastive training although there are initial similarities. In the text - image domain the two modalities are directly generated by the same latent variable which is the underlying semantic class. Whereas in phenomics molecule, the observed phenomics variable is actually conditioned on molecular structure and the initial state. There are two important conclusions from this: (1) This indicates that if molecular structure has no effect on the initial cell state, there will not be a positive pairing between the molecular structure and morphological patterns captured by phenomics, making it indistinguishable from a control image. (2) There is an underlying assumption that the initial cell state x 0 i is constant. In accordance with this assumption we utilize experiments with a fixed cell line, HUVEC-19 , and a constant genetic background. Future works can relax this assumption by taking into account phenomics experiments of the cells prior to the perturbation. This can allow the models to generalize beyond a single cell line and to diverse genetic backgrounds.

C Dataset

Models have been trained using our in house training set and we have conducted our evaluation on two novel datasets and an open-source molecule dataset [16]:

Training Set : Our training dataset comprises 1,316,283 pairs of molecules and concentration concentration combinations, complemented by fluorescent microscopy images generated through over 2,150,000 phenomic experiments.

Evaluation set 1 -Unseen Images + Seen Molecules : The first set consists of unseen images and seen molecules. Unseen microscopy images are associated with 15,058 pairs of molecules and concentrations from the training set and selected randomly.

Evaluation set 2 -Unseen Images + Unseen Molecules : The second set includes previously unseen molecules, and images (consisting of 45,771 molecule and concentration pairs). Predicting molecular identities of previously unseen molecular perturbations corresponds to zeroshot prediction. Scaffold splitting was used to split this validation dataset from training ensuring minimal information leakage.

Evaluation set 3 -Unseen Dataset : Finally, we utilize the RXRX3 dataset [16], an opensource out of distribution (OOD) dataset consisting of 6,549 novel molecule and concentration

pairs associated with phenomic experiments. The distribution of molecular structures differs from previous datasets, making this a challenging zero-shot prediction task.

C.1 Concentration Details

Additional details regarding the number of molecules at significant concentrations of each evaluation set are available in Table 6.

D Implementation Details

In our experiments we report the top 1% recall metric as it is agnostic to the size of the dataset used. Across different datasets, top 1 metric can correspond to varying levels of difficulty due to the number of negatives evaluated. Top 1% can be used to compare models with different batch sizes, datasets, and evaluations with different number of concentrations.

D.1 Hyperparameters

Our design choices and utilized hyperparameters for is presented in Table 7. We set batch size to 512 through experiments presented in top section of Table 1 and Figure 4 since training CLOOME model on images is not efficient compared to using pretrained models. In addition, results presented at bottom section of Table 1 are based on the best parameters found through described ablation studies (section E.5).

D.2 Resource Computation

We utilized an NVIDIA A100 GPU to train Molphenix using Phenom1 and MolGPS embeddings, which takes approximately ∼ 4.75 hours each. For loss comparison experiments, we run each model using 3 different seeds and 8 different losses, resulting in a total of 114 hours of GPU processing time. For concentration experiments we train 7 runs, one for each concentration, with 3 seeds each

-1.00

-0.50

0.00

0.50

1.00

arctan distance

0.0

0.2

0.4

0.6

0.8

1.0

Proportion

random mol

high pval

low pval

high-low

CDF of arctan distances across Phenom1 Perturbations

-1.00

-0.50

0.00

0.50

1.00

cosine distance

0.0

0.2

0.4

0.6

0.8

1.0

Proportion

random mol

high pval

low pval

high-low

CDF of cosine distances across Phenom1 Perturbations

totaling 21 runs per set of parameters. With 25 sets of parameters evaluated (13), that amounts to 2,500 A100 compute hours. Moreover, we employed 8 NVIDIA A100 GPUs to train CLOOME model on phenomics images, with an average of 40 hour usage per run. Across three seeds, that amounts to ∼ 1000 hours of A100 GPU usage (8 GPUs for 40 hours 3 times).

Note that, without accounting for the time to train Phenom1, MolPhenix is 8.4 × faster than the CLOOME baseline.

D.3 S2L Distance function

To calculate inter sample distances, we utilize arctangent of l2 distances between Phenom1 embeddings. More specifically, we calculate distances with

where c is a constant indicating the median l2 distance between a null set of embeddings. Empirically, we've found that setting similarities below a threshold k to 0 improves model performance: ⌈ w ⌉ k .

Usage of arctan-l2 distances is motivated by an observation that cosine similarities do not effectively separate inactive molecules from other molecular pairs (Figure 6). To alleviate inactive molecule challenge, we require significant separation of CDF curves of inactive perturbations (p value > .9) and active molecules (p < .01). We observe that in both the plots using arctangent and cosine similarities achieves this purpose. However, if we compare high p-value curves with high-low, we find that in the case of cosine similarities they are almost identical. This indicates that the distribution of cosine similarities between active - inactive molecules is almost identical to that of inactive - inactive molecules. In contrast, when using arctangent similarities, we observe that the two CDF curves are well separated.

This property of l2 distances can inform our model training to identify inactive-inactive molecules. These results informed our decision to utilize arctangent of l2 distances to specify sample similarities for the S2L loss.

D.4 Intuition for S2L Loss

In this section we aim to provide some additional intuition for the S2L loss and further relate it to previous works. We will first assess the conceptual similarities between InfoNCE and CWCL loss and then justify a similar extrapolation for the relationship between S2L and SigLIP losses.

InfoNCE can be considered a special case of the CWCL loss where w ij is set to 0 for all pairs of i and j unless i = j . Conceptually this is equivalent to stating that all the negative pairs are equally distant from the reference sample. We will consider a uni directional loss CWCL, for identifying X from M :

̸

If we set w ij = 0 when i = j and 1 otherwise then the term Σ N j =1 w X i,j evaluates to 1 and the above expression simplifies to:

In the case of CWCL, a non 0 w i,j determined by a within modality similarity function informed by a pre-trained model, allows for an additional inductive bias. It is especially beneficial in a training setting with a limited dataset-size and in the presence of inactive negative molecules.

̸

Similarly SigLip can be considered a special case of S2L when w X i,j = 0 when i = j and w X i,j = 1 in the case i = j . This is the formulation of S2L

̸

It can be simplified to SigLIP by setting w X i,j to 1 when y i,j = 1 thus setting the term (1 -w X i,j ) 1+exp ( α z x i · z m j + b ) , corresponding to i = j , we set w X i,j to 0 thus negating the first part of the L S 2 L loss, evaluating to:

Having a 0 ≤ w X i,j ≤ 1 allows us to inform the training by going between discrete negative labels to continuous informed by some prior information. This information is given by a pre-trained encoder θ Phi , in our case but can be informed by any pre-trained model.

There are a lot of domain specific choices that can be made to inform the choice for setting w X i,j which we discuss in the appendix. Briefly, we identify that a modified l 2 loss is most effective for identifying inactive molecules.

E Additional Results

E.1 Predicting molecular activity

Given the significance of identifying active molecules, we evaluate the ability of the chemical encoder to predict molecular activity. To do so, we assessed whether embeddings generated from the chemical

encoder can be used to predict calculated p-values for unseen molecules. We fit a logistic regression on molecular embeddings from the training set, classifying whether a molecular perturbation and concentration have a p-value below .01. We find that the trained logistic regression is capable of predicting molecular activity on two downstream datasets with a non-overlapping set of molecules, Figure 8. In addition, we provide a u-map of molecular embedding for the unseen dataset RXRX3, colored by p-value. We qualitatively observe a clustering of active molecules using a U-map (Figure 7). It demonstrates that predicting compounds activity is possible using MolPhenix chemical encoder as molecules representations are distinct, independent of the experimented dosage concentration.

UMAP Mol embeddings RXRX3 colored by p-values

0.01

0.1

1.0

UMAPMol embeddings RXRX3

UMAP Mol embeddings RXRX3

colored by p-values

colored by p-values

0.01

0.01

0.1

1.0

0.1

1.0

Concentration:0.25uM

Concentration:1.0uM

UMAP Mol embeddings RXRX3

UMAP Mol embeddings RXRX3

colored by p-values

colored by p-values

t0'0

0.1

1.0

10'0

1.0

0.1

Concentration:2.5uM

Concentration:10.0uM

0.0

0.2

0.4

0.6

0.8

1.0

FPR

0.0

0.2

0.4

0.6

0.8

1.0

TPR

ROC curve New Dataset

AUC=0.90

0.0

0.2

0.4

0.6

0.8

1.0

FPR

0.0

0.2

0.4

0.6

0.8

1.0

TPR

ROC curve Validation New Molecules

E.2 Zero Shot Biological Validation

We conduct a preliminary investigation into whether MolPhenix can be used to identify biological relationships without the need for conducting the underlying experiments. To this end, we evaluate on a subset of ChEMBL with curated pairs of gene knockouts and molecular perturbants [35]. These pairs of perturbations were curated due to the similarity of their effects on cells, although these might not always be captured through phenomic experiments. Thus, there is maximum performance that can be reached through just phenomic data, which we assume to be achieved by experimental data embedded using Phenom1.

To evaluate MolPhenix's ability to identify previously known biological associations directly from data, we embed phenomics experiments from gene knockouts using the vision encoder. To perform in-silico screening, we then embed the molecular structures associated with positive pairs using the chemical encoder. Generating molecular embeddings and the corresponding concentrations does not utilize any experimental data. We then calculate cosine similarities between embeddings of phenomics experiments evaluating gene knockouts, and representations of the chemical representations along with encoded concentrations. Using the computed cosine similarities we are then able to assess whether MolPhenix is capable of identifying known associations between gene knockouts and molecular structures. Since there is no information on molecular concentration at which the cells must be treated with, we repeat the experiment across 4 concentrations. To get a null distribution of cosine similarities we take pairs of genes knockouts and molecules for which there are no annotated relationships. We calculate a cut-off for a low and high percentiles, and then evaluate what percentage of pairs of genes and molecules with known relationships exceed the set thresholds.

Figure 9 demonstrates that in-silico screening using MolPhenix Molecular encoder is capable of recovering a significant portion of known interactions. This is performed without the use of experimental data on the molecular encoder. It is difficult to estimate an upper bound on the expected performance due to uncertainty in the quality of curation of known pairs, presence of unknown associations between genes and molecules, and uncertainty regarding molecular concentration. There is a clear trend however that MolPhenix molecular encoder is capable of recovering a meaningful fraction of these interactions.

0.5-99.5

1-99

2-98

5-95

10-90

0.0

0.1

0.2

0.3

0.4

0.5

Recall

Concentration: 0.0025

0.5-99.5

1-99

2-98

5-95

10-90

Concentration: 0.01

0.5-99.5

1-99

2-98

5-95

10-90

Concentration: 0.025

0.5-99.5

1-99

2-98

5-95

10-90

Concentration: 0.1

Model, p value: 0.1

(Experimental)

MolPhenix-Molecular

(In-Silico)

Cosine Similarity Percentiles (log scale)

Phenom1

E.3 Molecular Property Prediction

We expand our evaluation with additional experiments supporting the utility of MolPhenix beyond retrieval. We conduct a KNN evaluation of the MolPhenix latent space, assessing the learned embedding on 35 molecular property prediction tasks across the Polaris and TDC datasets (Table 8 and 9). We find that MolPhenix trained with fingerprint embeddings consistently outperforms standalone input fingerprints, demonstrating that the MolPhenix latent space effectively clusters molecules according to their biological properties. We observed an interesting effect where prediction quality is positively correlated with implied dosage, indicating that MolPhenix learns dosage-specific effects. In addition, utilizing

concentration

adme-fang-HCLint-1

adme-fang-HPPB-1

adme-fang-PERM-1

adme-fang-RCLint-1

adme-fang-RPPB-1

adme-fang-SOLU-1

ames

bbb_martins

bioavailability_ma

caco2_wang

clearance_hepatocyte_az

cyp2c9_substrate_carbonmangels

cyp2c9_veith

cyp2d6_substrate_carbonmangels

cyp2d6_veith

cyp3a4_substrate_carbonmangels

dili

half_life_obach

herg

hia_hou

ld50_zhu

lipophilicity_astrazeneca

pgp_broccatelli

pkis2-egfr-wt-c-1

pkis2-egfr-wt-r-1

pkis2-kit-wt-c-1

pkis2-kit-wt-r-1

pkis2-ret-wt-c-1

pkis2-ret-wt-r-1

ppbr_az

solubility_aqsoldb

vdss_lombardo

TDC Standardized Mean

Standardized Mean

metric

pearson

pearson

pearson

pearson

pearson

pearson

auroc

auroc

auroc

mae

spearman

spearman

auprc

auprc

auprc

auprc

auroc

auprc

auroc

spearman

auroc

auroc

mae

mae

auroc

auprc

pearson

auprc

pearson

auprc

pearson

mae

mae

spearman

rdkit

0.32

0.34

0.48 0.23

0.38 0.29 0.69

0.72 0.58 -0.54 0.25 0.45 0.31 0.45 0.45 0.29 0.54

0.59 0.71 0.26 0.61 0.71 -0.70 -0.84

0.76 0.20 0.42 0.33 0.53 0.36 0.45

-13.03

-1.63 0.22

-2.62

-0.51

-1.88

ecfp

0.46

0.60

0.49 0.43

0.60 0.39 0.69

0.75 0.48 -0.43 0.37 0.50 0.32 0.52 0.44 0.33 0.60

0.64 0.67 0.47 0.73 0.65 -0.73 -0.78

0.79 0.41 0.57 0.33 0.51 0.40 0.55

-9.91

-1.27 0.47

-1.96

0.03

-1.26

0.37

0.56

0.52 0.22

0.43 0.44 0.71

0.77 0.53 -0.47 0.35 0.42 0.32 0.49 0.45 0.32 0.62

0.61 0.75 0.43 0.66 0.70 -0.66 -0.83

0.79 0.21 0.35 0.25 0.32 0.44 0.49

-10.13

-1.47 0.46

-1.91

-0.45

-1.40

Concatnated fps

0.41

0.66

0.58 0.33

0.40 0.37 0.70

0.77 0.58 -0.43 0.38 0.52 0.33 0.54 0.42 0.33 0.57

0.62 0.74 0.45 0.70 0.72 -0.67 -0.80

0.84 0.36 0.56 0.34 0.57 0.44 0.57

-10.94

-1.46 0.48

-1.78

0.00

-1.15

Molphenix fingerprint

1

0.57

0.75

0.57 0.55

0.72 0.57 0.70

0.74 0.54 -0.48 0.29 0.46 0.32 0.57 0.47 0.38 0.59

0.64 0.77 0.55 0.67 0.69 -0.71 -0.70

0.80 0.20 0.41 0.30 0.43 0.31 0.39

-8.93

-1.10 0.55

-1.64

0.14

-1.01

Molphenix fingerprint

25

0.64

0.71

0.65 0.62

0.67 0.58 0.69

0.78 0.54 -0.42 0.30 0.43 0.32 0.56 0.49 0.42 0.60

0.67 0.77 0.38 0.69 0.74 -0.67 -0.66

0.84 0.17 0.42 0.32 0.39 0.37 0.46

-8.43

-1.02 0.50

-1.40

0.26

-0.82

concentration

adme-fang-HCLint-1

adme-fang-HPPB-1

adme-fang-PERM-1

adme-fang-RCLint-1

adme-fang-RPPB-1

adme-fang-SOLU-1

ames

bbb_martins

bioavailability_ma

caco2_wang

clearance_hepatocyte_az

cyp2c9_substrate_carbonmangels

cyp2c9_veith

cyp2d6_substrate_carbonmangels

cyp2d6_veith

cyp3a4_substrate_carbonmangels

dili

half_life_obach

herg

hia_hou

ld50_zhu

lipophilicity_astrazeneca

pgp_broccatelli

pkis2-egfr-wt-c-1

pkis2-egfr-wt-r-1

pkis2-kit-wt-c-1

pkis2-kit-wt-r-1

pkis2-ret-wt-c-1

pkis2-ret-wt-r-1

ppbr_az

solubility_aqsoldb

vdss_lombardo

TDC Standardized Mean

Standardized Mean

metric

pearson

pearson

pearson

pearson

pearson

pearson

auroc

auroc

auroc

mae

spearman

spearman

auprc

auprc

auprc

auprc

auroc

auprc

auroc

spearman

auroc

auroc

mae

mae

auroc

auprc

pearson

auprc

pearson

auprc

pearson

mae

mae

spearman

MolGPS

0.54

0.66 0.70

0.56 0.64 0.55

0.69 0.76

0.49 -0.50 0.40 0.57 0.30 0.62 0.50 0.41

0.66

0.68 0.81 0.52 0.70

0.74 -0.69

-0.71

0.84 0.34 0.51

0.44 0.55 0.30

0.48

-9.71 -0.98 0.63

-1.19

0.36

-0.65

Molphenix with Molgps

1 0.60

0.78 0.69

0.61 0.68 0.65

0.70 0.79

0.59 -0.49 0.36 0.51 0.29 0.62 0.55 0.42

0.58

0.67 0.72 0.45 0.74

0.79 -0.71

-0.65

0.83 0.14 0.33

0.34 0.44 0.32

0.42

-8.28 -1.00 0.63

-1.23

0.29

-0.69

Molphenix with Molgps

25 0.68

0.74 0.70

0.67 0.77 0.63

0.71 0.78

0.60 -0.47 0.38 0.53 0.33 0.62 0.50 0.43

0.66

0.67 0.79 0.40 0.73

0.83 -0.70

-0.62

0.84 0.12 0.29

0.41 0.45 0.29

0.43

-8.46 -0.97 0.62

-.92

0.38

-0.46

E.4 Addressing Challenges in Contrastive Phenomic Retrieval

Table 10 and 12 show the complete Top 1% and 5% results of evaluation on cumulative concentrations on only active and all molecules, respectively. Similarly, Table 11 and 13 presents the full retrieval results of held-out concentrations experiments. In comparison to prior loss functions, S2L loss objective demonstrates consistent high retrieval rate in all tasks and molecular groups (i.e. all or active molecules), while using the same modality (Phenom1) and with or without explicit concentration information.

E.5 Ablation Studies

Figure 10 and Table 15, 16, 17, 18 and 19 present top-1% recall accuracy across for the full ablation study on the variation of MolPhenix key components. We note that compact embedding sizes from pretrained models stabilize training. This indicates that embeddings are expressive and accurately capture intricate aspects of molecules. Larger batch sizes result in a greater number of negative samples, hence improving performance. This is in line with prior contrastive learning methods continuing to improve by increasing the batch size [12]. Increasing the number of parameters leads to more expressive models thereby enhancing retrieval performance. This result is in accordance with

top-1%

top-5%

Method

Explicit

Modality

Unseen Images

Unseen Images +

Unseen Dataset

Unseen Images

Unseen Images +

Unseen Dataset

Concentration (ours)

Unseen Molecules

(0-shot)

Unseen Molecules

(0-shot)

CLIP

✗

Phenom1

.2109

.2425

.1519

.2018

.4458

.4968

.3591

.4339

Hopfield-CLIP

✗

Phenom1

.1581

.2034

.1198

.1604

.3783

.4413

.3045

.3747

InfoLOOB

✗

Phenom1

.2122

.2496

.1501

.2040

.4443

.5003

.3515

.4320

CLOOME

✗

Phenom1

.2164

.2461

.1479

.2035

.4590

.4956

.3528

.4358

CLOOME

sigmoid

Phenom1

.3338

.2681

.1801

.2606

.6037

.5202

.3879

.5039

CLOOME

logarithm

Phenom1

.3094

.2345

.1665

.2368

.5960

.4874

.3534

.4790

CLOOME

one-hot

Phenom1

.3073

.2040

.1670

.2261

.5997

.4246

.3657

.4633

DCL

✗

Phenom1

.4717

.4027

.2841

.3861

.7352

.6579

.5322

.6417

DCL

sigmoid

Phenom1

.7282

.4100

.3560

.4980

.9226

.6561

.6015

.7267

DCL

logarithm

Phenom1

.6903

.3558

.3211

.4557

.8869

.6146

.5667

.6894

DCL

one-hot

Phenom1

.6562

.3607

.3272

.4480

.8831

.5983

.5659

.6824

CWCL

✗

Phenom1

.5731

.4403

.3232

.4455

.8218

.6833

.5706

.6919

CWCL

sigmoid

Phenom1

.7780

.4425

.3777

.5327

.9386

.6844

.6244

.7491

CWCL

logarithm

Phenom1

.7452

.3989

.3523

.4988

.9117

.6482

.5962

.7187

CWCL

one-hot

Phenom1

.7048

.4009

.3593

.4883

.9037

.6284

.6061

.7127

SigLip

✗

Phenom1

.5718

.4217

.3021

.4318

.8104

.6602

.5176

.6627

SigLip

sigmoid

Phenom1

.8366

.4640

.3830

.5612

.9623

.7023

.6080

.7575

SigLip

logarithm

Phenom1

.8097

.4391

.3747

.5411

.9437

.6746

.6046

.7409

SigLip

one-hot

Phenom1

.7561

.4020

.3345

.4975

.9279

.6248

.5557

.7028

MolPhenix (ours)

✗

Phenom1

.8334

.4615

.3792

.5580

.9638

.6937

.6128

.7567

MolPhenix (ours)

sigmoid

Phenom1

.8256

.4692

.3765

.5571

.9638

.7068

.6115

.7607

MolPhenix (ours)

logarithm

Phenom1

.7953

.4466

.3664

.5361

.9466

.6889

.5924

.7426

MolPhenix (ours)

one-hot

Phenom1

.7489

.4088

.3379

.4985

.9325

.6465

.5644

.7144

MolPhenix (ours)

✗

Phenom1 & MolGPS

.8277

.4739

.4071

.5695

.9602

.7041

.6798

.7813

MolPhenix (ours)

sigmoid

Phenom1 & MolGPS

.8218

.4771

.4287

.5758

.9588

.7117

.7045

.7916

MolPhenix (ours)

logarithm

Phenom1 & MolGPS

.7836

.4757

.4297

.563

.9402

.7138

.7011

.7850

MolPhenix (ours)

one-hot

Phenom1 & MolGPS

.7391

.4307

.3940

.5212

.9198

.6724

.6698

.7540

recent advances in language modelling and scaling laws across different data and compute budgets [25].

Table 14: Ablations across different model sizes. Larger capacity models are found to be more expressive.

E.6 Investigating Other Pre-trained Phenomic Encoders

To investigate the impact of pre-trained encoders, we perform additional experiments evaluating a supervised phenomic image encoder (Table 20). Instead of Phenom1, we trained Molphenix framework using AdaBN, a CNN-based supervised phenomic encoder, with an analogous implementation discussed in [55]. We find that the general trends between Phenom1 and AdaBN are consistent with a slight decrease in overall performance. These findings provide additional support to the generality of the proposed guidelines.

E.7 Integrating MolGPS Embeddings With Other Fingerprints

Molphenix architecture is flexible, allowing that the proposed components be replaced by other phenomic or molecular pretrained models. We leveraged from MolGPS, which is a MPNN based

256

512

1024

0.4

0.6

0.8

1.0

Recall@1

Unseen Images

Unseen Images +

Unseen Molecules

Unseen Dataset

10 7

Number of Parameters (log scale)

0.4

0.6

0.8

1.0

128

512

2048

8192

0.4

0.6

0.8

1.0

0.01

0.1

1.0

Cut-Off p-value (log scale)

0.4

0.6

0.8

1.0

MACCS

RDKit

Morgan

Multi-FPs

MolGPS

Fingerprint

0.4

0.6

0.8

1.0

W/O Embedding-Avg

With Embedding-Avg

0.5

0.6

0.7

0.8

0.9

1.0

GNN model with 1B parameters which allows us to maximize architecture expressivity while minimizing the risk of overfitting [34, 56]. For additional investigation, we combine MolGPS molecular embeddings with RDKIT, MACCS, and Morgan fingerprints and show that they can provide Molphenix with richer molecular information and yields overall higher performance of MolPhenix in both cumulative and held-out concentration scenarios. Results for active and all molecules retrieval of Molphenix trained on the discussed combinational molecular embeddings are available in table 21 and 22.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces MolPhenix for tackling the contrastive phenomolecular retrieval problem. MolPhenix employs a uni-modal pretrained phenomics model, an inter-sample similarity-aware loss function, and conditions on the representation of molecular concentration. This approach effectively addresses challenges such as experimental batch effects, inactive molecule perturbations, and encoding perturbation concentration. Experimental results demonstrate the model's effectiveness in the retrieval task.\n\nSTRENGTHS:\n- The paper is well-written and the motivation is clear.\n- The identified challenges are highly relevant and significant in the biological context, and the proposed methods are logical and well-conceived.\n- The experimental results indicate substantial improvements over baselines across various settings in the retrieval task.\n\nWEAKNESSES:\n- In Table 2 and Table 4, there is a significant performance increase for DCL, CWCL, SigLip, and S2L compared to other baselines. The source of these improvements is unclear. Conducting an ablation study for each of the three components of the method, corresponding to the three challenges, would provide more insights.\n- The paper primarily compares MolPhenix with CLOOME and a few other general domain objectives. However, numerous related studies specifically within the molecular-phenotype contrastive learning domain [1-6] are not discussed or compared.\n- The paper emphasizes the retrieval task, with experimental results showing only the top 1% recall accuracy. This limits the overall impact of the findings. Considering broader application scenarios based on the proposed method could enhance the paper's significance.\n\n[1] Cross-modal graph contrastive learning with cellular images. bioRxiv 2022.\n\n[2] Contrastive learning of image- and structure-based representations in drug discovery. ICLR MLDD 2022.\n\n[3] Molecule-Morphology Contrastive Pretraining for Transferable Molecular Representation. ICML CompBio 2023.\n\n[4] MMCL-CDR: enhancing cancer drug response prediction with multi-omics and morphology images contrastive representation learning. Bioinformatics 2023.\n\n[5] Removing Biases from Molecular Representations via Information Maximization. ICLR 2024.\n\n[6] Learning Molecular Representation in a Cell. arXiv 2024.\n\nQUESTIONS:\n- The retrieval accuracy results in the paper are reported as top 1% accuracy. However, this metric may be less informative when the retrieval set is very large, as the top 1% can still include a substantial number of molecules. Could the authors also provide results for top N accuracy, where N = 1, 10, and 100?\n\nLIMITATIONS:\nThe authors have noted several limitations in their methods and experiments. Additional limitations are listed in the **Weaknesses** and **Questions** sections.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper makes a significant step forward in the task of phenotype-molecular retrieval–the task to find the molecule applied to perturb a set of cells given a microscopy readout. This problem can be modeled using multi-modal contrastive learning. The proposed MolPhenix method leverages pre-trained foundation models to encode the microscopy images and molecular structures. Then a novel contrastive loss is used which takes into account domain specific issues like batch effect and different concentrations of the applied molecules. Through thorough evaluations and ablations the authors show increased performance over a baseline.\n\nSTRENGTHS:\n- The paper tackles a task relevant for drug discovery and shows a significant step forward in predictive performance\n- The paper is very well written\n- The authors perform extensive experiments on held out molecules, phenotypes and dataset\n- The CLOOME baseline is tuned convincingly\n- The proposed guidelines for the task are supported by ablation studies\n- The paper combines and improves upon SOTA methods in multi-modal contrastive learning\n- The authors show domain knowledge by incorporating reasonable data processing and training strategies (batch effect removal + undersampling of inactive molecules)\n\nWEAKNESSES:\n- The authors focus on retrieval instead of evaluation of the latent space. Given the expressiveness of the leveraged foundation models, generating phenotypes or molecules should be possible. It would be great if the authors could comment on this or other explorations of the phenotypic space.\n- The authors train on a private dataset which makes the paper almost not reproducible. However, due to detailed model descriptions, the guidelines could be evaluated on other datasets.\n\nQUESTIONS:\n- Please elaborate on generative or mechanistic opportunities of the model\n- You argue MolPhenix cannot be applied to images. Please elaborate since instead of ph-1 a simple non domain-specific image feature extractor could be used. Have you studied the results?\n- Please comment on which model weights of ph-1, Mol-1, and MolPhenix are publicly available or you will be releasing\n\nLIMITATIONS:\n- Limited reproducibility due to private datasets and the model weights likely not being released", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces MolPhenix, a framework for contrastive phenomolecular retrieval that integrates phenomic data and molecular structures into a joint embedding space. Key contributions include combining phenomic and molecular data for improved retrieval accuracy, proposing effective training guidelines, and addressing cumulative concentrations and label noise. The framework demonstrates significant performance improvements over existing methods, particularly in zero-shot settings, and is supported by comprehensive experiments and ablation studies.\n\nSTRENGTHS:\nThe paper is well-structured and comprehensive, showcasing rigorous experimentation through extensive ablation studies, comparisons with baseline methods, and evaluations. Its strength lies in the detailed and methodical approach to validating the MolPhenix framework, providing strong evidence for its effectiveness and potential applications. The experimental design is robust, demonstrating the framework's superiority in various scenarios and contributing valuable insights to the field. Additionally, the integration of molecular and phenomic data into a joint multi-modal embedding offers a fresh perspective, enhancing the overall impact and originality of the work.\n\nWEAKNESSES:\n1. The clarity of this work could be significantly improved. The task and background are not well stated, and the related work is not adequately introduced. Many biological terminologies are mentioned, creating a large gap between the introduction and the real dataset.\n2. The paper could benefit from including comparisons with a broader range of state-of-the-art methods, particularly those from adjacent fields such as multi-modal representation learning in genomics and proteomics. This would highlight the specific innovations of MolPhenix more distinctly.\n3. The significance of the work could be more explicitly articulated and demonstrated. Although terms like drug discovery are mentioned, the real-world impact is not clear for readers.\n\nQUESTIONS:\n1. For molecular representations, did you consider combining GNNs and fingerprints? In other tasks, such as property prediction, their combination often surpasses the performance of each method individually.\n2. Did you consider to report and compare metrics for top-5 and top-10 recall?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces MolPhenix, a model designed to learn a joint latent space between molecular structures and microscopy phenomic experiments, addressing the challenge of contrastive phenomolecular retrieval. The authors point out three key challenges in this domain: limited paired data & batch effects, inactive molecular perturbations, and variable concentrations. To address these issues, they propose a set of guidelines: 1) leveraging a pre-trained phenomics foundation model (ph-1), 2) mitigating the impact of inactive molecules through undersampling and a novel soft-weighted sigmoid locked loss (S2L), and 3) encoding molecular concentration both implicitly (within the S2L loss) and explicitly (by parsing dosage concentration).\n\nThe primary experiments focus on the task of phenomolecular retrieval. For active molecules, MolPhenix achieves up to 77.33% top-1% retrieval accuracy, representing an 8.1-fold improvement over the baseline. The paper includes necessary ablation studies and evaluations across multiple datasets, demonstrating the effectiveness of their approach in both cumulative and held-out concentration settings.\n\nSTRENGTHS:\n1. Novel approach: The paper introduces MolPhenix, a comprehensive framework that addresses several key challenges in contrastive phenomolecular retrieval by combining multiple innovative techniques. This new framework demonstrates notably strong performance.\n2. Concentration encoding: The study explores both implicit and explicit methods for encoding molecular concentration, enhancing the model's ability to capture dose-dependent effects and generalize across concentrations. The explicit concentration module and inactive molecule undersampling techniques may inspire future multi-molecule research.\n3. Detailed ablation: The paper provides an in-depth analysis of results, including the impact of various components (e.g., loss functions, concentration encoding methods) on model performance, presented in both the main body and appendix.\n\nWEAKNESSES:\nThe following are just some minor concerns, listed in descending order of importance:\n\n1. Insufficient justification for S2L loss: While the paper introduces the S2L loss as a key contribution, it lacks a thorough theoretical analysis explaining why this loss function is suitable for the phenomolecular retrieval task. A more rigorous mathematical analysis of S2L would strengthen the paper's contribution.\n2. Inadequate ablation of pretrained components: As a phenomolecular retrieval \"framework\", the study should include ablation studies using different molecular and phenomic encoders. However, it appears that experiments were conducted only with fixed molecular GNN and phenomic models.\n3. Oversimplified treatment of batch effects: The paper claims to address batch effects through embedding averaging, but this approach may be too simplistic. A more detailed explanation of why simple averaging can alleviate batch effects, or some minor adjustments to this batch effect removal procedure, would be beneficial.\n\nQUESTIONS:\nRegarding the equality of involved datasets:\n\n1. How would the model's performance vary if only the open-source RxRx3 data or only the private novel data were used?\n2. Given that many components of MolPhenix are publicly pretrained models, is it feasible to construct a comparable model using solely open-source resources?\n\nLIMITATIONS:\nThe authors have properly adressed the limitations in their paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work focuses on predicting the molecular impact on cellular functions and investigates the problem of contrastive phenomolecular retrieval. It introduces MolPhenix, a model that leverages a joint latent space between molecular structures and microscopy-based phenomic experiments using contrastive learning. The main contributions include the use of a pre-trained phenomics model, a novel inter-sample similarity-aware loss (S2L), and molecular concentration conditioning, leading to a significant improvement over the previous state-of-the-art in zero-shot molecular retrieval of active molecules.\n\nSTRENGTHS:\nThe introduction of the MolPhenix model, which utilizes a pre-trained phenomics model, a novel inter-sample similarity-aware loss (S2L), and molecular concentration conditioning, is original and demonstrates improvements over existing methodologies. \n\nThe experimentation of this work is comprehensive and detailed to justify the proposed method. \n\nThe paper is well-structured and clearly written with clear explanations of the methodologies. \n\nThe problem investigated is important in the field of drug discovery and has a growing interest.\n\nWEAKNESSES:\nFor the captions of Table 2~5, it is better to mention the experimental setting (cumulative concentration & held-out concentrations). The current captions are nearly the same (Table 2 vs. Table 4, and Table 3 vs. Table 5), which is kind of confusing.\n\nQUESTIONS:\nFor the pretrained GNN, are there any specific advantages to choosing the current one, given that there are many other pretrained GNNs that can be used to extract molecular representations?\n\nAre there any discussions regarding the results of different concentration encoding choices? In a cumulative concentration setting, one-hot performs the best, while in a held-out concentration setting, not using any explicit concentration is the best choice overall.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-17"}