Predicting Single-Cell Perturbation Responses for Unseen Drugs
Abstract
Single-cell transcriptomics enabled the study of cellular heterogeneity in responseto perturbations at the resolution of individual cells.
However, scaling highthroughput screens (HTSs) to measure cellular responses for many drugs remainsa challenge due to technical limitations and, more importantly, the cost of suchmultiplexed experiments.
Thus, transferring information from routinely performedbulk RNA HTS is required to enrich single-cell data meaningfully.
We introducea new encoder-decoder architecture to study the perturbational effects of unseendrugs.
We combine the model with an architecture surgery for transfer learningand demonstrate how training on existing bulk RNA HTS datasets can improvegeneralisation performance.
Better generalisation reduces the need for extensiveand costly screens at single-cell resolution.
We envision that our proposed methodwill facilitate more efficient experiment designs through its ability to generatein-silico hypotheses, ultimately accelerating drug discovery.
Introduction
Recent advances in single-cell methods allowed the simultaneous analysis of millions of cells,increasing depth and resolution to explore cellular heterogeneity (Sikkema et al., 2022; Han et al.
With single-cell RNA sequencing (scRNA-seq) and high-throughput screens (HTSs) one cannow study the impact of different perturbations, i.e., drug-dosage combinations, on the transcriptomeat cellular resolution (Yofe et al., 2020; Norman et al., 2019).
Unlike conventional HTSs, scRNAseq HTSs can identify subtle changes in gene expression and cellular heterogeneity, constituting acornerstone for pharmaceutics and drug discovery (Srivatsan et al., 2020).
Nevertheless, these newlyintroduced sample multiplexing techniques (McGinnis et al., 2019; Stoeckius et al., 2018; Gehringet al., 2018) require expensive library preparation and do not scale to screen thousands of distinctmolecules.
Even in its most cost-effective version, nuclear hashing, the acquired datasets contain nomore than 200 different drugs (Srivatsan et al., 2020).
Consequently, computational methods are required to address the limited exploration power ofexisting experimental methods and discover promising therapeutic drug candidates.
Suitable methodsshould predict the response to unobserved (combinations of) perturbations.
Increasing in difficulty,such tasks may include inter- and extrapolation of dosage values, the generalisation to unobserved(combinations of) drug-covariates (e.g., cell-type), or predictions for unseen drugs.
In terms ofmedical impact, the prediction of unobserved perturbations may be the most desirable, for examplefor drug repurposing.
At the same time, it requires the model to properly capture complex chemicalinteractions within multiple distinct cellular contexts.
Such generalisation capabilities can not yetbe learned from single-cell HTSs alone, as they supposedly do not cover the required breadth ofchemical interactions.
In this work, we leverage information across datasets to alleviate this issue.
We propose a new model that generalises previous work on Fader Networks by Lample et al.
(2017)and the Compositional Perturbation Autoencoder (CPA) by Lotfollahi et al.
(2021) to the challengingscenario of generating counterfactual predictions for unseen compounds.
Our method is as flexibleand interpretable as CPA but further enables us to leverage lower resolution but higher throughputassays, such as bulk RNA HTSs, to improve the model’s generalisation performance on single-celldata (Amodio et al., 2021).
Our main contributions are:
We introduce chemCPA, a model that incorporates knowledge about the compounds’ struc-
ture, enabling the prediction of drug perturbations at a single-cell level from molecularrepresentations.
We propose and evaluate a transfer learning scheme to leverage HTS bulk RNA-seq datain the setting of both identical and different gene sets between the source (bulk) and target
(single-cell) datasets.3. We show how chemCPA outperforms existing methods on the task of predicting unobserveddrug-covariate combinations. At the same time, we demonstrate chemCPA’s versatility and evaluate chemCPA on generalisation tasks that cannot be modeled using any previouslyexisting method.
Related Work
Over the past years, deep learning (DL) has become an essential tool for analysis (Lopez et al., 2020)and interpretation (Rybakov et al., 2020) of scRNA-seq data.
Representation learning in particular,has been useful not only for identifying cellular heterogeneity and integration (Gayoso et al., 2022),or mapping query to reference datasets (Lotfollahi et al., 2022), but also in the context of modellingsingle-cell perturbation responses (Rampášek et al., 2019; Seninge et al., 2021; Lotfollahi et al., 2019;
Ji et al., 2021).
Unlike linear models (Dixit et al., 2016; Kamimoto et al., 2020) or mechanistic approaches (Fröhlichet al., 2018; Yuan et al., 2021), DL is suited to capture non-linear cell-type-specific responsesand easily scales to genome-wide measurements.
Recently, Lotfollahi et al.
(2021) introduced the CPA method for modelling perturbations on scRNA-seq data.
CPA does not generalise to unseencompounds, hindering its application to virtual screening of drugs not yet measured via scRNA-seqdata, which is required for effective drug discovery.
For bulk RNA data, on the other hand, several methods have been proposed to predict gene expressionprofiles for de novo chemicals (Pham et al., 2021; Zhu et al., 2021; Umarov et al., 2021).
Crucially,the L1000 dataset, introduced by the LINCS programme (Subramanian et al., 2017), greatly facilitatedsuch advances on phenotype-based compound screening.
However, it remains unclear how to translatethese approaches to single-cell datasets that include significantly fewer compounds and, in manycases, rely on different gene sets.
Chemical Compositional Perturbation Autoencoder
We consider a dataset D
= { ( x i , y i ) } Ni =1 = { ( x i , ( d i , s i , c i ))
} Ni =1 , where x i ∈ R n describes then -dimensional gene expression and y i an attribute set.
For scRNA-seq perturbation data, we usuallyconsider the drug and dosage attributes, d i ∈ { drugs in D} and s i ∈ R , respectively, and the cell-linec i of cell i .
Note that this set of attributes Y depends on the available data and could be extended tocovariates such as patient, or species.
A possible approach to predicting counterfactual combinations is to encode a cell’s gene expressionx i invariantly from its attributes y i as a latent vector z i , called the basal state.
Afterwards, z i canbe combined with z attribute to encode any attribute combination y ′ i
̸
= y i , and decoded back to a geneexpression state
ˆ x i that corresponds to this new set of chosen attributes.
To this end, we divide ourmodel, the Chemical Compositional Perturbation Autoencoder (chemCPA), into three parts: (1) thegene expression encoder and decoder, (2) the attribute embedders, and (3) the adversarial classifiers,see Figure 1 for an illustration.
3.1 Gene encoder and decoder
Following Lotfollahi et al.
(2021), our model is based on an encoder-decoder architecture combinedwith adversarial training.
The encoding network E θ : R n → R l is a multi-layer perceptron (MLP)with parameters θ that maps a measured gene expression state x i ∈ R n to its l -dimensional
latentvector z i =
E θ ( x i ) .
Through adversarial classifiers, z i is trained to not contain any information aboutits attributes y i .
This gives us control over the latent space in which we update z i with an additiveattribute embedding of our choice and obtain z ′ i .
The decoder D ψ : R l → R 2 n is an MLP that takes z
′ i as input and computes the component-wiseparameters of the underlying distribution P of the gene expression data. Dependent on whether x i israw or pre-processed, P can follow a negative binomial or Gaussian distribution. Assuming a meanand variance parametrisation, we get in both cases µ = D µψ ( z ′ ) and[-variances σ 
-]
σ 2 = D σ 2 ψ ( z ′ ) for the descriptionof the decoded gene expression state.
While chemCPA supports both settings, we observed betterconvergence with a Gaussian likelihood for which the reconstruction loss becomes:
[Equation]
z
[Equation]
Next, we provide intuition about how we can meaningfully interpret latent space arithmetics and howwe encode drug and cell-line attributes.
3.2 Attribute embedding
Due to their different nature, we encode the drug attributes and cell-line information separately.
Forthe cell-lines, we use the same approach as Lotfollahi et al.
(2021), where a d -dim latent representationz c is optimised for each cell-line c .
Perturbation Network For the drugs, we train an embedding network P φ that maps molecularrepresentations—such as its graph or SMILES representation—and the used dosage to its latentperturbation state.
This perturbation network P φ consists of the molecule and perturbation encoders,
G and M , as well as the dosage scaler S , see Figure 1 (2).
The molecule encoder G : G → R m encodes the molecule representation g i ∈ G as a fixed sizeembedding h d i ∈ R m .
In a subsequent step, the perturbation encoder M : R m → R l takes themolecular embedding h d i as input and generates the drug perturbation z d i ∈ R l that is used inchemCPA’s latent space.
The dosage scaler S :
R m +
→ R also uses h d i and maps it together with the dosage s i to thescaled dosage value
ˆ s i .
We chose S to map back to a scalar value ˆ s as this allows us to computedrug-response curves in an easy fashion. In addition, this way of encoding matches the idea that z d iencodes the drug’s general effect, which is dosage independent.
Put together, we end up withˆ
s i × z d i = P φ ( g i , s i ) = S ( h d i , s i ) × M ( h d i ) with h d i =
G ( g i ) . The molecule encoder G can be any encoding network that maps molecular representations to fixedsize embeddings.
Due to the limited number of drugs available in scRNA-seq HTSs, we proposeto rely on a pretrained encoding model and freeze G during training.
We tested multiple differentoptions for G and include a detailed benchmark in the appendix A.1.
We found that in our setting RDKit features performed really well and report all following results for chemCPA with RDKit asthe molecule encoder G .
Additive latent space We assume an additive structure of the perturbation response in the latentspace. This choice of linearity makes the model interpretable for users such as biologists since itpermits to analyze and ablate components individually, e.g., allowing interpolation or extrapolation ofthe dose values.
Another advantage of the additive structure is its permutation invariance and that itallows for adding new covariates, e.g., during fine-tuning. While remaining interpretable, chemCPAis able to model complex relationships through the non-linear decoder.
By design, we can choose a new set of attributes for the drug and cell line and compute the new latentstate as z ′ i =
z i + z attribute at test time. Due to the perturbation network, chemCPA makes it possibleto predict drug perturbations for molecules that have not been experimentally observed ( d / ∈ D ). Incontrast, CPA can only make predictions for molecules that were present during training ( d ∈ D ).
Inboth cases, the latent representation is computed as: [Equation] We next describe how we “strip z i from its attribute information” to obtain a basal state representation.
3.3 Adversarial classifiers for invariant basal states
To generate invariant basal states and produce disentangled representations z i ,
z d i , and z c i , we useadversarial classifiers A drug ϕ and A cov ϕ .
Both adversary networks A jϕ : R l → R N j take the latentbasal state
z i as input and aim to predict the drug that has been applied to example i as well asits cell-line c i .
While these classifiers are trained to improve classification performance, we alsoadd the classification loss with a reversed sign to the training objective for the encoder E θ .
Hence,the encoder attempts to produce a latent representation
z i which contains no information about theattributes .
Note that this explicit separation of basal, drug, and covariate information, which wecall disentanglement, is an approximation to make the problem tractable.
At the same time, suchseparation is useful for attributing perturbation effects to specific sources, e.g., drug or cell line,which is relevant for biological applications and downstream analyses.
We use the cross-entropy loss for both classifiers
[Equation]
and
[Equation]
Following the CPA implementation from Lotfollahi et al. (2021), we add a gradient penalty to theloss function of the adversarial classifiers, to minimise
[Equation]
Empirically, the penalty improves the robustness of the classifier to measurement noise of geneexpressions.
During training, we alternate update steps between the following competing objectives
[Equation]
where λ dis balances the importance of good reconstruction against the encoder E θ ’s constraint togenerate disentangled basal states z i .
The gradient penalty is weigthed with λ pen .
Datasets and transfer learning
We use the sci-Plex3 (Srivatsan et al., 2020) and the L1000 (Subramanian et al., 2017) datasets forthe main evaluation on single-cell data and pretraining on bulk experiments, respectively.
Datasets The L1000 data contains about 1 .
3 million bulk RNA observations for 978 different genes.
It includes measurements for almost 20 k different drugs, some of which are FDA-approved, whileothers are synthetic compounds with no proven effect on any disease.
Compared to scRNA-seqdata, the L1000 data allows to explore a more diverse space of molecules which makes it ideal forpretraining.
The sci-Plex3 data is similar in size and contains measurements for 649 , 340 cells across 7561 drugsensitive genes.
On three human cancer cell lines—A549, MCF7, and K562—single-compoundperturbations for 188 drugs at four different dosages— 10 nM, 100 nM, 1 µ M, and 10 µ M— areexamined.
Note that all cell lines and about 150 compounds overlap with the L1000 data.
Inaddition, Srivatsan et al.
(2020) assigned to all compounds one of 19 different modes of action (MoA),also called pathways.
In contrast to the mechanism of action, which is related to the biochemicalinteraction between a molecule and a cell, the MoA describes the anatomical change that results fromthe exposure of cells to a drug-like molecule.
Transfer learning As we train chemCPA with a Gaussian likelihood loss, the dataset was firstnormalised and then log( x + 1)
-transformed.
Depending on the experiment, we further reduced thenumber of genes included in the single-cell data.
In Section 5.1, we first subsetted both datasets tothe same 977 genes which were identified via ensemble gene annotations.
For the final experiment in
Section 5.2, the considered gene set is increased as we hypothesize that more than the 977 L1000genes are required to capture the variability within the single-cell data.
To assess whether pretrainingon L1000 is still beneficial in this scenario, we included 1023 highly variable genes (HVGs) from thesci-Plex3 data.
That is, we consider 2000 genes in total.
For the extended gene set, chemCPA’s input and output dimensions have to be adjusted to match thetotal number of 2000 genes.
This is realised by adding a fully-connected layer to the input and outputof the encoder E θ and decoder D ψ , respectively.
This procedure differs from the architecture surgeryintroduced in Lotfollahi et al.
(2022), as whole layers are added instead of individual neurons and thetransfer is performed on the gene set instead of the dataset labels. Moreover, no encoder or decoderweights are frozen during the fine-tuning of chemCPA.
Experiments
Our evaluation strategy tests chemCPA’s ability to produce counterfactual predictions.
To this end, itis important to measure both the predictive performance of a trained model as well as the degree towhich the latent space components are disentangled.
Counterfactual predictions To perform a counterfactual prediction, chemCPA first encodes anunperturbed control observations, a cell treated with dimethyl sulfoxide.
The resulting basal state isthen combined with the encoding of the desired drug and cell line, and chemCPA decodes the result.
As we are free to choose any drug encoding, we refer to this process as counterfactual prediction.
Evaluation strategy Throughout our experiments, we use the coefficient of determination r 2 asthe main performance metric.
This score is computed between the actual measurements and thecounterfactual predictions on all genes and the 50 most differentially expressed genes (DEGs).
Itis necessary to consider all genes to evaluate the background and general decoder performance.
However, the resulting r 2 -scores can get inflated since most genes stay similar to their controls underperturbation.
In contrast, the DEGs capture the differential signal which reflects a drug’s effect.
Tofurther stress the importance to report both scores, note that the DEGs are unknown for unseen drugsas they depend on the drug and cell type.
Hence, the combination is essential to gauge the accuracyof the model’s predictions.
In order to classify the degree of disentanglement during evaluation, we train separate MLPs with fourlayers over 400 epochs and compute the prediction accuracy for drugs and covariates given the basal [Table] state.
We consider the resulting accuracies as our disentanglement scores. An optimally disentangledmodel achieves scores that match the ratios of the most abundant drug and cell line, respectively.
Since no model achieves perfect scores, we subset to models that are sufficiently disentangled.
Throughout our experiments on the sci-Plex3 data, we set the thresholds for perturbation and cell linedisentanglement to < 10% and < 70% , respectively, while values of 3% and 51% are optimal.
Notethat poor disentanglement will automatically lead to low scores due to computing the test scores onthe counterfactual predictions.
Before evaluating how well chemCPA can generalise to unseen drugs, we have to establish its competitive performance in a less ambitious setting. For this, we consider the scenario of generalisationto unobserved combinations of drugs and cell lines on the sci-Plex3 data and compare chemCPAagainst scGen (Lotfollahi et al., 2019) and chemCPA (Lotfollahi et al., 2021). As scGen cannot distinguish between different dosage values, we perform two separate experimentsfor the second and highest dose values, 1 µ M and 10 µ M, respectively.
Moreover, as both CPA andscGen require each individual component (drug d , cell line c ) to be part of the training data D , wecreate three distinct splits with only two of the three different drug set and cell line combinationsbeing present during training, the third one being left for testing. We choose to test nine different compounds: Dacinostat, Givinostat, Belinostat, Hesperadin, Quisinostat, Alvespimycin, Tanespimycin, TAK-901, and Flavopiridol. These drugs mostly belong tothree MoA—epigenetic regulation, tyrosine kinase signalling, and cell cycle regulation—and werereported among the most effective drugs in the original publication (Srivatsan et al., 2020). As discussed in the previous paragraph, we report mean and median r 2 values, which we averagedover the three splits and all drugs, for the sets of all genes and differentially expressed genes (DEGs).
We consider a model that discards all perturbation information as our baseline. As a consequence,one can understand the improvement over the baseline as a result of the additional drug encoding. Moreover, we use the L1000 data for the pretraining of chemCPA and subset to the same 977 genesfor the fine-tuning on sciPlex-3. This way, we are able to evaluate chemCPA in its pretrained andnon-pretrained version.
We report the results of this experiment in Table 1.
ChemCPA outperforms both scGen and CPA,demonstrating that the perturbation network together with pretraining leads to SOTA performance. Note also that the base version of chemCPA performs better than both CPA and scGen, indicatingthat the additional regularisation that comes from the perturbation networks P φ has beneficial effectson single-cell perturbation modelling.
To make a fair comparison, we optimised all CPA and chemCPA models identically and sweptover the same set of hyperparameters (random, 10 samples). scGen was optimised with defaultparameters and an adjusted KL annealing scheme to match the set number of epochs. Note that both [Figure]
M [Figure] [Figure]
[Figure] [Figure] [Figure] [Figure]
E [ r 2 ] on all genes [Figure] [Figure] [Figure]
in M [Figure] [Figure] [Figure]
[Figure]
[Table]
{+CPA and chemCPA can take control cells x i from all cell lines as input ( { A549, K562, MCF7 } →{ A549, K562, MCF7 } ) , while the cell line input for scGen has to match the test set ( A549 → A549 , K562 → K562 , MCF7 → MCF7 ) .5.
 Using chemCPA to predict single-cell responses for unseen drugs For the application of chemCPA to predict perturbation responses for unseen compounds, we usethe same nine drugs from sci-Plex3 data as in Section 5.1.+}
In addition to the shared gene set, wealso consider an extended gene set, cf. Section 4.
We include HVGs to account for the technologicaldifference between bulk and single-cell and to capture the variance of single-cell data.
To this end,the 977 genes present in both datasets are extended with 1023 HVGs of the sci-Plex3 data.
Note thatthrough this larger genes set, the 50 DEGs become a subset of the HVGs which makes it considerablymore difficult for pretrained models to leverage learned bulk expressions directly.
Shared gene set
Table 2 shows the test performance of chemCPA, averaged over all drugs andthe three cell lines, for the same gene set as used in Section 5.1. The pretrained chemCPA modelconsistently outperform the baseline and its base version. [Table]
[Figure]
The high baseline scores in Figure 2 shows that the drugs have almost no effect at low dosages.
Athigh dosages, however, we see how chemCPA’s predictions improve over the baseline.
Looking atthe prediction for all genes, the pretrained model has a significant advantage over its non-pretrainedversion.
As expected, the performance is lower for the DEGs.
Nevertheless, also in this scenario,chemCPA, especially the pretrained version, can explain gene expression values that must result fromthe drugs’ influence.
In Figure 3, latent perturbations z d are visualised. Note that the difference between the lowest andhighest doseage values results only from the non-linear dosage scaler S .
Extended gene sets
The extension to the larger gene set introduces a more difficult task forchemCPA.
In Table 3, we show the same analysis as for the shared gene set.
Again, the advantage ofthe pretraied chemCPA model translates to this scenario, while the base version is only slightly betterthan the baseline.
A more comprehensive view for all dosages is shown in Figure 4, see also A.5 forresults with different molecule encoders G .
This is a promising result, as it suggests thatthe transfer from abundant bulk RNA perturbation screens can be leveraged even in scenarioswhere the gene sets do not match.
Crucially, thisenables users to benefit from the proposed transfer learning and chemCPA’s modelling capacitywhile simultaneously accounting for the specialrequirements of scRNA-seq data.
In Figure 5 weshow an explicit example for chemCPA’s performance for two histone deacetylation drugs, seealso Figure 13 in the appendix for more details.
Yet, for real applications it is essential to makelimitations concerning the data and methodtransparent.
To this end, we propose an uncertainty measure that addresses some of the lim- itations related to the generalisation to unseen compounds
[Table]
5.3 Measure uncertainty on the drug embedding
Generalisation can only be achieved within the limits of the dataset a model is trained on.
For thesci-Plex3 data, less than 20% of the drug-dose combinations deviate from the controls’ phenotypes bymore than 35% in its r 2 scores.
In additon, we know from the original sci-Plex3 publication that onlydrugs from a few pathways—tyrosine kinase signaling, DNA damage and repair, cell cycle regulation,and epigenetic regulation—show a clear effect.
We assume that this technological noise is the reasonwhy the non-pretrained chemCPA version struggles to outperform the baseline on the extended geneset, whereas the pretrained model is more robust.
These data challenges are also reflected in Figure 3as we would expect chemCPA’s perturbation latent space to cluster according to the drugs’ MoA,similar to the cluster of histone deacetylation drugs.
We found that an imperfect clustering often correlates with high baseline scores and, as a result of that,chemCPA not being able to identify distinct perturbations, see Figure 15 in the appendix.
To makethe generalisation ability more transparent, we employ a measure of uncertainty.
A good indicatorfor chemCPA’s ability to generalise is the MoA prediction from the KNN-graph of the perturbationembedding space.
We further combine this measure with the average distance to neighbouring drugsas we recognise that larger distances indicate a distinct perturbation:
[Equation]
where d is the Euclidean distance, H is the Shannon entropy, and X the normalised pathwayprediction deduced from the neighbours
N i of drug
i .
This uncertainty measure combines two things:
First, chemCPA’s confidence on the drug’s MoA, measured by H , and second, whether chemCPAexpects the drug to have a distinct perturbation effect on the cell, measured by the inverse distance.
We provide an analysis of the uncertainty for the chemCPA model from Figure 3 in Table 4.
A plotthat includes all compounds is part of the appendix A.6.
The results show that the uncertainty score ufor unseen drugs correlates well with the accurate prediction of perturbed cells.
This illustrate howchemCPA’s compositional latent space can be leveraged for additional insights in order to evaluate itsgeneralisation ability.
Conclusion
In this paper, we introduced chemCPA, a model for predicting cellular gene expression responses forunseen drug perturbations by encoding the drugs’ molecular structures.
We showed how chemCPAoutperforms CPA and scGen on shared tasks, while generalising over existing methods by being applicable to the novel task of generalising to unseen drugs.
Applied to single-cell data, we demonstratedhow pretraining on bulk HTSs improves chemCPA’s generalisation performance.
This applies evenwhen the gene set of the single-cell dataset differs from the genes of the pretraining bulk RNA HTSdataset.
We further provided an uncertainty measurement that correlates well with the chemCPA’sgeneralisation ability to unseen drugs.
Taken all results together, we are confident that chemCPA willbenefit from higher-quality scRNA-seq HTSs in the future and can become a powerful aid in the drugscreening and drug discovery process.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data areproprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] See section claim 1 is supported by 3, claim 2 by A.1,claim 3 by 5.2 and 5.2.
(b) Did you describe the limitations of your work?
[Yes] See section 5.2 and 5.
(c) Did you discuss any potential negative societal impacts of your work?
[No] We do notforesee any potential harm by this work.(d)
Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A] Not applicable
(b) Did you include complete proofs of all theoretical results?
[N/A] Not applicable
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] Code link willbe included after double-blind review.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] Top 5 models that met disentanglement criteriain Table ??
, for the other experiments the best performing model according to the validation split was taken.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[No] The experiment files will be madepublic with the code release
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
Code link will be included after double-blind review.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A] Not applicable
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[No] All data comes from cell lines.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A] Not applicable
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A] Not applicable (c) Did
you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
N/A ] Not applicable
A Appendix
A.1 Benchmarking drug molecule encoders
Enabled by the flexibility of the molecule encoder G , we investigated what impact the architecturechoice has on the performance of chemCPA. For this, we compared multiple pretrained graph-basedmodels whose weights were frozen during the training. Next to predefined RDKit fingerprints (whichare non-differentiable, and hence not trainable), we included a GCN, MPNN, weave model, GROVERmodel, and a JT-VAE.
Table 5 summarises the results of this experiment on the L1000 dataset.
The weave model performsmuch worse than the others, achieving an r 2 -score of only 65 ± 8 on DEGs.
While the GCNdisentangles well, it is outperformed by the JT-VAE and GROVER models.
All experiments in themain text were ran across all three best performing models (GROVER, JT-VAE and RDKit), howeverdue to space constraints we only report RDKit results in the main text.
Table 6 shows the test performance of chemCPA for the nine unseen drugs across the three celllines in the transfer learning scenario with identical genes.
This is the same experiment as Table 2,but evaluated across more embedding models.
The fine-tuned chemCPA models for GROVER and
RDKit consistently outperform the baseline and their non-pretrained version with RDKit achievingthe highest median score on DEGs.
Interestingly, the fine-tuned JT-VAE model is better than thebaseline and other non-pretained chemCPA models but worse than its own non-pretrained version.
In Table 7, we show the same experiment as in Table 3.
Again, the pretrained chemCPA modelwith an RDKit molecule encoder G perform best.
We believe that this can be attributed to twothings.
First, the sci-Plex3 data is the first of its kind, and technological noise is still an issue.
When
[Table]
[Table]
evaluated over the whole training set, the baseline achieves r 2 -scores higher than 65% for more than96% of the observations.
This sparsity might hinder the more complex perturbation networks P φ ,which are based on GROVER and JT-VAE, from finding good perturbation representations.
Wesuspect that the same reason also explains the bad performance of non-pretrained models as these aremore susceptible to noise, whereas the fine-tuned models are more robust.
Second, the pretrainedembedding h that result from RDKit identifies the histone deacetylation drugs as a distinct cluster,see Figure 10.
Since these compounds show the strongest effect in the sci-Plex3 data, the inductivebias from RDKit give an explanation for the favourable generalisation performance.
[Table]
[Table]
A.2 Attribute embedding
A.3 Counterfactual prediction
To compute counterfactual predictions, we obtain basal states
z i for all control observationspresent in the test set.
For each combination of drug, dose, and cell line in the test set,we compute the latent attribute state z attribute and combine it with all z i .
Subsequently, wecompute the mean per gene across all predictions and likewise for the real measurements.
Asa result, we obtain two n dimensional vectors, where n is the number of genes (977 or 2000),for which we compute the r 2 score.
Taken together, we get one score per combination.
A 4 Additional information on the L1000 experiment
For infos on the RDKit sweep and resulting best run, see Table 9 and Table 10.
Architectures for best configuration of the perturbation networks P φ and adversary classifiers
are presented in Table 11.
For details on the performance of the best runs, see Table 12.
[Table]
A.5 Additional information on the sci-Plex3 experiments1.
The optimisation was performed similarly to the presented sweeps in Table 10 and Table 11for the perturbation network and adversary parameters for 10 samples each per category.2.
Boxplot results for RDKit, see Figures 6 and 8, and JT-VAE, see Figures 7 and 9.3.
Paired t-tests were performed for both settings, see Table 14 for the shared gene set and
Table 14 for the extended gene set.4.
More examples on the performance with respect to specific drugs are presented in Figure 11,
Figure 12, Figure 13, and Figure 14.5.
The Drug embedding that results from RDKit is shown in
A.6 Additional information on the uncertainty score1.
The uncertainty computation for the chemCPA model with an RDKit molecule embeddingfor the shared gene setting is shown in Figure 15. [Table] [Table] [Table]
[Figure]
[Figure] [Figure] [Figure]
[Table] [Table]