GANDALF: Generative AttentioN based Data Augmentation and predictive modeLing Framework for personalized cancer treatment
Authors: Aishwarya Jayagopal
Yanrong Zhang
Robert John Walsh
Tuan Zea Tan
Anand D Jeyasekharan
Vaibhav Rajan
Venue: ICLR 2025 Poster (2025)
Keywords: personalized drug response prediction
cancer
genomic data augmentation
diffusion model
pseudolabelling
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: A cancer drug response prediction model that addresses the problem of limited labelled data through a novel genomic data augmentation technique.

Abstract: Effective treatment of cancer is a major challenge faced by healthcare providers, due to the highly individualized nature of patient responses to treatment. This is caused by the heterogeneity seen in cancer-causing alterations (mutations) across patient genomes. Limited availability of response data in patients makes it difficult to train personalized treatment recommendation models on mutations from clinical genomic sequencing reports. Prior methods tackle this by utilising larger, labelled pre-clinical laboratory datasets (‘cell lines’), via transfer learning. These methods augment patient data by learning a shared, domain-invariant representation, between the cell line and patient domains, which is then used to train a downstream drug response prediction (DRP) model. This approach augments data in the shared space but fails to model patient-specific characteristics, which have a strong influence on their drug response. We propose a novel generative attention-based data augmentation and predictive modeling framework, GANDALF, to tackle this crucial shortcoming of prior methods. GANDALF not only augments patient genomic data directly, but also accounts for its domain-specific characteristics. GANDALF outperforms state-of-the-art DRP models on publicly available patient datasets and emerges as the front-runner amongst SOTA cancer DRP models.

Published as a conference paper at ICLR 2025
GANDALF: GENERATIVE ATTENTION BASED DATA AUGMENTATION AND PREDICTIVE MODELING FRAMEWORK FOR PERSONALIZED CANCER TREAT- MENT
Aishwarya Jayagopal', Yanrong Zhang', Robert John Walsh?, Tuan Zea Tan', Anand D Jeyasekharan', Vaibhav Rajan'
School Computing, National University of Singapore
National University Cancer Institute, Singapore Cancer Science Institute of Singapore (e0674492 e1068520) nus nus eduu, {robert.wall Henuhh edu. sg, [csittz, csiadj, vaibhav. ra jaja sg
ABSTRACT
Effective treatment of cancer a major challenge faced by healthcare providers. due to the highly individualized nature of patient responses treatment. This is caused by the heterogeneity seen in cancer-causing alterations (mutatioon) across patient genomes. Limited availability of response data in patients makes it dif- ficult to train personalized treatment recommendation models on mutations from clinical genomic sequencing reports. Prior methods tackle this by utilising larger. labelled pre-clinical laboratory datasets ( ('cell lines'), ), via transfer learning. These methods augment patient data by learning shared, Nomain-invarian represen- tation, between the cell line and patient domains, which is then used to train a downstream drug response prediction (DRP) model. This approach augments data in the shared space but fails to model patient-specific characteristics, which have a strong influence on their drug response. We propose novel generative attention-basse data augmentation and predictive modeling framework, GAN- DALF, to tackle this crucial shortcoming of prior methods. GANDALF not only augments patient genomic data directly, but also accounts for its domain-specific characteristics. GANDALF outperforms state-of-the-aar DRP models on publicly available patient datasets and emerges the front-runner amongst SOTA cancer DRP models.
INTRODUCTION
Cancer, a leading cause of deaths worldwide (Dattani et al., 2023). imposes a significant burden on global healthcare systems (Lopes) 2023). It is caused due to the presence of alterations (mutations) in the human genome, resulting in uncontrolled replication of cancer cells. Cancer patients exhibit a great deal of heterogeneity in their genomic mutation profiles,
when they have the same cancer type. This heterogeneity causes patients, of the same cancer type, to respond differently to the same treatment (Liao et 2023), making cancer treatment challenging (Wahida al., 2023). Treatment, today, is largely guideline-based and prescribes drugs based on the cancer type (Planchard et al.) 2018 Conroy al., 2023 Morris al. 2023). This approach fails account for heterogeneity in patient mutations, and its impact on treatment outcomes. Precision oncology (Sosinsky et al.) 2024 2015) is gradually shifting focus from a "one-size-fits- approach more
Collins & Varmus
personalized treatment strategies.
To aid precision oncology, cancer patients undergo genomic sequencing as part of clinical diag- nostics (Colomer et al., 2023). Clinical sequencing panels (Milbury et al.] 2022 Wei et al., 2022) identify the set of mutations present specific sections of the human genome (called genes), which have a known association with cancer. Cancer patients can exhibit a varying number of mutations in each of these genes (Saito et al.] 2021). These mutations interact with each other and the drug in complex ways to determine patient response treatment (Liu et al. 2022). While clinical tri- als have identified drugs that target specific mutations, these studies have largely been restricted to single mutations (Brachova et al. 2013 Randic al. 2023). Conducting large scale clinical trials

---

Published as a conference paper at ICLR 2025
Clinical requanninggreoo report
sequence
BICA QJ 7175577771
12732
umppiiicatio
Verying number mutationa
Observed DNA
FOP12emo4Tradiin
sequeence
Mutations identified
dimenatonnal
Drug Responas Prediction network
Futients
CellUnee Unee
Figure 1: Overview of clinical challenge in cancer drug response prediction.
for all possible combinations of mutations in ~ 20000 genes of the human genome is practically in- tractable, thereby limiting their ability to identify the right treatment when a patient exhibits multiple mutations.
Machine learning (ML) approaches provide a promising avenue to predict patient response Yp to drugs dz. based on the set of mutations Xp in their genomic profiles. However, guideline-basee baseed treatment in clinics prescribe only a small subset of drugs from all drugs approved for clinical use, thereby limiting the availability of labelled patient data (Xp. dp. Up). The resulting scarcity poses significant challenge training supervised MI. models to predict drug response patients. Prior methods in Drug Response Prediction (DRP) literature have tackled this using data from related domain called "cell lines". Cell lines (Ghandi et al. 2019) are cancer cells extracted from patients, which are then cloned under controlled laboratory settings. Each clone X is administered different drug de, and the corresponding response ye (e(Xe, de) is measured for various drug concentrations. Since these cells studied outside the human body, it is possible to obtain Ye for large set of drugs D, resulting in abundant labelled data.
However, models trained only (Xe; des ye) do not work well patients (Mourragui al. 2019 2021
et al. 2020). This is attributed to the inherent differences between patients and cell lines. As cell lines are studied outside the human body in the absence blood vessels and the immune system (called tumor microenvironmenty, these cells can acquire mutations differently compared patients, i.e. P(X.) + P(Xp). In addition, Ye [0 depends on drug concentra- tion and number of surviving cells (called Area Under the Dose Response Curve, AUDRC), while Up E {0, 1} indicates good or response (called Response Evaluation Criteria in Solid Tumors, RECIST, based change in tumor volume), i.e. domain(y..) * domain as shown in Figure
Prior DRP methods (Jayagopal al., 2024 2023 Kim et al.) 2024 He et al. 2022) have addressed representations Z, between X. and X,,, which are then used to train a downstream drug response prediction network f. Transforming X. to Z, increases samples in the shared space and allows f to the larger (Z,, de, yc) in training, thereby tackling the data scarcity issue. However, Zg does not capture patient-specific characteristics in X,, which can influence Up (Liao et al., 2023 Zhai & Liu 2024). To capture this, need augment X, directly. Prior DRP methods, except WISER, neglect this. WISER (Shubham et al. 2024) performs data augmentation by pseudolabelling unlabelled patient profiles X Xp(a) using (X des Me) and trains by combining (Xe; de, ye) and pseudolabelled Xp(u)- However, while combining the datasets, WISER assumes domain (ye) domain (yp), and does not account for P(Xp(w)) + P(X.). We tackle these issues using GANDALF, a Generative AttentioN based Data Augmentation and pre- dictive modeLing Framework. GANDALF augments Xp directly, by generating more "patient- like" samples Xaug leveraging available X also generates their response labels yaug to drugs dang D. Unlike WISER, it explicitly models domain + domain(y,) and P(Xp) x P(X.).
these differences by learning shared domain-invarre
2

---

Published as a conference paper at ICLR 2025
Data augmentation strategies are known to improve prediction performance in various fields of ML, like computer vision (Khosla & Saini 2020) and natural language processing (Shorten & Khosh- goftaar 2019). This is usually achieved through data transformations where identifying the label of the transformed data is relatively easy, e.g., a rotated image of a dog retains the label 'dog' after transformation. However, it is difficult to find such 'label-invariant" transformations for genomic data (Lacan et al.) 2023). Although genomic data can be augmented by interpolation of available samples or sampling new data points from known distribution, assigning labels to these samples is difficult. Data points, which may be "close" together in the representation space, can still exhibit different responses to drugs. If patients are represented by binary vectors (each element correspond- ing to a gene, 1 indicating presence of mutations in a gene and 0 the absence), perturbation is equivalent to addition or removal of a mutation. This perturbation can impact the functioning of the cells and the response to treatment (Hale et al. 2024). Identifying the response associated with each perturbation is difficult due to scarcity of labelled data, making data augmentation strategies challenging in DRP.
Though conclusively identifying labels for all possible perturbations is still an open problem, GANDALF takes step towards leveraging data augmentation in DRP, by utilising available la- belled data from cell lines. It generates Xaug by transforming X, and assigns Yang for generated DD by leveraging labelled information from (Xe, des Me). We use attention mechanisms to ensure that Xang retains information from X.. (Xaug: daug: Baug) is then used with (Xp, p. dp. Bp) to train a downstream DRP classifier. Our paper makes the following contributions:
(X augg dang); ); dang
We are the first to tackle, through novel data augmentation approach, the challenging problem of limited labels for sparse patient genomic data, in cancer drug response predic- tion.
We propose GANDALF, a generative, supersuper attention- data augmentation framework which uses labelled samples from the related cell line domain to generate la- belled patient data.
GANDALF performs data augmentation through a novel synthesis of denoising diffusion probabilistic models, transformers and multi-task learning.
GANDALF demonstrates an improvement of upto 10.96% over SOTA DRP methods, in predicting patient response to drugs, on key benchmark datasets comprising real patient samples with responses to clinically approved anti-cancer drugs. GANDALF also out- performs baseline genomic data augmentation and pseudo-labeling strategies by 21% and 2.5% respectively.
2 RELATED WORK
2.1 DRUG RESPONSE PREDICTION MODELS
Prior DRP models perform transfer learning between the source domain (cell lines) and target domain (patients). These methods can be inductive, transductive or unsupervised (Pan Yang. 2009), based on their use of labelled patient data. Inductive methods, like AITL (Sharifi-Noghabi ett al. 2020). drug2tme (Zhai Liu 2024) and TCRP (Ma et al. 202Tb use both labeled cell line and patient samples. They may either use multi-task learning approaches or few shot learning to capture the differences in label distribution across the two domains. Transductive methods like TUGDA (Peres da Silva et al.) (022)) WISER (Shubham et al. 2024),, PANCDR (Kim et al.) 2024) use labeled cell line and unlabeled patient samples. The unjustified assumption is that the response label does not change across the domains. To this end, most papers convert the continuous valued cell line response to discrete categories as seen in patients, using arbitrary thresholds. Few methods, like CODE-AE (He al. (022), rely on unsupervised transfer learning using unlabeled cell line and patient datasets in pre-training However, in most cases, the goal was to learn a shared representation space between the domains. The shared representation was then used to train a downstream DRP model. While the shared representation captures the similarities across the domains, this approach largely neglects the patient-specific characteristics, which relevant for drug response prediction.
2.2 GENOMIC DATA AUGMENTATION
Genomic data augmentation is difficult due lack of known label-invariant transforms (Lacan et al.
2023). Most existing methods augment transcriptomic data (Das & Shil (022) Chen et al.] 2020)
which is unavailable in clinical setting. A few recent methods (Yu et al. 2024 Lee et al. 2023
3

---

Published as a conference paper at ICLR 2025
Duncan et al., 2024 Lee et al. 2024) have augmented mutations, but they assume that the biological function and associated labels do not undergo changes during data transformation. Moreover, none of these methods focus cancer drug response prediction as the downstream task, where it is known that even the addition or removal of a mutation can cause a change in drug response (Liao et al. 2023). Thus, patient mutation data augmentation for cancer drug response prediction is an open problem. GANDALF proposes a way forward, by using prior information available in labelled cell lines augment patient mutation data and generate associated labels for DRP, rather than assuming label invariance.
3 METHOD
3.1 PROBLEM FORMULATION
Given a patient genomic mutation profile X, and drug dk, the goal in drug response prediction (DRP) is to classify whether the patient would respond well (label 3/p = 1) or not (label Y/, 0), i.c. learn a classifier Sa, (X,) R + {0, Let .M denote the set of all possible mutations found in set of sequenced genes G and A denote the set of possible alterations in G. Each mutation my E M can be separated out into a gene component gi E 9 and alteration as E A. Let D denote the set of chemotherapy drugs. Two related, albeit different datasets are available to perform the DRP task- labelled pre-clinical cell line data and clinical patient data. Cell line genomic data X. C P(M) and labelled patient genomic data p C P(M), where P(.) denotes the power set of M. No, = [Xe] and N₂- IXp denote the number of unique mutation profiles in each dataset. (p(ík) E 1 is binary RECIST response associated with patient-drug pair (Xpi. dx), while Me(s) E [0, 1] is the real-valued AUDRC response for cell line-drug pair (Fegj dz). To illustrate, patient mutation profile I1( = {ms = (92, @10), my = (g100. as)} }
has a response Up(13) = 1 for drug d3. The goal is to predict the response Уp(j2) for a new patient-drug pair (Ips- de). To achieve this, we perform patient data augmentation, i.e. generate (Xoug. daug. yaug) using (Xe, dey ye) and (Xp, dp. Up). de and dp denote the set of drugs available in labelled cell line and patient datasets, and daug S D. In general, [de| > Idpl. de S D and d, C DD as obtaining drug responses in cell lines for wide range of drugs is easier than in patients. The real and generated labelled patient data (X aag, daug- Vaug) U(Xp, p: dg. Up) can then be used to train a downstream DRP classifier f. Please note that can denote e or p in subsequent sections, to denote cell lines and patients respectively.
3.2 METHOD OVERVIEW
We propose a Generative AtteNtion based Data Augmentation and predictive modeLing Frame- work GANDALF, to tackle the labelled patient data scarcity issue via data augmentation. The complete algorithm is available in Algorithm [ GANDALF generates new patient-like samples from cell lines and assigns them labels in 5 steps (1) pretraining diffusion models to learn rep- resentations of X, and Xp. (2) generating new patient-like samples Xong from X., (3) training a multi-task learning network using (Xe, de, Nc) and (Xps dp.
dp. Yp), (4) assigning pscudolabels yaug for (Xoug; doag) )ddug E DD and selection of confident samples (Xe, de, ya) S (Xaugy daug. Yaug) and (5) training DRP classifier f on (X, U Xg. d, U dp, V. U y.)).
The goal is to learn g(.): Xaug 9(x)) ~ P(Xp), which accounts for patient-specifer characteris- tics. The intuition behind the transformation process is: we decompose each domain into domain- invariant Z, and domain-speen specific Zp (for patients) and Ze (for cell lines) representations (Lee & Pavlovic 202I), to transform Xx + ~Xps we introduce Zp over Z, obtained from X.. We can then augment (Xp. dpsUp) Up) using (X (Xangs duug. Yaug), ) daug D. where Yaug be generated by pseu- dolabelling (Lee et al. 20133, Kage et al., (024).. Our pseudolabelling approach assumes that ye and Up share certain characteristics, while differing in others.
3.2.1 STEP I: PRETRAINING DIFFUSION MODELS
In this step, we learn Z,, Zp and Z, representations. We assume Z, (O, 7), which can be mod- elled using denoising diffusion probabilistic model (DDPM) encoders (Ho et al., 2020). The DDPM decoders learn to remove the domain-specific noise, to reconstruct X. Transforming X. + X, would then involve the use of the patient DDPM decoder on Z.. We train two DDPM models (TD, and TD.), one per domain, such that they share a common Z,. In addition, we use the pretrained transformer encoder (Te) from (Jayagopal et al. 2024), with padding, to model varying number of mutations. We use domain alignment losses (Sun et al., 2016) to align Z, and KL-divergence loss to ensure Xang P(X,). We cross-attention ensure X Xang retains information from X..
4

---

Published as a conference paper at ICLR 2025
Algorithm 1 GANDALF training
Require:
Mutation profiles Xx, Xy. drugs D, cell line-drug labels ye. patient-drug labels yp. time steps t, pre-trained transformer encoder Te, DDPM networks TD., VAEs V., pre-train epochs cp, pseudolabel generation epochs c,. upper and lower thresholds and and DRP training epochs ea.
1: Step I: Pretraining diffusion models
2: Obtain transformer embedded samples Zt. (x..) E R..
3: Pre-train domain specific VAFs using Eq. 1 Tand 2
4: for in range(ep) do
5: Extract output from the tranformer- VAE encoder network E= (o(.)(T(())
Z ((u..o.) (S(.) = p. + o.e, where e NV Ho. = E. (X.))
6:
Z (e)
7:
X². denoise Zwoet, (Daa (Zur))
8: Ze = (X'))
9: ZAn softmaxl Zep Zuct
Zuct
Vi
10: Zepa denoise ZAu, L. Piia (Zau)) using Eq. Minimise loss LPRE until convergence.
12: end for
13: Step 2: Generating new patient-like samples Zuct (Dete) (Vote) (Zrc)
Xa Xaug = denoisel (Zect, f, <po): TDp(a) (Zrect
14: Step 3: Training multi-task learning network
15: for e in range(e,) do
16: Obtain cell line and patient embeddings Z... (X.))
17: Obtain drug embeddings Za. ga(d.)
18: For each sample, drug pair concatenate the embeddings to get Ond Ze. 1|
19: Obtain AUDRC and RECIST predictions: Ve = 9a (Oea): Up g. (0pp)
20:
Minimise LMTZ till convergence.
21: end for
22: Step 4: Assigning pscudolabels and selection of confident samples
23: Yang = 9- 99-X
for daug DD
24: Set ybun. as if Youg >= tu Yaug < t and otherwise.
25: Select confident tuples (non-abe abstained tuples) (X.,, d,, V=). Le. where yhur o 1.
26: Combine (X., d., ya) with (X" y. dy. Hp) to form (Xcomb, deomb Deumb)
27: Step 5: Training drug response prediction classifier
28: for in range(e.) do
29: Veomb = f(Xcombl (dcom))
30: Minimise loss LACE Eq. till convergence.
31: end for
Te takes input {m, my M). Each my has parts the gene part gi G and the alteration part or E A. 9I and at are tokenized separately, padded and concatenated to generate per-sample vector. In the embedding step, each a; is embedded following the variant annotation procedure
in (Jayagopal et al. 2024), to obtain a 23-dimensional embedding. This consists of a 17 dimensional binary vector from Annovar (Wang et al. 2010). 3 d-dimensional binary vector each from GPD (Li et al. (020) and Clin Var (Landrum et al. 2018). The embedding for each is passed through linear layer and concatenated with the corresponding gi embedding (obtained by one hot encoding), before being fed into Te. The resulting output is mean-aggregated to obtain sample embedding 7zx = (X,)) E N.N. where denotes the maximum sequence length. k: set based on maximum number of alterations in the training data, and all sequences was trained to predict the progression-Fee free survival (PFS) for (Xp, dp). PFS is indicative of the time after treatment that cancer patient survives without the cancer progressing. For further details, please refer to (Jaygooal et al. 2024).
padded to match k. T.
To ease training (Rombach et al. 2022 we reduce the dimensionality of Z,, from + 1,1 using variational autoencoders (VAEs) (Kingma & Welling 2013). We use 2 VAEs V. and V,, for cell line and patient domains respectively. These VAEs take as input Z., E R.N x and estimate the mean Hes Mp R.N.xt and standard deviation Tp.ap E R N. of each domain. Samples generated using the estimated j and a are used to train TD, .. 'Te VAEs are pretrained on each domain, to minimise reconstruction mean square enro and KI. divergence loss as in Eq. and 2 The VAE
5

---

Published as a conference paper at ICLR 2025
X
Feed forward Layers
Feed forward layers
Feed forward layera 7P
Feed forward leyers yD
Food forward layors
Cross ottention layer
Shared latent Space 2
Mean Aggregation
Layer Normolization
Feed forward Layers mD
Feed forward layers TD
Food forword layers
Zyc
Sampling
Layer Normelization
M
Multi
Encoder E
Encoder E
Query
Value
Gene1, 2
Mutation 10, 15
Cell lines X.
Patients X
Figure 2: GANDALF architecture used for pretraining domain speccifc diffusion models and to generate new patient-like samples using available cell line data. Circled numbers in blue indicate steps from Algorithm
pretraining loss is LV LR + LKLD.
La N. EN. EN.(Te. -Ze,)²
LKLD -0.5/N..) log (Zz.) )²) ) po Mo (2.2)))² o..())) -o.
(2)
where No denotes number of mutation profiles (N. or Npp,, Ze is the reconstructed VAE output. Pretrained T, attached to the encoder layers of the pretrained Ve and Vp, are henceforth referred to as encoder networks E. and Epi u.. o. E. (X.)) Parameters of T. are frozen for training.
The sampled output from E., Z.. =S(,, (5(.) μ4 + o.E denotes VAE sampling, where 6 (O 7)) is fed into TDe and TDp. with encoder TDx(e) and decoder TD Since 7 is vector, used feed forward linear layers in TD_ (Kotelnikov al., (023). To learn Z., we perform domain alignment, using CORAL loss (Sun al. 2016). CORAL loss minimises the co-variance between the latent spaces, as in Eq. 4 Although in theory, DDPM encoders should yield isotropic Gaussians as T + oc, the use of CORAL loss enforces that the two domains share Z.g, when 7' is finite. TD, and TD, are trained jointly with the CORAL loss using LALIGN
LDDPM LCORAL, in Eq. and
LDDPM = E(zue.c.. (ont 1)}² Eizop-cp.t) cpo 1))
(3)
[|] (() - Em. 2z)(Zs -Z;)²
LOORAL -EEE||(C(Z.L) C(Z
(4)
6. and Ep are ground truth noise distributions added to X and Xp. Zuct = TD (Zvc) and Zupt = TDD (Zup) are the noisy representations after timesteps through TDa(e)- Ecp and ep are estimated by (D) Z denotes mean. Zu is denoised using 6.0 (Eq. 5 obtain X' and X' p- These are passed through VAE decoders to obtain Zee = Volay (2" and Zep = Vp(as (X' p). 3r in Eq. is the variance schedule (Nichol Dhariwal 2021) of 6, and 6p at diffusion step time t.
X' denoise( 620); X", denoise tups, tp))
(5)
where denoise(XX, t,) (X.
ái = (as) = =1 3r
To ensure that Xaug preserves information from X., we use cross-attention Rombach et al. 2022
Given, Zuet and Zupt, we obtain Z = softmaz( Zezee 2. Vi
Zuct. ZAc pays attention to Zuct. Z
passed through TDp(d) p(d) and denoised using 6po to obtain Zupa- A KL. divergence loss LKLDA is also calculated between the distributions of Zup and Zopa ensure eventual adherence to P(X,), as

---

Published as a conference paper at ICLR 2025
in Equation 6 Additional mean square error terms LASE between Z4, and Zea and KL divergence
terms LKLDV for Zv. are caleulated as in Equation
LKLDA 0.52 (-1+ (o (-1g )²) log(o (xp(oo(o (oggooo((
(6)
-log(o(Zzppa")) + (u(Zup) 1 (xp(=))))))((((((((((
IMSE 1 LNN, (7r -Tra)²
N.
LKLDV (0.5/N.)E log(o- -1. (Z..))² -z(
The overall training loss is Lpne LALIGN + LKLDA + LKLDV + LMSE- Architecture details are available Figure 2 labeled data.
The training done an unsupervised manner and does not require
3.2.2 STEP 2: GENERATING NEW PATIENT-LIKE SAMPLES
To generate Xaug. we run inference on the trained model using X,- X₂ is first passed through Te, followed by Ve(e). to get Ze This is then passed through DD(ee) to get Zuct- This is analogous to removing Ze from the input samples. As the latent spaces of the DDPMs are already aligned, Zeet can be denoised using P(d(d) to obtain Xaug- This step corresponds to introducing Zp to Z, The red arrows in Figure 2 indicates the generation of X Xuug from X
3.2.3 STEP 3: TRAINING MULTI-TASK LEARNING NETWORK
In this step, the goal is to train network to assign Youg V (Xaug- daug), daug E D. A naive approach would involve training classifier f on (Xp: dp Up) and using it to predict yaug. However, dp C daug. since only a small subset of drugs are provided to patients as per clinical guidelines. This implies that P(Xp, dp, Vp) learnt by may not fully model P(Xp U Xaug. dp U daag. Upp U Voog). During inference, f may encounter drugs outside of the training set, yielding noisy Yaug- A similar constraint exists in using weak supervision methods (Ratner et al.] [017]] Zhang et al., (2022) to assign pseudo-labels. Further, f can be prone to overfitting, given the small size of (X,, dp- Up)
In this step we alleviate overfitting concerns using larger data (X yely... in multi-task learning (MTL) setup, with additional regularizing loss terms. Moreover, de ~ D, which allows the network to learn from drugs & dy. We also capture the shared traits between ye and y, by projecting labelled (Xe, de) and (Xp: dp) into a shared latent space O,, and capture the differences, via two separate prediction heads
classification head D3 gr (Xp. dp) {0. and regression head Ve (X (x) E [0, 1]. O₂ is learnt by aligning the latent representations, using CORAL loss (Sun) et al. (016), as in Equation 8 X, and Xp are first passed through the pretrained encoder network E.) to obtain Mes Up, oe and op. Sampling S is applied as before to obtain Zuo and Zup. Zve and Zep are concatenated with drug embeddings obtained from feedforward multi-layer perceptron (MLP) Za- = ga(d.) E RN. xi The resulting concatenated representations Ond Zvell Idd E R N222 and Opd = Zupll Záp x2x where || denotes concatenation, N₂ = [X der Vc)l and N₂ = 1(Xp. dp Up)l denote number of labelled sample, drug pairs (N, < Nc).
LCORALD- = Exd C(Oca)) - C(Op,))I C(2) (Z. - 2.)(Z. - -2,)"
(8)
n
Ocd is passed through a feed-forward MLP 9a to predict AUDRC values de = ga (Oea). Opa is
passed through another feed forward MLP ge predict RECIST values 1i = pod)). The entire
network is trained to minimise LMTT. = LACE + LMSE + LCORAL. o as in Equation 9 where (x) Ite-z. MTL architecture is shown in Figure 3eftt.
1 N.
LBCF
EN, (uplog( N
Mo) (1i)))); LMSE = ENN (Ve D6)²
3.2.4 STEP 4: ASSIGNING PSEUDOLABELS AND SELECTION OF CONFIDENT SAMPLES
To obtain Vaug. we first generate all possible Nr x (D) pairs (Xaugs daug), ), daug E D. We pass the drug representation dang through ga. We concatenate the resulting drug embedding gu (daug) with X Xaug- This is then passed through ge and a() to get Yang E (0, 1]. as shown in Figure 3 right, top).
7

---

Published as a conference paper at ICLR 2025
Fwed forward Layers
Fecolfowwrd a
Foudlorourd &
Drug Sana
Peesifarcant
lyyengg
Xaee
Soece
Parinnt-lke xamplis Xewa
Pscudo- laberiay
Drugh,
Feed forwerd loyers
Reass
Enoodert
Encoder
Reat samples
Pasenta.X,
Figure 3: GANDALF architecture for multi-task training (left), pseudolabel generation and selection of confident samples (right, top) and training downstream DRP model (right, bottom). Circled numbers in blue indicate steps from Algorithm
(Xangy dangs yaug) may however be noisy due to incorrect predictions from gr. Prior work on subset selection (Lang al. 2022) has identified that choosing subset of more confident pseudolabelled samples is more effective than using the complete pseudolabelled dataset. We use Maugs to select this subset. Ybin is generated by binning Youg into 3 groups, using upper and lower threshold tu and t,. Ybin 1, if Yaug >= tui Y/inn O, Yaug < t and Ybin = -1 otherwise (abstained samples). Only N. < (Ne x [DI) high confidence (non-abstained) samples (ybin + -1) used for the downstream DRP classifier training.
3.2.5 STEP 5: TRAINING DRUG RESPONSE PREDICTION CLASSIFIER
The non-abstained, high confidence generated "patient" drug pairs after pseudo labeling ((Xg, dgy size N,) are combined with N₂ (X' dp. Up) pairs to train a drug response pre-
dicting feed forward neural network f (Figure 3 right, bottom). f is trained to minimise BCE loss in Eq.
1 LECE N, (v.-oN.ullog(a(ii + (1- v.)log(1 - -
(10)
GANDALF offers several advantages. The use of VAEs and DDPMs makes the model generative in nature. While generation in DDPMs usually involves sampling from /N I) and denoising, here the sampling incorporates prior knowledge from X.. This also enables the use of (Xe, day Uo) in generating pseudo-labels for Xaug. When N₂ > 0, it reduces chances of overfitting.
4
EXPERIMENTS AND RESULTS 4.1 DATASETS
We used publicly available cell line and patient datasets, for all our experiments. Cell line mutation profiles were obtained from the Cancer Cell Line Encyclopedia (CCLE) DepMap (v23Q4) (Ghandi et al.) 2019 Barretina et al. 2012). AUDRC responses were obtained from the GDSCv2 (lorio et al., 2016 Yang et al. 2012). Patient mutation profiles and associated response labels for drugs were col- lected from The Cancer Genome Atlas (TCGA) (Weinstein al. (2013). ChioPortal (CBIO) (Hard-
ing et al. 2019; Nixon et al. 2019 de Bruijn et al.) al.] 2023; Gao et al. 2013 Cerami et al. 2012) and UC SinDicgo Moores Cancer Center (Moores) (Schwacderle et al. 2016 Patient response, mea- sured via RECIST were coalesced into binary labels (I: positive response; O: negative) (Peres da encoded using 2048 dimensional binary Morgan fingerprints (Mor- gan 1965). We exclude samples on multiple drug regimen and retain only patients given a single drug time. This results 1 197 CCLE samples, 541 TCGA, 44 Moores and 84 CBIO patient samples with documented response labels for 21 drugs in cell lines and 56 drugs across patients. We restrict our analysis to the 324 genes found in popular clinical sequencing panel, FoundationOne CDx (Milbury et al.) (202)) and removed samples without mutations in these genes. We also removed samples with responses to drugs without Morgan fingerprint. For the transformer pretraining, we used 71 non-small cell lung cancer and 71 colorectal cancer samples from GENIE (Choudhury et al.) 2023 Garcia et al., 2023), with a documented progression-fre survival. We had a total 156441 train, 17371 validation and 2588 test cell line, drug pairs We also had 488/488/487 train, 53/54/56 validation and 115/114/113 test patient, drug pairs over folds (folds 0/1/2 respectively) (details in
Silva et al. 2021). Drugs
Appendix Section A.I
8

---

Published as a conference paper at ICLR 2025
Table 1: Performance comparison across SOTA drug response prediction methods. Best performing results are highlighted in bold, while the second best performing results are underlined.
AUROC (Mean
Standard deviation)
Method
Cis
Flu
Gem
Pac
Tem
GANDALF
0.6343 + 0.0306
0.7309 + 0,0664
0.6188 1 0.0674
0.7728 + 0.1253
0.6451 = 0.0776
DrulD
0.6764
0.1447
0.6071 + 0.1988
0.5092 + 0.1005
0,5119 + 0.2324
0.6194 0.0420
PANCDR
0.6278 0.0308
0.4762 1 0.1798
0.4429 1 0.2268
0.4236 1 0.4168
0.6436 1 0023100
PREDICT-AI
0.5072 0.0331
0.3869 + 0.0372
0.5046 + 0.1181
0.6815 + 0.1786
0.5350 0.0606
drug2tme
0.5243 0.1301
0.7167 + 0.1957
0.4568 + 0.0857
0.3194 + 0.3127
0.5951 0.2541
WISER
0.4622 + 0. 1685
0.6095 + 0.193
0.4305 + 0.0867
0.3641 + 0.2522
0.5297 0.0738
CODE-AE
0.6322 0.1872
0.5381 + 0.1606
0.5085 + 0.0503
0.3611 - 0.3155
0.4332 = 0.3123
AUPRC (Mean
1 Standard deviation)
Method
Cis
Flu
Gem
Pac
Tem
GANDALF
0.9093 + 0.0355
0.8483 + 0.0933
0.5874 + 0.175
0.9558 + 0.024
0.2535 + 0.1
DrulD
0.9176 + 0.0671
0.7588 + 0.1484
0.4515 + 0.17
0.8897 + 0.0223
0.3014 + 0,103
PANCDR
0.9018 + 0.0324
0.6951 + 0.1530
0.4562 + 0.2270
0.8561 + 0.1019
0.3049 = 0.2653
PREDICT-AI
0,8622 + 0.0189
0.5885 + 0.0581
0.3873 + 0.0489
0.8687 = 0,1090
0.1373 = 0.0050
drug2tme
0,8754 1 0.0523
0.8092 + 0.1722
0.4826 1 0.0947
0.7824 1 0,1023
0,3058 + 0.1327
WISER
0,8454 + 0.0685
0.7505 + 0.0657
0.3901 + 0.0885
0.7724 + 0.1585
0.1762 = 0.0243
CODE-AE
0.9059 0.0521
0.6665 + 0.1435
0.4735 + 0.0701
0.8208 + 0.0574
0.17 0.0929
4.2 COMPARISON WITH CANCER DRUG RESPONSE PREDICTION METHODS
We compared GANDALF against 4 recent state of-te-rft (SOTA) methods which take sample, drug pairs as model inputs, namely, Drull) (Jayagopal et al., 2023), PREDICT-AI (Jayagopal al.) 2024), drug2tme (Zhai & Liu, 2024) and PANCDR (Kim et al. (024). We also compared GAN- DALF against CODE-AE (He et al. 2022) and WISER (Shubham et al. 2024), which train sepa- rate models per drug. We report performance metrics on 5 drugs, with samples available in all test folds, namely Cisplatin (Cis), Paclitaxel (Pac), 5-Fluorouracil (Flu), Gemcitabine (Gem) and Temozolomide (Tem). We do sample, drug pairs for the drug considered. For CODE-AE and WISER, we train separate models per drug. Apart from GANDALF, only PREDICT-AI could handle varying length inputs. For all other methods, we converted the mutation profiles into fixed length input vectors of 7776 dimen- sions, following the pre- preceeseesing (Jayagopal et al. 2023). Validation set correlation between predicted and actual response was used for early stopping and syyper-parametr selection. As shown Table GANDALF achieves the best AUROC in Flu, Gem, Pac and Tem and second-best in Cis. GANDALF achieves the best AUPRC score in Flu, Gem and Pac, and second-best in Cis.
model tuning in GANDALF, by only augmenting with
4.3 ABLATION STUDY
Next, we performed an ablation study to empirically verify the importance of each component in the architecture, We successively removed each component and measured the overall AUROC and AUPRC performance across all the drugs in the test set. The key components of GANDALF are the MTL network for pscudolabeling, cross-attention in pretraining DDPMs and use of transformers to model varying length inputs. We first removed the cell line head in the MTL network (W/O MTT.). Next, we removed the cross-attention KI. divergence loss LKLDA (W/O cross-attention We then removed the use of pretrained transfomer (W/O transformer) in the input to the network and instead used the 7776 dimensional input used by other SOTA methods. The full model with all components shows the best performance in terms of both AUROC and AUPRC, highlighting the importance of each component in the overall performance (Table 2 Ablation). The above ablation removes each component successively from the architecture. In Appendix [..]] we have also included ablation tests where only one component is removed at a time. We also analyse test performance sensitivity to increased volume of pseudolabelled data; details in Appendix Section A.2 A low to moderate volume of high confidence samples is better than large volume of low confidence samples.
4.4 COMPARISON WITH OTHER AUGMENTATION STRATEGIES
There are no known label-invariant mutation data augmentation approaches for cancer DRP (refer Section 2.2 for details). As baseline, we compare GANDALF against a naive data augmentation approach (Lee et al., 2023), where we perturb the 7776 dimensional inputs, using samples from (O, I). This is done once per patient, drug pair (W perturbation) in the training data, and the associated label is assumed to remain the same as in the original sample, resulting in a dataset of size 2 p- In addition, also compare GANDALF against vanilla feed-forward ML. (W/O

---

Published as a conference paper at ICLR 2025
Table Contribution of various components (ablation) in GANDALF, comparisons with other aug- mentation and pseudolabeling strategies.
Experiment
Method GANDALF W/O MTL
AUROC (mean = std)
AUPRCC (mean + std)
0.8409 + 0.0437
0.778 + 0.0255
0.753 = 0.1637
0.6448 + 0.1604
Ablation
W/O cross-are ress-aatentionn
0.752 = 0.165
0.6443 + 0.1636
W/O transformer
0.6007 = 0.08
0,5632 + 0.1101
W perturbation
0.6306 = 0.0255
0.5967 + 0.0611
Augmentation
W/O aug
0.6052 = 0.0219
0.5784 + 0.0394
Pscudolabeling
majority vote
0.8153 + 0.0541
0.756 + 0.0827
aug), trained using only (xp.dp: dp. Up). We compare the learning curves (Appendix F and test performance metrics (Table [ Augmentation). In both cases, we fix training epochs. In all folds, no augmentation and Gaussian perturbation strategies result overfitting, where the validation loss show an increase while the training loss remains low. This is consistent with the fact that smaller datasets can result in overfitting. The test performance metrics for these methods is lower than that of GANDALF. The slight improvement due to perturbation indicates the benefit of data augmentation in improving overall performance.
4.5 COMPARISON WITH MAJORITY VOTE BASED PSEUDOLABELING
We compared MTL based pseudolabeling strategy against another pseudolabeling strategy similar to Mong-DongChhrn & WeiGao 2018). The augmented data (Xoug. deag, Bang) is passed through separate feed-forward networks, trained on (Xp, dp Up). The pseudolabels generated by each net- work is aggregated by majority voting (Lang et al. 2022). As before, non-abstained samples are used to train the downstream DRP model, along with (X dp. Vy)) The results comparing GANDALF against this approach (W majority vote) are shown in Table 2 Pseudolabeling. While the majority voting strategy does perform well, GANDALF outperforms it overall AUROC and AUPRC. This may be potentially due to the use of the larger cell line labelled data, with more drugs, opposed to the smaller labelled patient dataset.
5 CONCLUSIONS AND DISCUSSION
In this paper, we propose GANDALF, generative patient data augmentation framework, to tackle the challenge of training cancer DRP model with limited labelled data. Unlike prior DRP methods that augment data in the shared space between patients and cell lines, we utilise the larger labelled cell line dataset to generate more patient-like samples as well as their pseudo-labels. GANDALF outperforms SOTA DRP methods, and also shows improved performance when compared to base- line genomic data augmentation and pseudo labeling approaches. GANDALF has large number of parameters and sub-modules, each of which needs pretraining, increasing overall training time. Learning the underlying data distributions is limited by available labelled cell lines and patients.
There several future directions to explore, which may improve GANDALF further. In this paper, we have only considered labelled patient profiles for training, although the pretraining stage sup- ports unlabelled data. Future work can evaluate the use of unlabelled patient profiles in all steps of training. We examined the quality of the generated samples by comparing the distributions against the original patient data. More extensive studies examine the biological significance of the gen- erated samples and their fidelity can shed light on the patterns captured by the model. Generative strategies, which can incorporate known biological information on co- oocuurunn mutations, can also be explored in the future. In the cell line datasets we used, we have included both solid and non-solid tumor types, that can lead to differences in pharmacological responses (Basu et al., 2013 Yao al., 2018 Gerdes et al. 2 Sharifi-Noghat et al.) 2021b). The effect of these tumor types on model performance can be examined in the future. We could even build tumor type specific models by fine-tuning the existing model using data specific to each cancer type. Overall, GANDALF sets the stage for using generative techniques in the field of cancer DRP research, and emphasises the im- portance of capturing patient domain-speciff characteristics for improving downstream prediction performance.
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY
Our code and data are made publicly available at htttp::/github.cco http:/thhwu..mm/aaaaaaaaaa
GANDALF
ACKNOWLEDGMENTS
This research is supported by the National Research Foundation, Singapore under its Al Singa- pore Programme (Award Number: by the Ministry of Health, Singapore, through the NMRC Clinician Scientist Award (MOH- CSAINV20nov-0010) Aishwarya Jayagopal is supported by the National University of Singapore Research Scholarship. We would like to acknowledge the American Association for Cancer Re- search in the development of the AACR Project GENIE registry, as well as members of the consor- tium for their commitment to data sharing. Images in this paper were created using Flatlcon and Freepik.
Anand D. Jeyasekharan supported
REFERENCES
Jordi Barretina, Giordano Caponigro, Nicolas Stransky, Kavitha Venkatesan, Adam A Margolin, Sungjoon Kim, Christopher J Wilson, Joseph Lehár, Gregory V Kryukov, Dmitriy Sonkin, et al. The cancer cell line encyclopedia enables predictive modelling anticancer drug sensitivity. Nature, (600333666666677 2012.
Amrita Basu, Nicole Bodycombe, Jaime H Cheah, Edmund Price, Ke Liu, Giannina Schae- fer, Richard Y Ebright, Michelle I. Stewart, Daisuke Ito, Stephanie Wang, et al. An interactive resource to identify cancer genetic and lineage dependencies targeted by small molecules. Cell, 154(5;1151-1161: 2013.
Pavla Brachova, Kristina W Thiel, and Kimberly K Leslie. The consequence of oncomorphic tp53 mutations in ovarian cancer. International journal of molecular sciences, 1499):19257-12275 2013.
Ethan Cerami, Jianjiong Gao, Ugur Dogrusoz, Benjamin E Gross, Selcuk Onur Sumer, Bülent Ar- man Aksoy, Anders Jacobsen, Caitlin J Byrne, Michael L. Heuer, Erik Larsson, et al. The cbio cancer genomics portal: an open platform for exploring multidimensional cancer genomics data. Cancer discovery, 2(5))401-404 2012.
Junjie Chen, Mohammad Erfan Mowlaei, and Xinghua Shi. Population-scale genomic data aug- mentation based on conditional generative adversarial networks. In Proceedings of the 1 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, pp. 1-6, 2020.
Noura J Choudhury, Jessica A Lavery, Samantha Brown, Ino de Bruijn, Justin Jee, Thinh Ngoc Tran, Hira Rizvi, Kathryn C Arbour, Karissa Whiting, Ronglai Shen, et al. The genie bpc nscle cohort: A real-world repository integrating standardized clinical and genomic data for 1,846 patients with non-small cell lung cancer. Clinical Cancer Research, 2911)):4418-1 18341883455 8-282,, 2023.
Francis Collins and Harold Varmus. A new initiative on precision medicine. New journal
of medicine, 372(9):793-795, 2015.
Ramon Colomer, Jesús Miranda, Nuria Romero-Laorden, Javier Hornedo, Lucfa González-Cortioo Silvana Mouron, Maria J Bueno, Rebeca Mondéjar, and Miguel Muintela-Fandine Usefulness and real-world outcomes of next generation sequencing testing in patients with cancer: an obser- vational study the impact of selection based clinical judgement. EClinicalMedicine, 60, 2023.
T Conroy, P Pfeiffer, V Vilgrain, Angela Lamarca, T Seufferlein, EM O' O'Reilly, T Hackert, T Golan, G Prager, Haustermans, et Pancreatic cancer: Esmo clinical practice guideline for diagnosis, treatment and follow-up. Annals of oncology, 34(1 11:987-10022 2023.
Supratim Das and Xinghua Shi. Offsprrng gan augments biased human genomic data. In Proceed- ings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, pp. 1-10, 2022.

---

Published as a conference paper at ICLR 2025
Saloni Dattani, Fiona Spooner, Hannah Ritchie, and Max Roser. Causes of death. Our World in Data, 2023. https///ouwoolkiidtttrr//uuuusss
Ino de Bruijn, Ritika Kundra, Brooke Mastrogiacomo, Thinh Ngoe Tran, Luke Sikina, Tali Mazor, Xiang Li, Angelica Ochoa, Gaofei Zhao, Bryan Lai, et al. Analysis and visualization of longitu- dinal genomic and clinical data from the aacr project genie biopharma collaborative in cbioportal. Cancer research, 83(23):3861-3880 2023.
W Dong- DongCennnn and ZH WeiGao. Tri-net for semi- suni-supervieed deep learning. In Proceedings of /wenty-seventh international joint conference on artificial intelligence, pp. 2014-2020, 2018.
Andrew G Duncan, Jennifer A Mitchell, and Alan M Moses. Improving the performance of super- vised deep learning for regulatory genomies using phylogenetic augmentation. Bioinformatics, 40(4):btae190) 2024.
Jianjiong Gao, Bülent Arman Aksoy, Ugur Dogrusoz, Gideon Dresdner, Benjamin Gross, S Onur Sumer, Yichao Sun, Anders Jacobsen, Rileen Sinha, Erik Larsson, et al. Integrative analysis of complex cancer genomics and clinical profiles using the chioportal. Science signaling, 6(269): pll-pl1, 2013.
Enrique Sanz Garcia, Eric Chen, Marios Giannakis, Gregory J Riely, Jeremy L Warner, Michele L LeNouu-Newton. Jessica Weiss, Katrina Hueniken, Kenneth Kehl, Deborah Schrag, et al. Ge- nomic characteristics and clinical outcomes of early onset colorectal cancer (eocrc): Findings from aacr project genie biopharma collaborative registry. Cancer Research, 83(7 SNV7_Supplement; 1177-1177, 2023.
Henry Gerdes, Pedro Casado, Arran Dokal, Maruan Hijazi, Nosheen Akhtar, Ruth Osuntola, Vinoth- ini Rajeeve, Jude Fitzgibbon, Jon Travers, David Britton, al. Drug ranking using machine learning systematically predicts the efficacy of anti-cancer drugs. Nature communications, 12(1):
1850, 2021.
Mahmoud Ghandi, Franklin W Huang, Judit Valé-Val Gregory Kryukov, Christopher C Lo. E Robert McDonald III, Jordi Barretina, Ellen T Gelfand, Craig M Bielski, Haoxin Li, et al. dext-generation characterization of the cancer cell line encyclopedia. Nature, 569(7757):503- 508, 2019.
Joseph Hale, Takeshi Matsui, Ilan Goldstein, Martin N Mullis, Kevin Roy, Christopher Ne Ville, Darach Miller, Charley Wang, Trevor Reynolds, Lars M Steinmetz, et al. Genome-scale analysis of interactions between genetic perturbations and natural variation. Nature Communications, 15 (1):4234, 2024.
James Harding, Subhiksha Nandakumar, Joshua Armenia, Danny Khalil, Melanie Albano, Michele Ly. Jinru Shia, Jaclyn F Hechtman, Ritika Kundra, Imane El Dika, et al. Prospective genotyping hepatocellular carcinoma: clinical implications of next-generation sequencing for matching patients to targeted and immune therapies. Clinical Cancer Research, 25(7):21162213 2019.
Di He. Qiao Liu, You Wu, and Lei Xie. A context-aware deconfounding autoencoder for robust prediction of personalized clinical drug response from cell-line compound screening. Nature Machine Intelligence, 4(10):879-892, 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 68511 2020.
Francesco Iorio, Theo A Knijnenburg. Daniel J Vis, Graham R Bignell, Michael P Menden, Michael Schubert, Nanne Aben, Emanuel Gonçalves, Syd Barthorpe, Howard Lightfoot, et al. A landscape of pharmacogenomic interactions in cancer. Cell, 166(3):740-754, 2016.
Aishwarya Jayagopal, Robert Walsh, Krishna Kumar Hariprasannan, Ragunathan Mariappan, De- babrata Mahapatra, Patrick William Jaynes, Diana Lim, David Shao Peng Tan, Tuan Zea Tan, Jason Pitt, et al. A multi-task domain domain-adapted model to predict chemotherapy response from mutations in recurrently altered cancer genes. medRxiv, pp. 2023-11, 2023.
12

---

Published as a conference paper at ICLR 2025
Aishwarya Jayagopal, Hansheng Xue, Ziyang He, Robert J Walsh, Krishna Kumar Hariprasannan, David Shao Peng Tan, Tuan Zea Tan, Jason J Pitt, Anand D Jeyasekharan, and Vaibhav Rajan. Personalised drug identifier for cancer treatment with transformers using auxiliary information. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp- 5138-514999 2024.
Patrick Kage, Jay Rothenherger, Pavlos Andreadis, and Dimitrios Diochnos. A review of pseudo- labeling for computer vision. arXiv preprint arXiv:2408.0722/ 2024.
Cherry Khosla and Baljit Singh Saini. Enhancing performance of deep learning models with dif- ferent data augmentation techniques: A survey. In 2020 International Conference Intelligent Engineering and Management (ICIEM), pp. 79-85. IEEE, 2020.
Juyeon Kim, Syn Park, and Hyunju Lee. Pancdr: precise medicine prediction using an adver- sarial network for cancer drug response. Briefings in Bioinformatics, 25(2):bbae088, 2024.
Diederik P Kingma and Max Welling, 1312.61 14, 2013.
Auto-encoding
variational bayes.
arXiv preprint
Akim Kotelnikov, Dmitry Baranchuk, Ivan Rubachev. and Artem Babenko. Tabddpm: Modelling tabular data with diffusion models. In International Conference on Machine Learning, pp. 17564- 17579. PMLR, 2023.
Alice Lacan, Michèle Sebag, and Blaise Hanczar. based data augmentation for transcriptomics: survey and comparative assessment. Bioinformatics, 39SSpplement 1):il11-i121 2023.
Melissa J Landrum, Jennifer M Lee, Mark Benson, Garth R Brown, Chen Chao, Shanmuga Chi- tipiralla, Baoshan Gu, Jennifer Hart, Douglas Hoffman, Wonhee Jang, et al. Clinvar: improving access to variant interpretations and supporting evidence. Nucleic acids research, 46(D1): D1067, 2018.
Hunter Lang, Aravindan Vijayaraghavan, and David Sontag. Training subset selection for weak supervision. Advances in Neural Information Processing Systems, 15:16023-160333 2022.
Dong-Hyun Lee et al. Pseudo-label: The simple and efficient cuni-supervise learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, pp. 896. Atlanta, 2013.
Hyunjung Lee, Utku Ozbulak, Homin Park, Stephen Depuydt, Wesley De Neve, and Joris Vanker- schaver. Assessing the reliability of point mutation as data augmentation for deep learning with genomic data. BMC bioinformatics, 25(1):170, 2024.
Mihee Lee and Vladimir Pavlovic. Private-shared disentangled multimodal vae for learning of la- tent representations. In Proceedings of the ieeelcvf conference computer vision and pattern recognition, pp. (692-1700 2021.
Nicholas Keone Lee, Ziqi Tang, Shushan Toneyan, and Peter K Koo. Evoaug: improving gener- alization and interpretability genomic deep neural networks with evolution-inspiree data aug- mentations. Genome Biology, 24(1):105, 2023.
Ginny XH Li, Dan Munro, Damian Fermin, Christine Vogel, and Hyungwon Choi. A protein- centric approach for exome variant aggregation enables sensitive association analysis with clinical outcomes. Human mutation, 41(5):934 945, 2020.
Jinzhuang Liao, Xiaoying Li, Yu Gan, Shuangze Han, Pengfei Rong, Wei Wang, Wei Li, and Li Zhou. Artificial intelligence assists precision medicine in cancer treatment. Frontiers in oncol- ogy, 12:998222, 2023.
Ruishan Liu, Shemra Rizzo, Sarah Waliany, Marius Rene Garmhausen, Navdeep Pal, Zhi Huang, Nayan Chaudhary, Lisa Wang, Chris Harbron, Joel Neal, al. Systematic pan-cancer analysis of mutation treetmmen interactions using large real-world clinicogenomics data. Nature Medicine, 2866):665611661 2022. 2022.
13

---

Published as a conference paper at ICLR 2025
Gilberto Lopes. The global economic cost of cancer estimating it is just the first step! JAMA oncology. 9(4):461-45) 2023.
Jianzhu Ma, Samson H Fong, Yunan Luo, Christopher J Bakkenist, John Paul Shen, Soufiane Mour- ragui, Lodewyk FA Wessels, Mare Hafner, Roded Sharan, Jian Peng, al. Few-shot learning creates predictive models of drug response that translate from tightthroughput screens to individ- ual patients. Nature Cancer, 2(2):233-244, 2021.
Coren A Milbury, James Creeden, Wai-Ki Yip. David L Smith, Varun Pattani, Kristi Maxwell, Bethany Sawchyn, Ole Gjoerup, Wei Meng, Joel Skoletsky, et al. Clinical and analytical vali- dation of @oundationone@ edx, a comprehensive genomic profiling assay for solid tumors. PLoS One, 17(3)xe2244138, 2022.
Harry L Morgan. The generation of unique machine description for chemical structures-a tech- nique developed at chemical abstracts service. Journal of chemical documentation, 5(2):107-113, 1965.
Van Morris, Erin Kennedy, Nancy Baxter, Al B Benson III, Andrea Cercek, May Cho, Kris- ten K Ciombor, Chiara Cremolini, Anjee Davis, Dustin A Deming, et al. Treatment of metastatic colorectal cancer: Asco guideline. Journal of Clinical Oncology, 41(3):678-700, 2023.
Soufiane Mourragui, Marco Loog, Mark A Van De Wiel, Marcel JT Reinders, and Lodewyk FA Wessels. Precise: domain adaptation approach transfer predictors of drug response from pre-clinical models to tumors. Bioinformatics, 35(14):55 35(14):510-i511) 10-i519, 2019.
Soufiane MC Mourragui, Marco Loog, Daniel J Vis, Kat Moore, Anna G Manjon, Mark A van de Wiel, Marcel JT Reinders, and Lodewyk FA Wessels. Predicting patient response with models trained on cell lines and patient-derived xenografts by nonlinear transfer learning. Proceedings of the National Academy of Sciences, 118(49):e21666221 2021.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusson probabilistic models. In International conference machine learning, pp. 8162-8171. PMLR, 2021.
Mellissa J Nixon, Luigi Formisano, Ingrid A Mayer, M Valeria Estrada, Paula I Forzslez--riisssnn, Steven J Isakoff, Andrés Porero-Torres Helen Won, Melinda E Sanders, David B Solit, et al. Pik3ca and map3kl alterations imply luminal a status and are associated with clinical benefit from pan-pic van-pi3k inhibitor buparlisib and letrozole in er+ metastatic breast cancer. NPJ Breast Cancer, 5 (1):31, 2019.
Sinno Jialin Pan and Qiang Yang. survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10))1345-133 2009.
Rafael Peres da Silva, Chayapom Suphavilai, and Niranjan Nagarajan. Tugda: task uncertainty guided domain adaptation for robust generalization of cancer drug response prediction from in 2021.
vitro to in vivo settings. Bioinformatics, 37(Supplement..1:176
Planchard. ST Popat, K KeTr, S Novello, EF Smit, Corinne Faivre-Finn, TS Mok, M Reck, PE Van Schil, MD) Hellmann, et al. Metastatic non-small cell lung cancer: Esmo clinical practice guidelines for diagnosis, treatment and follow-up. Annals of Oncology, 29:iv 1929i199-.v237. 2018
Tijana Randic, Stefano Magni, Demetra Philippidou, Christiane Margue, Kamil Grzyb, Jasmin Re- nate Preis, Joanna Patrycja Wroblewska, Petr V Nazarov, Michel Mittelbronn, Katrin BM Frauenknecht, et al. Single-cell transcriptomics of nras-mutated melanoma transitioning to drug resistance reveals p2rx7
indicator of early drug response. Cell Reports, 42(7), 2023.
Alexander Ratner, Stephen H Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher Ré. Snorkel: Rapid training data creation with weak supervision. In Proceedings of the VLDB en- dowment. International conference on very large data bases, volume 11, pp. 269. NIH Public Access, 2017.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
14

---

Published as a conference paper at ICLR 2025
Yuki Saito, Junji Koya, and Keisuke Kataoka. Multiple mutations within individual oncogenes. Cancer science, 112(2):483 -09))4849999,... 2021.
Maria Schwaederle, Barbara A Parker, Richard Schwab, Gregory A Daniels, David E Piccioni, Santosh Kesari, Teresa L Helsten, Lyudmila A Bazhenova, Julio Romero, Paul T Fanta, et al. Precision oncology: The uc san diego moores cancer center predict experience. Molecular cancer therapeutics, 15(4)7743-752, 2016.
Hossein Sharifi-Noghabi, Shuman Peng, Olga Zolotareva, Colin C Collins, and Martin Ester. Aitl: adversarial inductive transfer learning with input and output space adaptation for pharmacoge- 2020.
nomics. Bioinformatics, (666Supplement..1::380-----
Hossein Sharifi-Noghabi, Parsa Alamzadeh Harjandi, Olga Zolotareva, Colin C Collins, and Martin Ester. Out-of-distribution generalization from labelled and unlahelled gene expression data for drug response prediction. Nature Machine Intelligence, 3(11)962-972. 2021a.
Hossein Sharifi-Nohabbi, Soheil Taazhkii, Petr Smirnov, Casey Hon, Anthony Mam- moliti, Sisira Kadambat Nair, Arvind Singh Mer, Martin Ester, and Benjamin Haibe-Kains. Drug sensitivity prediction from cell line-based pharmacogenomies data: guidelines for developing machine learning models. Briefings in Bioinformatics, 22(6))bbab294, 2021b.
Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of big data, 6(1):1-48, 2019.
Kumar Shubham, Aishwarya Jayagopal, Syed Mohammed Danish, Prathosh AP, and Vaibhav Ra- jan. Wiser: Weak supervision and supervised representation learning to improve drug response prediction in cancer. arXiv preprint arXiv:2405. 04078, 2024.
Alona Sosinsky, John Ambrose, William Cross, Clare Turnbull, Shirley Henderson, Louise Jones, Angela Hamblin, Prabhu Arumugam, Georgia Chan, Daniel Chubb, et al. Insights for precision oncology from the integration of genomic and clinical data of 13,880 tumors from the 100,000 genomes cancer programme. Nature Medicine, 30(1):279-22) 2024.
Baochen Sun, Jiashi Feng, and Kate Saenko. Return of frustratingly easy domain adaptation. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.
Adam Wahida, Lars Buschhorn, Stefan Fröhling. Philipp J Jost, Andreas Schneeweiss, Peter Lichter, and Razelle Kurzrock. The coming decade in precision oncology: six riddles. Nature Reviews Cancer, 23(1):43-555 2023.
Kai Wang, Mingyao Li, and Hakon Hakonarson. Annovar: functional annotation of genetic variants from high-throughput sequencing data. Nucleic acids research, (8416))e164-e164, 2010.
Bo Wei, John Kang, Miho Kibukawa, Gladys Arreaza, Maureen Maguire, Lei Chen, Ping Qiu, Lixin Lang, Deepti Aurora-Gary Garo, Razvan Cristescu, et Evaluation of the trusight oncology 500 assay for routine clinical testing of tumor mutational burden and clinical utility for predicting response to pembrolizumab. The Journal of Molecular Diagnostics, 24(6):600-608, 2022.
John N Weinstein, Erie Collisson, Gordon B Mills, Kenna Shaw, Brad A Ozenberger, Kyle Ellrott, Ilya Shmulevich, Chris Sander, and Joshua M Stuart. The cancer genome atlas pan-cancer analysis project. Nature genetics,
113-1110,, 2013.
Wanjuan Yang, Jorge Soares, Patricia Greninger, Flena Edelman, Howard Lightfoot, Simon Forbes, Nidhi Bindal, Dave Beare, James A Smith, I Richard Thompson, et al. Genomies of drug sensitivity in cancer (gdsc): a resource for therapeutic biomarker discovery in cancer cells. Nucleic acids research, (1(D1):D955 D9611 2012.
Fupan Yao, Seyed Ali Madani Tonekaboni, Zhaleh Safikhani, Petr Smirnov, Nehme El-Hachem, Mark Freeman, Venkata Satya Kumar Manem, and Benjamin Haibe-Kains. Tissue specificity of in vitro drug sensitivity. Journal of the American Medical Informatics Association, 25(2): 158-166, 2018.
Yiyang Yu, Shivani Muthukumar, and Peter K Koo. Evoaug-tf tf extending evolution-inspire data augmentations for genomic deep learning to tensorflow. Bioinformatics, 40(3):btae092, 2024.
15

---

Published as a conference paper at ICLR 2025
Jia Zhai and Hui Liu. microenvironment impact on drug response. IEEE Journal of Biomedical and Health Informatics, 2024.
feature disentanglement for interpretable modeling of tumor
Jieyu Zhang, Cheng- Yu Hsieh, Yue Yu, Chao Zhang, and Alexander Ratner. A survey on program- matic weak supervision. arXiv preprint arX1v:2202.04333, 2022.
A APPENDIX
A.1 EXPERIMENT SETTINGS
A.1.1 DRUG SELECTION CRITERIA
The patient dataset we used had 56 drugs. For each of the 56 drugs in patients, we first consider those with least 20 labelled patient samples (He et al.) 2022) this reduced labelled data to 12 drugs. For each drug. we divided the samples into groups based on cancer type and data source. Each group with 20 samples was divided into 2:1 ratio in 3-fold label based stratified cross-validation. For some groups, no test samples were available. We excluded these to get 7 drugs. These drugs were used in the ablation studies in Table to report overall performance metrics. We removed drugs which had < 3 positive samples as it would cause issues in CV, where one fold may have test samples with only a single label this resulted in the five drugs shown in Table T
A.1.2 TRAIN-TEST SPLIT
RECIST labels in patients were initially coalesced into 2 groups - Complete and Partial Response label 1 (good response), Stable and Progressive Disease as label (bad response). The labelled patient samples obtained were grouped based on the drug, cancer type and source of dataset (TCGA, Moores, CBIO). Each group with 20 samples was divided into 3-fold cross validation train-test splits, stratified by label. Groups with < 20 samples were only used for training. The train and test labelled samples across all groups and folds were combined form 3 train-test folds respectively. Each of the 3 train folds were further divided in 90:10 ratio to obtain line data was also grouped in a similar fashion and divided into a single train-validation and test fold. The training and evaluation in all cases sample, drug pairs where the sample could be from either domain. We had a total of 156441 train, 17371 validation and 21589 test cell line, drug pairs. We also had 488/488/487 train, 53/54/56 validation and 1 14/1 13 test patient, drug pairs over the 3 folds. We run inference on test patient, drug pairs, and report the average AUROC and AUPRC metrics across 3 test folds.
vrainvvaidaation split. Cell
A.2 SENSITIVITY VOLUME PSEUDOLARELLED DATA
We examined the sensitivity of the overall model performance increasing the quantity of pseu- dolabelled data. We change the amount of pscudolabelled data by varying the upper and lower thresholds t, and tr. Increasing t and decreasing fu is equivalent adding more pseudolabelled samples. We varied & from 0.1 to 0.4, fu from 0.5 to 0.9, in increments of 0.1. In all cases, only a single parameter was changed while all others were left constant. Figure 4 indicates that lower value of f2 shows better performance. This may result in fewer non-abstained samples with label 0, and improve confidence the samples selected for the downstream DRP task. higher L, gen- eral improves performance with 0.8 yielding the best. If l,, is too low or 4 is too high, it may admit more low-confidence samples, Youg being closer to 0.5. If t, is too high, it may drastically reduce the number of positive labels available for downstream DRP training, also reducing performance. Table 3 indicates the number of pseudolabelled samples added in each case.
A.3 SENSITIVITY TO DIFFERENT AMOUNTS OF TRAINING DATA
We conducted two experiments to evaluate the effect of varying amounts of real and synthetic patient data.
16

---

Published as a conference paper at ICLR 2025
500
09
Reads: ibdl Craded
Figure 4: Sensitivity tests on value of pseudo label lower (left) and upper (right) thresholds.
Lower threshold val-
Fold pseudola-
Fold
pseudola-
Fold 2 pseudola-
belled responders /
belled responders
belled responders
non-responders
non-responders
non-responders
0.1
3830/60101
874/15668
241/7157
0.2
3830/192454
874/125098
241/81011
0.3
3830/355849
874/323572
241/274803
0.4
3830/481589
874/479348
241/462177
Upper threshold val- ues
Fold pseudola-
Fold pseudola-
Fold 2 pseudola-
belled responders /
belled responders
belled responders
non-responders
non-responders
non-responders
29599/60101
25932/15668
25554/7157
0.6
9568/60101
6336/15668
4023/7157
0.7
3830/60101
874/15668
241/7157
0.8
1578/60101 500/60101
27/15668
0/7157
0.9
0/15668
0/7157
Table Number of pseudolabelled samples used in sensitivity test of thresholds.
A.3.1 EFFECT OF VARYING AMOUNTS OF PSHUDOLABELLER DATA
We retain all the real train patient data and randomly sample 25%, 50%, 75% and 100% of the generated, confident pseudolabelled data, and use this in training the DRP model. 0% setting indi- cates augmented data in the DRP training. Results without any augmentation. Best AUROC is at 50% addition of pseudolabelled data, best AUPRC at 25% pseudolabelled data. Across 25-100% settings, the difference in performance is not statistically significant. For the case of 0%% vs any other level of augmentation, differences are statistically sig- nificant, indicating that adding pseudolabelled data improves performance. To answer the question of how much pscudolabelled is helpful we will need further studies possibly larger datasets.
shown the Table 4 0% does the worst,
A.3.2 EFFECT OF VARYING AMOUNTS OF REAL PATIENT DATA
We randomly sample 25%, 50%, 75% and 100% of real labelled patient data. In each case we sample twice the number of real samples from the pseudolabelled data. 100% setting thus refers to
% of pseudola- belled data
Average AUROC
Average AUPRC
Number of pscu- dolabelled patient data (fold 0)
Number of real labelled patient data (fold 0)
over 3 folds
over 3 folds
0%
0.5263 + 0,0195
0.5229 + 0.0249
488
25%
0.8584 = 0.0361
0.7838 = 0.0564
15983
488
50%
0.8613 + 0.0279
0.7796 + 0.0437
31966
488
75%
0.8577 + 0.0269
0.7677 + 0.0354
47948
488
100%
0.8409 + 0.0437
0.778 + 0.0255
63931
488
Table 4: Performance comparison for varying quantities of pseudolabelled data
17

---

Published as a conference paper at ICLR 2025
oeginel
arginall call limes X. oriiial potients X,
genersted sedients Xay
0.210
Seo
oeiginal pablents
0.008
Denoy
0.2
0.004
0.1
0.022
0.
10
0.000
PCA compenent2
1SRE component2
Figure 5: Kernel Density Estimation plot comparing the distribution of first PCA component (left) and first TSNE component (right) across original cell line, real patient and generated patient data
NA
Figure 6: Learning curves on (left to right) cross-validation folds, with orange line indicating train loss and blue indicating validation loss. Dotted lines indicate augmentation with Gaussian perturbation, solid lines indicate no augmentation, dashed lines indicate GANDALF augmentation.
3 times the size of real labelled patient data. general, as seen in Table as more real labelled data is added performance improves, generally expected.
A.4 HYPERPARAMETER SELECTION
For baseline models, we used the hyperparameter ranges defined in each of the papers. We did hyperparameter sweep over these ranges using Bayesian Optimization for maximum of 15 runs, to determine the best hyperparameters for our dataset. We did not tune epochs since we had early stopping all cases. Across methods, we focused on the last stage of DRP for tuning. For DruID, MTL learning rate range was [0.001, 0.05). RECIST prediction network dimensions for 1st hidden layer were tuned in 64, 32 and dimensions for second hidden layer in 16, 8. In PREDICT-AL, we tuned
tuned learning rate in the range [0.0001. 0.001]. batch size in 128, 64. In PANCDR. encoder bottleneck dimensions in 100, 128, 256, GCN dimensions in 100, 128, 256, learning rate in [0.0001, 0.001]. adversarial learning rate in [0.0001, 0.001), lambda in 1, 0.1, 0.01, batch size in 128, 256. In CODE-AE and WISER, we tuned dropout in 0, 1. In WISER we additionally tuned learning rate in the range [0.001, 0.1].
For GANDALF, we mainly focused on the hyperparameters in the supervised training stages, key being the lower and upper thresholds and learning rate parameters for the DRP and MTL models. We varied the lower threshold between 0.1 to 0.5 and upper threshold from 0.5 to 0.9, with increments done based on quantiles calculated from predicted probability of response after MTL training. This was done for each drug separately. The hidden layers from the VAE were set to 64 dimensions
% of real data (pseudolabelled
Average AUROC
Average AUPRC
Number of Number of real labelled patient data (fold 0)
over 3 folds
over 3 folds
pseudo labelled patient data (fold
data 2 x real
data)
0)
25%
0.5326 = 0,0152
0.5239 = 0.0222
244
122
50%
0.581 + 0.0216
0.5505 + 0.0274
488
244
75%
0.6888 + 0.0257
0.638 + 0.0348
732
366
100%
0.7086 + 0.0247
0.6533 + 0.0374
976
488
Table 5: Performance comparison for varying quantities of real patient data
18

---

Published as a conference paper at ICLR 2025
Table 6: Performance comparison across different ablation tests, where each test removes one com-
ponent from GANDALE. Best performing results are highlighted in bold.
AUROC (Mean Standard deviation)
Method
Cis
Flu
Gem
Pac
Tem
GANDALF
0.6343 + 0,0306
0.7309 + 0.0664
0.6188 + 0,0674
0.7728 + 0.1253
0.6451 = 0.0776
W/O MTL
0.3409 0.219
0.5333 + 0.075
0.5587 + 0.
0.2758 1 0,1461
0.7513 0.0805
W/O cross-
0.6061 + 0.0475
0.7309 1 0.0834
0.6188 1 0.0674
0.7728 1 0.1253
0.6152 + 1.14
attention
W/O trans-
0.3735 O. 0414
0.4143 + 0.1122
0.5718 + 0.0805
0.5625 + 0.3903
0,2106 0.0457
former
W/O VAE
Out of memory issues
W/O DDPM
0.4849 0.0909 0.6048 0.1185
0.6929 + 0.1189 0.6452 + 0.2304
0.5162 + 0.1247
0.5208 + 0,4161
0.3138 0.0647
psew-
0.6019 + 0.1891
0.6825 0.3345
0.5026 0.1647
dolabels
AUPRC (Mean
+ Stundard deviation)
Method
Cis
Flu
Gem 0.5874 = 0.175
Pac
Tem
GANDALF
0.9093 + 0.0355
0.8483 + 0.0933
0.9558 + 0,024
0.2535 = 0.1
W/O MTL
0.8101 1 0.0793
0.7345 + 0.1
0.5697 1 0.0628
0,7582 + 0,1012
0.3215 0.1623
W/O cross-
0.9183 + 0.0255
0.8483 + 0.0967
0.5873 + 0.1753
0.9558 + 0.024
0.2068 + 0.0585
atlention
W/O Irans-
0.8047 +: 0.0417
0.6400 + 0.1231
0.4760 + 0.0865
0.8478 + 0.1374
0.0993 + 0.0195
former
W/O VAE
Out of memory issues
W/O DDPM
0.8696 0.0098 0.919 + 0.035
0.8241 + 0.0741 0.8146 + 0.1368
0.4932 + 0.1867
0.8273 + 0,1636
0.1 0.0263
pseu-
0.5669 + 0.1321
0.9066 + 0.1301
0.2702 0.1045
dolabels
based on our GPU memory restrictions and batch size was 512. For the transformer we used 64 dimensional embeddings, 4 heads and 8 encoder layers. For the cell line VAE, we used 1024, 128 and 64 hidden units and for patient VAE we used 512, 128 and 64 hidden units. Both VAEs used tanh activation. The DDPM uses linear layers with dimensions as the VAE representation size, and uses dropout and ReLU. 33 was set based on the cosine beta scheduling in (Nichol & Dhariwal 2021]. The MTL network uses linear layers each in embedding drugs, predicting RECIST and predicting AUDRC, with ReLU activation. Max epochs were set to 500 for pretraining, 100 for the MTL and DRP training. Early stopping was done using patient validation set pearson correlation as in (Sharifi-oohabii et al. 202
A.5 ADDITIONAL ABLATION EXPERIMENTS
To understand the contribution of each individual component, we added additional ablation studies where each test removes just one component of the architecture. We also performed drug specific tuning in each case. Table o shows the results of each ablation test. Apart from the ablation tests in Table we also added three more tests W/O VAF, W/O DDPM and W/O pseudolabels. In W/O VAF, we attempt to directly feed the output of the transformer encoder layer to the domain-specific DDPMs, bypassing the VAEs. In W/O DDPM, VAES. The data augmentation is done by passing the cell lines through the cell line VAE encoder and the patient VAE decoder. The pseudolabelling and downstream DRP training remains the same as GANDALF in both cases. In W/O pseudolabels, to remove the influence of pseudolabelled data, directly use the MTL part of the network (after stage 3) run inference on the test patient data. The use of transformers and MTL appear to contribute the most to the model performance. The use of pseudolabelled data also helps improve average performance in most cases.
replace the DDPMs with two domain-specific
A.6 COMPARISON OF DISTRIBUTION
We examine the distribution of the Xang with respect to the real distributions of X" and X" We expect X Kang to be closer to the patient distribution than the original cell lines, while also retaining information from the original cell line data. Each dataset is further subjected to principal component analysis (principal components (PCs) from Xx) to obtain lower dimensional representations for eas- ier visualization. Figure] 7 (top. right) shows that original cell line data had lower variance compared to the real patient data (Figure 7 top left). However, the generated patient data (Figure 7 top mid- dle) is closer to the real patient data in terms of the variance of data points. This indicates that the generated data captures sppceic- heterogeneity. A similar trend is seen in the density plots of
19

---

Published as a conference paper at ICLR 2025
Original Patient date Kp
Genereted patient data Xauz
Original cell line data X
16
10
PCA companend
PCA companentz
PCA
componenti
compancnti.
componenti
Original patient dett X2
Generated patient data Kag
Original cell line data X.'
75
"SME companent:
40
"SME companent
"SME companent
50-
20
25
20-
25
-75
-75-
component
compunent
compunent
Figure 7: Comparison of distribution (left to right) across real and generated data, using PCA (top) and TSNE (bottom) methods.
TCGA-LGG
TCGA-GBM
TCGA-HBSC
TCGA-CECC
TCGA-COM COAD
TCGA-UIC
TCGA-MESO
TCGA-THCA
TSME component
TCGA-STAD
UCGA
TCGATOCT
TCGA-INC
ICAA-PRAD
TCGA BLCA
TEGA-USC
TCGA-PAAD
TCGA-ICH
ICGA-SARC
TCGA-OV
TCGA-ESCA ESCA
TCGA
-15
-15
-10
TSNE component
Figure 8: TSNE plots of first two components of the patient data in the representation space, color coded based on TCGA cancer types.
the first PC in Appendix Figure Quantitatively, we also examine the Golmogorov-Smimet (KS) test between the PCs of the 3 distributions. KS distance statistic between generated patient data and real patient data over 3 folds is 0.0694 + 0.0071, while the same between original cell line data and patients is 0.2524 = 0.0022. The PCs of the augmented data is closer to that of the real patient data, when compared to the distance between the PCs of the original cell line and patient data. This indicates that the augmented data starts resembling the patient data while retaining information from the original cell line data.
A.7 CHECKING FOR BATCH EFFECTS IN THE REPRESENTATION SPACE
Our patient data comes from three different sources TCGA, CBIO and Moore's. To ensure that these representations do not inadvertently capture batch effects, we perform a TSNE based visual- ization, where the patient latent representations TCGA). For Moore's 's and CBIO datasets, we identified the corresponding category in TCGA. Fig- ure shows the TSNE plot for the first two components, after embedding the patient data into the representation space. The lack of well defined boundaries across cancer types (indicated by various colors) suggest that there is no batch effect across the mutation datasets.
colored based on the cancer type (as coded in
20

---

Published as a conference paper at ICLR 2025
Cancer type
AUROC over 3 folds
AUPRC over folds
TCGA-BRCA
0.8947 + 0.0368
0.8720 + 0.0712
TCGA-CESC
0.3197 + 0.1403
0.7704 + 0.0716
TCGA-HNSC
0.7652 + 0.137
0.9788 + 0.0137
TCGA-STAD
0.7119 + 0.1318
0.8253 + 0.1222
TCGA-PAAD
0.6620 0.0765
0.6239 + 0.0516
TCGA-LGG
0,4309 + 0.0563
0,1406 + 0.0061
Table 7: Comparison of performance across various cancer types.
A.8 PERFORMANCE ACROSS CANCER TYPES
During the train-test split, we split the data based cancer type and drug. Then we divided each group into 2:1 ratio if more than 20 samples were present per group. The train data thus con- tained all available cancer types. The evaluation BRCA', "TCGA-CESC, "TCGA-HNSC', "TCGA-STAD", 'TCGA-PAAD"; "TCGA-LGG" Perfor- mance per cancer type from existing test splits caleulated the metries the available test splits by grouping based on cancer type. Table shows the results.
limited set of cancer types 'TCGA-
21


Reviews:
[{"review_id": "TUSsCeDFTz", "reviewer": "Reviewer_RAe2", "summary": "The authors propose GANDALF, a framework that leverages attention-based generative models to augment patient genomic data, addressing limitations in data availability and patient-specific characteristics within existing models. GANDALF demonstrates superior performance over state-of-the-art models in predicting drug responses for cancer treatments.", "questions": "See weaknesses above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The studied problem is important and practical. \n\n- The idea of incorporating patient-specific characteristics with drug response prediction models is novel. \n\n- The code is provided for reproducibility.", "weaknesses": "- The comparative performance analysis in Table 1 focuses on only 5 drugs, which is a small subset of the 56 drugs mentioned in the dataset. The paper does not justify why these specific drugs were chosen or demonstrate that the performance advantages generalize across the broader drug set. This limited evaluation makes it difficult to assess the method's robustness across different drug classes.  The paper needs to demonstrate results across a more diverse set of drug classes and provide statistical tests showing whether performance advantages hold across the full drug set. \n\n- The authors state \"we do drug-specific model tuning in GANDALF, by only augmenting with sample, drug pairs for the drug considered.\" This approach may cause extensive computational burden, as it requires fine-tuning for each new drug, potentially limiting its practical deployment.\n\n- The paper does not address how well GANDALF generalizes to novel drugs in out-of-domain scenarios—a critical consideration given that new drugs are regularly introduced into clinical practice.\n\n- Though the paper deals with the challenge of limited patient data, it doesn't systematically analyze how performance varies with different amounts of training data. There's no exploration of learning curves showing how model performance changes with increasing patient samples, or minimum data requirements for reliable predictions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pJHwklhdSG", "reviewer": "Reviewer_64hx", "summary": "This study presents GANDALF, a generative, semi-supervised framework addressing the challenge of limited labeled data in sparse patient genomic datasets by leveraging cell line data and augmenting patient samples with their clinical information. However, key concerns include the need for clearer model justification, additional technical details, and further clarity in experimental results.", "questions": "- The manuscript introduces a model with three distinct encoders, yet the rationale behind this multi-encoder structure lacks clarity and a convincing explanation. For example, the authors state, \"to ease training, we reduce the dimensionality … using variational autoencoders (VAEs),” but it is unclear why a VAE is required rather than a simpler embedding without a decoder and what specific role the VAE’s decoder serves. While VAEs are known for capturing latent features and supporting generative tasks, considering that DDPM is an autoencoder-structured generative model, the need for a secondary, nested autoencoder is not well justified. This makes the model more complicated. The authors are needed to address the following points; 1. why a simpler embedding without a decoder is insufficient in this context; 2. what the specific function of the VAE's decoder plays in the model's architecture; 3. how the combination of VAEs and DDPM provides benefits that DDPM alone does not.\n\n- How is the time step $t$ defined? Does it mean the time after treatment? This should be clarified in the problem formulation section. Additionally, how is $\\beta_t$ set?\n\n- How are the ground truth noise distributions $e_c, e_p$ defined, and how does the DDPM decoder remove domain-specific noise?\n\n- In the results section, only five drugs are reported out of about 100 test drugs. The criteria for selecting these specific drugs should be clarified, especially given that all drugs were included in the ablation study.\n\n- The study should assess the model’s ability to accurately label mutation-drug pairs when the drug in question is absent from the labeled training dataset (X_p, y_p, d_p). Were these five drugs excluded from the training and validation sets? If they were, a comparative analysis of model performance on drugs within and outside the training set would enhance the understanding of the model’s generalizability; if not, this experiment does not adequately validate the model’s intended benefits.\n\n- There are typos in Equation (6), specifically where L_MSE is indicated as L_R and L_KLDV as L_KLD?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- This study introduces GANDALF, a novel generative and semi-supervised data augmentation framework designed to overcome sparse labeled patient genomic data. \n- GANDALF leverages labeled samples from the cell line to augment patient data with critical information.\n- By incorporating attention mechanisms and generating augmented samples, GANDALF significantly enhances classifier performance in predicting patient drug response.", "weaknesses": "- The manuscript proposes a model with three distinct encoders, but the reasoning behind selecting this multi-encoder structure lacks clarity and strong justification.\n- Some technical details are missing. \n- Experimental results are not comprehensive.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "REQgnwhWwI", "reviewer": "Reviewer_w5ow", "summary": "Edit: Updated my score to 8 after the authors addressed my most major concerns\n\nThis paper proposes a method, GANDALF, for improving domain transfer for machine learning models, explicitly focusing on the cancer domain where we want to train chemo response models on cell line data, but apply them to patients. GANDALF works by creating artificial examples that are more similar to the target domain that can then be used for model training. The authors find that this data augmentation strategy significantly improves the performance of chemo response models on patients.", "questions": "Can you explain in more detail the differences between table 2 and 1 and why you chose a vastly different format between them?\n\nHow were standard deviations calculated?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Strengths:\n- Very well written paper, I really enjoy the step by step breakdown.\n- Very strong empirical improvement over prior art\n- Novel combination of techniques, incorporating a wide variety of tricks to improve performance\n- Very significant topic, as domain adaptation is critical for many important problems where in-domain data is hard to obtain", "weaknesses": "Major:\n- This paper is sorta a mix of two contributions, a very good “pseudo-labeler” MTL model and the data augmentation. However, it’s unclear which of those two contributions is important as it’s missing a simple, but important baseline: simply using the “pseudo-labeler” MTL model as a final classifier.\n\n- I don't see any discussion of hyperparmeters for both models and baselines. How were they selected and how did you ensure that equal hyperparameter search time was dedicated to your model and baselines?\n\n- It’s very hard to compare the ablation results in table 2 to table 1. Can you create an alternative table 2 that is side-by-side comparable with table 1?\n\nMinor:\n- There are some typos (for example the first row in table 2)\n- I think some of the analysis in 3.4 is incorrect. For example, looking at only the top 2 principal components in high dimensional data to compare variance does not seem correct as you might be ignoring a ton of the signal. However, section 3.4 doesn’t matter for the central hypothesis of the paper so it’s not that important. I would actually just remove section 3.4", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tTXJZoGufE", "reviewer": "Reviewer_1XQK", "summary": "Authors proposed GANDALF a method for data augmentation that takes labeled cell line and patients mutation data as input and makes drug response prediction via generating patient-driven samples in a 5 step process:\n- pretraining diffusion models on cell lines and patients\n- generating augmented samples \n- training a multitask predictor using labeled cell lines and patients \n- making predictions for augmented samples and selecting confident ones \n- training a final predictor on confident augmented samples and patients\n\nGANDALF was compared with state-of-the-art of drug response prediction across 5 chemotherapy drugs in terms of AUROC and AUPR and was able to outperform the majority of them.", "questions": "- There is no information regarding the applicability domain of GANDAL including what types of cancers was this method train/evaluated on? What was the performance per cancer type? similarly for cell lines, was the data from only solid tumours or both solid and non-solid tumours? \n\n- While somatic mutations can be predictive of drug response, other available data types such as RNA-Seq make more accurate predictions. To my knowledge, CCLE/GDSCv2 and TCGA/CbioPortal also have gene expression data, if the ultimate goal is to assist clinicians with choice of therapy why did you choose not to use other data types? \n\n- Why is GANDALF only applicable to chemotherapy drugs? Mutation data is also predictive of targeted therapeutics I can imagine it could be due to lack of labeled data for targeted drugs if so, isn’t this a huge disadvantage for the proposed method that requires labeled patient data as input? For targeted drugs like Erlotinib (Byers et al. An Epithelial-Mesenchymal Transition Gene Signature Predicts Resistance to EGFR and PI3K Inhibitors and Identifies Axl as a Therapeutic Target for Overcoming EGFR Inhibitor Resistance 2013) there are many labeled cell lines and smaller labeled patient data. Do you have any sense on the minimum required number of cell lines and patient samples for your method? I believe an experiment of percentage of labeled samples would be highly beneficial.  \n\n- The reported performance for WISER and Code-AE in this work are substantially lower than those reported in the original papers for similar drugs and datasets. It seems WISER and Code-AE had a similar setup so what’s the difference between your setup and their’s? \n\nI'd be happy to revise my score if authors answer all questions/concerns thoroughly particularly:\n- potential data issue\n- applicability domain\n- reproducibility \n- missing ablations", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The introduction of biological aspects was gentle and informative enough for those not familiar with the motivating application of the work. \n- While the separate components of GANDALF are not novel per se, components are linked together in a novel way for data augmentation - based on patient data for precision oncology. \n- The experimental results show improvements compared to baselines across multiple drugs", "weaknesses": "- The input patient data is itself coming from 3 different resources i.e, P(Xp1) != P(Xp2)!=P(Xp3) similarly, the cell line data is also coming from two resources (input from CCLE, output from GDSC) but there is no indication of that in neither the assumptions nor the setup or the method. What makes GANDALF capable of handling shifts like that?\n\n- Although authors have provided the source code, critical information is missing with respect to reproducibility of this work. For example, it is unclear if the reported numbers are from how many independent runs or hyper-parameters and the procedure for tuning them are not reported in the paper for any of the components e.g., DDPA, transformers, VAE, MTL, etc. \n\n- Citations of the utilized data are inaccurate and incomplete. For example, for CCLE and GDSC the seminal work like \n“Barretina et al., The Cancer Cell Line Encyclopedia enables predictive modelling of anticancer drug sensitivity 2012” or “Iorio et al. A Landscape of Pharmacogenomic Interactions in Cancer 2016” are not cited. Moreover, Although authors claimed that they’ve used GDSCv2, the citation is for 2012 which is more correlated with GDSCv1 timeline. Is this a citation error or GDSCv1 was used? \n\n- Authors used genomic data from CCLE and response data from GDSC, there is no justification for why the response profiles of these datasets are interchangeable. How many cell lines were in common between CCLE and GDSC, was the drug screening assay comparable between the two? this is important because GDSCv1 used Syto60 assay while CCLE and GDSCv2 used CellTiter-Glo (going back to the previous point on which GDSC was used).  \n\n- There are parts of the paper that is challenging to follow for example, “We had a total of 156441 train, 17371 validation and 21589 test cell line, drug pairs. We also had 488/488/487 train, 53/54/56 validation and 115/114/113 test patient, drug pairs over the 3 folds.” Is unclear to me. Is this a reference to different patient data sources? \n\n- W/o MTL in ablation seems misleading as it only removes the cell line part but the proper way of doing this is to replace it with a drug specific single task model train on both cell lines and patients. \n\n- Another missing ablation is w/o diffusion what happens if you apply VAE directly to the input data?\n\n- GANDALF learning curve is missing in Figure 7", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x7nI7Jb8QQ", "reviewer": "Reviewer_NotN", "summary": "The study shows an AI4Science application. It majorly focuses on using related cell line data and patients' clinical data to do drug and dosage response. The major contributions of this paper are: 1. proposing a composite framework (diffusion $+$ transformer) architecture to use cell line data to enrich limited patient clinical data. 2. the proposed framework also include a prediction module for the drug response prediction task and it beats SOTA models on the task.", "questions": "As I am not an expert in this application, I want to left the decision to the other reviewers and chairs that whether it deserves a highlight. But I think it is a very interesting application of diffusion models to this two modalities. \nGood work!", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The study introduces a complete framework to do the DRP task. The framework uses a smart way to use the cell-line data to augment the patient data in order to improve model's performance on the target task.\n- I like the idea that projects the patient embedding and cell-line data embedding data on the shared latent space and uses the cross attention strategy to keep the shared information. \n- The step 3 is also a smart move that addresss the limited drug issue in patients' clinical dataset.\n- I can see from Table 1 that the proposed framework achieves general improvement on the datasets compared with baselines. \n- The authors provided comprehensive ablation studies. Table 2 demonstrates the importance of each component of the framework.", "weaknesses": "- In figure 4, the aim is to show distribution similarity between generated patient data. But the PCA visualization is a linear projection method, and I cannot tell any difference between the distributions. Can you use some non-linear methods UMAP/t-SNE to show the distributions, and mark the labels using different colors? It would be better to see the distribution characteristics and differences.\n- Why you put related works at last? It is hard for a reader not in this field to follow the work. It would be better put the section right after the introduction. And unsupervised models has been introduced well in the paper, can you elaborate a little bit more on the brief definitions or examples of inductive and transductive approaches in the context of drug response prediction, and how their method differs from yours.\n- One key step is data augmentation. The author does provide the sample size for train/test/val in the appendix, but I didn't find the details about the sample size, i.e. how many psedo-patient data were added before training the final model. I think it is a key information for your experiments. And, I like your experiments in the appendix A.2, but the quantity there confused me. It would be better to provide the sample size directly rather than the thresholds.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)