Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
Authors: Jason Qin
Hans-Hermann Wessels
Carlos Fernandez-Granda
Yuhan Hao
Venue: ICML 2025 poster (2025)
Keywords: active learning
AI and Lab Loop
combination therapy
combinatorial perturbation
Primary area: applications->health_medicine

Abstract: Combinatorial CRISPR screening enables large-scale identification of synergistic gene pairs for combination therapies, but exhaustive experimentation is infeasible. We introduce NAIAD, an active learning framework that efficiently discovers optimal gene pairs by leveraging single-gene perturbation effects and adaptive gene embeddings that scale with the training data size, mitigating overfitting in small-sample learning while capturing complex gene interactions as more data is collected. Evaluated on four CRISPR datasets with over 350,000 interactions, NAIAD trained on small datasets outperforms existing models by up to 40\%. Its recommendation system prioritizes gene pairs with maximum predicted effects, accelerating discovery with fewer experiments. We also extend NAIAD to optimal drug combination identification among 2,000 candidates. Overall, NAIAD enhances combinatorial perturbation design and drives advances in genomics research and therapeutic development in combination therapy. Our code is publicly available at: https://github.com/NeptuneBio/NAIAD

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
Jason Qin' 1 Hans-Hermann Wessels Carlos Fernandez-Granda" 23 Yuhan Hao
Abstract
single-gene targeting (Nature Medicine, 2017; Hwangbo et al., 2023). One of the most notable successes in transi- tioning cellular phenotype is the discovery of the Yamanaka factors-a specific combination of four transcription fac- tors capable of reprogramming differentiated cells back to pluripotent state (Takahashi Yamanaka, 2006). This groundbreaking achievement demonstrates the significant potential within the combinatorial perturbation space to en- gineer cellular phenotypes.
Combinatorial CRISPR screening enables large- scale identification of synergistic gene pairs for combination therapies. but exhaustive experimen- tation is infeasible. We introduce NAIAD', an ac- tive learning framework that efficiently discovers optimal gene pairs by leveraging single-gene per- turbation effects and adaptive gene embeddings that scale with the training data size, mitigating overfitting in small- -aalle learning while cap- turing complex gene interactions more data is collected. Evaluated four CRISPR datasets with over 350,000 interactions, NAIAD trained on small datasets outperforms existing models by up to 40%. Its recommendation system priori- tizes gene pairs with maximum predicted effects, accelerating discovery with fewer experiments. We also extend NAIAD to optimal drug com- bination identification among 2,000 candidates. Overall, NAIAD enhances combinatorial pertur- bation design and drives advances in genomics research and therapeutic development in combi- nation therapy. Our code is publicly available at: hutps//gtthabbcoom/eeuueeBBBoo
The critical question now is how to systematically iden- tify additional effective combinatorial perturbations that can transform cells to achieve desired phenotypes. Comprehen- sively exploring this huge space presents a mathematical challenge due to the exponential growth of possible combi- nations. With approximately 20,000 protein-coding genes in the human genome, the total number of two-gene combi- nations approaches 200 million, and for four-gent combina- tions exceeds 6 quadrillion (1025). Experimentally testing all possible combinations is infeasible. Therefore, develop- ing computational models that can predict the most effective gene combinations essential for the efficient identification of the most effective combinatorial perturbations. Notably, active learning frameworks (Eisenstein, 2020), such the Al + Experiment Loop (Rood et al., 2024), have offered promising solutions by enabling efficient exploration of this space.
1. Introduction
We frame the discovery of optimal gene or drug combina- tions as machine learning problem of active search over nigh-dimensional combinatorial space, where evaluating each combination (via experiment) is costly. Our method trains a neural surrogate model that predicts the effects of unseen perturbation pairs by combining overparameterized encodings of single-gene outcomes with a learned gene em- bedding space that models interaction effects. The surrogate guides new experiment selection via acquisition strategies inspired by Bayesian optimization, with the ability to lever- age both exploitation and exploration.
Targeting multiple genes through drug combinations or polypharmacology offers transformative therapeutic ap- proach for developing effective treatments across diverse medical fields, including oncology (Al-Lazikani et al., 2012; Mokhtari et al., 2017), infectious diseases (Hammond et al., 2022; Shyr et al., 2021), and metabolic disorders (Samms et al., 2020; Jastreboff et al., 2023). Combinatorial gene per- turbations can yield additive or synergistic effects, enhanc- ing therapeutic outcomes beyond what is achievable with
'Neptune Bio, New York, NY, USA *Center for Data Science, New York University, New York, NY, USA *Courant Institute of Mathematical Sciences, New York University, New York, NY, USA. Correspondence Yuhan Hao yyuuan@@nptuun.bll
While this work is motivated by biological discovery, similar challenges arise across machine learning do- mains-including data augmentation policy search in vision (Cubuk et al., 2019), cold-start item selection in recom- mender systems (De Pessemier et al., 2021), and sample- efficient policy learning in robotics (Anwar al., 2025)-2220000 all
Proceedings of the ngnd International Conference on Machine I Lernnngg Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Named after the innermost satellite the planet Neptune

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
of which involve large discrete spaces, costly evaluations, and the need for adaptive modeling and decision making. In such domains, discrete components (e.g., transformations, items, actions) play a role analogous to gene or drug per- turbations in our framework: each is embedded in a latent space, and combinations of these embeddings are used to represent and evaluate complex configurations. Our frame- work shows how these components can be actively selected via a data-adaptive surrogate to enable efficient, scalable discovery.
turbation data. Variational Autoencoders (VAEs) (Kingma & Welling, 2014) have been employed model genetic and chemical combinatorial perturbations by simultaneously learning embeddings of single perturbations and capturing non-linear interactions in methods such as CPA (Lotfol- lahi et al., 2023), Com&VAE (Geiger-Schuller et al., 2023), SVAE+ (Lopez et al., 2023), and SAMS- VAE (Bereket & Karaletsos. 2023). These approaches facilitate the model- ing of complex relationships between genes compounds within the latent space of embeddings. Methods such as sVAF+ (Lopez et al., 2023) and SAMS-VAE, (Bereket & Karaletsos, 2023) have been developed to model sparsity in latent variable intervention effects. By disentangling the perturbation-related sparse latent space, these models ef- fectively identify critical features and interactions within high-dimensional biological data. SALT&PEPER imple- ments method to separately learn linear and non-linear effects of gene perturbations (Gaudelet et al., 2024). By using gene-embedding- -basdd autoencoder models, they ef- fectively decompose the interactions, enabling more inter- pretable models of gene effects. Recently, several single-cell foundation models, such as scGPT (Cui et al., 2024) and scFoundation (Hao et al., 2024), trained on all publicly available observational data, have demonstrated their ability to predict cellular responses following perturbations after model fine tuning. Additionally, GEARS leverages graph neural networks (GNNs) (Kipf& & Welling, 2017) to incor- porate prior biological knowledge into the network architec- ture (Roohani et al., 2023). GNNs facilitate the inference of gene-gene interactions by leveraging known pathways and interaction networks, thus enhancing the prediction of combinatorial effects.
In this work, we introduce NAIAD, an active learning frame- work for identifying the most effective gene or drug combi- nations. We initially train a model on a small dataset from experiments, which enables it to predict unseen combina- torial perturbation effects across the entire combinatorial space. These predictions guide the design of subsequent CRISPR screening libraries for targeted experiments, thus allowing us to iteratively refine the model and converge on optimal combinations (Figure IA). We focus on modeling the effects of 2-gene combinations, combining knowledge of single-gene effects with predictions of gene-gene interac- tions (Figure IB). Our key contributions are:
(1) A novel combinatorial perturbation model which incor- porates adaptive gene embeddings that scale with the train- ing data size, along with an overparametrized representation of single-gene perturbation effects.
(2) Maximum Predicted Effect (MPE)-based recommenda- tion system that suggests gene combinations for subsequent CRISPR library design, facilitating the discovery of syner-
gistic and effective gene combinations.
(3) An Al + Lab active learning framework that effectively identifies optimal gene combinations, significantly reducing the number of experimental iterations needed to achieve robust results.
However, most of these methods primarily focus on pre- dicting post-perturbation gene expression profiles and do not extend to the prediction of phenotypic outcomes. Also, the number of gene combinations from single-cell combi- natorial perturbations dataset is very limited (100-200 gene combinations), so it is difficult to evaluate if those deep learning models are generalizable in the entire combinato- rial perturbation space and can outperform linear models (Ahlmann- -ltzee et al., 2024).
2. Related Work
CRISPR combinatorial perturbation technologies can be broadly classified into two main categories (Norman et al., 2019): single-cell combinatorial perturbation and bulk com- binatorial perturbation. Single-cell combinatorial pertur- bation measures the entire transcriptome, capturing com- prehensive gene expression changes in individual cells, but with a limited number of gene combinations. In contrast, bulk combinatorial perturbation focuses on measuring sin- gle phenotype, enabling the investigation of a much broader range of gene combinations.
Some existing methods (i.e. GEARS (Roohani et al., 2023) and CPA (Lotfollahi et al., 2023)), are capable of predict- ing combinatorial perturbations from single-cell transcrip- tomic profiles, as well as from single measurements derived from bulk screens (i.e. cell viability). However, these ap- proaches generally assume the availability of sufficient data that would allow for a substantial portion to be used for training. In practice, we are often limited by the number of training samples due to the exponential growth of possible combinations in combinatorial perturbation data. This lim- itation necessitates having methods that can perform well with minimal data. Active learning frameworks offer solu-
Predicting combinatorial perturbations has been a significant certain gene combinations. Various machine learning approaches have been proposed to address this problem using single-cell combinatorial per-
challenge due to non- -ineeriyyy of
2

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
tion by optimizing model performance while using the least amount of training data possible. RECOVER utilizes active learning framework that iteratively selects the most promising drug combinations for testing through the Al + Lap loop (Bertin et al., 2023). RECOVER applies a bilinear operator to create permutation-invarian representations that integrate multiple single perturbation effects, learning the non-linear components of drug combinations. It leverages ensemble models to estimate uncertainty in the predictions through deep ensembles (Lakshminarayanan et al., 2017), guiding the selection of experiments for subsequent rounds based on both predicted effects and associated uncertainties.
action contributions between i and j. where applies permutation-invariant function capture the interactions of genes and j tthrough their embeddings X
By explicitly incorporating single-gene effects as input pa- rameters, we condition combinatorial predictions on the known single-gene effects, which substantially enhances the model's performance. The parameter W1 acts as an over-parame peeepaaramteri encoder, projecting the data from & di- mensions to a much higher-dimensiona. space R where >>>>> k. The projection of this data into a high-dimensional (though finite) space allows us to capture intricate patterns that are not discernible in the original low-dimensional space.
3. Methods
To model genetic interactions, we sum the gene embeddings of the perturbed gene combinations in the latent space to obtain single combined embedding. This summation is permutation-invarian operation, and thus independent of the order of the multiple perturbations. The combined embed- ding is then passed through an encoder that compresses it into a singular interaction value. This compression reduces the dimensionality of the data while retaining the essential information needed predict nonlinear interaction of genes. Finally, the compressed representations from both the over- parameterized single-gene effect and combined-gent -ene em- beddings are used to predict the phenotype. This also allows our model to learn higher-order interactions among gene combination perturbations, leading to improved predictive performance.
In this section, we present the architecture of our model for predicting the effects of combinatorial perturbations, along with our Al + Lab active learning framework (Figure 1A). Our NAIAD framework addresses two key objectives: (1) achieving predictive accuracy with limited training data from the initial experimental round; and (2) implementing a recommendation strategy to select additional gene pairs that maximize information gain, thereby accelerating conver- gence with fewer AI experimental iterations. Ultimately, our goal is to optimize the use of limited experimental re- sources, reducing the need for exhaustive testing of all pos- sible combinations and efficiently identifying effective gene combinations that drive cells toward desired phenotypes.
3.1. Model Design
We can interpret the model follows: the overparameter- ized single-gene perturbation effects tell the model how any pair of genes additively would impact the cellular pheno- type, and relationships between gene embeddings predict synergistic or buffering interactions between gene pairs.
Let X'ene E kRp be the learnable gene embedding matrix, where k is the number of unique genes perturbed within the dataset and p is the dimension of the gene embeddings, which is adapted to the number of training samples. Let Y; be scalar value representing the effects of single-gene (i) perturbation. In this work, we focus on the case of per- turbing two genes simultaneously, so the value Yi+j denotes the combined effect of the two-gene (i + j) perturbation. The target variable Y is a scalar that can represent various biological outcomes, such as cell fitness, marker enrichment levels, or projected gene signatures derived from single-cell transcriptomic data.
Additionally, while we focus on 2-gene perturbations in this study, our model can naturally be extended for sets of q >> simultaneously perturbed genes. For set involving S = (11, 'g) perturbations, we can model the effect of jointly perturbing the genes in S as:
Our model is formulated as:
Ys = & ((V,,, Yig.....
"f.....,Y,llW,
.Y,lWi) AT
Y1+, = ((Y, + f(o(W2X dl ( DA!
+ Σ Φ (W2x?
A7
(2)
HES
where ø is an activation function (ReLU or GeLU), and W, A are learnable matrix parameters. X.7em and X are the row-vectors for gene and j in X (iY, AAT models the over- wwerpparaeteriiee single-gene effects, and models the genetic inter-
where we use summation as the commutation-nnvariti func- tion that combines the embeddings of individual perturbed genes.
( "x"") (W2xx )) ),
3

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
(A)
(B)
Cumulative training data
Active learner model
X, Gene i embedding
X, Gene embedding
Training
Inforence
permutation invariant
Messlred Unmeasured
Predict all unseen conditions
Active Learning Experiments
Generic interaction term
Outpul: Y., Gene jeffect
Experiments
Recommendation
effects term
Combinatorial perturbation screening
In silico screening Identify the strongest pairs
W.
Y, Gene effect
Y, Gene effect
Perturb multiple genes
Figure 1. Illustration of active learning framework in CRISPR combinatorial perturbation (A) and our NAIAD model architecture with single single-gene effects and adaptive gene embedding modules (B).
overparameterized
3.2. Small Sample Learning as an Important Initial Step for Active Learning
ever, we noted that the experimental uncertainty estimated from the ensemble method more stable. Thus, in this work, we adopted the variance of ensemble predictions as the ensemble- based uncertainty estimator.
Unlike most existing models, we propose modeling gene embeddings within a latent space with varying dimensional- ity tailored to the training-date size. Training large models on small datasets can lead to overfitting. To mitigate this, we initialize the dimension p of the latent space to small value at the beginning of training. As active learning itera- tively incorporates new data, the size of the training dataset continually increases, and correspondingly increase p following predetermined schedule based on the average number of times each gene is seen in the dataset. As the training dataset expands, the increased dimensionality of the latent space enables the model to capture more complex patterns and interactions among genes without overfitting to noisy experimental measurements. By controlling the model complexity based the available data, this adapt- ability allows the gene embeddings effectively leverage both small and large training datasets, optimizing model performance across different data sizes.
In addition to sampling gene pairs with high uncertainty, we incorporate mexiimum-preiicte--fffee based sampling strategies to diversify our experimental se- lection. The MPE sampling focuses on gene pairs with strong predicted effects, identifying combinations that may yield substantial biological insights or therapeutic benefits. Additionally, to balance the exploitation of known MPE areas with the exploration of uncertain regions, we also combined ensemble prediction uncertainty with residual- based sampling. Residual-baas -ased sampling targets areas where the model's predictions deviate most strongly from linear model baseline, allowing exploration of complex interactions that the model has not yet captured and help- ing to uncover gene interactions that might be missed by linear models. Combining residual-base basedd sampling with un- certainty estimation is equivalent to the Upper Confidence Bound (UCB) sampling method used in RECOVER.
(MPE) and residual-
3.3. Recommendation System of NATAD
4. Experiments
A critical component of our active learning framework is the recommendation system for selecting gene combinations in subsequent experiment rounds to acquire new data (Figure IA). We evaluated an ensemble-based uncertainty estima- tion, where multiple models with different initializations are aggregated estimate prediction uncertainty through mea- sures like variance and entropy (Lakshminarayanann et al., 2017). This ensemble approach not only improves pre- dictive performance and enhances interpretability, but also quantifies epistemic uncertainty arising from limited data. We investigated uncertainty calculation by maximizing the likelihood P(Y (ū, o) under the assumption of conditional Gaussian distribution (Lahlou et al., 2023) as well. How-
4.1. Datasets
We evaluate our models on cell-viabiliyy measurements across two cell types from four bulk combinatorial CRISPR perturbation screening datasets and one drug combination
screening dataset (Norman et al., 2019; Simpson et al., 2023; Horlbeck al., 2018; Zheng et al., 2021; Bertin et al., 2023). We treat each gene combination or drug combination as one sample. Each dataset contains symmetric, pair-wise mea- surements for its gene or drug combinations, allowing to comprehensively compare model predictions with ground truth measurements. Detailed descriptions of these datasets

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
are provided in Appendix A.
Lincar Model
4.2. Downsampling Experiments
-4.-
NAJAD
Overparameteriwwe Model
MLP Modal
We evaluate model performance on different amounts of training data. Since our datasets vary in number of features and overall size, we split the data differently depending the experiment and dataset used. In our downsampling ex- periments in Section 5.2, we used [100, 200, 350, 500, 750, 1000, 1250, and 1500] samples during training for the Nor- man et al. (2019) dataset (6,328 combinations), and [100, 500, 1000, 2000, 3000, 4000, 5000, 6000] samples for train- ing on the Simpson al. (2023) (147,658 combinations) and Horlbeck et al. (2018) datasets (100,576 combinations for K562; 95,703 combinations for Jurkat T), along with 10% and 30% of each dataset for validation and testing, respectively.
-45
NAJAD Rxcd 4--im) Model
log MSE
-5.0
-5.5
-6.-
0.1
Training Fraction
Figure 2. Performance on test data of different gene embedding settings in NAIAD using the Norman dataset (4,429 training com- binations) across varying training data sizes, reported as log (MSE). The models without gene embeddings with low-dimensional em- beddings perform well with small training data but do not improve as more data are added. In contrast, the MLP model with larger gene embeddings outperforms these models when the training data exceeds 30%. The adaptive embedding approach achieves the best performance across all training data sizes.
4.3. Active Learning Experiments
To mimic an active learning scenario, in Section 5.3 we started with 100 samples from the Norman dataset in the first round, and incrementally included an additional 100 samples each active learning round for 4 additional rounds. For the Simpson and two Horlbeck datasets. we began with 500 samples in the first round, and incrementally included 500 more each round for 4 additional rounds.
performance (Figure 2).
We investigated different gene embedding configurations: an over wver-parameterized single-gene effect model without embeddings, NAIAD with fixed 4-dimensional gene em- heddings, and NAIAD with training-data-size adaptive beddings ranging from 2 to 128 dimensions. The results suggest that adaptive embeddings consistently achieve su- perior performance in most cases, regardless of the training data size (Figure 2) in the Norman dataset. This adaptability allows the model to effectively leverage the strengths of both embedding representations and combined single-gene effects. Of note, we observed that the compressed scalar val- ues from the single-gene components strongly correlate with the linear-model predicted results, demonstrating that the NAIAD model leverages the strong baseline performance of linear models. Moreover, the correlation between the scalar values from the gene embedding components and the linear residuals becomes stronger as the training data size increases (Appendix Figure 6).
The data for the first round is selected uniformly, and the incremental data added in each subsequent round is selected via an acquisition function (see Appendix C for descrip- tion of different acquisition functions used). The iterative data selection process allowed us to progressively improve the models by incorporating more data based on the active learning strategy.
In Section 5.4, we extend our active learning framework to a different intervention modality- drug treatment. We use the Bertin et al. (2023) drug dataset, and follow the same sampling schedule as for the Norman gene dataset.
5. Results
5.1. Small Sample Learning and Adaptive Gene Embeddings
One of the primary challenges in active learning is that it typically begins with a small training dataset (Chandra et al., 2020). When the amount of training data is limited, linear models consistently outperform deep learning models. We observed similar patterns in this study particularly when the training data size is below certain threshold (e.g., 10% in the (Norman et al., 2019) dataset, corresponding to 20 observations per gene during training). As the training data size increases beyond this threshold, the Multi-Layer Perceptron (MLP) begins to surpass the linear model in
5.2. Benchmark Analysis in Small Sample Learning
We benchmarked NAIAD against the linear model, MLP, GEARS, and RECOVER on the previously bulk combinatorial perturbation datasets that cover 6,328; 147,658; 100,576; and 95,703 gene combinations (detailed information of benchmark models is described in Appendix B). NAIAD consistently outperformed all other models, par- ticularly in situations with a limited number of observed gene pairs, as measured by log( Mean Square Error), Pear-
four
5

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
Table Root square error (RMSE) of five models on data when each gene observed in training data approximately 4 and 20 times, Error is standard error (SE) across three cross-fold replicates.
Dataset RMSE (x 10 2)
Gene Frequency
Model
Norman
Simpson
Horlbeck K562
Horlbeck Jurkat
Linear
6.2 (1.7)
3.3
6.4 (0.9)
3.9 (0.6)
MLP
7.7 (0.9)
4.3 (0.9)
7.8 (2.5)
5.5 (2.1)
4
GEARS
16.6 (19.8)
5.4 (3.4)
13.0 (11.6)
13.5 (16.1)
RECOVER
7.1 (2.8)
3.9 (0.4)
7.9 (2.0)
5.0 (0.7)
NAIAD
(1.8)
2.2 (0.1)
6.1 (1.9)
3.0 (0.6)
Linear
(1.1)
3.3 (0.2)
6.4 (0.9)
3.8 (0.6)
(1.4)
2.3)
(..1
3.0 (0.4)
20
GEARS
10.7
(2.0)
(4..0))4.)
20.7 (24.0)
RECOVER
(0.5)
1,9 (0.4)
(1.0)
3.0 (0.4)
NATAD
4.7 (0.1)
1.9 (0.2)
(0.6)
2.8 (0.6)
IAD's performance was consistently the best-over 40% better than the second-best model on average across the four datasets (Table 1) based root mean square error (RMSE). As the frequency of gene occurrence increased, the perfor- mance difference among the models gradually decreased (Table 1, Appendix Table 6). When each gene appeared 20 times within different gene combinations in the training dataset, all models achieved comparable performance levels (Table 1). This suggests that as more data becomes available, the gene embeddings learned from different deep learning models can all largely capture the genetic interactions that dominate performance. Notably, because Gene Occurrence depends on the number of unique genes included in the screen, achieving a high frequency of observations per gene becomes increasingly challenging when the aim is to cover wide range of genes. For example, screening 20,000 genes across the genome to obtain an average of 20 observations per gene would require measuring 200,000 combinations in the initial training dataset. This represents a substantial experimental cost and even exceeds the size of the largest current screen, which covers over 145,000 combinations (Simpson et al., 2023).
Norman Dataset 16,328 comb)
Simpson Dataset (147,858 comb)
MSE
log MSE
500
1000
1500
2000
4000
5000
Number of Training Gene Comb
Number of Training Gene Comb
beck Dataset (100,576 comb)
Horlbeck Dataset (95,703 comb)
MSE
log
-3"
2010
4200
1000
4000
6200
Number
Numbar Training
Figure 3. Benchmark analysis comparing the NALAD model with GEARS and RECOVER models, evaluated using test data (MSE) across different numbers of gene combinations train- ing data, Error bars SE across three cross-fold replicates.
These results demonstrate the robustness of NATAD in han- dling both small and large training datasets, making it valuable tool for exploring vast combinatorial spaces. This highlights its potential for discovery of effeetivee gene combi- nations in settings with constrained experimental resources.
son correlation coefficient, and true positive rate from the held-out test data (Figure 3, Appendix Figure 7).
Due to variations in the total number gene pairs measured across these datasets, we also analyzed model performance based on the average frequency of each gene's occurrence among the training data combinations. Gene occurrence was approximated as Gene Occurrence = X where N is the number of gene combinations in the training set (the factor 2 assumes a symmetrical screen), and M is the number of unique genes covered the screen. We observed that when each gene was seen on average four times, NA-
5.3. Effectiveness of Maximum Predicted Efffect Sampling in Identifying Effective Gene Pairs
2N
The second challenge in active learning frameworks is the design of the recommendation system or acquisition func- tion. We explored multiple acquisition functions -including uncertainty sampling, Maximum Predicted Effect (MPH)
6

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
Round 0
Round
Round 2
Round 3
Round 4
9.- Froston D Desppenee
Deccpnene
0.8
Frotton DEoONoO
Freddon Desowwed
0.8
Foston Desoverd
0.8
0.8-
0.6
98
18,50
Se-
Viattey
0.4
C4
0.4
0-
0.0
60
100
150 200
60
100
150
200
50
100
150
200
100
150
200
50
100
150
200
Top N Pertutations
Top N Parurtations
Top N Perfurbations
Perturbstions
Perturbations
Fraeten Fratern Deojveee
8-
Desoveedd
0.6
Fronton Frodonn Dicavenes
Fractoo Dicovete
Dicoress
0.0
(141250 camea
0.5
0.8
Frondoo D
0.8
0.4-
0.4
0.2
0.2-
0.2
0.0
0.0
80 100
150
200
200
50
100
150
200
100
150
200
00
100
130
200
Top N Perturtations
Peruroerions
Top N Porturations
Top Porturaations
Top Perturbations
Hoi Back 08s0 Dannet
24 Frecton Diooeeee
8--
Frecton Decoveeed
0.8
Foeton Disovess
0.a
Decoverd
0.5
Frostnn Doooveng
0.8
0.8
0.
0.5
(100570
0.4-
0.4
0.4
0.2-
0.2
50
Perturaations
100
200
0.0
Top Perurreeionss
90
150
200
0.0
Top N Perfuttations
50
100
150
200
Top N Perturbations
100
150
200
Perturbations
100
150
Postoon Decooeeedd
Deccoveedd
0.8
Foorton ForstnnDDavaae Dicawens
Decoweed
0.8
Fostion Depooerss
0.8
0.8
0.6
0.8
05
0.4
04
0.4
0.4
32
0.2
8.
0.0
0.0
0.
60
100
160
60
100
150
200
50
100
150
200
90
100
150
200
60
100
150
850
Pattrtations
Top
Parurowiooss
Thp
Porturbations
Top Perfurbstione
Peeturtations
Figure Comparison different acquisition functions evaluated by top-N prediction accuracy for the strongest perturbations across four iteration rounds (see Appendix D for full description of accuracy metric). Error hars are SE from three cross-fold replicates.
line ensemble of NAIAD models. We then used the trained model ensemble to infer unseen combinations across the entire combinatorial space and applied different acquisition functions on the corresponding ensemble metric (e.g. MPE or ensemble uncertainty) to select gene pairs for measure- ment as the additional training data for the next round. We repeated the sampling and retraining process across four iterations.
Table Number of top 200 gene combinations correctly discov- ered at round 4 of active lcarning.
Norman
Simpson
Horlbeck K562
Horlbeck Jurkat
Uniform
Y2
66
39
79
MPE
144
144
94
161
UCB
97
111
55
117
Uncertainty
45
84
We found that MPE sampling outperforms other sampling methods by identifying a higher fraction of the globally strongest gene pairs (Figure 4). The advantage of MPE sam- pling becomes larger with each additional iteration, even though all approaches show improved performance over iterations. By the fourth iteration, across the four datasets, were able uncover over twice many strong per- turbations using MPE sampling compared to uniform sam- pling, and nearly 1.5 times as many as UCB, the second-best method (Table 2). Specifically. the MPE method identified approximately 150 out of the top 200 strongest gene pairs in three of the datasets, achieving the highest marginal gain in each dataset (Table 3). We found this result be robust under different active learning ensemble sizes, ranging from model per ensemble up models per ensemble (Appendix Table 5).
Table 3. Marginal gain of correctly discovered top 200 gene com-
binations at round active learning.
Norman
Simpson
Horlbeck K562
Horlbcek Jurkat
Uniform
2.5
MPE
14
27
14.25
23
UCB
2.25
18.75
4.5
12
Uncertainty
1
9.5
2
3.75
sampling, and Upper Confidence Bound (UCB) sampling from RECOVER (Bertin et al., 2023), which combines resid- ual and uncertainty sampling- to evaluate their impact overall performance (Figure 4). To simulate the active learn- ing process using current publicly available symmetrical screening data, we started with the same uniformly sampled gene pairs for all acquisition functions and trained a base-
7

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
Table 4. Number of top 200 gene combinations correctly discovered round of active learning using NATAD and RECOVER with
different acquisition functions. Error is SE across three cross- replicates.
Model
Method
Norman
Simpson
Horlbeck K562
Horlbeck Jurkat
NAIAD
Uniform
93.3 (0.7)
70.3 (5.0)
38.7 (0.3)
81.3 (0.9)
RECOVER
Uniform
81.0 (0.0)
44.3 (0.3)
37.0 (0.6)
75.0 (0.6)
NAIAD
MPE
143.0 (1.5)
141.7 (3.7)
99.7 (4.2)
150.0 (4.0)
RECOVER
MPE
138.7 (2.7)
88.3 (28.2)
53.7 (23.7)
65.7 (10.4)
NAIAD
UCB
110.3 (2.5)
102.7 (8.1)
60.3 (3.3)
96.0 (8.5)
RECOVER
UCB
84.7 (3.5)
62.0 (14.1)
26.3 (1.3)
48.7 (7.1)
Although MPE sampling exhibited worse overall perfor- mance based on the test set MSE (Appendix Figure 8), this discrepancy likely due to the distribution of the sam- pled training data. MPE sampling skews the distribution toward strong-effee gene pairs, leading to more accurate predictions for these pairs but less accuracy actoss the entire dataset, thus affecting overall MSE (Appendix Figure 9).
The primary goal of our NAIAD active learning framework is not to accurately predict all gene pair interactions but to effectively select the strongest gene pairs that induce significant phenotypic changes. Therefore, the superior per- formance of NAIAD+MPE sampling in identifying potent gene combinations aligns well with our objectives. By using model that most effectively represents genetic perturbation interactions, and prioritizing the discovery of the strongest gene pairs, we can accelerate the identification of gene com- binations that are most relevant for therapeutic development.
We also considered alternative models and acquisition func- tions to assess their ability to identify strong pairs of per- turbations. Additional experiments performed in an active learning setting using the MPE acquisition function showed that NAIAD outperformed both RECOVER and GEARS (Appendix Table 7). We next examined the effect of com- bining exploration and exploitation during sampling by in- corporating an uncertainty-base sampling function (see Appendix C.5), inspired by the UCB strategy from RECOVER. This MPE+uncertainty function differs from UCB, which adds an ensemble uncertainty term to the residual (subtracting ensemble prediction mean from linear model baseline) rather than the mean of the ensemble estimate. We found that including an uncertainty term did not improve performance, and when assigned high weight in the sampling function, reduced the number of strong per- turbations discovered during acquisition (Appendix Table 8).
5.4. Extending NALAD for Drug Combination Predictions
To assess NAIAD's effectiveness beyond genetic interac- tions, we applied our framework to drug combination pre- dictions. Drug combination screening is commonly used approach for identifying optimal combination therapies that can enhance treatment efficacy while minimizing toxicity. We extended NAIAD to drug combination using the publicly available Zheng et al. (2021) dataset, which was adapted by the Bertin et al. (2023) RECOVER study (details in Ap- pendix A). Starting with randomized drug embeddings for the small molecules included in the screen, we incorporated MPE as the acquisition strategy.
score term to the MPE
We benchmarked NAIAD against RECOVER in an active learning framework for drug combination screening, eval- uating performance improvements over successive experi- mental iterations. As the number of experimental iterations increased, all methods showed performance gains. NA- TAD consistently outperformed RECOVER in identifying the strongest drug combinations, and the performance gap increased significantly starting from Round 2. This sug- gests that NAIAD learns meaningful drug interaction rep- resentations more efficiently than RECOVER, making it particularly advantageous when the number of experimental iterations is limited. In later rounds, we observed compara- ble performance between the two models when considering the top 50 pairs, but NALAD more effectively identifies the strongest combinations when considering both fewer than 50 and greater than 50 top combinations. Overall, NALAD
Furthermore, we disentangle the contributions of the MPE acquisition function and the NAIAD model architecture to- ward identifying optimal gene pairs (Table 4). We found that the MPE acquisition function robustly identifies strong combinations when using the RECOVER model, and addi- tionally that NAIAD outperforms RECOVER head in an active learning context for every acquisition strategy evaluated. The NAIAD model and MPE acquisition function Individually increased our ability to find effective combina- tions, and they jointly identified more top- pop-effornnng gene pairs than any other model-function pairing. Across the four gene perturbation datasets, after four rounds of active learning, NAIAD+MPE outperformed RECOVER+MPE by 1.7 times on average, and NAIAD+UCB by 1.5 times on average.
8

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
Round 0
Round 1
Round
Round
Round 4
0.3
06
0.6
0.3
(1828 conb)
0.
Thang
0.3
0-5
0.3
0.2
02
0.2-
NA.D MTC
292
10C
252
Top Perturbat on1
Tap N Perturbations
T5p Ni Part.rtattore
T5p Perturbattonn
Top N. Terturtations
Figure 5. Comparison NAIAD and RECOVER in drug combination screening, assessed by top-N prediction accuracy for the strongest N perturbations across four iteration rounds. The NAIAD model with uniform sampling is used as the bascline model. Error bars are SE from three cross-fold replicates.
is superior to RECOVER in identifying a broader range of effective drug pairs (Figure 5).
from single-cell combinatorial perturbation data. However, when projecting single-cell expression data into one rele- vant phenotype value, the NAIAD framework can still be adapted to predict that specific phenotype. Another lim- itation is that our model assumes we have knowledge of each gene's 's individual effect on the phenotype. For unseen individual genes, the current NAIAD model cannot predict the effeets of combinations involving these genes, especially when both genes in a pair are unseen. Incorporating properly pre-trained gene embeddings could potentially allow us to predict such unseen situations (Cui al., 2024). By incor- porating pre-trained gene embeddings, can obtain prior knowledge of the similarities between unseen and known genes within relevant latent spaces. Assuming that these gene relationships are conserved across different domains, the model can leverage this information to infer the single- gene effects of unseen genes, even to predict the outcomes of combinations involving two previously unseen genes. Additionally, refining the acquisition score function to be learnable component-such submodular regularization (Alieva et al., 2020; Wei et al., 2015; Golovin & Krause, 2010) could enable the model to adaptively prioritize experiments that maximize information gain, rather than relying on a fixed heuristic.
These results demonstrate NAIAD's robustness across both gene and drug perturbation domains. Its generalizability, combined with its efficiency in low-data regimes, makes NAIAD a practical tool for discovering effective therapeutic combinations under experimental constraints.
6. Discussion and Conclusion
In this work, we introduced NAIAD, active learning framework that efficiently identifies effective gene or drug pairs by leveraging single perturbation effects, adaptive gene emheddings, and an MPE acquisition function. Our frame- work leverages the principles of Bayesian optimization (Fra- zier, 2018), employing sequential experimentation and learn- ing to enable effective dociisinnmaakine with limited data. Whereas traditional Bayesian optimization employs fixed surrogate model, our framework adapts to learn different surrogate functions at varying sizes of training data. This adaptability enables NAIAD to adjust its modeling complex- ity based on the available data, effectively bridging the gap between linear models and deep learning techniques.
employing monotonic
By incorporating adaptive gene embeddings, NAIAD miti- gates overfitting in smalle training and captures plex interactions more data becomes available. The MPE acquisition function further enhances the model's 's efficiency by prioritizing gene pairs with significant predicted effects, accelerating the discovery process with fewer experimental iterations. As a result, NAIAD outperforms existing ap- proaches, particularly in scenarios with limited training data when using the MPE acquisition function. We demonstrate its effectiveness in covery rate for identifying the top N strongest combinations across the entire search space.
Our model also holds significant potential for higher-order combinatorial perturbations. With the advancement of com- binatorial CRISPR technologies, higher-order gene combi- nation datasets are becoming increasingly common (Tieu et al., 2024; Hsiung et al., 2024). NALAD is theoretically well-suited and can be casily adapted to accommodate inter- actions beyond gene pairs. Handling higher- CRISPR combinatorial perturbations would allow for the exploration of more complex genetic interactions, potentially leading to more effective induction of desired cellular phenotypes.
learning and its high dis-
Software and Data
Despite its advantages, there are areas where the NAIAD model could be further enhanced. Currently, NAIAD does not support predicting gene expression profiles resulting
The source code for NAIAD is available at http://ggiibbccm///ppuueetuuAAAD
9

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
Acknowledgements
Chandra, A. L.., Desai, S. V., Devaguptapu, C., and Bal- asubramanian, V. N. On initial pools for deep active learning, arXiv preprint, 2020. URL http:://axtti org/ abaabb//u /2011. 14696.
We thank our colleagues at Neptune Bio for creating a col- laborative and supportive environment that enabled us to complete this work.
Cubuk, E. D., Zoph, B.. Mané, D., Vasudevan, V., and Le, Q. V. Autoaugment: Learning augmentation strategies from data. In 2019 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 113-123, 2019. doi: 20.1109/CVPR.229900000
Impact Statement
This paper aims to advance the field of Machine Learn- ing with significant implications for biomedical research. NATAD's active learning framework has the potential to accelerate the discovery of effective combination therapies by identifying strong combinatorial and synergistic targets with enhanced efficacy and reduced toxicity. By optimiz- ing experimental resource allocation, our framework could contribute to more efficient drug development and improved therapeutic strategies.
Cui, H., Wang, C., Maan, H., Pang. K., Luo, F., Duan, N., and Wang. B. ScGPT: Toward building a foundation model for single-cell multi-omics using generative AL. Nature Methods, 21(8):1470-1488) 2024.
De Pessemier, T., Vanhove, S., and Martens, L. Batch versus sequential active learning for recommender sys- tems. In Proceedings of the 4th Workshop on On- line Recommender Systems and User Modeling (OR- SUM 2021) in conjunction with the 15th ACM Con- ference on Recommender Systems (RecSys 2021), pp. 11, 2021. URL httts:://rrsu.... http://orruummmsstccc.... www.ctec..p........m...m.. 2rsum202111 02summ2222//ssett////ss/////s
References
Ahlmann-Eältze, C., Huber, W., and Anders, S. Deep learning-based predictions of gene perturbation effects do not yet outperform simple linear methods. bioRxiv preprint, 2024. URL https::////// /twii 1101// 2024. 09. 16. 613342.
pdf.
Eisenstein, M. Active machine learning helps drug hunters tackle biology. Nature Biotechnology, 38(5):512-514 2020.
Al-Lazikani, B., Banerji, U., and Workman, P. Combina- torial drug therapy for cancer in the post-genomic era. Nature Biotechnology, 30(7):679-69) 692.699999222222 2012.
Frazier, P. I. A tutorial on bayesian optimization. arXiv preprint, 2018. URL http://arxiv.org 1807.02811.
Alieva, A., Aceves, A., Song. J., Mayo, S., Yue, Y., and Chen, Y. Learning to make decisions via sub- modular regularization. In Proceedings of the In- ternational Conference on Learning Representations (ICLR), 2020. URL https:/ https////peeeeee..eeee. pdfPid=ac22BvvG__7
Gaudelet, T., Del Vecchio, A., Carrami, E. M., Cudini, J., Kapourani, C.-A., Uhler, C., and Edwards, L. Sea- son Combinatorial Intervention Predictions with Salt Peper. arXiv preprint, 2024. URL http:// org/abs//404..11 16907.
Anwar, A., Gupta, R., Merchant, 7., Ghosh, S., Neiswanger, W., and Thomason, J. Efficient evaluation of multi-task robot policies with active experiment selection, 2025. URL https: //ariiv//axvv.../ org/ abs/2502, 09829.
Geiger-Schuller, K., Fraslan, B., Kuksenko, O., Dey, K. K., Jagadeesh, K. A., Thakore, P. I., Karayel, O., and et al. Systematically characterizing the roles of e3-ligase family members in inflammatory responses with massively par- allel perturb-seq. bioRxiv preprint, 2023. URL https: //doi. org 10.10000. 1111/1122111111 01.23.13. 525198.
Bereket, M. and Karaletsos, T. Modelling cellular perturba- tions with the sparse additive mechanism shift variational autoencoder. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc. doi: 00.5555/3666112..0 222.666662222366222
Golovin, D. and Krause, A. Adaptive submodularity:
Theory and applications in active learning and stochas- tic optimization. arXiv preprint, 2010. URL http: / //arxiv. wwt.iii.rg//bb//0003 3967..
Bertin, P., Rector-Brooks, J., Sharma, D., Gaudelet, T., Anighoro, A., Gross, T., Martínez-Peña, F., and et al. RECOVER Identifies Synergistic Drug Combinations in Vitro through Sequential Model Optimization. Cell Reports Methods, 3 3(10):00599, 2023. doi: 10.1016/j. seteth.202.1100555
Hammond, J., Leister-Tebbe, H., Gardner, A., Abreu, P., Bao, W., Wisemandle, W., Baniecki, M., and et al. Oral nirmatrelvir for high-risk, nonhospitalized adults with covid- 19. The New England Journal of Medicine, 386 (15):1397-1408, 2022.
10

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
Hao, M., Gong, J., Zeng, X., Liu, C., Guo, Y., Cheng, X., Wang, T., Ma, J., Zhang, X., and Song. I.. Large-scale foundation model on single-cell transcriptomics. Nature Methods, (188):44111444
Lopez, R., Tagasovska, N., Ra, S., Cho, K., Pritchard, J., and Regev, A. Learning causal representations of single cells via sparse mechanism shift modeling. In van der Schaar, M., Zhang, C., and Janzing, D. (eds.), Pro- ceedings of the Second Conference on Causal Learning and Reasoning, volume 213 of Proceedings of Machine Learning Reseurch, pp. 662-691. PMLR, 11-14 Apr 2023. http:::s: http///peoceee..pee/ee/ep/e/ssee mlr. press// v213/1opezz html.
2024.
Horlbeck, M. A., Xu, A., Wang, M., Bennett, N. K., Park, C. Y., Bogdanoff, D., Adamson, B., and et al. Mapping the genetic landscape of human cells. Cell, 174(4):953- 967.e22, 2018.
Hsiung, C. C., Wilson, C. M., Sambold, N. A., Dai, R., Chen, Q., Teyssier, N., Misiukiewicz, S., and et al. Engi- neered CRISPR-Cas for higher-order combinatorial chromatin perturbations. Nature Biotechnology, pp. 1-15, 2024. Published online May 2024.
Lotfollahi, M., Klimovskaia Susmelj, A., De Donno, C., Hetzel, L., Ji, Y., Ibarra, I. L., Srivatsan, S. R., and et al. Predicting cellular responses complex perturbations in high-throughput screens. Molecular Systems Biology, May 2023. doi: 10. (0.15252/mmb.22222111 1517.
Hwangbo, H., Patterson, S. C., Dai, A., Plana, D., and Palmer, A. C. Additivity predicts the efficacy of most approved combination therapies for advanced cancer. Na- 2023.
Mokhtari, R. B., Homayouni, T. S., Baluch, N., Mor- gatskaya, E., Kumar, S., Das, B., and Yeger, H. Combi- nation therapy in combating cancer. Oncotarget, 8(23); 38022, 2017.
ture Cancer,
Jastreboff, A. M., Kaplan, L. M., Frías, J. P., Wu, Q., Du, Y., Gurbuz, S., Coskun, T., Haupt, A., Milicevie, Z.,, and Hart- agonist retatrutide for obesity - a phase 2 trial. The New England Journal of Medicine, August 2023. doi: 10. 10.1060/NMMM023311777..
Nature Medicine. Rationalizing combination therapies. Na- ture Medicine, 23(10):111331111 2017.
man, M. L. seiple-hhormoe-reeee
Norman, T. M., Horlbeck, M. A., Replogle, J. M., Ge, A. Y., Xu. A., Jost, M., Gilbert, L. A., and Weissman, J. Exploring genetic interaction manifolds constructed from rich single-cell phenotypes. Science, 361(6455):786-993. 2019.
Kingma, D. P. and Welling, M. Auto-encoding variational bayes. In Bengio, Y. and LeCun, Y. (eds.), 2nd Interna- tional Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http: wwwaaa.ivooroo... abs/1312 6114.
Rood, J. E., Hupalowska, A., and Regev, A. Toward foundation model of causal cell and tissue biology with a perturbation cell and tissue atlas. Cell, 187(17)4450-- 4545, 2024.
Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https:// openreview http::/ee/t/eoeeoee wtt.orrruuunnnsSsSAAAAaYYee-
Roohani, Y., Huang, K., and Leskovec, J. Predicting tran- scriptional outcomes of novel multigene perturbations with GEARS. Nature Biotechnology, 42(6))927-935, 2023.
Samms, J.., Coghlan, M. P., and Sloop, K. W. How may gip enhance the therapeutic efficacy of glp-1? Trends in Endocrinology and Metabolism, 31(6):410-44) 421- 2020.
Lahlou, S., Jain, M., Nekoei, H., Butoi, V. L., Bertin, P., Rector-Brooks, J., Korablyov, M., and Bengio, Y. DEUP: Direct epistemic uncertainty prediction. Transactions Machine Learning Research, 2023. ISSN 2835-
Shyr, Z. A., Cheng, Y.SS., Lo, D. C., and Zheng, W. Drug combination therapy for emerging viral diseases. Drug Discovery Today, 26(10):2367, 2021.
8856. URL petttp::/openreeee id=eGLaVRvvv0, http://ppeneeeeeeeeeeefoootoe nww./oruumt
Simpson, D., Ling, J., Jing, Y., and Adamson, B. Mapping the genetic interaction network of parp inhibitor response. bioRxiv preprint, August 2023. URL https: doii org/10. 1101/2022 08. 19. 553986.
Lakshminarayanan, B., Pritzel, A., and Blundell, C. Sim- ple and scalable predictive uncertainty estimation ing deep ensembles. In Proceedings of the 31st In- ternational Conference on Neural Information Process- ing Systems (NeurIPS), NIPS' 17, pp.
Red Hook, NY, USA, 2017. Curran Associates Inc. doi: 10.5555/8295222.39955
Takahashi, K. and Yamanaka, S. Induction of pluripotent stem cells from mouse embryonic and adult fibroblast cultures by defined factors. Cell, 126(4):663-61) 2006.

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
Tieu, V., Sotillo, E., Bjelajac, R., Chen, C., Malipat- lolla, M., Guerrero, J. A., Xu, P., and et al. A versatile CRISPR-Cas platform for multiplexed transcriptomic regulation and metabolic engineering in primary human cells. Cell, 277-192228- 2024.
Wei, K., Iyer, R., and Bilmes, J. Submodularity in data subset selection and active learning. In Proceedings of the 32nd International Conference on Machine Learning (ICML), pp. 1954-1963. PMLR, 2015.
Zheng, S., Aldahdooh, J., Shadbahr, T., Wang, Y., Aldah- dooh, D., Bao, J., Wang. W. and Tang, J. Drugcomb update: more comprehensive drug sensitivity data repos- itory and analysis portal. Nucleic Acids Research, 49 (W1):W174-W184) 06 2021. ISSN 0305-1048. doi: 10. 10.10993/a/// URL https: httpaa:/ddooooooo//// 1093/nar/gkab43.
12

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
A. Dataset Summary
A.1. Genetic Perturbations
We utilize four bulk combinatorial perturbation datasets in our study;
Combinatorial CRISPRa on K562 cells (Norman et al., 2019): This dataset was generated using combinatorial CRISPR activation (CRISPRa), involving 112 genes and 6,328 unique gene combinations.
Large-scale combinatorial CRISPRi on K562 cells (Simpson et al., 2023): This dataset was generated using CRISPR interference (CRISPRi), involving 543 genes and 147,658 unique gene combinations.
Combinatorial CRISPRi on K562 cells (Horlbeck et al., 2018): This dataset was generated using combinatorial CRISPRi, involving 448 genes and 100,576 unique gene combinations.
Combinatorial CRISPRi on Jurkat Cells (Horlbeck et al., 2018): This dataset was generated using combinatorial CRISPRi, involving 437 genes and 95,703 unique gene combinations.
For the bulk cell viability datasets, we calculated the log-fold change in cell viability for each CRISPR-guide combination compared to negative control treatments. This normalization allows for the quantification of the effect size of each perturbation on cell survival. Next, we averaged these cell viability measurements across all pairs of guides targeting each unique pair of genes, to generate gene-level measurements of cell viability. To calculate the effect of single-gene perturbations, we used all guide pairs consisting of gene-targeting guide and one non-targeting guide. For a deeper explanation of calculating cell viability (also referred to as 7), we refer the reader to Simpson al. (2023).
A.2. Drug Treatment
We utilize the drug dataset prepared by Zheng et al. (2021) and used by Bertin et al. (2023) RECOVER to benchmark their active learning platform. The dataset was generated by assaying 2.320 cell lines with single and two-drug treatments. For benchmarking NAIAD,
the 4,349 unique two-drug treatments conducted on K562 cells.
We processed the data in several ways:
1. The drug database contains different dosage treatments for each drug the K562 cell line. Since drugs may have dosage-dependent treatment effects, we filter the assays to only include single dosage for each drug. For each drug, we choose the dosage closest to the IC50 of that drug in K562 cells, and discard all measurements at other dosages.
2. We only use combinations for which the corresponding single-drug measurements (at the same dosages) are also present in the dataset.
3. We remove conditions that saturate the cell viability measurements, i.e. combinations and individual drugs with 80% cell viability inhibition. We rationalize that at this detection limit of the experiment, a) the measurements are overly-noisy as result of low cell counts, and b) there are meaningful interactions between drugs since all cells are dead under each individual treatment.
After these filtering steps, are left with 1,828 unique measurements in K562 cells across combinations of 94 unique drugs.
B. Benchmark Models
Linear model: We employed simple linear regression model using the effects of two single-gene perturbations as independent variables to predict their joint effect. This model involved three parameters: two coefficients for the individual gene effects and an intercept term.
Multi-layer perceptron (MLP): We implemented an MLP model where each gene was represented by 128-dimensional embedding. For each two-gene perturbation, we created a joint embedding for the two genes by summing their embeddings along each dimension and passing it through a MLP layer that projects from 128 dimensions down to a single dimension corresponding to the phenotype value. The MLP captured non-linear interactions between gene embeddings.
GEARS: For the GEARS model, we adhered to the original settings as specified in their supplementary materials (Roohani
13

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
al., 2023) for cell viability prediction. This included their specific network architecture and parameters used in their tutorial. For bulk-combinatorial datasets lacking single-cell experiments expected for GEARS, generated synthetic Pertub-seq datasets of normalized gene expression matrices, with separate Gaussian (O, 1) used for sampling the expression of each gene.
RECOVER adaptation: RECOVER was originally developed for drug perturbations involving small molecule embeddings and bilinear operations to combine drugs (Bertin et al., 2023). The models were trained to predict Bliss synergy scores, which capture the non-linear components of phenotypic outcomes, instead of directly modeling the phenotypic outcomes themselves. We adapted RECOVER for gene perturbations by incorporating bilinear projection module to combine 1 128- 28-dimensional gene embeddings. This adaptation allowed to model gene-gene interactions using the same principles applied to drug combinations in the original RECOVER framework. We train the model to predict the gene-equivalent of Bliss scores, which is the difference between measured viability and the product of the viability of single-gene perturbations. We then convert the gene-Bliss- Sciss predictions back to overall cell viability predictions by re-adding the product of the viability of the Bogle-ggene perturbations.
C. Active Learning Sampling Strategies
In active learning framework, employ several sampling strategies to select the most informative experiments for subsequent rounds. Below, we provide the mathematical formulations for these strategies.
C.1. Uncertainty-Basee Sampling
We use ensemble of models {M, each initialized randomly. For candidate gene combination x, each model provides prediction Y: = M1).). We estimate the prediction uncertainty using the standard deviation of the ensemble predictions:
SD
(y 9)
where ÿ is the mean prediction across the ensemble:
1 N y
yi-
The acquisition function for uncertainty-brac based sampling is defined as:
Guncertainty (x) SD(v).
We select the candidate combinations with the highest Guncertainty (1) values, as they represent samples where the model is most uncertain and additional data could significantly improve the model.
C.2. Maximum Predicted Effect (MPE) Sampling
This strategy targets gene combinations predicted have strong effects. The acquisition function is:
MPPA x = y.
where ÿ is the mean prediction as defined above. We select candidates with the highest absolute predicted effects 7 prioritizing experiments likely yield substantial biological insights or therapeutic benefits.
C.3. Residual- Baasedd Sampling
To identify areas where nonlinear interactions are significant and the model may not have fully captured them, we compute the residual between predictions from non-linear model and a linear model. Let N N(x) be the prediction from the non-linear
14

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
model and L(x) be the prediction from linear approximation. The residual is calculated as:
(z) [N [(x)].
The acquisition function for residual- sampling is:
Gresidual (x)) = r(x))
We select candidate combinations with the highest residuals (residuat focusing on samples where the non-linear effects are most pronounced and the model's predictions differ significantly from linear expectations.
C.4. Upper Confidence Bound (UCB) Sampling
We perform UCB sampling following the strategy provided by RECOVER (Bertin et al., 2023):
aUCB (x)
Oresidual (x) * Guncertainty (x)
Where is a hyperparameter set to based the recommendation from the RECOVER paper.
By combining these sampling strategies, we aim to efficiently explore the combinatorial space of gene interactions. The goal is to maximize information gain with each experimental round, ultimately using a minimal number of experiments to identify effective gene combinations that can induce desired cellular phenotypes.
C.5. Mean Upper Confidence Bound (Mean-UCH) Sampling
Inspired by the UCB acquisition function from RECOVER, we develop an exploitationnexplraatiie balanced acquisition function that utilizes the mean of the ensemble prediction rather than the residual relative to a linear model. We call this sampling method Mean Upper Confidence Bound (Mean-UCH) sampling. We express the acquisition function as weighted combination:
score
x y + x SD(v)
where ÿ the mean across the ensemble predictions, and SD(y) is the standard deviation across ensemble predictions.
This unified view captures multiple acquisition strategies:
Uniform sampling: a = 333 =
Uncertainty-only (pure exploration):
MPE (pure exploitation): ==0
UCB (balanced trade-off): a
We evaluate the performance of the acquisition function on identifying the strongest perturbations in Appendix Table 8.
D. Model Evaluation Metrics
We use several metrics evaluate the performance of our models.
MSE: Mean Square Error
(wets)
mnth )2
N
pairs
Egenns
Fraction Discovered: Through active learning, we want to identify how many of the top P strong perturbations have been identified by our model of interest. Let M be the total number of rounds of active learning, and let N [Mo, 71...., nM] be the list that defines how many samples n; are used for each active learning round i 0, 0,l...., . M.
15

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
In each round of learning, train the ny unmasked samples chosen depending on the acquisition function. After training, make predictions for the entire unseen dataset. We concatenate the predictions the unseen data with the unmasked measured values from the seen data, and call this combined set of predictions and measurements X; for each round i.
Let X be the set of top P values in the set X, Let YI, be the set of top P measured (ground truth) values across all samples in round i. Let P; = i, n Yi,rl be the number of times a perturbation from the top P predictions is also the top P targets for round i. Fraction Discovered is then defined as f.
The marginal gain at round 4 is (o(top=200)- (o(top=200) where Pa(top 200) is the number of top-200 perturbations correctly identified in Round 4, and Po(lop 200) is the number of top-200 perturbations correctly identified in Round O.
Marginal Gain:
4
TPR: True positive rate
Let X be the set of top N measured combinations, and Predss be the set of top N predicted combinations. We find the number of matches Nmatch
x urgels n X preddy and calculate TPR as Naze.
E. Hardware Configuration
All model training was done on single Paperspace A100-80G server with 100GB of RAM.
F. Hyperparameter Selection
For all training the NAIAD and MLP models, we use learning-rate 10 and batch 1024. We also used linear rate scheduler with 10% of training steps used for warm up, and weight decay 0.
To identify these optimal hyperparameters, we testing hyperparameters across the following ranges:
n_epoch: [50, 100, 200, 500, 1000, 2000]
batch_size: [512, 1024, 2048, 4096]
leamiiggrate: [10 10-3, 10-2 10
d.embed: [2, 4, 8, 16, 32, 64, 128, 256]
d single-gene: [8, 16, 32, 64, 128, 256, 512]
decght [0, 10-1 10-3]
For training the RECOVER model, we use a learning_rate = 10-1. For GEARS, we keep all hyperparameters assigned by default from the package.
F.1. Adaptive Model Embedding Size Selection
For NAIAD model, we choose the embedding size hyperparameter based roughly on the number times each gene was seen in the training set, following the schedule shown Table 9, We determined these values using the idea that model size depends on the number of times each individual gene is seen during training, and that these properties hold across all datasets.
16

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
G. Supplemental Figures and Tables
training data size:
5%
20%
60%
single single-gent gen-eennee effects
Caneeao comporet medtioo
Compressed
022
01-
60
-33
-01
-12
-00
-04
+2
-05
-04
-0.
-04
-04
10
Linear model predicted value
training data size:
5%
20%
60%
-0.20
gene emberklings
Compressed
-64
-22
-03
-32
-01
-00 -04
Linear model residuals
Figure Evaluation of compressed single-gene effects and gene embeddings shows a strong correlation between the compressed single- gene effects and the values predicted by the linear model. As the training data increases, the correlation between gene embeddings and the residuals of the linear model predictions gradually becomes stronger.
Norman Dstaset (6,328 comb)
Simpson Dataset (147,658 comb)
Horlbeck K562 Dataset (100,576 comb)
Horibeck Jurkat Dataset
1.0
1.0
0.3
Condattonn
0.90
Connaation
0.8
Cormlation
0.3
0.85
0.6
0.6
0.7
0.80
NAV
RECOUS
0.4
0.4
0.2
500
1000
:500
2000
4000
6000
2000
6000
6000
0.7
0.9
0.4
Nodi
1000
2000
4000
6000
Number Training Gens Comb
Number of Training Gene Comb
Number of Training Gene Comb
Number of "Trining Gene Como
Figure Benchmark analysis comparing NAIAD model with GEARS and RECOVER models, evaluated using test data correlation and true positive rate (for identifying the top 200 perturbations of 30% held-out test set) across different numbers of gene combinations in training data. Error bars are SE across three cross-fold replicates.
17

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
Table Number of top 200 gene combinations correctly discovered round of active learning using different ensemble sizes. Error standard enror (SE) across three cross-fold replicates.
Ensemble Size
Norman
Simpson
Horlbeck K562
Horlbeck Jurkat
MPE
Uniform
MPE
Uniform
MPE
Uniform
MPE
Uniform
140.7 (2.3)
89.3 (2.0)
149.7 (2.7)
66.7 (4.7)
105.7 (3.9)
35.2)
153.3 (3.8)
79.7 (0.7)
3
141.3 (1.9)
(1.)
146.3 (3.8)
67.7 (4.6)
104.3 (4.3)
37.7 (0.9)
153.3 (5.4)
79.7 (1.5)
5
91.0 (2.5)
143.3 (2.6)
68.0 (2.6)
103.7 (3.8)
(1.0)
150.7 (2.8)
80.7 (1.5)
7
142.7 (1.8)
91.7 (0.7)
143.3 (3.0)
71.3 (4.4)
101.7 (5.8)
38.7 (0.9)
151.7 (3.8)
81.0 (2.5)
Norma
Simpson Datoset
Unfoom
Uniform
HPE
MPE
Uncertainty
Unnrectiint U.B
0.0025
MGE
0.0040
MSE
0.0020
0.0010
0.0025
0.0005
Round
Round
Horbbeck K562
Horibeck Unllcem
Unilorm
MPE
NPE
Uncertainty
0.0015
Uncertatnty
0.0050
0.0014
0.0045
MSE
MSE
0.0012
0.0010
0.0035
Round
Round
Figure Comparison of different acquisition functions evaluated by MSE across four iteration rounds, Error bars SE across three cross-fold replicates.
18

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
FOUNCO NoBOE
CSUNC
COUMOO
SUIIE TEUR00
CO
UCC
LICE
25
35
FARRAO VALL
BATTAAL JUNEE UnGAmen
made vidioe
Sear prided
cortact AODL anotr
Patteck 1502 MPC
holtech UCB 8:000
Nertuct IdE2 Uncarer
harback Brtect unlore
Nattec jroa Mnd
tolbuck anu LCB
Srttect Juka provten
Ionds 5:000 91
Scund Sp300 12
PUUNC CO0O0 31
DANES 50000
10r 20000 12
Ioand: 5:000 91
sundi S0000 18
HA- 1:000 3
Vork 90:00 13
CoreE Vent OF
cards 5:000 31
NomrA 0:200.11 18
Y00 PAOES
Figure 9. Comparison of training sample distribution changes across different iterations using four acquisition functions. The X-axis represents the linear predicted values, while the Y-axis shows the actual measured values. The plot highlights gene pairs belonging to the top 200 gene pairs with the strongest effects in the entire dataset.
19

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
Table RMSE of five models test data when each gene is approximately observed training data 10 times. Error is SE across three cross-fold replicates.
Dataset RMSE (x10 3)
Gene Frequency
Model
Norman
Simpson
Horlbeck K562
Horlbeck Jurkat
Linear
6.1 (1.3)
3.3 (0.2)
6.4 (0.9)
3.8 (0.6)
MLP
5.7 (1.3)
2.4 (0.4)
6.3 (1.1)
3.5 (1.0)
GEARS
15.3 (12.8)
8.7 (10.3)
(9.6)
24.2 (28.5)
RECOVER
5.9
2.5 (0.2)
6.1 (0.2)
3.7 (1.0)
NAIAD
4.7 (1.2)
2.0 (0.2)
5.7 (0.8)
3.0 (0.8)
Table 7. Number of top 200 gene combinations correctly discovered over four rounds of active learning by NAIAD, RECOVER, and GEARS on the Norman datasct. Error is SE across three cross- fold replicates
Model
Method
Round
Round
Round
Round
Round 4
NAIAD
MPE
86.0 (1.0)
100.3 (0.9)
112.3 (1.7)
128.0 (2.5)
(1.5)
RECOVER
МPE
82.0 (1.0)
101.0 (1.2)
110.7 (2.0)
125.0 (2.5)
138.7 (2.7)
GEARS
MPE
42.0 (2.1)
(1.5)
91.7 (0.7)
99.7 (0.3)
109.3 (1.5)
NALAD
Uniform
86.0 (1.0)
88.3 (2.0)
88.3 (0.3)
89.7 (1.2)
93.7 (0.7)
Table 8. Number of top 200 gene combinations correctly discovered at round of active learning using different 3 values in the Mcan-UCB acquisition function. We find that the True Positive Rate (TPR) of the top 200 gene combinations remains relatively stable for small values of (i.c., 3 < 1). This may be because, during carly active learning rounds, the effect magnitude dominates the predictive uncertainty. As increases, the influence of uncertainty grows, leading to a decrease TPR. Error standard error (SE) across three cross-fold replicates.
B
Norman
Simpson
Horlbeck K562
Horlbeck Jurkat
138.3 (2.2)
145.7 (3.8)
99.0 (2.6)
152.3 (4.1)
0.25
139.7 (2.0)
144.7 (3.7)
97.0 (3.0)
151.0 (3.8)
0.5
138.3 (0.9)
148.0 (2.6)
98.0 (0.6)
152.0 (4.0)
0.75
138.3 (0.9)
147.3 (2.0)
98.7 (0.3)
154.7 (4.9)
139.3 (1.7)
144.7 (2.6)
98.0
154.3 (4.6)
1.5
140.7 (0.3)
144.3 (2.3)
(2.9)
153.3 (4.6)
2
138.7 (2.9)
144.0
95.7 (1.9)
153.7 (4.5)
137.0 (4.7)
144.0 (2.6)
81.3 (1.8)
143.3 (4.8)
10
133.0
142.3 (2.6)
72.3 (6.4)
132.0 (6.9)
119.7 (3.4)
124.0 (4.5)
73.0 (5.8)
107.7 (1.8)
Uniform
93.3 (2.9)
(1.1)
39.0 (0.6)
80.3 (0.3)
20

---

Active Learning for Efficient Discovery of Optimal Combinatorial Perturbations
Table Training model hyperparameters for each data set and data split
Dataset
Train Epochs
Avg. Times All
Approx. Training Dataset Size
Embed
Single-Gene
Genes Seen
Dim
Dim
Norman al. (2019)
500
100
64
200
4
64
10
500
16
64
20
1000
16
64
30
1500
32
64
40
2000
64
64
60
3000
64
64
80
4000
128
64
100+
5000+
128
64
Simpson et al. (2023)
200
500
2
256
1000
4
256
10
5000
16
256
20
10000
256
30
15000
32
256
40
20000
64
256
60
30000
64
256
80
40000
128
256
100+
50000+
128
256
Horlbeck et al. (2018)
200
500
4
256
1000
16
256
10
5000
32
256
20
10000
32
256
30
15000
64
256
40
20000
64
256
60
30000
64
256
80
40000
128
256
100+
50000+
128
256
Zheng et al. (2021)
800
100
2
256
200
4
256
300
4
256
400
4
256
10
500
256
21


Reviews:
[{"review_id": "G5DgGDInxY", "reviewer": "Reviewer_vQva", "summary": "This paper presents an active learning framework that efficiently discovers optimal gene pairs by leveraging single-gene perturbation effects and adaptive gene embeddings. The experiments show that the proposed method achieve better performance than baseline methods.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer vQva,\n\nThank you sincerely for taking the time to review our work. We’re encouraged that you found the applications of our method interesting. We appreciate your thoughtful feedback about accessibility for a broader ML audience. We will incorporate a concise, ML-focused framing in the introduction that situates our work within well-known machine learning paradigms such as active learning and surrogate modeling.\n \nHere is the content:\n\n---\n\nWe frame the discovery of optimal gene or drug combinations as a machine learning problem of active search over a high-dimensional combinatorial space, where evaluating each combination (via experiment) is costly. Our method trains a neural surrogate model that predicts the effects of unseen perturbation pairs by combining overparameterized encodings of single-gene outcomes with a learned gene embedding space that models interaction effects. Crucially, the model’s capacity is dynamically scaled with the amount of training data. The surrogate guides new experiment selection via acquisition strategies inspired by Bayesian optimization, with the ability to leverage both exploitation (via maximum predicted effect) and exploration (via ensemble-based uncertainty). \n\nWhile this work is motivated by biological discovery, similar challenges arise across machine learning domains—including data augmentation policy search in vision [1], cold-start item selection in recommender systems [2], and sample-efficient policy learning in robotics [3]—all of which involve large discrete spaces, costly evaluations, and the need for adaptive modeling and decision making. In such domains, discrete components (e.g., transformations, items, actions) play a role analogous to gene or drug perturbations in our framework: each is embedded in a latent space, and combinations of these embeddings are used to represent and evaluate complex configurations. Our framework shows how these components can be actively selected via a data-adaptive surrogate to enable efficient, scalable discovery.\n \n[1] Cubuk, Ekin D., et al. \"Autoaugment: Learning augmentation policies from data.\" arXiv (2018).\n\n[2] De Pessemier, et al. \"Batch versus sequential active learning for recommender systems.\" arXiv (2022).\n\n[3] Anwar, Abrar, et al. \"Efficient Evaluation of Multi-Task Robot Policies With Active Experiment Selection.\" arXiv (2025).\n\n---\n\nAdditionally, we note that Reviewer kwgr regarded our methodology and framing as well-motivated, and our evidence as convincing. We have carefully considered the suggestions from both Reviewers kwgr and PDeb, and have expanded our experiments accordingly. These results are included in our rebuttal and will be incorporated into the final version of the paper.\n\nSpecifically, we have: \n\n(1) further isolated the contributions of the surrogate model and the MPE acquisition strategy\n\n(2) performed a more comprehensive benchmark analysis across various models in active learning iteration\n\n(3) performed hyperparameter tuning for the UCB acquisition function \n\n(4) evaluated the impact of the ensemble size on the performance of NAIAD. \n\n\nWe hope this added context addresses your concerns and helps clarify how our contributions relate to mainstream machine learning. Please let us know if you have any further questions. We will be actively available until the end of the rebuttal period. If you feel your concerns are addressed, please consider reevaluating our work. Looking forward to hearing from you!", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "Yes.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths\n1. the proposed method is straightforward with interesting applications. However, I have to admit that I have shallow understanding on this field. I cannot provide a fair evaluation on the novelty.\n\nWeaknesses\n1. Considering that this is a ML conference, I think the author should provide a preliminary section in the main content so the general reader can understand the task.", "other_comments_or_suggestions": "No", "questions_for_authors": "See Strengths And Weaknesses", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0tWOJEJO7h", "reviewer": "Reviewer_kwgr", "summary": "We can perturb the expression of various genes to achieve a desirable phenotype such as enhanced cell viability. However, given that there are close to 20k known human genes, it is not possible to test every gene combination to identify the optimal combination that leads to the most desirable phenotype. This paper presents an active learning method called NAIAD for discovering optimal gene pairs using existing perturbation data. NAIAD builds a model to predict the effect of jointly perturbing a pair of genes that not only accounts for additive effects but also for interactions been the genes. This model is coupled with an acquisition strategy, the most effective being Maximum Predicted Effects or MPE, to choose gene pairs over active learning iterations. Crucially, NAIAD also explicitly accounts for the need to efficiently model datasets with different sizes over active learning iterations by progressively increasing the dimensionality of gene embeddings as more data is obtained. In their experiments that use gene perturbation and drug combination data, the authors convincingly demonstrate the following:\n- Show that adaptive gene embedding sizes allow NAIAD to learn more effective models across training sets of different sizes when compared to using fixed embedding sizes\n- NAIAD outperforms existing baselines in modelling combinatorial gene perturbation datasets, especially when training data is limited.\n- In the active learning setting, the MPE acquisition function outperforms all other functions and identifies the most number of gene pairs that produce the greatest changes in the measured phenotype.\n- NAIAD + MPE outperforms the baseline RECOVER + UCB method in identifying optimal drug combinations.\n\n## Update after rebuttal\nThe additional results provided in the rebuttal further highlight the effectiveness of NAIAD and help understand the usefulness of MPE alone. All of my questions have been satisfactorily addressed and I will keep my original score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer kwgr,\n\nWe appreciate your supportive feedback and your valuable comments, which have helped us to improve our work further. We will incorporate the results from your suggested experiments in the revised manuscript. We address your questions below. Due to space constraints, we have the full tables with standard errors here: https://bit.ly/42bZt32. \n\n---\n\n> [Q1] how is it handled when the acquisition strategy queries a gene combination that is not present in the dataset? Do you only consider pairs for which data is available during acquisition?\n\n[A1]  We appreciate your point. The datasets used in our work were originally designed to be comprehensive, ensuring that all pairwise gene perturbation combinations were covered. In our simulated active learning framework, the full dataset includes all possible gene combinations. In contrast, during real active learning, each round selects gene pairs that have not been previously measured. These could then be measured through additional biological experiments. We will revise the manuscript to more clearly explain this.\n\n> [Q2] How do RECOVER and GEARS perform on the gene combination-based active learning task?\n\n[A2] Thank you for suggesting a comparison between RECOVER, GEARS and NAIAD in active learning tasks to enhance the benchmark's comprehensiveness. We've added an experiment evaluating RECOVER, GEARS, and NAIAD using MPE on the Norman dataset. The results below, showing the true positive rate (TPR) of the top 200 hits, demonstrate that NAIAD continues to outperform GEARS and RECOVER in active learning rounds.\n\nThe GEARS framework does not have active learning iterations and is not efficient for large combinatorial perturbation. Our active learning framework includes ensembles and multiple replicates, further increasing runtime. Thus, we focused the GEARS benchmark on the Norman dataset. A more comprehensive comparison with RECOVER is provided in [A3] and [A4].\n\n|Model|Method|Round0|Round1|Round2|Round3|Round4|\n|-|-|-|-|-|-|-|\n|NAIAD|MPE|86|100|112|128|143|\n|RECOVER|MPE|82|101|111|125|139|\n|GEARS|MPE|42|80|92|100|109|\n|NAIAD|Uniform|86|88|88|90|94|\n> [Q3] Figure 5 seems to indicate that using MPE leads to NAIAD outperforming RECOVER. To isolate the benefits of using MPE vs. using the NAIAD model, having results from RECOVER + MPE would be useful.\n\n[A3] We appreciate your thoughtful suggestion to disentangle the contributions of the MPE acquisition function and the NAIAD model architecture. In response, we added an experiment using the drug combination dataset, evaluating NAIAD and RECOVER paired with different acquisition strategies: uniform sampling, MPE, and residual-UCB (originally used in RECOVER). The results, summarized in the table showing the TPR of the top 200 hits at round 4, indicate that MPE has a stronger impact on performance than the choice of model architecture. Both RECOVER + MPE and NAIAD + MPE achieve comparable results. However, in the genetic perturbation data, we do see the advantages of both the NAIAD architecture and the MPE acquisition function [comments in A4]. \n\n|Model|Method|Drug Combination|\n|-|-|-|\n|NAIAD|MPE|131|\n|RECOVER|MPE|133|\n|NAIAD|residual_UCB|121|\n|RECOVER|residual_UCB|120|\n|NAIAD|Uniform|120|\n|RECOVER|Uniform|117|\n\n> [Q4] How much improvement comes from using MPE alone? Have the authors tried using MPE along with RECOVER in their benchmarking?\n\n[A4]  Thank you for your insightful comments to further explore the role of NAIAD and MPE in active learning. We included an experiment comparing the performance of RECOVER and NAIAD across all genetic perturbation datasets by calculating the TPR of the top 200 hits at round 4. These results highlight the advantages of both the NAIAD architecture and the MPE acquisition function. The discrepancy between the drug combination and genetic perturbation results is likely due to the relatively small size of the drug combination dataset, which includes only 1,800+ combinations, a substantially smaller number than in the genetic perturbation datasets.\n\n|Model|Method|Norman|Simpson|Horlbeck K562|Horlbeck Jurkat|\n|-|-|-|-|-|-|\n|NAIAD|Uniform|93|70|39|81|\n|RECOVER|Uniform|81|44|37|75|\n|NAIAD|MPE|143|142|100|150|\n|RECOVER|MPE|139|88|54|66|\n|NAIAD|res_UCB|110|103|60|96|\n|RECOVER|res_UCB|85|62|26|49|\n\n> [Q5] In lines 129-130, shouldn't k be the total number of genes for the row vector X_gene  to represent the i th gene's embedding?\n\n[A5] Yes.  You are correct that k should be the total number of genes within our dataset. Thank you for noticing this typo. We’ve fixed the statement to read:\n\n\"Let $X_{\\text{gene}} \\in \\mathbb{R}^{k \\times p} $ be the learnable gene embedding matrix, where $k$ is the number of genes perturbed ….\"\n\n---\nMany thanks to Reviewer kwgr for your professional, detailed, and valuable reviews! We have done our best to address each of your concerns and hope our response can resolve them. We will actively join the discussion until the end of the rebuttal period.", "claims_and_evidence": "The paper is very well-written and claims made throughout the paper are supported by clear and convincing evidence. Additionally, shortcomings and ways to improve on NAIAD are transparently described in the discussion section.", "methods_and_evaluation_criteria": "The method is well-motivated and the benchmarks used are quite comprehensive and meaningful. I have a few suggestions here to help improve the benchmarking:\n- Although the drug combination benchmark shows that NAIAD + MPE outperforms RECOVER + UCB, it would be useful to see active learning results from RECOVER and GEARS in the gene combination benchmarks to more convincingly demonstrate that NAIAD outperforms these methods in the active learning setting.\n- Figure 5 seems to indicate that using MPE leads to NAIAD outperforming RECOVER. To isolate the benefits of using MPE vs. using the NAIAD model, having results from RECOVER + MPE would useful.", "theoretical_claims": "The paper does not make theoretical claims.", "experimental_designs_or_analyses": "Yes, I checked the soundness of all the experiments presented and have mentioned my suggestions above.", "relation_to_broader_scientific_literature": "The main contribution of this paper is the NAIAD modelling framework that can accurately model effects of perturbations in a pair of genes from limited training data by incorporating both additive and interaction effects. The use of adaptive gene embedding sizes to account for differences in training data sizes across active learning iterations is novel to the best of my knowledge. RECOVER (Bertin et al. (2023)) is the closest prior work but their model does not explicitly have an additive effects term to the best of my knowledge (they only seem to have an interactive effects term). RECOVER was also only used for identifying drug combinations in the original paper. Although the MPE acquisition function is very simple, its use in this setting also appears to be novel and the authors demonstrate its effectiveness.", "essential_references_not_discussed": "To the best of my knowledge, relevant prior work has been adequately discussed.", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. In an active learning setting, how is it handled when the acquisition strategy queries a gene combination that is not present in the dataset? Do you only consider pairs for which data is available during acquisition?\n2. How do RECOVER and GEARS perform on the gene combination-based active learning task?\n3. How much improvement comes from using MPE alone? Have the authors tried using MPE along with RECOVER in their benchmarking?\n4. In lines 129-130, shouldn't $k$ be the total number of genes for the row vector $X^{i}_{gene}$ to represent the $i$th gene's embedding?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x7khQqDhTx", "reviewer": "Reviewer_PDeb", "summary": "This work proposes a new active learning framework for optimizing desirable properties in CRISPR screening, which is a combinatorial problem.\nThe main algorithmic contribution comes down to an adaptive training method that scales the embedding dimensionality of the predictive model with the size of the training size, which allows the representation to learn the necessary characteristics of the genes as the active learning loop progresses.\nThe authors conducted experiments to compare the performance of the proposed algorithm against various baselines.\n\n## update after rebuttal\n\nThe authors have sufficiently responded to my questions, and I will increase my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer PDeb,\n\nWe greatly appreciate your thoughtful feedback and questions. Your comments have helped us identify key areas where we could improve the presentation and clarity of our work. We will incorporate the additional results from the experiments in the revised manuscript. Below, we address your questions point by point. Due to space constraints, we have the full tables with standard errors here: https://bit.ly/42bZt32. \n\n---\n> [Q1] Do the authors have any comments on whether other values for β have been tried and what the trend is?\n\n[A1] You raised an important point regarding the role of β in the acquisition function. This prompted us to explore a more generalized formulation of our acquisition function. We now express the acquisition score as a weighted combination:\n score = α × mean + β × std.\nThis unified view captures multiple strategies:\n\nUniform sampling: α = β = 0\n\nUncertainty-only (pure exploration): α = 0\n\nMPE (pure exploitation): β = 0\n\nUCB: α = β = 1 (equal trade-off)\n\nTo better understand how this trade-off impacts performance, we added a new experiment that varies β while holding α = 1, testing β values from 0 to 25 on four genetic perturbation datasets.\n\nWe find that the True Positive Rate (TPR) of top 200 gene combinations remains relatively stable for small β (β ≤ 1). This may result from, in early rounds, the effect magnitude dominating the predictive uncertainty. As β increases, uncertainty begins to dominate and leads to a decrease in TPR.\n|β|Norman|Simpson|Horlbeck K562|Horlbeck Jurkat|\n|-|-|-|-|-|\n|0|138|146|99|152|\n|0.5|138|148|98|152|\n|1|139|145|98|154|\n|5|137|144|81|143|\n|10|133|142|72|132|\n|25|120|124|73|108|\n|Uniform|93|69|39|80|\n\nWe also want to clarify that UCB in RECOVER is defined differently: it uses residuals + β × std, since the model’s objective is to predict residuals from a linear baseline (synergy prediction). We will revise the manuscript to distinguish these UCB formulations more clearly.\nFinally, we expanded our experiments to compare the effectiveness of MPE and residual-UCB strategies across both NAIAD and RECOVER. The results are in our response to Reviewer kwgr [see A4]. Consistently, MPE outperforms residual-based UCB methods, further supporting our conclusion.\n\n> [Q2] Is embedding dimensionality scaled at every active learning step? Given the expensive cost of retraining multiple networks, have the authors evaluated performance versus ensemble size?\n\n[A2]  You're correct that we increase the embedding size during the active learning iterations. Retraining the ensemble from scratch each round is expensive (typically hours). However, this remains fast compared to the time and cost of biological experiments (weeks per cycle). \n\nYour suggestion to test the impact of ensemble size was valuable. We performed an experiment varying the number of ensemble models. The results indicate that increasing the ensemble size can improve performance under uniform sampling. In contrast, the MPE acquisition function demonstrates robust performance regardless of ensemble size.\n|Ensemble Size|Norman||Simpson||Horlbeck K562||Horlbeck Jurkat||\n|-|-|-|-|-|-|-|-|-|\n||MPE|Uniform|MPE|Uniform|MPE|Uniform|MPE|Uniform|\n|1|141|89|150|67|106|34|153|80|\n|5|141|91|143|68|104|37|151|81|\n|7|143|92|143|71|102|39|152|81|\n\n> [Q3] How scalable for the number of perturbations to be larger than 2? Does it take exponentially more effort to extend this framework?\n\n[A3] Thank you for highlighting this valuable point. Our framework extends efficiently to fit combinations with more than two gene perturbations. We will revise the manuscript to provide a more detailed explanation.\nFor a dataset with a higher-order combination involving p perturbations, we define the set $S=\\\\{i_1,i_2,…,i_p\\\\}$. We model the effect of jointly perturbing the genes in $S$ as:\n$$Y_{S} = f_1([Y_{i_1}, Y_{i_2}, … Y_{i_p}]W_1)A^T + \\sum_{i \\in S} f_2(W_2X^i_{gene})A^T_2$$\nHere, $Y_{i_1}, Y_{i_2}, \\dots Y_{i_p} $ represent the individual gene effects, while the interaction term $\\sum_{i \\in S} f_2(W_2X^i_{gene})A^T_2$ captures the high-order genetic interaction effect. We leverage a permutation-invariant mechanism for combining individual gene embeddings through summation, allowing the model to efficiently generalize across different lengths of perturbation sets.\n\n> [Q4]  Is there a reason why the different curves start off at different points in the plots?\n\n[A4] Thanks for pointing this out. In Figure 4, the x-axis should start at “5,” the minimum TPR for the top 5 gene combos. Since strategies choose different samples after round one, the curves diverge from there. We'll update the figure by removing “0” and marking the start as “5.”\n\n---\nThank you for your thoughtful and constructive feedback. It has helped strengthen and refine our work. If you have any further questions, we’d be happy to address them. If you feel all concerns have been resolved, we kindly invite you to consider re-rating your evaluation. Thank you.", "claims_and_evidence": "Yes, the claims are appropriately backed by the experiments.", "methods_and_evaluation_criteria": "Yes, the proposed methods and evaluation make sense.", "theoretical_claims": "There are no theoretical claims.", "experimental_designs_or_analyses": "Yes, the experimental designs are appropriate.\nThe analyses could be extended to get a better insight into what makes the active learning policy works as well as the scalability of the method.\nFor example, as I understand, MPE is a version of UCB that does not explore (setting the tradeoff parameter $\\beta = 0$, simply optimizing the predicted mean).\nIf so, I find it interesting that this version outperforms UCB with a non-zero $\\beta$.\nDo the authors have any comments on whether other values for $\\beta$ have been tried and what the trend is.\n\nIn terms of scaling the dimensionality of the embedding with the training data size, do we do this at every step of the active learning loop?\nThis seems to be quite an expensive endeavor, as we need to train multiple deep neural networks (since an ensemble is used) every time new data comes in.\nOn this note, have the authors considered how performance changes as a function of the size of the ensemble?\n\nFinally, could the authors provide some discussion regarding how scalable for the number of perturbations to be larger than 2? Does it take exponentially more effort to extend this framework?", "relation_to_broader_scientific_literature": "The proposed method is a new active learning framework for optimizing over gene combinations which outperforms baselines from previous works.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "Is there a reason why the different curves start off at different points in the plots?", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces NAIAD, an active learning framework for discovering optimal gene combinations using a neural surrogate model, adaptive gene embeddings, and a maximum predicted effect acquisition strategy. The work is motivated by the need for efficient exploration of large combinatorial perturbation spaces in gene and drug studies. The authors frame the task within established machine learning paradigms such as surrogate modeling and active acquisition, and they provide comprehensive empirical validation across multiple datasets.

The reviewers offer mixed but mostly positive evaluations:

- Reviewer kwgr gives a strong accept and highlights both the novelty and the empirical strength of the approach, especially the dynamic scaling of embedding dimensionality during training and the simplicity and effectiveness of the MPE acquisition function. 

- Reviewer PDeb raises several technical questions in the initial review but acknowledges that the authors addressed them thoroughly in the rebuttal, including with new experiments. 

- Reviewer vQva indicates limited familiarity with the topic and did not substantively engage during the discussion period, which limits the weight of that review.

The main contributions of the paper are empirical and systems-oriented rather than theoretical. The combination of adaptive embeddings and greedy acquisition is conceptually straightforward but is implemented thoughtfully and validated convincingly. The authors demonstrate that their approach outperforms existing baselines, and the empirical analysis is thorough, including ablation studies and robustness checks.
Decision: Accept (poster)