Identifying biological perturbation targets through causal differential networks
Authors: Menghua Wu
Umesh Padia
Sean H. Murphy
Regina Barzilay
Tommi Jaakkola
Venue: ICML 2025 poster (2025)
Keywords: perturbation
transcriptomics
causality
Primary area: applications->health_medicine
TL;DR: predict root causes of change in biological systems, based on differences between inferred causal graphs

Abstract: Identifying variables responsible for changes to a biological system enables applications in  drug target discovery and cell engineering. Given a pair of observational and interventional datasets, the goal is to isolate the subset of observed variables that were the targets of the intervention. Directly applying causal discovery algorithms is challenging: the data may contain thousands of variables with as few as tens of samples per intervention, and biological systems do not adhere to classical causality assumptions. We propose a causality-inspired approach to address this practical setting. First, we infer noisy causal graphs from the observational and interventional data. Then, we learn to map the differences between these graphs, along with additional statistical features, to sets of variables that were intervened upon. Both modules are jointly trained in a supervised framework, on simulated and real data that reflect the nature of biological interventions. This approach consistently outperforms baselines for perturbation modeling on seven single-cell transcriptomics datasets. We also demonstrate significant improvements over current causal discovery methods for predicting soft and hard intervention targets across a variety of synthetic data.

Identifying biological perturbation targets through causal differential networks
Menghua Wu! Umesh Padia Sean H. Murphy!
Regina Barzilay! Tommi Jaakkola 1
Abstract
variables, identifying the root causes, or targets, that induce these transitions has vast therapeutic implications, from cel- lular reprogramming (Cherry & Daley, 2012) to mechanism of action elucidation (Schenone et al., 2013).
Identifying variables responsible for changes to a biological system enables applications in drug tar- get discovery and cell engineering. Given a pair of observational and interventional datasets, the goal is to isolate the subset of observed variables that were the targets of the intervention. Directly applying causal discovery algorithms is challeng- ing: the data may contain thousands of variables with as few as tens of samples per intervention, and biological systems do not adhere classical causality assumptions. We propose causality- inspired approach to address this practical setting. First, we infer noisy causal graphs from the obser- vational and interventional data. Then, we learn to map the differences between these graphs, along with additional statistical features, to sets of vari- ables that were intervened upon. Both modules are jointly trained in supervised framework, on simulated and real data that reflect the nature of bi- ological interventions. This approach consistently outperforms baselines for perturbation modeling on seven single-cell transcriptomics datasets. We also demonstrate significant improvements over current causal discovery methods for predicting soft and hard intervention targets across variety of synthetic data.'
Machine learning methods have primarily been designed to infer the effects of perturbations on cells, with the goal of generalizing to unseen perturbations (Roohani et al., 2023), unseen cell distributions (Lotfollahi et al., 2023; Bunne et al., 2023). In principle, these models can be used within an active learning framework to discover intervention tar- gets (Huang et al., 2024; Zhang et al., 2023a). However, the number of candidate sets scales exponentially with the number of variables under consideration, and there are lim- ited training data for combinatorial perturbations (< 150 pairs in Norman et al. (2019)). Thus, this option may be impractical for larger search spaces.
Alternatively, some methods predict targets by comparing pairs of "before" and "after" cell states, similar to this paper. They do so by attributing observed differences to sparse, mechanistic changes. Specifically, relationships between variables are represented through a graph, e.g. a gene reg- ulatory network (Babu et al., 2004). Existing methods of- ten use data-mined relations as priors for this graph, and search for modifications to the set of edges that best ex- plain the interventional data (Cosgrove et al., 2008; Gon- zalez et al., 2024). However, biological knowledge graphs are incomplete and noisy priors. They contain heteroge- neous, potentially conflicting information, since the reported findings were observed under diverse contexts (Ashburner et al., 2000). Moreover, different cell populations may ex- hibit distinct gene regulatory behavior (Huttlin et al., 2021), leading to incomplete knowledge, especially of less well- characterized systems.
1. Introduction
Cells form the basis of biological systems, and they take a multitude of dynamical states throughout their lifetime. In addition to natural factors like cell cycle, external per- turbations (e.g. drugs, gene knockdown) can alter a cell's state. While perturbations can affect numerous downstream
In this paper, we propose causal differential networks (CDN): causality-inspired approach to identify variables that drive desired shifts in cell state, while estimating their mechanis- tic structure directly from data. Given a pair of observa- tional and interventional datasets, we train causal structure learner to predict causal graphs that could have generated each dataset. The pair of graphs is input to an attention- based classifier, which predicts whether each variable was subject to intervention. To address the challenges of data
DDpartment of Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA. Correspondence to: Menghua Wu <mwu @ mit.edu..
Proceedings of the 42nd International Conference on Machine I Leanning Vancouver, Canada. PMI.R 267, 2025. Copyright 2025 by the author(s).
Code available at htt.ss://gitteeet thubbbb.//tttbb///// com/ rmwu/ cdn

---

Identifying biological perturbation targets through causal differential networks
noise and sparsity, and to simulate the structure of biological interventions, these models are trained jointly in super- vised (amortized) framework (Ke et al., 2023; Petersen et al., 2023), over thousands of synthetic or real datasets.
cell lines, as certain genes or pathways are essential for survival. However, the empirical discrepancy can be quite large between experiments. For example, Nadig et al. (2024) reports that the median correlation in log-fold change of genes under the same perturbation, across two experiments in the same cell line, by the same lab, was 0.16.
We evaluate CDN on real transcriptomic data and synthetic settings, CDN outperforms the state-of-the-an in perturba- tion modeling (deep learning and statistical approaches), evaluated on the five largest Perturb-seq datasets at the time of publication (Replogle et al., 2022; Nadig et al., 2024) without using any external knowledge. Furthermore, CDN generalizes with minimal performance drop to unseen cell lines, which have different supports (genes), causal mech- anisms (gene regulatory networks), and data distributions. On synthetic settings, CDN outperforms causal discovery approaches for estimating unknown intervention targets.
Early efforts to map the space of cells measured the dis- tribution of gene levels as a surrogate for cell state (Lamb et al., 2006). Diseases, drugs, or genetic perturbations that induce similar changes in this distribution were thought to act through similar mechanisms. However, it is intractable to map all combinations of starting cell states and perturba- tions through experiment alone. This bottleneck has led to two directions in the in-silico modeling of perturbations.
To reduce experimental costs, which scale with the number of cell lines and perturbations, one option is infer the post- intervention distribution of cells. That is, given Dobs ~ Px and I, the goal is to infer Px. Works such as scGen (Lot- follahi et al., 2019), CPA (Lotfollahi et al., 2023), Chem- CPA (Hetzel et al., 2022). and CellOT (Bunne et al., 2023) aim to generalize to new settings Px, e.g. unseen cell lines or patients. Alternatively, models like GEARS (Roohani et al., 2023), graph (Wu et al., 2023), AttentionPert (Bai et al., 2024), and GIM (Schneider et al., 2024) infer the ef- fects of unseen perturbations. This setting is also a common application for single cell foundation models (Theodoris et al., 2023; Cui et al., 2024; Hao et al., 2024). These latter methods generalize by modeling the relationships between seen and unseen perturbations. For example. GEARS lever- ages literature-derived relationships in the Gene Ontology knowledge graph (Ashburner et al., 2000), and graph VCI learns to refine the gene regulatory relationships implied by ATAC-seq data (Grandi et al., 2022).
2. Background and related work
2.1. Problem statement
We consider a system of N random variables X; E X, whose relationships are represented by directed graph G = (V, E), where nodes i E V map to variables X,, and edges (i,j) E to relationships from X, to Xj. Samples from the data distribution Px are known as observational. We can perform interventions by assigning new conditionals
P(X: Xx,) < P(X: Xxx);
(1)
where T. denotes the parents of node i in G. Hard inter- ventions remove all dependence between X; and ni, while soft interventions maintain the relationship with a different conditional. We denote the joint interventional distribution as Px. Given an observational dataset Dobs ~ Px and an interventional dataset Dint Px, our goal is to predict the set of nodes I for which
Approaches for inferring the effects of interventions do not explicitly predict targets, but if the space of perturbations I can be efficiently enumerated, these models are useful for comparing different I (Zhang et al., 2023a; Huang et al., 2024). For example, given an oracle Φ Dats, I - P, the I' associated with P is
P(X; xx)) + P(X, X..,
(2)
In the context of molecular biology, a perturbation is a hard or soft intervention, and / is the set of perturbation targets,
where ||]| 2 1.
I = min d( (DDDne,) I)) IEI
(3)
2.2. Modeling perturbations
for some dissimilarity d. However, recent works have re- ported that the predictive ability of current models does not exceed that of naive baselines (Kernfeld et al., 2023; Ahlmann- Eltze et al., 2024; Märtens et al., 2024), reflecting that phenotype prediction is currently unsolved.
Perturbation experiments result in datasets of the form Dobs: {(Ix, ((I, }. where Debs ~ Px, D ~ Pt. In this paper, we focus on the transcriptomic data modality, where variables X; represent the levels of gene i; the number of variables N ranges from hundreds to thousands; Dots con- tains thousands of samples; and each Dim contains tens to hundreds of samples (Replogle et al., 2022).
Perturbation datasets can also be used to train models for predicting targets of interest, as is the focus of this work. A common strategy is to relate observed changes (c.g. discase, gene levels) to mechanistic explanations (e.g. pathways), in which the targets are involved. To draw this connec- tion, current approaches use external information such as
Each experiment is conducted in a single cell line, which represents a distinct set of variables X, distribution Px and graph G. These three objects may share similarities across
2

---

Identifying biological perturbation targets through causal differential networks
Unobserved changes to data- generating mechanisms result in observable differences in data.
If causal discovery algorithms be made scalable and data-efficient, their outputs can provide signal for predicting perturbation targets.
Starting cells
Target cells
Cell state engineering: Perturbations that induce desired cell states.
G
obs
ast
D obs
Inferred G obs
2
3
Diseased cells
Treated cells
Elucidating mechanism of action: Protein largets of drugs.
Predicted targets
D
D
D Inferred G
Data size: 10²-10² observations 10²-10* variables
(A) Perturbation target prediction
(B) Causal interpretation
(C) iausalit approach
Figure 1: (A) Biological applications. (B) Differences between observation dataset Dats and interventional dataset Das can
be attributed to changes in the underlying causal mechanisms. (C) We use an amortized causal discovery model to predict Gats and Gim, whose hidden representations are input a perturbation target classifier. Both modules are jointly trained.
knowledge graphs (Cosgrove et al., 2008; Gonzalez et al., 2024) and natural language (Roohani et al., 2025). However, these information are collected from highly inhomogenous sources (Ashburner et al., 2000), and it is difficult to quan- tify their correctness and completeness in each new setting. An alternative is to infer the mechanistic structure directly from data, either explicitly (di Bernardo et al., 2005) or as a model parameter (Noh & Gunawan, 2016). In this work, we follow the latter strategy, while allowing for richer (non-lineary class of relationships.
sumes linearity, while the latter requires least as many environments as ||]] (we only consider before and after per- turbation). Finally, several works aim infer differences between causal mechanisms (edges), rather than identify- ing the full causal structure (Wang et al., 2018; Belyaeva et al., 2021; Chen et al., 2023; Malik et al., 2024). Our work is also motivated by the idea that differences between summary statistics (e.g. regression coefficients in DC1) are informative of changes between systems. Thus, CDN can be viewed as implementing similar operations within an end-to-end, supervised framework.
2.3. Causal discovery
2.4. Amortized causal inference
The task of inferring causal mechanisms from data is known as causal discovery, causal structure learning (Spirtes al., 2001). In the simplest setting, the goal is infer G' from Dass The earliest works gorithms, which operated over the combinatorial space of potential graphs (Spirtes et al., 1995; Tian & Pearl, 2001; Chickering, 2002; Shimizu et al., 2006). More recently, a number of works have reframed the task as a constrained optimization over continuous adjacency matrices (Zheng al., 2018; Lachapelle al., 2020; Sanchez et al., 2023).
While the majority of causal discovery algorithms must be fit run from scratch for each data distribution, amortized causal discovery algorithms have addressed this task supervised machine learning problem. Using large numbers of synthetic datasets, generated by synthetic graphs, a neural network trained to map D directly to G (Li et al., 2020; Ke et al., 2023; Lorch et al., 2022: Petersen et al., 2023; Wu et al., 2025). Compared to traditional approaches, these algorithms trade explicit assumptions for implicit character- istics of the data simulation process (Montagna et al., 2024). These models are also fast and robust to noise. In this work, we simulate interventions that reflect biological perturba- tions, and our model architecture builds upon that of Wu et al. (2025), as scales easily to hundreds of variables.
discrete search al-
If data from multiple environments are available, they can provide additional identifiability regarding the system (Jaber et al., 2020; Zhang et al., 2023b). Differences between these environments, c.g. intervention targets I, are either given as input or part of the inference task. As the observational case, discovery algorithms can be categorized into discrete search (Hauser & Bühlmann, 2012; Ghassami et al., 2018: 8: Ke et al., 2019; Huang al., 2020; Mooij et al., 2020) or continuous optimization (Brouillard et al., 2020; Hägele et al., 2023). Additionally, Varici et al. (2022) and Yang et al. (2024) frame intervention target prediction as its own task, similar this paper. However, the former strictly as-
3. Methods
We introduce causal differential networks (CDN), a causality inspired method for predicting intervention targets. On high level, CDN is motivated by the idea that differences in observations Dat and Dim can be explained by the changes
3

---

Identifying biological perturbation targets through causal differential networks
to their underlying causal mechanisms (Figure IB).
pre-layer normalization on each self-attention, followed by dropout and residual connections:
Our model is composed of two modules: a causal structure learner (Section 3.1), and a differential network classifier (Section 3.2). Given a single dataset D, the causal structure learner predicts a putative G. The pair of datasets Dabs and Dim yields graphs Gobs and Gint. whose features are input to the differential network, to predict targets I (Figure IC). Both modules are trained jointly, supervised by classifi- cation losses for the graph and predicted targets. Finally, individual samples are known to be noisy in single-cell biol- ogy and are often analyzed in "pseudo-bule 2003). Thus, instead of operating over raw data, we also represent datasets in terms of their summary statistics.
h <- h + where h E R= x denotes an intermediate hidden repre- sentation. This design takes after Transformer models for higher dimensional data other domains, e.g. Axial Trans- formers for videos (Ho et al., 2019) and MSA Transformers for aligned protein sequences (Rao al., 2021). A key difference is the graph modality: when making edge or node-level predictions, the model should be equívariant to node labeling. This is accomplished by adding randomly- permuted "positional" embeddings to each node and edge representation. For more details, please see Appendix A.
(4)
(Blake et al.,
3.1. Causal structure learner
Edge-level outputs Let /im E N xd denote the final layer representations of each dataset, after pooling over the T dimension. These representations are input to the differential network. In addition, we introduce graph prediction head,
The causal structure learner takes as input dataset D and outputs the predicted G. In principle, this module can be implemented with any data-efficient causal discovery model. Here, we based the architecture on SEA (Wu et al., 2025), as its runtime complexity does not explicitly depend on the number of samples (which can be large for Dona).
Z1 = F..=FNN(hh,,, P(E., = o
(logits)
(5)
(softmax)
(6)
Input featurization We represent dataset D through two types of statistics. First, we compute global statistics P E RR- (pairwise correlation) over all variables. This statistic an inexpensive way for the model to filter out edges that are not likely to occur. Second, we obtain esti- mates of local causal structure to orient edges, by running classical causal discovery algorithms over small subsets of variables. Specifically, we run the FCI algorithm T times over subsets of k variables (Spirtes et al., 1995), which are sampled proportional to their pairwise correlation. We de- note the set of estimates as E' clxkxT, where E is the space of possible edge types inferred by FCI (e.g. ++,,,+) The size and number of local estimates, k and T, are hyper- parameters that trade off accuracy vs. runtime (Section 3.3).
where denotes softmax over the set of output edge types (forwards, backwards, loss LG is simply the cross entropy loss between the pre- dicted and true edge types, summed over the entire graph.
edge). The graph prediction
3.2. Differential network
Given a pair of graph representations hohs, Aim, the differen- tial network predicts 1, the set of intervention targets.
Input featurization h, we compute node-level statistics (mean, variance), which are mapped to RRd via a feed-forward network, and concate- nated along "incoming edge" dimension. This results in graph representation h' E R [NNxxxd)
In addition to pairwise representations
To compare P and E' in the same space, we linearly project elements of from IR to Rdd and map categorical edge types into Ed using a learned embedding, i.c. like token embed- dings in language models (Devlin et al., 2019). Thus, the dataset D is summarized into matrix of size TR NxNNXTxx
We combine the graph representations either through con- catenation or subtraction:
heat = [M Vaml E R. Nx(N+1)22
(7)
haif h/m 1/es E R Nx
(8)
Model architecture Both the causal structure learner and differential network are implemented with a series of attention- based blocks. Since scaled dot-product attention scales quadratically as sequence length (Vaswani et al., 2017), we attend over one dimension at a time. For ex- ample, given 222 entries in an adjacency matrix, naive self-attention would cost O(N*) operations. Instead, we apply self-attention along all nodes in the "outgoing edge" direction, with the "incoming edges" as a batch dimension,
The former can express richer relationships between the two graphs, while the latter is more efficient on large graphs.
Model architecture The differential network closely fol- lows the architecture of the causal structure learner. How- ever, since paired graph representation lacks the T dimen- sion (introduced by marginal estimates E'), the model only needs to attend over two "length" dimensions (rows and columns of adjacency matrix). After the attention layers, we mean over all "incoming edges" and linearly project to bi- nary node-level predictions, where a normalized indicates an intervention target, and 0 indicates otherwise.
followed by the opposite- resulting in two operations, each of cost O(N x N2). Following Rao et al. (2021), we use
4

---

Identifying biological perturbation targets through causal differential networks
NXNXd
Causal structure learner
vars,
M obs.
Attention Layers
Embed
FFN
E'
Graph loss X
Dataset D
Edge-level statistics
Graph features /
Predicted graph
True graph
Observational D oba
Differential network
Causal structure learner
Node-level statistics
FFN
Attention Layers
FFN
Interventional D
Target loss L,,
Causal
structure
(N+1))
N NX4 d
learner
Aggregate over incoming edges
Predicted targets
True
Concatenate subtract
Node-level
targets
FFN
statistics
Our model is composed causal structure learner and a differential network. Learned components in yellow (causal structure learner) and green (differential network). Both modules are trained jointly to optimize L LG Cy.
Figure 2:
The target prediction loss Cy the binary cross entropy loss between the predicted and true targets, computed indepen- dently for each node. This is to accommodate interventions with multiple targets, c.g. drugs with off off-target effects.
with hard and soft interventions, which vary in causal mech- anism, intervention type, and graph topology. To emulate transcriptomics data, in which perturbation effect sizes quantified in (log) fold change, we introduced soft interven- tions: "shift" +- f(T2) + C, where E R, and "scale" x + cf(mx), where c is positive scaling factor, with equal probability e $ 1. We also sampled interventions with multiple targets, to simulate off-target effects. Details are available Section 4.2 and Appendix D.2.
Remark We take an amortized causal discovery approach because it is hard to specify what assumptions biologi- cal system ought to follow. This makes difficult to pro- vide identifiability guarantees without simplifying aspects of either the data or modeling framework. Instead, in Ap- pendix C, we show that the differential network well- specified class of models. That is, there exist parametriza- tions of its attention architecture that map predicted graphs and global statistics (for hard and soft interventions) to the set of targets.
Model parameters Marginal estimates E' computed using the classic FCI algorithm (Spirtes et al., 1995) with k 5 and T 100, and global statistic p is pairwise correlation, as it is fast to compute and numerically stable. We swept over the number of differential network layers (Figure 4) on synthetic data, and we used layers for ham and 2 layers for han- Following SEA, adopted hidden dimension
3.3. Implementation details
64, the AdamW optimizer (Loshchilov & Hutter, 2019), learning rate le-4, batch size 16, and weight decay le-5. On the real data, where N = 1000, we changed to a batch size of 1, decreased the learning rate to 5e-6, and finetuned the models with half precision (FP16). We trained both heat and hant models on synthetic data, but only finetuned the har architecture on the real data (memory constraint).
Training procedure The overall training loss is L La + L, with 1.2 weight regularization. Both LG and L, supervise causal structure learner, while only the latter supervises the differential network. We use the synthetic and real datasets as follows.
1. Initialize the causal structure learner with pretrained weights from Wu et al. (2025) (except ablations).
2. Train both modules jointly on synthetic data.
3. Finetune both modules jointly with only C, (no ground truth graphs), on each training split of the real datasets.
4. Experiments
While our method is intended to be used in cases where true targets are unknown, we can only evaluate on datasets
For the synthetic training data, generated 8640 datasets
5

---

Identifying biological perturbation targets through causal differential networks
where the target is known. Thus, we demonstrate that CDN can recover perturbation targets on seven transcriptomies datasets, with comparisons to state-of-the-art models for these upplications (Section 4.1). For completeness, we also benchmarked CDN against multiple causal discovery algo- rithms for unknown interventions in variety of controlled settings (Section 4.2).
which the intended target was not trivially identifiable as the gene with the largest log-fold change.
While perturbation target prediction appears to be simple classification task, standard metrics are not well-suited for these real data. Not all genes are present in the baselines' domain knowledge (e.g. isolated node in graph), so their effects as perturbations cannot be predicted. In addition, due to genetic redundancy, it is common for multiple per- turbations to elicit similar responses (Kernfeld et al., 2023). An "incorrect" top 1 prediction may not necessarily reflcct poor performance. Thus, we propose the following metrics.
4.1. Biological experiments
Datasets We validate CDN on five Perturb-seq (Dixit et al.,
2016) datasets (genetic perturbations) from Replogle et al. (2022) and Nadig et al. (2024); as well as two Sci-Plex (Sri- vatsan et al., 2020) datasets (chemical perturbations) from FiFulinn-Figueroa et al. (2024). Each dataset is real- valued matrix of gene expression levels: the number of ex- amples M is the number of cells, the number of variables N is the number of genes, and each entry is a log-normalio nognooeml count of how many copies of gene j was measured from cell In Perturb-seq datasets, we aim to recover the gene whose promoter was targeted by the CRISPR guide, and in Sci- Plex datasets, we aim to identify the gene that corresponds to the drug's ss intended target (discussion in Appendix D.
We order genes based on the similarity of their pre- dicted effect to the ground truth target effect, or based on their predicted probability of being a target. The rank is the index of the true target, normalized by the number of candidate genes. Rank ranges from 0 (worst, bottom of list) to (best, top of list).
We measure the similarity between the perturbation ef- feet (log-fold change) of the top I predicted target and the true target with the Spearman rank correlation p and the Pearson correlation r. These range from -1 to 1, where is best.
Biologically, many perturbations do not affect the cell's production of mRNA (they may affect other factors like protein activity, which transcriptomies does not measure). To ensure that Poss and Pin are distinct, we filtered perturba- tions to those that induced over 10 differentially expeessed genes (statistically significant change, compared to control), of which the true target should be present. This excludes perturbations with insufficient cells (low statistical power), with minimal to no effect (uninteresting), and those that did not achieve the desired effect (low CRISPR efficiency). In the Sci-Plex data, several drugs have known off off-targets, but we did not observe differential expression of any of them (adjusted p-value 1).
To emulate "virtual screening," " we plot recall at p: the fraction of targets recovered within the p E (0, 1) candidates. Recall ranges from 0 to I, where is best. Rank is equivalent to the expected recall at p.
Baselines We compare to both algorithms that infer the effects of perturbations on cells (used scoring functions, Equation 3), and algorithms that directly predict targets. All baselines were run with their official implementations and/or latest releases. For details, please see Appendix B.2.
GEARS (Roohani et al., 2023) and GENEPT (Chen & Zou, 2023) predict the effects of unseen genetic perturbations. GEARS is graph neural network that regresses log-fold change upon perturbation, using the Gene Ontology knowl- edge graph as an undirected backbone. GENEPT is a set of large language model-derived embeddings that have been shown to achieve state-of-the-art performance (Märtens et al., 2024) with simple downstream models like logistic regression (used here). Both are trained per cell line.
We consider two splits: seen and unseen cell lines. In the former, models may be trained on approxi- mately half of the perturbations from each cell line, and are evaluated on the unseen perturbations. In the latter, we hold out one cell line at a time, and models may be trained on data from the remaining cell lines. To ensure that our train and test splits are sufficiently distinct, we cluster perturba- tions based on their log-fold change and assign each cluster to the same split (Figure 5). Since the Sci-Plex data are limited, we use them only as test sets without finetuning.
Evaluation
PDGRAPHERR (Gonzalez et al., 2024) is a graph neural network that predicts the perturbation targets of genetic or chemical perturbations, on seen or unseen cell lines, with the Human Reference Interactome (Luck et al., 2020) as the knowledge graph. To the best of our knowledge, PDGRA- PHER is the only published deep learning baseline for the target prediction task on transcriptomic data. Therefore, we compare against additional statistical and naive baselines.
We limited the set of candidate targets to the top 1000 dif- ferentially expressed genes by log-fold change, per pertur- bation. If too few genes passed the significance threshold (p < 0.05), we added additional candidates with the largest log-fold change until we reached minimum of 100 genes. Genetic perturbations have highly specific effects compared to chemical perturbations, so we focused on the subset for
CCDN predicts trivial perturbations nearly perfectly (Table 9).
6

---

Identifying biological perturbation targets through causal differential networks
DGE stands for differential gene expression (Love et al., 2014), i.e. the process of identifying statistically significant changes in genes between settings. Specifically, we run the Wilcoxon ranked-sum test with Hochber correction (Wilcoxon, 1945; Benjamini & Hochberg. 2000) between control and perturbed cells, as implemented by Wolf et al. (2018). Genes are ranked by adjusted p-value. LINEAR and MLP take as input the mean expression of all perturbation targets, plus the top 2000 highly-variable genes (Wolf et al., 2018). They are trained to predict binary label for each gene on each cell line independently.
sampled Erdös-Rényi graphs with N = 10, 20 nodes and F 2N 2N expected edges; causal mechanism parameters; and observations of each variable, in topological order. We sampled 3N distinct subsets of 1-3 nodes (N each) as in- tervention targets. Hard interventions set x <- 2, where 2 is uniform. For soft interventions (Section 3.3), we trained on shift and tested on scale for synthetic experiments (see Table 10 for explanation). Real experiments trained on all interventions.
Baselines We compare against discrete and continuous causal discovery algorithms for unknown interventions. UT- IGSP (Squires et al., 2020) infers causal graphs and un- known targets by greedily selecting the permutation of variables that minimizes their proposed score function. DCDI (Brouillard et al., 2020) and BACADI (Hägele et al., 2023) are continuous causal discovery algorithms that fit generative models to the data, where the causal graph and intervention targets are model parameters. DCDI's -G and DSF suffixes correspond to Gaussian and deep sigmoidal flow parametrizations of the likelihood. BACADI'S -E and - suffixes indicate empirical (standard) and mixture (boot- strap) variants.
Results CDN consistently outperforms all baselines on the five Perturb-seq datasets, in both the seen and unseen cell line settings (Table 1). There is minimal drop in be- tween seen and unseen cell lines, suggesting that the func- tional characteristies of genetic perturbations are highly transferrable, even to different data distributions. Notably, no other deep learning method exceeds simple statistical tests (DGE). This is more evident in Figure 3, in which CDN achieves higher recall at at nearly all points. Finally, if we ablate the finetuning step ("synthetic"" vs. "Perturb-seq"), we find that finetuning leads to significant improvements.
Chemical perturbations are much less specific than genetic perturbations (Figures 6 and 7), and it is less clear whether drug effects should be reflected in mRNA levels (via feed- back mechanisms), as drugs act upon their protein products. Still, on the two chemical perturbation datasets, CDN finds the true target within the top 100 candidates for 3/6 cases (Table 2). Due to the low performance of DGE on several cases, we hypothesized that gene-level changes may not be as predictive. Indeed, an ablation of CDN, trained without nove-level statistics, improves where DGE fails ("no p²).
MB+CI is inspired by Huang et al. (2020). We introduce domain index (indicator of environment), compute its Markov boundary using graph lasso, and use conditional mu- tual information to identify true children of the domain index (intervention targets). DCI with stability selection (Belyaeva et al., 2021) predicts edge-level differences between two causal graphs. We take each node's 's proportion of changed edges as its likelihood of being an intervention target. While DCI was also motivated by biological applications, it only scales to around a hundred variables at most, so we evaluate it alongside other causal discovery methods here.
4.2. Synthetic experiments
Evaluation In the synthetic case, we are not constrained by biological redundancy or incomplete predictions, so we re- port standard classification metries: mean average precision (mAP) and area under the ROC curve (AUC). Both metrics are computed independently for each variable and averaged over all regimes of the same number of targets. Their values range from 0 to (perfect). The mAP random baseline de- pends on the positive rate, while the AUC random baseline is 0.5 (per edge).
While it would be ideal to evaluate all algorithms real data, current causal discovery algorithms that support un- known interventions are not tractable on datasets with more than tens of variables. In fact, many baselines require hours on even N = 10 datasets, and they do not scale favorably (Table 6). Our transcriptomics datasets contain hundreds of genes, even after filtering to those that differentially expressed (Figure 8). At the same time, existing models for biological perturbations all rely on some form of domain knowledge, and their performance is inseparable from the choice and quality of these external data. Synthetic data al- low us to assess the model's capacity to predict intervention targets in isolation.
Results On synthetic data, CDN achieves high performance across intervention types and data nata-generating mechanisms (Table 3), while running in seconds (Table 6). As an ab- lation study, we investigated removing the graph loss and the pretrained weights, with no change to the graph-centric architecture. We found that supervising based on the graph is almost always helpful ("no LG" row), and pretraining leads more stable training dynamics (Figure 10).
Datasets Each evaluation setting consists of 5 unseen, i.i.d. sampled graphs. To assess the generalization capacity of our framework, we tested the model on seen (linear) and unseen causal mechanisms (polynomial, sigmoid), and un- seen soft interventions. To generate observational data, we
DC1 and MB+C1 are the best basclines, which is encourag-
7

---

Identifying biological perturbation targets through causal differential networks
Table 1: Results on 5 Perturb-seq datasets, Top:
Leave one cell line out of training. Test sets (unseen perturbations) are identical in both settings. CDN (synthetic) and DGE do not require training on real Perturb-seq data. Metries, from left to right: normalized rank of ground truth, top Spearman and Pearson correlations.
Train on all cell lines jointly. Bottom:
Uncertainty quantification in Table 7. Runtimes in Table 11.
K562 (gw)
K562 (es)
RPEI
HepG2
Jurkat
Setting Model
rank
0
r
rank
P
rank
P
rank
P
rank
p
LINEAR
0.51
0.18
0.23
0.48
0.22
0.25
0.49
0.42
0.52
0.51
0.35
0.43
0.50
0.20
0.23
MLP
0.45
0.18
0.21
0.49
0.08
0.11
0.53
0.32
0.41
0.48
0.33
0.40
0.45
0.24
0.29
Seen
GENEPT
0,52
0.29
0.32
0.42
0.16
0.19
0.41
0.37
0.46
0.31
0.32
0.40
0.51
0.31
0.36
cell line
GEARS
0.56
0.19
0.22
0.45
0.18
0.20
0.49
0.35
0.44
0.54
0.32
0.40
0.50
0.22
0.26
PDG
0.54
0.25
0.29
0.49
0.20
0.23
0.48
0.41
0.52
0.54
0.37
0.43
0.52
0.31
0.35
CDN (Perturb-seq)
0.92
0.72
0.74
0.95
0.81
0.82
0.92
0.67
0.76
0.94
0.69
0.75
0.98
0.83
0.84
DGE
0,82 0.50
0.54
0.89
0.60
0.62
0.79
0.56
0.64
0.86
0.51
0,51
0.84
0.45
0.50
Unseen
PDG
0.39
0.10
0.13
0.44
0.24
0.28
0.54
0.38
0.48
0.48
0.34
0.41
0.57
0.31
0.35
cell line
CDN (synthetic)
0.79
0.46
0.49
0.80
0.59
0.62
0.82
0.61
0.71
0.84
0.56
0.63
0.87
0.64
0.67
CDN (Perturb-seq)
0.91
0.69
0.71
0.95
0.81
0.82
0.92
0.65
0.74
0.93
0.67
0.72
0.97
0.84
0.85
K562 (gw)
K562 (es)
RPE1
HepG2
Jurkat
CDN (P)
1.0
CDN (S)
Recall
PDG
0.5
GEARS
GenePT
DGE
0.0
0
0.5
0
0.5 top p
1
0.5
1 0
0.5
1 0
0.5
Linear
MLP
top P
top p
top p
top p
Figure 3: Recall at p (normalized over number of candidate genes on 5 Perturb-seq datasets. For CDN, (P) and (S) denote "Perturb-seq" and "synthetic" training data.
Table 2: Rank of drug targets on chemical perturbation datasets MllFaaine-Figuere et al., 2024). A172 and T98G unseen cell lines. Highlighted: Within top 100. Drug names and details in Appendix D.I.
our goal is to lower the cost and scale of experiments, by reducing the search space. Our key technical contribution is the idea of using supervised causal discovery as a pretrain- ing objective, to obtain dataset representations useful for downstream tasks like inferring targets. Specifically. CDN trains causal graph learner that maps pair of observa- tional and interventional datasets to their data-generating mechanisms. and a differential network classifier to identify variables whose conditional independencies have changed. We demonstrated that our approach outperforms state-of- the-art methods in perturbation modeling and causal discov- ery, across seven transcriptomics datasets and variety of synthetic settings. Overall, these results highlight the poten- tial of our method to design more efficient experiments, by reducing the search space of perturbations.
A172
T98G
Model
infig.
nint.
palb.
doxo.
palb.
vola.
DGB
0.12
0.64
0.28
0.94
0.24
0.83
PDG
0.72
0.35
0.39
0.06
0.34
0.42
CDN
0.14
0.81
0.01
0.89
0.93
0.53
CDN (no p))
0.88
0.44
0.58
0.06
0.54
0.65
ing, as they are designed with the intuition that detecting differences is easier than reproducing the entire causal graph. Surprisingly, most other baselines perform poorly at recover- ing intervention targets. In the case of DCDI and BACADI, this may be because it is hard to select a single sparsity threshold for varying sizes of intervention sets, and to bal- ance the sparsity regularizer with the generative modeling objective.
There are several limitations of this work and directions for future research. First, our implementation does not address cyclic, feedback relationships or time-resolved, dynamic systems both of which are common in biology. Modeling these cases may require data simulation schemes that can sample from the steady-state of cyclic system, or archi- tectures that directly predict the parameters of a system of stochastic differential equations (Lorch et al., 2024). In addi- tion, current supervised causal discovery algorithms, which we use for the causal graph learner, tend to assume causal
5. Discussion
This work introduces causal differential networks (CDN), which focuses on the application of predicting perturbation targets for efficient experimental design. On a high level,
8

---

Identifying biological perturbation targets through causal differential networks
Table 3: Intervention target prediction results on synthetic datasets with N - 20, E = 40. Top: hard interventions (uniform);
bottom: soft interventions (scale). Neither polynomial nor "scale" soft interventions were seen during training. Number in parentheses indicates number of intervention targets. Uncertainty is standard deviation over 5 i.i.d. datasets. Runtimes in
Table Extended results in Table 12.
Linear (1)
Linear (3)
Polynomial (1)
Polynomial (3)
Type
Model
mAPt
AUC1
mAPt
AUC1
MAPt
AUCt
MAPt
AUC1
UT-IGSP
0.10
0.631.08
0.18
0.52:.03
0.19
0.70=.02 02
0.231103
0.562.03
DCDI-G
0,16:2.0 02
0.50..02
0.27:.05
0.50..04
0.151.03
0.45:.00
0.2611.0
0.51
DCDI-DSF
0,15:.02
0.46:.08
0.261..0 003
1.4
0.18
0.48:.05
0.28
0.52:.05
BACADI-E
0.18#.09
0.71=.00
0.28
0.69=.03
0.20#
0.84=.04
0.37+.07
0.82+.05
Hard
BACADI-M
0.10
0.662
0.21
0.63=.022
0.09
0.71=
0.24+.03
0.70=.05
DC1
0.58
0.85
2.
0.842.03
0.55#.08
0.82=.04
0.56#.08
0.79
MR+CI
0.47
0.682.08
0.58
0.71..06
0.772.05
0.88 .03
0.79*.04
0.882.02
CDN
0.82
0.962.02
0.88 04
0.95
0.85
0.97=.01
0.90#104
0.962.02
CDN (no Lc)
0.65+.u7
0.86#.00
0.72
0.84+.07
0.81+.05
0.95#.u2 .2
0.86+.03
0.94+.u1
UT-IGSP
0.102.or
0. 0.69
0.19
0.562.03
0.18+.02
0.77=.02
0.22:t.01
0.60#.01
DCDI-G
0.17:.03
0.50a.05
0.23
0.44
0.14:.03
0.48 03
0.30:+004
0.54 04
DCDI-DSF
0.17#.04
0.46#t.4
0.24
0.17=.03
0.16+.02
0.50=.05
0.26+.02
0.48#.02
BACADI-E
0.25#.07
0.65st.03
0.40
0.68 .08
0.43+.22
0.81=.09
0.49+.15
0.77#
Soft
BACADI-M
0.11#.01
0.61#.02
2.7
0.642.06
0.25#.10 10
0.80=.09
0.38+.10 10
0.76#
DCI
0.48+.02
0.78:t.02
0.53 .06
0.77 .04
0.622.09
0.892.04
0.55+.04
0.80#.04
MB+CI
0.46+.10
0.65
0.49 +.04
0.64 .03
0.97
0.98 1988..0m ..1
0.92
0.94
CDN
0.
0.95
0.76+.07
0.89#.04 .04
0.97+.03
1.00=.00
0.96+.02
0.99
CDN (no Lc)
0.77+.u6
0.95+..0
0.81+.os
0,93#.um .04
0.94+.03
0.99 .00
0.96+.02
0.99
exclude scale
0.32
0.64
0.34 09
0.65 .8
0.06#.02
0.63=.02 .02
0.08+.02
0.61
exclude scale
0.44
0.92+.01
0.43
0.90# .03
0.06+.01
0.72# .20
0.10+.02
0.73#.u2
sufficiency, i.e. that there are no unseen confounders. This assumption is unrealistic in biology, as it is not physically possible to measure all relevant variables. Thus, there is an opportunity to extend this work for the latent confound- ing setting. Finally, since it is difficult to simulate biology, there will be an inevitable domain shift between synthetic and real data. However, there does exist plentiful unlabeled single-cell data (Consortium* et al., 2022), which could be used with self-supervised objectives to bridge the gap (Sun et al., 2020). In conclusion, we hope that this work will enable efficient experimental design and interpretation, as well as future machine learning efforts towards these tasks.
A Acknowledgements
This material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No. 1745302. We would like to acknowledge support from the NSF Expeditions grant (award 1918839: Collabo- rative Research: Understanding the World Through Code), Machine Learning for Pharmaceutical Discovery and Syn- thesis (MLPDS) consortium, and the Abdul Latif Jameel Clinic for Machine Learning in Health.
Impact statement
Our work proposes machine learning method to improve understanding of large-scale perturbation experiments, in the context of molecular biology. Towards broader impact, this work may facilitate better understanding of drug mech- anisms; inform protocol design for cellular reprogramming: or identify causal targets of disease. While this work has numerous applications the life sciences, it does not di- rectly concern the design of molecules or other potentially harmful agents.
9

---

Identifying biological perturbation targets through causal differential networks
References
Chen, T., Bello, K., Aragam, B., and Ravikumar, iSCAN: identifying causal mechanism shifts among nonlinear additive noise models. Advances in Neural Information Processing Systems, 144466614447,,, 2023.
Ahlmann-Eltze, C., Huber, W., and Anders, S. Deep learning- beaniiggbbased predictions of gene perturbation effects do not yet outperfom simple linear methods. bioRxiv, 2024. doi: 10.1101/2024.09.1 16.663334.
Chen, Y. T. and Zou, J. GenePT: A simple but hard-to-beat foundation model for genes and cells built from ChatGPT. bioRxiv, 2023. doi: 10.1 10.1101/203, 10.11........ 10.16.662533.
Ashburner, M., Ball. A., Blake, J. A., Botstein, D., Butler, H. L., Cherry, J. M., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig, J. T., Harris, M. A., Hill, D. P., Issel-Tarver, L., Kasarskis, A., Lewis, S. E., Matese, J. C., Richardson, J. E., Ringwald, M., and Rubin, G. M. Gene ontology: tool for the unification of biology. Nature Genetics, 25: 25-29, 2000.
Cherry, A. and Daley, G. Reprogramming cellular identity for regenerative medicine. Cell, 148(6)11100-122) 2012. ISSN 0092-8674. doi: http:///bbior//10000666/// 02.031.
Chickering, D. M. Optimal structure identification with greedy search. Journal of Machine Learning Research, 3: 507-554, November 2002.
Babu, M. M., Luscombe, N. M., Aravind, L., Gerstein, M., and Teichmann, S. A. Structure and evolution of transcriptional regulatory networks. Current opinion in structural biology, 14(3):283-291, 2004.
Consortium T. T. S., Jones, R. C., Karkanias, J., Krasnow, M. A., Pisco, A. O., Quake, S. R., Salzman, J., Yosef, N., Bulthaup, B., Brown, P., et al. The Tabula Sapiens: A multiple- organ, single-cell transcriptomic atlas of humans. Science, 376(6594) 2022.
Bai, D., Ellington, C. N., Mo, S., Song, L.,, and Xing, E. AttentionPert: accurately modeling multiplexed genetic perturbations with multi-scale effects. Bioinformatics, 40:i453- 06 2024. ISSN 1367-4811. doi: 10.1093/ bioinformaaicsshnee244
Cosgrove, E. J., Zhou, Y.. Gardner, T. S., and Kolaczyk, E. D. Predicting gene targets of perturbations via network- based filtering of mRNA expression compendia. Bioin- formatics, 24(21):242--2490 09 2008. ISSN 1367-4803. doi: 10. 10.0093bionnoommiiccbtt//7
Belyaeva, A., Squires, C., and Uhler, C. DCI: learning causal differences between gene regulatory networks. Bioinformatics. 37118):3077-3069, 03 2021. ISSN 1367- 4803. doi: 10. 10.10///hoinffmmaii//bbbbbb...
Cui, H., Wang, C.,, Maan, H., Pang, K., Luo, F., Duan, N., and Wang. B. scGPT: toward building a foundation model for single-cell multi-omics using generative AI. Nature Methods, pp. 1-11, 2024.
Benjamini, Y. and Hochberg, Y. On the adaptive control of the false discovery rate in multiple testing with in- dependent statistics. Journal of Educational and Be- havioral Statistics, 25(1))60-88) 83, 2000. ISSN 10769986, 19351054.
Devlin, J., Chang. M.- M., Lec, K., and Toutanova, K. BERT: Pre-training of deep bidirectional transformers for lan- guage understanding. In Proceedings of the 2019 Confer- ence the North American Chapter of the Association for Computational Linguistics, pp. 4171-411111 1-166 Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 16555//1//11994423.
Blake, W. J., Kærn, M., Cantor, C. R., and Collins, J. Noise in eukaryotic gene expression. Nature, 422(6932): 633-637, 2003.
Brouillard, P., Lachapelle, S., Lacoste, A., Lacoste-Julies S., and Drouin, A. Differentiable causal discovery from interventional data. Advances in Neural Information Pro- cessing Systems,
di Bernardo, D., Thompson, M. J.. Gardner, T. S., Chobot, S. E.,, Eastwood, E. I.., Wojtovich, A. P., Elliotr, S. J., Schaus, S. E., and Collins, J. J. Chemogenomic profiling on a genome-wide scale using reverse-engineered gene networks. Nature Biotechnology, 23(3):377-383, 2005.
21877, 2020.
Bunne, C., Stark, S. G., Gut, G., Del Castillo, J. S., Levesque, M. Lehmann, K.-V., Pelkmans, L.. Krause, A., and Rätsch, G. Learning single-cell perturbation re- sponses using neural optimal transport. Nature Methods, 20(1 10(11)1175911768 2023.
Dixit, A., Parnas, O., Li, B., Chen, J., Fulco, C. P., Jerby- Amonn L., Marjanovic, N. D., Dionne, D., Burks, T., Raychowdhury. R., Adamson, B., Norman, T. M., Lan- der, E. S., Weissman, J. S., Friedman, N., and Regev, A. Perturb-seq: Dissecting molecular circuits with scal- able single-cell RNA profiling of pooled genetic screens. Cell, 16777) 2016. ISSN 0092-867 8674. doi: http:://bbi.org//0..0066... 11.038..
Chen, S., Chandra Tjin, C., Gao, X., Xue, Y., Jiao, H., Zhang, R., Wu, M., He, 7.,, Ellman, J., and Ha, Y. Phar- macological inhibition of pi5p4ka/3 disrupts cell energy metabolism and selectively kills p53-null tumor cells. Proceedings of the National Academy of Sciences, 118 (21 ee20022228 18, 2021.
10

---

Identifying biological perturbation targets through causal differential networks
Ghassami, A., Kiyavash, N., Huang, B., and Zhang, K. Multi-domain causal structure learning in linear systems. Advances in Neural Information Processing Systems, 31, 2018.
Huttlin, E. L.,, Bruckner, R. J., Mavarrete-Perea, J., Cannon, J. R., Baltier, K., Gebreab, F., Gygi, M. P., Thomock, A., Zarraga, G., Tam, S., et al. Dual proteome-scale networks reveal cell-specific remodeling of the human interactome. Cell, 184(1 3):3222-304,, 2021.
Gonzalez, G., Herath, I., Veselkov, K., Bronstein, M., and Zitnik, M. Combinatorial prediction of therapeutic pertur- bations using causally neural networks. bioRxiv, 2024. doi: 10.1 01000/2024.0100.57333330
Isik, Z., Baldow, C., Cannistraci, C. V., and Schroeder, M. Drug target prioritization by perturbed gene expression and network information. Scientific Reports, 5(1):17417, 2015.
Grandi, F. C., Modi, H., Kampman, L., and Corces, M. R. Chromatin accessibility profiling by ATAC-seq. Naturef Protocols, 17(6):5118-1522, 2022.
Jaber, A., Kocaoglu, M., Shanmugam, K., and Barein- boim, E. Causal discovery from soft interventions with unknown targets: Characterization and learning. Ad- vances in Neural Information Processing Systems, 33: 9551-9561, 2020.
Hägele, A., Rothfuss, J., Lorch, L., Somnath, V. R., Schölkopf, B., and Krause, A. BaCaDI: Bayesian causal discovery with unknown interventions. In Proceedings of 26th International Conference on Artificial Intelli- gence and Statistics, volume 206, pp. 1411-1436 PMLR, 25 27 Apr 2023.
Ke, N. R., Bilaniuk, O., Goyal, A., Bauer, S., Larochelle, H., Pal, C., and Bengio, Y. Learning neural causal models from unknown interventions. arXiv preprint 191001190000105, 2019.
Hao, M., Gong, J.. Zeng. X., Liu, C., Guo. Y., Cheng, X., Wang, T., Ma, J., Song, L.,, and Zhang, X. Large-scale foundation model on single-cell transcriptomics. Nature
Ke, N. R.,, Chiappa, S., Wang. J. X., Bornschein, J., Goyal, A., Rey, M., Weber, T., Botvinick, M., Mozer, M. C., and Rezende, D. J. Learning to induce causal structure. In International Conference on Learning Representations, 2023.
Methods, 2024. doi: 10.1038/44159--024-02233
Hauser, A. and Bühlmann, P. Characterization and greedy learning of interventional markov equivalence classes of directed acyclic graphs. Journal of Machine Learning Research, 13(79):2409-2466 2012.
Kernfeld, E., Yang, Y., Weinstock, S., Battle, A., and Cahan, A systematic comparison of computational methods for expression forecasting. bioRxiv, 2023. doi: 10.1101/2023.077 22..558553133
Hetzel, L., Boehm, S., Kilbertus, N., Günnemann, S., Theis, F., et al. Predicting cellular responses to novel drug per- turbations at a single-cell resolution. Advances in Neural Information Processing Systems, 35:26711-26722 2022.
Knox, C., Wilson, M., Klinger, C. M., Franklin, M., Oler, E., Wilson, A., Pon, A., Cox, J., Chin, N. E., Strawbridge, S. A., et al. DrugBank 6.0: the DrugBank knowledgebase for 2024. Nucleic Acids Research, 52(D1): 2024.
Hilberg, F., Roth, G. J., Krssak, M., Kautschitsch, S., Som- mergruber, W., Trontcch-Grun U.. Garin-Chesse P., Bader, G., Zoephel, A., Quant, J., et al. BIBF 1 120 triple angiok- inase inhibitor with sustained receptor blockade and good antitumor efficacy. Cancer Research, 68112)477444788 2008.
Lachapelle, S., Brouillard, P., Deleu, T., and Lacoste-Julic Julien, S. Gradient- neural DAG learning. In International Conference on Learning Representattinss 2020.
Lamb, J., Crawford, E. D., Peck, D., Modell, J. W., Blat, I. C.,. Wrobel, M. J., Lerner, J., Brunet, J.-P., Subramanian, A., Ross, K. N., Reich, M., Hieronymus, H., Wei, G., Armstrong. S. A., Haggarty, S. J., Clemons, P. A., Wei, R., Carr, S. A., Lander, E. S., and Golub, R. The connectivity map: Using gene-expression signatures to connect small molecules, genes, and disease. Science, 313:1929 1935, 2006.
Ho, J., Kalchbrenner, N., Weissenborn, D., and Salimans, T. Axial attention in multidimensional transformers. arXiv preprint arXiv: 1912. 22/2./22 12100 2019.
Huang, B., Zhang, K., Zhang, J., Ramsey, J., Sanchez- Romero, R., Glymour, C., and Schölkopf, B. Causal discovery from beterngenousshonssatioonyy data, Jour- nal of Machine Learning Research, 1-53)):1-53, 2020.
Li, H., Xiao, Q. and Tian, J. Supervised whole DAG causal discovery. arXiv preprint arXivv2006.044599 2020.
Huang, K., Lopez, R., Hütter, J.-C., Kudo, T., Rios, A., and Regev, A. Sequential optimal experimental design of perturbation screens guided by multi-modal priors. In International Conference on Research in Computational Molecular Biology, pp. 17-37. Springer, 2024.
Lorch, L., Sussex, S., Rothfuss, J., Krause, A., and Schölkopf, B. Amortized inference for causal structure learning. Advances in Neural Information Processing Systems, 35:13104-133 18, 2022.

---

Identifying biological perturbation targets through causal differential networks
Lorch, L., Krause, A., and Schölkopf, B. Causal modeling with stationary diffusions. In Proceedings of The 27th International Conference Artificial Intelligence and Statistics, volume 238, pp. 1927-1935. PMLR, 02-04 May 2024.
Mooij. J. M., Magliacane, S., and Claassen, T. Joint causal inference from multiple contexts. J. Mach. Learn, Res., 21(1), Jan 2020. ISSN 1532-4435.
Nadig, A., Replogle, J. M., Pogson, A. N., McCarroll, S. A., Weissman, J. S., Robinson, E. B., and O' O'Connor, L. J. Transcriptome wide characterization of genetic perturba- tions. bioRxiv, 2024. doi: 10.1101/2024.07.000 01.00202222.7.006199903..
Loshchilov, I. and Hutter, F. Decoupled weight decay regu- larization, 2019.
Newman, R., Ghaemmaghami, S., Ihmels, J., Breslow, D. K., Noble, M., DeRisi, J. L.,, and Weissman, J. S.
Lotfollahi, M., Wolf, F. A., and Theis, F. J. scGen predicts single-cell perturbation responses. Nature Methods, 16: 715 - 721, 2019.
Single-cell proteomic analysis of S. cerevisiae reveals the architecture of biological noise. Nature, 441(7095); 2006.
Lotfollahi, M., Klimovskaia Susmelj, A., De Donno, C.,, Hetzel, L.,, Ji, Y., Ibarra, I.., Srivatsan, R., Naghipour- far, M., Daza, R. M., Martin, B., et al. Predicting cellular responses complex perturbations in high-throughput screens. Molecular Systems Biology, pp. e11517, 2023.
Noh, H. and Gunawan, R. Inferring gene targets of drugs and chemical compounds from gene expression profiles. Bioinformatics, 32(14):2120-213) 2016.
Norman, T. M., Horlbeck, M. A., Replogle, J. M., Ge, A. Y., Xu, A., Jost, M., Gilbert, L. A., and Weissman, J. Exploring genetic interaction manifolds constructed from rich single-cell phenotypes. Science, 365(64555:786-79, 2019. doi: 10.1 16/sciincce.axx4433..
Love, M. I., Huber, W., and Anders, S. Moderated estima- tion of fold change and dispersion for RNA-seq data with DESeq2. Genome Biology, 15(12):550-555 -50000 2014. doi: 10.1186/s13053 10066//13300-11-0555500000000000
Pedregosa, E., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cour- napeau, D., Brucher, M., Perrot, M., and Duchesnay, Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830, 2011.
Luck, K., Kim, D.-K., Lambourne, L., Spirohn, K., Begg, B. E., Bian, w., Brignall, R., Cafarelli, T., Campos- Laborie, F. J., Charloteaux, B., et al. A reference map of the human binary protein interactome. Nature, 580 (7803):402-408, 2020.
Luecken, M. D. and Theis, F. J. Current best practices in single-cell RNA-seq analysis: tutorial. Molecular systems biology. 15(6):e8746, 2019.
Petersen, A. H., Ramsey, J., Ekstrøm, C. T., and Spirtes, P. Causal discovery for observational sciences using supervised machine learning. Journal of Data Science, 21(2):255-282 2023. ISSN 1680-743X. doi: 10.6339/ 23-JDS1083
Malik, V., Bello, K., Ghoshal, A., and Honorio, J. Identi- fying causal changes between linear structural equation models. In The 40th Conference on Uncertainty in Artifi- cial Intelligence, 2024.
Rao, R. M., Liu, J., Verkuil, R., Meier, J., Canny, J., Abbeel, P., Sercu, T., and Rives, A. MSA transformer. Interna- tional Conference on Machine Learning, pp. 8844-8856. PMLR, 2021.
Märtens, K., Donovan-Maiyy R., and Ferkinghoff- -Bor, Enhancing generative perturbation models with L.M-- informed gene embeddings. In ICLR 2024 Workshop on Machine Learning for Genomics Explorations, 2024.
Replogle, J. M., Saunders, R. A., Pogson, A. N., Hussmann, J. A., Lenail, A., Guna, A., Mascibroda, I.., Wagner, E. J., Adelman, K., Lithwick- Yanai, G., Iremadze, N., Oberstrass, F., Lipson, D., Bonnar, J. L., Jost, M., Nor- man, T. M., and Weissman, J. S. Mapping information- rich genotype-phenotyy landscapes with genome-scale Perturb-seq. Cell, 185(14):2559-2575, Jul 2022.
McFaline- FFguuroe, J. L., Srivatsan, S., Hill, A. J., Gasperini, M., Jackson, D. L.., Saunders, L., Domcke, S., Regalado, S. G., Lazarchuck, P., Alvarez, S., Mon- nat, R. J., Shendure, J., and Trapnell, C. Multiplex single-cell chemical genomics reveals the kinase depen- dence of the response to targeted therapy. Cell Ge- nomics, 100477 2024. ISSN 2666-979X. doi:
Roohani, Y., Huang, K., and Leskovec, J. Predicting tran- scriptional outcomes of novel multigene perturbations with gears. Nature Biotechnology, 2023.
Roohani, Y. H., Vora, J., Huang, Q. Liang, P., and Leskovec, J. BioDiscoveryAAgett An Al agent for designing genetic perturbation experiments. In International Conference on Learning Representations, 2025.
Montagna, F., Caimey-Leeming, M., Sridhar, D., and Lo- catello, F. Demystifying amortized causal discovery with transformers. arXiv preprint arXiv:2405. 16924, 2024.
12

---

Identifying biological perturbation targets through causal differential networks
Sanchez, P., Liu, X., O'Neil, A. Q. and Tsaftaris, S. A. Diffusion models for causal discovery via topologiaal ordering. In The Eleventh International Conference Learning Representations, 2023.
Tian, J. and Pearl, J. Causal discovery from changes. In Pro- ceedings of the Seventeenth conference on Uncertainty in artificial intelligence, pp. 512-521, 2001.
Varici, B., Shanmugam, K., Sattigeri, P., and Tajer, A. In- tervention target estimation in the presence of latent vari- ables. In Uncertainty in Artificial Intelligence, pp. 2013- 2023. PMLR, 2022.
Schenone, M., Dancík, V., Wagner. B. K., and Clemons, P. A. Target identification and mechanism of action in chemical biology and drug discovery. Nature Chemical Biology, 9 4:232 40, 2013.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, u., and Polosukhin, I. At- tention is all you need. In Guyon, I,, Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.. 2017.
Schneider, N., Lorch, L., Kilbertus, N., Schölkopf, B., and Krause, A. Generative intervention models for causal perturbation modeling, arXiv preprint arXiv:2411.14003 2024.
Shimizu, S., Hoyer, P. O., Hyvarinen, A., and Kerminen, A.
A linear non non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(72):2003-203 2006.
Wang. Y., Squires, C., Belyaeva, A., and Uhler, C. Direct estimation of differences in causal graphs. Advances in Neural Information Processing Systems, 31, 2018.
Spirtes, P., Meek, C., and Richardson. T. Causal inference in the presence of latent variables and selection bias. In Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, UAI'95, pp. 499-506, San Fran- cisco, CA, USA, 1995, Morgan Kaufmann Publishers Inc. ISBN 1558603859.
Wilcoxon, Individual comparisons by ranking methods. Biometrics Bulletin, 1(6):80 83, 1945. ISSN 00994987.
Wolf, A., Angerer, P., and Theis, F. J. Scanpy: large- scale single-cell gene expression data analysis. Genome Biology, 19:1-5, 2018.
Spirtes, P.. Glymour, C., and Scheines, R. Causation, Prediction, and Search. MIT Press, 2001. doi: https: httg///.755//mmprreesst. 155.0oss///754.0...00001
Wu, M., Bao, Y., Barzilay, R., and Jaakkola, T. Sample, esti- mate, aggregate: A recipe for causal discovery foundation models. Transactions on Machine Learning Research, 2025.
Squires, C., Wang, Y., and Uhler, C. Permutation-bastt based causal structure learning with unknown intervention tar- gets. In Peters, J. and Sontag, D. (eds.), Proceedings of the 36th Conference on Uncertainty in Artificial Intelli- gence (UAI), volume 124, pp. 1039-1048. PMLR, 03-06 Aug 2020.
Wu, Y., Barton, R., Wang, Z., Ioannidis, V. Donno, C. D., Price, L.. C.,, Voloch, I.. F., and Karypis, G. Predicting cellular responses with variational causal inference and refined relational information. In The Eleventh Interna- tional Conference Learning Representations, 2023.
Srivatsan, S. R., CidFaline-Figueeee J. L., Ramani, V., Saun- ders, L.., Cao, J., Packer, J., Pliner, H. A., Jackson, D. L., Daza, R. M., Christiansen, L.., Zhang, F., Steemers, F., Shendure, J., and Trapnell, C. Massively multiplex chem- ical transcriptomics at single-cell resolution. Science, 367 (6473):45-51, 2020 doi: 10.1
Yang, Y., Salehkaleybar, S., and Kiyavash, N. Learning unknown intervention targets in structural causal models from heterogeneous data, In Dasgupta, S., Mandt, S., and Li, Y. (eds.), Proceedings of The 27th International Con- ference on Artificial Intelligence and Statistics, volume 238, pp. 3187-3195. PMLR, 02-04 May 2024.
aax6234.
Su. J., Ahmed, M., Lu, Y., Pan, S., Bo, W., and Liu, Y. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 168:127063, 2024.
Yun, C., Bhojanapalli, S., Rawat, A. S., Reddi, J., and Kumar, S. Are Transformers universal approximators of sequence-to-sequence functions? CoRR, abs/1912.10077,, 2019.
Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A., and Hardt, M. Test-time training with self-supervision for generaliza- tion under distribution shifts. In International Conference on Machine Learning, pp. 9229-9248. PMLR, 2020.
Zhang, J., Cammarata, I.,, Squires, C., Sapsis, T. P., and Uhler, C. Active learning for optimal intervention design in causal models. Nature Machine Intelligence, 5:1066- 1075, 2023a.
Theodoris, C. V., Xiao, L., Chopra, A., Chaffin, M. D., Al Sayed, Z. R., Hill, M. C., Mantinco, H., Brydon, E. M., Zeng, Z., Liu, X. S., et al. Transfer learning enables predictions in network biology. Nature, 618(7965):616-
Zhang, J., Greenewald, K., Squires, C., Srivastava, A., Shan- mugam, K., and Uhler, C. Identifiability guarantees for
624, 2023.
13

---

Identifying biological perturbation targets through causal differential networks
causal disentanglement from soft interventions. In Thirty- seventh Conference Neural Information Processing Systems, 2023b.
Zheng, X., Aragam, B., Ravikumar, P. K., and Xing, E. DAGs with no tears: Continuous optimization for struc- ture learning. Advances in Neural Information Processing Systems, 31, 2018.
14

---

Identifying biological perturbation targets through causal differential networks
A. Model architecture
A.I. Attention layers
To balance expressivity and efficiency, our model is designed around series of "axial" attention layers, similar to Ho et al. (2019); Rao al. (2021); Wu et al. (2025). The causal structure learner closely resembles the architecture of SEA (Wu et al., 2025), while the differential network is slightly different (since the input space is different). Briefty, the input to the causal structure learner is matrix of shape N N T d. However, the (dense) global statistic does not span the T dimension, and the local graph estimates are generally sparse, Thus, the causal graph learner separately attends over the N x N x d and K x T x d components (K is the number of unique edges in sparse format), with "messages" to align the representations between each attention layer. In contrast, the differential network only operates over the dense N x (N + 1) x d (or 2d) representation. Thus, omits the message passing layers and directly stacks dense attention layers.
A.2. Edge embeddings
Since attention itself is invariant to input ordering, Transformer-style architectures use absolute (Devlin et al., 2019) or relative (Su et al., 2024) positional embeddings to distinguish input elements. Images and sequences have a natural ordering among their elements. This structure inherently induces distances between positions, which provides signal for learning models. In contrast, models over graphs should be equivariant to the labeling of nodes. That is, if + 2 is an edge, while 2 and are not connected, then re-labeling the nodes (1, 2, 3) (2,3, should result in edge 2 3 and edge between 3 and! 1.
If we assign an arbitrary ordering to each graphx, the Transformer will be able to distinguish between nodes. However, there are several practical issues. Since synthetic data are generated in the topological ordering of the nodes, always assigning the root nodes to low positions may lead to unintended information leakage. Furthermore, the model is trained on graphs of up to = 100, but tested on larger graphs, the positional embeddings associated with > 100 will be entirely random and out of distribution. Following Wu et al. (2025), we randomly sample a permutation of the maximum graph size to each graph. Each edge embedding is the concatenation of its constituent nodes' embeddings, combined via feed-forward network:
Embed 5) = FFN FFFFFFFENeeeee (i) Ej)ed(j)))
(mmbee(())
where j are arbitrarily assigned, but consistent within the graph. This embedding is added to the hidden representation hi,j before the first layer of attention.
In the differential network, the additional "column" associated with node-level statistics is not subject to any embedding (which implicitly differentiates from the remaining inputs), while the "row" identifies the node these statistics were computed for.
B. Implementation details
B.1. CDN training
We trained until convergence (no improvement for 50 epochs on synthetic; 10 epochs on Perturb-seq) and selected the best model based on validation mAP (Figure 4). The validation distribution was identical to training and did not contain test causal mechanisms. During training, we used 15 CPU workers (primarily for local graph estimates) and A6000 GPU. CDN took around hours train on synthetic data, and hour to finetune on real data. Unless otherwise noted, the causal structure leamner was initialized from the SEA inverse covariance, FCI pretrained weights (Wu al., 2025).
B.2. Baselines
We used the latest releases of all basclines.
GEARS is graph neural network that predicts the effects of unseen genetic perturbations, based on the Gene Ontology knowledge graph (Ashburner et al., 2000). We trained GEARS on each cell line separately, and predicted the effects of perturbing every expressed gene. Then we ranked candidates based on cosine similarity to the interventional data. Figure 9 depicts the low, but non-trivial proportion of differentially expressed genes for which GEARS was unable to make a prediction, due to lack of node coverage in their processed gene ontology graph (Ashburner et al., 2000). These genes were not considered in the rankings or evaluations for GEARS.
15

---

Identifying biological perturbation targets through causal differential networks
Target loss
Graph loss
Accuracy
mAP
AUC
0.80
0.950
0.9
0.9
0.3
0.925
1
0.75
0.8
0.8
0.2
0.900
0.875
0.7
4
0.70
0.7
2500
2500
2500
5000
2500
5000
Step
Step
Step
Step
Step
Figure 4: Hyperparameter search: number of differential network layers. We selected layers based on validation metries.
GENEPT utilized the v2 March 2024 update, which used newer models and additional protein data, compared to their initial We concatenated each perturbation's 's gene embedding to the log-fold change of the top 5000 highly-variable genes (Wolf et al., 2018) and trained logistic regression model on each cell line to predict true vs. decoy perturbations. Candidates were ranked based on predicted probability. Of the 2,842 unique genetic perturbations, only 2.819 (99.1%) mapped GENEPT embeddings. The remainder used the mean gene embedding (within the dataset) the language- -assede-bddddd embedding.
paper (GenePT_geet -aaeee-eeeeepp www.geeeeoteeellleeee proteiri i.eembedding
PDGRAPHER was published on the union of three distinct knowledge graphs, but their harmonized graphs were not available publicly, and the authors did not respond to requests for data sharing. As result, we relied on solely the human reference interactome (Luck et al., 2020) graph, as was the only one of the three that could be easily processed. All test perturbation targets could be inferred through PDGrAPHER
UT-IGSP used the official implementation from commit 7349396, since the repository is not back-compatible.
BACADI is evaluated using the fully-connected implementation, since it performed better than the linear version. However, since the linear version is significantly faster, we include its runtime in Table 6.
MB+CI was implemented using scikit-learn (Pedregosa et al., 2011).
C. Theoretical motivation
While this paper focuses biological applications, show that the differential network is well-specified as class of models, i.e. correct solution is contained within the space of possible parametrizations.
Preliminary note In this analysis, we assume that the "causal" representations h contains enough information both to retain global statistics and recover the true graph E. The former is reasonable due to high model capacity, and the latter is based on high empirical performance in graph reconstruction (Wu et al., 2025). We emphasize that the latter is an empirical judgment, which may not hold on all datasets in practice.
Attention- based architecture Our differential network is implemented using an attention layer, which is composed of two self-attention layers (one along each axis of the adjacency matrix) and a feed-forward network. For simplicity, we follow prior work (Yun al., 2019) and ignore layer normalization and dropout.
Our inputs h E x are 2d-dimension features, which represent pair of N x N causal graphs. We use h.j to denote a length N row for a fixed column j. and h;, to denote length N column for fixed row i. The attention layer implements:
Attn (h = h. + WoWyh. a (Wxhh.))WWhhh.,, Attn (ha,)) hi + WoWrh, (W } FFN(h) = h + Wz ReLU h b,) + b2,
where Wo E 2ddx2d Wv, WK, Wo E 2ddx2d W₂ E 22dxm W, E mx2d by E Rdd,b E R", and m is the FFN hidden dimension. We have omitted the i and subscripts on the Ws, but they use separate parameters. Any self attention can take on the identity mapping by setting Wo, Wv. WK. , Wo to 2d 2d matrices of zeros.
16

---

Identifying biological perturbation targets through causal differential networks
Hard interventions Let G (V, E) be causal graphical model associated with data distribution Px. Let G' (V, E') and Px denote the causal graph and data distribution after an unknown intervention, with ground truth targets T Ç V. For convenience, we use E, E" both to denote sets of edges, well as the equivalent adjacency matrices.
In the case of perfect interventions,
f(x;) + 2z, E
(10)
where II. X are independent random variables.
Px is associated with mutilated graph E', where
E" = E' U...E
(11)
IEI
In terms of the associated adjacency matrices, F E' has Is in each column i 7 and Os elsewhere.
Here, the attention layer should implement / - h', so that when we collapse over the incoming edges, the output is non-zero only at the edge differences. Suppose the first dimension of the 2d feature stores E, and the second dimension stores E". The row self-attention implements the identity (in the first two dimensions). Then we can set WX,Q to zero, Wy to the identity, and Wo to
Wo- 10 1 1
0
0
(12)
to account for the residual. The FFN implements the identity, so that when we take the mean over along the rows, we recover non-zero elements at all nodes whose incoming edges were removed.
Soft interventions We also study soft interventions the context of causal models with non-multiplicative noise, in which intervention targets are scaled by constant factors.
f(x)) + caf)),
(13)
where e, are sampled random per synthetic dataset. This choice inspired by the fact that biological perturbation effects are measured in fold-change. Here, the adjacency matrices are the same, but global statistics differ. In particular, we focus on two statisties: the correlation matrix R and the covariance matrix E. Note that while we do not explicitly provide the covariance matrix as input, we do provide marginal variances, from which the same information can be derived.
Suppose æ is an intervention target.
R R' non-zero in all entries j and j, where is a descendent of xr and j is any node for which Ras # 0 (e.g. ancestors, descendants, and x, if Px is faithful to G).
Σ Σ' is non-zero in all entries j and j, where is descendent of x or x. and j is any node for which Exg # 0 (e.g. ancestors, descendants, and x, if Px is faithful to G).
All descendants are always affected, due to the non-multiplicative noise term. These two differ in the row and column that correspond to x since
Corr(c- x2 Corr(z, y) Cov(e- x.,) = c Cov(rr
(14)
(15)
Therefore, to identify x, we should find the index in which Σ differs but not R. Suppose that dimensions 3-6 of h encode
R. R', E' Following the same strategy the hard interventions, we can use the row attention to compute R - RRRR Σ - E' and store them in dimensions 3, 4. Then we use the column attention to filter out variables that are independent from by storing the sum of each column in dimensions 5, 6. While not strictly impossible, it is unlikely that a variable dependent on o would result in a column that sums to exactly O. Thus, all columns with non-zero sums are either ancestors, descendants, or x. The feedforward network implements
-
h.33 = 0, h:,4 4 # h. + otherwise.
FFN(h.3-6)
(16)
This results in Is in the rows and columns where AR and AE differ. After collapsing over incoming edges and normalizing to probabilities, the maximum probabilities can be found at the intervention targets.
17

---

Identifying biological perturbation targets through causal differential networks
Supporting both intervention types Recall that the final output layer is a linear projection from 2d 1. If this layer implements simple summation over all 2d, the predicted intervention targets are consistent with both hard and soft interventions. For soft interventions, E' = E", so the hard intervention dimensions will be O. Likewise, for hard interventions, both R and Σ will differ as the same locations, as the underlying variable has changed, so the soft intervention dimensions will be O. Since the two techniques produce mutually exclusive predictions, this means that both hard and soft interventions can co-exist and be detected on different nodes.
D. Datasets
D.1. Biological datasets
Data processing We converted all single cell datasets to nog-oormalized, transcripts per 10,000 UMIs. Perturb-seq genes were mapped to standard identifiers and filtered by the authors. Sci-Plex dataset variables represented genes that appeared in at least 5,000 cells (threshold chosen to achieve a similar number of genes). We performed differential expression analysis via the scanpy package (Wolf et al., 2018), using the Wilcoxon signed-rank test (Wilcoxon, 1945) with Benjamini- Hochberg p-value correction and a threshold of adjusted p-value < 0.05. Table 4 reports statistics of the raw, unprocessed datasets.
For Perturb-seq datasets, we kept perturbations with > 10 petprenntially-xxpeeseeee genes (DEGs), and clustered them using k-means with k: = 200, chosen heuristically based on log-fold change heatmaps (Figure 5). These clusters were used to inform data splits for seen cell lines, where the largest cluster was allocated to the training set, and all remaining clusters were split equally among train and test. The largest cluster(s) appear to contain perturbations with smaller effects. For Sci-Plex datasets, only 6 drug perturbations across 2 cell lines resulted in differential expression of their known protein targets (Supplementary Table from ScFaline-Figueree et al. (2024)). Therefore, we used these exclusively as test sets. Table 5 reports statistics of the final, processed datasets. Figure 8 plots the full distribution of number of cells and DEGs per perturbation.
Top differentially expressed genes Here, we limited our analysis to the top 1000 DEGs. In the literature, it is quite common to restrict analysis to subsets of genes, though the threshold and criteria may vary (Luecken & Theis, 2019). Selecting the top 1000 genes (based on differential expression p-value) is reasonably permissive, since these genes are selected per perturbation. For sense of scale, Nadig et al. (2024) writes:
"In a genome-scale Perturb-seq screen, we find that typical gene perturbation affects an estimated 45 genes, whereas a typical essential gene perturbation affects over 500 genes."
classic approach for identifying drug targets argues that targets are not highly differentially expressed, but network-based approaches are useful for predicting them from gene expression (Isik al., 2015).Thii work also samples 1000 candidate targets as decoys.
Gene co-expression Figures 6 and 7 depict the differences in gene co-expression matrices between control and perturbed cells. Genetic perturbations tend to have much clearer phenotypes compared to chemical perturbations, whose effects are more diffuse.
Sci-Plex drug targets the authors FMigarine-Figueroa et al., 2024), and are each are well-documented in the literature.
The full names of the Sci-Plex drugs and their targets are as follows. These targets were collated by
Infigratinib ("infig") binds target FGFRI with nano-molar affinities (Knox al., 2024).
Nintedanib ("nint") and target FGFR1 have been probed through structural studies (Hilberg et al., 2008).
Palbociclib ("palb") has been co-crystallized (PDB 5L21) with its target CDK6.
Doxorubicin ("doxo") and target TOP2A are associated a number of works (Knox et al., 2024).
Volasertib ("vola") has been co-crystallized (PDB 3FC2) with PLK1 and a number of works confirm specificity for PLK2 (Chen et al., 2021).
18

---

Identifying biological perturbation targets through causal differential networks
Table 4: Extended biological dataset statistics (raw).
Type
Source
Accession
Cell line
# Perts
# Genes
# NTCs
# Cells
K562 gw
9,866
8,248
75,328
1,989,578
Replogle et al. (2022)
Figshare 20029387
K562 es
2,057
8,563
10,691
310,385
Genetic
RPEI
2,393
8,749
11,485
247,914
HepG2 Jurkat
2,393
9,624
4,976
145,473
Nadig al. (2024)
GSE220095
2,393
8,882
12,013
262,956
McFaline-Figuera et al. (2024)
A172 T98G
23
8,393
8,660
58,347
Chemical
GSM7056151
23
8,393
6,921
58,347
Table 5: Extended biological dataset statistics (processed).
Perturbations
Genes
Cells
Dataset
Train
Test
Trivial Non-trivial
Unique # DE Median # DE
K562 gw
1089
678
587
91
7,378
81
492,096
K562
640
420
348
72
8,492
226
213,552
RPE1
564
397
233
164
8,641
399
179,696
HepG2
364
263
162
101
9,282
271
79,309
Jurkat
679
333
262
71
8,432
162
174,698
A172
445
324
18,196
T98G
3
1,644
508
13,126
is important note that these drugs are also associated with known off-targets. However, we did not observe differential expression of any off-target, adjusted p x perhaps because the impact their mRNA expression negligible. Protein activity would be more suitable for quantifyyng the impact of a drug cell. However, this cannot be measured at scale. In fact, even protein abundance is difficult to measure at single-cell resolution, as it relies on mass spectrometry to differentiate proteins (Newman et al., 2006), while mRNA can be exhaustively enumerated and sequenced (based on the known human genome). Therefore, this chemical perturbation study is primarily a proof of concept, to probe whether targets can be inferred from mRNA expression, and so, to what extent.
D.2. Synthetic data
Synthetic data were generated using code modified from DCDI (Brouillard et al., 2020) for soft interventions. We used the following implementations for causal mechanisms f(z), where x is the variable in question, A a binary mask for the parents of v, X contains measurements all variables, E is independent Gaussian noise, and W is random weight matrix.
Linear: f(z) MXW + E.
Neural network, non-additive f(r) = Tanh((X, E) Wout
Neural network, additive: f(x) Tanh Wou +
Polynomial: f(x) = Wo + MXW, MX
Sigmoid: ((x Dd 1 W; sigmoid Xx)
Linear and the neural network variants were used for training. Polynomial and sigmoid were only used for testing.
Root causal mechanisms are uniform. For hard interventions, we set f(x) f- z, where z ~ Uniform 11, 1). For soft "scale" interventions, we
f(x) + Sign(x) f(x)
21 Uniform 4)
Uniform( -1, 1).
That we multiply (xx by a scaling factor that equal probability $ (constant across all observations).
19

---

Identifying biological perturbation targets through causal differential networks
K562 gw
K562 es
RPE1
Jurkat
HepG2
Figure 5: Heatmap of correlation between log-fold change, sorted by cluster, used for data splits.
ENSG00000275022
ENSG00000275700
ENSG00000278845
ENSG00000284024
-0.5
0.0
0.5
0.0
0.5
0.0
0.5
0.0
0.5
Figure 6: Difference between observational and interventional correlation matrices on K562 genome-wide, some cases, the changes are minimal and specific; in others, they are more diffuse.
For soft "shift" interventions, we set
f(x) + f(x) + Sign(2) 21 ~ Uniform(2 (2, 4)
2 ~ Uniform(-11 1).
That is, we translate ((x) by a scaling factor that equal probability ş 0 (constant across all observations).
E. Additional analyses
Runtime We compared the runtimes of various algorithms on the Perturb-seq and synthetic datasets. On the Perturb-seq data, all models finished running within minutes with the exception of GEARS (Figure 11). Runtimes of the various causal algorithms varied significantly (Table 6). The slowest method was DCDI, with an average runtime of around 10 hours on N datasets, while the fastest were UT-IGSP and the MLP variant of CDN. All models were benchmarked on equivalent hardware (A6000 GPU, CPU core).
Perturb-sey uncertainty quantification Due to space limitations, we report uncertainty estimates in Tables 7 and 8. Multiple baselines produce deterministic results (LINEAR, DGE, GENEPT), so instead of model randomness, we report uncertainties over the sampling of single cells (Table 7). Specifically, for each perturbation with M cells, we sample
M' (Mi,M, max(50, 0.8M))
(17)
cells uniformly at random, repeated times. We also consider the robustness of our model to samplings of the test set (Table 8). Typically, the largest cluster (assigned train, e.g. nearly 1/3 of K562) contains weak perturbations with minimal phenotype, different clusterings don't introduce significant heterogeneity. Instead, can partition all perturbations uniformly into 5 sets for the unseen cell line setting, across which the variation in results is minimal.
Perturb-seg trivial perturbations For the main results in Table 1, we focus on genetic perturbations for which the intended gene target was not the gene with the greatest log-fold change. This is because genetic perturbations are (unrealistically) specific, compared to drugs or transcription factors. In Table 9, we show that our model performs nearly perfectly this easy setting.
20

---

Identifying biological perturbation targets through causal differential networks
A172 Infigratinib
A172 Nintedanib
A172 Palbociclib
-02
0.0
0.25
0.00
0.25
0.00
0.25
T98G Doxorubicin
T98G Palbociclib
T98G Volasertib
0.5
0.0
0.5 -02
0.0
0.2 0.4
0.0
0.4
Figure 7: Difference between observational and interventional correlation matrices on Sci-Plex datasets. Effects are much more diffuse than in genetic perturbations (Figure 6).
# Cells per perturbation
DEGs per perturbation
# Cells per perturbation
# DEGs per perturbation
1000
1000
2000
500
2000
500
0
K562 gw K562 es RPE1
HepG2
Jurkat
K562 gw K562 es RPEI
HepG2
Jurkat
A172
T98G
A172
T98G
Figure 8: Transcriptomics dataset statistics, after processing. See Table 5 for details.
Graph loss We assess the value of predicting the causal graph, as opposed to only training a supervised classifier (Figure 10). Without modifying the architecture (which has inductive biases towards the graph modality), we train a version of CDN without any graph labels (causal graph learner no longer pretrained; remove graph loss LG). Across the validation loss and various metrics, find that the full CDn more stable, and leads better results. This model corresponds the "no Le" test results in Table 3.
Additional synthetic results During model development, we realized that if the model is allowed to train on all types of perturbations, it can casily overfit and learn these distributional shifts very well. For example, in Table 10, the models trained on both types of soft interventions performs perfectly (unrealistic) in many cases, even though the graphs and causal mechanism parameters are completely novel. Therefore, to assess whether our model can generalize, we chose to hold out a class of soft intervention for our main results in Table
Table 11 reports results larger graphs. There is some drop in performance, especially Polynomial. This could be attributed to error propagation from the causal graph predictor, which performs less well (at predicting graphs) on larger graphs, and could potentially be ameliorated by finetuning on larger synthetic data (similar to how Perturb-seq models were finetuned to be larger), Linear + Soft remains quite good, perhaps because the theory (Appendix C) suggests that the summary statistics are sufficient for predicting targets, without the graph prediction.
Table 12 reports results on all synthetic test datasets, averaging over all interventions for each dataset. We also include the
21

---

Identifying biological perturbation targets through causal differential networks
% ol DBUs not present in GEARS graph
200
150
Count
100
50
0
0.1
0.2
0.3
0.4
% DEGs per perturbation
Figure 9: Percentage of differentially expression genes (per perturbation) that were not in the GEARS knowledge graph, and whose effects could not be predicted.
Target loss
Accuracy
mAP
AUC
0.20
0.900
CDN (no G)
0.960
0.875
0.18
0.76
0.850
0.16
wul m mmm
0.955
0.74
0.825
0.14
0.950
CDN
0.72
CDN
0.800
CDN
CDN
CDN (no
CDN G)
0.70
0.775
1000
2000
3000
0
1000
2000
3000
1000
2000
3000
1000
2000
3000
Step
Step
Step
Step
Figure 10: Ablation study: We compare CDN trained synthetic data, the same model without any ground truth graph information (no pretraining or LG). We find that graph information leads to more stable and better results. Validation metrics and loss. One step is one batch (16 samples).
"full" version of CDN, i.c. trained on all mechanisms, in this analysis.
22

---

Identifying biological perturbation targets through causal differential networks
Inference runtime on Perturb-seq cell lines
Table 6: Runtimes on synthetic datasets (sec).
1000
Nodes Model
Min
Max
Mean
Std
Minutes
800
GEARS
UT-IGSP
98
26
31
600
PDGrapher
DCDI-G
27908
18731
9857
DCDI-DSF
7118
44440
23798
5688
GenePT
DCI
41
1438
404
365
400
10
Ours
BACADI
4005
9483
6284
1584
BACADI-L
1076
1403
1272
76
200
CDN (MLP)
1
CDN
17
147
39
27
K562
RPEI
HepG2
Jurkat
UT-IGSP
78
19
26
DCDI-G
45
46281
35032
16068
Dataset
DCDI-DSF
23181
55406
30076
7200
DCI
325
21414
4415
4707
Figure 11: Inference runtimes on Perturb-seq datasets. K562 datasets are reported together. All models run on a single A6000 GPU, no constraint on memory (up to 500G). Only GEARS required over 200 of memory. To the best of our ability, we normalized batch size to 1.
20
BACADI
15082
50737
26020
9044
BACADI-L
2862
3819
3461
CDN (MLP)
0
CDN
36
86
54
13
Table 7:
Uncertainty quantification on Perturb-seq datasets, seen cell lines, by sub- soob--smpling 80% of cells per perturbation (or 50, whichever is higher). Note that results may be slightly different from Table 1 due sub-sampling. Standard deviation reported over 5 seeds. GENEPT performance is highly variable. DGE is quite sensitive to sub-sampling on K562 genome-wide.
K562 gw
K562
RPE1
Model
rank
rank
rank
r"
LINEAR
0.50#.ou
0.17#.01
0.20#.u1
0.50+.01
0.25+
0.28+.01
0.48+t.0
0.40+.oz
0.49
MLP
0.48
0.171.00
0.20 .00
0.48+.00
0.162.00
0.19
0.53 00
0.30#.00
0.39
DGE
0.72..01
0.481.02
0.52.
0.861.00
0.59 02
0.611.02
0.74:.00
0.561.00
0,641.00
GENEPT
0.541.09
0.261.08
0.29
0.504.08
0.36
0.394.10
0.51
0.43:.08
0.511.07
GEARS
0.504.01
0.191.01 01
0.22::.00 01
0.51+.00
0,20=.01 .01
0.231.00
0.46:.00 00
0.33.1.01
0.411.02
PDG
0.19#.01
0.22#.00
0.25 o0
0.50+t.0
0.28
0.32+.01
0.49
0.38#.01
0.47
CDN
0.91
0.71
0.72.t.02
0.95
0.80
0.81
0.92
0.65
0.74
HepG2
Jurkat
rank
rank
LINEAR
0.49+.ou
0.34+.01
0.42#t.0
0.50+.00
0.23+.u1
0.27+.01
MLP
0.50+.ou
0.34+.00
0.41#.u0
0.474.00
0.25+.u0
0.31+.ou
DGE
0.83#.00
0.49+.01
0.50=.01
0.78+.01
0.44
0.494.02
GENEPT
0.502.13
0.39
0.45...06 .6
0.511.07
0.362.11
0.401.11
GEARS
0.512.00
0.38
0.45.1.00 .00
0.524.01
0.24#..0
0.30+.00
PDG
0.19#.00
0.34+.01
0.41
0.49+.01
0.29=..0
0.34+.00
CDN
0.93#.o0
0.67+.o1
0.73#.01
0.96+.01
0.83
0.85+.03
Table 8: To quantify the uncertainty across draws the test set (perturbations, instead of cells), we partition all perturbations uniformly into sets for the unseen cell line setting on K562 genome-wide. The variation across test sets minimal.
Test Split
rank
Recall @
R
Recall 20
Original (n 678)
0.974
0.915
0.921
0.699
0.900
0.957
New splits (n 353, x5)
0.973 +0.009
0.917 +0.017
0.923
10.016
0.696
#0.081
0,908 #:0.032
0.961 #0.021
23

---

Identifying biological perturbation targets through causal differential networks
Table 9: Results on Perturb-seq datasets, trivial perturbations (intended target has largest log-fold change, though not necessarily smallest adjusted p-value), Top: Train all cell lines jointly. Bottom: Leave one cell line out of training. Test sets (unseen perturbations) are identical in both settings. CDN (synthetic) and DGE do not require training on real Perturb-seq data. Metrics, from left to right: normalized rank of ground truth, top 1 Spearman and Pearson correlations.
K562 (gw)
K562 (es)
RPE1
HepP2
Jurkat
Setting
Model
rank
P
r
rank
p
rank
r
rank
P
rank
p
LINEAR
0.49
0.18
0.21
0.50
0.28
0.31
0.48
0.37
0.45
0.48
0.34
0.43
0.48
0.24
0.29
MLP
0.48
0.16
0.20
0.47
0.17
0.21
0.54
0.29
0.37
0.51
0.35
0.42
0.48
0.26
0.32
Seen
GENEPT
0.60
0.36
0.39
0.47
0.23
0.26
0.57
0.19
0.55
0.44
0.35
0.41
0.55
0.48
0.53
cell line
GEARS
0.49
0.15
0.18
0.51
0.25
0.27
0.49
0.32
0.40
0.48
0.32
0.40
0.47
0.20
0.24
PDG
0.49
0.23
0.26
0.52
0.31
0.34
0.51
0.34
0.42
0.46
0.34
0.40
0.49
0.30
0.35
CDN (Perturb-seg)
0.99
0.97
0.97
0.99
0.98
0.98
0.99
0.93
0.94
0.99
0.95
0.96
0.96
0.96
DGE
0.87
0.68
0.70
0.92
0.77
0.79
0.85
0.73
0.77
0.89
0.59
0.60
0.91
0.73
0.77
Unseen
PDG
0.40
0.12
0.15
0.41
0.22
0.24
0.48
0.28
0.37
0.47
0.33
0.41
0.53
0.28
0.33
cell line
CDN (synthetic)
0.91
0.72
0.74
0.92
0.83
0.84
0.94
0.83
0.86
0.90
0.73
0.77
0.95
0.82
0.84
CDN (Perturb-seq)
0.98
0.95
0.95
0.99
0.95
0.96
0.99
0.92
0.94
0.99
0.95
0.96
0.99
0.96
0.96
Table 10: Intervention target prediction results synthetic datasets with N = =700 E 40. Top: hard interventions (uniform); bottom: soft interventions (scale). We found that if the model is allowed to see the class of soft intervention ("all mechanisms") during training, it can easily learn (overfit) the distribution shift perfectly, even though test datasets and graphs are novel. Thus, our main results in Table 3 treat "scale" as an unseen soft intervention.
Linear (1)
Linear (3)
Polynomial (1)
Polynomial (3)
Type Setting
mAPt
AUC1
MAPt
AUC
mAPt
AUC1
mAPt
AUC1
all mechanisms (Aea))
0.86.1.09
0.97
0.91
0.971.03
0.86 04
0.974.01
0.91
0.97:101
Hard
all mechanisms (haa)
0.78#t.0
0.95+.03
0.89=.04
0.96#.02
0.83= 05
0.962.01
0.89: o3
0.96#t.02
exclude scale
0.82#.05
0.96+.02
0.88=.04
0.95+.02
0.85= 05
0.97
0.90#.04 04
0.96#.02
all mechanisms (Aear)
1.0#t.00
1.0+t.0
0.99
1.0+.00
1.0=.00
1.0#t.00
1.0+t.0 .0
1.0#t.0
Soft
all mechanisms (harr
0.99
1,0...0
0.98
0.99
1.0=.00
1.0.1.00
1.0::.00
1.0::.00
exclude scale
0.83.t.10
0.95..05
0.76..07
0.89
0.97 .03
1.0:..0
0.96...0
0.994.01
Table 11: While the majority of classic causal discovery models struggle to scale to larger graphs, we show that CDN still performs reasonably on N 100, = 200 graphs. AUC reported over 10 i.i.d. graphs.
Intervention
Linear (1)
Linear (3)
Sigmoid (1)
Sigmoid (3)
Polynomial (1)
Polynomial (3)
Hard
.64 + .04
.65 + 008
.63 + .02
.61 + .06
.50 + .01
.49 + .03
Soft
.92 + .01
.90 = .03
.72 = .02
.73 + .02
.53 + .03
.52 + .05
24

---

Identifying biological perturbation targets through causal differential networks
Table 12:
Intervention target prediction results on synthetic datasets, extended results. Uncertainty is standard deviation over i.i.d. datasets. Metrics are averaged over all 3/ perturbations for a given dataset (1-3 targets). CDN (full) denotes that we trained on all intervention mechanisms jointly, i.c as in Table 10.
N E Model
Linear (Hard)
Lincar (Soft)
Poly, (Hard)
Poly. (Soft)
Sigmoid. (Hard)
Sigmoid (Soft)
mAPt
AUC1
mAPt
AUCT
mAP
AUC1
mAPt
AUCT
mAPt
AUC1
mAPt
AUC1
UT-IGSP
29+.02
.574.08
.30
.63+.02
.334.04
.594.03
.32..0
.64+.02
.28..03
.584.04
.262.u2
.59+.03
DCDI-G
.94.04
.22..04
.40..01
.511.04
.38..03
.492.02
.62.03
.99..0
374.03
.99..08
.402.01
.0
.52..08
DCDI-DSF
361.02
.511.04
.40..05
.53..05
.37..02
.50..01
.38.002
.99..33
.37.002
.50..02
.37=.02
.481.03
BACADI-E
261.00
.611.05
.42..1
.631.11
.251.03
.61..05
.00.05
.611.04
.88..44
.641.06
.43:.12
.691.09
BACADI-M
.33.01
.564.03
.37.11
.62.11
.224.02
.562.04
.0
.61..04
.54.03
.614.05
.422.1
.69+.09
10 10
DCI
554.08
.792.04
.502.08
.702.05
.472.09
.712.06
.1
.67..01
.54.07
.80..0
.512.05
.762.04
MR+CI
534.05
.644.03
.572.10
.66
.704.09
.79=.07
.85
.89..0
844.08
.882.06
.96
.972.04
CDN
864.02
.31
.742.06
.85..08
.832.09
.11..5
.89 05
.94.00
.88.06
.44..04
.852.0r
.91
CDN (no La)
.77.09
.782.07
.822.05
.89..05
.824.08
.912.0
.91
.96
.89.05
.952.03
.94=.03
.972.02
CDN (full)
874.01
.94..o
.99+.u1
1.0+.00
.82+.08
.91+.um ..
1.0+.om on
1.0
.95..05
.98#..2
1.0=..0
1.0+.00
UT-IGSP
288..2
.564.08
.26
.594.03
.284.03
.584.04
.04
.27 01
600..1
25...0
.554.08 .03
.25 02
.56+.05
DCDI-G
.22.022
.544.02
.40
.51+.04
.40
.532.05
.92
.51
374.02
.50
.35=.03
.49
DCDI-DSF
.11.03
.522.03
.39
.52..05
.39..02
.51..02
.32
.50..0
.34.03
.502.05
.362.03
.492.03
BACADI-E
344.04
.684.03
.532.08
.71..04
.33..0
.722.04
.64
.78..05
.34..05
.712.04
.51
.732.04
BACADI-M
.77..u
.63+..u
.88..99
.71+.04
.27..04
.644.u8
.59
.77+.06
.284.03
.655.uu
.45#.us
.71+.05
20
DCI
59+.0a
.78+.u8
.57
.77+.03
.684.07
.82+.04
.65+.us
884.05
.70+.03
844..2
.66=.ue 0o
.834.00
MB+CI
.4+.11
.664.09
.9
.62t.04
.83+.04
.87..03
.06
.96..22
.804.99
.84..88
.96=.u2
.96+.02
CDN
88+.04
.54.022
.56
.71+.07
.64..22
.98
.952.04
.97
.44.04
.884.02
.68=.02 .2
.79+.02
CDN (no La)
.9+..0
.00..04
.77..04
.86+.02
.4+.01
.97..o
.95
.96+.04
.87..02
.93+.02
.94
.96..01
CDN (full)
894.03
.44.02
.98
.99
.97..01
.99-.00
1.0
1.0+.00
.64..03
.984.o1
.98=.o2
.99+.01
UT-IGSP
152.01
.54.01
.18
.65..02
.20..02 02
.602.02
.21 02
.67..02
.15..01
.57..02
.17
.65
DCDI-G
241.02
.49..02
.22..02
.99..2
.221.03
.50..03
.22..02 02
.49..03
.22..02
.502.04
.222 02
.50..03
DCDI-DSF
214.02
.50..02
23
.30.03
.22+.01
.504.02
.20
.8+.02
.22..2
.504.04
.21
.77.03
BACADI-E
.14.01
.61..08
.25
.60..o
.22..06
.762.06
.30
.74..0
.15..02
.652.03
.17=.00
.57+.00
BACADI-M
.12+:00
.58+.02
.172.03
.60+.05
.14.02
.66=.on
.22..0 .5
.71+.06
.12..01
.594.04
.14.002
.56+.05
20 20
DCI
A3t.03
.774.01
.52..8
.752.04
.84..5
.762.03
.502.02
.79+.02
.14..55
.752.02
.382.12
.73+.06
MB+CI
484.09
.664..8
..7
.70+.05
.72+.10
.84 06
.92 04
.94..02
.684.06
.804.04
.89 .03
.92+.02
CDN
.74..02
.90..01
.854.06
.964.02
.78+.09
.924.04
.92+.02
.98
.754.06
.914.03
.78
.92+.04
CDN (no La)
.04..s
.73+.07
.87+.03
.964.01
.764.09
.90+.0b .0.
.92+..a
.03
.98
.71+.04
.894.02
.2
.93+ 02
.99+.00
CDN (full)
.75+.03
.91..2
.99
1.0+.00
.79+.08
.93+.04
1.0+ .00
1.0+.00
.764.05
.92+.u2
1.0=.00
1.0+.00
UT-IGSP
.14.01
.574.0
.14..0
.61..02
.20+.01
.622.o2
.192.m m
.65..01
.134.01
.562.02
.142.00
.60
DCDI-G
22+.03
.49..03
.2
.84.03
.22..02
.92.02
.22 02
.50..02
.21..01
.88..02
.23
.51 2.02
DCDI-DSF
211.02
.46..03
.21
.72.02
.23+.02
.502.0
.21..
.94..3
.21..02
.492.02
.21
.49.022
BACADI-E
.33..05
.704.04
.0
.674.04
.294.08
.834.04
.45
.784.08
.20..5
.734.05
.51
.79+.04
BACADI-M
.15..01
.644.02
.194.01
.63+.02
.174.02
.704.06
.31+.1
.77+.07
.14.011
.654.03
.32+.06
.77 4.7..0
20 40
DCI
604.06
.854.u
.50
.77+.03
.564.08
.81+.u
.59+.u6
.84..4
.644.66
.87+.u2
.66+
.88+.03
MB+CI
.53..99
.70+.06
.47
.644.03
.784.04
.88+.02
.95+..3
.964.02
.834.03
.894.02
.96+.u4
.97
CDN
.864.04
.6+..u2
.79
.93+.os
.884.03
.97
.96
.99+.00
.884.03
.974.o1
.71
.87+.04
CDN (no La)
.694.09
854.07
.80..6
.941.03
.841.03
.94..01
.95
.991.00
.841.05
.931.03
.91=.02
.98
CDN (full)
.894.06
.974.02
.99
1.04.00
.894.03
.97..01
1.02.00
1.04.00
.931.03
.984.01
1.0=.00
1.0+.00
25


Reviews:
[{"review_id": "r0iH5BhpRR", "reviewer": "Reviewer_1GJy", "summary": "The paper introduces causal differential networks (CDN), a supervised method designed to identify which variables (eg. genes) were directly targeted by biological perturbations (genetic knockouts or chemical treatments) using gene expression data from single-cell transcriptomics experiments. CDN integrates a causal graph inference model (based on the SEA model architecture) and a differential perturbation classifier into a joint end-to-end framework. The causal graph inference module estimates causal graphs from observational and perturbed conditions separately; a subsequent attention-based differential module identifies intervention targets by comparing inferred causal graphs and related statistics (eg. mean, covariance, correlation).\n\nCDN employs a Transformer-based architecture with specialized axial attention layers to handle graph data in a permutation-invariant manner, allowing scalability and robustness. Training involves pre-training on synthetic causal graphs and fine-tuning on real single-cell transcriptomics data. CDN was tested on five genetic perturbation datasets (perturb-seq) and two chemical perturbation datasets (sci-plex). The model is shown to outperform sota causal discovery and deep-learning baselines on real genetic perturbations and showed reasonable success in the more challenging chemical perturbation setting.\n\nThe main findings demonstrate that CDN achieves better identification accuracy than prior methods, generalizes well to unseen cell lines with minimal drop in performance, and handles both hard (knockouts) and soft (scaling) interventions effectively. The paper provides a good experimental evaluation on real and synthetic data to validate CDN's performance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your questions and recommendations! We hope this response addresses your questions, but please let us know if you have any additional concerns.\n\n> Additional historical context\n\nThank you for these references and suggestions. We will incorporate them into the paper alongside additional discussion of the historical context.\n\n> Extension to dynamical systems\n\nAn empirical solution is to simulate cyclic data, and to use a classic discovery algorithm for cyclic graphs. \n\nWe can also model the dynamical system. In perturbation data, cells are sequenced after perturbations penetrate completely, e.g. 6-8 days [1], so we might assume that the data are sampled from the stationary density of a system of stochastic differential equations [2]. Here, the causal structure learner could predict the SDE parameters instead of the causal graph. Similar to the acyclic case, interventions could be \"read off\" the differences in the SDE parameters, e.g. $f$ and $\\sigma$ in [2]. The key design choice/challenge would be to select an input featurization that allows for the targets to be identified.\n\nFor actively differentiating cells (instead of pre/post differentiation), steady state assumptions might not hold. While there exist measurements like RNA velocity (the ratio of spliced to unspliced mRNA) that can partially order cells within a population and inform of causal structure [3, 4], modeling these continuous dynamics is out of the scope of this work.\n\n[1]. Replogle et al. Mapping information-rich genotype-phenotype landscapes with genome-scale Perturb-seq. Cell. 2022.\n\n[2] Lorch et al. Causal Modeling with Stationary Diffusions. AISTATS 2024.\n\n[3] Singh et al. Causal gene regulatory analysis with RNA velocity reveals an interplay between slow and fast transcription factors. Cell Systems 2024.\n\n[4] Atanackovic et al. DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNets. NeurIPS 2023.\n\n> Sensitivity to simulation\n\n- CDN is not sensitive to the type of non-linearity among graphs of the same size, as performance remains about the same for novel non-linearities (Polynomial, Table 3 and Sigmoid, Table 10).\n- CDN is more sensitive to the type of intervention (Table 9), so this is a key parameter to understand / simulate.\n- Mismatch issues: During development, we found that finetuning on Perturb-seq was not helpful for the Sci-Plex data, presumably because CRISPR perturbations are much cleaner and stronger than drugs. We did design the synthetic datasets with less specific / soft interventions in mind [below], which may render them a better fit.\n\n**Sim-to-real gap:** If a model has been trained and one would like to simulate additional finetuning data, one could be informed by the data-driven graph connectivity (e.g. based on pairwise correlation) and classic hypothesis tests (e.g. for data distribution, differences between pre/post intervention).\n\nHowever, these heuristics cannot fully capture complex systems like biology. Instead, perhaps one could use a self-supervised objective to finetune the model on observational data, a la test-time training approaches [5]. Alternatively, one could incorporate biological knowledge graphs with the understanding that they are *noisy* priors. E.g. as an additional input feature or as a finetuning objective for the causal graph learner.\n\n[5] Sun et al. Test-Time Training with Self-Supervision for Generalization under Distribution Shifts. ICML 2020.\n\n> Interpretability of intermediate graphs\n\nSince the causal graph learner is supervised on synthetic graphs, the predicted graphs are always available for inspection. This is not a primary focus of this work, due to the lack of cell line-specific \"ground truth\" graphs for evaluation. However, the predicted edges could represent gene regulatory relationships, and the differences between the observational and interventional graphs correspond to differences in gene regulation.\n\n> Multi-target identification\n\nCurrently, CDN is trained to make an independent prediction per gene (sigmoid, not softmax, 242-244). The synthetic experiments contain up to 3 targets per graph, and the model does output multiple high scores (Table 3).\n\nA limitation of this current approach is that the per-gene decision does not consider other genes in the proposed subset. To (efficiently) consider the entire subset, the prediction could be made autoregressively, where the order of predictions could be based on effect size, or be sampled at random during training.", "claims_and_evidence": "The paper's key claims, e.g. superior accuracy in identifying intervention targets, robustness to unseen biological domains, and improved performance over traditional causal discovery algorithms, are well supported by multiple experiments on real single-cell datasets and carefully designed synthetic benchmarks. CDN consistently ranks the true targets higher and shows stronger correlation between predicted and actual intervention effects compared to multiple sota baselines.\n\nI think the claim of generalization is especially convincing, backed by experiments explicitly evaluating CDN's performance on unseen cell lines, showing minimal performance drop. Similarly, CDN's ability to detect multiple simultaneous targets and handle different intervention types (hard and soft) is well supported by synthetic experiments. I believe the authors appropriately interpret results without exaggeration or unsupported claims.", "methods_and_evaluation_criteria": "The proposed CDN methodology, including the causal graph inference and perturbation classifier modules, is well justified for the problem domain. Using a combined graph representation and statistical features to capture intervention effects is sensible, given biological perturbation complexity. The authors provide clear details about model architecture, training, validation, and evaluation metrics.\n\nThe evaluation strategy is detailed, comparing CDN against a good range of baselines, including both domain-specific (eg. GEARS, PDGRAPHER) and general causal inference methods. Real data seem to be carefully preprocessed, with sensible candidate filtering and careful splitting to prevent data leakage. Metrics chosen (normalized rank, correlation, recall@p, and synthetic mAP/AUC) are standard and appropriate for capturing model accuracy.", "theoretical_claims": "The paper provides theoretical justifications rather than formal proofs. I (lightly) checked some key derivations ( Appendix C) showing how interventions affect graph structure and global statistics (correlation and covariance) and found them sound.", "experimental_designs_or_analyses": "The experimental design is appropriate, and well executed in my opinion. Data preprocessing, perturbation candidate selection, and train-test splits were carefully described and justified. Filtering trivial perturbations and ensuring splits by perturbation-effect clustering help prevent overfitting and leakage.\n\nMetrics chosen are appropriate for the task. The authors carefully assessed uncertainty through repeated runs or bootstrapping. CDN's advantage over baselines was large enough that minor statistical variations did not affect conclusions. The design explicitly tested CDN’s ability to generalize across cell lines and to handle different perturbation types, with thoughtful analyses like ablations and runtime measurements.", "relation_to_broader_scientific_literature": "I think the authors clearly position its contributions, distinguishing CDN's supervised, amortized causal discovery approach from earlier unsupervised causal discovery methods (UT-IGSP, DCDI, BACADI) and network-based or knowledge-driven perturbation target methods (GEARS, PDGRAPHER). CDN is appropriately recognized as building upon concepts from differential causal network analysis (e.g. DCI), extending them into high-dimensional, non-linear single-cell data.", "essential_references_not_discussed": "I think the paper could benefit by further contextualizing contributions in bioinformatics works on perturbation signature matching, earlier and more recent differential network approaches. Including these references would provide a bit historical context, emphasizing how CDN builds upon past methods and distinguishes from recent ones.\n\n1. **Lamb et al. (2006). \"The Connectivity Map: using gene-expression signatures to connect small molecules, genes, and disease.\"**\nThis work introduced the idea of connecting small molecule drugs to gene targets (and diseases) via gene expression signatures​. It laid the groundwork for using transcriptomic comparative analysis to infer drug targets (e.g. if a drug's expression profile is similar to that of a gene knockout, that gene might be the drug's target)​. \n\n2. **Noh & Gunawan (2016). \"Inferring gene targets of drugs and chemical compounds from gene expression profiles\".** \nI think this work is directly relevant, as it proposed DeltaNet, a method to infer drug targets by solving a regression problem linking expression changes to putative targets (without needing to first infer a full network)​. It outperformed network-based methods like MNI and SSEM, showing the value of a direct data-driven approach​. DeltaNet is, in a sense, a precursor to CDN but using a linear model and least-angle regression instead of a learned neural network​.\n\n3. **Wang et al. (2018). \"Direct estimation of differences in causal graphs.\"** I believe the DCI algorithm used in the cited work (Belyaeva\net al., 2021) stems from this paper from Wang et al.\n\n4. **Chen et al. (2023). \"iSCAN: identifying causal mechanism shifts among nonlinear additive noise models.\"** Similar setting to DCI but extends to nonlinear additive models.\n\n5. **Malik et al. (2024). \"Identifying causal changes between linear structural equation models.\"** Similar to DCI, the authors study the linear setting for difference networks but provide more analyses on the graphical conditions leading to identifiability of the intervention targets as well as the difference edges. Also related to the cited work (Varici et al., 2022).", "other_strengths_and_weaknesses": "### Strengths:\n\n* Original integration of causal structure learning with neural network prediction.\n* Rigorous experimental validation, detailed ablation and robustness analysis.\n* Good clarity, strong statistical robustness, transparent details aiding reproducibility.\n* Real-world biological application with clear potential impact.\n\n### Weaknesses:\n\n* Dependence on synthetic data realism and labeled perturbation targets (maybe potential domain adaptation challenge).\n* Implicit assumption of DAG structure; handling cycles not explicitly addressed.\n* Limited interpretability of intermediate causal graphs.", "other_comments_or_suggestions": "None", "questions_for_authors": "I would appreciate if the authors can comment on these questions:\n\n1. How does your method handle situations where the true causal structure is not acyclic? Biological pathways often contain feedback loops or reciprocal interactions. In such cases, I image CDN's performance might largely drop and a dynamic or time-series approach would be needed. How challenging is extending CDN's to dynamic causal graphs?\n\n2. Your training strategy heavily relies on simulated data to teach the model causal patterns. How sensitive is CDN to the choice of simulation parameters and assumptions? For instance, if the real data had a fundamentally different type of causal dynamics (e.g. non-linearities not in your simulation), would CDN struggle? Did you observe any mismatch issues when fine-tuning on real data? If one cannot perfectly simulate a domain, how would you recommend mitigating simulation-to-reality gaps? \n\n3. CDN is trained to output a set of targets. In your synthetic experiments it handled multiple simultaneous interventions. In real-world cases like drug perturbations, a drug can have multiple targets (on-target and off-target proteins). How well would CDN handle identifying multiple true targets? Is it capable of assigning high scores to more than one gene if multiple are responsible for the observed changes? If not directly, what modifications would be needed (e.g. a different loss or thresholding criterion) to allow multi-target identification?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CeIVfO8kiJ", "reviewer": "Reviewer_mWJN", "summary": "This paper proposes two modules for predicting true perturbation targets of cell perturbation datasets. Firstly, a causal structure learner is pre-trained on synthetic data in an attempt to recover causal graph representation from both observational and interventional dataset. Then, a differential network is trained on both synthetic data and downstream real-world cellular data to predict the intervention targets.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments! Please let us know if this response helps clarify your questions, and if you have any further concerns.\n\n> Technical novelty\n\nOur key contribution is the idea of supervised causal discovery (predicting graphs from simulated data) as a pretraining objective, to obtain dataset -> graph representations useful for downstream tasks like inferring targets. A direct solution to the task is to predict targets without inferring graphs. However, we empirically demonstrate that graph prediction leads to substantial improvement as an auxiliary loss (Table 3, no $L_G$) and starting from pretrained weights leads to more stable optimization dynamics (Figure 10). Furthermore, baselines that use data-mined graph priors perform substantially worse (Table 1, Figure 3).\n\nWe realize our idea with [1] as the causal structure learner, and a 2D Transformer-based architecture as the differential network, *neither of which* are our primary contributions. These modules were selected for their scalability and could (in principle) be replaced with alternate supervised learning algorithms [2,3] or models that operate pairs of graphs [4].\n\n[1] Wu et al. Sample, estimate, aggregate: A recipe for causal discovery foundation models. 2024.\n\n[2] Ke et al. Learning to Induce Causal Structure. ICLR 2023.\n\n[3] Lorch et al. Amortized inference for causal structure learning. NeurIPS 2022.\n\n[4] Li et al. Graph Matching Networks for Learning the Similarity of Graph Structured Objects. ICML 2019.\n\n> Identifiability\n\nOur setting is slightly different from the typical causal representation learning setting, where high-dimensional observations (e.g. images) are associated with (unobserved) low-dimensional factors, on which interventions are applied.\n\nWe follow the classic causality framework (Sec 2.1), where we measure all variables (causal sufficiency), and interventions are applied directly to these variables. Many classical causal discovery algorithms apply to the observational setting, in which certain graph topologies are still identifiable, under various assumptions [5].\n\nSupervised causal discovery algorithms do not have explicit assumptions, but they do follow implicit ones captured by the training data [6].\n\n[5] Sprites, Glymour, and Scheines. Causation, prediction, and search. 2001.\n\n[6] Montagna et al. Demystifying amortized causal discovery with transformers. 2024.\n\n> Value of synthetic data.\n\nEmpirically, the baselines that predict targets from real data without external information / synthetic data do not perform as well as our full method (Table 1). The synthetic soft interventions were loosely inspired by perturbations, e.g. in terms of fold-changes (796). In general, synthetic data pretraining also improves training stability (Figure 10).\n\n> Bidirected edges\n\nThe model's input and output spaces are different. The causal graph learner outputs only three edge types: \"A -> B,\" \"B -> A,\" \"no edge.\" FCI graph estimates are *inputs* to the model, along with correlation. The input edge type embedding includes bidirected edges because FCI may estimate them.\n\n> Prior work\n\nThank you for the recommendation! We will include this in the discussion of related work.\n\n> L75-80 right\n\nThese lines describe how a perturbation effect \"simulator\" could be used to identify targets. The actual implementation for GEARS and GenePT is described in B.2. We will update this section to be more descriptive and add a reference.\n\n- \"space of perturbations $\\mathcal{I}$\" refers to the set of sets of perturbation targets $I$ that are possible, e.g. the powerset of all variables, if we allow interventions of arbitrary # of targets\n- \"an oracle $\\phi : D_\\text{obs}, I \\to P$ refers to the simulator, which takes as input, an observational dataset (starting cells) and a set of intervention targets (genes)\n- \"the $I^*$ associated with $\\tilde{P*}$ refers to the set of ground truth targets, associated with the target distribution (Eqs 1 and 2)\n- \"dissimilarity d\" could be cosine distance, or mean squared error, or KL divergence between distributions\n- $\\arg\\min$ over $\\mathcal{I}$ implies an optimization over all possible sets of targets, e.g. naively we run the model $\\phi$ once per $I$ and output the $I$ whose prediction is closest to the target cells.\n\n> What are k and T?\n\nWe will update the paper to clarify. FCI is run $T$ times on different subsets of $k$ nodes to produce subgraph estimates (input to the causal graph learner)\n\n> Differential network\n\nThe differential network architecture is implemented using standard multi-head attention, applied along each axis. We will expand the descriptions in the main text and include references to the appropriate details.\n\n> Conclusion is missing\n\nWe will add a conclusion to highlight the key contributions, limitations, and future directions.", "claims_and_evidence": "The model architectures look pretty clear and straightforward to me, which is good. However, for the same reason, I'm not quite sure where the novelty lies. For causal structure learner, the authors stated that the proposed method is based on [1], then what is the difference of the causal structure learner in this work compared to [1]? And how does this difference improves on [1]? I don't think the novelty & contribution of this work on top of prior works is stated very clearly. [1] seems like an unpublished work and I don't think it is particularly well-known and well-accepted in the causal discovery community to the point that you can omit stating your contribution in comparison to it.\n\n\n[1] Wu,M., Bao,Y., Barzilay,R., and Jaakkola,T. Sample, estimate, aggregate: A recipe for causal discovery foundation models. arXiv2402.01929,2024.", "methods_and_evaluation_criteria": "The causal structure learner is quite heuristic and theoretically loose. It is generally well-known that identifying causal structure from observational dataset alone is not possible [1] and some additional data / prior knowledge / assumptions need to be present [2][3]. I do not believe pre-training the causal structure learner on unrelated synthetic dataset with true synthetic causal graph is going to help model's ability to identify causal representation on observational cellular datasets in the 2nd module, unless the synthetic dataset contains some true dynamics about cell perturbations. Therefore, I think $h_{obs}$ is just a graph representation that contains correlation information about the features in the observational dataset, not causal information. Besides, why is double edge allowed as an edge type (L196 left side)? Is causal structure not represented as directed acyclic graph?\n\n\n[1] Locatello, Francesco, et al. \"Challenging common assumptions in the unsupervised learning of disentangled representations.\" international conference on machine learning. PMLR, 2019.\n\n[2] Brehmer, Johann, et al. \"Weakly supervised causal representation learning.\" Advances in Neural Information Processing Systems 35 (2022): 38319-38331.\n\n[3] Shen, Xinwei, et al. \"Weakly supervised disentangled generative causal representation learning.\" Journal of Machine Learning Research 23.241 (2022): 1-55.", "theoretical_claims": "No theoretical claims.", "experimental_designs_or_analyses": "The soundness and validity of experimental designs are fine to me.", "relation_to_broader_scientific_literature": "This work is related to prior work that uses deep learning to predict intervention targets on cell perturbation dataset, such as [1]. The authors pointed out that it is also related to prior work that performs cellular response prediction such as [2], [3], through which intervention target can be implicitly inferred. More broadly, it is related to prior work in causal discovery and causal representation learning as the ones I cited in the \"Methods And Evaluation Criteria\" section.\n\n\n[1] Gonzalez, Guadalupe, et al. \"Combinatorial prediction of therapeutic perturbations using causally-inspired neural networks.\" bioRxiv (2024).\n\n[2] Lotfollahi, Mohammad, et al. \"Predicting cellular responses to complex perturbations in high‐throughput screens.\" Molecular systems biology 19.6 (2023): e11517.\n\n[3] Hetzel, Leon, et al. \"Predicting cellular responses to novel drug perturbations at a single-cell resolution.\" NeurIPS (2022).", "essential_references_not_discussed": "Key prior work [1] that outperformed CPA with convincing margins should be cited and discussed on L64-67 right side.\n\n[1] Wu, Yulun, et al. \"Predicting cellular responses with variational causal inference and refined relational information.\" ICLR (2023).", "other_strengths_and_weaknesses": "Strengths:\n\nGood evaluation results are shown for the proposed method.", "other_comments_or_suggestions": "1. I think the authors could be more clear in terms of details on equations and notations. For example, it is not quite clear to me how intervention target is inferred by the equation and descriptions on L75-80 right side, since the notations are not clearly defined and explained. I can only assume that given a cell, it is the intervention whose empirical response distribution on the dataset is the most correlated with this cell's expression profile? Besides, $k$ and $T$ on L195-198 left side is also quite confusing to me as there is no explicit explanations of what they are. For differential network, there is only a figure and verbal explanations regarding the mechanism but no exact equations on the attention and aggregation mechanisms. I managed to find the attention mechanism in Appendix C but it is not pointed to by the main text.\n\n2. The conclusion section is missing.", "questions_for_authors": "Questions are embedded in the previous sections.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qM6sZCTuCD", "reviewer": "Reviewer_gPde", "summary": "The paper proposes a new method called causal differential networks (CDN) for identifying variables responsible for biological perturbations (e.g. drug targets) from pairs of observational and interventional datasets. The main idea involves first inferring noisy causal graphs from data, and then learning a mapping from these causal graphs to the set of variables subjected to intervention. The authors demonstrate that CDN significantly outperforms existing methods across seven single-cell transcriptomics datasets and synthetic benchmarks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments! Please let us know if these answers are helpful, and if you have any further concerns.\n\n> Top 1000 DEGs\n\nIt's quite common to restrict analysis to subsets of genes, though the threshold and criteria may vary. Here, we believe that selecting the top 1000 genes (based on differential expression p-value) is reasonably permissive, since these genes are selected per perturbation.\n\nFor example, [1] writes: \"In a genome-scale Perturb-seq screen, we find that a typical gene perturbation affects an estimated 45 genes, whereas a typical essential gene perturbation affects over 500 genes.\"\n\nIn addition, [2] writes regarding highly-variable gene selection (global selection across all perturbations, based on variance to mean ratio): \"Depending on the task and the complexity of the dataset, typically between 1,000 and 5,000 HVGs are selected for downstream analysis [...]. Preliminary results from Klein et al (2015) suggest that downstream analysis is robust to the exact choice of the number of HVGs. While varying the number of HVGs between 200 and 2,400, the authors reported similar low‐dimensional representations in the PCA space.\"\n\nFinally, a classic approach for identifying drug targets [3] argues that targets are not highly differentially expressed, but network-based approaches are useful for predicting them from gene expression. There, they also sample 1000 candidate targets as decoys.\n\n[1] Nadig et al. Transcriptome-wide characterization of genetic perturbations. 2024.\n\n[2] Luecken and Theis. Current best practices in single‐cell RNA‐seq analysis: a tutorial. Mol Syst Biol (2019) 15: e8746.\n\n[3] Isik et al. Drug target prioritization by perturbed gene expression and network information. Scientific Reports 2015.\n\n> Larger synthetic experiments\n\nWe are happy to include experiments on larger graphs ($N=100, E=200$, AUC over 10 new datasets). There is some drop in performance, especially Polynomial. This could be attributed to error propagation from SEA, which performs less well (at predicting graphs) on larger graphs, and could potentially be ameliorated by finetuning on larger synthetic data (similar to how Perturb-seq models were finetuned to be larger). \n\nLinear + Soft remains quite good, perhaps because the theory (Appendix C) suggests that the summary statistics are sufficient for predicting targets, without the graph prediction.\n\n| Intervention | Linear (1) | Linear (3) |  Sigmoid (1) | Sigmoid (3) | Polynomial (1) | Polynomial (3) |\n|--|--|--|--|--|--|--|\n| Hard | .64±.04 | .65±.08 | .63±.02 | .61±.06 | .50±.01 | .49±.03 |\n| Soft |  .92±.01 |.90±.03 | .72±.02 |  .73±.02 | .53±.03 | .52±.05\n\n> Different test splits\n\nTypically, the largest cluster (assigned to train, e.g. nearly 1/3 of K562) contains weak perturbations with minimal phenotype, so different clusterings don't introduce as much heterogeneity.\n\nInstead, we can partition all perturbations uniformly into 5 sets for the unseen cell line setting. As seen here for K562 genome-wide, the variation across sets is minimal. We will include similar results for all cell lines.\n\n| Split | Rank | $\\rho$ | $r$ | Recall-1 | Recall-5 | Recall-20\n|--|--|--|--|--|--|--|\n| Original Test (n=678) | 0.974 | 0.915 | 0.921 | 0.699 | 0.900 | 0.957 |\n| New splits (n=353 x5)| 0.973 | 0.917 | 0.923 | 0.696 | 0.908 | 0.961 |\n| Std of above | 0.009 | 0.017 | 0.016 | 0.031 | 0.032 | 0.021", "claims_and_evidence": "I believe the claims made in the submissions are supported by convincing evidence (up to some reservations on the experimental setting).", "methods_and_evaluation_criteria": "Yes the proposed methods and benchmark datasets are very relevant to the application at hand (target prediction).", "theoretical_claims": "There are no theoretical claims in the paper.", "experimental_designs_or_analyses": "I have some doubts regarding the experimental setting: \n\n- The authors restrict the target search to the top 1000 differentially expressed genes. This choice can bias the evaluation because if the true perturbation target is already among those top candidates, even a simple baseline like differential gene expression (DGE) can appear very competitive. It’s not entirely clear whether this candidate restriction reflects a realistic use case or if it artificially favors methods that exploit strong differential signals. \n- The synthetic experiments are conducted on random graphs with only 10–20 nodes, which is a much simpler setting than the high-dimensional real transcriptomic data. Given the scalability of the proposed method, I would expect the authors to evaluate their approach on larger graphs.\n- The training and test splits are created by clustering perturbations based on log-fold change profiles. It would be useful if the authors conducted a sensitivity analysis—comparing results across various clustering methods (e.g., different k values in k-means or random initializations) to demonstrate that their conclusions hold regardless of how the data is partitioned.", "relation_to_broader_scientific_literature": "The key contributions are well discussed in the context of the broader scientific literature.", "essential_references_not_discussed": "To my knowledge, all the essential references are discussed.", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "- Is restricting the target search to the top 1000 differentially expressed genes a standard practice in the literature?\n- How sensitive are the results to the choice of clustering algorithm or the number of clusters (e.g., different k values in k-means)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UVxusMeIRi", "reviewer": "Reviewer_K5QP", "summary": "This paper introduces Causal Differential Networks (CDN), a method to identify which variables in a biological system are directly perturbed between observational and interventional conditions. The authors frame the problem in terms of causal discovery: they first learn approximate causal graphs from each condition, then use an attention-based neural network to detect which nodes have shifted causal mechanisms, thus identifying the perturbation targets. The pipeline is trained end-to-end in an amortized fashion using both simulated and real single-cell transcriptomics data.\nExperimental results suggest that this approach achieves superior accuracy in identifying causal perturbation targets compared to the baselines, both on large-scale real datasets of genetic and chemical perturbations and on carefully controlled synthetic datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments! Please let us know if this response helps clarify your questions, and if you have any further concerns.\n\n> Intended application\n\nWe will make this more explicit in the paper. This work focuses on the application of predicting perturbation targets, for efficient experimental design. The goal is not to replace experiment via simulation, but to lower the cost/scale of experiments, by reducing the search space.\n\nWe envision for actual use cases to resemble the chemical perturbation case more, e.g.\n- Inputs: before/after drug (unknown mechanism)\n- Model generates hypotheses for molecular mechanisms of drugs\n- Top predictions should be verified experimentally, i.e. knock down proposed genes and compare to \"after drug\" phenotype.\n\nThis type of method has an advantage to phenotype prediction methods when considering combinations of targets, e.g. messy drugs. Different off-targets may contribute orthogonal effects, which may make the predicted phenotype of any single target less similar to the overall profile. In addition, there are limited real training data for combinatorial perturbations, motivating methods that can leverage synthetic data.\n\n> Why Perturb-seq?\n\nSome experiments use Perturb-seq datasets because in terms of model evaluation, the ground truth targets are relatively \"clean.\" CRISPR is very specific compared to drugs, which may impact many proteins, known and unknown. While this is not the most \"realistic\" application, we chose these datasets for their size and ease of evaluation.\n\n> Single cell foundation models\n\nWe will include discussion of these works. Thank you for the suggestion!\n\n> Generalization under distribution shift\n\nOur \"unseen cell line\" case resembles \"zero-shot\" generalization, since experiments in different cell lines are necessarily conducted separately (282). While these datasets were produced by the same lab, they are actually quite incomparable at the level of individual genes, even for the same cell line. For example, [1] writes of K562 genome-wide and essential: \"The median correlation between log-fold-change point estimates [...] was only 0.16\"\n\n[1] Nadig et al. Transcriptome-wide characterization of genetic perturbations. 2024.\n\nIn controlled settings, our synthetic experiments hold out the \"scale\" soft intervention from training (368 right, Table 9). We include the exact formula for these interventions / causal mechanisms in D.2 (p16-17).\n\n> Interpretability of intermediate graphs\n\nSince the causal graph learner is supervised on synthetic graphs, the predicted graphs are always available for inspection. This is not a primary focus of this work, due to the lack of cell line-specific \"ground truth\" graphs for evaluation. However, the predicted edges could represent gene regulatory relationships, and the differences between the observational and interventional graphs correspond to differences in gene regulation.\n\n> Theoretical analysis\n\nWe show in Appendix C that the differential network is *well-specified* as a *model class*. Due to the complexity of the architecture, it is difficult to *guarantee* what the model will learn, i.e. in the classical identifiability sense. At the same time, it is *not* a given that an arbitrary Transformer can approximate any arbitrary function given limited precision, width, depth, [2,3]. Instead, we are inspired by the neural network expressivity literature to show that (page 15):\n\n- Hard interventions: given the sets of edges $E, E'$, where $E'$ represents the mutilated graph, there exists a setting of its parameters that maps $(E, E')$ to targets $I$ by detecting edges that disappeared.\n- Soft interventions: under \"scale\" interventions and non-multiplicative noise, there exists a setting of its parameters that maps the correlation and covariance matrices (which can be derived from model inputs) to targets $I$. In the linear case, Eq. 14-15 naturally hold and are relevant to the relationships between variables (i.e. if $x$ is an intervention target, for every $y\\ne x$, correlation($x,y$) does not change while cov($x,y$) does.\n\n[2] Yun et al. Are Transformers universal approximators of sequence-to-sequence functions? ICLR 2020\n\n[3] Sanford et al. Representational Strengths and Limitations of Transformers. NeurIPS 2023.\n\n> Typos\n\nThank you for pointing this out! We will fix it.", "claims_and_evidence": "The claims made in the paper are generally supported by empirical evidence, but I have the following concerns:\n\n*  Motivation and Practical Significance: The paper should more clearly justify the motivation and practical significance of identifying perturbation targets across different types of perturbation data.\n\n   * In chemical perturbation experiments, where precise molecular targets are often unknown, identifying perturbation targets is intuitive and valuable. However, in genetic perturbation experiments, where genes are directly targeted, the value lies in distinguishing direct vs. downstream effects and aiding in combinatorial perturbations. The significance of the proposed method is not fully discussed, leaving readers unclear about its impact.\n\n   * The paper's main contribution is identifying perturbation targets from observational and interventional data, but the authors do not demonstrate its practical use. It is unclear how the proposed method can be utilized. For instance, can it help learn a better causal model? Can it improve combinatorial perturbation prediction? Or is it meant to guide experimental design? These potential applications are not clearly articulated.\n\n* Interpretability and Biological Significance: From an interpretability perspective, the biological significance of the inferred causal relationships is not explored. This leaves questions about whether the model captures meaningful biological mechanisms, which is crucial for validating its utility in real-world biological research.", "methods_and_evaluation_criteria": "The proposed model architecture and the evaluation criteria make sense to me.", "theoretical_claims": "The paper does not introduce new proofs or theorems. However, it would benefit from a theoretical analysis on the identifiability guarantees of the proposed model. Specifically, it is important to understand under what conditions the model is guaranteed to accurately identify perturbation targets.", "experimental_designs_or_analyses": "The experimental design and analysis are generally well-strctured and sound.", "relation_to_broader_scientific_literature": "This paper bridges two key research domains: single-cell perturbation modeling and multi-environment causal discovery. While single-cell biology typically focuses on predicting intervention outcomes or mapping perturbations to known pathways without directly inferring causal structure, multi-environment causal discovery methods leverage varied conditions to enhance identifiability but often fail to scale to high-dimensional single-cell data.\nThe paper's key contribution is an amortized causal inference framework that learns local causal graphs from both observational and interventional data without relying on external knowledge networks. By analyzing differences between these graphs, the method identifies perturbation targets in noisy, high-dimensional single-cell gene expression datasets. This integration of approaches addresses limitations in both fields, offering a solution for causal inference in complex biological systems.", "essential_references_not_discussed": "All the essential related works are discussed, but it would be better to include works on single-cell foundation models as the perturbation prediction is one of the most important downstream tasks.", "other_strengths_and_weaknesses": "Weaknesses:\n\n* please see my comments on the previous parts.\n\n* Zero-Shot Generalization Concerns: The authors claim the proposed model can generalize to unseen perturbations, but it's unclear whether it can perform good zero-shot inference. Recent work by [1] has shown that the identifiability of amortized causal discovery methods depends heavily on the training data. While the authors mention generating 8640 datasets with hard and soft interventions, they provide insufficient details on how these datasets are simulated. The paper would benefit from a detailed discussion on generalization ability under distribution shift, especially if the prepared dataset is heterogeneous.\n\n* Theoretical Foundation: The paper lacks theoretical analysis regarding the identifiability conditions under which the model is guaranteed to correctly identify perturbation targets. Such analysis would strengthen the theoretical underpinnings of the approach.\n\n\n[1] Montagna, F., Cairney-Leeming, M., Sridhar, D., & Locatello, F. (2024). Demystifying amortized causal discovery with transformers. arXiv preprint arXiv:2405.16924.", "other_comments_or_suggestions": "* Please see Other Strengths And Weaknesses.\n\n* (Minor typos) The citation of “Rao et al. (2021)” at lines 579–584 on the left column of page 11 needs to be revised.", "questions_for_authors": "Please see Other Strengths And Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BhaVienKLL", "reviewer": "Reviewer_ryKm", "summary": "Identifying targets that induce cell state transitions has numerous implications in advancing targeted drug discovery and biological research. Authors propose causal differential networks (CDN), an attention-based framework independent of biological priors to identify targets of an intervention given a pair of observational and interventional datasets. The CDN framework is comprised of two modules – (1) a causal structure learner that predicts causal graphs that could have produced each dataset and (2) a differential network classifier that predicts targets of an intervention from the causal graphs – that are jointly trained via a supervised amortized framework. The paper benchmarks CDN on 7 real transcriptomic datasets containing genetic (Perturb-seq) and chemical (Sci-Plex) perturbations and shows that CDN outperforms the state-of-the-art perturbation modeling methods for both seen and unseen cell lines. Experiments on synthetic datasets show that CDN outperforms existing causal discovery methods on predicting intervention targets. Overall, the paper presents a data-driven framework to identify perturbation targets, a task that can help elucidate large-scale perturbation experiments, causal targets, and targeted drug discovery.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments! Please let us know if this response helps clarify your questions, and if you have any further concerns.\n\n> Architecture similarities and differences / novelty\n\nOur key contribution is the idea of supervised causal discovery (predicting graphs from simulated data) as a pretraining objective, to obtain dataset representations useful for downstream tasks like inferring targets. We realize our idea with [1] as the causal structure learner, and a 2D Transformer-based architecture as the differential network, neither of which are our primary contributions. These modules were selected for their scalability and could (in principle) be replaced with alternate supervised learning algorithms [2,3] or models that operate pairs of graph [4].\n\nFor a specific comparison to [1] –\n\nSimilarities: We take the aggregator in SEA as a pretrained module, which is finetuned.\n\nDifferences:\n- We finetune SEA using correlation as a summary statistic. The original paper uses the precision matrix instead (not suitable for single-cell data), but writes that it's possible to finetune weights for different statistics.\n- The differential network is separate and learned from scratch.\n- We generate our own datasets for training throughout.\n\n[1] Wu et al. Sample, estimate, aggregate: A recipe for causal discovery foundation models. TMLR 2025.\n\n[2] Ke et al. Learning to Induce Causal Structure. ICLR 2023.\n\n[3] Lorch et al. Amortized inference for causal structure learning. NeurIPS 2022.\n\n[4] Li et al. Graph Matching Networks for Learning the Similarity of Graph Structured Objects. ICML 2019.\n\n> Data noise and relevance of synthetic data\n\nWe agree that real data are noisy, hence the focus on summary statistics (correlation, mean, variance) as the primary inputs to the model.\n\nSimulating mRNA expression is a task on its own, and improvements in that regard are directly applicable here. In this work, the synthetic soft interventions were loosely inspired by perturbations, e.g. in terms of fold-changes (line 796). \n\n> Summary statistics\n\nWe'll make the inputs more clear. Thank you for pointing this out.\n\nYes, the causal structure learner uses pairwise correlations as the summary statistic. In addition to the paired graph features, the differential network takes marginal means and variances (218-219 right, inspired by differential expression analysis).\n\n> Rank criteria\n\nWe'll be more explicit regarding the definition.\n\nFor perturbation effect predictors (GEARS, GenePT), we order genes based on the similarity of their predicted effect to the ground truth target effect (Eq 3, B.2). Methods that directly predict targets order genes based on their predicted probability of being a target. The set of candidate genes is the set of top 1000 differentially-expressed genes.\n\nThe rank (309-311) is the index of the correct answer in any ordering, divided by the number of candidates (since different perturbations have different candidate pools, B.2), where 1 is best (top of light) and 0 is worst (bottom of list).\n\nThis is equivalent to the expected recall at p (Table 3), over all p.\n\nRank is inspired by the application of experimental design. Since it is common to select top predictions for further testing, \"rank\" quantifies how many perturbations must be tested on average, before finding the ground truth.", "claims_and_evidence": "Most are reasonably supported in the paper. It was unclear how the model handled the noisiness in the data and also if the synthetic data adequately represented the underlying properties of the real-world datasets.", "methods_and_evaluation_criteria": "- The proposed methods make sense – the paper uses a causal learner and differential network causal discovery framework to identify perturbation targets of interventions using a data-driven approach and does not rely on biological knowledge graph priors that may potentially be confounding. \n\n- Benchmark datasets make sense for the task –  the paper includes 5 Perturb-seq datasets for seen and unseen cell line settings, 2 chemical perturbation datasets, and synthetic datasets.\n\n- However, in the evaluation criteria, the paper uses the normalized rank of the true target, and it is unclear how the rank is computed. There is some additional information regarding ranking candidates in Appendix B.2 but only for GEARS and GENEPT.", "theoretical_claims": "I did not check the theoretical claims carefully.", "experimental_designs_or_analyses": "The experimental section is quite detailed", "relation_to_broader_scientific_literature": "In the broader scientific literature, many machine learning based methods exist to identify perturbation targets. CDN demonstrates great potential for target discovery, as it outperforms state-of-the-art perturbation modeling approaches. The significance of the proposed method lies in the data-driven causal framework to identify perturbation and causal targets, which has extensive applications to biological research and targeted therapeutic development.", "essential_references_not_discussed": "Most relevant work has been discussed", "other_strengths_and_weaknesses": "Strengths:\n\n- The originality of the work lies in the novel combination of a causal structure learner and differential networks. \nThe significance of the work is substantial, as it is a purely data-driven method for identifying perturbation targets and outperforms existing methods. \n\n- The evaluation framework includes a robust collection of benchmarking datasets and baselines. \nWeaknesses:\n\n- The paper states that the causal learner is largely adapted from the architecture of SEA (Wu et al.) but does not describe the architecture of SEA. What differences and similarities exist between the casual learner architecture of SEA and CDN? \n\n- The clarity of the paper could be improved. The paper mentions that datasets are represented “in terms of their summary statistics” but does not state exactly which statistics. In Methods 3.1 Causal structure learner (Input featurization), authors explain that global statistics “like pairwise correlation” are computed, while Methods 3.3 Implementation details (Model parameters) mention that the global statistic is a pairwise correlation. Is pairwise correlation the only type of global statistic computed?\n\n- The rank evaluation criteria could also be further explained.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Covered above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z9oVeTwlyD", "reviewer": "Reviewer_smYK", "summary": "The authors focus on the problem of identifying direct targets of intervention in single-cell perturbation data. The proposed method, Causal Differential Network (CDN), first extracts causal networks behind unperturbed and perturbed data. These networks are then compared using an axial attention-based classifier to determine which nodes (genes) are direct targets of the intervention. The method was evaluated on multiple single-cell sequencing screens involving gene and drug perturbations, as well as synthetic datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments! Please let us know if this response helps clarify your questions, and if you have any further concerns.\n\n> \"direct application of existing CDN algorithms to the target identification task\"\n\nCDN is the name of the method we propose; are you referring to causal discovery algorithms?\n\nOur key contribution is the idea of supervised causal discovery (predicting graphs from simulated data) as a pretraining objective, to obtain dataset representations useful for downstream tasks like inferring targets.\n\nA direct solution to the task is to predict targets without inferring graphs. However, we empirically demonstrate that graph prediction leads to substantial improvement as an auxiliary loss (Table 3, no $L_G$) and starting from pretrained weights leads to more stable optimization dynamics (Figure 10). Furthermore, baselines that use data-mined graph priors perform substantially worse (Table 1, Figure 3).\n\nWe realize our idea with [1] as the causal structure learner, and a 2D Transformer-based architecture as the differential network, *neither of which* are our primary contributions. These modules were selected for their scalability and could (in principle) be replaced with alternate supervised learning algorithms [2,3] or models that operate pairs of graph [4].\n\n[1] Wu et al. Sample, estimate, aggregate: A recipe for causal discovery foundation models. 2024.\n\n[2] Ke et al. Learning to Induce Causal Structure. ICLR 2023.\n\n[3] Lorch et al. Amortized inference for causal structure learning. NeurIPS 2022.\n\n[4] Li et al. Graph Matching Networks for Learning the Similarity of Graph Structured Objects. ICML 2019.\n\n> interpretable biological analyses\n\nThis work focuses on the application of predicting perturbation targets, for efficient experimental design. The goal is *not to replace* experiment via simulation, but to *lower* the cost/scale of experiments, by reducing the search space. We do not believe that this focus diminishes from the interpretability of the results.\n\n- Since the causal graph learner is supervised on synthetic graphs, the predicted graphs are always available for inspection. This is not a primary focus of this work, due to the lack of cell line-specific \"ground truth\" graphs for evaluation. However, the predicted edges represent gene-gene interactions, and the differences between the observational and interventional graphs correspond to differences in gene regulation.\n\n- Works that do predict phenotypes often share our intended application [5], and as we write in the introduction, directly predicting the target is more efficient at inference time, especially for combinations.\n\n- Directly predicting targets also has benefits in terms of label / evaluation quality. Perturbation target identity is one of the most reliable readouts from these single-cell data, as it is *non*-quantitative [6]. In contrast, due to technical and biological covariates, two datasets generated by the same lab, of the same cell line, can be largely inconsistent in terms of individual gene expression. For example, [7] writes of K562 genome-wide and essential: \"The median correlation between  log-fold-change point estimates [...] was only 0.16, suggesting very low replicability.\"\n\n[5] Roohani et al. GEARS: Predicting transcriptional outcomes of novel multi-gene perturbations. Nat Biotech 2024.\n\n[6] Luecken and Theis. Current best practices in single‐cell RNA‐seq analysis: a tutorial. Mol Syst Biol (2019) 15: e8746.\n\n[7] Nadig et al. Transcriptome-wide characterization of genetic perturbations. 2024.", "claims_and_evidence": "Yes, the performance results reported by the authors in the paper (e.g., Tables 1 and 3) adequately demonstrate the effectiveness of the proposed CDN method. The results of the ablation experiments are also sound.", "methods_and_evaluation_criteria": "The authors conducted experiments on five commonly used perturbation datasets and several synthetic datasets. The dataset selection and evaluation metrics appear reasonable.", "theoretical_claims": "This paper does not contain relevant theoretical proofs.", "experimental_designs_or_analyses": "Refer to Claims And Evidence and Methods And Evaluation Criteria.", "relation_to_broader_scientific_literature": "This paper combines the amortized causal discovery model[1] with the axial attention-based classifier[2], improving performance on the target identification task in perturbation datasets like [3], surpassing previously common models for this task like [4].\n\n[1] Wu, Menghua, et al. \"Sample, estimate, aggregate: A recipe for causal discovery foundation models.\" arXiv preprint arXiv:2402.01929 (2024).\n[2] Ho, Jonathan, et al. \"Axial attention in multidimensional transformers.\" arXiv preprint arXiv:1912.12180 (2019).\n[3] Dixit, Atray, et al. \"Perturb-Seq: dissecting molecular circuits with scalable single-cell RNA profiling of pooled genetic screens.\" cell 167.7 (2016): 1853-1866.\n[4] Roohani, Yusuf, Kexin Huang, and Jure Leskovec. \"Predicting transcriptional outcomes of novel multigene perturbations with GEARS.\" Nature Biotechnology 42.6 (2024): 927-935.", "essential_references_not_discussed": "None that I'm aware of.", "other_strengths_and_weaknesses": "The novelty of this paper is debatable. It appears to be a direct application of existing CDN algorithms to the target identification task, with minimal contribution to the machine learning algorithm aspect. The biological aspect also lacks more interpretable analyses (such as phenotype prediction or exploration of genetic interactions).", "other_comments_or_suggestions": "Please refer to Other Strengths And Weaknesses.", "questions_for_authors": "Please refer to Other Strengths And Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LSJOHtrHYE", "reviewer": "Reviewer_JJyS", "summary": "The authors considered the problem of identifying the target of interventions from merely observational/interventional data. In some biological applications, there might be thousands of variables in the system and few samples are available per intervention. The authors utilized an amortized approach (which has been explored for the task of causal discovery in the literature of causality) to detect the target of interventions. In particular, a network called causal structure learner is trained to return the causal graph from the observed data. Then, a differential network is considered where the graph obtained from observational data and the one from the interventional data are passed to an attention-based classifier to detect which nodes are intervened on. Both the causal structure learner network and the differential network are trained jointly with amortized data collected from different causal models and interventions. The experimental results showed that the proposed method performs better than previous work.\n\n### Update after rebuttal\nI have read the reviews and the rebuttal. In my view, the proposed approach remains quite heuristic and lacks theoretical grounding. That said, the experimental results are promising. Overall, I still recommend a weak reject, though I acknowledge it could be accepted.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments! Please let us know if this helps answer your questions, and if you have any further concerns.\n\n> Theoretical guarantees\n\nWe show in Appendix C that the differential network is *well-specified* as a *model class*. Due to the complexity of the architecture, it is difficult to *guarantee* what the model will learn, i.e. in the identifiability sense. At the same time, it is *not* a given that an arbitrary Transformer can approximate any arbitrary function given limited precision, width, depth [1,2]. We are inspired by the neural network expressivity literature to show that (page 15):\n\n- Hard interventions: given the sets of edges $E, E'$, where $E'$ represents the mutilated graph, there exists a setting of its parameters that maps $(E, E')$ to targets $I$ by detecting edges that disappeared.\n- Soft interventions: under \"scale\" interventions and non-multiplicative noise, there exists a setting of its parameters that maps the correlation and covariance matrices (which can be derived from model inputs) to targets $I$. In the linear case, Eq. 14-15 naturally hold and are relevant to the relationships between variables (i.e. if $x$ is an intervention target, for every $y\\ne x$, correlation($x,y$) does not change while cov($x,y$) does).\n\n[1] Yun et al. Are Transformers universal approximators of sequence-to-sequence functions? ICLR 2020\n\n[2] Sanford et al. Representational Strengths and Limitations of Transformers. NeurIPS 2023.\n\n> Alternative causal structure learners\n\nCSIvA [1] and AVICI [2] are supervised causal discovery algorithms that could work in principle.\n\n- CSIvA does not have a public codebase or supplemental files.\n\n- AVICI is available and runs well on smaller datasets, but there are some practical challenges. Since the Transformer attends over raw data, its complexity scales quadratically as the number of samples. Also, AVICI was only trained on datasets of up to 50 variables and 200 samples. On graphs of 100 variables (Table 7 in [2]), the performance drops to that of classical algorithms (GES, GIES).\n\nWe chose [3] since it reported reasonable performance on larger datasets (CausalBench, 622 variables). The use of summary statistics like correlation also mirrors common practices in single cell sequencing, since the raw data are often very noisy (150-152, right).\n\n[1] Ke et al. Learning to Induce Causal Structure. ICLR 2023.\n\n[2] Lorch et al. Amortized inference for causal structure learning. NeurIPS 2022.\n\n[3] Wu et al. Sample, estimate, aggregate: A recipe for causal discovery foundation models. TMLR 2025.\n\n> lines 193-198\n\nThank you for pointing this out, and please let us know if this makes more sense.\n\nFCI is run $T$ times on different subsets of $k$ nodes. The \"space of possible edge types\" (196) refers to the edge types output by FCI, which are mapped to categorical labels (e.g. no edge to 0, -> to 1, – to 2). These elements are \"embedded\" similar to word embeddings in language models (201-202).\n\nSo $E'$ contains $k\\times k \\times T$ elements ($T$ adjacency matrices, each of size $k\\times k$).\n\n> FCI noisy estimates and correlation\n\nOur causal graph learner, finetuned from SEA [3], uses correlation and FCI estimates and as input features. Correlation does not encode CI information, but it may help the model quickly filter out edges that are *not* likely to exist, especially important for larger datasets.\n\nFCI is run on subsets of $k=5$ throughout (above), rather than the full dataset [194]. These smaller subsets are sampled based on heuristics like high pairwise correlation, to prioritize CI testing for regions likely to be \"interesting.\"\n\nThere are classic algorithms for integrating (sub)graph estimates over different variable sets [4]. SEA trains a neural network to perform this task over datasets of varying assumptions, precisely because FCI estimates may be noisy in practice [3]. In this sense, the role of the neural network is to \"denoise\" the graph prediction and to learn to resolve contradictions.\n\n[4] Huang et al. Causal discovery from multiple data sets with non-identical variable sets. AISTATS 2020.\n\n> Permuting the edge embedding\n\nThis practice follows SEA [3] and loosely resembles the idea in [5], where permutations are sampled at random during training so that \"in expectation\" the Transformer does not rely on any particular ordering.\n\nIt's necessary to add embeddings to inform of node/edge identities across layers, since Transformers are permutation invariant. In principle, any ordering will do. However, since synthetic datasets are generated in topological order, \"node 1\" always corresponds to a root node and so on. As [6] shows, it's trivial for neural networks to memorize this spurious feature to recover topological order.\n\n[5] Yang et al. XLNet: Generalized Autoregressive Pretraining for Language Understanding. NeurIPS 2019.\n\n[6] Reisach et al. Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Game. NeurIPS 2021.", "claims_and_evidence": "The work is experimental and in the experiment section, the authors made a comprehensive comparison with previous work.", "methods_and_evaluation_criteria": "The authors considered several metrics for comparison such as rank, Spearman rank correlation, and Pearson correlation. Moreover, they considered several datasets in their evaluations.", "theoretical_claims": "The work is experimental and there is no theoretical claim in the paper.", "experimental_designs_or_analyses": "I did not check the codes, but I read the experiment section. Based on the text, it seems that the experimental results are sound and the authors also provided some explanations for the plots.", "relation_to_broader_scientific_literature": "One of the main applications of causal inference/discovery is in biology such as learning causal structures in gene regulatory networks or predicting the results of gene perturbations. This work is aligned with this line of research and aims to find the target of interventions (such as protein targets of a drug).", "essential_references_not_discussed": "As far as I checked the related work, the authors cited the most relevant previous work.", "other_strengths_and_weaknesses": "Strengths:\n- The authors used attention-based networks and proposed an architecture to learn the location of interventions with an amortized scheme.\n- They compared with several previous works in different synthetic and real datasets and showed better performance in almost all the datasets.\n\nWeakness:\n- The main weakness is that there is no theoretical guarantee in the work. That being said, I should emphasize that this comment is also applied to some of the previous work. Nevertheless, it is interesting to see whether, in simple settings, such as linear models or bivariate cases, the proposed method indeed recovers the correct intervention targets.", "other_comments_or_suggestions": "- It would be nice to study whether some theoretical guarantees can be added to the work in some specific settings. Moreover, it is good to check how this method performs in more challenging cases such as under latent confounding.", "questions_for_authors": "- Regarding the causal structure learner, what are the alternatives instead of SEA? Why did the authors consider this architecture?\n- The text in lines 193-198 is ambiguous. How is $E'$ formed from the output of FCI? What are $k$ and $T$ in this context?\n- Since FCI uses conditional independence (CI) tests, in the regime considered in this paper with thousands of variables, the output of FCI might have too many errors. It is not clear how the proposed method performs well with such a noisy input. Moreover, the correlation matrix does not encode any CI assertions (which are essential in learning the equivalence class). \n- I did not understand the sentence \"we randomly sample a permutation of the maximum graph size to each graph\" about the edge embedding. It is not clear how this ensures that the embedding is equivariant to the labeling of nodes. Moreover, why is this approach considered in this paper?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces Causal Differential Networks (CDN), a supervised method that integrates causal graph inference with perturbation target prediction to identify variables directly affected by biological interventions using single-cell gene expression data. 

Pros:

+ Uses permutation-invariant axial attention and a well-structured integration of causal inference and perturbation classification.

+ Shows minimal performance drop on unseen biological domains and handles both simple and complex intervention types.

Cons:

+ Causal graph inference relies on pre-training from synthetic data, raising concerns about its ability to learn true causal relationships in real-world scenarios.

+ Evaluation may be biased by restricting target candidates and using small synthetic graphs, which may not reflect real data complexity.
Decision: Accept (poster)