Original: G RANGER CAUSAL INFERENCE ON DAG S IDENTIFIES GENOMIC LOCI REGULATING TRANSCRIPTION
Revised: G RANGER CAUSAL INFERENCE ON DAG S IDENTIFIES GENOMIC LOCI REGULATING TRANSCRIPTION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: When a dynamical system can be modeled as a sequence of observations, Granger causality is a powerful approach for detecting predictive interactions between its variables.
Revised: When a dynamical system can be modeled as a sequence of observations, Granger causality is a powerful approach for detecting predictive interactions between its variables.

Original: However, traditional Granger causal inference has limited utility in domains where the dynamics need to be represented as directed acyclic graphs (DAGs) rather than as a linear sequence, such as with cell differentiation trajectories.
Revised: However, traditional Granger causal inference has limited utility in domains where the dynamics need to be represented as directed acyclic graphs (DAGs) rather than as a linear sequence, such as with cell differentiation trajectories.

Original: Here, we present GrID-Net, a framework based on graph neural networks with lagged message passing for Granger causal inference on DAG-structured systems.
Revised: Here, we present GrID-Net, a framework based on graph neural networks with lagged message passing for Granger causal inference on DAG-structured systems.

Original: Our motivating application is the analysis of single-cell multimodal data to identify genomic loci that mediate the regulation of speciﬁc genes.
Revised: Our motivating application is the analysis of single-cell multimodal data to identify genomic loci that mediate the regulation of speciﬁc genes.

Original: To our knowledge, GrID-Net is the ﬁrst single-cell analysis tool that accounts for the temporal lag between a genomic locus becoming accessible and its downstream effect on a target gene’s expression.
Revised: To our knowledge, GrID-Net is the ﬁrst single-cell analysis tool that accounts for the temporal lag between a genomic locus becoming accessible and its downstream effect on a target gene’s expression.

Original: We applied GrID-Net on multimodal single-cell assays that proﬁle chromatin accessibility (ATAC-seq) and gene expression (RNA-seq) in the same cell and show that it dramatically outperforms existing methods for inferring regulatory locus–gene links, achieving up to 71% greater agreement with independent population genetics-based estimates.
Revised: We applied GrID-Net on multimodal single-cell assays that proﬁle chromatin accessibility (ATAC-seq) and gene expression (RNA-seq) in the same cell and show that it dramatically outperforms existing methods for inferring regulatory locus–gene links, achieving up to 71% greater agreement with independent population genetics-based estimates.

Original: The loci identiﬁed by GrID-Net are also enriched for 10–50x more transcription-factor binding motifs than other methods.
Revised: The loci identiﬁed by GrID-Net are also enriched for 10–50x more transcription-factor binding motifs than other methods.

Original: By extending Granger causality to DAG-structured dynamical systems, our work unlocks new domains for causal analyses and, more speciﬁcally, opens a path towards elucidating gene regulatory interactions relevant to cellular differentiation and complex human diseases at unprecedented scale and resolution.
Revised: By extending Granger causality to DAG-structured dynamical systems, our work unlocks new domains for causal analyses and, more speciﬁcally, opens a path towards elucidating gene regulatory interactions relevant to cellular differentiation and complex human diseases at unprecedented scale and resolution.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Understanding the structure of a multivariate dynamical system often boils down to deciphering the causal relationships between its variables.
Revised: Understanding the structure of a multivariate dynamical system often boils down to deciphering the causal relationships between its variables.

Original: Since inferring true causality is often infeasible, requiring additional experiments or independent mechanistic insight, statistical analysis of observational data to identify predictive relationships between the system’s variables can be very valuable.
Revised: Since inferring true causality is often infeasible, requiring additional experiments or independent mechanistic insight, statistical analysis of observational data to identify predictive relationships between the system’s variables can be very valuable.

Original: The framework of Granger causality does exactly that: in a dataset where observations are temporally ordered, a time-dependent variable x (with value x t at time t ) is said to “Granger cause” a second variable y if the history of x at time t (i.e., x 1 , . . .
Revised: The framework of Granger causality does exactly that: in a dataset where observations are temporally ordered, a time-dependent variable x (with value x t at time t ) is said to “Granger cause” a second variable y if the history of x at time t (i.e., x 1 , . . .

Original: , x t − 1 ) is signiﬁcantly predictive of y at time t even after accounting for y ’s own history (Granger, 1969; Shojaie & Fox, 2021).
Revised: , x t − 1 ) is signiﬁcantly predictive of y at time t even after accounting for y ’s own history (Granger, 1969; Shojaie & Fox, 2021).

Original: Originating in econometrics, Granger causality has been a powerful tool in many domains including biology, ﬁnance, and social sciences (Fujita et al., 2007; Yao et al., 2015; Benhmad, 2012; Rasheed & Tahir, 2012).
Revised: Originating in econometrics, Granger causality has been a powerful tool in many domains including biology, ﬁnance, and social sciences (Fujita et al., 2007; Yao et al., 2015; Benhmad, 2012; Rasheed & Tahir, 2012).

Original: The prerequisite for applying Granger causality, however, is that there be a clear sequential ordering of observations, i.e., the data must conform to a total ordering along time.
Revised: The prerequisite for applying Granger causality, however, is that there be a clear sequential ordering of observations, i.e., the data must conform to a total ordering along time.

Original: Often, only a partial ordering of the observations is possible.
Revised: Often, only a partial ordering of the observations is possible.

Original: For instance, cell differentiation trajectories may have branches.
Revised: For instance, cell differentiation trajectories may have branches.

Original: In some social network settings, information ﬂows along a tree (e.g., the retweets of a meme by the followers of an earlier retweeter).

Original: In text-mining, the citation graph of publications captures the ﬂow of ideas and knowledge.
Revised: In text-mining, the citation graph of publications captures the ﬂow of ideas and knowledge.

Original: In such cases, the dynamics of the system are more suitably represented as a directed acyclic graph (DAG) corresponding to the partial ordering, with nodes of the DAG representing dynamical states and its edges indicating the ﬂow of information between states.
Revised: In such cases, the dynamics of the system are more suitably represented as a directed acyclic graph (DAG) corresponding to the partial ordering, with nodes of the DAG representing dynamical states and its edges indicating the ﬂow of information between states.

Original: The key conceptual advance of this work is extending the applicability of Granger causality to systems with partially ordered observations ( Figure 1a).
Revised: The key conceptual advance of this work is extending the applicability of Granger causality to systems with partially ordered observations ( Figure 1a).

Original: On a DAG where each node takes on multi- variate values, we deﬁne a variable x as Granger causing another variable y if the latter’s value y v at the DAG node v can be signiﬁcantly predicted by the values of x at the ancestor nodes of v , even after accounting for y
Revised: On a DAG where each node takes on multivariate values, we deﬁne a variable x as Granger causing another variable y if the latter’s value y v at the DAG node v can be signiﬁcantly predicted by the values of x at the ancestor nodes of v , even after accounting for y

Original: v ’s own ancestral history.
Revised: v ’s own ancestral history.

Original: We introduce GrID-Net ( Gr anger I nference on D AGs), a graph neural network (GNN) framework for Granger causal inference in dynamical systems that can be represented as DAGs ( Figure 1b).
Revised: We introduce GrID-Net ( Gr anger I nference on D AGs), a graph neural network (GNN) framework for Granger causal inference in dynamical systems that can be represented as DAGs ( Figure 1b).

Original: We modify the standard GNN architecture to enable lagged message-passing on the DAG, enabling us to accumulate past information for a variable according to the DAG’s structure.
Revised: We modify the standard GNN architecture to enable lagged message-passing on the DAG, enabling us to accumulate past information for a variable according to the DAG’s structure.

Original: This, combined with a mathematical formulation of Granger causality more amenable to neural networks, allows GrID-Net to recover nonlinear long-range Granger causal dependencies between the variables.
Revised: This, combined with a mathematical formulation of Granger causality more amenable to neural networks, allows GrID-Net to recover nonlinear long-range Granger causal dependencies between the variables.

Original: While our problem formulation is fully general and GrID-Net is broadly applicable, our motivating application is inferring non-coding genomic loci thatcausally inﬂuence the expression of a speciﬁc gene.
Revised: While our problem formulation is fully general and GrID-Net is broadly applicable, our motivating application is inferring noncoding genomic loci that inﬂuence the expression of a speciﬁc gene.

Original: We apply GrID-Net to single-cell multimodal studies that assay chromatin accessibility (ATAC-seq) and gene expression (RNA-seq) in the same cell, seeking to identifya Granger causal relationship between the accessibility of a chromatin region (ATAC-seq “peak”) and the expression ofa speciﬁc gene.
Revised: We apply GrID-Net to single-cell multimodal studies that assay chromatin accessibility (ATAC-seq) and gene expression (RNA-seq) in the same cell, seeking to identify Granger causal relationships between the accessibility of individual chromatin regions (ATAC-seq “peaks”) and the expression of speciﬁc genes.

Original: The system dynamics are represented by a DAG where each node corresponds to a cell and edges exist between cells in similar dynamic states (i.e. similar ATAC-seq and RNAseq proﬁles), withthe edge directions estimated by a pseudotime analysis (Haghverdi et al., 2016) ( Figure 1c).
Revised: The system dynamics are represented by a DAG where each node corresponds to a cell. Edges connect cells in similar dynamic states (i.e. similar ATAC-seq and RNA-seq proﬁles), with edge directions estimated by a pseudotime analysis (Haghverdi et al., 2016) ( Figure 1c).

Original: Our work addresses a pressing need to identify  causal relationships between non-coding loci and gene expression, as traditional approaches for doing so are expensive and slow.
Revised: Our work addresses a pressing need to identify temporally causal relationships between noncoding loci and gene expression, as traditional approaches for doing so are expensive and slow.

Original: They include population genetics techniques like expression quantitative trait locus (eQTL) studies, in which human genetic variability at a particular genomic locus is associated with expression changes in the gene of interest.
Revised: They include population genetics techniques like expression quantitative trait locus (eQTL) studies, in which human genetic variability at a particular genomic locus is associated with expression changes in the gene of interest.

Original: Alternatively, perturbation-based approaches directly seek to identify gene expression changes in response to CRISPR alterations of speciﬁc non-coding genomic loci (Fulco et al., 2016).
Revised: Alternatively, perturbation-based approaches directly seek to identify gene expression changes in response to CRISPR alterations of speciﬁc noncoding genomic loci (Fulco et al., 2016).

Original: The emergence of single-cell multimodal assays that proﬁle ATAC-seq and RNA-seq simultaneously offers a new way of discovering such regulatory relationships, on which we capitalize.
Revised: The emergence of single-cell multimodal assays that proﬁle ATAC-seq and RNA-seq simultaneously offers a new way of discovering such regulatory relationships, on which we capitalize.

Original: Rather than requiring the large population sizes of eQTL studies or being limited to speciﬁc noncoding loci as in the perturbation-based approaches, these multimodal experiments enable unbiased high-throughput, genome-wide estimations of peak–gene associations.
Revised: Rather than requiring the large population sizes of eQTL studies or being limited to speciﬁc noncoding loci as in the perturbation-based approaches, these multimodal experiments enable unbiased high-throughput, genome-wide estimations of peak–gene associations.

Original: However, leveraging this data will require algorithmic innovations.
Revised: However, leveraging this data will require algorithmic innovations.

Original: Over 99 .
Revised: Over 99 .

Original: 5% of ATAC-seq observations (“peaks”) in a cell are reported as empty (Singh et al., 2021; Zhang et al., 2019), and this sparsity constrains current approaches.
Revised: 5% of ATAC-seq observations (“peaks”) in a cell are reported as empty (Singh et al., 2021; Zhang et al., 2019), and this sparsity constrains current approaches.

Original: A key limitation of existing methods is also that they are based on correlation estimates and cannot capture thetemporal dynamics governing the  causal relationship between chromatin accessibility and gene expression.
Revised: A key limitation of existing methods is also that they are based on correlation estimates and cannot capture the dynamics governing the temporally causal relationship between chromatin accessibility and gene expression.

Original: For a noncoding locus to inﬂuence a gene’s expression, the locus typically must ﬁrst become accessible, upon which regulator proteins act on it to drive a change in gene expression ( Figure 1d).
Revised: For a noncoding locus to inﬂuence a gene’s expression, the locus typically must ﬁrst become accessible, upon which regulator proteins act on it to drive a change in gene expression ( Figure 1d).

Original: This causal mechanism, therefore, entails temporal asynchrony between the ATAC-seq and RNA-seq modalities (Lara-Astiaso et al., 2014;
Revised: This causal mechanism, therefore, entails temporal asynchrony between the ATAC-seq and RNA-seq modalities (Lara-Astiaso et al., 2014;

Original: Ostuni et al., 2013).
Revised: Ostuni et al., 2013).

Original: GrID-Net is speciﬁcally designed to recover the dynamics between peaks and genes while being robust to the noise and sparsity in single-cell data.
Revised: GrID-Net is speciﬁcally designed to recover this dynamics between peaks and genes while being robust to the noise and sparsity in single-cell data.

Original: We apply GrID-Net on three single-cell multimodal datasets that characterize a range of dynamic processes, including cancer drug responses and cellular differentiation.
Revised: We apply GrID-Net on three single-cell multimodal datasets that characterize a range of dynamic processes, including cancer drug response and cellular differentiation.

Original: We show that GrID-Net substantially outperforms current methods in identifying known peak–gene links, opening a path towards unlocking critical insights about gene regulation at genome scale.
Revised: We show that GrID-Net substantially outperforms current methods in identifying known peak–gene links, opening a path towards unlocking critical insights about gene regulation at genome scale.

Original: To our knowledge, this work presents both the ﬁrst framework for Granger causal inference on dynamical systems represented as DAGs and the ﬁrst single-cell method to explicitly model the temporal asynchrony between chromatin accessibility and gene expression.
Revised: To our knowledge, this work presents both the ﬁrst framework for Granger causal inference on dynamical systems represented as DAGs and the ﬁrst single-cell method to explicitly model the temporal asynchrony between chromatin accessibility and gene expression.

Original:  M ETHODS 2. B ACKGROUND ON G RANGER C AUSALITY : S TANDARD F ORMULATIONS
Revised:  M ETHODS 2. B ACKGROUND ON G RANGER C AUSALITY : S TANDARD F ORMULATIONS

Original: In time series analyses, the classical Granger causality framework uses the vector autoregressive (VAR) model, which can be expressed as follows (L ¨ utkepohl, 2005).
Revised: In time series analyses, the classical Granger causality framework uses the vector autoregressive (VAR) model, which can be expressed as follows (L ¨ utkepohl, 2005).

Original: [Equation]
Revised: [Equation]

Original: Here, y t ∈ R and x t ∈ R are values at time t for two stationary time series y and x , respectively; a ( (cid:96) ) 1 and a ( (cid:96) ) 2 are coefﬁcients that specify how lag (cid:96) affects the future of time series y ; and (cid:15) t is a zero-mean noise term.
Revised: Here, y t , x t ∈ R are values at time t for two stationary time series y and x , respectively; a ( (cid:96) ) 1 and a ( (cid:96) ) 2 are coefﬁcients that specify how lag (cid:96) affects the future of time series y ; and (cid:15) t is a zero-mean noise term.

Original: In this model,time series y  is assumed to be a linear combination of thepast L lags of y and x each, with x said to Granger cause time series y
Revised: In this model, y t is assumed to be a linear combination of the L most recent values of y and x each, with x said to Granger cause time series y

Original: if and only if a ( (cid:96) ) 2 (cid:54) = 0 for all (cid:96) .
Revised: if and only if a ( (cid:96) ) 2 (cid:54) = 0 for all (cid:96) .

Original: A more generalized formulation — the one we follow — for performing Granger causal inference is to consider two related models for forecasting time series y .
Revised: A more generalized formulation — the one we follow — for performing Granger causal inference is to consider two related models for forecasting time series y .

Original: A full model considers the past values of both y and x to forecast y , while a reduced model excludes the effect of x , only containing terms related to the past values of y .
Revised: A full model considers the past values of both y and x to forecast y , while a reduced model excludes the effect of x , only containing terms related to the past values of y .

Original: [Equation]
Revised: [Equation]

Original: Here, f ( full ) ( · ) and f ( reduced ) ( · ) are generalized functions that specify how the value of y t depends on past values of y and x .
Revised: Here, f ( full ) ( · ) and f ( reduced ) ( · ) are generalized functions that specify how the value of y t depends on past values of y and x .

Original: The predictions of each of these models are then compared, upon which a
Revised: The predictions of the two models are then compared, upon which a

Original: Granger causal relationship is declared if the full model’s predictions are signiﬁcantly more accurate than those of the reduced model.
Revised: Granger causal relationship is declared if the full model’s predictions are signiﬁcantly more accurate.

Original: 2. G RANGER C AUSALITY ON A DAG: G RAPH N EURAL N ETWORK F ORMULATION
Revised: 2. G RANGER C AUSALITY ON A DAG: G RAPH N EURAL N ETWORK F ORMULATION

Original: Let the data be represented by a DAG G =
Revised: Let the data be represented by a DAG G =

Original: ( V , E ) with n = |V| nodes (i.e., observations) andthe directed edges E indicating the information ﬂow or partial order between these observations.
Revised: ( V , E ) with n = |V| nodes (i.e., observations) and directed edges E indicating the information ﬂow or partial order between these observations.

Revised: For instance, when applied to the case of standard Granger causal inference on time series data, G would be the linear graph corresponding to the time series.

Original: Let y ∈ R n and x ∈ R n correspond to the values of variables y and x on the nodes in V , with x putatively causing y .
Revised: Let y , x ∈ R n correspond to the values of variables y and x on the nodes in V , with x putatively Granger-causing y .

Original: To infer Granger causality, we compare  full and reduced models as above:
Revised: To infer Granger causality, we compare the full and reduced models as above:

Original: [Equation]
Revised: [Equation]

Original: Here, ˜ h ( full ) y , ˜ h ( reduced ) y ,
Revised: Here, ˜ h ( full ) y , ˜ h ( reduced ) y ,

Original: ˜ h
Revised: ˜ h

Original: ( full ) x ∈ R
Revised: ( full ) x ∈ R

Original: n represent the historical information of y or x (as denoted by the subscript) that is aggregated by graph neural network (GNN) layers.
Revised: n represent the historical information of y or x (as denoted by the subscript) that is aggregated by graph neural network (GNN) layers.

Original: For example, at each node v ∈ V , ˜ h
Revised: For example, at each node v ∈ V , ˜ h

Original: [ v ] is the information accumulated from v ’s ancestors in G .
Revised: [ v ] is the information accumulated from v ’s ancestors in G .

Original: While ˜ h ( full ) y and ˜ h ( reduced )
Revised: While ˜ h ( full ) y and ˜ h ( reduced )

Original: y are outputs determined from similar architectures, their learned weights and biases end up being different since ˜ h ( full ) y is inﬂuenced by its interaction with ˜ h x .
Revised: y are outputs determined from similar architectures, their learned weights and bias terms end up being different since ˜ h ( full ) y is inﬂuenced by its interaction with ˜ h x .

Original: The coefﬁcient c mediates this interaction and describes the effect of x ’s history on y .
Revised: The coefﬁcient c mediates this interaction and describes the effect of x ’s history on y .

Original: In addition, (cid:15) ∈ R n is a zero-mean noise term.
Revised: In addition, (cid:15) ∈ R n is a zero-mean noise term.

Original: We also include g ( · ) as an optional link function that maps its input to the support of the distribution of variable y .
Revised: We also include g ( · ) as an optional link function that maps its input to the support of the distribution of variable y .

Original: In our single-cell application, we set g as the exponential function since the target variable ( y ) represents normalized transcript counts, which are non-negative.
Revised: In our single-cell application, we set g as the exponential function since the target variable ( y ) represents normalized transcript counts, which are non-negative.

Original: For brevity, we only describe how ˜ h ( full ) x is computed, with ˜ h
Revised: For brevity, we only describe how ˜ h ( full ) x is computed, with ˜ h

Original: ( full )
Revised: ( full )

Original: y and ˜ h ( reduced )
Revised: y and ˜ h ( reduced )

Original: y being  computed analogously.
Revised: y being independently computed analogously.

Original: Also, since x occurs only in the full model, for notational convenience we drop its superscript (full) below, writing just ˜
Revised: Also, since x occurs only in the full model, for notational convenience we drop its superscript (full) below, writing just ˜

Original: h x .
Revised: h x .

Original: We express ˜ h
Revised: We express ˜ h

Original: x
Revised: x

Original: as the mean of the outputs of L consecutive GNN layers, denoting the layerwise outputs as h ( (cid:96) ) x :
Revised: as the mean of the outputs of L consecutive GNN layers, denoting the layerwise outputs as h ( (cid:96) ) x :

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Here,  A and A + ∈ R n × n are matrices deﬁned by the DAG structure, designed to introduce a onestep lag in the ﬁrst GNN layer that is propagated by subsequent layers.
Revised: Here, w ( (cid:96) ) x , b ( (cid:96) ) x ∈ R are the per-layer weight and bias terms, respectively. σ ( · ) represents the nonlinear activation in each of the GNN layers, chosen here to be the hyperbolic tangent function. A and A + ∈ R n × n are matrices deﬁned by the DAG structure, designed to introduce a one-step lag in the ﬁrst GNN layer that is propagated by subsequent layers.

Original: A is the column-normalized adjacency matrix of G , with A ij = 

Revised: A is the column-normalized adjacency matrix of G , with A ij =

Original: d j if edge ( i, j ) ∈ E and 0 otherwise. d j is the in-degree of node j , ensuring the column normalization.
Revised: d j if edge ( i, j ) ∈ E and 0 otherwise, where d j is the in-degree of node j .

Original: We note that a DAG does not have self-loops sothat the diagonal terms of A are zero.
Revised: We note that a DAG does not have self-loops, so the diagonal terms of A are zero.

Original: In A +  we introduce these diagonal terms, setting them to the same value as others in the column ( d j is incremented by 1  to preserve normalization).
Revised: In A + , we include these diagonal terms, setting them to the same value as others in the column (i.e. d j is incremented by 1 in A + to preserve normalization).

Original: Together, A and A + allow us to extend the key intuition of Granger causality to DAGs: predicting a value at node v from the values of its ancestors that are within L steps.
Revised: Together, A and A + allow us to extend the key intuition of Granger causality to DAGs: predicting a value at node v from the values of its ancestors that are within L steps.

Original: The ﬁrst layer introduces the lag, using information at the parents of v but not the information at v itself.
Revised: The ﬁrst layer introduces the lag, using information at the parents of v but not the information at v itself.

Original: Each subsequent layer introduces the preceding set of parents, with the diagonal term in A + ensuring that information  stored at vfrom previous layers is integrated with the new information propagated via v ’s parents.
Revised: Each subsequent layer introduces the preceding set of parents, with the diagonal term in A + ensuring that information already stored at v is integrated with the new information propagated via v ’s parents.

Original: The sequence of GNN layers therefore reﬂects the successive aggregation of past information by traversing L steps backward along the DAG for each node in the graph.
Revised: The sequence of GNN layers therefore reﬂects the successive aggregation of past information by traversing L steps backward along the DAG for each node in the graph.

Original: Thus, we are able to  matchthe concept of lags in time series-based formulations (e.g., Eqn. 1) but with the crucial advantage of leveraging the richness of a DAG’s structure in directing the information ﬂow. We compute the GNN layers as follows (again, we only describe f x for brevity): [Equation]
Revised: Thus, we are able to conceptually match time series-based formulations (e.g., Eqn. 1) but with the crucial advantage of leveraging the richness of a DAG’s structure in directing the information ﬂow and aggregating sparse data.

Original: where w ( (cid:96) ) x , b ( (cid:96) ) x ∈ R are the per-layer weight and bias terms, respectively.

Original: σ ( · ) represents the nonlinear activation in each of the GNN layers, chosen here to be the hyperbolic tangent function.

Original: 2. C OMPARING F ULL AND R EDUCED M ODELS TO I NFER G RANGER C AUSALITY
Revised: 2. C OMPARING F ULL AND R EDUCED M ODELS TO I NFER G RANGER C AUSALITY

Original: Let Θ ( full ) y , Θ ( reduced ) y , Θ ( full ) x ∈ R 2 L denote the set of parameters for the full and reduced models, with Θ ( full ) x 
Revised: Let Θ ( full ) y , Θ ( reduced ) y , Θ ( full ) x ∈ R 2 L denote the set of parameters for the full and reduced models, with Θ ( full ) x = { [Equation] [Equation]

Revised: [Equation]

Original: = { w (1) x , ..., w ( L ) x ; b (1) x , ..., b( L ) x } and Θ ( full ) y , Θ ( reduced )
Revised: ; b [Equation] [Equation] , ..., b

Revised: [Equation]

Revised: L )

Revised: [Equation]

Revised: and Θ

Revised: [Equation]

Revised: , Θ ( reduced )

Original: y deﬁned analogously.
Revised: y deﬁned analogously.

Original: We propose to jointly learn all the Θ parameters  by minimizing the combined loss of the full and reduced models (Montalto et al., 2015):
Revised: All the Θ parameters are jointly learned by minimizing the combined loss of the full and reduced models (Montalto et al., 2015):

Original: [Equation]
Revised: [Equation]

Original: where ˆ y ( full ) v and ˆ y ( reduced ) v correspond to the predictions for observation v by the full and reduced models, respectively.
Revised: where ˆ y ( full ) v and ˆ y ( reduced ) v correspond to the predictions for observation v by the full and reduced models, respectively.

Revised: We note that the full and reduced models have completely separate parameters (see discussion in Appendix A.8).

Original: In a multivariate system, not all pairwise combinations of variables may be relevant: P is the subset of x, y variable pairs whose putative causal interactions are of interest.
Revised: Also, in a multivariate system, not all pairwise combinations of variables may be relevant: P is the subset of x, y variable pairs whose putative Granger-causal interactions are of interest.

Original: To infer Granger causal interactions between y and x , we compare the set of loss terms associated with the full model L (ˆ y ( full ) v , y v ) to those of the reduced model L (ˆ y ( reduced ) v , y v ) , noting that the precise functional form of L depends on the domain.
Revised: To infer Granger causal interactions between y and x , we compare the set of loss terms associated with the full model L (ˆ y ( full ) v , y v ) to those of the reduced model L (ˆ y ( reduced ) v , y v ) , noting that the precise functional form of L depends on the domain.

Original: If x does not Granger cause y , the full and reduced models will have similar explanatory power and hence similar loss values.
Revised: If x does not Granger cause y , the full and reduced models will have similar explanatory power and hence similar loss values.

Original: Otherwise, the full model will have a lower loss.
Revised: Otherwise, the full model will have a lower loss.

Original: We assess this by a one-tailed Welch’s t -test, where the alternative hypothesis is that the mean of the loss terms for the full model is less than the mean of theloss terms for the reduced model.
Revised: We assess this by a one-tailed F-test, comparing the residual sum of squares of the reduced and full models; an alternative assessment using Welch’s t

Original: We then score the interaction between each pair of variables as the − log 10 ( p ) -value associated with the Welch’s t -test and rank the set of candidate x – y interactions by these scores, with a higher score indicating amore signiﬁcant Granger causal interaction.
Revised: -test yielded very similar results (Appendix A.7). We rank the set of candidate x – y interactions by the F-statistic, with a higher score corresponding to stronger evidence for a Granger causal interaction.

Original: 2. D OMAIN - SPECIFIC M ODEL C USTOMIZATION
Revised: 2. D OMAIN - SPECIFIC M ODEL C USTOMIZATION

Original: The loss L should be chosen as appropriate for the domain.
Revised: The loss L should be chosen as appropriate for the domain.

Original: In our single-cell context, y corresponds to gene expression.
Revised: In our single-cell context, y corresponds to gene expression.

Original: We preprocessed RNA-seq transcript counts, normalizing and log transforming them so that the mean squared error loss became appropriate; we note that most methods based on
Revised: We preprocessed RNA-seq transcript counts, normalizing and log transforming them so that the mean squared error loss was appropriate; we note that most methods based on

Original: Pearson correlation of gene expression also seek to minimize squared loss, explicitly or implicitly.
Revised: Pearson correlation of gene expression also seek to minimize squared loss, explicitly or implicitly.

Original: The  modeldescribed above can also be customized to account for different degrees of lag.
Revised: The GrID-Net model can also be customized to account for different lags and lookbacks.

Original: The number of GNN layers L corresponds to the maximum amount of past information desired.
Revised: The number of GNN layers L corresponds to the maximum amount of past information desired.

Original: Similarly, to introduce a k -hop lag, the ﬁrst k GNN layers would use the matrix A while the later layers would use A + .
Revised: Similarly, to introduce a k -hop lag, the ﬁrst k GNN layers would use the matrix A while the later layers would use A + .

Original: Note that in the above, we described a one-hop lag, which we use for all analyses.
Revised: Note that in the above, we described a one-hop lag, which we used for all analyses.

Original: 2. T RAINING D ETAILS AND H YPERPARAMETERS
Revised: 2. T RAINING D ETAILS AND H YPERPARAMETERS

Original: GrID-Net models were trained using the Adam optimizer with a learning rate of 0.001 for 20 epochs or until convergence (deﬁned to be the point at which the relative change in the loss function is less than 0 .
Revised: GrID-Net models were trained using the Adam optimizer with a learning rate of 0.001 for 20 epochs or until convergence (deﬁned to be the point at which the relative change in the loss function is less than 0 .

Original: 1 / |P| across consecutive epochs).
Revised: 1 / |P| across consecutive epochs).

Original: A minibatch size of 1024 candidate peak–gene pairs was used during training, and trainable parameters in the model were initialized using Glorot initialization (Bengio & Glorot, 2010).
Revised: A minibatch size of 1024 candidate peak–gene pairs was used during training, and trainable parameters in the model were initialized using Glorot initialization (Bengio & Glorot, 2010).

Original: All GrID-Net models consisted of L = 10 GNN layers.
Revised: All GrID-Net models consisted of L = 10 GNN layers; the architectures of the three sub-models ( ˜ h ( reduced ) y , ˜ h ( full ) y , and ˜ h ( full ) x ) were identical but separate.

Original: All models were implemented in PyTorch and trained on a single NVIDIA Tesla V100 GPU.
Revised: All models were implemented in PyTorch and trained on a single NVIDIA Tesla V100 GPU.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Single-cell multimodal methods: The emergence of single-cell multimodal assays has led to the development of tools to leverage the joint proﬁling of multiple genomic, epigenetic, and functional features in each cell.
Revised: Single-cell multimodal methods: The emergence of single-cell multimodal assays has led to the development of tools to leverage the joint proﬁling of multiple genomic, epigenetic, and functional features in each cell.

Original: Many of these tools aim to synthesize the multiple modalities within a dataset to perform analysis tasks, such as dimensionality reduction, clustering, and visualization, that elucidate cell state or gene programs (Singh et al., 2021; Argelaguet et al., 2020;
Revised: Many of these tools aim to synthesize the multiple modalities within a dataset to perform downstream analyses that elucidate cell state or gene programs (Singh et al., 2021; Argelaguet et al., 2020;

Original: Gayoso et al., 2021).
Revised: Gayoso et al., 2021).

Original: Meanwhile, efforts to detect associations at the level of speciﬁc features across modalities have thus far been limited.
Revised: Meanwhile, efforts to detect associations at the level of speciﬁc features across modalities have thus far been limited.

Original: In particular, one of the primary opportunities made available by assays that proﬁle chromatin accessibility and gene expression in the same cell is the identiﬁcation of putative regulatory associations between accessible regions and genes.

Original: However, current approaches for inferring these associations predominately rely on simply calculating correlations between these two modalities (Ma et al., 2020; Zhu et al., 2019; Liu et al., 2019).
Revised: Current approaches for inferring peak–gene associations predominately rely on simply calculating correlations between these two modalities (Ma et al., 2020; Zhu et al., 2019; Liu et al., 2019).

Original: Not only do these correlationsbased approaches fail to account for the temporal asynchrony between chromatin accessibility and gene expression, but they are also sensitive to the inherent noisiness of single-cell data (Zhang et al., 2019).
Revised: Not only do these correlationsbased approaches fail to account for the temporal asynchrony between chromatin accessibility and gene expression, but they are also sensitive to the inherent noisiness of single-cell data (Zhang et al., 2019).

Original: These limitations point to the need for methods that go beyond analyzing mere correlations to leverage the joint dynamics of these features in causal inference frameworks, like Granger causality.
Revised: These limitations point to the need for methods that go beyond analyzing mere correlations to leverage the joint dynamics of these features in causal inference frameworks, like Granger causality.

Original: Granger causality: Recent approaches for extending Granger causality have primarily focused on enabling the detection of nonlinear  interactions in multivariate time series data.
Revised: Granger causality: Recent approaches for extending Granger causality have primarily focused on enabling the detection of nonlinear or graph-based interactions in multivariate time series data.

Original: One category of methods involves nonlinear kernel-based regression modelsthat infer linear Granger causality in a transformed feature space (Marinazzo et al., 2008; Ren et al., 2020).
Revised: One category of methods involves nonlinear kernel-based regression models (Marinazzo et al., 2008; Ren et al., 2020); another seeks to learn graphs that describe causal relationships between variables (Basu et al., 2015; Shojaie & Michailidis, 2010).

Original: Neural network-based Granger causality models have also more recently been proposed to account for more generalized nonlinear interactions (Tank et al., 2018; Marcinkeviˇcs & Vogt, 2021).
Revised: Neural network-based Granger causality models have also more recently been proposed to account for more generalized nonlinear interactions (Tank et al., 2018; Marcinkeviˇcs & Vogt, 2021).

Original: Of these methods, efforts have been made to leverage speciﬁc neural network architectures to more fully take advantage of the sequential ordering of observations in time series data.
Revised: Of these methods, efforts have been made to leverage speciﬁc neural network architectures to more fully take advantage of the sequential ordering of observations in time series data (Khanna & Tan, 2020; Nauta et al., 2019).

Original: Khanna & Tan (2020) proposed the economy statistical recurrent unit (eSRU) architecture to exploit the expressiveness of SRU units in a sample-efﬁcient manner.
Revised: Customized Granger causal approaches have also been designed for speciﬁc biological applications (Finkle et al., 2018).

Original: Similarly, Temporal Causal Discovery Framework (TCDF) employs 1D-convolutional neural network layers as a means of capturing long-range temporal information along a single temporal axis (Nauta et al., 2019).
Revised: As a clariﬁcation, we distinguish the DAGs referred to in this paper from the concept of “causal graphs”. Here, each node in the DAG corresponds to a cell with an associated multivariate observation, and edges describe the ﬂow of information between cells.

Original: In  our single-cell peak–gene inference application, performing Granger causal inference on a DAG(where each cell is a node and edges connect similar cells) allows us to capture the diversity of cell states as well as their dynamics.
Revised: In contrast, a causal graph describes a data generation process with nodes corresponding to the dynamical system’s variables. In our single-cell peak–gene inference application, performing Granger causal inference on a DAG allows us to capture the diversity of cell states as well as their dynamics.

Original: We note that using the standard Granger causality framework would require forcing this richly heterogeneous dataset into a sequential ordering of cells along pseudotime, a suboptimal choice due to the possibility of multiple differentiation trajectories, substantial biological and technical variation in single-cell data (Zhang et al., 2019), and multiple cells representing very similar cellular states (Baran et al., 2019). 
Revised: We note that using the standard Granger causality framework would require forcing the rich heterogeneity of a single-cell dataset into a sequential ordering of cells along pseudotime, a suboptimal choice due to the possibility of multiple differentiation trajectories, substantial biological and technical variation in single-cell data (Zhang et al., 2019), and multiple cells representing very similar cell states (Baran et al., 2019). We also note that a limitation of most Granger causal analyses — including ours — is the inability to rule out hidden confounders that might mediate the interaction between x and y (Mastakouri et al., 2021). Nonetheless, such analysis remains valuable in our peak–gene inference task, as it enables the identiﬁcation of peak–gene relationships that are temporally causal, upon which a shortlist of hypotheses can be generated for subsequent perturbational or population genetics studies.

Revised: More importantly, even indirect peak–gene interactions are biologically meaningful.

Revised: In future work, other approaches for causal inference (e.g., Entner & Hoyer (2010); Pﬁster et al.

Revised: including Bayesian inference (Brodersen et al., 2015; Glymour et al., 2019) or structural equation models (Spirtes & Zhang, 2016) could be applied to address this limitation.

Original:  R ESULTS
Revised:  R ESULTS

Original: Datasets and Preprocessing We analyzed three single-cell multimodal datasets with per-cell
Revised: Datasets and Preprocessing We analyzed three single-cell multimodal datasets with per-cell

Original: ATAC-seq and RNA-seq proﬁles that characterize a range of dynamical systems, including cell differentiation and drug-treatment responses (Cao et al., 2018;
Revised: ATAC-seq and RNA-seq proﬁles that characterize a range of dynamical systems, including cell differentiation and drug-treatment responses (Cao et al., 2018;

Original: Chen et al., 2019; Ma et al., 2020).
Revised: Chen et al., 2019; Ma et al., 2020).

Original: The sci-CAR dataset proﬁled 3,260 human lung adenocarcinoma-derived A549 cells after dexamethasone treatment.
Revised: The sci-CAR dataset proﬁled 3,260 human lung adenocarcinoma-derived A549 cells after dexamethasone treatment.

Original: The SNARE-seq dataset evaluated 5,081 cells from the mouse neonatal cerebral cortex.
Revised: The SNARE-seq dataset evaluated 5,081 cells from the mouse neonatal cerebral cortex.

Original: Lastly, the SHARE-seq dataset contained 34,774 cells from differentiating skin cells in mouse.
Revised: Lastly, the SHARE-seq dataset contained 34,774 cells from differentiating skin cells in mouse.

Original: We applied geometric sketching (Hie et al., 2019) to identify a representative subset of 5,000 cells for the SHARE-seq dataset, which we use for all downstream analyses.
Revised: We applied geometric sketching (Hie et al., 2019) to identify a representative subset of 5,000 cells for the SHARE-seq dataset, which we use for all downstream analyses.

Original: These datasets are high-dimensional (about 200 , 000 peaks and 30 , 000 genes per cell) and sparse (over 99 .
Revised: These datasets are high-dimensional (about 200 , 000 peaks and 30 , 000 genes per cell) and sparse (over 99 .

Original: 5% of peaks in a cell have zero counts).
Revised: 5% of peaks in a cell have zero counts).

Revised: For each dataset, GrID-Net produces results that are speciﬁc to the cell type/state composition of the study.

Revised: While we chose these studies to be tissue-speciﬁc, there remains some diversity in cell type and state due to cell-differentiation and perturbation-response variability.

Revised: To ﬂexibly hone in on speciﬁc cell types within a study, users can adapt GrID-Net by simply ﬁltering the dataset to preserve only the cells of interest.

Original: Following recommended practice (Hie et al., 2020), we preprocessed each dataset by applying log(1+ CPM / 10) and log(1+ CPM / 100) transformations to the raw RNA-seq and ATAC-seq count data, respectively; here CPM indicates counts per million.
Revised: Following recommended practice (Hie et al., 2020), we preprocessed each dataset by applying log(1+ CPM / 10) and log(1+ CPM / 100) transformations to the raw RNA-seq and ATAC-seq count data, respectively; here CPM indicates counts per million.

Original: The transformed RNA-seq data for each gene was also divided by the maximum value for that gene.
Revised: The transformed RNA-seq data for each gene was also divided by the maximum value for that gene.

Original: For each single-cell multimodal dataset, we then compiled a set of candidate peak–gene pairs (i.e., P in Eqn. 10) by selecting ATAC-seq peaks within 1 Mb from each gene (Appendix A.2).
Revised: For each single-cell multimodal dataset, we then compiled a set of candidate peak–gene pairs (i.e., P in Eqn. 8) by selecting ATAC-seq peaks within 1 Mb from each gene (Appendix A.3).

Original: We chose this genomic-distance cutoff because almost all enhancer-gene pairs reported in a recent benchmark study were within 1 Mb (Moore et al., 2019).
Revised: We chose this genomic-distance cutoff because almost all enhancer-gene pairs reported in a recent benchmark study were within 1 Mb (Moore et al., 2019).

Original: After this ﬁltering, we evaluated 507,408, 916,939, and 872,039 candidate peak–gene pairs for the sci-CAR, SHARE-seq, and SNARE-seq datasets, respectively.
Revised: After this ﬁltering, we evaluated 507,408, 916,939, and 872,039 candidate peak–gene pairs for the sci-CAR, SHARE-seq, and SNARE-seq datasets, respectively.

Original: Training GrID-Net on these sets of candidate peak–gene pairs took roughly 2–6 hours and required a maximum of 40 GB of 
Revised: Training GrID-Net on these sets of candidate peak–gene pairs took roughly 2–6 hours and required a maximum of 40 GB of RAM.

Original: RAM. Additional runtime and memory usage details can be found in Appendix A.7.
Revised: We further discuss scalability, runtime and memory usage details along with recommendations for using geometric sketching (Hie et al., 2019) for large datasets in Appendix A.12.

Original: DAG construction For each of the three multimodal datasets, a kNN( k =15) graph was constructed on representations of cells that uniﬁed information from both gene expression and chromatin accessibility via Schema (Appendix A.1, Singh et al.
Revised: DAG construction For each of the three multimodal datasets, a kNN graph was constructed on representations of cells that uniﬁed information from both gene expression and chromatin accessibility via Schema (Appendix A.1, Singh et al.

Original:  A pseudotime value was then inferred for each cell using diffusion pseudotime (Haghverdi et al., 2016), after which edges in the kNN graph are retained if they align with the direction of increasing pseudotime.
Revised: ; other approaches for integrating data modalities (e.g., manifold alignment (Cao et al., 2020)) could also be used here. A pseudotime value was then inferred for each cell, after which edges in the kNN graph are retained if they align with the direction of increasing pseudotime.

Original: Since each edge is oriented in the direction of increasing pseudotime, no cycles exist and the resulting graph is a DAG.
Revised: Since each edge is oriented in the direction of increasing pseudotime, no cycles exist and the resulting graph is a DAG.

Revised: In our results below, we used k =15 to build the kNN graph and the diffusion pseudotime algorithm

Revised: (Haghverdi et al., 2016) to infer pseudotime stamps, although we found GrID-Net to be robust (Appendix A.2) to the choice of k and other approaches for inferring pseudotime, speciﬁcally, Palantir (Setty et al., 2019) and Monocle (Qiu et al., 2017).

Original: Evaluating causality by using independent information A key contribution of this work is a framework for systematically evaluating putative  causal peak–gene relationships; to our knowledge, no such framework currently exists.
Revised: Evaluating causality by using independent information A key contribution of this work is a framework for systematically evaluating putative Granger causal peak–gene relationships; to our knowledge, no such framework currently exists.

Original: Since ground truth genome-scale perturbational data is unavailable, we propose that the prediction of peak–gene associations be compared against independent, cell type-matched information sources.
Revised: Since ground truth genome-scale perturbational data is unavailable, we propose that the prediction of peak–gene associations be compared against independent, cell type-matched information sources.

Original: One information source we use are eQTLs from cell types matched to those proﬁled in the single-cell studies.
Revised: One information source we use are eQTLs from cell types matched to those proﬁled in the single-cell studies.

Original: An eQTL is a locus-gene pair that associates naturally occurring genetic variability at a locus to gene expression variations observed in large populations, thus serving as a proxy for direct perturbation of the locus (Nica & Dermitzakis, 2013).
Revised: An eQTL is a locus-gene pair that associates naturally occurring genetic variability at a locus to gene expression variations observed in large populations, thus serving as a proxy for direct perturbation of the locus (Nica & Dermitzakis, 2013).

Original: We also evaluate against cell type-matched chromatin interaction (Hi-C/5C) data that report pairs of genomic loci that are spatially close in the 3D structure of the genome (Schoenfelder & Fraser, 2019).
Revised: We also evaluate against cell type-matched chromatin interaction (Hi-C/5C) data that report pairs of genomic loci that are spatially close in the 3D structure of the genome (Schoenfelder & Fraser, 2019).

Original: Intuitively, chromatin interaction and chromatin accessibility are both manifestations of the underlying gene regulatory process ( Figure 1d): a non-coding region becomes accessible (ATAC-seq) for binding by a gene regulator and the chromatin 3D structure is arranged (chromatin interaction) so that the regulator is proximal to the target gene’s promoter, thus enabling transcriptional control (Delaneau et al., 2019).
Revised: Intuitively, chromatin interaction and chromatin accessibility are both manifestations of the underlying gene regulatory process ( Figure 1d): a noncoding region becomes accessible (ATAC-seq) for binding by a gene regulator and the chromatin 3D structure is arranged (chromatin interaction) so that the regulator is proximal to the target gene’s promoter, thus enabling transcriptional control (Delaneau et al., 2019).

Original: As such, compared to random peak–gene pairs, causal pairs should more likely be in close spatial proximity.
Revised: As such, compared to random peak–gene pairs, causal pairs should more likely be in close spatial proximity.

Original: We note that, in absolute terms, a high overlap of  causal peak–gene pairs with these datasets should not be expected.
Revised: We note that, in absolute terms, a high overlap of Granger causal peak–gene pairs with these datasets should not be expected.

Original: Like ATAC-seq, both eQTLs and chromatin interaction capture the underlying biology only indirectly and are themselves prone to errors: eQTLs are confounded by correlated mutations elsewhere in the genome and Hi-C/5C data suffers from similar sparsity issues as ATACseq.
Revised: Like ATAC-seq, both eQTLs and chromatin interaction capture the underlying biology only indirectly and are themselves prone to errors: eQTLs are confounded by correlated mutations elsewhere in the genome and Hi-C/5C data suffers from similar sparsity issues as ATACseq.

Original: Therefore, we focus here on a relative evaluation, comparing GrID-Net’s overlap with these datasets against that of alternative approaches.
Revised: Therefore, we focus here on a relative evaluation, comparing GrID-Net’s overlap with these datasets against that of alternative approaches.

Original: Also, while good overlap with both eQTL and chromatin interaction datasets is desirable, we believe emphasis should be given to eQTLs since they are the better proxy for an actual perturbation.
Revised: Also, while good overlap with both eQTL and chromatin interaction datasets is desirable, we believe emphasis should be given to eQTLs since they are the better proxy for an actual perturbation.

Original: Benchmarking We evaluated two correlation-based techniques used in previous single-cell multimodal studies (Cao et al., 2018;
Revised: Benchmarking We evaluated two correlation-based techniques used in previous single-cell multimodal studies (Cao et al., 2018;

Original: Chen et al., 2019; Ma et al., 2020): a) Pearson correlation ( Pearson
Revised: Chen et al., 2019; Ma et al., 2020): a) Pearson correlation ( Pearson

Original: Corr ) of peak counts and gene expression, with each cell considered an independent observation; and b) Pearson correlation computed after averaging the RNA-seq and ATAC-seq proﬁles of each cell and its 50 neighbors in the kNN graph of cells ( Pseudocell Corr ).
Revised: Corr ) of peak counts and gene expression, with each cell considered an independent observation; and b) Pearson correlation computed after averaging the RNA-seq and ATAC-seq proﬁles of each cell and its 50 neighbors in the kNN graph of cells ( Pseudocell Corr ).

Original: The pseudocell approach, currently the state of the art, seeks robustness by replacing point estimates in the ATAC-seq (or
Revised: The pseudocell approach, currently the state-of-the-art, seeks robustness by replacing point estimates in the ATAC-seq (or

Original: RNA-seq) space with local averages.
Revised: RNA-seq) space with local averages.

Original: We use the absolute value of the correlation scores to rank peak–gene pairs in our analyses below.
Revised: We use the absolute value of the correlation scores to rank peak–gene pairs in our analyses below.

Original: To assess the advantage ofworking with a partial ordering and operating on the kNN DAG, we also computed a traditional Granger causal estimate using vector autoregression ( VAR Granger  ).
Revised: To assess the advantage of operating on a DAG for partially ordered data, we also computed a traditional, linear Granger causal estimate using vector autoregression ( VAR Granger ) and a state-of-the-art generalized vector autoregression method for nonlinear Granger causal inference ( GVAR ).

Original: There, data was forced into a total ordering along pseudotime by partitioning the pseudotime range into 100 bins of equal size and averaging theATAC-seq and
Revised: For both these methods, data was forced into a total ordering along pseudotime by partitioning the pseudotime range into 100 bins of equal size and averaging the

Original:  RNA-seq proﬁles across cells assigned to each bin (Appendix A.3).
Revised: ATAC-seq and RNA-seq proﬁles across cells assigned to each bin (Appendix A.4).

Revised: We note that the

Revised: Pseudocell Corr method uses information from just the kNN graph structure, while both the VAR

Revised: Granger and GVAR methods use only pseudotime information.

Revised: By making use of both the kNN graph and pseudotime information, GrID-Net is able to combine their strengths.

Original: Predicting eQTLs
Revised: Predicting eQTLs

Original: We obtained eQTL data from the GTEx project for the human cell types that most closely correspond to the cells from the single-cell multimodal assays (GTEx Consortium, 2013).
Revised: We obtained eQTL data from the GTEx project for the human cell types that most closely correspond to the cells from the single-cell multimodal assays (GTEx Consortium, 2013).

Original: For the lung adenocarcinoma-derived A549 cells in the sci-CAR dataset, we used eQTL data for human lungs.
Revised: For the lung adenocarcinoma-derived A549 cells in the sci-CAR dataset, we used eQTL data for human lungs.

Original: Meanwhile, for the mouse skin and neonatal cortex cells from the SHARE-seq and SNARE-seq datasets, respectively, we used eQTL data for human skin and thehuman cerebral cortex, mapping them to the mouse genome using liftOver (Gonzalez et al., 2021).
Revised: The SHARE-seq and SNARE-seq studies proﬁled mouse tissues, for which matching eQTL data is limited. Since human and mouse epigenomes show strong conservation (Gjoneska et al., 2015; Xiao et al., 2012), we used eQTL data for human skin and the cerebral cortex, respectively (see Appendix A.5), and mapped this data to the mouse genome.

Original: For each dataset, we retained peak–gene pairs with a matching locus–gene pair in the eQTL data (Appendix A.4).
Revised: For each dataset, we retained peak–gene pairs with a matching locus–gene pair in the eQTL data.

Original: In order to assess the effectiveness of GrID-Net in predicting eQTLs, we labeled peak–gene pairs associated with eQTLs as true (eQTL p < 10 − 10 ) or false (eQTL p > 0 . 9 ), discarding pairs that were not in either category.
Revised: To assess the effectiveness of GrID-Net in predicting eQTLs, we labeled peak–gene pairs associated with eQTLs as true (eQTL p < 10 − 10 ) or false (eQTL p > 0 . 9 ), discarding pairs that were not in either category.

Original: GrID-Net and the alternative methods were evaluated on their accuracy in predicting the true eQTLs based on the ranking of association scores generated by each method for each peak–gene pair.
Revised: GrID-Net and the alternative methods were evaluated on their accuracy in predicting the true eQTLs based on the ranking of association scores generated by each method for each peak–gene pair.

Original: Across allsingle-cell datasets, GrID-Netdramatically outperformed the other methods in predicting eQTLs, achieving the highest AUPRC (area under precision-recall curve) and AUROC (area under receiver operating characteristic) in all cases, indicating that peak–gene pairs prioritized by GrID-Net more closely align with evidence from population genetics (Table 1).
Revised: Across all datasets, GrID-Net outperformed the other methods in predicting eQTLs, achieving the highest AUPRC (area under precision-recall curve) and AUROC (area under receiver operating characteristic) in all cases, indicating that peak–gene pairs prioritized by GrID-Net more closely align with evidence from population genetics (Table 1).

Original: We also found the two correlation approaches to have similar predictive power to each other, suggesting that simply averaging values in a local neighborhood is not sufﬁcient.

Original: The VAR Granger method performed substantially worse than GrID-Net as well, which points to the advantage of  DAG-basedcausal inference  for such systems over standard causal inference on totally ordered data [Table]
Revised: The non-linear Granger approach (GVAR) performs slightly better than linear approach (VAR Granger); however, both methods performed substantially worse than GrID-Net, pointing to the advantage of the DAG-based inference approach for such systems over standard Granger-causal inference on forcibly-ordered data.

Original: [Table]
Revised: [Table]

Original: Predicting chromatin interactions We obtained chromatin interaction data for the speciﬁc cell types and experimental settings proﬁled in the three single-cell multimodal datasets (D’Ippolito et al., 2018; Dixon et al., 2012; Poterlowicz et al., 2017).
Revised: Predicting chromatin interactions We obtained chromatin interaction data for the speciﬁc cell types and experimental settings proﬁled in the three single-cell multimodal datasets (Davis et al., 2018; Dixon et al., 2012; Poterlowicz et al., 2017).

Original: Chromatin interactions were determined for a peak–gene pair by identifying the pairs of genomic windows in the corresponding chromatin interaction dataset that overlapped the peak and the transcription start site (TSS) of the gene.
Revised: Chromatin interactions were determined for a peak–gene pair by identifying the pairs of genomic windows in the corresponding chromatin interaction dataset that overlapped the peak and the transcription start site (TSS) of the gene.

Original: We then identiﬁed the peak–gene pairs with signiﬁcant chromatin interactions (Appendix A.5) and aimed to compare the accuracy of GrID-Net and alternative methods in predicting these pairs.
Revised: We then identiﬁed the peak–gene pairs with signiﬁcant chromatin interactions (Appendix A.6) and compared the accuracy of GrID-Net and alternative methods in predicting these pairs.

Original: We found that GrID-Net achieved the highest AUPRC  values in this prediction task across all datasets, demonstrating its ability to better identify these key functional features in gene regulation (Table 2).
Revised: We found that GrIDNet achieved the highest AUPRC and AUROC values in this prediction task across all datasets, demonstrating its ability to better identify these key functional features in gene regulation (Table 2) [Table]

Original: It also achieved the highest AUROC values in all but the sci-CAR dataset. As a measure, AUROC is more susceptible to class imbalances than AUPRC and this may explain GrID-Net’s muted performance on that metric in the sci-CAR data.
Revised: interaction prediction accuracy on three multimodal datasets

Original: [Table]
Revised: [Table]

Original: GrID-Net is more robust to sparse data than existing approaches With the sparsity of singlecell multimodal data expected to remain a concern, we assessed GrID-Net’s robustness to such sparsity.
Revised: GrID-Net is more robust to sparse data than existing approaches With the sparsity of singlecell multimodal data expected to remain a concern, we assessed GrID-Net’s robustness to such sparsity.

Revised: In particular, we hypothesized that GrID-Net’s message passing procedure, which aggregates sparse and noisy measurements across cells using the DAG’s structure

Original: Focusing on the subset of genes or peaks where data is especially sparse (fewer than 5% or 2% of cells for genes or peaks, respectively), we evaluated it on the eQTL prediction task, ﬁnding that its outperformance compared to the alternatives was accentuated in this subset (Table 3).
Revised: [Table] would offer it robustness to sparsity. Because of differing true-positive rates between the full set of candidate peak–gene pairs and the sparse-data subset (Appendix A.9, Table 15), the relative performance of methods are compared within rather than across settings. We focused on the subset of genes or peaks for which data is especially sparse (fewer than 5% or 2% of cells for genes or peaks, respectively).

Revised: Evaluating the various methods on the eQTL prediction task on this subset, we found that GrID-Net’s outperformance compared to the alternatives was accentuated (Table 3, additional details in Appendix A.9).

Original: GrIDNet also noticeably outperforms the other methods in predicting signiﬁcant chromatin interactions associated with peak–gene pairs consisting of sparse genes or peaks.
Revised: GrID-Net also noticeably outperforms the other methods in predicting signiﬁcant chromatin interactions associated with peak–gene pairs consisting of sparse genes or peaks.

Original: These results provide further evidence of GrID-Net’s utility in extracting insights from sparse single-cell data.
Revised: These results provide further evidence of GrID-Net’s utility in extracting insights from sparse single-cell data

Original: [Table]
Revised: [Table]

Original:  Peak–gene pairs detected by GrID-Net are enriched for TF binding motifs To assess the functional relevance of the detected peak–gene pairs, we aimed to relatethese peaks  to the transcription factors (TFs) that might bind there.
Revised: [Table] Peak–gene pairs prioritized by GrID-Net are supported by TF motif and ChIP-seq data To assess the functional relevance of the detected peak–gene pairs, we aimed to relate peaks in high scoring peak–gene pairs to the transcription factors (TFs) that might bind there.

Original: Accordingly, we selected the highest scoring 1% of peak–gene pairs from each of GrID-Net and its alternatives.
Revised: Accordingly, we selected the highest scoring 1% of peak–gene pairs from each of GrID-Net and its alternatives.

Original: We then used Homer (Heinz et al., 2010) to identify TF binding motifs that are co-enriched in sets of peaks linked to a particular gene (Appendix A.6).
Revised: We then used Homer (Heinz et al., 2010) to identify TF binding motifs that are co-enriched in sets of peaks linked to a particular gene.

Original: The presence of these co-enriched motifs is potentially indicative of coordinated regulatory control, whereby a gene is modulated by multiple TF binding events featuring the same or related sets of TFs (Pott & Lieb, 2015; Carleton et al., 2017).
Revised: The presence of these co-enriched motifs is potentially indica- tive of coordinated regulatory control, whereby a gene is modulated by multiple TF binding events featuring the same or related sets of TFs (Pott & Lieb, 2015; Carleton et al., 2017).

Original: Consequently, the co-occurrence of these motifs serves as a proxy for the functional signiﬁcance of the proposed peak–gene pairs.
Revised: Consequently, the co-occurrence of these motifs serves as a proxy for the functional signiﬁcance of the proposed peak–gene pairs.

Original: Across allthree datasets, we observed that the peak–gene pairs detected by GrID-Net were associated with 10–50 times more enriched TF binding motifs than the other methods (Table 4).
Revised: Across all datasets, we observed that the peak–gene pairs detected by GrID-Net were associated with 10–50 times more enriched TF binding motifs than the other methods (Table 4), with GrID-Net’s outperformance being statistically signiﬁcant ( p < 10 − 10 , one-sided binomial test with Bonferroni correction, details in Appendix A.10).

Original: These results suggest that GrID-Net may serve a valuable role in unveiling mechanistic insights into TF-gene regulatory control.
Revised: These results suggest that GrID-Net may serve a valuable role in unveiling mechanistic insights into TF-gene regulatory control.

Revised: We further evaluated the functional importance of detected peak–gene pairs by comparing peaks in these peak–gene pairs to TF binding sites supported by TF ChIP-seq experimental data (Oki et al., 2018).

Revised: We ﬁrst obtained TF ChIP-seq data for the speciﬁc cell type(s) represented in each singlecell multimodal dataset.

Revised: For each of these datasets, we then identiﬁed the 0.1% of peaks involved in the highest scoring peak–gene pairs and determined the proportion of such peaks that overlapped with ChIP-seq-derived TF binding sites.

Revised: We observed that peaks prioritized by GrID-Net more consistently overlapped TF binding sites relative to other methods across all three datasets (Table 4).

Revised: As TF binding sites are indicative of functional regulatory roles for noncoding loci (Valouev et al., 2008), GrID-Net’s effectiveness in prioritizing peaks bound by TFs provides additional evidence for its ability to reveal important gene regulatory roles for speciﬁc loci in the noncoding genome.

Original: [Table]
Revised: [Table]

Original: GrID-Net reveals link between genomic distance andtemporal regulatory control GrID-Net’s explicit consideration of the temporal asynchrony between chromatin accessibility and gene expression enables the genome-wide study of lags in gene regulation for the very ﬁrst time.
Revised: Investigating a link between genomic distance and regulatory control We also explored if larger peak–gene genomic distances correspond to greater temporal lags between peak accessibility and gene expression. By varying L (the number of GNN layers)

Original: We classiﬁed peak–gene pairs as proximal (genomic distance < 10kb) or distal ( ≥ 10 kb). We then evaluated different architectures of GrID-Net that vary in the number of GNN layers L , hypothesizing that architectures with higher L (i.e., the degree of lag between peaks and genes that can be captured) may detect more distal peak–gene pairs. Inspecting the top 1% of peak–gene pairs detected by different architectures, we found this to indeed be the case: across all datasets, architectures with higher L detected smaller proportions of proximal peak–gene pairs (Table 5), suggesting that distal peak– gene interactions may be marked by longer lags between peaks and genes. This ﬁnding indicates, to our knowledge, a previously unreported link between genomic distance and the lag between a regulatory element’s accessibility and its target gene’s expression.
Revised: [Table]

Original: One hypothesis is that the complex coordination of regulatory factors required to enable distal regulatory interactions is associated with greater lags between distal regulatory elements’ accessibility and the gene expression changes they affect (Panigrahi & O’Malley, 2021). GrID-Net, thus, presents a novel result that provides insight into the temporal dynamics governing distal regulatory elements and their gene expression control.
Revised: we allowed for greater pseudotime difference between peaks and genes, ﬁnding that architectures with larger L did produce a larger proportion of distal (vs. proximal) peak–gene pairs in their top hits (Appendix A.11; Tables 16, 17)

Original: [Table]
Revised: [Table]

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: We extend the applicability of Granger causal analysis to dynamical systems represented as DAGs, introducing GrID-Net to perform such inference.
Revised: We extend the applicability of Granger causal analysis to dynamical systems represented as DAGs, introducing GrID-Net to perform such inference.

Original: To best leverage the dynamics expressed in these

Original: DAG-based systems, GrID-Net takes advantage of the expressive power of graph neural networks, enabling the detection of long-range nonlinear interactions between variables.
Revised: GrID-Net takes advantage of the expressive power of graph neural networks, enabling the detection of long-range nonlinear interactions between variables.

Original: We focused on using GrID-Net to study multimodal single-cell gene regulatory dynamics, given the prevalence of graph-based representations of cellular landscapes in the ﬁeld.
Revised: We focused on applying it to study multimodal single-cell gene regulatory dynamics, given the prevalence of graph-based representations of cellular landscapes in the ﬁeld.

Original: GrID-Net demonstrated substantial improvements in accuracy relative to current methods in predicting peak–gene links from independent, cell type-matched information sources.
Revised: GrID-Net demonstrated substantial improvements in accuracy relative to current methods in predicting peak–gene links from independent, cell type-matched information sources.

Original: GrID-Net exhibited particularly strong performance in predicting links involving speciﬁc peaks and genes for which the data was sparse, a feat indicative of its value for extracting meaningful insights from sparse single-cell data.

Original: GrID-Net’s overall improvement in accuracy for the eQTL and chromatin interaction prediction tasks is also noteworthy, as it reveals important advantages that can be unlocked by using GrID-Net for studying gene regulation in future work.

Original: The large population sizes needed for eQTL studies and the relatively low resolution of chromatin interaction data (e.g., tens of kilobases in Hi-C) currently act as major limitations for exploring the space of genomic loci-gene interactions.

Original: With the higher resolution of ATAC peaks ( ∼ 1 kb), regulatory links detected by GrID-Net from single-cell multimodal assays may be used tomore precisely hone in on functional regions within noncoding loci oreven be used to complement existing chromatin interaction data to aid in this goal (Fulco et al., 2019; Nasser et al., 2021).
Revised: Capitalizing on the high resolution of ATAC peaks ( ∼ 1 kb), regulatory links detected by GrID-Net from single-cell multimodal assays may be used to precisely hone in on functional regions within noncoding loci or to complement existing chromatin interaction data towards this goal (Fulco et al., 2019; Nasser et al., 2021).

Original: This increased resolution, combined with the genome-wide scale of single-cell multimodal assays, positions GrID-Net to serve as a vital tool for furthering our understanding of key aspects of gene regulatory dynamics relevant to fundamental biology and disease.
Revised: This increased resolution, combined with the genome-wide scale of single-cell multimodal assays, positions GrID-Net to serve as a vital tool for furthering our understanding of key aspects of gene regulatory dynamics relevant to fundamental biology and disease.

Original: Lastly, the ever growing quantities of network data across numerous non-biological domains, including social media networks and ﬁnancial transaction networks, also points to the broader applicability of GrID-Net.
Revised: Additionally, the ever growing quantities of network data across numerous non-biological domains, including social media networks and ﬁnancial transaction networks, also points to the broader applicability of GrID-Net.

Original: The ability of GrID-Net to capture long-range, nonlinear interactions in DAGs opens the door to novel analyses on the many existing datasets that are characterized by these graph-based dynamics as well. 
Revised: The ability of GrID-Net to capture long-range, nonlinear interactions in DAGs opens the door to novel analyses on the many existing datasets that are characterized by these graph-based dynamics as well. A CKNOWLEDGEMENTS

Revised: We thank the anonymous reviewers of ICLR 2022 for their valuable feedback and suggestions.

Revised: Figure 1 was created using biorender.io.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: In order to ensure that the full set of results presented in this paper are reproducible, we have included source code for our method, which can be found in the supplementary materials.
Revised: In order to ensure that the full set of results presented in this paper are reproducible, we have included source code for our method, which can be found in the supplementary materials.

Original: In addition, all analyses described in the paper were performed on publicly available datasets, for which we have provided references throughout the main text.
Revised: In addition, all analyses described in the paper were performed on publicly available datasets, for which we have provided references throughout the main text.

Original: Similarly, the software tools used for biology-speciﬁc analyses (e g., Homer) are publicly available, and we have described the speciﬁc settings that were used for this study in the “Appendix” section.
Revised: Similarly, the software tools used for biology-speciﬁc analyses (e g., Homer) are publicly available, and we have described the speciﬁc settings that were used for this study in the “Appendix” section.

Original: We have also detailed pre-processing steps for the various single-cell datasets in the main text and included both pre-processing and analysis procedures related to the eQTL, Hi-C/5C, and TF binding motif data in the “Appendix” section.
Revised: We have also detailed pre-processing steps for the various single-cell datasets in the main text and included both pre-processing and analysis procedures related to the eQTL, Hi-C/5C, and TF binding motif data in the “Appendix” section.

Original: We also made sure to provide details on the speciﬁc training details and hyperparameter choices related to training our neural network model in a dedicated subsection in the “Methods” section entitled “Training
Revised: We also made sure to provide details on the speciﬁc training details and hyperparameter choices related to training our neural network model in a dedicated subsection in the “Methods” section entitled “Training

Original: Details and Hyperarameters”
Revised: Details and Hyperarameters”.

Original: A A PPENDIX A. DAG CONSTRUCTION 
Revised: A A PPENDIX A. DAG CONSTRUCTION DETAILS

Original: For the three single-cell multimodal datasets, we applied a standard log(1+ CPM / 10) transformation to the raw RNA-seq counts data and a log(1+ CPM / 100) transformation to the raw ATAC-seq counts data, where CPM indicates counts per million.
Revised: For the three single-cell multimodal datasets, we applied a standard log(1+ CPM / 10) transformation to the raw RNA-seq counts data and a log(1+ CPM / 100) transformation to the raw ATAC-seq counts data, where CPM indicates counts per million.

Original: Genes and accessible regions detected in fewer than 0.1% of cells in each dataset were excluded.
Revised: Genes and accessible regions detected in fewer than 0.1% of cells in each dataset were excluded.

Original: For each dataset, the expression of each gene was scaled to unit variance and zero mean, after which the top 2000 most highly variable genes (Satija et al., 2015) were retained.
Revised: For each dataset, the expression of each gene was scaled to unit variance and zero mean, after which the top 2000 most highly variable genes (Satija et al., 2015) were retained.

Original: In the original SHARE-seq and SNARE-seq studies, topic modelingbased approaches were used to transform ATAC-seq counts data for downstream trajectory analyses, prompting us to transform the ATAC-seq counts in these datasets using tf-idf , a procedure derived from text-based topic modeling that has been applied extensively to analyze single-cell ATAC-seq data (Cusanovich et al., b;c;a).
Revised: In the original SHARE-seq and SNARE-seq studies, topic modelingbased approaches were used to transform ATAC-seq counts data for downstream trajectory analyses, prompting us to transform the ATAC-seq counts in these datasets using tf-idf , a procedure derived from text-based topic modeling that has been applied extensively to analyze single-cell ATAC-seq data (Cusanovich et al., b;c;a).

Original: The normalized RNA-seq and ATAC-seq data were then projected onto their top 100 principal components (excluding components that have Spearman correlation ρ > 0 .
Revised: The normalized RNA-seq and ATAC-seq data were then projected onto their top 100 principal components (excluding components that have Spearman correlation ρ > 0 .

Original: 9 with the total counts per cell).
Revised: 9 with the total counts per cell).

Original: Schema (Singh et al., 2021) was then used to generate uniﬁed representations for each cell that synthesize information from both the RNA-seq and ATAC-seq representations.
Revised: Schema (Singh et al., 2021) was then used to generate uniﬁed representations for each cell that synthesize information from both the RNA-seq and ATAC-seq representations.

Original: For the sci-CAR and SNARE-seq datasets, RNA-seq was used as the primary modality in Schema with ATAC-seq acting as a secondary modality.
Revised: For the sci-CAR and SNARE-seq datasets, RNA-seq was used as the primary modality in Schema with ATAC-seq acting as a secondary modality.

Original: For the sci-CAR dataset, the time of data collection post-drug treatment was used as a tertiary modality.
Revised: For the sci-CAR dataset, the time of data collection post-drug treatment was used as a tertiary modality.

Original: Meanwhile, ATAC-seq served as the primary modality for the SHARE-seq dataset, as the original SHARE-seq study identiﬁed strong cell cycle effects represented in the RNA-seq data for certain subpopulations of cells and consequently used ATAC-seq as the modality for analyzing lineage dynamics for the dataset (Ma et al., 2020).
Revised: Meanwhile, ATAC-seq served as the primary modality for the SHARE-seq dataset, as the original SHARE-seq study identiﬁed strong cell cycle effects represented in the RNA-seq data for certain subpopulations of cells and consequently used ATAC-seq as the modality for analyzing lineage dynamics for the dataset (Ma et al., 2020).

Original: We set the minimum correlation between the primary and the remaining modalities to be 0.9 in Schema.
Revised: We set the minimum correlation between the primary and the remaining modalities to be 0.9 in Schema.

Original: For each dataset, a kNN graph ( k = 15 ) based on distances between the uniﬁed representations generated by Schema was constructed.
Revised: For each dataset, a kNN graph ( k = 15 ) based on distances between the uniﬁed representations generated by Schema was constructed.

Original: Pseudotime stamps were then inferred for each cell based on this kNN graph  (Haghverdi et al., 2016).
Revised: Pseudotime stamps were then inferred for each cell based on this kNN graph using diffusion pseudotime (Haghverdi et al., 2016).

Original: Finally, edges in the kNN graph that cohere with the direction of increasing pseudotime are retained to generate a DAG. 
Revised: Finally, edges in the kNN graph that cohere with the direction of increasing pseudotime are retained to generate a DAG. A. DAG ROBUSTNESS ANALYSIS In addition to the diffusion pseudotime algorithm [Table]

Revised: we also evaluated Palantir and Monocle for inferring pseudotime.

Revised: The performance of GrID-Net was robust to the choice of algorithm (Tables 5, 6,7,8).

Original:  A. S INGLE - CELL DATA PREPROCESSING AND CANDIDATE PEAK – GENE PAIR SELECTION
Revised: [Table] [Table] [Table] A. S INGLE - CELL DATA PREPROCESSING AND CANDIDATE PEAK – GENE PAIR SELECTION

Original: We applied the same CPM and log transformations as above to the raw RNA-seq and ATAC-seq counts data for all three single-cell multimodal datasets.
Revised: We applied the same CPM and log transformations as above to the raw RNA-seq and ATAC-seq counts data for all three single-cell multimodal datasets.

Original: We then scaled the RNA-seq data for each gene by the maximum value for that gene, so that expression values for all genes in each dataset are in the same [0,1] range.
Revised: We then scaled the RNA-seq data for each gene by the maximum value for that gene, so that expression values for all genes in each dataset are in the same [0,1] range.

Original: The peak–gene pairs considered for each dataset were determined by selecting genes and peaks detected in greater than 0.1% of cells and then retaining all peak–gene pair combinations in which the peak is within 1 Mb of the gene.
Revised: The peak–gene pairs considered for each dataset were determined by selecting genes and peaks detected in greater than 0.1% of cells and then retaining all peak–gene pair combinations in which the peak is within 1 Mb of the gene.

Original: For the SHARE-seq dataset, we only considered peaks detected in greater than 1% of cells so as to have a comparable set of candidate peak–gene pairs across the three datsets.
Revised: For the SHARE-seq dataset, we only considered peaks detected in greater than 1% of cells so as to have a comparable set of candidate peak–gene pairs across the three datsets.

Original: A. B ENCHMARKING
Revised: A. B ENCHMARKING

Original: For the VAR Granger model, peak–gene pairs were scored on the − log 10 ( p ) -value associated with the statistical signiﬁcance of Granger causality, as calculated using the grangercausalitytests function with default settings in the statsmodels package (Seabold & Perktold, 2010).
Revised: For the VAR Granger model, peak–gene pairs were scored on the − log 10 ( p ) -value associated with the statistical signiﬁcance of Granger causality, as calculated using the grangercausalitytests function with default settings in the statsmodels package (Seabold & Perktold, 2010).

Original: A. M ATCHING E QTL S WITH SINGLE - CELL ATAC- SEQ & RNA- SEQ DATA
Revised: A. M ATCHING E QTL S WITH SINGLE - CELL ATAC- SEQ & RNA- SEQ DATA

Original: The procedure to match a peak–gene pair with a eQTL variant–gene pair involved identifying all catalogued variants positioned in the peak of interest and selecting the variant with the most statistically signiﬁcant association with the gene in the peak–gene pair.
Revised: The procedure to match a peak–gene pair with a eQTL variant–gene pair involved identifying all catalogued variants positioned in the peak of interest and selecting the variant with the most statistically signiﬁcant association with the gene in the peak–gene pair.

Original: To perform this procedure for the 
Revised: To perform this procedure for the SNARE-seq and SHARE-seq datasets, we used liftOver (Gonzalez et al., 2021) with default [Table] settings to convert human genomic coordinates represented in the GTEx datasets to mouse coordinates. We chose to use human eQTL data here because it is important to obtain tissue-speciﬁc eQTL data.

Revised: Unfortunately, there is relatively little eQTL data available in mouse and, to our knowledge, when available it is organized around different strains of mice rather than tissues (Doss et al., 2005;

Original: SNARE-seq and SHARE-seq datasets, we used liftOver (Gonzalez et al., 2021) with default settings to convert human genomic coordinates represented in the GTEx datasets to mouse coordinates
Revised: Hofstetter et al., 2007). On the other hand, the transfer of eQTL knowledge from human to mouse is well-motivated because the epigenome is largely conserved across mouse and human (Gjoneska et al., 2015; Xiao et al., 2012) [Table]

Original: . A. C HROMATIN INTERACTION DATA PREPROCESSING
Revised: A. C HROMATIN INTERACTION DATA PREPROCESSING

Revised: For the sci-CAR, SHARE-seq

Original: For the SHARE-seq and SNARE-seq datasets, the corresponding chromatin interaction datasets contained scores indicating the frequency of interactions between genomic windows.
Revised: [Table] and SNARE-seq datasets [Table] the corresponding chromatin interaction datasets contained scores indicating the frequency of interactions between genomic windows.

Original: We retained peak–gene pairs that were matched with the genomic windows presented in the chromatin interaction datasets and chose the most frequent 1% of interactions as the set to be predicted.
Revised: We retained peak–gene pairs that were matched with the genomic windows presented in the chromatin interaction datasets and chose the most frequent 1% of interactions as the set to be predicted.

Revised: A. C OMPARISON OF F- TEST AND W ELCH ’ S T - TEST TO RANK PUTATIVE PEAK – GENE INTERACTIONS

Revised: For each candidate peak–gene pair, we compare the full and reduced models using an F-test, as is common in Granger causal inference.

Revised: Speciﬁcally, we evaluate the sum-of-squared-errors of the reduced and full models as per an F distribution with (2 L + 1 , n − 4 L − 1) degrees of freedom.

Revised: Here, n is the number of cells, and for each peak-gene pair, each of the L GNN layers includes two parameters per peak/gene, and c is an additional parameter in the full model).

Revised: Another approach would be to compare the full and reduced models’ losses with the Welch’s t -test, where the alternative hypothesis is that the mean of the loss terms for the full model is less than the mean of the loss terms for the reduced model.

Original: The chromatin interaction data associated with the sci-CAR dataset already included a set of statistically signiﬁcant chromatin interactions across the entire genome, so the  prediction task for the sci-car dataset involved predicting this set of interactions.
Revised: In both approaches, candidate interactions are then ranked by the test statistic with higher scores corresponding to a stronger likelihood of Granger causal interaction. We compared the performance of the two tests on the eQTL and chromatin interaction prediction tasks, ﬁnding that they yielded very similar results (Tables 9,10). This similarity is likely due to our use of the test statistics to rank peak–gene pairs, upon which the top-ranking pairs are selected. Moreover, the degrees of freedom are identical across all candidate pairs, so their ranking depends only on the relative losses of the full and reduced models.

Revised: [Table]

Revised: [Table]

Revised: A. C OMPARISON OF COMBINED AND SEPARATED LOSS FUNCTIONS FOR FULL AND REDUCED MODELS

Revised: During the GNN’s training, the sum of the losses of the full and reduced models is optimized.

Revised: Since the two models do not share any parameters, optimizing the sum of their losses corresponds to optimizing each loss separately.

Revised: We chose this joint-optimization strategy for efﬁciency and to avoid stochastic variations if the models were trained separately.

Revised: Nonetheless, to conﬁrm that optimizing the sum of losses does not introduce any unintended dependencies, we also evaluated

Revised: GrID-Net’s performance on the eQTL and chromatin interaction prediction tasks when the models were optimized in completely separate training runs.

Revised: The results for separate and joint training were essentially identical (Tables 11, 12), suggesting that our joint training works as intended

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: A. R OBUSTNESS OF G R ID-N ET TO SPARSITY

Revised: Tables 13 and 14 expand upon the results in Table 3 by also showing standard deviations (across three runs) of GVAR’s and GrID-Net’s AUPRC results; the other methods are fully deterministic and their results do not change across runs.

Revised: Table 13 shows the baseline proportions of true positives across the different comparison conditions to provide context for differences in the range of AUPRC values across these conditions

Revised: [Table]

Original:  A. TF BINDING MOTIF ENRICHMENT ANALYSIS
Revised: [Table] A. TF BINDING MOTIF ENRICHMENT ANALYSIS

Original: For each gene that was associated with more than 5 peaks in the highest scoring 1% peak–gene pairs, we applied HOMER with default parameters on the genomic windows deﬁned by the set of peaks to test for enrichment of TF binding motifs (Heinz et al., 2010). 
Revised: For each gene that was associated with more than 5 peaks in the highest scoring 1% peak–gene pairs, we applied HOMER with default parameters on the genomic windows deﬁned by the set of peaks to test for enrichment of TF binding motifs (Heinz et al., 2010). We used a BenjaminiHochberg-corrected p-value cutoff of 0 .

Revised: 01 to call enriched TF binding motifs.

Revised: Against each baseline, the statistical signiﬁcance of the difference between GrID-Net and the baseline in the number of enriched motifs was assessed via a one-sided binomial test, with Bonferroni correction for multiple hypothesis testing.

Revised: All comparisons were signiﬁcant at the p < 10 − 10 level.

Original: A.  R UNTIME AND MEMORY USAGE
Revised: A. G R ID-N ET SUGGESTS POSSIBLE LINK BETWEEN GENOMIC DISTANCE AND TEMPORAL REGULATORY CONTROL

Revised: GrID-Net’s explicit consideration of the temporal asynchrony between chromatin accessibility and gene expression enables the genome-wide study of lags in gene regulation for the very ﬁrst time. [Table] [Table]

Revised: We classiﬁed peak–gene pairs as proximal (genomic distance < 10kb) or distal ( ≥ 100 kb).

Revised: We then evaluated different architectures of GrID-Net that vary in the number of GNN layers L , hypothesizing that architectures with higher L (i.e., allowing greater pseudotime difference between peaks and genes) may detect more distal peak–gene pairs.

Revised: Inspecting the top 1% of peak–gene pairs detected by different architectures, we found this to indeed be the case: across all datasets, architectures with higher L detected larger ratios of distal-to-proximal peak–gene pairs (Tables 16, 17), suggesting that distal peak–gene interactions may be marked by longer pseudotime difference between peak and genes.

Revised: This ﬁnding suggests the possibility of a previously unreported link between genomic distance and the temporal lag between a regulatory element’s accessibility and its target gene’s expression.

Revised: However, like other Granger causal analyses, our approach also can not account for hidden confounders; this is especially pertinent here since the greater pseudotemporal lag between distal peaks and genes may be mediated by other regulatory mechanisms rather than being a direct inﬂuence.

Revised: One hypothesis is that the complex coordination of regulatory factors required to enable distal regulatory interactions is associated with greater lags between distal regulatory elements’ accessibility and the gene expression changes they affect (Panigrahi & O’Malley, 2021).

Original:  GrID-Net’s runtime and memory usage are dependent on a variety of factors, most notably the number of nodes in the DAG describing the system of interest.
Revised: [Table] A. R UNTIME AND MEMORY USAGE GrID-Net’s runtime and memory usage are dependent on a variety of factors, most notably the number of nodes in the DAG describing the system of interest.

Original: In our model, the DAG is represented by the N × N adjacency matrices A and A + , where n is the number of nodes in the DAG (i.e. number of cells in a multimodal single-cell dataset).
Revised: In our model, the DAG is represented by the N × N adjacency matrices A and A + , where n is the number of nodes in the DAG (i.e. number of cells in a multimodal single-cell dataset).

Original: These matrices serve as the primary bottlenecks for both runtime and memory usage.
Revised: These matrices serve as the primary bottlenecks for both runtime and memory usage.

Original: Memory usage scales quadratically with n , while calculations involving A or A + can be parallelized on a GPU to reduce the runtime.
Revised: Memory usage scales quadratically with n , while calculations involving A or A + can be parallelized on a GPU to reduce the runtime.

Revised: For very large singlecell datasets, we recommend the use of sketching techniques (Hie et al., 2019; DeMeo & Berger, 2020) so that the matrices will ﬁt into GPU memory.

Original: In addition, training time scales linearly with the number of candidate peak-gene pairs.
Revised: In addition, training time scales linearly with the number of candidate peak-gene pairs.

Original: In our results above, we evaluated 507,408, 916,939, and 872,039 candidate peak–gene pairs for the sci-CAR, SHARE-seq, and SNARE-seq datasets, respectively.
Revised: In our results above, we evaluated 507,408, 916,939, and 872,039 candidate peak–gene pairs for the sci-CAR, SHARE-seq, and SNARE-seq datasets, respectively.

Original: For these same datasets, we considered systems of 3,260, 5,000, and 5, cells.
Revised: For these same datasets, we considered systems of 3,260, 5,000, and 5,081 cells.

Original: The average total runtime for GrID-Net was 1.8 hours, 5.5 hours, and 5.3 hours, while peak memory usage was approximately 31 GB, 35 GB, and 39 GB for these respective datasets. 
Revised: The average total runtime for GrID-Net was 1.8 hours, 5.5 hours, and 5.3 hours, while peak memory usage was approximately 31 GB, 35 GB, and 39 GB for these respective datasets. [Table]