Learning Crossmodal Interaction Patterns via Attributed Bipartite Graphs for Single-Cell Omics
Authors: Xiaotang Wang
Xuanwei Lin
Yun Zhu
Hao Li
Yongqi Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Single-cell omics;  Crossmodal matching; Graph Transformer; Crossmodal Interaction Learning;
Primary area: machine_learning_for_sciences
TL;DR: Modeling single-cell crossmodal matching as graph classification task on attributed bipartite graphs

Abstract: Crossmodal matching in single-cell omics is essential for explaining biological regulatory mechanisms and enhancing downstream analyses. However, current single-cell crossmodal models often suffer from three limitations: sparse modality signals, underutilization of biological attributes, and insufficient modeling of regulatory interactions. These challenges hinder generalization in data-scarce settings and restrict the ability to uncover fine-grained biologically meaningful crossmodal relationships. 
Here, we present a novel framework which reformulates crossmodal matching as a graph classification task on Attributed Bipartite Graphs (ABGs).  It models single-cell ATAC-RNA data as an ABG, where each expressed ATAC and RNA is treated as a distinct node with unique IDs and biological features. To model crossmodal interaction patterns on the constructed ABG, we propose $\text{Bi}^2\text{Former}$, a **bi**ologically-driven **bi**partite graph trans**former** that learns interpretable attention over ATAC–RNA pairs. This design enables the model to effectively learn and explain biological regulatory relationships between ATAC and RNA modalities.
Extensive experiments demonstrate that $\text{Bi}^2\text{Former}$ achieves state-of-the-art performance in crossmodal matching across diverse datasets, remains robust under sparse training data, generalizes to unseen cell types and datasets, and reveals biologically meaningful regulatory patterns. 
This work pioneers an ABG-based approach for single-cell crossmodal matching, offering a powerful framework for uncovering regulatory interactions at the single-cell omics. Our code is available at: https://github.com/wangxiaotang0906/Bi2Former.

Learning Crossmodal Interaction Patterns via Attributed Bipartite Graphs for Single-Cell Omics
Xiaotang Wang
Xuanwei Lin Fuzhou University 1xw ambef ambefoxmail....mmmmmmmmmmmmm com
The Hong Kong University of Science and Technology (Guangzhou) xwang285@connect. www.cconnnnncctc.ttt..ooo. www.h-uttte cetist-gz.edu..o.. edu. cn
Yun Zhu Shanghai Artificial Intelligence Laboratory zhuyun@pjlab war..a.......................... en
Hao Li Academy of Military Medical Sciences teaa__thu113... com
Yonggi Zhang" The Hong Kong University of Science and Technology (Guangzhou) yonggizhangehkuustee edu cn
Abstract
Crossmodal matching in single-cell omics is essential for explaining biological regulatory mechanisms and enhancing downstream analyses. However, current single-cell crossmodal models often suffer from three limitations: sparse modality signals, underutilization of biological attributes, and insufficient modeling of regu- latory interactions. These challenges hinder generalization in data-scarce settings and restrict the ability to uncover fine- one-grained biologically meaningful crossmodal relationships. Here, we present a novel framework which reformulates crossmodal matching as a graph classification task on Attributed Bipartite Graphs (ABGs). It models single-cell ATAC-RNA data an ABG, where each expressed ATAC and RNA is treated as distinct node with unique IDs and biological features. To model crossmodal interaction patterns on the constructed ABG, we propose Bi² Former, a biologically-driven bipartite graph transformer that learns interpretable attention over ATAČ-RNA pairs. This design enables the model to effectively learn and explain biological regulatory relationships between ATAC and RNA modalities. Extensive experiments demonstrate that Former achieves ffite-of-the--ee art per- formance crossmodal matching across diverse datasets, remains robust under sparse training data, generalizes unseen cell types and datasets, and reveals biologically meaningful regulatory patterns. This work pioneers an ABG-based approach for single-cell crossmodal matching, offering powerful framework for uncovering regulatory interactions at the single-cell omics. Our code is available http:://gthhbb..omm///aggaaaongg
Introduction
a fundamental task in fields such as vision-language retrieval [11. protein-description matching 12] and drug-target [33 matching. where the goal is to determine whether two modality-specific inputs correspond to the same semantic entity. This task facilitates the learning of latent interaction patterns across different data domains. In single-cell omics study, each cell is profiled with multiple modalities, such as chromatin accessibility (scATAC-seq (10) and gene expression (scRNA-seg (30). These modalities are inherently correlated, resulting an intrinsic need for revealing the crossmodal interactions between them. Crossmodal matching in this context can help identify whether pair of ATAC and RNA profiles originates from the same cell. This
Crossmodal matching 41 40
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Nodalization Biologioa!
Attributes
Knowfedge
2Roz E003
MRRNA- die
Encoder Decoler
Unpairdd Cells
|
If mabchedf
Unpaied
Calls
0
Attritared Dipantite Goophh
6(V. x.A)x (A.3)
acUUAC
Sparse modality signals
(ui) Insulticient modeling
(i)) Underutilicaaion
crossmodall interaction
htological Teatures
Addrese
Challenge (ii)
Challlage (iti)
(a) Existing Challenges.
(b) Attributed Bipartite Graph Construction.
Figure 1: Comparison between (a) previous VAE- based crossmodal learning pipelines and (b) our ABG-based approach.
task offers unique opportunities uncover regulatory mechanisms in single-cell omics, the correct matches reflect the interaction patterns governed by underlying biology.
Existing crossmodal learning frameworks for single-cell omics, such Cobolt D4, CLUE (35,, MultiVI 31, and MIDAS [T8], primarily follow a Variational Autoencoder 24 (VAE)-based archi- tecture to perform modality alignment, data denoising, and shared latent representation learning. As shown in Figure Ial while such methods have demonstrated efficacy in downstream tasks, they exhibit three critical limitations: (i) Sparse modality signals: the expression vectors of RNA and ATAC are sparse; thus, modeling them with dense vectors will introduce noise from unexpressed signals 2 (ii) Underutilization of biological features: each RNA and ATAC signal carries rich biological attributes (e.g., statistical summaries, genomic annotations, and DNA sequences), which are effectively incorporated by existing methods. (iii) Insufficient modeling of crossmodal interaction: most current methods do not directly model the interaction between expressed ATAC and RNA signals, with limitations in understanding the underlying regulation. Moreover, most methods require paired multi-omics profiles for training, yet such data are costly and scarce. Thus leveraging limited paired data achieve robust generalization therefore remains a major challenge.
To address these challenges, we introduce a graph-based perspective built upon the concept of Attributed Bipartite Graphs (ABGs). ABGs naturally represent distinet node types with rich attributes and sparse interactions, making them well-suited for crossmodal biological data. This paradigm allows for explicit representation of interactions while leveraging both observed data and node-level features. Motivated by these strengths, we reformulate crossmodal matching in single-cell omics interaction learning problem via graph classification task on ABGs. In our setting, each expressed RNA and accessible ATAC peak is treated as a node with a unique ID and rich biological attributes, directly addressing Challenges (i) and (ii). Building the constructed graph, propose Former, a biologically-drive bipartite graph transformer that learns interpretable attention over potential ATAC-RM CNNA regulatory pairs, explicitly modeling crossmodal interactions (addressing Challenge (ii)). Extensive experiments show that FirForeer not only outperforms existing state-of-the-an methods in crossmodal matching accuracy across diverse datasets, but also demonstrates strong robustness under sparse training data and transfer capability across unseen cell types. Furthermore, it exhibits superior interpretability by revealing biologically meaningful regulatory patterns at both the cell-level and the cell-type-level.
Our contributions can be summarized as follows:
To model the crossmodal data single-cell omies which is sparse with rich attributes and important interactions, we provide a novel framework in modeling the expressed ATAC and RNA as nodes and their interactions with ABG. This establishes valuable training corpus for single-cell omics and advances the crossmodal matching learning in this field.

---

We propose Firmor, a biologically-dmin driven crossmodal graph transformer that integrates biologically-driver crossmodal attention module with a message- architecture. This design enables the model to effectively learn and explain regulatory relationships between ATAC and RNA modalities.
Through extensive experiments, Former achieves state-of-ther tart performance on crossmodal matching, including robustness under sparse training data and transfer capability across unseen cell types. Additionally, it provides superior interpretability by uncovering biologically meaningful ATAC-RNA interactions.
2 Related Work
2.1 Crossmodal Matching
Crossmodal matching aims to determine whether two modality-specific views correspond to the same underlying entity. It is a fundamental task in many domains, including vision-language retrieval [3] and audio-visual matching [9] In the biomedical domain, crossmodal matching has been applied
to problems such as croteinddescription matching [12), drug-target target matching [13), and medical image-report retrieval [46]. These applications demonstrate the potential of crossmodal matching to uncover meaningful interactions across modalities. Typical approaches adopt dual encoders or
cross-attention mechanisms to learn aligned representations across modalities.
In single-cell omics studies 37 crossmodal matching, together with crossmodal generation and joint embedding, constitutes the core tasks of the field [27]. For example, CLUE (35 introduces
the use of cross-encoders to construct latent representations from modality-incomplete observations.
Cobolt [14] uses a shared encoder-decoder architecture to integrate multiple modalities into a unified low-dimensional latent space. Multi VI [3] extends the variational autoencoder framework to jointly model RNA and ATAC distributions through modality-specific encoders.
In this work, we explicitly focus on crossmodal matching problem as proxy to learning interaction patterns. This formulation provides: (i) an efficient and lightweight supervision signal derived from naturally occurring cells, and (ii) aligns well with biological intuition that ATAC and RNA profiles from the same cell should reflect true regulatory interactions.
2.2 Learning with Attributed Bipartite Graphs
Attributed bipartite graphs (ABGs) model two distinct node types with heterogeneous features and sparse interactions, offering a powerful abstraction for many real-world problems. In recommendation systems [17]. users and items are modeled as nodes in bipartite graphs, where interactions learned through collaborative filtering 45 or Graph Neural Networks 36). In fraud detection, ABGs have been used to represent transactional patterns between customers and merchants, capturing anomalous links through attribute-awar substructures B3 Beyond these domains, ABGs have gained attention in biological settings for modeling drug-target or gene-disease associations
[29] Their strength lies
in combining structural signals from interactions with rich semantic content at the node level.
In this work, we adopt the ABG formalism to model expressed RNA and ATAC nodes within single cell. This allows us to filter out noise from unexpressed signals during graph construction and fully leverage the rich biological features associated with each expressed signals. In addition, the fine-grained regulatory dependencies can be captured by the proposed attention mechanism guided by biological priors over the two modalities. It is worth noting that although GLUE (6) and ScMoGNN [47] also employ graph structures, they primarily rely on prior knowledge (e.g., predefined guidance graphs) to facilitate multi-omics integration. By contrast, Former learns and reveals regulatory knowledge rather than depending on such priors.
3
Graph Construction
3.1 Problem Definition
Given a single cell C, we observe two modality-specific inputs: an RNA expression vector XRNA E B Nasa and ATAC accessibility vector XATAC [8 NaMse, where each element corresponds to the
3

---

expression of gene or chromatin region. Both vectors are digh-dimensional, sparse, and enriched with fomain-speciffi annotations recorded in metadata matrices HRNA (for RNA features) and HATAC (for ATAC. features). The objective of crossmodal matching is to predict whether given pair (XRNA XATAC) originates from the same biological cell. Instead of reporting soft probability scores 135 47 we compute hard accuracy based on each pair's binary prediction. This design emphasizes precise matching signals, which are particularly important for learning fine-grained interaction patterns across modalities. A label y E {0, 1} is assigned to each pair, where y 1 denotes a matched pair from the same cell, and y = denotes mismatched pair from different cells. The matched pairs given by signals detected in the same single cell by biological experiments, while the mismatched pairs are generated by negative sampling methods. To preserve biological diversity and avoid sampling bias, negative samples are drawn proportionally according to the distribution of cell types in the dataset, and the number of positive and negative samples maintained 1:1 ratio. To address the aforementioned challenges, transform each RNA-ATAC pair into an ABG and formulate the crossoodal matching problem as graph classification problem.
3.2 Graph Construction: From Modality Expression to Attributed Bipartite Graphs
Given paired single-cell expression vectors XRNa and XTACC we construct a bipartite graph G (V,X, A), where the node set represents the expressed features in each modality, the attribute set X contains the attributes of corresponding nodes, and the bipartite adjacency matrix A represents the interaction between RNA nodes and ATAC nodes.
Nodalizing Expressed Multimodal Signals into Bipartite Node Set. The node set V consists of two disjoint subsets: RNA nodes and ATAC nodes. Each RNA node corresponds to expressed gene with non-zero value in XRNA, and each ATAC node corresponds to a chromatin region with non-zero value in XAIAC- We denote them
VRnaa (RNA,, XRNA (m] + 0}, VATAC {ATAC,, XATAC [n] + 0},
(1)
thus, the full node set of the bipartite graph is: V = VRNA U VATAC
Embedding Biological Attributes into Node Features. Each node v E V is associated with biologically-ifoomae feature vector. The overall node features are:
X {XRNA E R Vexul xd, XAIAC R VATeel },
where each node feature constructed as a concatenation:
X² = Concat (Oneet(II())) (ID(v), Expr(v), BioAttr(u)).
(2)
where ID being unique identity per RNA/ATAC, Expr denotes the expression level of the node in the current cell, ie., the value from XRNA [m] or XATAc[u). and BioAttr encodes the biological metadata retrieved from HRNA or HATAC such as chromosomal location, expression statistics, or DNA sequence encodings.
Edge Design with Biological Prior Knowledge. Edges in the constructed bipartite graph reflect potential relationships between RNA and ATAC. For edge design, naive approach is connect all nodes in one modality with nodes in another. However, such full connectivity introduces substantial noise because regulatory interactions are often constrained to nearby genomic loci. Alternatively, define the adjacency matrix E {0, 1] |eess VArarl based on the biological prior knowledges, such as chromosomal co-location. Here, we introduce a chromosomal mask as adjacency matrix to constrain attention computation within chromosomally plausible regions. These prior-inforr inforiifrrmed connections improve inductive bias and reduce noise from fully associations.
The Constructed Attributed Bipartite Graph Formalism. Overall, each RNA-ATAC pair is encoded as an attributed bipartite graph G = (V, X, A), where nodes represent expressed RNAs and ATACS with biological features, and edges reflect regulatory potential under biological priors. Thus, we reformulate the crossmodal matching problem as a binary graph classification task, where the model f G + e {0, } is trained to predict if an ABG represents matched RNA-ATAC vector.

---

XRNA
XATAC
Biological
Pruning
Q
Qerress
Keys
Bi FirFormer
Values
Values
M Message
M essage
AATAC
RNA
Passing
Passing
Cell-level Explainability
Cell- Cell-type- Cell-type-level
TF Binding WWW
RNA
DE
Explainability
Matching Results
ATAC
peak
Model structure of Bi Forme. It consists biologically-drive crossmodal attention module and crossmodal message passing architecture. Moreover, the biological pruning strategy consists of a thresholding and a top-k filtering.
Figure 2:
4 Bi Former: An ABG-based Interaction Learner
To model ATAC-RNA regulatory interactions from the ABG perspective, we propose Bi Former, biologically-drine bipartite graph transformer that integrates biologically-dnnn driven crossmodal attention module with message passing architecture. Given a bipartite graph sample G = (V,X, A), our model processes node features and topology two key stages:
Biologically-driver Crossmodal Attention. To explicitly model the regulatory interactions be- tween RNA and ATAC, introduce biologically-driven bipartitite attention mechanism. Specifi- cally, we compute:
(3) where K,K Xdd query and key matrices, respectively, Wo, Wv. Rd,xd, and WK. Wv, Rdda are learnable weight matrices.
Q = XRNAWQ: K= XXXAC VINA = XRNA Wv. VAIAC = -XAtAW
In this module, we regard the information of RNA as queries Q and ATAC as keys K based on the biological intuition that ATAC regions regulate RNA expression. Different from general Transformer B8 modules, we compute two separate value matrices VRNA and VATAC for the two modalities such that the modality-sspciin information can be maintained. These designs can better adapt to the bipartite information in the two modalities and enable the model to explicitly learn crossmodal interaction patterns between RNA and ATAC.
As for attention values, we first compute raw crossmodal attention scores between RNA node and ATAC node a using scaled dot-product over the edges defined by the adjacency matrix A:
Q.KI Vdn
if A = V(r, E VRNA X VATAC-
Or
(4)
0,
otherwise
To mitigate noise from under-trained nodes and suppress uniform attention distributions, we employ biological pruning strategy. The attention scores are first passed through sigmoid activation and thresholded by a hyperparameter 7 to climinate low-confidence signals:
à Threshold(o(a), E Vexal x x|Vascl
Subsequently, we retain only the top-k attention scores aligned with each RNA node and binarize the scores, yielding sparse binary attention mask ā E (0, [ |eea This constraint aligns with the biological truth that each gene is typically regulated by limited number of ATAC peaks [15] The resulting binary attention matrix serves both as an interpretable crossmodal regulatory map (see details in Appendix C and structured graph to guide the subsequent message passing stage.
5

---

Crossmodal Message Passing. To generate enriched node representations that incorporate both intra- and inter-modal information, we design crossmodal message passing module guided by the binary attention matrix d. This module enables RNA nodes aggregate regulatory cues from attended ATAC nodes, while ATAC nodes receive feedback from their associated RNA targets via the transposed attention map. For each RNA node r and ATAC node a, we aggregate information from their relevant nodes:
X C
[r)
Σ VEVAcc ār.a Vxiacla];
eerosss ATAC [a] =
KIAC|a]
E revas ār,a VRNA [r],
(6)
To preserve node-specific intrinsic semantics, incorporate modality-speciffe self-update module:
XRNAPE [r] MLPRNA (XRRNA [r]) RNN [r], XATAC[a] MLP ATAC (AAxccXa)))))) XTAC [a] (7)
where M.)(.) denotes lightweight feedforward network that captures intra-modal patterns. The result a set of updated node embeddings containing both intra-modal and inter-modal knowledge.
Model Training. We adopt a graph-level binary classification objectíve, where the label {0, 1} indicates whether the RNA and ATAC graphs originate from the same cell. To obtain compact graph-level representation, we apply average pooling over the final-layer embeddings of the RNA and ATAC nodes, followed by a concatenation and a multi-layer prediction head:
i = Predictor [AvgPool (XRVa)
AvgPool
(NxIAe):
(8)
where XRNA and X ATAC denote the updated embeddings for all expressed RNA and ATAC nodes after message passing 7 AvgPool is the average pooling operator over rows, || denotes vector concatenation, and Predi is a 4-layer feedforward network with ReLU activations.
The model is trained end-to end-to-end by minimizing the binary cross-entropy loss RNA-ATAC pairs:
L y log (1 v) log (1 - 0)
using the Adam optimizer patterns that are predictive of cell identity alignment. The complexity analysis of Former shown in Appendix D and the limitations are analyzed in Appendix
23 This objective encourages the model to learn crossmodal regulatory
5 Experiments
In this section, we first introduce the datasets and the baselines in Section 5.1 Then we conduct extensive experiments to address the following research questions: RQ1: How well does model perform on the crossmodal matching task? RQ2: How robust is our model under limited paired training data? RQ3: How effective is the proposed method on the transfer capability across unseen cell types? RQ4: What is the contribution of each core component of our model to overall performance? RQ5: How do different hyperparameter settings affect model performance? RQ6: How can we use our framework for biological interpretation and discovery?
5.1 Experimental Setup
Datasets. To ensure the reliability and comparability of our evaluation, conduct experiments on five widely-used benchmark datasets for single-cell omics: ISSAAC-seq (50]. 10x Multiome PBMC [1]. SHARE-seq (28). SNARE-seq [8] and 10x genomics Multiome. We construct our graph
corpus following the description in Section 3 Details of the original datasets and the constructed
graphs are in Appendix [
Baselines. We compare Bi Former against two categories of baselines: (i) VAE-based models, including MultiVI B] CLUE (35). Cobolt [4, GLUE (6) scMoGNN [47] and scMaui 22 which are originally designed for joint embedding or modality reconstruction. We adapt these models by appending classification heads for the crossmodal matching task. (ii) Methods based on our constructed graph (i.e., ABG) corpus, including simple MLP, as well as advanced Graph Neural Networks (GNNs) such as GCNII (7), GraphSAGE [16], and Graph Transformer (GT) serving strong baselines.

---

Table 1: Crossmodal matching results across various datasets. We report both accuracy and ROC-AUC as evaluation metrics. Boldface indicates the best performance.
ISSAAC-9q
10PPMMC
SHARE--e
SNARE-10
Arg:
Menti
ACC
RDC
AUC
ACC
ROC AUC
ACC
ROC
AUC
ACC
ROC
ALC
ADC
ROC AUC
ACC
ALC
MutuVt
06.21
69.32
2.83
63.55
2.12
68.87
1.00
56.76 1.94
69.35
12
CLUB
1.22
75.01
a 0.95
65.73 a 1.67
12.26
63221
67.96 1.19
29.32
0.95
76.92
1.17
67.25
71.06
Caboli
73.69 2..1
61.65
66.74
586773314
61.74
60.1
71.61:
14.15
63.63 67.45
GELIE
77:40 0.9
72,51
19 0.71
66.9
143
1.11
64.47
1,20 68.2
1211
76.91
0.8
50.98
0.92
71.01 75:00
S:MGGNN
03.92
78.58
4 0.89
72:41 + 1.37
0.83
09.30 1.81
0.94
09.0
1.22
75,89 1.31
20.00 1.01
72.10
77.22
ScMaul
76.19 0:83
63.19
6.54
65.95
5 03.54:00555
75.07
7..5
66.35
1.19
71.04
0.79
61.25
55.87
58.97
0.74 62.53
$4.74
99.72
7044+
62.76
GCNI
1..6
2 0.63
0. 2
69
1.3 14.01 0.66
8271 1.00
0.59
76.28
0.76
70.95 75,98
GraPhSAGE
0..5
82.37
4 0.35
76.92 +: 1.52
0.63
67.56:1224
70.90
70.56:00:77
81.94 1.89
55.79 = 1.44
73.99 78.23
0.52
90,03 0.31
0.794
82406:0.47
0.517:0:53
0.334
0.67
BiFFomer
$4.40
0.31
2.33
0.36
0.29
84.96 0.15
0.33
00.30:
93.41 0.30
$3.39 57.46
Other settings. We report experimental results using hyperparameter settings detailed in Ap- pendix [..4] selecting those that achieve the highest validation performance. While our hyperparame- ter grids may not always be optimal, they cover broad range to ensure each model is adequately evaluated on every dataset. Each experiment is repeated with 10 different random seeds, and we report the mean and standard deviation across these runs.
5.2 Modal Matching (RQ1)
We evaluate the performance of Former on the task of crossmodal matching, where the goal is to determine whether a pair of ATAC and RNA sequences originates from the same cell. Table summarizes the results across four benchmark datasets.
First, VAE-based models (e.g.. MultiVI, CLUE, Cobolt, GLUE) perform poorly due to their reliance on sparse expression vectors and underutilization of biological attributes. Modeling both expressed and unexpressed elements introduces noise and weakens the meaningful regulatory signals. Second, MLP trained on our constructed ABG corpus benefits from denoised inputs and biological attributes, achieving modest results. However, its lack of structural and interaction- aawaee modeling limits its performance. Third, GNN models (i.e., GCNII, GraphSAGE, GT) further improve performance by leveraging structural information. However, their inductive biases are typically locality-dris -rivve and lack explicit mechanisms to model biologically meaningful crossmodal interactions.
Firmor addresses these limitations by a biologically -drivee crossmodal attention mechanism that filters low-confidence signals, incorporates chromosomal priors, and aligns with the biological truth. Furthermore, our crossmodal message passing preserves intra-modal semantics while capturing inter- modal dependencies, enabling Former to capture fine- sree-grrined interaction patterns that general GNNs cannot model explicitly. As result, Bi Former consistently outperforms baselines, surpassing the strongest VAE baseline by an average of 1 1.3% and the strongest ABG-based baseline by 8.9% in accuracy, with the largest improvement of 16.2% and 10.7% on 10XPBMC. These gains underscore the strength of our biologically grounded framework and its generalizability across diverse datasets.
5.3 Robustness under Sparse Supervision (RQ2)
To evaluate the ability to effectively handle sparsely paired datasets, which is significant challenge in biological measurement data, we conducted experiments comparing our model with other baselines under different training set sizes. As shown in Figure 3 VAE-based methods experience a notable performance drop when the amount of paired training data is reduced. In particular, when only 20% of the training pairs are available, most baselines lose the ability to effectively distinguish positive from negative samples. By contrast, FirFormer maintains strong performance even under such low-resourgs settings. This robustness arises from its biological attribute-awan design and its objective that explicitly captures fine-grained crossmodal interaction patterns, enabling efficient utilization of limited training data.
7

---

ISSAAC-SeU
10x PEMVC
Aocuaic
85
73
63
53
80%
40%
21%
10%
3%
40%
10%
MulbVI
CLUE
Cobolt
GLUB
ScMoGNN
stMaui
Former
Figure 3: Results for crossmodal matching task with different training sizes.
The PRANC CIEZ
Table 2: celose-ss prediction results for crossmodal matching across various datasets. We report accuracy evaluation metric. Boldface indi- cates the best performance.
0.9
A1 AL
A3
0.8
Danaset
18SAAC-4a
10xPBMC
SHARE-e0
SNARE-i0
MalaVI
47.32 1.39
54.72 168
52.81 1.51
2.18
CLUE
61.28 1.16
61.41 1.31
57.31 1.32
56.45 1.82
Cobolt
2.08
58.67
1.96
2.44
48.63
2.511
scMaui
62.43 1.84
58.54 1.79
52.49 1.74
Avg. Cosine Similrity
Avg. Positive Match Rate
MLP
65.72 1.13
61,84 1.69
58.74 1.12
53.67 1.33
Figure 4: mross- similarity of attention matrices. The metrics are computed between attention matrices of the same held-out cells from models with different train corpus.
GCNI
72.18:1 1.29
0.78
67.37 0.96
61.38 1.27
GraphSAGE
0.93
7 0.65
0.65
67.02 a 1.31
64.56 0.92
GT
71.92
76,43 a 0.81
69.94 0.95
67.85 0.81
Bi Booome
0.77
84.96 0.49
78.07 a 0.61
71.28 a 0.32
5.4 Cross-Cell-Type Generalization (RQ3)
Transfer Capability across Unseen Cell Types. To assess the generalization capability of our method, we evaluate the performance of Former under cross-cell-type setting. Specifically, we split each dataset into training and test sets with disjoint cell types in a 1:1 ratio (See details in Appendix B.3). This setup ensures that the model is evaluated on completely unseen biological categories. As shown Table 2 ABG-based methods significantly outperform traditional VAE- based baselines, emphasizing the benefits of denoised signals and biological attributes for improved generalization. Moreover, Bi Former consistently achieves the best performance across all settings, surpassing the strongest ABG-based baseline by average of 7.7%, highlighting the benefits of the biologically-driven design and its strong ability to generalize across cell types.
Analysis of Learned Attention Matrices across Cell Types. Beyond prediction accuracy, further investigate whether the attention learned by Bi Former captures transferable biological regulatory mechanisms across different cell types. Specifically, we train separate models on corpus different cell-type. Then we compare the attention matrices generated for the same test cells among these models. As shown in Figure 4 attention matrices generated by models trained on different groups of the same cell type (i.e,, Al and A2) remain highly consistent, with average cosine similarity of 0.93. Moreover, models trained on Al (one type of blood mononuclear cells from PBMC) and B (another type of blood mononuclear cells from PBMC) yield relatively similar regulatory patterns (0.80), whereas those trained on Al and C (one type of neuronal tissue cells from SNARE) show much lower similarity (0.63). These results suggest that Former captures cell-type-speciff regulatory interaction patterns that generalize more effectively across biologically similar populations.
5.5 Ablation Study (RQ4)
To understand the contribution of each key component in 2i² Former, we conduct an ablation study by selectively masking node ID embeddings (i.e., ID(u) in Eq. biological attributes (i.e., Expr( and BioAttr(v) in (h. biological pruming (BP) in the crossmodal attention module, and the edge structure informed by prior biological knowledge in the graph corpus.
Results are shown Table 3 performance drop than removing biological attributes. This suggests that with sufficient training
Interestingly, removing ID embeddings leads more significant

---

Table 3: Ablation study of masking different components in Former.
Methods
ISSAAC-seq
10XPBMC
SHARE-seq
SNARE-seq
Avg.A
BorFor
84,40 = 0.48
88.74 0.36
79.84 = 0.29
73.56 0.37
w/o
80.64 + 1.51
76.75 8300
1.87
70.97 1.01
3.78
w/o Attribute
81.47 # 0.49
0.38
7,474
2.93
w/o BP
83.87 = 0.64
0.46
77.97
0.36
1.43
w/o Edge
78.89 = 0.57
83.68 0.49
76.72 0.28
70315
4.28
w/o Attribute, BP, and Edge
72.39 + 1.12
70.86 1.74
69.78 + 2.14
67.45 1.13
11.52
data, the model is able to effectively learn the interaction patterns between nodes through repeated leoosurr-levergging latent co-occurrence signals across graphs, highlighting the model's ability to infer relationships in data-driven manner under fully supervised conditions.
We next remove the biological pruning, i.e., the sigmoid-threshold activation and top-k mask, which is designed to suppress low-confidence signals and reflect the biological constraint that genes are typically regulated by a limited number of ATAC peaks. This results modest performance drop, indicating that our attention sparsification pruning is able to align model with biological priors.
Furthermore, we evaluate the impact of removing the edges by replacing the prior-based adjaceney matrix with a fully connected bipartite graph between expressed ATAC and RNA nodes. This design removes biological priors and allows unrestricted attention computation across all node pairs. The performance degrades but remains competitive compared to the VAE-based baselines. Overall, these confirm two points: (i) our attention mechanism is capable of learning useful interactions even under noisy topologies, and (ii) biologically grounded edge priors serve as an effective inductive bias, guiding the model toward more interpretable and accurate regulatory patterns.
Finally, we remove the attributes, biological pruning, and edges, retaining only the expressed nodes. This modification results in the model only filtering out unexpressed nodes compared to traditional VAE- based models. We observed significant performance drop, but the model still outperformed the strongest VAE-based baseline. This further emphasizes the effectiveness of filtering out noise and focusing solely on the co-occurrence patterss on expressed nodes.
5.6 Hyperparameter Study (RQ5)
We investigate the influence of key hyperparameters in Firmoreer, focusing on the threshold strategy and the top-k selection strategy within the Biologically-drive Crossmodal Attention module.
Threshold т. The threshold parameter 7 is introduced filter out low-confidence attention signals. This design is motivated by our early observations that, during the initial training stages, under- trained nodes tend to produce uniformly distributed attention weights, which introduces considerable noise and deviates from biologically meaningful regulatory patterns. To address this, we first set the threshold r = 0.5, which effectively suppresses these uniformly noisy distributions. We then gradually increased the threshold to assess its impact on model performance. As shown in Figure 5 the optimal threshold varies slightly across different datasets, likely reflecting biological differences in RNA-ATAC interaction sparsity across distinct cell types. Higher thresholds enforee stricter gating, allowing only the most confident and specific regulatory links to be preserved.
Top-k. The top-k sparsification strategy following thresholding is introduced to further align with the biological assumption that each gene is regulated by a limited number of cis-regulatory elements. We experiment with & (5, 10, 15, 20). and the results shown in Figure Sindicate that k = 10 performs the best across most datasets. This finding supports the notion that a small number of ATAC regions contribute significantly to RNA regulation, aligning well with known biological priors.
5.7 Biological Interpretation and Discovery (RQ6)
Cell-level. In Figure 4 we evaluate the plausibility of the learned attention matrix from compu- tational perspective. To further interpret Former from a biological standpoint, we leverage the learned RNA-ATAC attention matrix á as proxy for regulatory interactions. At the cell-level, ä reveals how accessible ATAC potentially regulate gene expression (RNA), enabling fine-graines

---

Accuray (%)
76
72
0.5
0.7
08
0.9
0.95
Threshold r
Top k
ISS
10XPBMC
SHARE-seq
SANRE-seq
Figure 5: Experiment results of the hyperparameter of Formor.r.
cell-specific interpretation. As shown in Appendix C.I each RNA is regulated by limited number
of ATAC peaks, consistent with known biological principles of gene regulation.
Cell-type-level. By aggregating attention matrices across cells of the same type, we obtain population- level regulatory maps that reflect cell-iype-specifie transcriptional programs. These insights facilitate the comparative analysis of regulatory patterns across cell types. Moreover, to further validate the biological relevance of our model, we compare the cumulative attention scores against experimentally derived TF binding scores [21]. which reflect the actual activation strength of ATAC peaks in each cell type and serve as a proxy for the ground truth. The results show that the cumulative attention scores for CD4 cells exhibit strong agreement with CD4-specific TF binding signals, indicating that model successfully identifies biologically meaningful regulatory relationships. Details are provided in Appendix C.2
6 Conclusion
In this work, we present novel framework that formulates single-cell crossmodal matching as an interaction learning problem via graph classification task on Attributed Bipartite Graphs (ABGs). Our study introduces an interpretable ABG-basee based approach to single-cell crossmodal analysis, paving the way for more structured and insightful crossmodal learning biology. This perspective allows for explicit modeling of interaction while leveraging both observed data and node-level features. To model the regulatory interactions on these graphs, we propose Former, a biologically-dneg bipartite graph transformer that learns interpretable attention over potential ATAC-RNA regulatory pairs, explicitly modeling crossmodal interactions. Extensive experiments across diverse datasets show that our model achieves ofeee----he-rrr performance in crossmodal matching, generalizes well to unseen cell types. and uncovers biologically meaningful regulatory interactions. Our study introduces an interpretable ABG-based approach single-cell crossmodal analysis, paving the way for more structured and insightful crossmodal learning in biology.
7 Limitations
While 2i2 Former achieves strong performance and interpretability, it does not currently incorporate rich edge attributes, which can play crucial role in capturing fine-grained interactions in graph- based tasks [19]. Modeling such edge-level information requires the integration of more detailed biological priors and suitable encoding strategies. In future work, plan to extend our framework by introducing biologically informed edge attributes to fully exploit their representational power.
Acknowledgements
This work supported by Guangdong Basic and Applied Basic Research Foundation project 150010344,, Guangzhou Science and Technology Planning Project 2025A03J4491, National Key Research and Development Program of China 330740 H.L.).
10

---

References
[1] 10x Genomics. 2020.
PBMC from healthy donor, single cell multiome ATAC gene expres- https////upport.1xxeeeee sion demonstration data by Cell Ranger ARC 1.0.0. www.grraaoocee.ortteeeeee.oooede 10k single- citimmuliioonnoo tome- atac- wew/datasets/1.0t
com/
[] Ricard Argelaguet, Anna SE Cuomo, Oliver Stegle, and John C Marioni. 2021. Computational principles and challenges in single-cell data integration. Nature biotechnology 39, 10 (2021), 1202-1215
[3] Tal Ashuach, Mariano Gabitto, Rohan V Koodli, Giuseppe-Antone Saldi, Michael Jordan, and Nir Yosef. 2023. MultiVI: deep generative model for the integration of multimodal data. Nature Methods 20, 8 (2023), 1222-1231
[4] Alev Baysoy, Zhiliang Bai, Rahul Satija, and Rong Fan. 2023. The technological landscape and applications of single-cell multi-omics, Nature reviews. Molecular cell biology 24, 10 (October 2023), -73.
[5] Pietro Bongini, Niccolò Pancino, Franco Scarselli, and Monica Bianchini. 2022. BioGNN: how graph neural networks can solve biological problems. In Artificial Intelligence and Machine Learning for Healthcare: Vol. I: Image and Data Analytics. Springer, 211-231.
[6] Zhi-Jie Cao and Ge Gao. 2022. Multi-omies integration and regulatory inferenee for unpaired single-cell data with a graph-linked unified embedding framework. Nature biotechnology (2022).
[7] Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding. and Yaliang Li. 2020. Simple and deep graph convolutional networks. In International conference on machine learning. PMLR, 1725-1735.
[8] Shu Chen, Blue B Lake, and Kun Zhang. 2019. Joint profiling of chromatin accessibility and gene expression in thousands single cells. Science 361, 6409 (2019), 1380-1 1385.
[9] Soo- Cho-Whan Chung, Joon Son Chung, and Hong-Goo Kang. 2019. Perfect match: Improved modal embeddings for audio-visus synchronisation. ICASSP 2019-2019 IEEE International Conference on Acousticx, Speech and Signal Processing (ICASSP). JEEE, 3965-3969.
[10] Darren A. Cusanovich, Riza Daza, Andrew Adey, Hannah A. Pliner, Lena Christiansen, Kevin L.. Gunder- son, Frank J. Steemers, Cole Trapnell, and Jay Shendure. 2015. Multiplex single-cell profiling of chromatin accessibility by combinatorial cellular indexing. Science 348, 6237 (2015), 910-914
[11] Vijay Prakash Dwivedi and Xavier Bresson. 2020. A generalization of transformer networks to graphs.
[12] Noelia Ferruz, Steffen Schmidt, and Birte Höcker. 2022. ProtGPT2 is a deep unsupervised language model for protein design. Nature communications 13, (2022), 4348.
[13] Bowen Gao, Bo Qiang, Haichuan Tan, Yinjun Jia, Minsi Ren, Minsi Lu, Jingjing Liu, Wei- Yei-Ying Ma, and Yanyan Lan, 2023. Drugelip: Contrastive protein moleccl representation learning for virtual screening. Advances in Neural Information Processing Systems 36, 44595 44614.
[14] Boying Gong, Yun Zhou, and Elizabeth Purdom. 2021. Cobolt: integrative analysis of multimodal single-cell sequencing data. Genome biology 22 (2021), 1-21.
[15] Fiorella C Grandi, Hailey Modi, Lucas Kampman, and M Ryan Corces. 2022. Chromatin accessibility profiling by ATAC-seq. Nature protocols 17, 6 (2022),
[16] Will Hamilton, Zhitao Ying, and Jure Leskovee. 2017. Inductive representation learning large graphs Advances in neural information processing systems 30.
[17] Xiangnan He, Lizi I Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collahorative filtering.. 173-182 pages.
[18] Zhen He, Shuofeng Hu, Yaowen Chen, Sijing An, Jiahao Zhou, Runyan Liu, Junfeng Shi, Jing Wang, Guohua Dong, Jinhui Shi, Jiaxin Zhao, Le Ou- Yang, Yuan Zhu, Xiaochen Bo, and Xiaomin Ying. 2024. Mosaic integration and knowledge transfer of single-cell multimodal data with MIDAS. Nature Biotechnology 42 (01 2024), 1594-1605.
[19] Weihua Hu, Matthias Fey, Marinka Zitmik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovee. 2020. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems 33, 221 111188221 18-32133.
[20] Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. 2020. Heterogeneous graph transformer. 2704-2710 pages.

---

[21] Sachi Inukai, Kian Hong Kock, and Martha L Bulyk. 2017. Transcription factor-DNA binding: beyond binding site motifs. Current opinion in genetics & development 43 (2017), 110-119.
[22] Yunhee Jeong, Jonathan Ronen, Wolfgang Kopp, Pavlo Lutsik, and Altuna Akalin. 2024, scMaui: widely applicable deep learning framework for single-cell multiomics integration in the presence of batch effects and missing data. BMC bioinformaties 25, (2024), 257.
[23] Diederik P. Kingma and Jimmy Ba. 2017. 41Xiv:1412.69800 [cs.LG]
Adam:
Method for Stochastic Optimization.
[24] Diederik Kingma, Max Welling, et al. 2013. Auto-encoding variational bayes.
[25] Thomas N. Kipf and Max Welling. 2017. Supervisee Classification with Graph Convolutional Networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
[26] Rui Li, Xin Yuan, Mohsen Radfar, Peter Marendy, Wei Ni, Terrence J O' O'Brien, and Pablo M Casillas- Espinosa. 2021. Graph signal processing, graph neural network and graph learning on biological data: systematic review. IEEE Reviews Biomedical Engineering 16 (2021), 109-135.
[27] Malte Lücken, Daniel Burkhardt, Robrecht Cannoodt, Christopher Lance, Aditi Agrawal, Hananch Alice, Ann Chen, Louise Deconinck, Angela Detweiler, Alejandro Granados. Shelly Huynh, Laura Isacco, Yang Kim, Dominik Klein, Bony de Kumar, Sunil Kuppasani, Heiko Lickert, Aaron McGeever, Joaquin Melgarejo, Honey Mekonen, Maurizio Morri, Michaela Müller, Norma Neff, Sheryl Paul, Bastian Rieck, Kaylie Schneider, Scott Steelman, Michael Sterr, Daniel Treacy, Alexander Tong, Alexandra-Chle Villani, Guilin Wang, Jia Yan, Ce Zhang, Angela Pisco, Smita Krishnaswamy, Fabian J. Theis, and Jonathan M. Bloom. 2021. A sandbox for prediction and integration of DNA, RNA, and proteins in single cells.
[28] Siyuan Ma, Baohong Zhang, Lucas M LaFave, et al. 2020. Chromatin potential identified by shared single-cell profiling of RNA and chromatin. Cell 183, 4 (2020), 1103- 1116.200
[29] Georgios A Pavlopoulos, Panagiota I Kontou, Athanasia Pavlopoulou, Costas Bouyioukos, Evripides Markou, and Pantelis G Bagos. 2018. Bipartite graphs in systems biology and medicine: a survey of methods and applications. GigaScience 7, 4 (2018), giy014.
[30] Simone Picelli, K. Björklund. Omid Faridani. Sven Sagasser, Gosta Winberg, and Rickard Sandberg. 2013. Seart-seq2 for sensitive fell-length transcriptome profiling in single cells. Nat. Methods 10 (01 2013), 1096-1098.
[31] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. 8748 8763 pages.
[32] Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. 2022. Recipe for a general, powerful, scalable graph transformer. 14501-14515 pages. pages.
[33] Yuxiang Ren, Hao Zhu, Jiawei Zhang, Peng Dai, and Liefeng Bo. 2021. Ensemfdet: An ensemble approach to fraud detection based on bipartite graph. In 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, 2039-2044.
[34] Yongduo Sui, Xiang Wang, Jianean Wu, Min Lin, Xiangnan He, and Tat-Seng Chua. 2022. Causal attention for interpretable and generalizable graph classification. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, 1696-1705.
[35] Xinming Tu, Zhi-Jie Cao, Sara Mostafavi, Ge Gao, 2022. Cross-Linked Unified Embedding for cross- modality representation learning. Advances in Neural Information Processing Systems 35, 15942- 15955.
[36] Rianne Van Den Berg, N Kipf Thomas, and Max Welling, 2017. Graph convolutional matrix completion.
9 pages.
[37] Katy Vandereyken, Alejandro Sifrim, Bernard Thienpont, and Thierry Voct. 2023. Methods and applications for single-cell and spatial multi-omics Nature reviews. Genetics 24, 8 (August 2023), 494 515.
[38] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.
[39] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In International Conference Learning Representations.
12

---

[40] Kaiye Wang, Qiyue Yin, Wei Wang, Shu Wu, and Liang Wang. 2016. A comprehensive survey on cross-modal retrieval. arXiv preprint arXiv: 15 (2016).
[41] Tianshi Wang, Fengling Li, Lei Zhu, Jingjing Li, Zheng Zhang, and Heng Tao Shen. 2025. Cross-modal retrieval: systematic review of methods and future directions.
[42] Xiao Wang. Deyu Bo, Chuan Shi, Shaohua Fan, Yanfang Ye, and Philip S Yu. 2022. A survey on heterogeneous graph embedding: methods, techniques, applications and sources. 415-436 pages.
[43] Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference on Research and development Information Retrieval. 165-174.
[44] Xiaotang Wang, Yun Zhu, Haizhou Shi, Yongchao Liu, and Chuntao Hong- 2024. UniGAP: Universal and Adaptive Graph Upsampling Approach to Mitigate Over-Smmothing Node Classification Tasks. arXiv:2407.19422
[45] Xiaotang Wang, Zhu, Haizhou Shi Yongchao Liu, and Chuntao Hong. 2025. Graph Triple Attention Networks: A Decoupled Perspective... 12 pages.
[46] Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. 2022. Medelip: Contrastive learning from unpaired medical images and text. 3876 pages.
[47] Hongzhi Wen, Jiayuan Ding, Wei Jin, Yiqi Wang. Yuying Xie, and Jiliang Tang. 2022. Graph Neural Net- works for Multimodal Single-Cell Data Integration. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
[48] Chengxuan Ying. Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yamming Shen, and Tie- Yan Liu. 2021. Do transformers really perform badly for graph representation? 28877-28888 pages.
[49] Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. 2019. Heterogencous graph neural network. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 793-803.
[50] Lei Zhang, Zhe Sun, Yuchen Wang, et al. 2023. ISSAAC-seq enables joint analysis of chromatin accessibility and gene expression in single cells. Nature Methods 20, 1 (2023), 45 55.
[S1] Yongqi Zhang and Quamming Yao. 2022. Knowledge Graph Reasoning with Relational Digraph. In Proceedings of the ACM Web Conference 2022. ACM, 912-924.
[52] Yongqi Zhang, Quanming Yao, Ling Yue, Xian Wu, Ziheng Zhang, Zhenxi Lin, and Yefeng Zheng. 2023. Emerging drug interaction prediction enabled by a flow-based graph neural network with biomedical network. Nature Computational Science 12 (2023), 1023-1033.
[53] Yun Zhu, Haizhou Shi, Xiaotang Wang. Yongchao Liu, Yaoke Wang. Baci Peng, Chuntao Hong, and Siliang Tang. 2025. GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attribtedd Graphs (WWW '25).
[54] Yun Zhu, Yaoke Wang, Haizhou Shi, and Siliang Tang, 2024. Efficient Tuning Inference for Large Language Models on Textual Graphs. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Limitations are conclude Section 7
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification: NA
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The datasets, graph dataset generation code, model code, and hyperparameters are in the code repository.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification:
Our code is available at: https//gghalcccmm///aapaoooooooo
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ /pipp/.cc/pulli///uuii wwwiiiccguie//eeeeeeeeeic CoodeSubeiiie ssi ssionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
16

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Complexity analysis conclude in Appendix D
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps://neur: /ttes::///uips./.eeee www.eeew.......uu.eeeee httpp//ccccppp...tttttccccclleea
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
17

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee whworee.wthhhe....c.. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
18

---

Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
19

---

Justification: NA
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy htteas////eerrpppoccc///o/rreers for what should or should not be described.
20

---

Table Statisties of the original datasets.
Methods
#RNA
#ATAC
#Cell (Positive Sample)
#Cell Types
ISSAAC-seq
32,208
169,180
10,361
23
10XPBMC
29,095
107,194
9,631
19
SHARE-seq
21,478
340,341
32,231
22
SNARE-seq
28,930
241,757
9.190
22
10xMultiome
13,431
116,490
69,249
22
A Related Works of Graph Neural Networks and Graph Transformers
propagate and aggregate neighborhood information In the heterogeneous graph setting (49 20 42). such mechanisms allow flexible information fusion across
Graph neural networks (GNNs) 25 39 16
through message passing, making them well-suited for biological fields [5] 26 52 [1]
node types and modalities, which aligns with design of RNA-ATAC bipartite graphs.
48 32, 54] have gained popularity due to their global receptive fields and capacity to model complex dependencies. Attention not only captures relations but also enhances interpretability by quantifying the importance of each interaction
Recently, graph transformers [T
45 53
Building the ABG framework, we design a biologically-driver crossmodal graph transformer tailored the single-cell omics context. By incorporating biological priors and biological pruning. our model learns fine-grained regulatory patterns between ATAC and RNA. The attention module is carefully designed to highlight interpretable crossmodal signals, enabling us to uncover meaningful ATAC-RNA interactions patterns at both the cell-level and cell-type-level. Unlike generic graph attention methods, our approach grounds the attention weights in biological relevance, offering interpretability beyond performance.
B
Experimental Details
B.1 Details of the Original Datasets
We evaluate Bi² Former on four widely-used single-cell omics datasets that provide paired scRNA-seq and ScATC--e profiles from the same cells. Summary statisties are provided in Table
ISSAAC-seq. ISSAAC-seq [50] is a large-scale human multi-omics dataset that jointly profiles chromatin accessibility and gene expression at single-cell resolution. contains over 10,000 cells spanning 23 immune and epithelial cell types, making it suitable for evaluating both matching performance and generalization across diverse cell identities.
10x PBMC. The 10x Multiome PBMC dataset ① includes peripheral blood mononuclear cells from healthy donors, providing paired ATAC and RNA modalities with moderate sparsity and cell-type diversity. is benchmark dataset in many multimodal learning studies.
SHARE-seq. SHARE-seq 28 is one of the largest publicly available paired multi-omics datasets, capturing chromatin accessibility and gene expression across over 30,000 cells. Due to its large peak count and sparse feature distributions, it particularly challenging for cross-modal modeling.
SNARE-Seq. SNARE-Seq g enables simultaneous profiling of RNA and chromatin accessibility, particularly focused on neural tissues. Despite moderate sample size, its high ATAC dimensionality and -issuc-ppecific regulatory features make it useful for evaluating biological interpretability.
10x Multiome. The 10x Genomics Multiome dataset [27] originates from the 2021 NeurIPS Open
Problems in Single-Cell Analysis competition. The dataset comprises several thousand cells spanning major immune cell types, with moderate feature sparsity and well-balanced cell-type representa- tion. These properties make it a widely used benchmark for evaluating crossmodal matching, joint embedding, and preddctton methods under realistic paired-data conditions.
21

---

Table 5: Statistics of our ABG datasets.
Methods
#Graphs
Avg.HRNA Nodes
Avg.#ATAC Nodes
Avg.#Hdges
Avg.Sparsity
Split(S)
ISSAAC-seq
20,722
1,843
7,578
851,136
0.06
60/20/20
10XPBMC
19,262
1,924
7,379
762,927
0.05
60/20/20
SHARE-seq
64,462
619
3,971
157,132
0.06
60/20/20
SNARE-seq
18,380
937
2,452
133,146
0.05
60/20/20
B.2 Details of the ABG Datasets
The summary statistics of constructed ABG corpus are detailed in Table
As described in Section 3 during graph construction, encode biological attributes into the features of each RNA and ATAC node by BioAttr. Specifically, RNA nodes include attributes: ('chrom', means' variances sas___ommm strand', I'chrom', dddandneaueeeeeeeee seaus_seeeeee' J. Categorical and Statistics attributes are processed via one-hot encoding and direct numerical normalization, respectively. The iin_sequenne truncated 256 dims and encoded using sequential encoding scheme.
variable'), and ATAC nodes include:
While matched pairs reflect true biological interaction patterns across modalities, mismatched pairs are crucial for learning robust decision boundary. They help the model distinguish meaningful alignments from random co-occurrence. This discrimination is particularly important given the high dimensionality and noise in single-cell omics data. To preserve biological diversity and avoid sampling bias, negative samples drawn proportionally according to the distribution of cell types in the dataset. As summarized in Table 5] we maintain 1:1 ratio of positive to negative pairs, resulting graph dataset that contains twice the number of samples as the original single-cell dataset. Full implementation details are available in our code repository.
B.3 Details of the Cross-cell-types Setting
To assess the generalization capability of our method, evaluate the performance of Bi Former under a cross-cell-type setting. We split each dataset into training and test sets with disjoint cell types in 1:1 ratio. To ensure balanced partition, sort all cell types by their number of cells and assign those at odd and even indices to the training and test sets, respectively. Specifically:
ISSAAC-seq. Training cell types: {'R3 Ex-L5 IT", "R13 In-Drd2", "R8 Ex-L6 IT Bmp3", "R16 In-Sst", "R10 Ex-L6b", "R21 Oligo"); Test cell types: ("R7 Ex-L6 "R12 Misc", "R6 EX-L5-PT", "R20 OPC", "R9 Ex-L6 IT Oprk1", "R22 VLMC").
Training cell types: ("CD14 Mono", "CD8 Naive", "CD16 Mono", "CD8 TEM_1,, "Intermediate B", "CD4 TEM". "Treg", "MAIT", "pDC", "Plasma") Test cell types: ("CD4 Naive". "CD4 TCM", "NK", "CD8 TEM_21 2", "Memory B", "cDC", "gdT", "Naive B", "HSPC").
10x PBMC.
SHARE-seq. Training cell types: ("Basal", "TAC-1", "CD16 Mono". "alow CD34+ bulge", "Hair " "ORS". "Medulla", "Dermal Papilla", "IRS", "Dermal Sheath", "Macrophage DC". "Sebaceous Gland"); Test cell types: ("Infundibulum" "Spinous", "ahigh CD34+ bulge". "Dermal Fibroblast", "TAC-2", "Endothelial", "Isthmus", "K6+ Bulge/Companion Layer", "Granular", "Melanocyte", "Schwann Cell").
SNARE-seq. Training cell types: ("E2Rasgrf2", "E6Tle4". "E5Galnt14". "Ast", "InP", "E3Rmst", "InS", "InV", "OPC", "Mic", "Endo"); Test cell types: ("E3Rorb", "E4111rap12", "B4Thsd7a", "E5Parm1", "OliM". "E5Sulfi", "Clau", "InN", "Olil", "Peri").
B.4 Hyperparameters
Experimental results are reported on the hyperparameter settings below, where we choose the settings that achieve the highest performance the validation set. We choose hyperparameter grids that do
22

---

Threshold =0.5
Threhhodd = 0.8
Threshold 09
Threshold 0.95
Figure 6: Case study of RNA-ATAC attention matrices for a representative single cell under different threshold values Higher thresholds remove noisy interactions and highlight confident regulatory links.
not necessarily give optimal performance, but hopefully cover enough regimes so that each model is reasonably evaluated on each dataset.
learning_ rate E {le 3, 5e 4, 1e 4,5e 5, 5}
weight_decay E {le - 4, 4e 5, le
5,5e 1,e-5,5e
6, le 6}
dropout E (0,0.1,0.3,05,,00
For Bi2Former,
ID embedding dims (64, 128, 256, 512)
hidden dims E {64, 128, 256, 512]
layer_num E {1, 2}
Case of Biological Interpretation and Discovery
C.1 Cell-Level Regulatory Interaction.
To further demonstrate the interpretability of BirFormer and its capacity to reveal interaction between ATAC and RNA, we conduct a case study visualizing the learned RNA-ATAC attention matrix ă under varying threshold settings for representative single cell. As shown in Figure 6 increasing the threshold チ progressively suppresses low-confidence signals, resulting a sparser attention matrix and more specific and meaningful regulatory signals.
However, we observe that without further constraints, some RNA nodes may either lack any activated ATAC connections or remain dense connected-bbot of which deviate from biological priors. To address this, we introduce top-k constraint following thresholding. After training with top-k regularization, the model not only achieves better performance but also produces more interpretable attention patterns. As illustrated Figure each RNA regulated by limited number of ATAC peaks, consistent with known biological principles of gene regulation.
23

---

Figure Improved RNA-ATAC interaction map after applying top-k refinement. Each RNA is connected to limited number of ATAC peaks, aligning with known biological regulation mechanisms.
a_
TI TFPBiniing Soar
Naive
Caredattvve Altention Score Store
CD4 Nalve
Peak Inder
Cosine Similarity: 0.608 Cosine similarity on ranks: 0.909 Pearson Correlation Coefficient: 0.633 accard Index (TOP10000): 0.548
Figure 8: Comparison between TF binding scores and cumulative attention scores within the same cell type (CD4). The cumulative attention scores produced by our model align closely with the TF binding intensity in CD4 cells, suggesting that the learned crossmodal interactions accurately capture cell-type-specific regulatory signals.
C.2 Cell-Type-Level Regulatory Maps.
To investigate how ATAC-RNA regulatory patterns vary actoss cell types, aggregate ATAC expression signals across cells of the same type and analyze the resulting attention-informed regulatory maps. Specifically, compute the cumulative attention score the regulatory strength of each ATAC peak across different cell types to identify sell-type-specific activation patterns.
As shown in Figure Rb and Figure 9b, clear differences emerge between CD4 and CD14 cell populations, revealing divergent regulatory patterns that reflect their distinct biological functions. Moreover, to further validate the biological relevance of our model, we compare the cumulative attention scores against experimentally derived TF binding scores (21,, which reflect the actual activation strength of ATAC peaks in each cell type and serve as proxy for ground truth. In F the cumulative attention scores for CD4 cells exhibit strong agreement with CD4-specific TF binding signals, indicating that our model successfully identifies biologically meaningful regulatory relationships. In contrast, Figure 9 that CD14 cells exhibit different attention profiles relative to the CD4 ground truth, underscoring cell-type-specif regulatory patterns. These results highlight the capacity of our approach to uncover interpretable and biologically grounded crossmodal interactions the population level.
Overall, these cell-type-level maps offer powerful means to dissect cell identity through the strength of ATAC peak signals.
24

---

TF Binching Scorn
CD4 Nalve
b.
Cummbattvee A Atteetion Soore
CD14 Mono
Feak Index
Cosine Similarity: 0.330 Cosine similarity on ranks 0.674 P Pearson Correlation Coefficient: 0.358 J Jaccard Index (TOP 10000): 0.302
Figure 9: Comparison between TF binding scores from CD4 cells and cumulative attention scores in CD14 cells. CD14 cells exhibit distinct peak activation patterns with the CD4 ground truth, highlighting the distinct regulatory patterns across cell types.
D Complexity Analysis
In this section, we present the time and space complexity analysis Bir For simplicity, we assume that the feature dimension remains unchanged and that the number of model layers is set to 1, and the N, and N₂ denote the number of RNA and ATAC nodes in the graph, i.e,, |RNAl and |Atacc. I. E' is the number of edges (i.e., the number of non-zero entries in the adjacency matrix A), and d₃, is the hidden dimension.
D.1 Time Complexity
The primary time overhead arises from three components: the Ghologically-driverr Crossmodal Atten- tion, the Crossmodal Message Passing, and the Predictor. The time complexity of the Crossmodal Attention is ONN,d + Nadd Ed,). The Crossmodal Message Passing aggregates messages from sparse attention edges and includes self-attentions, with complexity O((N, + N, N.)dh + ). Finally, the complexity the Predictor is negligible overhead of (dh)). Thus the total time complexity of our method is O(N, + N₂ N. + F.d))-
Compared with VAE-based methods, which operate on full dense expression matrices with time complexity (NNe))) (where N is the total number of ATAC and RNA, typically of times larger than the number of expressed N, + N, per graph), our method is significantly more efficient.
D.2 Space Complexity
For each graph, we maintain hidden node embeddings, sparse attention matrices, and aggregated messages. Specifically, the feature storage and value projections require O(N,, + N₂ dd, ); and the attention matrix and message buffers take O(Ed,). Hence, the overall space complexity is O( + N. d - Eda).
Compared with VAE-based methods with space complexity O(Nd,), method reduces unneces- sary memory usage on unexpressed nodes, leading to significantly improved memory efficiency.
25


Reviews:
[{"review_id": "MsmDq8Tb8n", "reviewer": "Reviewer_htWu", "summary": "This work introduces Bi²Former, a biologically-informed bipartite graph transformer designed for single-cell cross-modal matching. The problem is framed as a graph classification task using Attributed Bipartite Graphs (ABGs), enabling explicit modeling of interactions between ATAC and RNA features. Bi²Former learns interpretable attention over regulatory pairs to uncover cross-modal relationships. Extensive experiments demonstrate state-of-the-art performance, strong generalization to unseen cell types, and biologically meaningful insights. The proposed framework offers an interpretable and structured approach to cross-modal learning in single-cell biology.", "questions": "For Cross-Cell-Type Generalization, can you train on one dataset to evaluate on a separate dataset with unseen cell types in training?", "limitations": "The tarining is based onthe paired scRNA-seq and scATAC-seq data, which is limited.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 1, "strengths_and_weaknesses": "Strengths:\n1. The work is easy to follow.\n\nWeaknesses:\n1. The proposed method is not novel to me.\n2. highly relevant and similar work missing, scMoGNN\nSince your method also uses a bipartite-graph formulation for modality matching, please compare it with scMoGNN—to my knowledge the first GNN-based approach for multi-omics prediction, modality matching, and joint embedding—which likewise builds a bipartite graph.\na. Bipartite graphs for multi-omics integration, ATAC-RNA, RNA-protein, and so on.\nb. also including prior knowledge\npaper: Graph Neural Networks for Multimodal Single-Cell Data Integration, KDD 2022, NeurIPS Multiomics Integration Winner solution\n3. A highly relevant baseline is missing, scMoGNN; please also include the NeurIPS 2021/2022 Multiomics integration dataset, which is larger.", "was_revised": true, "final_justification": "finer-grained modeling paradigm for graph neural networks is introduced in single-cell multi-omics, introducing attributed bipartite graphs that shift focus from cell-level to gene- and peak-level representation learning, enabling more biologically meaningful and generalizable interaction capture.", "rebuttal": "Dear Reviewer htWu，\n\nThanks for your constructive comments. Below we will address your concerns.\n\n***\n\n## **W1&W2. Regarding Novelty and Similar Work**\n\nWe sincerely thank you for raising this important point. However, we believe there is a misunderstanding. While scMoGNN also utilize graph neural networks for modeling single-cell multi-omics data, our methodology and contributions differ significantly from prior works in terms of **motivation, graph construction, model design, and task definition**.\n\n### 1. Motivation\nscMoGNN is primarily designed to learn integrated cell-level representations by modeling the cells and their omics features (e.g., RNA, ATAC) through a graph. It leverages GNNs for aggregation at the **inter-cell level**, yielding **integrated embeddings**.\n\nIn contrast, our motivation is to **uncover fine-grained intra-cell crossmodal interactions**(e.g., how chromatin accessibility influences gene expression), aiming for a model that is highly generalizable and biologically interpretable. To achieve this, we emphasize the **node-level** modeling of pairwise interactions between expressed RNA and ATAC within each cell as an independent interaction instance, enabling the model to capture the interaction patterns between them (Or any two or among more modalities).\n\n### 2. Graph Construction\nscMoGNN constructs a graph where **each cell is treated as a node**, and omics features (e.g., ATAC, RNA) are also represented as nodes. It is important to note that in scMoGNN, both the cell nodes and feature nodes (i.e., RNA and ATAC nodes) do not carry biological attributes as learnable features—thus it is a **non-attributed Bipartite Graph**, which prevents the model from learning generalizable properties from biological attributes (Leading challenge 2 in our paper). Moreover, the **reliance on predefined prior graphs** limits scalability and generalization across datasets, and prevents it from capturing fine-grained crossmodal interaction patterns, leading to a lack of interpretability (Suffering challenge 3).\n\nIn contrast, our model builds an **Attributed Bipartite Graph** (ABG) for each individual cell, in which RNA and ATAC features are treated as nodes enriched with biologically meaningful attributes. These attributes enable the model to learn representations that generalize across datasets and cell types. Additionally, our edge construction leverages biological priors in a dynamic and cell-specific manner—e.g., connecting ATAC and RNA nodes based on chromatin co-localization—to embed domain priors into the graph, allowing the graph structure to reflect fine-grained, biologically interpretable crossmodal interactions.\n\n### 3. Model Design\nIn line with its graph construction, scMoGNN primarily focuses on aggregating multi-hop neighborhood information to obtain cell-level embeddings using **a basic Graph Convolution Networks**. Its reliance on predefined and limited interaction priors means the model emphasizes leveraging known biological priors, rather than learning or discovering interaction patterns.\n\nIn contrast, our model is designed to learn potential crossmodal interaction patterns between RNA and ATAC features (Or any two or among more modalities) through a **biological-driven Graph Transformer**. Instead of using a fixed structure, we allow the model to learn interaction weights from data. The resulting attention maps not only align well with actual biological evidence (e.g., TF-binding signals), but also contributing to strong generalization capability of our model across cell types and datasets.\n\n### 4. Formulated Task \nIt is important to clarify that although scMoGNN includes a modality matching task, its formulation and purpose differ significantly from ours. Specifically, scMoGNN aims to predict the **probability** that two profiles from different modalities originate from the same cell, serving as an auxiliary task to learning high-quality cell-level representations. Learning the crossmodal interaction patterns remains a challange.\n\nIn contrast, our modality matching task serves as a proxy to uncover biological interaction patterns between modalities, as the correct matches reflect the interaction patterns governed by underlying biology. Rather than estimating the probability, our method explicitly models the **binary decision** of whether a given RNA–ATAC pair is a biologically meaningful match. This enables our model to determine whether unpaired modalities belong to a same cell without fine-tuning, thereby reducing the cost of experimental measurements. \n\nApart from the modality matching task (see the supplemented comparison results in the response to W3), our model also supports **crossmodal generation**. The specific generation pipeline of our method can refer to our response to L1.2 of reviewer e9JF. The results of generation task are shown in the table below:\n\n> Table A: The results of generation task from ATAC modality to RNA modality. We report MAE($\\downarrow$) as evaluation metric.\n\n||ISSAAC-seq|10xMultiome-PBMC|SHARE-seq|SNARE-seq\n:-|:-:|:-:|:-:|:-:\nMultiVI|0.097|0.132|0.114|0.129\nGLUE|0.108|0.140|0.107|0.117\nscMoGNN|0.095|0.101|0.098|0.112\nBi2Former(Ours)|0.072|0.082|0.079|0.078\n\nOverall, while basic GNNs have been explored in prior work as powerful tools for single-cell multi-omics and many other AI4Science problems, our study **addresses distinct and previously underexplored challenges** (uncovering the nature of crossmodal interaction patterns, achieving strong generalization through biological attributes, and enhancing biological interpretability). We will cite scMoGNN in the related work section and introduce these differences to prevent further misunderstanding.\n\n***\n\n## **W3. About the Baseline**\nWe appreciate the suggestion to include scMoGNN as a baseline and the NeurIPS dataset. We have integrated scMoGNN and NeurIPS 2021 dataset in our experiments and will include results and discussions in the revised version. The results are shown in the table below:\n\n> Table B: The results of more baselines of crossmodal matching task. We report ACC($\\uparrow$) as evaluation metric.\n\n||ISSAAC-seq|10xMultiome-PBMC|SHARE-seq|SNARE-seq|NeurIPS 2021 Multiomics|Avg.\n:-|:-:|:-:|:-:|:-:|:-:|:-:\nscMoGNN|73.72±0.96|72.41±1.37|69.84±1.81|69.03±1.22|75.49±1.31|72.10\nBi2Former(Ablation variant)|74.28±1.07|72.54±1.46|70.19±2.06|69.84±1.27|77.83±0.98|72.94\nBi2Former(Ours)|84.40±0.48|88.74±0.36|79.84±0.29|73.56±0.37|90.41±0.24|83.39\n\nOur model consistently achieves superior performance across all evaluated datasets. More importantly, when comparing against an ablation variant of our model (in which biological attributes and edges are removed, such that the primary difference lies in the model architecture), our method still outperforms scMoGNN. This provides evidence that our biologically-driven attention-based framework is more suitable for capturing interactions in single-cell data than simple GCN aggregation, further highlighting the novelty and significance of our proposed approach.\n\n***\n\n## **Q1. Further Generalization Ability Experiments**\nTo further demonstrate the robustness and generalization ability of our model, we have conducted zero-shot cross-dataset experiments. Specifically, we pretrained the model on source datasets and evaluated it directly on different target datasets without any fine-tuning. The results are summarized in the table below:\n\n> Table C: Zero-shot inference results for crossmodal matching task with training across various source dataset(s). The target dataset is ISSAAC-seq.\n\nSource dataset(s)|10xMultiome-PBMC|10xMultiome-PBMC+SHARE-seq|10xMultiome-PBMC+SHARE-seq+SNARE-seq\n:-|:-:|:-:|:-:\nMultiVI|47.82|48.19|51.24\nCLUE|51.07|49.67|52.93\nscMoGNN|54.74|61.02|58.89\nBi2Former(Ours)|71.14|79.16|80.03\n\nUnder the zero-shot setting, our model achieves a clear advantage over the baselines. As the size of the training dataset increases, the performance on the target dataset continues to improve, demonstrating that our model effectively generalizes by learning from biological attribute information and discovering crossmodal interaction patterns. \n\n***\n\n## **L1. About the Limitation**\nWhile it is true that paired data are required during training (which is necessary during training for almost all crossmodal integration models), our model is designed to be **highly data-efficient and generalizable**. By incorporating biological attributes as node representations and explicitly modeling crossmodal interaction patterns, our method requires fewer paired samples than previous approaches while achieving superior performance and strong generalization. The results of the performence in different training sizes are detailed in our response to W1&Q1.1&Q1.2 of Reviewer 7QNJ. In the revised version, we have added concerns about paired data to the limitations section.\n\nFurthermore, our framework is **not limited to scRNA-seq and scATAC-seq**. The Bi²Former architecture and its biological-driven attention mechanism are general and can be directly applied to other modality pairs (e.g., RNA–protein, ATAC–methylation) for tasks such as matching, generation, or interpretability with only minor modifications. More analysis and results are included in our response to W2&Q3 of Reviewer 7QNJ.\n\n***\nThanks again for your effort in reviewing our work. If you have any additional concerns or suggestions, please feel free to share them, we would be happy to discuss further. We sincerely value your feedback.\n\nSincerely,"}, {"review_id": "I0ThvS9wd0", "reviewer": "Reviewer_KGd3", "summary": "The paper presents  Bi$^2$ Former - a method for cross-modal matching between scRNA and ATAC data and predicting whether the data originates from the same cell or not. In order to do this, the authors leverage an attributed bipartite graph (with explicit features for scRNA data (in addition to gene expression) and ATAC data (in addition to chromatin accessibility)) and learn an attention-based mechanism for inferring cross-modal matching. They leverage domain-informed adjacency bipartite graph as well as biological pruning to eliminate noise and get a robust signal.\n\nThe authors show with evaluation on 4 datasets  that their model performs better than state-of-the-art for crossmodal matching. They also show ablation study highlighting contribution of different features including integration of additional node attributes, influence of domain-informed base adjaceny matrix, biological pruning and (learnt) node embeddings.", "questions": "- In the ablation study, the influence of Edge is highest. I understand the chromosomal mask as $A_{ij}=1$ defining whether gene $i$ lies on chromosome $j$ or $0$ if not. In the ablation without the intrachromosomal restriction, what is the fraction of inter-chromosomal interactions post biological pruning i.e., $ \\alpha_{ij}=1$ where $ A_{ij}=0 $ ?\n- How could the method be extended to more than two modalities?", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "# Strengths \n\n- Paper is well-written. \n- Strong empirical evidence is provided on the utility of the method. \n\n# Weaknesses\n- MultiModal data integration for scRNA-seq data and scATAC data is an important problem. However, there are now kits available, that allow collection of both these modalities from the same cell (e.g., https://www.10xgenomics.com/products/epi-multiome). Further, several methods have been developed for such aligned data including Bayesian approaches like MOFA (https://www.embopress.org/doi/full/10.15252/msb.20178124), MOFA+(https://genomebiology.biomedcentral.com/articles/10.1186/s13059-020-02015-1). Other approaches including linear algebraic or deep learning based are also relevant. Please see review in https://arxiv.org/abs/2501.17729 . \n\nMy first question: The paper should clearly position its contribution compared to existing approaches. Specifically, if only targetting scATAC-seq and scRNAseq, since there are technologies that make the whole problem moot  by providing both modalities for the same cell (when using 10X kits), the significance should be highlighted. \n\nAlso, would it be possible to show the results of the method on 3k cells from the frozen human healthy brain tissue on data by 10x Genomics here: https://www.10xgenomics.com/resources/datasets/frozen-human-healthy-brain-tissue-3-k-1-standard-2-0-0\n\n- Alternative modalities for which the data has to necessarily be matched are present, e.g.,  scRNAseq with Cut&Tag methods that can provide high-resolution signal compared to ChiPseq used in the past. How can the method be extended beyond scATACseq+snRNAseq integration? \n\n- I have not worked with scATAC data myself but am very familiar with previous single cell pipelines (e.g., SEURAT, scanpy, etc.). These pipelines are well-established in the field. For example, see https://satijalab.org/seurat/articles/seurat5_atacseq_integration_vignette . What would be the practicality of this method compared to previous methods.", "was_revised": true, "final_justification": "Changed to weak accept after review discussion", "rebuttal": "Dear Reviewer KGd3,\n\nThanks for your comments. Below we will address your questions.\n\n***\n\n## **W1 The Contribution (or Motivation & Utility) of the Work**\nWe sincerely apologize for any confusion, and we would like to clarify our contributions and utility from three perspectives:\n\n### 1). Reducing Experimental Measurement Costs\nWhile technologies such as 10X allow for profiling both modalities (e.g., RNA and ATAC) from the same cell, they require destructive and costly protocols, making the acquisition of paired multi-omics data **expensive and limited**. \nImportantly, our method is **not in conflict with such technologies**; instead, it leverages a small amount of paired data obtained via such technologies to train a model that generalizes well to unseen data. Our model is designed to be **highly data-efficient and generalizable**, and thus can significantly reduce the cost for paired data in the following scenarios:\n- **Semi-paired setting**: When only a subset of cells or cell types have paired measurements, our model can be trained on this limited data and generalize to unseen cells by leveraging biological attributes and interaction patterns learned during training.\n- **Unpaired setting (zero-shot)**: Existing VAE-based models typically require retraining or fine-tuning on new datasets. Our model can be pretrained on available source datasets and directly evaluated on target datasets without any fine-tuning. This simulates a real-world use case where no pairing is available at test time. Results from our zero-shot experiments are presented in the response to Q1 of Reviewer htWu.\n- **Crossmodal generation**: Beyond matching, our model also supports crossmodal generation, i.e., predicting one modality from another, even under the zero-shot setting. This allows estimation of unmeasured modalities, further reducing experimental costs. Compared to traditional VAE-based generative architectures, our generation mechanism offers greater biological intuition and interpretability, while also demonstrating superior performance. The pipeline and results of this generation task are detailed in our response to L1.2 of Reviewer e9JF.\n\n### 2). Uncovering Fine-Grained Crossmodal Interactions\nCurrent biological measurement techniques and prior deep learning-based models for crossmodal integration (or matching and generation) **fail to explicitly model or learn fine-grained interactions between modalities** (e.g., how chromatin accessibility influences gene expression). In contrast, the core utility of our approach lies not merely in performing matching, integration, or generation tasks(which is only a small part of our contributions), but rather in leveraging crossmodal matching as a proxy for **learning and uncovering the underlying biological interaction patterns** which are highly aligned with known biology. This leads to strong generalization, interpretability, and biological plausibility. We provide extensive empirical validation of the biological relevance, interpretability, and discovery potential of the learned attention patterns in our work. These results are discussed in detail in the response to W2&W3 of Reviewer e9JF.\n\n### 3). Generalization to Other Modalities \nOur framework is **not limited to scRNA-seq and scATAC-seq**. The Bi²Former architecture and its biological-driven attention mechanism are general and can be directly applied to other modality pairs (e.g., RNA–protein, ATAC–methylation) for tasks such as matching, generation, or interpretability with only minor modifications. More analysis and results are included in the response to Q2 below.\n\nOverall, while existing experimental technologies and modeling frameworks (e.g., Bayesian or deep learning-based integration) have enabled multi-omics analysis, our study **addresses distinct and previously underexplored challenges**.\n\n\n***\n\n## **W2 Results on Flash-Frozen Human Healthy Brain Tissue (3k) Dataset**\n\nWe have integrated the Flash-Frozen Human Healthy Brain Tissue (3k) dataset in our experiments. The results are shown in the table below:\n\n> Table D: The results of crossmodal matching task and generation from ATAC modality to RNA modality on Flash-Frozen Human Healthy Brain Tissue (3k) dataset.\n\n||Matching(ACC($\\uparrow$))|Generation(MAE($\\downarrow$))\n:-|:-:|:-:\nMultiVI|72.13|0.124\nCLUE|74.89|0.107\nscMoGNN|76.84|0.118\nBi2Former(Ours)|95.79|0.069\n\nWe noticed that chromosome information is not available in this dataset, so we constructed the graphs without edge connections. Nevertheless, Bi2Former still demonstrated strong performance, further highlighting the robustness of our approach.\n\n***\n\n## **W3 Extended beyond scATACseq+scRNAseq integration**\nPlease check the response to Q2 below.\n\n## **W4 The Practicality Ccompared to Previous Methods**\nPlease refer to the response to W1.\n\n***\n\n## **Q1 The Fraction of Inter-chromosomal Interactions**\nIn our framework, the graph edges serve multiple purposes:\n- They introduce biological priors into the Attributed Bipartite Graph (ABG) to enhance learning performance;\n- They provide inductive bias to guide the message aggregation process, improving training efficiency and convergence;\n- They help reduce computational complexity by constraining attention computation to the existing sparse edges.\n\nAs analyzed in Section 5.4 (Line 279–285), when edges are removed, the model's performance degrades but still remains competitive compared to VAE-based baselines. To further address your concern, we conducted additional experiments to evaluate the learned attention under a noisy topology (without edges):\n\n> Table E: The results of the fraction of inter-chromosomal interactions post biological pruning on ISSAAC-seq dataset.\n\n||ACC($\\uparrow$)|Inter-chromosomal(%) \n:-|:-:|:-:\nMultiVI| 66.21| -\nCLUE|71.28|-\nBi2Former|84.40|100.00\nBi2Former wo. Edge|78.89|94.87\n\nThese results demonstrate that our attention mechanism remains effective even without edge constraints. Notably, the Bi2Former without predefined edges still learns interaction patterns that are predominantly intrachromosomal, aligning with biological expectations. This highlights the model’s robustness and its capacity to infer biologically meaningful interactions even under edge-free or noisy topologies.\n\n***\n\n## **Q2 Generalization to Other Modalities**\n### 1). Two modalities\nOur framework is **not limited to scRNA-seq and scATAC-seq**. The Bi²Former architecture and its biological-driven attention mechanism are general and can be directly applied to other modality pairs (e.g., RNA–Protein, ATAC–Methylation) for tasks such as matching, generation, or interpretability with only minor modifications. \nSpecifically, the only modifications needed are in the biological attributes of nodes and the edge construction strategy, depending on the modalities involved. The core model architecture remains unchanged, showcasing the flexibility and extensibility of our approach. To further address your concern, we also empirically report the RNA-Protein matching results:\n\n> Table F: The results of crossmodal matching task between RNA and Protein. We report ACC($\\uparrow$) as evaluation metric.\n\n||NeurIPS 2021 Multiomics\n:-|-\nMultiVI|66.32±1.02\nCLUE|74.97±1.53\nBi2Former(Ours)|84.21±0.48\n\n### 2). Three or more modalities\nOur framework can be naturally extended to handle three or more modalities with only minor modifications. When multiple modality types (e.g., RNA, ATAC, and Protein) are involved in the graph, we generalize the ABG into a heterogeneous attributed graph. In this setting, each modality is treated as a distinct node type, and edges are constructed to represent biologically meaningful relationships between different modality pairs. During model training, we perform relation-specific attention and aggregation for each edge type. This design enables Bi2Former to support a wide range of tasks including matching, generation, and interaction pattern discovery across any pair of modalities. We evaluated our model on the NeurIPS 2021 Multiomics dataset, the results for the crossmodal matching task are summarized below:\n\n> Table G: The results of crossmodal generation task on the NeurIPS 2021 Multiomics dataset. We report MAE($\\downarrow$) as evaluation metric.\n\n||RNA2ATAC|RNA2Protein\n:-|-|-\nCLUE|0.121|0.147\nBi2Former(Ours)|0.083|0.102\n\n***\nThanks again for your comments. If you have any additional concerns or suggestions, please feel free to share them, and we sincerely value your feedback.\n\nSincerely,"}, {"review_id": "F4VcI8csTy", "reviewer": "Reviewer_e9JF", "summary": "This paper introduces a method based on ABG to match cells with different modalities. The paper is written well over. However, it primarily focuses on an artificial scenario created by the authors, namely, all the datasets they analyzed are from multiomes where the cells already have data from both RNA and ATAC modalities. There is no need to match cells. In this case, I am unsure whether it is a fair comparison with other methods designed for different tasks.", "questions": "The authors need to consider the case where the data are unpaired and with methods designed to match cells measured with one modality only.", "limitations": "The limitations discussed by the authors were not at the core of the problem, namely the artificial nature of their data set-up where the cell are already matched by the experimental design. In this setting, there is very limited utility of their method. Please also see my questions above.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths: The paper is written clearly and the results are well presented Weaknesses: The method is mostly designed for unpaired data, whereas all the data considered are already paired. It is not a fair comparison with other methods designed for paired data for different tasks. I have several questions and I hope the authors can answer them.\n\n1. The motivation is unclear, as the data should be unpaired data and the baselines are all designed for paired data. Including baselines such as scGLUE and scJoint will be helpful.\n\n2. Integration multi-omic data is already well-discussed and analyzed. Could the authors show more biology-driven discoveries to support their method?\n\n3. We do not find examples of tf-binding site analysis in the main text, if the authors intend to argue it as a downstream task, they should include analysis.", "was_revised": true, "final_justification": "I will keep my score.", "rebuttal": "Dear Reviewer e9JF，\n\nThank you for your valuable feedback. Below we will address your concerns.\n\n***\n\n## **W1 & Q1 Regarding Motivation and Experimental Setting**\n### 1). Clarification about the \"paired\" data and the experimental setting\nThank you for your feedback and valuable suggestions. First, we would like to clarify a possible misunderstanding regarding the experimental setting. Our training setup includes both paired and unpaired data, where unpaired data are generated via negative sampling, consistent with the definition in GLUE documentation[1]:\n\n*“Unpaired” means that different omics layers are not probed in the same single cells, but rather independent samples of (presumably) the same cell population.*\n\nGiven a sets of ATAC-RNA instances, where some instances are paired and others are not, our modality matching task is designed to determine whether a instance is paired (originates from the same cell). Importantly, in the testing set, the pairing status lables are **unknown**, meaning the evaluation is effectively conducted on data which not derived from any experimental pairing, making the comparison fair and unbiased.\n\n### 2). Our core motivation\nSecond, we emphasize that the core motivation of our work is not simply to integrate modalities for representation learning (which is only a small part of our contributions), but rather to **leverage crossmodal matching as a proxy** for learning the underlying biological interaction patterns. This yields a model with strong generalization capabilities and biological interpretability, which goes beyond conventional multimodal integration tasks. For a more detailed explanation, please refer to our extended discussion in the response to W1&W2 of Reviewer htWu.\n\n### 3). Fair comparison and more baselines\nFurthermore, all of the baselines we compared—such as crossmodal generation or representation integration models based on variational autoencoders (VAEs)—also require paired data for training. Their primary motivation is to learn integrated representations for downstream tasks. To ensure a **fair comparison**, we adopt modality matching as a downstream task for all baselines, generating representations from separate modalities and using a classifier head to predict whether two representations are paired. To further alleviate your concern, we have additionally included scGLUE[1] and scJoint[2] as baselines in our updated experiments. The results are shown in the table below: \n\n> Table H: The results of more baselines of crossmodal matching task. We report ACC($\\uparrow$) as evaluation metric. Our model consistently achieves superior performance across all evaluated datasets. \n\n||ISSAAC-seq|10xMultiome-PBMC|SHARE-seq|SNARE-seq|NeurIPS 2021 Multiomics|Avg.\n:-|:-:|:-:|:-:|:-:|:-:|:-:\nscMoGNN|73.72±0.96|72.41±1.37|69.84±1.81|69.03±1.22|75.49±1.31|72.10\nscGLUE|74.28±0.91|72.51±1.02|66.89±1.43|64.47±1.22|76.93±0.82|71.01\nscJoint|72.79±0.85|75.38±0.57|64.32±1.22|67.31±1.39|74.08±0.69|70.77\nBi2Former(Ours)|84.40±0.48|88.74±0.36|79.84±0.29|73.56±0.37|90.41±0.24|83.39\n\n[1] Multi-omics single-cell data integration and regulatory inference with graph-linked embedding. Nature Biotechnology 2022\n\n[2] scJoint integrates atlas-scale single-cell RNA-seq and ATAC-seq data with transfer learning. Nature Biotechnology 2022\n\n***\n\n## **W2 & W3 Biology-driven Discoveries**\nThank you for your feedback and we apologize for any confusion. We would like to clarify that we have included three types of biology-driven analyses in the main text and appendix. However, due to space limitations, two of them (b and c) are currently located in the appendix, with a brief summary provided in Section 5.5 (Line 308-323) of the main paper. We will expand these discussions in main text in the revised version. A summary of the biology-driven discoveries is provided below:\n\n### a). Cross-cell-type similarity of attention matrices (In Section 5.3)\nTo validate that our learned attention matrices reflect biologically plausible ATAC–RNA regulatory relationships, we leveraged the biological intuition that different cell types should exhibit distinct interaction patterns, and the degree of dissimilarity should correlate with cell-type divergence. We systematically analyzed and compared the pairwise similarity of learned attention matrices across:\n- Different splits of the same cell type.\n- Different cell types within the same dataset.\n- Different cell types across different datasets.\n\nThese analyses, summarized in Figure 3 and discussed in Lines 252–263, reveal that attention similarities decline as cell-type differences increase, supporting our hypothesis that Bi2Former captures cell-type-specific regulatory programs.\n\n### b). Cell-level attention visualization (In Appendix C.1)\n\nTo further support the biological plausibility of the learned attention matrices, we visualized the attention weights between ATAC and RNA nodes for individual cells. A known biological principle is that each gene is typically regulated by a limited number of accessible chromatin regions (ATAC peaks), often located on the same chromosome.\n\nIn Appendix C.1 (Lines 527–539, Figures 5 & 6), we show that for each RNA, only 5–15 ATAC peaks receive non-negligible attention weights, and these ATACs tend to be clustered by chromosome, aligning with known cis-regulatory mechanisms in gene regulation. These findings reinforce the interpretability and biological grounding of our model’s attention mechanism.\n\n### c). TF binding site alignment analysis (In Appendix C.2)\n\nWe further examined whether the attention scores learned by our model correlate with experimentally derived transcription factor (TF) binding sites, which represent real regulatory activity. Specifically, in Appendix C.2 (Lines 539–556, Figures 7 & 8), we aggregated attention weights across ATAC peaks for CD4 and CD14 cells and compared them with TF binding signals known to be specific to those cell types.\n\nThe results show that:\n- In CD4 cells, ATAC peaks with high attention weights from our model coincide with strong CD4-specific TF binding signals.\n- In contrast, CD14 cells show distinct attention patterns, differing from the CD4-specific TF ground truth.\n\nThis suggests that our model captures cell-type-specific regulatory mechanisms and aligns well with experimental regulatory evidence, further validating its biological relevance.\n\n***\n\n## **L1 About the Utility of Our Method**\n### 1). Clarification of the experimental setting\nAs clarified in our response to Q1, in the testing set, the pairing status lables are unseen for models, meaning the inference is effectively conducted on both **\"matched\" data** and **unmatched data**. Moreover, our model can determine whether unpaired modalities belong to a same cell without fine-tuning even in zero-shot settings (please check the response to Q1 of Reviewer htWu), which is hard for previous VAE methods, thereby further **reducing the cost of experimental measurements**. Our model is designed to be **highly data-efficient and generalizable**. By incorporating biological attributes as node representations and explicitly modeling crossmodal interaction patterns, our method requires fewer paired samples than previous approaches while achieving superior performance and strong generalization. The results of the performence in different training sizes are detailed in our response to W1&Q1.1&Q1.2 of Reviewer 7QNJ.\n\n### 2). Crossmodal generation and uncovering crossmodal interaction patterns\nThe model also supports **crossmodal generation**, thus further reducing experimental costs. For instance, given a set of ATAC nodes expressed within a cell, the model computes attention scores between these nodes and all RNA nodes in a reference database. By applying a thresholding strategy to the attention matrix, we infer which RNAs are likely to be expressed—effectively generating the RNA modality from the ATAC modality. Compared to traditional VAE-based generative architectures, our generation mechanism offers greater biological intuition and interpretability, while also demonstrating superior performance. The results of partial generation task are shown in the table below:\n\n> Table A: The results of generation task from ATAC modality to RNA modality. We report MAE($\\downarrow$) as evaluation metric.\n\n||ISSAAC-seq|10xMultiome-PBMC|SHARE-seq|SNARE-seq\n:-|:-:|:-:|:-:|:-:\nMultiVI|0.097|0.132|0.114|0.129\nGLUE|0.108|0.140|0.107|0.117\nscMoGNN|0.095|0.101|0.098|0.112\nBi2Former(Ours)|0.072|0.082|0.079|0.078\n\nBeyond crossmodal matching and generation, the core utility of our method lies in its ability to uncover and interpret fine-grained crossmodal regulatory interactions. By modeling these interactions at a fine scale, our framework supports interpretable crossmodal analysis, enabling new biological insights into cell-type-specific regulatory programs. As discussed in the response to W2&W3.\n\n### 3). Generalization to more modalities\nMoreover, our framework is **not limited to scRNA-seq and scATAC-seq**. The Bi2Former architecture and its biological-driven attention mechanism are general and can be directly applied to other modality pairs (e.g., RNA–protein, ATAC–methylation) for tasks such as matching, generation, or interpretability with only minor modifications. More analysis and results are included in our response to W2&Q3 of Reviewer 7QNJ.\n\nIn summary, the utility of our method is multi-faceted—it not only learns high-quality representations but also reduces experimental cost for matching unpaired data, enables interpretable crossmodal generation, and reveals biological interaction patterns across modalities. In the revised version, we will expand the relevant discussion in the Limitation section.\n\n***\nThanks again for your effort in reviewing our work. We hope our response helps to address your concerns. If you still have questions, please don't hesitate to let us know. We’re happy to discuss further.\n\nSincerely,"}, {"review_id": "XEA3qEGwEM", "reviewer": "Reviewer_7QNJ", "summary": "This paper introduces a method for crossmodal matching in single-cell omics by casting it as a graph classification task on Attributed Bipartite Graphs. The authors model single-cell omics data ( ATAC and RNA) modalities, as ABG with nodes representing individual ATAC and RNA features, incorporating IDs and biological attributes. They propose a transformer architecture to learn interpretable attention over ATAC and RNA pairs, aiming to capture regulatory relationships. Empirical experiments are conducted on several datasets to validate the method performance in cross-modal matching, its generalization to unseen cell types, and its ability to capture biologically meaningful regulatory patterns.", "questions": "1. Can the authors discuss the robustness of the method against the problem of unpaired data? Is training on large paired data required to achieve good performance? Can it handle unpaired or semi-paired settings?\n2. The ABG and transformer-based architecture might be computationally costly. Can the authors discuss this point?\n3. Can the proposed method generalize to other modalities beyond ATAC and RNA, such as methylation data, and what modifications would be needed?\n4. How sensitive is the method is to  to the negative sampling strategy as this can be related to robustness agains unpaired data, and does this affect its generalization?", "limitations": "The paper discuss the limitations of their method. I encourage the authors to revise this section, as reviewer feedback could help improve the limitations this section.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n* The paper is methodological work studying the problem of cross-modal matching in single-cell omics, which is important for both researchers and practitioners.\n* The paper is well written and easy to read. The challenges and limitations of existing literature are clearly stated and the proposed solutions are well motivated.\n*The experimental campaign validates the method performance and supports the presented claims.\n\n**Weaknesses**\n\n* The paper assumes access to paired ATAC and RNA data. However, it is well known that these omics require destructive sequencing techniques, so obtaining paired data is challenging. The paper does not discuss this limitation, which may affect the utility of the method.\n* Only two modalities are considered. While ATAC and RNA are popular, it would be interesting to see extensions of the method to other omics data.", "was_revised": true, "final_justification": "I give a score of 4 and recommend acceptance. The paper presents a well-motivated methodological contribution to the problem of cross-modal matching in single-cell omics, an important challenge in the field. It is clearly written, the limitations of prior work are well identified, and the proposed method is supported by strong experimental results. In particular, the new experiments demonstrating the method's effectiveness with a low number of paired omics samples are convincing. Emphasizing this strength more clearly would further improve the paper.", "rebuttal": "Dear Reviewer 7QNJ,\n\nThanks for your constructive and encouraging comments. Below we address your concerns.\n\n***\n\n## **W1 & Q1.1 & Q1.2 About the Unpaired Data**\nThank you for your feedback and valuable suggestions. \nWhile it is true that paired scRNA-seq and scATAC-seq data (by sequencing techniques) are required during training (which is necessary during training for almost all crossmodal integration models), our model is designed to be **highly data-efficient and generalizable**. By incorporating biological attributes as node representations and explicitly modeling crossmodal interaction patterns, our method requires fewer paired samples than previous approaches while achieving superior performance and strong generalization. To further address your concerns, we conducted experiments comparing our model with other baselines under different training set sizes on the ISSAAC-seq dataset:\n\n> Table I: Results for crossmodal matching task with different training sizes on ISSAAC-seq dataset.\n\nTraining Set Scale|60%|40%|20%|10%\n:-|:-:|:-:|:-:|:-:\nMultiVI|66.21|62.79|47.24|51.07\nCLUE|71.28|65.32|52.41|49.86\nscMoGNN|73.72|63.15|53.22|51.98\nBi2Former(Ours)|84.40|83.84|79.63|72.85\n\nAs shown in the table, VAE-based methods experience a notable performance drop when the amount of paired training data is reduced. In contrast, Bi2Former maintains strong performance even in low-resource settings, thanks to its biological attribute-aware design and its ability to capture fine-grained interaction patterns. We will expand the relevant discussion in the Limitation section of the revised version to clarify this point.\n\n## **Q1.3 Can it Handle Unpaired or Semi-paired Settings?**\nOur model is naturally compatible with both semi-paired and unpaired scenarios, and we have empirically validated its robustness and generalization in these settings.\n\nOur cross-cell-type generalization experiments (Section 5.3, Table 2) effectively simulate a **semi-paired setting**. During training, we utilize paired samples (along with corresponding negative samples) from only a subset of cell types. At test time, the model is evaluated on unseen cell types, whose data have no pairing information available in the training set.\n\nIn addition, we evaluate our model under a zero-shot cross-dataset setting, which is essentially an **unpaired setting**. Specifically, we pretrained the model on source datasets and evaluated it directly on different target datasets without any fine-tuning. This setup simulates a realistic deployment scenario where no pairing information is available during inference. The results are summarized below:\n\n> Table C: Zero-shot inference results for crossmodal matching task with training across various source dataset(s). The target dataset is ISSAAC-seq.\n\nSource dataset(s)|10xMultiome-PBMC|10xMultiome-PBMC+SHARE-seq|10xMultiome-PBMC+SHARE-seq+SNARE-seq\n:-|:-:|:-:|:-:\nMultiVI|47.82|48.19|51.24\nCLUE|51.07|49.67|52.93\nscMoGNN|54.74|61.02|58.89\nBi2Former(Ours)|71.14|79.16|80.03\n\nAs shown, our model outperforms all baselines significantly even when trained on a single source dataset (71.14% with one source vs. 58.89% with three sources). Moreover, as we incorporate more training data from additional source datasets, performance continues to improve, highlighting our model’s strong generalization capability. This improvement stems from the model’s ability to leverage biological attributes and learn biologically meaningful crossmodal interaction patterns, rather than relying solely on direct paired examples.\n\nThese results strongly support the applicability of our method to both semi-paired and fully unpaired crossmodal settings.\n\n***\n\n## **Q2 About the Computational Cost**\nAs detailed in Appendix D (Line 564-579), we provide a **theoretical analysis** of the time and space complexity of our model. Although Bi2Former computes attention matrices, the computational cost is significantly alleviated by two key strategies:\n- **Sparse Edge**: Attention is computed only on biologically meaningful edges, rather than across all node pairs.\n- **Node Reduction** via Expressed Features: The ABG include only the expressed RNA and ATAC features as nodes, which substantially reduces graph size and computation.\n\nThus, Bi2Former is computationally **more efficient than traditional VAE-based methods**. Specifically, the computational complexity of Bi²Former is $O((N_a + N_r)d^2 + Ed)$, which is more favorable compared to $O(Nd^2)$ for typical VAE-based methods (where $N$ is the total number of ATAC and RNA, typically tens of times larger than the number of expressed $N_r + N_a$ per graph).\n\nTo further address your concern, we also empirically report the training time per epoch (s/epoch) on the ISSAAC-seq and SHARE-seq datasets:\n\n||Time Complexity|Time(ISSAAC)|ACC(ISSAAC)|Time(SHARE)|ACC(SHARE)\n:-|-|-|-|-|-\nMultiVI|$O(Nd^2)$|218.7|66.21|732.1| 64.42 \nCLUE|$O(Nd^2)$|305.1|71.28|1031.9|63.21 \nBi2Former(Ours)| $O((N_a + N_r)d^2 + Ed)$|**121.2**|**84.40**|**365.3**|**79.84**\n\nAnalysis: Bi2Former has the outstanding performance while maintaining competitive running times.\n\n***\n\n## **W2 & Q3 Generalization to Other Modalities**\n### 1). Two modalities\nOur framework is **not limited to scRNA-seq and scATAC-seq**. The Bi²Former architecture and its biological-driven attention mechanism are general and can be directly applied to other modality pairs (e.g., RNA–Protein, ATAC–Methylation) for tasks such as matching, generation, or interpretability with only minor modifications. \nSpecifically, the only modifications needed are in the biological attributes of nodes and the edge construction strategy, depending on the modalities involved. The core model architecture remains unchanged, showcasing the flexibility and extensibility of our approach. To further address your concern, we also empirically report the RNA-Protein matching results:\n\n> Table F: The results of crossmodal matching task between RNA and Protein. We report ACC($\\uparrow$) as evaluation metric.\n\n||NeurIPS 2021 Multiomics\n:-|-\nMultiVI|66.32±1.02\nCLUE|74.97±1.53\nBi2Former(Ours)|84.21±0.48\n\n### 2). Three or more modalities\nOur framework can be naturally extended to handle three or more modalities with only minor modifications. When multiple modality types (e.g., RNA, ATAC, and Protein) are involved in the graph, we generalize the ABG into a heterogeneous attributed graph. In this setting, each modality is treated as a distinct node type, and edges are constructed to represent biologically meaningful relationships between different modality pairs. During model training, we perform relation-specific attention and aggregation for each edge type. This design enables Bi2Former to support a wide range of tasks including matching, generation, and interaction pattern discovery across any pair of modalities. We evaluated our model on the NeurIPS 2021 Multiomics dataset, the results for the crossmodal matching task are summarized below:\n\n> Table G: The results of crossmodal generation task on the NeurIPS 2021 Multiomics dataset. We report MAE($\\downarrow$) as evaluation metric.\n\n||RNA2ATAC|RNA2Protein\n:-|-|-\nCLUE|0.121|0.147\nBi2Former(Ours)|0.083|0.102\n\n***\n\n## **Q4. About the Negative Sampling Strategy**\nOur method is **robust to the choice of negative sampling ratio**. We have empirically observed that using a positive-to-negative ratio between 1:2 and 2:1 consistently leads to effective training and stable performance.\n\nAs illustrated in the table below, the model’s accuracy on both positive and negative samples remains stable across a range of sampling ratios. This indicates that the learned interaction patterns are not overly dependent on the sampling strategy and that the model captures meaningful crossmodal relationships.\n\n> Table J: Model performance on ISSAAC-seq dataset under different negative sampling ratios (e.g., 1:1, 1:2, 2:1).\n\n||ACC|Pos. Precision|Neg. Precision\n:-|-|-|-\n1:2|84.19±0.49|82.79±0.52|84.56±0.47\n1:1|84.40±0.48|84.75±0.42|83.17±0.51\n2:1|84.06±0.62|84.91±0.69|82.98±0.61\n\nWe will include this analysis in Appendix in the final version to clarify the robustness of our sampling strategy.\n\n***\nThanks again for your effort in reviewing our work. If you have any additional concerns or suggestions, please feel free to share them, and we sincerely value your feedback.\n\nSincerely,"}]

Author rebuttal:

Meta-review: This paper reframes single-cell cross-modal matching as graph classification on Attributed Bipartite Graphs (ABGs) and introduces Bi²Former, a biologically guided bipartite graph transformer that attends over ATAC–RNA pairs to capture putative regulatory interactions. The method delivers strong accuracy across multiple datasets, shows cross-cell-type and cross-dataset generalization (few/zero-shot), extends to RNA–protein and ATAC→RNA generation, and offers interpretable attention patterns consistent with cis-regulation and TF-binding signals. The model’s efficiency is supported by sparse attention on biologically plausible edges and node reduction to expressed features.

Concerns around novelty and task framing were substantively addressed. The authors positioned their contribution relative to scMoGNN and related GNN/transformer baselines, emphasizing per-cell attributed graphs and biologically driven attention that learn reusable interaction patterns; they added the missing baselines (scMoGNN, scGLUE, scJoint), a larger 10x brain 3k dataset, standardized metric comparisons (with high correlation to community matching scores), robustness to negative sampling, and runtime analyses showing competitive or better training time than VAE baselines. Biological validation—cell-type-specific attention structure and TF-binding alignment—was strengthened, and zero-shot/sparse-pairing studies convincingly demonstrate practical utility when paired data are limited.

Reviewer sentiment converged positively after rebuttal: htWu and KGd3 raised to weak-accept following clarifications and added experiments; 7QNJ remained positive, highlighting value in sparsely paired regimes; e9JF maintained reservations about the task’s utility, but did not engage in a discussion with the authors that have argued their case convincingly.
Decision: Accept (poster)