ChromFound: Towards A Universal Foundation Model for Single-Cell Chromatin Accessibiltiy Data
Authors: Yifeng Jiao
Yuchen Liu
Yu Zhang
Xin Guo
Yushuai Wu
Chen Jiang
Jiyang Li
Hongwei Zhang
LIMEI HAN
Xin Gao
Yuan Qi
Yuan Cheng
Venue: NeurIPS 2025 poster ()
Keywords: computational biology
scATAC-seq
cell representation
Primary area: machine_learning_for_sciences

Abstract: The advent of single-cell Assay for Transposase-Accessible Chromatin using sequencing (scATAC-seq) offers an innovative perspective for deciphering regulatory mechanisms by assembling a vast repository of single-cell chromatin accessibility data. While foundation models have achieved significant success in single-cell transcriptomics, there is currently no foundation model for scATAC-seq that supports zero-shot high-quality cell identification and comprehensive multi-omics analysis simultaneously. Key challenges lie in the high dimensionality and sparsity of scATAC-seq data, as well as the lack of a standardized schema for representing open chromatin regions (OCRs). Here, we present ChromFound, a foundation model tailored for scATAC-seq. ChromFound utilizes a hybrid architecture and genome-aware tokenization to effectively capture genome-wide long contexts and regulatory signals from dynamic chromatin landscapes. Pretrained on 1.97 million cells from 30 tissues and 6 disease conditions, ChromFound demonstrates broad applicability across 6 diverse tasks. Notably, it achieves robust zero-shot performance in generating universal cell representations and exhibits excellent transferability in cell type annotation and cross-omics prediction. By uncovering enhancer-gene links undetected by existing computational methods, ChromFound offers a promising framework for understanding disease risk variants in the noncoding genome. The implementation of ChromFound is available via https://github.com/JohnsonKlose/ChromFound.

ChromFound: Towards A Universal Foundation Model for Single-Cell Chromatin Accessibility Data
Yifeng Ji Yuchen Liu Yu Zhang", Xin Guo Yushuai Wu², Chen Jiang!?
Jiyang Li2, Hongwei Zhang Limei Han 12 Xin Gao2.345 Yuan Qil26, Yuan Cheng 1.2,"
Artificial Intelligence Innovation and Incubation Institute, Fudan University Shanghai Academy of Artificial Intelligence for Science
CComuter, Electrical and Mathematical Sciences and Engineering Division, KAUST
*Center of Excellence for Smart Health, KAUST
Center of Excellence on GenAI, KAUST
*Zhongshan Hospital, Fudan University
Correspondence to: (jaaoyifengggoxin|] saiis.rrg.n, [cheng_vaan.qiyunn ) @ fudan.edu.cn
Abstract
The advent of single-cell Assay for quencing (scATAC-seq) offers innovative perspective for deciphering regulatory mechanisms by assembling vast repository of single-cell chromatin accessibility data. While foundation models have achieved significant success in single-cell transcriptomics, there is currently no foundation model for ScATAC-seq that sup- ports zero-shot high-quality cell identification and comprehensive multi-omics analysis simultaneously. Key challenges lie in the high dimensionality and sparsity scATAC-seq data, as well as the lack standardized schema for representing open chromatin regions (OCRs). Here, present ChromFounde a foundation model tailored for ScATAC-seq. ChromFound utilizes hybrid architecture and genome-aware tokenization effectively capture genome-wide long contexts and regulatory signals from dynamic chromatin landscapes. Pretrained on 1.97 mil- lion cells from 30 tissues and 6 disease conditions, ChromFound demonstrates broad applicability across 6 diverse tasks. Notably, achieves robust zero-shot performance in generating universal cell representations and exhibits excellent transferability in cell type annotation and cross-omics prediction. By uncover- ing enhancer-gene links undetected by existing computational methods, Chrom- Found offers promising framework for understanding disease risk variants in the noncoding genome. The implementation of ChromFound available via http://iiiiiiiomoooooooonoooooee
Chromatin using se-
Introduction
The human genome harbors an extensive repository of open chromatin regions (OCRs) responsible for orchestrating precise spatial and temporal gene expression patterns [3I]. Identifying OCRs at single- cell level is pivotal for understanding the regulatory landscape of individual cells, offering profound
insights into cellular diversity and dynamic mechanisms of gene regulation [4 10] The single-cell Assay for transposase-Accessil prevalent genome-wil wide technique to enable the identification of OCRs at single-cell resolution. Its principal applications span cancer research, immunological studies, and neuroscience
Chromatin using sequencing (scATAC-seq) 4 stands as the most
[22
63
61 79
Equull contribution.
Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Furthermore, with initiatives such as the Human Cell Atlas 55 and other significant studies 17 83 the scale of ScATAC-seq data expanding rapidly, opening new possibilities and presenting novel challenges in the exploration of OCRs at the same time.
Pretrained language models (PLMs) have achieved great success in deciphering the complex language of single-cell RNA sequencing (scRNA-see -seq) data 164,1121 77], validating their capabilities of fine-tuning for a variety of downstream tasks. Nevertheless, the development of generalizable foundation models for ScATAC-seq remains under-explored, posing significant challenges for cellular representation learning and multi-tast transfer. Here, we identify and discuss three major challenges.
(I). ScATAC-seq data are inherently high-dimensional and sparse, with single cells spanning over millions of OCRs but typically less than 1% showing accessibility. This sparsity complicates the
utilization of Transformer [67] adopted in transcriptomics [64] 27 70 18 (77]. Other methods such
as transforming accessibility into gene activity [22 [9], filtering highly variable OCRs
22 62 20).
or binarizing accessibility 28 13 81] always cause substantial information loss 60)
(2). ScATAC-seq data are not formatted in a standardized feature space. Dynamic chromatin landscapes across diverse sources and platforms make dictionary- tokenization for genes and require
nucleotides unsuitable for OCRs. All deep learning models for ScATAC-seq [7] 72 dataset-specific training, leading to poor zero-shot performance.
(3). current methods lack a unified integration of genomic information with chromatin ac- cessibility profiles. Both genomic information defining regulatory loci and chromatin accessibil- ity profiles revealing their activity are essential in ScATAC-seq data. However, VAE-based mod- process accessibility matrices independently, while models like scBasset, CellSpace, and SANGO 81 62, 82) predict binary accessibility from genomic sequences. These methods struggle to generalize across diverse downstream tasks.
els [7 72 12
To address these challenges, we present ChromFound, a universal foundation model for single- cell chromatin accessibility data. Its hybrid architecture integrates Mamba block for efficient long-range sequence processing with self-attention block to capture local regulatory dependencies within +200 kb of transcriptional start sites, aligning with the typical range of enhancer- promoter mnhance--promoter interactions identified by Hi-C analysis [2] Chrom incorporates biologically informed OCR tokenization encoding genomic coordinates and non-binary accessibility profiles. This component ensures scalability and alignment across diverse scATAC-seq datasets from varied tissues, platforms, and protocols. Pretrained on over 1.97 million single cells from more than 30 organs or tissues and major disease categories (e.g., Alzheimer's, s. Parkinson's s. leukemia, glioma), ChromFound leverages 1.86 trillion training tokens, surpassing the scale of existing models such as Geneformer (64). scGPT M. and scFoundation [27], all of which use fewer than I trillion tokens.
To validate the effectiveness of ChromFound, we conduct comprehensive evaluations across multiple tasks and datasets. In cell representation, ChromFound outperforms baselines instead of additional training, achieving an average ARI improvement of 17.02% across 8 scATAC-seq datasets from tissues. Additional results demonstrate its robustness to denoise technical effect in zero-shot settings. For cell type annotation, ChromFound significantly improves accuracy and maero F1 score across all evaluated datasets and tissues. In cross-omics prediction, it accurately infers gene expression from chromatin accessibility,, surpassing all baselines across 5 datasets. Furthermore, in the K562 cell line, ChromFound effectively identifies gene-enhancer relationships and regulatory perturbation effects, highlighting its potential for interpretable cis-regulatory modeling.
Our main contributions can be summarized as follows:
(L) ChromFound, the first scATAC-seq foundation model, employs genome-wide architecture along with a genome-aware tokenization and is pretrained on a large corpus with 1.97 million single cells.
(2) ChromFound provides a strong zero-shot cell representation from an epigenetic perspective, as well as showing consistent robustness to data sparsity and batch effect.
(3) ChromFound shows great transfer learning capabilities in cell annotation and cross-omics predic- tion, validating biological significance in inferring enhancer-gene links and perturbation responses.
2

---

2 Related Work
Single-cell foundation models for scRNA-seq data have seen significant advancements, supporting tasks such as gene function prediction, cell annotation, and drug response modeling. Geneformer (64). scGPT [1. and scFoundation [27] are trained by large-scale datasets, while scBERT 777 and CellPLM [70] focus on specific objectives based on pre-training language models such as imputation and cell communications. However, existing models are predominantly tailored for ScRNA-seq data and fail to model SeATAC-seq data effectively. GET [18] achieves a level of predictive precision comparable to experimental replicates. It does so by integrating chromatin accessibility data and genomic sequence information in pseudo-bulk format, rather than relying on single-cell resolution.
Deep learning has been widely applied to ScATAC-seq data analysis for cell clustering and annota- tion. VI [ and Poisson VI 44] extend VAE-bbased frameworks, with the former disentangling biological from technical variation for batch correction and the latter introducing Poisson likelihood to preserve quantitative accessibility. SCALE 73 and SCALEX (72 employ VAEs for latent repre- sentation learning and multi-omic integration, while CASTLE [22 enhances cell-type annotation
via self-supervised learning. Beyond VAE-based methods, SnapATAC2 84] scales to millions of
cells using latent semantic indexing and spectral embedding, and Signac (59) provides Seurat-based
workflow for dimensionality reduction and multimodal integration. seBasset [8] links genomic
sequences with uccessibility profiles, and Cellcano 42 together with SANGO 82 employ two-step
MLPs for cell-type annotation. Despite their advances, these models remain limited by architectural constraints and data scarcity, restricting scalability and generalization.
3 Methods
3.1 OCR Tokenization
To encode chromatin accessibility data, we treat each OCR token by three essential components: chromosome embedding, positional embedding for genomic coordinates, and continuous accessibility embedding. The tokenization integrates these components to form a comprehensive representation of each OCR. Our unified token representation is designed to address two fundamental challenges. First, by encoding start and end genomic coordinates through sinusoidal positional embeddings, ChromFound captures the full extent of each OCR, accommodating varying fragment lengths. Second, by avoiding reliance on fixed OCR vocabulary, ChromFound flexibly represents tissue-specific or novel OCRs, thereby mitigating misalignment and dropout across tissues. In the following, we describe the encoding strategy for each component in detail.
Chromosome Embedding: Chromosome identity is encoded using a learnable embedding lookup table. The size of the chromosome embedding lookup table is 25, with 24 corresponding to the number of human chromosomes and designated for the padding token. We define the learnable embedding matrix W E R [C|x dassa and the chromosome index e, of OCR j. ChromFound maps the Cj to Eej by the following equation:
(I) Since some studies 33 76 have noted that state space models inherently handle sequential information through their recurrent nature, we use positional embedding to encode genomic coordinates instead of the sequence order. Specifically, the embedding of the starting position and the ending position for the OCR j is computed as:
E. = :.].,.].
Positional Embedding for Genomic Coordinates:
F
sin
sin
E
temp- 10000 74 mentte
E P.j
temp- 10000 2A/ Frodel
y? cos temp- (ampp1000022//com (andtl
cos temp-10000
where D3 and D represent the starting and end position for the OCR on the GRCh38 reference genome, temp is a hyperparameter set to 100000 in ChromFound, and dmuodet is the dimension of the embedding layer. The index i spans {0, 1.... dmodee/2), (2), with sine and cosine functions alternately assigned to even and odd dimensions of the embedding vector.
Accessibility Embedding: The accessibility value for the jth OCR, denoted as Xj. is continuous scalar processed by the procedures detailed in Appendix D To project the value X, into the hidden dimension, we use a linear transformation Ea Lin)
3

---

Model Architecture
Encoder Layer
Data Collection: -2.6 million cells from over 30 tissues
BOCR
OP
Accesibility Profiles
Chromosome identity
Genome Coodinates
FOCR
8
Pj
Lineur
Embedding
Positional Embeddding
POCR
E.
P.j
P.j
OCR Representation
BOCR.j
Expsa
Encoder Layer
Mamba Block
Emamma
Pooling
Projection
E Eout
Zero-shot Representation
(a) Cell representation
(a. 1) Denoising low count
(a.2) Denoising batch effect
Transfer Learning Usage
(b) Cell type
(c) Cross-omics prediction
Biological application
gene-enbanneer
sATAC--eq
SRNNA-ce
scTACC-e
parturlation response
Frozen
Tunable
Add
Figure 1: An overview of Chrom architecture. The methods of ChromFound are listed in
Section 3 OCR tokenization are described in Section 3.1 The hybrid encoder layer is described in
Section 3.2 Other pretraining details of datasets and implementation included in Section 3.3 All
evaluation results are detailed in Section
OCR Representation: The unified representation EOCR,j E R of the jth OCR is obtained by adding the previous embeddings together:
EOCR.j = Ees Ep + E. + Ea.j.
(3)
3.2 Hybrid Encoder Layer
The ChromFound encoder layer transforms the OCR representation EOCR E L.ID into a latent space, where L. denotes the sequence length and D is the embedding dimension. As illustrated in Fig. T ChromFound incorporates a self-attention module (Section 3.2.1 1) to effectively capture local dependencies among OCRs, and integrates Mamba layer (Section 3.2.2 to handle ultra-long OCR sequences. This architecture is motivated by biological insights, recognizing that both short- and long-range interactions among OCRs are critical to gene regulatory mechanisms (30 51 54]
3.2.1 Window Partition Self-Attention (WPSA)
Due to its computational complexity O(I2), the vanilla attention mechanism 667 not directly ap- plicable to ScATAC-seq data containing millions of OCRs. To address these challenges. ChromFound introduces an optimized self-attention layer named Window Partition Self-Attention (WPSA), which applies the attention mechanism within local windows of size W, where W < L.

---

Window Partition: Given the OCR representation EOCR E RIXDD L may not be multiple of W. We define the number of windows
and pad EOCR with zeros up to N W along the dimension L. As result, the padded sequence EOCR E R(N-W)x is reshaped as EOCR E IR NxWxD
Self-Attentoon: In practice, we set the window size to W -256, which approximately spans 200 kb upstream and downstream of transcriptional start site (TSS). This setting consistent with the scope of wwhanccr-promeeee promoter interactions suggested by high-resolution Hi-C analysis (2).
Window Restore: After applying self-attention within each window, we merge outputs of each window FOCR back into Eupsai
Eupsa- concat E% I NO R[N-W)x
(4)
Furthermore, we remove all zero padding tokens in Ewpsa and get the WPSA output Eupsn E RLXDD
3.2.2 Mamba Block
capable of processing extended sequences due to the linear-time complexity of state space models (SSMs) [26 23], making it particularly well suited for modeling high-resolution scATAC-seq data. When combined with the WPSA module, which amplifies local signals within predefined windows, the Mamba block enables effective integration of both short- and long-rang range dependencies among OCRs across the genome. This hybrid architecture mitigates the degradation of long-range information typically caused by relying solely zero-order discretization reduce parameter count and refine representations, we project Eupsa to a lower-dimensional space I.. Drww and then back, which is defined as:
Mamba 24
(50) 80). To
Edonn Eurpa W down: Emamba = Mamba (Faowa(
Eup- E Emmmba up:
(5)
where Wdown E Dx Diow (Diow < D) and Wup E Dlow
3.2.3
Overall Workflow
The overall workflow of the encoder layer is illustrated in Fig. Tad proceeded as:
E
EUCR + Wap Mamba
WWS. (RPSA(RMSNom(Eocc))
(6)
3.3 Pre-training
Datasets: We assemble a large-scale ScATAC-seq dataset comprising over 2.64 million cells detailed in Table For model pretraining, we select representative subset of 1.97 million cells from more than 30 distinct organs and tissues. To rigorously evaluate the generalization ability of ChromFound, additionally compile benchmark dataset containing 0.67 million cells from diverse sources, including tissues such as bone and retina excluded from pretraining.
ChromFound is trained for 5 epochs over 80 hours on compute cluster comprising 4 machines with total of NVIDIA A100 GPUs. The effective batch size set to 128. We use the AdamW optimizer (39 with maximum learning rate of 5 x 10-5, The embedding dimension and the hidden size of WPSA D are set to 128, while dimension Doow of the Mamba block is set to 32. The model architecture consists of 4 stacked encoder layers in total.
Implementation:
Training objective: Due to the high sparsity of scATAC-seq data, ChromFound adopts masking strategy that simultaneously targets both zero and non-zero OCR values. Inspired by x XTrimoGene [21]. the setting encourages the model to predict binary chromatin accessibility across the entire profile, mitigating the risk of learning representations dominated by zero entries. Poisson/ZINB losses often cause unstable optimization due to gradients dominated by nonzero entries. Instead, mean squared error (MSE) loss on log- srgrrradsforeed, normalized signals ensures stability. The MI. layer is employed as the decoder to reconstruct the original input X,. The reconstruction output of cell is denoted as X, The pretraining objective the MSE loss computed over the masked positions M, in each cell i:
Lisaeii
Σ HEM,
(x,
X,)
5

---

Table I: Results of cell clustering tasks. Result style: best, second best, relative gains.
Dataset
Tissue
Model
ARI(T)
FMI(1)
NMI(1)
AMI())
PeakVL
1
0.455840.0038
0.6286.0.
0.696440. 0012
0.6940=0.001
PoissonV11 48
0,367340.000
0.567340.0018
0.587810.0001
0.5845=0.0000
SnapATAC2
84
0.5118+0.0015
0.6740=0.0145
0.7202+0.0041
0.7179=0.0042
Signac
59
0,501 320.0507
0.7035=0.0177
0.558020.D0
0.5534=0.0165
1orabito 30K 48
Cortex
SCALE tk
0.5460+0
0.6996=0.0004
0.7282+0.001
0.7275=0.0016
(butch
SCALEX h2
0,4920=0.000 0022
0.6505=0.0013
0.6958+0.000)
0.6949=0.0000
CASTLE
12
0.442540.0029
0.6109:0.0019
0.708620.000
0.7077=0.0012
scBasset [8]
0.503910.00399
0.674710.0022
0.667410.0005
0.6647200.0005
ChromFound
0.6890=0.0387
0.7943=0.0144
0.7779:0.0037
0.7760=0.0038
Gains %)
26.20
15.17
6.81
6.66
PcakVI I
0551120000166
0.6394=0.0033
0,7050+0.00 0006
O. 7.7722000000 0006
PoissonVI 44
0,533410.000 0052
0.638810.0033
0.600
0 0.0852000000
SnipATAC2
84
0.53225000007
0.6380==0.0045
0.7038+0 0018
0. 0.7015=0.00 5.0.0018
Signac 39
0.5066920.0123
0.6478=#0.0064
0,588840. D033
0.5851=0.0003
Morabito 1 30K [8
(batch 2)
Cortex
SCALEX
SCALE 73
0.5306+0 0048
0.390S+00000
0.63944
0.5252+0
0,5361 (0028
0,6221+00 0007
0 0.6200==.00m 0007
0.5336=0.000
CASTLE
0,3743+0.00 0016
0.5087=0.0010
0,4636=0. 0.02]
0.4608=0.0022
scBusset
8
0.5573+0.002
0.6424==00015
0.6659#0. 0003
0. 0.6641=0 0003
ChromFound
0.6278:0000043
0.7156=0.0025
0.7321=0,0016
0.729920.0016
Gains %)
16.83
10.47
3.84
3.86
PcakVI 0
0,4159 0007
0.5434 0006
0.6024+0.000 0005
O 2.6000=0.00 0006
Poisson
0.3969+0
0.52322=0 0004
0.571 750 0001
0.5688=0.000 0001
SnapAIAC2
84
0.4571+000099
0.556420.006
0.6625#0.0013
0.6502=0.0013
Signac 59
0.4601=00000 #0.0028
D.52Y2200102X
0.5238=0.0009
Kuppe 139K 333
(donar av3)
Heart
SCALEX 121
SCALE 181
0.3819+0.0007
0.447540.0015
0 0.5654a0.00 0011
0.5061 +0 OODS
0,4829+0. 0000
0.600540.0003
0. 0.4797=0 0000
0 0.5978=0.0000:
CASTLE
0.5118=0..01 0.31%6+000011
0.4765+00
0. 0.4022=0000011
scBasset
[1
0.455340.0022
0.5729:0.0016
0.653910000006
0 0.6516=0.0006
Chrom
0,5828+010052
0.6757
0.7207=0.0008
0.7187=0.0008
Gains %)
27.75
17.94
8.78
10.31
PeakVi [T]
+0.0030
0.6383+0 0011
0.6149#0.
0.6144 0.022
Paisson 44
0.360640.0000
0. 0.5929 0027
0.447240. 0066
0.4464.0. 0066
SnupATAC2
84
0.5515+0.033
0.7285=0000 24
0.6867+0) 0060
0
0.6863=0.00 0060
Signac 50
0.115920.011 12
0.5093.0.005
0.2294 0076
Kuppe 139K 33
(donar av 10)
Heart
SCALEX W2
SCALE 13
0.474240.0100
0.4517+0.0088
0.6596=00 0004
0.6770=0.005
0.5397=0.000
0,504010..00
0070
0.5391=0.000 004
0.503320.008 008D
CASTLE
12
0.4739:0 00000
0.678620 0.000
0.520410.00
0.5197=0.000 00000
scBusset
81
0.582330.011
0.7459=0.0050
0.6780+0.00 0000
0.6776=0.00 0.00
Chrom Found
0.677420.0445
0.10991000169
0.736910.0137
0.7365=0.0136
Gains %)
16.34
8.71
7.31
7,31
PeakVT f
0,2885+0.0022
0.554320.0015 0019
0.564420.000
0.5638=0.000
Paisson 24g
0.316240.002
0.576620..00 0017
0.6081+0.0017
0. 0.6076=0.000 0017
SmapATAC2
8
0.560610
0.7207 1
0.704510..00 0085
0.7041
Signac
59
0.55.820.10ST
0.7451+0.0000
0.4931+0.055 0535
0.4919=0.055 0533
(sample D32)_13)
Liang154K [35]
Retina
SCALEX
SCALE
73
h2
0.5069+0
0.5512+0.0386
0.7285
0.7241 +0 0022
0.7172+0.000
0.6759H0.000
0.7168=0.0051
0'6755=0:0005
CASTLE
0.2424+0.00FF
0.5136=0
0.5298+0. D0.3
0. 0.5242=0.0003
scBusset
[8
0.4330+0.000 0049
0.6699=0.0029
0.6903+0.0015
0.6898=0.0015
ChromFoune
0.6668+0.0500
0.814990.0119
0.7644+0.0099
0.7641=0.0093
Gains %)
18.94
9.37
6.59
6.60
PeakVI 1
n.4702+0.006)
0.6318=00.0037
D. 0.7229=0.D022
0. 0.7224=0.0025
Paisson 2g
0.497840.0122
0.6527 0066
0.7400+0.0022
0.7395=0.00 0028
SnapATAC2
0.581410.0011
0.7173a0.001
0.788 178811000011
0.7877=0.001
Signac 59
0.514620.00 0.002
0.7339=0.0119
0.5235+0.0300
0.5222=0.033
(sample D19D008)
Liang154K (35]
Retina
SCALEX
SCALE tkl
72
0,342420004T
0.61 0.6129H0.013 31
0:687220.0023
0.7465=0.0113
D.69 7019
0.7981
O. 0..95%50.005K
0.6971=00001 =00000
CASTLE E
0.427240.0052
O. 0.5982=0 0034
0.661 4+0. 0034
0 66666==0000 0034
ScHasset ST
0.612740.01.99
0.7313=+0.0066
0.78%1 +00000000000
0.7878=0.00110
Chrom Found ound
0.668840.0264
0.7767=0.0123
0.8183460.0033
0.8179#0.0033
Gains %)
9.12
4.05
2.53
2.34
PeakVT f
0.541720.0022
0.634020.0015
0.663910.00 0009
0.6627=0.0000
PoissonV 44
0.6314+0.000
0.7073=00.001 8
0.735840.0012
0.7345=0.0011
SnupATAC2
84
0.6246+0 01 12
0.7085 0067
0.7551=0. 0023
0.7541=0.000 0023
Signac
0.183220.00
0,4543+000043
0,3064+00.
0. 0.3027=0.00 00.001
(batch VIB (1___[
PBMC169K LES
PBMC
SCALEX b2l
SCALE tkl
0.6158+0.0088
0.612110.0048
0.6959=01.0031
0..027
0.759310.00 D008
0.763244..0 0010
0. 0.77230000010
0.758410.0008
CASTLE
[2]
0.553010.000
0.643910.001
0.726310.0000 0004
0.7252=0.000 0004
scBasset
[ST
0.6240H0
0.7019=0.000
0.7660H0
0. 7651==000000
ChromFound
0.695320.0035
0.7601=0.0023
0.786020.0007
0.7852=0,0008
Gains %)
10.12
7.28
2.61
2.62
PeakVI 11
0.201550.0005
0. 0.5041 =0.000000 0004
0.4623+0.000 0004
0.4615=0.00 0005
PoissonVI 44
n.4103+0.0045
0 5994=1100028
0.5656=0.D003
0. 0.56.99=0.003.
0.005
SmaAATA22 84
0.412640.00
0.6149 20.99990199
0.5761
0 0.5754=0.0077
Signae
0.1885110.00m
0 0.5249 0006
0.285210:0000
0 000
(batch B1O __ddeeq__)
PBMC169K LS
PBMC
SCALEX
SCALE
0.408740.0119
0.4359x0.013
0.60652110110
0.6250=0.0100
0,559420.000 00.45
4.$51910.0066
0.5587=0.0045
0.551240.0066
CASTLE
0.3504+0..00 0042
0.5555=0.000
01512+000011
0. 0.550==0.0013
scBasset [T
0.4362400001
0.6223+40.0 0008
0.5577+0. 0008
0. 0.5571=0.0000
ChromFound
0.4835=0.0082
0.6604=0.0042
0.5950=0.0036
0.5944=0.0036
Gains %)
10.84
5.66
3.28
3.30

---

Experiments
To systematically evaluate the performance of ChromFound, we conduct experiments on various tissues and datasets. We first assess its zero-shot representation capabilities, demonstrating strong robustness to both data sparsity and batch effects. We then evaluate the transferabilityy in downstream tasks, including cell type annotation and cross-omics prediction. Finally, we validate the biological utility of ChromFound by accurately inferring gene-enhancer regulatory links and gene expression responses to enhancer perturbations. All ScATAC-seq datasets used in experiments are preprocessed according to the procedures detailed in Appendix D
4.1 Cell representation
ChromFound generates low-dimensional cell representations by applying PCA to the encoder outputs after the pooling layer. Details of the implementations are provided in Appendix B.I For comparison, we include scBasset [8]] and three VAE-based models [73] 72 [2]. which require self-supervised learning on each specific dataset. We conduct experiments on eight datasets from four tissue types and evaluate performance using four clustering metrics ensure fair and reliable comparison.
All results presented in Table D reveal that ChromFound outperforms existing SOTA methods in all tissues and metrics, with average improvements of 17.02%, 10.39%, 6.72% and 6.69% in ARI, FMI, NMI and AMI, respectively. The greater improvements observed in ARI compared to NMI and AMI infer the robustness of ChromFound to technical noise such as low read depth and batch effects, as ARI is particularly sensitive to local structures within the cell representation space. We discuss ChromFound's 's ability to denoise low count and batch effect below.
have acknowledged that the prevalence of missing signals in ScATAC-seq makes standard analyses challenging. Although all datasets in [ exhibit high sparsity (-99%) we further simulate increasingly sparse conditions by downsampling accessible OCRs to 10%, 20%, 30%, 40% and 50% of their original counts. We adopt the same benchmarking methods and clustering metrics as listed Table [ As shown in Fig. 2 ChromFound consistently maintains stable performance across all levels of downsampling. with its advantage over baseline methods becoming increasingly pronounced as the sparsity intensifies. Furthermore, observe that ChromFound performs better on the retina dataset under stronger downsampling. This improvement likely reflects protocol-specific artifacts, as retina data contain more frequently accessible OCRs, introducing redundancy and noise. Random downsampling suppresses these excessive signals, enhances the signal-to- to-iils-- ratio, and yields the observed performance gain.
Denoising low count: Some studies 6
291
Figure 2: Results of denoising low count. The left y-axis shows the absolute metric values (scatter plot) and the right y-axis indicates the relative gains (%) over the recond-besy methods (bar plot).
7

---

Denoising batch effect: Denoising batch effects remains a major challenge in ScATAC- analy- sis [4 We evaluate ChromFound on datasets from 4 tissues using biological conservation
metrics 41 consistent with scGPT [1]. In addition to Peak [ Poisson VI
[4].
SCALEX [72
and scBasset [1, we include five specialized batch integration methods 33
29
32
74
37 for
comparison. As shown in T ChromFound in zero-shot settings outperforms existing methods
by more effectively mitigating batch effects while better preserving biologically significant variation.
Table Results of denoising batch effect. Result style: best, second best, relative gains. AvgBIO is the average of cRI NMI and ASWeel cell measuring biological consistency. AvgBATCH is computed as the average of ASW batch and GraphConn to summarize the batch mixing performance.
Bone To326K 665
Kuppe139K
PBMC 169K [5]
Corfex Morabito
Model
batch 43/44
dosar av 3/vv00
HAR ddseq 1/2
batch 1/2
AvgBIO(1)
AvgBATCH(1)
AvgBIO(T)
AvgBATCH(1)
AvgBlO(1)
AvgBATCH(1)
AvgBIOK")
1vgBATCH(1)
scV1
[K
0.3713
0.9026
0.7303
0,8448
0.6101
0.7932
0.7238
0.9217
PcakV[ 0
0.2981
0.9173
0.5657
0.7564
0.6172
0.7889
0.7256
0.9254
PT
0.2936
0.9128
0.5622
0.8081
0.7240
0,9279
Scanorama
20
0.4386
0,8722
0.5320
0,7779
0.6036
0.7964
0.6895
0.9243
Harmony 1
0.3789
0.8558
0.5119
0,8390
0,6232
0.7923
0.7152
0.9101
scANV1 741
0.4351
0.8479
0.6810
0,8401
0.6028
0,8035
0.7185
0,9208
Liger (37)
0.1992
0.6471
0,7286
0,8019
0,6051
0,8009
0.6822
0,9208
SCALEX 72
0.9177
0,8004
0,4789
0.8728
scBasset 81
0.3650
0.9208
0.6216
0.8591
0.6024
0.8014
0.7045
0.9301
ChromFound
0.6408
0,9289
0,8180
0.8679
0,6443
0.8217
0,7440
0,9565
Gain( %)
46.07
0.88
7.69
1.03
3.37
2.25
2.78
2.84
4.2 Cell Type Annotation
Computational cell type identification a fundamental task in single-cell omics analysis. To adapt ChromFound for this task, introduce feature projection head followed by MLP layers as described in Appendix B.2 Our methods are compared to three strong baseline models designed for cell type annotation of ScATAC-seq. Cellcano 42 EpiAnno 7 and SANGO [82]. The variant of ChromFound trained from scratch is also included to assess the contribution of pretraining. Fig. Bl highlights that ChromFound consistently outperforms all benchmark methods, achieving the highest performance in macro F1 score and demonstrating its ability to accurately annotate diverse cell types. The confusion matrices on PBMC169K (15) for ChromFound and CeliCano 442 shown in Fig. [ The results indicate that Cellcano struggles to correctly classify rare cell types such as natural killer cells, CD16* monocytes, and dendritic cells, likely due to their low abundance. In contrast, ChromFound significantly improves the classification accuracy for these cell types, enabling a more precise understanding of immune cell composition and dynamics.
Figure 3: Results of cell type annotation, evaluated by Accuracy (left) and macro Fl Score (right).
4.3 Cross-omics Prediction
Although recent advances have enabled the simultaneous profiling of multiple assays, most single- cell datasets remain isolated, posing significant challenges for effective multi-omics analysis. To explore the potential of ChromFound in multi-omics prediction, we utilize ScATC-see data as the source modality and scRNA-seq data the target modality. Implementation details are provided in Appendix B.3 For comprehensive evaluation, we benchmark on five paired SeATAC-seq and scRNA- seq datasets
86 65
40 5 87] using two standard correlation metrics. Three representative bascline
8

---

methods 71 78 69 are included with default configurationss from the DANCE toolkit [6]. As shown in 3 ChromFound consistently outperforms all baselines across datasets in both PCC and CCC, highlighting its strong capability in cross-omics prediction. These results highlight the potential of ChromFound to facilitate biological analyses such as inferring gene-enhancer relationships and predicting transcriptional response to enhancer perturbation. The two biological applications are discussed in detail below.
Table 3: Cross-omics prediction results. Result style: best, second best, relative gains. ChromFound stands for the version of ChromFound without loading the pretrained checkpoint.
Cortex Zhw-55K PCC(1>
Bone
BMMC
BMMC
Cell lines
Model
To326K
65
2021 PCC(T)
30
alac2gox
2022 S CCC(T)
Zhul 87
CCC(t)
PCC(T)
CCC(*)
CCC(T)
PCCT
PCC(T)
CCC(t)
BABEL 711
0.7975
0.7687
0.8020
0.7223
0.3854
0.3695
0.8901
0.8329
0.9196
0.8608
0.7973
0.7525
0.7986
0.6685
0.3435
0.8983
0.7990
0.8539
scMM 477
0.7793
0.7325
0.7586
0.6513
0.3718
0.2983
0.8321
0.7027
0.8587
0.8062
ScMoGNN
69
0,8001
0.7210
0.7957
0.7233
0.4168
0.3911
0.9124
0.8655
0.7976
0,7263
ChromFound
0.7864
0.7674
0.7887
0.6919
0,4213
0. 0.3989
0.9279
0.8704
0.8992
0.8501
ChromFound
0.8316
0.8064
0.8304
0.7472
0.4249
0.4032
0.9293
0,8818
0.9449
0,9071
Gain( %)
4.10
4.67
3,54
3,30
1.94
3.09
1.85
1,88
2.67
5.11
Biological Application: Predicting enhancer-gene link and perturbation response
Enhancer elements in the human genome harbor thousands of genetic variants that regulate gene expression and contribute to the risk of common diseases [85]. In our approach detailed in Ap- pendix [ we simulate OCR knockdown and estimate the resulting changes gene expression. The absolute change reflects the likelihood of an enhancer-gene regulatory link, while the relative change captures the direction and strength of the transcriptional response. We perform experiments on 141 K562 cells from the test split in Zhul IK [87). using the cross-omics model trained evaluated in Table 3 Fulco4K [87] provides the ground truth of enhancer-gens links and quantitative effects through CRISPRi perturbations in K562 human erythroleukemia cells. For clarity, the evaluation focuses on the COPZI and HNRNPAI genes, which are involved in cancer-related processes such as stress adaptation and RNA splicing regulation [58] g BABEL [7 and CMAE (78) are included as benchmark methods due to their strong performance in cross-omics prediction in Zhul 1K (8) As shown in Fig. 4{a) ChromFound achieves the highest AUC scores with 0.77 for COPZI and 0.61 for HNRNPA1, 1, accurately identifying enhancer- gene links. Fig. [bb further shows that ChromFound better predicts transcriptional responses to enhancer perturbations. In contrast, BABEL 771 and CMAE (8) filter for the top 10,000 highly variable OCRs [16.. leading to zero predictions of most enhancer perturbations. This limitation is evident in both Fig 4(a) and Fig 4(b) highlighting the advantage of ChromFound's genome-wide modeling to support a more comprehensive understanding of regulatory mechanisms.
[
00P23
18.
SLRR OF Co
2.0
64 60
(a)
(b)
Figure 4: (a) ROC curves illustrate the performance of enhancer-gene link prediction, where true positive rates are plotted against false positive rates across varying prediction thresholds. Ground truth labels are defined based on the significance (FDR < 0.05) of expression changes after enhancer perturbation. (b) Scatter plots represent a specific enhancer perturbation. The y-axis shows the predicted scores of average gene responses of enhancer perturbation, while the x-axis represents the real effects of post- post-perturbation. Both magnitudes are rescaled to the unit norm.

---

4.4 Ablation study
In this section, we conduct an ablation study to address three questions using the PBMC169K [1]] sampled from batch VIB 1B_0xv1111. The results of cell representation are summarized in Table 4
Question Does our proposed ChromFound architecture contribute to performance?
We evaluate the individual impact of each key module in our model, including genome-aware tokenization (Section 3.1), the WPSA layer (Section (.2.T) and the Mamba block (Section (32.2). In
rows 2-4 of Table I we observe a clear performance drop in all clustering metrics compared to row 1, demonstrating that each component contributes complementarily to overall performance.
Question 2 Does scaling to large-scale data provide performance gains?
As shown in rows 5-6 of Table 4 reducing the data size from 1.97 million to 0.2 million and 20 thousand cells leads to consistent degradation in all evaluation metrics, confirming that large-scale data learning facilitates better generalization.
Question 3: Is long-context modeling necessary for high-quality representations?
As seen in rows 7-8 of T reducing the input length by half and by a quarter significantly hurts performance, highlighting the importance of long-context modeling for dynamic OCR landscapes.
Table 4: Ablation study. Red row indicates the full model of ChromFound. The other colored rows correspond to the topics discussed Section 4.4
Genome info
WPSA
Mambu
Data size
(million)
# of OCRs
per cell
FLOPS(10e9)
per cell
ARI(1)
FMI(1)
NMI(1)
AM
1.97
440000
19.72
0.695320.0035
0.7601=0.0002
0.786020.0007
0.7852:0.0000
1.97
440000
19.72
0.6452H0.0149
0.709510:0098
07221
0.7301=0.0017
1.97
440000
19.72
0.5897+0.0086
0.0001+000113
0.6419=0
0.6476+00.00 0015
1.97
440000
19.72
0.3075100.019
0.309820.00011
0.3101:0.0011
0.3326:10.0033
440000
19.72
0.653910.000 0057
0.726120.0037
0.759320.0012
0.758310 0012
0.02
440000
19.72
0.614240.0053
0.699560.0034
0.7354=0.000 0015
0.7345=0.0015
1.97
220000
9.87
0.401240.0047
0.450000.0031
0.4535=0.00 0007
0,44
1.97
110000
4,93
0.327620.0051
0.358920.0048
0.367420.0002
0.366810.0002
Conclusion
In this work, we present ChromFound, the best of our knowledge, the first foundation model specifically for ScATAC-seq data. To address the inherent challenges of high-dimensionallty. sparsity, and dynamic chromatin landscapes, we utilize a hybrid architecture that integrates short- and long- range dependencies with a genome-aware tokenization. Trained on 1.97 million single-cell profiles comprising over 30 human tissue types and disease categories, ChromFound achieves SOTA performance in 6 downstream tasks, showing great robustness in zero-shot cell representation. Its biological significance further demonstrated by its ability to accurately infer transcriptional responses to enhancer perturbations. In the future, we aim to explore the broader applicability of ChromFound in the large-scale mapping of enhancer-gene regulatory interactions, critical yet largely uncharted component of the cis-regulatory landscape of the human genome.
6
Acknowledgments
The authors gratefully acknowledge the following funding sources: AI for Science Program, Shanghai Municipal Commission of Economy and Information, National Natural Science Foundation of China (Grant No. 82394432, 92249302), Shanghai Municipal Science and Technology Major Project (023SHZDZX022,, the King Abdullah University of Science and Technology (KAUST) Office of Research Administration (ORA) under Award No REI/1/5234-01-00 REI/1/5414-01-001 REI/1/5289- URF/1/4663-01
01-01, RE1/1/5404-01-01, REI/1/5992-01- Health (KCSH) under award number 5932, and Center of Excellence on Generative Al under award number 5940. The computations in this research were performed using the Computing for the Future at Fudan (CFFF) platform of Fudan University.
Center of Excellence for Smart
10

---

References
[1] Tal Ashuach, Daniel Reidenbach, Adam Gayoso, and Nir Yosef. Peakvi: A deep generative model for single-cell chromatin accessibility analysis. Cell reports methods, 2(3), 2022.
[2] Boyan B. Bonev and Giacomo Cavalli. Organization and function of the 3d genome. Nature Reviews
Genetics, 17:661-678, 2016.
[3] Jason D Buenrostro, M Ryan Corces, Calch A Lareau, Beijing Wu, Alicia N Schep, Martin J Aryee, Ravindra Majeti, Howard Y Chang, and William J Greenleaf. Integrated single-cell analysis maps the continuous regulatory landscape of human hematopoietic differentiation. Cell. 173(6):1535-1548 2018.
[4] Jason D Buenrostro, Beijing Wu, Ulrike M Litzenburger, Dave Ruff, Michael L Gonzales, Michael P Snyder, Howard Y Chang. and William Greenleaf. Single-cell chromatin accessibility reveals principles of regulatory variation. Nature, 523(7561) ):486-490, 2015.
[5] Daniel Burkhardt. Malte Luecken, Andrew Benz, Peter Holderrieth, Jonathan Bloom. Christopher Lance. Ashley Chow, and Ryan Holbrook. Open problems multimodal single-cell integration. hattps://kagg
1e. co.com/cco coom/competiiinn/ /open- problems mul t timodal, 2022. Kaggle.
[6] Huidong Chen, Caleb Larcau, Iommaso Andreani, Michael E Vinyard, Sara P Garcia, Kendell Clement, Miguel A Cedrade-Navarro, Jason D Buenrostro, and Luca Pinello. Assessment of computational methods
for the analysis of single-cell atac-seq data. Genome biology, 20:1-25, 2019.
[7] Xiaoyang Chen, Shengquan Chen, Shuang Song, Zijing Gao, Lin Hou, Xuegong Zhang, Hairong Lv. and Rui Jiang. Cell type annotation single-cell chromatin accessibility data via supervised bayesian embedding. Nature Machine Intelligence, 4(2):1 16-126, 2022.
[8] Cynthia V Clower, Deblina Chatterjee. Zhenxun Wang, Lewis C Cantley Matthew G Vander Heiden, and Adrian R Krainer. The alternative splicing repressors hampp al/a2 and ptb influence pyruvate kinase isoform expression and cell metabolism. Proceedings of the National Academy of Sciences, (07(5))1894-11)) 2010.
[9] M Ryan Corces, Jason D Buenrostro, Beijing Wu, Peyton G Greenside, Steven M Chan, Julie L Koenig, Michael P Snyder, Jonathan K Pritchard, Anshul Kundaje, William J Greenleaf, et al. Lineage-specifie and single-cell chromatin accessibility charts human hematopoiesis and leukemia evolution. Nature genetics, (8(10)11933 1203, 2016.
[10] M Ryan Corces, Anna Shcherbina, Soumya Kundu, Michael Gloudemans, Laure Frésard, Jeffrey M Granja, Bryan H Louie, Tiffany Fulalio, Shadi Shams, S Tansu Bagdatli, et al. Single-cell epigenomic analyses implicate candidate causal variants at inherited risk loci for alzheimer's and parkinson's 's diseases. Nature genetics, 52(11))1158-116
2020.
[11] Haotian Cui, Chloc Wang. Hassaan Maan, Kuan Pang, Fengning Luo, Nan Duan, and Bo Wang. segpt: toward building a foundation model for single-cell multi-omics using generative ai. Nature Methods, pages 1-11, 2024.
[12] Xuejian Cui, Xiaoyang Chen. Zhen Li, Zijing Gao. Shengquan Chen, and Rui Jiang. Discrete latent embedding of single-cell chromatin accessibility sequencing data for uncovering cell heterogeneity. Nature Computational Science, pages 1-14, 2024.
[13] Darren A Cusanovich, Riza Daza, Andrew Adey, Hannah A Pliner, Lena Christiansen, Kevin L Gunder- son, Frank J Steemers, Cole Trapnell, and Jay Shendure. Multiplex single-cell profiling of chromatin accessibility by combinatorial cellular indexing. Science, 348(6237):910-))
2015.
[14] Darren A Cusanovich, James P Reddington, David A Garfield, Riza M Daza, Delasa Aghamirzaic, Raquel Marco-Ferreres, Hannah Pliner, Lena Christiansen, Xiaojie Qiu, Frank J Steemers, al. The cis- regulatory dynamics of embryonic development at single-cell resolution. Nature, 555(7697):538-544 2018.
[15] Florian V De Rop. Gert Hulselmans, Chris Flerin, Paula Soler- Vila, Albert Rafels. Valerie Christiaens, Carmen Bravo González-Blaz Domenica Marchese, Ginevra Caratu, Suresh Poovathingal, ct al. Systematic benchmarking of single-cell atac-scquencing protocols. Nature biotechnology, 42(6)916-999 9216, 2024. [16] Jiayuan Ding, Hongzhi Wen, Wenzhuo Tang, Renming Liu, Zhaoheng Li, Julian Venegas, Runze Su, Dylan Molho, Wei Jin, Wangyang Zuo, et al. Dance: A deep learning library and benchmark for single-cell analysis. bioRxiv. pages 2022-10. 10, 2022.
[17] Silvia Domeke, Andrew J Hill, Riza Daza, Junyue Cao, Diana R O'Day, Hannah A Pliner, Kimberly Aldinger. Dmitry Pokholok, Fan Zhang, Jennifer H Milbank, et al. A human cell atlas of fetal chromatin accessibility. Science, 370(65 1 :))eaba7612 2020,
[18] Xi Fu, Shentong Mo, Alejandro Buendia, Anouchka P Laurent, Anqi Shao, Maria del Mar divarz--Torres Tianji Yu, Jimin Tan, Jiayu Su, Romella Sagatelian, et al. A foundation model of transcription across human cell types. Nature, pages 1-9, 2025.
[19] Charles P Fulco, Joseph Nasser, Thouis R Jones, Glen Munson, Drew T Bergman, Vidya Subramanian, Sharon R Grossman, Rockwell Anyoha, Benjamin R Doughty, Tejal A Patwardhan, et al. byt contact model enhancer promoorr regulation from thousands of crispr perturbations. Nature genetics. 51(12):1644-166 2019.
[20] Shila Ghazanfar, Carolina Guibentif, and John C Marioni. Stabilized mosaic single-cell data integration using unshared features. Nature biotechnology. 42(2):284-292, 2024.
[21] Jing Gong, Minsheng Hao, Xingyi Cheng. Xin Zeng. Chiming Liu, Jianzhu Ma. Xuegong Zhang, Taifeng Wang, and Le Song. xtrimogene: an efficient and scalable representation learner for single-cell rna-se data. Advances in Neural Information Processing Systems, 36, 2024.

---

[22] Jeffrey M Granja, M Ryan Corces, Sarah E Pierce. S Tansu Bagdatli, Hani Choudhry. Howard Y Chang. and William J Greenleaf. Archr is a scalable software package for integrative single-cell chromatin accessibility analysis. Nature geneticx, 53(3):003-411 2021.
[23] Jeffrey M Granja, Sandy Klemm, Lisa M McGinnis, Arwa S Kathiria, Anja Mezger, M Ryan Corces, Benjamin Parks, Eric Gars, Michaela Liedtke, Grace XY Zheng. et al. Single-cell multiomic analysis identifies regulatory programs in mixed-phenotype acute leukemia. Nature biotechnology, 37(12):1458
1465, 2019.
[24] Albert Gu and Tri Dao. Mamba: Lincar-time sequence modeling with selective state spaces. arXiv preprint
S.tv:23/2.007552 2023.
[25] Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state
spaces. arXiv preprint arXiv:21 11.22/11100966 2021.
[26] Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré. Combining recurrent, convolutional, and continuous-time models with linear state space layers. Advances in neural
information proessing systems, 34:572-585, 2021.
[27] Minsheng Hao, Jing Gong, Xin Zeng. Chiming Liu, Yucheng Guo, Xingyi Cheng, Taifeng Wang. Jianzhu Ma, Xuegong Zhang, and Le Song. Large-scale foundation model on single-cell transcriptomics. Nature Methods, pages 1-11, 2024.
[28] Yuhan Hao. Stephanie Hao, Erica Andersen-Nissen. William M Mauck, Shiwei Zheng, Andrew Butler, Maddie J Lee, Aaron J Wilk, Charlotte Darby, Michael Zager, et al. Integrated analysis of multimodal single-cell data. Cell. 184(13)33573-333)
2021.
[29] Brian Hie, Bryan Bryson, and Bonnie Berger. Effficient integration of heterogeneous singlc-cell transcrip-
tomes using scamorama. Nature biotechnology, 37(6):685-691, 2019. [30] KyunK Kim and Ramin Shiekhattar. Architectural and functional commonalities between enhancers
and promoters. Cell. 162(5):948-95))) 2015.
[31] Sandy I.. Klemm, Zohar Shipony, and William James Greenleaf. Chromatin accessibility and the regulatory
epigenome. Nature Reviews Genetics, 20:207 220. 2019.
[32] Ily Korsunsky, Nghia Millard, Jean Fan, Kamil Slowikowski. Fan Zhang, Kevin Wei, Yuriy Baglaenko, Michael Brenner. Po-ru Loh, and Soumya Raychaudhuri. Fast, sensitive and accurate integration of single-cell data with harmony. Nature methods, (6(12): 111122991299 16(12):1899-129, 2019.
[33] Christoph Kuppe, Ricardo O Ramirez Flores, Zhijian Li, Sikander Hayat, Rebecca T Levinson, Xian Liao, Monica T Hannani, Jovan Tanevski, Florian Wünnemann, James S Nagai, et al. Spatial multi-omic map of human myocardial infarction. Nature, 6087792))776-777 2022.
[34] Zhijian Li. Christoph Kuppe, Susanne Ziegler. Mingbo Cheng. Nazanin Kabgani, Sylvia Menzel, Martin Zenke, Rafael Kramann, and Ivan G Costa. bbroomati--accessiiiiyyy estimation from single-cell atac-seq data with scopen. Nature communications, 12((1)6388 2021.
[35] Qingnan Liang, Xuesen Cheng. Jun Wang. Leah Owen, Akbar Shakoor, John L Lillvis, Charles Zhang, Michael Farkas, Ivana K Kim, Yumei Li, et al. A multi-omis atlas of the human retina at single-cell resolution. Cell genomics, 3(6), 2023.
[36] Chengkai Liu, Jianghao Lin, Jianling Wang, Hanzhou Liu, and James Caverlee. Mamba4rec: Towards efficient sequential recommendation with selective state space models. arXiv preprint arXiv:2403.0.990 2024.
[37] Jialin Liu, Chao Gao, Joshua Sodicoff, Velina Kozareva, Evan Z Macosko, and Joshua D Welch, Jointly defining cell types from multiple single-cell datasets using liger. Nature protocols, 15(1 15311)33632-362. 2020.
[38] Romain Lopez, Jeffrey Regier, Michael B Cole, Michael Jordan, and Nir Yosef. Deep generative modeling
for single-cell transcriptomics. Nature methods, 15(12):1053-1000 1058, 2018.
[39]
Loshchilov. Decoupled weight decay regularization. arXiv preprint 17X
2017.
[40]
Malte Luecken, Daniel Bernard Burkhardt, Robrecht Cannoodt. Christopher Lance, Aditi Agrawal,
Hananeh Aliee, Ann T Chen, Louise Deconinck, Angela M Detweiler, Alejandro A Granados, et al. A sandbox for prediction and integration of dna, ma, and proteins in single cells. In Thirty-fifth conference neural information processing systems datasets and benchmarks track (Round 2), 2021.
[41] Malte D Luecken, Maren Büttner, Kridsadakorn Chaichoompu, Anna Danese, Marta Interlandi, Michaela F Müller, Daniel C Strobl, Luke Zappia, Martin Dugas, Maria Colomé-Tatché.
al. Benchmarking atlas-level
data integration in single-cell genomics. Nature methods, 19(1):41-50 50, 2022.
[42] Wenjing Ma, Jiaying Lu, and Hao Wu. Cellcano: supervised cell type identification for single cell atac-scq data. Natue Communications, 14(1):1864, 2023.
[43] Laura D Martens, David S Fischer. Vicente A Yépez, Fabian Theis, and Julien Gagneur. Modeling fragment counts improves single-cell atac-seq analysis. Nature Methods, 21(1))28-3) 31,,-31,2210011 2024. S Fischer, Vicente A Yépez, Fabian Theis, and Julien Gagneur. Modeling fragment counts improves single-cell atac-seq analysis. Nature Methods, 21(1):28-31, 2024.
[44] Laura D Martens,
[45] Qiuchen Meng. Xinze Wu, Wenchang Chen, Yubo Zhao, Chen Li, Zheng Wei, Jiaqi Li, Xi Xi. Sijie Chen, Catherine Zhang, et al. A generic reference defined by consensus peaks for scatac-seq data analysis. bioRxiv, pages 2023-05, 2023.
[46] Zhen Miao, Jianqiao Wang. Kernyu Park, Da Kuang, and Junhyong Kim. Depth-corrected multi-factor dissection of chromatin accessibility for scatac-seq data with pacs. Nature Communications, 16(1):1-15,
2025.
12

---

[47] Kodai Minoura, Ko Abe, Hyunha Nam, Hiroyoshi Nishikawa, and Teppei Shimamura. scmm: Mixture- of-experts multimodal deep generative model for single-cell multiomics data analysis. bioRxiv, pages 2021-02, 2021.
[48] Samuel Morabito, Emily Miyoshi, Neethu Michael, Saba Shahin, Alessandra Cadete Martini, Elizabeth Head, Justine Silva, Kelsey Leavy, Mari PerezzRosendahl, and Vivek Swarup. Single-mucleus chromatin accessibility and transcriptomic characterization of alzheimer's 's disease. Nature genetics, 53(8):1 (3(8):1143-11) 143-1155, 2021.
[49] Ryan M Mulqueen, Dmitry Pokholok, Brendan I. O'Connell, Casey A Thornton, Fan Zhang, Brian J O' ''Raak, Jason Link, Galip Gürkan Yardiee, Rosalic C Scars, Frank J Steemers, al. High-content single-cell combinatorial indexing. Nature biotechnology, 39712)1574--188, 2021.
[50]
Antonio Orvieto, Samuel L. Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pascanu, and Soham De. Resurrecting recurrent neural networks for long sequences. In International Conference on 266988......... PMLR, 2023.
Machine Learning, pages
[51]
Anil Panigrahi and Bert W O' O'Malley. Mechanisms of enhancer action: the known and the unknown. Genome biology, 22(1), 108, 2021.
[52] Wang Qi, Yang Zhang, Bolei Zhang, Yao Fu, Xiaozhi Zhao, Jing Zhang, Ke Zuo, Yucxian Xing, Song Jiang, Zhaohui Qin, Erguang Li, Hongqian Guo, Hehe Liu, and Jingping Yang. Single-cell chromatin accessibility landscape in kidney identifics additional cell-of-origin in heterogenous papillary renal cell carcinoma. Nature Communications, 13(1):31, 2022.
[53] Aaron R Quinlan and Ira M Hall. Bedtools: flexible suite of utilities for comparing genomic features. Bioinformatics. 26(6):841 11188441.222. 2010.
[54] Shyam Ramasamy, Abrar Aljahani, Magdalena A Karpinska, TB Ngoe Cao, Taras Velychko, J Neos Cruz, Michael Lidschreiber, and Maricke Oudelaar. The mediator complex regulates enhanccr-promoter interactions. Nature Structural & Molecular Biology. 30(7):991-1000, 2023.
2023.
[55] Jennifer E Rood, Samantha Wynne, Lucia Robson, Anna Hupalowska, John Randell, Sarah A Teichmann, and Aviv Regev. The human cell atlas from a cell census to unified foundation model. Nature, pages 1-2, 2024.
[56] Adam J Rubin. Kevin R Parker, Ansuman T Satpathy, Yanyan Qi. Beijing Wu, Alvin Ong. Maxwell R Mumbach. Andrew L Ji. Daniel S Kim, Seung Woo Cho, al. Coupled single-cell crispr screening and epigenomic profiling reveals causal gene regulatory networks. Cell, 176(1):31-337, 2019.
[57] Ansuman T Satpathy, Jeffrey M Granja, Kathryn E Yost, Yanyan Qi, Francesca Meschi, Geoffrey P McDermott, Brett N Olsen, Maxwell R Mumbach, Sarah Pierce, Ryan Corces, et al. Massively parallel single-cell chromatin landscapes of human immune cell development and intratumoral I cell exhaustion. Nature biotechnology. 3788)9225-33. 2019.
[58] Michael Shtutman, Mirza Baig. Elina Levina, Gregory Hurteau, Chang-uk uk Lim, Eugenia Broude, Mikhail Nikiforov, Timothy T Harkins, C Steven Carmack, Ye Ding, et al. Tumor-specific silencing of cop22 gene encoding coatomer protein complex subunit (2 renders tumor cells dependent on its paralogous gene copz1. Proceedings of the National Academy of Sciences, 108(30) (08(3)):1444)) 22444-1224555 2011.
[59]
Tim Staart, Avi Srivastava, Shaista Madad, Caleb Lareau, and Rahul Satija. Single-cell chromatin state analysis with signac. Nature Methods, 2021. https://doi.org:
01586441592-021-01222
[60] Chang Su, Dongsoo Lee, Peng Jin, and Jingfei Zhang. scmultimap: Cell-type-specific mapping of enhancers and target genes from single-cell multimodal data. Nature Communications, 16(1):394 1:341, 2025.
[61] Laksshman Sundaram, Arvind Kumar, Matthew Zatzman, Adriana Salcedo, Neal Ravindra, Shadi Shams, Brian Louic, S Bagdatli, Shahab Sarmashghi, Hyo Choi, Wonyoung Choi, Kathryn Yost, Yanding Zhao, Jeffrey Granja, Toshinori Hinoue, D Hayes, Andrew Cherniack, Ina Felau, and Hui Shen. Single-cell chromatin accessibility reveals malignant regulatory programs in primary human cancers. Science, 185(6713) 2024.
[62] Zakich Tayyebi, Allison Pine, and Christina S Leslie. Scalable and unbiased sequence-inforee embedding of single-cell atac-seq data with cellspace. Nature Methods, 21(6):0014- 10222022,,1,,,-,,--1111111 2024.
[63] Nadezhda V. Terekhanova, Alla Karpova, Wen- Wei Liang, Alexander Strzalkowski, Siqi Chen, Li Yize, Austin N. Southard-Smith, Iglesia Michael D., Michael C. Wendl, Reyka G. Jayasinghe, Liu Jingxian, Yizhe Song. Song Cao. Andrew Houston. Xiuting Liu, Matthew A. Wyczalkowski, Rita Jui-Hsien Lu, Wagma Caravan, Andrew Shinkle, Nataly Naser Al Deen, John M. Herndon, Jacqueline Mudd. Cong Ma. Hirak Sarkar, Kazuhito Sato, Omar M. Ibrahim. Chia-Kues Mo. Sara E. Chasnoff, Eduard Porta-Pardo. Jason M. Hcld, Russell Pachynski, Julie K. Schwarz, William E. Gillanders, Albert H. Kim, Ravi Vij, John F. DiPersio, Sidharth V. Puram, Milan G. Chheda, Katherine C. Fuh, David G. DeNardo, Ryan C. Fields, Feng Chen, Benjamin J. Raphael, and Li Ding. Epigenetic regulation during cancer transitions across 11 tumour types. Nature, 62347986))432-444, 2023.
[64] Christina V Theodoris, Ling Xiao, Anant Chopra. Mark D Chaffin, Zeina R Al Sayed, Matthew C Hill, Helene Mantineo. Elizabeth M Brydon, Zexian Zeng. Shirley Liu, et al. Transfer learning enables predictions in network biology. Nature. 618(7965):616-622 2023.
[65] Ken To. Lijiang Fei. J Patrick Pett. Kenny Roberts, Raphael Blain, Krzysztof Polański, Tong Li, Nadav Yayon, Peng He, Chuan Xu, et al. A multi-omic atlas of human embryonic skeletal development. Nature, 635(8039):657-677 2024.
[66] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
[67] A Vaswani. Attention is all you need, Advances in Neural Information Processing Systems, 2017.
13

---

[68] Lin Wang, Husam Babikir, Sören Müller, Garima Yagnik, Karin Shamardani, Francisca Catalan. Gary Kohanbash, Beatriz Alvarado. Elizabeth Di Lullo, Amold Kriegstein, et al. The phenotypes of proliferating glioblastoma cells reside on single axis of variation. Cancer discovery, 1 2019.
[69] Hongzhi Wen, Jiayuan Ding. Wei Jin, Yiqi Wang. Yuying Xie, and Jiliang Tang. Graph neural networks for multimodal single-cell data integration. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 4153-4163, 2022.
[70] Hongzhi Wen, Wenzhuo Tang. Xinnan Dai, Jiayuan Ding. Wei Jin. Yuying Xie, and Jiliang Tang. Cellplm: pre-training of cell language model beyond single calls. bioRxiv, piggess 2023-10, 2023.
[71] Kevin E Wu, Kathryn É Yost, Howard Y Chang. and James Zou. Babel enables cross-modality translation between multiomic profiles at single-cell resolution. Proceedings of the National Academy of Sciences,
1 (8)5))2202300 18, 2021.
[72] Lei Xiong. Kang Tian, Yuzhe Li, Weixi Ning, Xin Gao, and Qiangfeng Cliff Zhang. Online single-cell data integration through projecting heterogeneous datasets into common cell-embedding space. Nature Communications, ( 2022.
[73] Lei Xiong, Kui Xu, Kang Tian, Yanqiu Shao, Lei Tang, Ge Gao, Michael Zhang, Tao Jiang. and Qiangfeng Cliff Zhang. Scale method for single-cell atac-seq analysis via latent feature extraction. Nature commmnications, 1051), 2019.
[74] Chenling Xu, Romain Lopez, Edouard Mehlman, Jeffrey Regier, Michael I Jordan, and Nir Yosef. Prob- abilistic harmonization and annotation of single-cell transcriptomics data with deep generative models.
Molecular systems biology. :7(1):,9620, 2021.
[75]
Kun Xu, Wenwen Zhang, Cong Wang. Longfei Hu, Runtian Wang, Cenzhu Wang, Lin Tang, Guohua Zhou, Bingjic Zou, Hui Xie, al. Integrative analyses of sema--sq and scatac-scq reveal excl 14 a key regulator of lymph node metastasis in breast cancer. Human Molecular Genetics, 30(5):370-33)
2021. [76] Xiongxiao Xu, Canyu Chen, Yueqing Liang. Baixiang Huang, Guangji Bai, Liang Zhao, and Kai Shu. Sst:
Multi-scale hybrid mamba-transforme transformner experts for long-short range time series forecasting, 2024.
[77] Fan Yang. Wenchuan Wang. Fang Wang, Yuan Fang. Duyu Tang, Junzhou Huang, Hui Lu, and Jianhua Yao. schert as a large-scale pretrained deep language model for cell type annotation of single-cell raa-eq data, Nature Machine Intelligence, 4(10):522-86) 866, 2022.
(78] Karren Dai Yang, Anastasiya Belyaeva, Saradha Vonkatachalapathy, Karthik Damodaran, Abigail Katcoff, Adityanarayanan Radhakrishnan, GV Shivashankar, and Caroline Uhler. Multi-domain translation between single-cell imaging and sequencing data using autocneoders. Nature communications, 12(1):31, 2021.
[79] Yu Yang. Xueyan Chen, Jieying Pan, Huiheng Ning, Yaojun Zhang, Yufei Bo, Xianwen Ren, Jiesheng Li. Shishang Qin, Dongfang Wang, Min-Min Min-Min Chen, and Zemin Zhang. Pan-cancer single-cell dissection reveals phenotypically distinct cell subtypes. Cell, 187(17): (7(17)44790-45 48114811212 Le22, 2024.
[80] Annan Yu, Arnur Nigmetov, Dmitriy Morozov, Michael W. Mahoney, and N. Benjamin Erichson. Robusti- fying state-space models for long sequences via approximate diagonalization. In International Conference Learning Repreventations, 2024.
[81] Han Yuan and David R Kelley. sebasset: sequence- based modeling of single-cell atac-seq using convolu- tional neural networks. Nature Methods, 19(9):1088 1096, 2022.
[82] Yuansong Zeng, Mai Luo, Ningyuan Shangguan, Peiyu Shi, Junxi Feng, Jin Xu, Ken Chen, Yutong Lu, Weijiang Yu. and Yuedong Yang. Deciphering cell types by integrating scatac-seq data with genome sequences. Nature Computational Science, 4(4):285-298, 2024.
[83] Kai Zhang, James Hocker, Michael Miller, Xiaomeng Hou, Joshua Chiou, Olivier B Poirion, Yunjiang Qiu, Yang E Li, Kyle Gaulton, Allen Wang, et al. A single-cell atlas of chromatin accessibility in the human genome. Cell. 184(24)-59856001, 2021.
[84] Kai Zhang. Nathan R Zemke, Ethan J Armand, and Bing Ren. A fast, scalable and versatile tool for analysis of single-cell omics data. Nature methods, 21(2):217-25) 227, 2024.
[85] Yu Zhang, Baole Wen, Yifeng Jiao, Yuchen Liu, Xin Guo, Yushuai Wu, Jiyang Li, Limei Han, Yinghui Xu, Xin Gao. et al. Script: Predicting single-cell long-range cis-regulation based on pretrained graph attention networks. Advanced Science, page e05021, 2025.
[86] Kaiyi Zhu, Jaroslav Bendl, Samir Rahman, James M Vicari, Claire Coleman, Tereza Clarence, Ovaun Latouche, Nadejda M Tsankova, Aiqun Li, Kristen J Brennand, et al. Multi-omic profiling of the developing human cerebral cortex at the single-cell level. Science advances, readg333337 2023.
[87] Qionghua Zhu, Xin Zhao, Yuanhang Zhang, Yanping Li. Shang Liu, Jingxuan Han, Zhiyuan Sun, Chunqing Wang. Daqi Deng. Shanshan Wang. et al. Single cell multi-omics reveal intra-cell-line heterogeneity across human cancer cell lines. Nature Communications, 14(1):81 170, 2023.
[88] Ryan S Ziffra, Chang N Kim, Jayden M Ross, Amy Wilfert, Tychele N Turner, Maximilian Hacussler, Alex M Casella, Pawel F Przytycki, Kathleen C Keough, David Shin, et al. Single-cell epigenomics reveals mechanisms of human cortical development. Nature, 59857879) 2021.
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providddd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly state ChromFound's contributions in advancing single-cell data analysis tasks like cell clustering 4.1 denoising batch effect [4.].
denoising low count 4.1 cell type annotation 4.2 cross-omics prediction 4.3 and predicting enhancer enhancer-gene gene link 4(a) and perturbation response 4(b) aligning with the experimental results presented in experiments section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The conclusion section highlights that ChromFound's pretraining data is currently restricted to human species, with ongoing efforts planned diversify the training dataset and refine the methodology to improve cross-species generalizability and widen its applicability.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specifiiattin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
16

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: As an application-driven study in Al for Science, the paper focuses evaluat- ing ChromFound through experimental validation across practical single-cell downstream tasks, such zero-shot cell clustering, cell type annotation, cross-omies prediction, pre- dicting enhancer-gens link and perturbation response rather than presenting theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
The paper provides comprehensive details in Sections D B.1 and B.3 covering data
preprocessing, model architectures, training settings (e.g.. 90%- 10% train-validation splits,
AdamW optimizer with learning rate of x 10 20 epochs for cell type annotation), and evaluation metrics [ for cell clustering, cell type annotation, and cross-omics prediction tasks, ensuring the main experimental results are reproducible.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution
specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (c.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
17

---

(c) If the contribution is new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We plan open-source the code and data upon acceptance to ensure full reproducibility; however, the paper provides comprehensive details in Section 4 and the appendix B I
D
and G to replicate the experiments, ensuring the results are solid and
reproducible.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/pu
Salcc/guies//Cae b sondddSuuumisssiooooooy.y for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
ciiiaps.cc/uullc///gi
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 3.3 details the pretraining pretrain implementation and G. details the training and test settings, including 90%-10% data splits, hyperparameters (e.g.. learning rate of 5 x 10-4), AdamW optimizer, and 50-step warmup schedule in in cell type annotation task.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
18

---

7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
and Table capturing variance due to downsampling levels, model component variations, and initialization differences in noise/sparsity analysis, ablation studies, and zero-shot cell clustering, respectively; these error bars represent the standard deviation over 20 independent runs, calculated directly from the sample variance as detailed in the corresponding sections, assuming normally distributed errors without asymmetric distributions that would lead to out-of-range values.
The paper reports error bars in Fig Table 4
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Section 3.3 details that pretraining was conducted on a cluster of machines with 32 NVIDIA A100 GPUs 5 epochs over 80 hours, while Sections G. and B.3 note that cell type annotation and cross-omic prediction experiments each ran on a single machine with 4 NVIDIA A100 GPUs for 20 and 10 epochs, respectively; cell clustering in Section B. required minimal compute as it operates in zero-shot mode; total compute across all experiments dominated by pretraining, with additional resources used for preliminary experiments on 32K cells as noted in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
19

---

Question: Does the research conducted in the paper conform, in every respect. with the NeurIPS Code of Ethics httpgh/twwpssp.aoc........ppopoa
Answer: [Yes]
Justification: The research adheres to the NeurlPS Code of Ethics, using publicly available
single-cell datasets with ethical concerns regarding data collection usage, confirmed in Table [
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The paper highlights positive societal impacts in Section [ noting that Chrom- Found's ' advancements in single-cell analysis can enhance biological research in fields like cancer, immunology, and neuroscience; however, it also acknowledges potential negative impacts, such as the risk of biased outcomes if trained on unrepresentative datasets, which could affect fairness in medical applications, and the possibility of incorrect predictions delaying scientific progress, with a suggestion for mitigation through careful dataset curation and validation as implied in the evaluation protocols of Section
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), faimess considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: ChromFound and the associated datasets pose no high risk for misuse, as they are designed for single-cell analysis with publicly available biological data, as noted in Section
20

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: A comprchensive data table in T lists all single-cell datasets used, including their sources, download links, and citations to the original creators, ensuring proper crediting; all datasets are publicly available and open-source, with their licenses respected as confirmed by their accessibility through the provided references and links.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use package should be provided. For popular datasets, paperswi www.wwtithhd........ com/ com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not generate new data as assets; however, ChromFound's model weights are new asset and while not released submission time, they will be available with complete documentation after acceptance.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
21

---

14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does involve crowdsoureing or research with human subjects, focusing solely on computational analysis of single-cell datasets.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve human subjects or crowdsourcing, as it focuses on computational methods for single-cell data analysis.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core methods of ChromFound not involve LLMs; the research focuses computational methods for single-cell data analysis without L.M components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (ttepe////arrrgeecc///oooererees for what should or should not be described.
22

---

A Discussion of comparing OCR tokenization with other peak-calling algorithms
Traditional ScATAC-seq pipelines often rely fixed OCR vocabulary (c.g., from reference atlas) to harmonize datasets, which ensures alignment consistency but introduces two major limitations: (1) loss of OCR length information, which encodes accessibility strength and resolution, and (2) reduced generalizability, as tissue-specct speuueeseccficceeee or novel OCRs are often excluded. ChromFound addresses these issues through actual genomic start and end positions using sinusoidal positional embeddings (Section 3.1). This position-aware representation naturally accommodates variable-length OCRs and preserves their spatial relationships, enabling the model to capture regulatory heterogeneity across tissues.
tokenization strategy that encodes each OCR by its
We perform experiment to directly compare ChromFound with VAE-based models trained on reference-per peaceeeppaaaaaaae yeak-aligned data. Specifically, we adopt the cPeaks reference set [45), which defines 1,657,194 peaks across the human genome. We use 20,000-cell PBMC subset (same as in Table F Row 6) as the training data. After mapping peaks to the cPeaks reference and filtering out peaks and cells with all-zero values, we obtain training set of 306,78. 784 peaks and approximately 18,000 cells.
We train four VAE-based baselines [73 72 44 on the aligned data and compare their performance on zero-shot cell clustering (Table D against ChromFound trained on the same aligned data. Evalu- ation metrics and datasets follow TableT aligned to 306,784 peaks. Results are summarized in Table
This comparison yields two key observations. First, reference-basee alignment substantially limits cross-tissue cross-tissue generalization: only 30% of peaks in retina, 35% in cortex, and 70% in heart overlap with the cPeeaks-alinee PBMC training peaks, leading to markedly lower performance of the baseline VAES on unseen tissues. Second, ChromFound, though robust even when trained on the reference- aligned data, performs best when using its native OCRs, suggesting that fixed weak harmonization fails to capture novel or shifted OCRs arising from tissue heterogeneity, disease-specific variation, or batch effects.
B Experiments details
B.1
Cell clustering
Since ChromFound configured in a zero-shot mode for the cell clustering task, no additional fine- tuning architecture or training parameters need to be specified. We directly propagate the data through the encoder structure of ChromFound, extracting the encoder output as the final representation. After performing pooling layer on the hidden dimension, we apply Principal Component Analysis (PCA) to reduce the dimensionality of the output 50. Based on this, we compute the metries for cell clustering and generate the UMAP clustering visualization. The benchmark methods are implemented with the default parameters provided by their source codes.
We compute the inference speed of ChromFound on a node equipped with 32 CPU cores, 256 GB of memory, and one NVIDIA A100 GPU. The input scATAC-seq dataset with 232,354 OCRs. The maximum CPU memory usage is 23.9 GB and the maximum GPU memory consumption is 12.7 GB out of 80 GB available, The inference speed with batch size 4 is 0.97 seconds per batch.
B.2
Cell type annotation
Building upon our four-layer pretrained model, we extract the encoder output as a general-purpose representation for downstream cell type annotation. To align the model with this task, we incorporate additional fine-tuning layers. Specifically, the hidden dimension produced by ChromFound's encoder is expanded from 128 to 256 before being projected down to single dimension, resulting in pooling tensor. Subsequently, additional multilayer perceptron (MLP) layers are employed to predict the logits corresponding to cell types. We freeze the decoder parameters from pretraining while enabling gradient flow through the rest of the pretrained backbone so that can adapt to the new classification objective. Dropout mitigates overfitting by randomly zeroing hidden units, while LayerNorm helps stabilize and accelerate fine-tuning. The benchmark methods are implemented with the default parameters provided by their source codes.
23

---

Table 5: Results of VAE-based models with cPeaks-aligned training against ChromFound. Result style: best, second best
Dataset
Model
VAE
ePeaks
ARI(()
FMI(O)
NMI(1)
AMI(()
SCALE 773
0.2932
0.5050
0.4127
0.4081
SCALEX 722
0.4230
0,6078
0.5543
0.5513
peakVI 1I
0.3690
0.5643
0.5023
0.4983
Cortex(batch
poisson VI
0.3364
0.5408
0.4636
0.4594
ChromFound
0.5858
0.7244
0.7519
0.7499
ChromFound
0.6475
0.7503
0.7701
0.7689
SCALE 73
0.2484
0.4052
0.3657
0.3609
SCALEX 772
0.4481
0.5679
0.5742
0.5710
peak [
0.3610
0,5035
0.4876
0.4835
Cortex(batch 2)
poisson VI
0.2820
0.4374
0.4104
0.4058
ChromFound
0.5751
0.6720
0.7145
0.7123
ChromFound
0.6092
0.7015
0.7276
0.7259
SCALE 73
0.3543
0.4817
0.5228
0.5195
SCALEX
0.2795
0.4150
0.4787
0.4752
Ve [1]
0.3023
0.4437
0.4816
0.4779
Heart(av 3)
poisson VI
0.3778
0.5025
0.5446
0,5414
ChromFound
0.5400
0.6347
0.6588
0.6567
ChromFound
0.5642
0.6601
0.6981
0.6953
SCALE (73
0.4051
0.6506
0.4461
0.4453
SCALEX 772
0.3013
0.5685
0.4199
0.4191
VeakVI I
0.3966
0.6080
0.5018
0.5011
Heart(av 10)
poisson
0.3939
0.5997
0.5010
0.5003
ChromFound
0.6150
0.7695
0.6968
0.6963
ChromFound
0.6432
0.7869
0.7220
0.7223
SCALE 73
LTLT FTTT
0.3065
0.5771
0.4447
0.4440
SCALEX 72
0.2430
0.5149
0.4127
0.4117
Peak I
0.2514
0.5260
0.4651
0.4644
Retina(D026_
poisson's
0.2347
0.5122
0.4063
0.4055
ChromFound
0.4888
0.7081
0.7241
0.7238
ChromFound
0.5802
0.7549
0.7410
0.7403
SCALE 73
0.3143
0.5092
0.4220
0.4210
SCALEX 72
0.2490
0.4542
0.3920
0.3909
Peak 1I
0.2664
0.4693
0.4745
0.4736
Retina(D19D00S
poisson VI
0.2396
0.4473
0.3937
0.3926
FoundFound
0.5368
0.6839
0.7664
0.7660
ChromFound
0.5956
0.7202
0.7832
0.7868
SCALE
0.5769
0.6756
0.7066
0.7057
SCALEX 772
0.5718
0,6514
0.7205
0.7196
Veak 1
0.5306
0.6249
0.7057
0.7046
PBMC(VIB_IOXxL_])
poisson VI 44
0.5593
0,6479
0,6742
0.6732
ChromFound
0.5999
0.6827
0.7434
0.7424
ChromFound
0.6142
0.6995
0.7354
0.7345
SCALE 73
0.4031
0.5797
0.5380
0.5374
SCALEX 72
0.3326
0.5430
0.5465
0.5458
Peak !
0.3830
0.5804
0.5737
0.5731
PBMC(BIO_ddsey
poisson VI
0.3959
0,5917
0.5509
0.5503
ChromFound
0.4347
0.6237
0.5867
0.5861
CoromFound
0.4579
0.6421
0.5902
0.5910

---

For experimental settings, we divide the training data into 90% for training and 10% for validation. We train for 20 epochs using the AdamW optimizer with an initial learning rate of 5 x 10 and linear warm-up schedule of 50 steps. The best model is chosen based on validation set performance and evaluated on the test set to obtain the reported accuracy and macro F1 score. All experiments are conducted on single machine equipped with four NVIDIA A100 GPUs.
B.3 Cross-omics prediction
We employ a two-layer Multilayer Perceptron (MIP) to perform the modality prediction task, with the hidden layer size set 1024. The learning rate is configured le-4, and AdamW is utilized the optimizer. A total of 10 epochs are conducted for training. All benchmark methods are implemented with the default parameters in the DANCE package [16]. All experiments conducted on single machine equipped with four NVIDIA A A10 GPUs. We divide the whole dataset into 80% for training, 10% for evaluation, and 10% for testing.
B.4 Biological application
To validate the biological relevance of predicted enhancer-gene interactions, we simulate enhancer knockdown using our cross-omics model trained on the ZhuliK dataset [87]. Specifically, we select 141 K562 cells from the test split and iteratively set the accessibility of each candidate enhancer to zero, mimicking CRISPRi-mediate expression, and the change relative to the unperturbed state is computed and averaged across all cells. The absolute magnitude of this change indicates the likelihood of a regulatory interaction, while the signed change reflects its direction and strength.
repression. The model then infers post-perturbation gene
We focus on two cancer-related genes, COPZ1 and HNRNPA1, which are extensively characterized in the Fulco4K CRISPRi dataset [87]. Each gene is associated with 117 candidate enhancers, among which 6 (COPZI) and 10 (HNRNPA1) are experimentally validated. To resolve discrepancies in genomic coordinates between Fulco4K 19 and Zhul IK (87), we perform enhancer mapping using bedtools (53). In total, the simulation spans 230,819 enhancers and 17,476 genes.
To evaluate performance, we compute the area under the ROC curve seen in Table 4(a) between the absolute expression change and the binary labels of enhancer- gene links tested with a statistical significance on gene expression at false discovery rate (FDR) < 0.05 [19.. ChromFound achieves the highest accuracy, with AUCs of 0.77 for COPZI and 0.61 for HNRNPAI, demonstrating its ability to identify functional regulatory links. Additionally, we calculate the Pearson correlation between signed responses and CRISPRi- quantitative effects plotted in Table 4bb)
The Benchmark methods BABEL 771) and CMAE [78 are included for comparison due to their strong cross-omics performance on ZhullK. However, both rely filtering the input ScATAC-seq profiles to the top 10,000 highly variable OCRs [16., which excludes the majority of candidate enhancers from Fulco4K. As result, these methods often fail produce meaningful predictions for enhancer perturbations, yielding near-zero expression changes in most cases. This limitation is evident both the ROC and scatter plots, underscoring the advantage of ChromFound's genome-wide modeling in capturing fine-grad fine-grained regulatory effects at scale.
C Details of datasets
We collect a large-scale human SCATCC-sc dataset that includes more than 2.65 million cells and 1.75 trillion tokens. ChromFound for pretraining is based on two fundamental datasets, the human atlas [83] and the fetal atlas [17), which contribute 1.32 million cells spanning multiple human organs and serve as the primary sources for pretraining ChromFound. We accept the CRCh38 as reference genome across all datasets to maintain uniformity, converting datasets originally in hg19 to hg38 when necessary. The resources of all datasets are detailed Table 6
D Pre-processing of ScATAC-seq data
We develop preprocessing pipeline for scATAC-seq data to address sparsity and high dimensionality, ensuring high-quality cells and OCRs for model pretraining and downstream analyses. Preprocessing
25

---

is applied after train/test splitting to avoid information leakage, and all datasets are stored in .5add format with genomic OCR coordinates the var schema.
Quality Control: We filter out cells with non-zero counts helow 1,000 or above 60,000 and remove OCRs with non-zero counts present in less than 5% of all cell types to preserve heterogeneity among rare populations. For unlabeled datasets, pseudo labels are inferred using TF-IDF followed by LSI for OCR filtering.
Cell aggregation: mitigate sparsity, we aggregate neighboring cells identified through Latent Semantic Indexing (LSI) and cosine similarity. LSI extracts components after removing technical noise (the first component), and cosine similarity determines the 10 nearest neighbors for aggregation.
Normalization and Log Transformation: To stabilize variance and scale the data, we apply total count normalization followed by log transformation.
Table 6: Dataset Information
Dataset
Tissue
Disease
Cell Numbers
OCR Numbers
Genome
GSE/GSM ID
HumanAtlas
83
Atlas
Health
615,998
1,154,611
bg38
GSE184462
FetalAtlas
Atlas
Health
707,043
1,154,646
bg38
GSE149683
Peripheral
PBMCBenchmark
Health
169,047
412,490
hg.38
GSE194028
Blood
Alzheimer's discase
Cortex 30K
48
Cortex
130,419
219.070
bg38
GSE174367
Cortex2K
4p
Cortex
Health
2,174
292.156
bg38
GSE174226
Zhu45K
86
Health
45,549
304,034
hg38
GSE204684
Peripheral
PВМСРK
2
Blood
Leukemia
9,215
108,344
hg38
GSE139369
BMМC1IK
Marrow
Health
11,384
452,004
hg
GSE194122
Rubin 56
Epidermis
Health
288
94,633
bg
GSE116428
Xu12K 75
Breast
Breast cancer
7,202
87,851
hg33
GSM4798906
5,642
76,850
GSM4798907
6,284
79,730
CSM4119511
129
1,169
GSM41 19514
Wang23K 68
Brain
Glioma
5,213
210.434
hg33
GSM41 19515
5,519
183,847
GSM41 19516
2,229
40,907
GSM41 19517
GSM41
Buearostro3K
3
Bone Marrow
Health
2,953
491,437
bg
GSE96769
CorcesO.5K 9
Bone Marrow/ Peripheral Blood
Leukemia
576
590.650
bg19
GSE74310
Corces70K 10
Brain
Alzheimer's und Parkinson's diseases
70,631
444,747
hg38
GSE147672
Marrow
Health
63,882
571,400
Peripheral
Satpathy LOK 37
Health
4,146
238.616
1g19
GSE129785
Blood
Peripheral
Health
4,786
152,367
Blood
TME
Cancer
37,818
580,789
Ziffia7KK 88
Forebrain
Health
77.354
459.953
bg38
GGE163018
Myocardial
Kuppe139K
Heart
139,835
429.828
bg38
infarction
Liung 54K
35
Retina
Health
154,775
264,833
hg19
GSE226108
TTo326K 65
Bone
Health
326,532
530,751
hg38
Zhu l KK
g7
Cell
Line
N/A
11,632
258.044
1g19
089118912
10,247
90,686
10k v1.1
9,688
144,023
TUK V2.0
Peripheral Blood
Health
135,377
bg19
1,004
82,579
V2.0
484
65,908
500 v2.0
26

---

E Supplementary results of ablation study
E.1 Hyperparameter sensitivity
We evaluate the hyperparameter sensitivity on the cell clustering task using PBMC169K (batch VIB_I0xvI dataset.
Positional embedding temperature: This parameter controls the frequency of sinusoidal position encodings and can be viewed as proxy for "genomic resolution". As shown in Table 7 performance slightly drops when temp is within 10e3 10e5, and degrades more substantially when temp is too large, likely due to loss of relative position sensitivity.
Table 7: Results of hyperparameter temp sensitivity experiment.
temp
ARI(1)
NMI(1)
AMI(t)
FMI(t)
1000
0.6535
0.7764
0.7755
0.7259
10000
0.6512
0.7709
0.7701
0.7243
100000 (Ours)
0.6953
0.7860
0.7852
0.7601
1000000
0.6036
0.7344
0.7334
0.6860
10000000
0.5852
0.7269
0.7258
0.6714
Mamba projection dimension: This parameter controls the compression within the Mamba block. As shown in Table 8 Diow 32 achieves strong balance between performance and efficiency. Our choice of 32 is primarily motivated by the trade-off between performance and computational efficiency. Larger values lead marginal gains but significantly increase FLOPPs and memory, with Diow = 128 exceeding the memory limits on A 80G GPUs.
Table 8: Results of Mamba projection dimension Diow sensitivity experiment.
Diow
#Parameter
FLOPs (10c11)
Inference Speed (s)
VRAM (GB)
ARI(1)
16 32 (Ours)
422,593
7.41
3.4027
48.8
0.6158
450,305
7.89
3,4624
60.7
0.6953
64
518,785
9.09
4.0053
72.3
0.6927
128
707,969
12.4
OOM
OOM
OOM
E.2
Comparison Transformer-only architectures
Training a vanilla Transformer on ScATAC-seq inputs containing nearly one million peaks is prac- tically infeasible due to its quadratic scaling memory and computation. To approximate a pure Transformer baseline under practical settings, we train two representative single-cell foundation mod- els, Geneformer and scGPT, on the same pretraining corpus with increasing peak lengths (4k-32k) and the same hyperparameters WPSA. The results of comparison the cell clustering task using the PBMC169K (batch VIB vIB_10xvl I_) dataset are detailed in Table
Table 9: Results of comparison on Transformer-only architectures.
Method
OCRs Length
ARI(1)
Geneformer
4096
0.0451
16384
0.0457
32768
0.0460
ScGPT
4096
0.3075
16384
0.3774
32768
0.3868
ChromFound
440,000
0.6953
27

---

Supplementary results of denoising batch effect
We show the plots of the denoising batch effect below. The results on To326K [65] from the Bone tissue are plotted in Fig. 5 The results on Kuppel1 13K [33] from the Heart tissue are plotted in The results on Morabitoll 30K 48 from the Cortex tissue are plotted in Fig. 7 The results on PBMC16 69K [5] from the PBMC tissue are plotted in Fig. In addition, the results of all metrics
Fig. 6
listed in Table I0
Table 10: Comparison of Batch Correction and Biological Conservation Metrics Across Datasets. The best performance for each metric within each dataset is highlighted in bold.
Dataset
Method
Batch Correction Metries
Biological Conservation Metrics
ASW harch
GraphConn
ASW sell
ARl
NMI
scVI
B8
0,8863
0,8595
0.5199
0.4537
0.4631
Pe
0.9135
0,9373
0.6022
0.8021
0.7725
VoissonVI
44
0.9205
0,9353
0.6021
0.7991
0.7707
Cortex Morabito130K 48
Scanorama
0,9391
0,9211
0.5318
0.8234
0.7585
Harmony
0.9174
0,9262
0,5793
0.8032
0.7892
batch 1/2
SCANVI 77
0.8790
0.9697
0.6585
0.6185
0.7916
Liger 337
0,8605
0.9598
0.5689
0.7899
0.7870
SCALEX 72
0.9143
0.9274
0.5791
0.7896
0.7869
scBasset 81
0,8052
0.9784
0.6361
0.6876
0.7230
ChromFound
0.9323
0.9808
0.6408
0.7531
0.8380
scVI [38]
0.9073
0.9283
0.4891
0.0070
0.0374
PeakVI 1
0.9088
0.9259
0.5145
0.1634
0.2167
0.9098 PoissonVV 44
0.9259
0.5006
0.1634
0.2167
Bone To326K 665
Scanorama
0.8836
0,9580
0,5115
0,2325
0.3513
batch 43/44
Harmony
0.8508
0,9544
0.5539
0.2535
0.3066
ScANVI 74
0.7939
0,9506
0.5721
0.2835
0.4604
Liger 37
0.8244
0.8874
0.5535
0.2167
0.3668
SCALEX 72
0.7452
0.9506
0.5616
0.2835
0.4604
scBasset
81
0.5163
0.7781
0.4339
0.0603
0.1035
ChromFound
0.8694
0.9885
0.6027
0.6426
0.6772
scVI 38
0.8128
0.8704
0.6247
0.8186
0.8355
PeakVI [
0.9149
0.5980
0.5215
0,6076
0.5680
Poisson'VI 44
0.9460
0.5980
0.5106
0.6076
0.5680
Heart
Scanorama
0,9378
0.7804
0.4707
0.6869
0.7072
K99 33
Harmony
0.8490
0.8407
0.5546
0.8066
0.8298
donar av3/av10
ScANVI [74]
0.7844
0.7715
0.4337
0,5070
0.6553
Liger
37
0,8374
0.8407
0.4196
0.4875
0.6287
SCALEX 72
0,8638
0.8164
0.5424
0.7296
0.7711
scBasset 8
0.7756
0,8282
0.6387
0.8055
0.7417
ChromFound
0.8589
0.8770
0.6155
0.9406
0.8980
scV1 38
0.7452
0,8557
0.5794
0.5286
0.6888
PeakVI 1
0.7511
0.8266
0.6054
0.5456
0.7005
PoissonVI 45
0.7895
0.8266
0.5626
0.5456
0.7005
PBMC 169K 15
Scanorama
0.7964
0.8066
0.5613
0.5456
0.7005
HAR ddseq 1/2
Harmony
0.7745
0.8120
0.5711
0.5525
0.7069
ScANVI 74
0,7480
0.8448
0.6223
0,5114
0.6773
Liger 337
0.7755
0,8091
0.6197
0.5481
0.7020
SCALEX
72
0,8221
0.7850
0.5534
0.5549
0.7003
scBasset
81
0.7753
0,8266
0.5695
0.5456
0.7005
ChromFound
0.7923
0.8511
0.6443
0.5730
0.7156
G Analysis of Cell Type Annotation
G.1 Metric Value of Cell Type Annotation
To adapt ChromFound for cell type annotation, we leverage four-layer pretrained model, extracting the encoder output as a general-purpose representation. The encoder's hidden dimension is expanded
28

---

Chromfound
Harmony
Scenorama
sen
LIGER
SCANVI
Figure 5: The plots illustrate ChromFound's superior performance in denoising batch effect on Bone To326K [65].
ChromPound
Harmany
LIGER
Figure 6:
The plots illustrate ChromFound's superior performance denoising batch effect on
Heart K3ppel139K 33
29

---

Hamony
Scanorama
scV
LIGER
SCAMI
Figure 7: The plots illustrate ChromFound's superior performance in denoising batch effect on Cortex M3r 48
Harmany
Scanorama
ScVI
LIGER
SCANMI
Figure 8: The plots illustrate ChromFound's superior performance in denoising batch effect on PBMC 169K [15]
30

---

from 128 to 256 and projected to single dimension, forming pooling tensor. This tensor is then processed by additional multilayer perceptron (MLP) layers predict cell type logits. During fine-tuning, the pretrained decoder parameters are frozen, while gradients flow through the rest of the backbone to align with the classification objective. Dropout and LayerNorm are applied to mitigate overfitting and stabilize training, respectively. Benchmark methods are implemented using their default parameters as provided in their source codes.
For the experimental setup, we split the training data into 90% for training and 10% for validation. Training conducted over 20 epochs using the AdamW optimizer with an initial learning rate of 5 x 10 and a 50-step learning rate warmup schedule. The model is selected based on validation performance and evaluated on the test set to compute accuracy and macro Fi-score. For certain configurations, such as training on EPF hydrop 2 and testing on VIB 10xv1 1, the learning rate reduced to 2.5 x 10-1 4 to ensure stable convergence. All experiments are performed on a single machine with four NVIDIA A100 GPUs.
G.2 Confusion Matrix of Cell Type Annotation
In the cell classification task where EPF hydrop_3 served as the training set and VIB 10xv1 as the test set, ChromFound achieves 4.71% % improvement in accuracy and 14.69% increase in macro F1 score compared to the previous SOTA method, CellCano. To further elucidate the specific advancements of ChromFound, present the confusion matrices of both methods for more detailed comparative analysis.
Here some biological insights from the comparison of confusion matrices:
(I) The observed misclassification rate of natural killer cells as cytotoxic T cells by the Cellcano model (19.35%) and its significant reduction by ChromFound (2.22%) highlights the importance of accurately distinguishing these functionally related yet distinct immune cell populations. Natural killer cells and cytotoxic T cells share cytotoxic properties, as both can mediate target cell lysis through perforin and granzyme pathways. However, they differ fundamentally in their ontogeny. activation mechanisms, and immune regulation.
(2) The high misclassification rate of CD16+ monocytes CD14+ T cells by Cellcano (91.91%) and its substantial reduction by ChromFound (66%), underscores the challenge of accurately dis- tinguishing myeloid from lymphoid lineages in cell type annotation based on ScATAC-seq. CD16+ monocytes, a subset of non-classical monocytes, exhibit distinct chromatin accessibility patterns associated with Fc receptor signaling, inflammatory responses, and patrolling behavior, whereas CD14+ T cells, a less well-characterized subset, retain a predominantly lymphoid epigenetic signature. The improved classification by ChromFound suggests a refined ability resolve lineage-speeific regulatory elements, likely by better capturing differential enhancer accessibility and transcription factor binding landscapes unique to myeloid versus lymphoid cell fate.
(3) Cellcano misclassifies dendritic cells as CD14+ monocytes at a rate of 65.7% while ChromFound reduces this misclassification to 53.33%. The misclassificttion of dendritic cells as CD14+ monocytes is particularly interesting because both cell types share similar transcriptional signatures, especially in immune responses. dendritic cells and monocytes both play critical roles in antigen presentation and inflammation, which may lead to similarities in the chromatin accessibility profiles captured by ScATAC-seq. From a biological perspective. this reduction misclassification also underscores the importance of distinguishing between functionally distinct but phenotypically similar immune cell subsets. Dendritic cells, being key mediators of immune tolerance and initiation, have distinct regulatory networks compared to monocytes, which are more directly involved in inflammatory responses. By refining the accuracy of cell type annotation, ChromFound enables a more precise understanding of immune cell dynamics, particularly in the context of immune responses and disease progression.
H Benchmark Methods for Downstream Tasks
This section describes the benchmark methods used for evaluating the performance of ChromFound across various downstream tasks. These methods are selected based on their established effectiveness in single-cell data analysis, spanning cell clustering, cell type annotation, and cross-omics modality prediction.
31

---

Celicane
ChromFound
CD4+ T -e.1-94.41 0.59 5.01 0.00
0.00
0.80
0.80
CD4+ Tcll
59.24
0.00 0.76 0.00 0.00 0.00 0.00
CD14+
CD14+
0.00
0.35 0.00 0.00 0.00 0.00
Cytatosic
Cytotoxic
15.50
3.88
0.00
5.83
cell
1.05
0.00
1.05
Natural kiler cell-
0.77
0.00
19.35
0.00
79.09
0.00
0.00
Natural killer tell 4.44
0.00
2:22
0.00
93.33
0.00
0.00
CD16+ monocyte 0.81
91:91
1.00 0.27
0.54
4.50
0.81
CD16+ monocyte
0.00
66.00
0.00
0.00
0.00
34.00
0.00
Dendritic cell-
0.58
65.70
1.16
0.00
0.00 0.58
31.98
Dandritic cell
0.00
$3.33
0.00
0.00
0.00
0.00
#6.67
CD4 call.
CD14+ menecyo
Cytatoiic call
call
Natural diler call.
CD16 menecyte
Dandratc call
CDE+ cell
CD14+ menesyte
Cytatoalc cell
coll
Natural kiler kiler cell
CD16- menesyte
Dendrati cell
Figure 9: Confusion matrix for cell type annotation task with comparison of Cellcano and Chrom- Found.
H.1
Cell Clustering
Cell clustering methods aim to group single-cell data into meaningful clusters, often leveraging deep learning and generative models to handle the high dimensionality and sparsity of ScATAC-seq data.
H.1.1 SCALE
SCALE [73 a deep generative framework that employs probabilistic Gaussian Mixture Models analyze high-dimensional ScATAC-seq data. It outperforms existing tools in clustering by effectively capturing latent structures in the data.
H.1.2 SCALEX
SCALEX 72] is a deep learning method designed for integrating single-cell data. projects cells into a batch-invariant embedding space in online manner, enabling robust cell representation and achieving excellent clustering performance.
H.1.3 CASTLE
CASTLE (12) is deep generative model tailored for single-cell epigenomic data. It uses vector- quantized variational autoencoder framework [66] extract discrete latent embeddings, excelling in cell clustering tasks by addressing the challenges of high dimensionality and sparsity.
H.1.4 scBasset
scBasset [81] is sequence-bac based convolutional neural network method designed for modeling single-cell ATAC-seq data. By utilizing DNA sequence information from accessibility peaks and the expressive power of neural networks, scBasset achieves superior performance in tasks such cell type identification and data integration across single-cell ATAC-seq and multiome datasets.
H.1.5 VeakVT
Peak VI [ is variational autoencoder base) probabilistic model for single-cell chromatin accessibility data. It captures both biological and technical variation through a hierarchical generative process, enabling effective batch correction and denoising. The learned latent representation supports diverse downstream tasks such as clustering and differential accessibility analysis, providing a strong probabilistic baseline for ScATAC-seq modeling.
32

---

H.1.6 PoissonVI VI
extends the VAE framework by modeling raw accessibility counts with Poisson likelihood instead of binarized inputs. This design preserves quantitative accessibility information and improves sensitivity subtle regulatory differences. PoissonVi achieves robust performance across clustering and integration tasks while maintaining a clear probabilistic interpretation.
Poisson VI
H.1.7 SnapATAC2
SnapATAC2 is a scalable framework for large-scale scATAC- analysis. It represents each cell by high-dimensional accessibility profile and applies diffusion- and spectral-based dimensionality reduction to reveal chromatin structure. With optimized memory management, parallel I/O, and GPU acceleration, SnapATAC2 efficiently handles datasets with millions of cells and serves strong non-deep learning baseline.
H.1.8 Signac
Signac [59] is an R package built on the Scurat ecosystem for single-cell chromatin accessibility analysis. It integrates preprocessing. feature selection, dimensionality reduction, and visualization within a unified workflow, and supports multimodal integration with scRNA-seq data. Owing to its usability and reproducibility. Signac remains widely adopted toolkit for exploratory and integrative single-cell epigenomic analysis.
H.2 Denoising Batch Effect
Batch effect removal methods aim align single-cell data across different batches while preserving biological signals, addressing technical variations that can obscure true biological differences. In this evaluation, we also benchmark SCALEX (2), previously introduced for cell clustering, and scBasset [8]]. noted for its requence-basee modeling, alongside other methods. These are included in the comparison for batch effect correction within their respective frameworks, with their detailed definitions provided in prior sections.
H.2.1 Harmony
Harmony 322 is prominent batch correction technique in single-cell analysis, designed to harmonize datasets from diverse experimental batches. It employs iterative strategy align cells into shared low-dimensional embedding, optimizing the similarity of cell types across batches while retaining biological variability, making it highly effective for downstream analyses single-cell genomics.
H.2.2 SeVI
seVI 38 is package for end-to-end analysis of single-cell omies data. The package is composed of several deep generative models for omics data analysis.
H.2.3 Scanorama
Scanorama [29 is designed to be used in scRNA-seq pipelines downstream noise-reduction methods, including those for imputation and highly-variable gene filtering. The results of Scanorama integration and batch correction can then be used as input to other tools for clustering, visualization, and analysis of scRNA sequences.
H.2.4 SCANVI
(single-cell ANnotation using Variational Inference; Python class SCANVI) is semi- supervised model for single-cell transcriptomics data. sense, can seen as a ScVI [8 extension that can leverage the cell type knowledge for subset of the cells present in the data sets to infer the states of the rest of the cells. For this reason, scANVI can help annotate a data set of unlabelled cells from manually annotated atlases.
scANVI 74
33

---

H.2.5
Liger
Liger (37] (Linked Inference of Genomic Experimental Relationships) integrates multi-omies single- cell data via integrative non-negative matrix factorization (iNMF) to define shared cell identities across protocols/species.
H.3 Cell Type Annotation
Cell type annotation methods focus supervised learning approaches to assign cells to predefined classes, leveraging advanced neural network architectures for improved accuracy.
H.3.1
Cellcano
Cellcano 422 utilizes gene expression scores as input and employs dual-phase training framework to achieve enhanced accuracy in cell type annotation across diverse datasets.
H.3.2
EpiAnno
EpiAnno implements probabilistic generative model with a Bayesian neural network. It delivers remarkable performance in cell type annotation, particularly when applied to diverse single-cell datasets.
H.4 Cross-omics Prediction
Cross-omics modality prediction methods aim infer one modality (e.g., ATAC-seq) from another (e.g., RNA-seq), often using integrative neural network frameworks to model the relationships between modalities.
H.4.1 BABEL
BABEL 71 leverages an interoperable neural network model to translate between the transcriptome and chromatin profiles of individual cells, enabling effective cross-omics prediction.
H.4.2 CMAE
CMAE (78 learns probabilistic coupling between differrnt data modalities using autoencoders. It provides a robust framework for integrating and translating between single-cell data modalities.
H.4.3 ScMoGNN
scMoGNN (69], an official winner in the overall ranking of modality prediction from the NeurIPS 2021 Competition, presents general Graph Neural Network framework facilitate multimodal single-cell data analysis, demonstrating superior performance in cross-omics prediction tasks.
H.4.4 scMM
scMM 47 is a novel deep generative model-based framework for single-cell multi-omics data
analysis (e.g.. transcriptome and chromatin accessibility). employs a mixture-of-expprts multimodal approach to extract interpretable joint representations and enable cross-modal generation, achieving end-to-end learning by modeling raw count data with modality speeific probability distributions.
H.S Biological Application: Predicting Enhancer-Gene Link and Perturbation Response
This section outlines the benchmark methods evaluated for the biological application of predicting enhancer seauncer-ggeeeeeee links and perturbation responses using ChromFound. This task leverages single-cell data to infer regulatory relationships and responses to perturbations, with a focus on maintaining biological relevance. BABEL 71 and CMAE 778, the second best models in cross-omics prediction tasks, are included in this application aim model the regulatory interactions between enhancers and genes,
34

---

I Evaluation Metrics for Downstream Tasks
This appendix details the evaluation metries used for the downstream tasks in our study, organized by task. Each metric is defined mathematically to ensure clarity and reproducibility, with ranges and performance interpretations provided where applicable.
I.1
Cell Clustering
Cell clustering is an unsupervised learning task that groups single-cell data into clusters. When ground-truth labels (e.g., cell types) available, evaluate clustering performance using the following metrics.
L1.1
Adjusted Rand Index (ARI)
The Rand Index (RI) measures the proportion of correctly grouped sample pairs:
TP TN
RI TN
where TP, TN, FP, and FN denote the numbers of true positives, true negatives, false positives, and false negatives, respectively, based on whether pairs are correctly grouped together or apart in both the clustering and ground-truth labels. The Adjusted Rand Index (ARI) corrects for chance by adjusting the expected RI under random labeling:
RI - R
ARI
max (RI) E
ranging from -1 to 1, where higher values indicate better agreement between clustering results and ground-truth labels.
1.1.2 Fowlkes-Mallows Index (FMI)
The Fowlkes-Mallon Index (FMI) combines precision and recall for clustering:
TP
TP
FMI
TP FP
TP+ FN'
where TP, FP, and FN are defined as above. It ranges from 0 to 1, with higher values indicating better clustering performance.
L1.3 Normalized Mutual Information (NMI)
NMI quantifies the shared information between clustering C and ground-truth Y:
2 C))
CMI(Y,C) H(Y) H(C)
where ((:-)) is mutual information and H(-) is entropy. It ranges from to 1, with higher values indicating greater alignment between the clustering and ground-truth labels.
I.1.4 Adjusted Mutual Information (AMI)
AMI adjusts NMI for chance by accounting for expected mutual information under random labeling:
AMI I(Y;C) E[((Y C)] max (T(Y:C)) B[((
ranging from 0 to 1, where higher values indicate a more robust agreement between clustering and ground-truth, especially in datasets with many clusters.
1.2 Denoising Batch Efffect
Denoising Batch Rffect aligns single-cell data across batches while preserving biological signals. We evaluate performance using biological conservation and batch mixing metrics implemented in [41]
35

---

1.2.1 Biological Conservation Metries
We use the following metrics to assess the preservation of biological signals, where C' denotes the set of cell types.
Normalized Mutual Information (NII
2 I(Y:C)
NMI = H(Y + H(C)
where I(::-) mutual information and H() is entropy. It ranges from 1, with higher values indicating better alignment with ground-truth cell types.
Adjusted Rand Index (ARlca))
RI [I
ARI
max (RI) [R[
TP+IN where RI TP+ and TP, TN, FP, FN are defined as in the clustering section. ranges from 1, with higher values indicating better clustering agreement with cell types.
Average Silhouette Width (ASW cell)
N
b(i) - a(i) max(a b(i))
ASW cell
N
where a(i) is the average distance of cell to others in its cell type, and b(i) is the smallest average distance to another cell type. ranges from -1 with higher values indicating better preservation of biological clustering.
Average Biological Score (AvgBIO)
AvgBIO ART + NMI ASW cell
averaging the biological conservation metrics, ranging from 0 to 1, with higher values indicating better preservation biological signals.
1.2.2 Batch Mixing Metrics
We assess batch mixing using the following metries, where denotes the set of batches.
Inverse Average Silhouette Width (ASW batch
N b(i) a(i) max(a (a(i),
ASW batch =
where aii) and b(i) are computed with respect to batch labels, with a(i) the average distance to other cells in the same batch and b(i) the smallest average distance to cells another batch. ranges from 0 to 1, with higher values indicating better batch mixing.
Graph Connectivity (GraphConn)
LCC(GANN)
GraphConn
No
CEC
where LCC(GINN) is the size of the largest connected component in the k-nearest neighbors (kNN) graph of cells ç cell type c, and N, the number of cells in c. ranges from 0 to 1, with higher values indicating better connectivity across batches.
36

---

Average Batch Score (AvgBATCH)
ASW batch GraphConn
AvgBATCH
2
averaging the batch mixing metrics, ranging from to 1, with higher values indicating better batch correction.
1.3
Cell Type Annotation
Cell type annotation supervised task that assigns cells to ground-truth cell type. We use the following metrics to evaluate performance.
1.3.1
Accuracy
Accuracy measures the proportion of correctly classified cells:
Accuracy = Eeec Ipc
EcEc No
where tp. is the number of true positives for cell type c, and N. is the total number of cells in c. ranges from with higher values indicating better classification performance.
1.3.2
Macro Score
The Macro F1 Score averages the Fl scores across all cell types:
F1. 2 Precisiin,. Recall,
Macro Fl
Σ
Fle.
[C]
Precision. + Recall,
rEC
1p where Precision, +p. and Recall, negatives for cell type c. ranges from to 1, with higher values indicating better balanced performance across classes.
p +in, with fp, and fn, as false positives and false
1.4 Cross-Omics Prediction
Cross-omics prediction infers one modality (e.g., ATAC-seq) from another (e.g., RNA-seq). We evaluate performance using correlation-bass metrics.
L4.1
Pearson Correlation Coefficient (PCC)
PCC measures the linear correlation between predicted (2x) and true (y:) values:
Σm1r (r: F) 7)
PCC
EN (z, 2) E 7)²
where 7 and y are the means of the predicted and true values, respectively. ranges from -1 with higher (closer to 1) values indicating stronger positive linear correlation.
L.4.2 Concordance Correlation Coefficient (CCC)
CCC extends PCC by accounting for bias and scale differences:
2 2.p.0z Fy
CCC
0 + (M, myy)
where p the PCC, T₂ and σg are the standard deviations of the predicted and true values, and Pa
and By are their means. It ranges from -1 to 1, with higher (closer to 1) values indicating better agreement in both correlation and scale.
1.5 Predicting Enhancer-Gene Links and Perturbation Response
These tasks involve predicting regulatory relationships or perturbation outcomes, often treated as classification problems. We use the following metrics.
37

---

15,1 Area Under the Receiver Operating Characteristic Curve (AUC-ROC)
The Receiver Operating Characteristic (ROC) curve plots the True Positive Rate (TPR) against the False Positive Rate (FPR) various classification thresholds:
TP TP FN
FP
TPR
FPR
FP + TN
where TP, TN, FP, and FN are true positives, true negatives, false positives, and false negatives, respectively, The Area Under the ROC Curve (AUC-ROC) quantifies the overall performance across all thresholds, ranging from 0 to with higher values indicating better classification performance, where represents perfect classification and 0.5 represents random guessing.
38


Reviews:
[{"review_id": "MinwWD2LI1", "reviewer": "Reviewer_dMbP", "summary": "ChromFound is a foundation model for single-cell ATAC-seq data pretrained on ~2 million cells that aims to learn batch corrected representations of cells that (a) cluster by cell type, (b) can be used to predict cell types for new cells, and (c) can be used to predict other modalities (such as RNA-seq). Across all three aforementioned tasks, the authors show that ChromFound outperforms baseline methods that train on specific datasets and are not pre-trained.\n\nThe model is pre-trained in a self-supervised fashion to predict masked accessibility values. Crucially, the model parametrizes open chromatin regions (OCRs) by their start and end location, as opposed to having a trainable embedding for a specific OCR, allowing the model to work on OCRs from new datasets, which will not exactly match OCRs from the pre-training datasets.", "questions": "Major suggestions/questions:\n1. Please compare to non-pretrained ChromFound for all experiments, not just Fig. 3, Table 3, and Table 4. \n2. Benchmark against Signac (TF-IDF + LSI), peakVI/poissonVI, and SnapATAC2 for cell clustering tasks, as these are the methods most commonly used by practioners. For each, the appendix should indicate the exact pre-processing done before applying the method (e.g. are peaks filtered by their observation frequency before passing them into peakVI?). \n3. Do not use scVI/scANVI for batch-effect correction, as they are designed for RNA-seq. Use peakVI/poissonVI instead.\n4. What input representations do you pass to Harmony for batch-effect correction? PCA or LSI or neither?\n5. What l2 regularization parameter do you use for scBasset for batch-effect correction? How do you choose this regularization parameter?\n6. A small linear layer on top of low-dimensional representations learned by Signac, peakVI, and SnapATAC2 can also be used for cell type annotation. Please compare to those baselines.\n \n\nMinor suggestions/questions:\n1. At test time, when cell type labels are unavailable, how does OCR filtering (as described on lines 611-612) work?\n2. Why do you use an MSE loss after log transformation instead of a Poisson loss on the raw counts, which likely better fits the generative process. \n3. In Fig. 4, are you using ABC predictions as ground-truth? This seems suspect as ABC itself is far from perfect and has several biases (i.e. overly discounting distal enhancers). \n4. Question 3 in the ablations asks if long-context modeling using Mamba layers helps, but I'm not sure if the experiment performed actually answers that question. My reading of Table 4 is that half and three-fourths of the OCRs are just removed from the input data, but that doesn't test the utility of the Mamba layers directly. A better test would be to use all OCRs, but pool after the WPSA layer without having any Mamba layers.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "To my knowledge, ChromFound is the first foundation model for scATAC-seq data. In my opinion, the paper's primary contribution is the demonstration that pre-training on millions of scATAC-seq cells enables zero-shot representation learning for completely new datasets, without requiring additional training or fine-tuning. (I do not think the model itself minus the pre-training is a general advance given the results in Fig. 3 and Table 3, which show that baselines are competitive with a non-pretrained ChromFound model.) This type of pre-training has mostly not been done before because different datasets have different peak/OCR boundaries and harmonizing peak sets is not straightforward. The authors cleverly solve this problem by parametrizing OCRs by a function of their start and end positions, as opposed to using an embedding table. This I find to be their primary computational contribution.\n\nHowever, there are other methods of harmonizing peak sets that would enable other architectures (such as a standard VAE) to be used for pre-training that the authors do not compare to. Computational biologists routinely (i) aggregate fragment files from multiple datasets to re-call a unified set of peaks or (ii) adopt the peak set of a large reference atlas. If fragment files are available for the new dataset, then peak counts can be called specifically against the reference peak set. But even if fragment files are not available and only a count matrix is, counts from the new peak set can be transferred to the reference peak set by examining peak overlaps. Demonstrating that ChromFound outperforms (a) both itself with a different harmonization strategy and (b) other methods pre-trained using one of the aforementioned harmonization strategies would substantially strengthen the paper.\n\nThe other notable weakness of the paper is the benchmarking to baselines. The authors omit several standardly used tools, do not systematically compare against the same tools for all tasks even though most methods are generally applicable, and crucially do not detail how they implement the baselines. For cell clustering and batch-effect correction, Signac, peakVI, and SnapATAC2 are the most commonly used tools and should be compared against.  For cross-omics predictions, Signac and multiVI should also be compared against. Additional recommendations are listed in the \"Questions\" section.", "was_revised": true, "final_justification": "I've increased my score from a 3 to a 4 based on the authors re-running of baselines with more suitably chosen hyperparameters. It seems like this method is at least as good as existing methods. And even if further hyperparameter optimization of the baselines reveals that this method is only slightly better than them (which I suspect to be the case), it at least offers a novel and interesting approach to merge OCRs from different datasets that will be valuable to the single-cell ATAC-seq community.", "rebuttal": "We appreciate the reviewer for recognition of ChromFound’s novelty and constructive feedbacks. Below, we summarize our key revisions and additional analyses in response:\n\n## [1]. Benchmark completeness\n\n1. We expand the clustering benchmarks (Table 1) by adding Signac (Sig), SnapATAC2 (SAT2), peakVI (pkVI), and poissonVI (psVI). All methods use the same preprocessing (Appendix C). Signac, peakVI, and poissonVI skip log normalization; scBasset uses binarized input.\n\n|Dataset|Model|ARI|FMI|NMI|AMI|\n|-|-|-|-|-|-|\n|Cortex(batch 1)|Sig|0.5013|0.7035|0.5580|0.5534|\n||SAT2|0.5118|0.6740|0.7202|0.7179|\n||pkVI|0.4558|0.6286|0.6964|0.6940|\n||psVI|0.3673|0.5673|0.5878|0.5845|\n||Ours|0.6890|0.7943|0.7779|0.7760|\n|Cortex(batch 2)|Sig|0.5069|0.6478|0.5888|0.5851|\n||SAT2|0.5325|0.6380|0.7038|0.7015|\n||pkVI|0.5512|0.6530|0.7050|0.7028|\n||psVI|0.5334|0.6388|0.6004|0.5983|\n||Ours|0.6278|0.7156|0.7321|0.7299|\n|Heart(av 3)|Sig|0.3123|0.5093|0.2294|0.2281|\n||SAT2|0.4571|0.7285|0.6867|0.6863|\n||pkVI|0.4159|0.6383|0.6149|0.6144|\n||psVI|0.3969|0.5929|0.4472|0.4464|\n||Ours|0.5828|0.6757|0.7207|0.7187|\n|Heart(av 10)|Sig|0.1159|0.4601|0.5272|0.5238|\n||SAT2|0.5515|0.5564|0.6625|0.6502|\n||pkVI|0.4301|0.5434|0.6024|0.6000|\n||psVI|0.3606|0.5232|0.5717|0.5688|\n||Ours|0.6774|0.8109|0.7369|0.7365|\n|Retina(D026_13)|Sig|0.5538|0.7451|0.4931|0.4919|\n||SAT2|0.5606|0.7207|0.7045|0.7041|\n||pkVI|0.2885|0.5543|0.5644|0.5638|\n||psVI|0.3162|0.5766|0.6081|0.6076|\n||Ours|0.6668|0.8149|0.7644|0.7641|\n|Retina(D19D008)|Sig|0.5146|0.7339|0.5235|0.5222|\n||SAT2|0.5814|0.7173|0.7881|0.7877|\n||pkVI|0.4702|0.6318|0.7229|0.7224|\n||psVI|0.4978|0.6527|0.7400|0.7395|\n||Ours|0.6688|0.7767|0.8183|0.8179|\n|PBMC(VIB_10xv1_1)|Sig|0.1832|0.4543|0.3064|0.3027|\n||SAT2|0.6246|0.7085|0.7551|0.7541|\n||pkVI|0.5417|0.6340|0.6639|0.6627|\n||psVI|0.6314|0.7073|0.7353|0.7343|\n||Ours|0.6953|0.7601|0.7860|0.7852|\n|PBMC(BIO_ddseq_1)|Sig|0.1885|0.5249|0.2852|0.2839|\n||SAT2|0.4126|0.6149|0.5761|0.5754|\n||pkVI|0.2915|0.5041|0.4623|0.4615|\n||psVI|0.4103|0.5994|0.5656|0.5649|\n||Ours|0.4835|0.6604|0.5950|0.5944|\n\n2. We expand batch correction evaluation (Table 2), including peakVI (pkVI) and poissonVI (psVI) to replace scVI and scANVI.\n\n|Tissue|Model|AvgBIO|AvgBATCH|\n|-|-|-|-|\n|Bone|pkVI|0.2981|0.9173|\n||psVI|0.2936|0.9178|\n||Ours|0.6408|0.9289|\n|Heart|pkVI|0.5657|0.7564|\n||psVI|0.5621|0.7719|\n||Ours|0.8180|0.8679|\n|PBMC|pkVI|0.6172|0.7889|\n||psVI|0.6029|0.8081|\n||Ours|0.6443|0.8217|\n|Cortex|pkVI|0.7256|0.9254|\n||psVI|0.7240|0.9279|\n||Ours|0.7440|0.9565|\n\n3. The descriptions for all benchmark methods are provided in Appendix H. The APIs and tools used are as follows:\n     - scBasset, peakVI, poissonVI, scVI: scvi.model\n     - Liger: the pyliger Python package\n     - Scanorama, Harmony: scanpy.external\n     - scANVI: scib.integration\n     - Cross-omics benchmarks: DANCE.modules.multi_modality.predict_modality\n     - Others: official source codes and tutorials from their respective publications\n\n    To directly address the reviewer’s question:\n     - L2 regularization parameter for scBasset is set to 1e-8;\n     - Input representation passed to Harmony is PCA.\n\n4. We include logistic regression results in cell type annotation (Fig. 3) using Signac (Sig), peakVI (pkVI), and SnapATAC2(SAT2) embeddings. The results of baselines are much worse than ChromFound's.\n\n|Tissue|Train|Test|Method|Accuracy|F1|\n|-|-|-|-|-|-|\n|PBMC|EPF_hydrop_1|VIB_10xv1_1|Sig|0.3546|0.1643|\n||||pkVI|0.2756|0.1430|\n||||SAT2|0.1256|0.1751|\n|PBMC|EPF_hydrop_3|VIB_10xv1_2|Sig|0.0841|0.0488|\n||||pkVI|0.1286|0.1961|\n||||SAT2|0.3548|0.1565|\n|Bone|batch_27|batch_26|Sig|0.2665|0.2477|\n||||pkVI|0.2107|0.1939|\n||||SAT2|0.4897|0.3525|\n|Bone|batch_43|batch_44|Sig|0.1597|0.1764|\n||||pkVI|0.2118|0.1795|\n||||SAT2|0.4479|0.4540|\n|Cortex|batch_2|batch_1|Sig|0.0240|0.0216|\n||||pkVI|0.0973|0.0717|\n||||SAT2|0.4007|0.2635|\n|Cortex|batch_3|batch_2|Sig|0.4930|0.3629|\n||||pkVI|0.5145|0.2610|\n||||SAT2|0.0880|0.0843|\n|Retina|D19D003|D018_13|Sig|0.0353|0.1037|\n||||pkVI|0.0935|0.0763|\n||||SAT2|0.0000|0.0000|\n|Retina|D021_13|D19D003|Sig|0.8088|0.5642|\n||||pkVI|0.0267|0.0393|\n||||SAT2|0.3032|0.2500|\n\n5. Clarification on non-pretrained ChromFound:\n\n   In manuscript, we have already included non-pretrained ChromFound as a baseline in Fig. 3 and Table 3. We emphasize that for tasks such as cell clustering and denoising, ChromFound is evaluated in zero-shot settings. Non-pretrained ChromFound loading randomly initialized weights in these scenarios is not a meaningful or fair baseline. We hope the reviewer understands our rationale, and we are happy to further discuss this point if needed.\n\n## [2]. Other peak calling methods and VAE pretraining\nWe sincerely thank the reviewer for insightful suggestions. In response, we perform an experiment to directly compare ChromFound with VAE-based models trained on reference-peak-aligned data. Specifically, we adopt the cPeaks reference set proposed by the recent work \"A generic reference defined by consensus peaks for scATAC-seq data analysis\", which defines 1,657,194 peaks across the human genome. We use a 20,000-cell PBMC subset (same as in Table 4, Row 6) as the training data. After mapping peaks to the cPeaks reference and filtering out peaks and cells with all-zero values, we obtain a training set of 306,784 peaks and approximately 18,000 cells.\n\nWe train four VAE-based baselines on the aligned data and compare their performance on zero-shot cell clustering (Table 1) against ChromFound trained on the same aligned data (Ours). Evaluation metrics and datasets follow Table 1, all aligned to 306,784 peaks. Results are summarized below:\n\n|Dataset|Model|ARI|FMI|NMI|AMI|\n|-|-|-|-|-|-|\n|Cortex(batch 1)|SCALE|0.2932|0.5050|0.4127|0.4081|\n||SCALEX|0.4230|0.6078|0.5543|0.5513|\n||pkVI|0.3690|0.5643|0.5023|0.4983|\n||psVI|0.3364|0.5408|0.4636|0.4594|\n||Ours|0.5858|0.7244|0.7519|0.7499|\n|Cortex(batch 2)|SCALE|0.2484|0.4052|0.3657|0.3609|\n||SCALEX|0.4481|0.5679|0.5742|0.5710|\n||pkVI|0.3610|0.5035|0.4876|0.4835|\n||psVI|0.2820|0.4374|0.4104|0.4058|\n||Ours|0.5751|0.6720|0.7145|0.7123|\n|Heart(av 3)|SCALE|0.3543|0.4817|0.5228|0.5195|\n||SCALEX|0.2795|0.4150|0.4787|0.4752|\n||pkVI|0.3023|0.4437|0.4816|0.4779|\n||psVI|0.3778|0.5025|0.5446|0.5414|\n||Ours|0.5400|0.6347|0.6588|0.6567|\n|Heart(av 10)|SCALE|0.4051|0.6506|0.4461|0.4453|\n||SCALEX|0.3013|0.5685|0.4199|0.4191|\n||pkVI|0.3966|0.6080|0.5018|0.5011|\n||psVI|0.3939|0.5997|0.5010|0.5003|\n||Ours|0.6150|0.7695|0.6968|0.6963|\n|Retina(D026_13)|SCALE|0.3065|0.5771|0.4447|0.4440|\n||SCALEX|0.2430|0.5149|0.4127|0.4117|\n||pkVI|0.2514|0.5260|0.4651|0.4644|\n||psVI|0.2347|0.5122|0.4063|0.4055|\n||Ours|0.4888|0.7081|0.7241|0.7238|\n|Retina(D19D008)|SCALE|0.3143|0.5092|0.4220|0.4210|\n||SCALEX|0.2490|0.4542|0.3920|0.3909|\n||pkVI|0.2664|0.4693|0.4745|0.4736|\n||psVI|0.2396|0.4473|0.3937|0.3926|\n||Ours|0.5368|0.6839|0.7664|0.7660|\n|PBMC(VIB_10xv1_1)|SCALE|0.5769|0.6756|0.7066|0.7057|\n||SCALEX|0.5718|0.6514|0.7205|0.7196|\n||pkVI|0.5306|0.6249|0.7057|0.7046|\n||psVI|0.5593|0.6479|0.6742|0.6732|\n||Ours|0.5999|0.6827|0.7434|0.7424|\n|PBMC(BIO_ddseq_1)|SCALE|0.4031|0.5797|0.5380|0.5374|\n||SCALEX|0.3326|0.5430|0.5465|0.5458|\n||pkVI|0.3830|0.5804|0.5737|0.5731|\n||psVI|0.3959|0.5917|0.5509|0.5503|\n||Ours|0.4347|0.6237|0.5867|0.5861|\n\nWe highlight two key observations from this experiment:\n1. Baseline models perform poorly on unseen tissues. We find that only 30% of peaks in retina, 35% in cortex, and 70% in heart overlap with the cPeaks-aligned PBMC training peaks, suggesting that cross-tissue peak heterogeneity is a major factor limiting generalization.\n2. Using cPeaks reduce ChromFound's performance compared to its native OCRs, suggesting that reference-based harmonization cannot capture novel or shifted OCRs introduced by batch effects or disease-specific variation. In contrast, ChromFound’s dynamic OCR tokenization allows the model to implicitly learn genomic distance and neighborhood structures, enhancing both robustness and extensibility.\n\n## [3]. Minor questions\n1. _OCR filtering when cell type labels are unavailable_\n\n   When cell labels are unavailable, we apply TF-IDF + LSI to infer pseudo labels for OCR filtering. We will clarify this in the manuscript.\n2. _Choice of loss function_\n\n   Due to the high sparsity of scATAC-seq data, we adopt a dual masking strategy (Sec 3.3) during pretraining, masking equal number of both nonzero and zero peaks. Under this setting, Poisson/ZINB losses often cause unstable optimization due to gradients dominated by nonzero entries. Instead, MSE loss on log-transformed, normalized signals ensures stability. This choice is also supported by xTrimoGene discussed in both their manuscript and rebuttal.\n3. _Ground truth in Fig. 4_\n\n   We would like to clarify that the ground truth labels in Fig. 4 are not ABC predictions, but experimental CRISPRi perturbation data in the Fulco4K dataset, as stated in lines 235–236 of the manuscript. Additionally, the citation of Fulco4K dataset will be corrected as \"Activity-by-contact model of enhancer–promoter regulation from thousands of CRISPR perturbations\".\n4. _Ablation study on the Mamba layer_\n\n   Thank you for the comment. To clarify, Question 3 in our ablation study is designed to assess the importance of long input peak sequences, not the effectiveness of the Mamba layers. Many existing scATAC-seq methods apply dimensionality reduction (e.g., highly variable peaks or LSI), but we aim to show that modeling long-range chromatin context leads to better cell representations and cross-omics performance. \n   \n   The effectiveness of the Mamba layers is directly evaluated in Question 1. The significant performance drop in this setting highlights the contribution of each component, especially Mamba layer.\n\n## Conclusion\nWe sincerely thank the reviewer for these thoughtful and professional suggestions. We hope our responses have addressed the reviewer’s concerns. Please let us know if further clarification is needed."}, {"review_id": "LEoDYBZ87b", "reviewer": "Reviewer_ybr7", "summary": "This paper presents ChromFound, the first foundation model developed specifically for single-cell chromatin accessibility (scATAC-seq) data. The authors propose a novel hybrid architecture that leverages a Mamba block for long-range genomic context and a windowed self-attention (WPSA) module for local regulatory interactions. A core contribution is the genome-aware tokenization scheme that encodes genomic coordinates and continuous accessibility values, addressing the challenge of heterogeneous data inputs. After pre-training on a large-scale corpus of 1.97 million human cells, the model is comprehensively evaluated across six downstream tasks, where it demonstrates state-of-the-art performance and notable zero-shot capabilities.", "questions": "1.  Could you elaborate on the design choice to project to a lower-dimensional space ($D_{low}=32$) for the Mamba block? What were the primary trade-offs you considered between performance and computational efficiency (e.g., parameter count, inference speed)?\n2.  The zero-shot performance for cell representation is quite strong, yet fine-tuning yields considerable gains for tasks like cell type annotation. What is your interpretation of this? Does it suggest that the pre-trained model captures general cell state well, but that cell-type-specific regulatory grammars are only fully resolved during task-specific fine-tuning?\n3.  Regarding the positional embeddings, could you comment on the model's sensitivity to the `temp` hyperparameter? Is performance stable across a range of values, or was this a carefully tuned parameter?\n4.  The fixed window for WPSA is well-justified for typical enhancer interactions. Have you explored its potential limitations for capturing regulatory phenomena over larger genomic distances, and is the Mamba block intended as the primary mechanism for these cases?\n5.  Given the pre-training corpus, what are your expectations for ChromFound's zero-shot or few-shot performance on cell types from disease contexts not well-represented in the training data, such as autoimmune disorders?", "limitations": "yes\n\nThe authors appropriately acknowledge the model's human-centric focus as a primary limitation. For a more comprehensive discussion, I would suggest also including a brief commentary on two other practical points: (1) the model's implicit reliance on the quality of user-provided peak calls, which is an important contextual factor for anyone applying the model, and (2) the practical compute resources required for fine-tuning, which is relevant for assessing the model's accessibility to the broader research community.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n\n* The work successfully establishes the first foundation model for a challenging and important biological data modality. This is a novel and impactful contribution that charts a new direction for the field.\n* The scale of the pre-training is impressive and a key factor in the model's strong performance. The subsequent evaluation is both rigorous and comprehensive, providing convincing evidence for the model's capabilities across a wide array of relevant tasks.\n* The hybrid Mamba-WPSA architecture is not merely a concatenation of popular methods; it's a well-justified design choice that reflects the multi-scale nature of genomic regulation, which is a commendable feature.\n\n**Weaknesses**\n\nWhile the work is strong overall, there are several areas where a deeper analysis would strengthen the paper's conclusions and provide a more complete picture for the research community. I've ordered these from minor points to more substantial considerations.\n\n* On a minor note, while key hyperparameters are provided, the paper would benefit from a brief discussion on the model's sensitivity to these choices (e.g., the `temp` value in positional embeddings or the Mamba projection dimension). This would provide readers with a better sense of the model's robustness and tuning requirements.\n* A more significant point is that the ablation study, while useful, could be more comprehensive. The current study primarily demonstrates that removing major architectural blocks is detrimental. However, it leaves open important questions about the sources of performance gain. For instance, a comparison to simpler, strong baselines (like a pure Mamba or a pure Transformer architecture) trained on the same massive dataset would be invaluable. Such an analysis would help disentangle the benefits of the proposed hybrid design from the undeniable power of the large-scale pre-training data.\n* Finally, the most substantial area for improvement relates to the model's generalizability and its dependency on upstream analytical choices. The model's performance is intrinsically linked to the input set of Open Chromatin Regions (OCRs), which can vary significantly based on the peak-calling algorithm used. A sensitivity analysis or at least a discussion on this crucial variable would be necessary to fully substantiate the model's claim of universality. Similarly, the model's impressive performance may not fully generalize to disease contexts (e.g., autoimmune disorders) that are not well-represented in the pre-training corpus, a key consideration for its application as a foundational tool for discovery.", "was_revised": true, "final_justification": "Authors have effectively addressed the key issues I raised:\n\n**Hyperparameter Analysis**: The systematic evaluation of both the positional embedding temperature and Mamba projection dimension provides exactly the type of sensitivity analysis I was looking for. The clear trade-off between performance and computational efficiency (particularly the memory constraints at d_proj=128) gives readers valuable guidance for practical implementation.\n\n**Enhanced Ablation Study**: Your comparison with pure Mamba and the creative approach to Transformer baselines (using Geneformer and scGPT as proxies) convincingly demonstrates that the performance gains stem from your hybrid architectural design, not merely from large-scale pretraining. The ~15% ARI drop when removing WPSA and the fundamental limitations of existing Transformer-based models on this data modality strengthen your architectural claims significantly.\n\n**Robustness and Generalization**: Your clarification on coordinate-based tokenization and the existing disease representation in your evaluation datasets addresses my concerns about generalizability. The downsampling experiments provide good evidence for robustness to preprocessing variations.\n\nThe addition of computational profiling results also addresses practical deployment considerations, which will be valuable for the research community.\n\nYour rebuttal demonstrates not only technical rigor but also a deep understanding of the biological context and practical implications of your work. The responses show that ChromFound represents a well-engineered solution to genuine challenges in single-cell chromatin accessibility analysis.\n\n**Updated Rating: 5 (Accept)** - This is technically sound work with clear contributions that will be valuable to the research community. The comprehensive evaluation and your responsive improvements to the analysis strengthen confidence in the work's impact and reliability.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and detailed feedback. We appreciate your recognition of ChromFound’s novelty, the biologically informed architectural design, and the breadth of downstream evaluations. Below, we address each of your suggestions and concerns.\n\n## [W3,L1] Discussion on Peak-calling Methods\nThank you for the important point. We agree that different upstream data pipelines can lead to variability in the cell-by-peak matrix and peak sets, particularly in **sparsity** and **peak boundary definitions**. ChromFound is designed to be robust to these variations. \n\n- To assess robustness to such sparsity differences, we perform downsampling experiments (Figure 2) simulating varying levels of accessible peaks per cell. ChromFound maintains stable performance across all conditions, indicating strong resilience to sparsity variation introduced by upstream preprocessing.\n\n- To address potential peak shift caused by different peak calling methods, ChromFound encodes OCRs using sinusoidal positional embeddings of their genomic coordinates. This design enables the model to capture relative spatial relationships between peaks, enhancing robustness to peak boundary shifts. Unlike dictionary-based approaches, our coordinate-based tokenization reduces reliance on exact peak definitions. We indeed conduct experiments aligning peaks across datasets to validate ChromFound’s robustness to peak shift. Due to space limitations, we kindly refer the reviewer to our response to Reviewer dMbP’s comment 2 for detailed results. We appreciate your understanding.\n\nIn summary, while the pretraining and evaluation datasets are from different upstream analytical tools, ChromFound still achieves strong performance and generalization. We will include a discussion of these points in the revised manuscript. Please feel free to discuss further if you have additional concerns.\n\n## [W3,Q5] Generalization to Disease Contexts\nWe appreciate your interest in ChromFound's generalization to disease contexts. As detailed in Appendix B Table 5, our pretraining corpus does span **6 disease types**, including Alzheimer’s, Parkinson’s, leukemia, glioma, myocardial infarction and breast cancer.\n\nMoreover, we have already included two disease benchmark datasets in downstream tasks: Morabito130K, derived from **Alzheimer’s disease**, and Kuppe139K, a heart dataset from patients with **myocardial infarction**. ChromFound demonstrates strong performance on both datasets (Table 1, Figure 2, Table 2), supporting the model’s robustness in disease contexts.\n\nWe hope these results address the reviewer’s concerns. In addition, We have not found a scATAC-seq dataset of autoimmune disorders with explicit cell labels. If the reviewer has specific datasets in mind, we would be happy to explore them further.\n\n## [W2] More Comprehensive Ablation Study\nThank you for the insightful suggestion. As requested, we compare ChromFound to both a pure Mamba and a pure Transformer baseline under the same pretraining corpus.\n1. The pure Mamba model (Row 3 of Table 4) removes WPSA and uses Mamba alone. This results in a ~15% ARI drop, confirming that local attention is critical and that the hybrid design contributes synergistically to performance.\n2. Training a vanilla Transformer on scATAC-seq inputs containing nearly one million peaks is practically infeasible due to its quadratic scaling in memory and computation. To approximate a pure Transformer baseline under practical settings, we train two representative single-cell foundation models, Geneformer and scGPT, on the same pretraining corpus with increasing peak lengths (4k–32k) and the same hyperparameters as WPSA. The results of comparison on the cell clustering task using the PBMC169K (batch VIB_10xv1_1) dataset are detailed in the table below.\n    - Geneformer sorts peaks by accessibility value before input, which disrupts the native genomic order. As a result, it fails to learn meaningful representations.\n    - scGPT applies a highly variable OCR selection strategy, limiting inputs to a small subset of peaks. Performance saturates at ARI ≈ 0.39, constrained by the maximum input length that Transformers can handle efficiently.\n\n|Method|Peak Length|ARI|\n|-|-|-|\n|Geneformer|4096|0.0451|\n||16384|0.0457|\n||32768|0.0460|\n|scGPT|4096|0.3075|\n||16384|0.3774|\n||32768|0.3868|\n|ChromFound|363066|0.6953|\n\n## [W1,Q1,Q3] Model Sensitivity to Hyperparameters\n\nThank you for this helpful suggestion. We have evaluated both hyperparameters during early-stage model development on the cell clustering task using PBMC169K (batch VIB_10xv1_1) dataset. The results and findings are summarized below.\n\n1. Positional embedding temperature: This parameter controls the frequency of sinusoidal position encodings and can be viewed as a proxy for “genomic resolution.” As shown below, performance slightly drops when temp is within 1e3 to 1e5, and degrades more substantially when temp is too large, likely due to loss of relative position sensitivity.\n\n|$temp$|ARI|NMI|AMI|FMI|\n|-|-|-|-|-|\n|1000|0.6535|0.7764|0.7755|0.7259|\n|10000|0.6512|0.7709|0.7701|0.7243|\n|100000(Ours)|0.6953|0.7860|0.7852|0.7601|\n|1000000|0.6036|0.7344|0.7334|0.6860|\n|10000000|0.5852|0.7269|0.7258|0.6714|\n\n2. Mamba projection dimension: This parameter controls the compression within the Mamba block. As shown below, $D_{low}$ = 32 achieves a strong balance between performance and efficiency. Our choice of 32 is primarily motivated by the trade-off between performance and computational efficiency. Larger values lead to marginal gains but significantly increase FLOPs and memory, with $D_{low}$ = 128 exceeding the memory limits on A100 80G GPUs.\n\n|$D_{low}$|Parameter Count|FLOPs|Inference Speed (s/sample)| GPU memory (GB) |ARI|\n|-|-|-|-|-|-|\n|16|422,593|7.41E+11|3.4027|48.8|0.6158|\n|32(Ours)|450,305|7.89E+11|3.4624|60.7|0.6953|\n|64|518,785|9.09E+11|4.0053|72.3|0.6927|\n|128|707,969|1.24E+12|OOM|OOM|OOM|\n\nWe hope these results address the reviewer's concerns. We will include this analysis in the revised manuscript.\n\n## [Q2] Interpretation of Fine-tuning Benefits\n\nThank you for the thoughtful and professional insight, and we strongly agree with your interpretation. The strong zero-shot performance indeed suggests that ChromFound learns generalizable representations of cellular states and underlying regulatory structures during pretraining. These representations are already informative for unsupervised clustering and can reveal biologically relevant subpopulations.\n\nAs you rightly pointed out, fine-tuning further improves performance by enabling the model to specialize in cell-type-specific regulatory features. Such features often involve subtle patterns or rare combinations of regulatory elements that are better resolved with supervision. This fine-tuning complements the global regulatory landscape already captured during pretraining. In addition, we have provided a detailed analysis of the cell type annotation confusion matrix in Appendix F.2, demonstrating ChromFound’s advantage in recognizing rare and long-tail cell types and cell-type-specific regulatory patterns.\n\nWe appreciate your expert perspective and will incorporate this interpretation into our revised discussion. Your feedback greatly enhances the clarity of our presentation and strengthens our understanding of how pretraining and fine-tuning synergize in ChromFound.\n\n## [Q4] Fixed WPSA Window\nThank you for raising this important point. Both Hi-C[1] and CRISPRi[2] have shown that the vast majority (over 90%) of validated enhancer-gene interactions occur within 200 kb of the transcription start site (TSS). Theoretically, the Mamba block modeling genome-wide dependencies allows ChromFound to infer potential distal enhancer-gene links. That said, we acknowledge that experimental validation of distal enhancer–gene links (>200 kb) remains limited, we view this as an important direction for future work. If the reviewer is aware of suitable datasets, we would be happy to incorporate such experiments.\n\n[1] Bonev, B. & Cavalli, G. Organization and function of the 3D genome. Nat Rev Genet 17, 661–678 (2016).\n\n[2] Fulco, C. P. et al. Activity-by-contact model of enhancer–promoter regulation from thousands of CRISPR perturbations. Nat Genet 51, 1664–1669 (2019).\n\n## [L2] Compute Resources for Fine-tuning\nWe thank the reviewer for highlighting the importance of computational efficiency. We provide profiling results on cell type annotation and cross-omics prediction tasks using a single A100 80G GPU:\n\n|Task|Dataset|Peak Length|Batch Size|Training Speed (s/step)|Peak Memory Usage (GB)|\n|-|-|-|-|-|-|\n|Cell Type Annotation|Bone 43/44|495,416|4|0.535|67.8|\n|Cross-omics Prediction|BMMC multiome 2021|269,544|4|1.060|56.7|\n\nWe note that the primary driver of memory consumption is the length of the input peak sequence, which can reach several hundred thousand tokens per cell. In contrast, the model architecture itself remains lightweight (parameter count: 450305). All current resource metrics will be included in the appendix to guide users in estimating compute requirements. In future work, we aim to further optimize memory efficiency and training speed for broader applications.\n\n## Conclusion\nWe hope these responses address your concerns and questions. We sincerely appreciate your valuable feedback and constructive suggestions. If you have any additional questions or suggestions, please feel free to let us know."}, {"review_id": "cdkumqcxAm", "reviewer": "Reviewer_2Z9j", "summary": "The authors proposed ChromFound, an encoder-decoder framework to build foundation model for scATAC-seq data. Genomic coordinate information and chromatin accessibility profile were used to represent open chromatin regions (OCR). The authors proposed window partition self-attention (WPSA) module to capture local dependencies among OCRs, and Mamba layer to handle ultra-long OCRs. The decoder is MLP layer.   \nChromFound shows improved performance on cell representation accommodating low count and batch effect, which are popular in scATAC-seq data. ChromFound also improves cell type annotation and cross -omics data prediction.Application of ChromFound on enhancer-gene link prediction and perturbation response prediction shows the results make biological sense.", "questions": "1. it is interesting to see how will it perform on gene regulatory network prediction?\n2. A nature way is to incorporate DNA sequence  and or scRNA-seq information into the model, since DNA sequence is already there and scRNA-seq data is more widely available.\n3. minor comments: \n   reference to Table 4.4 should be Table 4\n   In figure 6, the order of figure in Harmony panel is reversed compared with other methods.", "limitations": "1. The authors forget to include limitations, such as the model is only on human data, which can be extend to include mouse and other species", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "The submission is technically sound.  The experimental results are well supporting the authors' claim. The methods used are appropriate. The work is almost complete piece of work. The authors mentioned strengths of their work carefully and honestly.\n\nThe submission is clearly written and well organized. It adequately inform the reader. No source code was provided.\n\nThe authors propose for the first time foundation model for scATAC-seq and achieve promising results compared with existing approaches.  It is very likely other researchers will utilize or extend the approach to address more challenges in scATAC-seq data analysis. \n\nThe authors offer a novel combination of existing techniques, and the reasoning behind this combination is well-articulated. The work addresses challenges in generating foundation model in scATAC-seq with suitable techniques and adaption of existing technique when needed. The authors clearly distinguish their contributions with previous contributions.", "was_revised": true, "final_justification": "The authors have provided insights on future work on incorporating DNA sequence, scRNA-seq, and gene regulatory network to address my comments.", "rebuttal": "We sincerely thank you for your thoughtful and constructive feedback. We are encouraged by your recognition of the technical soundness, comprehensive evaluation, and potential impact of ChromFound. Below we address your comments in detail.\n\n## [W1] Source Code\nWe appreciate your comment regarding code availability. We are fully committed to promoting transparency and reproducibility. Upon acceptance, we will release the complete source code, pretrained weights and tutorials for all experiments.\n\n## [Q2] DNA sequence modeling\nThank you for this insightful suggestion. We totally agree that DNA sequence information can complement chromatin accessibility data by providing base-resolution regulatory context. We note that scBasset also incorporates local DNA sequence to predict binary chromatin accessibility and has demonstrated strong performance in cell representation and batch correction. \n\nWhile ChromFound currently does not explicitly encode DNA sequence, our tokenization schema can be extended to include local sequence features around each OCR. This would allow the model to jointly learn from static genomic sequences and dynamic chromatin accessibility profiles.\nOne possible way of integrating DNA sequence is to fuse the DNA sequence embeddings from DNA foundation models (e.g., Evo2, Nucleotide Transformer, Enformer, etc.) into the ChromFound architecture. This approach would allow ChromFound to benefit from both dynamic chromatin activity and static regulatory code, potentially improving its ability to resolve fine-grained enhancer-gene interactions. We will explore this direction in future work.\n\n## [Q2] scRNA-seq modeling\nThank you for your professional suggestion. We totally agree that incorporating scRNA‑seq is both natural and valuable. Many current methods jointly model scATAC‑seq and scRNA‑seq by using scATAC‑seq as input to predict scRNA‑seq expression (e.g., GET) or perform multi‑omics integration through shared embeddings (e.g., GLUE, scMODAL).\n\nHowever, these approaches typically treat peaks and genes as separate modalities linked only in downstream alignment. A current limitation in existing approaches is the lack of modeling under a unified genomic scale. We hope that the unified genomic scale will place peaks and genes on the same coordinate framework to enable end‑to‑end training from the genome sequence to transcriptomic output.\n\nIn our future work, we aim to bridge this gap by pretraining ChromFound in a truly genomic-centered manner: jointly embedding OCRs and genes in the same latent space from the start, enabling the model to internalize the central dogma (i.e., the information flow from DNA to RNA to protein) via self‑supervised objectives. While paired single-cell scATAC‑seq and scRNA‑seq data at sufficient scale remain limited for pretraining, we are actively pursuing large‑scale collection and simulation strategies to realize this vision.\n\n## [Q1] GRN prediction\nWe appreciate the reviewer’s suggestion on GRN prediction. ChromFound is currently designed to infer _cis_-regulatory interactions, such as enhancer-gene links and perturbation responses. Looking forward, we aim to expand ChromFound to support broader gene regulatory network (GRN) inference.\n\nAs discussed in the response regarding scRNA-seq modeling, we envision extending ChromFound to jointly model scATAC-seq and scRNA-seq data in a truly genomic-centered manner. This unified framework would allow the model to capture not only local _cis_-regulatory dependencies, but also _trans_-regulatory interactions. \nWhile data limitations currently constrain pretraining across paired multi-omics at single-cell resolution, we are actively exploring strategies to realize this integrative modeling paradigm in future work.\n\n## [Q3] Minor comments\nThank you for pointing this out. We will correct the table reference to Table 4 and revise Figure 6 in the revised manuscript to maintain consistent panel ordering.\n\n\n## [L1] Limitations on Human Data\nWe appreciate your suggestion to more explicitly discuss limitations. While we acknowledge that mouse datasets are often more abundant and widely available, our motivation is to prioritize human data due to its direct relevance for studying disease mechanisms and interpreting non-coding variants associated with human traits. We believe that this choice has greater significance for advancing biomedical research and understanding human-specific regulatory programs.\n\nNevertheless, extending ChromFound to other species is an important future direction. Our genome-aware tokenization is inherently adaptable, but cross-species modeling will require additional algorithmic revisions, such as species-aware OCR tokenization and the alignment of genomic coordinates across organisms. We plan to explore multi-species pretraining and cross-species transfer learning in future work.\n\n## Conclusion\nOnce again, we truly thank the reviewer for the valuable comments and strong endorsement of our work. We hope our responses have addressed your concerns, and we are happy to provide further clarifications if needed."}, {"review_id": "G0zKsbEawb", "reviewer": "Reviewer_VAFg", "summary": "The authors introduce a foundation model specifically designed for single-cell chromatin accessibility (scATAC-seq) data. The authors propose a hybrid encoder architecture that integrates Mamba blocks with local windowed self-attention and introduces a genome-aware tokenization scheme for encoding open chromatin regions (OCRs). Trained on 1.97 million cells from over 30 tissues and 6 disease conditions, ChromFound is evaluated on a broad set of downstream tasks: zero-shot cell representation, cell type annotation, cross-omics prediction, and enhancer-gene link inference.", "questions": "1. The authors should compare Chromfound with linear regression in cell type prediction and cross-omics prediction\n2. How do the authors unify the peaks to adapt Chromfound for different datasets?\n3. Does Chromfound achieve different performance across different tissues for all tasks?\n4. Do the authors compare the performance and computational complexity of WPSA and linear attention?\n4. Statistical Significance: Did you do any random seed runs or significance tests for the modest performance gain?", "limitations": "1. The novelty of model architecture is limited. There is no well-designed module for scATAC-seq data or tabular data, except for the position embeddings and mamba adoption.\n2. The training loss is not really suitable for the scATAC-seq data since the contextual signals of other regions are not significant with the dropout events.\n3. The downstream tasks are not comprehensive. The authors should conduct more research on perturbation prediction and temporal scATAC-seq prediction.\n4. More ablation studies should be conducted.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: $\\bullet$ This is a pioneer foundation model explicitly for scATAC-seq. $\\bullet$ In order to address the high dimension issue of scATAC, the authors propose a novel architectural design combining Mamba and windowed attention. $\\bullet$ It demonstrates strong performance across multiple tasks (representation, annotation, prediction).\n\nWeakness: \n$\\bullet$ The novelty of the model architecture is limited. The difference between Chromefound and other single-cell foundation models is that it adopts Mamba to process long sequential data. $\\bullet$ The training loss is not really suitable for the scATAC-seq data since the contextual signals of other regions are not significant with the dropout events. scATAC data is more sparse than scRNA data, however, the foundation model on scRNA does not really achieve robust performance when comparing to single linear regression. $\\bullet$ Unlike scRNA data which the gene set is already defined, scATAC fragment sets are not always the same among different datasets by different depths. The authors do not claim how they unify the formats of different datasets to employ Chromfound. $\\bullet$ As far as I know, there is another scATAC foundation model Epifoundation(https://www.biorxiv.org/content/10.1101/2025.02.05.636688v1) before Chromefound. $\\bullet$  Missing reference like Epifoundation, ATAC-Diff (https://arxiv.org/abs/2408.14801), and GET (https://www.nature.com/articles/s41586-024-08391-z). $\\bullet$ In the ablation study, the performance of Chromfound is close with much less training set. I am wondering if less OCR would decrease the performance.", "was_revised": true, "final_justification": "The authors have addressed all my concerns.", "rebuttal": "We sincerely thank the reviewer for their constructive feedbacks and recognizing our work’s novelty and robust performance across downstream tasks. We would like to clarify and elaborate on the following points of concern:\n\n## [W1,L1] Model Novelty\nAs our submission is under the _Machine Learning for Sciences_ area, our goal is to tailor the architecture to scATAC-seq characteristics for biologically meaningful representation and practical applications. We clarify the design of ChromFound’s each component as follows:\n- Unified OCR tokenization (Sec 3.1) encodes chromosomal identity, genomic coordinates, and continuous-valued accessibility, providing a generalizable architecture for large-scale pretraining of heterogeneous scATAC-seq datasets.\n- WPSA (Sec 3.2.1) captures local _cis_-regulatory dependencies within ±200 kb of transcriptional start sites, aligning with biological enhancer-promoter interactions.\n- Mamba (Sec 3.2.2) complements WPSA by modeling long-range interactions across hundreds of thousands of OCRs per cell, enabling ChromFound to learn both local and global chromatin accessibility patterns.\n- Pretrain objective (Sec 3.3) reconstructs masked both zero and non-zero OCRs using MSE loss, which ensures robustness to the high sparsity of scATAC-seq data.\n\n## [Q1] Shallow Model Baseline Comparison\n\nWe thank the reviewer for the helpful suggestion of including shallow baselines for both cell type annotation and cross-omics prediction.\n\n1. Cell Type Annotation\n    \n    We implement two logistic regression models (sklearn.linear_model.LogisticRegression) using full or HVG-filtered OCRs (5,000 peaks), trained with default hyperparameters. These baselines are evaluated across the same datasets in Fig. 3 and compared against ChromFound. Results are reported in the format of Accuracy/F1 score as below:\n\n|Tissue/train/test|Linear|Linear(HVG5k)|ChromFound|\n|-|-|-|-|\n|PBMC/EPF_hydrop_1/VIB_10xv1_1|0.8132/0.6741|0.4527/0.3189|0.8863/0.8135|\n|PBMC/EPF_hydrop_3/VIB_10xv1_2|0.8227/0.6709|0.4791/0.3355|0.9003/0.8134|\n|Bone/Batch27/Batch26|0.7198/0.6241|0.3884/0.3150|0.8230/0.5477|\n|Bone/Batch43/Batch44|0.7854/0.7873|0.4618/0.4414|0.8368/0.8335|\n|Cortex/Batch2/Batch1|0.8367/0.6031|0.4880/0.2403|0.9366/0.7715|\n|Cortex/Batch3/Batch2|0.8455/0.6598|0.4295/0.2804|0.9188/0.8063|\n|Retina/D19D003/D018_13|0.9217/0.8781|0.7737/0.5235|0.9792/0.9780|\n|Retina/D021_13/D19D003|0.9278/0.8862|0.8542/0.7473|0.9786/0.9762|\n\n2. Cross-Omics Prediction\n\n    For cross-omics prediction, we implement a shallow two-layer MLP that maps OCR inputs to genome-wide gene expression profiles. Given the complexity of predicting over 30,000 gene targets, we used a hidden layer size of 4096, ReLU activation, learning rate of 1e-4, and 50 training epochs. Results are reported in the format of PCC/CCC as below:\n\n|Model|Cortex_Zhu45K|Bone_To326K|BMMC_multiome_2021|BMMC_atac2gex_2022|Cell_lines_Zhu11K|\n|-|-|-|-|-|-|\n|MLP|0.1022/0.0721|0.0946/0.0709|0.1009/0.0627|0.1023/0.0558|0.2333/0.1729|\n|ChromFound|0.8316/0.8064|0.8304/0.7472|0.4249/0.4032|0.9293/0.8818|0.9449/0.9071|\n\nKey Conclusions:\n1. ChromFound consistently outperforms logistic regression, a strong baseline for annotation.\n2. Full OCR input significantly outperforms HVG-filtered input, reinforcing ChromFound's necessity of modeling genome-wide regulatory elements.\n3. The MLP baseline performs poorly on cross-omics prediction, revealing the limitations of simple linear or shallow models in capturing the complex _cis_-regulatory logic underlying chromatin-to-expression mapping. ChromFound is specifically designed to model this complexity through deep representation learning and large-scale self-supervised pretraining.\n\n## [W2,L2] Training Loss Suitability\n\nWe appreciate the reviewer’s concern regarding sparsity in scATAC-seq data. \nActually, we have mentioned that ChromFound selectively reconstructs both non-zero and zero OCRs (Sec. 3.3). The masking strategy ensures the training procedure is not affected by the dropout events.\nFollowed by xTrimoGene, we use the MSE loss for reconstructing the masking peaks, which is suitable for continuous-valued accessibility data.\nFig. 2 further demonstrates that ChromFound maintains stable performance under severe downsampling, showcasing its robustness to sparsity.\n\n## [W3,Q2] OCR Format Unification Across Datasets\n\nThank you for the question. ChromFound adopts a coordinate-based tokenization strategy (Sec. 3.1), where each OCR is represented by its chromosome ID, start/end positions on GRCh38, and continuous accessibility. This enables ChromFound to unify peaks across datasets without requiring pre-aligned peak sets. Notably, the cross-tissue OCR overlap is quite low, highlighting the necessity of position-aware encoding for robust generalization.\n\n## [W4,W5] References to Related Works\n\nWe sincerely thank the reviewer for pointing out these three important related works: Epifoundation, ATAC-Diff, and GET.\n- Epifoundation is quite impressive work, building a foundation model by leveraging paired scATAC-seq and scRNA-seq data to learn peak-gene alignments. However, its reliance on paired multi-omics data limits scalability as the paired data remains limited. \nIn contrast, ChromFound adopts a self-supervised approach that does not require paired modalities, enabling pretraining on over 2.6 million cells from diverse tissues and disease conditions. Furthermore, ChromFound extends beyond Epifoundation’s scope by supporting biological applications such as enhancer-gene link inference and perturbation response prediction.\n- ATAC-Diff is designed for identifying differential chromatin accessibility across conditions instead of supporting zero-shot transfer learning.\n- GET, as we have already acknowledged in Section 2, Line 83 of the manuscript, is a transcriptomic foundation model using pseudo-bulk pairs of chromatin accessibility and sequence data, differing in resolution and modality from ChromFound.\n\nIn response to your professional suggestion, we will add citations to Epifoundation and ATAC-Diff, and update Section 2 to include a more thorough comparison. In addition, we clarify the positioning of ChromFound as the first genome-wide, self-supervised foundation model for scATAC-seq with zero-shot generalization across tissues.\n\n## [Q3] Performance Variation Across Tissues\n\nThank you for the insightful question. We have thoroughly evaluated ChromFound across various tissue types on all downstream tasks. While minor variations in performance exist due to inherent differences in tissue complexity and data sparsity, ChromFound consistently achieves strong results across all tasks and tissue types. If any specific aspect of the results remains unclear, please do not hesitate to let us know. We would be happy to provide further clarification.\n\n## [Q4] Comparison Between WPSA and Linear Attention\n\nThank you for the question. ChromFound’s WPSA is implemented using FlashAttention, significantly improving efficiency compared to vanilla self-attention. To further compare with linear attention, we implement Performer and Linformer under the same conditions, using identical hidden dimensions, batch size of 4, and FP16 mixed-precision training.\nThe following results are obtained on the cell clustering task using the PBMC VIB_10xv1_1 dataset. As shown in the table below, WPSA outperforms both alternatives in terms of model performance and computational efficiency.\n\n|Model|Parameters|FLOPS| Inference speed(s/batch) |ARI(Trained on 0.02 million cells)|\n|-|-|-|-|-|\n|Ours|450305|7.88E+11|3.46|0.6142|\n|Performer|1838337|6.01E+12|3.89|0.5978|\n|Linformer|920833|1.62E+12|3.74|0.5751|\n\n## [Q5] Statistical Significance and Random Seed Control\n\nWe thank the reviewer for this professional and important question. \n\nFor cell clustering (Table 1) and low-count denoising (Figure 2), we perform each evaluation 20 times with different random seeds and report the mean and standard deviation to reflect performance stability.\nIn the perturbation response prediction task (Figure 4), we report p-values for the PCC metric to establish the statistical significance of the predictions.\nFor other tasks, we ensure that all benchmark methods are implemented and evaluated under the same random seed settings for fair comparison.\n\nTogether, these strategies ensure that all reported performance gains are both robust and statistically meaningful. We hope that all these clarifications have addressed reviewer's questions.\n\n## [W6,L4] Ablation Study of less OCR\n\nWe thank the reviewer for raising this important point. We try to understand the comment on \"less OCR\" in two possible ways, and we have conducted targeted ablation experiments to address both:\n1. Reduce OCR input per cell: In Section 4.4 (Question 3), we reduce the maximum number of OCRs per cell to one-half and one-quarter of the full set by highly variable peaks filtering. We observe a clear performance drop in both clustering and annotation tasks, demonstrating that genome-wide OCR input is critical for capturing rich regulatory information and ensuring robust generalization.\n2. Reduce pretraining data size: In Section 4.4 (Question 2), we also perform controlled experiments by reducing the pretraining corpus to one-tenth and one-hundredth of the original size. We observe a progressive decline in downstream performance, confirming the benefit of scaling up training data size.\n\n## [L3] More Downstream Tasks\n\nWe sincerely thank the reviewer for this suggestion. Actually, we have performed perturbation response prediction and presented the results in Figure 4. As the reviewer mentioned, temporal scATAC-seq prediction is quite challenging, and we are currently working on extending ChromFound to support this task.\n\n## Conclusion\n\nWe sincerely thank the reviewer for your valuable time and constructive feedback. We hope our responses have addressed your concerns. If there are any further questions, we would be happy to provide further clarification."}]

Author rebuttal:

Meta-review: The submission describes a foundation model for scATAC-seq data which can be pretrained on data from diverse cell types and tissues and used in a zero-shot or fine-tuned fashion on downstream tasks. Reviewers had high appraisal for the novelty and significance for this application domain. Questions about the technical details of the experiments led to a lively discussion, in which authors provided further experimental comparisons. 3/4 reviewers indicated positive appraisal of these new results presented in the rebuttal and the resolution of most concerns.
Decision: Accept (poster)