A versatile informative diffusion model for single-cell ATAC-seq data generation and analysis
Authors: Lei huang
Lei Xiong
Na Sun
Zunpeng Liu
Ka-Chun Wong
Manolis Kellis
Venue: NeurIPS 2024 poster (2024)
Keywords: scATAC-seq
diffusion model
generative model
Primary area: machine_learning_for_healthcare

Abstract: The rapid advancement of single-cell ATAC sequencing (scATAC-seq) technologies holds great promise for investigating the heterogeneity of epigenetic landscapes at the cellular level. The amplification process in scATAC-seq experiments often introduces noise due to dropout events, which results in extreme sparsity that hinders accurate analysis. Consequently, there is a significant demand for the generation of high-quality scATAC-seq data in silico. Furthermore, current methodologies are typically task-specific, lacking a versatile framework capable of handling multiple tasks within a single model. In this work, we propose ATAC-Diff, a versatile framework, which is based on a diffusion model conditioned on the latent auxiliary variables to adapt for various tasks. ATAC-Diff is the first diffusion model for the scATAC-seq data generation and analysis, composed of auxiliary modules encoding the latent high-level variables to enable the model to learn the semantic information to sample high-quality data. Gaussian Mixture Model (GMM) as the latent prior and auxiliary decoder, the yield variables reserve the refined genomic information beneficial for downstream analyses. Another innovation is the incorporation of mutual information between observed and hidden variables as a regularization term to prevent the model from decoupling from latent variables. Through extensive experiments, we demonstrate that ATAC-Diff achieves high performance in both generation and analysis tasks, outperforming state-of-the-art models.

A versatile informative diffusion model for single-cell ATAC-seq data generation and analysis
Lei Huang
Lei Xiong Massachusetts Institute of Technology
City University of Hong Kong Massachusetts Institute of Technology
Na Sun Massachusetts Institute of Technology
Zunpeng Liu Massachusetts Institute of Technology
Ka-Chun Wong
Manolis Kellis Massachusetts Institute of Technology
City University of Hong Kong
Abstract
The rapid advancement of single-cell ATAC sequencing (scATAC-seq) technologies holds great promise for investigating the heterogeneity of epigenetic landscapes the cellular level. The amplification process in ScATAC-seq experiments often in- troduces noise due dropout events, which results in extreme sparsity that hinders accurate analysis. Consequently, there is a significant demand for the generation high-quality ScATAC-seq data in silico. Furthermore, current methodologies are typically task-specific, lacking versatile framework capable of handling multiple tasks within single model. In this work, propose ATAC-Diff, versatile frame- work, which is based on a latent diffusion model conditioned on the latent auxiliary variables adapt for various tasks. ATAC-Diff the first diffusion model for the scATAC-seq data generation and analysis, composed of auxiliary modules encoding the latent high-level variables to enable the model learn the semantic information to sample high-quality data. Gaussian Mixture Model (GMM) as the latent prior and auxiliary decoder, the yield variables reserve the refined genomic information beneficial for downstream analyses. Another innovation is the incorporation of mutual information between observed and hidden variables as regularization term to prevent the model from decoupling from latent variables. Through extensive experiments, we demonstrate that ATAC-Diff achieves high performance in both generation and analysis tasks, outperforming state-of-the- art models.
Introduction
Assay for Transposase Accessible Chromatin with sequencing (ATAA--se)) data has shed light genome research to dissect gene regulatory landscapes and cellular heterogeneity. Particularly, single- cell ATAC-seq (scATAC-seq) can be harnessed probe the chromatin accessibility profile at single- cell level to reserve the diversity of cell types in heterogeneous tissue, which can reveal important genomic elements for transcription factor binding and regulating downstream gene expression, particularly at distal non-coding regulatory regions[[] However, the analysis of such data poses challenges due to the high levels of noise, sparsity, and data scale encountered. Furthermore, the complexity of biological systems, with their intricate intercellular communications and numerous
"Equal contribution.
t "ooresponniigg Author: Manolis Kellis.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

molecules involved, presents another challenge in analyzing them, especially when limited by small sample sizes, which can hinder reproducibility in biomedical research(3).
Recent advances in machine learning models especially generative models have driven rapid de- velopment in analyzing single-cell sequencing data to understand fundamental mechanisms of biology. These models achieved state-of-the-ar (SOTA) performances across a range of single-cell tasks 5 6 7 8). including clustering analysis, batch correction, data denoising and imputation, and in silico data generation[3]. However, most methods focused on single-cell RNA-seq data and there little work focusing on scATAC-seq data which is more sparse and high dimensional. Furthermore, these models treat each task with different frameworks, ignoring the potential correlations across these tasks.
To tackle these challenges, it's essential to create generative models that can not only produce high- quality ScATAC-seq data but also maintain the integrity of biological representations for subsequent analysis. Recently, diffusion models (9. 10 11 have emerged as a promising tool, demonstrating impressive results on realistic imaging generation [[2 audio synthesis [15]16], and molecular ligand design [17 18). Nevertheless, the latent variables of these diffusion models lack semantic meaning and are not interpretable since they aim learn the diffused noise, which is not suitable for ScATCC-se data representation learning to uncover the underlying biological patterns (i.e., cellular heterogeneity within a tissue or organism). Since single-cell sequencing data is typically represented as bag- of of-dodswordd, words, where there is no upper limit to the value of peak calling per cell and the order of peaks does not affect the cell, the diffusion model, which is designed for continuous data, is less suited for processing discrete data. To address this, we can employ latent diffusion model [3] that utilizes pretrained autoencoder to transform ScATAC-seq samples into a continuous latent space. This transformation facilitates the effective training of the diffusion model. Additionally, it provides access to an efficient and compact latent space where abstracted away, allowing the model to focus more on the important and variable genomic fragments.
and imperceptible details are
In this study, we propose ATAC-Diff, versatile informative latent diffusion model, to analyze and generate ScATAC-seq data by introducing semantically meaningful latent space within unified framework. Inspired by diffusion autoencoder [9]] and Info encoder equipped with Gaussian Mixture Model (GMM) as the prior of the latent space to learn the cell representation. The low dimensional cell embeddings could enhance the diffusion model to capture intrinsic high-level factors of variation present in the heterogeneous ScATC-seq data. By optimizing the mutual information between the cell embeddings and real data points, ATAC-Diff could avoid ignoring the latent variables the conditional information when utilizing ELBO as the objectivity [11] To the best of our knowledge, we are the first to utilize the diffusion model for ScATAC- data generation and analysis. We conduct comprehensive experiments to demonstrate that ATAC-diff achieves SOTA or comparable performances across a range of tasks, including realistic and conditional generation, denoising, imputation, and subgroup clustering, compared to the existing models specifically designed for these tasks.
we introduce an auxiliary
We summarize the contributions of this work as follows:
We propose versatile framework ATAC-Diff which can generate in silico ScATAC-seq data with conditional information and reverse the genomic latent representation to reveal the underlying principles.
We are the first study to harness diffusion based models for ScATAC-seq data generation and analysis.
We equip the diffusion backbone module with an auxiliary encoder which utilizes GMM as the latent feature extractor and mutual information to regulate the variational objectivities.
We provide evidence that ATAC-Diff exhibits the capability to yield high-quality scATAC- seq data and cell latent embeddings, surpassing or achieving performance levels comparable to SOTA models.
2 Background
The diffusion model is formulated as two Markov chains: diffusion process and reverse process. Given a variance preserve schedule, the diffusion process transforms the real data Xp to a latent variable distribution p(Xor)) and finally the data is diffused into predefined Gaussian noise TT

---

through the time step setting t = 1...T. The transform distribution is formulated as fixed Markov chain which gradually adds Gaussian noise to the data with user-defined variance schedule:
q (x+ xt-1) =N - 1,,,,111,,
where is the state of x at the time step / which is obtained by adding Gaussian noise to the former state X,-1 and B, controls the degree of the noise. The distribution delineates a Gaussian distribution centered on the incrementally corrupted data state Ir. We could obtain closed form from the input data Xg to Xy in a tractable way. The posterior distribution could be factorized as:
q (XL:T) (x) = II
x
X x
Let d; = -II 3., could sample 22 any arbitrary time step t in closed form by utilizing the reparameterization (rick[9]:
q (x, xo) = V (x:: Voixo (1 (1- -
If the time step is sufficiently large, the final distribution will approach that of a standard Gaussian distribution. The reverse process aims reconstruct the original data Xg from the diffused data xy, sampled from Gaussian distribution (O,, I), which is accomplished through the diffusion process. The reverse process can also factorized as Markov chain:
T
pe (Xo:T-1 xT) = II p (x xi) (-1
(4)
We aim to investigate the iterative reverse process p(x:-1 (x). However, Estimating the distribu- tion p (x,_1
Xx) hard estimate unless the gap between and is infinitesimally small (T = x 10]. Therefore, a learned Gaussian transitions pe (x,-1 x,) is devised to approximate
p [x:11 x2) at every time step:
po (xt-1 x1) = N (x1-1: Mo (x..)) oi
(5)
Following previous work (9)). pbo (Xe, t) can be modeled as:
1
Be vI-
ac En (xt: 1).
Bg (xx, -I-B.
(6)
where Eg is a neural network w.r.t trainable parameters 0. Having formulated the reverse process, we could maximize the likelihood of the training data as our object. Since directly calculating the likelihood is intractable, we adopt the evidence lower bound (ELBO) [9 to optimize.
E log pe (x) - LKL (q (XT xo) [pp (XT))
DKL (q (xi-1 X0) [ppo (x1-1 x:))
log Po (Xol x1) = LD.
Cr
3 Method
In this section, we formally present our proposed model ATAC-Diff for ScATAC-seq data analysis and generation. As illustrated in Figure I ATAC-Diff is based on the conditional latent diffusion model equipped with the auxiliary module which provides the additional latent variables. Specifically, we use fragment counts to represent the ScATAC-seq data. Then we compress the ScATAC-seq data (fragment counts) into lower dimensional latent space. Here, we adopt the autoencoder (AE) framework, where the encoder EFçAF encodes raw data Traw into latent domain and the decoder DECAE learns to decode Ti back to raw data Trow This framework can be trained by minimizing the reconstruction objective. The auxiliary encoder z Enc,(xo) learns map
EnC (Jraw)
3

---

an input ATAC-seq data to semantically meaningful representation 2. By incorporating latent variable of this nature, the diffusion model can prioritize high-level semantic information during the generation process, thereby producing data of superior quality. Additionally, this approach enables the utilization of meaningful representations at the hidden layer for downstream task analysis. Our work is inspired by the recent success of Diffusion Autoencoders [1],, but learning latent variables for the sparse ScATAC-seq data is however challenging. We address this by introducing GMM to extract the distinct features from the latent space and utilize mutual information to maximize the shared information between latent variables and real ScATAC-seq data. We elaborate on the auxiliary module in Section 3.1 and the conditional diffusion model in Section 3.2. Finally, we briefly describe the training and sampling scheme in section 3.3.
Mutual information
Latent generator (VAE)
Maximization I(X ox
Sample z-
X
GMM
Auxiliary Encoder
Auxiliary Decoder
Clustering
Cellsx Fragment Counts
Po (x,,-1xx;,2,,1)
Conditional LDM ((x,Ixx-11Xo)
X
X
Figure I: Overview of ATAC-Diff framework. ATAC-Diff is a versatile informative diffusion model of ScATAC-seq data analysis and generation. It leverages the auxiliary model yield low-dimensional meaningful semantic variables as the conditional information to help generate high-quality data while the reserved potential biological information could be applied for downstream tasks like cell type identification.
3.1
Informative auxiliary module
3.1.1
Semantic encoder
The goal of the informative auxiliary module is two-fold. The first to summarize an input ScATAC- seq data into semantic representation Ency(X,) which contains fine-grained information to assist the diffusion decoder pe (x;-1 1 z) denoise and predict the output ScATAC-sc data. The second one that the meaningful representation yielded by the auxiliary module could be utilised for downstream analysis such as cell relationships visualization and cell heterogeneity identification.
The latent variables 2, unlike the latent variables in the diffusion process, are flexible and can represent a ciw-dimensional vector latent factors variation which could be either continuous or discrete. In order infer 2, design the approximate posterior qo xn) which could be any architecture for the encoder. In our experiments, the encoder shares the same architecture in the diffusion backbone the transformer encoder, which is formulated
LayerNorm(xo + FFLA
(8)
where FFL the feed-forward layer, MHA is the multi-head self-attention layer.
3.1.2 Semantic prior
To learn the latent distribution of the input data and enable the unconditional sampling of in the diffusion model, we model the semantic prior p(z) over data features. Usually, the prior over the latent variables is commonly an isotropic Gaussian 22). However, utilizing Gaussian distribution as

---

the prior probability distribution p(2) restricts the latent representation to a single mode while the raw features the scATAC-seq data are multi-modal with different cell types [23]. Thus, we apply GMM as the posterior to learn multi-modal latent representations [8] [24]. We predefined categorical e E [T,...K} where is a predefined number of components in the mixture, e-g., the number of cell types. The joint probability p(xo, c) could be modeled as:
Ps (Xo: 2, e) - -plppppp(xp/ppp/ooooo (xpxx))))))(lcc)p())
where p(=/e) is mixture of Gaussian distribution parametrized by series of μr and de. Then define each factorized probability as:
p(c) Cat(()))) = (Nablc [cclue:oo21
(10)
We employ Kullback-Leibel (KL) divergence to calculate the regularization term to enforce the latent variable to the GMM manifold: RaMM D (2, xo)llp(z.c)
3.1.3 Consistent latent variables with mutual information maximization as regularizer
Diffusion models can considered as a special realisation of hicrarchical VAE[25] which the encoder contains no Iearnable parameters. The iterative decoder model complex decoding distribution (o(xo|X1:T>> 2) but suffers from the ignoring of low-dimensional latent variables (20) since high- dimensional Xy does not depend this issue by maximizing the mutual information (MI) between 2 and Xg, assuming meaningful semantic variables reserve high MI with the real data point. We define MI of xo and as:
leading degrading unconditional generation. We tackle
lo(2 Xo)
=) H()) (=|xx)) = (x..)) Doog
(11)
(z)
where (2)) and (2)x))) are the marginal information entropy and conditional information entropy. and (=) is the parameterized posterior of mixture of Gaussian distribution. Maximization of the mutual information enables model to generate X which can infer z, thus avoiding the ignorance
issue.
Finally, we employ an auxiliary decoder reconstruct the latent variables recover the real data point. The auxiliary decoder could help the auxiliary module to consistently generate semantic variables z by maximizing the likelihood of p(xolz). which can be considered as complementary information to mutual information / [xx;; 2). Furthermore, this could prevent the latent variable from generating into the pure mixture of Gaussians, which turn interferes with the generation of We define the reconstruction loss as:
C. E (e,,ci))) [logp(xo
(0xgox//) [z)]
(12)
3.2 Conditional diffusion decoder
Having formulated the latent auxiliary module, we conditioned the diffusion models on the latent variables and other conditional information y such as cell types, tissue, and other omics data (e.g. scRNA-seq data). Then the learning objectivity is converted to pe (x:-1 X,, 2, y). Hence, Eq. and Eq. 5 becomes the conditional format:
T II p (x-1 2.,
Po (Xo:T-1 - XT)
(13)
(x,-1 xx) = N' (x Ho (x,, t, v) of?l
(14)
Since the diffusion model predefined the diffusion process by the user-defined variance schedules, we could encode Zn to the stochastic x, by running the deterministic process in Eq. 3
X Vā v(1 -āā)c
(15)
where is sampled from N(O. 7). Similar to the auxiliary module, we also utilize transformer to model Hg (x,, y). For the conditional information y embedding, we also utilize MI.ss to convert the raw representation of the features to the final conditional set: Φ(y) (y)). where i 1,..L.
5

---

Then we utilize the cross attssttntnnioon mechanism to enable the prediction of Xo to be conditioned on the specific attribute information y and 2.
Attention (Q,K, softmax QKT
V,
vd
(16)
O,K,V Wq(Femb), Wk(o(u: 2)), W (v(o(v,
where Wo. WK. Wv are parameterized linear layers. For the categorical condition information such as cell type, we train a classifier on the latent embeddings 2, and then select the corresponding the latent embeddings for certain cell type.
3.3 Training process
Having formulated the informative auxiliary and the conditional diffusion decoder, the training of the reverse process is performed by optimizing the evidence lower bound (ELBO) negative log- likelihood (conditional form of Eq. 7 with the auxiliary module regulation. We train ATAC-Diff by the following form:
CATAC-Diff CD + L. + ax(ox =) XRGMM
T
= () (xo x1, w)1 - Σ
Exo.x (DRL (a (xt-1 Xx, xo) Iipo (xt-1 Xx, 2, v))]
t=2
+
+ (A - 1) (DKL(g ( c))
+ +aDrr(uo())llb)
(17)
The full derivation can be found in the Appendix. The third term and fourth term can be considered the ELBO objectivity of VAE model, which are feasible calculate. However, the last term
intractable to calculate since we cannot evaluate (z(2). Following (20). we could sample 2 ~
(=(xx))) which X ((x)) and then optimize it by likelihood free optimization techniques (26).
Empirically, the ELBO of diffusion models could be simplified as:
L simple = Exx.xx []]x+ - (x [)]
Ex [||x xolva + v1 o ox,,, ,,
In fact, we can directly model pa by utilizing x4 instead of co since Eq be rewritten as:
14g (xe, () Vas(( a1-1)
Vār-
x (xt, 1, 2,
(18)
Cle
In practice, we find that predicting Xo will enable the model converge faster. We speculate that the reason for this phenomenon that the ScATAC-seq data is highly sparse with noise, predicting e from x, which is close to the white noise is very hard, leading to decrease in sampling quality. For the reconstruction loss of the AE, we adopt 1.2 norm loss.
3.4 Sampling process
ATAC-Diff is different from vanilla Diffusion models which conditioned on the latent variables. In order to sample from ATAC-Diff model, we need to design a sampling strategy to sample 2 from the latent distribution. Since the prior distribution of latent variables is the mixture of Gaussians, it is hard sample for unconditional generation. Therefore, model to sample 2. In this study, we employ vanilla VAE to sample the latent variables. For data denoising and imputation, we just use the same data as the inputs for both diffusion model and the by
could train any arbitrary generative
auxiliary module. In general, we could calculate the mean of the reverse Gaussian transitions Eq. To sample the seATAC-seq data, we first sample the chaotic state Xy from N(00 /) or use the desired noised data as inputs. is sampled by the generative model or has the same value of ær. The next less chaotic state X generated by Eq. 5 The final state Xn is iteratively sampled x,-1 for T' times.

---

Experiments
In this section, we evaluate our proposed model ATAC-Diff across a range of experiments on three benchmark datasets, utilizing metrics which span generation quality, denoising and imputation effects, and latent space representation analysis. The extensive experimental results suggest that ATAC-Diff achieves higher or competitive performances compared with SOTA models which are designed for individual tasks.
4.1 Datasets
We adopt three datasets to benchmark our model and baseline models: Forebrain 277, Hematopoiesis [28). and PBMCC1 Forebrain dataset is derived from P56 mouse forebrain. Hematopoiesis dataset includes 2,000 cells during hematopoietic differentiation through FACS. PBMC1OK dataset comprises peripheral blood mononuclear cells isolated from a healthy donor, with granulocytes selectively removed through cell sorting, resulting cell population of approximately 10,000 cells for detailed analysis.
4.2 Metrics
For the latent representation analysis, we extract the latent representation of cells. We evaluate the clustering results of latent representations by Normalised Mutual Information (NMI), Adjusted Rand Index (ARI). Homogeneity score (Homo) and Average Silhouette Width(ASW). To evaluate the similarities between the generated and real cells, we adopt Spearman Correlation Coefficient (SCC) and Pearson Correlation Coefficient (PCC) for performance comparison.
4.3 Latent representation analysis
4.3.1 Experiment setting
We explore the quality of the latent representations by clustering examine if it could be used to identify cell types. For performance comparison, we apply Highly Variable Peak (HVP) method from SCANPY [29]. cisTopic SCALE(8), and Peak to obtain the corresponding dimensionality reduction features. Specifically, we leverage K-Means to obtain the clusters. We evaluate the clustering performance based on NMI, ARI, Homo, and ASW.
Table Clustering performance of ATAC-Diff and baseline methods on 3 ScATAC-seg datasets.
Datasets
Forebrain
Hematopoiesis
PRMCI0K
Mctrids/Metres
NMI
ARI
Homo
ASW
NMI
ARI
Homo
ASW
NMI
ARI
Homo
ASW
HVP
0.247
0.093
0.178
-0.563
0.083
0.023
0.065
-0.324
0.522
0.400
0.454
-0.672
PCA
0.359
0.182
0.329
0.081
0.041
0.023
0.034
0.171
0.626
0.419
0.679
0.393
cisTopic
0.665
0.540
0.650
0.467
0.639
0.457
0.660
0.272
0.736
0.499
0.502
0.467
SCALE
0.718
0.657
0.722
0.515
0.608
0.404
0.631
0.371
0.675
0.451
0.739
0.434
PeakV1
0.384
ATAC-DA Dif
0.740
0.674
0.673
0.533
0.647
0.492
0.666
0.423
0.733
0.506
0.800
0.386
4.3.2
Experimental results
The clustering results ATAC-DiA -iff and the baseline models on benchmark datasets are presented in Table 8 The first, second and third highest values colored by red, green and purple. The higher the value if the metrics, the better the clustering performance performance. Among the 3 datasets based the four metries, ATAC-Diff yields best scores and comparable scores compared with the baseline models. For instance, ATAC-Diff outperforms all the baseline models across all the metrics on Hematopoiesis dataset, defeats SOTA models by least 1. 1.25% in NMI, 7.66% in ARI, 0.91 % in Homo, and 10.16% in ASW. Furthermore, we observe that ATAC-Diff and SCALE achieve similar performance Forebrain dataset but ATAC-Diff surpasses it and achieves competitive performances on PBMC10K dataset.
DDwnloadedd PBMC10k dataset from https://support. 10xgenomicca cww.cooo/ciaoo ss////nglelelee cell mooll-mull multiome- atac- Beeee-ee//eee tex/datasets// wttpssss....0../... 0.0/pbmc_ granulocyte_sorted. generated from healthy donor after removing granulocytes through cell sorting
7

---

HVP
PCA
dis diToiiic
SCALE
peakvi
ATAC-DOff
Forebrain
Hematopolese
Figure 2: UMAP visualization of the highly variable peak values and extracted features from PCA, cisTopic. SCALE, VI and ATAC-Diff Forebrain and Hematopiesis datasets. The visualization of PBMC10K is included in the Appendix.
To determine whether the methods effectively separate the cell types in the latent space, we employ UMAP to visualize the extracted features (Figure 2). Our analysis reveals that ATAC-Diff exhibits superior performance in distinguishing the cell types, whereas HVP fails to differentiate between the cell types. Furthermore, some of the other methods show overlapping results for certain cell types. In addition, our findings demonstrate that ATAC-Diff has the ability to unveil the distance between distinct cell subpopulations and their developmental trajectories. For instance, observed that the three excitatory neuron cell types (EC1, EC2, EC3) from the Forebrain dataset exhibit close proximity to each other in the latent space of ATAC-Diff. Conversely, the distances between different cell types are significantly larger, indicating distinct cellular identities. In the Hematopoiesis dataset, we observed that the Multipotent Progenitor Cells (MPPs), which are downstream progenitor cells derived from Hematopoietic Stem Cells (HSCs), exhibit close proximity to each other in the latent space of ATAC-Diff. This finding aligns with their known differentiation relationship, further supporting the accuracy and biological relevance of our analysis.
4.4
Generation quality measurement
4.4.1
Experimental settings
In this section, we have devised two strategies to evaluate the generation capabilities of ATAC-Diff. The first strategy involves unconditional generation, where we assess whether the ScATAC-seg data generated by ATAC-Diff exhibits realistic characteristics. If our model excels capturing the data distribution, it can be leveraged to create simulated data for augmentation purposes rather than sequencing additional cells, thus conserving time and resources. Furthermore, visualizing the synthetic data generated can enhance researchers' comprehension of the distribution patterns and fundamental structure of scATAC-seq data. In this case, we anticipate that ATAC-Diff will be able to generate cells that possess the specific subpopulation features associated with the provided conditional information. Since we are the first to generate ScATAC-seq data from scratch, there are no baseline models for comparison. Therefore, we have made modifications to SCALE and PeakVI, which are VAE- WAE-based models, to enable unconditional generation by utilizing random noise as inputs. For conditional generation, have replaced the VAE module of SCALE with a conditional VAE moduld"] For the unconditional generation, we average all single cells as the ground truth. For the conditional generation, average all single cells of the same biological cell type the ground truth. To address the requirement of ATAC-Diff for latent variables as auxiliary information, we employ vanilla VAE sample the latent variables. We evaluate the generation performance based on the SCC and PCC between the mean values of generated samples and the two ground truth data.
'Unfortunately. due to the high integration of PeakVI II into the sevi-tools package, we were unable to make modifications to it.

---

4.4.2 Experimental results
We report the results of unconditional generation in T We observe that ATAC-Diff outperforms other methods in generating ScATAC-seq data from scratch, yielding more realistic results. Specifi- cally, ATAC-Diff achieves the highest SCC of 0.892 and PCC of 0.969 on Forebrain dataset, SCC of 0.822 and PCC of 0.973 on Hematopoiesis dataset, and PCC of 0.983 on PBMCIOK dataset. Table 6 also displays the mean correlation values of each cell type in the context of conditional generation. We observe that ATAC-Diff outperforms conditional SCALE among all the metrics. In addition, the performance of ATAC-Diff without conditional information degraded, indicating that ATAC-Diff can fuse the conditional cell type information to generate high-quality ScATAC-seq data of specific cell types. Overall, we prove that ATAC-Diff is capable of generating realistic ScATAC-seq data.
Table 2: Unconditional and conditional generation performance of ATAC-Diff and other baseline methods on 3 ScATCC-ee datasets.
Datasets
Forebrain
Hematoppiesis
PBMC10K
Methods/Metrics
SCC
PCC
SCC
PCC
SCC
PCC
Unconditional Generation
SCALE
0.548
0.728
0.799
0.719
0.892
0.451
PeakVI
0.330
0.366
0.759
0.762
0.824
0.860
ATAC-Diff
0.925
0.992
0.927
0.976
0.964
0.997
Conditional Generation
SCALE
0.576
0.746
0.708
0.816
0.838
0.819
ATAC-Diff w.o con
0.418
0.728
0.496
0.840
0.828
0.915
ATAC-Diff
0.688
0.770
0.850
0.923
0.846
0.922
4.5 Data denoising and imputation
4.5.1
Experimental settings
The ScATAC-seq data always contains both noised and a large number of missing values due to dropout events [30]. We design two approaches to test the ability of the method for denoising and recovering missing values (imputation) to address the issues under real scenarios. For ScATAC- seq data denoising, analyzing real datasets poses challenge due to the lack of ground truth data. Following previous work[8), we mitigate this challenge to average all single cells within the same cell type. resulting in a meta-cell that serves For SCATAC-seq data imputation, previous works chose to create a corrupted matrix by randomly dropping out some non-zero entries. In our case, we follow the setting used in DCA 31 simulate dropout events by masking 10% of the non-zero counts and setting them to zero. The probability of masking particular non-zero count follows an exponential distribution which peaks with lower expression levels have a higher likelihood of dropout compared to genes with higher expression levels 32). For comparison, choose SCALE and peakVI as baseline models and SCC and PCC as metrics.
good approximation of those individual cells.
4.5.2 Experimental Results
The results of scATAC-seq data denoising and imputation of ATAC-Diff and baseline models are presented in Table where the best result is highlighted in bold. For the data denoising evaluation, ATAC-Diff achieves the highest correlation of the denoised single cells with the corresponding meta-cells of each cell type PBMCIOK dataset and competitive results the other two datasets. Although the other generative models Peak and SCALE perform well on some of the datasets, their performances are not robust. One potential reason is that the auxiliary latent variables enable ATAC-Diff to capture the high-level semantics information. Similarly, ATAC-Diff performs stable on ScATAC-seg data imputation, where the generated data is closely related to the meta-cells.
5 Conclusion
In this work, we propose versatile framework ATAC-Diff which is based on diffusion model conditioned on the latent variables for ScATAC-seq data analysis and generation. ATAC-Diff could utilize the proposed auxiliary module to yield the latent variables which contain high-level meaningful

---

Table 3: Denoising and imputation performance of ATAC-Diff and other baseline methods on scATAC-seq datasets.
Datasets
Forebrain SCC
Hematopoiesis
PBMC10K
Metrods/Metries
PCC
SCC
PCC
SCC
PCC
Denoising
SCALE
0.777
0.870
0.676
0.726
0.858
0.945
VeakVT
0.710
0.760
0.874
0.879
0.860
0.948
ATAC-Diff
0.718
0.873
0.840
0.875
0.863
0.950
Imputation
SCALE
0.760
0.860
0.888
0.947
0.858
0.924
PeakVI
0.708
0.755
0.870
0.870
0.916
0.947
ATAC-Diff
0.716
0.861
0.892
0.909
0.860
0.949
information: it could be incorporated assist the data sampling during diffusion. Additionally, these variables could be examined for downstream analyses such as cell type annotation. Based on the formulation of the mutual information, we prevent the model from ignoring the low-dimensional latent variables. Furthermore, we employ Gaussian Mixture Model (GMM) as the latent prior and auxiliary decoder enhance the recovery of real data, allowing the latent variables to learn the genomic semantics.
We conducted extensive experiments evaluate the performance of ATAC-Diff on three datasets and compared it with SOTA models, indicating that the ATAC-Diff model can generate high-quality data in silico ScATAC-seq data while effectively disentangling the cell embeddings. An intriguing potential direction for ATAC-Diff involves exploring various conditional generation scenarios for genomic discovery, such identifying the connections between scRNA-seq data and perturbation prediction. We anticipate that ATAC-Diff can contribute to the advancement of genomic analysis in the near future personalized medicine the single-cell level.
6
Acknowledgment
The work was supported in part by NIH grants RO1 AG081017-01, RO1 AG067151, and HGO1 2009- 01. The work was partially supported by the grant from the Research Grants Council of the Hong Kong Special Administrative Region [CityU 11203723). The work described in this paper was partially supported by the grants from City University of Hong Kong (2021SIRG036, CityU 9667265, CityU 11 11002221) and Innovation and Technology Commission (TTB/FBL/90337/2/)))
References
[1] Jason D Buenrostro, Beijing Wu, Ulrike M Litzenburger, Dave Ruff, Michael L Gonzales, Michael P Snyder, Howard Y Chang, and William J Greenleaf. Single-cell chromatin accessi- bility reveals principles of regulatory variation. Nature, 523(7561):486 4906 2015.
[2] Sandy Klemm, Zohar Shipony, and William J Greenleaf. Chromatin accessibility and the regulatory epigenome. Nature Reviews Genetics, 20(4):207-25) 2019.
[3] Mohamed Marouf, Pierre Machart, Vikas Bansal, Christoph Kilian, Daniel S Magruder, Chris- tian F Krebs, and Stefan Bonn. Realistic in silico generation and augmentation of single-cell ma-seq data using generative adversarial networks. Nature communications, 11(1):166, 2020.
[4] Romain Lopez, Jeffrey Regier, Michael B Cole, Michael Jordan, and Nir Yosef. Deep generative modeling for single-cell transcriptomics. Nature methods, 10533) 2018.
[5] Tal Ashuach, Daniel A Reidenbach, Adam Gayoso, and Nir Yosef. Peakvi: deep generative model for single-cell chromatin accessibility analysis. Cell reports methods, 2(3), 2022.
[6] Han Yuan and David R Kelley, scbasset: bequence-based modeling of single-cell atac-seq using convolutional neural networks. Nature Methods, 19(9): 1088-1998 2022.
[7] Carmen Bravo González-Blas Liesbeth Minnoye, Dafni Papasokrati, Sara Aibar, Gert Hulsel- mans, Valerie Christiaens, Kristofer Davie, Jasper Wouters, and Stein Aerts. cistopie: cis- regulatory topic modeling single-cell atac-seg data. Nature methods, 16(5):397-400, 2019.
10

---

[8] Lei Xiong, Kui Xu, Kang Tian, Yanqiu Shao, Lei Tang, Ge Gao, Michael Zhang, Tao Jiang, and Qiangfeng Cliff Zhang. Scale method for single-cell atac-seq analysis via latent feature extraction. Nature communications, 10(1):4576, 2019.
[9]
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:68440-885,, 2020.
[10] Jascha
Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper-
vised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2265- PMLR, 2015.
[1I] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
[12) Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
[13] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[14) William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195-4205, 2023.
[15] Zhifeng Kong, Wei Ping. Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprini arXiv:2009.0977 2020.
[16] Rongjie Huang, Jiawei Huang. Dongchao Yang. Yi Ren, Luping Liu, Mingze Li, Zhenhui Ye, Jinglin Liu, Xiang Yin, and Zhou Zhao. Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models. arXiv preprini urXiv:2301. 12661, 2023.
[17] Lei Huang, Hengtong Zhang, Tingyang Xu, and Ka-Chun Wong. Mdm: Molecular diffusson
model for 3d molecule generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 5105-5112. 2023.
[18) Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International conference on machine learning, pages 8867-8887. PMLR, 2022.
[19] Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, and Supasom Suwajanakom. Diffusion autoencoders: Toward a meaningful and decodable representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10619-10629, 2022.
[20] Shengjia Zhao, Jiaming Song, and Stefano Ermon. Infovae: Information maximizing variational autoencoders. arXiv preprint arXiv:1706.0226.1 2017.
[21] Durk P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Improved variational inference with inverse autoregressive flow. Advances in neural information processing systems, 29, 2016.
[22] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint 1312621 14, 2013.
[23] Christopher Heje Grønbech, Maximillian Fornitz Vording, Pascal N Timshel, Casper Kaae Sønderby, Tune H Pers. and Ole Winther. sevae: variational auto-encoders for single-cell gene expression data. Bioinformatics, 36(16):4415-447)) 4422, 2020.
[24] Nat Dilokthanakul, Pedro AM Mediano, Marta Gamelo, Matthew CH Lee, Hugh Salimbeni. Kai Arulkumaran, and Murray Shanahan. Deep unsupervised clustering with gaussian mixture variational autoencoders. arXiv preprint arXiv: /. 1006448, 2016.
[25] Arash Vahdat and Jan Kautz. Nvae: A deep hierarchical variational autoencoder. Advances in neural information processing systems, 33:1 136677671997,, 2020.
[26] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial
networks. In International conference on machine learning, pages 214-223. PMLR, 2017.
[27] Sebastian Preissl, Rongxin Fang, Hui Huang, Yuan Zhao, Ramya Raviram, David Gorkin, Yanxiao Zhang, Brandon C Sos, Veena Afzal, Diane E Dickel, et al. Single-nucleus analysis of accessible chromatin in developing mouse forebrain reveals cell-type-specifi transcriptional regulation. Nature neuroscience, 21(3):432-433 2018

---

[28] Jason D Buenrostro, M Ryan Corces, Caleb A Lareau, Beijing Wu, Alicia N Schep, Martin Aryee, Ravindra Majeti, Howard Y Chang, and William Greenleaf. Integrated single-cell analysis maps the continuous regulatory landscape of human hematopoietic differentiation. Cell, 173(6):1535-1548, 2018.
[29] F Alexander Wolf, Philipp Angerer, and Fabian J Theis. SCANPY: large-scale single-cell gene expression data analysis. Genome Biol., 19(1):15, February 2018.
[30] David Van Dijk, Roshan Sharma, Juozas Nainys, Kristina Yim, Pooja Kathail, Ambrose Carr, Cassandra Burdziak, Kevin R Moon, Christine L Chaffer, Diwakar Pattabiraman, et al. Recovering gene interactions from single-cell data using data diffusion. Cell, 174(3)716-722, 2018.
[31] Gökcen Eraslan, Lukas M Simon, Maria Mircea, Nikola S Mueller, and Fabian J Theis. Single- cell rna-seq denoising using a deep count autoencoder. Nature communications, 10(1):390, 2019.
[32] Luke Zappia, Belinda Phipson, and Alicia Oshlack. Splatter: simulation of single-cell ma sequencing data. Genome biology, 18(1):174, 2017.
[33] Uya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 17X1v:1711.0500,,220 2017.
12

---

A Proof of the diffusion model
We provide proofs for the derivation of several properties in ATAC-Diff.
A.1 Marginal distribution of the diffusion process
In the diffusion process, we have the marginal distribution of the data at any arbitrary time step in closed form:y (x, xo) -V (xei vā,Xo, (1 (+) I).
Proof:
recall the posterior y (xt xx-1) Eq. 3 we can obtain x; using the reparameterization
trick. A property of the Gaussian distribution is that if we add off and (0 oīI), the new
distribution is (0, (oî + I
X Voixi-1 V1 0444-1
V0101-1X1-21 + a1(1 - at 04-1) v1 0
Vaj01-1X1-2 0,00-161-2
(19)
= Vāxxo + VI OGE,
where 1 3r, and é are sampled from independent standard Gaussian distributions. Then we could derive Eq.
A.2 The derivation of parameterized mean
learned Gaussian transitions po (x+-1 xx) is devised to approximate the g (x:-1 xx) of every
time step: pe (xt-1 xx) N (xr-1i Meg (xe, o phg is parameterized as follows:
Me (xe,
1 Var
32
60 (xx.1))
Proof: the distribution y (x:-1 xx) can expanded by Bayes' rule:
(xt-1 xx)
q -x Xt: X0.)
x1-1, q (x xo)
q (xt
q (x: xo)
9 (X
q (xi
x(-1)
q (Xt Xo)
(x:
(x:-1 1x0)
(xt varxo))
exp
1 1-
1
/2
2/ar-1
xe-1 + C (xx,x0) )) X5
exp
+
x2
Xi
3r
1
az-1
Br
āc-1
(1
14-1)
Vot--1Bt
exp(
2 xx?
x +
X (x))x--1)
ai
(20)
where C' (xx, Xp) is constant. We find that (x,-1 xx) is also Gaussian distribution. We
assume that:
q (x(-1 1
X1, Xn) = N (x (x),
(21)
where Br = =1/
(ae (0: --
1-ā .36
T-o;
and i., (Xe, xo)
Vay X, + vān-1 xo)
T-a
(m)
+1
--a
yaill-ār-1)
I-a, X, +
vār-
I-a;
Xo-
13

---

Then we could parameterize M4 (x,, yar(1-au-) 18 From Eq.
I-ar Xs yāu- Xa. which is presentned in Eq.
19 we have X, - vā,X0 + vI dge. We take this into ja:
Vot (1 -
V 1
(x, v1 -
X,
āt
as
vā,
1
3r Et
X
a
V
Ot
Hg is designed to model it. Therefore, He has the same formulation as jt but parameterizes e: 1 Vau [x VI-a; co (x).t)
Hg (x),t)
A.3 The derivation of loss function
hard to directly calculate the conditional log-likelihood of the data. Instead, we can derive its ELBO objective for optimizing. For simplicity, the in GMM is omitted
E. [log po (xo) (yy( log Byy(s) f po (xo=r(z) dxaer
= Ryy( log / po (XOrT, 2) dx
Ex)()) log (f q (xter xo) qo (2) (x1rr xx) go(22220 (o( dxi:dde) (22)
po (Xn:T, 2)
Pe (x
= -Eg(x,) log Exgxerr Boo() a (X1: (x))g
po (XI,T, 2) E (X log E Bys(elzo) (x xn) (2)
14

---

Then we further derive the conditional ELBO objective:
po (Xo:T; 2)
E Bxxx (o (2/xo)
log q (XL:T Xo) 96 (2(xx))
Po (xT) po (=) IT po (x,-1 X1, 2)
B
log
(z(x))))) II q (xx,
x(-1)
log po (XT) (=)=)
T
Po (xt-1 1
Xt: 2)
Fg
log
q (x+
x(-1)
Po (xT) (z)
T
Pu (Xt-1
Xx: x))
po (xo x1,=))
log
log
+ log
g0l2/ (2))
q (x+
X(-1)
ạ (x11 xn)
t=
log Pe (xT) (2)
Pg (x,-1
x),
(xx-1 xx0)
loog"" Po (xo
x1, 2)
Bg
log
(6(22 22|x0))
9 (xt-1
Xx, xo)
9 (x: xx0)
q (x1
x()
t=2
po (xT) po (=())
po (xt-1
X1, 2)
log ? (xt-1 xx)
=,,
+
log
(2(2(x))
q (x1-1
X1. Xo)
q (xt xo)
po (xo
x)
log
q (x1
Xo)
log Po (XT) (2)
T
Po (x:-1
x1,
q ((x) Xo)
Po (Xo
x1, 2)
En
log
+ log
+ llog
(=
(xt-1
Xr. xo)
(xr xn)
q (x1
(x0)
t=2
Po (xT (2)
T
po (x/-1
Xe, 2) Xe. x0)
po(2)
Bg
log
log
+ log pa (xo x), + log
q (XT
(x0)
(xt-1
(2 xo)
Eg
DEL (a (XT Xo) llpo (xx(z))
br
T
(6(2)
DEL (q (xt-1 Xx, Xo) |e (xc-1 Xx, ()) + log pa (xo x1, + log
To (=|xx)
t=
Lo
(23)
A.4 Proof of the ATAC-Diff Objectivity
For simplicity, we utilize the term log Pa(#)
(=(x) and the regular term of GMM and mutual
information to obtain the final form of the optimizing objectivity in Eq. (We also omit c in GMM for simplicity).
pz(z) + all (x032) + XRGMM
Eg log
(2 (x
log g (=(2|x0) (xo) + a log (x(2)
(=(z)
+ AAlog (x(2
Eg
(0(2(xx)
p(2)
log go(2|x0) [x)) A- (2)
E,g
p(z)a-1
(24)
log ( a (z)
E,g
p(=))-0-1p(())
log go (0) 1
p(2))-a-1 + log (2)
E,p
p(2)a
= (A - a 1) 1) + aD (DKr(go())lppa
15

---

Finally, we could derive the objectivity as:
CATAC-Diff
E 20) (Xo X1, 2,
T
Ex [KL (g (xx-1 xx) lipo (xe-1 xt, 2, v))
(25)
t=2
+ Ey (z.c) (z.c/xo) (1gpP(
+ (A - a - (())) (Del (gp(2, xo)llp(2,c)
+ +aDkr(a0()]llpe
B Experiments details
B.1 Dataset and implementation
We summarize the statistic information of all processed datasets in Table
Dataset
#cells
#peaks
#cell types
Reference
Forebrain
2088
11285
8
Preissl et al.,2018
Hematopoiesi
2034
103151
10
Buenrostro et al., 2018
PBMC10k
9631
107194
19
10xgenomics, 2020
Table 4: The statistic summary of datasets
For all three datastes, ATAC-Diff is trained by Adam [33 optimizer for 10K iterations with a batch
size of 256 and learning rate of 0.0001 one NVIDIA V100 GPU card. We set 0.1 and À as 1.1001.
B.2 Ablation study
In this study, we have removed the GMM and MI modules to conduct the ablation study for each task.
Table 5: Ablastion study for clustering.
Datasets
Forebrain
Hematopoiesis
PBMCIOK
Methods/Metries
NMI
ARI
NMI
Homo
ASW
NMI
ARI
Homo
ASW
ATAC w.o GMM
0.558
0.488
0.297
0.222
0.586
0.288
0.655
0.137
ATAC w. MI
0.596
0.451
0.602
0.120
0.490
0.299
0.510
0.077
0.590
0.231
0.657
0,032
Table 6: Ablation study for unconditional and conditional generation.
Datasets
Forebruin
Hematopoiesis
PBMC10K
Methods/Metriee
SCC
PCC
SCC
PCC
SCC
PCC
Unconditional Generation
ATAC w.o GMM
0.919
0.991
0.886
0.949
0.693
0.710
ATAC w.o MI
0.916
0.991
0.904
0.962
0.704
0.729
Conditional Generation
ATAC w.o GMM
0.678
0.768
0.845
0.910
0.823
0.911
ATAC w. MI 0.681
0.769
0.848
0.913
0.831
0.920
B.3 Euclidean distances different cell types
We present similarity matrix of the latent embedding based on Euclidean distances. Specifically, we average the latent embeddings within each cell type population and calculate the Fuclidean distances across cell types.
16

---

Table 7: Ablation study for denoising and imputation.
Datasets
Forebrain
Hematopoiesis
PBMCIUK
Methods/Metrics
SCC
PCC
SCC
PCC
SCC
PCC
Denoising
ATAC w.o GMM
0.701
0,867
0,823
0.851
0,843
0.932
ATAC woo MI 0.706
0.868
0.831
0.856
0.851
0.940
Imputation
ATAC w.o GMM
0.710
0.841
0.887
0.898
0.833
0.935
ATAC W. MI 0.712
0.845
0.887
0.901
0.831
0.931
Our results show that ATAC-Diff can effectively capture the relationships between distinct cell subpopulations and their developmental trajectories. For example, in the Forebrain dataset, the three excitatory neuron subtypes (EC1, EC2, EC3), which are biologically similar, cluster closely in ATAC-Diff's latent space based the correlation of their embeddings. In contrast, more biologically distant cell types display greater separation. Similarly, the Hematopoiesis dataset, Multipotent Progenitor Cells (MPPs) and Hematopoietic Stem Cells (HSCs) exhibit proximity, reflecting their known differentiation path and reinforcing the biological relevance of our findings.
Table 8: The Euclidean distances across different cell types.
Comparison
AC
EXI
EX2
EX3
INI
IN2
MG
OC
AC
0.8307
0.7431
0.7594
0.8508
0.7237
0.8550
0.8520
EX1
0.8307
0.4773
0.4667
0.6548
0.6219
0.7900
0,7514
EX2
EX3
0.7594
0.4667
0.1794
0.6153
0.5545
0.7607
0.7135
0.6548
0,5901
0.6353
0.6426
0.847]
0.8235
IN2
0.7237
0.6219
0.5183
0.5545
0.6426
0,7510
0,7188
MG
0.8550
0.7900
0.7592
0.7607
0.8471
0.7510
0.8530
OC
0.8520
0.7514
0.7171
0.7135
0.8135
0.7188
0.8530
C Additional figures
C.1 The UMAP visualization of the methods on PBMC10K
We have visualized the extracted features by different methods on PBMC1Ok dataset through UMAP, which is shown in Figure
PBMC10K
HVP
PCA
cisTopic
SCALE
peakyi
ATAC- DIff
?
Figure 3: UMAP visualization of the highly variable peak values and extracted features from PCA, cisTopic, SCALE, VeakVI, and ATAC-Diff on PBMC10k dataset.
17

---

NeurIPS Paper Checklist
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well vell-specification asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
18

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [No]
Justification:
This paper does not include theotrtical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
19

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: We will release the full source code soon. The link is https//giihub.com/L.ane Huang/ATAC-Diff.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
CodeSuomi ssinnoolicsy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
20

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httppp:///orrppp..pppppttttttttt
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
21

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
This paper poses such risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
22

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [No]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [No]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "XdZqN5FxSv", "reviewer": "Reviewer_feok", "summary": "Generating simulated scATAC-seq data is important for developing new methods and gaining a deeper understanding of the data. However, the simulation is challenging due to dropout and high noise in the data. Authors proposed a diffusion + VAE type of method to solve the problem. The general idea is first to use VAE to project original scATAC data to a lower embedding space. Then impose a diffusion process in the embedding space. The lower embedding space is a GMM rather than the classical isotropic Normal, which is the novelty part of the method. This configuration makes biological sense as cells can be grouped as different cell types. Due to the introduction of GMM distribution in the hidden GMM space, there exists complications in generalising the diffusion loss function. The authors have shown nice and solid derivations in the appendix. The method is then applied to three datasets on three different tasks and achieved comparable  performance as SOTA methods. It seems that authors have provided a convincing solution to the research question. While this paper is clearly written and the general idea is relatively easy to follow, it will be nice if the author could help to answer the following questions.\n1. In Eq. 11, what is the parametric form of q_\\phi(z|x_0)\n2. In Eq. 14, what is the actual meaning of conditional information y, could you list the parameters\n3. Could you provide a concrete network architecture of your network in a supplementary figure, i.e. including the tensors and their dimensions?\n4. In sec 4.4.1, what is the dropout rate distribution in your simulated data, are they similar to the real data?\nDepending on the answers, I may change my ratings in the future.", "questions": "Generating simulated scATAC-seq data is important for developing new methods and gaining a deeper understanding of the data. However, the simulation is challenging due to dropout and high noise in the data. Authors proposed a diffusion + VAE type of method to solve the problem. The general idea is first to use VAE to project original scATAC data to a lower embedding space. Then impose a diffusion process in the embedding space. The lower embedding space is a GMM rather than the classical isotropic Normal, which is the novelty part of the method. This configuration makes biological sense as cells can be grouped as different cell types. Due to the introduction of GMM distribution in the hidden GMM space, there exists complications in generalising the diffusion loss function. The authors have shown nice and solid derivations in the appendix. The method is then applied to three datasets on three different tasks and achieved comparable  performance as SOTA methods. It seems that authors have provided a convincing solution to the research question. While this paper is clearly written and the general idea is relatively easy to follow, it will be nice if the author could help to answer the following questions.\n1. In Eq. 11, what is the parametric form of q_\\phi(z|x_0)\n2. In Eq. 14, what is the actual meaning of conditional information y, could you list the parameters\n3. Could you provide a concrete network architecture of your network in a supplementary figure, i.e. including the tensors and their dimensions?\n4. In sec 4.4.1, what is the dropout rate distribution in your simulated data, are they similar to the real data?\nDepending on the answers, I may change my ratings in the future.", "limitations": "Generating simulated scATAC-seq data is important for developing new methods and gaining a deeper understanding of the data. However, the simulation is challenging due to dropout and high noise in the data. Authors proposed a diffusion + VAE type of method to solve the problem. The general idea is first to use VAE to project original scATAC data to a lower embedding space. Then impose a diffusion process in the embedding space. The lower embedding space is a GMM rather than the classical isotropic Normal, which is the novelty part of the method. This configuration makes biological sense as cells can be grouped as different cell types. Due to the introduction of GMM distribution in the hidden GMM space, there exists complications in generalising the diffusion loss function. The authors have shown nice and solid derivations in the appendix. The method is then applied to three datasets on three different tasks and achieved comparable  performance as SOTA methods. It seems that authors have provided a convincing solution to the research question. While this paper is clearly written and the general idea is relatively easy to follow, it will be nice if the author could help to answer the following questions.\n1. In Eq. 11, what is the parametric form of q_\\phi(z|x_0)\n2. In Eq. 14, what is the actual meaning of conditional information y, could you list the parameters\n3. Could you provide a concrete network architecture of your network in a supplementary figure, i.e. including the tensors and their dimensions?\n4. In sec 4.4.1, what is the dropout rate distribution in your simulated data, are they similar to the real data?\nDepending on the answers, I may change my ratings in the future.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Generating simulated scATAC-seq data is important for developing new methods and gaining a deeper understanding of the data. However, the simulation is challenging due to dropout and high noise in the data. Authors proposed a diffusion + VAE type of method to solve the problem. The general idea is first to use VAE to project original scATAC data to a lower embedding space. Then impose a diffusion process in the embedding space. The lower embedding space is a GMM rather than the classical isotropic Normal, which is the novelty part of the method. This configuration makes biological sense as cells can be grouped as different cell types. Due to the introduction of GMM distribution in the hidden GMM space, there exists complications in generalising the diffusion loss function. The authors have shown nice and solid derivations in the appendix. The method is then applied to three datasets on three different tasks and achieved comparable  performance as SOTA methods. It seems that authors have provided a convincing solution to the research question. While this paper is clearly written and the general idea is relatively easy to follow, it will be nice if the author could help to answer the following questions.\n1. In Eq. 11, what is the parametric form of q_\\phi(z|x_0)\n2. In Eq. 14, what is the actual meaning of conditional information y, could you list the parameters\n3. Could you provide a concrete network architecture of your network in a supplementary figure, i.e. including the tensors and their dimensions?\n4. In sec 4.4.1, what is the dropout rate distribution in your simulated data, are they similar to the real data?\nDepending on the answers, I may change my ratings in the future.", "weaknesses": "Generating simulated scATAC-seq data is important for developing new methods and gaining a deeper understanding of the data. However, the simulation is challenging due to dropout and high noise in the data. Authors proposed a diffusion + VAE type of method to solve the problem. The general idea is first to use VAE to project original scATAC data to a lower embedding space. Then impose a diffusion process in the embedding space. The lower embedding space is a GMM rather than the classical isotropic Normal, which is the novelty part of the method. This configuration makes biological sense as cells can be grouped as different cell types. Due to the introduction of GMM distribution in the hidden GMM space, there exists complications in generalising the diffusion loss function. The authors have shown nice and solid derivations in the appendix. The method is then applied to three datasets on three different tasks and achieved comparable  performance as SOTA methods. It seems that authors have provided a convincing solution to the research question. While this paper is clearly written and the general idea is relatively easy to follow, it will be nice if the author could help to answer the following questions.\n1. In Eq. 11, what is the parametric form of q_\\phi(z|x_0)\n2. In Eq. 14, what is the actual meaning of conditional information y, could you list the parameters\n3. Could you provide a concrete network architecture of your network in a supplementary figure, i.e. including the tensors and their dimensions?\n4. In sec 4.4.1, what is the dropout rate distribution in your simulated data, are they similar to the real data?\nDepending on the answers, I may change my ratings in the future.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your encouraging comments and the constructive advices on improving the manuscript. Your comments clearly helped a lot to improve the study. We have summarized the suggested comments (Questions) and made point-by-point responses and revisions as follows.\n\n1. Thank you for your valuable feedback. $q_\\phi(z|x_0)$ represents the amortized inference distribution, serving as an approximate variational posterior within the generative model. To achieve this parameterization, we employ an auxiliary encoder which is similar to the encoder of VAE model.\n2. Thank you for your insightful comments. In our study, $y$ represents the conditional information, encompassing factors like cell type, tissue, and other omics data (e.g., scRNA-seq data). We have chosen to utilize cell type as the guiding conditional information for the evaluation of our model. To address your feedback, we have revised the description of $y as follows:\n\n\"We conditioned the diffusion models on the latent 165 variables z and other conditional information y such as cell types, tissue, and other omics data (e.g. scRNA-seq data).\"\n\n3. Thanks for your valuable comments. We have incorporated a figure illustrating the dimensions of the tensors within the network. We will put this figure in the global response.\n4. We sincerely appreciate your valuable insights. In our methodology, we utilize an exponential distribution where peaks exhibiting lower expression levels are more prone to dropout events compared to those with higher expression levels. This differential dropout mechanism is employed to simulate realistic dropout events in our data. We hypothesize that peaks with higher expression levels are more likely to go undetected during the sequencing process. Consequently, we ensure that a minimum of 80% of values in the simulated dataset are zero, reflecting the sparsity inherent in single-cell sequencing data. This intentional introduction of sparse noise effectively mirrors the prevalent characteristics found in actual single-cell sequencing datasets, which are often characterized by a significant number of 'missing' or 'zero' values.\"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qzwkULLA7H", "reviewer": "Reviewer_kixm", "summary": "This submission introduces ATAC-Diff, a conditional latent diffusion model for scATAC-seq data. ATAC-Diff incorporates a few components to a basic diffusion model, including a Gaussian Mixture Model as a semantic prior over the latent variables and mutual information as a regularizer. ATAC-Diff is benchmarked on several tasks: latent representation clustering, (contitional) data generation, and denoising/imputation. ATAC-Diff is shown to do better than PeakVI and SCALE (among other methods for clustering).\n\n\n[UPDATE 8/9]: Adjusting score from Reject to Accept due to clarifications.", "questions": "Questions are integrated within Weaknesses (above).", "limitations": "This study seems interesting and their method ATAC-Diff may be an advance, but it is difficult to assess given the poor presentation. There are several weaknesses such as questionable ground truth within the benchmarks. Moreover, the task itself seems to be dated, focusing on cells x binary peaks. Moreover, the components of the proposed approach are not tested in any ablation study, bringing their purpose questionable. This approach seems to be a new method, but one of many in the growing universe of scATAC-seq data and it's not clear what impact this work will have due to these limitations.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- It seems to be the first application of a diffusion model to atac-seq data represented as cellsxpeaks\n- ATAC-Diff is benchmaked against other generative models based on VAEs, including SCALE and PeakVI\n- Performance of ATAC-Diff seems to be better than previous methods on 3 tasks (i.e., cell type clustering, data generation, and denoising). \n- ATAC-Diff can generate unconditionally and conditionally with assistance from a VAE", "weaknesses": "- presentation of ATAC-seq data is not clear. The cell x peak representation is a processed version that doesn't necessarily encapsulate the full ATAC-seq data. Clarity that they are not generating ATAC-seq data but rather processed ATAC-seq data in the form of cell x binary peaks is necessary.\n- Example: other approaches to analyze ATAC-seq data incorporate sequences, eg. chromBPnet and AI-TAC.\n- Weak evaluations. The evaluations of cell clustering are not clear. It is not clear how ground truth was determined, given it is likely given by another computational method. The metrics are not clear what they are and their strengths and limitations. The umap visualization is not reliable. The authors state that they observe EC1, EC2, EC3 from the Forebrain dataset are close proximity in latent space. But latent space can be highly warped, rendering any Euclidean distances in umap space not meaningful. \n- Generation quality task is confusing. \n\t- When is unconditional generation needed? Why is this a meaningful task?\n\t- Not clear how the metrics are calculated. What is the ground truth here?\n\t- Why are generated cells averaged? Shouldn't generation be assessed at the single-cell level? \n\t- The ATAC-Diff w.o con seems quite high. This affirms my concern that the evaluations might be weak. \n- The proposed components GMM and MI regularizer are not evaluated via an ablation study. Are they even needed?\n- This study treats ATAC-seq data as a cell x (binary) peaks representation, instead of fragment counts, which improves scATAC-seq analysis (see Martens et al, Nature Methods, 2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your detailed and insightful comments on our manuscript. Your comments clearly helped a lot to improve this manuscript! \n\n1. We clarify that in our approach, we utilize fragment counts rather than binary peaks to represent the scATAC-seq data. To alleviate any confusion, we have updated the descriptions related to the data for better clarity.\n\n\"Specifically, we use fragment counts to represent the scATAC-seq data.\n\n\"We compress the scATAC-seq data (fragment counts) into a lower-dimensional latent space.\"\n\n2. Our model is constructed solely based on the scATAC-seq data, without incorporating DNA sequence information. Given that our model operates as a conditional generative model, we intend to integrate DNA sequence information as a guiding conditional factor in our future work. In this version, we did not use DNA sequence since some datasets do not include the genome information.\n\n3. Ground truth cell annotations utilized in our study are extracted from the previous publication [1,2,3]. These annotations were extensively characterized through marker peaks corresponding to marker genes, supported by their distinct biological functions enriched with cell-type-specific peaks. We affirm that these cell annotations are not only reliable but also hold significant biological relevance.\n\nThe metrics like NMI, ARI, and ASW, which were employed in our evaluation, are commonly utilized to assess the conservation of biological variance in latent features for single-cell dataset benchmarks. This practice is well-documented in [https://www.nature.com/articles/s41592-021-01336-8] and several other prominent single-cell methodologies [4,5,6].\n\nWe acknowledge the concern that UMAP visualization can sometimes warp the space, particularly in regions with large distant gaps. Nevertheless, UMAP generally preserves distance relationships within closely-knit local subgroups that share similarities. For instance, subpopulations like EC1, EC2, and EC3, which belong to excitatory cells and are in close proximity, exhibit this preservation, aligning well with their biological characteristics.\n\nMoreover, we have computed the average latent embeddings within each cell type population and calculated the distances across different cell types. We have listed some distances due to the space limitation: EX1-EX2: 0.477, AC-EX1: 0.831. We have added other values in the Appendix.\n\n4.1 We employ unconditional generation to assess the model's ability to capture the entirety of the data distribution. Effective modeling of the data distribution enables us to generate synthetic data for data augmentation, potentially obviating the need for sequencing additional cells and thereby saving valuable time and resources. \n\nTo further address your concerns, we have refined Section 4.4.1 to provide a more detailed explanation of the unconditional task\n\n4.2 For the cluster task, we utilize the NMI, ARI, Homo, and ASW to evaluate the performance. The calculation of metrics are formulated as follows.\n\n$NMI(A,B)=\\frac{I(A,B)}{\\sqrt{H(A)H(B)}}$\n\n$ARI=\\frac{RI-\\text{Expected}_RI}{\\max{(RI)}-\\text{Expected}_RI}$\n\n$Homo = 1 - \\frac{H(C, Y)}{H(Y)}$\n\n$ASW = \\frac{1}{N} \\sum_{i=1}^{N} \\frac{b_i - a_i}{\\max(a_i, b_i)}$\n\nWe adopt the average of all scATAC-seq data as the ground truth to calculate the SCC and PCC for the unconditional generation task. For the conditional generation, we average all single cells of the same biological cell type as the ground truth. \nFor the denoising task, it is the same with conditional generation. For the imputation task, we calculate the SCC and PCC of the masked value and the imputed value.\n\n4.3 We cannot calculate the correlation of the generated data and ground truth at the single cell level as the data is not one to one generated. The scATAC-seq technology suffers from many sources of technical noise, leading to dropout events, which means even the ground truth scATAC-seq data is also not complete. \n\n4.4 The SCC and PCC of the conditional generation are lower than the unconditional generation since we average the SCC and PCC of different cell types for conditional generation while we calculate the SCC and PCC of the whole data for conditional generation. There are some cell types which are hard to learn due to limited data. Hence,  the PCC and SCC of such cell types are quite low, leading to the lower averaged SCC and PCC.\n\n4.5 We have conducted the ablation study.\n\n| Clustering | NMI   | ARI   | Homo  | ASW   |NMI   | ARI   | Homo  | ASW   |NMI   | ARI   | Homo  | ASW   |\n|-----------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|\n| ATAC w.o GMM       | 0.558 | 0.438 | 0.556 | 0.202 | 0.489 | 0.297 | 0.505 | 0.222| 0.586 | 0.288 | 0.655 | 0.137 |\n| ATAC w.o MI       | 0.596 | 0.451 | 0.602 | 0.120 | 0.490 | 0.299 | 0.510 | 0.077 | 0.590 | 0.231 | 0.657 | 0.032 | \n\n| Unconditional  | SCC  | PCC   | SCC  | PCC  | SCC   | PCC  | \n|-----------------|-------|-------|-------|-------|-------|-------|\n| ATAC w.o GMM       | 0.919 | 0.991 | 0.886 | 0.949 | 0.693 | 0.710 |\n| ATAC w.o MI       | 0.916 | 0.991 | 0.904 | 0.962 | 0.704 | 0.729 |\n\n| Conditional  | SCC  | PCC   | SCC  | PCC  | SCC   | PCC  | \n|-----------------|-------|-------|-------|-------|-------|-------|\n| ATAC w.o GMM       | 0.678 | 0.768 | 0.845 | 0.910 | 0.823 | 0.911 |\n| ATAC w.o MI       | 0.681 | 0.769 | 0.848 | 0.913 | 0.831 | 0.920 |\n\n| Denoising | SCC  | PCC   | SCC  | PCC  | SCC   | PCC  | \n|-----------------|-------|-------|-------|-------|-------|-------|\n| ATAC w.o GMM       | 0.701 | 0.867 | 0.823 | 0.851 | 0.843 | 0.932 |\n| ATAC w.o MI       | 0.706 | 0.868 | 0.831 | 0.856 | 0.851 | 0.940 |\n\n| Imputing | SCC  | PCC   | SCC  | PCC  | SCC   | PCC  | \n|-----------------|-------|-------|-------|-------|-------|-------|\n| ATAC w.o GMM       | 0.710 | 0.841 | 0.887 | 0.898 | 0.833 | 0.935 |\n| ATAC w.o MI       | 0.712 | 0.845 | 0.887 | 0.901 | 0.831 | 0.931 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BBoKuONQsP", "reviewer": "Reviewer_BJyq", "summary": "The paper introduces a new diffusion model ATAC-diff for scATAC-seq data generation and analysis. ATAC-diff using a latent diffusion model which is conditioned on latent auxiliary models to encode latent variables, and integrate GMM as the latent prior to capture genetic information. This paper introduces a mutual information regularization to maintain the connection between observed and latent variables. Extensive experiments show that ATAC-diff outperforms state-of-the-art models in both data generation and analysis tasks.", "questions": "1. Explain the discrepancy on baseline performance and cross validation with previous publications. \n2. Include more benchmark dataset for better cross validation of the model performance, such as Buenrostro2018.\n3. Add top baselines, such as scBasset into comparison.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "This paper introduces a uniform model that is applicable for multiple tasks in scATAC-seq analysis. The method section is clear and easy to follow, providing a clear explanation on auxiliary module, integration of GMM and mutual information regularization. \nThe paper provides an in-depth theoretical analysis, helps the audience to better understand the method. \nThe experiment covers a wide range of scATAC-seq analysis, including clustering, generation, denoising and imputation. The results are comprehensive and shows advantage of the new model.", "weaknesses": "Multiple baseline performance on the benchmark dataset does not match what was reported in previous publications. For example, [1] shows much higher PCA performance on PBMC10k dataset. This discrepancy makes me concern about the reliability of the experiment results. \nIf would be great to include a few benchmark dataset to help cross validate the model performance, for example Buenrostro2018. \nThere are top baselines that are not included in the experiment, please consider add the comparison on the latest baseline as well, such as scBasset. \n[1] Yuan H, Kelley D R. scBasset: sequence-based modeling of single-cell ATAC-seq using convolutional neural networks[J]. Nature Methods, 2022, 19(9): 1088-1096.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your detailed and insightful comments on our manuscript. Your comments clearly helped a lot to improve this manuscript! We have summarized your comments and made point-by-point responses and revisions to address your concerns.\n\n\n1. Thank you for your valuable comment. Our model outperforms the baseline models on both the analysis and generation task. The baseline models are only designed for the scATAC-seq data analysis by leveraging the approximate distribution like ZINB as the posterior. In contrast, our model incorporate both the latent distribution construction and data likelihood learning. These two strategies  can coordinate with each other, leading to the improved performance on both sides. The low dimensional cell embeddings could enhance the diffusion model to capture intrinsic high-level factors of variation present in the heterogeneous scATAC-seq data. Furthermore,  the mutual information between the cell embeddings and real data points enable ATAC-Diff to avoid ignoring the latent variables as the conditional information when utilizing ELBO as the objectivity compared to the baseline models which are based VAE framework (SCALE, PeakVI).\n\n2. Thank you for your considerable comment. Actually, the Hematopoiesis dataset is the Buenrostro2018 dataset. We utilize Hematopoiesis to indicate the biological process insetad of Buenrostro2018 dataset. You could check the reference [1]. Jason D Buenrostro, M Ryan Corces, Caleb A Lareau, Beijing Wu, Alicia N Schep, Martin J Aryee, Ravindra Majeti, Howard Y Chang, and William J Greenleaf. Integrated single-cell analysis maps the continuous regulatory landscape of human hematopoietic differentiation. Cell, 173(6):1535–1548, 2018.\n\n3. Thank you very much for your valuable comments. We did not compare our model with scBasset since it is only designed for single cell data analysis without any generation ability. Besides, scBasset requires the DNA sequences as the input. However, both Forebrain dataset and PBMC10K dataset do not provide the genome information. In order to further address your concern, we have reported the results of scBasset on the Hematopoiesis (Buenrostro2018) dataset.\n\n| Methods/Metrics | NMI   | ARI   | Homo  | ASW   |\n|-----------------|-------|-------|-------|-------|\n| scBasset        | 0.699 | 0.577 | 0.714 | 0.231 |\n\nWhile scBasset may outperform our model by leveraging supplementary genomic data for predicting chromatin accessibility, it faces limitations in capturing the distribution of single-cell data due to its reliance on restricted genomic information. Moreover, its deterministic nature restricts its utility in generating scATAC data.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thanks all the reviewers for their comments and constructive suggestions, which really help improve this manuscript. We have added the revised illustration of model in the PDF file. Moreover, we have computed the average latent embeddings within each cell type population and calculated the distances across different cell types.

| Comparison | AC    | EX1   | EX2   | EX3   | IN1   | IN2   | MG    | OC    |
|------------|-------|-------|-------|-------|-------|-------|-------|-------|
| AC         | -     | 0.8307| 0.7431| 0.7594| 0.8508| 0.7237| 0.8550| 0.8520|
| EX1        | 0.8307| -     | 0.4773| 0.4667| 0.6548| 0.6219| 0.7900| 0.7514|
| EX2        | 0.7431| 0.4773| -     | 0.3794| 0.5901| 0.5183| 0.7592| 0.7171|
| EX3        | 0.7594| 0.4667| 0.3794| -     | 0.6353| 0.5545| 0.7607| 0.7135|
| IN1        | 0.8508| 0.6548| 0.5901| 0.6353| -     | 0.6426| 0.8471| 0.8235|
| IN2        | 0.7237| 0.6219| 0.5183| 0.5545| 0.6426| -     | 0.7510| 0.7188|
| MG         | 0.8550| 0.7900| 0.7592| 0.7607| 0.8471| 0.7510| -     | 0.8530|
| OC         | 0.8520| 0.7514| 0.7171| 0.7135| 0.8235| 0.7188| 0.8530| -     |

Meta-review: The reviewers name the theoretical motivation and analysis of the method as well as good empirical performance as strengths. Writing is also generally speaking good, although some unclear parts were pointed out and should be clarified. Among weaknesses, some relevant comparisons were missing from the original manuscript and should be included for a more comprehensive evaluation with SOTA. Moreover, there are concerns about the evaluation metric (due to possibly misleading Euclidean distances in UMAP space).

Overall, it seems that the authors were able to address the most significant critical comments in their rebuttal (with the exception of the UMAP problem which remains an issue but, in my opinion, not a fatal one), and no further concerns were raised by the reviewers.
Decision: Accept (poster)