Multi-Modal and Multi-Attribute Generation of Single Cells with CFGen
Authors: Alessandro Palma
Till Richter
Hanyi Zhang
Manuel Lubetzki
Alexander Tong
Andrea Dittadi
Fabian J Theis
Venue: ICLR 2025 Poster (2025)
Keywords: scRNA-seq
Flow Matching
Generative modeling
Multiomics
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We devise a new method to generate discrete multi-attribute and multi-modal single-cell data using Flow matching.

Abstract: Generative modeling of single-cell RNA-seq data is crucial for tasks like trajectory inference, batch effect removal, and simulation of realistic cellular data. However, recent deep generative models simulating synthetic single cells from noise operate on pre-processed continuous gene expression approximations, overlooking the discrete nature of single-cell data, which limits their effectiveness and hinders the incorporation of robust noise models. Additionally, aspects like controllable multi-modal and multi-label generation of cellular data remain underexplored. This work introduces CellFlow for Generation (CFGen), a flow-based conditional generative model that preserves the inherent discreteness of single-cell data. CFGen reliably generates whole-genome, multi-modal, single-cell data, improving the recovery of crucial biological data characteristics while tackling relevant generative tasks such as rare cell type augmentation and batch correction. We also introduce a novel framework for compositional data generation using Flow Matching. By showcasing CFGen on a diverse set of biological datasets and settings, we provide evidence of its value to the fields of computational biology and deep generative models.

Published as a conference paper at ICLR 2025
MULTI-MODAY AND MULTI-ATTRIBUTE GENERATION OF SINGLE CELLS WITH CFGEN
Alessandro Palma 1.2
Till Richter!:2 1.2 Hanyi Zhang 1. Manuel Lubetzki'
Alexander Tong 3.4
Andrea Dittadi
Fabian J. Theis!.2
I Helmholtz Munich
Technical University of Munich Université de Montréal
Mila
MPI for Intelligent Systems, Tübingen
ABSTRACT
Generative modeling of single-cell RNA-seq data is crucial for tasks like trajectory inference, batch effect removal, and simulation of realistic cellular data. However, recent deep generative models simulating synthetic single cells from noise operate on pre-processed continuous gene expression approximations, overlooking the discrete nature of single-cell data, which limits their effectiveness and hinders the incorporation of robust noise models. Additionally, aspects like controllable multi- modal and multi-label generation of cellular data remain underexplored. This work introduces CellFlow for Generation (CFGen), a flow-based conditional generative model that preserves the inherent discreteness single-cell data. CFGen reliably generates whole-genon shooleeeeoome, multi-modal, single-cell data, improving the recovery of crucial biological data characteristics while tackling relevant generative tasks such rare cell type augmentation and batch correction. We also introduce a novel framework for compositional data generation using Flow Matching. By showcasing CFGen on diverse set of biological datasets and settings, provide evidence of its value to the fields of computational biology and deep generative models.
INTRODUCTION
Single-cell transcriptomicc has revolutionized our ability to study cell heterogeneity, revealing critical biological processes and cellular states (Rozenblatt- Rosen et al., 2017). Advances in single-cell RNA sequencing (scRNA-seq) enable high-throughput gene expression profiling across thousands of cells, providing valuable insights into cellular differentiation (Gulati et al., 2020), disease progression (Zeng & Dai, 2019), and responses drug perturbations (Ji et al., 2021). Recognizing the complexity of a cell's molecular state, modern studies increasingly integrate additional measurements beyond gene expression, such as DNA accessibility (Grandi et al., 2022) to better characterize gene regulatory mechanisms (Baysoy et al., 2023) or spatially resolved measurements to understand tissue orga- nization (Marx, 2021). Yet, technical bias and high experimental costs still hinder the homogeneous profiling of all possible cell states within the inspected biological process. Generative modeling offers a powerful approach to address these challenges by synthesizing biologically meaningful single-cell data, thereby uncovering underexplored cellular states and improving downstream analyses.
Generative models for single-cell data, in particular Variational Autoencoders (VAEs), have been extensively employed in representation learning (Lopez et al., 2018), perturbation prediction (Lot- follahi et al., 2019; 2023; Hetzel et al., 2022) and trajectory inference (Gayoso et al., 2023; Chen al., 2022). Recently, more complex approaches leveraging diffusion-based models (Luo al., 2024) or Generative Adversarial Networks (GAN) (Marouf et al., 2020) have paved the way for the task of synthetic data generation, demonstrating promising performance on realistic single-cell data modeling. Single-cell transcriptome data is inherently discrete, as gene expression is collected as the number of transcribed gene copies found experimentally. Due to the incompatibility of discrete data with continuous models such as Gaussian diffusion (Yang et al., 2023), most approaches generate data prepeeceesed through normalization and scaling. This limits their flexibility to support downstream tasks centered around raw counts, such batch correction (Lopez et al., 2018 88, differential gene expression (Love et al., 2014; Chen et al., 2025; Heumos et al., 2024), and analyses where the
"Correspondence fabi an the a@he mhol tz-muni ce

---

Published as a conference paper at ICLR 2025
Generating single-cell data with CFGen
Bemoulli decoder
DNA accessibility
Po
Probability path
P1
Generated cell
Pt
h +
b
XT
Compositional flow model
YOu
b DerneallissgmoddlAaae.)))))
(x y)
Z1
(x((a)))) w(x))
x ~. Ne(//scttaa<(//(()))))
h
X
U
Negative binomial decoder
Gene
Coll Type
Tissue
expression
CFGen generative model. A noise vector zo sampled from a Gaussian prior po transformed into latent cell representation 21 by compositional flow, conditioned on multiple biological and technical attributes. Decoders for gene expression and DNA accessibility map to the parameters of negative binomial and Bernoulli noise models, from which single-cell gene expression and DNA accessibility peaks sampled.
Figure I:
total number of transcripts in a cell is meaningful (Gulati et al., 2020). Additionally, technical and biological effects in single-cell counts have been formalized under effective discrete noise models (Hafemeister Satija, 2019), which should be incorporated into generative models for single-cell data to better approximate the underlying data generation process.
In this work, we present CellFlow for Generation (CFGen) (Fig. 1), a conditional flow-based generative model designed to reproduce multi-modal single-cell discrete counts realistically. Our approach combines the expressiveness of recent Flow Matching techniques (Albergo & Eijnden, 2023; Liu al., 2023; Lipman al., 2023; Dao al., 2023; Tong et al., 2024) with modeling the statistical properties of single-cell data across multiple modalities, each following a distinct discrete likelihood model. Moreover, we extend the current literature on Flow Matching by introducing the concept of compositional guidance, enabling the generation of cells conditioned on single attributes or combinations thereof a controlled setting.
Vanden-Eij Vanden-Eijnden,
We evaluate CFGen across multiple biological datasets, demonstrating its advantages in generative performance and downstream applications. Our main contributions are as follows:
We introduce CFGen, generative model for discrete multi-modal single-cell data that explicitly accounts for its key statistical properties under a specified noise model.
We extend the Flow Matching framework to incorporate guidance for compositional generation under multiple attributes.
We show that model's full-genome generative performance consistently outperforms existing single-cell generative models qualitatively and quantitatively on multiple biological datasets.
We showcase the application CFGen in enhancing downstream tasks, including robust data augmentation for improved classification of rare cell types and batch correction.
RELATED WORK
The synthetic generation of single-cell datasets is a well-established research direction pioneered by models using standard probabilistic methods estimate gene-wise parameters in single modality (Zappia et al., 2017; Li & Li, 2019) or multiple modality setting (Song et al., 2024). With the advent of deep generative models, VAE-based approaches have proven remarkably flexible, offering popular tools for batch correction (Lopez et al., 2018), modality integration (Gayoso et al., 2021), trajectory inference (Gayoso et al., 2023), and perturbation prediction (Lotfollahi et al., 2019; Bereket & Karaletsos, 2023). Despite their relevance, most of the mentioned approaches focus on learning meaningful cellular representations or counterfactual predictions rather than generating synthetic datasets from noise. Such task has instead been extensively explored by other works leveraging the expressive potential of diffusion models (Luo et al., 2024; Huang et al., 2025), Generative Adversarial Networks (GANs) (Marouf et al., 2020) and Large Language Models (LLMS)
2

---

Published as a conference paper at ICLR 2025
(Levine al., 2024) to produce realistic cells that approximate the observed data distribution. Our technical contribution builds upon Flow Matching (Albergo & Vanden- 2023; Liu et al., 2023; Lipman et al., 2023), an efficient formulation of continuous normalizing flows for generative modeling. Since its introduction, Flow Matching has been successfully applied to optimal transport (Tong et al., 2024; Eyring et al., 2024; Pooladian al., 2023), protein generation (Jing al., 2023; Yim et al., 2023), interpolation on general geometries (Chen & Lipman, 2024; Kapusniak et al., 2025), and guided conditional generation (Zheng al., 2023). Finally, Flow Matching showed promising performance in tasks involving scRNA-seq, such as learning cellular evolution across time (Tong al., 2024; Kapusniak et al., 2025) and responses to drugs (Klein al., 2025).
BACKGROUND
3.1 DEEP GENERATIVE MODELING OF SINGLE-CELL DATA
Single cells are represented as high-dimensional vectors of discrete counts, where each feature corresponds to gene and its measurement reflects the number of transcripts detected in cell. Technical bias and biological variation lead to unique characteristics in cells, including sparsity and over-dispersion. Sparsity arises from genes being inactive in specific cellular states (biological cause) or due to measurement dropouts in scRNA-seq (technical cause). Over-dispersion refers to the presence of greater variance than one would expect from a simple Poisson distribution of the count data (where the gene-wise variance equals the mean). This phenomenon is especially visible in highly expressed genes. Over-dispersed counts are typically modeled using a Negative Binomial (NB) distribution, parameterized by mean j and an inverse dispersion parameter 6. Formally, given nonnegative count expression matrix X E N with N cells and G' genes, entries Zing of the expression matrix are assumed to follow the negative binomial model:
Ing ~ NB(Ingg: 0g),
where Mag E Rgo is a cell-gene-specifee mean and 0, E R> is the gene-specific inverse dispersion. Thus, we assume each cell has an individual mean, while over-dispersion modeled gene-wise. This parameterization of the negative binomial can be derived from Doisson-gamma mixture, providing a natural formulation for the seRNA-seq likelihood (see Appendix B.
When scRNA is coupled with information on DNA accessibility, transcription measurements are complemented by a binary matrix B E (0. 1) 1)NX,, where P is the number of DNA regions profiled for accessibility measured as the presence (1) or absence (0) of a signal peak. Here, each measurement independently follows the Bernoulli model 6np ~ Bernoulli (Tpp)) with Tap indicating sell-gene-sppeiiee success probability.
In most single-cell representation learning settings, a deep latent variable model is trained to map latent space to the parameter space of the noise model via decoder maximizing the log- oig-liili of the data. Given a latent cell state z, the likelihood parameters of each modality are inferred as
P lp,
softmax (z)( π sigmoid (z))
where by and h are modality - specific decoders and / is the size factor, defined as the total number of counts of the generated cell. The vector p represents gene expression proportions.
3.2 CONTINUOUS NORMALIZING FLOWS AND FLOW MATCHING
Continuous Normalizing Flows (CNF). Chen et al. (2018) introduced CNFs as a generative model to approximate complex data distributions. Given data in a continuous domain 2 C Rd, we define a time-dependent probability path p [0, 1] x 2d Rz0. transforming tractable prior density Po into more complex data density pl, where we indicate the probability path time as Pt _.d - R20 such that f (z()) dz = 1. The probability path is formally generated by a time-dependent smooth vector field u, Rd + Idd with t E [0, 1]. satisfying the continuity equation dp. ăt
= - (p..))) The field is the time-derivative of an invertible flow o; Rdd R following the Ordinary Differential Equation (ODE) d
ät (z,) = 41(0) (Zu)), where Φu (Z)) = 2o and Zy is sampled from Pu- The flow & defines fush-forward transformation Pt = [ Po. transforming the prior Pu into the data density Pt. In other words, learning the vector field 2, that governs the flow allows transporting samples from po to pl by solving the ODE.
3

---

Published as a conference paper at ICLR 2025
Flow Matching. Assume the goal is to model complex data distribution q from prior Po by learning a continuous normalizing flow. One can marginalize the probability path Pt as Pz(z) = pr(z/z) (q )dzı, where 2 indicates a sample from the data distribution g and Pe(-Izı) is conditional probability path transporting noise to 2 under the boundary conditions po (z(2))) = po (z) and p1 z1) x ö(z 21). Here, 5 denotes Dirac delta measure, which places all probability mass at Z1. Note that, at t = 1, the marginal distribution pi approximates the data distribution q. Following the continuity equation, pr (z(z) generated by the marginal velocity field ue uz(z) that satisfies
uz(2) = (1(2)21) pr(z(zi)g(Z1) dz1,
pz(z)
where (z(zz) is called conditional vector field. Directly regressing uz(z) is intractable. However, Lipman et al. (2023) show that minimizing the Flow Matching objective
(E) (. [[lex, ue (z(z|z))l1
corresponds to learning approximate the marginal vector field u, with the cine-conditioned neural network V,gg with parameters Ç. Defining (z(z)z) = ((, o with the functions dg of controlling the noise schedule, u, uz(z(z) (z|21) has closed form, and Eq. (4) is tractable (see Appendix B.2 for more details). We define such a formulation as Gaussian marginal paths.
Classifier-Free Guidance (CFG). One can guide data generation a condition y by learning the conditional marginal field uy (z(z))) via a dime-conditioeel neural network vt.g ( y). Given a guidance strength hyperparameter
R. Zheng al. (2023) show that generating data points following the vector field (-yy) = (1 - ())u(.) + (-)(-))) approximates sampling from the distribution (zlyy) x ((z)-=q((e (zzly)) where q(z) and q(zly) are, respectively, the unconditional and conditional data distributions. The parameter w controls the trade-off between diversity and adherence to the condition. This approach enables guidance by interpolating between conditional and unconditional vector fields, both learned jointly during training.
4 CFGEN
Our objective is to define latent Nlw--Matching-tee -based generative model for discrete single-cell data, where each cell is measured through gene expression and, potentially, DNA accessibility. Our model, CFGen, is flexible: It can handle single and multiple modalities. Moreover, it supports guiding gener- ation conditioned on single or combinations of attributes without needing to train a different model for each. In what follows, we present the assumptions and generative process formulation in the uni-modal and multi-modal settings. We additionally illustrate our novel approach to compositional guidance.
4.1 UNI-MODAL AND SINGLE-ATTRIBUTE GENERATION
Let X E NANx be single-cell matrix where an observed single-cell vector is x No; with N and G being the number of cells and genes. Additionally, let y N be vector of categorical labels as- sociated with each observation. We also define ΣS
g= I Tg as the size factor of an individual cell x. The generative process. When the technical bias is negligible, we define the standard CFGen setting as the following generative model:
p(x, 2,2,,, v) p(x|z, (2lyy, Op())(().
where a continuous latent variable modeling the cell state, and we assumed that (1) x is independent of y conditionally on z and 1, and (2) l is independent of y. While Eq. (5) defines standard generative process, the factorization remains flexible based on data properties. Although related to existing VAE-based single-cell generative models, our proposed factorization is novel. We detail the relationship between Eq. (5) and existing generative models in Appendix B.5 and B.6.
Modeling the distributions in Eq. (5). Each factor of Eq. (5) is modeled separately: p(y) is categorical distribution Cat(N, nT,)) where N,, is the number of categories and ST, a vector of N, class probabilities, and p(1) = Log/Normal( of). The parameters of p(y) and p(l) can be learned maximum likelihood estimates
the dataset (see Appendix C.4). Given an attribute class y and size factor sampled from the respective distributions, p(zly, I) is approximated by a conditional continuous normalizing flow de (y, !), with & E [0, 1). learned via Flow Matching with Gaussian
4

---

Published as a conference paper at ICLR 2025
marginal paths (see Section 3.2, Appendix B.2 and Appendix C.2). Such flow transports samples Zy ~ ~N(0,, I) to latent cell representations z = 21 = Φ1 (z 1). Let z, = Φe (zo lg. 1). The time- derivative of the flow is parameterized velocity function ut,, ( y, (). Finally, p(x lz. 1) samples from a negative binomial distribution with mean parameterized by a decoder hg as in Eq. (2) and inverse dispersion modeled by a global parameter 0. In practice, hy and @ are optimized before training the flow, together with an encoder fg that maps the data to latent space (more details in Appendix C.I). We outline the reasons for training the encoder and the flow separately in Appendix C.5.
Sampling in practice. To generate cell using CFGen as illustrated in Eq. (5), we first sample a size factor 1 and a condition y (the latter to specify class). We then integrate the parameterized vector field (r,g(Z,, y. 1) with & E [0, 1]. starting from Za (N, I). We then take the simulated z1 = Φ (zolv. 1)
at & = 1 as our latent z in Eq. (5). Finally, we sample x ~ NB(1 softmax (hw (z1)), 0).
Size factor c E [.....,'), C}. we reformulate Eq. (5) as p(x, z, L, v) = き Ee p(x|z, (zzyyy, (p((())p(()) (()),, where assume that is independent of given I (i.e., I contains all necessary technical effect information to guide the flow), and y is independent of c. The last assumption derives from our choice of y as attribute encoding biological identity preserved across technical batches.
technical effect. When is influenced by technical effect under categorical covariate
4.2 MULTI-MODAL AND SINGLE CATLEBATTRBBTEE GENERATION
Let X and y be defined as in Section 4.1. In the multi-modal setting, we have additional access to a binary matrix B E f0. 1 NxPP representing DNA region accessibility, with P being the number of measured peaks. Each sample is, therefore, tuple (x, b. 1/), where and b are realizations of
different discrete noise models (negative binomial and Bernoulli). Following Eq. (2), both parameters of the negative binomial and Bernoulli noise models are functions of the same latent variable encoding a continuous cell state shared across modalities. We write the first factor in Eq. (5) as
pxx, b/z, 1) @2 ((x/z,))p(b/z, I) p(x|z, l)p(z)
(6)
where in (1) we use the fact that the likelihood of x and b are optimized disjointedly given z, and in (2) that b is independent of the size factor 1 (see Eq. (2)). In simple terms, all the modalities are encoded the same latent space used to train the conditional flow approximating o(zly, 1) in Eq. (5) (more details in Appendix C.I). During generation, separate decoders hy and h map a sampled latent variable z to the parameter spaces of the negative binomial and Bernoulli distributions, representing expression counts and binary DNA accessibility information, respectively (Fig. 1).
4.3 GUIDED COMPOSITIONAL GENERATION WITH MULTIPLE ATTRIBUTES
We extend CFG for Flow Matching (Zheng et al., 2023) to handle multiple attributes, enhancing control over the generative process in targeted data regions. This is especially relevant in scRNA- seq, where datasets are defined by several biological and technical covariates. Here, Y E N represents a matrix of K categorical attributes measured across N cells. Rather than training separate models for each attribute combination, we compose multiple single-attribure flow models.
Let q()ly) be the conditional data distribution, with y = (z1..... УK) being a collection of observed categorical attributes. In analogy with CFG in diffusion models (Ho & Salimans, 2021), we aim
to implement generative model to sample from g(zly) q(z) IT [(>]p>) where wi is the q(x)
w.
guidance strength for attribute i (see Appendix B.4). Diffusion models generate data by learning to approximate the score of the time-dependent density. V. log Pr (z) (zly). with neural network and using it to simulate a reverse diffusion Stochastic Differential Equation (SDE) transporting noise samples to generated data observations (Song et al., 2021). Importantly, the reverse diffusion SDE is associated with deterministic probability flow ODE with the same mame-nall densities (Yang et al., 2023).
CFG in diffusion models can be used to generate data compositionally from different attribute classes. More specifically, Liu et al. (2022) demonstrated that compositional CFG is achievable through parameterizing the drift of the generating reverse SDE with the compositional score:
K
Vz log (zly) = Vz lz) +
i=1 Wi V (z)l)))) - Vz (z)
Following the direct relationship between Flow Matching and CFG provided in Ho & Salimans (2021), we build the Flow Matching counterpart Eq. (7).
5

---

Published as a conference paper at ICLR 2025
Table I: Quantitative performance comparison of CFGen with conditional and unconditional single-cell gen- erative models. Evaluation performed based on distribution matching metries (RBF-kemel MMD and 2- Wasserstein distance). Results are averaged across datasets generated using three different seeds.
PBMC
Dentate
Tabula Muris
HLCA
MMD (
WD()
MMD (1)
WD
MMD (1)
WD
MMD (J)
WD
Conditional
C-CiGeen
0.85
4044
1.12 a o004
21.55 60.17
0.19
a..0
0.54 a.m:
10.72 0.88
scDiffusion
1.27 0.0
22.41 1:21
1.22 =00
22.56
10.0
0.24 2004
7.89 10.5
0.96 1004
15.82 a045
scVI
0.94 ans
17.66 6.29
1,15 +004
22.61 va2
0.26 0.00
9.76 +0.5
0.58 0.00
11.78 +0.19
Unconditional
u-CFGien
0.44.am
16.81 0.066
0.42 =001
21.20 0.02
0.08 0.00
8.54 : 0.06
0.15 s.0r
10.63 0..1
ScGAN
0.36
15.54 0.86
0.42
22.52 k.
0.25 0.00
0.18 0.00
10.81
Provided that we have access to Flow Matching models for the unconditional marginal vector field (z( and the single-attribute conditional fields {we (z ) the following holds:
Proposition 1 If the attributes yı, UK are conditionally independent given z, the vector field
Σ K
iz(z)y (z)
walua(zlys) ue(z)]
(8)
coincides with the velocity of the probability-flow ODE associated with the generative SDE of diffusion model with a compositional score as in Eq. (7).
We provide proof for Proposition 1 in Appendix B.4. In other words, the reversed diffusion SDE from compositional CFG admits a deterministic probability flow ODE with velocity as in Eq. (8). Consequently, CFG sampling from compositions of attributes is obtained by integrating the parameterized field E, (z, (z vt,c (z) Σ w:(er,s(2z y,) ft,e(z)) starting from samples from a Gaussian prior Po- Note that both conditional and unconditional fields are parameterized by the same model, which is learned by providing single-atribute conditioning with certain probability
during training (Algorithms and 2).
5 EXPERIMENTS
In this section, we compare CFGen with existing models in uni-modal (Section 5.1) and multi-modal (Section 5.2) generation across five datasets. We evaluate quantitatively by measuring distributional proximity between real and generated cells, and qualitatively by assessing how well models capture real data properties. In Section 5.3, we show the effectiveness multi-attribute generation in guiding synthetic samples towards specific biological labels and donors. Lastly, in Sections 5.4 and 5.5, demonstrate that CFGen enhances rare cell type classification through targeted data augmentation and performs batch correction on par with widely used VAE-based models.
5.1 COMPARISON WITH EXISTING METHODS ON UNI-MODAL SCRNA A SEQ GENERATION
We evaluate the performance of CFGen conditionally and unconditionally against three baselines. Baselines. We choose VI (Gayoso et al., 2021) and scDiffusion (Luo et al., 2024) as conditional models and scGAN (Marouf al., 2020) as unconditional baseline. The sc'I model is based on a VAE architecture with negative binomial decoder and performs generation by decoding low- dimensional Gaussian noise into parameters of the likelihood model. Conversely, scDiffusion and scGAN operate continuous-space domain, performing generation using standard latent diffusion (Rombach et al., 2022) and GAN (Goodfellow et al., 2014) models. Thus, we train them using normalized counts (more in Appendix D).
Datasets. We assess model performance on four datasets of varying size: (i) PBMC3K (2,638 cells from a healthy donor, clustering into 8 cell types), (ii) Dentate gyrus (La Manno et al., 2018) (18,213 cells from a developing mouse hippocampus), (iii) Tabula Muris (Tabula Muris Consortium et al., 2018) (245,389 cells from Mus muscudus across multiple tissues), and (iv) Human Lung Cell Atlas (HLCA) (Sikkema et al., 2023) (584,94 cells from 486 individuals across 49 datasets). Conditioning
Phttps://sade jalai.aalbbbbo... /tttas///attjjalaa.ooo///sse /tte http://uurrtt/rrrttte/////////// -utorri al hhm1

---

Published as a conference paper at ICLR 2025
is performed on cell type for all datasets except Tabula Muris, where use the rissue label. Dataset descriptions and pre- pro-procsssing details are in Appendix E and Table 4.
Quantitative evaluation. As evaluation metrics, Maximum Mean Discrepancy (MMD) (Borgwardt et al., 2006) and 2-Wasserstein distance) computed between the Principal Component (PC) projections of generated and real test data in 30 dimensions. The generated data is embedded using the PC loadings of the real data for comparability. For conditional models, we evaluate the metrics per cell type and average the results. All evaluations are performed on a held-out set of cells, considering the whole genome, with a filtering step for low expression genes (see Appendix E).
use distribution distances (RBF-kernel
Quantitative results. In Table 1, we evaluate the generative performance of CFGen conditionally (c-CFGenn) and unconditionally (u-CFGen) against the three baselines on the scRNA-sen generation task. CFGen consistently reaches the highest performance conditional generation across biological categories and overcomes scGAN on three out of four datasets on the unconditional generation task.
PBMC3K
Dantate gycus
gynus
0.9224
1.00
0.9926
0.73
Gane wwrkoo
0.73
0.0034
0.00
0.008:
6.22
0.23
0.8002
2.0
Tabolla Muris
6.5
Wiitlaice acGAN
Nem CTCen
0,0000
Tabula Muris
9.00025
HLCA
1.5
9.00020
8.0-
0.3
0.00015
9.0001
0.5
0.1
0.0001
8,00005
0.0
0.0
0.0000
0,00000
Dean Dog GEAPI
IDAuS Oag DEXPs
per
per
Figure 2: (a) Comparison between the gene- wise empirical mean- variance trend real data and samples from generative models. (b) Frequency of the number of zeroes per cell in real and generated data.
Qualitative evaluation. Evaluating realistic data generation in biology requires more than distribution-matchingg metrics. We compare CFGen with diffusion and GAN-based single-cell generators on the task of modeling the probabilistic properties of the single-cell data. Specifically, consider how well different methods recover the following aspects from real gene expression counts: (1) Sparsity: caused by technical biases in gene transcript detection gene inactivity in specific contexts. (2) Over-dispersion: a nonlinear mean-variance relationship, modeled through the inverse dispersion parameter of negative binomial distribution. (3) Discreteness.
Qualitative results. In Fig. 2, we provide qualitative evidence that CFGen is more effective in recovering properties (1) and (2) compared to scDiffusion and scGAN, which assume a continuous data space. Property (3) naturally follows when modeling discrete counts with CFGen. Specifically, Fig. 2a shows that explicitly modeling counts with gene-specific inverse dispersion leads to better alignment of the generated gene- wiie mean-variance relationship with real data. Additionally, Fig. 2b demonstrates the recovery the actual distribution of zero counts per cell. In contrast, scDiffusion often shifts towards actively expressed genes, while scGAN tends to either under overestimate data sparsity. Furthermore, CFGen is the only conditional model capable of generating plausible synthetic cells in terms of overlap with the real data distribution for large datasets such as the HLCA and Tabula Muris (see Appendix H.3 and Fig. A6).
5.2 MULTI-MODAL GENERATION
We evaluate the qualitative and quantitative performance of CFGen at generating multi-modal data comprising gene expression and binary DNA-region accessibility.
Baselines. We compare CFGen with a VAE-based multi-moda generative model (MultiVI) (Ashuach al., 2023). For completeness, add as basclines two single-modality generative models: Ve (Ashuach et al., 2022) (DNA accessibility) and se scVI (Lopez et al., 2018) (gene expression). Finally, include scDiffusion and uni-modal CFGen (CFGen RNA) baselines for scRNA-seq generation.
7

---

Published as a conference paper at ICLR 2025
Datasets. We use the multiome PBMCIOK dataset, made available by 10X Genomics 2_ Here, each cell is measured both in gene expression (RNA) and DNA accessibility (ATAC). The dataset consists of 10K cells across 25,604 genes and 40,086 peaks and was annotated with 15 cell types, with their respective marker peaks (enriched in accessible or inaccessible points) and genes.
Evaluation. We use the RBF-kernel MMD and 2- 2-Wasserstein distances in the same setting described in Section 5.1. Before comparison, we normalize both real and generated binary measurements of DNA accessibility via TF-IDF (Aizawa, 2003) (in analogy to text mining). The metrics are computed a 30-dimensional PC projection of the generated cells, using the PC loadings of the real data. RNA counts are treated in Section 5.1. In Appendix H.4 we compare CFGen and Multi more biologically. Specifically, we assess how well they approx-
Table 2: Comparison between CFGen, scDiffusion and VAE-based models on generating multiple single-cell modalities. We report distribution distance performance (RBF-kemel MMD and 2- Wasserstein distance) between real and generated cells across three seeds. Underlined values indicate the second- accond-best performance.
RNA
ATAC
MMD (L)
WD (.)
MMD (L)
(D)
CIGien multi.
0.89
13.90 2
0.92
18,86
CFGen RNA
2.82
+4.3
scDiff.
+018
MuluVI
0.80
15.92 2
a.0
21.09 2034
PeakVI
1.49
20.84
scVI
0.95
14.38
imate per-cell-type marker peaks and gene expression (see Appendix F.4). For each cell type, we compute the accessibility fraction and mean expression of literature-derived marker peaks and genes in both real and generated cells and report their correlation per cell type in Fig. A7b.
Results. CFGen outperforms both MultiVI and PeakVI in modeling accessibility data based on distribution matching metrics (see Table 2). When considering the RNA modality, our model surpasses VI and scDiffusion in all metrics and MultiVI in terms of 2-Wasserstein distance. Qualitatively, Fig. A7a shows substantial overlap between real and generated modalities. Finally, Fig. A7b demonstrates that CFGen better approximates average marker peak accessibility and gene expression, outperforming MultiVI across all cell type categories.
5.3 MUL.TI-ATTRIBUTE GENERATION AND GUIDANCE
We assess our approach to compositional guidance, as outlined in Section 4.3.
Datasets. We showcase guidance on datasets with extensive technical variation, one could combine different levels of biological and technical annotations to either augment rare cell type and batch combinations or control for the amount of technical effect added in simulation settings. Specifically, we consider (i) The NeurlPS 2021 dataset (Luecken et al., 2021a) 90,261 bone marrow cells from 12 healthy human donors. We use donor as batch attribute and cell type as a biological covariate. We also consider (ii) the Tabular Muris dataset described in Section 5.1, using tissue and Mouse ID as covariates.
Evaluation. The power of our guidance model is to generate data conditionally on arbitrary subset of attributes- including unconditional generation using a single trained model. For each pair of covariates (y,, Vy), we evaluate generation on 500 generated cells varying the parameter wj. keeping w; fixed. The expected result is conditional generation on Wi when Wj- = 0 and generation from the intersection between the two attributes as wj increases. We additionally test the unconditional model, given by Wi = W = O, expected to recover the whole single-cell dataset. In the unconditional case, we generate as many cells as there are in the dataset to better evaluate the coverage.
Results. Visual guidance results are shown in Fig. 3. with examples of double-attribute guidance between CD14+ monocytes and donor 1 for the NeurIPS 2021 dataset and tongue and mouse 18-M-52 for Tabula Muris. Unconditional generation recreates the original data (left-hand side) for both datasets. Setting guidance weights to zero for mouse IDD and donor attributes leads to single-attribute conditional generation. Increasing the guidance weight steers the generation to the intersection of the two attributes. Quantitative results on attribute intersection generation quality are in Appendix H.7.
5.4 APPLICATION: DATA AUGMENTATION TO IMPROVE CLASSIFICATION OF RARE CELL TYPES
We explore using CFGen to improve cell-type classifier generalization by augmenting rare cell types in datasets. Previous work has shown data augmentation enhances cell type classification (Richter et al., 2024). As a classifier, we use ScGPT (Cui et al., 2024), transformer pre-trained on 33 million cells.
httpss: : //www Oxgenomi cs. ctm.coo//saaa/ htmm///pprr//// / ngng e-ggle-oo cel1 11 1-mult ome- atac- stac-plus-gen e-expression
8

---

we invert the generative flow to remove the attribute structure, Next, we simulate the forward flow Published as a conference paper at ICLR 2025
Aái dence cllect
NeutPS dataset- cell typo and donor
Wolll type 0
Real CD14+ Mono
Manee Well
Wenner Weell
Real donor
Wool
Generated
Tabula tissue mouse ID
Wimnue 0
Add mouse ID effect
Whsso
Real Tongue
Wmonne
Watsoue 1
Wasouse id Weieoue
Wimouse id 18-M-52
Wisssse
1
Figure 3: Qualitative evaluation of guidance performance on attribute pairs in the NeurIPS 2021 and Tabula Muris datasets. Left: unconditional performance with guidance weights at O. Moving right: simulate 500 cells, progressively increasing the guidance strength of one attribute while keeping the counterpart unchanged.
Datasets. We leverage two large datasets: (i) PBMC COVID (Yoshida et al., 2022) - 422,220 blood cells from 93 patients ranging across paediatric and adult. (ii) The HLCA dataset described in Section 5. Both datasets are processed by selecting 2000 highly variable features and holding out cells from 20% of the donors.
Evaluation. We train CFGen on the PBMC COVID and HLCA training sets and succes- sively augment both to 800,000 samples by upsampling rare cell types. For each cell type, we compute -. where Net is the total number of cells from a cell type ct. We then generate observations to fill the gap between the dataset size and 800,000 cells, sampling cells propor- tional to the inverse of their cell type fieeueecy. This process yields significantly more observa- tions for rare cell types. However, we still do not reach uniformity, as class imbalance may be biologically meaningful. Following the original publication, we train kNN cell-type classifiers on scGPT's embeddings from the original and
Augmented Butter
Non- Augmmetgttt Better
Equal Performance
Hourson -0.31
Pearson -0.29
PBMC COVID
A Performance
HLCA & Performance
Co E
Relative Cell Type Prequency (%)
Figure 4: Cell-type classification recall difference be- fore and after augmentation as function of cell type frequency. The classifier is 10-ncarest neighbor (kNN) model trained the scGPT's representation space. augmented training sets, evaluating the recall performance on held-out donors. For each cell type, we assess if performance increases upon augmentation as function of its frequency in the dataset.
Results. Our results are displayed in Fig. 4 for the two datasets. Remarkably, most cell types in the held-out dataset are better classified after augmentation, suggesting that CFGen not only generates reliable cell samples but can be a valuable supplement to relevant downstream tasks. Moreover, the performance difference between before and after augmentation is inversely proportional to the frequency of the cell type in the dataset. Therefore, the improvement in generalization is more accentuated for rare cell types. Finally, Fig. A10 in the Appendix shows that augmentation via CFGen outperforms the competing methods at improving the generalization performance on rare cell types in unseen donors. We provide raw cell type recall metries in Table and Table
5.5 APPLICATION: BATCH CORRECTION
We apply multi-attribute CFGen to batch correction (see Fig. 5), a common use case for generative models in seRNA-seq (Tran al., 2020; Luecken al., 2021b). Given a dataset with batch labels,
(i) choose reference batch yhatch For latent cell representation 2, with attributes Ybatch and type'
from the obtained representations while fixing the cell type and assigning yref to all observations.
Guidance weights regulate the preservation of biological versus batch labels.

---

Published as a conference paper at ICLR 2025
Uncorrected
Noise
Currected
50c3 sied
Inverted flow
Generative flow
NeurIPS
y/ Noathh
L9 eype
baatch
L. type
UMAPI
Wituro 5C3 Parmo 563
Vistertion 300 minutes
Watersion minutes Vanterson minuues
C. Elegans
Inverted flow
Generative flow
UMAPI
Pbatch
Mond 1y2*
Shatch
Figure 5: To perform batch correction, the scRNA-sco -scq latent distribution is mapped to the prior distribution by inverting the flow model. The resulting points are then transported back to the data domain based common ref- erence batch label and the original cell type label to preserve the biological structure. Cells are colored by batch.
Datasets. We evaluate CFGen as a batch correction method on two datasets: (i) The NeurIPS dataset described in Section 5.3, using cell type as biological variable to preserve and acquisition site as batch variable. (ii) The C. Elegans molecular atlas (Packer et al., 2019), which profiles 89,701 cells across 7 sources (batches). Similarly to (i), we use cell type as a biological annotation.
Table 3: Average batch correction and bio- logical conservation metrics from the seIB package comparing CFGen with VAE-based batch correction models a 50-dimensional representation space. PC projections of the data are used to evaluate the uncorrected data.
NeurPPS
C. Elogans
Butch (?)
(1)
Batch (?)
(ia
Evaluation. We compare our model with established VAE-based integration methods: ScANVI (Xu et al., 2021), ScVI, and scPoli (De Donno et al., 2023). Using scIB met- rics (Luecken et al., 2021b), we assess batch correction and biological conservation based on neighborhood composition in the embedding space (see Ap- pendix F.7). All methods are evaluated on a 50-dimensional latent space, with scores from the PC representation of uncorrected data included for comparison. We find that setting Whatch 1 and type = 2 for C.Elegans and Whatch = 2 and weell type. = 1 for NeurIPS preserves cell type variation while correcting for technical variation (see Appendix H.8 for more details on the selection).
CHGen scPoli
0.63
0.61
0.68
0.63
0.55
0.64
0.61
0.56
SCANVI
0.68
0.59
seVI 0.45
0.63
0.58
0.55
Uncoreettd
0.33
0.62
0.40
0.53
Results. We present our technical effect correction approach alongside qualitative results in Fig. 5, illustrating batch mixing performance across both datasets. In Table 3, CFGen is benchmarked against baseline methods. Our model achieves superior batch correction, surpassing the second-best approach by 8% on the NeurIPS dataset and 7% on C. Elegans. On the latter dataset, CFGen additionally outperforms baseline models in preserving biological structure after correction.
CONCLUSION
We presented CFGen, a conditional latent flow-based generative model for single-cell discrete data that combines state-of-ft -fate-of-the-art generative approaches with rigorous probabilistic considerations. CFGen incorporates established noise models to sample realistic gene expression and DNA accessibility states, with promising applications in data augmentation and batch correction tasks. Furthermore, our model demonstrates improved performance over existing generative frameworks, reproducing data more faithfully across modalities. Our core machine learning contribution extends classifier-fre free guidance in Flow Matching with compositional generation of multiple attributes. Overall, CFGen represents a significant advancement in the simulation and augmentation of single-cell data, offering the research community powerful tools to support biological analysis.
Limitations. Our framework relies on multiple assumptions, including independence in the data, which may not hold in all biological contexts. Thus, exploring data characteristics is essential before using CFGen for generation. Furthermore, we currently train the autoencoder based representation framework separately from the generative flow, which can be inefficient and memory-intensive.
10

---

Published as a conference paper at ICLR 2025
ETHICS STATEMENT
This work explores the core features of single-cell data and examines how capturing complex, high- dimensional cellular information can assist answering biological questions. We aim to release CFGen as a user-friendly, open-source tool to facilitate its adoption in single-cell analysis, Given its application in biological research, CFGen may be utilized in sensitive environments that involve clinical data and patient information.
REPRODUCIBILITY STATEMENT
Reproduction details are reported in the Appendix and the main text. The proof for Proposition is extensively described in Appendix B.4, while prior knowledge on Flow Matching and classifier- free guidance is provided in Appendix B.2 and Appendix B.3. Algorithms for training and sampling with CFGen are reported Appendix G. We introduce a thorough model description of both the autoencoder and flow components, together with modeling choices in Appendix C. Basclines and their characteristics are reported in Appendix D. All datasets are publicly available, and their source publications referenced in the main text. We additionally summarize dataset characteristics in Table 4. Metrics and experimental setups are detailed in Appendix F.
ACKNOWLEDGMENTS
A.P. and T.R. are supported by the Helmholtz Association under the joint research school Munich School for Data Science (MUDS). Additionally, A.P., T.R. and FJ..T. acknowledge support from the German Federal Ministry of Education and Research (BMBF) through grant numbers 03 1L0289 and 18053A. T.R. and F.J.T. also acknowledge support from the Helmholtz Association's Initiative and Networking Fund via the CausalCellDynamics project (grant number Interlabs-0022), Additionally, F.J.T. acknowledges support from the European Union (ERC, DeepCell grant number 101054957). Finally, A.D. acknowledges support from G-Research.
REFERENCES
Akiko Aizawa. An information-theoreet perspective of tf-idf measures. Information Processing and; Management, 39(1):45-65 65, Jamuary 2003. ISSN 0306-4573. doi: 10.101660330-4553300000000
Michael Samuel Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representations, 2023.
Tal Ashuach, Daniel Reidenbach, Adam Gayoso, and Nir Yosef. Peakvi: A deep generative model for single-cell chromatin accessibility analysis. Cell Reports Methods, 203):10018,, 2022. ISSN 2667-2375. doi: https://doi.org/ 10. 1011//...meeh...... crmeth.2m22..... 2022.
Tal Ashuach, Mariano Gabitto, Rohan V Koodli, Giuseppe-Antonic Saldi, Michael I Jordan, and Nir Yosef. Multivi: deep generative model for the integration of multimodal data. Nature Methods, 2023. ISSN 1548-7105. doi: https://doi.or org/10. 10. 100033441122--2--099999999
Alev Baysoy, Zhiliang Bai, Rahul Satja,, and Rong Fan. The technological landscape and applications of single-cell multi-omics. Nature Reviews Molecular Cell Biology, 24(10))695-713, 2023. ISSN 1471-0080. doi: https://doi.or//100
Michael Bereket and Theofanis Karaletsos. Modelling cellular perturbations with the sparse additive mechanism shift variational autoencoder. Advances in Neural Information Processing Systems, 36: 1-12, 2023.
Karsten M. Borgwardt, Arthur Gretton, Malte Rasch, Hans-Peter Kriegel, Bernhard Schölkopf, and Alex J. Smola. Integrating structured biological data by Kernel Maximum Mean Discrepancy. Bioinformatics, 22(14) 7, 07 2006. ISSN 1367-4803. doi: 10. 2..00///ooffommatc///h2222.
Danila Bredikhin, Ilia Kats, and Oliver Stegle. Muon: multimodal omics analysis framework. Genome Biology, 23(1):42, 2022.
11

---

Published as a conference paper at ICLR 2025
Ricky T. Q. Chen and Yaron Lipman. Flow matching general geometries. In The Twelfth International Conference on Learning Representations, 2024.
Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
Yunshun Chen, Lizhong Chen, Aaron T L Lun, Pedro L Baldoni, and Gordon Smyth. edger v4: powerful differential analysis of sequencing data with expanded functionality and improved support for small counts and larger datasets. Nucleic Acids Research, 53(2), January 2025. ISSN 1362-4962. doi: 10. 10.1093/nar/gka000
Zhanlin Chen, William C. King, Aheyon Hwang, Mark Gerstein, and Jing Zhang, Deepvelo: Single- cell transcriptomic deep velocity field learning with neural ordinary differential equations. Science Advances, 8(48), December 2022. ISSN 2375-2548.
doi: 10.1 111///i/yy ab44375..
Pietro E Cippà and Thomas F Mueller. A first step toward a cross-tissue atlas of immune cells in humans. Transplantation, 107(1):8-9, 2023.
Haotian Cui, Chloe Wang, Hassaan Maan, Kuan Pang, Fengning Luo, Nan Duan, and Bo Wang. segpt: toward building a foundation model for single-cell multi-omics using generative ai. Nature Methods, pp. 1-11, 2024. ISSN 1548-7105. doi: httpps:/ddoi.or//1100 1..0000//111922001 022--220-----1---
Quan Dao, Hao Phung. Binh Nguyen, and Anh Tran. Flow matching in latent space. arXiv preprint
arXiv:2307.08698, 2023.
Carlo De Donno, Soroor Hediyeh-Zadeh, Amir Ali Moinfar, Marco Wagenstetter, Luke Zappia, Mohammad Lotfollahi, and Fabian J. Theis. Population-level integration single-cell datasets enables multi-scale analysis across samples. Nature Methods, 20(11):1683-1622 October 2023.
ISSN 1548-7105, doi: 10. 10.1038/<41592-022-00000
Luca Eyring, Dominik Klein, Théo Uscidda, Giovanni Palla, Niki Kilbertus, Zeynep Akata, and Fabian J Theis. Unbalancedness in neural monge maps improves unpaired domain translation. In The Twelfth International Conference on Learning Representations, 2024.
Rémi Flamary. Nicolas Courty, Alexandre Gramfort, Mokhtar Z. Alaya, Aurélie Boisbunon, Stanislas Chambon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, Léo Gautheron, Nathalie T.H. Gayraud, Hicham Janati, Alain Rakotomamonjy, levgen Redko, Antoine Rolet, Antony Schutz, Vivien Seguy, Danica Sutherland, Romain Tavenard, Alexander Tong, and Titouan Vayer. Pot: Python optimal transport. Journal of Machine Learning Research, 22(78):1-8, 2021.
Adam Gayoso, Zoë Steier, Romain Lopez, Jeffrey Regier, Kristopher I. Nazor, Aaron Streets, and Nir Yosef. Joint probabilistic modeling of single-cell multi-omic data with totalvi. Nature methods, 18 (2):272-282, 2021. ISSN 1548-7105. doi: https:/doi..rr// org/ 1008///1592-020--22
Adam Gayoso, Philipp Weiler, Mohammad Lotfollahi, Dominik Klein, Justin Hong, Aaron Streets, Fabian Theis, and Nir Yosef. Deep generative modeling of transcriptional dynamics for ma velocity analysis in single cells. Nature Methods, 21(1):50-59, September 2023. ISSN 1548-7105. doi: 10.1038/s41592-000 023-0199--0.....
Ian Goodfellow, Jean Pouget-Abadia Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In 7. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.
Fiorella C Grandi, Hailey Modi, Lueas Kampman, and Ryan Corces. Chromatin accessibility profiling by atac-seq. Nature protocols, 1186:1118-1552 2022. ISSN 1750-2799. doi: https:
019-8444156-002-006999
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723-773, 2012.
12

---

Published as a conference paper at ICLR 2025
Gunsagar S. Gulati, Shaheen S. Sikandar, Daniel J. Wesche, Anoop Manjunath, Anjan Bharadwaj. Mark Berger, Francisco Ilagan, Angera H. Kuo, Robert W. Hsich, Shang Cai, Maider Zabala, Ferenc A. Scheeren, Neethan A. Lobo, Dalong Qian, Feiqiao B. Yu, Frederick M. Dirbas, Michael F. Clarke, and Aaron M. Newman. Single-cell transcriptional diversity is a hallmark of developmental potential. Science, 367(6476):405 January 2020. ISSN 1095-9203. doi: 10. 126/science.aa x0249.
Christoph Hafemeister and Rahul Satjaa Normalization and variance stabilization of single-cell ma-seq data using regularized negative binomial regression. Genome Biology, 20(1), December 2019. ISSN 1474-760X. doi: 10.1 2..1166/s1300 10000//3999-191118777
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
doi: 10. N0.119//VPR.2016.9900
Leon Hetzel, Simon Boehm, Niki Kilbertus, Stephan Günnemann, Mohammad Lotfollahi, and Fabian Theis. Predicting cellular responses to novel drug perturbations at a single-cell resolution. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances Neural Information Processing Systems, volume 35, pp. 26711 222222 Curran Associates, Inc., 2022.
Lukas Heumos, Yuge Ji, Lilly May, Tessa Green, Xinyue Zhang, Xichen Wu, Johannes Ostner, Stefan Peidli, Antonia Schumacher, Karin Hrovatin, Michaela Müller, Faye Chong, Gregor Sturm, Alejandro Tejada, Emma Dann, Mingze Dong, Mojtaba Bahrami, Ilan Gold, Sergei Rybakov, Altana Namsaraeva, Amir Moinfar, Zihe Zheng. Eljas Roellin, Isra Mekki, Chris Sander, Moham- mad Lotfollahi, Herbert B. Schiller, and Fabian J. Theis. Pertpy: an end-to-end framework for perturbation analysis. bioRxiv, 2024. doi: 10. 10.1101/2024.08.04.66666 60.66666.......6
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurlPS 202/ Workshop on Deep Generative Models and Downstream Applications, 2021.
Lei Huang, Lei Xiong, Na Sun, Zunpeng Liu, Ka-Chun Wong, and Manolis Kellis. A versatile informative diffusion model for single-cell atac-seq data generation and analysis. Advances in Neural Information Processing Systems, 37:45462-45484, 2025.
Yuge Ji, Mohammad Lotfollahi, Alexander Wolf, and Fabian J. Theis. Machine learning for perturbational single-cell omics. Cell Systems, 12(6):522-537, 2021. ISSN 2405-4712. doi:
https://doi.or//10 c0116f.cels.202..05000
Bowen Jing, Bonnie Berger, and Tommi Jaakkola. Alphafold meets flow matching for generating protein ensembles. In NewrIPS 2023 Generative Al and Biology (GenBio) Workshop, 2023.
Kacper Kapusniak, Peter Potaptchik, Teodora Reu, Leo Zhang, Alexander Tong, Michael Bronstein, Joey Bose, and Francesco Di Giovanni. Metric flow matching for smooth interpolations on the data manifold. Advances in Neural Information Processing Systems, 37:135011-13500 2025.
Diederik Kingma. Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 21696-21707, Curran Associates, Inc.,
2021.
Dominik Klein, Théo Uscidda, Fabian Theis, and Marco Cuturi. Genot: Entropic (gromov) wasser- stein flow matching with applications to single-cell genomics. Advances in Neural Information Processing Systems. 37:103897-103944, 2025.
Gioele La Manno, Ruslan Soldatov, Amit Zeisel, Emelie Braun, Hannah Hochgemer, Viktor Petukhov, Katja Lidschreiber, Maria E Kastriti, Peter I. Lönnerberg, Alessandro Furlan, et al. Rna velocity of single cells. Nature, 560(7719):494 2018. ISSN 1476-4687. doi: https://doi.org/100 1038//4 1586-018-0414-6
Daniel Levine, Syed A Rizvi, Sacha Lévy, Nazreen Pallikkavaliyaveeti,, David Zhang, Xingyu Chen, Sina Ghadermarzi, Ruiming Wu, Zihe Zheng, Ivan Vrkic, Anna Zhong, Daphne Raskin, Insu Han, Antonio Henrique De Oliveira Fonseca, Josue Ortega Caro, Amin Karbasi, Rahul Madhav Dhodapkar, and David Van Dijk. Cell2Sentence: Teaching large language models the language
13

---

Published as a conference paper at ICLR 2025
of biology. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 27299- 27325. PMLR, 21-27 Jul 2024.
Wei Vivian Li and Jingyi Jessica Li. A statistical simulator scDesign for rational scRNA-seq experimental design. Bioinformatics, 35(14)i41-i50, 07 2019. ISSN 1367-4803, doi: 1093/bi oinformatics/bt2222..
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. The Eleventh International Conference on Learning Repre- sentations, 2023.
Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. In European Conference on Computer Vision, pp. 423-439. Springer, 2022.
Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023.
Romain Lopez, Jeffrey Regier, Michael B. Cole, Michael I. Jordan, and Nir Yosef. Deep generative modeling for single-cell transcriptomics. Nature Methods, 15(12):1053 1058, November 2018. ISSN 1548-7105. doi: 10. 11100///41592-01-00229999
Mohammad Lotfollahi, Alexander Wolf, and Fabian J Theis. segen predicts single-cell perturbation responses. Nature methods, 16(8):715-7211 2019. doi: 10. 10.1008/41192--1--4449999
Mohammad Lotfollahi, Anna Klimovskaia Susmelj, Carlo De Donno, Leon Hetzel, Yuge Ji, Ignacio Ibarra, Sanjay R Srivatsan, Mohsen Naghipourfar. Riza M Daza, Beth Martin, Jay Shendure, Jose L McFaline-Figueroa, Pierre Boycau, Alexander Wolf, Nafissa Yakubova, Stephan Günnemann, Cole Trapnell, David Lopez-Paz, and Fabian J Theis. Predicting cellular responses to complex perturbations in high-throughput screens. Molecular Systems Biology, 19(6), May 2023. ISSN 1744-4292. doi: (0.15252/msb..022111 202211517.
Michael I Love, Wolfgang Huber, and Simon Anders. Moderated estimation of fold change and dispersion for rna-seq data with deseq2. Genome Biology, 15(12), December 2014. ISSN 1474- 760X. doi: 10. 1886///155--0--0000
Malte Luecken, Daniel Burkhardt, Robrecht Cannoodt, Christopher Lance, Aditi Agrawal, Hananeh Aliee, Ann Chen, Louise Deconinck, Angela Detweiler, Alejandro Granados, Shelly Huynh, Laura Isacco, Yang Kim, Dominik Klein, Bony De Kumar, Sunil Kuppasani, Heiko Lickert, Aaron McGeever, Joaquin Melgarejo, Honey Mekonen, Maurizio Morri, Michaela Müller, Norma Neff, Sheryl Paul, Bastian Rieck, Kaylie Schneider, Scott Steelman, Michael Sterr, Daniel Treacy, Alexander Tong, Alexandra-Chle Villani, Guilin Wang. Jia Yan, Ce Zhang, Angela Pisco, Smita Krishnaswamy, Fabian Theis, and Jonathan M Bloom. sandbox for prediction and integration of dna, ma, and proteins in single cells. In J. Vanschoren and S. Yeung (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1, 2021a.
Malte Luecken, M. Büttner, K. Chaichoompu, Danese, M. Interlandi, M. F. Mueller, D. C. Strobl, L. Zappia, M. Dugas, M. Colomé and Fabian J. Theis. Benchmarking atlas-level data integration in single-cell genomics. Nature Methods, 19(1):41-50, December 2021b. ISSN 1548-7105. doi: 10. 111138//411592-201-11336666666
Erpai Luo, Minsheng Hao, Lei Wei, and Xuegong Zhang. scdiffusion: conditional generation of high-quality single-cell data using diffusion model. Bioinformatics, 40(9))btae511. 2024.
Mohamed Marouf, Pierre Machart, Vikas Bansal, Christoph Kilian, Daniel S Magruder, Christian Krebs, and Stefan Bonn. Realistic in silico generation and augmentation of single-cell ma-seq data using generative adversarial networks. Nature communications, 11(1):166, 2020. ISSN 2041-1723
doi:
14

---

Published as a conference paper at ICLR 2025
Vivien Marx. Method of the year: spatially resolved transcriptomics. Nature Methods, 18(1):9-14, January 2021. ISSN 1548-7105. doi: 10.10038644152--22--0003333
Jonathan S. Packer, Qin Zhu, Chau Huynh, Priya Sivaramakrishnan, Elicia Preston, Hannah Dueck, Derek Stefanik, Kai Tan, Cole Trapnell, Junhyong Kim, Robert H. Waterston, and John I. Murray, A lineage-resolved molecular atlas elegans embryogenesis at single-cell resolution. Science, 365(6459), September 2019, ISSN 1095-9203. doi: 10. 1el11///eecce.aaa19999
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary Vito,o, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Michael Poli, Stefano Massaroli, Atsushi Yamashita, Hajime Asama, Jinkyoo Park, and Stefano Ermon. Torchdyn: Implicit models and neural numerical methods in pytorch. In Neural Information Processing Systems, Workshop on Physical Reasoning and Inductive Biases for the Real World, volume 2, 2021.
Aram-Alexandre Pooladian, Heli Ben-Hamu, Carles Domingo-Enrich, Brandon Amos, Yaron Lipman, and Ricky T. Q. Chen. Multisample flow matching: Straightening flows with minibatch couplings. In Andreas Krause, Emma Brunskill, Kyunghyun Cho. Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 28100-28123 PMLR, 23-29 Jul 2023.
Till Richter, Mojtaba Bahrami, Yufan Xia, David Fischer, and Fabian Theis. Delineating the effective of self self-supervised learning in single-cell genomics. Nature Machine Intelligence, pp. 1-11, 2024.
Robin Rombach, Patrick Esser, and Bjorn Ommer. Network-to-netwwol translation with conditional invertible neural networks. Advances in Neural Information Processing Systems, 33:2784-2777 2020.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pp. 10684-10695, June 2022.
Orit Rozenblatt- Rosen, Michael J. T. Stubbington, Aviv Regev, and Sarah A. Teichmann. The human cell atlas: from vision to reality. Nature, 550(7677):45 453, October 2017. ISSN 1476-4687. doi: 10. 1038555/5044
Lisa Sikkema, Ciro Ramírez-Suástepee Daniel C Strobl, Tessa E Gillett, Luke Zappia, Elo Madissoon, Nikolay Markov, Emmanueleee Zaragosi, Yuge Ji, Meshal Ansari, et al. An integrated cell atlas of the lung in health and disease. Nature Medicine, 296))1163-11577 2023. ISSN 1546-170X doi: https://doi.org/ 10. 1038/441591-023-022277
Dongyuan Song, Qingyang Wang, Guanao Yan, Tianyang Liu, Tianyi Sun, and Jingyi Jessica Li. scdesign3 generates realistic in silico data for multimodal single-cell and spatial omics. Nature Biotechnology, 42(2):247-255 2024. ISSN 1546-1696. doi: https://doi.org/10. http://....rr//10000088811111100 1387//44777700 -01772-1.
Yang Song, Jascha
Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. International Conference on Learning Representations, 2021.
Tabula Muris Consortium, Overall coordination. Logistical coordination, Organ collection and pro-
cessing, Library preparation and sequencing, Computational data analysis, Cell type annotation, Writing group. Supplemental text writing group, and Principal investigators. Single-cell tran- scriptomics of 20 mouse organs creates tabula muris. Nature, 562(7727)337-372) October 2018.
Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang. Jarrid Rector- Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. Expert Certification.
15

---

Published as a conference paper at ICLR 2025
Hoa Thi Nhu Tran, Kok Siong Ang, Marion Chevrier, Xiaomeng Zhang, Nicole Yee Shin Lee, Michelle Goh, and Jinmiao Chen. A benchmark of batch-effect correction methods for single-cell sequencing data. Genome Biology, 21(1), January 2020. ISSN 1474-760X. doi: 10.1186/s130 59-019- 1850-9.
Guido Van Rossum and Fred L. Drake. Python 3 Reference Manual. CreateSpace, Scotts Valley, CA, 2009. ISBN 1441412697.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Zhizhong Wang, Lei Zhao, and Wei Xing. Stylediffusion: Controllable disentangled style transfer via diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7677-7689, 2023.
Alexander Wolf, Philipp Angerer, and Fabian Theis. Scanpy: large-scale single-cell gene expression data analysis. Genome biology, 19:1-5, 2018. ISSN 1474-760X. doi: http::/doi.org/10.. 186/s130 59-017-1382-0.
Chenling Xu, Romain Lopez, Edouard Mchlman, Jeffrey Regier, Michael Jordan, and Nir Yosef. Probabilistic harmonization and annotation of single-cell transcriptomics data with deep generative models. Molecular Systems Biology, 17(1), January 2021. ISSN 1744-4292. doi: 10. 10.15252/msb.22 0209620.
Ling Yang. Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys, 56(4):1-33) 2023.
Jason Yim, Andrew Campbell, Andrew YK Foong, Michael Gastegger, José Jiménez-Luna, Sarah Lewis, Victor Garcia Satorras, Bastiaan S Veeling, Regina Barzilay, Tommi Jaakkola, et al. Fast protein backbone generation with se (3) flow matching. arXiv preprint arXiv:2310.05297, 2023.
Masahiro Yoshida, Kaylee B Worlock, Ni Huang, Rik GH Lindeboom, Colin R Butler, Natsuhiko Kumasaka, Cecilia Dominguez Conde. Lira Mamanova, Liam Bolt, Laura Richardson, et al. Local and systemic responses to sars-cov-2 infection in children and adults. Nature, 602(7896):321-377, 2022. ISSN 1476-4687. doi: https://doi.org 10
Luke Zappia, Belinda Phipson, and Alicia Oshlack. Splatter: simulation single-cell ma sequencing data. Genome biology, 18(1):174, 2017. ISSN
doi:
https://doi.or//10.1 1ttp:///000..1111111000000000110
Tao Zeng and Hao Dai. Single-cell rna sequencing-based computational analysis to describe disease heterogeneity. Frontiers in Genetics, 10, July 2019. ISSN 1664-8021. doi: 1..3389ffgene.2019. 0629.
Qinqing Zheng, Matt Le, Neta Shaul, Yaron Lipman. Aditya Grover, and Ricky TQ Chen. Guided flows for generative modeling and decision making. arXiv preprint arXiv:2311.1344/ 2023.
16

---

Published as a conference paper at ICLR 2025
A DATASETS AND CODE
We make the code for CFGen as well as the links to pre-processe datasets available at ht atspp: //gittti wttpb//cc//tthiiiiceeeeeee.ee CFen.
B
THEORETICAL SUPPLEMENT
B.1 POISSON-GAMMA AND NEGATIVE BINOMIAL DISTRIBUTION
possible parameterization of the negative binomial distribution is via a mean j and an inverse dispersion parameter 8, with the Probability Mass Function (PMF):
pa 1,,0) r(0 +3) (0)
( 0 +
0
where I is the gamma function. One can show that the negative binomial distribution is obtained continuous mixture of Poisson distributions with gamma-distributed rate. More formally, define a Poisson model x ~ Poisson(A) ()) with À 2 O. The parameter À represents both the mean and variance of the distribution. Since the mean and variance for modeling over -ie-dispereed counts (i.e., when the variance exceeds the mean). A way to overcome this limitation is to model the rate of the Poisson distribution random variable following gamma distribution:
equal, the Poisson model is unsuitable
Poisson (A)ssonn(,..
(10)
A Gamma
(11)
where 0 >> 0 is the shape parameter and à is the scale parameter. Marginalizing out À in the PMF of the Poisson distribution recovers the PMF of negative binomial with mean μr and inverse dispersion
Crucially, the variance of this negative binomial parameterization is + 4.- Since the additional _ variance term & is always non-negative, the variance always exceeds the mean for finite 0, makinn the negative binomial distribution well-suited for modeling over-dispersed count data.
B.2 FLOW MATCHING WITH GAUSSIAN PATIS
Flow Matching (Lipman et al., 2023) learns time-dependent vector field (z), where t E [0, 1], generating a probability path (z). Here, Pu (O, I) is the standard Gaussian prior, and p1 represents more complex target distribution. The marginal (z(2) is commonly formulated mixture:
(z) pr (2)22)))((2) dzi,
where q denotes the target data distribution. The marginal vector field that generates such a mixture of paths is given by:
uy (z) = u (z/zz) Pr. Pr (q(z)) dz
(z(z)
While the marginal vector field u, (z) intractable, the conditional vector field uz(z/zz) (z] has closed- form expression when given an observed data point 2 and a pre-defined choice of the probability path Pr (z) ). This path satisfies the boundary conditions Po ) = Po and p1 pa(-1e (-|z; = 8(2 21). where the Dirac delta measure 8(z 21) ensures that the distribution at time f = 1 is a point mass at the observed data point z₁. Notably, regression on ui (z(zz)) admits the same minimizer as on uz(z). making it suitable for use target during training.
Following Lipman et al. (2023), we assume Gaussian probability paths for the transformation each time t, defined by:
(z(2z)) = (2zz),z2, of1),
where de and o the scheduler parameters, with ag 0, 01 0, a1 1, and ou 1. In this work, we use standard linear scheduling, where a; = t and o; = - t, linearly interpolating between the initial and final values.
17

---

Published as a conference paper at ICLR 2025
B.3 THE RELATIONSHIP BETWEEN FLOW MATCHING AND CLASSIFIER FREE GUIDANCE (CFG)
Zheng al. (2023) draw relationship between CFG in score-based models (Ho & Salimans, 2021) and the Flow Matching vector field dt. Specifically, the authors show that the following relationship between the score V log Pe (zly) and the marginal vector field uz(zly) () holds:
u4 (Zly) a, + be V log pr(lly)
(12)
both in the conditional case and when = 2, with a, á and by = (áror aror) a.
Let the equation
W) (z) + Vv log be the CFG score formulated by Ho & Salimans (2021) with guidance strength Zheng et al. (2023) define the vector field of classifier-free Flow Matching as
lo
Ty (zly) = (1 w))((z) + w (z
(13)
This vector field is related to the CFG score by
(z(y)y) az + by V2 (o
which is derived by substituting Eq. (12) into Eq. (13).
B.4 PROOF OF PROPOSITION
Proposition 1 If the attributes V1. .i...., UK are conditionally independent given z, the vector field
(z)y)) (z)
Σ [ue(z]y)) uz(z)]
coincides with the velocity of the probability-flow ODE associated with the generative SDE of diffusion model with the compositional score as in Eq. (7).
Proof. (Proposition 1) We first justify the conditional independence assumption and subsequently prove the equality.
Conditional independence assumption. Given a variable and a of attributes y 5/, .... yK. we define the attribute-condttioned marginal probability path Pr (z|B//, (/ ... yk), with & E [0, 1]. Under the assumption that the attributes are conditionally independent given z, can rewrite the marginal path as follows:
K
(r(2))))
(e .i. yK)) yK) x (z, .1..... yx) (z P((slz))) (z()) II
(14)
pr(z)
Taking the logarithm and then the gradient with respect to on both sides in Eq. (14), we obtain:
V log Ps (zly1, vK) Vz (z)
V (ogp(()))) Vz logp((z))
(15)
In CFG, the goal is to sample with attribute serebbtt--ppcificc guidance strengths {wi according to modified conditional data distribution
K
((l|vi, ... uK) x a(z)
II
(16)
<(z)
1=1
In terms of generative probabilistic paths, the score in Eq. (15) becomes:
Vx log Pr (2ly),, yK) Vx log
Wi
(olp)(Zlvs) V. logp:(z))
(17)
In score-based models, the formulation in Eq. (17) is used to parameterize the drift of the reverse-time SDE that generates data points conditionally on the attributes y...... Vx with guidance strengths
{w, k
18

---

Published as a conference paper at ICLR 2025
Proof of equality. Following the standard theory of score- score-based models (Yang al., 2023) and their compositional extension (Liu et al., 2022), we first note that one can use the compositional CFG score
V log V (o)) +
Σ wi V2 log V2 (r))
(18)
to simulate the generative probability- ODE:
== = 2 V2 logi((
(19)
Given a scheduling pair (04,0r), one can show (Kingma et al., 2021) that the drift and diffusion coefficients of score-based diffusion model following the formulation from Song al. (2021) satisfy
ft dlog at dt
do? di
dlog az 07 dl
(20)
From these, we derive
b. (áro) a, ot ar
d, a₁,
(21)
Thus, rewriting Eq. (19), we obtain the probability-flow ODE in the form:
à a, + be V log (zly)
(22)
Next, using results from Zheng et al. (2023), as discussed in Appendix B.3, we write the unconditional and conditional vector fields as
ux(2) = az + be log (z(): (z(Z)y:) a, + by V, log pz
(23)
(24)
Plugging Eqs. (23) and (24) into the Eq. (8) for the compositional flow, we obtain
(z(z)y) = (z)
wi (ue(z)))) u (z))
(25)
K
= a, + bt V (z)
+
wi (V loggp(((z))) - V. (z))
(26)
= 022 + b v, log pe (zly)
(27)
This matches Eq. (22), completing the proof.
B.5 RELATIONSHIP WITH EXISTING SINGLE-CELL GENERATIVE MODELS
Although Ikelihood-basee models are standard in the single-cell literature, CFGen leverages novel factorization scheme, as depicted in Eq. (5). Below, we outline key differences between our approach and standard single-cell VAEs:
In VI (Lopez et al., 2018), the conditioning is applied during the decoding phase rather than at the prior level for the latent variable p(z). In contrast, CFGen conditions directly on the latent prior p(zly), allowing it to sample from multiple modes. This enables more flexible generation compared traditional approaches that rely solely on lecoder-based conditioning.
Consequently, the likelihood term p(x|2, 1) differs from that of most single-cell VAEs. In CFGen, this term is modeled using an unconditional decoder since the conditioning on y is already incorporated in the flow-based generation of z. Conversely, standard conditional VAEs must explicitly feed the label into both the encoder and decoder.
While some VAEs incorporate conditional prior (Xu et al., 2021), they are primarily designed for representation learning rather than generative modeling. These models often under-regularize the latent space to favor structure and reconstruction. In contrast, CFGen enforces a strong conditional flow-based prior on the latent cell representation, avoiding the trade-off between Kullback-Leibe divergence minimization and likelihood optimization. As a result, it can generate high-quality samples without requiring highly regularized latent space.
19

---

Published as a conference paper at ICLR 2025
Although previous works have explored defining distribution over library size, approach uniquely integrates it into the generative process through our specifie factorization in Eq. (5). Typically, the library size is used merely a scaling factor in likelihood optimization, applied to the post-softmax output of the decoder. CFGen instead employs it as conditioning variable for sampling from the flow-hased conditional prior, ensuring a formally sound integration. Specifically, we define the library size as a conditioning attribute and factorize our latent variable model to generate z from p(z yy l). To the best of our knowledge, this formulation has not been explored before.
B.6 CONDITIONING ON THE SIZE FACTOR
Single-cell VAEs, like Vc, offer the option to learn a log-normal distribution over the size factor, which can then be sampled. This process is similar to our approach, both methods fit log-normal distribution over the library size in the data. However, the key distinction lies in how the size factor interacts with the latent cellular state.
In Vc,I, the latent cellular state and the size factor are sampled independently. In contrast, CFGen provides the option to bias the sampling by the size factor. More specifically, CFGen samples a latent state that inherently accounts for cell size, whereas scVI does not.
If the size factor is relatively uniform within target population of cells, the approach in sc might be sufficient. However, if we generate conditioned on a coarse annotation (e.g., the source study of dataset in modern atlases), the library size can vary significantly within the annotation category (see Fig. A17 for examples using different studies in the Human Cell Atlas dataset). Sampling latent state and size factor independently in such cases may lead to inconsiseencies-Scae -caleniiisssslninngge a decoded cell state by incompatible size factor could produce unrealistic results.
Instead, conditioning on the size factor is more appropriate, as it biases the sampling of the latent state toward regions of the latent space where that specific size factor is naturally represented. This strategy can be combined with coarsely annotated variables to enable more targeted conditional generation.
C MODEL DETAILS
The CFGen model is implemented in PyTorch (Paszke et al., 2017), version 2.1.2.
C.1 THE CFGEN AUTOENCODER
Before training the flow model to generate noise from data, we first embed the data using an autoencoder trained with maximum likelihood optimization.
Encoder. The encoder is a multi-layer perceptron (MLP) with two hidden layers of dimensions [512, 256). The final layer maps the input to a latent space, whose dimensionality is dataset- dependent. In our experiments, we use 50-dimensional latent space for most datasets, except for the Human Lung Cell Atlas (HLCA) and Tabula Muris, where we set the latent space to 100 dimensions for increased representational capacity. In the multi-modal setting, different data modalities are embedded into shared latent space. Fach modality first processed by modality-specific MILP encoder. Due to the high dimensionality of DNA accessibility data (referred to ATAC data from Assay for ceaapossss-acccsssbeee Chromatin), its encoder uses hidden layers of dimensions [1024, 512]. The outputs of the RNA and ATAC encoders are then concatenated and passed through shared encoder layer, mapping to a final 100-dimensional latent representation.
Decoder.
The decoder maps the latent space to the parameter space of likelihood model. For multi-modal data, each modality has its dedicated decoder.
The latent representation is mapped to the mean parameter μ of negative binomial likelihood, with one dimension per gene. Following Lopez et al. (2018), we apply a softmax transformation across genes produce normalized probabilities. These probabil- ities are then scaled by the library size (total transcript count per cell). The inverse dispersion parameter of the negative binomial distribution is learned model parameter, implemented
scRNA-seq:
20

---

Published as a conference paper at ICLR 2025
via torch. nn. Parameter. We offer the option to model inverse dispersion per gene gene-attribute pair, depending on dataset properties.
ATAC-seq: The decoder maps the latent space to continuous logit values. These are passed through an elementwise sigmoic in d function to produce probabilities per genomic region. Unlike RNA data, no size factor scaling is required.
Additional training details. We train the encoder and decoder networks jointly via likelihood optimization. Therefore, the weights of the networks are optimized (along with the inverse dispersion parameter) to produce the parameters that maximize the likelihood of the data under a predefined noise model. For scRNA-seq. we employ a negative binomial distribution, while for ATAC-seq. Bernoulli likelihood. The losses from different modalities backpropagation. Note that seRNA-seq data are provided by the encoder in their Tog-transformed version for training stability. However, the loss evaluated on the original count data.
summed before applying
For all settings, we set the learning rate to 0.001, with all layer pairs interleaved with one-dimensional batch normalization layers. We use the AdamW optimizer and the ELU activation function as the non-linearity.
C.2 THE VELOCITY MODEL
The vector field architecture. The vector field model takes as input the latent representation computed by the encoder and produces vector field used to simulate paths that generate data from noise. The architecture follows a deep ResNet (He et al., 2016) whose output has the same number of features as the input. The velocity model consists of the following components:
A linear projection layer that maps the input dimension to the hidden dimension of the flow model.
Three stacked ResNet blocks are responsible for representation learning and conditioning.
An output layer with a single non-linearity, implemented using a SiLU activation function.
A time embedder that encodes time using sinusoidal multi-dimensional embeddings (Vaswani et al., 2017) with frequency value of 1e4. This embedder is an MLP with two layers and SiLJ non-linear....
A size factor embedder that applies sinusoidal embeddings (Vaswani et al., 2017) to guide generation towards a predefined number of transcripts. This component is used only when the size factor a conditioning variable, i.e., when we do not assume p(zly, () = p(z y). Before being passed through the sinusoidal embeddings, the log size factor is normalized to a range approximately between and I, using the maximum and minimum log size factors in the dataset.
* Covariate embeddings for all different conditioning attributes.
Additional technical details. During training, the covariate embeddings are summed elementwise with the time embedding and, if applicable, the size factor embedding. Thus, all embeddings are either designed to have the same dimensionality or are transformed to a common size. This summed representation is then passed single vector to the ResNet blocks.
Both the summed conditioning embedding and the down-projected input are provided to the ResNet blocks, which consist of:
A non-linear input transformation of the state embedding.
A linear encoder for the covariate embedding.
A non-linear output transformation.
A skip connection.
The outputs of the non-linear input transformation and the covariate encoder are summed and passed through the output transformation. The result is then added to the input of the ResNet via the skip connection, following the traditional residual block structure (He et al., 2016). All non- linear transformations implemented as simple [si LU, Linear] stacks.
21

---

Published as a conference paper at ICLR 2025
In the standard setting, we train the flow model for 1, 000 epochs using the Adaml optimizer, learning rate of 0.001, and a batch size 256.
C.3 COVARIATE EMBEDDINGS
Covariate embeddings are trainable torch. nn Emboding layers of pre-defined size. our experiments, we use size of 100 in most of the settings.
C.4 SAMPLING FROM NOISE
To generate discrete observations from noise, we first draw a covariate from the associated categorical distribution, with proportions estimated from the observed data. Next, we sample size factor from LogNormal distribution, where the mean and standard deviation are set as the Maximum Likelihood Estimates (MLE) from the entire dataset conditioned on technical effect covariate. We then sample Gaussian noise and simulate latent observation from the real dataset conditionally. This is done by integrating the vector field computed by the neural network in Appendix C.2, starting from Gaussian noise. The integration is performed using the dopri5 solver with ad adjoint sensitivity and tolerance of 1 from the torchdyn package (Poli et al., 2021) in Python3 (Van Rossum & Drake, 2009), over the [0, 1] time interval. The generated latent vector is then decoded into the parameter space of the noise model for the negatve binomial for scRNA-seq or Bernoulli for ATAC-seq-afee after which single cells are sampled.
C.5 SEPARATE TRAINING
In CFGen, we train the encoder f, separately from the flow model. Initially, when we attempted to model the autoencoder and the flow jointly, we found that training the flow was unstable. Specifically, Flow Matching performs better when the state space is fixed. Alternating between autoencoder and flow updates leads to continuous changes in the data representation, as the autoencoder evolves with the flow. This dynamic hinders accurate velocity field estimation, particularly during the early updates of the autoencoder. One could initially train the autoencoder with a higher learning rate than the flow, periodically decreasing the former and increasing the latter. However, this approach is essentially similar to training the autoencoder and flow separately, which is the strategy we ultimately adopt to avoid the need for repeatedly retraining the autoencoder.
C.6 SCHEDULING
We use linear scheduling, following the original formulation from Lipman et al. (2023).
D BASELINE DESCRIPTION
D.1 SCVI, MULTIVI, PEAKVI
SCVI (Lopez et al., 2018), MultiVI (Ashuach et al., 2023), and PeakVI (Ashuach et al., 2022) are all VAE-based generative models designed for single-cell discrete data. Following the standard VAE framework, these models learn a Gaussian latent space, which is then decoded into the parameters of discrete likelihood models that describe different single-cell modalities. While sc and Peak VI generate single modalities- Vc for scRNA-seq and PeakVI for ATAC data-Multiv learns shared latent space across modalities while maintaining separate discrete decoders for each data type.
D.2 SCANVI AND SCPOLI
In the batch correction experiment described in Section 5.5, we compare CFGen with two additional VAE-based models: scANVI (Xu et al., 2021) and scPoli (De Donno et al., 2023), scANVI extends scVI by incorporating a latent cell type classifier to preserve biological structure in the representation space and introducing conditional prior the latent space. sePoli differs from ScANVI by using continuous embeddings instead of one-hot encodings as conditioning inputs to the VAE. Additionally, it enforces biological coherence by aligning cellular representations with latent cell-type prototypes.
22

---

Published as a conference paper at ICLR 2025
In simple terms, scPoli encourages cells to cluster around the average embedding vector of their respective cell types.
D.3 SCGAN
The scGAN model (Marouf al., 2020) is Generative Adversarial Network (GAN) (Goodfellow et al., 2014) designed for realistic scRNA-seq data generation. It minimizes the Wasserstein distance between real and generated cell distributions, employing generator network to produce synthetic samples and a critic network to distinguish real from generated cells. The architecture includes fully connected layers and a custom library-size normalization (LSN) layer for stable training. The model extends to conditional scGAN (cscGAN) for type-specific cell generation. Evaluation relies on metrics such as t-SNE visualization and marker gene correlation to assess the quality of generated cells. While scGAN has been explored for conditional generation, we found that conditioning on cell type led to significantly worse results compared to an alternative approach which the model is conditioned on data-driven Leiden cluster labels rather than real cell-type labels. We refer to this alternative as the unconditional version, as does not use predefined labels but instead leverages cluster-derived attributes.
D.4 SCDIFFUSION
scDiffusion (Luo et al., 2024) is a generative model that leverages diffusion models to generate realistic single-cell gene expression data. The model consists of three main components:
An autoencoder, which maps gene expression profiles to a latent space, enabling compres- sion and feature extraction from high-dimensional single-cell data.
A diffusion backbone network, which learns to reverse diffusion process applied to the latent embeddings, progressively refining noisy representations into meaningful biological signals.
A conditional classifier, which guides the generative process by incorporating cell type or other biological attributes, ensuring controlled cell generation.
During training, the autoencoder first encodes real single-cell data into a latent representation. Noise then progressively added to these embeddings following a predefined diffusion schedule. The diffusion backbone network is trained to learn the reverse process, reconstructing clean embeddings from noisy ones. Simultaneously, the conditional classifier is optimized predict labels from these latent representations, reinforcing biological relevance in the learned distribution.
At inference, scDiffusion starts from a random noise vector in the latent space and iteratively removes noise using the trained diffusion backbone, ultimately generating clean embedding. This embedding then decoded by the autoencoder to reconstruct synthetic gene expression data. The process enables controlled single-cell generation by conditioning specific biological attributes.
D.5 DISCUSSION: KEY DIFFERENCES BETWEEN SCDIFFUSION AND CFGEN
Handling Single-Cell Data Properties. scDiffusion applies Gaussian diffusion to preprocessed single-cell data, disregarding key properties such sparsity, overdispersion, and discreteness. While normalization ensures data continuity, most single-cell methods preserve zeros and non- non-linear mean- variance trends. Since continuous decoders like scDiffusion require centered, dense inputs, their design is suboptimal for scRNA-seq data.
Conditional Sampling. scDiffusion relies classifier-basee guidance, making conditional gen- cration highly dependent on classifier accuracy. This limits its ability to generate rare cell types or handle attributes that are difficult to classify.
Efficiency and Sampling Speed. due
CFGen is two to three orders of magnitude faster than scDiffusion
Efficient Sampling: Flow Matching directly maps noise to data along nearly straight paths, requiring only 5-10 integration steps compared to the >1000 steps needed for scfiffusion.
23

---

Published as a conference paper at ICLR 2025
Table 4: List of datasets considered in this work with the associated number of genes, cells, and cell types.
Dataset name
Number of cells
Number genes
Number cell rypes
PMBC3K
2,638
8,573
8
Dentate gyrus
18,213
17,002
14
Tabula Muris
249,389
19,734
123
HLCA PBMCIOK
584,944
27,997
50
10,025
25,604
14
NeurIPS
90.261
14.087
45
PBMC COVID
422,220
2,000
29
C.Elegans
89,701
17,747
(plus unknown)
Lower Dimensionality: CFGen operates in a compact latent space (50-100 dimensions), whereas scDiffusion performs denoising in much higher-dimensional space (1000 dimen- sions).
Guidance approach: Unlike scDiffusion, which relies on a classifier's gradient, CFGen uses CFG-based guidance, avoiding performance bottlenecks due to classifier accuracy.
E DATA PREPROCESSING AND DESCRIPTION
Single-cell data were preprocessed using scanpy (Wolf et al., 2018). Count normalization was applied only baseline models requiring real-valued inputs. In these cases, gene counts were library-size normalized to 1e4 and log-troe bog-transforme.. Since CFGen, MultiVI, PeakVI, and seVI operate in discrete space, trained them filtered out genes expressed in fewer than 20 cells across all datasets.
counts without normalization. Additionally,
F EXPERIMENT DESCRIPTION AND EVALUATION METRICS
F.1 2- 2-WASSERSTEIN DISTANCE AND MMD
We use the 2- 2assssesttein distance and the RBF-kernel Mean Maximum Discrepancy (MMD) with scales (0.01, 0.1, 1, 10, 100} (Gretton al., 2012) to measure the overlap between real and generated the Python Optimal Transport (POT) (Flamary et al., 2021)
data. To implement the former, package. For the MMD, we resort to the implementation proposed in 3
F.2 DISTRIBUTION METRICS COMPARISONS
To compute the metrics in Table 1, we generate three datasets per model, each matching the size of the original. In the conditional setting, we compute distribution metrics per cell type, comparing subsets of real and generated data. In the unconditional setting, we sample batches of 5, 000 cells from the full distribution. mitigate the curse of dimensionality, we compute MMD and Wasserstein distances in 20-dimensional Principal Component (PC) space. Generated cells are projected using PC loadings from real cells to ensure comparability. Since scDiffusion and scGAN generate normalized data while CFGen and sc produce discrete counts, normalize CFGen and Vc outputs to total of 1e4 counts per cell and then apply boggtransformation. The same processing is applied to real data, ensuring all models operate comparable quantities. All metrics are reported on the test
F.3 VARIANCE-MEAN TREND AND SPARSITY HISTOGRAMS
After pre-processing (as described in Appendix F.2), we compute the mean and variance of gene expression across cells and the frequency of unexpressed genes per cell. The mean-variance trend in raw count data is expected to be quadratic (see Appendix 1.)). However, normalization and terttrasffrmaaioo---uggee
for comparison with scDiffusion and scGAN-alte this trend. Despite this, examining the empirical mean-variance relationship remains informative, as should align with real data behavior.
https::/ thub. wtw.bcoomat..ooooo com//btog////// w/tnnn/////oiii /ondi tional f1 Tow-matching
24

---

Published as a conference paper at ICLR 2025
F.4 MULTI-MODAL EVALUATION
We generate multi-modal data and perform an unconditional comparison with the ground truth, as described in Appendix F.2. For ATAC data, we normalize both real and generated cells using the TF-IDF algorithm from the MUON package (Bredikhin et al., 2022).
To generate Fig. A7b, we follow these steps:
1. Compute the average gene expression and peak accessibility (i.e., fraction of accessible regions) per cell type for marker genes/peaks, following 4. This yields cell_typa ma rker matrix for both real and generated datasets, where each entry represents the mean expression or accessibility of a marker a given cell type.
2. Correlate the row vectors of these matrices between real and generated datasets. high correlation indicates that the generative model accurately captures mean marker expression and accessibility per cell type.
F.5 GUIDANCE STRENGTH EXPERIMENTS
Fig. illustrates the qualitative performance of guidance in CFGen. First, we train CFGen on each dataset following Algorithm 1. Upon successful training, we sample 500 cells under different guidance strength combinations, varying one attribute while keeping the other fixed, as shown in the figure. For unconditional generation (i.e., guidance strength of 0 for both attributes), many cells as in the real dataset to better visualize the overlap between real and synthetic data. When applying guidance, we train the guided CFGen model with an unconditional sampling probability of Puncond 0.2 (see Algorithm 1).
generate as
F.6 SCGPT (CUI ET AL., 2024) GENERALIZATION PERFORMANCE ENIIANCEMENT
We split the PBMC COVID and HLCA datasets into a training set and held-out set, ensuring challenging generalization task by leaving out all cells from 20% of the donors in both datasets. This results in 80 training and 27 test donors for HLCA and 60 training and 15 test donors for PBMC COVID. After augmenting the training set (see Section 5.4). we use a pre-trained scGPT model to embed both the training and validation sets for both the original and augmented data. We then fit a k-Nearest-Neighee on the held-out set. The results, shown in Fig. 4, illustrate how the classification performance in terms of recall on a cell type varies as a function of its frequency in the dataset after augmentation. performance improvement suggests that the additional synthetic examples help the model better distinguish the cell type in the L.M representation space.
(kNN) classifier on the training embeddings and evaluate its performance
F.7 BATCH CORRECTION EVALUATION
Correction with CFGen. Batch correction aims to remove technical effects while preserving biological variation in single-cell data. Given an observation x associated with a batch label yhatch and biological annotation Yeoll type- first encode x into latent variable z.
Flow Matching a generative model that maps a prior distribution to the data distribution via an invertible flow. This invertibility allows data distributions to be transported to noise. Inverting the flow back to the prior removes batch effects as well as cell-type variability from z (Rombach et al., 2020).
To perform batch correction, we first apply flow inversion to remove both biological and technical variation from z. Then, we simulate the flow forward again, starting from the noise representation. Given a reference batch bbathh we generate new observations conditioned on this batch while preserving the original biological label Yeell type- When applied across the dataset, this procedure aligns all observations to the same batch, effectively removing batch-specific variations.
ref
'tttps: hateee///moootttttoe rww..heooeeccccsc..oooooooossss. pbme1 1 0k/3 305///Mulll -a0-0MMttmmooo
integrationn on. html
25

---

Published as a conference paper at ICLR 2025
Remarks. In the context of CFG, the weights Wbatch and type control the degree of biological preservation. Notably, our batch correction approach is conceptually similar to style transfer methods in diffusion models (Wang et al., 2023).
Evaluation Setup. We train CFGen and all competing models using identical cell type and batch covariates. For fair comparison, use a latent space of 50 dimensions for all models. For uncorrected data, batch mixing is evaluated in the PC space.
All VAE-based models were trained for 100 epochs with default settings, while scPoli was pre-trained for 40 steps, following De Donno et al. (2023).
Metrics, To assess batch correction and biological conservation, et al., 2021b). Specifically, we employ five metrics for batch correction and five for biological conser- vation. The scores reported in Table 3 correspond the average of these metrics, as computed by scIB. All scores are normalized between 0 and I, where 1 indicates perfect correction or conservation.
the sclB package (Luecken
All metrics rely on kNN graphs, using batch and cell-type labels evaluate technical and biological mixing. Below, we briefly describe each metric, though we refer to Luecken et al. (2021b) for further details.
1. Batch Correction Metrics:
Silhouette Batch Measures the Average Silhouette Width (ASW) between batch clusters.
iLISI Computes the Inverse Simpson's 's Index based neighborhood composition in
kNN graphs, indicating batch mixing quality.
KBET Evaluates whether the local batch composition in cell's kNN neighborhood matches the expected global batch distribution.
Graph Connectivity Assesses whether cells sharing the same label form fully connected subgraph in the kNN representation.
PCR (Principal Component Regression) Quantifies batch-associated variance before and after correction.
2. Biological Conservation Metries:
Isolated Labels - Identifies rare cell types appearing in the fewest number of batches
and assesses their separation from other cell identities.
K-means NMI Computes the Normalized Mutual Information (NMI) between k-
means clustering and batch clusters.
K-means ARI - Measures the Adjusted Rand Index (ARI) between k-means clustering and batch clusters.
Silhouette Label - Represents the Average Silhouette Width (ASW) between cell type clusters.
cLISI A cell-type-specift version of the iLISI score, evaluating biological structure preservation.
Selection of the guidance weights for batch correction. Appendix H.8 provides an intuition for the selection process. In batch correction, cells are transported to noise and back to data guided by biological and batch covariates. The guidance strength parameters whin and Wbatch determine the emphasis biological conservation and batch correction. Based on the scIB metrics only, one might select the highest guidance strengths, as these maximize aggregation within cell types and batches. However, as shown in Fig. A15 and Fig. A16, scIB metrics alone can be misleading and should be paired with qualitative evaluation. Excessive guidance collapses variability beyond the batch and biological annotations, leading to artefacts. Parameters near Wbio Wbatch E (1,2) generally balance signal preservation and correction effectively. For example, Fig. A demonstrates that excessive biological preservation causes unnatural clustering. The extent of the batch effect in the data should also guide parameter selection. For C. Elegans, with mild batch effects, Whio 2, whatch 1 performs better than Wbio = 1, Wbatch = 2. Conversely, for NeurIPS, Whio = Wbatch = 2 avoids artifacts observed for Wbio 1 (Fig. A16). In summary, recommend assessing the batch effect severity, sweeping over guidance weights, and selecting parameters that optimize seIB metrics without compromising realistic single-cell representations.
26

---

Published as a conference paper at ICLR 2025
G ALGORITHMS
Algorithm and Algorithm 2 depict our training strategies. what follows, for notational simplicity, we indicate (z( with Zg, where t [0,
Algorithm Train CFGen with multiple attributes on scRNA-seq
Require: Probability unconditional generation Puncond, trained encoder fn, scheduling (at, oz).
1: Initialize Ut
2: while not converged do
3: Sample (x, V/, .... yk from the data
4: 21 < (x)
5: Sample from 0, 1]
6: + Sum of entries of x
7:
Sample from Bernoulli(Punooee
8:
if b then
9:
y 5-0
10: else
11:
y/ 6- sample uniformly label among ...... KK
12:
end if
13:
Zo ~ (0, I) (sample noise)
14:
0,21 + o,2g (noisy data point)
15:
&, + á + 0,20
16: Take gradient step on Vellve,((,,,11 2e 112
17:
end while
Output: Ut.E
Algorithm 2 Sampling from multi-attribute guided CFGen for scRNA-seq
Require:
Trained velocity field Ut,e, conditions y1, VKK guidance parameters WT, wI,..., WK. size factor distribution parameters (ni σi), number of ODE steps Todee
trained decoder hy, trained inverse dispersion parameter O.
Sample size factor 1 from LogNormal(p4,.00
2: ~ (0, [sample noise)
3: n f- 1/nude (step size]
4: (-(-) - Ve,e(,, 0,1) 1) + Σ [vx,e(s Vi, () ve.e (:, 0, [)] (guided velocity function]
5: for
n.....
do
6: Zi-n < ODEStep( ū, z:) (ODE solver step)
7: end for
8: x +- Sample from (B (oftmax(
Output:
27

---

Published as a conference paper at ICLR 2025
H ADDITIONAL RESULTS
H.1 ANALYSIS THE RUNTIME
In Fig. AI, we empirically evaluate how different hyperparameters impact CFGen's runtime. To do so, we generate synthetic data using untrained CFGen instance initialized with specific configuration and run our experiments on an NVIDIA 100 GPU. Each hyperparameter assessed across different latent space sizes, the latent space dimension serves bottleneck in Flow Matching models and expected to have the greatest influence on generation speed.
We consider the following hyperparameters:
1. Number of generated genes (default: 20,000).
2. Number of generated cells (default: 50,000).
3. Latent space dimensionality of the denoising model's bottleneck (default: 128).
4. Number of neural network blocks the denoising model (default: 3).
5. Embedding size for conditional inputs (default: 128).
When evaluating the effect of specific hyperparameter, all others are fixed at their default values to ensure controlled comparison.
Time Number of Genes
Time vs. Number of Cells 15
Time (seconds)
Time (secords))
100
10000
20000
30000 40000
0.75
1.00
Number
Number of Cells
168
Time vs. Hidden dimensions
Time vs. Number of NN blocks
3.5
3.25
Time (seconda)
Time (seconds)
3.00
3.0
2.75
2.50
2.25
2.0
200
400
2.00
Hidden dimensions
Number of NN blocks
Time vs. Cond. embedding size
Time (seconés)
Number Latents
10
3.0
500
1000
2.0
0
200 400
Cond. embedding
Figure Al: Runtime analysis of the CFGen generation process. Each panel represents different hyperparameter configuration. Different lines each plot correspond to varying latent space dimensionalities. which directly impact generation time. We examine how runtime changes as a function of five key hyperparameters: (1) number of generated genes, (2) number of generated cells, (3) hidden dimension of the denoising model, (4) number of blocks in the denoising model, and (5) size of the condition embedding. Results are reported in seconds. When varying one hyperparameter, all others are fixed their default values.
28

---

Published as a conference paper at ICLR 2025
In Fig. A1, observe that the most influential hyperparameters affecting generation speed are the number of cells and genes, while factors related to the neural network size have a smaller impact. As expected, the number of latent codes significantly influences the sampling speed, as it determines the dimensionality of the generation space. Additionally, we compare the training and sampling runtimes of CFGen against competing models across different datasets (see Table and Table 6).
Table 5: Training runtime table. Each entry corresponds the time in seconds required to train model on different datasets. The number of cells and genes composing each dataset is reported at the bottom the table. CFGen and scDiffusion are broken down into their different components that should be considered additively for overview of the total runtime. For all the models, the batch size is set 128.
PBMC3K
Dentate gyrus
Tabula muris
HLCA
PBMCIOK (scKNA-seq
CFGen Flow CFGen scVI MultiV1
1.02
7.13
69.00
192.12
3.23
6.31
68.40
253.21
6,30
0.08
2.11
18.13
65.12
2.15
22.12
scDiffusion Denoiser
1,03
2.13
18.02
53.62
4.48
scDiffusion AE
0.98
7.14
165.6
329.02
7.39
seDiffusion classifier
0.01
0.71
9.66
26.32
0.39
scGAN
0.98
5.15
20.41
181.12
2.40
No. of cells
2,638
18,213
245,389
584,944
10,025
No. of genes
8,573
17,002
19,734
27,997
25,604
Table 6: Generation runtime table. Each entry corresponds to the time in seconds required for a model to generate many cells and genes as in the original dataset. The number of cells and genes composing each dataset is reported the bottom of the table.
PBMC3K
Dentate gyrus
Tabula muris
HLCA
РВМСTОK (scRNA-seq))
CFGen
0.34
0.26
3.68
8.62
0.43
seVI
0.01
0.02
1.26
3.63
0.03
MultiVI
0.03
seDiffusion
48.79
105.08
1255.41
2004.00
113.41
scGAN
0.70
0.94
4.15
12.39
0.68
No. of cells
2,638
18,213
245.389
584,944
10,025
No. of genes
8,573
17,002
19,734
27,997
25,604
From the sampling runtime results in Table 6, we observe that VAE-based models (scVI and MultiVI) are generally faster. However, it is important to note that these models are inherently less expressive and perform worse than CFGen, particularly on large datasets (see Table and Fig. A6). Notably, CFGen outperforms scDiffusion in speed, accelerating sampling by orders of magnitude. This improvement attributed to the following factors:
CFGen requires fewer simulation steps than scDiffusion (5-10 steps in CFGen vs. >1000 in scDiffusion) while achieving superior empirical and quantitative results.
CFGen operates in a lower-dimensional latent space (50-100 dimensions 1000 dimen- sions for scDiffusion, as recommended in the manuscript).
CFGen employs classifier frree guidance, whereas scDiffusion relies on classifier- guidance. Consequently, CFGen's 's performance is not affected by the gradient of a classifier's prediction at each step.
Strikingly, CFGen can generate comprehensive atlases with over 500,000 cells, such as HLCA, in just 8 seconds. For faimess, we acknowledge that the speedup depends on the batch size that can fit into memory during sampling (10k cells in our case).
29

---

Published as a conference paper at ICLR 2025
H.2 EXAMPLE OF SYNTHETIC GENERATION BY CFGEN
Generation results PBMC3K
Real
Unconditional
B-cells
CD4 cels
CDS cells
Monorytes
Deadrttcc cells
FCGR3A+ Monocytos
Megekaryocytes
NK colls
CD14+ Mono
CD4 cells
CD8 T cells
Generated
Real
Figure A2: Uni-modal generation of scRNA-sey A-scq by CFGen on the PBMC3K dataset. Real and generated cells are embedded together and visualized as 2D UMAP coordinates.
Generation results on Dentate gyrus
Real
Unconditional
CAI-Sub
Glallrng
Gramae
Immlstre
NH2
Rediaklia
Rediskiled
GlialProg
Granule
ImmAstro
Real
Figure A3: Uni-modal generation of scRNA-seq CFGen the Dentate gyrus dataset. Real and generated cells are embedded together visualized as 2D UMAP coordinates.
30

---

Published as a conference paper at ICLR 2025
Generation results on Tabula Muris
Real
Unconditional
Bladder
Deart Ancta
Kitweyy
Large Jnteeiin
Limn, Macch
Livee
Lzng
Mecaawry Stand
Spiees
Kidney
Large intestine
Skin
Generated
Real
Figure A4: Uni-modal generation of scRNA-seq by CFGen the Tabula Muris dataset. Real and generated cells are embedded together and visualized as 2D UMAP coordinates.
Generation results HLCA
Real
Unconditional
B-cells
positire
CDlc+ Myeloid dendritic
CD4+ Alpha-Beta T cell
Alveolar Macrophage
Generated
Real
Figure A5: Uni-modal generation of scRNA-seg by CFGen on the HLCA dataset, Real and generated cells are embedded together and visualized as 2D UMAP coordinates.
31

---

Published as a conference paper at ICLR 2025
H.3 COMPARISON BETWEEN CFGEN, SCVI, AND SCDIFFUSION ON THE HLCA AND TABULA MURIS DATASETS
CFGen
scVI
scDiffusion
HILA
Generated
Real
Tabula
muris
Figure A6: Qualitative comparison of the generation results of CFGen, VT, and scDiffusion the HI.CA and Tabula Muris datasets. Comparison is performed by evaluating the similarity the generated results to real cells. Real and generated cells for all models are embedded together and visualized as 2D UMAP coordinates.
H.4 ADDITIONAL RESULTS ON MULTI-MODAL GENERATION
PBMC1OK- - ovzslep between real and EFGeet-gencrteed cells
ATAC
Geverated
RNA
Real
Commlsse naspeeeeed
Figure A7: 2D UMAP overlap between real and generated cells across modalities the PBMCIOK dataset. (b) Pearson correlation between average cal-ttye-specifff marker peak accessibility and marker gene expression between real data and samples from CFGen and MultiVI.
32

---

Published as a conference paper at ICLR 2025
Real
proup PX1
CFGen
MultiVT
Figure A8: Average marker expression per cell type in real and generated data the PBMC10K dataset. x-axis
marker genes. y y-axis cell types.
Real
CFGen
MultiVI
Froched of call
Has reprecdion
Figure A9: Average number of cells with accessible marker peaks per cell type in real and generated data on the PBMC1OK dataset. x-axis marker peaks. y-axis cell types.
33

---

Published as a conference paper at ICLR 2025
H.S
ADDITIONAL RESULTS DATA AUGMENTATION
Augmented Better
Non-Augmented Betler
Equal Performance
CFGie
scDiffusion
scVI
PBMC COVID
Performance
0.5
0.3
0.0
10-3
10"1
10
10
10"a
10
10a
10'
10-2
10-1
10"
10'
0.25
0.20
Performance
0.15
HLCA
0.10
0.05
0.00
-0.05
-010
0.15
10-1
10a
10'
10-1
10"
10'
10-1
107
101
Relative Cell Type Frequency (%)
Figure A10: Extension of Fig. Comparison CFGen with scDiffusion and sc on boosting the scGPT classifier performance in terms of recall on rare cell types.
Together with scGPT, in Fig. All I we investigate if using CFGen to augment individual cell types improves the performance of linear classifier like Cell'Typist (Cippà Mueller, 2023). We obtain similar result as scGPT, with the recall performance on real cell types improving after augmentation (hence negative correlation between the performance improvement and the cell type frequency).
Augmented Better
Non-Augmented Better
Equal Performance
PBMC COVID
HLCA
2.5
0.0
0.4
0.25
Performance
0.3-
Performance
0.15
0.10
0.95
0.0
0.00
10
107
101
10
10"1
109
10'
Relative Cell Type Frequency (%)
Figure Performance improvement linear cell type classifier after augmentation. The plot displays the cell-type classification performance difference in terms of recall as a function of cell type frequency before and after augmentation on PBMC COVID and HLCA datasets. As classifier, we use CellTypist (Cippà & Mueller, 2023), which is based on logistic regression. The held-out set includes cells from 20% of donors for both datasets.
34

---

Published as a conference paper at ICLR 2025
For better appreciation of the classification improvement of single cell type categories by scGPT, we include Table and Table 8.
Table Table reporting the cell lype classification recall of scGPT before (Recall Base) and after (Recall Aug) augmentation the PBMC covid dataset. The Relative Frequency (%) column reports how rare certain cell type is in the datasct. For each row, we highlight which setting leads to the highest recall.
Cell Type
Relative Frequency (%)
Recall Base
Recall Aug
naive thymus-derived CD4-positive, alpha-beta...
25.18
0.87
0.90
classical monocyte
16.61
0.98
0,98
natural killer cell CD4-positive helper T cell
10.22
0.91
0,90
9.63
0.80
0.75
naive dhymue-derived CD8-positive, alpha-beta naive B cell
8.70
0.83
0.79
5.92
0.96
0.98
CD8-positive, alpha-beta cytotoxic cell
4.98
0.78
0.67
non-classical monocyte
3.73
0,95
0.96
central memory CD8- CD8-positive, alpha-beta T cell regulatory T cell
2.72
0.45
0.33
2.27
0.28
0.32
conventional dendritic cell
1,42
0.79
0.84
CD16-negative, CD56-bright natural killer cell gamma-delta T cell
1.21
0.66
0.70
1.19
0.49
0.63
effector memory CD8-positive alpha-heta T cell class switched memory B cell
1.17
0.21
0.27
1.04
0.51
0.71
cell
0.75
0.21
0.39
mucosal invariant T cell
0.73
0.69
0.70
CD4-positive, alpha-beta cytotoxic T cell effector memory CD8-posiire positive, alpha-beta T cell
0.53
0.06
0.13
0.48
0.10
0.23
plasmacytoid dendritic cell platelct
0.40
1.00
1.00
0,40
1.00
1.00
plasma cell
0.23
0.97
0,90
hematopoietic precursor cell
0,20
0.97
0.86
mature NK T cell
0.10
0.00
0.16
innate lymphoid cell
0.08
0.21
0.40
erythrocyte
0.08
1.00
1.00
dendritic cell
0.02
0.47
0.80
plasmablast
0.02
0.93
1.00
granulocyte
0.00
1.00
1.00
35

---

Published as a conference paper at ICLR 2025
Table 8: Table reporting the cell type classification recall of scGPT before (Recall Base) and after (Recall Aug) augmentation on the HLCA dataset. The Relative Frequency (%) column reports how rare a certain cell type the dataset. For each row, we highlight which setting leads to the highest recall.
Cell Type
Relative Frequency (%)
Recall Base
Recall Aug
alveolar macrophage
20.00
0.95
0.95
lype II pneumocyte
13.51
0.99
0.99
respiratory basal cell ciliated columnar cell of tracheobronchial tree
8.63
0.92
0,90
6.67
0.97
0.94
nasal mucosa goblet cell
5,43
0,88
0.89
CD8-positive, alpha-heta cell
4.93
0.89
0.87
club cell
4.57
0.62
0.53
elicited macrophage
3.77
0.70
0.70
CD4-positive, alpha-heta cell vein endothelial cell
3.43
0.59
0.64
3.09
0.93
0.94
capillary endothelial cell
2.77
0.92
0.85
alveolar type fibroblast
2.54
0.94
0.90
classical monocyte
2,43
0.87
0.87
CD Iepposiievee myeloid dendritic cell
1.95
0.73
0.64
pulmonary artery endothelial cell
1.83
0.68
0.75
lung macrophage
1.75
0.36
0.57
lype pneumocyte
1.69
0.94
0.95
non-classical monocyte
1.61
0.55
0.54
natural killer cell
1.51
0.81
0.83
epithelial cell
1.14
0.55
0.66
endothelial cell of lymphatic vessel
0.97
0.91
0.93
epithelial cell of lower respiratory tract mast cell B cell
0.94
0.86
0.88
0.65
0.96
0.96
0.65
0.88
0.90
plasma cell
0.53
0.98
0.98
alveolar type fibroblast cell bronchus fibroblast of lung respiratory hillock cell tracheobronchial smooth muscle cell epithelial cell of alveolus of lung bronchial goblet cell
0.49
0.82
0.79
0.39
0.67
0.77
0.39
0.78
0.82
0.33
0.78
0.29
0.18
0.55
0.23
0.04
0.11
plasmacytoid dendritic cell acinar cell lung pericyte
0.18
0.87
0.94
0.15
0.67
0.81
0.10
0.88
0.89
ionocyte
0.09
0.77
0.85
T cell
0.09
0.57
0.55
tracheobronehial serous cell myofibroblast cell conventional dendritic cell secreting cell dendritic cell mesothelial cell
0.05
0.43
0.64
0.04
0.25
0.69
0.04
0,58
0.81
0.03
0.61
0.61
0.02
0.46
0.70
0.02
0.91
1.00
smooth muscle cell lung neuroendocrine cell brush cell of tracheobronchial tree stromal cell fibroblast
0.02
0.09
0.15
0.02
0.97
0.97
0.01
0.21
0.37
0.01
0.35
0.88
0.01
0.30
0.60
hematopoietic stem cell tracheobronchial goblet cell
0.01
0.78
0.89
0.01
0.00
0.50
36

---

Published as a conference paper at ICLR 2025
H.6 MISSING GENE IMPUTATION WITH CFGEN
In the VI paper (Lopez et al., 2018), 10% of data entries are masked and set to zero, with the model trained on this corrupted data. During inference, masked cells are passed through the encoder, and latent codes decodes z to infer masked counts. Similarly, we propose an imputation strategy using CFGen as follows:
(-xx are sampled from the posterior. The VAF, trained to handle noisy inputs,
Train CFGen on noisy data.
Encode a noisy input x into the latent representation 21 (x).
Invert the generative flow to compute 3o = Φn(2)), mapping 2 to noise.
Sample around %i N(zo, IT)).
Transport zó back to z Φ1 (z)), then decode to impute gene values for x.
We tested this strategy on four datasets, masking 10% the counts. Fig. A12 shows that our predictions correlate with pre-masking data, and Table 9 demonstrates superior imputation accuracy compared to VCVI three out of four datasets (Pearson correlation, mean absolute distance).
Table 9: Mean distance and correlation between real and imputed genes by sc and CFGen.
Mean L, distance real-imputed counts (1)
Pearson correlation real- real-imputed counts (*)
PBMC3K
Dentate gyrus
IILCA
Muris
PBMC3K
Dentale gyrus
IILCA
Muris
CFGen scVT
1.21
0.42
3.21
4.81
0.68
0.56 0.58
0.83
0.86
1.47
0.35
4.43
6.08
0.61
0.75
0.79
Tabula Muris
PBMC3K
30000
250
15000
Ground truth
150
Groand
10000
100
5000
Pradictions
Predictious
HLCA
Dentate gyyrus
600
300
Ground truth
Ground
10000
S00
7500
5000
Xn
2900
100
5000
20000
13000
Predictions
Predictious
Figure A12: Scatterplot between imputed and real gene expression values before masking across datasets. Correlations can be found in Table 9.
In Fig. A13, we study how the quality the imputation by CFGen varies as a function of the amount of noise used to sample around 2n. Notably, higher noise level leads to worse imputation results, since the generative modeling aspect takes over and samples a completely new cell, which loses the structure of the originally encoded noisy observation. Specifically, Fig. A13 highlights that should
37

---

Published as a conference paper at ICLR 2025
remain below 0.1 to avoid sampling distant z values, which generate unrelated cells and disrupt correlations with original gene expressions.
Doniate gyrus imputetion ttd 0.01
Dontato serae imguatiinn end
Dentarie gymmu Imggutation
gyran ingutatiio
200
200
230
203
Groaae tochh
200
Dentate gyrus
100
100
100
60-
Prstuction
Preduction:
150
20.0 Prodictioa
333
Impetation
trapmtttion
impotattin
Imputation
0000
6000
4C00
6038
2000
1000
3300
Cooend cruth
Grosed textt.
HLCA
2000
2000
tron
3000
1000
Tredadions
2010
2000
areo
Pnwdtction
1020
Prstxtions
2000
300
200
500
PRMCAI str o1
Imputation
imputation
imputaaion
200
Gooasd
201-
Gounz tren
2001
PBMC3K
UG0
ESQ
150
200
300
60-
50-
1990 Predictions
150
2001
200
500
1000
1900
labula Imputattin
Tahhils Marts Impertation and a
Tabriaa Muris imputaatinn snd
Tabril Inpuutation
50.0
5000
Gouod crtt
6000
Groend Emaah
1000
4000-
Tabula Muris
2003
2000
2000
1000
1000
2000 4000
5000 Peddetions
1000
3000
Tradidiioss
Predictaeo
Figure A13: Correlation between the CFGen- CFGen-imputed and real gene expression before masking function of the variance the added noise. Rows represent different datasets, columns represent standard deviation of the noise used to sample around the latent representation of the cell. Perfect correlation along the bisector is the best possible imputation result.
38

---

Published as a conference paper at ICLR 2025
H.7 ADDITIONAL RESULTS MULTI-LABEL GENERATION
Table 10: Extension to Fig. 3. We train a 3-layer MLP with a softmax head on the real data to predict the classes of the two attributes considered for each dataset. For different levels of the combination of guidance weights, the classifier applied predict the average probability that the generated observations are assigned with high probability
of certain guidance
class. When guided on a single attribute, it expected that generated only to the class of such an attribute. As guidance strength increases for the counterpart attribute, CFGen models the intersections between attributes increasingly better and, therefore, enables high classification probability for both guiding labels.
NeurlPS
Tabula Muris p( (Tongue)
Weights
p MCDM4 + M.)
p[donor 1)
Weights
P(18-M-52)
Wdonor
0.98
0.40
Winouse
0.0
0.98
0.19
Weell type 1
Wiissuue 1
Wilooor
0.96
0.87
Wmouse
0.98
0.69
Weell type
Wtissue 1
Welonor
0.96
1.00
Wmosse ID
2.5
0.96
0.97
Weell type
1
Wiissuee
Wasserstein distance Tapula Muris
MMD Tabula Muris
2.5
0.04
2.0
Vilue
0.03
Value
1.5
0.02
1.0
Ainus Mrouss 0.1
Oiaua 0.5. Urous 0.5
1, Minure =-1
Whoae Mewocc 2-2
Mboue Miner 25
Whose Meano
1.5.
weights
Guidence weights
b
MMD NeuriPS
3.0
0.05
2.5
0.04
2.0
Value
1.5
0.02
1.0
0.01
Oletnee 0-5 Mloee Di
1.5
Mec tae More
Modioo Oeeee
Wothre Mene
1.S.
rap
Guldance weights
Guidarce weights
Figure A14: Performance the generation of the intersection of attributes the Tabula Muris (top) and
NeurIPS (bottom) datasets based distributional metrics. the x-axis, we increase the guidance parameters for both conditioning attributes.
39

---

Published as a conference paper at ICLR 2025
H.8 ADDITIONAL RESULTS MULTI-ATTRIBUTE GUIDANCE RESULTS
In batch correction, cells transported noise and then back to data, guided by both biological and target batch covariate. The guidance strength parameters who and Whatch control the emphasis on biological conservation and batch correction, respectively.
Table 11:
The average hatch correction and hio conservation metrics from selB package evaluated at different levels of guidance strength.
Elegans
NeurlPS
Guidance weights
Batch Correction
Bio Conservation
Batch Correction
Bio Conservation
Whio
Wbatch =0
0.48
0.55
0.32
Whio
Wbateh
=1
0.67
0.55
0.61
Whio
1 Whatch -2
0.68
0.54
0.63
0.61
Wbio
2 Whatch =1
0.68
0.63
0.64
0.73
Whie 2 Wbatch =2
0.69
0.63
0.64
0,71
Whio =2 Wbatch =3
0.69
0.64
0.65
0.70
Whio =3 Wbatch =2
0.70
0.67
0.65
0.77
Wbio 3 Whastch =3
0.70
0,68
0.66
0.75
Wbio 3 Whatch -4
0.70
0.67
0.66
0.73
Wbip Wbatch
=
0.70
0.69
0.67
0.77
If one relies on the scIB metric Table 11, computed for different guidance strength parameters. the highest possible guidance strengths may appear optimal, as they yield the best aggregation within cell types and batches. However, Fig. A15 and Fig. A16 demonstrate that selB metries can be misleading and should be complemented by qualitative evaluation. Excessive guidance in the translation task causes an unnatural collapse of variability beyond what is explained by batch and biological annotations.
Aibs
Whio
Whbtch
Wio
Wbatch
Who
2, Whbscch
Waie 4, Whblch
Harryy
Mam
Wateritinn 201 minutes
Woterston 00t misutes
minutes batch
Wotersoon minuto baic
O,
1,
Whacc
2, Whathh =
4,whaat
siul
2002
st83
sted
Figure A15: The PCA plot of generated cells colored by batch the C.Elegans (a) and NeurIPS (b) datasets.
Each column represents difffernnt combination of guidance strength values.
We found that guidance strength parameters in the range whins Whathh E {1,2) effectively preserve biological signal while performing batch correction without gver-squahhing cell representations. example of these unwanted effeets is illustrated in Fig. A16, where excessive biological preservation results in unnatural clustering for both datasets.
Moreover, the severity of batch effects in the data should guide parameter selection. In the C.Elegans dataset, where batch effects are mild, we select Whio = 2, Whachh = as they provide better performance than Weio 1, Wbatch 2 (Table 11). Conversely. in the NeurlPS dataset, observe the opposite effect and thus select Whin = 1, Whatch 2. As shown in Fig. A16, choosing whin > leads to an unnatural biological structure, violating smooth temporal single-cell trajectories.
40

---

Published as a conference paper at ICLR 2025
In conclusion, recommend first assessing the severity of batch effects in the dataset, and then systematically sweeping over combinations of guidance weights. The optimal configuration should maximize scIB metric values while maintaining realistic single-cell representations.
Biological over-preservation NeurIPS
Whbl
Mbbich
wbatch
Biological over-preservation C. Elegans
Whoo 2, Whatch
Whoc
4, Whathh
Figure A16: UMAP plot generated cells colored by batch the (a) NeurlPS and (b) C.Elegans datasets. We show one example of generation with a reasonable guidance scheme (left columns) and one with guidance scheme causing unrealistic cell type distributions (right).
H.9
LIBRARY SIZE INFLUENCE IN SINGLE- CELL ATLASES
Study
Study
Study
Size
factor
25300
15000
Study
Study 5
Study
10000
Figure A17: UMAP plots of six studies included in the HLCA dataset colored by size factor.
41


Reviews:
[{"review_id": "2OQwp4hEAw", "reviewer": "Reviewer_kEhi", "summary": "This paper proposes CFGen, which is a latent flow-matching generative model for single-cell data, where the latent space is first learned by an autoencoder. To capture statistical properties specific to single-cell data, the autoencoders learn to decode the parameters of a negative binomial distribution and Bernoulli distribution, for RNA-seq and ATAC-seq data, respectively. Conditional generation is achieved through classifier guidance. Empirical results demonstrate that CFGen outperform other single-cell generative models in terms of (1) data generation to approximate the real data distribution, (2) data generation for rare cell type classification, and (3) batch correction.", "questions": "- For batch correction, is CFGen's performance (in terms of the Batch and Bio scores) sensitive to varying the guidance parameters? How does one tune the guidance parameters in practice?\n- For cell type classification, simple models such as logistic regression (with or without regularization) are often used. Does data augmentation with CFGen improve performance for a logistic regression model?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Adapting flow matching for single-cell data generation is a novel contribution.\n- The proposed framework CFGen can be easily adapted for different uni- and multi-modal scenarios, as long as there are modality-specific autoencoders with a common latent space.", "weaknesses": "- scVI should be included as a baseline in Figure 2 because scVI accounts for overdispersion and zero inflation, whereas the current baselines in Figure 2 (scDiffusion and scGAN) do not.\n- For downstream applications that rely on conditional generation, it is unclear how the classifier guidance strength is determined.\n- Quantitative results are lacking when evaluating the compositional classifier guidance in Section 5.3. The change in MMD and WD with respect to the target distribution when increasing guidance strength can suffice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W2KGg3AjI7", "reviewer": "Reviewer_DU3L", "summary": "The paper presents conditional flow-based generative models for single-cell RNA-seq and accessibility data. Single cell data is generally sparse, noisy, and has high feature variance. The authors suggest a flow matching based approach as a more expressive, and consistent generative model compared to VAEs, and GANs for generating synthetic cells. They also present a compositional variant of classifier-free guidance for flow-based models to allow conditioning on various attributes. Finally, they evaluate the model on two downstream tasks: (1) generating synthetic samples of rare cell-types and using them for data-augmentation,  (2) leveraging CFGen for batch correction.", "questions": "See weaknesses above.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper addresses an important problem in single-cell data generation by generating raw count values, and further extending this to multimodal generation.\n2. The paper is well-written, and the authors convey major limitations of their model clearly.\n3. The results show that CFGen is able to capture characteristics of the training dataset and generate single cell data with similar statistical properties.\n4. They also show the effectiveness of generating rare cell-types to improve classification performance for other models.\n\nPost Rebuttal comments:\n\nThe authors have addressed my concerns regarding the presentation. They have also added the additional details I addressed in the weaknesses below. After going through their responses to other reviewers, I believe the paper will be a valuable addition to ICLR. I am raising my score to accept.", "weaknesses": "1. Fig 3. is not really clear to me. Firstly, I suggest adding contrasting colors for points representing generated and real data. Secondly, what are the red points representing? I also suggest perhaps adding a quantitative metric (perhaps a oracle model that predicts the attributes) as well.\n2. I also suggest removing the bars from Fig. 2b as they make it hard to observe the overlapping density curves which are easier to infer from.\n3. For Sec 5.2, it might be worthwhile to also add a comparison with CFGen just trained on RNA-data in order to measure the effects of using multimodal data for training.\n4. A comparison of inference times might also be useful in this case, especially to compare scDiffusion and CFGen, since both require multiple time steps. Adding approximate training times for each of the comparable models would also be valuable.\n5. Fig.4 should also report the raw accuracy numbers for each of the cell-types to evaluate the effect of CFGen,", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tcA11cW6a5", "reviewer": "Reviewer_eqfJ", "summary": "The authors of this paper present CFGen, a flow-based generative model designed for multi-modal single-cell data. CFGen addresses the challenges of generating discrete, multi-modal data while allowing conditional generation based on various biological attributes. The model extends the flow matching framework to handle compositional guidance across multiple attributes and provides promising results on tasks like data augmentation, rare cell type classification, and batch correction.", "questions": "- Can CFGen be applied to gene expression imputation tasks? If so, could the authors describe how the current framework could handle imputation, or if modifications would be needed?\n\n- Could the authors provide details about the computational complexity of the model?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The authors nicely demonstrate practical applications of their method such as data augmentation in rare cell types, improving downstream classification, and performing batch correction. \n\n- The idea to extend flow matching for generation with multiple attributes is interesting and important for single-cell data.\n\n- The paper is well-written, the related work is appropriately referenced, and the experimental setup is detailed.", "weaknesses": "-  The authors do not discuss the computational complexity of the proposed method. A more detailed breakdown of computational requirements, including training and sampling times for the proposed method and the baselines, would improve the paper.\n\n- One important task in single-cell data analysis is gene expression imputation, where missing or zero-inflated gene expression values are inferred to provide a more complete view of cellular states. It is unclear from the paper whether CFGen can effectively handle this task, given its focus on generating new cells rather than imputing missing data within existing cells. Could the authors clarify if CFGen’s architecture or the flow matching framework could be adapted for imputation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hMd24NiyyI", "reviewer": "Reviewer_FqvM", "summary": "In summary, I initially rate this paper as  \"5: marginally below the acceptance threshold\". But I'm open to increase my score if authors properly answer my doubts in the rebuttal.\n\nSummary of paper: The paper proposes a generative model for scRNA as well as accessibility modalities. The model can take in a combination of attributes, which suits the biological settings where for each cell only a subset of attributes are available. The method is evaluated in generation, handling label imbalance in cell type classification for rate cell types, and batch correction.", "questions": "Please see the \"Weaknesses\" part.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The model is tailored to real biological settings: it handles 2 modalities (scRNA and ATAC) and any number of attributes.\n- The results properly support the good performance of the method. \n- Besides generation power, two very interesting applications are demonstrated: handing rare cell types in cell type classification and batch correction.", "weaknesses": "- Handling discrete count data via negative binomial distribution is presented as a \"contribution\" of this paper. But there is a plethora of methods that make use of negative binomial (or alternatives like poisson distribution) to handle count data as well as over-dispersion. So why should it be listed as a contribution of this paper?\n- According to the paper, \"... the proposed factorisation is novel\". In the factorisation of Eq. 5 what is the rational behind conditioning the latent factor z on library size?\n- In proposition 1, the attributes $y_1$, $y_2$, ... are assumed to be conditionally independent given $z$, but with the factorisation of Eq. 5 the attributes are connected to $z$, hence $z$ forms a V-structure which according to d-separation causes the attributes to be dependant given $z$ ?\n- Regarding the proposed guidance scheme, the only difference to the normal classifier-free guidance is that only some attributes (and one attribute during training) is fed to the decoder. Is this approach equivalent to the normal classifier-free guidance with all attributes plus some attributes being randomly dropped out? Even if so, it wouldn't decrease the value of the proposed method.\n- In Table 1 scDiffusion is heavily outperformed by the proposed method, but one may say diffusion models may perform on par with flow matching (apart from training stability etc.). In the paper I'd recommend providing an explanation for the superior performance of the proposed method compared to scDiffusion.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)