SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Authors: Suyuan Zhao
YIZHEN LUO
Ganbo Yang
Yan Zhong
Hao Zhou
Zaiqing Nie
Venue: ICML 2025 poster (2025)
Keywords: spatial transcriptomics
multi-scale
foundation model
Primary area: applications->health_medicine
TL;DR: We propose SToFM, a multi-scale foundation model for spatial transcriptomics, which can effectively capture and integrate multi-scale information in spatial transcriptomics data, establishing a comprehensive understanding of biological tissues.

Abstract: Spatial Transcriptomics (ST) technologies provide biologists with rich insights into single-cell biology by preserving spatial context of cells.
Building foundational models for ST can significantly enhance the analysis of vast and complex data sources, unlocking new perspectives on the intricacies of biological tissues. 
However, modeling ST data is inherently challenging due to the need to extract multi-scale information from tissue slices containing vast numbers of cells. This process requires integrating macro-scale tissue morphology, micro-scale cellular microenvironment, and gene-scale gene expression profile.
To address this challenge, we propose **SToFM**, a multi-scale **S**patial **T**ranscript**o**mics **F**oundation **M**odel.
SToFM first performs multi-scale information extraction on each ST slice, to construct a set of ST sub-slices that aggregate macro-, micro- and gene-scale information. Then an SE(2) Transformer is used to obtain high-quality cell representations from the sub-slices.
Additionally, we construct **SToCorpus-88M**, the largest high-resolution spatial transcriptomics corpus for pretraining. 
SToFM achieves outstanding performance on a variety of downstream tasks, such as tissue region semantic segmentation and cell type annotation, demonstrating its comprehensive understanding of ST data through capturing and integrating multi-scale information.

In single-cell transcriptomics, single-cell RNA sequenc- ing (seRNA-seq) data reflects the gene expression levels in SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Suyuan Zhao 12
Yizhen Luo
123
Ganbo Yang² Yan Zhong 4
Hao Zhou'
Zaiqing Nie 13
Abstract
Spatial Transcriptomics (ST) technologies pro- vide biologists with rich insights into single-cell biology by preserving spatial context of cells. Building foundational models for ST can signifi- cantly enhance the analysis of vast and complex data sources, unlocking new perspectives on the intricacies of biological tissues. However, model- ing ST data is inherently challenging due to the need to extract multi-scale information from tis- sue slices containing vast numbers of cells. This process requires integrating macro-scale tissue morphology. micro-scale cellular microenviron- ment, and gene-scale gene expression profile. To address this challenge, we propose SToFM, multi-scale Spatial Transcriptomies Foundation Model. SToFM first performs multi-scale in- formation extraction on each ST slice, to con- struct a set of ST sub-slices that aggregate macro- micro- and gene-scale information. Then an SE(2) Transformer is used to obtain high-quality cell representations from the sub-slices. Addition- ally, construet SToCorpus-88M, the largest high-resolution spatial transcriptomics corpus for pretraining. SToFM achieves outstanding perfor- mance on a variety of downstream tasks, such as tissue region semantic segmentation and cell type annotation, demonstrating comprehensive understanding of ST data through capturing and integrating multi-scale information.
(a) Macro Scale: Tissue Morphology
Intercellular information flow
(b) Micro Scale: Cell-cell Interaction
Gone 3
Gane 4
Gano
(c) Gene Scale:
Expression Levels
Gene Expression Profile
Figure 1: ST data contain biological information from multiple scales. (a). An example of a visualized ST slice of mouse brain. (b), A microscopic sub-slice of the brain slice. (c). Gene expression profile a single cell in the brain slice.
cells, providing a digital representation of cells (Ziegenhain et al., 2017). Machine learning approaches have achieved great success in analyzing scRNA-seq data and enhancing the understanding of complex biological systems (Szalata et al., 2024). However, ScRNA-seq technology disassoci- ates cells from their tissuc contexts, and therefore overlooks tissue-level information such as tissue morphology and in- tercellular relationships. This limitation compromises the application scope of scRNA-seq data in real-world studies involving cells. (Williams et al., 2022; Du et al., 2023).
Fortunately, Spatial Transcriptomies (ST) technology has opened up new possibilities to measure gene expression in cells without losing spatial arrangements of tissues. Specifi- cally, ST data are obtained in the form of tissue slices and can be computationally described 2D point cloud con- taining tens of thousands of points. Each of these points represents a cell or sequencing spot' containing up to approximately 20,000 gene expression values. Such rich information in ST data offers biologists unique perspective on single-cell biology (Du et al., 2023; Marx, 2021). Effec-
1. Introduction
CCorresponding author 'Institute for Industry Research (AIR). Tsinghua University 'Department of Computer Science and Tecnology. Tsinghua University of Mathematical Sciences, Peking University. Correspondence to: Suyuan Zhao <sxdtzsy @ com,, Zaiqing Nie <za- iqing @ air.tssinguu...uu.n.>
' PPharMolix Inc. "School
For simplicity, we refer to both cells and sequencing spots as "cells" in the remaining paper since we only use high-resolution ST data close single-cell resolution. See Appendix D. for further explanations.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
tively analyzing ST data can provide cell-level insights for various tissue structure-related studies, such embryology (Chen et al., 2022), neuroscience (Shi et al., 2023; Hasel et al., 2021), tumor microenvironmenn analysis (Wu et al., 2021; Jin et al., 2024), and disease studies related to specific organs (Wu et al., 2024; Franzén et al., 2024).
expression profiles from ST data. Secondly, to capture micro-scale information, divide ST slice into multi- ple ST sub-slices based on the cell coordinates, and use an SE(2) Transformer (Zhou et al., 2023) to capture spa- tial information and cell-cell interactions. This division not only emphasizes the spatially localized cell-cell interactions (Zormpas et al., 2023) but also improves the computational efficiency in fully harvesting ST data. Finally, to maintain macro-scale information within each sub-slice, we draw in- sights from multi-scale approaches in CV (Cao et al., 2019; Liu et al., 2021) and GNNs (Ying et al., 2018; Xu et al., 2018) and represent macro-scale information as few virtual cells through clustering, and inject them into the sub-slices. We design two pretraining tasks: masked cell modeling and pairwise distance recovery, to capture both gene expressions and spatial characteristics of ST data.
The large amount of ST data generated in recent years pose growing demand for analysis (Atta & Fan, 2021), which has promoted the application of machine learning meth- ods in this field. Early attempts design specific approaches for various ST downstream tasks, such as deconvolution, imputation and clustering (Biancalani et al., 2021; Abde- laal et al., 2020; Dong & Zhang, 2022). More recently, inspired by the success of pretrained models in single-cell transcriptomies (Yang et al., 2022; Cui et al., 2023; Hao et al., 2023; Theodoris et al., 2023), some research efforts have attempted to develop ST foundation models. For ex- ample, Nicheformer pretrains a Transformer-base model on single-cell gene expression profiles from both ST data and scRNA-seq data for transferability (Schaar et al., 2024). CelIPIM further integrates spatial information by encoding cell coordinates with sinusoidal position embeddings (Wen et al., 2023). These models demonstrate superior perfor- mance and transferability across series of downstream tasks.
To train SToFM, we construct SToCorpus-88M, the largest high-resolution ST pretraining corpus to date. This corpus includes approximately 2,000 high-resolution ST slices ob- tained by 6 different ST technologies, totaling 88 million cells. It surpasses the current largest ST corpus (Schaar et al., 2024) by 1.5 fold in scale and fold in ST technology diversity. SToCorpus-88M will be publicly released.
In order to validate the effectiveness of SToFM in integrat- ing multi-scale information from ST data, we establish a comprehensive benchmark containing several significant biological tasks. Specifically, SToFM surpasses the best baseline by 11.6% and 14.34% on F1 score in the cross- slice tissue region semantic segmentation task (Chan et al., 2019; He et al., 2024) and the cell type annotation task (Yang et al., 2022; Domínguez Conde et al., 2022), respectively. We further justify the informativeness and transferability of SToFM representations through additional qualitative and quantitative results. We attribute this to the integra- tion of multi-scale information, since tissue morphology, cell-cell interaction patterns and gene expression seman- tics all contribute to obtaining consistent and transferable representations across different ST slices.
Despite the promising advancements, we argue that one of the most important characteristics of ST data has not been well captured by existing works. As illustrated in Fig. 1, ST data contain biological information from multiple scales. From macro scale (Fig. la), we can extract tissue mor- phology and organ structure information such as functional zones and anatomical layers. From a micro scale (Fig. 1b), we can capture cellular contexts and cell-cell interactions by analyzing intercellular relationships with spatially adja- cent cells. From a gene scale (Fig. Ic), we can delve into detailed information for each cell by analyzing gene expres- sion profiles Analyzing ST data requires a comprehensive understanding of biological information across all the dif- ferent scales. For example, in order to comprehend brain structure and function in neuroscience, macro-scale informa- tion like brain anatomical regions, micro-scale information like neuron connections, and gene-scale information like spatially variable genes are all indispensable (Jung & Kim, 2023). This presents a unique challenge for establishing an ST foundation model: capturing and integrating multi-scale scale information from a large number of ST slices with appropri- ate model architecture and self-supervised objectives.
Our main contributions are summarized as follows:
(1) We propose SToFM, a multi-scall foundation model that captures and integrates information from macro, micro and gene scale of spatial transcriptomics.
(2) We construct SToCorpus-88M, the first large-scale spatial transcriptomics corpus suitable for pretraining. SToCorpus-88M will be publicly available.
(3) We demonstrate the outstanding performance of SToFM on various downstream tasks involving ST data.
To address this challenge, we propose SToFM, a multi-scale ST foundation model, which is shown in Fig. 2. Firstly, to capture gene-scale information, SToFM performs do- main adaptation an off-the-shelf pretrained cell encoder (Theodoris et al., 2023) by continual pretraining on gene
2

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomies
Clustering
Sub-slice 5
fSt
Virtual Cells
VC1
VC3
VC2
Macro Scale
Micro Scale
cell
Vc3
Celt
Celz
oder
Call
Mask
SE(2) Transformer
MCM
Cal2
CalE
Gene Expression gw)
Cell Embeddings F
Cell Representations Yat
Call
Slice
Genoe
Gene2
Gane'
f Fell
Gene Scale
VC2
Add
Gaussian
Encoder
Noise
Cell
C412
F
EpdR
Coll
Spatial Distance Matrix D
Pairwise Representations
n pair
Domain Adaptation
(a) Multi-scale Information Extraction
(b) Multi-scale ST Representation Learning
Figure 2: An overview of SToFM architecture. (a). Multi-scale processing integrates information on macro, micro, and gene scales into sub-slices. At gene scale, we extract the representations of each cell using a cell encoder that has undergone domain adaptation. At micro scale, we divide the ST slice into several sub-slices. At macro scale, we cluster cells based spatial and gene expression data, construct virtual cells for each cluster, and add the virtual cells into each sub-slice. Notably, The actual sub-slice contains much larger number cells ("1000) and virtual cells ("50) than are shown in the simplified figure. (b). During the pretraining process, first, the domain adapted dodain-adapted cell encoder is used to re-calculate cell embeddings, cnabling the cell encoder to be trained through backpropagation. Next, an SE(2) Transformer is employed to jointly model the randomly masked cell embeddings and the noise- augmented distance matrix. The output representations are then used to reconstruct the original cell embeddings and spatial distances. These two pretraining tasks are referred masked cell modeling (MCM) and pairwise distance recovery (PDR).
2. Related Work
scale pretraining. For example, Tangram (Biancalani et al., 2021) effectively performs deconvolution low-resolution ST data using reference scRNA-seq datasets. STAGATE (Dong & Zhang, 2022) excels in downstream tasks like tis- sue structure segmentation and spot deconvolution. SpaGE (Abdelaal et al., 2020) integrates ST and scRNA-seq datasets to impute the gene expression profiles of ST data. More recently, developing ST foundation models with large-scale unsupervised ST data has attracted rising research interest. Nicheformer (Schaar et al., 2024) is a Transformer based model pretrained on single-cell from ST data to mitigate the distributional gap introduced by spatial sequencing technol- ogy. It only utilizes gene expressions ST data and ignores the spatial coordinates. To our best knowledge, CelIPLM (Wen et al., 2023) is the only prior work that integrates spatial information with transeriptomies in pretraining by combining gene expression embeddings and position embed- dings as the Transformer's 's input embeddings. As nascent attempt, CelIPLM has preliminarily validated the feasibil- ity of establishing an ST foundation model. In contrast, SToFM captures fine- une-grained gene expression embeddings at gene scale and also proposes more advanced approaches for incorporating information at micro and macro scales.
Single-cell Foundation Models The development of scRNA-seq technology has led to an exponential growth of the data (Svensson al., 2018), which provides the basis for single-cell foundation models. Innovated by pretrained models in the NLP domain, most existing single-cell foun- dation models such as ScBERT and Geneformer leverage Transformers architecture and language modeling objectives like masked language modeling to capture the inherent gene expression patterns (Yang et al., 2022; Gong et al., 2023: Theodoris et al., 2023; Zhao et al., 2023; Cui et al., 2023; Hao et al., 2023). Other studies further integrate single-cell data with natural language for mudti-modal pretraining to expand biological insights (Zhao et al., 2024; Levine et al., 2023; Choi et al., 2024; Yuan et al., 2024). Benefiting from large-scale pretraining, these single-cell foundation models have demonstrated effectiveness on variety of downstream tasks in single-cell biology (Wenteler al., 2024; Liu al., 2023) and provided novel insights to biologists (Bordukova et al., 2024; Yang et al., 2024). We refer readers to (Szalata et al., 2024) for a more comprehensive review.
Deep Learning Methods for Spatial Transcriptomies Early machine learning-base based methods in ST data analy- sis primarily focus on specific downstream tasks, employing architectures with small parameter sizes and without large-
3

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
3. Methods
Algorithm Micro- and Macro-scale Integration
I: Input: E = (F F = [Fi) P = (P:)
In this section, we describe the implementation and work- flow of SToFM, which is illustrated in Fig. 2. First, we perform multi-scale information extraction to each ST slice, concentrating macro-, miero- and gene- scale information within sub-slices, which are feasible inputs for a neural network to handle, as described in Sec. 3.1. Then, for each sub-slice, we use an SE(2) Transformer to combine transeriptomics and spatial information for representation learning, as described in Sec. 3.2. Finally, in Sec. 3.3, we describe the pretraining objectives and strategies in detail.
a E [0, 1]: F. t a Normaliz((CA:((F) + (1 a)
Normalize(P)
3: Clusters 4- Clustering(A))
4: Fvc. Pvc -0
5: for C in Clusters do
6: FVC +- FVC F
U Fj/[C[]] JEC
pvc + pvcpvuus UI P//IC|
JEC
8: end for
3.1. Multi-scale Information Extraction
SubSlices <- Split(P)
10: S D
We represent an ST slice containing N cells as: So (E, P), where E' {E, E RT Ji=1 is the n-dimensional gene expression matrix, and P= (P. R | represents
11: for T in SubSlices do
12:
E(9) -- {Fíli E T)
13:
F(O) + {Fjli E T FVC
the 2D positional coordinates. As shown in Fig. 2a, in this process, we perform multi-scale information extraction on each slice to obtain a set of sub-slices that aggregate multi- scale information, defined as $ = {. S will be used as model input for the next process.
14:
p(t + &Pili pvc
15: S+SU(Si)
16:
end for
17: Return S
Gene Scale Domain Adaptation Given the high dimen- sionality and sparsity of gene expression profiles, we need to obtain their dimensionality reduction embeddings. Due to technical limitations, ST data often exhibit deficiencies such as limited gene coverage and high rates of "dropout zeros" (Avşar& & Pir, 2023; Li et al., 2023). Consequently, compared to scRNA-seq data, the quality of ST data is con- siderably lower. Incrementally training scRNA-seg founda- tion models on ST data facilitates to transfer the knowledge learned from scRNA-seq data to ST data with different dis- tributioon, achieving higher-quality cell embeddings. We initialize our cell encoder with Geneformer, one of the most advanced Transformer-based single-cell foundation mod- els, which achieves high-quality representation by encoding cells as gene sequences arranged in descending order of relative expressions (Theodoris et al., 2023). Please refer to Appendix A.1 for more details.
mately 1,000). This division empirically achieves a trade-off between maintaining computational efficiency and preserv- ing sufficient localized intercellular interactions. Then, to maintain macro-scale information, we use Leiden algorithm (Traag al., 2019) to cluster all cells in S based on both cell embeddings F and cell positions P. We aggregate each cluster into a virtual cell, whose embedding and position coordinates are averaged over all cells in the cluster. These virtual cells retain the main morphology and partition of the slice, serving as a compression of macro-scale information. We incorporate the virtual cells into each sub-slice, allowing the model to learn hicro-scale information while maintain- ing an ability to perceive macro-scale structural organization. As shown in Algorithm 1, we split S = (E, P) into set of sub-slices = ( = (E()) F(), p())} with virtual cells. The sub-slices encapsulate micro-scale information while also integrating macro- and gene-scale information through virtual cells and cell embeddings. respectively, No- tably, only calculate the embeddings FVC instead of the gene expressions for virtual cells, because the average gene expression profile of multiple cells is out of distribution and cannot be effectively encoded by the cell encoder.
We incrementally train our cell encoder on the cells from ST data to achieve domain adaptation. The domain domain-adapted adapted cell encoder feell is then used to process the gene expression profiles E of the ST data, resulting in h--dimensional features F = {F. e " where F; fea((E)) We add F as additional information of the cells to So. and obtain S (E, F.
Ji=1
3.2. Multi-scale ST Representation Learning
Micro- and Macro-scale Integration We design multi- scale approach to effectively integrate information from micro scale and macro scale. We divide each slice into multiple sub-slices based on spatial locations P. with each sub-slice containing a manageable number of cells (approxi-
The multi-scale ST representation learning process is shown in Fig. 2b. Given a sub-slice 50) obtained from Sec. 3.1, we obtain cell representations that incorporate transcriptomics and multi-scale spatial information.
4

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Second-time Forward with the Cell Encoder We per- form a second-time forward with the cell encoder, that the parameters of the cell encoder can be updated through gra- dient backpropagation. This step is only performed during training, and F()) can be used directly during inference.
randomly masked genes in the input gene sequence arranged by relative expression levels. In addition, inspired by Zhao et al., we introduce a self-supervised contrastive learning task (Gao et al., 2021) to enhance the representation quality of the cell encoder. See Appendix A. for details.
Representation Learning by SE(2) Transformer SToFM aims to jointly encode the cell embeddings F()) and the cell positions P(1, to obtain spatial-aware representations and capture intercellular interactions. In addition, the output representations should be invariant to 2D translations and rotations to P()), which is known as SE(2)-invariance. For this purpose, we use a widely- adopted SE(2) Transformer architecture" (Zhou et al., 2023). This simple and efficient architecture has shown excellent performance in representing proteins (Zheng et al., 2024) and small molecules (Zhou et al., 2023).
Multi-scale ST Representation Learning Objectives In order to better capture multi-scale information from ST data, we need to design objectives that harvest supervision signals from both gene expressions and cell coordinates. To this end, we design the following two tasks, masked cell modeling (MCM) and pairwise distance recovery (PDR):
In Masked Cell Modeling (MCM), we randomly mask 10% of the gene expression embeddings F(i) in the sub-slice, and use SToFM's output representations y-1 to predict the masked embeddings through a regression head. Given the impracticality of using micro-scale information to recon- struct macro-scale information, we do not mask virtual cell embeddings. We use the Mean Squared Error (MSE) loss function. The objective is defined as:
Specifically, a distance matrix (+) is used as input for lo- cation information, where D = [|p)() p( ll2. The distance matrix is first passed through Gaussian module (Shuaibi et al., 2021) to obtain the initial pairwise represen- tations. At each Transformer layer, the pair representations are updated by adding the attention matrix calculated from to it. Then, the updated pair representa- tions are used as the actual attention scores for updating cell representations. This approach of using pair representations as attention bias has been shown to be effective in various important works such Graphformer (Ying et al., 2021) and Alphafold series (Jumper et al., 2021; Abramson et al., 2024). Moreover, given the strong locality of cell-cell inter- actions (Zormpas et al., 2023), we envision that the distance information is suitable input for the pair representation in handling ST data. More details about the SE(2) Transformer architecture can be found in Appendix A.2.
LMCM = [M
1
Σ JEM,
(11600 - -r5")112
cell representations
Where Mi is the set of masked cells, ( is the j-th cell embedding, and F(() is the restored j-th cell embedding predicted with the j-th cell representation from y(i)
In Pairwise Distance Recovery (PDR), we randomly se- lect 10% cells in the sub-slice and add Gaussian noise to their 2D coordinates p(O), which modifies the corresponding rows and columns of the distance matrix D Our model then attempts to reconstruct the unperturbed distance matrix (i)
through a regression head using the pair representation pair' For the same cell, we make sure not to mask its embedding and perturb its coordinates at the same time. We use the MSE loss function. The objective is defined as:
We train the SE(2) Transformer fsr to encode each sub- slice ( obtaining cell representations y-t) and pairwise representations (i), follows:
y-O y(i) cell puir
= JST (Fii) p(i)
LPDR
1
(1D92- - -Djilla) (GA)EMM
Σ
MMall
In pretraining, y-() and y-()) pair are used to calculate the loss function. In downstream applications, we perform sub-slice division once and calculate the cell embeddings for each cell from the corresponding sub-slice.
Where My is the set of perturbed elements in the distance matrix, (.0 is the distance between cells j and k calculated from the unperturbed positions, and (:(0) the restored distance between cells j and k; predicted by regression
3.3. Pretraining Objectives and Strategies
head using the pair representations (()
Domain Adaptation Objectives During domain adap- tation, we adopt the masked gene modeling objective in Geneformer (Theodoris et al., 2023), which is to predict the
pair
Pretraining Strategies We adopt some strategies to help stabilize the training and accelerate convergence. In the early stage of pretraining, since the domain-adapted cell encoder has undergone pretraining while the SH(2)-
This architecture is originally an SE(3) Transformer, but it can he applied to 2D scenes with little architectural modifications.
5

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Transformer is trained from scratch, we perform synchro- nization by freezing the cell encoder and directly using F() as the inputs of the SE(2) Transformer. Once the training of the SE(2) Transformer has mostly converged, we tune the pa- rameters of both the cell encoder and the SE(()) Transformer. Considering the high computational cost of the cell encoder, we perform the second-tine cell encoding mentioned in Sec. 3.2 only on some randomly sampled cells within the sub-slice. This allows us update both the cell encoder and the SE(2) Transformer simultaneously via backpropagation, improving the model's 's learning capability.
formation in ST data, CelIPLM (Wen et al., 2023). These models have demonstrated superior performance over tra- ditional methods across various tasks. Since these models differ in parameters and architecture, their parameere fine-tuning performance could be heavily influenced by hy- perparameter settings and fine-tuning strategies. To ensure a fair comparison of representation quality, we perform head tuning on downstream tasks by training task-specific heads top of the pretrained model outputs (Wei et al., 2021). As supplement, we also conduct full-parameter fine-tuning experiments in Appendix B.1.
4. Experiments
4.2. Tissue Region Semantic Segmentation
Similar to image point cloud semantic segmentation, tis- sue region semantic segmentation is one of the key tasks in spatial transcriptomics (Zhou et al., 2024), aiming to determine which structural or functional region each cell belongs to. Accurate region segmentation not only helps reveal spatial transcriptional patterns within tissues but also provides foundation for understanding the functional spe- cialization of cells in biological processes. We evaluate SToFM's performance on tissue region semantic segmenta- tion in two biologically significant experimental scenarios: human embryonic structure segmentation and DLPFC layer segmentation. For each task, we collect dataset consisting of four slices and evaluate the model performance on each slice separately. Additionally, to assess the model's trans- ferability, we employ more challenging cross-slice setting, where the classification head is trained on three slices and tested on the remaining slice.
4.1. Experiment settings
Construction of SToCorpus-88M We organize publicly available ST data from multiple sources (Xu et al.. 2024; Yuan et al., 2023; Biology et al., 2023; 10xGENOMICS, 2025; Vizgen, 2025; nanoString, 2025; SeekGene, 2025). As some low-resolution ST data cannot accurately reflect gene expression at the gene level, we only select sequenc- ing methods with single-cell resolution or those whose spot diameters are close to cell diameters. Specifically, we se- lect six sequencing methods and construct SToCorpus-88M, the largest ST pretraining corpus, which contains approxi- mately 2,000 tissue slices and 88 million cells, surpassing the largest ST corpus to date by 1.6 fold (Schaar et al., 2024). SToCorpus-88N) includes slices from human and mouse, and gene expression profiles are aligned by homologous genes (Harrison et al., 2024). We standardize the corpus using gene names and Ensembl IDs, and filter out cells with less than 100 expressed genes ensure data quality. Please refer to Appendix D.2 for more details.
Human Embryonic Structure Segmentation Human embryonic development is a highly dynamic and complex process. Accurately segmenting human embryonic struc- tures aids in understanding cellular differentiation and tissue formation during development, and also uncovers the ori- gins and regulatory mechanisms of key organs and tissues. In this experiment, use human embryonic Stereo-seq ST data to perform structure segmentation (Pan et al., 2023; HESTA, 2025).
Pretraining Setup Pretraining is performed on SToCorpus-88M after removing the data used for down- stream tasks. We perform domain adaptation for one epoch. Then, we perform multi-scale ST representation learning for three epochs, where the cell encoder is frozen in the first two epochs, following the strategy in Sec. 3.3. The pretraining is performed with 4 NVIDIA Tesla A A100 GPUs and takes approximately 20 days. Detailed pretraining hyperparameters are provided in Appendix C.I.
DLPFC Layer Segmentation The dorsolateral prefrontal cortex (DLPFC), which is a key part of the brain's 's prefrontal cortex, plays a crucial role in various higher-order cognitive functions such as working memory and attention regulation.
Baseline Selection and Downstream Task Setup For baseline comparisons, we select pretrained models that only process single-cell transcriptomics data or ST data, without incorporating other modalities like text (Zhao et al., 2024; Levine et al., 2023) or image (Lin et al., 2024). The cho- sen baselines are single-cell models Geneformer (Theodoris et al., 2023) and scGPT (Cui et al., 2023), the ST-specialized single-cell model Nicheformer (Schaar et al., 2024), and the first model combining gene expression and spatial in-
The DLPFC is part of the brain's 's neocortex and has typical six- six-layer laminar structure. In this experiment, we use hu- man DLPFC 10x Visium ST dataset (Maynard et al., 2021), which has been manually annotated with layer information in previous studies. It is worth noting that 10x Visium data is not used during the pretraining process of SToFM or any of the baseline models. Therefore, this experiment also evaluates how well the models can adapt to data from a
6

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Table 1: Performance of tissue region semantic segmentation on human embryo and DLPFC slices. Expr: gene expressions. Pos: cell positions. M-S: Multi-scale. Acc: accuracy. F1: macro Fi score.
Embryonic Structure Segmentation
ST Pretraining
Embryo1
Embryo2
Embryo3
Embryo4
Average
Cross-slice
Model
Expr
Pos
M-S
Ace
Fi
Acc
Fi
Acc
Fi
Acc
F.
Acc
Fi
Acc
Fi
scGPT
x
x
X
0.7906
0.7872
0.7987
0.7210
0.7731
0.7146
0.8084
0.7572
0.7927
0.7450
0.5752
0.3947
Geneformer
X
X
0.7891
0.7789
0.7810
0.6854
0.8028
0.7572
0.7986
0.7651
0.7929
0.7467
0.5293
0.3745
Nicheformer
x
X
0.7260
0.7119
0.6870
0.5858
0.7180
0,6336
0.7306
0.6842
0.7154
0.6539
0.5359
0.3718
CelIPLM
x
0.8358
0,8261
0.8179
0.7325
0.8068
0.7566
0.8139
0.7737
0.8186
0.7722
0.5597
0.3985
SToFM
0.8514
0.8312
0.8380
0.8014
0.8175
0.7716
0.8380
0.8143
0.8362
0.8046
0.6059
0.4588
DLPFC Luyer Segmentation
ST Pretraining
DLPFCI
DLPFC2
DLPFC3
DLPFC4
Average
Cross-slice
Model
Expr
Pos
M-S
Acc
Fg
Fi
Acc
Fi
Aoc
Fi
Acc
Fi
scGPT
x
x
x
0.6943
0.6599
0.6565
0.5908
0.6919
0.6151
0.6696
0.6055
0.6781
0.6178
0.6689
0,5885
Geneformer
X
X
X
0.6404
0,5770
0.6286
0.5464
0.6373
0.5677
0.6157
0.5513
0.6305
0.5606
0.5981
0.5440
Nicheformer
X
X
0,5712
0,5319
0.5942
0.5300
0.5588
0.5192
0.5648
0.5141
0.5723
0.5238
0.5809
0.5138
CellPLM
V
X
0.7005
0.6553
0.6960
0.5923
0.7143
0.6482
0.6725
0.5918
0.6958
0.6219
0.6908
0.5953
SToFM
v
0,7082
0.6755
0.6974
0.6453
0,7157
0.6659
0.6856
0.6274
0.7014
0.6535
0.6981
0.6437
completely new sequencing technique.
Table 2: Performance of cell type annotation on mouse brain slices.
We use classification accuracy and F1 score evaluation metrics. The results in Table 1 demonstrate that SToFM outperforms existing methods across different tissue region semantic segmentation tasks. As the only baseline that uses both gene expression and location information of the cells, CelIPLM achieves second-best performance in most tasks, which further demonstrates the importance of jointly in- corporating these two types of information. Notably, we observe that SToFM surpasses other models to greater ex- tent in the cross-slice setting, demonstrating its robustness and transferability. We attribute this to the integration of micro- and macro-scale information in SToFM, since tissue morphology and cell-cell interaction patterns are more likely to transfer across different ST slices. In conelusion, SToFM achieves high-precision tissue region segmentation in vari- ous complex biological contexts, providing powerful tool for the biological interpretation of ST data.
Brain
Brain2
Models
Accuracy
Macro F1
Accuracy
Macro F1
Geneformer
0.5646
0,3853
0,8540
0.6742
Nicheformer CelIPLM SToFM
0,5186
0,3550
0.8447
0.7575
0,6001
0.4186
0.9256
0,7332
0.6349
0.4951
0.9289
0.8362
low-quality gene expression data.
4.4. Zero-shot Clustering and Visualization
Clustering and visualizing cells is important in bioinformat- ics analysis. In this section, we use cell representations from SToFM and baseline models for clustering and visual- ization, and evaluate how well the clustering results align with the ground truth of cell types on a MERFISH mouse brain slice (Allen et al., 2023). The experimental results in Fig. 3 demonstrate that clustering based on SToFM cell embeddings achieves the best performance in terms of NMI and ARI scores when compared to true cell type labels. The visualization further illustrates the ability of SToFM to pro- duce high-quality cell embeddings, where each cell type forms a distinct and tight cluster in the SToFM embedding space. Interestingly, the cell representations of SToFM also reflect some relationships between cells. For example, oligo- dendrocyte progenitor cell shares a common progenitor with neuron and can further differentiate into oligodendrocyte.
4.3. Cell Type Annotation in Spatial Transcriptomies
Cell type annotation is a fundamental problem in ST re- search. Compared to scRNA-seq data, the gene expression profiles of ST data have lower data quality due to fewer cap- tured genes or a higher probability of dropout zeros. This poses challenges for cell type annotation based on gene expression profiles. For this task, we use two mouse brain datasets obtained using the Stereo-seq and SeekSpace tech- nologies (Cheng et al., 2022; SeekGene, 2025). These two datasets have been manually annotated as 25 and 8 different cell types, respectively, as ground truth. The results in Table 2 show that our model significantly outperforms existing methods on both datasets. This confirms that incorporating spatial information can help improve cell type annotation on ST data. Specifically, the tissue region where the cells are located and the cell type composition in the cell's 's neigh- borhood can aid in inferring cell types even in the scenes of
4.5. Spatial Deconvolution
As described in Appendix D. in some ST methods, each data point actually represents a small region called spot which contains parts of different cells. Therefore, spot can be described as a proportion of various cell types. Predict- ing the cell type proportion in ST data is known as spatial deconvolution. Considering that foundation models can
7

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Geneformer ARI = 0.393, NMI 0.512
Nicheformer ARI 0.531, NMI 0.607
Cellp ARI 0.902, NMI 0.891
SToF M ARI 0.953, NMI 0.921
endothelial cell
astrocyte
digadendracyte
microglial cell
inhibitory interneuron
neuron
pericyte
oligodendracyte precursor cell
medium spiny neuron
Figure 3: UMAP visualization cell representations on a mouse brain slice, colored by cell types. ARI and NMI scores were calculated with Leiden clustering results and cell type labels.
Table 3: Performance of spatial deconvolution on mouse liver slices.
Table 5: Ablation study of the effect of different scales on SToFM performance. VCs: virtual cells. DA: domain adaptation.
Models
Pearsonf
LiverSample RMSEL
MAEU
Pearson
LiverCross RMSE
MAEL
Ablated Models
Gene
Multi-scale
Micro
Macro
EmbryoCross Macro Fi
Macro F.
Brain1
Geneformer
0,7152
0.0909
0.0446
0.6906
0.0970
0.0456
Cell encoder w/o DA
x
x
0,3745
0,3853
Nicheformer
0.7143
0.0909
0.0440
0.6938
0.0976
0.0447
Cell encoder w/ DA
0.4155
0.4725
CalIPI M
0.7458
0,0865
0.0423
0.7248
0.0921
0.0441
SToFM w/o VCs SToFM
x
0,4291
0,4893
SToFM
0.7666
0,0833
0.0408
0.7546
0,0867
0,0430
0.4588
0.4951
as labels and the remaining 327 as inputs. The evaluation metrics are Pearson correlation, RMSE and MAF. The ex- perimental results in Table show the excellent performance of SToFM this task.
Table 4: Performance of imputation on human skin slices.
SkinCross
Models
Pearsonf
RMSEJ
MAE]
Geneformer
0.3610
0,3671
0.1029
Nicheformer
0.2829
0.3772
0.1163
CellPLM
0.3985
0.3758
0.0905
4.7. Ablation Study
SToFM
0.4877
0.3356
0.0886
In order demonstrate the performance improvement re- sulting from modeling information at multiple scales, we conduct a set of ablation experiments shown in Table 5. Specifically, we first ablate the macro-scale information by removing the virtual cells. Then, we ablate the model's ability to jointly model multiple cells at the micro scale by removing the SE(2) Transformer. Finally, we remove do- main adaptation to test the model's original ability to handle gene-scale expression profiles. The ablation experiments are performed on two representative tasks: embryo structure segmentation and cell type annotation.
generate consistent representations for data from different slices, using foundation models to transfer deconvolution results from labeled slices to unlabeled slices provides promising solution for this task. This section explores the potential of using SToFM for ST deconvolution in a simple quantitative experiment. The experiment is conducted on a Stereo-seq mouse liver dataset (Wu et al., 2024), testing the models' ability to transfer deconvolution labels both within the same slice and across different slices. The ground truth is annotated by biologists (Wu et al., 2024). We adopt 3 evaluation metrics for regression: Pearson correlation, RMSE and MAE. Table 3 reports the results, showing that SToFM achieves the best performance in both within-slice and cross-slice deconvolution tasks. In addition, SToFM achieves more improvement under the cross-slice setting, which further emphasizes its excellent transferability.
The results have demonstrated the performance gains brought by each scale, confirming that the integration of multi-scale information is helpful for various downstream tasks. In addition, the cross-slic results also prove that more macroscopic information greatly enhances the trans- ferability of the model.
4.6. Spatial Transcriptomics Imputation
5. Conclusions and Limitations
ST technologies often cannot capture a large number of genes while maintaining high resolution, posing challenges for data analysis. ST imputation, i.c., inferring the uncap- tured gene expression levels, an important task in the analysis of these data (Abdelaal et al., 2020; Jiang et al., 2024). Similar to 4.5, we explore the potential of SToFM for ST imputation. We collect two 10x Xenium human skin datasets (10XGENOMICS, 2025) to be used as a training dataset and a test dataset. Each cell in these datasets has 377 known gene expressions, of which we randomly select
In this work, we propose SToFM, a multi-scale ST foun- dational model, pretrained on our large-scale ST corpus, SToCorpus-88M. By effectively capturing and integrating information across different scales, SToFM achieves a com- prehensive understanding of biological information in ST data. It demonstrates outstanding performance in various downstream tasks, introducing a new perspective for the analysis of ST data and driving progress in the field.
Currently, SToFM still has some limitations. SToFM only
8

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
considers three scales: macro, micro, and gene. In the future, may consider modeling and integrating more scales using methods like image pyramid (Adelson et al.,
nities in spatially resolved transcriptomic data analysis. Nature Communications, 12(1))5283, 2021.
1984), or introduce causal machine learning methods to model gene regulatory relationships (Tejada-Lapuerr) et al., 2025; Zhong et al., 2024). Furthermore, is also possible to enhance the model by using prior biological knowledge or other modalities, such as known ligand-receplor pairs or pathological images. We reserve these explorations for future works.
Avşar, G. and Pir, P. A comparative performance evaluation of imputation methods in spatially resolved transcrip- tomics data. Molecular Omics, 19(2):162 11731 2023.
Biancalani, T., Scalia, G., Buffoni, L., Avasthi, R., Lu, Z.,
Sanger, A., Tokcan, N., Vanderburg. C. R.,, Segerstolpe, A., Zhang, M., al. Deep learning and alignment of spatially resolved single-cell transcriptomes with tangram. Nature methods, 18(1 : 13:2-13133,, 2021.
Code Availability
Biology, C. S.-C., Abdulla, S., Aevermann, B., Assis, P., Badajoz, S., Bell, S. M., Bezzi, E., Cakir, B., Chaffer, J., Chambers, S., et al. Cz cellxgene discover: A single- cell data platform for scalable exploration, analysis and modeling of aggregated data. bioRxiv, pp. 2023-10, 2023.
SToFM will soon be added to the OpenBioMed toolkit: http://gwwwroooooaaaooo///////aa
Code is available at: http://giiibbbcom//PPaaoolistttt
Acknowledgements
Bordukova, M., Makarov, N., Rodriguez- R.,
Schmich, F,, and Menden, M. P. Generative artificial intelligence empowers digital twins in drug discovery and clinical trials. Expert Opinion Drug Discovery, 19(1)333-2, 2024.
This research is supported by the National Key R&D Pro- gram of China (No. 2022YFF1203002) and PharMolix Inc.
Impact Statement
Cao, Y., Xu, J., Lin, S., Wei, F., and Hu, H. Genet: Non-local networks meet squeeze-excitation networks and beyond. In Proceedings of the IEEE/CVF international conference on computer vision workshops, pp. 0-0, 2019.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Chan, L., Hosseini, M. S., Rowsell, C., Plataniotis, N., and Damaskinos, S. Histosegnet: Semantic segmentation of histological tissue type in whole slide images. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10662-10671, 2019.
References
10xGENOMICS. 10xgenomics datasets. https: htttps://wwy /www. 10xgenomicc cs. wtt///om//a///e datasets, 2025. Accessed: 2025-01-20.
Chen, A., Liao, S., Cheng, M., Ma. K., Wu, L.,, Lai, Y., Qiu, X., Yang, J., Xu, J., Hao, S., et al. Spatiotempo- ral transcriptomic atlas of mouse organogenesis using dna nanoball-pattereel arrays. Cell, 185(10):1777-1792, 2022.
Abdelaal, T., Mourragui, S., Mahfouz, A., and Reinders, J. Spage: spatial gene enhancement using sema-see. Nucleic acidy research, 48(18) 117181))107-e10 2020.
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, pp. 1-3, 2024.
Chen, J.. Zhou, M., Wu, W. Zhang. J., Li, Y., and Li, D. Stimage- 144m: A histopathology image-gene expression dataset for spatial transeriptomics. ArXiv, pp. arXiv-2406. 2024.
Adelson, E. H., Anderson, C. H., Bergen, R., Burt, P. J., and Ogden, J. M. Pyramid methods in image processing. RCA engineer, 29())33-41, 1984.
Chen, K. H., Boettiger, A. N., Moffitt, J. R., Wang, S., and Zhuang, X. Spatially resolved, highly multiplexed maa profiling in single cells. Science, 348(6233) :aaa2000 2015.
Allen, W. F., Blosser, T. R., Sullivan, Z. A., Dulac, C., and Zhuang, X. Molecular and spatial signatures of mouse brain aging at single-cell resolution. Cell, 186(1):194- 208, 2023.
Cheng, M., Wu, L., Han, L.,, Huang, X., Lai, Y., Xu, J., Wang, S., Li, M., Zheng, H., Feng, W., et al. A cellular resolution spatial transcriptomic landscape of the medial structures in postnatal mouse brain. Frontiers in Cell and Developmental Biology, 10:878346, 2022.
Atta, I. and Fan, J. Computational challenges and opportu-

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Choi, H., Park, J., Kim, S., Kim, J., Lee, D., Bae, S., Shin, H., and Lee, D. Cellama: Foundation model for single cell and spatial transcriptomics by cell embedding leveraging language model abilities. bioRxiv, pp. 2024-05 00,40-552220 2024.
Hasel, P., Rose, I. V., Sadick, J., Kim, R. D., and Liddelow,
S. A. Neuroinflammatory astrocyte subtypes in the mouse brain, Nature neuroscience, 24(10):1475-144 2021.
He, P. Qu, A., Xiao, S., and Ding, M. Detisseg: A dual- encoder network for tissue semantic segmentation of histopathology image. Biomedical Signal Processing and Control, 87:105544, 2024.
Cui, H., Wang, C. X., Maan, H., Pang, K., Luo, F,, and Wang. B. segpt: Towards building a foundation model for single-cell multi-omics using generative ai. bioRxiv, 2023. URL https: / seweiissmmaiiiccccooco org/Corpus : 6488664426.
He, S., Bhatt, R., Brown, C., Brown, E. A., Buhr, D. L., Chantranuvatana, K., Danaher, P., Dunaway, D., Garrison, R. G., Geiss, G., et al. High-plex imaging of rna and proteins at subcellular resolution in fixed tissue by spatial molecular imaging. Nature Biotechnology, 40(11):179) 1806, 2022.
Domínguez Conde, C., Xu, C., Jarvis, L.., Rainbow, D., Wells, S., Gomes, T., Howlett, S., Suchanek, O., Polanski, K., King. H., et al. Cross-tissue immune cell analysis reveals tissue-specific features in humans. Science, 376 (6594))eab15197, 2022.
HESTA. Hesta: Human embryogenesis spatiotemporal tran- scriptomic atlas. db.cngh org / omi cs /hesta/ brain/, 2025, Accessed: 2025-01-20.
Dong, K. and Zhang. S. Deciphering spatial domains from spatially resolved transcriptomics with an adaptive graph attention auto-encoder. Nature communications, 13(1): 1739, 2022.
Jaume, G., Doucet, P., Song, A., Lu, M. Y., Almagro Pérez, C., Wagner, S., Vaidya, A., Chen, R., Williamson, D., Kim, A., et al. Hest-lk: A dataset for spatial transcrip- tomics and histology image analysis. Advances in Neural Information Processing Systems, 37:53798-53833, 2024.
Du, J., Yang, Y-C.,, An, 7.-J., Zhang, M.-H., Fu, X.-H., Huang, Z.-F. Yuan, Y., and Hou, J. Advances in spa- tial transcriptomics and related data analysis strategies. Journal of Translational Medicine, 21(1):330, 2023.
Jiang, X., Xie, Y., and Xie, J. xmint: A multimodal integra- tion transformer for xenium gene imputation. In ICMI. 2024 Workshop on Efficient and Accessible Foundation Models for Biological Discovery, 2024.
Franzén, L., Olsson Lindvall, M., Hühn, M., Prasinski, V., Setyo, L., Keith, B. P., Collin, A., Oag. S., Volckaert, T., Borde, A., et al. Mapping spatially resolved transcrip- tomes human and mouse pulmonary fibrosis. Nature Genetics, 1625):172511736 2024.
Jin, Y., Zuo, Y., Li, G., Liu, W., Pan, Y., Fan, T., Fu, X., Yao, X., and Peng, Y. Advances in spatial transcriptomics and its applications in cancer research. Molecular Cancer, 23 (1):129, 2024.
Gao, T., Yao, X., and Chen, D. SimCSE: Simple con- trastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Nat- ural Language Processing, pp. 6894-6910, Online and Punta Cana, Dominican Republic, November 2021. As- sociation for Computational Linguistics. doi: 18.53/ URL https:///// h///a/aaaannooooo....o.oo org/ /tgg/2221.0 emnlp 552n.552.
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Zidek, A., Potapenko, A., et al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):533-58, 2021.
Gong, J., Hao, M., Cheng, X., Zeng. X., Liu, C., Ma, J., Zhang, X., Wang. T., and Song, L. xtrimogene: An efficient and scalable representation learner for single-cell ma-seq data. arXiv preprint arXiv:2311 15156, 2023.
Jung, N. and Kim, T.-K. Spatial transcriptomics in neuro- science. Experimental & Molecular Medicine, 55(10): 2105-2115 2023.
Levine, D., Rizvi, S. A., Lévy, S., Pallikkavaliyaveeti,, N., Zhang, D., Chen, X., Ghadermarzi, S., Wu, R., Zheng, 7.,, Vrkie, I., et al. Cell2sentence: teaching large language models the language of biology. BioRxiv, pp. 00, 2023.
Hao, M., Gong, J., Zeng, X., Liu, C., Guo. Y., Cheng, X., Wang, T., Ma, J., Song, L. T., and Zhang, X. Large scale foundation model on single-cell transeriptomics. bioRxiv, 2023. URL https: / http::::/ap..smmmiitccceeee semanticscholas cttp/Corpus::::::
Li, H., Zhou, J., Li, Z., Chen, S., Liao, X., Zhang, B., Zhang, R., Wang, Y., Sun, S., and Gao, X. A comprehen- sive benchmarking with practical guidelines for cellular deconvolution of spatial transcriptomics. Nature Commu- nications, 14(1) 1558,, 2023.
Harrison, P. W., Amode, M. R., Austine-Orimoloye O., Azov, A. G., Barba, M., Barnes, I., Becker, A., Bennett, R., Berry, A., Bhai, J., et al. Ensembl 2024. Nucleic acids research, 52(DI))D891 D8999, 2024.
10

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Lin, Y., Luo, L.,, Chen, Y., Zhang, X., Wang. Z., Yang, W., Tong, M., and Yu, R. St-align: A multimodal founda- tion model for image-gene alignment in spatial transcrip- tomics. arXiv preprint 2rXiv:2411.1679, 2024.
Westholm, J. O., Huss, M., et al. Visualization and analy- sis of gene expression in tissue sections by spatial tran- scriptomics. Science, 353(6244):788822 2016.
Stickels, R. R., Murray, E.,, Kumar, P., Li, J., Marshall, L., Di Bella, D. J., Arlotta, P., Macosko, E. Z., and Chen, F. Highly sensitive spatial transcriptomics at near-cellular resolution with slide-seqv2. Nature biotechnology. 39(3): 313-319, 2021.
Liu, T., Li, K. Wang, Y., Li, H., and Zhao, Evaluating the utilities of foundation models in single-cell data analysis. bioRxiv, pp. 2023-09, 2023.
Liu, 7.,, Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference computer vision, pp. 10012-10022, 2021.
Svensson, V., Pento-Tormo, R., and Teichmann, S. Ex- ponential scaling of single-cell raa-seq in the past decade. Nature protocols, 13(4):599-604 2018.
Szalata, A., Hrovatin, K., Becker, S., Tejada-Lapuerta, A., Cui, H., Wang. B., and Theis, F. Transformers in single-cell omics: review and new perspectives. Nature methods, 21(88)1430-144 1443, 2024.
Marx, Method of the year: spatially resolved transcrip- tomics. Nature methods, 18(1):9-14, 2021.
Maynard, K. R., Collado-Torres, L., Weber, L. M., Uytingco, C., Barry, B. K., Williams, S. R., Catallini, J. L., Tran, M. N., Besich, 7.,, Tippani, M., et Transcriptome- scale spatial gene expression in the human dorsolateral prefrontal cortex. Nature neuroscience, 24(3):425 436, 2021.
Tejada-Lapuerta, A., Bertin, P., Bauer, S., Aliee, H., Bengio, Y., and Theis, F. Causal machine learning for single-cell genomies. Nature Genetics, pp. 1-12, 2025.
Theodoris, C. V., Xiao, I.., Chopra, A., Chaffin, M. D., Sayed, Z. R. A., Hill, M. C., Mantineo, H., Brydon, E. M., Zeng. Z., Liu, X. S., and Ellinor, T. Transfer learning enables predic- tions in network biology. Nature, 618:616-66 624, 2023. URL petps://aphs org/ hetg/CorpuSTD.. /tsS/CorpuTT::25000005 259002047.
nanoString.
TM smi datasets.
https:
ciranostringg.compooodii cosme-spattill-
htpe-dataset//// 2025. Accessed: 2025-01-20.
Pan, J., Li, Y. Lin, Z., Lan, Q., Chen, H., Zhai, M., Sui, S., Zhang, G., Cheng, Y., Tang, Y., et al. Spatiotemporal transcriptome atlas of human embryos after gastrulation. bioRxiv, pp. 2023-0022000 04, 2023.
Traag. V. A., Waltman, L., and Van Eck, N. J. From lou- vain leiden: guaranteeing well-connected communities. Scientific reports, 9():1112,,019
Schaar, A. C., Tejada-Lapuerta, A., Palla, G., Gutgesell, R., Halle, L., Minaeva, M., Vornholz, L., Dony, L., Drummer, F., Bahrami, M., al. Nicheformer: foundation model for single-cell and spatial omics. bioRxiv, pp. 2024-04, 2024.
Vizgen. Data release program. https: /ttp://vviieeeeeeeee com/ data-re setaa-release-proggze 01-20.
2025. Accessed: 2025-
Wei, C., Xie, S. M., and Ma, T. Why do pretrained language models help in downstream tasks? an analysis of head and prompt tuning. Advances in Neural Information Processing Systems, 1116158-161000 2021.
SeekGene. Seekspace single-cell transcriptomies technology. https:/ www. seekgene com/ seek seekspacdxby jzbzsszzzzss........... 2025. Accessed: 2025-01-
Wen, H., Tang, W., Dai. X., Ding, J., Jin, W., Xie, Y., and Tang, J. Cellplm: pre-training of cell language model beyond single cells. bioRxiv, pp. 2023-10, 2023.
20.
Shi, H., He, Y., Zhou, Y., Huang, J., Maher, K., Wang, B., Tang, Z., Luo, S., Tan, P., Wu, M., al. Spatial atlas of the mouse central nervous system at molecular resolution. Nature, 622(7983):552-500 2023.
Wenteler, A., Occhetta, M., Branson, N., Huebner, M., Curean, V., Dee, W., Connell, W., Hlawkins-Hooke A., Chung, P., Ektefaie, Y., et al. Perteval-scfm Benchmark ing single-cell foundation models for perturbation effect prediction. bioRxiv, pp. 2024-10, 2024.
Shuaibi, M., Kolluru, A., Das, A., Grover, A., Sriram, A., Ulissi, Z., and Zitnick,
Rotation invariant graph neural networks using spin convolutions. arXiv preprint DrXiv:2106.09575. 2021.
Williams, C. G., Lee, H. J., Asatsuma, T., TortoTTomo, R., and Haque, A. An introduction to spatial transcriptomics for biomedical research. Genome Medicine, 14(1):68, 2022.
Ståhl, P. L., Salmén. F. Vickovic, S., Lundmark, A., Navarro, J. F., Magnusson, J., Giacomello, S., Asp, M.,

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Wu, B., Shentu, X., Nan, H., Guo, P., Hao, S., Xu, J. Shangguan, S., Cui, L.., Cen, J., Deng, Q., et al. A spa- tiotemporal atlas of cholestatic injury and repair in mice. Nature Genetics, pp. 1-15, 2024.
Zhao, S., Zhang, J., Wu, Y., Luo, Y., and Nie, Z. Langcell: Language-cell pre-training for cell identity understanding. arXiv preprint arXiv:2405.000.
2024.
Zheng. K., Long, S., Lu, T., Yang. J., Dai, X., Zhang, M., Nie, Z., Ma, W.-Y., and Zhou, H. Multi-scale protein language model for unified molecular modeling. bioRxiv, pp. 2024-03, 2024.
Wu, S. 7., Al-Eryani, G., Roden, D. L., Junankar, S., Harvey, K., Andersson, A., Thennavan, A., Wang. C., Torpy, J. R., Bartonicek, N., et al. A single-cell and spatially resolved atlas of human breast cancers. Nature genetics, 53(9): 1334-1347, 2021.
Zhong, Y., Wu, X., Zhang, L.,, Yang. C., and Jiang, T. Causal-iqa: Towards the generalization of image qual- ity assessment based on causal inference. In Forty-first International Conference on Machine Learning. 2024.
Xu, K., Hu, W. Leskovec, J., and Jegelka, S. How powerful are graph neural networks? arXiv preprint arXiv: 18/0. 810.00826, 2018.
Zhong, Y., Zhao, X., Zhao, G., Chen, B., Hao, F., Zhao, R., He, J., Shi, L., and Zhang, L. Ctd-inpainting: Towards the coherence of text-driven inpainting with blended dif- fusion. Information Fusion, pp. 103163, 2025.
Xu, Z., Wang, W. Yang, T., Li, L., Ma, X., Chen, J., Wang, J., Huang. Y., Gould, J., Lu, H., et al. Stomiesdb: com- prehensive database for spatial transcriptomices data shar- ing, analysis and visualization. Nucleic acids research, 520DI):D10S3-D106 2024.
Zhou, G., Gao, Z., Ding, Q., Zheng, H., Xu, H., Wei, Z., Zhang, L., and Ke, G. Uni-mol: A universal 3d molecular representation learning framework. 2023.
Yang, F., Wang, W., Wang, F., Fang, Y., Tang, D., Huang, J., Lu, H., and Yao, J. Schert as a large-scale pretrained deep language model for cell type anmotation of single-cell ma- seq data, Nature Machine Intelligence, 4(10):852-866, 2022. doi:
Zhou, Y., He, W., Hou, W., and Zhu, Y. Pianno: a prob- abilistic framework automating semantic annotation for spatial transcriptomics. Nature Communications, 15(1): 2848, 2024.
01015/442256-22-0055555
Yang, X., Liu, G., Feng, G., Bu, D., Wang, P., Jiang, J., Chen, S., Yang, Q., Miao, H., Zhang, Y., et al. Genecom- pass: deciphering universal gene regulatory mechanisms with knowledge-informedd cross- species foundation model. Cell Research, pp. 1-16, 2024.
Ziegenhain, C., Vieth, B., Parekh, S., Reinius, B., Guillaumet-Adkins, A., Smets, M., Leonhardt, H., Heyn, H., Hellmann, I,, and Enard, W. Comparative analysis of single-cell ma sequencing methods. Molecular cell, 65 (4):631-643, 2017.
Zormpas, E., Queen, R., Comber, A., and Cockell, J. Mapping the transcriptome: Realizing the full potential of spatial data analysis. Cell, 186(26);5677-566), 2023.
Ying, C., Cai, T., Luo, S., Zheng, S., Ke, G., He, D., Shen, Y., and Liu, T.-Y. Do transformers really perform badly for graph representation? Advances in neural information processing systems, 34:28877-28888, 2021.
Ying, Z., You, J., Morris, C., Ren, X., Hamilton, W., and Leskovee, J. Hierarchical graph representation learning with differentiable pooling. Advances in neural informa- tion processing systems, 31, 2018.
Yuan, X., Zhan, Z., Zhang, Z., Zhou, M.. Zhao, J., Han, B., Li, Y., and Tang, J. Cell-ontology guided transcriptome foundation model. arXiv preprint arXiv:2408.122 12373, 2024.
Yuan, Z., Pan, W. Zhao, X., Zhao, F,, Xu, Z., Li, X., Zhao, Y., Zhang, M. Q., and Yao, J. Sodb facilitates comprehen- sive exploration of spatial omics data. Nature Methods, 20(3):3873399, 2023.
Zhao, S., Zhang, J., and Nie, Z. Large-scale cell representa- tion learning via divide-and-conm wwide-and-conquer contrastive learning. arXiv preprint 0et.233660437., 2023.
12

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomies
Appendix
A. Architecture of the Model Components
In this section, we describe the detailed architecture of model components.
A.1. Cell Encoder
We use the single-cell foundation model, Geneformer (Theodoris et al., 2023), as the cell encoder of SToFM. The encoding process Geneformer is as follows: (1) Normalize the gene expression matrix for each cell. Then, divide the expression level of each gene by the median of its non-zero expression values across all cells to obtain the relative expression level of each gene in the cell. (2) Rank the genes in each cell based their relative expression levels to generate sequence representation for the cell. (3) Use BERT model to encode the serialized cell representation.
In the domain adaptation process, we follow Geneformer's pretraining task of Masked Gene Modeling(MGMI) where some genes the cell sequence representation are masked, and the model predicts the masked genes to reconstruct them. Additionnlly, we incorporate the self-supervised Contrastive Learning (CL) approach (Gao et al., 2021; Zhao et al., 2024) to enhance the model's 's ability to capture global information. The loss function of domain adaptation (LDA) is as follows:
(i,m(F Ft,
DD LMOM + fcL Σ
H H 02g)
Σ log T
1
2j=1 FF
where is the number of masked genes, Vig and biy respectively represent the label and predicted probability of the i-th masked position being identified as the j-th gene, is the cross entropy loss function. the batch size, sim is the cosine similarity function, the temperature parameter, F; and Ft represent the embedding of the i-th cell and its positive sample, respectively.
A.2. SE(2) Transformer
We draw inspiration from the Uni-Mol framework (Zhou al., 2023) implement our SE(2) Transformer, using simple yet effective architecture. The model structure is illustrated in Fig. A.1 and can be described as follows: (1). A distance matrix is used the spatial input ensure SE(2) invariance in straightforward manner. (2). The initial cell input is obtained by adding the cell features to learnable type embedding that distinguishes virtual cells from real cells. The distance matrix is passed through learnable Gaussian module (Shuaibi et al., 2021; Zhou et al., 2023) to project it to the dimensionality of the model's attention heads, serving as the initial pair representation. (3). Both cell representations and pair representations are propagated between layers. each layer, a self-attention matrix is computed for the cell representations and is added with the pair representation which serve as attention bias. The resulting combination used update both the pair representation and the attention matrix. This process illustrated follows:
2i (K
R R!A +
Vd
Attention softmax
R 1.t y
where R!.b is the pair representation matrix corresponding to the attention head h in layer and QuA,, Kli,, Vll are the attention parameters corresponding to the attention head in layer and head h.
B. More Experimental Results
B.1. Full-parameter tuning
In the main text, we aim to fairly compare the representation learning abilities of SToFM with a basic model serving as the baseline. Therefore, we only perform sead-tuning on downstream tasks. However, achieving better results downstream tasks through full- parameerrr tuning is also an important use case for foundation models. Therefore, in this section, we evaluate the performance of full- fall-parameter tuned SToFM on the cell type annotation task.
13

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
(2) Transformer (Sr))
Node Type Encoding
Multi- Head Attention
S Softmax & Matmul
Matmul & Scale
Add & Norm
Feed Forward
Add Norm
Gaussian
Multi-Head Attention with Pair Blas
Figure A.I: The architecture of SE(2) Transformer in SToFM
Table A.I: Performance of cell type annotation on head tuning and full-parameter tuning.
Brain
Brain2
Models
Accuracy
Macro F1
Accuracy
Macro Fi
SToFM head tuning
0.6349
0.4951
0.9289
0,8362
SToFM full-parameter tuning
0.6987
0.5532
0.9725
0.8903
B.2. More Ablation Studies
The resolution of clustering and the scale of sub-slices. They are critical hyperparameters in SToFM. In our experiments, used the Leiden algorithm for clustering, with the resolution ensured that each sub-slice contained approximately 1,000 cells, which aligns with the empirically suitable data scale for the SE(2) Transformer and includes sufficient microscopic information. Ablation experiments in Table A.2 confirmed that these hyperparameter settings fall within a reasonable range. Additionally, SToFM demonstrates robustness to changes in these hyperparameters within certain range.
1.0, typically resulting dozens of clusters. We
Micro-scale components and spatial distance matrix. We ablate the micro scale by limiting the scale of sub-slices to 1, to ensure that the cells could only interact with the virtual cells that represent macro-scale information, but not with other cells in the micro-environment Experimental results are shown in Table A.3. significant decrease in performance is observed, which demonstrates the effectiveness of incorporating the micro-scale information. Then, we conduct an ablation study on the spatial distance matrix. The spatial information is very underutilized in this case, only used to construct virtual cells and divide sub-slices. As shown in Table A.3, the results demonstrate that removing the spatial distance matrix significantly decreases model performance.
Sample rate of the second-time cell encoding. The purpose of the second time cell encoding is to enable LMCM and CPDR to optimize the cell encoder through backpropagation. To balance the training cost and model performance, we use only a small number of cells for this computation, which is similar selecting smaller batch size update the cell encoder. For the selection of the sampling number, we determined the number of samples to be 12, given that the original Geneformer paper gave training batch size of 12. Considering the computational cost, we test the impact of the sample number on a small amount of data (1/8 of the SToCorpus-88M), as shown in Table A.4. The results show that the model has some robustness to this hyperparameter, just as the batch size often only affects the convergence speed rather than the model performance. However, setting the sample size to 0, i.c. freezing the cell encoder, will lead to decrease in model performance.
14

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomies
Table A.2: Ablation study of sub-slice scale and Leiden resolution.
Embryo2
EmbryoCross
Sub-slice Scale
Leiden Resolution
Accuracy
Macro Fi
Accuracy
Macro Fi
10
1.0
0.7554
0.7066
0,5740
0,4291
100
1.0
0.8282
0.7418
0.5818
0.4510
500
1.0
0.8293
0.7899
0.5977
0.4475
2000
1.0
0.8018
0.7457
0.6002
0.4554
1000
0.1
0,8171
0.7755
0,5966
0,4410
1000
0.5
0.8014
0.7614
0,6008
0.4572
1000
1.5
0.8352
0.7523
0.6055
0.4403
1000
1.0
0.8380
0.8014
0.6059
0.4588
Table A.3: Ablation study of micro-scale components and spatial distance matrix.
Model
Embryo2 Macro Fi
EmbryoCross Macro Fi
w/o micro
0.721
0.425
w/o spatial matrix
0.721
0.413
SToFM
0.801
0.459
Data volume. We pretrain the multi-scale ST representation learning phase using 12.5% and 50% of the data, as shown in Table A.5. The results show that reduction in data volume led significant decrease in model performance. Considering that the SToCorpus-881) consists of approximately 2,000 ST slices, we believe that reducing the amount of data may reduce data diversity and limit the model's transferability.
in Algorithm 1. We have conducted experiments to show how different a values affect the model's performance, as shown Table A.6. The model has a certain robustness in alpha, and we believe this may be because cells that are closer in location are more likely to have similar gene expressions (Zormpas et al., 2023). The alpha=0.8 that we chose is essentially the optimal setting.
Combination ratio of the two loss functions. These two loss relatively close in scale, and in experiments, combining them in 1:1 ratio allows both converge normally. Considering the computational cost, test the impact of the ratio of the two losses on the speed of convergence and the performance of the model on a small amount of data (1/8 of the SToCorpus-88M), as shown in Table A.7. (y is the loss ratio in L = x LMOM + (1 (17)) x LPDR)
B.3. Zero-shot clustering
Further than in Sec. 4.4, we visualize the model's representation of mixed data from two mouse brain slices in Fig. A.2.
It is worth noting that this is not a common task of batch integration in single-cell analysis. Our goal is not to make cells from different slices indistinguishable. Instead, we believe that the cell representation generated by SToFM contains some global information from the slices. As a result, shown in the figure, SToFM establishes similar representations for cells of the same type from different slices, and preserve subtle differences brought by global slice information.
C. Experiment Settings for Pretraining and Downstream Tasks
C.1. Pretraining settings
The training process is conducted using the PyTorch framework. We utilize the AdamW optimizer, with a learning rate strategy that involved a warm-up phase followed by linear decay. Pretraining is carried out on four NVIDIA Tesla A100 GPUs, with both Domain Adaptation and pretraining taking approximately 10 days each to complete. Additional experimental configurations are detailed in Table A.8.
15

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Table A.4: Ablation study of sample number of the second-time cell encoding (on 1/8 SToCorpus-88M),
Sample number
Embryo2 Macro Fi
EmbryoCross Macro Fi
0
0.722
0.417
0.754
0.424
12
0.758
0.423
Table A.5: Ablation study of data volume. We use 12.5% and 50% of the SToCorpus-88N, for multi-scale ST representation learning pre- pre-training, and compare them with model pre-trained on the full dataset. The results show that a larger pre-trainn vre-training dataset can improve model performance. We attribute this to the increased diversity of the data.
Data volume
Embryo2 Macro Fi
EmbryoCross Macro F1
12.5%
0.758
0.423
50%
0.782
0.450
100%
0.801
0.459
C.2. Downstream tasks settings
For downstream tasks, we adhere to the following standardized settings:
All datasets filtered to remove special categories such "Other" or "Unknown" and categories with less than 1% of the total samples.
Tasks with inherent randomness are run three times, and the results are averaged.
For tasks involving splitting the same dataset into training and testing sets, 8:2 random split is applied. Additionally, 10% of the training set is randomly selected validation set, and early stopping strategy is applied based on performance on the validation set.
Both the classification and regression heads are implemented as three-layer fully connected neural networks with Lea activation functions.
C.3. Details of multi-class classification tasks
Tissue region semantic segmentation (Sec. 4.2) and cell type annotation (Sec. 4.3) are both multi-class classification tasks. The number of categories are shown Table A.9. We filtered out categories with fewer cells than 1% of the total number of cells.
Geneformer ARI = 0.328, NMI =0.480
Nicheformer ARI 0.502, NMI =
CellP ARI 0.9911 0.891
SToFM ARI 0.956 =
Spatial
Type esoethaka cell
accscyle
sigedendtooye
merogiaa cell
tnb
splny nouron
S Slice
Figure A.2: UMAP visualization of cell representations on two mouse brain slices, colored by cell types and slices.
16

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Table A.6: Ablation study of a in Algorithm 1.
Embryo2 Maero F EmbryoCross Macro Fi
0
0.751
0.435
0.2
0.767
0.458
0.4
0.778
0,440
0.6
0.796
0,436
0.8
0.801
0.459
1.0
0.769
0.448
Table A.7: Ablation study of a Algorithm 1.
Embryo2 Macro Fi EmbryoCross Macro Fi
0.683
0.409
0.2
0.729
0.423
0.5
0.758
0,423
0.8
0.753
0.429
1.0
0.701
0.415
D. Datasets
D.I. Sequencing techniques and data characteristics in ST
Spatial transcriptomics is an advanced technology that allows simultaneous acquisition of spatial tissue information and gene expression data. The main sequencing technologies can be categorized into image-based single-cell resolution techniques and sequencing-ban Spot techniques (I.i et al., 2023).
Image-based single-cell resolution technologies These technologies typically rely on fluorescence in situ hybridization (FISH) techniques, such as MERFISH (Chen et al., 2015), 10x Xenium and CosMx (He et al., 2022). By labeling specific RNA molecules and imaging them, these technologies achieve spatial expression profiling single-cell resolution. Their advantage lies in their high resolution, enabling precise localization of RNA within individual cells, but the number of genes that can be analyzed is usually limited. Some recent studies have compiled some image-based ST data (Jaume et al., 2024; Chen et al., 2024).
Sequencing-based spot technologies Sequencing- basse technologies typically involve placing capture areas of certain size, called spots, tissue sections (e.g., 10x Visium (Ståhl et al., 2016)). Each spot covers multiple cells, with size of approximately 50-100 microns. Newer techniques like Slide-seq (Stickels et al., 2021) use smaller beads with diameter of about 10 microns, significantly improving resolution. Additionally, Stereo-seq (Chen et al., 2022) leverages nanometer- scale lithography arrays, achieving spot sizes of 220 nanometers, approaching single-cell or even subcellular resolution. SeekSpace uses a special positional probe method to achieve a single-cell resolution sequencing- method.
Considering that some low-resolution technologies have spot diameters much larger than cell diameters, which cannot accurately reflect the gene expression distribution the single-cell level, exclude these data when constructing the datasets for pretraining and the vast majority of downstream tasks. Specifically, only low-resolution data in the DLPFC layer segmentation task (Sec. 4.2) to evaluate the model's transferability on this type of data.
D.2. SToCorpus-S8M
We selected data obtained from 6 high-resolution sequencing technologies to construct SToCorpus-88M. These include image-based technologies MERFISH, 10x Xenium, and CosMx, well as sequencing-based technologies Stereo-seq, Slide-seqv2, and SeekSpace. Among them, MERFISH, 10x Xenium, CosMx, and SeekSpace are single-cell resolution data, while Stereo-seg and Slide-seqv2 are data where the diameter of spots close to the diameter of cells.
Our data sources include six public databases or data release projects. We also obtain some unpublished SeekSpace data. The
17

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Table A.8: Experiment Configurations
Hyperparameter
Value
Leiden resolution
Multi-scale processing
in Algorithm
0.8
Split scale
1000
Vocab size
25426
Hidden size
512
Number of hidden layers
12
Max sequence length
2048
Cell Encoder
Number attention heads
Dropout
0.02
Hidden act
ReI.U.
LayerNorm eps
lc-12
Node Types
[Cell], [Virtaal Cell), [PAD]
Hidden size
256
Number of hidden layers
4
Max sequence length
2048
SE(2) Transformer
Number attention heads Dropout
0.1
Hidden act
GELU
LayerNorm eps
le-5
Optimizer
AdamW
Scheduler
Linear
Max learning rate
le-4
Pretraining
Warm up steps
Batch size
16
Gradient accumulation
4
Table A.9: Details of multi-class classification tasks.
Task
Data
Number of categories
Number of categories after filtering
Tissue region semantic segmentation (Sec. 4.2)
Embryo
18
16
Tissue region semantic segmentation (Sec. 4.2)
DLPFC
6
Cell type annotation (Sec. 4.3)
Brainl
25
21
Cell type annotation (Sec. 4.3)
Brain2
8
6
data volume and corresponding sequencing technologies from each source shown in Table A.10. We have removed the duplicated data obtained from different sources. In addition, individual slices in SToCorpus-88M are used for downstream tasks, and we exclude these data during pretraining.
These data are obtained from the following sources:
10xGENOMICS (10xGENOMICS, 2025): 10xgenomics. ctt.s/aaaadatattaaaaaaaaaaaaaaee
CELLAGENE (Biology et al., 2023): cell xgene wwt.eme.cziscieeeei... www.cciiiicccc..ooo.o com
nanoString (nanoString, pttpttttii..ccoooooooooo/////aaa f ffpe-dataset/
imager//
SODB (Yuan et al., 2023): gene ai. wenceetee.... wwtptttttccc//ppaaaaiiiiiioo
STOmicsDB (Xu et al., 2024); db. cngb. wwwp/rrgoiceee...
Vizgen (Vizgen, 2025): vizgen. rtw.
SeekSpace (SeekGene, 2025): not yet public, a demo can be found at seekgene. cttesneeccmm///s////ss// and downloaded at seekonetools-releat paa.laase.ooscc---biiiin...ooee aliyuncs com/ demo dat Mata/link link/ htw.lttered_ffauureebbb..e.
18

---

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomies
Table A.10: SToCorpus-88M data sources and statistics.
Sources
Sequencing technologies
Number of slices
Number of cells/spots
10xGENOMICS
10x Xenium
20
6,496,946
CELLxGENE
MERFISH, Slide-seqv2
762
23,531,670
nanoString
CosMx
6
1,118,500
SODB
seqve Stereo-seq, MERFISH
817
19,582,014
STOmicsDB
slide-seqv2. Stereo-seq, MERFISH
242
24,768,293
Vizgen
MERFISH
32
12,344,98 9,9
SeekSpace
SeekSpace
33
338,953
Total
technologies
1912
88, 18,1811,65
D.3. Details DownStream Task Datasets
Embryonic Structure Segmentation In this experiment, we use a human embryo Stereo-seq ST dataset (Pan et al., 2023; HESTA, 2025), which can be downloaded embryos Carnegie Stages (CS) 12-23. Considering that the morphology of later embryos is too complex, we only samples from CS12-13. We use four middle slices from CS12-13E2 (the slice position be viewed from the visual function of the data website), i.e., CS12-13E2S3, CS12-13E2S4, CS12-13E2SS and CS14-15E2S6, Embryol, Embryo2, Embryo3 and Embryo4 in Sec. 4.2. In addition, for the cross-slice task EmbryoCross in Sec. 4.2, we train on CS12-13E2S3, CS12-13E2S5, CS12-13E2S6, and test on CS12-13E2S4.
engb org/ st stomi es /hestaa. The dataset includes human
DLPFC Layer Segmentation In this experiment, we use human DLPFC 10x Visium ST dataset (Maynard et al., 2021). We select the four slices with the highest ids, 151673, 151674, 151675 and 151676, as DLPFCI, DLPFC2, DLPFC3 and DLPFC4 in section 4.2. For the cross-slice task DLPFCCross in Sec. 4.2, we train on slices 151674, 151675, and 151676, and test slice 151673.
Cell Type Annotation In this experiment, we use mouse brain Stereo-seq ST slice (Cheng et al., 2022) and mouse brain SeckSpace ST slice. The former can be downloaded from STOmiesDB (db. engb. wwwoototttoooooooooooooo ptt/stomiiee cs /ataaaatsseeeeeeeeee STDS00001333 13... The latter has not been publicly released before and will be released along with this paper. Both slices have been processed to single-cell resolution.
Zero-shot Clustering In this experiment, we use mouse brain MERFISH ST dataset (Allen et al., 2023), which can be downloaded from SODB (gene.. ai. tencent......... wwt.ttt...ccccoaaaacaacaaa aaw.aaaiaammmmmmmmmmmmmmmmm cs dataset MilatasetI-=1888 In Sec. 4.4, we use the first slice this dataset, MsBHainnggingppaaalDDooo..LLLLL _O. In Sec. B.3, we use two slices with some differences, MsBrainnginngSaatilDDee
and MsBBrinnAgngSSaaialDDDooo 4.0.
Spatial Deconvolution In this experiment, we use a mouse liver Stereo-seq ST dataset (Wu al., 2024), which can be downloaded from STOmicsDB (db. cngb. org/ s stomi stt.misssmssatttsssss htt.:/aattaaetts///s0000030033 sponding to the following 17 cell types regression labels: Cholangiocyte, Monocyte, KC, Fibroblast, HSC, LSEC, B_cell, T cell, LPLC, PC LVEC, PP LVEC, Neutrophil. NK, cDC, pDC, ILC1, VSMC. We select the slice D17 FR4 as LaverSample in Sec. 4.5, For the cross-slice task LiverCross in Sec. 4.5, we train on slice DO and test on slice D0.DY2.
We used the scores corre-
Imputation In this experiment, we use two human skin 10x Xenium ST slices (10xGE- NOMICS, 2025), which can be downloaded from www www.1Cxgenomics. com/ datasets/ human skin data iomiimmiie sum-human-mun mawaaan-mmliiitisssseeeee and cancer saancer-panee-e paael-1-standart cross-slice task SkinCross in Sec. 4.6, we train on slice Sample2, and test slice Samplel.
For
19


Reviews:
[{"review_id": "ssNMlemTUn", "reviewer": "Reviewer_TZDp", "summary": "The paper proposes SToFM, a multi-scale Spatial Transcriptomics foundation model, to effectively integrate macro-, micro-, and gene-scale information from Spatial Transcriptomics (ST) data. SToFM uses a combination of gene expression profiles, cell coordinates, and spatial relationships to learn representations of cells in their tissue context. It employs domain adaptation for gene expression embeddings, integrates spatial information using an SE(2) Transformer, and introduces novel pretraining tasks like masked cell modeling and pairwise distance recovery. The model outperforms existing methods on several biological tasks, demonstrating its ability to capture complex multi-scale biological information.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer TZDp:\n\nThanks for your appreciation and detailed review. We try our best to response the questions below:\n>Q1: Bridging the gap between ST and scRNA-seq data through transfer learning.\n\nST data consists of two parts: **spatial location** and **gene expression values**. As SToFM is a model for ST data, the purpose of domain adaptation is to utilize well-trained scRNA-seq model to better encode the **gene expression values** of ST data. The gene expression values of ST data have a similar distribution to scRNA-seq data and follow the same underlying gene co-expression patterns. Many well-established bioinformatics methods have also proven the effectiveness of transferring knowledge between scRNA-seq and ST data, such as Tangram[1]. Therefore, we believe it is reasonable to apply transfer learning between  scRNA-seq data and the **gene expression values** of ST data.\n\n>Q2: Since this model requires pretraining of the cell encoder, its scalability is limited.\n\nResearch like scGPT[2] has already shown that Transformer-based cell encoder models have good scalability. We believe that a well-pretrained cell encoder can help improve the model's scalability. \n>Q3: Computational costs. Lack of running time experiments and time complexity analysis.\n\n- The cell encoder is based on the Transformer architecture. And the time complexity of the SE(2) Transformer is similar to that of a standard Transformer [3]. Therefore, the time complexity of both parts of SToFM is **the same as the standard Transformer**. Specifically, the time complexity is $O(N *n^2+M *m^2)$, where $N$ and $n$ are the number of cells and genes, and $M$ and $m$ are the number and the scale of the sub-slices, respectively.\n- Section 4.2 of our paper provides details on the pretraining time cost. We have also added runtime experiments for inference, as shown in the Table. Specifically, running inference on an ST slice containing tens of thousands of cells typically takes 1–5 minutes, which we consider acceptable for practical applications.\n\n|Slice|Gene number|Cell number|Sub-slice number|Cell encoder time (s)|SE(2) Transformer time (s) | Total time (s)\n|-|-|-|-|:-:|:-:|:-:|\n|Allen1(Sec. 4.4)|355|21002|23|28.6|21.1|49.7|\n|Embryo2(Sec.4.2)|17552|12865|14|211.5|15.9|217.4|\n\n>Q4: Model's sensitivity to different alpha values.\n\nThank you for your suggestion! We have added experiments to show how different alpha values affect the model’s performance, as shown in the Table below:\n|alpha|Embryo2-F1| EmbryoCross-F1|\n|-|:-:|:-:|\n|0|0.751|0.435|\n|0.2|0.767|0.458|\n|0.4|0.778|0.440|\n|0.6|0.796|0.436|\n|0.8|**0.801**|**0.459**|\n|1.0|0.769|0.448|\n\nThe model has a certain robustness in alpha, and we believe this may be because cells that are closer in location are more likely to have similar gene expressions [4]. The alpha=0.8 that we chose is essentially the optimal setting.\n\n>Q5: The impact of the combination ratio of the two loss functions.\n\nWe will introduce further analysis on the loss ratio in the revised version of the paper. These two loss are relatively close in scale, and in our experiments, combining them in a 1:1 ratio allows both to converge normally. Considering the computational cost, we test the impact of the ratio of the two losses on the speed of convergence and the performance of the model on a small amount of data (1/8 of the SToCorpus-88M), as shown in Rebuttal-Fig.1 (https://anonymous.4open.science/api/repo/stofm-rebuttal/file/rebuttal-TZDp.pdf?v=5b2ab9f4) and the Table below. ($\\gamma$ is the loss ratio in $L=\\gamma*L_{MCM} + (1- \\gamma) * L_{PDR}$)\n\n|$\\gamma$|Embryo2-F1| EmbryoCross-F1|\n|-|:-:|:-:|\n|0|0.683|0.409|\n|0.2|0.729|0.423|\n|0.5|**0.758**|0.423|\n|0.8|0.753|**0.429**|\n|1.0|0.701|0.415|\n\nThe results show that the model's performance is robust to the ratio of loss functions. However, removing one of the loss functions will lead to a decrease in model performance. The $\\gamma$=0.5 that we chose is essentially the optimal setting.\n\n>Q6: The provided code does not run.\n\nWe had uploaded the code of the model and core algorithms along with the paper to help better understand the method. The executable code has been organized now, but anonymous links containing code are not allowed during rebuttal. We promise to release the executable code on github after the paper is published.\n___\nThank you again for your detailed review! Your insights are invaluable in aiding us to solidify our work. Should our responses satisfactorily address your queries, we would deeply appreciate your support for our work. \n\nRefs:\n\n[1] Deep learning and alignment of spatially resolved single-cell transcriptomes with Tangram\n\n[2] scGPT: toward building a foundation model for single-cell multi-omics using generative AI\n\n[3] Uni-Mol: a universal 3D molecular representation learning framework\n\n[4] Mapping the transcriptome: Realizing the full potential of spatial data analysis", "claims_and_evidence": "There may be a domain gap between ST data and scRNA-seq data. While ST data contains spatial information, scRNA-seq data only includes gene expression values. Therefore, the justification for bridging the significant gap between these two data types through transfer learning is insufficient.", "methods_and_evaluation_criteria": "Since this model requires pretraining of the cell encoder, its scalability is limited. Additionally, training both the cell encoder and the SE(2) Transformer can result in significant computational costs. However, the paper lacks running time experiments and time complexity analysis.", "theoretical_claims": "This paper does not present theoretical analysis.", "experimental_designs_or_analyses": "The alpha value in Algorithm 1 represents the combination ratio of cell embeddings and cell positions. However, the paper lacks an analysis of the model's sensitivity to different alpha values. It is necessary to examine how varying alpha values impact the integration of multi-scale information.\n\nThe impact of the combination ratio of the two loss functions, L_MCM and L_PDR, on the model's performance should be analyzed.", "relation_to_broader_scientific_literature": "The main contribution of this paper is the integration of multi-scale information from Spatial Transcriptomics (ST) data, including gene expression, cellular interactions, and tissue morphology. Unlike prior models, SToFM effectively combines micro-scale (cellular) and macro-scale (tissue) information using a multi-scale approach and SE(2) Transformer. This approach captures richer, more comprehensive biological insights from ST data.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "No other Strengths And Weaknesses", "other_comments_or_suggestions": "No other Comments", "questions_for_authors": "Please refer to the above contents.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6oeQDjNMnY", "reviewer": "Reviewer_Rgcj", "summary": "The paper proposes a multi-scale foundation model to integrate macro-scale tissue morphology, micro-scale cellular microenvironment and gene-scale gene expression profile of  spatial transcriptomics. The author constructs a large-scale spatial transcriptomics corpus containing approximately 2,000 tissue slices and 88 million cells for pretraining, which is claimed to be released. Various downstream tasks are validated to prove the performance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer Rgcj:\n\nThanks for your appreciation and detailed review. We try our best to response the questions below:\n>Q1: Ablation experiment on micro-scale components. **&** Q2: The effect of the spatial distance matrix. **&** Q4: The effect of incorporating spatial information.\n\nThank you for your suggestions for refining the ablation experiments! \n- For Q1, We ablate the micro-scale by limiting the scale of sub-slices to 1, to ensure that the cells could only interact informationally with the virtual cells that represent macro-scale information, but not with other cells in the micro-environment. Experimental results are shown in the Table below. A significant decrease in performance is observed, which demonstrates the effectiveness of incorporating the micro-scale information.\n- For Q2, we conducted an ablation study on the spatial distance matrix. The spatial information is very underutilized in this case, only used to construct virtual cells and divide sub-slices.  As shown in the Table below, the results demonstrate that removing the spatial distance matrix significantly decreases model performance.\n- For Q4, we had presented the results of completely ablating spatial information in the second row \"Cell Encoder w/ DA\" of Table 5 of our paper. We repeat the results in the Table below. Essentially, we rely on spatial information to establish relationships between multiple cells, with both micro- and macro-scale information being part of spatial information. If spatial information is removed, the model will not be able to interact between cells at all, essentially degrading to independently encoding each single cell.\n\n|Model|Embryo2-F1| EmbryoCross-F1|\n|-|:-:|:-:|\n|Q1: w/o micro|0.721|0.425|\n|Q2: w/o spatial matrix|0.721|0.413|\n|Q4: w/o spatial information (i.e. Cell Encoder w/ DA)|0.718|0.415|\n|SToFM|0.801|0.459|\n\n>Q3: The details on the second-time cell encoding mentioned in Sec. 3.2 should be written. If a sample rate is involved in the second-time cell encoding, the sensitivity of sample rate should be explored.\n- We will present more details in the revised paper, and will release detailed code to help readers better understand and reproduce our approach.\n- We will add experiments and discussion on the sample rate. The purpose of the second time cell encoding is to enable L_MCM and L_PDR to optimize the cell encoder through backpropagation. To balance the training cost and model performance, we use only a small number of cells for this computation, which is similar to selecting a smaller batch size to update the cell encoder. For the selection of the sampling number, we determined the number of samples to be 12, given that the original Geneformer paper gave a training batch size of 12. \n- Considering the computational cost, we test the impact of the sample number on a small amount of data (1/8 of the SToCorpus-88M), as shown in the Table below. The results show that the model has some robustness to this hyperparameter, just as the batch size often only affects the convergence speed rather than the model performance. However, setting the sample size to 0, i.e. freezing the cell encoder, will lead to a decrease in model performance.\n|Sample number|Embryo2-F1| EmbryoCross-F1|\n|-|:-:|:-:|\n|0|0.722|0.417|\n|4|0.754|0.424|\n|12|0.758|0.423|\n\n>Q5: This paper migrates the single-cell model through a domain adaptation strategy. However, there may be relatively large differences between different ST datasets, such as different technical platforms. How should we further improve adaptability?\n\n- Research such as scGPT [1] and LangCell [2] has already proven that large-scale pretraining is one of the best ways to remove batch effects in scRNA-seq data. Nicheformer [3] has also demonstrated that by pretraining, the model can gain modeling capabilities across different ST technology platforms. Indeed, despite the gap between different datasets, the underlying gene co-expression, intercellular signaling pathways, and other information should be largely uniform, which makes them have similar distributions that can be captured by the pre-trained model. \n- In the DLPFC experiments in Section 4.2, we purposely chose 10x Visium data that was not used in the pretraining. The excellent performance of SToFM on this task also proves that the model has gained the ability to transfer across technology platforms from the pretraining. We believe that our model and dataset can be of great help for future research in the application scenario of batch integration.\n___\nThank you again for your detailed review! Your insights have been invaluable in aiding us to solidify our work. Should our responses address your queries, we would deeply appreciate your support for our work. \n\nRefs:\n\n[1]  scGPT: toward building a foundation model for single-cell multi-omics using generative AI\n\n[2] LangCell: language-cell pre-training for cell identity understanding\n\n[3] Nicheformer: a foundation model for single-cell and spatial omics", "claims_and_evidence": "The paper claims to propose a multi-scale foundation model that captures and integrates information from macro, micro and gene scale of spatial transcriptomics. The ablation on the three scale is conducted in Table 5. Large performance improvement is shown at the second and fourth row in Tabel 5. However, performance improvement involving micro scale by adding the SE(2) Transformer is relatively limited. The author should include an ablation experiment evaluating the model’s performance when using only macro- and gene-scale features without micro-scale components. Besides, the author requires isolating the effect of the spatial distance matrix through experiments comparing performance with and without this component. The sensitivity of sample rate on the second-time cell encoding mentioned in Sec. 3.2 should be explored. And the effect of incorporating spatial information should also be evaluated in ablation study.", "methods_and_evaluation_criteria": "The evaluation criteria makes sense for the analysis of spatial transcriptomics representation. The downstream tasks about human embryonic structure segmentation, DLPFC layer segmentation, cell type annotation in spatial transcriptomics are validated in the experiments. Accuracy and F1 are reported to evaluate the segmentation and cell type annotation performance. Besides, the performance in Zero-shot Clustering and Visualization and Spatial Deconvolution illustrates the model’s ability  to produce high-quality cell embeddings.", "theoretical_claims": "No", "experimental_designs_or_analyses": "The experimental designs evaluates the model’s performance in various downstream tasks. The experiments on tissue region semantic segmentation evaluate the model’s ability to understand the functional specialization of cells. The author claims the experiments on cell type annotation in spatial transcriptomics confirm that incorporating spatial information can help improve cell type annotation. But the effect of incorporating spatial information should be evaluated in ablation study. Experiments on zero-shot clustering and visualization illustrate the ability of the model to produce high-quality cell embeddings.The model is also is evaluated to be a effective tool for spatial deconvolution. Experiments on spatial transcriptomics imputation shows the model’s ability to infer the uncaptured gene expression levels. The paper claims to propose a multi-scale foundation model that captures and integrates information from macro, micro and gene scale of spatial transcriptomics. The ablation on the three scale is conducted in Table 5. Large performance improvement is shown at the sencond and fourth row in Tabel 5. However, performance improvement involving micro scale by adding the SE(2) Transformer is relatively limited. It’s needed to evaluate what’s the performance when the model involve both Gene and Macro, except for Micro. Besides, the ablation on the effect of the spatial distance matrix is also needed to be evaluated. Also, ablation on the second-time cell encoding mentioned in Sec. 3.2 is needed to be explored. How does the sample rate influence the performance? The effect of incorporating spatial information should also be evaluated in ablation study.", "relation_to_broader_scientific_literature": "First, the author we construct SToCorpus-88M, the largest high-resolution ST pretraining corpus to date, which contains approximately 2,000 tissue slices and 88 million cells and is claimed to be released. This contribution will provide a large dataset for further ST analysis.  Second, the author proposes a multi-scale foundation model to integrate macro-scale tissue morphology, micro-scale cellular microenvironment and gene-scale gene expression profile of spatial transcriptomics, which provides a idea for integrate multi-scale information from ST.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "This paper is clearly written and easy to read. In terms of spatial transcriptome characterization methods, this thesis incorporates macro-, micro-, and gene-scale information and adds positional information, for which the largest spatial transcriptome dataset to date has been created. And the downstream task experiments in this paper are very rich. But the ablation experiment needs to be improved.", "other_comments_or_suggestions": "No", "questions_for_authors": "1. The author should include an ablation experiment evaluating the model’s performance when using only macro- and gene-scale features without micro-scale components. \n2. The author requires isolating the effect of the spatial distance matrix through experiments comparing performance with and without this component. \n3. The details on the second-time cell encoding mentioned in Sec. 3.2 should be written. If a sample rate is involved in the second-time cell encoding, the sensitivity of sample rate should be explored. \n4. The effect of incorporating spatial information should also be evaluated in ablation study.\n5. This paper migrates the single-cell model through a domain adaptation strategy. However, there may be relatively large differences between different ST datasets, such as different technical platforms. How should we further improve adaptability?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "No"}}, {"review_id": "o6R9TUQzkE", "reviewer": "Reviewer_LBxH", "summary": "This paper introduces a foundation model for cell spot representation of spatial transcriptomics. It fine-tunes the pretrained cell embedding (from existing scRNA foundation models) by incorporating the location information via masked feature prediction and noised distance information recovery, as well as involving the proposed virtual macro cells.   A series of down-stream tasks are performed with the pretrained spot embeddings to demonstrate the effectiveness of them.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer LBxH:\n\nThanks for your appreciation and detailed review. We try our best to response the questions below:\n\n>Q1: The ablation study is not sufficient to support the effectiveness of the fine-tuning with the incorporation of distance information.\n\nThank you for your suggestions! We conduct an ablation study on the spatial distance matrix. As shown in the Table below, the results demonstrate that removing the spatial distance matrix significantly decreases model performance. Essentially, the SE(2) Transformer relies on the spatial distance matrix to establish relationships between cells. If spatial distance matrix is removed, the ability of the model to perform intercellular information interactions will be compromised.\n\n|Model|Embryo2-F1| EmbryoCross-F1|\n|-|:-:|:-:|\n|w/o distance matrix|0.721|0.413|\n|SToFM|0.801|0.459|\n\n>Q2: For the Pairwise Distance Recovery (PDR), it is not sure that the cell embedding contains sufficient distance information to make the distance recovery mechanism feasible.\n\n- **Intuitively**, the spatial autocorrelation of cellular expression profiles [1], as well as the ligand-receptor intercellular signaling pathways of neighboring cells can help to recover the original distances from the noisy data.\n- **Experimentally**, the PDR loss can be converged well during our pre-training process. Moreover, following the discussion in Uni-Mol [2] about recoverability of distances, we only add limited noise to the cell coordinates, which does not obscure all spatial location information. This makes this pre-training task more feasible.\n\n>Q3: The proposed model only trained on cell resolution spatial transcriptomics data, and the feasibility of extending to low resolution ST remains unknown.\n\n- **SToFM can be applied to low-resolution ST data without domain adaptation.** In the DLPFC experiment of Section 4.2, we specifically chose the low-resolution 10x Visium data, and SToFM performed exceptionally well in this task, demonstrating that the model can be scaled to low-resolution ST data. \n- Advancements in biotechnology in recent years have continuously improved the resolution of ST data, making the analysis of high-resolution ST data a more valuable direction in the field of bioinformatics [3].\n\n>Q4: Not very sure the effectiveness of the way to formulate the virtual cell, i.e., simple by averaging of the representation and location.\n\n- **Intuitively**, the main purpose of constructing virtual cells is to provide a summary of information from global tissue sections. After clustering by combining expression embedding and location information, each cluster should contain a set of cells that are close in location and have similar expression. Thus, by averaging the embedding and position of the cells within a cluster, it is possible to say \"there is a cluster of cells with similar expression at this location\". We believe this is methodologically sound. The use of average or sum for simple yet effective pooling is also widely applied in fields such as graph representation learning [4].\n- **Experimentally**, our ablation experiments demonstrate that incorporating macro-scale information through virtual cells can effectively improve model performance. In addition, we calculate the similarity of expression embeddings and spatial positions of some virtual cells with each cell in the cluster, as shown in the Table below. \n|Sample ID|Pearson correlation of expression embeddings|Cosine Similarity of positions|\n|-|:-:|:-:|\n|1|0.863|0.804|\n|2|0.871|0.833|\n|3|0.893|0.853|\n\nThe results indicate a high similarity between the expression embeddings and positions of the virtual cell and individual cells within the cluster.\n___\nThank you again for your detailed review! Your insights have been invaluable in aiding us to enhance and solidify our work. Should our responses satisfactorily address your queries, we would deeply appreciate your support for our work. \n\nRefs:\n\n[1] Mapping the transcriptome: Realizing the full potential of spatial data analysis\n\n[2] Uni-Mol: a universal 3D molecular representation learning framework\n\n[3] Methods and applications for single-cell and spatial multi-omics\n\n[4] Graph pooling in graph neural networks: methods and their applications in omics studies", "claims_and_evidence": "The ablation study is not sufficient to support the effectiveness of the fine-tuning with the incorporation of distance information.", "methods_and_evaluation_criteria": "For the Pairwise Distance Recovery (PDR), it is not sure that the cell embedding contains sufficient distance information to make the distance recovery mechanism feasible.", "theoretical_claims": "No theoretical contribution is introduced in this work.", "experimental_designs_or_analyses": "The experiments are extensive and relatively comprehensive.", "relation_to_broader_scientific_literature": "It is an important direction to develop sing-cell rna expression, and there emerge a large body of pretrained foundation model for this purpose. This paper extend to incorporate location information of ST for better spot embedding learning.", "essential_references_not_discussed": "Most essential references are mentioned.", "other_strengths_and_weaknesses": "It is an important direction to pretrain spot representation with the consideration of location information.\n\nThis paper is well written and easy to follow. \n\nThe proposed model only trained on cell resolution spatial transcriptomics data, and the feasibility of extending to low resolution ST remains unknown.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Not very sure the effectiveness of the way to formulate the virtual cell, i.e., simple by averaging of the representation and location.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "30dChcBbOc", "reviewer": "Reviewer_tnYZ", "summary": "The authors introduce SToFM, a single-cell foundation model that incorporates not only the single-cell expression data but also their spatial locations. They propose SToCorpus-88M, one of the largest single-cell pretraining datasets curated to date, and also pretrain their model on the large pretraining dataset. It is demonstrated that SToFM is able to outperform a lot of recent SOTA ST FM such as Geneformer, Nicheformer, and scGPT.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer tnYZ:\n\nThanks for your appreciation and detailed review! We try our best to response your questions. Due to space limitations, we include **tables and references in the anonymous link** https://anonymous.4open.science/api/repo/stofm-rebuttal/file/rebuttal-tnYZ.pdf?v=af96a9f5, and refer to them in the following as *Rebuttal-Table X* and [X], separately.\n\n>Q1: About SToCorpus-88M.\n- **Is it the largest?** To our best knowledge, it is the cell-resolution ST pretraining corpus that contains the largest number of both slices and data points.\n- **Will it be released?** Yes, it will be publicly released.\n- **Recently-published ST datasets.** Thank you! We will add citations to the revised manuscript.\n\n>Q2: Lack of clarity and details.\n\nApologies for the lack of clarity. We promise to **add details to the revised version and release detailed code**.\n-  **Domain-adaptation**\nDomain adaptation performs an epoch in random order on all 88M data points. The loss had converged at the late stage of the first epoch.\n- **How are the sub-slice generated exactly?**\nFirst, we divide the slice into rectangles of the same size. Then, for each rectangle, we merge or split it according to the number of cells it contains. The code will be released for reproducing.\n-  **Number of morphological classes?**\nThey are multi-classifacation tasks.  We have provided experimental details in the *Rebuttal-Table 1*.\n-  **How is the ST imputation done?**\nWe trained a fully connected neural network with cell representations from SToFM to obtain 50 outputs, as a multi-target regression task.\n\n>Q3: Lack of ablations\n\nWe will **add more ablation studies in the revised version**.\n-  **Data efficiency**\nDue to the high computational cost, we pretrain the multi-scale ST representation learning phase using 12.5% and 50% of the data during rebuttal, as shown in *Rebuttal-Table 2*. The results show that reduction in data volume led to a significant decrease in model performance. Considering that the SToCorpus-88M consists of approximately 2,000 ST slices, we believe that reducing the amount of data may reduce data diversity and limit the model's transferability. Additionally, we would like to clarify that, as mentioned in Section 4.1, we conducted 1 epoch and 3 epochs in each of the two training stages.\n-  **SE(2) Transformer**\nWe conduct ablation studies on the PDR loss and the spatial distance matrix, as shown in *Rebuttal-Table 3*. For more analysis, please refer to the response Q1 to Reviewer LBxH. In addition, the effectiveness and efficiency of the SE(2) Transformer are discussed in detail in Uni-Mol[1]. And when incorporating spatial information in ST data, we primarily focus on the interactions between neighboring cells. Therefore, using the distance-based architecture is in line with the data characteristics.\n-  **Masking ratio**\nMasking ratios are set according to the specific data.  Each point in ST data contains high-dimensional features, which increase the data complexity. We had attempted to mask 20% of the cell expressions and perturb another 20% of the cell positions, but found training difficult to converge. In addition, [2] has demonstrated that models are usually robust to mask probabilities when they can converge properly.\n-  **Geneformer initialization**\nIf randomly initialized,  it will incur expensive computational overhead to train a cell encoder from scratch. And due to the lower quality of gene expressions in ST data compared to scRNA-seq data, our pre-experiments indicate that it is difficult to converge in early training. Considering that Geneformer is one of the SOTA models, and  domain adaptation was performed using a large amount of data, we believe there will not be a significant difference if using other SOTA models such as scGPT [3].\n\n>Q4: Batch effect correction.\n\nPlease refer to the response Q5 to Reviewer Rgcj.\n\n>Q5: Catastrophic forgetting.\n\nCatastrophic forgetting may cause the model's suitability for scRNA-seq data to decrease while becoming more suitable for ST data. \n- SToFM is a model specialized for ST data, and we do not recommend users to apply SToFM on scRNA-seq data scenarios.\n- We conduct an experiment on scRNA-seq data. As shown in *Rebuttal-Table 4*, there is almost no drop in performance. This may be because the distribution of gene expressions in ST data and scRNA-seq data is similar.\n\n>Q6: scGPT in Table 1.\n\nApologies for the confusion. The table header is \"ST Pretraining\", and what we want to express is that scGPT did not use gene expressions from ST data during pretraining.\n\n>Q7: Leiden cannot control the number of clusters.\n\nAfter clustering, the number of clusters will be checked. If it does not fall within 20-100, the resolution of Leiden will be adjusted to re-cluster until the cluster number meets the requirement. \n___\nThank you again for your insights which are invaluable in solidifying our work. Should our responses address your queries, we would deeply appreciate your support.", "claims_and_evidence": "Most are well-supported - However, I think the claim that SToCorpus-88M is the largest high-resolution ST pretraining corpus to date needs to be thoroughly validated.", "methods_and_evaluation_criteria": "Yes, the authors validate SToFM on diverse tasks such as ST imputation, morphological segmentation, and etc. and show universally better performance.", "theoretical_claims": "No theoretical claims are made.", "experimental_designs_or_analyses": "I think the explanations on each experiment is quite lacking. For instance, for morphological segmentation, it is unclear how many regions are being segmented. For imputation, it's unclear how SToFM is being used to infer the ST for certain cells. In short, I have doubts that the readers can \"reproduce\" the experimental results just based on what is provided in the paper.", "relation_to_broader_scientific_literature": "I think it is well-placed in terms of broad literature, especially with encouraging results against the SOTA baselines.", "essential_references_not_discussed": "I think maybe recently-published public ST benchmarks (although they are predominantly Visium-based) deserve some references.\n\n[1] Jaume, Guillaume, et al. \"Hest-1k: A dataset for spatial transcriptomics and histology image analysis.\" Advances in Neural Information Processing Systems 37 (2024): 53798-53833.\n[2] Chen, Jiawen, et al. \"STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics.\" ArXiv (2024): arXiv-2406.", "other_strengths_and_weaknesses": "While there are many strengths in this work, as demonstrated with diverse evaluation tasks against many SOTA baselines, there are several factors in this study that prevents me from giving higher score.\n\n**Lack of clarity & details**: The technical details behind SToFM and the experiment details are quite lacking, to the point that it is a bit hard to understand and replicate what the authors have proposed. For instance, in domain-adaptation phase, there are at least 7 different platforms from which the data comes from - Is the domain-adaptation performed on all 88M points? In what sequence? Does each batch randomly sample from 88M point each time? Why only one epoch? \n\nHow are the sub-slice generated exactly? The authors only mention that it roughly contains 1,000 genes per sub-slice and it's probably hard for readers to replicate.\n\nFor the experiments, it's unclear what number of morphological classes the segmentation is done on - Is it binary, due to the use of F1 and acc? But DLPFC typically has at least 5~6 morphological classes. How are the ST imputation done? How are the 327 inputs used to impute for 50 outputs?\n\nIn short, I would cut down verbose explanations on the architecture and paradigm and focus on providing more technical/experimental details.\n\n**Lack of ablations**: Right now, it feels like SToFM is a combination of existing parts (geneformer & SE(2) encoder & masking loss) put together. While I think that is fine and how the field advances, I feel like there is lack of motivation/explanation behind each of the component design choices. To really make strong claims about SToFM, I think the authors need to perform more extensive ablations (other than just cell/micro/macro ablations)\n- Data efficiency: That 88M is very big size is good, but is this really what drives the performance? Especially if the authors are only performing one epoch of pretraining. I would like the authors to try pretraining SToFM with only fractions of data (e.g., 12.5%, 25%, 50%) to see whether SToFM really follows the scaling law and that all 88 M datapoints are needed.\n- SE(2) Transformer: Can we try with different architecture (e.g., without spatial distance) to really show that the distance information is really required? I feel like expression reconstruction alone could be sufficient, but I might be wrong.\n- Masking ratio: The masking ratio is universally 10%. Given that typical use of Masked Image Modeling use higher masking ratio than 10%, I would like the authors to show the effect of this.\n- Geneformer initialization: What if it is randomly-initialized? What if other pretraining weights are used?", "other_comments_or_suggestions": "- How are the authors dealing with batch effect that arises from integrating 7 different sequencing platforms? There wasn't a reference to any efforts for batch effect correction, so I think it's very important.\n- Continual training can easily lead to catastrophic forgetting - Have the authors observed this + What have authors done to prevent any form of collapse?\n- scGPT in Table 1 says it doesn't use expression values - but scGPT uses expression values as well. What did the authors mean to say here?\n- Leiden clustering cannot control for exact number of clusters, i.e., virtual cells in each sub-slice. How do authors control for this?\n- Are the authors planning to release SToCorpus-88M publicly?", "questions_for_authors": "Please see above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Following the recent trend of foundation models, the paper proposes one for spatial transcriptomics. While many of the components are put together from existing work, the reviewers identify two main contributions of the paper: (1) from a technical point-of-view, incorporating distance information and (2) a data contribution, the new SToCorpus-88M dataset. The reviewers initially raised several concerns about missing empirical experiments and ablation studies but these were addressed by the authors during the course of the rebuttal period. As a result, I believe the paper makes a solid contribution to the conference and should be accepted.
Decision: Accept (poster)