SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data via Scalable Conditioned Neural Fields
Authors: David Keetae Park
Xihaier Luo
Guang Zhao
Seungjun Lee
Miruna Oprescu
Shinjae Yoo
Venue: ICML 2025 poster (2025)
Keywords: Spatiotemporal Learning
Scientific Data
Neural Fields
Implicit Neural Representation
Primary area: applications->chemistry_physics_and_earth_sciences
TL;DR: SCENT learns continuous spatiotemporal representations from sparse data, enabling joint reconstruction, interpolation, and forecasting.

Abstract: Spatiotemporal learning is challenging due to the intricate interplay between spatial and temporal dependencies, the high dimensionality of the data, and scalability constraints. These challenges are further amplified in scientific domains, where data is often irregularly distributed (e.g., missing values from sensor failures) and high-volume (e.g., high-fidelity simulations), posing additional computational and modeling difficulties. In this paper, we present SCENT, a novel framework for scalable and continuity-informed spatiotemporal representation learning. SCENT unifies interpolation, reconstruction, and forecasting within a single architecture. Built on a transformer-based encoder-processor-decoder backbone, SCENT introduces learnable queries to enhance generalization and a query-wise cross-attention mechanism to effectively capture multi-scale dependencies. To ensure scalability in both data size and model complexity, we incorporate a sparse attention mechanism, enabling flexible output representations and efficient evaluation at arbitrary resolutions. We validate SCENT through extensive simulations and real-world experiments, demonstrating state-of-the-art performance across multiple challenging tasks while achieving superior scalability.

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data via Scalable Conditioned Neural Fields
David Keetae Park
Xihaier Luo!
Guang Zhao'
Seungjun Lee! Miruna Oprescu Shinjae Yoo
Abstract
et al., 2018), medical imaging (Litjens et al., 2017), and video analysis (Wang et al., 2020). Achieving accurate spatiotemporal learning, however, presents significant chal- lenges due to the presence of complex spatial-temporal de- pendencies, including spatial heterogeneity and temporal non-stationarity, compounded by the high dimensionality of the data. These challenges are further amplified in scien- tific and engineering domains, where datasets are frequently characterized by irregular distributions (e.g., arising from sensor malfunctions) and large volumes (e.g., generated by high-fides sigh-fideiity simulations), introducing further complexities in both computation and modeling.
Spatiotemporal learning challenging due to the intricate interplay between spatial and temporal dependencies, the high dimensionality of the data, and scalability constraints. These challenges are further amplified in scientific domains, where data is often irregularly distributed (e.g., missing val- ues from sensor failures) and high-volume (e.g.. simulations), posing additional com- putational and modeling difficulties. In this paper. we present SCENT, novel framework for scal- able and continuity-informedd spatiotemporal rep- resentation learning. SCENT unifies interpolation. reconstruction, and forecasting within a single ar- chitecture. Built on a transformer- based encoder- processor-decoder backbone, SCENT introduces learnable queries enhance generalization and query-wise cross- coss-attenttoon mechanism to effec- tively capture multi-scale dependencies. To en- sure scalability in both data size and model com- plexity, we incorporate a sparse attention mech- anism, enabling flexible output representations and efficient evaluation at arbitrary resolutions. We validate SCENT through extensive simula- tions and real-world experiments, demonstrating state-of-the -tate-of-the-art performance across multiple chal- lenging tasks while achieving superior scalability.
To address the aforementioned challenges, significant re- search efforts have been dedicated developing solutions from both traditional signal processing and, more recently, machine learning perspectives. Among the emerging ma- chine learning approaches, Implicit Neural Representations (INRs) have garnered increasing attention due to their in- herent flexibility (Sitzmann et al., 2020b; Mildenhall et al., 2020). INRs parameterize data as a continuous function, mapping coordinates to signal values (e.g.. (z, y) - (r, g. b) for images), using neural network. Despite their potential, the adoption of INRs in scientific domains has been lim- ited. This can primarily be attributed to two key challenges: scalability and generalizability.
To bridge this methodological gap, we propose SCENT (Scalable Conditioned Neural Field for SpatioTemporal Learning), a novel framework designed to address the lim- itations of existing INR approaches. SCENT leverages Transformer-basee ture to efficiently process volume spatiotemporal data, demonstrating strong scalability with respect to both dataset size and model parameter count. Furthermore, SCENT incorporates trainable query mechanisms to enhance gen- eralizability, cireumventing the computational overhead as- sociated with existing strategies such latent optimiza- tion (Dupont et al., 2022) or meta-lcarning (Chen & Wang, 2022).
1. Introduction
mncoder-processor-dde
architec-
Spatiotemporal learning focuses on modeling and interpret- ing data that exhibit variations in both space and time. This approach crucial for analyzing intricate real-world phe- nomena where spatial structures are inextricably linked with temporal dynamics, including applications such climate modeling (Reichstein et al., 2019), traffic forecasting (Li
'Computing and Data Sciences, Brookhaven National Labora- tory *Comell University, Cornell Tech. Correspondence to: David Kectae Park <dpark1 @ bol.g
Overall, the major contributions of our work include:
We offer a unified framework for spatiotemporal leamn ing, capable performing joint interpolation, recon- struction, and forecasting.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
(a) Simulation for challenges
(b) Real Data: AirDelhi
GT
noise
malfunctions
sparsity
moving sensors
coarse
fine-grained
Input
Input
300
200
Output
Output
100
Figure 1. Challenging data scenarios motivating this study. (a) Learning continuous ground truth signal (GT) given noisy, mal- functioning, sparse, or moving sensors daunting challenge. However, these challenges common in scientific data, including AirDelhi (Chauhan et al., 2024) data which measure the particulate matter levels (PM2.5) from moving vehicles.
We empirically demonstrate that SCENT is scalable with larger models and datasets.
Problem Statement. We consider a spatiotemporal dis- crete observation denoted as u where X v, E R represents the spatial coordinates, and t E R denotes the sampled time. Given a set Ni input observations mea- sured at time 4,, we define:
We conduct extensive experiments to evaluate the effi- cacy of the proposed SCENT framework, demonstrat- ing superiority over state-of-the-art (SOTA) meth- ods in range of spatiotemporal learning tasks.
fu Uh = [xx "xo
XN,
2. Problem Setting
Our objective is to develop model F capable of learning the underlying continuous spatiotemporal function from these discrete Ni observations, enabling it to predict M target outputs at a different time to:
Background. Spatiotemporal learning is critical field for modeling data that evolves across space and time, with diverse applications in climate modeling (Reichstein et al., 2019), medical imaging (Litjens et al., 2017), and biochem- istry (Jumper et al., 2021). While traditional methods like RNNs and LSTMs (Hochreiter & Schmidhuber, 1997) ad- dress temporal sequences, they often struggle with long-
G = F (U*).
where the target outputs are given by:
range dependencies and spatial correlations, limitations that Transformer- based architectures like ViT (Dosovitskiy et al., 2021) and TimeSformer (Bertasius et al., 2021) have be- gun to overcome. Implicit Neural Representations (INRs) offer powerful, continuous approach to data modeling, mapping coordinates to function values using a neural net- work for compact encoding and resolution-agnostic inter- polation (Sitzmann et al., 2020b; Mildenhall et al., 2020). However, standard INRs lack generalizability, requiring retraining for new data instances; this has led to the devel- opment of Generalizable INRs (GINRs) leveraging meta- learning (Sitzmann et al., 2020a; Chen et al., 2021; Dupont al., 2022: Bauer et al., 2023). Conditioned Neural Fields (CNFs) are closely related to GINRs, providing powerful, single-step framework for solving PDEs by learning tinuous function representations conditioned on input pa- rameters, aligning with GINRs' goal of learning continuous functions and addressing challenges in high-dimensional and sparse-data problems (Wang et al., 2024: Li et al., 2022). detailed literature review provided in the Appendix
U = "
The specific task performed by (F) is determined by the relationship between the input and output spatiotemporal coordinates:
Reconstruction. If (1x)) (Ix)) the model is tasked with reconstructing the same ob- servations it was provided.
Spatiotemporal Interpolation.
((x) *
((x N. fo). the model estimates the function at novel spatiotemporal locations.
Forecasting. special case of interpolation where lo > 4, requiring the model to predict future values.
Our goal is to develop model (F) capable of jointly per- forming reconstruction, interpolation, and forecasting of complex scientific data given arbitrary spatiotemporal coor- dinates (x, y, 2,
2

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
time coordinates f,
Time-Targeted Spatial Encoder
Learnable queries
Temporal Warp
Fourier feature encoding
IR gMx
Processor
space coordinates x,y
IRN(xx
input (/4
Context
Embedding
RR RRx
Network
(a)
self- attention layers
Linear Projection
K,
sparse attention layer * Neenc
Roroc
Cross Attertionn
flattening
R
space coordinates x,y
Input U at 1==
Space interpolation
time coordinates
(b)
R Nox1
=0=
SCENT
output t/fa
Calbration Network
Fourier fesalure encoding
R Noxt
3=
fo=00
Reconstruction
Forecasting
sparse
attention
layer
K.V
Ridec
Cross Attention
R Mxl
Time-Conditioned Decoder
Figure 2. SCENT overview. (a) Detailed architecture of SCENT is illustrated. Our unique contribations are drawn with red boxes. Specifically, we introduce time coordinates both encoder and decoder for learning continuous time representations. Also, introduce Context Embedding Network and Calibration Network for improved spatial encoding and decoding, respectively. Thees Vlymee and fide denote the number of layers in encoder, processor, and decoder respectively. (b) At the inference stage, single SCENT model is capable to jointly perform reconstruction, spatiotemporal interpolation, and forecasting.
equations (Lee, 2022; Lee & Oh, 2024; Serrano et al., 2024; 2023; Li et al., 2023), where the architecture is particularly beneficial, as these models flatten the input and treat each sample point as an individual token. Additionally, this ar- chitecture facilitates learning moatinuity-informe represen- tations using inducing point learning within cross coss-attention layers (Jaegle et al., 2022: Lee & Oh, 2024). Building these advancements, we evaluate the models' potentials in complex real-world scientific problems, often characterized by intricate noise patterns and irregular sensor distributions. In remaining subsections, we describe notable develop- ments on top the existing architecture, displayed by red-colored objects in Fig. 2(a).
unrolled
(a)
T--
unrolled
(b)
warp
Figure 3. Warp-unrolling forecasting (WUF). (a) Conventional forecasting includes single-step unrolling which accumulates error. A dimming blue color is used to represent the increasing error. (b) WUF helps mitigate error accumulations caused by extensive unrolling steps.
3.2. Vime-Targete Spatial Encoder
3. SCENT
The encoder processes an input data U [u uz ... 2/22% representing N, samples from the space coordinate { given time 4, Here, {x, 1 can be structured grid irregular mesh. Using a cross-attention mechanism, the encoder transforms Uth into fixed-size set of tokens Z = {21, 22. 2M ) where M is the number of latent trainable query tokens.
{
3.1. Soniddr-Procsssr--Deee Framework
The pondderproceesor-deee
architecture is proposed general framework that scales linearly with input and out- put sizes, overcoming the quadratic scaling limitations of transformers with input sequence length (Jaegle et al., 2022). Prior works have leveraged this architecture to learn condi- tioned neural fields (CNFs) for solving partial differential
3

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
The encoder Eg is then expressed as:
it as a reference state for predicting subsequent time steps (Fig. 3). This ensures that at any given time state, only minimal number of prediction steps are required, thereby reducing the potential accumulation of errors. We denote this strategy as warp-unrolling forecasting ( (WUF) and use this for benchmark datasets (Table 2).
Eg: (U%, 2 ko) ZMe
where & is the input time, and fu is the targeted output time. Including to enhances attention to relevant information, lead- ing to improved performance (Table 3).
3.4. Time-Conditioned Decoder
Specifically, we encode the spatiotemporal coordinates {x,}ī=1, to using Fourier features (Tancik et al., 2020). Fourier features map input coordinates to a higher- dimensional space using sinusoidal functions of varying frequencies, enabling models to capture fine-grained and periodic variations in the data. Mcanwhile, the input sam- ples u2 are separately embedded using a linear projection layer with parameters frozen. This effectively increases the dimensionality of function value representation u empir- ically enhancing performance. Both Fourier features and encoded samples are concatenated, and sent through Con- text Embedding Network (CEN) which consists of sparse self-attention layers to enrich context encoded in individual tokens. Within CEN, each token attends to randomly sub- sampled S tokens where S < N₁. As the underlying data is in continuous field, sparse attention is an efficient way to encode global contexts to individual tokens. The final latent representation then summarized through cross attention against the learning queries which consist of M tokens, where M < N₁ We justify the benefits of the linear projection and CEN by the ablation study (Table 3).
The decoder Dy: Z - U utilizes the latent processed tokens (tt)) to approximate the function values at to. To this end, we apply Fourier features encoding to x and to. and use the resulting queries for tross-attention against ((to). Then we apply fldee sparse self-attention layers (Calibration Network, abbreviated as CN) to calibrate spatiotemporal de- coding. During training U includes No available samples at fo- During inference, however, (tt) may be evaluated on arbitrary points in x.
4. Experiments
We conduct evaluations using a diverse set of baseline mod- els, encompassing state-of-the-art regular-grid methods such as FNO (Li et al., 2020), adaptable transformer architectures represented by OFormer (Li et al., 2023), as well as neu- ral field based approaches like DINO (Yin et al., 2023), CORAL (Serrano al., 2023) and AROMA (Serrano et al., 2024). All training and evaluations are conducted using mean squared error (MSE), relative MSE (Rel-MSE), and root MSE (RMSE). RMSE is specifically used to measure PM2.5 levels for AirDelhi datasets (Section 4.1.3). Rel-
3.3. Temporal Warp Processor
MSE is defined as:
We define the Time Warp Processor (TWP) for learning continuous temporal dynamics, denoted as Pe Z Z/r. where to = f; + At, At E [0, ty] and ty represents hyperparameter for the maximal time horizon used during training. Depending on At, TWP can either perform input reconstruction (At = 0) or forecasting (At > 0), allowing joint reconstruction and forecasting with a single model. This flexibility also enables the of input-output pairs sampled at non-integer time intervals, which is particularly useful for spatiotemporal data with time-varying sampling rates (Chauhan et al., 2024; Nie et al., 2024).
2i=1 ( -U
Rel-MSE
(I)
Most experiments were performed on single NVIDIA H100 80GB HBM3 GPU. The largest model variant used in the scalability evaluation (Fig. 4) required distributed training across eight of them. Details of the algorithm can be found Appendix B, while information on the dataset and training procedures is provided in Appendices through G.
4.1. Datasets
Novel Unrolling Strategy. TWP can be leveraged to min- imize accumulating prediction errors during long-horizon forecasting, which empirically leads to improved perfor- mance over baselines (Section 4.5). Previous models have been reported to struggle with long-term forecasting due to rapidly accumulating errors during inference and un- rolling (Serrano et al., 2024). While we adhere to the stan- dard practice of training models for next-state prediction, we do not perform one-step unrolling for the entire forecast- ing time steps. Instead, we utilize time warping, advancing directly to the time horizon L, once to 1 > the using
1.1.1. BENCHMARK NAVTER-STOKER DATASETS
We use three benchmark Navier-Stokes datasets (Li et al., 2020), each corresponding to different viscosity coefficients. These are designed to model the dynamics of viscous and incompressible fluid governed by the 2D Ntoker-Stokes equation in vorticity form on the unit torus. The Navier- Stokes 1 x 10-3 (NS-3) dataset (Yin et al., 2023; Serrano et al., 2023), with a viscosity coefficient V = 1 x 10-3, includes 256 training trajectories and 32 testing trajectories. NS-3 models relatively slower fluid dynamics with a time
4

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
(a) Scalability on model size
(b) Scalability on dataset size
0.38
7.4M
FNO SCENT
0.36
3M
test error (relative MSE)
test error (relative MSE)
12.3M
17.80 7.8M
0.34
0.32
7.1M
12.5M
4M:4M
10M
3M
5M
0.30
25:714
124M
192M
10M
41 4M
0.28
328M
0.30
SM
0.26
193M
10M
0.28
0.24
1000
300
500
600
model width (d)
model width (d)
Figure 4. Scalability evaluations. (a) Texts next to each circle are the number of model parameters, and circle size is also proportional to it. Red dotted lines are scalability trends derived with exponential functions for comparisons. (b) Colors indicate training runs with identical model sizes. Texts next to each cirele are the number of training instances, and the circle size is also proportional to it.
horizon of first 20 steps beyond the initial condition. The Navier-Stokes 1 X 10-4 (NS-4) dataset, with a viscosity co- efficient V = 1 10-1, is more turbulent variant. Lastly, the Navier-Stokes 1 x 10 (NS-5) dataset, with the lowest viscosity coefficient y = 1 x 10-6, represents the most turbulent fluid dynamics. Both NS-4 and NS-5 contain 1000 training and 200 testing trajectories and maximum time steps of 30 and 20 steps, respectively. For all three datasets, use the vorticity to 10 as the initial condition for testing. These datasets provide a range of viscosity con- ditions, making them suitable for studying fluid dynamics across different turbulence levels.
malfunctioning (Nathaniel et al., 2023). We empty out three square areas with different sizes to simulate lost sensors. Randomly sparse sensors (S4): sensor locations may be randomly placed, such as for remote sensing (Myneni et al., 2001) in climate science. We randomly mask out 50% of the data to simulate the sparcity. Dynamically moving sensors (S5): A more challenging scenario arises when sampling locations are sparse and dynamically shift- ing (Chauhan et al., 2024; Nie al., 2024). To simulate this, we randomly select twenty non-overlapping 10 x 10 square regions as input locations. The output regions are then trans- lated by (h, v), where h, u E [-10, 10) denote horizontal and vertical shifts. Regions crossing image boundaries are mirrored to maintain continuity.
4.1.2. SIMULATED LARGE-SCALE COMPLEX DATASETS
We introduce a new class of datasets to specifically sim- ulate complex yet common real-world data scenarios as visualized in Fig. 1(a). This simulated dataset, derived from the Navier-Stokes equations, is designed to be significantly larger than benchmark datasets (Section 1.1.1), incorpo- rating real-world challenges such as sensor noise, missing values, and dynamically moving or reconstructed sensor locations.
4.1.3. REAL DATA 1: AIRDELHI
The AirDelhi (Chauban et al., 2024) dataset offers a compre- hensive collection of fine-grained spatiotemporal particulate matter (PM) measurements from Delhi, India. To address the limitations of static sensor networks, the researchers mounted lower-cost PM2.5 sensors on public buses through- out the Delhi-NCR region. The dataset includes PM2.5 measurements across various locations and times, with data collected at granularity of 20 samples per minute. The dynamic nature of the data, with sensors moving along pre- defined bus routes, introduces challenges such as sparse and temporally varying measurements. This necessitates the development of models capable of handling sparse and dy- namically moving sensors. Here we use three data variants for model performance comparisons.
Concretely, we introduce five datasets. While each variant represents different data challenges, they consist of the same underlying ground truth. + Ground truth (S1): We utilize the Navier-Stoker equation with viscosity and boundary con- ditions to simulate highly complex and fast diffusion dynam- ics (Appendix E). This setup ensures that our simulations capture the intricate interactions of fluid motion, enabling more realistic representation of challenging spatiotemporal processes. Noisy sensors (S2): multiplicative noise is sam- pled (Nartasilpa et al., 2016) and applied to SI following: u' (x, u(x,t t)- 17(x. t), where 77(x,)) t) ~ (( .), u is an input sample, is sample-level noise, and x = (ar, 2) and t denote the spatial and temporal coordinates respec- tively. We use μ ==1 and σ = 0.2. Sensor malfunctions (S3): scientific sensors are often temporarily unavailable or
This dataset was originally introduced as a benchmark for evaluation. Concretely, this data is collected between November 12, 2020, and January 30, 2021. In this data, initial days are omitted due to limited sample data and fewer instruments on buses. Also excluded are nightly data between 10:00 PM and 5:30 A.M, as buses remain stationary at bus depots during this period. The geo-
AirDelhi Benchmark (AD-B):
5

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
(a)
Input (t))
delta
GT
Pred
(ta)
GT
SCENT
FNO
AROMA
CORAL
OFORMER
(b)
100
80
Input (t))
delta
60
40
20
400
Pred 1
300
GT
(t,)
200
100
SCENT
FNO
AROMA
CORAL
OFORMER
Figure 5. Qualitative comparisons for forecasting, (a) Using partial input from the S5 dataset, each pretrained model is assessed the full mesh (GT), performing both joint forecasting and spatial interpolation. (b) Models forecast PM2.5 on the spatiotemporal locations where measurements are available (GT). 'delta' (upper rows) represents the absolute difference between each prediction and GT.
graphical region divided into km2 spatial grids, which further segmented into spatiotemporal cells with 30- minute time interval. The average of all samples within each spatiotemporal cell is calculated to determine repre- sentative PM values. We use the 'AB' and "CP" sets as training and test datasets, respectively. This dataset features differing numbers of samples for input (IV;) and output (No),, which demands a high degree model flexibility. Notably, SCENT stands out as the only model capable of naturally handling variable N; and No. AirDelhi Temporal (AD- T): this dataset still uses the 1 km grid, but increases time resolution from 30 minutes to finer minute, and averag- ing all samples within each spatiotemporal cell for repre- sentative PM values. This increases available data instances makes data spatially more sparse and thus challenging to predict. Unlike the original dataset, where the train-test split was based specific days, randomly shuffle all the data before splitting it into training and testing sets. This approach ensures there is no distribution shift between the training and test data. AirDelhi Fine-grained (AD-F): this dataset features a 0.02 km² spatial grid with a 1-minute temporal resolution. The high granularity of the spatial res- olution makes it nearly continuous, enabling the evaluation of models in learning continuous representations effectively.
4. 1.4. REAL DATA 2: KUROSHIO PATH
The Kuroshio current, originating from the North Equatorial Current (NEC) and flowing northward along the eastern side of the Philippine Islands, is the world's second-largess warm current. Accurately predicting its path is crucial because its variations significantly affect the exchange of water masses and heat between the North Pacific subtropical and subarctic circulations. We use 50-year records from the China Ocean Reanalysis (CORA) (Han et al., 2013) as our benchmark and follow the data processing guidelines established by Wu al. (2023). CORA provides daily oceanographic reanalysis data for the Kuroshio current spanning 50 years (January 1958- December 2007).
4.1.5. REAL DATA 3: RAINFOREST NOWCASTING
We use the RY product from the German Weather Service (DWD), a quality-controlled rainfall composite km x km spatial and 5-minute temporal resolution. Data from 2012 2016 used for training, and 2017 for testing. The task is predict rainfall fields for future timestamps E [5, 10...., 60) minutes, given four historical fields. Following RainNet (Ayzel et al., 2020), we use 173,345 43,456 instances for training test splits. We downsample the original 900 900 resolution to 64 64.
6

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
4.2. Robustness to Data Challenges
to
A robust model should reconstruct continuous fields from sparse, noisy data while capturing temporal dynamics. To evaluate this capability, assess forecasting performance across five challenging datasets (S1-S5, Fig. (()) using four representative baseline models. All models are trained with supervision on next-state prediction. Since FNO lacks mesh independence, we modify the original architecture to accommodate our sparse and irregular dataset. Sparse data variants are zero-padee zero-padded align with a regular grid representation for input processing. To prevent the model from trivially predicting zero values, data mask is applied to the output during loss computation. The time horizon (fx) is set to 3 for training. This implies that during training, the time step (At) is randomly selected from [0, 3), allowing the model learn time- continutt given varying temporal intervals well reconstruction (At = 0). Each training trajectory consists of 24 time steps, and total of 100k trajectories are used. Training is conducted using Rel-MSE as the supervision loss, a cosine learning rate scheduler over 50k iterations, and a batch size of 256. During validation, Al is fixed to
Inputs
Interpolation algorithms
Neural fields
G
Elo
Nearest
RBF
CORAL SCENT
GT
Mean
5,45x10-1
4.00x10-1
7.78x10-1 3.53x10-1 Ral-MSE
Figure 6. Joint reconstruction, interpolation, and forecasting. Given dataset S5 inputs shown on top, neural fields are tested for reconstnectoo//tteeppaaaia (t:) and forecasting at continuous time (fs., fo). For comparison purposes, interpolation results from mearest-neighhorhond and RBF algorithms are shown on the left. 'Mean Rel-MSE' denotes average performance at Eso evaluated with the validation datasct.
Results. Table presents forecasting performance compar- isons. SCENT consistently outperforms all baseline models on the simulated datasets, showcasing its resilience in chal- lenging environments and its ability to effectively capture spatiotemporal patterns. As expected, performance gener- ally degrades in more challenging environments, reflecting the increased difficulty of learning from sparse or noisy data. Surprisingly, FNO performs competitively despite its lack of mesh independence, likely due to its inherent bias toward learning low-frequency components, allowing it to maintain relatively strong performance.
sizes and shows better linear scalability, whereas FNO dis- plays clear convergence pattern. Additionally, Fig. 4(b) indicates larger datasets improve forecasting performance, the model effectively leverages additional samples to capture complex underlying patterns from sparse sensor data.
4.4. Learning Spatiotemporal Continuity
Our primary objective is to represent spatiotemporal field conditioned on a sparse input, as outlined in Section 2. An ideal model should, given an input at 4, be capable of both reconstruction and spatial interpolation at ty as well as forecasting at any continuous time within the given time horizon. To evaluate this capability, we assess and com- pare model performance, with particular focus on learning time-continuous representations. Specifically, given S5 data input U*, we infer the data field at three time points, t,, Eio, to. where f.o (t, + t.)/2 denotes the midpoint time. Reconstruction performances at ioo are examined against ground truth. We downsample each trajectory by a factor of two, reserving the rest for evaluating time conti- nuity in learning. We compare the results against CORAL, which also learns time-continuous conditioned neural fields (Table 4). Additionally, we evaluate our model against de- terministic interpolation methods, namely nearest-neighhor and radial basis function (RBF) interpolation. Unlike our approach, these methods require data at both t, and to to interpolate the spatiotemporal full mesh effectively.
4.3. Scalability Analysis
Scientific data volumes are rapidly increasing (Yu et al., 2023; Kitamoto al., 2023; Kaltenborn et al., 2023), reach- ing the exabyte scale in cases such as ATLAS (Peters & Janyst, 201 1). To assess scalability, we analyze model and dataset size variations, examining both model width (i.c., latent dimension) and depth (i.e., number of layers). To our knowledge, this is the first systematic scalability study of continuity-informtn architectures (c.g., INRs, CNFs), and we compare against FNO by varying model width to assess targe-scale adaptability. After constructing SCENT mod- els of varying sizes (Appendix F), we design corresponding FNO models with approximately matching parameter counts for fair comparison.
Results. Fig. 4(a) shows both FNO and SCENT exhibit decreasing Rel-MSE as model size increases. However, SCENT consistently outperforms FNO across all model
7

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
Table 1. Forecasting performance comparisons simulated (Rel-MSF) and real datasets (RMSE).
SIMULATED CHALLENGING ENVIRONMENTS
REAL DATA (AIRDELHI)
MODEL
S3
S4
AD-B
AD-T
AD-F
FNO
1.27 x 10-2
2.10 10-1
2.81
10-1
2.35 x 10
3.77 X 10-
48.79
55.04
55.66
OFORMER
1.63 x 10
2.57 x 10
3.30 10
3.00 x 10
6.17 x 10
70.62
57.00
54.58
CORAL
4.12 x 10
4.83 10
4.69 10
4.89 10
9.06 x 10
60.51
55.26
48.41
AROMA
1.29 x 10-1
2.38 x 10
2.83 x 10
6.67 x 10
5.25 x 10-1
40.78
63.06
47.49
SCENT
2.51 10
2.08
10-1
2.70
10-1
2.28 10
3.26
10
44.20
53.24
45.35
Table Long-term forecasting performances on Navier-Stokes benchmark datasets. MSE is used for NS-3 (for fair comparisons against reported prior arts), while Rel-MSE is used for NS-4 and NS-5.
4.5. Benchmark Performance and Model Ablations
In this section, we highlight the key innovations in our algo- rithm, compare against popular benchmark datasets, and present ablation studies on the SCENT architecture. eval- uate SCENT's ability to perform extended time forecasting, we test it on NS-3,4,5 (Section 4.1.1) using an unrolling approach. Specifically, all models are trained with super- vision on next-state prediction. At test time, we unroll the dynamics following the WUF framework, as illustrated in Section 3.3. Additionally, we conduct an ablation study on S5, systematically removing our key architectural compo- nents - CEN, CN, Proj, and TT (Fig. 2) - to assess their individual contributions performance.
BENCHMARK DATASETS
MODEL
NS-3
NS-4
NS-5
FNO
1.55 10 10-4
1.53 X 10
1.24 10-1
DINO
2.51 x 10
7.25 x 10
3.72 x 10
CORAL
5.76 10-4
3.77 10
3.11 x 10
OFORMER
7.76 10-3
1.36 x 10
2.40 x 10
GNOT
3.21 10
1.85 x 10
1.65 x 10
AROMA
1.32 x 10
1.05 x 10
1.24 x 10
SCENT
7.78 x 10
1.03 x 10
1.17 X 10
Table 3. Ablation study for SCENT dataset S5. Vindicates activated modules. Abbreviations: CEN=Context Encoding Net- work; CN=Calibration Network; Proj=linear projection; TT=Time- Targeted: whether to provide lour for the TTSE.
Results. Table 2 presents the results of ferm- forecast- ing performance across the benchmark datasets. SCENT outperforms all baseline models across all datasets, which attribute to WUF, fundamentally enabled by the time- continuity learned by the model. This advantage is particu- larly evident in the NS-3 3 dataset, where the fluid dynamics are relatively slower, hence error accumulation during next- state unrolling is more pronounced. On NS-4 and NS-5, SCENT and AROMA achieve comparable performances. Our ablation study in Table 3 highlights the contributions of individual architectural modifications introduced in SCENT. The performance gap relative to the best-performing model, referred to as contrast, demonstrates that all four compo- nents play a crucial role in the model's effectiveness. No- tably, performance deteriorates significantly when two or more modules are deactivated, with Rel-MSE increasing by 67.8% when all modules are used.
CEN
CN
PROJ
TT
REL- MSE
CONTRAST
/
3.26 x 10
X
1
/
/
4.02 10-1
+23.3%
X
/
V
3.64 x 10
+11.0%
V
X
V
3.12 x 10-1
+1.9%
AXX
X
3.40 x 10
+1.29%
X
V
4.61 x 10
+41.4%
V
4.82
10-1
+47.8%
X
X
X
X 5.47 x 10-1
+67.8%
Results. Fig. 6 presents the reconstructed results for given input. Notably, SCENT accurately interpolates and re- constructs all three time points, closely matching the ground truth (GT). While CORAL supports time interpolation. it inherits suboptimal performance from training with the com- plex S5 dataset (Table I), resulting in an inaccurate recon- struction of fields across different time points. Interpolation using radial basis functions (RBF) yields reasonable results; however, is important note that the inputs for interpola- tion methods and neural fields differ. Neural fields operate at disadvantage as they are provided only with the initial time state. To further evaluate time continuity, we report the mean reconstruction eтror against the GT at f.o on the full test dataset, with the quantitative results aligning well with the visual assessments.
4.6. Forecasting Performances on AirDelhi
We evaluate whether the superior performance observed in previous experiments extends to the more complex AirDelhi dataset. Similar to dataset S5, AirDelhi features sparse sen- with locations that vary across time, posing a significant challenge for spatiotemporal learning. Strong performance on this dataset would indicate that the model effectively infers the PM2.5 distribution from sparse observations and accurately predicts the future diffusion of particulate matter. While we use Mean Squared Error (MSH) as the training
8

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
RainNet
Relative Improvement (9)
Improvement
0.5
RMSE (mm//h)
RMSE (*)
1::0
0.4
0.15
0.3
EOF LSTM
CEEMIDAN LITH (TE)
EOF SEEMDAN LSTM (FE)
4.1
SCENT (NN)
10
100 120
Lead Time (days)
Lead Time (mins)
Figure 7. Kuroshio Path: RMSE (") comparisons at various lead Feature engineering, NN End-to-end neural networks, EOF = empirical orthogonal functions, CEEM- DAN = complete ensemble empirical mode decomposition with adaptive noise, LSTM long short-term memory.
Figure 8. Rainforest nowcasting: comparing RainNet and SCENT performances at varying
times. Abbreviations:
lead times. Green line corresponds to the relative gain of SCENT against RainNet baseline measured in percentage improvement.
baselines experiencing significant degradation.
loss, we adopt Root Mean Squared Error (RMSE) for evalua- tion to better capture the physical significance of the model's performance.
4.8. Rainforest Nowcasting
SCENT trained with a forecast horizon t, = 60. We report RMSE (mm h-!) Figure 8. SCENT consistently outperforms RainNet across all lead times, with the rela- tive improvement more salient with larger lead time. We attribute this gain in part to SCENT's ability to train with variable target times to, which serves as form of data augmentation.
Results. Table 1 compares forecasting performance across AirDelhi datasets. On AD-B, where the goal is to pre- diet PM2.5 values at sparse locations given three days of observations, SCENT ranks second after AROMA, likely due AROMA'S diffusion backbone effectively filtering noise. However, the qualitative results in Fig. 5(b) high- light SCENT's effectiveness in capturing and expressing PM2.5 levels. All of the five models outperform previously reported performances on AD-B (Appendix K). For larger and finer datasets (AD-T and AD-F), SCENT outperforms all baselines, demonstrating its strength in learning contin- uous representations and forecasting. Additional results (Appendix Fig. J) indicates that SCENT better captures the PM2.5 distribution.
5. Conclusion
SCENT (Scalable Conditioned Neural Field for Spatiotem- poral Learning) addresses the challenge of reconstruct- ing and forecasting spatiotemporal fields from sparse and noisy data. Through extensive evaluations, we demonstrate SCENT's superiority in learning continuous space-time rep- resentations, outperforming baselines in diverse forecasting and reconstruction tasks. SCENT the first single-step training model for learning continuous spatiotemporal rep- resentations, eliminating multi-stage optimization bottle- necks. Its scalability makes it suitable for large-scale ap- plications in geophysics, astronomy, epidemiology, and nu- clear physics (Reichstein et al., 2019; Gabbard et al., 2022; Massucci et al., 2016; Pata et al., 2024). Future work will focus on expanding SCENT's adaptability to extreme-scale datasets and real- welldworld deployments.
4.7. Longterm Forecasting on the Kuroshio Path
We compared SCENT against four baseline methods from Wu al. (2023) in 120-day Kuroshio path predic- tion experiment, training 40 years of data (1958-1997) and testing on the subsequent 10 years (1998-2007). We measured RMSE against true latitude at 10-day intervals. During training, SCENT is used to predict the Kuroshio path terms of latitude (ranging from N N to N) for a fixed forecast horizon. As shown in Fig. 8, while CEEMDAN- based feature engineering (FE) variants performed com- petitively, SCENT achieved second- best performance be- yond 50-day lead time. The FE methods' strength might stem from the limited dataset size, but our scalability study (Fig. 4) suggests SCENT could outperform them with more data. Importantly, SCENT maintained stable performance even as the lead time increased, sharp contrast to other
Further research will also explore enhancing SCENT's ca- pabilities for high-frequency phenomena, often challenging for neural fields. We will investigate integrating multimodal data sources, allowing SCENT to leverage diverse informa- tion for robust, accurate spatiotemporal modeling. We will also tackle multi-scale training problems, enabling SCENT to capture both fine- mne-grained details and large-scale trends within complex spatiotemporal systems.

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
Acknowledgements
Dosovitskiy, A., Beyer, L.,, Kolesnikov, A., Weissenborn
D., Zhai, X., Unterthiner, 'T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021.
This work was supported by the U.S. Department of Energy (DOE), Office of Science (SC), Advanced Scientific Com- puting Research program under award DE-SC-0012704 and used resources of the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility using NERSC award NERSC DDR-ERCAP0030592.
Dupont, E., Kim, H., Eslami, S. M. A., Rezende, D. J., and Rosenbaum, D. From data to functa: Your data point is a function and you can treat it like one. In Interna- tional Conference on Machine Learning, pp. 5694 5725. PMLR, 2022.
Impact Statement
This paper introduces robust, scalable, and unified machine learning approach for understanding and predicting complex spatiotemporal dynamics found in critical scientific datasets. Such advancements hold the potential to enhance predictive insights into complex systems and advance fundamental scientific knowledge.
Gabbard, H., Messenger, C., Heng, I. S., Tonolini, F., and Murray-Smith, R. Bayesian parameter estimation us- ing conditional variational autoencoders for gravitational- wave astronomy. Nature Physics, 18(1):112-117, 2022.
References
Han, G., Li, W., Zhang, X., Wang, X., Wu, X., Fu, H.
Zhang, X., Zhang, L., and Li, D. A new version of regional ocean reanalysis for coastal waters of china and adjacent seas. Advances Almospheric Sciences, 30: 974-982, 2013.
Aad, G., Abbott, B., Abdallah, J., et al. The atlas experiment at the cern large hadron collider. Journal of Instrumenta- tion, 3:S08003, 2019.
Ayzel, G., Scheffer, T., and Heistermann, M. Rainnet vl. O: convolutional neural network for radar-based precipita- tion nowcasting. Geoscientific Model Development, 13 (6):2631-2644, 2020.
Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural Computation, ((3):1735-11 1780, 1997.
Jaegle, A., Borgeaud, S., Alayrac, J.-B., Doersch, C., Ionescu, C., Ding, D., Koppula, S., Zoran, D., Brock, A., Shelhamer, E., Henaff, O. J., Botvinick, M., Zisser- man, A., Vinyals, O., and Carreira, J. Perceiver IO: general architecture for structured inputs & outputs. In International Conference on Learning Representations, 2022. URL https:ttt//.......t.m /tpp/reeeeeeeeeeeeeeeeeeeeeeeeee net/ forum? id id=fILj7WpI-g
Balcan, D., Colizza, V., et al. Multiscale mobility networks and the spatial spreading of infectious diseases. Pro- ceedings of the National Academy of Sciences, 106(51); 21484-21489, 2009.
Bauer, M., Dupont, E.,, Brock, A., Rosenbaum, D., Schwarz, J. R., and Kim, H. Spatial functa: Scaling functa to imagenet classification and generation. arXiv preprint arXiv:2302.031133 2023.
Jumper, J., Evans, R., Pritzel, A., et al. Highly accurate protein structure prediction with alphafold. Nature, 596: 583-589, 2021.
Bertasius, G.. Wang, H., and Torresani, L. Is space-time attention all you need for video understanding? In In- ternational Conference on Machine Learning (ICML), 2021.
Kaltenborn, J., Lange, C. E. E., Ramesh, V., Brouillard, P., Gurwicz, Y., Nagda, C., Runge, J., Nowack, P., and Rolnick, D. Climateset: model dataset for machine learning. In Thirty-eeeenth Conference Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https: / //openeeiiww net for umru ddei2212=2222222
large-scale climate
Chauhan, S., Patel, Z. B., Ranu, S., Sen, R., and Batra, N. Airdelhi: fine- aneengrinedd spatio-temporal particulate matter dataset from delhi for ml based modeling. Advances in Neural Information Processing Systems, 36, 2024.
Chen, Y. and Wang. X. Transformers as meta-learners for implicit neural representations. In European Conference on Computer Vision, pp. 170-187. Springer, 2022.
Kitamoto, A., Hwang, J., Vuillod, B., Gautier, L., Tian, Y.. and Clanuwat, T. Digital typhoon: Long-term satellite image dataset for the spatio-temporal modeling of tropi- cal cyclones. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Bench- marks Track, 2023. URL https: /ttp:////p/e.....//// net/ met//ooum?2 Sdn.miiddd-innnnnnnnn
Chen, Z., Wang, S., Liu, C., Yang, T., Dai, B., and Lin, D. Transinr: Encoding and decoding images as implicit neu- ral fields using data augmentation. Advances in Neural Information Processing Systems (NeurIPS), 2021.
10

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
Lee, D., Kim, C., Cho, M., and HAN, W. S. Locality-aware generalizable implicit neural representation. Advances in Neural Information Processing Systems, 36:48363- 48381, 2023.
Nartasilpa, N., Tuninetti, D., Devroye, N., and Erricolo, D. Let's share commrad: Effect of radar interference uncoded data communication system. In 2016 IEEE Radar Conference (RadarConf), pp. 1 1-5. IEEE, 2016.
Lee, S. Mesh-independent operator learning for partial differential equations. In ICML 2022 2nd Al for Science Workshop, 2022.
Nathaniel, J., Liu, J., and Gentine, P. Metaflux: Meta- learning global carbon fluxes from sparse spatiotemporal observations. Scientific Data, 10(1):440, 2023.
Lee, S. and Oh, T. Inducing point operator transformer: A flexible and scalable architecture for solving pdes. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pp. 153-161, 2024.
Nie, T., Qin, G., Ma, W., and Sun, J. Spatiotemporal implicit neural representation as a generalized traffic data learner. Transportation Research Part C: Emerging Technologies, 153:104890, 2024.
Li, Y., Yu, R., Shahabi, C., and Liu, Y. Diffusion con- volutional recurrent neural network: Data driven traffic forecasting. In International Conference on Learning Representations (ICLR), 2018.
Oreshkin, B. N., Carpov, D., Chapados, N., and Bengio, N-beats: Neural basis expansion analysis for inter- pretable time series forecasting, In International Confer- ence Learning Representations, 2020. URL https: / //openrevieee eww ew. www.w//weeeeeeeee...........ee.. fowmmfrruooooor d= ecqn
Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhat- tacharya, K., Stuart, A., and Anandkumar, A. Fourier neural operator for parametric partial differential equa- tions. arXiv preprini arXiv:2010.005:: 08895, 2020.
Pata, J., Wulff, E., Mokhtar, F., Southwick, D., Zhang, M., Girone, M., and Duarte, J. Improved particle-flow event reconstruction with scalable neural networks for current and future particle detectors. Communications Physics, 7 (1):124, 2024.
Li, Z., Kovachki, N., Azizzadenesheli, K., Bhattacharya, K., Anandkumar, A., and Stuart, A. Neural fields for pde solving. Neural Networks, 2022.
Li, Z., Meidani, K., and Farimani, A. B. Transformer for partial differential equations' operator learning. Transac- tions on Machine Learning Research, 2023. ISSN 2835- 8856. URL https: tetttps:/oppennee http///nneeeeeeeeee//ee forum? CG-EPPtt3EERT.
Patel, Z. B., Purohit, P., Patel, H. M., Sahni, S., and Batra, N. Accurate and scalable gaussian processes for fine fine-grained air quality inference. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 36, pp. 12080-12088, Jun. 2022. doi: 10.1609/ 136i1.3 1666111,2146.,. URL btts:ttp::/ orr/ index, hhtp://AAA///rrtiala/eee////////
Litjens, G., Kooi, T,, Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., Van Der Laak, A., Van Gin- neken, B., and Sánchez, C. I. A survey on deep learning in medical image analysis. Medical Image Analysis, 42: 60-88, 2017.
Peters, A. J. and Janyst, L. Exabyte scale storage cern. In Journal of Physics: Conference Series, volume 331, pp. 052015. IOP Publishing, 2011.
Massucci, F. A., Wheeler, J., Beltrán-Debón, R., Joven, J., Sales-Pardo, M., and Guimerà, R. Inferring propagation paths for sparsely observed perturbations on complex networks. Science advances, 2(10) 2016.
Reichstein, M., Camps-Vall Valls, G., Stevens, B., Jung, M., Denzler, J., Carvalhais, N., and Prabhat, F. Deep learning and process understanding for data-driven earth system science. Nature, 566(7743) 2019.
Meneveau, C. and Marusic, Turbulence in the atmosphere. Annual Review of Fluid Mechanics, 43:219-245, 2011.
Serrano, L., Le Boudec, I.., Kassai Koupai,, A., Wang, T. X., Yin, Y.. Vittaut, J.-N., and Gallinari, P. Operator learning with neural fields: Tackling pdes on general geometries. Advances in Neural Information Processing Systems, 36: 70581-70611, 2023.
Mildenhall, B., Srinivasan, P., Tancik, M., Barron, J. Ramamoorthi, R., and Ng, R. Nerf: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision (ECCV), pp. 405-421. Springer, 2020.
Serrano, L., Wang, T. X., Naour, E. L., Vittaut, J.NN., and Gallinari, P. AROMA: Preserving spatial struc- ture for latent PDE modeling with local neural fields. The Thirty-eighth Annual Conference on Newral In- formation Processing Systems, 2024. URL https: / //operreviee ew net forumm d= jakKka.
Myneni, R. B., Dong, J., Tucker, C. J., Kaufmann, R. K., Kauppi, P. E., Liski, J., Zhou, L., Alexeyev, V., and Hughes, M. A large carbon sink in the woody biomass of northern forests. Proceedings of the National Academy of Sciences, 98(26):1478 4784 14789, 2001.

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
Sitzmann, V., Lindell, D. B., and Wetzstein, G. Metasdf: Meta-learning signed distance functions. Advances in Neural Information Processing Systems (NeurIPS), 33: 10136-10147, 2020a.
Sitzmann, V. Martel, N., Bergman, W., Lindell, D. B., and Wetzstein, G. Implicit neural representations with periodic activation functions. Advances in Neural Infor- mation Processing Systems (NeurIPS), 33:7462-7473, 2020b.
Tancik, M., Srinivasan, P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J., and Ng. R. Fourier features let networks learn high fre- quency functions in low dimensional domains. Advances in neural information processing systems, 33:7537-7547, 2020.
Van Essen, D. C., Ugurbil, K., et al. Exploring the human connectome. Neuron, 70(5):713-720, 2010.
Wang, S., Seidman, J, H., Sankaran, S., Wang, H., Pappas, G. J., and Perdikaris, P. Bridging operator learning and conditioned neural fields: A unifying perspective. arXiv preprint 4rXiv:2405. 13998, 2024.
Wang, X., Girshick, R., Gupta, A., and He, K. Deep learning for video recognition: survey. IEEE Transactions Puttern Analysis and Machine Intelligence, 2020.
Woodcock, C. E. al. Globcover: Esa's global land cover map. Remote Sensing of Environment, 112:1951-1962, 2008.
Wu, X., Han, G., Li, W., Shao, Q. and Cao, L. Deep learning- based prediction of kuroshio path south of japan. Journal of Atmospheric and Oceanic Technology, 40(2): 175-190, 2023.
Yin, Y., Kirchmeyer, M., Franceschi, J.-Y., Rakotomamonjy, A., and patrick gallinari. Continuous PDE dynamics forecasting with implicit neural representations. In The Eleventh International Conference Learning Represen- tations, 2023. URL hitps: http://ppppeeeeeeeeeeeee nett/ forum? ddmmm1ddd-7dnns.ss
Yu, B., Yin, H., and Zhu, Z. Spatio-temporal graph convo- lutional networks: A deep learning framework for traffic forecasting. In International Joint Conference Artifi- cial Intelligence (IJCAI), 2017.
Yu. S., Hannah, W. Peng, L., Lin, J., Bhouri, M. A., Gupta, R., et al. Climsim: A large multi-scale dataset for hy- brid physics-ML climate emulation. In seieth-eventth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023, URL https: // //operevieww.o net forum? 1
12

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
A. Related Work
A.I. Spatiotemporal Learning
Spatiotemporal learning has emerged as a fundamental area of research, addressing the need to model and interpret data that evolve across both space and time. This field has far-reaching applications in diverse scientific and engineering disciplines, enabling advancements in climate modeling (Reichstein et al., 2019), traffic forecasting (Li et al., 2018), medical imaging (Litjens et al., 2017), and video analysis (Wang et al., 2020). Beyond these areas, spatiotemporal learning plays crucial role in biochemistry, where modeling molecular interactions and protein dynamics requires capturing complex temporal dependencies in high- -igennimensional spatial structures (Jumper et al., 2021). In nuclear particle physics, tracking subatomic particles in high-energy collisions demands precise spatiotemporal reconstruction to infer particle trajectories and decay chains (Aad et al., 2019). Neuroscience increasingly relies on spatiotemporal models to analyze large-scale neural recordings, such as EEG and IMRI, for understanding brain activity over time and across different brain regions (Van Essen et al., 2010). Similarly, in epidemiology, disease spread modeling depends on accurately capturing transmission dynamies across spatially distributed populations over time (Balean et al., 2009).
Other critical applications include remote sensing and Earth observation, where satellite imagery and geospatial data must be processed to track environmental changes, deforestation, and urbanization trends (Woodcock et al., 2008). In fluid dynamics, understanding turbulent flow patterns and their evolution over time is key to designing efficient aerodynamic structures and predicting oceanic and atmospheric circulation (Meneveau & Marusic, 2011). These applications highlight the growing importance of spatiotemporal learning in scientific discovery and engineering innovation. However, effectively capturing the complex dependencies inherent in spatiotemporal data presents significant challenges, including spatial heterogeneity (irregularly structured and non- non-uniformly distributed data), temporal con-stationarity (changes in statistical properties over time), and high dimensionality (large-scale data with intricate interdependencies). Addressing these challenges requires scalable, generalizable, and computationally efficient modeling approaches that can learn from noisy, sparse, and dynamically evolving spatiotemporal datasets.
A.2. Traditional Approaches to Spatiotemporal Learning
Historically, Recurrent Neural Networks (RNNs) and Long Short- Memory networks (LSTMs) have been employed to model temporal sequences (Hochreiter & Schmidhuber, 1997). For instance, LSTMs have been widely used in traffic forecasting to predict future traffic conditions based on historical data (Yu et al., 2017). Despite their effectiveness, these models often struggle with capturing long-range dependencies and may not fully exploit spatial correlations. To address these limitations, Transformer-best based architectures have been introduced, leveraging self-attention mechanisms to model long-range dependencies in both spatial and temporal dimensions. Vision Transformer (ViT) has demonstrated success in image analysis by treating images sequences of patches, enabling the modeling of global relationships (Dosovitskiy et al., 2021). Extending this idea, TimeSformer has been proposed for video understanding, jointly modeling spatial and temporal dependencies to achieve state-of-the-art results in action recognition tasks (Bertasius et al., 2021).
A.3. Implicit Neural Representations (INRs) for Learning Continuous Representations
Implicit Neural Representations (INRs) have emerged as flexible and powerful approach for modeling continuous signals. Unlike traditional grid-based representations, INRs parameterize data as continuous functions, mapping spatial coordinates to function values using a neural network (Sitzmann et al., 2020b). This formulation allows for compact data encoding, assolution-agnstio modeling, and seamless interpolation. INRs have been widely applied in 3D shape representation, scene reconstruction, and signal processing, where high-dimensional structured data needs to be represented efficiently. A notable example is Neural Radiance Fields (NeRF), which employs INRs for view synthesis, enabling the rendering of high-fidelity 3D scenes from sparse observations (Mildenhall al., 2020).
Despite these advantages, INRs face challenges in generalizability, as a trained INR typically encodes only single instance of data and does not naturally adapt to new instances. This limitation necessitates retraining the model for each new data sample, making INRs computationally expensive for large-scale applications. Generalizable INRs (GINRs) aim to overcome this by introducing mechanisms that allow a single model to adapt across multiple instances, rather than learning a fixed function for each data sample. Locality-Aware Generalizable Implicit Neural Representations introduce local feature conditioning that allows GINRs to adapt dynamically different regions of dataset, improving efficiency and generalization (Lee et al., 2023). Furthermore, MetaSDF and MetaSIREN employ gradient-basee meta-lcarning to
13

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
enable few-shot adaptation, allowing GINRs to learn priors across multiple data instances and generalize unseen samples more effectively (Sitzmann al., 2020a; Chen et al., 2021). Functa treats each data instance function and leveraging function-space representations for improved generalization (Dupont al., 2022). Spatial Functa extends this approach to large-scale datasets like ImageNet, introducing spatially aware latent spaces that enhance expressivity and enable GINRs to perform large-scale classification and generation tasks (Bauer et al., 2023).
A.4. Conditioned Neural Fields as GINRs
Recent work by Wang et al. (2024) et al., 2024) highlights the close relationship between CNFs and INRs, emphasizing that both frameworks model continuous fields but differ in their conditioning mechanisms. While INRs typically encode static signals without external conditioning, CNFs introduce input-dependent variations, making them well-suited for physics-informed learning in PDE solving. This connection unifies perspectives on operator learning and neural field-based modeling, bridging the gap between classical numerical methods and neural representations.
Conditioned Neural Fields (CNFs) provide a powerful framework for solving Partial Differential Equations (PDEs) by learning continuous function representations conditioned input parameters, initial conditions, constraints (Li et al., 2022). Unlike traditional numerical solvers that rely on discretization. CNFs upproximate time-evolving fields in a resolution- independent manner, making them particularly effective for high-dimensional and sparse-data problems. This approach aligns closely with GINRs, which parameterize data as continuous functions rather than grid-based representations (Sitzmann et al., 2020b).
A.5. Baseline Selection
Spatiotemporal learning requires modeling dynamic systems such as fluid flows, climate forecasting, and wave propagation, all of which are governed by PDEs. PDE-based neural solvers provide strong priors that enhance generalization, consistency, and interpretability. To evaluate our approach (SCENT) against established methods, we compare against FNO, which lears spectral representations for PDE solutions, OFormer, Transformer-basee operator learner, and CORAL and AROMA, which leverage neural fields for dynamic system modeling. These baselines offer diverse perspectives on how different architectures generalize across spatiotemporal interpolation, reconstruction, and forecasting tasks.
Table 4, Comparing model capacities for learning spatiotemporal continuity from discrete data.
MESH AGNOSTIC LEARNING
SPACE&TIME CONTINUOUS
MODEL
SINGLE-STEP TRAINING
LEARNING
FNO ET
ET AL., 2020)
X
OFORMER (LI ET AL.. 2023)
x
CORAL (SERRANO ET AT.,, 2023)
/
X
AROMA (SERRANO AL., 2024)
/
X
SCENT (OURS)
14

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
B. SCENT: Pseudo-Algorithm
We omit spatiotemporal mesh variables ((x)) 4, 4.10) 10)
in the algorithm for simplification.
Algorithm Training and Inference for Model F
Training Procedure:
Model P. training dataset with Mhy trajectories, each timesteps
Time horizon fa, total training steps Neain, loss function g
Initialize model parameters 0 for iteration 1 Nunin do
Select ith trajectory from D
Sample (U%, U**) with time horizon th, where to - 1/2 = At E (0,1,2......
Forward Pass: Compute model prediction Ühh F(U*)
Compute loss: - _(*> end return Trained model F
Backpropagation: Compute gradients Vol and update
Validation Procedure:
Validation dataset V with real trajectories with timesteps T. trained model F, validation metric Freeze model F [Disable gradient updates)
Initialize empty list B = []
for each validation trajectory in V do for each (U%, U**) where [0, 1] and to t; do
Forward propagate: t F(U
Compute loss: L= (0 (n))
Append L. to list B
end for
end for return
B Even {Return the average}
Forecasting Procedure:
Trained model F", of evaluation samples with desired targeted time t,
Initialize empty list B = =[]
for each evaluation sample Uh in & do Compute quotient and remainder: [tt. - tt)/tt J. t. = (to - ft) mod 6x
for j to do (Performing Warp-Unrollin Forecasting (Section 3.3)}
Update f. <- 4 + ba
Forward propagate: th F(U*))
Update t; 6- to
end for
if
then
Forward propagate: t F(U (One last forward pass with the remainder f,
end if
Append 0% to list B
end for
return list B (Return predicted samples)
15

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
C. Data statistics and Hyperparameters for training SCENT on Navier-Stokes Benchmark
Datasets
DATASET NAME
NS-3
NS-4
NS-5
STATISTICS
TRAJECTORIES
256
1000
1000
TRAJECTORIES -VALIDATION
64
200
MAXIMUM
30
30
20
INITIAL T
10
10
10
SPATIAL RESOLUTION
(64,64)
(64,64)
(64,64)
POINTS INPUTS (N)
4096
4096
4096
N POINTS INPUTS (M)
4096
4096
4096
MAX LR
0.001
0.0006
0.0008
MIN LR
0.000008
LR SCHEDULER WARMUP STEPS
COSINE
COSINE
COSINE
TRAINING
0
2000
2000
BATCH
100
100
100
TOTAL STEPS
150000
110000
110000
OPTIMIZER
ADAMW
ADAMW
ADAMW
BETAl
0.9
0.9
0.9
BETA2
0.999
0.999
0.999
TRAINING TIME HORIZON
WEIGHT DECAY
0.00001
0.00001
0.00001
EMBED DIM
128
128
LATENT DIM
128
128
128
LINEAR PROJECTION DIM # LEARNABLE QUERIES
64
64
64
64
256
256
# LAYERS PROCESSOR
2
2
MODEL
LAYERS ENCODER
# LAYERS DECODER
4
4
# HEADS
4
4
SPARSE ATTENTION GROUP SIZE
1
8
FF MULTIPLIER
OUTPUT SCALE
0.1
0.1
0.1
LATENT INIT SCALING (STD)
0.02
0.02
EMBEDDING
FOURIER FEATURES FREQUENCY BANDS
6
12
12
FOURIER FEATURES MAX RESOLUTION
20
20
20
16

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
D. Data statistics and Hyperparameters for Training SCENT on Simulated Datasets
DATASET NAME
S3
S4
TRAJECTORIES TRAIN
100000
100000
100000
100000
100000
TRAJECTORIES -VALIDATION
1000
1000
1000
1000
1000
MAXIMUM
50
50
50
50
DATA STATISTICS
INITIAL T
1
1
TEMPORAL SUBSAMPLE STEP SIZE
2
2
2
SPATIAL
(64,64)
(64.64)
(64,64)
(64,64)
(64,64)
POINTS INPUTS (N)
4096
4096
2840
2048
2000
N POINTS INPUTS (M)
4096
4096
2840
2048
2000
MAX LR
0.0008
0.0008
0.0008
0.0008
0.0008
MIN LR
0.00008
0.00008
0.00008
0.00008
0.00008
LR SCHEDULER
COSINE
COSINE
COSINE
COSINE
COSINE
WARMUP STEPS
2000
2000
2000
2000
2000
TRAINING
BATCH
256
256
256
TOTAL STEPS
100000
100000
50000
50000
50000
OPTIMIZER
ADAMW
ADAMW
ADAMW
ADAMW
ADAMW
BETAI
0.9
0.9
0.9
0.9
0.9
BETA2
0,999
0.999
0.999
0.999
0.999
TRAINING TIME HORIZON
3
3
WEIGHT DECAY
0.0001
0.0001
0.0001
0.0001
0.0001
EMBED DIM
164
164
164
164
LATENT DIM
128
128
128
128
128
LINEAR PROJECTION DIM
64
64
64
64
64
LEARNABLE QUERIES
128
128
128
128
128
LAYERS PROCESSOR
MODEL
LAYERS ENCODER
6
6
LAYERS DECODER
6
6
6
6
HEADS
4
4
4
SPARSE ATTENTION
8
8
FF MULTIPLIER
4
OUTPUT
0.1
0.1
0.1
0.1
0.1
LATENT INIT SCALING (STD)
0.02
0.02
0.02
0.02
0.02
EMBEDDING
FOURIER FEATURES FREQUENCY BANDS
12
12
12
12
12
FOURIER FEATURES MAX RESOLUTION
20
20
20
20
20
E. Additional Descriptions on Simulated Datasets
This dataset represents an incompressible fluid dynamics system governed by the vorticity transport equation:
dw
u
- + +
01
where the vorticity defined as:
w xu u=u
(2)
Here, u denotes the velocity field, and represents the vorticity. Both quantities are defined on spatial domain with periodic boundary conditions. The parameter represents the kinematic viscosity, and f is external forcing function applied to sustain turbulence.
The input at time t is given as wi. The spatial domain is defined as:
Ω = |- x|2
The vorticity field initialized using Gaussian Random Field (GRF) with parameters: alpha 2.5, 3.0.
17

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
Here, controls the smoothness of the initial vorticity distribution, while determines the correlation length scale of the spatial structures. The external forcing function used in the simulation is:
22), cos((122)
where ø2 represents the vertical spatial coordinate.
This forcing function introduces structured periodic forcing along the vertical direction, promoting rotational flow characteristics. The periodic nature of the cosine function ensures repeating vortex structure, which sustains turbulence and prevents energy dissipation over time. The negative sign maintains consistent direction of vorticity input, reinforcing the rotational dynamics within the system. As a result, this setup generates a persistent and well-defined turbulent flow pattern. Reynolds number of Re 100 is used, indicating a moderately turbulent regime where inertial forces are dominant over viscous forces, allowing for complex vortex interactions while maintaining numerical stability. This is particularly relevant for spatiotemporal learning, it provides complex yet structured temporal evolution of the vorticity field, making it an ideal testhed for evaluating models that aim learn continuous representations of dynamic physical systems. We generate in total 100k trajectories, with 50 time steps per trajectory. Among T 50 data, we use every two steps ((0, 2, 4, ..... 48)) for training and validation. The remaining time steps (f1,3, .,..., 49)) are reserved for evaluating the model's 's ability to learn continuous temporal representations.
18

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
F. Hyperparameters for Model Scalability EvaluationssFig. 4(a)) use dataset S5 for the model scalability evaluations.
MODEL NAME
M2
M5
M6
TRAINING
MAX LR LR SCHEDULER
0.0008
0.0007
0.0006
0,0005
0.0004
0.0003
0.0002
0.00008
0.00007
0.00006
0.00005
0.00004
0.00003
0.00002
COSINE
COSINE
COSINE
COSINE
COSINE
COSINE
COSINE
WARMUP STEPS
2000
2000
2000
2000
2000
2000
2000
BATCH SIZE
256
256
256
256
256
256
256
TOTAL
50000
50000
50000
50000
50000
50000
50000
OPTIMIZER
ADAMW
ADAMW
ADAMW
ADAMW
ADAMW
ADAMW
ADAMW
BETA
0.9
0.9
0.9
0.9
0.9
0.9
0.9
BETA2
0.999
0.999
0.999
0.999
0.999
0.999
0.999
TRAINING TIME
3
3
3
3
3
3
HORIZON
WEIGHT DECAY
0.0001
0.0001
0.0001
0.0001
0.0001
0.0001
0.0001
LATENT DIM
192
256
512
768
1024
# LEARNABLE QUERIES
128
138
164
192
224
224
256
# LAYERS PROCESSOR
2
2
2
2
4
# LAYERS ENCODER
6
6
6
6
6
6
# LAYERS DECODER
6
6
6
# HEADS PROCESSOR
4
8
8
12
MODEL
# HEADS ENCODER
2
2
2
4
4
# HEADS DECODER
2
2
2
4
4
4
EMBED DIM
164
164
164
164
164
164
164
LINEAR-
64
64
64
64
64
64
64
PROJECTION DIM
SPARSE ATTENTION
8
GROUP SIZE
MULTIPLIER
4
4
4
4
OUTPUT SCALE
0.1
0.1
0.1
0.1
0.1
0.1
0.1
LATENT INIT
EMBEDDING
0.02
0.02
0.02
0.02
0.02
0.02
0.02
SCALING (STD) FOURIER FEATURES # FREQUENCY BANDS
12
12
12
12
FOURIER FEATURES
20
20
20
20
20
20
20
MAX RESOLUTION
G. Hy perparameters for Dataset Scalability Evaluations (Fig. 4(b))
Other hyperparameters follow Appendix for each corresponding model size.
MODEL NAME
LATENT
DATASET NAME
BATCH SIZE
TOTAL STEPS
# TRAJECTORIES
MAX
MIN LR
DIM
S5-30
30000
50000
0.0008
0.00008
MI
128
S5-50
50000
128
50000
0.0008
0.00008
S5-100
100000
256
50000
0.0008
0.00008
$5-30
30000
76
50000
0.0006
0.00006
M2
256
85-50
50000
128
50000
0.0006
0.00006
S5-100
100000
256
50000
0.0006
0.00006
85-30
30000
76
50000
0.0004
0.00004
M3
512
85-50
50000
128
50000
0.0004
0.00004
S5 100
100000
256
50000
0.0004
0.00004
19

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
H. Learning Spatiotemporal Continuity: additional results
Two additional instances are illustrated. Please refer to main manuscript Fig. for interpretation.
input:
output: f
Additional instance
nearest
RBF
CORAL
SCENT
GT
input:
output: &
Additional instance 2
nearest
RBF
CORAL
SCENT
GT
20

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
I. Forecasting on S5 dataset: additional results
Four additional instances are illustrated. Please refer to main manuscript Fig. 5(a) for full details and interpretations.
Input (ti)
delta
1
GT
Pred
(t₂)
GT
SCENT
FNO
AROMA
CORAL
OFORMER
Input (t.)
delta
2
Pred
GT
SCENT
FNO
AROMA
CORAL
OFORMER
Input (t:)
delta
3
Pred
GT
SCENT
FNO
AROMA
CORAL
OFORMER
Input
delta
(t:)
4
Pred
(t.)
GT
SCENT
FNO
AROMA
CORAL
OFORMER
21

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
J. Forecasting AD-B dataset: additional results
Four additional instances are illustrated. Please refer to main manuscript Fig. 5(b) for details and interpretations.
100
Input (c:)
delta
400
dc
dc
R2
dc
300
GT
Pred
200
GT
SCENT
FNO
AROMA
CORAL
OFomer
100
Input
delta
20
2
400
Pred
(ta)
200
100
SCENT
FNO
AROMA
100
Input
delta
1
20
3
400
77
1
17
1
Pred
(ta)
200
100
SCENT
FNO
AROMA
CORAL
OFormer
100
Input (c)
delta
4
300
GT (t.)
Pred
200
GT
SCENT
FNO
AROMA
CORAL
OFormer
22

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
K. AirDelhi AD-B: Comparisons Against Previously Reported Benchmark Performances
Here, we report and compare performances against SCENT and other baselines. Our experiments with conditioned neural fields set new record in AD-B benchmark. Inverse Distance Weighting (IDW) computes the weighted average of all visible samples based on their distances, assigning this value to the held-out locations. Random Forest (RF) is a non-linear model designed to capture complex spatial relationships. It excels in non-linear regression tasks by utilizing an ensemble of decision trees, with the final prediction obtained as the mean output across all trees. XGBoost (XGB) incrementally enhances predictions by combining weak estimators. During training, it employs gradient boosting to optimize performance while sequentially adding new trees. ARIMA (Auto-Regressive Integrated Moving Average) is statistical model for time- series forecasting that applies linear regression. N-BEATS (Neural Basis Expansion Analysis for Time Series) (Oreshkin et al., 2020) is a deep learning model designed for zero-shot time-series forecasting. Non-Stationary Gaussian Process (NSGP) (Patel et al., 2022) a recent Gaussian process baseline that models non-stationary covariance for latitude and longitude, along with locally periodic covariance for time.
PRIOR REPORTED
UPDATED
PERFORMANCES
PERFORMANCES
MODEL
RMSE
MODEL
RMSE
IDW
86.52
FNO
48.79
RF
110.49
OFORMER
70.62
XGBOOST
102.68
CORAL
60.51
NSGP
95.83
AROMA
40.78
ARIMA
148.86
SCENT
44.2
N-BEATS
106.41
L. Computational Complexity Analysis
Summary. This section provides Big-O time comparisons between SCENT, AROMA, and FNO. In summary, SCENT's cost is sensitively affected by S which denotes the number of tokens attended within sparse attention layers in CEN (Section 3.2) and CN (Section 3.4). Meanwhile, AROMA with diffusion transformer backbone is costly if the number of refinement step K and unrolling steps T increase. Lastly, FNO is sensitive to the model width C' and also the unrolling steps T. We show that at the scale of the NS-3 experiment (Section 2), SCENT is the most expensive during training among three models, while the gap shrinks for larger unrolling steps. SCENT scales linearly with W, while AROMA scales linearly with K and T. Depending on their values, one could be more expensive than the other. Notably, W T' where ta is the time horizon of SCENT, thanks Warp-Unrollin Forecasting (Section 3.3, Fig. 3). Thus, for longer time forecasting SCENT more efficient. Please refer to following subsections for big-O time complexity derivations for individual models.
Table 5. Big-O time complexity
MODEL
FORMULA
FNO AROMA
O(T log NC + LNdC)) O((2N + 4KT + M.M)
SCENT
OLW + W WN + W 3)
L.1. Big-O time complexity of SCENT
For SCENT equipped with:
Sparse attention: Lg layers where each of the N tokens attends to S tokens (S < NN.
Cross-attention: Two layers between N and M tokens.
Self-attention: Lm layers operating on M tokens.
23

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
Table 6. Hyperparameters derived from experiments in Section 4.2.
PARAMETER
SYMBOL
VALUE
FOURIER LAYERS FNO
SPATIAL GRID POINTS
2000
FOURIER MODES
d
16
MODEL WIDTH
C
60
UNROLLING STEPS (FNO. AROMA)
OR 20
UNROLLING STEPS (SCENT)
OR 7
SPARSE ATTENTION LAYERS
Ls
6
SEL.F-ATTENTION LAYERS
T
2
SPARSE ATTENTION TOKENS
500
COMPRESSED TOKENS
128
LATENT CHANNELS
d
128
Table 7. Sample cost computed using hyperparameters in Table 6
UNROLLING
UNROLLING 20
MODEL
COST
RELATIVE SCALE
COST
RELATIVE SCALE
FNO
1.28E+08
1.0
2.56E+09
1.0
AROMA
2.29E+08
1.78
3,09E+09
1.21
SCENT
8.04E+08
6.28
5.63E+09
2.20
Sparse Attention Blocks
Each token from N attends to only S tokens, reducing the full self-attention cost from (NNd to:
O(L,NSd)
Cross-Attention Between N and M Tokens
Each cross-attention operation has cost:
O(N
With two such layers, the total remains:
M(N
Self-Attention on M Tokens
self-attention block on M tokens with L,, layers incurs:
M22)
Final Big-O Complexity
Summing contributions:
O(L.NSd+ NMd ddd
Considering W unrolling during inference, it becomes:
O(WL.NSd WNMd+ W L.
L.2. Big-O for AROMA
AROMA'S computational complexity is provided as following (Serrano et al., 2024):
O((2N 4KLLM L.M)Md)
(4)
24

---

SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data
where:
K Number of refinement steps
T = Number of autoregressive calls in unrolling
L2, L1 Number of layers in different parts of the architecture
Number of observations
Number tokens used to compress information
d Number of channels used in the attention mechanism
Diffusion Transformer incurs additional costs due iterative refinement:
L4
(5)
which scales with K (refinement steps) and T (autoregressive calls).
L.3. Big-O for FNO
For an FNO (Li et al., 2020) model with:
L Number of Fourier layers
N Number of spatial grid points
d Number of Fourier modes (spectral channels)
C Model width (number of feature channels per layer)
T = Number of autoregressive unrolling
The key computational operations are:
Fast Fourier Transform (FFT) and Inverse FFT (IFFT)
O(L log NC)
(6)
FFT is applied across all feature channels.
Linear Transform in Fourier Space
OLLN&C
(7)
Applies transformations to Fourier modes across all channels.
Total Complexity for Multiple Layers
O NC LNdC))
(8)
The unrolling factor 7 accounts for repeated forward passes in autoregressive prediction.
25


Reviews:
[{"review_id": "pQENI7Gc2J", "reviewer": "Reviewer_ujM5", "summary": "This paper addresses common issues in scientific data, such as sparsity, noise, and multi-scale problems, by proposing a method called SCENT (Scalable Conditioned Neural Field) that can handle various spatio-temporal learning tasks like interpolation, reconstruction, and prediction. The paper is well-structured with clear motivation, and extensive experiments on both synthetic and real-world data demonstrate the method's effectiveness and scalability.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's suggestion. We agree that the Kuroshio current provides an excellent yet challenging testbed for evaluating SCENT. We use 50‐year records from the China Ocean Reanalysis (CORA) [1] as our benchmark and follow the data processing guidelines established by Wu et al. (2023) [2].\n\n\n**Background.** The Kuroshio current, originating from the North Equatorial Current (NEC) and flowing northward along the eastern side of the Philippine Islands, is the world's second-largest warm current. Accurately predicting its path is crucial because its variations significantly affect the exchange of water masses and heat between the North Pacific subtropical and subarctic circulations. CORA provides daily oceanographic reanalysis data for the Kuroshio current spanning 50 years (January 1958-December 2007).\n\n**Implementation.** We adopt the four baseline methods from Wu et al. (2023) for comparison. We perform a 120-day prediction experiment, using data from the first 40 years (1958-1997) for training and the final 10 years (1998-2007) for testing. During training, SCENT is used to predict the Kuroshio path in terms of latitude (ranging from 29°N to 36°N) for various forecast horizons ($t_o$). In the test phase, we measure the root mean-squared error (RMSE, in degrees) against the true latitude for every $t_o \\in [1, 120]$ days, and we present comparisons at 10-day intervals. \n\n\n**Results.** Comparison analyses are shown below. We categorize baseline methods into those employing feature engineering (FE) and those that are purely neural network–based. LSTM is used as the baseline, and we also compare FE-enhanced LSTM variants, where FE includes empirical orthogonal functions (EOF) and complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN). The CEEMDAN-based variants perform competitively across all lead times, while SCENT exhibits second-best performance at and beyond a 50-day lead time. The relative strength of the FE methods may be due to the limited data scale—with 14,610 instances in the training split and 3,652 in the test split. However, our scalability study (Fig. 4) suggests that SCENT could outperform the FE methods when larger datasets become available. Additionally, SCENT maintains stable performance even as the lead time increases. This is in stark contrast to the other baselines, which experience significantly sharper performance degradation with longer lead times.\n\n\n| Type  | Model            |                   |                                                   |       |       |       |    Lead    |   Time     |   (days)    |       |       |       |     |     | \n|------------------|------------------|-------------------|:-----------------------------------------------------------------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|\n|            |                  |                   | 10 | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 90 | 100| 110| 120| \n | FE+NN | EOF_LSTM         |                   | 0.391  | 0.448  | 0.490  | 0.518  | 0.546  | 0.570  | 0.592  | 0.601  | 0.612  | 0.619  | 0.627  | 0.628  | \n | FE+NN| CEEMDAN_LSTM     |                   | $\\underline{0.243}$  | $\\underline{0.294}$  | $\\underline{0.328}$  | $\\underline{0.357}$  | 0.380  | 0.399  | 0.417  | 0.432  | 0.446  | 0.464  | 0.481  | 0.493  | \n | FE+NN| EOF_SEEMDAN_LSTM |                   | **0.176**  | **0.229**  | **0.262**  | **0.279**  | **0.303**  | **0.325**  | **0.337**  | **0.346**  | **0.357**  | **0.371**  | **0.386**  | **0.399**  | \n | Pure NN| LSTM             |                   | 0.403  | 0.468  | 0.511  | 0.546  | 0.576  | 0.591  | 0.605  | 0.617  | 0.628  | 0.643  | 0.646  | 0.647  | \n | Pure NN| SCENT            |                   | 0.354  | 0.348  | 0.360  | 0.365  | $\\underline{0.374}$  | $\\underline{0.391}$  | $\\underline{0.390}$  | $\\underline{0.408}$ | $\\underline{0.410}$  | $\\underline{0.419}$ | $\\underline{0.415}$  | $\\underline{0.430}$  | \n|            |                  |                   |  | | | | ||||| | Metric: | RMSE (°)  |\n\n\n**Thank you again for your time and feedback. We respectfully ask you to consider a revised score if you feel our responses address your concerns.**\n\n[1] Han, G. et al. \"A new version of regional ocean reanalysis for coastal waters of China and adjacent seas\", *Adv. Atmos. Sci*. 30, 974–982 (2013).\n\n\n[2] Wu, X. et al. \"Deep Learning–Based Prediction of Kuroshio Path South of Japan\", *J. Atmos. Oceanic Tech*. 40.2 (2023).", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "yes", "theoretical_claims": "yes", "experimental_designs_or_analyses": "yes", "relation_to_broader_scientific_literature": "yes", "essential_references_not_discussed": "yes", "other_strengths_and_weaknesses": "Weaknesses: I think the research scenario is too idealized. It could be worth considering real-world scenarios, like observational data from the Kuroshio, for example.", "other_comments_or_suggestions": "No", "questions_for_authors": "No", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_flag": "Flag this paper for an ethics review.", "ethics_expertise_needed": ["Other expertise"], "ethical_review_concerns": "No"}}, {"review_id": "YSU08inEzW", "reviewer": "Reviewer_WAR8", "summary": "This paper introduces SCENT, a framework for spatiotemporal learning using Scalable Conditioned Neural Fields (CNFs). The model is built on a Transformer-based encoder-processor-decoder architecture, incorporating learnable queries and a query-wise cross-attention mechanism to capture multi-scale dependencies. A sparse attention mechanism is used to improve scalability.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We truly appreciate your valuable comments. We agree on your concerns and suggestions, hence provide below our thoughts and additional experimental results for each of the questions. \n\n$\\ $\n\n**1. On Fourier feature** \n\nAlthough Fourier features are well established, their formulation can vary. Here, we describe our approach and present additional ablation studies. Let $R$ denote the maximum frequency resolution and $L$ the number of frequency bands. For the $i$th band, we define the frequency as \n$\nf_i = \\frac{iR}{L}, \\quad i=1,2,\\ldots, L.\n$\nThen, the positional encoding for a scalar $x$ is given by the concatenation of sine and cosine functions:\n$\n\\gamma(x)=\\operatorname{concat}_{i=1}^L \\Big[\\sin\\big(2\\pi f_i x\\big), \\cos\\big(2\\pi f_i x\\big)\\Big].\n$\n\nWe tune $R$ and $L$ to match the data’s inherent frequency characteristics: $R$ is set high enough to capture rapid variations, while $L$ is chosen to balance detail with computational efficiency. The following ablation experiments were conducted using a baseline SCENT model on dataset S5 with $M=32$ learnable queries, latent dimension $l=128$, and a batch size of 128.\n\n|# bands ($L$)|$\\quad$4|$\\quad$6|$\\quad$8|$\\quad$12|$\\quad$16|\n|:---:|:---:|:---:|:---:|:---:|:---:|\n|**Rel-MSE**|$\\underline{0.448}$|**0.446**|0.466|0.471|0.453|\n\n\n|Max resolution ($R$)|$\\quad$ 5|$\\quad$10|$\\quad$20|$\\quad$32|\n|:---:|:---:|:---:|:---:|:---:|\n| **Rel-MSE** | 0.514 |0.573| $\\underline{0.471}$ | **0.443** |\n\nThe results show that while the number of frequency bands $L$ has little impact on performance, a higher maximum resolution $R$ clearly improves it. Values were not explored beyond the Nyquist criterion (i.e., $R=32$).\n\n$\\ $\n\n**2. SCENT’s sparse attention mechanism**\n\nWe appreciate the reviewer's insightful question. Since scientific data typically appear as smooth and continuous signals, attending to all tokens may become redundant and inefficient for high sampling rates. In our approach, we employ random sparse attention for both the Context Embedding Network and Calibration Network (see Fig. 2), where each token attends to a random subset of $p$ tokens, reducing the complexity from $O(n^2)$ to $O(pn)$ with $p \\ll n$. This simple random sparse attention mechanism has demonstrated strong empirical performance. We also acknowledge the extensive literature on efficient attention mechanisms – such as Longformer and Linformer – which may work equally well; indeed, additional experiments with these methods (with roughly matched parameter sizes for fair comparison) reveal that the Longformer (replacing sparse attentions within SCENT) outperforms the others, suggesting promising avenues for future research.\n\n| Method|Big-O Complexity|$\\quad$ Variables/Notes|Rel-MSE\n|--|--|--|--|\n|Random Sparse Attention| $O(n·p$)| $p$: # tokens each query attends to|0.471|\n|SCENT+Linformer [1]|$O(n·k$) |$k$: projected dimension ($k \\ll n$, constant)|0.496|\n|SCENT+Longformer [2]|$O(n·(w+g)$)|$w$: sliding window size; $g$: # global tokens|**0.440**|\n\n$\\ $\n\n**3. Additional real-world datasets**\n\nThank you for raising concerns over limited real dataset employed. Here we report explore additional spatiotemporal datasets.\n\n(i) **Rainfall Nowcasting**: Using DWD radar rainfall data, SCENT nowcasts lead times from 5 to 60 minutes based on four consecutive 5-minute intervals. Compared to the CNN-based RainNet baseline, SCENT reduces the root-mean-squared error (RMSE) by approximately 30\\%. The table summarizes the performance metrics (RMSE in mm$~h^{-1}$) and the number of parameters for each method.\n\n| Method | # Params ||Lead|time|(mins)||\n|--|:---:|:--:|:--:|:--:|:--:|:--:|\n|||**5**|**10**|**30**|**60**| \n|RainNet [3]|1.93M|0.445|0.431|0.526|0.646| \n|SCENT|3.87M|**0.319**|**0.341**|**0.389**|**0.440**|\n||||Metric:|RMSE|(mm$~h^{-1}$)||||\n\n\n(ii) **Kuroshio Path Prediction**: Using 50 years of CORA [4] reanalysis data, SCENT predicts the Kuroshio current's latitude over a 120-day horizon (with data from 1958–1997 for training and 1998–2007 for testing). SCENT demonstrates stable performance beyond a 50-day lead time. Table below shows that SCENT achieves lower RMSE (in degrees) compared to an LSTM baseline, particularly for longer forecast horizons.\n\n|Model|||Lead|Time|(days)|\n|--|--|:--:|:--:|:--:|:--:|\n|||10|30|60|120| \n |LSTM||0.403|0.511|0.591|0.647| \n |SCENT||**0.354**|**0.360**|**0.391**|**0.430**| \n|||||Metric:|RMSE (°)|\n\n**We hope our responses have addressed your key concerns and would be grateful if you would consider a higher score in light of these clarifications.**\n\n[1] Wang, S, et al. \"Linformer: Self-attention with linear complexity\", *arXiv* 2020.\n\n[2] Beltagy, I, et al. \"Longformer: The long-document transformer\", *arXiv* 2020.\n\n[3] Ayzel, G, et al. \"RainNet v1.0: a convolutional neural network for radar-based precipitation nowcasting\", *GMD* (2020).\n\n[4] Han, G. et al. \"A new version of regional ocean reanalysis for coastal waters of China and adjacent seas\", *Adv. Atmos. Sci.* (2013).", "claims_and_evidence": "Some claims lack sufficient supporting evidence:\n1. Sparse Attention Justification – The paper claims sparse attention improves scalability, but does not compare against other sparse attention models (e.g., Longformer, Linformer).\n2. Fourier Features Impact – SCENT uses Fourier features, but the paper does not analyze how different frequency bands affect prediction quality.", "methods_and_evaluation_criteria": "1. Simulated datasets (Navier-Stokes, synthetic sensor data) are relevant, but real-world evaluation is limited (only AirDelhi). More diverse real-world datasets are needed.", "theoretical_claims": "The paper does not present formal theoretical proofs, but it makes implicit theoretical claims about the benefits of its architecture.", "experimental_designs_or_analyses": "The experimental design is generally sound, but there are some limitations:\n1. The paper does not compare SCENT’s sparse attention to other sparse attention models (e.g., Longformer, Linformer), making it unclear how much it contributes to performance gains.\n2. The paper does not analyze the impact of different frequency bands on spatial encoding, leaving a gap in understanding its effectiveness.", "relation_to_broader_scientific_literature": "The paper builds on spatiotemporal learning, implicit neural representations (INRs), and conditioned neural fields (CNFs) but lacks engagement with key works. It does not compare SCENT’s sparse attention to models like Longformer, Linformer, nor analyze Fourier features’ impact.", "essential_references_not_discussed": "The paper does not cite or compare SCENT’s sparse attention to established models like Longformer, Linformer, and others, which are essential for evaluating its efficiency.", "other_strengths_and_weaknesses": "Pros:\n1. The encoder-processor-decoder architecture is well-motivated, and the use of learnable queries and cross-attention improves scalability.\n2. SCENT outperforms baselines like FNO, AROMA, CORAL, and OFormer in most scenarios.\n\nCons:\n1. SCENT uses Fourier features for spatial encoding, but the impact of different frequency bands on prediction quality is not analyzed.\n2. Most evaluations focus on simulated datasets (Navier-Stokes, synthetic sensor data). The AirDelhi dataset is the only real-world test, and results are not significantly better than AROMA.\n3. The motivation and implementation details of the sparse attention mechanism in SCENT are unclear. The paper does not thoroughly explain why sparse attention is necessary beyond scalability and how it specifically enhances spatiotemporal learning. Additionally, it lacks a direct comparison with other sparse attention models, such as Longformer or Linformer, which could provide insight into the efficiency and effectiveness of SCENT’s attention mechanism.", "other_comments_or_suggestions": "Please refer to weaknesses.", "questions_for_authors": "1. How do different frequency bands in Fourier feature encoding affect SCENT’s performance? An ablation study could strengthen the justification for their use.\n2. Why is SCENT’s sparse attention mechanism chosen over existing methods like Longformer, or Linformer?\n3. SCENT performs well on simulated datasets, but its AirDelhi results are not significantly better than AROMA. Can the model generalize effectively to other real-world datasets?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BNQf2HDR5b", "reviewer": "Reviewer_RkwD", "summary": "The authors introduce a new model called SCENT for spatiotemporal modelling such as for differential equations like Navier-stokes. This model can take irregular input data and generate outputs at arbitrary locations and times, and so is capable of forecasting and spatial interpolation. This model has an encoder - processor - decoder architecture using latent tokens in the processing layers. The authors add several new components to this architecture such as sparse self attention layers during encoding and decoding, and providing the required output time during input and output, limiting the number of recurrent steps required for forecasting. This model is compared to many other current methods on simulated and real world data and shows good performance. The authors show that this model scales well with model and dataset size.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Hyperparameters**  \nWe appreciate the reviewer’s thoughtful questions regarding the extent of hyperparameter tuning conducted for SCENT in comparison to the baseline methods. This is indeed a crucial aspect when evaluating model performance fairly across methods. We included detailed hyperparameters in the appendix for transparency and reproducibility, and will release the code upon acceptance.\n\n**Inherited from Literature.** We clarify that we did not extensively tune hyperparameters for baseline methods, but instead adopted configurations from prior work. Many baselines, including IPOT and Perceiver-IO, share architectural similarities with our encoder-processor-decoder design, allowing us to inherit most of their recommended settings. For instance, Appendix Table C largely follows IPOT, with only minor changes such as warmup and total training steps.\n\n**Dataset Dependence.** Dataset-specific parameters (e.g., optimizer profiles, embedding configurations) reflect widely accepted strategies from prior literature, especially for common datasets like NS-3–5. These datasets exhibit varied dynamics but benefit from established training practices—such as long schedules for PDE dynamics and lower sensitivity to learning rate. As shown in Appendix Table D, most hyperparameters are consistent across datasets. We deliberately avoided aggressive tuning to promote generalizability. For example, embedding dimensions result from simple design choices—e.g., combining linear projections and Fourier features—rather than extensive tuning.\n\n**Baseline Hyperparameters.** When implementing the baselines, we made every effort to adhere to published or well-established hyperparameters. We first ensured each baseline was reproducible on its original dataset, then applied either the reported settings or those that worked well for SCENT—whichever yielded better performance. Given the complexity of our proposed data, which reflects realistic and large-scale scientific scenarios, we devoted substantial effort to adapting the baselines accordingly.\n\n**2. Newer FNO approaches**  \nWe thank the reviewer for highlighting recent FNO variants, including Geo-FNO and F-FNO. These works offer notable improvements, particularly in handling irregular geometries and sampling patterns. While we were not able to implement both approaches in full due to time constraints, we successfully implemented F-FNO on the S5 dataset and included those results in our evaluation. We evaluated two configurations of F-FNO with different parameter sizes (m1, m2). Notably, even the smaller variant (1M parameters) performs similarly to the standard FNO, demonstrating the efficiency of the architecture. When scaled to match the parameter count of SCENT and FNO (7.4M), F-FNO achieves significantly improved performance, approaching that of SCENT. We believe this provides meaningful insight into how newer FNO variants perform in challenging, non-grid-based scenarios.\n\n| Metric | SCENT | FNO  | F-FNO_m1 | F-FNO_m2 |\n|---|---|---|---|---|\n| # Params| 7.4M| 7.4M | 0.96M | 7.4M |\n| Rel-MSE| **0.326**| 0.377| 0.396| $\\underline{0.347}$ |\n\n**3. On Vaughan et al. (2024) [1]**  \nWe appreciate the reviewer’s reference to the Aardvark Weather system. The idea of sequentially trained processors extending the recursive rollout strategy is particularly interesting. By allowing each processor to specialize in longer lead times, the approach may indeed improve inference stability and long-range forecasting performance. While the sequential training of multiple models could introduce computational overhead, the modular nature of the design offers flexibility that might benefit certain applications. Additionally, although the current formulation appears to target fixed-step forecasting, integrating mechanisms for continuous-time inference could be an exciting direction for future work. We see this as a promising and complementary line of research and welcome further discussion on how such ideas might be integrated with or compared to SCENT’s joint modeling capabilities.\n\n**4. On CORAL**  \nAs the reviewer correctly notes, OFormer, CORAL, and AROMA are all trained using recursive forecasting with a fixed time step. However, CORAL distinguishes itself by introducing a Neural ODE solver, $g_{\\phi}$, as the autoregressive processor operating in the latent z-code space. As a result, CORAL adopts a two-stage training process: the first trains the autoencoder, and the second trains the Neural ODE for forecasting. The key advantage of this approach is that the learned ODE can be evaluated at any arbitrary time point, enabling CORAL to capture both spatial and temporal continuity.\n\n**We appreciate your thoughtful review and hope our clarifications address your questions.**\n\n[1] Allen, Anna, et al. \"End-to-end data-driven weather prediction.\" *Nature* (2025): 1–3.", "claims_and_evidence": "Yes the claims are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed datasets and evaluation metrics make sense, following those used in previous work. (e.g. \"Fourier Neural Operator for Parametric Partial Differential Equations\" (ICLR 2021) and \"AirDelhi: Fine-Grained Spatio-Temporal Particulate Matter Dataset From Delhi For ML based Modeling\" (NeurIPS 2023))", "theoretical_claims": "Only real theoretical claims are the big O analyses in the appendices, which look right to me.", "experimental_designs_or_analyses": "I investigated all of the experimental designs and analyses. I see from appendix C and D that some hyperparameter tuning was done for SCENT for the various different datasets / tasks. I would be interested if the same level of tuning / searching was performed for the various other approaches that are compared against. If not then some of the comparisons could be a bit unfair.", "relation_to_broader_scientific_literature": "The SCENT architecture is quite similar to other approaches in the literature with a few key additions. The encoder - processor - decoder architecture with latent tokens is seen in CORAL (\"Operator learning with neural fields: Tackling pdes on general geometries\" (NeurIPS 2023)), AROMA (\"AROMA: Preserving Spatial Structure for Latent PDE Modeling with Local Neural Fields\" (NeurIPS 2024)), IPOT (\"Inducing Point Operator Transformer: A Flexible and Scalable Architecture for Solving PDEs\" (AAAI 2024)) for example. \n\nThe sparse attention layers in the encoder and decoder do not seem to be present in earlier works. “WUF” approach to limit recurrent time marching steps seems novel as well and could be effective in limiting the accumulation of errors.\n\nThe paper compares their results to a wide variety of works in the area, which is helpful and is sometimes not done in previous work, and must have required significant effort. They also make use of several previous datasets and introduce new datasets to provide a good comparison between these approaches.", "essential_references_not_discussed": "The FNO (\"Fourier Neural Operator for Parametric Partial Differential Equations\" (ICLR 2021)) seems to be one of the most effective methods compared to SCENT even though it requires regular gridded data. The authors adapt this method to work with the less regular data that is used in this work by padding out the grid, and find that it is often quite competitive with their approach. However there have been works that build on FNO to remove this dependency to gridded data (Geo-FNO from \"Fourier Neural Operator with Learned Deformations for PDEs on General Geometries\") and additionally provide further improvements (F-FNO from \"Factorized Fourier Neural Operators\" (ICLR 2023)). It would have been enlightening to cite and compare to one or both of these works or discuss why they are not suitable for the datasets used in this work.", "other_strengths_and_weaknesses": "I appreciate the effort the authors have gone through to compare to a wide variety of previous work in this area, and test them on both real world and synthetic datasets.\n\nWhile the idea of essentially providing the required timestep at input and during decoding seems simple, it does not seem to be done in previous work and this could be quite helpful in preventing the accumulation of errors during recurrent processing steps usually required for forecasting. Vaughan et al. (2024) “Aardvark Weather: End-to-end data-driven weather prediction”, used for weather forecasting attempts to avoid this by training many different models, each for a different timestep, which feels less efficient than the approach used in this work.\n\nThe SCENT approach does show generally very good performance but in some cases it is only marginally better than alternative methods, and I wonder if these other approaches received the same level of hyperparameter optimization that is seen in the appendices for SCENT for each of the different datasets / tasks. If not then perhaps they would outperform SCENT with equivalent tuning. \n\nI would have liked to see a comparison to newer FNO approaches considering that the basic version of this with a probably suboptimal approach to non regular data showed good performance.", "other_comments_or_suggestions": "Some very minor issues I noticed:\n\nLine 182 - right - latex formatting error\n\nLine 326 - left - surely reconstruction is when delta t = 0?\n\nLine 661 - treatSeach -> treats each\n\nFigure 2 + table 3 - inconsistency in name of context encoding / embedding network\n\nTable 4 - I am not sure why CORAL is more space and time continuous than the other approaches - it is still based on recursively using the processor module to step forward in time like the other approaches. OFormer and AROMA seem to be able to do basically the same things as CORAL here.", "questions_for_authors": "1. How did you select hyperparameters for the approaches you compared to? And what was your process for selecting these for your approach? Answering this will give me more confidence in how your approach performs compared to alternatives.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3jhHdgp7Af", "reviewer": "Reviewer_9YaB", "summary": "This paper presents SCENT, which is a scalable and continuity-informed spatiotemporal learning framework designed to model complex scientific data. Using a transformer-based architecture with learnable queries and sparse attention, it unifies interpolation, reconstruction, and forecasting. Extensive experiments demonstrate its satisfied performance across various datasets, offering superior scalability and robustness against sparse and noisy data.\n\n## Update after rebuttal:\nThe authors addressed the main concerns I raised, including the addition of RainNet experiments and comparisons with STFNN. While full statistical significance testing is still limited, I appreciate the substantial effort made to strengthen empirical validation. I have updated my score to 3 accordingly.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the insightful discussions, suggested papers, and constructive comments. It was a pleasant surprise to find substantial similarities as well as subtle yet important distinctions between SCENT, STFNN, and the referenced works. We found STFNN's inference mechanism and gradient-based formulation particularly innovative, and have therefore reached out to the authors for further exploration. As suggested, we will move the related work section to the main manuscript and provide more in-depth discussions on spatiotemporal forecasting methods, such as AutoST and ST-ResNet.\n\n$\\ $\n\n**1. Comparing SCENT and STFNN**\n\n**Motivation and Similarities**. SCENT aims to develop a flexible model capable of generating continuous spatiotemporal fields from sparse observations, addressing the common scenario in scientific domains where sensor coverage is limited. Similarly, STFNN effectively infers unobserved regions using its sophisticated Pyramidal Inference. Both methods assume continuous fields and handle irregular, sparse, and noisy observations robustly.\n\n**Fundamental Differences in Generalization**. However, SCENT explicitly models a family of functions, acting as a generalizable implicit neural representation (GINR). It can represent various spatiotemporal scenarios conditioned on input data from different contexts. In contrast, STFNN models one specific spatiotemporal field (e.g., PM2.5 over China), allowing interpolation and extrapolation strictly within that field, rather than generalizing across distinct fields.\n\n**Architectural Design**. Both models utilize INRs but differ in structure and intent. SCENT employs an encoder–decoder architecture with attention-based querying to generalize across tasks. STFNN is more akin to a per-task INR, enhanced with gradient-based modeling and local graph-based correction to capture detailed local variations within a single domain.\n\n$\\ $\n\n**2. RainNet: Rainfall Nowcasting**\n\nWe appreciate for the suggestion. We have strengthened our experimental evaluation by incorporating RainNet and the new nowcasting dataset. \n\n**Dataset & Task**. We use the RY product from the German Weather Service (DWD), a quality-controlled rainfall composite at 1km~$\\times$~1km spatial and 5-minute temporal resolution. Data from 2012-2016 are used for training, and 2017 for testing. The task is to predict rainfall fields for future timestamps $t \\in [5, 10, \\dots, 60]$ minutes, given four historical fields. Following the official preprocessing, we use 173,345 / 43,456 training / test instances, and downsample the original 900~$\\times$~900 resolution to 64~$\\times$~64 for faster training during the rebuttal period. SCENT is trained with a forecast horizon ($t_h$ = 60).\n\n**Results.** We report root mean-squared error (RMSE [mm$~h^{-1}$]) in the table below. SCENT consistently outperforms RainNet across all lead times, reducing MSE by approximately 30%. We attribute this improvement in part to SCENT's ability to train with variable target times $t_o$, which serves as a form of data augmentation.\n\n\n| Method|#Params|||||Lead|time|(mins)|||||| \n|--|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|\n|||**5** |**10**|**15**|**20**| **25**|**30**| **35**| **40**|**45**|**50**|**55**|**60**| \n| RainNet|1.93M|0.445|0.431|0.439|0.460|0.491|0.526 |0.559|0.589|0.612 |0.628 | 0.639 | 0.646 | \n| SCENT|3.87M|0.319|0.341|0.354|0.366|0.377|0.389|0.399|0.409|0.417 | 0.425 | 0.434 | 0.440 | \n| Improvement |-| $28.3$% |$20.9$%|$19.4$%|$20.4$%|$26.0$%|$28.6$%|$30.6$% | $31.9$%|$31.9$%|$32.3$%|$32.1$% | $31.9$% | \n| | | | | | | | | | | | Metric:|RMSE|(mm$~h^{-1}$)|\n\n$\\ $\n\n**3. Choosing M**\n\n We agree with the reviewer that selecting the OPTIMAL number of latent tokens $M$ is important. While performance generally improves with larger $M$, we observe diminishing returns beyond $M=192$ on the S5 dataset—a saturation pattern also seen in Perceiver IO. This suggests that excessively large $M$ offers limited benefit while incurring higher computational cost, and a moderate $M$ can strike a better balance between performance and efficiency.\n\n| **M**|$\\ $32|$\\ $64|$\\ $96|$\\ $128|$\\ $192|$\\ $256|\n|--|:--:|:--:|:--:|:--:|:--:|:--:|\n|**Rel-MSE**|0.460|0.433|$\\underline{0.422}$|0.425|**0.400**|0.427|\n\n$\\ $\n\n**4. On statistical testing** \n\nWe appreciate the reviewer’s suggestion. While full significance testing is infeasible due to time constraints, we provide evidence of robustness through multiple runs: for example, our small model (Rel-MSE = 0.467) shows a standard deviation of only 0.0042 across five seeds. Our evaluation also spans multiple benchmarks and a range of lead times, demonstrating consistent performance across diverse tasks and forecasting horizons. We acknowledge the value of statistical testing and will consider incorporating it in future revision.\n\n$\\ $\n\n**We hope our responses address your concerns, and we’d be grateful if you’d consider updating your score accordingly.**", "claims_and_evidence": "The paper claims that SCENT is scalable and computationally efficient. However, while the model appears manageable for small values of M, its efficiency for large-scale datasets remains uncertain. How is M determined in practice? Different values of M may significantly impact the cross-attention mechanism, yet there are no ablation studies provided to support this claim. Adding such an analysis would strengthen the evidence for scalability and efficiency.", "methods_and_evaluation_criteria": "The study does not include additional real-world datasets across different domains to better demonstrate the model’s generalizability. For example, RainNet [1].\n\n[1] Rainnet v1. 0: a convolutional neural network for radar-based precipitation nowcasting. Geoscientific Model Development. 2020", "theoretical_claims": "The paper does not introduce new theoretical results.", "experimental_designs_or_analyses": "1.\tThe paper does not include several closely related methods. Specifically, it lacks comparisons with ST grid forecasting models, such as AutoST [2] and ST-ResNet [3], as well as ST field-based methods, such as STFNN [4]. \n2.\tThe reported experimental results (Table 1 and 2) lack standard deviations and statistical significance markers (e.g., confidence intervals). \n\nReference\n\n[2] Autost: Efficient neural architecture search for spatio-temporal prediction. KDD. 2020.\n\n[3]  Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction. AAAI 2017.\n\n[4] Spatio-Temporal Field Neural Networks for Air Quality Inference. IJCAI 2024.", "relation_to_broader_scientific_literature": "The key contributions of this paper align with and extend multiple areas of ST learning, INRs, and scalable deep learning models. Compared to FNOs, it generalizes better to real-world data but needs validation on diverse domains like RainNet and include a comparisons with STFNN.", "essential_references_not_discussed": "STFNN [3] also utilizes INRs and focuses on unified spatiotemporal modeling. A direct comparison between SCENT and STFNN would help clarify their differences and contributions.", "other_strengths_and_weaknesses": "Strengths: The paper includes clear visualization.\n\nWeakness: See previous section.", "other_comments_or_suggestions": "I noticed that the related work is included in the appendix. I recommend incorporating it into the main text to ensure a more comprehensive and cohesive presentation.", "questions_for_authors": "See previous section.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The proposed method achieves state-of-the-art performance on several benchmarks. The reviewers had doubts about the limited real-world benchmarks to evaluate the method. In the rebuttal authors provided in-depth clarifications and several additional experimental results that support the efficacy of the method.
I lean towards acceptance.
Decision: Accept (poster)