OmniField: Conditioned Neural Fields for Robust Multimodal Spatiotemporal Learning
Authors: Kevin Valencia
Thilina Balasooriya
Xihaier Luo
Shinjae Yoo
David Keetae Park
Venue: ICLR 2026 Poster (2026)
Keywords: Conditioned Neural Fields
Multimodal Learning
Spatiotemporal Learning
Scientific Data
Neural Fields
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: Spatiotemporal scientific data are inherently multimodal yet sparse, noisy, and irregular; we introduce OmniField, a multimodal conditioned neural field for unified robust spatiotemporal representation learning.

Abstract: Multimodal spatiotemporal learning on real-world experimental data is constrained by two challenges: within-modality measurements are sparse, irregular, and noisy (QA/QC artifacts) but cross-modally correlated; the set of available modalities varies across space and time, shrinking the usable record unless models can adapt to arbitrary subsets at train and test time. We propose OmniField, a continuity-aware framework that learns a continuous neural field conditioned on available modalities and iteratively fuses cross-modal context. A multimodal crosstalk block architecture paired with iterative cross-modal refinement aligns signals prior to the decoder, enabling unified reconstruction, interpolation, forecasting, and cross-modal prediction without gridding or surrogate preprocessing. Extensive evaluations show that OmniField consistently outperforms eight strong multimodal spatiotemporal baselines. Under heavy simulated sensor noise, performance remains close to clean-input levels, highlighting robustness to corrupted measurements.

Published as a conference paper at ICLR 2026
OMNIFIELD: CONDITIONED NEURAL FIELDS FOR ROBUST MULTIMODAL SPATIOTEMPORAL LEARNING
Thilina Balasooriya', Xihaier Luo", Shinjae You' & David Keetae Park 'UCLA "Columbia University 'Brokhhaven National Laboratory wwvinval04@uclln edu, tnb211 90collmbiiiiioo.om edu, [xluo, s syoo,
Kevin Valencia', Thilina
gov
ABSTRACT
Multimodal spatiotemporal learning real-world experimental data is con- strained by two challenges: within-modality measurements are sparse, irregular, and noisy (QA/QC artifacts) but cross-modally correlated; the set of available modalities varies across space and time, shrinking the usable record unless mod- els can adapt to arbitrary subsets at train and test time. We propose OmniField. a continuity- aware framework that learns a continuous neural field conditioned available modalities and iteratively fuses mross-modal context. A multimodal crosstalk block architecture paired with iterative cross-modal refinement aligns signals prior the decoder, enabling unified reconstruction, interpolation, fore- casting, and cross-modal prediction without gridding or surrogate preprocessing. Extensive evaluations show that OmniField consistently outperforms eight strong multimodal spatiotemporal baselines. Under heavy simulated sensor noise, per- formance remains close to clean-input levels, highlighting robustness corrupted measurements.
INTRODUCTION
Spatiotemporal data gathered from scientific experiments and observations are inherently multi- modal. In climate science, for instance, measurements of temperature, humidity, and wind speed are collected concurrently model atmospheric dynamics (Hersbach et al,, 2020). Similarly, air pol- lution studies rely data from various sensors measuring particulate matter, ozone, and nitrogen oxides (U.S. Environmental Protection Agency| 2025). This multimodality extends across numer- ous fields, including materials science, where stress and strain measured together (Sutton et al., 2009), ,, particle physics, which analyzes energy deposits and particle tracks simultaneously (ATLAS) Collaboration, 2008), and biology, where multimodal imaging captures diverse cellular processes (Bischof et al. 2024). Yet two practical challenges persist (Fig. T]:
Data challenge: measurements within each modality sparse and irregular, with QA/QC noise that can be sensor- or instance-specific (Cressie & Wikle (2011).
Modality challenge: coverage and fidelity vary across modalities. Sensors sit at different loca- tions, exhibit distinct sparsity patterns, and have modality-peecifie noise structures that can shift
over space, time, and even individual devices (Hall & Llinas 1997..
Prior efforts regarding these two challenges can be reviewed from two complementary perspectives: data and model. From the data perspective, pre-processing techniques like filtering, gridding/krig- ing, and imputation regularize irregular. noisy samples by constructing surrogate dataset before learning. While this can stabilize downstream training, it introduces systematic side effects like smoothing bias, which attenuates extremes and high-frequene as the surrogate is treated as ground truth and the uncertainty from these guesses is not carried for- ward in the analysis (Little & Rubin 2002). From the model perspective, methods designed for irregular sampling like continuous-tim latent models, neural ODEs, and graph-dynamic formula- tions respect -on-gdidded timing and can reduce reliance heavy pre- precescce...g. However, they typically assume fixed observation operators and a shared sampling index across modalities. In practice, sensor supports, coverage, and noise characteristics vary by modality, location, and time,
structure, and uncertainty collapse,

---

Published as a conference paper at ICLR 2026
(a) Noisy
(c)
Sparse sensor
Non-grid
Sensor malfunction
Moving sensors
Ozone
PM
1.0
2.5
no readings
fg
(b) Missing sensors
CO
NO2
Multimodal alignment
Missing modality
m1
m2
m1
m2
m2
Figure 1: (a) Data challenges: sparse, irregular, noisy, and dynamic measurements. (b) Modality challenges: misaligned supports, modality-specifft noise, and variable modality availability. (c)
Real-world example: ambient air pollution data collected from hundreds of monitors.
yiclding likelihood misspecification when these assumptions are violated (Chen et al. 2023 Gravinal
al. 2024 Feng et al. 2024).
We address these challenges with OmniField, a unified continuity-awarc framework that extends the principles of conditioned neural fields (CNFs) (Xie al.) 2022). Our approach first builds robust, high-fidelity CNF backbone capable of handling the data challenges and then introduces a novel, iterative fusion architecture and training strategy address the modality challenges. Our contributions are as follows:
We introduce OmniField, continuity- multimodal conditioned neural field for complex scientific observational data without gridding or surrogate preprocessing.
We demonstrate novel mechanisms for learning spatiotemporal multimodal correspon- dences while robust to noise: multimodal crosstalk (MCT) block for cross-modal information exchange and iterative cross-mit cross-modal refinement (ICMR) that progressively aligns signals.
We carry out extensive experiments which demonstrates the proposed model outperforms eight strong baselines, yielding a 22.4% average relative error reduction across our benchmarks, and remains near-clean under heavy simulated sensor noise, confirming robustness corrupted
measurements
We contribute ClimSim-LHW reflect realistic observational sparsity and ML-ready EPA-
AQS dataset, enabling systematic evaluation under data- and modality-challenge regimes.
RELATED WORKS
Neural Fields. Neural fields, implicit, coordinate-based networks, encode continuous functions that map coordinates to signals, enabling compact, high-frequency detail and resolution-fres sam- pling (Xie et al. 2022; Sitzmann et al. [020). They underpin high-fidelity reconstruction and syn- thesis across vision and science, with canonical instances including NeRF for novel view synthesis
and signed/occupancy fields for geometry [Mildenhall et al. 2020
Park et al. 2019).
Conditioned Neural Fields. Conditioned neural fields (CNFs) learn maps from auxiliary context (e.g., parameters, coefficients, boundary/sensor data) to continuous fields, so a single model spans family of signals. This operator-learing view is exemplified by Fourier Neural Operators (FNOs), which learn parametric PDE solution operators efficiently in spectral space (Li et al. 2020). Trans- former operator learners extend this perspective with attention over discretizations (Li al. 2023), while neural-field formulations general geometries demonstrate CNFs that natively accommodate complex domains (Serrano et al. (2023). Building on these trends, SCENT shows a scalable, explic- itly conditioned spatiotemporal CNF for scientific data that unifies interpolation, reconstruction. and forecasting, highlighting CNFs as practical vehicle for continuous spatiotemporal modeling (Park
al. 2025
Multimodal Conditioned Neural Fields. Multimodal CNFs fuse heterogeneous evidence as con- ditioning signals to improve reconstruction and forecasting under sparse partial observations. PROSE-FD illustrates this direction for physics: a multimodal PDE foundation model that learss multiple fluid-dynamics operators and leverages diverse inputs (e.g., coefficients, coarse fields, sen-
2

---

Published as a conference paper at ICLR 2026
streams)) to generalize across tasks (Liu et al. 2024). In vision, MIA meta-leams implicit neural representations with multimodal iterative adaptation, offering strong performance with limited ob- servations but relying bi-level optimization (Lee al. 2024).
BACKGROUND AND SETUP
Data. Let C Rd be the spatial domain and T C C time. We consider a fixed catalog of modalities Mall (1, M). For modality m, measurement at (x,1) X T is you (x, t). Example: in air quality, Mall = (PM2.5, O3, 2.
Context (inputs). At imput time iim each available modality E Min C. Mall contributes finite, irregular set Um = {(x, ym (x, {n,)) and C = {Um m E MMn]- Example: if only PM2.5 and NO2 arrived, then Min (PM2.5, NO2} and C = (UPM2.0 UNO.).
Queries (largets). Predictions are requested at time lont = lin + A/ with 0 < Al < ln (a time horizon hyperparameterr) for target modalities Mout S MMall and query locations Qm {x), giving Q = (m, x) m E Mout. X E Qm Example: for cross-modal prediction of ozone at current time, set At 0, Mout = {O3), and choose Qo, (locations of interest).
Goal. Our objective is to build neural network Fo that maps irregular, noisy, multimodal observa- tions continuous spatiotemporal field, avoiding gridding and heavy imputation.
Tasks. Under this setup, the network Fe(-) is designed to handle four tasks:
1. Reconstruction: At 0, Mout S Min and Qm {x E Um (predict provided observations).
2. Spatial interpolation: At = 0, Mout S Min. and Q. contains unseen locations not in Um-
3. Forecasting: At > 0 with arbitrary Q. (predict at future times).
4. Cross-modal prediction: some E Mous MMin (predict modalities not present at input).
Solution Strategy. The setup above calls for a function that can be queried at arbitrary space-time coordinates. A neural field provides this abstraction by mapping (x, L) to value via a coordinate- based network y Fel (x, 1) (Xie et al. 2022]. However, vanilla neural field typically models single signal (c.g.. one scene experiment) with fixed parameters, and does not directly general- ize across multiple signalsfinstanees without retraining. To model a family of signals with shared parameters, we adopt conditioned neural field: the network takes the coordinates and instance- specific summary derived from observations of that instance ý = Fe (x, f; c). where c is the latent embedding encoding the identity or properties of specific signal from a of signals.
METHOD
Overview, Following the solution strategy in Section 3, we introduce OmniField, an encoder- processor-decoder architecture (Jaegle et al. 2022 Park al. 2025). The full model is
Fo {Dw.m MEE., o Pe Eg.
0 = (ó, 16, w),
which maps query (x, t) and a context set to modality-speciff predictions. Specifically,
Encoder & (context to local summary). Given C' and query (x, the encoder builds query- local, coomuutaaionnnnvarrann summary c(x, 1) = (C x, 1), aggregating irregular observations (across space, time, and modalities) into lend-length representation.
Processor Py (coordinates + context to latent field). The processor fuses multi-resolluioon co-
ordinate encodings with the context summary: h(x, t) = Px()(x), r/(1), c(x, where 7, encode spatial and temporal coordinates. This stage constitutes the conditioned neural field. Decoder Du.m (latent to modality output). Each modality uses a lightweight decoder to produce
predictions: 1m (x, t) = (,,
The encoder-prrocessor-dd
backbone provides the right scaffold, but sparse, noisy, multinodal observations raise three practical questions: Q1: how do we mitigate low-frequency bias and pre- serve high-freue frequerreeecy detail? Q2: how do we align signals collected on mismatched supports, scales, and noise profiles? Q3: how do we operate when the set available modalities changes across space and time? Next, we address each of these questions in turn.
3

---

Published as a conference paper at ICLR 2026
(a)
(b)
(c)
postion k.30
signal
postiin 2.
sigral
M1: Og
MCT Block
Fourier Features
Fourler Features
Geueasian
Eyi
Mo CO
h
Loamable Ruandionnyy traialiiad
(MxD])
Learaaio Queries Sinusoifal Intialization
(Mx00)
teature
Cross Attention
Addition (broadcast)
Concatenate
(d)
Interpolation
h (nxc))
h
Ø (nxd)
(sxid)
MCT Block
MCT
MCT Black
average panling
average pooling
Reconstruction
Forecasting
feature
feature
Figure 2: Overview of Our Approach. We illustrate (a) prior SCENT (Park et al. 2025 encoder, (b) our proposed encoder (E), (c) our multimodal crosstalk (MCT) block, and (d) our proposed OmniField architecture equipped with iterative cross-modal refinement (ICMR) strategy.
4.1 GAUSSIAN EMBEDDINGS & LEARNABLE QUERIES
Learning fine-scale structure is notoriously hard for conditioned neural fields (CNFs): training amounts to fitting continuous function to irregular, noisy signals, and standard CNFs exhibit low-frequency (spectral) bias. In our setting, sparse, noisy scientific observations, this bias is am- plified by two factors: (i) coarse, fixed-frequency positional encodings (e.g., integer log-stepped bands) that under-represent high frequencies; and (ii) randomly initialized learnable query tokens that provide poor initial coverage of the spectrum, creating bottleneck for continuity-awar rep- resentations. To address this, we replace fixed sinusoidal Fourier features with Gaussian Fourier features (GFF), sampling
[dxx with entries Bg No0 o²) so that, for a coordinate x, ((x) concato (2x sin sin(2 Bx)) R2d yielding richer, less-biased spectral represen- tation that better captures high-fre gigh-frequency detail (Tancik et al., 2020). Additionally, we introduce sinusoidal initialization stabilize training and ensure balanced frequency coverage. Concretely, we initialize the M learnable queries with compact multi-scale sinusoidal pattern using log-spaced bands and unit-norm scaling (s d d'1/2), Our experiments show that GFF and sinusoidal initial- ization together improves CIFAR-I0 reconstruction loss and climate forecasting performance by x2. 74.74 and 30% (Table respectively. Full derivations, ablations, and discussion are deferred to Appendix B
4.2 MULTIMODAL ALIGNMENT
While general conditioned neural fields F(-) provide continuity-awarc representations, single pass of encode-xprocess sooood--ppooess--deeooo -decode
-decode can under-express fine- grained, cross-modal correspondences when modalities differ in support, scale, and noise (Serrano al. 2023 Park et al. 2025). Prior work such as MIA meta-leams per-instance implicit representations for sparse natural images (Lee et al.] 2024]; in contrast, ble and efficient for spatiotemporal, multimodal data. Specifically, we introduce two advances: a multimodal crosstalk (MCT) module that concatenates per-modality CNF tokens and conditions the processor with a lightweight global multimodal code, and iterative cross-modd cross-modal refinement (ICMR) that revisits this fusion over several processor steps. In effect, the model merges multimodal CNFs while being informed by global structure at each step, enabling adaptive alignment across scales and improved robustness to heterogeneous noise.
seek single-step, Shared-parameer training procedure that remains sta-
Multimodal Crosstalk (MCT). Let our unimodal encoder denoted as Em (Fig. 4). Our MCT outputs intermediate features follows:
h = MCT ((U)) mEM.i =
④
4

---

Published as a conference paper at ICLR 2026
Temperatura
Humidity (H)
Wind speed (W)
Ar. hours Forecasting
Muttimodal
Spatiotemporal
Temperature
Humidity Wind speed
Model
Ground Truth
OmniField
SCENT
PROSE-FD
MIA
OFORMER
RESNET
K
300
RMSE
4.38
5.02
9.44
8.91
12.2
13.3
260
H
12.5
2.22
2.51
3.33
3.30
4.55
4.99
7.5
w
-2.5
Figure 3: Qualitative Comparisons on ClimSim. Provided with highly sparse yet multimodal observations, models generate full-field forecasting At hours. We provide comparisons against the ground truth. RMSE against the Ground Truth is shown in white boxes.
where Em (U E Raxd denotes unimodal features, O denotes concatenation across modalities m, denotes addition with broadcasting, and Rixd represents global feature summarizing intermediate multimodal features. P denotes a multimodal processor equipped with multiple self- attention layers. The global feature z serves critical dual function; provides global information aggregated from all multimodal inputs to facilitate cross-modal communication, while also acting as compact information bottleneck that evolves throughout the network layers. 2 is given from prior layer as discussed further below.
Iterative Cross-Modal Refinement (ICMR). our ICMR strategy, the global feature acts as a communication bridge, relaying global multimodal information between the unimodal encoders. Given MCT blocks, we define ICMR follows. For le O....,
h (R) MCT(UUm JMEM: 2(()) B
(k+1)
(k) 10
Finally, the multimodal neural field is g (B-1)) The initial global feature (0)) filled with zero.
4.3 FLEXIMODAL FUSION
Fleximodal fusion treats modalities as m .M with a presence mask Tm, enabling one model to operate any input subset: absent channels are zero-gated at the encoders, masked out in cross- attention, and excluded from the loss (only supervised targets contribute), preventing leakage from missing inputs; this departs from training-only dropout (ModDrop; (Neverova al.] (015)) and
aligns with recent reximooal/nissinn-oddaii methods (Han et al.) 2024 Wu et al. 2024). Scien- tific datasets frequently have sensors variables intermittently unavailable, making such masking essential for robust deployment. In EPA-AQS air-quality data, true daily absences naturally set Tm for some m, so we evaluate with native day-specific masks without imputation. For fair- ness, we apply the same fleximodal masking across all baselines inclludig OmniField at train and test. We defer an in-depth discussion Appendix
EXPERIMENTS
We hypothesize that a continuity-infon informed multimodal model, instantiated as CNFs, can address the data and modality challenges outlined in Section and Fig, We begin by surveying prevalent strategies for multimodal training with sparse and serying-lcction sensors and summarizing their comparative performance (Section 5.2 We then present primary experiment: forecasting output

---

Published as a conference paper at ICLR 2026
fields from sparse imputs and benchmarking against eight widely used baselines simulated variant of ClimSim Yu et al. (2023), a climate reanalysis dataset. Next, we validate on real data air air-pollution network that exemplifies these challenges- again comparing the eight baselines with our approach, OmniField. Finally, ablation studies on CIFAR-10 (spatial), RainNet Ayzel et al.] (satiiottto) and ClimSim (multimodal spatiotemporal) quantify the contributions of our modeling choices, demonstrating the efficacy and robustness of continuity- informed neural-field conditioning across data regimes.
from EPA-AQS-
5.1 EXPERIMENTAL SETUP
Datasets. We evaluate our proposed method against baseline methods using four diverse datasets. Detailed information on data statistics for these datasets can be found in Appendix A
ClimSim-THW. We use a subset 10,000 consecutive 1-hour intervals from ClimSim and retain three modalities: temperature (T'; K), humidity (H; kg "g and wind speed (W; ms= The reanalysis fields are defined over 21,600 unique sensor locations; we select the lowest of the 60 vertical levels. To study sparse, partially co-located sensing, each modality is observed on 432 locations, with a triple overlap [T n n W| = 108, pairwise-only only regions [T n H| = [T n |w | n W 81, and modality-ecclusive regions [7] H [W| = 162, yielding union of 837 unique input locations. We use the time horizon 4, = hours.
EPA-AQS. The U.S. Environmental Protection Agency's 's Air Quality System (EPA-AQS) the national repository of quality-assured ambient air- air-pollution measurements. We use records from 1987-2017 and ingest the data as-is across six modalities Ozone (Os, ppm), PM25 (H/m²), PM10 (u/m²), NO₂ (ppm), CO (u/m² ( and SO2 (p/m²) 1 and parse it using calendar days as the reference timestep. Model inputs use native stations on the anchor day, and we use the time horizon 4, days.
CIFAR-10. We use CIFAR-10 as a purely spatial benchmark. We use all 32x32-resolution 60,000 natural images for ablating the reconstruction qualities.
RainNet. This dataset consists of 5-minute interval precipitation records from DWD radar com- posites (900 900 km domain). Following Park et al. (2025), we downsample this to 64 64 grid, and use the fixed split of 173,345 and 43,456 instances for training and validation, respectively.
Baselines. We benchmark OmniField against eight well-acknowledged and advanced
Table Comparing model capacities in spa- tiotemporal learning.
models: UNet (Ronneberger et al.; 2015),
ResNet (He al. (2016), FNO Li et al. (2020),
Model
Mush Agnostic Learning
Space Continuity
Time Continuity
Single-Step Training
OFormer Li et al. (2023), CORAL Serrano
Unet
X
X
X
al.
2023). PROSE-FD Liu et (2024).
ResNet
X
X
X
/
MIA
Lee et al.
2024), SCENT Park et al.
FNO
X
/
x
/
2025 Table I compares their reported capac- ities against our OmniField. Most baselines are inherently multimodal, hence we extend the prior architectures following the Mid-Fusion (Section 5.2 Nagrani et al. (2021)) scheme.
OFormer
x
x
CORAL
/
X
PROSE-FD
/
/
MIA
X
X
SCENT
7
V
OmmiField
For models not mesh-agnostic, we take the data a 1-dimensional vector attached with coordinate positions. All experiments were performed on a single NVIDIA H100 80GB HBM3 GPU.
5.2 MULTIMODAL GAINS FOR SPARSE SCIENTIFIC DATA
We first attempt to answer whether multimodality enhances performance across distinctive train- ing strategies for multimodal sparse data. We evaluate four strategies: (i) -rectrion-restrictt each modality to the common sensors so all modalities observe the same sensors; (ii) Interpola- tion-impute missing entries so every modality is defined on the union of all sensor locations; (iii) Mid-Fuusionn-presevee the original sparsity, encode each modality separately, then fuse features mid-network to condition single neural field; and (iv) our ICMR, which preserves sparsity while sharing cross- modal context before conditioning a unified neural field. We train OmniField either on single modality (T) or three modalities (T.H.W). in both cases forecasting all three modalities. Results: Fig. aa) shows the results. Multimodal inputs always improves the forecasting perfor- mances across all multimodal strategies and modalities. While Co-Location performs worst due

---

Published as a conference paper at ICLR 2026
[a]
(b)
(c)
ppm
out-m: 0
0 4..42
2.4
2.0
2.8
1.
Mai (TILW)
EDUAL
OFORNER
UNE:
MA
PROSaFI SCENT
5:622
18.3
14.9
12
10
10
kg
out-m:
8-
2g1o
S0,
50.9
PM
0.
w/m
PMp.
0.5
m am
aut-m:
CO
NO;
4.2
OFORMER
MIA
SCENT
Co-Loc
Interp
Mid Fusion
ICAR
JMG1-2
[Mil-4
JAC|-6
RESNET
PROSETO
Omnifieeld
Figure 4: (a) Multimodal training results on OmniField, comparing four training strategies (Co- Lo--CooLooction; Interp=interpolation; out-n=queried-ou modality) on ClimSim. (b) EPA-AQS baselines are trained for two, four, and six modalities and compared. (c) Models trained on full six modalities in EPQ-AQS are compared. We select six representative models for the illustration. All values RMSE in physical units.
to limited spatial coverage, Interpolation records slightly better. The superior performance of both Fid-Fusion and /CMR promotes the values of incorporating modelity-ppecffic sensors into model training.
5.3 BASELINE COMPARISONS ON MULTIMODAL FORECASTING
Having established in Section 5.2 that multimodality helps under sparse, partly co-located sens- ing, demonstrate how OmniField compares widely used baselines when forecasting real scientific fields from sparse multimodal inputs.
5.3.1 COMPARISONS ON CLIMSIM-THW
Our hypothesis is that continuity-ifformen conditioning with ICMR will outperform Co-Location, Interpolation, Mid-Fusion alternatives by preserving native sparsity while aligning modalities in a shared latent field prior to decoding. We use the ClimSim-THW setup from Section All base- lines trained with identical splits and normalization (per-modality z-score). Baselines inherit the originally suggested model sizes with additional overhead related to multimodal adaptation. We report per-modality errors over (T, H, W). Results: Table shows that OmniField achieves the best mean error and leads on most per-modality comparisons. Among triginally-unimode base- lines, Mid-Fusion is stronger than Co-Location and Interpolation, confirming that retaining native, modality-specine supports matters. Nevertheless, Mid-Fusion remains below OmniField, indicating that exchanging cross-modal context before conditioning a unified field is critical. Across archi- tectures, CNF-style models outperform (UNet/ResNet). Natively multimodal designs (e.g.. PROSE-FD, MIA) benefit from multiple chan- nels but still trail OmniField, suggesting that continuity- conditioning and iterative refinement confer additional gains beyond simple multi-channel fusion. Qualitative comparisons Fig. 3 illus- trate set of inputs, ground truths, and field forecasting results from six models. Given mere 3.87% sampling rate, OmniField is capable to accurately predict the underlying phenomena.
(e.g.. FNO/OFormer) and standard CNNs
5.3.2 COMPARISONS ON NATURALISTIC EPA-AQS
EPA-AQS stress-tests robustness under day-to-day nonstationarity, irregular station layouts, and asynchronous sensing. We test whether OmniField yields reliable multi-pollutant forecasts from sparse, partially co-located inputs and whether adding modalities improves performance when sta- tion supports vary by day. Inputs and targets are taken on their day-specific masks, while no impu- tation is performed. We evaluate: (i) modal-subset scaling, training/evaluating on |M| E {2, 4, 6) pollutants-Wy M2 [Og, )M,.s), M4 adds (PM10, NO2), and M6 adds (O2) and mea- suring forecasting error for O₂ and PM2.;: and (ii) full full-modal comparison, training/evaluating all methods on M6 and reporting per-pollutant performance. Results: Fig. 4b) shows that perfor-

---

Published as a conference paper at ICLR 2026
Table Baseline Comparisons for Multimodal Forecasting on ClimSim-THW, Mid-Fusion column denotes Mid-Fusion except natively multimodal (denoted in Architecture as MM), in which case respective multimodal fusion method used. T=Temperature (K); H-Humidity (10-9kg kg kg-l); W=Wind speed (ms- "); # Params-number of model parameters; All values are RMSE in physical
units, (refer to Appendix D on model sizes for EPA-AQS)
Co-Location
Interpolation
Mid-Fusion
Model
Architecture
Params
T
H
W
I
W
T
II
W
UNET
CNN
53.1M
5.31
1.96
5.63
4.60
1.84
5.52
4.49
1.74
5.41
RESNET
CNN
1.2M
9.14
3.80
5.43
8.72
3.51
5.42
8.13
3.26
5.36
OFORMER
Transformer
2.1M
12.56
4.99
6.02
11.27
4.20
5.98
11.20
4.24
5.84
FNO
Operator
1.MM
5.98
1.89
8.51
1.17
1.80
7.25
3.36
1.39
7.19
CORAL
Operator
2.0M
14.19
4.98
7.95
13.61
1.97
7.02
13.12
3.80
6.76
SCENT
CNF
29.3M
1.86
1.28
5.15
1.86
1.28
5.15
1.52
0.99
5.07
PROSE-FD
Operator (MM)
16.0M
5.20
1.65
5.30
5.18
1.65
5.30
5,20
1.65
5.30
MIA
CNF (MM)
0.3M
4.63
1.74
5.26
4.69
1.61
5.19
4.43
1.63
5.26
OmniField
CNF (MM)
37.4M
1.93
1.00
5.07
1.40
0.81
4.97
1.07
0.66
4.86
Table 3: Ablation of Our Architecture Vindicates activated modules. CIFAR-10 shows MSE, and ClimSim- columns show RMSE in physical units. Promotion is noted next to performance as relative increase in error.
ClimSim-LHW (RMSE)
Sinusoidal Init.
CIFAR-IK (MSE)
GFF
ICMR
Temperature (K)
Humidity
Wind
(10 kg
(m
/
0.0007 (x1.00)
1.07 (x1.00)
0.66 (x1.00)
4.86 (x1.00)
X
/
0.0097 (x 13.66)
2.61 (x2.44)
(x2.622
5.35 (x1.10)
V
0.0011 (x1.57)
1.08 (x1.01)
0.71 (x1.07)
4.87 (x1.00)
V
J
X
0.0053 (x7.57)
1.56 (x1.45)
0.82 (x1.24)
4.91 (x00)))
X
X
V
0.0098
(x 14.00)
1.79 1.79 (x1.67)
1.30 (x1.96)
5.24 (x100
X
X 0.0145 (x20.71)
2.92 (x2.72)
1.86 (x2.81)
5.37 (x1.11)
mance scales monotonically from M2 M4 + M6 for most methods, with OmniField leading at each level-evidence that more modalities help even when station supports differ daily. In the full-modal comparison (Fig. [c),, some baselines display strengths on particular pollutants; never- theless, OmniField attains the overall state of the art across pollutants.
5.4 ABLATIONS
We isolate which architectural choices drive gains across regimes of increasing difficulty: (i) purely spatial grids (CIFAR-10), (ii) spatiotemporal grids (RainNet), and (ii) sparse, multimodal spa- tiotemporal point-clouds
-TWW.. We ablate three components introduced for CNF: (a)
GFF for query and token emheddings; (b) sinusoidal initialization for cascaded latents; and (c) ICMR (replacing with a Mid-Fusion fallback). Each toggle changes one component while keep- ing others fixed. Results: Table B summarizes three consistent trends. (i) On CIFAR-10 recon- struction, combining GFF + sinusoidal init + ICMR records the best performance, substantially im- proving gigh-freueency recovery over any partial variant. (Appendix E. (ii) On RainNet (unimodal spatiotemporal), our architectural advances helps outperform both SCENT and the RainNet base- lineS (Fig. d)). (ii) On ClimSim-THW each component helps, and the full model consistently reduces error across T, H. and W. Overall, the ablations support our design choice: frequency-riet embeddings plus a continuity-award iteratively refined latent are necessary to perform well across spatial, spatiotemporal, and multimodal regimes.
5.5 ROBUSTNESS TO NOISY SENSORS
Observational networks exhibit outliers, dropouts, and calibration drift. A desirable property is ro- bustness corrupted -duullyideally the model should route information through cleaner channels
8

---

Published as a conference paper at ICLR 2026
(a) Temperatare (K)
(b) Humidity (10-"kg lg kg
Wind speed (m
Nowcasting (mm/h)
5.2-
ICMR
1.50
[CMR
ICMR
RainNet
SCENT
RMSE
1.25
RMMS
0.5
OmmiPioed
1.00
5.0
0.4
0.75
4.9
0.0
0.5
1.0
2.0
0.0
0.5
1.0
2.0
0.0
0.5
1.0
2.0
20
40
60
Noise
Noise (0)
Noise
Lead Time (mins)
Figure 5: (a)-(c) ICMR contrasted against noise severity. (d) Our OmniField outperforms both SCENT and RainNet on an established rainfall nowcasting task.
in an increasing amount instance-level
and suppress noisy ones. On ClimSim-THW, we compare ICMR versus Mid-Fusion under increas- ing degree of noise corruption. For each sample, we uniformly choose k E {1,2} modalities from {T, H.. W} and add Gaussian noise with standard deviation E {0.5, 1.0, 2.0} scaled by the sam- plc's observed input standard deviation for that modality. At most two modalities are corrupted per sample, so at least one remains clean. Evaluation uses clean inputs and targets. Results: Fig- a) (c) shows a clear advantage on robustness: ICMR remains near its clean-input accuracy across all corruption scales, whereas Mid-Fusion deteriorates steadily as noise increases. At the highest cor- ruption level, Mid-Fusion incurs large errors on temperature and humidity and noticeable drift on wind, while /CMR remains virtually unchanged on all three targets. Even under mild noise, ICMR's metrics move only marginally, indicating that its iterative cross-modal refinement routes informa- tion through cleaner channels and suppresses corrupted ones. In contrast, Mid-Fusion- lackiig this pre-conditioning eichange-amplife to the shared representation. For wind, Mid-Fusion shows a small error dip at low a: a noise- as-regularization effect where tiny zero-mean perturbations during training discourage overfitting to idiosyncratic station readings and slightly improve clean-test error; the gain vanishes and reverses as a increases.
noise introduced in any single modality and propagates
5.6 ADDITIONAL EXPERIMENTS
We extend our evaluation with compute/scale measurements, long-horizon rollouts, and downstream transfers. Overall, OmniField sits on a favorable accuracy-efficiency frontier, stays stable up to 48 h unrolled horizons, and learns representations that transfer across modalities and tasks (Ap- pendix Tables 8 Beyond aggregate scores, these experiments probe three deployment-relevant questions how much compute is needed, whether errors compound under rollout, and whether learned fields remain useful off-distribution
are central for practical use.
(A) Efficiency and scalability. We report ClimSim-THV forecasting RMSE together with param- eter counts and forward-pan FLOPS (Appendix 4 At an approximately matched budget (~100M parameters), OmniField substantially improves over FNO (e.g., RMSE 1.10 vs. 2.96) while re- maining far cheaper than PROSE-FD in FLOPP. The benefit is not capacity-only: a 1.3M-parameter OnniField variant already outperforms a much larger FNO configuration (Appendix 4 and scaling OmniField further continues to improve accuracy. Notably, these trends hold consistently across T/H/W rather than being driven by single variable, suggesting the gains come from the architec- ture's 's multimodal field coupling rather than metric-specific effects. This positions OmniField practical option when both accuracy and inference cost matter, e.g., for high-frequency forecasting or large ensembles.
ICMR iteration sensitivity. Sweeping refinement stages L. E (1,3,5, 5, 10) (Appendix 5) shows accuracy improves with more refinement; for example, moving from T.=1 to 1.=10 reduces THW RMSE from 1.383/0. /2.83/0900//5.00 5.289 to 0.95/0. 0.95/0.60/ /0.000/.//.. with diminishing returns beyond L R 5. Im- portantly, the FLOPs increase sublinearly relative to the error reduction, indicating that refinement an efficient way to allocate capacity within fixed backbone. In practice, this sweep motivates lecting L as a tunable knob: smaller L for strict latency budgets and moderate L for accuracy-critical settings.
(B) Long-horizon forecasting stability. We train OmniField using either 6h or 48 h supervised horizons and evaluate 6/24/48 h rollouts via normalized RMSE (Appendix (). With 48 h training.

---

Published as a conference paper at ICLR 2026
OmniField remains stable up to 48h (e.g.. 0.063/0.095/0.777 at 48h), while training only 6 degrades more at long rollouts. In contrast, naive FNO unrolling becomes unstable and diverges at longer horizons, consistent with compounding error; this supports our warp-unrolling design Park et al. (025). The gap between 6 and 48 h-trained OmniField suggests that long-horizon super- vision is not merely improving one-step accuracy, but calibrating the dynamics under repeated ap- plication. This stability pattern suggests that reference-aligned prediction helps prevent systematic drift, which is a common failure mode in long rollouts..
(C) Downstream generalization across tasks. OmniField's learned field representations transfer beyond the primary regression setting (Appendix (7) On CESNET C Koumar al. (2025). OmniField
improves over a strong baseline (MSE 0.755 vs. 0.859 for Time.ixxr Wang et al. (2024)). On sparse-pixel CIFAR-10 (20% observed), fine-tuning reaches 71.8% accuracy. On SEVIR Veillette et al. 2020)) anomaly detection, OmniField reaches ROC-AUC 0.97 (vs. 0.96 for an FNO baseline). These results span distinct input structures (irregular sparse pixels, multivariate sequences, and spa- tiotemporal radar imagery), suggesting that the continuous-field parameterization learns reusable primitives rather than dataset-specific shortcuts. Moreover, the competitiveness on classification and detection indicates that the learned features not limited to reconstruction, but also support discriminative objectives after minimal adaptation.
(D) Randomly missing modalities. Randomly dropping a fraction p of instances has limited impact (Appendix 8 at p=0.5, OmniField remains close to full-modality training (e.g., THW RMSE 1.11/0.711 /0.71//./.8 vs. 1.07 /0.66/4.86 p=0), indicating robustness to intermittent modality absence. This robustness is useful in realistic sensing pipelines where channels can be delayed, corrupted, or unavailable without requiring specialized retraining for each failure pattern. The mild degradation also suggests OmniField can implicitly compensate via cross cross-modal field consistency, rather than relying on any single modality.
two modalities during training for
DISCUSSION AND CONCLUSION
OmniField addresses the two practical obstacles introduced in Section iiregular, noisy mea- surements and variable, partially co-located modality coverage- by learning single continuity- aware neural field and repeatedly exchanging cross-modal context before decoding. The multi- modal crosstalk (MCT) module and iterative cross-modal refinement (ICMR) align per-modality tokens with a lightweight global code, while fleximodal fusion carries presence/quality masks so the same model operates on any subset of inputs. This design unifies reconstruction, spatial interpola- tion, forecasting, and cross- modal prediction without gridding or imputation, and empirically yields robust gains: multimodality consistently helps under sparse sensing (Fig. aa), OmniField outper- forms strong baselines on ClimSim- THW (Table 2). scales with added modalities on EPA-AQS (Fig. (b)), and remains stable under missing/irregular inputs (Fig. 5).
Limitations remain. Compute and memory scale with the number of tokens (e.g., stations) and la- tent capacity, which can challenge extreme-scale deployments; the current decoder provides point estimates without calibrated uncertainty; generalization under domain shift (seasonal changes, sen- sor re-sisings, policy-driven coverage changes) not yet fully quantified; and very long forecasting horizons beyond those studied here may require additional temporal structure.
These gaps suggest clear next steps. Efficiency can be improved via adaptive tokenization/prun: ing and model distillation (potentially with mixture-of-experts routing); reliability via uncertainty- aware objectives (e.g., quantile or heteroscedastic losses) and ensembling; and robustness via continual/domain-adaae
training. For longer horizons, hierarchical temporal warping or hybrid dynamical priors are natural extensions. Finally, we plan to incorporate and evaluate OmniField in broader scientific settings where multimodal, irregular sensing is ubiquitous-climan sis/forecasting, air-quality operations, materials characterization, particle physics, and multimodal bioimaging-llveereee leveraging the fleximodal design to handle missing or asynchronous channels in real deployments.
reanaly-
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was supported by the U.S. Department of Energy (DOE), Office of Science (SC), Ad- vanced Scientific Computing Research program under award DE-SC-0012704 and used resources of the National Energy Research Scientific Computing Center, a DOE Office of Science User Facil- ity using NERSC award NERSC DDR-ERCAP0030592.
REFERENCES
ATLAS Collaboration. The atlas experiment the cern large hadron collider. Journal of Instrumen- tation, 3:S08003, 2008. doi:
Georgy Ayzel, Tobias Scheffer, and Maik Heistermann. Rainnet v1. O: convolutional neural net- work for radar-based precipitation nowcasting. Geoscientific Model Development, 13(6):2631- 2644, 2020.
Johanna Bischof, Georgina Fletcher, Paul Verkade, Claudia Kuntner, Julia fernandez-Rodriguee. Linda Chaabane, Leor Ariel Rose, Andreas Walter, Michiel Vandenbosch, Mare A. M. van Zandvoort, Assaf Zaritsky, Antje Keppler, and Maddy Parsons. Multimodal bioimaging across disciplines and scales: challenges, opportunities and breaking down barriers. npj Imaging, 2(5),
2024. doi: 10.
Yuqi Chen, Kan Ren, Yansen Wang, Yuchen Fang, Weiwei Sun, and Dongsheng Li. Contiformer: Continuous-time transformer for imregllar time series model- ing. In Advances in Neural Information Processing Systems (NeurIPS), 2023, URL attps::/procee:
932 33220008eee642200111778681-----3
Noel Cressie and Christopher K. Wikle. Statistics for Opatio-Tempont Data. Wiley, Hoboken, NJ, 2011. ISBN 9780471692744.
Yutong Feng, Qiongyan Wang, Yutong Xia, Junlin Huang, Siru Zhong, and Yuxuan Liang. Spatio- temporal field neural networks for air quality inference. In Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI), pp. 7260-7268, 2024. doi: 10.24963/ijcai. 2024/803.
Alberto Gravina, Daniele Zambon, Davide Bacciu, and Cesare Alippi. Temporal graph odes for ireeularly-sampled time series. Proceedings of the 33rd International Joint Conference on Artificial Intelligence (ICAA), pp. 5025-4034 2024. doi: 00.24963/jccai222244444
David L. Hall and James Llinas. An introduction to multisensor data fusion. Proceedings of the IEEE, 85(1):6-23, 1997. doi: 10.1 109/5.554205.
Xing Han, Huy Nguyen, Carl Harris, Nhat Ho, and Suchi Saria. Fusemoe: Mixture-of-experrs transformers for fleximodal fusion. Advances Neural Information Processing Systems, 37: 67850-67900, 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 770-778, 2016.
Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Muñoz-Sabater, Julien Nicolas, et al. The era5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730):1999 20499999999999991911111119199999 2020. doi: 10. 10.1002/qj.3803.
Andrew Jaegle, Sebastian Borgeaud, Jean-Baptist Alayrac Carl Doersch, Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, Olivier Henaff, Matthew Botvinick, Andrew Zisserman, Oriol Vinyals. and Joao Carreira. Perceiver I0: A general archi- tecture for structured inputs & outputs. International Conference on Learning Representations, 2022. URL htt.htttp://ppoot htt.:r/ree...eeeeeoorroraaaaa...
ew hwt.//ee/eeeee....

---

Published as a conference paper at ICLR 2026
Josef Koumar, Karel Hynek, Tomáš Čejka, and Pavel Siška. Cesnet-timeserie22:: Time series dataset for network traffic anomaly detection and forecasting. Scientific Data, 12(1):338, Feb 2025. ISSN 2052-4463. doi: 10. 10.103%<41597-025-066000
[RLhtts:: org/10.
Yoshiki Kuramoto. Self-entrainment of population of coupled non-linear oscillators. In Interna- tional symposium on mathematical problems in theoretical physics: January 23-29, 1975, kyolo university, kyoto/Japan, pp. 420-422. Springer, 2005.
Sehun Lee, Wonkwang Lee, and Gunhee Kim. Meta-learning approach for joint multimodal signals with multimodal iterative adaptation. Transactions on Machine Learning Research (TMLR), 2024. URL
net / forum? inntrum?iddeee 1d=LVO4KBa I QQt Accepted; Open
Zijie Li, Kazem Meidani, and Amir Barati Farimani. Transformer for partial differential equations' operator learning. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL
htttps://openrewee
net
3uERT
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.00 08895, 2020.
Roderick J. A. Little and Donald B. Rubin. Statistical Analysis with Missing Data. Wiley, New York, 2 edition, 2002. ISBN 0471183865.
Yuxuan Liu, Jingmin Sun, Xinjie He, Griffin Pinney, Zecheng Zhang, and Hayden Schaeffer. PROSE-FD: A multimodal PDE foundation model for learning multiple operators for forecasting fluid dynamics. In Neurips 2024 Workshop Foundation Models for Science: Progress, Opportuni- ties, and Challenges, 2024. URL htLhttts::// http://ppereveeeeeee net forum? wtt.rummmddttttEAaee
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In Computer Vision - ECCV 2020, pp. 405-444444 421. Springer, 2020. doi: 10. 00077997833-30--0 58452-8 24.
Arsha Nagrani, Shan Yang. Anurag Arnab, Aren Jansen, Cordelia Schmid, and Chen Sun. Atten- tion bottlenecks for multimodal fusion. Advances neural information processing systems, 34: 14200-14213, 2021.
Natalia Neverova, Christian Wolf, Graham W. Taylor, and Florian Nebout. Moddrop: Adaptive multi-modal gesture recognition, 2015. arXiv preprint.
David Keetae Park, Xihaier Luo, Guang Zhao, Seungjun Lee, Miruna Oprescu, and Shinjae Yoo. Scent: Robust spatiotemporal learning for continuous scientific data via scalable conditioned neu- ral fields. In Proceedings of the International Conference on Machine Learning (ICMI 2025), 2025. URL https::///// cttp:///eperreieeeeemmmm net/ as arXiv:2504.1226.
juemZuRU Poster, also available
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In CVPR, 2019. doi: 10.1009/CVPR.2019.00022
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference Medical image computing and computer- assisted intervention, pp. 234-241. Springer, 2015.
Louis Serrano, Lise Le Boudec, Armand Kassaï Koupaï, Thomas X Wang. Yuan Yin, Jean-Noël Vittaut, and Patrick Gallinari. Operator learning with neural fields: Tackling pdes on general geometries. Advances in Neural Information Processing Systems, 36.70581-70611, 2023.
Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wet- zstein. Implicit neural representations with periodic activation functions. In NeurlPS, 2020.
Michael A. Sutton, Jean-José Orteu, and Hubert W. Schreier. Image Correlation for Shape, Motion and Deformation Measurements: Basic Concepts, Theory and Applications. Springer, Boston, MA, 2009. ISBN 9780387787480.
12

---

Published as a conference paper at ICLR 2026
Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Seddovich-Keil. Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let net- works learn high frequency functions in low dimensional domains. In Proceedings the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
U.S. Environmental Protection Agency. Air quality system (aqs) data mart, setps::/www. gov/aqs 2025, Accessed 2025-09-01.
Mark S. Veillette, Siddharth Samsi, and Christopher J. Mattioli. SEVIR: A storm event im- agery dataset for deep learning applications in radar and satellite meteorology. Ad- vances in Neural Information Processing Systems 33 (NeurIPS 2020). December 6-12, 2020, virtual, 2020. /wR//atttsss/ppoceednn.ss..
http://swssessaaoo.eeeeeeeeeraaa cc/ httpppps...cppppee////////ee/e/ fa78a o0d.:///ccdoo0001111184431111110 Abst-aaa333331099ttattttt... html
Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y. Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting, In The Twelfth International Conference on Learning Representations (ICLR 2024), Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https:/ /otp:///////eevvveeeeeee www. net. / fo forum? i 7oLshfETC2
Renjie Wu, Hu Wang, Hsiang- Ting Chen, and Gustavo Carneiro. Deep multimodal learning with missing modality: A survey. arXiv preprint M0Xi::2409.07825, 2024.
Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar. Neural fields in visual comput- ing and beyond. In Computer graphics forum, volume 41, pp. 641-676. Wiley Online Library, 2022.
Sungduk Yu. David Neelin, Laure Zanna, Nathan Urban, Janni Yuval, Ryan Aber- nathey, Pierre Baldi, et al. Climsim: A large multi-scale dataset for hybrid physics-ml climate modeling. In NeurIPS 2023 Datasets and Benchmarks, 2023. URL httpss//prrooeeinssseeeiuuss cc/paper files/paper/202 hash/ haa/:/55220131333300003332222322 DatasattttDatttesssssae and
Benchma rks. httml
13

---

Published as a conference paper at ICLR 2026
A DATA STATISTICS AND HYPERPARAMETERS
We report a detailed training and data specifications for better reproducibility. Data statistics and hyperparameters for OmniField on ClimSim-THi and EPA-AQS.
DATASET NAME
ClimSim-THW
EPA-AQS
DATA STATISTICS
Spatial domain / resolution
level, N 21600 grid pts
1048 fixed sites Os, PM... PM1o-
Modalities / targets
T. H. W - T, H,
NO,, CO, SO₃ same six time-aware split: 80% train 20% val (by day)
Train / Val size
9000 1000
Temporal horizon
random E [1-6) hr 0.02 (32 of 21600);
lead steps (1-5)
Input sparsity (per sample)
irregular (observations vary per
region=union variable (T.H.W sampled at 2%)
day/site)
Input points per modality
variable; min sites to sample per batch: 20, 5M 20, PM 15, NO:: 10, CO: 10, SO2: 10
Normalization
per-variable mean/std
per-pollutant mean/std
TRAINING
Batch size (train val)
8/1
4/4 epoch-based: 30 epochs AdamW (8, = 0.9, 8, 0.999)
Total optimizer steps Optimizer
100,000
AdamW (3) 0.9, 0.9999
Weight decay
x 10 Conealln WarmupRestarts
x 10 Cosine WasmupRestarts
LR schedule
Max Min LR
8 10 1 10- 1000
8 10 / 10
Warmup steps
10% of cycle
MODEL
# stages (L in (CMR)
3 (64, 64, 64) (64, 64, 64) (SA+FF each)
Latent dims per stage
(128, 128, 128)
latents per stage
(128, 128, 128) (SA+FF cach)
Self-attn trunk blocks (final)
Cross-attn heads / dim head
4 / 128
2/32
Self-attn heads / dim head Feed-forward multiplier
8 / 128
2 / 32
(GEGLU)
(GEGLU)
3 128 MLP (T/H/W), then concat pos enc
3- 128 MLP per pollutant (value, lat, lon)
Input projection per modality Decoder heads
three heads: T, H. W (each 1,)
six heads: O., PM2s, PM1.. NO., CO, SO,
EMBEDDING POSITIONAL FEATURES
Spatial encoding (GFF)
in:2 + 32 bands (sin+cos=64); scale 15.0
in:2 + 32 bands (sin+cos=64); scale 15.0
Time encoding (GFF)
in: 1 + 16 bands (sin+cos=32);; scale 10.0
in: 1 + 32 bands (sin+cos=64); scale 15.0
Query dim
64 (space) + 32 (time) = 96
64 (space) + 64 (time) combined by sum = 61
Per-modality input MLP dim
128 (hefore concat with pos enc)
128 (token dim pre-attention)
14

---

Published as a conference paper at ICLR 2026
B OMNIFIELD ENCODER DETAILS
We introduce the following two advances to improve high frequency learning aspects over the prior art, SCENT (Park al. 2025).
B. GAUSSIAN FOURIER FEATURES
In our architecture, we use Gaussian Fourier features (Tancik et al., 2020) in place of standard si- nusoidal Fourier features. This substitution is motivated by the need to capture high-frequenge fine-grained details more effectively. By sampling frequencies from continuous Gaussian distribu- tion, we provide the model with a richer and less biased spectral representation than a fixed, discrete set of functions allows (Tancik al. 2020). Our experimental results demonstrate that this approach significantly enhances the model's ability to reconstruct intricate details, leading to improved per- formance. For single coordinate variable æ; GFF is denoted follows:
[2os(2 Bx) sin Br)
7(2)
E R22d
BEReX1
By ~ ~((,00
B.2 SINUSOIDAL INITIALIZATION
To better capture high- frequecyy structure and stabilize training, we seed the M learnable queries with a compact multi-scale sinusoidal pattern. Let the model dimension be (even) and set d D/2. Define log-spaced frequency vector
V = Po...... va-1]'
Rdx1,
2k = b- k/d (k= 0. d - 1), b > 1.
Using the same feature map form in the GFF paragraph, for each E {0, ..... M 1} set
cos 2T mm) mm sin 2T v m)
IR2d =R,
Q()) [[4
[40 ::...:0M IR MxD
(0) (0)
Since sin? + cos elementwise,
1
11
sin (2 + cos² (2 sdd,
k=0
SO s = d 1/2 makes each row roughly unit norm, keeping initial attention logits well scaled.
15

---

Published as a conference paper at ICLR 2026
C
FLEXIMODAL FUSION
Multiple Unimodal/ Cross-madal Neural Fields
Fleximodal Fusion
model inputs
model inputs
M3
M9
M7
Unified Model
modality
modality
modality
model
model
outputs
Let M = (1,..... [il.....M M} index modalities. For instance, let Xm {(x,, e))) be the tokenized inputs modality m (possibly empty), and let π E 1), indicate observed modalities (T., = iff Nm 0). Fleximodal fusion learns single model that (i) conditions on any subset of modalities via presence masks, (ii) gates cross-modal interactions so absent channels contribute neither signal nor noise, and (ii) defines an objective valid for any subset of inputs/targets.
C.I MASKED ENCODERS AND GATED FUSION.
Each modality has encoder en that maps tokens to a latent set Zm Em (Xm). We gate absent channels at the source:
Zm Jm Em (X
(Zm when Tm = 0).
A fusion operator A (our MCT/ICMR stack) aggregates only present latents to produce the condi- tioned field representation G:
G = A(1 }mem),
with attention masks that zero cross- aoss-ntttnntion to absent sets. Concretely, for a cross-attention block with query Q and keys/values Km, Vm from modality
Atta(Q. K, softmax
QKT Va
+ log M M)V, M diag 1 L.....
(5)
present
absent
keys/values from effectively removed whenever T.
Query decoding and flexi-objective. Given query set Q (locations/times), per-modality decoders
Dm, read from G to produce predictions y,, Dm, (G; Q). Let {0, 13M indicate which targets are supervised on this instance (e.g., modality present on the target day). The loss is defined for any input/target subset:
M
C=
C
Σ m=1
Tm
a Ym),
(6)
with & the per-modality error (we use MSE in normalized space). This objective works for full, par- tial, or single-mone modalety inputs, and any supervised of outputs, without imputing missing channels.
Why this matters. Scientific systems frequently yield xm = 0 for some m (e.g., QA-filtered satel- lite AOD, down PM2.5 stations, asynchronous sampling). Fleximodal fusion prevents "hallucinated" evidence from absent channels and lets the same trained model operate across variable sensing con- ditions. This aligns with recent work on missing- modallty learning and FuseMoE for fleximodal fusion Han et al. (2024); (W al. (2024), and differs from multimodal dropout Neverova et al. (2015), which randomly drops modalities as training-time augmentation rather than handling truly absent channels.
16

---

Published as a conference paper at ICLR 2026
D FURTHER PREPROCESSING DETAILS ON CLIMSIM
Source and configuration. We use the High- Regh-Reesottion Real Geography configuration of Clim- Sim (E3SM-MMF) at 1.5" x 1.5% horizontal resolution (21,600 grid columns), saved every 20 min- utes for 10 simulated years. Data are hosted on Hugging Face under LEAP CAAP/CC /CI imm im www.ii..hhhhhhh hi- Each 20-minute step provides an input (mli) and output (mlo) NetCDF pair.
Modalities (T,H,W). We select three prognostic variables at a single vertical level (0- indexed index 59): air temperature state_t, specifie humidity state_q0001, and meridional wind state.v. The latitudelongitude for each column are taken from ClimS wee.semhig---r.riiii-aa
Download and subsample. We downloaded the first 10,000 input snap- shots (20-minute spacing) from the training split using scripted fetch from: /e
resolve/mainn
ml w......---------..------
For each file, we extract state_t [state 1evel=59], wwwat..go0000 (state. 1evel=59), and (etaae_v[ssaae 1eve1=59] into compressed .pp files (one snapshot per file). We use train/test split of 9,000 / 1, 1,000 snapshots.
Normalization and mesh. We precomputed per-modality normalization statistics (mean and stan- dard deviation) over the selected snapshots and applied them consistently across the split. All ex- periments use the provided high-resolution grid metadata to map each column to (pp, A).
Table: ClimSim (High-Res, Real Geography) THW subset used in our experiments.
QUANTITY
SYMBOL
VALUE
HORIZONTAL GRID
1.5 1.5" (21,600 COLUMNS) 20 MINUTES
TEMPORAL RESOLUTION
A
SNAPSHOTS USED
N
10,000 (FIRST IN TRAIN)
SPLIT (TRAIN/TEST)
9,000 1,000
MODALITIES
TESTATE_T, DESTATE_00001 W=STATELY INDEX (0-INDEXED)
VERTICAL LEVEL
GRID METADATA
CLIMSIM, MA,SIM.HE WWW.HIGH-RRB SEHH-RE_SRRIDING DIIDD-IOOCC
NORMALIZATION
PER-MODALITY (H, σ) OVER SELECTED SNAPSHOTS ONE SNAPSHOT PER NPZ (T,H.W AT k=59)
STORAGE
Preprocessing for Simulating Multimodal Sensor Networks When we sparsify each multi- modal instance of the ClimSim-THW dataset, we aimed to (i) have all three modalities to share fixed number sensors, while (ii) they each have a fixed number of exclusive sensors, and (iii) also a fixed number of paired two-modality sensors. This sophisticated design allows us to simulate a real scenario: where the scientific settings, data rarely available for all sensor locations, and more naturalistic to have partially shared sensors. Let X and
Rd be the continuous spatial domain
S { 21600 X
denote the fixed catalogue of candidate sensor coordinates. For three modalities m E {1, 2,3,, define fixed (instance-invariant)) modality-specifie sensor sets
Sm C,S
|mml 0.01 |S| 216.
Let the triple co-located subset be
Sn Si S Sa.
Snl 0.005 |S] 108.
Pairwise overlaps [S, S, are otherwise unconstrained (beyond [S; 0 Syl 2 |S,(), so the covered
union satisfies
S USUS US
Σ Sm
Σ
S |Sn| [324, 432].
1514353
17

---

Published as a conference paper at ICLR 2026
Le., between 1.5% and 2% of the 21,600 locations.
For each data instance E , and an input time f, the observations available to modal- ity m are the values at its fixed sensor coordinates:
Ul = ((tin))())) Joss.." with Sm independent of b.
Thus the sensor masks {Sm 13 m= are identical across all 10 instances (and across any forecast targets).
Define the fixed incidence matrix {0, 1 J2160003 by
A 1 [zz Sm).
Then each column sum is E Ag. = 216 and exactly 108 rows satisfy (Aj.1, Aj.s) = (1, 1, 1). Row sums encode co-location patterns (0, 1, 2, present at site), and the masks are fixed over all instances.
18

---

Published as a conference paper at ICLR 2026
E PRELIMINARY TEST 1: HIGH-FREQUENCY LEARNING ON CIFAR-10
Core Hypothesis The low frequency bias prevalent in prior works (Park et al. 2025) can im- proved by re-designing the major components of the encoder: positional encoding and learnable queries.
We show below the qualitative generation comparisons, ablating Gaussian Fourier Features (GFF). Sinusoidal Initialization (SI), and Iterative Cross-Model Refinement (ICMR). Here, since is a reconstruction task, the time encoding always zero: t사 O, and ICMR applies hier- archical encoding structure without multimodal mixing. We show eight test data instances for the comparison.
Ground Truth
O
X
O
O
O
X
X
X
X
X
X
Since the low resolution images of CIFAR- 10 makes it hard compare the results distinctively, report two additional results. First, we show the MSE performance as shown in the main manuscript Second, we show the power of the 2D fourier spectrum below:
Table 5.4
OFF
0
S1
0
X 0
X
0
(Power)
On a test instance, we show the original 32 x 32 image, and its corresponding power spectrum as defined by f= FF (FFT)x)) where FFT 2-dimensional fast fourier transform and is the input image. The Delta is the absolute difference between the ground truth power and the predicted power the 2D grid. In the Pred spectrum, the center area low-frequency, whereas boundaries are high-frequency components. We see a clear distinction between different model ablations. It notable that all three components show characteristic impacts on the power spectrum, while the absence of each component leading to inferior high frequency learning.
19

---

Published as a conference paper at ICLR 2026
F PRELIMINARY TEST 2: SYNTHETIC SINUSOIDAL MULTIMODAL DATA
Core Hypothesis. Differential performance of multimodal CNFs more easily and directly ob- servable in toy, synthetic multimodal data.
constructed a paired spatiotemporal dataset (,,, SS2 to test conjectures about multimodality in fully controlled environment. The driving modality S1 is a multiscale sinusoidal field, while the coupled modality 52 evolves under Kuramoto-style phase interactions (Kuramoto, 2005) with S1. This synthetic testbed allows us isolate the effects of fusion, cross cross-modal learning, and sparsity.
Driving modality (Si). Defined as a sum of sinusoidal components with different spatial/temporal frequencies:
Si (zx
Σ A sin(k,r wit + Φ.) rz(, f),
with N 3, domain æ E [0, 10] discretized X 100, and E [0,50] discretized 500. Noise was not included in the experiments reported here.
Coupled modality (S2). Phases 0j of M = oscillators evolve as
d0, = wt + dt
N
M (z 1 By (in(
(z, - (t)),
where u); are the S1 component phases. Coupling strength K tunes dependency: weak K yields nearly independent signals; strong K synchronizes S2 Sj. Unless otherwise stated, K 2.5.
Windows and splits. We form windows with input length Lin = 20 and horizon Lpred = 1, stride 8 1, producing Nwin 479 examples. Unless noted, we use an 80%/20% train/validation split. Sparsity masks (when used) are random with kept fraction of ~ 30 points, applied either jointly or per-modality. No normalization was applied.
Table: Synthetic S1/S2 dataset statistics (default configuration).
QUANTITY
SYMBOL
VALUE
SPATIAL POINTS
X
100
TIMESTEPS
T
500
WINDOW HORIZON STRIDE
(Lax; TIRED: 8)
(20, 1, 1)
# WINDOWS
Nwis
479
SPLIT (TRAIN/VAL/TEST)
80 / 20
COUPLING
K
2.5
NOISE
!
NONE
SPARSITY
OFF ON (~> 50 POINTS KEPT)
Experiment variants.
We evaluated seven complementary configurations to probe information
sharing and robustness:
(a) Multimodal reconstruction: Input (S1,S2), predict both at 1+1.
(b) Cross-modal (1~sS1,) Input S₁ only, predict both.
(c) Cross-modal SS1sS1; Input S₂ only, predict both.
(d) Mixed sparsity: One modality half dense/half sparse, the other dense.
(c) Unimodal sparse (~ 50 points): Input from single modality, sparsified to 50 spatial
points.
(f) Temporal continuity: Train/evaluate with fractional fout to test interpolation.
Summary. Across variants, multimodal fusion consistently improved reconstruction, forecasting, and modal prediction compared to unimodal baselines, with advantages persisting under spar- sity (both multimodal and unimodal at ~ 50 points) and fractional time interpolation. These sults provide a controlled validation of our multimodal neural field conjectures before applying the method to real scientific datasets.
20

---

Published as a conference paper at ICLR 2026
G
MORE QUALITATIVE RESULTS ON CLIMSIM-LHW
Temperature (T)
Humidity(H)
Wind speed (W)
OmniField
SCENT
PROSE-FD
MIA
300
Ground Truth
RMSE-4.38
9.44
260
CORAL
FNO
OFORMER
UNET
RESNET
25
16.0
8.29
12.2
12.7
13.3
10
OmniField
SCENT
PROSE-FD
MIA
10 2g kg
Ground Truth
17.5
RMSE=2.22 22
2.51
3.33
3.30
7.5
2.5
CORAL
FNO
OFORMER
UNET
RESNET
12
3.36
4.55
4.48
4.99
OmniField
SCENT
PROSE-FD
MIA
1
7.5
Ground Truth
RMSE=4,75
5.06
5.43
0.0
-7.5
CORAL
FNO
OFORMER
UNET
RESNET
6.43
7.81
9.35
5.86
5.78
Figure: ClimSin-LHW full qualitative evaluation. Gray-scale images denote the delta in absolute errrr.
21

---

Published as a conference paper at ICLR 2026
H
MORE QUALITATIVE RESULTS ON EPA-AQS
Qualitative Comparisons for forecasting (Ar = 5 days) on sparse EPA-AQS data. Visualized on
subset for six models and three modalities (Og, PM2.5. and CO).
Ground Truth
OmniField
SCENT
PROSE-FD
MIA
OFORMER
RESNET
ppm
Ozore
5.81
6.20
6.06
27.9
9.04
7.8
H m'
40
2M
6.66
14.0
16.0
32.3
22.4
16.4
10
ppm
00
Co
19.0
19.3
22.6
50.1
24.8
21.4
Below we show uncurated examples of the full multimodal forecasting comparisons on EPA- AQS, The first and second rows correspond to the set of inputs and ground truths, respectively. The input and output have varying sensor locations and statistics, posing significant challenges. Our OmniField's 's superior performance reveal the proven values of continuity- aware model architecture on top of the proposed ICMR robust spatiotemporal learning.
22

---

Published as a conference paper at ICLR 2026
Ozone Inputs
PM2.5 Inputs (none)
PMIO Inputs
NO2 Inputs
CO Inputs
150
S02 Inputs
20
20
20
Ozone GT
PM2.5 GT
PM10 GT
NO2 GT
CO GT
1:00
502 GT
60
20
20
20
OmniField (5.807)
OmniFiell (6.659)
OmniField (16.547)
OmniField (10. 668)
OmniFieed (18.998
1:00
OmniField (27.183)
50
SCENT (6.198)
SCENT (14.023)
SCENT (21.184)
SCENT (10.835)
SCENT (19.261)
SCENT (27 3725
100
400
90
PROSE-FD (8.063)
PROSE-FD (16.020)
PROSE-FD (20.562)
PROSE-FD (12.607)
PROSE-FD (22.615)
PROSE-FD (34.483)
100
90
MIA (27.951)
MIA (273177)
MIA (45. 515)
MIA 187)
MIA (50.148)
MIA (39.157)
100
90
CORAL (22.228)
CORAL (25.568) (68)
CORAL (21.389)
CORAL 814)847)
CORAL (25.627)
CORAL (36.647)
100
90
FNO (15. 585)
FNO (32.083)
FNO (81.58) (83)
FNO (40.013)
FNO (44.778)
FNO (82.962)
100
40
50
OFORMER (9.041)
OFORMER (22.408)
OFORMER (20.973)
OFORMER (16.874)
OFORMER (24.807)
OFORMER (36.807)
100
RESNET (7.846)
RESNET T (16.428)
RESNET 7633)
RESNET (10.434)
RESNET (21.427)
RESNET (29.215)
100
50
UNET (6. 134)
UNET (13:875)
UNET (18.636)
UNET (11.744)
UNET (20.340)
1.00
UNET (28.815)
60
100
50
Figure: EPA-AQS full qualitative evaluation instance 1
23

---

Published as a conference paper at ICLR 2026
Ozone Inputs
Inputs (none)
PM10 Inputs
NO2 Inputs
CO Inputs
502 Inputs
100
40
20
20
70
Ozone
PM2.5
PM10
NO2
CO GT
502 GT
66
100
40
20
20
OmniField (7.391)
OmniField (95)
OmniField (13.416)
OmniField (11.066)
OmniField (11 (74674
OmniField (30.619)
60
20
20
90
10
SCENT (8.637)
40
SCENT (11.153)
SCENT (14.021)
SCENT (12.506)
SCENT (13.555)
SCENT (9.957)
20
50
:0
PROSE-FD (16.559)
40
PROSE-FD 112.675 678)
PROSE-FD (16.557)
PROSE-FD (64.964)
PROSE-FD (16.373)
PROSE-FD (33.493)
20
90
100
MIA (25.859)
40
MIA (33.973)
MIA (35. 25.6681
40
MIA (31.052)
MIA (29.949 (29.94B)
MIA (55.500
20
50
CORAL (29.
CORAL (28.119)
CORAL (20.178)
CORAL (16.269)
CORAL (18.961)
CORAL (37. 337.129)
40
50
FNO (28. 348)
10
FNO (24.329)
FNO (98.522)
-60
FNO (43.685 443.685)
FNO (S5.600)
FNO (83.045)
50
OFORMER (62.502)
OFORMER (22.993 223.933)
OFORMER (17.594)
OFORMER (17.869)
OFORMER (19.343)
OFORMER (36.420)
90
RESNET (8.230)
40
RESNET (15.651)
RESNET (14.911)
RESNET (10.39) 396)
RESNET (13.635)
RESNET 206)
-00
40
50
UNET 346)
UNET (119977)
UNET (13.501)
UNET (12.438)
UNET (14.336)
UNET (31.388)
40
40
50
Figure: EPA-AQS full qualitative evaluation instance 2
24

---

Published as a conference paper at ICLR 2026
I FULL EXPERIMENTAL TABLES FOR FIG. 4
Table for Fig. a). For each input configuration, we show RMSE performance for each output modalities, separately. Note that for Co-Location strategy limit the test set to the intersection of all sensors from three modalities. In the meanwhile, for Interpolation, Mid-Fusion, or ICMR, we use all sensor locations for the test set. Therefore, we separate the test split and report them separately. Fig. 4aa corresponds to Test split = Intersection. It shows that, regardless of the Test split, ICMR outperforms outer strategies across all modalities. All values in physical units. Lower is better.
M (T)
Min (T, H, W,
Strategy
Test split
H
W
T
H
W
K
[10-3
kg kg
[ms-
[K]
[10-3
kg kg
[ms-
Interpolation Mid-Fusion
2.677
1.171
5.172
1.936
1.006
5.063
Union
1.887
1.024
5.155
1.398
0.807
4.967
ICMR
1.530
0.870
5.011
1.073
0.664
4.859
Co-Location
3.093
1,315
5.265
2.527
1.157
5.137
Interpolation
2.682
1.167
4.513
1.972
1.007
4.375
Intersection
Mid-Fusion
1.854
0.943
4.502
1.386
0.795
4.340
ICMR
1.655
0.841
4.380
1.081
0.645
4.258
Table for Fig. 4 b-c).. Comparison of models on the EPA-AQS dataset with varying number of training modalities. Test is always modalities.
2 Modality Train
4 Modality Train
6 Modality Train
Model
Params
Og/PM2.6
Og/PM2.5
PM
Oy/PM2.5
PM1u/NO;
CO/SO2
OmniField
4.3M
4.6/16.4
4.5/15.4
51.6/8.3
4.4/14.9
35.3/7.3
19.8/26.4
FNO
0.2M
11.3/17.6
8.7/17.2
56.5/9.2
8.3/15.6
43.9/8.8
32.8/31.3
SCENT
29.3M
5.7/16.9
5.6/16.1
54.3/8.5
5.5/15.8
44.4/7.6
25.6/26.7
MIA
1.5M
7.9/22
8/23.1
39.6/14.1
7.4/21.1
38.8/13.7
21.1/42.8
CORAL
1.9M
19.1/27
9.7/22.2
52.7/21.4
9.5/21.5
77.6/23.1
28.5/51.9
PROSE-FI FD
42.3M
5.5/19.3
7.9/20.6
38.4/7
5.4/19.7
40.3/12.8
19.9/39.4
OFORMER
2.5M
9,3/20.6
7.7/19.5
46.3/15.9
7.6/19.4
45.9/13.6
29.3/38.9
UNET
13.3M
7.8/18.3
6,5/18
46.4/11.4
7/17.6
45.1/13.2
30.7/36
RESNET
4.5M
10.2/18
8.2/18.5
43/11.7
8.6/17.9
49.4/10.5
32.7/34.2
25

---

Published as a conference paper at ICLR 2026
J ADDITIONAL EXPERIMENTAL TABLES
Table 4: Efficiency and scalability on ClimSim-THW. We report forecasting RMSE (physical
units) for temperature (T), humidity (H), and wind (W), together with parameter counts and forward- pass FLOPs.
Model / Setting
Params (M)
FLOPs
T[K]
[[10
kg kg
'1
W [m s
OmniField
1.3
1.6x 10"
1.21
0.74
5.13
OmniFiild
10.6
3.0x10''
1.10
0.68
4.94
OmniField
37.4
3.6x
1.07
0.66
4.86
FNO
10.5
5.0x109
2.96
1.16
6.79
FNO
38.1
1.7x104
2.53
1.13
6.72
PROSE-FD
10.6
3.1x10'2
5.40
1.95
5.66
5: ICMR refinement stage sweep. We vary the number of refinement stages T. E (1,3,5, 5,1)
and report THW RMSE along with parameters and FLOPS.
L
Params (M)
FLOPs
TRMSE
RMSE
W RMSE
2.8
1,383
0.905
5.289
3
10.6
3.0x10'0
1.10
0,68
4.94
18.5
3,2x10400
0.97
0.61
4.86
10
34.4
3.8x10
0.95
0.60
4.83
Table 6: Long-horizon rollout stability (normalized RMSE). We train OmniField with either
6h 48h h supervised horizon and evaluate 6/24/48 rollouts. We also report naive unrolled FNO baseline. Lower is better.
Model
Train horizon
Eval horizon
T
H
W
OmniField
48h
6h
0.056
0.084
0.700
OmniField
48h
24 h
0.056
0.092
0.722
OmniField
48h
48h
0.063
0.095
0.754
OmniField
6h
6h
0.062
0.093
0.707
OmniField
6h
24
0.085
0.134
0.728
OmniField
6h
48
0.087
0.138
0.789
FNO
6h
0.449
0.417
1.860
FNO
2.58 x 10
2.66 x 10
1.66
10a
FNO
48
9.36 x 10"
1.01 x 1020
6.60 x 10aD
Table 7: Downstream task transfer. OmniField representations transfer across tasks and data modalities.
Task Dataset
Metric
OmniField
Baselines
CESNET (multivariate TS forccasting) CIFAR-10 (20% sparse pixels)
Test MSE +
0.755
TimeMixer: 0.859; FNO: 0.790
Acc.
71.8%
UNet: 55.0%
SEVIR (lightning anomaly detection)
ROC-AUC t
0.97
FNO: 0.96
26

---

Published as a conference paper at ICLR 2026
Table 8: Training with randomly missing modalities. One or two modalities are randomly dropped during training for fraction p of instances (never dropping all modalities). Reported ClimSim- THW in physical units.
Drop fraction P
T RMSE [K]
RMSE [10 kg kg
W RMSE [m s
0.0
1.07
0.66
4.86
0.5
1.11
0.71
4.89
0.75
1.23
0.74
4.94
27


Reviews:
[{"review_id": "gh5dlwN6yA", "reviewer": "Reviewer_BeG4", "summary": "The paper proposes OmniField, a framework for robust multimodal spatiotemporal learning from sparse, irregular, and noisy real-world observational data.\nOmniField extends the continuous neural field paradigm (previously introduced in SCENT) by addressing two fundamental challenges: data sparsity and modality inconsistency.\nSpecifically, it integrates several components — Gaussian Fourier Features (GFF) and Sinusoidal Initialization (SI) for improved spatial continuity, and three new modules:\nMCT (Multimodal Crosstalk) for early cross-modal feature interaction,\nICMR (Iterative Cross-Modal Refinement) for multi-round alignment across modalities,\nand Fleximodal Fusion for handling missing or degraded modalities.\nExtensive experiments on four datasets (CIFAR-10, RainNet, ClimSim-THW, and EPA-AQS) and eight baselines show consistent performance improvements in both reconstruction and forecasting tasks, especially under conditions of high sparsity and missing modalities.", "questions": "* In Figure 4(b–c), the label “SCENT-MM” seems to correspond to OmniField, based on the text and tables. Could you please confirm whether this is a labeling oversight, or if SCENT-MM refers to a distinct model configuration?\n* I noticed that OmniField has a relatively large number of parameters compared to SCENT. To what extent might the performance improvement stem from model capacity rather than the proposed architectural changes?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "* The paper is well-structured and clearly motivates the problem by identifying two central challenges — data sparsity and multimodal inconsistency — and proposing targeted solutions through the MCT, ICMR, and Fleximodal Fusion modules. The organization is logical.\n\n* The figures are well-designed and self-explanatory, effectively supporting the paper’s claims and illustrating the benefits of the proposed modules. Quantitative results are straightforward and convincing, showing consistent gains over baselines.\n\n* The experimental results are solid and comprehensive, covering eight strong baselines and four diverse datasets. The results are straightforward to interpret and show consistent improvements across all settings. \n\n* The paper includes extensive ablation studies, noise-robustness analyses, and missing-modality evaluations, which together demonstrate strong empirical evidence for the proposed method's robustness and generality.", "weaknesses": "* While the paper is generally well written, some parts are conceptually dense and abstract. The presentation could benefit from additional intuition, clearer intermediate explanations, or a small running example to illustrate how each proposed component (MCT, ICMR, Fleximodal Fusion) functions in practice.\n* The forecasting horizon studied in the current experiments is relatively short (e.g., six-hour prediction on ClimSim-THW). Evaluating longer temporal horizons could provide deeper insights into the model’s stability and long-term reasoning ability. Incorporating additional temporal modeling structures might further enhance the method’s forecasting capability.\n* OmniField introduces additional modules (MCT and three rounds of ICMR), which substantially increase the computational cost compared to the baselines. While the model achieves lower RMSE and improved robustness, these gains come with notably higher architectural complexity. A more detailed analysis of runtime efficiency trade-offs would help clarify the impact of the proposed design.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MRnjDkRQmz", "reviewer": "Reviewer_JbX6", "summary": "This work presents a unified neural field paradigm that tackles the challenge of learning continuous, frequency-rich representations across heterogeneous spatial, temporal, and multimodal sensing regimes.\nThe authors blend gated fleximodal fusion with iterative cross-modal refinement, allowing the model to gracefully interpolate missing channels, suppress noisy sensors, and sustain high-frequency fidelity in sparse, real-world observation settings.\nThe authors evaluated the proposed method using climate simulation (ClimSim-THW), air-quality forecasting (EPA-AQS), and vision-centric reconstruction tasks, compared with baselines like SCENT and RainNet.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The proposed fleximodal fusion and iterative refinement approach provides a principled mechanism for handling missing and noisy modalities, improving robustness in settings with irregular or sparse sensors.\n- The incorporation of frequency-rich embeddings and sinusoidal initialization yields measurable gains in high-frequency signal reconstruction, particularly in spatiotemporal domains.\n- The method showed consistent performance improvements across multiple scientific datasets, suggesting a generalizable modeling framework rather than task-specific tuning.", "weaknesses": "- The evaluation focuses on a curated set of scientific benchmarks; broader assessment on diverse multimodal domains (robotics, remote sensing beyond climate/air quality) would strengthen claims of generality.\n- The computational and memory cost of iterative cross-modal refinement and continuous-field conditioning is not fully characterized. It's unclear how well the proposed method scales to higher-resolution or real-time applications.\n- While robustness to missing modalities is a central motivation, ablation analyses on modality-specific contributions are limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wusXgO9XpE", "reviewer": "Reviewer_BsBj", "summary": "This paper proposes a multimodal conditioned neural field (CNF) framework for learning from sparse, irregular, and noisy multimodal spatiotemporal signals, called OmniField. OmniField has three major components: Gaussian Fourier embeddings, sinusoid initialization to stabilize training, and iterative cross-modal refinement to align modalities. The authors evaluate OmniField on both simulated and real-world multimodal geo-spatio-temporal datasets and show notable performance gain over SOTA baselines. Ablation studies are performed on both spatial and spatio-temporal datasets to understand the contribution of each component.", "questions": "- How sensitive is the performance to the number of refinement iterations? Could fewer iterations achieve comparable results with lower computational cost?\n- The fusion mechanism prevents information leakage from missing modalities. Could the authors quantify how performance is under a randomly missing modalities scenario?\n- How does OmniField perform in other tasks, such as classification, where the learned field embeddings are fine-tuned for downstream objectives, rather than regression or forecasting?\n- How transferable is the approach to non-geospatial multimodal time-series domains (e.g., high-frequency sensor data)? Does the current formulation depend heavily on spatial continuity assumptions?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The datasets and benchmarks are comprehensive spanning across multiple applications\n- Omnifield shows robustness and performance gains\n- Proposed components are validated through ablation studies", "weaknesses": "- I am quiet concerned with the novelty. The core framework remains a straightforward extension of SCENT with a few architectural augmentations for multimodal data.\n- The authors have limited explanation of training efficiency and scalability. The computational complexity can grow with the number of tokens and modalities, but the paper has limited analysis on the training or inference efficiency, nor does it discuss how OmniField might perform on larger-scale or real-time systems. Given that some spatiotemporal models for some domains are often deployed on resource-constrained platforms, this efficiency is important.\n- Although the authors evaluated the methods on numerous datasets, the diversity of the downstream task evaluated is limited. The paper focuses almost entirely on reconstruction and forecasting accuracy. Other tasks such as classification, uncertainty quantification or anomaly detection could enhance the robustness. A stronger connection to practical use cases would also enhance the work’s relevance.\n- The model implicitly assumes shared information across modalities but does not explicitly disentangle shared vs. private modality features, which are important in multimodal learning. While the iterative fusion helps align features, it remains unclear how modality-specific noise or private components are handled. Can the authors elaborate on the integration with factorized multimodal learning?\n- Minor:\n    - More extensive citations in the Introduction, especially from the data/model perspective to contextualize multimodal spatiotemporal learning in prior literature can benefit.\n    - What is SCENT-MM? Is that OmniField? Figure 4b and 4c do not have OmniField", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)