VICON: Vision In-Context Operator Networks for Multi-Physics Fluid Dynamics Prediction
Authors: Yadi Cao
Yuxuan Liu
Liu Yang
Rose Yu
Hayden Schaeffer
Stanley Osher
Venue: Accepted by TMLR ()

Abstract: In-Context Operator Networks (ICONs) have demonstrated the ability to learn operators across diverse partial differential equations using few-shot, in-context learning. However, existing ICONs process each spatial point as an individual token, severely limiting computational efficiency when handling dense data in higher spatial dimensions. We propose \textit{Vision In-Context Operator Networks} (VICON), which integrate vision transformer architectures to efficiently process 2D data through patch-wise operations while preserving ICON's adaptability to multi-physics systems and varying timesteps. Evaluated across three fluid dynamics benchmarks, VICON significantly outperforms state-of-the-art baselines DPOT and MPP, reducing the average last-step rollout error by 37.9\% compared to DPOT and 44.7\% compared to MPP, while requiring only 72.5\% and 34.8\% of their respective inference times. VICON naturally supports flexible rollout strategies with varying timestep strides, enabling immediate deployment in \textit{imperfect measurement systems} where sampling frequencies may differ or frames might be dropped—common challenges in real-world settings—without requiring retraining or interpolation. In these realistic scenarios, VICON exhibits remarkable robustness, experiencing only 24.41\% relative performance degradation compared to 71.37\%-74.49\% degradation in baseline methods, demonstrating its versatility for deployment in realistic applications. Our scripts for processing datasets and code are publicly available at https://github.com/Eydcao/VICON.

Published in Transactions on Machine Learning Research (02/2026)
VICON: Vision In-Context Operator Networks for Multi- Physics Fluid Dynamics Prediction
Yadi Cao"t
yadicoo95 Caiicaao9Ssnlll..cm Gucla. edu
University of California, Los Angeles University of California, San Diego
Yuxuan Liu'
yaliu volla edu
University of Colifornia, Los Angeles
Liu Yang!
yangliuOnus.co edu.sg
University of Colifornia, Los Angeles National University of Singapore
Rose Yu
roseyn edu
University of Colifornia, San Diego
Hayden Schaeffer
hschaeffer edu
University of California, Los Angeles
Stanley Osher
sjo Grucla edu
University of California, Los Angeles
Reviewed on OpenReview: https: opeereveee. net/ forum? id- 6V3YMHULQ3
Abstract
In-Context Operator Networks (TCONs) have demonstrated the ability learn operators across diverse partial differential equations using few-shot, in-context learning. However, existing ICONs process each spatial point as individual token, severely limiting com- putational efficiency when handling dense data in higher spatial dimensions. We propose Vision In-Contert Operator Networks (VICON), which integrate vision transformer architec- tures to efficiently process 2D data through patch-wise operations while preserving ICON's adaptability to multi-physics systems and varying timesteps. Evaluated across three fluid dynamics benchmarks, VICON significantly outperforms trate-of-the-art baselines DPOT and MPP, reducing the average last-step rollout error by 37.9% compared DPOT and 44.7% compared to MPP, while requiring only 72.5% and 34.8% of their respective inference times. VICON naturally supports flexible rollout strategies with varying timestep strides, en- abling immediate deptoyment in imperfect measwrement systems where sampling frequencies may differ or frames might be dropped-comee common challenges in real-world settings without requiring retraining or interpolation. In these realistic scenarios, VICON exhibits remark- able robustness, experiencing only 24.41% relative performance degradation compared to 71.37%-74. 74.4%% degradation baseline methods, demonstrating its versatility for deployment in realistic applications. Our scripts for processing datasets and code are publicly available at https://github.co httpg/hhhb...mmmyyydd/////oo
settings-withour
"Equal contribution.
Work partially done while UCLA.

---

Published in Transactions on Machine Learning Research (02/2026)
1 Introduction
Machine learning has emerged powerful tool for solving Partial Differential Equations (PDEs). Traditional
approaches primarily operate discrete representations, with Convolutional Neural Networks (CNNs)
excelling at processing regular grids Gupta & Brandstetter (2022), Graph Neural Networks (GNNs) handling unstructured meshes Sanchez-Gonzalez et al. (2020); Lino et al. (2022 Cao et al. (2023) and transformers
capturing dependencies between sampling points within the same domains Cao (2021); ; Dang et al. (2022); ;
Jiang et al. (2023). However, these discrete approaches face a fundamental limitation: they lack PDE context. information (such as governing parameters) and consequently cannot generalize new parameters or PDEs
beyond their training set.
This limitation led to the development of "operator learning," " where neural networks learn mappings from input functions to output functions, such as from initial/boundary conditions to PDE solutions. This capability was first demonstrated using shallow neural networks Chen & Chen [1995b a), followed by specialized architectures inchuding the widely adopted Deep Operator Network (DeepONet) Lu et al. (2021). Fourier Neural Operator (FNO) Li et al. (2021]; ;; Kovachki et al. (2023), and recent transformer-baser operator learners Li et al. (2022:
2023 Ovadia al. (2024). While these methods excel at learning individual parametric PDEs, they cannot generalize across different PDE types, necessitating costly retraining for new equations or even different
timestep sizes.
Inspired by the success of large language models in multi-domain generation, researchers have explored "multi-physics PDE models" to address this single-operator limitation. Common approaches employ pre- training and fine-tuning strategies Chen et al. (2024); Herde et al. (2024); Zhang et al. (2024c), though these require substantial additional data (typically hundreds of frames) and fine-tuning before deployment. This requirement severely limits their practical applications in scenarios like online control and data assimilation, where rapid adaptation with limited data sampling is essential.
To climinate the need for additional data collection and fine-tuning, several works have attempted to achieve few-shot or zero-shot generalization across multi-physics systems by incorporating explicit. PDE information. For example, the PROSE approach Liu et al. (2024c b): Sun et al. (2024a); Jollie et al. (2024) processes the symbolic form of governing equations using an additional transformer branch. Similarly, PDElormer YYe et al., (2024) converts PDEs into directed graphs as input for graph transformers, while others embed symbol-tokent directly into the model Lorsung et al. (2024). Despite these ad such methods require explicit knowledge of the underlying PDEs, which is often unavailable when deploying models in new environments.
To further lower barriers for applying multi-physies models in real-world applications, the In- no-Cotext Operator Network (ICON) Yang et al. (2023a) offers fundamentally different approach: ICON implicitly encodes system dynamics through a few input-output function pairs, then extracts dynamics from these pairs in an in-context fashion. This approach enables few-shot generalization without retraining or explicit PDE knowledge, while the minimal required input-output pairs can be readily collected during deployment. A single ICON model has demonstrated success in handling both forward and inverse problems across various ODEs, PDEs, and mean-field control scenarios.
However, ICON's computational efficiency becomes a critical bottleneck when handling dense data in higher dimensions. Specifically, ICON treats each sampled spatial point as an individual token, leading to quadratic computational complexity with respect to the number of points. This makes it computationally prohibitive for practical 2D and 3D applications to date, only one instance of a 2D case using sparse data points has been demonstrated Yang et al. (2023a).
To address this limitation, we propose Vision In- Co-Contezt Operator Networks (VICON), which leverage vision transformers to process 2D functions using an efficient patch-wise approach. Going beyond classic vision transformers that typically process patches from a single image, VICON extends the architecture to handle sequences of input-output. function pairs, enabling "next. function prediction" capabilities while preserving the benefits of in-context operator learning. The current implementation focuses on passive, autonomous PDEs; extension to non-autonomous systems with external forcing is left for future work.
Our contributions include:
2

---

Published in Transactions on Machine Learning Research (02/2026)
Development of VICON, vision transformer- baasd in-context operator network for two-dimensional lime-dependent PDFs that maintains the flexibility of in-context operator learning while efficiently handling dense data in higher dimensions.
Comprehensive evaluation across diverse fluid dynamics systems comprising 879K frames with varying timestep sizos, demonstrating substantial improvements over state-of-the-art models MPP McCabe jet al. (2023) and DPOT Hao et al. (2024) in both accuracy and elficiency. Our approach reduces the averaged last-step rollout error by 97% compared to DPOT and 44.7% compared to MPP, while requiring only 72.5% and 34.8% of their respective inference times.
Enhanced flexibility supporting varying timestep strides and non-soquential measurements, offering substantial robustness in realistic scenarios with imperfect data collection, e.g. with missing frames. Under these scenarios, VICON exhibits only 24.41% % relative performance degradation versus 71.37%- 74.49% degradation in baseline methods.
2 Related Work
Operator Learning. Operator learning Chen & Chen 1995b a): Li et al. (202 Lu et al. (2021) addresses the challenge of approximaling operalors G U + V, where U and V are function spaces representing physical systems and differential equations, e.g., G maps initial/ /buudary conditions or system parameters to corresponding solutions. Among the various operator learning approaches, DeepONets Lu et al. 2021 2022
employ branch and trunk networks to independently process inputs and query points, while FNOs et al.
(2021) leverage fast Fourier transforms to efficiently compute kernel integrations for PDE solutions in regular domains. These methods have been extended to incorporate equation information Wang et al. [2021
ct al. 2024b), multiscale features Zhang ct al. (0224)] Wen et al. (2022] Liu et al.
(222a)) and adaptation
to heterogeneous and irregular meshes Zhang et al. 2003 2024b)
Li et al. (0024a):
Wu et al. 2024).
However, these approaches typically learn a single operator, requiring retraining when encountering different PDE lypes or timestep sizes. Our work aims to develop a unified model for multiple PDEs with few-shot. generalization capabilities based on limited observation frames.
Multi-Physics PDE Models. Foundation models in natural language processing Brown et al. (2020);
Touvron et al. (0223 and computer vision Ramesh et al. (2021) have demonstrated remarkable versatility across diverse tasks. Drawing inspiration from this paradigm, recent rescarch has explored developing unified models for multiple PDEs in domains such as PDE discovery Schaeffer (2017) and computational fluid dynamics Wang et al. (024a) Several approaches follow pre-training and fine-tuning strategies Chen et al.
(2024); Herde et al. (2024), though these require additional resources for data acquisition and fine-tuning before
deployment. In parallel, researchers have developed zero- or few-shot multi-physics models by incorporating
additional information; for instance, the PROSE approach Liu et. al. (2024c bb; [unn al. (2024a) Jollie
et al. 2024]; Sun et al. (2024b) directly encodes symbolic information of PDEs into the model.
Other
methods achieving zero- or few-shot generalization include using physics-informed tokens Lorsung et al. 2024
representing PDE structures as graphs Ye et al. (2024), or conditioning transformers on PDE descriptions Zhou
et al. 2025 However, all these techniques require prior knowledge of the PDEs as additional inputs. We
select two state-of-the-art models, MPP McCabe et al. (0033 and DPOT Hao et al.
2024 as our baselines.
In-Context Operator Networks. ICONs Yang et al. 20023 b Yang Osher (0024 learn operators by
observing input-output function pairs, enabling few-shot generalization across various PDEs without requiring explicit PDE representations fine-tuning. These networks have demonstrated success in handling forward and inverse problems for ODEs, PDEs, and mean-field control scenarios. However, ICONs face significant computational challenges when processing dense data, as they represent functions through scattered point tokens, resulting in quadratic computational complexity with respect to the number of sample points. This limitation has largely restricted their application to 1D problems, with only sparse sampling feasible in 2D cases Yang al. (2023a). Our work addresses these limitations by introducing a vision transformer architecture that efficiently handles dense 2D data while preserving the benefits of in-context operator learning.
3

---

Published in Transactions on Machine Learning Research (02/2020)
Predecien ofO
Prddciin of CoLs
m75)
ufGal
Data sequence with missing frames
Dataset
VICON Model
wetel
wits)
vita
wits)
Seq2saq
missing
Dataset
wite)
RiGy)
leFa)
wibs)
HiGa
+
6
Pair
Estopobis
MiGUl
mib))
pairs only using
Dataset
wite)
wisy)
wita))
wis)
vital
wis)
Pain1
Pain2
014 A4)
uafs Af)
Paint
Par
(8) COND-QOI peir formetion
(6) Model
Imperfect Meesurement
Figure 1: VICON model overview. (a) The formation process for conditions (COND) and quantitics of interest (QOI) pairs. At is randomly sampled during training. (b) Model illustration. The inputs to the model are pairs of COND and QOI, which are patchified and flattened before feeding into the transformer layers. The outputs, which represent different patches in the output frame, transformed back to obtain (c) With imperfect temporal mcasurements, VICON forms pairs using only clean data, and does not need to fill missing frames.
the final predictions.
3 Preliminaries
ICONs were introduced and developed Yang et al. (2023a b): in-context learning framework from large language models, aiming to train a model that learn operators from prompted function examples.
Yang & Osher (2024). They adapt the
Denote an ICON model as To, where T is transformer with trainable parameters 0. The model takes input as sequence comprising / pairs of conditions (c) and quantities of interest (q) as {(c,, H where each c, q contains multiple tokens representing a single function. The model outputs tokens representing future functions at the positions of c, Le., "next function prediction" similar to "next token predictions" in LLMs. More preciscly, for J e {I...., 1), the model predicts qu+1 given the leading J pairs and the condition C2+12
qu+1 Toles+1 ())1)-
(1)
For training parallelization, ICON uses special causal attention mask to perform autoregressive learning (i.e., the model only sees the leading J pairs and CJ41 when predicting (s+1) and enables output of all predictions within a single forward pass Yang et al. (023bb:
(q.) = Tolffe, 1.))
(2)
Training loss is computed as the mean squared error (MSE) between the predicted states qi and the ground truth states qi To ensure that the model receives sufficient contextual information about the underlying dynamics, only compute errors for the indices Imin: effectively requiring at least Imin examples in context. This approach has shown empirical improvements in model performance.
After training, ICON can process a new set of (c, q) pairs in the forward pass to learn operators in-context and employ Ihem to predict future functions using new conditions:
q = Telcu: (c., q H=l.
(3)
where J (Imin..... 1) is the number of in-context examples, Imia is the number of context examples exempted from loss calculation, and the index of the condition for future prediction.
4

---

Published in Transactions on Machine Learning Research (02/2026)
Importantly, all pairs in the same sequence must be formed with the same operator mapping (i.e., the same PDE and consistent timestep size within one sequence), while operators can vary across different. sequences/rows in the training batch to enable the model's generalization ability to different combinations of PDEs and timestep sizes.
4 Methodology
4.1 Problem Setup
We consider the forward problem for multiple time-dependent PDEs that are temporally homogeneous Markovian, defined on domain f2 _ R2 with solutions represented by u(x, t) f x [0, T) + Re, where a the number of channels. Given the initial To frames, {u, = u(;44) = l,..., ......,,ul Iu.. the task is to predict future solutions.
Each solution frame is discretized are the spatial grid sizes. Due to the homogeneous Markovian property, given the initial data fu, t 0} from some PDE indexed by ip. for a fixed At, there exists an operator L = = frame Ut+ for any t 2 0. Our goal is to train model that learns these operators c from the sequence of function pairs generated from the initial frames. Notably, even within the same dataset and fixed At, different trajectories can exhibit different dynamics (e.g., due to different Reynolds numbers Re). These variations are implicitly captured by the function pairs in our framework.
diree-dimensional tensor u, u(:,t) E RRNxXX where N₂ and Ng
(bp) that maps frame u,
4.2 Vision In-Context Operator Networks
The original ICONs represent functions as scattered sample data points, where each data point is projected as a single token. For higher spatial dimensions, this approach necessitates an excessively large nunber of sample points and extremely long sequences in the transformer. Due to the inherent quadratic complexity of the transformer, this approach becomes computationally infeasible for high-dimensional problems.
Inspired by the Vision Transformer (ViT) Dosovitskiy al. (2021). we address these limitations by dividing the physical fields into patches, where each patch is flattened and projected token. This approach, which call Vision In-Context Operator Networks (VICON), addresses the computational limitation of treating each spatial point as an individual token, which Icads to quadratic computational complexity with respect to the number of points. This patch-wise approach has its forward process illustrated in Figure (b) and detailed in the following.
First, the input u, and output ur+at functions are divided into patches {C* E RR Je and (Q' e N, Jim1, where Rz Ry are the resolution dimensions of the patch and No and N,, are the number of patches for the input and output functions, respectively. While Ng can differ from Ne (for instance, when input functions require additional boundary padding, resulting in N₂ > N₂), we maintain N₂ = Nr throughout our experiments. For notational clarity, we add the subscript E [I,...,I} to denote the pair index, where C represents the k-th patch of the input function and Q represents the 1-th patch of the output function in the i-th pair. These patches are then projected into unified d-dimensional latent embedding space using a shared learnable linear function f& : Rzz Ry c + Rdd
RRR xex
(4) where l: 1,..., No and / 1....,,, are the index of patches. We use the same projection fo for both conditions and Qols because in our setup, both represent the same physical fields at different timesteps; sharing the projection enforces consistency in latent space regardless of whether a frame serves context or target.
ct = fo(C).
a =
We inject two types of learnable positional encoding before feeding the embeddings into the transformer: (1) patch positional encodings to indicate relative patch positions inside the whole domain, denoted as Ep E Nxx, where Np = max(NN. and the encoding varies per patch (broadcast across points within each patch); (2) function positional encodings to indicate whether patch belongs to input function
5

---

Published in Transactions on Machine Learning Research (02/2026)
(using E R (x) output function (using E, E RR well as their indices in the sequence, where the encoding varies per condition/Qul in each pair (broadcast across patches and points of that cuditiinn/oolo
c = a. + (p( + Ec(1,, q. - q + Ep(() + (t).
(5)
The embeddings of and of then concatenated to form the imput sequence for the transformer:
cf...oo
No ......... ........c of oy ..... N₂
To support autoregressive prediction similar to Equation 2 VICON employs an alternating sized (in the case where N: + N₂) block causal attention mask, as opposed to the conventional triangular causal attention mask as in mainstream generative large language models Brown et al. (2020) and large vision models Bai The mask is defined as follows:
et al. 2024).
IN
0
0
0
1 Ng
NN
0
0
M =
(6)
IN.NN Ng.Ne
1N..Ng
1 N..N. N.
IN,,Ng
NN
where 1mxn denotes an all-ones matrix of dimension m nz, and 0 represents matrix of the corresponding dimensions.
After obtaining the output tokens from Equation we extract the tokens corresponding to the input patch indices cj ..,,, c and denote them as q ...... These tokens are then projected back to the original physical space using a shared learnable linear function I Rd R R2 x,x.
Q -
which predicts Q-
4.3 Prompt Normalization
To address the varying scales across different channels and prompts (i.e., sequences of pairs {(c, (=)) we normalize the data before feeding them into the model. A crucial requirement is to consistently normalize functions within the same sequence, to ensure that the same operator is learned. Denoting the normalization operators for and as Nr and N₂ respectively, and the operator in the original space C, the operator in the normalized space L' follows:
= ((c(u)))NN(CCC)))
(8)
In this work, we simply No N,,, mainly because our maximum timestep stride Smax 5 rela- tively small (i.e., the scale distribution does not change dramatically). Specifically, given input sequence 9.) compute the channel-wise mean mean(C1, C2 ,,,,). Cr) E Rd and standard deviation == st, .2....,cr) Hd of all conditions {c;Hen, which are then used to normalize both the conditions
and Qols. Training MSE loss is computed in the normalized space. To avoid division by zero, we set a minimum threshold of 10-4 for the standard deviation.
4.4 Datasets and Data Augmentation
We evaluate on three fluid dynamics datasets representing different physical regimes: 1) ODEArena-Incomp Gupta & Brandstetter (2022) (incompressible Navier-Stokes equations), containing 2,496/ 1688/68 /608 trajectories (train/valid, (testi with 56 timesteps each; 2) CoeeBench-Comp-HHihhhh Takamoto et al. (0222) (compressible Navier-Stokes with high viscosity), containing 40,000 trajectories of 21 timesteps
each; and 3) PDE.ench--Comph Vis Takamoto al. (022 (compressible Navier-Stoker with numerically

---

Published in Transactions on Machine Learning Research (02/2026)
zero viscosity), containing 4,000 trajectories of 21 timesteps each. For PDEArena-Incomp: we follow the origi- nal data split in Gupta & Brandstetter (2022). For PDF:Bench-Comp--iigh and PDFBench-Comp-Toww we randomly split trajectories in 80% 10% 10% proportions for training/ ialliing/valdaation validation /testing,tt respectively. For all datasets during rollout, we set Iu- = 10, i.c. we predict trajectories up to the last timestep given the initial frames.
The temporally homogeneous Markovian property (Section enables natural data augmentation for training and fiexible rollout strategy for inference (Section 4.5 by striding with larger timesteps to reduce the number of autoregressive steps in long-term prediction tasks: At= sÂTr, is the timestep size for recording in trajectory ip. During training, for cach trajectory in the training set, we first sample stride size s ~ li,,..., Smax), then form (uz, Ur+aAr) pairs for the corresponding operator. We illustrate this augmentation process Figure ! a), where different strides are randomly sampled during dataloading.
1,2,.... Smax, where Ti,
Additional dataset details appear in Appendix
4.5 Inference with Flexible Strategies
VTCON can make predictions with varying timestep strides. Given To initial frames {u:) can form in-context example pairs {(u,, ue+s) (247)) For j 2 1, set the question condition (cx in Equation 3 as uj to predict uj+a, enabling s-step prediction.
As mentioned in Section 4.4
For long-tern term rollout, we employ VICON in an autoregressive fashion. The ability to predict with different timestep strides enables various rollout strategics. We explore two natural approaches: singlo-step rollout and flexible-step rollout, where the latter advances larger strides to reduce the number of rollout steps.
For single-step rollout, we simply follow the autoregressive procedure with s 1. Flexible-step rollout involves more sophisticatod approach: given maximum prediction stride Smax, first make sequential predictions with a gradually growing each frame in this sequence as question condition, we then make consistent smax-step predictions while preserving all intermediate frames. This flexible-step strategy follows the approach in Yang & Osher (2024).
to 8max using U1 as the question condition, obtaining (wo++ J Using
This strategy also applicable to imperfect measurement cases where partial input frames are missing, e.g., due to sensor device error. In this case, we can still form pairs from the remaining framos, with minor adjustments to the strategy generation algorithms, as shown in Figure [c).
More details (including algorithms and examples) on rollout strategies provided in Appendix C.1 (for perfect measurements) and in Appendix C.2 (for imperfect measurements).
4.6 Evaluation Metric
We evaluate the rollout accuracy of VICON using two different strategies described in Section 4.5 evaluation uses relative and absolute 1 errors between the predicted and ground truth frames, starting from the frame To + 1. For relative scaling coefficients, we use channel-wise scaling standard deviation a of ground truth frames, which vary between different prompts.
The
5 Experimental Results
We benchmark VICON against two statc-of-tho-art scquence-to-soquenne models: DPOT Hao et al. 2004 (122M parameters) and MPP McCabe et al. (2023) (AViT-B architecture, 116M paramelers), as well as
Specialist U-Net baselines Gupta & Brandstetter (2022) trained individually per dataset. Implementation
details for all basclines are provided in Appendix B The vanilla ICON, which would require processing over 114K tokens per frame 128x7, exceeds our available GPU memory and is thus computationally
infeasible for direct comparison these datasets.
7

---

Published in Transactions on Machine Learning Research (02/2026)
(a)
VICON
DPOT
MPP
(b)
PDEAAnn-Incconp
7DEBBenca- Comp LowVis
7DEBencb Comp-
PDEArem--Iccomp
08
25
relative error
Rollout Errors
0.6
20
0.5
0.4
single
15-
step
0.2
10-
0.2
man step
slep
0.2
time
caslem drup
rasder
ranckon drup
bail Irspacncy
VIDON
DROT
MPP
Rollow perato
Mede:
25
120
Rollout Errors
20-
100
40
Figure 2: Main experiment results. (a) Last step rollout errors on 3 datasets. VICON outperforms MPP on all datasets and outperforms DPOT on 2 datasets. (b) VICON allows flexible rollout strategies to reduce error accumulation and demonstrates stride extrapolation. (c) VICON is robust to imperfect temporal measurements, while MPP and DPOT suffer from performance degradation. (d) VICON smaller size and has faster rollout time per step.
For evaluation, we use both absolute and relative RMSE (Section 4.6 on rollout predictions. Since VICON offers the flexibility of predictions with different timestep strides, we evaluate our single trained model using both single-step and flexible-step rollout strategies (Section 4.5 during inference.
For conciseness, present summarized plots and tables in the main text, and defer complete results and visual- izations to Appendix D Ablation studies examining key design choices of VICON--ICchlii positional encoding, and context length, as well as an alternative CNN-based architecture are presented in Appendix D.1 The chockerboard artifacts (c.g., in Figure of the COEBBench-Comp-Highh datasct are discussed in Appendix D.2
patch resolution,
Table Summary of Rollout Relative 2 Error (scaled by std) across different methods and datasets. The best results are highlighted in bold. For flexible step rollout, step 3 works best for the PDEArena-Incomp dataset. Specialist U-Net is trained individually per datasct with instance normalization matching VICON's preprocessing.
Rollout Relative 1! Error
Case
Ours (single step)
Ours (flexible step)
DPOT
MPP
Specialist U-Net
#DEAvens-nnooug
76.77
68.03
65.27
93.52
18.8
Last step
[ee2]
PDEBeach- Comp-Low Vis
39.11
39.11
48.92
65.32
55.9
PDEBench- DOEBench-Comp-ERe Homp-HigV-
61.41
61.41
2866
185.3
PDEArena- Incomp
56.27
48.50
41.20
55.95
27.9
average [1e-2]
PDEBeach. LowVis
27.08
27.08
37.72
46.68
35.7
PDEBench- Ceeteench-Coop--ee High-
30.06
30.06
821.9
72.37
19.9
5.1 Superior Performance on Long-term Rollout Predictions
As demonstrated in Figure aa), Table 1 and Figure 3 VICON consistently outperforms bascline methods on long-horizon predictions across all benchmarks with the exception of DPOT on the PDEArena-Income dataset, where performance is comparable. Overall, VICON achieves average

---

Published in Transactions on Machine Learning Research (02/2026)
VICON bingle sap) VICON (fechle
VICON bingle mapi VICON (Texhle
relative error
relative error
MPP
relative error
DPOT
DPOT
0.4-
10
0.4
0.5
VICON (fexahle step?
VICON (sngle step)
MP?
0.2-
10"
DPOT
20
time
time
time
(a) PDEBench COEEBenh--Comp-LoWWee Vis
(b) PDEBench- Seeth--Co----iggggh
(c) PDEArena-Inonne
Figure 3: Comparison of rollout errors (scaled by std) across different datasets and mod- els. We show errors for two VICON rollout strategies: single step rollout and flexible rollout strategy. For flexible step, step 3 works optimally for the PDEArena-Incomp datasct, while step works best for CreBeech-ComppLLww Vis and CoBBench-Comp--HilNWi
reduction in relative L RMSE at the final timestep of 37.9% compared to DPOT and 44.7% compared to MPP.
Notably, DPOT exhibits exceptionally poor performance PDEeBenh--Comp-iihh with an 821.9% error compared to our 30.06%. Our visualization of failure cases revcals that DPOT struggles with trajectories with small pressure values compared to the dalaset average. This may stem from DPOT's lack of prompt normalization, as compressible flow's pressure channel exhibits large magnitude variations.
For the PDEArena-Income datasct, while VICON slightly underperforms DPOT and MPP initially, quickly surpasses MPP and achieves comparable performance to DPOT for longer-step rollouts. This demonstrates VICON's robustness in long-term predictions. Despite DPOT's marginally better performance here, its poor dataset limits its applicability in multi-physics settings.
performance on the HelBeecch-Comp-llieee
To further contextualize VICON's performance, we compare against Specialist U-Net baselines (U-Net-Mod64 from PDEArena Gupta & Brandstetter (2022)) trained individually per dataset. The only modification we made adding instance normalization matching VICON's preprocessing, which we found essential for stabi- lizing training on PDEBench Sorth-Comp-oooo Vis and PDEBench- CowcchhCCo--ieee Caie-ooomp--iiii datasets. As shown in Table I while the specialist U-Net achieves strong performance individual datasets it was optimized for, VICON remains competitive across all datasets and even outperforms the specialist on DDEBBench-Comp-lle demonstrating the advantage of our multi-physics approach in handling diverse physical systems without per-dataset architectural engineering or retraining.
More detailed results are provided in Table 13 and Table 14 in Appendix D
5.2 Flexible Rollout Strategies
As demonstrated in Figure (b and Figure 5 VICON can select appropriate timestep strides based on each dataset's characteristics. While PDEBench- with single-step rollout, PDEArena-Tncom achieves optimal results with stride of using our flexible-step rollout stratogy.
CeBBenchCComp-LLw Comp-LowVis and PDEBench- FDEBench-ComppHHighe perform best
This dataset-dependent performance reflects the characteristics of each dataset and the challenges of learning larger timestep strides. PDEBench- TereBenchhCCopp-Lww// Vis and PDEBench-Come Comp-HighVis record data with larger timestep sizes, making the learning of larger stride predictions more challenging due to increased dynamics complexity. Additionally, the fewer number of total rollout steps (even with stride=1) in these datasets cancels out the benefits of reducing rollouts when using multi-stride. Conversely, PDEArena-Incom, records data with smaller timesteps, making both single- and multi-stride prediction lcarning more feasible. Under this condition, multi-stride reduces the total rollout steps and error accumulation, achieving an 11.4% error reduction with balanced stride (Smax = 3).
9

---

Published in Transactions on Machine Learning Research (02/2026)
When evaluated with unseen strides (Smax 7), VICON maintains comparable performance to single-step strategies for PDEArema-Tnoome indicating that it extracts underlying operators through context pairs rather than memorizing dynamics. This generalization capability provides tolerance when deployed to real-world settings where device sampling rates often differ from a fixed training sct-an an advantage we further demonstrate the next section.
5.3 Importance of In-Context Learning
To validate the importance of in-context learning, compare VICON against non-in-conlext VIT baseline (with similar architecture: 162M parameters, 12 layers, hidden dimension 1024 for attention, and 4096 for FFN, time stride At = 1 for training) on the PDEArena-Incomp dataset. The results in Table demonstrate that vanilla ViT cannot generalize to different timestep strides without in-context examples:
Table 2: Non-in-context ViT baselinc comparison on PDEArena-Incomp dataset
Model
Param Size
Stride 1 Error
Stride 2 Error
(1e-2)
(le-2)
VTCON (Last step)
88 M
76.77
68.03
VIT (Last step)
162 M
73.94
95.03
VICON (All average)
M
56.27
48.50
VIT (All average)
162 M
50.17
63.15
The significant performance degradation (95.03 vs 73.94 for last step error) demonstrates that vanilla ViT cannot generalize to different At without in-context examples, validating our approach's effectiveness.
Additionally, evaluated the impact of incorrect context pairs VICON's performance using the IDEArena-Incoma dataset (Table 3
Table 3: Impact of incorrect context pairs VICON performance
Experiment Setting
Average Rollout Error
Correct (stride 1)
0.55
Mix context pairs with stride and
0.88
Context pairs with stride
0.92
Random noise context pairs
1.57
The clear performance degradation with incorrect context pairs confirms that VICON effectively utilizes in-context information rather than simply memorizing patterns.
5.4 Robustness to Imperfect Temporal Measurements
Real-world experimental measurements frequently suffer from imperfections samplssnaannggggg rates may differ from the training set, and frames can be missing due device errors. Sequenceeto-ssquunee models like MPP and DPOT require either retraining with new data or interpolation that introduces noise. In contrast, VICON elegantly handles such imperfections by forming context pairs from available frames only (Figure (c). The algorithm for generating pairs with imperfect measurements appears in Appendix C.2
We evaluate VICON against baselines on all benchmarks under two scenarios: (1) half sampling rate (dropping every other frame) and (2) random frame dropping (removing 1-3 frames per sequence). Results for the PDEBencch-C PDEBench-CCmpp-oow
dataset Figure 2c) (with complete results in Figure X show that VICON expori- ences only 24.41% relative performance degradation compared to 71.37% 44.4%% baselines demonstratnn remarkable nosustness to measurement imperfections.
10

---

Published in Transactions on Machine Learning Research (02/2026)
5.5 Turbulence Kinetic Energy Analysis
Turbulence kinetic energy (TKE), defined as ( + ug), is a critical metric for quantifying model's i ability to capture turbulent flow characteristics. Here, u equilibrium state u.
denotes the fluctuation of velocity from its statistical
Within our datasets, only a subset of PDE:BenchCComp-ooww developed turbulent fields (see Appendix D.5 of Takamoto et al. (2022)) ) is suitable for TKE analysis. After filterigg these entries, we compare the mean absolute error (MAE) of the TKE between VICON, DPOT, and MPP. Our approach achieves TKE error of 0.016, significantly outperforming MPP's error of 0.049, while achioving performance comparable to DPOT's error of 0.012. Visualizations of TKE errors are presented in Figure in Appendix D
specifically those initialized with fully
5.6 Benefits of Multi-Physics Joint Training
We empirically examine whether VICON benefits from multi-physics training by comparing two strategies: (1) training a single model jointly on all three datasets versus (2) training separate models specialized for individual datasets. For fair comparison, we maintained identical batch sizes across all trainings while adjusting the steps for separate training to be slightly more than one-third of the joint training duration, ensuring comparable total computational costs between approaches.
As shown in Figure joint training significantly outperforms separate training across all three datasets. We attribule this performance gain to the underlying physical similarities across these fluid dynamics problems, where exposure to diverse yet related flow patterns enhances the model's generalization ion ability.
5.7 Context Sampling Strategy
By default, VICON forms context pairs from initial frames within a single trajectory, which we term intra- trajectory context. This design choice is motivated by practical online deployment scenarios where only a few initial mcasurements are available, and system parameters (e.g., viscosity, forcing terms) may be unknown-man making retrieval of historical trajectories with the same underlying operator infeasible.
As theoretical question, additionally investigate whether VICON can leverage cross-trajectory context, i.e., context pairs from different trajectories governed by the same operator. As detailed in Appendix D.3. VICON trained with cross-trajectory context achieves similar performance with mixed contexts during rollout (only 0.012 increase in average error). This demonstrates the model's capability to leverage cross-tras coos-trajectory information when available. Nevertheless, we argue that uutratrraecctoy context remains more practical it avoids the need for maintaining trajectory databases and online operator identification.
5.8 Computational Efficiency
In Figure d) and Table 16
compare the computational resources required by VICON and baselines. Our model demonstrates superior efficiency across multiple metrics. Compared to MPP, VICON requires approximately one-third of the inference time per frame while using 75% of the total parameters. VICON also outperforms DPOT, requiring 28% fewer parameters and 28% less inference time.
It is important to note that, theory, VICON incurs approximately 4x computational cost compared to non-paired, sequence-lo-ssqquncce baselines when all other settings (e.g., patch size) are identical, as it processes both condition and Qol, doubling the total token length with quadratic attention complexity. However, this computational overhead is justified by the significant performance improvements and unique capabilities (handling imperfect measurements, flexible timestep strides). Additionally, this cost can be reduced to 2x (linear) at inference time using KV caching, where all context pairs except the final pair remain fixed across rollout steps, allowing their key-value multiplications to be cached.
It important to note that while post-training (such as fine-tuning) on a sequence-to-ssquenec model may achieve the same accuracy on one physical condition and save this 2x cost overhead, the fine-tuned model
11

---

Published in Transactions on Machine Learning Research (02/2026)
loses the ability to predict accurately when physical parameters may change on-the-fly during deployment, while VICON provides superior flexibility.
6 Conclusion and Future Work
We present VICON, a physical fields through patch-wise operations. VICON overcomes the computational burden of original in- context operator networks in higher spatial dimensions while preserving the flexibility lo extract multi-physics dynamics from few-shot contexts. Our comprehensive experiments demonstrate that VICON achieves superior performance in long-term prodictions, roducing the average last-step rollout error by 37.9% compared to DPOT and 44.7% compared MPP, while requiring only 72.5% and 34.8% of their respective inference times. The model supports flexible rollout strategies with varying timestep strides, enabling natural application to imperfect real-world measurements where sampling frequencies differ frames are randomly -copped-scenarios scenarios where VICON experiences only 24.41% relative performance degradation compared to 71.37%-74.499 degradation in baseline models.
in-context operator network that efficiently processes dense
Despite these advances, several challenges remain for future investigation. We empirically showed in Section 5.6 the benefits of multi-physics training, yet scaling to larger and more diverse datasets presents practical challenges. Specifically, generating high-fidelity physics simulation data across multiple domains requires substantial computational resources and specialized domaín expertise including mosh generation, physical modeling, and numerical solver integration.
Furthermore, the current approach does not yet extend to 3D applications, as token sequence length would grow cubically, excoeding our computational budget. Potential remedics for 3D scaling include: (1) using VAE-like vechniques/transiine transformerss lo compress and evolve the fields the latent space, then applying attention mechanisms to decode at any location Wang et al. (2024b); Li et al. (2023), and (2) employing crop-like methods to learn evolution of fixthwidth windows similar to CNNs. These approaches could potentially be combined to address the cubic scaling challenge.
Since VICON is an autoregressive model, it still suffers from error accumulation of the final pair's condition frame during rollout. Future work could explore resolving error accumulation through techniques such as injecting Gaussian noise or using push-forwand methods Brandstetter et al. (2021).
The channel-union approach is also limited when incorporating domains beyond fluid dynamics with funda- mentally different state variables. Recent advances in inter-channel attention mechanisms Holzschuh et al. (2025) offer promising directions for handling channel scaling issues when expanding to more diverse physics systems.
Finally, adapting VICON to handle irregular domains such as graphs or meshes would broaden its applications to areas like solid mechanics and molecular dynamics. Techniques such as geometric deep learning extensions, masked patch modeling, or point-cloud modeling are potential future directions to handle irregular geometries. The original ICON framework has demonstrated success across various problem types including inverse problems and steady-state tasks Yang et al. (2023at, suggesting potential for extending VICON beyond forward temporal prodiction. Addressing these challenges will advance the promising paradigm of in-context. learning for a broader range of physical systems.
Acknowledgments
Yadi Cao and Rose Yu are supported in part by the US Army Research Office under Army-ECASE award 102201122-100031, the US Department of Energy, Office of Science, IARPA HAYSTAC Program, CDC- RFA-FT-23-0066 DARPA AIE FoundSci, DARPA YFA, NSF grants #2205093, #2100237, #2146343, and #2134274. Yuxuan Liu and Hayden Schaeffer are supported in part by AFOSR MURT FA9550-21-1-0088 and NSF DMS 2427558. Liu Yang is supported by the NUS Presidential Young Professorship. Stanley Osher is partially funded by STROBE NSF STCS87 DMR 1548924, AFOSR MURI FA9550-18 18-502 and ONR
N00014-20-1-277
12

---

Published in Transactions on Machine Learning Research (02/2026)
References
Y. Bai, X. Geng, K. Mangalam, A. Bar, A. L. Yuille, T. Darrell, J. Malik, and A. A. Efros. Sequential modeling enables scalable learning for large vision models. In Proceedings of the IEEE/CVP Conference Computer Vision and Pattern Recognition, pp. 22861-22877 2024.
J. Brandstetter, D. E. Worrall, and M. Welling. Message passing neural pde solvers. In International Conference on Learning Representations, 2021.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, and others. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
S. Cao. Choose a transformer: Fourier or galerkin. Advances in neural information processing systems, 34: 24924- 24924-24940, 2021.
Y. Cao, M. Chai, M. Li, and C. Jiang. Efficient learning of mesh-based physical simulation with bi-stride multi-scale graph neural network. In International Conference on Machine Learning, pp. 3541 3558, 2023.
T. Chen and H. Chen. Approximation capability to functions of several variables, nonlincar functionals, and operators by radial basis function neural networks. IEEE Transactions on Neural Networks, 6(4):904-910, 1995a.
T. Chen and H. Chen. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE transactions neural networks, 6(4): 911-917, 1995b.
W. Chen, J. Song, P. Ren, S. Subramanian, D. Morozov, and M. W. Mahoney. Data-efficient operator learning unsupervised pretraining and in-context learning. ai preprint arXiv:2402. 15794, 2024.
Y. Dang, 7. Hu, M. Cranmer, M. Fickenberg, and S. Ho. Tnt Vision transformer for turbulence simulations. arxiv preprint ar xrXi::2077044116 2022.
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, Uszkoreit, and N. Houlsby. An image worth 16x16 words: Transformers for image recognition scale. In International Conference on Learning Representations, 2021. URL
J. K. Gupta and J. Brandstetter. Towards mitiisppttiotemporllssaaee generalized pde modeling. arXiv preprint XrX:::2209. 15616, 2022.
Z. Hao, C. Su, S. Liu, J. Berner, C. Ying, H. Su, A. Anandkumar, J. Song, and J. Zhu, Dpot: Auto-regressive denoising operator transformer for large-scale pde pre-training. arXiv preprint arxiv:2403. 03540..2..3..... 2024.
M. Herde, B. Raonić, T. Rohner, R. Käppeli, R. Molinaro, F. Bézenac, and S. Mishra. Poseidon: Efficient. foundation models for pdes. arxie preprint 191101 2024.
B. Holzschuh, Q. Liu, G. Kohl, and N. Thuerey. Pde-transformer: Efficient and versatile transformers for physics simulations. arXiu preprint 2tb:2505.247111 2025.
J. Jiang, G. Li, Y. Jiang, L. Zhang, and X. Deng. Transcfd: A transformer-based decoder for flow field prediction. Engineering Applications of Artificial Intelligence, 123:106340, 2023.
D. Jollie, J. Sun, 7. Zhang, and H. Schaeffer. Time-series forecasting, knowledge distiilation, and refinement within a multimodal pde foundation model. arXiv preprint 3rXiv:2409.11609 2024.
N. Kovachki, Z. Li, B. Liu, K. Azizadeneshhel,, K. Bhattacharya, A. Stuart, and A. Anandkumar. Neural operator: Learning maps between function spaces with applications PDEs. Journal of Mochine Learning Research, 24(89):1-4) 2023.
13

---

Published in Transactions on Machine Learning Research (02/2026)
S. Li, Y. Zhao, R. Varma, O. Salpekar, P. Noordhuis, T. Li, A. Paszke, J. Smith, B. Vaughan, P. Damania, and others. Pytorch distributed: Experiences on accelerating data parallel training. arxiv preprinl ar arXiu:2006.15704, 2020.
Z. Li, N. B. Kovachki, K. Arizzaenesheli,, B. liu, K. Bhattacharya, A. Stuart, and A. Anandkumar. Fourier neural operator for paramctric partial differential equations. In International Conference on Learning Represenlations, 2021. URL
Z. Li, K. Mcidani, and A. B. Farimani. Transformer for partial differential equations' operator learning. arxiv preprint arXic:2205. 2022.
Z. Li, D. Shu, and A. Barati Farimani. Scalable transformer for pde surrogate modeling. Advances in Neural Information Processing Systems, 36:28010-28039, 2023.
Z. Li, N. Kovachki, C. Choy, B. Li, J. Kossaifi, S. Otta, M. A. Nabian, M. Stadler, C. Hundt, K. Azizzadenssheli, and others. Geometry-informeel neural operator for large-scale 3d pdes. Advances in Newral Information Processing Systems, 36, 2024a.
Z. Li, H. Zheng, N. Kovachki, D. Jin, H. Chen, B. Liu, K. Azizadenesheli,, and A. Anandkumar. Physics- informed neural operator for learning partial differential equations. ACM/JMS Journal of Data Science, (3):1 27, 2024b.
M. Lino, S. Fotiadis, A. A. Bharath, and C. D. Cantwell. Multi-scale rotation-equivariant graph neural networks for unstcady culerian fluid dynamics. Physics of Fluids, 34(8):087110, 2022.
X. Liu, M. Zhu, L. Lu, H. Sun, and J. Wang. Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics. Commumications Physics, 7(1):31, 2024a.
Y. Liu, J. Sun, X. He, G. Pinney, Z. Zhang, and H. Schaeffer. Prose-fd: A multimodal pde foundation model for learning multiple operators for forecasting fluid dynamics. arx preprint orXiv:2409.0900 09811, 2024b.
Y. Liu, Z. Zhang, and H. Schaeffer. Prose: Predicting multiple operators and symbolic expressions using multimodal transformers. Neural Networks, 180:106707, 2024c.
C. Lorsung, Z. Li, and A. B. Farimani. Physics informed token transformer for solving partial differential equations. Machine Learning: Science and Technology, 511:0055 2024.
L. Lu, P. Jin, G. Pang, Z. Zhang, and G. F. Karniadakis. Learning nonlinear operators via DeepONet basod on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218 229, 2021.
L. Lu, X. Meng, S. Cai, Z. Mao, S. Goswami, Z. Zhang, and G. E. Karniadakis. A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data. Computer Methods in Applied Mechanics and Engineering, 393:1 147788, 2022.
M. McCabe, B. Régaldo-Saine Saint Blancard, L. H. Parker, R. Ohana, M. Cranmer, A. Bictti, M. Eickenberg, S. Golkar, G. Krawezik, F. Lanusse, and others. Multiple physics pretraining for physical surrogate models. In NeurIPS 2023 AT for Science Workshop, 2023.
O. Ovadia, A. Kahana, P. Stinis, E. Turkel, D. Givoli, and G. E. Karniadakis. Vito: Vision transformer- operator. Computer Methods in Applied Mechanics and Engineering, 428:117109, 2024.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, 7. Lin, N. Gimelshein, L. Antiga, and others. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, and I. Sutskever. Zero-shot (ext-to-image generation. In International conference on machine learning, pp. 8821 8831. Pmlr, 2021.
A. Ganzaez, J. Godwin, T. Pfaff, R. Ying, J. Leskovee, and P. Battaglia. Learning to simulate complex physics with graph networks. In International conference on machine learning, pp. 8459 8468. PMIRR 2020.
14

---

Published in Transactions on Machine Learning Research (02/2026)
H. Schaeffer. Learning partial differential equations via data discovery and sparse optimization. Proceedings of the Royal Sociely A: Mathemalical, Physical and Engineering Sciences, 473((1977:20100444; 2017.
J. Sun, Y. Liu, Z. Zhang, and H. Schaeffer. Towards a foundation model for partial differential equation: Multi-operator learning and extrapolation. arXiv preprint 2xx::244411255, 2024a.
J. Sun, Z. Zhang, and H. Schaeffer. Lemon: Learning to learn multi-operator networks. ar preprinl arXiv:2408.1610 16168, 2024b.
M. Takamoto, T. Praditia, R. Leiteritz, D. MacKinlay, F. Alesiani, D. Pflüger, and M. Niepert. Pdebench: An extensive benchmark for scientific machine learning, Advances in Neural Information Processing Systems, 35:1596-166 2022.
H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Ham- bro, F. Azhar, and others. LLaMA: Open and efficient foundation language models. ar preprinl 2302 0ate:3302.19991,, 2023.
H. Wang, Y. Cao, Z. Huang, Y. Liu, P. Hu, X. Luo, Z. Song, W. Zhao, J. Liu, Sun, and , others. Recent advances machine learning for computational fluid dynamics: A survey. arXic preprint arX1::408..2217,, 2024a.
S. Wang, H. Wang, and P. Perdikaris. Learning the solution operator of parametric partial differential equations with duysics-informet DeepONets. Science Advances, x(40):eabi$605, 2021.
S. Wang, J. H. Seidman, S. Sankaran, H. Wang, G. J. Pappas, and P. Perdikaris. Cvit: Contimuous vision transformer for operator learning. arXiv preprint arxi:22405. 19998, 2024b.
G. Wen, Z. Li, K. Azizzadenesheli, A. Anandkumar, and S. M. Benson. U-fno an enhanced fourier neural operator-based deep-learning model for multiphase flow. Advances in Water Resources, 163:104180, 2022.
Wu, H. Luo, H. Wang, J. Wang, and M. Long. Transolver: A fast transformer solver for pdes on general geometries. Xixiv preprint ar orXiv:2402.0336 023666, 2024.
L. Yang and S. J. Osher. PDE generalization of in-context operator networks: A study on 1D scalar nonlincar conservation laws. Journal of Computational Physics, 519:113379, 2024. ISSN 0021-9991. doi: https://doo..rr/10...00 1tp.://11......2....1..1 pi1/80021999124006722
URL httpps://ww..eeiieeedere.t. http///ddrttttcccc///////////taa
L. Yang, S. Liu, T. Meng, and S. J. Osher. In-context operator learning with data prompts for differential equation problems. Proceedings of the National Academy of Sciences, 120(39)):0311114212) 2023a.
L. Yang, S. Liu, and S. J. Osher. Fine-tune language models as multi-modal differential equation solvers. Xixiv preprint ar Xix:2290 05061, 2023b.
Z. Ye, X. Huang, L. Chen, H. Liu, Z. Wang. and B. Dong. PDEformer: Towards a foundation model for one-dimensional partial differential equations. In ICLR 2024 Workshop on A1ADiferrniialBputiins In Science, 2024. URL https: /tttps://openreview wttp:arwwwww...eeeoeuueeeee net wtt.://ooormmii-GDDDiidid...
Z. Zhang, L. Wing Tat, and H. Schaeffer. Belnet: basis enhanced learning, a mesh-free neural operator. Proceedings of the Royal Society A, 20222004)),023304 2023.
Z. Zhang, C. Moya, W. T. Leung, G. Lin, and H. Schaeffer. Bayesian deep operator learning for homogenized to fine-scale maps for multiscale pdc. Multiscale Modeling 8 Simulation, 22(3): 2024a.
7. Zhang, C. Moya, L. Lu, G. Lin, and H. Schaeffer. D2no: Efficientt handling of heterogencous input function spaces with distributed deep neural operators. Computer Methods in Applied Mechanics and Engineering, 128:117084, 2024b.
Z. Zhang, C. Moya, L. Lu, G. Lin, and H. Schaeffer. Deeponet as multi-operatorr extrapolation model: Distributed pretraining with physics-informed fine-tuning. arXiv preprint ar arXic:2/11.07220 2024c.
15

---

Published in Transactions on Machine Learning Research (02/2026)
H. Zhou, Y. Ma, H. Wu, H. Wang, and M. Long. Unisolver: PDE-conditional transformers are universal
neural PDE solvers. In ICLR 2025 Workshop on Foundation Models in the Wild, 2025. URL https:
//openreview.o //ppenrevie..nt//forrrr// http://orrru//iiio//lgggg
16

---

Published in Transactions on Machine Learning Research (02/2026)
A Dataset Details
A.1 PDEArena-Incomp Dataset
The incompressible Navier-Stokes dataset comes from PDEArens rena Gupta & Brandstetter (2022 The data are generated from the equation
dc + u Vu = -Vp usu f:
(9)
V u O.
(10)
The space-time domain is [0,32] [18, 105] where dt 1.5 and the space resolution is 128 128. A scalar particle field is being transported with the fluids. The velocity fields satisfy Dirichlet boundary conditions, and the scalar field satisfies Neumann boundary conditions. The forcing term f is randomly sampled. The quantities of interest are the velocities and the scalar particle field.
A.2 PiEBinch--onpp--ihhiis and Co Datasets
The DoEBBecch-Comp--ighhy and PDEBench-Comp-Lone Vis datasels come from PDEBench Compressible Navier-Stokes dataset Takamoto et al. 2022 The data are generated from the equation
0.p+ V (pu) =
(11)
p00:u +u. Vu) VP + nAu + (//n33)VV u)
(12)
pu 0 8 2 - ( ( u u o d))
pu²
(13)
The space-time domain is T [0, where di 0.05. The datasets contain different combinations of shear and bulk viscosities. We group the ones with larger viscosities into the PDEBench CeateenchhCoop-Hiiee HighVis dataset, and the ones with extremely small (1e-8) viscosities into the PDEBench-Comp--Lnn PDEBench-Comp Comp-HighVis dataset has space resolution 128 x 128. The PDEBench- PDEBard-ComppLoww> Vis dataset has raw space resolution 512 x 512 and is downsampled to 128 x 128 through average pooling for consistency. The quantities interest are velocities, pressure, and density.
dataset. The
A.3 QOI Union and Channel Mask
Since each dataset contains different sets of quantities interest, we take their union to create a unifed representation. The unified physical field has 7 channels total, with the following ordering:
Density (p)
2. Velocity in the x direction (uz)
Velocity in the y dircction (uy)
4. Pressure (P)
Vorticity (w)
6. Passively transported scalar field (S)
Node type indicator (0: interior node, 1: boundary node)
For each dataset, we use a channel mask to indicate its valid fields and only calculate loss on these channels. The node type channel is universally excluded from loss calculations across all datasets.
17

---

Published in Transactions on Machine Learning Research (02/2026)
Table 4: Model Configuration Details
Patch Configuration
Input patch numbers
8 X
Output patch numbers
8 x
Patch resolution
16 16
Positional Encodings Shopes
Patch positional encodings
64, 1024
Function positional encodings
20, 1024
Transformer Configuration
Hidden dimension
1024
Number of attention heads Feedforward dimension
8
2048
Number layers
10
Dropout rate
0.0
Number of COND) & QOI pairs
10
Number of QOI excmpted from loss calculation
5
Table 5: Optimization Hyperparameters
Paramcter
Value
Learning Rate Schedule
Scheduler
Cosine Annealing with Linear Warmup
Peak learning rate
x 10
Final learning rate
1 X 10-7
Warmup steps
20,000
Total steps
200,000
Optimization Settings
Optimizer
AdamW
Weight decay
1 x 10-1
Gradient norm clip
1.0
B
Experiment Details
B.1
VICON Model Details
Here we provide key architoctural parameters of VICON model implementation in Table
B.2
Training Details
We implement method in PyTorch Paszke et al. (2019) and utilize data parallel training Li et al. (2020) across Iwo NVIDIA RTX 4090 GPUs. We employ the AdamW optimizer with cosine learning rate schedule that includes linear warmup phase. We apply gradient clipping with a maximum norm of 1.0 to ensure training stability. All optimization parameters are detailed in Table 5
B.3
MPP Details
For a fair comparison, we retrain MPP McCabe al. (2023) using our training datasct with the same model configurations and optimizer hyperparameters (batch size to the maximum possible on our device). We evaluate MPP using the same testing setup and metric.
18

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 1: CenerattSingccceesStragggg
Input: D: Number of demonstrations to use
R: Number of ground truth reference steps
T: Total steps in sequence
Output: S: Strategy list. of example pairs and question pairs
S []
Ei range(0, D)
/* example input indices: ...,...p. D-1 */
Fo <- raagee(, 1 D+11)
/* example output indices: 1,2,... */
for i from R to T-1 do
Qi i--
/* question input is previous frame */
Qo
/* question output is current frame */
(Fi,, Eo), Qi, Qo)
end return S
B.4 DPOT Details
For a fair comparison, we retrain DPOT Hao et al. (2024) using our training dataset with the same model configurations and optimizer hyperparameters (batch size set to the maximum possible on our device). We evaluate DPOT using the same testing setup and metric.
B.5 Specialist U-Net Details
We U-Net-Mod64, the strongest baseline from the PDEArena benchmark Gupta & Brandstetter (2022) trained individually per dataset. This architecture employs U-Net with modified residual blocks and 64 base channels. We keep the architecture consistent across all datasets, only adjusting the input/ aaput/outpe /output channel dimensions to match each dataset's state variables. We believe this is fair because realistic deployment, one cannot architecturally re-engincer a model for every new measurement stream encountered.
Additionally, we include instance normalization matching VICON's preprocessing (channel-wise normalization using statistics computed from initial frames), which we found essential for stabilizing training on the PDEBench datasets OPDEBench-Compp Coap--oomp--iiii LowVis and PDEBench-Comp HighVss... Without this normalization, training on compressible flows diverges due to the large magnitude variations in the pressure channel. The batch size is set to the maximum possible on our device. Each specialist model is trained on its respective dataset only, in contrast to VICON's joint multi-physics training.
C Algorithms and Examples of Strategy Generation
C.1 Strategy with Full Temporal Sequence
Algorithm for single-step strategy. Algorithm D presents approach for generating single-step rollout strategies when all temporal frames are available. This strategy maintains fixed stride of 1 between consecutive frames, providing stable but potentially error-accumulating predictions.
Algorithm for flexible-step strategy. Algorithm 2 describes our approach for generating flexible-step strategics with variable strides up to a maximum value. This approach reduces the total number rollout steps required, potentially mitigating error accumulation for long sequences.
Rollout Strategy Example. We demonstrate two rollout strategies in Tables and 7 showing single-stop and flexible-step strategies, respectively. In both cases, the initial frames span from time step 0 to 9, and we aim to predict the trajectory up time step 20.
19

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 2: GenorateFlxiibeeteeSStttee
Input: D: Number of demonstrations to use
R: Number of ground truth reference steps
M: Maximum stride between pairs
T: Total steps in sequence
Output: S: Strategy list of example pairs and question pairs
/* Require R > + 1 ensure examples exist for each stride
S <-[]
Ec + +
/* example condition indices by stride */
Eq + 0
/* example query indices by stride */
/* Prepare example pairs for each stride
for s from to M do
Nd min(D, R)
/* available examples for this stride */ / output indices for examples */
Eqo < range(R Nd, R)
/* If need more examples than available, repeat them reps - / Nd]
/* ceiling division */
Eqo < repeal (Eqo, (O:D]
/+ repeat and truncate to
Eqo.sort()
/* ensure increasing order
10
Ec[s] + Eqo -
input indices are 8 steps before outputs */
11
Eqls] Eqo
/* store output indices
12
end
/* Generate rollout strategy
for i from R to T-1 do dist - i- R + min(dist, M)
14
/+ distance from last reference frame
15
/ select appropriate stride / question input index question output index
16
Qi i-i
17
Qo i
18
S.pppend( Eq(s)), Qi, Qo)
19 end
20
return S
Table Single-step Rollout Strategy Example
Rollout index
Examples (COND, QOI)
Question COND
Predict QOI
(0,1) (1,2) (233) (3,4) (4,5) (5.6) (6,7) (7,8) (8.9)
9
10
2
(0,1) (1,2) (2.3) (3,4) (4,5) (5.6) (6,7) (7,8) (8,9)
10
11
3
(0,1) (1,2) (2.3) (3,4) (4.5) (5,6) (6,7) (7.8) (8.9)
11
12
(0,1) (2,3) (3,4) (4,5) (5.6) (6,7) (7,8) (8,9)
12
13
(0,1) (2.3) (3,4) (4,5) (5.6) (6,7) (7,8) (8,9)
13
(0,1) (1,2) (233) (3,4) (4,5) (5.6) (6,7) (7,8) (8,9)
14
(0,1) (1,2) (2.3) (3,4) (4.5) (5.6) (6,7) (7,8) (8.9)
15
16
(0,1) (1,2) (2.3) (3,4) (4.5) (5.6) (6,7) (7,8) (8.9)
16
17
9
(0,1) (1,2) (2.3) (3,4) (4.5) (5.6) (6,7) (7,8) (8.9)
17
18
10
(0,1) (1,2) (2.3) (3.4) (4.5) (5.6) (6,7) (7.8) (8.9)
18
19
11
(0,1) (1,2) (2.3) (3,4) (4,5) (5. (6,7) (7,8) (8.9)
19
20
As shown in Table 7 the flexible-step strategy initially uses smaller strides to build sufficient examples, then employs maximum strides for later rollouts, detailed in Scction 4.5] We note that repeated in-context examples appear in Table which is common when the maximum stride large and the initial frames cannot form enough examples. While the number of in-context. examples in VICON is flexible and our model
20

---

Published in Transactions on Machine Learning Research (02/2026)
Table 7: Flexible-step Rollout Strategy Example (5max = 5)
Rollout index
Examples (COND, QO1)
Question COND
Predict QOI
(0.1) (1,2) (2,3) (3,4) (4,5) (5,6) (6,7) (7,8) (8,9)
9
10
2
(0,2) (0.2) (1,3) (2,4) (3,5) (4,6) (5,7) (6,8) (7.9)
9
11
3
(0,3) (0,3) (1,4) (1,4) (2,5) (3.6) (4,7) (5.8) (6.9)
9
12
4
(0,4) (0,4) (1,5) (1,5) (2,6) (2.6) (3,7) (4,8) (5.9)
9
13
(0,5) (0,5) (1,6) (1,6) (2.7) (2.7) (3.8) 338 (4.9)
9
14
(0,5) (0,5) (1,6) (1,6) (2,7) (2.7) (3,8) (88 (4.9)
10
(0,5) (0,5) (1.6) (1,6) (2,7) (2.7) (8,8) (3,8) (4.
11
16
(0,5) (0,5) (1,6) (1,6) (2,7) (2.7) (3,8) (3,8) (4,
12
17
9
(0,5) (0,5) (1.6) (1,6) (2,7) (2.7) (3,8) (3,8) (4.9
13
18
10
(0,5) (0,5) (1,6) (1,6) (2,7) (2,7) (3,8) (3.8) (4.9)
14
19
11
(0,5) (0,5) (1,6) (1,6) (2,7) (2,7) (3,8) (3,8) (4,9)
15
20
Algorithm 3: AviilaaaallsPairs
Input: D: Number of demonstrations needed
dt: Desired time stride between pairs
Fa: List of indices of available frames
Output: P: List of input-output index pairs with the specified stride
Fa.sort()
P [+]]
/* Initialize empty pairs list */
/* Find all pairs with stride dt for from 0 len(Fa) do for j from i + 1 to len(Fa) do if Falj] Fafi] == then (Fali, Fali)))
end
end
end
if P emply then
return
end
/* If we don't have enough unique pairs, repeat them
13 if len(P) < then
14
reps + [D len(P)]
/* ceiling division */ /* repeat pairs and truncate to
15
RP < (P, [0:D]
16
P RP
17 end
18 return P
accommodate fewer examples than the designed length, our preliminary experiments indicate that the model performs better with more in-context examples, even when some examples are repeated.
C.2 Strategy with Imperfect Temporal Sequence
When facing imperfect temporal sampling where certain frames are missing, we can still form valid demonstra- tion pairs from the available frames. Algorithm presents this adaptive pair selection process, which serves fundamental building block for all strategy generation algorithms in scenarios with irregular temporal data.
21

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 4: CeneratSingllstqgttateyyyy WrthDrops
Input: D: Number of demonstrations to use
S: Fixed stride between pairs
T: Total steps in sequence
Fa: List of indices of available frames
Output: St: Strategy list of (E, Qi, Qo) tuples
St []]
Fa.sort()
Fs + Fa[-1]
/* the starting frame for rollout */
P & Ged S, Fa)
/* pairs with fixed stride */
if P is empty then
return
/* No available pairs with the specified stride */
end
Fe Fa.copy()
/* accumulated frames (current + predicted) */
for i from (F's 1) to (T 1) do
10
Ci -
/* potential condition index */
11
if Ci & Fe then
12
continue
/* Cannot predict frame i with stride */
13
end
14
Pdt P(s]
/* The example pairs for predicting frame */
Qi i-8
/* question input index
16
Qo +
/* question output index */
17
(ap,, Qi, Qo) Fc.append(Qo)
18
/* The predicted frame can be used for future steps */
19 end
20 return St
Algorithm for single-step strategy with drops. Algorithm f extends the single-step strategy to handle scenarios where frames are missing from the input sequence, adaptively forming strategies from available frames while maintaining the fixed stride constraint.
Algorithm for flexible-step strategy. Algorithm presents our solution for generating flexible-step rollout strategies when frames are missing, dynamically selecting appropriate strides based available frames and previous predictions.
Rollout Strategy Example with Missing Frames. Tables and illustrate our adaptive strategies when frames 2, 5, and 9 are missing from the initial sequence (with all other settings identical to those in Appendix C.I.. The tables demonstrate single-step and flexible-step (max stride: 3) approaches, respectively, highlighting how our algorithm dynamically selects appropriate example pairs to maintain prediction capability despite missing temporal data.
D Additional Experimental Results.
D.1 Ablation Studies
Impact of Patch Resolutions. We conducted ablation studics on patch resolution by varying patch sizes (4, 8, 16, 32, 64) to find a balance between spatial granularity and computational resource constraint. While smaller patches theoretically capture finer details, they generate longer token sequences, hitting memory caps due transformer's quadratic complexity. For patch size 8, we reducod token dimensions (512-2256) and feedforward dimensions (1024-512)). For patch size 4, we applied further reductions (token dim 128, feedforwant dim 256, layers 10-5) to fit within 80GB A100 memory- yet training still required 188 hours (vs
22

---

Published in Transactions on Machine Learning Research (02/2026)
3.0
10
potch mosnlation
monletion
poschructttion
relative error
relative error
-2-
relative error
20-
1.5-
0.5
26
30
40
time
time
time
(a) PDEBench- PDEBBencl-Comp-Lowww Comp-Low Vis
(b) CDEBBend-CCop--le
(c) PDEArena-Income
1.59
1.25
relative error
rolative error
6%.30"
posch mccding porch onod
relative
-2
escodiag
0.75
3K.16"
dite mooding poach eocating fune oncoding only
jucch
25.00"
10"1
0.25
patch crauding only
fune encoding only
20
time
time
time
(d) PDEBench- PDEBench-Comp Low Vis
(e) PDEBench-Come HighVis
(f) PDEArena-Income
0.90
+
10 exampe paia
cuumpl pans
6.8
exsmpp
relative error
0.35
ouumpi pairs
relative relative error
euumpl: pains
relative error
0.6
0.30
0.25
6.4
0.1
ctamplc poins ecmple pars
0.20
comple pain
0.15
time
time
time
(g) CeeBBench-CCmp-LLww Vis
(h) CrctBench-Comp-Hiiee
(i) PDEArena-Incomp
Figure 4: Ablation studies across the three datasets. Top row (a-c): Impact of patch resolutions (4, 8, 16, 32, 64) showing optimal performance at patch size 16. Middle row (d-f): Effect of different positional encoding combinations. Bottom row (g-i): Performance variation with different context lengths
(6, 10 pairs).
23

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 5:
Input: D: Number of demonstrations to use
M: Maximum stride between pairs for rollout
T: Total steps in the sequence
Fa: List of indices of available frames
Output: S: Strategy list of (E, Qi, Qo) tuples, where the list of example pair indices (Ei, Eo) and Qi, Qo the question and output indices
S [-]] Fa.sort() Fs + Fa[-1]
/* the starting frame for rollout */
P +
/* dictionary of available pairs */
for dt from to M do
Pdt GetdvvillablePairs(D.,, dt, Fa) if Pdl is not emply then
/* pairs with stride dt in Fa */
P[dt] + Pdt end end
Ms <- max maxx(P.koys)))
/* maximum available stride */
12 Fc t Fa.copy()
/* accumulated frames (current + predicted) */
13 for from (Fs + 1) to (T 1) do
14
dt - 1 Fys
15
Mt t min(dt, M, Ms)
maximum stride for current step */
16
found False
for in P.keps()soortdd/revvrss--- Truee) where S Mt do
18
Ci +- i-s
/* potential condition index */
19
if Ci E Fc then
/* Found a starting index to obtain frame
20
found + True
21
break
22
end
23
end
24
if not found then
/* We cannot predict frame i
25
continue
26
end
/* The example pairs predicting frame
*/
27
Pdt P[s]
Qi i-s
question input index */ /* question output index */
29
Qo i
30
(Pdd, Qi, Qo)
31
Fc.append(Qo) (Qo)
/* The Qo frame is obtained and can be used for future rollouts */
32 end
33
return S
58 hours for ps=16) and inference slowed to 9.55 sec/step (vs 8.7 ms for ps=16). Patches below 4 x 4 remained infeasible even with these aggressive reductions. Figure Ta-c) shows that patch size 16 achieves optimal performance across all datasets balancing sequence length and representational capacity. Performance degradation with coarser patches (32, 64) aligns with expectations, while degradation with finer patches stems from the necessary reduction hidden dimensions, highlighting a fundamental challenge in scaling 3D applications.
24

---

Published in Transactions on Machine Learning Research (02/2026)
Table 8: Single-step Rollout Strategy Example with Missing Frames
Rollout index
Examples (COND, QO1)
Question COND
Predict QO1
(0,1) (0,1) (0,1) (3,4) (3,4) (6,7) (6,7) (7,8) (7,8)
8
(0,1) (0,1) (0.1) (3,4) (3,4) (6,7) (6,7) (7,8) (7,8)
9
10
(0,1) (0,1) (0,1) (3,4) (3,4) (6.7) (6,7) (7,8) (7,8)
10
11
4
(0,1) (0,1) (0,1) (3,4) (3.4) (6.7) (6,7) (7,8)
(7,8 (7,8)
11
12
5
(0,1) (0.1) (0.1) (3,4) (3,4) (6.7) (7,8) (7.8)
12
13
(0,1) (0,1) (0,1) (3,4) (3,4 (6.7) (7,8) (7.8)
13
14
(0,1) (0,1) (3,4) (3,4) (6.7) (6,7) (7,8) (7,8)
14
(0,1) (3,4) (3,4) (6,7) (6,7) (7,8) (7,8)
15
(0.1) (0.1) (3,4) (3,4) (6.7) (6,7) (7,8) (7,8)
16
17
10
(0,1) (0,1) (0.1) (3,4) (3,4) (6.7) (6,7) (7,8) (7,8)
17
18
11
(0,1) (0,1) (0.1) (3,4) (3,4) (6,7) (6,7) (7,8) (7,8)
18
19
Table 9: Flexible-step Rollout Strategy Example with Missing Frames (max stride: 3)
Rollout index
Examples (COND, QOI)
Question COND
Predict QOI
(0,1) (0,1) (0,1) (3,4) (3,4) (6,7) (6,7) (7,8) (7,8)
8
(1,3) (1,3) (1,3) (4,6) (4,6) (4.6) (6,8) (6,8) (6,8)
8
10
(0,3) (0,3) (0.3) (1,4) (1,4) (3.6) (3,6) (4,7) (4.7)
8
11
4
(0,3) (0,3) (0.3) (1,4) (1,4) (3.6) (3,6) (4,7) (4,7)
9
12
5
(0,3) (0,3) (0.3) (1,4) (1,4) (3.6) (8,6) (4.7) (4.7)
10
13
(0,3) (0,3) (0.3) (1,4) (1,4) (8.6 (3.6) (4,7) (4.)) (4,7)
11
14
(0,3) (0.3) (0.3) (1,4) (1,4) (8.6) (8.6) (4.7) (4,7)
12
15
(0,3) (0.3) (0.3) (1,4) (1,4) (3.6) (3,6) (4,7) (4.7)
13
(0,3) (0.3) (0.3) (1,4) (1,4) (3.6) (3,6) (4,7) (4,7)
14
17
10
(0,3) (0.3) (0.3) (1,4) (1,4) (3.6) (3,6) (4,7) (4.7)
15
18
11
(0,3) (0.3) (0.3) (1,4) (1,4) (3.6) (3.6) (4,7) (4.7)
16
19
25

---

Published in Transactions on Machine Learning Research (02/2026)
Positional Encodings. We evaluated different combinations of positional encodings as shown in Figure 1 d-
(). Our architecture employs two types of encodings: patch position encodings (for spatial relationships between patches) and function encodings (differentiating input/output in different pairs) (Section 4.2)
The results demonstrate that including both encoding types consistently yields the best performance across all datasets. Notably, function encodings have more significant impact than patch encodings, highlighting the importance of distinguishing between the condition and qoi beyond what causal masking (equation provides.
The impact of patch encodings varies across datasets: they show minimal effect on the PDEBench datasets PPDEBench-ComppLooW/ and PDEBench-Comp- Comp-lligVii)i) yet significantly improve performance on PDEArena-Innomp. We attribute this to the different stiffness of these systems. In compressible flows (PDEBench), spatial interactions naturally decay with distance, creating consistent and monotonic spatial correlations between patches. In contrast, Navicr-Stokes equations exhibit infinite stiffness where correla- tions between any spatial locations are instantaneous and determined by the global velocity field, creating non-monotonic and desendepe relationships. Explicitly encoding spatial positions therefore becomes particularly beneficial for PDEArena-Incom Incomp, as it reduces the learning complexity for these non-local dependencies.
Varying Context Length. We investigated how the number of in-context examples affects model perfor- mance, as shown in Figure g-).. Following insights from the original ICON work Yang et al. (2023a), we evaluated context lengths of 6, 8, and 10 pairs, balancing performance against computational efficiency. The results show that 10 in-context examples deliver significantly better results for CrrBench-Comp-LowW// Vis com- pared to shorter contexts, while for the remaining datasets, 10 and 8 in-context examples perform similarly (both oulperforming 6 examples).
Alternative Architecture: CNN-ICON. We explored an alternative architecture (CNN-ICON) that uses a CNN encoder lo compress entire frames into single latent vectors, followed by the ICON translormer structure. The encoder consists of a 4-level CNN with residual blocks (7x x2x1 128 -> 512-dim latent via global average pooling), paired with a symmetric CNN decoder. This results in token per frame compared to VICON's 64 tokens (8x8 patches). We matched the model sizes: 86.5M parameters (vs VICON's 87 87.8M), identical transformer configuration (6 layers, 8 heads, dim=512), and same training protocol (200K steps).
Table 10: CNN-ICON vs VICON comparison (normalized RMSE, scaled by std)
Dataset
Model
Step
Step 5
Step 10
Last
Avg
CNN-ICON
0.323
0.467
0.643
1.194
0.912
PDEArena-Incomp
VICON
0.110
0.206
0.305
0.680
0.485
CNN-ICON
0.750
0.875
1.012
1.013
0.905
PDEIBench-Comp-LowWW
VICON 0.154
0.245
0.375
0.391
0.271
CNN-ICON
0.390
0.767
1.939
2.364
1.162
Comp-
VICON
0.052
0.163
0.498
0.614
0.301
As shown in Table TO VICON outperforms CNN-ICON by 1.9-3.9x across all datasets. We attribute this performance gap to information loss when compressing entire frames into single vectors, which discards spatial structure crucial for accurate PDE prediction. This finding motivated our patch- based design, which preserves local spatial information while maintaining computational efficiency.
D.2 Analysis of Checkerboard Artifacts in ODEBench-Comp-HHe
In our experiments, we observed block-like artifacts in the error/difference maps for the Pigg dataset that align with the model's 16x16 patch structure. Through detailed analysis, we identified two contributing factors: 1) drastic field value variation across the trajectory time horizon, and 2) gaaning--inferene normalization mismatch.
26

---

Published in Transactions on Machine Learning Research (02/2026)
Drastic field value variation across trajectory time horizon. High-viscosity flows rapidly dissipate kinetic energy, causing velocity fields to become nearly stationary after approximately 10 frames. Our analysis of standard deviations across different time ranges reveals these differences:
Table 11: Standard deviation analysis across time phases for PDEBench datasets. The bolded veloc- ity channels show CDEEBench-Comp--liglWWV has extreme variation between early and late phases, unlike PDEBench- Saaruch-oom--LoWW Vis with nearly consistent variation. This dynamic difference makes training a uni- formed model particularly challenging.
Dataset
Velocity
Velocity v
Pressure All
All
1-10
>10
All
1-10
>10
PDEBend-CCompp-igghVii
0.359
0.511
0.091
0.351
0.506
0.049
14.254
PDF-Benh--Comp-LowWW
0.989
1.104
0.873
1.036
1.165
0.903
29.687
Training-inference normalization mismatch. During training, context pairs are sampled uniformly across all frames. During inference, context pairs are sampled from initial frames only. As shown in Table []] the standard deviations at these two different time ranges vastly different, which results in normalization mismatch and exacerbates prediction errors in the already--hhallengnng problem. Future work could explore using global statistics from multiple trajectories instead of mngle-trajjctory instance normalization during deployment. Other potential remodies include using overlapping patches or moving patch positions during rollout, though these represent engineering improvements orthogonal to our core contributions.
D.3 Cross- Trajectryy Context Sampling
We investigate whether VICON can leverage context pairs from different trajectories governed by the same underlying operator, as suggested in the original ICON framework. Using the PDEArena-Incomp dataset (which provides 32 distinct trajectories per operator), we compare VICON trained with intra-trajectory context only versus mixture of intra- and cross-trajectory context (50/50 split).
Table 12: trajectrayy context experiment on PDEArena-Incom (normalized RMSE, scaled by std)
Training Setup
Inference Setup
Avg Error
VICON (trained with intra-traj only)
intra-traj context
0.563
cross-traj context
0.609
VICON (trained with mixed context)
untra-traj context
0.556
cross-traj context
0.568
As shown in Table 12 VICON is capable of leveraging cross-trajoctory information when trained appropriately. The model trained with mixed context achieves competitive performance in both settings, with only a marginal gap (0.556 0.568) between intra- and cross-traeector inference.
While these results demonstrate VICON's flexibility, maintain that intra-trajectory context is more practical for deployment scenarios. Cross-trajecton sampling requires: (1) a pre-existing database of trajectories, and (2) a retrieval mechanism to ensure sampled trajectories share the same (often unknown) operator. contrast, our intra-trajed tataa-rrajectoy approach is self-contained and better suited for real-time forecasting where only the current trajectory available.
D.4 More Results and Visualizations
Detailed Results. Tables 13 and 14 summarize the relative and absolute 2 rollout errors across different.
timesteps for all evaluated models the three datasets.
27

---

Published in Transactions on Machine Learning Research (02/2026)
Table 13: (Comparison) Summary of Rollout Relative L Error (scale by std) for different methods across various cases. The best results are highlighted in bold.
Rollout: Relative 1 Error
Case
Ours (single step)
Ours (flexible step)
DPOT
MPP
EetArena-Inoome
11.01
11.01
5.97
6.17
Step [1c-2]
PDEBeuch- CeBBeeh-CCoup--e Comp-Loow Low Vis
15.40
15.40
25.24
16.37
CeeBBech-Comp-llle
5.18
5.18
42.76
20,90
PDEArena-Income Incomp
22.68
20.62
11.70
14.81
Step 5 [1e-2]
PDEBench- Comp- Low Vis
24.45
24.45
36.03
45.45
PDEBench Chael-CoomC-iiii HighVis
16.25
16.25
286.3
42.93
PDEArena- Trnomp
35.67
30.53
20.74
27.89
Step 10 [1e-2]
PDEBench Comp-Low
37.54
37.54
49.47
64.11
CerBeecc-CCrup-Tiiii
49.83
49.83
2016
144.46
PDEArcua-Incomp
76.77
68. 03
65.27
93.52
Last step [le-2]
PDEBench-Comm Comp-Low Vis
39.11
39.11
48.92
65.32
PDE Saae--Coooppiiiii
61.41
61.41
2866
185.3
PDEArena- Incomp
56.27
48.50
41.20
55.95
All average [1e22]
PDEBench- Comp- LowVis
27.08
27.08
37.72
46.68
PDERench Comolh-CCmpp-ip Comp- HighVis
30.06
821.9
Table 14: (Comparison) Summary of Rollout Absolute L Error for different methods across various
cases.
The best results are highlighted in bold.
Rollout Z Error
Case
Ours (single step)
Ours (flexible step)
DPOT
MPP
PDEArena -ncoom
5.63
5.63
3.02
3.12
Step
PDEllench-Comp-LLee
21.74
21.74
29.47
24.47
[1o-2]
PDEBench- PDEBench-Comp-llee
1.43
1.43
6.22
4.73
PDEAreua-
10.38
9.46
5.37
6.79
Step
PDEBeuch-
31.23
31.23
43.57
57.50
[1o-2]
PDEBBench-Compp-igyh
2.34
2.34
14.48
6.52
TEEAArea-Incomn
12.55
8.55
11.49
Step 10 [1e-2]
CetBeenchCCop-Looo
45.39
45.39
59.43
78.54
PDEBBench-Comp-Higgl
3.21
3.21
25.79
8.98
IDEArena-Inoome
16.50
14.56
14.03
19.47
Last Step [le-2]
DDDDeenhh-Comp-Lee
48.69
48.69
63.06
85.60
PDEBench Canheenhh-Co-----ee Comp-HighVis
3.39
3.39
9.56
PDEArena- Incomp
16.26
14.31
11.86
15.77
All average [1o2]]
Ceetlench-Comp-Lotte PDEBench-Comme HighVis
34.44
34.44
46.60
60.68
2.48
2.48
16.86
7.04
Generalization to different timestep strides. Figure iillstrates VICON's performance with varying timestep strides (Smax 1,3, 5, 7) across all three datasets, demonstrating the model's ability to adapt to different temporal resolutions without retraining.
Turbulence Kinetic Energy (TKE) Predictions. Figure displays the visualization of TKE fields for ground truth and model predictions, highlighting VICON's superior ability to preserve critical turbulent flow structures compared to baseline approaches.
Comparison between joint training vs separate training. Figure 7 and Table 15 quantify the performance differences between jointly trained and separately trained models, demonstrating the significant. advantages of multi-physics training across all datasets.
Computational Efficiency. Table 16 summarizes the computational resources required by cach method, showing VICON's advantages in terms of training cost, inference speed, and model parameter count compared to both DPOT and MPP baselines.
28

---

Published in Transactions on Machine Learning Research (02/2026)
ringle dtp
cingle dep step max step?
singlsstry stop
10
06
relative error
step step
relative error
relative error
0:5
9.6
0.4
9.4
10"
02-
10
20
time
time
time
(a) PDEBBenc- P Vis
(b) CerBBench-CompHiiee
(c) PDEArena-Inoom
Figure 5: Comparison of rollout errors across different datasets, using single-step and flexible-step strategies
with varying maximum step sizes msmax 1,3, 5, 7).
VICON
DROT
1.0
1.0
1.0
1.0
0.8
0.5
944
0.4
02
0.2
0.2
02
GT VICOO
G7. DPOT
GT MPP
6.1
0.1
0.1
0.0
-02
-02
-02
Figure 6: Comparison of turbulence kinetic energy predictions. (Top-left) Ground truth TKE field, (Top-rightt) model predictions, (Bottom) model error.
Rollout with imperfect measurements. Figure show the rollout results with imperfect temporal
measurements.
Visualizations.
We compare the output different models in Figure g Figure 10 and Figure 11 Figures
12 and present additional visualizations of the VICON model outputs compared to ground truth and baseline predictions, highlighting the qualitative advantages of our approach.
29

---

Published in Transactions on Machine Learning Research (02/2026)
Joise Soparto
Josr Separoo
Soun
Sqparatz
relative error
0.5-
relative error
1.25
relative error
1.3
0.4-
1,00
0.75
0.5
0.3
0.4
422
time
time
time
(a) PDEBench-Comp- Comp-Low Vis
(b) PDEBench-Comp -ogh--ighVVis
(c) PDEArena IDEAArena-nccomp
Figure 7: Comparing rollout errors (single step, scale by std) for joint versus separate training strategies. For separate training (individual models for each dataset), we maintain the same batch sizes as in joint training while adjusting training steps be slightly more than one-third of the joint training duration, ensuring comparable total computational costs across both approaches.
0.8
undom nodom dmap 2rop3 bolf lnequecc
6.8
audom Coop
relative error
relative efror
unlom émp
relative errrr
imp à
0.6
0.6
bolf fuqueny
0.5
04-
0.4
randar drpp
pandam drop
02-
randoti dop
half froguccc
0.0
6.0
0.0
10
time
time
(a) VICON
(b) DPOT
(c) MPP
10
clese
dean
undom drap 1
25
nadumm drep
mandom deop
relative etror
random dopp
relative error
relative errOr
dmpp
nedum
meddlm deop
1.5
budf Gequency
half Sequancy
half frequeayy
15
1.0
10
10
10
time
time
time
(d) VICON
(e) DPOT
(f) MPP
modomm Goop
relative etror
relative etror
dnip
relative error
nudom doop
0.6
0.6
talf fnqquency
0.5
undom drap
sandar drop
nolom dmp
maddr crup
82
dipp
6.2
0.3
dop 3
belf inquencc
half fnuuurasy
20
10
40
10
20
40
20
50
40
time
time
time
(g) VICON
(h) DPOT
(i) MPP
Figure 8: Comparison of rollout errors (scale by std) with different input data noise levels across all datasets. Each column (left to right) shows VICON, DPOT, and MPP models, while each row (top bottom) represents PDEBench- ComB-nch-Comm-LooWiis
HDEBeech-Coomp-Highe For MPP and DPOT which require fixed di and context window, interpolation is used to generate missing frames, while VICON can directly handle irregular temporal data without interpolation.
and PDEArena-Income datasets.
30

---

Published in Transactions on Machine Learning Research (02/2026)
Step Shop
Stap
0.05
Target
0.00
0.15
MPP-5 1O 1777
-0.010
-0 00.15
0.035
DPOT-M (0. .6)
0.006
8.000
-0 00000
VICON 10.06) (6
8.000
0.0000
-0 0.0025
0030
Figure Comparing outputs from different models. The target the first output steps from DeDBBennh-CCmmp-lggiWW dataset (x-velocity channel). For each model (each row), we display the difference between target and model output. Errors (rescaled by std) for the full trajectory listed after the model names.
31

---

Published in Transactions on Machine Learning Research (02/2026)
Step
Step 5
45
Target
3.0
15
MPP-8 (0.33)
0.5
0.0
DPOT-M (0. 51)
-65
1.0
VICON (0.29)
-10
Figure 10: Comparing outputs from different models. The target is the first 6 output steps from BeEBeenhh-CompLLog Vis dataset (pressure channel). For each model (each row), we display the difference between target and model output. Errors (rescaled by std) for the full trajectory listed after the model names.
32

---

Published in Transactions on Machine Learning Research (02/2026)
Sea
Step
Seg
Step
Stpp
Stea
1.75
2.50
Target
0.75
0.50
0.25
0.75
MPP-8 (0.61)
0.35
2.00
DPOT-M 10.42 (2.42
VICON 00.47)
Figure 11: Comparing outputs from different models. The target is the first 6 output steps from PDEArena-Income datasct (particle density channel). For each model (each row), we display the difference between target and model output. Errors (rescaled by std) for the full trajectory are listed after the model names.
33

---

Published in Transactions on Machine Learning Research (02/2026)
Targer
Trrget
Trrget
Target
Torget
48.00
47,950
43.95
47.925
47.92
4..93
67.90
47,900
47.00
47.8 1585
47.85
47.0
47,850
47.06
47.030
47.94
47.05
47.92
47.90
47.90
47.900
47.85
47.80
47.05
47.03
47,075
47.88
47.06
0.02
0.00
0.00
0.00
0.00
0.00
-0.02
0,02
-6.07
0.00
(a) sequence output steps for TDEEBenci-Comp-Highhh dataset. The channel plotted the pressure field equation equation
1.0
5.0
1.0
1.0
05
3.5
0.0
0.0
-05
-10
-1.
-10
-10
3.5
0.0
0.0
0.5
-0.
-10
-1.
1.0
0.30
02
0.0
0.0
00
0.00
30
-0.
-.25
-05
-35
04
(b) sequence of output steps for PDEArena-Incomp dataset. The channel plotted is the x-velocity equation equation g
Figure 12: Example outputs for the VICON model. (a) The pressure field of PDEBench- PDEBench-Comp-HHige Comp-HighViss dataset, and (b) the x-velocity field of PDEArena Mneorena-Incomp dataset. Each col- umn represents a different timestep.
34

---

Published in Transactions on Machine Learning Research (02/2020)
Poficee
Poof 110
L5
05
Yewree
OPfeece
OPhennce
-23
04
-04
8.0
(a) The channel plotted is the y-velocity field in equation equation
bye
2.00
Tape
0.75
139
6J5
32
04
(b) The channel plotted is the density field in equation equation
Bree
Boye:
Tore
Texet
Trre
2a
15
E5
15
-05
(c) The channel plotted is the pressure field equation equation
Figure 13: More example outputs for the VICON model. Showing output steps for the PDEBench- Coweanchh-ooooowwohhe dataset as governed by equation equation (c) pressure fields. Each column represents different timestep.
(a) y-velocity, (b) density, and
35

---

Published in Transactions on Machine Learning Research (02/2026)
Table 15: Summary Rollout Relative L Error Metrics (single step, scale by std) for Training Strategies. The best results are highlighted in bold. For separate training (a model on each dataset), each run's batch size is controlled to be the same as joint training. To ensure fair comparison, we maintain the same batch sizes as joint training for each separate training while adjusting their training steps to be slightly more than one-third of the joint training duration, ensuring comparable total computational costs.
Rollout Relative L Error [l-2
Case
Joint
Separate
PDEArena- Incomp
11.10
9.66
Step
PDE.ench- Vis
15.61
19.68
FDEBench-Comee High
5.79
9.74
PDEArena-Incomm
23.00
24.09
Step 5
PDEBench- PDEBeuch-Comp-Low Vis
24.56
37.63
19.73
39.81
PDEArena-Incomp
36.18
41.66
Step 10
PDEBench- PDEBench-ComppLoow Vis
37.47
57.17
DDE.BechhCCoppHHHe
57.88
131.6
PDEArena-Incom
77.81
123.0
Last Step
JJDEBench--Comp-L.wwW/
39.03
59.11
PDEBench Careuchhh-om-llHe Comp-HighVis
71.17
163.5
PDEArena-Incomm
56.26
76.35
All average
ODEBBeccl-Comp-LLo Vis
27.08
40.75
IDE.Bench-Comp-lllee
30.06
65.19
Table 16: (Comparison) Summary of Resource and Timing Metrics for different methods.
Resource and Timing Metrics Ours
DPOT
MPP
Training cost [GPU hrs]
58
70
64
Rollout time per step [ms]
8.7
12.0
25.7
Model Param Size
88M
122M
116M
36


Reviews:
[{"review_id": "77pOqkwA93", "reviewer": "Reviewer_MrNX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces Vision In-Context Operator Networks (VICON) for predicting multi-physics fluid dynamics. VICON employs a ViT to process 2D data in a patch-wise manner rather than as individual data points, which enhances the computational inefficiency of the previous In-Context Operator Network (ICON) in handling dense, high-dimensional data, while preserving ICON's ability to learn from in-context few-shot samples. VICON significantly outperforms state-of-the-art baselines in both accuracy and speed on several fluid dynamics benchmarks. The paper also conducts abundant ablation studies and robustness analysis.", "claims_and_evidence_yn": "Yes", "claims_explanation": "1. The paper provides strong empirical evidence. VICON outperforms state-of-the-art models (DPOT, MPP) across multiple fluid dynamics benchmarks. It achieves lower prediction error, and is also computationally efficient.\n\n2. There are ablation studies on key design choices (e.g., patch resolution and positional encodings).\n\n3. The paper tests VICON against different sampling rates and missing frames, which demonstrates the robustness of the proposed method.", "audience_yn": "Yes", "audience_explanation": "The paper focuses on the task of multi-physics fluid dynamics prediction, a topic that has been widely studied in previous literature. I believe that at least some individuals in TMLR's audience would be interested in this paper.", "requested_changes": "1. In Section 4.3, it is unclear how the normalization is done. I would suggest the authors add more concrete procedural descriptions or formula for the normalization operation.\n\n2. The paper did not test ICON (one baseline method) on the three benchmarks, because the large number of input tokens \"exceeds our available GPU memory and is thus computationally infeasible.\" I think this might be solved by using tensor parallel or data parallel techniques that are usually supported by modern inference engines like Huggingface transformers or vLLM. Incorporating this result can enhance the soundness of the paper.\n\n3. Table 2 seems a bit confusing to me. Does it only show the result for non-in-context ViT baselines? As a comparative study, the performance of VICON (the proposed method) is also expected to be included.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fqb3egNv2m", "reviewer": "Reviewer_yK5h", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "## Summary\n\nThis paper presents VICON, which extends ICON by introducing the vision transformer architecture into the solving process, bringing better performance and efficiency in processing 2D data. To capture the correlations between query and condition, the authors adopt two types of learnable positional encoding: patch and function. Additionally, the authors explore more diverse inference scenarios, including varying time step strides and missing frames. Experimentally, VICON outperforms MPP and DOPT.\n\n## Strengths\n\n-\tEnhancing ICON by introducing 2D patches is reasonable, which not only enables 2D PDE solving but also boosts the model capacity.\n-\tThe proposed VICON delivers a favorable performance in long-term prediction and has better efficiency.\n-\tThis paper is well motivated and well-written, along with sufficient implementation details.\n\n## Weaknesses\n\nI appreciate the contribution of this paper, but I also have some concerns.\n\n(1)\tInsufficient baselines.\n\nRecently, some advanced baselines have been developed to perform a similar task to MPP and DPOT, such as Unisolver [1]. I think this is a highly relevant baseline, which also adopts a vision transformer for 2D fluid modeling and demonstrates better performance than DPOT.\n\nAdditionally, MPP and DPOT are old baselines; it is inappropriate to name them as “state-of-the-art”.\n\n[1] Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers, ICML 2025\n\n(2) Question about the unique advantage of VICON.\n\nSince VICON follows a similar training and inference framework as ICON, I am wondering whether the inference capability under varying time step strides and missing frames is a unique advantage of VICON or if ICON can also make the same capability.\n\n(3) Since the authors propose to use a vision transformer as VICON's model architecture, it is hard to handle simulations with irregular geometries. I think it is acceptable in this paper, but they should discuss this issue as a limitation or future work.", "claims_and_evidence_yn": "No", "claims_explanation": "Most of the claims are correct and well-supported. However, I do not think MPP and DPOT are \"state-of-the-art\" baselines.", "audience_yn": "Yes", "audience_explanation": "I think ICON is an interesting paper. It is nice to see its 2D extension.", "requested_changes": "I would like to suggest that the authors (1) provide more baselines for comparison, such as Unisolver [1]; (2) add more discussion on the VICON's limitation in irregular geometries.\n\n[1] Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers, ICML 2025", "broader_impact_concerns": "This is a technical paper; no ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "aAsZc9QWUX", "reviewer": "Reviewer_KURB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper extends the previously proposed ICON framework, which uses in-context learning to infer PDE operators from example pairs of input/output functions generated by the same operator. ICON showed that a transformer-based architecture could benefit from exposure to multiple 1D PDEs and extrapolate to unseen operators close to the training distribution.\n\nVICON builds upon ICON by introducing Vision Transformer (ViT) encoders that process spatial patches rather than pointwise 1D inputs. This modification enables the handling of higher-dimensional (2D) fields while avoiding memory limitations associated with pointwise processing. The paper reports improved performance on three fluid-dynamics datasets.\n\nHowever, the new model introduces trade-offs: (1) it is not backward compatible with ICON, since 1D PDEs cannot be processed using VICON, and (2) the scope of applicability is narrowed to 2D autonomous PDEs only.", "claims_and_evidence_yn": "No", "claims_explanation": "Overall, the evidence provided in the experimental section is **not sufficient to convincingly support the claimed benefits** of VICON.\n\n**Methodological novelty and validation**\n\nThe main architectural change — replacing ICON’s 1D encoder with a patch-based ViT — is technically incremental. It addresses a pragmatic limitation but does not introduce a new conceptual insight. Moreover, the introduction of ViT encoders makes VICON not backward compatible: the model is not an extension of ICON to 2D PDEs, but rather a derivation, since it is constrained to 2D problems.\n\nThe experiments, limited to three very similar fluid-dynamics datasets, provide only narrow evidence of the model’s capabilities. Unlike ICON, VICON’s evaluation does not include zero-shot generalization to unseen operators, which weakens the claimed extension of the “in-context operator learning” concept.\n\n**Realism of the setup**  \n\nAll datasets use 2D Navier–Stokes systems on uniform grids, which are not representative of modern fluid simulation practices that use adaptive or irregular meshes. Moreover, the patch-based design scales poorly toward 3D problems.\n\n**Limitation to autonomous problem**\n\nICON uses demonstrations $(c^i,  u^i)$ generated by the same operator, but from different trajectories, where $c^i = [c^i_1, ..., c ^i_T]$ is the input signal, and $u^i=[u^i_1, ..., u^i_T]$ is the output signal (to be predicted). The input signal $c^i$ can be either an external signal or the initial condition, in the case of an autonomous PDE.\n\nIn contrast, VICON is designed for autonomous 2D PDE only. The input/output context pairs provided to the model are pairs of consecutive observations $(u_t, u_{t+\\delta })$, which is a special case of ICON framework. The paper uses the $(c, u)$ notation throughout, but actually only considers the case $(u_t, u_{t+\\delta})$. This is explicitly stated in the experiment section, but also implicit in the model design itself: the model uses the same projection function $f_\\phi$​ for both $c$ and $u$; which is valid if $c$ and $u$ are of identical nature.\n\n**Interpretation of the learning setting**\n\nUncertainty remains about how the context pairs are sampled. It is unclear whether VICON’s demonstrations $(u_t​,u_{t+\\delta})$ are taken from:\n\n- multiple trajectories governed by the same operator (as in ICON), or\n- consecutive steps of a single trajectory (which would make VICON essentially autoregressive).\n\n*If context pairs are taken from different trajectories*, the model’s use of a causal mask per block and its autoregressive scheme are conceptually incoherent: causality among independent demonstrations is meaningless. In that case, a mask filled with $1$ except on the block diagonal would make more sense: each context pair could benefit from the other ones to make a prediction. The auto-regressive part is confusing as well: the model is initialized with $I_0=10$ context frames coming from different trajectories, which are progressively replaced with the latest prediction of the model, but then in which order ?\n\n*If they come from the same trajectory*, then VICON departs substantially from the ICON principle and reduces to a history-based forecaster requiring an external simulator to generate the first frames. In that case, stronger autoregressive baselines (e.g. _Learned Coarse Models for Efficient Turbulence Simulation_ (Stachenfeld et al.) or _Learning Mesh-Based Simulation with Graph Networks_ (Pfaff et al.)) should be compared.\n\n**Baselines and experimental thoroughness**  \n\nThe baselines are limited. Including a CNN encoder inside ICON would clarify whether the benefit arises from ViT architecture specifically. The definitions of the “vanilla ViT” and “incorrect context pairs” baselines are also unclear. If the “vanilla ViT” omits all context, its underperformance is trivial; if “incorrect context pairs” come from the same operator but with random perturbations, the experiment does not test operator identification.\n\nIn sum, the evidence does not yet convincingly support the claimed generalization of ICON to higher dimensions or to broader operator learning. Some properties from ICON are untested (generalization to unseen PDE, handling of non-autonomous operator, usage of context from different trajectories).", "audience_yn": "Yes", "audience_explanation": "The work could interest researchers exploring machine learning methods for PDE modeling and data-driven simulators, especially those using transformer-based architectures in physical domains. Practitioners working on operator learning for fluids might appreciate the engineering extension to 2D fields. However, because of the limited scope (2D uniform grids, autonomous PDEs, similar datasets), the current contribution is unlikely to attract broad attention beyond those specifically following the ICON line of work.", "requested_changes": "To address the above shortcomings and make the paper more convincing, the authors should:\n1. **Clarify the setup**\n    - Explicitly state upfront that the method handles autonomous PDEs only and clarify the difference with ICON.\n    - Clearly describe how context pairs are generated and ordered (same trajectory vs multiple trajectories).\n    - Explain the rationale behind the causal masking and autoregressive replacement scheme.\n2. **Methodological clarity**\n    - Justify using the same projection function $f_\\phi$ for both $c$ and $u$.\n3. **Baselines and experiments**\n    - Add or discuss baselines that reflect the actual setting: CNN-based ICON, and strong autoregressive simulators (e.g., Stachenfeld et al. or Pfaff et al.).\n    - Define “vanilla ViT” and “incorrect context pairs” precisely. Consider testing with context from _different operators_ to evaluate true operator identification.\n    - Extend the evaluation beyond fluid dynamics if feasible, or qualify the contribution as domain-specific.\n    - If context comes from the same trajectory, quantify performance drop when the model is prompted with context from the same operator, but from different trajectories.", "broader_impact_concerns": "No direct negative societal impact is apparent. However, if presented as a general solution for operator learning, the paper may unintentionally overstate its applicability. The experiments and setup correspond to a restricted subset of physical simulations and depend on having partially simulated trajectories. Clarifying these boundaries would prevent misconceptions about the generality or practical readiness of VICON.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)