Title: VICON: Vision In-Context Operator Networks for Multi-Physics Fluid Dynamics
Venue: NeurIPS 2025
Accepted: no
Keywords: AI4Science

Learning PDE

Fluid Dynamics

In-Context Learning
Abstract: In-Context Operator Networks (ICONs) have demonstrated the ability to learn operators across diverse partial differential equations using few-shot, in-context learning. However, existing ICONs process each spatial point as an individual token, severely limiting computational efficiency when handling dense data in higher spatial dimensions. We propose \textit{Vision In-Context Operator Networks} (VICON), which integrates vision transformer architectures to efficiently process 2D data through patch-wise operations while preserving ICON's adaptability to multiphysics systems and varying timesteps. Evaluated across three fluid dynamics benchmarks, VICON significantly outperforms state-of-the-art baselines: DPOT and MPP, reducing the averaged last-step rollout error by 37.9\% compared to DPOT and 44.7\% compared to MPP, while requiring only 72.5\% and 34.8\% of their respective inference times. VICON naturally supports flexible rollout strategies with varying timestep strides, enabling immediate deployment in \textit{imperfect measurement systems} where sampling frequencies may differ or frames might be dropped—common challenges in real-world settings—without requiring retraining or interpolation. In these realistic scenarios, VICON exhibits remarkable robustness, experiencing only 24.41\% relative performance degradation compared to 71.37\%-74.49\% degradation in baseline methods, demonstrating its versatility for depolying in realistic applications.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents VICON, a vision-transformer-based in-context operator network for multi-physics PDE prediction. The motivation is clear and well-justified: addressing the scalability issues of ICON in high-dimensional settings. The contributions are also clearly articulated, including an efficient patch-wise architecture and robust performance across diverse fluid systems. Overall, this is a well-structured and solid piece of work with strong motivation and clear contributions.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n1. The paper successfully extends the ICON framework to high-dimensional spatial domains by leveraging vision transformer architectures, significantly improving scalability while retaining the few-shot, in-context learning capability.\n\n2. The method addresses practical challenges such as variable timestep rollout and imperfect temporal measurements, making it valuable for real-world applications.\n\n3. The overall design and implementation are relatively simple and elegant, making the method easy to adopt and reproduce.\n\nWeaknesses:\n\n1. While flexible rollout is a core feature of VICON, in practice the best performance is often achieved with stride-1 predictions. This raises questions about how well the model truly benefits from variable stride training and whether longer-stride generalization is as effective as claimed. A deeper analysis of this behavior would be helpful.\n\n2. The current study focuses primarily on autoregressive forecasting tasks for time-dependent PDEs, which somewhat narrows the broader applicability of the original ICON framework (e.g., to inverse problems or steady-state tasks). Extending VICON to support more diverse problem types would further enhance its utility.\n\nQUESTIONS:\n1. *I have some concerns about the prediction stride.* While the flexible rollout design is a nice feature, it appears that in most cases, the best performance is still achieved when the stride is one. This is somewhat counter-intuitive: in traditional autoregressive training, a longer stride generally reduces error accumulation and improves performance for the same prediction horizon, since fewer steps are rolled out [1]. However, in Figure 5 of the appendix, “max step 7” seems to perform notably worse than “single step,” especially in the PDEBench datasets. Could the authors clarify why this occurs and whether it suggests a limitation in how VICON generalizes across stride lengths?\n2. *I’m interested in how this framework could generalize beyond forward temporal prediction and 2D spatial domains.* While the authors mention in the limitations that extending to 3D or broader PDE tasks may face computational or domain expertise constraints, it would be helpful to have a more concrete estimate of how computational requirements might scale in 3D. Additionally, what specific forms of “specialized domain expertise” would be needed—for example, are we talking about mesh generation, physical modeling, or numerical solver integration? A more detailed discussion would help evaluate the feasibility of future extensions.\n\n[1] Bi, Kaifeng, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. \"Accurate medium-range global weather forecasting with 3D neural networks.\" Nature 619, no. 7970 (2023): 533-538.\n\nLIMITATIONS:\nYes, the authors thoroughly discussed the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces vision in-context operator networks (VICON), a new operator learning model that is designed to predict the dynamics of complex physical systems, with a specific focus on fluid dynamics. The paper builds upon the in-context operator network (ICON) framework but overcomes the limitation of computational cost in the ICON methodology which treats each spatial point as a token. It achieves this by integrating a vision transformer architecture, which would process the 2D spatial domain through patch-wise operations. The paper highlights strong performance when compared against the baselines of DPOT and MPP across 3 distinct benchmarks. In particular, as a consequence of its formulation, VICON excels in handling long-term predictions and handling of imperfect or irregular data.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The paper is well motivated and effectively highlights computational cost as a shortcoming of existing in-context operator networks (ICONs) as a result of point-wise tokenization. The proposed solution of using a patch-wise encoding is a logical approach to mitigate this cost\n- The flexible time-step choice as a consequence of the in-context formulation is an elegant outcome. Further, the discussion on the robustness using the \"imperfect\" measurement study is an interesting demonstration which isn't often discussed for neural PDE surrogates. The innate ability to generalize across different rollout stride sizes using the same model is a particularly compelling result when compared against baselines.\n- The reported improvements in computational efficiency of using a smaller number of parameters and faster inference when compared against baselines is a practical advantage.\n\nWeaknesses:\n\n- I think the experiments being performed aren't the most complete. While I commend the authors on building a foundation model across multiple fluid benchmarks by demonstrating the improvement obtained using joint training / individual training, I would like to highlight two shortcomings: (i) Firstly all the data processed by the model seems to live on a 128 x 128 grid. I would like to see the performance when including a similar fluid mechanics benchmark (perhaps a candidate could be the Rayleigh Benard / Rayleigh Taylor / Shear Flow benchmark from TheWell if possible), (ii) While already mentioned as a shortcoming, would it be possible to include a dataset that isn't NS based (perhaps diffusion-reaction).\n- Further, it seems like the authors have only considered foundation models as a baseline of comparison. It would be helpful to see a comparison of the results obtained against other standard operator learning choices: FNO family and other vision transformer choices (ViTO, CViT). While these are bespoke models, they still provide a good reference of the cost of performance degradation at the cost of multi-physics capabilities.\n\nQUESTIONS:\n- Did you compare the performance of your model against more recent state-of-the-art foundation models like [Poseidon](https://arxiv.org/abs/2405.19101)?\n\n- Have you explored in-context generalization of the architecture to different resolutions? Similar to time-step generalization, would this also prevail in space? In this case, it would involve rethinking the choice of learnable PEs\n\n- Have you tried adding the stride in time as an additional conditioning variable? This way all the COND-QOI pairs may not require the use of the same stride within the same context\n\n- I noticed in the results for the PDEBench-Comp-HighVis dataset, specifically in the error/difference maps shown in Figure 9 and Figure 12a, there appear to be block-like artifacts. This pattern seems to align with the model's 16x16 patch structure. I was curious as to why they're particularly visible in this case\n\n- As I can see the number of tokens in an in-context learning framework would grow with increase in spatial resolution and in-context pairs. I know the authors have acknowledged this shortcoming. However, I was curious if you see a route to mitigate this cost especially when moving to higher resolution / complex problems / 3D?\n\nLIMITATIONS:\nThe authors have been clear about the shortcomings of the model and outlined scope for future work such as 3D models, irregular grid, more general design beyond channel based masking. Maybe one downside of not having an additional spatial query that the model accepts is that the current methodology isn't amenable to a physics informed loss.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce VICON, a vision transformer backbone that is integrated with in-context input-output pairs as part of the input sequence (sequence of patches from a 2D image grid) that can outperform SOTA models (DPOT and MPP) on spatiotemporal PDE emulation tasks. Specifically, they show the performance advantage on harder dynamical systems and systems with missing temporal samples (this mirrors some challenges in real-world), at a smaller training as well as inference cost. This motivates the use of VICON for PDE emulation tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* Vision in-context models for SciML is a novel methodology with strong implications for future SciML models, especially for downstream task adaptation\n* SOTA performance compared to MPP/DPOT with enhanced computational efficiency\n* Methods are clear with good figures illustrating the model\n\nWeaknesses:\n* My main concern is that it is difficult to judge the effect of the in-context learning (the main contribution) from the experiments - some key ablations may be missing.\n* [minor] The contributions are also not appropriately contextualized - there is the main contribution of the in-context vision model and some additional contributions with flexible temporal learning strategies that seems independent of the in-context model - again, some key ablations would help clarify these.\n\nQUESTIONS:\nIn-context learning contributions: I feel some ablations would help put the results in better perspective\n* As I understand, the vision transformer gets input-output pairs (u_t, u_(t+dt)) as part of the input sequence (and is concatenated) and hence the model has additional information about the inference task and can learn relationships purely through the above context (similar to LLMs). It seems like the first ablation is to show that a vision transformer without these pairs can fail in certain ways or have reduced performance. It would be good to see the same last-step rollout errors on in-distribution examples (from testing) and out-of-distribution examples (here, I would expect the vision transformer to fail steadily without prompting from the in-context examples). This strongly motivates VICON or in-fact any other model (MPP + ICON etc).\n* It would be useful to demonstrate that VICON indeed takes advantage of the in-context examples. It is unclear now since the model architecture has also changed with the comparisons to DPOT and MPP which I assume were different architectures(?). A quick ablation is to produce u_(t+dt) from the in-distribution examples (say lower Re) as in-context examples when evaluating on high Re and demonstrate that the model performance reduces as it pays attention to the lower Re (the wrong) examples. You could make it harder by prompting with the identity map (u_t and u_t pairs) and see the performance degradation - these should be purely inference runs so hopefully this is a realistic experiment to run in the revision period.\n* I am surprised at the computational expenses involved and would appreciate a discussion from the authors in the paper. VICON is cheaper to train and evaluate. I would expect that adding to the sequence would only blow up your computational costs (quadratically for the vision transformer). Maybe some quick back of the envelope calculations would be quite useful here to demonstrate why VICON is cheaper than DPOT/MPP. A related experiment from the first ablation suggested above is the following: the vanilla vision transformer would be cheaper to run than with extended sequence length. If you assume the same computational envelope, would it ever reach the performance of VICON (maybe through larger parameter count or through reduced patch size)? If so, then it motivates running larger DPOT models as well instead of just in-context examples.\n* The joint training is interesting - is that true for other models (vision transformer included)? Here, I think the out-of-distribution error analysis would be particularly useful to see where the in-context examples help?\n* Unless I misunderstood the performance of VICON, it seems like in-context examples are expensive and not scalable. For example, if we have several million pixels in the input frame, adding 10 other pairs would be prohibitive (100x inference costs). It might even be better to finetune in this case than rely on in-context (and offload cost to the training). To help understand this design space, it would be useful to show in-context vs finetuning performance on a computational envelope - this is mentioned in the introduction but there are no experiments to support that finetuning is worse and it requires 1000s of samples to reach the same accuracy as in-context learned models.\n\nOther contributions:\n* I may have misunderstood the temporal sampling: why is the MPP/DPOT framework not amenable to varying temporal step size? In principle, they could be trained on any sampling as well. Is this already done in the experiments? It seems like the simpler way to generalize to any time step size is to simply encode that information in the input rather than do any in-context training?\n\nLIMITATIONS:\nYes, some contextualization on scalability (see comments) above would be helpful to understand how the authors see the model being used in production.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Vision In‑Context Operator Networks (VICON), which augment the ICON framework by tokenizing 2D fields as vision‑transformer patches rather than individual points. This patch‑wise representation enables efficient in‑context learning of next‑step operators across multi‑physics fluid dynamics tasks.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n1. The paper innovatively combines vision-transformer patch tokenization with in-context operator networks, enabling efficient next-step prediction in multi-physics fluid simulations. The introduction of multiple timestep-sampling strategies further enhances flexibility across varying temporal resolutions.\n\n2. A comprehensive suite of experiments—evaluating inference speed, rollout error, and robustness under dropped frames—convincingly demonstrates VICON’s effectiveness and clearly communicates its advantages over current baselines.\n\n## Weaknesses\n\n- Generality on irregular data: Treating fixed grid patches as tokens constrains applicability to regular domains; many real-world problems involve irregular meshes or unstructured data, where patch-wise operations may not extend directly.\n\n- Training–inference mismatch: The reliance on teacher forcing during training assumes access to future ground truth, which is unavailable at deployment and differs from common operator-learning settings—this may artificially simplify learning and overestimate performance. A reduced dependence on ground truth (e.g., scheduled sampling) would better align with realistic use cases.\n\n- Experimental diversity: All benchmarks stem from variations of the same underlying PDE; the model’s generalization to qualitatively different physics (e.g., multiphase flows, reacting flows) remains untested. Broader validation on disparate datasets would strengthen claims of versatility.\n\nQUESTIONS:\n1. On line 134 the authors state that all training uses “the same PDE and consistent timestep size,” yet on line 88 they claim VICON is “a unified model for multiple PDEs.” Please clarify how VICON generalizes to different timesteps or entirely new PDEs or demonstrate zero‑shot transfer to a novel PDE or timestep at inference. Does the model require fine‑tuning or does it truly interpolate across unseen physics?\n2. The PDEBench‑Comp experiments only extrapolate 10 frames, yet you advertise “Superior Performance on Long‑term Rollout Predictions.” If possible, provide results for significantly longer rollouts (e.g. 50–100 frames) to substantiate long‑term stability.\n3.  In Figure 1(a), the top‑to‑bottom datasets are annotated with Δt = 1, 3, 2, which seems out of order. Please confirm whether the middle plot should be Δt = 2 (and bottom Δt = 3), or correct the figure caption.\n4. The paper mentions both “patch” and “function” positional encodings but omits implementation details. Please describe how to compute and inject each encoding—e.g., learned vs. sinusoidal, per-patch vs. per-point—and where in the transformer block they’re added.\n5. Lines 222–224 outline a “flexible-step” training strategy, but the description is hard to follow and Table 1 shows no accuracy gain over single‑step training. Please provide a step‑by‑step example of one flexible‑step training iteration and quantify its additional compute and explain why flexible‑step yields no apparent error reduction—what is its intended benefit?\n6. Converting the field into fixed patches may introduce discretization error compared to point‑wise tokens. Please include a baseline with patch size = 1 (i.e., one token per grid point) to quantify any accuracy loss from patching.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}