FlowBench: Benchmarking Optical Flow Estimation Methods for Reliability and Generalization
Authors: Shashank Agnihotri
Julian Yuya Caspary
Luca Schwarz
Xinyan Gao
Jenny Schmalfuss
Andres Bruhn
Margret Keuper
Venue: Accepted by TMLR ()

Abstract: Optical flow estimation is a crucial computer vision task often applied to safety-critical real-world scenarios like autonomous driving and medical imaging.
While optical flow estimation accuracy has greatly benefited from the emergence of deep learning, learning-based methods are also known for their lack of generalization and reliability.
However, reliability is paramount when optical flow methods are employed in the real world, where safety is essential.
Furthermore, a deeper understanding of the robustness and reliability of learning-based optical flow estimation methods is still lacking, hindering the research community from building methods safe for real-world deployment.
Thus, we propose FlowBench, a robustness benchmark and evaluation tool for learning-based optical flow methods. 
FlowBench facilitates streamlined research into the reliability of optical flow methods by benchmarking their robustness to adversarial attacks and out-of-distribution samples.
With FlowBench, we benchmark 57 checkpoints across 3 datasets under 9 diverse adversarial attacks and 23 established common corruptions, making it the most comprehensive robustness analysis of optical flow methods to date.
Across this wide range of methods, we consistently find that methods with state-of-the-art performance on established standard benchmarks lack reliability and generalization ability.
Moreover, we find interesting correlations between the performance, reliability, and generalization ability of optical flow estimation methods, under various lenses such as design choices used, number of parameters, etc.
The open-source code and weights for FlowBench are available in this GitHub repository: https://github.com/shashankskagnihotri/FlowBench.

Published in Transactions on Machine Learning Research (August/2025
FEOWBENCH: Benchmarking Optical Flow Estimation Meth- ods for Reliability and Generalization
Shashank Agnihotri" Data and Web Science Group, University of Mannheim, Germany
Julian Yuya Caspary*
julion. yuya. www.::aaaaaaaaa.... htw.g./www.ggaadd.o....mmmmmmmma
Data and Web Science Group, University of Mannheim, Germany
Luca Schwarz"
Data and Web Science Group, University of Mannheim, Germany
Xinyan Gao
Data and Web Science Group, University of Mannheim, Germany
Jenny Schmalfuss
Computer Vision Group, University of Stuttgart, Germany
Andrés Bruhn
Computer Vision Group, University of Stuligart, Cermany
Margret Keuper
keuper@hui-mmmnneim..c
Data and Web Science Group, University of Mannheim, Germany Mlax-Planck- Planck-Institute for Informatics, Saarland Informatics Campus, Germany
Reviewed on OpenReview: https: // opw.reviee.........mmm forum? id= Kw.bj//bbM
Abstract
Optical flow estimation is a crucial computer vision task often applied to safety-critical real-world scenarios like autonomous driving and medical imaging. While optical flow esti- mation accuracy has greatly benefited from the emergence of deep learning, learning-based methods are also known for their lack of generalization and reliability. However, reliability paramount when optical flow methods are employed in the real world, where safety is essential. Furthermore, deeper understanding of the robustness and reliability of learning- based optical flow estimation methods is still lacking, hindering the research community from building methods safe for real-world deployment. Thus, propose FLOWBEENHI robust- ness benchmark and evaluation tool for learning- based optical flow methods. FLOWBENCH facilitates streamlined research into the reliability of optical flow methods by benchmarking thoir robustness to adversarial attacks and out-of-distributio samples. With FLOWBENCH we benchmark 57 checkpoints across datasets under diverse adversarial attacks and 23 established common corruptions, making the most comprehensive robustness analysis of optical flow methods to date. Across this wide range of methods, we consistently find that methods with statc-of-the-art performance reliability and generalization ion ability. Moreover, we find interesting correlations between the performance, reliability, and generalization ability of optical flow estimation methods, under varions lenses such as design choices used, number of paramoters, ctc. The open-source code and weights for FEOWH are available This GilHub repository
established standard benchmarks lack

---

Published in Transactions on Machine Learning Research AAugust/2025)
L..d. Performance
Reliability
Generalization Ability
2.5
400
150
2.0
NARE20
3300
1.5
GA
100
1.0
200
50
0.5
100
0.0
0
0
09/2017
02/2020
03/2022 01/2024
09/2017
02/2020
03/2022 01/2024 Time
09/2017
02/2020
03/2022 01/2024 Time
Time
Figure 1: Optical flow estimation methods proposed over time and their reliability and generalization ability. In all three plots, the y-axis represents error, ie., lower is better. The error of optical flow estimation methods on independent and identically distributed data samples (i.id.)) has decreased over time, however, their reliability and generalization ability are stagnant if not deteriorating.
1 Introduction
The recent growth of Deep Learning (DL) has greatly benefited computer vision, in particular when con- sidering complex tasks such as the estimation of optical flow fields. In optical flow estimation, a method is supposed to estimate the movement of every pixel between at least two consecutive image frames in a subpixel-accurate manner. This task was carlier performed using model-driven approaches such as Horn & Schunck (1981) and Lucas & Kanade (198T). However, these methods have severe limitations leading to suboptimal estimations and, consequently, to the predominant use of DL to perform the estimations (Doso- vitskiy et al., 2015 [lg et al. 2017 Jahedi et al.] 2024b). The performance of barning-based optical flow estimation methods has improved over the years on independent and identically distributed data samples (i.i.d.), leading to lower validation errors as shown by Fig. [ (left). At the same time, DL.bbased methods are known to be unreliable GGeirhos et al. 2018 Prasad 2022 Agnihotri et al., 2023), they tend to learn shortcuts rather than meaningful feature representations Geirhos et al. 2020 Alshami et al. 2025) and can be easily deteriorated even by small corruptions. This can become a practical threat, as optical flow estima- tion highly relevant in safety-critical applications such as autonomous driving (Capito et al., 2020 Wang
et al. 2021), , robotic surgery (Rosa et al., 2019) and others. Thus, before deploying DL-based optical flow estimation methods, assessing their vulnerability and generalization ability is of paramount importance. We observe in Fig. that over the years, despite improvement in the performance of learning- based optical flow estimation methods on i.i.d. samples, their reliability under adversarial attacks and generalization ability to distribution shifts are almost unchanged, and far from the performance on i.i.d. sample. Had recent research focused on these factors, the newly proposed methods could have been more reliable and ready for practical use. Our proposed FLOWBENCH facilitates this study, streamlining for future rescarch to utilize.
Many works have highlighted the importance of such a study by reducing model vulnerability (Xu et al.
2b Croce et al. 2023 Agnihotri et al. 2024a Schrodi et al. 2022; Tran et al., 2022 Grabinski et al. 2022 Medi et al.
Agnihotri et al.) 2025a), showing that robustness does follow from high accuracy (sippas et al. 2019 Schmidt et al. 2018 Schmalfuss et al. (022b) or improving generalization (Hendrycks et al. 2020; 2 for various downstream tasks such as image classification, semantic segmentation, image restoration and others. To facilitate this research, robustness benchmarking tools and benchmarks like Croce al. (2021): Jung et al. (2023): Tang et al. (2021) have been proposed for image classification models. They look into the adversarial and Out-of-Distribution (OOD) robustness of DL models. However, these works are limited to image classification. A similar benchmarking tool and comprehensive benchmark
2025
Hoffmann et al.]
for optical flow is amiss.
"These authors contributed equally to this work.
2

---

Published in Transactions on Machine Learning Research (August/2255
To bridge this gap; we propose FLOWBENCH that facilitates robustness evaluations of optical flow models against adversarial attacks and image corruptions for OOD data and provides a unified evaluation scheme and streamlined code, We use white-box adversarial attacks constrained under a Lipchitz contimuity bound, to test the stability of an optical flow model's predictions to very small perturbations optimized with mali- cious intentions. This allows for lesting the reliability of the model's predictions. Additionally, FlowBench enables perturbing the input images with unseen synthetic corruptions, which allows to test for testing the generalization abilities of the optical flow models to distribution shifts. Using FLOWBENCH, we benchmark 57 model checkpoints over 3 commonly used optical flow estimation datasets. These model checkpoints include S optical flow estimation methods and evaluation methods, including SotA ad adversarial attacks and image corruption methods. FLOWBENCI is easy to use, and new methods, datasets, metrics, and evalu- ations, when proposed, can be easily integrated to benchmark their performance. This will help researchers build better models that are not limited to improved performance on identical and independently distributed (i.i.d.) samples and are less vulncrable to adversarial attacks while generalizing better image corruptions.
The main contributions of this work are follows:
We provide FLOWBENCH, a benchmarking tool evaluate the performance of DL-based optical flow estimation methods over different datasets and make 90 checkpoints over different datasets publicly available for streamlined benchmarking while enabling the research community to add further checkpoints.
We benchmark 57 of the aforementioned model checkpoints against SotA and other commonly used adversarial attacks and common corruptions that can be easily queried using FLOWBENCH.
We perform an in-depth analysis using FLOWBEENH, showing that methods that are SotA on i.d. are remarkably less reliable and generalize worse than other non-SotA methods.
We analyze correlations between performance, reliability, and generalization abilitics of optical flow methods, under various lenses such as design choices, and the number of learnable parameters.
We show that the optimization of white-box adversarial attacks for optical flow estimation can be performed without the availability of ground truth predictions, furthering the study of their reliability.
2
Related Work
is the first robustness benchmarking tool and benchmark for optical flow estimation methods that unifies adversarial and OOD robustness, taking inspiration from robustness benchmarks for other vision tasks such as image classification. While several previous works provide benchmarking tools for optical flow estimation, they only facilitate benchmarking of either adversarial or OOD robustness and are less comprehensive than FLOWBENCH. FLOWBENCH leverages the individual strengths of prior benchmarking tools, but casts them into a unified and easy-to-use robustness benchmark. Following, we discuss these related works in detail.
2.1 Robustness Benchmarking For Image Classification Methods
Goodfellow et al. 2015) proposed the Fast Gradient Sign Method (FGSM) attack which gave rise to the domain of adversarial attacks on image classification. Complementing adversarial attacks, [Hendrycks & Diet- terich 2019) proposed 2D Common Corruptions for image classification tasks the CIFAR-100 (Krizhevsky et al. 2009 and ImageNet-lk (Russakovsky et al. 2015) datasets and their variants. Since then, most ad- versarial attacks and OOD Robustness works have focused on image classification tasks, warranting a con- solidated benchmarking tool and benchmark for robust In the case of image classification, this gap was filled by multiple works such as RobustBench (Croce et al.) (021]) and RobustArts (Tang et al. 2021). Both works make multiple image classification model chockpoints publicly available, including checkpoints trained benchmarking tool that facilitates evaluating both
for improved robustness. Moreover, RobustBench adversarial and OOD robustness of image classification models. Other similar benchmarking tools exist, like
3

---

Published in Transactions on Machine Learning Research (Agusst2225))
DeepFool Moosavi-Dezzoooi et al. 2016), Torchattacks Kim 2020 , and Foolbox (Rauber et al. 2020). Yet, these are merely benchmarking tools and do not provide a comprehensive benchmark they only facilitate evaluating adversarial robustness but not the OOD robustness of the method. As of now, no benchmark- ing tool or benchmark cxists for optical flow estimation methods' robustness evaluations. Thus, we propose FLOWBBNCCY which enables benchmarking adversarial and OOD robustness and makes a multitude of model checkpoints available, providing the research community with the much needed tools.
2.2 Benchmarking Optical Flow Estimation Methods
Optical flow estímation has been a problem attempted to be solved for a long time. Over time multiple works have been proposed to streamline research in this direction by providing benchmarking libraries for i.id, performance of proposed methods. Such librarics include mmflow (Contributors) 2021). ptlflow (Morimitsu, 2021), and Spring (Mehl et al. 2023). These libraries also provide model checkpoints to facilitate evalu- ations. Spring, also provides a benchmark but the performance evaluations are limited to their proposed Spring dataset. Whereas, both mmnflow and pllfow do not provide a robustness benchmark but enable bench- marking on multiple optical flow datasets such as FlyingThings3] (Mayer et al. 2016), KITTI2015 (Menze) & Geiger 2015) and MPT Sintel (Butler et al. 2012). Most importantly, the evaluation abilities of these benchmarking tools are limited to i.idd data. Thus, we built FLOWBENCH, using ptlflow and publicly available model checkpoints to extend method evaluations to adversarial and OOD Robustness, consolidat- ing research towards reliability and generalization ability of optical flow estimation methods. Additionally, FLOW is the first to provide comprchensive benchmark on cxisting optical flow estimation methods over 3 dalasets and multiple adversarial atlacks and image corruptions. Please note, FLOWBENCH enables evaluations of methods on synthetic corruptions. Agnihotri et al. (2025b) show in their work that synthetic corruptions can serve as a reliable proxy for real-world corruptions. However, they additionally discuss that the synthetic corruptions do not accurately represent most real-world corruptions and challenging scenarios. Thus, to cover such scenarios, we require new datasets, like those proposed by Janai et al. (2017). In this work, we focus on datasets for which we could obtain pretrained checkpoints, and thus do not consider Janai et al. (0117). We anticipate the community to benchmark new methods, trained on new datasets using the open-source FLOWBENCH benchmarking tool.
2.3 Adversarial Attacks
As discussed in Sec. [1 DL models tend to learn shorteuts to map data samples from input to target dis- , leading to the model learning inefficient feature representations. In their work, Goodfellow et al. (2015) showed that this inefficient learning of feature representations can be casily exploited. Goodfellow et al. (2015) added noise to the input data samples which was optimized to increase loss using model information, such that the model was fooled into making incorrect predictions. This demon- strated the vulnerability and unreliability of model predictions as the perturbed input samples still appeared semantically similar to the human eye. They named this attack the Fast Sign Gradient Method (FGSM). This altack led to an increased interest by the research community to better optimize the noise inspiring multiple other works such as Basic Iteration method (BIM) (Kurakin et al. 2018), Projected Gradient De- scent (PGD) (Kurakin et al.) 2017), Auto-PGD (APGD) (Wong et al. 2020) and CosPGD (Agnihotri et al., Flow 2024bb which were direct extensions to FGSM, and other atlacks such as Celturbalion-Cousttrr
tribution (Geirhos et al.] 2020
Attack (PCFA) (Schmalfuss et al. 2022b) and Adversarial Weather (Schmalfuss et al. 2023), which are indirect extensions of FGSM.
3 Considered Optical Flow Estimation Methods
Given two consecutive image frames I, and I441, the goal of optical flow estimation is to predict dense motion field 7 Ω + R?, where Ω is the image domain and f (z.z) y) = (u, T) indicates the sub-pixel accurate horizontal and vertical displacement of the pixel of I, at location (z, y) E to the same point in I(+1, i.c. for non-occluded pixels, I((x,)) y) = 1141 + u, y + u). For our analysis, we consider variety of different methods for which model checkpoints were available. While our benchmark will be continuously
4

---

Published in Transactions on Machine Learning Research (Auuust/2025)
extended, our goal is to provide reasonable coverage across different modes such as to allow for analysis of properties that are shared between more reliable and generalizable approaches.
While DL-based optical flow estimation methods differ in diverse ways, for example, w.r.t. the model ar- chitecture, point matching mechanism or training data, we attempt to structure our analysis by grouping existing models according to their core architecture. We identify four model families, each inspired by one seminal work: FlowNet (Dosovitskiy et al., 2015), PWC-Net (Pyramid, Wrapping, and Cost Volume) (Sun et al. 2018), RAFT (Teed & Deng. 2020), and FlowFormer (Huang et al., 2022). Considering each model in the context of other models with similar working mechanisms allows to better analyze key differences that render a model more reliable. In the following, we briefly discuss the propertics of different considered models (please refer to Appendix B for more details). Please also note that there exist obvious similarities of methods across families as discussed in Appendix [B] such that there remains some ambiguity regarding their grouping. The following summary should therefore not be understood as a taxonomy but it merely provides a coarse understanding of commonalitics across approaches. Overall, we provide evaluations for 57 model checkpoints, including the following, diverse, approaches.
3.1 FlowNet Family
Dosovitskiy et al. (2055) proposed the first end-to-end differentiable DL.-based architectures for optical flow estimation, termed FlowNets, building on UNet-like architectures. Several works were inspired by FlowNet, and introduced various modifications. [Ig et al. (2017) introduce FlowNet2.. 0 that uses stacked FlowNet architecture including an iterative flow relinement using image warping, and improves the schedule of training data. Many later works follow this schedule.
Compared to FlowNet2.0, LiteFlowNet (Hui et al. 2018) introduces a more effective flow inference approach at each pyramid level through a lightweight cascaded network. A flow regularization layer ameliorates the issue of outliers and vague flow boundaries, and feature warping is used instcad of image warping. LiteFlowNed22 (Hui et al., 2020) further improves accuracy and latency by making minor architectural changes to the LiteFlowNer architecture and training schedule. LiteFlow:ett (Hui & Loy| 2020) further improve upon the LiteFlowNet2.00 by alleviating the issue of outliers in the cost volume. They amend each cost vector using an adaptive modulation before the flow decoding and replace each potentially inaccurately predicted optical flow with an accurate one from a near posit ion through a warping of the flow field.
3.2 PWC-Net Family
PWC-Net (Sun et al. 2018 and FlowNets share concepts such as employing multi-scale features, warping, and the usage of a cost volume. Both are based on CNN architectures. However, PWC-Net uses the warped features to construct a cost volume at every scale, significantly outperforming FlowNet2.0 on established benchmarks like KITTI2015 (Menze & Geiger 2015) and MPT Sintel (Butler el al. 2012). FastFlowNel (Kong) let al.) 2021) replaces the dual convolution feature pyramid in PWC-Net with the head-enhanced pooling pyramid (HEPP) for high-rosolution pyramid fcatures and reduced model size. A center dense dilated corre- lation layer (MFC) for constructs compact cost volume while keeping a large search radius. DICL (Wang) cost learning. The same 2D convolution-basel matching net is applied independently on each 2D displacement hypothesis to learn a 4D cost volume and avoid learning a 5D feature volume. HD3 (Yin et al. 2019) adapts a PWC-Net- like architecture to learn probabilistic pixel correspondences both optical flow and stereo matching using a pyramid feature extractor with Ima- geNet1k (Russakovsky et al.] pre-tiere weights. HD3 estimates local matching distributions at cach scale conditioned on the matching and warping at coarser scales. IRR (Hur & Roth 2019) take inspira- tion from classical energy minimization approaches and residual networks to propose an iterative residual refinement. IRR can be combined with both FlowNets and PWC-Net. In our evaluation, we consider their adaptation to PWC-Net because it has better i..dd performance.
et al. 2020 introduces cesplacement-invarian
ScopeFlow Haim & Wolf 2020) improves upon IRR-PWC-Net by improving the data sampling process while testing the regularization and augmentations used to mitigate the bias inducod by the training proto- cols. In MaskFlowNet, Zhao et al. (2020) propose an Occlusion-Awan Feature Matching Module (OFMM) and an Asymmetric Occlusion-Aware Feature Matching Module (AsymOFMM) in PWC-Net, to overcome
5

---

Published in Transactions on Machine Learning Research (August/2025)
the ambiguity caused due to occlusions. MaskFlowNets (Zhao et al, 2020 was proposed as the first stage of MaskFlowNet,, and inherits the network architecture from PWC-Net, but replaces the feature matching modules (FMMs) by AsymOFMMs. VCN (Yang & Ramanan 2019 address the 4D cost volume used by variants of the PWC family by proposing volumetric encoder-decoder architectures that efficiently capture large receptive fields, multi-channel cost volumes that capture multi-dimensional notions of pixel similarities, and separable volumetric filtering that significantly reduces computation and paramcters.
3.3 RAFT Family
Teed & Deng (2020) proposed Recurrent All-Pairs Field Transforms (RAFT) to extract per-pixel features and build multi-scale 4D correlation volume for all pairs of pixels. A recurrent unit is used to perform look-ups on these correlation volumes. The original RAFT uses additional data and fine-tuning compared to FlowNet2.0. CCMR (Jahedi et al.] 2024ab propose adapting RAFT to use attention-based motion grouping concepts for multi-scale optical flow estimation. CCMR first computes global multi-scale context features and then uses them to guide the actual motion grouping. While iterating both steps over all coarse-to to-fie scales, they adapt cross-covariance image transformers to allow for an efficient realization while maintaining scale-dependent properties. CRAFT (Sui et al., 2022) inherits the flow estimation pipeline of RAFT and revitalizes the correlation volume with two proposed components: Semantic Smoothing Transformer the features from the second frame, and Cross-Frame Attention Layer to compute the correlation volume, Both components are intended to suppress spurious correlations in the correlation volume. CSFlow (Shi eet al.) 2022) adapts the multi-layer GRU to optical flow. It further proposes a Cross Strip Correlation module (CSC) to encode global context into correlation volumes and a Correlation Regression Initialization module (CRI). Flow1D (Xu et al. (021aa proposes 1D attention and correlation operations to compute 3D cost volumes for optical flow regression, where they adopt RAFT's framework to estimate the optical flow iteratively.
GMA Jiang et al. 2021a) adapts a RAFT architecture to include their proposed global motion aggregation (GMA) module, a transformer- based approach to find long-range dependencies between pixels. This modified RAFT architecture with a GMA further inspired other optical flow approaches. GMFlow (Xu et al. 2022) identifies correspondences in image pairs using feature similarities. They use transformer-based modules to enhance extracted features, followed by self-attention modules for feature matching and flow propagation. Their feature extraction and feature upsampling modules are identical to RAFT. GMFlooNeet (Zhao et al. 2022 adopts the iterative update operator of RAFT as the optimization step. They propose pasch-based overlapping attention (POLA) instead of multi-headed self-attention of transformer blocks to extract large context features to improve the matching step. LCV (Khairi et al.) 2024) proposes a lightweight module for learnable cost volume that adds onto RAFT. LLA-Flow (Xu et al. 2023b) proposes local similarity aggregation for 4D cost volumes and presents lightweight operations to diminish the impact of outliers caused by lack of texture within RAFT. MS-RAFT: (Jahedi el al., (024b) adapts RAFT for combining hierarchical concepts at multiple scales. It builds on top of MS-RAFT by exploiting an additional finer scale.
MatchFlow Dong et al. 2023) proposes a new feature matching extractor (FME) to be used with RAFT and GMA module. SCV (Jiang et al. 2021bb adapts RAFT to use sparse correlation volume instead of dense one, Separable (Zhang et al.
2021] proposes a separable cost volume module, a drop-in replacement to RAFT's correlation cost volumes, that uses non-local aggregation layers to exploit global context cues and prior knowledge, to disambignate motions. SKFlow (Sun et al. 2022) uses Super Kernels that allow for larger receptive fields allowing it to recover occluded motions. They use the non-local GMA module from GMA for optical flow estimations. RapidFlow (Morimitsu et al. 2024a propose a recurrent feature encoder that uses single shared block with efficient 1D layers (NeXt1D) to generate feature pyramids of variable levels. Their decoder is similar to RAFT, with few changes inspired by SKFLow (Sun et al. 2022). RPKNet (Morimitsu ct al. 2024b) adapts RAFT to use their proposed Partial Kernel Convolution (PKConv) layers and Separable Large kernels (SLK). PKConv is used to produce variable multi-scale features with a single shared block, while SLK is used to capture large context information with low computational cost. SplatFlow (Wang et al. 2024) uses splatting for feature matching architectures like RAFT and GMA. As SplatFlow is proposod as multi-frame optical flow estimation method, requires three frames at a time for training as opposed to the two frames used by RAFT and GMA. VideoPlow (Shi et al. 2023a proposes a multi-frame optical
6

---

Published in Transactions on Machine Learning Research (Auuust/2025)
flow estimation method with an iterative flow refinement module using SKBlocks from SKFlow. For feature extractors, they take inspiration from FlowFormer (Huang et al. 2022) and use ImageNet 1k pre-trained Twins-SVT (Chu et al,, 2021). They use three and five-image frames during training. NeuFlow (Zhang ct al. 2024) is inspired by GMFlow and uses transformer-bast based blocks to implement global cross-attention. They use very similar upsampling module as GMFlow and RAFT. However, to obtain feature maps with finer details, they directly extract features from the original images using a CNN block, instead of using features for matching at the 1 th th and g
scale like RAFT and GMFlow.
16
3.4 FlowFormer Family
FlowFormer (Huang et al., 2022 is a transformer-base neural network architecture for optical flow esti- mation. It tokenizes the 4D cost volume, encodes the cost tokens into a cost memory with alternate group transformer (AGT) layers in a latent space, and decodes the cost memory via a recurrent transformer de- coder with dynamic positional cost queries. The two-stage Twins-VTT (Chu al. 2021 feature extractor is pro-trained on the ImageNet1k dataset. After that, the training procedure of the entire FlowFormer is similar to RAFT's training procedure. FlowFormer++ (Shi et al. 2023b) is built upon FlowFormer and includes Masked Cost Volume Autoencoding (MCVA). They pre-train the cost-volume encoder with a mask encoding strategy.
DIP (Zhngg et al. 2022) and StarFlow (Godet et al.] 2021 have significantly different architectures than the above summarized models. We therefore consider them separately in our analysis.
4 FLOWBENCH Usage
In the following, we describe the benchmarking tool, FLOWBRENCH It is built using plt.fooww MMoimitsu 2021 and supports 37 unique architectures, for example, RAFT, FlowFormer, FlowFormer CCMR, annd others (new architectures added to ptlflow over time are compatible with FLOWBENCH) and distinct datasets, namely
MMyye et al.) 2016), KITTI2015 (Menze & Geiger 2015), MPI Sintel (Butler et al. 2012) (clean and final) and Spring (Mehl et al. 2023 datasets (please refer Appendix C for additional details on the datasets). It enables training and evaluations on all aforementioned datasets including evaluations using SotA adversarial attacks such as CosPGD (Agnihotri et al.) 2024b) and PCFA {Schmalluss et al., 2022b), Adversarial weather (Schmalfuss et al., 2023), and other commonly used adversarial attacks like BIM (Kurakin et al., 2018), PGD (Kurakin et al. 2017), FGSM (Goodfellow et al.] 2015), under various Lipshitz (l)) norm bounds. Additionally, it enables evaluations for Out-of-Distributton (OOD) robustness by corrupting the inference samples using 2D Common Corruptions (Hendrycks & Dietterich 2019) and 3D Common Corruptions (Krr et al. ).22).
We follow the nomenclature set by MobustBench (Croce el al. 2021 and use "threat model" to define the kind of evaluation to be performed. When "threat model" is defined to be "None", the evaluation is performed unperturbed and unaltered images, if the "threat "mdel"" is defined to be an adversarial attack, for example "PGD", "CosPGD" or "PCFA", then FLOWBENCH performs an adversarial attack using the user-defined parameters. We elaborate on this in Appendix E.1 Whereas, if "threat "ithreat_mood""" is defined to be 22DCommonCorruptiion or "3DCommon Corrutonnoruutioons,, the FLOWBENCH performs evaluations after perturbing the images with 2D Common Corruptions and 3D Common Corruptions, respectively. We elaborate on this in Appendix F.2 Tf the queried evaluation already exists in the benchmark provided by this work, then FLOWBENCH simply retricves the evaluations, thus saving computation.
FLOWBENCH enables the use of all the attacks mentioned in Sec.2.3 help users better study the reliability of their optical flow methods. We choose to specifically include these white-box adversarial attacks they either serve as the common benchmark for adversarial attacks in classification litcrature (FGSM, BIM, PGD, APGD) or they are unique allacks proposed specifically for pixel-wise prediction tasks (CosPGD) and optical flow estimation (PCFA and Adversarial Weather). These attacks can either be Non-targeted, which are designed to simply fool the model into making incorrect predictions, irrespective of what the model eventually predicts, or can be Targeted, where the model is fooled to make a certain prediction. Most attacks can be designed to be either Targeted Non-targeted, these include FGSM, BIM, PGD, APGD, CosPGD,
7

---

Published in Transactions on Machine Learning Research (August/225)
and Adversarial Weather. However, by design, attacks are limited to being only one of the two, for example, PCFA which is targeted attack.
In the following we show the basic commands to use FLOWBENCH. We describe each attack and common corruption supported by FLOWBENCH in detail in Appendix Please refer to Appendix for details on the arguments and function calls.
4.1 Model Zoo
It is a challenge to find all checkpoints, while training them is a time and compute exhaustive process. Thus, we gather available model checkpoints from various sources such as ptlflow (Morimitsu 2021 and mmflow (Contributors, 2021). The trained checkpoints for all models available in FLOWBENCH can be obtained using the following lines of code:
from flowbench.o evals import load_me mod_mmdel model = load_model (model pomodel_amme=:: "RAFT', dataset= 'KITT12011))
Each model checkpoint can be retrieved with the pair of 'model name', the name of the model, and 'dataset' the dataset for which the checkpoint was last finetuned. In Appendix F we provide a complete overview of all the 90 available pairs of model checkpoints and datasets. Please note, we used checkpoints trained on KITTI 2015 and MPI Sintel for the benchmarking in this work, thus using 57 available pairs (or methods + checkpoints).
4.2 Adversarial Attacks
FLOWBENCH can be used to evaluate models on the discussed adversarial attacks using the following lines of code (please refer Appendix G.1 for details regarding the arguments):
from flowbench. evals import evaluate model, results-
evaluate (model lacool_ammmmra 'RAFT', dataset= 'KITTI2015', threat threat_model= mwdemmddellllm modell= iterations=20, alpha=0.01,
epsilon=8/255, ip_norm='Lin 'Linf',,,n', targeted-True, optim_wrt= aptin_wrt="grrund_ttrr ground_truth', vetrieve_existing-Tne
4.3 OOD Robustness
FLOWBENCI can be used to evaluate models on the 2D and 3D Common Corruptions using the following lines of code, following is an example for the latter (please refer Appendix G.3 (2D Common Corruptions)
and Appendix G.4 (3D) Common Corruption) for details regarding the arguments):
from flowbench. import evaluate model, results- evaluate (modell name= 'RAF', dataset= 'KITTI2015.
threat threat_model- model= "3DComonCorruptionn" severity=3, retrieve retrieve_existieeeeeeeeeeeeeee eewprregerttttteereeeeee
5 Metrics For Analysis At Scale
Analysis optical flow estimation methods at the same scale as this work, especially under the lens of reliability and generalization ability has not been attempted before. FLOWBENCE returns evaluations using common metrics. The most commonly (Schrodi al.
2022 Sehmalfuss al. 2022a Agnihotri al. 2024b;
Dosovitskiy et al. 2015 Teed & Deng 2020 1lg et al. 2017 Huang et al. 2022) used metric for evaluating the performance of a method is the mean End Point-Pinree (EPE), the mean Euclidean distance between the
predicted optical flow 7 and the ground truth 7xm for all pairs of frames in given datasot, where lower is better. Additionally, FLOWBENCH also returns 1-px error, 3-px error, 5-px error, and cosine distance more metrics for analysis.
between two vectors (see Appendix D for details). The users free to
8

---

Published in Transactions on Machine Learning Research (August/2055)
However, any single evaluation does not reflect the reliability and generalization ability of a method. More- over, this work has performed over 4500 experiments in total, such that individual EPE, comparison is hardly interpretable. Thus, we aggregate evaluations in our proposed metrics, the Reliability Error and Generaliza- tion Ability Error. For reliability and generalization, we look at the maximum possible value mcan EPE across allacks over multiple datasets. That is, we ask the question "What the worst possible performance of a given method?". An answer to this question tells us about the reliability and generalization ability of a method. In the following, we describe the measures for different scenarios in detail.
5.1 Generalization Ability Error
Inspired by multiple works (Croce et al. 2021 Hendrycks et al., 2020 Hoffmann et al., 2021) that use OOD Robustness of methods for evaluating the generalization ability of the method, we evaluate over every common corruption, that 2D Common Corruptions and 3D Common Corruptions combined. Then, we find the maximum of the mean EPE w.r.t. the ground truth for a given method, across all corruptions al a given severity and report this as Generalization Ability Error denoted by GAEzeverity level. For example, for severity 3, the mcasure would be denoted by GAE3- The less the GAE value, the better the generalization ability of the given optical flow estimation method. These corruptions perturb the images to cause distributions and domain shifts, such shifts often confuse the methods into making incorrect predictions.
For calculating GAE, we use all 15 2D Common Corruptions: 'Gaussian Noise', Shot Noise', 'Impulse Noise', 'Defocus Blur', 'Frosted Glass Blur', 'Motion Blur', 'Zoom Blur', 'Snow', 'Frost', 'Fog', 'Brightness', 'Contrast', 'Flastie Transform', 'Pixelate', *JPEG Compression', and eight. 3D Common Corruptions: 'Color Quantization', 'Far Focus', 'Fog 3D", 'ISO Noise', 'Low Light', 'Near Focus', 'XY Motion Blur', and "Z Motion Blur'. All the common corruptions are at severity 3. Kar et al. (2022) offers more 3D Common Corruptions, however computing them is resource intensive. Thus, given our limited resources and an overlap in the corruptions between 2D Common Corruptions and 3D Common Corruptions, we focus on generating 3D Common Corruptions that might be unique from their 2D counterpart, require fewer sources to generate, and are interesting from an optical flow estimation perspective. In Appendix [A] we show that these synthetic common corruptions can indeed be used as proxy for possible corruptions when in the wild.
5.2 Reliability Error
An adversarial attack is a perturbation made on the input images to fool a method into changing its pre- dictions while the input image looks semantically similar to a human observer. Most works that focus on the reliability of optical flow estimation methods perform adversarial attacks, however, these works either focus on targeted attacks or on non-targeted attacks, not both at the same time. The objective of targeted attacks is to optimally perturb the input image such that the method predictions are changed towards a specifically desired target, for example, target can be O flow i.e. attacking so that the flow prediction at all pixels should become zero. Conversely, non-targeted adversarial attacks do not intend to shift the method's predictions to a specific larget, they simply intend to fool the method into making any incorrect predictions. To streamline research into the reliability of these methods, we perform both targeted and non-targeted attacks.
Non- Targeted Attacks. Given the ground truth optical flow field 7 gt. the objective of a non-targeted attack is to fool the optical flow estimation model into predicting a flow field 7, such as to maximize the EPE of 7 w.r.t. 7g.. The notation for the metric measuring the EPE under non-targeted adversarial allack is NAREattack iterations, where NARE stands for Non-targeted Attack Reliability Error, and the subscript indicates the number of attack iterations used for optimizing the attack. For example, when 20 attack iterations were used lo optimize the attack, then the metric would be NARE20 The higher the NARE value, the worse the reliability of the optical flow method.
Targeted Attacks. Here, the objective is to minimize the EPE of the predicted flow field 7 w.r.t. a given target flow field 7 target- In this work, we consider two targets, 0, where o' (.. v) (0.0) V(x,)) e Ω
9

---

Published in Transactions on Machine Learning Research AAugust/2025
Targeted CosPGD Attack
PCFA
Adversarial Weather (Snow) Attack
Clean Input
Target Target -}
Target -J
Targel F Flow
Nou-targeted
Tnitial Prediction/mnt
Pacturbed Inpaut
Ground Truth Jx
Fical Prediction
Figure 2: Examples of MP1 Sintel images perturbed by adversarial attacks and the optical flow predictions using FlowFormer+++ These examples are intended to show the versatility of FLOWBENCH. Here, the first column shows the clean unperturbed input, initial flow prediction 7 init and ground truth flow field 7tt- The remaining columns show from left to right the CosPGD attack used targeted attack with targets 7f and -7, the PCFA attack, proposed as only a targeted attack, with target -J, and last, the Adversarial Weather Attack with snow particles optimized a non-targeted attack.
and -,, where -x 2) -mi (2, V(r, y) E St with finit being the initial model prediction before the attack. Note that both targets can be computed without knowledge of f gt. We measure the EPE of f w.r.l. 7 target- However, to standardize notations, we report the negative EPE in this case, thus, the higher the value, the worse the performance of the optical flow estimation method. The notation for this metric is, TAREfarget iterations' where TARE stands for Targeted Attack Reliability Error and the superscript informs about the larget used (zero vector or negative of the initial flow prediction) and the subscript informs about the number of attack itcrations used for optimizing the attack. For example, when the target is O and 20 attack iterations were used to optimize the attack then the metric would be TARE The higher the TARE value, the worse is the reliability of the optical flow method.
For calculating TARE and NARE values we used BIM, PGD, and CosPGD attack with step size a=0.01, 8 perturbation budget = 255 under the -orm bound, as targeted and non-targeted attacks respectively, We use -oom bound we observe in Appendix H that there is a high correlation between the performance of optical flow estimation methods when attacked using foo -ormn bounded attacks and fg-norm bounded attacks. We 20 attack iterations for calculating TARE and NARE we observe in Appendix H that at lower number of iterations, the gap in performance of different optical flow estimation methods is small, thus an in-depth analysis would be difficult, and we do not beyond 20 attack iterations computing each attack step for an adversarial attack is very expensive, and as shown by (Agnihotri et al. 2024b) and (Schmalfuss et al.) 2022b), 20 itcrations are enough to optimize an attack to truly understand the performance of the attacked method.
In Fig. 22 we visualize a few attack samples demonstrate what the targets might look like, in case of targeted attacks, and how both attacks optimize the perturbations the clean inputs to fool the model into predicting incorrect flow fields.
6 Model Analysis
To demonstrate the potential of FLOWBENCH, we use to perform multiple analyses, which provide us with better understanding of many optical flow estimation methods. In the following, we discuss the observations. We benchmark the performance of all prominent DL-basod optical flow estimation methods across three datasets, namely KITT12015, MPT Sintel (clean), and MPI Sintel (final), against SotA and
10

---

Published in Transactions on Machine Learning Research (August/2025)
Table 1: Overview of available model checkpoints (model X, trained for dataset Y) in FLOWBENCH and checkpoints used for benchmarking. We benchmarked all available checkpoints for KITTI2015 and MPI Sintel, which are marked in teal. Models for which only FlyingThings3D checkpoints are available were not benchmarked and thus in gray. FLOWWBENC supports total of 37 methods and 90 checkpoints + KITTI2015 + MPI Sintel), for which 57 checkpoints were available for KITTI2015 and MPI Sintel that were benchmarked, which are 31 unique methods.
Dataset
Model
Fiving TaingsiD
KITTlar Tlan15
MPI Sintel
Method Family
Time
Mager
Geiger 2016
Butler
2012
OCMR Sabelli e ml
2024A
RAFT
Jamuary 2024
CRAFT
A09
RAFT
March 2022
Febeuury
DICL
Wiug <5
372
PWC
October 2020
DIP
Zerig
3774
Doep Inwerse Patchmatch
April 2002
PWC
March
Flow1D
Xu 1m4aa 11
RAFT
April 2021
FlowFornher [ilxx
2122
FlowFormer
March 2/22
Former
Former
March
Flo. [g
December 2016
GMA
Tiaic e IL MI1-E
RAFT
April 2021
GMFlow
RAFT
Novemblrr 2021
GMFlowNNt
(Zan
RAFT
March
HDS Yis es kajy
PWC
December 2018
PWC
April 2019
July
FlowNoo
May 2018
LiteFlowNet2
FlowNet
Feheuary
LitoFowNNe3
Hin
July
LLA-Flow
Xir
Mie
RAFT
April 2023
MaakFlow
Ziaro
2770)
PWC
March 2023
MackFlowNNet
March
MS- +
Pabou e
RAFT
October 20/22
Mat Plow
RAFT
March 2023
March
PWC-Net
Stal mu 20:15
PWC
September 2017
RapidFlos
20244
RAFT
May 2024
March
RPKNN
Marlinisn ei Wi24hh
BAFT
March 2024
SeapeFlaw
Hiiin & Well 20120
PWC
Feheuazy 2020
SeparableFinz
October
SKTlow
Sun. 2i
53/4
RAFT
November 2022
SpletFlow
Waing et 10 2026
X
RAFT
Jammary, 2024
STaRFlow
Loonet
SSRE low
July
Uninncth
Novemlle 2022
VUN Yan, X Tesmmoni
PWC
December 2010
Videod
RAFT
March
commonly used adversarial attacks such as BIM, PGD, and CosPGD. Tab. provides an overview of all conducted experiments. Please refer to Appendix C for details on the dataset, Appendix D for additional implementation details, and Appendix H for additional results from the benchmarking.
6.1 Analyzing Method Families
We start by analyzing properties of the different optical flow method families as discussed in Sec. by comparing their i.dd. performance but also their NARE and TARE values. These are reliability error metrics, high values indicate low reliability. Refer to Appendix D for more implementation details.
In Fig. 3a we observe that methods belonging the FlowFormer-ffmmlly and RAFT-family and DIP (Zheng et al. 2022) have the best i.i.d. performance, however, given their relatively higher NARE20 and TARE
values, some exceeding 100, they appear not be reliable. Here observe that IRR (ur & Roth 2019) stands out
of the most reliable methods under adversarial attacks. Given that the primary differences between IRR-PWC and other methods from the PWC family are the classical energy minimization-inspin approach and the use of residual networks to propose an iterative residual refinement, one might hypothesize whether the inclusion model-driven components will in general lead to more reliable methods.
When considering generalization ability under common corruptions, we observe that all methods have poor performance. Methods such as LLA-Flow (Xu 2023b) and HD3 (Yin et al. 2019) from the RAFT- family and PWC-family respectively, have GAF values over 160. Here, SplatFlow (Wang et al. 2024] yields
11

---

Published in Transactions on Machine Learning Research 1ingus/22025)
160
3.0
-10
140
10
NARE2
-15
120
Clean EPE
-20
100
-25
1.0
10
-30
60
Methods
Family
FlowlD
MaskFlowsNatS
RPKNet
RAFT
OCMR
FlowFormer
LLA-Flow
MaskFoowet
SKPlow
PWC
CRAPT
FlowFormer++
Lite lowwbet
NeuFlow
STaRElow
Deep laverse Patchmatch
CSFlow
GMA
LitelooNee22
PWCNs
ScogeFlow
FlowFormer
DECL-Flow
GMPlow
LiteFloWNer3
RAPT
SplatElow
GMFlowtee
BAFT+
RAPIDHOwW
VideoFlow
SSRFlow
RastFloweet
HD3
(a) Analyzing correlations between the method family to which the optical flow estimation method belongs and its
corresponding performance, reliability, and generalization ability.
175
Cloan EPS (3:43
150
-10
NAREio
TAMPI
125
-20
-30
10"
10"
10"
Number Parametters
Number Parameters
Number Parameters
Number
(b) Analyzing correlation between the number of learnable parameters in a DL-baaed optical flow estimation method and its performance, reliability, and generalization ability. Colors show the different optical flow methods, while marker styles show the method family to which they belong.
Figure 3: We observe that the performance of methods, especially families is independent of the number of parameters. This observation is contrary known literature that shows methods in which larger number of parameters perform better (Liu et al. 2022 Ding et al. 2022 Hoffmann et al. 2021 Additionally, we observe methods from the FlowFormer family lack robustness despite good ii.d. performance. This contradicts the observations made for transformer-based methods for image classification (Paul & Chen 2022 Hoyer et al.) 2022) and semantic segmentation (Xie et al.) 2021). Thus, we gather that optical flow methods are unique and need be studied independently of other vision tasks as many known obscrvations do not translate from other tasks to optical flow estimation.
surprisingly good results. Its primary difference other RAFT-family methods is the use of splatting for feature matching.
Additionally, we observe in Fig. 3a that compared to other method families, the Flow Former family is very susceptible to targeted adversarial attacks. Given that the FlowFormer family comprises only transformer- based architectures for optical flow estimation, we note that this contradicts the observations made for transformer- based methods for image classification (Mahmood et al. 2021 Kim et al. 2024). J. However, this lack of generalization ability can also be attributed to the use of dynamic positional cost queries by both Flow and FlowFormer++ Interestingly, when considering the NARE metric, NeuFlow (Zhang al., 2024), which is RAFT-like method using transformer blocks, also performs marginally worse than similar methods.
6.2 Impact Of The Number Of Learnable Parameters and the Point Matching Mechanisms
Several works for classification have shown that Deep Neural Networks with more parameters and less vulnerable to adversarial attacks, and generalize better to common corruptions (Liu al. 2022 Ding et al., 2022 Hoffmann et al. 2021). Tn Fig. 3b we therefore analyze the effect of the number of model parameters for
12

---

Published in Transactions on Machine Learning Research (August/2025)
optical flow estimation methods. While the number of learnable parameters has an impact on the performance of the methods to some extent (other than the exceptions of MaskFlowNet and HD3), the same does not hold for reliability and generalization ability in general. Methods such as FlowFormer, FlowwFormer-+ (FlowFormer-fanml family,, and VidcoFlow (RAFT-family) have more parameters than other methods, however, they are less reliable and have a poor generalization ability. Conversely, methods like CSFlow and SplatFlow (both RAFT-family) have significantly fewer parameters but are more reliable under attack and generalize better than the other methods to distribution shifts. Within the PWC-family, the correlation between the number of parameters and the reliability is strongest, with Kendall tau rank correlation of only 0.24 for NARE (the second plot in Fig. 3b). It is obvious from the low correlation, even in this example, that further aspects, such as the employed point-matching mechanism, matter at least equally.
Even within our considered model families, the models' approach to estimating, for example, cost volumes for point matching, differs significantly. While methods such as CCMR, CRAFT, Flow1D, NeuFlow or FlowFormer employ attention, most other methods employ convolution/ccorclattio considering allention-based methods, CCMR stands out with rather low NARE and GAE. One key difference is their employed coarse-to-fine attention scheme, originally conceived to improve efficiency. When considering Multi-Scale approaches in general, we can further observe that MSRAFT+ performs even more reliably than CCMR. This follows the intuition that coarse-to-fiee matching has a positive effect on prediction stability. Furthermore, approaches that mainly focused on improving the cost volume computation, such as DICL-Flow or Litel'loWeet3, tend to improve in terms generalization over their respective predecessors.
cost volumes. When
6.3 Targeted v/s Non-targeted Adversarial Attacks
Next, we compare the NARE and TARE values (introduced in Sec. 5.2 and find correlations in their performance. We observe in Fig. 4a that there is a very high correlation between the TARE and TARE values of every optical flow estimation method. This shows that evaluating either one of the values can serve as a reliable proxy for the other. We use this finding in the later analysis. Additionally, in Fig, 4a we observe that most optical flow estimation methods like ScopeFlow (Bar-Haim & Woll 2020), MS-RAFT+ (Jahedì et al.] 2024b) and StarFlow (Godet al., (021) are relatively more susceptible to targeted attacks than they are to non-targeted attacks. On the other hand, some methods are highly susceptible to both and thus very unreliable, these include SKFlow (Sun et al. 2022), FastFlowNet (Kong et al.
2021 HD3 (Yin et al.
2019) and some SotA methods like FlowFormer (Huan et al., 2022) and FlowFormer
(hi et al.) 2023b)
Interestingly, IRR (Hur & Roth 2019) stands out as the most reliable optical flow estimation method as it is robust to both targeted and non-targeted adversarial attacks. While ScopeFlow (Bar-Haim & Wolf 2020).
GMFlowNet [Zhao el al., (2022) and MaskFlowNer (Zhao el al., 2020) are less reliable than IRR but more reliable than the other methods.
6.4 Reliability v/s Generalization
Following, we analyze if there is a correlation between the reliability and generalization ability of optical flow estimation methods. We observe in Fig. 4b that most methods that have a good performance on i.i.d. samples also generalize better to distribution shifts as measured by GAE However, methods like FlowFormer++ while having good i.i.d. performance, have relatively poor generalization ability. As observed in Sec. 6.3 HD3 (Yin et al., 2019) stands out as having poor performance and poor generalization ability. Interestingly, as shown by Fig. 4b there is a correlation between the generalization ability (GAEs values, introduced in Sec. 5.1. higher GAE value indicates lower gencralization ability) and reliability when measured using non-targeted adversarial atlacks (NARE20 values). Additionally, most methods identified in Sec. 6.3 to be reliable, for example, CSFlow, MaskFlowNett also have considerable generalization ability compared to the other methods. However, IRR, which stood out as the most reliable method, has low generalization abilities. It is interesting to note that CCMR (Jahedi et al. 2024a) offers good trado-off it has reasonably good performance, reliability, and generalization abilities.
13

---

Published in Transactions on Machine Learning Research Auggust/2225
0
-10
-20
-10
TARES
TARE
TARES
-20
-40
-30
-00
-30
NAREza
NAREz9
TARE
Methods
Family
MaskFlowNett
RPKNet
RAFT
CCMR
FlowEormer
LLA Flow
MaskFoowett
SKFlow
PWC
CRAFT
FlowFormert+
LiteFlowNet
NeuFlow
STaRFlow
Doeg Inverse Patchmatch
CSPlow
GMA
LiteFlowNetz
PWCNet
SoopaFlow
FlowFormer
DloL
GMFlow
LitePlowNett
RAFT
Splatflow
FlowNet
DIP
GMelooNNet
MS RAPT+
RAPIDFIOw
VideoFlow
RS
Flownntt
HD3
(a) Analyzing correlations between Targeted and Non- targetee adversarial attacks. model more reliable if has
low NARE value and low TARE value.
150
150
150
125
125
6
100
100
100
75
75
75
2 Clean EPE (i..d.)
3
100
200
300
400
-30
-20
-10
NARE
TARE
(b) Analyzing correlations between reliability and generalization ability of optical flow estimation methods.
Figure 4: We observe strong positive correlation in the performance of optical flow estimation methods against largeted allacks with different targets (0 and -}), thus evaluating one serves as good proxy for another. However, we do not find strong positive correlation between generalization ability against common corruption and reliability under adversarial attacks. This hints that solutions for these problems might need to be with independent evaluations the measures provide distinct information.
CosPGD Attack
PGD Attack
Optimized
200
1.50
30
Attack Iterations
Attack treratooss
Iterations
Attack tterations
FloLD
MaskFFownnt
ScopeFlow
Figure 5: Performance of selected optical flow estimation methods under different non-targeted adversarial attacks optimized using initial flow predictions the KITTI2015 dataset.
6.5 Optimizing Targeted Attacks using Initial Flow Predictions
Based on the observation in Sec. we identify several interesting methods whose performance warrants additional analysis and discussion. Following, we discuss our observations in detail.
One of the major limitations of white-box adversarial attacks that they require access to the ground truth to optimize the attack (Agnihotri et al. 2024b). However, access to the ground truth is not guaranteed
14

---

Published in Transactions on Machine Learning Research (August/2255)
in every scenario. Additionally, discussed by Schhmalfuss et al. (2022b), robustness is measure of the difference in a model's prediction on perturbed input w.r.t. the model's prediction on clean input samples. Thus, the goal of an attack should be to fool the method into changing its initial predictions (predictions when the mothod is not attacked), independent of the ground truth. Thus, we attempt to optimize the adversarial attack w.r.t. to the initial flow prediction on the unperturbed input sample before any allacks, as access to this almost guaranteed. This helps us ascertain if initial flow predictions can be used as a proxy to ground truth while optimizing attacks. Thus, in Eq. 4 Eq. (8). Eq. (9) and there places where applicable Y Y=x (please refer Appendix E.1p. However, this optimization is only possible for attacks that introduce certain randomness in the initial input sample, as shown by Eq. 7 This allows for there to exist a non-zero loss between the predictions on the clean input samples and the perturbed input samples allowing for oplimization. We report the evaluations for CosPGD and PGD attack using the KITTI2015 dataset for 10 interesting methods in Fig. 5 We choose the optical flow estimation methods on the basis of their performance in Soc. 0 and their performance on i.i.d. samples. For additional evaluation using more models please refer to Appendix H We observe in Fig. [ that there appears high correlation in the performance of all considered methods under attack when optimized using the ground truth flow and the initial flow prediction, Thus, initial flow predictions from methods do serve a strong proxy to the ground truth for optimizing attacks. This new finding over a big sample helps advance the study in the reliability of optical flow methods, even when ground truth predictions are not available.
7 Conclusion
FLOWBENCH is the first robustness benchmarking tool and a novel benchmark for optical flow estimation methods. It currently supports 90 model checkpoints, over distinct datasets, and all relevant robustness evaluation methods, including SotA adversarial attacks and image corruptions. We discuss the unique features of FLOWBENCH in detail and demonstrate that the library is user-friendly. Adding new evaluation methods optical flow estimation methods to FLOWBENCH is easy and intuitive. In Sec. 6.1 we show that methods from the FlowFormer family have good i.dd performance but are the most unreliable under largeted attacks, also that IRR stands out to have marginally better reliability. generalization abilities. In Sec. 6.2 we show that, unlike image classification, increasing the number of learnable parameters does not help increase the robustness of optical flow estimation methods, however, a couple of RAFT variants have marginally better generalization abilities even with fewer parameters. In Sec. 6.3 we find that there is a high correlation in the performance of optical flow estimation methods against targeted attacks using different targets, thus saving compute for future works as they need to evaluate only against one target. In Sec. 66.4] we observe the methods known to be SotA on i.i.d. samples not reliable and do not generalize well to image corruptions, demonstrating the gap in current research when considering real-world applications. Additionally, we observe here that there is no apparent correlation between generalization abilities and the reliability of optical flow estimation mothods. These observations help us conclude that, based on current works, different approaches might be required to attain reliability under attacks and generalization ability to image corruptions. Tastly, in Sec. 6.5 we show that white-box adversarial attacks on optical flow estimation methods can be independent of the availability of ground truth information, and can harness the information in the initial flow predictions to optimize attacks, thus overcoming huge limitation in the field. Such an in-depth understanding of reliability and generalization abilities to optical flow estimation methods can only be obtained using our proposed FLOWBENCH. We are certain that BENCH will be immensely helpful in gathering more such interesting findings and its comprehensive and consolidated nature would make things easier for the research community.
Future Work. For optical flow estimation, patch attacks are also interesting and widely studied (Ranjan ct al.] 2019 Schrodi et al., 2022 Scheurer et al. 2024). We plan to add such patch attacks to FLOw- BENCH in future iterations. Schmalluss et al. (2022b) proposed optimizing adversarial noise jointly for the consecutive image frames and also over the entire evaluation set. Only PCFA supports such optimization regimes in FEOCH so it would be interesting to extend such optimization to other adversarial attacks as well. Croce et al. (2021) show that the training methods used significantly impact the robustness of image classification methods. The same might be true for optical flow estimation methods, thus, robustness eval-
15

---

Published in Transactions on Machine Learning Research (August/2025)
uations under the lens of different training setups used would make interesting extension to the analysis in this work. Lastly, traditional non-DI-based optical flow estimation methods might be more robust to adversarial attacks than current DL-based methods. Thus, it would be interesting to study their robustness and hopefully adapt them to increase the reliability current methods. Another interesting line of work considers black-box adversarial attacks. While white-box adversarial attacks serve as a probe for the quality of representations learned by a DL method, and thus, the stability of the output under a. Lipschitz conti- nuity bound, black-box adversarial attacks help test their performance under real-world threat scenarios. Moreover, while synthetic corruptions help study the performance of DL methods, they might not cover all real-world scenarios. Thus, including existing datasets like those introduced by Janai et al. (2017) would further improve the assessment of the real-world applicability of existing optical flow methods. These are some interesting future directions for FLOWBENCH that we intend to pursue. FLOWBENCH is completely open-source, allowing the community to generate pull requests to add new methods, attacks, checkpoints, benchmarking results, and metrics, and thus pursuc these directions of work as well.
Reproducibility Statement
Every experiment in this work is reproducible and is part of an effort toward open-source work. FLOWBENCH is open source and publicly available, including all evaluation logs and model checkpoint weights. This work intends to help the research community build more reliable and generalizable optical flow estimation methods such that. they are ready for deployment the real world, even under safety-critical applications. FLOWBENCH built upon ptlflow and thus any new model added with ptlflow would most likely be supported by FLOWBENCU as well. The proposed FLOWBENCH benchuaarkngg tool is available as library in the following codebase: https://github. com/ slw.haakkkkagmhhotr//ooooaooaaa work as is due to anonymization. The FLOOBBENCH BENCH repository is completely open-source, allowing the community to generate pull requests to add their new results and optimize the code base to add new attacks, new evaluation metrics, and others.
The repository might not
There always exists stochasticity when evaluating adversarial attacks, due to the randomness these attacks exploit, and when evaluating common corruptions due different seeds and calculation approximations made by different. Python libraries. Therefore, for transparency and reproducibility, we evaluate different runs on the same seed and different runs of different seeds. We report these evaluations in Appendix [] using Tab. 5 for adversarial attacks and Tab. 0 for common corruptions, and observe that the variance is extremely low and the analysis performed in this work still stands.
Acknowledgments
Shashank Agnihotri and Margret Keuper acknowledge funding by the DFG Research Unit 5336. Jenny Schmalfuss and Andres Bruhn acknowledge support by the Deutsche Forschungsgemeinsccatt (DFG, Ger- man Research Foundation) Project-ID 533085500. Jenny Schmalfuss acknowledges support by the Interna- tional Max Planck Rescarch School for Intelligent Systems (IMPRS-IS). The authors gratefully acknowledge through bwHPC and computing time provided on the high-
support by the state of Baden-Württember performance computer HoreKa by the National Hligh-Performance Computing Center at KIT (NHR@KIT). This center is jointly supported by the Federal Ministry of Education and Research and the Ministry of Sci- ence, Research, and the Arts of (NHR) joint funding program /https://wee nhr- www.dd//e///eee wew/ee//orppaaeeeeeeeeeeee oorroouuerrarrss HoreKa is partly funded by the German Research Foundation (DFG). We would like to thank Jonas Jakubassa for his help with refactoring the code base for public use.
as part of the National figgh-Pefformncee Computing
References
Shashank Agnihotri, Kanchana Vaishnavi Gandikota, Julia Grabinski, Paramanand Chandramouli, and Margret Keuper. On the unreasonable vulnerability of transformers for image restoration-and an easy fix. Tn Proc. of the IEEF/CVF International Conference on Computer Vision (ICCV) pp. 3707-3717, 2023.
16

---

Published in Transactions on Machine Learning Research (August/2025)
Shashank Agnihotri, Julia Grabinski, and Margret Keuper. Improving feature stability during upsampling- spectral artifacts and the importance of spatial context. In European Conference on Computer Vision, pp. 357 376. Springer, 2024a.
Shashank Aguihotri, Steffen Jung, and Margret Keuper. CosPGD: an efficient white-box adversarial attack for pixel-wise prediction lasks. In Proc. International Conference on Machine Learning (ICML), 2024b.
Shashank Agnihotri, Amaan Ansari, Annika Dackermann, Fabian Rösch, and Margret Keuper. Dispbench: Benchmarking disparity estimation to synthetic corruptions. In Synthetic Data for Computer Vision Workshop@ CVPR 2025, 2025a.
Shashank Agnihotri, David Schader, Nico Sharei, Mehmet Ege Kaçar, and Margret Keuper. Are synthetic corruptions a reliable proxy for real-world corruptions? In Synthetic Data for Computer Vision Workshop@ CVPR 2025, 2025b.
Eyad Alshami, Shashank Agnihotri, Bernt Schiele, and Margret Keuper. Aim: Amending inherent inter- protability via self-supervieed masking. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), October 2025.
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEit: BERT pre-training of image transformers. In International Conference on Learning Represenlations, 2022. URL https://openreviex. htt.:/pprrrrviieeeeeerrrrrrrrrri net.forummid
p- B-ZZ229994
Aviram Bar-Haim and Lior Wolf. Scopeflow: Dynamic scene scoping for optical flow. In Proc. IEEE/CVE Conference Computer Vision and Pattern Recognition (CVPR), pp. 7998-80099 1008-8007,00 2020.
Daniel J. Butler, Jonas WulfT, Garrett B. Stanley, and Michael J. Black. A naturalistic open source movie for optical flow evaluation. In Proc. European Conference on Computer Vision (ECCV), LNCS 7577, pp. 611-625, 2012.
Linda Capito, Umit Ozguner, and Keith Redmill. Optical flow based visual potential field for autonomous driving. In 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 885-891. IEEE, 2020.
Xiangxiang Chu, Zhi Tian, Yuqing Wang, Bo Zhang, Haibing Ren, Xiaolin Wei, Huaxia Xia, and Chun- hua Shen. Twins: Revisiting the design of spatial attention in vision transformers. Advances in neural information processing systems, 34:9355-9366, 2021.
MMFlow Contributors. MMFlow: Openmmlab optical flow toolbox and benchmark. https://github. com/
open-nmlab/mon wataaammmmmloog 2021.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schicle. The cityscapes dataset for semantic urban scene understand- ing, 2016.
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robust.ench: a standardized adversarial robustness benchmark. In Advances Neural Information Processing Systems (NewrIPS), 2021.
Francesco Croce, Naman Singh, and Matthias Hein. Robust semantic segmentation: Strong adversarial attacks and fast training of robust models. arXiu preprint Xix:2306611944 2023.
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, (0116344-6335) 2022.
Xiaohan Ding, Xiangyu Zhang, Jungong Han, and Guiguang Ding. Scaling up your kernels to 31x31: Revisiting large kernel design in cnns. In Proc. of the IEEE/CVF conference on computer vision and pallern recognition, pp. 11963 11975, 2022.
17

---

Published in Transactions on Machine Learning Research (August/2025)
Qiaole Dong, Chenjie Cao, and Yanwei Fu. Rethinking optical flow from geometric matching consistent perspective. In Proc. IERE/CVI Conference on Computer Vision and Pallern. Recognition (CVPR), pp. 1337- 1347, 2023.
Alexey Dosovitskiy, Philipp Fischer, Eddy lgg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. Flownet: Learning optical flow with convolutional networks. In Proc. of the IEEE international conference on computer vision, pp. 2758 2766, 2015.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; incrcasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2018.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaclis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning deep neural networks. Nature Machine Intelli- gence, 2(11):665 673, nov 2020.
Pierre Godet, Alexandre Boulch, Aurélien Plyer, and Guy Le Besncrais. STaRFlow: A spatiotemporal recurrent cell for lightweight multi-frame optical flow estimation. In Proc. IEEE International Conference on Pattern Recognition (ICPR), pp. 2462-2469, 2021.
Tan Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing ad adversarial examples. In Proc. International Conference on Learning Representations (ICLR), 2015.
Julia Grabinski, Paul Gavrikov, Janis Keuper, and Margret Keuper. Robust models are less over-confident. NewrIPS, 2022.
Dan Hendryeks and Thomas Dielterich. Benchmarking neural network robustness to common corruptions and perturbations. In Proc. International Conference on Learning Representations (ICLR), 2019.
Dan Hendrycks, Norman Mu, Ekin D. Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. AugMix: A simple data processing method to improve robustness and uncertainty. Proc. of the Interna- tional Conference on Learning Representations (ICLR), 2020.
J Hoffmann, S Agnihotri, Tommoy Saikia, and Thomas Brox. Towards improving robustness of compressed cmns. In ICML Workshop on Uncertainty and Robustness in Deep Learning (UDL), 2021.
Berthold KP Horn and Brian G Schunck. Determining optical flow. Artificial intelligence, 17(133):185 203, 1981.
Lukas Hoyer, Dengxin Dai, and Luc Van Gool. Daformer: Improving network architectures and training strategies for domain-adaptivv semantic segmentation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 9924-99923 2022.
Hsin-Ping Huang, Charles Herrmann, Junhwa Hur, Erika Lu, Kyle Sargent, Austin Stone, Ming-Hsuan Yang, and Deqing Sun. Self-supervised autoflow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11412 11412-11421, June 2023.
Zhaoyang Huang, Xiaoyu Shi, Chao Zhang, Qiang Wang, Ka Chun Cheung, Hongwei Qin, Jifeng Dai, and Hongsheng Li. FlowFormer: A transformer architecture for optical flow. In Proc. Europeon Conference on Computer Vision (ECCV), LNCS 13677, pp. 668-685, 2022.
Tak- Wai Hui and Chen Change Loy. LiteFlowNett:: Resolving correspondence ambiguity for more accurate optical flow estimation. In Proc. European Conference on Computer Vision (ECCV), LNCS 12365, pp. 169-184, 2020.
Tak- Wai Hui, Xiaoou Tang, and Chen Change Loy. LiteFlownet: A lightweight convolutional neural network for optical flow estimation. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognation (CVPR), pp. 8981-888 9981-8989, 2018.
18

---

Published in Transactions on Machine Learning Research (August/2025)
Tak- Wai Hui, Xiaoou Tang, and Chen Change Loy. A lightweight optical flow CNN- data fidelity and regularization. IEEE Transactions on Poltern Analysis and Machine Intelligence (Tw. (TPAMT), 43(8): 2555 2569, 2020.
Junhwa Hur and Stefan Roth. Iterative residual refinement for joint optical flow and occlusion estimation. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5754-5763, 2019.
Eddy Igg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. Flownet 2.0: Evolution of optical flow estimation with deep networks. In Proc. IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2462 2470, 2017.
Woobin Im, Sebin Lee, and Sung-Eut Eui Yoon. Semi-supervised learning of optical flow by flow supervisor. In European Conference on Compuler Vision, pp. 302 318. Springer, 2022.
Azin Jahedi, Maximilian Luz, Marc Rivinius, and Andrés Bruhn. CCMR: High resolution optical flow estimation via coarse-to-fine context-guided motion reasoning. In Proc. IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 6899-6900 6908, 2024a.
Azin Jahedi, Maximilian Luz, Mare Rivinius, Lukas Mehl, and Andrés Bruhn. MS-RAFT+: high resolution multi-scale raft. International Journal of Computer Vision (IJCV), 132(5):1835 1856, 2024b.
Joel Janai, Fatma Guney, Jonas Wulff, Michael J Black, and Andreas Geiger. Slow flow: Exploiting high- speed cameras for accurate and diverse optical flow reference data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3597-3607, 2017.
Shihao Jiang, Dylan Campbell, Yao Lu, Hongdong Li, and Richard Hartley. Learning to estimate hidden motions with global motion aggregation. In Proc. IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9772-9781, 2021a.
Shihao Jiang, Yao Lu, Hongdong Li, and Richard Hartley. Learning optical flow from a few matches. In Proc. IEEE/CVF Conference on Computer Vision and Poltern Recognition (CVPR), pp. 16592 16600, 2021b.
Steffen Jung, Jovita Lukasik, and Margret Keuper. Neural architecture design and robustness: In ICLR. OpenReview. net, 2023.
dataset.
Oguzhan Fatih Kar, Teresa Yeo, Andrei Atanov, and Amir Zamir. 3d common corruptions and data aug- mentation. In Proc. IEEE/CVF Conference on Computer Vision and Pottern Recognition (CVPR), pp. 18963 18974, 2022.
Sarra Khairi, Etienne Meunier, Renaud Fraisse, and Patrick Bouthemy. Efficient local correlation volume for unsupervised optical flow estimation on small moving objects in large satellite images. In Proc. IEEE/CVP Conference Computer Vision and Pattern Recognition (CVPR), pp. 440 448, 2024.
Gihyun Kim, Juyeop Kim, and Jong-Seok Lee, Exploring adversarial robustness of vision transformers in the spectral perspective. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3976 3985, 2024.
Hoki Kim. Torchattacks: A pytorch repository for adversarial attacks. arXiv preprint arXiv:2010.01950, 2020.
Daniel Kondermann, Rahul Nair, Stephan Meister, Wolfgang Mischler, Burkhard Güssefeld, Sabine Hof- mann, Claus Brenner, and Bernd Jähne. Stereo ground truth with error bars. In Asian Conference Computer Vision, ACCV 2014, 2014.
Lingtong Kong, Chunhua Shen, and Jie Yang. Fastflownnt: A lightweight network for fast optical flow estimation. In IEEE International Conference on Robotics and Automation (ICRA), pp. 10310-10316, 2021.
19

---

Published in Transactions on Machine Learning Research (August/2255
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 and cifar-100 datasets, learning multiple layers of features from tiny images. URI: htlps://www. cs. toronlo. edu/kriz/cijar. himl, 6(1):1, 2009.
Alexey Kurakin, Tan J. Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In Proc. International Conference on Learning Representations (ICLR), 2017.
Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial Intelligence Safely and Securily, pp. 99 112. Chapman and Hall/CRC, 2018.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proc. of the IEEE/CVE conference. on computer vision and pottern recognition. pp. 11976 11986, 2022.
Bruce Lucas and Takeo Kanade. An iterative image registration technique with an application to stereo vision. In IJCAI'8: 7th international joint conference on Artificial intelligence, volume 2, pp. 674-679. 1981.
Kaleel Mahmood, Rigel Mahmood, and Marten Van Dijk. On the robustness of vision transformers to adversarial examples. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 7838 7847, 2021.
Nikolaus Mayer, Eddy llg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A large dataset train convolutional networks for disparity, optical flow, and scene flow estimation. In Proc. IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4040 4048, 2016.
Tejaswini Medi, Steffen Jung, and Margret Keuper. Fair-tal: Improving model fairness using targeted adversarial training. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 7827-7836. IEEE, 2025.
Lukas Mehl, Jenny Schmalfuss, Azin Jahedi, Yaroslava Nalivayko, and Andrés Bruhn. Spring: A high- resolution high-detail dataset and benchmark for scene flow, optical flow and stereo. In Proc. IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 4981-4991, 2023.
Moritz Menze and Andreas Geiger. Object scene flow for autonomous vehicles. In Proc. IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pp. 3061-3070, 2015.
Seyed-Mohsen Moosavi-Dezfooli, Alhusscin Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proc. of the IEEE conference on compuler vision and pallern recognition, pp. 2574 2182-582,2 2016.
Henrique Morinitsu. Pytorch lightning optical flow. httpgs////ttaabbaccc///oottttttt
2021.
Henrique Morimitsu, Xiaobin Zhu, Roberto M Cesar, Xiangyang Ji, and Xu-Cheng Yin. RAPIDFlow: Recur- rent adaptable pyramids with iterative decoding for efficient optical flow estimation. In IEEE International Conference on Robotics and Aulomation (ICRA), pp. 2946-2955 2024a.
Henrique Morimitsu, Xiaobin Zhu, Xiangyang Ji, and Xu-Cheng Yin. Recurrent partial kernel network for efficient optical flow estimation. In AAAI Conference on Artificial Intelligence (AAAI), 2024b.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, gigh-performanc deep learning library. In Advances in Neural Information Processing Systems 32, pp. 8024 8035. Curran Associates, Inc., 2019.
Sayak Paul and Pin- Yu Chen. Vision transformers are robust learners. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp.
2022.
20

---

Published in Transactions on Machine Learning Research (August/2025)
Adarsh Prasad. Towards Robust and Resilient Machine Learning. PhD thesis, Carnegie Mellon University, 2022.
Anurag Ranjan, Joel Janai, Andreas Geiger, and Michael J Black. Attacking optical flow. In Proc. IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2404 2413, 2019.
Jonas Rauber, Roland Zimmermann, Matthias Bethge, and Wieland Brendel. Foolbox native: Fast ad- versarial attacks to benchmark the robustness of machine learning models in pytorch, tensorflow, and jax. Journal of Open Source Software, 5(33):2607, 2020. doi: 221105/jss.02607 URL https:
7/doi. .rgg 10. 21ss5/oossssooee 02607
Stephan R Richter, Zeeshan Hayder, and Vladlen Koltun. Playing for benchmarks. In Proceedings of the IEEE international conference. on computer vision, pp. 2213 2222, 2017.
Benoit Rosa, Valentin Bordoux, and Florent Nageotte. Combining differential kinematics and optical flow for automatic labeling of continuum robots in minimally invasive surgery. Frontiers in Robotics and AI, 6:86, 2019.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211 252, 2015.
doi:
Christos Sakaridis, Dengxin Dai, and Luc Van Gool. ACDO: The adverse conditions dataset with corre- spondences for semantic driving scene understanding. In Proceedings of the IEEE/CVF International Conference on Compuler Vision (ICCV), October 2021.
Erik Scheurer, Jenny Schmalfuss, Alexander Lis, and Andrés Bruhn. Detection defenses: An empty promise against adversarial patch attacks optical flow. In Proc. IEEE/CVF Winter Conference Applications of Computer Vision (WACV), 2024.
Jenny Schmalfuss, Lukas Mehl, and Andrés Bruhn. Attacking motion estimation with adversarial snow. In Proc. ECCV Workshop on Adversarial Robustness in the Real World (AROW), 2022a. doi: 10.48550/ 112. URL htttp:://di.orgg 185..5855//..i 2210. 11242
Jenny Schmalfuss,, Philipp Scholze, and Andrés Bruhn. A perturbaation-constttiiin adversarial attack for evaluating the robustness of optical flow. In Proc. European Conference on Computer Vision (ECCV), LNCS 13682, pp. 183 183-200, 2022b.
Jenny Schmalfuss, Lukas Mchl, and Andrés Bruhn. Distracting downpour: Adversarial weather attacks for motion estimation. In Proc. IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10106-10116, 2023.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adversarially robust generalization requires more data. NIPS, 31, 2018.
Simon Schrodi, Tonmoy Saikia, and Thomas Brox Towards understanding adversarial robustness of optical flow networks. In CVPR, pp. 8916-8924, 2022.
Hao Shi, Yifan Zhou, Kailun Yang, Xiaoting Yin, and Kaiwei Wang. Csflow: Learning optical flow via cross strip correlation for autonomous driving. In IEEE Intelligent Vehicles Symposium (IV), pp. 1851 1858, 2022.
Xiaoyu Shi, Zhaoyang Huang, Weikang Bian, Dasong Li, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, Jifeng Dai, and Hongsheng Li. VidcoFlow: Exploiting temporal cues for multi-frame optical flow estimation. In Proc. IEEE/CVE International Conference on Computer Vision (ICCV), pp. 12469- 12480, 2023a.
21

---

Published in Transactions on Machine Learning Research (Aguust/2025)
Xiaoyu Shi, Zhaoyang Huang, Dasong Li, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, Jifeng Dai, and Hongsheng Ti. Flow Masked cost volume autoencoding for pretraining optical flow estimation. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1599-1610, 2023b.
Xiuchao Sui, Shaohua Li, Xue Geng, Yan Wu, Xinxing Xu, Yong Liu, Rick Goh, and Hongyuan Zhu. CRAFT: Cross-attentional flow transformer for robust optical flow. In Proc. IEEE/CVF Conference Computer Vision and Pattern Recognation (CVPR), pp. 17602-17611, 2022.
Deqing Sun, Xiaodong Yang, Ming-Yu Yu Liu, and Jan Kautz. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8934-8943, 2018
Shangkun Sun, Yuanqi Chen, Yu Zhu, Guodong Guo, and Ge Li. SKFlow: Learning optical flow with super kernels. Advances in Newral Information Processing Systems (NewrIPS), 35: 35:11313-11 11326, 2022.
Shiyu Tang, Ruihao Gong, Yan Wang, Aishan Liu, Jiakai Wang, Xinyun Chen, Fengwei Yu, Xianglong Liu, Dawn Song, Alan Yuille, Philip H.S. Torr, and Dacheng Tao. Robustart: Benchmarking robustness on architecture design and training techniques. https:// /arriv..vv. wrriii..rgg//e wrg/pdf/2109 http://119..021.pppd 05211.pdf. 2021.
Zachary Teed and Jia Deng. RAFT: Recurrent all-pairs field transforms for optical flow. In Proc. European Conference on Computer Vision (ECCV), LNCS 12347, pp. 402 419, 2020.
Dustin Tran, Jeremiah Liu, Michael W Dusenberry, Du Phan, Mark Collier, Jie Ren, Kehang Han, Zi Wang, Zelda Mariet, Huiyi Hu, et al. Plex: Towards reliability using pretrained large model extensions. arXiv preprint orxi:22207.00 07.407111 11, 2022.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In ICLR, 2019.
Bo Wang, Yifan Zhang, Jian Li, Yang Yu, Zhenping Sun, Li Liu, and Dewen Hu. Splatflow: Learning multi-frame optical flow via splatting. International Journal of Computer Vision, pp. 23, 2024.
Hengli Wang, Peide Cai, Yuxiang Sun, Lujia Wang, and Ming Liu. Learning interpretable end-to-end vision- based motion planning for autonomous driving with optical flow distillation. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 13731-1373) IEEE, 2021.
Jianyuan Wang, Yiran Zhong, Yuchao Dai, Kaihao Zhang, Pan Ji, and Hongdong Li. Displacement-innann invariian matching cost Icarning for accurate optical flow estimation. Advances in Newral Information Processing Systems (NeurIPS), 33:15220 15231, 2020.
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. ArXiv, abs/20011 009944 2020.
J. Wulff, D. J. Butler, G. B. Stanley, and M. J. Black. Lessons and insights from creating a synthetic optical flow benchmark. In A. Fusiello et al. (Eds.) (ed.), ECCV Workshop on Unsolved Problems in Optical Flow and Stereo Estimation, Part II, LNCS 7584, pp. 168-177. Springer-Verlag, October 2012.
Eaze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarcz, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems, 14:12077-10090 2021.
Haofei Xu, Jiaolong Yang, Jianfei Cai, Juyong Zhang, and Xin Tong. High-resolution optical flow from 1d attention and correlation. In Proc. IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10498-10507, 2021a.
Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. GMFlow: Learning optical flow via global matching. In Proc. IEEE/CVF Conference on Computer Vision and Pottern Recognation (CVPR), pp. 8121-81 8130, 2022.
22

---

Published in Transactions on Machine Learning Research (August/205))
Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, Fisher Yu, Dacheng Tao, and Andreas Geiger. Uni- fying flow, stereo and depth estimation. IEEE Transactions on Potlern Analysis and Machine Intelligence 2023a.
(TPAMI), 45(11):13941-1330)
Jiawei Xu, Zongqing Lu, and Qingmin Liao. I.LA-Flow: A lightweight local aggregation cost volume for optical flow estimation. In JEEE International Conference on Image Processing (ICIP), pp. 3220-3224, 2023b.
Xiaogang Xu, Hengshuang Zhao, and Jiaya Jia. Dynamic divide-and-conquer adversarial training for robust semantic segmentation. In IEEE/CVF International Conference on Compuler Vision (ICCV), pp. 7466- 7475, 2021b. doi: 10.100//CCCV480222222.000
Gengshan Yang and Deva Ramanan. Volumetric correspondence networks for optical flow. Advances in Newrul Information Processing Systems (NewrIPS), 32, 2019.
Zhichao Yin, Trevor Darrell, and Fisher Yu. Hierarchical discrete distribution decomposition for match den- sity estimation. In Proc. IEEE/CVF Conference on Computer Vision and Paltern Recognition (CVPR), pp. 6044-6053, 2019.
Feihu Zhang, Oliver J Woodford, Victor Adrian Prisacariu, and Philip HS Torr. Separable Flow: Learn- ing motion cost volumes for optical flow estimation. In Proc. IEEE/CVF International Conference on Computer Vision (CVPR), pp. 10807 10817, 2021.
Zhiyong Zhang, Huaizu Jiang, and Hanumant Singh. Neuflow: Real-time, high-accuracy optical flow esti- mation robots using edge devices. arviv preprint orXiv:2403.1044 10425, 2024.
Shengyu Zhao, Yilun Sheng, Yue Dong, Eric Chang, Yan Xu, et al. MaskFlownet: Asymmetric feature matching with learnable occlusion mask. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6278-6287, 2020.
Shiyu Zhao, Long Zhao, Zhixing Zhang, Enyu Zhou, and Dimitris Metaxas. Global matching with overlapping attention for optical flow estimation. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17592-17601, 2022.
Zihua Zheng, Ni Nie, Zhi Ling, Pengfei Xiong, Jiangyu Liu, Hao Wang, and Jiankun Li. DIP: Deep inverse patchmatch for high-resolution optical flow. In Proc. IEEE/CVF Conference on Computer Vision and Pallern Recognition (CVPR), pp. 8925 8934, 2022.
23

---

Published in Transactions on Machine Learning Research (August/2025)
FLOWBENCH: Benchmarking Optical Flow Estimation Methods for Reliability and Generalization
Supplementary Material
Table Of Content
The supplementary material covers the following information:
Appendix
A Do Synthetic Corruptions Represent The Real World?:
Yes
Appendix
Reasons For Categorizing Methods To Their Respective Families
Appendix
C
Details for the datasets used.
Appendix
C.1
FlyingThings3I
Appendix
C.2
KITT12015
Appendix
C.3
MPI Sintel
Appendix
Spring
Appendix D
Additional implementation details for the evaluated benchmark.
Appendix E In detail description of the attacks.
Appendix F comprehensive look-up table for the optical flow estimation model weights and dataset pairs available in FLOWBENCH and used for evaluating the benchmark.
Appendix G In detail explanation of the available functionalities of the FLOWBENCH benchmarking tool and description of the arguments for each function.
Appendix H Here we provide additional results from the benchmark evaluated using Flow Bench. For all evaluations except Adversarial Weather, the datasets used are KITTI2015, MPI Sintel (clean), and MP1 Sintel (final).
Appendix H.1
Adversarial Finetuning: We discuss using adversarial finetuning adversarial defense method for optical flow methods.
Appendix H.2 Additional Results: Adversarial Allacks.
Appendix H.2.1 Evaluations for all models against FGSM attack under -oom bound and norm bound, targeted (both targets O' and - and non-targeted attack.
Appendix H.2.2 Evaluations for all models against BIM attack under -orm bound and fz-norm bound, as targeted (both targets J and - and non-targeted attack, over multiple attack iterations.
Appendix H.2.3 Evaluations for all models against PGD) attack under form bound and fg-norm bound, as targeted (both targets O and - and non-targeted attack, over multiple attack iterations.
24

---

Published in Transactions on Machine Learning Research (August/2025)
Evaluations for all models against CosPGD attack under -orm bound and fz-norm bound, as targeted (both targets 0' and - and non-targeted attack, over multiple attack iterations.
Appendix H.2.4
Appendix H.2.5 Evaluations for all models against PCFA attack under l--norm bound, as targeted (both targets 0' and - attack, over multiple attack iterations.
Appendix H.2.6 Evaluations for all models against Adversarial Weather attack, all four
conditions: Fog, Rain, Snow, and Sparks, as targeted (both targets 0 and -f) and non- targeted attack.
We provide an overview of the performance of the benchmarked methods against 2D Common Corruptions and 3D Common Corruptions.
Appendix H.3
Appendix H.4 2D Common Corruptions
Appendix H.4.1 Evaluating generalization capabilities of semi-supervised and self-
supervised optical flow methods using 2D Common Corruptions.
Appendix H.4.2
Evaluations for all models under 2D Common Corruptions at severity 3, for KITT12015, MPI Sintel (clean), and MPI Sintel (final) datasets.
Appendix H.5p Evaluations for all models under 3D Common Corruptions at severity 3, for KITTI2015, MPT Sintel (clean), and MPT Sintel (final) datasets.
Appendix []
We discuss the limitations of FLOWBENCH.
Appendix J We discuss the reproducibility of our evaluations and show that the variance in metries
is extremely low and our analysis comfortably holds under these variances.
A Do Synthetic Corruptions Represent The Real World?
In their work Agnihotri et al. (2025b). they find the correlation between mean mloU over the ACDC eval- uation dataset (Sakaridis et al. 2021 and mean mIoU over each 2D Common Corruption (Hendrycks & Dietterich 2019 over the Cityscapes dataset (Cordts et al. 2016). We include Figure f from their work here for case of understanding. All models were trained using the training subset of the Cityscapes datasot. ACDC is the Adverse Conditions Dalaset with Correspondences for Semantic Driving Scene Understanding captured in similar scenes are cityscapes but under four different domains: Day/ /Night, Rain, Snow, and Fog in the wild. ACDC is community-used bascline for evaluating the performance of semantic segmentation methods on domain shifts observed in the wild. They find that there exists a very strong positive correlation between the two. This shows, that yes, synthetic corruptions can serve as a proxy for the real world. Unfortunately, a similar "in the wild" captured dataset does not exist for optical flow estimation to evaluate the effect of domain shifts on the performance of optical flow methods. However, given that for the task of semantic segmentation, we find very high positive correlation between the performance on real-world corruptions and synthetic corruptions, it is safe assumption that the same would hold true for optical flow estimation as well. Thus, in this work, evaluate against synthetic 2D Common Corruptions (Hendrycks & Dietterich 2019) and synthetic 3D Common Corruptions (Kar et al. 2022
B Reasons For Categorizing Methods To Their Respective Families
Over the years, various modifications have been proposed for DL-based optical flow estimation methods, These can be based on the training strategy used or new architectures. However, barring DIP Zheng et al. (2022) and StarFlow Godet et al. (201)) that appear to have significantly different architect.ures, most other optimal flow methods can be categorized into four major families: FlowNet (Dosovitskly et al. 2015), PWC (Pyramid, Wrapping, and Cost Volume) (Sun et al., 2018), RAFT (Teed & Deng 2020),, and FlowFormer (Huang et al., (202)). In the following, we discuss the reasoning for our categorization. Please note, that methods categorized in one family can have high similarity to methods in another family. We
25

---

Published in Transactions on Machine Learning Research (lggust/202))
55
ACDC mloU
40
(mean) 2D) Common Corruption mIoU Segmentation Method
Archicecture
PSPNet
ResNet10
Swin Small
MIT-B2
DaiT
Internlmage Largge
DeepLabV3
SegFormer
x
ResNet18
Swin-Ting
MIT-B3
BEiFLarge
Internlmage-Smen Small
+
UPPrNNe
ResNet50
MIT-B
MIT-B4
Internlmage-Base
Interlmage-Ting Interninage
Mask2Former
Backhone
MIT-
loterntmage-Huge
Figure 6: Results from work by Agnihotri et al. (2025b). Here they find very strong positive correlation between mean mloU over the ACDC evaluation dataset (Sakaridis et al., 2021) and mean mIoU over each 2D Common Corruption (Hendrycks & Dietterich, 2019) over the Cityscapes dataset (Cordts al. 2016). All models were trained using the training subset of the Cityscapes dataset. ACDC is the Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding, captured in similar scenes are cityscapes but under four different domains: Day/ /Nig/N, Rain, Snow, and Fog in the wild. ACDC is community-used baseline for evaluating the performance of semantic segmentation methods on domain shifts observed in the wild.
usually solve this categorization problem by looking at when the method was proposed and which similarity was intended, for example, LiteFlowNets also have pyramidal structure similar to PWCNets, however, they are categorized in the FlowNet family since the first LiteFlowNet paper was proposed around the same time the first PWCNet paper and the similarity was not intended. Moreover, the functioning of the pyramidal structure in LiteFlowNets is significantly different from PWCNets.
B.1 FlowNet Family
Dosovitskiy al. (2015) were the first propose an end-to-end differentiable DL-based architecture for optical flow cstimation, FlowNet. Many further works were inspired by FlowNet, making changes to FlowNet to propose novel optical flow estimation methods. These methods include:
FlowNett2.0 [lg et al. 2017): : They improve upon FlowNet by changes the schedule of training data usage, using stacked architecture include the warping of the second image with intermediate optical flow, and sub-network to focus small displacements.
LiteFlowNet (Hui al. 2018): Compared FlowNet2.0 they use more effective flow inference approach at each pyramid level through lightweight cascaded network. They also a flow regularization layer to ameliorate the issue of outliers and vague flow boundaries by using feature- driven local convolution, and they use feature warping instead of image warping. They use the same training schedule as FlowNet2.0 but they train their network stage-wise.
LiteFlowNet2 Hui et al. 2020 making minor architectural changes to LiteFlowNet. They follow the training schedule of FlowNet2.0 lo some extent and perform stage-wise training.
They improve the accuracy and latency from LiteFlowNet by
LiteFlowNet' (Hui Loy 2020
They further improve upon the NiteFlowNNt2..0 by amending each cost. vector using adaptive modulation before the flow decoding to alleviate the issue of
26

---

Published in Transactions on Machine Learning Research (August/2025)
outliers in the cost volume. Additionally, they replace each potentially inaccurately predicted optical flow with an accurate one from near position through warping of the flow field. They follow a special training schedule, first training the LiteFlow modules as mentioned in Hui et al. (2020). and then training the entire architecture again with the LiteFlowNet3 modifications to LiteF ltweF2owNet2 following the training protocol from Fet..N
B.2 PWC Family
While still using features from different at different scales, warping, and cost volume, Sun et al. (0118) proposed PWC-Net which with its architectural changes, presented a significant shift in architectures from the traditional FlowNet. (Snn et al. (2018) describe, "PWC-Net uses the current optical flow estimate to warp the CNN features of the second image. It then uses the warped features and features of the first image to construct a cost volume, which is processed by a CNN to estimate the optical flow:" This was faster than FlowNet2.0,, easier to train, and significantly outperformed it on established benchmarks like KITTI2015 (Menze & Geiger 2015) and MPI Sintel (Butler et al., 2012 PWC-Net uses similar training schedule and protocol FlowOO Many other works followed PWC-Net either changing the training strategy or making architectural changes to PWC-Net further improve iid. performance. These methods include:
FastFlowNet (Kong al. 2021; ; They replace the dual convolution feature pyramid in PWC- Net with the head enhanced pooling pyramid (HEPP) for enhancing the high-rcsolution pyramid feature and reducing model size, then, they propose center dense dilated correlation layer (MFC) for constructing compact cost volume while keeping the large search radius. followed by shuffle block decoders (SBD) at each pyramid level to regress optical flow with significantly cheaper computation. They follow the same training protocol mentioned by FlowNet2.0.
DICL WWang et al. 2020): They improve upon PWC-Net by decoupling the connection between 2D displacements and learn the matching costs at each 2D displacement hypothesis independently, i... displacement-invariaat cost learning. They apply the same 2D convolution-asse matching net independently on each 2D displacement hypothesis to learn a 4D cost volume and avoid learning a 5D feature volume, thus saving computing resources. They use the same training protocol as PWC-Net and FlowNNe2.00, and use the data augmentations proposed by VCN.
HD3 (Yin et al. 2019h: They adapt a PWC-Net-like architecture for the decomposition of the discrete probability distribution instead of the feature representations allowing them to learn prob- abilistic pixel correspondences in both optical flow and stereo matching. They decompose the full match density into multiple scales hierarchically and estimate the local matching distributions at each scale conditioned on the matching and warping at coarser scales. This allows the local dis- tributions to be composed together to form the global match density. They essentially follow the same training protocol as FlowNet2.0 while omitting some hard examples. Additionally, they use Ik (Russakovsky
MaaggeeetkK(Buusaoooosskk et al., 2015 -pett-pre-traiedd weights for their pyramid feature extractor.
They take inspiration from classical energy minimization approaches, well as residual networks to propose iterative residual refinement, they show that their proposed IRR can be combined with both FlowNets and PWC-Net. In our work, we consider their adaptation to PWC-Net as that has better i.i.d. performance. They use the same training procedure as PWC- Net but additionally set out-of-bound pixels (after applying augmentations the same as those in FlowNNe2.00) as occluded.
IRR (Hur & Roth 2019
MaskFlowNet (Zhao et al. 2020): Zhao et al. (020) apart their proposed Occlusion-Aware Feature Matching Module (OFMM) and Asymmetric Occlusion-AAare Feature Matching Module (AsymOFMM) in PWC-Net and consists to two cascaded subnetworks for obtaining dual feature pyramids. Their proposed method helps them overcome the ambiguity caused due to occlusions in images that induce inaccuracies in the flow fields during warping. They use the same train- ing protocol as
However, first, they train the MaskFlowNetk then keep its weights
27

---

Published in Transactions on Machine Learning Research (Auuust/2255)
frozen while training the entire MaskFlow.Net. They use additional data from KITTI2015 and HD1k dataset (Kondermann et al. 2014) for fine-tuning on MPI-Sintel.
2020): Proposed the first stage of MaskFlow NessFlowNetS inherits the network architecture from PWC-Net, but replaces the feature matching modules (FMMs) by their proposed AsymOFMMs. They use the same training procedure IRR-PWC-Net.
NesS (Zhao et al.
ScopeFlow (Bar-Haim & Wolf, 2020: Bar-Haim & Wolf 2020) improve upon IRR-PWC-net by improving the data sampling process while testing the regularization and augmentations used to mitigate the bias induced by the training protocols. They keep some aspects of the training protocols from FlowNet2.0 intact while changing a few like cropping, and regularization at different stages of the multi-phase training.
VCN Yang & Ramanan 2019 : They improve upon the 4D cost volume used by variants of the PWC family by proposing volumetric encoder-decoder architectures that efficiently capture large receptive fields, multi-channel cost volumes that capture multi-dinnensooall notions of pixel simi- larities, and separable volumetric filtering that significantly reduces computation and parameters while preserving i.i.d. performance. They use a very similar training procodure as FlowNet2.0 and PWC-Net, however, with fewer iterations.
B.3 RAFT Family
Teed & Deng (020)) proposed Recurrent All-Pairs Field Transforms (RAFT) to extract per-pixel features to build a multi-scale 4D correlation volume for all pairs of pixels. Here a recurrent unit used to perform lookups on these correlation volumes. They use additional data and fine-tuning compared to FlowNet2.0. RAFT was significant architectural change from PWC-Nets and inspired many future works that made modifications to RAFT to further improve i.i.d. performance. These methods include:
CCMR (Jhhddi et al. 2024a They propose adapting RAFT to attention-basel motion group- ing concepts for multi-scale optical flow estimation. CCMR first computes global multi-scale context features and then uses them to guide the actual motion grouping. While iterating both steps over all coarse-to-fine scales, Jahedi et al. (2024a) adapt cross-covariance image transformers to allow for an efficient rcalization while maintaining scale-dependent propertics. They use a training proce- dure similar to MS-RAFT+, after the traditionally followed training procedure of FlowNet2.00, they additionally finetune on a mixed set from KITTI and Viper dataset (Richter et al. 2017).
CRAFT (Sui el al. 2022 CRAFT inherits the flow estimation pipeline of RAFT and revitalizes the correlation volume computation part with two proposed components: the Semantic Smoothing Transformer on the features from the sccond frame, and Cross-Frame Attention Layer to compute the correlation volume. Sui et al. (2022) propose that these two components help suppress spurious correlations in the correlation volume. They use the same training procedure RAFT.
CSFlow (Shi et al. 2022): They propose, "Cross Strip Correlation module (CSC) and Correlation Regression Initialization module (CRI). CSC utilizes a striping operation across the target image and the attended image to encode global context into corrclation volumes while maintaining high efficiency. CRI is used to maxinally exploit the global context for optical flow initialization". They take inspiration from RAFT and adapt the multi-layer GRU from the stereo estimation task to optical flow. They follow a training procedure very similar to RAFT.
Flow1D (Xu et al. 2021a]: They take inspiration from transformers (aa et al. 2022 and propose a 1D attention operation that is first applied in the vertical direction of the target image, and then a simple 1D correlation in the horizontal direction of the atlended image to achieve 2D correspondence modeling effect. The directions of attention and correlation can also be exchanged, resulting in two 3D cost volumes that are concatenated for optical flow regression, where they adopt RAFT's framework to estimate the optical flow iteratively. They follow very similar training procedure RAFT, however for harder datasets, they use additional data for fine-tuning.
28

---

Published in Transactions on Machine Learning Research (Aguust/2025)
GMA (Jinng al. 2021a): They adapt RAFT architecture to include their proposed global motion aggregation (GMA) module, transformer-baseel approach to find long-range dependencies between pixels in the first image, and perform global aggregation on the corresponding motion features. This modified RAFT architecture with a GMA further inspired other architectures and works for optical flow estimation. GMA has a very similar training procedure to RAFT.
GMFlow [Xu et al. 2022 : They adapt RAFT to identify correspondences in image pairs by com- paring their feature similarities. They use transformer--ased modules to enhance extracted features, followed by self-attention modules for feature matching and flow propagation. Their feature extrac- tion and feature upsampling modules are identical to RAFT. They follow very similar training procedure to RAFT.
They adopt the iterative update operator of RAFT as the op- limization step for their proposed GMFllowNet. They use their proposed patch-based overlapping attention (POLA) instead of multi-headed self-attention of transformer blocks to extract large con- lext features to improve the matching step. They follow a very similar training procedure to RAFT.
GMFlowNet (Zao al. 2022:
LCV (Khairi et al. 2024]: They propose lightweight module for learnable cost volume that adds onto RAFT to improve i.i.d. performance. For training, they initialize their learnable cost volume kernels to be identity and directly load the pre-trained weights from RAFT, and then they follow a similar training schedule as RAFT but with significantly fewer iterations.
LLA-Flow (u et al. 2023b They propose the local similarity aggregation for 4D cost volume and present lightweight operations to diminish the impact of outliers caused by lack of texture. They apply their module on RAFT to improve ii.d. performance. They follow very similar training procedure to RAFT.
MS-RAFT+ (Jahedi et al. 2024b): MS-RAFT adapted RAFT for combining hierarchical concepts at multiple scales. MS-RAFT- builds top of MS-RAFT by exploiting additional finer scale for estimating the flow, which is made feasible using the on-demand cost computation proposed by RAFT. They follow a very particular training schedule which is in parts similar to RAFT, however, due to an overhoad of a number of learnable paramcters, requires more data and training time.
MatchFlow (Dong et al. 2023): They propose different feature matching extractor (FME) to be used with RAFT and GMA module, this proposed FME is pre-trained on a different dataset, which allows for increased i.i.d. performance due better feature extraction and matching. After incorporating the pre-trained FME, the resultant MatchFlow is trained very similarly to RAFT.
RapidFlow (Morinilsu et al. 2024a): Inspired by RAFT, Morimitsu et al. (2024a) propose Recur- rent Adaptable Pyramids with Iterative Decoding. They propose a recurrent feature encoder that uses single shared block with efficient 1D layers (NeXt1D) to generate feature pyramids of variable levels. Their decoder is similar to RAFT, with few changes inspired by SKFLow (Sun el al. 2022 They follow a very similar training procedure to RAFT.
RPKNet (Morimitsu et al. 2024b): They adapt RAFT to use their proposed Partial Kernel Con- volution (PKConv) layers and Separable Large kernels (SLK). PKConv is used to produce variable multi-scale features with a single shared block, while SLK is used to capture large context informa- tion with low computational cost. They follow a very similar training procedure to RAFT.
SCV (Jiang et al. 2021b): They adapt RAFT to use a sparse correlation volume instcad a dense correlation volume. They follow a very similar training procedure to RAFT.
SeparableFlow Zhang et al. 2021): They propose a separable cost volume module, a drop-in replacement to RAFT's correlation cost volumes, that uses non-local aggregation layers to exploit global context eues and prior knowledge, to disambiguate motions poorly constrained ambiguous regions. They follow a training procedure the same as RAFT.
29

---

Published in Transactions on Machine Learning Research (August/2255
SKFlow (Sun et al. 2022): They propose using Super Kernels that allow for larger receptive fields allowing it to recover occluded motions. Finally, they use the non-local GMA module from GMA for optical flow estimations. They follow a similar training procedure to RAFT.
SplatFlow (Wang et al. 2024]: They essentially propose to use splatting for feature matching in architectures like RAFT and GMA. As SplatFlow is proposed to be a multi-frame optical flow estimation method, requires three frames at a time for training as opposed to the two frames used by RAFT and GMA. We use their modified version with GMA. This requires first loading the pr-trained weights of GMA as proposed by GMA, freezing them, and training the GPU prediction and convex upsampling networks introduced by Splatflow. Then, all parameters are fine-tuned using dalaset-specific fineluning procedures very similar to RAFT.
VideoFlow (Shi et al. 2023a They propose multi-frame optical flow estimation method and use the same iterative flow refinement module as other methods in the RAFT family, specifically they use the SKBlocks from SKFlow. For feature extractors, they take inspiration from FlowFormer (Huang ct al. 2022) and use ImageNet1k pre-trained Twins-SVT (Chu et al. 2021). They use three and live-image frames during training while following training procedures slightly similar to RAFT.
NeuFlow [Zhang et al. 2024): Inspired by GMFlow, they use transformer- blocks to imple- ment global cross-attention, however, they use Flash Attention (Dao et al., 2022) for slight spoed improvements. They use a very similar upsampling module G.FFlow and RAFT. However, to obtain feature maps with finer details, they directly extract features from the original images using CNN block, instead of using features for matching at the th th 16 and scale like RAFT and GMFlow. They use a very similar training procedure as RAFT.
B.4 FlowFormer Family
Proposed by Huang al. 2022 FlowFormer marks a significant shift in the architecture of optical flow estimation methods compared to the RAFT family.
FlowFormer Huang et al. 2022 ; It is a transformer- neural network architecture for optical flow estimation. Huang al. (2022) describe, FlowFrrmer tokenizes the 4D cost volume built from an image pair, encodes the cost tokens into cost memory with alternate group transformer (AGT) layers in latent space, and decodes the cost memory via recurrent transformer decoder with dynamic positional cost queries. The two-stage Twins-SVT (Chu et al. 2021 feature extractor is pre-trained on the ImageNet1k datasct. After that, the training procedure of the entire FlowFormer is similar to RAFT's training procedure.
+lowFormer++ (Shi et al. 2023b): This is built upon FlowFormer to include Masked Cost Vol- ume Autoencoding (MCVA) to improve the i.dd. performance of FlowFormer by pre-training the cost-volume encoder with a mask encoding strategy proposed by them. FlowFormer+ requires sig- nificantly different pre-training, while the training and fino-tuning procedures are similar RAFT.
C Dataset Details
FLOwBENCH supports total of four distinct optical flow datascts. Following, we describe these datasets in detail.
C.1
This is a synthetic datasel proposed by Mayer el al. (2016) largely used for training and evaluation of optical flow estimation methods. This dataset consists of 25000 stereo frames, of everyday objects such as chairs, tables, cars, etc. flying around in 3D trajectorics. The idea behind this dataset to have large volume of trajectories and random movements rather than focus on real-world application. In their work, Dosovitskiy et al. (2015) showed models trained FlyingThingss can generalize to certain extent to other datasets.
30

---

Published in Transactions on Machine Learning Research AAugust/2025
C.2 KITTI2015
Proposed by Menze & Geiger 2015 this dataset is focused the real-world driving scenario. It contains total of 400 pairs of image frames, split equally for training and testing. The image framess were captured in the wild while driving around on the streets various citics. The ground-truth labels wore obtained by an automated process.
C.3 MPI Sintel
Proposed by Butler et al. (0122) and Wulff et al. (2012). this datasel is derived from an open-source animated short film and consists of a total of 1064 synthetic frames for training and 564 synthetic frames for testing, both at a resolution of 1024 436. The intention of this dataset is enforce realism while having a datasct al scale. This dataset is provided as two datasets, which are passes with more transformations and effects the frames that originally have constant albedo over time, these passes are,
MPI Sintel (clean): This is the clean pass that adds some realism to the images by adding some spectral effects, like illumination, shadows, and smooth shading.
MPI Sintel (final): This is the final pass that adds more realism by adding effects such as blur due to depth and camera. focus, blur due motion, and atmospheric effects such as snow during snow storms, etc.
C.4 Spring
Similar to MPI Sintel, Mchl et al. (2023) proposed a new dataset and benchmark for optical flow estimation, which is much larger than any other dataset before. Tt consists of frames from the open-source Blender movie "Spring" and consists of 6000 stereo image pairs from 47 sequences with SotA visual effects at full HD resolution (1920 1080 pixels).
D Implementation Details Of The Benchmark
Following we provide details regarding the experiments done for creating the benchmark used in the analysis. In Fig. we provide an overview of the FLOWBENCH benchmarking tool.
Compute Resources. Most experiments were done on a single 40 GB NVIDIA Tesla V100 GPU each, however, MS-RAFT+, FlowFormer, and FlowFormer++ are more compute-intensive, and thus 80GB NVIDIA A100 GPUs or NVIDIA H100 were used for these models, single GPU for each experiment.
Performing adversarial attacks and OOD robustness evaluations are very expensive and compute-intenssive. Thus, performing evaluation using all model-datasel pairs is not possible given the limited computing resources at our disposal. Thus, for the benchmark, we only use KITTI2015, MPI Sintel (clcan), and MPI Sintel (final) as these are the most commonly used datascts for evaluation [Ig ct al. 2017
Datasets Used.
Huang et al. Schunalfuss et al. 2022b Schrodi et al. 2022 Agnihotri et al. 2024b
2022:
Metrics Calculation. In Sec. we introduce three new metrics for better understanding our analysis, given the large scale of the benchmark created. For calculating TARE and NARE values we used BIM, & PGD, and CosPGD attack with step size a=0.01, perturbation budget 253 under the -orm bound, as targeted and non-targeted attacks respectively. We use -oom bound as we observe in Appendix H that there is a high correlation between the performance of optical flow estimation methods when allacked using loe normn bounded attacks and fg-norm bounded attacks. We use 20 attack iterations for calculating TARE and NARE as we observe Appendia: H that a lower number of iterations, the gap in performance of different optical flow estimation methods small, thus an in-depth analysis would be difficult, and we do not go beyond 20 attack iterations computing each attack step for adversarial attack is very expensive, and
31

---

Published in Transactions on Machine Learning Research (ingust/2025)
Optical Flow Method
FlyingThings3D Dataset
Thinggg3
Weights
Method (Untrained)
KITTI2015 Dalaset
KITTI-2015 Weights
Selected Dataset
Load Parameters
Input Images from selected dalasel
MPI Sintel Dataset
MPI Sintel Weights
Method (Trained)
Corrupt Dataset
Common Comuuptions
Perturbing Inputs with Threat
Evaluation
Corrupted Data
3D Common Corruptions
All recoded metrics
Adversarial Attacks
FlowBench
Figure 7: An overview of the FLOWBENCH benchmarking tool.
shown by Agnihotri et al. (024b) and Schmalfuss et al. (2022b). 20 iterations enough to optimize attack to truly understand the performance of the attacked method. For calculating GAE, we use all 15 2D Common Corruptions: 'Gaussian Noise', Shot Noise', 'Impulse Noise', 'Defocus Blur', "Frasted Glass Blur', 'Motion Blur', 'Zoom Blur', 'Snow', 'Frost', 'Fog', 'Brightness', 'Contrast', 'Elastic Transform', 'Pixelate', 'JPEG Compression', and cight 3D Common Corruptions: *Color Quantization', 'Far Focus', 'Fog 3D', 'ISO Noise', 'Low Light', 'Near Focus', 'XY Motion Blur', and 'Z Motion Blur: All the common corruptions are at severity Kar et al. (2022) offers more 3D Common Corruptions, however computing them is resource intensive. Thus, given our limited resources and an overlap in the corruptions between 2D) Common Corruptions and 3D Common Corruptions, focus generating 3D Common Corruptions that might be unique from their 2D counterpart, require fewer sources to generate, and are interesting from an optical flow estimation perspective.
Calculating the EPE. EPE is the Euclidean distance between the two vectors, where one vector is the predicted flow by the optical flow estimation method and the other vector the ground truth case of i.i.d. performance evaluations, non-targeted attacks evaluations, and OOD robustness evaluations, while it is the target flow vector, in case of targeted attacks. For each dataset, the EPE value is calculated over all the samples of the evaluation set of the respective dataset, and then the mean EPE value is used as the mean-EPE of the respective method over the respective datasct.
Other Metrics. Apart from EPE, FLOW also enables calculating a lot of other interesting metrics, such as lo, &2, fxo distance between the perturbations of each image before and after a threat. Apart from these, in all scenarios, we also capture the outlier error, 1-px error, 3-px error, 5-px error, and cosine distance between two vectors. These vectors are the same as those the case of EPE calculations. The 1-px, 3-px, and 5-px error metrics measure the percentage of pixels where the EPE between predicted and ground-truth
32

---

Published in Transactions on Machine Learning Research (August/2025)
optical flow exceeds 1, 5 pixels, respectively. These thresholds evaluate the accuracy of flow estimates at different levels of tolerance, with lower thresholds indicating stricter accuracy. The outlier error in optical flow estimation is the percentage of pixels where the endpoint error exceeds both a fixed threshold (e.g., 3 pixels) and a relative threshold (e.g., more than 5% of the ground-truth flow magnitude), highlighting severe mismatches. The cosine distance measures the angular difference between predicted and ground-Iruth flow vectors, capturing errors in flow direction irrespective of magnitude. Lower values indicate better directional alignment. These metrics have been used my multiple previous works on optical flow estimation, such Menze & Geiger (2015) Butler et al. 2012 Schmalfuss et al. (202bb;; Mehl et al (2023).
Let fg. (Mgt, 'gg)) be the ground truth optical flow vector and fpred (Mpred, Upred) be the predicted optical flow vector at a pixel. Then the formal definition for the metrics would be,
End-Point Error (EPE):
EPE [prd - Saoll2 - -v (Upred Ugt) 2 + (Upred Ugt))
Lo Distance (Number of non-zero flow errors):
Lo = Σ
((0)
pred + fel
L Distance:
L1 = [l - -folll1 = Upred Ugel + |ppeed - Uge
Tea Distance:
Loo = [| - fgalloo = max (lupred - Ugth, |Upred Ugt
We limited the analysis in this work to use EPE, since it is the most commonly used metric for evaluation, moreover, most works on optical flow estimation Aggihotri et al. 2024b Schrnalfuss et al. 2022b Schrodi
et al. 2022 Teed & Deng 2020 Jahedi et al. 2024b) show a very high correlation between performance evaluations using different metrics.
Models Used. All available checkpoints, as shown in Tab. for MP1 Sintel and KITT12015 dataset were used for creating the benchmark, except the following four models: Separableflow, SCV, VCN, Unimatch due lo special operations used in these models, they required specific libraries which were creating conflicts with all the others models, and as most of these models are very old and do not have performance close to SotA performance, we did not include them.
Adversarial Weather For generating adversarial weather attacks, we followed the implementation pro- posed by Schmmalfsss et al.] (2023). However, generating this attack highly compute-intensive. and thus doing so for all models not possible. Thus, based on the performance and reliability of all the models, we identified few (eight) interesting models and only attacked them using the four different attacks curtailed within adversarial weather. This done to demonstrate the capability of FLOWBENCu to perform this attack. The following are the specifications for the weather attacks:
Adversarial Weather:
Snow (random nowwlakes)
Number of Particles: 3000
Number of optimization steps: 750
Adversarial Weather: Rain (rain streaks of length 0.15 with motion blur
Number of Particles: 20
Number of optimization steps: 750
33

---

Published in Transactions on Machine Learning Research (August/2025)
Adversarial Weather: Fog (random large less opacity particles)
Number of Particles: 60
Number of optimization steps: 750
Adversarial Weather: Sparks (random red sparks)
Number of Particles: 3000
Number of optimization steps: 750
Please note, that these specifications are identical to the optimal ones proposed by Schmalfuss et al.
(023
E Description of FLOWBENCH
Following, describe the benchmarking tool, FLOWBENCH. It is built using pitilow Morimitsu 2021
and supports 37 unique architectures and 4 distinct datasets, namely
(Mayer et al. 2016
KITTI2015 (Menze & Geiger 2015), MP1 Sintel (Butler et al., 2012) (clean and final) and Spring (Mehl ict al.
2023) datasets (please refer Appendix [ for additional details on the datasets). It enables training
and evalualions on all aonee dataasets including evaluations using Sot.A adversarial attacks such as
CosPGD (Agnihotri et al., 2024b) and PCFA (Schmalfuss et al.) (2022b), Adversarial weather (Schmalfuss
et al.
2023
and other commonly used adversarial attacks like BIM (Kurakin al. 2018 PGD (Kurakin FGSM (Goodfellow et al. 2015), under various lipshitz (4p) norm bounds.
et al.
2017
Additionally, it enables evaluations for Out-of-Distribution (OOD) robustness by corrupting the inference samples using 2D Common Corruptions HHendrycks & Dietterich 2019) and 3D Common Corruptions (Kar
ce al. 2022).
"threat_model" We follow the nomenclature set by RobustBench (Croce et al. 2021 and use "threat. model" to define the kind of evaluation be performed. When "threat model" is defined to be "None", the evaluation is performed unperturbed and unaltered images, if the "threat model" is defined to be adversarial attack, for example "PGD", "CosPGD" or "PCFA", then FLOWBENCH performs an adversarial attack using the user-defined parameters. We claborate on this in Appendix E.1 Whercas, if "threat "atreat_mmde""" is defined to be "DDCoomonCorrupttions" or DDCCommonCorruptionn after perturbing the images with 2D Common Corruptions and 3D Common Corruptions respectively. We claborate on this in Appendix E.2
the FLOw BENCH performs evaluations
If the queried evaluation already exists in the benchmark provided by this work, then BEnCh simply retrieves the evaluations, thus saving computation.
E.1 Adversarial Attacks
FLOWBENCH enables the use of all the attacks mentioned in Soc. 2.3 to help users better study the reliability of their optical flow methods. We choose to specifically include these while-box adversarial attacks as they either serve as the common benchmark for adversarial attacks in classification literature (FGSM, BIM, PGD, APGD) or they are unique attacks proposed specifically for pixel-wise prediction tasks (CosPGD) and optical flow estimation (PCFA and Adversarial Weather). These attacks can either be Non-targeted which are designed to simply fool the model into making incorrect predictions, irrespective of what the model eventually predicts, or can be Targeted, where the model is fooled to make a certain prediction. Most atlacks can be, designed to be either Targeted or Non-targeted, these include, FGSM, BIM, PGD, APGD, CosPGD and Adversarial Weather. However, by design, some attacks are limited to being only one of the two, for example, PCFA which is a targeted attack. Following, we discuss these attacks in detail and highlight their key differences.
FGSM. Assuming non-targeted attack, given a model fo and an umperturbed input sample eleen and ground truth label Y, FGSM attack adds noise to X clean as follows,
34

---

Published in Transactions on Machine Learning Research AAugust/2025
X Xadv Xcleam + -iggnnnV x xeleau ((f (X x Y),
(1)
(x Xxleean)
(2)
Xadv =0" ("
clean +)..
(3)
Here, L(-) is the loss function (diffcrentiable at lcast once) which calculates the loss between the model prediction and ground truth, Y. a is small value of that decides the size of the step to be taken in the direction the gradient of the loss w.r.t. the input image, which leads to the input sample being perturbed such that the loss increases. X Xadv is the adversarial sample obtained after perturbing X Xelean, To make sure that the perturbed sample is semantically indistinguishable from the unperturbed clean sample to the human eye, steps from Eq. (2) and Eq. (3) are performed. Here, function 6f is clipping the in e-ball for lo-mnom bounded attacks the e-projection in other 1p-norm bounded attacks, complying with the -orm other 4--norm constraints, respectively. While function s" clips the perturbed sample, ensuring that it is still within the valid input space. FGSM, as proposed, is a single-step attack. For targeted attacks, Y is the target and a is multiplied by -1 so that a step is taken to minimize the loss between the model's prediction and the target prediction.
BIM. This is the direct extension of FGSM into an iterative attack method. In FGSM, Xelean was perturbed just once. While in BIM, X is perturbed iteratively for time steps E [0. T), such that e Zt, where T are the total number of permissible attack iterations. This changes the steps of the attack from FGSM to the following,
Xadvei1 Xadvi sign V wavee T.So ([ (x Y),
(4)
8 = (xxaddv++ Xeleau).
(5)
xadve+1 0² (Xolean + 5).
=x"(xxlee
(6)
Here, at t=0, xadve= = =
Since in BIM, the initial prediction always started from X Xelean the attack required a significant amount of steps to optimize the adversarial noise and yet it was not guaranteed that in the permissible e-bound, Xadvi+1 was far from Xelean, Thus, PGD proposed introducing stochasticity to ensure random starting points for attack optimization. They achieved this by perturbing xelean with U( c), a uniform distribution in [-e,e], e], before making the first prediction, such that, at t=0
PGD.
Xadv = (x olean + u((-ec)).
(7)
APGD. Auto-PGD is an effective extension to the PGD attack that effectively scales the step size over attack iterations considering the compute budget and the success rate of the attack.
CosPGD. All previously discussed attacks were proposed for the image classification task. Here, the input sample is 2D image of resolution H x W, where H and W are the height and width of the spatial resolution of the sample, respectively. Pixel-wise information is inconsequential for image classification. This led to the pixel-wiee wise loss L(-) being aggregated to L(-), as follows,
L(fo(Xadv.)
Σ C(fo(Xadv:),, Y.).
(8)
H W
35

---

Published in Transactions on Machine Learning Research (Auuust/202))
This aggregation of [()) fails to account for pixel-wise information available in tasks other than image classification, such as pixel-wise prediction tasks like Optical Flow estimation. Thus, in their work Agnihotri ett al. (2024b) propose an effective extension of the PGD attack that takes pixel-wise information into account by scaling L() by the alignment between the distribution the prodictions and the distributions Y before aggregating leading to a better-oplimized allakk, modifying Eq.
(4) as follows,
X addv44 = =x x-adv + a sign V Xadva
Σ
cOs (1bsa(( X ).), V(Y.)) L (Jo (xad)))) Yi).
(9)
iEHxW
Where, functions s and are used to obtain the distribution over the predictions and Y,, respectively, and the function cos calculates the cosine similarity between the two distributions. CosPGD is the unified Sot.A adversarial attack for pixel-wise prediction tasks.
PCFA. Recently proposed by Schmalfuss et al. 2002b), is the Sot targeted adversarial attack specifi- cally designed for optical flow estimation. It optimizes the input perturbation = X X X ellea within a given ly bound to obtain a given target flow Y Mathematically, PCFA transforms the constrained opti- mization problem to find the most. destructive perturbation under an l2 constraint é2 into unconstrained optimization problem by adding a term that penalizes deviations from the l2 constraint:
xadv = Xadve + ([foin Ylars) + p. ReLU([]81 - (E2V2 x HxW))))
(10)
8
Hero, L(-) is a generic loss function, like EPE or cosine distance. The penalty scaling parameter influences how severely deviations from the per-pixel lg bound E2 are penalized. The optimization problem argminn.)
is solved with an L-BFGS optimizer.
Adversarial Weather. Unlike the previous attacks which introduced per-pixel modifications, adver- sarial weather Schmalfuss et al. 2023 2022a) attacks optical flow methods through optimizing the motion trajectories of rendered weather particles P like snow flakes, rain drops or fog clouds. The particle trajecto- ries are modelled as positions p= (P,, P2} } in the two frames 41, 12. Consequently, Xadv(P) is generated by difforentiably rendering the particles with thoir respective 3D positions to the 2D images. The update step optimizes the particle positions to achieve a certain target flow ytarg while simultancously limiting the position offset size 8pt- = pinit Pc:
16
10p dj
BB
(pttl) = Xady xpadr(p'+ +a aa- Vp. (EPE(fo(X'dy(P)) y
Σ
81
(11)
SE1,2
jEP
Here, B balances the two optimization goals of reaching the larget flow and limiting trajectory offsets. The allowed trajectory offsets are further scaled with the particle depth d in the scene, to generate visually pleasing results.
Fig. 2
shows adversarial examples created using the SotA atlacks and how they alfect the model predictions.
E.2
Out-of-Distriuutin Robustness
While adversarial attacks help explore vulnerabilities of inefficient feature representations learned by a model, another important aspect of reliability generalization ability. Especially, generalization to previously unscen samples or samples from significantly shifted distributions compared to the distribution of the samples seen while learning model parameters. As one cannot cover all possible scenarios during model training, a certain degree of generalization ability is expected from models. However, multiple works (Hendrycks & Dictterich 2019 Kar et al. 2022. Hoffmann et al. 2021 showed that models are surprisingly less robust to distribution shifts, even those that can be caused by commonly occurring phenomena such as weather changes, lighting changes, etc. This makes the study of Out-of-Distributioon (OOD) robustness an interesting avenue for research. Thus, to facilitate the study of robustness to such commonly occurring corruptions, FLOWBENCH enables evaluating against prominent image corruption methods. Following, we describe these methods in detail.
36

---

Published in Transactions on Machine Learning Research (August/2025)
Color Quantization
Far Focus
Fog 3D
ISO Noise
Low Light
Near Focus
XY Motion blur
Z- Moooio Blur
Figure 8: Examples of images from KITTI2015 corrupted using 3D Common Corruptions for evaluation of OOD robustness.
2D Common Corruptions. Hendrycks Dietterich (019) propose introducing distribution shift the input samples by perturbing images with a total of 15 synthctic corruptions that could occur in the real world. These corruptions include weather phenomena such as fog, and frost, digital corruptions such jpeg compression, pixelation, and different kinds of blurs like motion, and zoom blur, and noise corruptions such as Gaussian and shot noise amongst others corruption types. Each of these corruptions can perturb the image at different severity levels between and 5. The final performance of the model is the mean of the model's performance on all the corruptions, such that every corruption is used to perturb each image in the evaluation dataset. Since these corruptions are applied to a 2D image, they are collectively termed 2D Common Corruptions.
3D Common Corruptions. Since the real world is 3D, Kar et al. (2022) extend 2D Common Corruptions to formulate more realistic-looking corruptions by leveraging depth information (synthetic depth information when real depth is not roadily available) and luminescence angles. They name these image corruptions 3D Common Corruptions. Fig. shows examples of KITTI2015 images corrupted using 3D Common Corruptions.
F Model Zoo
The trained checkpoints for all models available in FLOWBBECH can be obtained using the following lines of code:
from flowbench. evabls import load_model model model lloddmodeel (model_namee name= 'RAFT', Dittase==KITTI2000
Each model checkpoint can be retrieved with the pair of 'model name', the name of the model, and 'datasct', the datasel for which the checkpoint was last fine-tuned. In Table 2 we provide a comprehensive look-up table for 'model name' and 'dataset" pairs for which trained checkpoints are available in FlowBench.
G FLOWBENCH Usage Details
Following we provide a detailed description of the evaluation functions and their arguments provided in FlowBeech.
G.1 Adversarial Attacks
To evaluate a model for given dataset, on allack, the following lines of code are required.
from flowbench. evals import evaluate
model, results evaluate (model name= RAFT', dataset= 'KITTI2015',
threat_moder = CoaPGD', iterations=20, alpha=0.00 01,
37

---

Published in Transactions on Machine Learning Research (August/2255
Table Overview of all available model checkpoints (model X, trained for dataset Y) in FLOWBENCI.
Datamet
Model
Fiving Thine33
KTTT22015
MPI Sintel
Method Familly
Time
Mayer
Monse Getger 2015
Butler
2011
Mizia
Jamury
GRAFT
Ni manel.24 AB
RAFT
March 2022
CSFlow
RAFT
Februury 2022
DIP
3127
Diep Inverse Patchmatch
April 2022
Fust Fot
m1
2122
PWC
March 2721
2021
Flow:Prmerr
2122
FlowFormee
March 2022
Flow
S3 cl
412210
FlawFormmer
March 2/23
Fot..N
Flow Not
December 2016
GMA
Jigue 2f
MOMA
RAFT
April 2021
GMIFON
RAFT
Nevember 2021
GMFlouNr
RAFT
March
HDS
Mi
PWC
December 2018
IRR
[Tur Hotn
23
PWC
April 2019
LCV
RAFT
July
LitoF Farvet
May
LiteFlon[ett2
Thu et
FlowNes
Febeuary 2020
LitoPloWNetd
Flow
July 2020
Maok Nrt
(A122)
PWC
March 2023
MuckFloeNet
Zhab
A7/0
PWC
March 2023
MatekFloow
Pong Be
RAFT
March 2/23
NoaFlow
Flow Not
March 2024
PWC
September
RapidFlon
21240
RAFT
May 2014
RAFT
TRu Maay 2020
RAFT
March 2020
Murminet
7022h
March
ScopeFlow
[ar & Voll M0100
PWC
Febeuary 2010
Separable kow
Zraamy 1E
2012
RAFT
October 2021
SkCPlow
Sun
RAFT
Novemler 2022
SplatFloe
Wrimp
Jamuayy 2024
STaRFlow
MIPT
SSRE low
July 2020
Unimatch
RAFT
November 2022
December 2019
Video Jow Sli 66
RAFT
March 2023
epsilon-8/265, 1p_ porm='Linf" targeted-True, optim_ wrt- wrt='ground truth', retrieve, existing
The argument description is as follows:
'moeel name' is the name the optical flow estimation method be used, given as string.
dataset' is the name of the dataset to be used also given as string.
chreat_model
model' is the name of the adversarial attack to be used, given string.
'iterations' are the number attack iterations, given as an integer.
eepsllon' is the permissible perturbation budget given floating point (float).
'alpha' is the step size of the attack, a, given floating point (float).
"pp norm' is the Lipschitz continuity norm (g--norm) to be used for bounding the perturbation,
possible options are 'Linf and 'L2' given as a string.
'targeted' is boolean flag that decides if the attack must be targeted or not. If targeted="True then by default the target is 0, passed as target='zero', this can be changed to negative of the initial flow by passing 'negaiv=.
optim wrt' decides wrt what attack should be optimized, available choices 'ground truth' and 'initial flow" as string. Please note, this only works well with attacks that utilize Eq. 7
'retrieve_ retrieve_eciisinng is boolean flag, which when set to "True' will retrieve the evaluation from the benchmark if the queried evaluation exists in the benchmark provided by this work, else BRoCBBecH
38

---

Published in Transactions on Machine Learning Research (August/2025)
will perform the evaluation. If the 'retrieve_extee existingg boolean flag is set to 'False' then FLOWBENCH will perform the evaluation even if the queried evaluation exists in the provided benchmark.
G.2 Adversarial Weather
As attack, adversarial weather works slightly different compared to other adversarial attacks, thus we additionally mention the commands for using adversarial weather.
from flowbench.evee evals import evaluate
model, results evaluate (model _aals___aaa_____eeeee
"aaeT=RAFT' RAFT', dataset= Sintee--ial
threat_modell Adversarial Weather', "eccher''now'. num wi__particles-10000 targeted-True, retrieve retrieve_existing existing-True)
The argument description is as follows:
'moee name' is the name of the optical flow estimation method to be used, given string,
dataset' the name of the dataset to be used also given as string.
ttreat__roee model' is the name of the adversarial attack to be used, given as string.
'weather' the name of the weather condition in adversarial weather attack to be used, given as string, options include 'snow', 'fog', 'rain' and 'sparks'
'num particles" is the number particles per frame to be used, given as a integer.
'targeted' is boolean flag that decides if the attack must be targeted or not. If targeted=TTrue" then by default the target is 0' passed as target='zero', this can be changed to negative of the initial flow by passing harget='negatin
opptmm wrt' decides wrt what attack should be optimized, available choices are 'ground truth' and 'initial 'low'' as string. Please note, this only works well with attacks that utilize Eq. 7
'retrieve existing' is boolean flag, which when set 'True' will retrieve the evaluation from the benchmark if the queried evaluation exists in the benchmark provided by this work, else FLOWBENCH will perform the evaluation. If the 'retrieve _eirrr____tiinge boolean flag is set to 'Falsc' then FLOWBENCH will perform the evaluation even if the queried evaluation exists in the provided benchmark.
Please note, adversarial weather data requires additional particle data as computed by Schmalfuss et al. 2023) and path to the same should be provided demonstrated in the GitHub repository: https: com httpc///wwws.a....ooooooo///eeae
/
G.3 2D Common Corruptions
To evaluate model for a given datasot, with 2D Common Corruptions, the following lines of code are required.
from flowbench. evals import evaluate model, results evaluate (modde_namee 'RAFT', dataset- 'KITTI0011,
threat_model 2DCommonCorruption" severity=3, retrieve retrieve_existineee existing=True)
from flowbench, evals evals import evaluate
model, results evaluate (aooell_mmmmmenaaaaaaa nam_name==ep RAFT', dataset= KKTTT221
'RAFT',
retrieve_ seaiieee_eriettin-Trrree
39

---

Published in Transactions on Machine Learning Research AAugust/2025
The argument description is as follows:
'model_nann name" is the name the optical flow estimation method to be used, given as a string.
dataset' the name of the dataset be used also given as string.
threat model' is the name of the common corruption to be used, given string.
'severity is the severity of the corruption, given as an integer between 1 and 5 (both inclusive).
'retrieve retrieve__xxisiing' is a boolean flag, which when set to "True' will retrieve the evaluation from the benchmark the queried evaluation exists the benchmark provided by this work, else FLowBENCH will perform the evaluation. If the 'retrieve existing' boolean flag is set to 'False' then Flow BENch will perform the evaluation if the queried evaluation exists in the provided benchmark.
FLOW BENCH supports the following 2D Common Corruption: 'gaussian noise', shot. noise', 'im- pulse noise', 'defocus blur', 'frosted glass blur', 'motion blur', 'zoom blur', 'snow', 'frost', 'fog', 'bright- ness', 'contrast', 'clastic', 'pixclatc', 'jpeg'. For the evaluation, FLOWBENCH will evaluate the model on the validation images from the respective dataset corrupled using each of the aforementioned corruptions for the given severity, and then report the mean performance all of them.
G.4 3D Common Corruptions
To evaluate a model for a given dataset, with 3D Common Corruptions, the following lines of code are required.
from flowbench. evals import evaluate
model, results evaluate (model_name= RAFT', 'RAFT', dataset= KITTT12011,,
threat threat_model= model= 3DCommonCorruption"" severity=3, retrieve existing-True)
The argument description is as follows:
'model name' is the name of the optical flow estimation method be used, given as string,
ddtasset is the name of the dataset to be used also given string.
threat model' is the name of the common corruption to be used, given as string.
'sverrtyy is the severity of the corruption, given as an integer between 1 and 5 (both inclusive).
'retrieve existing" is boolean flag, which when set "True' will retrieve the evaluation from the benchmark if the queried evaluation exists in the benchmark provided by this work, else FLOWBENCH will perform the evaluation. If the 'retrieve existing' boolean flag is set to 'False' then FLOw Bench will perform the evaluation even if the queried evaluation exists in the provided benchmark.
FLOWBENCH supports the following 3D Common Corruption: 'color _uant', 'far focus', 'fog 3d', isso noise', 'low light', 'near focus', 'xy motion blur', and 'z motion blur! For the evaluation, FLow- BENCH will evaluate the model on the validation images from the respective dataset corrupted using each of the aforementioned corruptions for the given severity, and then report the mean performance over all of them.
H Additional Results
Following, include additional results from the benchmark made using FROWB
40

---

Published in Transactions on Machine Learning Research (August/202)
Table 3: Results from a subset of methods adversarially finetuned on the KITTI2015 training dataset using non-targeted -orm constrained 3 iterations PGD attack with с with non-tat con-targeted lx-norm constrained CosPGD, PGD, and BIM attack with Number of Attack Iterations E [3. 5, 10, 20).
4 8 and a 0.01 and attacked
255
1255 255 255 253
8
and
Model
Training
Clean
CosPGD
PGD
BIM
203
B3G
253
205
255
253
B5G
253
205
255
Attack Iterations =
FastFlowNee
Baseline
2.17
11.98
25.31
46.30
57.53
11.58
25,58
47.38
59.25
15.87
28. 35
44.81
65.94
FastFlowNed
Adv =
29.55 89.81
29.11
29.
29.99
30. 12
29.00
29.02
29.27
29.16
28. 70
29,
30.37
31.77
FastFlowNeet
Adv 2015
30.41
30.60
31.15
31.32
29.93
30.81
29.62
30.18
31.50
33.00
Flom:Frrner
Baseline
0.49
3.90
8.65
14.39
17.43
3.58
8.66
14:31
17.40
5.24
9.39
15.80
24.75
FlowFormer
Adv = 73
36.74
36,74
36.74
36.74
96. 73
36.74
36.74
36.74
36.73
38.74
36.74
96.74
36.73
Flow Fowmore
Adv e
36.74
36.74
36.74
36.73
36. 72
36.74
36.74
36.73
36.72
36.74
36.74
36.73
36.71
IRR-PWC
Baseline
1.80
2.28
2.61
1.59
1.78
2:25
1.61
1.85
2.37
3.57
PWC
Adv 405
35.38
35.40
35.43
35.46
35.49
35.40
35.43
35.45
35.48
35.39
35.43
35.50
35.60
IRR-PWC
Adv = 285
35.74
35.75
33.77
35.80
35.82
35.75
33.77
35.79
35.82
35.75
35.77
35.82
35,89
RAFT RAFT
Baseline
0.64
3.67
8.85
15.01
17.54
3.71
8.85
15.14
17.48
5.69
10.13
16.93
27.16
Adv 7
1.81
7.11
10.61
12.012
3.95
7.13
11.97
4.89
7.64
11.37
16.26
RAFT
Adv
205
1.99
4.25
7.38
10.69
12.06
4.15
7.30
10.65
12.002
5.14
7.87
11.45
16.04
Attack Irerations
FastFlowNe
Baseline
2.17
14.15
37.44
80.86
120.66
13.36
35.80
81.47
118.29
20.13
41.95
72.80
108.27
NlstFlowNet
Adv zg
29.81
29.37
29.61
30.17
31.07
29.00
29.03
29.42
30.22
28.77
29.15
30.22
31.71
FostFlowNet
Adv 205
30.41
30.28
30.53
31.43
31.90
29.86
30.06
30.56
31:35
29.68
30.28
31.48
33.15
FlowForrerr
Baseline
0.49
3.15
10.46
30.48
49.88
3.19
10.37
30.36
49.118
7:24
13.16
17.65
24.38
FlowFFrmer
Adv = 7
36.74
36.74
36.74
36.73
36.70
36.74
36.74
36.73
36.71
36.74
36.74
36.73
36.68
FlowoFormer
Adv c 285
38.74
36.74
36.74
36.72
36.68
36.74
36.74
38.72
36.68
36.74
36.74
36.73
36.67
IRR-PWC
Baseline
1.11
1.60
1.81
2:31
2.99
1.60
1.79
2.35
3.05
1.62
1.87
2.37
3.12
TRR-PWC
Adv z01
35.38
35.40
35.43
35.46
35.50
35.40
35.43
35:46
35.50
35.39
35.43
35.50
35.59
PRC-PWC
Adv 201
33.77
35.75
335.77
35.79
35.88
335.75
33.77
35.79
35.82
35.75
35.77
35.82
35.88
RAFT RAFT
Baseline
0.64
3.39
12.12
31.58
43.72
3.44
12.29
31.66
42.84
8.05
15.51
24.80
37.33
Adv
7
1.81
3.83
8:06
17.98
21.75
3.71
8.11
18.06
21.73
5.71
9.51
14.33
20.40
RAFT
Adv
205
1.99
3.99
8.29
17.88
21.53
3.87
8.24
17:52
21.29
5.98
9.62
14.00
19.51
Astack Iterationn = 10
FustFlowNe
Baseline
2.17
14.72
56.74
155.64
258.25
14.61
59.04
152.13
258.05
29.05
69.71
125.29
173.005
FastFlowNet
Adv 4
29.
29.86
29.89
30.76
32.37
29.32
31.61
28.77
28.91
30.11
24
FastFlowNe
Adv 205
30.41
30.80
301.89
31.81
33.
30.30
30.37
31.112
32.44
29.81
30.01
31.33
33.31
FlomFFormer
Baseline
0.49
2.96
9.75
38.32
124.63
2.93
9.47
37.85
115.52
7.00
12.86
22.33
35.79
FlowFormer
Adv == 2
36.74
36.74
36.74
36.72
36. 42
36.74
36.74
36.72
36.43
36.74
36.74
36.77
36.59
FlowFormer
Adv 285
36.74
36.74
36.74
36.71
36.35
36.74
36.74
36.71 71
36.35
36.71
36.74
36.72
36.58
IRR-PWC
Baseline
1.44
1.60
1.79
2.38
3.399
1.60
1.80
2.35
3.48
1.60
1.80
2.33
3.46
PWC
Adv 405
35.38
35.40
35.42
35.47
35.51
35.40
35.42
35.46
35.51
35.39
35.42
35.47
35.57
IRR-PWC
Adv e =
35.774
35.75
335.77
35.80
35.84
33..75
33.77
35.50
35.83
35.775
35.76
35.80
35.87
RAFT RAFT Adv = 37
Baseline
0.64
3.18
11.59
40.53
97.49
3.17
11.23
39.64
96.35
7.73
13.64
24.77
43.54
1.81
3.55
7.61
20.38
42.78
3.52
7.61
20.89
42.66
5.51
8.82
14.38
20.58
RAFT Adv 285
1.99
3.74
7.75
20.45
40.90
3.70
7.44
19.61
40.72
5.52
8.86
13.42
20.16
Astack Iterations
20
FlstelowWett
Baseline
2.17
15.27
77.80
213.35
158.65
15.18
76.99
215.23
453.35 52
37.60
98. 10
171.28
213.98
Nos
Adv
29.81
29.85
29.98
31.27
34.75
29.41
29.27
30.41
34.30
28. 28.84
28.92
30.60 60.66
33.95
FastFlowNer
Adv
my
30.41
30.82
31.01
32.18
34.55
30.42
30.38
31.70
33.93
29.81
30.12
31.79
34.64
Formorreer
Baseline
0.49
2.72
9.57
56.61
201.03
2.83
10.33
52.50
206.1 17
7.26
13:41
23.88 82
36.45
Former
Adv == T
36.71
36.71
36. 34
36.71
35.91
36.74
36.74
36.71
35.97
36.71
36.74
36.
36.55
Flow Former
Adv
205
36.74
36.74
36.74
36.70
36.02
36.74
36.74
36:70
36.01
38.74
36.74
36.72
36.55
IRR-PWC
Baseline
1.41
1.59
1.80
2:17
3.73
1.59
1.79
2.12
3.88
1.60
1.84
2.37
3.41
IRR-PWC
Adv e m zp
35.38
35.40
35.43
35.47
35.52
35.40
35.42
35.47
35.52
35.30
35.42
35.48
35.57
IRR-PWO
Adv e = zr
35.74
35.75
35.77
35.80
35.84
35.75
35.77
35.50
35.83
35.75
35.77
35.81
35.87
RAFT
Baseline
0..64
12.014
40.70
1220.51
3.16
11.34
43.72
1224.46
8:13
15.45
24.92
RAFT
Adv -
1.81
3.50
7.63
21.94
19. 12
3.10
7,89
21.29
19.72
5.82
9.02
13.89
20.02
RAFT Adv 205
1.99
3.70
7.88
22.19
48.
3.60
7.78
21.38
48.55
5.84
9.04
14.10
20.07
41

---

Published in Transactions on Machine Learning Research AAugust/2025)
H.1 Adversarial Training
In Table report results from subset of methods adversarially finetuned (=100k iterations with starting learning rate=10-6 and same learning rate scheduler as used by the method during training) on the KITTI2015 training dataset using -oorm constrained 3 iterations PGD attack with 6
and
2 255
a = 0.01 and attacked with -orm constrained CosPGD, PGD, and BIM attack with e E 255 and Number of Attack Iterations
255 255 253
[3, 5, 10, 20].
For finetuning, each mini-batch is adversarially attacked and used to finetune the model.
We observe that adversarial training does, in fact, improve the performance of all considered optical flow methods, but IRR-PWC, against adversarial attacks. For IRR-PWC, we observe significant drop in clean and adversarial performance when naively finet adversarially. For other methods, we observe a gain. This gain in performance is significant for RAFT while losing only a small amount of performance on clean data. However, other methods like Fast and FlowFormer, where there a gain in adversarial performance with adversarial finetuning, there is significant drop in clean performance.
Please note that in this work, image-wide white-box adversarial attacks have been used as probe for the reliability of the feature representations learned by the model when trained normally. While adversarial training is a known adversarial defense method, using it in this context defeats the purpose of using image- wide white-box adversarial attacks as probe for reliability.
H.2
Adversarial Attacks
Here we report additional results for all adversarial attacks.
H.2.1
FGSM Attack
2015 POSM Ustargeted
KITTI POSM
EBE W.S. Grousd Truth
15.0
7.5
0.0
0.000
0.005 0,000 6.015 8,020 0.025 0.000
0.00
0.05
0.10
0,15 Epsiion
0,20
0.25
Model
Model
BowFoctse
LiteFloWeet
FlowForeer
LitelowNee
RAPLDELOW
C3AFT
RPKNet
-RAFT
FlowPormere
LioFoooNees
RPKNet
CSPlow
C2AA
LLA-Flow
Scopeflow
CSRlow
GMA
LLA-How
SoopeHoow
Flo
CMPlow
MaskFloweet
SKEToW
DICL Flow
GMFlew
NaskFlownet
SXFlow
CMPlowNet
MS
SToRflow
GMPloWNet
STURFlow
Vidwozlow
FlowDD
Flow1D
Figure 9: Evaluations for non-targeted FGSM attack under norm bound using the KITTI2015 dataset. The attack was optimized w.r.t. the ground truth predictions.
Here we report the evaluations using FGSM attack, both as targeted (both targets: d and -) and non- largeted attacks optimized under the -orm bound and the fg-norm bound. For formnorm bound, pertur- bation budget 255 while for bg -orm bound, perturbation budget
64
255
Attack evaluations include Fig. 9
Fig
Fig. 23
Fig.
12
Fig.
13
Fig.
Fig.
15
Fig.
16
Fig Fig.
17
Fig
Fig.
19
Fig.
20
ig
21
Fig. and
ig.
H.2.2
BIM Attack
Here we report the evaluations using BIM attack, both targeted (both targets: J and F) and non- targeted attacks optimized under the form bound and the Ez-norm bound over multiple attack iterations.
42

---

Published in Transactions on Machine Learning Research AAuuust/2225
K7TT-2015- FOSM Zeco-Forget-
KITTI-2015 - PGSM - Zero- Target
13P wett Ground Truth
ERT wet Turgel Flow
90
40
60
Egeilon
Epsilon
Modal
Model
RAPIDFOOM
LiteFlowNet
RAPIDELOw
CSrfow
EMAA
Scupeflow
CSPlow
GMA
Scopaffuw
Flow
G34Flow
Maskk Flownet
SK.Tow
DICL- Flow
GMFlow
MaskFlowmst
SSFFlow
RIP
PMPPlowNet
MS-RAATT
STaltTlow
DIP
GMPlowNet
STERPPow
PantFloweet
HI23
RAPT
Videoflow
Peot Flossset
HIDS
BAPT
Vidwo/low
FlowID
FlowID
Figure 10: Evaluations for targeted FGSM attack with target O under lx norm bound using the KITTI2015 dataset. The attack was optimized w.r.t. the ground truth predictions.
Target
Target
EPP wrr Dround Truth
EPE M.et. Tampet Floot
52
188
0.50
0.05
0.10
0:15
0.20
0.25
0.05
0:10
0.15
0.20
0.25
Egailon
Epsilon
Model
CCMR
HowPeeme
RAPIDfIOw
CCMB
Howlomer
LitePoowNet
-.. RAPINP
CRAPT
CHAPT
LiaelowNath
RPKNet
CSTlow
EMA
LLA-flow
Stopeflow
CSPlow
GMA
LLA-flow
SoapeNlow
SKTTow
MarkTloweet
MSS-RAPT
Slatt/ow
GMFlewNNe
NS-RAPT4
StaitFlow
Pastilounnt
HDD
RAFT
Videoflow
PhatFlowmet
HDD
RAPT
Video
FlowDD
HRR
Mine1D
TIR
Figure 11: Evaluations for targeted FGSM attack with target O under 4--norm bound using the KITT12015 dataset. The attack was optimized w.r.t. the ground truth predictions.
For form bound, perturbation budget
255 and step size a=0.01, while for f2-norm bound, pertur-
bation budget
64
and step size 0=1.1. Attack evaluations include Fig.
24
Fig.
25
Fig.
26
Fig. 27
Fig. 28
Fig.
29
Fig.
Fig.
31
Fig.
32
Fig.
33
Fig.
Fig. 35
Fig.
36
Fig.
37
and Fig.
38
H.2.3
PGD Attack
Here report the evaluations using PGD attack, both as targeted (both targets: 0 and 7 and non- targeted attacks optimized under the lo -orm bound and the fg-norm bound over multiple attack iterations. For formnorm bound, perturbat ion budget = 255 and step size o=0.01, while for f2-norm bound, pertur- bation budget
and step size a=1.1. Attack evaluations include Fig, 39
Fig.
40
Fig.
Fig.
42
255
Fig. 43
Fig.
44 Fig.
Fig.
Fig. Fig. 48 Fig.
Fig. 50 Fig. 51
Fig. and
53
43

---

Published in Transactions on Machine Learning Research (August/2025)
XITI 2015 FOSM Neçative Target L.
KI171-2015 FCSM Negative Target
BPU wizt Sround Truth
Vlow
1.00
40
0.025
0.100 8.000 0.030
Epsilon
Model
Model
CRAPT
DUKNeL
CHAPT
HPKNet
CSslow
COSA
LLA-Flow
ScopeFlow
CSF3ow
GMA
LLA, How
ScopeHoow
BlCI-Plow
RMPlow
5K7Tmw
DICI-FFlw
GMPlow
...
Markflownet
SXPPlw
GMMPloWN
MS-RAPPT
STall/oow
DIP
GMPlewNe:
MS-BAPTH
StaitPloo
PastFlomnet
HDD3
RAFT
VideoFlow
RusFlloneet
SEDO
RAPT
Videoflow
Figure 12: Evaluations for targeted FGSM attack with target - under -orm bound using the KITTI2015 dataset. The attack was optimized w.r.t. the ground truth predictions.
KITTI 2015 PGSM Negative Target Lg
XITII 2015 PGSM Nogatiie-Tarpe
Enng wet Oovare fruth.
E7F wet Target Finw
90
50
Epsilon
Upsilen
Modal
Model
RAPIDHlww
RAPIDElan
CRrlow
EMA
Senpefloow
CSNlow
GMA
Soopaflow
Flo
224Howw
Sk.low
DiCL-Plow
GMPlow
Mookflownet
SSEFlow
DIP
CMPloowNet
MS SAFT+
STaRTTow
DIP
GMPloWNet
MS RAPT+
STARPlow
Paetfflwent
RAPP
Videoflow
RAPT
Vidwo
FloIDD
FlowID
Figure 13: Evaluations for targeted FGSM attack with target - under fz-norm bound using the KITTI2015 dataset. The attack was optimized w.r.t. the ground truth predictions.
Slntel-Clean FGSM Untargeted
L.
Siatel Clean FOSM Untargeted L
3.5
1222 WLEL. Ground Bryth
3.0
EPR wert Ground
2.5
2.0
1.5
1.0
0.5
Fpelion
Epsilen
Model
Model
CCMR
Rowhhemers
RAPIDIOO
CCMR
Howformre
RAPInMlow
ScopePlow
MoskFlomnet
Soope
HD3
MS-RAITT+
SKiTow
DIP
ND3
MS-HART+
SX0flow
Pastfimmnnt
PWITNW
Pood flownee
STaitPiow
FlowID
LiboFlowhezz
RAFT
Vidoo
FlowID
Libe PlowNee2
RAFT
Video
Thonformer
Flowformer
Figure 14: Evaluations for non-targeted FGSM attack under fx-norm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
44

---

Published in Transactions on Machine Learning Research (Aguust/222))
Sactel Clean FOSM Zero Target
Sintel Clean FGSM Target
1222 WEE Ground Trath
wet. Targeat Flow
0.000
Model
Model
DOCL/Tlow
GMPlow
MaskFloomeet
StopePlow
DCCL- Flow
GMPlow
MaskFlowmet
Scapeflow
DIP
HID3
MS-RAFTT+
SK7Tow
DIP
MDD
NSC HART++
SXPflw
PasEllwastt
LiteFloNNet
PWCNNe
SToRFTow
Fustllowett
Lite PloWeet
PACNe
STURPlow
FlowID
LiteBloWeez
RAFT
VideoFlow
FlowID
LiteBloNNerr
RAPT
Video7low
Figure 15: Evaluations for targeted FGSM attack with target o under -orm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
Zerc-T Zare-Tenga--
Sintel-Clein- PGSM Zeru- Target
2.5
BPE Ground
worL Turgut
2.0
1.5
12.0
0.5
11.5
0.90
0.05
0.30
0:15
0.20
8.25
0.05
0:10
0.15
0.20
0.22
Egailon
Epsilon
Model
Medel
COMR
LiNeS
RAPIDROM
CCMR
FloFoomeet
LieelooNec
RAPIDELOW
CRAPP
GMA
FLw-Flow
CRAPT
GMA
LLA-How
RPKNet
ScopeFlow
MaskFloweet
SKETow
Pastifoweet
LiteblowNas
PWINNw
Slatt/oww
Pasd
PWCCNN
Slaitbloo
FlowID
LitaFloNNez2
RAPT
ViduoFlow
Flow1D
Liteee
RAFT
VideoSlow
PlnwPormar
Figure 16: Evaluations for targeted FGSM attack with target Of under 4--nrm bound using the MP1 Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
Statel Clean PGSM Negotive
Sintel Clean PGSM Nega Tarpet 50
Ground Trushh
EPP WeL. Torget Fiom
EPP
0.000 0.005 6,000 6.035 8,020 0.025 0.000 Lppalon
0.000 0.005 0,010 0.015 0,020 0.025 0.030 Üpstion
FowFormeeee
LitelooNees
LibeFooNNed
GRAFT
EMA
LLA-Flow
BOKNet
CRAFT
GMA
LLA-Tlow
RPKNot
RMPiow
Scnpafllow
DICL-Plow
GMPlow
Markflownet
ScopeHfow
DIP
HD3
MS RAFT+
SKSTow
DIP
BID3
NS RAET+
SXFlow
Pastflomnnt
LibeBlowNet
PWCNet
STARTTow
LiteBloweeet
PWIGNet
STERBllw
Vidwo71mm
Figure 17: Evaluations for targeted FGSM attack with target 7 under norm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
45

---

Published in Transactions on Machine Learning Research AAuuust22225
Sintal-Clenn POSM- Nagatico-Targee 2
Sintal-Clenn- POSM Nagutisn-Tory
3.0
27.0
EP5 Ground
26.0
2.0
25.0
1.0
24.0
0.05
8:15
8.55
0.15
Epsilon
Model
Model
LiteFlowNets
FlowForeert
LtceoowNed
CRAPT
LLA-Flow
DOKNet
CHAPT
GMA
DICL
G34 low
NaskFlowatt
Scopeflow
DICL Flow
GMFlow
MankFlomnst
ScopeHlow
RIP
MS-RAFTT
5K7Tow
DIP
NXc MS-HARFT
SXPPlw
PWENN
STatt/oww
Flounee
PWCCNE
StaitPloo
Vidsoflow
HlowDD
Vidws2low
Figure 18: Evaluations for targeted FGSM attack with target - under 4--norm bound using the MP1 Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintal-Final- FCSM Untargated
L.
Sintal-Final-Fon FCSM Untarquted
4.0
12212 WEE. Ground Truth
wet Ground
3.0
2.0
1.0
0.5
0,000 0.005
6.015 0.025 0.00U
0.25
Model
Model
Fow
LiNeFooNeet
LbellowNed
BEKNNt
DSCL Flow
CME Low
MaskFloowe Flownet
ScopeFlow
DCCL Flow
GMFlow
MaskFlowmet
Scope
DiP
HUL
MS-RAPTS
5k.fow
DUP
HUS
NS HAPTH
SS.Plow
PastFlomoet
LiteFlowNet
PWCNet
STattTow
FlatFlowast
LitaFeeWet
PWCNN
STURPlow
LitaFlowNet2
RAPT
VidcoFow
FlowID
RAFT
Videaflow
Figure 19: Evaluations for non-targeted FGSM attack under -oom bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintal-Findl FOSM
Sintel-Fina FDSM Zero-Turged
1F2 WIZ. Ground Trunh
EIRE w.r. Thrgel Flow
16
0.000 0.055 0,000 6.015 8,020 0.025 0.000
0.000 0.D0S 0,CLO 0,015 0.520 0.025 2.050
Epaiion
RAPIDHHO
CEMI
FlowPhmmrr
LiteblowNaat
RAPLDFlaw
CSAPT
C3AA
LLA Flow
REKNat
CRAFT
GMA
LLA How
RPKNet
DCCL Flow
CMBLow
MishFlowmet
ScopePlow
DICL Flow
GMPlw
NaskFloweet
Soope
MS- NS-RAPPT
Skifow
DiP
HIS
MS MS-BAPF
SKFFlow
PWCNNe
STulslow
PWCNN
STuRFlow
FlowID
LiteFlowNez2
RAFT
VidcoFoww
FlowID
RAFT
VidooFlow
Flooformen
PluwPormer
Figure 20: Evaluations for targeted FGSM attack with target O' under fx-norm bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
46

---

Published in Transactions on Machine Learning Research (Aguust/2025)
Sintol Final PGSM Zeco Target Eg
Sintel Pinal FCSM Zero Target
13.4
32.0
12.8
122
0.5
12.0
Epsilon
Epsilon
Model
Model
DSCL/Tlow
GMPlow
MaskFloomet
StogePlow
DDCL- Flow
GMPlow
NaskFloweet
Scopeflow
DIP
HD3
-..
KS SAPT
SK.Tow
DIP
HLD3
NS RAFT
SXFFlo
PosEFlonnet
LitelooNeet
PWCNNt
SToRSTow
FustFloweet
LitePloNNe
PACNE
STURPlow
FlowID
LitelooNeez
RAFT
VideoFlow
FlowID
RAPT
Video7llw
FlowFormser
Figure 21: Evaluations for targeted FGSM attack with target 7 under 4--norm bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
Negative Turget
Nogative Target
LEPT WEL Grouad Truth
EPE Wet. Target Flome
-44
0.000 0.005 0,030 0.015 8.020 0.025 0.030 Epoilon
0.000 0.005 0.010 0.015 0.520 0.025 0.030
Epstion
Model
Model
CCMR
RAPIDiTOw
CCMIR
RAPINP
CRAFT
CMA
LLA-FloW Flow
REKNet
CRAFT
GMA
LLA Rowe
RPKNet
DRCL
EMPlow
Maskiloomet
Stopefloww
DICL-
GMPlow
MarkTloweet
SoopeNlow
SKTTow
FostFlommet
LiteFlowNee
STaRFlow
STaRFlow
FlowID
LiheBlowNee2
RAIT
Videoflow
BlowiD
LiteBoowNen
RAPT
Video
FlowFortner
FlowEarmer
Figure 22: Evaluations for targeted FGSM attack with target 7 under norm bound using the MPI Sintel (final) dataset. The allack was optimized w.r.t. the ground truth predictions.
Sintel Fhnal PGGM Negative
Sintel Itnal PGSM Negattve
Turget Flom
26.5
2.5
2.0
25.5
1.0
0.5
0.00
0.05
0.30
0.15
0.20
0.25
0.00
0,05
0.10
0.35
0,20
0.25
Lipallon
Upstlon
RAPIDHHOw
LitabloNNat
RAPLDSlaw
CRAFT
EMA
LLA-Flow
BOKNet
CRAFT
GMA
LLA-Hlow
RPKNot
BlG
PMPiow
Maskiimonet
Senpalllow
DICL--Poo
GMPlow
Narkflownet
Seopwlfow
HI2
MS MS-RAPTS
Skilow
DCP
HUS
MS-BAPF4
SKFFlow
Pastiloonnt
LibeBlowoNet
PWCNet
STARTTow
LiteBloweett
PWICNet
STERBllw
Figure 23: Evaluations for targeted FGSM attack with target - under fe-norm bound using the MPI Sintel (fina)) dataset. The attack was optimized w.r.t. the ground truth predictions.
47

---

Published in Transactions on Machine Learning Research Augusst/225))
KITTI-2015 Untargeted
KITTI-2015 BIM Untargeted
E993 wxrt. (rrondd Treeth
EP9 B99 Ground Trech
220
1.50
1.00
0.0
2.5
5.0
7.5
10.0
12.5
150
17.5 20.0
0.0
255 50
12.5 15.0 17.5 25.0 Nom. of Mttckk Berations
7.5
10.0
Num, of Attoci Ieeations
Model
Model
RAPIDHfow
RAPInFlor
LiteFlowNee3
CRAFT
LiselowNect
CRTlow
EMA
LLA-flow
Sengefflow
GMA
Sospaffom
DSCL Flow
CMPlow
MaskFloowet
SKSTow
DICL Flow
GMPlow
MaskFlommet
SXFlow
DIP
GMPlowNet
MS SAFT+
STaRTTow
DIP
GMPloWNet
MS RAFT+
STaRPlow
Paetiflmmet
RAPT
Videoflow
RAPT
Vidwo7low
FloNID
FlowID
Figure 24: Evaluations for non-targeted BIM attack under fo-m bound using the KITTI2015 dataset.
The attack was optimized w.r.t. the ground truth predictions.
KITTI-2015 5 BIM Zero Target L.
K7TT1-2015
BIM Zero Target
2
EPB Mec Ground Truth
80
60
w.r. Turgel
220
80
20
30
1.0
00
17.5
12.5
17.5
Num. of Attack Nerations
Num. of Attack Berations
Modal
RAPIDHOWw
CUMR
LiteloweNa
RAPLDF
CSAPT
RowFecmsre
ReKNat
CRAFT
Bowforere
LiteFloNNat
RPKNet
CSTlow
PMA
LLA-Tlow
Scnpwlllw
CSPlow
GMA
L1A-Hlm
Soopwffow
FlC
GMM
MaskFloomet
Skilow
DiCL-Flow
GMFlew
Mankfloweet
SRFFlow
GMFlowNet
STutclow
GMFlewNet
STuRFlow
FlolDD
Mow1D
DIII
Figure 25: Evaluations for targeted BIM attack with target O' under -oom bound using the KITTI2015 dataset. The attack was optimized w.r.t. the ground truth predictions.
2015 Target L2
Target 4
wel. Target Flon
EPI wit. Ground Truth
12
2.5
5.0
7.5
10.
12.5
15.0
17.5 20.0
2.5
2.5
10.0
12.5
12.5 20.0
Num. of Attackk Hecctions
Nom. of Allack Barations
Model
Model
CRAFT
ROKNet
CRAPT
BPKNet
CSTlow
CMA
Flow
ScopeFlow
-SPlow
GMA
LLA Flose
Soope
DOCL-Tlow
EMPlow
-4- Maskiioomet
SKTTow
DICL- Flow
GMPlow
NorkTlownet
SXPlow
DIP
GMElowNet
MS SAPT
STaRFlow
DIP
GMFleWNet
NS RAFT+
STLRELO
FastFlonmet
HD3
RAFT
VidcoFoow
FustFloweet
SED3
RAFT
Video.7ow
Figure 26: Evaluations for largeted BIM attack with target O' under f2-norm bound using the KITTI2015 dataset. The attack was optimized w.r.t. the ground truth predictions.
48

---

Published in Transactions on Machine Learning Research (August/2025)
KITTI-2015 Negative Target
KITTI-2015 Negative Target L.
EPE wist. Thapeat Flow
120
60
40
0.0
2.5
5.0
75
10.0
12.5
15.0
17.5 20.0
0.0
2.5
7.5 10.0 12.5 150 17.5 25.0 Nom. of Attack Iterations
50
Num, of Attocis Ieeations
Model
Model
RowForter
Litelowheet
RAPIDROM
FlowForee
LibeFloNeet
RAPIDELOW
LiselowNect
CSSlow
ScopeFlow
C3Flow
ScopeHlww
DSCL Flow
CMPlow NoM
MishFFooweet
SKFlow
DICL Flow
GMFOOW GMPleWNeL
MaskFlowmet
SKFlow
DiP
MS--APTS
STatt/oow
DUP
NS HAPT+
StaitPow
PustElummet
HD3
RAPT
VideoFoww
FustFlowss
HLDS
RAFT
VideoFlow
FlowID
Flow1D
Figure 27: Evaluations for targeted BIM attack with target 7 under fo-norm bound using the KITTI2015 dataset. The attack was optimized w.r.t. the ground truth predictions.
KITTI-2015 BIM Negative Target
42
KTTTI 2015 BIM Negative Target L2
72.5
Wi2L. Ground
87.5
68.0
62.5
60.0
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
0.0
2.5
5.0
7.5
10,0
12.5
15.0
17.5 20.0
Attack Beeratonn
Model
Model
CCMR
RowFortee
LiteflowNet
RAPIDHOM
CCCR
Flowfomer FlonFomeeT
LiteFlowNet
RAPIDeluw
RowForteee
B2KNet
CRAFT
LiteFloNNet
CSElow
LLA
ScopePlow
Soope
DiC
MaskFlownet
SK3Tow
DUCL-Flow
GMFlew
MuskFlownat
DIP
CMFlooNNe
MS SAPT
STaRSlow
DIP
GMFloWNee
NS RAFT
STURFlOw
PostFlomnet
HD3
RAFT
Vidco
Fustllwwett
HED3
RAFT
Video
FlawID
BUR
FlowID
LII
Figure 28: Evaluations for targeted BIM attack with target - under 42-norm bound using the KITT12015 dataset. The attack was optimized w.r.t, the ground truth predictions.
Sintel- Clean BIM Untargeted
Lx
Sintel-Cless BIM Untarpeted La
BPC wet. Ground Truits
wirt Ground
00
0.6
2.5
5.0
7.5
10,0
12.5
15.0
17.5 20,0
0.0
2.5
7.5
16,0
12,5
15.0 17.5 20.0
Iterntiion
Mttck
Model
Model
COMR
BowEsrnetet
LiteFWowNeet
RAPIDROM
CCMR
HloFormert
LibeFloNNec
RAPIDELWW
CHAPP
GMA
LLA-Flow
CHAPP
GMA
LLA-How
RPKNNt
NaskFlowaa
ScopeFlow
MarkFlownet
ScopeHlow
HD3
MS
SKETow
HDS
Pastrfoweet
LiteblowNet
PWENNt
Stat/fow
Pes Flownne
PANGNN
Siaittllow
FlowID
LiteFlowNet2
RAPT
VideoHlow
Plow
Line
RAFT
Vidwoslow
FlawEumee
FlowEormee
Figure 29: Evaluations for non-targeted BIM attack under -orm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
49

---

Published in Transactions on Machine Learning Research (lngust/2025),
Sintel-Clean BIM Zero Target-
EPP wett Target Elow
1.0
12
0.0
2.5
5.0
7.5 108 12.5
15.9 17.5 20.0
Recations
Model
FlowForneee
RAPIDFIOW
LLV-flow
ScopeFlow
HDD
MS-RAPT+
PastFlownet
SowNe
PWCNNt
Slaikflow
FlowED
LineFkoNe122
RAPT
VideoFlow
Plon/Pummrr
Figure 30: Evaluations for targeted BIM attack with target 0' under -orm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintel-Clean BDM Zero Target L
1P2 wat.. Tagget Tlw
122
Num. Attack lberations
CCMR
FlowFormeee
RAPIDFIOW
CRAFT
RMA
LLA-ttom
RPKN
DICL-Tlow
BMPlow
Mackflownet
Soupetfor
HDO
RAFT+
SKFFow
FlowtD
LibeFSoWNe12
RAFT
VidcoFlow
FlowFormer
Figure 31: Evaluations for targeted BIM attack with target O' under fg-norm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
Nogative Target
UPE wett Target Dlw
15
20
15
0.0
2.5
5.0
15.0 Num. of Attack Recations
7.5
10.0
12.5
17.5 20,0
Modal
FlowFormeee
NoteFooWNet
RPXNot
DICL-VlOw
MaekFlownet
Scape
DIP
HDO
MS-RAFT4
SxF3oo
FistFlownet
LiteFRoNeet
PWCNet
MowiD
RAPT
Vidlecflow
Figure 32: Evaluations for targeted BIM attack with target - under loc norm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
50

---

Published in Transactions on Machine Learning Research (August/2025)
Siatel-Clean BIM Negative Target-
20
224
23
22
0.0
2.5
5.0 7.5
10.6
12.5
15.0
17.5 20.0
Num. of Attack Itecations
Model
FlowForeeee
RAPIDFIOW
DICI-Plow
Mankflownet
Sompe Piom
HDO
MS RAFT+
SXF3oo
PastFipmnnt
LiteFloNNet
PWCNet
5Th.RFlow
FlawED
LineFooNe122
RAPT
VideoFlow
FlowFormer
Figure 33: Evaluations for targeted BIM attack with target -f under 4z-norm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintel-Final Untargeted
Sintel-Final BIM Untargeted
ESPE wet Gronnt Truth
60
80
(ert
100
0.0
2.5
5.0
75
10c
12.5
15.0
175 20.0
0.0
255 s. 75
10. 12.8 15.0 17.5 20.0 Num. Berations
Num, lerations
Model
Model
CCMR
Rowfhemerce
LiteloowNees
RAPIDIOOM
CCMR
Howforert
RAPIDELOn
DiC
MaokFloontt
Scupellow
DICL-Flow
MankFlownet
SeopaHfuw
HD3
MS-BAFT+
SKTTow
DIP
MDD
2S--ARTT+
SXPioo
Pastiimment
PWINNt
Paud Floweee
STaRtPlow
FlowID
LitoPloWNec2
RAPT
VidcoFoow
FlowID
RAFT
VideoFlow
Plonformer
FlowFormer
Figure 34: Evaluations for non-targeted BIM attack under -oom bound using the MPI Sintel (final)
dataset. The attack was optimized w.r.t. the ground truth predictions.
Statel-Final BIM Zero Target-
Sintel-Final BIM Zero Target L.
20.0
Crouse
15
17.5
30
12.5
10.0
2.5
5.0
7.5
10.8
12.5
15.0
17.5 20.0
2.5
2.5 10.0 12.5
12.5
Num. of Attack Becutinns
Num. of Allack Barations
Model
Model
EMA
LLA-foow
ROKNet
CHAPT
GMA
BPKNet
DOCL Flow
CMPlow
MaskFloonet
ScopeFlow
DCCL Flow
GMPlow
MaskFlomeet
Soope
DIP
HD3
MS MS-RAATT+
SK.Tow
DIP
RDD3
NS RART+
SXPlow
PWINNW
STattToww
Rood Flouree
FlowDD
LibaFlowNett
RAFT
VidcoFlow
FlowiD
RAFT
Video
Figure 35: Evaluations for targeted BIM attack with target O' under -orm bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
51

---

Published in Transactions on Machine Learning Research (Aguust/2225
Sintel-Final Zero Target L2
Sintal-Final BIM Zero Target
EP2 WEt Tarpet Blow
EFE Ground
1.0
0.0
2.5
5.0
75
10.
12.
15.0
17.5 20.0
0.0
2.5
5.0
7.5
10.0
12.5
15.0 17.5 20.0
Attack Becations
Num. Attack Berations
Model
Model
CCMB
RAPIDIfow
CCMB
Mowhammens
LitefoowNell
RAPINelo
How
Flow
ScopeFlow
DICL
GMFlow
MaskFlownet
Scope
HB3
MS--RAPT
Skrfow
HDS
NS-HAATT+
PastFlowset
Liteblowiit
PWCNN
Shatt/oow
Past Flowaae.
PWCNNe
StaitPlow
FlowID
LieelowwNee2
RAPT
Vidsofloww
PlowID
RAPT
Video/Toww
Flooffrme
Plowformer
Figure 36: Evaluations for targeted BIM attack with target 0' under f2-norm bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintol- Pinlll-inal--a BIM Negative Target
Sintel-Pina BIM Negative Target-
E70 wett Target
30
Ground
2.5
25
20
15
12.5
17.5
Num. of Attack Nerations
Num. of Attack Berations
RAPIDHHOw
CUMR
CSAPT
C3AA
Flow
ReKNat
CRAFT
GMA
LLA How
RPKNet
DCCL Flow
CMPlow
MishFlownet
ScopeFlow
DICL Flow
GMPRow
MaskFFoweet
Scope
Hil
MS-RAPTS
Skifow
DEP
HUS
NS-BAPTH
SKPFoo
PWCNNet
STutclow
PWCNE
STuRFFow
FlowID
LiteFloNNc22
RAFT
VidcoFoow
Flow1D
LiteBewNe22
RAFT
VidooFlow
Flosforme
PlumPormer
Figure 37: Evaluations for largeted BIM atlack with target - under lec-norm bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintel Final
Negative Target 42
Final Negative Target 4
27
Target Elow
EPE wULL Ground
25
24
23
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
0.0
2.5
15.0 17.5 20.0 Num. of Mttack Beeations
5.0
7.5
10.0
12.5
Num. of Attacis Ieerations
Model
Model
RowForteres
LiueSooNNet
RAPIDHOM
LitaFlowNett
RAPIDELOW
CMA
BEKNet
RPKNet
DOCL Flow
CMPLow
MaskFlooeet
ScopePlow
DDCL Flow
GMPlow
MaskFlommet
Soope
DiP
HD3
M55- SAPTS
SK.fow
DUP
HDS
NS RAFTE
SxFFlow
RastFlomaet
LiteFloNNet
PWCNet
STaRSlow
FustFlowest
LiteFeeWeet
PACNN
STLRELW
FlowDD
LitaFlowNet2
RAFT
VidcoFloww
FlowiD
RAFT
Video.low
Figure 38: Evaluations for targeted BIM attack with target F under E--nrrm bound using the MPT Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
52

---

Published in Transactions on Machine Learning Research AAguust22255
KITTI-2015 Untargeted
KITTI-2015 Untargeted L
E992 wrtt Ground Tredt
600
E99 B995 Ground Treath
300
1.00
0.0
2.5
5.0
75
10.0
12.5
15.0
17.5 20.0
0.0
255
12.5 150 17.5 25.0 Nom. of Mttcck Itrrtions
5.
7.5
10.0
Num, Attock Rerations
Model
Model
RAPIDHfow
RAPInFlor
LiteFloNNeeS
LiselowNest
CRTlow
EMA
LLA-flow
Sengefflow
GMA
Sospaffom
DSCL Flow
CMPlow
MiskFlooweet
SKSTow
DICL Flow
GMPlow
MaskFlommet
SXFlow
DIP
GMPlowNet
MS SAFT+
STaRTTow
DIP
GMPloWNet
MS RAFT+
STaRPlow
Paetiflmmet
RAPT
Videoflow
RAPT
Vidwo7low
FloNDD
FlowID
Figure 39: Evaluations for non-targeted PGD attack under -orm bound using the KITTI2015 dataset.
The attack was optimized w.r.t. the ground truth predictions.
KrT'I--015 5- PGD Zero Target L.-
KITTI-2015 PGD Zero Target L.
30
I2P wet Target Blow
80
Ground
:20
1.0
Num. of Attaacck Nerations
Num. of Attack Berations
Madel
Model
Nowe
RAPIDHOWw
CUMR
LiteloweNa
CSAPT
RowFectsrre
LinessoWNets
ReKNat
CRAFT
Bowforere
LiteFloWNact
RPKNet
CSTlow
PMA
LLA-flow
Scnpwlllo
CSPlow
GMA
L1A-Hlm
Soopwffow
FlC
GMM
MaskFloomet
Skilow
DiCL-Flow
GMFlew
Mankfloweet
SRFFlow
GMPlowNet
STatclow
GMFlewNet
STuRFlow
FlolDD
Mow1D
DIII
Figure 40: Evaluations for targeted PGD attack with target T under -oom bound using the KITTI2015 dataset. The attack was optimized w.r.t. the ground truth predictions.
2015
Target 42
2015
Target L2
wet Target 2llww
30
w.L.. Ground
33
2.5
15.0 Num. of Attack Hecutions
5.0
7.5
10.
12.5
175 20.0
2.5
7.5
10.0
12.5
12.5 20.0
Num. of Allack Barations
Model
Model
GRAPT
RKKNet
CRAPT
BPKNet
CSTlow
CMA
Flow
ScopeFlow
-SPlow
GMA
LLA Flose
Soope
DOCL-Tlow
EMPlow
-4- Maskiiowmet
SKTTow
DICL-
GMPlow
-...
NorkTlownet
SXPlow
DIP
GMElowNet
MS SAPT
STaRFlow
DIP
GMFleWNet
NS RAFT+
STLRELO
FastFlommet
HD3
RAFT
VidcoFoow
FustFloweet
SED3
RAFT
Video.7lw
Figure 41: Evaluations for targeted PGD altack with target d' under Ez-norm bound using the KITTI2015 dataset. The attack was optimized w.r.t, the ground truth predictions.
53

---

Published in Transactions on Machine Learning Research (August/2025)
KITTL-2015 Negative Target Lx
XITTI-2015 Negative Target- L.
wiet T..ppet Flow.
820
80
40
EPE
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
0.0
2.5
5. 7.5 10.0 12.5 15.0 17.5 25.0 Nom. of Attack Iterations
Num. of Attoris Iteraioons
Model
Model
RowForter
Lieelowheet
RAPIDROM
FlowForee
LibeFloweet
RAPIDELOW
LiselowNect
CSElow
ScopeFlow
C3Flow
ScopeHoww
DSCL Flow
CMPlow
MishFFooweet
SKFlow
DICL Flow
GMFOOW GMPleWNeL
MaskFlowmet
SKFlow
DiP
MS--APTS
STatt/oow
DUP
NS HAPT+
StaitPow
PustElummet
HD3
RAPT
VideoFoww
FustFoowss
HLDS
RAFT
VideoFlow
FlowID
Flow1D
Figure 42: Evaluations for targeted PGD attack with target 7 under &x-norm bound using the KITTI2015 dataset. The attack was optimized w.r.t. the ground truth predictions.
KITTI 2015 PGD Negative Target 42
KITTI 2015 PGD Negative Target 42
72.5
WIZL Ground
67.5
68.0
62.5
60.0
0.0
2.5
5.0
7.5
10,0
12.5
15.0
17.5 20.0
0.0
2.5
5.0
7.5
10,0
12.5
15.0
17.5 20.0
Attack Beeratonn
Model
Model
COMR
RowFormee
LiteflowNet
RAPIDHOM
CCCR
Flowfomer FlonFomeeT
LiteFlowNet
RAPIDelaw
RowForteee
LineSwwNeet
B2KNet
CRAFT
LiteFloNNet
CSElow
LLA
ScopePlow
Soope
DiC
MaskFlownet
SK3Tow
DUCL-Flow
GMFlew
MuskFlownat
DIP
CMFlooNee
MS SAPT
STaRSlow
DIP
GMFloWNee
NS RAFT
STURFlOw
PosiFlomnet
HD3
RAFT
VidcoFFow
Fustllwwett
HED3
RAFT
Video
FlawID
BUR
Flow1D
LII
Figure 43: Evaluations for targeted PGD attack with target - under f2-norm bound using the KITTI2015 dataset. The attack was optimized w.r.t, the ground truth predictions.
Sintel- Maaell-Clenn--e PGD
Untargeted L.
Sintel Cleen---lnn--ee PGD Untargeted
400
3P0 wet.. Dround Trotb
wet. Ground Trotb
200
1.00
0.6
2.5
5.0
7.5
10,0
12.5
15.0
17.5 20,0
0.0
2.5
7.5
16,0
12,5
15.0
17.5 20.0
Iterations
Mttckk
Model
Model
COMR
BowEsrnete
RAPIDROM
CCMR
FloFoomeet
LibeFloNNec
RAPIDELWW
CHAPP
GMA
LLA-Flow
CHAPP
GMA
LLA-How
RPKNet
MaskFlowaa
ScopeFlow
MarkFlownet
ScopeHlow
HD3
MS
SKETow
HDS
Pastrfoweet
LiteblowNet
PWENNt
Stat/Tow
Pes Flownee
PANGNN
Siaittllo
FlowID
LiteFlowNet2
RAPT
VideoHlow
Plow
Line
RAFT
Vidwoslow
FlawEumee
FlowEormee
Figure 44: Evaluations for non-targeted PGD attack under foo norm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
54

---

Published in Transactions on Machine Learning Research AAguust22255
Sintel-Clean PGD Zero Target-
Sintel-Clese PGD Zero Target-
15.0
west Target Fllow
12.5
LPP Grouns
10.0
0.0
2.5
3.0
75
10.8
12.5
15.0
17.5 20.0
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
Num. Atlack Berations
Berations
Model
Model
Lneelowheet
RAPIDROM
FloFoomeee
LieFoowNedd
RAPIDELOW
How
Flow
ScopeFlow
DICL
CMFlww
MaskFlownet
Scope
HB3
MS--RAPT
Skrfow
HDS
MS-HAAT+
PastFloweet
Liteblowist
PWUNNL
Shatt/oow
Past Flowaae
PWCNNe
StaitPoow
FlawID
LiNeFooNec2
RAPT
VideoFoww
FlowID
LiteRewNeiz
RAFT
VidesFlow
PlowPormer
Figure 45: Evaluations for targeted PGD attack with target O under -orm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintel-Clean - PGD Zero Target Lg
Sintel-Clean PCD Zero Target L
wist Thagge Slow
1.3
Webt Ground
12
1.1
7.5
Num. Attaci Iterations
Num. Attakk Berntions
COMR
RowEormere+
LiteFlowNNetS
RAPIDITIM
CCMR
RloFormeree
LibeFloWNed
RAPIDFIow
CRAPT
FMA
LLA-Tlow
ROKNet
CRAFT
GMA
LLA--lor Ploe
RPKNet
GMPlow
Scopelllow
DiCL-Flow
GMPlew
Maokfloweet
Seopaffow
HD3
MS BAFT+
SKFlow
HDD
SXPlow
FlowLD
LitelloNec22
RAFT
VidcoFoww
Flow1D
LiteBowNe22
RAFT
VidooFlow
FlowEorterr
FlowEormer
Figure 46: Evaluations for targeted PGD attack with target O under &n-nrrm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
Clean Negative Target
Clean Nogative Target
Tarpet Blow
Ground
155
E212
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
0.0
2.5
15.0 17.5 20.0 Num. of Mttack Berations
5.0
7.5
10.0
12.5
Num. of Attaria Herations
Modal
Model
RowForteree
LiuetooNeet
RAPIDHOM
LitaFlowNett
RAPIDELOW
CRAFT
BEKNet
CRAFT
RPENet
BiCL-Plow
Scopefloow
DICL-Plow
GMBlew
MaokFlownet
Scopatfuw
DIP
HD3
MS5 SAPTS
Sk.tow
DIP
HDS
NS RAFT
SKFFow
RastFlomaet
LiteFloNNet
PWCNet
STaRSlow
FustFlowss
PACNN
STLRFLO
FlovID
RAPT
Videotfiw
MoolD
DAPT
Vidwo
Figure 47: Evaluations for targeted PGD attack with target 3 under norm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
55

---

Published in Transactions on Machine Learning Research (August/2225
Sintel- Cretel-Cleen-eee Negative Target
Sintel-Cless Negative Target-
BPC wast Thapps Stow
22
0.0
2.5
5.0
7.
10.0
12.5
15.0
17.5 20.0
0.0
2.5
5. 7.5 10.0 12.5 15.0 17.5 25.0 Nom. of Attack Iterations
Num, Attoris Ieeations
Model
Model
Lieelowheet
RAPIDROM
FloFoomeee
LiteoowNeed
RAPIDELOW
DICL
Sengefflow
DICL-Flow
GMPllow
Mask Flownet
Sospaffow
DEP
HD3
MS SAFT+
SKFTow
DIP
HDD
MS RAFT
SXFiow
PastFlommnt
LitePlowNet
PWCNet
STaRTTow
PestFoowet
Lite PlowNee
PACNN
SThRPlow
FlawED
LiNeFooNeC2
RAPT
VideoFoww
FlowID
RAFT
VideoFlow
FluwEutter
FlowFormer
Figure 48: Evaluations for targeted PGD attack with target - under éy-norm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintel-Final PGD Untargeted
Sintel-Final PGD Untargeted
E392 wet Geouzt Trrth
600
EPP wErt Ground Truth
200
1.
0.0
2.5
5.0
75
10.
12.5
15.0
175 20.0
0.0
255 s. 75
16.0 12.5 15.0 17.5 20.0 Num. Alback Berations
Num, Attuck Berations
Model
Model
CCMR
Rowfhrmerse
RAPIDIOOM
CCMR
Howforert
LiellowNeet
RAPIDMLOn
DiC
MaokFloontt
ScupePlow
DICL-Flow
MankFlownet
SoopaHfuw
HD3
MS-BAFT+
SKTTow
DIP
MDD
PS--AAT+++
SXPfoo
Pastiimment
PWrNNt
Paod Floweee
STaRtPlow
FlowID
LitoPloWNec2
RAPT
VidcoFoww
FlowID
RAFT
VideoFlow
Plonformer
FlowFormer
Figure 49: Evaluations for non-targeted PGD attack under formorm bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintel-Final PGD Zero Target
Lx
Statel-Final PGD Zero Target L.
Cround t5
30.0
30
7.5
5.0
5.0
7.5
10.6
12.5
15.0
17. 20.0
2.5
2.5
10.0
12.5
12.5 20.0
Num. of Attack Hecctions
Numt of Allack Barations
Model
Model
GRAPT
EMA
LLA-foow
ROKNet
CHAPT
GMA
BPKNet
DOCL Flow
CMPlow
MaskFlownet
ScopeFlow
DCCL Flow
GMPlow
MaskFlomeet
Soope
DIP
HD3
MS MS-RAPTT
SKTTow
DIP
RDD3
NS RART+
SXPlow
PWITNW
STattToww
Rood flouree
PAACNN
FlowDD
LibaFlowNett2
RAFT
VidcoFlow
FlowiD
RAFT
Video
Figure 50: Evaluations for targeted PGD attack with target O' under norm bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
56

---

Published in Transactions on Machine Learning Research (Agusst/205))
Sintel-Final PGD Zero Target- lg
Sintel-Final PGD Zer Target L2
EFE M.L Ground Truth
WEt Tarpet Slow
13
1.2
111
10
0.0
2.5
5.0
75
10.
12.5
15.0
17.5 20.0
0.0
2.5
5.
7.5
10.0
12.5
15.0 17.5 20.0
Attack Berations
Num. Attack Berations
Model
Model
CCMB
RAPIDIfow
CUMB
Mowhammens
LitefoowNell
RAPInelo
How
Flow
ScopeFlow
DICL
GMFlww
MaskFlownet
Scope
HB3
MS--RAPT
Skrfow
HDS
NS-HAATT+
PastFlowset
Liteblowiit
PWCNNL
STatt/oow
Past Flowaae.
PWCNNe
StaitPlow
FlowID
LieelowwNee2
RAPT
Vidsofloww
PlowID
LJtnflowNes
RAFT
Video/Toww
Flooformer
Plowformer
Figure 51: Evaluations for targeted PGD attack with target O under f--norm bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintel-Final PGD Negative Target- L.
Sintel- Pinal PGD Negnative Target L.
Met Tarpet Elow
25
Ground
20
15
1.5
30
12.5
17.5
Num. Attaccck Nerations
Num. of Attack Berations
RAPIDHHOw
CUMR
RAPIDF
CSAPT
COSA
Flow
ReKNat
CRAFT
GMA
LLA How
RPKNet
DCCL Flow
CMPlow
MishFlownet
ScopeFlow
DICL Flow
GMPPow
MaskFFoweet
Soope
Hil
MS-RAPTS
Skifow
DEP
HUS
NS-BAPT4
SKPFoow
PWCNNet
STutclow
PWCNN
STuRFFow
FlowID
LiteFloNNc22
RAFT
VidcoFoow
Flow1D
LiteBewNe22
RAFT
VidooFlow
Flosforme
PlumPormer
Figure 52: Evaluations for largeted PGD atlack with larget - under norm bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintel Pinal
Negative Target 42
Final Nogative Target L2
watk Target Flow Flowe
26
ERR
22
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
0.0
2.5
15.0 17.5 20.0 Num. of Mttack Beeations
5.0
7.5
10.0
12.5
Num. of Attacia Ieerations
Model
Model
RowForneres
LitefooNNett
RAPIDHOM
LitaFlowNett
RAPIDELOW
CMA
BEKNet
RPKNet
DOCL Flow
CMPLow
MaskFlooeet
ScopePlow
DDCL Flow
GMPlow
MaskFlommet
Soope
DiP
HD3
MS5- SAPTS
SK.fow
DUP
HDS
NS RAFTY
SxFFlow
RastFlomaet
LiteFloNNet
PWCNet
STaRSlow
FustFlowest
LiteFeeWeet
PWCNN
STLRELW
FlowDD
LitaFlowNet2
RAFT
VidcoFlow
FlowiD
RAFT
Video5low
Figure 53: Evaluations for targeted PGD attack with target 3 under forn bound using the MPT Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
57

---

Published in Transactions on Machine Learning Research AAugust/222)
H.2.4 CosPGD Attack
KITTI-2015 CosPCD Untargeted Lx
KITTI-2015 CosPGD Untargeted -
6eP5 Groupd 1roth
EPPE EPEWwrt Ground (roth
800
200
Num Attaci
Iterations
Num. Atback Barations
COMR
RowFermer
LiteflowNee
RAPIDNO
-CCCR
HowPorner
LibelloNeet
RAPIDFIOw
CRAPT
Lite
ROKNet
-CRAIT
Howformer++
1it=PlowNedd
-4 RPKNes
CSElUw
GMA
LLA Flow
ScopeFlow
CSElow
GMA
LLA Flo
Scopeflow
DCCL/
GMPlow
MaskFlomeet
SKFlow
DICL
GMPlow
MaskFlomeet
SXPlow
RastFlomaet
HD3
RAFT
Vid:oFlow
FastFloweet
RAFT
VidooFlow
FlowID
IR3
FlowiD
IRR
Figure 54: Evaluations for non- non-targeted CosPGD attack under norm bound using the KITTI2015 dataset. The attack was optimized w.r.t. the ground truth predictions.
KETTI-2015 CosPGD Zero Target
KITTI-2015
CosPGD Zero Target -o
50
wstt Teagge Flow
Ground
00
220
12.5
2.5 Berations
12,5
19.5
OCMIt
HowFiemee
LowNe
RAPIDflOw
CCNB
... RAPRIFlow
CRAPT
ReKNet
CHAPT
Rowfomert
LiteFooNNtt
RPKNet
CSTlow
EMA
LLA-flow
Stngeflow
CSPlow
GMA
1LA-flow
Sospaflow
SKrlow
SKPloo
STaRFFow
GMPloWNet
SThRFlow
Pastifmment
RAIT
Videoffinw
MDD
RAPT
Vidwo/Toww
FlowDD
HUR
MlnwID
JIII
Figure 55: Evaluations for targeted CosPGD attack with target D' under forn bound using the KITTI2015 dataset. The attack was optimized w.r.t. the ground truth predictions.
Here we report the evaluations using CosPGD attack, both as targeted (both targets: 0 and -} and non-targeted altacks optimized under the form bound and the fg-norm bound over multiple atlack iterations. For form bound, perturbation budget c 255 and step size a=0.01, while for fg-norm bound, perturbation budget.
64 and step size a=0.1. Attack evaluations include Fig.
54
Fig.
55
Fig.
56
Fig.
57
Fig.
Fig.
59
Fig.
60
Fig.
61
Fig.
62
Fig.
Fig.
Fig.
65
Fig. 66
Fig.
67
and
Fig.
H.2.5
PCFA Attack
Here we report the evaluations using PCFA attack, targeted (both targets: D and -}) optimized under the fo-m bound over multiple attack iterations. Here the perturbation budget 0.05 and step size le
Attack evaluations include Fig. 69 and Fig. 70
H.2.6 Adversarial Weather Attack
Here we report the ovaluations using different Adversarial Weather, both as targeted (both targets: d' and
-) and non-targeted attacks. Attack evaluations include Fig.
71
Fig.
72
Fig.
73
and Fig. 74
58

---

Published in Transactions on Machine Learning Research (August/2025)
KITTI-2015 5 CosPGD Zero Target 12
KrrT1-2015 CosPGD Zero Target
wet Target Dlow
wED Ground
00
12.5
7.5
12.5
12.5
Num.
Attack Berations
Model
CCMI
Lowee
RAPIDITOw
... CUXI
Nowlarer
LitelowNeet
... RAPIDFIOn
CRAPT
CHAFF
Howfomar
LiteFooNatt
RPKNeL
CSTlow
DMA
LLA-flow
Scopefllow
CSPloo
GMA
L.A-Hoow
Seapeflow
SKiTow
Maskflownet
MS--RAPT
Statt/oow
GMFlewNet
HAPT4
StaitFlow
Pastifoumnt
RAIT
Videoflow
Floweet
MDD
RAPT
Video/7oww
FlowDD
BRR
Mlnw1D
JII
Figure 56: Evaluations for targeted CosPGD attack with target 0 under 4--norm bound using the KITTI2015 dataset. The attack optimized w.r.t. the ground truth predictions.
KITTI 2015 5 CosPGD Negative Target
KITTI-2015 CosPCD Negative Target
ERS west Torpet Slow
80
70
130
30
0.0
255
5.0
75
10.
12.5
15.0
175 20.0
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
Num, Atluck Becations
Num. Allack Barations
Model
Model
CCMR
Rowthemer
LitefooNNe
RAPIDITOw
CCMM
MowPomer
RAPINMlor
CRAFT
RowFormer++
LiteFooheett
R2KNet
FlonFomeee
LiteFooNeet
RPKNet
ScopePlow
SoopeFlow
DICL-Tlow
EMPlow
SKTTow
DICL-Flow
GMNllww
MaokTloweet
DIP
CMFlowNet
MS SAFT+
SToRSlow
DIP
CMFloWNee
NS RAFT+
STuRFlow
PosiFlommnt
HD3
RAFT
VideoFoow
FustFloweet
HDD
RAFT
Video7low
TlorDD
PlowID
TRR
Figure 57: Evaluations for targeted CosPGD attack with target =} under form bound using the KITTI2015 dataset. The attack was optimized w.r.t. the ground truth predictions.
H.3 Common Corruptions Overview
Following, we provide an overview of the performance over all corruptions. This is reported in Fig. 75
H.4
2D Common Corruptions
Following, we report additional results for 2D Common Corruptions.
H.4.1 Supi-Supe And Sele-Supervised optical Flow Methods
Following, we evaluate the generalization capabilities of semi-supervised Im et al. (2022) and self- supervised Huang al. (2023) optical flow methods, using 2D Common Corruptions. We take common architecture, RAFT, for the results to be comparable. We report results in Table We observe that pretraining on the AutoFlow Huang et al. (2023) dataset provides better gencralization capabilitics to the RAFT model, however, it lowers its performance on the clean data. Whereas, the semi-supervised met Flow Supervisor Im et al. (2022) provides better or comparable generalization to RAFT on all common corruptions except Snow and Frost, leading a worse mean performance against corruptions. Please note, we do not observe a significant difference in performance for AutoFlow checkpoints pretrained only on their dataset and the checkpoint finetuned for KITTI2015, except that the "only pretrained" checkpoint, slightly
59

---

Published in Transactions on Machine Learning Research (August/202))
KFTTI-2015 CosPGD Negative Target-
KITTI-2015 CosPGD Negative Target-
72.5
70.0
Ground
87.5
85.0
00.0
12.5
7.5
12.5
19.5
Attack Berations
Model
CCMI
Lowee
RAPIDiTOw
... CUXI
Nowlorer
LitelowNNet
... RAPIDFloo
CRAPT
CHAFF
Howwomar
LiteFooNatt
RPKNeL
CSTlow
DMA
LLA-flow
Scopefllow
CSPllo
GMA
L.A-flow
Seapeflow
SKiTow
Maskflownet
MS--RAPT
Statt/oow
GMFlewNet
StaitFlow
Pastifoumnt
RAIT
Videoflow
Flovmet
MDD
RAPT
Vidwo/7oww
FlowDD
BRR
Mlnw1D
Figure 58: Evaluations for targetod CosPGD attack with target 7 under 4--norm bound using the KITTI2015 dataset. The atlac was optimized w.r.t. the ground truth predictions.
Sintel Clean Sintel-Clean
CosPGD
Untargeted
-.
Sintel Clean CosPGD
Untargeted L2
EPP wor.. Greund Inath
EPE wont Greund Trath
800
200
12.5
12,5
Attack Beraatonns
Mttck Berations
Model
Model
COMR
HowFecmerre
LineslowNets
RAPIDHOW
... CCCR
BowFomert
LiteFlowNatt
RAPIDFlOw
CRAFT
CMA
LLA Flow
REKNet
CRAFT
GMA
LLA Row
RPENet
DCCL
GMPlow
MaskFlowmer
ScopeFlow
DSCL
MaskFlomeet
Soope
HD3
SKFlow
SSFlow
RustFlomset
LiteFloNNce
STaRFFlo
FusFFlowet
LiteBewNett
STuRFlow
FlowID
LitePloNNe22
RAFT
VidcoFoow
FlowiD
RAFT
VideoFlow
FlosfFormer
PlowFormer
Figure 59: Evaluations for non-targeted CosPGD attack under -orm bound using the MPI Sintel (clean)
dataset. The attack was optimized w.r.t. the ground truth predictions.
but significantly outperforms the checkpoint finetuned on KITTI2015. We suspect that the checkpoints provided are incorrect or mislabeled. Thus, we recommend caution when interpreting these results.
H.4.2
Benchmarking Results
Here we report evaluations using different 2D common corruptions over all considered datasets. OOD Robustness evaluations with 2D Common Corruptions include Fig.
76 Fig.
77
and Fig. 78
H.5 3D Common Corruptions
Here we report evaluations using different considered 3D common corruptions over all considered datasets. OOD Robustness evaluations with 3D Common Corruptions include Fig.
79
Fig.
80
and Fig.
81
Limitations
Benchmarking optical flow estimation methods a computationally and labor-intensive endeavor. Thus, best utilizing available resources, we use FLOWBENCH to benchmark limited number of settings. The benchmarking tool itself offers significantly more combinations that can be benchmarked. Nonetheless, the benchmark provided is comprehensive and instills interest to further utilizing FLowBENCH
60

---

Published in Transactions on Machine Learning Research 22gu))/2025)
Sintel-Clean CosPGD Zero Target- L.
Sintel-Clean - CosPGD - -Zer Target-
DPC Groand Treth
15.0
ESPE wext Target Blow
12.5
1.0
10.0
7.5
0.0
12.5
17.5 20.0
2.5
7.5
12,5
12.5
Num.
Neerations
Attack Berations
Model
Model
CCMR
Lowel
RAPIDfIOw
CUXB
Nowwhrmarr
LiteFlowNec
CRAPT
GMA
LLA Flow
BOKNet
CHAFT
GMA
LLA- How
RPKNet
DlC
EMPlow
Scop=fllow
DICL-
GMPllww
MaskTlowee
Seapeltow
SKrlow
SKiPlow
LiteblowNad
PWWNNe
Statt/oww
StaitFlow
Flow1D
LiMellowNe2
RAIT
Videoffow
MlolID
RAPT
Vidwo/7oww
Flosffommr
PlowFortar
Figure 60: Evaluations for targeted CosPGD attack with target O under -oom bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintel-Clean CosPGD Zero Target- L2
Sintel-Clean CosPGD> Zero Target 2/
EPPS wett Tarpet Flow
EPE WEL Ground
12
1.0
0.0
2.5
5.0
75
108
12.5
15.0
175 20.0
0.0
2.5 5.0
7.5
10,0
12.5 15.0
17.5 20.0
Num. Attack Berations
Num. Attack Rerations
Modal
Model
CCMB
RAPIDFfOw
CCMB
RAPIDelaw
BOKNNt
How
Scopefllow
Sespallow
HB3
MS-RAPTT
SKifow
DIP
HDS
NS-IAATT
SXPPoo
PastFlowset
LitebloWNst
PWCNNL
Slatt/low
LitaBleNee
PWCNNe
StaitFlow
FlolDD
LibePloWNec?
RAPT
Videofliww
Plow
RAFT
Vidwo
Moooforme
Plowoormer
Figure 61: Evaluations for targeted CosPGD attack with target O' under fz-norm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
J Reproducibility Of Evaluations
There always exists stochasticity when evaluating adversarial attacks, due to the randomness these attacks exploit, and also common corruptions due to variations in seeds and calculation approximations made by various python libraries. Therefore, for transparency and reprodueibility, we evaluate different runs on the same seed and different runs of different seeds. We report these evaluations here, using Tab. 3 for adversarial attacks and Tab. 0 for common corruptions, and observe that the variance is extremely low and the analysis performed in this work still stands.
To ensure the reproducibility of our adversarial attack evaluations repeat experiments in two ways: first, three different runs with the same seed, and second, one run each for three different seeds. We observe very minute variations in results in both cases which can be attributed to calculation approximations made by different libraries such as pytorch (Paskee et al., 2019 Due to the compute-hungry nature of these evaluations, we limit them to using one method: RAFT on the KITTI2015 dataset, and the attack used is CosPGD. We evaluate multiple settings: different fp-norm bounds, different attack optimization methods (optimizing w.r.t. ground truth flow and optimizing w.r.l. initial flow prediction.), and for targeted allacks, two different targets. The attack settings are consistent with the paper.
61

---

Published in Transactions on Machine Learning Research (Auuust/2225)
Sintel-Clean CosPGD Negative Target-L. L
Sintel-Clean CosPCD Negative Target-
27.5
25.0
22.0
3.0
17.5
EPE
12.5
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
0.0
2.5
5.
7.5
10.0
12.5
15. 17.5 25.0
Num, of Attocis Iterations
Nom. of Attack Iterations
Model
Model
Lneelowheet
RAPIDROM
CCMR
FlowFomeee
LieFoowNedd
RAPIDELOW
DSCL Flow
MaskFloweet
ScopeFlow
DICL
CMFlww
MaskFlownet
ScopeHlow
DIP
HD3
MS RAFT+
SKFTow
DEP
HDD Litn
MS RAFT
SKFlow
PastFloweet
Liteblowist
POUNNL
STatt/oow
Flowaet
PWCNN
StaitPow
FlawID
LiteFooNec2
RAPT
VideoFoww
FlowID
LiteBleweez
RAFT
VideoFlow
FlowFormer
Figure 62: Evaluations for targeted CosPGD attack with target 3 under -norm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintel Clean CosPGD Negative Target
Sintel Clean CnsPGD Negative Target
Farget flow
26
WiL Ground
25
23
22
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
0.0
2.5
5.0
7.5
10,0
12.5 15.0
17.5 20.0
Herations
Mttack Iterations
Model
Model
CCMR
LitellowNetS
RAPIDHON
CCMR
Flowfomrr
LiteFlowNett
RAPIDeluw
Flow
B2KNet
DECL
ScopePlow
DICL
MaskFlameet
Soope
HD3
MS-SAPT
SK3Tow
HDS
38-RAFT4
FastFlomzet
LitaFloNeet
PWCCNe
STaRflow
FustFloweet
LiteBewNet
PWCNe:
STuRFlow
FlowID
LitePloWNez2
RAPT
VidcoFoow
FlowID
LitePooNee2
RAFT
VideoFlow
PlonFormee
FlowForrer
Figure 63: Evaluations for targeted CosPGD attack with target 7 under f2-norm bound using the MPI Sintel (clean) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintal-Final CosPGD Untargeted
Sintel-Final Final CosPGD Untargeted
BP0 wet. Orgund Tratb
BPC wist. Ground Trubb
300
200
1.00
0.6
2.5
5.0
7.5
10.0
13.5
15.0
17.5 20,0
0.0
2.5
7.5
16,0
12,5 15.0
17.5 20,0
Iteratiion
Mttck Ieeratoon
Model
Model
COMR
BowEsrmetet
LiteFWodNett
RAPIDROM
CCMR
FloFoomert
LibeFloNNec
RAPIDELWW
CHAPP
GMA
LLA-Flow
CHAPP
GMA
LLA-How
RPKNet
Flow
NaskFlowas
ScopeFlow
MarkFlownet
ScopeHlow
HD3
MS
SKETow
HDS
Pastrfoweet
LiteblowNet
PWENNt
Stat/fow
Pes Flowane
PANGNN
Siaittllow
FlowID
LiteFlowNst2
RAPT
VideoHlow
PlowlD
Lite
RAFT
Vidwoslow
FlawEumee
FlowEormee
Figure 64: Evaluations for non targeted CosPGD attack under lo norm bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
62

---

Published in Transactions on Machine Learning Research (Auuust/2025
Sintal-Final CosPGD Zeer Target-
Sintel-Final CosPGD Zero Target
15.0
w.... Cround
10.0
5.0
0.0
2.5
3.0
75
10.8
12.5
15.0
17.5 20.0
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
Num. Attack Berations
Num. Berations
Model
Model
RowEortee
Lieelowheett
RAPIDROM
CCMR
FlowForeeee
LioFoowNedd
RAPIDELOW
How
Flow
ScopeFlow
DICL
CMFlww
MaskFlownet
Scope
HB3
MS--RAPT+
Skrfow
HDS
MS-HAAT++
PastFlowset
Liteblowist
PWUNNL
STatt/oww
Past Flowaae
PWCNNe
StaitPlow
FlawLD
LiNeFooNee2
RAPT
VideoFoww
FlowID
LiteRewNei2
RAFT
VideaFlow
Plowformer
Figure 65: Evaluations for targeted CosPGD attack with target O under -orm bound using the MPL Sintel (final) dataset. The attack was optimized wrr.t. the ground truth predictions.
Sintel-Final CosPCD Zero Target La
Sintel-Final CosPGD Zero Target L
I3P wast: Thogge Nlw
wett Ground
122
7.5
Nom. Attaci Iterntions
Num. Atback Bterations
Modal
Model
COMR
RowEormere+
LiteFlowNNetS
RAPIDITIO
CCMR
FloFormeere
LibePloWNed
RAPIDFIOw
GRAPT
FMA
LLA-Tlow
BRKNet
CRAFT
GMA
LLA- Ploe
RPKNet
GMPlow
Scopelllow
DiCL-Flow
GMPlew
Maokfloweet
Sespaffow
HD3
MS BAFT+
SKFlow
HDD
RAFT
SXPlow
LitellowNet
FlowLD
LitelloNec22
RAFT
VidcoFoww
Flow1D
LieeBowNe22
RAFT
VidooFlow
FlowEurter
FlowFormer
Figure 66: Evaluations for targeted CosPGD attack with target O' under fg-norm bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
Sintel Final CosPGD Negative Target L.
Sinte] Pinal CasPGD Negative Target
27.5
25.0
E212 wort. Ground
10
22.5
00
17.5
15.0
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
Num. of Attacis Ierratonns
Num. of Mttack Beertions
Modal
Model
RowForteres
LiuetooNNet
RAPIDHOM
LitaFlowNett
RAPIDELOW
CRAFT
CMA
Flow
B3KNet
CRAFT
RPENet
DiCL-Plow
Scopefloow
DICL-Plow
GMBlew
MaokFlownet
Scopatfuw
DiP
HD3
MS5 SAPTS
SkFlow
DIP
HDS
NS RAFT+
SxFlow
RastFlomaet
LiteFloNNet
PWCNet
STaRSlow
FustFlowsst
Liteee
PACNN
STLRFLOW
FlovDD
RAPT
Videotfiw
MoolD
BAPT
Vidwe
Figure 67: Evaluations for targeted CosPGD attack with target - under le -orm bound using the MPI Sintel (final) dataset. The attack was optimized w.r.t. the ground truth predictions.
63

---

Published in Transactions on Machine Learning Research (iggus//0025)
CosPGD Nogative Target- Lg
Sintel-Final CosPCD Negative Target L2
Ground
25
0.0
2.5
500
7.5
10.0
12.5
15.0
17.5 20.0
0.0
2,5
5.0
7.5
10.0
12.5 15.0
17.5 20.0
Num. of Attaci Nerations
Num. of Attack Berationa
Model
Model
RAPIDHfow
RAPInFlor
EMPlow
Scogefllow
DXCL--llw
GMfllow
Maskflownet
Sospaffow
DIP
HIL
MS-RAPTT
SK/low
DIP
HIDS
NS-HAPT+
SccPlow
PastFlommnt
LisePloNeet
PWCNet
SToRTTow
FestFloweet
PACGNet
STaRPlow
TloiDD
RAPT
Videoflow
Mow11
BART
Vidwo
FluwEurmer
FlowFurmer
Figure 68: Evaluations for targeted CosPGD attack with target - under by-norm bound using the MPI Sintel (final) dataset. The at attack was optimized w.r.t. the ground truth predictions.
EPE calculated w.r.t. Ground Truth
EPE calculated w.r.t. Target 0
KITTI-2015- PCFA Zero Target L
KFTT1-2015 PCFA Zero Target- 42
800
80
Tarpet
80
wezt.
40
20
12.5 15.0
Iterations
Model
Model
CCMR
FlowFormer
IRR
RAPIDFlow
CCMR
RomPoomer
IRR
RAPIDFIow
CRAFT
FlowFFrmeri
LioFlowNeet
RPKNet
CRAFT
FlawPormrr
RPKNot
LLA-foow
SoopnHlow
LLA-Flow
Stopel
DICI-Flow
GMFlaw
Flownet
SKFwww
DICL-Flow
GMFlow
MarkFlownst
SKFlow
CMFlowNet
-AFT,
STuRFlow
DIP
GMFlowNet
MS-RAFT+
STaRFlow
RusFlownet
HDD
RAFT
VideoFlow
PastFooweet
HDD
RAPT
Videoflow
FlowID
FlowDD
Figure 69: Evaluating all optical flow estimation methods against PCFA attack with target J over multiple attack iterations.
64

---

Published in Transactions on Machine Learning Research (August/2255)
EPE calculated w.r.t. Ground Truth
EPE calculated w.r.t. Target
KITTI-2015- PCFA- Negative Target- L2
KITTI-2015 PCFA- Negative Target- L
150
125
EPE Target Flow
1:2
EPE wezt. Ground
1000
75
50
25
2.5
5.0
7.5
10.0
12.5
15.0
17.5 20.0
0.0
2.5
12.5 Num. of Attack Iterations Mode!
7.5
10.0
15.0
17.5 20.0
Num. of Attack Iberations Model
CCMR
Plowformer FlowFormer++
IRR
RAPID/low
CCMR
Pow/oomer
IRR
RAFIDUIOW
CHAРT
Now
RPKNNt
CRAFT
Alwroomrr+
RPKNot
CSFlow
GMA
LLV Flow
ScopePlow
CSFlow
GMA
LLA Flow
ScopeFlow
DICL-Flow
GMPlow
Mask/lownet
--- SKrlow
DICL-Flow
GMIllow
Movidtlownet
- SKPllw
DIP
GMFlowNet
MS-RAFT+
SToR
DIP
GMPlowNet
MS-RAFT+
FustFlownet
HDS
RAFT
VideoFlow
RostFlownet
HD3
RAFT
VideoFlow
FlowID D
FlowlD D
Figure 70: Evaluating all optical flow estimation methods against PCFA attack with target - over multiple attack iterations.
R4a24 Radd Aierraad Pay Unceepted
Faud Adherrnd Warterr Png 2eace Easid
Sund Real Alwonaid Motther Pay Ri Degin
Modal
Medel
Medal
Praforee
Figure 71: Evaluations for Adversarial Weather attack with Fog optimized as an non-targeted atlack (left), and targeted attack with targets o' (center) and 7 (right).
Saggsiie Zerget
DMA
TMCH4
OMA
Figure 72: Evaluations for Adversarial Weather attack with Rain optimized as an non-targeted attack (left), and targeted attack with targets O' (center) and 7 (right).
Riaoditts: Afteriical Weither- Zean Unlerpenc
Sasel/Taal Aovererti Westher Soowe Zero Deppe
Raluitas -tverarti Weatber Stow: Magetho Zlrget
Signlaree
Astated
Make
TenToruw
Husd H54007T
WaaToor
Mulllose
KS
Figure 73: Evaluations for Adversarial Weather attack with Snow optimized as an non-targeted attack (left), and targeted attack with targets O' (centor) and 7 (right).
65

---

Published in Transactions on Machine Learning Research AAggust/2025
Spath7ral: Adveraraa Vesther Sperte Untargeeed
Caiakflaa Acrmacc Weather Spactr Steo Cape
Sala/7/all Achonatta Weater: Sgana Negubre Sarpt
13
Jopatered
Aotated
Modal
Mode
Medd
XAPT
Weefee
RXPT
Figure 74: Evaluations for Adversarial Weather attack with Sparks optimized as an non-targeted attack (left), and targeted attack with targets 0' (center) and -3 (right...
2D Common Corruptions
3D Common Corruptions
150
150
E7E
1 200
50
50
Caursian Natio
Impuiio Nomt
Notse
Defecir Mla
Metan flar
Front
Zeor Bioor
Comtrnt
Flg
Brightners
Pitolate
Elastte Tranafurns
Jen Comprraaton Glass Rlar
Near Ronus
Far Ponta
Fog 3p
Noss
Low Libht
Ty Metion Blur
Color Quaet
Menisn Biar
Corrigtin Type
Cormuption Type
Model
CCMR
FlowFormer+
A-A-Flow
RAPIDFlow RPKNet ScopePlow
FastFlownet
MaskFlownet
STaRFlow
CSFlow
FlowiD
GMFlow
LiteFloWNet
MS RAPT+
VidenTlow
DICL-Flow
FlowFrrmer
GMFlowNet
LiteFloNNet3
RAPT
Figure 75: Performance of various optical flow estimation methods after corruptions on the KITTI2015 dataset.
Table 4: We report the mean EPE the baseline method RAFT (Tedd & Deng 2020 and RAFT trained in a semi-supervisod way using Flow Supervisor (Im 2022 and RAFT trained self-supervised way using AutoFlow (Huang et al. 2023), and finally RAFT trained using AutoFlow and then finetuned on the KTTT12015 training dataset 'AutoFlow Finetuned:.
2D Common Corruptions
RAFT
Flow Supervisor Im et al.
AutoFlow Pretrained
AutoFlow Finetuned
Teed & Deng 2020
2022
Huang et al.
2023
Huang et al.
2023
Gaussian Noise
6.02
4.10
2.16
2.35
Shot Noise
4.17
2.85
2.18
2.36
Impulse Noise
6.07
3.70
2.15
2.36
Defocus Blur
2.02
2.02
2.27
2.40
Glass Blur
2.39
2.16
2.23
2.36
Motion Blur
4.08
3.92
2.29
2.43
Zoom BLur
4.81
4.74
2.20
2.38
Snow
40.42
53.31
1.82
2.60
Frost
26.51
95.53
1.93
3.04
Fog
1.22
2.99
2.28
2.43
Brightness
1.24
2.60
2,26
2.43
Contrast
1.19
1.22
2.28
2.43
Elastic Transform
1.05
1.13
2,29
2.44
Pixelate
0.79
0.86
2.28
2.44
JPEG Compression
1.94
1.89
2.13
2.11
mean Corruptions
6.93
12.20
2.18
2.46
Clean
0.61
0.73
2.29
2.44
66

---

Published in Transactions on Machine Learning Research (lngust/2025)
0153211-- Creave Compatinn
M05-011-- Cenerie Comptiees Oreter
K11.0015- Cencees Sebves
Voca
Vodal
CHAFT
85344
DRAPT
FaciRorat
103
Comptonn
Oiperated
(33
DM4
107
Shiller
Fivd)
Phel
17152011 Creren Compteen
1252201 Compatinn
Uipetabeed
Moddl
DMAF
DRAPT
MiLL-T
Comptione Jag Cxepresse
Upp
Digetatedd
Oipetaaked
Intuceed
(134
3ire3
EXP-2215- Conmus Compptane 1ha
R20-005- Conrcc Compgaata 20ow
R15-311 Canoua Compptees Zaen lik
Pepeteted
Voca
M664
Vodal
(OUTT
Dodforeree
2000
CRAFT
20004
(RAIT
Ravia
Figure 76: Evaluating optical flow estimation methods against all 2D Common Corruptions on the KITTI2015 dataset.
67

---

Published in Transactions on Machine Learning Research (lngust/2025)
Rand/ber Draple Trabow
CaldCos Drrestappeen Carred
Eaid Dreve Gmpiinn Inten
Uppeta3ed
hntieed
Digenated
Dipenated
Petubeed
CCNR
-RAPT
CRAFT
Tuilbora
ID3
Blod3
Rwil
Cooe Drmplns Hoda
11.0
Uppet2Sed
hnteled
Dipenated
Dipenated
Ietubeed
0396
CCN:
CCMOL CRAFT
CRAPT
TucTooat
(D)
7W00
Bovt
hopala
Uppetaeed
hnteceed
Diperaaeed
Detubee
Upgetuaaed
CCMIL
DSOT
CRA2T
TucTocat
1D3
DAY
Flovl
The
Mutathau Cerredann
Danglen Madas
Oepetcted
hnteceed
Doonattd
Derabed
Upperaree
Detucbed
(156
Dodfore Bodivanee
CCHR CRATT
CCMEL CRAUT
USOT
TutTorat
HD3
3
Leaflootie
Hepetatdd
Pnticled
Dooncted
Dera
Deticted
Dodforer
Unlleedee
CCHR CRATT
Ravioren
CCME)
3420
1S07
(RAIT
H03
Dive3
Tawi
Figure 77: Evaluating optical flow ostimation methods against all 2D Common Corruptions on the MPI Sintel (clean) dataset.
68

---

Published in Transactions on Machine Learning Research (lngust/2025)
tedethel Deeen Compless Inplen
Eaxitbll Cerva Comples Darcel
Uppeta3ed
Dipenated
Petubeed
CCNR
-RAPT
DRAFT
Tuilbora
ID3
Blxt3
Rwi
Naxitbul Complee:
Phut Conpen
Dosses Durretton hret
Uppea2eed
hntelee
Dipenated
Ieratee
Dipenated
Petubee
CCN:
CCMBL
CRAPT
CRAFT
TucTooat
(D3
7W00
Bort
Deel
Sode Ral Daees Carrgg
Uppetaeed
Inticeed
Iipenatee
Detubee
C156
Upgetraed
CCMIL
DSOT
CRAUT
TucTocat
(D3
DAY
The
Oepttcted
hnticeed
Doonattd
Derabee
Uppeaared
Detucbed
(156
Dodfore Dodforneee
CCHR CRATT
CCMEL CRAUT
1SOT
TutTorat
1D3
Sadal Rad. tappeat
Dipetatedd
Peticted
Dooneted
Dera
Detiched
(O50
Dodforer
Unlleeded
CCHR CRATT
Ravioren
COME)
3420
1SU7
(RAIT
103
Dive3
Figure 78: Evaluating optical flow ostimation methods against all 2D Common Corruptions on the MPI Sintel (final) datasel.
69

---

Published in Transactions on Machine Learning Research (August/2025
Figure 79: Evaluating optical flow estimation methods against the considered 3D Common Corruptions on the KITTI2015 dataset.
Figure 80: Evaluating optical flow estimation methods against the considered 3D Common Corruptions on the MPI Sintel (clean) dataset.
1385
Figure 81: Evaluating optical flow estimation methods against the considered 3D Common Corruptions on the MPI Sintel (final) datasct.
70

---

Published in Transactions on Machine Learning Research (iguust/2025,
Table 5: To ensure the reproducibility of our adversarial attack evaluations we repeat experiments in two ways: first, three different runs with the same seed, and second, one run each for three different seeds. We observe very minute variations results in both cases which can be attributed to calculation approximations made by different librarics such as pytorch (Paszke et al., 2019). Due the compute-hungry nature of these evaluations, we limit them to using one method: RAFT on the KITTI2015 dataset, and the attack used is CosPGD. We evaluate multiple settings: different f--norm bounds, different attack optimization methods (oplimizing w.r.t. ground truth flow and optimizing w.r.t. initial flow prediction.), and for largeted atlacks, two different targets. The attack settings are consistent with the paper. Target 'None' means the attack was Non-targeted.
EPE mean + std
px3 error mean + std
fo-norm bound
Target
Attack Optimized w.r.t.
Three different runs on the same seed
t0o norm -orm loo -orm
None
Ground Truth Flow
119.504 + 2.95E+0
0.078 + 6.76E-3
Ground Truth Flow
45.357 4.26E-1
0.200 + 7.84E-4
Ground Truth Flow
10.674 + 2.74E-1
0.647 + 1.06E-2
fo-morm norm
None
Ground Truth Flow
0.644 = 2.72E-6
0.968 + 3.38E-6
Ground Truth Flow
73.454 = 3.12E-5
0.129 + 2.86E-7
fo-m f--mrrm
Ground Truth Flow
36.724 = 2.11E-5
0,170 + 4.41E-7
None
Initial Flow Pred
0.643 + 7.74E-6
0.968 1.49E-6
One run each using three different seeds
-orm -nom
None
Ground Truth Flow
119.692 + 1.75E+0
0.077 + 4.27E-3
Ground Truth Flow
45.149 = 9.16E-1
0.202 + 1.65E-3
-oom
Ground Truth Flow
11.016 + 4.91E-1
0.625 9.90E-3
norm
None
Ground Truth Flow
0.644 = 7.46E-6
0.968 + 6.05E-6
no-m
Ground Truth Flow
73.454 + 1.15E-4
0.129 + 1.85E-7
fo-mo
Ground Truth Flow
36.724 + 1.10E44
0.170 + 7.92E-7
f--mrrm
None
Initial Flow Pred
0.643 + 1.44E4
0.968 + 1.55E-5
71

---

Published in Transactions on Machine Learning Research (oggus/20025)
Table 6: To ensure reproducibility of Common Corruptions evaluations we repeat experiments in two ways: first, three different runs with the same seed, and second, one run each for three different seeds. We observe extremely minute variations in results which can be attributed to differences in seeds and calculation approximations made by the Python libraries. Due to the compute-hungry nature of these evaluations, we limit them to using one method: RAFT on the KITTI2015 datasct, and all the fifteen 2D Common Corruptions.
EPE
px3 error mean std
2D Common Corruption Name
mean + std
Three different runs on the same seed
brightness
1.235 + 0.000
0.935 + 0.000
contrast
1.187 + 0.000
0.938 + 0.000
defocus blur
2.026 + 0.000
0.899 + 0.000
clastic transform
1.043 = 0.000
0.954 = 0.000
fog frost
1.221 + 0.000
0.936 + 0.000
29.640 + 0.000
0.383 + 0.000
gaussian noise glass blur impulse noise
5.931 = 0.000
0.732 = 0.000
2.409 = 0.000
0.861 + 0.000
6.098 + 0.220
0.736 + 0.002
jpeg compression motion blur
1.942 + 0.000
0.892 + 0.000
5.513 = 0.000
0.549 = 0.000
pixelate
0.785 + 0.000
0.960 + 0.000
shot noise
4.435 + 0.000
0.780 + 0.000
snow
41.974 + 0.000
0.354 + 0.000
zoom blur
4.808 + 0.000
0.746 = 0.000
One run each using three different seeds
brightness
1.235 + 0.000
0.935 + 0.000
rontrast
1.187 + 0.000
0.938 + 0.000
defocus blur
2.026 + 0.000
0.899 +-2 0.000
clastic transform
1.041 = 0.009
0.953 = 0.001
fog frost gaussian noise glass blur impulse noise
1.282 + 0.076
0.937 + 0.001
28.783 + 0.827
0.391 + 0.019
5.877 +: 0.026
0.735 +' 0.001
2.532 = 0.105
0.859 = 0.002
5.856 + 0.067
0.737 + 0.002
jpeg compression motion blur
1.942 + 0.000
0.892 + 0.000
4.135 = 0.141
0.565 + 0.010
pixelate shot noise snow
0.785 + 0.000
0.960 + 0.000
4.336 + 0.143
0.781 + 0.004
42.984 + 4.786
0.362 + 0.007
zoom blur
4.808 + 0.000
0.746 + 0.000
72


Reviews:
[{"review_id": "xeWc6Nf2b6", "reviewer": "Reviewer_8XL8", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The experiments appear to be thorough, covering a wide range of methods, datasets, corruption techniques, and attacks. This comprehensive evaluation along with the checkpoints will be highly valuable for the community.\n2. The authors provide insightful analyses, including correlations between different evaluation metrics, model parameters and model architectures.\n\nWeaknesses:\n1. The writing and presentation of the paper can be greatly improved. There are places that are confusing and require careful editing. I highly recommend that the authors to use LLMs for editing their work in general. Specific suggestions are provided in the requested changes section.\n2. For a benchmarking paper, it is crucial that the codebase is readable and easily extendable. Unfortunately, the authors did not provide any code for review.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper benchmarks the performance, reliability, and generalization capabilities of several optical flow estimation methods across multiple datasets. The authors present their experimental results along with additional analysis and provide checkpoints for users to reproduce or build upon their work.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. In the introduction and abstract, you mention benchmarking 63 model checkpoints, but later state that you provide 91 checkpoints. This discrepancy is confusing—could you please clarify what the 63 refers to?\n2. It would also be very helpful to summarize the experiments in a table, similar to Table 1 in the appendix. This would give readers a clearer understanding of the full scope of the evaluation.\n3. At the beginning of Section 4, you mention that FlowBench supports 36 methods and several datasets. Could you please clarify what “supports” means in this context? \n4. I would encourage the authors to include a diagram or figure illustrating the FlowBench framework. This would make the design and workflow easier to understand.\n5. Several metrics used in Section 5—such as 1-px error, 3-px error, 5-px error, and EPE—are not defined anywhere in the paper. Please define these metrics in the main text or the appendix, along with appropriate references.\n6. Regarding the use of the zero vector and the negative of the initial flow prediction: it would be helpful to explain their meanings more clearly, and to provide the actual mathematical notations rather than describing them only in words.\n7. Finally, would it be possible to share the codebase with reviewers? This would significantly help in evaluating the reproducibility and usability of the benchmark.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "jm6oRIGej4", "reviewer": "Reviewer_eUgT", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I believe this paper leans more toward a technical report than a traditional academic research paper, primarily due to the lack of novel methodological contributions. While I acknowledge and appreciate the authors' efforts, particularly in consolidating benchmarking tools and conducting a large-scale evaluation, the work would benefit from deeper analytical insights and a more comprehensive review of the current landscape of deep learning-based optical flow estimation methods.\n\nThe paper does an excellent job of presenting a clear and well-structured background and introduction. The major strength of the paper lies in providing an easy-to-use interface for benchmarking optical flow models. Running over 4,500 experiments is a commendable effort and a valuable contribution to the field.\n\nHowever, several areas of the analysis lack depth. For instance, in Section 6.1, the authors state, \"When considering generalization ability under common corruptions, we observe that all methods have poor performance.\" This result is unsurprising, as the evaluated models were not designed or trained with robustness to such corruptions in mind. Therefore, the observation lacks novelty. Moreover, the grouping of methods into four families appears to offer little insight, as no meaningful patterns or design principles are derived from this taxonomy.\n\nIn the same section, the authors further claim that “Fig. 2a that compared to other method families, the FlowFormer family is very susceptible to targeted adversarial attacks. ... ... We note that this contradicts the observations made for transformer-based methods for image classification (Paul & Chen (2022) and Hoyer et al. (2022)). However, these works primarily address robustness to common corruptions and domain adaptation, not adversarial attacks. This weakens the comparison and the basis of the claimed contradiction.\n\nSimilar issues arise in Section 6.3. The models under study were not designed with adversarial robustness or input corruptions in mind. Consequently, many of them exhibit similar performance under both NARE and TARE metrics. This outcome, while expected, raises questions about the value of this particular analysis. In contrast, results from RobustBench — a benchmark for adversarial robustness and corruption robustness in image classification — show that models like WideResNet-94-16, WideResNet-70-16, and Swin-L achieve top scores on CIFAR10, CIFAR100, and ImageNet, respectively. These findings suggest a stronger correlation between model capacity (i.e., number of parameters) and robustness in classification tasks. The lack of such correlation in this paper’s analysis may be more reflective of model selection or training setup than of inherent optical flow estimation characteristics, which should be discussed.\n\nFinally, in Section 6.4, the authors state, “We observe in Fig. 3b that most methods that have a good performance also generalize better.” However, this conclusion is vague and fails to yield any actionable or insightful findings. The analysis does not effectively distinguish the nuanced relationship between clean performance, generalization, and robustness.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces FlowBench, a robustness benchmark and evaluation tool for learning-based optical flow methods and evaluation of 63 models  across 3 standard datasets (KITTI2015, MPI Sintel clean/final) using 7 adversarial attacks and 23 corruption types (2D and 3D).", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would suggest that the model trained with adversarial examples or corrupted inputs should be involved. Also, deeper analytical insights and a more comprehensive review of the current landscape of deep learning-based optical flow estimation methods.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "IB98m6I3B4", "reviewer": "Reviewer_HeGh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. This paper systematically benchmarks a range of optical flow methods, both recent and established, by introducing 7 diverse adversarial attacks and 23 common corruptions.\n2. Several interesting research questions are explored through the benchmark analysis, such as which families of optical flow methods are more robust, and how the number of learnable parameters and the choice of point matching mechanisms impact performance.  \n3. Figure 6 illustrates examples of both targeted and non-targeted attacks and effectively demonstrates how these attacks work. Please include it in the main paper rather than in the appendix.\n\nMajor weaknesses\n1. The datasets used for benchmarking the optical flow methods were also part of their training, and as a result, most of these methods are already overfitted to them. Creating a new dataset with ground-truth optical flow for this benchmark would provide a more accurate evaluation of generalization.\n2. There are also unsupervised and semi-supervised optical flow approaches [1,2,3]. While these methods may not achieve the same level of accuracy as supervised ones on the datasets used in this benchmark (as they may not be overfitted), they could demonstrate greater robustness when applied to real-world videos. How do the authors plan to incorporate such methods into the proposed benchmark?\n3. This paper focuses on benchmarking the reliability and generalizability of optical flow methods for real-world scenarios and videos (e.g., YouTube videos). However, generating ground truth for real-world videos is non-trivial. Therefore, evaluating optical flow methods under perturbations that commonly occur in real-world settings is a reasonable approach. However, it may still be even more meaningful to propose a new dataset that captures the performance of these methods on such real-world videos. How do the authors propose to address this challenge?\n4. Most of the attacks considered in this work are perturbation-based and do not account for spatial attacks [5], which could be particularly important for evaluating optical flow methods.\n5. This paper brings attention to a key issue: the lack of generalization in optical flow methods. However, the more pressing need is a dataset that enables benchmarking in real-world scenarios, which remains a significant challenge. To date, only one effort, SlowFlow [6], has attempted to address this, and no recent work or new datasets have emerged to tackle the problem. Introducing such a dataset would greatly enhance the value and impact of FlowBench.\n6. Since, in a real-world scenario, the attacker may not have access to the models, please clarify how these attacks can be formulated as black-box attacks. The benchmark, as presented, does not seem to fully capture real-world scenarios, and it would be helpful if this limitation were discussed in more detail in the paper.\n\nMinor Weaknesses\n\n- It would be helpful to include method names as a legend in Figure 1 for clarity. Additionally, please define reliability and generalization early in the paper.\n\n[1] RealFlow: EM-Based Realistic Optical Flow Dataset Generation from Videos, Han et al., ECCV 2022. \n\n[2] Self-supervised AutoFlow, Huang et al. CVPR 2023. \n\n[3] What Matters in Unsupervised Optical Flow, Jonschkowski et al., ECCV 2020. \n\n[4] Semi-Supervised Learning of Optical Flow by Flow Supervisor, Im et al., ECCV 2022. \n\n[5] Spatially Transformed Adversarial Examples, Xiao et al., ICLR 2018. \n\n[6] Slow Flow: Exploiting High-Speed Cameras for Accurate and Diverse Optical Flow Reference Data, Janai et al., CVPR 2017.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper evaluates the robustness of optical flow methods under adversarial attacks and common corruptions, revealing a key limitation, which is their poor generalization ability. While several new optical flow estimation methods have been proposed recently, most overlook critical issues of reliability and generalization, often overfitting to the synthetic dataset Sintel and the real-world KITTI2015 dataset, which is limited to driving scenarios on the street.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see strenghts and weaknesses.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)