S PATIALLY CONSTRAINED A DVERSARIAL A TTACK D ETECTION AND L OCALIZATION IN THE R EPRESENTATION S PACE OF O PTICAL F LOW N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Optical ﬂow estimation have shown signiﬁcant improvements with advances in deep neural networks.
However, these ﬂow networks have recently been shown to be vulnerable to patch-based adversarial attacks, which poses security risks in real-world applications, such as self-driving cars and robotics.
We propose SADL, a Spatially constrained adversarial Attack Detection and Localization framework, which does not require dedicated training.
The detection of an attacked input sequence is performed via iterative optimization on the activations from the inner layers of ﬂow networks, without any prior knowledge of the attacks.
The novel spatially constrained optimization ensures that the detected anomalous subset of features comes from a local region.
To this end, SADL provides a subset of nodes within a spatial neighborhood that contribute more to the detection, which will be utilized to localize the attack in the input sequence.
The proposed SADL is validated across multiple datasets (i e., MPI-Sintel and KITTI) and ﬂow networks (i e., FlowNetC, FlowNet2, PWCNet and RAFT).
With patch attacks 4 .
8% of the size of the input image resolution on RAFT, our method successfully detects and localizes them with an average precision of 0 .
946 and 0 .
951 for KITTIand MPI-Sintel datasets, respectively.
The results show that SADL consistently achieves higher detection rates than existing methods and provides new localization capabilities.
I NTRODUCTION
Motion estimation is a key problem in computer vision.
Motion in video is often represented in the form of dense optical ﬂow ﬁelds, which specify the motion of each pixel from one frame to the next.
It has diverse application areas, such as tracking (Shi & Tomasi, 1994), action recognition (Diba et al., 2017), 3D reconstruction (Agarwal et al., 2009), video denoising (Buades & Lisani, 2018) and video compression (Le Gall, 1991).
With the emergence of deep learning techniques and availability of large scale datasets, optical ﬂow estimation performance has been signiﬁcantly improved (Dosovitskiy et al., 2015; Ilg et al., 2017;
Sun et al., 2018;
Teed & Deng, 2020) across existing benchmarks (Butler et al., 2012; Menze &
Geiger, 2015).
However, recent work (Ranjan et al., 2019; Schrodi et al., 2021; 2022) demonstrated that these networks are also vulnerable to adversarial attacks just as any other deep learning approaches (Papernot et al., 2017; Akhtar & Mian, 2018;
Akinwande et al., 2020; Kim et al., 2022), thereby raising security issues, e.g., in self-driving cars.
These attacks on optical ﬂow estimators come in the form of patches that are pasted onto the input image frames at the same pixel coordinates.
This is motivated by cases where something gets stuck on the video camera (see Figure 1).
These patch-based attacks cause occlusions on areas where the attack is placed, as well as motion boundaries or discontinuities in ﬂow along the boundaries of the attack, which is exactly where current optical ﬂow estimators suffer (Kim et al., 2021).
Ranjan et al.
(2019) is the ﬁrst work to tackle optical ﬂow estimators with adversarial patches that are optimized so that the direction of the original ﬂow estimations are shifted towards the opposite direction.
Schrodi et al.
(2021; 2022) continue this work to show that the location these adversarial patches are applied on the input images is important.
Figure 1 shows the effect of these patch attacks
Attacked I t
Original ˆf
Attacked ˜f
Attacked I t
Original ˆf
Attacked ˜f on two ﬂow estimators, FlowNetC (Dosovitskiy et al., 2015) (top) and RAFT (Teed & Deng, 2020) (bottom), the current state of the art on ﬂow estimation.
While RAFT is more robust to these attacks than FlowNetC, these attacks still disrupt the ﬂow estimations nearby these attacks, e.g. estimations on moving white car (left) and on arm and face (right).
Schrodi et al.
(2021; 2022) further explain that the vulnerability of ﬂow estimators comes from the attacks’ self-similar patterns.
However, there is no existing work, to the best of the authors’ knowledge, that was proposed to detect, localize, and characterize these potential attacks.
We present a Spatially constrained adversarial Attack Detection and Localization (SADL) framework that aims to detect and localize patch-based attacks given any pre-trained ﬂow estimators in an unsupervised fashion without prior knowledge about these attacks.
Speciﬁcally, we propose analyzing the inner layers’ activations with a local-neighborhood constraint across the spatial dimension.
The main contributions of this paper are:
• First to detect and localize patch-based adversarial attacks for optical ﬂow neural networks based on spatially constrained subset scanning on the inner layer activations, without any training of a dedicated model nor prior knowledge of the patch-based attacks.
• Analysis of the effect of the attacks across various inner layers of the ﬂow networks.
• Validate our method across several state-of-the-art optical ﬂow networks and datasets.
R ELATED W ORK
In this Section, we reviewed the related work on ﬂow networks and adversarial attacks on them.
We also summarized subset scanning techniques from anomalous pattern detection literature.
Flow networks Recent methods (Sun et al., 2018; Xu et al., 2017; Dosovitskiy et al., 2015; Teed & Deng, 2020) on optical ﬂow estimation employed deep learning to achieve promising results with a faster inference speed.
Starting with Chen & Koltun (2016), these deep learning approaches used cost volumes (Rhemann et al., 2013), which comprised of feature correspondence scores between frames to compute optical ﬂow.
Sun et al.
(2018) built the cost volumes at multiple scales, and
FlowNet (Dosovitskiy et al., 2015) included the full cost volume at a single scale.
Ilg et al.
(2017) proposed FlowNet2 by stacking four FlowNet-based models.
Teed & Deng (2020) achieved state-ofthe-art performance with RAFT using a complete, multi-scale, and four-dimensional cost volumes for all pairs of pixels on the input images.
Adversarial attacks in optical ﬂow neural networks Ranjan et al.
(2019) is the ﬁrst work to apply adversarial attacks on optical ﬂow models, by pasting an p ⇥ p adversarial patch onto the input image frames I 1 , I 2 at the same location.
This adversarial patch can be obtained using any pre-trained ﬂow network by minimizing the cosine similarity between the estimated ﬂow maps before ˆf = ( u, v ) and after the attack ˜f
=
(˜ u, ˜ v ) , i.e., ˆf · ˜f / || ˆf |||| ˜f || .
This work was later extended by Schrodi et al.
(2021) to show that patch-based attacks trigger matching ambiguities in the correlation output, which are successively spread in the decoder into a wider neighborhood.
In their subsequent work, Schrodi et al.
(2022) demonstrated the importance of the spatial location that these adversarial patches applied on the input image frames.
Large ﬂow regions, e.g., fast-moving objects, are susceptible to a severe deterioration of ﬂow estimations.
Yamanaka et al.
(2021) used patch-based adversarial attacks to simultaneously attack two CNNs developed for optical ﬂow and monocular depth estimation.
Wortman (2021) showed that these patch attacks for optical ﬂow estimation can [Figure] Repeat [Figure]
be hidden by adjusting their alpha values while still affecting the ﬂow estimations.
Global noisebased attacks were recently reported Schrodi et al.
(2022); Schmalfuss et al.
Schrodi et al. showed that optical ﬂow networks are robust to white-box global noise-based attacks, and Schmalfuss et al.
(2022) proposed perturbation constrained ﬂow attack.
Unlike the existing works, we focus on detecting the patch-based attacks in an unsupervised fashion without requiring any prior knowledge of the attacks.
Our work further localizes these patch attacks in the input images, ﬁnding the origin of the attack, which can be useful in future mitigation efforts.
To the best of the author’s knowledge, we are the ﬁrst work to detect or localize adversarial attacks on optical ﬂow estimators.
Subset Scanning for Anomalous Pattern Detection in Neural Networks Subset scanning has been used to detect anomalous samples in various computer vision and audio tasks, including creativity sample characterization (Cintas et al., 2022), audio adversarial attacks (Akinwande et al., 2020) and skin condition classiﬁcation (Kim et al., 2022).
To detect anomalous samples, subset scanning searches for the “most anomalous” subset of node activations from the inner layers of the given test sample on any pre-trained networks, and uses these subset of nodes to obtain the anomalousness score of the test sample.
While existing works employed subset scanning to detect adversarial attacks, we are the ﬁrst to use it to detect patch-based attacks as opposed to global noise-based attacks, on sequential images across temporal dimension, and on networks that estimates regression instead of classiﬁcation task.
Since patch attacks in optical ﬂow estimation occur in a local spatial region, we propose to apply spatial constraints to the scanning step to improve the detection and localization of the attacks.
P ROPOSED APPROACH : S PATIALLY C ONSTRAINED S UBSET S CANNING
Given node activations of an inner-layer, L , of a pre-trained optical ﬂow network ⇥ , we aim to detect a subset of test sequences with adversarial attacks and localize these attacks.
To this end, we propose SADL: Spatially constrained Adversarial attack Detection and Localization framework (see
Fig. 2) to be applied on pre-trained optical ﬂow estimators, in an unsupervised fashion without any prior knowledge of the attacks.
Overview Figure 2 illustrates an overview of our proposed method.
We extract the node-level activations across a given inner-layer of the ﬂow network for a pair of input images.
We compare this observed activation against an empirical distribution of expected activations extracted from clean (un-attacked) images at that same node.
This node-level comparison provides a p-value for each node’s observed activation.
If the test image pair has the same representation as un-attacked images in this ﬂow network layer, then these observed p-values across nodes should appear uniformly distributed.
However, if an adversarial attack systematically increases the activations, then the observed p-values would appear non-uniform.
Our scoring function, to quantify the deviation of the observation from the expectation, is a nonparametric goodness-of-ﬁt statistic which measures the Kullback-Leiblier divergence between the observed distribution of p-values and the expected, uniform distribution Berk & Jones (1979).
A high-scoring image pair suggests it has a different inner-layer representation than the clean, unattacked images.
Critically, this scoring function satisﬁes the Linear-time subset scanning (LTSS) do
P c
FlattenSpatial w
RandomlySelectRows
; c
F do property Neill (2012);
McFowland et al.
(2013) which allows for exact, efﬁcient maximization over subsets of nodes.
However, un- constrained maximization across all subsets of nodes may lead to high scores even when a test pair of images is not attacked (false positive due to matching to noise).
Therefore, SADL implements channel constraints and proximity constraints which result in higher detection power and accuracy for localized patch attacks.
The end result of SADL is a high-scoring subset of node activations in a local, spatial neighborhood and subset of channels.
The score is used for detection power calculations, while the subset of spatial nodes is used for detection accuracy.
Details on the constrained optimization process are provided below.
p-value Computation Let D H 0 be a dataset of M background images that have not been attacked.
Feeding the background image pairs into ⇥ , we can extract M background activations A H
j from each node O j 2 L .
Given a test sample I 62 D H 0 , we extract the activation a j at node O j in L .
We can compute the p-value of the activation a j of this test sample by computing the proportion of activations in A H 0 j that are greater than a j :
[Equation]
where [ · ] is the indicator function.
Spatial Alignment The inner layer under consideration, L , has spatial dimensions w (width), h (height) and channels c .
In order to properly incorporate spatial information across channels, we make the alignment assumption that if spatial location w 0 ⇥ h 0 is to be evaluated, then activations (p-values) from all channels under consideration also located at w 0 and h 0 must be scored as well.
This intuitive assumption is visualized at the top of Figure 2.
Although spatial alignment is always enforced, SADL does not require all channels to be included in the returned subset.
This extension is covered in the Spatial-Channel Optimization section below.
Non-parametric Scan Statistics ( OptimizeNPSS ) Given a tensor of p-values, P , computed with
Eq. 1 and with the same w ⇥ h ⇥ c dimensions as the inner layer L , our goal is to ﬁnd the subset of nodes, aligned at spatial locations, such that the corresponding p-values of these node activations is the most non-uniform .
We quantify divergence away from the uniform distribution using a nonparametric scan statistic (NPSS), such as Berk-Jones (Berk & Jones, 1979), with a general form:
[Equation]
where S is the subset p-values (nodes) under consideration, N ( S ) is the number of p-values in S ,
N ↵ ( S ) is the number of signiﬁcant p-values less than a threshold ↵ .
NPSS is preferred as we make minimal assumptions on how the activations at a given node are distributed.
Without spatial alignment enforced, NPSS can be trivially maximized by creating a subset consisting of all-and-only p-values that are less than ↵ .
However, this naive maximization would likely violate the spatial alignment requirement by not necessarily including all p-values from the same spatial location.
Enforcing spatial alignment makes this maximization more difﬁcult because some not-signiﬁcant p-values may be included in the highest-scoring spatially aligned subset, and some signiﬁcant p-values may be excluded.
To this end, we employ the Linear-time Subset Scanning (LTSS) property of the Berk-Jones scan statistic (Neill, 2012; McFowland et al., 2013).
Returning to the dimensions of the tensor, P , each of the wh spatial locations contains up to c p-values.
There are 2 wh  1 possible subsets of spatial locations to consider with each location contributing c p-values to the scoring function.
The LTSS property reduces this search space from exponential, O (2 wh ) to linear O ( wh ) while still guaranteeing exactness (Neill, 2012).
To do this, each spatial location is sorted by a priority function, N ↵ c .
A spatial location with a priority of 1 means every one of the p-values at that location across the c channels is signiﬁcant at level ↵ .
A priority value of 0 .
5 means only half of the p-values at that spatial location are less than ↵ .
Once the wh locations are sorted by this priority, the LTSS property states that the highest-scoring subset is guaranteed to be one of the linearly-many subsets created by incrementally including the c p-values of the next-highest priority location.
Critically, the LTSS property turns the Berk-Jones from a metric that measures divergence into an objective scoring function that can be efﬁciently maximized over exponentially-many subsets.
SADL, in its base form, searches over the exponentially many subsets of spatial locations, , to ﬁnd spatially-aligned subset of p-values, S ⇤ , with the highest score, F ⇤ :
[Equation]
where F ( S ) is a NPSS that satisﬁes the LTSS property such as the Berk-Jones scan statistic.
Spatial-Channel Optimization (SCO)
The base version of SADL optimizes a scoring function over the 2 wh  1 possible subsets of spatial locations in a tensor with w ⇥ h ⇥ c dimensions and is forced to include all c channels in the returned subset.
See the top part of Figure 2 for a visual example.
This forced inclusion of all channels naively implies that all channels in the inner layer being scanned are impacted by a patch attack on an optical ﬂow neural network.
The more complicated reality is that only some subset of the c channels may be affected by the attack.
By introducing
Spatial-Channel Optimization features to SADL we are returning a subset of spatial locations r max crossed with a subset of channels c max .
Without SCO in place, SADL is forced to return a subset of spatial locations spanning all channels.
Including SCO in the search process can be done through two additional insights.
The ﬁrst is recognizing that the same process that maximizes a scoring function over subsets of spatial locations can also be used to maximize over a subset of channels.
In other words, the LTSS property can be applied across a different mode of the tensor, P .
The second is using an iterative ascent procedure that optimizes over spatial locations ﬁrst, then over channels, and then back to spatial locations.
Each step of the ascent is conditioned on the highest-scoring subset found so far, and this ascent is guaranteed to converge to a local maxima such that any change to either the subset of spatial locations or a subset of channels would decrease the score (McFowland et al., 2013).
Two random restarts are used to approach the global maximum.
Proximity Constraint (PC) While SCO allows us to ﬁnd an anomalous subset of node activations across spatial locations, these detected anomalous locations may be far apart from each other, spanning the entire frame of the given test image pairs.
We want to further constrain the subset scanning so that the detected anomalous locations strictly come from a local spatial neighborhood.
Thus, we enforce a proximity constraint where our optimization is only applied to the k ⇥ k spatial region (yellow region in Figure 2) within the given h ⇥ w ⇥ c p-values where k << h and k << w (see
Algorithm 1).
This will ensure that the detected subset of anomalous locations only comes from this k ⇥ k spatial neighborhood.
We do this for all k ⇥ k locations in the given h ⇥ w locations with a stride of s = k/ 2 , and return the subset of p-values from a k ⇥ k neighborhood that yields the highest NPSS score F max .
Patch Attack Localization Given the subset of detected p-values from a k ⇥ k neighborhood with
F max , we can easily localize the attack by ﬁnding which locations these detected p-values occur in the inner layer feature map.
These detected locations in the feature maps can be up-sampled, if needed, to match the resolution of the input images, or vice versa.
For instance, if a pixel at location (10 , 10) is detected as anomalous in a feature map of size 20 ⇥ 20 ⇥ c under consideration, then
SADL predicts 3 ⇥ 3 attack to be in the input images of size 60 ⇥ 60 ⇥ 3 centered at location (30 , 30) .
The localized attacks can be utilized for attack mitigation, which we leave for future work.
E XPERIMENTAL S ETUP
Flow Networks We validate our approach using four state-of-the-art ﬂow estimators, namely,
FlowNetC (Dosovitskiy et al., 2015), FlowNet2 (Ilg et al., 2017), PWCNet (Sun et al., 2018), and
RAFT (Teed & Deng, 2020).
When selecting inner layers to apply our proposed method, we choose the ﬁrst layer in each of the network’s components.
Speciﬁcally, we select the ﬁrst layer of the encoder and decoder module, and their correlation layer.
FlowNet2 contains four FlowNet-based models and a fusion module, and we apply our proposed subset scanning on the ﬁrst layer in each of the components.
For PWCNet, we select the ﬁrst layer from the encoder , the cost volume layer and the optical ﬂow estimator.
Finally, for RAFT, we select the ﬁrst layer in the feature encoder and context encoder, and correlation and ﬂow layer in the iterative update block.
For layers that process multiple features, (e.g. Siamese encoder of Teed & Deng (2020)), we concatenate the features across the channel dimension and apply subset scanning on the concatenated features.
Datasets Following the literature (Schrodi et al., 2022; Teed & Deng, 2020), we use KITTI 2015 Menze & Geiger (2015), raw KITTI Geiger et al.
(2013), MPI-Sintel Butler et al.
(2012), and raw Sintel Liu et al.
(2019) datasets.
KITTI consists of road scene images with sparse optical ﬂow labels (KITTI 2015) and without labels (raw KITTI).
MPI-Sintel contains 23 sequences from computer-animation short “Sintel” with ﬂow labels.
Its raw frames without labels have also been used in previous un- or semi-supervised ﬂow estimators (Liu et al., 2019; 2020; Yuan et al., 2022).
Generating patch-based attacks Following Ranjan et al.
and Schrodi et al.
, we construct patch-based adversarial attacks on the four ﬂow networks.
Table 1 shows the change in end-point error (EPE) with and without the adversarial patch attack.
We use patches of four different sizes ( p ) in respect to the input image resolution, 4.8% ( 153 ⇥ 153 ), 2.1% ( 102 ⇥ 102 ), 0.5% ( 51 ⇥ 51 ), 0.1% ( 25 ⇥ 25 ).
These patches are trained using KITTI raw and Sintel raw datasets for each network, and evaluated on the labeled test set of KITTI 2015 and on MPI-Sintel, respectively.
As expected, we see worse performances as we increase the patch attack sizes.
In terms of ﬂow networks, these patch attacks harm the performance of FlowNetC the most and RAFT the least (see Figure 1).
[Table]
Performance Evaluation We use the labeled data of Sintel and KITTI datasets to evaluate the attack detection and localization method.
For KITTI 2015 dataset, we use the test split of its data as the training split has been used in training the existing ﬂow networks.
The test split of KITTI 2015 consists of 200 image pairs, and we use 100 of these image pairs as background image pairs ( I H 0and
I H 0 2 ) to obtain our expected distribution of activation, the other 100 sample pairs as our test images ( I t 1 and I t 2 ), where 50 are non-attacked clean image pairs and the other 50 are anomalous image pairs with the patch attacks.
Similarly, for Sintel dataset, we randomly select 200 image pairs for evaluation, 100 image pairs for background image pairs ( I H 0and
I H 0 2 ), 50 sample pairs for anomalous test samples, and the last 50 sample pairs for clean test samples ( I t 1 and I t 2 ).
To evaluate the detection power of adversarial attacks, we follow the subset scanning in neural networks literature Cintas et al.
Kim et al.
and use the area under the receiver operating characteristic curve (AUC).
For localization of the attacks, we report average precision (AP) and average recall (AR), where we compute precision and recall of the correctly detected patch attack locations for each sample and average over the entire test set.
Here, the correctly detected locations occur when the detected anomalous locations on the given inner layer feature overlap with the true attacked location in the input space, down-sampled to match the resolution of the feature.
R ESULTS
In this section, we show our results on the detection and localization of adversarial patch attacks on optical ﬂow estimation.
As SADL is the ﬁrst work to detect and localize these patch attacks, we use a naive scanning method without any spatial constraints as the baseline method.
Patch Attack Detection We show in Table 2 the performance of patch attack detection in terms of AUC comparing the proposed SADL (ours) with the baseline (Cintas et al.
Overall, we see higher detection performance with our proposed method (bolded).
Figure 3 visualizes the distribution of F max of the clean (blue) and attacked (orange) test set corresponding to the values in Tablefor a patch attack of size p = 153 on KITTI 2015.
The two distributions are more separable using SADL than the baseline.
While our method shows clear improvements across multiple networks and datasets, both methods suffer on MPI-Sintel for FlowNetC and on KITTI 2015 for PWCNet.
We show in Section 6 that the hyper-parameter k can be optimized to improve their performance.
[Table]
SADL (ours) 0. FlowNetC FlowNet
PWCNet RAFT Cintas et al [Figure]
[Figure]
Patch Attack Localization Table 3 lists the performance of patch attack localization in terms of average precision and average recall (see Section 4).
As expected, we see higher localization performances for larger patches.
While our method sometimes fails to localize patch attacks of size p = 51 , these small patch attacks comprise 0.5% of the input images and do not have much effect on the ﬂow estimation performance as shown in Table 1.
Similar to attack detection results above,
SADL struggles to localize attacks on KITTI 2015 for PWCNet and on MPI-Sintel for FlowNetC, partly because our hyper-parameters are not optimized (see Section 6).
Figure 5 shows an example visualization of a detected subset of anomalous locations in the feature space for the attacks of size p = 153 for KITTI 2015 (top) and MPI-Sintel (bottom).
SADL successfully detects a subset of anomalous locations that align with the location of the patch attack.
[Table]
A BLATION S TUDY
Performance across Layers Figure 4 shows the change in AUC for detecting 153 ⇥ 153 patch attacks as we go deeper in the four ﬂow networks, i e., FlowNetC, FlowNet2, PWCNet, RAFT, on KITTI 2015 and MPI-Sintel dataset.
For FlowNetC, we see better detection power as we go deeper into the network.
This supports the claim made by Schrodi et al.
(2022) that states that
FlowNetC ampliﬁes the effect of the patch attack going deeper into the decoder layers, making it easier to detect these anomalous behaviors in the deeper layers.
The ﬂow estimation by FlowNetC in
Figure 1 also shows this pattern, where the effect of the patch attack is ampliﬁed to its neighborhood.
For the other three networks, FlowNet2, PWCNet and RAFT, we don’t see this amplifying effect in the prediction, and they show better detection power in the earlier layers of the network.
This observation aligns with Cintas et al.
(2020), which shows that adversarial attacks can be better detected in the earlier layers of the network before their effects get saturated in the deeper layers.
Performance without the Proposed Components We show the performance changes in detection (Table 4) and localization (Fig. 5), without our proposed components, i.e., proximity constraint (PC) and spatial-channel optimization (SCO).
Without PC, the detected subset of anomalous locations could be spatially far apart, which would make it hard to localize the patch attacks in the inputs.
Without SCO, the anomalous score F max are computed using the activation across the entire channel in the detected locations, including the channels that are not anomalous.
Overall, the use of the proposed components (SCO+PC) is shown to achieve the highest detection performance across different networks and patch sizes.
Figure 5 further demonstrates the capability of the proposed PC component to successfully ﬁnd the most anomalous local region where the attack occurs.
Dataset
[Table]
RAFT
[Table] [Table]
[Table] [Table]
Impact of Local Region Size for Proximity Constraint Figure 6 shows the effect of k used in our proximity constraint on the AUC of attack detection on FlowNetC. Speciﬁcally, the x-axis of the
SCO + PC
FlowNetC
FlowNet
PWCNet
RAFT
[Figure]
C ONCLUSION
We propose SADL, the ﬁrst method to detect and localize patch-based adversarial attacks in optical ﬂow estimators.
We use a spatially constrained subset scanning on the inner layer in an unsupervised manner without any training nor prior knowledge of the attacks.
We further give insights on which layers are most affected by these attacks for various ﬂow networks.
We validated SADL with multiple ﬂow networks and datasets and compared it with a baseline method, across which SADL consistently performed well.
Immediate next step could utilize detected and localized attacks to devise mitigation techniques for ﬂow estimators.
Future work also aims to explore various hyperparameters and further analyze why the performances differ across various networks and datasets.