Attacking Optical Flow
Andreas Geiger
Autonomous Vision Group
University of Tübingen / MPI for Intelligent Systems Tübingen

August 23, 2020

University of Tübingen
MPI for Intelligent Systems
Autonomous Vision GroupCollaborators

Anurag Ranjan

Joel Janai

Aditya Prakash

Eshed Ohn-Bar

Kashyap Chitta

Aseem Behl

Michael Black

Andreas Geiger

2

Self-Driving must be RobustSelf-Driving must be Robust

4

Self-Driving must be Robust

Robustness to environment

Robustness to perturbations

5

Situational Driving

Proposed
Situational
2. Context
EmbeddingDriving (LSD) Framework
EnvironmentLearningqÁz

1. Expert Policies1

¼µ

2

¼µ:::

3. Task-Driven
Refinement¼KµDemonstrations

I Learn mixture of imitation experts {πθk }I Task-driven policy reﬁnement

Ohn-Bar, Prakash, Behl, Chitta and Geiger: Learning Situational Driving. CVPR, 2020.

6

Data Aggregation

I Data aggregation: query expert for difﬁcult situations

Prakash, Behl, Ohn-Bar, Chitta and Geiger: Exploring Data Aggregation in Policy Learning for Vision-based Urban Autonomous Driving. CVPR, 2020.

7

Self-Driving must be Robust

Robustness to environment

Robustness to perturbations

8Adversarial Attacks on Image Classiﬁcation

L-BFGS Attack:I Given classiﬁer f : Rm → {1, . . . , L}
I Find adversarial example for image x:
x + argmin {k∆xk2 : f (x + ∆x) 6= f (x)}∆x

I All images classiﬁed as “ostrich”(right column)

Szegedy, Zaremba, Sutskever, Bruna, Erhan, Goodfellow and Fergus: Intriguing properties of neural networks. ICLR, 20149Adversarial Attacks on Semantic Segmentation

I Attack on semantic segmentation manipulates label map
Metzen, Kumar, Brox and Fischer: Universal Adversarial Perturbations Against Semantic Image Segmentation. ICCV, 2017.

10

Physical Adversarial Attacks“Adversarial perturbation methods applied to stop sign detection only work in carefully
chosen situations, and our preliminary experiment shows that we might not need toworry about it in many real circumstances, speciﬁcally with autonomous vehicles.”Lu, Sibai, Fabry and Forsyth: NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles. Arxiv, 2017.

11

Robust Adversarial Attacks

I Demonstrate existence of robust adversarial examples in the physical worldI Maximize expectation over transformation T (EOT):
argmax Et∼T log P (y|t(x0 )) − λk(t(x0 ) − t(x)k2
x0I Larger distributions require larger perturbations
Athalye, Engstrom, Ilyas and Kwok: Synthesizing Robust Adversarial Examples. ICML, 2018.

12

Robust Adversarial Attacks

I Robust adversarial example designed to mimic “grafﬁti”
Eykholt, Evtimov, Fernandes, Li, Rahmati, Xiao, Prakash, Kohno and Song: Robust Physical-World Attacks on Deep Learning Visual Classiﬁcation. CVPR, 2018.

13

Adversarial Patch Attacks

I Patch attacks use EOT idea, but also optimize across many images
I Easy to apply in real-world settings (attaching patch to an object)
Brown, Mane, Roy, Abadi and Gilmer. Adversarial patch. Arxiv, 2017.

14

Optical Flow

Low-Level Perception

Low-Level Perception

Optical Flow16

Motion Estimation

I Accurate motion estimation is critical for self-driving
I Allows for making predictions about the future
Wedel, Brox, Vaudrey, Rabe, Franke and Cremers: Stereoscopic scene ﬂow computation for 3D motion understanding. IJCV, 2011.

17

Optical FlowI Optical ﬂow describes the 2D pixel motion between two frames
I Optical ﬂow contains information about 3D geometry and 3D motion
Gibson: The Perception of the Visual World. 1950.

18

Approaches to Optical Flow

Variational Optical Flow

Variational Energy:
ZZ
E(u, v) =(I(x + u(x, y), y + v(x, y), t + 1) − I(x, y, t))2|{z}
penalize brightness change+ λ · k∇u(x, y)k2 + k∇v(x, y)k2 dx dy|{z}penalize ﬂow change

Horn and Schunck: Determining optical ﬂow. Artiﬁcial Intelligence, 1981.

20

Encoder-Decoder Networks

FlowNet 2.0:
I Multiple stacked encoder-decoder networks
Ilg, Mayer, Saikia, Keuper, Dosovitskiy and Brox: FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks. CVPR, 2017.

21

Spatial Pyramid Networks

PWC-Net:
I Coarse-to-ﬁne optical ﬂow estimation
Sun, Yang, Liu and Kautz: PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume. CVPR, 2018.

22

Motivation

Optical ﬂow on KITTI dataset computed via FlowNet2 [Ilg et al., CVPR 17]
I Until 2016: Classic variational optical ﬂow methods state-of-the-art
I Since 2016: Leaderboards dominated by deep learning based methods
I But so far no investigation of adversarial robustness of optical ﬂow approaches

23

Attacking Optical Flow

Attacking Optical Flow

Given an image pair from a video sequence ...

Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

25

Attacking Optical Flow

Given an image pair from a video sequence ...

Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

25

Attacking Optical Flow... FlowNet2 predicts a smooth optical ﬂow ﬁeld.

Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

25

Attacking Optical Flow

Can we obtain a small (< 1%) attack patch p̂ ...

Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

25

Attacking Optical Flow... that successfully attacks the optical ﬂow network?

Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

25

Attacking Optical Flow
Let F (I, I 0 ) denote an optical ﬂow network and I a dataset of frame pairs.
Let A(I, p, t, l) denote image I with patch p transformed by t inserted at location l.
Let T denote a distribution over afﬁne 2D transformations.
Let L denote a (uniform) distribution over the image domain.
Goal: Find a patch p̂ such thatp̂ = argmin E(I,I 0 )∼I, t∼T , l∼Lp

with(u, v) · (ũ, ṽ)k(u, v)k2 · k(ũ, ṽ)k2(u, v) = F (I, I 0 )(ũ, ṽ) = F (A(I, p, t, l), A(I 0 , p, t, l))Intuition: Find patch p̂ that reverses the direction of the optical ﬂow.
Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

26

Results

White-Box Attacks
Setting:
I Attack each network in separation
I Location sampled uniformly within image
I Scale: ±5%
I Rotation: ±10%
I Optimize on 32k unlabeled KITTI frames
I Flow predictions as pseudo ground truth
I Learn patches of 4 different sizes

Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

28

White-Box Attacks

Network

Unattacked
EPE

25x25
EPE
Rel

153x153
EPE
Rel

FlowNetC [Fischer et al. ]

14.56

29.07

+100 %

95.32+555 %

FlowNet2 [Ilg et al. ]11.90

17.04+43 %59.58+400 %SpyNet [Ranjan and Black]

20.26

20.59+2 %21.00+4 %PWCNet [Sun et al. ]11.03

11.37+3 %12.52+13 %Back2Future [Janai et al. ]

17.49

18.04

+3 %

18.43+5 %I FlowNetC and FlowNet2 use encoder-decoder architectures
I SpyNet, PWCNet and Back2Future use a spatial pyramid

Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

29

White-Box Attacks

I Attacks extend beyond region of patch for encoder-decoder architectures
Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

30

White-Box Attacks

I Attacks extend beyond region of patch for encoder-decoder architectures
Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

30

Black-Box Attacks

Setting:
I Patch is optimized over several networks(we use FlowNet2 and PWCNet)
I Patch is used to attack all networks
I Patch is moved as if it was part of the scene

Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

31

Black-Box Attacks

UnattackedEPE

Attacked
EPE

AttackedRel

FlowNet2 [Fischer et al. ]11.90

30.99+160 %PWCNet [Ilg et al. ]11.03

11.16+1 %FlowNetC [Ranjan and Black]

14.56

77.78+434 %SpyNet [Sun et al. ]20.26

20.65+2 %Back2Future [Janai et al. ]17.49

17.76+2 %Epic Flow [Revaud et al. ]4.52

4.57+1 %LDOF [Brox and Malik]

9.20

9.30

+1 %

I FlowNetC and FlowNet2 use encoder-decoder architectures
I SpyNet, PWCNet and Back2Future use a spatial pyramid
I Epic Flow and LDOF are classical variational approaches
Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

32

Black-Box Attacks

Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

33

Real-World Attack

I Printed patch attached to desk lamp
Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

34

Insights into Attacks

Zero-Flow Test

What happens when CNN sees
I Identical images?
I Identical images with identical attack patch?
Ideally
I The network should output a zero ﬂow ﬁeld
I The feature maps of the attacked and unattacked images should be similarRanjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

36

Zero-Flow Test

I Feature activations are not spatially invariant, even without an attack
I Deconvolution layers cause checkerboard artifacts
I Feature maps of encoder-decoder architectures are very different
Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

37

Zero-Flow Test

I Feature activations are not spatially invariant, even without an attack
I Deconvolution layers cause checkerboard artifacts
I The pyramid networks predict large motion in coarser ﬂow levels
Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

38

SummarySummary

I Placing a patch in the scene may lead to failure of optical ﬂow networks
I Patch attacks are invariant to translation and small changes in scale and rotation
I Patch attacks work in the physical world (to some extent)
I Encoder-decoder architectures like FlowNetC and FlowNet2 are strongly affected
I Spatial pyramid methods like SPyNet and PWC-Net are quite robust
I Classical methods like LDOF and EpicFlow are not affected
I Zero-Flow test can provide insights into networks

Ranjan, Janai, Geiger and Black: Attacking Optical Flow. ICCV, 2019.

40

Thank you!
http://autonomousvision.github.io