CAMOU: Learning Physical Vehicle Camouflages to Adversarially Attack Detectors in the Wild

In this paper , we conduct an intriguing experimental study about the physical adversarial attack on object detectors in the wild . In particular , we learn a camouflage pattern to hide vehicles from being detected by state-of-the-art convolutional neural network based detectors . Our approach alternates between two threads . In the first , we train a neural approximation function to imitate how a simulator applies a camouflage to vehicles and how a vehicle detector performs given images of the camouflaged vehicles . In the second , we minimize the approximated detection score by searching for the optimal camouflage . Experiments show that the learned camouflage can not only hide a vehicle from the image-based detectors under many test cases but also generalizes to different environments , vehicles , and object detectors . 1 INTRODUCTION . Is it possible to paint a unique pattern on a vehicle ’ s body and hence hide it from being detected by surveillance cameras ? We conjecture the answer is affirmative mainly for two reasons . First , deep neural networks will be widely used in modern surveillance and autonomous driving systems for automatic vehicle detection . Second , unfortunately , these neural networks are intriguingly vulnerable to adversarial examples ( Akhtar & Mian , 2018 ) . Szegedy et al . ( 2013 ) found that adding imperceptible perturbations to clean images can result in the failure of neural networks trained for image classification . This motivates a rich line of work on developing defense techniques for the neural networks ( Akhtar & Mian , 2018 ) and powerful attack methods to defeat those defenses ( Athalye et al. , 2018a ) . Moreover , the adversarial attack has been extended to other tasks , such as semantic segmentation ( Arnab et al. , 2018 ) , object detection ( Xie et al. , 2017 ) , image captioning ( Chen et al. , 2018a ) , etc . It is worth noting that the adversarial examples in the works mentioned above are not physical , i.e. , the adversary directly manipulates image pixels . Although it is arguably more challenging to create physical adversarial objects than to produce adversarial images , some existing works have shown promising results with adversarial patches ( Brown et al. , 2017 ) , stop signs ( Eykholt et al. , 2018b ; Chen et al. , 2018b ) , and small objects like baseballs and 3D turtle models ( Athalye et al. , 2018b ) . To this end , we are reasonably optimistic about designing a special pattern to camouflage a 3D car , in order to make it difficult to detect by the deep learning based vehicle detectors . It is undoubtedly challenging to run experiments in the real world considering financial and time constraints . In this paper , we instead demonstrate results using a simulation engine ( Unreal ) with a high-fidelity 3D sedan model and a 3D SUV . Fig . 1 shows that the vehicle in the simulation is photo-realistic such that , even covered with random camouflage , it can still be detected by the Mask R-CNN detector ( He et al. , 2017 ) trained on COCO ( Lin et al. , 2014 ) . The simulation engine enables us to test the physically adversarial cars under a considerable spectrum of environmental conditions : lighting , backgrounds , camera-to-object distances , viewing angles , occlusions , etc . In contrast , existing experiments on physical adversarial attacks are all executed in simplified scenarios . Eykholt et al . ( 2018b ) , Eykholt et al . ( 2018a ) , and Chen et al . ( 2018b ) attack neural classifiers and detectors of stop signs . While projective transformations could be used to render the planar stop signs to various images , it is more involved to image the nonplanar 3D vehicles considered in this paper ; we learn a neural approximation function instead . Athalye et al . ( 2018b ) synthesize objects ( e.g. , baseball , turtle , etc . ) which are adversarial within a small range of camera-to-object distances and viewing angles . Given a 3D vehicle model in the simulation engine , we learn a camouflage for it by following the expectation over transformation ( EoT ) principle first formalized by Athalye et al . ( 2018b ) . The main idea is to consider a variety of transformations under which the camouflage can consistently hide the vehicle from a neural detector . A transformation imitates the imaging procedure and produces an image of the 3D vehicle model in the simulated environment . If a camouflage works under many transformations seen in the training phase , it is expected to also generalize to unseen transformations in the test phase . One of the major challenges is that the simulator ’ s image generation procedure is non-differentiable . A seemingly plausible solution is to train a neural network to approximate this procedure . The network takes as input the environment , a camouflage pattern , and the 3D vehicle model and outputs an image as close as possible to the one rendered by the simulator . Although this approach is viable , it is extremely difficult to generate high-resolution images . State-of-the-art methods ( e.g. , RenderNet ( Nguyen-Phuoc et al. , 2018 ) ) can only generate simple 3D objects without any backgrounds . We tackle the above challenge by drawing the following observation . In EoT ( Athalye et al. , 2018b ; Chen et al. , 2018b ) , the gradients propagate back to the physical object from the detector/classifier ’ s decision values . If we jointly consider the object detector and the imaging procedure of the simulator as a whole black box , it is easier to learn a function to approximate this black box ’ s behavior than to train the image generation neural network . Hence , we learn a substitute neural network which takes as input a camouflage , the vehicle model , and the environment and outputs the vehicle detector ’ s decision value . Equipped with this substitute network , we can readily run the EoT algorithm ( Athalye et al. , 2018b ) over our simulator in order to infer an adversarial camouflage for the vehicles . Finally , we make some remarks about the significance and potential impact of this work . In the real world , multiclass visual object detection neural networks ( He et al. , 2017 ; Redmon & Farhadi , 2018 ) have become the cornerstone of multiple industrial applications , such as surveillance systems ( Nair et al. , 2018 ) , autonomous driving ( Huval et al. , 2015 ) , and military systems ( Pellerin , 2017 ) . Among these applications , cars are one of the most crucial objects . Attacking vehicle detectors in the physical world will be enormously valuable and impactful from the perspective of the malicious adversaries . Compared with the stop sign , it is legal in the United States to paint a car while defacing a stop sign is criminal . This poses a more significant threat to autonomous driving systems since anyone has access to perturb public machine learning based systems legally . This observation motivates us to focus our approach on cars . We limit our camouflage within the legally paintable car body parts , which means that we will leave discriminative visual cues , such as the tires , windows , grille , lights , etc . unaltered for the detectors . 2 RELATED WORK . 2.1 ADVERSARIAL ATTACK AND ITS GENERALIZATION . Currently , the adversarial attack is powerful enough to attack image classification ( MoosaviDezfooli et al. , 2017 ) , object detection ( Arnab et al. , 2018 ) , semantic segmentation ( Xie et al. , 2017 ) , audio recognition ( Carlini & Wagner , 2018 ) and even bypass most of the defense mechanism ( Athalye et al. , 2018a ) . The mainstream adversarial machine learning research focuses on the in silico ones or the generalization within in silico ( Liu et al. , 2017 ) . Such learned perturbations are practically unusable in the real world as shown in the experiments by Lu et al . ( 2017b ) , who found that almost all perturbation methods failed to prevent a detector from detecting real stop signs . The first physical world adversarial attack by Kurakin et al . ( 2016 ) found perturbations remain effective on the printed paper . Eykholt et al . ( 2018b ) found a way to train perturbations that remain effective against a classifier on real stop signs for different viewing angles and such . Athalye et al . ( 2018b ) trained another perturbation that successfully attacks an image classifier on 3D-printed objects . However Lu et al . ( 2017c ) found that Eykholt et al . ( 2018b ) ’ s perturbation does not fool the object detectors YOLO9000 ( Redmon & Farhadi , 2017 ) and Faster RCNN ( Ren et al. , 2015 ) . They argue that fooling an image classifier is different and easier than fooling an object detector because the detector is able to propose object bounding boxes on its own . In the meanwhile , Lu et al . ( 2017a ) ’ s and Chen et al . ( 2018b ) ’ s work could be generalized to attack the stop sign detector in the physical world more effectively . However , all the above methods aim to perturb detecting the stop signs . Blackbox attack is another relevant topic . Among the current blackbox attack literature , Papernot et al . ( 2017 ) trained a target model substitution based on the assumption that the gradient between the image and the perturbation is available . We do not have the gradient since the simulation is nondifferentiable . Chen et al . ( 2017 ) proposed a coordinate descent to attack the model . However , we found that our optimization problem is time-consuming , noisy ( see Sec . G.2 ) , empirically nonlinear and non-convex ( see Sec . H ) . Time constraints made this approach unavailable to us since it requires extensive evaluations during coordinate descent . Besides , coordinate descent generally requires precise evaluation at each data-point . 2.2 SIMULATION AIDED MACHINE LEARNING . Since the dawn of deep learning , data collection has always been of fundamental importance as deep learning model performance is generally correlated with the amount of training data used . Given the sometimes unrealistically expensive annotation costs , some machine learning researchers use synthetic data to train their models . This is especially true in computer vision applications since state-of-the-art computer-generated graphics are truly photo-realistic . Ros et al . ( 2016 ) ; Richter et al . ( 2017 ) proposed synthetic datasets for semantic segmentation . Gaidon et al . ( 2016 ) proposed virtual KITTI as a synthetic replica of the famous KITTI dataset ( Geiger et al. , 2012 ) for tracking , segmentation , etc . And Varol et al . ( 2017 ) proposed using synthetic data for human action learning . Zhang et al . ( 2017 ) ; Bousmalis et al . ( 2017 ) adapt RL-trained robot grasping models from the synthetic environment to the real environment . Tremblay et al . ( 2018 ) trained a detection network using the same Unreal engine that we are using . 3 PROBLEM STATEMENT . In this paper , we investigate physical adversarial attack on state-of-the-art neural network based object detectors . The objective is to find a camouflage pattern such that , when it is painted on the body of a vehicle , the Mask R-CNN ( He et al. , 2017 ) and YOLO ( Redmon & Farhadi , 2018 ) detectors fail to detect the vehicle under a wide spectrum of variations ( e.g. , locations , lighting conditions , viewing angles , etc. ) . We learn the camouflage in a black-box fashion , without the need for accessing the detectors ’ network architecture or weights . Expectation-over-transformation ( EoT ) . We formalize the physical adversarial attack problem with the EoT framework ( Athalye et al. , 2018b ) . Denote by t a transformation which converts a camouflage pattern c to a real photo . Such a transformation actually represents an involved procedure : paint the pattern to a vehicle ’ s body , drive the vehicle to a location , configure a camera , and take a picture of the vehicle . The transformation conveniently abstracts away enormous factors ( e.g. , quality of painting , camera , etc . ) , each of which could play a role in this procedure . Denote by Vt ( c ) the detection score ( e.g. , by Mask-RCNN ) over an image which is a result of a transformation t and a camouflage c. The physical adversarial attack problem is then described by argmin c Et∼T Vt ( c ) ( 1 ) where T is a distribution over all the possible transformations { t } . In other words , we search for a camouflage c that minimizes the vehicle detection score in expectation . Transformation in simulation . Due to financial and time constraints , we conduct our study with the photo-realistic Unreal 4 game engine . It supplies us sufficient configuration parameters , such as the resolution and pattern of the camouflage , 3D models of vehicles , parameters of cameras and environments , etc . Fig . 2 shows a camouflage pattern , a high-fidelity 3D model of Toyota Camry , a corner of the virtual city used in this paper , and finally the picture taken by a camera after we apply the camouflage to the Camry and drive it to that corner — in other words , the rightmost image is the result of a certain transformation t acting on the three images on the left of Fig . 2 .

Summary: The authors investigate the problem of learning a camouflage pattern which, when applied to a simulated vehicle, will prevent an object detector from detecting it. The problem is frames as finding the camouflage pattern which minimises the expected decision scores from the detector over a distribution of possible views of this pattern applied to a vehicle (e.g. vantage point, background, etc). This expectation is approximated by sampling detector scores when applying the detector considered to images synthesised using a number of vantage points and scene contexts. In order to generate a gradient signal with respect to the camouflage applied (the simulated image rendering is non-differentiable) the approach considers learning a clone network which takes as input the camouflage pattern, the vehicle model and a given environment and outputs the vehicle detector’s devision values. The clone network and the optimal camouflage are learned alternately in order to obtain a relatively faithful approximation. The approach is evaluated in simulation using two standard object detectors (Mask R-CNN and YOLOv3) on two vehicle models over a range of transformations.