CAMOU: Learning Physical Vehicle Camouflages to Adversarially Attack Detectors in the Wild

In this paper , we conduct an intriguing experimental study about the physical adversarial attack on object detectors in the wild . In particular , we learn a camouflage pattern to hide vehicles from being detected by state-of-the-art convolutional neural network based detectors . Our approach alternates between two threads . In the first , we train a neural approximation function to imitate how a simulator applies a camouflage to vehicles and how a vehicle detector performs given images of the camouflaged vehicles . In the second , we minimize the approximated detection score by searching for the optimal camouflage . Experiments show that the learned camouflage can not only hide a vehicle from the image-based detectors under many test cases but also generalizes to different environments , vehicles , and object detectors . 1 INTRODUCTION . Is it possible to paint a unique pattern on a vehicle ’ s body and hence hide it from being detected by surveillance cameras ? We conjecture the answer is affirmative mainly for two reasons . First , deep neural networks will be widely used in modern surveillance and autonomous driving systems for automatic vehicle detection . Second , unfortunately , these neural networks are intriguingly vulnerable to adversarial examples ( Akhtar & Mian , 2018 ) . Szegedy et al . ( 2013 ) found that adding imperceptible perturbations to clean images can result in the failure of neural networks trained for image classification . This motivates a rich line of work on developing defense techniques for the neural networks ( Akhtar & Mian , 2018 ) and powerful attack methods to defeat those defenses ( Athalye et al. , 2018a ) . Moreover , the adversarial attack has been extended to other tasks , such as semantic segmentation ( Arnab et al. , 2018 ) , object detection ( Xie et al. , 2017 ) , image captioning ( Chen et al. , 2018a ) , etc . It is worth noting that the adversarial examples in the works mentioned above are not physical , i.e. , the adversary directly manipulates image pixels . Although it is arguably more challenging to create physical adversarial objects than to produce adversarial images , some existing works have shown promising results with adversarial patches ( Brown et al. , 2017 ) , stop signs ( Eykholt et al. , 2018b ; Chen et al. , 2018b ) , and small objects like baseballs and 3D turtle models ( Athalye et al. , 2018b ) . To this end , we are reasonably optimistic about designing a special pattern to camouflage a 3D car , in order to make it difficult to detect by the deep learning based vehicle detectors . It is undoubtedly challenging to run experiments in the real world considering financial and time constraints . In this paper , we instead demonstrate results using a simulation engine ( Unreal ) with a high-fidelity 3D sedan model and a 3D SUV . Fig . 1 shows that the vehicle in the simulation is photo-realistic such that , even covered with random camouflage , it can still be detected by the Mask R-CNN detector ( He et al. , 2017 ) trained on COCO ( Lin et al. , 2014 ) . The simulation engine enables us to test the physically adversarial cars under a considerable spectrum of environmental conditions : lighting , backgrounds , camera-to-object distances , viewing angles , occlusions , etc . In contrast , existing experiments on physical adversarial attacks are all executed in simplified scenarios . Eykholt et al . ( 2018b ) , Eykholt et al . ( 2018a ) , and Chen et al . ( 2018b ) attack neural classifiers and detectors of stop signs . While projective transformations could be used to render the planar stop signs to various images , it is more involved to image the nonplanar 3D vehicles considered in this paper ; we learn a neural approximation function instead . Athalye et al . ( 2018b ) synthesize objects ( e.g. , baseball , turtle , etc . ) which are adversarial within a small range of camera-to-object distances and viewing angles . Given a 3D vehicle model in the simulation engine , we learn a camouflage for it by following the expectation over transformation ( EoT ) principle first formalized by Athalye et al . ( 2018b ) . The main idea is to consider a variety of transformations under which the camouflage can consistently hide the vehicle from a neural detector . A transformation imitates the imaging procedure and produces an image of the 3D vehicle model in the simulated environment . If a camouflage works under many transformations seen in the training phase , it is expected to also generalize to unseen transformations in the test phase . One of the major challenges is that the simulator ’ s image generation procedure is non-differentiable . A seemingly plausible solution is to train a neural network to approximate this procedure . The network takes as input the environment , a camouflage pattern , and the 3D vehicle model and outputs an image as close as possible to the one rendered by the simulator . Although this approach is viable , it is extremely difficult to generate high-resolution images . State-of-the-art methods ( e.g. , RenderNet ( Nguyen-Phuoc et al. , 2018 ) ) can only generate simple 3D objects without any backgrounds . We tackle the above challenge by drawing the following observation . In EoT ( Athalye et al. , 2018b ; Chen et al. , 2018b ) , the gradients propagate back to the physical object from the detector/classifier ’ s decision values . If we jointly consider the object detector and the imaging procedure of the simulator as a whole black box , it is easier to learn a function to approximate this black box ’ s behavior than to train the image generation neural network . Hence , we learn a substitute neural network which takes as input a camouflage , the vehicle model , and the environment and outputs the vehicle detector ’ s decision value . Equipped with this substitute network , we can readily run the EoT algorithm ( Athalye et al. , 2018b ) over our simulator in order to infer an adversarial camouflage for the vehicles . Finally , we make some remarks about the significance and potential impact of this work . In the real world , multiclass visual object detection neural networks ( He et al. , 2017 ; Redmon & Farhadi , 2018 ) have become the cornerstone of multiple industrial applications , such as surveillance systems ( Nair et al. , 2018 ) , autonomous driving ( Huval et al. , 2015 ) , and military systems ( Pellerin , 2017 ) . Among these applications , cars are one of the most crucial objects . Attacking vehicle detectors in the physical world will be enormously valuable and impactful from the perspective of the malicious adversaries . Compared with the stop sign , it is legal in the United States to paint a car while defacing a stop sign is criminal . This poses a more significant threat to autonomous driving systems since anyone has access to perturb public machine learning based systems legally . This observation motivates us to focus our approach on cars . We limit our camouflage within the legally paintable car body parts , which means that we will leave discriminative visual cues , such as the tires , windows , grille , lights , etc . unaltered for the detectors . 2 RELATED WORK . 2.1 ADVERSARIAL ATTACK AND ITS GENERALIZATION . Currently , the adversarial attack is powerful enough to attack image classification ( MoosaviDezfooli et al. , 2017 ) , object detection ( Arnab et al. , 2018 ) , semantic segmentation ( Xie et al. , 2017 ) , audio recognition ( Carlini & Wagner , 2018 ) and even bypass most of the defense mechanism ( Athalye et al. , 2018a ) . The mainstream adversarial machine learning research focuses on the in silico ones or the generalization within in silico ( Liu et al. , 2017 ) . Such learned perturbations are practically unusable in the real world as shown in the experiments by Lu et al . ( 2017b ) , who found that almost all perturbation methods failed to prevent a detector from detecting real stop signs . The first physical world adversarial attack by Kurakin et al . ( 2016 ) found perturbations remain effective on the printed paper . Eykholt et al . ( 2018b ) found a way to train perturbations that remain effective against a classifier on real stop signs for different viewing angles and such . Athalye et al . ( 2018b ) trained another perturbation that successfully attacks an image classifier on 3D-printed objects . However Lu et al . ( 2017c ) found that Eykholt et al . ( 2018b ) ’ s perturbation does not fool the object detectors YOLO9000 ( Redmon & Farhadi , 2017 ) and Faster RCNN ( Ren et al. , 2015 ) . They argue that fooling an image classifier is different and easier than fooling an object detector because the detector is able to propose object bounding boxes on its own . In the meanwhile , Lu et al . ( 2017a ) ’ s and Chen et al . ( 2018b ) ’ s work could be generalized to attack the stop sign detector in the physical world more effectively . However , all the above methods aim to perturb detecting the stop signs . Blackbox attack is another relevant topic . Among the current blackbox attack literature , Papernot et al . ( 2017 ) trained a target model substitution based on the assumption that the gradient between the image and the perturbation is available . We do not have the gradient since the simulation is nondifferentiable . Chen et al . ( 2017 ) proposed a coordinate descent to attack the model . However , we found that our optimization problem is time-consuming , noisy ( see Sec . G.2 ) , empirically nonlinear and non-convex ( see Sec . H ) . Time constraints made this approach unavailable to us since it requires extensive evaluations during coordinate descent . Besides , coordinate descent generally requires precise evaluation at each data-point . 2.2 SIMULATION AIDED MACHINE LEARNING . Since the dawn of deep learning , data collection has always been of fundamental importance as deep learning model performance is generally correlated with the amount of training data used . Given the sometimes unrealistically expensive annotation costs , some machine learning researchers use synthetic data to train their models . This is especially true in computer vision applications since state-of-the-art computer-generated graphics are truly photo-realistic . Ros et al . ( 2016 ) ; Richter et al . ( 2017 ) proposed synthetic datasets for semantic segmentation . Gaidon et al . ( 2016 ) proposed virtual KITTI as a synthetic replica of the famous KITTI dataset ( Geiger et al. , 2012 ) for tracking , segmentation , etc . And Varol et al . ( 2017 ) proposed using synthetic data for human action learning . Zhang et al . ( 2017 ) ; Bousmalis et al . ( 2017 ) adapt RL-trained robot grasping models from the synthetic environment to the real environment . Tremblay et al . ( 2018 ) trained a detection network using the same Unreal engine that we are using . 3 PROBLEM STATEMENT . In this paper , we investigate physical adversarial attack on state-of-the-art neural network based object detectors . The objective is to find a camouflage pattern such that , when it is painted on the body of a vehicle , the Mask R-CNN ( He et al. , 2017 ) and YOLO ( Redmon & Farhadi , 2018 ) detectors fail to detect the vehicle under a wide spectrum of variations ( e.g. , locations , lighting conditions , viewing angles , etc. ) . We learn the camouflage in a black-box fashion , without the need for accessing the detectors ’ network architecture or weights . Expectation-over-transformation ( EoT ) . We formalize the physical adversarial attack problem with the EoT framework ( Athalye et al. , 2018b ) . Denote by t a transformation which converts a camouflage pattern c to a real photo . Such a transformation actually represents an involved procedure : paint the pattern to a vehicle ’ s body , drive the vehicle to a location , configure a camera , and take a picture of the vehicle . The transformation conveniently abstracts away enormous factors ( e.g. , quality of painting , camera , etc . ) , each of which could play a role in this procedure . Denote by Vt ( c ) the detection score ( e.g. , by Mask-RCNN ) over an image which is a result of a transformation t and a camouflage c. The physical adversarial attack problem is then described by argmin c Et∼T Vt ( c ) ( 1 ) where T is a distribution over all the possible transformations { t } . In other words , we search for a camouflage c that minimizes the vehicle detection score in expectation . Transformation in simulation . Due to financial and time constraints , we conduct our study with the photo-realistic Unreal 4 game engine . It supplies us sufficient configuration parameters , such as the resolution and pattern of the camouflage , 3D models of vehicles , parameters of cameras and environments , etc . Fig . 2 shows a camouflage pattern , a high-fidelity 3D model of Toyota Camry , a corner of the virtual city used in this paper , and finally the picture taken by a camera after we apply the camouflage to the Camry and drive it to that corner — in other words , the rightmost image is the result of a certain transformation t acting on the three images on the left of Fig . 2 .

Summary: Adversarial attacks and defences are of growing popularity now a days. As AI starts to be present everywhere, more and more people can start to try to attack those systems. Critical systems such as security systems are the ones that can suffer more from those attacks. In this paper the case of vehicles that attack an object detection system by trying to not be detected are tackled.