CAMOU: L EARNING A V EHICLE C AMOUFLAGE FOR P HYSICAL A DVERSARIAL A TTACK ON O BJECT D ETECTORS IN THE W ILD
Anonymous authors
Paper under double-blind review
A BSTRACT
In this paper, we conduct an interesting experimental study about the physical adversarial attack on object detectors in the wild.
In particular, we learn a camouﬂage pattern to hide vehicles from being detected by state-of-the-art convolutional neural network based detectors.
Our approach alternates between two threads.
In the ﬁrst, we train a neural approximation function to imitate how a simulator applies camouﬂages to vehicles and how a vehicle detector performs given an image generated by the simulator.
In the second, we minimize the approximated detection score by searching for the optimal camouﬂage.
Experiments show that the learned camouﬂage can not only hide a vehicle from the image-based detectors under many cases, but also generalizes to different environments, vehicles, and object detectors.
I NTRODUCTION
Is it possible to paint a unique pattern on a vehicle’s body and hence hide it from being detected by surveillance cameras?
We conjecture that the answer is afﬁrmative mainly for two reasons.
First, deep neural networks will be widely used in modern surveillance and autonomous driving systems for automatic vehicle detection.
Second, unfortunately, these neural networks are intriguingly vulnerable to adversarial examples Akhtar & Mian (2018).
The authors in Szegedy et al.
(2013) found that adding imperceptible perturbations to clean images can result in the failure of neural networks trained for image classiﬁcation.
This motivates a rich line of work on developing defense techniques for the neural networks Akhtar & Mian (2018) and powerful attack methods to defeat those defenses (Athalye et al., 2018a).
Moreover, the adversarial attack has been extended to other tasks, such as semantic segmentation (Arnab et al., 2018), objection detection (Xie et al., 2017), image captioning (Chen et al., 2018a), etc.
It is worth noting that the adversarial examples in the works mentioned above are not physical, i.e. the adversary directly manipulates image pixels.
Although it is arguably more challenging to create physical adversarial objects than producing adversarial images, some existing works have shown promising results with adversarial patches (Brown et al., 2017), stop signs (Eykholt et al., 2018;
Chen et al., 2018b), and small objects like baseballs and 3D turtle models (Athalye et al., 2018b).
To this end, we are reasonably optimistic about designing a special pattern to camouﬂage a 3D car, in order to make it difﬁcult to detect with deep learning-based vehicle detectors.
It is undoubtedly challenging to run experiments in the real world considering ﬁnancial and time constraints.
In this paper, we instead demonstrate results using a simulation Unreal engine with a high-ﬁdelity 3D sedan model and a 3D SUV.
Fig.1 shows that the vehicle in the simulation is realistic enough that even covered with random camouﬂage, it still would be detected by the COCO trained detector.
The simulation engine enables us to test the generated adversarial cars under a considerable spectrum of environmental conditions: lighting, backgrounds, camera-to-object distances, viewing angles, occlusions, etc.
In contrast, existing experiments on physical adversarial attacks are all executed in simpliﬁed scenarios.
Eykholt et al.
and
Chen et al.
(2018b)
respectively attack neural classiﬁers and detectors using the front views of a stop sign.
Athalye et al.
(2018b) synthesize objects (e.g., baseball, turtle, etc.) which are adversarial within a small range of camera-to-object distances and view angles.
Given a 3D vehicle model in the simulation engine, we learn a camouﬂage for it by following the expectation over transformation (EoT) principle of Athalye et al.
(2018b).
The main idea is to consider a variety of transformations under which the camouﬂage can consistently hide the vehicle from a neural detector.
A transformation imitates the imaging procedure and produces an image of the 3D vehicle model in the simulated environment.
If a camouﬂage works under many transformations seen in the training phase, it is expected to also generalize to unseen transformations in the test phase.
One of the challenges in our context is that the simulator’s image generation procedure is nondifferentiable.
A seemingly plausible solution is to train a neural network to approximate this procedure.
The network takes as input the environment, a camouﬂage pattern, the 3D vehicle model and outputs an image as close as possible to the one rendered by the simulator.
Although this approach is viable, it is extremely difﬁcult to generate high-resolution images.
E.g. the current state-of-the-art
RenderNet (Nguyen-Phuoc et al., 2018) is only able to generate single simple 3D object without background.
We tackle the above challenge by drawing the following observation.
In EoT (Athalye et al., 2018b;
Chen et al., 2018b), the gradients propagate back to the physical object from the detector/classiﬁer’s decision values.
If we jointly consider the object detector and the imaging procedure of the simulator as a whole black box, it is easier to learn a function to approximate this black box’s behavior than training the image generation neural network.
As a result, we learn a substitute neural network which takes as input a camouﬂage, a vehicle model, and the environment and outputs the vehicle detector’s decision values.
Equipped with this substitute network, we can readily run the EoT algorithm (Athalye et al., 2018b) over our simulator in order to infer an adversarial camouﬂage for our vehicles.
Finally, we make some remarks about the signiﬁcance and potential impact of this work.
In the real world, multiclass visual object detection neural networks He et al.
(2017); Redmon & Farhadi (2018) have become the cornerstone of multiple industrial applications such as surveillance systems (Nair et al.
autonomous driving (Huval et al.
and military systems (Pellerin (2017)).
Among these applications, the car is one of the most crucial objects.
Attacking vehicle detectors in the physical world will be enormously valuable and impactful from the perspective of the malicious adversaries.
Compared with the stop sign, it is legal in the United States to paint a car while defacing a stop sign is criminal.
This poses a more signiﬁcant threat to autonomous driving systems since anyone has access to perturb public machine learning based system legally.
This observation motivates us to set our goal as the car.
We limit our camouﬂage within the legally paintable car body parts, which means that we will leave crucial visual cues, such as the tires, windows, grille, lights, etc, for the detector.
And this will make our task further challenging.
R ELATED W ORK 2. A DVERSARIAL A TTACK AND ITS G ENERALIZATION
Currently, the adversarial attack is powerful enough to attack image classiﬁcation (MoosaviDezfooli et al.
object detection (Arnab et al.
semantic segmentation (Xie et al.
(2017)), audio recognition (Carlini & Wagner (2018)) and even bypass most of the defense mechanism (Athalye et al.
(2018a)).
The mainstream adversarial machine learning research focuses on the in silico ones or the generalization within in silico (Liu et al.
Such learned perturbations are practically unusable in the real world as shown in the experiment by Lu et al.
(2017b) who found that almost all perturbation methods failed to prevent a detector from detecting real stop signs.
The ﬁrst physical world adversarial attack by Kurakin et al.
(2016) found perturbations remain effective on the printed paper.
Eykholt et al.
(2018) found a way to train perturbations that remain effective against a classiﬁer on real stop signs for different view angles and such.
Athalye et al.
(2018b) trained another perturbation that successfully attacks an image classiﬁer on 3D-printed objects.
However Lu et al.
(2017c) found that Eykholt et al.
’s perturbation does not fool the object detectors (YOLO9000 (Redmon & Farhadi, 2017) and Faster RCNN (Ren et al., 2015)).
They argue that fooling an image classiﬁer is different and easier than fooling an object detector because the detector is able to propose object bounding boxes on its own.
In the meanwhile, Lu et al.
(2017a)’s
and
Chen et al.
(2018b)’s work could be generalized to attack the stop sign detector in the physical world more effectively.
However, all the above methods aim to perturb detecting the stop signs.
Blackbox attack is another relevant topic.
Among the current blackbox attack literature, Papernot et al.
(2017) trained a target model substitution based on the assumption that the gradient between the image and the perturbation is available.
We do not have the gradient since the simulation is nondifferentiable.
Chen et al.
(2017) proposed a coordinate descent to attack the model.
However we found that our optimization problem is time-consuming, noisy (see Sec.
G.2), empirically nonlinear and non-convex (see Sec. H).
Time constraints made this approach unavailable to us since it requires extensive evaluations during coordinate descent.
Besides, coordinate descent generally requires precise evaluation at each data-point.
2. S IMULATION AIDED M ACHINE L EARNING
Since the dawn of deep learning, data collection has always been of fundamental importance as deep learning model performance is generally correlated with the amount of training data used.
Given the sometimes unrealistically expensive annotation costs, some machine learning researchers use synthetic data to train their models.
This is especially true in computer vision applications since state-of-the-art computer-generated graphics is virtually photo-realistic.
Ros et al.
Richter et al.
(2017) proposed synthetic datasets for semantic segmentation.
Gaidon et al.
(2016) proposed virtual KITTI as a synthetic replica of the famous KITTI dataset (Geiger et al., 2012) for tracking, segmentation etc.
And Varol et al.
(2017) proposed using synthetic data for human action learning.
Zhang et al.
; Bousmalis et al.
(2017) adapt RL-trained robot grasping models from the synthetic environment to the real environment.
Tremblay et al.
(2018) trained a detection network using the same Unreal engine that we are using.
P ROBLEM S TATEMENT
In this paper, we investigate physical adversarial attack on state-of-the-art neural network based object detectors.
The objective is to ﬁnd a camouﬂage pattern such that, when it is painted to the body of a vehicle, the Mask R-CNN (He et al., 2017) or YOLO (Redmon & Farhadi, 2018)
detectors fail to detect the vehicle under a considerable spectrum of variations (e.g., locations, lighting conditions, view angles, etc.).
We learn the camouﬂage in a black-box fashion, without the need of accessing the detectors’ network architecture or weights.
Expectation-over-transformation (EoT).
We formalize the physical adversarial attack problem by the EoT framework (Athalye et al., 2018b).
Denote by t a transformation which converts a camouﬂage pattern c to a real photo.
Such a transformation actually represents a procedure: paint the pattern to a vehicle’s body, drive the vehicle to a location, and take a picture of the vehicle.
Moreover, the transformation conveniently abstracts away enormous factors (e.g., quality of painting, camera, etc.), each of which could play a role in this procedure.
Denote by V t ( c ) the detection score (e.g., by Mask-RCNN) over an image which is a result of a transformation t and a camouﬂage c .
The physical adversarial attack problem is then described by
[Equation]
where T is a distribution over all the possible transformations { t } .
In other words, we search for a camouﬂage c that minimizes the vehicle detection score in expectation.
Transformation in simulation.
Due to ﬁnancial and time constraints, we conduct our study with the photo-realistic Unreal 4 game engine.
It supplies us sufﬁcient conﬁguration parameters, such as the resolution and pattern of the camouﬂage, 3D models of vehicles, parameters of cameras and environments, etc.
Figure 2 shows a camouﬂage pattern, a high-ﬁdelity 3D model of Toyota Camry, a corner of the virtual city used in this paper, and ﬁnally the picture taken by a camera after we apply the camouﬂage to the Camry and drive it to that corner — in other words, the rightmost image is the result of a certain transformation t acting on the three left images of Figure 2.
A PPROACH
In this section, we present two key techniques for solving the problem min c E t V t ( c ) (cf. eq.
(1) and the text there).
One is to estimate the expectation E t by an empirical mean over many transformations.
The other is to train a neural network to clone the joint behavior V t ( c ) of the black-box detector and the nondifferentiable simulator.
4. S AMPLING TRANSFORMATIONS TO ESTIMATE E t
Recall that a transformation t speciﬁes a particular procedure from applying the camouﬂage c to a vehicle until the corresponding image captured by a camera.
In the context of the simulation engine, the camouﬂage c is ﬁrst programmed to textures, which are then warped onto the 3D model of an object.
The simulator can teleport the object to different locations in the environment.
The simulator also has multiple cameras to photograph the teleported object from various distances and view angles.
We identify some key factors involved in this procedure, including vehicle, location, camera-toobject distance, and view angle.
The combinations of them also give rise to variations along other dimensions.
For instance, the lighting condition changes from one location to another.
The vehicle of interest is occluded to different degrees in the images captured by the cameras.
Denote by T S all the sampled transformations that are used for learning the camouﬂage.
Fig.
3 illustrates the positions where the cameras are placed.
We can see how the view angles and camera-to-object distances vary.
The ones shown in the green color are used in the training.
We left out some cameras shown in the red color for the testing.
Since utilizing the camera is computationally expensive, we arrange the cameras in different heights and distances to create as much variance as possible instead of permuting all possible height/ distance combination.
L: 21mH: 4m
L: 21mH: 4m L: 21mH: 4m L: 21mH: 4m
L: 6mH: 1.5m L: 6mH: 1.5m L: 6mH: 1.5m
L: 6mH: 1.5m
L: 20mH: 4m … …
L: 20mH: 4m … L: 20mH: 4m …
L: 20mH: 4m
L: 13mH: 2m
L: 13mH: 2m L: 13mH: 2m L: 13mH: 2m
L: 5.5mH: 1.5m L: 5.5mH: 1.5m
L: 5.5mH: 1.5m L: 5.5mH: 1.5m
L: 5.5mH: 1.5m L: 5.5mH: 1.5m
L: 5.5mH: 1.5m L: 5.5mH: 1.5m
I
III
IV
II I II
III
IV
4. L EARNING A CLONE NETWORK V θ ( c, t ) TO APPROXIMATE V t ( c )
If we unroll the detection score V t ( c ) , it contains at least two major components.
The ﬁrst renders an image based on the camouﬂage c by following the procedure speciﬁed by the transformation t .
The second is to obtain the detection score of a vehicle detector (e.g., Mask-RCNN) over the rendered image.
The ﬁrst component is nondifferentiable while the second could be a black box in practice.
Therefore, we propose to consider them jointly as a black box.
Furthermore, we learn a clone neural network V θ ( c, t ) to imitate the input-output behavior of this extended black box.
As the transformation t itself is involved and hard to represent, we instead input its consequence to the network: a background image and the cropped foreground of a white vehicle.
Figure 4b shows the network architecture.
It takes as input a camouﬂage pattern c , the background image due to a sampled transformation t , and the cropped foreground.
The network V θ
( c, t ) has a single output to approximate the detection score V t ( c ) .
To this end, we are ready to write down an approximate form of problem (1),
[Equation]
Thanks to the differentiable network V θ ( c, t ) with respect to the camouﬂage c , we can solve the problem above by standard (stochastic) gradient descent.
It is important to note that the ﬁdelity of problem (2) depends on the size and diversity of the sampled set of transformations T S as well as the quality of the clone network V θ ( c, t ) .
It is straightforward to generate a large training set for the clone network by randomizing the camouﬂages and transformations.
However, if the optimal camouﬂage is unfortunately not covered by this training set, a discrepancy would occur when we solve problem 2.
In other words, the network may fail to well approximate the detector scores at the region around the optimal camouﬂage.
We address this discrepancy by an alternative learning algorithm as follows.
4. J OINTLY LEARNING THE CLONE NETWORK AND THE OPTIMAL CAMOUFLAGE
We alternatively learn the clone network V θ ( c, t ) and solve the problem (2).
Once a new camouﬂage pattern is found due to optimizing problem (2), it is converted to multiple images by the training
[Figure]
transformations T S .
We obtain the detection score for each of the images by querying a detector.
The camouﬂage pattern, along with the detection scores are then added to the training set for learning the clone network V θ ( c, t ) .
Figure 4 illustrates this process.
Implementation details.
Denote by H [ p, q ] := − p log q − (1 − p ) log(1 − q ) the cross-entropy loss.
We alternately optimize the following two problems,
[Equation]
where C is the collection of all camouﬂages in the training set for learning the clone network V θ ( c, t ) .
s is the stored V t ( c ) .
The (cid:96) 2 regularization λ (cid:107) θ (cid:107) over the network weights is essential.
Without this term, the two loss functions may cause oscillations.
We set λ
= 10 in the experiments.
As the approximation accuracy of the clone network near the optimal camouﬂage is more important than the other regions, we weigh the samples newly added to the training set 10 times the old ones at every iteration.
Algorithm 1 in the Appendix gives more details.
E XPERIMENTS
Since the primary objective of this paper is to learn camouﬂage patterns that deceive vehicle detectors, we introduce two baseline camouﬂage patterns in the experiments: 6 most popular car colors and 800 random camouﬂages in different resolutions.
We then analyze how the resolution of the camouﬂage affects the detection performance.
For the vehicles, we employ two 3D models: a
Toyota Camry and a virtual SUV.
In addition to the main comparison results, we also test the transferability of the learned camouﬂage across different car models, environments, camera positions, and object detectors.
5. E XPERIMENT S ETUP
We describe the detailed experimental setup in this section, including the simulator, vehicle detector, evaluation metrics, and the baseline camouﬂage patterns.
5.1. T HE SIMULATORS
As shown in Fig.
5a, we used the Unreal engine to build our ﬁrst simulation environment from the photo-realistic DownTown environment.
It is modeled after downtown Manhattan in New York.
There are the skyscrapers, cars, trafﬁc signs, a park, and roads, resembling a typical urban environment.
We sample 32 different locations along the streets.
Eight cameras perch at each location, each taking pictures of the size 720 × 360 .
The relative position of a camera is indexed by the view angle and camera-to-object distance, as shown in Fig.
In addition, we install another 16 cameras to test the transferability of camouﬂages across view angles, distances, etc.
In total, we use 18 locations
[Figure]
Our second environment is based on a totally different countryside scene Landscape Mountains, shown in Fig.
5b.
The roads lie on high-altitude mountains and cross bridges, forest, snow, and a lake.
We use this scene to test the generalization ability of our camouﬂage across different environments; this scene is not used in the training.
Like the DownTown environment, we samplelocations along the roads for the purpose of testing.
The two vehicles used in the experiments are shown in Figure 6.
One is a 2015 Toyota Camry
XLE sold in the European Union.
The other is a virtual SUV from AirSim (Shah et al.
It is worth noting that the Toyota sedan appears multiple times in the MS-COCO dataset Lin et al.
(2014) (cf.
some examples in Fig.12).
Since the object detectors are trained on MS-COCO, it is more challenging to hide the Camry than the virtual SUV.
5.1. V EHICLE DETECTOR
We study two state-of-the-art detectors: Mask R-CNN (He et al., 2017) and YOLOv3-SPP Redmon & Farhadi (2018).
Mask R-CNN is one of the most powerful publicly available object detectors; it currently ranks 4th place in the MS COCO detection leaderboard.
Both detectors are pre-trained on MS COCO.
For Mask R-CNN, we use the one implemented by Abdulla (2017).
Its backbone network is ResNet-101 (He et al., 2016).
YOLOv3 has comparable performance with Mask R-CNN.
Its network architecture is very different from Mask R-CNN’s, causing challenges to the transfer of the camouﬂage between the two detectors.
We use the spatial pyramid pooling (SPP) variant of
YOLOv3 in the experiments.
In the rest of the paper, we experiment with Mask R-CNN except the transfer experiments in Sec. 5.4.
5.1. E VALUATION M ETRICS
We adopt two metrics to evaluate the detection performance.
The ﬁrst one is a variation of the mean Intersection over Union (mIoU) proposed by Everingham et al.
The IoU between a predicted box and the groundtruth bounding box is deﬁned as IoU ( A, B ) =
A ∩ B A ∪ B .
Since IoU was originally proposed to evaluate the results of multi-object detection, as opposed to the single vehicle detection in our context, we modify it to the following to better capture the vehicle of interest: max p ∈ P IoU ( p, GT ) , where P is the set of all detection proposals in an image.
We average this quantity across all the test images and denote the average by mIoU.
Our second metric is precision at 0.5 or P@ 0 . 5 .
Everingham et al.
(2015) set a 50% threshold for the detection IoU in the PASCAL VOC detection challenge to determine whether a detection is a hit or miss.
We report the percentage of the hit detections across all observations as our P@ 0 .
We also report the relative performance drop over baseline colors whenever possible.
5.1. B ASELINES
Our ﬁrst baseline is the detector’s performance when a vehicle is colored in popular real car colors.
We select 6 basic car colors (red, black, silver, grey, blue, and white) which cover over 90% of the car colors worldwide (Axalta).
We obtain their RGB values according to the X11 Color Names.
One may concerned that the learned camouﬂage successfully attacks the detector not because it exploits the CNN’s structural weakness, but because it takes advantage of the domain gap between the real data used to train the detector and our simulated data used to test the detector.
To address this concern, we generate 800 random camouﬂages in different resolutions ranging in { 2 i × 2 i | i ∈ [1 .. 8] } .
Since we ﬁnd that camouﬂages with strong contrasts work better, we generate half of the camouﬂages using RGB values ∈ [0 , 255] and the other half of them using RGB values ∈ { 0 , 255 } .
After we obtain these camouﬂages and the corresponding detection scores, we use those with proper resolutions to initialize the training set for the clone network V θ ( c, t ) .
5. R ESOLUTION OF THE C AMOUFLAGES
We ﬁrst report the random camouﬂages’ performance on hiding the Camry from the detectors in the Downtown environment.
Fig.7 shows the results.
The ﬁrst observation is that, although ran-
256 Camou Resolution 69% 70% 71% 72% 73% 74% 75% 76%
D e t e c t i o n m I
o U mean mIoU (a) mIoU.
256 Camou Resolution 76% 78% 80% 82% 84% 86%
D e t e c t i o n P r e c
i s i o n @ 0 .mean
Precision (b) Precision.
dom camouﬂages are visually very different from conventional car paintings (cf.
Fig.8), the Mask
R-CNN detector is still able to detect them without any challenge.
Another slightly counter-intuitive yet interesting observation is that the detector’s performance does not always decrease as the camouﬂage’s resolution increases.
This is probably because some ﬁne-grained patterns displayed by the high-resolution camouﬂage become harder to observe from a distance.
Since the high-resolution camouﬂage does not bring any additional beneﬁts beyond the 16 × 16 resolution, we use camouﬂages of size 16 × 16 in the remaining experiments.
5. T OYOTA C AMRY IN U RBAN A REA
Here we report the detection results on the Camry.
Table 1 summarizes the results of the Camry with baseline colors, random camouﬂages, and our learned camouﬂage in the urban environment.
We can see from the table that the Mask R-CNN is surprisingly robust against different types of random camouﬂages.
The random camouﬂages’ detection scores are close to the baseline colors’.
Moreover, the standard deviation of the random camouﬂages’ scores is also very low, indicating that the difference in the random camouﬂages does not really impact the performance.
Note that we apply the camouﬂage to the body of the Camry, leaving tires, windows, grilles, etc., as informative visual cues to the detector.
Despite that, our learned camouﬂage reduces the precision by around 30% over baseline colors in both training and testing scenes in the urban environment.
Virtual SUV.
Appendix A presents results on the virtual SUV.
The learned camouﬂage decreases
Mask R-CNN’s detection precision by around 40% relative to the precision with baseline colors.
5. T RANSFERABILITY EXPERIMENTS
We show that the camouﬂage learned to attack Mask R-CNN can actually also defeat YOLOv to a certain degree.
The results are reported in Table 2.
Similarly, appendices A to D report the transferabilities of the learned camouﬂage across different environments, vehicles, camera view angles and distances to the object.
[Table] [Table]
5. Q UALITATIVE R ESULTS
We present some of our detection results on the baseline color, random camouﬂaged
[Table]
and learned camouﬂaged Camry in Fig.11.
We can draw a lot of interesting observations from the qualitative results that we cannot from the quantitative ones.
We discover that there are 3 types of successful attacks: (1) Camouﬂages lower the objectiveness of the car and the car is not detected or only partially detected; (2) The car is successfully detected but misclassiﬁed (i.e., kite, cake, truck, potted plant as shown in the examples) or the classiﬁcation score is too low to pass the detection classiﬁcation score threshold.
The car is successfully detected and classiﬁed, but the camouﬂage resulted in another incorrect overlapped detection (the car in the 5th row, for example, is being detected as both a car and a boat for some reason).
One can see that the context or background plays a vital role in object detection.
Although some cars have the same pose, the detector makes completely different predictions for them.
Also, these qualitative results imply that the detector works in a way different from human vision.
In the landscape scenarios, our learned camouﬂage has the strongest contrast to the background compared to other tested patterns.
However, the detector is still sometimes not able to detect these cars.
C ONCLUSION
In this paper, we investigate whether it is possible to camouﬂage objects with complex shapes, i.e., vehicles, to hide them from state-of-the-art detectors such as Mask R-CNN.
We found this to be feasible.
We proposed a clone network to mimic the simulation and the detector.
Then a camouﬂage could be proposed by minimizing the output of the clone network.
Our learned camouﬂage signiﬁcantly reduced the detectability of these vehicles and we further showed that the camouﬂage is transferable across different environments.
We plan to look into possible ways to white-box the entire process to propose a stronger camouﬂage in our future work.
A V IRTUAL SUV IN U RBAN A REA
We then report the camouﬂage performance on the newly modeled SUV in the urban environment in Table.
[Table]
Judging by the results
[Table]
it is clear that the Mask R-CNN is a very generalized detector.
The SUV’s baseline color and random camouﬂage detection scores are even higher than the Camry correspondence although the detector has never seen it before.
This might because the SUV has much less polygon and is built to resemble the shape of a general SUV as shown in Fig.6.
However, the price for not seeing this vehicle during training is that Mask R-CNN is more likely to be affected by the camouﬂage.
The std of the random camouﬂage mIoU/ precision is higher (3.26/3.48 vs. 0.8/1.2) than Camry.
And our camouﬂage achieves lower detection precision despite both baseline colors and random camouﬂage’s detection scores are higher than Camry’s.
The detectability reduced is almost 1.5 times of Camry’s results in this case.
This experiment shows that
Mask R-CNN might well generalized to unseen vehicles, but it is easier to get attacked.
B T RANSFERABILITY ACROSS E NVIRONMENTS
Another important question is that what if we transfer the camouﬂage to a not only previously unseen but a totally different environment?
To quantitatively answer this question, we build a countryside environment (Fig.5b) to test our camouﬂages trained in urban environment (Fig.5a) using the Camry vehicle.
The results are reported in Table.4.
Some qualitative results are shown in Fig.11.
Given the barren landscape with few objects in sight, the detector detects the car better than it did in the urban environment for both baseline colors (82.04 vs. 77.57) and random camouﬂages (77. vs. 71.42) possibly due to the absence of distractions.
However, our directly transferred camouﬂage is still able to beat both of them by more than 46% regarding both detection mIoU and precision without any ﬁne-tuning.
It is interesting to notice that the Camry with grey color looks almost identical to the background in this environment (Fig.11).
However, it still could be perfectly detected.
Meanwhile, our leaned camouﬂage results far better stealth despite it has a sharp contrast to the background.
[Table]
[Table]
C T RANSFERABILITY ACROSS V EHICLES
It will be impractical to retrain a speciﬁc camouﬂage for each vehicle whenever we need it.
Hence it would be interesting to look into the transferability between vehicles in this scenario.
We swap the camouﬂages of SUV and Camry and see how they would perform in the urban environment.
We present the testing precision after swapping in Table.5.
First, both camouﬂages are deﬁnitely transferable.
It is also interesting to see that the Camry learned camouﬂage is not as good as the SUV learned camouﬂage even when being applied on the Camry itself.
This might due to that the SUV resembles more generic car feature and hence learning camouﬂage on SUV is less likely to encounter local minima during optimization.
D T RANSFERABILITY ACROSS V IEW P OSITION
One of the probabilities most concerned question is that whether this camouﬂage is robust to the change of the camera position.
To answer this question, we set up another 16 new cameras, as shown in Fig.3, surrounding the vehicle in different relative locations.
We then test the learned camouﬂage’s performance on these new cameras.
The results are shown in Table.6.
Our camouﬂage performance drop has slightly decrease 5% from Table.1.
This indicates that the change of the camera locations would impact the performance, but the performance drop is still way beyond the stand deviation of random camouﬂages’ scores.
Given that the new cameras’ views cover more perspectives as shown in Fig.3, this result is reasonable.
E I MPACT OF C LONE N ETWORK Q UALITY
How does the clone network’s quality affect the camouﬂage’s performance?
Is the alternative optimization necessary?
We quantitatively show the ﬁrst 300 simulation calls of our system in Fig.
9 and meanwhile evaluate the camouﬂages proposed by the clone network.
Note that initially the clone network has already been trained with 800 random camouﬂages.
However, the proposed camouﬂage’s score does not fall until the new camouﬂages from iteration scheme join the optimization.
This suggests that without out iterative optimization mechanism, the clone network could only ﬁnd camouﬂage with mIoU around 70%, which is the same as the random camouﬂage.
Those new samples serve as the hard samples.
They gradually calibrate clone network’s global minima to V t () ’s and help it to generate better camouﬂages.
Note that despite the score descend quickest during the ﬁrst 50 iterations, it does not ﬁnd
the global best camouﬂage until near the end (296th iteration).
This graph also shows the classiﬁcation score is a suitable choice to be minimized as it is highly correlated with mIoU.
Note that the system automatically re-initialize the clone network’s parameter to prevent it from falling into local minima during the optimization whenever we add a new sample to the training set
C .
Hence there are some spikes in the graph.
[Figure]
We can see how the new samples helped the clone network to ﬁnd the minimal.
F D ETECTION A TTENTION
How exactly does our camouﬂage work against the detector?
We partially answer this question by visualizing Mask-RCNN’s completely/partially successful detection attention on our grey and camouﬂaged Camry.
Since our Mask-RCNN implementation does not explicitly yield failed detection’s prediction, we are unable to visualize the failed detection’s attention w.r.t.
the input image if the failed detection does not exist.
There two main visualization approaches: Grad-CAM (Selvaraju et al., 2017) and saliency (Simonyan et al., 2013).
Approximately speaking, grad-CAM visualization the gradient of output w.r.t.
penultimate (pre-fully-connected layer) convolutional layer feature map output; Saliency visualize the gradient of output w.r.t.
initial input image.
Grad-CAM is generally considered superior as the last convolutional layer’s feature map contains much more abstracted semantic information, leading to less noisy visualization.
However, we ﬁnd it is hard to deﬁne the single “penultimate layer” in
Mask-RCNN: It has multiple penultimate layers, tracing back to different stages of the network, prior to the ROI pooling layer.
Each of those penultimate layers contains varying levels of information.
We choose to use saliency in this case.
It is clear to deﬁne the “attention” in the image classiﬁcation scenario:
It will be the gradient heatmap of the classiﬁcation score scalar w.r.t.
the entire input image.
In another hand, an end-to-end detection neural network often yields multiple structure predictions.
Each structure contains bounding box and classiﬁcation score etc.
We choose to visualize the gradient of best bounding-box’s classiﬁcation score w.r.t.
the input image.
Our visualizations are presented in Fig.10.
It is surprising that the window, roof and upper bodies are playing the predominant role in car detection.
Upper body attention exists even when the upper body is not included in the detection bounding box(row 3).
Given that the classiﬁcation network makes the classiﬁcation decision based on the proposed feature map box region, such attention must have been already included in the detection proposal before the ROI layer, i.e. detection network.
This may also explain why it is easier to fail the front-view and the rear-view detectors: frontview (row 1) and the rear-view (row 5) detectors place their attention on the hood and trunk, where camouﬂage pattern is presented.
A partially successful attack (row 4) was carried out by wiping out detector’s attention on the car hood.
In another hand, the side-view detector is harder to attack since the detector merely place any attention of the car body (row 2) where the camouﬂages mainly are.
However, our camouﬂages on the roof could still fail the side-view detector partially (row 3).
Since we only visualize the (partially) successful detections, there are still many cases to explore.
G S IMULATION S ETUP G. S IMULATION I MPLEMENTATION
The iterative optimization framework works on two Nvidia GTX 1080 Ti in our experiment.
We use one to run the detector and another one to run the simulation and training/ prediction of evaluation network.
The simulation was implemented using AirSim by Shah et al.
(2017) and UnrealEnginePython.
All submodules are implemented asynchronously to run in parallel and are communicating with each other using RPC/ RPyC.
All the camouﬂages and the baseline colors are implemented using or based on the ofﬁcial Unreal Automotive Material.
Each evaluation in the simulation, which is the most time-consuming part, takes around 15 to 20 second.
G. S IMULATION E RROR
Despite our best effort, we observe V t ( c ) come with a standard deviation of 0 .
due to the inherent and mostly necessary random processes in the rendering (i.e., Monte Carlo in path tracing, etc.)
This unfortunately makes V t ( c ) a noisy function.
We reduce this error by repeat sampling V t ( c ) 5 times whenever use it.
H N ON - LINEARITY AND N ON - CONVEXITY
Since this is a blackbox optimization problem, it is important to exam some important features of the E t V t ( · ) .
We ﬁrst verify its convexity via the convexity deﬁnition.
We test the convexity of E t V t ( · ) by test the convexity of subsampled correspondence 1 | T S | (cid:80) t ∈ T S V t ( · ) via: ∀ c 1 , c 2 ∈ C : | T S | (cid:88) t ∈ T S
V t ( c 1 + c ) ≤ | T S | (cid:88) t ∈ T S
[Equation] (4) where C is a set of camouﬂages.
We sampled 1000 pairs of random camouﬂages from C and half of them does not meet the equation.
Hence 1 | T S | (cid:80) t ∈ T S V t ( · ) is nonconvex.
Besides, we ﬁnd a simple linear MLP is insufﬁcient to approximate 1 | T S | (cid:80) t ∈ T S V t ( · ) , which empirically shows it is nonlinear.
I S UPPLEMENTAL F IGURES
Input : Clone network parameter V θ ( · ) ; Simulation and detection V T S ( · ) ; Transformation T S which are parameterized as rendered background and foreground images; Regularization tradeoff α ; Random camouﬂage set C R .
θ arg min θ
[Equation]
[Figure]
[Equation]
[Equation]
[Table]
[Table] [Equation]
s
C ←
if then
: Best learned
N o t d e t ec t e d “ t r u c k ” “ po tt e d p l a n t ”
P a r ti a ll y d e t ec t e d “ bo a t ” & “ca r ” “ca k e” & “ca r ”
N o t d e t ec t e d “ t r u c k ”
N o t d e t ec t e d “ k it e”
N o t d e t ec t e d
[Figure]
[Figure]