ViewFool: Evaluating the Robustness of Visual Recognition to Adversarial Viewpoints
Abstract
Recent studies have demonstrated that visual recognition models lack robustness to distribution shift.
However, current work mainly considers model robustness to 2D image transformations, leaving viewpoint changes in the 3D world less explored.
In general, viewpoint changes are prevalent in various real-world applications (e.g., autonomous driving), making it imperative to evaluate viewpoint robustness.
In this paper, we propose a novel method called ViewFool to find adversarial viewpoints that mislead visual recognition models.
By encoding real-world objects as neural radiance fields (NeRF), ViewFool characterizes a distribution of diverse adversarial viewpoints under an entropic regularizer, which helps to handle the fluctuations of the real camera pose and mitigate the reality gap between the real objects and their neural representations.
Experiments validate that the common image classifiers are extremely vulnerable to the generated adversarial viewpoints, which also exhibit high cross-model transferability.
Based on ViewFool, we introduce ImageNet-V, a new out-of-distribution dataset for benchmarking viewpoint robustness of image classifiers.
Evaluation results on 40 classifiers with diverse architectures, objective functions, and data augmentations reveal a significant drop in model performance when tested on ImageNet-V, which provides a possibility to leverage ViewFool as an effective data augmentation strategy to improve viewpoint robustness.
Introduction
One of the fundamental challenges of deep learning is to generalize reliably to unseen or shifted test distributions [16, 23, 44, 58].
Typically, deep learning models can be easily deceived by adversarial examples [19, 53], which are crafted by applying small perturbations to natural examples.
Although the adversarial perturbations are imperceptible, they are quite sophisticated and can hardly exist in the wild [3, 30].
Recent works have revealed the vulnerability of visual recognition models to some natural (image) transformations, including rotation and translation [13], geometric transformations [27], image corruptions [23], and others [17, 31, 61, 62].
Out-of-distribution (OOD) generalization is thus emerging as an essential research topic in closing the gap between human and machine vision [16, 18].
Despite the progress, most works concentrate on the robustness of visual recognition models to 2D image transformations [13, 16, 23], while less effort has been made to explore model robustness to 3D variations in the physical world, such as viewpoint changes .
In numerous real-world applications (e.g., autonomous driving, surveillance), viewpoint changes are arguably more natural and prevalent, which can cause severe security and safety consequences if the models cannot deal with such changes.
[Figure] As human vision exhibits a strong ability to robustly recognize objects from varying viewpoints [7], it is of significant importance to study the viewpoint robustness of visual recognition models to identify and understand their weaknesses before they are deployed in safety-critical applications.
Some prior works [5, 25] build new image datasets with different viewpoints to benchmark model robustness, but they cannot evaluate the worst-case performance of the model.
Other works [2, 64] propose to adversarially estimate the physical conditions (e.g., viewpoints, illumination) that cause misclassification, but these methods can only perform attacks for synthetic 3D object models, which do not necessarily correspond to real-world objects.
In general, evaluating the viewpoint robustness of visual recognition models in the physical world is considerably challenging, since it requires modeling the real-world 3D objects with high fidelity but there is always a gap between the real-world objects and their digital representations.
Moreover, even if the adversarial viewpoints can be generated, it is hard to control the pose of the real camera to precisely match a specific adversarial viewpoint, making the captured image fail to consistently mislead the model in the physical world.
In this paper, we propose ViewFool , a novel method to systematically generate adversarial viewpoints that can mislead visual recognition models in the physical world, as shown in Fig.
Motivated by the recent progress of neural rendering, we represent the real-world 3D objects by neural radiance fields (NeRF) [41], which can synthesize photorealistic images from novel viewpoints.
The rendered images are then fed into the target model to obtain the losses.
By optimizing a parameterized distribution of adversarial viewpoints under an entropic regularizer, ViewFool can characterize diverse adversarial viewpoints to mitigate the gap between the real objects and their neural representations and improve the resistance to camera fluctuations.
For optimization, we find that gradient calculation requires unacceptable GPU memory usage despite the differentiability of the rendering process (detailed in
Sec. 3.3).
Therefore, we resort to search gradients that only require forward propagation through the rendering process and the target model, making ViewFool applicable under a black-box setting with only query access to the model.
We conduct extensive experiments to evaluate the viewpoint robustness of image classifiers on the
ImageNet [45] dataset.
Our results demonstrate that ViewFool can effectively generate a distribution of adversarial viewpoints against the common image classifiers, which also exhibit high transferability across different models.
Moreover, we introduce a new OOD benchmark— ImageNet-V for viewpoint robustness evaluation of image classifiers.
We provide evaluation results on 40 image classifiers with diverse network architectures (e.g., ResNet [21], vision transformer [12]), training objective functions (e.g., self-supervised learning [22], adversarial training [46]), and data augmentation strategies (e.g.,
AugMix [24], DeepAugment [25]).
The performance of these classifiers degrades significantly when evaluated on ImageNet-V, demonstrating that the existing classifiers are substantially vulnerable to viewpoint changes, which may motivate future work on improving viewpoint robustness.
Related work
Deep learning robustness.
A major obstacle of deep learning models towards human-level performance is the lack of robustness to distribution shift [16, 18, 23, 44, 58].
It has been shown that deep learning models are vulnerable to adversarial examples [19, 53], which are maliciously crafted by perturbing the natural examples with small perturbations.
Some methods have successfully generated adversarial examples in the physical world [3, 30], raising a severe threat to the deployment of deep learning models.
However, these adversarial examples are elaborately designed by humans, that can hardly exist in the wild.
Other works have studied the vulnerability of deep learning models to natural transformations.
For example, Engstrom et al.
[13] find that image translations and rotations suffice to mislead a target model.
Hendrycks & Dietterich [23] introduce ImageNet-C/P to benchmark model robustness to natural image corruptions and perturbations.
The gap between human and machine vision for out-of-distribution generalization has also been analyzed [16, 18].
To address the robustness issues, various kinds of defense and robustness improvement techniques have been proposed, e.g., [24, 25, 33, 38, 63, 65], to name a few.
Robustness to 3D variations.
As deep learning models have been increasingly deployed in numerous real-world applications, it is imperative to study model robustness in the 3D world.
The ObjectNet [5] and DeepFashion Remixed [25] datasets have been introduced to evaluate model performance under varying backgrounds, camera viewpoints, and object rotations.
However, they cannot evaluate the worst-case performance of the model to 3D variations.
Zeng et al.
[64] generate adversarial examples by perturbing the 3D physical conditions, including 3D rotations and translations, illuminations, etc.
Alcorn et al.
[2] further generate adversarial poses of the 3D objects, such that the rendered images can mislead deep learning models.
However, these methods require synthetic 3D object models that do not correspond to real-world objects, and thus they cannot be utilized in the physical world.
Our work differs from them mainly in that we can generate adversarial viewpoints for real-world objects by encoding them as neural representations.
Novel view synthesis and neural rendering.
It is a long-standing problem to synthesize novel views of an object/scene given a set of captured images.
Early methods [20, 32] can construct photorealistic novel views given densely sampled input views.
Recently, plenty of works [14, 36, 39, 40, 41, 43, 50, 51] adopt neural networks to model 3D objects and can render high-quality images for sparser sets of input views.
Among these approaches, neural radiance fields (NeRF) [39] show promising results by representing 3D objects/scenes as volumetric radiance fields parameterized by a multi-layer perceptron (MLP).
In this paper, we adopt NeRF to learn neural representations of real-world 3D objects and then generate adversarial viewpoints based on the rendered images.
Although NeRF can synthesize photorealistic images, there is still a reality gap between the rendered images and the real images, affecting the effectiveness of the adversarial viewpoints given the real objects.
Our proposed method eliminates this issue by learning a distribution of adversarial viewpoints, as detailed below.
Methodology
In this section, we detail the proposed ViewFool method.
ViewFool adopts NeRF to model real-world 3D objects as the digital representation and performs optimization in the search space of viewpoint parameters.
In the following, we first introduce the background knowledge of NeRF and then present the problem formulation and optimization algorithm of ViewFool.
3.1 Preliminary: neural radiance fields (NeRF)
NeRF [41] encodes a real-world object/scene as a continuous volumetric radiance field F : ( x , d ) → ( c , τ ) .
F is approximated by a multi-layer perceptron (MLP) which takes a 3D location x ∈ R 3 and a unit-norm viewing direction d ∈ R 3 as inputs, and outputs an emitted RGB color c ∈ [0 , 1] 3 and a volume density τ ∈ R + .
The volumetric radiance field can then be rendered into a 2D image from a specific viewpoint by computing the color of every pixel using volume rendering.
Let r ( t )
=
o
+ t d denote the camera ray emitted from the camera center o through a given pixel on the image plane.
The color of this pixel can be approximated by
[Equation]
) · α ( τ ( t i ) · δ i ) · c ( t i ) , where T ( t i ) = exp
[Equation]
where a set of quadrature points { t i } Ni =1 is randomly selected by stratified sampling,
c ( t i ) and τ ( t i ) are the color and density at r ( t i ) , α ( x ) = 1 − exp( − x ) , and δ i =
t i +1 − t
i is the distance between two adjacent points. Camera Rotation ! "
Camera Translation Camera % & ' % & ' % & ' Rendering Δ , ,Δ . ,Δ / 3D Object 2D Image To train the volumetric radiance field network F , NeRF minimizes the total squared error between the rendered and true pixel values given a set of images with known camera poses.
NeRF also adopts the positional encoding and hierarchical volume sampling strategies to improve its performance.
The network F can leverage the multiview consistency among the calibrated images to implicitly capture the 3D nature of the underlying object, enabling it to render photorealistic novel views.
3.2 Problem formulation
Given a trained NeRF with the fixed parameters, ViewFool then estimates the adversarial viewpoints such that the rendered images can mislead a target visual recognition model f .
As shown in Fig.
2, for a pre-defined coordinate system xyz and the initialized camera pose, we first apply 3D rotations to the camera about the z - y ′ - X axes in sequence by the Tait–Bryan angles ( ψ, θ, ϕ ) , which are also known as yaw, pitch, and roll.
We then translate the camera position by (∆ x , ∆ y , ∆ z ) along the three axes.
We let v :=
[ ψ, θ, ϕ, ∆ x , ∆ y , ∆ z ] ∈ R 6 denote the transformation parameters of the camera.
v is bounded in [ v min , v max ] to make the captured images recognizable by humans.
The setting of v min and v max will be specified in Sec. 4.
Given the new camera pose, we can render a 2D image based on NeRF.
Note that the rendered image is only determined by the viewpoint parameters v , thus we denote it as I :=
R ( v ) , where R is the differentiable rendering process as introduced in Sec. 3.1.
Instead of searching for a single adversarial viewpoint, ViewFool captures a diverse set of adversarial viewpoints using a distribution p ( v ) .
By optimizing the distribution p ( v ) (under proper parameterization), any viewpoint v drawn from it is likely to fool the model.
Therefore, we formulate ViewFool as solving the following optimization problem:
[Equation]
where f is an image classification model, y is the ground-truth label of the object, L is the classification loss (e.g., cross-entropy loss), H ( p ( v ))
=
− E p ( v ) [log p ( v )] is the entropy of the distribution p ( v ) , and λ is a balancing hyperparameter between the two loss terms.
The entropic regularization is used to avoid the degeneration problem [11], i.e., max p ( v ) E p ( v )
[ L ( f ( R ( v )) , y )] ≤ max v L ( f ( R ( v )) , y ) , indicating that without the entropic regularization, the optimal distribution would degenerate into a
Dirac one and cannot capture heterogeneous adversarial viewpoints.
Therefore, ViewFool adopts the entropic regularization to address this issue with the following merits.
First , ViewFool can explore the space of adversarial viewpoints by learning the underlying distribution, which could help us better understand model vulnerabilities.
Second , as it is hard to precisely control the real camera pose, learning a wider range of adversarial viewpoints can be more resistant to camera fluctuations, enabling to consistently mislead the target model in the physical world.
Third , although
NeRF shows promise for synthesizing photorealistic novel views of complex objects, there is still a gap between the real object and its neural representation, making the rendered images somewhat different from the real captured images.
As a result, the adversarial viewpoint generated based on
NeRF may not fool the model given the real object due to the appearance difference (which can be understood as a kind of adversarial noise introduced in NeRF rendering).
ViewFool eliminates this issue by optimizing a continuous distribution of adversarial viewpoints because it is unlikely that the appearance difference can consistently fool the model under a variety of viewpoints if not optimized to do so [3].
Fourth , the entropic regularizer can help to avoid dropping into poor local optimal of the optimization problem to alleviate overfitting [10], leading to better cross-model transferability of the adversarial viewpoints.
These benefits of ViewFool are validated by extensive experiments in Sec. 4.
3.3 Optimization algorithm
In ViewFool, we parameterize the distribution of adversarial viewpoints with trainable parameters.
To define the distribution p ( v ) whose support is contained in [ v min , v max ] , we take the transformation of random variable approach as [Equation] [Equation] b , where u ∼ N ( µ , σ 2 I ) , a = v max − v min
[Equation]
where u follows a diagonal Gaussian distribution with mean µ ∈ R 6 and standard deviation σ ∈ R 6 , and v is obtained by transforming u via tanh with proper normalization.
Note that we utilize the diagonal Gaussian distribution in this paper for the sake of simplicity.
Although the true distribution of adversarial viewpoints may not be Gaussian or unimodal, we could apply our method multiple times with different initializations or adopt mixture distributions.
Our method is generally compatible with more expressive distributions, e.g., multiplicative normalizing flows [37] or diffusion probabilistic models [52], which we leave to future work.
Given Eq.
(3), problem (2) becomes
[Equation]
where the second term is the negative log density, whose expectation is the entropy H ( p ( v )) .
To solve problem (4), we need to calculate the gradients of the loss w.r.t.
the distribution parameters ( µ , σ ) .
A common method to back-propagate the gradients from random samples to the distribution parameters is the low-variance reparameterization trick [8, 29].
In particular, we reparameterize u by u = µ + σϵ , where ϵ is an auxiliary random variable following the standard Gaussian distribution
N ( 0 , I ) .
In principle, with the reparameterization, the gradients of the objective function in Eq.
(4) w.r.t.
µ and σ can be calculated, since the NeRF rendering process R is naturally differentiable.
However, we need to render the whole image of all pixels simultaneously, which requires a significant amount of GPU memory with a high image resolution.
For example, it consumes more than 300G
GPU memory for rendering a 400 × 400 image with N = 128 quadrature points each ray, making it impractical to directly calculate the gradients through the rendering process R .
To address this problem, we adopt the search gradients to update the distribution parameters motivated by natural evolution strategies (NES) [60].
The gradients of the first loss in Eq.
(4) can be derived as
[Equation] [Equation]
Note that the gradients derived in Eq.
(5) can be estimated with only forward propagation through the rendering process R and the classifier f , such that we do not need to render all pixels at the same time to reduce the memory usage.
Moreover, it enables our method to operate in a black-box manner with only query access to the model.
In practice, we adapt the plain search gradients by natural gradients to stabilize the optimization process [60].
For the second loss (i.e., entropy) in Eq.
(4), we still adopt the reparameterization trick to analytically calculate its gradients to reduce the variance.
Overall, the gradients of the objective function in Eq.
(4) w.r.t.
µ and σ can be derived (in Appendix A) as
[Equation]
In practice, we approximate the expectation in gradient calculation by k Monte Carlo (MC) samples.
We perform multi-step gradient ascent to update the distribution parameters ( µ , σ ) until convergence.
Experiments
We consider visual recognition models on ImageNet [45] in the experiments.
To make the evaluation results fairer and more reproducible, we first conduct experiments on synthetic 3D object models in Sec. 4.1.
We then study several aspects of ViewFool in Sec.
We present the experimental results of ViewFool on real-world 3D objects in Sec. 4.3.
Finally, we introduce a new OOD dataset called ImageNet-V to benchmark viewpoint robustness.
We provide evaluation results on 40 image classifiers with different architectures, training objective functions, and data augmentation strategies in Sec. 4.4.
More results are provided in Appendix C.
The code to reproduce the experimental results is publicly available at https://github.com/Heathcliff-saku/ViewFool_ .
[Table] 4.1 Performance on synthetic 3D objects
Experimental settings.
We collect 100 3D models of rigid objects from BlenderKit 2 belonging to the 1000 ImageNet classes to form our dataset.
We do not adopt public datasets such as ShapeNet [9] due to the lack of realistic textures.
More details about our dataset (including license, visualization) are provided in Appendix B.
We provide the experiments on the more realistic Objectron dataset [1] in Appendix C.7.
Following [41], we train a NeRF model for each 3D object using 100 images from varying viewpoints sampled on the upper hemisphere.
We consider two image classification models, including the CNN-based ResNet-50 [21] and the Transformer-based ViT-B/16 [12].
They achieve 80 .
72% and 79 .
28% Top-1 accuracy on the naturally sampled images for training NeRF.
In
ViewFool, we initialize the camera at [0 , 4 , 0] as shown in Fig.
We set the range of rotation angles as ψ ∈ [ − 180 ◦ , 180 ◦ ] , θ ∈ [ − 30 ◦ , 30 ◦ ] , ϕ ∈ [20 ◦ , 160 ◦ ] , and the range of translation distances as ∆ x ∈ [ − 0 .
5 , 0 . 5] , ∆ y ∈ [ − 1 , 1] , ∆ z ∈ [ − 0 .
We study viewpoint robustness of these models to camera translations only (with the fixed rotation angles [ ψ, θ, ϕ ] =
[0 ◦ , 0 ◦ , 65 ◦ ] ), camera rotations only (with the fixed translation [∆ x , ∆ y , ∆ z ] =
[0 , 0 , 0] ), and the composition of camera rotations and translations.
We also study different ranges of rotation angles in Appendix C.5 and compare to adversarial 2D transformations in Appendix C.6.
We set λ = 0 .
01 in the experiments and conduct an ablation study on λ in Sec.
We approximate the gradients in Eq.
(6) with k = 50 MC samples and adopt the Adam optimizer [28] to update the distribution parameters ( µ , σ ) for 100 iterations.
Evaluation metrics.
After obtaining the optimal parameters ( µ ∗ , σ ∗ ) , we have the optimal distribution of adversarial viewpoints p ∗ ( v ) as well as the mean of the distribution v ∗ = a · tanh( µ ∗ ) + b with a and b defined in Eq.
v ∗ can be understood as the optimal adversarial viewpoint .
We first measure the attack success rate (i.e., misclassification rate) of the classifier given the rendered image
R ( v ∗ ) by NeRF.
We also calculate the attack success rate given the real image taken at the adversarial viewpoint v ∗ (denoted as Real( v ∗ ) ) for comparison.
Moreover, to evaluate the performance of the learned distribution of adversarial viewpoints, we measure the attack success rate given the rendered images from 100 viewpoints sampled from p ∗ ( v ) , denoted as R ( p ∗ ( v )) .
All those attack success rates are averaged over all 100 objects in our dataset.
Experimental results.
Table 1 shows the experimental results, where we compare ViewFool using λ = 0 .
01 with two baselines, including ViewFool without the entropic regularizer (i.e., λ = 0 ) and the random search method.
For random search, we only report the results for R ( p ∗ ( v )) evaluated by 100 random viewpoints per object since there is no optimal viewpoint v ∗ .
We have the following findings based on the results.
(1) ViewFool with the entropic regularizer achieves higher attack success rates given the rendered image— R ( v ∗ ) and the real image— Real( v ∗ ) from the optimal viewpoint v ∗ than that with λ = 0 , [Figure] R e a l I m ag e f r o m N a t u r a l
V i e w p
o i n t
R e nd ere d I m ag e f r o m A d v er s a r i a l V i e w p o i n t
R e a l I m ag e f r o m
A d v er s a r i a l V i e w p
o i n t because the entropic regularizer enables to sufficiently explore the space of adversarial viewpoints and converge to better local optima. (2) The performance gap between R ( v ∗ ) and Real( v ∗ ) is smaller when using λ = 0 . in most cases, indicating that ViewFool can better mitigate the reality gap between the rendered images and the real images.
Thus, the generated adversarial viewpoints can consistently mislead the classifier.
(3) Using the entropic regularizer can result in lower attack success rates of R ( p ∗ ( v )) , which is mainly because that the learned distributions with λ = 0 are more concentrated and cannot capture diverse adversarial viewpoints, as further verified in Table 2.
(4) We can see that optimizing the translation and rotation parameters simultaneously can lead to better performance than optimizing each of them alone.
Thus, we consider the composition of camera translations and rotations in the following experiments.
(5) ViT-B/16 is more resistant to viewpoint changes than ResNet-50, showing the superiority of the transformer architecture, which is further validated with more architectures in Sec. 4.4.
Visualization.
Fig.
3 shows the visualization results of the adversarial viewpoints v ∗ generated by
ViewFool against ResNet-50.
It can be seen that the real images taken from the natural viewpoints can be correctly classified by the model.
By changing the viewpoints to v ∗ , the rendered images and the real images are misclassified by the model to incorrect classes, while those images are still natural and recognizable for humans.
In most cases, the model misclassifies the rendered image and the real image to the same class, although they appear differently in some details.
However, there are some counter-examples that the rendered image and the real image are misclassified to different classes (see the second last column), or the real image is not misclassified (see the last column). 4.2 Additional results and ablation studies The effects of λ . We adopt ResNet-50 as the target and consider λ = 0 , 0 . 0001 , 0 . 001 , 0 . 01 , 0 . 1 , and 1 . 0 , respectively.
For each value of λ , we optimize the distribution of adversarial viewpoints by solving Eq.
(4) for each object in our dataset.
We measure the attack success rates given the rendered images from the optimal adversarial viewpoint R ( v ∗ ) and the optimal distribution R ( p ∗ ( v )) . To measure the diversity of the adversarial viewpoints, we further calculate the standard deviation of viewpoint parameters [ ψ, θ, ϕ, ∆ x , ∆ y , ∆ z ] . The results are shown in Table 2. On one hand, we can observe that the standard deviation of every parameter increases along with λ , which is reasonable since we emphasize more on the entropic regularizer with a larger λ .
On the other hand, as the distribution tends to cover a larger region of adversarial viewpoints, it inevitably contains unsuccessful viewpoints, leading to a reduction of attack success rate given R ( p ∗ ( v )) .
However, it does not affect
[Table]
[Table]
the attack success rate given the rendered image R ( v ∗ ) from the optimal viewpoint.
As we discussed above, using an appropriate λ > 0 can achieve a higher attack success rate given R ( v ∗ ) .
Cross-model transferability.
We then study the transferability of the generated adversarial viewpoints across different models.
Specifically, we adopt 6 CNN-based models including VGG-16 [49],
Inception-v3 [54], Inception-ResNet-v2 [55], DenseNet-121 [26], EfficientNet-B0 [56], MobileNetv2 [47], 2 Transformer-based models including DeiT-B [59], Swin-B [35], and 1 MLP-based model
MLP-Mixer [57].
We test the attack success rates of the adversarial viewpoints sampled from the optimal distribution p ∗ ( v ) learned by ViewFool with λ = 0 and λ = 0 .
01 against ResNet-50 and
ViT-B/16.
We present the results in Table 3.
The adversarial viewpoints have very high attack success rates against the black-box classifiers with different architectures, showing that the vulnerability of visual recognition models to viewpoint changes is prevalent and intrinsic among these models.
Besides, using λ = 0 .
01 can achieve better cross-model transferability, since the entropic regularizer can avoid the overfitting problem [10] by escaping from poor local maxima of the optimization landscape.
[Figure]
Resistance to camera fluctuations.
In the physical-world experiments, we cannot control the real camera pose to precisely match the adversarial viewpoints v ∗ , thus it is important to ensure the resistance of the adversarial viewpoints to camera fluctuations.
We simulate the physical-world experiments by changing the viewpoints within a percentage of camera fluctuations.
Given a certain percentage r % , we uniformly sample 20 random viewpoints from [ v ∗ − ( v max − v min ) · r % , v ∗ + ( v max − v min ) · r %] and test the attack success rates given the rendered images.
Fig.
4 shows the curves of attack success rates of ViewFool against ResNet-50 with the percentage of fluctuations ranging from 1% to 10% .
The results indicate that using λ = 0 .
01 can lead to better resistance to camera fluctuations that may be encountered in the physical world.
4.3 Performance on real-world 3D objects
In this section, we conduct two sets of real-world experiments to evaluate the performance of our method.
The first set of experiments involves 8 real-world objects, including 2 warplanes, 1 pineapple,
[Table]
[Table]
1 school bus, 1 keyboard, 1 shoe, 1 orange, and 1 French loaf.
We train a NeRF model for each object given 100 captured images of resolution 1600 × 900 using a handheld cellphone.
We place white paper underneath the object to simplify the background for NeRF training.
We use COLMAP [48] to estimate the camera parameters of each image.
After training NeRF, we adopt ViewFool with λ = 0 .
01 to optimize the distribution of adversarial viewpoints.
The rendered images are resized and cropped to match the input size of the target model.
We choose ResNet-50 and adopt the same hyperparameters as the previous experiments.
After obtaining the optimal adversarial viewpoint v ∗ for each object, we aim to take photos in the physical world to approximate v ∗ , as shown in Fig.
Since we cannot precisely match the adversarial viewpoint v ∗ , we take a video for each object around v ∗ .
We then extract 100 frames from the video to calculate the attack success rate in the physical world.
The results are shown in Table 4.
ViewFool can successfully mislead the target model on 7 out of the 8 objects, showing the effectiveness.
For the second set of experiments, we further consider another 4 objects, including two indoor objects (chair and keyboard) and two outdoor objects (street sign and traffic light).
In this experiment, we do not place white paper underneath the object to be more realistic in the wild.
We adopt the same experimental settings.
Fig.
1 and Fig.
C.4 in Appendix C.8 show the visualization results.
ViewFool successfully generates adversarial viewpoints for all these 4 objects in the real world.
4.4 ImageNet-V benchmark
In this section, we introduce the ImageNet-V dataset for viewpoint robustness evaluation of image classifiers.
Specifically, we render 100 images per object from varying viewpoints, which are sampled from the distribution p ∗ ( v ) optimized by ViewFool with λ = 0 .
We choose ResNet-50 as the target model since it leads to better cross-model transferability than ViT-B/16, as shown in Table 3.
Therefore, ImageNet-V consists of 10000 images of 100 different objects.
Note that the background of all ImageNet-V images is white.
This is conducive to evaluating viewpoint robustness solely.
We evaluate the performance of 40 image classifiers with different network architectures (including
VGG [49], ResNet [21], Inception [54, 55], DenseNet [26], EfficientNet [56], MobileNet-v2 [47],
[Table]
[Table]
[Table]
[Table]
[Table]
I n ce p t i o n - v3 I n ce p t i o n - v
I n ce p t i o n - R e s N e
t- v
D e n s e N e tD e n s e N e tD e n s e N e tE ff i c i e n t N e t - B
E ff i c
i e n t N e t - B
E ff i c
i e n t N e t - B
E ff i c
i e n t N e t - B
E ff i c
i e n t N e t - B
M o b il e N e tv2 - 120 d
M o b il e N e tv2 - 140 V i T - B / 16 V i T - L / 16 D e i T - T
i D e i T - S D e i T - B S w i n - T i S
w i n - S S w i n - B S w i n - L M
i x er - B M
i x er - L
R e s N e te p s 1 .
R e s N e te p s 3 .
R e s N e te p s 5 .
M A E : V i T - B M A
E : V i T - L
M A E : V i T - H
A u g M i x
D ee p
A u g m e n t
A u g M i x + D ee p
A u g m e n t
N a t u r a l A cc u r a c y
I m ag e
N e tVA cc u r a c y
ViT [12], DeiT [59], Swin Transformer [35], and MLP Mixer [57]), objective functions (including adversarial training [46] and self-supervised MAE [22]), and data augmentation strategies (including
AugMix [24] and DeepAugment [25]).
Fig.
6 shows the classification accuracy of these models on ImageNet-V.
For comparison, we also evaluate their performance on 10000 images ( 100 per object) from natural viewpoints.
Most classifiers achieve 70% ∼ 80% accuracy on natural images, while none of them exceed 50% accuracy on
ImageNet-V, demonstrating that they are vulnerable to viewpoint changes.
Among them, transformerbased models achieve the best performance (e.g., Swin-L obtains 47 .
40% accuracy and MAE with
ViT-H obtains 49 .
85% accuracy), showing the superiority of the transformer architectures on OOD generalization [4, 6, 42].
Besides, a larger model size within the same architecture family tends to perform better.
Adversarial training and data augmentation techniques, which show promise for adversarial and corruption robustness, do not obtain good results on ImageNet-V, demonstrating that
ImageNet-V performance may not be correlated with that on other OOD robustness benchmarks.
Conclusion
This paper proposes ViewFool to generate adversarial viewpoints against image classification models.
ViewFool adopts NeRF to represent real-world 3D objects and learns a distribution of adversarial viewpoints under an entropic regularizer for each object.
Extensive experiments demonstrate the effectiveness of ViewFool and also reveal the vulnerability of common image classifiers to viewpoint changes.
We successfully generated adversarial viewpoints for real-world 3D objects.
Moreover, we established ImageNet-V for viewpoint robustness evaluation of any classifier.
Besides evaluating viewpoint robustness, ViewFool has the possibility to be used as an effective data augmentation strategy to improve viewpoint robustness.
A potential negative societal impact of ViewFool is that it could be used by malicious adversaries to cause security/safety issues in real-world applications.
Limitations:
As an initial attempt for viewpoint robustness, this work also has several limitations.
First, this work adopts a smaller dataset of 100 synthetic 3D objects for evaluation, which may be biased over all 1000 ImageNet classes, as elaborated in Appendix B. Second, our method requires high computational cost for NeRF training and optimizing adversarial viewpoints, as discussed in
Appendix C.1.
Third, this work only focuses on generating adversarial viewpoints (i.e., attacks) but does not propose a defense algorithm, which remains an open problem.
We will address these limitations in the following work.
Acknowledgement
This work was supported by the National Key Research and Development Program of China (2020AAA0106000, 2020AAA0104304, 2020AAA0106302),
NSFC Projects
Nos. 62061136001, 62076145, 62076147, U19B2034, U1811461, U19A2081, 61972224), Beijing NSF Project (No.
JQ19016), BNRist (BNR2022RC01006), Tsinghua Institute for Guo Qiang, and the High Performance Computing Center, Tsinghua University.
Y. Dong was also supported by the China National
Postdoctoral Program for Innovative Talents and Shuimu Tsinghua Scholar Program.
J. Zhu was also supported by the XPlorer Prize.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work? [Yes] See Sec. (c) Did you discuss any potential negative societal impacts of your work? [Yes] See Sec.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix A. 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] Our code is publicly available at https://github.com/Heathcliff-saku/ViewFool_ .
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] Since running ViewFool on all objects in our dataset is time-consuming, we only ran one time.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix C.1.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] See Appendix B. (c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[Yes] See Appendix B. (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[Yes] See Appendix B. 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Proof of Eq. (6)
In this section, we derive the gradients of the objective in Eq.
(4) w.r.t.
the distribution parameters ( µ , σ ) .
For the first loss L 1 := E N ( u ; µ , σ 2 I )
[ L ( f ( R ( a · tanh( u ) + b )) , y )] in Eq.
(4), We adopt the search gradients as shown in Eq.
(5), in which we can derive that
[Equation]
where ϵ follows the standard Gaussian distribution
[Equation]
As we mentioned in Sec. 3.
[Equation]
we adapt the plain search gradients by natural gradients to stabilize the optimization process, as suggested in [60].
The natural gradient is defined as
[Equation]
where F is the Fisher information matrix as
[Equation]
Therefore, we can derive that
[Equation]
By integrating Eq.
(A.5) and Eq.
(A.6) into Eq.
[Equation]
we can observe the gradients of the first loss as
[Equation]
For the second loss H :
= E N ( u ; µ , σ 2 I )
[ − log p ( a · tanh( u ) + b )] , we take the transformation of random variable approach to rewrite H as H = E N ( ϵ ; 0 , I )
[ − log p ( a · tanh( µ + σϵ ) + b )] .
The log density of the distribution can be analytically calculated as follows.
Note that each dimension of the random variable is independent, thus we only consider one dimension.
The density of ϵ is p ( ϵ )
[Equation]
√
[Equation]
π exp( − ϵ
[Equation]
The density of u := µ + σϵ is p ( u ) =
[Equation]
Let v := a tanh( u ) + b , then the inverse transformation is u = tanh − 1 ( v − ba )
= 12 log( a + v − b a − v + b ) .
The derivative of u w.r.t.
v is 1 a (1 − tanh( u ) 2 ) .
By applying the transformation of variable approach, we have the density of v as
[Equation]
Hence, the negative log density of p ( v ) is
[Equation]
+
[Equation]
a.
(A
[Equation]
Sum over all dimensions, we have
[Equation]
where d is the dimension index.
Given Eq.
(A.11)
, we can simply calculate the gradients of H w.r.t.
µ and σ as
[Equation]
As the overall loss is L 1 + λ · H , by combining Eq.
(A.7) and Eq.
(A.12), Eq.
(A.8) and Eq.
(A.13), we obtain the gradients in Eq.
hotdog, hotdog, red hot # # # # # # # # # #10 French loaf cheeseburger lemon butternut squash agaric
Granny Smith pretzel hotdog, hot dog, red hot cheeseburger #11 sports car, sport car # # # # # # # # #20 tank, army tank car wheel car wheel motor scooter, scooter forklift mountain bike traffic light warplane canoe # # # # # # # # # #30 studio couch, day bed crate toaster rocking chair, rocker washbasin studio couch, day bed grand piano, grand folding chair washer vase microphone, mike notebook computer mouse reflex camera remote control lampshade electric guitar smoothing iron computer keyboard typewriter keyboard ashcan water bottle running shoe basketball sandal pool table street sign street sign street sign manhole cover analog clock digital clock warplane folding chair sports car studio couch sunglass cellular telephone acoustic guitar volleyball barrel, cask microphone, mike coffeepot moped carpenter's kit, tool kit clog, geta, patten, sabotcandle, taper, wax light vase teapot airliner beach wagon soap dispenser iPod cassette studio couch, day bed lipstick, lip rouge sax, saxophone barrow, garden cart shopping cart barrel, cask spotlight, spot desk espresso cup goblet umbrella crash helmet syringe swing teddy, teddy bear screwdriver piggy bank padlock abacus letter opener hourglass hammer dial telephone binoculars dumbbell # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
B Dataset
In the experiments, we collect a dataset of 100 3D object models from BlenderKit.
These objects were selected based on the following criteria.
They are common in the real world, including cars, street signs, etc; 2) They are easily recognizable by humans; and 3)
They belong to the ImageNet classes such that the adopted visual recognition models (e.g., ResNet, ViT) can classify them from natural viewpoints with high accuracy.
The royalty free license ( https://www.blenderkit.com/ docs/licenses/ ) states that: “This license protects the work in the way that it allows commercial use without mentioning the author, but doesn’t allow for re-sale of the asset in the same form (eg.
a
3D model sold as a 3D model or part of assetpack or game level on a marketplace)”.
As this work is not for commercial use, we do not violate the license.
Fig.
B.1 visualizes these objects from natural viewpoints.
We also show their corresponding groundtruth labels belonging to the 1000 ImageNet classes.
A limitation of the dataset is the smaller size.
It is because training NeRF for each object is computationally expensive as discussed in Appendix C.1.
The dataset does not contain all classes in ImageNet, especially those with deformable shapes (e.g., animals), which may potentially lead to biased evaluations.
Nevertheless, we think that the dataset is highly valuable for benchmarking the viewpoint robustness of visual recognition models, since it is important to understand model vulnerabilities to viewpoint changes in safety-critical applications while few efforts have been devoted to this area.
We will continuously enlarge the dataset in the future.
We provide the additional experimental results in this section. C.1 Computation complexity All of the experiments were conducted on NVIDIA GeForce RTX 3090 GPUs.
For each object, it takes about 6 .
5 hours to train the NeRF model and about 4 .
5 hours to generate a distribution of adversarial viewpoints with a single GPU.
So in total, running all of the experiments (including ablation studies and physical-world experiments) requires about 500 GPU days.
A potential limitation of ViewFool is that it runs slowly due to the time-consuming rendering process of NeRF and querybased optimization algorithm.
It can be further accelerated by adopting advanced NeRF variants with faster inference [15, 34], which we leave to future work.
And therefore we prefer to establish a benchmark for viewpoint robustness evaluation of more classifiers rather than running ViewFool for each of them.
C.2 NeRF results
The first step of our algorithm is to train a NeRF model for each object.
The quality of the NeRF rendering also affects the performance of ViewFool since a smaller reality gap between the rendered images and the real images can make the generated adversarial viewpoints more robust in the real world.
We provide the results of NeRF in Table C.1.
The training images for NeRF are submitted in the code in the supplementary material
. C.3 Visualization of the optimal adversarial viewpoints
To supplement the visualization results in Fig.
3, we provide the visualization of the optimal adversarial viewpoints v ∗ of all 100 objects in our dataset.
Fig.
C.1 shows the visualization results for all objects with the predicted labels and confidences.
These images from adversarial viewpoints can be easily recognized by humans and look natural, which identifies the vulnerability of visual recognition models to natural changes of the inputs.
From the figure, we can observe that some of the objects (e.g., #1, #9, #41) are flipped over, while some others (e.g., #3, #11, #19) are viewed from the above (i.e., bird’s eye view).
Since theses views are uncommon in the training datasets (e.g., ImageNet), the model is not endowed with the robustness.
However, these strange viewpoints can occur in the real world, which can lead to severe security/safety problems.
C.4 Diversity of ImageNet-V
ImageNet-V consists of 10000 images of 100 objects, in which we adopt 100 images from varying viewpoints sampled from the adversarial distributions.
As shown in Table 2, we learn a wide range of viewpoints by ViewFool, such that the images in ImageNet-V do not overlap with each other and have diversity.
To further demonstrate this, Fig.
C.2 shows some images in ImageNet-V.
We can observe that the sampled images for the same object are also different with each other, thus the diversity of
ImageNet-V is improved.
[Table] lipstick 31. grand piano 27. sax 27.
folding chair 35.
barrow, garden cart 30.
washer 36.
shopping cart 29.
vase 31.
barrel, cask 33.
microphone 28.
spotlight 34.
notebook 32.
desk 35.
computer mouse 35.
espresso 26.
reflex camera 30.
cup 36.
remote control 34.
goblet 34.
lampshade 33.
umbrella 36.
electric guitar 35.
crash helmet 33.
smoothing iron 31.
syringe 36.
computer keyboard 36.
swing 34.
typewriter keyboard 24.
teddy, teddy bear 32.
ashcan 33.
screwdriver 39.
water bottle 33.
piggy bank 38.
running shoe 33.
padlock 35. basketball 33. abacus 36. sandal 28.
letter opener 43.
pool table 33. hourglass 32. street sign 36. hammer 40.
street sign 33.
dial telephone 30.
street sign 34.
binoculars 32.
manhole cover 29.
dumbbell 33.
C.5 Different ranges of rotation angles
We study the performance of using different ranges of rotation angles.
Besides the baseline setting in Table 1 that ψ ∈ [ − 180 ◦ , 180 ◦ ] , θ ∈ [ − 30 ◦ , 30 ◦ ] , ϕ ∈
[20 ◦ , 160 ◦ ] , we consider three settings that
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Figure]
we only optimize one rotation angle while keep the other two rotation angles fixed.
We also consider two more settings that we decrease the range of rotation angles.
In this ablation study, we do not optimize the translation parameters.
The results on ResNet-50 are shown in Table C.2.
Generally, a larger range of rotation angles leads to better performance due to the larger search space.
[Table]
[Table]
C.6 Comparison to adversarial 2D transformations
We further study the performance of ViewFool compared to adversarial 2D transformations.
Note that 2D transformations (including 2D rotation, scaling, cropping) are a subset of 3D viewpoint changes studied in this paper.
Specifically, if we keep the rotation angles ψ and ϕ fixed
[Table]
and optimize other viewpoint parameters including θ and [∆ x , ∆ y , ∆ z ] , the changes correspond to 2D image rotation, translation, scaling
[Table]
and cropping.
So in general, adversarial viewpoints can lead to a higher attack success rate than adversarial 2D transformations.
To validate this, we conduct an ablation study by keeping ψ = 0 ◦ , ϕ = 90 ◦ fixed, and perform optimization over the other parameters using the same algorithm and experimental settings.
The results on ResNet-50 are shown in Table C.3.
It can be seen that 3D viewpoint changes lead to better attack success rates than 2D image transformations.
C.7 Experiments on the Objectron dataset
We conduct experiments on the Objectron dataset [1], which contains object-centric videos in the wild.
We select 10 videos that the objects can be correctly classified by the ResNet-50 model (otherwise it is meaningless to study viewpoint robustness).
We adopt the same experimental settings to train NeRF models and generate adversarial viewpoints for those objects.
Fig.
C.3 shows the visualization of the adversarial viewpoints against ResNet-50.
ViewFool successfully generates adversarial viewpoints
Bicycle:
[Table]
[Table]
Horizontal
[Table]
[Table]
Hamber
[Table]
Camera: [Table] Binoculars: [Table]
[Table]
Binoculars
[Table] Chair: [Table] Board:
[Table]
Chair [Table] [Table] Beer
[Table] Jellyfish: [Table]
[Table]
Laptop: [Table] Binder:
[Table] [Table]
Sandal: [Table]
Chocolate:
[Table]
Sandal: [Table]
Diaper: [Table]
R e a l I m ag e f r o m
N a t u r a l V i e w p
o i n t
R e nd ere d I m ag
e f r o m
A d v er s a r i a l
V i e w p
o i n t
R e a l I m ag e f r o m
N a t u r a l V i e w p
o i n t
R e nd ere d I m ag
e f r o m
A d v er s a r i a l
V i e w
[Figure]
i n
[Figure]
for all objects (i.e., the attack success rate is 100% ).
However, since we do not have the ground-truth 3D models of objects/scenes, we cannot obtain the real images taken from the adversarial viewpoints.
C.8 More real-world experiments
We further conduct real-world experiments on another 4 objects, including two indoor objects (chair and keyboard) and two outdoor objects (street sign and traffic light).
In this experiment, we do not place white paper underneath the object to be more realistic in the wild.
Besides Fig.
1, Fig.
C.4 shows the visualization results of rendered images and more captured images from adversarial viewpoints.
ViewFool successfully generates adversarial viewpoints for all these 4 objects in the real world.
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]