Tensor Train Decomposition for Adversarial Attacks on Computer Vision Models

## Abstract

Abstract Deep neural networks (DNNs) are widely used today, but they are vulnerable to adversarial attacks.
To develop effective methods of defense, it is important to understand the potential weak spots of DNNs.
Often attacks are organized taking into account the architecture of models (white-box approach) and based on gradient methods, but for real-world DNNs this approach in most cases is impossible.
At the same time, several gradient-free optimization algorithms are used to attack black-box models.
However, classical methods are often ineffective in the multidimensional case.
To organize black-box attacks for computer vision models, in this work, we propose the use of an optimizer based on the low-rank tensor train (TT) format, which has gained popularity in various practical multidimensional applications in recent years.
Combined with the attribution of the target image, which is built by the auxiliary (white-box) model, the TT-based optimization method makes it possible to organize an effective black-box attack by small perturbation of pixels in the target image.
The superiority of the proposed approach over three popular baselines is demonstrated for five modern DNNs on the ImageNet dataset.

## 1 Introduction

In recent years, extensive research has been carried out on the topic of adversarial attacks on DNNs, including for classification problems in computer vision as described, for example, in reviews (Qiu et al., ; Akhtar et al., ; Chakraborty et al., ).
New successful adversarial attacks allow us to better understand the functioning of DNNs and develop new effective methods of protection against real attacks in various practical applications.
White-box attacks, which assume that the internal structure of the attacked DNN is known and the gradient of the target output can be calculated, are very successful in many cases.
However, for real-world models, a black-box attack is much more relevant, in which information about the internal structure of the DNN is not used, and it is only possible to make requests to the network and obtain the corresponding outputs.

In the case of black-box attacks, gradient-free optimization methods (Larson et al., ) are most often used to find the optimal perturbation of the target image, which leads to a change in the model’s prediction.
These methods are based either on constructing an estimate of the gradient of the objective function followed by conventional gradient descent, or on various sampling heuristics, including genetic algorithms and evolutionary strategies.
However, in the case of a substantially multidimensional search space, these methods may not be effective enough, and it becomes attractive to use the TT-format (Oseledets, ), which is a universal approach for manipulating multidimensional data arrays (tensors).

The TT-format provides a wide range of methods for constructing surrogate models and effectively performing linear algebra operations (solution of linear systems, integration, convolution, etc.).
Recently it was shown in (Sozykin et al., ; Chertkov et al., ) that the TT-format can also be successfully applied for optimization problems, and the quality of the solution in many cases turns out to be higher than for methods based on alternative gradient-free approaches.
Therefore, in this work, we propose the use of a new TT-based optimization method PROTES (Batsheva et al., ) for black-box attacks.
To further speed up the computation procedure, we build an attribution map for the target image by a popular Integrated Gradients method (Sundararajan et al., ) using an auxiliary pre-trained DNN for which gradients are assumed to be available.
In terms of an untargeted attack, we select a set of pixels that have the highest attribution value, and then only change them during the optimization process.
As a result, we come to a new effective method(^1^11 The program code with all numerical examples from this work is publicly available in the repository [https://github.com/AndreiChertkov/tetradat](https://github.com/AndreiChertkov/tetradat).) TETRADAT (TEnsor TRain ADversarial ATtacks) for generating successful adversarial attacks on DNNs (please, see illustration in Figure [1](#S1.F1)).

This work is organized as follows.
In Section [2](#S2), we briefly describe the TT-decomposition and gradient-free optimization methods based on it, discuss the attribution problem for DNNs, and then review modern methods of adversarial attacks on computer vision models.
Next, in Section [3](#S3), we present in detail the proposed TETRADAT method for adversarial attacks on DNN classifiers.
In Section [4](#S4), we report the results of adversarial attacks performed for five popular models on the ImageNet dataset with our method and three well-known black-box-based baselines.
Finally, in Section [5](#S5), we formulate conclusions and discuss further possible improvements and applications of the proposed approach.

Figure: Figure 1: Schematic representation of the proposed method for black-box adversarial attacks.
Refer to caption: x1.png

## 2 Background and Related Work

In this section, we formulate the problem, describe the main components of the proposed approach (TT-decomposition, TT-based black-box optimization, attribution for DNNs), and then discuss modern methods of adversarial attacks on computer vision models.

We consider adversarial attacks on DNN classifiers in computer vision.
An input image of size $d_{1}\times d_{2}$ pixels (for simplicity, we do not take into account color channels) may be represented as a vector(^2^22 We denote vectors with bold letters ($\bm{a},\bm{b},\bm{c},\ldots$), we use upper case letters ($A,B,C,\ldots$) for matrices, and calligraphic upper case letters ($\mathcal{A},\mathcal{B},\mathcal{C},\ldots$) for tensors with $d>2$. A tensor is just an array with a number of dimensions $d$ ($d\geq 1$); a two-dimensional tensor ($d=2$) is a matrix, and when $d=1$ it is a vector. The $(n_{1},n_{2},\ldots,n_{d})$th entry of a $d$-dimensional tensor $\mathcal{Y}\in\mathbb{R}^{N_{1}\times N_{2}\times\ldots\times N_{d}}$ is denoted by $\mathcal{Y}[n_{1},n_{2},\ldots,n_{d}]$, where $n_{k}=1,2,\ldots,N_{k}$ ($k=1,2,\ldots,d$), and $N_{k}$ is a size of the $k$-th mode.) $\bm{x}\in\mathbb{R}^{d}$, where $d=d_{1}\cdot d_{2}$, and $\bm{x}[i]$ is a value of the $i$-th pixel ($i=1,2,\ldots,d$).
When feeding the image $\bm{x}$ to the DNN, we obtain the probability distribution

$$ $\textsf{$\bm{F}$}(\bm{x})\in\mathbb{R}_{[0,1]}^{C},\quad\sum_{\tilde{c}=1}^{C} \textsf{$\bm{F}$}(\bm{x})[\tilde{c}]=1,$ (1) $$

for $C>1$ possible classes, where the vector function $\bm{F}$ denotes the action of the DNN.
We define the image class prediction $c$, and the corresponding score $y_{c}$ as

$$ $c=\textsf{argmax}(\textsf{$\bm{F}$}(\bm{x})),\quad y_{c}=\textsf{f}(\bm{x}) \equiv\textsf{$\bm{F}$}(\bm{x})[c]\in\mathbb{R}_{[0,1]}.$ (2) $$

Then(^3^33 In this work, we consider the case of untargeted attacks, and do not deal with targeted attacks, when we need to change the network prediction to a predetermined target class, which is specified by the user.) the problem may be formulated as $\bm{z}_{opt}=\textsf{min}_{\bm{z}}\,\,\textsf{f}(\bm{x}+\bm{z})$, where $\bm{z}\in\mathbb{R}^{d}$ is a small perturbation of the target image limited in amplitude, e.g., $||\bm{z}||_{l}\leq\epsilon$ ($l=0,1,2,\infty$, $\epsilon>0$).
The attack is successful if $c\neq\textsf{argmax}(\textsf{$\bm{F}$}(\bm{x}+\bm{z}_{opt}))$ for the found optimal perturbation $\bm{z}_{opt}$.

### 2.1 Tensor Train Decomposition

Let consider the perturbation $\bm{z}$ as a discrete quantity, i.e.,

$$ $\bm{z}[i]\in\biggl{\{}\left(2\cdot\frac{n_{i}-1}{N_{i}-1}-1\right)\cdot \epsilon\,\,\,\Big{|}\,\,\,n_{i}=1,2,\ldots,N_{i}\biggr{\}}\quad\textit{for all}\,\,\,i=1,2,\ldots,d,$ (3) $$

where $N_{i}$ is a number of possible discrete values for perturbations of the $i$-th pixel, and $\epsilon$ is the maximum amplitude of the perturbation.
The values of the function $y=\textsf{f}(\bm{x}+\bm{z})$ form a tensor $\mathcal{Y}\in\mathbb{R}^{N_{1}\times N_{2}\times\ldots\times N_{d}}$ such that $\mathcal{Y}[n_{1},n_{2},\ldots,n_{d}]=\textsf{f}(\bm{x}+\bm{z})$, where $\bm{z}$ is determined by a multi-index $(n_{1},n_{2},\ldots,n_{d})$ according to ([3](#S2.E3)).
Then we come to the problem of discrete optimization, i.e., we search for the minimum of the implicitly specified multidimensional tensor $\mathcal{Y}$, and to solve it we can use the TT-format.

A tensor $\mathcal{Y}\in\mathbb{R}^{N_{1}\times N_{2}\times\ldots\times N_{d}}$ is said to be in the TT-format (Oseledets, ), if its elements are represented by the following formula

$$ $\begin{split}\mathcal{Y}[n_{1},n_{2},\ldots,&n_{d}]=\sum_{r_{1}=1}^{R_{1}}\sum _{r_{2}=1}^{R_{2}}\cdots\sum_{r_{d-1}=1}^{R_{d-1}}\\ &\mathcal{G}_{1}[1,n_{1},r_{1}]\;\mathcal{G}_{2}[r_{1},n_{2},r_{2}]\;\ldots \mathcal{G}_{d-1}[r_{d-2},n_{d-1},r_{d-1}]\;\mathcal{G}_{d}[r_{d-1},n_{d},1], \end{split}$ (4) $$

where $\bm{n}=[n_{1},n_{2},\ldots,n_{d}]$ is a multi-index ($n_{i}=1,2,\ldots,N_{i}$ for $i=1,2,\ldots,d$), integers $R_{0},R_{1},\ldots,R_{d}$ (with convention $R_{0}=R_{d}=1$) are named TT-ranks, and three-dimensional tensors $\mathcal{G}_{i}\in\mathbb{R}^{R_{i-1}\times N_{i}\times R_{i}}$ ($i=1,2,\ldots,d$) are named TT-cores.
The TT-decomposition ([4](#S2.E4)) allows to represent a tensor in a compact and descriptive low-parameter form, which is linear in dimension $d$, i. e., it has less than $d\cdot\max_{i=1,\ldots,d}(N_{i}R_{i}^{2})$ parameters.
In addition to reducing memory consumption, the TT-format also makes it possible to obtain a linear in dimension $d$ complexity of many algebra operations, including element-wise addition and multiplication of tensors, calculation of convolution and integration, solving systems of linear equations, calculation of statistical moments, etc.

It is important that there is a set of methods that allow us to quickly construct a TT-decomposition for a tensor specified by a training data set (Chertkov et al., ) or as a function that calculates any of its requested elements (Oseledets and Tyrtyshnikov, ).
Note that in both cases there is no need to store the full tensor in memory, and the algorithms usually require only a small part of the tensor elements to construct a high-quality approximation.
Due to the described properties, the TT-format has become widespread in various practical applications (Cichocki et al., , ; Oseledets et al., ; Chertkov and Oseledets, ; Khrulkov et al., ), including surrogate modeling, solving differential and integral equations, acceleration and compression of neural networks, etc.

### 2.2 Gradient-free Optimization

Figure: Figure 2: Schematic representation of the optimization method PROTES.
Refer to caption: x2.png

TT-format can be applied for black-box optimization problems.
In the recent work (Sozykin et al., ) the TTOpt algorithm was proposed and its superiority over many popular optimization approaches, including genetic algorithms and evolutionary strategies, was shown.
A more powerful algorithm PROTES was presented in later work (Batsheva et al., ).
The applicability of these methods to various problems was considered in (Nikitin et al., ; Chertkov et al., , ; Morozov et al., ; Belokonev et al., ).

In PROTES, the discrete non-normalized probability distribution $\mathcal{P}_{\theta}\in\mathbb{R}^{N_{1}\times N_{2}\times\ldots\times N_{d}}$ is introduced for the tensor $\mathcal{Y}$ being optimized.
The distribution $\mathcal{P}_{\theta}$ is represented in the TT-format ([4](#S2.E4)), and parameters $\theta$ relate to the values of the corresponding TT-cores.
We start from a random non-negative TT-tensor $\mathcal{P}_{\theta}$
and iteratively perform the following steps
until the budget is exhausted or until convergence (please, see illustration in Figure [2](#S2.F2)):

- 1.
Sample $K$ candidates of $\bm{n}_{min}$ from the current distribution $\mathcal{P}_{\theta}$:
$\mathcal{N}^{(K)}=\{\bm{n}^{(1)},\ldots,\bm{n}^{(K)}\}$;
- 2.
Compute the corresponding black-box values
$\{y^{(1)},\ldots,y^{(K)}\}$;
- 3.
Select $k$ best candidates with indices $\mathcal{S}=\{s_{1},\ldots,s_{k}\}$
from $\mathcal{N}^{(K)}$ with the minimal objective value, i. e., $y^{(j)}\leq y^{(J)}$ for all $j\in\mathcal{S}$ and $J\in\{1,\,\ldots,\,K\}\setminus\mathcal{S}$;
- 4.
Update the probability distribution $\mathcal{P}_{\theta}$ ($\theta\rightarrow\theta^{(new)}$) to increase the likelihood of selected candidates $\mathcal{S}$.
We make several ($k_{gd}$) gradient ascent steps with the learning rate $\lambda$ for the tensor $\mathcal{P}_{\theta}$, using the loss function
$\widehat{L}_{\theta}(\{\bm{n}^{(s_{1})},\,\ldots,\,\bm{n}^{(s_{k})}\})=\sum_{i
=1}^{k}\log\left(\mathcal{P}_{\theta}[\bm{n}^{(s_{i})}]\right).$

After a sufficient number of iterations,
we expect that the distribution $\mathcal{P}_{\theta}$
will concentrate near the optimum, and among the values requested by the method, there will be multi-indices close to the exact minimum $\bm{n}_{min}$.
It is important to note that all operations described in the above algorithm can be efficiently performed in the TT-format, which ensures the high performance of the method.
The values $K$, $k$, $k_{gd}$, $\lambda$ and the number of parameters in $\theta$ (i. e., the average TT-rank $r$ of the TT-decomposition) are the hyperparameters of the PROTES algorithm, and their choice is discussed in detail in (Batsheva et al., ).

### 2.3 Attribution of Neural Networks

Attribution plays an important role in interpreting artificial intelligence models (Zhang et al., ; Matveev et al., ).
It allows for a given input image $\bm{x}\in\mathbb{R}^{d}$ to construct an attribution map, that is, the vector $\bm{a}\in\mathbb{R}^{d}$, such that $\bm{a}[i]$ is the degree of significance of the $i$-th pixel on the model score $\textsf{f}(\bm{x})\equiv\textsf{$\bm{F}$}(\bm{x})[c]$ for the image class prediction $c$.
In the context of adversarial attacks, attribution is of interest because it allows us to select only a part $\hat{d}$ of the pixels of the original image based on their semantic significance for the attack.

The Saliency Map method (Simonyan et al., ) is a basic approach to white-box model attribution in which the derivative of the DNN output is computed for each pixel of the input image, i.e.,

$$ $\bm{a}[i]=\frac{\partial\,\textsf{f}(\bm{x})}{\partial\,\bm{x}[i]},\quad \textit{for all}\,\,i=1,2,\ldots,d.$ $$

However, such a naive approach in some cases turns out to be insufficiently accurate, for example in local extrema and flat regions of the score $\textsf{f}(\bm{x})$.

In many cases, the more successful method is Integrated Gradients (Sundararajan et al., ) in which a sequence of linear transformations of the image $\bm{x}$ into a baseline $\bm{x}^{\prime}$ (a completely black image or a noise-based image) is considered.
Then the attribution is computed as the averaging of gradients over all transformed images

$$ $\bm{a}[i]=\left(\bm{x}[i]-\bm{x}^{\prime}[i]\right)\cdot\int_{\xi=0}^{1}\frac{ \partial\,\textsf{f}\left(\bm{x}^{\prime}+\xi\cdot(\bm{x}-\bm{x}^{\prime}) \right)}{\partial\,\bm{x}[i]}\,d\xi,\quad\textit{for all}\,\,i=1,2,\ldots,d.$ (5) $$

As proven in (Sundararajan et al., ), such attribution has useful properties, in particular, high sensitivity and independence from the internal implementation of the DNN.
Therefore, we use the Integrated Gradients method as part of our approach for adversarial attacks.
Its hyperparameters are the number of gradient descent steps for approximate derivative calculation and the number of discretization nodes for the integral in ([5](#S2.E5)).

### 2.4 Black-box Adversarial Attacks

Black-box attacks can be divided into query-based attacks and transfer-based attacks.
White-box auxiliary models and existing white-box attacks are often used for transfer-based attacks to generate adversarial perturbations, which are then fed to the target DNN in a black-box setting.
The effectiveness of such attacks depends heavily on transferability across different models, and it seems unlikely that a great deal of knowledge can be extracted from one auxiliary model.
As the auxiliary model is trained to be representative of an attacked DNN in terms of its classification rules, it becomes computationally expensive and hardly feasible when attacking large models.
To increase the transferability of such attacks, various methods are proposed to escape from poor local optima of white-box optimizer (Dong et al., ) or increase the diversity of candidates with augmentation-based methods (Wang and He, ).

In the case of query-based attacks, only interaction with the target model is carried out to generate an adversarial example, and various methods for approximate gradient construction followed by gradient descent (Ilyas et al., ), or gradient-free heuristics (Su et al., ; Andriushchenko et al., ; Pomponi et al., ) can be used.
The heuristic-based approach is usually more query-efficient since estimating the gradient (with finite differences or stochastic coordinate descent) requires additional queries to the DNN.

We classify our method TETRADAT as a query-based method with gradient-free heuristics, despite the use of an auxiliary white-box model, since we need it only to identify the semantic features of the image and its internal structure (provided that the semantic map is correct) has little impact on the quality of our results.
Therefore, as the closest alternative approaches we consider Onepixel, Square, and Pixle methods.

The Onepixel attack algorithm (Su et al., ) applies the differential evolution (Storn and Price, ) to search the suitable pixels and only modify them to perturb target images.
The vector norm of perturbation when using this method is rather small, but attacks may be perceptible to people since the changes in the image are very intense and irregular.

The Square attack algorithm (Andriushchenko et al., ) performs randomly selected squared perturbations, i.e., it selects localized square-shaped updates of the target image at random positions so that at each iteration the perturbation is situated approximately at the boundary of the feasible set.
This method turns out to be query efficient, but the distortion it introduces into the target image is in many cases easily distinguishable visually.

The Pixle attack algorithm (Pomponi et al., ) generates adversarial examples by rearranging a small number of pixels using random search, i.e., a patch of pixels
is iteratively sampled from the target image, and then some pixels within it are rearranged using a predefined heuristic function.
The specificity of image changes in this approach potentially makes it possible to obtain adversarial examples that are practically visually indistinguishable from the original image, however, in some cases this heuristic may not work effectively.

## 3 Adversarial Attacks with Tensor Trains

Figure: Algorithm 1 The TETRADAT method for black-box adversarial attacks.

Figure: Algorithm 2 Image perturbation in the TETRADAT method.

The proposed method TETRADAT for black-box adversarial attacks is schematically illustrated in Figure [1](#S1.F1) and presented in Algorithm [1](#alg1), and [2](#alg2).
First, we establish the top class $c$ predicted by the attacked model $\bm{F}$ for the given input image $\bm{x}$ having $d$ pixels.
Then we build an attribution map $\bm{a}$ using the auxiliary DNN (white-box) $\hat{\textsf{$\bm{F}$}}$ by the Integrated Gradients method and select $\hat{d}$ ($\hat{d}\leq d$) pixel positions $\bm{q}$ with the highest attribution value.

Next, we carry out successive runs of the PROTES optimizer, each time reducing the amplitude of the perturbation $\epsilon$ by half until the budget $m$ is exhausted.
Each run of the optimizer continues until a perturbation $\bm{n}_{adv}$ appears, which leads to a successful attack.
During restarts, we initialize the PROTES optimizer by the probabilistic tensor in the TT-format $\mathcal{P}_{\theta}$ obtained at the end of the previous step (on the first run we generate a random TT-tensor).
This strategy allows us to effectively reuse information on each subsequent run, and achieve a balance between the attack success rate and the smallness of the perturbations.

With PROTES, we minimize the function loss, which returns the score of the attacked DNN for the proposed
discrete perturbation multi-index(^4^44 PROTES requests a batch of values at once, but for simplicity, we demonstrate only one multi-index.) $\bm{n}\in\mathbb{R}^{\hat{d}}$, as presented in Algorithm [1](#alg1).
We consider the simplest case of a discrete grid like ([3](#S2.E3)) with $3$ nodes, that is, for each pixel there are three options: decrease the value by $\epsilon$ (for the index $1$), do not change the value (for the index $2$), and increase the value by $\epsilon$ (for the index $3$).
We can perturb the pixel values in the same way for all three RGB (Red, Green, Blue) color channels at once or consider the optimization of each color channel separately, but the following approach turns out to be more useful in terms of attack success rate and visual assessment of generated adversarial images.
As presented in Algorithm [2](#alg2), we convert the RGB value to the HSV (Hue, Saturation, Value) color model.
Then for the case of decreasing intensity we change the S-factor, and for the case of increasing intensity, we change the V-factor.

Thus, the hyperparameters of our method are the DNN $\hat{\textsf{$\bm{F}$}}$ used for attribution (as mentioned above, the result is practically independent of the choice of model, provided that it builds a semantically correct attribution map), the initial amplitude of the perturbation $\epsilon$ (for simplicity, we always take it equal to $1$), the number of pixels used for optimization $\hat{d}$ (empirically it turns out that $10$ percent of the total number of pixels is enough), as well as hyperparameters of the Integrated Gradients and PROTES method.

## 4 Numerical Experiments

To evaluate the performance of the proposed method TETRADAT, we conducted a series of numerical experiments in the following setting.

#### Dataset.

We consider the ImageNet dataset (Deng et al., ) since it corresponds to high-resolution images of real objects, attacks on which seem relevant in practical applications.

#### Models for attack.

We consider classic popular architectures Alexnet (Krizhevsky, ), Googlenet (Szegedy et al., ), Inception V3 (Szegedy et al., ), Mobilenet V3 LARGE (Howard et al., ), and Resnet 152 (He et al., ) for validation of our method.
In experiments, we used pre-trained models from the torchvision package.(^5^55 See repository [https://github.com/pytorch/vision](https://github.com/pytorch/vision).)

**Table 1: Attack success rate for the baselines and the proposed TETRADAT method.**
|  | Onepixel | Pixle | Square | TETRADAT |
| --- | --- | --- | --- | --- |
| Alexnet | $27.60\,\%$ | $100.00\,\%$ | $94.22\,\%$ | $99.28\,\%$ |
| Googlenet | $26.95\,\%$ | $98.41\,\%$ | $96.30\,\%$ | $99.08\,\%$ |
| Inception | $30.12\,\%$ | $94.11\,\%$ | $91.97\,\%$ | $97.86\,\%$ |
| Mobilenet | $9.99\,\%$ | $92.16\,\%$ | $96.08\,\%$ | $97.22\,\%$ |
| Resnet | $4.69\,\%$ | $61.49\,\%$ | $80.26\,\%$ | $85.68\,\%$ |

**Table 2: $L_{1}$ norm of the perturbations averaged over all successful attacks for the baselines and the proposed TETRADAT method.**
|  | Onepixel | Pixle | Square | TETRADAT |
| --- | --- | --- | --- | --- |
| Alexnet | $449.6$ | $1961.2$ | $10251.9$ | $2222.0$ |
| Googlenet | $441.8$ | $1705.2$ | $10246.8$ | $1750.9$ |
| Inception | $434.4$ | $1660.4$ | $10242.0$ | $1834.0$ |
| Mobilenet | $433.8$ | $3051.3$ | $10249.6$ | $1890.8$ |
| Resnet | $435.3$ | $3061.9$ | $10249.2$ | $3495.0$ |

**Table 3: $L_{2}$ norm of the perturbations averaged over all successful attacks for the baselines and the proposed TETRADAT method.**
|  | Onepixel | Pixle | Square | TETRADAT |
| --- | --- | --- | --- | --- |
| Alexnet | $31.4$ | $57.2$ | $26.7$ | $33.1$ |
| Googlenet | $31.0$ | $53.1$ | $26.6$ | $26.3$ |
| Inception | $30.5$ | $50.8$ | $26.6$ | $27.4$ |
| Mobilenet | $30.4$ | $70.8$ | $26.6$ | $28.2$ |
| Resnet | $30.3$ | $72.4$ | $26.6$ | $51.7$ |

Figure: Figure 3: Visualization of results for five random successful adversarial attacks on the Mobilenet. The correct classes for the presented images correspond to (from left to right): “dung beetle”, “irish wolfhound”, “red fox”, “refrigerator, icebox”, “blenheim spaniel”. The network’s predictions for the attacked images are presented in the corresponding titles.
Refer to caption: extracted/5306064/figures/mobilenet.png

Figure: Figure 4: Visualization of results for five random successful adversarial attacks on the Resnet. The correct classes for the presented images correspond to (from left to right): “warplane, military plane”, “bouvier des flandres”, “armadillo”, “whiptail”, “lakeside”. The network’s predictions for the attacked images are presented in the corresponding titles.
Refer to caption: extracted/5306064/figures/resnet.png

Figure: Figure 5: The result of a randomly selected successful attack on the Mobilenet with TETRADAT. We present (from left to right): the original image (the correct class is “white wolf, arctic wolf, canis lupus tundrarum”), the computed attribution map from the auxiliary model, the proposed image perturbation (pixel intensity is increased $10$ times for visibility), and the final adversarial image (the related prediction of the model is “timber wolf, grey wolf, gray wolf, canis lupus”).
Refer to caption: extracted/5306064/figures/base.png

#### Model for attribution.

We consider the pre-trained VGG 19 (Simonyan and Zisserman, ) model from the torchvision package to build an attribution map.

#### Images for attack.

We consider one image from each of the $1000$ classes in the ImageNet dataset,(^6^66 The images from repository [https://github.com/EliSchwartz/imagenet-sample-images.git](https://github.com/EliSchwartz/imagenet-sample-images.git) are used.) and the attack was carried out only if the corresponding class was correctly predicted by the attacked model and by the model for attribution(^7^77 Excluding from consideration images in which the auxiliary model gives an incorrect prediction does not limit the generality of the method, since in this case, we can use any other model that gives the correct prediction for the image or even manually select semantic areas of significance in the image.).
Thus, for Alexnet we used $692$ images for attack, for Googlenet – $757$ images, for Inception – $747$ images, for Mobilenet – $791$ images, and for Resnet – $831$ images.
The top-1 accuracy of the considered models on the used set of images is $74.2\%$ for Alexnet, $79.5\%$ for Googlenet, $82.0\%$ for Inception, $84.7\%$ for Mobilenet, $95.0\%$ for Resnet, and $84.0\%$ for VGG.

#### Hyperparameters of the Integrated Gradients method.

We used our implementation of the method, with $15$ gradient descent steps, and $15$ nodes for discretization.
For subsequent optimization, we selected $\hat{d}=5000$ pixels with the highest attribution value (that is, approximately $10\%$ of the number of pixels in the image).

#### Hyperparameters of the PROTES method.

We used the official implementation of the method(^8^88 See repository [https://github.com/anabatsh/PROTES](https://github.com/anabatsh/PROTES).) with the following parameters for all computations: $K=100$, $k=10$, $k_{gd}=100$, $\lambda=0.01$, and $r=5$ (please see the description of these parameters in the section dedicated to the PROTES optimizer above).
The computational budget for the PROTES method (that is, the number of requests to the black-box model) was limited to $10^{4}$.

#### Baselines.

We consider three well-known black-box methods: Onepixel, Pixle, and Square, which were discussed in the literature review section above.
We used implementations from a popular library torchattacks (Kim, ), and set the default hyperparameters for all methods, with the following exceptions.
For the Onepixel method, we increased the number of attacked pixels from $1$ to $100$, since with a lower value the percentage of successful attacks is too low, and with a higher value the distortion of the original image becomes too large.
For the Square method, we reduced the maximum perturbation value from $8/255$ to $4/255$, since otherwise the distortions of the attacked image turn out to be dramatic (bright stripes and rectangles running throughout the entire image).
The computational budget for all methods was limited to $10^{4}$.

#### Results.

We report the attack success rates for all considered DNNs and methods in Table [1](#S4.T1).
The average $L_{1}$ and $L_{2}$ norms of the perturbations are presented in Table [2](#S4.T2), and Table [3](#S4.T3) respectively.
Generated adversarial images for five random successful adversarial attacks for the Mobilenet and Resnet model are visualized in Figure [3](#S4.F3), and
Figure [4](#S4.F4) respectively.
In Figure [5](#S4.F5) we demonstrate an example of the attack on the Mobilenet model with our method.

#### Discussion.

As follows from the presented results, the proposed method gives a higher percentage of successful attacks in four out of five cases (for the Alexnet model, the Pixle method was slightly more accurate).
According to the $L_{1}$ norm, our method is significantly superior to the Square method, and according to the $L_{2}$ norm, it is better than the Pixle method, while visually TETRADAT produces the most realistic adversarial images (this becomes more noticeable as images are zoomed in).

## 5 Conclusions

In this work, we presented a new method TETRADAT for untargeted adversarial query-based black-box attacks on computer vision models.
Our approach is based on gradient-free optimization with the low-rank tensor train format and additional dimensionality reduction by using the semantic attribution map of the target image.
TETRADAT with a constant set of hyperparameters demonstrates a high percentage of successful attacks for various modern neural network models in comparison with well-known alternative black-box methods, while image distortion in many cases turns out to be negligible.
As part of the further work, we plan to extend our approach to targeted attacks and other relevant problems, including label-based and universal attacks.

Acknowledgments and Disclosure of Funding

Andrei Chertkov was supported by Non-Commercial Foundation for Support of Science and Education “INTELLECT”, and he thanks Dr. Konstantin Anokhin for productive discussions on the topic of this work. Ivan Oseledets was supported by the Ministry of Science and Higher Education of the Russian Federation (Grant No. 075-15-2020-801).

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: 1 The experiment scope is too narrow. You consider using IG to generate attribution. Thus, you introduce extra information compared with the baselines in your experiment.Additionally, you should consider comparing with transfer-based attacks with VGG as the source model. Especially, you should consider to compare with attribution-based adversarial attacks [1, 2]. Otherwise, the effectiveness of your approach cannot be fully validated. Furthermore, the victim models are outdated, so you should also consider transformer-based models to attack.

2 What is the benefit of your approach? You adapt TT to the field of adversarial attacks, but you fail to demonstrate the benefit of the adoption of TT-based transformation. You should further emphasize your motivation for this paper.

3 What is the time complexity of your approach? It turns out that your approach needs black-box optimization and searching. Thus, you should illustrate the time efficiency compared with baselines.

4 Why do you keep the hyper-parameter setting of the PROTES instead of searching for the optimal setting under the adversarial attack scenario? If the hyper-parameters are already optimal, you should show ablation studies on those hyper-parameters. 

5 The format is a little bit weird. The template of equations seems to be done by Word instead of Latex.

[1] Transferable adversarial attack based on integrated gradients. ICLR 2022.

[2] Improving adversarial transferability via neuron attribution-based attacks. CVPR 2022.
soundness: 2
strengths: Gradient-free adversarial attack is interesting.
summary: This paper proposes a gradient-free adversarial attack method. Their approach is based on the low-rank tensor train format and combined with the attribution of the target image with any auxiliary model. Their approach can devise an effective black-box attack by small perturbation of pixels in the target image. Compared with three popular baselines on the ImageNet dataset.
weaknesses: The experiment scope is too narrow.

Motivation is not clear.

Time complexity is missing.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 1
questions: - How is the auxiliary model selected or trained? Is it a model with the same architecture as the target, or is it a simpler/standard model trained on the same data?
- What are the parameters of the PROTES optimization process? For example, the learning rate, or convergence criteria. Understanding these would clarify the practical tuning required to implement the attack.
- Would using alternative tensor decomposition techniques (e.g., CP or Tucker decomposition) impact performance? It would be interesting to know if the authors tested other decomposition methods or if TT was chosen based on specific benefits in this context.
- How effective is TETRADAT against adversarial defenses? Did the authors test the method against defenses like adversarial training or detection algorithms, and if so, how resilient is it?
soundness: 2
strengths: - The paper introduces a unique combination of Tensor Train (TT) decomposition and PROTES optimization to handle high-dimensional perturbations efficiently in black-box settings. According to the experimental results, the proposed method performs better than some query-based methods.
summary: The paper presents TETRADAT, a novel black-box adversarial attack method for Deep Neural Networks (DNNs) that doesn’t require model details. Using Tensor Train (TT) decomposition and an optimization technique called PROTES, TETRADAT efficiently crafts small perturbations to fool DNNs with limited queries. It outperforms existing black-box methods in terms of success rates and computational efficiency across various models on the ImageNet dataset. TETRADAT is effective for testing DNN vulnerabilities, offering improved robustness evaluation for AI models.
weaknesses: - The authors classify TETRADAT as a query-based black-box attack method due to its reliance on querying the target model during the optimization process. However, since TETRADAT also uses an auxiliary white-box model to guide the attack, it shares characteristics with transfer-based methods, which typically require no queries to the target model.
Given this similarity, a fairer and more comprehensive evaluation would involve comparing TETRADAT with transfer-based attack methods. This would provide insights into whether querying the target model yields significant advantages over transfer-based approaches, especially when both methods assume access to a white-box auxiliary model.

- While the paper fixes a query budget of $10^4$ for TETRADAT and other baseline methods, it does not provide a detailed comparison of the actual number of queries required to achieve a successful attack across different methods.
Query efficiency is a critical measure of black-box attack performance, especially in scenarios where query access is limited or costly. Without a direct comparison of query counts, it is difficult to assess whether TETRADAT offers an advantage in terms of query efficiency relative to other black-box methods.

- TETRADAT relies on several hyperparameters, including TT ranks, perturbation amplitude, learning rate for the PROTES optimization, and the number of discrete perturbation levels. These hyperparameters require careful tuning to achieve optimal performance.
The complexity of tuning these parameters makes the method less practical for deployment, especially for users without extensive experience in hyperparameter optimization. Additionally, there is limited analysis of how these hyperparameters affect the attack's effectiveness, which could have provided valuable guidance for setting these values.

- TETRADAT discretizes the perturbation levels for each pixel, simplifying the search space but potentially limiting the granularity of the perturbations. The chosen number of discrete levels influences both the subtlety and effectiveness of the attack.
This discretization could restrict the method's flexibility to find the optimal perturbation, particularly in cases where finer adjustments are needed. Continuous optimization methods might perform better in scenarios requiring precise control over perturbation values.

- The evaluation is limited to ImageNet, lacking tests on additional datasets, which would demonstrate the method’s generalizability. A more diverse evaluation, including tests with varied hyperparameter settings, would offer a fuller understanding of TETRADAT’s robustness and effectiveness. Besides, a more detailed analysis of experimental results would be preferred.

- The presentation of this paper doesn't meet the ICLR standards and needs further improvement.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. This method claimed the high attack success rate. Could the author further clarify the soecific advantages of TT decomposition over other decomposition methods? What's the main different/novel points with the TT decomposition method?
	2. For creating the attribution maps, would the accuracy significantly different if use a different model? For example, can we use transformer-based model, ViT-B/L?
	3. Because this paper is focusing on untargeted attacks, is this method also works with the target attack? Because the target attack is more challenge and it is able to showcase the achievement of the purposed method.
	4. Do the authors have insights on applying this method on larger/deeper model? Such as ViT-L or different domains model like NLP? 
	5. This method is very relying on the auxiliary DNN model. If using multiple auxiliary DNN models, will it improve the attack accuracy? 
As for generating the perturbating examples, how long it needs to take for generate one sample? If increase the iteration steps, what's the accuracy difference?
soundness: 3
strengths: This method outperforms three baseline methods (Onepixel, Square, and Pixle) in attack tasks across seven models, including adversarially trained models.

This method focuses only on the key pixels, keeping the adversarial attacks not being observed by human eyes, which allows it to achieve high success rate with minimal image distortion.
summary: This paper presents a novel method for using black-box adversarial attacks on the deep neural networks. Unlike the traditional methds that rely on the gradient information or gradient-free optimization methods, this paper uses Tensor Train decomposition for high-dimensional data. It is able to attack the network even without fully access the model. By using low-rangk optimizer and the surrogate model, this paper can generate high-impact pertubations and targeted attack the network with efficiency.
weaknesses: The weakness of the paper is that the study focuses on untargeted attacks. It also needs to consider and compare with the targeted attack results. In adddition, the paper does not address if the approach is feasible for larger and deeper models.

---

confidence: 5
contribution: 1
initial_rating: 3
presentation: 2
questions: 1. Could the authors elaborate on the novel aspects of TETRADAT that differentiate it from the original PROTES framework or other existing works?


2. Is this method superior to advanced query-based black box attack methods?

3. The paper describes the integration of attribution maps with the PROTES optimizer but does not include any ablation studies. Could the authors explain the rationale behind this omission?

4. Regarding the results presented, it seems that Figures 3 and 4 convey similar information. Could the authors clarify the distinction between these two figures and justify their separate inclusion?

5. The L1 and L2 norm results do not clearly demonstrate the superiority of the TETRADAT method over the baselines. Could the authors offer an explanation for this observation and discuss the implications for the practical application of their method?

6. Could the authors elaborate on the basis for their claim that TETRADAT produces the most realistic adversarial images? What observations or preliminary data led to this conclusion?
soundness: 1
strengths: 1. The paper is easy to understand.

The paper is logically expressed, and the description of the method is complete and intuitive. The authors explain the principles and details of the TETRADAT with pictures and text, which makes it easy to understand.
summary: This paper introduces TETRADAT, a novel method for conducting black-box adversarial attacks on deep neural networks (DNNs) in the field of computer vision. The method leverages the Tensor Train (TT) decomposition, a format popular for its efficiency in handling multidimensional data, combined with an attribution map generated by an auxiliary white-box model. The authors propose using the PROTES optimizer, a gradient-free technique, to perturb pixels in the target image that have the highest attribution value, thereby causing misclassification by the DNN with minimal distortion.
weaknesses: 1. The method proposed in this paper is not innovative enough.

(1) While the paper introduces the TETRADAT method for performing black-box adversarial attacks, it relies primarily on the PROTES optimiser, as described in the paper by Batsheva et al.[1], without making any innovative modifications to the PROTES algorithm itself.

(2) While this paper presents an approach that integrates a saliency map with the PROTES optimisation framework to perform adversarial attacks on deep neural networks, the concept of using saliency maps for such attacks is not new. Previous works, such as Liu et al.[2] and Dai et al.[3], have explored the use of saliency maps in the context of adversarial attacks.

2. There are some issues with the experimental section of this paper.

(1) Insufficient Benchmarking:
The paper utilizes Onepixel, Pixle, and Square as baselines for comparison, all of which focus on perturbing a small number of pixels to achieve adversarial attacks. However, the paper lacks a comparison with query-based black-box attack methods, such as the approach presented by Bai et al. [4].

(2) Absence of Ablation Studies:
The paper combines the Tensor Train-based methods with the attribution maps generated by auxiliary white-box models. However, there is a noticeable absence of ablation studies that demonstrate the effectiveness of incorporating the attribution maps. Ablation studies are crucial for understanding the incremental benefits of the components included in the proposed approach.

(3) Lack of Result Analysis:
The paper presents results in Table 1, Table 2, Table 3, Figures 3, and Figure 4, but there is a lack of textual analysis to accompany these results. Moreover, Figures 3 and 4 seem to convey the same information, which may indicate a redundancy in the presentation of results. A thorough discussion of the results, including the implications and insights gained from the experimental data, is missing.

(4) Poor Experimental Results:
The average L1 and L2 norms reported for the TETRADAT method in Tables 2 and 3 do not appear to demonstrate a clear advantage over the baseline methods. A discussion on why the TETRADAT method did not outperform the baselines in terms of these norms is necessary to assess the practical applicability of the proposed technique.

3. Some of the views are not supported by evidence.

The paper claims that TETRADAT produces the most realistic adversarial images (this becomes more noticeable as images are zoomed in), as stated in lines 496-500. However, this claim lacks evidence, as the authors have not provided any experiments with human observers to back up the assertion that TETRADAT's images are more realistically altered.

**References**

[1] Batsheva A, Chertkov A, Ryzhakov G, et al. PROTES: probabilistic optimization with tensor sampling[J]. Advances in Neural Information Processing Systems, 2023, 36: 808-823.

[2] Liu H, Zuo X, Huang H, et al. Saliency Map-Based Local White-Box Adversarial Attack Against Deep Neural Networks[C]//CAAI International Conference on Artificial Intelligence. Cham: Springer Nature Switzerland, 2022: 3-14.

[3] Dai Z, Liu S, Li Q, et al. Saliency attack: towards imperceptible black-box adversarial attack[J]. ACM Transactions on Intelligent Systems and Technology, 2023, 14(3): 1-20.

[4] Bai Y, Wang Y, Zeng Y, et al. Query efficient black-box adversarial attack on deep neural networks[J]. Pattern Recognition, 2023, 133: 109037.

Meta-review: This paper proposes TETRADAT, a gradient-free adversarial attack method based on the low-rank tensor train decomposition, a format popular for its efficiency in handling multidimensional data, combined with an attribution map generated by an auxiliary white-box model. The reviewers think the paper's novelty is limited, and the experimental results are inadequate. For example, since the method uses an auxiliary white-box model to guide the attack, it should also be compared to transfer-based attacks. The paper uses a fixed query budget of $10^4$
 for TETRADAT and other baseline methods, but it does not provide a detailed comparison of the actual number of queries required to achieve a successful attack across different methods. The proposed method also relies on several hyperparameters. Based on the reviewers' comments, I recommend rejection.

Decision: Reject