SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Ao-Xiang Zhang
Yu Ran
Weixuan Tang*
Yuan-Gen Wang*
Guangzhou University, China {zax, ranyu}@e.gzhu.edu.cn, {tweix, wangyg}@gzhu.edu.cn
No-Reference Video Quality Assessment (NR-VQA) plays an essential role in improving the viewing experience of end-users. Driven by deep learning, recent NR-VQA models based on Convolutional Neural Networks (CNNs) and Transformers have achieved outstanding performance. To build a reliable and practical assessment system, it is of great necessity to evaluate their robustness. However, such issue has received little attention in the academic community. In this paper, we make the first attempt to evaluate the robustness of NR-VQA models against adversarial attacks, and propose a patch-based random search method for black-box attack. Specifically, considering both the attack effect on quality score and the visual quality of adversarial video, the attack problem is formulated as misleading the estimated quality score under the constraint of just-noticeable difference (JND). Built upon such formulation, a novel loss function called Score-Reversed Boundary Loss is designed to push the adversarial video's estimated quality score far away from its ground-truth score towards a specific boundary, and the JND constraint is modeled as a strict  $L_{2}$  and  $L_{\infty}$  norm restriction. By this means, both white-box and black-box attacks can be launched in an effective and imperceptible manner. The source code is available at https://github.com/GZHU-DVL/AttackVQA.
In recent years, the service of "we-media" has shown an explosive growth. It is reported that Facebook can produce approximately 4 billion video views per day [1]. Nevertheless, the storage and transmission of such volumes of video data pose a significant challenge to video service providers [2]. To address this challenge, video coding is employed to balance the tradeoff between coding efficiency and video quality. Therefore, Video Quality Assessment (VQA) has become a prominent research topic. In general, according to whether pristine video is applied as a reference, VQA can be divided into Full-Reference VQA (FR-VQA), Reduced-Reference VQA (RR-VQA), and No-Reference VQA (NR-VQA). Recently, Convolutional Neural Networks (CNNs) and Transformers have made great progress in NR-VQA. However, according to recent studies, Deep Neural Networks (DNNs) show vulnerability against adversarial examples [19]. To build a reliable and practical assessment system, it is of great necessity to evaluate the robustness of the NR-VQA models against adversarial attacks.
The threat of adversarial attacks has been studied in the task of image classification. Szegedy et al. [4] first found that an image injected with a small amount of imperceptible adversarial perturbations can mislead DNNs with high confidence. Moosav-DezFool et al. [5] suggested that universal perturbations can achieve attack effect on different samples. Athalye et al. [6] showed that adversarial examples can be applied in physical world. Since then, numerous adversarial attacks on CNNs and Transformers have been proposed [7; 8; 9; 24; 32].
* denotes corresponding author.
$L_{2} = 3 / 255$ $L_{\infty} = 5 / 255$
Figure 1: The original video frame and the perturbed ones constrained by the  $L_{2}$  and  $L_{\infty}$  norm.
$L_{2} = 8 / 255$ $L_{\infty} = 10 / 255$
Compared with its rapid development in the classification task, there have been few investigations of adversarial attacks in Quality Assessment (QA), including Image Quality Assessment (IQA) and Video Quality Assessment (VQA). Two pioneering works [44; 50] were the closest to this topic in scope. [44] first investigated the adversarial attack on IQA models. It optimized the perturbations with propagated gradients by NR-IQA models. Such perturbations were added to the original image to yield adversarial images. Multiple adversarial images could be generated under the constraint of different Lagrange multipliers, and those adversarial images added with the most perceptually invisible perturbation were selected according to human subjective experiments. This can be understood as a constraint limiting the perturbations below the just-noticeable difference (JND) threshold. [50] designed a universal perturbation as the fixed perturbation trained by a given IQA/VQA model to increase its output scores. Despite [44] and [50] have made early attempt of adversarial attack on IQA/VQA model, there still remain some important issues to be solved. Firstly, since human subjective experiments are both time-consuming and labor-intensive, the test set of [44] includes only 12 images, which is too small to be practical. Secondly, [50] aims to merely increase the estimated quality score outputted by the target model, which is not in line with the definition of adversarial attack. Thirdly, [50] cannot control the visual quality during optimization, and thus the adversarial videos suffer from obvious artifacts. Fourthly, the adversarial attacks in both [44] and [50] are ineffective in more practical black-box scenarios.
To address the above problems, this paper comprehensively investigates the robustness of NR-VQA models by adversarial attacks. Firstly, adversarial attack on NR-VQA model is mathematically modelled. The target NR-VQA model takes in the adversarial video and outputs the estimated quality score. Meanwhile, we define the anchor quality score as far away from its mean opinion score (MOS) in a specific direction. The optimization goal of such attack problem is to mislead the estimated quality score by means of pulling in the distance between the estimated and anchor quality scores under the constraint of restricting the distortion between the adversarial and original videos below the JND threshold. Specifically, the Score-Reversed Boundary Loss is designed to push the adversarial video's estimated quality score far away from its MOS towards a specific boundary. And limiting the  $L_{2}$  and  $L_{\infty}$  norm of the perturbations below a rather strict threshold can be interpreted as a kind of JND constraint. In this paper, unless otherwise specified, the  $L_{2}$  norm indicates the pixel-level  $L_{2}$  norm averaged on the whole video. An example of applying the constraint of  $L_{2}$  and  $L_{\infty}$  norm is given in Fig. 1. It can be observed that by means of limiting the  $L_{2}$  and  $L_{\infty}$  norm of perturbations within a rather strict threshold of 3/255 and 5/255, the perturbations are imperceptible to human eyes, which can be considered to be below the JND threshold. The contributions of this work are summarized as follows:
- Adversarial attacks on NR-VQA models are formulated as misleading the estimated quality score by means of pulling in the distance between the estimated and anchor quality scores under the JND constraint. To the best of our knowledge, this is the first work to apply adversarial attacks to investigate the robustness of NR-VQA models under black-box setting.
- A novel loss function called Score-Reversed Boundary Loss is proposed to push the adversarial video's estimated quality score far away from its MOS towards a specific boundary. By means of minimizing such loss function, the generated adversarial video can effectively disable the NR-VQA models.
- Adversarial attacks on NR-VQA models are implemented under white-box and black-box settings. Furthermore, a patch-based random search method is designed for black-box attack, which can significantly improve the query efficiency in terms of both the spatial and temporal domains.
2.1 NR-VQA models
With the rapid development of deep learning techniques, NR-VQA models based on CNNs and Transformers have received tremendous attention. Li et al. [25] extracted the 3D shearlet transform features from video blocks. Afterwards, logistic regression and CNN were simultaneously applied to exaggerate the discriminative parts of the features and predict the quality score. Li et al. [26] utilized ResNet [27] to extract features for each frame within video, which were fed into GRU [28] to construct the long-term dependencies in temporal domain. You and Korhonen [29] employed LSTM [30] to predict quality score based on the features extracted from small video cubic clips. Zhang et al. [40] reorganized the connections of human visual system, and comprehensively evaluated the quality score from spatial and temporal domains. Li et al. [31] transferred the knowledge from IQA to VQA, and applied SlowFast [34] to obtain the motion patterns of videos.
Xing et al. [35] were the first to introduce Transformer [38] into NR-VQA to implement space-time attention. Wu et al. [36] designed a spatial-temporal distortion extraction module, wherein the video Swin Transformer [42] was utilized to extract multi-level spatial-temporal features. Li and Yang [37] proposed a hierarchical Transformer strategy, wherein one Transformer was utilized to update the frame-level quality embeddings, and another Transformer was utilized to generate the video-level quality embeddings. To reduce the computational complexity of Transformers, Wu et al. [41] segmented the video frames into smaller patches, which were partially concatenated into fragments and fed into video Swin Transformer [42].
2.2 Adversarial attacks in classification
According to the knowledge of the attacker, adversarial attacks can be roughly divided into white-box and black-box attacks. In general, most white-box attacks rely on the propagated gradients of the target model. Goodfellow et al. [16] suggested that DNNs were susceptible to analytical perturbations due to their linear behaviors, and proposed the Fast Gradient Sign Method (FGSM). On this basis, Kurakin et al. [17] launched the attack via several steps, and clipped the pixel values within the  $\epsilon$ -neighbourhood after each step. Similarly, Madry et al. [20] proposed a multi-step variation of FGSM called Projected Gradient Descent (PGD), wherein the updated adversarial example was projected on the  $L_{\infty}$  norm in each PGD iteration. Carlini and Wagner [3] formulated the adversarial attack as an optimization problem with different objective functions, which aimed to find adversarial examples with minimum perturbations. Papernot et al. [18] created adversarial examples by iteratively modifying the most significant pixel based on the adversarial saliency map.
However, in real-world scenarios, the gradients of the model may not be accessible to the attacker. Therefore, the black-box setting is a more practical attack scenario. Papernot et al. [21] performed a black-box attack by training a local surrogate model, and then used this surrogate model to generate adversarial examples that could mislead the target model. Chen et al. [22] proposed the Zeroth Order Optimization (ZOO) based attack to estimate the gradients of the target model. To improve the query efficiency of ZOO, Ilyas et al. [23] replaced the conventional finite differences method with natural evolution strategy in gradient estimation. Guo et al. [24] proposed a simple black-box attack, which iteratively injected perturbations that could lead to performance degradation of target model in a greedy way. Xiao et al. [43] proposed AdvGAN to generate perturbations via feed-forward process, which could accelerate the generation of adversarial examples in the inference stage.
In this part, the adversarial attack on NR-VQA model is mathematically modelled as
$$
\underset {\mathbf {I} _ {x} ^ {\mathrm {a d v}}} {\operatorname {a r g m i n}} \mathcal {L} \left(f \left(\left\{\mathbf {I} _ {x} ^ {\mathrm {a d v}} \right\} _ {x = 1} ^ {X}\right), b \left(\left\{\mathbf {I} _ {x} \right\} _ {x = 1} ^ {X}\right)\right), \text {s . t .} \mathcal {D} \left(\left\{\mathbf {I} _ {x} \right\} _ {x = 1} ^ {X}, \left\{\mathbf {I} _ {x} ^ {\mathrm {a d v}} \right\} _ {x = 1} ^ {X}\right) \leq \mathrm {J N D}, \tag {1}
$$
where  $X$  denotes the number of frames within a video, and  $\{\mathbf{I}_x^{\mathrm{adv}}\}_{x = 1}^X$  and  $\{\mathbf{I}_x\}_{x = 1}^X$  denote the adversarial video and original video, respectively.  $f(\cdot)$  denotes the estimated quality score outputted
Algorithm 1 White-box attack on NR-VQA model
1: Require: Estimated score by target NR-VQA model  $f(\cdot)$ , anchor score  $b(\cdot)$ , number of frames optimized in one round  $T$ , number of iterations in one round  $K$ , and step size  $\beta$   
2: Input: Original video  $\{\mathbf{I}_x\}_{x=1}^X$   
3: Output: Adversarial video  $\{\mathbf{I}_x^{\mathrm{adv}}\}_{x=1}^X$   
4: for  $x = 0$  to  $X$  by  $T$  do // The optimization process contains  $[X/T]$  rounds.  
5:  $\{\mathbf{A}_x\}_{x}^{x + T} \gets$  Initialize the perturbations from a discrete set  $\{-1/255, 0, 1/255\}$   
6:  $\{\mathbf{I}_x^{\mathrm{adv}}\}_{x}^{x + T} \gets \{\mathbf{I}_x\}_{x}^{x + T} + \{\mathbf{A}_x\}_{x}^{x + T}$  // Initialize the adversarial video.  
7: for  $k = 0$  to  $K$  do // One round contains  $K$  iterations.  
8:  $g \gets \nabla_{\{\mathbf{I}_x^{\mathrm{adv}}\}_{x}^{x + T}} \mathcal{L}_{srb}\left(f\left(\{\mathbf{I}_x^{\mathrm{adv}}\}_{x}^{x + T}\right), b\left(\{\mathbf{I}_x\}_{x=1}^X\right)\right)$  // Compute the direction.  
9:  $\{\mathbf{I}_x^{\mathrm{adv}}\}_{x}^{x + T} \gets \{\mathbf{I}_x^{\mathrm{adv}}\}_{x}^{x + T} + \beta \cdot g$  // Update the adversarial video.  
10:  $\{\mathbf{I}_x^{\mathrm{adv}}\}_{x}^{x + T} \gets$  Limit the pixel-level  $L_2$  and  $L_\infty$  norm of perturbations
by the target NR-VQA model, and  $b(\cdot)$  denotes the anchor quality score which is far away from its MOS in a specific direction.  $\mathcal{L}(\cdot)$  is the loss function that measures the distance between the estimated and anchor quality scores. The closer the distance between the estimated and anchor quality scores, the further the distance between the estimated quality score and its MOS, and the stronger the effect of adversarial attack on misleading the estimated quality score.  $\mathcal{D}(\cdot)$  is the distortion between the original and adversarial videos. Restricting the distortion below the JND threshold indicates that the perturbations are imperceptible to human eyes. By means of minimizing the loss function under the JND constraint, the adversarial attack on NR-VQA model can be launched in an effective and undetectable manner.
To implement the adversarial attack according to Eq. (1), it is essential to design the loss function  $\mathcal{L}$ . Although Mean Squared Error (MSE) Loss has been widely applied in regression problem, it is not suitable for adversarial attack on NR-VQA model. The reason is that a well-performed adversarial attack on NR-VQA model should possess the capability of leading the estimated quality score far away from its MOS in a specific direction, i.e., misleading the NR-VQA model to assign high-quality score to low-quality video, and vice versa. Although applying MSE Loss may push the estimated quality score away from the MOS, it cannot control the leading direction. In this paper, a loss function called Score-Reversed Boundary Loss  $(\mathcal{L}_{srb})$  is proposed, which is formulated as
$$
\mathcal {L} _ {s r b} = \left| f \left(\left\{\mathbf {I} _ {x} ^ {\mathrm {a d v}} \right\} _ {x = 1} ^ {X}\right) - b \left(\left\{\mathbf {I} _ {x} \right\} _ {x = 1} ^ {X}\right) \right|, \tag {2}
$$
$$
b \left(\left\{\mathbf {I} _ {x} \right\} _ {x = 1} ^ {X}\right) = \text {b o u n d a r y} = \left\{ \begin{array}{l} 0, \text {i f} \left\{\mathbf {I} _ {x} \right\} _ {x = 1} ^ {X} \text {i s h i g h q u a l i t y ,} \\ 1, \text {i f} \left\{\mathbf {I} _ {x} \right\} _ {x = 1} ^ {X} \text {i s l o w q u a l i t y ,} \end{array} \right. \tag {3}
$$
where the high or low quality of a video in Eq. (3) can be determined by its MOS in practice, and the boundaries of 0 and 1 are regarded as the anchor quality scores.
Besides, the perturbation distortion  $\mathcal{D}$  between the original and adversarial videos is another issue to be addressed. In general, limiting  $\mathcal{D}$  below the JND threshold indicates that such perturbations are invisible to human eyes. However, JND is empirically determined by many factors, including brightness, contrast between the background, and individual visual sensitivity. Therefore, some fields convert it into certain objective metrics for simple modeling, such as PSNR in video coding [10], SSIM in image/video enhancement [11], and  $L_{p}$  norm in adversarial attacks [13]. Zhang et al. [44] introduced human subjective experiments to judge whether the perturbation distortion is below the JND threshold. However, such human subjective experiments are both time-consuming and labor-intensive. In this paper, we refer to the works in adversarial examples to construct the JND constraint [12; 13; 14]. Considering a pair of original and adversarial videos, the problem of constraining the distortion below the JND threshold can be converted into the problem of restricting their pixel-level  $L_{2}$  and  $L_{\infty}$  norm to be lower than a rather strict threshold. Consequently, the optimization based on subjective JND can be automatically performed by the objective metrics.
Figure 2: Illustration of the proposed patch-based random search method for black-box attack.
3.2 White-box adversarial attack
In the case of white-box attack, the parameters of the target model are accessible to the attacker. Therefore, the steepest descent method can be applied to minimize the Score-Reversed Boundary Loss in Eq. (2). The process of generating an adversarial video has  $\lfloor X / T\rfloor$  rounds, wherein  $T$  denotes the number of frames optimized in one round together. At the beginning of each round, the adversarial videos are generated by means of adding the original videos with initialized perturbations denoted as  $\{\mathbf{A}_x\}_{x}^{x + T}$ , whose element is independently sampled from a discrete set  $\{-1 / 255,0,1 / 255\}$ . Afterwards,  $K$  iterations are applied in this round to optimize the perturbations, and the detailed steps of one iteration are as follows. Firstly, the obtained adversarial videos are fed into the target NR-VQA model, and the steepest descent direction of  $\mathcal{L}_{srb}$  can be calculated to optimize the perturbations. Secondly, the projected gradient descent method is utilized to limit the pixel-level  $L_{2}$  and  $L_{\infty}$  norm of the perturbations within 1/255, which can be regarded as a kind of strict JND constraint. The pseudo-code of generating adversarial example for NR-VQA model under white-box setting is given in Algorithm 1.
3.3 Black-box adversarial attack
Compared with white-box attack, black-box attack is a more practical attack scenario, wherein only the final output of the model can be queried to optimize the adversarial examples. Note that the number of queries is an important evaluation metric in black-box attack. However, black-box attack on NR-VQA model pose great challenges from the following two aspects. Firstly, in most existing outstanding black-box attacks, the number of queries is positively correlated with the resolution of the video frame. As NR-VQA models are widely applied to 1080p and 2160p high-resolution datasets such as LIVE-VQC [15], YouTube-UGC [45], and LSVQ [46], its query resources may be quite expensive. Secondly, different frames within one video have different content characteristics. Therefore, to obtain better attack effect, each frame should be generated with a specific perturbation. However, one video may consist of hundreds of frames, which can lead to larger number of queries.
To reduce the query resources from the spatial and temporal perspective under black-box attack, we design a patch-based random search method, as illustrated in Fig. 2. In this part, the height and width of a video frame are denoted as  $H$  and  $W$ , respectively. The height and width of a patch are denoted as  $h$  and  $w$ , respectively. Each frame is divided into red, green, and blue channels, and each channel is further split into non-overlapping  $\left\lfloor \frac{H}{h} \right\rfloor \times \left\lfloor \frac{W}{w} \right\rfloor$  patches. In this way, each frame contains  $\left\lfloor \frac{H}{h} \right\rfloor \times \left\lfloor \frac{W}{w} \right\rfloor \times 3$  patches.
Like the white-box attack, we suppose that the process of generating an adversarial video has  $\lfloor X / T\rfloor$  rounds. In one round,  $N$  queries are performed. The detailed steps of the optimization process in the  $n$ -th query are as follows. Firstly, for each of the  $T$  frames,  $Z$  patches, which are located in the same positions within these  $T$  frames, are randomly selected to be perturbed. Note that in each round of the attack, each of the  $Z$  patches would be selected once and only once, and all patches would be selected in  $N$  queries. Therefore, we can obtain  $N\times Z = \lfloor \frac{H}{h}\rfloor \times \lfloor \frac{W}{w}\rfloor \times 3$ . For the convenience of
Algorithm 2 Black-box attack on NR-VQA model
1: Require: Estimated score by target NR-VQA model  $f(\cdot)$ , anchor score  $b(\cdot)$ , number of frames optimized in one round  $T$ , number of queries in one round  $N$ , perturbation magnitude  $\gamma$ , size of video frame  $H \times W$ , and size of patch  $h \times w$   
2: Input: Original video  $\{\mathbf{I}_x\}_{x=1}^X$   
3: Output: Adversarial video  $\{\mathbf{I}_x^{\mathrm{adv}}\}_{x=1}^X$   
4: for  $x = 0$  to  $X$  by  $T$  do // The optimization process contains  $\lfloor X / T \rfloor$  rounds.  
5:  $\{\mathbf{I}_x^{\mathrm{adv}}\}_{x}^{x + T} \gets \{\mathbf{I}_x\}_{x}^{x + T}$  // Initialize the adversarial video.  
6:  $R \gets \mathcal{L}_{srb}\left(f\left(\{\mathbf{I}_x^{\mathrm{adv}}\}_{x}^{x + T}\right), b\left(\{\mathbf{I}_x\}_{x=1}^X\right)\right)$   
7: for  $n = 0$  to  $N$  do // One round contains  $N$  queries.  
8:  $Z \gets \max\left(\lfloor \frac{H}{h} \rfloor \times \lfloor \frac{W}{w} \rfloor \times 3 / N \rfloor, 1\right)$  // Select  $Z$  patches to perturb in one frame.  
9:  $\mathbf{p}_n \gets$  Encode information of the positions of the selected patches  
10:  $\mathbf{m}_n \gets$  Generate universal perturbation map from a discrete set  $\{-\gamma, +\gamma\}$   
11: for operation  $\in \{-, +\}$  do  
12:  $\left\{\tilde{\mathbf{I}}_x^{\mathrm{adv}}\right\}_{x}^{x + T} \gets$  Selected patches are injected with  $\mathbf{m}_n$  according to given operation  
13:  $R' \gets \mathcal{L}_{srb}\left(f\left(\left\{\tilde{\mathbf{I}}_x^{\mathrm{adv}}\right\}_{x}^{x + T}\right), b\left(\left\{\mathbf{I}_x\right\}_{x=1}^X\right)\right)$  // Evaluate the attack effect.  
14: if  $R' < R$  then  
15:  $\left\{\mathbf{I}_x^{\mathrm{adv}}\right\}_{x}^{x + T} \gets \left\{\tilde{\mathbf{I}}_x^{\mathrm{adv}}\right\}_{x}^{x + T}$  // Update the adversarial video.  
16:  $R \gets R'$   
17: break
illustration, the information of the positions of the selected patches is encoded into a vector denoted as  $\mathbf{p}_n^{K\times 1}$ , where  $K = \lfloor \frac{H}{h} \rfloor \times \lfloor \frac{W}{w} \rfloor \times 3$ . Each element in  $\mathbf{p}_n$  corresponds to a specific patch within the frame, and the element equals to 1 if such patch is perturbed, and 0 otherwise. Note that each of the  $T$  frames optimized in one round shares the same  $\mathbf{p}_n$ . Secondly, for these  $T \times Z$  patches to be perturbed, a universal perturbation map  $\mathbf{m}_n$  is generated, wherein its dimension is the same as the dimension of the patch, and its element is independently sampled from a discrete set  $\{-\gamma, +\gamma\}$ . Thirdly, the selected  $T \times Z$  patches are subtracted or added with the universal perturbation map, and the generated adversarial video is fed into the target NR-VQA model. Such perturbations would be kept if the Score-Reversed Boundary Loss in Eq. (2) decreases, and abandoned otherwise. Following the above three steps, an adversarial video with  $X$  frames can be generated. The pseudo-code of generating adversarial example for NR-VQA model under black-box setting is given in Algorithm 2.