SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zhiyu Lin $^{1}$ , Yifei Gao $^{1}$ , Yunfan Yang $^{1}$ , Jitao Sang $^{1,2*}$
<sup>1</sup>Beijing Jiaotong University, China
$^{2}$ Peng Cheng Lab, Shenzhen 518066, China
{zyllin, yf-gao, yfyang, jtsang}@bjtu.edu.cn
A widely discussed hypothesis regarding the cause of visual models' lack of robustness is that they can exploit human-imperceptible high-frequency components (HFC) in images, which in turn leads to model vulnerabilities, such as the adversarial examples. However, (1) inconsistent findings regarding the validation of this hypothesis reflect in a limited understanding of HFC, and (2) solutions inspired by the hypothesis tend to involve a robustness-accuracy trade-off and leaning towards suppressing the model's learning on HFC. In this paper, inspired by the long-tailed characteristic observed in frequency spectrum, we first formally define the HFC from long-tailed perspective and then revisit the relationship between HFC and model robustness. In the frequency long-tailed scenario, experimental results on common datasets and various network structures consistently indicate that models in standard training exhibit high sensitivity to HFC. We investigate the reason of the sensitivity, which reflects in model's under-fitting behavior on HFC. Furthermore, the cause of the model's under-fitting behavior is attributed to the limited information content in HFC. Based on these findings, we propose a Balance Spectrum Sampling (BaSS) strategy, which effectively counteracts the long-tailed effect and enhances the model's learning on HFC. Extensive experimental results demonstrate that our method achieves a substantially better robustness-accuracy trade-off when combined with existing defense methods, while also indicating the potential of encouraging HFC learning in improving model performance.
The disparity between deep visual models and the human visual system (HVS) in image signal processing enables the models to outperform the HVS on vision benchmarks [16, 33]. However, it is worth noting that most visual models lack the robustness exhibited by HVS [36, 17]. Specifically, the generalization advancement of visual models can be partially explained by the utilization of human-imperceptible information, such as high frequency components (HFC) [4, 39, 29]. Additionally, the adversarial vulnerability is also closely related to this information, as observed in adversarial examples phenomena [36]. The coexistence of both conditions gives rise to a widely discussed hypothesis: the vulnerability of the model stems from the utilization of imperceptible high-frequency information in images.
Based on this hypothesis, a line of research has been conducted to explain and enhance the robustness of the model [43, 1, 7, 9, 8, 35, 13]. However, these works share a common limitation: most of them solely rely on qualitative analysis of frequency components observed by humans, without further delving into the statistical properties. This limitation gives rise to the following issues in existing
works: (i) the division of the frequency spectrum is intuitively based on manually set radius, resulting in inconsistent conclusions across different datasets and models; and (ii) The understanding of high frequency utilization is limited. For example, improving model robustness often involves suppressing the learning on HFC, which is in stark contrast to the promotion of high frequency utilization to improve generalization performance. This leads to a trade-off between robustness and accuracy in some defense methods.
In this paper, to overcome the challenges posed by the aforementioned limitation, we are motivated by the long-tailed characteristics in the spectrum. Specifically, the power spectral density of an image naturally follows a power-law distribution, where a few low-frequency bands accounts for the majority of the spectrum power while a large number of high-frequency bands maintains a low power distribution. This new realization encourages us to revisit the significance of high-frequency components in addressing the problem of model robustness under the guidance of deep long-tailed learning.
In deep long-tailed learning [46], the presence of a large number of tail classes plays a critical role in improving model performance. These tail classes offer benefits such as ensuring data integrity and diversity [5] and capturing rare events [28]. However, it is commonly observed that models tend to under-fit these classes due to their numerical disadvantage. Intuitively, the long-tailed characteristics of the frequency spectrum also exhibit similar phenomena. We confirm the model's (standard training) under-fitting behavior on high frequencies through the lens of loss landscape [27]. Under the context of model under-fitting, the discussion regarding HFC and model generalization performance can be compatible in most studies [29, 39], as the common underlying cause is the low generalization resulting from under-fitting. In this paper, we establish a link between under-fitting behavior and model robustness, specifically identifying that the model's under-fitting of high-frequency components leads to high sensitivity towards them. We validate this relationship across different datasets and models, consistently confirming our findings. Model's under-fitting behavior on high frequency leads to both low generalization and high sensitivity towards HFC. Thus, it is crucial to improve the under-fitting behavior and further mitigate the trade-off between accuracy and robustness.
Drawing inspiration from deep long-tail learning, we introduce spectral entropy as a measure of the information content of frequency components and provide an explanation for model's under-fitting behavior on HFC. We attribute model's under-fitting behavior to the low spectral entropy of HFC. Based on this perspective, the principle of maximum entropy [23, 3, 32] guides us in constructing an image spectrum with a more balanced energy distribution to address the issue of low information entropy. Finally, our quantitative experiments and theoretical analyses inspire us to propose a simple yet highly effective Balance Spectrum Sampling (BaSS) strategy, which can work in conjunction with different training paradigms and enables the model to achieve a better trade-off between robustness and accuracy.
Our major contributions are as follows: (1) To the best of our knowledge, we are the first to propose focusing on the long-tailed distribution in instance-wise, specifically in the frequency domain. This novel perspective provides insights for analyzing and improving model performance. (2) We revisit the relationship between HFC and model robustness, revealing the under-fitting phenomenon and its association with high sensitivity and low generalization at high frequencies. To address the trade-off between accuracy and robustness, we propose to enhance the high-frequency learning. (3) We explain the under-fitting behavior through the lens of spectral entropy. Drawing inspiration from the principle of maximum entropy, we propose a simple yet highly effective spectral sampling strategy (BaSS) to improve the model's under-fitting behavior. By incorporating BaSS with other training paradigms, such as adversarial training and AugMix, model achieves better trade-off between accuracy and robustness. Comprehensive experiments and analyses reveal the effectiveness of our method.
2 The long-Tailed Distribution in Frequency Domain
2.1 Notations and Preliminaries
Image Processing in the Frequency Domain. We perform 2-D discrete Fourier transform (referred to as the DFT)  $\mathcal{F}$  on image  $X\in \mathbb{R}^{H\times W}$  to get Fourier spectrum  $\tilde{X}\in \mathbb{C}^{H\times W}$ . Unless otherwise specified, the low frequencies are shifted towards the center of the Fourier spectrum. We denote  $\tilde{X} [u,v]$  as the frequency component where  $u\in \{0,\dots ,H - 1\}$  and  $v\in \{0,\dots ,W - 1\}$ . Addition-
ally,  $\omega_{u,v}$  denotes the Fourier Basis function [38] and  $r_{u,v}$  denotes the Euclidean Distance from the spectrum coordinate  $[u,v]$  to the center of spectrum, where a larger distance corresponds to a higher frequency, and vice versa.
Power Spectral Density. The power spectral density is defined by azimuthally averaging the magnitude of Fourier coefficients over radial frequencies. Specifically, the value of power density is estimated as the azimuthal integration (referred to as  $AI$ ) over radial frequencies  $\phi$  according to Eq.1.
$$
A I \left(r _ {k}\right) = \int_ {0} ^ {2 \pi} \| \tilde {X} \left[ r _ {k} \cdot \cos \phi , r _ {k} \cdot \sin \phi \right] \| ^ {2} d \phi \tag {1}
$$
where  $r_k$  is the Euclidean Distance from the  $k^{th}$  frequency band to the center of the spectrum [6, 14],  $k \in \{0, \dots, H/2 + 1\}$ .
2.2 Definition of Frequency Long-Tailed Scenario
Real-world data generally demonstrates a long-tailed distribution, where a small number of dominant categories comprise the majority of the data, and a large number of minority categories possess limited samples. Although previous studies [46, 37] have primarily investigated the degradation in model performance due to the long-tailed problem at the class level, they have not adequately considered the potential existence of a long-tailed issue within individual images. Through our analysis of the Fourier spectrum, we observe that the power spectral density follows a power-law distribution, which also exhibits the long-tailed characteristics. Specifically, the distribution (y-axis) of  $i^{th}$  frequency band (x-axis) is calculated by the proportion of power in  $i^{th}$  band to the total power of the spectrum, which is denoted as  $\pi_{i} = AI(r_{i}) / \sum_{k}AI(r_{k})$ . Without loss of generality,  $\{\pi_i\}$  naturally present in decreasing trend (i.e., if  $k_{1} < k_{2}$ , then  $\pi_{k_1} < \pi_{k_2}$ ).
Consequently, we summarize the statistics of the frequency feature from the long-tailed model perspective as follows: (i) The head part, which is consisted of a few low-frequency bands, accounts for the majority of the spectrum power and is highly semantic and information dense; (ii) The tail part, which is composed of a large number of high-frequency bands, however maintains a low power distribution. This part includes high-frequency semantic information (e.g., texture) as well as human-imperceptible information.
$$
LFC = \{\tilde{X} [ u,v] \mid \max_{n}\sum_{\substack{i = 0\\ H / 2 + 1}}^{n}\pi_{i}\leq 80\% ,r_{u,v}\leq r_{n}\} \tag{2}
$$
$$
H F C = \{\tilde {X} [ u, v ] \mid \min  _ {n} \sum_ {i = n} ^ {H / 2 + 1} \pi_ {i} \geq 20 \%, r _ {u, v} \geq r _ {n} \}
$$
Long Tail based Spectrum Division. Previous analysis [15, 39, 31] indicating that a few low-frequency components form the image's main content, while most high-frequency components comprise a minor part. The rule that few factors determine main outcomes is encapsulated by the Pareto Principle and the "80/20" quantitative rule, found across various domains, where  $80\%$  of effects are caused by  $20\%$  of the reasons. Guided by this rule, we defined the boundary between frequencies using an 80/20 spectral energy ratio. We formally define the low- and high-frequencies according to the Eq.2, where the  $r_n$  is the Euclidean Distance from the  $n^{th}$  frequency band to the center of the spectrum. The set composed of  $\tilde{X}[u,v]$  occupying  $80\%$  of the total power in the spectrum is defined as the low-frequency component, and the remaining portion of the spectrum is defined as the high-frequency component. We experimented with various datasets, finding that different resolution images' high-low frequency boundaries are at a  $20\%$  radius from the spectrum's center (as shown in Appendix B.2). This confirms the stability and reasonableness of the classification system using spectral energy and the Pareto Principle.
3 Investigating and Explaining Model Robustness Behaviors: A Frequency Long-tailed View
Vision models have demonstrated a capability in learning frequency information [24, 39, 29, 43]. Intuitively, model's behavior could be affected by the long-tailed characteristics in the frequency
Figure 1: Loss landscape visualization of HFC (a) and LFC (b). We conduct experiments on ResNet-18 naturally trained on CIFAR10 dataset. The black star symbol represents model parameters  $\theta$ . A noticeable phenomenon is that the model exhibits a learning bias towards different frequency components: specifically, the learning for high-frequency components has not converged, while the learning for low-frequency components has already reached a local optimum.
domain, which has not been fully explored in previous studies, and is crucial for improving model performance in visual tasks. In this section, we first summarize the impact of long-tailed characteristics as a phenomenon of the model's under-fitting of HFC (Sec. 3.1). We then explore the model's sensitivity to HFC from the perspective of under-fitting, and revisit the relationship between HFC and robustness (Sec. 3.2). Finally we explain the model's under-fitting behavior from the perspective of spectral entropy (Sec. 3.3)
3.1 Model's Under-fitting Behavior on HFC
In the long-tailed theory, the tail part is considered more deserving of attention than the head part. On one hand, the tail part is often overlooked by the system due to its numerical disadvantage(e.g., the sample size of the tail class is far less than that of the head class in the context of class-wise long tail). On the other hand, it plays a crucial role in enhancing the system's performance, primarily due to two characteristics (i) providing data integrity and diversity [5], and (ii) capturing rare events [28]. In this subsection, we explore the learning behavior of models regarding frequency components. Specifically, we visualize and analyze the loss landscape [27] of the classifier on both LFC and HFC. We present the detailed experimental procedure in the Appendix C.
Phenomenon of Model Under-fitting on HFC. The convergence result of LFC and HFC learning exhibits significant differences as shown in Fig. 1. Fig. 1(b) suggests that the loss landscape on LFC exhibits a benign characteristic: the parameter space around the model  $\theta$  is relatively flat and close to the local minima, indicating that the model has precisely converged on LFC. In contrast, the loss landscape on HFC shown in Fig. 1(a) manifests a dramatic non-convexity in certain regions. It can be observed that the model  $\theta$  are located in a steep region of the landscape, with loss rapidly decreasing when moving along the gradient directions (i.e., which are perpendicular to the contours). Additionally, in the bottom-left region of the Fig. 1(a), we also observe a better local minimum far away from current model. These observations indicate that the model still has the potential to improve its ability to learn HFC.
Positive Correlation between Generalization and HFC Learning. It is widely recognized that under-fitting is a cause of low generalization performance. Recent works have confirmed that models exhibit lower generalization ability on HFC compared to LFC [39, 29]. Meanwhile, ignoring the HFC can lead to inaccurate representation of the data distribution, resulting in a degradation in model generalization [39, 4]. This correlation can be attributed to the first characteristic of tail components mentioned above. Hence, it is vital to devise and employ strategies that alleviate or eliminate the under-fitting of HFC, as it substantially contributes to the enhancement of model generalization performance [39, 29].
(b) Tiny-ImageNet
(c) ImageNet
Figure 2: Model sensitivity to frequency components on CIFAR10, Tiny-ImageNet and ImageNet datasets. We present the sorted average gradient magnitude on test set in the first row, with the  $x$ -axis denoting the rank index. In the second row, we select part of rank positions, and display the density distribution of frequency radius. The gray dashed line is served as the boundary between low- and high-frequencies. The overall phenomenon is that naturally trained model is highly sensitive to tail frequency components.
3.2 Revisiting the Relationship between HFC and Model Robustness
3.2.1 Motivation
The phenomenon of under-fitting inspires us to revisit the issue of model robustness. Robustness is typically expressed as the sensitivity of model outputs to perturbations in inputs, and it can be evaluated in two ways: (i) changes of the outputs lead by introducing additional perturbations to the inputs; and (ii) inputs gradients with respect to outputs. The results shown in Fig. 1(a) indicate that standard training classifier still exhibit significant loss values in HFC and may have large gradients with respect to high-frequencies, thus resulting in model vulnerability. In the context of long-tailed theory, the tail part usually represents special or extreme cases and thus is more likely to be affected by rare events, making HFC crucial for evaluating and analyzing model robustness. Existing studies [43, 7, 9] suggest that the vulnerability of models is attributed to HFC, however, this conclusion is not consistent across different datasets and models. We attribute the potential causes of inconsistency to the definition of HFC. We aim to further explore relationship between HFC and model robustness.
3.2.2 Analyzing Model Sensitivity to Frequency Components
Model sensitivity to different frequency components has been explored using the Fourier heatmap, which quantifies the changes in outputs when applying Fourier basis perturbations to images [43]. However, there are a few concerns about its applicability: (i) The theory of Fourier basis is based on linear networks while deep neural networks typically exhibit high non-linearity, (ii) The perturbations is excessively sensitive to the magnitude of the basis disturbance. Thus, we take inspiration from [29, 7] and employ the loss gradient across various frequency components to examine sensitivity.
Estimation of Model Sensitivity. Given an input image  $X$  with label  $y$ , the output of classifier  $f$  is  $f(X)$ . The objective function of model's training is formulated as  $\mathcal{L}(f(X),y)$ . Consequently, the gradient of the loss w.r.t. the original input  $X$  can be expressed as  $g = \nabla_X\mathcal{L}(f(X),y)\in \mathbb{R}^{H\times W}$ , whose magnitude can indicate the sensitivity of the classifier to pixel domain perturbations. To further assess the model's sensitivity in frequency domain, we employ the set of orthogonal Fourier basis vectors  $\{\omega_{u,v}\}$  and project  $g$  into frequency domain coordinate system. The corresponding coordinate of projection in  $\omega_{u,v}$  is defined as follows:
$$
c _ {u, v} = \left\langle g, \omega_ {u, v} \right\rangle = \sum_ {m = 0} ^ {H - 1} \sum_ {n = 0} ^ {W - 1} g [ m, n ] \cdot e x p \{- i 2 \pi \cdot \left(\frac {m u}{H} + \frac {n v}{W}\right) \}. \tag {3}
$$
The sensitivity of model to the frequency components corresponding to the spectrum point  $[u, v]$  is measured by the gradient magnitude on  $\omega_{u, v}$ , which is defined as  $\| c_{u, v} \|_2$ . In order to identify the frequency domain sensitivity patterns of the model at the dataset level, we first sort  $w_{u, v}$  according to  $\| c_{u, v} \|_2$  on a single image. For ease of statistical analysis, we replace  $w_{u, v}$  with the frequency radius  $r_{u, v}$  as the feature vector. Finally, we get the set of feature vector for all images and estimate the density of frequency radius using kernel density estimation (KDE) within the same rank index. We analyze the frequency sensitivity across the CIFAR10 [25], Tiny-ImageNet [26], and ImageNet [12] datasets with resolutions of  $32^2$ ,  $64^2$ , and  $224^2$  pixels, respectively. We show detailed settings in the Appendix D.
Results and Discussions. From the first row of Fig. 2, we observe an imbalanced distribution of gradients in different frequency bands, and the degree of imbalance is quite pronounced, resembling a similar long-tailed distribution. The results in the second row of Fig. 2 indicate that the density peaks of frequency radius in high-magnitude gradients are mainly concentrated in the tail high-frequencies (i.e., represented by the darker colored curves). These observations clearly demonstrate the relationship between HFC and model vulnerability, indicating that perturbations in HFC are more easier to cause errors in the model. In the result of ImageNet as shown in Fig. 2, we notice a particular phenomenon where there are also large gradient magnitude in the head components. This could be due to the wide spectral range causing similar vulnerabilities in the head components. However, this phenomenon does not affect the consistency of our conclusions. It also suggests that addressing robustness in the ImageNet dataset is a more challenging task.
3.3 Reasons leads to the Under-fitting Behavior
In deep long-tailed learning, the lack of sufficient generalizable information on tail classes directly causes biased behavior in models. Similar phenomena inspire us to explain the under-fitting of the tail part from an information-theoretic perspective in the case of frequency long-tailed problems.
3.3.1 The Spectral Entropy of HFC is Much Lower than LFC
Spectral Entropy To quantitatively characterize the amount of information contained in a given image, spectral entropy is utilized analogously to Shannon entropy in information theory. The continuous frequency domain is discretized and represented by a discrete variable  $\xi$ , with  $\xi_{l}$  and  $\xi_{h}$  as its minimum and maximum values, respectively. Given that the spectral density of natural images follows a power-law distribution, its density function can be computed and normalized as follows:  $p(\xi = k) = \frac{1 - \alpha}{\xi_h^{1 - \alpha} - \xi_l^{1 - \alpha}}\cdot k^{-\alpha}$ , where  $\alpha$  is the parameter determining the magnitude of the long-tailed effect, and its smaller value indicates a more significant long-tailed effect. We use  $\pi_{i}$  to estimate  $p_i$ , and then the spectral entropy of  $k^{th}$  band is derived by applying the Shannon entropy formula:  $H_{k} = -\pi_{k}\log \pi_{k}$ . Consequently, the flat spectrum corresponds to maximal spectral entropy, while the spectrum of a single frequency signal has minimal spectral entropy, which is zero.
Theorem 1 Given any natural image  $X$ ,  $k$  represents the order of frequency band, then the spectral entropy  $H_{k}$  is given by:  $H_{k} = (\alpha A_{\alpha}\log k + H(A(\alpha)))\cdot k^{-\alpha}$ , where  $A(\alpha) = (1 - \alpha)\cdot (\xi_h^{1 - \alpha} - \xi_l^{1 - \alpha})^{-1}$ . Then we have,  $\lim_{k\to 0}H_k = \infty$  and  $\lim_{k\to \infty}H_k = 0$ .
Theorem 1 demonstrates an inverse proportional relationship between the frequency band and spectral entropy, decreasing at an  $\alpha$  order. This indicates that high-frequency bands in the tail contain substantially less information than low-frequency bands in the head. Unlike measuring the number of image samples, we provide a new way to measure image information from the frequency domain perspective, explaining the model's under-learning behavior for low-information data. Moreover, Theorem 1 also suggests that high-resolution data, such as ImageNet, often lack effective generalizable information in the tail. This finding aligns with Fig. 2, illustrating that the model fails to generate gradient components in the ultra-high frequency range of ImageNet data.
3.3.2 Balanced Spectral Entropy Benefits the Under-fitting Phenomena
The maximum entropy principle seeks to make reliable predictions on unknown data distributions by selecting probability distributions with the highest entropy, without introducing unnecessary biases or prior knowledge. Both generalization and robustness reflect a model's ability to infer unknown samples, which depends on its capacity to capture information during training. Based on