SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Textual Prototypes To establish textual prototypes, we do not estimate  $\mathbf{s}^c$  from instances in one class (e.g., average) considering the insufficient, noisy nature of metadata. Instead, we refer to WordNet [85] for the vocabulary hierarchy [86, 78, 24]. For the  $c$ -th class, we extract its definition in WordNet and expand context with its siblings (synonyms), children (hyonyms), and parents (hypernyms). Then, we get  $\mathbf{t}^c$  and encode it for  $\mathbf{s}^c$ . Such prototypes  $\mathbf{s}^c$  have two advantages: 1) It enriches semantic representations of classes. For instance, the comprehensive text of the class "tiger cat" is a cat having a striped coat; domestic_cat, house_cat, felis_domesticus, felis_catus: any domesticated member of the genus Felis. It provides disambiguation to web instances of "tiger-cat" (medium-sized wildcat in Central South America) and "tiger, cat" (large feline of forests in most of Asia having a tawny coat with black stripes; endangered). 2) It reveals the underlying inter-class relationship by language encoding. The structural information of class hierarchy is hard to infer from metadata instances but can be directly indexed in WordNet.
Text Matching With textual prototypes as queries, web instances with correct semantics can be retrieved by matching queries to their embeddings as keys. To improve precision, the same distance measurement in Eq. (3) for  $k$ -reciprocal-NN encoding is adopted to rerank the matched candidates. We sort samples by distance in an ascending order, and select the top- $K$  as clean set  $D_K$ .
$$
D _ {K} = D _ {K} ^ {1} \cup D _ {K} ^ {2} \cup \dots \cup D _ {K} ^ {C}, D _ {K} ^ {c} = \left\{\left(\mathbf {x} _ {i}, \mathbf {t} _ {i}, y _ {i}\right) | \left(y _ {i} = c\right) \wedge \left(d ^ {*} \left(\hat {\mathbf {s}} _ {i}, \mathbf {s} ^ {c}\right) \leq \sigma_ {K} ^ {c}\right) \right\}, \tag {4}
$$
where  $\sigma_K^c$  denotes the  $K$ -th smallest distance in the  $c$ -th class.
Visual Prototypes  $D_K$  plays an anchoring role in shaping visual prototypes. We initialize the  $c$ -th prototype  $\mathbf{z}^c$  by averaging instances in  $D_K^c$ :  $\hat{\mathbf{z}}^c = \frac{1}{K}\sum_{\mathbf{x}_i\in D_K^c}\mathbf{z}_i$ ,  $\mathbf{z}^c = \frac{\hat{\mathbf{z}}^c}{\|\hat{\mathbf{z}}^c\|_2}$ . Given such a good starting point, visual prototypes are consistently polished up by trustworthy web examples with a momentum coefficient  $m_p$ :  $\hat{\mathbf{z}}^c = m_p\mathbf{z}^c + (1 - m_p)\mathbf{z}_i$ ,  $\mathbf{z}^c = \frac{\hat{\mathbf{z}}^c}{\|\hat{\mathbf{z}}^c\|_2}$ . We perform instance-prototype contrastive learning to pull instances around their prototypes and push apart different class clusters. Instance-level discrimination is also encouraged to improve separation across classes.
$$
\mathcal {L} _ {i} ^ {\text {p r o}} = - \log \frac {\exp \left(\mathbf {z} _ {i} \cdot \mathbf {z} ^ {y _ {i}} / \tau\right)}{\sum_ {c = 1} ^ {C} \exp \left(\mathbf {z} _ {i} \cdot \mathbf {z} ^ {c} / \tau\right)}, \mathcal {L} _ {i} ^ {\text {i n s}} = - \log \frac {\exp \left(\mathbf {z} _ {i} \cdot \mathbf {z} _ {i} ^ {\prime} / \tau\right)}{\sum_ {j = 1} ^ {Q} \exp \left(\mathbf {z} _ {i} \cdot \mathbf {z} _ {j} ^ {\prime} / \tau\right)}, \tag {5}
$$
where  $\tau$  is a temperature coefficient.
Noise Removal Noisy instances can be filtered out by self-prediction and instance-prototype similarity. We refer to MoPro [28] for rules of label adjustment by  $\mathbf{o}_i\in \mathbb{R}^C$  ..
$$
\mathbf {o} _ {i} = \alpha \mathbf {p} _ {i} + (1 - \alpha) \mathbf {r} _ {i}, \mathbf {r} _ {i (k)} = \frac {\exp \left(\mathbf {z} _ {i} \cdot \mathbf {z} ^ {k} / \tau\right)}{\sum_ {c = 1} ^ {C} \exp \left(\mathbf {z} _ {i} \cdot \mathbf {z} ^ {c} / \tau\right)}, \tag {6}
$$
where  $\alpha$  balances two terms. Given a threshold  $0\leq \gamma \leq 1$ , the pseudo-label  $\hat{y}_i$  is estimated by:
$$
\hat {y} _ {i} = \left\{ \begin{array}{l l} y _ {i} & \text {i f} \mathbf {x} _ {i} \in D _ {K}, \\ \arg \max  _ {c} \mathbf {o} _ {i (c)} & \text {e l s e i f} \max  _ {c} \mathbf {o} _ {i (c)} > \gamma , \\ y _ {i} & \text {e l s e i f} \mathbf {o} _ {i (y _ {i})} > 1 / C, \\ \operatorname {N u l l} (O O D) & \text {o t h e r w i s e .} \end{array} \right. \tag {7}
$$
The above control flow guarantees continuous guidance from  $D_K$  on cluster separation. If the highest score of  $\mathbf{o}_i$  is above  $\gamma$ , the label will be changed accordingly. To prevent aggressive elimination of hard examples, we keep an instance till the next epoch so long as its confidence is above average. Otherwise, it is removed as OOD. The refined label  $\hat{y}_i$  successively affects Eqs. (1) and (5).
3.3 Collective Bootstrapping
Due to memorization [87], as the training epoch increases, deep models will be prone to overfit noise even with the carefully designed logic of noise removal. We assume that overfitting occurs less dramatically when a majority can be consulted for the model to avoid over-confident decision on one single instance. With regard to the consultancy basis, the low-dimensional embedding is a good choice because its distilled description about visual contents is robust enough. Therefore, we propose to exploit the large dictionary, which is originally set up for instance-wise contrastive learning, to realize collective bootstrapping by dictionary look-up. The matching scores of the current query  $\mathbf{z}_i$  to all keys  $\mathbf{z}_j^{\prime}$  in the dictionary act as the weights for the bootstrapped representations  $\mathbf{b}_i\in \mathbb{R}^C$ .
$$
\mathbf {b} _ {i} = \sum_ {j = 1} ^ {Q} w _ {i j} \left(\alpha \mathbf {q} _ {j} ^ {\prime} + (1 - \alpha) \mathbf {r} _ {j} ^ {\prime}\right), w _ {i j} = \frac {\exp \left(\mathbf {z} _ {i} \cdot \mathbf {z} _ {j} ^ {\prime} / \tau\right)}{\sum_ {j = 1} ^ {Q} \exp \left(\mathbf {z} _ {i} \cdot \mathbf {z} _ {j} ^ {\prime} / \tau\right)}, \mathbf {r} _ {j (k)} ^ {\prime} = \frac {\exp \left(\mathbf {z} _ {j} ^ {\prime} \cdot \mathbf {z} ^ {k} / \tau\right)}{\sum_ {c = 1} ^ {C} \exp \left(\mathbf {z} _ {j} ^ {\prime} \cdot \mathbf {z} ^ {c} / \tau\right)}. \tag {8}
$$
We minimize the difference between predictions and bootstrapping targets via a KL-divergence loss.
$$
\mathcal {L} _ {i} ^ {\mathrm {b t s}} = D _ {K L} \left(\mathbf {q} _ {i} \| \mathbf {b} _ {i}\right) = \sum_ {c = 1} ^ {C} \mathbf {q} _ {i (c)} \log \frac {\mathbf {q} _ {i (c)}}{\mathbf {b} _ {i (c)}}. \tag {9}
$$
It not only allows collaborative contribution to individual soft label estimation, but also encourages consistent performance on visually similar examples. Note that such regularization is imposed on the auxiliary classifier  $\mathbf{q}_i$ . Compared with  $\mathbf{p}_i$ , constraints on  $\mathbf{q}_i$  coincide with our contrastive learning setting without potential conflicts with the hard label assignment in Eq. (7). The total objective is:
$$
\mathcal {L} = \sum_ {i = 1} ^ {N} (1 - \lambda^ {\mathrm {b t s}}) \mathcal {L} _ {i} ^ {\mathrm {c l s}} + \lambda^ {\mathrm {b t s}} \mathcal {L} _ {i} ^ {\mathrm {b t s}} + \lambda^ {\mathrm {p r j}} \mathcal {L} _ {i} ^ {\mathrm {p r j}} + \lambda^ {\mathrm {p r o}} \mathcal {L} _ {i} ^ {\mathrm {p r o}} + \lambda^ {\mathrm {i n s}} \mathcal {L} _ {i} ^ {\mathrm {i n s}}.
$$
We evaluate CAPro on WebVision1k [3] (Google500 [78]) and NUS-WIDE (Web) [88] for single-label and multi-label representation learning, respectively. They contain image-text pairs which are in line with our WSL setting. All datasets under investigation are described in Sec. A. We perform ablation studies on Google500 and NUS-WIDE for low cost without losing generalization [22, 78]. The R50 [89]/MiniLM (L6) [30] are used as image/text encoders by default. Exhaustive details about hyper-parameters, implementation, and training are elaborated in Secs. B C and Algo. 1.
4.2 Comparison with the SOTA
Table 1 reports the top1/top5 accuracy of WebVision1k and Google500. Results of the SOTA methods trained and evaluated on the same datasets are quoted here. Due to different choices of image encoders and training strategies, prior methods may not be directly comparable. For example, VSGraph adopts
Table 1: Results on WebVision1k and Google500. Best/2nd best are marked bold/underlined.
Method	Back-bone	WebVision1k	ImageNet1k	Google500	ImageNet500
Top1	Top5	Top1	Top5	Top1	Top5	Top1	Top5
MentorNet [17]	IRV2 [90]	72.6	88.9	64.2	84.8	-	-	-	-
Curriculum [16]	IV2 [91]	72.1	89.1	64.8	84.9	-	-	-	-
Multimodal [92]	IV3 [93]	73.2	89.7	-	-	-	-	-	-
Vanilla [22]	R50D [94]	75.0	89.2	67.2	84.0	75.4	88.6	68.8	84.6
SCC [22]	R50D	75.3	89.3	67.9	84.7	76.4	89.6	69.7	85.3
\( Vanilla^† \)[78]	R50	74.2	89.8	68.2	86.2	66.9	82.6	61.5	78.8
CoTeach [20, 78]	R50	-	-	-	-	67.6	84.0	62.1	80.9
\( VSGraph^† \)[78]	R50	75.4	90.1	69.4	87.2	68.1	84.4	63.1	81.4
Vanilla [28]	R50	72.4	89.0	65.7	85.1	-	-	-	-
SOMNet [8]	R50	72.2	89.5	65.0	85.1	-	-	-	-
Curriculum [16]	R50	70.7	88.6	62.7	83.4	-	-	-	-
CleanNet [18]	R50	70.3	87.7	63.4	84.5	-	-	-	-
SINet [24]	R50	73.8	90.6	66.8	85.9	-	-	-	-
NCR [58]	R50	73.9	-	-	-	-	-	-	-
\( NCR^† \)[58]	R50	75.7	-	-	-	-	-	-	-
MILE [26]	R50	75.2	90.3	67.1	85.6	-	-	-	-
MoPro [28]	R50	73.9	90.0	67.8	87.0	-	-	-	-
Vanilla (ours)	R50	72.6	89.7	67.0	86.8	69.9	86.5	64.5	83.1
CAPro (ours)	R50	74.2	90.5	68.0	87.2	76.0	91.3	72.0	89.2
$\dagger$  Results on WebVision1k are under optimized training settings with batch size of 1024.
the same R50, but is trained with a batch-size of 1024. The benefits of a larger batch size have been validated in NCR, where the same method achieves  $75.7\%$  and  $73.9\%$  in top1 accuracy respectively for the batch size of 1024 and 256. We believe batch size is the reason that a vanilla baseline [78] surpasses most SOTAs. Due to the limited budget, training with a batch size of 1024 is currently not affordable, but we will experiment in the future. In this case, methods within each row group of Table 1 are fairly comparable with each other, including the vanilla trained by a cross-entropy loss.
CAPro achieves quite competitive performance on WebVision1k, with an improvement of  $1.6\%$  (top1 accuracy) over our vanilla. Although SCC and VSGraph respectively opt for stronger backbones (e.g., R50D) and longer training epochs (e.g., 150) with a larger batch size (e.g., 1024), CAPro still excels in terms of the top5 accuracy. Furthermore, our gain of  $1\%$  (top1 accuracy) on ImageNet1k demonstrates that CAPro is robust to the domain gap between web and real-world datasets. Web data include advertisements, artworks, and renderings that differ from realistic photographs. On Google500 and ImageNet500, CAPro outperforms existing methods despite our disadvantages.
Table 2: Results on NUS-WIDE (Web).
Method	Back-bone	NUS-WIDE
C-F1	O-F1	mAP
Vanilla [78]	R50	37.5	39.6	43.9
VSGraph [78]	R50	38.6	40.2	44.8
MCPL [95]	R101	22.5	17.2	47.4
Vanilla (ours)	R50	37.8	42.4	38.3
CAPro (ours)	R50	39.3	45.4	48.0
Table 3: Results on open-set recognition.
Method	WebVision C-F1	ImageNet C-F1
Vanilla [78]	50.5	46.4
CoTeach [20, 78]	51.0	47.7
VSGraph [78]	57.2	52.8
Vanilla (ours)	54.6	48.3
CAPro (ours)	62.2	57.8
Table 2 reports per-Class F1 (C-F1), Overall F1 (O-F1), and mean Average Precision (mAP) on NUS-WIDE. Most prior multi-label methods are developed for ground-truth labels, while we are concerned with noisy WSL settings. Under this circumstance, CAPro is compared with methods that are trained on NUS-WIDE (Web) and evaluated on clean testing set. Following [96, 78], the top three categories of an image by prediction confidence are chosen for metric calculation. CAPro reaches the SOTA with a significant increase of  $1.5\%$  (C-F1),  $3.0\%$  (O-F1), and  $9.7\%$  (mAP) over our vanilla.
4.3 Discussion on Open-Set Recognition
To verify if CAPro can identify outliers of unknown categories, we conduct experiments on open-set recognition. Specifically, we train CAPro on Google500 and validate on the testing sets of WebVision1k and ImageNet1k. Images from the remaining 500 classes all belong to one open-set
category. We follow [97, 98, 78] to classify an image as open-set if its highest prediction confidence is below a threshold. The average C-F1 is adopted to reflect whether a model can discriminate between base and novel classes (501 in total). Table 3 confirms our superiority over existing methods, showing that CAPro is capable of detecting semantic novelty. More analysis can be found in Sec. E.
Table 4: Ablation study on text encoding, enhancement, and reference provider.
Text Encoding	Text Enhancement	Reference Provider	Google500	ImageNet500	NUS-WIDE
Top1	Top5	Top1	Top5	C-F1	O-F1	mAP
×	×	×	71.5	87.8	66.5	84.6	37.2	42.4	46.2
MiniLM	VSGraph [78]	×	72.0	88.0	66.9	85.4	39.2	44.4	46.8
MiniLM	✓ (ours)	×	75.5	91.0	71.5	88.8	39.3	44.9	47.4
XLNet	VSGraph [78]	×	71.6	87.8	66.8	84.8	38.6	43.4	47.6
XLNet	✓ (ours)	×	75.4	91.0	71.5	88.8	39.3	45.1	47.5
GPT-Neo	VSGraph [78]	×	72.0	88.0	67.2	85.3	39.2	45.0	47.4
GPT-Neo	✓ (ours)	×	75.7	91.1	71.6	88.8	39.2	45.1	47.6
MiniLM	✓ (ours)	Mix-up (MU) [99]	75.7	90.9	71.4	88.6	38.7	45.3	47.2
MiniLM	✓ (ours)	Bootstrap [33]	75.5	90.8	71.3	88.4	38.1	43.2	46.0
MiniLM	✓ (ours)	Label smooth [100]	75.4	90.8	71.2	88.4	36.9	42.1	46.8
MiniLM	✓ (ours)	SCC [22]	73.8	89.9	70.2	88.0	35.6	41.3	45.0
MiniLM	✓ (ours)	NCR [58]	75.5	91.1	71.5	88.8	37.6	43.4	46.8
MiniLM	✓ (ours)	✓ CB (ours)	76.0	91.3	72.0	89.2	39.3	45.4	48.0
MiniLM	✓ (ours)	✓ CB (ours) + MU	76.5	91.1	71.9	88.8	40.4	46.7	49.9
GPT-Neo	✓ (ours)	✓ CB (ours)	76.1	91.4	72.1	89.4	39.3	44.9	47.7
GPT-Neo	✓ (ours)	✓ CB (ours) + MU	76.5	91.2	72.0	88.8	40.7	45.2	50.0
Text Encoding and Enhancement Table 4 reveals the benefit from cross-modality alignment. For the method without text encoding and enhancement, we sample  $K$  examples randomly from each category. These instances barely provide reliable prototypes with semantic correctness. With respect to the text encoder, we additionally validate XLNet (base) [29] and GPT-Neo (1.3B) [31]. MiniLM surpasses XLNet by a minor margin, but both exhibit similar performance with our enhancement. GPT-Neo displays its power even with the plain  $k$ -NN-based smoothing in VSGraph [78], implying that advanced a Large Language Model (LLM) would boost CAPro. Note that encoders in CLIP [76] are not applicable here to avoid visual data leakage. All methods with text encoding and enhancement outperforms the vanilla one, validating the guidance from textual knowledge. Besides, we notice an increase up to  $3.8\% / 4.7\%$  on Google500/ImageNet500 by our text enhancement, showing that proper noise suppression is indispensable. Fig. 3 presents a comparison on the top- $K$  matched instances. Enhancement in VSGraph can filter out OOD to a certain degree, but can do nothing with lexical confusion (e.g., food in Drumstick and players in Tiger cat). More qualitative results are in Sec. D.1.
Reference Provider Our collective bootstrapping is compared against commonly used regularization methods which provide reference on targets. Surprisingly, most existing methods bring about no or even negative impact. In one respect, bootstrapping and label smoothing are already implicitly inherited in our framework as Eqs. (6) and (7). Therefore, no further gains can be seized. As for SCC, its estimated confidence may not comply with our Eq. (6), which leads to incompatible optimization. NCR has two drawbacks: 1) The chance of similar instances appear in one mini-batch is fairly small for large web datasets. 2) It only counts on self-prediction as reference source which is fragile to noise. In contrast, CAPro is enlightened by MoCo to maintain the dictionary as a queue, which enlarges the number of reference objects beyond instances in a mini-batch. Our enriched sources from both self-prediction and instance-prototype similarity expedite a steady learning progress. Moreover, mix-up improves CAPro in top1 but lowers top5. It adopts convex combinations for both inputs and targets, enforcing a stronger regularization than our CB where we only recombine targets. For WebVision1k, examples with noisy labels still resemble prototypes and therefore neighbor knowledge brings useful reference. Mix-up does not consider appearance similarity and causes over-regularization.
$\lambda^{\mathrm{bts}}$  and top- $K$  Fig. 4 shows that an increasing  $\lambda^{\mathrm{bts}}$  triggers off worse results on Google500 and ImageNet500. This suggests that collective knowledge should not overwhelm individual instance decision. With regard to top- $K$  on prototype initialization, there exists a trade-off between domain-variety and class-purity. The rise of  $K$  increases diversity but also the risk of noise.
More ablation studies on the threshold  $\gamma$ , the update frequency of visual prototypes, and the noise removal policy can be found in Secs. D.2, D.3, and D.4, respectively. Empirical guidelines on hyper-parameter tuning are concluded in Sec. F. We also provide analysis of failure cases in Sec. G. Our computation cost with respect to performance gains can be found in Sec. H.
Tiger cat (domestic cat)
(b)
Figure 3: Top-matched WebVision1k instances are chosen: (a) without text enhancement, (b) with text enhancement in VSGraph [78], and (c) with our text enhancement.
Figure 4: Impact of hyper-parameters  $\lambda^{\mathrm{bts}}$  and top-  $K$  on CAPro.
CAPro utilizes web datasets to learn visual representations that are aligned with correct semantics. Cross-modality alignment and collective bootstrapping are corroborated as two keys to improve WSL. The benefits of building prototypes are two-fold: 1) Noisy web data whose visual and textual descriptions can be efficiently removed by simply measuring the distance between an instance and its prototype in the embedding space. 2) The inter-class relationship can be statistically studied by comparing each instance to all class prototypes, which may shed light on visual similarity for species. Three potential drawbacks should be considered: 1) the limited intra-class diversity with less tolerance to the minority in one category. Images crawled from websites follow the long-tailed distribution, which means that the more common or typical one instance is, the greater the likelihood that it gets exposed online. Over-emphasis on the purity of class prototypes leads to false negatives on the recognition of atypical samples. One possible solution is to introduce randomness into the initialization and update of prototypes to improve generalization. 2) the noteworthy domain gap and bias of web data. Even image contents are correct, their styles (e.g., advertising photos, artworks, and rendering) are different from the realistic datasets. When it comes to modalities such as infrared or medical tomography images, there exist very few images online. Therefore, it is encouraged to prepare realistic images for guided-training and evaluation, where early-stopping and normalization techniques can be used to avoid overfitting. 3) the accuracy of prior knowledge about class hierarchy. Since definitions of class prototypes rely on the systematic understanding of concepts, improper, coarse-grained, or even wrong descriptions would devalue the semantic alignment. A thorough analysis on class concepts is a requisite to developing prototypes. Future work includes extension to other modalities (e.g., audio and video clips) and to other tasks (e.g., semi-supervised learning).
Broader Impact CAPro manoeuvres language models for visual concept learning, where LLMs (e.g., GPT-Neo) can deliver promising results for future development. Regardless of data sources, we showcase a cost-efficient and practical way to utilize cross-modality data. It also promotes rethinking the key-value matching mechanism for creative usages. For example, the visual dictionary originally built for instance-wise contrastive learning is re-discovered for our collective bootstrapping.
The authors would like to express gratitude to the anonymous reviewers who greatly help improve the manuscript. In addition, the authors thank Hao Cheng from the University of California, Santa Cruz for the valuable comments.
[1] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 248–255. IEEE, 2009.
[2] Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta. Revisiting unreasonable effectiveness of data in deep learning era. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 843-852, 2017.
[3] Wen Li, Limin Wang, Wei Li, Eirikur Agustsson, and Luc Van Gool. Webvision database: Visual learning and understanding from web data. arXiv preprint arXiv:1708.02862, 2017.
[4] Zeren Sun, Yazhou Yao, Xiu-Shen Wei, Yongshun Zhang, Fumin Shen, Jianxin Wu, Jian Zhang, and Heng Tao Shen. Webly supervised fine-grained recognition: Benchmark datasets and an approach. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10602-10611, 2021.
[5] Jonathan Krause, Benjamin Sapp, Andrew Howard, Howard Zhou, Alexander Toshev, Tom Duerig, James Philbin, and Li Fei-Fei. The unreasonable effectiveness of noisy data for fine-grained recognition. In European Conference on Computer Vision, pages 301–320. Springer, 2016.
[6] Parneet Kaur, Karan Sikka, and Ajay Divakaran. Combining weakly and webly supervised learning for classifying food images. arXiv preprint arXiv:1712.08730, 2017.
[7] Chuanyi Zhang, Yazhou Yao, Huafeng Liu, Guo-Sen Xie, Xiangbo Shu, Tianfei Zhou, Zheng Zhang, Fumin Shen, and Zhenmin Tang. Web-supervised network with softly update-drop training for fine-grained visual classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 12781-12788, 2020.
[8] Yi Tu, Li Niu, Junjie Chen, Dawei Cheng, and Liqing Zhang. Learning from web data with self-organizing memory module. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12846–12855, 2020.
[9] Huafeng Liu, Haofeng Zhang, Jianfeng Lu, and Zhenmin Tang. Exploiting web images for fine-grained visual recognition via dynamic loss correction and global sample selection. IEEE Transactions on Multimedia, 24:1105-1115, 2021.
[10] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64 (3):107-115, 2021.
[11] Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens Van Der Maaten. Exploring the limits of weakly supervised pretraining. In European Conference on Computer Vision, pages 181–196, 2018.
[12] Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Large scale learning of general visual representations for transfer. arXiv preprint arXiv:1912.11370, 2(8), 2019.
[13] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=Sy8gdB9xx.