SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Two proposed objectives  $\mathcal{L}_{\text{align}}$  and  $\mathcal{L}_r$  work in a complementary manner to convey the semantic relations between labeled concepts to primary model, whereas the classical empirical risk minimization helps the model to discriminate. Combining the three losses, our proposed method achieves fully exploitation of the information inside human annotations, and improves the data utilization efficiency. The final optimization objective of LSG is the weighted sum of the three losses joined by trade-off parameters  $\lambda$  and  $\mu$ . The full two-stage training process is shown in Fig. 1.
$$
\min  _ {\theta , \phi , \psi} \quad \mathcal {L} = \mathcal {L} _ {e m p} + \lambda \mathcal {L} _ {\text {a l i g n}} + \mu \mathcal {L} _ {r}. \tag {7}
$$
Remark: The GCN and correspondingly the projector is not used during inference, and the classification result is still produced by the primary model. For this reason, no extra computational cost is needed in inference time.
3.3 Extend LSG to Unlabeled Data
We further discuss a straightforward extension of the proposed LSG on unlabeled training dataset  $\{\pmb{u}\}$  that share a same label space with the labeled data. By simply assigning a hard pseudo-label to each of them, these unlabeled data can also be connected to the semantic graph and be leveraged to guide the model. We find surprisingly that the vanilla Pseuso-Labeling strategy [34] works satisfactorily with our method in semi-supervised learning. To be specific, we assign pseudo-labels to all unlabeled data according to the maximum class in model output:  $\hat{y} = \arg \max_c\delta (C\circ F(\pmb {u}))$  , and these pseudo-labels are updated after each epoch. To this end, we extend LSG to semi-supervised learning by jointly utilize labeled and unlabeled data for constructing the augmented semantic graph, and the objective now becomes
$$
\min  _ {\theta , \phi , \psi} \quad \mathcal {L} _ {s s l} = \mathcal {L} _ {e m p} (\boldsymbol {x}) + \lambda \mathcal {L} _ {a l i g n} (\boldsymbol {x}, \boldsymbol {u}) + \mu \mathcal {L} _ {r} (\boldsymbol {x}, \boldsymbol {u}). \tag {8}
$$
With these pseudo-labeled samples, our method achieves significant performance boost despite that there exists noisy pseudo-labels. We hypothesis that it is because the pseudo-labels are leveraged only for the auxiliary branch and hence do not bias the classification head, as also suggested in [5]. We believe that further improvements can be achieved via combining our method with more advanced pseudo-labeling strategies in SSL, which will be left as future works.
Datasets and Models. We conduct experiments on 7 standard datasets that are intensively studied in Transfer Learning [77, 41, 27] and Semi-supervised learning [67, 24] and cover input data ranging images, videos and audios. For image datasets, we adopt FGVC Aircraft [44] (10,000 images for 100 aircraft variants), Stanford Cars [30] (16,185 images for 196 car categories) and CUB-200-2011 [66] (11,788 images for 200 bird species) for fine-grained classification analysis and Office Home [65] (four domains, each contains roughly 4,000 images for 65 categories) to evaluate out-of-distribution performance. For video datasets, we use UCF-101 [59] (13,320 video clips in 101 categories) and HMDB51 [31] (6,766 clips form 51 actions) For audio dataset, we report the performance on AudioSet-20K [16].
Table 1: Classification accuracy (%) on three fine-grained image benchmarks under fully-supervised and semi-supervised learning settings. ResNet-50 pretrained on ImageNet-1k is adopted as backbone and SSL indicates whether the rest training data is leveraged as unlabeled samples.
Method	SSL	FGCV Aircraft	Stanford Cars	CUB-200
15%	30%	50%	100%	15%	30%	50%	100%	15%	30%	50%	100%
Fine-tuning	X	41.6	57.8	68.7	80.2	41.1	65.9	78.4	87.8	51.2	64.6	74.6	81.8
LWF [37]	X	44.1	60.6	68.7	82.4	44.9	67.0	77.6	87.5	56.7	66.8	73.4	81.5
DELTA [36]	X	43.6	59.5	69.6	81.2	43.3	67.6	79.6	88.0	53.4	66.7	76.0	82.0
BSS [7]	X	44.4	61.9	71.4	82.7	45.0	68.4	79.6	88.4	54.8	67.3	76.3	82.3
StochNorm [29]	X	44.3	60.6	70.1	81.5	44.4	68.1	79.3	87.9	54.8	66.8	75.8	82.2
Co-Tuning [77]	X	45.9	61.6	72.7	83.9	49.0	70.6	81.9	89.5	57.6	70.1	77.3	82.5
LSG	X	55.6	72.0	79.5	86.7	55.4	75.5	83.8	90.7	57.7	70.6	77.5	82.2
Pseudo-Labeling [34]	✓	46.8	62.8	73.2	-	40.9	67.0	78.7	-	45.3	62.0	72.3	-
FixMatch [58]	✓	55.5	71.6	78.3	-	49.9	77.5	84.8	-	44.1	63.5	76.0	-
SimCLRv2 [6]	✓	40.8	59.0	68.5	-	45.7	61.7	77.5	-	45.7	62.7	71.1	-
Self-Tuning [67]	✓	64.1	76.0	81.2	-	72.5	83.6	88.1	-	64.2	75.1	80.2	-
DebiasMatch [68]	✓	59.5	71.2	77.1	-	75.3	86.1	90.0	-	64.7	75.1	77.7	-
LSG (extended)	✓	71.4	85.3	87.1	88.6	79.3	87.7	90.9	91.9	66.1	76.1	80.6	82.6
Table 2: Classification accuracy (%) with self-supervised pretraining model (CLIP ViT-B).
Method	FGCV Aircraft	Stanford Cars	CUB-200
15%	30%	50%	100%	15%	30%	50%	100%	15%	30%	50%	100%
CoOp [84]	34.8	38.2	42.9	49.8	75.0	81.3	83.7	84.0	64.4	72.3	76.3	78.7
ProDA [43]	33.9	36.6	43.5	50.8	75.4	81.5	84.0	84.2	65.5	73.1	76.4	79.2
Tip-Adapter [81]	36.9	41.7	46.7	53.8	75.6	81.4	84.1	85.0	69.0	74.9	77.9	81.2
LSG	48.9	58.6	65.0	74.5	79.4	83.2	86.1	90.1	70.2	78.4	81.9	85.4
Several deep neural networks with different architecture and pretraining dataset are included in the experiments. For image tasks, we use ResNet-50 [21] and ConvNext-S [40] pretrained on ImageNet-1k [10], ViT-B pretrained on CLIP WIT [51], and Swin-B [39] pretrained on ImageNet-21k. For video and audio tasks, we adopt pretrained ViT-L [13] from VideoMAE [61] and ViT-B from Audio-MAE [22], respectively.
Implementation Details. We utilize a pretrained BERT-L [25] as the language model to transform labels to text embeddings. For each concept, we contextualize it into complete sentences using 20 handcrafted prompts. When constructing the Language Semantic Graph, the similarity threshold  $\tau$  is determined adaptively to include the top  $\rho = 0.3\%$  edges that connects between nodes of different labels of the fully connected graph. The GCN is trained on full graph for 5,000 iterations.
For LSG guided training process, the projector  $H$  is implemented by a fully-connected layer with an output dimension of 1024 and randomly initialized weights. We find that setting  $\lambda$  and  $\mu$  to 1.0 and 8.0 generally achieves satisfying results within all the experiments. In image classification tasks, we adopt SGD with a momentum of 0.9 as the optimizer. The learning rate is set as 1e-3 for the visual backbone in most experiments and a  $10\times$  larger value is applied for the classifier and projector in SSL and SDG. In video and audio tasks, we adopt the same configurations as finetuning in the official released VideoMAE and Audio-MAE codebase. Please refer to the appendix for more details.
4.1 Results of Image Experiments
To examine the effectiveness of our method, we first conduct experiments on three fine-grained image classification benchmarks FGVC-Aircraft, Stanford Cars and CUB-200-2011 following [77, 67]. Similar to previous works, we analyze the performance of LSG under labeled data partition ratio of  $15\%$ ,  $30\%$ ,  $50\%$  as well as the full training set. We adopt ImageNet-1k pretrained ResNet-50 as backbone and compare LSG with previous transfer learning baselines: vanilla fine-tuning, LWF [37], DELTA [36], BSS [7], StochNorm [29] and Co-Tuning [77]. In addition, to show that LSG is applicable to self-supervised pretraining models, we adopt CLIP pretrained ViT-B and compare LSG to vision-language fine-tuning methods CoOP [84], ProDA [43], Tip-Adapter [81]. Finally, we extend LSG to semi-supervised learning setting by including the rest training data as unlabeled data. As discussed in § 3.3, we adopt Pseudo-Labeling [34] strategy for unlabeled data (which can be regarded as the baseline for extended LSG) and compare the extended LSG with SSL methods including FixMatch [58], DebiasMatch [68], SimCLRv2 [6] and Self-Tuning [67].
Table 3: Out-of-distribution accuracy (%) for single domain generalization on Office-Home. Backbone ConvNext-S and Swin-B are pre-trained on ImageNet-1k and ImageNet-22k respectively.
Backbone	Method	Source:Ar	Source:Cl	Source:Pr	Source:Rw	Avg. (ID)	Avg. (OOD)
Ar	Cl	Pr	Rw	Cl	Ar	Pr	Rw	Pr	Ar	Cl	Rw	Rw	Ar	Cl	Pr
ConvNext-S	ERM [27]	85.0	53.4	72.7	78.6	85.0	67.5	72.9	75.4	94.7	61.8	49.0	80.0	91.4	72.2	52.7	80.9	89.0	68.1
LP-FT [32]	85.1	55.3	73.3	79.5	85.6	70.4	73.4	77.5	95.1	63.9	52.0	82.0	91.4	72.0	54.1	82.2	89.3	69.6
LSG	85.8	57.7	74.0	79.9	86.3	71.7	75.4	77.8	95.1	65.8	54.7	82.1	91.2	73.8	58.0	82.3	89.6	71.1
Swin-B	ERM [27]	91.8	70.7	86.1	88.5	89.1	80.6	84.3	86.7	96.5	77.9	66.1	88.3	95.2	82.6	69.1	90.4	93.2	81.0
LP-FT [32]	91.5	70.4	85.9	88.6	89.4	81.3	85.4	87.1	96.7	78.8	68.0	88.9	95.1	82.9	70.3	90.5	93.2	81.5
LSG	92.4	72.4	85.7	88.3	89.8	83.7	87.1	88.3	96.7	80.9	69.9	89.4	95.1	83.0	70.8	90.5	93.5	82.5
The results are shown in Table 1 and Table 2. We observe that our method outperforms its counterparts on each task and in both fully-supervised and semi-supervised scenarios. In fully supervised experiments, LSG significant outperforms other methods on FGVC-Aircraft and Stanford Cars especially when the available labeled data is more scarce. It also achieves comparable performance on CUB200. At  $15\%$  sampling ratio, LSG improves from the vanilla fine-tuning by  $14.0\%$ ,  $14.3\%$  and  $6.5\%$  on three datasets, and surpasses Co-tuning by an average accuracy of  $5.3\%$ . Similarly, LSG shows promising potential in semi-supervised setting, achieving the best performance across all of the labeling rate and all three datasets. Notably, since the extended LSG incorporates data augmentation techniques adopted in semi-supervised learning, its results improvement over LSG under  $100\%$  training data shows that the proposed method is mutually beneficial to other data-efficient strategies. When applied on self-supervised pretrained model, LSG also demonstrates consistent gains comparing to fine-tuning methods that are specifically designed for vision-language models.
We also conduct experiments on cross-domain benchmark Office-Home to evaluate the out-of-distribution performance of the LSG guided training. Following [27], we consider the challenging single domain generalization setting where models are trained on one single source domain and tested on the rest target domains without seeing any data from them. We compare LSG against the vanilla empirical risk minimization (ERM) baseline and the more advanced approach LP-FT [32].
We report the prediction accuracies of both in-distribution (ID) and out-of-distribution (OOD) in Table 3. Specifically, samples in the source domain are randomly partitioned into  $80\%$  training data and  $20\%$  test data, and the results refers to accuracy on test data. OOD results are obtained evaluating all data in each target domain. The results show that LSG improves model performance on both ID and OOD samples, demonstrating that the label semantic relations help the model learn features that are more robust to distribution shift.
4.2 Results of Video and Audio Experiments
Category labels in video and audio tasks also have their natural language descriptions, such as "push up", "kiss" in HMDB51 dataset and "singing", "groan" in AudioSet. This enables LSG to construct semantic graph and improve the model training process. We adopt VideoMAE pretrained ViT-L as backbone for small-scaled video datasets UCF-101 and HMDB51, and use AudioMAE pretrained ViT-B for audio benchmark AudioSet.
Table 4: Accuracies  $(\%)$  of video action recognition on UCF-101 and HMBD51 under label rate of  $1\%$  and  $10\%$ . Backbone: ViT-L pretrained by VideoMAE.
Method	SSL	UCF-101	HMDB51
1%	10%	1%	10%
VideoMAE ft [61]	×	26.80	73.26	11.82	46.85
LSG	×	29.07	74.49	13.28	50.50
Pseudo-Labeling [34]	✓	28.11	76.08	11.97	47.72
FixMatch [58]	✓	30.51	76.45	13.12	48.30
CMPL [73]	✓	32.24	77.82	13.58	50.81
LSG (extended)	✓	35.32	80.36	15.40	52.65
Following previous data efficient studies [24, 73] on the two video benchmarks, we train the model using  $1\%$  and  $10\%$  labeled data respectively. We also conduct SSL experiments similar to image experiments and compare LSG against SSL approaches in video. From the results in Table 4, we observe that LSG consistently improves the fine-tuning accuracy over
Table 5: Classification accuracies  $(\%)$  on AudioSet-20K with label proportion of  $10\%$ ,  $25\%$  and  $50\%$ . (ft: Fine-tuning)
Method	10%	25%	50%
Audio-MAE ft [22]	2.59	15.60	25.96
LSG	11.20	21.80	27.82
all tasks with only limited amount of labeled samples available. With unlabeled data being included, two classical SSL methods Pseudo-Labeling and FixMatch both achieve certain improvements, whereas CMPL [73], a method designed specifically for video SSL tasks, performs better. On top of that, LSG obtains superior results than these method on all of the tasks, boosting the accuracy from
Table 6: Ablation study of losses and alternative design choices on FGVC Aircraft,  $15\%$  labeled data.
Method	Lemp	Lalign	Lr	Lr w/ sg.	classifier	language head	Acc.
LSG w/o Lr, Lalign	✓	n/a	✓	41.6 (-14.0)
LSG w/o Lr	✓	✓	n/a	✓	44.2 (-11.4)
LSG w/o Lalign	✓	✓	✓	✓	50.7 (-4.9)
LSG	✓	✓	✓	✓	✓	55.6 (+0.0)
Classifier→Language Head	✓	✓	✓	✓	✓	47.9 (-7.7)
LSG w/o stop gradient	✓	✓	✓	✓	48.3 (-7.3)
Table 7: Ablation study using different alignment strategy to transfer language semantic knowledge to the primary model on four image datasets.  ${OH}$  denotes for Office-Home dataset.
Method	FGCV Aircraft	Stanford Cars	CUB-200	OH Avg. (ID)	OH Avg. (OOD)
15%	30%	50%	15%	30%	50%	15%	30%	50%
Vanilla (No Alignment)	41.6	57.8	68.7	41.1	65.9	78.4	51.2	64.6	74.6	89.0	68.1
Text Prototype Alignment	51.1	68.8	76.5	52.4	73.0	82.6	56.0	70.1	76.8	89.0	68.8
Contrastive Alignment	54.2	70.7	78.3	54.3	73.8	83.2	56.7	69.9	77.1	89.4	69.3
LSG	55.6	72.0	79.5	55.4	75.5	83.8	57.7	70.6	77.2	89.6	71.1
its baseline Pseudo-Labeling by  $4.28\%$  and  $4.93\%$ , and achieves  $80.36\%$  and  $52.65\%$  accuracies on two datasets with only  $10\%$  labeled data.
For audio experiment, we utilize the Audio-MAE ViT-B model pretrained on the full AudioSet-2M and report its finetuning results on a class-balanced AudioSet-20K subset using  $10\%$ ,  $25\%$  and  $50\%$  labeled audio clips. Our results is illustrated in Table 5 where we compare LSG with vanilla fine-tuning method. As a consequence, LSG achieves an average of  $5.56\%$  accuracy enhancement from the baseline, which demonstrates that guiding the audio representation by their label semantic information could also be beneficial. These results on video and audio experiments demonstrate that LSG is widely application to various modalities.
Ablation Study. We report the accuracy of different LSG variants on FGVC-Aircraft with  $15\%$  labeled data (Table 6). We begin with respectively removing the regularization loss  $\mathcal{L}_r$  and the alignment loss  $\mathcal{L}_{\text{align}}$  from the total objective. The performance drops of  $14.4\%$  and  $4.9\%$  prove that both objectives contribute to the semantic guidance. Moving on, we investigate two variants: replacing the trainable classifier by a fixed head from language embeddings, and removing the stop gradient in  $\mathcal{L}_r$ . Both variants perform much worse than the current design.
Table 8: Calinski-Harabasz Indexes of the original label embeddings from pretrained language model and the GCN refined label embeddings.  ${OH}$  denotes for Office-Home dataset.
Method	Aircraft	StanfordCars	CUB200	OH
Original	86.5	137.2	157.8	85.5
GCN refined	1364.1	1481.7	916.3	1395.2
Effectiveness of the GCN model. Here we conduct two experiments to verify the effectiveness of the GCN model on refining the label embeddings and promoting semantic knowledge transfer from LSG to the primary model. First, we compares the Calinski-Harabasz Index between the original and refined la
bel embeddings, where higher index indicates better clustering performance. We use the node embeddings of the penultimate layer of the GCN as the refined label embeddings. As shown in Table 8, the index of the refined embeddings is significantly higher than the original ones, which demonstrates that the GCN model learns the discriminative information and improves the label embedding quality.
Second, we compare the performance of LSG with two alternative alignment strategies: Text Prototype Alignment and Contrastive Alignment. For the former, we use the average of the label embeddings as the class prototypes and minimize the cosine similarity between features and their corresponding prototypes. For the latter, we adopt supervised contrastive loss [26] between features and label embeddings. The results are shown in Table 7. We observe that both alignment strategies improve the performance of the vanilla finetuning, which demonstrates that learning semantic relation via the alignment loss is beneficial to the model training. However, the performance of LSG is still
Figure 2: Analytical experiments: (a) and (b) Model accuracy on validation sets during training. (c) Comparison between GCN and GAT. (d) Effect of the threshold hyper-parameter  $\tau$ .
superior to the two alternatives, which indicates that the GCN model is more effective in transferring the label semantic knowledge to the primary model.
LSG speeds up model training. It is discovered that applying LSG to guide the representation learning encourages the model to achieve a satisfactory performance faster. The accuracy curves on validation set of LSG and its baseline in both video (UCF-101) and audio (AudioSet) training are depicted in Fig. 2(a)(b). The accuracy curve of LSG rises faster and higher than vanilla fine-tuning.
Substitute GCN by GAT [63]. To further investigate the effect of the graph neural network in our system, we change the implementation from GCN to Graph Attention Network (GAT), which is another wildly adopted GNN. The biggest difference is that GAT automatically learns the weight between connected nodes instead of relying on manual definition as GCN does. We compare the two variants on FGVC Aircraft SSL with  $15\%$  labeled data and discover an interesting trend. As shown in Fig. 2(c), GAT outperforms GCN in low label regime (labeled data  $\leq 15\%$ ) whereas GCN is more beneficial with more labeled samples.
Analysis of the LSG edge connection threshold  $\tau$ . The hyper-parameter  $\tau$  in Eq. (2) controls the topology (i.e., the number of edges that connect different labels) of the Language Semantic Graph. We study the effect of this threshold on both the GCN node classification accuracy and more importantly the primary model accuracy. We observe (in Fig. 2(d)) that as the ratio of cross-label edges decrease, the graph topology becomes more simple and GCN accuracy increases. However, due to lack of beneficial cross-label interaction, the primary model performs worse. In contrast, if the ratio is overwhelmingly high, the graph topology becomes too complicated and results in degradation in both GCN and primary model performance.
The proposed semantic graph is a general concept that aims to capture the high-level semantic relations between labeled concepts. In this paper, we only use the natural language descriptions of the labeled concepts and leverage the pretrained language model to construct the graph. Although using language embedding is effective in many data-efficient scenarios across various modalities, it is still limited when the labeled concepts in the task are not well described by natural language. However, the semantic relations between concepts can also be captured by other ways, such as manually determining the relationship between categories based on expert knowledge or leveraging a knowledge graph. In the future, we will explore how to combine the semantic relations from different modalities to construct a more comprehensive graph.
Meanwhile, since the proposed LSG is a simple and lightweight modification to the standard training process and does not alter the inference procedure, another promising direction would be to extend the proposed method to other data-efficient scenarios, such as semantic segmentation and object detection.
In this paper, we study how to exploit the semantic information in labels for improving data efficiency. We present LSG, a novel and effective data-efficient learning method that leverages Language Semantic Graph to guide the model learn semantic relations between labeled concepts. LSG consists of two parts: an auxiliary graph neural network that extracts knowledge from the semantic graph and two novel optimization objectives that transfer the knowledge to primary model. We demonstrate that LSG is applicable on image, video and audio models and brings significant performance gains to the model under Transfer Learning and Semi-Supervised Learning scenarios.
This paper was supported by National Key R&D Program of China (No. 2021YFB3301503), the National Natural Science Foundation of China (No. 62376026), and also sponsored by Beijing Nova Program (No. 20230484296), CCF-Tencent Rhino-Bird Open Research Fund.
[1] A. Adadi. A survey on data-efficient algorithms in big data era. Journal of Big Data, 8(1):24, 2021.
[2] Y. Balaji, S. Sankaranarayanan, and R. Chellappa. Metareg: Towards domain generalization using meta-regularization. In NeurIPS, 2018.
[3] D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver, and C. A. Raffel. Mixmatch: A holistic approach to semi-supervised learning. In NeurIPS, 2019.
[4] D. Berthelot, R. Roelofs, K. Sohn, N. Carlini, and A. Kurakin. Adamatch: A unified approach to semi-supervised learning and domain adaptation. In ICLR, 2022.
[5] B. Chen, J. Jiang, X. Wang, P. Wan, J. Wang, and M. Long. Debiased self-training for semi-supervised learning. In NeurIPS, 2022.
[6] T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and G. E. Hinton. Big self-supervised models are strong semi-supervised learners. In NeurIPS, pages 22243-22255, 2020.
[7] X. Chen, S. Wang, B. Fu, M. Long, and J. Wang. Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning. In NeurIPS, 2019.
[8] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, pages 3213-3223, 2016.
[9] E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le. Randaugment: Practical automated data augmentation with a reduced search space. In CVPR Workshops, pages 702-703, 2020.
[10] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248–255, 2009.
[11] T. Dinh, Y. Zeng, R. Zhang, Z. Lin, M. Gira, S. Rajput, J.-y. Sohn, D. Papailiopoulos, and K. Lee. Lift: Language-interfaced fine-tuning for non-language machine learning tasks. In NeurIPS, pages 11763-11784, 2022.
[12] J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. In ICML, pages 647-655, 2014.
[13] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2020.
[14] Y. Ganin and V. Lempitsky. Unsupervised domain adaptation by backpropagation. In ICML, pages 1180-1189, 2015.
[15] V. Garcia and J. Bruna. Few-shot learning with graph neural networks. In ICLR, 2018.
[16] J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter. Audio set: An ontology and human-labeled dataset for audio events. In ICASSP, pages 776-780, 2017.
[17] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl. Neural message passing for quantum chemistry. In ICML, pages 1263–1272, 2017.
[18] R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, pages 580-587, 2014.