SlowGuess/NAACL_2021_OCR huggingface dataset:
Yuanhe Tian, Yan Song, and Fei Xia. 2020c. Supertagging Combinatory Categorical Grammar with Attentive Graph Convolutional Networks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6037-6044.
Yuanhe Tian, Yan Song, Fei Xia, and Tong Zhang. 2020d. Improving Constituency Parsing with Span Attention. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1691-1703.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30, pages 5998-6008.
Kai Wang, Weizhou Shen, Yunyi Yang, Xiaojun Quan, and Rui Wang. 2020. Relational Graph Attention Network for Aspect-based Sentiment Analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3229-3238.
Yequan Wang, Minlie Huang, Li Zhao, et al. 2016. Attention-based LSTM for Aspect-level Sentiment Classification. In Proceedings of the 2016 conference on empirical methods in natural language processing, pages 606-615.
Hu Xu, Bing Liu, Lei Shu, and S Yu Philip. 2019. BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2324-2335.
Yan Xu, Lili Mou, Ge Li, Yunchuan Chen, Hao Peng, and Zhi Jin. 2015. Classifying Relations via Long Short Term Memory Networks Along Shortest Dependency Paths. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1785-1794.
Wei Xue and Tao Li. 2018. Aspect Based Sentiment Analysis with Gated Convolutional Networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2514-2523.
Chen Zhang, Qiuchi Li, and Dawei Song. 2019a. Aspect-based Sentiment Classification with Aspect-specific Graph Convolutional Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4560-4570.
Hongming Zhang, Yan Song, and Yangqiu Song. 2019b. Incorporating Context and External Knowledge for Pronoun Coreference Resolution. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational
Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 872-881.
A. Hyper-parameter Settings
Table 5 reports the hyper-parameters tested in training our models. We test all combinations of them for each model and use the one achieving the highest accuracy score in our final experiments.
Hyper-parameters	Values
Learning Rate	5e-6, 1e-5, 2e-5, 3e-5
Warmup Rate	0.06, 0.1
Dropout Rate	0.1
Batch Size	8, 16, 32
Max Input Length	100
Table 5: The hyper-parameters tested in tuning our models, where the best ones used in our final experiments are highlighted in boldface.
B. Model Size and Running Speed
Table 6 reports the number of trainable parameters and the inference speed (sentences per second) of the baseline models (BERT) and our best performing models (i.e., T-GCN and ALE using BERT-large encoder with sentence-aspect pair input) on all datasets. All models are performed on an Nvidia Quadro RTX 6000 GPU.
C. Mean and Deviation of the Results
In our experiments, we run models using BERT-base and BERT-large encoders with different configurations, where models using single sentence input (S) or sentence-aspect pair input (P) as well as models using normal GCN (+ GCN) or T-GCN (+ T-GCN) are tested. For each model, we train it with the best hyper-parameter setting using five different random seeds. We report the mean  $(\mu)$  and standard deviation  $(\sigma)$  of the experimental results (accuracy and F1 scores) on all datasets in Table 7.
D. Effect of T-GCN layer
In our ablation study, we run models with different configurations of type-aware graph (TG), attention (Att), and ALE, where three T-GCN layers are used. For settings without ALE (ID: 4-6 and 8), we also try different number of layers and report the results in Table 8, where similar trend is observed.
Models	LAP14	REST14	REST15	REST16	TWITTER	MAMS
Para.	Speed	Para.	Speed	Para.	Speed	Para.	Speed	Para.	Speed	Para.	Speed
BERT-Base	109.5M	37.1	109.5M	38.1	109.5M	37.3	109.5M	38.5	109.5M	38.2	109.5M	38.0
Full Model	114.8M	31.4	114.8M	30.9	114.8M	30.6	114.8M	29.6	114.8M	30.7	114.8M	30.2
BERT-Large	335.1M	20.0	335.1M	20.1	335.1M	20.5	335.1M	20.5	335.1M	19.6	335.1M	20.0
Full Model	344.6M	16.4	344.6M	17.4	344.6M	16.8	344.6M	17.1	344.6M	17.6	344.6M	17.3
Table 6: Numbers of trainable parameters (Para.) in different models and the inference speed (sentences per second) of these models on the test sets of all datasets.
Models	LAP14	REST14	REST15	REST16	TWITTER	MAMS
Acc	F1	Acc	F1	Acc	F1	Acc	F1	Acc	F1	Acc	F1
μ	σ	μ	σ	μ	σ	μ	σ	μ	σ	μ	σ	μ	σ	μ	σ	μ	σ
BERT-base (S)	77.61	0.74	72.85	0.74	82.45	0.41	72.62	0.74	81.48	0.61	64.00	0.64	89.05	0.46	69.04	1.94	73.89	0.24
+ GCN	78.95	0.37	74.71	0.62	85.39	0.36	78.72	0.64	83.65	0.75	69.66	1.93	91.37	0.63	74.99	1.14	75.05	0.71
+ T-GCN	79.33	0.68	75.29	1.00	85.86	0.30	79.19	0.61	84.44	0.75	69.61	2.34	91.61	0.53	76.63	1.03	75.48	0.75
BERT-base (P)	78.97	0.49	74.75	0.30	85.49	0.75	78.54	1.23	83.40	0.73	69.12	2.83	91.04	0.87	74.93	2.41	74.88	0.16
+ GCN	79.00	0.82	74.90	1.14	85.76	0.13	79.62	0.73	83.69	0.79	69.86	1.59	91.60	0.57	76.87	0.61	75.31	0.95
+ T-GCN	79.10	0.87	75.16	0.95	86.19	0.24	79.56	0.72	84.16	0.81	69.95	2.03	92.36	0.33	77.70	1.69	75.65	0.91
BERT-large (S)	78.11	0.30	74.00	0.67	82.35	0.87	71.75	1.77	82.03	0.51	68.04	0.39	89.30	0.90	70.24	1.39	74.47	0.87
+ GCN	80.77	0.58	77.56	0.69	86.87	0.51	81.00	0.50	84.93	0.58	69.83	1.49	92.18	0.76	76.84	1.90	75.39	0.47
+ T-GCN	81.00	0.98	77.89	0.96	87.02	0.10	81.26	0.16	85.57	0.35	69.73	1.07	92.23	0.18	76.78	1.48	75.52	0.34
BERT-large (P)	80.06	0.48	76.50	0.67	86.17	0.33	79.10	0.76	81.59	1.87	59.23	7.40	89.29	2.04	65.32	9.02	75.34	0.31
+ GCN	81.22	0.50	77.15	0.75	86.83	0.22	80.18	0.32	85.26	0.56	68.37	1.50	92.21	0.65	77.86	1.68	75.65	0.30
+ T-GCN	81.37	0.68	77.94	0.92	87.11	0.25	81.33	0.85	85.89	0.72	70.06	2.53	92.74	0.28	78.60	1.33	77.73	0.41
Table 7: The mean  $\mu$  and standard deviation  $\sigma$  of accuracy and F1 scores of all models on six benchmark datasets. "GCN" refers to the normal GCN model without using type-aware graph, attention mechanism and ALE. "S" and "P" refer to the settings that the input is a single sentence and a sentence-aspect pair, respectively.
Setting	LAP14	REST14	REST15	REST16	TWITTER	MAMS
TG	Att	ALE	Acc	F1	Acc	F1	Acc	F1	Acc	F1	Acc	F1	Acc	F1
4	✓	✓	×	81.50	78.42	87.35	82.17	85.44	72.32	92.48	78.26	76.58	75.53	83.21	82.53
5	✓	×	×	81.21	77.79	87.26	81.64	86.13	73.96	92.42	75.69	74.88	73.89	83.18	82.39
6	×	✓	×	81.32	77.75	87.21	81.43	86.19	73.78	92.43	75.87	75.17	73.91	83.33	82.56
8	×	×	×	80.88	77.63	86.84	81.07	85.85	72.11	91.89	75.18	75.04	73.99	82.26	82.13
(a) 1 T-GCN layer
Setting	LAP14	REST14	REST15	REST16	TWITTER	MAMS
TG	Att	ALE	Acc	F1	Acc	F1	Acc	F1	Acc	F1	Acc	F1	Acc	F1
4	✓	✓	×	81.34	77.84	87.31	81.57	86.19	73.99	92.48	75.64	75.00	73.96	83.18	82.47
5	✓	×	×	81.13	77.68	87.16	81.42	86.08	73.86	92.38	75.61	74.90	73.82	83.08	82.32
6	×	✓	×	81.38	77.85	87.35	81.58	86.23	73.98	92.58	75.89	75.08	73.99	83.28	82.52
8	×	×	×	80.83	77.57	86.68	81.02	85.88	72.03	91.92	75.24	75.00	73.96	82.18	82.07
(b) 2 T-GCN layers
Table 8: Experimental results of ablation study on the six datasets, with different configurations applied to our best model without attentive layer ensemble (ALE). "TG" refers to the type-aware graph; "ATT" denotes the attention mechanism in T-GCN. "√" and "×" represent if a corresponding component is used or not. (a) reports the results where only 1 T-GCN layer is used; (b) reports the results where 2 T-GCN layers are used.