SlowGuess/NAACL_2021_OCR huggingface dataset:
Yaowei Zheng, Richong Zhang, Samuel Mensah, and Yongyi Mao. 2020. Replicate, walk, and stop on syntax: An effective neural network model for aspect-level sentiment classification. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 9685-9692. AAAI Press.
A Experiments on non-English Datasets
In this section, we provide details about our experiments on non-English datasets.
We conduct experiments on three non-English datasets, which are named Dutch, French, and Spanish, respectively. All of them are restaurant review datasets from SemEval-2016 task 5 (Pontiki et al., 2016), whose languages are the same as dataset names. Detailed data statistics can be found in Table 6. Following previous works, we remove samples with conflicting polarities or with "NULL" aspect terms in all datasets.
Dataset	Split	Positive	Negative	Neutral
Dutch	Train	720	386	108
Test	229	120	23
French	Train	833	683	98
Test	320	253	54
Spanish	Train	1308	443	79
Test	505	171	33
Table 6: Data statistics.
A.2 Tree Structures
We obtain five kinds of trees for every dataset. The first one is to use the off-the-shelf dependency tree parser to get parser-provided dependency trees,
written as “Dep.”. Specifically, we utilize the spaCy parser for the non-English datasets. The second method is to induce the trees from the pre-trained mBERT and XLM-R (Conneau et al., 2020) base models by the Perturbed Masking method (Wu et al., 2020), written them as “BERT Induced Tree” and “RoBERTa Induced Tree”, respectively. The third method is to use the same method as above to induce trees from the mBERT and XLM-R after fine-tuning in the corresponding datasets with the same model structure as English datasets. These two are written as “FT-BERT Induced Tree” and “FT-RoBERTa Induced Tree” to have a uniform form as the English datasets. Similarly, we add “Left-chain” and “Right-chain” as baselines. “Left-chain”, “Right-chain” mean that every word deems its previous or next word as the dependent child word.
Similar to the English datasets, Experiments incorporating tree-based ALSC models with different trees are conducted on non-English datasets, as well as the fine-tuning of PLMs. All experiments are conducted on the NVIDIA GTX1080Ti.
For experiments with tree-based models, we use the 300-dimension pre-trained embeddings (Ruder et al., 2016) for non-English datasets. We keep the word embeddings fixed to avoid overfitting. Other parameters are initialized with original models. It is worth noting that in RGAT Model reproduction, since the induced tree does not provide relation labels, we assign virtual relations for every dependency in a uniform way.
We retain the fine-tuning experiments with batch size  $b = 32$ , dropout rate  $d = 0.1$ , learning rate  $\mu = 2\mathrm{e - }4$  using the AdamW optimizer with the default settings.
As for the induced trees, We choose the trees induced from the 11th layer in all of our experiments.
A.4 Experimental Results
A.4.1 ALSC Performance with Different Trees
The comparison between models with different trees is presented in Table 7, which comprises experiments results of non-English datasets. Experimental results show that: (1) Incorporating FT-RoBERTa Induced Tree leads to the best results on all datasets, which proves the effectiveness and advantage of FT-RoBERTa Induced Tree in non
Model	Tree Features	Tree Sturcture	Dutch	French	Spanish
Acc.	F1	Acc.	F1	Acc.	F1
BiLSTM	-	-	83.30	62.50	80.0	67.50	85.30	62.10
ASGCN	Topological Structure	Dep.	84.18	70.06	79.23	65.0	87.6	67.36
BERT Induced Tree	84.45	67.25	79.23	66.31	87.10	67.58
FT-BERT Induced Tree	83.37	68.12	79.38	62.27	86.70	69.07
RoBERTa Induced Tree	84.45	70.94	79.53	67.20	86.70	68.19
FT-RoBERTa Induced Tree	84.99	68.26	80.31	67.4	87.8	72.88
PWCN	Relative Distance	Dep.	83.38	67.82	79.23	66.28	86.25	67.95
BERT Induced Tree	84.18	67.37	78.46	64.6	87.09	66.57
FT-BERT Induced Tree	84.18	68.17	78.62	66.57	86.53	67.87
RoBERTa Induced Tree	84.90	68.30	78.62	63.27	85.97	66.38
FT-RoBERTa Induced Tree	85.25	70.21	80.0	67.9	87.23	64.93
RAGT	Structure & Distance	Dep.	84.45	59.85	79.53	66.16	86.14	56.44
BERT Induced Tree	84.45	57.36	76.92	58.14	86.53	61.70
FT-BERT Induced Tree	84.18	59.67	78.61	60.79	85.50	62.66
RoBERTa Induced Tree	84.71	67.60	78.15	61.10	86.81	61.88
FT-RoBERTa Induced Tree	85.25	69.53	81.38	66.97	87.37	65.30
Table 7: The averaged performance(%) of tree-based ALSC models incorporating different tree structures on three non-English datasets. Dep. refers to the dependency tree generated by spaCy. As mentioned in English datasets, BERT Induced Tree, RoBERTa Induced Tree, FT-BERT, and FT-RoBERTa Induced Tree refer to tree structures extracted from corresponding PLMs.
Embedding	Model	Tree Structure	Dutch	French	Spanish
Acc.	F1	Acc.	F1	Acc.	F1
Static Embedding	BiLSTM	-	83.3	62.5	80.0	67.5	85.3	62.1
SA-LSTM-P#	-	87.3	-	82.4	-	88.0	-
Our ASGCN	Dep.	81.6	61.0	75.5	63.0	85.0	59.0
Our RGAT	Dep.	81.0	62.1	75.1	53.3	84.6	55.2
Our PWCN	Dep.	84.1	69.2	78.4	66.7	86.9	67.5
mBERT	MLP	-	80.37	63.43	78.06	65.04	88.21	68.03
XLM-R	MLP	-	88.36	76.29	85.95	74.72	91.48	77.96
ASGCN	Dep.	87.97	74.38	86.43	77.14	91.91	77.49
FT-RoBERTa	88.2	75.23	86.04	76.21	92.47	78.74
PWCN	Dep.	88.36	75.72	86.4	76.8	91.51	77.32
FT-RoBERTa	88.1	75.54	86.69	77.42	91.44	78.13
RGAT	Dep.	88.31	70.57	85.92	75.14	91.61	76.41
FT-RoBERTa	87.86	70.97	86.41	74.38	92.11	76.62
Table 8: The results(%) of ALSC models incorporating with different tree structures on non-English datasets. The definition of tree structures retains the same as the aforementioned. The results with “#” are retrieved from the original papers.
English datasets. Moreover, we find that the results of the FT-RoBERTa Induced Tree usually have more stable  $F_{1}$  scores. (2) Subjected to the quality of the parser of non-English languages, models using the PLMs induced trees achieve slightly better performance compared to "Dep." This illustrates that the dependency tree could be very sensitive to parser and quality of corpus. (3) Similarly, from "RoBERTa Induced Tree" and "FT-RoBERTa Induced Tree", we conclude that fine-tuning can substantially enhance the ALSC performance through trees induced from PLMs.
A.4.2 Comparison between ALSC models
Similarly, we compare the performance between the fine-tuned XLM-R and a set of top high-performing models. The results are presented in Table 8. We could see that XLM-R with an MLP is enough to achieve SOTA or near SOTA results in non-English datasets.
B Sentiment words set
positive sen-timent words	great, good, like, just, will, well, even, love, best, bet-ter, back, want, recommend, worth, easy, sound, right, ex-cellent, nice, real, fun, sure, pretty, interesting, stars
negative sen-timent word	too, little, bad, game, down, long, hard, waste, disap-ointed, problem, try, poor, less, boring, worst, trying, wrong, least, although, prob-lems, cheap
Table 9: The sentiment words used in our analysis, derived from Tian et al. (2020).
To calculate the Aspects-sentiment Distance of different tree structures on English datasets, we predefine a set of sentiment words, shown in Table 9. Specifically, we use the sentiment words described in Tian et al. (2020), which are the selected 50 most frequent sentiment words counted on Amazon-2.