SlowGuess/NAACL_2021_OCR huggingface dataset:
Tree Structure	Rest14	Laptop14	Twitter
Dep.	0.509	0.500	0.509
Left-chain	1.000	1.000	1.000
Right-chain	1.000	1.000	1.000
BERT	0.710	0.690	0.741
RoBERTa	0.702	0.705	0.722
FT-BERT	0.606	0.519	0.666
FT-RoBERTa	0.506	0.480	0.485
Table 3: Proportion of neighboring connections of different trees in all datasets. We use the short name of induced trees here as well as Table 4 and Table 5.
To further investigate the reasons for the difference between trees, we propose a set of quantitative metrics, presented in Table 3 and Table 4.
The Proportion of Neighboring Connections is to calculate the proportion of neighboring connections in the sentence, shown in Table 3. A neighboring connection links the word to its left/right neighbor word. From Table 3, we observe that on average over  $70\%$  relations in BERT/RoBERTa Induced Tree are neighboring connections. This will damage the performance of models using topological structures of trees. Thus, PTMs induced trees usually perform worse than "Dep.", with a slight
4The Left/Right-chain are exactly the same input files after the data preprocessing in these three models.
(a) The parser-provided Tree
(b) The RoBERTa Induced Tree
(c) The FT-RoBERTa Induced Tree
Figure 2: Visualization of different trees. The colored box refers to the aspect terms. Since ROOT has no directional relation arcs, we omit the ROOT notation here. For the same two sentences, trees from dependency parser, RoBERTa and fine-tuned RoBERTa are displayed. As Figure 2b shows, trees induced from RoBERTa tend to have more neighboring connections. As the bottom two figures show, trees induced from fine-tuned RoBERTa tend to have connections between sentiment words and others words.
improvement over left/right-chains.
In comparison with RoBERTa Induced Tree, a significant decline of the proportion is shown in FT-RoBERTa Induced Tree in Table 3. We see the same tendency in BERT Induced Tree and FT-BERT Induced Tree. This marks the consistent structure change in the fine-tuning process, indicating the transition to a more diverse structure. As shown in Figure 2b, RoBERTa Induced Tree has a clear pattern to depend on words in their neighbor side. Yet FT-RoBERTa Induced Tree in Figure 2c shows a more diverse dependency pattern.
Aspects-sentiment Distance is the average distance between aspect and sentiment words. We pre-define a sentiment words set  $C$ . For a sentence  $S_{i}$  in datasets  $S$ , the set of aspects words in  $S_{i}$  is termed as  $w$ .  $S_{i} \cap C$  is the set of sentiment words appearing both in the sentence  $S_{i}$  and the sentiment words set  $C$ . The Aspects-sentiment Distance(AsD) is calculated as follows:
$$
A s D \left(S _ {i}\right) = \frac {\sum_ {w} ^ {w _ {i}} \sum_ {C ^ {\prime} = S _ {i} \cap C} ^ {C _ {i} ^ {\prime}} d i s t \left(C _ {i} ^ {\prime} , w _ {i}\right)}{| w | | C ^ {\prime} |} \tag {4}
$$
$$
A s D = \frac {\sum_ {S} ^ {S _ {i}} A s D \left(S _ {i}\right)}{| S |} \tag {5}
$$
where  $|\cdot|$  is the number of elements in the set and  $\text{dist}(x_i, x_j)$  represents the relative distance be
tween  $x_{i}$  and  $x_{j}$  in the tree. Specifically,  $C$  contains sentiment words counted on Amazon-2 from Tian et al. (2020), which can be found in the Appendix. As for the Rest14 and Laptop14, Xu et al. (2020) provides the paired sentiment words with its corresponding aspect. We also calculate the paired Aspects-sentiment Distance(pAsD) on these two datasets, which only counts the distance between aspect and its corresponding sentiment words.
Tree Structure	Rest14	Laptop14	Twitter
Dep.	4.46 / 3.19	3.77 / 3.13	4.26
Left-chain	7.49 / 6.06	6.48 / 5.97	7.90
Right-chain	7.49 / 6.06	6.48 / 5.97	7.90
BERT	5.85 / 4.20	5.06 / 4.19	5.87
RoBERTa	5.05 / 3.61	4.49 / 3.67	5.39
FT-BERT	3.85 / 3.58	3.65 / 3.22	5.06
FT-RoBERTa	3.56 / 2.92	3.35 / 2.88	3.55
Table 4: The Aspects-sentiment Distance of different trees in all datasets. The less result indicates shorter distance between aspects and sentiment words. The values of Rest14 and Laptop14 are formed like “pAsD / AsD”.
We present the Aspects-sentiment Distance (AsD) of different trees in English datasets in Table 4. Results show that FT-RoBERTa has the least AsD value, indicating the shortest aspects-sentiment distance. Compared to PTMs induced trees, the trees from FT-PTMs have less AsD, indicating shortened aspects-sentiment distance. This shows that the FT-PTMs induced
Embedding	Model	Tree Structure	Rest14	Laptop14	Twitter
Acc.	F1	Acc.	F1	Acc.	F1
Static Embedding	BiLSTM†	-	77.59	67.05	70.06	64.46	71.39	69.45
LSTM+SynATT#	Dep.	80.45	71.26	72.57	69.13	-	-
AdaRNN#	Dep.	-	-	-	-	66.30	65.90
TD-GAT#	Dep.	80.35	76.13	74.13	72.01	72.68	71.15
BERT	MLP	-	85.35	78.38	78.36	74.16	75.92	74.41
DGEDT#	Dep.	86.30	80.0	79.80	75.60	77.90	75.40
RGAT#	Dep.	86.60	81.35	78.21	74.07	76.15	74.88
RACL#	-	-	81.61	-	73.91	-	81.61
RoBERTa	MLP	-	87.37	80.96	83.78	80.73	77.17	76.20
RoBERTa-ASC#	Dep.	82.82	75.12	74.12	70.52	-	-
LCFS-ASC-CDW#	Dep.	86.71	80.31	80.52	77.13	-	-
ASGCN	Dep.	86.90	80.75	81.66	78.31	75.28	74.38
FT-RoBERTa	86.87	80.59	83.33	80.32	76.10	75.07
PWCN	Dep.	87.41	81.07	84.16	81.18	76.63	75.60
FT-RoBERTa	87.35	80.85	84.01	81.08	77.02	75.52
RGAT	Dep.	87.43	80.61	83.43	80.28	74.42	72.93
FT-RoBERTa	87.52	81.29	83.33	79.95	75.81	74.91
Table 5: The results(%) of SOTA ALSC models on English datasets. The results with “†” are retrieved from Sun et al. (2019b), and those with “‡” are retrieved from the original papers. Those without additional symbols are on our own. We highlight the best results on bold.
trees are more sentiment-word-oriented, which partially reveals that the fine-tuning in ALSC encourages the aspects to find sentiment words. However, for the "Dep.", we notice that some Twitter results in Table 2 can not be fully explained by these two proposed metrics. We conjecture that the grammar casualness features the Twitter corpus, which makes the parser hard to provide an accurate dependency parsing tree. Still, these two metrics can be suitable for the induced trees.
Taken together, as the conclusion to Q2, these analyses demonstrate that the fine-tuning on ALSC could adapt the induced tree implicitly. On the one hand, less proportion of neighboring connections after fine-tuning indicates the increase of long range connections. On the other hand, less Aspects-sentiment Distance after fine-tuning illustrates the shorter distance between aspects and sentiment words, which helps to model connections between aspects and sentiment words. Thus, as shown in Section 5.1, fine-tuning RoBERTa in ALSC not only makes induced tree better suit the ALSC task but also outperform the dependency tree when combined with different tree-based ALSC models.
5.3 Comparison between ALSC models
Additional, we explore how well the fine-tuned RoBERTa model could achieve in the ALSC task. We select a set of top high-performing models of ALSC as state-of-the-art alternatives. The compari
son results are shown in Table 5.
Comparing with all these SOTA alternatives, surprisingly, the RoBERTa with an MLP layer achieve SOTA or near SOTA performance. Especially, compared to other datasets, we notice that significant improvement is obtained on the Laptop14 dataset. We assume that the pre-training corpus of RoBERTa may be more friendly to the laptop domain since the RoBERTa-MLP already obtains much better results than the BERT-MLP on Laptop14. For these BERT-based models in the second row of Table 5, similar experiments using RoBERTa are conducted. However, limited improvements have been made over the RoBERTa-MLP. We expect that induced trees from models specifically pre-trained for ALSC (Tian et al., 2020) may provide more information, which is left for the future works.
The FT-RoBERTa Induced Tree could be beneficial to Glove based ALSC models. However, incorporating trees over the RoBERTa brings no significant improvement, even the decline can be seen in some cases. This may be caused by failure to reconcile the implicitly entailed tree with external tree. We argue that incorporating trees over the RoBERTa in currently widely-used tree methods may be the loss outweighs the gain. Additionally, in the review of previous ALSC works, we notice that very few works employ the RoBERTa as the base model. We would attribute this to the
difficulty of optimizing the RoBERTa-based ALSC models. As the higher architecture, which is usually randomly initialized, needs a bigger learning rate compared to the RoBERTa. The inappropriate hyperparameters may be the cause reason for the lagging performance of previous RoBERTa-based ALSC works (Phan and Ogunbona, 2020).
In this paper, we analyze several tree structures for the ALSC task including parser-provided dependency tree and PTMs-induced tree. Specifically, we induce trees using the Perturbed Masking method from the original PTMs and ALSC fine-tuned PTMs respectively, and then compare the different tree structures on three typical tree-based ALSC models on six datasets across four languages. Experiments reveal that fine-tuning on ALSC task forces PTMs to implicitly learn more sentiment-word-oriented trees, which can bring benefits to Glove based ALSC models. Benefited from its better implicit syntactic information, the fine-tuned RoBERTa with an MLP is enough to obtain SOTA or near SOTA results for ALSC task. Our work can lead to several promising directions, such as PTMs-suitable tree-based models and better tree-inducing methods from PTMs.
We would like to thank the anonymous reviewers for their helpful comments. This work was supported by the National Key Research and Development Program of China (No. 2020AAA0106700) and National Natural Science Foundation of China (No. 62022027).
Danqi Chen and Christopher Manning. 2014. A fast and accurate dependency parser using neural networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 740-750, Doha, Qatar. Association for Computational Linguistics.
Muhao Chen, Yingtao Tian, Haochen Chen, Kai-Wei Chang, Steven Skiena, and Carlo Zaniolo. 2019. Learning to represent bilingual dictionaries. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 152-162, Hong Kong, China. Association for Computational Linguistics.
Peng Chen, Zhongqian Sun, Lidong Bing, and Wei Yang. 2017. Recurrent attention network on mem
ory for aspect sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 452-461, Copenhagen, Denmark. Association for Computational Linguistics.
Y. J. Chu and T. H. Liu. 1965. On the shortest arborescence of a directed graph. Science Sinica, 14.
Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What does BERT look at? an analysis of BERT's attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276-286, Florence, Italy. Association for Computational Linguistics.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettle-moyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440-8451, Online. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive recursive neural network for target-dependent Twitter sentiment classification. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 49-54, Baltimore, Maryland. Association for Computational Linguistics.
Timothy Dozat and Christopher D. Manning. 2017. Deep bioaffine attention for neural dependency parsing. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
Jack Edmonds. 1967. Optimum branchings. Journal of Research of the national Bureau of Standards B, 71(4):233-240.
Jason M. Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics.
Ruidan He, Wee Sun Lee, Hwee Tou Ng, and Daniel Dahlmeier. 2018. Effective attention modeling for
aspect-level sentiment classification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1121-1131, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2733-2743, Hong Kong, China. Association for Computational Linguistics.
John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4129-4138, Minneapolis, Minnesota. Association for Computational Linguistics.
Binxuan Huang and Kathleen Carley. 2019. Syntax-aware aspect level sentiment classification with graph attention networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5469-5477, Hong Kong, China. Association for Computational Linguistics.
Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent Twitter sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 151-160, Portland, Oregon, USA. Association for Computational Linguistics.
Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif Mohammad. 2014. NRC-Canada-2014: Detecting aspects and sentiment in customer reviews. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 437-442, Dublin, Ireland. Association for Computational Linguistics.
Xin Li, Lidong Bing, Wai Lam, and Bei Shi. 2018. Transformation networks for target-oriented sentiment classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 946-956, Melbourne, Australia. Association for Computational Linguistics.
Jiangming Liu and Yue Zhang. 2017. Attention modeling for targeted sentiment. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 572-577, Valencia, Spain. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis,
Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692.
Dehong Ma, Sujian Li, Xiaodong Zhang, and Houfeng Wang. 2017. Interactive attention networks for aspect-level sentiment classification. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017, pages 4068-4074. ijcai.org.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.
Minh Hieu Phan and Philip O. Ogunbona. 2020. Modelling context and syntactical features for aspect-based sentiment analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3211-3220, Online. Association for Computational Linguistics.
Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, Mohammad Al-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clercq, Véronique Hoste, Marianna Apidianaki, Xavier Tannier, Natalia V. Loukachevitch, Evgeniy V. Kotelnikov, Núria Bel, Salud María Jiménez Zafra, and Gülsen Eryigit. 2016. Semeval-2016 task 5: Aspect based sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2016, San Diego, CA, USA, June 16-17, 2016, pages 19-30. The Association for Computer Linguistics.
Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 27-35, Dublin, Ireland. Association for Computational Linguistics.
Xipeng Qiu, TianXiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. 2020. Pre-trained models for natural language processing: A survey. SCIENCE CHINA Technological Sciences.
Sebastian Ruder, Parsa Ghaffari, and John G. Breslin. 2016. A hierarchical model of reviews for aspect-based sentiment analysis. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 999-1005, Austin, Texas. Association for Computational Linguistics.
Chi Sun, Luyao Huang, and Xipeng Qiu. 2019a. Utilizing bert for aspect-based sentiment analysis via constructing auxiliary sentence. In Proceedings of the 2019 Conference of the North American Chapter of
the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 380-385.
Kai Sun, Richong Zhang, Samuel Mensah, Yongyi Mao, and Xudong Liu. 2019b. Aspect-level sentiment analysis via convolution over dependency tree. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5679-5688, Hong Kong, China. Association for Computational Linguistics.
Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2016a. Effective LSTMs for target-dependent sentiment classification. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3298-3307, Osaka, Japan. The COLING 2016 Organizing Committee.
Duyu Tang, Bing Qin, and Ting Liu. 2016b. Aspect level sentiment classification with deep memory network. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 214-224, Austin, Texas. Association for Computational Linguistics.
Hao Tang, Donghong Ji, Chenliang Li, and Qiji Zhou. 2020. Dependency graph enhanced dual-transformer structure for aspect-based sentiment classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6578-6588, Online. Association for Computational Linguistics.
Jialong Tang, Ziyao Lu, Jinsong Su, Yubin Ge, Linfeng Song, Le Sun, and Jiebo Luo. 2019. Progressive self-supervised attention learning for aspect-level sentiment analysis. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 557-566, Florence, Italy. Association for Computational Linguistics.
Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. 2018. Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 5956-5963. AAAI Press.
Hao Tian, Can Gao, Xinyan Xiao, Hao Liu, Bolei He, Hua Wu, Haifeng Wang, and Feng Wu. 2020. SKEP: Sentiment knowledge enhanced pre-training for sentiment analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4067-4076, Online. Association for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz
Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998-6008.
Duy-Tin Vo and Yue Zhang. 2015. Target-dependent twitter sentiment classification with rich automatic features. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25-31, 2015, pages 1347-1353. AAAI Press.
Kai Wang, Weizhou Shen, Yunyi Yang, Xiaojun Quan, and Rui Wang. 2020. Relational graph attention network for aspect-based sentiment analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3229-3238, Online. Association for Computational Linguistics.
Shuai Wang, Sahisnu Mazumder, Bing Liu, Mianwei Zhou, and Yi Chang. 2018. Target-sensitive memory networks for aspect sentiment classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 957-967, Melbourne, Australia. Association for Computational Linguistics.
Yequan Wang, Minlie Huang, Xiaoyan Zhu, and Li Zhao. 2016. Attention-based LSTM for aspect-level sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 606-615, Austin, Texas. Association for Computational Linguistics.
Zhiyong Wu, Yun Chen, Ben Kao, and Qun Liu. 2020. Perturbed masking: Parameter-free probing for analyzing and interpreting BERT. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4166-4176, Online. Association for Computational Linguistics.
Lu Xu, Hao Li, Wei Lu, and Lidong Bing. 2020. Position-aware tagging for aspect sentiment triplet extraction. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2339-2349, Online. Association for Computational Linguistics.
Wei Xue and Tao Li. 2018. Aspect based sentiment analysis with gated convolutional networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2514-2523, Melbourne, Australia. Association for Computational Linguistics.
Chen Zhang, Qiuchi Li, and Dawei Song. 2019a. Aspect-based sentiment classification with aspect-specific graph convolutional networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4568-4578, Hong Kong, China. Association for Computational Linguistics.
Chen Zhang, Qiuchi Li, and Dawei Song. 2019b. Syntax-aware aspect-level sentiment classification with proximity-weighted convolution network. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019, Paris, France, July 21-25, 2019, pages 1145-1148. ACM.
Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2016. Gated neural networks for targeted sentiment analysis. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA, pages 3087-3093. AAAI Press.