## 4 Tasks and Datasets  
We used English restaurant review datasets (Available Online: 1. [[http://alt.qcri.org/semeval2016/task5/index.php?id=data-and-tools](http://alt.qcri.org/semeval2016/task5/index.php?id=data-and-tools)]([http://alt.qcri.org/semeval2016/task5/index.php?id=data-and-tools](http://alt.qcri.org/semeval2016/task5/index.php?id=data-and-tools)); 2. [[http://alt.qcri.org/semeval2015/task12/index.php?id=data](http://alt.qcri.org/semeval2015/task12/index.php?id=data)]([http://alt.qcri.org/semeval2015/task12/index.php?id=data](http://alt.qcri.org/semeval2015/task12/index.php?id=data))  
Figure 2: Example of score function. and-tools) provided in the SemEval-2016 and SemEval-2015. Each sentence in the restaurant reviews dataset was annotated with aspect terms (e.g., \"pizza\", \"fish\", \"food\", \"restaurant\") and those aspect terms were assigned/labeled to their aspect categories (e.g., \"FOOD\", \"QUALITY\") with polarities (e.g., \"Negative\", \"Positive\"). SemEval-2016 English restaurant reviews dataset contains 2000 training and 676 test sentences. In these sentences there were 1708 training and 587 test sentences labeled with aspect categories and remaining sentences were labeled with 'outOfScope' or 'None' categories. Sentences with 'outOfScope' or 'None' categories were not used in the final evaluation of the ACD results in SemEval-2016 Task 5. Similarly, SemEval-2015 English restaurant reviews contain 1120 training and 582 test sentences with categories. Each restaurant review consists of multiple sentences annotated with their respective aspect categories. Aspect categories are the combination of Attribute and Entity (E#A) pairs as discussed before. A sample of a single customer review from the restaurant dataset is shown in Figure 3.  
Aspect categories are unevenly distributed across the training and test review sentences; therefore, the dataset is highly unbalanced. This can affect the training and prediction accuracies. The distribution of aspect categories in SemEval-2016 Task 5 is shown in Table 2. We have disregarded the repetition for the same categories of a single sentence and consider their counts as one as discussed in SemEval-2015 and SemEval-2016. *<Review rid=\"1727363\"> *<sentences> *<sentence id=\"1727363:0\"> *<text>1 actualty feel like i should keep it a secret.</text> </sentence> *<sentence id=\"1727363:1\"> *<text>  This is a wonderful place on all stand points especially value ofrmoney. *</texts> *<Opinions> *<Opinion target=\"place\" [category=\"RESTAURAN#PRICES\"] polarity=\"positive\" from=\"20\" to\"25\"/> *<Opinion target=\"place\" [category=\"RESTAURAN#GENERAL\"] polarity=\"positive\" from=\"20\" to\"25\"/> *</Opinions> *<sentence> *<text>=\"all\"1727363:2\"> *<text>=\"all\"27363:2\"> *<text>=\"all\"27364:\"> *<Opinions> *<Opinion target=\"service\" [category=\"SERVICE#GENERAL\"] polarity=\"positive\" from=\"13\" to\"20\"/> *</Opinions> *</sentence> *</sentences> *</Review> \begin{table}
\begin{tabular}{c c c} \hline \hline
**Categories** & **Training** & **Test** \\ \hline FOOD\#QUALITY & 681 & 226 \\ SERVICE\#GENERAL & 419 & 145 \\ RESTAURANT\#GENERAL & 421 & 142 \\ AMBIENCE\#GENERAL & 226 & 57 \\ FOOD\#STYLE\_OPTIONS & 128 & 48 \\ RESTAURANT\#MISCELLANEOUS & 97 & 33 \\ FOOD\#PRICES & 82 & 22 \\ RESTAURANT\#PRICES & 80 & 21 \\ DRINK\#QUALITY & 46 & 21 \\ DRINKS\#STYLE\_OPTIONS & 30 & 12 \\ LOCATION\#GENERAL & 28 & 13 \\ DRINKS\#PRICES & 20 & 3 \\ TOTAL: 12 & TOTAL: 2258 & TOTAL: 743 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Aspect categories distribution across training and test sentences. Figure 3: Annotation of sentences in a single training review instance in English restaurant reviews dataset. In SemEval-2015 and SemEval-2016, ACD systems are divided into two categories. One is constrained (C) systems and the second is unconstrained (U) systems. In constrained systems, no external training dataset is allowed during training while in unconstrained systems, it is allowed to use datasets from outside sources (e.g., Yelp, Amazon).