# Improved Graph-Based Arabic Hotel Review Summarization Using Polarity Classification
## 5 Experimental Results  
We carried out several experimental evaluations to demonstrate the proposed approaches' potential. First, summarization was performed without sentiment grouping, i.e., using the combined mixed reviews. Second, summarization was performed based on polarity grouping, which means grouping the reviews according to their polarity before feeding the TextRank algorithm. These experiments were divided further into five sub-experiments, including five and three rating grouping, five and three sentiments grouping, and grouping based on the sentiment analysis of the reviews using Majazak API [37]. Thus, the total number of experiments for the two hotels' summaries for both reviewers before and after applying the proposed evaluation method was 48. We have calculated the _BLEU_, _ROUGE_-1, _ROUGE_-2, _ROUGE_-L, precision, recall, and F1 scores for all the experiments. To demonstrate the difference between summarizing all reviews and summarizing by sentiment based on rating score, a hotel with 55 reviews was selected. The reviews were summarized by both techniques, with a summary size of 10%. The result of this experiment is shown in Table 7. Positive reviews are underlined, and negative reviews are in bold. Positive and negative reviews are shared with the combined reviews; however, that is not always the case, as the neutral review is not part of the combined summary. Thus, the two techniques produce different results, so combining the three polarity classes does not yield the combined summary, as noted by the neutral summary. This explains the difference in the evaluation scores, as shown in Tables 8-11. The example highlights other essential aspects of the summarization techniques. First, TextRank summarizes by selecting a whole sentence ending with a period, so if a sentence contains many sentences without a period, it is considered one sentence. Second, the sentiment depends on the rating score, so it may not be very accurate, as explained in Section 4.3. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Sentence 1**} & \multirow{2}{*}{**Sentence 2**} & \multicolumn{4}{c}{**Before Stemming**} & \multicolumn{4}{c}{**After Stemming**} \\ \cline{2-9}  & & \multirow{2}{*}{_BLEU_} & **ROUGE-** & **ROUGE-** & **ROUGE-** & **ROUGE-** & **ROUGE-** & **ROUGE-** \\  & & & **1** & **2** & **L** & **1** & **2** & **L** \\ \hline \multirow{4}{*}{_BLEU_} & \multirow{4}{*}{1} & F: 0.9 & F: 0.9 & F: 0.9 & & & F: 0.9 & F: 0.9 & F: 0.9 \\  & & P: 1.0 & P: 1.0 & P: 1.0 & P: 1.0 & 1 & P: 1.0 & P: 1.0 & P: 1.0 \\  & & R: 1.0 & R: 1.0 & R: 1.0 & & & R: 1.0 & R: 1.0 & R: 1.0 \\ \hline \multirow{4}{*}{_BLEU_} & \multirow{4}{*}{0} & F: 0 & F: 0 & F: 0 & & & F: 0.66 & F: 0 & F: 0.66 \\  & & P: 0 & P: 0 & P: 0 & P: 0 & 0.6 & P: 0.66 & P: 0 & P: 0.66 \\  & & R: 0 & R: 0 & R: 0 & & & R: 0.66 & R: 0 & R: 0.66 \\ \hline \multirow{4}{*}{_BLEU_} & \multirow{4}{*}{0} & F: 0 & F: 0 & F: 0 & & & F: 0.66 & F: 0 & F: 0.66 \\  & & P: 0 & P: 0 & P: 0 & P: 0 & 0.6 & P: 0.66 & P: 0 & P: 0.66 \\  & & R: 0 & R: 0 & R: 0 & & & R: 0.66 & R: 0 & R: 0.66 \\ \hline \multirow{4}{*}{_BLEU_} & \multirow{4}{*}{0.3} & F: 0.39 & F: 0 & F: 0.39 & & & F: 0.79 & F: 0.79 \\  & & P: 0.5 & P: 0 & P: 0.5 & 0.6 & P: 0.66 & P: 0 & P: 0.66 \\  & & R: 0.33 & R: 0 & R: 0.33 & & & R: 1 & R: 0 & R: 1 \\ \hline \multirow{4}{*}{_BLEU_} & \multirow{4}{*}{0.5} & F: 0.49 & F: 0 & F: 0.49 & & & F: 0.9 & F: 0.9 & F: 0.9 \\  & & P: 0.5 & P: 0 & P: 0.5 & 1 & P: 1.0 & P: 1.0 & P: 1.0 \\  & & R: 0.3 & R: 0 & R: 0.5 & & & R: 1.0 & R: 1.0 & R: 1.0 \\ \hline \multirow{4}{*}{_BLEU_} & \multirow{4}{*}{0.8} & F: 0.59 & F: 0 & F: 0.39 & & & F: 0.9 & F: 0.9 & F: 0.39 \\  & & P: 0.6 & P: 0 & P: 0.4 & 1 & P: 1.0 & P: 0 & P: 0.4 \\  & & R: 0.6 & R: 0 & R: 0.4 & & & R: 1.0 & R: 0 & R: 0.4 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Examples of _BLEU_ and _ROUGE_ scores of simple Arabic reviews illustrating the stemming effect.