# Self-Interaction Attention Mechanism Based Text Representation for Document Classification
## 5 Results and Analysis
### Impact of the Number of Sentences  
To answer **RQ 2**, we manually group the documents according to the number of sentences, e.g., \((0,5]\), \((5,10]\), \((10,15]\), \((15,20]\), \((20,25]\) and \((25,30]\) (the truncation number of the sentence is 30), and then examine the performance of our proposals as well as the baselines on groups of documents with various sentence numbers. We plot the results in Figure 5a,b on Yelp 2016 and Amazon Reviews (Electronics), respectively.  
Clearly, for both datasets, we find that the performance of all discussed models declines monotonously as the sentence number increases. The higher the sentence number, the more complex the relation between sentences in the document, making it more difficult to get a good document representation.  
On the Yelp 2016 dataset, in particular, when the number of sentences increases from \((0,5]\) to \((25,30]\), the accuracy of all discussed models presents a significant drop. For instance, regarding the baseline methods, TextRNN and TextHAN decreases by around 20% and 6% in terms of accuracy, respectively. Regarding our proposals with self-interaction attention mechanism, generally, the TextSAM models present a relatively stable decrease in terms of accuracy when the number of sentences increases. For instance, the drop rate of TextSAM\({}_{AVE}\) achieves at most 5% when performing on documents with sentence number from \((0,5]\) to \((25,30]\). In addition, our proposals consistently outperform the baseline models when the number of sentences exceeds \((15,20]\). On the Amazon Reviews (Electronics) dataset, similar results can be found. In general, the baseline models present a stable decline as the number of sentences increases. However, our proposals show an obvious drop in terms of accuracy before the number of sentences reaches \((15,20]\). After that, different from the  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Model** & **Yelp 2016** & **Amazon Reviews (Electronics)** \\ \hline TextRNN [11] & 0.4532 & 0.5211 \\ TextHAN [14] & 0.5537 & 0.5435 \\ TextSAM\({}_{AVE}\) & 0.5543 & 0.5632 \\ TextSAM\({}_{MAX}\) & **0.5919** & **0.6293** \\ TextSAM\({}_{ATT}\) & 0.5602 & 0.5726 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Accuracy on the document classification task in the k-fold cross-validation. The results of the best baseline and the best performer in each column are underlined and boldfaced, respectively.  
stable descending trend on Yelp 2016, the performance of TextSAM models consistently jumps until the number of sentences arrives at \((25,30]\).  
From the above observation, we would like to conclude that, compared with the baseline models, our proposals can obviously alleviate the impact brought about by the increase of sentence number on the performance of document classification. Since the baseline models are typically based on the LSTM (Long Short-Term Memory) architecture, which suffers from the problem of gradient vanishing [29] and make a descending performance as the number of sentences increases. Instead, our proposals can tackle such a problem by introducing the interaction between source elements and the context into the hierarchical architecture, which leads us to retain the overall semantics of text and help improve the performance of document classification.