## 7 Our Approach--Hierarchical Weighted Attention Network
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Reuters** & **Ohsumed** & **20 Newsgroups** & **BBC** & **BBCSport** \\ \hline BoW & 27.16 & 20.42 & 68.50 & 5.79 & 1.20 \\ TFIDF & 0.95 & 0.95 & 2.31 & 0.54 & 0.11 \\ \hline LSA & 25.51 & 45.82 & 38.70 & 8.48 & 1.88 \\ LDA & 363.29 & 179.01 & 947.30 & 193.08 & 69.32 \\ P-SIF & 363.14 & 356.04 & 777.95 & 164.63 & 71.26 \\ Doc2 Vec-DM & 225.34 & 247.14 & 311.22 & 42.67 & 13.67 \\ Doc2 Vec-DBoW & 183.42 & 190.18 & 241.81 & 34.31 & 11.94 \\ HAN & 3611.17 & 2816.97 & 593.19 & 771.30 & 254.61 \\ Longformer base & 64,935.80 & 74,314.70 & 105,622.34 & 14,396.79 & 4919.38 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Average total training and inference time in seconds. the attention score for the whole document. Our study examined three different operations: concatenation (denoted in Tables 3 and 4 as _concat_), addition (_add_) and multiplication (_mul_) of word attention and statistical features. The models are named as follows: the first part of the name specifies the type of statistical feature and the second part specifies the type of information appending operation. Assuming that the statistical feature is BoW and the operation is _add_, we get the name BoW + add. The model parameters are the same as for the baseline HAN model. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Reuters** & **Ohsumed** & **20 Newsgroups** & **BBC** & **BBCSport** \\ \hline HAN & 81.33, \(\sigma=0.0046\) & 54.39, \(\sigma=0.0029\) & **89.57**, \(\sigma=0.0066\) & **99.24**, \(\sigma=0.0046\) & **100**, \(\sigma=0.0\) \\ \hline TFIDF + add & **82.85**, \(\sigma=0.0022\) & **55.64**, \(\sigma=0.0027\) & 88.68, \(\sigma=0.0030\) & **99.69**, \(\sigma=0.0011\) & **100**, \(\sigma=0.0\) \\ TFIDF + mul & 72.76, \(\sigma=0.0051\) & 34.03, \(\sigma=0.0313\) & 70.51, \(\sigma=0.0168\) & 98.88, \(\sigma=0.0045\) & 69.51, \(\sigma=0.1957\) \\ TFIDF + concat & **82.44**, \(\sigma=0.0011\) & 53.33, \(\sigma=0.0044\) & 85.40, \(\sigma=0.0086\) & 99.64, \(\sigma=0.0027\) & 99.59, \(\sigma=0.0054\) \\ \hline TF + add & **82.81**, \(\sigma=0.0014\) & **55.39**, \(\sigma=0.0032\) & 88.68, \(\sigma=0.0038\) & **99.60**, \(\sigma=0.0026\) & 99.73, \(\sigma=0.0054\) \\ TF + mul & 73.50, \(\sigma=0.0057\) & 37.61, \(\sigma=0.0145\) & 72.41, \(\sigma=0.0088\) & 99.01, \(\sigma=0.0066\) & 87.29, \(\sigma=0.1613\) \\ TF + concat & **82.44**, \(\sigma=0.0017\) & 53.00, \(\sigma=0.0042\) & 85.86, \(\sigma=0.0051\) & **99.55**, \(\sigma=0.0038\) & **100**, \(\sigma=0.0\) \\ \hline BOW + add & **82.88**, \(\sigma=0.0012\) & **55.10**, \(\sigma=0.0038\) & 88.01, \(\sigma=0.0131\) & 98.92, \(\sigma=0.0048\) & 98.51, \(\sigma=0.0079\) \\ BOW + mul & **81.97**, \(\sigma=0.0011\) & 52.20, \(\sigma=0.0234\) & 71.51, \(\sigma=0.0079\) & **99.51**, \(\sigma=0.0030\) & **100**, \(\sigma=0.0\) \\ BOW + concat & **82.46**, \(\sigma=0.0017\) & 51.18, \(\sigma=0.0081\) & 75.08, \(\sigma=0.0101\) & 99.19, \(\sigma=0.0048\) & 99.32, \(\sigma=0.0043\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Averaged overall classification accuracy (in percent) for HAN and variants of HWAN; cross-validation for Ohsumed, BBC and BBCSport, test-train split for Reuters and 20 Newsgroups. Restricting input length at 30 sentences 100 words each. Numbers in bold denotes results better than baseline HAN model. Figure 18: The architecture of our Hierarchical Weighted Attention Network.