# Long Length Document Classification by Local Convolutional Feature Aggregation
## 3 Model
### Model_3: CNN Feature with Recurrent Attention Model  
In the previous two models, we use the random sub-sampling technique to extract parts of the text. As a result, the blocks of words that the two models selected are purely random, which do not represent the most salient parts of the document. In view of this, we consider introducing a smart sampling agent that can quickly find the best positions for those blocks. Inspired by the seminal recurrent attention model (RAM) [9], we expect to employ reinforcement learning to control the locations of sampled blocks, so that we could select more suitable blocks of words that better represent the characteristics of the document.  
The diagram of the model CNN_RAM_Agg is shown in Figure 4. This document classification framework incorporates the recurrent attention mechanism, which includes a glimpse network module \(f_{g}(\cdot|\theta_{g})\) for local CNN feature extraction, a recurrent neural network module \(f_{h}(\cdot|\theta_{h})\) for CNN feature aggregation, and an attention network \(f_{a}(\cdot|\theta_{a})\) for sampling location predication. The algorithm of this model is shown in Algorithm 1.  
```
1:Input: Document D, GloVe dictionary, initial network parameter \([\theta_{a},\theta_{h},\theta_{g}]\) (\(\theta_{a}\) for attention network, \(\theta_{h}\) for recurrent network, \(\theta_{g}\) for glimpse network), number of glimpse T, maximum iteration maxIter.
2:for i = 0, 1, 2,..., maxIter do
3:for t = 0, 1, 2,...  Tdo (\(l_{0}\) is initialized by a random location) Extract words near the position \(l_{t}\) and use GloVe to obtain the word vectors Use the glimpse network \(f_{g}(\cdot|\theta_{g})\) to extract feature vectors \(o_{t}\) Input \(o_{t}\) to the LSTM network \(f_{h}(\cdot|\theta_{h})\) to obtain a new hidden state \(h_{t}\) Predict the next location \(l_{t+1}\) by the attention network \(f_{a}(\cdot|\theta_{a})\) with \(h_{t}\)
4:endfor
5: Using the aggregated feature \(h_{T+1}\) emitted from the last step of LSTM to obtain the predicted label.
6: If predicted label is correct then get reward 1 otherwise get none reward.
7: Update \(f_{a}(\cdot|\theta_{a})\) parameters \(\theta_{a}\) with reinforcement learning; update \(f_{h}(\cdot|\theta_{h})\) parameters \(\theta_{h}\) and \(f_{g}(\cdot|\theta_{g})\) parameter \(\theta_{g}\) with back propagation
8:endfor
```  
**Algorithm 1**CNN_RAM_Agg  
Figure 3: Model CNN_LSTM_AGG architecture.  
Following the mechanism of RAM [9], at the \(t_{th}\) glimpse step, the location \(l_{t}\) is predicted by the attention network \(f_{a}(\cdot|\theta_{a})\) of previous step. Then, a block of words is extracted near this location and the local CNN features \(o_{t}\) are extracted by the glimpse network \(f_{g}(\cdot|\theta_{g})\). Next, \(o_{t}\) is aggregated with the previous hidden state \(h_{t-1}\) to obtain a new hidden state \(h_{t}\) by the LSTM network \(f_{h}(\cdot|\theta_{h})\). Then, the attention network \(f_{a}(\cdot|\theta_{a})\) predicts the next location \(l_{t+1}\) for the next glimpse step. After the model glimpses \(T\) locations, the last state of LSTM \(h_{T+1}\) is used for document classification. Note that we use a reinforcement learning technique to train the attention network \(f_{a}(\cdot|\theta_{a})\) because the predicted location \(l_{t}\) is not differentiable with respect to \(\theta_{a}\). Thus, if the predicted label is correct, our model receives reward \(r=1\), otherwise \(r=0\). Then, the attention network \(f_{a}(\cdot|\theta_{a})\) is optimized by maximizing the expected cumulative return, as Equation (5):  
\[J(\theta)=\mathrm{E}_{p(s_{1:T}|\theta)}[\sum_{t=1}^{T}r_{t}]=\mathrm{E}_{p(s_ {1:T}|\theta)}[R] \tag{5}\]  
where \(p(s_{1:T}|\theta)\) is the optimized policy, which predicts the next position according to its glimpse at each step; \(r_{t}\) is the reward for each step; and \(R\) is the cumulative return, which is defined as follows:  
\[R=\sum_{t=1}^{T}\gamma^{t-1}r_{t} \tag{6}\]  
where \(\gamma\) is the decaying coefficient.