## 3 The ASLM Architecture
### Model Description
#### 3.3.2 Attention-Based Short-Term Intent Layer
Finally, the GRU produces the output vector, namely the target hidden states vector \(\left[\mathbf{h}_{1}\ \mathbf{h}_{2}\ \ldots\ \mathbf{h}_{j}\right]\) where \(\mathbf{h}_{j}\) is a target hidden state regarding the current target item prediction \(y_{j}\). The target hidden states vector, together with the context vector \(\mathbf{c}_{j}\) stated below, is used for the calculation of the target item prediction. _Architecture of Attention Mechanism_. The main part of the attention mechanism is illustrated in Figure 3. The context vector \(\mathbf{c}_{j}\) is used to capture relevant source-side information to help provide the current target item prediction \(y_{j}\). Given the target hidden state \(\mathbf{h}_{j}\) and the context vector \(\mathbf{c}_{j}\) calculated in Equation (2), we use a concatenation to integrate both vectors to produce an attentional hidden state:  
\[\mathbf{a}_{j}=f\big{(}\mathbf{c}_{j},\mathbf{h}_{j}\big{)}=tanh\big{(} \mathbf{W}_{c}\big{[}\mathbf{c}_{j};\mathbf{h}_{j}\big{]}\big{)}, \tag{1}\]where  
\[\mathbf{c}_{j}=\sum_{i}\mathbf{\alpha}_{ji}\overline{\mathbf{h}}_{i}, \tag{2}\]  
where \(\mathbf{W}_{c}\) is the model parameter, and \(\mathbf{c}_{j}\) is computed by the sum of the product of each attentive weight \(\mathbf{\alpha}_{ji}\) and source hidden state \(\overline{\mathbf{h}}_{i}\). A dense layer is used to fully connect the neurons in the encoder and the decoder. \(\mathbf{\alpha}_{ji}\) is the amount of attention that the output prediction of \(\mathbf{h}_{j}\) should pay to \(\overline{\mathbf{h}}_{i}\), which can be calculated by the SoftMax function:  
\[\mathbf{\alpha}_{ji}=\frac{exp(\mathbf{\alpha}_{ji}^{{}^{\prime}})}{\sum_{i}exp(\mathbf{ \alpha}_{ji}^{{}^{\prime}})}, \tag{3}\]  
where \(\mathbf{\alpha}_{ji}^{{}^{\prime}}\) is used to compare the target hidden state \(\mathbf{h}_{j}\) with each of the source hidden state \(\overline{\mathbf{h}}_{i}\):  
\[\mathbf{\alpha}_{ji}^{{}^{\prime}}=\mathbf{h}_{j}^{\top}\mathbf{W}\overline{\mathbf{h }}_{i} \tag{4}\]  
Denote \(s_{t}^{u}\) as the embedded vector representation of the session \(S_{t}^{u}\). After the calculation of the attentional vector \(\mathbf{a}_{j}\), it is fed through another SoftMax layer to produce the predictive distribution. Given the predictions \(y_{<j}=\left\{y_{1},y_{2},\dots,y_{j-1}\right\}\), \(s_{t}^{u}\) and the weight \(\mathbf{W}_{1}\) of \(\mathbf{a}_{j}\), the predictive distribution of items is formulated as:  
\[p\left(y_{j}\big{|}y_{<j},s_{t}^{u}\right)=softmax\left(\mathbf{W}_{1}\mathbf{a}_{ j}\right) \tag{5}\]  
We apply two methods to generate the session representations \(s_{t}^{u}\) similar to that in [7]. One is to generate \(s_{t}^{u}\) with the average of the embedded vector representations of the items in the session, as illustrated in Part _Method (a):__ASLM-AP_ of Figure 2. The other is to use the last hidden state of attention-based short-term preference layer as the session representation, illustrated in Part _Method (b):__ASLM-LHS_ of Figure 2. On one hand, although there is more useful information for the Attention-based Short-term Intent Layer to learn from the final hidden state, it is more a representation of the end of the session, rather than the entire session. On the other hand, since the average of the embeddings is irrelevant to the sequence order of the items, it can represent the entire session.