# Logit Averaging: Capturing Global Relation for Session-Based Recommendation
## 2 Proposed Method
### Logit Averaging  
We propose a simple module that averages logits from a prediction layer for the better recommendation, so the other parts of a recommender system, such as an item encoder, a session encoder and a prediction layer, can be any models. In other words, our module can easily be plug-and-play for any other existing neural network-based session recommendation models. As shown in Figure 1, our proposed technique \"Logit Averaging (LA)\" is located between the session encoder and the final prediction layer. Most neural network-based recommendation models are structured as the white box in Figure 1. Sessions in a batch go into the model as inputs and an item encoder learns item embeddings. Session embeddings are generated from a session encoder using item  
Figure 1: General deep-learning-based session-based recommendation models framework with the proposed methods. embeddings. The final prediction is commonly calculated by the cosine similarity between all unique item embeddings and session embeddings of the batch. Logit averaging is a module that performs as augmentation, which can inject global context at the same time. As mentioned above, recent deep learning-based session recommendation models recommend items in respect of a single session. It is not enough to make better embeddings when solely using intra-session transition and not considering inter-sessions. Sessions have item consequences where several sessions have common items. This means that the sessions that share the same items have similar item transitions and can help to supplement the information with each other. Then, it is easier to predict items than use a single session alone. GCE-GNN [22] tried to solve the session recommendation task with a global transition using a global item graph. The graph merges all the items in the entire sessions into a single global graph. This shadows the hidden global contexts in sessions and cannot distinguish the predominant or minor trends. To overcome the problem, we propose a method to average logits of sessions that share the same target items instead of constructing a large graph. In detail, logits of each sample are obtained by calculating cosine similarity between session embeddings and item embeddings, and the logits are replaced by the mean value of logits that have the same target items. The Logit Averaging and replacing are only performed by sessions that have short-head items as target items. As shown in Figure 2, all the four public session datasets have long-tail item distribution, meaning that the majority of sessions share a few numbers of target items [34]. We assume that sessions with the same target items will have shared a global context. The short-head items take up seventy-five percent of sessions, so it can inject global relations into the majority of sessions without adopting all items. In other words, averaging logits of short-head items easily supplements the global relations information and increases the recommendation performance. It is not enough to utilize averaged logits only for making better predictions because averaging can ignore the original item transition characteristics of sessions. Therefore, we  
Figure 2: Long-tail distribution of real-world session datasets. From the left, the datasets are Diginetica, Yoochoose, Retailrocket, Tmall. The red part represents short-head and the blue part represents long-tail. adopt two loss functions, one is for the original logits and the other for the averaged logits. We get the final loss by the sum of two losses with \(\lambda\). \[L_{org}(y,Z^{org})=-\sum_{i=1}^{m}y_{i}\log(Z^{org}_{i}) \tag{1}\]  
\[L_{org}(y,Z^{LA})=-\sum_{i=1}^{m}y_{i}\log(Z^{LA}_{i}) \tag{2}\]  
\[L=L^{org}+\lambda\cdot L^{LA}. \tag{3}\]  
The detailed process is shown in Algorithm 1.