# DACSR: Decoupled-Aggregated End-to-End Calibrated Sequential Recommendation
## 6 Results and Analysis
### RQ1: Overall Performance
On the MI-1m dataset, the CaliRec-GC model achieves the lowest \(C_{KL}\) value among all models, including our proposed model (e.g., 0.3847 vs. 0.7262 of \(C_{KL}\)@20). While on the Tmall dataset, the CaliRec-GC model cannot provide calibrated recommendation lists. For example, the \(C_{KL}\)@20 of CaliRec-GC is 2.0459, which is close to the original SASRec model. We think this phenomenon results from two aspects. On the one hand, the number of item attributes of the Tmall dataset is much more than that of the MI-1m dataset. The MI-1m dataset contains 18 different item attributes, while the Tmall dataset has 70 attributes. On the other hand, the average length of the user behavior sequence of the Tmall dataset is less than the MI-1m dataset, as shown in Table 1. The shorter sequence and larger item attributes set lead to the lower coverage of item attributes. The CaliRec-GC model adopts an adaptive selection of the trade-off factor \(\lambda\) for calibration based on the coverage of item attributes. The greater coverage leads to the higher value of \(\lambda\). Therefore, it performs best in terms of calibration on the MI-1m dataset, and almost does not work on the Tmall dataset. The differences between the two datasets also lead to the different calibration performances of the two datasets. In general, performances of \(C_{KL}\) on the MI-1m dataset are better than the Tmall dataset. For example, the \(C_{KL}\)@20 of our DACSR model is 1.6240 on the Tmall dataset, which is much higher than the 0.7262 on the MI-1m dataset. This is similar for the original SASRec model, with 2.1103 vs. 0.8548 on the two datasets. A possible reason is that the lower coverage of item attributes mentioned above results in the higher divergence. The large amount of 0 in \(p_{\text{}}(s)\) makes it difficult in achieving calibration, especially with the concern of accuracy. **Time Consumption** In addition, we also compared the response time of our model against the post-processing-based CaliRec model and the original SASRec model. We focused on the average time required to generate the recommended list for each sequence. For the SASRec model, we reported the time consumption when the dimension of hidden states equals to 64 and 128 (namely \(SASRC_{D64}\) and \(SASRC_{D128}\)). This is because the size 64 is the setting of each sequence encoder of our DACSR model. We conducted experiments on the same device, and removed the GPU acceleration for a fair comparison. The performance is listed in Table 3. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Response Time (10\({}^{-4}\) s)** & **MI-1m** & **Tmall** \\ \hline \(SASRC_{D64}\) & 2.01 & 1.32 \\ \(SASRC_{D128}\) & 3.28 & 1.96 \\ CaliRec & 580.15 & 668.69 \\ DACSR & 4.24 & 5.76 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Average time consumption for each sequence. Compared to the original SASRec model, our model needs more computation. For example, on the MI-1m dataset, the time consumption for each sequence of the \(SASRC_{D64}\) model is 2.01 \(\times\) 10\({}^{-4}\) s, which is approximately half of our DACSR model. This is because it incorporates two SASRec encoders and an extraction net, which is more complex than the single SASRec model. In contrast, our model can provide more accurate and calibrated recommendations than the original SASRec model. The \(SASRC_{D128}\) requires more time than \(SASRC_{D64}\) because it contains a larger scale of parameters. Compared to the CaliRec model, our model costs much less time to generate recommendation lists. For a single sequence, our model only needs 4.24 and 5.76 \(\times\) 10\({}^{-4}\) s on the MI-1m and Tmall datasets, respectively. In contrast, the CaliRec requires approximately 0.06 s for a sequence, which needs 200 times more time than our model. This is because the CaliRec model needs an extra ranking stage. The original SASRec model provided scores of all items, and selected the top-100 items with the highest scores.