# Efficient Tree Policy with Attention-Based State Representation for Interactive Recommendation
## 4 Results and Discussion
### Influence of the Attention-Based State Representation (RQ3)  
We also conduct experiments to show the influence of the attention-based state representation in ATRec comparing with four state representation models:  
* Caser [51]: A popular CNN-based model for sequential recommendation by embedding the sequence with multiple convolutional filters to capture the user dynamics.
* ATEM [26]: An attention-based model for sequential recommendation. Compared to the proposed state representation model, ATEM ignores the user feature and feedback.
* TPGR's state representation model [5]: An RNN-based state representation model that encodes the state with the final output of RNN.
* DRR-att [4]: An attention-based state representation model that uses an attention mechanism and average pool to obtain the user's feature. Compared to DRR-att, our method introduces the user's feedback and preprocessed the user and item feature with a fully connected layer.  
To make a fair comparison, for each state representation model, we use the same tree policy network as ATRec. The only difference between the baseline methods and ATRec is the state representation model. The method is completely trained online. We report the hit rate@30 of the five state representation models under five different episode lengths: 8, 16, 32, 64, and 128. The experimental result is shown in Figure 5.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{2}{c}{**Average Decision Time per Item**} & \multicolumn{3}{c}{**Average Decision Time for Top-K Items**} \\ \cline{2-6}  & **Instant Video** & **Musical Instruments** & **Baby** & **Instant Video** & **Musical Instruments** & **Baby** \\ \hline TPGR & 0.00073 & 0.00072 & 0.00071 & 6.03045 & 12.66505 & 5.91112 \\ ATRec & 0.00939 & 0.01135 & 0.000814 & 0.00939 & 0.01135 & 0.000814 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Efficiency evaluation for decision making (seconds).  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Method** & **Instant Video** & **Musical Instruments** & **Baby** \\ \hline DDPG-KNN (\(k=1\)) & 0.56430 & 0.23432 & 1.42474 \\ DDPG-KNN (\(k=0.1N\)) & 0.88891 & 1.15585 & 1.72415 \\ DQN-R & 0.64808 & 5.64286 & 1.75619 \\ TPGR & 0.07419 & 0.07971 & 0.06205 \\ ATRec & **0.05439** & **0.06201** & **0.04382** \\ \hline Improv. & 36.4\% & 28.5\% & 41.6\% \\ \hline \hline \end{tabular}
\end{table}
Table 3: Learning efficiency comparison (seconds).  
For the two sequential recommendation models Caser and ATEM, poor performances were obtained with the IRS settings. The proposed ATRec clearly outperforms all the baseline methods in the three benchmark datasets. Compared to DRR-att, the preprocessing of the input user and item features improves the effectiveness of the state representation model. In addition, with the increase in episode length, our method can obtain better performances in Instant Video and Musical Instruments datasets. The performance of DRR-att falls as the length of the episode grows, especially in the Instant Video dataset.  
We also notice that the ATRec method with TPGR's state representation outperforms the original TPGR, which implies that the complete tree policy with a parameter-shared online learning framework is more effective than the offline one.