# Plugging Small Models in Large Language Models for POI Recommendation in Smart Tourism
## 5 Experiments
### Results and Analysis  
#### 5.3.1 Overall Comparison  
Table 2 presents the experimental results of our model compared to mainstream classic models. Table 3 shows the results of the comparison between our model and LLM-based models. The results show that PSLM4 ST outperforms the baselines in most cases, validating our model's advantages. It is worth noting that, from the variations in suboptimal models across different datasets, recent research competition in the field of POI recommendation has been quite intense. For instance, on the NYC dataset, MTNet and ROTAN--both of which focus on temporal awareness--achieved suboptimal results on different metrics. This indicates that users in the NYC dataset exhibit more regular temporal patterns, and the optimal-performing PSLM4 ST is capable of effectively capturing the time slot preferences. On the TKY dataset, STHGCN, based on hypergraph Transformers, achieved suboptimal results, while MTNet and ROTAN performed relatively worse. This might be due to the fact that users in the TKY dataset had relatively richer check-in trajectories, and both STHGCN and PSLM4 ST demonstrated strong capabilities in capturing global collaborative signals. Although ROTAN also considers point-of-interest transition relationships, it only incorporates them as pre-trained embeddings, and subsequent extensive learning through sequence models does not lead to significant final performance. On the final CA dataset, PSLM4 ST and ROTAN engaged in intense competition. Similar to the situation in the NYC dataset, PSLM4 ST performed relatively better in the Acc@5 and Acc@10 metrics. This suggests that PSLM4 ST possesses strong generalization capabilities when expanding the prediction scope, enabling it to cover correct results more comprehensively. Compared to LLM-based methods, the related work only adopts Acc@1 evaluation, and we maintain consistency. GenUP achieves higher accuracy by utilizing supervised fine-tuning and constructing user profiles, outperforming LLM-ZS's basic ICL approach. Due to the limited context length of LLMs, the actual POI visited by the user may not be present in their context. This means that the true POI may not have been included in the candidate set determinable by the LLM (e.g., through random sampling or by selecting the preceding segment of historical check-in data). Our plugin model can provide a more accurate \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Methods**} & \multicolumn{4}{c}{**NYC**} & \multicolumn{4}{c}{**TKY**} & \multicolumn{4}{c}{**CA**} \\ \cline{2-13}  & **Acc@1** & **Acc@5** & **Acc@10** & **MRR** & **Acc@1** & **Acc@5** & **Acc@10** & **MRR** & **Acc@1** & **Acc@5** & **Acc@10** & **MRR** \\ \hline UTopRec & 0.1654 & 0.3350 & 0.3588 & 0.2464 & 0.1490 & 0.3269 & 0.3590 & 0.2314 & 0.1311 & 0.2591 & 0.2983 & 0.1938 \\ FPMC & 0.1003 & 0.2126 & 0.2970 & 0.1701 & 0.0814 & 0.2045 & 0.2746 & 0.1344 & 0.0383 & 0.0702 & 0.1159 & 0.0911 \\ STGN & 0.1716 & 0.3381 & 0.4122 & 0.2598 & 0.1689 & 0.3391 & 0.3848 & 0.2422 & 0.0982 & 0.3167 & 0.4064 & 0.2040 \\ GETNext & 0.2435 & 0.5089 & 0.6143 & 0.3621 & 0.2254 & 0.4417 & 0.5287 & 0.3262 & 0.1357 & 0.2852 & 0.3590 & 0.2103 \\ STHGCN & 0.2734 & 0.5361 & 0.6244 & 0.3915 & 0.2950 & 0.5207 & 0.5980 & 0.3986 & 0.1730 & 0.3529 & 0.4191 & 0.2558 \\ MTNet & 0.2620 & 0.5381 & 0.6321 & 0.3855 & 0.2575 & 0.4977 & 0.5848 & 0.3659 & 0.1453 & 0.3419 & 0.4163 & 0.2367 \\ ROTAN & 0.3106 & 0.5281 & 0.6131 & 0.4104 & 0.2458 & 0.4626 & 0.5392 & 0.3475 & **0.2199** & 0.3718 & 0.4334 & **0.2931** \\ \hline PSLM4 ST & **0.3388** & **0.5894** & **0.6787** & **0.4464** & **0.3059** & **0.5596** & **0.6493** & **0.4172** & 0.1948 & **0.3794** & **0.4581** & 0.2855 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance comparison against baselines on three datasets.