# Plugging Small Models in Large Language Models for POI Recommendation in Smart Tourism
## 4 Methodology
### Next POI Recommendation  
#### 4.4.1 Model Inference and Optimization  
We apply the activation function \(softmax(\cdot)\) to estimate the output of the preference vectors for each module. Each module can output its own predictions. Then, the plugin model will select the top-N locations with the highest probability as the preliminary prediction. For our independent plugin model, we use the hyperparameter \(\alpha\) to balance users' habit preferences and novelty preferences, thereby reflecting their needs for POIs (categories or regions) in each time slot. Subsequently, we integrate the personal location transfer preference and the global location transfer preference. These two types of preferences are derived from the internal logic of location transfer as reflected in personal habits and global regularities. We utilize the hyperparameter \(\beta\) to regulate the balance between them.  
\[\hat{y}_{hn}=\alpha\cdot softmax(v_{hab})+(1-\alpha)\cdot softmax(v_{nov}), \tag{8}\]  
\[\hat{y}_{tf}=\beta\cdot softmax(v_{ptf})+(1-\beta)\cdot softmax(v_{gtf}), \tag{9}\]  
where both \(\alpha\) and \(\beta\) are hyperparameters ranging from 0 to 1. Then, we adopt the same method to combine the above preferences. \(\hat{y}\) denotes the prediction probabilities in all locations by adding \(\hat{y}_{hn}\) and \(\hat{y}_{tf}\). And \(\hat{y}_{hn}\) represents the prediction results modeled through a copy-generation mechanism for user behavior patterns, whereas \(\hat{y}_{tf}\) denotes the prediction results based on location transfer modeling.  
The plugin model serves as a temporal knowledge graph reasoning model for the next POI (category or region) recommendation. When given the user's historical check-in trajectory \(T_{u}\) and current query \(q=(u,t,?,d)\), predicting the target POI (category or region)can be regarded as a multi-category classification task, where each category corresponds to a POI. The learning objective is to minimize the following cross-entropy loss \(\mathcal{L}_{CE}\) on all check-in records of our TKG snapshots that exist during training.  
\[\mathcal{L}_{CE}=-\sum_{d=1}^{|\mathcal{D}|}\sum_{u=1}^{|\mathcal{U}|}\sum_{t=1 }^{|\mathcal{T}|}\sum_{k=1}^{K}y(x_{k}^{(u,t,d)})\log(g_{k}^{(u,t,d)}), \tag{10}\]  
where \(y(\cdot)\) is an indicator function that is equal to 1 if \(x_{k}^{(u,t,d)}\) is the ground truth POI (category or region) of the \(k\)-th query \((u,t,?,d)\) in the snapshot \(\mathcal{G}_{d}\) and 0 otherwise.  
Next, in order to promote the model to learn robust representations, we jointly learn three prediction subtasks through a multi-task training framework. The overall loss function is as follows:  
\[\mathcal{L}_{final}=\frac{1}{2\sigma_{l}^{2}}\mathcal{L}_{CE}^{l}+\frac{1}{2 \sigma_{c}^{2}}\mathcal{L}_{CE}^{c}+\frac{1}{2\sigma_{a}^{2}}\mathcal{L}_{CE }^{a}+log\sigma_{l}\sigma_{c}\sigma_{a}, \tag{11}\]  
where \(\sigma_{l}\), \(\sigma_{c}\), and \(\sigma_{a}\) are learnable parameters, and the last term serves as a regularization term for denoising. \(\mathcal{L}_{CE}^{l}\), \(\mathcal{L}_{CE}^{c}\), and \(\mathcal{L}_{CE}^{a}\) are the cross-entropy losses for the three subtasks of POI recommendation, category recommendation, and region recommendation, respectively.