Mobility-LLM: Learning Visiting Intentions and Travel Preference from Human Mobility Data with Large Language Models

Authors: letian  gong
yan  lin
xinyue  zhang
yiwen  lu
xuedi  han
yichen  liu
shengnan  guo
youfang  lin
huaiyu  wan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 16:58:54

Abstract: Location-based services (LBS) have accumulated extensive human mobility data on diverse behaviors through check-in sequences. These sequences offer valuable insights into users’ intentions and preferences. Yet, existing models analyzing check-in sequences fail to consider the semantics contained in these sequences, which closely reflect human visiting intentions and travel preferences, leading to an incomplete comprehension. Drawing inspiration from the exceptional semantic understanding and contextual information processing capabilities of large language models (LLMs) across various domains, we present Mobility-LLM, a novel framework that leverages LLMs to analyze check-in sequences for multiple tasks. Since LLMs cannot directly interpret check-ins, we reprogram these sequences to help LLMs comprehensively understand the semantics of human visiting intentions and travel preferences. 
Specifically, we introduce a visiting intention memory network (VIMN) to capture the visiting intentions at each record, along with a shared pool of human travel preference prompts (HTPP) to guide the LLM in understanding users’ travel preferences. These components enhance the model’s ability to extract and leverage semantic information from human mobility data effectively. Extensive experiments on four benchmark datasets and three downstream tasks demonstrate that our approach significantly outperforms existing models, underscoring the effectiveness of Mobility-LLM in advancing our understanding of human mobility data within LBS contexts.

Main paper: 
 introduction: 
 Location-based services (LBS) such as Gowalla, Weeplace, and Foursquare enable users to share and discover location information and nearby services. This results in the collection of extensive human mobility data, often presented in the form of check-in sequences. These sequences record users' visits to different points of interest (POIs) like restaurants and hospitals at various times, reflecting significant semantics about their intentions and preferences. Analyzing these check-in sequences is crucial as it offers valuable information on human mobility data, which can positively impact individuals, businesses, and urban management.The key to effectively mining check-in sequences lies in understanding their rich semantics. Existing methods primarily focus on specific tasks, such as location prediction [10, 59, 52, 26] , time prediction [40, 45] , and trajectory user linking [34, 13, 62] , rather than delving into the semantics of human behaviors. This narrow focus often results in limited optimization goals and a shallow understanding of the semantics contained in check-in sequences. Recently, large language models (LLMs) have demonstrated impressive capabilities in semantic understanding and contextual information processing, demonstrating successful adaptability across different domains. LLMs trained on extensive corpora surpass task-specific models in their potential to understand semantic information. Inspired by this, we aim to utilize pre-trained LLMs as powerful check-in sequence learners.Nevertheless, LLMs encounter a significant obstacle in their inability to directly interpret check-in sequences. As typical sequential data, check-in sequences contain a wealth of semantic information that reflects various near-term regularities and inherent characteristics. The future intention of an individual is prone to be dictated by near-term regularities that are close to recent visits, termed visiting intentions. Furthermore, an individual's inherent characteristics tend to persist over time and determine their travel preferences, which is necessary to analyze them across multiple domains for a comprehensive understanding. Hence, our main challenge is to enable LLMs to effectively extract semantics from check-in sequences and comprehensively understand human visiting intentions and travel preferences.To address this challenge, we present a novel unified framework called Mobility-LLM for various check-in sequence analysis tasks. It leverages pre-trained LLMs for general check-in sequence analysis. Our contributions can be summarized as follows:• We propose a unified framework called Mobility-LLM that uses a pre-trained LLM to achieve a SOTA or comparable performance across various check-in analysis tasks including location prediction, trajectory user link, and time prediction. We extract the semantics of check-in sequences to enable LLMs to gain a comprehensive understanding of human visiting intentions and travel preferences. • A visiting intention memory network (VIMN) is proposed for capturing users' visiting intentions of users at each check-in record by prioritizing relevant check-in records. • A shared pool of human travel preference prompts (HTPP) in different domains is introduced, which enables a comprehensive understanding of human travel preferences and matches appropriate prompts from multiple domains. • Our model's exceptional performance is validated through extensive experiments on four benchmark datasets involving three tasks. Our robust outcomes in cross-domain pre-training exhibit an average enhancement of 17.8% and an average of 23.6% to 38.3% on the few-shot scenario. 
related works: 
 In this section, we provide short reviews of literature in the areas of mobility data mining and crossdomain applications of LLMs. We postpone the detailed discussion of works to the Appendix A, due to limited space.Mobility Data Mining has emerged as a promising research area due to the proliferation of locationbased services. This has led to the development of three significant tasks that enhance service quality: next location prediction (LP), next time prediction (TP), and trajectory user link (TUL). The LP task aims to anticipate a user's future location based on their historical movement. Several notable models have emerged as leading approaches in LP [10, 59, 51, 48, 5, 23, 58] . The TUL task focuses on establishing connections between different trajectories and users, facilitating the analysis of user movement patterns and uncovering valuable insights into their behavior [13, 7, 60, 16] . The TP task focuses on estimating the time at which a user is likely to visit their next location. Various models have been developed to model the intensity function representing the rate or density of event occurrences, effectively making accurate time predictions [57, 17, 53] .Cross-domain Application of LLMs has gained attention in recent studies, which adopt large language models to address the challenge of limited training data. In the field of time series analysis, LLM4TS [4] is a pioneering method that aligns pre-trained large language models with temporal characteristics, introducing a two-level aggregation method to effectively incorporate multi-scale temporal data into pre-trained LLMs. One-Fits-All [63] is a unified framework that leverages a frozen pre-trained language model to attain state-of-the-art or comparable performance across various major types of time series analysis tasks. AutoTimes [30] facilitates the tokenization of time series into the embedding space of LLMs and intelligently utilizes the inherent token transitions to effectively predict time series in an autoregressive manner. In the field of computer vision, LM4VE [35] incorporates a frozen transformer block from an LLM as a general-purpose visual encoder layer. To tackle graph-related tasks, TAPE [19] utilizes semantic knowledge generated by LLMs to enhance the quality of initial node embeddings in GNNs. MoleculeSTM [27] aligns GNNs and LLMs within a shared vector space, integrating textual knowledge into graphs to enhance reasoning capabilities.3 Preliminaries POI Visiting Record In the check-in datasets, a user's visit to a certain place is represented by a POI visiting record R = (L p , t) generated by the user u. L p indicates the visited POI at time t. L p is represented by (L id , L lon , L lat , L category ), comprising L id as a POI index, and accurate longitude L lon and latitude L lat . L category denotes the category of the visited POI (e.g., hospital, restaurant).Check-in Sequence A user's movement during a specific period can be represented by sequential POI visiting records, which we refer to as a check-in sequence. We denote a check-in sequence asC =< R 1 , R 2 , • • • , R n >, where the POI visit records are ordered by their visited time, and n is the length of the sequence.Problem Statement Given a check-in sequence C, our objective is to encode this sequence into a meaningful representation. This representation can be used for various tasks. In this paper, we choose three typical check-in prediction tasks: 1) Identifying the user û who generated the check-in sequence (TUL task). 2) Predicting the next location ŝn+1 the user will arrive at (LP task). 3) Predicting the arrival time tn+1 at this location (TP task).Question: Does the paper discuss the limitations of the work performed by the authors?Answer:[Yes]Justification: See Section 6 Limitations.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
theory assumptions and proofs: 
 Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [NA]• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
methodology: 
 We introduce a novel unified framework, Mobility-LLM, designed to address a variety of check-in sequence tasks. The overall structure of this framework is illustrated in Fig. 1 . 1) Initially, to embed POIs by incorporating category semantics, we introduce the POI Point-wise Embedding Layer (PPEL) to generate the embedding of POIs in the current check-in record (referred to as PPE). 2) Subsequently, we feed the PPEs and timestamps of a check-in sequence into the Visiting Intention Memory Network (VIMN) to capture the visiting intentions of users at each check-in record. 3) A Human Travel Preference Prompt (HTPP) pool is introduced to extract users' preferences from check-in sequences, which act as cues to assist the LLM in comprehending users' travel preferences more effectively. 4) Finally, we use the different parts outputs of LLM (corresponding to VIMN, HTPP), each with its own projection head, to forecast the user's next location, the estimated arrival time, and the user who generates the check-in sequence.POI Point-wise Embedding Layer (PPEL) is designed to generate the semantic information embedding for each POI in a check-in record. This is especially important since POI categories contain a wealth of semantic information. It has been observed that the category descriptions of POIs in the original check-in datasets are often vague or unidentified. When we refer to "vague," we are indicating that the descriptions are too broad or contain abbreviations, making it challenging to determine the specific POI categories accurately. To address this issue, we have developed a category word pool (see Appendix I for the whole categories list) that allows each POI to match with the most appropriate categories automatically. We reindex the POI IDs and word IDs in each dataset, making it convenient for subsequent encoding of these IDs. As shown in Fig. 1a , we use the learnable embedding E L id of each POI ID as the query, the learnable embedding E C id of the category word ID as the key, and the corresponding category word token E C token from the LLM tokenizer as the value.A Point-wise attention mechanism is used to calculate the i-th PPE s i of POIs:EQUATIONwhere Que(•), Key(•), and Val(•) denote linear projections, d is the dimension of E C token , GeoHash(•, •) encodes geographic coordinates (latitude and longitude) into an embedding vec- tor (see Appendix H for details). We omit the MLP Layer and Layer Normalizations in Transformer layers to simplify the representation in Eq. 1.Visiting Intention Memory Network (VIMN) is proposed to capture the visiting intentions of users at each check-in record by prioritizing relevant check-in records. As shown in Fig. 1b , we feed the timestamp of each check-in record and the time interval between the adjacent records into the Imminent GRU layer. We adopt a dual encoding approach for time representation: 1) Periodic encoding for timestamps t as T (t) = [cos(ω 1 t), sin(ω 1 t), . . . , cos(ω k t), sin(ω k t)], where {ω k } are frequencies determined to capture periodicity across various temporal scales. 2) Logarithmic encoding for time intervals ∆t represented as ∆T = log(1 + ∆t), which adjusts the GRU's forget gate based on the time interval ∆t. This adjusted factor, denoted as ∆T , affects the forget gate of the GRU unit through G forget (∆T ) = σ(W f ∆T + b f ). The other components like G update remain unchanged from the original GRU configuration. The Imminent GRU layer can be depicted as:EQUATIONwhere z i represents the output of the Imminent GRU at time step t i , and H i-1 denotes the hidden state at time step i -1. This setup allows for filtering out less relevant, temporally distant data. Subsequently, the outputs Significant Score[z i-r+1 , • • • , z i-1 , z i ]••• ( ) = 𝒉 ! 𝒉 !"# 𝒉 $ Figure 2:The architecture of HTPP.Human Travel Preference Prompt (HTPP). This paragraph introduces a method for extracting users' travel preferences to help the LLM enable a comprehensive understanding of human travel preferences and match appropriate prompts from multiple domains. Prompting strategies have demonstrated encouraging outcomes in various applications that aid model predictions [3] . Previous works primarily focus on utilizing a fixed prompt to boost the pre-trained models' performance through fine-tuning [30] or learnable prompts lacking reality semantic meaning [41, 3] . User behavior is diverse and cannot be accurately summarized as a single fixed prompt to describe the user's travel preferences. Adopting a prompt-based approach with meaningless learnable prompt vectors can only handle sequences with simple semantics, such as time series [41, 3] . However, it lacks the capability to adequately extract human travel preferences. To address this limitation, we introduce a shared pool of travel preference prompt words across D = 3 domains (e.g., occupation, activity type, lifestyle) as illustrated in Fig. 2 . Each domain includes a selection of m = 16 prompt words. For every prompt word, we create a key-value pair where the value represents the word token obtained from the LLM tokenizer, and the key is derived from the word token through a trainable linear transformation. Specifically, these key-value pairs are defined as follows:EQUATIONwhere k D m , V D m ∈ R L E is a key-value pair, and we maintained it at the same embedding size L E as visiting intentions h i . We then employ a scoring functionϕ(h i , k D m ) = h i • k D m /∥h i ∥∥k D m ∥ to determine the significance score of each h i , where ϕ : R L E × R L E → R.The significant score of each h i indicates the relevance of the current visiting intention to each word in the travel preference prompts pool. Subsequently, we aggregate the significant scores of each h i to identify the top-K most significant pairs in a domain. By defining {m j } K j=1 as a subset of indices for the selected top-K prompt words in each domain, we obtain the output[V D m1 ; • • • ; V D m K ]of HTPP as prompts (illustrated in Fig. 1c ) to more accurately recognize and comprehensively understand user's travel preferences at the level of individual check-in sequences.Training Task In Fig. 1d , the input to the LLMs consists of the output of VIMN H= [h 1 , h 2 , • • • , h n ],HTPP, and the user embedding U i , denoted as x input :EQUATIONWe concatenate all the tokens along the check-in length dimension. As shown in Fig. 1e , α denotes the output of the LLM corresponding to VIMN (i.e., the first n output of the LLM), while the remaining outputs are denoted as β, where n is the length of this check-in sequence. Various projection heads are used to predict the next location ŝn+1 the user will arrive at and its arrival time tn+1 with the β. We use the β and α with a mean pooling projection head to predict the user û who generates this check-in sequence. 
experiments: 
 To evaluate the performance of our proposed model, we carry out extensive experiments on four real-world check-in sequence datasets, targeting three different types of downstream tasks: Next Location Prediction (LP), Trajectory User Link (TUL), and Time Prediction (TP). We use TinyLlama-1B [56] as the default backbone unless stated otherwise. The code of Mobility-LLM is released at https://github.com/LetianGong/Mobility-LLM.Baselines: For the LP task, we cover five state-of-the-art LP models to demonstrate the superiority of our model: DeepMove [10] , LightMove [22] , PLSPL [47] , HMT-LSTM [25] , LSTPM [59] .For the TUL task, we select four end-to-end models for comparison: TULER [13] , TULVAE [62] , MoveSim [11] , S2TUL [7] . For time prediction methods, we select four SOTA models for comparison: IFLTPP [40] , THP [64] , NSTPP [17] , DSTPP [53] . NSTPP and DSTPP can also be applied to the LP task. For sequence representation methods ReMVC [55] , VaSCL [54] , SML [61] , CACSR [15] , we apply them to learn the representation of the check-in sequence and serve different downstream tasks. More details are in Appendix D.In our experiments, we use four real-world datasets derived from Gowalla3 , WeePlace [28, 31] , Brightkite4 , and FourSquare [49, 50] . To ensure data consistency, we set a maximum historical time limit of 120 days and filter out users with fewer than 10 records and places visited fewer than 10 times. Appendix C provides statistical information for each processed dataset. We shuffle all the samples and then split the datasets into training, validation, and test sets in a 6:2:2 ratio based on the number of samples.Implement Details: During the training phase, we adopt a partially frozen strategy to fine-tune the pre-trained LLM. We apply different parameter freezing strategies to the 1 -L F layers and the L F -L F +U layers. In order to enhance the model's performance on trajectory data, we employ the Low-Rank Adaptation (LoRA) algorithm [21] to incorporate additional parameters into the LLMs. Details of the partially frozen strategy and other settings can be found in Appendix B. We run each set of experiments 5 times and reported their mean valuesTable 1 : Next location prediction (LP) performance results. A higher value indicates better performance. Red: the best, Blue: the second best. The units of all metics are expressed as e-2.Datasets Gowalla WeePlace Brightkite FourSquareMetric (e-2) Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Method DeepMove 10. 51 Metric (e-2) Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Method TULER 55. 85 Setups: We consider the LP task as a multi-classification problem. Given a check-in sequence C Ui from a specific user U i , we feed it to our framework G LLM to obtain the check-in sequence representation G LLM (C Ui ). As shown in Fig. 1e , a multi-class projection head f multi θ (β) = softmax (W β + b)) is used to predict the next location ŝn+1 with the corresponding output β of VIMN. We maximize the conditional log-likelihood for a given N observations as follows:L M LE (θ) = N i=1 log f multi θ (ŝ n+1 | β),where N is the number of POIs. We maintain all baseline user embeddings at 256 dimensions and POI embeddings at 128 dimensions. The evaluation metrics include Acc@k and mean reciprocal rank (MRR). The details of the implementation and metric can be found in Appendix B.2.Results: Our brief results are shown in Tab. 1, and consistently surpass all baselines. The comparison with CACSR is particularly noteworthy since it is the latest check-in sequence learning model. We note average performance gains of 17.19% and 7.49% over CACSR and ReMVC, respectively. Compared with the SOTA task-specific models, PLSPL and LSTPM realized an average MRR improvement of 9.32% and 19.88%. Relative to the time point process models, e.g., NSTPP and DSTPP, our improvements are also pronounced, exceeding 14.86% and 13.22%.Setups: Unlike the LP task, the TUL task requires predicting which user generated a given check-in sequence. Therefore, the input information cannot contain any details about the user. We use the Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Task Method DeepMove 7. 58 corresponding output β of HTPP and α with a mean pooling projection head to predict the user û who generates this check-in sequence (TUL task). Other settings and evaluation metrics are the same as those used in the LP task.Results: Mobility-LLM consistently surpasses all baselines in Tab. 2, outperforming the all baselines by an average of 47.3%. Mobility-LLM remains competitive even when compared with the SOTA model, ReMVC, by 17.17%. Our model performs exceptionally well in the TUL task thanks to its effective extraction of users' travel preferences, allowing for precise identification of users.Setups: For the time prediction task, we follow the method of IFLTPP [40] using an intensity-free method to model the interaction time as a mixture distribution. We first obtain the mixture weights w, means µ and standard deviations s from the corresponding output β of LLM with linear layer. Then we use the TPP projection head built by a mixed distribution function and sample to get the prediction time tn+1 as follows:EQUATIONwhere k represents the number of independent Gaussian distributions in the mixed distribution, a denotes the mean of the whole set and b denotes the standard deviation of the whole set. We sample from the mixture model in the parsing solution. The evaluation metrics include root mean square error (RMSE) and mean absolute error (MAE).Results: Our brief results are shown in Tab. 3. Due to the diversity of user behaviors and the unpredictability of activity timing, even the state-of-the-art (SOTA) models for Temporal Point Processes (TPP) struggle to achieve excellent results across all datasets. However, our model outperforms all baselines in most cases and does so significantly for most of them.Setups: LLMs have recently demonstrated remarkable few-shot learning capabilities [29] . This section assesses whether our reprogrammed LLM retains this ability in different tasks. In our experiments, we maintain consistent splits for training, validation, and test sets in both standard learning (where the full training set is used) and few-shot learning. For few-shot scenarios, we intentionally limit the training data percentage of sample number (i.e., using first 20%, 5%, 1% samples of the training dataset).Results: Our 5% few-shot learning results are in Tab. 4 remarkably excel over all baseline methods, and we attribute this to the successful knowledge activation in our reprogrammed LLM. Interestingly, our approach on both LP and TUL tasks consistently surpasses other competitive baselines, further underscoring the potential prowess of language models as proficient human behavior analysis machines. Concerning recent SOTA models such as NSTPP, DSTPP, S2TUL, ReMVC, and CACSR, our average enhancements surpass 21.4%, 21.7%, 86.6%, 46.2%, and 45.2% w.r.t. average on all the metrics. Even with only 5% of the training dataset, our model achieves results comparable to other models using 100% of the training dataset. This is particularly significant for privacy-protected and typically smaller Check-in datasets, as our model can effectively understand the distribution patterns of human behaviors with minimal data.Language Model Variants. We compare eight representative backbones with varying capacities (A.1-8 in Tab. 5). We find that the TinyLlama (A.1 in Tab. 5) backbone model performed the best overall for our task, while its Chat version (A.2) is relatively less suitable for reprogramming. Our results indicate that the scaling law is not strictly retained after the LLM reprogramming. Even the Pythia-70M (A.5 in Tab. 5) backbone model, which has fewer parameters, can demonstrate advantages in certain datasets and tasks. Therefore, it can be inferred that the suitability of backbone models for different tasks is significantly related to the training strategy of the backbone model, and not just to the number of parameters. For example, in time prediction tasks, GPT-2 (A.8) performs better than the 2.7B+ parameter backbone model (A.4,7).WeePlace-TUL Brightkite-TUL Gowalla-TUL MRR Acc@5 FourSquare-TUL MRR Acc@5 MRR Acc@5 MRR Acc@5 (%) (%) (%) (%) (%) (%) (%) (%) WeePlace-LP Brightkite-LP Gowalla-LP MRR Acc@5Ablation Study. As shown in Tab. 5, we find that the removal of (B.1) stands as a pivotal element in harnessing the efficiency for the TUL task. We observe a notable average performance degradation of 5.16% on the TUL task. The inherent habits of human activities can effectively reflect a user's behavioral characteristics, which strongly validates the importance of (B.1) in capturing the inherent habits of people's activities. Ablation of this component (B.2) results in over 4.09% degradation on LP task, as people's actions are largely influenced by visiting intentions. Additionally, (B.3) allows for more accurate embedding of POI types and semantics, providing a foundation for subsequent modules and tasks. In the w/o LLM setting (B.4), we replace the large language model with a standard transformer. (B.4) also shows that even without using an LLM as the base model, our reprogramming still achieves excellent results, even surpassing many baselines.Reprogramming Interpretation. We provide three case studies to visualize the improvements brought by HTPP and VIMN. In Fig. 3a , we visualize the significance of different prompt words in three domain pools within the HTPP model. Orange indicates high significance, and Purple indicates low significance. It can be observed that as training progresses, the prompt words for this sample in each domain concentrate on a few words. This also indicates that user behavior is diverse and personalized. To more intuitively see the effect of HTPP, we select 10 different users, each with several samples. We visualize the output of LLM with t-SNE [43] corresponding to the samples generated by these 10 users in Fig. 3b and Fig. 3c . It can be seen that with the addition of HTPP, the model can more comprehensively capture user behavior patterns and better recognize the behavioral characteristics of each user. Similarly, to better visualize the effect of VIMN, we select a continuous sequence of five user behavior points. Fig. 3d-f show the Pearson correlation coefficients between the five outputs h without using the VIMN, without considering Delta Time as input, and when using VIMN, respectively. Fig. 3g shows the time intervals among these five points. It can be observed that with the use of the VIMN, the closer the time, the greater the influence on the current state, which aligns with human behavior patterns. Since plans cannot keep up with changes, people's decisions are mostly influenced by recent behaviors. Therefore, our model uses Delta Time as a constraint for correlation, effectively mimicking human activity patterns and achieving better prediction results.Hyperparameter Analysis. There is a significant difference in user behavior patterns across different datasets. As shown in Fig. 4a , in larger datasets like Foursquare, user behavior is more diverse, and larger K values yield better results. In Fig. 4b , it can also be seen that the number of nearby cycles in VIMN is highly correlated with the distribution of time intervals in the dataset.Reprogramming Efficiency. Tab. 6 provides an overall efficiency analysis with and without the backbone LLM. Our proposed reprogramming network itself is lightweight in activating the LLM's ability for different tasks (i.e., fewer than 10 million trainable parameters; only around 3.4% of the total parameters in TinyLlama), and the leveraged backbones actually cap the overall efficiency. This is favorable even compared to the parameter-efficient fine-tuning methods (e.g., QLoRA [8] ) The core of these models is the modeling of check-in sequences, which leads to improved accuracy in location prediction and trajectory analysis.LP aims to anticipate a user's future location based on their historical movement. Several notable models have emerged as prominent approaches in LP. DeepMove [10] leverages RNNs and attention mechanisms to capture the spatial-temporal intentions in users' location data and predict their next destination. STAN [33] introduces a spatial-temporal attention network that incorporates spatial and temporal contexts for accurate prediction. LSTPM [59] focuses on long and short-term patterns in user trajectory using an attention-based LSTM [20] model. SERM [52] utilizes an encoder-decoder architecture with a spatial-temporal residual network to capture user preferences and predict future locations. PLSPL [47] trains two LSTM models for location-and category-based sequences to capture the user's preference. LightMove [22] designs neural ordinary differential equations to enhance robustness against sparse or incorrect inputs. HMT-GRN [25] alleviates the data sparsity problem by learning different User-Region matrices of lower sparsities in a multitask setting. Graph-Flashback [38] constructs a spatial-temporal knowledge graph to enhance the representation of POIs, having a great advantage when dealing with the historical sequence input of the same length.GETNext [51] introduces a user-agnostic global trajectory flow map as a means to leverage the abundant collaborative signals.TUL is a significant task that focuses on establishing connections between different trajectories, facilitating the analysis of user movement patterns, and uncovering valuable insights about their behavior. Notable models have been specifically developed to address the challenge of predicting trajectory links. TULER [13] takes advantage of advanced algorithms to establish links between trajectories, allowing for a comprehensive understanding of user movement patterns. TULVAE [62] uses latent variables to model the variability in trajectories, capturing hierarchical and structural semantics and improving the identification and linking performance. MoveSim [11] simulates human mobility using a generative adversarial framework that incorporates attention mechanisms to capture complex spatial-temporal transitions in human mobility. DeepTUL [34] utilizes deep learning techniques to extract representations from trajectory data and facilitate the prediction of trajectory links. S2TUL [7] utilizes graph convolutional networks and sequential neural networks to capture trajectory relationships and intra-trajectory information. GNNTUL [60] employs graph neural networks for human mobility and associates the traces with users on social networks.TP focuses on estimating the time at which a user is likely to visit their next location. To accomplish this, it is common practice to use intensity functions to represent the rate or density of event occurrences, various models have been developed to model the intensity function and make accurate time predictions effectively. Modeling the intensity function using RNNs or attention mechanisms is a common approach for predicting the occurrence of events. IFLTPP [40] approximates any distribution of inter-event times using normalizing flows and mixture distributions. RMTPP [9] utilizes RNNs to model the intensity function. SAHP [57] combines the Hawkes process with self-attention mechanisms to capture the temporal dependencies and spatial influences in event sequences. THP [64] combines the Hawkes process with transformer-based architectures to capture temporal dependencies in event sequences. NSTPP [17] utilizes neural ODEs to model discrete events in continuous time and space, enabling the learning of complex distributions in spatial and temporal domains. IMTPP [18] models the generative processes of observed and missing events and utilizes unsupervised modeling and inference methods for time prediction. DSTPP [53] proposes a novel parameterization framework that uses diffusion models to learn complex joint distributions.In recent years, many sequence representation models have become a hot topic, including those for natural language sequences and spatial-temporal trajectory sequences. For natural language sequences, SimCSE [14] is a classical sequence representation language model that utilizes a straightforward contrastive learning framework. It employs standard dropout as noise, making it easy to implement. However, it cannot effectively distinguish between hard negative samples or semantically similar sequences. Thus, VaSCL [54] leverages neighborhood information to generate virtual augmentations, improving the quality of data transformations compared to traditional methods. Inspired by models for natural language sequence representation, researchers in the spatial-temporal domain have also conducted similar studies on spatial-temporal sequences. SML [61] first employs self-supervised contrastive learning to effectively manage sparse and noisy trajectory data, enhancing trajectory representation through spatial-temporal data augmentation. However, the spatial-temporal augmentation in SML needs to be manually specified, which is relatively cumbersome. Therefore, ReMVC [55] effectively addresses this issue by adopting a cross-view contrastive approach. It uses contrastive learning to extract meaningful region representations, improving intra-view and inter-view learning. And CACSR [15] effectively adds adversarial perturbations and automated data augmentation, enhancing contrastive training processing. While it lacks interpretability and does not understand the semantic information of time and space from the actual semantics of human activities.In summary, it can be seen that whether it is end-to-end models designed for specific tasks or spatiotemporal sequence representation learning models, their understanding of human activities is not yet deep enough, and they lack a profound understanding of the essence and spatial-temporal patterns behind human activities.We have witnessed the great success of Large Language Models (LLMs) in natural language processing [37] , and some cross-domain (such as computer vision [36] , time series, and graph-related tasks.In the field of graph theory, To tackle graph-related tasks, Graph Neural Networks (GNNs) have emerged as one of the most popular choices for processing and analyzing graph data. While GNNs excel at capturing structural information, their reliance on semantically constrained embeddings as node features limits their ability to fully express the complexities of the nodes. By incorporating LLMs, GNNs can be augmented with stronger node features that effectively capture both structural and contextual aspects. TAPE utilizes semantic knowledge that is pertinent to the nodes (e.g., papers) generated by LLMs to enhance the quality of initial node embeddings in GNNs. Furthermore, InstructGLM replaces the predictor in GNNs with LLMs, harnessing the expressive capabilities of natural language through techniques like graph flattening and prompt-based instruction design. MoleculeSTM aligns GNNs and LLMs within a shared vector space, integrating textual knowledge into graphs (e.g., molecules) to enhance reasoning capabilities.In the field of Computer Vision (CV), ViT is an image classification model based on the Transformer architecture, which has achieved excellent performance on multiple benchmark image classification datasets. CLIP is a large-scale pre-trained model developed by OpenAI that aligns images and text, enabling simultaneous learning of representations for both modalities, which allows for interactive understanding between images and text. DALL-E utilizes a large-scale pre-trained language model to transform text into images through a generative approach, showcasing remarkable capabilities in image generation. TEST [41] , as a time series (TS) embedding method tailored for Large Language Models (LLMs), generates embeddings that capture similarity, instance-wise, feature-wise, and text-prototype alignment for TS tokens. Time-LLM [24] introduces a reprogramming framework.Given the challenge of limited training data, recent studies turn to adopting large language models (LLMs) to address cross-domain applications. In the field of time series analysis, LLM4TS [4] is the pioneering method that aligns pre-trained Large Language Models with temporal characteristics, introducing a two-level aggregation method to effectively incorporate multi-scale temporal data into pre-trained LLMs. One-Fits-All [63] is a unified framework that leverages a frozen pre-trained language model to attain state-of-the-art or comparable performance across various major types of time series analysis tasks. AutoTimes [30] facilitates the tokenization of time series into the embedding space of LLMs and intelligently utilizes the inherent token transitions to effectively predict time series in an autoregressive manner. In the field of computer vision, LM4VE [35] incorporates a frozen transformer block from an LLM as a general-purpose visual encoder layer.To tackle graph-related tasks, TAPE [19] utilizes semantic knowledge that is pertinent to the nodes generated by LLMs to enhance the quality of initial node embeddings in GNNs. MoleculeSTM [27] aligns GNNs and LLMs within a shared vector space, integrating textual knowledge into graphs to enhance reasoning capabilities.GPT4TS [42] utilizes PLMs by freezing the self-attention feed-forward layers. For visual encoding tasks, LM4VisualEncoding [35] incorporates a frozen transformer block from a PLM as a generalpurpose visual encoder layer. RLMRec [39] integrates the semantic space of PLMs and collaborative relational signals using an alignment framework. An LLM agent framework is proposed for flexible and efficient personal mobility generation [46] .While these studies provide valuable insights, it is important to recognize that their methods cannot be directly applied to the domain of trajectory learning. Trajectory data exhibits distinct spatialtemporal characteristics and contains unique information that requires customized approaches and considerations.The Mobility-LLM model was constructed using the PyTorch 5 . The loss function is a cross-entropy loss for the LP and TUL tasks and an MAE loss for the TP task. The performance on the validation sets determines the hyper-parameters and the best models. All experiments are performed five times, and the means and standard deviations are calculated. To make a fair comparison, for all methods, the embedding dimension d is 256, while the hidden state h has a size of 256. The learning rate is 0.001. The model is pre-trained for 100 epochs on the training sets with the early-stopping mechanism of 10 patients. All trials have been conducted on Intel Xeon E5-2620 CPUs and NVIDIA RTX A40 GPUs.Four metrics are used for evaluating the models: mean absolute error (MAE), root mean squared error (RMSE), Accuracy at K (ACC@K), and Mean Reciprocal Rank (MRR). MAE and RMSE quantify absolute errors. ACC@K measures the accuracy of the predictions within the top K ranks. MRR calculates the average of the reciprocal ranks, where a higher rank indicates a better performance.EQUATION)where m is the number of predicted values, Y i is the predicted value, Y i is the target value.EQUATION)where m is the total number of queries, rank i is the rank position of the first relevant result for the i -th query.C Check-in Datasets We can see that the dataset we selected includes the Gowalla dataset, which has a large number of samples, users, and Points of Interest (POI), as well as the Foursquare dataset, which has a small number of users but a large number of POIs. Additionally, we have the Brightkite and WeePlace datasets, which have a small number of both users and POIs. These datasets cover different scenarios and can fully validate the model's comprehensive capabilities.The frozen pre-trained transformer (FPT) has demonstrated effectiveness in a variety of downstream tasks across non-language modalities [32] . We adopt a partially frozen attention (PFA) LLM, specifically designed to enhance prediction accuracy in check-in sequences prediction.The difference between the FPT and our PFA primarily lies in the configuration of frozen attention layers. In the FPT framework, both the multi-head attention and feed-forward layers are frozen during training, as these layers contain the most significant portion of the learned knowledge within the LLM. In the PFA, we maintain the first F layers identical to the FPT, but crucially, we unfreeze the last U multi-head attention layers since the attentions offer effective handling of spatial-temporal dependencies in data. Consequently, our Mobility-LLM can adapt to check-in datasets while preserving the foundational knowledge acquired during pre-training.DeepMove [10] is an attentional recurrent network designed for predicting human mobility from lengthy and sparse trajectories. It introduces a multi-modal embedding recurrent neural network that captures complex sequential transitions by embedding multiple factors influencing human mobility. DeepMove further incorporates a historical attention model with dual mechanisms to effectively capture multi-level periodicity. This historical attention model enhances the recurrent neural network's ability to predict mobility patterns by leveraging the inherent periodic nature of human movement.LightMove [22] is a lightweight and accurate deep learning-based method developed for predicting the next locations of taxicabs in order to enhance targeted advertising on taxicab rooftop devices.The paper focuses on Motov, a leading company in South Korea's taxicab rooftop advertising market, and aims to leverage demographic information of locations to improve the preparation of targeted advertising campaigns.PLSPL [47] addresses the task of recommending the next Point of Interest (POI) for users based on their historical check-in data. The main objective is to capture both the users' general taste and their recent sequential behaviors, as these factors are crucial in making accurate recommendations. However, existing methods often assume the same dependencies for all users, disregarding the fact that different users may have varying preferences and dependencies on these two aspects.HMT-LSTM [25] addresses the challenging task of predicting the next Point-of-Interest (POI) that a user is likely to visit in personalized recommender systems. One of the main difficulties in this task is the large search space of possible POIs in the region, which leads to data sparsity issues in existing works and hampers performance.LSTPM [59] proposes a method for POI recommendation by modeling long-term and short-term preferences. It uses a nonlocal network for capturing long-term dependencies and a geo-dilated RNN for considering geographical relationships. This approach improves the reliability of recommendation results compared to existing methods.TULER [13] addresses the task of understanding human trajectory patterns in Location-Based Social Networks (LBSNs) applications. This task is crucial for various applications like personalized recommendation and preference-based route planning. Existing methods often classify trajectories or their segments into predefined categories based on spatial-temporal values and activities, such as walking or jogging. However, the paper focuses on a novel problem called Trajectory-User Linking (TUL), which aims to identify and link trajectories to the users who generated them in LBSNs.TULVAE [62] addresses the important task of Trajectory-User Linking (TUL) in Geo-tagged social media (GTSM) applications. It utilizes a neural generative architecture with stochastic latent variables that span hidden states in an RNN. By incorporating variational autoencoder techniques, TULVAE can capture the hierarchical and structural semantics of trajectories using high-dimensional latent variables. Moreover, TULVAE addresses the data sparsity challenge by leveraging large-scale unlabeled data, providing more comprehensive representations of user mobility patterns.MoveSim [11] is a framework for the realistic simulation of massive human mobility data. Existing solutions for mobility simulation have limitations in generating high-quality data due to complex transitions and intricate regularities in human mobility patterns.S2TUL [7] is a flexible Semi-Supervised framework for Trajectory-User Linking (TUL) with five main components.IFLTPP [40] is a novel approach that models event sequences with irregular intervals using temporal point processes. Unlike the standard intensity-based methods, IFLTPP directly models the conditional distribution of inter-event times. Drawing inspiration from normalizing flows, IFLTPP captures complex dependencies in the data while maintaining practicality and interpretability through a simple mixture model.THP [64] is a model designed for handling massive event sequence data with complex temporal dependencies. To address the limitations of existing recurrent neural network-based models, THP utilizes the self-attention mechanism of Transformers, which effectively captures short-term and long-term dependencies in the data.NSTPP [17] is an approach that addresses the retrieval problem of continuous-time event sequences (CTESs) using marked temporal point processes (MTPP). It improves the retrieval performance by applying a trainable unwarping function to the query sequence, making it comparable to the corpus sequences.DSTPP [53] is a framework designed for modeling spatial-temporal point processes (STPPs) by leveraging diffusion models. It addresses the limitations of existing solutions that assume conditional independence between time and space.ReMVC [55] is a model designed for unsupervised region representation learning from unlabeled urban data. It addresses the limitations of previous methods by leveraging contrastive learning for multi-view region representation learning.VaSCL [54] is a framework designed for data augmentation in contrastive representation learning, with a focus on the challenging context of natural language processing (NLP). Unlike other domains, NLP lacks general rules for effective data augmentation due to the discrete nature of language.SML [61] is a framework that tackles the challenges of extracting meaningful supervised signals from sparse and noisy human mobility data obtained from location-based services (LBS). The framework aims to leverage this data for various applications, including location recommendation, anomaly trajectory detection, crime discrimination, and epidemic tracing.CASCR [15] is a model that addresses the accurate representation learning of user-generated check-in sequences in human mobility data. It introduces a contrastive pre-training approach specifically designed for check-in sequence representation learning, eliminating the need for manual adjustments to data augmentation strategies. TinyLlama adopts exactly the same architecture and tokenizer as Llama 2. This means TinyLlama can be seamlessly integrated into many open-source projects built upon Llama. Additionally, TinyLlama is compact with only 1.1B parameters, allowing it to cater to a multitude of applications demanding a restricted computation and memory footprint.TinyLlama-Chat [6] . The TinyLlama-Chat model is fine-tuned on top of TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T. Following HF's Zephyr's training recipe, the model was "initially fine-tuned on a variant of the UltraChat dataset, which contains a diverse range of synthetic dialogues generated by ChatGPT. We then further aligned the model with TRL's DPOTrainer on the openbmb/UltraFeedback dataset, which contains 64k prompts and model completions that are ranked by GPT-4.".LiteLlama [12] . LiteLlama Series Models Published by Xiaotian Han, Texas A&M University. The LiteLlama series of models have a wide range of applications on edge devices, such as smartphones, IoT devices, and embedded systems, which usually have limited computational power and storage space, and they are unable to run large language models efficiently. Therefore, it is particularly important to explore small models in depth.The main models in the LiteLlama family are LiteLlama-460M. LiteLlama presents an open-source reproduction of Meta AI's LLaMa 2. It has 460M parameters with 1T tokens. LiteLlama-460M-1T was trained on the RedPajama dataset and the text was tokenized using GPT2Tokenizer. The authors evaluated the model on the MMLU task, comparing the parameter sizes to those of a large model of equal capacity, and the results demonstrate that with a significantly reduced number of parameters, LiteLlama-460M-1T still achieves results comparable to or better than the other models.Phi-2 [1] . The Phi family of language models are small language models introduced by Microsoft. The goal of the Phi family of language models is to demonstrate that by constructing high-quality pre-training data, small models can achieve significantly better performance than larger models at lower training costs. The Phi family of language models includes Phi-1, Phi-1.5, and Phi-2.Phi-2 is a Transformer with 2.7 billion parameters. Trained using the same data sources as Phi-1.5, Phi-2 incorporates a new data source consisting of various NLP synthetic texts and filtered websites (for safety and educational value). It showcases a nearly state-of-the-art performance among models with less than 13 billion parameters, especially when assessed against benchmarks testing common sense, language understanding, and logical reasoning.Pythia-70M, 1B, 2.8B [2] . The Pythia Scaling Suite is a collection of models developed to facilitate interpretability research. It contains two sets of eight models of sizes 70M, 160M, 410M, 1B, 1.4B, 2.8B, 6.9B, and 12B. For each size, there are two models: one trained on the Pile, and one trained on the Pile after the dataset has been globally deduplicated. All 8 model sizes are trained on the exact same data, in the exact same order. Additionally, 154 intermediate checkpoints per model are hosted on Hugging Face as branches.The Pythia model suite was deliberately designed to promote scientific research on large language models, especially interpretability research. Despite not centring downstream performance as a design goal, the models match or exceed the performance of similar and same-sized models, such as those in the OPT and GPT-Neo suites.GPT-2 [37] . GPT-2 is a transformers model pre-trained on a very large corpus of English data in a self-supervised fashion. This means it was pre-trained on the raw texts only, with no humans labeling them in any way. More precisely, it was trained to guess the next word in sentences.Inputs are sequences of continuous text of a certain length, and the targets are the same sequence, shifted one token to the right. The model uses internally a masking mechanism to make sure the predictions for the i-th token only use the inputs from 1 to i but not the future tokens.To further evaluate the effects of different components in Mobility-LLM, we conduct ablation experiments and analyze experimental results on all datasets. We compare these four variants on three downstream tasks.• w/o HTPP: We remove the HTPP Module. The rest of the settings are the same as Mobility-LLM. We use this setting to evaluate the function of the HTPP Module. • w/o VIMN: We use full-connection to replace the VIMN module. The rest of the settings are the same as Mobility-LLM. We use this setting to evaluate the function of the VIMN module. • w/o PPEL: We use a learnable parameter to represent a POI. We use this setting to evaluate the function of the PPEL.• w/o LLM: We use a Transformer layer to replace the LLM. The rest of the settings are the same as Mobility-LLM. We use this setting to evaluate the function of the LLM. Language Model Variants. We compared eight representative backbones on WeePlace dataset with varying capacities in Tab. 8. We found that the TinyLlama backbone model performed the best overall for our task. In particular, TinyLlama achieved the highest scores in LP (Acc@1, Acc@20) and TUL (Acc@1, Acc@5, Acc@20, MRR) tasks. The Chat version of TinyLlama was relatively less suitable, showing lower performance across most metrics. Interestingly, LiteLlama exhibited competitive performance, achieving the best Acc@5 and MRR in the LP task. Despite having fewer parameters, Pythia-70M demonstrated advantages in some tasks, achieving the second-best Acc@5 in the TUL task and the best MAE in the TP task. This suggests that the suitability of backbone models for different tasks is significantly influenced by the training strategy rather than just the number of parameters. For example, in trajectory prediction tasks, GPT-2 performed better than some models with larger parameter sizes such as Pythia-2.8B.Ablation Study. As shown in Tab. 8, we conducted an ablation study to understand the impact of different components on performance. Removing HTPP (w/o HTPP) led to a significant decrease in performance, particularly in the LP task where Acc@20 decreased of 9.29% and MRR of 7.94%. This highlights the importance of HTPP in the model's ability to make accurate predictions. Similarly, removing VIMN (w/o VIMN) resulted in noticeable performance drops in both LP and TUL tasks, indicating its role in maintaining robust prediction capabilities. Without PPE (w/o PPE), the model showed a moderate decrease in performance, with MRR in the LP task decreased of 5.75%. The removal of LLM (w/o LLM) had the most pronounced negative effect across all tasks, with LP Acc@1 decreased of 13.7% and MRR of 15.1%, and TUL metrics significantly reduced. This underscores the crucial role of the large language model in the overall framework, as even replacing it with a standard transformer could not achieve comparable performance. Language Model Variants. We compared various language model variants on the Brightkite dataset across different tasks, as shown in Tab. 9. The TinyLlama (default) variant achieved the best performance in LP tasks, with the highest Acc@1 (53.18%) and Acc@20 (74.11%), and the second-best MRR (59.89%). It also excelled in TUL tasks, achieving the highest scores across all metrics, indicating its robust performance. LiteLlama performed exceptionally well, securing the highest Acc@5 (69.13%) and MRR (59.98%) in LP tasks, and the best MAE in TP tasks.TinyLlama-Chat showed slightly lower performance compared to TinyLlama, particularly in TUL tasks, where it fell behind the other variants. Pythia-70M, despite having fewer parameters, performed competitively, achieving the second-best Acc@5 (87.63%) in TUL tasks and the best RMSE in TP tasks. Pythia-2.8B demonstrated strong performance as well, securing the second-best Acc@20 (73.38%) in LP tasks and showing consistency in TUL tasks.Overall, the results suggest that the TinyLlama variants, especially the default one, are highly effective across different tasks, while LiteLlama and Pythia variants show specific strengths, indicating that the choice of backbone model can be tailored to the specific requirements of the task.Ablation Study. The ablation study results in Tab. 9 highlight the importance of different components in the model. Removing HTPP (w/o HTPP) led to a significant drop in performance across all tasks. In TUL tasks, MRR decreased of 5.95%, while in LP tasks, there was a notable decrease in performance metrics, with Acc@20 of 7.91% and MRR of 5.26% indicating the critical role of HTPP in maintaining prediction accuracy.Removing VIMN (w/o VIMN) also resulted in decreased performance, particularly in LP and TUL tasks, where the metrics showed a notable average performance degradation of 4.07%.The absence of PPE (w/o PPE) led to moderate performance degradation, with MRR in LP tasks decreased of 4.01% and in TUL tasks of 3.38%. The removal of LLM (w/o LLM) had the most pronounced negative effect, especially in LP tasks where Acc@5 decreased of 17.03% and MRR of 14.68%. This underscores the importance of the LLM component, as its absence significantly hinders the model's overall performance across all tasks. We evaluated multiple language model variants on the FourSquare dataset across different tasks, with results presented in Tab. 10. The TinyLlama (default) variant demonstrated the best performance in LP tasks, achieving the highest Acc@1 (17.29%), Acc@20 (53.16%), and the second-best MRR. In TUL tasks, it also performed exceptionally well, securing the second-best Acc@1 (72.08%) and the best Acc@20 (84.32%) and MRR. This indicates its overall robustness in handling various tasks.The LiteLlama variant excelled with the highest Acc@5 (37.50%) and MRR in LP tasks and matched TinyLlama in TUL tasks with the best Acc@1 (72.10%), although it slightly underperformed in other TUL metrics. TinyLlama-Chat, while showing strong performance, did not outperform the default TinyLlama variant, particularly in TUL tasks where it scored lower across all metrics.Pythia-70M and Pythia-2.8B showed competitive results, with Pythia-70M achieving the second-best Acc@5 (78.79%) in TUL tasks and Pythia-2.8B securing the second-best Acc@1 (17.13%) in LP tasks. Pythia-1B also performed well, particularly in TUL tasks with the highest Acc@20 (84.39%). GPT-2, while generally performing well, did not achieve top ranks in any specific metric, indicating it may not be the best choice for these tasks compared to other variants.Ablation Study. The ablation study in Tab. 10 highlights the importance of various components in the model. Removing HTPP (w/o HTPP) led to a noticeable drop in performance across all tasks. For instance, in LP tasks, Acc@20 decreased of 7.35% and MRR of 4.69%, while in TUL tasks, MRR decreased of 6.51%, underscoring the critical role of HTPP in maintaining high performance.Removing VIMN (w/o VIMN) also resulted in decreased performance, particularly in LP and TUL tasks, where metrics showed a notable average performance degradation of 4.32%.The absence of PPE (w/o PPE) led to moderate performance degradation, with MRR in LP tasks decreased of 4.19% and in TUL tasks of 3.76%. The removal of LLM (w/o LLM) had the most significant negative effect, especially in LP tasks where Acc@5 decreased of 17.19% and MRR of 14.69%. This emphasizes the importance of the LLM component, as its absence significantly hinders the model's overall performance across all tasks.The results of 1% few-shot learning are presented in Tab. 11 and they significantly outperform all baseline methods. We attribute this to the efficient knowledge activation in our reprogrammed LLM. Remarkably, our approach consistently surpasses other competitive baselines across both LP and TUL tasks, further highlighting the potential of language models in proficient human behavior analysis.When compared to recent state-of-the-art models such as NSTPP, DSTPP, S2TUL, ReMVC, and CACSR, our average improvements exceed 20.3%, 22.9%, 84.5%, 48.2%, and 41.2% respectively, across all metrics. Even with only 1% of the training dataset, our model achieves results comparable to other models using 100% of the training dataset. This is particularly significant for privacy-protected and typically smaller Check-in datasets, as our model can effectively understand the distribution patterns of human behaviors with minimal data. Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR The results of 20% few-shot learning are presented in Tab. 12 and they significantly outperform all baseline methods. We attribute this to the efficient knowledge activation in our reprogrammed LLM. Remarkably, our approach consistently surpasses other competitive baselines across both LP and TUL tasks, further highlighting the potential of language models in proficient human behavior analysis. When compared to recent state-of-the-art models such as NSTPP, DSTPP, S2TUL, ReMVC, and CACSR, our average improvements exceed 21.4%, 21.8%, 86.6%, 46.3%, and 45.1% respectively, across all metrics. Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR Acc@1 Acc@5 Acc@20 MRR We aim to effectively fuse the sequences[z i-r+1 , • • • , z i-1 , z i ] and [s i-r+1 , • • • , s i-1 , s i ]using a gated feed-forward layer inspired by transformers. The fusion mechanism is designed to be sophisticated while maintaining the core principles of the feed-forward structure.Gating Mechanism and Gated Inputs: To enhance the fusion process, we introduce gating mechanisms g x and g z to control the flow of information from input vectors x and z. Then, apply the gating mechanisms to the inputs:EQUATIONwhere σ is the sigmoid activation function, W gx ∈ R dg×dx , b gx ∈ R dg , W gz ∈ R dg×dz , and b gz ∈ R dg , and ⊙ denotes element-wise multiplication.Multi-Layer Perceptron (MLP), Layer Normalization, and Residual Connection:Concatenate the gated inputs and process them through a multi-layer perceptron, apply layer normalization to stabilize the training process, and incorporate a residual connection to preserve the original information:EQUATIONwhereW 1 ∈ R d1×(dx+dz) , b 1 ∈ R d1 , W 2 ∈ R d2×d1 , b 2 ∈ R d2 .Perform a final linear transformation to produce the fused output:EQUATIONwhereW 3 ∈ R d f ×(dx+dz) , b 3 ∈ R d f .This design leverages the powerful feed-forward layer structure with enhancements like gating mechanisms, MLPs, layer normalization, and residual connections to achieve a sophisticated and effective fusion of the input sequences[z i-r+1 , • • • , z i-1 , z i ] and [s i-r+1 , • • • , s i-1 , s i ].The Geohash embedding function converts geographic coordinates (latitude and longitude) into a Geohash-encoded embedding vector. This involves two primary steps: first, encoding the latitude and longitude into a Geohash string, and second, converting this Geohash string into an embedding vector.Geohash Encoding Given a latitude L lat and longitude L lon , the Geohash encoding function G maps the coordinates to a Geohash string g:EQUATIONThe function G performs the following operations: Interleave the binary representations of latitude and longitude. 2. Encode the resulting binary string into base32 characters to form the Geohash string g.Geohash to Embedding Vector Let E be the embedding function that converts a Geohash string g into an embedding vector v in R d :EQUATIONThe embedding function E can be implemented using various methods such as a lookup table or a neural network that maps the discrete Geohash values to continuous vector spaces.Combined Function Combining the two steps, we define the overall Geohash embedding function F:EQUATIONThis can be expanded as:EQUATIONwhere F first encodes the latitude L lat and longitude L lon into a Geohash string using G, and then maps the Geohash string to an embedding vector using E. Justification: See Abstract and Section 1 Introduction.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: See the Tab. 6 in Appendix B.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: See Section 6.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: The paper does not suffer from this risk.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer:[NA]Justification: The paper does not use existing assets.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.https://snap.stanford.edu/data/loc-Gowalla.htmlhttps://snap.stanford.edu/data/loc-brightkite.htmlHTPPs://pytorch.org 
 
 Appendix:  


Decision: The paper introduces MobilityLLM, a novel architecture designed to enhance the analysis of human mobility behaviors using large language models (LLMs). The proposed approach includes the Visiting Intention Memory Network (VIMN) to encode visit intentions in each check-in record and a shared pool of Human Travel Preference Prompts (HTPP) to capture the semantics of these intentions. The LLMs are then fine-tuned using the Low-Rank Adaptation (LoRA) algorithm. Evaluations on four benchmark datasets for tasks such as next location prediction, trajectory user linking, and time prediction demonstrate promising performance. The paper has received generally positive reviews, and the authors have addressed most reviewers' concerns. Given that this is the first application of reprogramming techniques to LLMs for understanding human mobility, I recommend its acceptance.

Reviews: review: paper_summary: paper_summary: This paper aims to leverage large language models (LLMs) to predict human mobility behavior recorded in social media check-ins. The authors argue existing models fail short to model the visiting intentions and travel preferences embedded in check-in sequences, which could be addressed by the semantic understanding capabilities of LLMs. The authors propose to use visiting intention memory network (VIMN) to encode the visit intention in each check-in record, which is feed into the LLMs along with the shared pool of human travel preference prompts (HTPP). The employed LLMs are trained with Low-Rank Adaptation (LoRA) algorithm. The proposed method is evaluated on four benchmark datasets with the tasks of next location prediction, trajectory user link and time prediction.
strength_weakness: strength_weakness: 1. The performance gain is not substantially enough. The main results in Table 1 show the performance improvement for next location prediction is often less than 1%. Moreover, it is unclear if the performance gain is statistically significant.
2. The proposed Mobility-LLM framework leverages large language models for mobility prediction, which have substantially larger model size compared to other deep learning baselines. The authors should provide a cost analysis (e.g. complexity of training and inference) to discuss whether the performance gain justifies additional cost.
3. The model design of visiting intention memory network (VIMN) is not adequately explained. The authors propose to use LLM to capture the visiting intention and travel preferences in check-in sequences, but then they claim these semantics information cannot be directly interpreted by LLM, which needed to be reprogramed by VIMN. This design seems a bit self-contradictory and VIMN needs better justification.
4. In the Limitation section, the authors mention their Mobility-LLM is hindered by the differing user and POI counts in check-in data. This is an important observation and it should be discussed with experiment results.
questions: questions: 1. Please provide the statistical significance of the performance gains.
2. Please provide a cost analysis of the proposed method.
3. Please explain the design choice of visiting intention memory network. 
4. Please discuss the limitation of different data sizes in more detail.
limitations: limitations: The limitations are discussed in the paper.
paper_summary: paper_summary: This paper combines large language models (LLMs) to better analyze check-in sequences and understand human mobility behaviors, and proposes  a visiting intention memory network(VIMN) and a shared pool of human travel preference prompts (HTPP) to capture the semantics of human visiting intentions . The experiments show promising improvement. It is a good work in both theoretical and experimental perspectives.
strength_weakness: strength_weakness: 1. In the figure1, it’s not clear how the user embedding U_i\ is obtained?It is best to explain it in the article.
2. There are some inconsistencies in the writing of this article. For example, in Section 5.1, the LP task uses the MLE loss, but in Appendix B, the LP task uses the cross-entropy loss.
3. The model training part in the article is not detailed enough. For example, what is the parameter L_F when fine-tuning LLM? Are the three downstream tasks trained separately or uniformly?
questions: questions: 1. In the HTPP module, if there is possible that the k prompt words selected in each domain have semantic contradictions?
2. When doing TUL tasks, are user embeddings also passed into LLM as part of the input? User information should not be included in the input.
3. In section 5.5, why not experiment with the GPT-3.5 as a language model variant?As a large language model that performs well in various fields, is it possible to achieve better results by using GPT-3.5 as the backbone network?
limitations: limitations: When doing downstream tasks in this article, different parts of the LLM output are used for different tasks. What are the design considerations for this part? I concerned what should I do if I want to apply this model to other downstream tasks.
paper_summary: paper_summary: The paper proposes a novel architecture, MobilityLLM, for utilizing pre-trained LLMs with various specific embedding modules for various mobility tasks. Specifically, the authors proposed to use any pretrained LLM with adding four unique components: PPEL (POI location embedding), VIMN (timestamp embedding with GRU), HTPP (user travel preference prompting), and Multi-task training strategy. MobilityLLM finds a unique place in LLM-based human mobility modeling applications by incorporating spatiotemporal aspects of mobility characteristics using learnable components (neural networks). The experiments are tested on three different tasks: location prediction, trajectory user-link, and time-prediction. The proposed model performs consistent outperformance compared to benchmarks.
strength_weakness: strength_weakness: * The paper is relatively well written; however, there are multiple typos and abbreviation errors throughout the paper.  Here the two typo examples: (reference error in table 4. Instead of Table 3 it references Table 1) and (Figures 3 and 4 have typos: HIBP -> HTTP, IIMN -> VIMN)
* While it is designed more on check-in datasets, I wonder how applicable the model is to trajectory generation tasks. 
* Figure 1 is informative but very complicated to read without the methodology section. 
* I am aware of the page limitation, but some important details are not mentioned in the main body and are left to appendixes. For instance, few-shot learning(prediction) requires a more detailed explanation; it is not intuitively clear in the main body. 
* The Related Works section is fragile in terms of the compared baselines. They are not discussed very well in the main body and appendix. For instance, CACSR is not even discussed in both places.
questions: questions: 1. Can you elaborate on the statement at the end of the methodology section, "We use the β and α with a mean pooling projection head to predict the user uˆ who generates this check-in sequence?"

2. While the metrics mainly evaluate the performance in terms of piece-wise similarities, it would be interesting to see distributional similarities as well, such as the JS similarity metric, which can be employed.
limitations: limitations: LLMs are powerful machines but the proposed model is right now specific to one dataset. It would be interesting to explore the combination of diverse scenarios and datasets.
paper_summary: paper_summary: This submission presents a framework that leverages a large language model (LLM) as the backbone architecture for human-mobility-related tasks, including user identification, next location prediction, and arrival time prediction. The authors introduce a POI (Point of Interest) Point-wise Embedding Layer and a Visiting Intention Memory Network as an encoder to process check-in data. Additionally, they establish a behavior prompt pool to incorporate behavior preference information with the embeddings from the Visiting Intention Memory Network. The framework is evaluated using four real-world datasets, and the proposed model outperforms baseline methods in terms of most metrics.
strength_weakness: strength_weakness: 1.	While the framework offers a novel approach to addressing human mobility-related tasks, the robustness of the modules is questionable, particularly in the formulation of the human travel preference prompt. Additionally, the overall technical contribution is limited.
2.	The paper provides no discussion of choice of scoring function.
3.	There is a lack of novelty in PPEL and HTPP.
questions: questions: 1.	The reviewer suggests that the HTPP is the most significant idea in this study. However, However, several concerns need to be addressed: (i) The use of cos-similarity based scoring function to determine the significance score needs more illustration. Why is cos-similarity sufficient to evaluate the semantic similarity in urban mobility tasks? What is the impact of choice of the scoring function? (ii) What is the motivation of the selected domains as well as the prompt words? Are there any theoretical support? 
2.	This paper essentially treats each domain separately during top-k prompts selection. The reviewer suggests that it is the combination of prompts of each domain that presents meaningful and promising semantic information, i.e., a doctor (Domain 2) goes to exercise (Domain 1) to keep healthy (Domain 3).
limitations: limitations: The authors have discussed the limitations adequately.

Comments: comment: In the LP task, our model performed exceptionally well on Foursquare, and across all datasets, our model demonstrated outstanding performance in the TUL task. A key strength of our model is its robustness and stability across multiple datasets. In the context of check-in sequence prediction, where data variability is common, our model consistently performs well across different datasets. This generalization ability is crucial, as it indicates that our model is not overfitted to specific scenarios but is instead capable of adapting to various types of human activity data. We also discussed our views on marginal performance improvements in the Comment section under reviewer 6rUV: **"To verify the effectiveness of our model on a larger dataset, we increased the sample size by including users with fewer than 10 check-ins (while other papers typically exclude users with fewer than 50 or 100 check-ins). This inclusion of more sparse samples makes further prediction improvements more difficult. The performance gap between these baselines is minimal, reaching a bottleneck where further improvements with conventional deep learning models are challenging. Thus, the current performance improvements are considered valuable."** 

Additionally, the value of our work extends beyond marginal performance improvements. The primary contribution of our model lies in its innovative application of LLMs to check-in sequence prediction, introducing a new methodology that bridges the gap between language models and human activity prediction. Evaluating our work solely based on performance metrics does not fully capture the broader implications and future potential of this approach. We are pioneering a new direction in the field, demonstrating that these models can be utilized to extract and interpret complex human activity patterns. We believe that the introduction of such a novel methodology provides a foundation for future advancements and improvements in the field, which may yield even greater performance gains as the techniques mature.

Regarding concerns about the model size, it's important to note that our model's memory, which is under 12GB, is well within the capabilities of modern GPUs. With the widespread availability of GPUs with large memory capacities, our model is practical and accessible for both academic research and potential industry applications. Additionally, we have demonstrated through our experiments **with the pythia-70M base model that our approach can achieve superior performance while requiring less memory (2.74GB) and computational time compared to other baseline models**. As LLMs continue to evolve, particularly with the trend towards smaller and more efficient models, our approach is poised to become even more effective. The ongoing advancements in LLM technology will allow our reprogrammed models to achieve better results with reduced computational resources. Our model is designed to be versatile and adaptable. It can be easily integrated with any large model available on huggingface, ensuring that our approach remains relevant and capable of benefiting from the latest developments in LLMs. This adaptability makes our model a future-proof solution for human activity prediction.
Thank you for your thoughtful and constructive comments. We will include all the experimental contents of the rebuttal in our final version (if accepted).
Thank you for your further explanation. I mean in your rebuttal reply to reviewer NycQ, the Table Efficiency comparison for TUL task shows that Mobility-LLM (TinyLlama) - the computational efficiency of the model is slightly weaker than that of ReMVC, and the memory it occupies is much larger than that of ReMVC, but the performance improvement is very significant. This observation seems to prove the superiority of your method. At present, your model is superior to GETNext, which seems to prove the effectiveness of the model to a certain extent. I suggest that you include all the experimental contents of the rebuttal in your final version (if accepted). At the same time, for the experimental part, it is best to compare each different task with the latest and most popular baseline under the current task to increase credibility. In addition, I suggest that you explain the question of marginal performance gain cannot justify the much larger model size to the reviewer NycQ more clearly. Overall, based on your further rebuttal comments, I am willing to restore my original score.
Thank you for your detailed feedback. We appreciate the concerns raised and would like to provide a comprehensive response that emphasizes the significance and contributions of our work:

1) Innovative Use of Reprogramming Techniques

    Our model represents the first application of reprogramming techniques (e.g. VIMN and HTPP Module) to Large Language Models (LLMs) specifically for understanding and predicting human activity sequences from check-in data. This approach is novel and transformative within the field of check-in sequence prediction, allowing LLMs—traditionally used for natural language processing—to be effectively adapted for this unique task. By reprogramming LLMs for the prediction of check-in sequences, we are pioneering a new direction in the field, demonstrating that these models can be utilized to extract and interpret complex human activity patterns, which were previously challenging to model with traditional methods.

2) Semantic Understanding and Accurate Predictions

    Our experiments clearly show that our reprogramming design enables the LLM to understand the semantic information embedded within human activity sequences. This understanding is crucial, as check-in data is not just about locations and times; it carries rich context about human intentions, behaviors, and routines. The model's ability to leverage this semantic understanding to make accurate predictions of future check-ins highlights a significant advancement in the field. It demonstrates that LLMs, when reprogrammed appropriately, can effectively capture and utilize the intricate patterns and meanings in human activity data, leading to more precise and contextually aware predictions.

3) Beyond Pure Performance Metrics

    We recognize the importance of performance metrics; however, the value of our work extends beyond marginal performance improvements. The primary contribution of our model lies in its innovative application of LLMs to check-in sequence prediction, introducing a new methodology that bridges the gap between language models and human activity prediction.     Evaluating our work solely based on performance metrics does not fully capture the broader implications and future potential of this approach. We believe that the introduction of such a novel methodology provides a foundation for future advancements and improvements in the field, which may yield even greater performance gains as the techniques mature.

4) Model Size and Memory Requirements

    Regarding concerns about the model size, it's important to note that our model's memory footprint, which is under 12GB, is well within the capabilities of modern GPUs. With the widespread availability of GPUs with large memory capacities, our model is practical and accessible for both academic research and potential industry applications. Additionally, we have demonstrated through our experiments with the pythia-70M base model that our approach can achieve superior performance while requiring less memory and computational time compared to other baseline models. This showcases our model's efficiency and adaptability, ensuring it remains practical even in resource-constrained environments.

5) Robustness Across Multiple Datasets

    A key strength of our model is its robustness and stability across multiple datasets. In the context of check-in sequence prediction, where data variability is common, our model consistently performs well across different datasets. This generalization ability is crucial, as it indicates that our model is not overfitted to specific scenarios but is instead capable of adapting to various types of human activity data. The consistent improvements observed across four different datasets validate the effectiveness of our design and its potential for broader application in diverse real-world scenarios, such as personalized services, urban planning, and more.

6) Future Prospects and Model Versatility

    As LLMs continue to evolve, particularly with the trend towards smaller and more efficient models, our approach is poised to become even more effective. The ongoing advancements in LLM technology will allow our reprogrammed models to achieve better results with reduced computational resources. Our model is designed to be versatile and adaptable. It can be easily integrated with any large model available on huggingface, ensuring that our approach remains relevant and capable of benefiting from the latest developments in LLMs. This adaptability makes our model a future-proof solution for human activity prediction.

In conclusion, we believe that our work represents a pioneering effort in the field of check-in sequence prediction and human trajectory analysis. The innovative application of reprogramming techniques to LLMs on our tasks not only demonstrates the feasibility and effectiveness of this approach but also provides a robust and adaptable solution that can inspire and guide future research.
### Other-Q1
Yes

### Other-Q2
We conducted experiments on GETNext, and the results of GETNext is comparable to PLSPL:

|Datasets|Gowalla||||WeePlace||||Brightkite||||FourSquare||||
|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|
|Metric (e-2)|Acc@1|Acc@5|Acc@20|MRR|Acc@1|Acc@5|Acc@20|MRR|Acc@1|Acc@5|Acc@20|MRR|Acc@1|Acc@5|Acc@20|MRR|
| GETNext | 11.15 | 24.04 | 33.58 | 17.23 | 18.69 | 37.17 | 53.16 | 27.73 | 51.22 | 65.23 | 71.40 | 57.33 | 13.67 | 30.54 | 43.05 | 21.28 |

### Other-Q3
I might not have fully understood your point. When you mentioned ReMVC, were you possibly referring to the PLSPL model? The line "Mobility-LLM (TinyLlama)-Patience 3 11.3G 2.96h 10" is intended to illustrate that compared to PLSPL, our model requires less training time, uses more memory, but offers significant performance improvement. If compared to ReMVC, the performance improvement would be even more significant.

### Other-Q4
We have modified the figure according to your request. The `q` and `k` were positioned at the bottom due to a formatting error when exporting to PDF, which has been corrected. The updated version is available on the anonymous GitHub.
I agree with Reviewer 6rUV that the small performance gain is an important problem. Such marginal perfomance gain cannot justify the much larger model size. In the followup response, the authors justify this with different experiment settings. But these differences are not contextualized in the original submission or rebuttal. It is difficult to evaluate these claims. The authors also argue the proposed method is multi-task. However, as also noted by other reviewers, it was only applied to check-in data, which is a subset of mobility data that contain rich semantic information that probably will favor an LLM solution.

I was really on the fence about this paper, because it also had relatively good presentation and completeness. However, after reading other reviewer's interactions with the authors, I also lean towards a rejection.
Thank you for your further reply, which basically solved my concerns (Yours part). Please also answer my comments on other questions.

【Other-Q1】
Taking Table 1 as an example, for acc@1, can I understand that the performance of deepmove under the default settings is only 10.51/1.37=7.67?

【Other-Q2】
What about baseline GETNext [1], which supports variable-length input? This is the most popular baseline for next location prediction in the past two years.

[1] Yang, Song, Jiamou Liu, and Kaiqi Zhao. "GETNext: trajectory flow map enhanced transformer for next POI recommendation." Proceedings of the 45th International ACM SIGIR Conference on research and development in information retrieval. 2022.

【Other-Q3】
Mobility-LLM (TinyLlama)-Patience 3 11.3G 2.96h 10
The results in this row seem to indicate that the computational efficiency of the model is slightly weaker than that of ReMVC, and the memory it occupies is much larger than that of ReMVC, but the performance improvement is very significant, right?

【Other-Q4】
In the main figure, LLM (Partially-Frozen+LoRA) is missing the number (d)
In sub-figure (a), the order of the attention module is v, k, q, and in sub-figure (d) the order is q, k, v. It is recommended to unify the order. In addition, why are q and k in sub-figure (d) below the weight matrix, while v is part of the weight matrix?
We apologize that our previous response did not provide the help you needed.
We have optimized the overall Figure 1 by reducing the coupling of elements, enhancing clarity, and improving the layout. This makes Figure 1 clearer and more intuitive. The revised version is available in our anonymous github(our manuscript line 187). 
Also, we have refined the Related Work section and placed the improved version in section A.1 of the Appendix, along with the full list of references in our anonymous github.
### 【Other-Q4】
We have optimized the overall Figure 1 by reducing the coupling of elements, enhancing clarity, and improving the layout. This makes Figure 1 clearer and more intuitive. The revised version is available in our anonymous gitHub(our manuscript line 187).
### 【Other-Q5】
We have refined the Related Work section and placed the improved version in section A.1 of the Appendix, along with the full list of references in our anonymous github.
### 【Other-Q3】
As you mentioned, the number of epochs alone cannot effectively demonstrate the convergence speed. The total training time should be used to prove the speed of convergence. In further experiments, we tried reducing the early stopping patience from 10 to 3 to decrease the model's training time. So, we adjusted the patience to 3 and were surprised to find that some of the model's metrics actually improved with less traing time. After our analysis, we found that the reason is that the patience setting during the training of our model was relatively large compared to the total number of training epochs. This led to some unnecessary time overhead and caused our model to slightly overfit.

Below, we present the GPU memory usage, training time for the LP task on the WeePlace dataset, and the results of the LP task under the new setting on four datasets. We also included variants of Python-70M (as presented in Table 5 and A.6 of the paper).

|Method|Memory|Training Time|Training Epoch|
|-|-|-|-|
|PLSPL-Patience 10|3.4G|4.73h|79|
|PLSPL-Patience 3|3.4G|4.45h|75|
|Mobility-LLM (TinyLlama)-Patience 10|11.3G|5.45h|17|
|Mobility-LLM (TinyLlama)-Patience 3|11.3G|2.96h|10|
|Mobility-LLM (pythia-70M)-Patience 10|2.74G|2.76h|39|
|Mobility-LLM (pythia-70M)-Patience 3|2.74G|1.47h|21|

|Datasets|Gowalla||||WeePlace||||Brightkite||||FourSquare||||
|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|
|Metric (e-2)|Acc@1|Acc@5|Acc@20|MRR|Acc@1|Acc@5|Acc@20|MRR|Acc@1|Acc@5|Acc@20|MRR|Acc@1|Acc@5|Acc@20|MRR|
|The Best Baseline|11.47±0.03|24.12±0.17|33.82±0.15|17.44±0.05|19.66±0.04|37.68±0.18|53.44±0.13|28.46±0.06|51.42±0.12|66.25±0.08|71.46±0.11|57.59±0.14|16.92±0.04|36.05±0.04|49.39±0.16|26.02±0.04|
|Mobility-LLM (TinyLlama)-Patience 10|11.87±0.04|25.14±0.04|36.36±0.09|18.29±0.11|20.47±0.13|39.22±0.19|56.69±0.10|29.21±0.15|53.18±0.17|68.31±0.14|74.11±0.18|59.89±0.03|17.29±0.03|37.17±0.02|53.16±0.20|26.47±0.05|
|Mobility-LLM (TinyLlama)-Patience 3|11.90±0.03|25.17±0.05|36.43±0.11|18.30±0.10|20.50±0.07|39.27±0.17|56.71±0.09|29.24±0.13|53.18±0.16|68.34±0.11|74.17±0.18|59.91±0.04|17.31±0.04|37.20±0.03|53.19±0.22|26.44±0.02|
|Mobility-LLM (pythia-70M)-Patience 10|11.03±0.16|24.86±0.11|35.74±0.07|17.91±0.17|19.88±0.18|38.02±0.03|54.19±0.04|28.46±0.13|52.03±0.10|66.34±0.02|72.09±0.05|57.93±0.22|17.01±0.04|36.20±0.09|50.88±0.13|25.84±0.16|
|Mobility-LLM (pythia-70M)-Patience 3|11.29±0.13|24.98±0.18|35.56±0.16|17.98±0.02|19.93±0.11|38.21±0.22|54.69±0.03|28.79±0.07|52.14±0.04|66.87±0.10|73.21±0.17|58.90±0.05|17.12±0.09|36.33±0.04|52.13±0.04|26.03±0.14|

The experimental results show that our model can maintain its previous advantages while using less training time than the conventional deep learning models, with some metrics even improving.

The new experimental setup shows that the TinyLlama base model can achieve better results in less time. We also conducted further experiments with a variant of pythia-70M and found that setting patience to 10 indeed caused slight overfitting. Our pythia-70M base model can outperform the PLSPL baseline model with less time and memory usage. To verify whether PLSPL also suffers from overfitting, we conducted experiments with patience set to 3, and the results are as follows:

|Datasets|Gowalla||||WeePlace||||Brightkite||||FourSquare||||
|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|
|PLSPL-Patience 10|11.26±0.02|24.12±0.17|33.82±0.15|17.44±0.05|18.77±0.11|37.31±0.02|53.31±0.18|27.86±0.10|51.42±0.12|65.34±0.09|71.46±0.11|57.59±0.14|13.73±0.07|30.65±0.10|43.18±0.09|21.42±0.04|
|PLSPL-Patience 3|11.22±0.07|24.08±0.17|33.81±0.02|17.40±0.02|18.81±0.14|37.26±0.10|53.31±0.09|27.84±0.10|51.46±0.09|65.31±0.04|71.44±0.11|57.57±0.18|13.70±0.12|30.60±0.11|43.17±0.15|21.38±0.05|

We observe that conventional deep learning models do not exhibit overfitting under the patience 10 setting And the total training time was also roughly the same.

In summary, we appreciate the reviewer for raising these issues, which have helped us improve and further explore our model's potential. We were previously unaware of the impact of training epochs on our model. After changing the patience setting to 3, we were pleasantly surprised by better performance in less time than the conventional deep learning models. In Section 5.5 of the paper, we also discussed that the parameter size of the base model does not necessarily correlate with performance improvements. With the future advancement of more "refined" large language base models in huggingface, our model's potential will be further unleashed.
### 【Other-Q2】
#### 【Other-Q2-1】
We first explain why we did not include Graph-Flashback as a baseline model from the perspectives of application scenarios, data processing methods, model architecture input, and dataset support.

1. **Application Scenario**: Graph-Flashback is a classic and effective baseline model, but it requires a fixed-length historical sequence as input. Our study focuses on the representation and prediction of variable-length sequences. Therefore, the baselines in our paper are models that can handle variable-length sequences.
2. **Data Processing Methods**: In next location prediction research, it is common to handle variable-length sequences. Processing samples often involves filtering out POIs with less than 10 visits, users with less than 10 check-ins, limiting historical days to no more than 30, and ensuring that the next location visit interval does not exceed one day to qualify as a label. This approach leads to variable-length sample sequences that better reflect real-world scenarios. In contrast, the data processing method of Graph-Flashback, as mentioned in the original paper, is: "We discard inactive users who have less than 100 check-ins and sort each user’s check-ins in ascending order of timestamp. The first 80% check-ins of each user are split into multiple length-equally (e.g., 20) sequences, which are chosen as the training set. Likewise, the remaining 20% are regarded as the testing set."
3. **Model Architecture Input**: The input to this model is a fixed-length sequence. We thoroughly reviewed the code of Graph-Flashback, and the historical sequences it processes are fixed-length sequences of 20. We also attempted to modify it into a variable-length prediction model, but since the entire framework is designed for fixed-length sequences, almost the entire framework would need adjustment, and many components can only handle fixed-length sequences. Further modifications would cause the model to deviate significantly from the original design of Graph-Flashback.
4. **Dataset Support**: Not all datasets provide friendship information. Only the Gowalla and Foursquare datasets provide friendship networks, while Weeplace and BrightKite do not disclose friendship networks due to privacy reasons, or the disclosed friendship networks are unreliable. Therefore, Graph-Flashback cannot run on our other two datasets.

We firmly believe that Graph-Flashback is a classic baseline model, and we have cited it in the Related Work section and will include Graph-flashback in our related work and make fully discussion in our revised version. However, due to the multiple issues mentioned above that cannot be unified or fairly addressed, Graph-Flashback could not be fairly included in our baseline model comparison.

#### 【Other-Q2-2】
Next, we explain why the comparison between our model, Graph-Flashback, and DeepMove is not valid:

1. **Different Scenario Settings**: The models we selected, such as DeepMove, are designed to handle variable-length sequences, and their advantages are more evident in such scenarios. Although Graph-Flashback includes many baseline models that handle variable-length sequences, these baselines, including DeepMove, cannot fully exploit their advantages in the scenario set by Graph-Flashback. In the comparison experiment under Graph-Flashback's setting, the gaps between the baselines are large, while in our setting, the gaps are small. These differences are not on the same scale, so a direct comparison cannot distinguish the superiority of the models.
2. **Different Experimental Settings**: We were surprised to find that the DeepMove model's performance improved by 37% when the Dropout parameter was set to 0.9. Although this parameter may seem unreasonable, we still respect the facts. However, we are unsure whether Graph-Flashback conducted experimental adjustments and a comprehensive parameter search for the Dropout parameter of DeepMove during the experiment.
3. **Unified Framework**: Our model is a unified framework capable of handling multiple tasks. To demonstrate the powerful performance of our framework, we compared it with the most classic and SOTA end-to-end models in various downstream tasks. Therefore, more attention should be paid to the overall performance improvement.
### 【Other-Q1】
We will explain from three aspects: parameter search, performance bottleneck, and multi-task framework.

1. To ensure fairness, we did not use default parameters for the experiments to save time. Instead, we conducted a comprehensive parameter search for all baseline models, selecting the optimal hyperparameters for each on different datasets to ensure they perform at their best. During the search, we found that some baseline models performed significantly better with certain unexpected parameter combinations (e.g., the DeepMove model's performance improved by 37% when the Dropout parameter was set to 0.9 compared to the default parameters). Therefore, the results we reported reflect the best possible performance of the baseline models, surpassing their default settings.
2. To verify the effectiveness of our model on a larger dataset, we increased the sample size by including users with fewer than 10 check-ins (while other papers typically exclude users with fewer than 50 or 100 check-ins). This inclusion of more sparse samples makes further prediction improvements more difficult. The performance gap between these baselines is minimal, reaching a bottleneck where further improvements with conventional deep learning models are challenging. Thus, the current performance improvements are considered valuable.
3. Our model can handle multi-task frameworks. To demonstrate its powerful performance, we compared it with multi-task framework models(such as ReMVC, VaSCL, SML, CACSR) and various SOTA models designed for specific downstream tasks. Our model achieved SOTA results in nearly all metrics, with improvements exceeding 26% in some cases, and outperformed other SOTA multi-task framework models by over 15%.
We sincerely apologize for not accurately addressing your question. We realize that we did not fully understand your comments before. In this discussion, we will first address the questions raised in your current comment in detail and then respond to the issues raised by other reviewers in the subsequent comments.


### 【Yours-Q1】
The selection of domains in the HTTP module is based on our research summary of various materials and reports describing human activity characteristics. Initially, besides these three domains, we also considered other fields such as age, travel time, and activity range. 

After testing on a small dataset, we discovered that the selected three domains had the most significant impact on the model's performance. For each set of terms within one domain, we asked an LLM to provide a comprehensive set of commonly used terms for that domain.

We found that selecting 16 terms sufficiently met our experimental needs. Choosing more could lead to redundancy, while fewer terms would fail to cover the domain knowledge comprehensively. Ultimately, this exploration led to the current experimental setup.

### 【Yours-Q2】
Cosine similarity measures the angle between two vectors rather than the distance, making it suitable for measuring directional similarity rather than magnitude. In high-dimensional spaces, text or user behavior data is represented as high-dimensional sparse vectors, making it more effective in capturing textual feature similarity without being disrupted by data sparsity. The direction of the vectors often reflects the semantic similarity of words or sentences.

In contrast, Euclidean distance and Manhattan distance reflect absolute positional differences, which can be problematic in high-dimensional sparse data. The presence of zero dimensions might reduce similarity, leading to indistinguishable distance calculations that fail to capture actual similarity[1]. Therefore, it is more reasonable to use cosine similarity in the HTPP module compared to other measurement methods.

[1] Turney, P. D., & Pantel, P. (2010). From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37, 141-188.

### 【Yours-Q3】
We acknowledge that our previous explanation may have been unclear, and we apologize for the confusion.

**The case study mentioned in our previous response actually refers to the discussion in Q2 of the initial Rebuttal** and is not related to the case study in the paper.

We would like to elaborate on the point we intended to express in the rebuttal as follows.

At the outset of our model design, we tried three different approaches, including the one you mentioned in Q2:

1. Listing all possible combinations of all domains，each combination is composed of one term from each domain.
2. Manually selecting reasonable combinations in approach 1.
3. Our current approach.

We compared the performance and efficiency of these three approaches. During the performance comparison, we looked into the results of approaches 1 and 3 through a case discussion. We found that the results from approach 3 were more reasonable, while the combinations from approach 1 were often unsatisfactory. During the training, the semantic of a sequence might match multiple terms in each domain, which means that if approach 1 was adopted, more than one combinations might be matched. Since the combinations contain semantic from multiple domains is coupled together in approach 1, this could cause interference among the matched combinations, resulting in unsatisfactory selections. However, in approach 3, each domain is decoupled and matched separately, so this issue does not exist.
Additionally, approach 1 was time-consuming, so after the case discussion, we discarded it. 

If the approach 2 is adopted, where multiple terms are selected from each domain, the number of combinations will be extremely large, making it impractical to manually select reasonable combinations.

Therefore, approach 3 is optimal in terms of efficiency and performance, so we ultimately chose approach 3. This choice of approach and the above case discussion were not mentioned in the paper before. We will also add the above content to Section 5.5 of the paper. We are grateful for your suggestion, which helped us make our experiment and paper clearer and more complete.
Thank you for the response. I am not going to change my score but I could not find any insight from your responses regarding my questions. I would suggest you to pay more attention in your next rebuttal.
Thanks for the response. I maintain my origin score.
I am disappointed with the author's rebuttal. The author did not sufficiently address my questions; the responses lacked clarity and evaded the main issues. For instance, I don't understand the purpose of the combined response to W1&W2Q1. Your answer is completely off-topic, and I feel that W2's question should correspond to W3. Specifically, my questions are:

1. I have concerns about the robustness of the human travel preference prompt (HTPP) module. For example, your HTPP module includes several critical parameters, such as D. Why is the domain set to 3? Is this a limitation of the dataset, an arbitrary definition, or derived from experiments? I am unclear. What is the reason for setting m to 16? Even if the author did not mention this in the paper, it should be adequately addressed in the rebuttal. Unfortunately, I did not find any relevant answers.

2. My question is why you chose cosine similarity instead of other methods like Euclidean distance or Manhattan distance. These could all be discussed. Of course, I know that cosine similarity can measure the similarity of vectors in a unified semantic space, but what are its specific advantages? Did you determine through experiments that cosine similarity is better? Can't these topics be further discussed?

3. Your response claims that the existing solution is superior to the idea I proposed in Q2. The original text states, "We provide three case studies to visualize the improvements brought by HTPP and VIMN." I do not understand the relevance of the cases you provided to my question. I need a clearer explanation, such as comparing my question's definition with your case studies. Your response only added to my confusion.

Other reviewers have also provided insights from different perspectives, for example:

1. Reviewer NycQ believes that the performance improvement is not significant enough, which is indeed a problem. Your rebuttal mentioned that the t-value between Mobility-LLM and the best baseline is very high. However, I think this is because both models are very stable, and changing the random seed does not overly affect the model's performance, leading to a low standard deviation and a high t-value. This is meaningless. The core issue is that your improvement over the best baseline is only 0.4%. While such an improvement might be acceptable for other deep learning papers, this paper is based on LLM, requiring better GPU, more computing time, and potential API costs, making the slight performance improvement questionable.

2. For example, the classic baseline Graph-Flashback [1] for next location prediction shows more than double the performance improvement on Gowalla compared to DeepMove, and 4% higher on Foursquare. Why didn't you compare with some classic baselines, or if your performance improvement could also reach this level, it would be more reasonable. DeepMove is only a baseline published in 2018, but according to Table 1, your model only shows about a 1% improvement over DeepMove on Gowalla and Foursquare.

3. The authors claim that their convergence speed is faster than other baselines, but I did not see any relevant evidence. If you are referring to fewer epochs, I think this is unreasonable. The time each model consumes per epoch is completely inconsistent. The speed of convergence should consider the overall training time. What I observed is that Mobility-LLM consumes more memory, requires more training time, and inference time. The increased computational cost only brings minimal performance gains.

4. Reviewer Mti9 considers your Figure 1 to be rather cluttered. Why didn't the authors use the global response to upload a PDF showing the improved figure? After your adjustments, we are not clear about what the final version will look like.

5. Additionally, CACSR was not discussed. Why didn't you present a relevant discussion in the rebuttal?

Overall, although the authors initially received an overall positive comment, I do not see the authors taking the reviewers' questions seriously in the rebuttal, and the evasive answers have deepened my doubts. It is undeniable that the completion of this paper, including the production of charts and experimental results, is very high. However, the vague explanations of many details, the additional computational costs, and the insufficiently superior performance lead me to lean towards rejecting this paper, despite the positive attitude of other reviewers.

[1] Rao, Xuan, et al. "Graph-flashback network for next location recommendation." Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 2022.
I thank the authors for providing additional experiment results. They should be included in the manuscript for completeness. However, I still think the performance gain is not substantial enough, especially considering the significantly larger model size. After reading other reviewers' comments, I lean towards a rejection now.