Provide only relevant keywords to facilitate an online search related to the user's message and remember that they should be few words and should not be repetitive: System: Based on the search results given, can you provide a concise overview of the main hypothesis or research question addressed in the article, along with the key methodologies used by the researchers to investigate it? Additionally, could you summarize the main findings obtained from the experiments and discuss their implications for advancing state-of-the-art research in the field? Lastly, please identify any limitations of the study and suggest potential areas for future research, while also evaluating the strengths and weaknesses of the presented conclusions. The answer should be well structured and between 300 and 400 words.

User: SEARCH RESULTS:

[Page no. 8] "evaluate whether the model can find the truth. For the k-nearest search task, we use Precision to measure the coverage of the top-k results. D. Performance Comparision 1) Overall Performance: Table II reports the overall results on the three downstream tasks. Based on the table, we can make the following observations. • Our START achieves the best performance in terms of all metrics on these three tasks for the two real- world datasets. It confirms the superior performance of our framework in learning trajectory representations by introducing temporal regularities and travel semantics in the pre-training phase. • The encoder-decoder models with reconstruction outper- form the sequence representation models (Transformer, BERT). It could be because the pre-training methods of these two models from the natural language processing domain are not suitable for trajectory data, ignoring the spatial-temporal characteristics. • Trembr performs best among all baselines because it considers the visit timestamp of each road in the de- coding process, highlighting the importance of temporal information in the trajectories. • The performance of the two-stage models, i.e., PIM and Toast, is unsatisfactory due to two factors. First, they consider trajectories as ordinary road sequences and ignore the temporal information. Second, their road rep- resentation learning method does not adequately consider the travel semantics, such as road visit frequencies. 2) Performance of Trajectory Travel Time Estimation: We fine-tune all models with the objective function (16). Note that no time information is fed into the model during fine-tuning, except for the departure time to avoid information leakage. In addition to the overall performance in Table II, to investigate the performance of the model under different scenarios and verify the role of pre-training with temporal regularities, we present the MAPE results on different departure times, whether it is a weekend or not, and the hops"

[Page no. 2] "Specifically, the TAT-Enc fuses minute and day-of-week indexes with road segment representations to capture the periodic temporal patterns of urban traffic and adopts a Time Interval-Aware Self-Attention to process irregular time interval information. We also design two self-supervised tasks to train our START. The first is span-masked trajectory recovery, which masks consecutive subsequences in trajectories to capture the local features and order information. The second is trajectory contrastive learning, which employs four data augmentation methods that consider the spatial-temporal characteristics of trajectories to train the contrastive learning loss. Compared to the traditional self-supervised tasks such as sequence re- construction and MLM, the proposed tasks fully consider the spatial-temporal characteristics of the trajectories. The effectiveness of the proposed method is verified by extensive experiments on two large-scale datasets for three downstream tasks. The results show that START significantly outperforms the state-of-the-art models. In summary, the main contributions of this paper are sum- marized as follows: • We propose a two-stage TRL method incorporating tem- poral regularities and travel semantics into trajectory representations. Compared to previous TRL research, the proposed method can utilize more spatial-temporal characteristics of trajectories for downstream tasks. • We design two self-supervised tasks to train our START. Compared to traditional self-supervised tasks for gen- eral sequence representation learning, such as sequence reconstruction and MLM, the proposed tasks are more suitable for TRL since they account for the spatial- temporal characteristics of trajectories. We believe these tasks can be applied to other TRL model training. • In addition to superior performance, the experiments also demonstrate that the proposed self-supervised tasks can use fewer data to outperform the supervised model. Moreover, our methods can be transferred across hetero- geneous road network datasets. To the best of our knowl- edge, this is the first TRL method with this feature, which is"

[Page no. 8] "of the trajectory on BJ in Figure 3. Here we compare three models, including START, a variant without temporal (noted as w/o Temporal) where the time embeddings and the time interval matrix are removed, and the best baseline Trembr. We can observe the following phenomena: (1) START consistently outperforms others, regardless of the weekday or weekend or the trajectory hop size. Moreover, START shows excellent performance, especially in the late peak periods (16:00-21:00) and when the trajectory is between 20 to 100 hops. (2) The no temporal variant cannot capture temporal regularities and therefore has worse performance than START, highlighting the importance of temporal regularities when pre-training. 3) Performance of Trajectory Classification: We fine-tune all models with the objective function (17). We use whether the taxi carries passengers as a binary classification label in BJ and the driver ID as the label in Porto for the multi-classification (435 classes). As shown in Table II, our model consistently outperforms all baselines because it can capture the underlying travel semantics and achieves accurate performance. 4) Performance of Trajectory Similarity Search: The sim- ilarity measure is a fundamental problem with various appli- cations, such as identifying popular routes and similar drivers in trajectory analysis. A recent study [8] proposes to use the most similar trajectory search and the k-nearest trajectory search to evaluate the effectiveness of different methods. We adopt it in the experiments as it is currently the best evaluation method that proves the effectiveness of the model from multiple perspectives. Here we directly use the trajectory representations obtained from the pre-training without fine- tuning and use the Euclidean distance of the representations to represent the similarity between the trajectories, i.e., the smaller the distance, the greater the similarity. (a) Most Similar Trajectory Search: The most similar trajectory search task is to"

[Page no. 11] "is a pre-trained method for learning repre- sentations that can be trained offline, in practicality, we are more concerned with the time cost of encoding the trajectories into representations and applying them to downstream tasks. Here we use the default settings described in Section IV-C. We report only the results for Porto due to space limitations, and the results for BJ are similar. First, we report the inference time of START and baselines, i.e., the time cost of embedding 100k-400k trajectories in Figure 10(a). The results show that self-attention models are faster than RNN models. This is because RNN model need O(L) sequential operations to process the trajectory while self-attention model only need O(1). Here L is the length of the trajectory. Besides, START is slightly slower than other self-attention models because it introduces the TPE-GAT layer and the time interval matrix, a tradeoff between performance and efficiency. Even so, it takes only 25.8 seconds to encode 100,000 trajectories, and it can get even faster as the batch size gets larger during inference. In addition, we also compare the time cost of similarity search. Figure 10(b) shows the average time cost of a query for performing the most similar search with different query sizes and database sizes. The size of the query and detour dataset Nq varies from 10 to 10,000, and the size of the negative samples Nneg is ten times Nq. In addition to the two representative deep models Toast and Trembr, we also compare some traditional algorithms including Dynamic Time Warping (DTW) [32], Longest Common SubSequence (LCSS) [33], Fr’echet Distance [34], and Edit Distance on Real Sequence (EDR) [35]. For the deep models, we sum the time cost of obtaining the representations and computing the similarities using the representations. We can see deep models are at least"

[Page no. 6] "the test dataset. 2) Trajectory Classification: This task aims to classify tra- jectories based on a specific label, such as carrying passengers or not, the driver ID, the transportation, etc. We employ a simple fully connected layer with the softmax activation to obtain the predicted value as ˆyi = softmax(FC(pi)). Then we optimize the model with the cross-entropy loss: Lclassify = 1 N N � i=1 C � c=1 −yi(c) log(ˆyi(c)), (17) where yi is the ground truth, N is the total number of trajec- tories in the test dataset, and C is the number of categories. 3) Trajectory Similarity Computation and Search: In this task, we design two sub-tasks: the most similar trajectory search and the k-nearest trajectory search. Here we directly use the representation pi obtained from the pre-training task without fine-tuning. The most similar trajectory search task is to find out the most similar trajectory from a large database given a query. In the k-nearest trajectory search task, given a trajectory, models need to find top-k similar trajectories from candidates ignoring the rank. The detailed settings for these two tasks can be found in Section IV-D4. IV. EXPERIMENTS In this section, we conduct extensive experiments to evaluate the performance of the START framework. The experiments include five parts: • Performance Comparison. We compare the performance of START with eight baselines on two large-scale datasets for three downstream tasks. The experiment results show the superior overall performance of START. • Pre-training Effect Study. We demonstrate the effec- tiveness of the self-supervised pre-training tasks over small-size datasets and across datasets. The results show that the self-supervised tasks can effectively reduce the usage of training data, and the model can be transferred across heterogeneous datasets. This nature is beneficial for solving the problem of insufficient training data. • Ablation Experiment."

SUMMARY:
Assistant: