Understanding the Geospatial Reasoning Capabilities of LLMs: A Trajectory Recovery Perspective

## Abstract

Abstract We explore the geospatial reasoning
capabilities of Large Language Models (LLMs), specifically, whether LLMs can read road network maps and perform navigation. We frame trajectory recovery as a proxy task, which requires models to reconstruct masked GPS traces, and introduce GlobalTrace , a dataset with over 4,000 real-world trajectories across diverse regions and transportation modes. Using road network as context, our prompting framework enables LLMs to generate valid paths without accessing any external navigation tools. Experiments show that LLMs outperform off-the-shelf baselines and specialized trajectory recovery models, with strong zero-shot generalization.
Fine-grained analysis shows that LLMs have strong comprehension of the road network and coordinate systems, but also pose systematic biases with respect to regions and transportation modes. Finally, we demonstrate how LLMs can enhance navigation experiences by reasoning over maps in flexible ways to incorporate user preferences.

## 1 Introduction

Figure: Figure 1: An example of the trajectory recovery task of a sample in GlobalTrace. The masked segment to recover is in green, which is part of a trajectory shown in blue.
Refer to caption: demo_trace.png

Large Language Models (LLMs) are increasingly recognized as general-purpose systems, showing strong performance across domains ranging from mathematics and coding to vision and robotics. An emerging yet underexplored question is whether these models possess geospatial understanding, the ability to reason about maps, paths, and spatial relationships. Such capabilities are fundamental to many real-world applications, e.g., autonomous vehicle navigation, logistics, and urban planning.
While prior work has studied LLMs in contexts such as geographic knowledge retrieval (Manvi et al., 2024a; b) and map-based multiple-choice question answering (Dihan et al., 2025), the ability of LLMs to read road networks and plan paths has not been systematically evaluated.

We investigate whether LLMs can perform navigation through the trajectory recovery
task: reconstructing masked segments of GPS traces from the road network context, to bypass the restriction of relying on shortest path-type of ground truth which may not reflect human navigation pattern in practice (Golledge, 1995; Duckham & Kulik, 2003).
To facilitate this evaluation, we introduce GlobalTrace, a dataset of over 4,000 real-world trajectories spanning multiple continents and diverse transportation modes, collected from public traces on OpenStreetMap (OpenStreetMap contributors, 2017). Unlike existing datasets, GlobalTrace covers multiple activities, i.e., transportation modes (e.g., driving, cycling, walking, and hiking), and geographic regions, enabling a comprehensive evaluation of model generalization. Our dataset is framed in away that is harder than the traditional point-wise trajectory recovery task (Newson & Krumm, 2009; Song et al., 2017; Si et al., 2024), and closer to the higher-level navigation problem. As [Figure 1](https://arxiv.org/html/2510.01639v1#S1.F1) shows, to successfully reconstruct a masked segment of a trajectory, a model must be able to (1) understand the context of travel (e.g., mode of transportation, speed, and direction) as well as the constraints of the road network (e.g., one-way road and road connectivity) to select an appropriate route and (2) generate smooth coordinates following the route.

Traditionally, the task of trajectory recovery is tackled using either probabilistic models and map-matching techniques (Newson & Krumm, 2009; Huang et al., 2018; Yin et al., 2018), such as Hidden Markov Models (HMM), or interpolation heuristics (Zheng et al., 2008), to reconstruct missing points from noisy or sparse GPS traces. While effective in constrained settings, these approaches are limited in capturing long-range dependencies, or adapting to complex moving patterns. Following the pre-training trend after Transformer (Vaswani et al., 2017) is introduced, there is also effort to build foundation trajectory models (Liang et al., 2025; Lin et al., 2024; Si et al., 2024; Yu et al., 2025b) to encode geospatial knowledge from large amount of training data. This approach relies on training using specialized in-domain trajectory data and struggles to generalize to unseen regions.

To address these issues, we explore the generalization capability of LLMs and present a prompting-based framework that allows LLMs to reconstruct valid trajectories without reliance on external navigation tools. Our system reasons over the available road network data to perform reconstruction without the need of additional in-domain training.
Our experiments compare LLMs against both off-the-shelf navigation systems and specialized trajectory recovery models. Results show that state-of-the-art LLMs achieve superior performance, with strong zero-shot generalization across regions and activity types. Fine-grained analysis highlights their ability to plan over complex road networks, adhere to geometric constraints, and generate realistic coordinates, while ablation studies reveal that compact topological representations with directional cues are the most effective input format.

This combination of geospatial reasoning with LLMs’ semantic strengths to understand flexible user queries opens new opportunities for navigation. For example, beyond recovering trajectories, an LLM could generate a route that is not only efficient but also tailored to a user’s preferences, such as scenic views, safety, or avoiding noisy streets—by integrating contextual knowledge already embedded in the model. Such preference-aware navigation lies beyond the scope of traditional systems, but emerges naturally when map reasoning is combined with LLMs’ flexible language and reasoning capabilities.

Our contributions are threefold:

- •
We introduce GlobalTrace, a benchmark of over 4,000 real-world trajectories across regions and transportation modes to benchmark models’ geospatial reasoning capabilities.
- •
We propose a prompting-based framework that elicits LLMs’ capability to perform trajectory recovery directly from the road network context, without external navigation tools.
- •
We provide extensive evaluation and analysis, showing that LLMs outperform specialized trajectory recovery models, generalize zero-shot, exhibit systematic regional and activity biases, and can serve as a foundation for preference-based navigation. We release the reproducible code for the experiment and our system together with the GlobalTrace dataset at [https://github.com/joey234/llm_traj_rec](https://github.com/joey234/llm_traj_rec).

## 2 Related Work

We covered a summary of existing works on trajectory recovery in the last section. Below, we outline the literature that studies the geospatial understanding capabilities of LLMs. A full discussion on the related works is included in [Appendix A](https://arxiv.org/html/2510.01639v1#A1).

Recent studies have probed whether large language models possess an implicit grasp of geographic space and spatial reasoning. Gurnee & Tegmark (2024) show that LLMs are able to map coordinates to corresponding regions. Manvi et al. (2024b) show that LLMs embed substantial geospatial knowledge (e.g., population density and mean income), but simply feeding geographic coordinates into prompts yields poor results on those questions.
The authors augment coordinate inputs with auxiliary map data (from OpenStreetMap), such as point-of-interest (POIs) around the coordinates,
and achieve large improvement over naive baselines. Manvi et al. (2024a) further extend this analysis, showing that LLMs exhibit systematic geographic biases (e.g., against lower-socioeconomic status regions).
MapEval (Dihan et al., 2025) is a benchmark of 700 multiple-choice, map-grounded questions (textual/API/visual) across 180 cities and 54 countries. Across 30 models tested, none exceeds 67% accuracy and all lag behind human performance, with particular difficulties in distance/direction inference, route planning, and visual map understanding.
Unlike previous task formulations of knowledge extraction, bias diagnostic, and multiple-choice question answering, we tackle trajectory recovery as a path reconstruction problem and investigate whether general-purpose LLMs can perform this in a zero-shot manner, without in-domain pretraining or extra supervision.

## 3 GlobalTrace

To enable comprehensive benchmarking, we curate a diverse dataset named GlobalTrace covering multiple modes of transportation and geographic regions. The dataset is derived from publicly available user traces on OpenStreetMap, collected via its official API (Overpass API, 2025)
across a predefined set of regions and cities (detailed in [Appendix B](https://arxiv.org/html/2510.01639v1#A2)). As these traces are voluntarily shared by contributors and contain no personally identifiable information (PII), their use raises no privacy concerns. We further constrain all trajectories to lie within a target length range of 500 m to 30,000 m to provide sufficient context for meaningful reconstruction.
To lessen the risk of data contamination, we only consider traces from 2024 onward.
Each trace is annotated with an activity type (i.e., mode of transportation) using keyword matching over the trace name and description, with an LLM-based classifier serving as a fallback to allow semantic-based mapping and for cases such as non-English text. For every trace, we generate two
masked variants by masking consecutive points: one with a small gap and another with a large gap, providing different difficulty levels. To facilitate different experiment settings, we partition the dataset into training, development, and test splits, stratified to balance both activity types and regional coverage. We provide the geographic coverage, activity type distribution, and route length statistics of GlobalTrace in [Figure 2](https://arxiv.org/html/2510.01639v1#S3.F2).

#### Problem Statement

Given a partial trajectory $T=\langle p_{1},p_{2},\ldots,p_{s-1},p_{s},p_{e},p_{e+1},\ldots,p_{|T|}\rangle$, where each point $p_{i}$ is represented by its coordinates (latitude and longitude), and there is a consecutive segment $G=\langle p^{*}_{1},p^{*}_{2},\ldots,p^{*}_{|G|}\rangle$ between $p_{s}$ and $p_{e}$ that is missing (i.e., masked), our *masked trajectory reconstruction* task aims to generate a series of points $R=\langle\hat{p}_{1},\hat{p}_{2},\ldots,\hat{p}_{|R|}\rangle$ that minimizes the *deviation* of $R$ from $G$. In [Section 5](https://arxiv.org/html/2510.01639v1#S5), we will detail how to measure the deviation.

Our masked trajectory reconstruction task departs significantly from two common setups in the literature. First, it is not a map-matching problem (Yin et al., 2018; Ren et al., 2021)
where the goal is to align noisy or low-quality GPS points to the underlying road network. In our problem, the coordinates to be recovered (i.e., $G$) are entirely withheld rather than available in degraded form. Second, our trajectory reconstruction problem is more challenging than point-wise recovery tasks (Si et al., 2024; Zhu et al., 2024), where single (or a few) missing points are inferred from its immediate neighbors.
Instead, we remove a contiguous segment of the trajectory, spanning hundreds to thousands of meters. A model must therefore reason over global context (movement context before and after the gap, activity type, and road network topology) to generate coherent continuation. This long-horizon prediction makes our problem substantially more challenging , and better reflects the complexity of real-world mobility data.

Figure: Figure 2: GlobalTrace dataset overview: (a) geographic coverage and trajectory-length statistics, (b) activity type distribution.
Refer to caption: x1.png

## 4 Methods

Traditional Transformer-based methods in trajectory reconstruction require a large amount of training data (e.g., 2.4M trajectories (Zhu et al., 2024)).
Moreover, such models fail to generalize beyond the region covered by their training data (typically just a city).
By contrast, we propose a framework that enables LLMs to perform navigation-style reasoning without relying on external routing engines or domain-specific training.

A naive solution is to prompt an LLM with the partial trajectory to be recovered, $T$, and the road network context of the region covering $T$. Empirically, we find that this naive solution fails to elicit the geospatial reasoning capabilities of LLMs, due to a large amount of unorganized data being fed to the LLM. We detail such alternative solutions in [Section 6.2](https://arxiv.org/html/2510.01639v1#S6.SS2).

To elicit the geospatial reasoning capabilities of LLMs, we adopt a two-stage, schema-constrained design: (1) Path selection over road and intersection IDs; and (2) Coordinate generation grounded by road geometries, as illustrated in [Figure 3](https://arxiv.org/html/2510.01639v1#S4.F3).
This two-stage design helps evaluates whether LLMs can reconstruct trajectories by reasoning over road network topology rather than delegating to external navigation engines.

Figure: Figure 3: Two-stage LLM trajectory reconstruction.
Refer to caption: x2.png

Our two-stage approach decouples the two distinct goals and allow fine-grained qualitative analysis: in the first stage, we exploit LLMs’ strong reasoning abilities to generate a verbose plan, while in the second stage, we instruct the models to strictly adhere to the plan by grounding against road geometries to generate precise coordinates (i.e., points in $R$).
Unlike prior work that either performs map-matching from noisy GPS points or predicts a single missing location given surrounding context, our formulation demands recovering an entire long masked segment of trajectory. This setup is significantly harder, as the model must maintain consistency across multiple steps, align with motion summaries, and adhere to road network constraints simultaneously.

#### Stage 1: Path Selection

This stage evaluates the ability of LLMs to read the road network surrounding the masked segment and perform planning to craft an appropriate path connecting the start and end points $p_{s}$ and $p_{e}$ with respect to the trajectory context.
We provide the following context to an LLM:
(1) Masked segment information: coordinates of $p_{s}$ and $p_{e}$, activity type, and masked segment length.
(2) Trajectory context summaries: timing, speed, and heading narratives before and after the masked segment, which provide more context to help models understand the movement.
(3) Road network slice: a road network enclosed by a bounding box surrounding the masked segment. Each road in the road network has ID, name, type, cardinal direction, and the list of roads that it connects to along with the corresponding intersection ID.
coordinates, and precomputed bearing to destination. We detailed the road network construction in [Appendix C](https://arxiv.org/html/2510.01639v1#A3) and compare different ways to represent it in our ablation study ([Section 6.2](https://arxiv.org/html/2510.01639v1#S6.SS2)).
We enforce further requirements for the navigation steps to make sure that LLMs produce unambiguous description with clear anchors (full prompt in [Appendix D](https://arxiv.org/html/2510.01639v1#A4)).

#### Stage 2: Coordinate Generation

Once a valid step-by-step plan is produced, the second stage requests explicit coordinates (points in $R$) for each step. We perform this iteratively for each step, grounded against the road network to ensure models generate valid coordinates that adhere to the roads. To be able to successfully generate coordinates, the LLM must (1) be able to align the generation to the provided road geometries, and (2) generate coordinates in the order that respect the moving direction
in the generated step descriptions. Please refer to [Appendix D](https://arxiv.org/html/2510.01639v1#A4) for an example of the prompt and response.

## 5 Experimental Setup

#### Models Evaluated

We consider three classes of models to benchmark against GlobalTrace. The settings and parameters are detailed in [Appendix E](https://arxiv.org/html/2510.01639v1#A5).

- 1.
Baselines. Traditional baselines for trajectory recovery. We consider: Linear interpolation (“Linear”), and Linear interpolation + HMM map-matching (“Liner+HMM”), following Yu et al. (2025b). We also consider a popular navigation system Google Maps. Although not designed for trajectory recovery, we convert the path generated by Google Maps Direction API (as polylines) into coordinates by decoding the Google’s Encoded Polyline Algorithm Format (Google Maps Platform,) for comparison with other methods.
- 2.
Pre-trained models. We compare with a strong pre-trained Transformer model for trajectory recovery: TrajFM (Lin et al., 2024). The model was pre-trained using the Didi dataset (Didi-Chuxing, 2018)
containing vehicle trajectories in two cities in China. We fine-tuned the model on our GlobalTrace training set.
- 3.
LLM models. We consider strong models from different model families, including three proprietary models: GPT-4.1, GPT-4.1-mini, and Claude 4 Sonnet, and four open-weight models: Deepseek V3, Llama 4 Maverick, Qwen3-235B, and Qwen3-30B.

#### Metrics

We focus on two evaluation metrics defined based on $G=\langle p^{*}_{1},p^{*}_{2},\ldots,p^{*}_{|G|}\rangle$, the ground-truth points of a masked segment, and $R=\langle\hat{p}_{1},\hat{p}_{2},\ldots,\hat{p}_{|R|}\rangle$, the reconstructed points.

*Mean Absolute Error*
(MAE; $\downarrow$), a commonly used metric in the trajectory recovery literature (Zhu et al., 2024), measures the average distance deviation (the the Haversine distance, denoted by $d(\cdot,\cdot)$) between the original trajectory points and their respective closest points on the reconstructed trajectory, normalized by the total masked segment length $L_{G}$.

$$ $\text{MAE}_{G\to R}=\frac{1}{|G|\cdot L_{G}}\sum_{i=1}^{|G|}\min_{j=1}^{|R|}d(p^{*}_{i},\hat{p}_{j})\cdot 100.$ (1) $$

To penalize spurious, off-route reconstruction, we also compute $\text{MAE}_{R\to G}$ that
measures the average distance deviation between the reconstructed trajectory points and their respective closest points on the ground-truth trajectory, expressed as a percentage of the reconstructed trajectory length $L_{R}$.
We combine $\text{MAE}_{G\to R}$ and $\text{MAE}_{R\to G}$ to calculate the harmonic mean.

$$ $\text{MAE}_{F1}=\frac{2\cdot\text{MAE}_{G\to R}\cdot\text{MAE}_{R\to G}}{\text{MAE}_{G\to R}+\text{MAE}_{R\to G}}.$ (2) $$

While commonly used, MAE is sub-optimal to measure the path reasoning capabilities of LLMs as it is a point-to-point metric, which can be affected when we have different point density between the ground truth and the generated trajectory. To address this, we propose a point-to-segment metric.

*Point-on-Trajectory* (PoT; $\uparrow$) measures the percentage of original trajectory points that fall within $\tau$ ($\tau=10$ by default) meters of the reconstructed trajectory. For each ground-truth point, we calculate the minimum distance to the reconstructed trajectory (considering all segments between any two consecutive points). We count the number of points within the $\tau$-meter tolerance. Higher PoT values indicate better ground-truth route coverage. This metric is less sensitive to minor positional errors, focuses on overall route adherence, and penalizes reconstructions that miss significant portions of the original trajectory.

$$ $\text{PoT}_{G\to R}=\frac{1}{|G|}\sum_{i=1}^{|G|}\mathbb{I}\!\left(\min_{j=1}^{|R|-1}d_{\text{seg}}(p^{*}_{i},\overline{\hat{p}_{j},\hat{p}_{j+1}})\leq\tau\right)\cdot 100.$ (3) $$

Here, $\mathbb{I}(\cdot)$ is the indicator function, and $d_{\text{seg}}(\cdot,\cdot)$ is the distance from a point to a line segment. Like before, we also compute $\text{PoT}_{R\to G}$ that measures the percentage of reconstructed trajectory points that fall within $\tau$ meters of the ground-truth trajectory. The harmonic mean, which we call the symmetric coverage score, is then calculated as:

$$ $\text{PoT}_{F1}=\frac{2\cdot\text{PoT}_{G\to R}\cdot\text{PoT}_{R\to G}}{\text{PoT}_{G\to R}+\text{PoT}_{R\to G}}.$ (4) $$

## 6 Main findings

### 6.1 RQ1: Can LLMs Perform Trajectory Recovery?

[Table 1](https://arxiv.org/html/2510.01639v1#S6.T1) reports the trajectory recovery results on the GlobalTrace test set.
Overall, we can see that, LLMs achieve strong results, on par with Google Maps, which is the state-of-the-art (SOTA) navigation system. Even Google Maps has a relatively low results for this task, showing that actual user trajectories do not always follow the optimal paths recommended by professional navigation service, especially for exploration type activities such as walking or hiking (cf. [Figure 6(b)](https://arxiv.org/html/2510.01639v1#A9.F6.sf2)).

Results of different LLMs tend to fall in similar range of 50-60%, with GPT-4.1 producing the best results, followed by Claude-4-Sonnet. The two smaller variants, GPT-4.1-mini and Qwen-3-30B, have worse results compared to their larger counterparts, indicating that geospatial reasoning capability emerges with scale.
The pre-trained model, TrajFM, on the other hand, performs the worst, and most of the time it just draws a straight line in small gap cases, or otherwise generate coordinates in a completely different region. This is expected as we ensure no overlap between the regions in the train/development and the test sets. It shows that generalizing to an unseen cities/region is not a trivial fine-tuning objective and would likely require large amount of training data across different regions in the world. We discuss more on the mismatch between the two metrics and show cases where LLMs perform better than Google Maps in [Appendix I](https://arxiv.org/html/2510.01639v1#A9), as well as the detailed results for each reconstruction direction ($G\rightarrow T$ and $T\rightarrow G$) in [Section H.1](https://arxiv.org/html/2510.01639v1#A8.SS1).

Table: Table 1: Reconstruction performance on the GlobalTrace test set. Best scores are in bold, second-best are underlined.

#### Stage-Based Analysis

As our method consists of two stages, we also perform analyses for each stage to assess their output quality. For path finding, we measure the path connectivity, i.e., whether the generated step-by-step navigation can form a valid path connecting the start and end points of the masked trajectory. We extract road segments from the generated step-by-step navigation and cross-check them against the road network graph to verify if all the elements are connected. As shown in [Table 2](https://arxiv.org/html/2510.01639v1#S6.T2), all tested LLMs show
above average scores, with GPT-4.1 being the strongest model (i.e., all road segments in the generated path are connected for 76.2% of the test instances), and Qwen-3-30B the weakest.

We also report the road network adherence (“Net. adh.”) score, which measures the proportion of the generated roads IDs and intersection IDs strictly presented in the road network. The scores for all LLMs tested
are almost perfect, showing almost zero hallucination in generating road IDs and intersection IDs adhering to the provided road network. The average number of steps (“Avg. # steps”) are between 3 and 5 for most models, while the average number of gaps (“Avg. # gaps”) longer than 200 m (which is considered a large gap) between steps is below 0.2, showing smooth transition between navigation steps. These results show that LLMs pose strong path finding capabilities. They can plan and reason over complex and large road network graphs to find valid paths.

For coordinate generation, we evaluate whether the generated coordinates for each step are consistent with the step description.
This includes geometry adherence (percentage of the generated coordinates presented in the road geometry, “Geo. adh.”), and bearing error (how far the directions of the generated coordinates deviate from the step descriptions, “Bearing”). To determine the direction of the coordinates for each step, we calculate the bearing using the start and end points for the step, then measure the error with respect to the direction (north, east, south, or west) in the step description. The bearing errors of all models are in the acceptable quadrant (i.e., $<90\degree$), with the Qwen and DeepSeek models having slightly larger deviation. The coordinate generation analysis demonstrates LLMs’ strong awareness of the GPS coordinate systems, i.e., the models can correctly generate coordinates following the correct direction of the selected road segments.

Overall, the fine-grained quality metrics have high correlation with the overall trajectory recovery performance, i.e., the models score consistently higher (or lower) for both evaluations.

Table: Table 2: Quality analysis for the two-stage LLM-based approach. Best scores are in bold, second-best are underlined. The metrics are defined in [Appendix G](https://arxiv.org/html/2510.01639v1#A7).

### 6.2 RQ2: What Road Network Context Is Optimal for Trajectory Recovery?

To determine the optimal amount of context, we need to balance information completeness with computational efficiency. While providing comprehensive road network data might seem beneficial, excessive context can overwhelm LLMs and degrade performance and significantly increase computational costs.
We conducted experiments on the development set with the following configurations ([Appendix F](https://arxiv.org/html/2510.01639v1#A6) includes example prompts for each configuration):

(1) No network: An end-to-end approach where LLMs are asked to generate the final coordinates to recover the missing segments.

(2) Raw network - Direct: We include the full road network in the bounding box surrounding the masked segment for direct coordinate generation (the naive solution described in [Section 4](https://arxiv.org/html/2510.01639v1#S4)).

(3) Raw network - Two-stage: We use the two-stage approach described in [Section 4](https://arxiv.org/html/2510.01639v1#S4), with the full road network in the bounding box surrounding the masked segment as above.

(4) Adjacent list - Two-stage:
Similar to (3), while to improve readability, we transform the retrieved road network into an adjacency list-based graph representation, where each road includes explicit connection information, along with relevant road metadata such as road types, oneway constraint.

(5) Topology-only - Two-stage: Similar to (4), and we remove all road geometry, retaining only topology information.

(6) Topology-only + Direction - Two-stage (current system): Similar to (5), and we pre-compute the direction for each road segment in the road network by calculating its bearing using the road geometry to provide models with cues about the direction of the road, helping models have a general sense of the roads to choose to move toward the destination.

**Table 3: Ablation result of GPT-4.1 on the GlobalTrace development set. Best scores are in bold, second-best are underlined.**
| Variant | $\text{PoT}_{F_{1}}$ $\uparrow$ | $\text{MAE}_{F_{1}}$ $\downarrow$ | Avg. # Tokens |
| --- | --- | --- | --- |
| No network | 39.9 | 13.0 | 780 |
| Raw network - Direct | 42.3 | 11.6 | 23 655 |
| Raw network - Two-stage | 45.2 | 11.3 | 25 947 |
| Adjacent list - Two-stage | 53.1 | 6.6 | 19 920 |
| Topology-only - Two-stage | 52.3 | 7.5 | 12 892 |
| Topology-only + Direction - Two-stage | 58.7 | 4.2 | 10 708 |

The results in [Table 3](https://arxiv.org/html/2510.01639v1#S6.T3) demonstrate a clear pattern where structured representations and explicit guidance significantly improve path finding performance. While the raw road network approaches struggled with information overload, using topology-only with direction guidance produces optimal performance by providing just the essential information LLMs need for effective path finding. The minimal performance difference between Adjacent list and Topology-only approaches suggests that geometric details beyond intersection points contribute little to path finding accuracy while substantially increasing computational overhead.

### 6.3 RQ3: Do LLMs Exhibit Geospatial Bias?

To further investigate potential biases, we breakdown the results based on activity types, regions, and training data cutoff date of LLMs.

Figure: Figure 4: Reconstruction performance ($\text{PoT}_{F_{1}}$) of all LLMs tested, breakdown by regions and activity types.
Refer to caption: x3.png

#### Regional Bias

From [Figure 4](https://arxiv.org/html/2510.01639v1#S6.F4), we can clearly see that the LLMs consistently perform worse on the Global South (Africa, Middle East, Oceania, South America, and South Asia) than they do on the Global North (Europe, North America, and Southeast Asia).
The discrepancy between the two is non-trivial (with up to 50% difference, e.g., Africa vs. North America for Llama, DeepSeek, and Qwen).
These results are unsurprising as most training data are concentrated to the western world, and LLMs would likely have more geospatial knowledge about those regions. One outlier is the abnormally high result of GPT-4.1 on South Asia. Most of those are hiking trails in Bhutan, which could potentially be a bias specific to this model. This finding also provides more context to a previous finding that LLMs are biased toward higher socioeconomic regions (Manvi et al., 2024a).

#### Activity Type Bias

Cycling, bus, and driving are among the the best-performing activity types. These structured transportation modes utilize well-defined infrastructure. For these modes, LLMs consistently perform better. For pedestrian activities, i.e., walking and hiking, LLMs show poorer performance, while trajectories of flying and boat activities have the worst reconstruction quality. This pattern is consistent across all LLM models, suggesting systematic biases rather than model-specific limitations. The performance hierarchy appears to correlate with infrastructure definition and predictability: activities following established routes (roads, railways, and bike paths) achieve better reconstruction than free-form movements (e.g., walking and hiking).

#### Data Contamination

We study the performance trend on traces before vs. after cutoff date for each model
and observe no significant difference between the two periods, showing that LLMs do possess geospatial reasoning capabilities beyond just memorization (detailed in [Appendix J](https://arxiv.org/html/2510.01639v1#A10)).

### 6.4 RQ4: Can LLMs Incorporate User Preference to the Navigation?

The finding that LLMs can comprehend road networks to construct paths offers new opportunities to enhance navigation experience beyond just finding the shortest or fastest routes between two locations.
We conduct a case study to explore such opportunities.

In practice, when in familiar areas, people usually choose paths that suit their personal preferences, e.g., safe, scenic, close to water, or going through shops and cafes.
There are several works that use POIs (e.g., landmarks, parks, and shops), environmental features (e.g., greenery, lighting, and street type) to plan travel (Ju et al., 2025; Wang et al., 2025; Yu et al., 2025a).
The survey by Siriaraya et al. (2020) introduces the SWEEP (Safety, Well-being, Exploration, Effort, Pleasure) taxonomy, outlining different qualities in pedestrian paths. Based on SWEEP,
we craft different scenarios, focusing mostly on the Exploration and Pleasure categories, embedding users’ preferences and the relevant POIs to the context provided to the model. Below, we demonstrate one specific manually-crafted scenario. Refer to the full prompts and other scenarios in [Appendix K](https://arxiv.org/html/2510.01639v1#A11).

Figure: Figure 5: Comparison of “Urban foodie” paths generated by our system (green) and Google Maps (black)
Refer to caption: melbourne_foodie.png

#### Urban Foodie

We select a scenario in Melbourne, famous for its diverse food scene. The start and end points are in two different ends of the city center, and the preference is a pedestrian path with many food options. The path generated by Google Maps is just a straightforward path that goes through the main streets with minimal detours. On the other hand, our system suggests a path that goes through alleys and lanes, where there could be more hidden gems and local options. Interestingly, the path managed to go through Hardware Lane, a famous alley for foods and drinks, despite this street was not retrieved as a POI. This show that the model can leverage its internal knowledge into route planning.

#### Additional Results

We also crafted the “First-time tourist” to show that our system works well in cases where Google Maps failed to suggest due to the lack of maps data, and “Waterfront cyclist” where model was able to suggest an alternative route that goes through more scenic POIs ([Appendix K](https://arxiv.org/html/2510.01639v1#A11)).

## 7 Conclusion

We introduced GlobalTrace, a benchmark for probing whether general-purpose LLMs can read road networks and reconstruct long masked trajectory segments without specialized training or external routing engines. Using a two-stage, schema-constrained framework, we showed that strong LLMs can produce valid, connected routes, outperforming specialized trajectory models and approaching the performance of an enterprise-grade navigation system.
Our analysis suggests that structured map topology is key to elicit these capabilities, and reveals persistent gaps and biases across regions and transportation modes.
Crucially, we also demonstrate that LLMs can go beyond shortest paths and flexibly integrate user preferences into route recommendation.
In a broader sense, these findings show that LLMs have huge potential to support applications in mobility, accessibility, and urban decision making.

While our results demonstrate that LLMs possess non-trivial geospatial reasoning capabilities, several limitations remain. First, trajectory recovery, while informative, captures only a narrow slice of geospatial understanding. Second, the definition of “ground truth” recoveries is itself imperfect. Our benchmark is constructed from user-generated GPS traces, which may be noisy and reflect individual rather than standard choices of movement.

#### Ethics Statement

This work involves the use of user-generated data, in the form of GPS traces. Those data are uploaded voluntarily and contain no personal identifiable information (PII).
One more concern is on sending those data to proprietary LLM APIs.
Again, as those traces are readily-available and allow crawling through the official API, there should not be any violation regarding the terms and agreements of OpenStreetMap.

#### Reproducibility Statement

To ensure full reproducibility and to facilitate future research, we release the full GlobalTrace dataset ([Section 3](https://arxiv.org/html/2510.01639v1#S3)) along with the code to replicate the findings in this work ([Section 6](https://arxiv.org/html/2510.01639v1#S6)) at [https://github.com/joey234/llm_traj_rec](https://github.com/joey234/llm_traj_rec). We also document the settings that were used for each model ([Appendix E](https://arxiv.org/html/2510.01639v1#A5)).

## References

- Cheng et al. (2024)
Jeffrey Cheng, Marc Marone, Orion Weller, Dawn Lawrie, Daniel Khashabi, and Benjamin Van Durme.
Dated data: Tracing knowledge cutoffs in large language models.
In *Proceedings of the First Conference on Language Modeling*, 2024.
- Didi-Chuxing (2018)
Didi-Chuxing.
GAIA Initiative.
[https://outreach.didichuxing.com](https://outreach.didichuxing.com), 2018.
Accessed: 2025-09-24.
- Dihan et al. (2025)
Mahir Labib Dihan, Md Tanvir Hassan, Md Tanvir Parvez, Md Hasebul Hasan, Md Almash Alam, Muhammad Aamir Cheema, Mohammed Eunus Ali, and Md Rizwan Parvez.
MapEval: A map-based evaluation of geo-spatial reasoning in foundation models.
In *Proceedings of the 42nd International Conference on Machine Learning (ICML)*, 2025.
- Duckham & Kulik (2003)
Matt Duckham and Lars Kulik.
“Simplest” Paths: Automated Route Selection for Navigation.
In *International Conference on Spatial Information Theory (COSIT)*, pp. 169–185, 2003.
- Golledge (1995)
Reginald G. Golledge.
Path selection and route preference in human navigation: A progress report.
In *International Conference on Spatial Information Theory (COSIT)*, pp. 207–222, 1995.
- (6)
Google Maps Platform.
Encoded polyline algorithm format.
[https://developers.google.com/maps/documentation/utilities/polylinealgorithm](https://developers.google.com/maps/documentation/utilities/polylinealgorithm).
Accessed: 2025-09-25.
- Gurnee & Tegmark (2024)
Wes Gurnee and Max Tegmark.
Language models represent space and time.
In *Proceedings of the 12th International Conference on Learning Representations (ICLR)*, 2024.
- Huang et al. (2018)
Xingyu Huang, Yong Li, Yue Wang, Xinlei Chen, Yu Xiao, and Lin Zhang.
CTS: A cellular-based trajectory tracking system with GPS-level accuracy.
*Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies*, 1(4):1–29, 2018.
- Ilyankou et al. (2024)
Ilya Ilyankou, Meihui Wang, Stefano Cavazzi, and James Haworth.
Quantifying geospatial in the common crawl corpus.
In *Proceedings of the 32nd ACM International Conference on Advances in Geographic Information Systems (SIGSPATIAL)*, pp. 585–588, 2024.
- Ju et al. (2025)
Chenlu Ju, Jiaxin Liu, Shobhit Sinha, Hao Xue, and Flora Salim.
Trajllm: A modular llm-enhanced agent-based framework for realistic human trajectory simulation.
In *Companion Proceedings of the ACM on Web Conference 2025*, pp. 2847–2850, 2025.
- Liang et al. (2025)
Yuxuan Liang, Haomin Wen, Yutong Xia, Ming Jin, Bin Yang, Flora Salim, Qingsong Wen, Shirui Pan, and Gao Cong.
Foundation models for spatio-temporal data science: A tutorial and survey.
In *Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2*, pp. 6063–6073, 2025.
- Lin et al. (2024)
Yan Lin, Tonglong Wei, Zeyu Zhou, Haomin Wen, Jilin Hu, Shengnan Guo, Youfang Lin, and Huaiyu Wan.
TrajFM: A vehicle trajectory foundation model for region and task transferability.
*arXiv preprint arXiv:2408.15251*, 2024.
- Manvi et al. (2024a)
Rohin Manvi, Samar Khanna, Marshall Burke, David Lobell, and Stefano Ermon.
Large language models are geographically biased.
In *Proceedings of the 41st International Conference on Machine Learning (ICML)*, 2024a.
- Manvi et al. (2024b)
Rohin Manvi, Samar Khanna, Gengchen Mai, Marshall Burke, David B. Lobell, and Stefano Ermon.
GeoLLM: Extracting geospatial knowledge from large language models.
In *Proceedings of the 12th International Conference on Learning Representations (ICLR)*, 2024b.
- Newson & Krumm (2009)
Paul Newson and John Krumm.
Hidden Markov map matching through noise and sparseness.
In *Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL)*, pp. 336–343, 2009.
- OpenStreetMap contributors (2017)
OpenStreetMap contributors.
Planet dump retrieved from https://planet.osm.org .
[https://www.openstreetmap.org](https://www.openstreetmap.org), 2017.
- Overpass API (2025)
Overpass API.
Overpass API – OpenStreetMap wiki.
[https://wiki.openstreetmap.org/wiki/Overpass_API](https://wiki.openstreetmap.org/wiki/Overpass_API), 2025.
Accessed: 2025-09-25.
- Ren et al. (2021)
Huimin Ren, Sijie Ruan, Yanhua Li, Jie Bao, Chuishi Meng, Ruiyuan Li, and Yu Zheng.
MTrajRec: Map-constrained trajectory recovery via seq2seq multi-task learning.
In *Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining*, pp. 1410–1419, 2021.
- Si et al. (2024)
Junjun Si, Jin Yang, Yang Xiang, Hanqiu Wang, Li Li, Rongqing Zhang, Bo Tu, and Xiangqun Chen.
TrajBERT: BERT-based trajectory recovery with spatial-temporal refinement for implicit sparse trajectories.
*IEEE Transactions on Mobile Computing*, 23(5):4849–4860, 2024.
- Siriaraya et al. (2020)
Panote Siriaraya, Yuanyuan Wang, Yihong Zhang, Shoko Wakamiya, Péter Jeszenszky, Yukiko Kawai, and Adam Jatowt.
Beyond the shortest route: A survey on quality-aware route navigation for pedestrians.
*IEEE Access*, 8:135569–135590, 2020.
- Song et al. (2017)
Xin Song, Yuanxin Ouyang, Bowen Du, Jingyuan Wang, and Zhang Xiong.
Recovering individual’s commute routes based on mobile phone data.
*Mobile Information Systems*, 2017(1):7653706, 2017.
- Vaswani et al. (2017)
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin.
Attention is all you need.
*Advances in Neural Information Processing Systems (NeurIPS)*, 30, 2017.
- Wang et al. (2025)
Kaimin Wang, Yuanzhe Shen, Changze Lv, Xiaoqing Zheng, and Xuan-Jing Huang.
TripTailor: A real-world benchmark for personalized travel planning.
In *Findings of the Association for Computational Linguistics: ACL 2025*, pp. 9705–9723, 2025.
- Yin et al. (2018)
Yifang Yin, Rajiv Ratn Shah, Guanfeng Wang, and Roger Zimmermann.
Feature-based map matching for low-sampling-rate GPS trajectories.
*ACM Transactions on Spatial Algorithms and Systems (TSAS)*, 4(2):1–24, 2018.
- Yu et al. (2025a)
Dazhou Yu, Riyang Bao, Gengchen Mai, and Liang Zhao.
Spatial-RAG: Spatial retrieval augmented generation for real-world spatial reasoning questions.
*arXiv preprint arXiv:2502.18470*, 2025a.
- Yu et al. (2025b)
Xie Yu, Jingyuan Wang, Yifan Yang, Qian Huang, and Ke Qu.
BIGCity: A universal spatiotemporal model for unified trajectory and traffic state data analysis.
In *Proceedings of the 41st IEEE International Conference on Data Engineering (ICDE)*, pp. 4455–4469, 2025b.
- Zheng et al. (2008)
Yu Zheng, Quannan Li, Yukun Chen, Xing Xie, and Wei-Ying Ma.
Understanding mobility based on GPS data.
In *Proceedings of the 10th International Conference on Ubiquitous Computing (UbiComp)*, pp. 312–321, 2008.
- Zhu et al. (2024)
Yuanshao Zhu, James Jianqiao Yu, Xiangyu Zhao, Xuetao Wei, and Yuxuan Liang.
UniTraj: Learning a universal trajectory foundation model from billion-scale worldwide traces.
*arXiv preprint arXiv:2411.03859*, 2024.

## Appendix A Full Discussion on Related Works

#### Trajectory recovery

Traditional trajectory recovery methods
have typically relied on probabilistic modeling and map-matching techniques to reconstruct missing points from noisy or sparse GPS traces.
More recent, there are efforts that construct foundation models for trajectories. Most notably, TrajBERT (Si et al., 2024) introduces a BERT-style Transformer for sparse
trajectories where location points are missing from poor GPS signal; it learns bidirectional temporal patterns with spatial-temporal refinement to recover missing points.
Following the same direction, TrajFM (Lin et al., 2024) builds a vehicle trajectory foundation model for region and task transferability. The paper propose to integrate spatial, temporal, and POIs, and uses trajectory masking/recovery to pre-train a standard Transformer model.
Instead of relying purely on coordinates-level data, BigCity (Yu et al., 2025b) merge trajectories with population-level traffic state data (e.g. flow, speed, density across road segments) to perform additional tasks like next-location prediction, and traffic forecasting.
Most of these works, however, focus on point recoveries (rather than a large continuous segment). Furthermore, the biggest bottleneck is generalizability, as models trained for a specific regions are not transferrable to other unseen regions. To the best of our knowledge, our work is the first to explore this by applying general-purpose LLMs to this task, with the focus on zero-shot capabilities without the need of training or additional supervision signals.

#### Geospatial understanding capabilities of LLMs

Recent studies have probed whether large language models possess an implicit grasp of geographic space and spatial reasoning. Gurnee & Tegmark (2024) show that LLMs are are able to map coordinate to corresponding regions. Manvi et al. (2024b) show that LLMs embed substantial geospatial knowledge (e.g., population density and mean income), but simply feeding geographic coordinates into prompts yields poor results on those questions.
The authors augment coordinate inputs with auxiliary map data (from OpenStreetMap), such as point-of-interest (POIs) around the coordinates,
which lead to large improvement over naive baselines. Manvi et al. (2024a) further extend this analysis, showing that LLMs exhibit systematic geographic biases (e.g., against lower-socioeconomic status regions).
MapEval (Dihan et al., 2025) is a benchmark of 700 multiple-choice, map-grounded questions (textual/API/visual) across 180 cities and 54 countries. Across 30 foundation models tested, none exceeds 67% accuracy and all lag behind human performance, with particular difficulties in distance/direction inference, route planning, and visual map understanding.
Unlike previous task formulations of knowledge extraction, bias diagnostic, and multiple-choice question answering, we tackle trajectory recovery as long-segment route reconstruction and investigate whether general-purpose LLMs can perform this in a zero-shot manner, without in-domain pretraining or extra supervision.

## Appendix B More details on GlobalTrace

We show the cities, activity types, and regions covered by GlobalTrace in [Table 4](https://arxiv.org/html/2510.01639v1#A2.T4) to [Table 6](https://arxiv.org/html/2510.01639v1#A2.T6).

Table: Table 4: Cities covered by GlobalTrace.

Table: Table 5: GlobalTrace dataset statistics by region and activity type.

**Table 6: GlobalTrace dataset coverage by geographical regions.**
| Geographical Region | Regions | Dev | Test | Train | Total Traces |
| --- | --- | --- | --- | --- | --- |
| Europe | 39 | 3 | 148 | 1896 | 2047 |
| East Asia | 5 | 60 | 41 | 344 | 445 |
| Southeast Asia | 5 | 260 | 203 | 0 | 463 |
| South Asia | 4 | 22 | 3 | 260 | 285 |
| Central Asia | 1 | 0 | 17 | 0 | 17 |
| Middle East | 2 | 88 | 30 | 0 | 118 |
| North America | 5 | 126 | 53 | 213 | 392 |
| South America | 5 | 0 | 3 | 217 | 220 |
| Africa | 2 | 0 | 2 | 0 | 2 |
| Oceania | 4 | 37 | 69 | 0 | 106 |
| Total | 72 | 596 | 569 | 2930 | 4095 |

## Appendix C More details in road network construction

We retrieve a road network surrounding the masked segment using the Overpass API (Overpass API, 2025). Given the start and end coordinates for the masked segment, we draw a rectangle around the straight line between the start and end points. To ensure enough coverage, we also expand this rectangle on all sides using a gap‑aware buffer in meters (150 m for small gap and 500 m for large gap). This yields a compact box that encloses the segment rather than a large city‑wide area. Inside this bounding box, we then retrieve relevant roads specific to the type of activity:

- •
Walking/Hiking: footway, pedestrian, path, steps, living_street, track, bridleway, road, residential, service, unclassified, tertiary, tertiary_link, secondary, secondary_link, primary, primary_link, cycleway, trunk, trunk_link
- •
Cycling: cycleway, path, living_street, track, residential, service, unclassified, tertiary, tertiary_link, secondary, secondary_link, primary, primary_link
- •
Driving/Bus: motorway, motorway_link, trunk, trunk_link, primary, primary_link, secondary, secondary_link, tertiary, tertiary_link, unclassified, residential, service
- •
Train: public_transport=station, railway=station, railway=subway_entrance, public_transport=platform/stop_position.
- •
Flying/Boat: No specific filtering

An example of a query for walking activity would look like:

Endpoint: [https://overpass-api.de/api/interpreter](https://overpass-api.de/api/interpreter)

Request:

## Appendix D Sample Prompts and Responses

#### Path Selection

#### Coordinate Generation

## Appendix E Model Details

#### Linear

Reconstruct the masked segment by connecting the segment’s start and end GPS coordinates with a straight line and sampling evenly along that line.

#### Linear + HMM

Produce a simple, connected candidate polyline via linear interpolation, then snap it to the road/path network using a standard HMM-based map-matching service. For this step, we call the OSRM’s HMM-based Map Matching API. The HMM combines (1) emission probabilities: likelihood of each observed point being near a road candidate (distance-based) and (2) transition probabilities: likelihood of moving between candidates given road graph connectivity and plausible travel costs; OSRM encapsulates this logic and returns a most-likely path (Viterbi).

#### TrajFM

We adapt the pre‑trained TrajFM to GlobalTrace as a masked coordinate recovery model by aligning data, projection, and supervision to the task. Concretely, we keep the TrajFM
architecture fixed and load pre‑trained weights, then project trajectories into metric space using per‑group UTM coordinates with the correct zone. We convert GlobalTrace supervision into the model’s tokenized format using a task‑specific padder that masks only spatial features at indices flagged by the dataset’s mask column, and finetune with the following settings:

**Table 7: Finetuning hyperparameters for TrajFM**
| Hyperparameter | Value |
| --- | --- |
| Optimizer | Adam (weight decay $1\times 10^{-6}$) |
| Learning rate | $5\times 10^{-5}$ |
| Epochs | 20 |
| Scheduler | CosineAnnealingLR ($T_{\text{max}}{=}20$, $\eta_{\min}{=}1\times 10^{-6}$) |
| Batch size | 16 |
| Loss weights | spatial=2.0, temporal=0.5, token=0.5 |
| Gradient clipping | max_norm=1.0 |
| Task/Padder | gt_mask (masked coordinate recovery) |

#### LLMs

Table: Table 8: LLM settings used in experiments: checkpoint, parameter size (if known), and training cutoff date.

## Appendix F More Details on Ablation Study

## Raw road network

## Adjacent list

## Topology only

## Appendix G More details on stage-based analysis metrics

#### Connectivity

We first extract all mentions of road ID from the generated step-by-step navigation (in order). Then using this ordered list, we will cross-check each consecutive road ID pair against the provided road network (already transformed into topology-only format with explicit connection defined). Then the Connectivity is just the percentage of pairs that are connected out of the total number of pairs.

#### Network adherence

Also extract all mentions of road ID from the generated step-by-step navigation. Using this list, we check against the provided road network to see if the IDs are actually exist. The score is just the percentage of valid road ID out of all the generated road IDs

#### Geometry adherence

Similar to road network adherence, but this metric compare whether the generate coordinates are actually presented in the provided road geometry. The score is then just the percentage of valid coordinates out of all the generated coordinates

#### Bearing

For each LLM-produced navigation step, we extract the intended cardinal direction (e.g., north, southeast) and map it to a canonical expected bearing in degrees: N=0, NE=45, E=90, SE=135, S=180, SW=225, W=270, NW=315. We then compute the actual bearing between the step’s start_point and end_point coordinates and normalize it to [0, 360). The per‑step bearing error is the circular angular distance between expected and actual:
$error=min(|expected-actual|,360-|expected-actual|)$.

## Appendix H Detailed results

### H.1 Results for each reconstruction direction

Table: Table 9: Combined metrics: F1 (symmetric) and directional (GT→REC, REC→GT) on test set.

Table: Table 10: Directional metrics: Small gap.

Table: Table 11: Directional metrics: Large gap.

A clear pattern shown in [Table 9](https://arxiv.org/html/2510.01639v1#A8.T9) to [Table 11](https://arxiv.org/html/2510.01639v1#A8.T11) is that many LLMs show high GT→REC and much lower REC→GT (e.g., good coverage of the ground-truth path but poor precision). Interpreting the directions:
GT→REC measures recall (how much of the GT path is covered by the reconstruction), while REC→GT measures precision (how much of the reconstructed path lies on or near the GT). So a
low REC→GT alongside a high GT→REC typically means the model covers most of the true route but also adds extra, off-route geometry—spurs, loops, or side streets—so a large share of
reconstructed points are not close to the GT.

### H.2 Activity results

We show the activite bias results in [Table 12](https://arxiv.org/html/2510.01639v1#A8.T12).

Table: Table 12: Activity bias results: mean PoT (%) by model and activity

### H.3 Regional results

We show the regional bias results in [Table 13](https://arxiv.org/html/2510.01639v1#A8.T13).

Table: Table 13: Regional bias results: mean PoT (%) by model and region.

## Appendix I Google Maps errors

The mismatch between PoT and MAE also reveals some interesting insights. For linear and our two-stage LLM-based solutions, we notice that the MAE scores are strong, while Google Maps clearly favors PoT.(^1^11Also the low value of MAE can be misleading as it is normalized by the masked segment length. For example, a 5% error for a 2 km masked segment is 100 m, which is actually quite high, while a 15% error of a 200 m masked segment is just 30 m.) We notice that Google Maps often takes realistic detours to adhere to road rules, which may not match well for free-form movements like walking or hiking.
[Figure 6](https://arxiv.org/html/2510.01639v1#A9.F6) shows an example.
In [Figure 6(a)](https://arxiv.org/html/2510.01639v1#A9.F6.sf1), Google Maps is constrained to follow the road direction (the black line), achieving a lower MAE to the user trajectory (dark green),
which takes a parallel segment that run in the other direction. This shows
PoT is more robust to evaluate those cases as the two trajectories are within the same corridor. Also, in areas without sufficient road network data, Google Maps may generate a different trajectory ([Figure 6(b)](https://arxiv.org/html/2510.01639v1#A9.F6.sf2),
or may not be able to generate any route at all (see [Appendix K](https://arxiv.org/html/2510.01639v1#A11))

Figure: (a)
Refer to caption: gmaps_mae.png

## Appendix J Cutoff analysis

Figure: Figure 7: Reconstruction performance (PoT) of pre- vs. post-cutoff date for each model. ns denotes a non-significance change based on an independent t-test. Given a more recent cutoff date, Claude-4-Sonnet was excluded as we cannot find enough samples for post-cutoff analysis.
Refer to caption: x4.png

LLMs training corpora contains a huge amount of geolocation data (Ilyankou et al., 2024), raising a concern of data leakage given their strong performance.
We observe the performance trend before vs. cutoff date for each model. To facilitate a fair comparison, we perform stratification to get a balance distribution in term of on activity types, regions, and masking strategies for the pre- and post-cutoff sets. Overall, we observe no significant different between the two periods ([Figure 7](https://arxiv.org/html/2510.01639v1#A10.F7)), where most models in fact have slightly better performance post cutoff date, showing that LLMs do posses geospatial reasoning capabilities rather than just memorization.
Regardless, given that all the traces are from 2024 onward, it is unlikely that data contamination is a concern as it has been shown that the LLMs have a much more distant effective knowledge cutoff date compared to the reported date (Cheng et al., 2024).

## Appendix K Preference demo

Figure: Figure 8: “First time tourist” routes comparison. Our systems route consists of steps with different colors, while Google Maps failed to suggest a valid pedestrian route.
Refer to caption: seoul_our.png

#### First‑time Tourist

Maximizes novel POIs coverage within distance/time; strings landmarks/markets/pedestrian areas; balances detours. For this scenario, we select a route between two popular tourist location in Seoul: From Myeongdong Shopping Street to N Seoul Tower. Google Maps was not able to suggest a walking route due to unavailable map data in this region while our system successfully construct a route going route various landmarks and popular tourist areas such as
along Myeongdong Shopping Street, Underground shopping centre, Namsan Park.

Figure: Figure 9: “Scenic cyclist” routes comparison. Our systems route consists of steps with different colors. The Google Maps route is in black.
Refer to caption: sydney_cycling.png

#### Scenic Cyclist

Continuous rivers/lakes/bay boardwalks and promenades; avoids trunk roads. For this scenario, we select a trace in Sydney CBD, where there are a combination of waterfronts and gardens. Our system route start along the waterfront, similar to Google Maps, but then make a detour through several cycleways, and connect through the Royal Botanic Garden before moving toward the destination.
Though this show that the generated route do follow user preference, our system output is not necessarily better compared to Google Maps.

## Appendix L Disclose of LLM usage

We use GPT-5 through the web interface ([https://chatgpt.com/](https://chatgpt.com/)) to aid in the writing of this paper, including generating latex commands to improve paper formatting and polish writing (grammar, collocation, rephrasing).

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: See weakness
soundness: 2
strengths: - Introduce a new task for geospatial reasoning via trajectory recovery
- Constructs a comprehensive dataset (GLOBALTRACE) across regions and transportation modes
- Design a prompts to tackle
- Comp[are with LLM and non LLM HMM, Google Maps, TrajFM, etc.)
- Clear metric design (MAE, PoT, F1 variants) with detailed ablations
summary: This paper explores whether LLMs can perform geospatial reasoning by reconstructing masked GPS trajectories from road network data. It introduces GLOBALTRACE, a diverse dataset of 4K real-world trajectories, and proposes a two-stage prompting framework for path selection and coordinate generation. Experiments show that strong LLMs outperform specialized trajectory models and approach Google Maps’ performance, revealing both reasoning ability and regional biases. The paper also showcases preference-aware navigation scenarios, highlighting new applications of LLM-based spatial reasoning.
weaknesses: - This work lacks the clear motivation and its impact, other than can LLM do this? And how is it distinct from path finding or routing or navigation given source and destination in term of consequence? 
- No visual or multimodal grounding—limited to text-based reasoning. While the application seems valuable, the current text form is not much, rather I think a multi-modal task would be more significant.  
- The benchmark is seems easy, not challenging, on par the Google maps-which limits its applicability. No clear motivation why to use LLM instead of these methods. Also High token/inference cost limits real-world scalability but there was no study analysis.
- No discussion on human performance as well. 
- Section seems not method rather evaluation protocol. 
- Data collection section with details is missing  
- For Preference aware path finding analyis, Popular path finding is a common task, why not sudy on that?
- It does not discuss why or how the two stage prompt is derived, neigher consider any other reasoning prompts. 
- I dont understand why the dataset is based on OpenStreetMap and the evaluation consider Google maps. 
- Bias discussion shallow—not linked to data imbalance quantitatively
- Missing robustness tests on noisy or incomplete maps or any such discussion
- Not much intuition on error corrections, why it was making the error and how to fix.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 3
questions: The paper addresses most of the relevant questions.
soundness: 3
strengths: - The paper proposes a new benchmark to assess geospatial reasoning capabilities of llms
- The benchmark is created with diverse range of trajectories from real world coordinates, and tries to reduce data imbalance and biasness.
- This can be a good addition to existing llm benchmark suites and provide a unified evaluation framework.
- The paper is clearly written and explains in detail the steps used for creating the benchmark and evaluating models.
summary: The paper explores to understand the geospatial reasoning capabilities of large language models. They propose a benchmark named GlobalTrace consisting of 4000 real-world trajectories, LLMs are tasked with reproducing masked segments of a trajectory i.e a sequence of longitude and latitude points using a two-stage task description. The evaluation shows that existing llms exhibit a high degree of geospatial awareness similar to existing algorithms such as those used by Google Maps.

The key contributions of this work is the design and release of a new benchmark to assess geospatial reasoning capabilities.
weaknesses: - The authors claim that there have not been previous works that evaluate the ability of llms to plan paths or read road networks or generate coordinates. This is false as there are multiple existing works in this domain such as (https://arxiv.org/pdf/2306.00020) who have already shown that llms contains a strong degree of groundness to real world data and can reason and plan using geospatial data. 
- The benchmark only evaluates the capability of llms in using their internalized knowledge and world coordinates. I am not sure how this might be useful in real world applications, as one can easily call an external api or service to convert between locations and coordinates. Moreover llms are still not reliable in generating very accurate and fine floating point values such as longitude and latitude coordinates, as exhibited by the high MAE scores. The benchmark can be more useful if it addresses how language agents with access to external tools might perform in this scenario.
- Being able to recover a trajectory does not guarantee a high degree of understanding or reasoning capability. It shows that the llm was trained with a lot of geospatial data and can retrieve that knowledge given the correct prompt. Tasks such as RQ4 are more relevant to understanding whether the llm can actually use the knowledge and apply it to solve other tasks such as personalized recommendation. I think the authors can improve the benchmark much more by designing tasks such as these that require multi step reasoning, rather just recalling geospatial coordinates from memory.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: *Why would people want to use LLMs for path planning when deterministic algorithms do this quite well?

*Does the level of effort required in setting up the prompts to generate outputs that LLMs can handle justify the effort, especially in light of the success of traditional trajectory planning approaches?

*How brittle is this approach in light of slight prompt variations?
soundness: 1
strengths: Using trajectories to test whether LLMs can spatially reason is a good idea.
summary: The authors attempt to show that LLMs can reconstruct trajectories to the same degree as traditional mapping platforms. This paper does too much though and the authors need to break the paper into different focus area.
weaknesses: *The prompts were extremely detailed and there were no prompt sensitivity analyses. The authors want to claim that they showed the LLMs were on par with Google Maps but the inputs were very different and the detailed prompts all but ensured the systems would perfir reasonably.

* The research questions were introduced in the results section, and were never motivated. 

*Not enough detail was provided on exactly what trajectories needed to be generated. Appendix B was too light on details.

* Results of different LLMs fell in the range of 50-60% in the best cases, this is barely better than flipping a coin so these results do no engender confidence.

*The authors claim to show their approach can incorporate user preferences, but just using a couple of examples is not sufficient, especially for their claims. And the authors fail to consider that humans do not consider trajectory reconstruction.The authors should focus on this study separately in another paper.

It was not clear why cycling, bus, and driving was included in the same study as pedestrian activities, such analyses necessarily require different map resolutions and none of this information was included. For the authors to make the claims they did, they need to include all information so people could replicate their studies.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 2
questions: How much time does the two-stage prompting framework take to recover one trajectory in practice?

Does this strong geometric prior in both stages simplify the task (geospatial reasoning) too much?

In Fig. 2a, the average “Full trajectory (km)” differs between the small- and large-gap subsets, although each trajectory is said to have both masking variants. Could authors clarify these?
soundness: 1
strengths: The introduced GLOBALTRACE benchmark is diverse and realistic.

The evaluation metrics go beyond standard MAE, providing more interpretable path-level measures such as PoT and PoTF1.

The paper is well-structured.
summary: This paper investigates whether LLM can perform geospatial reasoning by reconstructing masked segments of GPS trajectories from road network context. The paper proposes a two-stage prompting framework, path selection followed by coordinate generation. Experiments show that LLMs outperform some baselines, generalize zero-shot across regions, and demonstrate structured reasoning over road networks. Further analyses reveal regional and activity-type biases and show that LLMs can integrate user preferences into route planning.
weaknesses: The method focuses on prompting design, but does not report any inference or token cost. Since some configurations involve 10K~20K tokens per query.

In Stage 1, the model receives a pre-selected local road network, which already restricts the possible routes to a small area, which may reduce task difficulty and partly explain its high accuracy compared to TrajFM.

Stage 2 relies on explicit road geometries, which introduces a strong spatial prior and reduces the need for actual geographic reasoning.

The current small/large gap split provides only a coarse difficulty granularity: the large-gap range is broad, and longer masked segments would more effectively evaluate genuine geospatial reasoning. In addition, the paper does not report how many samples fall under each difficulty level, which would be important for assessing the balance and relative contribution of each subset to the overall results.