Enhancing Maritime Trajectory Forecasting via H3 Index and Causal Language Modelling (CLM)
Authors: Nicolas Drapier
Aladine Chetouani
Aurélien Chateigner
Venue: Accepted by TMLR ()

Abstract: The prediction of ship trajectories is a growing field of study in artificial intelligence. Traditional methods rely on the use of LSTM, GRU networks, and even Transformer architectures for the prediction of spatio-temporal series. This study proposes a viable alternative for predicting these trajectories using only GNSS positions. It considers this spatio-temporal problem as a natural language processing problem. The latitude/longitude coordinates of AIS messages are transformed into cell identifiers using the H3 index. Thanks to the pseudo-octal representation, it becomes easier for language models to learn the spatial hierarchy of the H3 index. The method is qualitatively compared to a classical Kalman filter and quantitatively to Seq2Seq and TrAISformer models. The Fréchet distance is introduced as the main evaluation metric for these comparisons. We show that it is possible to predict ship trajectories quite precisely up to 8 hours ahead with 30 minutes of context, using solely GNSS positions, without relying on any additional information such as speed, course, or external conditions — unlike many traditional methods. We demonstrate that this alternative works well enough to predict trajectories worldwide.

Published in Transactions on Machine Learning Research (03/2025)
Enhancing Maritime Trajectory Forecasting via H3 Index and Causal Language Modelling (CLM)
Nicolas Drapier*
racolus.drapier@@ass--mppp
PRISME Laboratory, SAS Impact
University of Orléans, Orléans, France
Aladine Chetouani
aladbbe..dclooumn@@@aiirrraat
PRISME Laboralory
Universily of Orléans, Orléans, France
Aurélien Chateigner
aurelien pnredeenchateiigrrrBaa---ppp
SAS Impact
Orléans, France
Reviewed on OpenReview: https: // openreview. forum? id= tIf96jy09f
Abstract
The prediction of ship trajectories is a growing field of study in artificial intelligence. Tradi- tional methods rely on the use of LSTM, GRU networks, and even Transformer architectures for the prediction of spatio-temporal series. This study proposes a viable alternative for predicting these trajectories using only GNSS positions. considers this spatio-temporal problem a natural language processing problem. The AIS messages are transformed into cell identifiers using the H3 index. Thanks to the pscudo- octal representation, it becomes easier for language models to learn the spatial hierarchy of the H3 index. The method is qualitatively compared to a classical Kalman filter and quantitatively to Seq2Seq and TrAlSformer models. The Fréchet distance is introduced as the main ovaluation metric for these comparisons. We show that it is possible to predict ship trajectories quite precisely up to 8 hours ahead with 30 minutes of context, using solely GNSS positions, without relying on any additional information such as speed, course, or ex- ternal conditions unlike many traditional hods. We demonstrate that this allernative works well enough to predict trajectories worldwide.
coordinates of
1 Introduction
Predicting ship trajectories is an essential task for maritime stakeholders, encompassing economic, security, and logistical considerations. Accurate trajectory prediction plays pivotal role in optimising shipping routes, ensuring maritime safety, and managing resources efficiently. However, this endeavour has posed several challenges due to the vast amount of trajectory data generated in real-time and the intricate interplay of spatial and temporal factors.
Traditionally, Long Short Term Memory (LSTM) (Hochreiter & Schmidhuber, 1997) and Gated Recurrent Units (GRU) (Cho et al., 2014) networks have been employed to model sequential and temporal data, and many researchers have tried to adapt these recurrent neural network (RNN) architectures to the spatio- temporal domain. While these RNN-based approaches have demonstrated success in various applications
"Principal author, Conceptualisation, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data
Curation, Writing original draft, Visualisation
WWriting review & editing, Visualisation, Supervision, Project administration
"Conceptualisation, Methodology, Validation, Formal analysis, Writing review editing, Visualisation, Supervision, Project administration

---

Published in Transactions on Machine Learning Research (03/2025)
(Connor et al., 1994; Assaad et al., 2008; Salinas et al., 2020; Khan et al., 2020), they typically neglect the crucial spatial component inherent in ship trajectories, such as the geographical coordinates and the intricate relationships between waypoints in a trajectory.
More recently, the introduction of the Transformer architecture by Vaswani et al. (2017) has sparked a revolution different fields, such as the Natural Language Processing (NLP) (Brown et al., 2020; Radford et al., 2019; Devlin et al., 2018) and time-series prediction (Nie et al., 2022; Zeng et al., 2022; Liu et al., 2023; Wu et al., 2022), democratising the development of state-of-tho-art models due to their popularity, scalability, and ease of use. Models like Informer (Zhou et al., 2021) and Autoformer (Wu et al., 2021) have shown exceptional prowess in capturing long-term dependencies and intricate relationships within temporal data, making them valuable tools in the realm of time-series prediction.
However, despite their successes in temporal modelling, these models based on the Transformer architecture have not been adoquately tailored to address the spatial nuances inherent in ship trajectorics, limiting their applicability in the maritime domain. This research effort seeks to bridge this gap by exploring novel approaches that combine the strengths of Transformer-base the potential to revolutionise ship trajectory prediction in real-time industrial applications. In doing so, we aim to address the limitations of existing models, making maritime operations more efficient, secure, and economically viable.
architectures with spatial awareness, offering
In this research, we adopt a Causal Language Model (CLM) due to its ability to effectively handle sequential dependencies, a crucial aspect of ship trajectory prediction. The reto-regressvve nature of CLMs aligns with the temporal progression of vessel movements, where each position is inherently influenced by prior locations. By leveraging the capacity of CLMs to predict future tokens based precoding ones, we ensure that the spatial and temporal continuity of trajectories is accurately captured.
We believe that combining Transformer- based models with better spatial representation can enhance the accuracy of ship trajectory prediction. In maritime trajectory prediction, accurately representing Global Navigation Satellite System (GNSS) coordinates is challenging due to the inherent precision of GNSS mea- surements. Even a minor error of one hundredth of degree in latitude consistently translates to a difference of approximately 1.11 kilometres, while in longitude, the impact varies considerably basod on Earth's loca- tion. For instance, at the equator, it approximates 1.11 kilometres, but at 80 longitude, it diminishes to around 200 metres. These variations lead to profound disparities in actual locations, making it difficult to increase the dimensionality of GNSS positions using an nutoencoder-basee approach.
In light of these considerations, we set out to explore an alternative solution: the discretisation method utilising Uber's Hexagonal Hierarchical Spatial Index (H3) (Uber, 2018). H3 is a geospatial indexing system that partitions the world into hexagonal cells. H3 cells are written to 64 bits and can be represented in hexadecimal format, providing creative solution to the challenges associated with the continuous represen- tation of GNSS coordinates. While the concept of partitioning trajectories into hexagonal cells has already been explored by Spadon et al. (2024), their approach does not strictly adhere to Uber's original H3 method. Instead, they propose a similar discretisation technique, which inspired us to refine the use of H3 for our purposes. Our central aim with this discretisation approach is to reconfigure a ship's trajectory from Auto- matic Identification System (AIS) messages (Organization, 2015) into a structured narrative, akin to the composition of words in a sentence. By doing so, we effectively counteract the potential detrimental impacts of minor coordinate inaccuracies, and we expect to benefit from the ability of Transformer archilectures to understand the relationships between tokens.
Given the limited usefulness of the initial 12 bits of the H3 cells for improving our understanding of space, we propose an innovative pseudo-octal format. This representation is a cornerstone of our study. By tailoring this and integrating it with a customised tokenisation scheme, we have constructed a tokeniser vocabulary of just 270 tokens. This streamlined encoding not only optimises the computational load but also facilitates the Transformer models' spatial comprehension. Together, this pseudo-octal representation and tokenisation allow the models to grasp the structure of ship trajectories, offering a significant ad advantage in spatial pattern recognition within our Transformer--assee architecture.
The Automatic Identificntion System an automatic tracking system used to monitor and locate ships in real time. It uses Very High Frequency (VHF) transceivers to transmit data such as the vessel's identifier, position, speed, and direction.
2

---

Published in Transactions on Machine Learning Research (03/2025)
To clarify the role of H3 in our method, it is important to note that while H3 plays crucial role in structuring our data, it is not the core innovation of our approach. H3 serves as a tool to add hierarchical spatial structure to the contimuous GNSS coordinates, enabling us to represent ship trajectories as sequences of tokens. This structured representation is essential for our CLM to process the data effectively. Our primary focus lies in leveraging Large Language Models (LLMs) to process time-series data by representing these sequences in 'natural' language with task-specific grammar, where the grammar is defined by the spatial structure provided by H3. This approach allows us to treat trajectory prediction as a language modelling task, where the model learns to prediet the future sequence based on the patterns and relationships it identifies in the data.
To analyse the results, we propose utilising the Fréchet distance (Fréchet, 1957). The rationale behind this choice lies in its ability to capture the similarity between two trajectories while considering both spatial proximity and sequential order of points. The Fréchet distance, by comparing the geometric similarity between the actual and predicted trajectories, serves as a stringent measure of prodictive fidelity.
In the forthcoming sections, we delve into the methodologies adopted, with primary emphasis the utilisation of the H3 index for spatial representation. We elaborate the rationale behind selecting H3 at zoom level of 10, which directly influences the gramularity of the discretised ship trajectories. This choice plays a critical role in the quality of predictions, as the level of precision in the H3 index determines the amount of zigzagging introduced into a trajectory after discretisation. A finer resolution could capture more detailed movements, but may also increase noise, whereas a coarser resolution might miss important spatial nuances.
Furthermore, present experimental results and validate our approach against existing methods, demon- strating the promise of integrating Transformer architectures with spatial awareness within the maritime domain.
2
Background
GNSS play a pivotal role in modern artificial intelligence, particularly in applications related to Geospatial Intelligence (GEOINT). GEOINT, as defined by the USA, refers to the analysis and exploitation of geospatial information to gain insights and support joint operations (Sharp, 2011; Clark, 2020). GNSS, which includes systems like GPS, GLONASS, and Galileo provides essential geospatial data that underpins these intelligence operations.
A complementary technology in the maritime domain is the Automatic Identification System (AIS) (Orga- nization, 2015), which is increasingly intertwined with GNSS systems. AIS transponders on ships broadcast their GNSS-derived positions, allowing for real-time tracking and identification. This integration has revo- lutionised maritime operations, enabling vessel tracking, collision avoidance, and efficient port management.
However, AIS is limited due inherent characteristies. The protocol's reliance on VHF radio transmissions limits its range, making it susceptible to signal loss in remote congested areas and not always accurate due to delays in transmitting the data. Additionally, while ATS data provides valuable information, it may not always reflect the full scope of a vessel's activities, particularly in scenarios involving illicit or evasive behaviour.
One of the fundamental challenges in utilising GNSS and ATS data is the open nature of the ocean. Unlike terrestrial environments, where fixed road networks and landmarks facilitate navigation, the vast and unpre- dictable expanse of the sea makes it difficult to determine a vessel's trajectory with absolute certainty. The dynamic and often erratic nature of maritime operations poses significant obstacles to generating accurate predictions.
In the field of natural language processing (NLP) research, Transformer architectures with progressively extended model capabilities are at the heart of research. They are originally designed for NLP applications, and they have undergone substantial improvements order to optimise their efficiency in the processing of sequential data. Recent progress in this line of rescarch has been directed towards improving their ability handle extended sequences efficiently (Xiong et al., 2021; Beltagy et al., 2020; Child et al., 2019; Press et al.,
3

---

Published in Transactions on Machine Learning Research (03/2025)
2021; Dai et al., 2019; Su et al., 2024), making transformers an appropriate choice for the complex analysis of large datasets on maritime trajectories.
3 Related Work
In recent years, the field of vessel trajectory prediction and maritime safety has witnessed siguificant devel- opments, driven by advances in artificial intelligence and the utilisation of AIS data. Researchers have been actively working towards enhancing the safety and efficiency of maritime systems, particularly in the context of collision avoidance.
A comprehensive survey by Tu et al. (2018) examines the exploitation of ATS data for intelligent maritime navigation. AIS, which tracks vessel movements through electronic data exchange, offers valuable insights for maritime safety, security, and efficiency. This survey explores various facets of AIS data sources and their applications, including traffic anomaly detection, route estimation, collision prediction, and path planning. It underscores the synergy between data and methodology in marine intelligence research, highlighting the importance of both components.
Although the utilisation of autoencoders in vessel trajectory prediction shows promise based theoretical merits, some concerns can be raised about their practical application. For instance, the study presented by Murray & Perera (2020) by Murray and Prasad Perera proposes a novel dual linear autoencoder method for predicting vessel trajectories using historical AIS data and unsupervised learning for trajectory clustering and classification. This technique offers several advantages, such as generating multiple possible trajectories and estimating positional uncertainties through latent distribution estimation. However, critics argue that the method may only provide accurate predictions for short time horizons, typically less than 30 minutes, given its reliance on single initial condition current position in the trajectory. Additionally, since the algorithm operates based on previously observed states, it might struggle to effectively process new, unseen data. Thus, further investigation into enhancing the adaptability and accuracy of autoencoder- approaches for longer-term vessel trajectory predictions is warranted.
To address the complexity of maritime traffic patterns and improve Maritime Situational Awareness, Pallotta et al. (2013) have introduced methodologies like Traffic Route Extraction and Anomaly Detection (TREAD). TREAD relies unsupervised and incremental learning extract maritime movement patterns from AIS data. It serves as the basis for automatic anomaly detection and trajectory projection, enabling more effective synthesis of relevant behaviours from raw data to support makking processes. Trajectory predictions are made by context-based tracking algorithms, using the direction of mean velocity and the series of trajectorios taken by other vessels of the same type. However, the trajectory of a fishing vessel is highly anarchic and follows the movements of shoals of fish. It very difficult. to model such behaviour by taking account of positions alone. The statistical model used is based on automatic knowledge discovery. We believe that its ability to prodict ship movements away from shipping routes is limited because the context of other ships at the same location is reduced non-existent.
In the context of increasing concerns about collision accidents in the shipping industry, an advanced ship tra- jectory prediction model has been proposed (Bao et al., 2022). This model combines a multi-head attention mechanism with bidirectional gate recurrent units (MHA-BiGRU) to enhance precision and real-time capa- bilities AlS-based ship trajectory prediction. It effectively retains long-term sequence information, filters and modifies historical data, and models associations between past and future trajectory states, ultimately leading to improved accuracy. But, the approach taken in this model raises certain concerns, particularly regarding the normalisation of longitude and latitude data. While this is a common practice in trajectory prediction models, it can lead to potential inaccuracics. Normalising these coordinates to a 0-1 scale can distort critical spatial relationships, especially near meridians and poles. This simplification might. also inadequately represent the real-world physical distance, as a degree's distance varies with latitude. Such distortions are crucial in the shipping industry, where prccision in trajectory prediction is paramount for collision avoidance. In 2022, the French Bureau d'enquêtes des évènements de mer (BEAMer) recorded 472 accidents, 68% of which were caused by fishing vessels (BEAMer, 2022).
4

---

Published in Transactions on Machine Learning Research (03/2025)
Urban mobility research has also contributed to the understanding of human transportation modes. Tra- jectoryNet (Jiang et al., 2017), neural network architecture for point-based trajectory classification using GPS traces, offers an innovative perspective. It embeds the original feature space into a higher-dimensional representation, enriches the feature space with segment-based information, and employs Maxout activations to enhance the predictive power of Recurrent Neural Networks (RNNs). TrajectoryNel achieves impressive classification accuracy when identifying various transportation modes from GPS traces. Significantly, we were particularly inspired by the application of NLP techniques in this context. The concept of embedding, commonly used in NLP to convert words into vector representations, resonated with us for its potential in our own research. We admired how this approach effectively captures the underlying semantics of continu- ous features, like speed, in varying contexts. This insight guided us in designing our experiment, where we aimed lo similarly interpret and represent the semantics of continuous geospatial features, providing a richer understanding of the data beyond its numerical value.
While the use of Automatic Identification System (AIS) data for predicting vessel trajectorics has seen sig- nificant advancements, such as the successful deployment of the TrATSformer model by Nguyen & Fablet (2024), however, there are number of areas for improvement regarding this approach. The TrAISformer model represents an innovative step forward by employing discrctised, multidimensional encoding of AIS data and introducing a distinctive loss function tailored to handle variability and multiple modes within ship- ping movement data. However, it is important to note that traditional spatial binning techniques (Nguyen et al., 2018) have proven effective for local entity analysis. Nevertheless, these methods face considerable challenges when attempting to scale up for global applications due to resource limitations. Therefore, while the TrATSformer model presents promising solution, further research is required to assess its efficacy under diverse operational conditions and evaluate its potential impact on computational resources.
A recent study by Spadon et al. (2024) introduces a novel approach to enhance vessel trajectory forecasting through the integration of probabilistic modeling and advanced neural network techniques. The authors made significant contributions to maritime safety through vessel trajectory forccasting. One koy innovation is the use of a probabilistic model that leverages historical AIS data to create Route Probability and Destination Probability Matrices. These matrices generate probable routes and destinations, adding valuable features to the vessel trajectory data. This approach significantly enhances the model's decision-making capacity, particularly for complex or curved paths, by reducing mean and median errors. The integration of these probabilistic features allows the model to distill generalized patterns and rules from the data, making it more robust and adaptable to various conditions.
Another notable aspect is the positional-aware attention mechanism, which prioritizes more recent time steps of the input sequence. This methodological focus ensures that recent data points have a greater impact on the learning process, enhancing the precision of the trajectory reconstruction task. By giving higher importance to the latest time steps, the model can capture muanced temporal patterns that improve its ayyecttor-fitttngg capabilities. This approach allows the model to navigate the complexity of maritime navigation more effectively, making it well-suited for real-time applications processing streaming AIS data.
The authors' methodology emphasises the significance of recent data points in the learning process. However, similar to the TrAISformer model (Nguyen & Fablet, 2024), the dataset used in this study is limited to a specific goographical region. Although the objective is not to generalisc across all global trajectorics, the application of a probabilistic model to enhance learning indicates that the transferability of this model to other regions may require recalculating the probability matrix.
While exploring the literature, we encountered several other methods for ship trajectory prediction (Yang et al., 2022; Li et al., 2024; Sørensen et al., 2022). However, these methods are not autoregressive and do not align with the specific scope and requirements of our study. As a result, they have been excluded from our detailed analysis and comparison.
These various projects represent a selection of research activities that have made significant contributions to the advancement of ship trajectory prediction, the use of AIS data and maritime safety. This research study builds on these foundations and presents a unique approach to GNSS coordinate discretisation and trajectory prediction using transformer architectures.
5

---

Published in Transactions on Machine Learning Research (03/2025)
4 Methodology
This section details the methodology described above. The cornerstone of our approach is the transformation of the traditional
coordinate system into a pseudo-octal format using the H3 index. On this basis, we deploy simplified version of the Mixtral 8x7B model specifically adapted for trajectory prodiction. The aim of this adaptation is to make the model accessible for exccution on consumer computers while minimising its resource requirements. To assess the accuracy our predictions, we introduce the use of the Fréchet distance, which serves as robust measure for evaluating the quality of predictions. The summary is given in the figure 1.
Bu20690047477
1e55 028435077777
9429690047711!
1055 028439677777
Start
Lait ng coordinates
H3 Transtormation
Paeudo-octal representation
[I. S.
Tokenisation
End
Lat/Ung coordinatess
H3 Transformtion
Prediction
[1.. 3... Mc. V.
Figure 1: Flowchart of the method presented throughout the article. Green rectangles represent trunsformation processes, yellow rectangles correspond to eramples.
4.1 H3 Index
The methodology section of this research paper delves into crucial decisions that form the basis of our innovative approach in maritime trajectory prediction. In this section, we explore the pivotal choice of utilising the H3 index developed by Uber as the cornerstone of our spatial representation.
Initially, we considered representing each spatial point using a 768 dimensional vector learned through an autoencoder - model renowned for its effectiveness in conventional contexts. However, we confronted substantial challenge unique to the realm of geographical coordinates. Even the slightest deviations in these coordinates translated into significant errors in the real-world domain, as previously discussed in the introduction. This limitation prompted us to look for a more robust solution.
The discretisation of a near-continuous space such as the GNSS coordinate system", through the use of the H3 index a rational choice for mitigating inaceuracies in geographical coordinates. By representing spatial data discrete hexagons, the H3 index ensures that small deviations in coordinates do not translate into significant errors. Each hexagon serves well-defined spatial unit, reducing the scnsitivity of measure- ments to minor fluctuations in GNSS data. This contrasis with continuous coordinate systems, where slight variations can lead to substantial discrepancies. Furthermore, the multiple resolution levels of H3 allow for adaptability across diffcrent spatial scales, ensuring both accuracy and computational efficiency. This approach not only alleviates the impact of measurement uncertainties but also enhances the robustness of spatial data representation, making it a practical solution for maritime trajectory prediction.
In addition to its robustness, the H3 index offers several advantages that align well with objectives. H3 represents geographical locations as hexagons, ensuring that any two neighbouring hexagons are always at
2 Although the CNSS coordinate system technically discrete, with smallest measurable unit of mm, refer "nearrcontimoous because this level of precision negligihle compared the scale of decimal degrees. For instance, first. digit in decimal degrees corresponds a precision of approximately 111 km (ISO 6709:2008/ 1:2009).

---

Published in Transactions on Machine Learning Research (03/2025)
1th
10
NOMAD
HUNTERS POINT
DISTRICT
EOR Drive
Holland
Coll Giee
GREEN
West
McCo
Poot
DE DOIVE
CHENATOWN
SIDE
CVIC CENTER
NEW YORK
CITY
Figure 2: Illustration of the hcxagons of the H3 index at different resolutions over Manhattan centred on the latitude/ latitude/longituuee longitude coordinates 40.73129, 40.73129,,73..9928, -73.99288. Resolution is shown in blue, resolution in red and resolution 8 in green.
absolute distance of 1, unlike the S2 representation system (Google, 2015). The geometric nature of hexagons makes them less deformable the poles, and H3 provides multiple resolution levels, making it adaptable various spatial scales. The 64-bit cell identifiers in H3 are highly efficient for handling extensive geographical data, aligning scamlessly with the demands of big data applications. Unlike alternative methods (Sahr et al., 2003), such as Geohash and Hexbin, H3 demonstrates reduced visual distortions when subjected to different map projections, ensuring the preservation of spatial fidelity.
In our research, we opted for a resolution level 10, which effectively covers an area of approximately 1.5 hectares. To put it in perspective, this resolution corresponds to a square with sides approximately 122 metres, roughly equivalent to the area of 1.4 football fields. This choice represents a. balanced trade-off between the number of cells and computational efficiency, facilitating the practical implementation of our methodology. It's important to note that pursuing greater precision in spatial representation could lead to diminishing returns, as GNSS inaccuracies could result in non-stop coordinate jumps, rendering excessively high resolutions impractical for our purposes.
To facilitate efficient representation and manipulation of H3 cells, we introduced the concept of a pscudo- octal representation. This involved discarding the initial 12 bits (Table in A), considering a fixed resolution mode, and encoding the basic cell as a 2-character hexadocimal string. Each resolution level was then encoded in octal form, ensuring a consistent and concise representation that streamlined subsequent processing steps. Detailed algorithms for this transformation and its inverse be found in algorithms 1 and 2.
The proposed algorithms for converting H3 cell identifiers to and from a pseudo-octal representation are crucial for optimising the spatial encoding process and addressing limitations in traditional representations. The default hexadecimal representation of the H3 index is inadequate for our use case, as it merely translates the cell identifier without encoding spatial characteristics. This makes cumbersome for our tokenisation process, as does not map directly to the hierarchical structure of H3 cells. Likewise, using the 64- binary representation would necessitate remely long sequences for each position, limiting the efficiency of

---

Published in Transactions on Machine Learning Research (03/2025)
trajectory predictions. In contrast, the pscudo-octal representation leverages the hierarchical nature of the H3 index, where each subsequent resolution is formed using a 7-cell aperture. By discarding the initial 12 bits and encoding each resolution level in octal form (except the base cell, which remains in hexadecimal), we achieve a compact and efficient representation aligned with the spatial hicrarchy of the H3 index. This encoding scheme particularly valuable for our application predicting maritime trajectories, where optimising sequence length enhances the model's ability predict trajectories accurately and efficiently.
Algorithm Couverts an H3 cell identifier to a pseudo-octal string representation.
a string representing the H3 cell identifier in hexadecimal format Ensure: A pseudo-octal string representation of the given H3 cell identifier h3 :: str to (nt((ell)
Require: cell
1 be < [-rrmal( >> (52 - 7), 02x) for from 42 down to 0 step -3 do
digit + ((f >> u)& + ORD("0")
append char(digit) end for return bc + join(l)
Example: 8439690647477Jf 1c55102643507777
Algorithm Converts a pseudo-octal cell identifier a hexadecimal string representation.
a string representing the H3 cell identifier pseudo-octal format Ensure: A hexadecimal string representation of the given pseudo-octal cell identifier
Require: cell
b 6 2]
value t (((1 << 7)/10) << (nnt 16)
for each E cell[2 do
value + (value << in)
end for return volue
Example: 1c551026435077777 8a39690b4777fff
Tokenisation plays pivotal role in managing this spatial representation within our text-based model. Each two-character base cell is treated as a single token, with every subsequent resolution level encoded as a distinct token. Since we use resolution 10, all tokens beyond this resolution (levels 11 to 15) are non-informative and set to (an empty octal value). While we could have removed these or tokenised them as a single token to reduce computational complexity, we opted to relain them in our methodology for now. This leaves open the possibility of revisiting the tokenisation in the future, should we choose to adjust the resolutions without modifying the algorithms.
In the following sections, we look at the practical implementation of these choices, showing their impact on improving the prediction of maritime trajectories. Our meticulous approach to spatial representation and tokenisation prepares the ground for the transformative application of Causal Language Modelling (CLM) in maritime trajectory prediction, a promising way to raise the accuracy and efficiency of models in this domain.
4.2 Model Architecture
In the pursuit of prodicting ship trajectorics, the sclection of an appropriate model architecture is crucial. This section delves inlo the rationale behind choosing a causal language model and explores the intricacies of the Mixtral8x7B framework. Causal models, with their auto-regressive nature, are particularly suited for tasks that demand sequential prediction, making them idcal for capturing the temporal and causal structure of the trajectories. This section will detail the specific configuration of our architecture, highlighting its capacity and flexibility in handling the complexities of maritime trajectory prediction.
8

---

Published in Transactions on Machine Learning Research (03/2025)
embedding
P Tensor
Norm
Attention
Tensor
x10
Tensor
Norm
Exports
MoE
E5
E6
E1
E2
Top-2
Tensor
Norm
Tensor
Linear Projection
Decocling
Figure 3: Model architecture of MiXTRAJ. We use the some base as Miatrol, with a Miature of Experts. The only major modification is the vocabulary size (270) as we built our own tokenizer work accordingly with the pseudo-octal representation. See table for details. Figure from MistrolKil (2023).
4.2.1 Rationale for Selecting a Causal Model
The choice of causal language model, also referred to as auto-regressive model, is grounded in its inherent ability to predict future tokens based solely preceding ones (Radford, 2018; Radford et al., 2019; Brown et al., 2020). This auto-regressive framework aligns naturally with many linguistic tasks, as language generation inherently sequential. In CLM, each word or token is predicted based on all the prior tokens, capturing the temporal and causal structure of language. This makes CLMs particularly effective for tasks where the generation of each token depends on the context provided by the previous ones (unidirectional, left-to-right in our case).
One of the primary strengths of causal models is their ability to handle long-range dependencies within a sequence. Given that language often exhibits dependencies between distant elements, CLMs are well-suited for capturing these relationships over extended sequences. As result, they can generate text that remains coherent and contextually appropriate over longer spans, which critical for many downstream applications like our case.
In the context of predicting ship trajectories, the use of a causal language model is particularly justified by the sequential nature of vessel movement, where each GNSS coordinate is dependent prior positions. By transforming each coordinate into pscudo-octal representation, we effectively discretise the spatial domain, capturing the hierarchical and spatial characteristics of each location. The aulo-regressive nature of the causal language model ensures that each subsequent position is predicted based on the previous sequence of H3 cells, maintaining consistency in the spatial path. This approach is particularly robust in scenarios where long- range dependencies must be captured- such as forecasting futurr movement over extended distances-whie the structured encoding facilitates the model's understanding of spatial continuity.

---

Published in Transactions on Machine Learning Research (03/2025)
4.2.2 Mixtral8x7B
Our research leverages the powerful architecture provided by the Mixtral8x7B model (Jiang et al., 2024). This model is based upon the Mistral 7B architecture (Jiang et al., 2023) and incorporates a Mixture of Experts approach (Shazecr et al., 2017; Zoph ct al., 2022), significantly enhancing its capacity to handle complex lasks. Each expert can focus on different part the pseudo-octal representation, which improves predictive accuracy by enabling more granular analysis.
The inclusion of sliding window mechanism further sets it apart, allowing for attention span of 2560 tokens feature partieularly beneficial when dealing with lengthy trajectories. In our specific case, this translates to an attention span of over 2 hours, providing the ability to capture intricate patterns in maritime movements. Additionally, the utilisation of Grouped Query Attention not only facilitates faster inference but also reduces cache size requirements, ensuring efficient model operation.
Additionally, Mixtral's integration with the HuggingFace Transformers library (Wolf, 2019) made it an appealing option for development. This compatibility simplifies the model's implementation and ensures that our solution is maintainable and extensible".
The complete architecture configuration for our application is defined as follows in the table and can be seen in the figure 3. This comprehensive architecture with 220M parameters is tailored to our maritime trajectory prediction task, offering the capacity and flexibility to handle the complexities of ship movement patterns.
Table 1: Mixtral8x7B Model Configuration
Parameter
Value
Vocabulary Size Hidden Size
270
1024
Sliding Window
256
Number of Hidden Layers
10
Number of Attention Heads
16
Number of Key-Value Value Heads
Number of Experts
Experts per Token
4.3 Dataset, hardware and training process
The model was trained on a machine equipped with an Intel Core 19-13900K processor, an NVIDIA GeForce RTX 4090 graphies card with 24GB of RAM Manjaro 23.1.3 operating system. The model imple- mented using the HuggingFace and the PyTorch frameworks.
4.3.1 Dataset and Preprocessing
The AIS data utilised in this study were collected over a period of one week in August 2023, within a defined geographical region spanning saiitude/longitude coordinates from (-84.88, 180.00; 90.00, 179.65) as shown in figure While the collection area covers wide range, the majority of the recorded vessel trajectories are concentrated in the North Sea, the English Channel, the Atlantic Ocean (along the French, Portuguese, and Spanish coastlines), and the Mediterranean Sea.
As highlighted in the introduction, ATS data are derived from GNSS coordinates, which may suffer from inaccuracies. Common issues encountered included erratic jumps in position, sudden "teleportations" of points, and multiple vessels sharing the same Maritime Mobile Service Identity (MMSI), despite the AIS
3 At the time of the revision of this article, more models have heen released, and have reconducted the experiments with these newer models.
10

---

Published in Transactions on Machine Learning Research (03/2025)
standard requiring MMSIs to be unique. While the protocol itself stipulates unique MMSI assignments, this is not strictly enforced, either technically or by software, leading to potential confusion in trajectory analysis.
To address these challenges, we employed several data cleaning and clustering techniques. The first two teanes-erratee points and teleportation were resolved using the Sonsity-Basee Spatial Clustering of Ap- plications with Noise (DBSCAN) algorithm, as described by Ester et al. (1996), with specífic optimisations implemented by Wang al. (2020). We set the DBSCAN epsilon parameter to 10, which filtered out points that did not belong to any meaningful trajectory, as isolated points were considered noise.
For the issue of duplicate MMSIs, a more intricate process was required. The data were first grouped by MMSI, and for each group, we implemented a systematic cleaning procedure. We began by removing duplicate positions and timestamps, ensuring only unique points remained. Additionally, any point where the Speed Over Ground (SOG) was recorded as zero was discarded, as such records are typically associated with stationary vessols.
Following this, the data were sorted chronologically, and a secondary DBSCAN with an epsilon value of 0.1 was applied. This allowed for clustering of spatially related points, with clusters representing coherent vessel trajectorics. Any points labelled as noise by the algorithm were excluded. Subsequently, we computed the centroid of each cluster, as well as the average distance from the cluster's 's points to this centroid. Clusters with an average point-to-centroid distance of less than 15 metres were considered artefacts of GNSS imprecision and were removed, as they likely represented stationary vessels (for example, a docked ship).
At this stage, the remaining clusters represented valid trajectories. To detect instances where multiple vessels might share the same MMSI, we examined the temporal gaps between consecutive points, assigning new group identifier whenever time difference exceeding five hours was detected. A unique hash was generated for each point based on the cluster and group number, facilitating the identification of distinct vessels using the same MMSL.
We further refined the trajectories by calculating the Haversine distance between consecutive points and esti- mating the vessel's actual speed, independent of the recorded SOG. Any points indicating a speed exceeding 100 km/h were flagged as 'teleportations", and the entire trajectory containing such points was discarded, based the actual hash. Consceutive clusters were then iteratively merged by comparing the final point of one cluster with the initial point of the next. If the transition was feasible based on the average speed of the last 10 points (within a +5% tolerance), the two clusters were combined into a single trajectory, with cluster B inheriting the hash and group number of cluster A.
Finally, a consolidated hash was generated for each trajectory, incorporating the MMSI, trajectory number, and individual cluster hash. Any resulting trajectorics with fewer than 10 points were excluded from further analysis to maintain data integrity and avoid bias from insufficiently sampled paths. To standardise the temporal resolution, we then applied a downsampling interval of one minute, ensuring consistency across all trajectories.
In addition to the trajectory refinement, the next step involved transforming the AIS points into spatially encoded identifiers using the H3 indexing system. As described earlier in the article, this transformation fol- lowed the procedures outlined in algorithms 1 and 2, effectively converting the vessel locations into hexagonal cells for more efficient spatial representation.
Once transformed, the tokenisation process was applied to prepare the data for trajectory forecasting. Fach unique trajectory was tokenised, with the resulting sequence split into chunks of size 2560 whenever a trajectory excceded the context size of the model. Overlapping tokens were preserved, and padding was introduced with an attention mask set to zero. This approach allowed us to handle large datasets without compromising model accuracy. The final dataset comprises 1,731,686 unique trajectories spread across approximately 32 million H3 cells. After tokenisation, it consists of billion tokens.
'Given that the theoretical maximum speed is 188.904 km/h (meaning that the vessel helicopter is travelling at or above this speed).
11

---

Published in Transactions on Machine Learning Research (03/2025)
This multi-step approach, incorporating spatial clustering, temporal segmentation, H3 transformation, to- kenisation, and trajectory filtering, successfully mitigated issues of duplicate MMSIs and erroneous data points, ensuring a high-quality dataset for subsequent analysis and forecasting.
(a)
(b)
Figure 4: Comparison of maritime trajectories before and after dataset cleaning. (a) Before cleaning, the trajectories exhibit numerous errors, with irregular jumps crossing over land and a high concen- tration of anomalies covering Europe. These errors indicate poor capture of the actual ship movements. (b) After applying our cleaning procedure, the trajeclories are corrected and closely follow the marilime contours of the continents, removing artefacts and errors that cross landmasses. This process ensures more accurate representation of the actual maritime routes.
4.3.2 Training and Validation
The training of the model took 175,000 steps using an Adam optimiser with an initial learning rate of 5 times 10-4 and a gradient accumulation of 8 (see figure 11 in appendix B for the learning curves).
The entire captured dataset has been used for training and validation (respectively 85% and 15% of the dataset). The testing phase was carried out using another dataset that we captured on the fly during 3 days in February 2024 and represents approximately 15,000 cleaned trajectories.
4.4 Evaluation metric
To further enhance the evaluation of our trajectory prediction models, we employ the Fréchet distance key metric given by the equation 1. This choice is crucial in addressing the limitations of traditional error metrics such as Mean Absolute Error (MAF) or Mean Squared Frror (MSE) (Nguyen & Fablet, 2024; Spadon et al., 2024), which compare points in isolation and assume strict correspondence between them, potentially overlooking the overall shape and progression of the trajectory. The Fréchet distance, by contrast, evaluates the entire trajectory, accounting for both the geometry and the order of points, making better suited for spatial and temporal analysis. This allows us to capture the continuity and structure of maritime movements more effectively, providing more comprehensive assessment of model performance as shown in figure 5. We implemented the algorithm using the Haversine function, method commonly employed for calculating distances between points on the Earth's surface due to its accuracy in spherical geometry.
Consider two curves and B in the metric space S. The Fréchet distance between these curves is the infimum of the maximum distances between points on A and B, mapped through reparametrisations a and B of the interval [0, 1). This distance measures the dissimilarity between A and B by evaluating the maximal spatial discrepancy across all possible point correspondences established by a and B. The metric d in S is the Haversine distance, delineated in equation 2.
12

---

Published in Transactions on Machine Learning Research (03/2025)
Ground truth
Prediction example
MAE 49.928km
4008.800kML 810km²
FD 113.057km
170
Ground truth distance 596.290km
Figure 5: Comparison of the Fréchet distance versus Mean Absolute Error (MAE) and Mean Squared Error (MSE). Over distance of 596 km, the predicted trajectory deviates significantly from the ground truth, making such prediction unacceplable. The MAF gives the lowest error, but this value is misleading as it does not capture the true nature of the deviation, being too low due to its point by-point comparison. In contrast, the Fréchet distance accounts for the overall shape of the trajectories, providing a higher and more realistic measure of enror.
F)A, inf max o ((((O))). a,8 rE
(1)
where the distance function of S defined in 2.
Li P2-21
L2 A=-A1
d:
P1, P2, A1, Xu + 2r Bracssin (Vsin* (L1) + cos((21) (os((p2) sin" (1))
(2)
[-90; 90] 90 90] |- 180; 180] x [-180; 180] I+
where r is the Earth's radius, (o1 and 622 are the latitudes, A1 and A2 are the longitudes.
4.5 Qualitative and Quantitative Evaluation of Trajectory Predictions
To the best of our knowledge, there is currently no method capable of predicting ship trajectorics globally and over long distances without being tailored to specific geographic regions. Most existing approaches are limited to particular areas, restricting their applicability on a broader scale. In contrast, we sought to develop a solution that not constrainod by these regional limitations.
For the qualitative evaluation on test dataset, have chosen to compare our approach with Kalman filters (Kalman, 1960). This widely used method is known for its adaptability in trajectory prediction, yet it still requires regional adjustments, making it apt baseline to highlight the advantages of our model. In our implementation, only GPS positions positions-longituude (lon) and latitude (lat)-are used to predict trajectories. Each prediction is represented by a matrix G E xR=x2 where 7 is the length of the prediction. To align with our approach, which relies solely on positional data, we restricted the state in the Kalman filter to include only four values: longitude (lon), latitude (lat), and their respective velocities (Vion and Utae). Although Extended Kalman filters can incorporate more complex data such as sea current patterns, we opted roduce dependency external data by limiting the parameters of both our model and the Kalman filter. Since the Kalman filter predicts a single state and relies real observations in ils update phase, and real observations are not yet available in our maritime case, we modified the Kalman filter to operate in an auto-rogressive mode over т steps. In this sctup, cach predicted state becomes the subsequent input in the prediction cycle, as expressed equation 3.
6 FAJR-1
(3)
PAJ-11 = FR- Q
13

---

Published in Transactions on Machine Learning Research (03/2025)
The state is vector z (lat, lon, Viat, Vlon). The transition matrix Fk E R4x4 the time between two points, in our case 60 seconds:
x4 is as follows considering A.
1
0
4
0
0
1
0
Ar
FE
0
0
1
0
0
0
0
1
We define Qu to be the process noise covariance matrix as an identity matrix R multiplied by factor of 1 x 10-3 and PkIk-1 to be the state covariance matrix as an identity matrix R4x4,
Additionally, we conducted a quantitative comparison with TrATSformer and Seq2Seq Sutskever et al. (2014), introduced in earlier sections as a state-of-the-art reference. To ensure consistency, we reproduced the TrAISformer experiment ourselves using the code provided by the original authors. However, we modified the evaluation metric to the Fréchet distance, discussed previously, and used TrAtSformer's evaluation evaluation dataset to mcasure the performance of model, MixTRAJ, both with and without finc-tuning.
5
Results and discussions
5.1
Metrics analysis
Our analysis the predictive accuracy of ship trajectory models shown in the table 2, centred on the Fréchet distance as key metric, reveals nuanced insights into the model's performance in real-world navigation scenarios. By applying a stringent selection criterion, we isolated trajectories where the model's predictions were not only completed but also exceeded a prediction length of rj minutes, observed in our prior analysis and where 7) med(x)) 5, with med(X)) the median of the series of prediction minutes X. This filtration yielded dataset conducive to a focused analysis on the effectiveness of our predictive model under conditions of substantial navigationa complexity.
The violin plots in figure depict the distribution of two metrics used to evaluate the performance of our model predicting ship trajectories: the prediction distance and the Fréchet distance. Sub-figure (a) shows the distribution of the prediction distance (in metres), calculated as the straight-line distance between the last point of the context and the last point of the predicted trajectory. With mean of approximately 29,790 metres and density peak around 32,401 metres, this value represents the average distance covered in 90 minutes of prediction. This metric helps to contextualise the model's output by quantifying the length of trajectory being predicted over time, as our model outputs tokens only.
In sub-figure (b), the Fréchet distance used to compare the predicted trajectory with the actual one. Fréchet distance measures the similarity between the full predicted and actual paths, capturing the accuracy of the model over the entire trajectory rather than just the endpoints. This distance calculated using the closest ground-truth point to the centroid formed by the N predictions in given context. The mean Fréchet distance is 2,806 metres, with median of 1,938 retres and a density peak at. 1,061 metres. The proximity of the density peaks in both sub-figures suggests that the model performs consistently, with a typical prediction error of around kilometre for an overall predicted distance of 32 kilometres. This demonstrated that the model effectively captures the global patterns of ship movement under most conditions (see appendix C and D).
The long tails in both plots highlight instances where the model's predictions deviate more significantly from the actual trajectories. Such deviations could arise from unpredictable environmental factors (like sudden weather changes), complex maritime traffic conditions, or unique navigational challenges not fully encapsulated by the model's parameters such as the trajectories of fishing boats, which are difficult to predict by taking only GNSS positions into account. These cases underscore the challenges of modelling the intricacies of real-world navigation, where unforeseen variables can impact a ship's course.
The general effectiveness of the model in wide range of conditions is a strong proof of its utility in maritime operations, offering a robust tool for route planning, risk assessment, and decision-makingg support.
14

---

Published in Transactions on Machine Learning Research (03/2025)
Mean, median and density peak for a 60-minute contextual window and 90-minute prediction.
Prediction distance without outliers
Fréchet distance without outliers
Values: Mean 0022999000 89
Values:
Mean 2806. 44
Median = -30698.90
Median 1938
Density 32401.89
Density Peak 1061.
10000
20000 30000
40000
50000
60000
2000
4000
6000
8000
10000
(a) Prediction distance (meters)
(b) distance (meters)
Figure 6: Violin plot of the prediction distance. Without oulliers, highlighting the mean (green star, indicated by the green arrow head), median, and density peak for a context length of 60 minates and a prediction length of 90 minutes. See figures 14, 15 and 16 in E for the full analysis.
However, the instances of significant prediction error reflected in the higher Fréchet distances signal the need for caution. They suggest that reliance the model should be balanced with situational awareness and potentially supplemented with real-time data and human expertise, especially in navigating complex unfamiliar waters. By reducing instances of high Fréchet distances, we would improve the model's reliability across a wider array of navigational scenarios.
Table 2: Relative deviation between predictions and ground truth for mean, median and peak density. The context is expressed in minutes, while the prediction is expressed in the number of tokens generated. It can seen that for a doubled distance, the relative error is approximately doubled, which suggests that the model has generalised sufficiently to produce linear error.
Prediction
2560
5120 (x2)
7680 (x3)
Context
Mean
30 (540 tokens)
11.46%
18.30%
(x1.60)
22.89% (x2.00)
60 (1080 tokens)
9.42%
18.84%
(x2.00)
25.61% (x2.71)
100 (1800 tokens)
6.97%
14.78%
(x2.12)
23.33%
(x3.34)
Median
30 (540 tokens)
7.80%
13.23%
(x1.70)
17.54%
(x2.24)
60 (1080 tokens)
6.31%
13.67%
(x2.16)
19.46%
(x3.08)
100 (1800 tokens)
4.88%
11.33%
(x2.32)
17.82%
(x3.65)
Density Peak
30 (540 tokens)
4.51%
5.95%
(x1.32)
10.33%
(x2.29)
60 (1080 tokens)
3.27%
6.51%
(x1.99)
7.98%
(x2.44)
100 (1800 tokens)
2.67%
6.86%
(x2.57)
8.55%
(x3.20)
The figure highlights that the prediction error generally increases with the prediction distance, as shown by the comparison between (a) and (b). It. is generally observed that the prediction errors greater
15

---

Published in Transactions on Machine Learning Research (03/2025)
Comparison trajectory prediction methods using public AIS data from TrAISformer. The evaluation metric used is the Fréchet distance (in meters). The Seq2Seq models (LSTM and GRU) share the same architecture and were trained on TrAlSfommer data. TrAISformer was implemented using the source code provided by the original authors.
Table 3:
MixTRAJ models were evaluated in three different scenarios:
Few-Shot: Trained on private dataset, where 2% of the geographical area overlaps with the TrAISformer dataset for a duration of one week, and evaluated on the TrAISformer dataset.
Fine-Tuned: Trained directly on the public TrAlSformer datasct, and evaluated on the TrAISformer dataset.
Zero-Shot: Evaluated on private dataset from the Singapore Strait, without prior exposure to this region.
Models
Parameters
FLOPS (G)
MACs (G)
Error (2h)
Seq2Seq (LSTM ours) (Sutskever et al., 2014)
967k
0.2
9 X 10
15033.4 4
Seq2Seq (GRU ours) (Sutskever et al., 2014)
730k
0.1
9 x 10-5
6364.0
TrAISformer (Nguyen & Fablet, 2024)
57.5M
13.7
6.8
6945.2
MixTRAJ with finetuning (ours)
221M
4044.1
2018.8
2528.4
MixTRAJ (Few-Shot task, ours)
221M
4044.1
2018.8
5407.6
MixTRAJ (Zero-Shot task, ours)
221M
4044.1
2018.8
6198.7
(a)
(b)
Figure 7: Path prediction performance comparison. This graph compares trajectory prediction per- formance for different configurations of context windows and prediction distances. (a) 60 minutes of contert with a prediction at 90 minules, (b) 100 minules of context with a prediction al 90 minules and (c) 60 min- utes of contezt with a prediction at 20 hours. The colours represent the contezt window (red), the trajectories predicted by the model (green) and the actual trajectory (blue). We can see zigzags between the points due the geometric arrangement of the hexogons.
(a) and (b) are the same scale, (c) is zoomed out.
the context length is reduced, and the prediction length increased. On the other hand, we can see that, despite certain hallucinations which we will discuss later, the model seems have generalised the nature of trajectory, shown in image (c) of figure 17 in appendix F.
We compare model to the Kalman filter the same dataset. However, did not apply the H3 transformation for the Kalman pass, as the filter is then less sensitive to trajectory zigzags. We changed the colour of the predictions to yellow for the Kalman filter in order to differentiate them from the green predictions of our model.
The figure 8 depicts the results of the Kalman filter. We notice that the Kalman filter appears draw straight vector from the last known position. As we know, our auto-regressive Kalman filter does not take into account the history of the ship and Kalman filter is not designed to know the trends of other
16

---

Published in Transactions on Machine Learning Research (03/2025)
(a)
15 km
Upnoitsher (b)
15 km
(c)
15 km
Figure 8: Context (red), ground-truth (blue) and prediction (yellow) using a Kalman filter. The figure shows that the Kalman filters, in our case, predict linear trajectories becouse they auto- regressive, and they are not very sophisliculed (e.g. inlegration of headings and marine drift models). This seems to work very well when ship is going straight along its route (a, b), but not so well when there are sharp bends or changes of direction (c).
ships. We are trying to demonstrate here that the Kalman filter is not totally reliable indicator for ship because it has no knowledge of the ship's past, the ship's current trend or the shipping trend of other ships. However, we can qualify this by considering that a Kalman filter can be good and highly sufficient on a defined maritime route (for example for cargo ships and container ships) because these ships have straight trajectorics.
In contrast, shown in Table 3, MixTRAJ demonstrates a substantial enhancement in predictive accuracy, outperforming the baseline TrAlSformer model by 28.4% without fine-tuning and by an impressive 174.7% with fine-tuning on TrAffformer's evaluation dataset. This result highlights that, TrAISformer's data and despite the limited representation of this specific region in our initial dataset, our baseline model performs better, suggesting it has successfully generalised the intrinsic nature of trajectory patterns. Notably, we opted to use the median error for model ovaluation, whereas the initial TrAISformer implementation used the minimum error; we adapted this to ensure a more robust assessment.
without finc-tuning
Additionally, we evaluated our model through Zero-Shot task in a region unseen before (table 3). This region is a zoomed-in view of the Singapore Strait, from (15.44, 92.91) 9 to (-9.57, 128.03) in latitude/longituud coordinates. We observed that our model performs relatively well in this new context. This Shot evaluation underscores the model's ability to generalize to new and unseen regions while also indicating the importance of fine-tuning for achieving the highest accuracy. It remains the most effective approach to adapt to the specific characteristics of a region that influence trajectory patterns.
5.2 Trajectory density
In our study, we aim to assess the veracity of ship trajectories by generating a set of thirty predictive paths. The primary objective is to evaluate the accuracy of these trajectorics by examining the proximity of the centroid of polygon, formed by the endpoints of the generated trajectories, to the actual ground truth position of the ship. This method provides an estimate of the reliability of the predictions made by our model.
We have observed two distinct scenarios in the outcomes of our analysis. In the first scenario depicted in figure 9, significant majority of the trajectories coincide, indicating high likelihood that these paths accurately reflect the ship's movement. This consensus among the trajeclories suggests a precise and reliable prediction. In the second scenario, the generated trajectories vary widely, resulting in what could be described as chaos (figure 10, left). However, this dispersion does not necessarily indicate flaw in the model but rather a multitude of possible paths the ship could take. In such cases, further analysis and additional context are required to ascertain the exact trajectory of the ship.
17

---

Published in Transactions on Machine Learning Research (03/2025)
Lastly, it is crucial to highlight that some trajectories generated by model are physically unfeasible for ship to execute. We have coined the term "hallucinations" to describe these unachievable trajectories, concept further elaborated in the following section of our paper. This distinction aids in refining the accuracy of our model by eliminating non-viable predictions and focusing on plausible trajectories.
Figure 9: Illustration of the trajectory consensus phenomenon. This figure shows three ezamples of sets of trajectories where between 85 and 95% of the predicted trajectories align closely, representing high degree of agreement between the model's predictions. Each set demonstrates the model's ability to accurately predict ship motion, highlighting the reliability of consensus trajectories in reflecting the actual navigation path.
5.3 Hallucination
In the development of this predictive model for ship trajectory forecasting, an interesting phenomenon that we called "hallucination" has been observed. This term used to describe generated trajectories by the model that are not feasible for a ship to execute in reality. Examples of such unrealistic trajectories include abrupt U-turns, light turns, and improbable GPS jumps, which are highly unlikely in our context. These anomalies can be visually identified in the model's output, as showcased in the subsequent figure 10.
Figure 10: An example of the phenomena we have called "hallucinations" A red line represents the contert used lo generale the green trajectory. A blue line represends the ground fruth. See more ezample in the figures 18 and 19 in appendiz G
One of the key reasons behind the occurrence of these hallucinations is linked to the nature of the Causal Language Model we used and its auto-regressive architecture. In an auto-regressive framework, each predic- tion is based solely on the previous tokens without any direct consideration of future points, making it prone to accumulating small errors over time. As these errors compound, especially in longer sequences, they can lead to unrealistic outputs such as abrupt turns or improbable position jumps.
To address and potentially mitigate the generation of hallucinatory trajectories, future research could explore the application of reinforcement. learning solutions. By incorporating concepts of correct versus incorrect
18

---

Published in Transactions on Machine Learning Research (03/2025)
trajectories, techniques such Reinforcement Learning from Human Feedback (RLHF) (Schulman et al., 2015; 2017; MacGlashan et al., 2017; Stiennon et. al., 2020; Ouyang et al., 2022). Reinforcement Learning with Artificial Intelligence Feedback (RLAIF) (Lee et al., 2023), or Direct Preference Optimization (DPO) (Rafailov et al., 2023) might offer promising avenues for improvement. Additionally, integrating a posi- tioning vector token within a pseudo-octal representation of the cell could enhance the spatial structure understanding of the model, promoting more realistic trajectory predictions.
This exploration into advanced learning methodologies and structural enhancements remains prospective area of research. Such investigations could significantly refine the predictive accuracy and reliability of ship trajectory models, ensuring that hallucinations are effectively minimized or altogether climinated in future iterations.
6 Conclusion
In conclusion, our foray into the H3 index system has significantly enhanced our approach to spatial data representation, particularly in the context of maritime trajectory prediction. It is important to clarify that our model is designed for manual user actions in GIS software and is not intended for real-time streaming applications. The ability of the H3 index to reduce visual distortions across different map projections and to maintain spatial fidelity has been crucial in our work.
The introduction of pseudo-octal representation for the H3 cells marked pivotal point in our research. This innovative approach, involving the discarding of initial bits and encoding cells in more compact format, significantly streamlined our data processing. The algorithms we developed for this transformation and its inverse showcase our commitment to enhancing the efficiency of spatial data manipulation.
Looking at the practical implementation of these choices, becomes clear that they have impacted the prediction of maritime trajectorics. Our methodical approach spatial representation and tokenisation has laid the groundwork for the application of Causal Language Modelling (CLM) in this domain. The integration of CLM promises to improve maritime trajectory prediction, enhancing both the accuracy and efficiency of predictive models.
Although the Mixtral8x7B model architecture is very elfective at capturing the complex patterns of maritime trajectories, we recognise the potential benefits of Fourier transforms. Recent advances, particularly in FNet (Leo-Thorp et al., 2021); have demonstrated the usefulness of Fourier transforms in sequence modelling tasks. We believe that the application of Fourier transforms could further improve our model's ability to recognise additional patterns and nuances in ship trajectories, which could lead to even more accurate predictions. This line of exploration represents an intriguing future direction in pursuit of improving the accuracy of maritime trajectory prediction.
This research not only contributes to the field of geospatial data analysis but also opens new pathways for future exploration, particularly in the application with advanced modelling techniques like CLM. We antici- pate that our findings and methodologies will pave the way for more sophisticated and accurate applications in maritime trajectory prediction and beyond. Additionally, this research is geared towards manual GIS workflows, highlighting its distinction from real-time applications.
Data Availability
Due the sensitive nature of the data, which was collected using proprietary system of the French Navy, are unable to share the raw dataset. However, have provided a detailed description of the data preparation procedure, ensuring that others replicate the cleaning and tokenisation steps. If needed, we can also provide the code used for data cleaning. Morcover, the data can be recap- tured dynamically, as we did during the testing phase, or exported from publicly available paid-platforms such as MarineTrafti and VesselFinder, allowing for similar experiments to be conducted with alterna- tive datascts. For free data, refer to AISHub (https:/ www.aishuu.nct//) Norwegian coastguard's server (https:// https//www.ystterkt.... www.o////eeaaaainnnn.oo aavigatioo--nddmonnntrnnggggg /ass/ hatp://css-s-t-i-datttttt to-ais-data/) or the data history of the Danish Maritime Authority (https://wwol wttp://wbbbbi...kk aisdata/).
19

---

Published in Transactions on Machine Learning Research (03/2025)
References
Mohammad Assaad, Romuald Boné, and Hubert Cardot. A new boosting algorithm for improved time-series forecasting with recurrent neural networks. Information Fusion, 9:41-55, 1 2008. ISSN 15662535. doi: 10.10166/j.nffuu.22006 10.009.
Kexin Bao, Jinqiang Bi, Miao Gao, Yuc Sun, Xuefeng Zhang, and Wenjia Zhang. An improved ship trajectory prediction based on ais data using mha-bigru. Journal of Marine Science and Engineering, 10, 6 2022. ISSN 20771312. doi: 10.3390/jmse1006000
ВЕАМer. Rapport d'activités. Feb 2022. CRL http://wwbbbae.eree http:://wwww.beerrrr..deeaalpaap gouv fr/
spip.php php?paee-somaalee
Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arxiv:0004.050 05150, 2020.
Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901 2020.
Rewon Child, Scott Gray, Alec Radford, and llya Sutskever. Generating long sequences with sparse trans- formers. arXiv preprinl ar 10509, 2019.
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rmm encoder-decoder for statisti- cal machine translation. arXiv preprint 0rXiv:1406.1077 2014.
Robert M. Clark. Geospatial Intelligence. Origins and Evolution. Georgetown University Press, 2020.
J.T. Connor, R.D. Martin, and L.E. Atlas. Recurrent neural networks and robust time series prediction. IEEE Transoctions on Neural Networks, 5:240-254, 1994. ISSN 1045-9227. doi: 10.1109/72.279188.
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdi- nov. Transformer-xl: Attentive language models beyond fixed-length context. arXio preprint arXiv:1901 028860, 2019.
Jacob Devlin, Ming-Wor Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.0/44 04805, 2018.
Martin Ester, Hans- Kriegel, Jörg Sander, Xiaowei Xu, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, pp. 226-231, 1996.
M. Fréchet. Sur la distance de deux lois de probubilité. Feb 1957.
Google. $2 geometry: Computational geometry and spatial indexing on the sphere. https://github. com/ google/ stggl/////emmmeryy 2015.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Newral compulation, 9(8):1735 1780, 1997.
ISO 6709:2008/ by coordinates. Standard, International Organization for Standardization, Geneva, CH, 2009.
C0:008/Cor CCo 1:2009. ISO 6709:2008/ Cor 1:2009, Standard representation geographic point location
Albert Q, Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b. 10 2023.
20

---

Published in Transactions on Machine Learning Research (03/2025)
Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie- Anne Lachaux, Pierre Stock, Sandcop Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Témothée Lacroix, and William El Sayed. Mixtral of experts. 2024. URL http://arxiv. htt.aaawwi..g..aaaaeeeeeee abs /200// 04088.
Xiang Jiang, Erico N de Souza, Ahmad Pesaranghader, Baifan Hu, Daniel L Silver, and Stan Matwin. Trajectorynet: An embedded gps trajectory representation for point-based classification using recurrent neural networks. arXiv preprint arXiv:1703 700.02666, 2017.
Rudolph Emil Kalman. new approach to linear filtering and prediction problems. Transactions of the ASME-JJurnal of Basic Engineering, 82(Series D):35-45, 1960.
Zulfigar Khan, Tanveer Hussain, Amin Ullah, Seungmin Rho, Miyoung Lee, and Sung Baik. Towards efficient electricity forecasting in residential and commercial buildings: A novel hybrid enn with Istm-ae based framework. Sensors, 20:1399, 3 2020. ISSN 1424-8220. doi: 00.3390/s20051300
Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and Sushant Prakash. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. 2023.
James Lee- Thoop, Joshua Ainslie, llya Eckstein, and Santiago Ontanon. Fnet: Mixing tokens with fourier transforms. arXiv preprint arXiv:2105. 1XX::2105500824 2021.
Mingze Li, Bing Li, Zhigang Qi, Jiashuai Li, and Jiawei Wu. Enhancing maritime navigational safety: Ship trajectory prediction using acoatt-Istin and data. ISPRS International Journal of Gea-Information, (3), 2024. ISSN 2220-9964. doi: 10.3390 10.3390/ijgi13000 /tt//ii/333000555 URL htth:tts:/// //ww:. wwwp..mdp.... com/ 2ttmm//200-000 2ttp//222299999//3333
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. 10 2023.
James MacGlashan, Mark K Ho, Robert Loftin, Bei Peng, Guan Wang, David Roberts, Matthew E. Taylor, and Michael L. Littman. Interactive learning from policy-dependent human feedback. 1 2017.
Contributors MistralKit. Opencompass: universal evaluation platform for foundation models. https:
2023.
Brian Murray and Lokukaluge Prasad Perera. A dual linear autoencoder approach for vessel trajectory prediction using historical ais data. Ocean Engineering, 209:107478, 8 2020. ISSN 00298018. doi: wwt.://.ee...ccennn.....mo occaneng. 2020. 107...07777..tpp////
10.1016/j..ceaneeng.. http::eceaaenn222..07777777 2020. linkinghub. elsevier. con///ecrcoeee11///220000000000
URL https://6ol..rr/10.. 1016/
Duong Nguyen and Ronan Fablet. A transformer network with sparse augmented data representation and cross entropy loss for ais-based vessel trajectory prediction. IEEE Access, pp. 1-1, 2024. ISSN 2169-3536. doi: 10.1109/ ACCESS. SA0.SS..224.333400 2024.334
Duong Nguyen, Rodolphe Vadaine, Guillaume Hajduch, René Garello, and Ronan Fablet. A multi-task deep learning architecture for maritime surveillance using ais data streams. In 2018 IEEE 5th International Conference on Dala Science and Advanced Analylics (DSAA), pp. 331 340, 2018. doi: 10.1109/DSAA. 2018.00044.
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. 11 2022.
International Maritime Organization. Revised guidelines for the onboard operational use of shipborne Auto- matic Identification Systems (AIS). IMO, December 2015. Adopted on 2 December 2015 (Agenda item 10).
21

---

Published in Transactions on Machine Learning Research (03/2025)
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. 3 2022.
Giuliana Pallotta, Michele Vespe, and Karna Bryan. Vessel pattern knowledge discovery from ais data: framework for anomaly detection and route prediction. Entropy, 15:2218 2244,, 2013. ISSN 10994300. doi: 10.3390/c15002218.
Ofir Press, Noah A Smith, and Mike Lewis. Train short, test long: Attention with linear biases enables input length extrapolation. arXiv preprinl Xrxi:22008. 12409, 2021.
Alec Radford. Improving language understanding by generalive pre-training. OpenAl blog, 2018.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn, Direct preference optimization: Your language model is secretly reward model. 5 2023.
Kevin Sahr, Denis White, and A. Jon Kimerling. Geodesic discrete global grid systems. Cartography and Geographic Information Science, 30:121-134, 2003. ISSN 1523-0406. doi: 10. 00555//15330000000000 1090.
David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Jamuschowski. Deepar: Probabilistic forecasting with autorogressive recurrent networks. International Journal of Forecasting, 36:1181-1191, 2020. ISSN 01692070. doi: /0000//|iiooreass..01.......0..0
John Schulman, Sergey Levine, Philipp Moritz, Michael Jordan, and Pieter Abbeel. Trust region policy optimization. 2 2015.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov, Proximal policy optimiza- tion algorithms. 7 2017.
Walter L. Sharp. Geospatial Intelligence Support to Joint Operations. DIANE Publishing Company, 2011.
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoe Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. 2017.
Gabriel Spadon, Jay Kumar, Derek Eden, Josh van Berkel, Tom Foster, Amilcar Soares, Ronan Fablet, Stan Matwin, and Ronald Pelot. Multi-path long-term vessel trajectorics forecasting with probabilistic
feature fusion for problem shifting. Ocean Engineering, 312:119138, 11 2024. ISSN 0029-8018. doi:
Nisan Stiemnon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodci, and Paul Christiano. Learning to summarize from human foodback. 2020.
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Newrocomputing, 568:127063, 2 2024. ISSN 09252312. doi: 10.1016/j.neucomm2023 200.
llya Sutskever, Oriol Vinyals, and Quoe V Le. Sequence to sequence learning with neural networks. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014. URL https: /ptoce/e//..cc.o....... courips.cc/papeee http:/pppr..1leepppp//////////1/ htt.:////.....aaa22227772/......
pdf.
Kristian Aalling Sørensen, Peder Heiselberg, and Henning Heiselberg. Probabilistic maritime trajectory prediction complex scenarios using deep learning. Sensors, 22(5), 2022. ISSN 1424-8220. doi: 10.3390/
$22052058. URL https: http///www.a// /oddpiicom/1444822///2/2
22

---

Published in Transactions on Machine Learning Research (03/2025)
Enmei Tu, Guanghao Zhang, Lily Rachmawati, Eshan Rajabally, and Guang Bin Huang. Exploiting ais data for intelligent maritime navigation: A comprehensive survey from data to methodology- IEEE Transactions on Intelligent Transportation Systems, 19:1559 1582, 5 2018. ISSN 15249050. doi: 10.1109/TITS.2017. 2724551.
Uber. H3: Hexagonal hicrarchical goospatial indexing system.
2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30,
2017.
Yiqiu Wang, Yan Gu, and Julian Shun. Theoretically-effcceen and practical parallel dbscan. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, SIGMOD '20, pp. 2555 2571, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450367356. doi: 10.1145/ 3318464.3380582. URL https://doii org/10. 1145//311/338 3380582.
T Wolf. Huggingface's transformers: State-of-the-art natural language processing.
ar preprint
arXiv:1910.037 2019.
Haixu Wu, Jiehui Xu, Jiamnin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, Advances in Neural Information Processing Systems, 34: 22419 22430, 2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-vaiaation modeling for general time series analysis. 10 2022.
Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, and Vikas Singh. Nyströmformer: A nyström- based algorithm for approximating self-attention. volume 35, pp. 14138 14138-14148, 2021.
Gheng-Hong Yang, Chih-Hsien Wu, Jen-Chung Shao, Yi-Chuan Wang, and Chih-Min Hsieh. based intelligent vessel trajectory prediction using bi-lstm. IEEE Access, 10:24302 24315, 2022. doi: 10.1109/ ACCESSS.022.331554 2022.31548...
24345,
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting?
5 2022.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang, In- former: Beyond elficient transformer for long sequence time-series forecasting, In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11106-11115, 2021.
Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeff Dean, Noam Shazeer, and William Fedus. St-moe: Designing stable and transferable sparse expert models. 2 2022.
23

---

Published in Transactions on Machine Learning Research (03/2025)
A
H3 Index Bit Layout
Table 4: H3 Index Bit Layout
Start
Bits
Type
Reserved
1
4
Index Mode
5
3
Mode-Dependent
8
4
Resolution
12
7
Base Cell
19
3
Res digit
22
3
Res digit
3
Res digit
28
3
Res digit
31
3
Res digit
34
3
Res 6 digit
37
3
Res digit
40
3
Res digit
43
Res digit
46
3
Res 10 digit
49
3
Res 11 digit
52
3
Res 12 digit
55
3
Res 13 digit
58
3
Res 14 digit
61
3
Res 15 digit
24

---

Published in Transactions on Machine Learning Research (03/2025)
B
Learning curves
0.0
eval loss
train loss
-0.5
-1.0
-1.5
-2.0
-2.
-3.0
0
25000
50000 75000 100000 125000 150000 175000
Unified training and evaluation loss curves. It shows the loss value compared to the number of steps in the training process. Convergence of training (orange) and evoluation (blue) loss curues indicales effeclive model generalisation without overfilling.
Figure 11:
25

---

Published in Transactions on Machine Learning Research (03/2025)
C
Density plot of Fréchet distance versus prediction distance
Conjoined density heatmap
12000
10000
8000
Fréchet distance
6000
4000
2000
30000
40000
50000
60000
Prediction distance
Figure 12: Graph showing the density of the Fréchet distance in metres compared with the prediction distance in metres for 90-minute predictions with 60 minutes of context. We can see that the errors measured by the Fréchet distance are minimal for prediction lengths of around 30km. Prediction distonces that are too short too long be interpreted teleportations or hallucinations.
26

---

Published in Transactions on Machine Learning Research (03/2025)
D Example of river prediction
Budernh
MOLLEN
VOHRUM
OVERERUCA
Rheebberg
WAISIM
HOLTE
REFELEN
fuayn
DUISBURC
km
DESTRUM
Figure 13: Illustration of prediction of ship's trajectory in a river. The context is represented by 60 ménutes long red line, the 155 minutes long green lines represent predictions. The ground truth is shown in blue. It is interesting to note that accuracy in these areas is lower, nolably due to the inaccuracy of GNSS.
27

---

Published in Transactions on Machine Learning Research (03/2025)
E
Distribution of predictions
Mean, median and density peak for a 60-minute contextual window and 90-minute prediction.
Prediction distance without outliers
Fréchet distance without outliers
Values: Mean 00299.099 89
Values:
Mean = 2806 44
Median = 333698.90
Median 1938.55
Density 32401.89
Density Peak 1061.00
10000
20000
30000
40000
50000
60000
2000
6000
8000
10000
Prediction distance (meters)
Fréchet distance (meters)
Figure 14: Violin plot of the prediction dislance without oulliers, highlighting the mean (green slar), median, and density peak for a context length of 60 minutes and a prediction length of 90 minutes.
28

---

Published in Transactions on Machine Learning Research (03/2025)
Mean, median and density peak for 60-minute contextual window and 240-minute prediction.
Prediction distance without outliers
Fréchet distance outliers
Values: Mean 88002.73
Values:
Mean 15063.32
Modian 8-7744.60
Median 11174.91
Density Peak 87799.67
Density Peak 5718.16
25000
50000
75000
100000
125000
150000
10000
30000
40000
50000
Prediction distance (meters)
Fréchet distance (meters)
Figure 15:
Violin plot of the prediction distance without outliers, highlighting the mean (green star), median, and density peak for a contert length of 60 minutes and a prediction length of 240 minutes.
Mean, median and density peak for 60-minute contextual window and 390-minute prediction.
Prediction distance without outliers
Fréchet distance without outliers
Values:
Values:
Mean = 122907.79
Mean 33147.25
Median 127441172
Median 244789.9 97
Density Peak 147868.29
Density Peak - 11798.45
100000
200000
60000 Fréchet distance (meters)
100000
Prediction distance (meters)
Figure 16: Violin plot of the prediction distance without oulliers, highlighting the mean (green slar), median, and density peak for a contert length of 60 minutes and a prediction length of 390 minutes.
29

---

Published in Transactions on Machine Learning Research (03/2025)
F
Multiple trajectories grid
(a)
12 km
20 km
12 km
(b)
17
17 km
17
(c)
Figure 17: Zoom out path prediction performance comparison of figure 7.
30

---

Published in Transactions on Machine Learning Research (03/2025)
G
Hallucinations
Figure 18: More examples of hallucinations. The red point is the barycentre of the polygon formed by the green points at the ends of the green trajectories.
31

---

Published in Transactions on Machine Learning Research (03/2025)
Figure 19: More examples of hallucinations.
32


Reviews:
[{"review_id": "18NjzJEupU", "reviewer": "Reviewer_1ide", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- Ship trajectory mining is an important AI area with a lot of promise and applications\n- The discretization step is an essential tool to model patterns and improve prediction\n- Experimental results show the promise of the solution with qualitative and quantitative results\n\nWeaknesses:\n\n- Additional baselines are needed to understand improvements over SOTA methods\n- Discussion about the scalability of the solution is also necessary considering the need to run such models in streaming manner\n- The extension of H3 lacks technical depth but seems to work reasonably well", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Ship trajectory prediction is an evolving area in AI, with traditional methods relying on techniques like LSTMs, GRUs, and Transformers. This study offers an alternative approach, predicting trajectories using only GNSS positions by treating the problem as a natural language processing task. AIS latitude/longitude data is converted into cell identifiers via the H3 index, making it easier for language models to capture spatial patterns. The Fréchet distance is used for evaluation and results demonstrate the promise of this solution against an established baseline.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Additional baselines are needed to understand improvements over SOTA methods\n\nThe paper is lacking in terms of comparisons with baselines. From traditional forecasting methods to modern DNN and Foundation Models, such methods have to be included to understand the improvement over SOTA methods.\n\n- Discussion about the scalability of the solution is also necessary considering the need to run such models in streaming manner\n\nIn practice, such methods have to be applied in real-time and therefore, some discussion and experiments (vs. baselines) are needed to understand the computational cost of this solution.\n\n- The extension of H3 lacks technical depth but seems to work reasonably well\n\nThe solution is mainly a simple extension of H3. It seems to work and improve performance, yet, it lacks technical depth (it appears as a simple transformation step, which is unclear if this has to appear as a full paper). Novelty is somewhat low for this work.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "AvDafUFENL", "reviewer": "Reviewer_2Vsr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1. Novel Research Topic: The focus on maritime trajectory forecasting, especially using GNSS data alone, is an uncommon and intriguing application in machine learning, providing valuable insights for specific domains like logistics and maritime safety.\n2. Robust Methodology: The use of H3 geospatial indexing for discretizing spatial data is well-suited for this task, mitigating the challenges of GNSS inaccuracies and enabling hierarchical spatial representation. The model’s integration of classic transformer techniques with spatial data highlights practical applications of NLP methods in non-traditional fields.\n\nWeakness:\n1. Limited Benchmarking: The model comparison is limited to Kalman filters and TrAISformer. There is no comparison with traditional RNN-based methods, such as LSTM or GRU, commonly used for trajectory forecasting. This weakens the claim that the proposed approach is superior for this task\n2. Insufficient Generalization Analysis: Although the model generalizes well globally, the lack of specific regional analysis or adaptability to diverse maritime environments limits insights into its robustness across unique operational settings.\n3. Dependence on Data Availability: The reliance on proprietary AIS data limits reproducibility and wider adoption within the machine learning community. Without open datasets, the value of this work to the broader research community remains unclear.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper focuses on applying transformer to the prediction of maritime trajectories, presenting an application-oriented study that leverages established techniques in an innovative context. The contributions can be summarized as follows:\n1. Interesting Application.\n2. Methodology Based on Classic Model Mistral 8x7B.\n3. Predictable but Useful Results.\n4. Limited Comparison Against Existing Methods:", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Expand Generalization Analysis: Provide a detailed evaluation of the model’s performance across diverse geographic regions and maritime scenarios to validate its robustness and adaptability.\n2. Address Motivation for Transformer Adoption: Include a comparison with LSTM or GRU-based models to demonstrate clear advantages (e.g., accuracy, long-term dependencies, computational efficiency).", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZM4ag1he7W", "reviewer": "Reviewer_Nwg3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths \n1. In terms of methodological innovation, to address the limitations of traditional methods in ship trajectory prediction, this study innovatively combines the Transformer architecture with spatial representation, using H3 spatial indexing and pseudo-octal representation to resolve GNSS precision issues.\n2. This study directly addresses the practical challenges in maritime operations, offering significant practical value.\n\nWeaknesses\n1. The article title emphasizes the H3 index and causal language models, but the introduction lacks a description of the causal language model.\n2. The article provides limited comparison between the proposed method and other existing methods.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1. This paper proposes an innovative approach combining Transformer architectures with spatial representation to address the omission of spatial features in traditional ship trajectory prediction methods, improving accuracy.\n2. This paper utilises Uber's H3 spatial indexing system to discretise ship trajectories into hexagonal cells, resolving GNSS precision issues, and introduces a pseudo-octal H3 representation to enhance Transformer's spatial understanding and computational efficiency.\n3. This paper explores the advantages of causal language models in ship trajectory prediction, leveraging their auto-regressive nature to capture temporal and causal structures, improving performance in complex maritime scenarios.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. It would be more logical to include a description of the causal language model in the introduction, even though a detailed explanation is provided in the later sections.\n2. The model chooses Mixtral8x7B, and providing a simple architecture diagram for this model would help make the description clear.\n3. If possible, a comparison with more existing methods should be included.", "broader_impact_concerns": "It seems that this paper does not involve ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: When responding to the reproducibility concerns raised by reviewer 2Vsr, the authors pointed out that there exist free, publicly available AIS datasets. While adding links in the Data Availability section would help, greater impact could be made if even some preliminary experiments using these datasets would be added to the paper to demonstrate the generalization capabilities of the proposed method.
Decision: Accept (poster)