# Survey of Federated Learning Models for Spatial-Temporal Mobility Applications

Yacine Belal, Sonia Ben Mokhtar, Hamed Haddadi, Jaron Wang, Afra Mashhadi

► To cite this version:

Yacine Belal, Sonia Ben Mokhtar, Hamed Haddadi, Jaron Wang, Afra Mashhadi. Survey of Federated Learning Models for Spatial-Temporal Mobility Applications. ACM Transactions on Spatial Algorithms and Systems, In press, 10.1145/3666089. hal-04613220

HAL Id: hal-04613220

https://hal.science/hal-04613220

Submitted on 15 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Survey of Federated Learning Models for Spatial-Temporal Mobility Applications

Yacine Belal, Sonia Ben Mokhtar, Hamed Haddadi, Jaron Wang, Afra Mashhadi

► To cite this version:

Yacine Belal, Sonia Ben Mokhtar, Hamed Haddadi, Jaron Wang, Afra Mashhadi. Survey of Federated Learning Models for Spatial-Temporal Mobility Applications. ACM Transactions on Spatial Algorithms and Systems, In press, 10.1145/3666089. hal-04613220

HAL Id: hal-04613220

https://hal.science/hal-04613220v1

Submitted on 15 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
SURVEY OF FEDERATED LEARNING MODELS FOR
SPATIAL-TEMPORAL MOBILITY APPLICATIONS

A PREPRINT

<figure><img src="image_2.png" /></figure>

Yacine Belal

National Institute of Applied Sciences
Lyon, France

yacine.belal@insa-lyon.fr

<figure><img src="image_3.png" /></figure>

Sonia Ben Mokhtar

National Institute of Applied Sciences
CNRS

Lyon, France

sonia.benmokhtar@insa-lyon.fr

<figure><img src="image_4.png" /></figure>

Hamed Haddadi

Imperial College London
London, UK

h.haddadi@imperial.ac.uk

Jaron Wang

University of Washington
Bothell, USA

jierui98@uw.edu

<figure><img src="image_5.png" /></figure>

Afra Mashhadi

University of Washington
Bothell, USA

mashhadi@uw.edu

June 15, 2024

ABSTRACT

Federated learning involves training statistical models over edge devices such as mobile phones such that the training data is kept local. Federated Learning (FL) can serve as an ideal candidate for training spatial temporal models that rely on heterogeneous and potentially massive numbers of participants while preserving the privacy of highly sensitive location data. However, there are unique challenges involved with transitioning existing spatial temporal models to federated learning. In this survey paper, we review the existing literature that has proposed FL-based models for predicting human mobility, traffic prediction, community detection, location-based recommendation systems, and other spatial-temporal tasks. We describe the metrics and datasets these works have been using and create a baseline of these approaches in comparison to the centralized settings. Finally, we discuss the challenges of applying spatial-temporal models in a federated setting and by highlighting the gaps in the literature we provide a road map and opportunities for the research community.

**Keywords** Spatio-Temporal Applications, Federated Learning, Privacy-Preserving, Mobility Prediction, Transportation, Location Based Social Networks

# 1 Introduction

Spatio-temporal (ST) mobility data collected by location-based services (LBS) [72] and other means such as Call Data Records (CDR), WiFi hotspots, smart watches, cars, etc. is very useful from a socio-economical perspective as it is at the heart of many useful applications (e.g., navigation, geo-located search, geo-located games) and it allows answering numerous societal research questions [86]. For example, CDR has been successfully used to provide real-time traffic anomaly as well as event detection [155, 161], and a variety of mobility datasets have been used in shaping policies for urban communities [47] or epidemic management in the public health domain [129, 128]. From an individual-level perspective, users can benefit from personalized recommendations when they are encouraged to share their location data with third parties [37].

While there is no doubt about the usefulness of location-based applications, privacy concerns regarding the collection and sharing of individuals’ mobility traces or aggregated flow of movements have prevented the data from being utilized to their full potential [151, 17, 89]. A mobility privacy study conducted by De Montjoye et al. [33] illustrates that four spatio-temporal points are enough to identify 95% of the individuals, which exacerbates the user re-identification risk and could be the origin of many unexpected privacy leakages. Indeed, various studies have shown that numerous threats
SURVEY OF FEDERATED LEARNING MODELS FOR
SPATIAL-TEMPORAL MOBILITY APPLICATIONS

A PREPRINT

<figure><img src="image_2.png" /></figure>

Yacine Belal

National Institute of Applied Sciences
Lyon, France

yacine.belal@insa-lyon.fr

<figure><img src="image_3.png" /></figure>

Sonia Ben Mokhtar

National Institute of Applied Sciences
CNRS

Lyon, France

sonia.benmokhtar@insa-lyon.fr

<figure><img src="image_4.png" /></figure>

Hamed Haddadi

Imperial College London
London, UK

h.haddadi@imperial.ac.uk

Jaron Wang

University of Washington
Bothell, USA

jierui98@uw.edu

<figure><img src="image_5.png" /></figure>

Afra Mashhadi

University of Washington
Bothell, USA

mashhadi@uw.edu

June 15, 2024

ABSTRACT

Federated learning involves training statistical models over edge devices such as mobile phones such that the training data is kept local. Federated Learning (FL) can serve as an ideal candidate for training spatial temporal models that rely on heterogeneous and potentially massive numbers of participants while preserving the privacy of highly sensitive location data. However, there are unique challenges involved with transitioning existing spatial temporal models to federated learning. In this survey paper, we review the existing literature that has proposed FL-based models for predicting human mobility, traffic prediction, community detection, location-based recommendation systems, and other spatial-temporal tasks. We describe the metrics and datasets these works have been using and create a baseline of these approaches in comparison to the centralized settings. Finally, we discuss the challenges of applying spatial-temporal models in a federated setting and by highlighting the gaps in the literature we provide a road map and opportunities for the research community.

**Keywords** Spatio-Temporal Applications, Federated Learning, Privacy-Preserving, Mobility Prediction, Transportation, Location Based Social Networks

# 1 Introduction

Spatio-temporal (ST) mobility data collected by location-based services (LBS) [72] and other means such as Call Data Records (CDR), WiFi hotspots, smart watches, cars, etc. is very useful from a socio-economical perspective as it is at the heart of many useful applications (e.g., navigation, geo-located search, geo-located games) and it allows answering numerous societal research questions [86]. For example, CDR has been successfully used to provide real-time traffic anomaly as well as event detection [155, 161], and a variety of mobility datasets have been used in shaping policies for urban communities [47] or epidemic management in the public health domain [129, 128]. From an individual-level perspective, users can benefit from personalized recommendations when they are encouraged to share their location data with third parties [37].

While there is no doubt about the usefulness of location-based applications, privacy concerns regarding the collection and sharing of individuals’ mobility traces or aggregated flow of movements have prevented the data from being utilized to their full potential [151, 17, 89]. A mobility privacy study conducted by De Montjoye et al. [33] illustrates that four spatio-temporal points are enough to identify 95% of the individuals, which exacerbates the user re-identification risk and could be the origin of many unexpected privacy leakages. Indeed, various studies have shown that numerous threats
<figure><img src="image_6.png" /><figcaption>Figure 1: Number of FL Surveys and the underlying applications tackled. **Note the absence of surveys on FL-ST works**, which contrasts with the ever-growing number of FL-ST methodological papers (e.g., 26 papers in 2023).</figcaption></figure>

arise if location data falls into the hands of inappropriate parties. These threats include re-identification [111], the inference of sensitive information about users [89, 165] (e.g., their home and work locations, religious beliefs, political interests or sexual orientation). In some extreme cases, sharing geo-located data may even endanger users' physical integrity (e.g., the identification of protesters in dictatorial regimes or during wars<fnref n="1" />) or their belongings (e.g., robbery<fnref n="2" />).

One way to consider addressing privacy challenges is to break from centralized data collection and maintain the location data on user devices. In this decentralized paradigm, a variety of solutions are plausible. A possible approach is to create accurate enough synthetic location data to train spatial-temporal models and then fine-tune them on users' devices to perform best for the individuals' location data [67]. Although this is a promising approach, the research has shown that mobility trajectory generation is an extremely difficult task that may not portray data heterogeneity accurately [88, 107]. Moreover, trajectories from different locations, periods, or user groups may have distinct characteristics that the pre-trained model does not handle effectively. Although this is an ongoing field of research, to date there exists little evidence as to whether models can transfer mobility knowledge across cities [67], as with a significant distribution shift between the data used for pre-training and the data for the target task, transfer learning may not generalize well.

In parallel, an alternative approach that is increasingly considered a promising approach is *Federated Learning* (FL) [117]. FL relies on clients (be it users' smartphones or edge devices) to train a machine learning model on their local training data and share the model weights with a central server (called the FL server) that aggregates the received clients' contributions. As such, FL can inherently alleviate some privacy concerns [149] while enabling clients to collaboratively train and utilize a global model without ever sharing their private data. This characteristic renders FL more compliant with recent privacy regulations (e.g., General Data Protection Regulation (GDPR) [158], California Consumer Privacy Act (CCPA) [63]) and an increasing user awareness [156]. Additionally, it facilitates the emergence of new use cases (e.g., digital healthcare [144, 92]). Indeed, several real-world applications leveraging FL have already emerged including drug discovery [133] and fraud detection [4].

Compared with other applications of FL, ST data and models possess certain characteristics and properties that introduce challenges. For instance, in domains involving ST data, the observations made at nearby times and locations cannot hold the independent property due to the auto-correlation (e.g., changes in traffic activity occur smoothly over time and space). As a result, classical FL algorithms that assume independence among observations are not suited for ST applications [11]. Furthermore, ST data is not identically distributed, that is, it does not meet the assumption that every instance has the same probability to belong to a set. These two properties are referred to as *iid* (independent and identically distributed). Finally, ST data is often both spatially and/or temporally sparse and often created at high frequencies, especially in real-time applications, resulting in a high velocity and volume of data generation.

**Related Surveys** Since FL emerged half a decade ago, many survey papers [95, 164, 34, 49, 125, 140] have reviewed relevant literature from various perspectives. In 2023 alone, 91 surveys on federated learning were published [164, 13, 109] with the majority being on applications-specific topics, predominantly focused on healthcare [142, 58, 138, 163], finance [130], blockchain [74, 200, 135, 126], and the Internet of Things (IoT) [131, 6, 29, 157]. Figure 1 presents the

<footnote n="1">¹https://www.independent.co.uk/news/ukraine-ap-russia-gps-kyiv-b2093310.html</footnote>

<footnote n="2">²https://pleaserobme.com</footnote>
<figure><img src="image_6.png" /><figcaption>Figure 1: Number of FL Surveys and the underlying applications tackled. **Note the absence of surveys on FL-ST works**, which contrasts with the ever-growing number of FL-ST methodological papers (e.g., 26 papers in 2023).</figcaption></figure>

arise if location data falls into the hands of inappropriate parties. These threats include re-identification [111], the inference of sensitive information about users [89, 165] (e.g., their home and work locations, religious beliefs, political interests or sexual orientation). In some extreme cases, sharing geo-located data may even endanger users’ physical integrity (e.g., the identification of protesters in dictatorial regimes or during wars<fnref n="1" />) or their belongings (e.g., robbery<fnref n="2" />).

One way to consider addressing privacy challenges is to break from centralized data collection and maintain the location data on user devices. In this decentralized paradigm, a variety of solutions are plausible. A possible approach is to create accurate enough synthetic location data to train spatial-temporal models and then fine-tune them on users’ devices to perform best for the individuals’ location data [67]. Although this is a promising approach, the research has shown that mobility trajectory generation is an extremely difficult task that may not portray data heterogeneity accurately [88, 107]. Moreover, trajectories from different locations, periods, or user groups may have distinct characteristics that the pre-trained model does not handle effectively. Although this is an ongoing field of research, to date there exists little evidence as to whether models can transfer mobility knowledge across cities [67], as with a significant distribution shift between the data used for pre-training and the data for the target task, transfer learning may not generalize well.

In parallel, an alternative approach that is increasingly considered a promising approach is *Federated Learning* (FL) [117]. FL relies on clients (be it users' smartphones or edge devices) to train a machine learning model on their local training data and share the model weights with a central server (called the FL server) that aggregates the received clients' contributions. As such, FL can inherently alleviate some privacy concerns [149] while enabling clients to collaboratively train and utilize a global model without ever sharing their private data. This characteristic renders FL more compliant with recent privacy regulations (e.g., General Data Protection Regulation (GDPR) [158], California Consumer Privacy Act (CCPA) [63]) and an increasing user awareness [156]. Additionally, it facilitates the emergence of new use cases (e.g., digital healthcare [144, 92]). Indeed, several real-world applications leveraging FL have already emerged including drug discovery [133] and fraud detection [4].

Compared with other applications of FL, ST data and models possess certain characteristics and properties that introduce challenges. For instance, in domains involving ST data, the observations made at nearby times and locations cannot hold the independent property due to the auto-correlation (e.g., changes in traffic activity occur smoothly over time and space). As a result, classical FL algorithms that assume independence among observations are not suited for ST applications [11]. Furthermore, ST data is not identically distributed, that is, it does not meet the assumption that every instance has the same probability to belong to a set. These two properties are referred to as *iid* (independent and identically distributed). Finally, ST data is often both spatially and/or temporally sparse and often created at high frequencies, especially in real-time applications, resulting in a high velocity and volume of data generation.

**Related Surveys** Since FL emerged half a decade ago, many survey papers [95, 164, 34, 49, 125, 140] have reviewed relevant literature from various perspectives. In 2023 alone, 91 surveys on federated learning were published [164, 13, 109] with the majority being on applications-specific topics, predominantly focused on healthcare [142, 58, 138, 163], finance [130], blockchain [74, 200, 135, 126], and the Internet of Things (IoT) [131, 6, 29, 157]. Figure 1 presents the

<footnote n="1">¹https://www.independent.co.uk/news/ukraine-ap-russia-gps-kyiv-b2093310.html</footnote>

<footnote n="2">²https://pleaserobme.com</footnote>
growing number of application-specific FL surveys<fnref n="3" />, which is paralleled by an emergence of papers on FL-ST topics. In motivating the need for this survey, we discuss the perspectives that IoT-related and generic FL surveys have covered and highlight their shortcomings when applied to ST models. More specifically, FL Surveys on smart cities including those of IoT applications are closest to our work. Zheng et al. [199] reviewed applications of FL in smart cities. They provided a broad overview of various applications of smart cities including IoT, transportation, communication, medical care, and finance. They created a glossary of the papers in these domains without discussing the diversity of existing approaches or drawing comparisons between these works. Their work mainly focused on security and privacy challenges and outlined algorithmic efficiency as a future direction but did not consider the heterogeneity and sparsity often associated with the location data. The closest survey to ours is the work of Pandya et al. [131], which has a subsection discussing the FL transportation applications in smart cities. However, only one of the research works discussed in their survey is revisited in our work, as their focus predominantly revolves around a different type of use cases (e.g., number plate recognition, railway ... etc.).

Other FL surveys that are *not* application-specific and review methodological approaches of FL regarding heterogeneity, model convergence, and personalization fall short in drawing a comparison between these different techniques when applied to ST data. Although some of the challenges such as non-iid are well covered in those surveys, the other characteristics of the ST data are not covered. Finally, we note that despite its unique properties and the increasing number of spatial-temporal research papers emerging in FL, no survey paper has attempted to categorize, review, and compare this emergence of FL-ST methodological approaches (Figure 1).

**Our Contribution** The contributions of this survey are as follows:

* In this survey, we study 38 existing spatial-temporal works that have been tailored for the FL paradigm. As we review these recent advancements, we provide brief yet detailed descriptions of each approach, supplemented by a meta-analysis comparing their efficacy in addressing the encountered challenges.

* As part of this survey we create an open-source repository<fnref n="4" /> where we host the existent FL-ST algorithms that are available as well as sample datasets. We also include available mobility data handling (pre-processing, analysis, and visualization) software. We hope that this open-source repository enables the research community to contribute their work and enable a greater number of baselines.

* We discuss open challenges and a roadmap that we envision for the research community to explore in the coming years. Some of the research questions that emerge from our survey are: *How can FL effectively achieve better personalization through the integration of semantic information?* (See Section 5.1) *What novel techniques and solutions can be developed to detect and mitigate Byzantine behaviors in FL-ST?* (See Section 5.2) *How can current ST mobility frameworks be optimized to enhance communication efficiency?* (See Section 5.3) *How does fairness of models behave in FL-ST and more importantly, how can it (i.e., fairness) be defined in this context?* (See Section 5.4) *How can the ST mobility research community establish and promote standardized practices?* (See Section 5.5) *How can realistic cross-silo spatial-temporal datasets be built in order to address ownership challenges and support benchmarking?* (See Section 5.6) and finally, what gaps must be addressed to facilitate a seamless transition towards real-world FL-ST applications? (See Section 5.7).

**Organization of this survey** The remainder of this paper is structured as follows. First, we present a background on FL in Section 2. Then, we present a set of spatial-temporal applications under study in Section 3 and detail the used FL approaches for these applications in Section 4. Finally, we discuss open research challenges in Section 5 and conclude the paper in Section 6.

## 2 Federated Learning : Preliminaries

*Federated Learning (FL)* is a paradigm to perform distributed machine learning at the edge [20, 80, 116, 87]. In FL, a global (joint) model is trained in a decentralized fashion without the need to collect and process user data centrally. The central service provider, often called aggregator, distributes a shared ML model to multiple users for training on local data, and then aggregates the resulting models into a single, more powerful model, using an aggregation method (e.g., Federated Averaging [116]).

More formally, let $\theta$ be the global model that an FL instance aims to learn and $F(\theta)$ be the cost function that evaluates $\theta$. Let $\mathcal{N} = \{1, 2, ..., n\}$ be the set of users, with each user $i$ possessing a local dataset $\mathcal{D}_i$. Note that we use 'users' and 'participants' interchangeably throughout the paper.

<footnote n="3">³Our methodology in harvesting data about all survey papers on FL is outlined in the Appendix.</footnote>

<footnote n="4">⁴https://github.com/YacineBelal/Survey-of-Federated-Learning-Models-for-Spatial-Temporal-Mobility-Applications</footnote>
growing number of application-specific FL surveys<fnref n="3" />, which is paralleled by an emergence of papers on FL-ST topics. In motivating the need for this survey, we discuss the perspectives that IoT-related and generic FL surveys have covered and highlight their shortcomings when applied to ST models. More specifically, FL Surveys on smart cities including those of IoT applications are closest to our work. Zheng et al. [199] reviewed applications of FL in smart cities. They provided a broad overview of various applications of smart cities including IoT, transportation, communication, medical care, and finance. They created a glossary of the papers in these domains without discussing the diversity of existing approaches or drawing comparisons between these works. Their work mainly focused on security and privacy challenges and outlined algorithmic efficiency as a future direction but did not consider the heterogeneity and sparsity often associated with the location data. The closest survey to ours is the work of Pandya et al. [131], which has a subsection discussing the FL transportation applications in smart cities. However, only one of the research works discussed in their survey is revisited in our work, as their focus predominantly revolves around a different type of use cases (e.g., number plate recognition, railway ... etc.).

Other FL surveys that are *not* application-specific and review methodological approaches of FL regarding heterogeneity, model convergence, and personalization fall short in drawing a comparison between these different techniques when applied to ST data. Although some of the challenges such as non-iid are well covered in those surveys, the other characteristics of the ST data are not covered. Finally, we note that despite its unique properties and the increasing number of spatial-temporal research papers emerging in FL, no survey paper has attempted to categorize, review, and compare this emergence of FL-ST methodological approaches (Figure 1).

**Our Contribution** The contributions of this survey are as follows:

* In this survey, we study 38 existing spatial-temporal works that have been tailored for the FL paradigm. As we review these recent advancements, we provide brief yet detailed descriptions of each approach, supplemented by a meta-analysis comparing their efficacy in addressing the encountered challenges.

* As part of this survey we create an open-source repository<fnref n="4" /> where we host the existent FL-ST algorithms that are available as well as sample datasets. We also include available mobility data handling (pre-processing, analysis, and visualization) software. We hope that this open-source repository enables the research community to contribute their work and enable a greater number of baselines.

* We discuss open challenges and a roadmap that we envision for the research community to explore in the coming years. Some of the research questions that emerge from our survey are: *How can FL effectively achieve better personalization through the integration of semantic information?* (See Section 5.1) *What novel techniques and solutions can be developed to detect and mitigate Byzantine behaviors in FL-ST?* (See Section 5.2) *How can current ST mobility frameworks be optimized to enhance communication efficiency?* (See Section 5.3) *How does fairness of models behave in FL-ST and more importantly, how can it (i.e., fairness) be defined in this context?* (See Section 5.4) *How can the ST mobility research community establish and promote standardized practices?* (See Section 5.5) *How can realistic cross-silo spatial-temporal datasets be built in order to address ownership challenges and support benchmarking?* (See Section 5.6) and finally, what gaps must be addressed to facilitate a seamless transition towards real-world FL-ST applications? (See Section 5.7).

**Organization of this survey** The remainder of this paper is structured as follows. First, we present a background on FL in Section 2. Then, we present a set of spatial-temporal applications under study in Section 3 and detail the used FL approaches for these applications in Section 4. Finally, we discuss open research challenges in Section 5 and conclude the paper in Section 6.

## 2 Federated Learning : Preliminaries

*Federated Learning (FL)* is a paradigm to perform distributed machine learning at the edge [20, 80, 116, 87]. In FL, a global (joint) model is trained in a decentralized fashion without the need to collect and process user data centrally. The central service provider, often called aggregator, distributes a shared ML model to multiple users for training on local data, and then aggregates the resulting models into a single, more powerful model, using an aggregation method (e.g., Federated Averaging [116]).

More formally, let $\theta$ be the global model that an FL instance aims to learn and $F(\theta)$ be the cost function that evaluates $\theta$. Let $\mathcal{N} = \{1, 2, \dots, n\}$ be the set of users, with each user $i$ possessing a local dataset $\mathcal{D}_i$. Note that we use ‘users’ and ‘participants’ interchangeably throughout the paper.

<footnote n="3">³Our methodology in harvesting data about all survey papers on FL is outlined in the Appendix.</footnote>

<footnote n="4">⁴https://github.com/YacineBelal/Survey-of-Federated-Learning-Models-for-Spatial-Temporal-Mobility-Applications</footnote>
<figure><img src="image_7.png" /><figcaption>Figure 2: Federated Learning general workflow. On the left, a cross-device setting where each client is a device (e.g., smartphone) and on the right is cross-silo setting where each client represents an organization. The main difference between the two is the magnitude and availability, both in terms of clients and local data.</figcaption></figure>

The aggregator initiates by randomly sampling $S = \{1, 2, \dots, m\}$ with $S \subset \mathcal{N}$ being a subset of users chosen to participate to a specific round $t$. The aggregator transmits the global model $\theta^t$ to $S$. Each selected participant $i$ trains $\theta^t$ on $\mathcal{D}_i$ to obtain an updated model, denoted $\theta_i^{t+1}$. Then, $i$ send the model updates $\nabla F(\theta_t, \mathcal{D}_i)$ (or $\theta_i^{t+1}$) back to the aggregator. Communication overhead can be reduced by applying a random mask to the model weights [87]. The central server then aggregates the received updates [141] to create the new global model, $\theta_i^{t+1}$. This round-based mechanism is repeated until some convergence criteria of $\theta$ is reached. Equation 1 presents a Federated Averaging-based aggregation step.

$$ \theta^{t+1} = \theta^t - \frac{\eta}{\sum_{i=1}^{m} |\mathcal{D}_i|} \sum_{i=1}^{m} |\mathcal{D}_i| \cdot \nabla F(\theta_t, \mathcal{D}_i) \quad (1) $$

Where $\eta$ denotes the global learning rate and $|\cdot|$ denotes the set cardinality operation.

This described algorithm represents the general FL workflow. However, it is essential to note that there exist several flavors of FL. Based on the order of magnitude of the number of participants and the assumptions regarding their computational capabilities, two FL settings are distinguished in this paper: cross-device FL (CD FL) and cross-silo FL (CS FL) [70].

CD FL typically refers to scenarios where the number of users is in the order of millions of individuals, each possessing limited and heterogeneous computational resources (e.g., personal laptops, cell phones). In contrast, cross-silo FL operates at a (much) smaller scale of the number of users, representing each a large entity (e.g., hospitals, cellular operators, and data centers), with substantial computational capabilities. Figure 2 presents an overview of both schemes of FL.

## 2.1 Challenges

There are various challenges in FL that the research community has been studying in the past years (see [80] for full reference). Two of these challenges impacting FL models are generally regarding **data heterogeneity** and **model personalization**.

* Data heterogeneity refers to the scenario where clients' data are not-iid which could lead to domain shift problems making learning a generalizable representation a difficult task. Consider two users $i$ and $j$, with data following probability distributions $P_i$ and $P_j$, respectively. These probabilities are typically rewritten as $P_i(Y|X)P_i(X)$ and $P_j(Y|X)P_j(X)$, respectively, where $X$ and $Y$ are the features' (e.g., visited POIs)
<figure><img src="image_7.png" /><figcaption>Figure 2: Federated Learning general workflow. On the left, a cross-device setting where each client is a device (e.g., smartphone) and on the right is cross-silo setting where each client represents an organization. The main difference between the two is the magnitude and availability, both in terms of clients and local data.</figcaption></figure>

The aggregator initiates by randomly sampling $S = \{1, 2, \dots, m\}$ with $S \subset \mathcal{N}$ being a subset of users chosen to participate to a specific round $t$. The aggregator transmits the global model $\theta^t$ to $S$. Each selected participant $i$ trains $\theta^t$ on $\mathcal{D}_i$ to obtain an updated model, denoted $\theta_i^{t+1}$. Then, $i$ send the model updates $\nabla F(\theta_t, \mathcal{D}_i)$ (or $\theta_i^{t+1}$) back to the aggregator. Communication overhead can be reduced by applying a random mask to the model weights [87]. The central server then aggregates the received updates [141] to create the new global model, $\theta_i^{t+1}$. This round-based mechanism is repeated until some convergence criteria of $\theta$ is reached. Equation 1 presents a Federated Averaging-based aggregation step.

$$ \theta^{t+1} = \theta^t - \frac{\eta}{\sum_{i=1}^{m} |\mathcal{D}_i|} \sum_{i=1}^{m} |\mathcal{D}_i| \cdot \nabla F(\theta_t, \mathcal{D}_i) \quad (1) $$

Where $\eta$ denotes the global learning rate and $|\cdot|$ denotes the set cardinality operation.

This described algorithm represents the general FL workflow. However, it is essential to note that there exist several flavors of FL. Based on the order of magnitude of the number of participants and the assumptions regarding their computational capabilities, two FL settings are distinguished in this paper: cross-device FL (CD FL) and cross-silo FL (CS FL) [70].

CD FL typically refers to scenarios where the number of users is in the order of millions of individuals, each possessing limited and heterogeneous computational resources (e.g., personal laptops, cell phones). In contrast, cross-silo FL operates at a (much) smaller scale of the number of users, representing each a large entity (e.g., hospitals, cellular operators, and data centers), with substantial computational capabilities. Figure 2 presents an overview of both schemes of FL.

## 2.1 Challenges

There are various challenges in FL that the research community has been studying in the past years (see [80] for full reference). Two of these challenges impacting FL models are generally regarding **data heterogeneity** and **model personalization**.

* Data heterogeneity refers to the scenario where clients' data are not-iid which could lead to domain shift problems making learning a generalizable representation a difficult task. Consider two users $i$ and $j$, with data following probability distributions $P_i$ and $P_j$, respectively. These probabilities are typically rewritten as $P_i(Y|X)P_i(X)$ and $P_j(Y|X)P_j(X)$, respectively, where $X$ and $Y$ are the features' (e.g., visited POIs)
and labels' (e.g., next POI) probability distributions. Considering this notation, data can be non-iid in several ways: the marginal distribution of features can vary between clients (i.e., $P_i(X) \neq P_j(X)$), such is the case for clients who have significantly different preferences in a POI recommendation use case. This is known as a covariate shift. Alternatively, the marginal distribution of labels can be different, also known as the prior distribution shift (i.e., $P_i(Y) \neq P_j(Y)$). This is especially the case for clients tied to different geo-regions. Finally, there can also be a concept shift, that is, the conditional distributions $P(X|Y)$ and $P(Y|X)$ can highly vary across clients and context (e.g., weather, time... etc). As we will discuss in the next sections, the human trajectory is extremely unique [33] and strongly influenced by different meta-parameters. This leads to the coexistence of different types of non-iid property and calls for an increased need for personalization.

*   **Model Personalization** on the other hand refers to the process of assigning different model parameters (weights) to different clients. The objective behind this is twofold: i) improve the global model's individual performance and ii) mitigate the impact of non-iid property. To this end, various methodologies have emerged. One approach involves incorporating context features into the model during training, known as personalization through featurization. Another technique, meta-learning, focuses on adapting the global model to the local learning task. In its simplest form, it boils down to fine-tuning the global model or a subset of it. Understanding what level of generalization can be learned globally and what layers of the models need to be personalized locally is an active area of research.

There are several other FL challenges that we consider throughout this paper. Firstly, concerns over data and model parameter privacy, which are addressed through methods such as Differential Privacy (DP) [118], Trusted Execution Environments (TEEs) [124] and encryption-based methods [152, 110, 69] (i). Secondly, Byzantine Resilience [59], defined as the capability to train an accurate statistical model despite arbitrary behaviors, presents another significant challenge (ii). Finally, Communication Efficiency stands as a crucial aspect to consider (iii).

In addition to these challenges, there are specific considerations within the realm of ST mobility applications. These include: 1) the constraints on client resources, 2) the online learning setting - specifically, whether clients train while simultaneously collecting/creating data, and 3) the comparison of solutions with at least one federated competitor, aside from standalone FL. This comprehensive evaluation framework has allowed us to conduct a meta-comparison, analyzing the strengths and limitations of the main works within each approach. Moreover, it has facilitated the identification of potential research directions (See Section 5).

## 2.2 Frameworks

As FL establishes itself as the standard machine learning paradigm, numerous frameworks have emerged to offer scalable and flexible solutions for privacy-preserving machine learning in distributed settings. Selecting an appropriate FL framework for designing new algorithms is a non-trivial task. Therefore, in this section, we define specific criteria to facilitate the selection process of an FL framework. The following criteria will be considered:

*   **Supported Frameworks:** Most FL frameworks can be extended to integrate with various machine learning frameworks. However, this process is often complex. Hence, we will outline the machine learning frameworks inherently supported by each FL framework and we will qualify by "native" each framework that provides its own optimization library.

*   **Privacy tools:** As privacy stands as a central motivation for FL, it is imperative to recognize that standalone FL lacks privacy [140, 80]. Within this context, we previously mentioned a formal model, DP, as well as hardware-based solutions like TEEs. Additionally, a range of privacy-enhancing technologies are based on cryptographic tools such as Secure Multi-Party Computation (SMPC) [152, 110], Homomorphic Encryption (HE) [69, 182] and Secret Sharing [150]. Evaluating which of these tools an FL framework supports is crucial due to the complexity of integrating them posthoc.

*   **Scenario:** This criterion assesses if the framework was primarily designed for real-world deployments, addressing physical environments, networking challenges, and scalability, or if its focus was solely on simulations.

*   **Benchmarking:** To enhance the efficiency of the research workflow, the majority of Federated Learning frameworks come equipped with predefined machine learning tasks, including models, datasets, and benchmarks. These tasks will be highlighted for each framework. It's important to note that while frameworks come with suggested tasks, they are not restricted solely to those tasks.

Table 1 presents a summary of the specified evaluation criteria for the main FL frameworks. Notably, most frameworks are extensively benchmarked for tasks in computer vision and Natural Language Processing (NLP), with relatively limited consideration for spatial mobility applications. *FedScale* [90] distinguishes itself by offering starting pack
and labels' (e.g., next POI) probability distributions. Considering this notation, data can be non-iid in several ways: the marginal distribution of features can vary between clients (i.e., $P_i(X) \neq P_j(X)$), such is the case for clients who have significantly different preferences in a POI recommendation use case. This is known as a covariate shift. Alternatively, the marginal distribution of labels can be different, also known as the prior distribution shift (i.e., $P_i(Y) \neq P_j(Y)$). This is especially the case for clients tied to different geo-regions. Finally, there can also be a concept shift, that is, the conditional distributions $P(X|Y)$ and $P(Y|X)$ can highly vary across clients and context (e.g., weather, time... etc). As we will discuss in the next sections, the human trajectory is extremely unique [33] and strongly influenced by different meta-parameters. This leads to the coexistence of different types of non-iid property and calls for an increased need for personalization.

*   **Model Personalization** on the other hand refers to the process of assigning different model parameters (weights) to different clients. The objective behind this is twofold: i) improve the global model's individual performance and ii) mitigate the impact of non-iid property. To this end, various methodologies have emerged. One approach involves incorporating context features into the model during training, known as personalization through featurization. Another technique, meta-learning, focuses on adapting the global model to the local learning task. In its simplest form, it boils down to fine-tuning the global model or a subset of it. Understanding what level of generalization can be learned globally and what layers of the models need to be personalized locally is an active area of research.

There are several other FL challenges that we consider throughout this paper. Firstly, concerns over data and model parameter privacy, which are addressed through methods such as Differential Privacy (DP) [118], Trusted Execution Environments (TEEs) [124] and encryption-based methods [152, 110, 69] (i). Secondly, Byzantine Resilience [59], defined as the capability to train an accurate statistical model despite arbitrary behaviors, presents another significant challenge (ii). Finally, Communication Efficiency stands as a crucial aspect to consider (iii).

In addition to these challenges, there are specific considerations within the realm of ST mobility applications. These include: 1) the constraints on client resources, 2) the online learning setting - specifically, whether clients train while simultaneously collecting/creating data, and 3) the comparison of solutions with at least one federated competitor, aside from standalone FL. This comprehensive evaluation framework has allowed us to conduct a meta-comparison, analyzing the strengths and limitations of the main works within each approach. Moreover, it has facilitated the identification of potential research directions (See Section 5).

## 2.2 Frameworks

As FL establishes itself as the standard machine learning paradigm, numerous frameworks have emerged to offer scalable and flexible solutions for privacy-preserving machine learning in distributed settings. Selecting an appropriate FL framework for designing new algorithms is a non-trivial task. Therefore, in this section, we define specific criteria to facilitate the selection process of an FL framework. The following criteria will be considered:

*   **Supported Frameworks:** Most FL frameworks can be extended to integrate with various machine learning frameworks. However, this process is often complex. Hence, we will outline the machine learning frameworks inherently supported by each FL framework and we will qualify by "native" each framework that provides its own optimization library.

*   **Privacy tools:** As privacy stands as a central motivation for FL, it is imperative to recognize that standalone FL lacks privacy [140, 80]. Within this context, we previously mentioned a formal model, DP, as well as hardware-based solutions like TEEs. Additionally, a range of privacy-enhancing technologies are based on cryptographic tools such as Secure Multi-Party Computation (SMPC) [152, 110], Homomorphic Encryption (HE) [69, 182] and Secret Sharing [150]. Evaluating which of these tools an FL framework supports is crucial due to the complexity of integrating them posthoc.

*   **Scenario:** This criterion assesses if the framework was primarily designed for real-world deployments, addressing physical environments, networking challenges, and scalability, or if its focus was solely on simulations.

*   **Benchmarking:** To enhance the efficiency of the research workflow, the majority of Federated Learning frameworks come equipped with predefined machine learning tasks, including models, datasets, and benchmarks. These tasks will be highlighted for each framework. It's important to note that while frameworks come with suggested tasks, they are not restricted solely to those tasks.

Table 1 presents a summary of the specified evaluation criteria for the main FL frameworks. Notably, most frameworks are extensively benchmarked for tasks in computer vision and Natural Language Processing (NLP), with relatively limited consideration for spatial mobility applications. *FedScale* [90] distinguishes itself by offering starting pack
<table><caption>Table 1: Available functionalities and bench-marking across different FL frameworks.</caption><thead><tr><th>Name</th><th>Supported Frameworks</th><th>Privacy tools</th><th>Scenario</th><th>Benchmarking</th></tr></thead><tbody><tr><td>Tensorflow Federated (TFF)</td><td>Tensorflow</td><td>DP<br>SMPC</td><td>Simulation</td><td>Computer Vision<br>NLP</td></tr><tr><td>Pysift & Pygrid</td><td>Tensorflow<br>Pytorch</td><td>DP<br>SMPC</td><td>Simulation<br>Real</td><td>Computer Vision</td></tr><tr><td>FATE</td><td>Tensorflow<br>Pytorch<br>Sickit-learn</td><td>HE</td><td>Simulation<br>Real</td><td>Computer Vision</td></tr><tr><td>Flower</td><td>Framework-agnostic</td><td>DP</td><td>Simulation</td><td>Computer Vision</td></tr><tr><td>PaddleFL</td><td>Native</td><td>DP<br>SMPC<br>Secret Sharing</td><td>Simulation<br>Real</td><td>Computer Vision<br>NLP<br>Recommendation</td></tr><tr><td>FederatedScope</td><td>Tensorflow<br>Pytorch</td><td>DP<br>SMPC</td><td>Simulation<br>Real</td><td>Computer Vision<br>NLP<br>Recommendation<br>Graph Learning</td></tr><tr><td>LEAF</td><td>Tensorflow</td><td>×</td><td>Simulation</td><td>Computer Vision<br>NLP</td></tr><tr><td>FedML</td><td>Framework-agnostic</td><td>SMPC</td><td>Simulation<br>Real</td><td>Computer Vision<br>NLP</td></tr><tr><td>FedScale</td><td>Tensorflow<br>Pytorch</td><td>DP<br>SMPC</td><td>Simulation<br>Real</td><td>Computer Vision<br>NLP<br>Recommendation<br>Mobility Prediction<sup>5</sup><br>RL</td></tr><tr><td>FS-Real</td><td>Tensorflow<br>Pytorch</td><td>×</td><td>Simulation<br>Real</td><td>Speech Recognition</td></tr><tr><td>MindSpore Federated</td><td>MindSpore</td><td>SMPC<br>LDP</td><td>Simulation<br>Real</td><td>Computer Vision<br>NLP<br>Recommendation</td></tr></tbody></table>

mobility datasets, including a task for taxi trajectory prediction utilizing the TaxiPorto dataset [120]. Additionally, it incorporates the Waymo Motion Dataset, comprising object trajectories and corresponding 3D maps for 103,354 scenes, utilized in autonomous driving scenarios.

A noteworthy feature is observed in FederatedScope [167], which introduces benchmarks for graph learning, potentially applicable to Graph Neural Network (GNN)-based approaches for flow prediction tasks.

Regarding machine learning framework support, it's notable that only two of the existing solutions offer a framework-agnostic approach to Federated Learning—namely, Flower [18] and FedML [65].

Considering privacy enhancements, most frameworks integrate DP and SMPC functionalities. However, LEAF [24] does not provide support for privacy-enhancing technologies. On the other hand, FATE [104] stands out by offering unique Homomorphic Encryption support.

In terms of compatibility with physical networking infrastructures, device heterogeneity, and client scalability—referred to as real-world scenarios- FedScale [90], FS-Real [51], and Mindspore Federated [190] emerge as the only frameworks capable of accommodating such requirements. Notably, FS-Real builds upon FedScale, enhancing its scalability to accommodate up to 100k devices and improving compatibility with heterogeneous hardware and software configurations across devices. However, the integration of FedScale's privacy tools into FS-Real remains unclear. In contrast, Mindspore Federated achieves comparable scalability by leveraging Elastic Load on the cloud, while incorporating both client-level DP (i.e., LDP) and SMPC techniques. However, it does require the usage of Mindspore's machine learning framework.

For an in-depth comparative analysis across the described FL frameworks, refer to [102]. Additionally, extensive comparisons of peer-to-peer FL frameworks have been conducted in [15].

<footnote n="5">⁵FedScale is the only framework that comes with an off-the-shelf mobility dataset benchmark.</footnote>
<table><caption>Table 1: Available functionalities and bench-marking across different FL frameworks.</caption><thead><tr><th>Name</th><th>Supported Frameworks</th><th>Privacy tools</th><th>Scenario</th><th>Benchmarking</th></tr></thead><tbody><tr><td>Tensorflow Federated (TFF)</td><td>Tensorflow</td><td>DP SMPC</td><td>Simulation</td><td>Computer Vision NLP</td></tr><tr><td>Pysift & Pygrid</td><td>Tensorflow Pytorch</td><td>DP SMPC</td><td>Simulation Real</td><td>Computer Vision</td></tr><tr><td>FATE</td><td>Tensorflow Pytorch Sickit-learn</td><td>HE</td><td>Simulation Real</td><td>Computer Vision</td></tr><tr><td>Flower</td><td>Framework-agnostic</td><td>DP</td><td>Simulation</td><td>Computer Vision</td></tr><tr><td>PaddleFL</td><td>Native</td><td>DP SMPC Secret Sharing</td><td>Simulation Real</td><td>Computer Vision NLP Recommendation</td></tr><tr><td>FederatedScope</td><td>Tensorflow Pytorch</td><td>DP SMPC</td><td>Simulation Real</td><td>Computer Vision NLP Recommendation Graph Learning</td></tr><tr><td>LEAF</td><td>Tensorflow</td><td>×</td><td>Simulation</td><td>Computer Vision NLP</td></tr><tr><td>FedML</td><td>Framework-agnostic</td><td>SMPC</td><td>Simulation Real</td><td>Computer Vision NLP</td></tr><tr><td>FedScale</td><td>Tensorflow Pytorch</td><td>DP SMPC</td><td>Simulation Real</td><td>Computer Vision NLP Recommendation Mobility Prediction <sup>5</sup> RL</td></tr><tr><td>FS-Real</td><td>Tensorflow Pytorch</td><td>×</td><td>Simulation Real</td><td>Speech Recognition</td></tr><tr><td>MindSpore Federated</td><td>MindSpore</td><td>SMPC LDP</td><td>Simulation Real</td><td>Computer Vision NLP Recommendation</td></tr></tbody></table>

mobility datasets, including a task for taxi trajectory prediction utilizing the TaxiPorto dataset [120]. Additionally, it incorporates the Waymo Motion Dataset, comprising object trajectories and corresponding 3D maps for 103,354 scenes, utilized in autonomous driving scenarios.

A noteworthy feature is observed in FederatedScope [167], which introduces benchmarks for graph learning, potentially applicable to Graph Neural Network (GNN)-based approaches for flow prediction tasks.

Regarding machine learning framework support, it's notable that only two of the existing solutions offer a framework-agnostic approach to Federated Learning—namely, Flower [18] and FedML [65].

Considering privacy enhancements, most frameworks integrate DP and SMPC functionalities. However, LEAF [24] does not provide support for privacy-enhancing technologies. On the other hand, FATE [104] stands out by offering unique Homomorphic Encryption support.

In terms of compatibility with physical networking infrastructures, device heterogeneity, and client scalability—referred to as real-world scenarios- FedScale [90], FS-Real [51], and Mindspore Federated [190] emerge as the only frameworks capable of accommodating such requirements. Notably, FS-Real builds upon FedScale, enhancing its scalability to accommodate up to 100k devices and improving compatibility with heterogeneous hardware and software configurations across devices. However, the integration of FedScale's privacy tools into FS-Real remains unclear. In contrast, Mindspore Federated achieves comparable scalability by leveraging Elastic Load on the cloud, while incorporating both client-level DP (i.e., LDP) and SMPC techniques. However, it does require the usage of Mindspore's machine learning framework.

For an in-depth comparative analysis across the described FL frameworks, refer to [102]. Additionally, extensive comparisons of peer-to-peer FL frameworks have been conducted in [15].

<footnote n="5">⁵FedScale is the only framework that comes with an off-the-shelf mobility dataset benchmark.</footnote>
# 3 Applications

**Mobility Prediction** Mobility prediction can be defined as algorithms and techniques to estimate the future locations of users. Predicting the next location of users can help with a range of applications including networking (e.g., handover management), pandemic management (e.g., contact tracing), etc. This type of prediction is performed on individual users' traces where the historical trend of the user's visited locations can help in predicting the likelihood of their next location.

**Transportation** With the rising availability of transportation data collected from various sensors like road cameras, GPS probes, and IoT devices, there is an enormous opportunity for city planners to leverage these types of data to facilitate various tasks such as traffic flow prediction. Different from *trajectory* data that records a sequence of locations and time in each trip, crowd flow data only have the start and end locations of a trip, and how many people flow in and out of a particular region can be counted. Indeed, traffic flow prediction using spatial-temporal data has been one of the main focuses of the research community (See comprehensive survey [77]). In the context of transportation, this problem is often considered as forecasting which is to predict traffic speed or traffic flow of regions or road segments based on historical aggregated mobility data.

**Community Detection** Community detection is an important aspect of urban planning as it allows researchers and planners to identify patterns and trends in human movement. By identifying groups of individuals or locations that are highly connected, researchers and planners can gain insight into how people move through a city, which can inform the design of transportation systems and urban spaces. Additionally, community detection can help identify areas of a city that are at risk of overcrowding or under-utilization, allowing for proactive measures to be taken to address these issues. The underlying enabler of identifying urban communities [54, 47] is spatial temporal data that presents the amount of time spent in different parts of the city. Researchers have shown that human mobility exhibits a strong degree of non-linearity [32] and models that rely on non-linear clustering algorithms, such as the one proposed by Ferreira et al. [46], to detect urban communities have been shown to outperform traditional approaches such as Principal Component Analysis (PCA), and Model-Based Clustering (MBC) and DB-SCAN techniques on a variety of centralized geospatial traces.

**Location Based Social Networks** LBSNs such as Foursquare and Flickr are social networks that use GPS features to locate the users and let the users broadcast their locations and other content from their mobile devices. LBSNs do not merely mean that the locations are added to the user-generated content (UGC) in social networks so that people can share their location information but also reshape the social structure among individual users that are connected by both their locations in the physical world and their location-tagged social media content in the virtual world. LBSNs contain a large number of user check-in data which consists of the instant locations of each user. Such social networks could also be thought of as the underlying application of Location-based recommendation systems.

# 4 Approaches

In this section we summarize FL-ST approaches in four main categories of i) trajectory predictive approaches which focus on the next-point prediction of user's trajectories, ii) traffic flow prediction approaches, iii) Top-K Location-based recommendation approaches and iv) other approaches. For each approach, we present the evaluation metrics, the datasets used and the various FL strategies. Table 2 provides an overview of the most important works.

## 4.1 Trajectory Predictive Approaches

Given the trajectory of a user, these approaches aim at predicting the next position. In a centralized setting where the training data containing all users' trajectories are available, Recurrent Neural Networks (RNN)-based approaches including Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU) can be broadly applied in dealing with trajectories and predictive tasks [99, 42, 52, 101]. As this type of data (i.e., trajectory) is highly privacy sensitive, one of the challenges that the research community has been focusing on is creating models that can be tuned for privacy and utility, namely, PUTs (Privacy-Utility Trade-off). Models such as [180, 139, 37] leverage the centralized training data to enhance the privacy level of the traces by reducing user's re-identification risk and at the same time optimizing the utility of the predictions (i.e., higher accuracy of next point predictions).

The challenges of trajectory predictive approaches in FL are different. Firstly, distributed/decentralized learning creates a heterogeneity problem, as different statistical distributions between clients can lead to a phenomenon called *client drift*, where clients' model updates/gradients have opposing directions, making their aggregate senseless [170]. As stated in the introduction, this challenge can, in principle, be mitigated to some extent [172]. However, the unique properties of
# 3 Applications

**Mobility Prediction** Mobility prediction can be defined as algorithms and techniques to estimate the future locations of users. Predicting the next location of users can help with a range of applications including networking (e.g., handover management), pandemic management (e.g., contact tracing), etc. This type of prediction is performed on individual users' traces where the historical trend of the user's visited locations can help in predicting the likelihood of their next location.

**Transportation** With the rising availability of transportation data collected from various sensors like road cameras, GPS probes, and IoT devices, there is an enormous opportunity for city planners to leverage these types of data to facilitate various tasks such as traffic flow prediction. Different from *trajectory* data that records a sequence of locations and time in each trip, crowd flow data only have the start and end locations of a trip, and how many people flow in and out of a particular region can be counted. Indeed, traffic flow prediction using spatial-temporal data has been one of the main focuses of the research community (See comprehensive survey [77]). In the context of transportation, this problem is often considered as forecasting which is to predict traffic speed or traffic flow of regions or road segments based on historical aggregated mobility data.

**Community Detection** Community detection is an important aspect of urban planning as it allows researchers and planners to identify patterns and trends in human movement. By identifying groups of individuals or locations that are highly connected, researchers and planners can gain insight into how people move through a city, which can inform the design of transportation systems and urban spaces. Additionally, community detection can help identify areas of a city that are at risk of overcrowding or under-utilization, allowing for proactive measures to be taken to address these issues. The underlying enabler of identifying urban communities [54, 47] is spatial temporal data that presents the amount of time spent in different parts of the city. Researchers have shown that human mobility exhibits a strong degree of non-linearity [32] and models that rely on non-linear clustering algorithms, such as the one proposed by Ferreira et al. [46], to detect urban communities have been shown to outperform traditional approaches such as Principal Component Analysis (PCA), and Model-Based Clustering (MBC) and DB-SCAN techniques on a variety of centralized geospatial traces.

**Location Based Social Networks** LBSNs such as Foursquare and Flickr are social networks that use GPS features to locate the users and let the users broadcast their locations and other content from their mobile devices. LBSNs do not merely mean that the locations are added to the user-generated content (UGC) in social networks so that people can share their location information but also reshape the social structure among individual users that are connected by both their locations in the physical world and their location-tagged social media content in the virtual world. LBSNs contain a large number of user check-in data which consists of the instant locations of each user. Such social networks could also be thought of as the underlying application of Location-based recommendation systems.

# 4 Approaches

In this section we summarize FL-ST approaches in four main categories of i) trajectory predictive approaches which focus on the next-point prediction of user's trajectories, ii) traffic flow prediction approaches, iii) Top-K Location-based recommendation approaches and iv) other approaches. For each approach, we present the evaluation metrics, the datasets used and the various FL strategies. Table 2 provides an overview of the most important works.

## 4.1 Trajectory Predictive Approaches

Given the trajectory of a user, these approaches aim at predicting the next position. In a centralized setting where the training data containing all users' trajectories are available, Recurrent Neural Networks (RNN)-based approaches including Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU) can be broadly applied in dealing with trajectories and predictive tasks [99, 42, 52, 101]. As this type of data (i.e., trajectory) is highly privacy sensitive, one of the challenges that the research community has been focusing on is creating models that can be tuned for privacy and utility, namely, PUTs (Privacy-Utility Trade-off). Models such as [180, 139, 37] leverage the centralized training data to enhance the privacy level of the traces by reducing user's re-identification risk and at the same time optimizing the utility of the predictions (i.e., higher accuracy of next point predictions).

The challenges of trajectory predictive approaches in FL are different. Firstly, distributed/decentralized learning creates a heterogeneity problem, as different statistical distributions between clients can lead to a phenomenon called *client drift*, where clients' model updates/gradients have opposing directions, making their aggregate senseless [170]. As stated in the introduction, this challenge can, in principle, be mitigated to some extent [172]. However, the unique properties of
people's mobility [33] as well as the different factors that dictate the local distributions of mobility data [192, 83, 100] (e.g., preferences, temporal, spatial, urban characteristics ... etc.) render generic techniques ineffective. Second and as a result of the first challenge, creating a global model for predicting the next location of users that works equally well for all the users becomes an extremely challenging task. That is, one must decide to what extent should clients adopt the global model and when to opt-in for a purely personalized model. In this section we review the existing works in this domain, review how they account for the mentioned challenges and more generic ones (See Table 3 for the meta-analysis). Finally, we compare their performances in Table 4 against centralized predictive approaches namely ST-RNN [101], MCARNN [99], DeepMove [42], and VANext [52].

<table><caption>Table 2: CD denotes Cross-Device approaches and CS for cross-silo</caption><thead><tr><th>Model</th><th>Year</th><th>Approach</th><th>Dataset</th><th>Federated Strategy</th></tr></thead><tbody><tr><td colspan="5"><strong>Trajectory Predictive Approaches</strong></td></tr><tr><td>Fan et al. [40]</td><td>2019</td><td>Transfer-learning</td><td>Private Mobile Phone Traces</td><td>CD</td></tr><tr><td>PMF [43]</td><td>2020</td><td></td><td>Foursquare [171], DenseGPS Twitter [183]</td><td>42, CD, Attack-resilient</td></tr><tr><td>STSAN [94]</td><td>2020</td><td>ST Attention Layer</td><td>Foursquare[171], Twitter [184], Yelp [105]</td><td>CD<br>Adaptive Model Fusion</td></tr><tr><td>Ezequiel et al.[39]</td><td>2022</td><td>GRU-Spatial and Flashback</td><td>Foursquare [171], Gowalla [27]</td><td>CD</td></tr><tr><td>STLPF [162]</td><td>2022</td><td>AutoEncoder with Global/Local attention</td><td>Foursquare[171]</td><td>CD</td></tr><tr><td colspan="5"><strong>Flow Predictive Approaches</strong></td></tr><tr><td>FedGRU [103]</td><td>2020</td><td>GRU</td><td>PeMS [25]</td><td>CS, FedAvg</td></tr><tr><td>FedTSE [178]</td><td>2022</td><td>Reinforcement Learning</td><td>England Freeway Dataset[36]</td><td>CS, FedAvg</td></tr><tr><td>FedSTN [177]</td><td>2022</td><td>GNN</td><td>Taxi-NYC [1], Taxi-BJ [75]</td><td>CS, Vertical FL</td></tr><tr><td>CNFGNN [121]</td><td>2021</td><td>GNN</td><td>PeMS-BAY [98], METRALA [75]</td><td>CS</td></tr><tr><td>CTFL [185]</td><td>2022</td><td>GNN</td><td>PeMSD4, PeMSD7</td><td>Clustered FL, CS</td></tr><tr><td>MVFF [38]</td><td>2022</td><td>GRU+GNN</td><td>Yelp [105], NY-Bike [2]</td><td>Vertical FL, CS</td></tr><tr><td colspan="5"><strong>Community Detection Approaches</strong></td></tr><tr><td>F-DEC [114]</td><td>2021</td><td>Deep Embedded Clustering</td><td>GeoLife [198]</td><td>CD, FedAvg</td></tr><tr><td colspan="5"><strong>Other</strong></td></tr><tr><td>EDEN [84]</td><td>2021</td><td></td><td>Privacy Optimization</td><td>CD</td></tr><tr><td>PREFER [60]</td><td>2021</td><td></td><td>Location Rec Sys</td><td>CD</td></tr><tr><td>PEPPER [14]</td><td>2022</td><td></td><td>Location Rec Sys</td><td>CD, Gossip Learning</td></tr><tr><td>MTSSFL [186]</td><td>2021</td><td></td><td>Transport Mode Inference</td><td>CD</td></tr><tr><td>Fed-DA [181]</td><td>2021</td><td></td><td>Network Traffic</td><td>CS</td></tr><tr><td>Fed-NTP [147]</td><td>2022</td><td></td><td>Network Traffic</td><td>CS</td></tr></tbody></table>

### 4.1.1 Metrics

Trajectory prediction models are commonly benchmarked on a handful of available datasets using Acc@K metric which is computed as an average of how many times the correct location was within the top-K predicted places (sorted by the model's predictions). For example, for an Acc@5 metric, the target (or actual output) is compared against a vector of the top-5 most probable locations output by the model. If the target is an element of the top-5 vector, the prediction is correct (or true positive). In practice, papers commonly report an Average Percentile Rank (APR).

### 4.1.2 Federated Trajectory Predictive Approaches

Table 3 offers a concise overview of the challenges addressed by the existing literature in trajectory predictive approaches. In this application domain, the non-iid nature stemming from a substantial number of users, each with unique characteristics, as well as privacy considerations, takes precedence over other challenges. Notably, while the former challenge appears to have been adequately addressed by the majority of the reviewed works, the latter has only been specifically tackled in [43]. In addition, it is noteworthy that most works have not taken into account federated
people's mobility [33] as well as the different factors that dictate the local distributions of mobility data [192, 83, 100] (e.g., preferences, temporal, spatial, urban characteristics ... etc.) render generic techniques ineffective. Second and as a result of the first challenge, creating a global model for predicting the next location of users that works equally well for all the users becomes an extremely challenging task. That is, one must decide to what extent should clients adopt the global model and when to opt-in for a purely personalized model. In this section we review the existing works in this domain, review how they account for the mentioned challenges and more generic ones (See Table 3 for the meta-analysis). Finally, we compare their performances in Table 4 against centralized predictive approaches namely ST-RNN [101], MCARNN [99], DeepMove [42], and VANext [52].

<table><caption>Table 2: CD denotes Cross-Device approaches and CS for cross-silo</caption><thead><tr><th>Model</th><th>Year</th><th>Approach</th><th>Dataset</th><th>Federated Strategy</th></tr></thead><tbody><tr><td colspan="5"><strong>Trajectory Predictive Approaches</strong></td></tr><tr><td>Fan et al. [40]</td><td>2019</td><td>Transfer-learning</td><td>Private Mobile Phone Traces</td><td>CD</td></tr><tr><td>PMF [43]</td><td>2020</td><td></td><td>Foursquare [171], DenseGPS Twitter [183]</td><td>42, CD, Attack-resilient</td></tr><tr><td>STSAN [94]</td><td>2020</td><td>ST Attention Layer</td><td>Foursquare[171], Twitter [184], Yelp [105]</td><td>CD<br>Adaptive Model Fusion</td></tr><tr><td>Ezequiel et al.[39]</td><td>2022</td><td>GRU-Spatial and Flashback</td><td>Foursquare [171], Gowalla [27]</td><td>CD</td></tr><tr><td>STLPF [162]</td><td>2022</td><td>AutoEncoder with Global/Local attention</td><td>Foursquare[171]</td><td>CD</td></tr><tr><td colspan="5"><strong>Flow Predictive Approaches</strong></td></tr><tr><td>FedGRU [103]</td><td>2020</td><td>GRU</td><td>PeMS [25]</td><td>CS, FedAvg</td></tr><tr><td>FedTSE [178]</td><td>2022</td><td>Reinforcement Learning</td><td>England Freeway Dataset[36]</td><td>CS, FedAvg</td></tr><tr><td>FedSTN [177]</td><td>2022</td><td>GNN</td><td>Taxi-NYC [1], Taxi-BJ [75]</td><td>CS, Vertical FL</td></tr><tr><td>CNFGNN [121]</td><td>2021</td><td>GNN</td><td>PeMS-BAY [98], METRALA [75]</td><td>CS</td></tr><tr><td>CTFL [185]</td><td>2022</td><td>GNN</td><td>PeMSD4, PeMSD7</td><td>Clustered FL, CS</td></tr><tr><td>MVFF [38]</td><td>2022</td><td>GRU+GNN</td><td>Yelp [105], NY-Bike [2]</td><td>Vertical FL, CS</td></tr><tr><td colspan="5"><strong>Community Detection Approaches</strong></td></tr><tr><td>F-DEC [114]</td><td>2021</td><td>Deep Embedded Clustering</td><td>GeoLife [198]</td><td>CD, FedAvg</td></tr><tr><td colspan="5"><strong>Other</strong></td></tr><tr><td>EDEN [84]</td><td>2021</td><td></td><td>Privacy Optimization</td><td>CD</td></tr><tr><td>PREFER [60]</td><td>2021</td><td></td><td>Location Rec Sys</td><td>CD</td></tr><tr><td>PEPPER [14]</td><td>2022</td><td></td><td>Location Rec Sys</td><td>CD, Gossip Learning</td></tr><tr><td>MTSSFL [186]</td><td>2021</td><td></td><td>Transport Mode Inference</td><td>CD</td></tr><tr><td>Fed-DA [181]</td><td>2021</td><td></td><td>Network Traffic</td><td>CS</td></tr><tr><td>Fed-NTP [147]</td><td>2022</td><td></td><td>Network Traffic</td><td>CS</td></tr></tbody></table>

### 4.1.1 Metrics

Trajectory prediction models are commonly benchmarked on a handful of available datasets using Acc@K metric which is computed as an average of how many times the correct location was within the top-K predicted places (sorted by the model's predictions). For example, for an Acc@5 metric, the target (or actual output) is compared against a vector of the top-5 most probable locations output by the model. If the target is an element of the top-5 vector, the prediction is correct (or true positive). In practice, papers commonly report an Average Percentile Rank (APR).

### 4.1.2 Federated Trajectory Predictive Approaches

Table 3 offers a concise overview of the challenges addressed by the existing literature in trajectory predictive approaches. In this application domain, the non-iid nature stemming from a substantial number of users, each with unique characteristics, as well as privacy considerations, takes precedence over other challenges. Notably, while the former challenge appears to have been adequately addressed by the majority of the reviewed works, the latter has only been specifically tackled in [43]. In addition, it is noteworthy that most works have not taken into account federated
competitors, preventing a comprehensive evaluation of their impact. This concern is pivotal not only for the current task but for all ST-FL, as elaborated in Section 5.5. Similar apprehensions extend to the robustness against Byzantine behaviors. However, this criterion holds less significance in the context of CD FL compared to the CS FL setting. In the following, we delve into a more comprehensive discussion of these works.

**Fan et al. [40]** proposed a federated attention-based personalized human mobility prediction. They apply a few-shot learning human mobility predictor that makes personalized predictions based on a few records for each user using an attention-based model. Furthermore, they take advantage of pre-training strategies where the predictor is trained on another smaller mobility dataset to accelerate the FL training on devices. However, even with the pre-training and attention-based strategy, the model requires over 1000 rounds of data communication rounds and is not sufficiently robust for the irregular nature of the human movement.

**STSAN [94]** Li et al. in 2020 proposed a cross-silo personalized next point prediction model named STSAN (Spatial-Temporal Self-Attention Network) which integrates AMF (Adaptive Model Fusion Federated Learning) for offering a mixture of a local and global model. The spatial attention layer allows for capturing the user's preference for geographic location, and temporal attention captures the user's temporal activity preference. To overcome the non-iid challenge, the AMF function enables the algorithm to learn specific personalization at each aggregation step on the FL server. The approach is evaluated on Foursquare, Twitter, and Yelp datasets. Table 4 reports its performance on the Foursquare dataset against centralized prediction approaches and shows the superior performance of 99% APR and 6% increase in Acc@1 compared to VANext [52].

**PMF [43]** Feng et al. proposed PMF, a privacy-preserving mobility prediction framework that uses FL to train general mobility models in a privacy-aware manner. In PMF, every participating device trains locally a representation of the global (centralized) model by using only the locally available dataset at each device. The framework also accounts for attack cases in the mobility prediction task and uses a group optimization algorithm on mobile devices to tackle these attacks. In the group optimization procedure, the whole model is divided into the risky group trained with protected data and the secure group trained with normal data. Furthermore, an efficient aggregation strategy based on robust convergence and an effective polling schema for fair client selection in the centralized server. The results of this model on DenseGPS and Twitter dataset show similar top-1 performance to DeepMove [42] but on Foursquare Tokyo it performs poorer than centralized baselines and other FL approaches.

**STLPF [162]** Wang et al. proposed a spatial-temporal location prediction framework (STLPF) where the next point prediction algorithm is trained based on a self-attention layer that enables information to be learned between long sequences in both local and global models. Furthermore, as part of the framework, the authors propose an approach that enables clients to cooperatively train their models in the absence of a global model. Their evaluation shows marginal improvement in APR on the Foursquare dataset when compared to the centralized approaches, and their approach performs similarly to STSAN [94] (See Table 4).

**Ezequiel et al. [39]** develop two implementations of GRU-Spatial and Flashback on FL for predicting the next location in human mobility. To the best of our knowledge, they are currently the only work that has worked on baselining these different approaches in an FL framework, namely Flower [18] and measuring the computational complexity of the model. They evaluate their model on Foursquare NY and Gowalla datasets.

Building upon our exploration of trajectory predictions, it is noteworthy to acknowledge a distinct but closely related research direction—mobility mode prediction. In this realm, the focus shifts towards understanding and dissecting individuals’ personal transportation modes, a facet that holds paramount importance in the efficient management of public transportation systems. Particularly in bustling metropolis cities marked by substantial populations and

<table><caption>Table 3: Summary of the challenges tackled by the reviewed Trajectory Predictive works. Observe that current works do not consider byzantine resilience.</caption><thead><tr><th></th><th>Privacy</th><th>Byzantine Resilience</th><th>Non-IIDness</th><th>Resource constraints</th><th>Overhead assessment</th><th>Federated competitors</th><th>Online Learning</th></tr></thead><tbody><tr><td>Fan et al. [40]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>STSAN [94]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>PMF [43]</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td></tr><tr><td>Ezequiel et al. [39]</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>STLPF [162]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td></tr></tbody></table>
competitors, preventing a comprehensive evaluation of their impact. This concern is pivotal not only for the current task but for all ST-FL, as elaborated in Section 5.5. Similar apprehensions extend to the robustness against Byzantine behaviors. However, this criterion holds less significance in the context of CD FL compared to the CS FL setting. In the following, we delve into a more comprehensive discussion of these works.

**Fan et al. [40]** proposed a federated attention-based personalized human mobility prediction. They apply a few-shot learning human mobility predictor that makes personalized predictions based on a few records for each user using an attention-based model. Furthermore, they take advantage of pre-training strategies where the predictor is trained on another smaller mobility dataset to accelerate the FL training on devices. However, even with the pre-training and attention-based strategy, the model requires over 1000 rounds of data communication rounds and is not sufficiently robust for the irregular nature of the human movement.

**STSAN [94]** Li et al. in 2020 proposed a cross-silo personalized next point prediction model named STSAN (Spatial-Temporal Self-Attention Network) which integrates AMF (Adaptive Model Fusion Federated Learning) for offering a mixture of a local and global model. The spatial attention layer allows for capturing the user's preference for geographic location, and temporal attention captures the user's temporal activity preference. To overcome the non-iid challenge, the AMF function enables the algorithm to learn specific personalization at each aggregation step on the FL server. The approach is evaluated on Foursquare, Twitter, and Yelp datasets. Table 4 reports its performance on the Foursquare dataset against centralized prediction approaches and shows the superior performance of 99% APR and 6% increase in Acc@1 compared to VANext [52].

**PMF [43]** Feng et al. proposed PMF, a privacy-preserving mobility prediction framework that uses FL to train general mobility models in a privacy-aware manner. In PMF, every participating device trains locally a representation of the global (centralized) model by using only the locally available dataset at each device. The framework also accounts for attack cases in the mobility prediction task and uses a group optimization algorithm on mobile devices to tackle these attacks. In the group optimization procedure, the whole model is divided into the risky group trained with protected data and the secure group trained with normal data. Furthermore, an efficient aggregation strategy based on robust convergence and an effective polling schema for fair client selection in the centralized server. The results of this model on DenseGPS and Twitter dataset show similar top-1 performance to DeepMove [42] but on Foursquare Tokyo it performs poorer than centralized baselines and other FL approaches.

**STLPF [162]** Wang et al. proposed a spatial-temporal location prediction framework (STLPF) where the next point prediction algorithm is trained based on a self-attention layer that enables information to be learned between long sequences in both local and global models. Furthermore, as part of the framework, the authors propose an approach that enables clients to cooperatively train their models in the absence of a global model. Their evaluation shows marginal improvement in APR on the Foursquare dataset when compared to the centralized approaches, and their approach performs similarly to STSAN [94] (See Table 4).

**Ezequiel et al. [39]** develop two implementations of GRU-Spatial and Flashback on FL for predicting the next location in human mobility. To the best of our knowledge, they are currently the only work that has worked on baselining these different approaches in an FL framework, namely Flower [18] and measuring the computational complexity of the model. They evaluate their model on Foursquare NY and Gowalla datasets.

Building upon our exploration of trajectory predictions, it is noteworthy to acknowledge a distinct but closely related research direction—mobility mode prediction. In this realm, the focus shifts towards understanding and dissecting individuals’ personal transportation modes, a facet that holds paramount importance in the efficient management of public transportation systems. Particularly in bustling metropolis cities marked by substantial populations and

<table><caption>Table 3: Summary of the challenges tackled by the reviewed Trajectory Predictive works. Observe that current works do not consider byzantine resilience.</caption><thead><tr><th></th><th>Privacy</th><th>Byzantine Resilience</th><th>Non-IIDness</th><th>Resource constraints</th><th>Overhead assessment</th><th>Federated competitors</th><th>Online Learning</th></tr></thead><tbody><tr><td>Fan et al. [40]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>STSAN [94]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>PMF [43]</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td></tr><tr><td>Ezequiel et al. [39]</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>STLPF [162]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td></tr></tbody></table>
<table><caption>Table 4: Baselines for next location prediction models</caption><thead><tr><th rowspan="2"></th><th colspan="2">Foursquare NY</th><th colspan="2">Foursquare Tokyo</th></tr><tr><th>Acc@1</th><th>APR</th><th>Acc@1</th><th>APR</th></tr></thead><tbody><tr><th colspan="5">Centralized Baselines</th></tr><tr><td>ST-RNN [101]</td><td>0.2633</td><td>0.9431</td><td>0.2567</td><td>0.9536</td></tr><tr><td>MCARNN [99]</td><td>0.3167</td><td>0.9595</td><td>0.2770</td><td>0.9532</td></tr><tr><td>DeepMOVE [42]</td><td>0.3010</td><td>0.9221</td><td>0.2668</td><td>0.9257</td></tr><tr><td>VaNext [52]</td><td>0.3627</td><td>0.9792</td><td>0.3436</td><td>0.9735</td></tr><tr><th colspan="5">Federated Approaches</th></tr><tr><td>STSAN [96]</td><td>0.4297</td><td>0.9902</td><td>0.3906</td><td>0.9847</td></tr><tr><td>STLPF [162]</td><td>0.4067</td><td>0.9893</td><td>0.3887</td><td>0.9856</td></tr><tr><td>PMF [43]</td><td>0.2407</td><td>0.6182</td><td>0.2310</td><td>0.5842</td></tr><tr><td>Ezequiel [39]</td><td>0.1133</td><td>NA</td><td>NA</td><td>NA</td></tr></tbody></table>

heavy traffic volumes [112]. Decoding users' traffic modes and aggregating traffic patterns on a broader scale can profoundly influence decision-making processes. For instance, it provides valuable insights for adaptively allocating public transportation resources, and strategically assigning more support during peak times or on congested streets to alleviate traffic burdens.

Within this avenue, the work presented by [174] takes center stage. They propose a federated VGG-like model expressly designed for predicting users' transportation modes, utilizing the GeoLife dataset. This distinctive approach contributes to the evolving landscape of mobility mode prediction, offering insights that extend beyond trajectory predictions and bear significance in refining resource allocation strategies and bolstering the overall management of public transportation systems. Mensah et al. [123] extends this setting by considering three different architectures (GRU, LSTM and 1D Convolutional NN (CNN)) trained simultaneously, obtaining three global models after convergence. These global models' outputs are combined through an ensemble learning technique (i.e., stacking) with a Multi Layer Perceptron (MLP) playing the role of the meta-learner. The objective of this MLP is to find the most confident label. (i.e., majority vote). The approach is tested against a federated version of each of the three architectures, showing clear improvement.

### 4.1.3 Datasets

The following mobility datasets are used to evaluate the trajectory predictive approaches in the research community:

* **Foursquare [171]:** It comprises two sub-datasets, Tokyo and New York data. The Tokyo dataset contains 0.5 million check-ins in Tokyo while the New York one contains over 0.2 million, both collected over a span of about 10 months (from 12 April 2012 to 16 February 2013). Each check-in includes an anonymized user ID, timestamp, and location information, e.g., GPS coordinates and semantic meaning (represented by fine-grained venue categories).

* **Twitter [184]:** It contains around 1.1 million geo-tagged tweets from Los Angeles. These tweets are collected from 1 August 2014 to 30 November 2014. Every geo-tagged tweet consists of four parts, e.g., an anonymized user ID, location information (GPS coordinates), timestamp, and the message published by the user. Compared with the other two platforms, Twitter data is very sparse when location service is not a frequently-used function for Twitter users.

* **Gowalla [27]:** It consists of two parts: a check-in dataset and a friendship network dataset. The check-in dataset contains over 6.4 million check-ins contributed by more than 196,000 users, collected over the period of February 2009 to October 2010. Similarly to Foursquare, each check-in includes an anonymized user ID, latitude and longitude coordinates, a timestamp, and a location ID. As for the friendship network dataset, it contains information about social relationships between users, represented by over 95,000 undirected edges.

* **Brightkite [27]:** This dataset is similar to Gowalla in that it includes check-in data and a friendship network. However, this dataset is significantly sparser, as check-ins were deliberately shared by users, leading to a sparser dataset. Quantitatively, the dataset includes nearly 4.5 million check-ins and 58,228 users, collected between April 2008 and October 2010.

* **Weeplaces [188]:** This dataset has been sourced from Weeplaces, a website that provides visual representations of users' check-in activities on location-based social networks (LBSNs). The platform has been integrated with other location-based social networking services, such as Facebook Places, Foursquare, and Gowalla, through APIs. This dataset contains 7,658,368 check-ins from 15,799 users, as well as their friends present on Weeplaces.
<table><caption>Table 4: Baselines for next location prediction models</caption><thead><tr><th rowspan="2"></th><th colspan="2">Foursquare NY</th><th colspan="2">Foursquare Tokyo</th></tr><tr><th>Acc@1</th><th>APR</th><th>Acc@1</th><th>APR</th></tr></thead><tbody><tr><td colspan="5">Centralized Baselines</td></tr><tr><td>ST-RNN [101]</td><td>0.2633</td><td>0.9431</td><td>0.2567</td><td>0.9536</td></tr><tr><td>MCARNN [99]</td><td>0.3167</td><td>0.9595</td><td>0.2770</td><td>0.9532</td></tr><tr><td>DeepMOVE [42]</td><td>0.3010</td><td>0.9221</td><td>0.2668</td><td>0.9257</td></tr><tr><td>VaNext [52]</td><td>0.3627</td><td>0.9792</td><td>0.3436</td><td>0.9735</td></tr><tr><td colspan="5">Federated Approaches</td></tr><tr><td>STSAN [96]</td><td>0.4297</td><td>0.9902</td><td>0.3906</td><td>0.9847</td></tr><tr><td>STLPF [162]</td><td>0.4067</td><td>0.9893</td><td>0.3887</td><td>0.9856</td></tr><tr><td>PMF [43]</td><td>0.2407</td><td>0.6182</td><td>0.2310</td><td>0.5842</td></tr><tr><td>Ezequiel [39]</td><td>0.1133</td><td>NA</td><td>NA</td><td>NA</td></tr></tbody></table>

heavy traffic volumes [112]. Decoding users' traffic modes and aggregating traffic patterns on a broader scale can profoundly influence decision-making processes. For instance, it provides valuable insights for adaptively allocating public transportation resources, and strategically assigning more support during peak times or on congested streets to alleviate traffic burdens.

Within this avenue, the work presented by [174] takes center stage. They propose a federated VGG-like model expressly designed for predicting users' transportation modes, utilizing the GeoLife dataset. This distinctive approach contributes to the evolving landscape of mobility mode prediction, offering insights that extend beyond trajectory predictions and bear significance in refining resource allocation strategies and bolstering the overall management of public transportation systems. Mensah et al. [123] extends this setting by considering three different architectures (GRU, LSTM and 1D Convolutional NN (CNN)) trained simultaneously, obtaining three global models after convergence. These global models' outputs are combined through an ensemble learning technique (i.e., stacking) with a Multi Layer Perceptron (MLP) playing the role of the meta-learner. The objective of this MLP is to find the most confident label. (i.e., majority vote). The approach is tested against a federated version of each of the three architectures, showing clear improvement.

### 4.1.3 Datasets

The following mobility datasets are used to evaluate the trajectory predictive approaches in the research community:

* **Foursquare [171]**: It comprises two sub-datasets, Tokyo and New York data. The Tokyo dataset contains 0.5 million check-ins in Tokyo while the New York one contains over 0.2 million, both collected over a span of about 10 months (from 12 April 2012 to 16 February 2013). Each check-in includes an anonymized user ID, timestamp, and location information, e.g., GPS coordinates and semantic meaning (represented by fine-grained venue categories).

* **Twitter [184]**: It contains around 1.1 million geo-tagged tweets from Los Angeles. These tweets are collected from 1 August 2014 to 30 November 2014. Every geo-tagged tweet consists of four parts, e.g., an anonymized user ID, location information (GPS coordinates), timestamp, and the message published by the user. Compared with the other two platforms, Twitter data is very sparse when location service is not a frequently-used function for Twitter users.

* **Gowalla [27]**: It consists of two parts: a check-in dataset and a friendship network dataset. The check-in dataset contains over 6.4 million check-ins contributed by more than 196,000 users, collected over the period of February 2009 to October 2010. Similarly to Foursquare, each check-in includes an anonymized user ID, latitude and longitude coordinates, a timestamp, and a location ID. As for the friendship network dataset, it contains information about social relationships between users, represented by over 95,000 undirected edges.

* **Brightkite [27]**: This dataset is similar to Gowalla in that it includes check-in data and a friendship network. However, this dataset is significantly sparser, as check-ins were deliberately shared by users, leading to a sparser dataset. Quantitatively, the dataset includes nearly 4.5 million check-ins and 58,228 users, collected between April 2008 and October 2010.

* **Weeplaces [188]**: This dataset has been sourced from Weeplaces, a website that provides visual representations of users' check-in activities on location-based social networks (LBSNs). The platform has been integrated with other location-based social networking services, such as Facebook Places, Foursquare, and Gowalla, through APIs. This dataset contains 7,658,368 check-ins from 15,799 users, as well as their friends present on Weeplaces.
<figure><img src="image_8.png" /><figcaption>(a) Horizontal entropy $H_u$ across different check-in datasets.</figcaption></figure>

<caption>(b) Vertical entropy $H_l$ across different check-in datasets.</caption>

<caption>Figure 3: Entropy Experiments.</caption>

<table><caption>Table 5: Global Statistics across check-in datasets.</caption><thead><tr><th></th><th>N<sub>u</sub></th><th>N<sub>l</sub></th><th>N<sub>c</sub></th><th>Ñ<sub>u</sub></th><th>Ñ<sub>l</sub></th></tr></thead><tbody><tr><td>Foursquare-Tokyo</td><td>2293</td><td>61858</td><td>573703</td><td>3.43</td><td>92.44</td></tr><tr><td>Foursquare-NYC</td><td>1083</td><td>38333</td><td>227428</td><td>2.37</td><td>84.05</td></tr><tr><td>Gowalla</td><td>107092</td><td>1280969</td><td>6442892</td><td>3.11</td><td>37.18</td></tr><tr><td>Brightkite</td><td>50686</td><td>772780</td><td>4491080</td><td>1.39</td><td>21.16</td></tr><tr><td>Weeplaces</td><td>15799</td><td>971308</td><td>7658368</td><td>2.72</td><td>166.64</td></tr></tbody></table>

* **GeoLife [198]**: It is a GPS trajectory dataset that was collected from 182 users over a span of five years (from April 2007 to August 2012). This dataset comprises 18,670 trajectories, each represented by a sequence of time-stamped points containing latitude, longitude, and altitude information. These trajectories capture a diverse range of users' outdoor movements, including routine activities (e.g., going home), as well as leisure activities (e.g., shopping, hiking, and cycling). Recently, 69 out of the 182 users have labeled their trajectories with transportation modes, such as driving, taking a bus, riding a bike, and walking. The labels for transportation mode are stored in a separate file for each user's folder.

* **Yelp [105]**: It is a collection of businesses, reviews, and user data extracted from the Yelp platform. This dataset is regularly updated and contains almost 7 million reviews of over 150,000 businesses located in 11 metropolitan areas across the United States and Canada. The data includes information on individual users such as their name, the number and nature of their reviews, and their list of friends. Additionally, the dataset also includes check-ins, which provide information about the frequency and duration of customer visits to businesses.

* **Priva'Mov [16]**: It comprises data collected from multifarious sensors, including WIFI, GPS, and Cellular, and contains around 286.7 million records, where each record is a timestamped trajectory point containing latitude, longitude,, and a user ID. It was collected from 100 users over a period of 15 months and primarily focuses on urban mobility around Lyon city of France.

* **DenseGPS [42]**: It is a dataset that includes private data from a major mobile application provider in China with 5000 users with one-month dense location records. This dataset is not available for the research community to use.
<figure><img src="image_8.png" /><figcaption>(a) Horizontal entropy $H_u$ across different check-in datasets.</figcaption></figure>

<caption>(b) Vertical entropy $H_l$ across different check-in datasets.</caption>

<caption>Figure 3: Entropy Experiments.</caption>

<table><caption>Table 5: Global Statistics across check-in datasets.</caption><thead><tr><th></th><th>N<sub>u</sub></th><th>N<sub>l</sub></th><th>N<sub>c</sub></th><th>Ñ<sub>u</sub></th><th>Ñ<sub>l</sub></th></tr></thead><tbody><tr><td>Foursquare-Tokyo</td><td>2293</td><td>61858</td><td>573703</td><td>3.43</td><td>92.44</td></tr><tr><td>Foursquare-NYC</td><td>1083</td><td>38333</td><td>227428</td><td>2.37</td><td>84.05</td></tr><tr><td>Gowalla</td><td>107092</td><td>1280969</td><td>6442892</td><td>3.11</td><td>37.18</td></tr><tr><td>Brightkite</td><td>50686</td><td>772780</td><td>4491080</td><td>1.39</td><td>21.16</td></tr><tr><td>Weeplaces</td><td>15799</td><td>971308</td><td>7658368</td><td>2.72</td><td>166.64</td></tr></tbody></table>

* **GeoLife [198]**: It is a GPS trajectory dataset that was collected from 182 users over a span of five years (from April 2007 to August 2012). This dataset comprises 18,670 trajectories, each represented by a sequence of time-stamped points containing latitude, longitude, and altitude information. These trajectories capture a diverse range of users' outdoor movements, including routine activities (e.g., going home), as well as leisure activities (e.g., shopping, hiking, and cycling). Recently, 69 out of the 182 users have labeled their trajectories with transportation modes, such as driving, taking a bus, riding a bike, and walking. The labels for transportation mode are stored in a separate file for each user's folder.

* **Yelp [105]**: It is a collection of businesses, reviews, and user data extracted from the Yelp platform. This dataset is regularly updated and contains almost 7 million reviews of over 150,000 businesses located in 11 metropolitan areas across the United States and Canada. The data includes information on individual users such as their name, the number and nature of their reviews, and their list of friends. Additionally, the dataset also includes check-ins, which provide information about the frequency and duration of customer visits to businesses.

* **Priva'Mov [16]**: It comprises data collected from multifarious sensors, including WIFI, GPS, and Cellular, and contains around 286.7 million records, where each record is a timestamped trajectory point containing latitude, longitude,, and a user ID. It was collected from 100 users over a period of 15 months and primarily focuses on urban mobility around Lyon city of France.

* **DenseGPS [42]**: It is a dataset that includes private data from a major mobile application provider in China with 5000 users with one-month dense location records. This dataset is not available for the research community to use.
### 4.1.4 Data sparsity and heterogeneity in trajectory prediction tasks

Mobility literature defines the highest potential accuracy of predictability of any individual, termed as “maximum predictability” ($Π_{max}$) [108]. Maximum predictability is defined by the entropy of information of a person’s trajectory (frequency, sequence of location visits, etc.). We adopt this measure to compute the non-iid property of the mobility traces. We use Shannon’s Entropy $H(x)$ to get a sense of both the sparsity and non-iidness of several check-in datasets. In this process, we adjust the metric proposed by [145] to extend the individual entropy of users (i.e., horizontal) with an individual entropy of point-of-interests (i.e., vertical). We argue that having these two dimensions for the entropy is necessary to draw conclusions w.r.t to sparsity and heterogeneity. To quantify this relationship, we first measure the horizontal entropy in the following manner:

$$H_u(x) = - \sum_{i=1}^{n} P_u(x_i) \log_2[P_u(x_i)] \qquad (2)$$

Where $n$ is the number of POIs and is the size of the probability vector. $P_u(x_i)$ is the probability of an individual user $u$ visiting location $x_i$ considering exclusively spatial pattern.

We compute the vertical entropy as follows:

$$H_l(x) = - \sum_{u \in U} P_u(l) \log_2[P_u(l)] \qquad (3)$$

Where $U$ is the set of all users. $P(x)$ is the probability of user $u$ visiting location $l$.

In Figure 3, a comparison of entropy levels is shown for different check-in datasets. Meanwhile, Table 5 provides a summary of more conventional statistics about these datasets. More specifically, $N_u$ denotes the number of users, $N_l$ denotes the number of POIs, $N_c$ denotes the number of check-ins, $\tilde{N}_u$ denotes the average number of users that visited each point of interest and $\tilde{N}_l$ denotes the average number of POIs visited by each user. Figure 3 seems to indicate that Brightkite has the least entropy, as expected. In fact, Subfigure 3a even shows that a significant proportion of Brightkite users have zero entropy, indicating that some users have only one check-in. This correlates with the high sparsity levels of the dataset (refer to Table 5). This downward trend is also evident when viewed through the lens of vertical entropy, although it may be less pronounced. Considering these observations, we see this dataset as an appropriate option to tackle the sparsity problem in distributed learning on mobility data. Conversely, Weeplaces exhibits two important trends: Firstly, there is a considerably higher level of horizontal entropy compared to what is seen on Brightkite, without a corresponding increase in vertical entropy. This suggests that in general users are more active and less predictable, but have not explored a much wider spectrum of points-of-interest. Secondly, Figure 3a exhibits a significant number of user outliers, particularly those with low entropy, indicating the presence of a short tail. These observations, correlated with the results of the table 5, which shows a high $N_l$ and a quite low $N_u$, indicate low heterogeneity levels within this dataset. Oppositely, Gowalla exhibits more horizontal entropy, which indicates that users are more active globally while also having a higher vertical entropy, indicating that users have substantially less in common than in WeePlaces. These two observations suggest a more heterogeneous, yet less sparse dataset. This makes Gowalla a perfect candidate for works tackling heterogeneity. However, the presence of a significant proportion of highly mobile outlier users should also be noted. Finally, Foursquare datasets show good entropy levels on both axes, making them an excellent choice for sanity tests.

## 4.2 Traffic Flow Prediction Approaches

Traffic Flow Prediction (TFP) is a problem with multifarious applications. For instance, it allows to mitigate traffic, evaluate air pollution, estimate travel time, and improve driving experience. Approaches in predicting the traffic patterns range from parametric approaches, which are based on statistical metrics (e.g., Auto-Regressive Integrated Moving Average (ARIMA) [56]), to more advanced ML models such as those based on RNNs and GNNs. For a comprehensive survey of deep learning models for traffic predictions in centralized settings refer to [77]. TFP approaches require slightly different settings in FL than trajectory prediction. Unlike individual predictive approaches where each client is assumed to correspond to an individual with their mobility traces, in flow predictive approaches, the clients are often considered to be entities or organizations that are maintaining their flow data private to comply with privacy regulations. Considering this, CS FL appears as a promising paradigm for ensuring that records are not shared outside of each organization/sensor. Moreover, as the number of clients is smaller, and the datasets sizes per client is greater, this configuration alleviates some of the non-iidness concerns. However, this paradigm shift also ushers in two pivotal challenges: First, as mentioned by Li et al. [97], flow conditions can be influenced by many random variables (e.g., daily schedules, weather, accidents), which makes it highly dynamic. As such, TFP demands real-time adaptation. Specifically, the FL process cannot afford to wait until a significant data pool accumulates. Implementing online
### 4.1.4 Data sparsity and heterogeneity in trajectory prediction tasks

Mobility literature defines the highest potential accuracy of predictability of any individual, termed as “maximum predictability” ($Π_{max}$) [108]. Maximum predictability is defined by the entropy of information of a person’s trajectory (frequency, sequence of location visits, etc.). We adopt this measure to compute the non-iid property of the mobility traces. We use Shannon’s Entropy $H(x)$ to get a sense of both the sparsity and non-iidness of several check-in datasets. In this process, we adjust the metric proposed by [145] to extend the individual entropy of users (i.e., horizontal) with an individual entropy of point-of-interests (i.e., vertical). We argue that having these two dimensions for the entropy is necessary to draw conclusions w.r.t to sparsity and heterogeneity. To quantify this relationship, we first measure the horizontal entropy in the following manner:

$$H_u(x) = - \sum_{i=1}^{n} P_u(x_i) \log_2[P_u(x_i)] \qquad (2)$$

Where $n$ is the number of POIs and is the size of the probability vector. $P_u(x_i)$ is the probability of an individual user $u$ visiting location $x_i$ considering exclusively spatial pattern.

We compute the vertical entropy as follows:

$$H_l(x) = - \sum_{u \in U} P_u(l) \log_2[P_u(l)] \qquad (3)$$

Where $U$ is the set of all users. $P(x)$ is the probability of user $u$ visiting location $l$.

In Figure 3, a comparison of entropy levels is shown for different check-in datasets. Meanwhile, Table 5 provides a summary of more conventional statistics about these datasets. More specifically, $N_u$ denotes the number of users, $N_l$ denotes the number of POIs, $N_c$ denotes the number of check-ins, $\tilde{N}_u$ denotes the average number of users that visited each point of interest and $\tilde{N}_l$ denotes the average number of POIs visited by each user. Figure 3 seems to indicate that Brightkite has the least entropy, as expected. In fact, Subfigure 3a even shows that a significant proportion of Brightkite users have zero entropy, indicating that some users have only one check-in. This correlates with the high sparsity levels of the dataset (refer to Table 5). This downward trend is also evident when viewed through the lens of vertical entropy, although it may be less pronounced. Considering these observations, we see this dataset as an appropriate option to tackle the sparsity problem in distributed learning on mobility data. Conversely, Weeplaces exhibits two important trends: Firstly, there is a considerably higher level of horizontal entropy compared to what is seen on Brightkite, without a corresponding increase in vertical entropy. This suggests that in general users are more active and less predictable, but have not explored a much wider spectrum of points-of-interest. Secondly, Figure 3a exhibits a significant number of user outliers, particularly those with low entropy, indicating the presence of a short tail. These observations, correlated with the results of the table 5, which shows a high $N_l$ and a quite low $N_u$, indicate low heterogeneity levels within this dataset. Oppositely, Gowalla exhibits more horizontal entropy, which indicates that users are more active globally while also having a higher vertical entropy, indicating that users have substantially less in common than in WeePlaces. These two observations suggest a more heterogeneous, yet less sparse dataset. This makes Gowalla a perfect candidate for works tackling heterogeneity. However, the presence of a significant proportion of highly mobile outlier users should also be noted. Finally, Foursquare datasets show good entropy levels on both axes, making them an excellent choice for sanity tests.

## 4.2 Traffic Flow Prediction Approaches

Traffic Flow Prediction (TFP) is a problem with multifarious applications. For instance, it allows to mitigate traffic, evaluate air pollution, estimate travel time, and improve driving experience. Approaches in predicting the traffic patterns range from parametric approaches, which are based on statistical metrics (e.g., Auto-Regressive Integrated Moving Average (ARIMA) [56]), to more advanced ML models such as those based on RNNs and GNNs. For a comprehensive survey of deep learning models for traffic predictions in centralized settings refer to [77]. TFP approaches require slightly different settings in FL than trajectory prediction. Unlike individual predictive approaches where each client is assumed to correspond to an individual with their mobility traces, in flow predictive approaches, the clients are often considered to be entities or organizations that are maintaining their flow data private to comply with privacy regulations. Considering this, CS FL appears as a promising paradigm for ensuring that records are not shared outside of each organization/sensor. Moreover, as the number of clients is smaller, and the datasets sizes per client is greater, this configuration alleviates some of the non-iidness concerns. However, this paradigm shift also ushers in two pivotal challenges: First, as mentioned by Li et al. [97], flow conditions can be influenced by many random variables (e.g., daily schedules, weather, accidents), which makes it highly dynamic. As such, TFP demands real-time adaptation. Specifically, the FL process cannot afford to wait until a significant data pool accumulates. Implementing online
learning strategies becomes imperative to continuously update the model with incoming data streams. Second, the units responsible of collecting the data (e.g., sensors) often grapple with limited computational and spatial resources [73]. As such, considering efficiency when designing solutions becomes crucial to ensure their practicality and adaptability to real world applications. In the following, we categorize TFP works into two groups, depending on the types of models employed: grid-based approaches relying on temporal models (e.g., RNNs, LSTM, etc.) and graph-based approaches employing GNNs. We first introduce the metrics, then compare the literature on a high level before diving deeper into each work. Finally, We describe the datasets used for this task.

## 4.2.1 Metrics

The forecasting performance for traffic flow predictions is commonly measured as a mean absolute error (MAE), mean square error (MSE), root MSE (RMSE), and mean absolute percentage error (MAPE).

## 4.2.2 Grid based Approaches

In grid-based approaches, the input data into the model is a sequence of flow data per location over time (e.g., average speed, number of vehicles). These approaches mainly focus on deep neural models such as RNN, GRU and LSTM to capture past historical traffic information as a predictor for future instances. Table 6 provides a comprehensive summary of the challenges addressed by the current body of literature in this category. Notably, our assessment reveals a notable gap in addressing the resource limitations inherent in sensor networks, encompassing constraints related to memory and computational capabilities. Moreover, with the exception of two notable works [119, 178], the overhead imposed by the proposed solutions remains largely unquantified.

Furthermore, the current array of works largely overlooks the critical aspect of robustness—neglecting to account for potential vulnerabilities wherein sensors may transmit compromised model updates due to malicious interference or system faults.

It is also imperative to highlight that, apart from the noteworthy contribution by Sepasgozar and Pierre [147], there remains a lack of comparative analyses with federated competitors beyond standalone FL—an aspect that deserves more attention in this field. A more comprehensive discussion of these shortcomings is expounded upon in Section 5.

<table><caption>Table 6: Summary of the challenges tackled by the reviewed Grid-Based works. The main takeaway is the absence of solutions tackling the resource constraints concerns.</caption><thead><tr><th></th><th>Privacy</th><th>Byzantine Resilience</th><th>Non-IIDness</th><th>Resource constraints</th><th>Overhead assessment</th><th>Federated competitors</th><th>Online Learning</th></tr></thead><tbody><tr><td>FedGRU[106]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>Akallouch et al. [5]</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>FedLSTM [153]</td><td>✓</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>BFRT[119]</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>FedTSE[178]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>Fed-NTP[147]</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td></tr><tr><td>Zeng et al. [179]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✓</td></tr></tbody></table>

**FedGRU [106]** In their work, Liu et al. [106] introduced a federated learning algorithm for highway flow prediction, leveraging a GRU. Additionally, they extended the client sampling phase of federated learning through a joint-announcement protocol. This protocol allows willing clients to announce their participation, strategically aiming to reduce communication costs associated with the FedAvg aggregation step. To address the challenges posed by non-iid data, especially in the context of highly heterogeneous locations, the authors suggested clustering clients based on their location information into *k* clusters, utilizing a constrained K-means approach [21]. This results in a model per cluster, and ensemble learning is then applied to identify the optimal subset of global models, enhancing prediction accuracy.

Their methodology was rigorously tested using real-world data from the Caltrans Performance Measurement System (PeMS) [25], comprising 39,000 individual sensors monitoring the freeway system in real-time across major metropolitan areas of California. The results demonstrated comparable performance to the centralized baseline of the GRU model and various other centralized models.

**Akallouch et al. [5]** in this work, the authors address privacy concerns in the context of TFP. They tackle this issue by introducing LDP to provide theoretically provable privacy guarantees. Their approach involves training an LSTM model in a federated fashion, with clients transmitting noisy gradients where the noise is sampled from a Gaussian
learning strategies becomes imperative to continuously update the model with incoming data streams. Second, the units responsible of collecting the data (e.g., sensors) often grapple with limited computational and spatial resources [73]. As such, considering efficiency when designing solutions becomes crucial to ensure their practicality and adaptability to real world applications. In the following, we categorize TFP works into two groups, depending on the types of models employed: grid-based approaches relying on temporal models (e.g., RNNs, LSTM, etc.) and graph-based approaches employing GNNs. We first introduce the metrics, then compare the literature on a high level before diving deeper into each work. Finally, We describe the datasets used for this task.

### 4.2.1 Metrics

The forecasting performance for traffic flow predictions is commonly measured as a mean absolute error (MAE), mean square error (MSE), root MSE (RMSE), and mean absolute percentage error (MAPE).

### 4.2.2 Grid based Approaches

In grid-based approaches, the input data into the model is a sequence of flow data per location over time (e.g., average speed, number of vehicles). These approaches mainly focus on deep neural models such as RNN, GRU and LSTM to capture past historical traffic information as a predictor for future instances. Table 6 provides a comprehensive summary of the challenges addressed by the current body of literature in this category. Notably, our assessment reveals a notable gap in addressing the resource limitations inherent in sensor networks, encompassing constraints related to memory and computational capabilities. Moreover, with the exception of two notable works [119, 178], the overhead imposed by the proposed solutions remains largely unquantified.

Furthermore, the current array of works largely overlooks the critical aspect of robustness—neglecting to account for potential vulnerabilities wherein sensors may transmit compromised model updates due to malicious interference or system faults.

It is also imperative to highlight that, apart from the noteworthy contribution by Sepasgozar and Pierre [147], there remains a lack of comparative analyses with federated competitors beyond standalone FL—an aspect that deserves more attention in this field. A more comprehensive discussion of these shortcomings is expounded upon in Section 5.

<table><caption>Table 6: Summary of the challenges tackled by the reviewed Grid-Based works. The main takeaway is the absence of solutions tackling the resource constraints concerns.</caption><thead><tr><th></th><th>Privacy</th><th>Byzantine Resilience</th><th>Non-IIDness</th><th>Resource constraints</th><th>Overhead assessment</th><th>Federated competitors</th><th>Online Learning</th></tr></thead><tbody><tr><td>FedGRU[106]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>Akallouch et al. [5]</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>FedLSTM [153]</td><td>✓</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>BFRT[119]</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>FedTSE[178]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>Fed-NTP[147]</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td></tr><tr><td>Zeng et al. [179]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✓</td></tr></tbody></table>

**FedGRU [106]** In their work, Liu et al. [106] introduced a federated learning algorithm for highway flow prediction, leveraging a GRU. Additionally, they extended the client sampling phase of federated learning through a joint-announcement protocol. This protocol allows willing clients to announce their participation, strategically aiming to reduce communication costs associated with the FedAvg aggregation step. To address the challenges posed by non-iid data, especially in the context of highly heterogeneous locations, the authors suggested clustering clients based on their location information into *k* clusters, utilizing a constrained K-means approach [21]. This results in a model per cluster, and ensemble learning is then applied to identify the optimal subset of global models, enhancing prediction accuracy.

Their methodology was rigorously tested using real-world data from the Caltrans Performance Measurement System (PeMS) [25], comprising 39,000 individual sensors monitoring the freeway system in real-time across major metropolitan areas of California. The results demonstrated comparable performance to the centralized baseline of the GRU model and various other centralized models.

**Akallouch et al. [5]** in this work, the authors address privacy concerns in the context of TFP. They tackle this issue by introducing LDP to provide theoretically provable privacy guarantees. Their approach involves training an LSTM model in a federated fashion, with clients transmitting noisy gradients where the noise is sampled from a Gaussian
distribution. Despite the common expectation of a performance drop with such privacy-preserving mechanisms, their experimental results defy this trend, revealing competitive performance comparable to centralized baselines.

**FedLSTM [153]** the authors strive to introduce both privacy and resilience guarantees into the context of TFP, operating under the distinctive assumption of computationally capable Roadside Units (RSUs) alongside traditional FL clients, represented by individual vehicles. In this scenario, the authors devise a protocol wherein clients engage with RSUs during each FL round, transmitting their model updates. These RSUs, equipped with uniform test sets, evaluate and validate the models through consensus mechanisms involving miners. The validated models are subsequently published on the blockchain, with local aggregation by clients occurring at a later stage.

To enhance privacy, the authors adopt a strategy akin to [5], leveraging LDP. However, they employ a distinct mechanism involving Laplacian noise, introduced to the count of vehicles in specific areas. The efficacy of this protocol is assessed using an LSTM and compared against various centralized baselines. Notably, the evaluation extends to scenarios with malicious users introducing poisoned models, showcasing the protocol's resilience due to the presence of verifiers. This work contributes a unique perspective by considering both privacy and resilience aspects in TFP.

**BFRT [119]** In this contribution, the authors also employ blockchain technology to enhance accountability and verification aspects within FL. Specifically, they utilize a permissioned blockchain, implemented using Hyperledger Fabric [9], to establish a framework where clients can only submit their model updates with permission from a designated group of verifiers (referred to as peers). Acceptance is contingent upon approval by a group of orderers through a consensus algorithm. While sharing a conceptual similarity with [153], the distinguishing factor lies in the unique assumptions of this work. Here, the peers and orderers are presumed to be under the ownership of an institution, such as a service provider or government, with RSUs serving the role of clients in this setup.

Notably, the authors do not explicitly outline the criteria used by the verifiers (peers) to validate models. Evaluation of this protocol is conducted on a Delaware Department of Transportation (DelDOT) dataset, encompassing both GRU and LSTM architectures. A key distinction of this work, in contrast to previous ones [82, 106, 5, 153], is its focus on real-time TFP, even within the experimental settings, as opposed to offline learning scenarios. This real-time consideration adds a practical dimension to their exploration of FL in the context of TFP.

**FedTSE [178]** In FedTSE, authors proposed a framework for Travel State Estimation (TSE). They consider an LSTM model as the local training model for joint prediction of vehicular speed and traffic flow. A unique characteristic of FedTSE is that it relies on a deep reinforcement learning (DRL)-based algorithm to adjust model parameter uploading/downloading decisions such that it improves the estimation accuracy of local models and balances the trade-off between computation and communication cost. They evaluate their approach on the England Freeway Dataset which includes flow and speed for the entire year of 2014. They also consider three different aggregation strategies corresponding to a synchronous aggregation with the same number of epochs per client, namely, *FedTSE-Syn*; an asynchronous aggregation with different numbers of epochs, namely, *FedTSE-Asyn* and a weighted version of FedTSE-Asyn where clients that have trained more are given more weight.

**Fed-NTP [147]** proposed an LSTM model implemented in FL to predict network traffic based on the most influential features of network traffic flow in the Vehicular Ad-Hoc Network (VANET). Even though this work is not aiming in predicting vehicle traffic and focuses on network flow, it has some relevance with the existing models such as FedGRU [106]. Fed-NTP shows to outperform FedGRU on the same V2V dataset. Other similar works that have focused on network traffic prediction such as FedDA [181] exist in the literature. However, as they solely focus on network traffic problems and are not intended for mobility applications, we do not review them in the survey. We encourage the reader to refer to [78] for a full survey of network traffic prediction techniques in FL.

**Zeng et al. [179]** To our knowledge, this work stands as the sole work introducing a multi-task learning method for the TFP challenge within the FL framework. The motivation behind this approach is to craft personalized models capable of adjusting to the diverse forms of heterogeneity previously outlined (See Section 2). The authors' methodology begins with a hierarchical clustering of clients' (i.e., sensors) local data based on various criteria such as weather conditions, time, and special events. This clustering ensures the presence of consistent clusters across clients. Subsequently, an LSTM model is trained in a federated manner for each cluster to capture temporal features and address time travel prediction. This process yields a dynamic graph delineating sub-segments connecting different stations, along with their respective travel times—this graph evolves over time. Interestingly, the authors propose a modification of the A\* algorithm to determine the optimal route utilizing this time-dependent graph. The experimental comparison with Google Maps' route prediction demonstrates the advantages of this approach. Nonetheless, the paper lacks explicit details on the clustering criteria employed and the mechanisms for client consensus regarding these clusters.
distribution. Despite the common expectation of a performance drop with such privacy-preserving mechanisms, their experimental results defy this trend, revealing competitive performance comparable to centralized baselines.

**FedLSTM [153]** the authors strive to introduce both privacy and resilience guarantees into the context of TFP, operating under the distinctive assumption of computationally capable Roadside Units (RSUs) alongside traditional FL clients, represented by individual vehicles. In this scenario, the authors devise a protocol wherein clients engage with RSUs during each FL round, transmitting their model updates. These RSUs, equipped with uniform test sets, evaluate and validate the models through consensus mechanisms involving miners. The validated models are subsequently published on the blockchain, with local aggregation by clients occurring at a later stage.

To enhance privacy, the authors adopt a strategy akin to [5], leveraging LDP. However, they employ a distinct mechanism involving Laplacian noise, introduced to the count of vehicles in specific areas. The efficacy of this protocol is assessed using an LSTM and compared against various centralized baselines. Notably, the evaluation extends to scenarios with malicious users introducing poisoned models, showcasing the protocol's resilience due to the presence of verifiers. This work contributes a unique perspective by considering both privacy and resilience aspects in TFP.

**BFRT [119]** In this contribution, the authors also employ blockchain technology to enhance accountability and verification aspects within FL. Specifically, they utilize a permissioned blockchain, implemented using Hyperledger Fabric [9], to establish a framework where clients can only submit their model updates with permission from a designated group of verifiers (referred to as peers). Acceptance is contingent upon approval by a group of orderers through a consensus algorithm. While sharing a conceptual similarity with [153], the distinguishing factor lies in the unique assumptions of this work. Here, the peers and orderers are presumed to be under the ownership of an institution, such as a service provider or government, with RSUs serving the role of clients in this setup.

Notably, the authors do not explicitly outline the criteria used by the verifiers (peers) to validate models. Evaluation of this protocol is conducted on a Delaware Department of Transportation (DelDOT) dataset, encompassing both GRU and LSTM architectures. A key distinction of this work, in contrast to previous ones [82, 106, 5, 153], is its focus on real-time TFP, even within the experimental settings, as opposed to offline learning scenarios. This real-time consideration adds a practical dimension to their exploration of FL in the context of TFP.

**FedTSE [178]** In FedTSE, authors proposed a framework for Travel State Estimation (TSE). They consider an LSTM model as the local training model for joint prediction of vehicular speed and traffic flow. A unique characteristic of FedTSE is that it relies on a deep reinforcement learning (DRL)-based algorithm to adjust model parameter uploading/downloading decisions such that it improves the estimation accuracy of local models and balances the trade-off between computation and communication cost. They evaluate their approach on the England Freeway Dataset which includes flow and speed for the entire year of 2014. They also consider three different aggregation strategies corresponding to a synchronous aggregation with the same number of epochs per client, namely, *FedTSE-Syn*; an asynchronous aggregation with different numbers of epochs, namely, *FedTSE-Asyn* and a weighted version of FedTSE-Asyn where clients that have trained more are given more weight.

**Fed-NTP [147]** proposed an LSTM model implemented in FL to predict network traffic based on the most influential features of network traffic flow in the Vehicular Ad-Hoc Network (VANET). Even though this work is not aiming in predicting vehicle traffic and focuses on network flow, it has some relevance with the existing models such as FedGRU [106]. Fed-NTP shows to outperform FedGRU on the same V2V dataset. Other similar works that have focused on network traffic prediction such as FedDA [181] exist in the literature. However, as they solely focus on network traffic problems and are not intended for mobility applications, we do not review them in the survey. We encourage the reader to refer to [78] for a full survey of network traffic prediction techniques in FL.

**Zeng et al. [179]** To our knowledge, this work stands as the sole work introducing a multi-task learning method for the TFP challenge within the FL framework. The motivation behind this approach is to craft personalized models capable of adjusting to the diverse forms of heterogeneity previously outlined (See Section 2). The authors' methodology begins with a hierarchical clustering of clients' (i.e., sensors) local data based on various criteria such as weather conditions, time, and special events. This clustering ensures the presence of consistent clusters across clients. Subsequently, an LSTM model is trained in a federated manner for each cluster to capture temporal features and address time travel prediction. This process yields a dynamic graph delineating sub-segments connecting different stations, along with their respective travel times—this graph evolves over time. Interestingly, the authors propose a modification of the A\* algorithm to determine the optimal route utilizing this time-dependent graph. The experimental comparison with Google Maps' route prediction demonstrates the advantages of this approach. Nonetheless, the paper lacks explicit details on the clustering criteria employed and the mechanisms for client consensus regarding these clusters.
<figure><img src="image_9.png" /><figcaption>Figure 4: Figure from FedTSE [178] showing the interactions between Edge Computing (EC) Server and Road Side Units (RSU) acting as a cross-silo unit. The LSTM model weights are aggregated using FedAvg, and a Deep Reinforcement Learning (DRL) Agent to maximize reward.</figcaption></figure>

<table><caption>Table 7: Number of parameters and performance of baseline centralized models on PEMSD7M [173] dataset.</caption><thead><tr><td>Model</td><td>Year</td><td>Number of Parameters</td><td>Performance (MAPE%)</td></tr></thead><tbody><tr><td>STGCN [173]</td><td>2017</td><td>330K</td><td>5.02%</td></tr><tr><td>MTGNN [166]</td><td>2020</td><td>433K</td><td>5.02%</td></tr><tr><td>DCRNN [98]</td><td>2017</td><td>610K</td><td>5.33%</td></tr></tbody></table>

### 4.2.3 GNN Based Approaches

State-of-the-art multi-layer GNNs effectively address the spatial-temporal nature of traffic prediction in centralized settings. However, when applied in FL scenarios, GNN-based approaches encounter specific challenges. Notably, vertical FL poses a hurdle where each silo possesses a partial, overlapping view of the graph.

Motivated by intelligent transportation systems, such as RSUs, individual units exhibit unique traffic patterns across a city (See Figure 4). The primary challenge in using GNN models for traffic forecasting within FL arises from their abundance of parameters, contrasting with simpler models like ARIMA with significantly fewer parameters (on the order of hundreds). Table 7 shows the magnitude of parameters for baseline traffic prediction models over the PEMSD7M dataset [173].

Moreover, the dynamic nature of these graphs over time demands adaptation through online learning, posing another significant challenge. Existing works aim to address these issues, as summarized in Table 8. While most studies tackle heterogeneity challenges, considerations regarding the cost of training and storing these complex models are rarely highlighted. Notably, [194] stands out by addressing these concerns, delegating the bulk of learning to a centralized server while clients handle and store manageable parameter subsets.

Furthermore, similar to grid-based approaches, the current literature lacks exploration into the models' robustness against faults and attacks that could compromise the learning process.

<table><caption>Table 8: Summary of the challenges tackled by the reviewed GNN-based works. Similarly to grid-based approaches, we note the absence of considerations w.r.t byzantine resilience.</caption><thead><tr><td></td><td>Privacy</td><td>Byzantine Resilience</td><td>Non-IIDness</td><td>Resource constraints</td><td>Overhead assessment</td><td>Federated competitors</td><td>Online Learning</td></tr></thead><tbody><tr><td>Fed-STGRU[82]</td><td>X</td><td>X</td><td>✓</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>FedAGCN[136]</td><td>X</td><td>X</td><td>✓</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>FedSTN[177]</td><td>X</td><td>X</td><td>X</td><td>X</td><td>✓</td><td>✓</td><td>X</td></tr><tr><td>CTFL[185]</td><td>X</td><td>X</td><td>✓</td><td>X</td><td>✓</td><td>X</td><td>X</td></tr><tr><td>GOF-TTE[194]</td><td>✓</td><td>X</td><td>✓</td><td>✓</td><td>X</td><td>✓</td><td>✓</td></tr></tbody></table>
<figure><img src="image_9.png" /><figcaption>Figure 4: Figure from FedTSE [178] showing the interactions between Edge Computing (EC) Server and Road Side Units (RSU) acting as a cross-silo unit. The LSTM model weights are aggregated using FedAvg, and a Deep Reinforcement Learning (DRL) Agent to maximize reward.</figcaption></figure>

<table><caption>Table 7: Number of parameters and performance of baseline centralized models on PEMSD7M [173] dataset.</caption><thead><tr><th>Model</th><th>Year</th><th>Number of Parameters</th><th>Performance (MAPE%)</th></tr></thead><tbody><tr><td>STGCN [173]</td><td>2017</td><td>330K</td><td>5.02%</td></tr><tr><td>MTGNN [166]</td><td>2020</td><td>433K</td><td>5.02%</td></tr><tr><td>DCRNN [98]</td><td>2017</td><td>610K</td><td>5.33%</td></tr></tbody></table>

### 4.2.3 GNN Based Approaches

State-of-the-art multi-layer GNNs effectively address the spatial-temporal nature of traffic prediction in centralized settings. However, when applied in FL scenarios, GNN-based approaches encounter specific challenges. Notably, vertical FL poses a hurdle where each silo possesses a partial, overlapping view of the graph.

Motivated by intelligent transportation systems, such as RSUs, individual units exhibit unique traffic patterns across a city (See Figure 4). The primary challenge in using GNN models for traffic forecasting within FL arises from their abundance of parameters, contrasting with simpler models like ARIMA with significantly fewer parameters (on the order of hundreds). Table 7 shows the magnitude of parameters for baseline traffic prediction models over the PEMSD7M dataset [173].

Moreover, the dynamic nature of these graphs over time demands adaptation through online learning, posing another significant challenge. Existing works aim to address these issues, as summarized in Table 8. While most studies tackle heterogeneity challenges, considerations regarding the cost of training and storing these complex models are rarely highlighted. Notably, [194] stands out by addressing these concerns, delegating the bulk of learning to a centralized server while clients handle and store manageable parameter subsets.

Furthermore, similar to grid-based approaches, the current literature lacks exploration into the models' robustness against faults and attacks that could compromise the learning process.

<table><caption>Table 8: Summary of the challenges tackled by the reviewed GNN-based works. Similarly to grid-based approaches, we note the absence of considerations w.r.t byzantine resilience.</caption><thead><tr><th></th><th>Privacy</th><th>Byzantine Resilience</th><th>Non-IIDness</th><th>Resource constraints</th><th>Overhead assessment</th><th>Federated competitors</th><th>Online Learning</th></tr></thead><tbody><tr><td>Fed-STGRU[82]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>FedAGCN[136]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>FedSTN[177]</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>CTFL[185]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td></tr><tr><td>GOF-TTE[194]</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td></tr></tbody></table>
**DST-GCN** [160] The authors parallel FedGRU [106] by adopting a GRU model to predict temporal dependencies in TFP. However, they extend this approach by integrating a Graph Convolutional Network (GCN) to capture spatial features, recognizing the GCN's aptitude for modeling inter-spatial characteristics. Augmenting this fusion, an attention mechanism is employed to identify and elevate critical spatio-temporal dependencies that dynamically evolve. Notably, the study promotes a client participation scheme based on both willingness and the evaluation of clients' models prior to aggregation. Their experimentation with the PeMS dataset substantiates that this client selection strategy significantly enhances performance compared to traditional FL.

**Fed-STGRU** [82] Similar to DST-GCN [160], this work propose a combination of a GRU and a GCN to solve TFP. Moreover, the authors also focus on improving the clients selection process of FL through a clustering strategy, where they opt for the Fuzzy c-means algorithm. This alternative is highlighted as less computationally demanding compared to the constrained k-means proposed in [106].

In terms of performance, this work demonstrates superior results compared to a standalone FedAvg algorithm, showcasing the effectiveness of their hybrid GRU-GCN model and the efficiency of Fuzzy k-means for client clustering in the context of traffic flow prediction.

**FedAGCN** [136] In this work, the authors propose a comprehensive solution to address the efficiency of GCNs in handling spatial correlations, a critical aspect of traffic flow data. Previous methods, particularly those not based on graph models, rely on individual clients learning on their local view of the system, leading to a neglect of spatial associations of the traffic network's topology. To overcome this limitation, the authors introduce GraphFed, a carefully devised algorithm aimed at enhancing communication efficiency while preserving the ability to capture spatial correlations. The key innovation lies in the division of the graph network into sub-graphs, with representative clients randomly selected for each sub-graph. These representatives aggregate data from all clients within their respective sub-graphs, facilitating the classification of the model's parameters into global and local categories, the latter being tailored specifically to spatial relationships.

Importantly, the practicality of GraphFed is evident as it mitigates the need to store the entire traffic network at the client level, a cumbersome requirement for large traffic networks. By operating on disjoint regions of the traffic network, representative clients efficiently avoid the necessity of sharing local parameters with the FL server, ensuring that only global parameters are learned in a federated manner, thereby minimizing communication overhead. The authors also adopt the ADGCN model [137], enhancing its efficiency by refining the convolution operation. This meticulous approach guarantees the adeptness of the model in addressing spatial correlations and overcoming challenges associated with FL on a broad scale within traffic networks.

**FedSTN** [177] The FedSTN approach as formalized by Yuan et. al is a newly proposed solution to solving the TFP problem. To accomplish this task, authors proposed a Graph-based Representation Learning for CS FL using three main modules: a Recurrent Long-term capture Network (RLCN) module, an Attentive Mechanism Federated Network (AMFN) Module, and a Semantic Capture Network module (SCN). The RLCN module is responsible for learning long-term traffic behaviors as geometric time series data of fixed long-term interval *p*. The data as input into this module is comprised of inflow-outflow values for certain grid spaces, as initially computed for the data.

The AFMN module is responsible for learning short-term spatial-temporal features in a privacy-preserving manner. This module includes long-term contextual data such as meteorology and federated graph attention (FedGAT). Lastly, there is the SCN module, which takes into account Point of Interest Feature Components (POI) and non-euclidean connection relationships. Points of interest and their interactions have significant effects on TFP but are not incorporated into the raw time series prediction data. Furthermore, transportation networks also have methods of connection outside adjacent-grid connections, such as trains, highways, etc. These flows also have an important effect on TFP that is not incorporated elsewhere, so this module is to address this issue. The output of all these modules is then connected via an FC layer followed by a Tanh activation function for the final output.

**CNFGNN** [121] Meng et al. proposed a new architecture named Cross Node Federated Graph Neural Network (CNFGNN) to predict the flow. CNFGNN works by decomposing the problem into two stages: first, it uses an encoder-decoder network to extract temporal features locally (See right side of Figure 5), and then a GNN to capture spatial relations across devices, as is illustrated in the left side of Figure 5. On each training step, the server processes a temporal encoding update, a partial gradient update, and an on-node graph embedding update. Each iteration updates the client side weights and then ships the model, hidden layer, and gradients back to the server.

One large issue of the proposed approach is the communication overhead in the training stage, with split learning requiring the global model to fetch all hidden states from each node, and ship gradients of node embeddings to each node. Then, it must receive gradients of node embedding and send gradients of hidden states in the back-propagation
**DST-GCN** [160] The authors parallel FedGRU [106] by adopting a GRU model to predict temporal dependencies in TFP. However, they extend this approach by integrating a Graph Convolutional Network (GCN) to capture spatial features, recognizing the GCN's aptitude for modeling inter-spatial characteristics. Augmenting this fusion, an attention mechanism is employed to identify and elevate critical spatio-temporal dependencies that dynamically evolve. Notably, the study promotes a client participation scheme based on both willingness and the evaluation of clients' models prior to aggregation. Their experimentation with the PeMS dataset substantiates that this client selection strategy significantly enhances performance compared to traditional FL.

**Fed-STGRU** [82] Similar to DST-GCN [160], this work propose a combination of a GRU and a GCN to solve TFP. Moreover, the authors also focus on improving the clients selection process of FL through a clustering strategy, where they opt for the Fuzzy c-means algorithm. This alternative is highlighted as less computationally demanding compared to the constrained k-means proposed in [106].

In terms of performance, this work demonstrates superior results compared to a standalone FedAvg algorithm, showcasing the effectiveness of their hybrid GRU-GCN model and the efficiency of Fuzzy k-means for client clustering in the context of traffic flow prediction.

**FedAGCN** [136] In this work, the authors propose a comprehensive solution to address the efficiency of GCNs in handling spatial correlations, a critical aspect of traffic flow data. Previous methods, particularly those not based on graph models, rely on individual clients learning on their local view of the system, leading to a neglect of spatial associations of the traffic network's topology. To overcome this limitation, the authors introduce GraphFed, a carefully devised algorithm aimed at enhancing communication efficiency while preserving the ability to capture spatial correlations. The key innovation lies in the division of the graph network into sub-graphs, with representative clients randomly selected for each sub-graph. These representatives aggregate data from all clients within their respective sub-graphs, facilitating the classification of the model's parameters into global and local categories, the latter being tailored specifically to spatial relationships.

Importantly, the practicality of GraphFed is evident as it mitigates the need to store the entire traffic network at the client level, a cumbersome requirement for large traffic networks. By operating on disjoint regions of the traffic network, representative clients efficiently avoid the necessity of sharing local parameters with the FL server, ensuring that only global parameters are learned in a federated manner, thereby minimizing communication overhead. The authors also adopt the ADGCN model [137], enhancing its efficiency by refining the convolution operation. This meticulous approach guarantees the adeptness of the model in addressing spatial correlations and overcoming challenges associated with FL on a broad scale within traffic networks.

**FedSTN** [177] The FedSTN approach as formalized by Yuan et. al is a newly proposed solution to solving the TFP problem. To accomplish this task, authors proposed a Graph-based Representation Learning for CS FL using three main modules: a Recurrent Long-term capture Network (RLCN) module, an Attentive Mechanism Federated Network (AMFN) Module, and a Semantic Capture Network module (SCN). The RLCN module is responsible for learning long-term traffic behaviors as geometric time series data of fixed long-term interval *p*. The data as input into this module is comprised of inflow-outflow values for certain grid spaces, as initially computed for the data.

The AFMN module is responsible for learning short-term spatial-temporal features in a privacy-preserving manner. This module includes long-term contextual data such as meteorology and federated graph attention (FedGAT). Lastly, there is the SCN module, which takes into account Point of Interest Feature Components (POI) and non-euclidean connection relationships. Points of interest and their interactions have significant effects on TFP but are not incorporated into the raw time series prediction data. Furthermore, transportation networks also have methods of connection outside adjacent-grid connections, such as trains, highways, etc. These flows also have an important effect on TFP that is not incorporated elsewhere, so this module is to address this issue. The output of all these modules is then connected via an FC layer followed by a Tanh activation function for the final output.

**CNFGNN** [121] Meng et al. proposed a new architecture named Cross Node Federated Graph Neural Network (CNFGNN) to predict the flow. CNFGNN works by decomposing the problem into two stages: first, it uses an encoder-decoder network to extract temporal features locally (See right side of Figure 5), and then a GNN to capture spatial relations across devices, as is illustrated in the left side of Figure 5. On each training step, the server processes a temporal encoding update, a partial gradient update, and an on-node graph embedding update. Each iteration updates the client side weights and then ships the model, hidden layer, and gradients back to the server.

One large issue of the proposed approach is the communication overhead in the training stage, with split learning requiring the global model to fetch all hidden states from each node, and ship gradients of node embeddings to each node. Then, it must receive gradients of node embedding and send gradients of hidden states in the back-propagation
<figure><img src="image_10.png" /><figcaption>Figure 5: Figure from CNGNN [121] presents (a) the server-side graph neural network with a systematic overview of training steps: (1) Federated learning of on-node models. (2) Temporal encoding update. (3) Split Learning of GN. (b) Client *i* Auto-encoder architecture.</figcaption></figure>

step. To mitigate this, they propose an alternative training approach in which the temporal encoder-decoder and the node embeddings are trained separately. First, the node embeddings are fixed and optimization is performed on the encoder-decoder. Then, after a fixed interval, the global model is updated fixing the node-level models. This drastically reduces communication overhead. The FL local models and the GNN model with only a local objective function. They perform alternating optimization to update clients' model weights with GNN model weights fixed and then update GNN model weights with the FL local model weights fixed, over multiple rounds.

CTFL [185] proposes a Clustering-based hierarchical and Two-step-optimized FL (CTFL), to overcome the large number of parameters that are needed for aggregation in the GNN-based models such as STGCN [173], DCRNN [98] and MTGNN [166]. CTFL employs a divide-and-conquer strategy, clustering clients based on the closeness of their local model parameters. It also accounts for optimization by applying a two-step strategy where the central server uploads only one representative local model update from each cluster, thus reducing the communication overhead associated with model update transmission in the FL.

GOF-TTE [194] This work addresses a different formulation of the TFP problem, namely, Time Travel Estimation (TTE), where the task is to predict the travel time needed to go from a point *a* to a point *b* considering spatial features (*e.g.*, road network map) and temporal features (*e.g.*, time of the day). [194] tackles this challenge within the context of taxi-driving scenarios, with potential applications to diverse driving types. In this work, generic spatio-temporal features are used to learn the global state of the network in a federated manner, allowing to obtain a global model. Subsequently, each client fine-tunes this model in locally (*i.e.*, Localized global model), before incorporating a personalized model trained on the clients' profile features.

The input architecture involves a dual graph representing road segments and intersections. These elements are translated into latent representations (*i.e.*, embeddings) before being passed to a GCN layer, allowing for the capture of spatial dependencies. Spatial and temporal representations are then subjected to a cross-product operation. Temporal aspects, such as day of the week and time of the day, are encoded, with an attention mechanism considering both current and past temporal impacts. This cross-product results in a global state of the road network.

In a subsequent phase, the global state is fine-tuned using clients' local data before being obfuscated with Laplacian noise through the DP mechanism. A personalized model is later introduced, comprising a fully connected layer of client profile feature embeddings (*e.g.*, frequently visited regions, average driving distance). This personalized model is integrated with the output of the localized global model to mitigate biases arising from the high heterogeneity of the data.

Notably, this work distinguishes itself from prior research by acknowledging the relationship between road segments and intersections, in contrast to road-segment-based solutions for instance. Furthermore, it introduces a novel personalization technique that accounts for the current global state of the network, enabling real-time personalized predictions—a combination of considerations not concurrently addressed in existing literature.

Finally, it is worth noting that other works similar to flow prediction exists which are focused on identifying travel modality inference (TMI). For instance [186] proposes MTSSFL which trains a deep neural network ensemble under a novel semi-supervised FL framework. It achieves a highly accurate score for a crowd-sourced TMI without depending on the availability of massive labeled data.

### 4.2.4 Datasets

For flow predictive approaches the following datasets are widely used to benchmark the comparison of various algorithms.
<figure><img src="image_10.png" /><figcaption>Figure 5: Figure from CNGNN [121] presents (a) the server-side graph neural network with a systematic overview of training steps: (1) Federated learning of on-node models. (2) Temporal encoding update. (3) Split Learning of GN. (b) Client *i* Auto-encoder architecture.</figcaption></figure>

step. To mitigate this, they propose an alternative training approach in which the temporal encoder-decoder and the node embeddings are trained separately. First, the node embeddings are fixed and optimization is performed on the encoder-decoder. Then, after a fixed interval, the global model is updated fixing the node-level models. This drastically reduces communication overhead. The FL local models and the GNN model with only a local objective function. They perform alternating optimization to update clients' model weights with GNN model weights fixed and then update GNN model weights with the FL local model weights fixed, over multiple rounds.

CTFL [185] proposes a Clustering-based hierarchical and Two-step-optimized FL (CTFL), to overcome the large number of parameters that are needed for aggregation in the GNN-based models such as STGCN [173], DCRNN [98] and MTGNN [166]. CTFL employs a divide-and-conquer strategy, clustering clients based on the closeness of their local model parameters. It also accounts for optimization by applying a two-step strategy where the central server uploads only one representative local model update from each cluster, thus reducing the communication overhead associated with model update transmission in the FL.

GOF-TTE [194] This work addresses a different formulation of the TFP problem, namely, Time Travel Estimation (TTE), where the task is to predict the travel time needed to go from a point *a* to a point *b* considering spatial features (*e.g.*, road network map) and temporal features (*e.g.*, time of the day). [194] tackles this challenge within the context of taxi-driving scenarios, with potential applications to diverse driving types. In this work, generic spatio-temporal features are used to learn the global state of the network in a federated manner, allowing to obtain a global model. Subsequently, each client fine-tunes this model in locally (*i.e.*, Localized global model), before incorporating a personalized model trained on the clients' profile features.

The input architecture involves a dual graph representing road segments and intersections. These elements are translated into latent representations (*i.e.*, embeddings) before being passed to a GCN layer, allowing for the capture of spatial dependencies. Spatial and temporal representations are then subjected to a cross-product operation. Temporal aspects, such as day of the week and time of the day, are encoded, with an attention mechanism considering both current and past temporal impacts. This cross product results in a global state of the road network.

In a subsequent phase, the global state is fine-tuned using clients' local data before being obfuscated with Laplacian noise through the DP mechanism. A personalized model is later introduced, comprising a fully connected layer of client profile feature embeddings (*e.g.*, frequently visited regions, average driving distance). This personalized model is integrated with the output of the localized global model to mitigate biases arising from the high heterogeneity of the data.

Notably, this work distinguishes itself from prior research by acknowledging the relationship between road segments and intersections, in contrast to road-segment-based solutions for instance. Furthermore, it introduces a novel personalization technique that accounts for the current global state of the network, enabling real-time personalized predictions—a combination of considerations not concurrently addressed in existing literature.

Finally, it is worth noting that other works similar to flow prediction exists which are focused on identifying travel modality inference (TMI). For instance [186] proposes MTSSFL which trains a deep neural network ensemble under a novel semi-supervised FL framework. It achieves a highly accurate score for a crowd-sourced TMI without depending on the availability of massive labeled data.

### 4.2.4 Datasets

For flow predictive approaches the following datasets are widely used to benchmark the comparison of various algorithms.
* **Taxi Datasets:** The following datasets contain taxi in-out flow data collected via GPS, which include pick-up time, drop-off time, and trip distance.

- **TaxiBJ [189]** contains data collected in Beijing from four time intervals: 2013/7/1 to 2013/10/30, 2014/3/1 to 2014/6/30, 2015/3/1 to 2015/6/30, and 2015/11/1 to 2016/4/10. The data was collected at 30-minute intervals and include trajectories of over 34,000 taxis. It additionally contains meteorology data such as weather conditions, temperature, and wind speed.

- **TaxiNYC [1]** contains data collected in New York City, which was collected between 2016/4/1 and 2016/6/30 and contains over 35 million records. A more extensive version of this dataset, collected from 2009 to 2022, is also available.

- **TaxiPorto [120]** comprises a full year (from 2013/7/1 to 2014/6/30) of trajectories for all the 442 taxis running in the city of Porto, Portugal. It also contains information on the type of taxi call: central-based, stand-based, and demanded on a random street.

- **T-Drive [197]** contains GPS traces collected in Beijing, China. The dataset was collected over a period of three years from 2008 to 2011, and it consists of over 10,000 taxi drivers' GPS trajectories. The dataset contains a total of 1.07 billion GPS points, covering approximately 150 million kilometers. That being said, the sample released by Microsoft is over a span of one week only, containing around 15 million GPS points and covering a total distance of trajectories that reaches 9 million kilometers.

* **METR-LA [75]:** is Los Angeles traffic collected using 207 sensors mounted around highways, and 1515 edges from 2012/3/1 to 2012/6/30.

* **PeMS** is a traffic flow dataset collected from California Transportation Agencies Performance Measurement System (PeMS).

- **PeMS-BAY [98]:** It contains 325 nodes (traffic sensors) and 2369 edges in the Bay Area from 2017/1/1 to 2017/5/31.

- **PeMSD7M:** It is a sub-sample of PeMS published as part of [173], also collected from PeMS. It covers 228 traffic sensors with a 5-minute sampling rate corresponding to 2012/5/1 to 2012/6/30.

* **NY-Bike [2]:** Spanning a period of ten years, from June 2013 to January 2023, this dataset includes comprehensive information about daily bike orders by people in New York City and is regularly updated. More specifically, it contains information about bike trips, such as duration, starting and ending point, and location, as well as details about bikers, including user type, gender, and year of birth.

* **Highways England network journey [35]:** This dataset consists of average journey time, speed and traffic flow information for 15-minute periods since April 2015 on all motorways and 'A' roads managed by Highways England.

* **DelDOT dataset [119]:** represents roadway traffic data collected by the Delaware Department of Transportation at a 5-minute time frequency.

## 4.3 Top-K Location-Based Recommendation Approaches

Location-based recommendation task, while resembling trajectory prediction, often entails lower demands. This task involve identifying a top-K set of relevant Points of Interest (POIs) for a user based on their historical POIs. In Table 9, we summarize the challenges addressed by the community for this use case. Notably, none of the reviewed works have considered an online learning setting. It can be argued that this task may not necessarily require online learning since the preferences of users might not significantly change over time, unlike the trajectory prediction task, which inherently encompasses a natural time dimension. However, we argue that exploring online learning could still be valuable. Moreover, recent research has demonstrated that users' preferences tend to evolve over time. While other challenges have been reasonably explored, it's noteworthy that several works have based their privacy solutions on not sharing all model parameters, but recent studies on similar models have shown this approach not to be entirely private [191, 176]. This is indicated in the table using the "≈" symbol. Finally, similar to previous approaches/tasks, the exploration of robustness has been lacking. This is unfortunate, given that issues like shilling attacks [71] are significant concerns for recommendation systems and represent just one type of robustness concern not explored in this federated setting. Other types include malicious users fabricating fake profiles to manipulate recommendations, promote specific POIs, and introduce unfairness.

Given that the evaluation metrics and datasets align with those used in trajectory prediction approaches, the rest of this section focuses solely on the main works in this domain.
* **Taxi Datasets:** The following datasets contain taxi in-out flow data collected via GPS, which include pick-up time, drop-off time, and trip distance.

- **TaxiBJ [189]** contains data collected in Beijing from four time intervals: 2013/7/1 to 2013/10/30, 2014/3/1 to 2014/6/30, 2015/3/1 to 2015/6/30, and 2015/11/1 to 2016/4/10. The data was collected at 30-minute intervals and include trajectories of over 34,000 taxis. It additionally contains meteorology data such as weather conditions, temperature, and wind speed.

- **TaxiNYC [1]** contains data collected in New York City, which was collected between 2016/4/1 and 2016/6/30 and contains over 35 million records. A more extensive version of this dataset, collected from 2009 to 2022, is also available.

- **TaxiPorto [120]** comprises a full year (from 2013/7/1 to 2014/6/30) of trajectories for all the 442 taxis running in the city of Porto, Portugal. It also contains information on the type of taxi call: central-based, stand-based, and demanded on a random street.

- **T-Drive [197]** contains GPS traces collected in Beijing, China. The dataset was collected over a period of three years from 2008 to 2011, and it consists of over 10,000 taxi drivers' GPS trajectories. The dataset contains a total of 1.07 billion GPS points, covering approximately 150 million kilometers. That being said, the sample released by Microsoft is over a span of one week only, containing around 15 million GPS points and covering a total distance of trajectories that reaches 9 million kilometers.

* **METR-LA [75]**: is Los Angeles traffic collected using 207 sensors mounted around highways, and 1515 edges from 2012/3/1 to 2012/6/30.

* **PeMS** is a traffic flow dataset collected from California Transportation Agencies Performance Measurement System (PeMS).

- **PeMS-BAY [98]**: It contains 325 nodes (traffic sensors) and 2369 edges in the Bay Area from 2017/1/1 to 2017/5/31.

- **PeMSD7M:** It is a sub-sample of PeMS published as part of [173], also collected from PeMS. It covers 228 traffic sensors with a 5-minute sampling rate corresponding to 2012/5/1 to 2012/6/30.

* **NY-Bike [2]**: Spanning a period of ten years, from June 2013 to January 2023, this dataset includes comprehensive information about daily bike orders by people in New York City and is regularly updated. More specifically, it contains information about bike trips, such as duration, starting and ending point, and location, as well as details about bikers, including user type, gender, and year of birth.

* **Highways England network journey [35]**: This dataset consists of average journey time, speed and traffic flow information for 15-minute periods since April 2015 on all motorways and 'A' roads managed by Highways England.

* **DelDOT dataset [119]**: represents roadway traffic data collected by the Delaware Department of Transportation at a 5-minute time frequency.

## 4.3 Top-K Location-Based Recommendation Approaches

Location-based recommendation task, while resembling trajectory prediction, often entails lower demands. This task involve identifying a top-K set of relevant Points of Interest (POIs) for a user based on their historical POIs. In Table 9, we summarize the challenges addressed by the community for this use case. Notably, none of the reviewed works have considered an online learning setting. It can be argued that this task may not necessarily require online learning since the preferences of users might not significantly change over time, unlike the trajectory prediction task, which inherently encompasses a natural time dimension. However, we argue that exploring online learning could still be valuable. Moreover, recent research has demonstrated that users' preferences tend to evolve over time. While other challenges have been reasonably explored, it's noteworthy that several works have based their privacy solutions on not sharing all model parameters, but recent studies on similar models have shown this approach not to be entirely private [191, 176]. This is indicated in the table using the "≈" symbol. Finally, similar to previous approaches/tasks, the exploration of robustness has been lacking. This is unfortunate, given that issues like shilling attacks [71] are significant concerns for recommendation systems and represent just one type of robustness concern not explored in this federated setting. Other types include malicious users fabricating fake profiles to manipulate recommendations, promote specific POIs, and introduce unfairness.

Given that the evaluation metrics and datasets align with those used in trajectory prediction approaches, the rest of this section focuses solely on the main works in this domain.
<table><caption>Table 9: Summary of the challenges tackled by the reviewed Top-K POI recommendation works. Byzantine resilience and online learning have not been tackled by current works.</caption><thead><tr><th></th><th>Privacy</th><th>Byzantine Resilience</th><th>Non-IIDness</th><th>Resource constraints</th><th>Overhead assessment</th><th>Federated competitors</th><th>Online Learning</th></tr></thead><tbody><tr><td>FPL [10]</td><td>≈</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td></tr><tr><td>DFL-PC [61]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>PREFER [60]</td><td>≈</td><td>✗</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>MVFF [38]</td><td>≈</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>PriRec[26]</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td></tr><tr><td>PEPPER[14]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>FedPOIRec[134]</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td></tr></tbody></table>

**FPL [10]** This work extends the bayesian pair wise ranking (BPR) [143] algorithm to the federated setting. The authors train the sensitive user embeddings locally and provide users with the option to share these embeddings with a certain probability, while training the less sensitive parameters in a federated manner. This approach can be viewed as a flexible framework for factorization models, where clients can decide how many parameters to share while maintaining convergence of the model. They evaluate their approach on Foursquare dataset, considering different countries and levels of sparsity and compare themselves with a federated movies recommender [7].

**DFL-PC [61]** This work focuses on optimizing the aggregation process in federated learning. The authors pre-train deep models consisting of a GCN and a GRU connected to an MLP, which are used to estimate the system parameter space. These estimates are then optimized at the server level through a reinforcement learning algorithm. The evaluation results show that their solution surpasses traditional centralized deep models due to the added optimization step based on reinforcement learning.

In terms of system architecture, alternative solutions have been proposed for more efficient federated POI recommendation. The following works fall within this scope:

**PREFER [60]** Guo et al. proposed a two-set recommendation model training. First, for privacy purposes, they propose to train the user-dependent model parameters strictly locally. Subsequently, user share and aggregate less sensitive parameters (i.e., user-independent) in a multiple-edge server architecture, instead of remote cloud servers, with an aim to improve real-time response capability and reduce communication cost. They validate their approach both analytically and empirically on two standard POI recommendation models, namely, Distance2Pre [28] and PRME-G [44], and two check-in datasets, Foursquare [171] and Gowalla [27], and show the competitiveness of their approach with centralized and federated competitors.

**MVFF [38]** proposes a vertical federated learning framework for mobility data forecasting for CS FL applications where each organization holds a partial subset of data. Using a local learning model, each organization extracts the embedded spatio-temporal correlation between its locations. To account for global learning, a global model synchronizes with the local models to incorporate the correlation between all the organizations' locations.

**PriRec [26]** proposes a peer-to-peer approach to learn sensitive user embeddings, while less sensitive ones(e.g., feature interaction model) are learned in a federated manner. This is achieved through the introduction of secret sharing [148] in the decentralized gradient descent (DGD) topology and considering geographical information when building this topology, that is, users closely geo-located learn (privately) collaboratively their respective user embeddings. As for the items' features, they are aggregated through a secure aggregation protocol (SA) [76]. Authors evaluate their approach on standard POI datasets' and compare it with a centralized factorization machine (FM), which it seems to compete with, while guaranteeing a better level of privacy.

**PEPPER [14]** emphasizes personalization of POI models through the aggregation step while using a gossip communication protocol to eliminate the central FL server. In this work, nodes gossip their models with their neighbors and aggregate them after evaluating their contribution. The authors also introduce a peer-sampling protocol that acts as a clustering over time, ensuring each node has similar users in its neighborhood. Results from their experiments show that fully decentralized FL can be competitive with centralized solutions, while offering scalability and personalization.

**FedPOIRec [134]** introduces a framework that leverages social relationships between users to make more personalized models. The authors first train a global model using FL and SMPC to protect the aggregation phase from a curious
<table><caption>Table 9: Summary of the challenges tackled by the reviewed Top-K POI recommendation works. Byzantine resilience and online learning have not been tackled by current works.</caption><thead><tr><th></th><th>Privacy</th><th>Byzantine Resilience</th><th>Non-IIDness</th><th>Resource constraints</th><th>Overhead assessment</th><th>Federated competitors</th><th>Online Learning</th></tr></thead><tbody><tr><td>FPL [10]</td><td>≈</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td></tr><tr><td>DFL-PC [61]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>PREFER [60]</td><td>≈</td><td>✗</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>MVFF [38]</td><td>≈</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>PriRec[26]</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td></tr><tr><td>PEPPER[14]</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>FedPOIRec[134]</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td></tr></tbody></table>

**FPL [10]** This work extends the bayesian pair wise ranking (BPR) [143] algorithm to the federated setting. The authors train the sensitive user embeddings locally and provide users with the option to share these embeddings with a certain probability, while training the less sensitive parameters in a federated manner. This approach can be viewed as a flexible framework for factorization models, where clients can decide how many parameters to share while maintaining convergence of the model. They evaluate their approach on Foursquare dataset, considering different countries and levels of sparsity and compare themselves with a federated movies recommender [7].

**DFL-PC [61]** This work focuses on optimizing the aggregation process in federated learning. The authors pre-train deep models consisting of a GCN and a GRU connected to an MLP, which are used to estimate the system parameter space. These estimates are then optimized at the server level through a reinforcement learning algorithm. The evaluation results show that their solution surpasses traditional centralized deep models due to the added optimization step based on reinforcement learning.

In terms of system architecture, alternative solutions have been proposed for more efficient federated POI recommendation. The following works fall within this scope:

**PREFER [60]** Guo et al. proposed a two-set recommendation model training. First, for privacy purposes, they propose to train the user-dependent model parameters strictly locally. Subsequently, user share and aggregate less sensitive parameters (i.e., user-independent) in a multiple-edge server architecture, instead of remote cloud servers, with an aim to improve real-time response capability and reduce communication cost. They validate their approach both analytically and empirically on two standard POI recommendation models, namely, Distance2Pre [28] and PRME-G [44], and two check-in datasets, Foursquare [171] and Gowalla [27], and show the competitiveness of their approach with centralized and federated competitors.

**MVFF [38]** proposes a vertical federated learning framework for mobility data forecasting for CS FL applications where each organization holds a partial subset of data. Using a local learning model, each organization extracts the embedded spatio-temporal correlation between its locations. To account for global learning, a global model synchronizes with the local models to incorporate the correlation between all the organizations' locations.

**PriRec [26]** proposes a peer-to-peer approach to learn sensitive user embeddings, while less sensitive ones(e.g., feature interaction model) are learned in a federated manner. This is achieved through the introduction of secret sharing [148] in the decentralized gradient descent (DGD) topology and considering geographical information when building this topology, that is, users closely geo-located learn (privately) collaboratively their respective user embeddings. As for the items' features, they are aggregated through a secure aggregation protocol (SA) [76]. Authors evaluate their approach on standard POI datasets' and compare it with a centralized factorization machine (FM), which it seems to compete with, while guaranteeing a better level of privacy.

**PEPPER [14]** emphasizes personalization of POI models through the aggregation step while using a gossip communication protocol to eliminate the central FL server. In this work, nodes gossip their models with their neighbors and aggregate them after evaluating their contribution. The authors also introduce a peer-sampling protocol that acts as a clustering over time, ensuring each node has similar users in its neighborhood. Results from their experiments show that fully decentralized FL can be competitive with centralized solutions, while offering scalability and personalization.

**FedPOIRec [134]** introduces a framework that leverages social relationships between users to make more personalized models. The authors first train a global model using FL and SMPC to protect the aggregation phase from a curious
server. A trusted third party is then tasked with finding similar users based on encrypted embeddings for personalized aggregation. The encryption is done using a leveled variant of the CKKS fully homomorphic encryption (FHE) scheme. This works also encompasses an adaptation of the setting of [10] and CESAR [154], a sequential recommender based on a convolutional neural network. The authors validate their approach through an evaluation on five versions (cities) of Foursquare as well as a formal privacy analysis.

Finally, another use case that has been explored in the context of location-based recommendation systems is the driver recommendation use case, as tackled by [159]. In this work, cab companies use federated learning to strengthen RSUs with the computational capability to develop an intelligent recommender system that recommends the appropriate driver for a subsequent trip. To this end, they consider both the driver's stress and past behavioral patterns.

## 4.4 Other Approaches

### 4.4.1 Clustering Based Approaches

Most existing works in clustering in FL are focused on methods to identify and self-organize devices into communities are so to conduct model sharing within those communities. In [22], authors introduce and evaluate a hierarchical clustering for vision models, where the local model is a CNN that is trained under a supervised fashion. The extensive evaluation presents the improvement that hierarchical clustering can bring to federated learning under a non-iid setting where each client holds partitioned data. In [85], the authors propose the dynamic Generative Adversarial Network (GAN)-based clustering in FL to improve the time series forecasting for cases such as cell tower handover prediction. Their proposed approach accounts for the adaptive clusters and non-iid data. IFCA proposed by [55], starts by randomly initializing *k* models, one per cluster. Each client assigns itself to a cluster at the start of each round of training by evaluating all *k* models on its local data and choosing the model with the lowest loss to train for *m* epochs. At the end of each round, the server performs federated averaging within each cluster of clients separately.

Although this theoretical line of work is receiving a great deal of attention from the FL research community we have seen almost no adaptation, except one, to the spatial-temporal models in FL. F-DEC [114] proposed a deep embedded clustering for urban community detection in FL. They expanded on the centralized model proposed by [47] and trained an autoencoder based on heatmap images of mobility trajectories transformed using the frequency of visits (where brighter pixels show more frequently visited areas). They then used a KL divergence loss for clustering similar heatmaps together. Furthermore, this work is the only early evidence found that measures the computational complexity of such algorithms when it is deployed on ordinary smartphones.

### 4.4.2 Privacy and Attacks in Spatial-Temporal FL Models

FL was initially designed to protect user privacy by sharing model parameters instead of data. However, research has shown that sharing these parameters can still reveal sensitive information, especially in models that use embeddings/latent features to capture user or point-of-interest semantics. To address this issue, researchers have proposed mainly three approaches: i) "a share less" policy, ii) injecting noise using techniques like DP, and iii) using cryptographic primitives (e.g., FHE). Various studies have proposed solutions using each of these approaches, but each approach has its limitations. For instance, [60, 8] proposed sharing only user-independent embeddings to be learned in a federated manner while training user embeddings locally. [43] first proposed a practical attack, demonstrating that user check-ins can be easily inferred by a curious FL server based on POIs embeddings. Later on, they proposed to alleviate this attack by training these embeddings on noisy data generated using DP. They show that if the rest of the network (i.e., non-sensitive layers) is frozen during this noisy training, and is pre-trained on real data, then the performance remains reasonable. Nevertheless, they did not quantify the impact of their attack nor the degree of protection provided by their solution. Another line of work, [134], opts for the third approach and uses SMPC to hide the individual contributions of the users from a curious FL server. Unfortunately, this approach opens the door to malicious users, whose goal would be to corrupt the learning, and who would be difficult to detect due to SMPC.

A more privacy-oriented solution was proposed by [26]. They aggregated less sensitive embeddings using SMPC and categorized sensitive embeddings into two parts: those related to POIs and those related to users. They used Local DP to add noise at the user level to the POI-related embeddings before sharing them with the server. They also proposed to share the user-dependent embeddings in a peer-to-peer fashion using secret sharing. To reduce overhead and the attack surface, they considered geographic information to build the peer neighborhood. Thus, a user shares its sensitive embeddings only with geographically close neighbors. This solution has competitive prediction performances with non-private solutions and undeniably provides more privacy guarantees. However, the impact of the assumption that sensitive embeddings can be safely shared with nearby users is still unclear.
server. A trusted third party is then tasked with finding similar users based on encrypted embeddings for personalized aggregation. The encryption is done using a leveled variant of the CKKS fully homomorphic encryption (FHE) scheme. This works also encompasses an adaptation of the setting of [10] and CESAR [154], a sequential recommender based on a convolutional neural network. The authors validate their approach through an evaluation on five versions (cities) of Foursquare as well as a formal privacy analysis.

Finally, another use case that has been explored in the context of location-based recommendation systems is the driver recommendation use case, as tackled by [159]. In this work, cab companies use federated learning to strengthen RSUs with the computational capability to develop an intelligent recommender system that recommends the appropriate driver for a subsequent trip. To this end, they consider both the driver's stress and past behavioral patterns.

## 4.4 Other Approaches

### 4.4.1 Clustering Based Approaches

Most existing works in clustering in FL are focused on methods to identify and self-organize devices into communities are so to conduct model sharing within those communities. In [22], authors introduce and evaluate a hierarchical clustering for vision models, where the local model is a CNN that is trained under a supervised fashion. The extensive evaluation presents the improvement that hierarchical clustering can bring to federated learning under a non-iid setting where each client holds partitioned data. In [85], the authors propose the dynamic Generative Adversarial Network (GAN)-based clustering in FL to improve the time series forecasting for cases such as cell tower handover prediction. Their proposed approach accounts for the adaptive clusters and non-iid data. IFCA proposed by [55], starts by randomly initializing *k* models, one per cluster. Each client assigns itself to a cluster at the start of each round of training by evaluating all *k* models on its local data and choosing the model with the lowest loss to train for *m* epochs. At the end of each round, the server performs federated averaging within each cluster of clients separately.

Although this theoretical line of work is receiving a great deal of attention from the FL research community we have seen almost no adaptation, except one, to the spatial-temporal models in FL. F-DEC [114] proposed a deep embedded clustering for urban community detection in FL. They expanded on the centralized model proposed by [47] and trained an autoencoder based on heatmap images of mobility trajectories transformed using the frequency of visits (where brighter pixels show more frequently visited areas). They then used a KL divergence loss for clustering similar heatmaps together. Furthermore, this work is the only early evidence found that measures the computational complexity of such algorithms when it is deployed on ordinary smartphones.

### 4.4.2 Privacy and Attacks in Spatial-Temporal FL Models

FL was initially designed to protect user privacy by sharing model parameters instead of data. However, research has shown that sharing these parameters can still reveal sensitive information, especially in models that use embeddings/latent features to capture user or point-of-interest semantics. To address this issue, researchers have proposed mainly three approaches: i) "a share less" policy, ii) injecting noise using techniques like DP, and iii) using cryptographic primitives (e.g., FHE). Various studies have proposed solutions using each of these approaches, but each approach has its limitations. For instance, [60, 8] proposed sharing only user-independent embeddings to be learned in a federated manner while training user embeddings locally. [43] first proposed a practical attack, demonstrating that user check-ins can be easily inferred by a curious FL server based on POIs embeddings. Later on, they proposed to alleviate this attack by training these embeddings on noisy data generated using DP. They show that if the rest of the network (i.e., non-sensitive layers) is frozen during this noisy training, and is pre-trained on real data, then the performance remains reasonable. Nevertheless, they did not quantify the impact of their attack nor the degree of protection provided by their solution. Another line of work, [134], opts for the third approach and uses SMPC to hide the individual contributions of the users from a curious FL server. Unfortunately, this approach opens the door to malicious users, whose goal would be to corrupt the learning, and who would be difficult to detect due to SMPC.

A more privacy-oriented solution was proposed by [26]. They aggregated less sensitive embeddings using SMPC and categorized sensitive embeddings into two parts: those related to POIs and those related to users. They used Local DP to add noise at the user level to the POI-related embeddings before sharing them with the server. They also proposed to share the user-dependent embeddings in a peer-to-peer fashion using secret sharing. To reduce overhead and the attack surface, they considered geographic information to build the peer neighborhood. Thus, a user shares its sensitive embeddings only with geographically close neighbors. This solution has competitive prediction performances with non-private solutions and undeniably provides more privacy guarantees. However, the impact of the assumption that sensitive embeddings can be safely shared with nearby users is still unclear.
Another type of attack that has received significant attention in recent mobility research is re-identification attacks [111, 50]. The fundamental concept behind such attacks is that a malicious service provider could exploit background knowledge to associate anonymized user traces with their respective owners, thereby compromising users' anonymity. To address this issue, Khalfoun et al. proposed a federated protocol for assessing the risk of re-identification on mobility data. This protocol involves training a re-identification model in a federated manner using users' traces and subsequently utilizing this model to select the optimal combination and hyperparameters of location privacy protection mechanisms (LPPMs) that can protect a user's privacy whenever they transmit their data to an untrusted Mobile Crowd Sensing Server (MCS). Notably, this solution appears to be the only privacy risk assessment mechanism for mobility data that does not necessitate the presence of a trusted curator, to the best of our knowledge. For a full survey of privacy and security techniques in FL see [125].

Based on our established taxonomy, an analysis of relevant studies, and a meta-analysis, we have observed that some key factors in certain tasks (e.g., privacy and non-iidness in recommendations, resource constraints and computational costs in TFP) are often not given sufficient consideration. Instead, other criteria are erroneously deemed more significant. For instance, in TFP, privacy is typically not a primary concern because the data collected from sensors is generally aggregated, however, current works have considered it more often than other criteria. Moreover, there are several still open-problems as well as a lack of standardization, both in data and code. In light of these observations, we outline a proposed roadmap in the following section. This roadmap is designed to accelerate progress in FL-ST, with a focus on bridging the gap with real-world applications, which are currently underrepresented in the literature. We believe that this approach will not only streamline research in this field but also facilitate its practical implementation.

# 5 Discussion and open research challenges

Based on the review of the above papers, we see the following opportunities and roadmap for the research community to explore.

## 5.1 Semantic Location Embedding and Context-Awareness Modelling

One of the biggest opportunities that we see in continuing research on location and point trajectory predicting, is in regards to integrating more semantic and contextual information about types of places instead of focusing primarily on coordinates. For example, this contextual information can include information on whether a point in trajectory represents someone's workplace, their frequently visited locations, or a potential point of interest in a new town. While this is not limited to FL applications and previous works have often used open source maps to infer information about the type of places (e.g., popularity, socio-economic level), FL can bring a new level of anonymity and personalization to this integration. The semantic representation of the locations can then be learned over time on users' devices, maintaining users' privacy while allowing for better-personalized models.

## 5.2 Byzantine Resilient Spatio-Temporal Mobility Federated Learning

Within the realm of ML, Byzantine Resilience commonly denotes the ability to train an accurate statistical model amid the presence of arbitrary behaviors, commonly referred to as Byzantine users [59]. These behaviors manifest either due to faults or malicious users, encompassing scenarios like a compromised sensor in the TFP problem or a user aiming to promote specific items in a Top-K POI recommendation use case. In the best-case scenario, the trained model proves unusable, resulting in a cost loss incurred during its training. However, in the worst-case scenario, if the attack goes undetected (e.g., backdoors on GNN [193]), the model might exhibit concealed yet malicious behaviors. This predicament poses a significant challenge owing to the difficulty of interpreting model parameters.

Distinguishing between a Byzantine model update transmitted by a client or sensor and an honest but out-of-distribution client often presents a non-trivial task. This complexity amplifies in highly heterogeneous configurations typical of FL-ST mobility applications. Its pertinence highlights this challenge as a substantial research track for Distributed ML and FL. Surprisingly, this aspect remains relatively unexplored in the context of FL-ST mobility applications. Among the multitude of reviewed papers, only Meese et al. [119] and Tang et al. [153] have delved into such adversarial settings. However, neither of these works has considered the novel solutions proposed in this field, such as bucketing [66] and variance reduction techniques [41]. This gap is more concerning given the pronounced heterogeneity characteristic of these applications.

Consequently, there is an urgent imperative to delve into this challenge and contemplate environments accommodating Byzantine behaviors within FL clients. This pursuit should be seen as a fundamental step to bridge the gap between