# Estimating Spatial Mean Speeds from Local Sensors: a Machine Learning Approach

Nandan Maiti, Ludovic Leclercq

► To cite this version:

Nandan Maiti, Ludovic Leclercq. Estimating Spatial Mean Speeds from Local Sensors: a Machine Learning Approach. Data Science for Transportation, 2025, 7, 37p. 10.1007/s42421-025-00120-w. hal-05096954

HAL Id: hal-05096954

https://hal.science/hal-05096954v1

Submitted on 4 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Estimating Spatial Mean Speeds from Local Sensors: a Machine Learning Approach

Nandan Maitiª, Ludovic Leclercqª

ªUniv. Eiffel, ENTPE, LICIT-ECO7, Lyon, 69500, France

## Abstract

Accurate Network Macroscopic Fundamental Diagram (NMFD) estimation requires accurate spatial mean speed estimates that are hard to get from loop detector devices (LDD) as they capture local speeds only. This study introduces a correction method able to reconstruct the mean speed from loop data leveraging floating car devices (FCD) during the training phase. This significantly improves LDD-based NMFD estimation. Unlike previous studies focusing on local speed corrections, our approach integrates LDD and FCD using machine learning techniques to enhance link-average speed estimation. We compare the performance of Ordinary Least Squares Regression (OLSR), Random Forest (RF), Multilayer perceptron (MLP), and XGBoost models, using a comprehensive dataset to evaluate their effectiveness in aligning LDD speeds with FCD as ground truth average speed. The RF model showed the highest accuracy, with a 37.44% improvement over original LDD speeds and a Root Mean Squared Error (RMSE) of 9.54 km/hr. Additionally, we investigated the impact of LDD positions and demand by separating loading and unloading periods. The RF model's adjustments were especially effective under these conditions, resulting in an improved accuracy of 44.9%. Our analysis of NMFDs further validated the RF model's robustness and accurate estimation of NMFDs using corrected LDD. We validate our methodology across two urban networks, Athens and Lyon, demonstrating its spatial and temporal transferability. Results show that bias correction significantly improves LDD speed accuracy, leading to more reliable NMFD estimation. This work advances data fusion methods for urban traffic monitoring and provides a robust framework for addressing LDD positional bias in large-scale traffic studies.

**Keywords:** Loop detectors, Floating cars, Spatial imputation, Macroscopic fundamental diagrams, Machine learning models.

## 1. Introduction

Empirical estimation of NMFD is crucial for gaining insights into the complex dynamics of traffic networks, especially in response to varying demands and incidents. The primary tools used for capturing these dynamics are loop detectors and floating cars. LDD provide continuous and localized data on
traffic flow, typically installed at strategic points throughout road networks. They offer detailed insights into parameters such as flow rates, local speeds, and occupancy levels at specific locations. However, their spatial coverage is limited to predefined locations, restricting the comprehensive understanding of network-wide traffic patterns (Qiu et al., 2010). In contrast, an FCD utilizes GPS-enabled vehicles to track traffic conditions dynamically across broader areas of the network. They offer real-time data on travel times, route choices, and traffic densities, providing a more extensive view of network-wide dynamics (Kerner et al., 2005). However, deploying floating cars can be costly and may not cover all parts of the network uniformly. Both LDD and FCD have inherent limitations regarding data granularity (Saeedmanesh and Geroliminis, 2016), coverage (Leclercq et al., 2014), and cost-effectiveness (Huang et al., 2019). To overcome these limitations and enhance the accuracy of NMFD estimation, data fusion techniques can be employed.

Data fusion involves integrating multiple data sources, such as LDD (Geroliminis and Daganzo, 2008; Gayah and Daganzo, 2011; Geroliminis and Sun, 2011; Mahmassani et al., 2013; Leclercq et al., 2014; Saberi et al., 2014; Tsubota et al., 2014; Ambühl and Menendez, 2016; Du et al., 2016; Ambühl et al., 2017; Shim et al., 2019), FCD (Geroliminis and Sun, 2011; Leclercq et al., 2014; Tsubota et al., 2014; Ambühl et al., 2017), and public transport data (PTD) (Geroliminis et al., 2014; Loder et al., 2017; Huang et al., 2019; Fu et al., 2020), to provide more accurate NMFD estimations by addressing the biases present in individual data sources. The challenges with LDD are the heterogeneous spatial distribution within the network, link selection, and the uneven relative positioning to the nearest intersection (Leclercq et al., 2014). These factors contribute to biased and systematic errors when estimating traffic speed. Unlike vehicle counts, which can be converted to equivalent network flow, speed measurements at the detector level are localized and cannot capture the spatial speed dynamics related to congestion propagation. This makes LDD unreliable, thereby enhancing the utility of FCD in accurately estimating network-level speed. Many existing studies use both LDD and FCD data to calibrate NMFD (Leclercq et al., 2014; Tsubota et al., 2014; Ambühl et al., 2017). However, challenges with FCD data include accurately estimating the penetration rate, long-term availability of data, and developing a robust fusion methodology to integrate it with LDD (Fu et al., 2020). The penetration rate of FCD is crucial for calibrating speed in data fusion methods (Lu et al., 2018). Both LDD and FCD measure all modes together, making them suitable for unimodal NMFD estimation, which represents all modes in a single plot.

Although the existing studies of Geroliminis and Daganzo (2008); Gayah and Daganzo (2011); Geroliminis and Sun (2011) started using FCD with LDD to estimate the NMFD with pivotal goal of enriching LDD estimation, Leclercq et al. (2014); Tsubota et al. (2014); Ambühl et al. (2017) adopted FCD implicitly for network speed and LDD for flow estimation. Ambühl et al. (2017) proposed for the first time a fusion algorithm for estimating network average variables using LDD and FCD by incorporating coefficients such as probe penetration rate ($ρ$) and no. of link with detector to link ratio ($φ$). The accuracy of the
fusion method can be evaluated against the ground truth NMFD. It could be argued that setting $\phi = 1$ or $\rho = 1$ might aid in estimating the ground truth NMFD. However, the former is linked with the challenges posed by LDD, while the latter could be prohibitively expensive (Ambühl et al., 2017; Du et al., 2016). While LDD count is widely recognized for accumulation and flow estimation, it is generally not recommended for speed estimation in a region because it captures the localized speed of the vehicles (Leclercq et al., 2014; Fu et al., 2020). Lee et al. (2023) analyzed the LDD of 28 cities and found that speeds from the LDD are biased and directly dependent on the position with respect to the intersection. Leclercq et al. (2014) proposed a novel analytical approach to correct the LDD speed using the network flow parameters such as wave speed, jam density, saturation flow, and free-flow speed with the signal cycle, which can be difficult to estimate for an unfamiliar network. Most existing studies focus on local speed and bias corrections of LDD (Sin et al., 2002; Peeta and Anastassopoulos, 2002; Coifman and Kim, 2009; Yufei Yuan et al., 2010; Corey et al., 2011; Coifman and Neelisetty, 2014), with less emphasis on correcting LDD speed in relation to link-average speed. This gap is notable, as link-average speed is a critical variable for accurately estimating network-wide macroscopic dynamics using LDD.

While several studies have explored the application of data fusion of LDD and FCD, most have faced challenges due to the inaccuracy of LDD and the sparse FCD (Maiti et al., 2024, 2025). The study focused on the former, addressing the discrepancies in LDD speed using the parsed FCD and developing a methodology to estimate the accurate speed from LDD. Unlike traditional statistical models, we choose machine learning (ML) approaches to capture complex, non-linear relationships in traffic data without the need for explicit network parameterization. Also, ML models can effectively learn spatial and temporal patterns, making them particularly suited for LDD speed estimation and bias correction. This adaptability allows ML models to generalize across different traffic conditions, enhancing their robustness and applicability in large-scale traffic studies. Therefore, this study aims to develop an ML-based correction factor using FCD to accurately measure LDD rather than fusing LDD and FCD together.

## 1.1. Motivation and Objectives

In vast transportation networks, LDD are strategically positioned at specific distances from intersections to monitor traffic flow. However, due to network congestion and the dynamics of traffic signals, the speed data collected from these LDD often fail to accurately reflect the average link speed. Consider, for instance, an intersection with four links as illustrated in Figure 1 (a). Two LDD were installed near the intersection, as detailed in the inset. The average link speed, estimated from FCD, was compared with local speed data from the detectors ($LD_1$ and $LD_2$). This comparison is depicted in Figure 1 (b) and (c). It is evident from the figures that when $LD_1$ is positioned upstream of the intersection, it consistently underestimates the speed relative to the average link speed derived from FCD. Conversely, when $LD_1$ is located downstream, it
tends to overestimate the speed compared to the average link speed from FCD. This discrepancy arises primarily due to two factors:

1. Upstream Congestion Propagation: LDD positioned near the upstream side of intersections frequently experience congestion spillover, especially during red light phases, leading to reduced speed readings.

2. Downstream Speed Surge: Vehicles passing LDD on the downstream side of intersections may travel at higher speeds immediately after the signal turns green, only to encounter congestion further downstream as they approach the next intersection.

Given these factors, the speed measurements from LDD can significantly differ from the actual average speeds captured by FCD. While analytical models can estimate the impact of congestion on LDD speeds using parameters such as signal cycle length, red time, and congestion wave speed, these models are often complex and computationally expensive, especially for large-scale networks. Therefore, the study objectives include:

<figure><img src="image_2.png" /><figcaption>Figure 1: Speed comparison of LDD and FCD on 01/11/2022 (a) Two detectors are placed near to intersection, $LD_1$ at upstream of intersection on $l_1$ link and $LD_2$ at downstream of intersection on $l_2$ link (b) Speed from the FC at $l_1$ and $LD_1$ (c) Speed from the FCD at $l_2$ and $LD_2$.</figcaption></figure>
1. This study aims to enhance the accuracy of LDD for NMFD estimation by developing and training a correction factor based on data-driven machine-learning models using FCD.

2. Identifying network and traffic variables contributing to bias in LDD speed, providing insights into the underlying causes of inaccuracies.

3. Empirical investigation to evaluate biases in LDD results during peak and off-peak traffic periods, highlighting temporal discrepancies.

Towards this end, this paper is organized as follows. Section 2 provides an overview of the data and network used in this study. Section 3 introduced the network variables using data fusion, loading-unloading periods identification, variable estimation for ML models, and proposed the methodology adopted for LDD speed correction using ML models. Section 4 discusses the results and accuracy of the models in different traffic conditions. Finally, the study concludes with major highlights in Section 5.

## 2. Data Description

### 2.1. Athens Downtown

We used LDD data and FCD data from downtown Athens, Greece, to develop the methodology. Figure 2(a) presents a map view of the Athens city, and the link-level details can be found in Figure 2(b). The total network coverage area is approximately (7.3 × 4.8)km². We defined nodes as intersections and links as road segments. The study analyzed a total of 2,652 links and 1,423 nodes. Among these nodes, 332 were identified as signalized intersections. The LDD are spatially distributed in the network, as shown in Figure 2(b). The road spaces are shared by different vehicle classes and modes, such as cars, medium vehicles, taxis, heavy vehicles, two-wheelers, and buses; more details about the vehicle classes can be found in Barmpounakis and Geroliminis (2020). The data collection spanned four weekdays, starting from November 1st to November 4th, 2022. LDD and FCD both provide 24-hour traffic data.
<figure><img src="image_3.png" /><figcaption>Figure 2: Network and LDD descriptions (a) Athens map view (b) LDD position in the network with links.</figcaption></figure>

### 2.1.1. LDD

LDD in the network provides critical data, including total vehicle counts and average speeds, with a frequency of one hour. These LDDs are predominantly installed on Type 1 and Type 2 road links. A total of 264 detectors have been installed across the network, all of which were operational continuously throughout the data collection period. The distribution of LDDs across the network is illustrated in Figure 3 (a), showing an exponential distribution of LDD positions relative to the downstream intersections. Specifically, it is evident from Figure 3 (a) that a significant proportion of the detectors, approximately 135 out of 264, are located within 25 meters downstream of intersections. This placement could be influenced by legacy installation practices, space constraints, or the need to monitor discharge rather than approach flow. While this configuration may not be ideal for capturing queueing dynamics, it still provides valuable insights into overall network performance. Furthermore, the aggregated average counts from all LDDs are depicted in Figure 3 (c). This figure highlights the temporal variation in vehicle counts, with the ascending and descending slopes indicating periods of network loading and unloading, respectively (more details in section 3.3). The attributes of LDD used in this study are total vehicle counts, average speed, distance from intersection, date-time, and detector lat-long.
<figure><img src="image_4.png" /><figcaption>Figure 3: Data description of the study (a) distribution of LDD distance from the downstream of the intersection (b) distribution of the link length in the network (c) LDD counts over the study duration.</figcaption></figure>

### 2.1.2. FCD

The probe vehicles equipped with GPS tracking systems, commonly referred to as floating cars, are integral to this study. We estimated the average speed on each link by utilizing the harmonic mean speed of these probe vehicles, with data collected at 5-minute intervals. This estimation was based on the distance traveled and the time taken by the probes on each link. To assess potential biases in the LDD, we extracted the average speed of links where LDD are installed and compared these values with the FCD. The corresponding links and LDD are depicted in Figure 2 (b). Additionally, the distribution of link sizes is shown in Figure 3 (b), indicating that most links range between 50 to 200 meters, with the maximum link size being 1390 meters in our study. The attributes of FCD utilized in this study include link-wise total probe count, distance traveled, travel time, harmonic mean speed, date-time, and latitude-longitude coordinates. These attributes enable precise estimation of link-level average speed and probe density. The speed confidence level of FCD is discussed in Appendix A.
## 2.2. Lyon Downtown

We employed an additional empirical dataset from downtown Lyon, France, to validate the proposed methodology and evaluate its applicability across diverse network configurations. The study area spans approximately 13 km × 20 km and includes data from 400 loop detectors. The LDD was sourced from an open-access platform (https:// avatar.cerema.fr) and covers the period from August 1 to August 7, 2018. The LDD provides only occupancy and flow traffic variables. Therefore, we estimate density and speed using a calibrated density-occupancy equation based on Leclercq (2005). To complement this, we utilized average link speeds derived from FCD for the same timeframe. The attributes of both the FCD and LDD datasets align with those used in the Athens case study. Figure 4 illustrates the loop detector locations and the network layout for Lyon.

<figure><img src="image_5.png" /><figcaption>Figure 4: Network and LDD descriptions (a) Villeurbanne, Lyon map view (b) all the LDD with links</figcaption></figure>
## 3. Methodology

This section details the development of a methodology for correcting LDD speed estimations using information on LDD positions and traffic demand. The objective is to evaluate the adaptability of ML models in reducing the bias inherent in LDD speed estimations. The ML models were trained using FCD speeds as the ground truth. The process began with the identification of variables that are correlated with and influence LDD speed measurements. These variables were then used to construct the ML models. The models' hyperparameters were optimized by comparing their outputs with the ground truth speeds from FCD data. To enhance model performance, training was conducted separately for periods of network loading and unloading (explained in detail in Section 3.3.2). Finally, the Root Mean Squared Error (RMSE) of speed estimations from different ML models and the original LDD were compared against the FCD speeds. Additionally, the NMFDs derived from these models were evaluated and compared. This comprehensive approach ensures a robust correction mechanism, improving the accuracy of LDD speed estimations and, consequently, NMFDs.

### 3.1. Estimating the NMFD

Theoretically, an empirical NMFD requires all link information leads to have at least an LDD in a link. But, in practice, only a certain portion of links are installed with an LDD. Bias in speed sensing can occur depending on their position. Therefore, FCD is integrated with LDD to construct a sound NMFD. Hence, we defined the traffic variables from both sources and fused them to acquire accurate network flow information.

#### 3.1.1. LDD

In this study, we estimate speed ($v$) and flow ($q$)- NMFD. The LDD provides total counts and average speeds at an hourly frequency. Consequently, the estimation of link flows ($q_{i,LDD}$) is notably precise. However, due to the absence of occupancy ($o_{i,LDD}$) data at the detector level, a direct derivation of link density ($k_{i,LDD}$) from the detectors is not feasible. To estimate the average network flow from LDD ($\hat{q}_{LDD}$), we calculate $q_i$ weighted by the corresponding link length, $l_i$. Thus the network variables from LDD are expressed as (1).

$$ \hat{q}_{LDD} = \frac{\sum_{i} q_{i,LDD} l_i}{\sum_{i} l_i} \qquad \hat{v}_{LDD} = \frac{\sum_{i} v_{i,LDD}}{n} \qquad \hat{k}_{LDD} = \frac{\hat{q}_{LDD}}{\hat{v}_{LDD}} \quad (1) $$

The network level average speed ($\hat{v}_{LDD}$) is the average speed of all the $n$ detectors in the network, where $v_{i,LDD}$ is the detector level speed at $i^{th}$ link. Thus, estimated average network density ($\hat{k}_{LDD}$) is defined as in (1).

#### 3.1.2. FCD

In this study, we used FCD to estimate link speed. Each link had at least one probe during the study period. Let $p$ denote the total number of probes
traveling on the $i^{th}$ link. For the $p^{th}$ probe, the travel distance is $d_{i,p}$ and travel time is $t_{i,p}$. The average speed of a link from FCD ($v_{i,FCD}$) is calculated by dividing the total distance traveled by all probes on that link by the total time spent (see (2)). Similarly, the average network-level speed from FCD ($\hat{v}_{FCD}$) is determined by dividing the total distance traveled by all probes across the network by the total time spent within a specific time interval.

$$v_{i,FCD} = \left( \frac{\sum_{p} d_{i,p}}{\sum_{p} t_{i,p}} \right) \qquad \hat{v}_{FCD} = \left( \frac{\sum_{i} \sum_{p} d_{i,p}}{\sum_{i} \sum_{p} t_{i,p}} \right) \qquad (2)$$

The link speed derived from FCD represents the average travel speed of the probes, offering greater spatial coverage compared to LDD. Consequently, FCD speed can be regarded as the ground truth for link-level vehicle speed.

Network-level traffic flow variables can be estimated for a network of length ($\sum_{i} l_{i}$) during time period ($T$) from FCD, as shown in Equation (3). Here, $\sum_{i} l_{i}$ represents the network length equipped by FCD. However, the estimation of $\rho$ over the spatio-temporal domain is both tedious and expensive (Leclercq et al., 2014). Therefore, it is highly desirable to estimate network flow from LDD and speed from FCD for improved accuracy of NMFD.

$$\hat{q}_{FCD} = \frac{\sum_{i} \sum_{p} d_{i,p}}{\sum_{i} l_{i} T \rho} \qquad \hat{k}_{FCD} = \frac{\sum_{i} \sum_{p} t_{i,p}}{\sum_{i} l_{i} T \rho} \qquad \hat{v}_{FCD} = \frac{\hat{q}_{FCD}}{\hat{k}_{FCD}} \qquad (3)$$

### 3.1.3. LDD+FCD

As this study used both FCD and LDD, the ground truth NMFD of a network can be expressed as shown in (4).

$$\hat{q} = \hat{q}_{LDD} \qquad \hat{v} = \hat{v}_{FCD} \qquad \hat{k} = \frac{\hat{q}_{LDD}}{\hat{v}_{FCD}} \qquad (4)$$

Eq (4) refrees the ground truth NMFDs from the equipped network of FCD and LDD. Mariotte et al. (2020) proposed a scaling factor as a division of total network length ($L$) and total equipped network length ($\sum_{i} l_{i}$) to obtain the network production. Therefore, by using a similar scaling factor, we can estimate the variables of the entire network traffic flow. However, this is beyond the scope of the paper. Also, the spatial coverage of FCD should be sufficiently extensive to accurately represent the average network speed. This paper aims to demonstrate that training a model with LDD data combined with FCD over a short period enables accurate estimation of NMFD using LDD data alone, as needed. Thus, it is crucial to address the biases in LDD and quantify them based on network traffic characteristics and detector positions. In the following section, we propose a methodology to rectify LDD speed estimates by comparing them with the ground truth provided by FCD at the link level, where both data sources are available. Subsequently, this model is applied to LDD data where FCD is not available. By addressing LDD errors and biases in this manner, we aim to estimate a more accurate NMFD than would be possible using LDD data
alone.

## 3.2. ML Models for LDD Correction

We introduce an LDD correction method utilizing ML models within large-scale urban networks. The following subsections provide an in-depth explanation of the methodology employed.

### 3.2.1. Problem statement

Figure 5 (a) illustrates a scenario where congestion propagates upstream to the LDD. Consequently, a vehicle traveling at speed $v_i(x, t)$, with an average speed of $v_{i,FCD}$, may be recorded at a reduced speed $v_{i,LDD}$ by the LDD. This paper proposes the development of an ML model aimed at correcting LDD speed measurements. The proposed ML model will take the following inputs:

* $x_i$: Position of the LDD in the $i$-th link from the upstream of the intersection.

* $q_i(t)$: Traffic demand at time $t$ in the $i$-th link.

* $v_{i,LDD}(t)$: Speed measured by LDD at time $t$ in the $i$-th link.

The objective is to train the ML models to output a corrected speed ($v_{i,LDD}^c(t)$) that closely approximates the actual average link speed, as determined by FCD (see (5)). Figure 5 (b) shows a flowchart for the proposed ML models and LDD correction.

$$v_{i,LDD}^c(t) = f(v_{i,LDD}(t), x_i, q_i(t)) \quad \exists \quad v_{i,LDD}^c(t) \approx v_{i,FCD}(t) \quad (5)$$

By leveraging ML techniques, we aim to develop a robust and scalable solution that effectively aligns LDD speed measurements with actual prevailing speeds. This correction mechanism is expected to enhance the accuracy of NMFD from LDD across large transportation networks.

## 3.3. Pre-processing, Data Labeling, and Parametric Analysis

### 3.3.1. Pre-processing

The collected LDD and FCD data for the entire network appeared unfiltered and contained noise artifacts. The initial challenge in data processing stemmed from the non-overlapping nature of LDD and FCD within a link. Due to minor discrepancies in the latitude and longitude data, these points did not always align precisely. To address this issue, we introduced a buffering zone using geopandas-Python package (Jordahl et al., 2020) with a 5-meter radius around each LDD location. Any link intersecting this buffering zone was assigned to the same link as the LDD, ensuring accurate data alignment, used in the previous work of (Paipuri et al., 2020). Due to some mechanical errors in LDD, sometimes the counts from LDD may become very unreasonable. We excluded the LDD data of zero counts and per lane greater than 3500 veh/hr. Similarly, zero speed and more than 100 Km/hr speed were also excluded from further
<figure><img src="image_6.png" /><figcaption>Figure 5: (a) Shock-wave and time-space analysis at an LDD position upstream of an inter-section during red signal timing (b) Flow chart for the LDD correction model</figcaption></figure>

analysis. The data used in this study were devoid of systematic error in LDD. We opted for a higher threshold, ensuring that the filtering process accounts for the unique traffic characteristics of the Athens network while minimizing the risk of discarding relevant observations.

The frequency of FCD is 5 minutes. To align with the LDD aggregation time, we calculate the average link-wise FCD speed on an hourly basis. We exclude link speed information from the FCD data set if each 5-minute speed information is absent for a unique link. We discard FCD for probes with travel times exceeding 15 minutes within a link, as these likely indicate stops or prolonged waiting times.

### 3.3.2. Identification of loading-unloading periods

As we already explained in Figure 5 (a), the effect of demand and LDD position play a crucial role in determining LDD bias results. The demand profile of a network can be broadly categorized into loading and unloading periods, and the dynamics of the traffic network may differ based on whether it is in the loading or unloading phase (Buisson and Ladier, 2009; Geroliminis and Sun, 2011; Leclercq and Paipuri, 2019). Loder et al. (2019) demonstrated a hysteresis pattern in NMFD by distinguishing the loading and unloading periods within the empirical dataset. Therefore, in this study, we initially partitioned the data set into loading and unloading periods. Subsequently, we developed two distinct ML models using the FCD and LDD data. Later, a comparison of results was made to demonstrate the enhanced accuracy of LDD corrections using models specific to loading and unloading periods. Ambühl et al. (2020) introduced a dynamic time-warping-based approach to identify similar demand
patterns across different days, focusing on temporal variations in loading and unloading periods. The transition time between these phases was defined by (Mousavizadeh and Keyvan-Ekbatani, 2024) and was utilized in this study as shown in (6).

$$ \text{period} = \begin{cases} \text{loading, } \hat{k}(t) \ge \hat{k}(t-1) & \& \quad sd_i(t) \ge sd_i(t-1) \\ \text{unloading, } \hat{k}(t) \le \hat{k}(t-1) & \& \quad sd_i(t) \le sd_i(t-1) \end{cases} \quad (6) $$

The network loading periods are characterized by an increasing trend in the aggregated network density ($\hat{k}(t)$), and the standard deviation of density across all links ($sd_i(t)$) of the network at time step $t$. Conversely, unloading periods are defined by a decreasing trend in these metrics. Figure 6 (a) illustrates the temporal variation of aggregated density in the study network, highlighting the corresponding loading and unloading periods. Similarly, Figure 6 (b) shows the corresponding average speed across all links. It is evident from these figures that network characteristics, in terms of density and speed, differ significantly between the two periods, necessitating distinct modeling approaches. Consequently, the following section proposes two ML modeling approaches for LDD corrections.

<figure><img src="image_7.png" /><figcaption>Figure 6: Loading and unloading periods identification for (a) density and (b) speed.</figcaption></figure>

### 3.3.3. Parametric analysis

In this study, we conducted a parametric analysis to predict the corrected loop detector speed ($v_{i,LD}(t)$) based on the $v_{i,LD}(t)$, $x_i$, $q_i(t)$, $v_{i,FCD}(t)$. To identify the key factors influencing LDD speed, we computed the correlation matrix using the Pearson correlation coefficient for all the variables involved (Figure 7 (a)) and plotted pair plots (Figure 7 (b)) to visualize the relationships between them.

The correlation matrix (Figure 7 (a)) reveals significant relationships between LDD speed and several other variables. Specifically, the correlation with
LDD position is 0.11, indicating a positive correlation. Conversely, the correlation with traffic demand is -0.24, suggesting a negative relationship. Additionally, there is a negative correlation with the total number of lanes in a link, which is accounted for by considering demand per lane. As expected, the correlation with actual link speed (FCD speed) is 0.47, demonstrating a moderate positive correlation. The pair plots (Figure 7 (b)) provide further insights into these relationships, stratified by the class labels Unloading and Loading demand periods. The diagonal plots, which show the distribution of each variable, reveal differences in the distribution patterns between the two classes. For example, the LDD speed distributions show more observations of fast vehicles during unloading and more slow speeds during loading. The off-diagonal plots highlight the interactions between variables. For instance, the scatter plots between LDD speed and loop detector position illustrate a more clustered pattern for the loading class, while the unloading class shows a broader spread, indicating that the loop detector position has a varying impact on LDD speed depending on the class. The relationship between traffic demand and LDD speed shows a more dispersed pattern, with lower speeds generally associated with higher traffic demand. This negative correlation aligns with the intuition that higher traffic demand can lead to congestion, thereby reducing speeds. Overall, the correlation analysis and pair plot visualizations substantiate the hypothesis that LDD speed is influenced by loop detector position and traffic demand. These findings are pivotal for feature selection in developing a predictive model, as they highlight the most influential variables, ultimately aiding in enhancing the model's predictive accuracy for corrected loop detector speed.

<figure><img src="image_8.png" /><figcaption>Figure 7: (a) Overall correlation coefficient of different variables associated with LDD (b) Network demand-wise pair plots of the variables associated with LDD</figcaption></figure>

### 3.4. *Model Development*

The overall process involves several key steps, as illustrated in Figure 8. The process begins with data preparation, where LDD data, including $v_{i,\text{LDD}}(t)$,
$q_{i,\text{LDD}}(t)$, and $x_i$, are separated for both network loading and unloading periods. In the model training phase, the training inputs ($X_{\text{train}}$) from LDD data and the training output ($Y_{\text{train}}$) from FCD data ($v_{i,\text{FCD}}(t)$) are utilized to train a predictive model through parameter fitting and validation. Predictive models are then developed for both loading and unloading periods. During the model testing phase, test inputs ($X_{\text{test}}$) from LDD data are applied to the predictive models to generate predicted outputs ($\hat{Y}$). These predicted outputs ($v_{i,\text{LDD}}^c(t)$) are compared with test outputs ($Y_{\text{test}}$) from FCD data to evaluate prediction accuracy. Finally, a longitudinal analysis is conducted to assess if the difference between $Y_{\text{test}}$ and $\hat{Y}$ is significant, with iterations and refinements made as necessary to improve the model's accuracy. This comprehensive approach ensures robust and accurate prediction of LDD speed under varying network conditions.

<figure><img src="image_9.png" /><figcaption>Figure 8: Methodology flowchart</figcaption></figure>

### 3.4.1. *Training, validation, and testing*

The proposed ML model leverages a structured approach involving distinct training and testing phases. The training dataset ($X_{\text{train}}$) comprises LDD data collected over two days, including variables such as $v_{i,\text{LDD}}(t)$, $q_{i,\text{LDD}}(t)$, and $x_i$. Correspondingly, the training output ($Y_{\text{train}}$) consists of FCD data ($v_{i,\text{FCD}}(t)$) from the same period. This dataset is used to train the predictive model, enabling it to learn the underlying patterns and relationships. From the training
data set, 20 % of the data were chosen randomly for validation of the model training and tuning the model hyperparameter. The inclusion of performance metrics from the validation dataset in the model configuration process tends to introduce greater bias into the evaluation. Thus, during the model test, a separate test dataset ($X_{test}$) containing LDD data from the other two days is used. The model's performance is evaluated by comparing the predicted outputs ($\hat{Y}$) to the actual test outputs ($Y_{test}$) from the FCD data.

### 3.4.2. LDD Speed Correction Models

The model development process involves implementing four distinct ML algorithms: Regression, Random Forest, Multilayer perception, and XGBoost, each utilizing the training and testing datasets described previously. The prediction function can be formulated as follows (7):

$$ \hat{v}_{i,LDD}(t) = f(v_{i,LDD}(t), x_i, q_i(t)) \quad (7) $$

Where $f$ represents different models, which is discussed in the following sections. Ordinary Least Squares regression (OLSR): Regression serves as a baseline for comparison. It aims to establish a linear relationship between the input features ($X_{train}$) and the target variable ($Y_{train}$). Parameters include coefficients for each feature, which are optimized using ordinary least squares to minimize the mean squared error between predicted and actual values. The simplicity and interpretability of regression make it a useful initial step in the model development process.

$$ \hat{Y}_{OR} = \alpha_0 + \sum_j \alpha_j X_j + \epsilon, \quad X_j \in (v_{i,LDD}(t), x_i, q_i(t)) \quad (8) $$

Random Forest (RF): The RF model is an ensemble learning method that constructs multiple decision trees during training and outputs the mean prediction of the individual trees. Mathematically, the prediction from an RF is the average of the predictions from individual decision trees:

$$ \hat{Y}_{RF} = \frac{1}{T} \sum_{t=1}^{T} f_t(X, \Theta_t) \quad (9) $$

where $\hat{Y}_{RF}$ is the final predicted output, $T$ is the total number of trees, and $f_t(X, \Theta_t)$ represents the prediction from the $t$-th tree associated with independent and identically distributed (iid) random vectors $\Theta_t$. Each tree is built using a random subset of the training data and features, a process known as bootstrap aggregating or bagging (Tin Kam Ho, 1995). Key parameters include the number of trees in the forest ($n \_ estimators$), the maximum depth of the trees ($max \_ depth$), and the minimum number of samples required to split an internal node ($min \_ samples \_ split$). The model is trained using $X_{train}$ and $Y_{train}$, with hyperparameter tuning performed via cross-validation to optimize performance (Tin Kam Ho, 1995).
*Multilayer Perception (MLP)*: we also included an MLP model in our analysis. The MLP consisted of two dense layers with 64 neurons each, using the ReLU activation function and the Adam optimizer. These choices were based on hyperparameter tuning to achieve optimal performance, as shown in (Maiti and Chilukuri, 2023).

*XGBoost*: XGBoost (Extreme Gradient Boosting) is an advanced ensemble technique that builds trees sequentially, with each new tree correcting errors made by the previous ones. The model prediction in XGBoost is formulated as follows:

$$ \hat{Y}_{XG}^{(t)} = \sum_{k=1}^{t} f_k(X, \Phi_k) \qquad (10) $$

where $\hat{Y}^{(t)}$ is the prediction at the t-th boosting round and $f_k$ is the prediction function of the k-th tree, and $\Phi_k$ is the iid random vectors. Important parameters include the learning rate (learning_rate), the maximum depth of the trees (max_depth), and the number of boosting rounds (n_estimators). The model is trained on $X_{train}$ and $Y_{train}$ using gradient descent to minimize the loss function, and hyperparameters are tuned using techniques such as grid search or randomized search for optimal results (Chen and Guestrin, 2016).

To implement the predictive models, we utilized the Python packages XG-ols, BRegressor from the xgboost library (Chen and Guestrin, 2016) and Random-ForestRegressor from the scikit-learn library (Pedregosa et al., 2012). It allows for the specification of various hyper-parameters to control the behavior of the models. The most crucial hyper-parameters are the ‘objective’ of the XGBoost regressor and the ‘criterion’ for the RF regressor. In our case, they are set to minimize the squared error loss function.

All models are developed within a unified framework that ensures consistency in data preprocessing, feature selection, and evaluation metrics. The training phase involves fitting the models to the $X_{train}$ and $Y_{train}$ datasets over three days, while the testing phase evaluates their performance on the one-day $X_{test}$ and $Y_{test}$ datasets. Performance metrics such as mean squared error (MSE), root mean squared error (RMSE), and R-squared ($R^2$) are used to compare the models. This ensures robust model development and facilitates a thorough comparison of each algorithm’s predictive capabilities.

# 4. Results and Discussions

This section presents the results of optimizing hyper-parameters for the ML models and demonstrates the subsequent improvement in LDD speed estimation. Finally, we compare NMFDs derived from corrected LDD data against those obtained from the original, uncorrected data.

## 4.1. Model Optimization

To achieve optimal predictive performance, we conducted a comprehensive hyper-parameter tuning process for all discussed models, OLSR, RF, and XG-
Boost, focusing on minimizing the Mean Absolute Percentage Error (MAPE) (as shown in (11).

$$MAPE = \frac{100\%}{T} \sum_{t=1}^{T} \left| \frac{\hat{Y}_t - Y_{t, test}}{Y_{t, test}} \right| \quad (11)$$

The parameter optimization was carried out using grid search and cross-validation techniques, ensuring robust and reliable parameter settings for each model by randomly choosing the validation data set. For the basic OLSR model, we did not perform any kind of parameter optimization cause it is not so much affected by the hyper-parameters tuning. In RF and XGBoost, the main hyper-parameters for tuning were chosen based on the existing works of (Maiti and Chilukuri, 2023). The optimal parameters for each model were summarized in Table 1, which highlights the selected values that resulted in the lowest MAPE during validation.

<table><caption>Table 1: Optimal hyper-parameters</caption><thead><tr><th>RF</th><th>XGB</th></tr></thead><tbody><tr><td>n_estimators = 100</td><td>n_estimators = 100</td></tr><tr><td>max_depth = 70</td><td>max_depth = 70</td></tr><tr><td>min Samples Split = 2</td><td>learning_rate = 0.1</td></tr><tr><td>min Samples Leaf = 1</td><td>booster = 'gblinear'</td></tr><tr><td>bootstrap = 'True'</td><td>min_child_weight = 1</td></tr><tr><td>criterion = 'squared_error</td><td>base_score = 0.5</td></tr><tr><td>max_features = 'sqrt'</td><td>objective = 'reg : squarederror'</td></tr></tbody></table>

## 4.2. LDD Corrections

In the previous section, we discussed the ML models and their optimal hyperparameters. We now compare the models' efficiency in correcting LDD speed by estimating speeds close to FCD (ground truth) data using the corresponding training dataset. The test data consists of observations taken at a 1-hour frequency over 2 days for 264 LDD, totaling 12,672 observations. From this dataset, we randomly selected 300 observations to demonstrate the models' fit with the ground truth FCD and the original LDD speed. The entire dataset was used to calculate the MAPE and accuracy. Figure 9 illustrates the fits of all three models for the random observations when the models were trained without separating the loading and unloading periods. Analysis of these observations reveals that the LDD predominantly overestimated speed when the average link speed (FCD) was low (around 10-20 km/hr). Among the models, the simple OLSR performed the worst due to its reliance on linear relationships within the training data. Consequently, OLSR predictions closely followed the original LDD speed trend, with only a moderate improvement of 12.85% compared to the original LDD speed, as shown in Figure 9 (a). In contrast, the XGBoost model demonstrated superior performance with an RMSE of 10.92 km/hr, achieving
an improvement of 28.39% over the original LDD speed. The RF model out-
performed both, with an RMSE of 9.54 km/hr and an improvement of 37.44%
for the entire test dataset. While MLP did not perform well compare to the RF
and XGBoost with RMSE of 12.97 km/hr and an improvement of 14.95% for
the entire dataset. Notably, when comparing the black-circled areas in Figure
9 (b) for RF, Figure 9 (c) for XGBoost with Figure 9 (d) for MLP, it is evident
that RF provides slightly better corrections for lower speed areas. This indi-
cates that while both models significantly enhance the accuracy of LDD speed
estimations, RF offers marginally superior performance in scenarios where the
actual link speed is low.
<figure><img src="image_10.png" /><figcaption>Figure 9: Comparison of original LDD speed and corrected LDD speed for the test dataset using (a) OLSR, (b) RF, (c) XGBoost, and (d) MLP Models without separating training data for loading and unloading periods.</figcaption></figure>
The results presented in Figure 9 indicate that model performance varied across different speed regimes. Consequently, we segregated the data into loading and unloading periods based on average network density, as depicted in Figure 6. The loading periods correspond to times when network density increased and speed decreased, while unloading periods represent times when density decreased and speed increased. Figure 10 illustrates the comparative performance of the RF and XGBoost models, which were the top two performers from the previous results, during these periods. During the loading periods, the FCD speed was consistently below 40 km/hr, while during unloading periods, it fluctuated between 20 km/hr and 80 km/hr. Due to its poor performance in the initial analysis, the OLSR model was excluded from further evaluation. In the loading periods, both models showed improved performance, but Figure 10 (a) and (c) reveal that the RF model provided a slight edge in fitting with the FCD data, particularly in the observations between 250 and 300. Similarly, during unloading periods, depicted in Figure 10 (b) and (d), the RF model again demonstrated a better fit, especially for observations between 200 and 250. The summary of the performance is given in Table 2. Percentage improvements for the models during training and testing remain close, which signifies the robustness of the modeling framework. The RMSE and percentage improvement metrics further support the superiority of the RF model under varying traffic conditions. Specifically, the RF model achieved a 44.91% improvement over the original LDD speed during loading periods and a 42.87% improvement during unloading periods during testing. These results underscore the RF model’s robustness and effectiveness in accurately capturing the dynamics of network traffic conditions, outperforming the XGBoost model in both scenarios.
<figure><img src="image_11.png" /><figcaption>Figure 10: Comparison of original LDD speed and corrected LDD speed for the test dataset with separating training data for (a) RF on loading period, (b) RF on unloading periods, (c) XGBoost on loading period, and (d) XGBoost on unloading periods.</figcaption></figure>
<table><caption>Table 2: Performance of models for LDD speed correction</caption><thead><tr><th rowspan="2">Period</th><th rowspan="2">Model</th><th colspan="2">Training</th><th colspan="2">Testing</th></tr><tr><th>RMSE (km/hr)</th><th>% Improvement</th><th>RMSE (km/hr)</th><th>% Improvement</th></tr></thead><tbody><tr><td rowspan="5">All</td><td>LDD original</td><td>16.48</td><td>-</td><td>15.25</td><td>-</td></tr><tr><td>XGBoost</td><td>11.63</td><td>29.43</td><td>10.92</td><td>28.39</td></tr><tr><td>RF</td><td>10.16</td><td><strong>38.32</strong></td><td>9.54</td><td><strong>37.44</strong></td></tr><tr><td>MLP</td><td>13.11</td><td>20.44</td><td>12.97</td><td>14.95</td></tr><tr><td>OLSR</td><td>14.36</td><td>12.85</td><td>13.29</td><td>12.85</td></tr><tr><td rowspan="5">Loading</td><td>LDD original</td><td>16.25</td><td>-</td><td>15.25</td><td>-</td></tr><tr><td>XGBoost</td><td>9.19</td><td>43.45</td><td>8.7</td><td>42.95</td></tr><tr><td>RF</td><td>8.84</td><td><strong>45.55</strong></td><td>8.4</td><td><strong>44.91</strong></td></tr><tr><td>MLP</td><td>11.96</td><td>26.40</td><td>11.54</td><td>24.32</td></tr><tr><td>OLSR</td><td>14.00</td><td>13.87</td><td>13.12</td><td>13.96</td></tr><tr><td rowspan="5">Unloading</td><td>LDD original</td><td>16.30</td><td>-</td><td>15.79</td><td>-</td></tr><tr><td>XGBoost</td><td>9.85</td><td>39.56</td><td>9.7</td><td>38.56</td></tr><tr><td>RF</td><td>9.13</td><td><strong>43.97</strong></td><td>9.02</td><td><strong>42.87</strong></td></tr><tr><td>MLP</td><td>12.49</td><td>23.37</td><td>12.11</td><td>23.30</td></tr><tr><td>OLSR</td><td>13.74</td><td>15.67</td><td>13.37</td><td>15.32</td></tr></tbody></table>

The comparison of ground truth FCD speed with the original LDD speed and the corrected LDD speed using the RF model, as shown in Figure 11, highlights the significant improvements achieved through model correction. Figure 11 (a) reveals that LDD consistently overestimates network average speed ($\hat{v}_{LDD}$) than the FCD speed ($\hat{v}_{FCD}$). The clustering of these points below the diagonal line $\hat{v}_{LDD} = \hat{v}_{FCD}$ indicates a systematic bias where the original LDD speeds are higher than the actual FCD speeds. One probable reason for this trend could be attributed to the distribution of LDD positions within the network, as illustrated in Figure 3 (a). A significant number of LDD are positioned very close (20 meters) to the downstream of intersections, which leads to mainly detection of free-flow speeds under saturated conditions. This proximity likely contributes to the observed overestimation of speeds by the LDD. Conversely, the corrected LDD speed ($\hat{v}_{LDD}^c$) using the RF model (Figure 11 (b)) shows a marked improvement. The corrected speeds align more closely with the FCD speeds, as evidenced by the more symmetrical distribution of data points around the line $\hat{v}_{LDD}^c = \hat{v}_{FCD}$. This alignment is particularly notable during loading periods, where the RF model effectively reduces the overestimation bias.

Based on the discussions and the results presented in Table 2, it is evident that the RF model demonstrates superior predictive performance under network loading conditions characterized by slower vehicle movements. One explanation for this observation could be that during network loading, the independent variables in the model exhibit a higher correlation with the dependent variable. Specifically, LDD speeds during loading periods are significantly affected and
require more correction, especially if the loop detectors are positioned close to
the downstream of intersections. Conversely, if the detectors are positioned near
the upstream, intense correction will be required for unloading time.

<figure><img src="image_12.png" /><figcaption>Figure 11: Comparison of ground truth FCD speed with (a) original LDD speed, and (b) corrected LDD speed using RF model for loading and unloading period separately on test data.</figcaption></figure>

4.3. NMFD Comparison

This section compares actual NMFDs (refer Eq. (4)) from LDD and FCD
with those from only LDD (refer Eq. (1)) and corrected LDD during the train-
ing and testing phase. The corrected speed from the RF model (Eq. (5)) is
used as LDD speed in Eq. (1) to estimate NMFDs from corrected LDD. During
training, Figure 12 (a) shows that only LDD overestimated speed and the max-
imum observed density from LDD data is lower than the actual estimated from
LDD+FCD. Because at the same flow, LDD estimated a higher average speed in
the network, which led to a lower density. Therefore, Figure 12 (b) shows a lesser
critical density at the same saturation flow level. This observation supports our
previous findings in Figure 11 (a), where we explained during network loading,
the LDD were more affected by systematic errors. The corrected LDD speed
and flow NMFDs both align with the one estimated from LDD+FCD, which
shows a clear improvement over the original LDD in the training data. Figure
12 (c,d) presents the comparison between actual NMFDs and those estimated
from the corrected LDD, separately during loading and unloading periods for
the training periods. Upon visual inspection, it is evident that the corrected
NMFDs closely align with the actual NMFDs. During loading periods, cor-
rected LDD and LDD+FCD exactly overlapped with others, which exhibited
better correction in network loading. Hence, the corrections derived from the
RF model for LDD are deemed superior and suitable for NMFD estimation.

The main visuals of NMFDs from the test data in Figure 13 indicate that
corrected LDD speeds are precisely aligned within the band of LDD+FCD,
with reduced scattering. This indicates that the speeds corrected from the RF
model are more deterministic with the scenarios, which are unable to capture
the different speeds at a unique demand. However, during unloading periods,
the corrected LDD speeds tend to lie in the upper portion of the actual NMFDs,
whereas, during loading periods, they nearly overlap (see Figure 13 (c)). This
observation corroborates the findings from Figure 11(b), suggesting that the
corrections applied to LDD during loading periods are more effective compared
to unloading periods. Another notable observation from Figure 12 and 13 is the
disparity in scatter between actual NMFDs and those estimated from LDD. The
actual NMFDs exhibit greater scatter, reflecting real traffic hysteresis within the
network. In contrast, because LDD data has a longer aggregation time, the es-
timated NMFDs display less variation, capturing fewer details of the network
traffic dynamics. We also observed that loading periods exhibited greater vari-
ance in q-k NMFDs plots (Figure 12, Figure 13). During loading periods, traffic
congestion builds up progressively as demand increases, leading to greater dis-
persion in the flow-density relationship. This is because vehicles experience
varying congestion levels depending on network bottlenecks, signal timing, het-
erogeneous driver behavior, etc. Speed fluctuations become more pronounced
as congestion develops, resulting in a wider spread of data points in the NMFD.
In contrast, congestion dissipates more systematically during unloading periods,
leading to a more stable and predictable relationship between flow and density.

Overall, these insights underscore the effectiveness of the RF model corrections for LDD in improving NMFD estimation, particularly during loading periods, while also highlighting the limitations of LDD data in capturing the full extent of traffic variability observed in real-world conditions.
<figure><img src="image_13.png" /><figcaption>Figure 12: Comparison of NMFDs during model training from Athens data: (a) speed-density NMFD estimated without separating loading and unloading periods, (b) flow-density NMFD estimated without separating loading and unloading periods, (c) speed-density NMFD estimated with separated loading and unloading periods, and (d) flow-density NMFD estimated with separated loading and unloading periods from original LDD and corrected LDD</figcaption></figure>
<figure><img src="image_14.png" /><figcaption>Figure 13: Comparison of NMFDs during model testing from Athens data: (a) speed-density NMFD estimated without separating loading and unloading periods, (b) flow-density NMFD estimated without separating loading and unloading periods, (c) speed-density NMFD estimated with separated loading and unloading periods, and (d) flow-density NMFD estimated with separated loading and unloading periods from original LDD and corrected LDD</figcaption></figure>

4.4. *Testing and Validation Over Lyon City*

We conducted model training and testing on the Lyon dataset using a five-
day training period and two days of testing. The results demonstrated a perfor-
mance similar to the Athens dataset. Figure 14 compares the NMFD derived
from uncorrected LDD (LDD only), LDD combined with FCD (LDD+FCD),
and corrected LDD using the proposed RF model. Similar to Athens, most loop
detectors (LDs) in Lyon are located near intersections downstream, leading to
an overestimation of speed by LDD relative to the average link speed derived
from FCD. As shown in Figure 14 (a,b), the overestimated speed from LDD lies
consistently above the ground truth (LDD+FCD), whereas the corrected LDD
aligns closely with the ground truth. The correction further improves when
loading and unloading periods are treated separately. Figures 14 (c,d) depict
NMFD comparisons during network loading and unloading phases. In Lyon,
unloading periods predominantly occur at very low densities (approximately
5 veh/km-In), while loading periods tend to dominate beyond this range. By
comparison, the NMFD results for Athens in Figure 13 (c,d) show unloading phases extending to densities as high as 25 veh/km-In. This difference can be attributed to the relatively short unloading periods in the low-density regime in Lyon. Overall, the proposed method effectively reduces local biases in LDD for both datasets, achieving the primary objective of this study. Importantly, the methods success in two distinct network settings (Athens and Lyon) highlights its robustness and applicability to other network configurations.

<figure><img src="image_15.png" /><figcaption>Figure 14: Comparison of NMFDs during model testing from Lyon data: (a) speed-density NMFD estimated without separating loading and unloading periods, (b) flow-density NMFD estimated without separating loading and unloading periods, (c) speed-density NMFD estimated with separated loading and unloading periods, and (d) flow-density NMFD estimated with separated loading and unloading periods from original LDD and corrected LDD</figcaption></figure>

## 5. Conclusions

In this research, we addressed the challenges of utilizing speed data from LDD for estimating NMFDs using empirical data from Athens, Greece, and Lyon, France. The study introduced ML models with inputs limited to LDD position, LDD original speed, and network demand. Hyperparameters were
calibrated using test data from FCD. The developed models aim to provide a
robust solution that captures systematic errors and biases inherent in LDD data
within an urban network without relying on strict theoretical assumptions or
analytical models. This method proves crucial for reproducing network vari-
ables in scenarios where most data sources are LDD-based. Key insights and
highlights of the study include:

* The RF model demonstrates superior predictive performance, especially under network loading conditions where vehicle speeds are slower. The RF model achieved an RMSE of 9.54 km/hr with a 37.44% improvement over the original LDD speed, compared to the XGBoost model, which had an RMSE of 10.92 km/hr and a 28.39% improvement.

* Separating loading and unloading periods further enhances the RF model's performance, with a 44.91% improvement during loading and a 42.87% improvement during unloading, compared to the without separation, with a 37.44%.

* The analysis of the model's spatial and temporal transferability in Appendix B demonstrates that while the model performs best when trained and tested on the same location, it can still be applied across different networks and time periods with slightly reduced accuracy.

* The proximity of LDD to intersections influences speed estimation accuracy, with detectors near the downstream likely detecting free-flow speeds under saturated conditions, leading to overestimation.

* Corrected LDD speeds using the RF model improved the alignment of NMFDs with actual NMFDs, validating the model's effectiveness in mitigating overestimation bias.

This study introduces a correction methodology for LDD and estimates the
NMFD using data exclusively from an equipped subset of the network. How-
ever, the objective of NMFD is to capture the overall network dynamics, which
are not adequately represented when relying solely on data from a network sub-
set. Consequently, future research should focus on scaling NMFDs from the
LDD-equipped network subset to the entire network. As discussed in Section
3, Mariotte et al. (2020) proposed a scaling factor based on the ratio of to-
tal network length to the equipped network length. However, validating this
scaling factor is crucial. In this context, refining LDD correction methods to
estimate an NMFD for the entire network presents a promising avenue for future
research. In this study, RF emerged as the best model due to its superior per-
formance, interpretability, and computational efficiency. While XGB is a strong
alternative, it requires more tuning and computational resources. MLP, while
theoretically powerful, is less practical for this task due to its complexity, lack of
interpretability, and higher risk of overfitting. Future work could explore hybrid
approaches or further optimize XGBoost and MLP for specific scenarios where
their strengths might be more beneficial. The limitations of this study include
the use of long aggregation periods for LDD and the quality of FCD. The LDD, obtained as open-source data from city transportation agencies, was constrained to an aggregation period of one hour. The one-hour aggregation period for LD data, while practical, restricts temporal resolution, potentially obscuring short-term traffic fluctuations and reducing sensitivity to demand variations. This limitation may introduce biases in speed estimations and weaken correlation analyses. Future research may investigate shorter aggregation periods (e.g., 5 or 15 minutes) to enhance temporal granularity, improve model accuracy, and support real-time traffic management. However, this limitation does not affect the methodological development of the study. Using shorter aggregation periods could enhance data correction and analysis. In future work, we also plan to partition the dataset such that certain links are exclusively reserved for testing, ensuring that the model is evaluated on data it has never encountered during training. This approach will provide a more robust assessment of the model's generalizability and reduce the risk of overfitting. Additionally, we will explore cross-validation techniques across different subsets of links to further validate the model's performance. These steps will strengthen the reliability of our methodology and ensure its applicability to diverse urban networks.

## 6. Acknowledgment

This research has received funding from the European Unions Horizon Europe research and innovation program under Grant Agreement no. 101103808 (ACUMEN). The authors would like to thank Prof. Eleni I. Vlahogianni and her team from NTUA Athens for their support in providing traffic data.
# Appendix A. Link Speed Accuracy of FCD

The confidence metric Z for floating car speed data is derived from the statistical concept of confidence levels. It is calculated using A.1.

$$Z = \frac{\Delta\sqrt{N-1}}{\max(\sigma, \lambda)} \qquad (A.1)$$

where N is the number of probe samples, σ is the standard deviation of these samples, and Δ is the deviation factor. Constant (λ) accounts for measurement characteristics, such as integer-based speed data increasing standard deviation. Z is then mapped into predefined confidence ranges (e.g., 5, 10, 15, ..., 40) indicating reliability levels such as ‘Gap-filled’, ‘Suggestive’, or ‘Highly Confident’. Lower confidence values (ending in 5) correspond to gap-filled records, whereas higher values (e.g., 40) indicate more reliable observations. Analysts can use this confidence field to filter out low-quality data, reducing the impact of outliers in speed analysis. The histogram in Figure A.15 illustrates the distribution of confidence levels in the observed speed data.

<figure><img src="image_16.png" /><figcaption>Figure A.15: Confidence level of FCD Speed estimation for the observation periods</figcaption></figure>

# Appendix B. Spatial Transferability of The Models

To assess the temporal and spatial transferability of our model, we conducted additional analyses using the Athens dataset to complement the Lyon results and vice versa. Spatial transferability was evaluated by comparing LDD speed estimations from models trained in one city and tested in another. As shown in Table B.3, applying a model trained in one city to another resulted in a performance decline of approximately 10% in RMSE, indicating some limitations in spatial generalization. We also analyzed the models’ transferability during
loading and unloading periods. The results indicate that while both models
improved speed correction, the loading phase exhibited slightly better transfer-
ability than the unloading phase. However, despite this decline, the model still
achieved a reasonable correction across different urban networks, demonstrating
its adaptability to diverse traffic conditions. We ensured that distinct training,
testing, and validation datasets were used for temporal transferability in both
cities. The results confirm that the model trained on data from specific days
performed effectively when tested on different days within the same network,
reinforcing its ability to generalize over time.
<table><caption>Table B.3: Evaluation of RF Model Performance for Spatial Transferability in LDD Speed Correction</caption><thead><tr><th rowspan="2">Period</th><th rowspan="2">Data</th><th colspan="3">Training</th><th colspan="5">Testing</th><th rowspan="2">Performance Change (%)</th></tr><tr><th>LDD Original RMSE (Km/h)</th><th>RF Model RMSE (km/h)</th><th>% Improvement</th><th>Data</th><th>LDD Original RMSE (Km/h)</th><th>RF Model RMSE (km/h)</th><th>% Improvement</th></tr></thead><tbody><tr><td rowspan="4">All</td><td>Athens</td><td>16.48</td><td>10.16</td><td>38.32</td><td>Athens</td><td>15.25</td><td>9.54</td><td>37.44</td><td>-</td></tr><tr><td>Lyon</td><td>12.58</td><td>8.76</td><td>30.34</td><td>Athens</td><td>15.25</td><td>10.47</td><td>31.34</td><td><b>-16.3</b></td></tr><tr><td>Lyon</td><td>12.58</td><td>8.76</td><td>30.34</td><td>Lyon</td><td>13.86</td><td>9.81</td><td>29.22</td><td>-</td></tr><tr><td>Athens</td><td>16.48</td><td>10.16</td><td>38.32</td><td>Lyon</td><td>13.86</td><td>10.9</td><td>21.35</td><td><b>-26.93</b></td></tr><tr><td rowspan="4">Loading</td><td>Athens</td><td>16.25</td><td>8.84</td><td>45.55</td><td>Athens</td><td>15.25</td><td>8.4</td><td>44.91</td><td>-</td></tr><tr><td>Lyon</td><td>9.06</td><td>5.15</td><td>43.08</td><td>Athens</td><td>15.25</td><td>8.71</td><td>42.88</td><td><b>-4.52</b></td></tr><tr><td>Lyon</td><td>9.06</td><td>5.16</td><td>43.08</td><td>Lyon</td><td>10.11</td><td>5.79</td><td>42.72</td><td>-</td></tr><tr><td>Athens</td><td>16.25</td><td>8.84</td><td>45.55</td><td>Lyon</td><td>10.11</td><td>6.39</td><td>36.82</td><td><b>-13.8</b></td></tr><tr><td rowspan="4">Unloading</td><td>Athens</td><td>16.3</td><td>9.13</td><td>43.97</td><td>Athens</td><td>15.79</td><td>9.02</td><td>42.87</td><td>-</td></tr><tr><td>Lyon</td><td>10.12</td><td>5.77</td><td>42.95</td><td>Athens</td><td>15.79</td><td>9.97</td><td>36.84</td><td><b>-14</b></td></tr><tr><td>Lyon</td><td>10.12</td><td>5.77</td><td>42.95</td><td>Lyon</td><td>10.96</td><td>6.39</td><td>41.67</td><td>-</td></tr><tr><td>Athens</td><td>16.3</td><td>9.13</td><td>43.97</td><td>Lyon</td><td>10.96</td><td>6.85</td><td>37.54</td><td><b>-9.9</b></td></tr></tbody></table>
References

T. Z. Qiu, X.-Y. Lu, A. H. F. Chow, S. E. Shladover, Estimation of Freeway Traffic Density with Loop Detector and Probe Vehicle Data, Transportation Research Record: Journal of the Transportation Research Board 2178 (2010) 21–29. doi:10.3141/2178-03.

B. Kerner, C. Demir, R. Herrtwich, S. Klenov, H. Rehborn, M. Aleksi, A. Haug, Traffic state detection with floating car data in road networks, in: Proceedings. 2005 IEEE Intelligent Transportation Systems, 2005., IEEE, 2005, pp. 700–705. doi:10.1109/ITSC.2005.1520133.

M. Saeedmanesh, N. Geroliminis, Clustering of heterogeneous networks with directional flows based on "Snake" similarities, Transportation Research Part B: Methodological 91 (2016) 250–269. doi:10.1016/j.trb.2016.05.008.

L. Leclercq, N. Chiabaut, B. Trinquier, Macroscopic Fundamental Diagrams: A cross-comparison of estimation methods, Transportation Research Part B: Methodological 62 (2014) 1–12. URL: http://dx.doi.org/10.1016/j.trb.2014.01.007. doi:10.1016/j.trb.2014.01.007.

C. Huang, N. Zheng, J. Zhang, Investigation of Bimodal Macroscopic Fundamental Diagrams in Large-Scale Urban Networks: Empirical Study with GPS Data for Shenzhen City, Transportation Research Record 2673 (2019) 114–128. doi:10.1177/0361198119843472.

N. Geroliminis, C. F. Daganzo, Existence of urban-scale macroscopic fundamental diagrams: Some experimental findings, Transportation Research Part B: Methodological 42 (2008) 759–770. doi:10.1016/j.trb.2008.02.002.

V. V. Gayah, C. F. Daganzo, Clockwise hysteresis loops in the Macroscopic Fundamental Diagram: An effect of network instability, Transportation Research Part B: Methodological 45 (2011) 643–655. doi:10.1016/j.trb.2010.11.006.

N. Geroliminis, J. Sun, Properties of a well-defined macroscopic fundamental diagram for urban traffic, Transportation Research Part B: Methodological 45 (2011) 605–617. URL: http://dx.doi.org/10.1016/j.trb.2010.11.004. doi:10.1016/j.trb.2010.11.004.

H. Mahmassani, T. Hou, M. Saberi, Connecting networkwide travel time reliability and the network fundamental diagram of traffic flow, Transportation Research Record (2013) 80–91. doi:10.3141/2391-08.

M. Saberi, H. S. Mahmassani, T. Hou, A. Zockaie, Estimating network fundamental diagram using three-dimensional vehicle trajectories: Extending edie's definitions of traffic flow variables to networks, Transportation Research Record 2422 (2014) 12–20. doi:10.3141/2422-02.
T. Tsubota, A. Bhaskar, E. Chung, Macroscopic Fundamental Diagram for Brisbane, Australia, Transportation Research Record: Journal of the Transportation Research Board 2421 (2014) 12-21. doi:10.3141/2421-02.

L. Ambühl, M. Menendez, Data fusion algorithm for macroscopic fundamental diagram estimation, Transportation Research Part C: Emerging Technologies 71 (2016) 184-197. doi:10.1016/j.trc.2016.07.013.

J. Du, H. Rakha, V. V. Gayah, Deriving macroscopic fundamental diagrams from probe data: Issues and proposed solutions, Transportation Research Part C: Emerging Technologies 66 (2016) 136-149. doi:10.1016/j.trc.2015.08.015.

L. Ambühl, A. Loder, M. Menendez, K. W. Axhausen, Empirical Macroscopic Fundamental Diagrams: New insights from loop detector and floating car data (2017). URL: https://doi.org/10.3929/ethz-b-000167171. doi:10.3929/ethz-b-000167171.

J. Shim, J. Yeo, S. Lee, S. H. Hamdar, K. Jang, Empirical evaluation of influential factors on bifurcation in macroscopic fundamental diagrams, Transportation Research Part C: Emerging Technologies 102 (2019) 509-520. doi:10.1016/j.trc.2019.03.005.

N. Geroliminis, N. Zheng, K. Ampountolas, A three-dimensional macroscopic fundamental diagram for mixed bi-modal urban networks, Transportation Research Part C: Emerging Technologies 42 (2014) 168-181. doi:10.1016/j.trc.2014.03.004.

A. Loder, L. Ambühl, M. Menendez, K. W. Axhausen, Empirics of multi-modal traffic networks: Using the 3D macroscopic fundamental diagram, Transportation Research Part C: Emerging Technologies (2017). doi:10.1016/j.trc.2017.06.009.

H. Fu, Y. Wang, X. Tang, N. Zheng, N. Geroliminis, Empirical analysis of large-scale multimodal traffic with multi-sensor data, Transportation Research Part C: Emerging Technologies 118 (2020). doi:10.1016/j.trc.2020.102725.

S. Lu, V. L. Knoop, M. Keyvan-Ekbatani, Using taxi GPS data for macroscopic traffic monitoring in large scale urban networks: Calibration and MFD derivation, in: Transportation Research Procedia, volume 34, Elsevier B.V., 2018, pp. 243-250. doi:10.1016/j.trpro.2018.11.038.

G. Lee, Z. Ding, J. Laval, Effects of loop detector position on the macroscopic fundamental diagram, Transportation Research Part C: Emerging Technologies 154 (2023). doi:10.1016/j.trc.2023.104239.

J. Sin, S.-H. Lee, K. Choi, A single loop detector-based freeway speed estimation, KSCE Journal of Civil Engineering 6 (2002) 1-9. doi:10.1007/BF02829034.
S. Peeta, I. Anastassopoulos, Automatic Real-Time Detection and Correction of Erroneous Detector Data with Fourier Transforms for Online Traffic Control Architectures, Transportation Research Record: Journal of the Transportation Research Board (2002).

B. Coifman, S. Kim, Speed estimation and length based vehicle classification from freeway single-loop detectors, Transportation Research Part C: Emerging Technologies 17 (2009) 349–364. doi:10.1016/j.trc.2009.01.004.

Yufei Yuan, J. W. C. van Lint, Th. Schreiter, S.P. Hoogendoorn, J. L. M. Vrancken, Automatic Speed-Bias Correction with Flow-Density Relationships, in: Networking, Sensing and Control (ICNSC), IEEE, 2010.

J. Corey, Y. Lao, Y. J. Wu, Y. Wang, Detection and correction of inductive loop detector sensitivity errors by using gaussian mixture models, Transportation Research Record (2011) 120–129. doi:10.3141/2256-15.

B. Coifman, S. Neelisetty, Improved speed estimation from single-loop detectors with high truck flow, Journal of Intelligent Transportation Systems: Technology, Planning, and Operations 18 (2014) 138–148. doi:10.1080/15472450.2013.801708.

N. Maiti, M. Seppecher, L. Leclercq, Scaling Methods To Estimate Macroscopic Fundamental Diagrams in Urban Networks with Sparse Sensor Coverage, 2024. URL: https://doi.org/10.48550/arXiv.2411.19721. doi:https://doi.org/10.48550/arXiv.2411.19721.

N. Maiti, M. Seppecher, L. Leclercq, Unveiling Challenges in Empirical Estimation of Multimodal Network Macroscopic Fundamental Diagrams: A Data Fusion Perspective, in: 104th Transportation Research Board Annual Meeting (TRB 2025), 2025.

E. Barmpounakis, N. Geroliminis, On the new era of urban traffic monitoring with massive drone data: The pNEUMA large-scale field experiment, Transportation Research Part C: Emerging Technologies 111 (2020) 50–71. URL: https://doi.org/10.1016/j.trc.2019.11.023. doi:10.1016/j.trc.2019.11.023.

L. Leclercq, Calibration of Flow-Density Relationships on Urban Streets, Transportation Research Record: Journal of the Transportation Research Board 226 (2005) 226–234.

G. Mariotte, L. Leclercq, S. F. Batista, J. Krug, M. Paipuri, Calibration and validation of multi-reservoir MFD models: A case study in Lyon, Transportation Research Part B: Methodological 136 (2020) 62–86. doi:10.1016/j.trb.2020.03.006.

K. Jordahl, J. V. d. Bossche, M. Fleischmann, J. Wasserman, geopandas: v0.8.1, 2020.
M. Paipuri, Y. Xu, M. C. González, L. Leclercq, Estimating MFDs, trip lengths and path flow distributions in a multi-region setting using mobile phone data, Transportation Research Part C: Emerging Technologies 118 (2020). doi:10.1016/j.trc.2020.102709.

C. Buisson, C. Ladier, Exploring the impact of homogeneity of traffic measurements on the existence of macroscopic fundamental diagrams, Transportation Research Record (2009) 127–136. doi:10.3141/2124-12.

L. Leclercq, M. Paipuri, Macroscopic Traffic Dynamics Under Fast-Varying Demand, Transportation Science 53 (2019). URL: https://hal.science/hal-02497472v2. doi:10.1287/trsc.2019.0908{"{i}}.

A. Loder, L. Ambühl, M. Menendez, K. W. Axhausen, Understanding traffic capacity of urban networks, Scientific Reports 9 (2019) 1–10. doi:10.1038/s41598-019-51539-5.

L. Ambühl, A. Loder, M. C. Bliemer, M. Menendez, K. W. Axhausen, A functional form with a physical meaning for the macroscopic fundamental diagram, Transportation Research Part B: Methodological 137 (2020) 119–132. doi:10.1016/j.trb.2018.10.013.

O. Mousavizadeh, M. Keyvan-Ekbatani, On the important features for a well-shaped reduced network MFD estimation during network loading and recovery, Transportation Research Part C: Emerging Technologies 161 (2024). doi:10.1016/j.trc.2024.104539.

Tin Kam Ho, Random decision forests, in: Proceedings of 3rd International Conference on Document Analysis and Recognition, IEEE Comput. Soc. Press, 1995, pp. 278–282. doi:10.1109/ICDAR.1995.598994.

N. Maiti, B. R. Chilukuri, Does anisotropy hold in mixed traffic conditions?, Physica A: Statistical Mechanics and its Applications 632 (2023) 129336. doi:10.1016/j.physa.2023.129336.

T. Chen, C. Guestrin, XGBoost: A Scalable Tree Boosting System (2016). doi:10.1145/2939672.2939785.

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, A. Müller, J. Nothman, G. Louppe, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, . Duchesnay, Scikit-learn: Machine Learning in Python (2012).