## 5 Validation of the Proposed CARLA+
### Experiments
#### Dataset Creation  
For this experiment, we aim to model the effect of weather and time of day on traffic count, pedestrian counts, and vehicle speed in a dynamic environment. After a thorough search, we did not find any relevant dataset which contained data about traffic count, traffic speed, pedestrian count, and weather. Therefore, we created such a dataset ourselves and selected the dataset for New York City, as it satisfied all of our data requirements. The detail of all the data sources that we used to create the new dataset are discussed below:  
* **Traffic Count:** for the traffic count, we used New York City's open data platform [20] which contains data for traffic volume counts. It contains traffic volume data of different streets in the boroughs of New York City, taken at 15-minute intervals.
* **Pedestrian Count:** for pedestrians counts, we used the Brooklyn Bridge Automated Pedestrian Counts data [21], which contains pedestrian count as well as basic weather data, taken at 1-hour intervals.
* **Traffic Speed:** we used New York City's Real-Time Traffic Speed data [22] which is comprised of the traffic speed as well as the borough where the data were captured.
* **Weather:** the NYC weather data are taken from Open-Meteo's Historical Weather API [23] for the city of Brooklyn.  
We selected the data from the year 2019 as they contained the largest number of entries in all the datasets, and created a unified dataset containing 8670 rows corresponding to every hour in the year 2019. Our new dataset is openly available for researchers at [[https://www.kaggle.com/datasets/aadimator/brooklyn-2019-traffic-data](https://www.kaggle.com/datasets/aadimator/brooklyn-2019-traffic-data)]([https://www.kaggle.com/datasets/aadimator/brooklyn-2019-traffic-data](https://www.kaggle.com/datasets/aadimator/brooklyn-2019-traffic-data)) (accessed on 2 January 2023) to experiment and validate their solutions.  
Figure 9: Stages to learn the Bayesian Network.  
Data Preparation and Feature Engineering  
Once we have data corresponding to every hour in the year 2019, we filtered it to only contain the data for Brooklyn City, as the pedestrian count only corresponds to that. To handle the missing entries, we employed the _backward fill_ method, where the null entry is replaced with the previous entry. For this experiment, we aim to validate the effect of weather and time of day on traffic and pedestrian counts, as well as their effect on traffic speed. Thus, we filtered only the relevant columns from the dataset, resulting in the following final columns:  
* Hour
* Pedestrians
* Traffic
* Speed
* Rain
* Fog
* Clouds  
As most of these columns contain continuous values, we further discretize them to be used in a BN. The variable Hour is classified into the time of days, as shown in Table 5. Moreover, the Rain, Fog, and Clouds variables are discretized into four categories based on the rules defined in Table 6. As for the discretization of Traffic, Pedestrians, and Speed parameters, they are divided into three bins based on their data distributions, as shown in Table 7. The final processed and discretized dataset is made publicly available at the following link [[https://www.kaggle.com/datasets/aadimator/brooklyn-2019-traffic-data](https://www.kaggle.com/datasets/aadimator/brooklyn-2019-traffic-data)]([https://www.kaggle.com/datasets/aadimator/brooklyn-2019-traffic-data](https://www.kaggle.com/datasets/aadimator/brooklyn-2019-traffic-data)) (accessed on 2 January 2023).