## 5 Open Traffic Data Framework in the Context of Deep Learning  
Most recently, a significant increase in the availability of real-time traffic data can be noticed in the complex and ever-expanding traffic environments. This increase corresponds with the two modern trends that are happening in parallel. The first trend is related with a high percentage of smartphone ownership, implementation of innovative traffic sensors, and development of infrastructure for fast and secure data exchange, while the second trend is related with the increase in traffic demand for daily migrations due to a growing number of personal vehicles and traffic network capacity expansion in overpopulated urban regions. Datasets obtained from those evolving traffic environments are nowadays mainly stored at the cloud-oriented _data lakes_. Those datasets contain various traffic data such as GPS trails of vehicles/mobile phone users; vehicle positioning and queue counting at intersections by using advanced video processing [56] and other sensor technologies such as inductive loops, weather, and condition of road surface data; satellite and drone imagery; social network data; etc. The existing traffic datasets will be significantly expanded by the upcoming CAVs. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**DRL Method/Year** & **DNN Configuration** & **DNN Parameters**1  & **Learning Episodes** \\ \hline \multirow{4}{*}{Regional A3C + PER 2019 [17];} & Single A3C & 1: (32, 4\(\times\)4, 2) & \multirow{4}{*}{4000} & \multirow{4}{*}{1 Million} \\  & CNN & (64, 2\(\times\)2, 1) & & \\  & (1: 3: 5-Conv, & (128, 2.2, 12) & & \\  & 2: 1\(\times\)FC, & 2: 128 & & /32 & \\  & 3: 2\(\times\)LSTM 2  & \multirow{4}{*}{3} \\  & Single A2C & 1 input 28 & \multirow{4}{*}{Orthogonal} & \multirow{4}{*}{1000} & \multirow{4}{*}{1400} \\  & 3 inputs & 2 input 32 & & \\  & per 1\(\times\)FC, & 3 inputs 64 & RMSprop as & & (1 episode \\  & 1\(\times\)LSTM 3  & \multirow{4}{*}{LSTM: 4} \\  & Regional & Agent: & \multirow{4}{*}{1000} & \multirow{4}{*}{1000} \\  & A2C agents: & Critic net & & \\  & 2: 3\(\times\)(MLP-ReLU) 4  & \multirow{4}{*}{Adam} & \multirow{4}{*}{-64} & \multirow{4}{*}{1 episodes} \\  & Global layer: & & & \\  & 1\(\times\)FC+ReLU & (200, 200, 100) & & \\ \hline \multirow{4}{*}{3 DQN + PER 2019 [46];} & \multirow{4}{*}{\ \begin{tabular}{} \end{tabular}  } & 1: (32, 3\(\times\)15, 3) & \multirow{4}{*}{100,000} & \multirow{4}{*}{1000} \\  & 3\(\times\)Conv & 2: (64, 2\(\times\)2, 2) & & \\  & 2: 3\(\times\)(2, 2, 2, 1) & & /32 ; & \\ \hline \multirow{4}{*}{ResNet based A2C 2018 [18];} & Single A2C DNN, & Filters & \multirow{4}{*}{1000} & \multirow{4}{*}{1000} \\  & 4\(\times\)ResNet Blocks & for Conv & & \\  & (2\(\times\)Conv, 2:=BN, ReLU) & in ResNet & Adam & /64x\(\times\) \\  & Actor and Critic each & (32, 64, & (16 agents) & (1 episode is 3600 TS) \\  & (1\(\times\)conv, 1:=BN, 2:FC) & 128, 256 & & \\ \hline \multirow{4}{*}{DRG 2017 [20]} & \multirow{4}{*}{ \begin{tabular}{} \end{tabular}  } & Critic (4\(\times\)nd & \multirow{4}{*}{-} & \multirow{4}{*}{1000} \\  & 1\(\times\)BN+ReLU & + np, 1\(\times\)nd & & \\  & Gaussian Noise) & Actor: (2\(\times\)nd & [PERSON] & /- & episodes \\  & (4\(\times\)[PERSON]+[PERSON]) 6 [FOOTNOTE:6]Footnote 6: \(\Those vehicles are equipped with advanced on-board sensors that can generate more accurate traffic data and they have the ability to extend its sensory range by exchanging data with other CAVs and road-side-sensory-units [57]. According to the above, current traffic datasets can be categorized as _big data_ considering their volume, variety, and velocity. Traffic _big data_ are created based on mobile users data, live traffic video signal processing, CAVs, and other traffic sensors telemetry, and it can complete missing pieces of data through the data fusion process. The DNN models in DRL frameworks dedicated for traffic control require a lot of raw traffic data which must be systematically extracted from different sources, fused, and arranged in the large sets of image-like data structures.