## 4 Experiment Results and Discussion
### Simulation Results  
The performance of the obtained control policy for CACC based on SRL is investigated via simulation with two ideal driving cycles, including the step acceleration cycle and the sinusoidal velocity cycle, and a collected real driving cycle. In [22], a linear feedforward and feedback control algorithm for CACC is proposed and tested on a vehicle. Here, the linear controller is also adopted for comparison. With the first two ideal cycles, the SRL control policy trained by samples of Driver 1 is compared with the linear control algorithm. With the real driving cycle, the SRL control policies trained by samples of Driver 1 and Driver 2 are compared to evaluate the learning ability of SRL and validate the effect of the supervisor. The time headway is set as 1 s, and the safe distance at standstill is 2 m.  
Figure 5 presents the simulation results with the cycle of step accelerating and decelerating commands, followed by the maximum, average, and variance of the inter-vehicle distance error listed in Table 2. The initial distance, the initial velocity of the preceding vehicle, and the host vehicle are 7 m, 36 km/h, and 18 km/h, respectively. Note that for a fair comparison, the desired distance curve is calculated using the velocity of the preceding vehicle in this and following figures in this section. With the SRL-based control policy, the host vehicle tracks the velocity of the preceding vehicle well while maintaining the desired inter-vehicle distance. The velocity error and distance error of the SRL control policy damp more quickly compared with the linear control. More specifically, at the beginning of the simulation and the rising/falling edge of the step command, the SRL policy tends to generate a larger acceleration to achieve a faster response and more accurate inter-vehicle distance control. The maximum, average, and variance of the distance error of the SRL control policy are less than that of the linear controller, which indicates that the SRL control policy achieves a better following accuracy. Figure 6 shows the simulation results of the sinusoidal velocity cycle. The corresponding maximum, average, and variance of the inter-vehicle distance error are listed in Table 3. At the beginning of the simulation, the SRL policy provides a larger acceleration than the linear controller to pursue the velocity of the preceding vehicle in a short period of time. After that, the SRL policy performs as well as the linear controller in velocity tracking. As shown in Figure 6c, the proposed SRL policy exceeds the linear controller in maintaining the desired distance. The distance error of SRL policy is less than the linear controller as shown in Table 3. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Algorithm** & **Number of Experiments** & **Success Rate** & **Average Number of Trials** \\ \hline SRL & 100 & 100\% & 128 \\ RL & 100 & 34\% & 724 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Training results of SRL and RL. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Control Policy** & **Maximum (m)** & **Average (m)** & **Variance** \\ \hline SRL control & 7.50 & 0.48 & 2.05 \\ Linear control & 9.86 & 0.52 & 4.57 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Maximum, average, and variance of distance error, with the cycle of step acceleration and deceleration. Figure 5: Simulation results with the cycle of step acceleration and deceleration: (**a**) velocity, (**b**) acceleration, and (**c**) distance. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Control Policy** & **Maximum (m)** & **Average (m)** & **Variance** \\ \hline SRL control & 2.84 & 0.23 & 0.31 \\ Linear control & 5.07 & 0.43 & 1.83 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Maximum, average, and variance of distance error, with the cycle of sinusoidal velocity. Moreover, the performance of the SRL control policies trained by different drivers are compared using a real driving cycle collected on an urban road in Beijing.