  Deep deterministic policy gradient (DDPG) based car-following strategy can
break through the constraints of the differential equation model due to the
ability of exploration on complex environments. However, the car-following
performance of DDPG is usually degraded by unreasonable reward function design,
insufficient training and low sampling efficiency. In order to solve this kind
of problem, a hybrid car-following strategy based on DDPG and cooperative
adaptive cruise control (CACC) is proposed. Firstly, the car-following process
is modeled as markov decision process to calculate CACC and DDPG simultaneously
at each frame. Given a current state, two actions are obtained from CACC and
DDPG, respectively. Then an optimal action, corresponding to the one offering a
larger reward, is chosen as the output of the hybrid strategy. Meanwhile, a
rule is designed to ensure that the change rate of acceleration is smaller than
the desired value. Therefore, the proposed strategy not only guarantees the
basic performance of car-following through CACC, but also makes full use of the
advantages of exploration on complex environments via DDPG. Finally, simulation
results show that the car-following performance of proposed strategy is
improved significantly as compared with that of DDPG and CACC in the whole
state space.
