# A Survey on Theories and Applications for Self-Driving Cars Based on Deep Learning Methods
## 4 Applications Overview of Deep Learning in the Field of Self-Driving Cars
### Other Applications
#### 4.4.1 Path Planning  
Path planning plays a significant role in autonomous driving and has been extensively studied for decades. There are lots of methods used for path planning, such as Particle Swarm Optimization (PSO), Genetic Algorithm (GA), and so on [77; 78]. However, these conventional planning algorithms are not very suitable for the path planning task of self-driving cars under complex environments. Owing to impressive advantages in extracting features, deep learning is promising to overcome the bottlenecks of conventional planning algorithms and learn to plan paths efficiently under various conditions for self-driving cars. For example, [PERSON] et al. [79] proposed a path planning method based on deep reinforcement learning. This method can deal with the problem of the model training with continuous input and output. Thus, it can output the control action and trajectory sequence directly.  
The flow chart of the method proposed in [79] is shown in Figure 18. There are an actor policy network and a critic evaluation network. The two neural networks are both based on DenseNet (Dense Convolutional Network). The input of the actor policy network is \(s_{a}=(s,v,a_{old})\), where \(s\) are the states obtained by sensors, \(v\) is the speed and \(a_{old}\) is the old action. Three fully connected networks are used as the hidden layers of the actor policy network. Tanh function is used as the activation function of this network's last layer. The output of the actor policy network is the new action \(a_{new}\). The input to the critic evaluation network is the union of the state \(s_{a}\) and the new action \(a_{new}\). Thus, the output of the critic evaluation network is the corresponding Q-value \(Q(s_{a},a_{new})\), which is as follows:  
\[y=kx+b, \tag{16}\]  
where \(y\) is the output Q-value; \(x\) is the input; \(k\) is the weight and \(b\) is the bias.  
In the method of [79], the Actor network includes an online policy network and a target policy network. In this actor network, an action is gotten based on the deterministic strategy from the current state. The online policy network of the actor is updated with sampling gradient:  
\[\
abla_{\theta^{\mu}}\mu\left|s_{i}\right.\approx\frac{1}{N}\sum_{i}\
abla_{a} Q(s,a\left|\theta^{Q}\right.)\left|s_{s=s_{i},a=\mu(s_{i})}\right.\
abla_{\theta^{ \mu}}\mu(s\left|\theta^{\mu}\right.)\left|s_{i}\right., \tag{17}\]  
where \(\theta^{Q}\) and \(\theta^{\mu}\) are the parameters of critic online Q network and actor's online policy network respectively; \(N\) is the number of batches; and \(\mu(s_{i})\) denotes current strategy at the state \(s_{i}\).  
Figure 18: The flow chart of the method proposed in [79].  
The critic network includes an online Q network and a target Q network. In this critic network, the Bellman equation is used to evaluate the quality of action. The online Q network in the critic network is updated by:  
\[L{=}\frac{1}{N}\sum_{i}\Big{(}y_{i}-Q(s_{i},a_{i}\left|\theta^{Q}\right.\Big{)}^ {2}. \tag{18}\]  
In the method proposed in [79], the reinforcement learning model is trained based on a deep deterministic policy gradient and a vehicle dynamic model. So it has the advantages of deep Q network and can ensure the convergence of the network. The performance of this method is better than the traditional trajectory planning method.