# Design of a Reinforcement Learning-Based Lane Keeping Planning Agent for Automated Vehicles
## 2 Environment
### Vehicle Model
The equations of the chassis are based on [PERSON]'s second law:  
\[\dot{x}=\frac{1}{m}(F_{f,x}+F_{r,x}+F_{d,x}) \tag{1}\]  
\[\dot{y}=\frac{1}{m}(F_{f,y}+F_{r,y}+F_{d,y}) \tag{2}\]  
\[\ddot{\Psi}=\frac{1}{\theta}(l_{f}F_{l,f}^{V}-l_{r}F_{r,y}^{V}) \tag{3}\]  
where the \(F_{[f/r],[x,y]}\) variables are the tire forces of the vehicle, the forces derived from aerodynamic drag is modeled and calculated as follows,  
\[F_{d,x}^{V}=\frac{1}{2}c_{D}A_{f}\rho_{A}\dot{x}^{V}\sqrt{(\dot{x}^{V})^{2}+( \dot{y}^{V})^{2}} \tag{4}\]  
\[F_{d,y}^{V}=\frac{1}{2}c_{D}A_{f}\rho_{A}\dot{y}^{V}\sqrt{(\dot{x}^{V})^{2}+( \dot{y}^{V})^{2}} \tag{5}\]  
where \(\rho_{A}\) is the density of the air, \(c_{D}\) is the coefficient of drag, and \(A_{f}\) is the size of the frontal surface of the vehicle. Only the equations of the front wheel are presented because the rear and front are modeled identically, the forces of the tire are derived from the motion of the wheel, and the equations are formalized by taking into consideration [PERSON]'s second law and the dynamic slip equations from the work in [30]:  
\[\ddot{\phi}=\frac{1}{\theta_{f}}(M_{f,d}-r_{f}F_{f,x}^{W}-M_{f,b}-M_{f,rr}) \tag{6}\]  
\[\dot{s}_{f,x}=\frac{1}{I_{f,x}}(r_{f}\dot{\phi_{f}}-\dot{x}_{f}^{W}-|\dot{x}_{ f}^{W}|s_{f,x}) \tag{7}\]  
\[\dot{s}_{f,y}=\frac{1}{I_{f,y}}(-\dot{y}_{f}^{W}-|\dot{y}_{f}^{W}|s_{f,y}) \tag{8}\]The variable \(\hat{x}_{f}^{W}\) is the longitudinal velocity of the center of the wheel, while the \(\hat{y}_{f}^{W}\) is the lateral velocity of the same wheel center. The relaxation lengths which depend on the lateral and longitudinal slips are calculated as follows,  
Figure 3:0  
The \(l_{f,[x/y],0}\) represents the values at wheel lock or wheel spin, while \(l_{f,[x/y],min}\) notes the value for the standstill case. The SAE J2452 standard is used to calculate the torque of the rolling resistance \(M_{fr,r}\), and the Magic formula provides the computational model for the calculation of the tire forces, which looks as follows,  
\[ \begin{split}\hat{F}_{f,[x/y]}^{W}=&\mu_{f}F_{f,z} ^{W}\sin\{C_{f,[x/y]}\arctan(B_{f,[x/y]}\hat{s}_{f,[x/y]}-\\ & E[B_{f,[x/y]}\tilde{s}_{f,[x/y]}-\arctan(B_{f,[x/y]}\tilde{s}_{ f,[x/y]})])\}.\end{split}  \tag{10}\]  
To enhance and maintain the stability of the numerical computational process, dumped slip values are used in the calculation of the forces, which looks as follows,  
\[\tilde{s}_{f,x}=s_{f,x}+\frac{k_{f,x}}{B_{f,x}C_{f,x}\mu_{f}F_{f,z}^{W}}(r_{f }\dot{\rho}_{f}-\dot{x}_{f}^{W}) \tag{11}\]  
\[\tilde{s}_{f,y}=s_{f,y} \tag{12}\]  
where \(k_{f,x}\) is the factor that depends on the velocity, and it is calculated as follows,  
\[k_{f,x}= \begin{cases}\frac{1}{2}k_{f,x,0}(1+\cos(\pi\frac{|\hat{x}_{f}^{W}|}{ v_{low}}))&\text{if }\dot{x}_{f}^{W}\leq v_{low}\\ 0,&\text{otherwise}\end{cases}  \tag{13}\]  
where the damping factor \(k_{f,x,0}\) belongs to the zero velocity case, and the damping factor is switched off for the \(v_{low}\) case. The friction ellipse method is used to calculate the superposition of the forces, which is given as follows,  
\[F_{f,x}^{W}=sign(\tilde{s}_{f,x})\sqrt{\frac{(\hat{F}_{f,x}^{W}\hat{F}_{f,y}^ {W})^{2}}{(\hat{F}_{f,y}^{W})^{2}+(\frac{\hat{s}_{f,y}}{\hat{s}_{f,x}}\hat{F}_ {f,x}^{W})^{2}}} \tag{14}\]  
\[F_{f,y}^{W}=sign(\tilde{s}_{f,y})\sqrt{\frac{(\hat{F}_{f,x}^{W}\hat{F}_{f,y}^ {W})^{2}}{(\hat{F}_{f,x}^{W})^{2}+(\frac{\hat{s}_{f,x}}{\hat{s}_{f,y}}\hat{F}_ {f,y}^{W})^{2}}} \tag{15}\]  
An explicit Ordinary Differential Equation solver is used with a step size of approximately 1 ms for the introduced wheel model. To further enhance the computational efficiency of the vehicle model, it is implemented in C instead of python. For more details about the vehicle model see the works in [29; 31].