# Reinforcement Learning Decision-Making for Autonomous Vehicles Based on Semantic Segmentation
## 2 Materials and Methods
### Agent Model
#### 2.3.3 Incentive Mechanism  
The reward mechanism determined the convergence speed and decision accuracy of the agent model. The following were the five designed reward mechanism indicators:  
(1) Safe distance threshold \(R_{o}\) and collision reward \(R_{c}\)  
Reward outcome for evaluating the relationship between the ego vehicle and obstacles \(R_{o}\):  
\[R_{o}=\left\{\begin{array}{l}C\cdot\omega,\min(d_{adv1},d_{adv2})>T\\ -C\cdot\omega,\text{otherwise}\end{array}\right. \tag{7}\]  
In the formula, \(C\) is a constant representing the baseline magnitude affecting this component of the total reward, \(\omega\) is the adjustment weight of the constant \(C\), and \(T\) is the threshold for determining whether the vehicle maintains a safe distance from obstacles ahead [20]. \(\min(d_{adv1},d_{adv2})\) is the minimum distance between the ego vehicle and the detected obstacle.  
The reward value \(R_{c}\) was assigned a negative value when the autonomous vehicle collided.  
Its calculation formula is as follows:  
\[R_{c}=\left\{\begin{array}{l}-C_{c}\cdot\omega_{c},\text{Collision with an obstacle}\\ 0,\text{otherwise}\end{array}\right. \tag{8}\]  
In the formula, \(C_{c}\) is a constant representing the baseline magnitude affecting the total reward; \(\omega_{c}\) is the adjustment weight of \(C_{c}\).  
(2) Lane deviation reward \(R_{i}\)  
The correctness of the ego vehicle's driving on the lane was determined by the distance difference, calculating the lateral distance (perpendicular to the driving direction) between the vehicle's current position and the lane centerline. The smaller the distance, the higher the reward given, with the reward reaching its maximum value when the distance was 0 (exactly on the centerline). Its calculation formula is:  
\[R_{i}=\text{max\_}R-k\times|d_{c}| \tag{9}\]  
In the formula, max_\(R\) is the maximum reward value set when the vehicle is exactly on the centerline, \(k\) is a scaling function used to adjust the sensitivity of the reward to the distance, and \(d_{c}\) is the lateral distance between the vehicle's current position and the lane centerline.  
Speed and steering angle variation rewards \(R_{s}\), \(R_{t}\)  
The driving stability of the self-driving vehicle in the test scenario needs to be given to the driving direction and driving speed of the two indicators to set the appropriate reward values \(R_{s}\), \(R_{t}\). As such, the model in the training process controls the driving direction and speed of the self-driving vehicle without excessive deviation, ensuring the stability of the driving vehicle. If the direction and speed of the two changes, compared to the previous, did not occur with many deviation changes, the reward value feedback to the model is high, and if vice versa, the value feedback is low. Therefore, the formula for the two reward values is:  
\[R_{s}=(T_{s}-|Sr_{t}-Sr_{t-1}|)\times w_{s} \tag{10}\]  
\[R_{t}=(T_{t}-|Th_{t}-Th_{t-1}|)\times w_{t} \tag{11}\]  
In the formula, \(T_{s}\) represents the threshold for whether the vehicle suddenly makes a sharp change in direction. \(Sr_{t}\) and \(Sr_{t-1}\) represent the steering angles of the vehicle at time \(t\) and \(t-1\) (\(t>0\)), respectively. \(w_{s}\) is the adjustment weight for the steering reward value. \(T_{t}\) is the threshold for determining whether the vehicle suddenly makes a sharp change in speed. \(Th_{t}\) and \(Th_{t-1}\) represent the vehicle's speed at time \(t\) and \(t-1\) (\(t>0\)), respectively, and \(w_{t}\) is the adjustment weight for the vehicle's speed reward value.  
The total reward value was composed as the sum of the above five reward values. The equation is:  
\[R_{A}=R_{o}+R_{i}+R_{c}+R_{s}+R_{t} \tag{12}\]