## 3 Dpd3 qn
### Priority Experience Replay
#### 3.2.2 Prioritization Based on Segment Sampling  
Based on the TD error-based priority, this paper further introduces Stochastic Prioritization [20], which is inspired by the mini-batch gradient descent [21] in the Deep Deterministic Policy Gradient (DDPG) deep reinforcement learning algorithm. By dividing the data into multiple small batches, segmented sampling can achieve efficient learning and stability in complex environments. This method effectively improves the utilization of data in the experience pool and enables samples of different priorities to be effectively extracted, thereby accelerating the convergence of the model and avoiding overfitting or insufficient data utilization in a high-dimensional continuous action space.  
The specific steps of segmented sampling priority are as follows: First, the features are divided into two main segments: the position segment, including the distance to the preceding vehicle or the relative lane position, and the speed segment, covering the vehicle speed range. The data of the entire experience replay buffer are divided into different segments according to these features. Then, the segment to which each sample \(i\) belongs is defined as \(s(i)\) and a weight is set for each segment.  
For the position segment, the priority can be set according to the scarcity and importance of the preceding vehicle distance. In certain specific distance ranges (such as close or far distances), the number of samples is small, and a higher weight should be given to ensure that the scenes of these critical distances are fully paid attention to. This diversity can be captured by measuring the standard deviation of samples within a position segment. The specific weight setting can be expressed as Equation (21):  
\[w_{\text{position}}(s(i))=g(n(s(i)),\sigma_{\text{pos}}(s(i))) \tag{21}\]  
where \(w_{\text{position}}(s(i))\) is the weight of the position segment \(s(i)\), \(n(s(i))\) is the number of samples in segment \(s(i)\), and \(\sigma_{\text{pos}}(s(i))\) is the standard deviation of the distance to the preceding vehicle in segment \(s(i)\). A higher standard deviation indicates that the scene changes more within the segment, so more attention is needed. The function g(\(\cdot\)) is a mapping function that combines the number of samples and the variability in position to determine the importance of the position segment. It typically increases with larger\(\sigma_{\text{pos}}\) and may decrease or normalize based on \(n(s(i))\), thereby balancing between data abundance and scene dynamics.  
Meanwhile, the weight of the speed segment can be expressed as Equation (22):  
\[w_{\text{speed}}(s(i))=h(n(s(i)),\sigma_{\text{speed}}(s(i))) \tag{22}\]  
where \(w_{\text{speed}}(s(i))\) is the weight of speed segment \(s(i)\) and \(\sigma_{\text{speed}}(s(i))\) variability is determined by the standard deviation of the speed measurements within segment \(s(i)\). In addition, emergency situations on highways (such as emergency braking or avoidance) are more important than ordinary driving scenarios, so these segments are artificially given higher weights. The function h(\(\cdot\)) is similar to g(\(\cdot\)), serving as a mapping from the speed variability and the segment size to a scalar importance value, emphasizing segments with abrupt speed changes or rare events.  
The weight of ordinary driving can be set to \(w_{\text{normal}}=1\), the weight of emergency braking to \(w_{\text{emergency}}=3\), the weight of lane change operation to \(w_{\text{lane change}}=3\), and the weight of close driving to \(w_{\text{close}}=2\). By setting priorities in this way, we ensure that important scenarios are fully paid attention to during training, thereby improving the overall performance and adaptability of the model.