# Borehole Trajectory Optimization Design Based on the SAC Algorithm with a Self-Attention Mechanism
## 4 Experimental Design and Result Analysis
### Comparative Analysis of the Self-Attention Mechanism  
In order to evaluate the actual effect of the self-attention mechanism in wellbore trajectory design tasks, based on the same training environment and parameter settings, this paper compares the performance of the original SAC algorithm and the improved SAC algorithm with the self-attention mechanism in the process of double-obstacle trajectory optimization. Figure 16 shows the trend of evaluation mean reward (EMR) with the number of steps during training for both algorithms.  
Figure 16 shows the comparison of reward convergence curves. The horizontal axis is labeled \"Training Steps\" (range: 0-30,000 steps, linear scaling), and the vertical axis is labeled \"Average Evaluation Reward\" (range: \(-2.0\) to \(-1.5\), linear scaling). Units: Steps (no unit) for the horizontal axis; the reward value is a dimensionless metric. Scaling is clarified as linear, with equal intervals between tick marks (5000 steps per interval on the horizontal axis; 0.1 per interval on the vertical axis) to ensure clear visualization of convergence trends.  
The experimental results show that the strategy after introducing the self-attention mechanism shows faster convergence ability in the early stage of training. In the first 20,000 training steps, the average reward of the improved model increases rapidly and tends to be stable, while the performance of the baseline SAC algorithm lags significantly at the same stage. In addition, the average reward value in the final stable stage is also better. The SAC + Attention strategy is maintained at a high level of about \(-1.76\) for a long time, while the baseline SAC is stable at about \(-1.77\), indicating that after introducing the attention mechanism, the strategy can more effectively avoid obstacles, optimize paths, and improve the quality of task completion.  
After introducing the attention mechanism, the model showed smaller reward value fluctuations during the training process, and the overall stability was significantly improved. This phenomenon shows that the model has stronger anti-interference ability and the ability to adapt to new scenarios when dealing with complex obstacle environments. Therefore, it can continuously generate high-quality action trajectories, effectively avoiding violent shocks or instability that may occur during the exploration of the strategy. Moreover,  
Figure 16: Comparison of reward convergence before and after adding the self-attention mechanism.  
the core advantage of the self-attention mechanism is that it can capture the correlation between the elements within the state information. This ability enables the strategy model to dynamically and sensitively perceive changes in key spatial features in the environment, such as the precise position of obstacles, the adjustment of wellbore direction, and the movement of target points. This enhanced perception directly improves the performance of the trajectory control strategy: when faced with multi-stage and multi-source spatial interference, the strategy can respond faster and more accurately, showing higher environmental adaptability and operational sensitivity. On the one hand, the introduction of the self-attention mechanism optimizes the learning efficiency of the strategy and improves the overall effect of path planning. On the other hand, it greatly enhances the operational stability and decision-making intelligence level of the model in complex and changeable environments. These provide strong support for the practical application of the reinforcement learning algorithm in the intelligent optimization task of wellbore trajectory.