# Borehole Trajectory Optimization Design Based on the SAC Algorithm with a Self-Attention Mechanism
## 4 Experimental Design and Result Analysis  
A graphical flowchart of the experimental design is added as Figure X to clarify the logic of test scenarios. The flowchart illustrates the overall process: (1) Generating 3D geological models with obstacles using WGAN technology; (2) Constructing the reinforcement learning environment by defining state space, action space, and reward function; (3) Training the SAC algorithm integrated with self-attention mechanism using the generated geological data; (4) Testing the trained model in typical obstacle scenarios (initial obstacle, mid-borehole obstacle, terminal obstacle, double obstacles, and lateral obstacle);and (5) Evaluating model performance through indicators such as convergence speed, trajectory smoothness, and obstacle avoidance success rate. In order to verify the performance of the SAC trajectory optimization model incorporating the self-attention mechanism proposed in this paper under complex obstacle environments, a comparative experimental platform was constructed, and the relevant hyperparameters were set in detail. In the following section, the hardware and software environment configurations used for the experiments (Table 1) and the main hyperparameter settings of the SAC reinforcement learning algorithm (Table 2) are introduced to ensure the reproducibility of the experiments and the scientificity of the methodology, respectively. A comparative validation approach was adopted to evaluate the superiority of the proposed SAC algorithm with a self-attention mechanism. The baseline model for comparison was the original SAC algorithm without the self-attention mechanism. Both models were trained and tested under identical experimental conditions: the same 3D geological environments generated by WGAN, consistent state/action spaces, reward function settings, and hyperparameters (as listed in Table 2). The comparison focused on key performance metrics in typical obstacle scenarios (especially the double-obstacle environment), including convergence speed (number of steps to reach stable performance), average reward value, trajectory smoothness (dogleg severity), and obstacle avoidance success rate. This design ensures that the observed performance differences can be attributed to the introduction of the self-attention mechanism. \begin{table}
\begin{tabular}{c c} \hline \hline
**Item** & **Unit Name** \\ \hline CPU & Core i7-12700K \\ GPU & RTXTM 4070 \\ memory & 32 g \\ GPU memory & 8 G \\ operating system & Windows 11 \\ Python version & Python 3.10 \\ Pytorch version & torch=1.2.0 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Experimental conditions. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Hyperparameter Name** & **Meaning** & **Value** \\ \hline \multirow{2}{*}{Learning Rate} & Step size used by the optimizer to update policy and & \multirow{2}{*}{0.00003} \\  & Q-networks & \\ \hline \multirow{2}{*}{Entropy Coefficient, \(\alpha\)} & Weight of entropy in the objective function, & \multirow{2}{*}{Auto-adjusted} \\  & controlling exploration & \\ \hline Discount Factor, \(\gamma\) & Determines the decay rate of future rewards & 0.98 \\ \hline Batch Size & Number of samples used for training in each iteration & 64 \\ \hline Replay Buffer Size & Maximum capacity of the experience replay buffer & 1 \(\times\) 10\({}^{6}\) \\ \hline Network Size & Hidden layer sizes for policy and Q-value networks & [28] \\ \hline Soft Update Tau & The factor used for softly updating the target networks & 0.005 \\ \hline \multirow{2}{*}{Temperature for Entropy} & If using auto entropy adjustment, this parameter determines & \multirow{2}{*}{Auto-adjusted} \\  & the initial temperature & \\ \hline Number of Updates per Step & Number of network updates performed per environment step & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Hyperparameter indexes and values (for the SAC algorithm). The experiment was conducted with 1000 training episodes to ensure the model sufficiently converges.