# Research on Wellbore Trajectory Optimization and Drilling Control Based on the TD3 Algorithm
###### Abstract
[PERSON] [8] established a CORA multi-constraint model to quantify and integrate the influencing factors of non-vertical well trajectory for the first time. Taking Shengli Oilfield as an example, [PERSON] [9] proposed a design scheme for reducing the dogleg degree and anti-collision well to address the problem of easy collapse in Yan 227 block. [PERSON] [10] and [PERSON] et al. [11] studied the reef-shoal gas reservoir in Yuanba gas field, Sichuan, and established the dynamic optimization technology of horizontal well trajectory. [PERSON] et al. [12] innovated the well trajectory optimization model for formation uncertainty and experimentally verified its annual net present value coefficient as >0. [PERSON] et al. [13] proposed a double two-dimensional trajectory design method and confirmed that it can reduce the difficulty of shale gas well construction. [PERSON] [14] introduced formation parameters, such as the vertical depth of the build-up point, into ae model to form a platform well deployment and trajectory optimization scheme. [PERSON] et al. [15] carried out technical research on the directional well trajectory and friction control of extended reach wells for shallow problems in Bohai Oilfield. The formation constraint trajectory model and wellbore stability optimization model were constructed by [PERSON] [16]. [PERSON] et al. [17] took the southeastern part of the Sulige gas field as the research area. In view of the complexity of the spatial distribution of fluvial sand bodies, three-dimensional geological modeling technology was used to improve the prediction accuracy of inter-well sand bodies. [PERSON] et al. [18] constructed a geomechanical model of the Changning shale gas reservoir in Sichuan Province. Based on the logging data, the influence of rock mechanical properties on the distribution of in situ stress was analyzed. An improved plate-weak surface failure criterion was proposed for wellbore stability analysis, which provided a basis for drilling fluid density optimization. Aiming at the deterioration of rock mechanical properties in the upper part of the Dibera fault block in Niger Oilfield, [PERSON] [19] constructed a horizontal well trajectory optimization control system based on numerical simulation and mathematical statistics. The proposed scheme has the advantage of high precision, which provides a reference paradigm for trajectory planning with formation characteristics constraints. The drilling practice shows that because of the dynamic change in formation conditions, the wellbore trajectory needs to be adjusted iteratively during the construction process. Different from the planning method based on model planning, reinforcement learning does not require a complete geological model. It uses the process of interacting with the environment to learn how to maintain an efficient drilling process under non-standard formation conditions. It can adjust to emergency situations, so as to more effectively plan the wellbore trajectory, avoid risks, and increase the economic value of oil and gas field development and construction. Reinforcement learning-based wellbore trajectory planning represents an important step towards intelligent and adaptive wellbore trajectory planning. Because of the high complexity of the current geological environment, the traditional reinforcement learning method has difficulty solving the problem of wellbore trajectory planning. Therefore, most of the current research focuses on the combination of deep reinforcement learning (DRL) and wellbore trajectory planning. [PERSON] [20] (2022) and [PERSON] [21] (2022) proposed an intelligent guidance method based on the DQN algorithm to solve the problem of the low intelligence level in wellbore trajectory planning at this stage. However, [PERSON] improved the DQN algorithm based on [PERSON]'s research and proposed an NDQN algorithm with an attention mechanism to realize online drilling decision planning.