# Rate of Penetration Prediction Method for Ultra-Deep Wells Based on LSTM-FNN
## 1 Introduction  
Deep and ultra-deep well drilling has become the key focus of the future of oil and gas exploitation [1]. With the increase of well depth, the geological conditions become more complex, well formation becomes more difficult and drillability worsens, resulting in a low drilling efficiency, high safety risk, long drilling cycles and other problems [2]. However, the prediction of the rate of penetration (ROP) can lower drilling risks and optimize drilling parameters in advance, which is of great significance to the improvement of the efficiency of deep drilling [3]. With the progress of information technology, intelligent drilling and completion technology has begun to enter the field. Information technology, microelectronics technology, robotics technology, communication network technology and drilling and completion engineering have been closely combined and integrated, which has greatly promoted the innovation of oil drilling and production technology [4]. The intelligent prediction of the ROP is one of the most important aspects to intelligent drilling and completion. Although the ROP prediction is very important to improve drilling efficiency, it is difficult to match these factors with the ROP and establish an accurate ROP prediction model [5]. In recent years, with the deepening of the oilfield digitization process, the massive drilling data and efficient computing power of computers provide good conditions for data-driven ROP prediction. There are several Machine Learning (ML) methods [2, 6], which have been applied in ROP prediction, such as Random Forest (RF) [7], Gradient Boosting Machines (GBM), K-nearest Neighbors (KNN) [3] and Support Vector Machine (SVM) [8, 9]. However, when these machine learning models are trained using a largeamount of drilling data, they facing the challenges of long training times and overfitting. When the volume of drilling data is very large, deep learning possesses a clear advantage, as computational efficiency and single-well prediction accuracy are significantly improved. A large number of researchers have begun to use fully connected neural network (FNN) to establish the ROP prediction [4; 5; 10; 11; 12], and compare it with machine learning models and ROP equations [13]. The results show that the established model has high precision and a fast training speed, which is better than the conventional machine learning and ROP equation model [14; 15; 16]. [PERSON] [4] et al. optimized the key parameters of the ROP and selected the density of rock, porosity and point load index as input parameters, and ROP as an output parameter, and a drilling data set with a total sample of 71 was established. Using these drilling data, an ROP prediction model is established based on the FNN algorithm, and the relative error on the test set is 14%. Furthermore, considering the timing characteristics between the drilling parameters and ROP, some scholars established an ROP prediction model by using recurrent neural network (RNN) [17; 18; 19; 20], long short-term memory neural network (LSTM) and gated recurrent neural network (GRU). The hyperparameters of the neural network algorithm, such as network structure, activation function and weight parameters, have significant influence on the prediction effect of the model [21; 22]. Some scholars have attempted to use a genetic algorithm (GA) [23], particle swarm optimization (PSO) and other algorithms to optimize the model's hyperparameters [24; 25] and to obtain the optimal hyperparameters and achieve the best prediction effect. Some scholars have also improved the effect of the model by combining the structures of various neural networks, as seen with [PERSON] [26]. However, for the problem of deep well drilling, the existing intelligent prediction models for the ROP tend to ignore the characteristics of deep well drilling, such as high temperature and high pressure. In addition, multiple drilling parameters do have time series correlation with ROP.