###### Abstract  
Fiber-optic gyroscopes (FOGs)-based Measurement While Drilling system (MWD) is a newly developed instrument to survey the borehole trajectory continuously and in real time. However, because of the strong vibration while drilling, the measurement accuracy of FOG-based MWD deteriorates. It is urgent to improve the measurement accuracy while drilling. Therefore, this paper proposes an innovative scheme for the vibration error of the FOG-based MWD. Firstly, the nonlinear error models for the FOGs and ACCs are established. Secondly, a 36-order Extended Kalman Filter (EKF) combined with a calibration method based on 24-position is designed to identify the coefficients in the error model. Moreover, in order to obtain a higher accurate error model, an iterative calibration method has been suggested to suppress calibration residuals. Finally, vibration experiments simulating the drilling vibration in the laboratory is implemented. Compared to the original data, compensated the linear error items, the error of 3D borehole trajectory can only be reduced by a ratio from 10% to 34%. While compensating for the nonlinear error items of the FOG-based MWD, the error of 3D borehole trajectory can be reduced by a ratio from 44.13% to 97.22%. In conclusion, compensation of the nonlinear error of FOG-based MWD could improve the trajectory survey accuracy under vibration. nonlinear error; vibration; FOG-based MWD; borehole trajectory survey; EKF 2021 8 8 July 2021  
a big threat to the MWD [9]. Drilling vibration includes axial vibration, lateral vibrations and torsional vibration [10]. It is characterized by large vibration magnitude, random frequency and a long period of time. [PERSON] et al. [11, 12] and [PERSON] et al. [13] analyzed the influence of downhole vibration on FOG-based MWD measurement accuracy. They gave us the conclusion that the drilling vibrations worsen the measurement accuracy because the error characteristic of the sensors were changed in the vibration. However, it did not propose a solution to solve it. As we all know, the original data of the FOGs and ACCs have different types of errors, such as bias error, scale factor error and random noise [14]. So, before using them, the error should be compensated by the established error model [15]. In a general way, only the linear error item has been compensated and the nonlinear error items have been ignored [16]. However, while in the drilling environment, downhole vibrations, including non-sinusoidal motions and random high-magnitude shocks, are high dynamic motions. If the nonlinear error is still ignored, it leads to an inaccurate survey result. Aiming to improve the measurement accuracy of the FOG-based MWD under vibration, the nonlinear error must be recognized and compensated correctly. At present, the centrifuge has been used to calibrate the nonlinear error coefficients of the ACCs [17, 18, 19]. However, the centrifuge itself has a lot of errors, for example, mechanical structure error, the installation error and the axial rotation dynamic error, which will make the calibration results inaccurate. In addition, using the centrifuge can only recognize the nonlinear error coefficient, while the linear error coefficient needs to calibrate using other equipment. Studies about the nonlinear error model in terms of the FOG seem to be rare. [PERSON] and [PERSON] [20] studied a method to recognize the gyro error coefficient related to the acceleration, but it could not recognize the nonlinear error coefficient related to itself. In this paper, the nonlinear error item of the FOG-based MWD is recognized and compensated by a turntable instead of a centrifuge. First, taking the nonlinear error item into consideration, the nonlinear error model for the FOG-based MWD is established. Second, a 36-order Extended Kalman Filter (EKF) is designed based on the nonlinear error model [21]. Next, a calibration method based on 24 positions is put forward. With these steps, both the nonlinear and the linear error coefficients could be recognized.