# Storage and Processing of Big Data for Geomagnetic Support of Directional Drilling
## 6 Conclusions  
This paper presents the first step in transferring the geomagnetic measurement database of the GC RAS to the Big Data technological base. The Arenadata database based on the Greenplum open source DBMS was selected as the storage technology and deployed on a physical cluster. This is a massively parallel DBMS that is widely used in commercial data processing systems and that has not been used in magnetometric information research before.  
An important quality of the Arenadata DB, in contrast to classic DBs, is its support for fast horizontal scaling by adding new segments to the computer cluster. The results of the tests the new MAGNUS BD system show that it significantly outperforms the MAGNUS system, which has existed since 2016, in terms of the speed of executing standard queries, and provides additional tools for working with data.  
The capabilities of the new system make it possible to simplify and speed up the process of geomagnetic field modeling using satellite and ground-based data by parallelizing computations.  
The use of the Big Data approach will allow the accumulation and prompt transmission of processed and consolidated information from various observatories for the prompt management of the directional drilling process. This will allow a multi-faceted improvement in the accuracy of positioning when developing hard-to-recover oil and gas reserves, including those located in the Arctic region.  
**Author Contributions:** Conceptualization, A.D.G.; Data curation, I.O.B.; Funding acquisition, B.A.D.; Methodology, D.V.K. and A.D.G.; Project administration, A.D.G., I.M.N. and B.A.D.; Resources, A.A.G. and B.A.D.; Software, D.V.K., I.O.B. and A.A.G.; Supervision, A.D.G.; Visualization, I.M.N., I.O.B. and R.I.K.; Writing--original draft, D.V.K., A.D.G., I.M.N., I.O.B., B.A.D., A.A.G., B.V.D. and R.I.K.; Writing--review & editing, D.V.K., A.D.G., I.M.N., I.O.B., B.A.D., A.A.G., B.V.D. and R.I.K. All authors have read and agreed to the published version of the manuscript.  
This work was carried out in the framework of budgetary funding of the Geophysical Center of RAS (No. 075-00443-24-01) and the Schmidt Institute of Physics of the Earth of RAS (No. 075-00273-24-00), adopted by the Ministry of Science and Higher Education of the Russian Federation.  
Not applicable.  
The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.  
This work employed data and services provided by the Shared Research Facility \"Analytical Geomagnetic Data Center\" of the Geophysical Center of RAS ([[http://ckp.gcras.ru/](http://ckp.gcras.ru/)]([http://ckp.gcras.ru/](http://ckp.gcras.ru/)) (accessed on 3 September 2024)).  
The authors declare no conflicts of interest.  
The following abbreviations are used in this manuscript:  
\begin{tabular}{l l} ADCM & Arenadata Cluster Manager \\ API & Application Programming Interface \\ BD & Big Data \\ DB & Database \\ DBMS & Database Management System \\ DD & Directional Drilling \\ DED & Deadhorse Observatory \\ EMF & Earth's Magnetic Field \\ GB & Gigabyte \\ GNSS & Global Navigation Satellite Systems \\ ISCWSA & The Industry Steering Committee on Wellbore Survey Accuracy \\ MPP & Massively Parallel Processing \\ TB & Terabyte \\ UAV & Unmanned Aerial Vehicles \\ USGS & US Geological Survey \\ \end{tabular}