# Storage and Processing of Big Data for Geomagnetic Support of Directional Drilling
## 2 Geomagnetic Data and Data Management  
From the middle of the 19 th century up to the present moment, geomagnetic observatories are the main source of long-term observations of EMF parameters. A magnetic observatory is a sophisticated scientific measuring complex where high-precision magnetometers are installed in special pavilions constructed of non-magnetic materials. The purpose of a magnetic observatory is to measure the magnetic field vector with high accuracy in real time. Currently, there are more than 120 high-quality magnetic observatories operating around the world, united in the international INTERMAGNET network (www.intermagnet.org (accessed on 3 September 2024)). Magnetic observatories are equipped with a set of standard instrumentation, which necessarily includes vector and scalar magnetometers, as well as a non-magnetic theodolite with a built-in single-component magnetometer. This set of instruments allows for the measurement of all the components of the full geomagnetic vector [25]. In 2009, the Deadhorse Observatory (DED) was deployed for the first time in Alaska for joint use by the US Geological Survey (USGS) for scientific purposes and the oilfield service company Schlumberger for supporting DD [13]. The case of this observatory demonstrates that, when using geomagnetic navigation for DD, several tasks are distinguished:  
* Modeling the main EMF with high resolution [26];
* Identifying and modeling the lithospheric field on a regional scale [14];
* Monitoring and accounting for EMF variations associated with space weather [11; 13]. Data from magnetic observatories are used to varying degrees in solving each of the three problems listed above. While modern EMF modeling largely uses data from low-orbit satellites [27], the problem of recording EMF variations near drilling sites is solved by constructing a magnetic observatory [13]. In this regard, the interest of oilfield service companies in observatory magnetic observations is growing. At the same time, in some fields, it is possible to use data from existing observatories if they are located close enough to drilling sites. Regular disturbances of the magnetic field are observed at high latitudes that are close to the auroral zone [16; 28]. Thus, the efficiency of mineral extraction in the Arctic zone of the Russian Federation can be increased by using the technology of the geomagnetic support of directional drilling [17; 29], similar to how it was used in Alaska [13]. Turning to the technology, it is important to note that the data obtained at the observatory must be processed with minimal delay [30] and transmitted to the control center of the drilling station. The center compensates for direction errors or suspends drilling in the event of extreme disturbances. Here, processing means a whole complex of algorithms that eliminate anthropogenic and instrumental interference, calculate the temperature coefficients of magnetometers, etc. [30]. This leads to the need to create systems for processing geomagnetic data in real time. Today, in the Russian Federation, there is the only full-fledged system for collecting, storing, and promptly processing geomagnetic data, MAGNUS (Monitoring and Analysis of Geomagnetic aNomalies in Unified System) [[[http://mag.gcras.ru/](http://mag.gcras.ru/)]([http://mag.gcras.ru/](http://mag.gcras.ru/))] (accessed on 3 September 2024), created and operating at the Geophysical Center of the Russian Academy of Sciences (GC RAS). The basic functional diagram of MAGNUS is shown in Figure 1.  
Figure 1: Structural diagram of the MAGNUS system. MAGNUS receives data from 22 observatories located in Russia and neighboring countries, a significant part of which corresponds to the international INTERMAGNET standard. A map of existing and prospective observatories is shown in Figure 2. It is planned to expand the number of observatories supplying the MAGNUS system with data. The MAGNUS 1.0 analytical software package was created in 2016 based on the MySQL database.