# Research on 3D Geological Modeling Method Based on Deep Neural Networks for Drilling Data
## 4 Data Scenario
### Normalization  
Normalization is a process that transforms data into a common scale or range to make them more suitable for analysis. Typically, this involves scaling data to have a mean of 0 and a standard deviation of 1, or rescaling them to a specific range, such as between 0 and 1. In this paper, we utilize the batch normalization method [46] for preprocessing drilling data, a choice tailored to accommodate the distinctive characteristics of geological drilling data.  
Within the framework of a deep neural network, where we denote the net input to the lth layer as \(\mathbf{z}(\mathbf{l})\) and represent the output of the neuron in that layer as \(\mathbf{a}(\mathbf{l})\), the relationship between input and output is expressed in Equation (15).  
\[\mathbf{a}(\mathbf{l})=\mathrm{f}(\mathbf{z}(\mathbf{l}))=\mathrm{f}(\mathbf{ Wa}(\mathbf{l}-1)+\mathbf{b}) \tag{15}\]  
In the equation, the function f(*) represents the activation function, while \(\mathbf{W}\) and \(\mathbf{b}\) denote the weights and biases.  
To enhance optimization efficiency, each dimension of \(\mathbf{z}(\mathbf{l})\) is standardized to follow a standard normal distribution using a normalization method, as depicted in Equation (16).  
\[\hat{\mathbf{z}}(\mathbf{l})=\frac{\mathbf{z}(\mathbf{l})-\mathrm{E}[\mathbf{ z}(\mathbf{l})]}{\sqrt{\mathrm{var}(\mathbf{z}(\mathbf{l}))+\epsilon}} \tag{16}\]  
In the equation, \(\mathrm{E}[\mathbf{z}(\mathbf{l})]\) and \(\mathrm{var}(\mathbf{z}(\mathbf{l}))\) represent the expectation and variance of each dimension of \(\mathbf{z}(\mathbf{l})\) across the entire training dataset with the current parameters, while \(\epsilon\) is a small constant introduced to avoid a zero variance.  
In the case of a small-batch sample set consisting of K samples, Equations (17) and (18) can be used to calculate the mean and variance of the net inputs \(\mathbf{z}(\mathbf{l},\mathbf{l})\), \(\ldots\), \(\mathbf{z}(\mathbf{K},\mathbf{l})\) for the neurons in layer \"\(\mathbf{l}\)\".  
Figure 2: Illustration of the resampling process. (**a**) Location of study area. (**b**) Resampling in the planar space. (**c**) Resampling in the depth direction.  
\[\mu_{B}=\frac{1}{K}\sum_{k=1}^{K}\mathbf{z}^{(k,l)} \tag{17}\] \[\sigma_{B}^{2}=\frac{1}{K}\sum_{k=1}^{K}\left(\mathbf{z}^{(k,l)}- \mu_{B}\right)\odot(\mathbf{z}^{(k,l)}-\mu_{B}) \tag{18}\]  
The values of the net inputs are centered around 0 through standard normalization. This interval closely aligns with the linear transformation range, reducing the nonlinearity of the neural network. To ensure that normalization does not compromise the network's expressiveness, an additional scaling and translation transformation can usually be applied to adjust the interval, as shown in Equation (19).  
\[\hat{\mathbf{z}}^{(1)}=\frac{\mathbf{z}^{(l)}-\mu_{B}}{\sqrt{\sigma_{B}^{2}+ \varepsilon}}\odot\gamma+\beta \tag{19}\]  
In the equation, the parameter vectors for scaling and translation, represented as \(\gamma\) and \(\beta\), respectively, are included. The batch normalization operation can be viewed as a specialized neural network layer added before each layer with a nonlinear activation function, as depicted in Equation (20).  
\[\mathbf{a}^{(l)}=\mathbf{f}\left(\mathbf{BN}_{\gamma,\beta}\left(\mathbf{z}^ {(l)}\right)\right)=\mathbf{f}\left(\mathbf{N}_{\gamma,\beta}\left(\mathbf{ Wa}^{(l-1)}\right)\right) \tag{20}\]  
Batch normalization not only improves optimization efficiency but also acts as an implicit regularization technique. During training, the prediction for an individual sample by the neural network is impacted not only by that specific sample but also by other samples within the same batch. As batch selection is randomized, it prevents the neural network from overfitting to a particular sample, thus enhancing the network's generalization capability.