Unsupervised Learning of Full-Waveform Inversion: Connecting CNN and Partial Differential Equation in a Loop

1 INTRODUCTION . Geophysical properties ( such as velocity , impedance , and density ) play an important role in various subsurface applications including subsurface energy exploration , carbon capture and sequestration , estimating pathways of subsurface contaminant transport , and earthquake early warning systems to provide critical alerts . These properties can be obtained via seismic surveys , i.e. , receiving reflected/refracted seismic waves generated by a controlled source . This paper focuses on reconstructing subsurface velocity maps from seismic measurements . Mathematically , the velocity map and seismic measurements are correlated through an acoustic-wave equation ( a second-order partial differential equation ) as follows : ∇2p ( r , t ) − 1 v ( r ) 2 ∂2p ( r , t ) ∂t2 = s ( r , t ) , ( 1 ) where p ( r , t ) denotes the pressure wavefield at spatial location r and time t , v ( r ) represents the velocity map of wave propagation , and s ( r , t ) is the source term . Full Waveform Inversion ( FWI ) is a methodology that determines high-resolution velocity maps v ( r ) of the subsurface via matching synthetic seismic waveforms to raw recorded seismic data p ( r̃ , t ) , where r̃ represents the locations of the seismic receivers . A velocity map describes the wave propagation speed in the subsurface region of interest . An example in 2D scenario is shown in Figure 2a . Particularly , the x-axis represents the horizontal offset of a region , and the y-axis stands for the depth . The regions with the same geologic information ( velocity ) are called a layer in velocity maps . In a sample of seismic measurements ( termed a shot gather in geophysics ) as depicted in Figure 2b , each grid in the x-axis represents a receiver , and the value in the y-axis is a 1D time-series signal recorded by each receiver . Existing approaches solve FWI in two directions : physics-driven and data-driven . Physics-driven approaches rely on the forward modeling of Equation 1 , which simulates seismic data from velocity map by finite difference . They optimize velocity map per seismic sample , by iteratively updating velocity map from an initial guess such that simulated seismic data ( after forward modeling ) is close to the input seismic measurements . However , these methods are slow and difficult to scale up as the iterative optimization is required per input sample . Data-driven approaches consider FWI problem as an image-to-image translation task and apply convolution neural networks ( CNN ) to learn the mapping from seismic data to velocity maps ( Wu & Lin , 2019 ) . The limitation of these methods is that they require paired seismic data and velocity maps to train the network . Such ground truth velocity maps are hardly accessible in real-world scenario because generating them is extremely time-consuming even for domain experts . In this work , we leverage advantages of both directions ( physics + data driven ) and shift the paradigm to unsupervised learning of FWI by connecting forward modeling and CNN in a loop . Specifically , as shown in Figure 1 , a CNN is trained to predict a velocity map from seismic data , which is followed by forward modeling to reconstruct seismic data . The loop is closed by applying reconstruction loss on seismic data to train the CNN . Due to the differentiable forward modeling , the whole loop can be trained end-to-end . Note that the CNN is trained in an unsupervised manner , as the ground truth of velocity map is not needed . We name our unsupervised approach as UPFWI ( Unsupervised Physical-informed Full Waveform Inversion ) . Additionally , we find that perceptual loss ( Johnson et al. , 2016 ) is crucial to improve the overall quality of predicted velocity maps due to its superior capability in preserving the coherence of the reconstructed waveforms comparing with other losses like Mean Squared Error ( MSE ) and Mean Absolute Error ( MAE ) . To encourage fair comparison on a large dataset with more complicate geological structures , we introduce a new dataset named OpenFWI , which contains 60,000 labeled data ( velocity map and seismic data pairs ) and 48,000 unlabeled data ( seismic data alone ) . 30,000 of those velocity maps contain curved layers that are more challenge for inversion . We also add geological faults with various shift distances and tilting angles to all velocity maps . We evaluate our method on this large dataset . Experimental results show that for velocity maps with flat layers , our UPFWI trained with 48,000 unlabeled data achieves 1146.09 in MSE , which is 26.77 % smaller than that of the supervised method , and 0.9895 in Structured Similarity ( SSIM ) , which is 0.0021 higher than the score of the supervised method ; for velocity maps with curved layers , our UPFWI achieves 3639.96 in MSE , which is 28.30 % smaller than that of supervised method , and 0.9756 in SSIM , which is 0.0057 higher than the score of the supervised method . Our contribution is summarized as follows : • We propose to solve FWI in an unsupervised manner by connecting CNN and forward modeling in a loop , enabling end-to-end learning from seismic data alone . • We find that perceptual loss is helpful to boost the performance comparable to the supervised counterpart . • We introduce a large-scale dataset as benchmark to encourage further research on FWI . 2 PRELIMINARIES OF FULL WAVEFORM INVERSION ( FWI ) . The goal of FWI in geophysics is to invert for a velocity map v ∈ RW×H from seismic measurements p ∈ RS×T×R , where W and H denote the horizontal and vertical dimensions of the velocity map , S is the number of sources that are used to generate waves during data acquisition process , T denotes the number of samples in the wavefields recorded by each receiver , and R represents the total number of receivers . In conventional physics-driven methods , forward modeling is commonly referred to the process of simulating seismic data p̃ from given estimated velocity maps v̂ . For simplicity , the forward acoustic-wave operator f can be expressed as p̃ = f ( v̂ ) . ( 2 ) Given this forward operator f , the physics-driven FWI can be posed as a minimization problem ( Virieux & Operto , 2009 ) E ( v̂ ) = min v̂ { ||p− f ( v̂ ) ||22 + λR ( v̂ ) } , ( 3 ) where ||p − f ( v̂ ) ||22 is the the ` 2 distance between true seismic measurements p and the corresponding simulated data f ( v̂ ) , λ is a regularization parameter and R ( v̂ ) is the regularization term which is often the ` 2 or ` 1 norm of v̂ . This requires optimization per sample , which is slow as the optimization involves multiple iterations from an initial guess . Data-driven methods leverage convolutional neural networks to directly learn the inverse mapping as ( Adler et al. , 2021 ) v̂ = gθ ( p ) ≈ f−1 ( p ) , ( 4 ) where gθ ( · ) is the approximated inverse operator of f ( · ) parameterized by θ . In practice , gθ is usually implemented as a convolutional neural network ( Adler et al. , 2021 ; Wu & Lin , 2019 ) . This requires paired seismic data and velocity maps for supervised learning . However , the acquisition of large volume of velocity maps in field applications can be extremely challenging and computationally prohibitive . 3 METHOD . In this section , we present our Unsupervised Physics-informed solution ( named UPFWI ) , which connects CNN and forward modeling in a loop . It addresses limitations of both physics-driven and data-driven approaches , as it requires neither optimization at inference ( per sample ) , nor velocity maps as supervision . 3.1 UPFWI : CONNECTING CNN AND FORWARD MODELING . As depicted in Figure 1 , our UPFWI connects a CNN gθ and a differentiable forward operator f to form a loop . In particular , the CNN takes seismic measurements p as input and generates the corresponding velocity map v̂ . We then apply forward acoustic-wave operator f ( see Equation 2 ) on the estimated velocity map v̂ to reconstruct the seismic data p̃ . Typically , the forward modeling employs finite difference ( FD ) to discretize the wave equation ( Equation 1 ) . The details of forward modeling will be discussed in the subsection 3.3 . The loop is closed by the reconstruction loss between input seismic data p and the reconstructed seismic data p̃ = f ( gθ ( p ) ) . Notice that the ground truth of velocity maps v is not involved , and the training process is unsupervised . Since the forward operator is differentiable , the reconstruction loss can be backpropagated ( via gradient descent ) to update the parameters θ in the CNN . 3.2 CNN NETWORK ARCHITECTURE . We use an encoder-decoder structured CNN ( similar to Wu & Lin ( 2019 ) and Zhang & Lin ( 2020 ) ) to model the mapping from seismic data p ∈ RS×T×R to velocity map v ∈ RW×H . The encoder compresses the seismic input and then transforms the latent vector to build the velocity estimation through a decoder . Since the number of receivers R and the number of timesteps T in seismic measurements are unbalanced ( T R ) , we first stack a 7×1 and six 3×1 convolutional layers ( with stride 2 every the other layer to reduce dimension ) to extract temporal features until the temporal dimension is close to R. Then , six 3×3 convolutional layers are followed to extract spatial-temporal features . The resolution is down-sampled every the other layer by using stride 2 . Next , the feature map is flattened and a fully connected layer is applied to generate the latent feature with dimension 512 . The decoder first repeats the latent vector by 25 times to generate a 5×5×512 tensor . Then it is followed by five 3×3 convolutional layers with nearest neighbor upsampling in between , resulting in a feature map with size 80×80×32 . Finally , we center-crop the feature map ( 70×70 ) and apply a 3×3 convolution layer to output a single channel velocity map . All the aforementioned convolutional and upsampling layers are followed by a batch normalization ( Ioffe & Szegedy , 2015 ) and a leaky ReLU ( Nair & Hinton , 2010 ) as activation function . 3.3 DIFFERENTIABLE FORWARD MODELING . We apply the standard finite difference ( FD ) in the space domain and time domain to discretize the original wave equation . Specifically , the second-order central finite difference in time domain ( ∂ 2p ( r , t ) ∂t2 in Equation 1 ) is approximated as follows : ∂2p ( r , t ) ∂t2 ≈ 1 ( ∆t ) 2 ( pt+1r − 2ptr + pt−1r ) +O [ ( ∆t ) 2 ] , ( 5 ) where ptr denotes the pressure wavefields at timestep t , and p t+1 r and p t−1 r are the wavefields at t+ ∆t and t−∆t , respectively . The Laplacian of p ( r , t ) can be estimated in the similar way on the space domain ( see Appendix ) . Therefore , the wave equation can then be written as pt+1r = ( 2− v2∇2 ) ptr − pt−1r + v2 ( ∆t ) 2str , ( 6 ) where∇2 here denotes the discrete Laplace operator . The initial wavefield at the timestep 0 is set zero ( i.e . p0r = 0 ) . Thus , the gradient of loss L with respect to estimated velocity at spatial location r can be computed using the chain rule as ∂L ∂v ( r ) = T∑ t=0 [ ∂L ∂p ( r , t ) ] ∂p ( r , t ) ∂v ( r ) , ( 7 ) where T indicates the length of the sequence .

Summary: The paper introduces a novel way to solve Full-Waveform Inversion problem which is a common problem in geological surveys. Their method is based on CNN, giving a reconstruction of velocity field for measured seismic data, and the loss function which connects CNN to a discretized version of the governing partial differential equations (the wave equation). The power of the method lies in unsupervised learning which allows one to use more data without expensive data labelling. This is demonstrated using numerical results using their simulated seismic dataset.