A Unified Framework for Forward and Inverse Problems in Subsurface Imaging using Latent Space Translations

Authors: Naveen Gupta, Medha Sawhney, Arka Daw, Youzuo Lin, Anuj Karpatne
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: Machine Learning, Inverse Problems, Full-Waveform Inversion, Seismic Imaging, ML4Science, OpenFWI

Abstract:
In subsurface imaging, learning the mapping from velocity maps to seismic waveforms (forward problem) and waveforms to velocity (inverse problem) is important for several applications. While traditional techniques for solving forward and inverse problems are computationally prohibitive, there is a growing interest to leverage recent advances in deep learning to learn the mapping between velocity maps and seismic waveform images directly from data. Despite the variety of architectures explored in previous works, several open questions still remain unanswered such as the effect of latent space sizes, the importance of manifold learning, the complexity of translation models, and the value of jointly solving forward and inverse problems. We propose a unified framework to systematically characterize prior research in this area termed the Generalized Forward-Inverse (GFI) framework, building on the assumption of manifolds and latent space translations. We show that GFI encompasses previous works in deep learning for subsurface imaging, which can be viewed as specific instantiations of GFI. We also propose two new model architectures within the framework of GFI: Latent U-Net and Invertible X-Net, leveraging the power of U-Nets for domain translation and the ability of IU-Nets to simultaneously learn forward and inverse translations, respectively. We show that our proposed models achieve state-of-the-art (SOTA) performance for forward and inverse problems on a wide range of synthetic datasets, and also investigate their zero-shot effectiveness on two real-world-like datasets. The code is available at https://github.com/KGML-lab/Generalized-Forward-Inverse-Framework-for-DL4SI

Decision: accept (poster)
Reviewer scores: 8, 5, 6, 6

Reviews and discussion:
<thread>
creation date:
1730372912676

thread_type:
official_review

<thread_object>
creator: reviewer_nmvk

<content>
summary:
This work summarizes recent advances in deep learning-based mapping between velocity maps and seismic waveforms, introducing a unified framework (GFI) to systematically characterize prior research. Using GFI, key factors affecting mapping performance are investigated, alongside the proposal of two novel model architectures. Comprehensive experiments reveal the impact of these factors, with the proposed models achieving state-of-the-art (SOTA) performance in both forward and inverse problems.

strengths:
1. This work is the first to investigate key factors influencing mapping performance.
2. Extensive experiments were conducted to conclude the influence of these factors.
3. Two novel networks are introduced for mapping between velocity maps and seismic waveforms, achieving SOTA performance.

weaknesses:
1. This paper resembles a survey and might be more suitable for journal submission.
2. A significant portion of the content is placed in the appendices, making it less reader-friendly.
3. Aside from extensive experiments on factor influences, the work offers limited novelty and lacks theoretical analysis.
4. Additional factors should be investigated, such as the robustness of data noise across different network architectures.

questions:
Invertible Neural Networks (INNs) often have limited flexibility in adjusting the dimensions of input and output, the number of layers, filters, and convolutional kernel sizes. These constraints can impede performance for complex tasks.

When mapping between velocity maps and seismic waveforms—data from two distinct domains (one spatial and the other spatio-temporal), why Invertible X-Net could demonstrate superior performance?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730392375995

thread_type:
official_review

<thread_object>
creator: reviewer_74kd

<content>
summary:
The paper proposes a new training scheme combined with two new model architectures for the problem of learning the forward and inverse map in deep learning for subsurface imaging. 
The training scheme consists of jointly learning the encoders and decoders for the two domains (velocity maps and seismic waveforms) with the translation network transforming the latent space representations from one domain to another.
The two architectures differ in the characteristics of the translation network with one consisting of two separate unidirectional U-nets (latent U-net) and the other of a single bidirectional invertible network (invertible X-net).
Both architectures achieve a what seems to be significant improvement in performance over existing works.
Finally the paper formulates a unifying framework in which existing approaches can be categorized and distinguished.

strengths:
1.	(Quality) High quality of experimental evaluation: Experiments are extensive over a range of datasets including an extensive evaluation of out-of-distribution performance and qualitative results are provided in addition to quantitative results. Hence, the conclusion of generally improved performance seems sound. 
2.	(Significance) The improvements in performance seem to be significant, however interpretability of how significant could be improved (see weaknesses point 5)
3.	(Significance) The finding that jointly solving forward and inverse problems is helpful for the forward problem but not for the inverse problem is interesting. 
4.	(Originality) The proposed unifying framework helps to quickly understand the different existing approaches and will be helpful for future works to expand the framework or propose improvements within it.
5.	(Clarity) The paper is overall well written and easy to follow also for researches not actively involved in the field of subsurface imaging.

weaknesses:
Major concerns

1. (Clarity) The experiment 5.1.3 addresses the question ‘should we train latent spaces solely for reconstruction?’, which I assume corresponds to the question 3) formulated in line 083 ‘what is the role of manifold learning?’.  
1.1 For increased clarity I’d suggest not changing the phrasing of the questions.  
1.2 The setup is unclear to me. What are training fractions? A fraction of the training set or a fraction of the training time? And why is it instructive to look at 5% and 10% training fraction?  
1.3 So, is the conclusion form the experiment that learning manifolds is not important? It is difficult to grasp the conclusion if the wording is not consistent. Is “influence the training of the latent spaces based on the translation objectives” identical to “not learning manifolds”?  
1.4 To better answer the question I think the following baseline is missing that combines the two alternatives (reconstruct then translate and directly translate) into directly learning the translation networks while having at the same time reconstruction losses (basically what is described in lines 243-245). Seeing how this combination performs would help answering the question if manifold learning helps.  
2. (Originality) The idea of the Latent U-net seems to be very similar to the InversionNet following the comparison in Table 1.  
2.1 The Modelling Mode is the same because the forward and inverse part of the Latent U-net are completely disjoint and hence correspond to an InversionNet and a ForwardNet (as long as no manifold learning is performed).  
2.2 While the latent U-net allows manifold learning, the experimental results pertain to the case without manifold learning same as the InversionNet.  
2.3 Latent space  translation is said to be identity for the InversionNet and U-net for the Latent U-net, but couldn’t we also just claim some middle layers of the InversionNet to perform latent space translation?  
2.4 The size of the latent space is low in both cases.  
--> So to my understanding the difference between the two boils solely down to the architectural design and size of the network but not to any conceptual differences. If that is the case, it should be stated more directly in the paper and the exact architectural differences should be explained in more detail. 
3. (Clarity) From Section 5.1.4 it is not entirely clear what the answer to the question raised in the paragraph (Is it useful to jointly solve forward and inverse problem) is. Is the answer that no it is not useful for learning the inverse map but yes it is useful for learning the forward map (both answers based on the findings in Figure 3)?

Minor concerns

4. (Clarity) Please provide links to the exact Section in the Appendix rather than just referencing the entire appendix (e.g. lines 308, 311, 314, 377, 388). 
5. (Significance) Include quantitative scores in the qualitative examples (e.g. in Figures 4 and 8) to give a better understanding of how differences in scores translate to significantly perceptible differences.

questions:
See weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730526112362

thread_type:
official_review

<thread_object>
creator: reviewer_vwh3

<content>
summary:
This paper explores deep learning methods to address the forward and inverse problems in subsurface imaging by mapping between velocity maps and seismic waveforms. The authors introduce the Generalized Forward-Inverse (GFI) framework, which unifies past approaches and addresses open questions around latent spaces, manifold learning, and model complexity. They propose two novel architectures, Latent U-Net and Invertible X-Net, which achieve state-of-the-art results on synthetic datasets and show promising zero-shot performance on real-world-like datasets.

strengths:
The paper presents a unified architecture for addressing both forward and inverse subsurface imaging problems, while previous studies typically focus only on inversions. Modeling these problems together within a single framework could offer valuable insights, though it may also introduce potential challenges.

weaknesses:
1. The paper’s narrow focus on subsurface imaging may limit its relevance for a broader audience at ICLR. If there are analogous problems in other fields where their framework could be applied? 
2. Comparisons to prior work are outdated, with main competing methods from 2019, like InversionNet and VelocityGAN. The authors are encouraged to consider more recent advances, such as “Physics-Informed Robust and Implicit Full Waveform Inversion Without Prior and Low-Frequency Information (2024)” and other state-of-the-art methods from 2023, for a fairer and more relevant assessment.
3. While the authors aim to model the forward problem using neural networks, it is uncertain if the learned mapping can capture complex physical dynamics, as seen in the governing equations of seismic data. Extensive out-of-domain experiments are recommended to assess the model’s robustness and generalizability to real-world data.

4. The paper's structure could be streamlined to highlight the proposed methodology. Currently, a substantial background section precedes a brief description of the methods. Specific sections of the introduction and related work and preliminary knowledge could be condensed.

5. The model specifications lack detail; adaptations made to standard UNet for subsurface imaging and the mechanisms enabling X-Net’s invertibility are unclear. Clarifying these aspects and specifying when each model might be preferred would improve understanding.
6. Experimental comparisons with recent baselines (published within the last 2-3 years) would add rigor to the evaluation, as several recent methods could serve as relevant benchmarks. To name a few, “Physics-Informed Robust and Implicit Full Waveform Inversion Without Prior and Low-Frequency Information 2024”; “Full-waveform inversion using a learned regularization 2023”; “Wasserstein distance-based full-waveform inversion with a regularizer powered by learned gradient 2023”; “Full-waveform inversion using a learned regularization 2023”, and more.
7. Key numerical results in Section 5 are deferred to the appendix, which reduces clarity. Summarizing key findings directly in the main text would enhance readability.
8. The Invertible X-Net significantly underperforms other models, raising questions about its ideal use cases. Further context on scenarios where the X-Net's design offers advantages would be valuable.

questions:
please see my concerns in the Weakness
</content>
</thread_object>

</thread>
<thread>
creation date:
1730561736799

thread_type:
official_review

<thread_object>
creator: reviewer_lx2y

<content>
summary:
Subsurface imaging is a technique for identifying the geophysical properties of layers underneath the Earth's surface. There are two directions: learning the mapping from velocity maps to seismic waveforms, called the forward problem, and the inverse problem which is the mapping from seismic data to velocity maps. Most of the previous deep learning-based research in the field focuses only on the inverse problem also referred to as full waveform inversion (FWI). This paper proposes a generalized forward-inverse (GFI) framework that unifies both directions, i.e., FWI and the forward problem. The framework builds on the assumption of manifolds and latent space translations, proposing two model architectures for the latter, namely Latent U-Net and Invertible X-Net. Latent U-Net architecture employs two U-Nets for translation between the velocity and waveform latent representations and vice versa, while Invertible X-Net uses a single IU-Net that simultaneously learns forward and inverse translations. The GFI framework encompasses previous works in deep learning for subsurface imaging, at the same time trying to answer questions such as the effect of latent space size, the importance of manifold learning and the value of jointly solving forward and inverse problems. The models were evaluated on the synthetic OpenFWI dataset and their generalization ability was tested on two real-world-like datasets.

strengths:
- The paper is clearly presented and well-organized.
- Even though the GFI framework is not the first idea that unifies seismic imaging forward and inverse problems (e.g., Auto-Linear [1]), it tries to systematically characterize and unify prior research in this area. The paper answers a few questions that were open in the research field. The paper concludes the following: the size of the latent space should be decided based on the complexity of the problem, the training of the latent spaces should be influenced based on the translation objectives and jointly solving forward and inverse problems is helping the model to achieve better forward solutions. In my opinion, these are answers to important questions that were backed up by the experiments in the paper.
- The paper introduces U-Net and IU-Net architectures for learning translations between velocity and seismic waveforms in the latent space. This leads to two new architectures for unified seismic imaging, namely Latent U-Net and Invertible X-Net. In comparison, the Auto-Linear framework's architecture comprises two separate linear layers trained for the forward and inverse translations in the latent space.
- Extensive and systematic experiments. The proposed approach was compared to the existing state-of-the-art methods both for the forward and inverse seismic imaging problems. The proposed method employing Latent U-Net architecture outperforms the existing methods in most of the datasets in both forward and inverse directions.

---
[1] Y. Feng, Y. Chen, P. Jin, S. Feng, Y. Lin, "Auto-Linear Phenomenon in Subsurface Imaging
." ICML, PMLR:235, 2024, pp. 13153-13174.

weaknesses:
- Section 5.1.1. "What is the effect of latent space sizes on translation performance?" is kind of vague. It states that the ideal size of the latent space should be decided based on the complexity of the problem being solved. It is not clear how to determine the complexity in real-world applications. I think the authors should be more specific in this section.
- In Section 5.1.2. "Do we need complex architectures for translations?", there is a comparison between large and small Latent U-Net. The only difference was the complexity (size) of the U-Nets for the latent space translation and the influence of having skip connections. What is the relation between the number of parameters in encoders and decoders and the U-Net and how does it influence performance? The Invertible X-Net was not included in the experiments and only one size was used throughout the paper. It would be useful to see how its size influences the performance.
- The Invertible X-Net model achieves worse results than the Latent U-Net model for the inversion problem and often fails to outperform other state-of-the-art methods. In contrast, it outperforms most of the methods for the forward problem. It is not clear from the paper what are the benefits of having such a unified framework. I think the limitations should be stated more clearly in the paper.
- In the results section, in qualitative comparison, Figure 4. and Figure 8. are missing a measure of error (at least MAE). The picked images should have approximately the same MAE as the reported results in the tables presenting a realistic case and fair comparison. It is not clear whether the presented results were in a sense cherry-picked, e.g., the proposed method achieved better results than the average and other methods below the average for the illustrated example.
- Instead of having a qualitative comparison of the methods on the two real-world-like datasets, I think it would be much more representative to have a quantitative comparison. Here, the presented examples might also be cherry-picked.

questions:
1. Could you comment on the complexity of the problems and how to "calculate" it when choosing the latent space size? How to choose this size for real-world applications and are there any limitations in real-world scenarios?
2. Why have you decided to remove Invertible X-Net from the experiments in Section 5.1.2.? Have you tried different model sizes for this architecture?  What is the relation between the number of parameters in encoders and decoders and the U-Net and how does it influence performance? 
3. It is not clear from the paper what are the benefits of having such a unified framework. Could you comment on the limitations? What would be your recommendation on which architecture to use and when?
4. Could you comment on the examples in Figure 4. and Figure 8. and the performance of the models achieved in terms of MAE?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732327874434

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Global Response to Review Comments

comment:
We sincerely thank all the reviewers for their constructive and detailed feedback. We are encouraged that the reviewers appreciated several key aspects of our work and found:

1. Our Generalized Forward-Inverse (GFI) framework to be a significant contribution in unifying forward and inverse seismic imaging problems [*Reviewers Lx2Y, 74KD, VWh3*].
2. Our proposed Latent U-Net and Invertible X-Net architectures to be novel and effective, demonstrating state-of-the-art performance on most datasets [*Reviewers Lx2Y, nmvk*].
3. Our experimental evaluation to be extensive and systematic, including insights into key factors affecting performance on both forward and inverse problems [*Reviewers Lx2Y, 74KD, nmvk*].
4. Our writing to be clear and accessible even to researchers outside the field of subsurface imaging [*Reviewers Lx2Y, 74KD*].

To address the reviewer comments, we have performed additional experiments and added new figures, tables, and text explanations in the revised manuscript (revisions highlighted in red). Here is a **summary of the new experiments:**

1. Analyzed effect of adding noise on forward and inverse problems for Flat Fault-B and Curve Fault-B datasets (Appendix Tables 11-14) (addressing Reviewer nmvk).
2. Performed ablations to study the effect of training with reconstruction loss and translation loss together for Latent U-Net (Large and Small) and Invertible X-Net (Figure 6 and Appendix Figure 22) (addressing Reviewer 74KD).
3. Analyzed effect of varying latent dimensions on Invertible X-Net’s performance for all OpenFWI datasets (Figure 5c) (addressing Reviewer Lx2Y).

Before addressing each of the reviewer’s comments individually, we address some of the shared concerns raised by the reviewers in our general response below: 


**Comment: Show more evaluation metrics in visualizations; how are the visualization examples picked? (Lx2Y, 74KD)**

> We have updated the paper to report both MAE and SSIM metrics alongside all visualizations of velocity and waveform predictions in the main paper (Figures 4 and 8) and the appendix (Figures 23-34). We can see that the metrics of the chosen examples are representative of the numbers reported in the Appendix Tables 7 and 9. All examples used for visualizations in the main paper were randomly sampled so no cherry-picking was involved. We also show several other visualization examples in the Appendix. 

**Comment: Why choose Invertible X-Net when it is not always the best? What is the advantage of using Invertible X-Net compared to Latent U-Net? (Lx2Y, vWh3, 74KD)**

> To analyze the overall performance of Invertible X-Net across all 10 datasets of OpenFWI, we have now included the average rank of all methods across all datasets for metrics: MAE, MSE, and SSIM in Appendix Tables 7 and 9. Lower average rank is better. We summarize this analysis of average ranks in the tables below. 

***Average Ranks for the Inverse problem:***

| **Model** | **MAE Rank** | **MSE Rank** | **SSIM Rank** |
|-----------------------|----------|----------|-----------|
| Inversion Net     | 2.8        | 2.8        | 3       |
| Auto-Linear        | 3.5        | 3.5        | 3.8    |
| Latent U-Net      | 1.3        | 1.3        | 1.3    |
| Invertible X-Net  | 2.7        | 2.4        | 1.9    |

***Average Ranks for the Forward problem:***

| **Model** | **MAE Rank** | **MSE Rank** | **SSIM Rank** |
|-----------------------|----------|----------|-----------|
| FNO                   | 2.4        | 3.1        | 3.1    |
| Auto-Linear        | 3.9        | 3.5        | 3.5    |
| Latent U-Net      | 2.3        | 2.6        | 2.3    |
| Invertible X-Net  | 1.4        | 1.8        | 1.1    |


Note that it is important to consider all metrics because while SSIM is more suited to capture visually perceptible changes related to geological similarity, MAE and MSE measure average errors across entire domains. From these tables, we can make the following observations:
- **For inverse problems:** Latent U-Net (Large) achieves the *best* performance with an average SSIM rank of 1.3. Invertible X-Net is a *close second* with an average SSIM rank of 1.9.
- **For forward problems:** Invertible X-Net performs the *best* across all metrics with an average SSIM rank of 1.1, followed by Latent U-Net that has an average SSIM rank of 2.3.

Based on these observations, we make the following recommendations:
- If we **only** want to solve **inverse** problems, use Latent U-Net (Large).
- If we **only** want to solve **forward** problems, use Invertible X-Net.
- If we want to solve **both forward and inverse** problems, use Invertible X-Net. Even though Invertible X-Net has second-best performance on inverse problems, it is highly parameter-efficient as it only requires 26M parameters to jointly solve forward and inverse while Latent U-Net (Large) requires a total of 70M parameters (35M x 2).

We have included these recommendations explicitly in Section 5.1.4 to make this analysis reader-friendly.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a joint forward--inverse solver for wave imaging problems using two autoencoders whose latent spaces (of possibly different dimensions) are paired via learned "latent space translations". The reviewers praised clarity of presentation, and extensive experiments. Initially there was some concern about the contribution and vagueness, but this has been in good part eliminated in the rebuttal phase. My opinion is that this is a nice paper very slightly above the acceptance bar. It is clear, the prior work section is very nice, and the experiments are complete and informative. I will recommend acceptance but I would not mind if it ends up being rejected. The reason for hesitation is that there is a lot of very related work which has not been acknowledged or compared to, for example

https://arxiv.org/pdf/2405.13220
https://arxiv.org/pdf/2305.13314

but also that I think that as a community we should finally go beyond OpenFWI benchmarks with 2D 70x70 pixel, highly stylized, layered velocity models which have a handful of degrees of freedom.