Paper: Vibroacoustic Frequency Response Prediction with Query-based Operator Networks (ICLR-2024)

Reviewer comment (Reproducibility):
The anonymous repository's documentation would benefit from some improvement. For example, it seems to be using Weights & Biases for logging but no explanation on how to use it is provided. Furthermore, the command in the README under "Train a model" section is not correct as the paths do not correspond to the folder structure. The setup script does not provide specific versions for the packages being installed (except for pytorch's cuda version) and, thus, this will impact the reproducibility of this work.

Author response:
We acknowledge the issues you raised regarding our repository’s documentation. We will incorporate the suggested improvements by updating our readme with instructions, adding version numbers to the requirements.txt and adding an explanation for Weights and Biases setup.

Reviewer comment (Experiments):
The datasets are synthetic, and do not seem to come from real-world measures. I understand from the paper that this is cost prohibitive, and that is why this new dataset is relevant, but I got the impression that this applied domain has implications in the real world, and therefore I'd guess an experiment in a real-world dataset, even if small, would be relevant.

Author response:
Real-world experiments are beyond the scope of this work due to prohibitive costs for manufacturing and measurements of field solutions (a measurement would require laser scanning vibrometry). However, previous research validated the use of the Mindlin model for the plate and errors of the finite element method are well understood in the engineering community [1].

Reviewer comment (Experiments):
What's the distance between the simulations and the data collected in the real world? If the authors can conduct some experiments on real-world equipment, it would be very great.

Author response:
Our focus in this work is on establishing a data-driven benchmark dataset. Real-world experiments are beyond the scope of this work since special equipment for manufacturing and measurement would be needed. However, previous research provides good evidence for the validity of the Mindlin model and FEM in real world settings (see general answer). Generally, we are highly interested in real-world applicability and follow-up work should address this.

Reviewer comment (Presentation):
The mesh geometries in this paper are not represented in vector form, and they are blurred. This should not happen and should be fixed in the rebuttal phase.

Author response:
The mesh geometries are displayed (e.g. in Fig. 3) in their original discretized (an $81 \times 121$ grid). We did not apply interpolation or other image improvements to have a faithful presentation. This will be explained in the figure caption.

Reviewer comment (Experiments):
Why does Table 3 not have baselines' performances? It's better to test baseline methods on the transfer learning setup as well, and this is very critical.

Author response:
We agree that baselines are important and follow your suggestion of adding new baselines to the revised manuscript for a more comprehensive comparison of transfer learning performance.

Reviewer comment (Presentation):
It seems that the visualization of Figure 6 is not very good because (b) and (d) are not very similar. Also, I think it's better to add baseline comparisons or even videos (videos can be provided in the supplementary or the anonymous link).

Author response:
We appreciate your feedback on Figure 6 and suggestions for further visualizations. Regarding Figure 6, we want to make the point that despite visible errors in the field solution prediction, the frequency response (=average) is well estimated. 
We agree that providing more visualizations of the model results is a good idea and will add them. For now, we provide several gifs showcasing the predicted and ground truth field solutions of the FQO-UNet. 
https://drive.google.com/drive/folders/1EUeeAtPm0ZFdC6oIBsxJQbsOhhfXVUk4?usp=sharing

Reviewer comment (Reproducibility):
The mechanical model should be put into the main text and linked to the dataset construction or methodology. This is very essential in the PDE domain.

Author response:
We agree on the importance of linking the mechanical model to our methodology and dataset construction. This will be changed in the revised manuscript.

Reviewer comment (Experiments):
What do grid-based models mean, and why is FNO not based on grids? I think FNO should have an option to work in regular grids. If grid-based methods are stronger, then FNO should also be put in the similar mode.

Author response:
By grid-based models we refer to models, that can only make predictions for predefined, equidistant frequencies (i.e. a 1-dimensional grid). As you correctly point out, this holds for our implementation of the FNO method as well. The Grid-UNet and Grid-RN18 are ablation variations of the FQO-UNet and FQO-RN18, where we removed the frequency query mechanism.
We understand that the distinction between baseline methods and the ablation variations of our proposed method needs to be clarified. We will do so in our updated manuscript.

Reviewer comment (Experiments):
It seems that in the numerical experiments, there is no noise added to the data. Predicting data without noise is not a difficult task from a data-fitting perspective, especially with large models like deep networks.

Author response:
**Absence of Noise in Data:**
Our work concerns learning models, that predict the frequency response of a vibrating plate. The primary goal is to learn a faithful surrogate model. All experiments evaluate scores on the test set, which consists of unseen plate geometries. Hence, they measure generalization and not just how well the data is fitted. Nonetheless, we agree that robustness to noise is important and particulary relevant for real world transfer (for example when manufacturing is imperfect) and should be addressed by follow-up work.

Reviewer comment (Reproducibility):
The details of the finite element method and linear solver are missing, which is crucial to the generation of the data.

Author response:
**Details for Finite Element Method and Solver:** 
We agree that the details of the finite element method and linear solver are crucial. We employ a specialized FEM implementation for acoustics [1]. The setup involves shell elements with 9 nodes, 6 degree of freedom and quadratic ansatz functions for discretization. The direct solver MUMPS [2] is employed. More comprehensive descriptions will be included in the final manuscript.

Reviewer comment (Theory):
It seems that in the calculation of the aggregation frequency response, the paper only describes how it is done, but no justification is given. Are there any references which can justify the calculation steps?

Author response:
**Aggregated Frequency Response Justification:** 
Calculating the aggregated frequency response by taking the mean squared velocity is a common choice in vibroacoustic simulation. The mean squared velocity is an important measure for the radiated sound power. Therefore, acoustic engineers use this quantity to evaluate structural design measures [3]. We will clarify this in the paper.

Reviewer comment (Experiments):
In the dataset generation, the displacement is set to 0, which means that a term in the plate theory differential equation is set to 0. Does it mean that the plate is static? Is it a common setting in existing work?

Author response:
**Question: Dataset Generation Settings:** 
Thank you for highlighting this aspect. In our dataset, the zero displacement condition is applied exclusively at the plate's edges. This means the central part of the plate can move (vibrate) freely, while the edges are fixed. This modeling choice is common in structural analysis, especially in engineering applications where plates are often components of more complex assemblies. To prevent further misunderstandings, we clarify this in our manuscript.

Reviewer comment (Evaluation):
The data is generated by solving the differential equation use FEM and a linear solve to (1). Thus, the accuracy of the solution is crucial here. What is the error to equation (1) after applying the linear solver? How well does the generated data satisfy the differential equation?

Author response:
**Accuracy of FEM and Solver:**
Generally, the finite element approach in combination with linear direct solvers converges to the exact solution [6]. The accuracy of our solutions in relation to equation (1) is assured by a suitable discretization. In particular, we have chosen the mesh size such that the smallest occurring bending wavelength is discretized with at least 10 nodes. The smallest bending wavelength occurs at the highest frequency and can be analytically calculated for thin plate structures. This approach is common practice for choosing the mesh size of vibroacoustic models [7].

Reviewer comment (Experiments):
How does the proposed method perform on frequencies that it is not trained on? Since one strength of the proposed operator model is that it can be evaluated at any frequency values, it is important to show that it can empirically generalize to other frequencies well.

Author response:
**Performance on Untrained Frequencies:**
We recognize the importance of our model's generalization capabilities, which we've explored for different amounts of beadings as detailed in our transfer learning section. Extending this to unseen frequency ranges is a great idea! We conducted this experiment and present our results in the general answer.

Reviewer comment (Novelty):
Considering that this work is applicational and has little contribution to methodology, I do worry that this work does not have enough audience in the ICLR community.

Author response:
1. **Concerns on Audience and Contribution:**
We would like to emphasize the position of our research in the established fields of *Machine learning for physics and science*. While machine learning applications to engineering are a fairly new topic in the machine learning community, there is great potential for impactful research. Applications of machine learning to other domains were highly successful, e.g. AlphaFold in biology [1]. Our work seeks to bridge the gap between the communities by challenging data-driven approaches with a mechanical engineering problem. While most existing research models the temporal evolution of dynamic systems, our work's novelty is to directly predict solutions in the frequency domain. To address this task, we contribute a benchmark dataset, scores of relevant baselines, and the new FQO-UNet model.

Reviewer comment (Novelty):
How do you distinguish different methods as baselines and proposed methods? I feel that they are all applications of existing models with minor changes.

Author response:
2. **Question on Baselines vs. Proposed Method:**
As baselines, we selected methods that were proposed in the literature for related physics-based problems. In contrast, our proposed FQO-UNet is specifically designed to address the unique challenges of vibroacoustic frequency response prediction. 
For instance, the prediction of resonance peaks is addressed by the frequency-query approach. However, we agree that the distinction between baseline methods and the ablation variations of our proposed method needs to be emphasized and we will do so in our updated manuscript.