Venue: MIDL 2023 Conference

Reviews:

final_score: rating: 4: Weak accept
review_content: summary: The paper proposes an Ultrasound (US) physics-inspired rendering module in an implicit neural representation-based method that uses a set of 2D US images for novel view synthesis of target anatomy from any arbitrary viewing direction. For any 3D point in the target anatomy, a neural network first learns to map it to five parameters of a US image formation model. Then a rendering module learns to synthesize a 2D image from these parameters. Tracked 2D US images from a phantom and images synthesized from CT scans are used to train and evaluate the method. Initial results show that the network can synthesize US images, including the properties such as shadows.
strengths: The idea of estimating parameters related to the US image formation model and the US physics-based rendering module within an end-to-end deep neural network is novel and interesting to my knowledge. 

Initial results show that the method is able to synthesize various aspects of an ultrasound image, such as shadows. 
weaknesses: - Some aspects of the paper are difficult to understand, such as the figures and the details on how the network is trained (see detailed comments). 

- The experiments are carried out on phantom and synthetic data that do not consider the real-world scenario of motion of target anatomy. It is unclear how exactly the novel view synthesis would be used in real-life scenarios, and whether the method can be trained with moving target anatomy. 

- The main contribution of the paper is to learn the US physics-inspired parameters, but there are very limited results (single figure with little explanation) showing whether these parameters estimated by the network consistently correspond to the expected properties across several images.
detailed_comments: Anisotropy in ultrasound: the term is not well-defined in the paper; it would be helpful for the readers to define it in the paper’s context. 

Fig 1 (b) is difficult to understand. The captions (or labels in the image) should describe what the three rows and columns represent. The first row seemed to be compounded images, but then it seems to be one 2D view, as the term compounding in this paper seems to have been used only for the 3D surface view. I couldn’t understand the relationship between the angles and colors. The mid-row seems to show TSDF representation of the anatomy. Does it represent only the surface of a target anatomy or the whole volume? I.e. what would slices of the TSDF look like? From the figure, it seems it only represents the surface of the white structure seen in the first row. 

Fig 5. What values do the colors correspond to? The figure needs a color bar.  

There is limited analysis of Fig 5 in the discussion section. Since rendering with US physics parameters is the main contribution of the paper, it would be better to provide a more detailed analysis (with a few more randomly sampled images) on whether the estimated parameters are consistent to what they are meant to represent. For example, beta seems to be high almost everywhere (if yellow/whitish > blue/blackish). The dark regions in the US image can represent shadows or fluid (it seems there are both in Fig 5 as well; a vertical shadow and an elliptical fluid region), but both seem to have similar values for all the parameters except for scattering density and amplitude. 

What are t_n & t_f in Eq 1&2? What’s “d” in Eq 3, perhaps “t”? 

Is the rendering module fully differentiable for the loss to backpropagate?
recommendation: Poster
review_title: null

final_score: rating: 3: Borderline
review_content: summary: The authors propose using NeRF-like architecture for ultrasound imaging. They follow the NeRF settings where the main objective is to obtain renderings from an arbitrary view. In the US context, the NeRF-like model is trained to learn a mapping from 3D to 5D. The authors show that they can synthesize novel views from US sweeps. They show the performance of their method on synthetic data and phantom data.
strengths: NeRF-like architecture (neural field) is a promissing approach for representing data, and is useful for various modalities. It is therefore interesting to see, how neural representation can be utilized for ultrasound imaging. 
weaknesses: It is unclear what is the main objective of this work: generate novel views or infer 3D structure? The work seems very preliminary. It is only tested on synthetic and phantom data. No comparison with any baseline method is provided.
recommendation: []
review_title: null

final_score: rating: 2: Weak reject
review_content: summary: The author presented a physics-enhanced implicit neural representation (INR) for ultrasound (US) imaging that learns tissue properties from overlapping US sweeps. The presented work is the first to address view-dependent US image synthesis using INR. The author used synthetic and phantom B-mode images datasets to validate the performance. 
strengths: The presented work is the first to address view-dependent US image synthesis using INR. This paper is very innovative. It supports progress towards integrating the implicit 3D US representation exemplified with NeRF into medical applications.
weaknesses: 1. Figure 2 presents an overview of the system of models. Page 5, section 3.2 put some words briefly explains the architecture of the framework.  It is reasonable, due to the page limit, it is difficult to elaborate more in the main context. But, it is worth using more words to clearly present the design of the model, either in the main context or in the appendix.
In addition to this, in Appendix A, It is suggested to provide a complete explanation of your design of the architecture. You cannot only put an addition or complement of Section 3.2.
It is suggested to present, 1) What is the input and output of this architecture? 2) Why did you design this type of model? What are the reasons you selected MLP, not other types of neural network?

2. In Figure 2, $\lambda L_{S S I M}\left(I^{\prime}, I\right)+(1-\lambda) L_2\left(I^{\prime}, I\right)$, is this a loss function?
The reason of designing and choosing this loss function is not clearly demonstrated. The explanation of this loss function is missing in the main context. The explanation of 'Our loss is a weighted sum of SSIM and L2 between the rendered and true B-mode images and controlled by a parameter λ ∈ [0, 1]. In our experiments we used λ = 0.9.' in the appendix is not enough. It is suggested to clearly explain why you choose this loss function and the parameters.

3. In this paper, a 8 layer MLP is selected. Will this cause overfitting? Normally we use 3-4 layers of MLP, and this is enough. 8 layers of MLP is questionable. It is suggested to provide a separate section, called 'hyper-parameter tuning' to show the process you investigate this. Also, Adam optimizer does not need to be cited. It is so commonly used.

4. I wish the author could perform more experiments to show that their methods have good performances. Experimental results and analysis of the results can go into the Appendix section. In the appendix section, It is suggested to explain more on the experiments.


recommendation: []
review_title: null

final_score: rating: 4: Weak accept
review_content: summary: The authors propose using neural radiance fields (NeRF) to model the interaction of ultrasound waves with tissue that gives rise to BMode images using a neural network, and propose to use this to do volumetric reconstructions of anatomical images by compounding multi-view predicted images. The method is demonstrated in synthetic data and in a phantom where the BMode images and the relative position and orientation of these images is available.

The paper is well written, and proposes a very novel and interesting approach to inverse problem solving in medical ultrasound imaging. The validation is somewhat limited to phantom and synthetic data from one subject each, and although quantitiative data form tissue (attenuation, density etc) can be calculated, this is left out of the validation.

Overall, an interesting proof of concept paper that could benefit from a more thourough validation and a discussion on how this will move forward to real in vivo data.
strengths: * A novel approach to image reconstruction for ultrasound that combines ultrasound physics, and neural implicit representations. I believe this would be of interest for the community.
* Early results suggest ability to recover physical properties of tissue from sweeps.
* Authors commit to make data available, enabling further research on this which can help move the field forward

weaknesses: * Figures need to be improved. Figures appear at non intuitive locations. For example, Fig. 3 appears before Fig 1 is mentioned in the text. Also, figure 1 is referred to after figure 2 which should be reversed. Also, caption information is incomplete. For example in Fig. 3, please describe all columns and the relation between the left and right ( I am assuming right is synthetic data but it is not specified). Also there is n discussion on any visual differences in the gt and generated synthetic data. What is the relevance of that synthetic dataset, provided it looks very different to real ultrasound images from a phantom?
* The volumes can be generated by compounding generated images; however, it seems like the learned model should contain the information about the object to be reconstructed within.  Why is compounding necessary, instead of retrieving the object directly from the learned NeRF? Why not use density, which is computed as shown in Fig 5?
* Figure 5, which shows the estimated tissue properties, is described in the discussion instead of in the results. Also, these values are not validated; if gt is not available for the phantom, it should at least be available for the synthetic dataset. 
* No discussion on to what extent this method relies on perfect alignment of the input images, and therefore how likely it is that this can be used in real patients and what what type of tracking. Also no discussion on how motion will be dealt with.
* The proposed method reconstruct one subject at a time -is there any space for generalising to multiple subjects/larger datasets, or is this something that will be done subject by subject? How is the amount and diversity of input images affecting the accuracy of the result.
recommendation: Poster
review_title: null

Meta-review: metareview: The authors propose using NeRF-like architecture to obtain renderings from an arbitrary view of ultrasound imaging. The method is validated on synthetic data and on a phantom. While results on real data would have been a plus, this first feasibility study is interesting. 

confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)