Venue: MIDL 2023 Conference

Reviews:

final_score: rating: 3: Borderline
review_content: summary: This work compared the robustness of CT reconstruction methods when adversarial attacks/noise are presented. 6 CT reconstruction methods including traditional methods such as FBP and deep learning-based methods are evaluated on the low-dose parallel beam (LoDoPaB) CT dataset. The authors considered two types of adversarial attacks: untargeted attacks and localized attacks. Experimental results show that deep learning-based methods are more prone to untargeted attacks, but both traditional and deep learning methods are affected by localized attacks.
strengths: + The performance metrics are well-designed to evaluate different methods comprehensively.
+ It is interesting that the localized attacks affect both classical approaches and deep learning approaches, but hardly change the data consistency.
weaknesses: - It is not clear what criteria are used to select the compared methods, especially deep learning approaches (see "Questions To Address In The Rebuttal" for details).
- The manuscript, particularly Section 4, is somewhat cumbersome and hard to digest.
detailed_comments: Table 1 and 2 are too dense to get any meaningful conclusion quickly. It would be very helpful to simply/redesign the presentation in a more efficient way.
recommendation: []
review_title: null

final_score: rating: 4: Weak accept
review_content: summary: The paper focuses on the robustness against adversarial attacks for classic and neural network-based image formation approaches used in low-dose CT imaging. The authors consider three types of neural network-based image formation approaches, including i) models that directly predict images from undersampled sinograms, ii) models that denoise back-projected images, and iii) unrolled optimization networks, and compare their robustness against untargeted global and local attacks. For the classical approach, filtered back-projection with TV regularized is used.  They consider white box attacks where the attacker has complete knowledge of the image formation network and the classifier network. Projected gradient descent (PGD) with different levels of perturbations is used for attacking sinograms, and the authors compare the robustness of the neural network-based attack with the classical approach. They conclude that NN-based solutions are more brittle against untargeted attacks, while both classical and NN-based approaches are brittle against localized attacks (which is due to the brittle nature of the NN-classifier!). 
strengths: * The paper is well written
* Comparison of the robustness of classical approaches against deep neural network-based solutions could be of interest to researchers
* The experiments support the claims of the paper
weaknesses: Main concern: 

* Studying the vulnerabilities of NN-based image formation approaches demonstrates a worst-case behavior of these networks. However, the proposed attack has little to no physical-world realization. Here is why:
  1. The attacker has full knowledge of both image formation and classifier modules,
  2. The attacker has access to the raw sinogram, which means that the attacker can insert noise in the middle of the image formation pipeline,
  3. The attacker has time to optimize an image-specific adversarial perturbation. 
These assumptions make the proposed approach trivial and of little interest to physical-world imaging. 

* The paper contains no novelty. In particular, there are no specific considerations required for attacking the sinograms, and this has been done in prior publications, for instance, see: "Wu, W., Hu, D., Cong, W., Shan, H., Wang, S., Niu, C., ... & Wang, G. (2022). Stabilizing deep tomographic reconstruction: Part B. Convergence analysis and adversarial attacks. Patterns, 3(5), 100475." Hence, the only novelty is the comparison of the classical approach against NN-based approach. 

 


recommendation: Poster
review_title: null

final_score: rating: 3: Borderline
review_content: summary: This work focuses on the robustness of CT reconstructions algorithms to adversarial attacks. They evaluate the performance of several reconstruction algorithms, including classical methods and deep networks, under two types of adversarial attacks: untargeted and localized attacks. The results suggest that deep learning based reconstruction algorithms are more susceptible to the untargeted attacks than classical ones. Furthermore, both deep learning based and classical algorithms are easily affected by localized attacks highlighting the need of better regularisation.
strengths: - The paper is written clearly and easy to understand.
- This discover a phenomenon that it is easy to attack reconstruction algorithms such that the resulting CT reconstructions change malignancy prediction evaluated by a pre-trained predictor.
- The localised adversarial attack is interesting,
weaknesses: - The technical contribution of this paper is limited. The fact that deep learning reconstruction methods are susceptible to adversarial attacks is known and has been actively researched. It could be better to point out some potential solutions as well.

detailed_comments: - I believe better figure captions and better presentation of figures will be helpful.

recommendation: []
review_title: null

Meta-review: metareview: After reading the reviews, and authors responses, I believe the authors have generally addressed most important concerns raised by the reviewers. From the discussions, I believe that this work can lead to interesting in-person discussions/debate regarding the transferability of the adversarial attacks and the practical significance of localized attacks. Thus, I recommend the acceptance of this submission.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)