Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: The paper is well written, and seems to solve the proposed problem well. I am very confused however about the point of the problem you are trying to solve. From what I understand you want to take a voxelized version of an MRI scan and distort it such that the observed face is no longer identifiable with the original patient, but the useful information in the MRI is preserved. Part of your method requires or extracts the brain from the MRI. In either case you claim that that is information is available and sufficient for downstream tasks, and everything can be removed. Would a superior method here not just be to take this brain information directly.  The De-Identification Quality metric which you use would obviously get the optimal 20% accuracy here as the we have no way of associating images of brains with images of faces, and even if we do, you have not demonstrated that you method would perform well on it either. I see no point in altering the face of the scans as the face information will be useless for any downstream task, and all other information is apparently preserved, so why not just take this preserved data as the privacy preserving MRI information? While this may fail to perform well under SIENAX, I would say this is a problem with using this software as a metric, as the brain would clearly be preserved. 

If I am mistaken here, I apologies, and am happy to change by rating and comments given sufficient reasoning. However, from the knowledge I posses in the field I do not think the approach has merit due to superior, trivial baselines. 

I also have an issue with the experiment for assessing privacy preservation. I would say that the mechanical turk experiment is valuable, but only if you also demonstrate that deep learning based approaches cannot identify the associate the faces as well. Without this I have no way of knowing if your method fails to a simple CNN. 


review_title: Confusion over point of method

final_score: rating: 6: Marginally above acceptance threshold
review_content: Pros.

    • This work develops a novel approach to de-identify privacy-sensitive regions of the MRI scans that enables sharing the data for downstream tasks like segmentation, classification etc.

    • The presented de-identification framework is non-reversible and also preserves medically sensitive regions.

    • The performance of the presented methodology is established through various simulations.


Cons.

    • While experimental results supports author’s claims regarding privacy-preservation without adversely affecting the medically relevant features, further justification is required for the superiority of the proposed method over existing algorithms. For instance, while with the DEFACE method illustrated in Figure 1 it seems impossible to identify the patient, the de-identification quality for DEFACE in Figure 3 does not agree with that. Similar argument can be made for the QUICKSHEAR method as well.

    • While it has been argued in the beginning of the manuscript that the removal-based methods can compromise the medically relevant features for the downstream tasks, this is not inferred from Figure 4 in which FACEMASK and QUICKSHEAR methods achieve higher Dice index for brain segmentation as well as lower bias (s.d.) for age estimation. Authors can improve the presentation to clarify these concerns.
review_title: This paper addresses a crucial need in medical imaging community, i.e. an automated and reliable anonymization method to remove patient’s identifiers from MRI scans. Instead of removing privacy-sensitive regions, the presented approach aims at remodeling privacy-sensitive facial structures. The experiments exhibit higher de-identification measures compare to some existing methods.

final_score: rating: 3: Clear rejection
review_content: This paper proposes a GAN-based method to remodel the privacy information of an MRI scan, i.e., the face. They first propose a probabilistic solution to construct a surface representation and then compute the convex hull of the head. The convex hull was used to instruct the GAN to generate a synthetic face. The main contribution of this paper is that they use GAN to generate a 3D volume MRI in which the face has been remodeled rather than removed. Experiment results showed the proposed method outperforms several existing removed-based methods.
Cons:
My main concern about this paper is the purpose of remodeling the face. In my opinion, removing the original face is enough for the de-identification of an MRI scan. Why spent so much effort to generate a new one? Is the new face provides extra information to the downstream medical analyses? The authors point out that their method did not alter any content of medically relevant data. Therefore, I think remodeling the fake face is unnecessary and the method may have little practical uses.

review_title: ADVERSARIAL PRIVACY PRESERVATION IN MRI SCANS OF THE BRAIN

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors proposed a conditional GAN-based de-identification method for MRI scans. This method aimed to prevent possible personal information from leaking from the face surface of MRI data. The study is useful in certain clinical situations. However, the applied methods lack enough novelty in the aspect of deep learning theory. C-GAN that can manipulate the generation results have been proposed and applied to a lot of tasks in recent years. In my opinion, this submission is more suitable for medical imaging conferences such as  MICCAI, ISBI, or EMBC. 

Feedbacks from authors are constructive to solve my puzzle and I decided to increase my rating. 
review_title: An interesting task but does not have wide audience in this conference.

final_score: rating: 7: Good paper, accept
review_content: The authors developed a GAN based method for privacy preservation in Neuroimaging called C-DeID-GAN. Providing an overview of the related work, they highlighted the shortcomings of the commonly applied methods to preserve privacy while simultaneously maintaining facial structures to enable commonly applied operations such as brain segmentation. To address these challenges the authors built on similar GANs approaches and adjusted it to the 3D dimensionality and the added criteria to fully preserve the brain area in the transformation. They experimentally validated the proposed method using two MR imaging datasets (ADNI and OASIS-3) for a) privacy preservation b) Common image processing (Brain/VCSF/GM/WM segmentation and age prediction). Finally they demonstrated a PoC for different image resolutions.

The scope of the paper is very relevant to the community. The paper is well presented within the scientific context. The methodology development and the experimental validation are well done. Overall the contribution is justified.

Major comments:

* While the experimental validation is well designed, a statistical analysis is missing and is quite essential to support some of the authors' claims. Claims such as “the model performs on par with the other models” are not meaningful enough without a statistical test to confirm it. This specifically refers to the analysis results shown in Figures 3 and 4. 
* One of the main advantages of MRI compared to other neuroimaging methods is the multi-modality, i.e. utilization of different MRI sequences (such as DWI, PWI, FLAIR and others) to retrieve different properties of the brain tissue. The authors tested the operations of segmentation and age prediction but did not address this aspect. A common process that is usually done with MRI multi-modal imaging is co-registration. Where does that come into play with regard to the DeID method? How is that addressed or will be potentially addressed? I would find it useful if the authors could relate to that point. 
* Regarding the analysis on controlling synthesis via the privacy transform, I’ve found the motivation for the analysis as well as the conclusion are not entirely clear. I find this analysis not directly relevant to the study objectives. I would therefore suggest moving it to the appendix and instead include the study limitations.

Minor comments:

* Page 3, Figure 2 legend line 5-6: There’s a double “to” in the sentence - please delete it.
* Page 6, Benchmark De-Identification methods: The authors state that the comparison methods are used “day-to-day basis” in the clinic. Can you support this claim by a relevant reference or other source?





review_title: While needing to add statistical analysis, overall justified contribution in a relevant topic

Meta-review: The paper received diverging review feedback. While reviewers found merits in the work, they also raise serious concerns over experimental validation, comparison with the existing methods, and practicality of the proposed method. It appears that the paper can benefit from better writing and more experimental validations clarifying all these points. 
Decision: Reject