Venue: MIDL 2023 Conference

Reviews:

final_score: rating: 3: Borderline
review_content: summary: This paper proposed a representation learning approach for MRI slices and demonstrated its utility for MRI retrieval. The main goal is to learn representations unique to each subject but invariant to age/disease status and other scanner-related differences. Their approach uses two losses --- Barlow twins to impose invariance to image transformations & infoNCE captures unique information about each subject to learn the representations. The proposed method is tested on ADNI and a synthetic dataset created from ADNI to exaggerate contrast variations. The main innovation of the paper is in the MRI-specific image transformation.
strengths: - The paper does an excellent job of explaining the infoNCE and Barlow twins. However, some details about data and augmentation could be better (See detailed comments). 
- The authors compared against several baselines and highlighted traditional baselines' failures in capturing the contrast differences by creating a synthetic dataset. This provided more insights and increased my faith in their method. 

weaknesses: My main complaint with this paper is its poor explanation of supervised/self-supervised terminology. From what I understand, supervised means naturally paired data or scans of the same subject at different times are available, and self-supervised means scans are generated by image transformation. This differentiation appears artificial. Eventually, there is a set of scans belonging to the same subject --- naturally available + generated by augmentation. It would help if the authors could clarify this. 

With the above interpretation in mind, I have some specific questions --- 
    
1. Are some of the baselines unfairly crippled? For example, SimCLR can use both observed and transformed scans; but it only uses transformed scans in the SS+DT case. 
     
2. Is the difference between DeepBrainPrint and infoNCE (FS+DT) just the use of Barlow-twin loss and not? If yes, it is interesting to contrast why using Barlow twins in the early part of training improves the result. DeepBrainPrint reduces the contribution of image transformations gradually (because it is used with Barlow-twins), and this begs the question if using the loss function is essential or the decreasing effect of image transformations. 
detailed_comments: **Results:** I understand that the choice of k=3 is due to each subject having at least 3 scans. However, this may not be realistic as it is not known apriori how many scans of a subject are in the database. What would the precision and recall look like if the retrieval is done by thresholding similarity? 

**Dataset:** How many unique subjects are in the dataset? Sec 3 only mentions the final number of train/test/validation scans. Did you ensure that the subjects did not overlap between the train and test sets?

**Writing:**
- It would be helpful to see more descriptions/mathematical forms of image transformation, especially about the "bias field."
- "The main challenge of this tool is to preserve privacy when searching for data, including avoiding the use of the subject's name." --- This needs to be clarified. How does the tool preserve privacy? Instead, it reveals a loophole in the anonymization and shows that anonymized scans can be almost perfectly identified. 
- Use \citet for inline citations. For example, "...Specifically, we extend the approach in (Zbontar et al., 2021)..."

recommendation: Poster
review_title: null

final_score: rating: 1: Strong reject
review_content: summary: This paper proposes a semi-self-supervised pipeline for creating fingerprints of brain MRI scans in real-time and efficiently retrieving images of the same subject within a large dataset. A fully supervised branch and a self-supervised branch are used,  the self-supervised branch allows the method can learn robustness to synthetic transforms. Results are favorable in the specific context here, however unfortunately are much below existing state of the art for the same task and the same data.

Post rebuttal:
The premise of the original paper was the state-of-the-art performance of their novel network for the brain MRI reidentification task, having relatively high accuracy. However, the paper overlooked the keypoint approach, which achieves perfect accuracy on the same ADNI data and task, including data an order of magnitude larger[1] and multiple real MRI modalities[a]. The authors make efforts to include and understand the keypoint approach, however it appears that a major rewrite is necessary to revise the motivation and situtate this work in the literature, so unfortunately I cannot recommend accept.

Several misunderstandings remain regarding the complexity and performance of algorithms, memory search, particularly as the number of data N grows large. Experimental comparisons, if properly performed, will show the keypoint approach has perfect accuracy for reidentificaiton real or simulated modalities used here. Since the authors approach makes use of 2D images, it would be natural to evaluate this approach also for 2D face reidentification. 

strengths: The paper addresses an interesting topic, re-identification from brain MRI, which may be very relevant for personalized medicine applications and preventing errors.
The idea of incorporating self-supervised learning of transforms is interesting.
weaknesses: The main weakness in this work is that it is far from state-of-the-art in medical image re-Identification. Specifically for brain MRI, uncited work [1] obtains perfect accuracy for the same task and data, using an order of magnitude more subjects (7500 MRIs including ADNI, OASIS, HCP). The approach here stuggles in top-3 accuracy on a small subset of the data.

Notably [1], requires no training, preprocessing or registration, and is invariant to full 3D similarity transforms (3D scaling, rotation, translation), intensity variations (the approach here considers 2D rotations, no scaling). Numerous instances of mislabelled subjects are identified and verified by dataset administrators, including in ADNI. 

[1] Chauvin, L., Kumar, K., Wachinger, C., Vangel, M., de Guise, J., Desrosiers, C., ... & Alzheimer’s Disease Neuroimaging Initiative. (2020). Neuroimage signature from salient keypoints is highly specific to individuals and shared by close relatives. NeuroImage, 204, 116208.

detailed_comments: Recommend for the authors to consider the work [1], including practical aspects as the number of datapoints becomes large.

The paper mentions real-time performance, however no timing is given, nor limitations as to the number of subjects that can be enrolled in training.

recommendation: Poster
review_title: null

final_score: rating: 4: Weak accept
review_content: summary: This paper proposes an innovative approach, combining existing methods, for Brain-MRI Re-Identification using a semi-self-supervised contrastive DL approach. The pipeline is divided in two branches (supervised and self-supervised) and includes several transformation as data augmentation strategy. The proposed approach is validated on the ADNI database and synthetic database, where it shows improved performances compared to many other state-of-the-art methods. I believe that both the methods and results would be interesting for the MIDL community.
strengths: I found that the paper was well written and easy to read. All technical details are specified and the code is available which makes the approach reproducible. The comparison with many existing methods is interesting and shows the benefits of the proposed method. 
weaknesses: A few important information are missing (see comments about section 3 and 4). Some more results could be reported to better compare the tested methods (see comments about section 4; 4.1 and 5) but I understand that space is limited.
recommendation: Oral
review_title: null

Meta-review: metareview: After extensive consideration on the part of the program committee, we agree with the area chair that the paper should be conditionally accepted based on the requirements specified in their comment, e.g, inclusion of the updated results and lacking literature.

For future improvements on this paper, I encourage the authors to consider shortcomings, e.g., having less performance than other methods in real data (ADNI), and that the real time execution is not very important for most applications.  
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)