Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper tackles the problem of multiple instance learning in pathology imaging where annotations are available only at slide level and the goal is to predict it at instance level. The author exploits the low-rank structure in the data to design a new contrastive learning approach. LRC is proposed as an extension SupCon approach for low-rank data by defining the loss function on the most- and least-distant subspaces. At the feature aggregation level, an iterative approach is proposed using transformers. A learnable low-rank latent matrix L is used across the layers to encode global features. The transformer model named ILRA is based on two transformer modules GABf which reduces the space dimension and GABb restores it. The ILRA transformer layer is applied k times. The proposed approaches are benchmarked on three datasets, showing the improved performance 
strength_and_weaknesses: Strengths
Low-rank property provides good inductive bias in applications such as in computational pathology

Weakness
No validation with multi-center datasets
The computation time has not been discussed and could be prohibitive for real-world usage where inference time needs to be reduced.

clarity,_quality,_novelty_and_reproducibility: The paper is not so easy to read. Its clarity could be further improved.

summary_of_the_review: I think the main drive for improvements in the proposed model is the contrastive learning loss LRC rather than the proposed global features. ILRA-MIL alone does not do better than the baseline methods. This would require further investigation and limits the novelty of the paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents a novel framework for Whole Slide Image (WSI)  classification based on two
low-rank methods:
1. A Low-Rank Constraint (LRC) feature embedder to extract features from pathological slides in
a self-supervised fashion.
2. An Iterative Low-Rank Attention MIL (ILRA-MIL) model for bag-level classification , derived
from the vision transformers architecture.

Both methods rely on low rank assumptions, i.e. the existence of a data representation in a smaller dimension space that exhibit more discriminatory characteristics for feature embedding on the one hand, and lower computational complexity for bag-level prediction on the other.
strength_and_weaknesses: Strength : 
The approach presented here is quite new in the field of digital pathology, and the authors show
that the enforcement of low-rank constraints on top of existing self-supervision and attention-based
MIL strategies improves the results on several whole slide image classification datasets.

Weaknesses : some explanations are not clear enough or suffer from ambiguous notations, and overall there are several errors along the manuscript.
clarity,_quality,_novelty_and_reproducibility: The authors should carefully proofread their work again to correct all the spelling mistakes, missing words
and other typos.
For instance :
- In equation (5), the brackets should cover the entire set of conditions, and i and j should not be written as indices
- In the same section, one can read: \... over the most- and least-distant subspace C1(a), Cr(a) ...".
Most and least distant should be switched to be coherent with the rest of the explanations. The same mistake is repeated in the appendix.
- Where does the index k come from after equation (6)? Is there a relation between i and k or j and k? Overall, this small paragraph after equation (6) is not very clear. Could the authors give more explanations as to why this threshold is needed?
- The first sentence of section 3.2.3 does not make much sense. Could the authors rephrase? Perhaps
the word \image" is missing after \low-rank".
- What exactly is xb? A linear layer? What is \rho ? Is it the same \rho as in equation (10)?
summary_of_the_review: The paper brings an interesting idea worth being introduced in the community. But it lacks clarity and is impaired by many typos, missing words and unprecise formulas.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper addresses the problem of WSI classification by optimizing the feature embedding and feature aggregation with low-rank properties. Firstly, the paper improves the vanilla contrastive loss with additional low-rank constraints to collect more positive samples for contrast. Then, the authors devise an iterative low-rank attention feature aggregator to make efficient non-local interactions among instances. Experiments show the effectiveness of the proposed model. 
strength_and_weaknesses: Strength:
(1) The authors extend contrastive learning with a low-rank constraint (LRC) to learn feature embedding using unlabeled WSI data.
(2) The authors design an iterative low-rank attention MIL (ILRA-MIL) to process a large bag of instances, allowing it to encode cross-instance interactions.
(3) Experiments are conducted on multiple datasets and the effectiveness is well studied. 

Weakness:
(1) In the Abstract, the authors state that “we highlight the importance of instance correlation modeling but refrain from directly using the transformer encoder considering the O(n^2) complexity”. However, in the main body part, there are not any details to discuss the importance of instance correlation. Besides, in the experimental section, the importance should be investigated.
(2) The authors should also provide more details to discuss cross-instance interaction, and whether is it helpful to boost the classification performance?
(3) Some details of experimental settings are missing. For example, k is the total number of layers. Thus, in the experiments, how to set k? Besides, for the used contrastive loss, which type of augmentation is adopted in this study? 


clarity,_quality,_novelty_and_reproducibility: This paper leverages the properties of the apparent similarity in high-resolution WSIs, which essentially exhibit low-rank structures in the data manifold, to develop a new MIL with a boost in both feature embedding and feature aggregation. The authors have claimed that code will be available. 
summary_of_the_review: Overall, the structure is clear and the proposed model is well-validated.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper propose a new model architecture and training method to classify digital pathology whole-slide images (WSI). These images are in the giga-pixel range and cannot be classified directly by a model. The paper proposes a multi-instance learning approach with contrastive self-supervised learning for patch level feature extraction and an iterative low-rank attention model for the multi-instance feature aggregation. The method is tested on 3 public digital pathology datasets and shown to significantly outperform SOTA. In an ablation study, other feature extraction and MIL techniques are compared to the proposed one, showing superior results of individual components. In a parameter analysis, low-rank size, pooling method and number of iterations are studied.


strength_and_weaknesses: Pros:

* A sophisticated MIL model that significantly beats SOTA on several public datasets

* Both feature-extraction and MIL steps are improved, as shown by the ablation study.

* An exploration of several hyper-parameters show a sweet spot for the number of iterations in MIL, an asymptotic behavior for the rank of the low-rank decomposition and the superiority of nonlocal pooling against regular max-pooling or local-attention-pooling.


Cons:

* There are no speed information given for training and inference. How long does it take to classify a slide on average ? How long does it take to train the feature extractor, the MIL model ?

* no details are given on which type of augmentation is used in the contrastive loss.


clarity,_quality,_novelty_and_reproducibility: * The paper is very well written, well organized and clear.

* The method proposed contains several contributions: improved loss on feature extraction (low-rank constraints added to SupCon loss); new MIL aggregator (their iterative gated attention low-rank blocks appears to be novel)

* The authors claim that code will be available. Without it, i think it is hard to reproduce this approach as it consists of a complex loss for the self-supervised feature learning and a complex model for the MIL part. Also details on what augmentation to use for the contrastive learning are missing, 


summary_of_the_review: Overall, a good paper with a clear and convincing approach that beats SOTA.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The submission proposes a number of engineering innovations for whole slide image classification, a task whose main challenge is the large size of the images, in the gigapixel range.  The two main design components - a contrastive learning step and MIL - both incorporate a low rank approach, bringing some common theme to the paper.  Experimental results show improvements in the 2% accuracy range across several benchmark datasets, and I would request that the reviewers provide error bars for these results.  The paper received some borderline reviews and one more positive assessment.

The main significance to the ICLR audience is probably as an applications paper, in which a challenging task with very large images is tackled, and effective solutions are demonstrated.  The main weak points of the two reviewers who gave the lowest assessment seem to have been addressed.  The main points by Reviewer oayv were the suggestion for a multi-center dataset and timing information.  I somewhat agree with the authors that the benchmarks here are sufficient for showing the feasibility of the solution, and that a more clinically relevant multi-center study is a different stage of the development process.  Timing information was provided, showing some significant overhead, but something that could conceivably be incorporated into a processing pipeline.  Reviewer NvEE raised concrete issues in the presentation aspects of the paper, and the authors seem to have addressed those that were listed in the review.  The submission promises a code release.

The submission is therefore a borderline case.  It could make an acceptable applications paper if there is room in the program, but it is not without its flaws (statistical significance being a remaining one, but with 2% accuracy increase it is likely to be there, and possibly easy to demonstrate for the camera ready version).
justification_for_why_not_higher_score: The submission contains an interesting application, but it has 3 less enthusiastic reviews.  It will be of potential interest to those working on very large images or in medical imaging, but not necessarily a wider ICLR audience.
justification_for_why_not_lower_score: It would be OK to bump this paper down.  It's a fine paper and could be accepted, but it's also OK if there's not room for it in the program.
Decision: Accept: poster