Venue: MIDL 2023 Conference

Reviews:

final_score: rating: 4: Weak accept
review_content: summary: The authors propose and compare three different graph structures, as well as different combinations of node labels, to capture the spatial layout of tumor buds and T-cells, in order to investigate them as potential risk predictors for CRC progression. The best-performing GNN architecture is able to increase specificity by 20% compared to the currently recommended risk stratification based on histopathological risk factors, without losing any sensitivity. The work was developed with a dataset of 626 tumor budding hotspots from 575 patients, that will be made available publicly.

strengths: - Overall good work
- Clear writing, with good English
- Extended ablation study and set of metrics for evaluation
- Interesting application of GNN, with promising results for risk assessment improvement
weaknesses: - Paper structure is not suitable to follow ideas easily
- Some key ideas are presented in Appendix
- The methodology is not well described in the main text
- A concrete proposal of the framework is missing


detailed_comments: - Section 2 should be in Appendix. 
- Table 3 from Appendix B should be in section 3.
- Section 3: It would be better to have some summary on the workflow used for dataset creation, instead. Also, Figure 1 is not very informative (so, maybe, it can be removed?). Figure 2 should be referenced in Section 3.2
- Section 3.2.1: which kind of post-processing was used after Net?
- Section 3.2.1: how exactly was the output quality assessed?
- Section 3.2.1: typo in 200 x 200 pixels (letter x instead of times symbol)
- It is suggested to avoid to start paragraphs with Tables description. The authors can try to include reference to tables after some kind of explanation.
- Section 4.1: z is not defined.
- Figure 5 for Appendix C should be in the main text, as well as its description.
- All results tables are missing units.


recommendation: Poster
review_title: null

final_score: rating: 4: Weak accept
review_content: summary: The authors propose to investigate two clinically relevant risk factors, namely tumor budding and lymphocyte infiltration at the invasive front. In order to capture the spatial layout of tumor buds and T-cells, the authors compare using three popular graph neural network architectures, as well as combinations of the node labels. Additionally, the others produced a new pT1 Hotspot Tumor Budding T-cell Graph (pT1-HBTG) dataset consisting of 626 tumor budding hotspots from 575 patients, which the authors have stated will be made open source.
strengths: 1. The authors collected a new dataset which will be made open source. 
2. The authors compare three popular GCN methods on this new dataset and demonstrate that the best of these is able to increase specificity by 20% compared to the currently recommended risk stratification based on histopathological risk factors, without losing any sensitivity.
3. The paper is well written, the analysis is thorough, and the reviewer is particularly impressed with the figures which were extremely helpful in demonstrating the utility of graph-based approaches for capturing these particular clinical features.
weaknesses: 1. The choice to only report results for 10-ensemble models is a bit confusing. Ensembling is typically done to squeeze out some additional performance, but most papers do not report results only of ensembles. For practical clinical considerations, GCNs are already expensive to run compared to regular CNNs, and adding in a need to ensemble 10-models together makes this even more costly.
recommendation: Poster
review_title: null

final_score: rating: 4: Weak accept
review_content: summary: This paper investigates spatial layout of tumor buds and T-Cells as potential risk factors for colon cancer with deep graph models. Experiments were done with 626 tumors from 575 patients. Authors propose and compare three different graph structures. Specificity is increased by amount 20% compared to current convention. 
strengths: -- exploring interaction between tumor buds and T-cells using GNN is innovative.
-- large data set 
-- smooth and good structured paper
-- promising results with new directions
-- clinical problem is important


weaknesses: -- comparisons with SOTA not available
-- evaluation and model selection parts are not entirely clear
-- there is no external data available for generalization check
-- not clear how this can be embedded into routine clinics although this reviewer recognizes the importance of the work

detailed_comments: The paper is written good, overall innovation is solid as GNN was never applied to this problem before, and exploring new markers with DL sounds grounded. Results seem improved compared to baseline, but there is no comparison with any other method, and the evaluation routine about to chose best model is not clear.
recommendation: Poster
review_title: null

final_score: rating: 3: Borderline
review_content: summary: The authors applied graph neural networks to extract spatial features from tumour budding and lymphocytic infiltration at the invasive front. The authors assessed various technical considerations in building a GNN classifier to predict high/low-risk patients. The GNN-based classifier is shown to potentially improve the risk assessment of patients, especially in reducing false positives in a mid-sized cohort. 

strengths: - The paper has chosen a clinically relevant question, where significant improvement could lead to patient benefit. 
- The authours have conducted a thorough ablation study, where a large number of technical variations were tested.
weaknesses: - While having a mid-size cohort in histology studies, the number of patients seems low considering training GNNs with very big graphs. In addition, the data is highly unbalanced. The risk of overfitting is high. In table 2 and the results in the appendix, the uncertainties in the key performance metric TNR are very high, suggesting widespread overfitting in the configurations tested. 
- There is no data in this study suggesting the GNN approaches learned biologically meaningful signals that separate high vs low-risk patients. This again adds to the risk of the GNNs overfitting to circumstantial signals in the potentially noisy graphs. 
- There are significant variations between the graph constructed by the three different approaches. The best model uses one that is drastically different from the other two. The implication of this is not clear.  
- There is no non-GNN baseline to establish the added value of GNN approaches.
detailed_comments: While I appreciated the clinical value of the task, some epidemiological information on pT1 patients would help readers to better understand its significance.  

recommendation: Poster
review_title: null

final_score: rating: 5: Strong accept
review_content: summary: In the paper, the authors introduce a new dataset for risk stratification of colorectal cancer patients. The dataset represents a collection of digital pathology slides with sparse labeling into two classes (tumor buds and immune system T-cells). In addition to the semantic labeling, the authors provide various graph representations reflecting the relationship between the tumor buds and T-cells. Finally, clinical meta-data such as lymphovascular invasion, tumor grade, etc. is provided.

The dataset is benchmarked using various graph-based neural nets to carry out risk stratification into low- and high-risk.
strengths: It is a very well-written paper. Was a pleasure to read! Thank you for your efforts!

The new digital pathology dataset (of 575 patients) is gonna be released upon acceptance that would contribute to the community working on graph-based medical imaging algorithms. 

Proper benchmarking with existing SOTA methods for GNN is performed.

A detailed discussion of the experiments and the data is provided throughout the text and in the appendix.
weaknesses: My only concern is that such benchmarking of new datasets is typically published in journals (for example, since longer back-and-forth discussions with reviewers can provide more thorough feedback and reveal dataset details that could be useful to include in the paper). But if authors prefer to release it via MIDL, I will not object much.



recommendation: Oral
review_title: null

Meta-review: metareview: This an excellent and well-written study, with interest to the community. I recommend it for an oral presentation.
confidence: 5: The area chair is absolutely certain
Decision: Accept (Poster)