###### Abstract  
Background: In this study focusing on colorectal carcinoma (CRC), we address the imperative task of predicting post-surgery treatment needs by identifying crucial tumor features within whole slide images of solid tumors, analogous to locating a needle in a histological haystack. We evaluate two approaches to address this challenge using a small CRC dataset. **Methods:** First, we explore a conventional tile-level training approach, testing various data augmentation methods to mitigate the memorization effect in a noisy label setting. Second, we examine a multi-instance learning (MIL) approach at the case level, adapting data augmentation techniques to prevent over-fitting in the limited data set context. **Results:** The tile-level approach proves ineffective due to the limited number of informative image tiles per case. Conversely, the MIL approach demonstrates success for the small dataset when coupled with post-feature vector creation data augmentation techniques. In this setting, the MIL model accurately predicts nodal status corresponding to expert-based budding scores for these cases. **Conclusions:** This study incorporates data augmentation techniques into a MIL approach, highlighting the effectiveness of the MIL method in detecting predictive factors such as tumor budding, despite the constraints of a limited dataset size. histopathology; CRC; budding; supervised segmentation; classification +
Footnote †: journal: Medical  
many biological aspects [5; 6]. Of note, in 3D, it can be appreciated that some tumor buds are tentacle-like tumor protrusions, whereas others are detached tumor cell aggregates. These detached, small tumor cell aggregates are tumor buds per definition [5]. In several publications on colorectal carcinoma (CRC) (and other solid tumors), it proved useful in predicting nodal status and overall survival [7; 8]. For CRC, there are at the moment two main scenarios where tumor budding has a potential influence on clinical decision-making: First, in the case of pT1 carcinoma regarding planning the surgical approach, and second in UICC stage II, where the role of adjuvant chemotherapy still needs to be clarified in detail [6]. Though a consensus on tumor budding evaluation in CRC could be achieved, there remains considerable inter- and intra-observer variability [8; 9; 10]. Such prediction uncertainties hamper the potential clinical interpretation of such measurements. Clinical decisions could only be made if the underlying scoring system poses enough reliability. To sum it up, in theory, the task of determining the budding score per slide or, rather, slide region is a good use case for applying computer vision tools for classification (regarding a budding score) or regression (regarding tumor bud numbers). In addition, even the direct prediction of the nodal status based on the histology of the primary tumor seems to be a promising task [11]. Besides sparing humans from tedious counting tasks, the most significant benefit would be that machine learning-based classification for budding status, and subsequent nodal status prediction could reduce inter-observer variability. This would make tumor budding a reliable histomorphological biomarker. However, the relevant parts of the image--the metaphoric needles--would first have to be found or defined within the large histological sections--the metaphorical haystack. Against this background, in the presented work, we first challenge the concept of tumor budding. This is achieved through model training for nodal status prediction, followed by an examination of (a) the regions within the images harboring the most pertinent information and (b) the presence of tumor budding within these identified regions. This investigation aligns with our hypothesis concerning the significance of morphological biomarkers in histology. Subsequently, (c) we assess the prognostic efficacy of the histology-based prediction and juxtapose it with established biomarkers such as grading within our dataset.