NCT-CRC-HE: Not All Histopathological Datasets Are Equally Useful
Paper • 2409.11546 • Published
YOLO — colorectal tissue classification (NCT-CRC-HE)
Nine-class tissue classifier for 224 × 224 px H&E patches of colorectal cancer tissue, built with Ultralytics YOLO (classification). Trained with one fixed recipe that we apply unchanged to every dataset, without dataset-specific tuning.
95.32 % accuracy and 0.997 macro AUC on patients the model never saw — above the dataset authors' own model (94.3 %) and the classic CNN baselines, trained on about one third of the available images.
| Metric on CRC-VAL-HE-7K (7,180 patches) | Value |
|---|---|
| Accuracy | 95.32 % |
| Balanced accuracy | 93.89 % |
| Macro AUC (one-vs-rest) | 0.997 |
The dataset
NCT-CRC-HE is one of the most used public benchmarks in computational pathology: nine tissue classes — adipose, background, debris, lymphocytes, mucus, smooth muscle, normal mucosa, stroma, tumour epithelium.
| Split | Patches | Role |
|---|---|---|
| NCT-CRC-HE-100K | 100,000 (86 whole-slide images) | training |
| CRC-VAL-HE-7K | 7,180 (separate patients) | external test |
The problems with this dataset
1. The random-split trap
The training set has no slide or patient identifiers, so a random train/test split puts patches from the same slide on both sides. On a random 10 % of NCT-CRC-HE-100K this model scores 99.8 %; on the external test set it scores 95.3 %. Only the external number is a fair measure — it is the one reported everywhere on this page.
2. A colour shortcut
Ignatov & Malivenko (2024) showed that three numbers per image — mean red, green and blue — already give 53.8 % accuracy, and a colour histogram 82.2 %, without looking at a single cell.
3. JPEG artifacts and corrupted patches
The same study found class-dependent JPEG compression and patches corrupted by wrong dynamic-range handling after stain normalisation — signals a model can learn instead of tissue morphology.
What others reported on CRC-VAL-HE-7K
Same protocol for every method: trained on NCT-CRC-HE-100K, tested on CRC-VAL-HE-7K.
| Method | Accuracy |
|---|---|
| DINO · self-supervised ViT | 95.9 % |
| iBOT · self-supervised ViT-L | 95.8 % |
| YOLO (this model) | 95.32 % |
| VGG16 | 95.3 % |
| ResNet-50 | 94.8 % |
| Inception-v3 | 94.8 % |
| VGG19 — Kather et al. 2019 (dataset paper) | 94.3 % |
| DenseNet | 92.9 % |
Methods designed specifically around this dataset's colour and artifact issues score higher (up to 98.33 % for an ensemble of two EfficientNet-B0 models). All published values as compiled in Ignatov & Malivenko 2024, Table 2.
Results in detail
| Class | Recall |
|---|---|
| Adipose | 97.4 % |
| Background | 88.0 % |
| Debris | 99.7 % |
| Lymphocytes | 99.8 % |
| Mucus | 98.6 % |
| Smooth muscle | 97.8 % |
| Normal mucosa | 99.2 % |
| Stroma | 66.3 % |
| Tumour epithelium | 98.3 % |
Ranking quality
This model has the highest macro AUC of our two models (0.997): it ranks stroma and smooth muscle almost perfectly (AUC 0.978 and 0.994). Its stroma errors — 142 of 421 patches, 90 of them called smooth muscle — come from the decision rule, not from failing to recognise the tissue.
Training and evaluation
| Training data | at most 4,000 patches per class from NCT-CRC-HE-100K (≈ 36,000 of 100,000) |
| Training | 50 epochs, 224 px, batch 64, Ultralytics default augmentation |
| Model selection | best epoch on a held-out 10 % of NCT-CRC-HE-100K |
| Test | CRC-VAL-HE-7K, evaluated once, no tuning on the test set |
| Hardware | one NVIDIA T4 |
Usage
from huggingface_hub import hf_hub_download
from ultralytics import YOLO
model = YOLO(hf_hub_download("ghostsas001/nct-crc-yolo", "yolo.pt"))
result = model.predict("patch.jpg", imgsz=224)[0]
print(model.names[int(result.probs.top1)])
Labels: ADI, BACK, DEB, LYM, MUC, MUS, NORM, STR, TUM.
Licence
Built with Ultralytics YOLO, which is licensed under AGPL-3.0; these weights are distributed under the same licence.
Limitations
Research model, not a medical device. Trained on Macenko-normalised patches from one source; other scanners, stains or magnifications may reduce accuracy. One training run, no confidence intervals.
References
- Kather J.N. et al. Predicting survival from colorectal cancer histology slides using deep learning. PLoS Medicine 16(1): e1002730, 2019. doi:10.1371/journal.pmed.1002730
- Dataset: NCT-CRC-HE-100K and CRC-VAL-HE-7K, Zenodo 1214456
- Ignatov A., Malivenko G. NCT-CRC-HE: Not All Histopathological Datasets Are Equally Useful. 2024. arXiv:2409.11546
- Downloads last month
- 44