mamounyosef/knee-abnormality-detection

Knee Abnormality Detection (RSNA, Kaggle)

Trained models from my solution to the RSNA Knee Abnormality Detection Kaggle competition. Each model takes one knee MRI study and scores 12 abnormalities: ACL and MCL tears, medial and lateral meniscus tears, medial, lateral and patellofemoral osteoarthritis, effusion, synovitis, Baker's cyst, bone contusion and fracture. Code, training recipes and the full write-up: GitHub.

Best results (macro AUROC on Kaggle's hidden test set): the 14-model ensemble scored 0.942 public / 0.928 private; the 4-model ensemble 0.942 / 0.927; the best single model (convnext_base_v5_96slices_per_finding_attn) 0.939 / 0.922.

How it was trained, in short

• Only 58 of 4,407 training studies have expert image labels, so training labels were built from each study's radiology report (9 languages, translated, then scored by an LLM), refined with MedGemma image opinions and then with out-of-fold predictions of earlier image models ("teachers"). Label versions: v2 report labels; v3 = v2 + MedGemma on 5 findings; v4 = v3 blended 50/50 with teacher predictions.
• Models read several MRI series (sagittal, coronal and axial; plain and fluid-sensitive) as 2D images per slice, 2.5D images of 3 slices, or bags of 3-slice windows, and pool them into 12 scores.
• "All data" models were retrained on every labelled study, so they have no validation score.

Models

Each folder holds one model: model.safetensors, config.json, a standalone modeling.py, metrics.json, its training log, and a model card with every detail. Val = validation macro AUROC on the held-out split (report labels); Audit-58 = macro AUROC on the 58 expert-labelled studies.

The 2 DINOv3 models (dinov3_convnext_base, dinov3_convnext_base_v5) are in mamounyosef/knee-abnormality-detection-dinov3, because DINOv3 fine-tunes must be distributed under the DINOv3 License.

4-model ensemble (public 0.942 / private 0.927)

Weighted logit mean 0.42 / 0.18 / 0.20 / 0.20 (ensembles/ensemble_4.json).

Model	What	Labels	Val	Audit-58	Alone: public / private
convnext_base_v5_96slices_per_finding_attn	The best single model: ConvNeXt-Base over 96 slices with a per-finding attention head	v4	0.9213	0.9272	0.939 / 0.922
medimageinsight_davit_huge_v5_96slices_per_finding_attn_sam	The per-finding-attention recipe on MedImageInsight DaViT-Huge, trained with the SAM optimiser	v4	0.9195	0.9271	0.937 / 0.925
dinov3_convnext_base_v5	DINOv3 ConvNeXt-Base, per-slice at 512 px, on v4 labels	v4	0.9211	0.926	-
swinv2_base_in22k_v5_warm	SwinV2-Base, per-slice at 512 px	v4	0.9152	0.9278	0.934 / 0.920

14-model ensemble (public 0.942 / private 0.928, the best score)

Plain logit mean of all 14 (ensembles/ensemble_14.json).

Model	What	Labels	Val	Audit-58	Alone: public / private
convnext_base_full_all_knees_same_orientation	ConvNeXt-Base, per-slice at 512 px, trained on all labelled data	v2 (all data)	full data	full data	-
mricore_vitb_cross_attention	MRI-CORE ViT-B/16 (DINOv2-style weights) with cross-slice attention	v2	0.8962	0.8951	-
mricore_vitb_mae_cross_attention_fulldata_flip	MRI-CORE ViT-B/16 (MAE weights) with cross-slice attention, trained on all labelled data	v2 (all data)	full data	full data	-
dinov3_convnext_base	DINOv3 ConvNeXt-Base, per-slice at 512 px, on v2 labels	v2	0.9011	0.8957	-
convnext_base_25d_all_knees_same_orientation_fulldata	ConvNeXt-Base, 2.5D at 336 px, trained on all labelled data	v2 (all data)	full data	full data	-
swinv2_base_in22k	SwinV2-Base, per-slice at 512 px, on v2 labels	v2	0.9053	0.9101	-
convnext_base_25d_poolattn_fulldata_flip	ConvNeXt-Base, 2.5D at 336 px with attention pooling, trained on all labelled data	v2 (all data)	full data	full data	-
convnext_base_25d_perfinding_slots_and_pooling_fulldata_flip	ConvNeXt-Base, 2.5D over 6 series, where each finding reads its own views and pools them its own way; trained on all labelled data	v2 (all data)	full data	full data	-
medimageinsight_davit_huge_perslice_480_warm	MedImageInsight DaViT-Huge, per-slice at its native 480 px	v2	0.9095	0.9061	-
medimageinsight_davit_huge_336_v3_f0	MedImageInsight DaViT-Huge, 2.5D at 336 px, fold 0 of 5 (v3 labels)	v3	0.9111	0.9221	-
medimageinsight_davit_huge_336_v3_f1	MedImageInsight DaViT-Huge, 2.5D at 336 px, fold 1 of 5 (v3 labels)	v3	0.9134	0.9116	-
medimageinsight_davit_huge_336_v3_f2	MedImageInsight DaViT-Huge, 2.5D at 336 px, fold 2 of 5 (v3 labels)	v3	0.9046	0.9095	-
medimageinsight_davit_huge_336_v3_f3	MedImageInsight DaViT-Huge, 2.5D at 336 px, fold 3 of 5 (v3 labels)	v3	0.9112	0.9117	-
medimageinsight_davit_huge_336_v3_f4	MedImageInsight DaViT-Huge, 2.5D at 336 px, fold 4 of 5 (v3 labels)	v3	0.9168	0.9116	-

PerFindingAttn-96 folds (public 0.937 / private 0.921 together)

5-fold cross-validation of the best recipe. Their out-of-fold predictions are the teacher behind the final (w50r10) labels.

Model	What	Labels	Val	Audit-58	Alone: public / private
convnext_base_v5_96slices_per_finding_attn_f0	Fold 0 of 5 of the per-finding-attention ConvNeXt (v4 labels, round-2 folds)	v4	0.9186	0.9223	-
convnext_base_v5_96slices_per_finding_attn_f1	Fold 1 of 5 of the per-finding-attention ConvNeXt (v4 labels, round-2 folds)	v4	0.9239	0.9303	-
convnext_base_v5_96slices_per_finding_attn_f2	Fold 2 of 5 of the per-finding-attention ConvNeXt (v4 labels, round-2 folds)	v4	0.9145	0.9268	-
convnext_base_v5_96slices_per_finding_attn_f3	Fold 3 of 5 of the per-finding-attention ConvNeXt (v4 labels, round-2 folds)	v4	0.9197	0.9261	-
convnext_base_v5_96slices_per_finding_attn_f4	Fold 4 of 5 of the per-finding-attention ConvNeXt (v4 labels, round-2 folds)	v4	0.9225	0.9243	-

ConvNeXt label teachers (public 0.918 / private 0.907 together)

5-fold 2.5D ConvNeXt models on v2 labels. Their out-of-fold predictions, with the MedImageInsight folds, built the v4 labels.

Model	What	Labels	Val	Audit-58	Alone: public / private
convnext_base_25d_18slices_f0	2.5D ConvNeXt-Base, fold 0 of 5 (v2 labels)	v2	0.8936	0.9069	-
convnext_base_25d_18slices_f1	2.5D ConvNeXt-Base, fold 1 of 5 (v2 labels)	v2	0.9045	0.9027	-
convnext_base_25d_18slices_f2	2.5D ConvNeXt-Base, fold 2 of 5 (v2 labels)	v2	0.891	0.8978	-
convnext_base_25d_18slices_f3	2.5D ConvNeXt-Base, fold 3 of 5 (v2 labels)	v2	0.8899	0.8986	-
convnext_base_25d_18slices_f4	2.5D ConvNeXt-Base, fold 4 of 5 (v2 labels)	v2	0.9021	0.9006	-

Using a model

from huggingface_hub import snapshot_download
import sys

name = "convnext_base_v5_96slices_per_finding_attn"
folder = snapshot_download("mamounyosef/knee-abnormality-detection", allow_patterns=f"{name}/*") + "/" + name
sys.path.insert(0, folder)
from modeling import load_model

model = load_model(folder)   # each model card gives its exact input shape

The DICOM-to-tensor preprocessing lives in the GitHub repository (runs/shared_code.py).

Licence and intended use

• Weights: CC BY-NC 4.0 (non-commercial use, credit required). Backbones keep their own licences, listed in each model card: ConvNeXt (Apache 2.0), SwinV2 (MIT), MRI-CORE (Apache 2.0), MedImageInsight (MIT).
• Trained on RSNA competition data, licensed for research use only (RSNA MIRA terms). No competition images, labels or reports are included.
• Research use only. Not a medical device and not for clinical use.
• Thanks to RSNA and Kaggle for the dataset, and to the authors of the pretrained backbones.