Model Card for DALE-CT-0-L

Authors: Evan W. Damron · Mahmut S. Gokmen · Mitchell A. Klusty · Caroline N. Leach · Emily B. Collier · V. K. Cody Bumgardner — Institute for Biomedical Informatics Center for Applied AI (IBI-CAAI), University of Kentucky

This repository hosts the backbone weights for DALE-CT-0-L (Depth-Aware Latent-Euclidean Computed Tomography — Large corpus), a foundational Vision Transformer (ViT-Large) trained entirely self-supervised, from scratch, on a ~296,000-case multi-source chest-CT pool — to our knowledge the largest chest-CT pretraining corpus reported to date. It scales the supervision-free DALE-CT-0 recipe by ~11.5× in data with no auxiliary labels.

This is the recommended general-purpose DALE-CT backbone: it achieves the best external transfer of the 2D family (RAD-ChestCT retrained-probe AUROC 0.7572), matches the anatomically supervised DALE-CT-1S-v2 in-domain without any labels, and preserves the anatomical world model (frozen slice embeddings linearly decode volumetric position, R² = 0.973). For maximum in-domain CT-RATE performance, use DALE-CT-2S.

Quick Load (timm)

import timm
model = timm.create_model("hf-hub:Kentucky-Open-Science/DALE-CT-0-L", pretrained=True)
model.eval()

Inputs must be Hounsfield-Unit slices preprocessed exactly as during training (clipping + z-score; see the full example below).

The DALE-CT Family

All numbers are our own head-to-head measurements: every model (including the public 3D baselines in the paper) is probed under one linear-probing MIL protocol on shared splits (CT-RATE n = 992 test scans; RAD-ChestCT n = 360). See the paper for the full protocol and confidence intervals.

Model	CT-RATE Macro AUROC	RAD-ChestCT AUROC (frozen / retrained probe)	Role
DALE-CT-0-L ⭐	0.8156	0.6281 / 0.7572	Recommended general-purpose backbone — best 2D external transfer; supervision-free at ~287k-scan scale
DALE-CT-2S	0.8247	0.6252 / 0.7389	Best in-domain (CT-RATE)
DALE-CT-1S-v2	0.8098	0.6284 / 0.7334	Anatomical (TotalSegmentator) dense supervision only
DALE-CT-0	0.8057	0.5946 / 0.7477	Pure self-supervised, CT-RATE
Finetuned DINOv2	0.7953	0.6252 / 0.7550	Continual-pretraining baseline

Paper: DALE-CT: Depth-Aware 2D Slice Encoders Learn an Anatomical World Model of Chest CT · Code: Kentucky-Open-Science/DALE-CT

Model Details

• Model Type: Vision Transformer (ViT-Large) for chest CT analysis.
• Developed by: Institute for Biomedical Informatics Center for Applied AI (IBI-CAAI), University of Kentucky
• Base Model Architecture: vit_large_patch14_dinov2 (via timm), randomly initialized and trained from scratch with patch_size=16, img_size=512, in_chans=1, dynamic_img_size=True.
• Input: 1-channel grayscale CT slice (Hounsfield Units, preprocessed as below — note the normalization statistics differ from the CT-RATE-trained variants).
• Output: class token and patch tokens (embedding dimension 1024).
• License: CC BY-NC-SA 4.0 — non-commercial use only. The pretraining pool includes an institutional chest-CT archive in addition to public collections; the weights are released for research use.

Training Data

• Corpus: a multi-source chest-CT pool of 296,429 cases across 32 collections (287,302 used for training), stored at native resolution in true-HU form. Two cohorts dominate: the National Lung Screening Trial (~130k scans) and an institutional chest-CT archive, together roughly three-quarters of the pool; the remainder comes from ~30 public collections (RSNA pulmonary embolism, STOIC, 4D-Lung, COVID cohorts, CT-RATE, among others). See the paper for details.
• Preprocessing: HU clipped to [-940.8, 923.1] (0.5/99.5 foreground percentiles fit on the full pool), mapped to [0, 1], then z-score normalized (pool mean -25.03, std 246.87 in HU space). These statistics differ from the CT-RATE-trained DALE-CT variants — use the values above with this model.

Training Procedure

• DDP, bf16, 16×H100 GPUs; 3 epochs over the pool (191,357 iterations, global batch 384), no auxiliary head.
• Depth-aware multi-crop: two global 256² crops from the slab center slice and eight local 144² crops drawn from a 3-slice native-resolution axial slab; local crops guided to TotalSegmentator foreground (p=0.8). No ReX guidance (the pool lacks ReX labels outside CT-RATE).
• Objective: pure LeJEPA (invariance + SIGReg, λ=0.02); no labels of any kind.

Preprocessing Example

import torch, numpy as np, timm

model = timm.create_model("hf-hub:Kentucky-Open-Science/DALE-CT-0-L", pretrained=True)
model.eval()

clip_min, clip_max, mean_hu, std_hu = -940.8, 923.1, -25.03, 246.87  # DALE-CT-0-L stats
rng = clip_max - clip_min
norm_mean, norm_std = (mean_hu - clip_min) / rng, std_hu / rng

hu_slice = np.random.uniform(-1000, 1000, size=(512, 512))  # replace with real HU data
x = torch.from_numpy(hu_slice).float().clamp(clip_min, clip_max)
x = ((x - clip_min) / rng - norm_mean) / norm_std
x = x[None, None]  # (1, 1, H, W)

with torch.no_grad():
    cls_feature = model(x)                    # (1, 1024)
    tokens = model.forward_features(x)        # (1, 1 + N_patches, 1024)

Citation

If you use this model, please cite the DALE-CT paper (https://arxiv.org/abs/2606.07775).