deit-cifar100

Dense fine-tuned weights of the 3 DeiT variants on CIFAR100 (starting from the ImageNet-1K official weights, low-LR fine-tuning, no distillation), produced by the LPViT pipeline strategy=dense. One subdirectory per model:

Model	Subdirectory	Reference accuracy
DeiT-Tiny	deit_tiny_patch16_224_cifar100/	~81.65%
DeiT-Small	deit_small_patch16_224_cifar100/	~87.54%
DeiT-Base	deit_base_patch16_224_cifar100/	~90.04%

Each subdirectory contains model.safetensors (standard DeiT keys), config.json, and a README.md (loading examples). Common input stats (CIFAR100 @224): mean=[0.5071, 0.4867, 0.4408], std=[0.2675, 0.2565, 0.2761].

Hyperparameters (dense fine-tune)

• Initialization: ImageNet-1K DeiT official weights (timm Hugging Face)
• Model / input: DeiT patch16 @224, 100 classes
• Epochs: 30
• Batch size: 128
• Optimizer: AdamW (eps=1e-8)
• Learning rate: effective 5e-5 (recipe lr=2e-4, scaled by lr * batch * world / 512)
• LR scheduler: cosine, min_lr=1e-5, warmup=5 epochs
• Weight decay: 5e-2
• Label smoothing: 0.1
• RandAugment: rand-m9-mstd0.5-inc1
• Mixup: 0.8
• CutMix: 1.0
• Random Erasing: 0.25
• Distillation: none (no teacher)
• Backbone: fully updated (not frozen)
• Precision: bf16
• Input normalization: mean=[0.5071, 0.4867, 0.4408], std=[0.2675, 0.2565, 0.2761]