# Model checkpoints for [PMF](https://github.com/hushell/pmf_cvpr22)

NOTE: for DINO-small, peak VRAM is about 32GB; for DINO-base, peak VRAM is about 42GB.

Meta-testing with `dino_small_batch16` trained on full Meta-Dataset:
```
python -m torch.distributed.launch --nproc_per_node=8 --use_env test_meta_dataset.py --data-path ../../datasets/meta_dataset --dataset meta_dataset --arch dino_small_patch16 --deploy finetune --output outputs/md_full_dinosmall --resume md_full_128x128_dinosmall_fp16_lr5e-5/best.pth --dist-eval --ada_steps 100 --ada_lr 0.0001
```

Meta-testing with `dino_small_batch16` trained on ImageNet domain of Meta-Dataset:
```
python -m torch.distributed.launch --nproc_per_node=8 --use_env test_meta_dataset.py --data-path ../../datasets/meta_dataset --dataset meta_dataset --arch dino_small_patch16 --deploy finetune --output outputs/md_inet_dinosmall_6gpus --resume pmf_metadataset_dino/md_inet_128x128_dinosmall_fp16_lr5e-5/best.pth --dist-eval --ada_steps 100 --ada_lr 0.0001
```

## Results
The validated meta-test learning rate using 5 episodes for each domain is shown in the bracket.  
Method                     |ILSVRC (test)              |Omniglot                   |Aircraft                   |Birds                      |Textures                   |QuickDraw                  |Fungi                      |VGG Flower                 |Traffic signs              |MSCOCO
---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------
[md_full_128x128_dinosmall_fp16_lr5e-5](https://huggingface.co/hushell/pmf_metadataset_dino/blob/main/md_full_128x128_dinosmall_fp16_lr5e-5/best.pth)             |73.52±0.80&nbsp;(lr=0.0001)       |92.17±0.57&nbsp;(lr=0.0001)     |89.49±0.52&nbsp;(lr=0.001)        |91.04±0.37&nbsp;(lr=0.0001)       |85.73±0.62&nbsp;(lr=0.001)      |79.43±0.67&nbsp;(lr=0.0001)     |74.99±0.94&nbsp;(lr=0)       |95.30±0.44&nbsp;(lr=0.001)        |89.85±0.76&nbsp;(lr=0.01)        |59.69±1.02&nbsp;(lr=0.001)
[md_inet_128x128_dinosmall_fp16_lr2e-4](https://huggingface.co/hushell/pmf_metadataset_dino/blob/main/md_imagenet_128x128_dinosmall_fp16_lr2e-4/best.pth)          |75.51±0.72&nbsp;(lr=0.001)       |82.81±1.10&nbsp;(lr=0.01)       |78.38±1.09&nbsp;(lr=0.01)       |85.18±0.77&nbsp;(lr=0.001)     |86.95±0.60&nbsp;(lr=0.001)       |74.47±0.83&nbsp;(lr=0.01)     |55.16±1.09&nbsp;(lr=0)       |94.66±0.48&nbsp;(lr=0)       |90.04±0.81&nbsp;(lr=0.01)       |62.60±0.96&nbsp;(lr=0.001)