Package: dinov2
DINOv2: Learning Robust Visual Features without SupervisionMeta AI Research, FAIRMaxime Oquab,
Timothée Darcet,
Théo Moutakanni,
Huy V. Vo,
Marc Szafraniec,
Vasil Khalidov,
Patrick Labatut,
Armand Joulin,
Piotr Bojanowski[Paper] [Blog] [Demo] [BibTeX]PyTorch implementation and pretrained models for DINOv2. For details, see the paper:DINOv2: Learning Robust Visual Features without Supervision.DINOv2 models produce high-performance visual features that can be directly employed with classifiers as simple as linear layers on a variety of computer vision tasks; these visual features are robust and perform well across domains without any requirement for fine-tuning. The models were pretrained on a dataset of 142 M images without using any labels or annotations.https://user-images.githubusercontent.com/60359573/230078733-5faffa19-e6ce-4c55-9200-62dd76f8236a.mp4Visualization of the three first principal components of the patch features of all frames, mapped to RGB values.Pretrained modelsmodel# ofparamsImageNetk-NNImageNetlineardownloadViT-S/14 distilled21 M79.0%81.1%backbone onlyViT-B/14 distilled86 M82.1%84.5%backbone onlyViT-L/14 distilled300 M83.5%86.3%backbone onlyViT-g/141,100 M83.5%86.5%backbone onlyPretrained models via PyTorch HubPlease follow the instructionshereto install PyTorch (the only required dependency for loading the model). Installing PyTorch with CUDA support is strongly recommended.A correspondingmodel cardis included in the repository.importtorchdinov2_vits14=torch.hub.load('facebookresearch/dinov2','dinov2_vits14')dinov2_vitb14=torch.hub.load('facebookresearch/dinov2','dinov2_vitb14')dinov2_vitl14=torch.hub.load('facebookresearch/dinov2','dinov2_vitl14')dinov2_vitg14=torch.hub.load('facebookresearch/dinov2','dinov2_vitg14')InstallationThe training and evaluation code requires PyTorch 2.0 andxFormers0.0.18 as well as a number of other 3rd party packages. Note that the code has only been tested with the specified versions and also expects a Linux environment. To setup all the required dependencies for training and evaluation, please follow the instructions below:conda(Recommended)- Clone the repository and then create and activate adinov2conda environment using the provided environment definition:condaenvcreate-fconda.yaml
condaactivatedinov2pip- Clone the repository and then use the providedrequirements.txtto install the dependencies:pipinstall-rrequirements.txtData preparationImageNet-1kThe root directory of the dataset should hold the following contents:<root>/test/ILSVRC2012_test_00000001.JPEG<root>/test/[..]<root>/test/ILSVRC2012_test_00100000.JPEG<root>/train/n01440764/n01440764_10026.JPEG<root>/train/[...]<root>/train/n15075141/n15075141_9993.JPEG<root>/val/n01440764/ILSVRC2012_val_00000293.JPEG<root>/val/[...]<root>/val/n15075141/ILSVRC2012_val_00049174.JPEG<root>/labels.txtImageNet-22kPlease adapt thedataset classto match your local setup.:warning: To execute the commands provided in the next sections for training and evaluation, thedinov2package should be included in the Python module search path, i.e. simply prefix the command to run withPYTHONPATH=..TrainingFast setup: training DINOv2 ViT-L/16 on ImageNet-1kRun DINOv2 training on 4 A100-80GB nodes (32 GPUs) in a SLURM cluster environment with submitit:pythondinov2/run/train/train.py\--nodes4\--config-filedinov2/configs/train/vitl16_short.yaml\--output-dir<PATH/TO/OUTPUT/DIR>\train.dataset_path=ImageNet:split=TRAIN:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>Training time is approximately 1 day and the resulting checkpoint should reach 81.6% on k-NN eval and 82.9% on linear eval.The training code saves the weights of the teacher in theevalfolder every 12500 iterations for evaluation.Long setup: training DINOv2 ViT-L/14 on ImageNet-22kRun DINOv2 training on 12 A100-80GB nodes (96 GPUs) in a SLURM cluster environment with submitit:pythondinov2/run/train/train.py\--nodes12\--config-filedinov2/configs/train/vitl14.yaml\--output-dir<PATH/TO/OUTPUT/DIR>\train.dataset_path=ImageNet22k:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>Training time is approximately 3.3 days and the resulting checkpoint should reach 82.0% on k-NN eval and 84.5% on linear eval.The training code saves the weights of the teacher in theevalfolder every 12500 iterations for evaluation.EvaluationThe training code regularly saves the teacher weights. In order to evaluate the model, run the following evaluation on a single node:k-NN classification on ImageNet-1kpythondinov2/run/eval/knn.py\--config-file<PATH/TO/OUTPUT/DIR>/config.yaml\--pretrained-weights<PATH/TO/OUTPUT/DIR>/eval/training_24999/teacher_checkpoint.pth\--output-dir<PATH/TO/OUTPUT/DIR>/eval/training_24999/knn\--train-datasetImageNet:split=TRAIN:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>\--val-datasetImageNet:split=VAL:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>Logistic regression classification on ImageNet-1kpythondinov2/run/eval/log_regression.py\--config-file<PATH/TO/OUTPUT/DIR>/config.yaml\--pretrained-weights<PATH/TO/OUTPUT/DIR>/eval/training_24999/teacher_checkpoint.pth\--output-dir<PATH/TO/OUTPUT/DIR>/eval/training_24999/logreg\--train-datasetImageNet:split=TRAIN:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>\--val-datasetImageNet:split=VAL:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>Linear classification with data augmentation on ImageNet-1kpythondinov2/run/eval/linear.py\--config-file<PATH/TO/OUTPUT/DIR>/config.yaml\--pretrained-weights<PATH/TO/OUTPUT/DIR>/eval/training_24999/teacher_checkpoint.pth\--output-dir<PATH/TO/OUTPUT/DIR>/eval/training_24999/linear\--train-datasetImageNet:split=TRAIN:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>\--val-datasetImageNet:split=VAL:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>We release the weights from evaluating the different models:modelImageNettop-1linear evaluationViT-S/14 distilled81.1%linear head weightsViT-B/14 distilled84.5%linear head weightsViT-L/14 distilled86.3%linear head weightsViT-g/1486.5%linear head weightsThe performance of the provided pretrained model weights can be evaluated as follows on ImageNet-1k:pythondinov2/run/eval/linear.py\--config-filedinov2/configs/eval/vitg14_pretrain.yaml\--pretrained-weightshttps://dl.fbaipublicfiles.com/dinov2/dinov2_vitg14/dinov2_vitg14_pretrain.pth\--train-datasetImageNet:split=TRAIN:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>\--val-datasetImageNet:split=VAL:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>LicenseDINOv2 code and model weights are released under the CC-BY-NC 4.0 license. SeeLICENSEfor additional details.ContributingSeecontributingand thecode of conduct.Citing DINOv2If you find this repository useful, please consider giving a star :star: and citation :t-rex::@misc{oquab2023dinov2,
  title={DINOv2: Learning Robust Visual Features without Supervision},
  author={Oquab, Maxime and Darcet, Timothée and Moutakanni, Theo and Vo, Huy V. and Szafraniec, Marc and Khalidov, Vasil and Fernandez, Pierre and Haziza, Daniel and Massa, Francisco and El-Nouby, Alaaeldin and Howes, Russell and Huang, Po-Yao and Xu, Hu and Sharma, Vasu and Li, Shang-Wen and Galuba, Wojciech and Rabbat, Mike and Assran, Mido and Ballas, Nicolas and Synnaeve, Gabriel and Misra, Ishan and Jegou, Herve and Mairal, Julien and Labatut, Patrick and Joulin, Armand and Bojanowski, Piotr},
  journal={arXiv:2304.07193},
  year={2023}
}