Package: vision-transformers
vision_transformersA repository for everything Vision Transformers.Currently Supported ModelsImage ClassificationViT Base Patch 16 | 224x224: Torchvision pretrained weightsViT Base Patch 32 | 224x224: Torchvision pretrained weightsViT Tiny Patch 16 | 224x224: Timm pretrained weightsVit Tiny Patch 16 | 384x384: Timm pretrained weightsSwin Transformer Tiny Patch 4 Window 7 | 224x224: Official Microsoft weightsSwin Transformer Small Patch 4 Window 7 | 224x224: Official Microsoft weightsSwin Transformer Base Patch 4 Window 7 | 224x224: Official Microsoft weightsSwin Transformer Large Patch 4 Window 7 | 224x224: No pretrained weightsQuick SetupStable PyPi Packagepip install vision-transformersORLatest Git Updatesgit clone https://github.com/sovit-123/vision_transformers.git
cd vision_transformersInstallation in the environment of your choice:pip install .Importing Models and UsageIf you have you own training pipeline and just want the modelReplacenum_classes=1000with you own number of classes.fromvision_transformers.modelsimportvitmodel=vit.vit_b_p16_224(num_classes=1000,pretrained=True)# model = vit.vit_b_p32_224(num_classes=1000, pretrained=True)# model = vit.vit_ti_p16_224(num_classes=1000, pretrained=True)fromvision_transformers.modelsimportswin_transformermodel=swin_transformer.swin_t_p4_w7_224(num_classes=1000,pretrained=True)# model = swin_transformer.swin_s_p4_w7_224(num_classes=1000, pretrained=True)# model = swin_transformer.swin_b_p4_w7_224(num_classes=1000, pretrained=True)# model = swin_transformer.swin_l_p4_w7_224(num_classes=1000)If you want to use the training pipelineClone the repository:git clone https://github.com/sovit-123/vision_transformers.git
cd vision_transformersInstallpip install .From thevision_transformersdirectory:If you have no validation splitpython tools/train_classifier.py --data data/diabetic_retinopathy/colored_images/ 0.15 --epochs 5 --model vit_ti_p16_224In the above command:data/diabetic_retinopathy/colored_images/represents the data folder where the images will be inside the respective class folders0.15represents the validation split as the dataset does not contain a validation folderIf you have validation splitpython tools/train_classifier.py --train-dir data/plant_disease_recognition/train/ --valid-dir data/plant_disease_recognition/valid/ --epochs 5 --model vit_ti_p16_224In the above command:--train-dirshould be path to the training directory where the images will be inside their respective class folders.--valid-dirshould be path to the validation directory where the images will be inside their respective class folders.All Available Model Flags for--modelvit_b_p32_224
vit_ti_p16_224
vit_ti_p16_384
vit_b_p16_224
swin_b_p4_w7_224
swin_t_p4_w7_224
swin_s_p4_w7_224
swin_l_p4_w7_224DETR TrainingThe datasets annotations should be in XML format. The dataset (according to--dataflag) given in following can be found here =>https://www.kaggle.com/datasets/sovitrath/aquarium-datapython tools/train_detector.py --model detr_resnet50 --epochs 2 --data data/aquarium.yamlExamplesViT Base 16 | 224x224 pretrained fine-tuning on CIFAR10ViT Tiny 16 | 224x224 pretrained fine-tuning on CIFAR10DETR image inference notebookDETR video inference script(Fine Tuning Coming Soon) ---Check commands hereDETR Video Inference CommandsAll commands to be executed from the root project directory (vision_transformers)Using default video:python examples/detr_video_inference.pyUsing CPU only:python examples/detr_video_inference.py --device cpuUsing another video file:python examples/detr_video_inference.py --input /path/to/video/file