Package: vitcifar10
DescriptionThis repository contains the Python packagevitcifar10, which is a Vision Transformer (ViT) baseline code for training and testing onCIFAR-10. This implementation only supports CUDA (no CPU training).The idea of this repository is not to have a very flexible implementation, but one that can be used as a baseline for research with results on testing close to the state of the art.The code in this repository relies ontimm, which is a Python package with many state-of-the-art models implemented and pretrained.Use within a Docker containerIf you do not have Docker,hereyou can find a tutorial to install it.Buildvitcifar10Docker image:$gitclonehttps://github.com/luiscarlosgph/vitcifar10.git
$cdvitcifar10/docker
$dockerbuild-tvitcifar10.Launchvitcifar10container:$dockerrun--namewild_vitcifar10--runtime=nvidia-v/dev/shm:/dev/shmvitcifar10:latest&Get a terminal inside the container (you can execute this command multiple times to get multiple container terminals):$dockerexec-itwild_vitcifar10/bin/zsh
$cd$HOMELaunch CIFAR-10 training:$ python3 -m vitcifar10.train --lr 1e-4 --opt adam --nepochs 200 --bs 16 --cpdir checkpoints --logdir logs --cpint 5 --data ./dataLaunch CIFAR-10 testing:$ python3 -m vitcifar10.test --data ./data --resume checkpoints/model_best.pt --bs 1If you want to kill the container run$ docker kill wild_vitcifar10.To remove it execute$ docker rm wild_vitcifar10.Install with pip$pipinstallvitcifar10--userInstall from source$gitclonehttps://github.com/luiscarlosgph/vitcifar10.git
$cdvitcifar10
$python3setup.pyinstallDownload CIFAR-10 training and testing data$python3-mvitcifar10.download_data--data./dataTrain on CIFAR-10Launch training:$python3-mvitcifar10.train--lr1e-4--optadam--nepochs200--bs16--cpdircheckpoints--logdirlogs--cpint5--data./dataResume training from a checkpoint:$python3-mvitcifar10.train--lr1e-4--optadam--nepochs200--bs16--cpdircheckpoints--logdirlogs--cpint5--data./data--resumecheckpoints/epoch_21.ptOptions:--lr: learning rate.--opt: optimizer, choose eithersgdoradam.--nepochs: number of epochs to execute.--bs: training batch size.--cpdir: checkpoint directory.--logdir: path to the directory where the Tensorboard logs will be saved.--cpint: interval of epochs to save a checkpoint of the training process.--resume: path to the checkpoint file you want to resume.--data: path to the directory where the dataset will be stored.--seed: fix random seed for reproducibility.Launch Tensorboard:$python3-mtensorboard.main--logdirlogs--bind_allTrain multiple models from scratch (different random seeds)$python3-mvitcifar10.run--lr1e-4--optadam--nepochs100--bs16--cpdircheckpoints--logdirlogs--cpint5--niter5--datadataOptions:--niter: number of training cycles to run, e.g.--niter 5will lead to training five networks.The rest of the options are identical to those ofvitcifar10.train.Train with a custom dataloaderThis is useful when you want to experiment with the input/labels without modifying the model. To do so,vitcifar10.runaccepts the parameter--data-loader, e.g.:$python3-mvitcifar10.run--lr1e-4--optadam--nepochs100--bs16--cpdircheckpoints--logdirlogs--cpint10--niter5--datadata--data-loader<python_class_name>The dataloader class that you pass must align with the following requirements:Constructor__init__should accept the batch size as first parameter.Methods__next__and__iter__should be implemented.Test on CIFAR-10$python3-mvitcifar10.test--data./data--resumecheckpoints/model_best.pt--bs1Options:--data: path to the directory where the dataset will be stored.--resume: path to the checkpoint file you want to test.Perform inference on a single imageAfter training, you can classify images such as thisdogor thiscatfollowing:$python3-mvitcifar10.inference--imagedata/dog.jpg--modelcheckpoints/model_best.ptItisadog!$python3-mvitcifar10.inference--imagedata/cat.jpg--modelcheckpoints/model_best.pt
Itisacat!Training | validation | testing splitsThis code usestorchvisionto download and load the CIFAR-10 dataset. The constructor of the classtorchvision.datasets.CIFAR10has a boolean parameter calledtrain.In our code we settrain=Trueto obtain the images for training and validation, using 90% fortraining(45K images) and 10% forvalidation(5K images). The validation set is used to discover the best model during training (could also be used for hyperparameter tunning or early stopping). Fortesting, we settrain=False. The testing set contains 10K images.AuthorLuis C. Garcia Peraza Herrera (luiscarlos.gph@gmail.com).LicenseThis repository is shared under anMIT license.