Package: crainets
CRAI-NetsThe CRAI-Nets ProjectThis is just another model-zoo and utility library combined for developing deep learning models. The main reasons for this project to exist is to avoid boilerplate code across projects, letting others tap in on your work, making benchmarking/expermenting easy and fast while also sticking to readibility and reproducibility. The goal of the project is to include as many useful models as possible and also smart customized metrics and loss functions. The project, as of now, is aimed towards computer vision, although contribution within NLP or RL is more than welcome.Getting started0. RequirementsThe library is platform agnostic although we strongly suggest to use Linux or Mac for ML development. We also suggest to usepoetryorpyenvfor dependency management unless you are on Win where Conda is the defacto(satans speed to you). Make sure to have python version 3.8 or later installed.1. Install the packageAs recommended, use poetry to install the package by running:$ poetry add crainets2. What you need to considerThe Trainer class you can use for simple benchmarking or fast expermenting expects mainly the following:A model configuration dict containing hyperparametersA dict containing your loss functionsA dict containing your metrics (you can specify multiple)Train and test data that you should prep in dataloader class that inherits from the pytorchdatasetclassThe model architecture imported from crainets model-zooWe suggest to write your code modular such that configurations come from aconfig.pyscript and the dataloader comes from adataloader.pyscript.3. ExampleLets write up two dataloaders that will lazy evaluate our data durng runtime when its batched for training. Cifar10 is used in this example and the only reason why is for brevity.importtorchimporttorchvisionimporttesting.configasconfigimporttorch.utils.dataasdata_utilstransform=torchvision.transforms.Compose([torchvision.transforms.ToTensor(),torchvision.transforms.RandomHorizontalFlip(),torchvision.transforms.Normalize((0.5,0.5,0.5),(0.5,0.5,0.5))])transform_test=torchvision.transforms.Compose([torchvision.transforms.ToTensor(),torchvision.transforms.Normalize((0.5,0.5,0.5),(0.5,0.5,0.5))])train=torchvision.datasets.CIFAR10(config.DATA_PATH,train=True,download=True,transform=transform)test=torchvision.datasets.CIFAR10(config.DATA_PATH,train=False,download=True,transform=transform_test)train_loader=torch.utils.data.DataLoader(train,batch_size=config.batch_size_train,shuffle=True)test_loader=torch.utils.data.DataLoader(test,batch_size=config.batch_size_test,shuffle=True)Now that we have our data, lets write up a config dict for our network to use.importosimporttorchROOT=os.getcwd()DATA_PATH=os.path.join('/data')CHCKPT=os.path.join('/checkpoints')batch_size_train=100batch_size_test=50TRAIN_CONFIG={"n_gpu":1,"optimizer":{"type":"Adam","args":{"lr":1e-3,"weight_decay":0,"amsgrad":True}},"loss":"nll_loss","metrics":["accuracy","top_k_acc"],"lr_scheduler":{"type":"StepLR","args":{"step_size":500,"gamma":0.1}},"trainer":{"epochs":2,"iterative":False,"iterations":5,"images_pr_iteration":100,"val_images_pr_iteration":10,"save_dir":CHCKPT,"save_period":5,"early_stop":1}}METRICS={'CrossEntropy':torch.nn.CrossEntropyLoss()}Note that we also included METRICS as a config in the script. We could define many more metrics in the dict than what is written in the example.Now lets tie it all together in a controller script for running the network. We are going to use the sexyefficient-netin this example.# Internal importsfromdata_loaderimporttrain_loader,test_loaderfromconfigimportconfig# CRAI-Nets importsfromcrainets.trainer.trainerimportTrainerfromcrainets.models.efficientnetimportEfficientNetfromcrainets.essentials.multi_lossimportMultiLossfromcrainets.essentials.multi_metricimportMultiMetric# specifiy the needed configmodel=EfficientNet.from_name(in_channels=3,num_classes=10,model_name='efficientnet-b0')loss=[(1,torch.nn.CrossEntropyLoss())]loss=MultiLoss(losses=loss)# Add metrics in the metrics dict from the config filemetrics=MultiMetric(config.METRICS)# Instantiate zhe classtrainer=Trainer(model=model,loss_function=loss,metric_ftns=metrics,config=config.TRAIN_CONFIG,data_loader=train_loader,valid_data_loader=test_loader,seed=666,accumulative_metrics=True)# Gut gut! Now run the network training und zmile!trainer.train()The project is mainly developed and maintained by CRAI at the university hospital of Oslo