Package: bcos
B-cos Networks v2arXiv|codeB-cos Alignment for Inherently Interpretable CNNs and Vision TransformersMoritz Böhle, Navdeeppal Singh, Mario Fritz, Bernt SchieleTable of ContentsIntroductionQuick StartInstallationbcosPackageTraining Environment SetupSetting Data PathsUsageEvaluationTrainingModel ZooLicenseIntroductionThis repository contains the code for the B-cos v2 models.These models are more efficient and easier to train than theoriginal v1 B-cos models. Furthermore, we make a
large number of pretrained B-cos models available for use.If you want to take a quick look at the explanations the models generate,
you can try out the Gradio web demo on.If you prefer a more hands-on approach,
you can take a look at thedemo notebook onColabor load the models directly viatorch.hubas explained below.If you simply want to copy the model definitions, we provide a minimal, single-file reference implementation including
explanation mode inextra/minimal_bcos_resnet.py!UPDATE: We have also released our ViT models! SeeModel Zoo.Quick StartYou only need to make sure you havetorchandtorchvisioninstalled.Then, loading the models viatorch.hubis as easy as:importtorch# list all available modelstorch.hub.list('B-cos/B-cos-v2')# load a pretrained modelmodel=torch.hub.load('B-cos/B-cos-v2','resnet50',pretrained=True)Inference and explanation visualization is as simple as:fromPILimportImageimportmatplotlib.pyplotasplt# load imageimg=model.transform(Image.open('cat.jpg'))img=img[None].requires_grad_()# predict and explainmodel.eval()expl_out=model.explain(img)print("Prediction:",expl_out["prediction"])# predicted class idxplt.imshow(expl_out["explanation"])plt.show()Each of the models has its inference transform attached to it, accessible viamodel.transform.
Furthermore, each model has a.explain()method that takes an image tensor and returns a dictionary
containing the prediction and the explanation, and some extras.See thedemo notebookfor more details on the.explain()method.Furthermore, each model has aget_classifierandget_feature_extractormethod that return the
classifier and feature extractor modules respectively. These can useful for fine-tuning the models!InstallationDepending on your use case, you can either install thebcospackage
or set up the development environment for training the models (for your custom models or for reproducing the results).bcosPackageIf you are simply interested in using the models (pretrained or otherwise),
then we provide abcospackage that can be installed viapip:pipinstallbcosThis contains the models, their modules, transforms, and other utilities
making it easy to use and build B-cos models.
Take a look at the public APIhere.
(I'll add a proper docs site if I have time or there's enough interest.
Nonetheless, I have tried to keep the code well-documented, so it should be easy to follow.)Training Environment SetupIf you want to train your own B-cos models using this repository or are interested in reproducing the results,
you can set up the development environment as follows:Usingconda(recommended, especially if you want to reproduce the results):condaenvcreate-fenvironment.yml
condaactivatebcosUsingpippipinstall-rrequirements-train.txtSetting Data PathsYou can either set the paths inbcos/settings.pyor set the environment variablesDATA_ROOTIMAGENET_PATHto the paths of the data directories.TheDATA_ROOTenvironment variable should point to the data root directory for CIFAR-10
(will be automatically downloaded).
For ImageNet, theIMAGENET_PATHenvironment variable should point to the directory containing
thetrainandvaldirectories.UsageFor thebcospackage, as mentioned earlier, take a look at the public APIhere.For evaluating or training the models, you can use theevaluate.pyandtrain.pyscripts, as follows:EvaluationYou can use evaluate the accuracy of the models on the ImageNet validation set using:pythonevaluate.py--datasetImageNet--hubconfresnet18This will download the model fromtorch.huband evaluate it on the ImageNet validation set.
The default batch size is 1, but you can change it using the--batch-sizeargument.
Replaceresnet18with any of the other models listed inModel Zoothat you wish to evaluate.TrainingShort version:pythontrain.py\--datasetImageNet\--base_networkbcos_final\--experiment_nameresnet18Long version: SeeTRAINING.mdfor more details on
how the setup works and how to train your own models.Model ZooHere are the ImageNet pre-trained models available in the model zoo.
You can find the links to the model weights below
(uploaded to theWeightsGitHub release).Model/EntrypointTop-1 AccuracyTop-5 Accuracy#ParamsDownloadresnet1868.736%87.430%11.69Mlinkresnet3472.284%90.052%21.80Mlinkresnet5075.882%92.528%25.52Mlinkresnet10176.532%92.538%44.50Mlinkresnet15276.484%92.398%60.13Mlinkresnext50_32x4d75.820%91.810%25.00Mlinkdensenet12173.612%91.106%7.95Mlinkdensenet16176.622%92.554%28.58Mlinkdensenet16975.186%91.786%14.08Mlinkdensenet20175.480%91.992%19.91Mlinkvgg11_bnu69.310%88.388%132.86Mlinkconvnext_tiny77.488%93.192%28.54Mlinkconvnext_base79.650%94.614%88.47Mlinkconvnext_tiny_bnu76.826%93.090%28.54Mlinkconvnext_base_bnu80.142%94.834%88.47Mlinkdensenet121_long77.302%93.234%7.95Mlinkresnet50_long79.468%94.452%25.52Mlinkresnet152_long80.144%94.116%60.13Mlinksimple_vit_ti_patch16_22459.960%81.838%5.80Mlinksimple_vit_s_patch16_22469.246%88.096%22.28Mlinksimple_vit_b_patch16_22474.408%91.156%86.90Mlinksimple_vit_l_patch16_22475.060%91.378%178.79Mlinkvitc_ti_patch1_1467.260%86.774%5.32Mlinkvitc_s_patch1_1474.504%91.288%20.88Mlinkvitc_b_patch1_1477.152%92.926%81.37Mlinkvitc_l_patch1_1477.782%92.966%167.44Mlinkstandard_simple_vit_ti_patch16_22470.230%89.380%5.67Mlinkstandard_simple_vit_s_patch16_22474.470%91.226%21.96Mlinkstandard_simple_vit_b_patch16_22475.300%91.026%86.38Mlinkstandard_simple_vit_l_patch16_22475.710%90.050%178.10Mlinkstandard_vitc_ti_patch1_1472.590%90.788%5.33Mlinkstandard_vitc_s_patch1_1475.756%91.994%20.91Mlinkstandard_vitc_b_patch1_1476.790%92.024%81.39Mlinkstandard_vitc_l_patch1_1477.866%92.298%167.54MlinkYou can find these entrypoints inbcos/models/pretrained.py.LicenseThis repository's code is licensed under the Apache License 2.0
which you can find in theLICENSEfile.The pre-trained models are trained on ImageNet (and are hence derived from it), which is
licensed under theImageNet Terms of access,
which among others things, only allows non-commercial use of the dataset.
It is therefore your responsibility to check whether you have permission to use the
pre-trained models foryouruse case.