Package: bregmanet
BregmaNet : Bregman Neural NetworksBregmaNetis a PyTorch library providing multipleBregman Neural Networks.
To date, implemented models cover Bregman variants of multi-layer perceptrons and various residual networks.Contributor:Jordan Frécon (INSA Rouen Normandy, France)Table of ContentsRequirements and InstallationGetting StartedList of Supported ModelsCitationContribution and AcknowledgmentsRequirements and Installation:clipboard: RequirementsPyTorch version >=1.7.1Python version >=3.6Torchvision version >=0.8.2:hammer: Installationpip install bregmanetIn development versions can be foundhere.Getting Started:warning: PrecautionsAll images should be scaled within the domain range of the activation function.MLP models provided work only for 1-dimensional data inputs.MLP models are designed without a softmax final layer.All models need to be trained first. If you wish to provide your pretrained models, pleasecontribute.:rocket: DemosMultiple demo files can be foundtherein thedemosfolder. It contains:demo_toy_mlp.py: training of MLP on the Two-spiral toy dataset.demo_mnist_mlp.py: training of MLP on the MNIST dataset.demo_cifar10_resnet.py: training of ResNet20 on the CIFAR-10 dataset.demo_cifar100_resnet.py: training of ResNet20 on the CIFAR-100 dataset.demo_imagenet_resnet.py: training of ResNet18 on the ImageNet dataset.:page_with_curl: Loading a ModelTo date, all Bregman neural models provided are not trained.
If needed, a training procedure is made availabletherein thedemos/utilsfolder.
In order to load a model, proceed as follows.Multi-Layer PerceptronsFor asigmoid-based MLP witha linear input accepting 1d tensors of size 10243 hidden layers of size (1024, 1024, 512)a linear output layer mapping to 1d tensors of size 10importbregmanetmodel=bregmanet.MLP(activation='sigmoid',num_neurons=[1024,1024,512],input_dim=1024,output_dim=10)ResNetFor a BregmanResNet20 with SoftPlus activation function:importbregmanetmodel=bregmanet.bresnet20(activation='softplus')List of Supported ModelsThe following list reports all models currently supporting a Bregman variant.
If you have any issue with one of them or wish to provide your own, pleasecontact us.MLPResNet18ResNet20ResNet32ResNet34ResNet44ResNet56ResNet101Resnet110ResNet152Resnet1202ResNeXt50_32x4dResNeXt101_32x8dWideResNet50_2WideResnet101_2CitationIf you use this package, please cite the following work:@inproceedings{2022_Frecon_J_p-icml_bregmanet,
  title = {{Bregman Neural Networks}},
  author = {Frecon, Jordan and Gasso, Gilles and Pontil, Massimiliano and Salzo, Saverio},
  url = {https://hal.archives-ouvertes.fr/hal-03132512},
  series    = {Proceedings of Machine Learning Research},
  booktitle = {Proceedings of the 39th International Conference on Machine Learning,
               {ICML} 2022, 17-23 July 2022, Baltimore, USA},
  year = {2022},
}Contribution and AcknowledgmentsJordan Frecon would like to express his gratitude to theDepartment of Computational Statistics and Machine Learning(IIT, Genova, Italy) where part of this work was conducted during his postdoctoral position. The authors gratefully acknowledge the financial support of the French Agence Nationale de la Recherche (ANR), under grant ANR-20-CHIA-0021-01 (project RAIMO).The proposed BregmanResNets for CIFAR-10 are based on a rework of the ResNet implementation ofYerlan Idelbayev.
Other ResNet models are devised by hinging upon the official PyTorch/TorchVision repository. For more information, please refer to:ResNet:"Deep Residual Learning for Image Recognition"ResNeXt:"Aggregated Residual Transformation for Deep Neural Networks"WideResNet:"Wide Residual Networks"All kind of contributions are welcome, do not hesitate tocontact us!