Package: torch-salad
🥗 salad========**S**\ emi-supervised **A**\ daptive **L**\ earning **A**\ cross **D**\ omains.. figure:: img/domainshift.png:alt:``salad`` is a library to easily setup experiments using the currentstate-of-the art techniques in domain adaptation. It features several ofrecent approaches, with the goal of being able to run fair comparisonsbetween algorithms and transfer them to real-world use cases. Thetoolbox is under active development and will extended when newapproaches are published.Contribute on Github: `https://github.com/domainadaptation/salad`_Currently implements the following techniques (in ``salad.solver``)-  VADA (``VADASolver``),`arxiv:1802.08735 <https://arxiv.org/abs/1802.08735>`__-  DIRT-T (``DIRTTSolver``),`arxiv:1802.08735 <https://arxiv.org/abs/1802.08735>`__-  Self-Ensembling for Visual Domain Adaptation(``SelfEnsemblingSolver``)`arxiv:1706.05208 <https://arxiv.org/abs/1706.05208>`__-  Associative Domain Adaptation (``AssociativeSolver``),`arxiv:1708.00938 <https://arxiv.org/pdf/1708.00938.pdf>`__-  Domain Adversarial Training (``DANNSolver``),`jmlr:v17/15-239.html <http://jmlr.org/papers/v17/15-239.html>`__-  Generalizing Across Domains via Cross-Gradient Training(``CrossGradSolver``),`arxiv:1708.00938 <http://arxiv.org/abs/1804.10745>`__-  Adversarial Dropout Regularization (``AdversarialDropoutSolver``),`arxiv.org:1711.01575 <https://arxiv.org/abs/1711.01575>`__Implements the following features (in ``salad.layers``):-  Weights Ensembling using Exponential Moving Averages or StoredWeights-  WalkerLoss and Visit Loss(`arxiv:1708.00938 <https://arxiv.org/pdf/1708.00938.pdf>`__)-  Virtual Adversarial Training(`arxiv:1704.03976 <https://arxiv.org/abs/1704.03976>`__)Coming soon:-  Deep Joint Optimal Transport (``DJDOTSolver``),`arxiv:1803.10081 <https://arxiv.org/abs/1803.10081>`__-  Translation based approaches📊 Benchmarking Results----------------------One of salad's purposes is to constantly track the state of the art of a variety of domainadaptation algorithms. The latest results can be reproduced by the files in the ``scripts/``directory... figure:: img/benchmarks.svg:alt:💻 Installation---------------Requirements can be found in ``requirement.txt`` and can be installedvia.. code:: bashpip install -r requirements.txtInstall the package via.. code:: bashpip install torch-saladFor the latest development version, install via.. code:: bashpip install git+https://github.com/bethgelab/domainadaptation📚 Using this library---------------------Along with the implementation of domain adaptation routines, thislibrary comprises code to easily set up deep learning experiments ingeneral.This section will be extended upon pre-release.Quick Start~~~~~~~~~~~To get started, the ``scripts/`` directory contains several python scriptsfor both running replication studies on digit benchmarks and studies ona different dataset (toy example: adaptation to noisy images)... code:: bash$ cd scripts$ python train_digits.py --log ./log --teach --source svhn --target mnistRefer to the help pages for all options:.. code::usage: train_digits.py [-h] [--gpu GPU] [--cpu] [--njobs NJOBS] [--log LOG][--epochs EPOCHS] [--checkpoint CHECKPOINT][--learningrate LEARNINGRATE] [--dryrun][--source {mnist,svhn,usps,synth,synth-small}][--target {mnist,svhn,usps,synth,synth-small}][--sourcebatch SOURCEBATCH] [--targetbatch TARGETBATCH][--seed SEED] [--print] [--null] [--adv] [--vada][--dann] [--assoc] [--coral] [--teach]Domain Adaptation Comparision and Reproduction Studyoptional arguments:-h, --help            show this help message and exit--gpu GPU             Specify GPU--cpu                 Use CPU Training--njobs NJOBS         Number of processes per dataloader--log LOG             Log directory. Will be created if non-existing--epochs EPOCHS       Number of Epochs (Full passes through the unsupervisedtraining set)--checkpoint CHECKPOINTCheckpoint path--learningrate LEARNINGRATELearning rate for Adam. Defaults to Karpathy'sconstant ;-)--dryrun              Perform a test run, without actually training anetwork.--source {mnist,svhn,usps,synth,synth-small}Source Dataset. Choose mnist or svhn--target {mnist,svhn,usps,synth,synth-small}Target Dataset. Choose mnist or svhn--sourcebatch SOURCEBATCHBatch size of Source--targetbatch TARGETBATCHBatch size of Target--seed SEED           Random Seed--print--null--adv                 Train a model with Adversarial Domain Regularization--vada                Train a model with Virtual Adversarial DomainAdaptation--dann                Train a model with Domain Adversarial Training--assoc               Train a model with Associative Domain Adaptation--coral               Train a model with Deep Correlation Alignment--teach               Train a model with Self-EnsemblingReasons for using solver abstractions~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~The chosen abstraction style organizes experiments into a subclass of``Solver``.Quickstart: MNIST Experiment~~~~~~~~~~~~~~~~~~~~~~~~~~~~As a quick MNIST experiment:.. code:: pythonfrom salad.solvers import Solverclass MNISTSolver(Solver):def __init__(self, model, dataset, **kwargs):self.model = modelsuper().__init__(dataset, **kwargs)def _init_optims(self, lr = 1e-4, **kwargs):super()._init_optims(**kwargs)opt = torch.optim.Adam(self.model.parameters(), lr = lr)self.register_optimizer(opt)def _init_losses(self):passFor a simple tasks as MNIST, the code is quite long compared to otherPyTorch examples `TODO <#>`__.💡 Domain Adaptation Problems-----------------------------Legend: Implemented (✓), Under Construction (🚧)📷 Vision~~~~~~~~~-  Digits: MNIST ↔ SVHN ↔ USPS ↔ SYNTH (✓)-  `VisDA 2018 Openset and Detection <http://ai.bu.edu/visda-2018>`__(✓)-  Synthetic (GAN) ↔ Real (🚧)-  CIFAR ↔ STL (🚧)-  ImageNet to`iCubWorld <https://robotology.github.io/iCubWorld/#datasets>`__ (🚧)🎤 Audio~~~~~~~~-  `Mozilla Common Voice Dataset <https://voice.mozilla.org/>`__ (🚧)፨ Neuroscience~~~~~~~~~~~~~~-  White Noise ↔ Gratings ↔ Natural Images (🚧)-  `Deep Lab Cut Tracking <https://github.com/AlexEMG/DeepLabCut>`__ (🚧)🔗 References to open source software-------------------------------------Part of the code in this repository is inspired or borrowed fromoriginal implementations, especially:-  https://github.com/Britefury/self-ensemble-visual-domain-adapt-  https://github.com/Britefury/self-ensemble-visual-domain-adapt-photo/-  https://github.com/RuiShu/dirt-t-  https://github.com/gpascualg/CrossGrad-  https://github.com/stes/torch-associative-  https://github.com/haeusser/learning\_by\_association-  https://mil-tokyo.github.io/adr\_da/Excellent list of domain adaptation ressources: -https://github.com/artix41/awesome-transfer-learning👤 Contact----------Maintained by `Steffen Schneider <https://code.stes.io>`__. Work is partof my thesis project at the `Bethge Lab <http://bethgelab.org>`__. ThisREADME is also available as a webpage at`salad.domainadaptation.org <http://salad.domainadaptation.org>`__. Wewelcome issues and pull requests `to the official githubrepository <https://github.com/bethgelab/domainadaptation>`__.