Package: domainlab
DomainLab: modular python package for training domain invariant neural networksDistribution shifts, domain generalization and DomainLabNeural networks trained using data from a specific distribution (domain) usually fails to generalize to novel distributions (domains). Domain generalization aims at learning domain invariant features by utilizing data from multiple domains (data sites, corhorts, batches, vendors) so the learned feature can generalize to new unseen domains (distributions).DomainLab is a software platform with state-of-the-art domain generalization algorithms implemented, designed by maximal decoupling of different software components thus enhances maximal code reuse.DomainLabDomainLab decouples the following concepts or objects:task $M$: In DomainLab, a task is a container for datasets from different domains. (e.g. from distribution $D_1$ and $D_2$). Task offer a static protocol to evaluate the generalization performance of a neural network: which dataset(s) is used for training, wich dataset(s) used for testing.neural network: a map $\phi$ from the input data to the feature space and a map $\varphi$ from feature space to output $\hat{y}$ (e.g. decision variable).model: structural risk in the form of $\ell() + \mu R()$  where$\ell(Y, \hat{y}=\varphi(\phi(X)))$ is the task specific empirical loss (e.g. cross entropy for classification task).$R(\phi(X))$ is the penalty loss to boost domain invariant feature extraction using $\phi$.$\mu$ is the corresponding multiplier to each penalty function factor.trainer:  an object that guides the data flow to model and append further domain invariant losses
like inter-domain feature alignment.We offer detailed documentation on how these models and trainers work in our documentation page:https://marrlab.github.io/DomainLab/DomainLab makes it possible to combine models with models, trainers with models, and trainers with trainers in a decorator pattern like line of codeTrainer A(Trainer B(Model C(Model D(network E), network E, network F)))which correspond to $\ell() + \mu_a R_a() + \mu_b R_b + \mu_c R_c() + \mu_d R_d()$, where Model C and Model D share neural network E, but Model C has an extra neural network F. All models share the same neural network for feature extraction, but can have different auxilliary networks for $R()$.Getting startedInstallationFor development version in Github, seeInstallation and Dependencies handlingWe also offer a PyPI version herehttps://pypi.org/project/domainlab/which one could install viapip install domainlaband it is recommended to create a virtual environment for it.Task specificationWe offer various ways for the user to specify a scenario to evaluate the generalization performance via training on a limited number of datasets. See detail inTask SpecificationExample and usageCommand linedomainlab -c ./examples/conf/vlcs_diva_mldg_dial.yaml(if you install via pip)or if you clone this the code repository for DomainLabpython main_out.py -c ./examples/conf/vlcs_diva_mldg_dial.yamlwhere the configuration file below can be downloadedherete_d: caltech                       # domain name of test domain
tpath: examples/tasks/task_vlcs.py  # python file path to specify the task
bs: 2                               # batch size
model: dann_diva                    # combine model DANN with DIVA
epos: 1                             # number of epochs
trainer: mldg_dial                  # combine trainer MLDG and DIAL
gamma_y: 700000.0                   # hyperparameter of diva
gamma_d: 100000.0                   # hyperparameter of diva
npath: examples/nets/resnet.py      # neural network for class classification
npath_dom: examples/nets/resnet.py  # neural network for domain classificationSee details inCommand line usageor Programm against DomainLab APISee example here:Transformer as feature extractor, decorate JIGEN with DANN, training using MLDG decorated by DIALBenchmark different methodsDomainLab provides a powerful benchmark functionality.
To benchmark several algorithms(combination of neural networks, models, trainers and associated hyperparameters), a single line command along with a benchmark configuration files is sufficient. See details inbenchmarks documentation and tutorialOne could simply runbash run_benchmark_slurm.sh your_benchmark_configuration.yamlto launch different experiments with specified configuraiton.For example,  the following result (without any augmentation like flip) is for PACS dataset using ResNet.Benchmark results plot generated from DomainLab, where each rectangle represent one model trainer combination, each bar inside the rectangle represent a unique hyperparameter index associated with that method combination, each dot represent a random seeds.Temporary citation@manual{domainlab,title={{DomainLab: modular python package for training domain invariant neural networks}},author={{Xudong Sun, et.al.}},organization={{Institute of AI for Health}},year={2023},url={https://github.com/marrlab/DomainLab},note={temporary citation for domainlab}}