Package: fastda
FastDAIntroductionFastDA is a simple framework for domain adaptation training.It relies onMMCVvia borrowing a lot of useful tools and mechanisms (e.g.,Config,Registry,Hook). MMCV acts as a foundational library for computer vision research and supports many projects such asMMClassification,MMSegmentation,MMDetection.We hopeFastDAcan also support domain adaptation training for different research areas.Design ideaTo make FastDA more suitable for domain adaptation, we first review the differences between traditional supervised training and domain adaptation training.Training in mmcls/mmseg/mmdetTraining in domain adaptationDatasetsingle train datasetmultiple train datasetsModelsingle model with single optimizermultiple models (e.g., base model and domain classifier)sometimes different models has different optimizer parametersOptimizersingle loss (summation of all losses)call loss.backward() once in each iterationmultiple loss.backward() for calculating gradients ofdifferent modelsIt can be seen that the training process in domain adaptation are more complicated.1. Datasets and ModelsFor dataset and model which are the basic parts for training, we define a set of new rules when writing dataset and model configs.  By parsing the dataset config, we can get multiples training datasets. By parsing the model config, we can get aModuleDictwhere the key is an user defined name and the value contains a instantiated model and its optimizer and learning rate scheduler.2. Trainer and ValidatorFor the training and testing process, MMCV use theRunnerclass to control them. Two kinds of runners are provided, namelyEpochBasedRunnerandIterBasedRunner. The latter one is more natural for domain adaptation training since the number of samples in the source and target datasets are always different and it is hard to define an epoch. Actually, most methods report their implementation details based on iterations but not epochs.Instead of directly using IterBasedRunner where the whole training process are wrapped in a single top-level model's train_step function, we put interaction among different models in aTrainer. It can be regarded as aminimumimplementation of basic running process, which can take care of  text logging, tensorboard logging, checkpoint saving, resume training, scheduler step etc. This is achieved by registering predefinedhooksto trainer.Besides Trainer, we also introduceValidatorto control the validation (testing) process. There is not default hook for validator. When building your own project, you should create a task-specific evaluation hook and register it to validator.Note: For a more concise and easy implementation, we impose some restrictions as follows:We only consider training and testing based on GPU device.Use DistributedDataParallel instead of original model or DataParallel. More specifically, we use MMDistributedDataParallel in mmcv which is more consistent with the DataContainer.In the BaseTrainer class, we do not impose operations of optimizer such as optimizer.zero_grad() and optimizer.step(). You can either register OptimizerHook in your trainer's init function or manipulate it yourself.InstallationPrepare environment: Installpytorchandmmcv.pip3installtorch
pip3installmmcvNote: Since MMCV requires Python 3.6+, FastDA also maintains this requirement.Install FastDApip3installfastdaTrain procedureBuild your own projectLicenseThis project is released under theMIT License.