Package: paddletransfer
PaddleTransferIntroductionPaddleTransfer, a transfer learning tool of PaddlePaddle, provides a variety of algorithms to transfer the knowledge learned from source tasks to target task. It enables users to use the state-of-the-art transfer learning algorithms on main-stream model archtechtures.PaddleTransfer provides various transfer learning algorithms, includingMMD(JMLR'12),L2-SP(ICML'18), DELTA(ICLR'19), RIFLE(ICML'20), Co-Tuning(NIPS'20)and supports many main-stream model archtechtures, includingResNet, MobileNet and ViT. With several lines of codes, users can apply these algorithms on our predifined model or their own model easily.ContentsKey HighlightsInstallationUsage GuidelineProvided AlgorithmsKey HighlightsHigh AccuracyPaddleTransfer provides various transfer learning algorithms, users can conveniently apply them on their models, and select the most appropriate one with high accuracy for further usage.Easy to UsePaddleTransfer provides unified API to invoke different algorithms, users can easily apply them on their models with several lines of codes.Fully SupportPaddleTransfer supports most commonly used models including Resnet, MobileNet and ViT, and will iterates rapidly to support more model archtectures.InstallationInstall by pippython -m pip install paddletransferDependenciesif you want to use our package in your own code, the following dependencies are requiredpython >= 3.7numpy >= 1.21paddlepaddle >= 2.2 (with suitable CUDA and cuDNN version)If you want to run our demo script, please make sure the following packages are installed correctly on your machine.visualdlUsage GuidelineQuick StartUsers can run our demo code for a quick startpython finetune.py --name [experiment_name] --train_dir [path_to_train_dir] --eval_dir [path_to_eval_dir] --model_arch [model_archtechture] --algo [finetune_algorithm] --gpu [device_for_experiment]For model_arch argument, please choose one from "resnet18", "resnet34", "resnet50", "resnet101", "resnet152", "mobilenet_v2" and "vit", mistyping may lead to unexpected behavior.If you want to finetune the ViT model, please make sure you have set the configuration file and pretrained parameters file correctly, and remember to add the corresponding argumengts(--cfg and --model_path) in your command. You can get the configuration file and pretrained model fromPaddleVit.Use PaddleTransfer in Your Own CodeImport dependenciesfrom paddletransfer.finetuning.img_cls import *We strongly recommand you to use the model architechtures we provide inbackbonesby following codes. The parameters setting are the same as the implementations inpaddle.vision.models, remember to setpretrained=Truefor finetuning.from backbones import resnet18, resnet34, resnet50, resnet101, resnet152, mobilenet_v2, build_vitYou can only import the model archtechtures that you need.If you want to use ViT model, please put the config.py file either fromour projectorPaddleVitunder your working directory and set other files correctly as described inQuick StartIf you want to use your self-defined model, please make sure that the name of layers are consistent with the implementations inpaddle.vision.modelsand your model has two outputs: The first one is the original output of the network and the second one is the features(the intermediate output before average pooling layer and fc layer).Initialize the algorithm objectalgo = FinetuneDELTA(model, model_arch)To customize the hyperparameter for finetune algorithm, please add the following arguments to the initializer, for details about hyperparameter setting, please refer toprovided algorithmsconfs = confs # a directory of the hyperparameter settingGet the parameters list which need to update and pass them to optimizerparams = algo.params()
...
optimizer = paddle.optimizer.Optimizer(...,parameters = params,...)In most cases, it is equal to model.parameters() and you may not want to invoke this method. But remember to do this if you are using Co-tuning algorithm since it has extra parameters to updateGet the regularization loss and merge it to the classification lossloss_cls = paddle.nn.CrossEntropyLoss(y_data,logits)
loss_all = {'loss_cls': loss_cls}
loss_reg = algo.loss(x_data, y_data, logits, features, epoch, batch_id)
loss_all.update(loss_reg)
loss = sum(loss_all.values())
...
loss.backward()Provided AlgorithmsSo far we have provided 5 algorithms for finetune, which areMMD, L2-SP, DELTA, RIFLE and Co-Tuning. If you do not want to use any finetune algorithms, just use the following code for vanilla finetune, the corresponding code for invoking different algorithms are in the respective sections.algo = FinetuneBASE(model,model_arch)MMDUse the following code for invoking MMD algorithmalgo = FinetuneMMD(model,model_arch,confs=_confs)The default hyperparameters for MMD algorithms is as follows, if you want to modify them, please pass your own confs object to the initializer._confs = {'reg_weight': 0.1, 'kernel_mul': 2.0, 'kernel_num': 5}L2-SPUse the following code for invoking L2-SP algorithmalgo = FinetuneL2SP(model,model_arch,confs=_confs)The default hyperparameters for L2SP algorithms is as follows, if you want to modify them, please pass your own confs object to the initializer._confs = {'reg_weight': 0.01}DELTAUse the following code for invoking DELTA algorithmalgo = FinetuneDELTA(model,model_arch,confs=_confs)The default hyperparameters for DELTA algorithms is as follows, if you want to modify them, please pass your own confs object to the initializer._confs = {'reg_weight': 0.01}RIFLEUse the following code for invoking RIFLE algorithmalgo = FinetuneRIFLE(model,model_arch,confs=_confs)The default hyperparameters for RIFLE algorithms is as follows, if you want to modify them, please pass your own confs object to the initializer._confs = {'reinit_epochs': [10,20,30]} # with total epochs = 40Co-TuningUse the following code for invoking Co-Tuning algorithm. The data_loader is used for relationship learning.algo = FinetuneCOT(model, model_arch, data_loader=data_loader, confs=_confs)If you want to use Co-Tuning algorithm on ViT, use the following code. The vit_config is used for auxillary imagenet classifier construction. Please make sure that your vit_config object passed to the initializer has the same structure with the official implementation inPaddleVit, you can use the get_config() function they provide to generate one easily.algo = FinetuneCOT(model, model_arch, data_loader=data_loader, vit_config=config, confs=_confs)The default hyperparameters for Co-Tuning algorithms is as follows, if you want to modify them, please pass your own confs object to the initializer._confs = {'reg_weight': 2.3}Algorithm PerformanceWe have conducted some experiments on several dataset(CUB_200_2011, indoorCVPR_09 and dtd) using algorithms provided by PaddleTransfer. Most of experiments use the default hyper parameter setting in finetune.py, except Co-Tuning. For Co-Tuning, We use default hyper parameters on bird classification task, lr = 0.002 and wd = 0.0005 on scene classification task, lr = 0.001 and wd = 0.0005 on texture classification task. The outcomes are as follows.BirdSceneTextureBASE80.2577.3462.07MMD80.4178.2763.15L2SP80.3977.5765.34DELTA81.0478.7967.47RIFLE80.7478.0863.15CoTuning81.0778.0069.39