Package: peft-machinify
🤗 PEFTState-of-the-art Parameter-Efficient Fine-Tuning (PEFT) methodsParameter-Efficient Fine-Tuning (PEFT) methods enable efficient adaptation of pre-trained language models (PLMs) to various downstream applications without fine-tuning all the model's parameters. Fine-tuning large-scale PLMs is often prohibitively costly. In this regard, PEFT methods only fine-tune a small number of (extra) model parameters, thereby greatly decreasing the computational and storage costs. Recent State-of-the-Art PEFT techniques achieve performance comparable to that of full fine-tuning.Seamlessly integrated with 🤗 Accelerate for large scale models leveraging DeepSpeed and Big Model Inference.Supported methods:LoRA:LORA: LOW-RANK ADAPTATION OF LARGE LANGUAGE MODELSPrefix Tuning:Prefix-Tuning: Optimizing Continuous Prompts for Generation,P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and TasksP-Tuning:GPT Understands, TooPrompt Tuning:The Power of Scale for Parameter-Efficient Prompt TuningGetting startedfromtransformersimportAutoModelForSeq2SeqLMfrompeftimportget_peft_config,get_peft_model,LoraConfig,TaskTypemodel_name_or_path="bigscience/mt0-large"tokenizer_name_or_path="bigscience/mt0-large"peft_config=LoraConfig(task_type=TaskType.SEQ_2_SEQ_LM,inference_mode=False,r=8,lora_alpha=32,lora_dropout=0.1)model=AutoModelForSeq2SeqLM.from_pretrained(model_name_or_path)model=get_peft_model(model,peft_config)model.print_trainable_parameters()# output: trainable params: 2359296 || all params: 1231940608 || trainable%: 0.19151053100118282Use CasesGet comparable performance to full finetuning by adapting LLMs to downstream tasks using consumer hardwareGPU memory required for adapting LLMs on the few-shot datasetought/raft/twitter_complaints. Here, settings considered
are full finetuning, PEFT-LoRA using plain PyTorch and  PEFT-LoRA using DeepSpeed with CPU Offloading.Hardware: Single A100 80GB GPU with CPU RAM above 64GBModelFull FinetuningPEFT-LoRA PyTorchPEFT-LoRA DeepSpeed with CPU Offloadingbigscience/T0_3B (3B params)47.14GB GPU / 2.96GB CPU14.4GB GPU / 2.96GB CPU9.8GB GPU / 17.8GB CPUbigscience/mt0-xxl (12B params)OOM GPU56GB GPU / 3GB CPU22GB GPU / 52GB CPUbigscience/bloomz-7b1 (7B params)OOM GPU32GB GPU / 3.8GB CPU18.1GB GPU / 35GB CPUPerformance of PEFT-LoRA tunedbigscience/T0_3Bonought/raft/twitter_complaintsleaderboard.
A point to note is that we didn't try to sequeeze performance by playing around with input instruction templates, LoRA hyperparams and other training related hyperparams. Also, we didn't use the larger 13Bmt0-xxlmodel.
So, we are already seeing comparable performance to SoTA with parameter efficient tuning. Also, the final checkpoint size is just19MBin comparison to11GBsize of the backbonebigscience/T0_3Bmodel.Submission NameAccuracyHuman baseline (crowdsourced)0.897Flan-T50.892lora-t0-3b0.863Therefore, we can see that performance comparable to SoTA is achievable by PEFT methods with consumer hardware such as 16GB and 24GB GPUs.Parameter Efficient Tuning of Diffusion ModelsGPU memory required by different settings during training is given below. The final checkpoint size is8.8 MB.Hardware: Single A100 80GB GPU with CPU RAM above 64GBModelFull FinetuningPEFT-LoRAPEFT-LoRA with Gradient CheckpoitningCompVis/stable-diffusion-v1-427.5GB GPU / 3.97GB CPU15.5GB GPU / 3.84GB CPU8.12GB GPU / 3.77GB CPUTrainingAn example of using LoRA for parameter efficient dreambooth training is given in~examples/lora_dreambooth/train_dreambooth.pyexportMODEL_NAME="CompVis/stable-diffusion-v1-4"#"stabilityai/stable-diffusion-2-1"exportINSTANCE_DIR="path-to-instance-images"exportCLASS_DIR="path-to-class-images"exportOUTPUT_DIR="path-to-save-model"acceleratelaunchtrain_dreambooth.py\--pretrained_model_name_or_path=$MODEL_NAME\--instance_data_dir=$INSTANCE_DIR\--class_data_dir=$CLASS_DIR\--output_dir=$OUTPUT_DIR\--train_text_encoder\--with_prior_preservation--prior_loss_weight=1.0\--instance_prompt="a photo of sks dog"\--class_prompt="a photo of dog"\--resolution=512\--train_batch_size=1\--lr_scheduler="constant"\--lr_warmup_steps=0\--num_class_images=200\--use_lora\--lora_r16\--lora_alpha27\--lora_text_encoder_r16\--lora_text_encoder_alpha17\--learning_rate=1e-4\--gradient_accumulation_steps=1\--gradient_checkpointing\--max_train_steps=800Try out the 🤗 Gradio Space which should run seamlessly on a T4 instance:smangrul/peft-lora-sd-dreambooth.Parameter Efficient Tuning of LLMs for RLHF components such as Ranker and PolicyHere is an exmaple intrllibrary using PEFT+INT8 for tuning policy model:gpt2-sentiment_peft.pyExample using PEFT for both reward model and policy [ToDo]INT8 training of large models in Colab using PEFT LoRA and bits_and_bytesHere is now a demo on how to fine tuneOPT-6.7b(14GB in fp16) in a Google colab:Here is now a demo on how to fine tunewhishper-large(1.5B params) (14GB in fp16) in a Google colab:andSave compute and storage even for medium and small modelsSave storage by avoiding full finetuning of models on each of the downstream tasks/datasets,
With PEFT methods, users only need to store tiny checkpoints in the order ofMBsall the while retaining
performance comparable to full finetuning.An example of using LoRA for the task of adaptingLayoutLMForTokenClassificationonFUNSDdataset is given in~examples/token_classification/PEFT_LoRA_LayoutLMForTokenClassification_on_FUNSD.py. We can observe that with only0.62 %of parameters being trainable, we achieve performance (F1 0.777) comparable to full finetuning (F1 0.786) (without any hyerparam tuning runs for extracting more performance), and the checkpoint of this is only2.8MB. Now, if there areNsuch datasets, just have these PEFT models one for each dataset and save a lot of storage without having to worry about the problem of catastrophic forgetting or overfitting of backbone/base model.Another example is fine-tuningroberta-largeonMRPCGLUEdataset suing differenct PEFT methods. The notebooks are given in~examples/sequence_classification.PEFT + 🤗 AcceleratePEFT models work with 🤗 Accelerate out of the box. Use 🤗 Accelerate for Distributed training on various hardware such as GPUs, Apple Silicon devices etc during training.
Use 🤗 Accelerate for inferencing on consumer hardware with small resources.Example of PEFT model training using 🤗 Accelerate's DeepSpeed integrationDeepSpeed version requiredv0.8.0. An example is provided in~examples/conditional_generation/peft_lora_seq2seq_accelerate_ds_zero3_offload.py.
a. First, runaccelerate config --config_file ds_zero3_cpu.yamland answer the questionnaire.
Below are the contents of the config file.compute_environment:LOCAL_MACHINEdeepspeed_config:gradient_accumulation_steps:1gradient_clipping:1.0offload_optimizer_device:cpuoffload_param_device:cpuzero3_init_flag:truezero3_save_16bit_model:truezero_stage:3distributed_type:DEEPSPEEDdowncast_bf16:'no'dynamo_backend:'NO'fsdp_config:{}machine_rank:0main_training_function:mainmegatron_lm_config:{}mixed_precision:'no'num_machines:1num_processes:1rdzv_backend:staticsame_network:trueuse_cpu:falseb. run the below command to launch the example scriptacceleratelaunch--config_fileds_zero3_cpu.yamlexamples/peft_lora_seq2seq_accelerate_ds_zero3_offload.pyc. output logs:GPUMemorybeforeenteringthetrain:1916GPUMemoryconsumedattheendofthetrain(end-begin):66GPUPeakMemoryconsumedduringthetrain(max-begin):7488GPUTotalPeakMemoryconsumedduringthetrain(max):9404CPUMemorybeforeenteringthetrain:19411CPUMemoryconsumedattheendofthetrain(end-begin):0CPUPeakMemoryconsumedduringthetrain(max-begin):0CPUTotalPeakMemoryconsumedduringthetrain(max):19411epoch=4:train_ppl=tensor(1.0705,device='cuda:0')train_epoch_loss=tensor(0.0681,device='cuda:0')100%|████████████████████████████████████████████████████████████████████████████████████████████|7/7[00:27<00:00,3.92s/it]GPUMemorybeforeenteringtheeval:1982GPUMemoryconsumedattheendoftheeval(end-begin):-66
GPUPeakMemoryconsumedduringtheeval(max-begin):672GPUTotalPeakMemoryconsumedduringtheeval(max):2654CPUMemorybeforeenteringtheeval:19411CPUMemoryconsumedattheendoftheeval(end-begin):0CPUPeakMemoryconsumedduringtheeval(max-begin):0CPUTotalPeakMemoryconsumedduringtheeval(max):19411accuracy=100.0
eval_preds[:10]=['no complaint','no complaint','complaint','complaint','no complaint','no complaint','no complaint','complaint','complaint','no complaint']dataset['train'][label_column][:10]=['no complaint','no complaint','complaint','complaint','no complaint','no complaint','no complaint','complaint','complaint','no complaint']Example of PEFT model inference using 🤗 Accelerate's Big Model Inferencing capabilitiesAn example is provided in~examples/causal_language_modeling/peft_lora_clm_accelerate_big_model_inference.ipynb.Models support matrixCausal Language ModelingModelLoRAPrefix TuningP-TuningPrompt TuningGPT-2✅✅✅✅Bloom✅✅✅✅OPT✅✅✅✅GPT-Neo✅✅✅✅GPT-J✅✅✅✅GPT-NeoX-20B✅✅✅✅LLaMA✅✅✅✅ChatGLM✅✅✅✅Conditional GenerationModelLoRAPrefix TuningP-TuningPrompt TuningT5✅✅✅✅BART✅✅✅✅Sequence ClassificationModelLoRAPrefix TuningP-TuningPrompt TuningBERT✅✅✅✅RoBERTa✅✅✅✅GPT-2✅✅✅✅Bloom✅✅✅✅OPT✅✅✅✅GPT-Neo✅✅✅✅GPT-J✅✅✅✅Deberta✅✅✅Deberta-v2✅✅✅Token ClassificationModelLoRAPrefix TuningP-TuningPrompt TuningBERT✅✅RoBERTa✅✅GPT-2✅✅Bloom✅✅OPT✅✅GPT-Neo✅✅GPT-J✅✅Deberta✅Deberta-v2✅Text-to-Image GenerationModelLoRAPrefix TuningP-TuningPrompt TuningStable Diffusion✅Image ClassificationModelLoRAPrefix TuningP-TuningPrompt TuningViT✅Swin✅Note that we have tested LoRA forViTandSwinfor fine-tuning on image classification. However, it should be possible to use LoRA for any compatible modelprovidedby 🤗 Transformers. Check out the respective
examples to learn more. If you run into problems, please open an issue.The same principle applies to oursegmentation modelsas well.Semantic SegmentationModelLoRAPrefix TuningP-TuningPrompt TuningSegFormer✅Caveats:Below is an example of using PyTorch FSDP for training. However, it doesn't lead to
any GPU memory savings. Please refer issue[FSDP] FSDP with CPU offload consumes 1.65X more GPU memory when training models with most of the params frozen.frompeft.utils.otherimportfsdp_auto_wrap_policy...ifos.environ.get("ACCELERATE_USE_FSDP",None)isnotNone:accelerator.state.fsdp_plugin.auto_wrap_policy=fsdp_auto_wrap_policy(model)model=accelerator.prepare(model)Example of parameter efficient tuning withmt0-xxlbase model using 🤗 Accelerate is provided in~examples/conditional_generation/peft_lora_seq2seq_accelerate_fsdp.py.
a. First, runaccelerate config --config_file fsdp_config.yamland answer the questionnaire.
Below are the contents of the config file.command_file:nullcommands:nullcompute_environment:LOCAL_MACHINEdeepspeed_config:{}distributed_type:FSDPdowncast_bf16:'no'dynamo_backend:'NO'fsdp_config:fsdp_auto_wrap_policy:TRANSFORMER_BASED_WRAPfsdp_backward_prefetch_policy:BACKWARD_PREfsdp_offload_params:truefsdp_sharding_strategy:1fsdp_state_dict_type:FULL_STATE_DICTfsdp_transformer_layer_cls_to_wrap:T5Blockgpu_ids:nullmachine_rank:0main_process_ip:nullmain_process_port:nullmain_training_function:mainmegatron_lm_config:{}mixed_precision:'no'num_machines:1num_processes:2rdzv_backend:staticsame_network:truetpu_name:nulltpu_zone:nulluse_cpu:falseb. run the below command to launch the example scriptacceleratelaunch--config_filefsdp_config.yamlexamples/peft_lora_seq2seq_accelerate_fsdp.pyWhen usingP_TUNINGorPROMPT_TUNINGwithSEQ_2_SEQtask, remember to remove thenum_virtual_tokenvirtual prompt predictions from the left side of the model outputs during evaluations.For encoder-decoder models,P_TUNINGorPROMPT_TUNINGdoesn't supportgeneratefunctionality of transformers becausegeneratestrictly requiresdecoder_input_idsbutP_TUNING/PROMPT_TUNINGappends soft prompt embeddings toinput_embedsto create
newinput_embedsto be given to the model. Therefore,generatedoesn't support this yet.When using ZeRO3 with zero3_init_flag=True, if you find the gpu memory increase with training steps. we might need to set zero3_init_flag=false in accelerate config.yaml. The related issue is[BUG] memory leak under zero.InitBacklog:Explore and possibly integrate(IA)^3Add testsAdd more use cases and examplesCiting 🤗 PEFTIf you use 🤗 PEFT in your publication, please cite it by using the following BibTeX entry.@Misc{peft,title={PEFT: State-of-the-art Parameter-Efficient Fine-Tuning methods},author={Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul},howpublished={\url{https://github.com/huggingface/peft}},year={2022}}