Package: deltatuner
DeltatunerDeltatuner is an extension forPeftto improve LLM fine-tuning speed through multiple optimizations, including leveraging the compact model constructorDE-NASto construct/modify the compact delta layers in a hardware-aware and train-free approach and adding more new deltatuning algorithms.IntroductionKey ComponentsSupported parameter efficient finetuning algorithmsLoRAalgorithm: It freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks.Scaling and Shifting(SSF) algorithm: Scale and Shift the deep features in a pre-trained model with much fewer parameters to catch up with the performance of full finetuningDe-Nas: Automatically construct compact and optimal delta layers with train-free and hardware-aware mode (more detailshere)step1: Generate search space for delta layersstep2: Search algorithm populates delta layers for LMstep3: Train-free score evaluates LM with adaptive delta layersFeaturesEasy-to-use: provide package install; just need to inject a few codes into the original codeAuto-tuning: automatically select the best algorithms and delta structure for finetuning modelValuesSaving computation power: reduce the computation power and time required to fine-tune a model by reducing parameter size and memory footprint.Improve accuracy: ensure the same or no accuracy regression.Get StartedInstallationinstall the python packagepipinstalle2eAIOK-deltatunerFast Fine-tuning on Base modelsBelow is an example of optimizingMPTmodel by adding the following few-lines to use the delatuner optimizations. It use the DE-NAS in delatuner to optimize a LLM with LoRA layers to a LLM with compact LoRA layers, so as to efficiently improve the LLM fine-tuning process in peak memory reduction and time speedup.from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig, get_peft_model

# import model from huggingface
model_id =  "mosaicml/mpt-7b"
model = AutoModelForCausalLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# adding the lora componenents with peft
config = LoraConfig()
lora_model = get_peft_model(model, config) 

# delatuner optimize the model with best lora layer configuration+ import deltatuner+ deltatuner_config = {"algo": "lora", "denas": True, "best_model_structure": f"/path/to/best_structure.jsonl"}+ deltatuner_model = deltatuner.optimize(lora_model, **deltatuner_config)...API referenceIn above examples,deltatuner.optimizeis a python function to using deltatuner supported optimization algorithms to optimize the model.defoptimize(model,tokenizer,deltatuning_args=None,**kwargs)->DeltaTunerModel:'''Parameters:- model - a HuggingFace PreTrainedModel or LoraModel. Specifies the model to be optimized- tokenizer - a corresponding tokenizer for text preprocessing, it is optional, and default set to None, if not specify the deltatuning_args.best_model_structure, then tokenizer cannot be ommitted- deltatuning_args (optional) – the deltatuner configuration.- deltatuning_args.denas is to use the denas in the optimization (default: True)- deltatuning_args.algo Specifies what type of parameter efficient tuning algorithm (default: auto)- "auto" – If the input model is mpt, the algorithm is ssf; elif the algorithm is lora- "lora" – use the lora algotihm- "ssf" – use the ssf algotithm- deltatuning_args.best_model_structure Specifies the pre-searched delta best structure so the model can be directly initialized without searching with DE-Nas algorithm.- kwargs - used to initialize deltatuning_args through key=value, such as algo="lora", "denas"=TrueReturnDeltaTunerModel - a wrapper of model, which composed of the original properties/function together with advanced properties/function provided by deltatuner'''Detailed examplesPlease refer toexample pagefor more use cases on fine-tuning other LLMs with the help of DeltaTuner.Model supported matrixWe have uploaded the searched delta best structure to theconf dir, so that users can directly use our searched structure for directly fine-tuning by passing theDeltaTunerArguments.best_model_structureto thedeltatuner.optimizefunction.Causal Language ModelingModelLoRASSFGPT-2✅GPT-J✅✅Bloom✅✅OPT✅✅GPT-Neo✅✅Falcon✅✅LLaMA✅✅MPT✅✅