Package: glmtuner
ChatGLM Efficient TuningFine-tuning 🤖ChatGLM-6Bmodel with 🤗PEFT.👋 Join ourWeChat.[ English |中文]If you have any questions, please refer to ourWiki📄.NoticeThis repo willnot be maintainedin the future. Please followLLaMA-Efficient-Tuningfor fine-tuning the language models (including ChatGLM2-6B).Changelog[23/07/15] Now we develop an all-in-one Web UI for training, evaluation and inference. Trytrain_web.pyto fine-tune ChatGLM-6B model in your Web browser. Thank@KanadeSiinaand@codemayqfor their efforts in the development.[23/07/09] Now we releaseFastEdit⚡🩹, an easy-to-use package for editing the factual knowledge of large language models efficiently. Please followFastEditif you are interested.[23/06/25] Now we align thedemo APIwith theOpenAI'sformat where you can insert the fine-tuned model in arbitrary ChatGPT-based applications.[23/06/25] Now we support fine-tuning theChatGLM2-6Bmodel with our framework![23/06/05] Now we support 4-bit LoRA training (akaQLoRA). Try--quantization_bit 4argument to work with 4-bit quantized model. (experimental feature)[23/06/01] We implemented a framework supporting the efficient tuning of LLaMA and BLOOM models. Please followLLaMA-Efficient-Tuningif you are interested.[23/05/19] Now we support using the development set to evaluate the model while training. Try--dev_ratioargument to specify the size of development set.[23/04/29] Now we support training ChatGLM withReinforcement Learning with Human Feedback (RLHF)! We provide several examples to run RLHF training, please refer to theexamplesfolder for details.[23/04/20] Our repo achieved 100 stars within 12 days! Congratulations![23/04/19] Now we supportmerging the weightsof fine-tuned models trained by LoRA! Try--checkpoint_dir checkpoint1,checkpoint2argument for continually fine-tuning the models.[23/04/18] Now we support training thequantized modelsusing three fine-tuning methods! Tryquantization_bitargument for training the model in 4/8 bits.[23/04/12] Now we supporttraining from checkpoints! Use--checkpoint_dirargument to specify the checkpoint model to fine-tune from.[23/04/11] Now we support training withcombined datasets! Try--dataset dataset1,dataset2argument for training with multiple datasets.DatasetsFor supervised fine-tuning:Stanford Alpaca (en)Stanford Alpaca (zh)GPT-4 Generated Data (en&zh)Open Assistant (multilingual)Self-cognition (zh)ShareGPT (zh)RefGPT (zh)Guanaco Dataset (multilingual)BELLE 2M (zh)BELLE 1M (zh)BELLE 0.5M (zh)BELLE Dialogue 0.4M (zh)BELLE School Math 0.25M (zh)BELLE Multiturn Chat 0.8M (zh)Firefly 1.1M (zh)LIMA (en)CodeAlpaca 20k (en)Alpaca CoT (multilingual)Web QA (zh)UltraChat (en)WebNovel (zh)For reward modelling:HH-RLHF (en)Open Assistant (multilingual)GPT-4 Generated Data (en&zh)Please refer todata/README.mdfor details.Some datasets require confirmation before using them, so we recommend logging in with your Hugging Face account using these commands.pipinstall--upgradehuggingface_hub
huggingface-cliloginFine-Tuning MethodsOur script now supports the following fine-tuning methods:LoRAFine-tuning the low-rank adapters of the model.P-Tuning V2Fine-tuning the prefix encoder of the model.FreezeFine-tuning the MLPs in the last n blocks of the model.Full TuningFine-tuning all the parameters of the model.RequirementPython 3.8+ and PyTorch 1.13.1+🤗Transformers, Datasets, Accelerate, PEFT and TRLfire, protobuf, cpm-kernels and sentencepiecejieba, rouge-chinese and nltk (used at evaluation)gradio and matplotlib (used in train_web.py)uvicorn, fastapi and sse-starlette (used in api_demo.py)Andpowerful GPUs!Getting StartedData Preparation (optional)Please refer todata/example_datasetfor checking the details about the format of dataset files. You can either use a single.jsonfile or adataset loading scriptwith multiple files to create a custom dataset.Note: please updatedata/dataset_info.jsonto use your custom dataset. About the format of this file, please refer todata/README.md.Dependence Installation (optional)gitlfsinstall
gitclonehttps://github.com/hiyouga/ChatGLM-Efficient-Tuning.git
condacreate-nchatglm_etuningpython=3.10
condaactivatechatglm_etuningcdChatGLM-Efficient-Tuning
pipinstall-rrequirements.txtIf you want to enable the quantized LoRA (QLoRA) on the Windows platform, you will be required to install a pre-built version ofbitsandbyteslibrary, which supports CUDA 11.1 to 12.1.pipinstallhttps://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.39.1-py3-none-win_amd64.whlAll-in-one Web UICUDA_VISIBLE_DEVICES=0pythonsrc/train_web.pyCurrently the web UI only supports training ona single GPU.Fine-tuning with a Single GPUCUDA_VISIBLE_DEVICES=0pythonsrc/train_bash.py\--stagesft\--model_name_or_pathpath_to_your_chatglm_model\--do_train\--datasetalpaca_gpt4_en\--finetuning_typelora\--output_dirpath_to_sft_checkpoint\--per_device_train_batch_size4\--gradient_accumulation_steps4\--lr_scheduler_typecosine\--logging_steps10\--save_steps1000\--learning_rate5e-5\--num_train_epochs3.0\--plot_loss\--fp16Please refer to ourWikiabout the details of the arguments.Distributed Fine-tuning with Multiple GPUsaccelerateconfig# configure the environmentacceleratelaunchsrc/train_bash.py# arguments (same as above)Training Reward ModelCUDA_VISIBLE_DEVICES=0pythonsrc/train_bash.py\--stagerm\--model_name_or_pathpath_to_your_chatglm_model\--do_train\--datasetcomparison_gpt4_en\--finetuning_typelora\--resume_lora_trainingFalse\--checkpoint_dirpath_to_sft_checkpoint\--output_dirpath_to_rm_checkpoint\--per_device_train_batch_size4\--gradient_accumulation_steps4\--lr_scheduler_typecosine\--logging_steps10\--save_steps1000\--learning_rate1e-5\--num_train_epochs1.0\--plot_loss\--fp16Training with RLHFCUDA_VISIBLE_DEVICES=0pythonsrc/train_bash.py\--stageppo\--model_name_or_pathpath_to_your_chatglm_model\--do_train\--datasetalpaca_gpt4_en\--finetuning_typelora\--resume_lora_trainingFalse\--checkpoint_dirpath_to_sft_checkpoint\--reward_modelpath_to_rm_checkpoint\--output_dirpath_to_ppo_checkpoint\--per_device_train_batch_size2\--gradient_accumulation_steps4\--lr_scheduler_typecosine\--logging_steps10\--save_steps1000\--learning_rate1e-5\--num_train_epochs1.0\--plot_lossEvaluation (BLEU and ROUGE_CHINESE)CUDA_VISIBLE_DEVICES=0pythonsrc/train_bash.py\--stagesft\--model_name_or_pathpath_to_your_chatglm_model\--do_eval\--datasetalpaca_gpt4_en\--finetuning_typelora\--checkpoint_dirpath_to_checkpoint\--output_dirpath_to_eval_result\--per_device_eval_batch_size8\--max_samples50\--predict_with_generatePredictCUDA_VISIBLE_DEVICES=0pythonsrc/train_bash.py\--stagesft\--model_name_or_pathpath_to_your_chatglm_model\--do_predict\--datasetalpaca_gpt4_en\--finetuning_typelora\--checkpoint_dirpath_to_checkpoint\--output_dirpath_to_predict_result\--per_device_eval_batch_size8\--max_samples100\--predict_with_generateIf you want to predict the samples with empty responses, please kindly fill theresponsecolumn withdummy tokensto ensure the sample will not be discarded throughout the preprocessing phase.API Demopythonsrc/api_demo.py\--model_name_or_pathpath_to_your_chatglm_model\--finetuning_typelora\--checkpoint_dirpath_to_checkpointVisithttp://localhost:8000/docsfor API documentation.CLI Demopythonsrc/cli_demo.py\--model_name_or_pathpath_to_your_chatglm_model\--finetuning_typelora\--checkpoint_dirpath_to_checkpointWeb Demopythonsrc/web_demo.py\--model_name_or_pathpath_to_your_chatglm_model\--finetuning_typelora\--checkpoint_dirpath_to_checkpointExport modelpythonsrc/export_model.py\--model_name_or_pathpath_to_your_chatglm_model\--finetuning_typelora\--checkpoint_dirpath_to_checkpoint\--output_dirpath_to_exportHardware RequirementsFine-tune methodBatch sizeModeGRAMSpeedLoRA (r=8)16FP1628GB8ex/sLoRA (r=8)8FP1624GB8ex/sLoRA (r=8)4FP1620GB8ex/sLoRA (r=8)4INT810GB8ex/sLoRA (r=8)4INT48GB8ex/sP-Tuning (p=16)4FP1620GB8ex/sP-Tuning (p=16)4INT816GB8ex/sP-Tuning (p=16)4INT412GB8ex/sFreeze (l=3)4FP1624GB8ex/sRM  methodBatch sizeModeGRAMSpeedLoRA (r=8) + rm4FP1622GB-LoRA (r=8) + rm1INT811GB-RLHF methodBatch sizeModeGRAMSpeedLoRA (r=8) + ppo4FP1623GB-LoRA (r=8) + ppo1INT812GB-Note:ris the lora rank,pis the number of prefix tokens,lis the number of trainable layers,ex/sis the examples per second at training. Thegradient_accumulation_stepsis set to1. All are evaluated on a single Tesla V100 (32G) GPU, they are approximated values and may vary in different GPUs.Fine-tuning ChatGLM: A CaseTraining ResultsWe use the wholealpaca_gpt4_zhdataset to fine-tune the ChatGLM model with LoRA (r=8) for one epoch, using the default hyper-parameters. The loss curve during training is presented below.Evaluation ResultsWe select 100 instances in thealpaca_gpt4_zhdataset to evaluate the fine-tuned ChatGLM model and compute the BLEU and ROUGE scores. The results are presented below.ScoreOriginalFZ (l=2)PT (p=16)LoRA (r=8)BLEU-415.7516.8516.0617.01 (+1.26)Rouge-134.5136.6234.8036.77 (+2.26)Rouge-215.1117.0415.3216.83 (+1.72)Rouge-l26.1828.1726.3528.86 (+2.68)Params (%)/4.35%0.06%0.06%FZ: freeze tuning, PT: P-Tuning V2 (we usepre_seq_len=16for fair comparison with LoRA), Params: the percentange of trainable parameters.ProjectsSupritYoung/RLHF-Label-Tool: A tool for ranking the responses of LLMs to generate annotated samples used in RLHF training.Compared with Existing ImplementationsTHUDM/ChatGLM-6BOfficial implementation of fine-tuning ChatGLM withP-Tuning v2on theADGENdataset.Our fine-tuning script is largely depend on it. We further implement theLoRAtuning method. Additionally, wedynamicallypad the inputs to the longest sequence in the batch instead of the maximum length, to accelerate the fine-tuning.mymusise/ChatGLM-TuningAn unoffical implementation of fine-tuning ChatGLM withLoRAon theStanford Alpacadataset.We borrowed some ideas from it. Our fine-tuning scriptintegratesthe data pre-processing part into the training procedure, so we need not generate a pre-processed dataset before training.ssbuild/chatglm_finetuningAn unofficial implementation of fine-tuning ChatGLM with several PEFT methods on theStanford Alpacadataset.Our fine-tuning script is implementedpurelywithHugging Face transformersand is independent of thedeep_trainingframework.lich99/ChatGLM-finetune-LoRAAn unofficial implementation of fine-tuning ChatGLM withLoRAon theStanford Alpacadataset.We use theHugging Face PEFTto provide the state-of-the-art PEFT methods.liucongg/ChatGLM-FinetuningAn unofficial implementation of fine-tuning ChatGLM with several methods including Freeze, LoRA and P-Tuning on the industrial dataset.We are aim to incorporate more instruction-following datasets for fine-tuning the ChatGLM model.yanqiangmiffy/InstructGLMAn unofficial implementation of fine-tuning ChatGLM that explores the ChatGLM's ability on the instruction-following datasets.Our fine-tuning script integrates the data pre-processing part in to the training procedure.TODOEmployingLangChainto easily build applications that are capable of leveraging external knowledge upon fine-tuned ChatGLM models.Implementing the alignment algorithms to align human preferrences.RLHFRRHFRAFTIncorporatingChinese datasetsinto the training sets.BELLEpCLUECLUECorpusGuanacoDatasetFireflyDatasetIncorporatingChatGPT&GPT-4self-chat data into the training sets.BaizeGPT-4-LLMImplementing the Freeze-Tuning and P-Tuning method.Supporting Multi-GPUs fine-tuning.Adding script for evaluation.Loading from checkpoint.Fine-tuning the quantized model.Writing a guidebook about how to fine-tune ChatGLM with this framework.Combining with state-of-the-art model editing algorithms. (e.g.MEND)Incorporating theOpenAssistant Conversations Datasetfor SFT and alignment.Incorporating the high quality Chinese instruction datasetCOIG.LicenseThis repository is licensed under theApache-2.0 License. Please follow theModel Licenseto use ChatGLM-6B model.CitationIf this work is helpful, please cite as:@Misc{chatglm-efficient-tuning,title={ChatGLM Efficient Tuning},author={hiyouga},howpublished={\url{https://github.com/hiyouga/ChatGLM-Efficient-Tuning}},year={2023}}AcknowledgementThis repo benefits fromChatGLM-6B,ChatGLM-Tuningandyuanzhoulvpi2017/zero_nlp. Thanks for their wonderful works.Star History