AutoTrain documentation

LLM Finetuning

You are viewing v0.8.8 version. A newer version v0.8.21 is available.
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

LLM Finetuning

With AutoTrain, you can easily finetune large language models (LLMs) on your own data!

AutoTrain supports the following types of LLM finetuning:

  • Causal Language Modeling (CLM)
  • Masked Language Modeling (MLM) [Coming Soon]

Data Preparation

LLM finetuning accepts data in CSV format.

Data Format For SFT / Generic Trainer

For SFT / Generic Trainer, the data should be in the following format:

text
human: hello \n bot: hi nice to meet you
human: how are you \n bot: I am fine
human: What is your name? \n bot: My name is Mary
human: Which is the best programming language? \n bot: Python

An example dataset for this format can be found here: https://huggingface.co/datasets/timdettmers/openassistant-guanaco

For SFT/Generic training, your dataset must have a text column

Data Format For Reward Trainer

For Reward Trainer, the data should be in the following format:

text rejected_text
human: hello \n bot: hi nice to meet you human: hello \n bot: leave me alone
human: how are you \n bot: I am fine human: how are you \n bot: I am not fine
human: What is your name? \n bot: My name is Mary human: What is your name? \n bot: Whats it to you?
human: Which is the best programming language? \n bot: Python human: Which is the best programming language? \n bot: Javascript

For Reward Trainer, your dataset must have a text column (aka chosen text) and a rejected_text column.

Data Format For DPO/ORPO Trainer

For DPO/ORPO Trainer, the data should be in the following format:

prompt text rejected_text
hello hi nice to meet you leave me alone
how are you I am fine I am not fine
What is your name? My name is Mary Whats it to you?
What is your name? My name is Mary I dont have a name
Which is the best programming language? Python Javascript
Which is the best programming language? Python C++
Which is the best programming language? Java C++

For DPO/ORPO Trainer, your dataset must have a prompt column, a text column (aka chosen text) and a rejected_text column.

For all tasks, you can use both CSV and JSONL files!

< > Update on GitHub