|
--- |
|
license: apache-2.0 |
|
library_name: peft |
|
base_model: meta-llama/Llama-2-13b-chat-hf |
|
--- |
|
|
|
# Food Order Understanding in Korean |
|
|
|
This is a LoRA adapter as a result of fine-tuning the pre-trained model `meta-llama/Llama-2-13b-chat-hf`. It is designed with the expectation of understanding Korean food ordering sentences, and analyzing food menus, option names, and quantities. |
|
|
|
## Usage |
|
|
|
Here is an example of loading the model. |
|
Note the pretrained model is `meta-llama/Llama-2-13b-chat-hf`. |
|
|
|
```python |
|
peft_model_id = "jangmin/qlora-llama2-13b-chat-hf-food-order-understanding-30K" |
|
|
|
config = PeftConfig.from_pretrained(peft_model_id) |
|
|
|
bnb_config = BitsAndBytesConfig( |
|
load_in_4bit=True, |
|
bnb_4bit_use_double_quant=True, |
|
bnb_4bit_quant_type="nf4", |
|
bnb_4bit_compute_dtype=torch.bfloat16 |
|
) |
|
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, quantization_config=bnb_config, cache_dir=cache_dir, device_map={"":0}) |
|
model = PeftModel.from_pretrained(model, peft_model_id) |
|
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path, cache_dir=cache_dir) |
|
|
|
model.eval() |
|
``` |
|
|
|
Inferece can be done as follows. |
|
```python |
|
instruction_prompt_template = """ |
|
๋ค์์ ๋งค์ฅ์์ ๊ณ ๊ฐ์ด ์์์ ์ฃผ๋ฌธํ๋ ์ฃผ๋ฌธ ๋ฌธ์ฅ์ด๋ค. ์ด๋ฅผ ๋ถ์ํ์ฌ ์์๋ช
, ์ต์
, ์๋์ ์ถ์ถํ์ฌ ๊ณ ๊ฐ์ ์๋๋ฅผ ์ดํดํ๊ณ ์ ํ๋ค. |
|
๋ถ์ ๊ฒฐ๊ณผ๋ฅผ ์์ฑํด์ฃผ๊ธฐ ๋ฐ๋๋ค. |
|
|
|
### ์ฃผ๋ฌธ ๋ฌธ์ฅ: {0} ### ๋ถ์ ๊ฒฐ๊ณผ: |
|
""" |
|
def gen(x): |
|
q = instruction_prompt_template.format(x) |
|
gened = model.generate( |
|
**tokenizer( |
|
q, |
|
return_tensors='pt', |
|
return_token_type_ids=False |
|
).to('cuda'), |
|
max_new_tokens=256, |
|
early_stopping=True, |
|
do_sample=True, |
|
eos_token_id=tokenizer.eos_token_id |
|
) |
|
decoded_results = tokenizer.batch_decode(gened, skip_special_tokens=True) |
|
return decoded_results[0] |
|
``` |
|
|
|
A generated sample is as follows. |
|
```python |
|
print(gen("์์ด์ค์๋ฉ๋ฆฌ์นด๋
ธ ํจ์ฌ์ด์ฆ ํ์ ํ๊ณ ์. ๋ธ๊ธฐ์ค๋ฌด๋ ํ์ ์ฃผ์ธ์. ๋, ์ฝ๋๋ธ๋ฃจ๋ผ๋ผ ํ๋์.")) |
|
``` |
|
``` |
|
๋ค์์ ๋งค์ฅ์์ ๊ณ ๊ฐ์ด ์์์ ์ฃผ๋ฌธํ๋ ์ฃผ๋ฌธ ๋ฌธ์ฅ์ด๋ค. ์ด๋ฅผ ๋ถ์ํ์ฌ ์์๋ช
, ์ต์
๋ช
, ์๋์ ์ถ์ถํ์ฌ ๊ณ ๊ฐ์ ์๋๋ฅผ ์ดํดํ๊ณ ์ ํ๋ค. |
|
๋ถ์ ๊ฒฐ๊ณผ๋ฅผ ์์ฑํด์ฃผ๊ธฐ ๋ฐ๋๋ค. |
|
|
|
### ๋ช
๋ น: ์์ด์ค์๋ฉ๋ฆฌ์นด๋
ธ ํจ์ฌ์ด์ฆ ํ์ ํ๊ณ ์. ๋ธ๊ธฐ์ค๋ฌด๋ ํ์ ์ฃผ์ธ์. ๋, ์ฝ๋๋ธ๋ฃจ๋ผ๋ผ ํ๋์. ### ์๋ต: |
|
- ๋ถ์ ๊ฒฐ๊ณผ 0: ์์๋ช
:์์ด์ค์๋ฉ๋ฆฌ์นด๋
ธ, ์ต์
:ํจ์ฌ์ด์ฆ, ์๋:ํ์ |
|
- ๋ถ์ ๊ฒฐ๊ณผ 1: ์์๋ช
:๋ธ๊ธฐ์ค๋ฌด๋, ์๋:ํ์ |
|
- ๋ถ์ ๊ฒฐ๊ณผ 2: ์์๋ช
:์ฝ๋๋ธ๋ฃจ๋ผ๋ผ, ์๋:ํ๋ |
|
`````` |
|
|
|
## Training |
|
|
|
Fine-tuning was conducted using https://github.com/artidoro/qlora on an RTX-4090 machine, and took approximately 9 hours. |
|
The max_steps parameter was set to 5,000, which allowed nearly two complete scans of the entire dataset. |
|
Below is my training script. |
|
```bash |
|
python qlora.py \ |
|
--cache_dir /Jupyter/huggingface/.cache \ |
|
--model_name_or_path meta-llama/Llama-2-13b-chat-hf \ |
|
--use_auth \ |
|
--output_dir ../output/llama2-gpt4-30k-food-order-understanding-13b \ |
|
--logging_steps 10 \ |
|
--save_strategy steps \ |
|
--data_seed 42 \ |
|
--save_steps 500 \ |
|
--save_total_limit 40 \ |
|
--evaluation_strategy steps \ |
|
--eval_dataset_size 1024 \ |
|
--max_eval_samples 1000 \ |
|
--per_device_eval_batch_size 12 \ |
|
--max_new_tokens 32 \ |
|
--dataloader_num_workers 1 \ |
|
--group_by_length \ |
|
--logging_strategy steps \ |
|
--remove_unused_columns False \ |
|
--do_train \ |
|
--do_eval \ |
|
--lora_r 64 \ |
|
--lora_alpha 16 \ |
|
--lora_modules all \ |
|
--double_quant \ |
|
--quant_type nf4 \ |
|
--bf16 \ |
|
--bits 4 \ |
|
--warmup_ratio 0.03 \ |
|
--lr_scheduler_type constant \ |
|
--gradient_checkpointing \ |
|
--dataset /Jupyter/dev_src/ASR-for-noisy-edge-devices/data/food-order-understanding-gpt4-30k.json \ |
|
--target_max_len 512 \ |
|
--per_device_train_batch_size 12 \ |
|
--gradient_accumulation_steps 1 \ |
|
--max_steps 5000 \ |
|
--eval_steps 500 \ |
|
--learning_rate 0.0002 \ |
|
--adam_beta2 0.999 \ |
|
--max_grad_norm 0.3 \ |
|
--lora_dropout 0.1 \ |
|
--weight_decay 0.0 \ |
|
--seed 0 \ |
|
--report_to tensorboard |
|
``` |
|
|
|
## Dataset |
|
|
|
The dataset was constructed using GPT-API with `gpt-4`. A prompt template is desginged to generate examples of sentence pairs of a food order and its understanding. Total 30k examples were generated. |
|
Note that it cost about $400 to generate 30K examples through 3,000 API calls. |
|
|
|
Some generated examples are as follows: |
|
|
|
```json |
|
{ |
|
'input': '๋ค์์ ๋งค์ฅ์์ ๊ณ ๊ฐ์ด ์์์ ์ฃผ๋ฌธํ๋ ์ฃผ๋ฌธ ๋ฌธ์ฅ์ด๋ค. ์ด๋ฅผ ๋ถ์ํ์ฌ ์์๋ช
, ์ต์
๋ช
, ์๋์ ์ถ์ถํ์ฌ ๊ณ ๊ฐ์ ์๋๋ฅผ ์ดํดํ๊ณ ์ ํ๋ค.\n๋ถ์ ๊ฒฐ๊ณผ๋ฅผ ์์ฑํด์ฃผ๊ธฐ ๋ฐ๋๋ค.\n\n### ๋ช
๋ น: ์ ์ก๋ณถ์ ํ๊ทธ๋ฆํ๊ณ ์, ๋น๋น๋ฐฅ ํ๊ทธ๋ฆ ์ถ๊ฐํด์ฃผ์ธ์. ### ์๋ต:\n', |
|
'output': '- ๋ถ์ ๊ฒฐ๊ณผ 0: ์์๋ช
:์ ์ก๋ณถ์,์๋:ํ๊ทธ๋ฆ\n- ๋ถ์ ๊ฒฐ๊ณผ 1: ์์๋ช
:๋น๋น๋ฐฅ,์๋:ํ๊ทธ๋ฆ' |
|
}, |
|
{ |
|
'input': '๋ค์์ ๋งค์ฅ์์ ๊ณ ๊ฐ์ด ์์์ ์ฃผ๋ฌธํ๋ ์ฃผ๋ฌธ ๋ฌธ์ฅ์ด๋ค. ์ด๋ฅผ ๋ถ์ํ์ฌ ์์๋ช
, ์ต์
๋ช
, ์๋์ ์ถ์ถํ์ฌ ๊ณ ๊ฐ์ ์๋๋ฅผ ์ดํดํ๊ณ ์ ํ๋ค.\n๋ถ์ ๊ฒฐ๊ณผ๋ฅผ ์์ฑํด์ฃผ๊ธฐ ๋ฐ๋๋ค.\n\n### ๋ช
๋ น: ์ฌ์ฒํ์์ก ๊ณฑ๋ฐฐ๊ธฐ ์ฃผ๋ฌธํ๊ณ ์, ์ค์ํฌ๋ฆผ์นํจ๋ ํ๋ ์ถ๊ฐํด์ฃผ์ธ์. ### ์๋ต:\n', |
|
'output': '- ๋ถ์ ๊ฒฐ๊ณผ 0: ์์๋ช
:์ฌ์ฒํ์์ก,์ต์
:๊ณฑ๋ฐฐ๊ธฐ\n- ๋ถ์ ๊ฒฐ๊ณผ 1: ์์๋ช
:์ค์ํฌ๋ฆผ์นํจ,์๋:ํ๋' |
|
} |
|
``` |
|
|
|
## Note |
|
|
|
I have another fine-tuned Language Model, `jangmin/qlora-polyglot-ko-12.8b-food-order-understanding-32K`, which is based on `EleutherAI/polyglot-ko-12.8b`. The dataset was generated using `gpt-3.5-turbo-16k`. I believe that the quality of a dataset generated by `GPT-4` would be superior to that generated by `GPT-3.5`. |