README.md

---
license: apache-2.0
datasets:
- Dongwookss/q_a_korean_futsal
language:
- ko
tags:
- unsloth
- trl
- transformer
---

### Model Name : 풋풋이(futfut) 

#### Model Concept 

- 풋살 도메인 친절한 도우미 챗봇을 구축하기 위해 LLM 파인튜닝과 RAG를 이용하였습니다.
- **Base Model** : [zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) 
- 풋풋이의 말투는 '해요'체를 사용하여 말끝에 '얼마든지 물어보세요~! 풋풋~!'로 종료합니다.

<img src="https://cdn-uploads.huggingface.co/production/uploads/66305fd7fdd79b4fe6d6a5e5/7UDKdaPfBJnazuIi1cUVw.png" width="400" height="400">


#### Summary:

- **Unsloth** 패키지를 사용하여 **LoRA** 진행하였습니다.
- **SFT Trainer**를 통해 훈련을 진행
- 활용 데이터
  - [q_a_korean_futsal](https://huggingface.co/datasets/Dongwookss/q_a_korean_futsal)
    - 말투 학습을 위해 '해요'체로 변환하고 인삿말을 넣어 모델 컨셉을 유지하였습니다.
   
- **Environment** : Colab 환경에서 진행하였으며 L4 GPU를 사용하였습니다. 

<details>
  <summary>How to use</summary>
  
  **Model Load**
  
  ``` python
  
  #!pip install transformers==4.40.0 accelerate
  import os
  import torch
  from transformers import AutoTokenizer, AutoModelForCausalLM
  
  model_id = 'Dongwookss/small_fut_final'
  
  tokenizer = AutoTokenizer.from_pretrained(model_id)
  model = AutoModelForCausalLM.from_pretrained(
      model_id,
      torch_dtype=torch.bfloat16,
      device_map="auto",
  )
  model.eval()
  ```

  **Query**

  ```python
  from transformers import TextStreamer
  PROMPT = '''Below is an instruction that describes a task. Write a response that appropriately completes the reques문"
  
  messages = [
      {"role": "system", "content": f"{PROMPT}"},
      {"role": "user", "content": f"{instruction}"}
      ]
  
  input_ids = tokenizer.apply_chat_template(
      messages,
      add_generation_prompt=True,
      return_tensors="pt"
  ).to(model.device)
  
  terminators = [
      tokenizer.eos_token_id,
      tokenizer.convert_tokens_to_ids("<|eot_id|>")
  ]
  
  text_streamer = TextStreamer(tokenizer)
  _ = model.generate(
      input_ids,
      max_new_tokens=4096,
      eos_token_id=terminators,
      do_sample=True,
      streamer = text_streamer,
      temperature=0.6,
      top_p=0.9,
      repetition_penalty = 1.1
  )
  
  ```

</details>

<details>
  <summary>Fine-Tuning with Unsloth(SFT Trainer)</summary>

```python

from unsloth import FastLanguageModel
import torch
from trl import SFTTrainer
from transformers import TrainingArguments

max_seq_length = 256
dtype = None
load_in_4bit = False
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="HuggingFaceH4/zephyr-7b-beta",
    max_seq_length=max_seq_length,
    dtype=dtype,
    load_in_4bit=load_in_4bit,
    #token = ,
)

model = FastLanguageModel.get_peft_model(
    model,
    r=32,
    lora_alpha=64,
    lora_dropout=0.05, 
    target_modules=[
        "q_proj",
        "k_proj",
        "v_proj",
        "o_proj",
        "gate_proj",
        "up_proj",
        "down_proj",
    ],  # 타겟 모듈
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=123,
    use_rslora=False,
    loftq_config=None,
)

tokenizer.padding_side = "right"

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=max_seq_length,
    dataset_num_proc=2,
    packing=False,
    args=TrainingArguments(
        per_device_train_batch_size=20,
        gradient_accumulation_steps=2,
        warmup_steps=5,
        num_train_epochs=3,
        max_steps = 1761,
        logging_steps = 10,
        learning_rate=2e-5,
        fp16=not torch.cuda.is_bf16_supported(),
        bf16=torch.cuda.is_bf16_supported(),
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="cosine",
        seed=123,
        output_dir="outputs",
    ),
)

trainer.train()

```


</details>