---
library_name: transformers
tags:
- unsloth
- trl
- sft
datasets:
- mintaeng/llm_futsaldata_yo
license: apache-2.0
language:
- ko
---
### Model Name : 풋풋이(futfut)
#### Model Concept
- 풋살 도메인 친절한 도우미 챗봇을 구축하기 위해 LLM 파인튜닝과 RAG를 이용하였습니다.
- **Base Model** : [zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)
- 풋풋이의 말투는 '해요'체를 사용하여 말끝에 '얼마든지 물어보세요~! 풋풋~!'로 종료합니다.
### Serving by Fast API
- Git repo : [Dongwooks](https://github.com/ddsntc1/FA_Chatbot_for_API)
#### Summary:
- **Unsloth** 패키지를 사용하여 **LoRA** 진행하였습니다.
- **SFT Trainer**를 통해 훈련을 진행
- 활용 데이터
- [llm_futsaldata_yo](https://huggingface.co/datasets/mintaeng/llm_futsaldata_yo)
- 말투 학습을 위해 '해요'체로 변환하고 인삿말을 넣어 모델 컨셉을 유지하였습니다.
- **Train for 7H 23M**
- **Environment** : Colab 환경에서 진행하였으며 L4 GPU를 사용하였습니다.
**Model Load**
``` python
#!pip install transformers==4.40.0 accelerate
import os
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = 'Dongwookss/big_fut_final'
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
model.eval()
```
**Query**
```python
from transformers import TextStreamer
PROMPT = '''Below is an instruction that describes a task. Write a response that appropriately completes the request.
제시하는 context에서만 대답하고 context에 없는 내용은 모르겠다고 대답해'''
messages = [
{"role": "system", "content": f"{PROMPT}"},
{"role": "user", "content": f"{instruction}"}
]
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
terminators = [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<|eot_id|>")
]
text_streamer = TextStreamer(tokenizer)
_ = model.generate(
input_ids,
max_new_tokens=4096,
eos_token_id=terminators,
do_sample=True,
streamer = text_streamer,
temperature=0.6,
top_p=0.9,
repetition_penalty = 1.1
)
```
## Model Details
### Model Description
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
- **Developed by:** Dongwookss
- **Model type:** [More Information Needed]
- **Language(s) (NLP):** Korean
- **Finetuned from model :** HuggingFaceH4/zephyr-7b-beta
### Model Sources [optional]
- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]
## Uses
### Direct Use
[More Information Needed]
### Downstream Use [optional]
[More Information Needed]
### Out-of-Scope Use
[More Information Needed]
## Bias, Risks, and Limitations
[More Information Needed]
### Recommendations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
## How to Get Started with the Model
Use the code below to get started with the model.
[More Information Needed]
## Training Details
### Training Data
[More Information Needed]
### Training Procedure
#### Preprocessing [optional]
[More Information Needed]
#### Training Hyperparameters
- **Training regime:** [More Information Needed]
#### Speeds, Sizes, Times [optional]
[More Information Needed]
## Evaluation
### Testing Data, Factors & Metrics
#### Testing Data
[More Information Needed]
#### Factors
[More Information Needed]
#### Metrics
[More Information Needed]
### Results
[More Information Needed]
#### Summary
## Model Examination [optional]
[More Information Needed]
## Environmental Impact
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
- **Hardware Type:** [More Information Needed]
- **Hours used:** [More Information Needed]
- **Cloud Provider:** [More Information Needed]
- **Compute Region:** [More Information Needed]
- **Carbon Emitted:** [More Information Needed]
## Technical Specifications [optional]
### Model Architecture and Objective
[More Information Needed]
### Compute Infrastructure
[More Information Needed]
#### Hardware
[More Information Needed]
#### Software
[More Information Needed]
## Citation [optional]
**BibTeX:**
[More Information Needed]
**APA:**
[More Information Needed]
## Glossary [optional]
[More Information Needed]
## More Information [optional]
[More Information Needed]
## Model Card Authors [optional]
[More Information Needed]
## Model Card Contact
[More Information Needed]