File size: 3,599 Bytes
d3f870e 7564663 02d1762 d3f870e 02d1762 3fe08a7 443103f d3f870e 72adf1d d3f870e c1bf985 d3f870e 11822f6 72adf1d 364054f 9d1e250 72adf1d bd7197a 72adf1d a7b78a9 6277244 ca2b2f4 6277244 bd7197a f9290c4 ca2b2f4 a7b78a9 ca2b2f4 a7b78a9 72adf1d ca2b2f4 02475fe |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 |
---
language:
- en
license: apache-2.0
tags:
- transformers
- unsloth
- llama
- trl
- sft
- peft
base_model: unsloth/llama-3-8b-bnb-4bit
library_name: peft
datasets:
- myzens/alpaca-turkish-combined
---
# Llama 3-8B Turkish Model
This repo contains the experimental-educational fine-tuned model for the Turkish Llama 3 Project and its variants that can be used for different purposes.
The actual trained model is an adapter model of [Unsloth's Llama 3-8B quantized model](https://huggingface.co/unsloth/llama-3-8b-bnb-4bit), which is then converted into .gguf format using llama.cpp and into .bin format for vLLM.
The model is open to further development, we will continue to train the model when we obtain quality data. We can't use every Turkish dataset since some of them has poor quality of translation from English.
You can access the fine-tuning code [here](https://colab.research.google.com/drive/1QRaqYxjfnFvwA_9jb7V0Z5bJr-PuHH7w?usp=sharing).
Trained with NVIDIA L4 with 150 steps, took around 8 minutes.
## Example Usages
You can use the adapter model with PEFT.
```py
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model = AutoModelForCausalLM.from_pretrained("unsloth/llama-3-8b-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "myzens/llama3-8b-tr-finetuned")
tokenizer = AutoTokenizer.from_pretrained("myzens/llama3-8b-tr-finetuned")
alpaca_prompt = """
Instruction:
{}
Input:
{}
Response:
{}"""
inputs = tokenizer([
alpaca_prompt.format(
"",
"Ankara'da gezilebilecek 3 yeri söyle ve ne olduklarını kısaca açıkla.",
"",
)], return_tensors = "pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
You can use it from Transformers:
```py
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("myzens/llama3-8b-tr-finetuned")
model = AutoModelForCausalLM.from_pretrained("myzens/llama3-8b-tr-finetuned")
alpaca_prompt = """
Instruction:
{}
Input:
{}
Response:
{}"""
inputs = tokenizer([
alpaca_prompt.format(
"",
"Ankara'da gezilebilecek 3 yeri söyle ve ne olduklarını kısaca açıkla.",
"",
)], return_tensors = "pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=192)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
Transformers Pipeline:
```py
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
tokenizer = AutoTokenizer.from_pretrained("myzens/llama3-8b-tr-finetuned")
model = AutoModelForCausalLM.from_pretrained("myzens/llama3-8b-tr-finetuned")
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
alpaca_prompt = """
Instruction:
{}
Input:
{}
Response:
{}"""
input = alpaca_prompt.format(
"",
"Ankara'da gezilebilecek 3 yeri söyle ve ne olduklarını kısaca açıkla.",
"",
)
pipe(input)
```
Output:
```
Instruction:
Input:
Ankara'da gezilebilecek 3 yeri söyle ve ne olduklarını kısaca açıkla.
Response:
1. Anıtkabir - Mustafa Kemal Atatürk'ün mezarı
2. Gençlik ve Spor Sarayı - spor etkinliklerinin yapıldığı yer
3. Kızılay Meydanı - Ankara'nın merkezinde bulunan bir meydan
```
### **Important Notes**
- We recommend you to use an Alpaca Prompt Template or another template, otherwise you can see generations with no meanings or repeating the same sentence constantly.
- Use the model with a CUDA supported GPU.
Fine-tuned by [emre570](https://github.com/emre570). |