metadata

language:
  - en
license: apache-2.0
tags:
  - transformers
  - unsloth
  - llama
  - trl
  - sft
  - peft
base_model: unsloth/llama-3-8b-bnb-4bit
library_name: peft
datasets:
  - myzens/alpaca-turkish-combined

Llama 3-8B Turkish Model

This repo contains the fine-tuned model for the Turkish Llama 3 Project and its variants that can be used for different purposes.

The actual trained model is an adapter model of Unsloth's Llama 3-8B quantized model, which is then converted into .gguf format using llama.cpp and into .bin format for vLLM.

You can access the fine-tuning code below.

Example Usage

You can use the adapter model with PEFT.

from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("unsloth/llama-3-8b-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "myzens/llama3-8b-tr-finetuned")
tokenizer = AutoTokenizer.from_pretrained("myzens/llama3-8b-tr-finetuned")

alpaca_prompt = """
Instruction:
{}

Input:
{}

Response:
{}"""

inputs = tokenizer([
    alpaca_prompt.format(
        "",
        "Ankara'da gezilebilecek 3 yeri söyle ve ne olduklarını kısaca açıkla.",
        "",
)], return_tensors = "pt").to("cuda")


outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Output:

`Instruction:

Input: Ankara'da gezilebilecek 3 yeri söyle ve ne olduklarını kısaca açıkla.

Response:

Anıtkabir - Mustafa Kemal Atatürk'ün mezarı
Gençlik ve Spor Sarayı - spor etkinliklerinin yapıldığı yer
Kızılay Meydanı - Ankara'nın merkezinde bulunan bir meydan`

Important Notes

We recommend you to use an Alpaca Prompt Template or another template, otherwise you can see generations with no meanings or repeating the same sentence constantly.
Use the model with a CUDA supported GPU.