Morfoz-LLM-8b-v1.0

This model is an extended version of a Llama-3 8B Instruct-based Large Language Model (LLM) for Turkish. It was trained on a cleaned Turkish raw dataset. We utilized Turkish instruction sets created from various open-source for fine-tuning with the LORA method.

Model Details

• Base Model: Meta Llama 3 8B Instruct
• Tokenizer Extension: Specifically extended for Turkish
• Training Dataset: Cleaned Turkish raw data with custom Turkish instruction sets
• Training Method: Fine-tuning with LORA

LORA Fine-Tuning Configuration

• lora_alpha: 16
• lora_dropout: 0.05
• r: 64
• target_modules: "all-linear"

Usage Examples

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("Morfoz-Aigap/Morfoz-LLM-8b-v1.0")
model = AutoModelForCausalLM.from_pretrained("Morfoz-Aigap/Morfoz-LLM-8b-v1.0", torch_dtype=torch.bfloat16, device_map={"": 0},low_cpu_mem_usage=True)

messages = [
    {"role": "user", "content": "Kırmızı başlıklı kız adında kısa bir çocuk hikayesi yazabilir misin?"}

]

top_k = 50
top_p = 0.9
temperature = 0.6
def get_formatted_input(messages):

    for item in messages:
        if item['role'] == "user":
            item['content'] = item['content']
            break

    conversation = '\n\n'.join(["User: " + item["content"] if item["role"] == "user" else "Assistant: " + item["content"] for item in messages]) + "\n\nAssistant:"
    formatted_input = "\n\n" + conversation

    return formatted_input

formatted_input = get_formatted_input(messages)
print(formatted_input)
tokenized_prompt = tokenizer(tokenizer.bos_token + formatted_input, return_tensors="pt").to(model.device)

terminators = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = model.generate(input_ids=tokenized_prompt.input_ids, do_sample = True, attention_mask=tokenized_prompt.attention_mask, max_new_tokens=256, eos_token_id=terminators, top_p=top_p, temperature=temperature)

response = outputs[0][tokenized_prompt.input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))