README.md

---
language:
- en
library_name: transformers
tags:
- orpo
- llama 3
- sft
datasets:
- Open-Orca/OpenOrca
---

# Model description

Meta-Llama-3-8B-OpenOrca is a fine-tuned version of the [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) on 1.5k subsamples of the 
[OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca) dataset. 

This model follows the ChatML chat template!

## How to use

````
import torch
from transformers import AutoTokenizer, pipeline

model = "MuntasirHossain/Meta-Llama-3-8B-OpenOrca"
tokenizer = AutoTokenizer.from_pretrained(model)

llm = pipeline(
    task = "text-generation",
    model=model,
    eos_token_id=tokenizer.eos_token_id,
    torch_dtype=torch.float16,
    max_new_tokens=256,
    do_sample=True,
    device_map="auto",
)

def generate(input_text):
  system_prompt = "You are a helpful AI assistant."
  messages = [
      {"role": "system", "content": system_prompt},
      {"role": "user", "content": input_text},
      ]
  prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
  outputs = llm(prompt)
  return outputs[0]["generated_text"][len(prompt):]

generate("What is a large language model?")
````