README.md

---
library_name: transformers
language:
- en
- fa
pipeline_tag: text-generation
---
# Intro

First of all, we would like to express our gratitude to [PartAI](https://huggingface.co/PartAI) for their efforts in expanding large language models in the Persian language by releasing the ["Dorna"](https://huggingface.co/PartAI/Dorna-Llama3-8B-Instruct) model.

The quantized version of the "Dorna" language model requires only 6GB of GPU memory for loading, while the original model requires 40GB of GPU memory.

This model based on AWQ quantize method that decrease the volume of model in minimum decrease of Accuracy by changing type of weights




## Usage

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM


tokenizer = AutoTokenizer.from_pretrained("amir-ma71/Dorna-Llama3-8B-Instruct-AWQ")
model = AutoModelForCausalLM.from_pretrained(
    "amir-ma71/Dorna-Llama3-8B-Instruct-AWQ",
    torch_dtype=torch.bfloat16,
    device_map="cuda",
)

messages = [
    {"role": "system",
     "content": "You are a helpful Persian assistant. Please answer questions in the asked language."},
    {"role": "user", "content": "پایتخت ایران کجاست؟"},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

terminators = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = model.generate(
    input_ids,
    max_new_tokens=256,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
```
## Output
```bash
پایتخت ایران، تهران است. تهران شهر بزرگ و مهمی در مرکز ایران است که جمعیت قابل توجهی دارد و از مهم‌ترین شهرهای این کشور است.
```

## Contributing

feel free to contact me:

* email: aliabadi.am@gmail.com

* [linkedin](https://www.linkedin.com/in/amir-mohammad-aliabadi-038107125/)