README.md

---
library_name: transformers
license: other
datasets:
- teknium/OpenHermes-2.5
- LDJnr/Capybara
- Intel/orca_dpo_pairs
- argilla/distilabel-intel-orca-dpo-pairs
language:
- en
---

# Quyen
<img src="quyen.webp" width="512" height="512" alt="Quyen">

# Model Description
Quyen is our first flagship LLM series based on the Qwen1.5 family. We introduced 6 different versions:

- **Quyen-SE (0.5B)**
- **Quyen-Mini (1.8B)**
- **Quyen (4B)**
- **Quyen-Plus (7B)**
- **Quyen-Pro (14B)**
- **Quyen-Pro-Max (72B)**

All models were trained with SFT and DPO using the following dataset:

- *OpenHermes-2.5* by **Teknium**
- *Capyabara* by **LDJ**
- *distilabel-intel-orca-dpo-pairs* by **argilla**
- *orca_dpo_pairs* by **Intel**
- and Private Data by **Ontocord** & **BEE-spoke-data**

# Prompt Template
- All Quyen models use ChatML as the default template:

```
<|im_start|>system
You are a sentient, superintelligent artificial general intelligence, here to teach and assist me.<|im_end|>
<|im_start|>user
Hello world.<|im_end|>
<|im_start|>assistant
```

- You can also use `apply_chat_template`:

```python
messages = [
    {"role": "system", "content": "You are a sentient, superintelligent artificial general intelligence, here to teach and assist me."},
    {"role": "user", "content": "Hello world."}
]
gen_input = tokenizer.apply_chat_template(message, return_tensors="pt")
model.generate(**gen_input)
```

# Benchmarks:

- Coming Soon! We will update the benchmarks later

# Acknowledgement
- We're incredibly grateful to **Tensoic** and **Ontocord** for their generous support with compute and data preparation.
- We want to say a special thank you to the **Qwen** team for the amazing base model and allowing us to get access to the models early.