---
license: other
license_name: qwen
language:
- th
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- openthaigpt
- qwen
---
# ðđð OpenThaiGPT 72b 1.5 Instruct
[More Info](https://openthaigpt.aieat.or.th/)
ðđð **OpenThaiGPT 72b Version 1.5** is an advanced 72-billion-parameter Thai language chat model based on Qwen v2.5 released on September 30, 2024. It has been specifically fine-tuned on over 2,000,000 Thai instruction pairs and is capable of answering Thai-specific domain questions.
## Highlights
- **State-of-the-art Thai language LLM**, achieving the highest average scores across various Thai language exams compared to other open-source Thai LLMs.
- **Multi-turn conversation support** for extended dialogues.
- **Retrieval Augmented Generation (RAG) compatibility** for enhanced response generation.
- **Impressive context handling**: Processes up to 131,072 tokens of input and generates up to 8,192 tokens, enabling detailed and complex interactions.
## Benchmark on [OpenThaiGPT Eval](https://huggingface.co/datasets/openthaigpt/openthaigpt_eval)
** Please take a look at ``openthaigpt/openthaigpt1.5-72b-instruct`` for this model's evaluation result.
| **Exam names** | **scb10x/llama-3-typhoon-v1.5x-70b-instruct** | **meta-llama/Llama-3.1-70B-Instruct** | **Qwen/Qwen2.5-72B-Instruct** | **openthaigpt/openthaigpt1.5-72b-instruct** |
|:------------------------------:|:---------------------------------------------:|:-------------------------------------:|:-----------------------------:|:----------------------------------:|
| **01_a_level** | 59.17% | 61.67% | 75.00% | 76.67% |
| **02_tgat** | 46.00% | 40.00% | 48.00% | 46.00% |
| **03_tpat1** | 52.50% | 50.00% | 55.00% | 55.00% |
| **04_investment_consult** | 60.00% | 52.00% | 80.00% | 72.00% |
| **05_facebook_beleble_th_200** | 87.50% | 88.00% | 90.00% | 90.00% |
| **06_xcopa_th_200** | 84.50% | 85.50% | 90.00% | 90.50% |
| **07_xnli2.0_th_200** | 62.50% | 63.00% | 65.50% | 70.50% |
| **08_onet_m3_thai** | 76.00% | 56.00% | 76.00% | 84.00% |
| **09_onet_m3_social** | 95.00% | 95.00% | 90.00% | 95.00% |
| **10_onet_m3_math** | 43.75% | 25.00% | 37.50% | 37.50% |
| **11_onet_m3_science** | 53.85% | 61.54% | 65.38% | 73.08% |
| **12_onet_m3_english** | 93.33% | 93.33% | 96.67% | 96.67% |
| **13_onet_m6_thai** | 55.38% | 60.00% | 60.00% | 56.92% |
| **14_onet_m6_math** | 41.18% | 58.82% | 23.53% | 41.18% |
| **15_onet_m6_social** | 67.27% | 76.36% | 63.64% | 65.45% |
| **16_onet_m6_science** | 50.00% | 57.14% | 64.29% | 67.86% |
| **17_onet_m6_english** | 73.08% | 82.69% | 86.54% | 90.38% |
| **Micro Average** | 69.97% | 71.09% | 75.02% | 76.73% |
Thai language multiple choice exams, Test on unseen test set, Zero-shot learning. Benchmark source code and exams information: https://github.com/OpenThaiGPT/openthaigpt_eval
(Updated on: 30 September 2024)
## Benchmark on [scb10x/thai_exam](https://huggingface.co/datasets/scb10x/thai_exam)
| Models | **Thai Exam (Acc)** |
|:----------------------------------------------------------:|:-------------------:|
| **api/claude-3-5-sonnet-20240620** | 69.2 |
| **openthaigpt/openthaigpt1.5-72b-instruct*** | 64.07 |
| **api/gpt-4o-2024-05-13** | 63.89 |
| **hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4** | 63.54 |
| **Qwen/Qwen2-72B-Instruct** | 58.23 |
| **meta-llama/Meta-Llama-3.1-70B-Instruct** | 58.23 |
| **scb10x/llama-3-typhoon-v1.5x-70b-instruct** | 58.76 |
| **Qwen/Qwen2.5-14B-Instruct** | 57.35 |
| **api/gpt-4o-mini-2024-07-18** | 54.51 |
| **openthaigpt/openthaigpt1.5-7b-instruct*** | 52.04 |
| **SeaLLMs/SeaLLMs-v3-7B-Chat** | 51.33 |
| **openthaigpt/openthaigpt-1.0.0-70b-chat** | 50.09 |
* Evaluated by OpenThaiGPT team using [scb10x/thai_exam](https://huggingface.co/datasets/scb10x/thai_exam).
## Licenses
* Built with Qwen
* Qwen License: Allow **Research** and **Commercial uses** but if your user base exceeds 100 million monthly active users, you need to negotiate a separate commercial license. Please see LICENSE file for more information.
## Sponsors
## Supports
- Official website: https://openthaigpt.aieat.or.th
- Facebook page: https://web.facebook.com/groups/openthaigpt
- A Discord server for discussion and support [here](https://discord.gg/rUTp6dfVUF)
- E-mail: kobkrit@aieat.or.th
## Prompt Format
Prompt format is based on ChatML.
```
<|im_start|>system\n{sytem_prompt}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n
```
### System prompt:
```
āļāļļāļāļāļ·āļāļāļđāđāļāđāļ§āļĒāļāļāļāļāļģāļāļēāļĄāļāļĩāđāļāļĨāļēāļāđāļĨāļ°āļāļ·āđāļāļŠāļąāļāļĒāđ
```
### Examples
#### Single Turn Conversation Example
```
<|im_start|>system\nāļāļļāļāļāļ·āļāļāļđāđāļāđāļ§āļĒāļāļāļāļāļģāļāļēāļĄāļāļĩāđāļāļĨāļēāļāđāļĨāļ°āļāļ·āđāļāļŠāļąāļāļĒāđ<|im_end|>\n<|im_start|>user\nāļŠāļ§āļąāļŠāļāļĩāļāļĢāļąāļ<|im_end|>\n<|im_start|>assistant\n
```
#### Single Turn Conversation with Context (RAG) Example
```
<|im_start|>system\nāļāļļāļāļāļ·āļāļāļđāđāļāđāļ§āļĒāļāļāļāļāļģāļāļēāļĄāļāļĩāđāļāļĨāļēāļāđāļĨāļ°āļāļ·āđāļāļŠāļąāļāļĒāđ<|im_end|>\n<|im_start|>user\nāļāļĢāļļāļāđāļāļāļĄāļŦāļēāļāļāļĢ āđāļāđāļāđāļĄāļ·āļāļāļŦāļĨāļ§āļ āļāļāļĢāđāļĨāļ°āļĄāļŦāļēāļāļāļĢāļāļĩāđāļĄāļĩāļāļĢāļ°āļāļēāļāļĢāļĄāļēāļāļāļĩāđāļŠāļļāļāļāļāļāļāļĢāļ°āđāļāļĻāđāļāļĒ āļāļĢāļļāļāđāļāļāļĄāļŦāļēāļāļāļĢāļĄāļĩāļāļ·āđāļāļāļĩāđāļāļąāđāļāļŦāļĄāļ 1,568.737 āļāļĢ.āļāļĄ. āļĄāļĩāļāļĢāļ°āļāļēāļāļĢāļāļēāļĄāļāļ°āđāļāļĩāļĒāļāļĢāļēāļĐāļāļĢāļāļ§āđāļē 8 āļĨāđāļēāļāļāļ\nāļāļĢāļļāļāđāļāļāļĄāļŦāļēāļāļāļĢāļĄāļĩāļāļ·āđāļāļāļĩāđāđāļāđāļēāđāļĢāđ<|im_end|>\n<|im_start|>assistant\n
```
#### Multi Turn Conversation Example
##### First turn
```
<|im_start|>system\nāļāļļāļāļāļ·āļāļāļđāđāļāđāļ§āļĒāļāļāļāļāļģāļāļēāļĄāļāļĩāđāļāļĨāļēāļāđāļĨāļ°āļāļ·āđāļāļŠāļąāļāļĒāđ<|im_end|>\n<|im_start|>user\nāļŠāļ§āļąāļŠāļāļĩāļāļĢāļąāļ<|im_end|>\n<|im_start|>assistant\n
```
##### Second turn
```
<|im_start|>system\nāļāļļāļāļāļ·āļāļāļđāđāļāđāļ§āļĒāļāļāļāļāļģāļāļēāļĄāļāļĩāđāļāļĨāļēāļāđāļĨāļ°āļāļ·āđāļāļŠāļąāļāļĒāđ<|im_end|>\n<|im_start|>user\nāļŠāļ§āļąāļŠāļāļĩāļāļĢāļąāļ<|im_end|>\n<|im_start|>assistant\nāļŠāļ§āļąāļŠāļāļĩāļāļĢāļąāļ āļĒāļīāļāļāļĩāļāđāļāļāļĢāļąāļāļāļĢāļąāļ āļāļļāļāļāđāļāļāļāļēāļĢāđāļŦāđāļāļąāļāļāđāļ§āļĒāļāļ°āđāļĢāļāļĢāļąāļ?<|im_end|>\n<|im_start|>user\nāļāļĢāļļāļāđāļāļāļĄāļŦāļēāļāļāļĢ āļāļ·āđāļāđāļāđāļĄāļĒāļēāļ§āđāļāļ·āļāļāļ°āđāļĢ<|im_end|>\n<|im_start|>assistant\n
```
##### Result
```
<|im_start|>system\nāļāļļāļāļāļ·āļāļāļđāđāļāđāļ§āļĒāļāļāļāļāļģāļāļēāļĄāļāļĩāđāļāļĨāļēāļāđāļĨāļ°āļāļ·āđāļāļŠāļąāļāļĒāđ<|im_end|>\n<|im_start|>user\nāļŠāļ§āļąāļŠāļāļĩāļāļĢāļąāļ<|im_end|>\n<|im_start|>assistant\nāļŠāļ§āļąāļŠāļāļĩāļāļĢāļąāļ āļĒāļīāļāļāļĩāļāđāļāļāļĢāļąāļāļāļĢāļąāļ āļāļļāļāļāđāļāļāļāļēāļĢāđāļŦāđāļāļąāļāļāđāļ§āļĒāļāļ°āđāļĢāļāļĢāļąāļ?<|im_end|>\n<|im_start|>user\nāļāļĢāļļāļāđāļāļāļĄāļŦāļēāļāļāļĢ āļāļ·āđāļāđāļāđāļĄāļĒāļēāļ§āđāļāļ·āļāļāļ°āđāļĢ<|im_end|>\n<|im_start|>assistant\nāļāļ·āđāļāđāļāđāļĄāļāļāļāļāļĢāļļāļāđāļāļāļĄāļŦāļēāļāļāļĢāļāļ·āļ \"āļāļĢāļļāļāđāļāļāļĄāļŦāļēāļāļāļĢ āļāļĄāļĢāļĢāļąāļāļāđāļāļŠāļīāļāļāļĢāđ āļĄāļŦāļīāļāļāļĢāļēāļĒāļļāļāļĒāļē āļĄāļŦāļēāļāļīāļĨāļāļ āļ āļāļāļĢāļąāļāļāļĢāļēāļāļāļēāļāļĩāļāļđāļĢāļĩāļĢāļĄāļĒāđ āļāļļāļāļĄāļĢāļēāļāļāļīāđāļ§āļĻāļāđāļĄāļŦāļēāļŠāļāļēāļ āļāļĄāļĢāļāļīāļĄāļēāļāļāļ§āļāļēāļĢāļŠāļāļīāļ āļŠāļąāļāļāļ°āļāļąāļāļāļīāļĒāļ§āļīāļĐāļāļļāļāļĢāļĢāļĄāļāļĢāļ°āļŠāļīāļāļāļīāđ\"
```
## How to use
### Huggingface
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "openthaigpt/openthaigpt1.5-72b-instruct"
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
prompt = "āļāļĢāļ°āđāļāļĻāđāļāļĒāļāļ·āļāļāļ°āđāļĢ"
messages = [
{"role": "system", "content": "āļāļļāļāļāļ·āļāļāļđāđāļāđāļ§āļĒāļāļāļāļāļģāļāļēāļĄāļāļĩāđāļāļĨāļēāļāđāļĨāļ°āļāļ·āđāļāļŠāļąāļāļĒāđ"},
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
generated_ids = model.generate(
**model_inputs,
max_new_tokens=512
)
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
```
### vLLM
1. Install VLLM (https://github.com/vllm-project/vllm)
2. Run server
```bash
vllm serve openthaigpt/openthaigpt1.5-72b-instruct --tensor-parallel-size 4
```
* Note, change ``--tensor-parallel-size 4`` to the amount of available GPU cards.
3. Run inference (CURL example)
```bash
curl -X POST 'http://127.0.0.1:8000/v1/completions' \
-H 'Content-Type: application/json' \
-d '{
"model": ".",
"prompt": "<|im_start|>system\nāļāļļāļāļāļ·āļāļāļđāđāļāđāļ§āļĒāļāļāļāļāļģāļāļēāļĄāļāļĩāđāļāļĨāļēāļāđāļĨāļ°āļāļ·āđāļāļŠāļąāļāļĒāđ<|im_end|>\n<|im_start|>user\nāļŠāļ§āļąāļŠāļāļĩāļāļĢāļąāļ<|im_end|>\n<|im_start|>assistant\n",
"max_tokens": 512,
"temperature": 0.7,
"top_p": 0.8,
"top_k": 40,
"stop": ["<|im_end|>"]
}'
```
### Processing Long Texts
The current `config.json` is set for context length up to 32,768 tokens.
To handle extensive inputs exceeding 32,768 tokens, we utilize [YaRN](https://arxiv.org/abs/2309.00071), a technique for enhancing model length extrapolation, ensuring optimal performance on lengthy texts.
For supported frameworks, you could add the following to `config.json` to enable YaRN:
```json
{
...,
"rope_scaling": {
"factor": 4.0,
"original_max_position_embeddings": 32768,
"type": "yarn"
}
}
```
### GPU Memory Requirements
| **Number of Parameters** | **FP 16 bits** | **8 bits (Quantized)** | **4 bits (Quantized)** | **Example Graphic Card for 4 bits** |
|------------------|----------------|------------------------|------------------------|---------------------------------------------|
| **7b** | 24 GB | 12 GB | 6 GB | Nvidia RTX 4060 8GB |
| **13b** | 48 GB | 24 GB | 12 GB | Nvidia RTX 4070 16GB |
| **72b** | 192 GB | 96 GB | 48 GB | Nvidia RTX 4090 24GB x 2 cards |
### Authors
* Sumeth Yuenyong (sumeth.yue@mahidol.edu)
* Kobkrit Viriyayudhakorn (kobkrit@aieat.or.th)
* Apivadee Piyatumrong (apivadee.piy@nectec.or.th)
* Jillaphat Jaroenkantasima (autsadang41@gmail.com)
* Thaweewat Rugsujarit (thaweewr@scg.com)
* Norapat Buppodom (new@norapat.com)
* Koravich Sangkaew (kwankoravich@gmail.com)
* Peerawat Rojratchadakorn (peerawat.roj@gmail.com)
* Surapon Nonesung (nonesungsurapon@gmail.com)
* Chanon Utupon (chanon.utupon@gmail.com)
* Sadhis Wongprayoon (sadhis.tae@gmail.com)
* Nucharee Thongthungwong (nuchhub@hotmail.com)
* Chawakorn Phiantham (mondcha1507@gmail.com)
* Patteera Triamamornwooth (patt.patteera@gmail.com)
* Nattarika Juntarapaoraya (natt.juntara@gmail.com)
* Kriangkrai Saetan (kraitan.ss21@gmail.com)
* Pitikorn Khlaisamniang (pitikorn32@gmail.com)
Disclaimer: Provided responses are not guaranteed.