Hugging Face's logo

openthaigpt
/
openthaigpt1.5-7b-instruct

Text Generation
Transformers
Safetensors
GGUF
Thai
English
qwen2
openthaigpt
qwen
conversational
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Community
kobkrit commited on
Commit
102f450
•
1 Parent(s): 20c26dd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -25
README.md CHANGED
@@ -11,11 +11,11 @@ tags:
11
  - qwen
12
  ---
13
 
14
- # 🇹🇭 OpenThaiGPT 7b 1.5.0 Chat
15
  ![OpenThaiGPT](https://1173516064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvvbWvIIe82Iv1yHaDBC5%2Fuploads%2Fb8eiMDaqiEQL6ahbAY0h%2Fimage.png?alt=media&token=6fce78fd-2cca-4c0a-9648-bd5518e644ce)
16
  [More Info](https://openthaigpt.aieat.or.th/)
17
 
18
- 🇹🇭 **OpenThaiGPT 7b Version 1.5.0** is an advanced 7-billion-parameter Thai language chat model based on Qwen v2.5 released on September 30, 2024. It has been specifically fine-tuned on over 2,000,000 Thai instruction pairs and is capable of answering Thai-specific domain questions.
19
 
20
  ## Highlights
21
  - **State-of-the-art Thai language LLM**, achieving the highest average scores across various Thai language exams compared to other open-source Thai LLMs.
@@ -25,28 +25,26 @@ tags:
25
 
26
  ## Benchmark on [OpenThaiGPT Eval](https://huggingface.co/datasets/openthaigpt/openthaigpt_eval)
27
  ** Please take a look at ``openthaigpt/openthaigpt1.5-7b-instruct`` for this model's evaluation result.
28
- | **Exam names** | **scb10x/llama-3-typhoon-v1.5x-70b-instruct** | **meta-llama/Llama-3.1-70B-Instruct** | **Qwen/Qwen2.5-72B-Instruct** | **openthaigpt/openthaigpt1.5-72b-instruct** |
29
- |:------------------------------:|:---------------------------------------------:|:-------------------------------------:|:-----------------------------:|:----------------------------------:|
30
- | **01_a_level** | 59.17% | 61.67% | 75.00% | 76.67% |
31
- | **02_tgat** | 46.00% | 40.00% | 48.00% | 46.00% |
32
- | **03_tpat1** | 52.50% | 50.00% | 55.00% | 55.00% |
33
- | **04_investment_consult** | 60.00% | 52.00% | 80.00% | 72.00% |
34
- | **05_facebook_beleble_th_200** | 87.50% | 88.00% | 90.00% | 90.00% |
35
- | **06_xcopa_th_200** | 84.50% | 85.50% | 90.00% | 90.50% |
36
- | **07_xnli2.0_th_200** | 62.50% | 63.00% | 65.50% | 70.50% |
37
- | **08_onet_m3_thai** | 76.00% | 56.00% | 76.00% | 84.00% |
38
- | **09_onet_m3_social** | 95.00% | 95.00% | 90.00% | 95.00% |
39
- | **10_onet_m3_math** | 43.75% | 25.00% | 37.50% | 37.50% |
40
- | **11_onet_m3_science** | 53.85% | 61.54% | 65.38% | 73.08% |
41
- | **12_onet_m3_english** | 93.33% | 93.33% | 96.67% | 96.67% |
42
- | **13_onet_m6_thai** | 55.38% | 60.00% | 60.00% | 56.92% |
43
- | **14_onet_m6_math** | 41.18% | 58.82% | 23.53% | 41.18% |
44
- | **15_onet_m6_social** | 67.27% | 76.36% | 63.64% | 65.45% |
45
- | **16_onet_m6_science** | 50.00% | 57.14% | 64.29% | 67.86% |
46
- | **17_onet_m6_english** | 73.08% | 82.69% | 86.54% | 90.38% |
47
- | **Micro Average** | 69.97% | 71.09% | 75.02% | <b style="color:blue">76.73%</b> |
48
-
49
-
50
 
51
 
52
  Thai language multiple choice exams, Test on unseen test set, Zero-shot learning. Benchmark source code and exams information: https://github.com/OpenThaiGPT/openthaigpt_eval
@@ -74,7 +72,8 @@ Thai language multiple choice exams, Test on unseen test set, Zero-shot learning
74
 
75
  ## Licenses
76
  * Built with Qwen
77
- * Qwen License: Allow **Research** and **Commercial uses** but if your user base exceeds 100 million monthly active users, you need to negotiate a separate commercial license. Please see LICENSE file for more information.<br>
 
78
 
79
  ## Sponsors
80
  <img src="https://cdn-uploads.huggingface.co/production/uploads/5fcd9c426d942eaf4d1ebd30/3kjN6kuTzXDXQ6o1RFvHX.png" width="600px">
 
11
  - qwen
12
  ---
13
 
14
+ # 🇹🇭 OpenThaiGPT 7b 1.5 Instruct
15
  ![OpenThaiGPT](https://1173516064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvvbWvIIe82Iv1yHaDBC5%2Fuploads%2Fb8eiMDaqiEQL6ahbAY0h%2Fimage.png?alt=media&token=6fce78fd-2cca-4c0a-9648-bd5518e644ce)
16
  [More Info](https://openthaigpt.aieat.or.th/)
17
 
18
+ 🇹🇭 **OpenThaiGPT 7b Version 1.5** is an advanced 7-billion-parameter Thai language chat model based on Qwen v2.5 released on September 30, 2024. It has been specifically fine-tuned on over 2,000,000 Thai instruction pairs and is capable of answering Thai-specific domain questions.
19
 
20
  ## Highlights
21
  - **State-of-the-art Thai language LLM**, achieving the highest average scores across various Thai language exams compared to other open-source Thai LLMs.
 
25
 
26
  ## Benchmark on [OpenThaiGPT Eval](https://huggingface.co/datasets/openthaigpt/openthaigpt_eval)
27
  ** Please take a look at ``openthaigpt/openthaigpt1.5-7b-instruct`` for this model's evaluation result.
28
+ | **Exam names** | **scb10x/llama-3-typhoon-v1.5x-8b-instruct** | **meta-llama/Llama-3.1-7B-Instruct** | **Qwen/Qwen2.5-7B-Instruct_stat** | **openthaigpt/openthaigpt1.5-7b** |
29
+ |:------------------------------:|:--------------------------------------------:|:------------------------------------:|:---------------------------------:|:---------------------------------:|
30
+ | **01_a_level** | 46.67% | 47.50% | 58.33% | 60.00% |
31
+ | **02_tgat** | 32.00% | 36.00% | 32.00% | 36.00% |
32
+ | **03_tpat1** | 52.50% | 55.00% | 57.50% | 57.50% |
33
+ | **04_investment_consult** | 56.00% | 48.00% | 68.00% | 76.00% |
34
+ | **05_facebook_beleble_th_200** | 78.00% | 73.00% | 79.00% | 81.00% |
35
+ | **06_xcopa_th_200** | 79.50% | 69.00% | 80.50% | 81.00% |
36
+ | **07_xnli2.0_th_200** | 56.50% | 55.00% | 53.00% | 54.50% |
37
+ | **08_onet_m3_thai** | 48.00% | 32.00% | 72.00% | 64.00% |
38
+ | **09_onet_m3_social** | 75.00% | 50.00% | 90.00% | 80.00% |
39
+ | **10_onet_m3_math** | 25.00% | 18.75% | 31.25% | 31.25% |
40
+ | **11_onet_m3_science** | 46.15% | 42.31% | 46.15% | 46.15% |
41
+ | **12_onet_m3_english** | 70.00% | 76.67% | 86.67% | 83.33% |
42
+ | **13_onet_m6_thai** | 47.69% | 29.23% | 46.15% | 53.85% |
43
+ | **14_onet_m6_math** | 29.41% | 17.65% | 29.41% | 29.41% |
44
+ | **15_onet_m6_social** | 50.91% | 43.64% | 56.36% | 58.18% |
45
+ | **16_onet_m6_science** | 42.86% | 32.14% | 57.14% | 57.14% |
46
+ | **17_onet_m6_english** | 65.38% | 71.15% | 78.85% | 80.77% |
47
+ | **Micro Average** | 60.65% | 55.60% | 64.41% | <b style="color:blue">65.78%</b> |
 
 
48
 
49
 
50
  Thai language multiple choice exams, Test on unseen test set, Zero-shot learning. Benchmark source code and exams information: https://github.com/OpenThaiGPT/openthaigpt_eval
 
72
 
73
  ## Licenses
74
  * Built with Qwen
75
+ * Qwen License: Allow **Research** and
76
+ **Commercial uses** but if your user base exceeds 100 million monthly active users, you need to negotiate a separate commercial license. Please see LICENSE file for more information.<br>
77
 
78
  ## Sponsors
79
  <img src="https://cdn-uploads.huggingface.co/production/uploads/5fcd9c426d942eaf4d1ebd30/3kjN6kuTzXDXQ6o1RFvHX.png" width="600px">