Update README.md
Browse files
README.md
CHANGED
@@ -7,7 +7,7 @@ license: llama3
|
|
7 |
---
|
8 |
**Llama-3-Typhoon-1.5X-70B-instruct: Thai Large Language Model (Instruct)**
|
9 |
|
10 |
-
**Llama-3-Typhoon-1.5X-70B-instruct** is a 70 billion parameter instruct model designed for Thai 🇹🇭 language. It demonstrates competitive performance with GPT-4-0612, and is optimized for **
|
11 |
|
12 |
Built on Typhoon 1.5 70B (not yet released) and Llama 3 70B Instruct. this model is a result of our experiment on **cross-lingual transfer**. It utilizes the [task-arithmetic model editing](https://arxiv.org/abs/2212.04089) technique, combining the Thai understanding capability of Typhoon with the human alignment performance of Llama 3 Instruct.
|
13 |
|
@@ -27,15 +27,15 @@ We evaluated the model's performance in **Language & Knowledge Capabilities** an
|
|
27 |
- **Language & Knowledge Capabilities**:
|
28 |
- Assessed using multiple-choice question-answering datasets such as ThaiExam and MMLU.
|
29 |
- **Instruction Following Capabilities**:
|
30 |
-
- Evaluated based on
|
31 |
-
- **Human Alignment & Reasoning**: Ability to generate responses that are
|
32 |
-
- Evaluated using [MT-Bench](https://arxiv.org/abs/2306.05685) — How LLMs can
|
33 |
-
- **Instruction-following**: Ability to adhere to specified constraints in the
|
34 |
- Evaluated using [IFEval](https://arxiv.org/abs/2311.07911) — How LLMs can follow specified constraints, such as formatting and brevity.
|
35 |
-
- **Agentic
|
36 |
-
- Evaluated in agent use-cases using [Hugging Face's
|
37 |
|
38 |
-
Remark: We developed the TH
|
39 |
|
40 |
### ThaiExam
|
41 |
|
@@ -80,7 +80,7 @@ Remark: We developed the TH pair by translating the original datasets into Thai
|
|
80 |
|
81 |
## Insight
|
82 |
|
83 |
-
|
84 |
|
85 |
## **Usage Example**
|
86 |
|
@@ -144,7 +144,7 @@ This model is experimental and might not be fully evaluated for all use cases. D
|
|
144 |
|
145 |
## **SCB 10X Typhoon Team**
|
146 |
|
147 |
-
- Kunat Pipatanakul, Potsawee Manakul, Sittipong Sripaisarnmongkol, Pathomporn Chokchainant, Kasima Tharnpipitchai
|
148 |
- If you find Typhoon-1.5X useful for your work, please cite it using:
|
149 |
|
150 |
```
|
|
|
7 |
---
|
8 |
**Llama-3-Typhoon-1.5X-70B-instruct: Thai Large Language Model (Instruct)**
|
9 |
|
10 |
+
**Llama-3-Typhoon-1.5X-70B-instruct** is a 70 billion parameter instruct model designed for Thai 🇹🇭 language. It demonstrates competitive performance with GPT-4-0612, and is optimized for **application** use cases, **Retrieval-Augmented Generation (RAG), constrained generation**, and **reasoning** tasks.
|
11 |
|
12 |
Built on Typhoon 1.5 70B (not yet released) and Llama 3 70B Instruct. this model is a result of our experiment on **cross-lingual transfer**. It utilizes the [task-arithmetic model editing](https://arxiv.org/abs/2212.04089) technique, combining the Thai understanding capability of Typhoon with the human alignment performance of Llama 3 Instruct.
|
13 |
|
|
|
27 |
- **Language & Knowledge Capabilities**:
|
28 |
- Assessed using multiple-choice question-answering datasets such as ThaiExam and MMLU.
|
29 |
- **Instruction Following Capabilities**:
|
30 |
+
- Evaluated based on beta users' feedback, focusing on two factors:
|
31 |
+
- **Human Alignment & Reasoning**: Ability to generate responses that are clear and logically structured across multiple steps.
|
32 |
+
- Evaluated using [MT-Bench](https://arxiv.org/abs/2306.05685) — How LLMs can align with human needs.
|
33 |
+
- **Instruction-following**: Ability to adhere to specified constraints in the instructions.
|
34 |
- Evaluated using [IFEval](https://arxiv.org/abs/2311.07911) — How LLMs can follow specified constraints, such as formatting and brevity.
|
35 |
+
- **Agentic Capabilities**:
|
36 |
+
- Evaluated in agent use-cases using [Hugging Face's Transformer Agents](https://huggingface.co/blog/agents) and the associated [benchmark](https://huggingface.co/blog/open-source-llms-as-agents).
|
37 |
|
38 |
+
Remark: We developed the Thai (TH) pairs by translating the original datasets into Thai through machine and human methods.
|
39 |
|
40 |
### ThaiExam
|
41 |
|
|
|
80 |
|
81 |
## Insight
|
82 |
|
83 |
+
We utilized **model editing** techniques and found that the most critical feature for generating accurate Thai answers is located in the backend (the upper layers of the transformer block). Accordingly, we incorporated a high ratio of Typhoon components in these backend layers to enhance our model’s performance.
|
84 |
|
85 |
## **Usage Example**
|
86 |
|
|
|
144 |
|
145 |
## **SCB 10X Typhoon Team**
|
146 |
|
147 |
+
- Kunat Pipatanakul, Potsawee Manakul, Sittipong Sripaisarnmongkol, Natapong Nitarach, Pathomporn Chokchainant, Kasima Tharnpipitchai
|
148 |
- If you find Typhoon-1.5X useful for your work, please cite it using:
|
149 |
|
150 |
```
|