kunato commited on
Commit
4f7a3ff
·
verified ·
1 Parent(s): 9e9b63d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -10
README.md CHANGED
@@ -7,7 +7,7 @@ license: llama3
7
  ---
8
  **Llama-3-Typhoon-1.5X-70B-instruct: Thai Large Language Model (Instruct)**
9
 
10
- **Llama-3-Typhoon-1.5X-70B-instruct** is a 70 billion parameter instruct model designed for Thai 🇹🇭 language. It demonstrates competitive performance with GPT-4-0612, and is optimized for **production** environments, **Retrieval-Augmented Generation (RAG), constrained generation**, and **reasoning** tasks.
11
 
12
  Built on Typhoon 1.5 70B (not yet released) and Llama 3 70B Instruct. this model is a result of our experiment on **cross-lingual transfer**. It utilizes the [task-arithmetic model editing](https://arxiv.org/abs/2212.04089) technique, combining the Thai understanding capability of Typhoon with the human alignment performance of Llama 3 Instruct.
13
 
@@ -27,15 +27,15 @@ We evaluated the model's performance in **Language & Knowledge Capabilities** an
27
  - **Language & Knowledge Capabilities**:
28
  - Assessed using multiple-choice question-answering datasets such as ThaiExam and MMLU.
29
  - **Instruction Following Capabilities**:
30
- - Evaluated based on our beta users' feedback, focusing on two factors:
31
- - **Human Alignment & Reasoning**: Ability to generate responses that are understandable and reasoned across multiple steps.
32
- - Evaluated using [MT-Bench](https://arxiv.org/abs/2306.05685) — How LLMs can answer embedded knowledge to align with human needs.
33
- - **Instruction-following**: Ability to adhere to specified constraints in the instruction
34
  - Evaluated using [IFEval](https://arxiv.org/abs/2311.07911) — How LLMs can follow specified constraints, such as formatting and brevity.
35
- - **Agentic** **Capabilities:**
36
- - Evaluated in agent use-cases using [Hugging Face's agent implementation](https://huggingface.co/blog/agents) and the [benchmark](https://huggingface.co/blog/open-source-llms-as-agents).
37
 
38
- Remark: We developed the TH pair by translating the original datasets into Thai and conducting a human verification on them.
39
 
40
  ### ThaiExam
41
 
@@ -80,7 +80,7 @@ Remark: We developed the TH pair by translating the original datasets into Thai
80
 
81
  ## Insight
82
 
83
- Utilized model editing technique. We found that the most critical feature for generating Thai answers is located in the backend (the upper layers of the transformer block). Accordingly, we incorporated a high ratio of Typhoon in these backend layers.
84
 
85
  ## **Usage Example**
86
 
@@ -144,7 +144,7 @@ This model is experimental and might not be fully evaluated for all use cases. D
144
 
145
  ## **SCB 10X Typhoon Team**
146
 
147
- - Kunat Pipatanakul, Potsawee Manakul, Sittipong Sripaisarnmongkol, Pathomporn Chokchainant, Kasima Tharnpipitchai
148
  - If you find Typhoon-1.5X useful for your work, please cite it using:
149
 
150
  ```
 
7
  ---
8
  **Llama-3-Typhoon-1.5X-70B-instruct: Thai Large Language Model (Instruct)**
9
 
10
+ **Llama-3-Typhoon-1.5X-70B-instruct** is a 70 billion parameter instruct model designed for Thai 🇹🇭 language. It demonstrates competitive performance with GPT-4-0612, and is optimized for **application** use cases, **Retrieval-Augmented Generation (RAG), constrained generation**, and **reasoning** tasks.
11
 
12
  Built on Typhoon 1.5 70B (not yet released) and Llama 3 70B Instruct. this model is a result of our experiment on **cross-lingual transfer**. It utilizes the [task-arithmetic model editing](https://arxiv.org/abs/2212.04089) technique, combining the Thai understanding capability of Typhoon with the human alignment performance of Llama 3 Instruct.
13
 
 
27
  - **Language & Knowledge Capabilities**:
28
  - Assessed using multiple-choice question-answering datasets such as ThaiExam and MMLU.
29
  - **Instruction Following Capabilities**:
30
+ - Evaluated based on beta users' feedback, focusing on two factors:
31
+ - **Human Alignment & Reasoning**: Ability to generate responses that are clear and logically structured across multiple steps.
32
+ - Evaluated using [MT-Bench](https://arxiv.org/abs/2306.05685) — How LLMs can align with human needs.
33
+ - **Instruction-following**: Ability to adhere to specified constraints in the instructions.
34
  - Evaluated using [IFEval](https://arxiv.org/abs/2311.07911) — How LLMs can follow specified constraints, such as formatting and brevity.
35
+ - **Agentic Capabilities**:
36
+ - Evaluated in agent use-cases using [Hugging Face's Transformer Agents](https://huggingface.co/blog/agents) and the associated [benchmark](https://huggingface.co/blog/open-source-llms-as-agents).
37
 
38
+ Remark: We developed the Thai (TH) pairs by translating the original datasets into Thai through machine and human methods.
39
 
40
  ### ThaiExam
41
 
 
80
 
81
  ## Insight
82
 
83
+ We utilized **model editing** techniques and found that the most critical feature for generating accurate Thai answers is located in the backend (the upper layers of the transformer block). Accordingly, we incorporated a high ratio of Typhoon components in these backend layers to enhance our model’s performance.
84
 
85
  ## **Usage Example**
86
 
 
144
 
145
  ## **SCB 10X Typhoon Team**
146
 
147
+ - Kunat Pipatanakul, Potsawee Manakul, Sittipong Sripaisarnmongkol, Natapong Nitarach, Pathomporn Chokchainant, Kasima Tharnpipitchai
148
  - If you find Typhoon-1.5X useful for your work, please cite it using:
149
 
150
  ```