Update README.md
Browse files
README.md
CHANGED
@@ -1,5 +1,10 @@
|
|
1 |
---
|
2 |
license: llama2
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
4 |
# Bailong-orpo 7B
|
5 |
<div align="center">
|
@@ -18,7 +23,16 @@ This model card contains the information and the results of our released Bailong
|
|
18 |
- **Bailong-bench:** Most existing language models claiming to support Traditional Chinese are adapted from continuously pre-trained open-source models, primarily trained on English data. In certain cases, models fine-tuned with instructions using this approach may respond to Traditional Chinese instructions in English and vice versa. This could pose a significant problem when deploying the model for real-world applications. Consequently, it is essential to have a benchmark dataset specifically designed to assess a model's proficiency in following both English and Traditional Chinese instructions. To address this issue, we propose Bailong-bench, a benchmark dataset crafted not only to evaluate the model's performance in various real-world application scenarios but also to assess its ability to maintain language consistency.
|
19 |
- **Technical report:** In our [technical report](https://arxiv.org/abs/2404.00862), we document the model training process and the details regarding the sources of training data.
|
20 |
- **Bailong-orpo 7B:** Leveraging monolithic odds ratio preference optimization
|
21 |
-
algorithm, [ORPO](https://arxiv.org/abs/2403.07691), we further fine-tune Bailong-instruct 7B with 180k preference
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
22 |
|
23 |
## Applications (Bailong-orpo 7B)
|
24 |
The following tables present, but are not limited to, several possible scenarios for the applications of Bailong-orpo 7B. All the following model outputs are generated under the same generation configuration (temperature=0.6, top-p=0.9, top-k=40, repetition_penalty=1.1)
|
|
|
1 |
---
|
2 |
license: llama2
|
3 |
+
library_name: transformers
|
4 |
+
base_model: INX-TEXT/Bailong-instruct-7B
|
5 |
+
pipeline_tag: text-generation
|
6 |
+
tags:
|
7 |
+
- orpo
|
8 |
---
|
9 |
# Bailong-orpo 7B
|
10 |
<div align="center">
|
|
|
23 |
- **Bailong-bench:** Most existing language models claiming to support Traditional Chinese are adapted from continuously pre-trained open-source models, primarily trained on English data. In certain cases, models fine-tuned with instructions using this approach may respond to Traditional Chinese instructions in English and vice versa. This could pose a significant problem when deploying the model for real-world applications. Consequently, it is essential to have a benchmark dataset specifically designed to assess a model's proficiency in following both English and Traditional Chinese instructions. To address this issue, we propose Bailong-bench, a benchmark dataset crafted not only to evaluate the model's performance in various real-world application scenarios but also to assess its ability to maintain language consistency.
|
24 |
- **Technical report:** In our [technical report](https://arxiv.org/abs/2404.00862), we document the model training process and the details regarding the sources of training data.
|
25 |
- **Bailong-orpo 7B:** Leveraging monolithic odds ratio preference optimization
|
26 |
+
algorithm, [ORPO](https://arxiv.org/abs/2403.07691), we further fine-tune Bailong-instruct 7B with 180k preference pair data to derive Bailong-orpo 7B. We also provide f16 GGUF version of Bailong-orpo 7B for efficient inference and storage purposes.
|
27 |
+
|
28 |
+
## Model information
|
29 |
+
- License: [Llama-2 License](https://ai.meta.com/llama/license/)
|
30 |
+
- Base Model: [INX-TEXT/Bailong-instruct-7B](https://huggingface.co/INX-TEXT/Bailong-instruct-7B)
|
31 |
+
- Type: decoder-only transformer architecture
|
32 |
+
- Model Size: 6.96B
|
33 |
+
- Context length: 2048
|
34 |
+
- Vocabulary size: 59241
|
35 |
+
- Language: English and Traditional Chinese
|
36 |
|
37 |
## Applications (Bailong-orpo 7B)
|
38 |
The following tables present, but are not limited to, several possible scenarios for the applications of Bailong-orpo 7B. All the following model outputs are generated under the same generation configuration (temperature=0.6, top-p=0.9, top-k=40, repetition_penalty=1.1)
|