Change name
Browse files
README.md
CHANGED
@@ -10,27 +10,27 @@ language:
|
|
10 |
|
11 |
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/62cd3a3691d27e60db0698b0/2peGbPRq4jE-OoS9ndkOx.jpeg)
|
12 |
|
13 |
-
#
|
14 |
|
15 |
-
|
16 |
|
17 |
## Performance
|
18 |
|
19 |
-
|
20 |
|
21 |
### Fact-based Evaluation (Open LLM Leaderboard)
|
22 |
|
23 |
| **Metric** | **MMLU** | GSM8K | **HellaSwag** | **TruthfulQA** | **Arc** | **Winogrande** |
|
24 |
| -------------- | --------- | --------- | ------------- | -------------- | ----------- | -------------- |
|
25 |
| **Yi-9B-200K** | 65.73 | 50.49 | 56.72 | 33.80 | 69.25 | 71.67 |
|
26 |
-
| **
|
27 |
|
28 |
### Long-context Modeling (LongBench)
|
29 |
|
30 |
| **Name** | **Average_zh** | **Average_en** | **Code Completion** |
|
31 |
|----------------|----------------|----------------|---------------------|
|
32 |
| **Yi-9B-200K** | 30.288 | 36.7071 | 72.2 |
|
33 |
-
| **
|
34 |
|
35 |
<details>
|
36 |
<summary>Score breakdown</summary>
|
@@ -38,7 +38,7 @@ Fi-9B enhances its ability compared to Yi-9B-200K in most dimensions, especially
|
|
38 |
| **Name** | **Few-shot Learning_en** | **Synthetic Tasks_en** | **Single-Doc QA_en** | **Multi-Doc QA_en** | **Summarization_en** | **Few-shot Learning_zh** | **Synthetic Tasks_zh** | **Single-Doc QA_zh** | **Multi-Doc QA_zh** | **Summarization_zh** |
|
39 |
|----------------|--------------------------|------------------------|----------------------|---------------------|----------------------|--------------------------|------------------------|----------------------|---------------------|----------------------|
|
40 |
| **Yi-9B-200K** | 60.6 | 22.8 | 30.9 | 38.9 | 25.8 | 46.5 | 28.0 | 49.6 | 17.7 | 9.7 |
|
41 |
-
| **
|
42 |
|
43 |
</details>
|
44 |
|
@@ -49,10 +49,4 @@ Fi-9B enhances its ability compared to Yi-9B-200K in most dimensions, especially
|
|
49 |
| **Name** | MMLU | **CMMLU** |
|
50 |
| -------------- | --------- | --------- |
|
51 |
| **Yi-9B-200K** | 65.73 | 71.97 |
|
52 |
-
| **
|
53 |
-
|
54 |
-
|
55 |
-
## Current Limitations
|
56 |
-
|
57 |
-
- This version of Fi-9B may not be able to stop generation in some scenarios. I will fix that soon.
|
58 |
-
- Compared to the original Yi-9B-200K, Fi-9B has degraded ability for code completion. This may be due to the lack of raw code data during instruction tuning.
|
|
|
10 |
|
11 |
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/62cd3a3691d27e60db0698b0/2peGbPRq4jE-OoS9ndkOx.jpeg)
|
12 |
|
13 |
+
# Faro-Yi-9B
|
14 |
|
15 |
+
Faro-Yi-9B is an improved [Yi-9B-200K](https://huggingface.co/01-ai/Yi-9B-200K) with extensive instruction tuning on [Fusang-V1](https://huggingface.co/datasets/wenbopan/Fusang-v1). Compared to Yi-9B-200K, Faro-Yi-9B has gained greater capability in various downstream tasks and long-context modeling thanks to the large-scale synthetic data in Fusang-V1.
|
16 |
|
17 |
## Performance
|
18 |
|
19 |
+
Faro-Yi-9B enhances its ability compared to Yi-9B-200K in most dimensions, especially in long-range modeling and bilingual (English, Chinese) understanding. Fi is competitive among all open-sourced models at around 9B parameters. Fi-9B is good at both factual tasks and preferred by LLM-judges.
|
20 |
|
21 |
### Fact-based Evaluation (Open LLM Leaderboard)
|
22 |
|
23 |
| **Metric** | **MMLU** | GSM8K | **HellaSwag** | **TruthfulQA** | **Arc** | **Winogrande** |
|
24 |
| -------------- | --------- | --------- | ------------- | -------------- | ----------- | -------------- |
|
25 |
| **Yi-9B-200K** | 65.73 | 50.49 | 56.72 | 33.80 | 69.25 | 71.67 |
|
26 |
+
| **Faro-Yi-9B** | **68.80** | **63.08** | **57.28** | **40.86** | **72.58** | 71.11 |
|
27 |
|
28 |
### Long-context Modeling (LongBench)
|
29 |
|
30 |
| **Name** | **Average_zh** | **Average_en** | **Code Completion** |
|
31 |
|----------------|----------------|----------------|---------------------|
|
32 |
| **Yi-9B-200K** | 30.288 | 36.7071 | 72.2 |
|
33 |
+
| **Faro-Yi-9B** | **41.092** | **40.9536** | 46.0 |
|
34 |
|
35 |
<details>
|
36 |
<summary>Score breakdown</summary>
|
|
|
38 |
| **Name** | **Few-shot Learning_en** | **Synthetic Tasks_en** | **Single-Doc QA_en** | **Multi-Doc QA_en** | **Summarization_en** | **Few-shot Learning_zh** | **Synthetic Tasks_zh** | **Single-Doc QA_zh** | **Multi-Doc QA_zh** | **Summarization_zh** |
|
39 |
|----------------|--------------------------|------------------------|----------------------|---------------------|----------------------|--------------------------|------------------------|----------------------|---------------------|----------------------|
|
40 |
| **Yi-9B-200K** | 60.6 | 22.8 | 30.9 | 38.9 | 25.8 | 46.5 | 28.0 | 49.6 | 17.7 | 9.7 |
|
41 |
+
| **Faro-Yi-9B** | **63.8** | **40.2** | **36.2** | 38.0 | **26.3** | 30.0 | **75.1** | **55.6** | **30.7** | **14.1** |
|
42 |
|
43 |
</details>
|
44 |
|
|
|
49 |
| **Name** | MMLU | **CMMLU** |
|
50 |
| -------------- | --------- | --------- |
|
51 |
| **Yi-9B-200K** | 65.73 | 71.97 |
|
52 |
+
| **Faro-Yi-9B** | **68.80** | **73.28** |
|
|
|
|
|
|
|
|
|
|
|
|