wenbopan commited on
Commit
0038b20
1 Parent(s): 42fa051

Change name

Browse files
Files changed (1) hide show
  1. README.md +7 -13
README.md CHANGED
@@ -10,27 +10,27 @@ language:
10
 
11
  ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/62cd3a3691d27e60db0698b0/2peGbPRq4jE-OoS9ndkOx.jpeg)
12
 
13
- # Fi-9B
14
 
15
- Fi-9B is an improved [Yi-9B-200K](https://huggingface.co/01-ai/Yi-9B-200K) with extensive instruction tuning on [Fusang-V1](https://huggingface.co/datasets/wenbopan/Fusang-v1). Compared to Yi-9B-200K, Fi-9B has gained greater capability in various downstream tasks and long-context modeling thanks to the large-scale synthetic data in Fusang-V1.
16
 
17
  ## Performance
18
 
19
- Fi-9B enhances its ability compared to Yi-9B-200K in most dimensions, especially in long-range modeling and bilingual (English, Chinese) understanding. Fi is competitive among all open-sourced models at around 9B parameters. Fi-9B is good at both factual tasks and preferred by LLM-judges.
20
 
21
  ### Fact-based Evaluation (Open LLM Leaderboard)
22
 
23
  | **Metric** | **MMLU** | GSM8K | **HellaSwag** | **TruthfulQA** | **Arc** | **Winogrande** |
24
  | -------------- | --------- | --------- | ------------- | -------------- | ----------- | -------------- |
25
  | **Yi-9B-200K** | 65.73 | 50.49 | 56.72 | 33.80 | 69.25 | 71.67 |
26
- | **Fi-9B-200K** | **68.80** | **63.08** | **57.28** | **40.86** | **72.58** | 71.11 |
27
 
28
  ### Long-context Modeling (LongBench)
29
 
30
  | **Name** | **Average_zh** | **Average_en** | **Code Completion** |
31
  |----------------|----------------|----------------|---------------------|
32
  | **Yi-9B-200K** | 30.288 | 36.7071 | 72.2 |
33
- | **Fi-9B-200K** | **41.092** | **40.9536** | 46.0 |
34
 
35
  <details>
36
  <summary>Score breakdown</summary>
@@ -38,7 +38,7 @@ Fi-9B enhances its ability compared to Yi-9B-200K in most dimensions, especially
38
  | **Name** | **Few-shot Learning_en** | **Synthetic Tasks_en** | **Single-Doc QA_en** | **Multi-Doc QA_en** | **Summarization_en** | **Few-shot Learning_zh** | **Synthetic Tasks_zh** | **Single-Doc QA_zh** | **Multi-Doc QA_zh** | **Summarization_zh** |
39
  |----------------|--------------------------|------------------------|----------------------|---------------------|----------------------|--------------------------|------------------------|----------------------|---------------------|----------------------|
40
  | **Yi-9B-200K** | 60.6 | 22.8 | 30.9 | 38.9 | 25.8 | 46.5 | 28.0 | 49.6 | 17.7 | 9.7 |
41
- | **Fi-9B-200K** | **63.8** | **40.2** | **36.2** | 38.0 | **26.3** | 30.0 | **75.1** | **55.6** | **30.7** | **14.1** |
42
 
43
  </details>
44
 
@@ -49,10 +49,4 @@ Fi-9B enhances its ability compared to Yi-9B-200K in most dimensions, especially
49
  | **Name** | MMLU | **CMMLU** |
50
  | -------------- | --------- | --------- |
51
  | **Yi-9B-200K** | 65.73 | 71.97 |
52
- | **Fi-9B-200K** | **68.80** | **73.28** |
53
-
54
-
55
- ## Current Limitations
56
-
57
- - This version of Fi-9B may not be able to stop generation in some scenarios. I will fix that soon.
58
- - Compared to the original Yi-9B-200K, Fi-9B has degraded ability for code completion. This may be due to the lack of raw code data during instruction tuning.
 
10
 
11
  ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/62cd3a3691d27e60db0698b0/2peGbPRq4jE-OoS9ndkOx.jpeg)
12
 
13
+ # Faro-Yi-9B
14
 
15
+ Faro-Yi-9B is an improved [Yi-9B-200K](https://huggingface.co/01-ai/Yi-9B-200K) with extensive instruction tuning on [Fusang-V1](https://huggingface.co/datasets/wenbopan/Fusang-v1). Compared to Yi-9B-200K, Faro-Yi-9B has gained greater capability in various downstream tasks and long-context modeling thanks to the large-scale synthetic data in Fusang-V1.
16
 
17
  ## Performance
18
 
19
+ Faro-Yi-9B enhances its ability compared to Yi-9B-200K in most dimensions, especially in long-range modeling and bilingual (English, Chinese) understanding. Fi is competitive among all open-sourced models at around 9B parameters. Fi-9B is good at both factual tasks and preferred by LLM-judges.
20
 
21
  ### Fact-based Evaluation (Open LLM Leaderboard)
22
 
23
  | **Metric** | **MMLU** | GSM8K | **HellaSwag** | **TruthfulQA** | **Arc** | **Winogrande** |
24
  | -------------- | --------- | --------- | ------------- | -------------- | ----------- | -------------- |
25
  | **Yi-9B-200K** | 65.73 | 50.49 | 56.72 | 33.80 | 69.25 | 71.67 |
26
+ | **Faro-Yi-9B** | **68.80** | **63.08** | **57.28** | **40.86** | **72.58** | 71.11 |
27
 
28
  ### Long-context Modeling (LongBench)
29
 
30
  | **Name** | **Average_zh** | **Average_en** | **Code Completion** |
31
  |----------------|----------------|----------------|---------------------|
32
  | **Yi-9B-200K** | 30.288 | 36.7071 | 72.2 |
33
+ | **Faro-Yi-9B** | **41.092** | **40.9536** | 46.0 |
34
 
35
  <details>
36
  <summary>Score breakdown</summary>
 
38
  | **Name** | **Few-shot Learning_en** | **Synthetic Tasks_en** | **Single-Doc QA_en** | **Multi-Doc QA_en** | **Summarization_en** | **Few-shot Learning_zh** | **Synthetic Tasks_zh** | **Single-Doc QA_zh** | **Multi-Doc QA_zh** | **Summarization_zh** |
39
  |----------------|--------------------------|------------------------|----------------------|---------------------|----------------------|--------------------------|------------------------|----------------------|---------------------|----------------------|
40
  | **Yi-9B-200K** | 60.6 | 22.8 | 30.9 | 38.9 | 25.8 | 46.5 | 28.0 | 49.6 | 17.7 | 9.7 |
41
+ | **Faro-Yi-9B** | **63.8** | **40.2** | **36.2** | 38.0 | **26.3** | 30.0 | **75.1** | **55.6** | **30.7** | **14.1** |
42
 
43
  </details>
44
 
 
49
  | **Name** | MMLU | **CMMLU** |
50
  | -------------- | --------- | --------- |
51
  | **Yi-9B-200K** | 65.73 | 71.97 |
52
+ | **Faro-Yi-9B** | **68.80** | **73.28** |