TommyZQ commited on
Commit
f1298d9
1 Parent(s): 7030745

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +117 -22
README.md CHANGED
@@ -1,25 +1,120 @@
1
-
2
  ---
 
 
 
 
 
3
  license: apache-2.0
4
  ---
5
-
6
- ### wukong-chat
7
- csg-wukong 英文对话版
8
-
9
- ### 下载
10
- ```
11
- git lfs install
12
- git lfs clone https://opencsg.com/models/baicai/CSG-Wukong-1B-English-chat.git
13
- ```
14
-
15
- ### 网页推理
16
- 执行以下命令安装依赖包:
17
- ```
18
- pip install -U streamlit transformers peft
19
- ```
20
- 执行以下命令启动网页推理:
21
- ```
22
- streamlit ./CSG-Wukong-1B-English-chat/run web_streamlit_for_wukong.py ./CSG-Wukong-1B-English-chat/ --theme.base="dark"
23
- ```
24
-
25
- ![对话演示](image.png)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - code
7
  license: apache-2.0
8
  ---
9
+
10
+
11
+
12
+ # **csg-wukong-1B-chat-v0.1** [[中文]](#chinese) [[English]](#english)
13
+
14
+ <a id="english"></a>
15
+
16
+ <p align="center">
17
+ <img width="900px" alt="OpenCSG" src="./csg-wukong-logo.jpg">
18
+ </p>
19
+
20
+ <p align="center"><a href="https://portal.opencsg.com/models">[OpenCSG Community]</a> <a href="https://github.com/opencsgs">[github]</a> <a href="https://cdn-uploads.huggingface.co/production/uploads/64c71b27d43e4dee51a8b31a/HU6vz21qKTEmUBCWqCFh9.jpeg">[wechat]</a> <a href="https://twitter.com/OpenCsg">[Twitter]</a> </p>
21
+
22
+
23
+ </div>
24
+ OpenCSG stands for Converged resources, Software refinement, and Generative LM. The 'C' represents Converged resources, indicating the integration and full utilization of hybrid resources. The 'S' stands for Software refinement, signifying software that is refined by large models. The 'G' represents Generative LM, which denotes widespread, inclusive, and democratized generative large models.
25
+
26
+ The vision of OpenCSG is to empower every industry, every company, and every individual to own their models. We adhere to the principles of openness and open source, making the large model software stack of OpenCSG available to the community. We welcome everyone to use, send feedback, and contribute collaboratively.
27
+
28
+
29
+
30
+
31
+ ## Model Description
32
+
33
+
34
+
35
+
36
+ **csg-wukong-1B-chat-v0.1** was finetuned on csg-wukong-1B
37
+ <br>
38
+
39
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/661790397437201d78141856/sZvOqCJY4gOEvVhpmlH_N.png)
40
+
41
+ ## Model Evaluation results
42
+
43
+ We submitted csg-wukong-1B on the [open_llm_leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard), and
44
+ the results show our model ranked the 8th among the ~1.5B pretrained small language models.
45
+
46
+
47
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/661790397437201d78141856/_HRTxL6N0qnNPNt-P8k9k.png)
48
+
49
+
50
+
51
+ # Training
52
+
53
+ ## Hardware
54
+
55
+ - **GPUs:** 6 V100
56
+ - **Training time:** 6 hours
57
+
58
+ ## Software
59
+
60
+ - **Orchestration:** [Deepspeed](https://github.com/OpenCSGs)
61
+ - **Neural networks:** [PyTorch](https://github.com/pytorch/pytorch)
62
+ - **BP16 if applicable:** [apex](https://github.com/NVIDIA/apex)
63
+
64
+
65
+ <a id="chinese"></a>
66
+
67
+ <p>
68
+
69
+ </p>
70
+
71
+ # OpenCSG介绍
72
+
73
+
74
+ <p align="center">
75
+ <img width="300px" alt="OpenCSG" src="https://cdn-uploads.huggingface.co/production/uploads/64c71b27d43e4dee51a8b31a/GwYXPKuEoGCGcMICeW-sb.jpeg">
76
+ </p>
77
+
78
+ <p align="center"><a href="https://opencsg.com/models">[OpenCSG 社区]</a> <a href="https://github.com/opencsgs">[github]</a> <a href="https://cdn-uploads.huggingface.co/production/uploads/64c71b27d43e4dee51a8b31a/HU6vz21qKTEmUBCWqCFh9.jpeg">[微信]</a> <a href="https://twitter.com/OpenCsg">[推特]</a> </p>
79
+
80
+
81
+
82
+ </div>
83
+ OpenCSG中 Open是开源开放;C 代表 Converged resources,整合和充分利用的混合异构资源优势,算力降本增效;S 代表 Software refined,重新定义软件的交付方式,通过大模型驱动软件开发,人力降本增效;G 代表 Generative LM,大众化、普惠化和民主化的可商用的开源生成式大模型。
84
+
85
+ OpenCSG的愿景是让每个行业、每个公司、每个人都拥有自己的模型。 我们坚持开源开放的原则,将OpenCSG的大模型软件栈开源到社区,欢迎使用、反馈和参与共建,欢迎关注。
86
+
87
+
88
+
89
+ ## 模型介绍
90
+
91
+
92
+ **csg-wukong-1B-chat-v0.1** 在csg-wukong-1B模型上微调而成。
93
+ <br>
94
+
95
+
96
+
97
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/661790397437201d78141856/YrpSwEsRGdaQj56__8o0U.png)
98
+
99
+
100
+ ## 模型评测结果
101
+
102
+ 我们把csg-wukong-1B模型提交到[open_llm_leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)榜单上,结果显示我们的模型目前在~1.5B小语言模型中排名第8。
103
+
104
+
105
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/661790397437201d78141856/ZfWZ1Fd7ccKrJVx0okV9z.png)
106
+
107
+
108
+
109
+ # 训练
110
+
111
+ ## 硬件资源
112
+
113
+ - **GPU数量:** 6 V100
114
+ - **训练时间:** 6小时
115
+
116
+ ## 软件使用
117
+
118
+ - **微调训练框架:** [Deepspeed](https://github.com/OpenCSGs)
119
+ - **深度学习框架:** [PyTorch](https://github.com/pytorch/pytorch)
120
+ - **BP16:** [apex](https://github.com/NVIDIA/apex)