Commit
•
890a426
1
Parent(s):
844cdad
Update README.md
Browse files
README.md
CHANGED
@@ -9,21 +9,20 @@
|
|
9 |
🤖 <a href="https://modelscope.cn/organization/codefuse-ai" target="_blank">ModelScope</a>
|
10 |
</p>
|
11 |
|
12 |
-
DevOps-Model
|
13 |
|
14 |
-
|
15 |
|
16 |
<br>
|
17 |
-
|
18 |
-
|
19 |
<br>
|
20 |
<br>
|
21 |
|
22 |
-
#
|
23 |
-
|
24 |
|
25 |
-
|
|
26 |
-
|
27 |
| CMMLU | Computer science | 204 |
|
28 |
| CMMLU | Computer security | 171 |
|
29 |
| CMMLU | Machine learning | 122 |
|
@@ -31,9 +30,10 @@ DevOps-Model-14B-Chat 是我们经过高质量 DevOps 语料训练基于 Qwen-14
|
|
31 |
| CEval | Computer architecture | 21 |
|
32 |
| CEval | Computernetwork | 19 |
|
33 |
|
34 |
-
我们分别测试了 Zero-shot 和 Five-shot 的结果,我们的 DevOps-Model-14B-Chat 模型可以在测试的同规模的开源 Chat 模型中取得最高的成绩,后续我们也会进行更多的测试。
|
35 |
|
36 |
-
|
|
|
|
|
37 |
|--|--|--|--|
|
38 |
|**DevOps-Model-14B-Chat**|**14B**|**74.04**|**75.96**|
|
39 |
|Qwen-14B-Chat|14B|69.16|70.03|
|
@@ -42,59 +42,38 @@ DevOps-Model-14B-Chat 是我们经过高质量 DevOps 语料训练基于 Qwen-14
|
|
42 |
|
43 |
<br>
|
44 |
|
45 |
-
#
|
46 |
-
|
47 |
-
|
48 |
-
## 要求
|
49 |
-
- python 3.8 及以上版本
|
50 |
-
- pytorch 2.0 及以上版本
|
51 |
-
- 建议使用CUDA 11.4及以上
|
52 |
-
|
53 |
|
54 |
-
##
|
55 |
-
下载模型后,直接通过以下命令安装 requirements.txt 中的包就可以
|
56 |
```bash
|
57 |
cd path_to_download_model
|
58 |
pip install -r requirements.txt
|
59 |
```
|
60 |
|
61 |
-
##
|
62 |
|
63 |
```python
|
64 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
65 |
from transformers.generation import GenerationConfig
|
66 |
|
67 |
-
tokenizer = AutoTokenizer.from_pretrained("path_to_DevOps-Model
|
|
|
68 |
|
69 |
-
model =
|
70 |
|
71 |
-
|
72 |
-
model.generation_config = GenerationConfig.from_pretrained("path_to_DevOps-Model-14B-Chat", trust_remote_code=True)
|
73 |
-
|
74 |
-
# 第一轮对话
|
75 |
-
resp, hist = model.chat(query='你是谁', tokenizer=tokenizer, history=None)
|
76 |
-
print(resp)
|
77 |
-
# 我是 DevOps-Model,一个由蚂蚁金服平台技术事业群风险智能团队和北京大学联合研发的人工智能机器人,可以与您进行自然语言交互,帮助您解答各种问题。
|
78 |
-
|
79 |
-
# 第二轮对话
|
80 |
-
resp2, hist2 = model.chat(query='Java 中 HashMap 和 Hashtable 有什么区别', tokenizer=tokenizer, history=hist)
|
81 |
-
print(resp2)
|
82 |
-
# HashMap 和 Hashtable 都是 Java 中常用的哈希表实现,它们的区别在于 Hashtable 是线程安全的,而 HashMap 不是。Hashtable 中的方法都是同步的,因此在多线程环境下使用 Hashtable 是安全的,而 HashMap 中的方法没有同步机制,因此在多线程环境下使用 HashMap 需要手动加锁来保证线程安全。此外,Hashtable 中的键和值都不能为 null,而 HashMap 中的键和值都可以为 null。
|
83 |
-
|
84 |
-
# 第三轮对话
|
85 |
-
resp3, hist3 = model.chat(query='线程安全代表什么', tokenizer=tokenizer, history=hist2)
|
86 |
-
print(resp3)
|
87 |
-
# 线程安全指的是在多线程环境下,程序能够正确地处理并发访问的情况,不会出现数据不一致、死锁等问题。线程安全的程序在多线程环境下可以保证数据的一致性和正确性,避免出现并发问题。在 Java 中,线程安全可以通过同步机制来实现,例如使用 synchronized 关键字或者使用线程安全的类。
|
88 |
```
|
89 |
|
90 |
|
91 |
|
92 |
-
#
|
93 |
-
|
94 |
-
|
|
|
95 |
|
96 |
|
97 |
-
#
|
98 |
-
|
99 |
- [LLaMA-Efficient-Tuning](https://github.com/hiyouga/LLaMA-Efficient-Tuning)
|
100 |
-
- [
|
|
|
9 |
🤖 <a href="https://modelscope.cn/organization/codefuse-ai" target="_blank">ModelScope</a>
|
10 |
</p>
|
11 |
|
12 |
+
DevOps-Model is a Chinese **DevOps large model**, mainly dedicated to exerting practical value in the field of DevOps. Currently, DevOps-Model can help engineers answer questions encountered in the all DevOps life cycle.
|
13 |
|
14 |
+
Based on the Qwen series of models, we output the **Base** model after additional training with high-quality Chinese DevOps corpus, and then output the **Chat** model after alignment with DevOps QA data. Our Base model and Chat model can achieve the best results among models of the same scale based on evaluation data related to the DevOps fields.
|
15 |
|
16 |
<br>
|
17 |
+
At the same time, we are also building an evaluation benchmark [DevOpsEval](https://github.com/codefuse-ai/codefuse-devops-eval) exclusive to the DevOps field to better evaluate the effect of the DevOps field model.
|
|
|
18 |
<br>
|
19 |
<br>
|
20 |
|
21 |
+
# Evaluation
|
22 |
+
We first selected a total of six exams related to DevOps in the two evaluation data sets of CMMLU and CEval. There are a total of 574 multiple-choice questions. The specific information is as follows:
|
23 |
|
24 |
+
| Evaluation dataset | Exam subjects | Number of questions |
|
25 |
+
|:-------:|:-------:|:-------:|
|
26 |
| CMMLU | Computer science | 204 |
|
27 |
| CMMLU | Computer security | 171 |
|
28 |
| CMMLU | Machine learning | 122 |
|
|
|
30 |
| CEval | Computer architecture | 21 |
|
31 |
| CEval | Computernetwork | 19 |
|
32 |
|
|
|
33 |
|
34 |
+
We tested the results of Zero-shot and Five-shot respectively. Our 7B and 14B series models can achieve the best results among the tested models. More tests will be released later.
|
35 |
+
|
36 |
+
|Model|Zero-shot Score|Five-shot Score|
|
37 |
|--|--|--|--|
|
38 |
|**DevOps-Model-14B-Chat**|**14B**|**74.04**|**75.96**|
|
39 |
|Qwen-14B-Chat|14B|69.16|70.03|
|
|
|
42 |
|
43 |
<br>
|
44 |
|
45 |
+
# Quickstart
|
46 |
+
We provide simple examples to illustrate how to quickly use Devops-Model-Chat models with 🤗 Transformers.
|
|
|
|
|
|
|
|
|
|
|
|
|
47 |
|
48 |
+
## Requirement
|
|
|
49 |
```bash
|
50 |
cd path_to_download_model
|
51 |
pip install -r requirements.txt
|
52 |
```
|
53 |
|
54 |
+
## Model Example
|
55 |
|
56 |
```python
|
57 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
58 |
from transformers.generation import GenerationConfig
|
59 |
|
60 |
+
tokenizer = AutoTokenizer.from_pretrained("path_to_DevOps-Model", trust_remote_code=True)
|
61 |
+
model = AutoModelForCausalLM.from_pretrained("path_to_DevOps-Model", device_map="auto", trust_remote_code=True, bf16=True).eval()
|
62 |
|
63 |
+
model.generation_config = GenerationConfig.from_pretrained("path_to_DevOps-Model", trust_remote_code=True)
|
64 |
|
65 |
+
resp, hist = model.chat(query='What is the difference between HashMap and Hashtable in Java', tokenizer=tokenizer, history=None)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
66 |
```
|
67 |
|
68 |
|
69 |
|
70 |
+
# Disclaimer
|
71 |
+
Due to the characteristics of language models, the content generated by the model may contain hallucinations or discriminatory remarks. Please use the content generated by the DevOps-Model family of models with caution.
|
72 |
+
If you want to use this model service publicly or commercially, please note that the service provider needs to bear the responsibility for the adverse effects or harmful remarks caused by it. The developer of this project does not assume any responsibility for any consequences caused by the use of this project (including but not limited to data, models, codes, etc.) ) resulting in harm or loss.
|
73 |
+
|
74 |
|
75 |
|
76 |
+
# Acknowledgments
|
77 |
+
This project refers to the following open source projects, and I would like to express my gratitude to the relevant projects and research and development personnel.
|
78 |
- [LLaMA-Efficient-Tuning](https://github.com/hiyouga/LLaMA-Efficient-Tuning)
|
79 |
+
- [QwenLM](https://github.com/QwenLM)
|