summit4you commited on
Commit
5ee4f00
1 Parent(s): 52dadfa

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +66 -1
README.md CHANGED
@@ -14,4 +14,69 @@ tags:
14
  - medical
15
  - code
16
  - biology
17
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  - medical
15
  - code
16
  - biology
17
+ ---
18
+
19
+ # Model Summary
20
+
21
+ Llama3-8B-COIG-CQIA is an instruction-tuned language model for Chinese & English users with various abilities such as roleplaying & tool-using built upon the Meta-Llama-3-8B-Instruct model.
22
+
23
+ Developed by: [Wenfeng Qiu](https://github.com/summit4you)
24
+
25
+ - License: [Llama-3 License](https://llama.meta.com/llama3/license/)
26
+ - Base Model: Meta-Llama-3-8B-Instruct
27
+ - Model Size: 8.03B
28
+ - Context length: 8K
29
+
30
+ # 1. Introduction
31
+
32
+ Training framework: [unsloth](https://github.com/unslothai/unsloth).
33
+
34
+ Training details:
35
+ - epochs: 1
36
+ - learning rate: 2e-4
37
+ - learning rate scheduler type: linear
38
+ - warmup steps: 5
39
+ - cutoff len (i.e. context length): 2048
40
+ - global batch size: 2
41
+ - fine-tuning type: full parameters
42
+ - optimizer: adamw_8bit
43
+
44
+ # 2. Usage
45
+
46
+ Inference, use to `llama.cpp` or a UI based system like `GPT4All`. You can install GPT4All by going [here](https://gpt4all.io/index.html).
47
+
48
+ Here is the example in `llama.cpp`.
49
+
50
+ ```python
51
+ from llama_cpp import Llama
52
+
53
+ model = Llama(
54
+ "/Your/Path/To/Llama3-8B-COIG-CQIA.Q8_0.gguf",
55
+ verbose=False,
56
+ n_gpu_layers=-1,
57
+ )
58
+
59
+ system_prompt = "You are a helpful assistant."
60
+
61
+ def generate_reponse(_model, _messages, _max_tokens=8192):
62
+ _output = _model.create_chat_completion(
63
+ _messages,
64
+ stop=["<|eot_id|>", "<|end_of_text|>"],
65
+ max_tokens=_max_tokens,
66
+ )["choices"][0]["message"]["content"]
67
+ return _output
68
+
69
+ # The following are some examples
70
+
71
+ messages = [
72
+ {
73
+ "role": "system",
74
+ "content": system_prompt,
75
+ },
76
+ {"role": "user", "content": "你是谁?"},
77
+ ]
78
+
79
+
80
+ print(generate_reponse(_model=model, _messages=messages), end="\n\n\n")
81
+
82
+ ```