Update 'Recommended Inference Parameters' section
Browse files
README.md
CHANGED
@@ -67,10 +67,10 @@ widget:
|
|
67 |
<|im_start|>assistant
|
68 |
inference:
|
69 |
parameters:
|
70 |
-
add_special_tokens: true
|
71 |
max_new_tokens: 250
|
72 |
penalty_alpha: 0.5
|
73 |
-
top_k:
|
|
|
74 |
---
|
75 |
|
76 |
# A Llama Chat Model of 101M Parameters
|
@@ -90,8 +90,6 @@ inference:
|
|
90 |
|
91 |
## Recommended Prompt Format
|
92 |
|
93 |
-
The recommended prompt format is as follows:
|
94 |
-
|
95 |
```
|
96 |
<|im_start|>system
|
97 |
{system_message}<|im_end|>
|
@@ -102,10 +100,8 @@ The recommended prompt format is as follows:
|
|
102 |
|
103 |
## Recommended Inference Parameters
|
104 |
|
105 |
-
To get the best results, add special tokens and use [contrastive search](https://huggingface.co/docs/transformers/main/en/generation_strategies#contrastive-search):
|
106 |
-
|
107 |
```yml
|
108 |
-
add_special_tokens: true
|
109 |
penalty_alpha: 0.5
|
110 |
-
top_k:
|
|
|
111 |
```
|
|
|
67 |
<|im_start|>assistant
|
68 |
inference:
|
69 |
parameters:
|
|
|
70 |
max_new_tokens: 250
|
71 |
penalty_alpha: 0.5
|
72 |
+
top_k: 4
|
73 |
+
repetition_penalty: 1.105
|
74 |
---
|
75 |
|
76 |
# A Llama Chat Model of 101M Parameters
|
|
|
90 |
|
91 |
## Recommended Prompt Format
|
92 |
|
|
|
|
|
93 |
```
|
94 |
<|im_start|>system
|
95 |
{system_message}<|im_end|>
|
|
|
100 |
|
101 |
## Recommended Inference Parameters
|
102 |
|
|
|
|
|
103 |
```yml
|
|
|
104 |
penalty_alpha: 0.5
|
105 |
+
top_k: 4
|
106 |
+
repetition_penalty: 1.105
|
107 |
```
|