Commit
•
b1aba82
1
Parent(s):
2c58152
Update README.md
Browse files
README.md
CHANGED
|
@@ -19,6 +19,18 @@ license: apache-2.0
|
|
| 19 |
|
| 20 |
**This model is converted from [MiniCPM-S-1B-sft](https://huggingface.co/openbmb/MiniCPM-S-1B-sft/) as a LLaMA format to make its usage more convenient.**
|
| 21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
### Introduction
|
| 23 |
|
| 24 |
The utilization of activation sparsity, namely the existence of considerable weakly-contributed elements among activation outputs, is a promising method for inference acceleration of large language models (LLMs) ([Liu et al., 2023](https://proceedings.mlr.press/v202/liu23am/liu23am.pdf); [Song et al., 2023](https://arxiv.org/pdf/2312.12456.pdf)). Concretely, acceleration methods based on activation sparsity usually achieve higher inference speed by making wiser resource allocation and computation policies to avoid resource waste on these weakly-contributed parameters.
|
|
|
|
| 19 |
|
| 20 |
**This model is converted from [MiniCPM-S-1B-sft](https://huggingface.co/openbmb/MiniCPM-S-1B-sft/) as a LLaMA format to make its usage more convenient.**
|
| 21 |
|
| 22 |
+
### Chat Template
|
| 23 |
+
|
| 24 |
+
To make the model sophisticatedly respond to a query, it is recommended to use a standard chat prompt, such as:
|
| 25 |
+
|
| 26 |
+
```
|
| 27 |
+
<用户>{prompt}<AI>
|
| 28 |
+
```
|
| 29 |
+
|
| 30 |
+
where `prompt` is the query text, while `<用户>` and `<AI>` are prompt tokens.
|
| 31 |
+
|
| 32 |
+
Also, make sure that you have **a bos token `<s>` at the beginning of any input**, or the model can sometimes behave improperly.
|
| 33 |
+
|
| 34 |
### Introduction
|
| 35 |
|
| 36 |
The utilization of activation sparsity, namely the existence of considerable weakly-contributed elements among activation outputs, is a promising method for inference acceleration of large language models (LLMs) ([Liu et al., 2023](https://proceedings.mlr.press/v202/liu23am/liu23am.pdf); [Song et al., 2023](https://arxiv.org/pdf/2312.12456.pdf)). Concretely, acceleration methods based on activation sparsity usually achieve higher inference speed by making wiser resource allocation and computation policies to avoid resource waste on these weakly-contributed parameters.
|