Update README.md
Browse files
README.md
CHANGED
|
@@ -145,7 +145,8 @@ print(processor.decode(out[0], skip_special_tokens=True))
|
|
| 145 |
</details>
|
| 146 |
|
| 147 |
##### In 8-bit precision (`int8`)
|
| 148 |
-
|
|
|
|
| 149 |
<details>
|
| 150 |
<summary> Click to expand </summary>
|
| 151 |
|
|
|
|
| 145 |
</details>
|
| 146 |
|
| 147 |
##### In 8-bit precision (`int8`)
|
| 148 |
+
>**Important:** Paper results only use int8 for the LLM weights while this loads all weights in int8.
|
| 149 |
+
> We see that this gives slightly worse results but currently, int8 for some model parts is not supported by HuggingFace.
|
| 150 |
<details>
|
| 151 |
<summary> Click to expand </summary>
|
| 152 |
|