Update README.md
Browse files
README.md
CHANGED
|
@@ -28,7 +28,7 @@ Refer to the [original model](https://huggingface.co/databricks/dolly-v2-12b) fo
|
|
| 28 |
|
| 29 |
- total model size is only ~12.5 GB!
|
| 30 |
- this enables low-RAM loading, i.e. Colab :)
|
| 31 |
-
|
| 32 |
## Basic Usage
|
| 33 |
|
| 34 |
|
|
|
|
| 28 |
|
| 29 |
- total model size is only ~12.5 GB!
|
| 30 |
- this enables low-RAM loading, i.e. Colab :)
|
| 31 |
+
- **update**: generation speed can be greatly improved by setting `use_cache=True` and generating via contrastive search. [example notenook here](https://colab.research.google.com/gist/pszemraj/12c832952c88d77f6924c0718a2d257d/dolly-v2-12b-8bit-use_cache-bettertransformer.ipynb)
|
| 32 |
## Basic Usage
|
| 33 |
|
| 34 |
|