Update README.md
Browse files
README.md
CHANGED
|
@@ -39,6 +39,12 @@ Please note that these GGMLs are **not compatible with llama.cpp, or currently w
|
|
| 39 |
|
| 40 |
[KoboldCpp](https://github.com/LostRuins/koboldcpp) just added GPU accelerated (OpenCL) support for MPT models, so that is the client I recommend using for these models.
|
| 41 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
## Repositories available
|
| 43 |
|
| 44 |
* [2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/mpt-30B-instruct-GGML)
|
|
|
|
| 39 |
|
| 40 |
[KoboldCpp](https://github.com/LostRuins/koboldcpp) just added GPU accelerated (OpenCL) support for MPT models, so that is the client I recommend using for these models.
|
| 41 |
|
| 42 |
+
**Note**: There is currently a bug with loading this model in KoboldCpp Release 1.32: it will wrongly detect it as a GPT-NeoX model.
|
| 43 |
+
|
| 44 |
+
To resolve this, add argument `--forceversion 500`
|
| 45 |
+
|
| 46 |
+
This should be fixed in the next release of KoboldCpp, so if you are running a version later than 1.32 it should not be necessary.
|
| 47 |
+
|
| 48 |
## Repositories available
|
| 49 |
|
| 50 |
* [2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/mpt-30B-instruct-GGML)
|