Upload README.md
Browse files
README.md
CHANGED
|
@@ -54,6 +54,7 @@ It is also now supported by continuous batching server [vLLM](https://github.com
|
|
| 54 |
|
| 55 |
* [AWQ model(s) for GPU inference.](https://huggingface.co/TheBloke/airoboros-l2-7B-gpt4-m2.0-AWQ)
|
| 56 |
* [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/airoboros-l2-7B-gpt4-m2.0-GPTQ)
|
|
|
|
| 57 |
* [Jon Durbin's original unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/jondurbin/airoboros-l2-7b-gpt4-m2.0)
|
| 58 |
<!-- repositories-available end -->
|
| 59 |
|
|
|
|
| 54 |
|
| 55 |
* [AWQ model(s) for GPU inference.](https://huggingface.co/TheBloke/airoboros-l2-7B-gpt4-m2.0-AWQ)
|
| 56 |
* [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/airoboros-l2-7B-gpt4-m2.0-GPTQ)
|
| 57 |
+
* [2, 3, 4, 5, 6 and 8-bit GGUF models for CPU+GPU inference](https://huggingface.co/TheBloke/airoboros-l2-7B-gpt4-m2.0-GGUF)
|
| 58 |
* [Jon Durbin's original unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/jondurbin/airoboros-l2-7b-gpt4-m2.0)
|
| 59 |
<!-- repositories-available end -->
|
| 60 |
|