---
license: cc-by-sa-4.0
---

This repository contains the most relevant quantizations of Stability AI's
[StableLM-3B-4E1T](https://huggingface.co/stabilityai/stablelm-3b-4e1t) model
in GGUF format - ready to be used with
[llama.cpp](https://github.com/ggerganov/llama.cpp) and similar applications.

For this model, Stability AI claims: "_StableLM-3B-4E1T achieves
state-of-the-art performance (September 2023) at the 3B parameter scale
for open-source models and is competitive with many of the popular
contemporary 7B models, even outperforming our most recent 7B
StableLM-Base-Alpha-v2._"

According to them "_The model is intended to be used as a foundational base 
model for application-specific fine-tuning. Developers must evaluate and 
fine-tune the model for safe performance in downstream applications._"

Right now, the following quantizations are available:

* [stablelm-3b-4e1t-Q4_K_M](https://huggingface.co/rozek/StableLM-3B-4E1T_GGUF/blob/main/stablelm-3b-4e1t-Q4_K_M.bin)
* [stablelm-3b-4e1t-Q5_K_M](https://huggingface.co/rozek/StableLM-3B-4E1T_GGUF/blob/main/stablelm-3b-4e1t-Q5_K_M.bin)
* [stablelm-3b-4e1t-Q6_K](https://huggingface.co/rozek/StableLM-3B-4E1T_GGUF/blob/main/stablelm-3b-4e1t-Q6_K.bin)
* [stablelm-3b-4e1t-Q8_K](https://huggingface.co/rozek/StableLM-3B-4E1T_GGUF/blob/main/stablelm-3b-4e1t-Q8_K.bin)

These files are presented here with the written permission of Stability AI (although
access to the model itself is "gated").

Any technical details can be found on the
[original model card](https://huggingface.co/stabilityai/stablelm-3b-4e1t) and in
a paper on [StableLM-3B-4E1T](https://stability.wandb.io/stability-llm/stable-lm/reports/StableLM-3B-4E1T--VmlldzoyMjU4?accessToken=u3zujipenkx5g7rtcj9qojjgxpconyjktjkli2po09nffrffdhhchq045vp0wyfo).
The most important ones for using this model are

* context length is 4096
* there does not seem to be a specific prompt structure - just provide the text
you want to be completed

**License**: The original "_Model checkpoints are licensed under the Creative Commons license
([CC BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)). Under this 
license, you must give [credit](https://creativecommons.org/licenses/by/4.0/#) 
to Stability AI, provide a link to the license, and 
[indicate if changes were made](https://creativecommons.org/licenses/by/4.0/#). 
You may do so in any reasonable manner, but not in any way that suggests the Stability AI endorses you or your use._"

So, in order to be fair and give credits to whom they belong:

* the original model was created and published by [Stability AI](https://huggingface.co/stabilityai)
* besides quantization, no changes were applied to the model itself