Quantization made by Richard Erkhov. [Github](https://github.com/RichardErkhov) [Discord](https://discord.gg/pvy7H8DZMG) [Request more models](https://github.com/RichardErkhov/quant_request) TinyLlama-1.1B-1T-OpenOrca - GGUF - Model creator: https://huggingface.co/jeff31415/ - Original model: https://huggingface.co/jeff31415/TinyLlama-1.1B-1T-OpenOrca/ | Name | Quant method | Size | | ---- | ---- | ---- | | [TinyLlama-1.1B-1T-OpenOrca.Q2_K.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q2_K.gguf) | Q2_K | 0.4GB | | [TinyLlama-1.1B-1T-OpenOrca.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.IQ3_XS.gguf) | IQ3_XS | 0.44GB | | [TinyLlama-1.1B-1T-OpenOrca.IQ3_S.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.IQ3_S.gguf) | IQ3_S | 0.47GB | | [TinyLlama-1.1B-1T-OpenOrca.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q3_K_S.gguf) | Q3_K_S | 0.47GB | | [TinyLlama-1.1B-1T-OpenOrca.IQ3_M.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.IQ3_M.gguf) | IQ3_M | 0.48GB | | [TinyLlama-1.1B-1T-OpenOrca.Q3_K.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q3_K.gguf) | Q3_K | 0.51GB | | [TinyLlama-1.1B-1T-OpenOrca.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q3_K_M.gguf) | Q3_K_M | 0.51GB | | [TinyLlama-1.1B-1T-OpenOrca.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q3_K_L.gguf) | Q3_K_L | 0.55GB | | [TinyLlama-1.1B-1T-OpenOrca.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.IQ4_XS.gguf) | IQ4_XS | 0.57GB | | [TinyLlama-1.1B-1T-OpenOrca.Q4_0.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q4_0.gguf) | Q4_0 | 0.59GB | | [TinyLlama-1.1B-1T-OpenOrca.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.IQ4_NL.gguf) | IQ4_NL | 0.6GB | | [TinyLlama-1.1B-1T-OpenOrca.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q4_K_S.gguf) | Q4_K_S | 0.6GB | | [TinyLlama-1.1B-1T-OpenOrca.Q4_K.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q4_K.gguf) | Q4_K | 0.62GB | | [TinyLlama-1.1B-1T-OpenOrca.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q4_K_M.gguf) | Q4_K_M | 0.62GB | | [TinyLlama-1.1B-1T-OpenOrca.Q4_1.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q4_1.gguf) | Q4_1 | 0.65GB | | [TinyLlama-1.1B-1T-OpenOrca.Q5_0.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q5_0.gguf) | Q5_0 | 0.71GB | | [TinyLlama-1.1B-1T-OpenOrca.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q5_K_S.gguf) | Q5_K_S | 0.71GB | | [TinyLlama-1.1B-1T-OpenOrca.Q5_K.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q5_K.gguf) | Q5_K | 0.73GB | | [TinyLlama-1.1B-1T-OpenOrca.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q5_K_M.gguf) | Q5_K_M | 0.73GB | | [TinyLlama-1.1B-1T-OpenOrca.Q5_1.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q5_1.gguf) | Q5_1 | 0.77GB | | [TinyLlama-1.1B-1T-OpenOrca.Q6_K.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q6_K.gguf) | Q6_K | 0.84GB | | [TinyLlama-1.1B-1T-OpenOrca.Q8_0.gguf](https://huggingface.co/RichardErkhov/jeff31415_-_TinyLlama-1.1B-1T-OpenOrca-gguf/blob/main/TinyLlama-1.1B-1T-OpenOrca.Q8_0.gguf) | Q8_0 | 1.09GB | Original model description: --- license: apache-2.0 datasets: - Open-Orca/OpenOrca - bigcode/starcoderdata - cerebras/SlimPajama-627B language: - en --- [Built with Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) #### Base model: PY007/TinyLlama-1.1B-intermediate-step-480k-1T #### Dataset: Fine tuned on OpenOrca GPT4 subset for 1 epoch,Using CHATML format #### Model License: Apache 2.0, following the TinyLlama base model. #### Quantisation: - GPTQ:https://huggingface.co/TheBloke/TinyLlama-1.1B-1T-OpenOrca-GPTQ - AWQ:https://huggingface.co/TheBloke/TinyLlama-1.1B-1T-OpenOrca-AWQ - GGUF:https://huggingface.co/TheBloke/TinyLlama-1.1B-1T-OpenOrca-GGUF #### Hardware and training details: Hardware: 1*RTX A5000, ~16 hours to complete 1 epoch. GPU from autodl.com, cost around $3 for this finetuning. https://wandb.ai/jeff200402/TinyLlama-Orca?workspace= for more details.