Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+TestMixtral-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+TestMixtral-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+TestMixtral-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+TestMixtral-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+TestMixtral-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+TestMixtral-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+TestMixtral-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+TestMixtral-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+TestMixtral-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+TestMixtral-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+TestMixtral-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+TestMixtral-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,70 @@
+---
+library_name: transformers
+tags:
+- TensorBlock
+- GGUF
+base_model: artek0chumak/TestMixtral
+---
+
+<div style="width: auto; margin-left: auto; margin-right: auto">
+<img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;">
+</div>
+<div style="display: flex; justify-content: space-between; width: 100%;">
+    <div style="display: flex; flex-direction: column; align-items: flex-start;">
+        <p style="margin-top: 0.5em; margin-bottom: 0em;">
+            Feedback and support: TensorBlock's  <a href="https://x.com/tensorblock_aoi">Twitter/X</a>, <a href="https://t.me/TensorBlock">Telegram Group</a> and <a href="https://x.com/tensorblock_aoi">Discord server</a>
+        </p>
+    </div>
+</div>
+
+## artek0chumak/TestMixtral - GGUF
+
+This repo contains GGUF format model files for [artek0chumak/TestMixtral](https://huggingface.co/artek0chumak/TestMixtral).
+
+The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
+
+## Prompt template
+
+```
+
+```
+
+## Model file specification
+
+| Filename | Quant type | File Size | Description |
+| -------- | ---------- | --------- | ----------- |
+| [TestMixtral-Q2_K.gguf](https://huggingface.co/tensorblock/TestMixtral-GGUF/tree/main/TestMixtral-Q2_K.gguf) | Q2_K | 0.005 GB | smallest, significant quality loss - not recommended for most purposes |
+| [TestMixtral-Q3_K_S.gguf](https://huggingface.co/tensorblock/TestMixtral-GGUF/tree/main/TestMixtral-Q3_K_S.gguf) | Q3_K_S | 0.005 GB | very small, high quality loss |
+| [TestMixtral-Q3_K_M.gguf](https://huggingface.co/tensorblock/TestMixtral-GGUF/tree/main/TestMixtral-Q3_K_M.gguf) | Q3_K_M | 0.005 GB | very small, high quality loss |
+| [TestMixtral-Q3_K_L.gguf](https://huggingface.co/tensorblock/TestMixtral-GGUF/tree/main/TestMixtral-Q3_K_L.gguf) | Q3_K_L | 0.005 GB | small, substantial quality loss |
+| [TestMixtral-Q4_0.gguf](https://huggingface.co/tensorblock/TestMixtral-GGUF/tree/main/TestMixtral-Q4_0.gguf) | Q4_0 | 0.005 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
+| [TestMixtral-Q4_K_S.gguf](https://huggingface.co/tensorblock/TestMixtral-GGUF/tree/main/TestMixtral-Q4_K_S.gguf) | Q4_K_S | 0.006 GB | small, greater quality loss |
+| [TestMixtral-Q4_K_M.gguf](https://huggingface.co/tensorblock/TestMixtral-GGUF/tree/main/TestMixtral-Q4_K_M.gguf) | Q4_K_M | 0.006 GB | medium, balanced quality - recommended |
+| [TestMixtral-Q5_0.gguf](https://huggingface.co/tensorblock/TestMixtral-GGUF/tree/main/TestMixtral-Q5_0.gguf) | Q5_0 | 0.006 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
+| [TestMixtral-Q5_K_S.gguf](https://huggingface.co/tensorblock/TestMixtral-GGUF/tree/main/TestMixtral-Q5_K_S.gguf) | Q5_K_S | 0.006 GB | large, low quality loss - recommended |
+| [TestMixtral-Q5_K_M.gguf](https://huggingface.co/tensorblock/TestMixtral-GGUF/tree/main/TestMixtral-Q5_K_M.gguf) | Q5_K_M | 0.006 GB | large, very low quality loss - recommended |
+| [TestMixtral-Q6_K.gguf](https://huggingface.co/tensorblock/TestMixtral-GGUF/tree/main/TestMixtral-Q6_K.gguf) | Q6_K | 0.007 GB | very large, extremely low quality loss |
+| [TestMixtral-Q8_0.gguf](https://huggingface.co/tensorblock/TestMixtral-GGUF/tree/main/TestMixtral-Q8_0.gguf) | Q8_0 | 0.007 GB | very large, extremely low quality loss - not recommended |
+
+
+## Downloading instruction
+
+### Command line
+
+Firstly, install Huggingface Client
+
+```shell
+pip install -U "huggingface_hub[cli]"
+```
+
+Then, downoad the individual model file the a local directory
+
+```shell
+huggingface-cli download tensorblock/TestMixtral-GGUF --include "TestMixtral-Q2_K.gguf" --local-dir MY_LOCAL_DIR
+```
+
+If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try:
+
+```shell
+huggingface-cli download tensorblock/TestMixtral-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'
+```

TestMixtral-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd1d5aad033309a718786e981d485e84d8983d2dbed5d1ca79e9b4a21658d5fa
+size 5378720

TestMixtral-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c33e24ffab79c8c867b29bbdd8d64d8096af866544de3a9c738738ca2630584
+size 5595296

TestMixtral-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57e4e143793d751ad8d844f211efcbd1250311f788d0e8ddc6a6284b7977256e
+size 5492640

TestMixtral-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:673c4baf4ad038f8070938350ed007d2bf256d3e7d866b819a13c3b058cb79bf
+size 5378720

TestMixtral-Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4052c1a848fc29a9e7fb5be46331ff07ababd8c8a42180a3360f906b39a12271
+size 5483168

TestMixtral-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da62ef1a666f59ab5d0abc0fdbd2297332a2977fdf2edc53bcf17fdb035faddd
+size 6060224

TestMixtral-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:377ce15bb87cc41cec81b1486aa1bb75bd4b8fe09105d4ab96c53c92b2f9811b
+size 5962944

TestMixtral-Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7e625fb0041628336553f982f8e65faebda0e75458580a959cf1b9bbc74561c
+size 6052544

TestMixtral-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b3306fe07fa671dce2c0cc4a10a227b433f0847b6aa54647e7a1d75f4e8dc2a
+size 6344160

TestMixtral-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cbd7585d272e91a44592e65b4d43e0d3f6650d76d9d37c37f11e0342e9808a70
+size 6288096

TestMixtral-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0529a3a2cfa29d66440e3700d602c05936eb43da65e83dd8ca38d8a1839dabfd
+size 7570240

TestMixtral-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:813b8839789fa1057d47a3ca8c578f4767260c135a8d59b82b08cde751100bad
+size 7760704