Duplicate from localmodels/LLM

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,79 @@
+---
+duplicated_from: localmodels/LLM
+---
+# Vicuna 30B GGML
+
+From LMSYS: https://huggingface.co/lmsys/vicuna-33b-v1.3
+
+---
+
+### Original llama.cpp quant methods: `q4_0, q4_1, q5_0, q5_1, q8_0`
+
+Quantized using an older version of llama.cpp and compatible with llama.cpp from May 19, commit 2d5db48.
+
+### k-quant methods: `q2_K, q3_K_S, q3_K_M, q3_K_L, q4_K_S, q4_K_M, q5_K_S, q6_K`
+
+Quantization methods compatible with latest llama.cpp from June 6, commit 2d43387.
+
+---
+
+## Files
+| Name | Quant method | Bits | Size | Max RAM required, no GPU offloading | Use case |
+| ---- | ---- | ---- | ---- | ---- | ----- |
+| vicuna-33b.ggmlv3.q2_K.bin | q2_K | 2 | 13.71 GB | 16.21 GB | New k-quant method. Uses GGML_TYPE_Q4_K for the attention.vw and feed_forward.w2 tensors, GGML_TYPE_Q2_K for the other tensors. |
+| vicuna-33b.ggmlv3.q3_K_L.bin | q3_K_L | 3 | 17.28 GB | 19.78 GB | New k-quant method. Uses GGML_TYPE_Q5_K for the attention.wv, attention.wo, and feed_forward.w2 tensors, else GGML_TYPE_Q3_K |
+| vicuna-33b.ggmlv3.q3_K_M.bin | q3_K_M | 3 | 15.72 GB | 18.22 GB | New k-quant method. Uses GGML_TYPE_Q4_K for the attention.wv, attention.wo, and feed_forward.w2 tensors, else GGML_TYPE_Q3_K |
+| vicuna-33b.ggmlv3.q3_K_S.bin | q3_K_S | 3 | 14.06 GB | 16.56 GB | New k-quant method. Uses GGML_TYPE_Q3_K for all tensors |
+| vicuna-33b.ggmlv3.q4_0.bin | q4_0 | 4 | 18.30 GB | 20.80 GB | Original llama.cpp quant method, 4-bit. |
+| vicuna-33b.ggmlv3.q4_1.bin | q4_1 | 4 | 20.33 GB | 22.83 GB | Original llama.cpp quant method, 4-bit. Higher accuracy than q4_0 but not as high as q5_0. However has quicker inference than q5 models. |
+| vicuna-33b.ggmlv3.q4_K_M.bin | q4_K_M | 4 | 19.62 GB | 22.12 GB | New k-quant method. Uses GGML_TYPE_Q6_K for half of the attention.wv and feed_forward.w2 tensors, else GGML_TYPE_Q4_K |
+| vicuna-33b.ggmlv3.q4_K_S.bin | q4_K_S | 4 | 18.36 GB | 20.86 GB | New k-quant method. Uses GGML_TYPE_Q4_K for all tensors |
+| vicuna-33b.ggmlv3.q5_0.bin | q5_0 | 5 | 22.37 GB | 24.87 GB | Original llama.cpp quant method, 5-bit. Higher accuracy, higher resource usage and slower inference. |
+| vicuna-33b.ggmlv3.q5_1.bin | q5_1 | 5 | 24.40 GB | 26.90 GB | Original llama.cpp quant method, 5-bit. Even higher accuracy, resource usage and slower inference. |
+| vicuna-33b.ggmlv3.q5_K_M.bin | q5_K_M | 5 | 23.05 GB | 25.55 GB | New k-quant method. Uses GGML_TYPE_Q6_K for half of the attention.wv and feed_forward.w2 tensors, else GGML_TYPE_Q5_K |
+| vicuna-33b.ggmlv3.q5_K_S.bin | q5_K_S | 5 | 22.40 GB | 24.90 GB | New k-quant method. Uses GGML_TYPE_Q5_K for all tensors |
+| vicuna-33b.ggmlv3.q6_K.bin | q6_K | 6 | 26.69 GB | 29.19 GB | New k-quant method. Uses GGML_TYPE_Q8_K - 6-bit quantization - for all tensors |
+| vicuna-33b.ggmlv3.q8_0.bin | q8_0 | 8 | 34.56 GB | 37.06 GB | Original llama.cpp quant method, 8-bit. Almost indistinguishable from float16. High resource use and slow. Not recommended for most users. |
+
+---
+
+# Vicuna Model Card
+
+## Model Details
+
+Vicuna is a chat assistant trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT.
+
+- **Developed by:** [LMSYS](https://lmsys.org/)
+- **Model type:** An auto-regressive language model based on the transformer architecture.
+- **License:** Non-commercial license
+- **Finetuned from model:** [LLaMA](https://arxiv.org/abs/2302.13971).
+
+### Model Sources
+
+- **Repository:** https://github.com/lm-sys/FastChat
+- **Blog:** https://lmsys.org/blog/2023-03-30-vicuna/
+- **Paper:** https://arxiv.org/abs/2306.05685
+- **Demo:** https://chat.lmsys.org/
+
+## Uses
+
+The primary use of Vicuna is research on large language models and chatbots.
+The primary intended users of the model are researchers and hobbyists in natural language processing, machine learning, and artificial intelligence.
+
+## How to Get Started with the Model
+
+Command line interface: https://github.com/lm-sys/FastChat#vicuna-weights.  
+APIs (OpenAI API, Huggingface API): https://github.com/lm-sys/FastChat/tree/main#api.  
+
+## Training Details
+
+Vicuna v1.3 is fine-tuned from LLaMA with supervised instruction fine-tuning.
+The training data is around 140K conversations collected from ShareGPT.com.
+See more details in the "Training Details of Vicuna Models" section in the appendix of this [paper](https://arxiv.org/pdf/2306.05685.pdf).
+
+## Evaluation
+
+Vicuna is evaluated with standard benchmarks, human preference, and LLM-as-a-judge. See more details in this [paper](https://arxiv.org/pdf/2306.05685.pdf).
+
+## Difference between different versions of Vicuna
+See [vicuna_weights_version.md](https://github.com/lm-sys/FastChat/blob/main/docs/vicuna_weights_version.md)

vicuna-33b.ggmlv3.q2_K.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:088d4cf962be3dc6543d781f05190fcd4dbed122ab1cbd09e87e43f397b35442
+size 13705131392

vicuna-33b.ggmlv3.q3_K_L.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b146f1e44356958821aea106e6fd99c8c31b59dc4b8dec70b73fddb978bcbf00
+size 17279469952

vicuna-33b.ggmlv3.q3_K_M.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a753ece15d2649e96a4b8285499c4cb81b38a4f89b812615bc6260034bd2fc77
+size 15720368512

vicuna-33b.ggmlv3.q3_K_S.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87168669c42c6c92fd8ff8a813b4fcd2f41900e9f86b94e29fd5c90d1910163d
+size 14063823232

vicuna-33b.ggmlv3.q4_0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a478dd408ea0193ae4579326c7a163a51a7e13f38bdc9cbb6f2d1719f972f3da
+size 18300766592

vicuna-33b.ggmlv3.q4_1.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8612896713898ee47e75b04b338d0c16cce0d8fea9611f1f3aca51abf9cff06f
+size 20333775232

vicuna-33b.ggmlv3.q4_K_M.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09124ef4b91ca0b2cd2749803b0bd8056b796be9cd19c55b3aa881acc0a0389d
+size 19620851072

vicuna-33b.ggmlv3.q4_K_S.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:574be572239eb5dbfdb22d44cd741d2b5b0878e98a842135c925b88e9b89c4ce
+size 18355678592

vicuna-33b.ggmlv3.q5_0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:92f6edf814fc46b741fdd0a5449b18461e574748f8ca71699c9bcb97254da224
+size 22366783872

vicuna-33b.ggmlv3.q5_1.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1edaff8d1ecf86b55d85ef0c334701039d7306bdd8a95ad11a95b6761fbb7651
+size 24399792512

vicuna-33b.ggmlv3.q5_K_M.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fa565bdb7c8bd30455ef709a0ff5123216de8188e5c822f7bc28dfe561cfef8
+size 23046827392

vicuna-33b.ggmlv3.q5_K_S.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5eec17d851539d763d3aa2be6725a4fc43cf7d691dcc26724eb64db77339f84
+size 22395071872

vicuna-33b.ggmlv3.q6_K.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4802cd02b842584f78933ff232ccf3d48b32c6583cd4e158a7193043b85fc1f1
+size 26686927232

vicuna-33b.ggmlv3.q8_0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:638f515e564cc4dad561c48df534a8282848125f8259275aa9206620d86af7a6
+size 34564835712