|
--- |
|
license: apache-2.0 |
|
base_model: v2ray/Mixtral-8x22B-v0.1 |
|
inference: false |
|
model_creator: MaziyarPanahi |
|
model_name: Mixtral-8x22B-v0.1-GGUF |
|
pipeline_tag: text-generation |
|
quantized_by: MaziyarPanahi |
|
tags: |
|
- quantized |
|
- 2-bit |
|
- 3-bit |
|
- 4-bit |
|
- 5-bit |
|
- 6-bit |
|
- 8-bit |
|
- 16-bit |
|
- GGUF |
|
- mixtral |
|
- moe |
|
--- |
|
|
|
# Mixtral-8x22B-v0.1-GGUF |
|
|
|
in progress ... |
|
|
|
## Load sharded model |
|
|
|
`llama_load_model_from_file` will detect the number of files and will load additional tensors from the rest of files. |
|
|
|
``` |
|
main --model Mixtral-8x22B-v0.1.fp16-00001-of-00005.gguf -ngl 64 |
|
``` |
|
|
