init

README.md

@@ -1,3 +1,37 @@
 ---
 license: apache-2.0
+inference: false
+library_name: mlx
 ---
+
+# Model Card for Mixtral-8x7B 4 bit
+
+The Mixtral-8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts. The Mixtral-8x7B outperforms Llama 2 70B on most benchmarks we tested.
+
+For full details of this model please read our [release blog post](https://mistral.ai/news/mixtral-of-experts/).
+
+## Instruction format
+
+This format must be strictly respected, otherwise the model will generate sub-optimal outputs.
+
+The template used to build a prompt for the Instruct model is defined as follows:
+```
+<s> [INST] Instruction [/INST] Model answer</s> [INST] Follow-up instruction [/INST]
+```
+Note that `<s>` and `</s>` are special tokens for beginning of string (BOS) and end of string (EOS) while [INST] and [/INST] are regular strings.
+
+## Run the model
+
+```bash
+# Install mlx, mlx-examples, huggingface-cli
+pip install mlx
+pip install huggingface_hub hf_transfer
+git clone https://github.com/ml-explore/mlx-examples.git
+
+# Download model
+export HF_HUB_ENABLE_HF_TRANSFER=1
+huggingface-cli download --local-dir Mixtral-8x7B-Instruct-v0.1-4-bit https://huggingface.co/hurongliang/Mixtral-8x7B-Instruct-v0.1-4-bit
+
+# Run example
+python mlx-examples/llms/mixtral/mixtral.py --model_path Mixtral-8x7B-Instruct-v0.1-4-bit
+```

config.json

@@ -0,0 +1,19 @@
+{
+    "dim": 4096,
+    "n_layers": 32,
+    "head_dim": 128,
+    "hidden_dim": 14336,
+    "n_heads": 32,
+    "n_kv_heads": 8,
+    "norm_eps": 1e-05,
+    "vocab_size": 32000,
+    "moe": {
+        "num_experts_per_tok": 2,
+        "num_experts": 8
+    },
+    "quantization": {
+        "group_size": 64,
+        "bits": 4
+    },
+    "model_type": "mixtral"
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

weights.0.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:527927721a735aa0eb9a539b164dc9f9ba79e7d76e33d046cc96cd97b80e37d5
+size 3601584424

weights.1.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5920ffee12ebc6f970e502fa1f3c2821b24039c23ed3b76ee830e94c527e2adc
+size 3601584424

weights.2.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc52bf84d4a5ff6f13e2dc87b4d19ee6da3b79b278588973dae321a70cd106d3
+size 3601584772

weights.3.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a98570f0c2456d613ff975c3635340aefd2f0b48856291e76b060ef5ebc2f12
+size 3601585120

weights.4.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d38cfe4ab8f91adfb39fb2b030f0c367f248d2dc9ec366bc92fc883b633996e
+size 3601585120

weights.5.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02d5faab866d587a69eb6686ef83d4e233fa99e86e859f57137e40592653ca8c
+size 3601585120

weights.6.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eee13a3035f6ad3808376a55d3e2c6b5be713b546533dc64af5ca645005c2c1c
+size 3601585120

weights.7.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebf0780fd807327750538e4b01d3ed4163bbe8fd3efdc54c7c371e923f27aa18
+size 3601585120