Update README.md
Browse files
README.md
CHANGED
@@ -50,11 +50,21 @@ Mixtral 8x7B is a generative Sparse Mixture of Experts (SMoE) model designed to
|
|
50 |
|
51 |
### Core Library
|
52 |
|
53 |
-
Mixtral 8x7B Instruct
|
54 |
|
55 |
-
**Primary Framework**: `
|
56 |
-
**Alternate
|
57 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
58 |
|
59 |
### Safety and Responsible Use
|
60 |
|
|
|
50 |
|
51 |
### Core Library
|
52 |
|
53 |
+
Mixtral 8x7B Instruct is supported by multiple libraries to ensure flexibility for deployment and development. The primary frameworks include:
|
54 |
|
55 |
+
- **Primary Framework**: `llama.cpp`
|
56 |
+
- **Alternate Frameworks**:
|
57 |
+
- `transformers` for initial integration into Hugging Face's ecosystem.
|
58 |
+
- `vLLM` for highly optimized inference with low-latency serving.
|
59 |
+
|
60 |
+
You can access the model components and libraries here:
|
61 |
+
|
62 |
+
- **Model Base**: [mistralai/Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1)
|
63 |
+
- **Common Utilities**: [mistralai/mistral-common](https://github.com/mistralai/mistral-common)
|
64 |
+
- **Inference Optimization**: [mistralai/mistral-inference](https://github.com/mistralai/mistral-inference)
|
65 |
+
- **Quantization Support**: [ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp)
|
66 |
+
|
67 |
+
These resources provide a complete ecosystem for deployment, fine-tuning, and scaling sparse mixture models.
|
68 |
|
69 |
### Safety and Responsible Use
|
70 |
|