Instructions to use text-generation-inference/Mixtral-8x7B-Instruct-v0.1-medusa with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use text-generation-inference/Mixtral-8x7B-Instruct-v0.1-medusa with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("text-generation-inference/Mixtral-8x7B-Instruct-v0.1-medusa", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update config.json
#6 opened 10 months ago
by
Biswarup001
Generating the custom model
#5 opened over 2 years ago
by
ngupta24
Throughput Measurement
#4 opened over 2 years ago
by
jacktenyx
What about latencies
👍 1
#3 opened over 2 years ago
by
LorenzoCevolaniAXA