How to use text-generation-inference/Mixtral-8x7B-Instruct-v0.1-medusa with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("text-generation-inference/Mixtral-8x7B-Instruct-v0.1-medusa", device_map="auto")
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("text-generation-inference/Mixtral-8x7B-Instruct-v0.1-medusa", device_map="auto")