TheBloke
/

Mixtral-8x7B-Instruct-v0.1-AWQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions Community

Use ybelkada/Mixtral-8x7B-Instruct-v0.1-AWQ with VLLM instead

#10

by sqreept - opened Apr 10

sqreept

Apr 10

This repo doesn't produce output with VLLM. But ybelkada/Mixtral-8x7B-Instruct-v0.1-AWQ is working correctly.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment