Air Striker Mixtral
Hello! Could you please quantize LoneStriker's Air Strikers? Both ZLoss and Instruct-ZLoss would be appreciated!
https://huggingface.co/LoneStriker/Air-Striker-Mixtral-8x7B-ZLoss-GGUF
https://huggingface.co/LoneStriker/Air-Striker-Mixtral-8x7B-Instruct-ZLoss-GGUF
The models are recommended by AliCat, but unfortunately only static GGUF quants are available, and as such, Q3 performance leaves a lot to be desired. Ant just so happens to be the optimal model precision to run on a system with 32GB of RAM and 10-12GB of VRAM. Also, since the author didn't release full precision models, only LORAs, and it presents a problem for you, I'd suggest using the merges provided by sandmanbuzz
https://huggingface.co/sandmanbuzz/Air-Striker-Mixtral-8x7B-ZLoss
https://huggingface.co/sandmanbuzz/Air-Striker-Mixtral-8x7B-ZLoss-Instruct
indeed, my pipeline can't automatically deal with loras, sot hanbks for providing the alternative urls. both models are now in the queue. static quants will be generated first.
They should be incoming within the next hours or day:
https://huggingface.co/mradermacher/Air-Striker-Mixtral-8x7B-ZLoss-i1-GGUF
https://huggingface.co/mradermacher/Air-Striker-Mixtral-8x7B-ZLoss-Instruct-i1-GGUF