Weighted GGUFs Available
#4
by
InferenceIllusionist
- opened
Hi there. As always, appreciate your efforts in pushing the limits of what's possible (and am a big fan of mergekit as well!).
Just a heads up I went ahead and made some weighted GGUF quants here so that others can have a chance to test this unique model too:
https://huggingface.co/InferenceIllusionist/llama3-42b-v0-iMat-GGUF
More files will be added to the repo shortly but just wanted to say thanks for sharing your process around pruning. Looking forward to seeing the final evaluation results.
This is great, thanks for doing this!