Check new, much better, version of this model
This model has issues (trained without BOS token), please use the following preview models instead:
But no quants :|
Yea, I'm waiting on quants as well. I can just BARELY not run the full model on my VRAM haha.
I just spotted GGUF for one of them:
https://huggingface.co/localfultonextractor/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5-Q8_0-GGUF
Well, I was watching this drama and wanted to wait till a more "final" version appears. But I've put both in the queue and a full set of static quants should be available in a few hours.
Spoke too soon:
NotImplementedError: Unknown rope scaling type: dynamic
the models are not supported by llama.cpp at the moment it seems. Not without disabling rope scaling at least.
You can remove that from the config and use llama.cpp's own rope scaling.
Though I am surprised it throws an error like this.
llama.cpp can throw a lot of interesting errors, even with old models that did convert fine at the time :)
https://huggingface.co/mradermacher/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5-GGUF
BTW, anybody can request quants from me at https://huggingface.co/mradermacher/model_requests in cases I overlooked it. Can save the model creators a lot of time, too :)
Awesome, thank you @mradermacher !