Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
eaddario 
posted an update 10 days ago
Post
2064
Squeezing out tensor bits, part II

At post time, watt-ai/watt-tool-70B continues to top the Berkeley Function-Calling Leaderboard, with the 8B version occupying the 4th place. A remarkable achievement for a model of that size!

The "squeezed" version is now available at eaddario/Watt-Tool-8B-GGUF

(For context please see: https://huggingface.co/posts/eaddario/832567461491467)
UICO

Well done! Your technique is very impressiove! BTW，Could you provide quantization for QWQ-32B?

·
eaddario

Thank you @UICO , but at the moment rather than a technique, it's more of a mix of brutish-force, educated guesses, trial and error and the occasional luck, but will tackle QwQ 32B next as it will help me validate an idea (see my next post)

In this post