Hugging Face's logo

llm-agents
/
tora-code-34b-v1.0

Text Generation
Transformers
PyTorch
English
llama
code
math
text-generation-inference
Inference Endpoints
Model card Files Community
4

GPTQ version pls

#1
by Yhyu13 - opened
Discussion
Yhyu13

@TheBloke

Would you like to make quantized versions for this model which claims

is currently the first and only open-source model to achieve over 50% accuracy (pass@1) on the MATH dataset, which significantly outperforms GPT-4’s CoT result (51.0 vs. 42.5), and is competitive with GPT-4 solving problems with programs.

https://github.com/microsoft/ToRA

zubingou
 LLM-Agents org

Thanks for your interest, you can load with checkpoint with gptq directly.

zubingou changed discussion status to closed
TheBloke

@Yhyu13 I've added this to the queue for GGUF, GPTQ and AWQ quants.

@zubingou GPTQs have to be pre-made, you can't load a model checkpoint with GPTQ directly. Maybe you're thinking of BitsandBytes?

zubingou
 LLM-Agents org

I apologize for the confusion, and thank you for your clarification and support, @TheBloke !

Sign up or log in to comment