Is `model.gguf` t5-small or flan-t5-small?

by Felladrin - opened Oct 26, 2023

Discussion

Felladrin

Oct 26, 2023

•

edited Oct 26, 2023

Hi @lmz ,

I would appreciate it if you could clarify whether model.gguf is related to t5-small or flan-t5-small.
Asking because this repository has a config-flan-t5-small.json without a corresponding model-flan-t5-small.gguf, and by @radames ' space (https://huggingface.co/spaces/radames/Candle-T5-Generation-Wasm), I'd guess the file is related to t5-small:

See also: Code reference

If that's the case, could you upload a model-flan-t5-small.gguf?

lmz

Owner Oct 26, 2023

It's indeed t5-small, you can see where the mapping is done in our command line example here. This was the first model to be quantized hence the generic name.

Felladrin

Oct 26, 2023

Thanks for confirming! So I'm going to generate flan-t5-small.gguf following those instructions from the readme.

Felladrin changed discussion status to closed Oct 26, 2023

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment