Template format
#7
by
elsatch
- opened
Hi!
I have been trying to convert and quantize the model so it can be used in Ollama. One of the steps requieres a TEMPLATE definition, that describes the kind of format used in in the model. I haven't found that info in the model card. I have looked for the documentation of the BLOOM project and the format appears to be:
TEMPLATE "[INST] {{ .Prompt }} [/INST]"
However, when I use that format with the base FLOR-6.3B model, I am getting lots of repetitions in the output, including the [/INST] token. Maybe this is because this format is only used in the Instruct version.
Is there any particular format that needs to be used to produce the outputs of the FLOR-6.3B base model?