Update Tokenizer Chat Template
Fixed 'chat_template' so it uses the proper prompt formatting... hopefully...
Luna Translator doesn't have any options to edit instruct format, so I assume it pulls from the model metadata.
Thanks! The updated template should indeed work better, but did you encounter any issues with the previous one?
Luna Translator doesn't have any options to edit instruct format, so I assume it pulls from the model metadata.
That depends on your inference engine. Some don't pull from the model and instead define the format in their own way.
but did you encounter any issues with the previous one
Not yet so far, I'm going through a range of backends. Tried llamacpp and ollama so far.
llamacpp doesn't allow custom templates, however while using it I found out that Luna's regenerate translation feature doesn't remove the old translation from the context.
Because of this, while I found it works well, Go Template doesn't allow math like jinja2 does, which I'll need to create a chat template that removes subsequent user duplicate chat pairs.
I'll be trying Ooba tomorrow. IIRC it reads from the metadata as well as allow setting/creating a template within the Web UI, so hopefully its api works with Luna.
EDIT: Scratch that, apparently llamacpp does read metadata, so you CAN use custom templates through that. I'll try the Jinja script through that. It's just used to pick the closest template... God dammit...