TheBloke

/

Tess-M-v1.3-GGUF

like 11

@migtissera In tokenizer_config.json you have "eos_token": "</s>", - is this correct?

Yep! That’s how I trained it.

Maybe its a llama.cpp setting of mine causing it?

#!/bin/bash

./main -n -1 -ins -m "$1" --color -c 4096 --multiline-input -t 14 -ngl 48 --log-disable

Thats my boot script

The GPTQ version works well. I had to add EOS and BOS tokens to the tokenizer.

Can you check here @dillfrescott ? https://github.com/ggerganov/llama.cpp/issues/1776

Hmm interesting. What should I set that line to before compiling it? Im bad at this

I think the issue is that its not throwing the EOS token when it should, and just rambling on until forced to stop.

Maybe @wolfram can help here?