IlyaGusev

/

saiga_13b_ggml

like 18

for token in generator:
# detokenize function may return incomplete bytes (not for printing), because one letter can consist of several tokens - see what we can make with it there: https://abetlen.github.io/llama-cpp-python/#__codelineno-0-673
# errors="ignore" - just simple trick to avoid exception
token_str = model.detokenize([token]).decode("utf-8", errors="ignore")

Fixed here: https://github.com/IlyaGusev/rulm/commit/5b31a0b5d6c0b564366025f42bc50ffc3b2c0b89