Repository: Mozilla-Ocho/llamafile

Issue #646: Bug: Broken display in Hebrew letters for chat interface in llamafiler-0.8.17
Created: 2024-12-01T02:44:05Z
Contact Details
nh.local11@gmail.com
What happened?
When using a Hebrew model in the chat interface, the Hebrew content was displayed in a distorted way. At the time of the articles in English, the problem did not exist.

Version
llamafiler 0.8.17
What operating system are you seeing the problem on?
Windows
Relevant log output
note: if you have an AMD or NVIDIA GPU then you need to pass -ngl 9999 to enable GPU offloading
2024-12-01T02:43:04.206826 llamafile/server/listen.cpp:41 server listen http://127.0.0.1:8080
2024-12-01T02:43:04.209006 llamafile/server/main.cpp:111 server warning: this OS doesn't support pledge() security
2024-12-01T02:43:15.394912 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/index.html
2024-12-01T02:43:15.410383 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/chatbot.css
2024-12-01T02:43:15.412911 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight.css
2024-12-01T02:43:15.413364 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/ctype.js
2024-12-01T02:43:15.413429 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/clipboard.js
2024-12-01T02:43:15.422203 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight.js
2024-12-01T02:43:15.424159 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_txt.js
2024-12-01T02:43:15.426316 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_cpp.js
2024-12-01T02:43:15.426460 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_markdown.js
2024-12-01T02:43:15.431832 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_c.js
2024-12-01T02:43:15.432774 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_d.js
2024-12-01T02:43:15.435739 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_cxx.js
2024-12-01T02:43:15.438625 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_ada.js
2024-12-01T02:43:15.440089 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_asm.js
2024-12-01T02:43:15.440109 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_basic.js
2024-12-01T02:43:15.440560 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_cobol.js
2024-12-01T02:43:15.442594 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_csharp.js
2024-12-01T02:43:15.444589 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_forth.js
2024-12-01T02:43:15.447003 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_fortran.js
2024-12-01T02:43:15.447170 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_go.js
2024-12-01T02:43:15.447629 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_haskell.js
2024-12-01T02:43:15.448217 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_js.js
2024-12-01T02:43:15.453629 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_css.js
2024-12-01T02:43:15.454430 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_php.js
2024-12-01T02:43:15.462153 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_html.js
2024-12-01T02:43:15.462411 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_java.js
2024-12-01T02:43:15.462846 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_julia.js
2024-12-01T02:43:15.463938 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_kotlin.js
2024-12-01T02:43:15.465346 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_ld.js
2024-12-01T02:43:15.467466 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_lisp.js
2024-12-01T02:43:15.472629 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_lua.js
2024-12-01T02:43:15.472842 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_m4.js
2024-12-01T02:43:15.473084 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_make.js
2024-12-01T02:43:15.473512 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_matlab.js
2024-12-01T02:43:15.475573 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_ocaml.js
2024-12-01T02:43:15.476327 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_pascal.js
2024-12-01T02:43:15.480527 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_perl.js
2024-12-01T02:43:15.481226 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_r.js
2024-12-01T02:43:15.480612 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_python.js
2024-12-01T02:43:15.482604 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_ruby.js
2024-12-01T02:43:15.489304 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_rust.js
2024-12-01T02:43:15.491199 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_scala.js
2024-12-01T02:43:15.499722 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_shell.js
2024-12-01T02:43:15.500234 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_sql.js
2024-12-01T02:43:15.501331 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_tcl.js
2024-12-01T02:43:15.502669 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_swift.js
2024-12-01T02:43:15.504998 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_tex.js
2024-12-01T02:43:15.505745 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_typescript.js
2024-12-01T02:43:15.512989 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_zig.js
2024-12-01T02:43:15.515922 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/highlight_cmake.js
2024-12-01T02:43:15.517339 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/render_markdown.js
2024-12-01T02:43:15.517423 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/chatbot.png
2024-12-01T02:43:15.517525 llamafile/server/client.cpp:783 127.0.0.1 served /zip/www/chatbot.js
2024-12-01T02:43:25.008172 llamafile/server/slot.cpp:304 127.0.0.1 prefilled 42 tokens (after removing 0 and reusing 0)
2024-12-01T02:43:41.304813 llamafile/server/client.cpp:484 127.0.0.1 writev failed EPIPE
2024-12-01T02:43:41.305112 llamafile/server/slots.cpp:104 127.0.0.1 relinquishing slot

Duplicate issue #722: Bug: Chinese coding error in Server V2
Created: 2025-03-18T16:55:41Z
What happened?

When using Chinese, the output will be with coding error (I guess this also happens using other non-ascii language). Server V1 is fine.
Version
v0.9.1
What operating system are you seeing the problem on?
Windows
Relevant log output

Closing comment (2025-03-18T17:03:50Z): I think this is probably similar to #646
@corebonts I know you are already doing a ton of work in the UI (thank you so much), not sure if you have time to check this one out as well. If you do, thank you, and if not, no problem.