oskusalerma/trelby — Issue #85: Support more than ISO-8859-1 (Latin-1)
Author: oskusalerma
Created: 2012-01-04 19:08:49+00:00
State: open

Common request people make is for Trelby to support more than the ISO-8859-1 (also known as Latin-1) character set to write scripts in. We already support most Western European languages (see http://en.wikipedia.org/wiki/ISO/IEC_8859-1 for the full list), but there are plenty of characters used in other languages not found in Latin-1.

Things that would need to be implemented to support arbitrary Unicode characters in scripts, ranked in decreasing order of difficulty:

* Unicode characters in PDF. Hard problem; PDF spec is oriented towards Latin-1 and basically says that to use anything else, you must embed your own font, subset the characters you use, and do complicated lookup tables to encode the text. That's how it was 5 years ago anyway when I last looked at it; maybe the spec has improved since then? See http://stackoverflow.com/questions/128162/unicode-in-pdf and http://blogs.adobe.com/insidepdf/2008/07/text_content_in_pdf_files.html for some discussions about this.

* Text input. Right now, OnKeyChar gets a key event and it contains an 8-bit character code. To support input of unicode characters, we need to do more. Maybe use the wxKeyEvent::GetUnicodeKey function? See http://docs.wxwidgets.org/2.8/wx_wxkeyevent.html. Note that Latin-1 characters are easy to input, they're a single keypress on most keyboards. Fancier unicode characters sometimes have very sophisticated input systems where people type in something, a popup shows up, they select the character they want, etc. I have no idea how those input systems integrate to wxWidgets, or if they do at all.

* All the text processing in Trelby would need to be redone to do it in Unicode instead. util.py has a whole bunch of string handling functions, for example.

* Probably more that I'm forgetting right now.