tiarno/plastex — pull request #3 by paternal (2014-07-06 18:00:30+00:00)
Title: Solved encoding bug

At several places in plasTeX, the string ``string.letters`` is used. It works well when manipulating English strings, but in a French environment, as the ``string.letters`` [depends on the current locale](https://docs.python.org/2/library/string.html#string.letters), it contains characters with accents (like `é`), and encoding matters, and we get the usual:
```python
[...]
File "/usr/lib/pymodules/python2.7/plasTeX/TeX.py", line 387, in parse
for item in tokens:
File "/usr/lib/pymodules/python2.7/plasTeX/TeX.py", line 46, in next
return self._next()
File "/usr/lib/pymodules/python2.7/plasTeX/TeX.py", line 275, in __iter__
token = next()
File "/usr/lib/pymodules/python2.7/plasTeX/TeX.py", line 244, in itertokens
t = inputs[-1][-1].next()
File "/usr/lib/pymodules/python2.7/plasTeX/Tokenizer.py", line 440, in __iter__
if token[-1] in string.letters:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xaa in position 52: ordinal not in range(128)
```

This patch solves this. A unittest is given, that fails without the patch, and succeed with it.

The principle of this patch is: instead of using directly `string.letters`, we convert it to unicode using the encoding of the current locale if it is set.

By the way: is there any plan to publish a new version? The last one (0.9.3) is [five years old](https://github.com/tiarno/plastex/commit/0e8f235cd04fcc807a841754e0fe416681612815), and the versions that can be installed using pip or GNU/Linux distributions are very old : [pip](https://pypi.python.org/pypi/plasTeX), [debian](https://packages.debian.org/source/squeeze/plastex), [opensuse](http://software.opensuse.org/package/plasTeX?search_term=plastex)…