astropy/astropy — pull request #2028 by taldcroft (2014-01-31 22:08:00+00:00)
Title: Refactor io.ascii to use six without unicode_literals

This is an alternative to #1950 which is basically the same except that it does not `import unicode_literals` in any files. The motivation is:

- The basis for `io.ascii` is the std. lib `csv` module, which explicitly does not support unicode directly in Python 2.
- `csv` breaks if any of the inputs are unicode.
- Solutions for Python 2 unicode support basically involve encoding in UTF-8 and then working with the resulting bytestrings.
- There is processing in `io.ascii` that involves mixing user-provided input (i.e. lines of table strings) and code literals. If any of those literals are unicode then strings get upcast to unicode and everything breaks.

With this, the current state is that `io.ascii` really only supports ASCII data in Python 2. Using `unicode_literals` in the source just forces a lot of explicit `str()` conversions back to the `str` type. Subsequent code maintenance will be require remembering to always wrap string literals with `str()`.

In theory we could provide some support for parsing unicode input files, but that is a separate issue. I would say this is better handled by using Python 3.

cc: @mdboom (resident six / unicode expert)