pydata/pandas — pull request #2224 by y-p (2012-11-11 18:25:14+00:00)
Title: Unicode : change df.to_string() and friends to always return unicode objects

**Note**: Although all the tests pass with minor fixes, this PR has an above-average chance of
breaking things for people who have relied on broken behaviour thus far.

```python
a=pd.Series([u"\u05d0"]*1000)
a.name= 'title1'
repr(a)

pandas/pandas/core/series.py in _tidy_repr(self, max_vals)
869 name=False)
870 result = head + '\n...\n' + tail
--> 871 return '%s\n%s' % (result, self._repr_footer())
872
873 def _repr_footer(self):

UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 6: ordinal not in range(128)
```

`df.tidy_repr` combines several strings to produce a result. when one component is unicode
and other other is a non-ascii bytestring, it tries to convert the latter back to a unicode string
using the 'ascii' codec and fails.

I suggest that `_get_repr` -> `to_string` should always return unicode, as implemented by this PR,
and force_unicode be deprecated.

The `force_unicode` argument in `to_string` conflates two things:
- which codec to use to decode the string (which will always be a hack/guess,
unless we got that information by other means).
- whether to return a unicode() object or str() object,

The first is now no longer necessary since `pprint_thing` already resorts to the same hack
of using utf-8 (with errors='replace') as a fallback.
I believe the making the latter an option is wrong, precisely because it brings about situations
like the test case above result, `to_string`, like all internal functions , should utilize
unicode objects, whenever feasible.