Configurability of unicode/console encoding
State: closed

#1620

checks whether there are non-ascii characters and tries to convert them if possible
allows encoding to be set in set_printoptions (default UTF8)


Labels:

Comments:
Crap, I fixed this on the plane. Looks like you did a better job by
adding a configuration option

Sent from my mobile device

On Jul 20, 2012, at 6:20 PM, Chang She
reply@reply.github.com
wrote:

> checks whether there are non-ascii characters and tries to convert them if possible
> allows encoding to be set in set_printoptions (default UTF8)
> 
> You can merge this Pull Request by running:
> 
>  git pull https://github.com/changhiskhan/pandas nonascii
> 
> Or you can view, comment on it, or merge it online at:
> 
>  https://github.com/pydata/pandas/pull/1654
> 
> -- Commit Summary --
> - BUG: try to convert non-unicode non-ascii characters in repr #1620
> 
> -- File Changes --
> 
> M pandas/core/common.py (8)
> M pandas/core/format.py (33)
> M pandas/tests/test_format.py (7)
> 
> -- Patch Links --
> 
>  https://github.com/pydata/pandas/pull/1654.patch
>  https://github.com/pydata/pandas/pull/1654.diff
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/pydata/pandas/pull/1654

Doh, I claimed it but forgot to push yesterday, my bad

On Jul 20, 2012, at 6:24 PM, Wes McKinney wrote:

> Crap, I fixed this on the plane. Looks like you did a better job by
> adding a configuration option
> 
> Sent from my mobile device
> 
> On Jul 20, 2012, at 6:20 PM, Chang She
> reply@reply.github.com
> wrote:
> 
> > checks whether there are non-ascii characters and tries to convert them if possible
> > allows encoding to be set in set_printoptions (default UTF8)
> > 
> > You can merge this Pull Request by running:
> > 
> > git pull https://github.com/changhiskhan/pandas nonascii
> > 
> > Or you can view, comment on it, or merge it online at:
> > 
> > https://github.com/pydata/pandas/pull/1654
> > 
> > -- Commit Summary --
> > - BUG: try to convert non-unicode non-ascii characters in repr #1620
> > 
> > -- File Changes --
> > 
> > M pandas/core/common.py (8)
> > M pandas/core/format.py (33)
> > M pandas/tests/test_format.py (7)
> > 
> > -- Patch Links --
> > 
> > https://github.com/pydata/pandas/pull/1654.patch
> > https://github.com/pydata/pandas/pull/1654.diff
> > 
> > ---
> > 
> > Reply to this email directly or view it on GitHub:
> > https://github.com/pydata/pandas/pull/1654
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/pydata/pandas/pull/1654#issuecomment-7143211

My standard comment for anything unicode related - are we sure this won't mess things up in Python 3? I'm not sure when these functions get called, but, for instance, if `_conv_unicode()` is called on a `bytes` string, it will always return False.

Hey @changhiskhan I pushed the fix I'd cooked up ae70acc and 4e94fba. Unit test suite is passing-- it would be nice to incorporate the configurability of the encoding in this PR, so I'm going to leave it open for now. 

#2005 also fixes this, replaces the small helper functions with the generic 
print_thing() function, and fixes up the repr to always use the encoding in 
print_config on py2.x,  and leave things well alone on py3.

unless I missed something , vote to close? 

Agreed-- it looks like the encoding has already been made configurable in `set_printoptions` so we can close this.