Repository: pandas-dev/pandas

Issue:
TST/BUG: stata failures on big endian
related #5778

Investigate/fix whether the test failures (currently skipped on big-endian) are actual bugs or just an odd test.

cc @PKEuS 



Discussion:
@jreback , what where the other failing tests? need to mark everything dirty until fixed.
I suspect the user base for stata on sparc numbers somewhere in the lower 0'z, so perhaps
it's just fine to blacklist the whole thing and call it duly mitigated.

If this isn't resolved by 0.14 release time please ping me and I'll just nuke it from space.

@y-p there are 5 skips; my brief investigation is that it needs to do some kind of byte swapping in order to compose some of the data on read-back. I don't know enough about what its supposed to do.

Hi guys -- would anyone have time to look at this one?
It keeps buildbots unhappy on the sparc box (http://nipy.bic.berkeley.edu/builders/pandas-py2.x-sid-sparc/builds/768/steps/shell_7/logs/stdio) and other glorious architectures e.g. https://buildd.debian.org/status/fetch.php?pkg=pandas&arch=powerpc&ver=0.14.0~rc1%2Bgit79-g1fa5dd4-1&stamp=1401244093 

Would anyone have time/interest to look into it?

just going to disable the tests (again)

oh no... ok -- let me look into it at least briefly.

On Wed, 28 May 2014, jreback wrote:

>    just going to disable the tests (again)
> 
>    —
>    Reply to this email directly or [1]view it on GitHub.
> 
> References
> 
>    Visible links
> 1. https://github.com/pydata/pandas/issues/5781#issuecomment-44443632

## 

Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,            Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

I think the reading/writing is prob incorrect (e.g. may need byteswapping on big endian) as these are raw byte reads...but noone I know has a big endian :) so not sure what can do about it

On Wed, 28 May 2014, jreback wrote:

>    I think the reading/writing is prob incorrect (e.g. may need byteswapping
>    on big endian) as these are raw byte reads...but noone I know has a big
>    endian :) so not sure what can do about it

I think I gave you access to a big endian already... at least I see a
jreback user on that vagus box ;)

## 

Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,            Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

I know, that's how I debugged the value_counts issue :)

no time to do anything about these stata issues (and its a low-level code reading)...this issue will stay open, maybe someone will get to it at some point


Resolving patch:
diff --git a/pandas/io/stata.py b/pandas/io/stata.py
--- a/pandas/io/stata.py
+++ b/pandas/io/stata.py
@@ -561,7 +561,7 @@ def _read_header(self):
                 raise ValueError("Version of given Stata file is not 104, "
                                  "105, 108, 113 (Stata 8/9), 114 (Stata "
                                  "10/11), 115 (Stata 12) or 117 (Stata 13)")
-            self.byteorder = self.path_or_buf.read(1) == 0x1 and '>' or '<'
+            self.byteorder = struct.unpack('b', self.path_or_buf.read(1))[0] == 0x1 and '>' or '<'
             self.filetype = struct.unpack('b', self.path_or_buf.read(1))[0]
             self.path_or_buf.read(1)  # unused
 


Test patch:
diff --git a/pandas/io/tests/test_stata.py b/pandas/io/tests/test_stata.py
--- a/pandas/io/tests/test_stata.py
+++ b/pandas/io/tests/test_stata.py
@@ -19,9 +19,6 @@
 from pandas.util.misc import is_little_endian
 from pandas import compat
 
-if not is_little_endian():
-    raise nose.SkipTest("known failure of test_stata on non-little endian")
-
 class TestStata(tm.TestCase):
 
     def setUp(self):