Repository: pandas-dev/pandas

Issue:
test_missing_value_conversion on ubuntu 13.10 32bit KeyError: 2147483647
```
======================================================================
ERROR: test_missing_value_conversion (pandas.io.tests.test_stata.TestStata)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/buildd/pandas-0.15.1+git125-ge463818/debian/tmp/usr/lib/python3/dist-packages/pandas/io/tests/test_stata.py", line 651, in test_missing_value_conversion
    parsed_113 = read_stata(self.dta17_113, convert_missing=True)
  File "/tmp/buildd/pandas-0.15.1+git125-ge463818/debian/tmp/usr/lib/python3/dist-packages/pandas/io/stata.py", line 69, in read_stata
    order_categoricals)
  File "/tmp/buildd/pandas-0.15.1+git125-ge463818/debian/tmp/usr/lib/python3/dist-packages/pandas/io/stata.py", line 1278, in data
    missing_value = StataMissingValue(um)
  File "/tmp/buildd/pandas-0.15.1+git125-ge463818/debian/tmp/usr/lib/python3/dist-packages/pandas/io/stata.py", line 646, in __init__
    self._str = self.MISSING_VALUES[value]
KeyError: 2147483647
```

doesn't happen on amd64

```
INSTALLED VERSIONS
------------------
commit: None
python: 3.3.2.final.0
python-bits: 32
OS: Linux
OS-release: 3.2.0-4-amd64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: C
LANG: C

pandas: 0.15.1.dev
nose: 1.3.0
Cython: 0.19
numpy: 1.7.1
scipy: 0.12.0
statsmodels: None
IPython: None
sphinx: 1.1.3
patsy: None
dateutil: 2.0
pytz: 2012c
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.2.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.2.0
html5lib: None
httplib2: None
apiclient: None
rpy2: None
sqlalchemy: None
pymysql: None
psycopg2: None
```



Discussion:
cc @bashtage

Bizzare.  This is straight python

``` python
MISSING_VALUES = {}
bases = (101, 32741, 2147483621)
for b in bases:
    MISSING_VALUES[b] = '.'
    for i in range(1, 27):
        MISSING_VALUES[i + b] = '.' + chr(96 + i)
```

This certainly appears to create `MISSING_VALUE[2147483621 + 26]` (2147483647).

I don't have any 32 bit to test on - I suppose that is is a 32 bit issue since this is the largest 32-bit integer

so on 32-bit
the max values i `np.iinfo(np.int32).max` (which is that number)

you cannot add to it (well you can, but it squelches the overflow) and its 'undefined' IIRC.
however I have seen where the max value 'wraps' around, the the max values i one less that as show here.

are these bases a stata thing? 

Yes, these are a Stata choice to use the highest integer values of each type to represent missing values.  From what I can see, it should be perfectly fine since it is not adding anything to this value.

Just some thoughts - I am guessing that `hash(np.int32(2147483647))` is not the same as `hash(int(2147483647))` - perhaps numpy is treating it as  `2147483647L` which might have a different hash on 32 bit.  I suppose one way to workaround this would be to explicitly cast any integer values to int when looking up (any missing < 2147483647  must be an int).


Resolving patch:
diff --git a/pandas/io/stata.py b/pandas/io/stata.py
--- a/pandas/io/stata.py
+++ b/pandas/io/stata.py
@@ -611,9 +611,10 @@ class StataMissingValue(StringMixin):
     MISSING_VALUES = {}
     bases = (101, 32741, 2147483621)
     for b in bases:
-        MISSING_VALUES[b] = '.'
+        # Conversion to long to avoid hash issues on 32 bit platforms #8968
+        MISSING_VALUES[compat.long(b)] = '.'
         for i in range(1, 27):
-            MISSING_VALUES[i + b] = '.' + chr(96 + i)
+            MISSING_VALUES[compat.long(i + b)] = '.' + chr(96 + i)
 
     float32_base = b'\x00\x00\x00\x7f'
     increment = struct.unpack('<i', b'\x00\x08\x00\x00')[0]
@@ -643,6 +644,8 @@ class StataMissingValue(StringMixin):
 
     def __init__(self, value):
         self._value = value
+        # Conversion to long to avoid hash issues on 32 bit platforms #8968
+        value = compat.long(value) if value < 2147483648 else float(value)
         self._str = self.MISSING_VALUES[value]
 
     string = property(lambda self: self._str,
@@ -1375,13 +1378,6 @@ def _pad_bytes(name, length):
     return name + "\x00" * (length - len(name))
 
 
-def _default_names(nvar):
-    """
-    Returns default Stata names v1, v2, ... vnvar
-    """
-    return ["v%d" % i for i in range(1, nvar+1)]
-
-
 def _convert_datetime_to_stata_type(fmt):
     """
     Converts from one of the stata date formats to a type in TYPE_MAP


Test patch: