Repository: pandas-dev/pandas

Issue:
computation.tests.test_eval.TestScope.test_global_scope generates errors with numexpr engine and numpy 1.7.0
```
$ nosetests -v pandas.computation.tests.test_eval:TestScope.test_global_scope
pandas.computation.tests.test_eval.TestScope.test_global_scope('_var_s * 2', 'python', 'python') ... ok
pandas.computation.tests.test_eval.TestScope.test_global_scope('_var_s * 2', 'python', 'pandas') ... ok
pandas.computation.tests.test_eval.TestScope.test_global_scope('_var_s * 2', 'numexpr', 'python') ... ERROR
pandas.computation.tests.test_eval.TestScope.test_global_scope('_var_s * 2', 'numexpr', 'pandas') ... ERROR

======================================================================
ERROR: pandas.computation.tests.test_eval.TestScope.test_global_scope('_var_s * 2', 'numexpr', 'python')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/obrienjw/src/github/pandas/pandas/computation/tests/test_eval.py", line 1430, in check_global_scope
    parser=parser))
  File "/home/obrienjw/src/github/pandas/pandas/computation/eval.py", line 208, in eval
    ret = eng_inst.evaluate()
  File "/home/obrienjw/src/github/pandas/pandas/computation/engines.py", line 53, in evaluate
    self.expr.terms.return_type)
  File "/home/obrienjw/src/github/pandas/pandas/computation/align.py", line 250, in _reconstruct_object
    ret = ret_value.item()
IndexError: index 140737488325120 is out of bounds for axis 0 with size 10

======================================================================
ERROR: pandas.computation.tests.test_eval.TestScope.test_global_scope('_var_s * 2', 'numexpr', 'pandas')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/obrienjw/src/github/pandas/pandas/computation/tests/test_eval.py", line 1430, in check_global_scope
    parser=parser))
  File "/home/obrienjw/src/github/pandas/pandas/computation/eval.py", line 208, in eval
    ret = eng_inst.evaluate()
  File "/home/obrienjw/src/github/pandas/pandas/computation/engines.py", line 53, in evaluate
    self.expr.terms.return_type)
  File "/home/obrienjw/src/github/pandas/pandas/computation/align.py", line 250, in _reconstruct_object
    ret = ret_value.item()
IndexError: index 140737488325120 is out of bounds for axis 0 with size 10

----------------------------------------------------------------------
Ran 4 tests in 0.007s

FAILED (errors=2)
```

Here are the version details of my environment.

```
$ git describe
v0.12.0-1110-gd250d64
$ ci/print_versions.py

INSTALLED VERSIONS
------------------
Python: 2.7.5.final.0
OS: FreeBSD 9.2-PRERELEASE FreeBSD 9.2-PRERELEASE #0 r254955: Tue Aug 27 11:01:00 EDT 2013     root@drivel.saltant.net:/usr/obj/usr/src/sys/NARB amd64
byteorder: little
LC_ALL: None
LANG: None

pandas: Not installed
Cython: 0.19.1
Numpy: 1.7.0
Scipy: 0.12.1
statsmodels: Not installed
    patsy: Not installed
scikits.timeseries: Not installed
dateutil: 2.1
pytz: 2013.8
bottleneck: Not installed
PyTables: Not Installed
    numexpr: 2.2.2
matplotlib: 1.2.0
openpyxl: Not installed
xlrd: Not installed
xlwt: Not installed
xlsxwriter: Not installed
sqlalchemy: Not installed
lxml: Not installed
bs4: Not installed
html5lib: Not installed
bigquery: Not installed
apiclient: Not installed
```

Edited title to reflect latest analysis; affected code depends on the version of numpy, not the platform.



Discussion:
Weird question: does it generate exactly the same error each time?

Yes.

```
$ for i in $(seq 100); do nosetests -v pandas.computation.tests.test_eval:TestScope.test_global_scope 2>&1 | grep IndexError; done | sort -u
IndexError: index 140737488325120 is out of bounds for axis 0 with size 10
```

Thanks, didn't mean to make you run that 100 times there.

I wouldn't have known the answer without running it a bunch of times. ;-)


Resolving patch:
diff --git a/pandas/computation/align.py b/pandas/computation/align.py
--- a/pandas/computation/align.py
+++ b/pandas/computation/align.py
@@ -249,6 +249,9 @@ def _reconstruct_object(typ, obj, axes, dtype):
 
     try:
         ret = ret_value.item()
-    except ValueError:
+    except (ValueError, IndexError):
+        # XXX: we catch IndexError to absorb a
+        # regression in numpy 1.7.0
+        # fixed by numpy/numpy@04b89c63
         ret = ret_value
     return ret


Test patch: