Numexpr 2.4.6
State: closed

 - [x] closes #15213 
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry

This addresses #15213.  Actually, the recommended version for me is 2.6.2, but I understand that this is too recent still.

Labels:
name: Dependencies
description: Required and optional dependencies

Comments:
@FrancescAlted we have a few occurences of numpexpr pinned at older versions in ci/requirement files (I think the 2.7_COMPAT.run and 3.4_SLOW.run ones). Best to change those versions to 2.4.6 then.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15383?src=pr&el=h1) Report
> Merging [#15383](https://codecov.io/gh/pandas-dev/pandas/pull/15383?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/d9e75c7e724e5f7449c8c57624ce9395c9ffe11a?src=pr&el=desc) will **decrease** coverage by `-0.02%`.
> The diff coverage is `100%`.


```diff
@@            Coverage Diff             @@
##           master   #15383      +/-   ##
==========================================
- Coverage   90.42%   90.41%   -0.02%     
==========================================
  Files         134      134              
  Lines       49357    49375      +18     
==========================================
+ Hits        44632    44640       +8     
- Misses       4725     4735      +10
```


| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/15383?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/computation/__init__.py](https://codecov.io/gh/pandas-dev/pandas/compare/d9e75c7e724e5f7449c8c57624ce9395c9ffe11a...c081199fe35de6b4d3510e56a9ab4621e03bbe16?src=pr&el=tree#diff-cGFuZGFzL2NvbXB1dGF0aW9uL19faW5pdF9fLnB5) | `90.9% <100%> (+13.98%)` | :white_check_mark: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/compare/d9e75c7e724e5f7449c8c57624ce9395c9ffe11a...c081199fe35de6b4d3510e56a9ab4621e03bbe16?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `16.46% <ø> (-0.42%)` | :x: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/compare/d9e75c7e724e5f7449c8c57624ce9395c9ffe11a...c081199fe35de6b4d3510e56a9ab4621e03bbe16?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `96.33% <ø> (ø)` | :white_check_mark: |
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/compare/d9e75c7e724e5f7449c8c57624ce9395c9ffe11a...c081199fe35de6b4d3510e56a9ab4621e03bbe16?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `94.15% <ø> (ø)` | :white_check_mark: |
| [pandas/core/missing.py](https://codecov.io/gh/pandas-dev/pandas/compare/d9e75c7e724e5f7449c8c57624ce9395c9ffe11a...c081199fe35de6b4d3510e56a9ab4621e03bbe16?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvbWlzc2luZy5weQ==) | `84.95% <ø> (+0.19%)` | :white_check_mark: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15383?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15383?src=pr&el=footer). Last update [d9e75c7...c417fe2](https://codecov.io/gh/pandas-dev/pandas/compare/d9e75c7e724e5f7449c8c57624ce9395c9ffe11a...c417fe2bcde23dbe7ea7516dc703cd7554fd5cb3?el=footer&src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
There are a couple of ["allowed failures" in TravisCI](https://travis-ci.org/pandas-dev/pandas/builds/201215599).  Any hint on what's going on there?
For the MacOSX one, I see 

```
UnsatisfiableError: The following specifications were found to be in conflict:
  - numexpr 2.4.6
  - pytables 3.0.0 -> numexpr 2.2*
```


Uh, I see the Mac OSX build that completed just fine.  Also, the AppVeyor failure seems like a transitory failure during conda packages access:

```
conda.exceptions.CondaRuntimeError: Runtime error: RuntimeError: Runtime error: HTTPError: 502 Server Error: Bad Gateway for url: https://repo.continuum.io/pkgs/free/win-64/xlsxwriter-0.9.6-py35_0.tar.bz2: https://repo.continuum.io/pkgs/free/win-64/xlsxwriter-0.9.6-py35_0.tar.bz2
```

But I am still unsure on the level of importance of the ["allowed failures" in TravisCI](https://travis-ci.org/pandas-dev/pandas/builds/201215599).
Sorry, I mislooked, not the MacOSX build of course, build number 6 with the old versions (2.7_COMPAT), the error now is:

```
UnsatisfiableError: The following specifications were found to be in conflict:
  - numexpr 2.4.6 -> numpy 1.10*
  - numpy 1.7.1
Use "conda info <package>" to see the dependencies for each package.
```

We want to keep the numpy 1.7 there for now (although there is an issue to bump that requirement as well), so I would just keep the older numexpr in this build (then it is a test that you get the correct error for that).

> But I am still unsure on the level of importance of the "allowed failures" in TravisCI.

They are not 'allowed' to fail in principle (only for specific things, eg compat issue with numpy master that is not yet fixed). I think it is like that to get quicker feedback from travis.

There is still a lint error:

```
pandas/computation/__init__.py:18:17: E128 continuation line under-indented for visual indent
```

The appveyor issue indeed does not seem to be related to this PR.


@FrancescAlted Was it needed to also revert the version change in the 3.4_SLOW build? As this one was not failing I think.
Yes, it was: https://travis-ci.org/pandas-dev/pandas/builds/201238220.  I am not familiar with the convention to run different configurations, but I'd say that stems from: https://github.com/pandas-dev/pandas/blob/master/.travis.yml#L131.  I just reverted the numexpr versions existing in this two scenarios.  My intention is to look at the possible errors after the revert and try to fix them.
i would change this version to 2.4.6, otherwise lgtm. https://github.com/pandas-dev/pandas/blob/master/ci/requirements-3.4_SLOW.run

ping on green.
I think we will still have a remaining error [here](https://github.com/pandas-dev/pandas/blob/master/pandas/tests/computation/test_compat.py#L29).  But the assert will not raise the error as such, and we would need something like `from pandas.computation import __NUMEXPR_INSTALLED`.  However, there is a line at the beginning saying `from pandas.computation import _MIN_NUMEXPR_VERSION` which already raises the warning, and hence the error is not re-raised.  Not sure how to cope with that.  Opinions?
So probably that test never worked .. What you could do is move the import inside the test and in a "catch warnings" context, and then check that a warning is raised in the specific case and not otherwise (https://docs.python.org/3.5/library/warnings.html#testing-warnings)?
no it worked but the import process is changed by this PR
it needs to be protected

I have been looking at the original code, and I don't think it worked because `assert not _NUMEXPR_INSTALLED` cannot possibly raise a Warning; can it?
so I think with the changed up order of tests, this ``test_compat`` that is failing (on 2.7_COMPAT) is not able to assert the warning as its already been produced (you can see in the output actually).

So I guess just change the test then.
Ok, I think that I have found the issue and fixed it in e1b34a9.  Basically, one needs to force the `reload()` of `pandas.computation` module to trigger the `UserWarning` in import time.  Hopefully we should see all green after this one.
Well, it turns out that `reload()` is tricky and it still cannot raise the `UserWarning` in Python 2.7 (although 3.4 seems to work well).  After googling a bit, I have read that [`reload()` is very tricky to get right](http://stackoverflow.com/a/684229/3095325), so my vote is to remove this `UserWarning` test completely (unless you have a better proposition).
@FrancescAlted I would just remove the check for the warning (leave the test just don't assert that it produces)
lgtm. ping on green.
Done.  Hopefully this will give us all green (bar some temporary failures that apparently are usual in TravisCI).
@FrancescAlted well there have been some flakiness with some gbq tests; I turned them off for now. We *should* have all green. 

FYI we just have the fail fast to make the results appear quicker. We have LOTS of configurations to test. So allowed failures are really NOT allowed :>


one more FYI. every push on a PR generates a new travis run (which can take hours). So generally like to bunch them up if possible. This PR of course is a case that its necessary to test little changes so no big deal.
thanks @FrancescAlted