CI: Fixed NumPy pinning in conda-build
State: closed

Closes #19572 
closes #19153 

(testing locally as well too)

Labels:
name: CI
description: Continuous Integration

Comments:
Built fine locally.
local building has always worked fine, its branches that are failing *only* sometimes.
Ah, OK.

Thoughts on failing that worker entirely if the conda-build command fails?

I also see that the command we run is `time conda build ./conda.recipe --numpy 1.13 --python 3.5 -q --no-test`

I suppose that should be bumped?
yes should bump that. I *think* there is an issue about 1.14 somewhere as well.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/19575?src=pr&el=h1) Report
> Merging [#19575](https://codecov.io/gh/pandas-dev/pandas/pull/19575?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/6b0c7e72b141831b7a9a5651f9e19eef53ec9e76?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/19575/graphs/tree.svg?src=pr&token=eZ4WkYLtcO&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/19575?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #19575      +/-   ##
==========================================
+ Coverage   91.62%   91.62%   +<.01%     
==========================================
  Files         150      150              
  Lines       48781    48795      +14     
==========================================
+ Hits        44695    44708      +13     
- Misses       4086     4087       +1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.99% <ø> (ø)` | :arrow_up: |
| #single | `41.72% <ø> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/19575?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/category.py](https://codecov.io/gh/pandas-dev/pandas/pull/19575/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9jYXRlZ29yeS5weQ==) | `97.26% <0%> (-0.26%)` | :arrow_down: |
| [pandas/core/groupby.py](https://codecov.io/gh/pandas-dev/pandas/pull/19575/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS5weQ==) | `92.12% <0%> (-0.05%)` | :arrow_down: |
| [pandas/core/indexes/datetimelike.py](https://codecov.io/gh/pandas-dev/pandas/pull/19575/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZWxpa2UucHk=) | `97.05% <0%> (-0.05%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/19575/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.42% <0%> (ø)` | :arrow_up: |
| [pandas/core/sparse/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/19575/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL2ZyYW1lLnB5) | `94.81% <0%> (ø)` | :arrow_up: |
| [pandas/core/indexing.py](https://codecov.io/gh/pandas-dev/pandas/pull/19575/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhpbmcucHk=) | `93.11% <0%> (ø)` | :arrow_up: |
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/19575/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `96.25% <0%> (+0.01%)` | :arrow_up: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/19575/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `95.93% <0%> (+0.01%)` | :arrow_up: |
| [pandas/core/reshape/merge.py](https://codecov.io/gh/pandas-dev/pandas/pull/19575/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9tZXJnZS5weQ==) | `94.25% <0%> (+0.03%)` | :arrow_up: |
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/19575/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `94.87% <0%> (+0.13%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/19575?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/19575?src=pr&el=footer). Last update [6b0c7e7...6f5f87a](https://codecov.io/gh/pandas-dev/pandas/pull/19575?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

numba packages for numpy 1.14 are out now, so I removed that pin.

Trying without any pins for the conda build recipe. We'll see.
so this is still initially installing 1.13 and building, then upgrading to 1.14 after, not sure whey that is. I also *think* we DO need the pinnning on the conda build, it looks like its using a pretty old version (1.11) for building. It still works but is slightly odd.
OK, I think I have it so we build against 1.11 and run against NumPy latest (lmk if you want something else for run). I think we do want to build against 1.11 though, since that's the oldest ABI compatible NumPy version that supports Python 3.5, and it's what we use for conda-forge / manylinux builds.
yeah that kind right
we want to build against the oldest version we support for ABI - i guess u can build with 1.9 but maybe we have something in the recipe which doesn’t allow that

this still looks odd. we are installing 1.14, then building with 1.11 (i guess min numpy version), but end up with 13.3 (see show_versions() at the end)

```
# Name                    Version                   Build  Channel
pandas                    0.23.0.dev0     np111py35_259_g5d52a3a    local
```
Missed a pin. Hopefully now.
Still ended up with 1.13. One of these packages is forcing a downgrade.

```
The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    python-dateutil-2.6.1      |           py35_0         237 KB  conda-forge
    parquet-cpp-1.2.0          |                0         1.2 MB  conda-forge
    openssl-1.0.2n             |                0         3.5 MB  conda-forge
    numpy-1.13.3               |py35_nomklhb7e7d7c_0         3.8 MB
    pandas-0.22.0              |           py35_0        25.6 MB  conda-forge
    certifi-2017.11.5          |           py35_0         195 KB  conda-forge
    feather-format-0.4.0       |           py35_2           5 KB  conda-forge
    arrow-cpp-0.5.0            |      np113py35_2         2.0 MB  conda-forge
    pyarrow-0.5.0              |      np113py35_0         600 KB  conda-forge
    jemalloc-4.4.0             |                0         2.3 MB  conda-forge
    ca-certificates-2017.11.5  |                0         145 KB  conda-forge
    ------------------------------------------------------------
                                           Total:        39.6 MB

The following NEW packages will be INSTALLED:

    arrow-cpp:       0.5.0-np113py35_2           conda-forge
    feather-format:  0.4.0-py35_2                conda-forge
    jemalloc:        4.4.0-0                     conda-forge
    pandas:          0.22.0-py35_0               conda-forge
    parquet-cpp:     1.2.0-0                     conda-forge
    pyarrow:         0.5.0-np113py35_0           conda-forge
    python-dateutil: 2.6.1-py35_0                conda-forge

The following packages will be UPDATED:

    ca-certificates: 2017.08.26-h1d4fec5_0                   --> 2017.11.5-0                 conda-forge
    openssl:         1.0.2n-hb7f436b_0                       --> 1.0.2n-0                    conda-forge

The following packages will be DOWNGRADED:

    certifi:         2018.1.18-py35_0                        --> 2017.11.5-py35_0            conda-forge
    numpy:           1.14.0-py35_nomklh7cdd4dd_0             [nomkl] --> 1.13.3-py35_nomklhb7e7d7c_0             [nomkl]

```

pyarrow is pinned to 0.5 for this build. Do you have a preference for

1. pyarrow 0.5 & NumPy 1.13
2. pyarrow / NumPy latest

If 1, this can be merged. Else I'll remove that pyarrow pin.
> pyarrow 0.5 & NumPy 1.13
> pyarrow / NumPy latest

hmm would just bump the pyarrow pin to 0.6.0 i think.
0.7.1 was the first one that has a binary that supports numpy 1.14.

On Fri, Feb 9, 2018 at 6:09 AM, Jeff Reback <notifications@github.com>
wrote:

> pyarrow 0.5 & NumPy 1.13
> pyarrow / NumPy latest
>
> hmm would just bump the pyarrow pin to 0.6.0 i think.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/19575#issuecomment-364417154>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHInkt7h6VjscT0uih_PRYp7tlfdYSks5tTDWQgaJpZM4R8zHy>
> .
>

OK, https://api.travis-ci.org/v3/job/339442588/log.txt did it correctly. To summarize

1. Build against NumPy 1.11: https://travis-ci.org/pandas-dev/pandas/jobs/339442588#L1131
2. Run in an env with NumPy 1.1.4: https://travis-ci.org/pandas-dev/pandas/jobs/339442588#L947
3. Pyarrow is pinned to 0.7.1: https://travis-ci.org/pandas-dev/pandas/jobs/339442588#L1038