CI: use versioneer to have PEP440 versions
State: closed

- use versioneer, for PEP440 version strings, closes #9518 
- use binstar build
- OSX stack testing, closes #7127

This removes all of the original manual-ish logic to create/manage the versioning in `setup.py`, and replaces with `versioneer.py` (bundled inline) to handle correct versioning for release, development, and conda builds.

This actually was tricky simply because of the way the tags are setup, they are like: `v0.16.2` which is pretty non-standard, e.g. most would simply have `0.16.0`. This messes with comparision using `LooseVersion` (of course you _can_ replace this tag as versioneer does, its called the `tag_prefix`, but his makes other things require a slightly different method).

So bottom line we get PEP compliant versions like this:

```
In [1]: pd.__version__
Out[1]: '0.16.2+10.gc01927b'
```

IOW, 10 commits past the last tag and the git hash.

Now also able to build on `anaconda.org` for all platforms (`linux-32,linux-64,OSX,win-32,win-64`). This part is not 100% done, but quite close. The intention is to have testing and artifact building (well at least for say 2.7/3.4) for linux,osx,win that can be directly `conda install pandas -c pandas` to get the dev version.


Labels:
name: Build
description: Library building on various platforms
name: CI
description: Continuous Integration

Comments:
Big :+1: 
It will be really good that people can quickly test the dev version by just conda installing it!

Some comments:
- We should maybe use a 'pandas' organisation on anaconda.org? (as now they seem to be uploaded to your user name) There is already https://anaconda.org/pandas, but I don't know if this is an organisation or a normal user account. And it should be a bit updated anyway (maybe it does not need a picture of Philip .. :-))
- Should we use a 'dev' channel on anaconda.org for the nightly builds?
- Should the generated `_version.py` be included in git?
- So there is no place in the code (as now in setup.py) anymore where you have to change a number for the release? Does it get it then from the git tags?

> We should maybe use a 'pandas' organisation on anaconda.org? (as now they seem to be uploaded to your user name) There is already https://anaconda.org/pandas, but I don't know if this is an organisation or a normal user account. And it should be a bit updated anyway (maybe it does not need a picture of Philip .. :-))
- the `pandas` account that exists now will be the 'offical' account. Should change the picture, and I will add the other core devs access.

> Should we use a 'dev' channel on anaconda.org for the nightly builds?
- its actually simpler to get it from `pandas` but can do this. I think @cpcloud had some arguments about this for `blaze`.

> Should the generated _version.py be included in git?
- yes its included in the repo.

> So there is no place in the code (as now in setup.py) anymore where you have to change a number for the release? Does it get it then from the git tags?
- yes it gets it from the git tag (AND it save it when the tagging happens / sdist and such), so that it will work in dev and production. The original code (what I deleted), basically did this, but versioneer handles all of the edge cases (as well as makes the string formatting configurable).

I added @shoyer & @jorisvandenbossche 

I think you can read-write here `https://anaconda.org/pandas`

@TomAugspurger @sinhrks if you would setup an `anaconda.org` account (free, just sign up). Then I will add you guys.

Great!! I've created my account. https://anaconda.org/sinhrks

 https://binstar.org/taugspurger

Thanks for setting this up Jeff.

Is there actually a way we can change the 'pandas' profile (I only find my own profile). Because eg there is a typo in the website.

I added @TomAugspurger and @sinhrks 
fixed the url; now you guys should be able to edit anything .

@jreback thanks for the detailed answers!

> > Should the generated _version.py be included in git?
> > yes its included in the repo.

I actually wanted to ask whether it should rather _not_ be included (was not very clear), as the generated `_version.py` is now also not included in git, and you seem to explicitly state in the manifest that this should be included in archives.

@jorisvandenbossche the `pandas/_version.py` IS included in the repo. Its similar to the old `pandas/version.py` but will dynamically rather than statically give you the version number.

I'm still _slightly_ concerned about making it impossible to install pandas dev versions with old versions of pip, but all things considered probably better just to adopt this approach and move on

did you try with an old version of pip? why wouldn't it work?

@jreback see https://github.com/pydata/pandas/issues/9518#issuecomment-93519732

These new PEP440 compliant versions work with pip 6.0 or later, which was released on 2014-12-22.

We should probably add a note to this effect in the dev install instructions.

I don't think that's useful as the older pip doesn't handle a PEP version string. I did manage to repro the error though; fails for `pip` < 6. All that said, the recommended way to actually install is:

`python setup.py install`, so I don't think this is really an issue

my point is that where does it say to use `pip install -e .`. I have personally never see that as a method of installing anything.

Okay, fair enough. 

But I think `pip install https://github.com/pydata/pandas/archive/master.zip` is maybe something that people use. I just recommended it to a colleague who had to install a dev version and didn't have git installed (but it was for a pure python package, I wouldn't recommend it for pandas).

If I read the versioneer explanation, I understand that on the moment `git archive` is performed, the version tag is inserted in the `pandas/_version.py` file. I tried this with your branch, but if I download the archive, the lines:

```
    git_refnames = "$Format:%d$"
    git_full = "$Format:%H$"
```

are replaced by:

```
    git_refnames = " (build)"
    git_full = "6b1055c765c488e6fb411eed452abf75ff8df4bb"
```

So the commit hash is included, but I don't see any mention of 0.16.2. But didn't try to install it to see what the version gives.

@jreback I suppose the `_version.py` file now needs to be in the git repo for when you download it from github the version gets included, but now, the `version.py` file was not included, it was explicitly ignored in `.gitignore` . So you can probably remove that from `.gitignore`

If you install master version but without git, you get:

```
>>> pandas.__version__
'0+unknown'
```

which is indeed a bit unfortunate.. 
But when downloading the zipped version of a tagged commit, then it works.

Quoting from https://github.com/warner/python-versioneer/blob/master/details.md:

> Sometimes you use the VCS tools to make a tarball directly, like git archive, without using setup.py sdist. Unpacking such a tarball results in the third environment, "from-keyword". The resulting source tree will contain expanded keywords in the _version.py file, which tells it a git revision, and an exact tag (if any), but cannot generally detect things like commits-since-recent-tag, or recent-tag at all. 

any issues with this

@jorisvandenbossche 
@shoyer 
@cpcloud 

otherwise going to bombs away (soon)

let's do it

ok, bombs away. 

Great!

@jreback Will this enable builds on the pandas anaconda channel? As for now, if I look at https://anaconda.org/pandas/pandas/builds, there is nothing to see yet. 
If this becomes possible, we can also update the docs that if you want to test a development version, you can install this with conda.