BLD: Add useful shortcuts to Makefile
State: closed

Add some shortcuts to make it easier to develop with pandas.

now there's a set of commands in the `Makefile` in the top-level pandas directory with the following functionality
- `make clean` will delete the `build` and `dist` directories + all `*.pyc` and `*.so` files
- `make clean_pyc` just removes `*.pyc` files
- `make build` will build extensions inplace
- `make develop` will install `pandas` in your environment but will place a link to the dev dir so that you can make changes and they will show up immediately
- `make doc` will build the documentation from scratch (erases generated and build directories)


Labels:

Comments:
@jtratner looks good
@cpcloud @hayd 
anyone have an issue with this?

can u add

``` makefile
doc: develop
    -rm -rf doc/source/generated
    -cd doc
    -python make.py clean
    -python make.py html
    -cd ..
```

or something like it? i'm always building docs now that it's like a second mini testing suite :)

the only thing is I always have to add my path to `doc/source/conf.py` for some reason whenver I need to builds docs...e.g.

`sys.path.insert(0,'/home/jreback/pandas'`

or is this just me being stupid?

hm i don't get that, but i remember i used to get it on another project and do the same thing to my path, i never changed it so i'm not sure what the fix is

@cpcloud I added the task you recommended. Suggestion for you too: you might try installing [guard](https://github.com/guard/guard) and then using this [Guardfile](https://gist.github.com/jtratner/5733131) that I wrote up for the pandas repo. This should, in general, automatically run tests as you change files (and I added a part so that it would automatically rebuild the docs if you make changes). That said, are you sure that you want the `doc` task to also call `develop`? That causes it to remove the build folder, rebuild all the extensions and then install it again into your site-packages. You might want to remove develop and keep it separate.

Hold on - don't merge this, made a mistake in the Makefile.

@jtratner ur right def should remove the `develop` dependency. i'm not sure i want tests run every time i change a file, that would lead to so many breaks since i cannot write correct code the first time every time... ;) i like `pyflakes` on save in `vim` cuz it's fast and catches the stupid errors whereas i leave crazy runtime errors and bugs to nose...maybe not the most efficient. sometimes i use `pep8` too but because all of the interesting files in pandas are so not pep8 compliant that's slow too :(

on second thought i'm not sure i like removing the extensions since they take so long to build what about just removing pyc and having another make directive that removes the extensions explicitly like `make rmbuild` or something. i often rebase and then build and maybe a SINGLE extension has changed so i just want to do `develop` on pandas right after i pull in those changes...make sense? any other opinions?

Sure, I think that makes sense . `make clean` handles removing the build file, so that doesn't need to change. I also added `clean_pyc` as a target to the `clean` task.

I believe this all works now.

@jtratner oh right I forgot about this feature/quirk. I haven't used make in a while. 

definitely a quirk

On Sat, Jun 8, 2013 at 1:15 AM, Phillip Cloud notifications@github.comwrote:

> @jtratner https://github.com/jtratner oh right I forgot about this
> feature/quirk. I haven't used make in a while.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/pull/3803#issuecomment-19143307
> .

@jtratner just noticed that the `doc` directive should contain `-rm -rf doc/build` as well.

wonder if there are any other mindless tasks we can delegate to the gods of `make`.

@cpcloud we could improve `make test` since it only tests sparse (which makes little sense to me).

I was thinking it could be improved to:

``` Makefile
test: build
     -./test_fast.sh
```

Another option could be:

``` Makefile
perf: build
      -./test_perf.sh
```

`test_fast.sh` yields a 3.5x ish speedup for me. i go back and forth with how useful that is since you might just add more time having to fix bugs that weren't caught by the "fast" and you still have to run the "slow" one at that point anyway.

there's a psychological benefit to not having to wait 3.5 minutes for the full test suite to finish especially if i didn't make any changes in the things that run slow...HOWEVER of course you could have touched something in the "fast" parts that breaks something in the "slow" parts.

can u have args to make directives? like `make test fast` (run `./test_fast`) and `make test` (default test everything) note that both `test.sh` and `test_fast.sh` take args i don't remember how `make` deals with passing arguments from the shelll....

@cpcloud it's tricky to [add arguments to Makefiles](http://stackoverflow.com/questions/6273608/how-to-pass-argument-to-makefile-from-command-line). I'd be okay with adding `test_fast`, which runs `test_fast.sh` and `test` which runs all tests, but going in any more detail will probably end up making the Makefile more fragile.

I'm sticking to using a Guardfile with pandas, because it makes it so you don't have to think about running the tests :)

ur right that looks outrageous

@cpcloud so, which tasks do you want here?

up to u but i'm partial to `test_fast` and `test`...b4 by "sparse" i thought u meant `test_fast` not only testing sparse data strcts...my bad

although i think u could prolly leave em out since it's less to type `./test_<TAB>`

this looks good 2 me...@jreback ?

this is ok too

@jtratner anything else b4 i merge?

@cpcloud do you think that build should really be silenced? (i.e., if it encounters an error it ignores it) Maybe it makes more sense to have it fail loudly.

not sure what u mean...is that what the "-" does?

@cpcloud Yes. so right now, if you did `make develop` and `make build` failed with an error code, it would still try to run `python setup.py develop` even though the build didn't work.

i would say always propagate errors since you don't want things to just blindly continue if something failed...:) even though errors are of course annoying better to fail fast then fail much, much later after 1000 things have gone wrong! i just thought that the "-" was `make` syntax for "do this in a shell" or some kind of strange legacy syntax..

@jtratner Sorry about that. Not a huge make user. I usually end up just writing bash functions to do my bidding... 

@cpcloud this is ready to go now. It will fail on all the build tasks, but ignores failures on cleans (which is what you want, because it ignores things like "no matching files" if you haven't built anything yet).