python-imaging/Pillow — pull request #594 by hugovk (2014-04-02 11:05:19+00:00)
Title: Add coverage reports for Python code on Travis CI

For Travis CI, this runs the tests and measures coverage, and creates reports on Coveralls like this:

https://coveralls.io/r/hugovk/Pillow
https://coveralls.io/builds/647958

Coveralls is free for open source projects, and integrates easily with Travis CI. A repo admin/owner will need to enable it here: https://coveralls.io/repos/new

This PR also provides a summary at the end of the build log:
```
$ coverage report
Name Stmts Miss Cover
-----------------------------------------
PIL/BmpImagePlugin 118 93 21%
PIL/GifImagePlugin 243 164 33%
PIL/Image 907 513 43%
PIL/ImageColor 49 11 78%
PIL/ImageDraw 203 139 32%
PIL/ImageFile 264 168 36%
PIL/ImageFilter 92 26 72%
PIL/ImageMath 153 59 61%
PIL/ImageMode 20 1 95%
PIL/ImagePalette 127 96 24%
PIL/JpegImagePlugin 307 203 34%
PIL/JpegPresets 154 0 100%
PIL/PngImagePlugin 343 283 17%
PIL/PpmImagePlugin 71 19 73%
PIL/PyAccess 163 75 54%
PIL/__init__ 3 0 100%
PIL/_binary 25 11 56%
PIL/_util 12 5 58%
-----------------------------------------
TOTAL 3254 1866 43%
```

Notes:

* `coverage` is run with `--append` because it's run for both `selftest.py` and the main ` Tests/run.py`. `coverage erase` is run once first to make sure there's no old report. This doesn't matter on the CI but of course does if you run locally. If `coverage` was run just once for, say, ` Tests/run.py`, `coverage erase` and `--append` may be removed.

* `--include=PIL/*` means only those files under PIL/ are included. Without it, it would include files like `/usr/local/pypy/lib_pypy/cffi/ffiplatform.py` which we don't care about. Current coverage with this option: 43.09%. https://coveralls.io/builds/647958 This is the option in this PR.

* `--include=PIL/*` also means only executed files are covered. For a more complete report, that includes those files under PIL that aren't even executed (and thus lowers the coverage, replace `--include=PIL/*` with `--source=PIL`. Current coverage with this option: 15.1%. https://coveralls.io/builds/647930 Not used in this PR.

* Higher coverage is generally better, it shows if tests execute the code (and Travis CI says if they passed) for each build and gives more confidence. 100% isn't needed and there's clearly other priorities to work toward than increasing coverage. Pushing for 100% , but knowing the coverage may help writing new tests or making sure new code is properly tested.

* Doesn't make build times noticeably slower. My last covered build took 16 min 53 sec. The non-covered build I forked from took 16 min 51 sec.