sympy/sympy-bot — pull request #161 by asmeurer (2013-04-12 02:45:34+00:00)
Title: Add testing of doctest coverage with --doc-coverage

To run, use `--doc-coverage`. `--doc-coverage` automatically enables
`--build-docs`, because the docs must be built to get the Sphinx coverage.

The output so far looks something like this:

**[SymPy Bot][sympy-bot] Summary**: :white_check_mark: Passed after merging asmeurer/travis-sphinx (2e1817c7a1fb39a0c798d557ad48e54a0cabe5c1) into master (0ec0ba5a953f8c52d971f9bd1aff9c6089b98766).
:white_check_mark: **Sphinx 1.2b1**: [pass](http://reviews.sympy.org/report/agZzeW1weTNyDAsSBFRhc2sYhpB1DA)
:red_circle: **Doc Coverage**: ([Failed](http://reviews.sympy.org/report/agZzeW1weTNyDAsSBFRhc2sY7Zd1DA))
36% (1983 of 5425) of functions have doctests
40% (2221 of 2221) of functions are imported into Sphinx

Still TODO:

- [ ] (Only if someone wants it) allow to customize the args sent to
`coverage_doctest.py`. This would allow testing the coverage of specific
modules, or not testing Sphinx coverage.

- [ ] Cleanup display of the output. I was thinking of using some kind of
table. HTML tables should be supported. I'm open to suggestions here.

- [ ] Add comparative testing against master, so that we can see if the
coverage improved or got worse.

- [ ] Figure out a good [Emoji](http://www.emoji-cheat-sheet.com/) to use for
the above. I was thinking :chart_with_upwards_trend: if the coverage
improved and :chart_with_downwards_trend: if it got worse, but those look
kind of the same. What I really want is a green arrow pointing up and a red
arrow pointing down. Maybe we should just grab some custom images and host
them in the sympy-bot repo, and just show those. Again, I'm open to
suggestions.

- [ ] Should we pass -v to coverage_doctest.py? The answer is probably yes,
though the output is very verbose in that case.

- [ ] Test this.