biocore/bipy — pull request #134 by jrrideout (2014-03-10 18:20:14+00:00)
Title: DOC: remove 'bipy.' prefix and add support for documenting special/private methods

RST files now have shorter names with the removal of the 'bipy.' prefix. For example, ```bipy.core.sequence.rst``` became ```core.sequence.rst```. This matches scipy's doc layout.

Now for the interesting part- apologies for the novel, but I think these details are important for discussion and future reference.

Added the latest dev version of numpydoc to the repo, as I needed to extend/customize it a bit. This seems to be a standard practice in other packages, such as pandas and scikit-learn. I included numpydoc's license at licenses/numpydoc.txt.

numpydoc does not support building documentation for private (i.e. methods that start with ```_```) or special (e.g., ```__getitem__```, ```__str__```, etc.) methods. Since we have a number of classes that implement various special methods, we need a way to have their docstrings processed during the build. This was why I needed to customize numpydoc.

There were a number of different ways to solve this, and I toyed around with a few different "solutions" (they are all hacks...). A few ideas I had:

- numpydoc already has a hack in place that makes an exception for ```__call__```. If ```__call__``` is defined in a class, its docs are built, but all other private/special methods are ignored. This was hardcoded into numpydoc. We could extend that list to include other special methods that we expect bipy classes to define. I don't like this approach because the number of special methods in Python is [very large](http://docs.python.org/2/reference/datamodel.html#specialnames), may change in the future, and it's hard to tell what special methods we actually want to generate docs for, because many of them have default implementations in ```object```. This leads to classes having a bunch of special methods documented when they didn't override them.

- assume method names starting and ending with ```__``` should be documented. Again, we run into the issue above where we're generating documentation for the default implementations in ```object```. A check could be put in place to only process special methods that are defined directly within a class, but then subclasses wouldn't have these methods documented. This would be inconsistent with the current system, where subclasses have inherited methods included in their docs.

- tag private/special methods to let numpydoc know that the docs should be built. I ended up going this route and added a new "directive" to numpydoc. If you add the line ```.. shownumpydoc``` to a method's docstring, numpydoc will build the docs for that method instead of ignoring it.

I like this solution because:

- the directive is unobtrusive: If viewing the docstring in an interpreter, it simply looks like a bit of markup, which we already have in our docstrings. When processed by numpydoc, the directive is parsed as a sphinx comment and omitted from the generated file (so it doesn't show up on the website).

- it is explicit: devs must explicitly tag special methods that they want documented. This is a good thing because there may be cases where we *don't* want a special method documented (maybe the behavior is very obvious and there's no point to documenting it, or maybe we don't want a certain special method showing up as part of the public interface). It'll force devs to think about whether or not they want a special method documented, and gives direct control over this decision, instead of relying on implicit rules/exceptions.

- it works with inheritance: if you tag a special method in a parent class, all children will automatically have their inherited methods tagged as well.

- the implementation is maintainable: adding the directive to numpydoc was just a single line change.

This is definitely a hack, but so are the other options (and there are several hacks within numpydoc already, so I don't feel too dirty). Of course, this will be well documented when #120 is completed.

What do you guys think? Hate it, can live with it, or don't care 'cause you didn't want to read the novel? :D

Fixes #129 #131.