Remove Sphinx conf `numpydoc` requirement.

This will likely be needed later, however, the requirement is
affecting the RTD build.