I would be in favor of including the sphinx utilities in ``astropy_helpers`` too. At the moment, one has to install ``astropy`` in RTD for example, rather than just be able to 'mock' it, which slows down documentation builds a lot. It seems a bit over the top to have to install astropy just to build a bunch of HTML pages.