Revert "Re-organize the setup.py so that Astropy is not required for egg_info"

This reverts commit 9aa533907f20f346b0cc55749224f6a15ff7e22a.