Make astropysics a test-only dependency

Note that you do need install astrophysics for full functionality. scipy is required by astropy, so it is also listed as a test dependency.