Fix quirks in rotation_matrix

As discussed in more detail on the astropy list, the  implementation of `rotation_matrix` for arbitrary axes had some rounding and sign convention issues.  This PR fixes that, updates `angle_axis` to be more consistent with it, and adds a few tests of these functions.
Should we consider `np.asarray` here?  It probably amounts to the same thing but I find `asarray` more descriptive of intent than `array(..., copy=False)`.