Only convert array if needed

This is a fix for https://github.com/scipy/scipy/issues/4203
is this somehow different from `np.asarray(numpy_array, dtype=detype)`?