MAINT: Remove unnecessary check from before swapaxes (API change)

This check was a speed optimization in the numpy code. Due to
a small change in the MapIter API, the check is now wrong in
corner cases and becomes fully unnecessary.
The previous API had not been public in any release numpy
version.