Update with explanation of why it breaks this way

tl;dr Layer size affects the transform, make the m34 property on the appropriate CATransform3D rotation small enough so that it won't throw out the view's frame to unusable proportions