MAINT: optimize: remove dead code

ndim of arrays that were previously flattened were being checked.
(The check and flatten could be swapped, but that would change current
behavior.)
Merge pull request #4156 from larsmans/optimize-dead-code

MAINT: optimize: remove dead code