BUG: optimize: fix botched test for scalars

Any row-vector or other ndarray with shape[0] == 1 was passed through.
Merge pull request #4077 from larsmans/optimize-scalar-check

BUG: optimize: fix botched test for scalars