BUG: optimize: fix botched test for scalars

Any row-vector or other ndarray with shape[0] == 1 was passed through.
This is still ugly.
What's the problem with the context manager version I suggested before?