Revert "Add workaround for older SciPy rejecting 0-shape matrix"

This reverts commit 6ad195bba15d1c03e95f5d294b23bea0f660c5e0.