Remove vector assertions

These were initially added to get tracebacks for invalid vector errors, but it
didn't work and tracebacks have since been properly fixed in the core.