registration: py3 fixes (Q_TRIU_INDICES)

The Q_TRIU_INDICES constant was initialised using the zip() builtin. On
Py3 this returns a generator and not the list itself. This doesn't
matter the first time it's used but subsequently Q_TRIU_INDICES acts
like a zero-length vector.
registration: remove non-singular hack

There was a dirty hack whereby solvetransform() would succeed even with
singular matrices. Restore existing warning and NaN behaviour.