Avoid converting to and from quaternions

Fixes bug #678