Correct color equality and refactor the <=> code a bit

* fixes #692