Only invert matching direction if absolutely necessary

Only invert the direction to constrain from face_1 to face_2 when
absolutely necessary. With this, unnecessary invertions of the
transformation matrix can be avoided.

Manually verified that constraint entries (beside direction) do not change.