Simplify how nodes are determined.

Doesn't keep the left vs right values anymore, but the code
is a lot nicer to look at.