layout: Stop adding flows to the leaf set that are obviously never going
to be leaves.

60% improvement in flow tree construction time on the rainbow page.