layout: Stop adding flows to the leaf set that are obviously never going

to be leaves.

60% improvement in flow tree construction time on the rainbow page.

r? @larsbergstrom