is there any way to optimize this to not recurse portions of the tree that need not be considered? For example portions of the tree that have not changed since the last shared face pass?

If we keep a timestamp for the last shared face culling pass, than we should be able to identify branches of the tree that have not changed since that pass was completed.