Only keep track of visible views.

We only care about visible children since invisible and gone views have no effect on the 3D model.