Don't keep invisible or out-of-view objects in the quad tree