Compute frustum culling only when needed