The best we can do for limiting the `removeEmptyView` work is to check childViews to make sure `emptyView` is there before calling remove. Otherwise, we always need to remove the emptyView before calculating the delta (which is based on the number of childViews) and before attempting to position the child elements (since empty views won't necessarily have the correct interface), but we can at least make sure to only run it whenever it's actually a `childView`.