Don't try to remove members from an empty structure