Fix bug where we were trying to remove objects from a frozen array