Remove destroyed surfaces from the list