Don't attempt to destroy if already destroyed