Clean up keys on runtime exit