Remove the offline keys from the server