Move regenerated keys to secrets file