Clean up the OAuth folder