Temporarily remove recoverable passwords