Only try to revoke passwords if there are any to revoke