Only remove user if not logged in