Only look for a User if the user_id is set