Only refresh token if we have one!