Allows an expiration date without using a refresh token

In some cases you may want to use an expiration date for access tokens,
but no refresh tokens. This small fix allows that.