Ensure we don't examine empty token

Fixes issue #163.
Merge pull request #166 from kortschak/issue163

Ensure we don't examine empty token