Handle invisibles at the token level to fix char width measurement

Fixes #3188