Fix bug with empty string tokens being emitted by filterchains.

Allowing empty token generation caused a subtle corruption bug in the
vocabulary_map, which made the index fail to function properly.