Add Tokeniser.add_iterator.
Use where clauses.
Split lib and tokeniser.
Split builders from lib.
Implement C-style tokeniser.