Moved versions to the subpackages.
Some clarification on tokenizer contract.
Fixed infinite loop on empty input string.
Introduced integration test.
Fixed test.