temporarily hack indexer for install/build purposes; fix tokenizer to use only byte slices and not strings; add more to test file and refactor