added tokenizer start
switched to trigram identifier
fixed directory traversal
its reading documents now