completed rev. 0 word normalization and stemming tested to compatibility, next need to work on tokenizers