folded retweet metadata into Stats class and associated fields in indexing.
Added new delimiters and fixed a small bbug which caused the buffer used by LowerCaseEntityPreservingFilter to be overwritten if entity delimiters were used outside of an entity.
Remove an unused import and some print staements used for examples
Merge branch 'prototype-mefron' into trec2013-api
Merge branch 'jamesmc-tokenizer' of https://github.com/JamesMcMinn/twitter-tools into trec2013-api