Added a discardDuplicateTokens property

This option is false by default. This is an optimization that can be turned on that avoids indexing the same token two or more times for the same string in the same object. This cuts indexing time by 3-4 for long form texts
Added more details about optimizations