potatolondon/djangae — pull request #91 by benvand (2014-11-05 11:00:02+00:00)
Title: Update indexing.py

It looks like duplicates are extraneous in this index. This change will allow storage of longer values and reduce storage/ iteration on lookup if the value to be indexed contains any duplication.
Works on ContainsIndexer and IContainsIndexer index classes.

The indexer can now index a string made up of 500 of the same character in an index list 500 long too hence the <= 500:

>>> value=''.join(['a' for i in range(500)])
>>> len(value)
500
>>> length=len(value)
>>> len(list(set([value[i:j + 1] for i in xrange(length) for j in xrange(i, length)])))
500
>>> len([value[i:j + 1] for i in xrange(length) for j in xrange(i, length)])
125250