piskvorky/gensim — pull request #93 by strongh (2012-08-11 05:54:54+00:00)
Title: HashDictionary 

This addresses issue #76 - I couldn't help taking a pass at it! (apologies to @raoshivani)

The `HashDictionary` is based on `Dictionary` and is very similar. The tests are also based on the tests for `Dictionary`. There's a debug option which will keep track of collisions in the hash's range (as sets).

The class `gensim.corpora.RestrictedHash` contains most the logic for imitating the `token2id` dict. The hash function is exposed as a construction argument. The default hash function is `zlib.adler32`, for no particularly good reason. Let me know if you have any feedback!