piskvorky/gensim — pull request #67 by felixmiddendorf (2011-11-19 12:09:59+00:00)
Title: Added option for tf-length-normalization to models.TfidfModel

See the discussion in the mailing list: http://groups.google.com/group/gensim/browse_thread/thread/11e19beedd70470c

This implementation calculates the document length (not geometric length!) for each vector and applies the following division to each of the vector's components: `tf / document_length`.

This is done before tf is multiplied with idf.