mapbox/carmen — pull request #76 by yhahn (2013-11-22 16:17:01+00:00)
Title: Drop use of freq index @ query time

Per https://github.com/mapbox/carmen/issues/74,

1. Switches encoding of terms to be 28/4 with 28 bits dedicated to the term hash and 4 bits free for encoding additional info. (Degenerate distance, phrase term weight).
2. Encodes IDF term weights (phrase-relative) directly into the phrase index.
3. Updates querying to use the encoded term weight rather than loading the freq index to calculate weights.

The freq index will still be kept around for doing incremental updates -- it represents the canonical freq count of each term.