dbpedia-spotlight/dbpedia-spotlight — pull request #245 by jodaiber (2013-11-19 00:57:41+00:00)
Title: [WIP] Statistical implementation performance and memory size improvements and better case handling

This pull request (still work in progress) addressess the following issues:

- Issue #196: Added support for reading lowercase counts for better modeling of case.
- Improved performance of context retrieval from the data store, getting rid of temporary hash maps that blew up the memory during heavy requests.
- Issue #205: Decreased memory requirement by quantizing counts (all counts are 2 byte instead of 4 now)

Because of #196 and #205, this pull request will break model compatability. New Spotlight models will require:

a) a quantized count store (holding a short <-> int mapping)
b) counts for the lowercase versions of surface forms.