inverse index tokens now contain frequecy and position data. We are not doing anytihng with these yet. Everything functioning normally with it in there. Some memory increase asa result.
Adds basic relevency sorting. Based on term frequency across all fields.