WIP - convert dictionary into a more search friendly data structure

A single continguous vector for all term vectors is better for a linear search
as required by nearest neighbours.