sanger-pathogens/crawl2 — pull request #72 by gv1 (2012-01-11 23:46:44+00:00)
Title: read the sequence directly from the gff files, i.e. not from elasticsearch

This approach should fix the memory issues that arise regarding the storing of sequence inside elasticsearch indices.

When creating a region, we store instead the path to the GFF file as a feature property. This file path can then be retrieved by the regions/sequence* services, and is at that point parsed. After the first time the sequence is parsed, it is cached in the local hazelcast cache, so the file doesn't have to be reopened. Nonetheless, the file reader is thread safe (in case two simultaneous requests for that file occur before the sequence has been chached).

This should also speed up regions/inorganism calls, because the amount of data stored as region types is vastly reduced.