Include hashie, remove procrastinate.
Rewrite everything to a process structure like:

Master
|
- search index
|
- web worker 1
- web worker 2
- web worker 3

The web workers have a channel to the search index
for updating and querying.