use reconnecting socket.  page through queries.  queue adding results to index to avoid race condition.