Project: usergrid
Issue USERGRID-752: Update Usergrid 1.0 search to use concurrent search per shard
Currently we merge all shards together with every page seek in usergrid 1.0. This means we select 20k columns, and discard 19k with each page selection. Instead, I propose that we create an AST per shard, and execute the iterator on these shards concurrently. We then merge on the last step, greatly reducing the amount of network I/O and discarded data on queries of large data sets. Proposed Changes: 1 AST instance per shard, evaluated concurrently via some worker pool. Change final gather of concurrent iterators. Change cursor generation to create the cursor from the merged trees.
Story points: 3