Smarter index data rotation. Per "shard".
remove metadata.json in favor of merging into rewrite.json.
Save json data in arrays.