Project: datamanagement
Issue DM-2119: W16 Optimize Secondary Index - Research
Work on the secondary index (objectId --> chunkId / subChunkId mapping). This needs to be scalable to 40B entries. Since we are planning to ingest all data from DRP in <2 days, building should take <2 days. This epic involves researching applicable technologies (including experimenting with most promising ones). Deliverable: proposed technology / architecture along with measures performance at production scale (40 B entries).
Story points: 48