Project: datamanagement
Issue DM-4763: Exploration of In-memory database packages used in time critical applications
Begin evaluation of potential in-memory data storage tools - selecting memcached and redis to start. - 4 With the intent to gain familiarity with these tools, procurred introductory volume on redis and began writing prototype python code to prototype lists, hashes, and lists of hashes. Sketched out and implemented base python class with virtual save method, then wrote child classes for replicators, replicator health, replicator jobs etc. and tested this code and implemented the save methods. - 4 Installed the above code on a nebula instance that acts as a job manager, then ran real job messages through the system and simulated task assignment and completion, using redis to track jobs. - 2 Exploring how a logging and visualization harness might be included so job activity could 1) be observed in realtime, and 2) so a session could be played back as an after action review to investigate errors, bottlenecks, cold restart behavior, etc. - 2 To finish out this story, redis replication must be included in the above prototype.
Story points: 12