We will be relying on the logs completely to do our analysis. We still have some Hadoop ETL work to be done to make this a reality, but this is the plan. Note that the logger is pluggable and we can decide to replace it with something that logs to a DB.

The reason we use Redis for assignments is because we need quick lookups. I am considering renaming "subject storage" to "assignment cache" for that reason.