organized into streaming and mapreduce