refactored to support Hadoop hooks, and provides a hadoop MR test illustrating it works