luigi (conda): Luigi handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more. It also comes with Hadoop support built in.
Keywords: hadoop|luigi|orchestration-framework|python|scheduling