spotify/luigi: Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in. 
Category: ml-ops
Language: Python
Topics: hadoop, luigi, orchestration-framework, scheduling