add support for running the entire pipeline (upload, process, add to hive)