Make sure ingest jobs with empty directories finish
Merge pull request #734 from rcordovano/ingest_scheduler_improvements

Ingest scheduler improvements