ingest job refactoring