refactoring clean-up for batch ingest