improve error handling in batch ingest process; closes #525; closes #527