mostly completed refactoring batch ingest spec tests