begin refactoring ingest validation