small changes to corpus ingestion