Moved some ingestion logic to lib