support crawl based ingest of data into OGP