new design facilitating scraping and import

New infrastructure with all endpoints thought to be working