pull development work into web crawling work