Change how it works, now crawls the site. Need to add sane limits and port the tech data back to the site document