Add URL discovery task type
Added crawl job states
Add basic CrawlJob entity, migration and persist test
Make URL discovery task type unselectable
Remove crawl jobs before removing jobs
Complete CrawlJobService create() and hasForJob()
Start crawl job via route
Add CrawlJobService->getForJob()
Append crawl status to job status
Making breaking changes to CrawlJob entity
Add crawl job type
Create and prep crawl job
Ensure crawl job is set to correct state to allow tasks to be selected for assigment
Set timeperiod on newly-started crawl job
Add method to CrawlJobContainerService to process url discovery task results
Correctly named tests
Get processed and discovered url lists for crawl job
Process url discovery task results on task completion
Ensure task result processing is idempotent
Check account plan constraints when receiving URL discovery task results, update crawljob accordingly