Add 'hs_' prefix to spider attributes
Remove idle_spider function, instead of processing multiple set of batches per job, the idea is now to run multiple jobs and use the closing reason to determine whether the job items are valid or not
Send the new links to the HCF as soon as they are available instead of waiting until spider_close, this way we make better use of the batch uploader