make HS_ENDPOINT optional

There is default endpoint in python-hubstorage package.
small docs fix
Merge pull request #20 from kmike/hs-optional-endpoint

make HS_ENDPOINT optional. LGTM
Merge pull request #21 from kmike/docs-fix-1

small docs fix. OK
Add 'hs_' prefix to spider attributes
Remove idle_spider function, instead of processing multiple set of batches per job, the idea is now to run multiple jobs and use the closing reason to determine whether the job items are valid or not
Send the new links to the HCF as soon as they are available instead of waiting until spider_close, this way we make better use of the batch uploader
Cleanup tests
Change default slot assignation function to use multiple slots, add HS_NUMBER_OF_SLOTS parameter
fixed typo
Merge pull request #22 from andresp99999/hcf-middleware

Hcf middleware