The idea is to limit the number of links retrieved from the HCF per every run, if the number is too large and the spider dies those requests would have to be reprocessed, if the number is too small it won't be too efficient due the overhead of the call to the HCF.
The number of links returned per batch is not constant, so we can not rely on just counting the number of batches to enforce this restriction. I am open to change the name of the setting I don't like it much either.