save spots to disk as space separated
move list of bulk urls to new directory earlier. May fix Defect #2218 if there is something that is causing the bulk job to restart before this function returns
use "expand" option when calling Diffbot
Merge branch 'diffbot-dan' into diffbot-testing
when doing a query reindex on a json
child url we need to add the spider request
of the original parent url and make sure
it does not get "EDOCUNCHANGED" error.
then the possibly new json child objects
won't get indexed.