if indexing diffbot url and it had
error, do NOT add a spider reply.