fix some cores.
NOTE: emails disabled here... need to fix.
added debug statements for email bug
if indexing diffbot url and it had
error, do NOT add a spider reply.
fix a core. do not get sections of non-html
or non-text documents. was causing EMALFORMED
sections error on diffbot json.
Merge branch 'diffbot' of github.com:gigablast/open-source-search-engine into diffbot
for json docs only give them a single
xmlnode in the Xml.cpp class. hopefully
will not get "malformed sections" error
anymore. i think that was a result of the
json having html tags in it and making
unnested html structures which the
sections class did not like.
TODO: probably do this for CT_TEXT etc.
as well.
fixed core
fixes for sudden revitilization of dead crawls.
fix core from double free