do not send "children" docs (robots.txt roots etc)
to diffbot.