corrected some bugs in document crawler