adding logging to detect errors at crawling