fixing crawling logic