Fixed issue with cralwer missing pages.