fix getting headers from crawled pages