implemented retry behavior for network errors, broke crawl logic into functions, handling timeouts, errors, and resources loaded to determine when page is ready to be evaluated