added a boolean failNotConnectedToInternetURLs that will add URLs to failed URLs if YES and will ignore that logic if NO