fixes redirection error in scraping methods