Package: botasaurus-proxy-authentication
Botasaurus Proxy AuthenticationBotasaurus Proxy Authentication provides SSL support for authenticated proxies.Proxy providers like BrightData, IPRoyal, and others typically provide authenticated proxies in the format "http://username:password@proxy-provider-domain:port". For example, "http://greyninja:awesomepassword@geo.iproyal.com:12321".However, if you use an authenticated proxy with a library like seleniumwire to scrape a Cloudflare protected website like G2.com, you will surely be blocked because you are using a non-SSL connection.To verify this, run the following code:First, install the necessary packages:python-mpipinstallselenium_wirechromedriver_autoinstallerThen, execute this Python script:fromseleniumwireimportwebdriverfromchromedriver_autoinstallerimportinstall# Define the proxyproxy_options={'proxy':{'http':'http://username:password@proxy-provider-domain:port',# TODO: Replace with your own proxy'https':'http://username:password@proxy-provider-domain:port',# TODO: Replace with your own proxy}}# Install and set up the driverdriver_path=install()driver=webdriver.Chrome(driver_path,seleniumwire_options=proxy_options)# Navigate to the desired URLdriver.get("https://ipinfo.io/")# Prompt for user inputinput("Press Enter to exit...")# Clean updriver.quit()You will definetely encounter a block by Cloudflare:However, using proxies with botasaurus_proxy_authentication prevents this issue. See the difference by running the following code:First, install the necessary packages:python-mpipinstallbotasaurusThen, execute this Python script:frombotasaurusimport*@browser(proxy="http://username:password@proxy-provider-domain:port")# TODO: Replace with your own proxydefscrape_heading_task(driver:AntiDetectDriver,data):driver.get("https://ipinfo.io/")driver.prompt()scrape_heading_task()Result:NOTE: To run the code above, you will need Node.js installed.Usage with Botasaurusfrombotasaurusimport*@browser(proxy="http://username:password@proxy-provider-domain:port")# TODO: Replace with your own proxydefvisit_ipinfo(driver:AntiDetectDriver,data):driver.get("https://ipinfo.io/")driver.prompt()visit_ipinfo()Usage with Seleniumfromseleniumimportwebdriverfromselenium.webdriver.chrome.optionsimportOptionsfromchromedriver_autoinstallerimportinstallfrombotasaurus_proxy_authenticationimportadd_proxy_options# Define the proxy settingsproxy='http://username:password@proxy-provider-domain:port'# TODO: Replace with your own proxy# Set Chrome optionschrome_options=Options()add_proxy_options(chrome_options,proxy)# Install and set up the driverdriver_path=install()driver=webdriver.Chrome(driver_path,options=chrome_options)# Navigate to the desired URLdriver.get("https://ipinfo.io/")# Prompt for user inputinput("Press Enter to exit...")# Clean updriver.quit()BotasaurusWe encourage you to learn aboutBotasaurus. The All-in-One Web Scraping Framework with Anti-Detection, Parallelization, Asynchronous, and Caching Superpowers.ThanksKudos to the Apify Team for creatingproxy-chainlibrary. The implementation of SSL-based Proxy Authentication wouldn't be possible without their groundbreaking work onproxy-chain.Love It?Star It! ⭐Become one of our amazing stargazers by giving us a star ⭐ on GitHub!It's just one click, but it means the world to me.Made with ❤️ in Bharat 🇮🇳 - Vande Mataram