Package: nodriver
NODRIVERThis package provides next level webscraping and browser automation
using a relatively simple interface.This is the official successor of theUndetected-Chromedriverpython package.No more webdriver, no more seleniumDirect communication provides even better resistance against web applicatinon firewalls (WAF’s), while
performance gets a massive boost.
This module is, contrary to undetected-chromedriver, fully asynchronous.What makes this package different from other known packages,
is the optimization to stay undetected for most anti-bot solutions.Another focus point is usability and quick prototyping, so expect a lot to work-as is-,
with most method parameters havingbest practicedefaults.
Using 1 or 2 lines, this is up and running, providing best practice config
by default.While usability and convenience is important. It’s also easy
to fully customizable everything using the entire array ofCDPdomains, methods and events available.Some featuresA blazing fast undetected chrome (-ish) automation libraryNo chromedriver binary or Selenium dependencyThis equals bizarre performance increase and less detections!Up and running in 1 line of code*uses fresh profile on each run, cleans up on exitsave and load cookies to file to not repeat tedious login stepssmart element lookup, by selector or text, including iframe content.
this could also be used as wait condition for a element to appear, since it will retry
for the duration of  until found.
single element lookup by text using tab.find(), accepts a  best_match flag, which will not
naively return the first match, but will match candidates by closest matching text length.descriptive __repr__ for elements, which represent the element as htmlutility function to convert a running undetected_chromedriver.Chrome instance
to a nodriver.Browser instance and contintue from therepacked with helpers and utility methods for most used and important operationsInstallationSince it’s a part of undetected-chromedriver, installation goes viapip install undetected-chromedriverIn case you don’t want undetected-chromedriver, this package can be installed
usingpip install nodriverusage exampleThe aim of this project (just like undetected-chromedriver, somewhere long ago)
is to keep it short and simple, so you can quickly open an editor or interactive session,
type or paste a few lines and off you go.importasyncioimportnodriverasucasyncdefmain():browser=awaituc.start()page=awaitbrowser.get('https://www.nowsecure.nl')awaitpage.save_screenshot()awaitpage.get_content()awaitpage.scroll_down(150)elems=awaitpage.select_all('*[src]')foreleminelems:awaitelem.flash()page2=awaitbrowser.get('https://twitter.com',new_tab=True)page3=awaitbrowser.get('https://github.com/ultrafunkamsterdam/nodriver',new_window=True)forpin(page,page2,page3):awaitp.bring_to_front()awaitp.scroll_down(200)awaitp# wait for events to be processedawaitp.reload()ifp!=page3:awaitp.close()if__name__=='__main__':# since asyncio.run never worked (for me)uc.loop().run_until_complete(main())A more concrete example, which can be found in the ./example/ folder,
shows a script to create a twitter accountimportrandomimportstringimportlogginglogging.basicConfig(level=30)importnodriverasucmonths=["january","february","march","april","may","june","july","august","september","october","november","december",]asyncdefmain():driver=awaituc.start()tab=awaitdriver.get("https://twitter.com")# wait for text to appear instead of a static number of seconds to wait# this does not always work as expected, due to speed.print('finding the "create account" button')create_account=awaittab.find("create account",best_match=True)print('"create account" => click')awaitcreate_account.click()print("finding the email input field")email=awaittab.select("input[type=email]")# sometimes, email field is not shown, because phone is being asked instead# when this occurs, find the small text which says "use email instead"ifnotemail:use_mail_instead=awaittab.find("use email instead")# and click itawaituse_mail_instead.click()# now find the email field againemail=awaittab.select("input[type=email]")randstr=lambdak:"".join(random.choices(string.ascii_letters,k=k))# send keys to email fieldprint('filling in the "email" input field')awaitemail.send_keys("".join([randstr(8),"@",randstr(8),".com"]))# find the name input fieldprint("finding the name input field")name=awaittab.select("input[type=text]")# again, send random textprint('filling in the "name" input field')awaitname.send_keys(randstr(8))# since there are 3 select fields on the tab, we can use unpacking# to assign each fieldprint('finding the "month" , "day" and "year" fields in 1 go')sel_month,sel_day,sel_year=awaittab.select_all("select")# await sel_month.focus()print('filling in the "month" input field')awaitsel_month.send_keys(months[random.randint(0,11)].title())# await sel_day.focus()# i don't want to bother with month-lengths and leap yearsprint('filling in the "day" input field')awaitsel_day.send_keys(str(random.randint(0,28)))# await sel_year.focus()# i don't want to bother with age restrictionsprint('filling in the "year" input field')awaitsel_year.send_keys(str(random.randint(1980,2005)))awaittab# let's handle the cookie nag as wellcookie_bar_accept=awaittab.find("accept all",best_match=True)ifcookie_bar_accept:awaitcookie_bar_accept.click()awaittab.sleep(1)next_btn=awaittab.find(text="next",best_match=True)# for btn in reversed(next_btns):awaitnext_btn.mouse_click()print("sleeping 2 seconds")awaittab.sleep(2)# visually see what part we're actually inprint('finding "next" button')next_btn=awaittab.find(text="next",best_match=True)print('clicking "next" button')awaitnext_btn.mouse_click()# just wait for some button, before we continueawaittab.select("[role=button]")print('finding "sign up"  button')sign_up_btn=awaittab.find("Sign up",best_match=True)# we need the second oneprint('clicking "sign up"  button')awaitsign_up_btn.click()print('the rest of the "implementation" is out of scope')# further implementation outside of scopeawaittab.sleep(10)driver.stop()# verification code per mailif__name__=="__main__":# since asyncio.run never worked (for me)# i useuc.loop().run_until_complete(main())