Package: selenium-driverless
Selenium-Driverless (Non-commercial use only!)DocumentationUse Seleniumwithout chromedriverCurrently passesCloudflare,Bet365,Turnstile, and othersMultiple tabs simultaneouslyMultiple Incognito-contexts with individual proxy & cookiesAsync (asyncio) and sync (experimental) supportProxy-auth support (experimental,example code)Network-interceptionheadless supportedQuestions?Feel free to join theDriverless-CommunityonDiscord:)Also, seedev-branchfor the latest implementations.dev-installation (click to expand)pipuninstall-yselenium-driverless
pipinstallhttps://github.com/kaliiiiiiiiii/Selenium-Driverless/archive/refs/heads/dev.zipStill getting detected?Feel free to give me a feedback!You're a company and looking for another solution? Maybeundetect.iois smth for youAlso, feel free toDependenciesPython >= 3.7Google-Chromeinstalled (Chromium not tested)InstallingInstallGoogle-Chromepip install selenium-driverlessUsagewith asynciofromselenium_driverlessimportwebdriverfromselenium_driverless.types.byimportByimportasyncioasyncdefmain():options=webdriver.ChromeOptions()asyncwithwebdriver.Chrome(options=options)asdriver:awaitdriver.get('http://nowsecure.nl#relax')awaitdriver.sleep(0.5)awaitdriver.wait_for_cdp("Page.domContentEventFired",timeout=15)# wait 10s for elem to existelem=awaitdriver.find_element(By.XPATH,'/html/body/div[2]/div/main/p[2]/a',timeout=10)awaitelem.click(move_to=True)alert=awaitdriver.switch_to.alertprint(alert.text)awaitalert.accept()print(awaitdriver.title)asyncio.run(main())synchronousasyncified, might be buggyfromselenium_driverless.syncimportwebdriveroptions=webdriver.ChromeOptions()withwebdriver.Chrome(options=options)asdriver:driver.get('http://nowsecure.nl#relax')driver.sleep(0.5)driver.wait_for_cdp("Page.domContentEventFired",timeout=15)title=driver.titleurl=driver.current_urlsource=driver.page_sourceprint(title)custom debugger addressfromselenium_driverlessimportwebdriveroptions=webdriver.ChromeOptions()options.debugger_address="127.0.0.1:2005"# specify if you don't want to run remote# options.add_argument("--remote-debugging-port=2005")asyncwithwebdriver.Chrome(options=options)asdriver:awaitdriver.get('http://nowsecure.nl#relax',wait_load=True)Network-Interceptionuse CDP events (seechrome-developer-protocollfor possible events)Notes: synchronous might not work properlyExample Code (Click to expand)warning: network interception withFetch.enablemight have issues with cross-domain iframes, maximum websocket message size or Font requests.You might try using`Network.setRequestInterception(officially deprecated) or narrowing the patternimportasyncioimportbase64importsysimporttimeimporttracebackfromcdp_socket.exceptionsimportCDPErrorfromselenium_driverlessimportwebdriverasyncdefon_request(params,global_conn):url=params["request"]["url"]_params={"requestId":params['requestId']}ifparams.get('responseStatusCode')in[301,302,303,307,308]:# redirected requestreturnawaitglobal_conn.execute_cdp_cmd("Fetch.continueResponse",_params)else:try:body=awaitglobal_conn.execute_cdp_cmd("Fetch.getResponseBody",_params,timeout=1)exceptCDPErrorase:ife.code==-32000ande.message=='Can only get response body on requests captured after headers received.':print(params,"\n",file=sys.stderr)traceback.print_exc()awaitglobal_conn.execute_cdp_cmd("Fetch.continueResponse",_params)else:raiseeelse:start=time.perf_counter()body_decoded=base64.b64decode(body['body'])# modify body herebody_modified=base64.b64encode(body_decoded).decode("ascii")fulfill_params={"responseCode":200,"body":body_modified,"responseHeaders":params["responseHeaders"]}fulfill_params.update(_params)ifparams["responseStatusText"]!="":# empty string throws "Invalid http status code or phrase"fulfill_params["responsePhrase"]=params["responseStatusText"]_time=time.perf_counter()-startif_time>0.01:print(f"decoding took long:{_time}s")awaitglobal_conn.execute_cdp_cmd("Fetch.fulfillRequest",fulfill_params)print("Mocked response",url)asyncdefmain():asyncwithwebdriver.Chrome(max_ws_size=2**30)asdriver:driver.base_target.socket.on_closed.append(lambdacode,reason:print(f"chrome exited"))global_conn=driver.base_targetawaitglobal_conn.execute_cdp_cmd("Fetch.enable",cmd_args={"patterns":[{"requestStage":"Response","urlPattern":"*"}]})awaitglobal_conn.add_cdp_listener("Fetch.requestPaused",lambdadata:on_request(data,global_conn))awaitdriver.get("https://nowsecure.nl/#relax",timeout=60,wait_load=False)whileTrue:# time.sleep(10) # no. cloudflare would hangawaitasyncio.sleep(10)asyncio.run(main())Multiple tabs simultaneouslyNote: asyncio is recommended, threading only works on independent webdriver.Chrome instances.Example Code (Click to expand)fromselenium_driverless.syncimportwebdriverfromselenium_driverless.utils.utilsimportreadfromselenium_driverlessimportwebdriverimportasyncioasyncdeftarget_1_handler(target):awaittarget.get('https://abrahamjuliot.github.io/creepjs/')print(awaittarget.title)asyncdeftarget_2_handler(target):awaittarget.get("about:blank")awaittarget.execute_script(script=read("/files/js/show_mousemove.js"))awaittarget.pointer.move_to(500,500,total_time=2)asyncdefmain():options=webdriver.ChromeOptions()asyncwithwebdriver.Chrome(options=options)asdriver:target_1=awaitdriver.current_targettarget_2=awaitdriver.new_window("tab",activate=False)awaitasyncio.gather(target_1_handler(target_1),target_2_handler(target_2))awaittarget_1.focus()input("press ENTER to exit")asyncio.run(main())Unique execution contextsexecutejavascriptwithout getting detectedExample Code (Click to expand)fromselenium_driverless.syncimportwebdriverfromselenium_driverlessimportwebdriverimportasyncioasyncdefmain():options=webdriver.ChromeOptions()asyncwithwebdriver.Chrome(options=options)asdriver:awaitdriver.get('chrome://version')script="""const proxy = new Proxy(document.documentElement, {get(target, prop, receiver) {if(prop === "outerHTML"){console.log('detected access on "'+prop+'"', receiver)return "mocked value:)"}else{return Reflect.get(...arguments)}},});Object.defineProperty(document, "documentElement", {value: proxy})"""awaitdriver.execute_script(script)src=awaitdriver.execute_script("return document.documentElement.outerHTML",unique_context=True)mocked=awaitdriver.execute_script("return document.documentElement.outerHTML",unique_context=False)print(src,mocked)asyncio.run(main())Pointer Interactionsee@master/dev/show_mousemove.pyfor visualizationpointer=awaitdriver.current_pointermove_kwargs={"total_time":0.7,"accel":2,"smooth_soft":20}awaitpointer.move_to(100,500)awaitpointer.click(500,50,move_kwargs=move_kwargs,move_to=True)Iframesswitch and interact with iframesiframes=awaitdriver.find_elements(By.TAG_NAME,"iframe")awaitasyncio.sleep(0.5)iframe_document=awaitiframes[0].content_document# iframe_document.find_elements(...)use preferencesfromselenium_driverlessimportwebdriveroptions=webdriver.ChromeOptions()# recommended usageoptions.update_pref("download.prompt_for_download",False)# oroptions.prefs.update({"download":{"prompt_for_download":False}})# supportedoptions.add_experimental_option("prefs",{"download.prompt_for_download":False})Multiple Contextsdifferent cookies for each contextA context can have multiple windows and tabs withindifferent proxy for each contextopens as a window as incognitoExample Code (Click to expand)fromselenium_driverlessimportwebdriverimportasyncioasyncdefmain():options=webdriver.ChromeOptions()asyncwithwebdriver.Chrome(options=options)asdriver:context_1=driver.current_contextawaitdriver.set_auth("username","password","localhost:5000")# proxy not supported on windows due to https://bugs.chromium.org/p/chromium/issues/detail?id=1310057context_2=awaitdriver.new_context(proxy_bypass_list=["localhost"],proxy_server="http://localhost:5000")awaitcontext_1.current_target.get("https://examle.com")awaitcontext_2.get("https://examle.com")input("press ENTER to exit:)")asyncio.run(main())Custom exception handlingYou can implement custom exception handling as followingimportselenium_driverlessimportsyshandler=(lambdae:print(f'Exception in event-handler:\n{e.__class__.__module__}.{e.__class__.__name__}:{e}',file=sys.stderr))sys.modules["selenium_driverless"].EXC_HANDLER=handlersys.modules["cdp_socket"].EXC_HANDLER=handlerHelpYou found a bug? Feel free to open an issue:)
You've got other questions or proposials? feel free to join theDiriverless-CommunityonDiscordor open a discusionNote:please check the todo's below at first!Todo'sClick to expandimplementationsWebElementsimprovemid_locationcalculationaddWebElement.screenshotInputMousemousemoveclickscrolldrag&dropwriteTouchtouchmoveTouchTapscollpinch//zoomKeyBoardSendKeyssend filessupportoptions.add_extension()support prefssyncmove sync to threaded for allowing event_handlerssupport multithreading with sync versionon independent driver instanceson same driver instancecheckPython 3.12.0AuthorsCopyright and Author:Aurin Aegerter(akaSteve)LicenseShield:Unless specified differently in a single file, this work is licensed under aCreative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.DisclaimerI am not responsible what you use the code for!!! Also no warranty!AcknowledgmentsInspiration, code snippets, etc.selenium_driverless/utils/find_chrome_executablecdp-socketjsobjectpycdp/browser.py