Package: undetected-chromedriver-arthur
undetected_chromedriverhttps://github.com/ultrafunkamsterdam/undetected-chromedriverOptimized Selenium Chromedriver patch which does not trigger anti-bot services like Distill Network / Imperva / DataDome / Botprotect.io
Automatically downloads the driver binary and patches it.Tested until current chrome beta versionsWorks also on Brave Browser and many other Chromium based browsers, some tweakingPython 3.6++**Message for allI will be putting limits on the issue tracker. It has beeen abused too long.any good news?Yes, i've openedUndetected-Discussionswhich i think will help us better in the long run.3.4.5What a week. Had the recent advancedments in Automation-Detection algorithms pwned (so i thought) with 3.4.0, but apparently, for some OS-es this caused an error when    interacting with elements. Had to revert back using a different method, fix bugs, and now eventually was still able to stick to the initial idea (+ fixing bugs)Update to chrome 110 caused another surprise, this time for HEADLESS users.although headless is unsupported officially, i did patch it!happy to announce IT IS NOW UNDETECTED AS WELL (but still unsupported ;))special thanks here to@mdmintzand@abdulzain6also special thanks to@sebdelsolfor his help troughout the issues section completely voluntarily, you must be crazy :)3.4.0Big update! be careful as it -potentially- could break your code.rewritten the anti-detection mechanism instead of removing and renaming variables, we just keep them, but prevent them from being injected in the first place. This will keep us safe from detection at least for the near future.rewritten the file naming, to prevent ending up with 1000 of {randomstring}_chromedriver.exe 's instead it is just called undetected_chromedriver.execleanup removed compat,v2 files and tests folder3.2.0added an example containing some typical webdriver code, answers to commonly asked questions, pitfalls + showcasing some tricks to ditch
the need for multithreading.>>>> example code here <<<<added WebElement.click_safe() method, which you can try in case you get detected after clicking a link. This is not guaranteed t o work.added WebElement.children(self, tag=None, recursive=False)
to easily get/find child nodes. example:body = driver.find_element('tag name', 'body')

# get the 6th child (any tag) of body, and grab all img's within (recursive). 
images = body.children()[6].children('img', True)
srcs = list(map(lambda _:_.attrs.get('src'), images))added example.py where i can point people at when asking silly questions
(no, its actually quite cool, everyone should see it)added support for lambda platformadded support for x86_32added support for systems reporting as linux2some refactoring3.1.6still passing stronguse_subprocess now defaults to True. too many people don't understand multiprocessing andname== 'main, and after testing, it
seems not to make a difference anymore in chrome 104+added no_sandbox, which defaults to True, and this without the annoying "you are using unsecure command line ..." bar.updateDocker image. you can now vnc or rdp into your container to see the
actual browser windowof course, "regular" mode works as well3.1.0this versionmightbreak your code, test before update!added new anti-detection logic!v2 has become the main module, so no need for references to v2 anymore. this mean you can now simply use:importundetected_chromedriverasucdriver=uc.Chrome()driver.get('https://nowsecure.nl')for backwards compatibility, v2 is not removed, but aliassed to the main module.Fixed "welcome screen" nagging on non-windows OS-es. For those nagfetishists who ❤ welcome screens and feeding google with even more data,
use Chrome(suppress_welcome=False).replacedexecutable_pathin constructor in favor ofbrowser_executable_pathwhich should not be used unless you are the edge case (yep, you are) who can't add your custom chrome installation folder to your PATH
environment variable, or have an army of different browsers/versions and automatic lookup returns the wrong browser"v1" (?) moved to _compat for now.fixed dependency versionsChromeOptions custom handling removed, so it is compatible withwebdriver.chromium.options.ChromiumOptions.removed Chrome.get() fu and restored back to "almost" original:nowithstatements needed anymore, although it will still work for the sake of backward-compatibility.no sleeps, stop-start-sessions, delays, or async cdp black magic!this will solve a lot of other "issues" as well.test success to date: 100%just to mention it another time, since some people have hard time reading:headless is still WIP. Raising issues is needless3.0.4 changeschange process creation behavior to be fully detachedchanged .get(url) method to always use the contextmanagerchanged .get(url) method to use cdp under the hood.... thewithstatement is not necessary anymore ..todo: work towards asyncification and selenium 4words of wisdom:Whenever you encounter the dauntedfrom session not created: This version of ChromeDriver only supports Chrome version 96 # or what ever versionthe solution is simple:importundetected_chromedriverasucdriver=uc.Chrome(version_main=95)July 2021: Currently busy implementing selenium 4 for undetected-chromedrivernewsflash:https://github.com/ultrafunkamsterdam/undetected-chromedriver/pull/255Installationpip install undetected-chromedriverUsageTo prevent unnecessary hair-pulling and issue-raising, please mind theimportant note at the end of this document.easyLiterally, this is all you have to do. Settings are included and your browser executable is found automagically. This is also the snippet i
recommend using in case you experience an issue.importundetected_chromedriverasucdriver=uc.Chrome()driver.get('https://nowsecure.nl')# my own test test site with max anti-bot protectionmore advanced way, including setting profie folderLiterally, this is all you have to do. If a specified folder does not exist, a NEW profile is created. Data dirs which are specified like
this will not be autoremoved on exit.importundetected_chromedriverasucoptions=uc.ChromeOptions()# setting profileoptions.user_data_dir="c:\\temp\\profile"# use specific (older) versiondriver=uc.Chrome(options=options,version_main=94)# version_main allows to specify your chrome version instead of following chrome global versiondriver.get('https://nowsecure.nl')# my own test test site with max anti-bot protectionexpert mode, including Devtool/Wire eventsLiterally, this is all you have to do. You can now listen and subscribe to the low level devtools-protocol. I just recently found out that
is also on planning for future release of the official chromedriver. However i implemented my own for now. Since i needed it myself for
investigation.importundetected_chromedriverasucfrompprintimportpformatdriver=uc.Chrome(enable_cdp_events=True)defmylousyprintfunction(eventdata):print(pformat(eventdata))# set the callback to Network.dataReceived to print (yeah not much original)driver.add_cdp_listener("Network.dataReceived",mylousyprintfunction)driver.get('https://nowsecure.nl')# known url using cloudflare's "under attack mode"defmylousyprintfunction(message):print(pformat(message))# for more inspiration checkout the link below# https://chromedevtools.github.io/devtools-protocol/1-3/Network/# and of couse 2 lousy examplesdriver.add_cdp_listener('Network.requestWillBeSent',mylousyprintfunction)driver.add_cdp_listener('Network.dataReceived',mylousyprintfunction)# hint: a wildcard captures all events!# driver.add_cdp_listener('*', mylousyprintfunction)# now all these events will be printed in my consoledriver.get('https://nowsecure.nl'){'method':'Network.requestWillBeSent','params':{'documentURL':'https://nowsecure.nl/','frameId':'F42BAE4BDD4E428EE2503CB5A7B4F700','hasUserGesture':False,'initiator':{'type':'other'},'loaderId':'449906A5C736D819123288133F2797E6','request':{'headers':{'Upgrade-Insecure-Requests':'1','User-Agent':'Mozilla/5.0 (Windows NT ''10.0; Win64; x64) ''AppleWebKit/537.36 (KHTML, ''like Gecko) ''Chrome/90.0.4430.212 ''Safari/537.36','sec-ch-ua':'" Not A;Brand";v="99", ''"Chromium";v="90", "Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0'},'initialPriority':'VeryHigh','method':'GET','mixedContentType':'none','referrerPolicy':'strict-origin-when-cross-origin','url':'https://nowsecure.nl/'},'requestId':'449906A5C736D819123288133F2797E6','timestamp':190010.996717,'type':'Document','wallTime':1621835932.112026}}{'method':'Network.requestWillBeSentExtraInfo','params':{'associatedCookies':[],'headers':{':authority':'nowsecure.nl',':method':'GET',':path':'/',':scheme':'https','accept':'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','accept-encoding':'gzip, deflate, br','accept-language':'en-US,en;q=0.9','sec-ch-ua':'" Not A;Brand";v="99", ''"Chromium";v="90", "Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0','sec-fetch-dest':'document','sec-fetch-mode':'navigate','sec-fetch-site':'none','sec-fetch-user':'?1','upgrade-insecure-requests':'1','user-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; ''x64) AppleWebKit/537.36 (KHTML, like ''Gecko) Chrome/90.0.4430.212 ''Safari/537.36'},'requestId':'449906A5C736D819123288133F2797E6'}}{'method':'Network.responseReceivedExtraInfo','params':{'blockedCookies':[],'headers':{'alt-svc':'h3-27=":443"; ma=86400, h3-28=":443"; ''ma=86400, h3-29=":443"; ma=86400','cache-control':'private, max-age=0, no-store, ''no-cache, must-revalidate, ''post-check=0, pre-check=0','cf-ray':'65444b779ae6546f-LHR','cf-request-id':'0a3e8d7eba0000546ffd3fa000000001','content-type':'text/html; charset=UTF-8','date':'Mon, 24 May 2021 05:58:53 GMT','expect-ct':'max-age=604800, ''report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"','expires':'Thu, 01 Jan 1970 00:00:01 GMT','nel':'{"report_to":"cf-nel","max_age":604800}','permissions-policy':'accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()','report-to':'{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report?s=CAfobYlmWImQ90e%2B4BFBhpPYL%2FyGyBvkcWAj%2B%2FVOLoEq0NVrD5jU9m5pi%2BKI%2BOAnINLPXOCoX2psLphA5Z38aZzWNr3eW%2BDTIK%2FQidc%3D"}],"group":"cf-nel","max_age":604800}','server':'cloudflare','vary':'Accept-Encoding','x-frame-options':'SAMEORIGIN'},'requestId':'449906A5C736D819123288133F2797E6','resourceIPAddressSpace':'Public'}}{'method':'Network.responseReceived','params':{'frameId':'F42BAE4BDD4E428EE2503CB5A7B4F700','loaderId':'449906A5C736D819123288133F2797E6','requestId':'449906A5C736D819123288133F2797E6','response':{'connectionId':158,'connectionReused':False,'encodedDataLength':851,'fromDiskCache':False,'fromPrefetchCache':False,'fromServiceWorker':False,'headers':{'alt-svc':'h3-27=":443"; ma=86400, ''h3-28=":443"; ma=86400, ''h3-29=":443"; ma=86400','cache-control':'private, max-age=0, ''no-store, no-cache, ''must-revalidate, ''post-check=0, ''pre-check=0','cf-ray':'65444b779ae6546f-LHR','cf-request-id':'0a3e8d7eba0000546ffd3fa000000001','content-type':'text/html; charset=UTF-8','date':'Mon, 24 May 2021 05:58:53 GMT','expect-ct':'max-age=604800, ''report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"','expires':'Thu, 01 Jan 1970 00:00:01 GMT','nel':'{"report_to":"cf-nel","max_age":604800}','permissions-policy':'accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()','report-to':'{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report?s=CAfobYlmWImQ90e%2B4BFBhpPYL%2FyGyBvkcWAj%2B%2FVOLoEq0NVrD5jU9m5pi%2BKI%2BOAnINLPXOCoX2psLphA5Z38aZzWNr3eW%2BDTIK%2FQidc%3D"}],"group":"cf-nel","max_age":604800}','server':'cloudflare','vary':'Accept-Encoding','x-frame-options':'SAMEORIGIN'},'mimeType':'text/html','protocol':'h2','remoteIPAddress':'104.21.5.197','remotePort':443,'requestHeaders':{':authority':'nowsecure.nl',':method':'GET',':path':'/',':scheme':'https','accept':'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','accept-encoding':'gzip, deflate, ''br','accept-language':'en-US,en;q=0.9','sec-ch-ua':'" Not ''A;Brand";v="99", ''"Chromium";v="90", ''"Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0','sec-fetch-dest':'document','sec-fetch-mode':'navigate','sec-fetch-site':'none','sec-fetch-user':'?1','upgrade-insecure-requests':'1','user-agent':'Mozilla/5.0 ''(Windows NT 10.0; ''Win64; x64) ''AppleWebKit/537.36 ''(KHTML, like Gecko) ''Chrome/90.0.4430.212 ''Safari/537.36'},'responseTime':1621835932177.923,'securityDetails':{'certificateId':0,'certificateTransparencyCompliance':'compliant','cipher':'AES_128_GCM','issuer':'Cloudflare Inc ECC ''CA-3','keyExchange':'','keyExchangeGroup':'X25519','protocol':'TLS 1.3','sanList':['sni.cloudflaressl.com','*.nowsecure.nl','nowsecure.nl'],'signedCertificateTimestampList':[{'hashAlgorithm':'SHA-256','logDescription':'Google '"'Argon2021' "'log','logId':'F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3','origin':'Embedded ''in ''certificate','signatureAlgorithm':'ECDSA','signatureData':'30450221008A25458182A6E7F608FE1492086762A367381E94137952FFD621BA2E60F7E2F702203BCDEBCE1C544DECF0A113DE12B33E299319E6240426F38F08DFC04EF2E42825','status':'Verified','timestamp':1598706372839.0},{'hashAlgorithm':'SHA-256','logDescription':'DigiCert ''Yeti2021 ''Log','logId':'5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA','origin':'Embedded ''in ''certificate','signatureAlgorithm':'ECDSA','signatureData':'3046022100A95A49C7435DBFC73406AC409062C27269E6E69F443A2213F3A085E3BCBD234A022100DEA878296F8A1DB43546DC1865A4C5AD2B90664A243AE0A3A6D4925802EE68A8','status':'Verified','timestamp':1598706372823.0}],'subjectName':'sni.cloudflaressl.com','validFrom':1598659200,'validTo':1630238400},'securityState':'secure','status':503,'statusText':'','timing':{'connectEnd':40.414,'connectStart':0,'dnsEnd':0,'dnsStart':0,'proxyEnd':-1,'proxyStart':-1,'pushEnd':0,'pushStart':0,'receiveHeadersEnd':60.361,'requestTime':190011.002239,'sendEnd':41.348,'sendStart':41.19,'sslEnd':40.405,'sslStart':10.853,'workerFetchStart':-1,'workerReady':-1,'workerRespondWithSettled':-1,'workerStart':-1},'url':'https://nowsecure.nl/'},'timestamp':190011.06449,'type':'Document'}}{'method':'Page.frameStartedLoading','params':{'frameId':'F42BAE4BDD4E428EE2503CB5A7B4F700'}}{'method':'Page.frameNavigated','params':{'frame':{'adFrameType':'none','crossOriginIsolatedContextType':'NotIsolated','domainAndRegistry':'nowsecure.nl','gatedAPIFeatures':['SharedArrayBuffers','SharedArrayBuffersTransferAllowed'],'id':'F42BAE4BDD4E428EE2503CB5A7B4F700','loaderId':'449906A5C736D819123288133F2797E6','mimeType':'text/html','secureContextType':'Secure','securityOrigin':'https://nowsecure.nl','url':'https://nowsecure.nl/'}}}{'method':'Network.dataReceived','params':{'dataLength':9835,'encodedDataLength':0,'requestId':'449906A5C736D819123288133F2797E6','timestamp':190011.093343}}{'method':'Network.loadingFinished','params':{'encodedDataLength':10713,'requestId':'449906A5C736D819123288133F2797E6','shouldReportCorbBlocking':False,'timestamp':190011.064011}}{'method':'Network.requestWillBeSent','params':{'documentURL':'https://nowsecure.nl/','frameId':'F42BAE4BDD4E428EE2503CB5A7B4F700','hasUserGesture':False,'initiator':{'stack':{'callFrames':[{'columnNumber':51,'functionName':'','lineNumber':114,'scriptId':'8','url':'https://nowsecure.nl/'},{'columnNumber':9,'functionName':'','lineNumber':115,'scriptId':'8','url':'https://nowsecure.nl/'}]},'type':'script'},'loaderId':'449906A5C736D819123288133F2797E6','request':{'headers':{'Referer':'https://nowsecure.nl/','User-Agent':'Mozilla/5.0 (Windows NT ''10.0; Win64; x64) ''AppleWebKit/537.36 (KHTML, ''like Gecko) ''Chrome/90.0.4430.212 ''Safari/537.36','sec-ch-ua':'" Not A;Brand";v="99", ''"Chromium";v="90", "Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0'},'initialPriority':'Low','method':'GET','mixedContentType':'none','referrerPolicy':'strict-origin-when-cross-origin','url':'https://nowsecure.nl/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=65444b779ae6546f'},'requestId':'17180.2','timestamp':190011.106133,'type':'Script','wallTime':1621835932.221325}}{'method':'Network.requestWillBeSent','params':{'documentURL':'https://nowsecure.nl/','frameId':'F42BAE4BDD4E428EE2503CB5A7B4F700','hasUserGesture':False,'initiator':{'columnNumber':13,'lineNumber':117,'type':'parser','url':'https://nowsecure.nl/'},'loaderId':'449906A5C736D819123288133F2797E6','request':{'headers':{'Referer':'https://nowsecure.nl/','User-Agent':'Mozilla/5.0 (Windows NT ''10.0; Win64; x64) ''AppleWebKit/537.36 (KHTML, ''like Gecko) ''Chrome/90.0.4430.212 ''Safari/537.36','sec-ch-ua':'" Not A;Brand";v="99", ''"Chromium";v="90", "Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0'},'initialPriority':'Low','method':'GET','mixedContentType':'none','referrerPolicy':'strict-origin-when-cross-origin','url':'https://nowsecure.nl/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=65444b779ae6546f'},'requestId':'17180.3','timestamp':190011.106911,'type':'Image','wallTime':1621835932.222102}}{'method':'Network.requestWillBeSent','params':{'documentURL':'https://nowsecure.nl/','frameId':'F42BAE4BDD4E428EE2503CB5A7B4F700','hasUserGesture':False,'initiator':{'type':'parser','url':'https://nowsecure.nl/'},'loaderId':'449906A5C736D819123288133F2797E6','request':{'headers':{'Referer':'https://nowsecure.nl/','User-Agent':'Mozilla/5.0 (Windows NT ''10.0; Win64; x64) ''AppleWebKit/537.36 (KHTML, ''like Gecko) ''Chrome/90.0.4430.212 ''Safari/537.36','sec-ch-ua':'" Not A;Brand";v="99", ''"Chromium";v="90", "Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0'},'initialPriority':'Low','method':'GET','mixedContentType':'none','referrerPolicy':'strict-origin-when-cross-origin','url':'https://nowsecure.nl/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=65444b779ae6546f'},'requestId':'17180.4','timestamp':190011.109527,'type':'Image','wallTime':1621835932.224719}}{'method':'Page.domContentEventFired','params':{'timestamp':190011.110345}}{'method':'Network.requestWillBeSentExtraInfo','params':{'associatedCookies':[],'clientSecurityState':{'initiatorIPAddressSpace':'Public','initiatorIsSecureContext':True,'privateNetworkRequestPolicy':'WarnFromInsecureToMorePrivate'},'headers':{':authority':'nowsecure.nl',':method':'GET',':path':'/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=65444b779ae6546f',':scheme':'https','accept':'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8','accept-encoding':'gzip, deflate, br','accept-language':'en-US,en;q=0.9','referer':'https://nowsecure.nl/','sec-ch-ua':'" Not A;Brand";v="99", ''"Chromium";v="90", "Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0','sec-fetch-dest':'image','sec-fetch-mode':'no-cors','sec-fetch-site':'same-origin','user-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; ''x64) AppleWebKit/537.36 (KHTML, like ''Gecko) Chrome/90.0.4430.212 ''Safari/537.36'},'requestId':'17180.3'}}{'method':'Network.requestWillBeSentExtraInfo','params':{'associatedCookies':[],'clientSecurityState':{'initiatorIPAddressSpace':'Public','initiatorIsSecureContext':True,'privateNetworkRequestPolicy':'WarnFromInsecureToMorePrivate'},'headers':{':authority':'nowsecure.nl',':method':'GET',':path':'/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=65444b779ae6546f',':scheme':'https','accept':'*/*','accept-encoding':'gzip, deflate, br','accept-language':'en-US,en;q=0.9','referer':'https://nowsecure.nl/','sec-ch-ua':'" Not A;Brand";v="99", ''"Chromium";v="90", "Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0','sec-fetch-dest':'script','sec-fetch-mode':'no-cors','sec-fetch-site':'same-origin','user-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; ''x64) AppleWebKit/537.36 (KHTML, like ''Gecko) Chrome/90.0.4430.212 ''Safari/537.36'},'requestId':'17180.2'}}{'method':'Network.requestWillBeSentExtraInfo','params':{'associatedCookies':[],'clientSecurityState':{'initiatorIPAddressSpace':'Public','initiatorIsSecureContext':True,'privateNetworkRequestPolicy':'WarnFromInsecureToMorePrivate'},'headers':{':authority':'nowsecure.nl',':method':'GET',':path':'/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=65444b779ae6546f',':scheme':'https','accept':'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8','accept-encoding':'gzip, deflate, br','accept-language':'en-US,en;q=0.9','referer':'https://nowsecure.nl/','sec-ch-ua':'" Not A;Brand";v="99", ''"Chromium";v="90", "Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0','sec-fetch-dest':'image','sec-fetch-mode':'no-cors','sec-fetch-site':'same-origin','user-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; ''x64) AppleWebKit/537.36 (KHTML, like ''Gecko) Chrome/90.0.4430.212 ''Safari/537.36'},'requestId':'17180.4'}}{'method':'Network.responseReceivedExtraInfo','params':{'blockedCookies':[],'headers':{'accept-ranges':'bytes','cache-control':'max-age=7200\npublic','cf-ray':'65444b781d1de604-LHR','content-length':'42','content-type':'image/gif','date':'Mon, 24 May 2021 05:58:53 GMT','etag':'"60a4d856-2a"','expires':'Mon, 24 May 2021 07:58:53 GMT','last-modified':'Wed, 19 May 2021 09:20:22 GMT','server':'cloudflare','vary':'Accept-Encoding','x-content-type-options':'nosniff','x-frame-options':'DENY'},'requestId':'17180.3','resourceIPAddressSpace':'Public'}}{'method':'Network.responseReceivedExtraInfo','params':{'blockedCookies':[],'headers':{'accept-ranges':'bytes','cache-control':'max-age=7200\npublic','cf-ray':'65444b781d1fe604-LHR','content-length':'42','content-type':'image/gif','date':'Mon, 24 May 2021 05:58:53 GMT','etag':'"60a4d856-2a"','expires':'Mon, 24 May 2021 07:58:53 GMT','last-modified':'Wed, 19 May 2021 09:20:22 GMT','server':'cloudflare','vary':'Accept-Encoding','x-content-type-options':'nosniff','x-frame-options':'DENY'},'requestId':'17180.4','resourceIPAddressSpace':'Public'}}{'method':'Network.resourceChangedPriority','params':{'newPriority':'High','requestId':'17180.4','timestamp':190011.171057}}{'method':'Network.responseReceived','params':{'frameId':'F42BAE4BDD4E428EE2503CB5A7B4F700','loaderId':'449906A5C736D819123288133F2797E6','requestId':'17180.3','response':{'connectionId':0,'connectionReused':False,'encodedDataLength':214,'fromDiskCache':False,'fromPrefetchCache':False,'fromServiceWorker':False,'headers':{'accept-ranges':'bytes','cache-control':'max-age=7200\npublic','cf-ray':'65444b781d1de604-LHR','content-length':'42','content-type':'image/gif','date':'Mon, 24 May 2021 05:58:53 GMT','etag':'"60a4d856-2a"','expires':'Mon, 24 May 2021 07:58:53 GMT','last-modified':'Wed, 19 May 2021 ''09:20:22 GMT','server':'cloudflare','vary':'Accept-Encoding','x-content-type-options':'nosniff','x-frame-options':'DENY'},'mimeType':'image/gif','protocol':'h3-29','remoteIPAddress':'104.21.5.197','remotePort':443,'requestHeaders':{':authority':'nowsecure.nl',':method':'GET',':path':'/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=65444b779ae6546f',':scheme':'https','accept':'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8','accept-encoding':'gzip, deflate, ''br','accept-language':'en-US,en;q=0.9','referer':'https://nowsecure.nl/','sec-ch-ua':'" Not ''A;Brand";v="99", ''"Chromium";v="90", ''"Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0','sec-fetch-dest':'image','sec-fetch-mode':'no-cors','sec-fetch-site':'same-origin','user-agent':'Mozilla/5.0 ''(Windows NT 10.0; ''Win64; x64) ''AppleWebKit/537.36 ''(KHTML, like Gecko) ''Chrome/90.0.4430.212 ''Safari/537.36'},'responseTime':1621835932265.169,'securityDetails':{'certificateId':0,'certificateTransparencyCompliance':'compliant','cipher':'AES_128_GCM','issuer':'Cloudflare Inc ECC ''CA-3','keyExchange':'','keyExchangeGroup':'X25519','protocol':'QUIC','sanList':['sni.cloudflaressl.com','*.nowsecure.nl','nowsecure.nl'],'signedCertificateTimestampList':[{'hashAlgorithm':'SHA-256','logDescription':'Google '"'Argon2021' "'log','logId':'F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3','origin':'Embedded ''in ''certificate','signatureAlgorithm':'ECDSA','signatureData':'30450221008A25458182A6E7F608FE1492086762A367381E94137952FFD621BA2E60F7E2F702203BCDEBCE1C544DECF0A113DE12B33E299319E6240426F38F08DFC04EF2E42825','status':'Verified','timestamp':1598706372839.0},{'hashAlgorithm':'SHA-256','logDescription':'DigiCert ''Yeti2021 ''Log','logId':'5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA','origin':'Embedded ''in ''certificate','signatureAlgorithm':'ECDSA','signatureData':'3046022100A95A49C7435DBFC73406AC409062C27269E6E69F443A2213F3A085E3BCBD234A022100DEA878296F8A1DB43546DC1865A4C5AD2B90664A243AE0A3A6D4925802EE68A8','status':'Verified','timestamp':1598706372823.0}],'subjectName':'sni.cloudflaressl.com','validFrom':1598659200,'validTo':1630238400},'securityState':'secure','status':200,'statusText':'','timing':{'connectEnd':26.087,'connectStart':0,'dnsEnd':0,'dnsStart':0,'proxyEnd':-1,'proxyStart':-1,'pushEnd':0,'pushStart':0,'receiveHeadersEnd':40.709,'requestTime':190011.109386,'sendEnd':26.346,'sendStart':26.182,'sslEnd':26.087,'sslStart':0,'workerFetchStart':-1,'workerReady':-1,'workerRespondWithSettled':-1,'workerStart':-1},'url':'https://nowsecure.nl/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=65444b779ae6546f'},'timestamp':190011.174536,'type':'Image'}}{'method':'Network.dataReceived','params':{'dataLength':42,'encodedDataLength':0,'requestId':'17180.3','timestamp':190011.174737}}{'method':'Network.dataReceived','params':{'dataLength':0,'encodedDataLength':44,'requestId':'17180.3','timestamp':190011.17524}}{'method':'Network.loadingFinished','params':{'encodedDataLength':258,'requestId':'17180.3','shouldReportCorbBlocking':False,'timestamp':190011.152073}}{'method':'Network.responseReceived','params':{'frameId':'F42BAE4BDD4E428EE2503CB5A7B4F700','loaderId':'449906A5C736D819123288133F2797E6','requestId':'17180.4','response':{'connectionId':0,'connectionReused':True,'encodedDataLength':178,'fromDiskCache':False,'fromPrefetchCache':False,'fromServiceWorker':False,'headers':{'accept-ranges':'bytes','cache-control':'max-age=7200\npublic','cf-ray':'65444b781d1fe604-LHR','content-length':'42','content-type':'image/gif','date':'Mon, 24 May 2021 05:58:53 GMT','etag':'"60a4d856-2a"','expires':'Mon, 24 May 2021 07:58:53 GMT','last-modified':'Wed, 19 May 2021 ''09:20:22 GMT','server':'cloudflare','vary':'Accept-Encoding','x-content-type-options':'nosniff','x-frame-options':'DENY'},'mimeType':'image/gif','protocol':'h3-29','remoteIPAddress':'104.21.5.197','remotePort':443,'requestHeaders':{':authority':'nowsecure.nl',':method':'GET',':path':'/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=65444b779ae6546f',':scheme':'https','accept':'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8','accept-encoding':'gzip, deflate, ''br','accept-language':'en-US,en;q=0.9','referer':'https://nowsecure.nl/','sec-ch-ua':'" Not ''A;Brand";v="99", ''"Chromium";v="90", ''"Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0','sec-fetch-dest':'image','sec-fetch-mode':'no-cors','sec-fetch-site':'same-origin','user-agent':'Mozilla/5.0 ''(Windows NT 10.0; ''Win64; x64) ''AppleWebKit/537.36 ''(KHTML, like Gecko) ''Chrome/90.0.4430.212 ''Safari/537.36'},'responseTime':1621835932268.067,'securityDetails':{'certificateId':0,'certificateTransparencyCompliance':'compliant','cipher':'AES_128_GCM','issuer':'Cloudflare Inc ECC ''CA-3','keyExchange':'','keyExchangeGroup':'X25519','protocol':'QUIC','sanList':['sni.cloudflaressl.com','*.nowsecure.nl','nowsecure.nl'],'signedCertificateTimestampList':[{'hashAlgorithm':'SHA-256','logDescription':'Google '"'Argon2021' "'log','logId':'F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3','origin':'Embedded ''in ''certificate','signatureAlgorithm':'ECDSA','signatureData':'30450221008A25458182A6E7F608FE1492086762A367381E94137952FFD621BA2E60F7E2F702203BCDEBCE1C544DECF0A113DE12B33E299319E6240426F38F08DFC04EF2E42825','status':'Verified','timestamp':1598706372839.0},{'hashAlgorithm':'SHA-256','logDescription':'DigiCert ''Yeti2021 ''Log','logId':'5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA','origin':'Embedded ''in ''certificate','signatureAlgorithm':'ECDSA','signatureData':'3046022100A95A49C7435DBFC73406AC409062C27269E6E69F443A2213F3A085E3BCBD234A022100DEA878296F8A1DB43546DC1865A4C5AD2B90664A243AE0A3A6D4925802EE68A8','status':'Verified','timestamp':1598706372823.0}],'subjectName':'sni.cloudflaressl.com','validFrom':1598659200,'validTo':1630238400},'securityState':'secure','status':200,'statusText':'','timing':{'connectEnd':-1,'connectStart':-1,'dnsEnd':-1,'dnsStart':-1,'proxyEnd':-1,'proxyStart':-1,'pushEnd':0,'pushStart':0,'receiveHeadersEnd':42.415,'requestTime':190011.110341,'sendEnd':25.713,'sendStart':25.609,'sslEnd':-1,'sslStart':-1,'workerFetchStart':-1,'workerReady':-1,'workerRespondWithSettled':-1,'workerStart':-1},'url':'https://nowsecure.nl/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=65444b779ae6546f'},'timestamp':190011.175727,'type':'Image'}}{'method':'Network.dataReceived','params':{'dataLength':42,'encodedDataLength':0,'requestId':'17180.4','timestamp':190011.175856}}{'method':'Network.dataReceived','params':{'dataLength':0,'encodedDataLength':44,'requestId':'17180.4','timestamp':190011.176133}}{'method':'Network.loadingFinished','params':{'encodedDataLength':222,'requestId':'17180.4','shouldReportCorbBlocking':False,'timestamp':190011.153335}}{'method':'Network.responseReceivedExtraInfo','params':{'blockedCookies':[],'headers':{'alt-svc':'h3-27=":443"; ma=86400, h3-28=":443"; ''ma=86400, h3-29=":443"; ma=86400','cache-control':'max-age=0, must-revalidate','cf-ray':'65444b781d1ee604-LHR','cf-request-id':'0a3e8d7f140000e60496387000000001','content-encoding':'br','content-type':'text/javascript','date':'Mon, 24 May 2021 05:58:53 GMT','expect-ct':'max-age=604800, ''report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"','nel':'{"report_to":"cf-nel","max_age":604800}','report-to':'{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report?s=ZtI%2Bx8B7DpI8%2FsDA72maecFVCPvIsfBOyJjT8weyiqfmrHrmcBYpRhc%2FI%2F6JmIlnxW%2F%2BBohxLi1F8mpjAUabJ0kXLYnmjGKp2Ndio9M%3D"}],"group":"cf-nel","max_age":604800}','server':'cloudflare','vary':'Accept-Encoding'},'requestId':'17180.2','resourceIPAddressSpace':'Public'}}{'method':'Network.responseReceived','params':{'frameId':'F42BAE4BDD4E428EE2503CB5A7B4F700','loaderId':'449906A5C736D819123288133F2797E6','requestId':'17180.2','response':{'connectionId':0,'connectionReused':True,'encodedDataLength':510,'fromDiskCache':False,'fromPrefetchCache':False,'fromServiceWorker':False,'headers':{'alt-svc':'h3-27=":443"; ma=86400, ''h3-28=":443"; ma=86400, ''h3-29=":443"; ma=86400','cache-control':'max-age=0, ''must-revalidate','cf-ray':'65444b781d1ee604-LHR','cf-request-id':'0a3e8d7f140000e60496387000000001','content-encoding':'br','content-type':'text/javascript','date':'Mon, 24 May 2021 05:58:53 GMT','expect-ct':'max-age=604800, ''report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"','nel':'{"report_to":"cf-nel","max_age":604800}','report-to':'{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report?s=ZtI%2Bx8B7DpI8%2FsDA72maecFVCPvIsfBOyJjT8weyiqfmrHrmcBYpRhc%2FI%2F6JmIlnxW%2F%2BBohxLi1F8mpjAUabJ0kXLYnmjGKp2Ndio9M%3D"}],"group":"cf-nel","max_age":604800}','server':'cloudflare','vary':'Accept-Encoding'},'mimeType':'text/javascript','protocol':'h3-29','remoteIPAddress':'104.21.5.197','remotePort':443,'requestHeaders':{':authority':'nowsecure.nl',':method':'GET',':path':'/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=65444b779ae6546f',':scheme':'https','accept':'*/*','accept-encoding':'gzip, deflate, ''br','accept-language':'en-US,en;q=0.9','referer':'https://nowsecure.nl/','sec-ch-ua':'" Not ''A;Brand";v="99", ''"Chromium";v="90", ''"Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0','sec-fetch-dest':'script','sec-fetch-mode':'no-cors','sec-fetch-site':'same-origin','user-agent':'Mozilla/5.0 ''(Windows NT 10.0; ''Win64; x64) ''AppleWebKit/537.36 ''(KHTML, like Gecko) ''Chrome/90.0.4430.212 ''Safari/537.36'},'responseTime':1621835932301.817,'securityDetails':{'certificateId':0,'certificateTransparencyCompliance':'compliant','cipher':'AES_128_GCM','issuer':'Cloudflare Inc ECC ''CA-3','keyExchange':'','keyExchangeGroup':'X25519','protocol':'QUIC','sanList':['sni.cloudflaressl.com','*.nowsecure.nl','nowsecure.nl'],'signedCertificateTimestampList':[{'hashAlgorithm':'SHA-256','logDescription':'Google '"'Argon2021' "'log','logId':'F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3','origin':'Embedded ''in ''certificate','signatureAlgorithm':'ECDSA','signatureData':'30450221008A25458182A6E7F608FE1492086762A367381E94137952FFD621BA2E60F7E2F702203BCDEBCE1C544DECF0A113DE12B33E299319E6240426F38F08DFC04EF2E42825','status':'Verified','timestamp':1598706372839.0},{'hashAlgorithm':'SHA-256','logDescription':'DigiCert ''Yeti2021 ''Log','logId':'5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA','origin':'Embedded ''in ''certificate','signatureAlgorithm':'ECDSA','signatureData':'3046022100A95A49C7435DBFC73406AC409062C27269E6E69F443A2213F3A085E3BCBD234A022100DEA878296F8A1DB43546DC1865A4C5AD2B90664A243AE0A3A6D4925802EE68A8','status':'Verified','timestamp':1598706372823.0}],'subjectName':'sni.cloudflaressl.com','validFrom':1598659200,'validTo':1630238400},'securityState':'secure','status':200,'statusText':'','timing':{'connectEnd':-1,'connectStart':-1,'dnsEnd':-1,'dnsStart':-1,'proxyEnd':-1,'proxyStart':-1,'pushEnd':0,'pushStart':0,'receiveHeadersEnd':78.885,'requestTime':190011.107975,'sendEnd':27.934,'sendStart':27.809,'sslEnd':-1,'sslStart':-1,'workerFetchStart':-1,'workerReady':-1,'workerRespondWithSettled':-1,'workerStart':-1},'url':'https://nowsecure.nl/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=65444b779ae6546f'},'timestamp':190011.188468,'type':'Script'}}{'method':'Network.dataReceived','params':{'dataLength':31556,'encodedDataLength':0,'requestId':'17180.2','timestamp':190011.188663}}{'method':'Network.dataReceived','params':{'dataLength':6737,'encodedDataLength':11251,'requestId':'17180.2','timestamp':190011.198249}}{'method':'Network.dataReceived','params':{'dataLength':0,'encodedDataLength':2049,'requestId':'17180.2','timestamp':190011.200943}}{'method':'Network.loadingFinished','params':{'encodedDataLength':13810,'requestId':'17180.2','shouldReportCorbBlocking':False,'timestamp':190011.198142}}{'method':'Page.loadEventFired','params':{'timestamp':190011.204711}}{'method':'Page.frameScheduledNavigation','params':{'delay':12,'frameId':'F42BAE4BDD4E428EE2503CB5A7B4F700','reason':'metaTagRefresh','url':'https://nowsecure.nl/'}}{'method':'Page.frameStoppedLoading','params':{'frameId':'F42BAE4BDD4E428EE2503CB5A7B4F700'}}{'method':'Network.requestWillBeSent','params':{'documentURL':'https://nowsecure.nl/','frameId':'F42BAE4BDD4E428EE2503CB5A7B4F700','hasUserGesture':False,'initiator':{'type':'other'},'loaderId':'449906A5C736D819123288133F2797E6','request':{'headers':{'Referer':'https://nowsecure.nl/','User-Agent':'Mozilla/5.0 (Windows NT ''10.0; Win64; x64) ''AppleWebKit/537.36 (KHTML, ''like Gecko) ''Chrome/90.0.4430.212 ''Safari/537.36','sec-ch-ua':'" Not A;Brand";v="99", ''"Chromium";v="90", "Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0'},'initialPriority':'High','method':'GET','mixedContentType':'none','referrerPolicy':'strict-origin-when-cross-origin','url':'https://nowsecure.nl/favicon.ico'},'requestId':'17180.5','timestamp':190011.210491,'type':'Other','wallTime':1621835932.325683}}{'method':'Network.requestWillBeSentExtraInfo','params':{'associatedCookies':[{'blockedReasons':[],'cookie':{'domain':'nowsecure.nl','expires':1621839532,'httpOnly':False,'name':'cf_chl_prog','path':'/','priority':'Medium','sameParty':False,'secure':False,'session':False,'size':12,'sourcePort':443,'sourceScheme':'Secure','value':'e'}}],'clientSecurityState':{'initiatorIPAddressSpace':'Public','initiatorIsSecureContext':True,'privateNetworkRequestPolicy':'WarnFromInsecureToMorePrivate'},'headers':{':authority':'nowsecure.nl',':method':'GET',':path':'/favicon.ico',':scheme':'https','accept':'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8','accept-encoding':'gzip, deflate, br','accept-language':'en-US,en;q=0.9','cookie':'cf_chl_prog=e','referer':'https://nowsecure.nl/','sec-ch-ua':'" Not A;Brand";v="99", ''"Chromium";v="90", "Google ''Chrome";v="90"','sec-ch-ua-mobile':'?0','sec-fetch-dest':'image','sec-fetch-mode':'no-cors','sec-fetch-site':'same-origin','user-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; ''x64) AppleWebKit/537.36 (KHTML, like ''Gecko) Chrome/90.0.4430.212 ''Safari/537.36'},# hopefullly you get the idea.the easy way (v1 old stuff)importundetected_chromedriverasucdriver=uc.Chrome()driver.get('https://distilnetworks.com')target specific chrome version  (v1 old stuff)importundetected_chromedriverasucuc.TARGET_VERSION=85driver=uc.Chrome()monkeypatch mode  (v1 old stuff)Needs to be done before importing from selenium packageimportundetected_chromedriverasucuc.install()fromselenium.webdriverimportChromedriver=Chrome()driver.get('https://distilnetworks.com')the customized way  (v1 old stuff)importundetected_chromedriverasuc# specify chromedriver version to download and patchuc.TARGET_VERSION=78# or specify your own chromedriver binary (why you would need this, i don't know)uc.install(executable_path='c:/users/user1/chromedriver.exe',)opts=uc.ChromeOptions()opts.add_argument(f'--proxy-server=socks5://127.0.0.1:9050')driver=uc.Chrome(options=opts)driver.get('https://distilnetworks.com')datadome.co example  (v1 old stuff)These guys have actually a powerful product, and a link to this repo, which makes me wanna test their product. Make sure you use a "clean"ip
for this one.## STANDARD selenium Chromedriver#fromseleniumimportwebdriverchrome=webdriver.Chrome()chrome.get('https://datadome.co/customers-stories/toppreise-ends-web-scraping-and-content-theft-with-datadome/')chrome.save_screenshot('datadome_regular_webdriver.png')True# it caused my ip to be flagged, unfortunately## UNDETECTED chromedriver (headless,even)#importundetected_chromedriverasucoptions=uc.ChromeOptions()options.headless=Trueoptions.add_argument('--headless')chrome=uc.Chrome(options=options)chrome.get('https://datadome.co/customers-stories/toppreise-ends-web-scraping-and-content-theft-with-datadome/')chrome.save_screenshot('datadome_undetected_webddriver.png')Check both saved screenhotshere