github.com/go-rod/rod (go): Package rod is a high-level driver directly based on DevTools Protocol.  This example opens https://github.com/, searches for "git", and then gets the header element which gives the description for Git.    Rod use https://golang.org/pkg/context to handle cancellations for IO blocking operations, most times it's timeout. Context will be recursively passed to all sub-methods. For example, methods like Page.Context(ctx) will return a clone of the page with the ctx, all the methods of the returned page will use the ctx if they have IO blocking operations. Page.Timeout or Page.WithCancel is just a shortcut for Page.Context. Of course, Browser or Element works the same way.   Shows how we can further customize the browser with the launcher library. Usually you use launcher lib to set the browser's command line flags (switches). Doc for flags: https://peter.sh/experiments/chromium-command-line-switches   Shows how to change the retry/polling options that is used to query elements. This is useful when you want to customize the element query retry logic.   When rod doesn't have a feature that you need. You can easily call the cdp to achieve it. List of cdp API: https://github.com/go-rod/rod/tree/main/lib/proto   Shows how to disable headless mode and debug. Rod provides a lot of debug options, you can set them with setter methods or use environment variables. Doc for environment variables: https://pkg.go.dev/github.com/go-rod/rod/lib/defaults    We use "Must" prefixed functions to write example code. But in production you may want to use the no-prefix version of them. About why we use "Must" as the prefix, it's similar to https://golang.org/pkg/regexp/#MustCompile   Shows how to share a remote object reference between two Eval   Shows how to listen for events.   Shows how to intercept requests and modify both the request and the response. The entire process of hijacking one request:  The --req-> and --res-> are the parts that can be modified.       Show how to handle multiple results of an action. Such as when you login a page, the result can be success or wrong password.   Example_search shows how to use Search to get element inside nested iframes or shadow DOMs. It works the same as https://developers.google.com/web/tools/chrome-devtools/dom#search   Shows how to update the state of the current page. In this example we enable the network domain.   Rod uses mouse cursor to simulate clicks, so if a button is moving because of animation, the click may not work as expected. We usually use WaitStable to make sure the target isn't changing anymore.   When you want to wait for an ajax request to complete, this example will be useful.
Keywords: automation|cdp|chrome-devtools|chrome-devtools-protocol|chrome-headless|crawling|devtools|devtools-protocol|go|golang|gorod|headless|rod|scraper|testing|web|web-scraping