Repository: go
Issue: proposal: x/net/html: parser hooks
Labels: Proposal

### Proposal Details

The following is not something I _need_ right now, but my use case made me think that there is a valid reason for the code calling `html.Parse` to be able to influence the parsing process.

I'm playing around with a crazy idea, to create a headless browser written in Go (I have a POC that proves I can write the DOM in Go, and expose the objects to JavaScript - I have a v8 engine integrated).

I have used `x/net/html` to parse HTML; but there are some rules for constructing the DOM in the browser that the library doesn't support (and I don't think it should _implement_ those rules - but let the caller deal with them).

For example, when a `<script>` element is connected, the script is executed. Which means the script doesn't see the entire "HTML source" (or the DOM representation of the entire source to be exact). Scripts aren't the only types of element with these types of rules wither.

I can work around that right now; but makes me want to ask the question, if support should be added to `x/net/html` that allows the caller to install hooks in the process. Events like `inserted`, `connected`, `removed` seems like candidates.

As per the [DOM tree specs](https://html.spec.whatwg.org/multipage/infrastructure.html#dom-trees)

> An HTML element can have specific HTML element insertion steps, HTML element post-connection steps, and HTML element removing steps, all defined for the element's [local name](https://dom.spec.whatwg.org/#concept-element-local-name).

**My current approach**

The path I have decided to take forward is to first use the `x/net/html` package to construct a node tree, and then iterate that tree to construct my own tree. So basically I'll perform two passes of DOM processing, and I can implement the necessary rules in the 2nd pass.

I will most likely use the `html.Node` instance as a "backing field" for DOM data, as it gives the benefits:
- I get HTML rendering out of the box, e.g. `element.outerHTML`/`innerHTML`
- Libraries exist already for XPath search

Even if I could hook into the parser process I still need to create wrapper types for various reasons. E.g., I need to expose a different interface to JavaScript, and I need a binding layer to JavaScript.

**What benefits would the hooks give**

If the package supported hooks you could generate the "correct DOM" in one pass.

In my case, the wrapper objects could then just be created lazily.

**Alternate solution**

The suggested _hooks_ is just _one possible solution_ to the problem;

Another potential solution could be to let the caller pass in some kind of "Node Factory". But that does seem more complex.

**A bit background about the idea**

The idea was basically to be able to test HTTP apps written in Go; but where client-side script plays an important role. The standard library provides everything necessary if the test only need to inspect the HTTP response, e.g. response headers or body. But if you want to verify that the behavior JavaScript, you need something more. An example could be using HTMX and Go, a tech-combo that seems to be gaining some popularity. Here, the behavior of the app depends on setting the right attributes on specific HTML elements, and be sure that the returned HTTP responses play well with those tags.

Using a _real_ browser in headless has a huge overhead; which tends to discourage TDD; which otherwise provides a fast feedback loop.

Being in written in Go, it easily bypass the TCP layer entirely and connect directly to the root `net/http.Handler`. This makes it easy to let tests run in parallel with each test using with their own http handler; which could possibly have different dependencies mocked or stubbed for different test cases.

The project is as (but this is an extremely early prototype) is in https://github.com/stroiman/go-dom