UNDERSTANDING HTML WITH LARGE LANGUAGE MODELS
Authors: Izzeddin Gur
Ofir Nachum
Yingjie Miao
Mustafa Safdari
Austin V Huang
Sharan Narang
Aakanksha Chowdhery
Noah Fiedel
Aleksandra Faust
Venue: Submitted to ICLR 2023 (2023)
Keywords: html understanding
web navigation
large language models
semantic classification
description generation

Abstract: Large language models (LLM) have shown exceptional performance on a variety of natural language tasks. Yet, their capabilities for HTML understanding – i.e., parsing the raw HTML of a webpage, with applications to automation of web-based tasks, crawling, and browser-assisted retrieval – have not been fully explored. We contribute HTML understanding models (fine-tuned LLMs) and an in-depth analysis of their capabilities under three tasks: (i) Semantic Classification of HTML elements, (ii) Description Generation for HTML inputs, and (iii) Autonomous Web Navigation of HTML pages. While previous work has developed dedicated architectures and training procedures for HTML understanding, we show that LLMs pretrained on standard natural language corpora transfer remarkably well to HTML understanding tasks. For instance, fine-tuned LLMs are 12% more accurate at semantic classification compared to models trained exclusively on the task dataset. Moreover, when fine-tuned on data from the MiniWoB benchmark, LLMs successfully complete 50% more tasks using 192x less data compared to the previous best supervised model. To promote further research on LLMs for HTML understanding, we create and open-source a large-scale HTML dataset distilled and auto-labeled from CommonCrawl. We show evidence that T5-based models due to the bidirectional encoder-decoder architecture are the best choice and that for practitioners larger models are not necessarily better.

Under review as a conference paper at ICLR 2022
UNDERSTANDING HTML WITH LARGE LANGUAGE MODELS
Anonymous authors Paper under double-bling review
ABSTRACT
Large language models (LLMs) have shown exceptional performance on a va- riety of natural language tasks. Yet, their capabilities for HTML understanding - i.e. parsing the raw HTML a webpage, with applications to automation web-based tasks, crawling, and browser-assisted retrieval have not been fully explored. We contribute HTMI understanding models (fine-tuned L.MSs) and an in-depth analysis of their capabilities under three tasks: (i) Semantic Classifica- tion of HTML elements, (ii) Description Generation for HTML inputs, and (iii) Autonomous Web Navigation of HTML pages. While previous work has devel- oped dedicated architectures and training procedures for HTML understanding, we show that LLMs pretrained on standard natural language corpora transfer re- markably well to HTML understanding tasks. For instance, fine-tuned LLMs are 12% more accurate at semantic classification compared models trained exclu- sively on the task dataset. Moreover, when fine-tuned data from the MiniWoB benchmark, LLMs successfully complete 50% more tasks using 192x less data compared to the previous best supervised model. Out of the LLMs ate, we show evidence that T5-based models are ideal due to their bidirectional encoder-decoder architecture. To promote further research LLMs for HTML understanding, we create and open-source a large-scale HTML dataset distilled and auto-labeled from CommonCrawl!
evalu-
INTRODUCTION
Web crawling Olston et al. 2010), form- filliiiiing DDaz et al. 2013 Gur et al. 2021), or information retrieving web agents (Nogueira & Cho 2016) important for both automating and assisting users in web-based tasks. These and similar applications rely on models that can search for specific content controls on web page as well as navigate website autonomously. Since web page in its raw form is represented as an HTML--based text sequence, the success of models for web-based tasks relies on their ability understand HTML semantics, structure, and embedded interactions.
The predominant approach to web automation and HTML understanding is to train specialized mod- els, i.e., gathering application-specific datasets and designing neural network (NN) architectures to leverage inductive biases of the HTML's structure; see, e.g., Liu et al. (2018); Toyama et al. (202T); Gur al. (202I) [Humphreys al. (2022). However, both dataset collection and neural architecture design are expensive, time-consuming, and require highly-specialized, domain-specife knowledge.
Meanwhile, the natural language processing (NLP) literature, large language models (LLMs)) have emerged as a solution to the difficulties of dataset collection and specialized NN design (Kaplan) cet al. 2020 Bommasani et al. 202I). A popular paradigm in NLP is to take an off-the-shelf LLM pretrained on a large text corpus via unsupervised and task-agnostic learning objective and either fine-tune or prompt the LLM on small task-specific dataset. This paradigm has shown exceptional performance
variety of NLP tasks (Xue al. 2020 Brown et al. 2020; Austin ct al. 2021). Whether LLMs can be applied to HTML understanding especially given the much larger context and sequence lengths remains an under-explored question.
In this paper, we investigate whether LLMs can be applied to HTML understanding to produce better-performing, more sample-efficient HTML understanding models and without the need for
See visualizations of the results at httpp:://stttt google www/vwww//////eeo..o 144ht11 /home

---

Under review as a conference paper at ICLR 2022
shar>
<fors classe"log "logt-oooo--orr
<die>
Enter Emil aotress
<label>>
tore"pars's
Enter Fastzord:
<labol>
Encoder Only username
Lirgut typte small" shand
parsntrd"
span closs "hid0o"">
Classification Layer
EOS
/diis
type SSign
riform
s/boay-
ENCODER
DECODER
Classifcattion
Sementic
Description Generation
Nevigation
<ipoeeaaee classe fart- fare stame "Caaci addrasse (a0e)
Encoder-Decoder
Usemame
Enter Amateee
Dypeltome- UNamn" Youreer "ysaeer Vigiskconn)
Class Taidder ohleas exter pastea srr prrpanniiii
targetoroqpe typer ode pass seepas
(a)
(b)
Figure 1: a) HTML example page with highlighted salient element, an element of interest (dashed box). All canonical tasks evaluate distinct interaction with this element, either by classifying as one set of categories, generating a text description of its purpose, or applying an action as part of sequential navigation of a multi-page website. b) LLM architectures overview. Dashed boxes denote sub-mu sub-modules that are specific to either encoder-only or encoder-decoder models. For encoder-only models, we add extra classification layer. Decoder-only models (not in the diagram) are similar encoder-decoder models, main difference is that the HTML snippet is fed to the decoder and processed from left-to-right.
custom NN architecture design. To that end, we present a suite of three benchmarking tasks for HTML understanding that capture the essence of these applications and require understanding both structure and content. First, we devise Semantic Classification as a task that requires a model to classify a given HTML element into one of etc., with application to automated form-filling. Second, we present Description Generation, label-extraction task where model is given an HTML snippet and asked to produce natural language description. For instance for an email field, the description might be "Please enter your email address." " Note that in the majority of web pages, this connection between input elements and description content is only implicit in the raw HTML code and inferring such links is prerequisite for higher-level navigation objectives. The third task is Autonomous Web Navigation (Shi et al,, 2017). A model is presented with HTML page paired with natural language command and must apply appropriate actions on a sequence of HTML pages to satisfy the command. See Figure la for simplified example of these tasks.
of categories, such as address, email, password
With these benchmark tasks in hand, we evaluate the transfer capabilities a variety of pretrained LLMs (Table []. varying in architecture (encoder-only, encoder-decoder, or decoder-only), model size (from 24.6M to 62B parameters), and training data corpora (both including and excluding pre- training NLP and HTML corpus) While prior work universally pre-parses the HTML input the 2021 Liu et al. 2018 Nakano et al. 2021). ours - to the best of our knowledge - is the first work that uses raw, unprocessed HTML. Our results show that LLMs demonstrate a remark- able level of HTML understanding across all tasks, with up to 192x more sample-efficieng than models trained from scratch, and achieving a new SoTA for supervised learning on the Mini MoB benchmark suite (Shi et al.) (017). The -ncoder-decoder architectures with bi-directional attention show the best performance across the board even when their pretraining does not include HTML. In addition, we show that the performance scales sub-linearly with the model size.
model (Gur et al.
The broader objective of this research is advance the integration of L.IMs with autonomous web agents. It has only been in the last year that researchers have begun to utilize LLMs outside of NLP and integrate them as core capabilities in autonomy (Lu et al. (201]); Ahn et al. (2022)). In this context, LLMs are reasoning engines for sequential decision making agents interacting with environments.
The present work is the first in the research literature to embed LLM and train it agent for autonomous web navigation. This requires new implementations to adapt LLM training for behavior cloning in addition designing interfaces for integrating text generation into a perception-compute-
2

---

Under review as a conference paper at ICLR 2022
action cycle operating in stateful web environment. Our implementation allows to answer new questions regarding trade-offs among various model characteristics.
We believe these contributions expand the scope of language models and connect their unique capa- bilities with autonomous agents for the web. We provide a new perspective on machine learning for HTML understanding and web automation, showing that pretrained LLMs can achieve significant performance on such tasks, reducing the need for specialized architectures and training protocols. To encourage further research in this direction, we open sourced | model weights for agents used in the WoB environment and plan to open-source our dataset for description generation and.
2 RELATED WORK
HTML Understanding Autonomous web navigation has been a popular application for neural net- work models, and variety of works propose simulated websites for training web-based agents, with
application to task fulfillment (Yao et al. 2022 Gur et al., 2021] Burns et al. [2022] Mazumder & Riva 2020 Shi al. 2017 Liu et al. 2018 as well as information retrieval or question-anse wnnweinnnn phs et al.) 2021, Nogueira & Cho 2016). Simulated websites provide an easy way to evaluate models online, and for this reason we use the existing MiniWoB benchmark (Shi et al. 2017) 7) for our web navigation setting. However, it is still important to have mechanism for evaluating models on a wide variety of real-world websites. This was the key motivation for generating our own dataset for the description generation task, which is distilled and auto-labeled from CommonCrawi and is key contribution of our paper.
Alongside these benchmarks, many works have developed models for web navigation and related subtasks (Pasupat et al. 20188) Boommasan et al. [02T] [e et al.] 2021 Gur et al. 202T] Humphreys et al. [/2)|Liu et al., [2188] Dia al. [2019). These works often rely specialized neural network architectures that leverage inductive blases of HTML structure, or on preprocessing of HTML to
a b)). In contrast, our work takes a minimalist approach, providing HTML in text form with minimal processing and using widely-adopted trans- former networks.
make it easier to input model (Li et al.
LLMs and HTML Works that explore the intersection of LLMs and HTML generally fall into two categories. The first category uses LLMs to assist web navigation (Nakano et 202 Yao al., 2022), and typically relies on a custom preprocessing to map the context and structure of a web page to natural language, thus severely restricting what HTML pages the model can parse. The second category pretrains LLMs on large corpora of HTML text (Aghajanyan et al., (021). However, these works typically restrict the model evaluation to standard NLP tasks, e.g., summarization and question/answering
opposed to tasks more relevant to HTML understanding and web automation. Our work can be thought of as the reverse: We keep the pretraining of LLMs unchanged and focus on the mechanisms for transferring the pretrained LLMs to HTML-relevant tasks.
BRIEF BACKGROUND ON HTML AS SEMI-STRUCTURED TEXT DATA
HTML is a markup language, used to organize web page structure and content. Consider the example HTML page in Figure la] This web page includes two adjacent input elements, one for e-mail and another for password, with their corresponding labels 1s separate branch of the page. These inputs and 1 labels are one of many possible elements that serve as HTML building blocks. Each element has set of attributes key and value pair that describe the element's 's content, such as style and human-readable text. When rendered in a browser, these attributes will be responsible for how the element shown and where it is positioned. In the example in Figure Tal the first input has three attributes, tag=" " "iipput"" type= "email" ", and id="uName" ", that identify the element an email input with an identifier ("uName"") that can be accessed programmatically.
3

---

Under review as a conference paper at ICLR 2022
Model
Task
Dataset
Size
Input
Architecture
Output
Task Output
Autenomous Web Navigation
MiniWoB Demos (Shi et al. 2017
12K
Page
Enc-Doe
Dec
Text
Dictionary
Semuniic Clazzification
Anmotated Shopping Webpages Glui et al. 2021
28K
Snippet
All
Tex
Calegury
Description Generation
CommonCCrrwl (new)
85K
Snippet
Enc-Dec
Text
Text
Dec
Table I: Task, datasct, and model summary. All models receive raw HTML. Autonomous Web Navigation receives the entire HTML, while the other tasks receive HTML snippets extracted given salient element.
CANONICAL TASKS FOR HTML UNDERSTANDING
We devise three canonical tasks to study HTML understanding capabilities of LLM-based web agents, These tasks require correctly interpreting both structure and content to varying degrees make predictions, with autonomous navigation being the most challenging capability of the three.
Autonomous Web Navigation. This task evaluates how well a model navigates multi-page web- sites as sequential decision-making problem (Shi et al,, 2017 Liu et al. 2018). At the beginning of an episode, the agent is given natural language instruction, e.g. Enter the username "Iyda" and the password "N22r" into the text fields and press login. The agent applies actions to a se- quence of HTML pages, where each action is of the form function (selector, text). The funct ion is one of click or rype, selector is an integer pointer that uniquely identifies an ele- ment, and txx is text to input if the type functionality is activated. An episode terminates when either the page reaches a terminal state (e.g., the 'sign in' button is clicked) or the maximum number of steps is reached.
Semantic Classification. Many HTML understanding applications require a model that can classify HTML elements into standardized categories. For example, in automated form-filling (Diaz al.) 2013 Guret al. 2021). it is useful to identify a 'submit button' across many websites (e.g., shopping, flight booking, utility application) with various button representations (e.g., position, color, text). Thus, we formulate Semantic Classification as classifying elements into role categories. Take the example HTML in Figure La which includes two input elements and a submit button. Let's pick the first i Input as an element of interest to be classified by the system, also called a salient element. The system should classify this element as username, since it appears on a login page and it has a label with Email Address which is typically associated with the username in form-fill form-filling applications. To solve this, the system can aggregate information from multiple sources the page the label that says Enter Email Address, the input attributes (type= "email" and id= "uName" "), or even the ordering of other elements in the page such as 'password' and 'sign in'.
Description Generation. Motivated by applications in accessibility-mine minded web browser trol (Jorgensen & Binsted, (2005). we formulate description generation as an extractive problem where the goal is locate the textual description of element in the HTML and generate it as output. For instance, the description of the salient element in Figure Ta is Enter Email Address; when rendered, this label will appear above the 'email' input field. HTML provides large amount of flexibility, and so in general a descriptive text that appears alongside a specific element when rendered can very far from that element when looking at the HTML plaintext. Thus, this task evaluates a model's 's ability to understand the structure of HTML as it would appear to user, despite not having access to the rendered web page directly.
DATASETS
Each of our canonical tasks requires a separate dataset, with the description generation task using a newly contributed, auto-labelled dataset based CommonCrawl.
Autonomous Web Navigation. We use the 12K demonstrations included in the publicly available MiniWoB benchmark (Shi et al. (017). which encompass 62 website applications ranging from email forwarding to social media interactions. Each demonstration is a sequence of (instruction, HTML, action) tuples. Every element in a Mini WoB demonstration is accompanied by a reference number unique within its respective pages. This number can be used as an element selector, making the action space unified across all tasks and time steps. For instance, the action in Figure would be
4

---

Under review as a conference paper at ICLR 2022
rype(ref=5, "username weaaiaaeeemmaal.ommmm.. where 5 refers to the index of the imput when counted from top-to-bottom. As model input, we concatenate the natural language instruction and HTML into a single text input sequence. Similarly, we treat the action
text sequence for the model to predict.
Semantic Classification. We use a dataset of 28K labelled examples, containing 66 different cat- egories, of the form (HTML, element, category), previously used in the context of environment generation (Gur et al. 2021]. The dataset consists of HTMLL from real-world shopping websites and categories relevant to form-filling during payment and checkout on these websites.
Description Generation. For this task, we derive a dataset from CommonCraw P CommonCrawl does not include renderings or annotations that would reveal what text in the HTML is associated with which elements. Instead, we infer descriptions of various elements by exploiting special attribute in the HTML schema known as for. As an example in Figure Ia the first label in the HTML has a for attribute with value uName, which is the id of the element described by label; in this case, the id is that of the first input in the page. This annotation does not affect the rendering of the page and is typically used for accessibility purposes. We utilize the information given by these for attributes to create a large-scale dataset to study description generation. small sample is available in the supplemental material, while the entire dataset will be available upon publication.
Specifically, we collected 100 WARC (from April 2019) files from the CommonCrawl project and extracted all HTML labels that have a for attribute. Removing non-Unicode and alphanumeric text in HTML labels results in 400K example datset. We balance the distribution of labels, effectively downsampling the dataset to 85K samples. Each example is represented as (HTML, element, description), where HTML is the HTML plaintext of the page, element is the element whose i attribute matches that appearing in the label's I's for attribute, and description is the text inside the label element (see example Figure Iap. More details of the dataset can be found in Appendix A.1
6 PRE-PROCESSING
In treating HTML as token sequences, we minimize any HTML tree pre-processing prior to model input. We thus provide HTML as raw text (i.e., sequences of text tokens) and only apply snippet extraction pre-processing for pages which are too large to fit into the typical LLMs context windows.
Snippet Extraction. Real HTML pages can grow extremely large, reaching thousands of elements, far beyond the context window of the largest LLM that we studied (1920 tokens in PaLM (Chowdh- ery et al. 2022) ) LLMs typically truncate such long sequences, which can be detrimental to HTML. understanding as HTMLS are not linearly structured. We take an element-centric approach and ex- tract HTML snippets (a small portion of HTML code) surrounding a salient element (Figure 55. A simple heuristic, which controls the tree's width and depth, guides the process: Start with salient element and traverse its ancestors in the HTML tree until a stopping condition is satisfied. As we traverse up, we estimate the height of the tree and the increased number of descendants of the new root. We stop when either metric violates pre-defined limit and take the resulting sub-tree as the snippet. We mark the salient element using a special attribute, called target, to distinguish it from other elements. We perform the snippet extraction for the semantic classification and description generation datasets, and keep the full HTML pages in Mini WoB because these pages are typically much smaller than real-world HTMI..
HTML un-Parsing. We provide the models with the unparsed plaintext HTML in the form of sequence of tokens. This canonical representation does not require specific model architectures such as hierarchical networks (Liu et al., [2018] Gur et al.] (222)) and can be fed into any LLM. We transform all datasets by converting every HTML page snippet into sequence. For MiniWoB, we additionally concatenate (action history, instruction, HTML) tuples into a single sequence.
http::/ccmmmneee .org
5

---

Under review as a conference paper at ICLR 2022
7 MODEL TRAINING
We study a variety of transformer-base LLMs ( Vasswani et al., 2017) with different sizes and archi- tectures for HTML understanding tasks (Table In the rest of the text, we prefix models fine-tuned for Autonomoux Web Navigation, Description Generation, and Semantic Classification with WebN- WebD-, and WebC-, respectively. For instance, WebD-T5-3B is the three billion parameter T5 model (Raffel al., 2020) fine-tuned for the Description Generation task. The rest of this section elaborates on training details.
Encoder-Decoder and Decoder-only Models. We train encoder-decoder models, i... T5 (Raffel et al. 2020), and decoder-only models, i.e., LaMDA (Toppilan et al. 2022) and PaLM (Chowdh- ery al., allaal]20000 2022), with text input and text output (Figure lbj. Inputs raw HTML pages or snippet texts; similarly, outputs are categories, natural language descriptions, or actions represented as text. Namely, for Semantic Classfsccaiiowe use the textual representation of categories, similar previ- ous classification problems in NLP (Raffel et al. 2020). For Autonomous Web Navigation, actions are converted into text by first converting them into key and value pairs and then concatenating the pairs.
Many websites in MiniWoB require multiple interactions, such as click-button-ssqqueet -eeueonssequeene or click- checkboxes, where each interaction might cause subtle change in the website state. For instance, after clicking on a checkbox the checkkhhck website, its value flips from positive to negative or the other way around, which is not always reflected in LLMs' predictions and leads to action repetitions. We solve this issue by augmenting tuples in the dataset with a sequence of past actions, (action history, instruction, HTML, action), and allowing LLMs learn from past experience.
Encoder-only Models. We train encoder-only models, i.c., BERT (Devlin et al. 2018). with text input and categorical output. We keep semantic categories discrete one-hot classes. To train encoder-only models, we add a new classification layer after the final encoder layer to produce a distribution over semantic categories. In addition to the typical BERT models, we study Mobile- BERT (Sun et al. 2020), distilled from BERT-large with inverted bottlenecks, and Albert-XL (Lan al. 2020), with parameter sharing and embedding split.
8 RESULTS
We now present the results of fine-tuned LLMs for HTML understanding. We compare the models' performance with the existing baselines where possible (autonomous web navigation) and against other LLM architectures and training regimes (all tasks). Sections 8.1 8.2 and 8.3 evaluate task- specific performance, while Section 8.4 assesses the performance across all the tasks.
Metrics: For autonomous web navigation we evaluate models' Success Rate, which is averaged over 100 episodes per task. For the other tasks, we use Accuracy to measure exact match between predic- tion and ground truth. In the description generation task, additionally provide evaluations using alternative 'soft' text evaluation metrics, BLEU and ROUGE-1, measuring the similarity between predicted and ground truth text.
8.1 AUTONOMOUS WEB NAVIGATION RESULTS
For Autonomous Web Navigation we fine-tune two WebN- encoder-decoder architectures (WebN- T5- and WebN-T5-3BI on 12k demonstrations from
real websites. We eval- uate the models on MiniWob (Liu al. 2018) benchmark, and compare with specialized architec- tures trained using supervised learning (SL) on 2.4 million human expert demonstrations CC-Net (SI.) (Humphreys et al. (022), and two RL models bootstrapped with SI., CC-Net (SI.) (CC-Net (SL & RL) (Humphreys et al. 2022), and WGE (SL & RL) (Liu et al. 2018)). Additionally, compare with the decoder-only architecture (WebN-Lambda-1 IB) and perform an ablation study on the impact of including the action history in the input.
Comparison to SoTA. Since previous works report success on only a subset of websites in Mini- WoB, we evaluate on 48 out of 62 websites that are common across all models. Table g in the Appendix reports ine-grained results while Figure 2a presents results averaged over all websites. Compared to CC-Net (SL.) which is trained on all 2.4M demonstrations, WebN-T5-3B improves the

---

Under review as a conference paper at ICLR 2022
12K Demoa
2.4M Demce
Model Name
Success (%)
Model Size
TT-large
800M
LaMDA-IB
15.6
1B
T5-3B
11.1
3B
WchN-T55-arge
46.4
800M
15.0gs
12
15-4
CC-M
WehN-LaMDA-IB
WebN-T5-3B
51.8
3B
Model
(a) Baseline comparison.
(b) Pre-training effect.
Figure 2: a) WebN-T5 performance compared to the previous SOTA models MiniWoB benchmark. WehN-T5-3B improves the task success 16% while using 192 times less data, compared the best supervised learning (SL.) model, CC-Net (SI.). LLMs performance only surpassed by works utilizing RI, requiring ders magnitude more online experience interaction with websites. b) LLMs with and without pretraining on Autonomous Web Navigation task. Those with pretraining (denoted by the 'WebN-' prefix) show a 2.5-4.5x performance improvement.
Model Name WebC-MobileBRRT
Test (%)
Dev (%)
Model Size
Code in training Corpus
78.1
77.7
24.6 M
WchC-Albert XL
83.5
83.1
58.9 M
WobC-BERT-smallee
84.4
83.6
38.7 M
WebC-BER' -ma--B
84.4
85.2
52.8 M
WebC-BERTTmmdiin
0%
WebC-BERT-baad
83.9
84.8
109.5 M
WebC-BERT-larg.
84.1
85.8
335.2 M
WebC-T5-baas
86.8
89.9
250 M
WebC-T5-large
89.3
800
WebC-T5-3B
87.7
90.3
3
WebC-LaMDA--IB
87.4
87.1
IB
12.5% Code
WebC-PaL.M
86.6
89.9
8 B
5%
Code (0.875% HTML)
WebC-PaLMM-62B
90.5
62 B
5% Code (0.875% HTML)
TS-large
76.4
800
T5-3B
77.2
73.8
3
0%
PaLM-8B
73.3
70.1
8B
Table 2: LLMs performance on the Semantic Classification task. fine-tuning off thels pretrained LLMs (model names with prefix 'Weh*') helps L.Ms transfer better compared to training the same architecture from scratch on the HTML dataset (model names without prefix "Web*'), improving the accuracy of M-L. more than 12%. While BebC-Pal.M-633 clearly performed better than all other models, found WebC-T5-large competitive with much larger models such as WebC-LaMDA-IID or WebC-PaLM-SB.
success 16% while only training on 12K publicly-avillabl demonstrations, yielding over 192x im- provement in sample-efficienyy. We find that all choices of LLMs outperform previous SL models. Notably, WebN-T5-3B significantly improves websites requiring multiple-action sequences such as click_checkboxes or websites requiring entering text such as login .user (Table We observe that the performance LLMs only surpassed by previous works utilizing RL, which require orders of magnitude more online experience interaction. Extending our fine-tuned L.LMS to an RI. setting is a promising avenue for future work.
Action history ablation. Across all LLMs we consistently observe decrease in success, av- erage 6.4%, when past actions are excluded from the inputs (Figure 2a). Action history helps with websites that require entering multiple texts, well understanding minor changes that could be difficult to detect (e.g. dlick.checkboxes and multi layout. texts in the website where the layout is randomized at each episode, yet, surprisingly, even the (rel- atively smaller) WebN-T5-large model without action history outperforms the CC-Net (SL) model; illustrating that incorporating action history is not the only contributing factor for the better success.
layout requires entering 3 different

---

Under review as a conference paper at ICLR 2022
Accuracy
75
50
25
pozawen rew
colk poloy sal tactoon countr
gent LonLonn JpM
dreal sunnayy
aae care eip neseeed
parteao
wome and conttiio
posa seeell Lode
nams las
sreat con Co virr sdftee
number J city
headl
sou can numbol si ftattinnn cels
moa and on mn yf
un
iin con nu cne
con
IIv's
adesiau"
secey seo
come ole poe aoo
ndcatt on cne soe an
mrsee montte vots cart coe to roccoot
use
git
12 suues
the
Categories
Figure 3: Accuracy per classification category of the WebC-T5-33 3B model on the development dataset.
WebC--PLLM WebC-T5-38
T5-38 (full data no pretreining)
05
New
Height
Test (%)
Dev (%)
descendants (%)
25
3
87.7
90.3
25
4
88.6
89.2
50
3
88.4
90.0
50
89.3
89.2
300
5
87.8
88.8
500
7
75.8
74.5
1500
Data Size (h)
(a)
Figure 4: a) Effect of snippet extraction parameters WebC-T5-3B. Increases above 50% in new descendants and height of 4. Large increases in both parameters lead large snippets and decrease in accuracy. b) Accu- racy over training data size. Using only 1000 labeled examples (4.4% of training dataset), WebC-T5-3B outperforms T5-3B (full data without pretraining) which trained on all available labeled data (approximately 30k examples), and outperforms WebC-PaLM-a which is an order of magnitude larger.
8.2 SEMANTIC CLASSIFICATION TASK RESULTS
To evaluate the Semantic Classification task, compare the T5 encoder-decoder architecture's 's three size variants (WebC-T5-base. WebC-T5-large, and WebC-T5-3B) fine-tuned on 22K real, human-labeled training websites. We compare with fine-tuned encoder only architectures (WebC-"BERT"), three fine-tuned decoder-only architectures (WebC-LaMDA and PaL.M,, and both encoder-decoder and decoder-only models trained on human labeled websites from scratch. Results are presented in Table-2 where we find that all WebC-LLMS perform well and significantly better than the same architectures without pretraining.
Accuracy per category. In Figure 3 we present accuracy distribution of the WebC-T5-3B model on the development dataset. The fine-tuned encoder-decoder model performs strongly on a majority of the categories (Figure 3) even on those with very few samples. For instance, the model is 100% accurate on password new which has only 56 training examples, because the class is unambiguous. On the other hand, unsurprisingly, the performance drops when the category is ambiguous, such as the email category which is frequently mistaken as username.
Snippet generation ablation. Two hyper-parameters govern snipper generation: percentage of new descendants and height of the new root. While small variations of both parameters not change the performance, increasing both degrades the performance significantly (Table 4ap. With new descendants up to 500% and height up to 7, the performance drops by more than 15%. Note that snippet generation returns the full-page HTML when both parameters increase indefinitely.
Data size impact. When varying the fine-tuning training data sizes (1, 5, 10, 20, or 50 samples per class) in Figure 4b) WebC-T5-3B slightly outperforms WebC-PaLM--8B which is an order of mag- nitude larger. Compared to T5-3B that is trained on all available HTML data without pretraining, WebC-T5-3B achieves better performance while using only 3.4% of labeled data (1000 samples),
8

---

Under review as a conference paper at ICLR 2022
Test
Dev
Model Name
(ccuracy($e)
BLEU
ROUGE-1
Accuracy (%)
BLEU
ROUGE-1
WehD--T5-lage
83.22
90.2
90.5
84.3
91.7
91.5
WebD-LaMDA-IB
83.3
87.5
90.2
84.3
88.6
91.2
WebD-T5-3B
84
90.8
90.9
85.2
92.1
91.9
Closest Description
57.4
59.2
60.8
23.9
62.1
Table 3: Description generation accuracy of LLMs.
thus highlighting the benefit of using standard off-the-shelf pretrained LLMs for HTML understand- ing.
8.3 DESCRIPTION GENERATION TASK RESULTS
For Description Generation split the CommonCrawl dataset based URL top-level domains to test LLMs' capabilities to generalize to unseen HTML. We fine-tune encoder-decoder architectures (WebD-T5* and decoder-only models also evaluate strong heuristic baseline which simply finds the description closest to the salient element the HTML text (Closest Description).
with results presented Table 3 We
Accuracy and Similarity Performance We show results of our evaluations in Table All models achieve high scores across all metrics, achieving = 84% on the accuracy terms of exact match and a higher non-exact match score based BLEU and ROUGE-1 (s 91%). This difference indicates that the models are capable of locating the descriptions, but not always generating the exact output.
8.4
HTML UNDERSTANDING LLMS PERFORMANCE ANALYSIS ACROSS TASKS
We now analyze our results in aggregate derive our main conclusions.
8.4.1 PRETRAINING EFFECT: PRETRAINING LARGE TEXT CORPORA MATTERS
Fine-tuned pretrained LLMs outperform LLMs trained on HTML-only data, improving the perfor- mance by more than 34.1% on the Autonomous Web Navigation (Table 2b and 10% to 12.7% on Semantic Classification task (T.
Since Autonomous Web Navigation is the most difficult task, the improved performance is an en- couraging evidence of the value of LLMs in HTML understanding tasks. Specifically, we observe that LLMs without pretraining are comparable to fine-tuned pretrained models only websites that require simple text matching. In contrast, for websites such as biikkcheekboxes. text matching is harder and find that pretraining key to good performance. We also found that without pretrain- ing, model outputs were frequently in an incorrect format such as invalid dictionaries or invalid refs with non-integer values. This suggests that the large corpora used for pretraining helps models to learn general HTML structure.
8.4.2 ARCHITECTURE EFFECT: T5-BASED MODELS PERFORM BEST ACROSS ALL TASKS
Encoder-decoder T5 based models perform better across all three tasks. On the Autonomous Web Navigation task, encoder-decoder (WebN-T5) architectures are better or comparable WebN- LaMDA-IB (Figure 2a). On the Semantic Classification, the smallest encoder-decode model (WebC-T5-baal performs comparably much larger decoder-only models (WebC-LaMDA-IIB or WebC-PaL.M-8BB and the largest encoder-only model (WebC-BERT--aa
which has 85M more pa- rameters (Table We also observe that decoder-only M-L. performs worse than much-smaller encoder-decoder T5-large when trained only HTML data. Finally, on the Description Generation encoder-decoder architecture has higher BLEU score.
One possible explanation for the strong performance of T5-based moels is the encoder-decoder architecture of these models. Namely, T5 models utilize an encoder with a bidirectional attention mechanism, not present in the LaMDA and PaLM decoders. The bidirectional attention mechanism can process HTML pages from both ends, potentially overcoming the loss information when tree-structured HTML. pages are converted into fixed linear text sequences.

---

Under review as a conference paper at ICLR 2022
8.4.3 MODEL SIZE EFFECT: SIZE (SUB-LINEARLY) MATTERS
Across the tasks appears that the architecture plays an important role in the model performance. Model size and performance are also positively correlated, although they reach diminishing returns. For instance, the model performance is roughly Ollog log n) with respect to model size on Seman- tic Classification (Figure 4bjin Appendix). On the Autonomous Web Navigation task, performance grows slowly with the model size (Table while on the Description Generation it plateaus (Ta-
ble 3
8.5 DISCUSSION
Bi-directional attention vs training corpora: Pretraining on large corpora matters, yielding <4.5x performance improvements. Larger models tend to be better and we credit the bidirectional attention for T5's best overall performance across the tasks. PaLM and LaMDA include HTML and other code their pretraining corpora, while BERT and T5 architectures did not, showing that pretraining on HTML is not necessary for strong performance when fine-tuned for HTML understanding. This strengthens the hypothesis behind the role of the bidirectional attention, and opens up the possibility to further improve the performance of T5 architectures by pretraining them on corpora with HTML.
Practical impact on labeling: When available, the pretrained I.I.Ss need very little new expert data (200x and 30x reduction the web navigation and classification tasks, respectively). This has a big potential impact on practical applications, reducing the data collection time and cost by orders of magnitude.
Bigger not always better: When choosing the model size, the expected performance gains (sub- linear at best and asymptotic worst) should be considered alongside the model's training and inference time and cost. For instance, on the classification task, the largest model WebC-PaLM-667 takes several days fine-tune, and evaluates 30 Hz, while WebC-T55large fine-tunes in several hours and evaluates at 700 Hz. - an order of magnitude more expensive for single percent uplift in accuracy. BERT models on the other hand train in minutes. If the application does not require high precision, these might be a good choice.
Context window a bottleneck: The major bottleneck for the HTML understanding tasks seems to be the context window length that the current LLMs support, even with models that accept 1000+ to- kens. It remains prohibitive evaluate web navigation tasks on real websites that are orders of mag- nitude larger than pages in MiniWok Similarly. we observed that increasing the snippet size leads to major performance degradation. This makes HTMI. understanding an interesting benchmark for future LLM development. For instance, new methods may need to be developed to compress the state representation web content for use in LLM context windows.
CONCLUSION
We presented canonical tasks and fine-tuned LLMs for HTML understanding. The comprehensive evaluations and analyses over range of architectures, dataset sizes, and baselines yields practical findings and highlights current limitations of these models. find that a) pretraining is critical for the performance and can reduce labeled data requirements, improving sample efficiency up to 200x; b) model architecture is the second-mov important factor, and T5 models with bidirectional attention and recoder-decoder architecture perform the best across the board; c) given choice, model size should be evaluated in the context of the model's training and inference performance, as the model size sub-linearly correlates with its performance. Finally, the proposed HTML understanding tasks highlight the relatively short context window that limits current LLMs, suggesting possibilities for future research that incorporate or eliminate this constraint.
REFERENCES
Leonard Adolphs, Benjamin Boerschinger, Christian Buck, Michelle Chen Huebscher, Massimil- iano Ciaramita, Lasse Espeholt, Thomas Hofmann, and Yannic Kilcher. Boosting search engines with interactive agents. arXiv preprint 2rxiv:2119.00555 2021.
10

---

Under review as a conference paper at ICLR 2022
Armen Aghajanyan, Dmytro Okhonko, Mike Lewis, Mandar Joshi, Hu Xu, Gargi Ghosh, and Luke Zettlemoyer. Htlm: Hyper-text pre-training and prompting of language models. arXiv preprint arXiv:2107.06955, 2021.
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint wrXiv:2204.01699 2022.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Temy. Quoc Le, al. Program synthesis with large language models. arXiv preprini arXiv:2 (.Xiv:2108.0777: 077332, 2021.
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportu- nities and risks of foundation models. arXiv preprint arXiv:2108.0722 2021.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Andrea Burns, Deniz Arsan, Sanjna Agrawal, Ranjitha Kumar, Kate Saenko, and Bryan A Plummer. Interactive mobile app navigation with uncertain or under-specified natural language commands. arXiv preprint arXiv:2202.023111 2022.
Aakanksha Chowdhery, Sharan Narang. Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 2022.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1rXiv:1810.04800 2018.
Oscar Diaz, Itziar Otaduy, and Gorka Puente. User-driven automation of web form filling. In International Conference Web Engineering, pp. 171-185. Springer, 2013.
Izzeddin Gur, Natasha Jaques, Yingjie Miao, Jongwook Choi, Manoj Tiwari, Honglak Lee, and
Aleksandra Faust. Environment generation for zero-shot compositional reinforcement learning. Advances in Neural Information Processing Systems, 14:4157-4169 2021.
Zecheng He, Srinivas Sunkara, Xiaoxue Zang, Ying Xu, Lijuan Liu, Nevan Wichers, Gabriel Schu- biner, Ruby Lee, and Jindong Chen. Actionbert: Leveraging user actions for semantic under- standing of user interfaces. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 5931-5938, 2021.
Peter C Humphreys, David Raposo, Tobias Pohlen, Gregory Thornton, Rachita Chhaparia, Alistair Muldal, Josh Abramson, Petko Georgiev, Adam Santoro, and Timothy Lillicrap. A data-driven approach for learning to control computers. In International Conference on Machine Learning, pp. 9482- PMLR, 2022.
Sheng Jia, Jamie Ryan Kiros, and Jimmy Ba. DOM-q-NET: Grounded RL on structured lan- guage. In International Conference on Learning Representations, 2019. URL https:// openreview wttpriiwwweeeeeeeee www.ooorummmdndddllid..d..
Chuck Jorgensen and Kim Binsted. Web browser control using emg based sub vocal speech recog- nition. In Proceedings of the 38th Annual Hawaii International Conference on System Sciences, pp. 294c-294c. IEEE, 2005.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.0836 2020.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Sori- cut. Albert: A lite bert for self- self-supervised learning of language representations. In International Conference Learning Representations, 2020.
11

---

Under review as a conference paper at ICLR 2022
Chenliang Li, Bin Bi, Ming Yan, Wei Wang. Songfang Huang, Fei Huang. and Luo Si. Structurallm: Structural pre-training for form understanding. arXiv preprint 1ttv:21051111000, 2021a.
Junlong Li, Yiheng Xu, Lei Cui, and Furu Wei. Markuplm: Pre-training of text and markup language for visually-rich document understanding. arXiv preprint 10.iv:21 2021b.
Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang. Reinforcement learning on web interfaces using workflow-guided exploration. arXiv preprint arXiv: 1802.0882, 2018.
Kevin Lu, Aditya Grover, Picter Abbeel, and Igor Mordatch. Pretrained transformers universal computation engines. arXiv preprint arXiv:2103.0522 2021.
Sahisnu Mazumder and Oriana Riva. Flin: A flexible natural language interface for web navigation. arXiv preprint arXiv:2010. 2020.
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christo- pher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332. 2021.
Rodrigo Nogueira and Kyunghyun Cho. End-to-end goal-driven web navigation. Advances in neural information processing systems, 29, 2016.
Christopher Olston, Mare Najork, et al. Web crawling. Foundations and Trends in Information Retrieval, 4(3):155-244 2010.
Panupong Pasupat, Tian-Shun Jiang, Evan Zheran Liu, Kelvin Guu, and Percy Liang. Mapping natural language commands to web elements. arXiv preprint arXiv:1808.09) 091880088011,... 2018.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, al. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., (1((11)))
(140): 106, 2020.
Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Perey Liang, World of bits: An open-domain platform for web-based agents. In International Conference on Machine Learning, pp. 3135-3144. PMIR, 2017.
Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, and Denny Zhou Mobile- BERT: a compact task-agnostic BERT for resource-limited devices. In Proceedings of the 58th Annual Meeting the Association for Computational Linguisties. Association for Computational Linguistics, 2020.
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tza -zee Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Mare Pickett, Kathleen S. Meier- Hellstern, Meredith Ringel Morris, Tulsce Doshi, Renelito Delos Santos, Toju Duke, Johnny So- raker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera-Areas, Claire Cui, Marian Croak, Ed H. Chi, and Quoc Le. Lamda: Language models for dialog applications. CoRR, 2022.
Daniel Toyama, Philippe Hamel, Anita Gergely, Gheorghe Comanici, Amelia Glaese, Zafarali Ahmed, Tyler Jackson, Shibl Mourad, and Doina Precup. Androidenv: a reinforcement learn- ing platform for android. arXiv preprint arXiv:2 13231, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- fion processing systems, 30, 2017.
12

---

Under review as a conference paper at ICLR 2022
HTML
expanddable:
wripoet
orpand one level
Sotherwise OUEGUT
elements
generation on
rubates
up
Figure 5: High-level overview of our pre- processing pipeline for generating snippets from full HTML web- page. Given the page, we detect salient elements and for each one of them we extract snippets by recursively moving up in the HTML tree until validation hcuristic fails.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. Huggingface's transformers: State-of-the-art natural language processing. CoRR, abs/1910.03771, 2019. URL
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami AI-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mt5: massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010. 1934, 2020.
Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real- world web interaction with grounded language agents. arXiv preprint arXiv:2207.01206, 2022.
A APPENDIX
A.1 DATASET DETAIL
Examining the description distribution, we found the original 400K dataset to be very skewed; only 20 descriptions (such Email and Password) were covering 50% of the dataset. We sub-sampled the dataset so that each unique description has at most 10 data points. We also found that for attributes are almost always defined for HTML labels. This could cause a model overfit and just find the 1 label element in the HTML and ignore everything else. To avoid this sort of 'cheating' we replace the tags of HTML labels by randomly sampling from fdiv, span, a, label}. These tags are also frequently used to inject text in HTML but they are very rarely used with for attributes. Finally, we removed examples where there are only single text in the HTML since models can trivially generate descriptions by finding the only text in the HTML, which biases model weights and evaluation metrics. After this final step, we have a total of 85K labeled examples.
A.1.1 SNIPPET GENERATION
In Figure 5 we give a high-level overview of our snippet generation procedure.
A.2
ADDITIONAL RESULTS
A.2.1 SEMANTIC CLASSIFICATION
Error Analysis. We manually examined 50 errors of T5-3B model over the development set (Ta- 4 and assigned them into one of the 9 error types that we devised. We found that 32% of the errors are due to lack of information in the HTML snippets, which is mainly the result of lost in- formation during snippet extraction process. Annotation errors or email/username ambiguity make up 30% of the errors. These can't 't be improved without revising the annotated data or adding extra information to resolve the ambiguity. We also found that the model sometimes picks a more general category, or a nearby text misleads the model; the latter usually happens when the HTML snippet is long where majority of the elements are noise.
13

---

Under review as a conference paper at ICLR 2022
Error Type
Percentage Examples
Not enough information in the HTML snippet
30
Incorrect annotation (ex: "unknown_rve role" instead of "organization")
12
Annotation tool translates user selection incorrectly
Email/Username ambiguity
10
More general category (ex: "header" instead of "cart_header"
Immediate neighboring text misleads
8
Incorrect date formatting (ex: "mm" instead of "mmm") No information in the HTML snippet
2
Others
18
Table 4: Types of errors over 50 manually examined examples. 32% of errors are due to lack of information in HTML snippets, 30% of errors related to annotations or can't It be improved due to ambiguity (email/user- name), and the remaining errors are incorrect predictions the model.
Few-Shot Prompting In Table we present few-shot prompting performance of 540B PaLM model. We probe the model using prompt template <html> Role: <category> with ex- ample per category and generate categories using we found that few-shot prompting achieves only 45.6 accuracy, much lower than model fine-tuned on the same data (F) We found two common problems the model not able canonicalize predictions into categories and many of the examples dropped due context length.
In our preliminary experiments,
We developed post-processing methods to al- leviate the canonicalization problem and pre- processing methods to reduce lengths of ex- amples. Adding dictionary-bast based mapping on predictions a manually curated paraphrase dictionary improves the performance 52. We also tried rewriting predictions by chang- ing the order of tokens around "_" such as name first to first name which further improved 57.9. Finally, we cleaned examples in the prompt by removing certain el- ements such as "svg", "path", ", "img". and "iframe" and also removing class attribute from every element; this pre-processing step gives 64.2.
Model Name
Test
Dev
PaLM-540B
64.2
60.3
w/o Example Cleaning
57.9
57.2
w/o Category Rewriting
52.1
50.7
w/o Dictionary Mapping
45.6
45.1
Table Few-shot prompting performance with differ-
ent pre- and post-processing steps.
the performance
Figure 6: Performance comparison w.r.t. increasing model size. As the model size increases, we observe increase in overall accuracy with PaLM-62B model achieving the highest accuracy while being 7x larger than PaLM-8B.
100
dev encoder
test encoder
test fit dev fit
dev enc-dec
x test enc-dec
dev decoder
x test decoder
Accuracy (%)
90
X
85
x
80
75
10a
10a
10*
Model Size (in millions)
14

---

Under review as a conference paper at ICLR 2022
A.3 SAMPLE EPISODES FROM MINIWOB
See Table 6 for an example episode of web navigation inferred by fine-tuned LLM.
A.4 DETAILED MINIWOB RESULTS
See Table 7 for detailed performance of various models on Mini Wob.
A.5
RESOURCE REQUIREMENTS
See Table g
A.6 STRUCTURE DEPENDENCE ABLATION STUDY
We conducted an ablation study to examine the sensitivity of model performance to preserving structural information. To do so, we evaluate the model's performance on HTML imput with criti- cal structure components removed. We kept the order of elements and their attributes fixed while corrupting the nesting structure by removing closing tags.
Removing closing tags corresponds to a valid traversal (BFS) and keeps the order of elements the same as the text based input.
As simple example:
<div id=" " =d
would be converted into:
<div id= "d==ffrrm>><>>> "dm==xiiiiixix id= "d="username">>
We evaluated the trained WebN-T5-3B model on the same set of synthetic websites from the MiniWoB benchmark with this aspect of structure removed from the HTMI, pages. WebN-T5- 3B achieves a 45.4% success rate, 6% lower than before, suggesting that WebN-T5-3B is at least partially dependent on the DOM topology.
A.7 TASK-SPECIFIC MODELS
An alternative to LLMs is to adapt bespoke task-specific architectures tailored towards processing of structured documents and HTML (Li et al. (2021b]a).
StructuralLM (Li et al. (221aa) is an approach specifically tailored for document understanding (i.e.. combinations of images and text), and thus makes several simplifying assumptions for its model that limit its applicability to HTML understanding (i.c., trees elements with a richer structure and functionality). It is trained only the textual content of document the markup information is ignored. For example, any input field or dropdown in a document would be missing from the model inputs. All of the tasks we study require knowledge of this information. For example, in autonomous navigation the model needs to interact with input elements (e.g. text, checkboxes, dropdowns) such as username and password in the login-user task MiniWoB. Typically, "type" action with reference to an element and a text argument is generated by the model. Without knowing which input elements are available in the page, it impossible generate reference to any input element.
While MarkupLM (Li et al. (2021b) is better tailored for understanding HTML pages, it has similar that it focuses solely on text and structure of text while ignoring
drawbacks as Structurall.N everything else in the markup. To illustrate our point better, we used the open source implementation of MarkupLM from the HuggingFace library (Wolf et al. (2019) to process the sample HTML snippet in Ib) The MarkupLM ignores all input elements, both username and password, and generates <>> Email AddressEnter Password: text input the MarkupLM Transformer. Classifying this text username or password is not possible without the additional context on which input element is the salient element (in this context is the username). See helow for the code reproduce result.
enter your password. 
 which is the
15

---

Under review as a conference paper at ICLR 2022
from transformers import MarkupLMProcessor
processor =
snippet <div><label f "orruaaa"mi Address
</label><label class="form-labbs" for="pass">Enter Password:
</label>>/ddxxxxdvi>ddrupu type=* id=="Name" target><<nput
type="password" id- "pass"><ppan class "iads="niddeen"PPPesss enter your password.
</span></div>>''
encoding = prrcessorr
print (processor. htwpraaattccccchccdeeeeeeeeeeooe (encoding (wtroddiingiiunn........ ("input_ide"]]
Ma is also evaluated on NLP-like tasks such as QA or entity classification where understand- ing page content is paramount, whereas we focus on HTML understanding tasks such as autonomous navigation where both content and the page's ' layout structure need to be understood.
We perform quantitative evaluation of MarkupLM on our tasks to understand how significant these limitations are. We fine-tune the MarkupLM-baase model on the semantic classification task, using the same setup other WebC models but with the suggested hyperparameters from (Li al. (2021b). We use the MarkupLM implementation from the HuggingFace library (Wolf et al. (2099). On development and test sets, Markupl MM-base achieves 65% and 66% accuracy, respectively. These results are more than 16% lower compared to similar size WebC-BERT-bass results that we report in our work. This suggests that although domain specific models may be suitable for processing HTML for NLP tasks, the generality, flexibility, and sample efficiency LLMs provide advantages for autonomous navigation tasks.
16

---

Under review as a conference paper at ICLR 2022
Table 6: A sample web page and corresponding episode using the T5-3B model. At each time step, previous actions, instruction, and HTML are concatenated into single HTML text. Note that at the beginning of episode, there no past actions and we simply concatenate instruction and HTML. Action is generated as sequence of tokens which is later parsed into dictionary. The ref in the action points to an element that has ref attribute with the same value. For instance, at the beginning of episode, ref: 6 corresponds to an input with ref=6. At the end of the episode, the model clicks on the submit button and the episode terminates.
Web page
Select cwe, mMa YVMMONa, DUKG, 1532 and Submit
Qw
mMpY
YVMMOVV
DURG
113Z
Submit
HTML Text
Action Text
Select qu, sMp,, YVYMVVa, DJFG, 1138 and click Subnit. <body ">cdvvv
id- id="wrap" ref-i2"><div id-"ares" ref="3"><div id="boxes" "scaaee> ref="5"> <input "checkboo id-"ch0" ref="6" "False" "s</inppttt
class- "TEXT\ CLASS" ref="Bone" "tpp///><//abbe>>iaaae type="checkbox" id="ch1" ref="s" value value-"Palse" rat="None"
ref-"?'s ">><ipput
class "TEXT) CLASS"
raf="9 ">"icppt type="cbeckal id="ch2"
ref="10" "alse="aale"><//ine
class- "EXTT) CLASS" ref- "oEe"Non"">en "//p"//>
</labei><label ref="11"><inpu" type- "checkbox" id="ch3' ref-"12" value- "False">
</ingut><t class-"Txx"" .CCASS' ref="None" >VVmoVa httee//tt/abbbiaaaaseea ref='13'>
<input type="checkber "checkbox" id-"chd' ref-"14" value- 'Talee" >></inpu>><t
class='IExT CLASS" rene"Bone" />D/f/////////////////////////// tt></abbe><saabbe ref-ntt"><input type "cheskboo"
value- "False" "ee/>>>/npppt class "TEXT', CLASS" oef='None">>) 1113<</t>//aabl><///// <biton id="subbtn' class-"secondee "secondary-action:
/wwp///////////////////// wttps/////////////////// /body>
faction: click, ref: 6}
(action: click, ref: Select qu, stp,, vingya, DJEG, 1231 and click Subait. <body ref="1"><div 'd='vrap" ref= bef="2"><div id id="area" ref="3" "3'x<div id= "boxe" ref="4 "ot='e">cabbel ref='5"><inpo <icinutt type= "checkboo" id="cho" rete'6" "True>" </input><<t "haxs="TExY\_CC CCLASS" ref- "None""o"""" "None""qu<//"""" "aaeaiigg//////aa/aa///a /bbbbabbaaaaabbe "txinut type-"checkbon id='chl' ref-"8" value- "Yalue" >></ngttt<t class- CLASS"
ref-"None" =ef="Kone">MlfDDnbd htllf//dk//////aaeeea>>///
sypee"checkbor id""ch2"
ref-"10" value='False >></inpt>>< class- "TEXT), CLASS" "None" ">Mpp<</t>>>
</label><label
"cheokbox" id-"ch3' ref-"12" value='False" "False">
</input><t class "EETT), CLASS" ref "None" wwtp//////<</aaaae>>>>>
"13'
<input type= "checkbox" id="ch4" ref="14" "alse=" >><ng>>><</nntt>
clasa- CCASS"
"ype="checkbox" id='ch5' ref="16" value= "alueeFallle"eee <</inpu><>> class- "reee"reeeeee CLASS*
id-" "subbta" class-" "eean=""ccoddryaacll Sunmii"""nitt///uutttti /dioe////vv/////////////// divs> /bddy>
{action: click, ref: 10}
(action: click, ref: 6), (action: click, refi 10) Select qu, mip,, IVINQUa, DJEC, 1135 click Subnit. <body ref="1 1'sdiiv id="vrap" zef="2"><div id "area" refxi3">< id="boxes" "efx"a""<labe" "ef="5"><input
"checkobto id="cho" ref="6" value= >></nput>>t class "TEXT) CLASS" ref= /atnaa/ggg////laaabaaaba <label ref="7" ref=-n"<<ipput type=" checkbox" id="ch1" ref="g" value= "False"><//i >>/inptt>>>>>>> <t clans-"TEXY? CLASS" ref= "ooe""Non""" Mon heffbnnt//><</abbbb>ee
ref- "iscintts
"checkoop" id="ch2" reé="10" value- "True" "></ "ouuueeS//nninnee ingut><t
(1877) CLASS*
ref-"None" Noe""Nooe'"MyV"" /MMy<</eoc/aabbb<labbe
"<1"ncipput type- "checkbox" id""ch3"
ref-"12" value="ralse" "></i>ptt></ class- "TEXT) CLASS" ref- "Hoee <>vrngas//>>>
</label><label "ef="13"><inpu" type- "checkbox" id="ch4" ref-"14" value- "Palse"s
</input><t "lasT=" CLASS" ref="Mone" boff://></laaee><aaaae
ref "15*s
kinput sype="checkbox id='ch5" ref="16" value= "<<inptt==///
class="EEXT), CLASS' ref- "None" ">113<</1>>//00 /tt/////aaaab/////eeeaa class= "sccondary- ref="17" Stbniisbbt///////
id= "abbttt"
faction: click. ref: 12)
(action: click, ref: (action: click, ref: 10), (action: click, refi 12) Select gw, zMpy, YVYиgVa, DJIC, 1132 and click Subnit. <body " idr'wrap" ref="2"><div id="area" "3"xdvv id- "bxeexxs" ref="4"x<labe typa= "cho" ref="6" value- "True" >></ingu>><t class- "TEXT), CLASS*
ref="None'se Nef="Nonn">qe</bee wttgg///////bbb////e/ee
ref=""">cinput type-"cheekbox" "checboox" id""ch1" ref-"g"
value="Yalse" "elee"<////pueee cless- "TXX7) CLASS" ref- "None"
ref- tef-"9"><input "cheokboo" "ch2" ref="10" value-"frue "></input><t
class "TEXTV CLASS" ref- "oon"" wagpn//og/////oo wwtgg///>///a/elll/>//a/// <>><//abe><<aabe
type-"checkbox id""oh3" ref="12" value- "Frue" "></ ></input><t class "TEXT) CLASS"
ref ref="None" "e=="Moee=<VTNNOe ceaeeerrrooaa////////// /ttpoo////b//abbbblaaaaal ref= def= "ii"xiputtt
"checkooo cd="ch4"
ref="14" value- "False ></lpput>tt class="VEXT) CLASS" ref- "None"oone"ee DJEC</t>>>>
</labeD><abell ref= "1*tppputt type="checkbon "checkbox" id="ch5' ref="16" value= "False">
</inpat><t "EXT, CLASS" ret= hed="Kone>>>13e 1atee/i/33///////"/ http://////bbbb/////ee/aaa
"du'uubbea" class-" seconddry--ctionn ref="17" Sttr'111""Suum//////" http:///ttttttt/oa//////////////
{action: click, ref: 14)

---

Under review as a conference paper at ICLR 2022
(action: click, ref: (action: click, ref: (action: click, ref: 12), (actions click, ref: 14) Select qu, xMpY, WYMgVa, DJEG, 1138 and click Subnit. <body ref="]"><div id-"wrap" ref-"2"><div id- ide"aren" ref-"3"><div id="boxes" ref="4"><lale ref="s"scinput type-
id-"cho" ref-"6" value='True"> </input><t "EXX"" CLASS" ref-"Mone httpg//s////be////////ee////
ref-" "i<iiputt
type="cheoken checkbox" id- "c"""hh" ref="g" value value='False Taalse"c//npppsa
class "ZXXT) CLASS"
ref baafbee</t><//abbe
ref="9"><ia
type- "checkboxx id-"ch2"
ref="10" value= "True" "oaeee"""eee"e/utttttteeee <</input><t class "IEXT) CLASS" ref="Bore" "Bope'>aMpp />tpp<//>><//// L
<label ret=iii"><iapot type= "checkooo id="ch3" ref="12" value= "True" >></nppt>>>>
<t class='rExP), CLASS" sef="None www.o////////e/eo wVNoua//b>//aabbiiiiaa
"ofinit"><
"checkcchckooo" id- "ch4" ref="14" value="Prue" >></inpb><t elasg-"QEXT! CLASS
ref- "Nonn' "Nonn">affo<t>></ /tte///><//abbb>>iabbii ref- "it=-iut sype-"checkbos id""ch5"
></input>ct class-
ref-
</Label<</dio>-butttee id-"subbtn" class- "secondaty--cction" ref- 117">Summit hioye/
faction: click, ref: 16)
(action: click, ref: 6), (action click, ref: (action: click, ref: 12), (action: click, ref: 14), (action: click, ref: 16) Select qu, nMp,, YVYMOVa, DJÉG, 1135 and click Sabmit. <body pef-"1"s<dis id-"vrap" ref-"2"><div" id-'ared ref="3"><div "boxes"
"tcblabbel ref- "St=inu" type="cheelle "checkbox" id-"ch0" reč-"6" value- "Zrue" "></inpat><t class "TEXT), CLASS' ref "None" ">qus//></labbe
<label red="""><inppt type- "checkbox" id-'chl" ref-"3" value= "False" ></nnptt>< class- CLASS" ref= "ofe"None" hBffoa///t></aaee>>>ee
>ef="""><input
type= type="checcbox" id="ch2" ref="10" value="
value="Frue "Frue >></ippu>><t class "TXX7) CLASS
vet="Hone"" "otone""///7//// (ateek//s//aabe>iiaaa
ref- "il'><input type- "checkbox" id="ch3"
ref="12" value='Frue"'a >//e""<//a <oou=>><</nnp>>tt class='TEXTT CLASS" ref= ref='Nion" "Konn" WVTgWa///>>>>> "
</labei><label ref- "af="13"><iaput "cpec"checkbox" id="ch4" ref="14" value- "frue"
</input><t claxs='TEXP\, CLASS" ref="Kone" -tff////////bbbbbaaaaa
ref="15'>cinpu
"ype-"checkan id- ref="16" value="Frue "Frue"></innub>>> ">/iigut><< class-"1RXY) CLASS"
pef="None" 1<33<>>11/<//>><//11
http://t///abbb////eeaaeeeeaaaet heoll///////va/oo/ divoobbtttonn id="subbta" class-'s class="seconde "secondary-action
{action: click, ref: 17)
18

---

Under review as a conference paper at ICLR 2022
Table 7: Success rate comparison of various models MiniWoB tasks. Baseline results are borrowed from
Humphreys et al. 2022 Note that these are normalized between O and 1.
Lames
DOMOONS
LeunEE sAta (Nugraske
AGgPORAE A
Agg-aREZ (hegreend)
WAM8-1388
WebN-1S-1 (anlerrg))
ISL MILY
(SL)
1tAks equanaioon
Fatol expleratin
excigux
IHL.
boll Hight
113
035
33
23
122
119
0125
100
HEE new Denan Viga
11:1
106
2-naiin
661
0.56
606
oren patt-3
645
0.55
0.56
0.6
0.76
dagals
WEST dal
WEST WESTERECCE
102
100
123
WAT
Toon EeT
102
mee lile
307
Hasdhengee
sieple didta
wealinndee
arrrzal
596
566
0.99
19

---

Under review as a conference paper at ICLR 2022
Table 8: Resource requirements and running time L.Mss.
Model Name
TPU version
Batch size
Input sequence length
Examples per sec (training)
Examples per sec (inference)
Pal
62B
TPU v4
1920
9.313
30.51
TS
3B
TPU v4
128
512
163.8
734.5
LaMDA
TPU
1416
20


Reviews:
[{"review_id": "GH-g3558JF", "reviewer": "Reviewer_5MM1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work addresses the problem of using large language models for understanding HTML. Unlike prior work which attempt to solve this problem using dedicated architectures and training procedures and/or large HTML corpora, this work employs large language models pretrained on natural language text and evaluates their performance on three HTML understanding tasks - Semantic Classification of HTML elements, Description Generation for HTML inputs, and Autonomous Web Navigation of HTML pages, thus potentially eliminating the need for dedicated architectures and training procedures. Further, using only a small HTML corpus for finetuning a pretrained LM, the work reports encouraging results compared to LMs trained exclusively on the task dataset.\n\n\nThe key question asked by this work is can off-the-shelf LLM trained on a large text corpus be used in tasks that require some level of understanding of HTML. As canonical tasks in HTML understanding, the work looks at three tasks. \n\nIn Semantic Classification, the ask from the model is to classify a salient HTML element into one of a set of role categories that are commonly used in automated form-filling applications. E.g. address, email, password. \n\nIn Description Generation, the ask from the model is to, given a HTML snippet as the input, extract a small text sequence from the snippet as the natural language description of the snippet. \n\nIn Autonomous Web Navigation, the ask from the model is to, given a HTML page and a natural language command as the input, identify the appropriate HTML elements and the actions on those elements that would satisfy the command. \n\n\nThe work tests the idea of using pre-trained LLM for the three canonical tasks with several pretrained LLMs with different architecture  encoder-only, encoder-decoder, or decoder-only, different model size, and training data. Best results are obtained with encoder-decoder architectures with bi-directional attention.\n\nThe input to the models is the raw HTML text sequence. However, when the sequence is too big to fit into the context window of LLM, a snippet of appropriate size is extracted using a heuristic algorithm.\n\nThe work uses MiniWoB benchmark (demonstrations like email forwarding and social media interactions) for Autonomous Web Navigation task, a new dataset consisting of URLs from the real shopping websites for Semantic Classification, and a dataset derived from CommonCrawl for Description Generation.\n\n", "summary_of_the_review": "This work asks the question can off-the-shelf LLM trained on natural language text be used effectively for tasks that involve HTML pages. It proposes three tasks as canonical tasks in understanding HTML. It employs a variety of LLM to build models for the three tasks using a small amount of HTML data for fine tuning. It shows that LLM does help these tasks significantly. One key question not answered in this context is how much of HTML structure analysis and understanding is truly required for these questions. ", "strength_and_weaknesses": "Strengths:\n\t1. That pre-trained natural language LLM can be effective for tasks involving HTML pages  is interesting and can potentially find use in several interesting practical applications.\n\t2. As no retraining of LLM with large HTML datasets is necessary, models for tasks  involving HTML pages can be developed quickly and less expensively. \n\t3. That raw HTML text can be used as input without needing parsing is an advantage.\n        4. Experimental results are very encouraging and validate the claim that pretrained LLMs can be effective for the three tasks.\n\nWeaknesses:\n        1. It is claimed that these three tasks require understanding of both structure and content of the web-page. While it is easy to see that textual content plays a key role in each of the three tasks, the role played by the structure of the web-page is not clear. It can be argued that no significant HTML structure analysis or understanding is needed for these tasks. For example, in Semantic Classification, what is most important for classifying HTML element 'input' into, say, 'username' is the value of its two attributes,  'type' and 'id'. As these attributes are in the close neighbourhood of 'input', parsing of HTML is not strictly necessary. Therefore, it might a good idea to do some experiments that demonstrate unequivocally the need for HTML structure analysis or understanding in these tasks. One such experiment could be to map all HTML tags in the web-page except the salient tags to the same token (say, ***) so that the input is now a sequence of salient tags, and ***. \n      2. There is not much novelty in the methodological aspects of the work. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Fgd1GEiQZfD", "reviewer": "Reviewer_wKTj", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper investigates whether LLMs can be applied to HTML understanding to produce better-performing, more sample-efficient HTML understanding models without the need for custom NN architecture design. To achieve this goal, the authors present a suite of three benchmarking tasks for HTML understanding: (1) Semantic Classification of HTML elements, (2) Description Generation for HTML inputs, and (3) Autonomous Web Navigation of HTML pages. The authors find that a) pretraining is critical for the performance and can reduce labeled data requirements / improve sample efficiency up to 200x; b) model architecture is the second most important factor, and T5 models with bi-directional attention and encoder-decoder architecture performs the best across the board; c) given a choice, model size should be evaluated in the context of the models training and inference performance, as the model size sub-linearly predicts its performance. ", "summary_of_the_review": "In my opinion, the audience could gain insights from the thorough empirical analysis of HTML understanding from this paper. However, the lack of technical novelty & contribution makes this work slightly below the acceptance bar of the conference. Overall, I recommend a weak reject. ", "strength_and_weaknesses": "Strengths:\n1. The paper is well-written and easy to follow.\n2. The authors conduct comprehensive evaluations and analyses over a range of architectures, dataset sizes, and baselines to make the experimental conclusion convincing.\n3. The authors create and open-source a new dataset/benchmark for HTML understanding.\n\nWeaknesses:\n1. The novelty is primarily empirical, and the technical approach is essentially an application of known techniques (the standard pretrain-finetune paradigm of LLMs).\n2. I examined the supplementary material provided by the authors and only found the data files. It would be better if there were codes to prove its reproducibility.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "4KEhfqKJDLi", "reviewer": "Reviewer_hMUi", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies how pre-trained large language models (LLMs) perform on three HTML understanding tasks: (1) semantic classification of HTML elements, (2) description generation for HTML inputs and (3) autonomous web navigation of HTML pages. It found that pre-trained LLMs can work on these tasks effectively after fine-tuning. They require less task-specific data while perform better than the previous best supervised model.", "summary_of_the_review": "I like the idea of studying LLMs on HTML texts. However, it seems to me that the experiment setup (re. models to compare against) cannot support the main claim of why one would like to do so. Besides, the technical novelty of the paper is limited for a ML conference like ICLR; a NLP conference might be a better fit for this submission.", "strength_and_weaknesses": "## Strength\n- The paper is easy to follow.\n- The study of how LLMs perform on raw HTML texts is interesting and novel.\n\n## Weaknesses\n- Models that are trained on task specific data to compare against is questionable. I expect to see more task-specific models such as StructuralLM [1], MarkupLM [2] that respect the tree structure of HTML (or some other methods mentioned in the paper) to be evaluated against. However, the results seem to focus on LLMs only. I don't see the point of compare different variants of LLMs with or without pre-training. Basically, I need to see the following question answered: \"For specific web tasks, do I want to use a HTML specific model trained on the small dataset or just to fine-tune LLMs\"?\n\n[1] Li, C., Bi, B., Yan, M., Wang, W., Huang, S., Huang, F. and Si, L., 2021. StructuralLM: Structural Pre-training for Form Understanding. arXiv preprint arXiv:2105.11210.\n[1] Li, J., Xu, Y., Cui, L. and Wei, F., 2021. MarkupLM: Pre-training of text and markup language for visually-rich document understanding. arXiv preprint arXiv:2110.08518.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "41HQMZzi4Tq", "reviewer": "Reviewer_d1QN", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes two main contributions to the direction of HTML modeling/understanding:\n\n1. A new task and dataset, **Description Generation**, derived from CommonCrawl with 85K (HTML, element, description) tuples. \n\n2. Experiments across a suite of three HTML tasks (Description Generation, semantic classification from Gur et al., MiniWob web navigation), investigating the importance of pre-training corpora (natural language and/or HTML), model architecture (encoder-only, decoder-only, encoder-decoder), the scaling effect, and so on. The main finding is the effectiveness of large language models (LLM) pre-trained on natural language and fine-tuned on HTML task data, with T5-like encoder-decoder architecture being the best.", "summary_of_the_review": "I believe the paper makes solid empirical contributions by introducing a new task, and studying many LLM variants on three HTML tasks, with some insights about what's important for modeling HTML. My main concern is that the new task is not clearly justified, and the modeling part might be too standard -- there is no HTML-specific model design involved, just studying basic LLMs. I think the paper can benefit from more evidence about the value of the new benchmark (e.g. what new insights/modeling capabilities it enables), or more interesting modeling.", "strength_and_weaknesses": "Pros: The direction of HTML understanding is increasingly important, as recent work has started to show the promise of utilizing the semantics of HTML and web navigation. The paper adds solid contributions to this direction by creating a novel and interesting task, and conducting many empirical studies with insights about architecture, (sub-linear) scaling effect, importance of context window/action history, etc. \n\nCons: \n\n- On the proposed Description Generation task, authors only spend two paragraphs presenting results, where WebD-T55-3B can obtain 90.8% accuracy (exact string match??). Does it mean the task is too easy? How to justify it from the existing semantic classification task, as in Figure 1(a) they look similar? Why in Table 3, dev numbers are much lower than test set? I feel the new task/dataset needs more explanation and justification.\n\n- The modeling part lacks novelty, just basic pre-training + fine-tuning on LLMs, and seems there's no model design based on the features of HTML?\n\n- On MiniWoB, it's not very surprising that using a larger model with pre-training, you can beat smaller models from scratch. Instead, the fact that LLMs are worse than RL performances should be discussed more -- does it mean the task is bad, or RL is important (but it's hard to do RL for LLM)?\n\n- (Minor) In intro authors claim \"no processing of HTML\", but there is still some pre-processing?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject