SCRIBES: Web-Scale Script-Based Semi-Structured Data Extraction with Reinforcement Learning
Authors: Shicheng Liu
Kai Sun
Lisheng Fu
Xilun Chen
Xinyuan Zhang
Zhaojiang Lin
Rulin Shao
Yue Liu
Anuj Kumar
Wen-tau Yih
Xin Luna Dong
Venue: ICLR 2026 Poster (2026)
Keywords: Semi-Structured Data
Reinforcement Learning
Information Extraction
Primary area: foundation or frontier models, including LLMs

Abstract: Semi-structured content in HTML tables, lists, and infoboxes accounts for a substantial share of factual data on the web, yet the formatting complicates usage, and reliably extracting structured information from them remains challenging. Existing methods either lack generalization or are resource-intensive due to per-page LLM inference. In this paper, we introduce SCRIBES (**SCRI**pt-**B**ased Semi-Structured Content **E**xtraction at Web-**S**cale), a novel reinforcement learning framework that leverages layout similarity across webpages within the same site as a reward signal. Instead of processing each page individually, SCRIBES generates reusable extraction scripts that can be applied to groups of structurally similar webpages. Our approach further improves by iteratively training on synthetic annotations from in-the-wild CommonCrawl data. Experiments show that our approach outperforms strong baselines by over 13\% in script quality and boosts downstream question answering accuracy by more than 4\% for GPT-4o, enabling scalable and resource-efficient web information extraction.

Published as a conference paper at ICLR 2026
SCRIBES: WEB-SCALE SCRIPT-BASED SEMI- STRUCTURED DATA EXTRACTION WITH REINFORCE- MENT LEARNING
Shicheng Liu'; Kai Sun", Lisheng Fu', Xilun Chen', Xinyuan Zhang", Zhaojiang Lin", Rulin Shao Yue Liu?, Anuj Kumar?, Wen-tau Yih Xin Luna Dong
1 Stanford University, 3 FAIR at Meta, Univeessity of Washington
2 Meta Reality Labs,
shichenge cs. stanford.edu, sunkaicnemeta.co
ABSTRACT
Semi-structured content in HTML tables, lists, and infoboxes accounts for sub- stantial share of factual data on the web, yet the formatting complicates usage, and reliably extracting structured information from them remains challenging. Existing methods either lack generalization or are resource-intensive due to per- page LLM inference. this paper, we introduce SCRIBES (SCRIpt-Basse Semi-Structured Content Extraction at Web-Scale), novel reinforcement learn- ing framework that leverages layout similarity across webpages within the same site reward signal. Instead of processing each page individually, SCRIBES generates reusable extraction scripts that can be applied to groups of structurally similar webpages. Our approach further improves by iteratively training syn- thetic annotations from in-the-wild CommonCrawl data. Experiments show that our approach outperforms strong baselines by over 13% in script quality and boosts downstream question answering accuracy by more than 4% for GPT-4o, enabling scalable and resource-efficient web information extraction
INTRODUCTION
A substantial volume of web data is stored in semi-structured formats such as HTML (Hyper- Text Markup Language) tables, lists, and infoboxes (Dong et al. 2014: Sun et al. 2025 Such
content offers rich source of factual information, yet its formatting complicates effective usage in downstream applications like question answering (Tan et al., 2025 Sun et al., 2025). Knowl- edge extraction aims transform such data from raw HTML into structured representations (e.g.. triples) (Wilks 1997). but despite decades of research, this remains a major challenge at large scale. Existing approaches fall into two main categories. Traditional information extraction (IE) meth- ods, such as wrapper induction (Kushmerick et al.) 1997). graph mining (Crescenzi et al., 2001 Liu et al., 2003), layout-based methods (Zhai & Liu 2005 Lockard al., 2018), and Deep Neural Networks (Dalvi et al. 2 Lockard et al.) 2020). tend to be brittle and struggle to generalize over unseen data or schema. More recently, Large Language Model (L.LM)-based methods have emerged that parse individual pages construct Knowledge Graphs (KGs) using large models (Gutiérrez Zhang & Sohl 20244] Ning et al., 2023) Chen & Bertozzi 2023 Zhang et al., 2023 Bai
ct al. 2024
al.
2025 Although these methods can produce high-quality outputs, they are eesource-intensie
to apply at scale because they require invoking an LLM for every page.
Can extract knowledge from semi-structured content at the web scale both effectively and efficiently? In this paper, we introduce SCRIBES: SCRIpt-Based Semi-Structured Content Extraction at Web-Scale, novel approach for large-scale knowledge extraction. Given webpage, SCRIBES leverages an LLM to generate an extraction script that applies to other pages within the same domain, which typically share highly similar layouts (Figure 2) Executing the script incurs
WWor done Meta
'Code will he released at https:: / //gi thub. com/ / www.aaaooooeeerreooooooooaaaaaae
HES See Appendix B for discussion of different types of webpages with semi-structured content.

---

Published as a conference paper at ICLR 2026
Training-time
One Webpage Input
Model Generates one Script per Group
HTML
Extraction Result
PEAR Deta, 2010-11-28
Dedup
Cenerate
Post
Apply to -
Iren eccrerr
P. 2510-38
trais eum
Reward Calculation In-Group Webpages
Weights Update
Human Annotation
Synthetic Annotation
Inference-time
4
Generalize Unseen Groups of Websites
(est_as/1.Be hank AVENNIS
HTML
Extraction Result
Generate
Psst
Apply to
P-0 C20050 Company,
teal 4n/5
LAN (MBUS)
Figure 1: SCRIBES organizes similar webpages into groups under each website. During training. the model receives one representative webpage per group as input (pt. 1) and tasked with generat- ing a single extraction script applicable all similar webpages within the group (pt. 2). Extraction results are then compared against human annotations for labeled data and synthetic annotations for unlabeled CommonCrawl webpages. The resulting scores used update the model weights (pt.
3). At inference time, SCRIBES enables the model to generalize to new, unseen websites by gener- ating scripts that can be applied across similar webpages (pt. 4).
only negligible resource cost compared with running an L.---dsed extraction on every individual page.
Although the idea appears straightforward, current LLMs struggle to produce high-quality, gener- alizable extraction scripts. Fine-tuning them for this ability is cumbersome, as creating annotations for such scripts difficult even for expert labelers. The success of SCRIBES lies in a Reinforce- ment Learning (RL) framework that leverages structural similarities across related webpages: given a group of similar webpages, the model is rewarded when script generated for one webpage also works others. This encourages learning scripts that generalize beyond individual examples.
SCRIBES draws training data from two sources. First, learns from a small set of annotated examples (192 pages from 34 groups) (Figure [1 parts 1-3). For each group, SCRIBES takes one webpage as input and prompts the model to generate a script intended to generalize across the group. The script is then executed the remaining pages, and its outputs are compared with annotations to compute the reward. Second, SCRIBES leverages in-the-wild websites from Common Craw/ to fur- ther enhance its capabilities. We develop iterative approach that starts from a checkpoint trained on annotated data and then refines the model to continue learning from their failed predictions on the in-the-wild websites. To provide supervision at scale, we employ LLMM-bsed direct extractions as synthetic annotations, reducing reliance annotations or hand-crafted parsers.
Extensive experiments show that our RL-trained model outperforms strong agentic baselines by more than 13% in generating robust, reusable parsing scripts. Moreover, we demonstrate that im- proved extraction translates into downstream benefits: in QA tasks requiring structured reasoning over HTML,, incorporating triples produced by SCRIBES boosts accuracy across wide range of LLMs, including SOTA models such as GPT-4o by over 4%.
2 RELATED WORKS
2.1 SEMI-STRUCTURED DATA PROCESSING
Flattening: In complex QA or retrieval settings that mix texts, tables, and knowledge bases, a common practice is to "Iinearize" everything into plain text (Oguz al. 2022 Zhang et al. 2024
2

---

Published as a conference paper at ICLR 2026
Ma et al. 2022
Christmann et al. 2022). This is also popular practice when dealing with HTML pages. Trafilatura is a widely used HTML cleaning and text extraction toolkit designed for large- scale web processing [Barbaresi 202T), among many other HTML conversion packages (Firecrawl 2025 Paraschiv 2024). While effective for general text extraction, these utilities typically discard flatten structural elements such as tables, lists, and infoboxes. Similar findings in complex QA that highlight the importance of structural cues (Liu et al., 2024b] Zhang et al., (2024), recent work on RAG with raw HTML shows that converting plain text discards headings, table structures, and other layout information critical for downstream tasks TTn et al.) 2025).
Traditional IE Methods: A classical approach to extracting structured data from semi-structured web content is wrapper induction, which learns extraction procedures ("wrappers") from small set of labeled examples instead of hand-crafted rules (Kushmerick et al., 1997). Extensions in- clude boosted wrapper induction, which combines simple patterns for greater robustness (Freitag & Kushmerick] 2000), and large-scale methods that handle noisy data and template drift (Dalvi al,) 2011). While effective regular site structures with clean annotations, these methods are brittle to structural changes and generalize poorly across diverse domains. In contrast, our approach learns HTML, allowing the system to generalize beyond fixed rules and adapt automatically without manual template design.
executable scripts, iee. full extraction programs that operate directly
LLM-based methods: Several recent advances utilize LLMs to extract semi-structured contents. For instance, Wang et al. (2025) train a LLM to convert HTMLs into Markdown and JSON us- ing SFT and RL methods. Similarly, Pozzanski al. (2025) use VLM to convert PDFs into clean, readable format retaining tabular structures, Many related works also exist on LLM-assisted knowledge-base construction (Gutiérrez al.. 2024 Zhang Soh, 2024] Ning et al.) 2023 Chen
& Bertozzi 2023 Zhang et 20233] [ai et al. (025). ). However, calling an LLM per page remains
resource-intensive at web-scale; moreover, they typically treat each page independently, missing the cross-page layout regularities that SCRIBES exploits.
2.2 RL WITHOUT ANNOTATIONS
A growing body of work explores reinforcement learning in settings without explicit annotations. Zuo et al. (2025) show that models can refine themselves at test time by turning consensus among rollouts into rewards, while et al. (2025) and Prabhudesai et al. (205)) demonstrate that internal signals such as self-certainty or confidence are sufficient to drive continued improvement. [Shao et al. (2025) find that even spurious random rewards can produce surprising gains, suggesting that models can bootstrap from imperfect signals. Like prior work, we reduce dependence on annotations by iteratively refining the model from its own failures, but instead of relying solely on internal signals, we utilize LLM-based direct extractions as synthetic annotation for reward calculation.
SCRIBES FRAMEWORK
3.1 PROBLEM DEFINITION
Figure 2: Three webpages containing semi-structured content under the same website.
Knowledge extraction: Let G {p..... . . pn } be group of semi-structured webpages that structurally similar. The knowledge extraction task parses each page p;, i E [1,n], to list of triples (subjects, predicates, and objects). We denote by yš, the ground truth triples for page pi.
3

---

Published as a conference paper at ICLR 2026
Extraction script generation: We propose to solve the knowledge extraction problem by gener- ating an extraction script that applies to every page in G. Formally, our goal is to train model LM that, given any webpage p G, predicts y to every page G generates triples close to ground truth triples (yp, pi E G). For instance, in Figure 2, model-generated script should robustly handle variations across webpages, such as differences in table sizes and values.
extraction script 9, LM (p), such that applying
3.2 HTML DEDUPLICATION (DEDUP)
The raw HTMLs of webpages are typically very long and can easily surpass the maximum context window of even the long-context LLMs. We propose simple yet effective method for deduplicating HTMLs: repeated HTML blocks are collapsed into a compact representation of the form "n more ... elements," which substantially reduces context length. Ablation experiments confirm that this deduplication step significantly improves model performance. We therefore apply throughout our SCRIBES -tarnned models. An example of the dedup process is shown in Figure 6 and further details and analysis are provided Appendix
3.3 RL SETUP
Annotating such extraction scripts for training is challenging even for expert human annotators. To address this, rather than relying on demonstrations, we propose adopting Reinforcement Learning with Verifiable Rewards (RLVR) for this task.
We define r(p - g) = 5( (g), vū) [0, 1] as the score obtained when the script Üp is executed on (possibly different) page q, where is a scoring function that measures similarity between predicted and annotated tuples. To compute this score, we follow prior works (Liu et al.) [024aa] Sun et al. 2025) and adopt bipartite matching algorithm that aligns predicted triples with gold triples by maximizing their pairwise fuzzy matching score. Based this matching,
com-
pute fuzzy precision pluzay, recall Rluzzy, and Fi score F.luzay Since fuzzy string similarity may fail to fully capture semantic equivalence, we additionally employ an LLM--as-a-uddge (set evaluate the aligned triples (Prompt [7). We choose Llama to ensure consistency with prior work (Sun et al. (2025) and, by fixing the checkpoint, to enable reproducible experiments. This yields LLM-based precision PLM recall RLM and Fi score L.LM
Llama-3 703- Insttruttt
During training, we set = F-furry the triple-level fuzzy F1 score. Refer to Appendix for addi- tional details on metrics and an optimized implementation of Ffuzzy during training.
3.3.1 REWARD SIGNAL FROM LABELED DATA
We define the following notations:
1. the self-score is (per(()) = r(p rpp-p). while
2. each cross-score is Peross (p r(p-+q) for q 7 p.
SCRIBES optimizes model using Group Relative Policy Optimization (GRPO) (Shao 2024) based on the following reward function for each training sample p:
rSCRIBEE (p)
Σ gEG(p)
r(p -)) rarlp) |Gp))-1 G(D]
Teross (p, ().q)
T
gEG(p).p7gg
Within this framework, each self-score contributes only [G(p] the final reward, while cross-scores constitute the majority of the reward signal. This design strongly encourages the model generalize
by accounting for potential variations across other, unseen webpages within the same group. We study the effect of different reward formulations through ablation studies in Section 4.4
3.3.2 REWARD SIGNAL FROM UNLABELED DATA IN THE WILD
When training on annotated data, SCRIBES can directly leverage the gold human annotation y for each page as the reward signal. However, because the only high-quality annotated dataset available from Sun et al. (2025) is relatively small, it is inherently difficult achieve broad coverage of diverse
4

---

Published as a conference paper at ICLR 2026
6
3
Retain
Group Webpages
lang = Filter
Blacklist Filters
CommonCrawl (Sample)
Size Groups
6
8
9
10
Retain >em % Semi-Structured
1: Train:Reward Data Creation
LLM Direct Extraction
Checkpoint
Training Data
Empty Prediction
Figure 3: Processing pipeline for unlabeled data from CommonCrawl in Section 3.3.2
website layouts using annotated data alone. To address this limitation, we propose novel approach that leverages unlabeled in-the-wild webpages from CommonCrawl (abbreviated as CC) (Common)
Crawl 2025).
Our data collection pipeline is illustrated in Figure 3 (pt. 1) Starting from a sample of CC, (pt. 2) we first apply the blacklist filters from Penedo et al. (2024) to remove adult or explicit content. (pt. 3) We then apply language filters to select English content websites and (pt. 4) group webpages by domain, (pt. 5) retaining only groups containing least webpages. (pt. 6) Next, use an LLM-based classifier (Prompt[[5) to identify webpages containing stri-struetured content, and we retain only those website groups where at least m% of the pages are classified as semi-structured. (pt. 7) Finally, we sample one webpage as the training example and associate it with up to k: S n in-group webpages for reward calculation. In our experiments, we apply the following thresholds: = 30, m = 90, and l: = 13.
At this stage, we obtain a collection of in-the-wild webpage groups containing semi-structured con- tent. However, without human annotations, it is unclear what reward signal should be used for train- ing. (pt. 8) To address this, we propose using LLM-based direct extraction (Prompt [6p as a proxy for gold annotations. Our experiments show this to be the strongest baseline. Nevertheless, because such direct extraction far from perfect (achieving only about 40% Fi for the best baseline), we aim to prevent noisy rewards from degrading model performance. (pt. 9) To this end, we start from checkpoint trained on annotated data and identify a subset of webpages where the model's predicted scripts fail to produce any results. By concentrating training these failure cases, increase the likelihood that the additional synthetic data improves the model's performance. Ablation studies on the necessity of this subset presented in Section 4.4
4 EXPERIMENTS
4.1 DATASET
Annotated dataset:
Existing datasets for semi-structured knowledge extraction from raw webpages
are limited. SemiBench (Sun et al., (025)) presents dataset of webpages drawn from 139 popular websites in CommonCrawl, annotated with triples. Their collection includes 83 websites with a single webpage, 46 groups of 3 similar webpages, and 10 groups of 13 similar webpages each. This grouping scheme provides a valuable opportunity to evaluate generalization in the SCRIBES setting. We select the 56 groups containing more than I webpage each for experiments this work. We divided the annotated dataset into training and test sets using a 60%-40% split across groups; that is, we assign entire groups either the training test set, and we do not split within any group. For group of size n in the training/test set, we create n training/test examples, each using one webpage as input and all group elements used for reward calculation. All evaluation metrics are reported on the test set, which contains only websites from groups that the model did not see during training. Refer additional details Appendix [
In-the-wild webpages: To construct groups directly from CommonCrawl, we employ simple heuristic: two webpages are grouped together if they share the same URL prefix up to the final substring. For example, example. com mttmmmmlssss// /slb and example foom/midll ssub2 be- long to the same group, while example com/ com/mide mid2 does not. The LLM used in our pipeline is GPT-OSS-120B We randomly sampled 50 webpages and estimated classifier accuracy at 90.0% precision and 72.0% recall. In total, 19,566 groups satisfied the n 2 30 condition, among which 2,003 also satisfied the > 90 condition. After direct extraction with the L.M,, 1,898 examples
5

---

Published as a conference paper at ICLR 2026
Model and Method
All
Example
Holdout
RLM
pLM
F
RL
PLM
FI
R
pLM
F
Baselines
(Direct L.M Extraction)
1.70B Sun 2025
24.3
15.7
19.1
Fine-tuned Sun 2025
21.4
27.1
23.9
GPT-4o (Sun
2025
35.1
23.8
28.3
Q-14B flatten
30.5
36.5
29.9
Q-32B flatten
28.7
37.4
29.9
GO-20B 2-shot flatten
33.2
47.1
34,9
GO-120B 2-shot flatten
42.3
46.3
40.4
Baselines (Script-gen)
Q-14B agentic-3-iter 2-shot
8.6
11.1
8.0
13.2
18.0
12.6
6.3
7.8
5.7
1.-00 agentic-3-iter
10.1
15.5
10.5
16.7
23.8
16.8
6.9
11.2
7.4
Q-72B agentic-3-iter 2-shot
16.4
19.4
15.0
24.1
28.6
21.8
13.3
15.8
12.4
Q-32B agentic-3-iter 2-shot
18.6
27.2
19.4
24.5
34.8
25.9
15.8
23.9
16.4
GO-20B agentic-3-iter
24.7
23.2
20.9
29.3
26.4
27.7
22.5
21.8
18.9
GPT-4o agentic-3-iter 2-shot
26.0
33.0
24.4
33.0
36.5
31.2
22.5
31.3
21.1
GO-120B agentic-3-iter 2-shot
33,9
41.0
34.3
35.8
42.3
36.6
33.0
40.5
33.3
SCRIBES
(Script-gen)
Q-14B
23.0
24.3
19.9
31.2
29.8
26.7
19.0
21.7
16.7
Q-14B (+ CC)
25.2
23.0
21.8
34,9
31.0
30.0
20.5
19.1
17.7
Q-32B
29.9
31.5
28.1
32.0
33,9
30.3
28.8
30.3
26.8
Q-32B (+ CC)
37.4
36.0
33.2
39.5
35.5
34.6
36.2
36.2
32.4
Table LLM-judged metrics are reported separately for All, Examples (the webpage model used to generate the script), and Holdout (similar webpages where the same script applied). Columns show macro-averaged PLM, RLLM, and FLM For each model and block, we report only the strongest baseline here. The full baseline results, including LLM-based agentic baselines Hip- PoRAG (Gutiérrez et al. 2024) and AutoSchemaKG (Bai et al., (025), which exhibit lower scores,
provided Table 11 Appendix G.4 (") Numbers reported by Sun al. (2025) the full set.
were retained (the remainder corresponding prediction failures or empty outputs). This entire process used less than 1% of the CC-MAIN-2025-30 crawl. We hypothesize that this pipeline can be scaled to larger portions of CommonCrawi for broader coverage; this paper, we focus on establishing its feasibility.
4.2 TRAINING SETUP AND BASELINES
Training We train Qwen2 5 5 Instruct family models and perform minimal hyperparameter tuning to ensure stability during model training. Refer to Appendix D for additional details.
Baselines We experiment with both SOTA close-source and open-source models, including: gpt 4o, L1ama-3. 7--00---- instruct (abbreviated as L-B0B), Qwen2. 2e 5-Inssttuct (abbre- viated Q-xB) family, and gpt-oss (abbreviated GO-xB) family. We implement the following baselines for comparison (99. By default, all baselines use Dedup as the SCRIBES E--tain-t models. We explore multiple configurations to construct strong baseline models.
1. agentic-n-ited After the model outputs script given example, if the script fails to produce output or produces empty output, we feed the execution feedback to the model and the output script prediction. We repeat this ReAct-
ask it retry. Otherwise style Yao et al., (022) procedure up to n times;
2. n-shot: We feed in n HTMLs and their corresponding gold extraction results as in-context learning examples;
3. flatten:
We directly flatten the HTMI and use it as model's input. Note that there is no generalizability requirement or dedup involved in this setup.
4. Recent, SOTA LLM-based KG construction pipelines, including HippoRAG (Gutiérrez) et 2024 and AutoSchemaKG (Bai al. 2025). See Section G. for details.
Beaut ful Soup (htm1 content, #htm1 .parser") get text

---

Published as a conference paper at ICLR 2026
4.3 RESULTS
RQI: Does SCRIBES framework bring improvements to models in terms of their capability to extract semi-structured data?
For each example in our test set, models generate a script Dp = (p) and we apply to all examples Gpp). We derive score
S(p) = G(())
Σ
S(Ops Vg))
(2)
gEG(p)
where we set to be recall, precision, or F1 score, defined in Section 3.3
We refer this
aggregate score as "All." To further investigate the performance gap between the example provided to the model ("Example") and the other webpages to which the model-generated script is applied ("Holdout"), we decompose the score in Eq. 2 into two separate components:
Sexamp (p) S(p:: v5))
Sholdout (p) -
E
S(0g V7)
@EG(D), opp
In Table we report the macro average of RLM pLM FI-M by averaging individual Sp) scores.
SCRIBES Se-EES-traiie -tained models drastically outperform strong agentic baselines. The best Q-14B and Q- 32B models outperform the few-shot agentic base model performance by 13.8% in FLM and our best Q-32B model performs on-par with the few-shot agentic GO-120B model.
RQ2: Does using SCRIBES enable eesource-efficieee web-scale extraction?
To demonstrate the --ram-----rss applicabiliity to web-scale semi-structured con- tent extraction, evaluate
Estimated FLOPs Webpeges, Q-32B
1024
St310s5, ke1
leftover sub- set of CommonCrawl data that was not used in model training. To keep the experiment tractable, we capped each group at 30 webpages and required least 13 webpages per group, meaning this evaluation covers only a tiny frac- tion of the available data. On this small subset with 113,129 webpages, our model extracted 2,788,760 triples. Remarkably, only 4,661 re- quired direct model predictions, while the vast majority were generated automatically through model- produced scripts.
SCHIEE k-18
SCRRE5, ke120
SCRIBES k=1000
Pe-page LLM irierence
10x
1024
10a
10
10'
webpaages
Figure 4: Estimated GPU FLOPs usage compar- ing SCRIBES-traned model with per-page LLM inference for Q-32B. The results show that, even when training compute included, SCRIBES- trained models scale more efficiently at web scale.
On average, processing a webpage with dedu- plicated HTML requires 8,879 tokens, whereas using flattened HTML requires 2,399 tokens. $879
Let p 2399
3.7 denote this relative per-
page token ratio. Our approach quickly becomes more efficient as long as the target website contains
least 4 structurally similar pages. In fact, the token speedup of our scribe-based method relative to flattening grows linearly with & (the number of structurally similar pages), following:
speedup k
We further compare the total GPU cost of SCRIBES, including training, with per-page LLM infer- ence Figure Let g denote the number of groups processed. While per-page inference (dashed purple line) increases linearly with both the number of groups g and the group size k, the SCRIBES- trained model yields substantial FLOP savings, with the magnitude savings growing proportion- ally group size. For instance. with 100 pages per group, SCRIBES can already provide compu- tational saving of 1.12 x 1021 FLOPS when processing 10 groups. Additional details on the FLOP estimates provided in Appendix D.3
Thus, compared to approaches that require ver-page LLM inference Bai et al. 2025 ). SCRIBES significantly cut down the GPU resource usage for web-scale extraction.
7

---

Published as a conference paper at ICLR 2026
Model and Method
All
Example pLla
Holdout
Rt
pLM
F|
Rt-M
F1-
R
PLM
FL
Q-14B (Reward Eq.
3
15.6
19.6
15.7
29.1
36.2
27.9
8.8
11.0
9.5
Q- 141 (SCRIBES)
23.0
24.3
19.9
31.2
29.8
26.7
19.0
21.7
16.7
Table 2: Ablation study of reward design (Eq. 3 enhances performance on holdout webpages.
showing that SCRIBES 's reward significantly
Method
All
Example
Holdout
RLM
pLM
FJM
RLU
PLM
FLM
RLMM
pLM
FLM
Q-14B (Annotated only)
23.0
24.3
19.9
31.2
29.8
26.7
19.0
21.7
16.7
Q-14B (+ All CC)
22.0
30.2
22.0
28.9
35.1
28.1
18.4
27.6
18.8
Q-14B Failure-Case CC)
25.2
23.0
21.8
34.9
31.0
30.0
20.5
19.1
17.7
Q-32B (Annotated only)
29.9
31.5
28.1
32.0
33.9
30.3
28.8
30.3
26.8
Q-32B (+ All CC)
31.1
34.1
29.7
35.2
37.0
36.1
32.9
29.0
28.
Q-32B (+ Failure-Cass Case CC)
37.4
36.0
33.2
39.5
35.5
34.6
36.2
36.2
32.4
Table Ablation study on CC data subsets, showing that models trained with the failure-case subset generally perform better.
Model and Method
All
Example
Holdout
RL.
pLM
PLM
R.M
pLM
R
PLM
P.LM
Q-14B agentic 3-iter 2-shot
9.5
13.7
8.8
23.2
24.2
20.0
12.4
Q-14B (SCRIBES)
20.7
22.2
19.4
31.8
36.1
30.4
14.5
12.0
12.2
Table 4: Ablation study on cross-domain transferability, showing that the SERIBES-trainee model demonstrate strong cross-domain transfer skills and outperform the baseline by more than 10%.
4.4 ABLATIONS
RQ3: Does the SCRIBES reward design improve the model's capability in generating scripts that generalize to holdout elements?
To answer this question, we train Q-14B model with the following reward for each training exam- ple pt
ro(p) (p)
Compared to Equation] [ this reward encourages the model only to generate scripts suited to the current training example, without considering other in-group elements. We still use the same input prompt as in our SCRIBES-traine models (Prompt 19), which instructs the model to produce scripts that generalize across similar webpages. The training setup remains unchanged.
As shown in Table 2 although this model outperforms Q-14B (SCRIBES) on the examples en- countered during inference (+1. (+1.2%), generalizes much more poorly similar webpages where the script is applied (-7.2%) resulting in worse overall performance in the "All" column (-4.2%). This shows that the SCRIBES reward design can more effectively instill in models the capability to produce generalizable scripts.
RQ4: Does using CommonCrawl data bring further improvements to our models?
We apply the technique described in Section 3.3.2 3..2 to the final checkpoints of the SCRIBES-traine Q-14B and Q-32B models on the annotated dataset. As shown in Table D additional training on synthetic data derived from CommonCrawl further improves performance, yielding gains of roughly 2% for Q-B and 5% for Q-32B overall.
To better understand the impact of noisy rewards, conducted the following ablation studies: (1) training directly on CC data, and (2) training on mixture CC and annotated data at a 1:1 ratio. Neither approach led to performance improvements, therefore hypothesize that is essential to first train the model with gold rewards to establish strong prior knowledge of this task. Subsequent training with noisy rewards can then expose the model to
shown in Table IO (Appendix G.2 We
8

---

Published as a conference paper at ICLR 2026
More numerous structures
complex structures
Breakdown
PLM
RLM
FT.
0.6
HT
Nested list
23.6
18.7
19.7
LLM
0.4
AVP
FF
All
Multi-column
23.3
39.5
33.9
35.5
34.6
18.6
(b) Comparison of the best Q-32 model's per- formance across nested-list and multi-column cases on our test set.
11
12-
28
28-
70
76-
199
Page Type
A nested list is defined content containing table another list embedded within an outer list, while multi-column content refers to in- stances where headers span multiple columns.
Structure Ratio
(a) Performance of our best Q-32B model by amount of structure and page type.
Figure 5: Error analysis: (a) model performance by structure complexity; (b) comparison across nested lists and multi-column formats.
Additional reference
Q-1.5B
Q-3B
Q-7B
Q-14B
Q-32B
GPT-4o
Flattened HTML
50,2
53.8
62.9
74.2
70.8
82.5
+ Best Direct L.M Extraction triples
52.8
61.2
66.6
73.5
73.1
82,7
+
Best Q-32B triples
52.9
54.3
64.1
77.3
73.2
86.6
Groond truth triples
60.5
64.9
70.5
78.2
74.8
87.4
Table 5: QA accuracy (%) with triple augmentations (evaluated by 3.aaam-.--nnstutttteeee
Prompt 20
SCRIBES 's predicted triples boost QA performance across many models.
more diverse inputs, not only preserving but further improving performance, analogous to findings in Shao et al. (2025).
RQ5: What's the effect of selecting the failure case subset continue CommonCrawl trainings?
As discussed in Section 3.3.2 we select the subset of CC data where our model produced scripts with no valid triples extracted. We examine whether restricting training to this subset is necessary by training both 14B and 32B model on the full CC dataset ("All CC") and only the subset where no triples were extracted ("Failure-Casc CC"). Results are reported in Table 3 We highlight two findings: (I) Training either All CC Failure-Case CC improves performance compared to using annotated data alone, and (2) Failure-Case CC yields stronger gains for Q-32B compared to All CC (+3.5%). while performance for Q-14B remains comparable across the two settings.
RQ6: Do PERIBES-trainee models transfer across domains? For example, does a model trained on finance legal tables generalize product encyclopedia pages?
To investigate this question, we conduct an ablation study using a train-test split in which the test set contains all product and encyclopedia pages, while the training set excludes webpages from these domains entirely. Details on this setup are provided in Appendix G.33 As shown in Table 4 the SCRIBES-traine model still substantially outperforms the strongest agentic baseline of the same model by more than 10%. To develop a model capable of web-scale extraction, we would still rec- ommend training on a dataset that encompasses diverse domains and page layouts, as demonstrated by our CommonCrawl processing in Section 3.3.2
4.5 ERROR ANALYSIS
We perform an error analysis to understand the failures of the best-performing Q-32B model. We break down performance by the amount of structure in a webpage (approximated by the ratio of raw HTML length to flattened text length) and by webpage type. As shown on the left of Figure 5a where webpages grouped into five equal-sized bins (by number of webpages) and the respective medians are reported, performance declines as webpages contain more structure. On the right, the model performs best webpages with Horizontal Tables (HT), followed by Attribute- Value Pairs (A- (P-))), and performs worst on Free-Form (F-F) pages. These results suggest that webpages with more numerous complex structures are particularly challenging for our model.

---

Published as a conference paper at ICLR 2026
We also compare the performance of model's outputs contents involving multi-column and nested lists. As shown in T5b we observe that such content is more challenging for our model. Further prediction examples showcased in Appendix
5 DOWNSTREAM APPLICATIONS
5.1 QUESTION ANSWERING OVER REMI-STRUCTURED WEB DATA
We demonstrate that our script-extracted triples can enhance QA performance, even for the most capable LLMs. Although there exist many ceneral-purpose QA datasets (Yang et al., 2018 Ra- jpurkar et al. 2016) and datasets focused on semi-structured databases (Chen et al, 2020 Zhu
al. 2021 Chen et al., 2021), very few address the setting where the input consists of HTML. SemiBench (Sun et al. (025) fills this gap, containing QA pairs with aligned triple annotations. This makes a strong testbed for evaluating whether triple extraction improves QA over semi-structured web data. We select the subset of QA data (a total of 416 QA pairs) associated with our test set and evaluate broad range of models as QA backbones, using the following reference conditions in Prompt 18 (1) Flattened HTML only; (2) Flattened HTML with best- pest-performing direct LLM- extracted triples (GO-120B 2-shot flatten): (3) Flattened HTML with our model motel-extracted triples; and (4) Flattened HTML with gold triples. We report the result on the QA pairs associated with our validation examples in T Our SCRIBES- models yield consistent gains across diverse QA backbones, including improvement of more than 4% for GPT-4o.
We further observe that although the SCRIBES- Strined models slightly underperform the strongest per-page LLM-inference baseline in Table I they nonetheless deliver comparable downstream QA gains. As shown in Table 5 using SCRIBES generated triples improves QA performance for Q- 14B and GPT-4o, yields roughly similar performance for Q-1.5B and Q-32B, and performs worse for Q-3B and Q-7B. These results indicate that higher IE accuracy does not necessarily translate into better downstream QA performance. Instead, using (CRIBES-produce triples can deliver much better efficiency and a similar level of downstream QA improvement.
EeRIBES--enerate
5.2 FURTHER DISCUSSIONS
The efficiency benefits of SCRIBES open up additional opportunities, and we highlight two direc- tions for future explorations:
Multi-page, Complex QAs: SCRIBES -xttracted triples enable queries that require aggregation or ranking across multiple webpages. For example, standard RAG solution would struggle with questions like "What the latest report filed?" when answering against the website Figure 2 In triples can efficiently support such queries, eliminating the need for resource-intensive. page-by sagge-by-pace KG construction with LLMs.
contrast,
Pretraining: Most open-source pretraining corpora systematically filter out semi-structured con- tent. For instance, C4 (Raffel et al., 2023) applies a "punctuation filter" that removes sentences not ending with valid punctuation. Recent popular corpora such Dolma (Soldaini et al., 2024) and FineWeb (Penedo et al.) 2024) inherit this bias, resulting in a near-comple absence of semi- structured data. We believe SCRIBES can address this gap by enabling efficient and resource- effective extraction and incorporation of such content into pretraining datasets.
6 CONCLUSION
This work introduces a novel RL framework, SCRIBES, for training models to generate general- izable extraction scripts across structurally similar webpages for semi-structured content extraction. We also propose a new method for generating synthetic training data, which further improves model performance, by leveraging in-the-wild webpages from CommonCrawl. Experiments on our dataset demonstrate that SCRIBES EBBBES-tE -rained models yield substantial gains in question answering semi- structured data. We hope that SCRIBES will facilitate further research on semi-structured content, such complex QA and pretraining, and serve valuable tool for the community.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Jiaxin Bai, Wei Fan, Qi Hu, Qing Zong, Chunyang Li, Hong Ting Tsang, Hongyu Luo, Yauwai Yim, Haoyu Huang, Xiao Zhou, Feng Qin, Tianshi Zheng, Xi Peng, Xin Yao, Huiwen Yang, Leijie Wu, Yi Ji, Gong Zhang, Renhai Chen, and Yangqiu Song. Autoschemakg: Autonomous knowledge graph construction through dynamic schema induction from web-scale corpora, 2025.
URL https: /aapiv/ org/ 22055 23628
Adrien Barbaresi. Trafilatura: Web Scraping Library and Command Line Tool for Text Discov- ery and Extraction. In Proceedings of the Joint Conference of the 59th Anmual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natu- ral Language Processing: System Demonstrations, pp. 122-131. Association for Computational
Linguistics, 2021. URL htttpp://allanthoooge www.tttwwooooooooooooooooooo hwtp://g.gg.0...... acl demo , 15
Bohan Chen and Andrea L. Bertozzi. Autokg: Efficient automated knowledge graph generation for language models. In 2023 IEEE International Conference on Big Data (BigData), pp. 3117-3126, 2023. doi: 10. 10.1 1at10/// 1at.:/990042222133866663353 10386454.
Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, and William Yang Wang- HybridQA: A dataset of multi-hop question answering over tabular and textual data. In Trevor Cohn, Yulan He, and Yang Liu (eds.), Findings of the Association for Computational Linguis- fics: EMNLP 2020, pp. 1026-1036, Online, November 2020. Association for Computational Lin- guistics. doi: 10. 10.18853/w1/20// 2020. findingsnnn--ii... emn 917
wtw.://w..i.iinnennn....... URL http:://acaaattoollttt.c oogg
Wenhu Chen, Ming- Wei Chang, Eva Schlinger, William Wang, and William W. Cohen, Open ques- tion answering over tables and text, 2021.
wwtp:/s////aaaa/oeeooooos 20100. 10439
Philipp Christmann, Rishiraj Saha Roy, and Gerhard Weikum. Conversational question answering on heterogeneous sources. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development
Information Retrieval, SIGIR '22, pp. 144-154, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450387323. doi: 10.1145/ 3477495.3531815 URL /t/.ttppe///oo..rrr////111/11111 3531815
Common Crawl. Common crawl.
2025. Accessed: 2025-08.
Valter Crescenzi, Giansalvatore Mecca, and Paolo Merialdo. Roadrunner: extraction from large web sites. In Proceedings of the 27th International Conference on Very Large Data Bases, VLDB '01, pp. 109-118, San Francisco, CA, USA, 2001, Morgan Kaufmann Publishers Inc. ISBN 1558608044.
Towards automatic data
Nilesh Dalvi, Ravi Kumar, and Mohamed Soliman. Automatic wrappers for large scale web extrac- tion, 2011.
Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. Knowledge vault: a web-scale approach to proba- bilistic knowledge fusion. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 14, pp. 601-610, New York, NY, USA, 2014. Association for Computing Machinery. ISBN 9781450329569, doi: 10.11452623330.2222222 2623623
URL /TT 1
Firecrawl. firecrawl: The web data api for ai turn entire websites into Ilm-ready markdown or struc- tured data. hhttps:://iitubb.cmm/ htw.:ttbb.ccooomee recrawel hfwf///rrrrrallllooo September 2025. GitHub repository, licensed under AGPL-3.0, 54.3k stars, 4.6k forks (as of Sept 2025).
Dayne Freitag and Nicholas Kushmerick. Boosted wrapper induction. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 577-583, 2000.
Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. Hipporag: Neuro- biologically inspired long-teo tern memory for large language models. In The Thirty-eighth Anmual Conference on Neural Information Processing Systems, 2024. URL ht htttss://ppeneeiie. met//orum?id jvAVVsg

---

Published as a conference paper at ICLR 2026
Nicholas Kushmerick, Daniel S Weld, and Robert B Doorenbos. Wrapper induction for information extraction. In Proceedings of the Fiffeenth International Joint Conference on Artificial Intelli- gence (ICAI). pp. 729-737, 1997.
Bing Liu, Robert Grossman, and Yanhong Zhai. Mining data records in web pages. In Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03, pp. 601-606, New York, NY. USA. 2003. Association for Computing Machinery. ISBN 1581 137370. doi: 10.11449956550.5566 URL /tL//tttss///o...o////.... Tttp:/11.11199997797 956826
Shicheng Liu, Sina Semnani, Harold Triedman, Jialiang Xu, Isaac Dan Zhao, and Monica Lam. SPINACH: SPARQL-based information navigation for challenging real-world questions. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Compu- tational Linguistics: EMNLP 2024, pp. 15977-16001, Miami, Florida, USA, November 2024a.
Association for Computational Linguistics. doi: 10. 10.18653/ 1/204, http://22222ii2nnneeeeeee sondings-emnl.9955 URL httts::/aaclane woc.hooooy....o org/ 2024. f ndi wadinggsa ngg emn 1p. 938/
Shicheng Liu, Jialiang Xu, Wesley Tjangnaka, Sina Semnani, Chen Yu, and Monica Lam. SUQL: Conversational search over structured and unstructured data with large language models. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Association for Com- putational Linguistics: NAACL 2024, pp. 4535 4555, Mexico City, Mexico, June 2024b. Association for Computational Linguistics. URL
Findings naacl. .283
Colin Lockard, Xin Luna Dong, Arash Einolghozati, and Prashant Shiralkar. Ceres: distantly supervised relation extraction from the semi-structured web. Proc. VLDB Endow, 11(10): 1084-1096, June 2018. ISSN 2150-8097. doi: 10.14778/2317551323377 777.10117777131111222222
URL https:
Colin Lockard, Prashant Shiralkar, Xin Luna Dong, and Hannaneh Hajishirzi. ZeroShotCeres: Zero- shot relation extraction from semi-structured webpages. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.). Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. -117- Online, July 2020. Association for Computational Linguistics. doi: 10.
URL https::/aclaattholl/l
org/
2020.ac1= main. 7217
Kaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg, and Jianfeng Gao. Open domain ques- tion answering with unified knowledge interface. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1605-1620, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. 10.18653/1/22222.ac-onnn httpss//accaanthoooryyyoo wtt.::ooooooo....//00/ ott.:/oooooooooooo2222222..oooa.
URL
aww-alng...............o..o..... 1137]
Yansong Ning, Hao Liu, Hao Wang, Zhenyu Zeng, and Hui Xiong. Uukg: Unified urban knowledge graph dataset for urban spatiotemporal prediction. Advances in Neural Information Processing Systems, 36:62442-62456, 2023.
Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Schlichtkrull, Sonal Gupta, Yashar Mehdad, and Scott Yih. UniK-QA: Unified representations of structured and unstructured knowledge for open-domain question answering. In Marine Carpuat, Marie-Catherinn de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Findings of the Association for Computational Linguistics: NAACI. 2022, pp. 1535-1546, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 18.186 1/2022. finddigss--aaaeee 15 URL findingnisisss naacl. 115/
Andrei Paraschiv. newspaper4k: Article scraping & curation, a continuation of newspaper3k.
March 2024. GitHub repos- itory, a fork of Newspaper3k by codelucas; latest release v0.9.3 (March 18 2024), MIT license.
Guilherme Penedo, Hynek Kydlfček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. The fineweb datasets: Decanting the web for the finest text data scale, 2024. URL https: http///xwww..or/////////00//0 11024406......5 17557
12

---

Published as a conference paper at ICLR 2026
Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yarn: Efficient context window extension of large language models, 2023.
Jake Poznanski, Jon Borchardt, Jason Dunkelberger, Regan Huff, Daniel Lin, Aman Rangapur, Christopher Wilhelm, Kyle Lo, and Luca Soldaini. olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models, 2025. URL htttps://arxiiv.ogr/ab///// 18443
Mihir Prabhudesai, Lili Chen, Alex Ippoliti, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. Maximizing confidence alone improves reasoning. arXiv preprint arXiv:2505.22664 2025.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023. URL https: http://aavvvvve. org wwwaaaasb// 1910. 1 10683
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text, 2016. URL httttpp://arxv..ooo wwwpxii.oogg/ab///i0000
Josh Schulman. Approximating kl divergence. Blog post, 2020.
Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, et al. Spurious rewards: Rethinking training signals in rlvr. arXiv preprint Sxkiv:2506.194477 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models, 2024. URL /tattss://axiv.. org/ abs/ 2402.
03300
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Evan Walsh, Luke Zettlemoyer. Noah Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, and Kyle Lo. Dolma: an open corpus of three trillion tokens for language model pretraining re- search. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15725-157888 Bangkok, Thailand, August 2024. Association for Computational Linguis- ac URL httpps/////caaaoooo.....oo org/2024
tics. doi: 10. 115533/1/2022.acllloo acl-long 8407
Kai Sun, Yin Huang, Srishti Mehra, Mohammad Kachuee, Xilun Chen, Renjie Tao, Zhaojiang Lin, Andrea Jessee, Nirav Shah, Alex Betty, Yue Liu, Anuj Kumar, Wen tau Yih, and Xin Luna Dong. Knowledge extraction on semi-structured content: Does it remain relevant for question answering in the era of llms? 2025.
Jiejun Tan, Zhicheng Dou, Wen Wang, Mang Wang. Weipeng Chen, and Ji-Rong Wen. Htmirag: Html is better than plain text for modeling retrieved knowledge in rag systems. In Proceedings of the ACM on Web Conference 2025, WWW '25, pp. 1733-1746, New York, NY, USA, 2025, Association for Computing Machinery. ISBN 77794000112766 doi: 10.1145/3694411337145566
URL
3714546
Feng Wang, Zesheng Shi, Bo Wang, Nan Wang, and Han Xiao. Readerlm-v2: Small language model for html to markdown and json, 2025. URL /oL.:tttpps//rrxx.or///b/b 220.0///......////20....... 01151
Yorick Wilks. Information extraction as core language technology. In International Summer School on Information Extraction: A Multidisciplinary Approach to an Emerging Information Technology, SCIE '97. pp. 1-9, Berlin, Heidelberg, 1997. Springer-Verine
ISBN 354063438X.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio. William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse. explainable multi-hop question answering, 2018. htRl::tts:// wwtp://wwwwraaa////oee///
09600
13

---

Published as a conference paper at ICLR 2026
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022.
Yanhong Zhai and Bing Liu. Web data extraction based on partial tree alignment. In Proceedings of the 14th International Conference on World Wide Web, WWW '05, pp. 76-85, New York, NY, USA, 2005, Association for Computing Machinery. ISBN 1595930469. doi: 10.1145/1060745. 1060761. URL /attps://don
Bowen Zhang and Harold Soh. Extract, define, canonicalize: An LLM-based framework for knowl- edge graph construction. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Pro- ceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp- 9820-9836, Miami, Florida, USA, November 2024. Association for Computational Linguis- tics. doi: 10.
Ut / /acl anthe waathelogy.oo 0r9/2024.
@mmn 1 1-- 1.. 5487]
Heidi Zhang, Sina Semnani, Farhad Ghassemi, Jialiang Xu, Shicheng Liu, and Monica Lam. SPAGHETTI: Open-domain question answering from heterogeneous data sources with retrieval and semantic parsing. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics; ACI. 2024, pp. 1663-1677 Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.11655/w1/2024.fmmii findings-acl.90 URL httppss//aalannhologgy. org/ 2024 findings--ani 967]
Kai Zhang, Bernal Jimenez Gutierrez, and Yu Su. Aligning instruction tasks unlocks large lan- guage models zero-shot relation extractors. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL. 2023, pp. 794- 812, Toronto, Canada, July 2023. Association for Computational Linguisties. doi: 10.18653/ v1/2023, findings-acl.55. URL
findings -cll
50/
Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song. Learning to reason without external rewards. arXiv preprint 250X:::9005.99900 2025.
Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, Alban Desmaison, Can Balioglu, Pritam Damania, Bernard Nguyen, Geeta Chauhan, Yuchen Hao, Ajit Mathews, and Shen Li. Pytorch fsdp: Experi- ences on scaling fully sharded data parallel, 2023. WRL/httss://aat http///wwww...r/////////4///4 11277
Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. TAT-QA: A question answering benchmark on a hybrid of tabular and textual content in finance. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Naturul Language Prucessing (Volume 1: Long Papers), pp. 3277-3287, Online, August 2021. Association for Computational Linguistics. doi: 18.18v53/v1/ 2021.acl-long.. long.254. URL wttphttpp//. http:////aaaathhhoooooooo http:////o.otttttt////aaaattttor org.
254/
Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, al. Ttrl: Test-time reinforcement learning. arXiv preprint MrXiv:2504.16085 2025.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
A Use of LLMs in this Research
16
Websites with Semi-Structured Content
16
HTML Dedup Algorithm Details
16
D Training Hyperparameters and Other Details
17
D.I Data Pre-processing
17
D.2 Trrinnig Details
18
D.3 Detailed on Compute Comparison
18
E Dataset Comparison
18
Metries and their implementation
19
Details on the Fuzzy Match Algorithm
19
Reward during RL implementation
20
F3
Human verification of RL reward
20
Additional Experiments
20
Additional LLaed Agentic Baselines
20
G.2
Additional Ablation Experiment on Impact of Noisy Reward
21
G.3
Additional Ablation Experiment on Domain Transfcrability
G.4
Complete Baseline Numbers
22
Details on QA dataset used in Sec. 5.1
22
Example Model Predictions
23
Prompts Used
25
15

---

Published as a conference paper at ICLR 2026
A USE OF LLMS IN THIS RESEARCH
We utilize LLMs two main ways in this research:
Assistance with Code Writing: During the implementation of RL training and evaluation scripts, LLMs were occasionally used assistants. All code was subsequently double- checked and verified by the authors.
2. Paper Language and Related Works: During the writing process, we occasionally uti- lized LLMs to improve the clarity and fluency of the English. We also occasionally use LLM-assisted search systems to find additional related works. All final text was reviewed by the authors.
B WEBSITES WITH SEMI-STRUCTURE CONTENT
We can broadly classify webpages with semi-structured content into three categories:
Horizontal Tables: These webpages primarily present information a tabular format.
2. StrributeVaalue Pairs: Information organized as attribute-valus pairs, typically dis- played across multiple rows in an "infobox" -like format.
3. Free Form: stritstructured content is distributed throughout the page, often combining both horizontal tables and attribute-value pairs.
For additional information and more details on these breakdowns, refer to Sun et al. 2005
C HTML DEDUP ALGORITHM DETAILS
<IEECTVPE hIRts
Removed
Rape//iitacc
<dlr Claster boader" (6
</hesD
sal Class" prouct-grii" www.-ottponnnaaeeooooooo "eteeppntlltttmmiic....
<ite classe' teake" Los Yool teador"
wo class Lusse"proesch www.ert--ttttiiiiiooooooo troake bwits
odia clase grockc-ersst cata-catagary "atectenretsssseeeeeee
class www.ii--illl -erodcct Scht-
prabacircree 2ste-1de3" styter bender: 3px setid Rese""
plass""riie"
opan class pwioe 90/./0/
odle class" praductcard sata-dper2hs
seroeenaair holee
cane protuct-ann
styler lps solid #eeep"> -ittitrrrrrttttt
<apas prace spens
clack Toodk"breeett wtw.etttttteeeeeeeeeee B0T5>
2015 220
wire Classe comd anto-tars stylee bonderi 184 sel3d Roce's
aper class blase bracd 1529, Merssam
Figure 6: An example illustrating Algorithm is shown here. The original HTML appears on the left, while the compressed HTML is shown on the right. The dashed-highlighted section near the top, containing script and style elements, has been removed. The repeated HTML content near the bottom has been deduplicated, retaining up to 3 elements.
Raw HTMLS are often long and repetitive. We propose a simple and effective dedup algorithm to significantly cut down the token length of HTML pages while still maintaining its structure. Algorithm shows the implementation of this algorithm. We set z 3 in our experiments.
Table 6 shows the token saving effect of our dedup algorithm. Removing whitespaces in HTML only brings minimal token savings (< 2%), while our dedup algorithm brings significant token savings, cutting down token usage from > >-1k <1k We also profiled performance gains of baselines models using dedup. As shown in Table 7 employing deduplicated HTML yields clear improvements compared to using HTML. Most notably, deduplication significantly increases the Non- Non-Empty Rate of baseline performance by enabling more data points to within the model's context window.
16

---

Published as a conference paper at ICLR 2026
Algorithm Structure-Preserving HTML Deduplication (keep-z)
Require: Raw HTML string H, integer = 2 1 (default 2-3)
Ensure:
Compressed, structure-preservingg HTMLL
1: Parse H into DOM It (fallback parser if needed; return H on failure)
2: RemoveTTag t Ascript,stylee noscript,
iframe, embed, abject,
meta, 1ink,,nk,,b,,
role, 3: KeepAttrs +- rid,
"xpee,href.srrcaaltttttee
for, method,
value,
requi data data-x,aria-s,
4: Remove all nodes with tag in RemoveTags
5: Remove all HTML comments except those starting with "..."
6: for all element nodes e in R do
7: for all attributes of do
8: if a & KeepAttrs and not prefixed by data- or aria- then delete attribute from 10: end
9:
11: end for
12: end for
13: for all nodes in traversal of R do
14: if n..ta E oul, div, sect i on, tbody, thead, select] then
children + [c n.children is an element]
16:
Group children by sig(c) + (c.tag, sort(c.class or (]))
for all group G do
18:
if G then
19:
Keep the first z in (order preserved); remove the rest After the z-th kept node, insert comment:
20:
21:
|G| more <lag class=' ...'> elements
22:
end if end for
23:
24:
end if
25:
end for
Optionally normalize whitespace and excessive blank lines
27: return serialized DOM
Processing Stage
Avg Tokens
Percentage
Original tokens
114,318.6
100.0%
After whitespace removal
112,279.0
98.2%
After dedup
16,985.1
14.9%
Reductions
Whitespace token savings
2,039.6
1.8%
Total dedup token savings
97,333.5
85.1 %
Table 6: Token reduction analysis across the webpages collected by Sun et al. (2025). profiled with GPT-4o tokenizer, accessed via
Tokens were cww/ wwt.://...oooppaaa.a.oaa
D TRAINING HYPERPARAMETERS AND OTHER DETAILS
D.1 DATA PRE-PROCESSING
During training, we set the maximum prompt length to 28672 tokens and the maximum response length to 4096 tokens. This results in total model context window of 32768 tokens, which is the maximum length before needing to apply YaRN (Peng et al. 2023 for the Qwen-2.5 series model
We observed empirically that model training with YaRN becomes much more unstable and difficult to converge.
17

---

Published as a conference paper at ICLR 2026
Model & Format
PLM
RLM
F..LM
Non-Empty Rate
L.77BB w/ Raw HTML
3.4
3.7
3.5
37,9
L-70B w/ Dedup HTML
14.2
9.5
11.3
46.4
GPT-4o w/ Raw HTML
13.7
15.4
14.5
63.8
GPT-4o w/ Dedup HTMI.
19.1
23.0
20.9
94.9
Table 7: Performance comparison of baseline models using raw dedup-ed HTML. Here, we feed each page in one-by-one in this dataset and only evaluate the model's performance on one given page. Non-Empty Rate is set to 1 if the model's generated code produced at least 1 triple on this page, and if otherwise.
SemiBench (Sun et al. 2025 includes a subset of 268 webpages drawn from 56 groups, each con- taining more than one webpage. We partition the groups into training and test sets at an approxi- mately 6:4 ratio, resulting in 34 groups (192 webpages) for training and 22 groups (76 webpages) for testing. After applying the maximum-contex constraint described above, 141 training webpages and 65 test webpages remain.
D.2 TRAINING DETAILS
During GRPO training, we do not apply entropy loss. We set the KL loss coefficient to 0.001 and the KL loss to be the ky loss using the approximation described in Schulman 22000 i.e.
nnew (a)
new (a)
kz(a)
log
1
Told (a)
oold (a)
We the default model rollout parameters (for (wwe-22.5-nnstrue these are top -1, top top-p= 1, and temperature 1) and validation/inference parameters (for Qwen-2.5-instrue these are top k= 0). We do not use LoRA and instead perform full-pare aallmpareetter
-1, top-p= 1, and temperature finetuning with FSDP (Zhao et al., 2023). We trained the models the annotated set for total of 50 epochs, and on CommonCraw data for epoch. For each update, we collect 8 rollouts to perform GRPO update. For the 32B model, we apply 0.5 gradient clipping, which found to lead to more stable trainings. We set the learning rate to be a constant 1e 6.
D.3 DETAILED ON COMPUTE COMPARISON
To train the Q-14B SCRIBES model, throughput of 3,958 TFLOPs across all GPUs, yielding an estimated total training compute of 1.71 x 102e FL.OPs. For the Q-32B SCRIBES model, similarly trained for about 12 hours at average of 8,232 TFLOPs, resulting in an estimated total compute of 3.56 x 10 FLOPs.
approximately 12 hours of training at average
To estimate the per-page inference cost reported in Figure 4 we assume that each forward pass requires roughly 2 times the parameter count per token.
E DATASET COMPARISON
We compare several statistics of HTML webpages in Table S Below, we define each statistic:
The maximum depth of the Document Object Model (DOM) tree in an HTML document. This measures how deeply the elements are nested; higher DOM Max Depth indicates more extensive nesting.
DOM Max Depth:
The lengths of the HTML content before and after applying the deduplica- tion algorithm described in Appendix C (in characters). This quantifies redundancy in the HTML structure; a lower Deduplication Ratio indicates greater redundancy.
Deduplication Ratio:
Structure Ratio:
The ratio of the HTML length to the flattened text length (in characters). This approximates how much structural markup the HTML contains relative to its textual content; a higher Structure Ratio reflects more structural complexity.
18

---

Published as a conference paper at ICLR 2026
Tag Count: The number of all tags in an HTML documen This measures the structural complexity of the HTML; a higher Tag Count indicates a more complex document.
Feature
Metric
Train
Test
CC (After Step 6 in Fig. 3)
Mean
20.2
18.4
20.2
Median
19.0
17.0
17.0
DOM Max Depth
Std
4.94
6.98
21.7
Min
10.0
10.0
5.00
Max
37.0
37.0
455
Mean
0.215
0.174
0.353
Median
0.213
0.166
0.344
Deduplication Ratio
Std
0.111
0.0986
0.178
Min
0.0302
0.0324
0.000480
Max
0.484
0.422
1.02
Mean
46.1
43.5
20.8
Median
28.7
27.4
13.8
Structure Ratio
Std
40.0
45.5
48.7
Min
2.24
6.00
1.19
Max
174
199
1960
Mean
1650
1820
655
Median
1260
1080
496
Tag Count
Std
2140
2550
559
Min
224
154
18.0
Max
27800
12300
5070
Table 8: Summary statistics (Mean, Median, Std, Min, Max) for HTML-derived features across datasets.
This comparison shows that the labeled training and test sets share similar summary statisties, whereas the CommonCrawl portion differs noticeably. In particular, the CommonCraw/ data is less redundant (lower Deduplication Ratio), contains less structural markup (lower Structure Ratio), and is structurally simpler (lower Tag Count). Across all metrics, it also exhibits greater variability, as indicated by the higher standard deviations. These observations suggest that incorporating this portion of the CommonCrawi data into training can meaningfully broaden the distribution of inputs, exposing the models to examples that differ substantially from those the labeled dataset.
F METRICS AND THEIR IMPLEMENTATION
F.I DETAILS ON THE FUZZY MATCH ALGORITHM
Formally, let G {g1. 92- ... Im. denote the set of gold triples and P {pi P2..... Pn.) the predicted triples. Instead of requiring exact equality, we define similarity function finay (gi, P2) e [0, 1] that quantifies the degree of match between gold triple 9i and predicted triple pj as the tio of character-level matching To ensure one-to-one alignment, bipartite matching between G and P. where the weight of each edge is fluzyy (.). This assign- ment is efficiently solved using the Jonker- Volgenant algorithm" Precision, recall, and F, are then generalized as:
compute maximum- weightt
pluzzy Eiamem fluzey (g.p)
Elariem flunyy
2. pfuzzy Rluzzy plucy Rluzy
pluzzy
F furzy
P
'1en (soup. Find_al1 (True))
'Implemented via /ttppss//tttttuu www.aaa....o.eaaaooooooo.e...eee wttp:///ttteeeeeeeezaeeeooo htw.:/geeek//uzzywwzzyy ratio function, which caleu-
late a ratio character-leve matching using Levenshtein distance
"mpleemented via httpttsst////oot sci py. http://///////./../eeeee/e////ea
op ze. near aataarssm_______ gnment. html
19

---

Published as a conference paper at ICLR 2026
Method
RLM
PLM
PL
R.uzzy
pluzay
F Ffuzay
Q-14B flatten
30.46
36.46
29.87
45,96
52.37
43,50
Q-32B flatten
28.73
37.44
29.93
41.62
54.25
42.26
GO-20B 2-shot flatten
33.18
47.10
34,93
46.53
65.21
49.77
GO-120B 2-shot flatten
42.27
46.26
40.40
56.01
61.42
53.37
Q-14B 3-iter 2-shot
8.59
11.13
8.01
17.17
25.57
16.53
Q-72B 3-iter 2-shot
16,40
19.41
14.97
28.73
37.96
28.60
Q-32B 3-iter 2-shot
18.56
27.20
19,41
27.49
44.67
30.39
GO-20B 3-iter
24.70
23.22
20.87
52.30
41.83
39.58
GPT-4o 3-iter 2-shot
25.95
33.04
24.42
45.58
60.57
44.46
GO-120B 3-iter 2-shot
33.86
40.96
34.30
49.79
65.72
52.02
Comparison of LLM-judged metrics and fuzzy-matching metrics for baselines reported in Table 1 for the "All" column. Gray-highlighted columns denote F[-M and Fluezy This comparison shows that the two metrics show similar performance trend across models and configurations.
Table 9:
where M C G x P denotes the optimal matching. Given M, the LLM-based metric evaluates correctness by invoking LLM the final matched pairs of gold and predicted triples. For each pair (g, p) E M, the model outputs a binary judgment fLM (g.p) E {0, 13, where denotes a true We then define LLM-based precision,
match and denotes failed match according Prompt 17
recall, and Fi as:
PLM EGDDEM FIM (g.p)
R.L. Eigpless fIM (g.p)
2. A.PLM. RLM PIM RIM
FLM
Pl
|G|
Empirically, observe correlation between pruzzy and FI.. The latter tends to yield slightly lower absolute scores but exhibits the same performance trend across models and configurations. A comparison showing the two metrics and the associated precision and recall metrics for the baselines are shown Table We calculated the correlation coefficient between Fruzyy and FJ to be 0.957 with p-value of 1.4 x 10 showing strong positive correlation.
F.2 REWARD DURING RL IMPLEMENTATION
We use Fluzzy during training as proxy for FlM, thereby avoiding LLM calls. Because computing fuzzy Fi exactly requires solving a maximum- weight bipartite matching, runtime can become too long for large sets of triples. We thus approximate all candidate pairs of gold and predicted triples scored by fluny, sorted in descending order, and added sequentially to the matching as long as they do not conflict with previously chosen pairs. This impose 60-seconds cutoff for
matching with a greedy heuristic. Specifically,
yields fast, albeit sub-optimal, alignment. To ensure scalability, evaluation. If timeout occurs, we further project the total similarity score by extrapolating from the average score of observed matches the remaining unmatched capacity.
F.3 HUMAN VERIFICATION OF RL REWARD
We followed the same evaluation metrics as defined in Sun et al. (022)), which reported 95% agreement rate between the LLM-based Fl metric 12 and human judgments, indicating strong alignment.
G ADDITIONAL EXPERIMENTS
G.1 ADDITIONAL LLM-BASED AGENTIC BASELINES
In addition the simple 2-shot baseline, profile two promising LLM-based agentic knowledge- base-consee construccion baselines: HippoRAG (Gutiérrez et al.) 2024) and AutoSchemaKG (Bai et al, 2025). representative of recent L..M-driven KG construction pipelines.
HippoRAG a retrieval-ad auggeettd generation framework that builds knowledge graph as an embedding index, mimicking the role of the hippocampus in human memory. We use the first stage
20

---

Published as a conference paper at ICLR 2026
Method
All
Example
Holdout
RIM
pLM
F.
R
pLM
F.
RDM
ptM
P..
Q-14B (Annotated mixed with CC)
6.5
8.0
6.5
8.1
9.6
7.9
5.7
6.4
5.7
Q-14B (CC only
7.7
15.8
9.2
8.9
18.4
10.8
7.2
14.7
8.4
Q-14B (Annotated followed by CC)
25.2
23.0
21.8
34,9
31.0
30.0
20.5
19.1
17.7
Table Ablation study on the impact noisy reward. We compare three training configurations: (1) CC data only, (2) annotated data mixed with CC data at a 1:1 ratio, and (3) training first on annotated data followed by CC data. Results show that noisy reward alone or mixed training does not improve performance, whereas staged setup, first training annotated data before continuing with CC, yields substantial gains.
of their KG construction pipeline, which consists of two prompts, one for named entity recognition (NER) and one for triple extraction. We also replace their 1-shot example with the same 2-shot examples used in our baseline.
pretraining-scale corpus. calls three LLM modules on each webpage: (1) an entity-entity relationship extractor, (2) an entity-event relationship extractor, and (3) event-event relationship extractor. These prompts all zero-shot and are challenging to adapt, so we retain them as originally specified.
AutoSchemaKG is framework for web-scale KG construction
As shown in Table 11 the simple 2-shot baseline outperforms both LLM-based baselines across all models evaluated on our task, including by more than 20% for the strongest model, GPT-OSS-I20P Moreover, they inherit the same cost inefficiencies, as each webpage requires multiple LLM calls.
G.2 ADDITIONAL ABLATION EXPERIMENT ON IMPACT OF NOISY REWARD
To further investigate the role of noisy reward, we conduct additional ablation experiments under three training configurations: (1) training on CC data only, (2) training on mixture of CC and annotated data at 1:1 ratio, and (3) training first on annotated data and then continuing on CC data. Results are reported in Table TO
G.3 ADDITIONAL ABLATION EXPERIMENT ON DOMAIN TRANSFERABILITY
In this ablation study, reorganized the dataset by assigning each website to of the following content categories:
Finance & Economics
Legal & Regulatory
Developer & Software
Science & Research
Science & Database
Sports
Gaming Entertainment
Media & Entertainment
Real Estate
Social Platforms
Weather & Enviromment
Jobs & Careers
Travel & Hospitality
Products & Brands
Encyclopedias & Reference
21

---

Published as a conference paper at ICLR 2026
Method
RIM
PLM
FII.MM
FL.
Baselines (Flattened)
Q-14B w/ AutoSchemaKG (Bai et al.
2025
2.1
8.26
3.35
8.17
Q-14B w/ HippoRAG (Gutierrez et al.
2024
2-shot
8.49
32.24
13.43
16.12
Q-14B flatten
30.46
36.46
33.19
29.87
Q-32B AutoSchemaKG (Bai
2025
2.64
11.14
4.27
Q-32B w/ HippoRAG Gutierrez el al.
2024
2-shot
10.12
39.7
16.13
20.03
Q-32B flatten
28.73
37.44
32.51
29.93
GO-20B w/ AutoSchemaKG (Bai
2025
5.57
11.96
7.6
9.26
GO-20B HippoRAG (Gutierrez 2024
2-shot
8.26
23.06
12.16
14.02
GO-20B flatten
36.94
37.88
37.40
33.61
GO-20B 2-shot flatten
33.18
47.10
38.93
34.93
GO-120B w/ AutoSchemaKGi (Bai et al.
2025
6.52
16.97
9.42
12.28
GO-120B w/ HippoRAG (Gutierrzz
2024
2-shot
28.57
12.12
17.02
17.22
GO-120B flatten
36.43
34.59
35.49
31.74
GO-120B 2-shot flatten
42.27
46.26
44.18
40.40
Baselines (Script-gen)
Q-14B agentic-3-iter
8.11
8.26
8.18
7.14
Q-14B agentic-3-iter 2-shot
8.59
11.13
9.70
8.01
Q-32B agentic-3-iter
10.41
9.08
9.70
8.74
Q-32B agentie-3-iter 2-shot
18.56
27.20
22.07
19.41
Q-72B agentic-3-iter
9.67
9.65
9.66
7.19
Q-72B agentic-3-iter 2-shot
16,40
19.41
17.78
14.97
GO-20B agentic-3-iter
24.70
23.22
23.94
20.87
GO-20B agentic-3-iter 2-shot
13.06
27.30
17.66
14.40
GO-120B agentic-3-iter
27.63
24.76
26.12
23.30
GO-120B agentic-3-iter 2-shot
33,86
40.96
37.07
34.30
GPT-4o agentic-3-iter
19.05
14.72
16.61
13.81
GPT-4o agentic-3-iter 2-shot
25,95
33.04
29.07
24.42
L--7BB agentic-3-iter
10.05
15.49
12.19
10.47
L-70B agentic-3-iter 2-shot
8.23
8.08
8.15
7.10
SCRIBES
Q-14B
22.96
24.26
23.59
19.91
14B (+CC)
25.24
22.98
24.05
21.77
Q-32B
29.88
31.53
30.68
28.05
Q-32B (+CC)
37.41
36.03
36.71
33.24
Table 11: List of all baselines and SCRIBESStrained models. LLM-judged metrics on all data. PLM RLM, harmonic F. H.H.MM and average per-example IrL.M.
We placed Products & Brands and Encyclopedias & Reference in the test set, with all remaining categories assigned to the training This split yielded 196 training examples and 72 examples. After applying the maximum-conteed constraint described in Section D.I. 147 training examples and 59 test examples remained.
G.4 COMPLETE BASELINE NUMBERS
For Fi, we provide two variants:
(i) the macro-average of per-example Fj scores, and (ii)
harmonic-mean variant defined as
2PR
F
(4)
P
where P and R denote the mean precision and recall, respectively. The complete list of baseline
performance is shown in Table TTlad
H DETAILS ON QA DATASET USED IN SEC. 5.1
The QA pairs in Sec. 5.I are collected by Sun et al. 2025 through an LLM- -eg-eeeretted followed by human-aue human-auditing process. We summarize their process below:
22

---

Published as a conference paper at ICLR 2026
Method
Example
Holdout
RIM
ptM
F.ILM
FT
R
pLM
FIM
Baselines
Q-14B agentic-3-ite
11.96
11.81
11.88
10.57
6.47
6.90
6.68
5.77
Q-14B agentic-3-itet 2-shot
13.21
17.97
15.23
12.63
6.29
7.79
6.96
5.73
Q-32B agentic-3-iter
18.84
17.17
17.97
16.46
6.36
5.33
5.80
5.07
Q-32B agentic-3-iter 2-shot
24.53
28.79
25.90
15.79
23.91
19.02
Q-72B agentic-3-iter
13.03
13.15
13.09
10.12
8.20
8.12
8.16
5.94
Q-72B agentic-3-iten 2-shot
24.11
28.59
26.16
21.78
13.26
15.83
14.43
12.38
GO-20B agentic-3-iter
29.25
26.38
27.74
24.91
22.51
21.78
22.14
18.94
GO-20B agentic-3-iter 2-shot
13.48
27.68
18.13
14.41
13.07
27.11
17.64
14.66
GO-120B agentie-3-iter
31.32
26.76
28.86
25.70
25.86
23.86
24.82
22.16
GO-120B agentic-3-iter 2-shot
35.83
42.27
38.78
36.60
32.98
40.47
36.34
33.26
GPT-4o agentic-3-iter
25.19
18.35
21.23
18.47
16.00
12.89
14.28
11.47
GPT-4o agentic-3-iter 2-shot
32.98
36.48
34.64
31.19
22.52
31.32
26.20
21.11
L-70B agentic-3-iter
23.76
19.58
16.78
11.16
8.49
7.36
L-70B agentic-3-iter 2-shot
7.77
6.77
7.23
8.42
8.54
7.51
SCRIBES
Q-14B Q-14B (+CC)
31.22
29.81
30.50
26.71
19.01
21.65
20.24
16.66
34.88
30.96
32.80
29.96
20.45
19.06
19.73
17.69
Q-32B
31.99
33.88
32.90
30.32
28.79
30.28
29.51
26.83
Q-32B (+CC)
39.54
35.48
37.40
34.60
36.24
36.15
36.20
32.41
Table 12:
List of all baselines and SCRIBES-traine models by Example and Holdout. LLM-
judged metries all data. pLM, RI. harmonic F.ILLM, and average per-example FLM..
First, 70B Llama model generated initial question-answer pairs using webpage content and ground truth data. Then, these pairs were refined by:
1. Removing overly complex questions that required heavy reasoning, focusing instead on comprehension of semi-structured webpages.
2. Eliminating compound questions that combined multiple queries into one to avoid in- flated difficulty.
3. Filtering out trivial questions that all tested models answered correctly, ensuring better differentiation among model performances.
Finally, human auditors reviewed and removed any pairs that were ungrounded in the source con- tent or contained incorrect answers.
For more details and statistics, refer to Sun et al. 2025).
I EXAMPLE MODEL PREDICTIONS
Here, we show an example our best model's prediction on the eval set to better illustrate the task and associated difficulties.
Forihhs website, our model generated the code shown Table 13 The prediction versus gold entries
shown in Table [4 This code achieves the following scores: RLM = 53.3, Ptm 50.0, FLM 51.6.
The model code misses some information such as the "Business Address" and "Mailing Address" fields, penalizing the recalls. It also produces some false positives, including triples where the relation is "sequence" but corresponding object is returned, reducing precision. Additionally, the model incorrectly treats the filing document number as the subject throughout, rather than using more appropriate subjects such "Class/Contrac C000008114:
23

---

Published as a conference paper at ICLR 2026
from bs4 import Beaut BcautifulSoup from typing import List
def main (html: str) [ist
soup Beaut falutfuullaae FulSoup (html, 'html, .arrser)) triples = 1|
Extract title title soup, soup.title.tex. if scup.title else " triples. append ( ["EDGAR Filing", "title", title])
Extract form information form
soup.find findddivv, id="formName")
if form_name:
triples.append app.nd [[title, "form", Corm_name.exx..tzeee...
sec_num soup, find id="secNum")
if sec num: hhples.appenne [[title, "accessi seccessionNumber,, sec_nur. str.surrttxtttrrip..........
Extract filing details
info heads soup. find all ("div" class_ "aass_=iinfolead""
info values info_vvlues_ foup.find. all ("div, class class_="info")
for head, value in zip (info heads, info_values). silples.ppendd ([title, tead.text.stripll
sexts
Extract document table information table soup. find findffaandaabee ("table", class_="tableFiin if table:
rews = table. find_all ("tr") (a1ttra
for in rows|l:l: $ Skip header
cols aow.tindaalll("tt""
"th"])
if [cn(s) >= 5:
seq - cols[[]] description - ceolssl).tettsstrigg
document cols(2].
doc_type
size = cols(3].
triples.append((tttteee "sequence", seq])
ariples.agpend "description", description))
aaallss.ppcen((titt.)) "document", document))
triples.append (ppplees,ppcnd(((iicc. "type", doc_typel) triples.appent ([title, "size", size])
Extract series and classes information series table soup. soup.find if series_tables
find ("tablee, class_=ttalesseries")
rows = series table. find all altr" for row in rows # Skip header rows
cols (ow.find_all((ttt) "th"]}
if (cols) >- 4:
status name = cols[2] ttss(2).text.striipp
ticker (all(3))text.striit
apapll "status", status])
append ([title, "name", namel) shiples.apenti ([title, "tickerSymbol", ticker])
return triples
Table 13: An example SCRIBES-generated code discussed in Appendix
24

---

Published as a conference paper at ICLR 2026
Subject
Relation
Object
EDGAR Filing Documents for 1901199419-11--022877
Filing Dute
2010-08-10
EDGAR Filing Documents 0300891189-100 199-00002275
Accepted
2010-08-10
10:58:25
EDGAR Filing Documents (0008941891000010 13-100002275
Documents
EIGAR Filing Documents for (000999188-104002233
Period of Report
2010-06-33
EDGAR Filing Documents for (0908991199-0-0022667
Effectiveness Date
2010-08-10
cmpiricc 63010ng
Seq
(ispinic_6300mme htm
Description
QUARTERLY NOTICE OF PORTFOLIO HOLDINGS
empini 630 60000600000n...
Type
N-Q
Size
530257
certs.
Seq
Inm
Deseription
OFFICER CERTIFICATIONS
certs.htm
Type
EX 99 CERT
cents htm
Size
24222
Deseription
Complete submission text tile
(000894/89-10-0025555
Size
556496
Series S000012964
Name
Core Equity Fund
Class/Contract C00008114
Class/Conract CX00081
Ticker Symbol
EMCCX
ChasoCantmer (X0XX18 15
Name
Class/Contmet 1100001115
Symbol
EMCAX
EIGAR Filing Documents For 18-1000085
EMPIRIC HUNDS.
0000000099 (sce all compuny filings)
INC (Filer) CIK No.
EDGAR Filing Documents 0000891899 89-10002275
742759654
EDGAR Filing Documeats for 0000891189-10 189-00000885
State of Incorp.
TX
EDGAR Filing Documents
Fiscal Year End Type
931
EIGARR Filing Documents 00008891189-144888 889-000028875
N-Q
EDGAR Filing Documcats for 0000891:89-10-0 199-000028855
Act
40
EDGAR Filing Documents 0000894189. 10-087077
File No.
811-09088
EDGAR Filing Documents for 00008911159-0--0000 10-000007
Film No.
101002905
EDGAR Filing Documents for 0000894189 19-00-08875
Business Address
6300 BRIDGEPOINT PARKWAY BUILDING II, SUITE
AUSTIN TX 512328932X11
EDGAR Filing Documents for (000891119-1-0002380
Mailing Address
6300 BRIDGEPOINT PARKWAY BUILDING IL SUITE
105 AUSTIN TX 78730
Predicted
EDGAR Filing
tile
EDGAR Filing Documents for 0000894 189- 10-002875 Form N-Q Quarterly Schedale of partfolio holdings
EDGAR Filing Documents for 0200894189-D08 191-0002875
form
management investment companies:
EDGAR Filing Documents for 0000891189-10
10-007777
accessieNunbber
SEC Accession No. 0000894159-1000 1199-10000000 002875
EDGAR Filing Documents (10099418--1000011)
Filing Date
2010-08-18 10
EDGAR Filing Documents for (0000991889-104444 189-100002875
Acceptied
2010-08-11 10:58:25
EDGAR Filing Documeats for 0300891159-100 189-0000288755
Documents
EDGAR Filing Documents
Report
2010-06-30
EDGAR Filing Documents for 0000894]89-100
Effectiveness Date
2010-08-10
EDGAR Filing Documents for 0001999189-1000
89-10-008875
sequence
EDGAR Filing Documents 010009918991100 199-0002779
description
QUARTERLY NOTICE OF PORTFOLIO HOLDINGS
EDGAR Filing Documents for 0000891159-D00
199-0000287555
document
empiric 63010ng him N-Q
EDGAR Filing Documents
01111944189-1--022225
type
EDGAR Filing Documents
size
530257
EDGAR Filing Documents for 000000044
sequencc
EDGAR Filing Documents For 10011994189-1--077200
description
OFFICER CERTIFICATIONS
EDGAR Filing Documents
1000891159--00002201
document
certs.htm htm
EDGAR Filing Docunscnts for (1000899119-100020200
type
EX-99 CERT 24222
EDGAR Filing Documents 0000894189--000
189-00002855
EDGAR Filing Documents 1000894189-0--00
sequence
EDGAR Filing Documents For 0001894189-100
19-100002755
description
Complete submission text tile
EDGAR Filing Documents 0000094188 8991000012875
document
00000991183 111004188-100-1100 10-1180002875.
EDGAR Filing Documeetts for 0000894189-100 89-100002877
type
EDGAR Filing Documents for 0300894189. 1899941119-100 1S--10002287
size
556499
EDGAR Filing Documents
CIK 0001000069
EDGAR Filing Documents For 1700089418911-000223
name
EDGAR Filing Documents for 0000994189-1100 89-10-022875
tickenSymbol
EDGAR Filing Documents
89-10-012875
status
Class/Cotract CO00000
EDGAR Filing Documents for 0000894159-
10-0028755
name
EDGAR Filing Documents for 0000894188
899-10-002875
tickerSymbol
EMOCX
EDGAR Filing Documents 0000894189.00 00-002875
status
Class/Contract C000008115
EDGAR Filing Documents for 0001894189-10-002888
name
EDGAR Filing Documents for 0000894189 199-000022575
tickerSymibol
EMCAX
Table 14: Comparison of predicted and gold triples for Code
13
J PROMPTS USED
All prompts used in our experiments are shown here in Jinja2 format, including the classifier prompt (Prompt L.M direct extraction prompt [Prompt[T). LLLM-as-ajjudge prompt (Prompt QA prompt (Prompt 18) the main script generation prompt (Prompt [9] used in both baseline and in SCRIBES training data, and the QA evaluation prompt (Prompt 20).
25

---

Published as a conference paper at ICLR 2026
Inetruction
Your tast is to claseity an input RIML to see whether it containe semt-structured concent,
You below wish tollowing
soni-streetered
vithout.
Output Reason shou speat fy your chasn of thought and decision should be one of:
"reason"
Seni-structured contents Respond with "Yes" it ETNG contains seni-structured content,
No somi-structared content: Respend with "No" it the HINL does not ccntain any seni-structured content.
Explicit content: Respona with "Exclude" it the ETN concains explicit content adult
violencel.
input
Exaples containing following HIM:
[1 aXVi_exmmple__ 1)
"reason"s "This EIN, contains cable which falla into the definition of sant-stractured conteat", "dacicion": "Yos"
inpuz
output
"ceason"s "Evnn though this HTML contains atractared discussions and Osda, it doea not have tablea oz infoboxes",
"decicion": "Ho"
An: HIN, with the Following infus
output
"reason"s Mhala RIT'; show dases infobos, which should be treated an swni scructured content." "decision"; PVen?
Inpuc
html
Table 15: Classifier prompt used to determine whether webpage contains semi-structured content or not.
26

---

Published as a conference paper at ICLR 2026
instruck iun
You be doc in ITSS and ise citle, Please return all (subject, predicate, object triples appean ssccions luns Iew farn Lexky you should keep then as jectes. not attempl to break big chunks of cexce down into snaller portione.
descrigtions
object should generally gaines toon Please only inallde complete triplesi if for any section the presicate or obgect missing from the doc, you may skip it. Output List Lists, where inner list is triple, will python''
Supject, precicate,
opens in the doc the
Inpu:
it it example__loba L.llbaa__hh Lhnnl triples NJ
Here are (1 exengle global heml triplee length H1 examples of flattened IIT'', and their expected triples: hatplll Exanple iI Losp. indexu 1I Elatzenes HTML 11 eirgle
kingle_snapple
eerggleexxoppeeee exonple| "htnl, Hlateen") 12
"xample If leop, indexd 11 Expected Triples est if single,ess exarplal perriplesannntatee annotation") endror
(t 1f exampic_cripien 6]
Stre 10 triples expecting andi-
randonly chosens cample_triples
#4 title hemi_titic
#* HTMG
11 htmL 3F
Table 16: LLM direct extraction prompt used to directly generate triples from a webpage.
instruction
You given Lwo ject, powdicate, ob ject) Leiples. Your should be "Yos" triples aenartically the thay somaatically diffarens.
inpuc -2
Table LLM-ae-a-judge prompt for judging whether two triples are semantically equivalent.
inat ruch fon
are given question reterense may may not holp answer quastion. Please answer the question, Be concise.
quest ton
tte Raterence
reterence
Table 18: Question Answering prompt with reference.
27

---

Published as a conference paper at ICLR 2026
instruction
Your task is to generate senancic triples fron giver. HIL.
tripic contains subfect, predicate, and object. You write python
extract tripies The final exeautable fonction ahould be called 'def main nain (himi) -> Lisc[tupla(ate, ate, acallry wtare it will output 11st of triplet.
STM.
the python should any text other
only, Feul free code your reaponse.
explain your code.
DPOOTTNT: will zeruse the case sccipt. far other webpages with similar HTG contencs. ra-usable
should make not hardcads fer values this particular HIML!
different vabsites
inpu:
exanple oloballllllllloooooooooao beatl triples 1)
aare are (1 excmole_cios wataaaaattntitttooo tripies length 11 exampies of other ard aites and what the script generaled ed out
looking fus:
s_global__hnn______ stripeeetriee Exampie I[ leop, indexo 1) RTHL: 11 single_pxamola [" 1)
single_eranp
in
Example If loop, indedd 1] Expected Outgutat (f siagle_ssappee (at.sselllll"ippleeeeo aisiiple..nmttttiirrrr
(1 endfor 1)
andi
STM:: (1 html 11
LI example ipllss
aze 10 tziplea are expecting outpuz zandonly chosen: examplo_triple
19 wndi 91
(5 if Lri_triples 1)
the triples 41
expacting octpet: ait_triples
if script
previnuxly generated scrapt: prev_seript. []
This script generated the fol loling result:
teedback
1F you think the resulta asw guod enough, stop and cutput the same script. not, incorporate the reedback generating ecript.
endif
Table 19: Main script generation prompt for baselines and SCRIBES-traine models.
instruction
You need to check whether the prediecion of queat iorvvvaarin system to question is correct. truch grovided
You should fudmmnt based
Your response should be "corract" if the prediction is correct oz "Locrractt" 12 the prediction is wrorg.
Impus
Quastions 11 question }1
Ground Lruths [l geld }]
Psadiction:
Correctness:
Table 20: QA evaluation prompt.
28


Reviews:
[{"review_id": "BqSGUsiPu5", "reviewer": "Reviewer_oCsj", "summary": "The paper addresses the task of knowledge extraction, i.e. extraction of triplets (subjects, predicates, and objects), from semi-structured web content such as tables and infoboxes. This paper proposes to extract such knowledge with LLM-generated extraction script that is designed to be applicable to similar pages from the same source website. Furthermore, the authors use reinforcement learning (RL) to train LLMs to produce these scripts. The results show that for the 14B and 32B models, this script generation approach notably outperforms direct LLM-based direct extraction, and the application of RL provides a subsequent performance gain.", "questions": "As in weakness", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Script-based knowledge extraction holds promise for significantly improving the speed and efficiency of the extraction process compared to direct, single-instance LLM extraction.\n2. The application of RL brings a clear and significant improvement in performance compared to the base models.", "weaknesses": "1. While the extraction of knowledge triplets can ideally benefit downstream tasks such as knowledge-intensive question answering, I am not completely convinced that it is necessary. The knowledge triplet is a relatively narrow format, and may not be sufficient to comprehensively represent the knowledge. A promising alternative could be to directly use the source data through RAG.\n2. The reported performance seems inconsistent. For example, the script generation method using the gpt-oss-120b model appears to significantly lag behind the results achieved by direct extraction with the same model.\n3. The comparison against established baselines is not comprehensive enough. I would love see a more thorough comparison, including against the numbers previously reported for the SemiBench paper itself, and other related baselines such as specialized relation extraction models, LLM-based relation extraction agents, etc.\n4. Could the authors present more details regarding the results in Section 5.1? Where does the QA data come from, and how is it selected?\n5. I would also appreciate more implementation details about HTML flattening and cleaning, and whether different strategies were explored. These pre-processing steps can have a dramatic impact on both the performance and the resulting behavior of the extraction system.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A7l67HREFL", "reviewer": "Reviewer_yg8p", "summary": "The paper proposes a web-scale structured information extraction framework called SCRIBES. Instead of calling a large model on every single webpage, the key idea is to first let an LLM generate a reusable extraction program (a Python script) and then apply this script to many structurally similar pages. To make the script generalize, the authors use reinforcement learning on a group of webpages to produce scripts that work across pages. Experiments show that, compared to existing script-generation / agentic baselines, their RL-based training improves group-level reusability, and the extracted triples can also benefit downstream HTML QA.", "questions": "* Could you include one or two representative examples of the generated Python extraction scripts in the appendix , so that readers can better understand what the model actually produces?\n* Is it feasible to integrate a lightweight Open IE (or similar semantic post-processing) component into the generated script to improve its ability to handle less strictly structured content?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Well-motivated problem. Web-scale information extraction is an important and realistic task, and the cost of applying an LLM to every single page is prohibitive; reducing LLM calls is therefore a meaningful objective.\n* Novel paradigm. By generating a script for each layout type (i.e., each group of structurally similar webpages), the method reframes the task from page-level IE to program induction and reuse, clearly setting it apart from pure agent-style, one-page-at-a-time extraction.", "weaknesses": "* Absolute performance gap. Despite gains over script-generation baselines at comparable model sizes, the approach still trails the strongest page-wise direct LLM extraction (e.g., GO-120B with flattening).\n* Fragile grouping assumption. The heuristic “same domain + similar URL prefix ⇒ similar structure” may not hold on sites with mixed templates, limiting robustness and generalization.\n* Bias toward structure-heavy pages. Performance degrades on free-form, text-heavy pages that require deeper semantic understanding.\n* Insufficient deployment analysis. A clearer cost–accuracy trade-off (e.g., a curve or table) is needed to show when the script-based approach is preferable to per-page LLM extraction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wAnEllWdME", "reviewer": "Reviewer_vXwT", "summary": "In this paper, the authors target the problem of semi-structured data extraction.\nThey develop a script-based semi-structured content extraction at web-scale framework to improve generalization of this task. The proposed reinforcement learning framework leverages layout similarity across webpages within the same\nsite as the reward signal. The proposed approach outperforms strong baselines in script quality and boosts downstream question answering accuracy.", "questions": "1. A \"script\" is essentially akin to a type of automatically generable rule, yet the paper does not clearly distinguish it from traditional wrappers, templates, or XPath rules.\nThere is a potential risk of overstating the method's novelty. It is necessary to more explicitly articulate the advantages of RL-generated scripts—such as generalization ability and automatic learning strategies—highlighting their unique value compared to manually crafted rules.\n\n2. The paper emphasizes black-box data analysis while providing limited discussion on the limitations of the DSL and the handling of anomalous webpages. Its capability to adapt to irregular structures—such as multi-row/multi-column tables, nested lists, or JavaScript-rendered content—remains unclear.\n\n3. The paper claims support for \"web-scale\" and multi-domain scenarios, yet it does not present the domain differences between the webpages used for training and those used for testing. Including such statistics would make the experimental conclusions more convincing.\n\n4. The selected baselines appear relatively weak. Many other LLM-based methods are mentioned in Section 1; it remains unclear how the proposed method performs in comparison with them.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper addresses an important and practical problem:  extracting structured data from the vast amounts of semi‑structured web content.\n2. The framework is novel in that it focuses on webpage grouping and employs a semi-supervised-like approach, enhancing the model's ability to generate highly reusable scripts, thereby improving task performance while reducing costs.\n3. The experimental results reveal that the designed framework is efficient and effective in both generating robust, reusable scripts and improving performace on downstream tasks.", "weaknesses": "1. The advantages of script-based extraction need to be further elaborated.\n2. The domain scalability of the method requires further elaboration.\n3. The evaluation baselines seem to be somehow weak.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0ueltHi2ye", "reviewer": "Reviewer_TGsp", "summary": "The paper introduces SCRIBES, a reinforcement learning (RL) framework for scalable information extraction from semi-structured web data (HTML tables, lists, infoboxes). Rather than invoking a large language model (LLM) for each page individually, SCRIBES trains an LLM to generate reusable extraction scripts that can be applied to groups of structurally similar webpages (e.g., pages from the same site).", "questions": "How sensitive is performance to noise in synthetic LLM-based rewards? Any quantitative correlation between F_fuzzy and F_LM?\n\nDoes a model trained on finance/legal tables transfer to product or encyclopedia pages?\n\nWhat is the total compute cost (training + inference) compared to a standard per-page LLM pipeline?\n\nCould you show examples where the generated scripts fail catastrophically (e.g., mismatched tags or wrong data fields)?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The script-generation idea bridges wrapper induction and LLM-based extraction, offering a path to web-scale efficiency.\n\nThe use of cross-page generalization as a verifiable RL reward is elegant and practical.\n\nThe empirical results also show some promising evidence.", "weaknesses": "The system largely combines existing ingredients (RLVR, GRPO, LLM-as-a-judge, fuzzy matching) rather than introducing a fundamentally new algorithm. This is my biggest concern to give a positive score. The “fuzzy F1” and LLM-judged metrics may still be noisy; no human verification of RL reward quality was reported. In addition, although SCRIBES aims for efficiency, it still relies on large proprietary models (e.g., GPT-4o, Qwen-32B) for training and reward calculation, limiting replicability and true scalability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)