# A FAIR Resource Recommender System for Smart Open Scientific Inquiries
## 2 Related Works
### Novelty of FAIRFind and DWCP  
Conventional deep web crawlers, like Google's deep web crawler, Form-Focused Crawler, and ACHE, all emphasize detecting and submitting HTML forms to access hidden content [27; 28; 29]. Google's crawler is heuristic-driven and domain-agnostic, while Form-Focused and ACHE are focused crawlers, using classifiers and adaptive learning to guide form discovery within specific domains. They often failed to understand the semantic context of form inputs, struggled with multi-step interactions, and generated non-reproducible access paths. In contrast, FAIRFind leverages large language models to semantically interpret forms, infer meaningful input values, and construct human-readable, machine-executable workflows that can be reused reliably. Key innovations include the following:  
* Uses LLMs to understand field semantics and generate contextually valid inputs, avoiding brute-force submission strategies. * Captures detailed access paths and form behaviors into persistent DWCP records, ensuring that interactions can be reproduced by other agents. * Filters irrelevant or non-scientific forms through LLM-guided semantic classification, increasing precision of data discovery. * Allows for operating at scale, unlike ad hoc or session-bound systems, because it uses an agent-based design with asynchronous execution and message queues. * Bridges FAIRsharing metadata and actionable endpoints to make resources not just discoverable but also machine-interpretable and reusable. * DWCP formalizes interactions in JavaScript Object Notation (JSON)-based specifications with semantic and operational details, supporting optional features like message IDs, session tracking, and signatures for robust, transparent workflows. Recent LLM-driven web agents such as AutoWebGLM, Steward, YURAScanner, BardeenAgent, Skyvern, and WebVLN-Net (Table 1) have explored diverse strategies for automating web tasks, combining text- and vision-based perception with innovations like curriculum learning, caching, semantic abstraction, and vision-first navigation. However, these systems are primarily oriented toward task efficiency, RPA, security auditing, or human-like browsing, and do not generate standardized, reusable, and FAIR-compliant interaction specifications. \begin{table}
\begin{tabular}{p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt}} \hline \hline
**Agent System** & **Base LLM (s)** & **Perception Modality** & **HTML Handling Strategy** & **Core Innovation/Training Method** & **Primary Use Case** \\ \hline AutoWebGLM [33] & ChatGLM3-6B & Text (HTML) & Custom simplification algorithm & 3-step training & (Curriculum Learning, RL, Rejection Sampling) & General Web Navigation \\ Steward [34] & GPT-3.5/4 & Text + Vision & 3-step cleaning pipeline (filter, match, clean) & \
\begin{tabular}{c} Caching of action \\ sequences for cost/runtime optimization \\ \end{tabular} & Scalable Web Automation \\ YURAScanner [41] & GPT-4 & Text (Abstracted HTML) & Semantic abstraction of interactive elements & Task-driven crawling for security vulnerability discovery & Web Application Security Testing \\ BardeenAgent [37] & Pluggable (e.g., GPT-4) & Text (HTML) & Leverages regular structure of lists/tables &
\begin{tabular}{c} Record/Replay \\ mechanism generates \\ executable programs \\ \end{tabular} & Structured Data Extraction at Scale \\ Skyvern [35] & Vision LLMs & Vision (Screenshots) & Bypasses HTML parsing for visual interaction & Vision-first approach for resilience to & Robotic Process Automation (RPA) \\ WebVLN-Net [44] & VLNBERT & Text + Vision & Extracts clickable elements from HTML & Fuses visual and HTML to emulate human browsing & Vision-and-Language Navigation \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparative analysis of modern LLM-based web agent architectures. Each system combines different LLMs, perception modalities, and innovations to address diverse web automation and navigation challenges.