# A FAIR Resource Recommender System for Smart Open Scientific Inquiries
## 3 The Deep Web Communication Protocol
### Message Structure and Data Model
\begin{table}
\begin{tabular}{l l} \hline \hline
**Feature** & **DWCP** \\ \hline Data format & JSON \\ Mandatory fields & version, type, target, access\_type, payload related fields \\ Optional extensions & message\_id, timestamp, session\_id, signature \\ Supported access\_types & form, htal\_table, file\_download \\ Transport & HTTP (S) or message queues \\ Execution & Fully reproducible, executable by agents \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Summary of DWCP as a protocol, highlighting its structure, supported features, and robustness for agent-based execution. ### Metadata Harvesting

We begin by collecting metadata from the FAIRsharing registry, which lists 2349 curated scientific databases spanning diverse domains. FAIRsharing provides structured records that describe data policies, access methods, and process pages for each entry. From these records, FAIRFind extracts descriptive fields such as database name, summary, creation year, homepage, and the set of associated data access links. In order to distinguish between user-facing web interfaces, downloadable file endpoints, and FTP repositories, each access link is examined to identify its type. We exclude entries with REST API endpoints, as programmatic APIs are outside the scope of this study. During processing, we encountered several records whose access links failed to load or returned invalid content. These were likely due to outdated or deprecated process pages and were excluded from further processing. After filtering and validation, our final dataset includes 924 FAIRsharing entries with 1800 data-containing web pages. These pages form the basis of the downstream crawling, access abstraction, and semantic indexing pipeline. ### Form Discovery and Interaction Modeling

On the web, a large portion of scientific data remains hidden behind interactive interfaces like forms, filters, and custom input panels that need human intervention to access the underlying datasets. It can be difficult to index, query, or reuse these inter

Figure 1: FAIRFind system architecture and workflow. The system consists of a data indexing pipeline (**top**) and a query engine (**bottom**). In the pipeline, metadata from FAIRsharing is used to discover web pages, extract forms, tables, and FTP schemas, and generate machine-actionable workflows using LLMs. These workflows are stored in an indexed dataset. In the query engine, user queries are matched to relevant schemas, and the system retrieves structured data by parsing tables or submitting forms with LLM-inferred inputs. faces programmatically because they are usually created for user experience rather than automated systems. Finding these interactive forms, comprehending how they function, and creating a reusable representation of the interaction pathway are the main goals of this pipeline stage. We call this an access path abstraction. The process unfolds through four core stages: detecting interactive inputs, semantic filtering, form simulation, and constructing access paths. #### 4.3.1 Detecting Interactive Inputs

The system starts by loading a specific web page, making sure that all client-side content is completely rendered. It then performs a thorough scan to find possible locations for data entry. This comprises text fields, dropdown menus, input fields, form elements, and submit buttons. After a form is detected, it is parsed into a structured candidate block that includes all related input fields, the action endpoint, and the HTTP method (GET or POST). The name, type, and XPath expression of each field are annotated. An initial collection of interaction candidates is the end result, but many of these are later filtered as irrelevant, such as newsletter sign-ups or login forms. #### 4.3.2 Semantic Filtering and Selection

The system analyzes and filters the previously identified candidates using a language model to make sure we concentrate on relevant forms. The forms on the page that are most likely to produce organized scientific results are given priority by this model.