# A FAIR Resource Recommender System for Smart Open Scientific Inquiries
## 3 The Deep Web Communication Protocol
### Message Structure and Data Model
The resulting output includes headers and sample rows, and the access type is set to \"html-table\". * When data are provided via downloadable links (e.g., CSV, XLS), the system downloads and parses the file. The first row is treated as headers, and the next two rows are extracted as samples. Regardless of access type, each output is then passed to the embedding and indexing module. This step finalizes the conversion of human-facing interfaces into machine-actionable, FAIR-aligned data entries. In the following stage, these records are vectorized and stored in a retrieval-ready format to support semantic search and ranking. Algorithm 1 describes the complete indexing pipeline used to convert FAIRsharing records and their associated access links into structured, queryable DWCP metadata entries. The process is designed to be modular, extensible, and compatible with a wide variety of data delivery mechanisms found throughout the scientific deep web. The algorithm operates over a set of records \(\mathcal{R}=\{r_{1},r_{2},\ldots,r_{n}\}\), where each record contains a homepage and one or more access links. For each access link \(l_{ij}\), the pipeline applies a branching logic depending on its declared access method, FTP or user interface (UI), as annotated in FAIRSharing entries. This distinction is essential, as each access mode requires a distinct handling strategy. ```
Input: Set of FAIRsharing records \(\mathcal{R}=\{r_{1},r_{2},\ldots,r_{n}\}\) Output: DWCP metadata index \(\mathcal{M}\) containing structured access objects
1 foreachrecord\(r_{i}\in\mathcal{R}\)do
2 Extract homepage \(h_{i}\) and access links \(\{(l_{ij},m_{ij})\}_{j=1}^{k}\);
3 foreachlink\(l_{ij}\) with access method \(m_{ij}\)do
4 if\(m_{ij}=FTP\)then
5 Download file from \(l_{ij}\) and parse into \((\text{headers},\text{sample\_rows})\);
6 Construct DWCP object \(o_{ij}\) with \(\text{access\_type}\leftarrow\text{ftp}\);
7
8 elseif\(m_{ij}=UI\)then
9 Load \(l_{ij}\) in headless browser to retrieve HTML \(c_{ij}\);
10 Extract all forms \(\mathcal{F}_{ij}\) and interactive inputs from \(c_{ij}\);
11 if\(\mathcal{F}_{ij}\
eq\emptyset\)then
12 Use LLM \(\Lambda\) to select and summarize target form \(f^{*}\in\mathcal{F}_{ij}\);
13 Generate sample input values \(v^{*}\) using inference model \(\Gamma\);
14 Simulate interaction using DOM mapping to produce rendered page \(r_{ij}\);
15 Parse \(r_{ij}\) to extract table headers and sample rows;
16 Construct DWCP object \(o_{ij}\) with:
17 access_type \(\leftarrow\) form
18 headers, sample_rows
19 form_summary and access path abstraction
20
21 else
22 Scan \(c_{ij}\) for static HTML tables;
23 iftable foundthen
24 Extract \((\text{headers},\text{sample\_rows})\);
25 Construct DWCP object \(o_{ij}\) with \(\text{access\_type}\leftarrow\text{html\_table}\);
26
27 Embed \(o_{ij}\) into vector space via encoder \(\phi(o_{ij})\in\mathbb{R}^{d}\);
28 Store \(o_{ij}\) and its embedding in metadata store \(\mathcal{M}\);
29
30 return\(\mathcal{M}\)
```
**Algorithm 1**DWCP-based Deep Web Indexing Pipeline

For links labeled with the FTP access method, the system directly downloads and parses the associated file. A rule-based parser extracts the header and a small sample of rows. This extracted information provides sufficient semantic content to populate a DWCP object containing representative metadata and example data. The most common and complex case involves user interface (UI) access links, which point to dynamic web pages requiring full rendering and interaction. First, the target page is loaded, ensuring that all dynamic content is rendered and the full Document Object Model (DOM) is available for analysis. Once the page is fully rendered, the system extracts all form elements and free-standing input controls, parsing them into structured candidates for potential interaction. These candidates are then assessed by a locally hosted open-source language model, which chooses the form that is most likely to produce structured data and provides a brief overview of its intended purpose.