# A FAIR Resource Recommender System for Smart Open Scientific Inquiries
## 3 The Deep Web Communication Protocol
### Message Structure and Data Model
Additionally, it generates a brief summary in natural language that explains the specifics and intent of each form. This synopsis is essential for providing retrieval systems' search space with context. In addition to classification, the model generates inputs intended to yield reliable results by inferring sample values. This semantic reasoning is driven by a structured prompt that combines the HTML context with parsed form metadata. The output at this stage provides the selected form's identifier, its semantic description, and a set of sample inputs. #### 4.3.3 Simulating Form Interaction

After a target form is selected and populated with sample inputs, the system simulates its submission. Forms demand full DOM-level interaction. To achieve this, the system uses the previously recorded XPath references to pinpoint relevant input fields and submit buttons. A headless browser then programmatically fills out and submits the form. This interaction handles JavaScript-based redirects, AJAX updates, and other dynamic elements common in modern web environments. The final rendered HTML from this submission is then captured for subsequent data extraction. #### 4.3.4 Constructing Access Path Abstractions

To ensure that form interactions can be reliably repeated and interpreted by other systems, the system records each successful submission as a structured access path abstraction. This object includes the HTTP method used (GET or POST), the full action URL of the form, the list of fields and sample values submitted, and metadata such as XPath expressions or DOM attributes for fields and buttons. An example of this access path abstraction, as stored in the DWCP output message, is shown in Listing 3. ```
''form_method'''':''post'', ''input_1'':{ ''name'':''search-box'', ''xpath'':''/*(@id='search-box']'', ''value'':''CRISPR'' }, ''button_1'':{ ''xpath'':''/*(@id='submit']'', ''value'':''Search'' } }
```
This structure enables downstream agents to re-execute the interaction using only the information encoded in the DWCP message. ### Data Extraction and Access Normalization

Data extraction from the resultant web page comes next, once form interactions are reproduced or direct links are resolved. This stage creates a normalized schema from the visual or code-level representation of a dataset, thereby preparing it for indexing, search, and reuse. This module performs three main tasks: identifying data structures, extracting headers and sample rows, and packaging the information into a DWCP-compliant output format. It is designed to handle both static content, like pre-rendered HTML tables, and dynamic content, such as tables loaded via JavaScript or rendered after a form submission. #### 4.4.1 Table Detection and Parsing

The rendered page is parsed using a dual strategy. If HTML-table elements are present, a rule-based parser is applied to extract column headers and a small number of representative data rows. We use a fallback language model-based HTML interpreter for more complicated pages whereby tables are dynamically rendered or styled using various containers. Even in cases when traditional tags are absent, this LLM-assisted method heuristically detects tabular areas and the structured content of extracts. For each discovered table \(t_{i}\), we constructed a representation:

\[t_{i}=(\text{headers}_{i},\text{rows}_{i}) \tag{2}\]

where headers\({}_{i}\) is a list of column names and rows\({}_{i}\) contains two representative entries sufficient for embedding and preview. Each extracted data unit is wrapped into a DWCP-compliant output object, regardless of how it was retrieved, whether through form submission, direct table access, or file download. #### 4.4.2 Access Normalization Across Types

The extraction logic adapts based on the type of access mechanism. * Data are extracted from the page rendered after simulating a completed form submission. The DWCP output includes the full form object, and the access type is set to \"form\". * Pages containing static tables are parsed directly without interaction.