# Ad Hoc Data Foraging in a Life Sciences Community Ecosystem Using SoDa +
## 3 SoDa Architecture
### Application Design
#### 3.4.1 SoDa Query Language  
QBuild uses our previously developed BioFlow query language construct extract to access the referenced tables in real time and treats the extracted tables as traditional tables in the back-end relational database, MySQL. The extract statement has a general form, as shown in Figure 5.  
In this statement, a wrapper is a function that hides the internal details of a database or web site and offers an abstract view of it. In this view, a deep web database or a web site can be viewed as a table returned by the database or a site when it is supplied with a set of input values. For example, the table in Figure 3 is returned by the DisGeNet database when the GeneID 29974 (gene symbol A1 CF) is submitted to it. A wrapper would accept this GeneID and scrape the table from this HTML page and return a table in.txt or.CSV format. A schema matcher, on the other hand, would resolve the schema disparities between two views. For example, a user may refer to a column name as Gene, while the DisGeNET database may refer to it as G_ID. A schema matcher may establish the correspondence that says Gene and G_ID are identical. Finally, a filler is a function that helps mimic form filling in web sites, such as submitting the GeneID 29974 to the DisGeNET database form at the following URL: [[https://disgenet.com/](https://disgenet.com/)]([https://disgenet.com/](https://disgenet.com/)) (accessed on 1 December 2024). For any table \(r\), the set of column names by which it is defined is called the scheme or \(r\), denoted as \(r(R)\) where \(R\) is the set of column names. The extract statement above returns  
Figure 4: SoDa wrapper bank: the wrappers generated using CroW, as in Figure 3, can be materialized and stored for posterity. The table in this picture shows a searchable index of materialized wrappers ready to be used in user applications. Figure 5: BioFlow extract statement syntax: the extract statement can retrieve and return a table with columns \(A_{1},A_{2},\ldots,A_{n}\) from a deep web database at the URL \(\varphi\) by submitting each row in table \(r\) as parameters to the database. Schema disparities are resolved automatically using the schema matcher \(M\), and tables are extracted using the wrapper \(W\). If a form filling operation is required, it is carried out using the filler \(F\). a table \(t\) with column names \(A_{1},A_{2},\ldots,A_{n}\) when it is supplied with a table \(r\) with columns \(B_{1},B_{2},\ldots,B_{m}\). The scheme \(S_{o}=\{A_{1},A_{2},\ldots,A_{n}\}\) is called the output scheme of the extract statement, while the scheme of \(r\), i.e., \(S_{i}=\{B_{1},B_{2},\ldots,B_{m}\}\), is called the input scheme of the extract statement. BioFlow approaches the deep web resource querying problem declaratively. In this approach, a deep web database is modeled as a function in an internet repository \(\varphi\). This function, when prompted with a table \(r_{i}\) over a scheme \(S_{i}\), returns a table \(r_{o}\) over a scheme \(S_{o}\). The extract statement above provides the syntax needed to implement this function. The interesting and unique component of the extract statement is its using clause. This option specifies the set of required tools needed to access a hidden web database. The application of these tools provides the needed abstraction of the function and hides the complexity of retrieving data resolving possible schema disparity using a schema matcher and possible scraping of tables (in case it is returned as an HTML document) using the wrapper \(W\) from the returned document. The wrapper essentially captures all necessary access details, as discussed in Section 3.3, including converting returned data by database \(d\) at \(\varphi\) into a tabular form.