# Query Rewriting for Incremental Continuous Query Evaluation in HIFUN
## 4 Methods
### System Implementation
#### 4.2.3 Translating Continuous HIFUN Queries to SQL  
The abstract definition of a query evaluation can be realized when the involved dataset \(D\) is stored in an unbounded append-only relation table; and also, mapping this abstract definition to the existing physical-level mechanism using the semantics of the SQL exploiting group-by SQL queries of Spark Structured Streaming. The basic idea in Structured Streaming is treating continuously arriving data as a table that is being continuously appended. Structured Streaming runs in a micro-batch execution model as well. Spark waits for a time interval and combines into batches all events that were received during that interval. The mapping mechanism defines a query on the input table, as if it were a static table, computing a result table that will be updated through the data stream. Spark automatically converts this batch-like query to a streaming execution plan. This is called incrementalization: Spark figures out what needs to be maintained to update the result each time a new batch arrives. At each time interval, Spark checks for new rows in the input table and incrementally updates the result. As soon as a micro-batch execution is complete, the next batch is collected and the process is reapplied. Generic Evaluation Schema to SQL  
In [30; 31] it was already proven that HIFUN queries can be mapped to SQL group-by queries. In general, for the query \(Q=(g_{A},m_{B},op)\), two cases are distinguished. **Case 1**. The attributes A and B appear in the same table (e.g., T). In this case we can obtain the answer of Q using the following group-by statement of SQL. \[Select\ A,op(B)\ as\ ans_{Q}(A)\ From\ T\ GroupBy\ A \tag{3}\]  
**Case 2**. The attributes A and B appear in two different tables (e.g., S and T). In this case we can obtain the answer of Q using the following group-by statement of SQL. \[Select\ A,op(B)\ as\ ans_{Q}(A)\ From\ join(T,S)\ GroupBy\ A \tag{4}\]  
To this end, next we present examples of mapping analytic queries directly to SQL. We shall use the context of the Figure 5 and we assume that the dataset is stored in the form of a relation using the star schema shown in that figure. In general, a star schema includes one or more fact tables, indexing a number of associated dimension tables. In our example, this star schema consists of the fact table FT and two-dimensional tables: the dimensional table \(DT_{Branch}\) of the branch and the dimensional table \(DT_{Product}\) of the product. The edges of the context are embedded in these three tables as functional dependencies that the tables must satisfy, and the underlying attribute in each of these three tables in the key of the table. Our implementation handles the above relation schema as follows: The fact table is represented as an unbounded table containing the primary incoming streaming data, and the dimensional tables \(DT_{Branch}\) and \(DT_{Product}\) are represented as static tables which are connected to the fact table. By using static dimensional tables, we avoid the stream-stream joins. The problem of generating inner join results between the two data streams is that, at any time, the view of the dataset is incomplete for both sides of the joining, making it inefficient to find the matching values between two inputs data streams. Any row received from the input stream can match with any future not yet received row from the other input stream. Thus, the solution for this is to buffer the past input as a streaming state to match every future input with past input and accordingly to generate join results. As such, our implementation supports joins between a streaming and static relational table. In this setting, consider the query \(Q=(b,q,sum)\) in the context of Figure 5, asking for totals by branch. In this query, the grouping and measuring attributes appear in the same fact table. This query will be mapped to the following SQL query:  
**Select Branch, sum(Quantity) As \(ans_{Q}(Branch)\) From** FT
**Group by** Branch  
Next we present another example.