# Revisiting the Design of Parallel Stream Joins on Trusted Execution Environments
## 5 Profiling Study
### Preliminary Results  
In the following, we show how the three design aspects (i.e., execution approaches, partitioning schemes, and join methods) interact with TEEs-enabled multicore processors.  
**The Effect of Varying Execution Approaches**. In practice, most off-the-shelf sort-merge algorithms cannot be readily ported on an SGX enclave without anticipating a considerable performance loss. Sort-merge algorithms require sorting the entire relations first, which can prove troublesome with very large datasets given the limited enclave memory available. Nonetheless, algorithms which have been optimised for NUMA architectures such as MWAY show considerable flexibility that can be leveraged in TEE. Nonetheless, as illustrated in Figures 5 and 6, lazy approaches in joining dramatically impact performance. We observe an exponentially increasing gap in both latency and throughput between the insecure and secure processing nodes as the number of joins to be processed increases. While the native mode is able to achieve up to 250,000 tuples/s on the YSB dataset, the SGX counterpart achieves less than 60,000 tuples/s. The significant throughput differences for different input sizes are mainly due to enclave memory swaps operations as the large data cannot be loaded and processed within the limited memory available to the enclave. In contrast, aggressive eager approaches also suffer from enclave call performance overhead that adds up for in each subset of tuples processed. Hence, a well informed hybrid approach needs to be considered.  
**The Effect of Varying Partitioning Schemes**. In order to evaluate the impact of the varying partitioning schemes configurations, we first run mini-batches of different size streams \(R\) and \(S\), such that: \(M_{R}\times M_{S}=10^{6}\). Such a small number of inputs fits entirely within the enclave memory. Averaging the results over 20 runs, we notice that square partitioning (where \(M_{R}=M_{S}\)) consistently yields minimal latency (up to \(10\times\) less), whilst all other configurations fit within the reverse bell curve illustrated in Figure 7. This behavior is consistent across datasets and execution approaches.  
**The Effect of Varying Join Methods**. In Figures 5 and 6, we observe that with a few exceptions, SH generally outperforms MWAY in terms of throughput and latency for smaller workloads. This native behavior is carried forward to the secure implementation where small input sizes achieve a performance that is comparable to native runs. For larger workloads requiring EPC paging, MWAY appears to have a slight edge.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & \begin{tabular}{c} **Tuple Size** \\ **(in Bytes)** \\ \end{tabular} & \begin{tabular}{c} **Attributes** \\ **Joined** \\ **Oven in R** \\ \end{tabular} & \begin{tabular}{c} **Attributes** \\ **Joined** \\ **Oven in S** \\ \end{tabular} \\ \hline \begin{tabular}{c} Rovio \\ \end{tabular} & 30 & 1.2 & 1.2 \\ \hline \
\begin{tabular}{c} YSB \\ \end{tabular} & 100 & 1 & 1 \\ \hline
\begin{tabular}{c} DEBS \\ \end{tabular} & 5000 & 3.5 & 4.6 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Join attribute and size of tuple per dataset.  
Figure 5: Average Latency with Different Processing Volume.  
Figure 6: Average Throughput with Different Processing Volume.