# Partitioning Strategies for Parallel Computation of Flexible Skylines
## 5 Experiments  
In this section, we measure the efficiency of the proposed algorithmic pattern for the computation of both ND and PO. We measure our indicators of efficiency in different settings according to the operating parameters described in Table 1. In particular, we target anticorrelated datasets, which are the most challenging in any skyline-like scenario. The set \(\mathcal{F}\) of functions considered for computing flexible skylines is simply characterized by the constraint \(w_{1}\geq w_{2}\), which prunes 50% of the space of weights and can be applied to any dataset with at least \(d\geq 2\) dimensions. Our experiments are conducted on a computational infrastructure with Spark comprising virtual machines equipped with a total of 30 cores and 8 GB of RAM. We observe that the times measured in this experimental setting are very different and should not be compared with those obtained in [3], but should rather be used to appreciate the trends of improvements offered by parallelization. When running on PySpark, execution times can be up to two orders of magnitude slower compared with a single-machine setup using a compiled language like Swift. This slowdown is primarily due to the overhead introduced by PySpark's distributed framework, which involves data serialization, inter-process communication, and task scheduling. Additionally, Python's interpreted nature and reliance on the JVM for Spark execution further contribute to inefficiencies, especially when compared with Swift's direct compilation to a native machine code and its optimized memory usage, which eliminates the need for a virtual machine or garbage collection overhead. **Varying the dataset size. ** Our first experiments test how efficiency varies as we vary the dataset size \(N\) and keep all other parameters set to a default value, i.e., \(d=4\) dimensions, \(p=100\) partitions, and \(c=30\) cores. \begin{table}
\begin{tabular}{l l} \hline \hline
**Full Name** & **Tested Value** \\ \hline Dataset size (\(N\)) & 200 K, 500 K, **1 M**, 2 M, 5 M, 10 M \\ \# of dimensions (\(d\)) & 2, 4, 6, 7 \\ \# of partitions (\(p\)) & 10, 50, **100**, 150, 200, 300 \\ \# of cores (\(c\)) & 5, 10, 20, **30** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Operating parameters for testing efficiency (defaults in bold). Figure 3a shows that, while, of course, execution times grow as the size grows, the Sliced strategy is much more efficient than Angular and, especially, Grid in exploiting parallelism during the computation of ND with the SVE1F algorithm. In particular, Figure 3b shows the time spent during the parallel phase, highlighting how Grid underperforms and is inefficient in that many partitions are unbalanced, while the other two partitioning strategies are much more balanced in this respect. To compare our results with a sequential baseline, Figure 3a also shows the time incurred by a non-parallelized execution with SFS (indicated as Serial), which is clearly more costly than all our parallelized scenarios. Finally, Figure 4 shows the percentage of tuples that are removed during the parallel phase, highlighting again that Grid is the least efficient choice, while Angular has the highest trimming power, although it is overall less efficient than Sliced because it pays an overhead due to the partial imbalance between partitions (which, instead, are perfectly balanced in case of Sliced). Table 2 shows the breakdown of the execution times between the parallel and the sequential phase, along with the number of tuples surviving after the parallel phase, for a 4d dataset with 2 M tuples, confirming the highest benefits of the Sliced strategy. As also observed in Figure 4, the first numeric column of the table reports that, after the parallel phase, between 1.4% and 2% of the initial 2 M tuples are retained. We perform a similar analysis with PO using the POPI2 algorithm.