# A New Big Data Benchmark for OLAP Cube Design Using Data Pre-Aggregation Techniques
## 3 Method
### The Apache Kylin Case
#### 3.3.1 Cube Design for Apache Kylin
Kylin supports two dimensions types:  
* **Normal:** A dimension that will be combined with the other \"Normal\" dimensions and facts in order to generate data pre-aggregation. These dimensions fully contribute to the Curse of Dimensionality. For example, the measure Credits Approved, combined with the Degree (Computer Science, Maths) and Academic Year (2016, 2017) dimensions, will be pre-aggregated and stored on the OLAP cube for the combinations (Computer Science, 2016), (Computer Science, 2017), (Maths, 2016), (Maths,2017). This will reduce the aggregation needed during query execution time, thus reducing query latency. However, \"Normal\" dimensions must be defined carefully, as they lead to a combinatorial explosion between dimensions. * **Derived:** Often dimensions are stored in tables separated from fact table, as in Star or Snowflake schemas [1]. In these cases, the dimension table columns needed to determine the relationship between this table and the fact table are called Surrogate Keys (SK) [1] and Kylin manage them as \"Normal\" dimensions by default. However, the remaining columns of a dimension table can be defined as \"Normal\" or \"Derived\" dimensions. If we define a dimension column as \"Derived\",  
Figure 2: Apache Kylin architecture. this dimension is not used for pre-aggregation, thus it has no effects on Curse of Dimensionality. These dimensions will be obtained from the SK of the dimension table and aggregated with other dimensions and facts during query execution time, therefore increasing query latency. However, in what degree a \"Derived\" dimension could impacts query latency, depends of the cardinality relationship between SK and its \"Derived\" dimension column, e.g., the cardinality relationship between SUBJECT_ID (SK dimension) and SUBJECT_DES (subject description) is 1 to 1, thus in execution time only is required mapping between these values, due to data are aggregated at the same level in both dimensions. However, if we define SUBJECT_TYPE as \"Derived\", where the cardinality relationship with SUBJECT_ID is higher (1/16), more computation for data aggregation will be required during execution time. In some scenarios, we have a lot \"Normal\" dimensions, due to (i) the presence of many different dimensions tables, having each one at least one SK treated as a \"Normal\" dimension, or (ii) many dimension columns with a high cardinality relationship with SK columns, thus we prefer define it as \"Normal\" dimensions to avoid a high computation requirements during execution time. For these scenarios with many \"Normal\" dimensions, Kylin provides another optimization called **Aggregation Groups**. This optimization allows to group \"Normal\" dimensions that will be pre-aggregated together, enabling cube designers to apply one the following three powerful optimizations to reduce combinations between \"Normal\" dimensions included on this Aggregation Group:  
* **Mandatory Dimensions:** If a \"Normal\" dimension is defined as \"Mandatory\", all the combinations and pre-aggregations without this dimensions are pruned, e.g., if we define Student, Subject and Academic Year as \"Normal\" dimensions and also we mark Student as Mandatory, all the combinations between these three dimensions will include the Student dimension. * **Dimension Hierarchies:** We can define a hierarchical relationship for a group of \"Normal\" dimensions. Then only combinations and pre-aggregation for these dimensions in defined hierarchical order are computed during cube building, e.g., we can define the hierarchy SUBJECT_TYPE_ID\(\rightarrow\) SUBJECT_ID. This means that only the combinations (SUBJECT_TYPE_ID) and (SUBJECT_TYPE_ID, SUBJECT_ID) will be computed during cube building. * **Joint Dimensions:** Sometimes we find dimensions that are queried together very often. E.g., Id and description columns are usually queried together, such as SUBJECT_ID and SUBJECT_DES in our data source, however Kylin considers them as two separated dimensions. In this case, we can define a Joint group that includes these two dimensions.