# SMARTEN--A Sample-Based Approach towards Privacy-Friendly Data Refinement
## 2 Data Refinement Process
\begin{table}
\begin{tabular}{l l l} \hline \hline
**CRISP-DM** & **KDD** & **SEMMA** \\ \hline \hline Business Understanding & _Preceding the KDD Process_ & - \\ \hline \multirow{2}{*}{Data Understanding} & Selection & Sample \\ \cline{2-3}  & Pre-Processing & Explore \\ \hline Data Preparation & Transformation & Modify \\ \hline Modeling & Data Mining & Model \\ \hline Evaluation & Interpretation/Evaluation & Assess \\ \hline Deployment & _Succeeding the KDD Process_ & - \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Contrasting Juxtaposition of the Individual Process Steps of CRISP-DM, KDD, and SEMMA. Figure 2: A data refinement process model in accordance with [PERSON] et al. [26]. get them refined. The IoT facilitates the capturing of different types of raw data. These data constitute the foundation for any smart service. Data producers therefore have an incentive to let their data be prepared and processed in order to feed these services. The processing itself is handled by the _data processor_. The processor first transforms the heterogeneous raw data into a processable form and then applies various analysis techniques to them in order to gain knowledge. This knowledge is then offered to a _data consumer_, who utilizes it, e.g., to operate a smart service [29]. Sensitive data, e.g., confidential or private data, may be involved in this process. A data producer therefore not only has privacy requirements with regard to the data consumer, who utilizes the knowledge contained in the data, but also with regard to the data processor, who gains deep insights into the raw data during data refinement. In our work, the stages of the refinement process in which the data processor is involved are particularly relevant. Here, not only the transformation of the data into information takes place, but also the analysis of the information that reveals the knowledge. This part of the process can be divided into two phases, the _data preparation phase_ and the _data processing phase_. Data Preparation Phase.This part of the data refinement process deals with data wrangling. That is, a _data steward_ selects a set of base data from the available raw data that is worth further investigation. These target data must then be cleansed, since raw data always have quality issues. For instance, missing values have to be filled in, wrong values have to be corrected, redundant data have to be removed and imbalances in the datasets have to be corrected. The cleansed data must then be converted into a uniform structure that can be processed. A data steward needs deep insight into individual datasets in order to identify the quality issues within the base data [30]. Data Processing Phase.Based on the compiled information, a _data analyst_ can recognize contained information patterns. Applied to the right situation, these patterns represent knowledge. Such patterns are supposed to answer questions such as \"What happened?\" (descriptive), \"Why did it happen?\" (diagnostic), \"What will happen?\" (predictive), and \"What should I do?\" (prescriptive). Thus, the base data are reflected in this knowledge only in a highly aggregated and condensed form. Therefore, a data analyst does not need an insight into individual datasets to accomplish his or her task. However, the derived knowledge reveals much more insight than a single set of raw data [31]. Although it might seem that the second phase is more relevant, as it reveals the valuable knowledge, the focus of data refinement is actually on the first phase. In fact, the data preparation phase accounts for up to 80% of the data refinement, and, without thorough data wrangling, the subsequent data analysis cannot be conducted successfully [32]. However, since such data refinement can reveal a lot of confidential knowledge, data subjects have to be protected. Legislators have therefore enacted data protection laws that are designed for the age of the IoT and big data analytics, e.g., the _General Data Protection Regulation_ (_GDPR_) [33] which is in effect in the European Union.