# Missing Data Imputation in the Internet of Things Sensor Networks
## 4 Experiments
### Our Approach
#### 4.3.2 Stage 2: Defining the Imputation Strategy  
The second stage is initiated after the dataset has been group into clusters with similar records. We trained a random forest model on each group before aggregating the data. This ensured that strong predictors \(X_{s}=1,...,p\) were used in the training process.  
In our approach, we assume an \(n\times p\)-dimensional matrix where X = \((X_{1},...,X_{n})\). We first of all use the RF algorithm to fill in missing observations in each partition created by the \(k\)-Means algorithm. A built-in routine is added to the RF algorithm for handling missing values by considering the frequency of values in the recorded variables with their RF proximities after initially training the model on the dataset pre-imputed with mean values [47]. This approach mostly requires a fully observed response variable before the RF model can be trained. However, we directly estimated the missing values by using the RF model on a training set containing only the observed data, with X, being the matrix with complete data and \(X_{s}\) representing the sample with missing vales \(i^{(s)}_{miss}\subseteq\{1,...,n\}\). To better understand the training process, we separate the data into four parts as described below:  
1. \(y^{(s)}_{obs}\rightarrow\) representing the values that are present in the variable \(X_{s}\)
2. \(y^{(s  
It is important to note that \(t_{obs}^{(s)}\) points only to the observed values in \(X_{s}\). Therefore, \(x_{obs}^{(s)}\) is not completely observed and likewise, \(x_{miss}^{(s)}\) is not completely missing.  
Similar to the work in [17], the process is initiated by pre-imputing the missing values in \(X\) with the mean of the distribution or any imputation method, after which the predictors \(X_{s}=1,...,p\) are stacked in ascending order considering the amount of values that are missing. Each missing value in \(X_{s}\) is then imputed by first of all fitting the RF on the response \(y_{obs}^{(s)}\) and predictor variable \(x_{obs}^{(s)}\). Next, the trained RF model is applied to \(x_{miss}^{(s)}\) to predict the missing values of \(y_{miss}^{(s)}\). The imputation process is repeated until the set stopping criterion (\(\gamma\)) has been met. This is achieved when the difference between the most recent imputed data matrix and the old matrix has an increase for the first time, considering the variable types present. In our approach, we take the \(n\times p\) matrix to be a set of set of continuous variables. Therefore, the difference in the new and previous imputed matrix \(\mathbf{N}\) is defined as:  
\[\Delta_{N}=\frac{\sum_{j\in N}(X_{new}^{imp}-X_{old}^{imp})^{2}}{\sum_{j\in N }(X_{new}^{imp})^{2}} \tag{3}\]  
This step is followed by two additional imputation phases where missing values in each cluster is also estimated using a \(k\)NN and linear regression model.  
Still considering an \(n\times p\) matrix \(X_{s}\), the procedure for next imputation phase is described as follows;  
1. The missing values in each cluster matrix \(C_{i}\) are located.
2. The \(k\)NN vectors are defined by; \(x_{(1)}^{D},...,x_{n}^{D}\) with \(d(x_{i}x_{(1)})\leq,...,\leq d(x_{i}x_{(k)})\), where \(x_{(1)}^{D}\) represents the rows of the matrix \(X^{D}\), and \(d(x_{i}x_{(k)})\) is the distance given by Equation (1).
3. For each point \((y)\) in \(C_{i}\), the distance \((x,y)\) between the missing point and nearest imputed value is stored in a similarity array \((S)\).
4. The array \((S)\) is sorted in descending order and the top \(K\) data for \((y)\) in \(C_{i}\) is selected for imputation.  
The linear regression imputation process follows, as described below;  
1. For each matrix \(C_{i}\) the data was split into four parts similar to the RF method where a regression model was trained on the response \(y_{obs}^{(s)}\) and predictor variable \(x_{obs}^{(s)}\).
2. The trained regression model is then used to predict the missing values in \(X_{s}\).