# Adaptive Imputation of Irregular Truncated Signals with Machine Learning
## 3 Materials and Methods
### Introduction of Missing Data  
Once we had our synthetic dataset, the next phase of the methodology was to introduce missing data to the dataset. To do this, and to allow each of the three types of missing data to be introduced, we wrote a function in Python to take three parameters: the Pandas DataFrame containing our synthetic data, the type of missing data to add, and the percentage of values to make missing. The pseudocode for this function is shown in Algorithm 2.  
```
1 Requires()
2 The original DataFrame
3:type
4:The type of missing data (MCAR, MAR, or MNAR)
5:percentage
6:add
7:\(df\)Copy \(df.copy\)()
8:Create a copy of the DataFrame
9:featuresToConsider \(\leftarrow\) [Temperature', 'Pressure', 'Vibration']
10:iftype = 'MCAR'then
11:forfeature in featuresToConsiderdo
12:missingValues \(\leftarrow\) int(len(dfCopy) \(\times\) percentage/100)
13:missingIndices \(\leftarrow\) random choice from dfCopy.index of size missingValues
14:dfCopy.loc[missingIndices] \(\leftarrow\) NaN
15:endfor
16:elseiftype = 'MAR'then
17:\(\mathit{highTemperatureIndices}\)\(\leftarrow\) dfCopy[dfCopy[Temperature'] > dfCopy[Temperature'],mean().index
18:\(\mathit{MAR\_missing}\)VibrationIndices \(\leftarrow\) random choice from highTemperatureIndices of size int(len(highTemperatureIndices) \(\times\) percentage/100)
19:\(\mathit{dCovp.loc[MAR\_missing}\)VibrationIndices, 'Vibration'] \(\leftarrow\) NaN
20:\(\mathit{MAR\_missing}\)TemperatureIndices \(\leftarrow\) random choice from highTemperatureIndices of size int(len(highTemperatureIndices) \(\times\) percentage/100)
21:\(\mathit{dCovp.loc[MAR\_missing}\)TemperatureIndices, 'Temperature'] \(\leftarrow\) NaN
22:elseiftype = 'MNAR'then
23:\(\mathit{highPressureIndices}\)\(\leftarrow\) dfCopy[dfCopy['Pressure'] < dfCopy['Pressure'],quantitile(0.25)) or dfCopy['Pressure'],quantitile(0.75)).index
24:\(\mathit{highVibrationIndices}\)\(\leftarrow\) dfCopy[dfCopy['Vibration'] < dfCopy['Vibration'],quantitile(0.25)) or dfCopy['Vibration'],quantitile(0.75)).index
25:\(\mathit{MNAR\_missing}\)Indices \(\leftarrow\) highPressureIndices or highVibrationIndices
26:\(\mathit{MNAR\_selectedIndices}\)\(\leftarrow\) random choice from dfCopy[MNAR\_missing}\(\mathit{missingIndices}\)].index of size int(len(dfCopy/MNAR\_missing}\times\) percentage/100)
27:\(\mathit{dCovp.loc[MNAR\_missing}\)Vibration \(\leftarrow\) NaN
28:\(\mathit{dCovp.loc[MNAR\_missing}\)Vibration \(\leftarrow\) NaN
29:endif
30:return dfCopy
```  
**Algorithm 2** Missing Data Introduction  
The logic of this function can be described as follows:  
* Since missing data is classified as MCAR when the missingness has no relationship with any data, observed or missing, if 'MCAR' is passed into the 'type' parameter, then completely random numerical values are made missing by replacing the existing values with Not a Number (NaN) values.
* Missing data is classified as MAR when the missingness has a relationship with observed data but not with the missing data itself. There were a variety of ways that we could have simulated MAR data for this research, but the method we chose was introduce missing \"Vibration\" values more frequently when the value in \"Temperature\" is high, illustrating a systematic relationship between the two variables. In addition to introducing missing \"Vibration\" values when the temperature is high, we also definedlogic to increase the likelihood that there will be missing values in \"Temperature\", albeit at a reduced frequency compared to the first case.
* As with MAR data, there were a variety of ways that MNAR data, where missingness is related to the data that's missing itself, could have been simulated. The method we chose is to introduce missing \"Pressure\" values more frequently when the values in \"Pressure\" are considered too low or too high, showing a relationship between \"Pressure\" with its own values. We also apply this same logic to the values in \"Vibration\".  
A visualization of the dataset with each of the three types of missing data added is shown in Figure 2.