# Adaptive Imputation of Irregular Truncated Signals with Machine Learning
## 3 Materials and Methods
### Adaptive Imputation  
Once we were able to automatically assess the missing data percentage, we could begin to implement our adaptive imputation system. Our system is based on the logic that the most effective imputation method for a particular problem can vary based on what type of missing data scheme is present as well as what percentage of the data is missing. Table 4 shows each imputation method that was selected for the types and percentages of missing data. The logic behind these selections will be explained in this subsection. For MCAR, where the missing data is independent of both observed and unobserved data, the chosen imputation method varies with the percentage of missing data. For small amounts (1-10%), mean imputation is efficient and maintains dataset integrity by replacing missing values with the mean of observed data, preserving the overall dataset's statistical properties. This method is simple and computationally light, making it ideal for quick fixes in datasets with minimal missing entries. When the missingness is moderate (11-30%), KNN imputation leverages the proximity to the nearest neighbors in the dataset to estimate missing values. This method utilizes the inherent patterns and similarities found in nearby data points to provide a more contextually accurate imputation. As the level of missing data increases (31-50%), MICE becomes particularly advantageous. It iteratively performs multiple regressions, filling in missing values several times to better approximate the inherent uncertainty in the data. For substantial missing data percentages (51-90%), autoencoders, which are neural networks that learn the underlying patterns and structures within data by compressing and decompressing the dataset, can unearth deep correlations, making them particularly effective for reconstructing significantly incomplete datasets. In MAR scenarios, where the missing data is related to the observed data, the imputation techniques adapt to reflect these dependencies. Even with minimal missing data, MICE proves effective because it can account for the relationships between observed variables and missing data. This robust method enhances the imputation strategy for datasets where missingness depends on observed variables. KNN continues to be a reliable choice for moderate levels of missing data, preserving data integrity by imputing values based on the characteristics of similar observed data. As the percentage of missing data increases, MICE provides a more extensive imputation by analyzing interdependencies within the data, crucial for maintaining the fidelity of more complex datasets. For extensive missing data, autoencoders' capability to model complex data relationships efficiently makes them invaluable. They reconstruct data based on learned representations, providing a potent tool for detailed and accurate data restoration. MNAR presents the most challenging scenario, where the missingness depends on unobserved data. Selection models and MIMM are designed to address the specific mechanisms behind MNAR data for small to moderate missing data levels. These methods provide a tailored approach that incorporates assumptions about the reasons behind the missingness into the imputation process, enhancing the accuracy in the presence of biased missing data. For significant levels of missing data, joint modeling utilizes all available data in the estimation process, effectively leveraging both the observed and missing data patterns to make more accurate inferences. For severe missingness, autoencoders' ability to model complex, non-linear relationships allows them to reconstruct datasets effectively, where traditional methods might not adequately capture the underlying data structure.