Package: ldr
LDR stands forLatent Dimensionality Reduction. It is a generic method for interpreting models and making accurate predictions when features are missing. It is deployedhereas a python module.AboutThe purpose of LDR is to solve a common data science problem. Often models that have a higher predictive accuracy are more complex. These complex models are sensibly referred to asblack box models. This is frustrating for many data scientists, as they end up with a model that performs well, butcan't explain why. Inability to explain the model frequently causes the model tofail in critical situations, that are difficult to test for. Also, black box models typically require all features to be present to make a prediction. LDR provides a solution to both of these problems.Getting StartedPrerequesitesPython3.Note: Some examples contain additional that are not installed as dependencies by default, such asPyTorch.Installationpython3 -m pip install ldr --userExamplesAn example analysis of a classification problem can be foundhere.Imputation from densityAfter calculating the density estimate of the trained model usingdensity_estimate(), usingpredict()on new data will automatically impute missing features.Note: currently only missing columns will be imputed, not the odd value here or there.Note: This has not been fully implemented, so the optimal accuracy has not been achieved, and it's a bit slow.Contribution welcome:).VisualizationsThe examples above generate either 1D or 2D visualizations. These demonstrate the certainty of the model across the sample space it is trained on. Take the 1D visualizations from the classification example, which describe themodel's certainty of malignancy classification of mean area and area error of breast cancersamples in astudy.1 Dimensional VisualizationHere what can be seen is that if a cancer has amean area of less than 250then the model is most likely toclassify it as benign, whereas if it has amean area of more than 1200then it is almost certainly going to beclassified as malignant.With the area error, what can be seen is thatbetween 280 and 480, there isnot enough training datafor the model to reliably make a prediction in that area, as the total model certainty drops to 0 for both classifications.Between 80 and 210the model is very biased towards amalignant classification.2 Dimensional VisualizationInterpolating two classification certainties of individual features results in the heatmap in the bottom right. What can be seen is that the majority of samples have a low mean area and a low area error. When samplesdeviate away from the bottom left, with increasing mean area and area errors, theprobability of a malignancy classification dramatically increases.n Dimensional VisualizationsEven though only 1 and 2 dimensional visualizations are given, the entire classifier certainty across the sample space has been mapped. In this study there are actualy30 features, but by using LDR the individual components can be easily extracted. In order for a human to interpret an algorithmic model, they must be able to see it. AsDonald Knuthsays:"An algorithm must be seen to be believed, and the best way to learn what an algorithm is all about is to try it."Note: most successful black box models come from algorithmic techniques, such as neural networks; this is why I draw this similarity.Why LDR is Better Than Naive TechniquesNaive techniques for analysis black box models consist of analysing individual features (or composite functions of features) directly against their classifications resulting from each sample in the sample space. Because of this, the extrapolation that happens when applying the model to real world situations is never experimented against. Because LDR draws samples randomly from a KDE (kernel density estimate) of the sample space, by default there is extrapolation. Because of the KDE of the model, features can be imputed during prediction by taking the typical value of variables at the highest density, giving a more accurate representation of the feature space.The LDR RecipeAs machine learning algorithms go, LDR is pretty simple. It's effectively just amonte-carlo integrationacross the sample space, using theVEGAS algorithmas the sampling method.Min-max scale the data so that it falls into [0, 1] intervals.Train a predictive model on the scaled data.Create a kernel density estimate of the training samples.Sample n new points from the kernel density estimate, using the predictive model to make a prediction at each point.Bin the samples according to regular intervals. For each dimension, group points with resolution r, reducing the value of the bin to the mean prediction across it.See thepaperfor more detail.Additional NotesIf you find this package useful, please considercontributing!