  Despite their popularity, machine learning predictions are sensitive to
potential unobserved predictors. This paper proposes a general algorithm that
assesses how the omission of an unobserved variable with high explanatory power
could affect the predictions of the model. Moreover, the algorithm extends the
usage of machine learning from pointwise predictions to inference and
sensitivity analysis. In the application, we show how the framework can be
applied to data with inherent uncertainty, such as students' scores in a
standardized assessment on financial literacy. First, using Bayesian Additive
Regression Trees (BART), we predict students' financial literacy scores (FLS)
for a subgroup of students with missing FLS. Then, we assess the sensitivity of
predictions by comparing the predictions and performance of models with and
without a highly explanatory synthetic predictor. We find no significant
difference in the predictions and performances of the augmented (i.e., the
model with the synthetic predictor) and original model. This evidence sheds a
light on the stability of the predictive model used in the application. The
proposed methodology can be used, above and beyond our motivating empirical
example, in a wide range of machine learning applications in social and health
sciences.
