Shining a light on the “Black Box” of machine learning
By: Matt Scott Frank
Tags: Data Science, Data Visualization, Statistics, Machine Learning

Variable Relationships

Variable importance plots like the one above can be a good initial peek under the hood into how our model is arriving at its conclusions. However, it is also limited in a number of ways.

For instance, though we know history of election violence is our most important variable, we do not know if high or low values yield higher probabilities of election violence. In addition, even if we knew this, we still do not know what sort of interaction effects there might be between multiple predictors and our target variable.

Given these limitations, variable importance plots are best thought of as an initial clue into what further, more nuanced avenues of analysis will be fruitful.

For instance, given what we have learned from the above plot, we know that gaining a more nuanced understanding of the relationship between election violence and historic levels of election violence will be important, and we may additionally be curious how precipitation levels interact with this, for example.

This is a stage where subject-matter expertise can become very important. Such knowledge can point the researcher to especially interesting relationships that may or may not conform with existing research.

In order to explore these relationships, there are four additional visualizations we will explore: partial dependence plots, individual conditional expectation plots, accumulated local effects plots and Shapley values.

Partial Dependence Plots

We will begin here with partial dependence plots (PDP). At their most basic level, PDP’s display the average effects of a given predictor on a target variable for every value of the predictor variable. Essentially, for a given predictor value, the effects of that value on the target variable are aggregated over a marginal distribution while also accounting for the average effects of all other predictors. What this yields is a plot that can intuitively visualize a potentially very complicated variable relationship.

In addition, they are extremely straightforward to make. The only drawback is that the pdp package used to make them does not work with caretList objects (yet), meaning that we cannot use the full ensemble model in our PDP. For now, we will just be using a random forest object.

Below we simply put our model object into the partial() function, note which variable we want to display and what our positive class is. Then we display the plot by putting that object into the plotPartial() function.

pd1 <- partial(rf_ev, pred.var = "process1",

which.class = "vio") plotPartial(pd1, xlab = "History of Election Violence",

ylab = "Violence")

What we get is a very basic chart that shows that the proportion of elections classified as “violent” increases as the history of election violence increases.

The y-axis values on this PDP are all negative because our data is imbalanced. Recall that 70% of our observations have negative target values (no election violence).

If we consider that the partial dependence function is calculating the cumulative effect of each predictor’s individual values on the classification of the target, then these cumulative effects are unsurprisingly skewed towards the negative predictor value.

Because of this, it is better to interpret the relationship in more simple directional terms. The more positive the line, the more it is associated with the positive target value.

Beyond interpretation, we may also note that the above plot is pretty boring. It is just a plain, choppy line plotted against a white background.

Thankfully, the pdp package makes it easy to branch out from the plotPartial() function if we wish because the partial() function returns a dataframe with predictor values and their associated ŷ estimates. This allows us to use ggplot to make a nicer visualization.

ggplot(pd1, aes(x = process1, y = yhat)) +

geom_smooth() +

theme_bw() +

labs(x = "History of Election Violence", y = "Violence")

Now we have a much nicer, smoothed line that forms a classic sigmoidal curve and even has fancy-looking, shaded confidence intervals.

While this is a better plot to use for external-facing materials, we can also plot multiple predictors at the same time in order to see interactions between these predictors. This is where we get the really whiz-bang plots.

Below we are essentially doing the same thing as we did with one predictor, we just list the two variables we want in a vector for our partial() function. That being said, there is a bit more going on in our plotPartial() function that is worth going through.

First, we need to set the levelplot argument to false in order to have a three-dimensional plot.

Second, setting the drape argument to true displays the colors we see on the actual three-dimensional plane in our plot below.

Third, the colorkey argument is a logical of whether or not the plot should include the key on the right side of our plot.

Finally, the screen argument indicates the angles at which the plot should be displayed. What values you want here will depend on what the actual plot looks like and will necessarily entail some trial and error.

pd2 <- partial(rf_ev, pred.var = c("logIMR", "SPI")) pd2 <- plotPartial(pd2, levelplot = F, zlab = "Violence",

xlab = "Infant Mortality", ylab = "Precipitation",

drape = T,

colorkey = T,

screen = list(z = -300, x = -50))

The three variable PDP above displays a very interesting interaction between the overall positive relationship between infant mortality rate and election violence and a parabolic relationship between precipitation and and election violence.

Even though lower infant mortality rates in general have a negative effect on the probability of election violence, very high and very low levels of precipitation offer an exception to this and can increase the risk of election violence even in cases of very low infant mortality rates. As infant mortality rises, the association with violent elections surges in both times of drought and abundant rainfall.

Individual Conditional Expectation Plots

Individual Conditional Expectation (ICE) plots show a disaggregated version of PDP’s, plotting individual expected effects of each observation rather that the overall average effect of the variable.

In essence, for each observation in our dataset we see how its predicted value is expected to vary across all predictor values.

Why might we want to do this?

Because PDP’s display an averaged effect, we may want to see where greater heterogeneity exists within this average and at which points this heterogeneity is introduced. ICE plots can help us visualize this greater level of nuance.

Here we will be using the ICEbox package. However, it is worth noting that there are a couple of other ways you can create ICE plots. For instance, the pdp package has an option to make ICE plots, while the iml package can also be used to create ICE plots.

Below we are using the ice() function which, for classification problems, has four key arguments.

The first three are fairly straightforward. The object argument asks for the model object we wish to use, while we list the training data used for the X argument and which predictor we wish to visualize for the predictor argument.

The predictfcn argument appears slightly more complicated, however, all we are doing is writing a function which will return a vector of probabilities made using the predict function.

The ice function will automatically pass what we specified as our object and X into this function as the object and newdata , respectively.

After that we simply use the brackets to specify which column of predictions we want. In this case, the first column is the probabilities of election violence occurring.

From here, we put our resulting ICE object into the plot() function.

However, there are a number of ways to customize this plot. Below we use the frac_to_plot argument to tell R we want it to randomly plot 60% of the given observations. We do this to minimize how crowded the resulting plot is.

ice_ob <- ice(object = rf_ev, X = trane, predictor = "process1",

predictfcn = function(object, newdata){

predict(object, newdata, type = "prob")[,1]

}

) plot(ice_ob, frac_to_plot = .6, xlab = "History of Election Violence", ylab = "Violence")

Above, each individual gray line represents a single observation, while each gray dot represents that observation’s observed value.

The thick black line with the yellow outline is the PDP. In general, we see that there is a rather large amount of heterogeneity in the effect that the history of election violence has on the probability of classifying election violence.

Of particular interest is the cluster of observed values in the upper-right corner which appear to drag the PDP line up. One note is that our plotted PDP line here looks different than the PDP line we got when using the pdp package.

This may be a result of slightly different ways the packages calculate the lines, as well as a scale issue given the much larger range of values plotted above and below the PDP line.

There are two other ways we can use ICE plots to visualize variable relationships.

The first is by centering all of our lines around a particular predictor value and then displaying the effects of our selected predictor on our response variable relative to that centered value.

This is called a centered ICE plot or c-ICE plot. The default for the ICEbox package is to center the predictor values around first percentile.

In other words, it centers the plot around the lowest value of our chosen predictor, although this can be changed by using the centered_percentile argument in the plot() function.

Other than that, the only different thing we have to do below is set the centered argument to true.

plot(ice_ob, centered = T, frac_to_plot = .6, xlab = "History of Election Violence", ylab = "Violence")

In the c-ICE plot above, we can see that when compared to cases with the lowest levels of historical election violence, the probability of future election violence mostly remains unchanged until our history of election violence value is greater than zero, while there is also quite a spike at right around one.

Meanwhile, the right-hand vertical axis offers a measure of the magnitude of the predictor effect by displaying the change in our target variable’s value as a fraction of its range (which in this case is between zero and one because it is a probability).

Another option for working with ICE plots is to use a derivative ICE plot, or d-ICE.

While this is not as useful in a general sense as the previous two ICE plots, it can be useful for identifying the places in which greater heterogeneity is introduced in the data.

In order to create d-ICE plots, we simply place our ICE object into the dice() function, and then place the resulting d-ICE object into the plot() function as we would with our regular ICE objects.

dice_ob <- dice(ice_obj = ice_ob) plot(dice_ob, frac_to_plot = .6, xlab = "History of Election Violence", ylab = "Derivative of ICE Curve")

In the above d-ICE plot, we can see that there is a slight increase in the derivatives at around zero on the x-axis, with the larger spike being around one.

This largely reflects what we already saw in the c-ICE plot above, however, the d-ICE plot can be useful when trying to emphasize these spikes. In addition, the sub-plot below the d-ICE curves is a chart of the standard deviations of the derivatives at each point on the line and is a good visualization of where there is greater volatility along the curve.

One final note on customizing your ICE charts is that if you do not wish to display the PDP line you can set the plot_pdp argument to false in the plot() function.

Likewise, you can choose not to display the individual dots for each observed value by setting plot_orig_pts_preds to false.

Accumulated Local Effects Plots

Accumulated Local Effects (ALE) plots attempt to answer essentially the same question as PDP’s: how does the effect of a predictor on a target variable vary with the predictor’s value? However, they differ from PDP’s in the way in which they are calculated, and can offer some benefits depending on the data.

When calculating PDP’s, the cumulative effects of a given predictor value are calculated over a marginal distribution, while ALE plots do the same thing over a conditional distribution. The important difference here is that when predictors are highly correlated, the marginal distribution used by PDP’s can be much wider than the conditional distribution and include predictor areas where no data actually exists.

This can be problematic because these regions can contain unrealistic predicted values because models are not penalized for wild predictions where no training data exist. The narrower conditional distribution used by ALE plots helps to mitigate this issue, which can make ALE plots preferable in cases where predictors are highly correlated.

Here we will be creating ALE plots with the iml package. The syntax for the iml package may be unfamiliar to a lot of users, however, it offers a flexible method for creating ALE plots (among a lot of other things).

Below we use the method Predictor$new() in order to create an object that contains the random forest model, the training data and a specification for what our target variable is.

We then plug that into the similar FeatureEffect$new() and specify which predictor we want. Note that the default method used by FeatureEffect$new() is to make ALE plots, however, it can also be used to make PDP and ICE plots.

From here we can extract the results from the resulting object and then use the resulting dataframe with ggplot .

predictor <- Predictor$new(rf_ev,

data = trane,

y = train_df2$elecViolence1,

class = "vio") ale_iml <- FeatureEffect$new(predictor, feature = "process1") p1_df <- ale_iml$results ggplot(p1_df, aes(x = process1, y = .ale)) +

geom_line(size = 1) + theme_bw()

Above, we get an ALE plot which is quite similar to the PDP we made for history of election violence. This is good news because it means the results we got for our PDP were not biased by correlation amongst features. Indeed, looking at the smoothed line below, we also get a similar sigmoidal-ish curve.

ggplot(p1_df, aes(x = process1, y = .ale)) +

geom_smooth(method = "loess") +

theme_bw() +

labs(x = "History of Election Violence", y = "Violence (ALE)")

We can also use the ALE plots to explore interactions between different variables, much as we did with PDP’s. Below we follow the same steps as we did for a single predictor, but just list a vector of two predictors for the feature argument. Below we display the interaction between precipitation and time since the last election.

ale_iml.2 <- FeatureEffect$new(predictor,

feature = c("SPI", "lastelection"))

ale_iml.2$plot()

Here our ALE plot displays two areas with larger positive effects on the classification of election violence: in the area with very low precipitation and higher time since last election and at very high precipitation with a lower time since last election.

The area of very low association with election violence in the upper right corner of the graph (high precipitation and high time since last election) is somewhat surprising, however, looking at the distributions of our data displayed along the axes, we can see that very few observations fall into this category. As such, we should take those results with a grain of salt.

Shapley Values

While the previous methods have looked at general relationships between predictors and the target variable, Shapley values can offer a more specific look at how each predictor value influenced each observation’s classification.

In other words, here we can look at how much each predictor contributed to a given prediction.

In brief, the calculation of the Shapley value attempts to explain the difference between an observed predicted value and the average prediction.

For our purposes, the average predicted probability for election violence is 30%, meaning that we will be attempting to explain a given observation’s deviation from this average.

In order to achieve this, Shapley values are calculated by taking a single observation (let’s say Angola’s 2017 election) and its predicted probability (87%), and then replacing individual predictor values with those drawn from other randomly selected observations.

For instance, Angola’s quality of democracy score was -2 for its 2017 election. As part of this process, its original value of -2 might be replaced with a randomly selected observation’s quality of democracy score, and its predicted probability would be recalculated.

Let’s say its recalculated predicted probability was 73% after its quality of democracy score was replaced with a value of 5. We would then say that, relative to a value of 5, Angola’s quality of democracy of -2 contributed 14% to its predicted probability.

This process is repeated over all possible coalitions and the Shapley value itself is then calculated as the average marginal contribution to all possible coalitions.

Essentially, this value indicates how much each predictor value contributes to the observed difference between an observation’s actual predicted probability and the average predicted probability.

In order to calculate and plot these values, we will again be using the iml package.

First, we will add a vector of predicted probabilities to the dataframe we will put into our Predictor object. Once we have done this, we specify type = "probs" within our Predictor$new() , keeping everything else the same more or less.

After that we calculate the Shapley values using Shapley$new() and list our Predictor object as well as the row within our data that we are interested in with the x.interest argument. After that we can extract the results and use the resulting dataframe with ggplot .

rf_preds <- predict(rf_ev,

newdata = train_df2,

type = "prob") trane.2 <- trane



trane.2$probs <- rf_preds$vio shap_pred <- Predictor$new(rf_ev,

data = trane.2,

y = "probs",

type = "prob",

class = "vio") angola <- Shapley$new(shap_pred,

x.interest = trane.2[2188,]) ang_data <- angola$results ang_data <- ang_data[order(ang_data$phi,

decreasing = T),] var_names <- c("History of Election Violence", "Regime Tenure", "Infant Mortality Rate", "Quality of Democracy", "Executive Constraint", "Population", "GDP per capita", "GDP growth", "Year", "Precipitation", "Level of Political Competition", "Election Anticipation", "Time since Last Election", "Coup Risk") ang_data$var_names <- var_names ang_data$var_names <- factor(ang_data$var_names,

levels = ang_data$var_names) ggplot(ang_data, aes(x = var_names, y = phi)) +

geom_col(fill = "cornflowerblue",

alpha = .7,

col = "blue") +

theme_classic() +

coord_flip() +

labs(x = "Variable",

y = "Shapley Value",

title = "Angola Election 2017",

subtitle = "Actual prediction = 0.87; Average Prediction = 0.3")

Above we can see a stacked ranking of the estimates of how much each predictor contributed to the difference between the average and actual predicted probability.

While the history of election violence made by far the greatest contribution, we can also see that regime tenure made a significant contribution. This is interesting because at the time of this election Angola had a regime tenure of 456 months, roughly 373 months above the average of 83!

In terms of interpretation, it is important to note that despite a similar graphical representation, Shapley values are not similar to variable importance.

Variable importance measures how adversely a model’s accuracy would be affected if a given predictor were removed, while Shapley values measure how much a given predictor value contributes to the difference between an observation’s predicted value and the average predicted value.

In addition, it should be emphasized that despite my choice to simply display the variable names on the x-axis, it is the effect of the actual predictor values that is being measured, not the predictor itself per se.

In the default iml output, each predictor value is listed along the x-axis. This is preferable if you need a plot that is more technically specific, however, for the purposes of creating a less busy plot they are excluded here.

Obviously this is a trade-off and the appropriate choice will vary based on your audience/needs.