Package: model-inspector
Model Inspectormodel_inspectoraims to help you train betterscikit-learn-compatible models by providing insights into their
behavior.UseTo usemodel_inspector, you create anInspectorobject from ascikit-learnmodel, a feature DataFrameX, and a target Seriesy.
Typically you will want to create it on held-out data, as shown below.importsklearn.datasetsfromsklearn.ensembleimportRandomForestRegressorfromsklearn.model_selectionimporttrain_test_splitfrommodel_inspectorimportget_inspectorX,y=sklearn.datasets.load_diabetes(return_X_y=True,as_frame=True)X<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>agesexbmibps1s2s3s4s5s600.0380760.0506800.0616960.021872-0.044223-0.034821-0.043401-0.0025920.019907-0.0176461-0.001882-0.044642-0.051474-0.026328-0.008449-0.0191630.074412-0.039493-0.068332-0.09220420.0852990.0506800.044451-0.005670-0.045599-0.034194-0.032356-0.0025920.002861-0.0259303-0.089063-0.044642-0.011595-0.0366560.0121910.024991-0.0360380.0343090.022688-0.00936240.005383-0.044642-0.0363850.0218720.0039350.0155960.008142-0.002592-0.031988-0.046641.................................4370.0417080.0506800.0196620.059744-0.005697-0.002566-0.028674-0.0025920.0311930.007207438-0.0055150.050680-0.015906-0.0676420.0493410.079165-0.0286740.034309-0.0181140.0444854390.0417080.050680-0.0159060.017293-0.037344-0.013840-0.024993-0.011080-0.0468830.015491440-0.045472-0.0446420.0390620.0012150.0163180.015283-0.0286740.0265600.044529-0.025930441-0.045472-0.044642-0.073030-0.0814130.0837400.0278090.173816-0.039493-0.0042220.003064442 rows × 10 columnsy0      151.0
1       75.0
2      141.0
3      206.0
4      135.0
       ...  
437    178.0
438    104.0
439    132.0
440    220.0
441     57.0
Name: target, Length: 442, dtype: float64X_train,X_test,y_train,y_test=train_test_split(X,y)rfr=RandomForestRegressor().fit(X_train,y_train)rfr.score(X_test,y_test)0.4145806969881506inspector=get_inspector(rfr,X_test,y_test)You can then use various methods ofinspectorto learn about how your
model behaves on that data.The methods that are available for a given inspector depends on the
types of its estimator and its targety. An attribute calledmethodstells you what they are:inspector.methods['plot_feature_clusters',
 'plot_partial_dependence',
 'permutation_importance',
 'plot_permutation_importance',
 'plot_pred_vs_act',
 'plot_residuals',
 'show_correlation']ax=inspector.plot_feature_clusters()most_important_features=inspector.permutation_importance().index[:2]axes=inspector.plot_partial_dependence(features=[*most_important_features,most_important_features])axes[0,0].get_figure().set_size_inches(12,3)inspector.permutation_importance()bmi    0.241886
s5     0.153085
sex    0.003250
s3     0.000734
bp     0.000461
s4    -0.002687
s2    -0.004366
s1    -0.008953
s6    -0.018925
age   -0.022768
dtype: float64ax=inspector.plot_permutation_importance()ax=inspector.plot_pred_vs_act()axes=inspector.plot_residuals()inspector.show_correlation()<style type="text/css">
#T_c8180_row0_col0, #T_c8180_row1_col1, #T_c8180_row2_col2, #T_c8180_row3_col3, #T_c8180_row4_col4, #T_c8180_row5_col5, #T_c8180_row6_col6, #T_c8180_row7_col7, #T_c8180_row8_col8, #T_c8180_row9_col9, #T_c8180_row10_col10 {
  background-color: #ff0000;
  color: #f1f1f1;
}
#T_c8180_row0_col1, #T_c8180_row1_col0 {
  background-color: #ffc6c6;
  color: #000000;
}
#T_c8180_row0_col2, #T_c8180_row0_col5, #T_c8180_row2_col0, #T_c8180_row5_col0, #T_c8180_row5_col9, #T_c8180_row9_col5 {
  background-color: #ffd2d2;
  color: #000000;
}
#T_c8180_row0_col3, #T_c8180_row2_col5, #T_c8180_row3_col0, #T_c8180_row5_col2 {
  background-color: #ffd0d0;
  color: #000000;
}
#T_c8180_row0_col4, #T_c8180_row4_col0 {
  background-color: #ffc4c4;
  color: #000000;
}
#T_c8180_row0_col6, #T_c8180_row6_col0 {
  background-color: #f4f4ff;
  color: #000000;
}
#T_c8180_row0_col7, #T_c8180_row3_col7, #T_c8180_row7_col0, #T_c8180_row7_col3 {
  background-color: #ffcece;
  color: #000000;
}
#T_c8180_row0_col8, #T_c8180_row8_col0 {
  background-color: #ffb6b6;
  color: #000000;
}
#T_c8180_row0_col9, #T_c8180_row9_col0 {
  background-color: #ffaeae;
  color: #000000;
}
#T_c8180_row0_col10, #T_c8180_row1_col8, #T_c8180_row8_col1, #T_c8180_row10_col0 {
  background-color: #ffdede;
  color: #000000;
}
#T_c8180_row1_col2, #T_c8180_row1_col7, #T_c8180_row2_col1, #T_c8180_row7_col1 {
  background-color: #ffb4b4;
  color: #000000;
}
#T_c8180_row1_col3, #T_c8180_row3_col1 {
  background-color: #ffb0b0;
  color: #000000;
}
#T_c8180_row1_col4, #T_c8180_row4_col1 {
  background-color: #f2f2ff;
  color: #000000;
}
#T_c8180_row1_col5, #T_c8180_row5_col1 {
  background-color: #ffeaea;
  color: #000000;
}
#T_c8180_row1_col6, #T_c8180_row6_col1 {
  background-color: #9898ff;
  color: #f1f1f1;
}
#T_c8180_row1_col9, #T_c8180_row9_col1 {
  background-color: #ffb8b8;
  color: #000000;
}
#T_c8180_row1_col10, #T_c8180_row10_col1 {
  background-color: #ffbaba;
  color: #000000;
}
#T_c8180_row2_col3, #T_c8180_row3_col2 {
  background-color: #ff7272;
  color: #f1f1f1;
}
#T_c8180_row2_col4, #T_c8180_row4_col2 {
  background-color: #ffd8d8;
  color: #000000;
}
#T_c8180_row2_col6, #T_c8180_row6_col2 {
  background-color: #9292ff;
  color: #f1f1f1;
}
#T_c8180_row2_col7, #T_c8180_row7_col2 {
  background-color: #ff8c8c;
  color: #000000;
}
#T_c8180_row2_col8, #T_c8180_row8_col2 {
  background-color: #ff9292;
  color: #000000;
}
#T_c8180_row2_col9, #T_c8180_row9_col2 {
  background-color: #ff8282;
  color: #f1f1f1;
}
#T_c8180_row2_col10, #T_c8180_row5_col7, #T_c8180_row7_col5, #T_c8180_row10_col2 {
  background-color: #ff5656;
  color: #f1f1f1;
}
#T_c8180_row3_col4, #T_c8180_row4_col3, #T_c8180_row4_col10, #T_c8180_row5_col10, #T_c8180_row10_col4, #T_c8180_row10_col5 {
  background-color: #ffe8e8;
  color: #000000;
}
#T_c8180_row3_col5, #T_c8180_row5_col3 {
  background-color: #fff6f6;
  color: #000000;
}
#T_c8180_row3_col6, #T_c8180_row6_col3 {
  background-color: #ccccff;
  color: #000000;
}
#T_c8180_row3_col8, #T_c8180_row8_col3 {
  background-color: #ffa2a2;
  color: #000000;
}
#T_c8180_row3_col9, #T_c8180_row9_col3 {
  background-color: #ff8e8e;
  color: #000000;
}
#T_c8180_row3_col10, #T_c8180_row10_col3 {
  background-color: #ff7c7c;
  color: #f1f1f1;
}
#T_c8180_row4_col5, #T_c8180_row5_col4 {
  background-color: #ff1e1e;
  color: #f1f1f1;
}
#T_c8180_row4_col6, #T_c8180_row6_col4 {
  background-color: #ffeeee;
  color: #000000;
}
#T_c8180_row4_col7, #T_c8180_row7_col4 {
  background-color: #ff6c6c;
  color: #f1f1f1;
}
#T_c8180_row4_col8, #T_c8180_row8_col4 {
  background-color: #ff7e7e;
  color: #f1f1f1;
}
#T_c8180_row4_col9, #T_c8180_row9_col4 {
  background-color: #ffbebe;
  color: #000000;
}
#T_c8180_row5_col6, #T_c8180_row6_col5 {
  background-color: #d6d6ff;
  color: #000000;
}
#T_c8180_row5_col8, #T_c8180_row8_col5 {
  background-color: #ffc2c2;
  color: #000000;
}
#T_c8180_row6_col7, #T_c8180_row7_col6 {
  background-color: #4646ff;
  color: #f1f1f1;
}
#T_c8180_row6_col8, #T_c8180_row8_col6 {
  background-color: #a0a0ff;
  color: #f1f1f1;
}
#T_c8180_row6_col9, #T_c8180_row9_col6 {
  background-color: #b4b4ff;
  color: #000000;
}
#T_c8180_row6_col10, #T_c8180_row10_col6 {
  background-color: #8a8aff;
  color: #f1f1f1;
}
#T_c8180_row7_col8, #T_c8180_row8_col7 {
  background-color: #ff6464;
  color: #f1f1f1;
}
#T_c8180_row7_col9, #T_c8180_row7_col10, #T_c8180_row9_col7, #T_c8180_row10_col7 {
  background-color: #ff9696;
  color: #000000;
}
#T_c8180_row8_col9, #T_c8180_row9_col8 {
  background-color: #ff7a7a;
  color: #f1f1f1;
}
#T_c8180_row8_col10, #T_c8180_row10_col8 {
  background-color: #ff8888;
  color: #f1f1f1;
}
#T_c8180_row9_col10, #T_c8180_row10_col9 {
  background-color: #ffa6a6;
  color: #000000;
}
</style>agesexbmibps1s2s3s4s5s6targetage1.000.220.180.190.230.18-0.040.190.280.320.13sex0.221.000.290.31-0.050.08-0.410.300.130.270.27bmi0.180.291.000.550.160.18-0.430.450.430.490.66bp0.190.310.551.000.090.04-0.200.190.360.440.51s10.23-0.050.160.091.000.880.070.570.500.260.09s20.180.080.180.040.881.00-0.160.660.230.180.09s3-0.04-0.41-0.43-0.200.07-0.161.00-0.72-0.37-0.30-0.46s40.190.300.450.190.570.66-0.721.000.600.410.41s50.280.130.430.360.500.23-0.370.601.000.520.46s60.320.270.490.440.260.18-0.300.410.521.000.35target0.130.270.660.510.090.09-0.460.410.460.351.00Scopemodel_inspectormakes some attempt to support estimators from popular
libraries other thanscikit-learnthat mimic thescikit-learninterface. The following estimators are specifically supported:Fromcatboost:CatBoostClassifierCatBoostRegressorFromlightgbm:LGBMClassifierLGBMRegressorFromxgboost:XGBClassifierXGBRegressorInstallpip install model_inspectorAlternativesYellowbrickYellowbrickis similar to Model
Inspector in that it provides tools for visualizing the behavior ofscikit-learnmodels.The two libraries have different designs. Yellowbrick usesVisualizerobjects, each class of which corresponds to a single type of
visualization. TheVisualizerinterface is similar to thescikit-learntransformer and estimator interfaces. In constrast,model_inspectorusesInspectorobjects that bundle together ascikit-learnmodel, anXfeature DataFrame, and aytarget Series.
TheInspectorobject does the work of identifying appropriate
visualization types for the specific model and dataset in question and
exposing corresponding methods, making it easy to visualize a given
model for a given dataset in a variety of ways.Another fundamental difference is that Yellowbrick is framed as a
machine learningvisualizationlibrary, while Model Inspector treats
visualization as just one approach to inspecting the behavior of machine
learning models.SHAPSHAPis another library that
provides a set of tools for understanding the behavior of machine
learning models. It has a somewhat similar design to Model Inspector in
that it usesExplainerobjects to provide access to methods that are
appropriate for a given model. It has broader scope than Model Inspector
in that it supports models from frameworks such as PyTorch and
TensorFlow. It has narrower scope in that it only implements methods
based on Shapley values.AcknowledgmentsMany aspects of this library were inspired byFastAI
courses, including bundling together a model
with data in a class and providing certain specific visualization
methods such as feature importance bar plots, feature clusters
dendrograms, tree diagrams, waterfall plots, and partial dependence
plots. Its primary contribution is to make all of these methods
available in a single convenient interface.