Package: oae
OAEThis package implements thispaperin which the author tries to address the problem of interpretability and actionability of tree-based models. The author of the paper presents a novel framework to post-process any tree-based classifier to extract an optimal actionable plan that can change a given input to a desired class with a minimum cost. Currently this package only supports scikit-learn's implementation of Random Forest.Installpip install oaeHow to useimportnumpyasnpimportpandasaspdfromoae.coreimport*fromoae.treeimport*fromoae.optimizerimport*fromsklearn.ensembleimportRandomForestClassifierfromsklearn.model_selectionimporttrain_test_splitasttsfromsklearn.metricsimportaccuracy_score,roc_auc_scoreSEED=41np.random.seed(SEED)data=get_external_dataset(URLS['BREAST_CANCER'])data.target.value_counts(normalize=True)2    0.655222
4    0.344778
Name: target, dtype: float64Convertbenignrepresented as2andmalignantrepresented as4to0and1respectively.# convert benignalbls,lbl_map=pd.factorize(data['target'])Let's look at the data-type of the featuresdata.dtypescode_number                     int64
clump_thickness                 int64
cell_size_uniformity            int64
cell_shape_uniformity           int64
marginal_adhesion               int64
single_epithelial_cell_size     int64
bare_nuclei                    object
bland_chromatin                 int64
normal_nucleoli                 int64
mitoses                         int64
target                          int64
dtype: objectdata.bare_nuclei.unique()array(['1', '10', '2', '4', '3', '9', '7', '?', '5', '8', '6'],
      dtype=object)Let's replace this?with-1and convert it into int64 like othersdata=data.assign(bare_nuclei=data.bare_nuclei.str.replace('?','-1').astype(np.int))data=data.assign(target=lbls);data.head()<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>code_numberclump_thicknesscell_size_uniformitycell_shape_uniformitymarginal_adhesionsingle_epithelial_cell_sizebare_nucleibland_chromatinnormal_nucleolimitosestarget0100002551112131101100294554457103210210154253111223110310162776881343710410170234113213110data.iloc[:,1:-1].nunique()clump_thickness                10
cell_size_uniformity           10
cell_shape_uniformity          10
marginal_adhesion              10
single_epithelial_cell_size    10
bare_nuclei                    11
bland_chromatin                10
normal_nucleoli                10
mitoses                         9
dtype: int64All of the features of interest ( excluding code_number and target ) are categorical variables. Let's create a holdout set and train a Random Forest Classifier.SEED=41np.random.seed(SEED)features=data.columns[1:-1]Xtr,Xte,ytr,yte=tts(data.loc[:,features],data.target,test_size=.2,random_state=SEED)clf=RandomForestClassifier(n_estimators=10,n_jobs=-1,random_state=SEED)clf.fit(Xtr,ytr)print(f'train accuracy:{accuracy_score(ytr,clf.predict(Xtr))}')print(f'holdout accuracy:{accuracy_score(yte,clf.predict(Xte))}')train accuracy: 0.998211091234347
holdout accuracy: 0.9714285714285714Let's select an instance from holdout set and look at the ground. We realize that the classifier marks it asmalignantand we want to know what features could be changed so that classifier would mark it asbenign.instanceidx=4print(yte.iloc[instanceidx],' ',clf.predict_proba(Xte.iloc[instanceidx:instanceidx+1]))1   [[0. 1.]]Now we will try to extract an optimal action problem by posing this problem as an Integer Linear Programming problem.atm=ATMSKLEARN(clf,data.loc[:,features].values)instance=Instance(Xte.iloc[instanceidx],['categorical']*len(features))We would be using the following cost function so our  OAE problem minimize the number of changed features, i.e. Hamming distance.But we don' need to restrict ourselves to this particular cost function, you can design your cost function and pass it to the solver.In this example our input has ground label1and we want to find out how to tweak features with minimum cost such that classifier classifies it as label0withzbeing the target threshold.$F(x) = \frac{1}{w_{t}} \sum_{k=1}^{m_t} h_{t,k}\phi_{t,k} \geq z$, where $h_{t_k} \in R$$F(x)$ represents the probability estimate from Random Forest Classifier.opt=Optimizer(cost_matrix,combine,z=0.45,class_=0)v_i_j_sol,phi_t_k_sol=opt.solve(atm,instance)The package would help suggest changes that should be made to the feature to move it from classified asmalignantto beingbenign.atm.suggest_changes(v_i_j_sol,instance)['no change, current value: 5',
 'no change, current value: 3',
 'no change, current value: 5',
 'no change, current value: 1',
 'no change, current value: 8',
 'current value: 10, proposed change: [-1, 1]',
 'current value: 5, proposed change: [3, 4]',
 'no change, current value: 3',
 'no change, current value: 1']An action plan is extracted which says that we need to change5th featurewhich currently has value10to-1and6th featureto3and then our classifier would classify it as label0. Let's find it out.X_transformed=atm.transform(v_i_j_sol,instance);X_transformed<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>clump_thicknesscell_size_uniformitycell_shape_uniformitymarginal_adhesionsingle_epithelial_cell_sizebare_nucleibland_chromatinnormal_nucleolimitoses053518-1331clf.predict_proba(X_transformed)array([[0.6, 0.4]])Indeed we can see that classifier will label it as0and probability is also greater thanz=0.45so it also satisfies that concern as well.ApplicationsOne example coult be in targeted marketing, we can use the action plan generated per customer to better understand which all levers can we pull to get desired results.