Module Import alias Import alias Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Import alias Import alias identifier Import alias identifier Import alias Import alias Import alias Import alias Import alias ImportFrom alias Import alias Import alias Import alias ImportFrom alias alias Import alias Import alias FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call NameLoad For NameStore NameLoad body Assign NameStore Call AttributeLoad NameLoad attr ListComp SubscriptLoad NameLoad Index Num comprehension NameStore NameLoad Assign NameStore Call NameLoad ListComp SubscriptLoad NameLoad Index NameLoad comprehension NameStore NameLoad FunctionDef arguments args NameParam defaults body Assign NameStore Call NameLoad For NameStore NameLoad body Assign NameStore Call AttributeLoad NameLoad attr ListComp SubscriptLoad NameLoad Index Num comprehension NameStore SubscriptLoad NameLoad Index NameLoad Return BinOpDiv Call NameLoad ListComp SubscriptLoad NameLoad Index NameLoad comprehension NameStore NameLoad NameLoad decorator_list Assign AttributeStore NameLoad attr Call NameLoad NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults ListLoad body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad AttributeLoad NameLoad attr Assign NameStore Num While CompareNotEq Call NameLoad NameLoad NameLoad body Assign NameStore Num Assign NameStore SubscriptLoad NameLoad Index Num For NameStore NameLoad body Assign NameStore BinOpSub Call NameLoad BinOpAdd NameLoad ListLoad NameLoad NameLoad If CompareGt NameLoad NameLoad body Assign NameStore NameLoad Assign NameStore NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad AugAssignAdd NameStore NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad body FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad For NameStore NameLoad body For TupleStore NameStore NameStore NameLoad body If UnaryOpNot NameLoad body Assign NameStore Num AugAssignAdd SubscriptStore NameLoad Index NameLoad Call AttributeLoad NameLoad attr NameLoad For NameStore NameLoad body Assign SubscriptStore NameLoad Index NameLoad Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index NameLoad Assign NameStore Call NameLoad NameLoad NameLoad Return NameLoad Assign NameStore Call NameLoad NameLoad NameLoad Return NameLoad decorator_list If CompareIn NameLoad ListLoad Str Str body Assign NameStore ListComp SubscriptLoad NameLoad Index Num comprehension NameStore SubscriptLoad SubscriptLoad NameLoad Index Str Index NameLoad Assign NameStore ListComp SubscriptLoad NameLoad Index Num comprehension NameStore SubscriptLoad SubscriptLoad NameLoad Index Str Index NameLoad orelse Assign NameStore SubscriptLoad SubscriptLoad NameLoad Index Str Index NameLoad Assign NameStore SubscriptLoad SubscriptLoad NameLoad Index Str Index NameLoad Assign NameStore Call NameLoad NameLoad Assign NameStore Call NameLoad NameLoad If UnaryOpNot NameLoad body Return TupleLoad Call NameLoad NameLoad NameLoad Call NameLoad NameLoad NameLoad orelse Assign NameStore ListLoad For NameStore Call NameLoad NameLoad body Assign NameStore IfExp CompareEq BinOpMod NameLoad Num Num NameLoad NameLoad Assign NameStore Call NameLoad NameLoad BinOpAdd NameLoad Num NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore Call NameLoad Call NameLoad SubscriptLoad SubscriptLoad NameLoad Index Str Index NameLoad Return TupleLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad Call NameLoad SubscriptLoad SubscriptLoad NameLoad Index Str Index NameLoad NameLoad keyword NameLoad Return TupleLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Assign NameStore Call NameLoad For NameStore NameLoad body If CompareIn NameLoad ListLoad Str Str body Assign TupleStore NameStore NameStore SubscriptLoad NameLoad Index NameLoad orelse Assign NameStore SubscriptLoad NameLoad Index NameLoad Assign NameStore Call AttributeLoad NameLoad attr ListComp SubscriptLoad NameLoad Index Num comprehension NameStore NameLoad CompareIn SubscriptLoad NameLoad Index Num NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Assign NameStore Lambda arguments args NameParam NameParam defaults BoolOpOr BoolOpAnd CompareGt NameLoad Num CompareGt NameLoad Num BoolOpAnd CompareLtE NameLoad Num CompareLtE NameLoad Num Assign NameStore Lambda arguments args NameParam NameParam defaults CompareEq Call NameLoad ListComp SubscriptLoad NameLoad Index Num comprehension NameStore NameLoad CompareIn SubscriptLoad NameLoad Index Num NameLoad Num Assign NameStore Num For NameStore Call NameLoad Call NameLoad NameLoad body If CompareIn NameLoad ListLoad Str Str body Assign TupleStore NameStore NameStore SubscriptLoad NameLoad Index NameLoad If CompareEq NameLoad Str body Assign NameStore BinOpAdd Call NameLoad ListComp SubscriptLoad NameLoad Index Num comprehension NameStore NameLoad NameLoad orelse If CompareEq NameLoad Str body Assign NameStore NameLoad Assign NameStore BinOpSub NameLoad Call NameLoad ListComp SubscriptLoad NameLoad Index Num comprehension NameStore NameLoad CompareIn SubscriptLoad NameLoad Index Num NameLoad If UnaryOpNot Call NameLoad NameLoad NameLoad body AugAssignAdd NameStore Num orelse Assign NameStore SubscriptLoad NameLoad Index NameLoad If UnaryOpNot Call NameLoad NameLoad NameLoad body AugAssignAdd NameStore Num Return NameLoad decorator_list FunctionDef arguments args defaults body Assign NameStore Call AttributeLoad NameLoad attr keyword Str Expr Call AttributeLoad NameLoad attr Str Str keyword NameLoad keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr Str Str keyword NameLoad keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr Str Str keyword NameLoad keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr Str Str keyword NameLoad keyword NameLoad keyword Str keyword Str Expr Call AttributeLoad NameLoad attr Str Str keyword NameLoad keyword NameLoad keyword Num keyword Str Expr Call AttributeLoad NameLoad attr Str Str keyword NameLoad keyword NameLoad keyword Num keyword Str Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore AttributeLoad NameLoad attr Assign NameStore Lambda arguments args NameParam NameParam NameParam NameParam defaults IfExp CompareGt NameLoad NameLoad CompareLt NameLoad NameLoad CompareGt NameLoad NameLoad Assign NameStore ListLoad Str Str Str Str Assign NameStore Num Assign NameStore AttributeLoad NameLoad attr Assign NameStore Num If CompareEq AttributeLoad NameLoad attr Str body Assign NameStore NameLoad orelse If CompareEq AttributeLoad NameLoad attr Str body Assign NameStore Lambda arguments args NameParam NameParam NameParam defaults Call NameLoad NameLoad NameLoad NameLoad keyword NameLoad orelse If CompareEq AttributeLoad NameLoad attr Str body Assign NameStore NameLoad Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Lambda arguments args defaults ListLoad Assign NameStore Call AttributeLoad NameLoad attr Lambda arguments args defaults ListLoad For NameStore Call NameLoad NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Lambda arguments args defaults ListLoad For NameStore SubscriptLoad Call AttributeLoad NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr BinOpMod Str AttributeLoad NameLoad attr Slice Num body AugAssignAdd NameStore Num Assign NameStore Call AttributeLoad NameLoad attr Call NameLoad NameLoad Assign NameStore SubscriptLoad NameLoad Index Str Assign NameStore SubscriptLoad NameLoad Index Str Assign NameStore SubscriptLoad NameLoad Index Str Assign NameStore SubscriptLoad NameLoad Index Str Assign NameStore SubscriptLoad NameLoad Index Str For NameStore NameLoad body If Call AttributeLoad NameLoad attr Str body Assign TupleStore NameStore NameStore Call NameLoad NameLoad Str NameLoad Assign NameStore SubscriptLoad SubscriptLoad NameLoad Index Str Index Str Assign NameStore SubscriptLoad SubscriptLoad NameLoad Index Str Index Str orelse If Call AttributeLoad NameLoad attr Str body Assign TupleStore NameStore NameStore Call NameLoad NameLoad Str NameLoad Assign NameStore SubscriptLoad SubscriptLoad NameLoad Index Str Index Str Assign NameStore SubscriptLoad SubscriptLoad NameLoad Index Str Index Str orelse Assign TupleStore NameStore NameStore Call NameLoad NameLoad NameLoad NameLoad Assign NameStore SubscriptLoad SubscriptLoad NameLoad Index Str Index NameLoad Assign NameStore SubscriptLoad SubscriptLoad NameLoad Index Str Index NameLoad If CompareNotEq AttributeLoad NameLoad attr Str body Assign NameStore Call NameLoad NameLoad NameLoad NameLoad NameLoad Assign NameStore Call NameLoad NameLoad NameLoad NameLoad NameLoad orelse Assign NameStore NameStore NameLoad Assign NameStore Call NameLoad NameLoad NameLoad NameLoad NameLoad Assign NameStore Call NameLoad NameLoad NameLoad NameLoad NameLoad While CompareEq NameLoad NameLoad body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Num Num Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Num Num Expr Call AttributeLoad SubscriptLoad NameLoad Index NameLoad attr Call NameLoad Call NameLoad NameLoad NameLoad NameLoad NameLoad Expr Call AttributeLoad SubscriptLoad NameLoad Index Str attr Call NameLoad Call NameLoad NameLoad NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad AttributeLoad NameLoad attr attr For NameStore NameLoad body Expr Call AttributeLoad SubscriptLoad NameLoad Index NameLoad attr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index NameLoad Print Str For NameStore NameLoad body Print NameLoad Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index NameLoad decorator_list If CompareEq NameLoad Str body Expr Call NameLoad
N/A N/A sys N/A copy N/A N/A N/A N/A sys path append .. N/A time N/A numpy np N/A scipy sp N/A scipy.stats N/A sklearn N/A xgboost N/A xgboost.sklearn N/A explainers load_datasets * N/A glob N/A argparse N/A collections sklearn ensemble cross_validation N/A pickle N/A parzen_windows submodular_fn N/A N/A explanations feature_value N/A N/A N/A TODO: Detail this N/A z_words N/A set N/A exp explanations N/A N/A z_words N/A N/A z_words union N/A N/A x N/A 0 N/A x exp N/A normalizer N/A sum N/A N/A feature_value N/A w N/A w z_words fnz N/A N/A x N/A N/A N/A all_words N/A set N/A doc x N/A N/A all_words N/A N/A all_words union N/A N/A x N/A 0 N/A x N/A explanations N/A doc N/A N/A N/A sum N/A N/A feature_value N/A w N/A w all_words normalizer N/A N/A N/A fnz num_items N/A len explanations N/A fnz N/A greedy N/A N/A submodular_fn k chosen N/A N/A N/A N/A chosen N/A N/A copy deepcopy chosen N/A all_items N/A range N/A submodular_fn num_items N/A current_value 0 N/A N/A N/A len chosen k N/A N/A best_gain 0 N/A best_item N/A all_items N/A 0 N/A i all_items N/A N/A gain N/A N/A submodular_fn N/A chosen N/A i current_value N/A N/A gain best_gain N/A N/A best_gain gain N/A best_item i N/A N/A N/A chosen append best_item N/A N/A N/A all_items remove best_item N/A current_value best_gain N/A chosen N/A submodular_pick N/A N/A pickled_map explainer B use_explanation_weights alternate N/A False False N/A get_function N/A N/A exps N/A N/A N/A feature_value N/A N/A collections defaultdict float N/A exp exps N/A N/A N/A f v exp N/A N/A N/A use_explanation_weights N/A N/A v 1 N/A N/A feature_value N/A f N/A N/A np abs v N/A f feature_value N/A N/A N/A feature_value N/A f N/A N/A np sqrt N/A feature_value N/A f N/A submodular N/A submodular_fn exps feature_value N/A submodular N/A out N/A greedy submodular B N/A out N/A N/A N/A explainer N/A parzen lime N/A N/A exps1 N/A N/A x N/A 0 N/A x N/A N/A pickled_map N/A exps1 N/A explainer N/A exps2 N/A N/A x N/A 0 N/A x N/A N/A pickled_map N/A exps2 N/A explainer N/A N/A exps1 N/A N/A pickled_map N/A exps1 N/A explainer N/A exps2 N/A N/A pickled_map N/A exps2 N/A explainer N/A fn1 N/A get_function exps1 N/A fn2 N/A get_function exps2 N/A N/A alternate N/A N/A N/A N/A greedy fn1 B N/A greedy fn2 B N/A N/A ret N/A N/A i N/A range B N/A N/A fn N/A N/A N/A i 2 0 fn1 fn2 N/A ret N/A greedy fn N/A i 1 ret N/A ret N/A all_pick N/A N/A pickled_map explainer B N/A N/A N/A list_ N/A range N/A len N/A N/A pickled_map N/A exps1 N/A explainer N/A N/A list_ list_ N/A random_pick N/A N/A pickled_map explainer B N/A N/A N/A list_ N/A N/A N/A np random choice N/A range N/A len N/A N/A pickled_map N/A exps1 N/A explainer B replace False N/A N/A list_ list_ N/A find_untrustworthy N/A N/A explainer exps instances untrustworthy N/A N/A N/A found N/A set N/A i instances N/A N/A N/A explainer N/A lime parzen N/A N/A N/A exp mean N/A exps N/A i N/A N/A exp N/A exps N/A i N/A found N/A N/A found union N/A N/A x N/A 0 N/A x exp N/A N/A x N/A 0 untrustworthy N/A found N/A tally_mistrust N/A N/A explainer exps predict_probas untrustworthy N/A N/A N/A trust_fn N/A N/A N/A prev curr N/A N/A N/A N/A prev 0.5 N/A curr 0.5 N/A N/A prev 0.5 N/A curr 0.5 N/A trust_fn_all N/A N/A N/A exp unt N/A N/A N/A len N/A N/A x N/A 0 N/A x exp N/A N/A x N/A 0 unt 0 N/A mistrust 0 N/A i N/A range N/A len exps N/A N/A N/A explainer N/A lime parzen N/A N/A N/A exp mean N/A exps N/A i N/A N/A explainer lime N/A N/A prev_tot N/A N/A sum N/A N/A x N/A 1 N/A x exp mean N/A N/A N/A explainer parzen N/A N/A prev_tot mean N/A tot N/A prev_tot N/A sum N/A N/A x N/A 1 N/A x exp N/A N/A x N/A 0 untrustworthy N/A N/A N/A trust_fn tot prev_tot N/A N/A mistrust 1 N/A N/A exp N/A exps N/A i N/A N/A N/A trust_fn_all exp untrustworthy N/A N/A mistrust 1 N/A mistrust N/A main N/A N/A N/A N/A N/A parser N/A N/A argparse ArgumentParser description Evaluate some explanations N/A N/A N/A parser add_argument --dataset -d type str required True help dataset name N/A N/A N/A parser add_argument --output_folder -o type str required True help output folder N/A N/A N/A parser add_argument --num_features -k type int required True help num features N/A N/A N/A parser add_argument --pick -p type str required False default all help all, submodular, submodular2 or random N/A N/A N/A parser add_argument --num_instances -n type int required False default 1 help number of instances to look at N/A N/A N/A parser add_argument --num_rounds -r type int required False default 10 help num rounds N/A args N/A N/A parser parse_args N/A dataset N/A args dataset N/A got_right N/A N/A N/A test1 test2 mistrust1 mistrust2 N/A N/A N/A test1 test2 N/A mistrust1 mistrust2 N/A mistrust1 mistrust2 N/A names N/A lime parzen random greedy N/A num_exps 0 N/A B N/A args num_instances N/A rounds 1 N/A N/A N/A args pick all N/A N/A pick_function all_pick N/A N/A N/A N/A args pick submodular N/A N/A pick_function N/A N/A N/A a b c N/A N/A submodular_pick a b c use_explanation_weights True N/A N/A N/A N/A args pick random N/A N/A pick_function random_pick N/A rounds N/A args num_rounds N/A accuracy N/A N/A collections defaultdict N/A N/A N/A N/A N/A N/A right N/A N/A collections defaultdict N/A N/A N/A N/A N/A N/A r N/A range rounds N/A N/A right N/A N/A collections defaultdict N/A N/A N/A N/A N/A N/A filez N/A N/A N/A glob glob N/A N/A N/A os path join N/A args output_folder N/A comparing_%s* N/A args dataset N/A 800 N/A N/A num_exps 1 N/A pickled_map N/A N/A pickle load N/A open filez N/A predict_probas N/A pickled_map N/A predict_probas1 N/A predict_probas2 N/A pickled_map N/A predict_probas2 N/A test1 N/A pickled_map N/A test_acc1 N/A test2 N/A pickled_map N/A test_acc2 N/A untrustworthy N/A pickled_map N/A untrustworthy N/A explainer names N/A N/A N/A N/A explainer startswith lime N/A N/A N/A pick1 pick2 N/A pick_function pickled_map lime B N/A exps1 N/A N/A pickled_map N/A exps1 N/A lime N/A exps2 N/A N/A pickled_map N/A exps2 N/A lime N/A N/A N/A N/A explainer startswith parzen N/A N/A N/A pick1 pick2 N/A pick_function pickled_map parzen B N/A exps1 N/A N/A pickled_map N/A exps1 N/A parzen N/A exps2 N/A N/A pickled_map N/A exps2 N/A parzen N/A N/A N/A pick1 pick2 N/A pick_function pickled_map explainer B N/A exps1 N/A N/A pickled_map N/A exps1 N/A explainer N/A exps2 N/A N/A pickled_map N/A exps2 N/A explainer N/A N/A N/A args pick all N/A N/A unt1 N/A find_untrustworthy explainer exps1 pick1 untrustworthy N/A unt2 N/A find_untrustworthy explainer exps2 pick2 untrustworthy N/A N/A unt1 unt2 untrustworthy N/A mistrust1 N/A tally_mistrust explainer exps1 predict_probas unt1 N/A mistrust2 N/A tally_mistrust explainer exps2 predict_probas2 unt2 N/A N/A mistrust1 mistrust2 N/A N/A mistrust1 N/A N/A N/A np random randint 0 10 N/A mistrust2 N/A N/A N/A np random randint 0 10 N/A N/A N/A N/A right N/A explainer append N/A int N/A got_right test1 test2 mistrust1 mistrust2 N/A N/A N/A N/A right N/A random_choice append N/A int N/A got_right test1 test2 N/A N/A N/A np random random N/A N/A N/A np random random N/A name right N/A N/A N/A N/A N/A accuracy N/A name append N/A N/A np mean N/A right N/A name N/A Mean accuracy: N/A name right N/A N/A name N/A N/A np mean N/A accuracy N/A name N/A N/A N/A __name__ __main__ N/A N/A N/A main