Package: glocalx
GLocalX - Global through Local ExplainabilityExplanations come in two forms: local, explaining a single model prediction, and global, explaining all model predictions. The Local to Global (L2G) problem consists in bridging these two family of explanations. Simply put, we generate global explanations by merging local ones.The algorithmLocal and global explanations are provided in the form of decision rules:age < 40, income > 50000, status = married, job = CEO ⇒ grant loan applicationThis rule describes the rationale followed given by an unexplainable model to grant the loan application to an individual younger than 40 years-old, with an income above 50000$, married and currently working as a CEO.Setupgitclonehttps://github.com/msetzu/glocalx/cdglocalxDependencies are listed inrequirements.txt, a virtual environment is advised:mkvirtualenvglocalx# optional but reccomendedpip3install-rrequirements.txtRunning the codeInstalling from pypi does not give direct access to quickstart data, first download thedata/folder from theoriginal repository.Python interfacefromtensorflow.keras.modelsimportload_modelimportlogzerofromglocalx.glocalximportGLocalX,shut_up_tensorflowfromglocalx.modelsimportRule# Set log profile: INFO for normal logging, DEBUG for verbositylogzero.loglevel(logzero.logging.INFO)shut_up_tensorflow()# Load black box: optional! Use black_box = None to use the dataset labelsblack_box=load_model('data/dummy/dummy_model.h5')# Load data and headerdata=genfromtxt('data/dummy/dummy_dataset.csv',delimiter=',',names=True)features_names=data.dtype.namestr_set=data.view(float).reshape(data.shape+(-1,))# Load local explanationslocal_explanations=Rule.from_json('data/dummy/dummy_rules.json',names=features_names)# Create a GLocalX instance for `black_box`glocalx=GLocalX(oracle=black_box)# Fit the model, use batch_size=128 for larger datasetsglocalx=glocalx.fit(local_explanations,tr_set,batch_size=2,name='black_box_explanations')# Retrieve global explanations by fidelityalpha=0.5global_explanations=glocalx.rules(alpha,tr_set)# Retrieve global explanations by fidelity percentilealpha=95global_explanations=glocalx.rules(alpha,tr_set,is_percentile=True)# Retrieve exactly `alpha` global explanations, `alpha/2` per classalpha=10global_explanations=glocalx.rules(alpha,tr_set)Command line interfaceYou can invokeGLocalXfrom the api interface inapi.py:>python3api.py--help
Usage:api.py[OPTIONS]RULESTR

Options:-o,--oraclePATH--namesTEXTFeaturesnames.-cbs,--callbacksFLOATCallbackstep,eitherintorfloat.Defaultsto0.1-m,--nameTEXTNameofthelogfiles.--generateTEXTNumberofrecordstogenerate,ifgiven.DefaultstoNone.-i,--intersectTEXTWhethertousecoverageintersection('coverage')orpolyhedraintersection('polyhedra').Defaultsto'coverage'.-f,--fidelity_weightFLOATFidelityweight.Defaultsto1.-c,--complexity_weightFLOATComplexityweight.Defaultsto1.-a,--alphaFLOATPruningfactor.Defaultsto0.5-b,--batchINTEGERBatchsize.Setto-1forfullbatch.Defaultsto128.-u,--undersampleFLOATUndersamplesize,touseapercentageoftherules.Defaultsto1.0(Noundersample).--strict_joinUsetousehighconcordance.--strict_cutUsetousethestrongcut.--global_directionUsetousetheglobalsearchdirection.-d,--debugINTEGERDebuglevel.--helpShowthismessageandexit.A minimal run simply requires a set of local input rules, a training set and (optionally) a black box:python3.8apy.pydata/dummy/dummy_rules.jsondata/dummy/dummy_dataset.csv--oracledata/dummy/dummy_model.h5--namedummy--batch2If you are interested, the folderdata/dummy/contains a dummy example.
You can run it withpython3.8apy.pymy_rules.jsontraining_set.csv--oraclemy_black_box.h5\--namemy_black_box_explanationsThe remaining hyperparameters are optional:--names $namesComma-separated features names--callbacks $callbacksCallback step, either int or float. Callbacks are invoked every--callbacksiterations of the algorithm--name $nameName of the log files and output. The script dumps here all the additional info as it executes--generate $sizeNumber of synthetic records to generate, if you don't wish to use the provided training set--intersect $strategyIntersection strategy: eithercoverageorpolyhedra. Defaults tocoverage--fidelity_weight $fidelityFidelity weight to reward accurate yet complex models. Defaults to 1.--complexity_weight $complexityComplexity weight to reward simple yet less accurate models. Defaults to 1.--alpha $alphaPruning factor. Defaults to 0.5--batch $batchBatch size. Set to -1 for full batch. Defaults to 128.--undersample $pctUndersample size, to use a percentage of the input rules. Defaults to 1.0 (all rules)--strict_jointo use a more stringentjoin--strict_cutto use a more stringentcut--global_directionUse to evaluate merges on the whole validation set--debugDebug level: the higher, the less messages shownValidationGLocalXoutputs a set of rules (list ofmodels.Rule) stored in a$names.rules.glocalx.alpha=$alpha.jsonfile.Theevaluators.validatefunction provides a simple interface for validation. If you wish to extend it, you can directly extend either theevaluators.DummyEvaluatororevaluators.MemEvaluatorclass.Run on your own datasetGLocalX has a strict format on input data. It accepts tabular datasets and binary classification tasks. You can find a dummy example for each of these formats in/data/dummy/.Local rulesWe provide an integration withLorerules through theloadersmodule.
To convert Lore rules to GLocalX rules, use the providedloaders.lore.lore_to_glocalx(json_file, info_file)function.json_fileshould be the path to a json with Lore's rules, andinfo_fileshould be the path to a JSON dictionary holding the class values (keyclass_names) and a list of the features' names (keyfeature_names).
You can find dummy examples of the info file indata/loaders/adult_info.json.More examples on other local rule extractors to follow.Rules [/data/dummy/dummy_rules.json]Local rules are to be stored in aJSONformat:[{"22":[30.0,91.9],"23":[-Infinity,553.3],"label":0},...]Each rule in the list is a dictionary with an arbitrary (greater than 2) premises. The rule prediction ({0, 1}) is stored in the keylabel. Premises on features are stored according to their ordering and bounds: in the above,"22": [-Infinity, 91.9]indicates the premise "feature number 22 has value between 30.0 and 91.9".Black boxes [/data/dummy/dummy_model.h5]Black boxes (if used) are to be stored in ahdf5format if given through command line. If given programmatically instead, it suffices that they implement thePredictorinterface:classPredictor:@abstractmethoddefpredict(self,x):passwhen called to predictnumpy.ndarray:xthe predictor shall return its predictions in anumpy.ndarrayof integers.Training data[/data/dummy/dummy_dataset.csv]Training data is to be stored in a csv, comma-separated format with features names as header. The classification labels should have feature namey.Docs and referenceYou can find the software documentation in the/html/folder and a powerpoint presentation on GLocalX can be foundhere.You can cite this work with@article{SETZU2021103457,
  title = {GLocalX - From Local to Global Explanations of Black Box AI Models},
  journal = {Artificial Intelligence},
  volume = {294},
  pages = {103457},
  year = {2021},
  issn = {0004-3702},
  doi = {https://doi.org/10.1016/j.artint.2021.103457},
  url = {https://www.sciencedirect.com/science/article/pii/S0004370221000084},
  author = {Mattia Setzu and Riccardo Guidotti and Anna Monreale and Franco Turini and Dino Pedreschi and Fosca Giannotti},
  keywords = {Explainable AI, Global explanation, Local explanations, Interpretable models, Open the black box},
}Useful functions & OverridesCallbacksThefit()function provides acallbacksparameter to add any callbacks you desire to be invoked everycallbacks_stepiterations. The callback should implement thecallbacks.Callbackinterface. You can find the set of parameters available to the callback inglocalx.GLocalX.fit().Serialization and deserializationYou can dump to disk and loadGLocalXinstances and their output with theRuleobject and theserializationmodule:frommodelsimportRuleimportserializationrules_only_json='input_rules.json'run_file='my_run.glocalx.json'# Load input rulesrules=Rule.from_json(rules_only_json)# Load GLocalX outputglocalx_output=serialization.load_run(run_file)# Load a GLocalX instance from a set of rules, regardless of whether they come from an actual run or not!# From a GLocalX run...glocalx=serialization.load_glocalx(run_file,is_glocalx_run=True)# From aglocalx=serialization.load_glocalx(rules_only_json,is_glocalx_run=False)Extending themergefunctionTo override the merge function, simply extend theglocalx.GLocalXobject and override themergefunction with the following signature:merge(self,A:set,B:set,x:numpy.ndarray,y:numpy.ndarray,ids:numpy.ndarray)whereAandBare the sets ofmodels.Ruleyou are merging,xis the training data,yare the training labels andidsare the batch ids. The ids are used by theMemEvaluatorto store pre-computed results.Extending thedistancefunctionThedistancebetween explanations is computed by theevaluators.Evaluatorobjects. To override it, override either theevaluators.DummyEvaluatororevaluators.MemEvaluatorobject with the following signature:distance(self,A:set,B:set,x:numpy.ndarray,ids:numpy.ndarray)->numpy.ndarraywhereA,Bare the two (sets of) explanation(s),xis the training data andidsare the ids for the current batch.Extending the merge/acceptanceWhether a merge is accepted or rejected is decided by theglocalx.GLocalX.accept_merge()function with signature:accept_merge(union:set,merge:set,**kwargs)->bool