# Open and Extensible Benchmark for Explainable Artificial Intelligence Methods
## 3 Benchmark Structure
### Users and Use Cases
#### 3.5.2 Use Cases  
The use cases presented in this section are based on the potential users identified in the previous one. The use cases are divided into groups depending on how users can interact with the benchmark and its the main functions. The first group of use cases is utilization of the benchmark. This is an experiment that results in reproduction to verify metric values that were published. There are also different kinds of evaluations as follows: a single method on a specific setup, a method against a full set of metrics, several methods on some setup, etc. To reproduce the experiment results on the full setup using the XAIB, one needs to install it and run an evaluation. The benchmark provides installation instructions on its documentation website and GitHub page. The instructions themselves are not so complex, since the package is available on PyPI and can be installed with a single command. The list of additional dependencies is also available and can be installed in the same fashion for users wanting a full setup. After installation, users can run the default evaluation scripts. The evaluation itself is separated into modules by explanation type. For each type of explanation, the results are written in files along with the visualizations in the same directory as the evaluation script. Reproducing methods is important, but for the users who want to test some specific existing method or their own method, something less general is needed. For this purpose, users can use special interfaces that were mentioned in the Structure section. In the Listing 2, an example of how to use an existing method is offered. There is a need to create and configure entities that is not within the scope of this experiment. To not overwhelm the user and create additional complexities, all of it is encapsulated within the benchmark itself. Since the particular explainer is the focus of this experiment, the user is responsible for creating and configuring it. If the user wants to evaluate this method on all of the cases, the previous example can be continued as shown in Listing 3. The second group is result inspection. The main function of a benchmark is to provide users with a ranking of competing methods. To be able to compare methods in general, users should be able to see the aggregated results for each method. For a more in-depth view of each method, in particular, metric values for each method should be presented separately. Since the XAI evaluation is very high-dimensional, it requires special care when visualizing results. For a benchmark, the representation of results is one of the most important aspects. ```
fromxaib.explainers.feature_importance.lime_explainerimportLimeExplainer fromxaib.evaluationimportDatasetFactory,ModelFactory train_ds,test_ds =DatasetFactory().get(\"synthetic\") model =ModelFactory(train_ds,test_ds).get(\"svm\") explainer =LimeExplainer(train_ds,labels=[0,1]) sample =[test_ds[i][\"it=\"]foriinrange(10)] explanations =explainer.predict(sample,model)
```  
Listing 2 Existing method evaluation example. ```
fromxaib.evaluation.feature_importanceimportExperimentFactory fromxaib.evaluation.utilsimportvisualize_results experiment_factory =ExperimentFactory( repo_path =\"results\", explainers = {\"lime\":explainer}, test_ds =test_ds, model =model, labels = [0,1], batch_size = 10 ) experiments =experiment_factory.get(\"all\") fornameinexperiments: experiments[name]() visualize_results(\"results\",\"results/results.png\")
```  
Listing 3: Existing method evaluation example. The XAIB can help those wanting to inspect evaluation results by using the information published on the documentation website. For a general comparison, there is an aggregated bar plot with all the methods averaged over all setups. The website also provides results for each method per dataset for comparing the influence of datasets on the method's performance. In addition to that, a full table with all metric values for each combination tested is also published.