zhiminy/EvalEng huggingface dataset:
Repository: understandable-machine-intelligence-lab/Quantus
Issue: Support Model-Agnostic Explainers
Root cause: Architectural Constraint

Hi there, firstly-- kudos for this amazing project :)

### Description of the problem

After reviewing the examples and API usage, I understood that I needed to rely on Tensorflow or Torch. Moreover, even ModelInterface is coupled to Computer Vision-related models.

For example:
```
class ModelInterface(ABC, Generic[M]):
    """Base ModelInterface for torch and tensorflow models."""

    def __init__(
        self,
        model: M,
        **channel_first: Optional[bool] = True,**
        softmax: bool = False,
        model_predict_kwargs: Optional[Dict[str, Any]] = None,
    ):
        """
        Initialisation of ModelInterface class.

        Parameters
        ----------
        model: torch.nn.Module, tf.keras.Model
            A model this will be wrapped in the ModelInterface:
        channel_first: boolean, optional
             Indicates of the image dimensions are channel first, or channel last. Inferred from the input shape if None.
```
Also, the `explanation_fun` is heavily coupled to existing libraries, making it difficult to use if the user wants to rely on a library different than `captum`; `zennit` or `tf_explain`, like `shap`. 

### Description of a solution

Before finding this project, I've worked with Captum before. There, it is flexible enough that even though the library provides its own attribution methods, the `metrics` API lets the user define any explanation method that she/he wants and easily integrates into the model. I only need to return a Tensor of attributions.

Take this as an example:
```
from captum.metrics import sensitivity_max
import numpy as np
import shap

masker = shap.maskers.Independent(X_test)
def model_log_odds(x):
    log_prob = np.log(model.predict_proba(x) + 1e-20)
    result = log_prob[:, 1] - log_prob[:, 0]
    return result

exact_shap = shap.explainers.ExactExplainer(model_predict, masker)

def shap_exact_function(inputs):
    shap_values = exact_shap(input_tensor.cpu().numpy())
    attributions_shap = torch.tensor(shap_values.values)
    return attributions_shap

**sensitivity_score = sensitivity_max(
    explanation_func=shap_explanation_function,
    inputs=X_test_tensor,
    perturb_function=my_custom_perturbation_fn_generator,  # Adjust as needed
    n_perturb_samples=100,
)**

```
As I'm new to the project, I'm unsure if Quantum supports such an approach. I tried to look into the docs, but I didn't find it.

The main advantage of such design is that automatically the library becomes more agnostic, making it easy to experiment to models other than images, such as text or tabular; Moreover, it also turn possible to use models that are different than tensorflow or pytorch.


Please let me know what do you think. Thanks!


Hi @abarbosa94, 
I agree with both statements. Those are known design flaws, and we've talked about them multiple times. Unfortunately, we've never managed to come to conclusion.
Repository: understandable-machine-intelligence-lab/Quantus
Issue: The problem faced by GradCAM and Occlusion Sensitivity methods when dealing with images with more than 3 bands.
Root cause: Validation Gap

### Description
- Imagine we have a task for image classification in the deep learning model that takes a tensor with the shape of m*n*c. Here, m and n are the height and width of the input tensor respectively and c is the number of channels. The Quantus toolkit for GradCAM and Occlusion Sensitivity faces an error that originates from opencv packages and mentions that the number of bands in the input image is more than 3. Wide range of tasks work with more than 3 bands. So, maybe you can modify functions to avoid such errors! Thanks

@annahedstroem




Hi @BEEILAB thanks for sharing this! Can you paste the code that causes this error? It is helpful to know more e.g., the type fo model (torch or tf) and the dims of your data etc. Thanks!
Repository: understandable-machine-intelligence-lab/Quantus
Issue: Remove dependency of pytest-lazy-fixture which causes problems with pytest==8.0.0
Root cause: External Dependency Breakage

### Description of the problem
- Remove dependency of pytest-lazy-fixture which causes problems with pytest==8.0.0
- See problem discussed here: https://github.com/TvoroG/pytest-lazy-fixture/issues/65

Links:
- https://docs.pytest.org/en/stable/proposals/parametrize_with_fixtures.html
### Minimum acceptance criteria
- Specify what is necessary for the issue to be closed.
- @mentions of the person that is apt to review these changes e.g., @annahedstroem
Repository: understandable-machine-intelligence-lab/Quantus
Issue: ROAD takes 6 seconds for single image with GPU enabled
Root cause: Documentation Deficiency

### Description

Hello there, I think the ROAD metric may not be utilizing GPU during computation.

I have a VGG-16 based model and evaluation of single input tensor using ROAD metric takes ~6 seconds.

### Steps to reproduce the behavior

This is the code:
```py
import quantus
road = quantus.metrics.ROAD(
    percentages=[90],
)
road(
    model=model,
    x_batch=x.cpu().numpy(),
    y_batch=y.cpu().numpy(),
    a_batch=lrp_attributions.sum(axis=1).cpu().numpy(),
    device="cuda",
)
```

When I change the device to cpu (and switch model as well), the computation takes around 6 seconds as well. Attributions are precomputed using `captum`.

### Minimum acceptance criteria

Could you please verify, that this specific metric is indeed converting the tensor to CPU? I found, that in docs you mention to specify device as either `"cpu"` or `"gpu"`, but looks like the `"gpu"` option is obsolete and `"cuda"` should be used instead. Thanks in advance.

random tag since the issue guidelines say so: @annahedstroem


Looks like my mistake - apparently the 90 will remove the top 90% of attributions and this takes some while. Closing the issue, since I need to remove 10% and keep 90 :).
Repository: understandable-machine-intelligence-lab/Quantus
Issue: ROAD returns 0 all the time
Root cause: Algorithmic Error

### Description
Hello there, as the title says - looks like ROAD returns 0 all the time. In the [one of provided tutorials](https://github.com/understandable-machine-intelligence-lab/Quantus/blob/main/tutorials/Tutorial_ImageNet_Example_All_Metrics.ipynb), the output of ROAD is `0` for all percentages. In my own experiments, the metric acts the same.

### Steps to reproduce the behavior
- Initialize road metric
- Evaluate it on arbitrary data
- I would expect to at least some of the percentage to display different result than `0`
- Explanations were generated using `captum`'s LRP, I tried for Occlusion as well, same result
- Screenshot from local development
![Screenshot 2024-02-13 at 10 03 04 PM](https://github.com/understandable-machine-intelligence-lab/Quantus/assets/58729669/62222888-ce0a-4cae-a44e-18e3dd88a3b4)

Code for easier reproducibility:
```py
import quantus
road = quantus.metrics.ROAD(
    percentages=[10, 15, 20, 25, 30],
    display_progressbar=True,
)
road(
    model=model.model,
    x_batch=x.cpu().numpy(),
    y_batch=y.cpu().numpy(),
    a_batch=lrp_attributions.sum(axis=1).cpu().numpy(),
    softmax=True,
    device="cuda",
)
```

### Minimum acceptance criteria
- Please double check the metric whether it works as supposed. It is weird to consistently return 0 for all inputs
- Random tag because the description guideline says so: @annahedstroem

Thanks in advance!

I just tested implementation from `pytorch-grad-cam` and it expectedly reports the non-zero values.

@annahedstroem were you able to confirm?

hi @krebso ! thanks so much for this. we'll look into what is causing this.

what dataset did you use for this? would it be possible for you to try ROAD with a low-dim dataset?
Repository: understandable-machine-intelligence-lab/Quantus
Issue: Wrong calculation of selectivity
Root cause: Documentation Deficiency

### Description

If I am not mistaken, there is an error the calculation of the selectivity:
`results[patch_id] = y_pred_perturb`, however, AFAIK one should calculate the difference of the prediction on the original and perturbed input: `results[patch_id] = y_pred - y_pred_perturb`. 

This is also mentioned in the comment above:
`# Predict on perturbed input x and store the difference from predicting on unperturbed input. `

I am fairly new to the domain so if I'm wrong please excuse me. I already made a fix and have created a PR.
@annahedstroem 




Hi, thanks so much for your suggestion.

According to Figure 8 and the description in Section 7.2 in the original paper https://arxiv.org/pdf/1706.07979.pdf, I do however believe that the current implementation is correct.

Please share your source so we can compare!

(If you want, you can also add this functionality by creating a boolean flag like "compute_prediction_difference" (or with a better name) so to make your implementation available. Please then add 1-3 pytests to test it).

Hey Anna, thanks for the reply.

I've reread the paper and you're right. Sorry, I missed that.

It is not a different implementation, I think I was just confused by the comment
 [ # Predict on perturbed input x and store the difference from predicting on unperturbed input.](https://github.com/understandable-machine-intelligence-lab/Quantus/blob/47a41b15be43a164c6f42716aa9014a7dde90c99/quantus/metrics/faithfulness/selectivity.py#L363C12-L363C106) 
 and the calculation of y_pred in
[y_pred = float(model.predict(x_input)[:, y])](https://github.com/understandable-machine-intelligence-lab/Quantus/blob/47a41b15be43a164c6f42716aa9014a7dde90c99/quantus/metrics/faithfulness/selectivity.py#L298C1-L298C53),
Which is not necessary then, right?

ah, good that you noticed it! I'll update the comment.
Repository: understandable-machine-intelligence-lab/Quantus
Issue: import cv2 complain when installing quantus with python == 3.9 in virtual env 
Root cause: External Dependency Breakage

### Description

```
>>> python3.9 -m venv venvs/testenv
>>> source venvs/testenv/bin/activate
>>> pip install git+https://github.com/understandable-machine-intelligence-lab/Quantus.git # succesfully installs
>>> import quantus
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/annahedstroem/Projects/venvs/quantustrans/lib/python3.9/site-packages/quantus/__init__.py", line 11, in <module>
    from quantus.evaluation import evaluate
  File "/Users/annahedstroem/Projects/venvs/quantustrans/lib/python3.9/site-packages/quantus/evaluation.py", line 19, in <module>
    from quantus.functions.explanation_func import explain
  File "/Users/annahedstroem/Projects/venvs/quantustrans/lib/python3.9/site-packages/quantus/functions/explanation_func.py", line 17, in <module>
    from quantus.helpers import constants
  File "/Users/annahedstroem/Projects/venvs/quantustrans/lib/python3.9/site-packages/quantus/helpers/constants.py", line 14, in <module>
    from quantus.functions.perturb_func import *
  File "/Users/annahedstroem/Projects/venvs/quantustrans/lib/python3.9/site-packages/quantus/functions/perturb_func.py", line 14, in <module>
    import cv2
  File "/Users/annahedstroem/Projects/venvs/quantustrans/lib/python3.9/site-packages/cv2/__init__.py", line 181, in <module>
    bootstrap()
  File "/Users/annahedstroem/Projects/venvs/quantustrans/lib/python3.9/site-packages/cv2/__init__.py", line 153, in bootstrap
    native_module = importlib.import_module("cv2")
  File "/usr/local/Cellar/python@3.9/3.9.17_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: dlopen(/Users/annahedstroem/Projects/venvs/quantustrans/lib/python3.9/site-packages/cv2/cv2.abi3.so, 2): Symbol not found: __ZNSt3__113basic_filebufIcNS_11char_traitsIcEEE4openEPKcj
  Referenced from: /Users/annahedstroem/Projects/venvs/quantustrans/lib/python3.9/site-packages/cv2/.dylibs/libvmaf.1.dylib (which was built for Mac OS X 12.0)
  Expected in: /usr/lib/libc++.1.dylib
```

### Steps to reproduce the behavior

Solution that works:
```python
pip install opencv-python==4.8.1.78        
```

### Minimum acceptance criteria
- Specify what is necessary for the issue to be closed.
- @mentions of the person that is apt to review these changes e.g., @annahedstroem
Repository: understandable-machine-intelligence-lab/Quantus
Issue: Region Perturbation plotting cannot deal with certain multi-dimensional data (image no. > 1, explain_func no. == 1) 
Root cause: Algorithmic Error

### Description
The Region Perturbation plotting function throws a shape mismatch error, when trying to plot e.g. results obtained for a batch of three images for one explain function. In this case, region perturb will return a list of np.shape(batch_size, perturbation_steps). For the plotting to work, we need the perturbation steps as x_value and the results as y_value. However as is, the x_value is determined by len(results), which will return the batch_size instead of the perturbation steps.

I will raise a pull request to fix this issue right after opening this issue.

### Steps to reproduce the behavior
  - Calculate Region Perturbation results for len(x_batch) > 1 for one explainer, using quantus.RegionPerturbation
  - Try to plot the results using quantus.RegionPerturbation.plot(results=results)

<img width="875" alt="image" src="https://github.com/user-attachments/assets/ec4997bb-4a86-4c1f-ac3f-3a5854b3757e">


### Minimum acceptance criteria
- allow for plotting of one-dimensional and multi-dimensional results lists.
- briefly discussed with @annahedstroem


Fixed in PR: https://github.com/understandable-machine-intelligence-lab/Quantus/commit/bbe77038ad4b6111336ba6a3b62e27582c2d4002

Thanks @tlabarta
Repository: understandable-machine-intelligence-lab/Quantus
Issue: Application on A Customized Method 


- If I want to apply it in a customized method
- I want the pseudo code or the general scheme for this purpose

- Would You Please guide me @annahedstroem


Hi @soumiadjellikh what do you mean by a customised method? Can you provide more details so I can help? 

hello @annahedstroem , I mean a newly proposed XAI method that I want to evaluate and compare with other XAI methods using Quantus

Please check the documentation — we have examples for that use case, i.e., to create your own Callable. 

See e.g., https://quantus.readthedocs.io/en/latest/getting_started/getting_started_example.html.

Thanks! Let me know if it does not solve your issue and we'll re-open it.
Repository: understandable-machine-intelligence-lab/Quantus
Issue: Documentation on MPRT and other Randomisation Metrics not available

### Description

Documentation on MPRT and other Randomisation metrics not accessible [LINK](https://quantus.readthedocs.io/en/stable/docs_api/quantus.metrics.randomisation.model_parameter_randomisation.html).

### Minimum acceptance criteria

I can access the documentation via the source code, but I'd like to know if there is a reason for this or if it's just a bug.

Best Regards
Nicolas 


Hi Nicolas! We will look into this in the next PR! Thanks
Repository: understandable-machine-intelligence-lab/Quantus
Issue: RelativeInputStability Metric Returns Wrong Name (Relative Output Stability)

### Description
I  found an issue where the metric RelativeInputStability in Quantus returns an incorrect name "Relative Output Stability". This causes issues when matching metric names dynamically.

### Steps to reproduce the behavior

```python
import quantus
print(quantus.RelativeInputStability.name)  # Expected: 'Relative Input Stability'
# But it prints: 'Relative Output Stability'
```

### Minimum acceptance criteria
- Modify line 61 of quantus/metrics/robustness/relative_input_stability.py from 'Relative Output Stability' to 'Relative Input Stability'. 


Thanks! It is fixed in commit [68a19a1](https://github.com/understandable-machine-intelligence-lab/Quantus/commit/68a19a1125f98f0f596c12d102fbc15d31515c80)
Repository: understandable-machine-intelligence-lab/Quantus
Issue: ValueError: 'subset_size' must be smaller than input size. [1 >= 1.0] in FaithfulnessCorrelation for TABULAR DATA
Root cause: Validation Gap

**Description:**
I encountered a `ValueError` when running the `FaithfulnessCorrelation` metric in Quantus. The error message indicates that `subset_size` must be smaller than the input size, but even after setting it to a reasonable value, the issue persists.

**Error Message:**
```
ValueError: 'subset_size' must be smaller than input size. [1 >= 1.0]
```

**Steps to Reproduce:**
1. Load a tabular dataset.
2. Train a TensorFlow model on the dataset.
3. Compute SHAP explanations for the model.
4. Pass the model, input data, labels, and explanations to `FaithfulnessCorrelation`.
5. Observe the error.

**Expected Behavior:**
The metric should execute without errors if `subset_size` is properly defined within the valid range.

**Code Snippet:**
```python
import quantus as q
import numpy as np
import pandas as pd
import tensorflow as tf
import shap

# Load dataset
df = pd.read_csv("your_data.csv")
X = df.drop(columns=["target"]).values  # Features
y = df["target"].values  # Target

# Load model
model = tf.keras.models.load_model("your_model.h5")

# Generate SHAP explanations
explainer = shap.Explainer(model, X)
shap_values = explainer(X)
explanations = shap_values.values

# Fix subset_size issue
num_features = X.shape[1]
subset_size = max(1, min(num_features - 1, int(num_features * 0.5)))

faithfulness_metric = q.FaithfulnessCorrelation(
    nr_runs=100,
    subset_size=subset_size,
    perturb_func=q.perturb_func.feature_removal_by_indices,
    similarity_func=q.similarity_func.correlation_pearson,
    abs=False,
    return_aggregate=False
)

# Compute Faithfulness Correlation
faithfulness_scores = []
for x_batch, y_batch, a_batch in zip(x_batches, y_batches, a_batches):
    score = faithfulness_metric(model=model, x_batch=x_batch, y_batch=y_batch, a_batch=a_batch)
    faithfulness_scores.append(score)

print("Faithfulness Correlation Scores per Batch:", faithfulness_scores)
```

**Environment:**
- OS: Ubuntu 22.04
- Python: 3.11
- TensorFlow: 2.3
- Quantus: Latest
- SHAP: Latest


Hi there @officialarijit , I am facing the same issue. Were you able to resolve this? Thanks
Repository: understandable-machine-intelligence-lab/Quantus
Issue: Add option to evaluate `RandomLogit` with multiple samples
Root cause: Unimplemented Feature Gap

If I understand the [current implementation](https://github.com/understandable-machine-intelligence-lab/Quantus/blob/85bcf2c8e88f7b8f3334f552ee72f8a37e870e1a/quantus/metrics/randomisation/random_logit.py#L262-L306) of `RandomLogit` correctly, only a single randomized target logit is attributed. 
In my experience, this causes scores returned by the metric to be quite sensitive and random. More robustness could be obtained by averaging over more (or all) target logits.
Repository: understandable-machine-intelligence-lab/Quantus
Issue: Docs: Include MPRT metrics

The [documentation for the `randomization` module](https://quantus.readthedocs.io/en/latest/docs_api/quantus.metrics.randomisation.html) is currently missing docstrings for `MPRT`, `EfficientMPRT` and `SmoothMPRT` that [appear to be implemented](https://github.com/understandable-machine-intelligence-lab/Quantus/tree/main/quantus/metrics/randomisation).

Sorry, just noticed that this is a duplicate of #358.
Repository: understandable-machine-intelligence-lab/Quantus
Issue: Shape mismatch in NonSensitivity when features_in_step != 1
Root cause: Algorithmic Error

When calling the Non-Sensitivity metric with a features_in_step parameter different from 1, a ValueError occurs during evaluation.

Error message: ValueError: operands could not be broadcast together with shapes (17,24) (17,150528)


Cause

The error originates from this line in non_sensitivity.py:

return (preds_differences ^ non_features).sum(-1)


non_features has shape (batch_size, n_features) - one boolean array per image, where each element indicates whether a pixel’s attribution value is below eps.

preds_differences has shape (batch_size, n_perturbations) - one boolean array per image, where each element represents whether the model output changed after each perturbation step.

Because these arrays have different dimensions (n_perturbations ≠ n_features when features_in_step != 1), the XOR (^) operation fails due to incompatible shapes.

Steps to Reproduce

import quantus
scores = quantus.NonSensitivity(
    eps=1e-5,
    features_in_step=6272,   #numbers that  != 1
)(model=model, x_batch=x_batch, y_batch=y_batch, device=device)


Expected Behavior

The metric should handle batch evaluation correctly when features_in_step > 1, either by aggregating or reshaping appropriately so that the boolean comparison is valid.


Environment

Quantus version: 0.6.0

Python version: 3.12.4 

Framework: PyTorch 

OS: Windows 11

I'm working on this issue now and will update soon.
Repository: understandable-machine-intelligence-lab/Quantus
Issue: Potential Bug in ROAD during Linear Imputation
Root cause: Algorithmic Error

### Description
When trying to calculate ROAD scores for precomputed explantions i get an error due to a shape mismatch, when the image is tried to be masked/imputed based on the relevance scores. Specifically, i pass x_batch (images) in shape (B, C, W, H) and a_batch (explanations) in shape (B, 1, W, H).

In class ROAD(..).evaluate_batch(..), a_batch is broadcasted to x_batch's shape:

```
if x_batch.shape != a_batch.shape:
        a_batch = np.broadcast_to(a_batch, x_batch.shape)
```

then a_batch is flattened to shape (B, C\*W\*H):
```

# Flatten the attributions.
batch_size = a_batch.shape[0]
a_batch = a_batch.reshape(batch_size, -1)
n_features = a_batch.shape[-1]
```

Finally, the indices for masking/noisy_linear_imputation are computed from the flattened a_batch **across the channel dimension** (besides height and width). In other words, the indices relate to color channel values, not pixels:

`ordered_indices = np.argsort(-a_batch, axis=1)`

Then, the code iterates over batch_images and passes single images of shape (CxWxH) to the perturb_function:

```
for x_element, top_k_index in zip(x_batch, top_k_indices):
    x_perturbed_element = self.perturb_func(  # type: ignore
        arr=x_element,
        indices=top_k_index,
    )
```

The perturb function (`noisy_linear_imputation()`), however, behaves like the indices passed relate to pixels and not color-channel-values:

```
arr_flat = arr.reshape((arr.shape[0], -1))

mask = np.ones(arr_flat.shape[1])
mask[indices] = 0
```

This throws an "index out of bounds error", since the indices passed, that relate to the image with color channels, are too large for indexing pixels independent of color channels.

This is a bug i guess? Also, what behavior is desired? I assume that we'd like to impute pixels rather than color channel values, but that's also not 100% clear to me from the Rong et al. (2022) paper.

### Steps to reproduce the behavior

```
 metric = quantus.ROAD(
      abs=False,
      normalise=True,
      disable_warnings=True,
      display_progressbar=False,
      return_aggregate=False,
  )

x_batch = np.random.rand(32, 3, 299, 299)
a_batch = np.random.rand(32, 1, 299, 299)
y_batch = np.random.rand(32)
model = models.resnet50(weights="IMAGENET1K_V2") # from torchvision

scores = metric(
    model=model,
    x_batch=x_batch,
    y_batch=y_batch,
    a_batch=a_batch
    device=device,
    softmax=False
)
```

### Minimum acceptance criteria
Fix the implementation so that pixels are correctly indexed or add an option for deciding whether to impute entire pixels or single color channel values.

- @mentions of the person that is apt to review these changes e.g., @annahedstroem

Best regards
Tjade