Report for bhadresh-savani/distilbert-base-uncased-emotion

#145
by ZeroCommand - opened
Giskard org

Hi Team,

This is a report from Giskard Bot Scan 🐢.

We have identified 2 potential vulnerabilities in your model based on an automated scan.

This automated analysis evaluated the model on the dataset dair-ai/emotion (subset split, split validation).

Giskard org
👉Performance issues (1)

For records in the dataset where text contains "know", the Precision is 5.25% lower than the global Precision.

Level Data slice Metric Deviation
medium 🟡 text contains "know" Precision = 0.885 -5.25% than global

Taxonomy

avid-effect:performance:P0204

Examples are too long to be displayed in this area.

Giskard org
👉Robustness issues (1)

When feature “text” is perturbed with the transformation “Add typos”, the model changes its prediction in 22.2% of the cases. We expected the predictions not to be affected by this transformation.

Level Metric Transformation Deviation
major 🔴 Fail rate = 0.222 Add typos 222/1000 tested samples (22.2%) changed prediction after perturbation

Taxonomy

avid-effect:performance:P0201

Examples are too long to be displayed in this area.

Sign up or log in to comment