Report for bhadresh-savani/bert-base-uncased-emotion

#132
by giskard-bot - opened
Giskard org

Hi Team,

This is a report from Giskard Bot Scan 🐢.

We have identified 1 potential vulnerabilities in your model based on an automated scan.

This automated analysis evaluated the model on the dataset dair-ai/emotion (subset split, split validation).

👉Robustness issues (1)

When feature “text” is perturbed with the transformation “Add typos”, the model changes its prediction in 22.2% of the cases. We expected the predictions not to be affected by this transformation.

Level Data slice Metric Deviation
major 🔴 Fail rate = 0.222 222/1000 tested samples (22.2%) changed prediction after perturbation

Taxonomy

avid-effect:performance:P0201
🔍✨Examples
text Add typos(text) Original prediction Prediction after perturbation
656 i feel a little bit more nostalgic when those memories come to mind i feel a little bit more nosftalic when those memories comwe to mind love (p = 1.00) joy (p = 0.97)
734 i can talk to her about almost anything i want to and she just listens and she doesnt make me feel like a whiney brat and she helps me sort my thoughts and make decisions while keeping me where she feels im safe i can talk to her about almost anything i want to and she just lisrens and she doesnt make me feel liek a shiney brat and she helps me sort my thoughts and make decisions while keeping me where she fes im safe sadness (p = 1.00) joy (p = 1.00)
1403 i feel the need to preface this by saying that i am strongly in favor of keeping violent or otherwise inappropriate videogames out of the hands of minors and i believe that this is an issue that parents and the government need to work on together i feel the need to preface this by saying that i am ateongly in faor of keeping volent or otherwise inappropriate videogames outo f yhe hands of minor san di believe that this is an issue that parents and the government need to work on tovether anger (p = 0.99) sadness (p = 0.94)

Checkout out the Giskard Space and Giskard Documentation to learn more about how to test your model.

Disclaimer: it's important to note that automated scans may produce false positives or miss certain vulnerabilities. We encourage you to review the findings and assess the impact accordingly.

Sign up or log in to comment