Report for HooshvareLab/bert-fa-base-uncased-sentiment-deepsentipers-binary
Hi Team,
This is a report from Giskard Bot Scan 🐢.
We have identified 2 potential vulnerabilities in your model based on an automated scan.
This automated analysis evaluated the model on the dataset sst2 (subset default, split validation).
You can find a full version of scan report here.
👉Performance issues (2)
For records in the dataset where text_length(text) >= 151.500 AND text_length(text) < 165.500, the Precision is 20.03% lower than the global Precision.
| Level | Data slice | Metric | Deviation |
|---|---|---|---|
| major 🔴 | text_length(text) >= 151.500 AND text_length(text) < 165.500 |
Precision = 0.407 | -20.03% than global |
Taxonomy
avid-effect:performance:P0204🔍✨Examples
| text | text_length(text) | label | Predicted label |
|
|---|---|---|---|---|
| 9 | in exactly 89 minutes , most of which passed as slowly as if i 'd been sitting naked on an igloo , formula 51 sank from quirky to jerky to utter turkey . | 154 | negative | positive (p = 1.00) |
| 11 | it takes a strange kind of laziness to waste the talents of robert forster , anne meara , eugene levy , and reginald veljohnson all in the same movie . | 152 | negative | positive (p = 1.00) |
| 26 | the action switches between past and present , but the material link is too tenuous to anchor the emotional connections that purport to span a 125-year divide . | 161 | negative | positive (p = 1.00) |
For records in the dataset where text contains "movie", the Precision is 17.22% lower than the global Precision.
| Level | Data slice | Metric | Deviation |
|---|---|---|---|
| major 🔴 | text contains "movie" |
Precision = 0.421 | -17.22% than global |
Taxonomy
avid-effect:performance:P0204🔍✨Examples
| text | label | Predicted label |
|
|---|---|---|---|
| 11 | it takes a strange kind of laziness to waste the talents of robert forster , anne meara , eugene levy , and reginald veljohnson all in the same movie . | negative | positive (p = 1.00) |
| 14 | even horror fans will most likely not find what they 're seeking with trouble every day ; the movie lacks both thrills and humor . | negative | positive (p = 1.00) |
| 18 | ... the movie is just a plain old monster . | negative | positive (p = 1.00) |
Checkout out the Giskard Space and Giskard Documentation to learn more about how to test your model.
Disclaimer: it's important to note that automated scans may produce false positives or miss certain vulnerabilities. We encourage you to review the findings and assess the impact accordingly.