Spaces:
Running
Report for cardiffnlp/twitter-xlm-roberta-base-sentiment-multilingual
Hey Team!🤗✨
We’re thrilled to share some amazing evaluation results that’ll make your day!🎉📊
We have identified 3 potential vulnerabilities in your model based on an automated scan.
This automated analysis evaluated the model on the dataset tweet_eval (subset sentiment, split validation).
👉Performance issues (3)
| Vulnerability | Level | Data slice | Metric | Transformation | Deviation |
|---|---|---|---|---|---|
| Performance | major 🔴 | text contains "day" |
Precision = 0.095 | — | -39.17% than global |
🔍✨Examples
For records in the dataset where `text` contains "day", the Precision is 39.17% lower than the global Precision.| text | label | Predicted label |
|
|---|---|---|---|
| 1 | "National hot dog day, national tequila day, then national dance day... Sounds like a Friday night." | positive | negative (p = 0.86) |
| 54 | "If I'm off from work again tomorrow, I'm spending the entire day catching up on The Walking Dead." | neutral | negative (p = 0.62) |
| 58 | "Tomorrow is National Ice Cream Day. Just in case you can't make it to the dining hall to satisfy your craving, here are some stores......" | positive | negative (p = 0.83) |
| Vulnerability | Level | Data slice | Metric | Transformation | Deviation |
|---|---|---|---|---|---|
| Performance | major 🔴 | text contains "friday" |
Precision = 0.102 | — | -34.81% than global |
🔍✨Examples
For records in the dataset where `text` contains "friday", the Precision is 34.81% lower than the global Precision.| text | label | Predicted label |
|
|---|---|---|---|
| 1 | "National hot dog day, national tequila day, then national dance day... Sounds like a Friday night." | positive | negative (p = 0.86) |
| 27 | every time I hear alright by Kendrick I think it's j Cole's Black Friday | neutral | negative (p = 0.72) |
| 32 | Friday! How can you argue with 5 beautiful women who sound this good playing Iron Maiden! | positive | negative (p = 0.98) |
| Vulnerability | Level | Data slice | Metric | Transformation | Deviation |
|---|---|---|---|---|---|
| Performance | medium 🟡 | text contains "night" |
Precision = 0.145 | — | -6.86% than global |
🔍✨Examples
For records in the dataset where `text` contains "night", the Precision is 6.86% lower than the global Precision.| text | label | Predicted label |
|
|---|---|---|---|
| 1 | "National hot dog day, national tequila day, then national dance day... Sounds like a Friday night." | positive | negative (p = 0.86) |
| 9 | Irving Plaza NYC Blackout Saturday night. Got limited spots left on the guest list. Tweet me why you think you deserve them | neutral | negative (p = 0.67) |
| 42 | A FB friend of mine just posted that seeing Magic Mike XXL was the best night of her life. If only she knew what my typical sat night is. | positive | negative (p = 0.99) |
Disclaimer: it's important to note that automated scans may produce false positives or miss certain vulnerabilities. We encourage you to review the findings and assess the impact accordingly.
💡 What's Next?
- Checkout the Giskard Space and improve your model.
- The Giskard community is always buzzing with ideas. 🐢🤔 What do you want to see next? Your feedback is our favorite fuel, so drop your thoughts in the community forum! 🗣️💬 Together, we're building something extraordinary.
🙌 Big Thanks!
We're grateful to have you on this adventure with us. 🚀🌟 Here's to more breakthroughs, laughter, and code magic! 🥂✨ Keep hugging that code and spreading the love! 💻 #Giskard #Huggingface #AISafety 🌈👏 Your enthusiasm, feedback, and contributions are what seek. 🌟 Keep being awesome!