Spaces:
Running
Report for cardiffnlp/twitter-roberta-base-irony
Hey Team!🤗✨
We’re thrilled to share some amazing evaluation results that’ll make your day!🎉📊
We have identified 2 potential vulnerabilities in your model based on an automated scan.
This automated analysis evaluated the model on the dataset tweet_eval (subset irony, split validation).
👉Performance issues (2)
| Vulnerability | Level | Data slice | Metric | Transformation | Deviation |
|---|---|---|---|---|---|
| Performance | major 🔴 | text contains "love" |
Accuracy = 0.333 | — | -36.21% than global |
🔍✨Examples
For records in the dataset where `text` contains "love", the Accuracy is 36.21% lower than the global Accuracy.| text | label | Predicted label |
|
|---|---|---|---|
| 120 | Dont we all just love those people who message you out of nowhere and act like you guys are close cus they want something from you? | non_irony | irony (p = 0.86) |
| 133 | I love waking up at 4am everyday and then running 5 miles.. favorite part of the Army :) | non_irony | irony (p = 0.99) |
| 155 | I love how today is going | non_irony | irony (p = 0.98) |
| Vulnerability | Level | Data slice | Metric | Transformation | Deviation |
|---|---|---|---|---|---|
| Performance | medium 🟡 | text contains "day" |
Accuracy = 0.490 | — | -6.26% than global |
🔍✨Examples
For records in the dataset where `text` contains "day", the Accuracy is 6.26% lower than the global Accuracy.| text | label | Predicted label |
|
|---|---|---|---|
| 41 | Oh, thank GOD - our entire office email system is down... the day of a big event. Santa, you know JUST what to get me for xmas. | non_irony | irony (p = 0.93) |
| 62 | First day skiing #killedit #iwasshit #tignes #seasonlife | non_irony | irony (p = 0.72) |
| 121 | LOL..It was an RSS agenda modi bribed UN officals to make yoga day international.. #secularindia #AntiConversionLaw #hypocrites | non_irony | irony (p = 0.98) |
Disclaimer: it's important to note that automated scans may produce false positives or miss certain vulnerabilities. We encourage you to review the findings and assess the impact accordingly.
💡 What's Next?
- Checkout the Giskard Space and improve your model.
- The Giskard community is always buzzing with ideas. 🐢🤔 What do you want to see next? Your feedback is our favorite fuel, so drop your thoughts in the community forum! 🗣️💬 Together, we're building something extraordinary.
🙌 Big Thanks!
We're grateful to have you on this adventure with us. 🚀🌟 Here's to more breakthroughs, laughter, and code magic! 🥂✨ Keep hugging that code and spreading the love! 💻 #Giskard #Huggingface #AISafety 🌈👏 Your enthusiasm, feedback, and contributions are what seek. 🌟 Keep being awesome!