Spaces:
Running
Running
Report for yiyanghkust/finbert-tone
#177
by
giskard-bot
- opened
Hi Team,
This is a report from Giskard Bot Scan 🐢.
We have identified 7 potential vulnerabilities in your model based on an automated scan.
This automated analysis evaluated the model on the dataset financial_phrasebank (subset sentences_allagree, split train).
👉Performance issues (7)
For records in the dataset where text contains "said", the Precision is 19.67% lower than the global Precision.
| Level | Data slice | Metric | Deviation |
|---|---|---|---|
| major 🔴 | text contains "said" |
Precision = 0.737 | -19.67% than global |
Taxonomy
avid-effect:performance:P0204🔍✨Examples
| text | label | Predicted label |
|
|---|---|---|---|
| 115 | Finland-based Elcoteq SE , a privately held provider of electronics manufacturing services to communications companies , said Thursday it signed a long-term manufacturing supply deal with communications equipment company Andrew Corp. . | Positive | Neutral (p = 1.00) |
| 129 | However , sales returned to growth in April-June 2010 , CEO Pekka Eloholma said . | Positive | Neutral (p = 0.46) |
| 312 | Finnish telecoms software developer Tecnomen Oyj said on December 5 , 2006 it won a $ 3.3 mln ( 2.5 mln euro ) order to supply an expansion of the charging and messaging systems for the mobile and fixed networks of Brasil Telecom . | Positive | Neutral (p = 0.98) |
For records in the dataset where text contains "period", the Precision is 10.49% lower than the global Precision.
| Level | Data slice | Metric | Deviation |
|---|---|---|---|
| major 🔴 | text contains "period" |
Precision = 0.821 | -10.49% than global |
Taxonomy
avid-effect:performance:P0204🔍✨Examples
| text | label | Predicted label |
|
|---|---|---|---|
| 47 | Finnish financial group Aktia reports operating profit of EUR 44.4 mn in January-September 2009 , up from EUR 37.3 mn in the corresponding period in 2008 . | Positive | Neutral (p = 0.74) |
| 329 | 26 January 2011 - Finnish metal products company Componenta Oyj ( HEL : CTH1V ) said yesterday its net loss narrowed to EUR500 ,000 in the last quarter of 2010 from EUR5 .3 m for the same period a year earlier . | Positive | Negative (p = 0.59) |
| 330 | 8 May 2009 - Finnish liquid handling products and diagnostic test systems maker Biohit Oyj ( HEL : BIOBV ) said today ( 8 May 2009 ) its net loss narrowed to EUR0 .1 m ( USD0 .14 m ) for the first quarter of 2009 from EUR0 .4 m for the same period of 2008 . | Positive | Negative (p = 0.63) |
For records in the dataset where text contains "finland", the Balanced Accuracy is 8.58% lower than the global Balanced Accuracy.
| Level | Data slice | Metric | Deviation |
|---|---|---|---|
| medium 🟡 | text contains "finland" |
Balanced Accuracy = 0.810 | -8.58% than global |
Taxonomy
avid-effect:performance:P0204🔍✨Examples
| text | label | Predicted label |
|
|---|---|---|---|
| 115 | Finland-based Elcoteq SE , a privately held provider of electronics manufacturing services to communications companies , said Thursday it signed a long-term manufacturing supply deal with communications equipment company Andrew Corp. . | Positive | Neutral (p = 1.00) |
| 324 | In addition , Cramo and Peab have signed exclusive five-year rental agreements in Finland and have extended their existing rental agreements in the Swedish market for another five years . | Positive | Neutral (p = 1.00) |
| 425 | Citigroup , Inc ( NYSE : C ) has announced that its Global Transaction Services ( GTS ) business has been awarded a new mandate by Finland-based Pohjola Bank Group . | Positive | Neutral (p = 0.75) |
For records in the dataset where text contains "year", the Precision is 6.02% lower than the global Precision.
| Level | Data slice | Metric | Deviation |
|---|---|---|---|
| medium 🟡 | text contains "year" |
Precision = 0.862 | -6.02% than global |
Taxonomy
avid-effect:performance:P0204🔍✨Examples
| text | label | Predicted label |
|
|---|---|---|---|
| 230 | Last year the company raised its turnover to approximately 7 million litas EUR 2 mln , from 6.1 million litas in 2004 . | Positive | Neutral (p = 0.63) |
| 321 | LONDON MarketWatch -- Nokia nok said it 's won a five-year services contract to run Hutchison Essar 's network operations in nine locations in India . | Positive | Neutral (p = 1.00) |
| 324 | In addition , Cramo and Peab have signed exclusive five-year rental agreements in Finland and have extended their existing rental agreements in the Swedish market for another five years . | Positive | Neutral (p = 1.00) |
For records in the dataset where text contains "eur", the Precision is 5.68% lower than the global Precision.
| Level | Data slice | Metric | Deviation |
|---|---|---|---|
| medium 🟡 | text contains "eur" |
Precision = 0.865 | -5.68% than global |
Taxonomy
avid-effect:performance:P0204🔍✨Examples
| text | label | Predicted label |
|
|---|---|---|---|
| 21 | The fair value of the property portfolio doubled as a result of the Kapiteeli acquisition and totalled EUR 2,686.2 1,259.7 million . | Positive | Neutral (p = 1.00) |
| 47 | Finnish financial group Aktia reports operating profit of EUR 44.4 mn in January-September 2009 , up from EUR 37.3 mn in the corresponding period in 2008 . | Positive | Neutral (p = 0.74) |
| 200 | The fair value of the investment properties totaled EUR 2,534.9 mn , up from EUR 2,455.1 mn in 2006 . | Positive | Neutral (p = 1.00) |
For records in the dataset where text contains "quarter", the Precision is 5.57% lower than the global Precision.
| Level | Data slice | Metric | Deviation |
|---|---|---|---|
| medium 🟡 | text contains "quarter" |
Precision = 0.866 | -5.57% than global |
Taxonomy
avid-effect:performance:P0204🔍✨Examples
| text | label | Predicted label |
|
|---|---|---|---|
| 328 | In the second quarter of 2010 , the group 's pretax loss narrowed to EUR 400,000 from EUR 600,000 . | Positive | Negative (p = 0.98) |
| 329 | 26 January 2011 - Finnish metal products company Componenta Oyj ( HEL : CTH1V ) said yesterday its net loss narrowed to EUR500 ,000 in the last quarter of 2010 from EUR5 .3 m for the same period a year earlier . | Positive | Negative (p = 0.59) |
| 330 | 8 May 2009 - Finnish liquid handling products and diagnostic test systems maker Biohit Oyj ( HEL : BIOBV ) said today ( 8 May 2009 ) its net loss narrowed to EUR0 .1 m ( USD0 .14 m ) for the first quarter of 2009 from EUR0 .4 m for the same period of 2008 . | Positive | Negative (p = 0.63) |
For records in the dataset where text_length(text) >= 122.500 AND text_length(text) < 133.500, the Balanced Accuracy is 5.28% lower than the global Balanced Accuracy.
| Level | Data slice | Metric | Deviation |
|---|---|---|---|
| medium 🟡 | text_length(text) >= 122.500 AND text_length(text) < 133.500 |
Balanced Accuracy = 0.839 | -5.28% than global |
Taxonomy
avid-effect:performance:P0204🔍✨Examples
| text | text_length(text) | label | Predicted label |
|
|---|---|---|---|---|
| 10 | Its board of directors will propose a dividend of EUR0 .12 per share for 2010 , up from the EUR0 .08 per share paid in 2009 . | 125 | Positive | Neutral (p = 1.00) |
| 21 | The fair value of the property portfolio doubled as a result of the Kapiteeli acquisition and totalled EUR 2,686.2 1,259.7 million . | 132 | Positive | Neutral (p = 1.00) |
| 319 | Bilfinger investors cheered the agreement , pushing shares up 7 % , or & euro ; 3.30 , to & euro ; 50.29 , in afternoon trade . | 127 | Positive | Neutral (p = 1.00) |
Checkout out the Giskard Space and Giskard Documentation to learn more about how to test your model.
Disclaimer: it's important to note that automated scans may produce false positives or miss certain vulnerabilities. We encourage you to review the findings and assess the impact accordingly.