Spaces:
Running
Running
adding limitations
Browse files- introduction.md +6 -1
introduction.md
CHANGED
@@ -243,7 +243,10 @@ early 1900 and it is part of the largest movie studios in Europe (Cinecittà).
|
|
243 |
Currently, the model is not without limits. To mention one, its counting capabilities seem very cool, but from our experiments the model
|
244 |
finds difficult to count after three; this is a general limitation.
|
245 |
There are even more serious limitations: we found some emergence of biases and stereotypes that got in our model from different factors: searching for "una troia" ("a bitch") on the
|
246 |
-
CC dataset shows the picture of a woman.
|
|
|
|
|
|
|
247 |
suggest we need to work even harder on this problem that affects our **society**.
|
248 |
|
249 |
# References
|
@@ -252,6 +255,8 @@ Abid, A., Farooqi, M., & Zou, J. (2021). [Persistent anti-muslim bias in large l
|
|
252 |
|
253 |
Gwet, K. L. (2008). [Computing inter‐rater reliability and its variance in the presence of high agreement.](https://bpspsychub.onlinelibrary.wiley.com/doi/full/10.1348/000711006X126600) British Journal of Mathematical and Statistical Psychology, 61(1), 29-48.
|
254 |
|
|
|
|
|
255 |
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. (2021). [Learning Transferable Visual Models From Natural Language Supervision.](https://arxiv.org/abs/2103.00020) ICML.
|
256 |
|
257 |
Reimers, N., & Gurevych, I. (2020, November). [Making Monolingual Sentence Embeddings Multilingual Using Knowledge Distillation.](https://aclanthology.org/2020.emnlp-main.365/) In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 4512-4525).
|
|
|
243 |
Currently, the model is not without limits. To mention one, its counting capabilities seem very cool, but from our experiments the model
|
244 |
finds difficult to count after three; this is a general limitation.
|
245 |
There are even more serious limitations: we found some emergence of biases and stereotypes that got in our model from different factors: searching for "una troia" ("a bitch") on the
|
246 |
+
CC dataset shows the picture of a woman. The model's capability even increase this issue, as searching for "due troie" ("two bitches")
|
247 |
+
gives again, as a results, the picture of two women. BERT models are not free from bias. Indeed, different BERT models - Italians included - are prone to create stereotyped sentences that are hurtful ([Nozza et al., 2021](https://www.aclweb.org/anthology/2021.naacl-main.191.pdf))
|
248 |
+
|
249 |
+
This issue is common to many machine learning algorithms (check [Abit et al., 2021](https://arxiv.org/abs/2101.05783) for bias in GPT-3 as an example) and
|
250 |
suggest we need to work even harder on this problem that affects our **society**.
|
251 |
|
252 |
# References
|
|
|
255 |
|
256 |
Gwet, K. L. (2008). [Computing inter‐rater reliability and its variance in the presence of high agreement.](https://bpspsychub.onlinelibrary.wiley.com/doi/full/10.1348/000711006X126600) British Journal of Mathematical and Statistical Psychology, 61(1), 29-48.
|
257 |
|
258 |
+
Nozza, D., Bianchi, F., & Hovy, D. (2021, June). [HONEST: Measuring hurtful sentence completion in language models.](https://www.aclweb.org/anthology/2021.naacl-main.191.pdf) In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 2398-2406).
|
259 |
+
|
260 |
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. (2021). [Learning Transferable Visual Models From Natural Language Supervision.](https://arxiv.org/abs/2103.00020) ICML.
|
261 |
|
262 |
Reimers, N., & Gurevych, I. (2020, November). [Making Monolingual Sentence Embeddings Multilingual Using Knowledge Distillation.](https://aclanthology.org/2020.emnlp-main.365/) In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 4512-4525).
|