Biases and limitations introduced by machine translation in the PORTULAN ExtraGLUE benchmark include:
Translation Errors: Inaccuracies in translation, especially for gendered nouns, pronoun resolution, and idiomatic expressions, can lead to incorrect labels and impact model performance.
Cultural Nuances: Machine translation may not capture cultural nuances specific to Portuguese, affecting the authenticity of the translated datasets.
Named Entities: Inconsistent translation of proper names and entities can lead to confusion and errors in downstream tasks.
To mitigate these issues in future iterations:
Human Validation: Incorporating human validation and correction of translations to ensure linguistic accuracy and cultural relevance.
Post-Editing: Implementing post-editing processes to refine machine-translated texts and correct errors introduced during translation.
Task-specific Evaluation: Conducting thorough evaluations of translated datasets to identify and address biases and limitations, especially in tasks sensitive to translation quality.
Continuous Improvement: Iteratively refining the translation process based on feedback and evaluation results to enhance the quality and reliability of the benchmark.