Bias auditing for all: moving into the territory
Channel: csvconf
Date: 2023-05-31

Presented by: Laura Alonso Alemany

Automated decision-making systems (automatic recommenders, credit risk classifiers, image recognition systems, translators, among others) involve increasingly important aspects of our lives, even affecting human rights. For this reason, auditing them becomes a necessity. However, systems involving artificial intelligence components are particularly difficult to audit, especially in cases where machine learning is applied, as they are often opaque and unintelligible to the person inspecting them. Many methods for auditing these systems have been proposed in academic circles, but most of the proposals involve technical complexities that exclude people with relevant experience for auditing, such as experience in discrimination or in the contexts of application of the technologies. In this talk we want to present a tool and a methodology to facilitate the involvement of experts, but without specific technical skills, in the audit of language technology biases. We will describe how we conceptualize inspection methods for these technologies by focusing on people with relevant experience, and making the technical complexities transparent. We will relate two practical experiences with people with expertise in very different types of biases and discriminations, which showed good feasibility of the tool. Finally, we will detail our future plans: to consolidate the tool as a software library, to compile, make available and make visible linguistic resources that allow systematizing the analysis of local biases, in collaboration with international teams with the same objectives, and finally to promote the adoption of the methodology associated with this tool in organizations and companies in our environment. A demo of the tool can be found at https://huggingface.co/vialibre.

"