# Fit for What Purpose? NER Certification of Automatic Captions in English and Spanish
## 1 Introduction  
Audiovisual translation and media accessibility are concerned with making audiovisual content available across cultures and languages for a wide range of users with different needs. These fields have always relied heavily on technology. However, the so-called artificial intelligence (AI) revolution [1] has brought about a significant paradigm shift in this area. Dubbing and captioning workflows are now increasingly cloud-based and, most importantly, partially (and, in some cases, even fully) automated [2]. A case in point is live captioning, defined by [3] as \"the real-time transcription of spoken words, sound effects, relevant musical cues and other relevant audio information\" to provide access to live programmes or events for users with or without hearing loss. Live captions can be produced through different methods (standard QWERTY keyboards, dual keyboard, Velotype, stenography, resepaking, etc. ), all of which rely on technology [4]. Until recently, the preferred live subtitling method for most broadcasters and access service providers was stenography, which uses a system of machine shorthand in which letters phonetically represent syllables, words, phrases and punctuation marks [5], and resepaking, which relies on a live captioner who listens to the original sound of a programme or event and resepaks it to a speech-recognition software that turns the recognized utterances intosubtitles on the screen [6]. Although still commonly used, these methods are now gradually being replaced by automatic captioning [7]. Powered by the latest AI-based speech recognition technology, automatic captioning turns the original speech of a programme or live event into captions without the need for a respeaker. It is thus a clear example of full automation in audiovisual translation and accessibility. Needless to say, the increasing use of automatic (live) captioning has triggered concerns amongst professional translators and access professionals [8], who complain that this turn towards automation may have a negative impact on the quality of the access received by the users. Assessing the quality of these new automated captions thus becomes of the essence. In 2017, the Galician Observatory for Media Accessibility (GALMA), a research centre at the Universidade de Vigo concerned with the analysis of media accessibility quality, set up LiRICS, the Live Respeaking International Certification Standard. The original aim of LiRICS was to help set and maintain high international standards in the live captioning profession in order to create a pool of captioners who could provide high-quality access through captions for live TV programmes and live events [9]. LiRICS is an online certification process designed to assess the candidates' ability to caption across two broad live contexts: TV, on the one hand, and education and live events, on the other. Since it was set up, the LiRICS certification was undertaken by professionals from different companies and countries, including 26 captioners from AI-Media and 20 from the Global Alliance for Speech-to-Text Captioning. It has so far never been applied to automatic captions. In order to ascertain whether automatic captions have reached an accuracy level that can match that of human captions, this article puts the latest AI-based live captions produced by the leading international access service company AI-Media through the LiRICS certification, which has so far only been applied to human captions. This is crucial to understand if the current move towards automatic captions recently initiated by the access industry is justified by empirical data regarding live captioning quality. The research questions to be answered in this paper are: Do Lexi's captions pass the LiRICS certification? How do Lexi's captions perform compared to human captions? What are the specific limitations of Lexi when applied to challenging content? Section 2 reviews the most relevant literature in this area. Section 3 describes the materials and method used in this analysis.