# Fit for What Purpose? NER Certification of Automatic Captions in English and Spanish
## 3 Materials and Methodology: Assessment of AI-Media's Automatic Captions in English and Spanish
### Materials  
As mentioned above, this is the first time that the LiRICS certification of captioning quality has been used with automatic captions. However, for the sake of consistency and comparability, the audiovisual materials are the same used for the certification of human captioners. These materials are divided into three different levels of difficulty. Level 1 is the lowest level of assessment and the entry level for all candidates. Once a candidate has passed this assessment, they are able to move on to Levels 2 and 3. A number of parameters differentiate the difficulty between levels, content being one of them. Level 1 test materials are of a more general nature and not as specialized as those of Levels 2 and 3. Other aspects are also taken into account, such as sound quality, the number and delivery of the speakers (in terms of pronunciation, accent, speech rate and whether they speak spontaneously or read a written text) or whether there are any visual aids that support the verbal information. These aspects are detailed in assessment-setting proformas produced by the LiRICS team to check that the selected video clips are set at the correct level. The proformas include information about genre, provenance, title, publication date, synopsis of the video with the time frame that has been selected to be captioned and finally, a list of captioning challenges, such as changes of speaker, proper nouns, specialized terminology and the need to restructure sentences. In the study presented here, a Level 3 Extra clip was added in both English and Spanish in order to include an example of material representative of content currently streamed on social networks and to test the limits of automatic captioning. The material used in the test may be described as follows:  
Level 1--English:  
(1A_EN) Primary level science lesson exploring the question 'How do we hear sounds?'. This lesson comes at the start of the topic and the children are given instructions to build a model of an ear drum. (1B_EN) A conference presentation by scientist [PERSON] exploring the concept of simplicity by discussing everyday objects and their practical and scientific uses in our lives. (1C_EN) A Q&A interview with [PERSON] discussing her UN speech about gender equality and societal issues surrounding feminism. Level 1--Spanish:  
(1A_ES) A speech by former Spanish Prime Minister [PERSON]. (1B_ES) A masterclass by Spanish journalist [PERSON] about public speaking. It features one main speaker and three questions from the audience. (1C_ES) An interview between politician [PERSON] and writer [PERSON]. Level 2--English:  
(2A_EN) A young secondary school class learning about thermal energy transfer, more specifically about radiation, convection and conduction. The teacher recaps a previous class and allows the students to interact frequently. (2B_EN) A talk by [PERSON] about how his life experiences led him to learn Slovak, Hungarian and English and about his journey to become a polyglot. (2C_EN) An interview with [PERSON] as part of the United State of Women Summit in 2018 about how she found her voice as a child. Level 2--Spanish:  
(2A_ES) A masterclass by author [PERSON] about the value of attitude. (2B_ES) An interview with Professor [PERSON] about pensions. (2C_ES) A podcast about mindfulness in the classroom. Level 3--English:  
(3A_EN) An English literature secondary school class analysing a specific extract of a literary text. The class works together to discuss the text with a special focus on why certain terminology has been used by the author. (3B_EN) A talk by Dr. [PERSON] on the importance of eating well and how our bodies process the food we eat. The talk is supported by a PowerPoint presentation and other visual aids. (3C_EN) A conversation between [PERSON] and [PERSON] about Harry Potter, sharing previous (off-screen) subject matter in an on-camera conversation. The image of them talking is interspersed with occasional images of the people and places they refer to.