# Methodology for Obtaining High-Quality Speech Corpora
## 4 Assessing the Quality of Speech Corpora
### Usability and UX  
Usability and UX are other, often neglected, criteria for assessing the quality of speech corpora. UX estimates the efficiency of using the investigated corpus and measures the time, completeness, and precision of the test users' performance, as well as user satisfaction.  
The users' opinions are also collected, and they can indicate how new users (not the corpus creators or users familiar with the interface) cope with the use of the corpus. UX can be assessed at the design step using mockups, and the interface can be improved even before implementation based on the usability test results. Eye tracking can be applied in these tests, and the results allow for assessing whether the most relevant information has been viewed based on fixations and eye movements.  
A well-designed interface facilitates completing the tasks to be performed by the user. The time needed to complete the task can be applied as one of the usability criteria; the shorter the time, the better. If the interface is intuitive, and not overwhelming with numerous available functionalities, then the user is encouraged to use the corpus.  
The usability of various systems (such as databases, including speech corpora) is often assessed based on their effectiveness, but usability assessment standards also exist. For instance, the ISO standard 9241-11:2018 Ergonomics of Human-System Interaction, Part 11, describes \"how usability can be interpreted in terms of user performance and satisfaction\" [92]. Usability is defined here as \"the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use\". Effectiveness is defined as the \"accuracy and completeness with which users achieve specified goals\", and efficiency is defined as the \"resources used in relation to the results achieved\", where resources usually represent time, human effort, costs, and materials. Satisfaction is defined as the \"extent to which the user's physical, cognitive and emotional responses that result from the use of a system, product or service meet the user's needs and expectations\". The SQuaRE series of standards, ISO/IEC 25000:2014 [93], aims to specify software quality requirements, as well as systems and software quality evaluation, including instructions for testing. The ISO/IEC 25062:2006 [94] standard provides a method for reporting usability test results. Therefore, these standards can be followed in usability testing, with the completeness and accuracy of the performed tasks being used as usability measures, and with time and cost being used as the estimations of resources needed.  
Usability tests should be performed before the final version is released to the public, and they can indicate corrections to be made in the corpus design. Typically, five to ten test participants (generally, between three and twenty), possibly potential corpus users, are required to complete usability tests for problem discovery; see [95] for details.  
In general, the corpus is usable if it is easily accessible and if all tools are easy to use for the target users. Since speech corpora are often used by linguists, who are non-technical users, they may encounter technical problems more often when using a corpus and its tools compared to IT specialists. Therefore, linguist specialists can be invited to participate in usability testing, as well as the persons responsible for speech data acquisition, if speech acquisition tools are included in the corpus. Corpus tools to be tested may include corpus querying tools, annotation tools, etc. The corpus website should also undergo usability tests. Exemplary outcomes of the corpus website's usability can be found in [96]; these tests indicated that \"the information could be better interlinked and organized in a more novice-user oriented way\" [96].