Gli errori di un sistema di riconoscimento automatico del parlato. Analisi linguistica e primi risultati di un progetto di ricerca interdisciplinare
Authors: Maria Palmerini, Renata Savy (Università di Salerno. Cedat 85, 2014)
Topic: Multimodal

Abstract
The work presents the results of a work of classification and line-guistic analysis of errors of an automatic re-knowledge system (ASR), produced by Cedat‚Äô85. This is the first phase of a search to develop errors reduction strategies.

Introduction
The research project was born from a collaboration-in between the University of Salerno and Cedat 85, the leading company in Italy in the self-matic treatment of the speaker. The purpose of the project is a accurate assessment of errors produced by an automatic speaker transcription system (ASR), passed to the sequel of a more detailed linguistic analysis and subsequent metadata. The most used word error rate (WER) estimate of an ASR system is calculated automatically and is based on the analysis of a manual transcription (aligned to the signal) and the relative transcription obtained by the ASR system. This comparison identifies the wrong words (Substitutions), the missing words (Deletetions) and the wrongly inserted words (Insertions) as well as the total words (N) for an assessment: WER = (S+D+I)x100 N This estimate does not enter into the cause n≈Ω of the relevance of the error, making it rather a maximum reference for a gross assessment of an ASR system, without any indication on its real utility and adequacy, n≈Ω on the possibilities of intervention and improvement. Most of the last-generation ASR systems, which work on spontanely speaking, use technologies and algorithms that can best exploit the enormous power of calculation currently available, but differ significantly in the choice of parameters, intermediate steps, in the selection criteria of most likely candidates, in the tools for the processing of ad-training data. A quality criterion, as well as quantitative, of error assessment is necessary for an adaptation of the system to the reference environment, and for the indication of possible improvement interventions. Recent studies, both technological and lin-guistic and psycholinguistic, indicate correlations between errors and frequency in the vocabulary or in the use of words, speaking speed, ambiguity (homophony) and acoustic confundability (minimum and sub-minimum couples). However, there are missing systematic studies that take into account the correlation with morph-lesical classes, phonological and sillabic structures, syntagmatic sequences, the order of the constituents and above all, prosodic factors. In this contribution we present a first part of the results of a broader research on the weight of these factors, suffering on the criteria of the language classification of data and on the correlations achieved between the presence (and type) of error and phono-morphological and morphosynthetic categories.

Conclusion
In this preliminary phase of analysis, it seems to be possible to draw a first important conclusion: the quantitative assessment of the word error rate surpasses the failure of recognition of an ASR system. The metadata made and the subsequent quality assessment normalizes the data of the WER and redirect the mag-the best rate to unpredictable phenomena as little significant for the measurement of the efficiency of the system. In this context, indeed, the indecision and confusion of graphic rendering are almost equal to self-matic and manual transcription. Nevertheless, the weight of the wrong recognitions of these segments can be reduced by adopting a thinner annotation scheme, both in terms of more solid standards for transcribers and as a model for the ASR system. We end up supposing that some secondary interventions on the phone set, the enrichment of the vocabulary with the possible phonetic variants, and a better treatment of prosodic phenomena could improve the performance of the system to some extent.