Evaluating Speech Synthesis on Mathematical Sentences
Authors: Alessandro Mazzei, Michele Monticone, Cristian Bernareggi (Università di Torino, 2019)
Topic: Chatbots and Dialogue Systems

Abstract
In this paper we present the main features of a rule-based architecture to transform a LATEX encoded mathematical expression into its equivalent mathematical sentence form, i.e. a natural language sentence expressing the semantics of the mathematical expression. Moreover, we describe the main results of a first human based evaluation of the system for Italian language focusing on speech synthesis engines.

Introduction
Computational linguistics can help people in many ways, especially in the field of assistive technologies. In the case of mathematical domain, blind people can access to a mathematical expression by listening its LATEX source. However, this process has several drawbacks. First of all, it assumes the knowledge of the LATEX. Second, listening LATEX is slow and error-prone, since LATEX is a typographical language, that is a language designed for specifying the details of typographical visualization rather than for efficiently communicate the semantics of a mathematical expression. For instance, the simple LATEX expression f (x) is a typographical description and so it represents both the function application of f to x, and the multiplication of the variable f for the variable x surrounded by parenthesis. There are many lines of research to enable people with sight impairments to access mathematical contents. It is possible to embed mathematical expressions in web pages not only as images but through MathML or MathJax (Cervone, 2012) and in PDF documents produced from LaTeX (Ahmetovic et al., 2018). Other research directions concern conversion into Braille (Soiffer, 2016) and speech reading (Raman, 1996; Waltraud Schweikhardt, 2006; Sorge et al., 2014). In this paper we follow another direction: we consider the possibility to produce a mathematical sentence, i.e. a natural language sentence expressing the semantics of a mathematical expression. Indeed, the idea to use mathematical sentences for improving the accessibility of mathematical expressions has been previously presented and experimented for Spanish in (Ferres and Fuentes Sepúlveda, 2011; Fuentes Sepúlveda and Ferres, 2012). However, in contrast to previous work on mathematical sentences, in this work we use a natural language generation (NLG) architecture rather than a template-based one for generating sentences. By using NLG architecture we obtain (i) more portability, and (ii) a major and simple customization of the output. We have two research goals in this paper. The first goal is to describe a system for transforming a mathematical expression natively encoded in LATEX in its equivalent mathematical sentence (cf. Figure 1). The processing flow follows a wellknown approach, called interlingua in the field of machine translation (Hutchins and Somer, 1992). Indeed, the process of generating a mathematical expression from its LATEX source is a two-step algorithm. In the first step the LATEX is analyzed and its semantics is represented in Content MathML (CMML henceforth), a W3C standard for expressing the syntax and the semantics of mathematical expressions. In the second step, the CMML representation is used as input of the S2S (Semantics to Speech) module, that is a NLG module generating the mathematical sentence. Note that the S2S module inserts in the sentence parenthesis and pauses too. The sentence will finally be transformed in audio format encoding by an external synthesis engine. The second goal of this paper is to give a first evaluation of the performance of two distinct synthesis engines in the domain of mathematical sentence. With a pilot experimentation conducted with four blind people, we will compare the perception of the mathematical sentences of a neural network based speech engine and of a formant based speech engine. In Section 2 we will describe the main features of the developed system, in Section 3 we will describe the experimentation and finally in Section 4 we end the paper with some conclusions and introducing future work.

Conclusion
In this paper we have presented a study on the generation of mathematical sentences, i.e. natural language sentences encoding mathematical expressions6 . In particular, we have described the main features of the system and the a first experimentation centred on the evaluation of two distinct speech engine. The results of the experimentation suggests a good performance of the formantbased synthesis engine with respect to the neuralnetwork base synthesis engine. However, more data is necessary to achieve statistical significance. In future work we intend to repeat the evaluation of the system for Italian with a larger number of users and to repeat the experiment by using English lanaguage too.