Patent ID: 12210816

DETAILED DESCRIPTION

Since the embodiments described in the following are in no way limiting, it is possible, in particular, to envisage variants of the present disclosure that comprise only a selection of features described in the following, in a manner isolated from the other features described, if this selection of features is sufficient for providing a technical advantage or for distinguishing the present disclosure from the prior art. This selection comprises at least one feature, preferably functional and without structural details, or having some of the structural details if this part alone is sufficient for providing a technical advantage or for distinguishing the present disclosure from the prior art.

In the figures, an element that appears in several figures is provided with the same reference sign.

An embodiment of a system100, as well as an embodiment of a method M for generating a response A to a voice signal Sv, are described at the same time in the following, with reference to the drawings.

FIG.1shows an embodiment of a system100for generating the response A to the voice signal Sv.

The system100comprises an automatic speech recognition engine200configured to generate, during a step E1, a voice-signal transcription TSv from the voice signal Sv.

It is possible, for example, to use the Kaldi toolkit or the CMU Sphinx collection of software and tools in order to construct, or interact with, the speech recognition engine200. It is also possible, in the case of the embodiment of an “end-to-end” speech recognition engine, to use the TensorFlow or PyTorch deep learning software frameworks, or to rely on a toolkit such as ESPNet, which proposes training methods for various models of this kind.

The system also comprises a semantic engine400, which is configured to generate, during a step E2, the response A in response to an injection of a question Q.

The semantic engine400is equipped with at least one ontology Oms, labelled with reference numeral410, comprising a vocabulary space Vms, labelled with reference numeral412.

It is possible, for example, to create a system made up of: an ontology made up using Protégé, comprising knowledge to restore; a natural language processing stack based on the Stanford NLP tools, and a machine learning system such as BERT, trained to associate a sequence of input words with possible knowledge present in the ontology.

According to the prior art, the voice-signal transcription TSv is used as a question Q to be injected into a semantic engine400and to obtain the response A. This step is modelled by the barred arrow AA.

Differently from the prior art, the system100further comprises a device300for modifying, during a step M, the voice-signal transcription TSv, functionally interposed between the automatic speech recognition engine200and the semantic engine400.

More precisely, the device300receives, at the input, the voice-signal transcription TSv, generated by the automatic speech recognition engine200, from a voice signal Sv. Prior to the generation of the response A by the semantic engine400from the injection of the question Q, the semantic engine being equipped with the ontology410comprising the vocabulary space Vms, the modification device is configured for:determining the set I of all of the words of the voice-signal transcription TSv not belonging to the vocabulary space412, which can be written:
I={x∈TSv,x∉Vms}for each word of the determined set:performing word-embedding of the word in the vocabulary space,determining semantic distances between the word-embedding of the word and each of the words of the vocabulary space embedded in the vocabulary space,selecting at least one word of the vocabulary space, the selection being made from the semantic distances determined,modifying the transcription TSv by replacing the word of the transcription with the at least one selected word in order to form the modified transcription MTSv.

Embedding a word in a vocabulary space is a well-known technique.

“Word-embedding,” or also lexical embedding, is a training method of a representation of words used, in particular, in automatic processing of languages.

This technique makes it possible to represent each word of a dictionary by a corresponding vector of real numbers. This facilitates the semantic analysis of the words. Two approaches for encoding the context (the vicinity) of a word are distinguished. The frequency-based approaches count the words that co-occur with a given word, and then distill this information so as to create dense, small vectors. An example is latent semantic analysis. Word-embedding constitutes a second approach by which it is attempted to predict a given mode, using its context, and vice versa. This is the process that creates the word-embedding. A popular implementation of the calculation of word-embedding is provided by Tomas Mikolov in his program word2vec.

The general idea is to project a set of words of a vocabulary of size V in a vector space, where the vectors of the words are of a relatively small size N. Moreover, the vector representation of each word of V must be determined such that the words having similar representations appear in similar contexts. Researcher Tomas Mikolov's approach is based on artificial neural networks for constructing the vectors. These models are trained on high-volume corpuses.

It is possible, for example, to use the group of models word2vec or GloVE, which can be implemented in a deep learning computer library such as TensorFlow or PyTorch, in order to implement all the word-embedding of the present disclosure. Within the scope of implementing word-embedding by composition, it is possible to use the library Java Neural Network (JNN), implemented by the authors, or to use the preceding deep learning computer libraries for personalized implementation. In particular, the device for modifying the voice-signal transcription TSv generated by the automatic speech recognition engine200from the voice signal Sv can be configured to determine, during an initial step, the word-embedding, in the vocabulary space Vms, of each of the words of the vocabulary space Vms, for example, by calling up the command word2vec as many times as necessary.

EXAMPLE

It is supported that a user produces the voice signal Sv “family of canines,” which is correctly transcribed by the automatic speech recognition engine200into a voice-signal transcription TSv.

Usually, the signal TSv is addressed to the semantic engine400as a question Q.

When the semantic engine400does not include the word “canine” in its vocabulary space Vms associated with the ontology used, in the present case an animal ontology, it is not capable of processing the words.

Even so, the vocabulary Vms associated with the ontology can include the words “Canidae,” associated with other words, which are the genera “Atelocynus cabrera,” “Canis linnaeus,” “CerdocyonHamilton-Smith,” “Chrysocyon,” etc.

Thus, when a user produces the question Q in the form of a voice-signal transcription TSv “what are the genera of the family of the canines,” the semantic engine400cannot provide an adequate response, although the responses are known.

The method according to the present disclosure comprises the following steps, during step M:during an initial step, for each of the words of the vocabulary space Vms, the result of the word-embedding of the word in the vocabulary space, i.e., each of the words “Canidae,” but also “Atelocynus cabrera,” “Canis linnaeus,” “CerdocyonHamilton-Smith,” “Chrysocyon,” etc., is calculated.the set I of all of the words of the voice-signal transcription not belonging to the vocabulary space is determined: I=“canine,”the word “canine” is word-embedded, into the vocabulary space, by composition,the semantic distances and/or other measures of similarity between the word-embedding of the word and each of the words of the vocabulary space is determined,a word of the vocabulary space that is closest, in this case the word “canidae,” is determined, andthe transcription is modified by replacing the word of the transcription with the at least one word in order to obtain a modified signal transcription MTSv: “what are the genera of the family of the Canidae.”

The transcription of the modified signal is then injected, as a question Q, into the semantic engine400, and the adequate response A is obtained.

Example

It is supposed that a user produces the voice signal Sv “Where can I bunk cheaply?”, which is correctly transcribed by the automatic speech recognition engine200into a voice-signal transcription TSv.

Usually, the signal TSv is addressed to the semantic engine400as a question Q.

When the semantic engine400does not include the word “bunk” in its vocabulary space Vms associated with the ontology used, in the present case a hotel and catering ontology, it is not capable of processing the request.

Even so, the vocabulary Vms associated with the ontology can include the words “sleep,” “have a meal,” “stay,” associated with the class “hotel and catering service.”

Thus, when a user produces the question Q associated with the voice signal Sv, the semantic engine400cannot provide an adequate response, although the responses are known.

The method according to the present disclosure comprises the following steps, during step M:during an initial step, for each of the known words of the vocabulary space Vms, the result of the word-embedding of the word in the vocabulary space, i.e., each of the words “sleep,” “have a meal,” “stay,” etc., is calculated.the set I of all of the words of the voice-signal transcription not belonging to the vocabulary space is determined: I=“bunk,”word-embedding by composition is performed for the word “bunk” in a vocabulary space referred to as Vunk,unknown words are projected in a common space (Vms U Vunk), and the N-best candidates are selected, considering the semantic distance and/or measures of similarity,considering criteria (time, cost, performance) defined by the user, the candidate word is selected. In this example, the word “sleep” will be retained, andthe transcription is modified by replacing the word of the transcription with the at least one word in order to obtain a modified signal transcription MTSv: “Where can I sleep cheaply?”

The transcription of the modified signal is then injected, as a question Q, into the semantic engine400, and the adequate response A is obtained.

Of course, the present disclosure is not limited to the embodiments described above, and a number of developments can be made to the embodiments, without departing from the scope of the invention as defined by the claims. Moreover, the various features, types, variants, and embodiments of the present disclosure may be associated with one another, in accordance with various combinations, insofar as they are not mutually incompatible or exclusive.