TEXT TO SPEECH PROMPT TUNING BY EXAMPLE

According to one embodiment, a method, computer system, and computer program product for customizing the rendering of a synthesized speech prompt is provided. The present invention may include extracting prosodic information from a received audio recording of a prompt by parsing the text corresponding with the prompt and generating phonetic units, aligning the phonetic units with the audio recording, and calculating, based on the alignment, prosodic values for the phonetic units. The invention may further include adapting the prosodic values to match a text-to-speech voice in use, and then synthesizing speech for the prompt based upon the adapted prosodic information.

BACKGROUND

The present invention relates, generally, to the field of computing, and more particularly to speech synthesis.

Speech synthesis is the artificial production of human speech by a computer system. As computers become more advanced and more deeply integrated into users' everyday lives, convenient means of interfacing between humans and computers are of increasing interest. Speech is a natural avenue to pursue as a user interface method; after all, it is already the means by which humans primarily interact with other humans. However, the use of speech as an interface method introduces new levels of complexity. Beyond mere intelligibility of synthesized speech, which is crucial in its own right, the rendering of a given phrase conveys a great deal of additional meaning: whether the phrase constitutes a statement, question, or command, the presence of irony or sarcasm, emphasis, contrast, focus, the mood or intent of the speaker, and more. As such, a correct rendering is crucial to the future success of speech synthesis as a human interface method.

SUMMARY

According to one embodiment, a method, computer system, and computer program product for customizing the rendering of a synthesized speech prompt is provided. The present invention may include extracting prosodic information from a received audio recording of a prompt by parsing the text corresponding with the prompt and generating phonetic units, aligning the phonetic units with the audio recording, and calculating, based on the alignment, prosodic values for the phonetic units. The invention may further include adapting the prosodic values for use with the text-to-speech voice in use, and then synthesizing speech for the prompt based upon the adjusted prosodic information.

DETAILED DESCRIPTION

Embodiments of the present invention relate to the field of computing, and more particularly to speech synthesis. The following described exemplary embodiments provide a system, method, and program product to, among other things, analyze recorded speech from a user, extract prosodic information from the recorded speech, and utilize the prosodic information for speech synthesis. Therefore, the present embodiment has the capacity to improve the technical field of speech synthesis by providing a means of incorporating prosodic information from speech recordings to modify and correct the rendering of synthesized speech.

As previously described, speech synthesis is the artificial production of human speech by a computer system. As computers become more advanced and more deeply integrated into users' everyday lives, convenient means of interfacing between humans and computers are of increasing interest. Speech is a natural avenue to pursue as a user interface method; after all, it is already the means by which humans primarily interact with other humans. One method of synthesizing speech is by storing short clips of recorded human speech, from whole words down to individual sounds, and combining these recorded sounds to create words and sentences. Another method involves utilizing a synthesizer which can model the vocal tract and human voice characteristics to create purely artificial speech from scratch. More recent methods involve the use of deep neural networks to predict acoustic features of the speech and to encode the resulting audio.

However, speech synthesis often fails to achieve the desired result; even where synthesized speech comprises all the correct phonemes, synthesis errors can still be enough to render the synthesized speech unintelligible, unsatisfactory or lacking in expressiveness. Synthesized speech may fail to convey any of a host of additional linguistic features that humans rely on for context and clear communication. Such problems are often encountered, for example, by designers for computer applications, where the application needs to say a number of messages (prompts) to the user running the application. For instance, an application may need to ask the user for her account number. However, during testing, the designers often realize that the prompt doesn't sound the way that it was intended. For example, the text-to-speech engine synthesizing the speech from the prompt may place emphasis on the wrong word, pause in inappropriate places or for an inappropriate duration, pronounce a word incorrectly, produce synthesized speech that is technically correct but sounds unnatural, add an awkward inflection, or introduce other flaws to the audio.

Users may try to address such issues by adding punctuation, changing the pronunciation, or changing the text of the prompt in the hope that the text-to-speech engine will be able to synthesize a different prompt correctly. These approaches are inconsistent in their success, and are extremely limited in the control they afford a user over the synthesis of the prompts.

In some cases, these issues might have been addressed by using the original voice talent providing the voice of a given text-to-speech program to record prompts and include these recordings in the generated voice, splicing phrases into the prompt as needed in a method known as “phrase splicing.” However, this option requires the original voice talent to be available, and introduces a significant delay between when the voice talent is available to record and when the corrected recording is available to the user.

Arguably the most useful tool currently available to address synthesis issues is a suite of commands included in the speech synthesis markup language (SSML). SSML commands allow a user finer control over synthesized speech, such as by enabling the user to specify the pronunciation of words in the text, add pauses, specify text normalization rules, change the speaking rate, or alter the base pitch. This can go a long way towards correcting prompts, and where a prompt is already synthesized correctly, SSML commands can still be used to make subtle changes which may include resulting quality.

However, composing SSML text minutely detailing the rendering of synthesized speech is a lengthy and manually intensive process which must be performed on each prompt. In the most difficult cases, a user may spend half an hour tuning an individual prompt. Furthermore, language expertise may be needed to tune difficult prompts; in many cases, it is difficult for casual users to even express what is wrong with the synthesized audio. Common complaints include: “sounds robotic,” “not human-like,” “the tone is all wrong,” et cetera. Without a certain level of linguistics expertise, a user may not know which SSML commands address which problems. Even given time, effort and expertise, SSML commands may still be insufficient to the task of producing the desired quality. As such, it may be advantageous to, among other things, implement a simple and intuitive mechanism by which users can improve the synthesis of specific prompts without any prior knowledge of linguistics or intensive SSML code, by submitting samples of correct audio recordings of incorrectly rendered synthesized speech prompts to a system; this system extracts the prosodic components of the correct audio recording to determine the correct realization, and applies that knowledge to future synthesis of the speech prompt in question.

Used herein, the term “prompt” may be used to refer to a discrete segment of language of any length and in any form, for instance textual or audible, which may be targeted to be rendered as speech by a speech synthesis application, or may correspond to audible speech that has already been rendered.

According to at least one embodiment, the invention is a system for correcting or modifying synthesized speech for a given prompt, which receives an audio recording and corresponding text of the prompt from a user, extracts prosodic information from the audio, associates the prosodic information with the prompt by means of a customization identification, and stores the prosodic information.

In some embodiments, the audio recording may be a recording of a user reading the prompt in a fashion which the user desires the system to emulate when synthesizing speech for the prompt. The user may select the prompt to submit audio recordings for based on imperfections or undesired properties of synthesized speech corresponding with the prompt; for example, in the case of user Dave, Dave may be attempting to program an application to audibly express the line, “Snowfall is expected to reach 10 inches today,” and he would like to hear the number 10 stressed. If the synthesis of this prompt fails to stress 10, Dave may submit a recording of himself reading this prompt with the number 10 emphasized.

In some embodiments, the user may submit an audio recording to customize the synthesis of a prompt; in other words, a prompt may be synthesized as speech correctly, but a user may wish to modify the rendering to convey a different meaning, emotion, or implication, to suit different contexts, to emphasize different words, et cetera. For instance, the word “goodbye” could be pronounced in a variety of different ways depending on context; in a happy context, for example where a text to speech application was able to help a user, “goodbye” pronounced in a cheerful tone with a rising inflection may be most appropriate. Conversely, where a text to speech application was unable to help a user, “goodbye” pronounced with a downward inflection or in a more neutral tone may be desired. In another example, a user may include a pun in the prompt, and may wish the synthesized speech to place greater emphasis on the pun.

In some embodiments, prosodic information may be any information regarding a multitude of linguistic properties that comprise a speech realization, such as intonation, rhythm, stress, and tone. In some embodiments, the prosodic information of the audio recording may be all information necessary to reproduce the realization of the audio recording when synthesizing speech for the corresponding prompt.

In some embodiments, the prosodic information extracted from the audio recording may be associated with the prompt to which it corresponds via a customization identification (ID). The customization ID may be a unique identifier associated with a customization, or rendering of the prompt described by the prosodic information. The customization ID may identify to the system that a customization for the prompt exists, and allows the customization to be specifically invoked, for example where a user desires to utilize a particular realization in synthesizing speech for the prompt. In embodiments where multiple customizations exist for the same prompt, the customization ID may distinguish the customizations from each other and may contain additional information to this end. For instance, the customization ID may contain information regarding the context of the customization; where one customization is pronounced as a command, and another customization is pronounced as a question, the customization ID may identify the former as a “command” and the latter as a “question.” In another example, where one customization is cheerful and uses an upward inflection, another is neutral, and a third is gloomy and uses a downward inflection, the customization ID of each may further read “happy,” “neutral,” and “sad,” respectively.

In some embodiments, the system may be a real-time or near-real-time interactive system. For example, the system may be responsive to user inputs and submissions, and may respond to user inputs and reply to the user in real-time or near-real-time.

In some embodiments, the system may prompt the user to submit the audio recording and corresponding text. In other embodiments, the system may provide the user with a graphical user interface for submitting the audio recording and corresponding text of the prompt. In some embodiments, the system may prompt or enable the user to record multiple audio recordings for the prompt, and may allow the user to hear synthesized output resulting from each of the recordings and enable the user to select the preferred one to keep for customization.

In some embodiments, the system may receive prompts that contain fixed and dynamic language. Such prompts may be lines where one subset of the prompt occurs unchanged in synthesis requests, while another subset of the prompt changes across multiple instances. The subsection of the prompt that occurs unchanged may be fixed, while the subsection of the prompt that changes across instances may be dynamic. For example, the prompt “Your account balance is 802.32 dollars” may occur multiple times with a different dollar number in the account balance; in such case, the subsection of the phrase “your account balance is . . . ” and “ . . . dollars” may be the fixed language, and the number, here “802.32,” may be the dynamic language. In some embodiments, the system may receive the audio recording and/or corresponding text already flagged as containing fixed/dynamic language, and/or with fixed/dynamic language sections specifically identified by a user or administrator. In some embodiments, the system may identify the presence of fixed or dynamic language by reading the flags associated with the prompt or by automatically detecting the presence of or a likelihood of fixed dynamic language; for instance, the system may automatically detect subsections that are typically dynamic such as currency amounts or dates. In some embodiments, the system may query the user as to whether such subsections should be flagged as dynamic.

In some embodiments, for example where fixed and dynamic language has been identified, the system may only extract prosodic information from the audio recording corresponding with the fixed subset or subsets of the prompt; because the fixed subset is the only subset that would reoccur, extracting prosodic information from the entire prompt including the dynamic subset would result in a customization that could not be applied to instances of the prompt where only the dynamic language has changed. In some embodiments, the system may separately extract prosodic information from the dynamic subset and the fixed subset, and may store the renderings of the two subsets as separate customizations, with separate customization IDs.

In some embodiments, the system may adapt the prosodic information of a customization to the individual voice being used to synthesize speech. Speech synthesis programs may use any number of voices; in order to integrate the prompt with any number of speech synthesis programs, or any number of possible voices, the system may adapt the prosodic information to match the voice being used to synthesize the speech. Adapting the prosodic information may include uniformly adjusting the pitches contained in the prosodic information of the customization to match the vocal range of the voice being used.

According to at least one embodiment, the invention is a system that extracts prosodic information from an audio recording of a prompt by parsing the text corresponding with the prompt and generating phonetic units, aligning the phonetic units with the audio recording, and calculating, based on the alignment, prosodic values for at least one of the phonetic units.

In some embodiments, the phonetic units may be distinct sounds which, when combined together, create speech. The system may use any units that correspond to the distinctive sounds of a language. For example, the system may use phonemes as the phonetic units, which may be the minimal categorical unit of sound that can be used to distinguish between words in a language. However, in some embodiments, the phonetic units may be smaller (for example, subphonemes), or larger, for instance including combinations of sounds such as phonemes, syllables, et cetera. In the context of text, phonetic units may be the sounds represented by each letter and/or word of the text.

In some embodiments, parsing the plurality of received text into phonetic units may include processing the received text to identify each phonetic unit, or segment of sound, represented by the text. For example, the system may identify and delineate every individual syllable represented by the received text.

In some embodiments, the prosodic values may be the numerical values or metrics by which the prosodic information is enumerated. In some embodiments, the prosodic values may be the starting and ending pitch of a phonetic unit, and/or any representation of pitch within the phonetic unit. The prosodic values may include measures of volume or energy at points within the phonetic unit, duration of a phonetic unit, et cetera. Prosodic values may represent additional speech features such as stress, vowel length, et cetera.

According to at least one embodiment, the invention is a system for synthesizing speech for a previously corrected or modified prompt by receiving a customization identification, extracting stored prosodic information corresponding with the received customization identification, and synthesizing speech for the prompt based on the extracted prosodic information.

The following described exemplary embodiments provide a system, method, and program product to analyze recorded speech from a user, extract prosodic information from the recorded speech, and utilize the prosodic information for speech synthesis.

Referring toFIG. 1, an exemplary networked computer environment100is depicted, according to at least one embodiment. The networked computer environment100may include client computing device102and a server112interconnected via a communication network114. According to at least one implementation, the networked computer environment100may include a plurality of client computing devices102and servers112, of which only one of each is shown for illustrative brevity.

Client computing device102may include a processor104that is enabled to host and run a text to speech engine106A and a prompt tuning program110A and communicate with the server112via the communication network114, in accordance with one embodiment of the invention. Client computing device102may be, for example, a mobile device, a telephone, a personal digital assistant, a netbook, a laptop computer, a tablet computer, a desktop computer, or any type of computing device capable of running a program and accessing a network. As will be discussed with reference toFIG. 7, the client computing device102may include internal components702aand external components704a, respectively.

The server computer112may be a laptop computer, netbook computer, personal computer (PC), a desktop computer, or any programmable electronic device or any network of programmable electronic devices capable of hosting and running a text to speech engine106B and a prompt tuning program110B and a database116and communicating with the client computing device102via the communication network114, in accordance with embodiments of the invention. As will be discussed with reference toFIG. 7, the server computer112may include internal components702band external components704b, respectively. The server112may also operate in a cloud computing service model, such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS). The server112may also be located in a cloud computing deployment model, such as a private cloud, community cloud, public cloud, or hybrid cloud.

According to the present embodiment, the text to speech engine106A,106B may be a program enabled to synthesize human speech from text. In some embodiments, text to speech engine106A,106B may be enabled to convert normal language text into speech, and/or to convert symbolic linguistic representations such as phonetic transcriptions. The text to speech engine106A,106B may be located on client computing device102or server112or on any other device located within network114. Furthermore, text-to-speech engine106A,106B may be distributed in its operation over multiple devices, such as client computing device102and server112. The text to speech engine106A,106B may operate or otherwise be in communication with a speaker capable of reproducing human speech.

According to the present embodiment, the prompt tuning program110A,110B may be a program enabled to analyze recorded speech from a user, extract prosodic information from the recorded speech, and utilize the prosodic information for speech synthesis. The prompt tuning program110A,110B may be located on client computing device102or server112or on any other device located within network114. Furthermore, prompt tuning program110A,110B may be distributed in its operation over multiple devices, such as client computing device102and server112. The prompt tuning program110A,110B may be a subroutine or otherwise integrated into text to speech engine106A,106B, or may be a separate and/or standalone program. The prompt tuning program110A,110B is depicted as being located on the same computing device as text to speech engine106A,106B, but may be located on different computing devices relative to text to speech engine106A,106B. The prompt tuning method is explained in further detail below with respect toFIG. 2.

Referring now toFIG. 2, an operational flowchart illustrating a prompt tuning process200is depicted according to at least one embodiment. At202, the prompt tuning program110A,110B receives an audio recording and associated text of a prompt from a user. The audio recording may comprise the same words as the text, and both the text and the audio recording may comprise the same words as the prompt. While an advantage of the prompt tuning process200is that it simplifies the process of adjusting a prompt for a user to the mere step of submitting an audio recording and corresponding text, it may be desirable in some embodiments (for example, where there are multiple audio recordings corresponding to the same prompt that could benefit by being distinguished from one another, or where an audio recording is best suited to a particular context), to request or accept additional information from the user; in some embodiments, the user may further submit information describing the audio recording, such as the context, part of speech, or intended emotion to be conveyed by the user's reading of the prompt. For instance, the user may indicate if the rendering is intended to convey sarcasm or irony, anger, incredulity, happiness, et cetera. The user may indicate whether the rendering casts the prompt as a command, query, statement, et cetera.

Next, at204, the prompt tuning program110A,110B assigns an identification number (ID) to the prompt. The ID may be a unique identifier associated with a customization, or rendering of the prompt described by the prosodic information. In some embodiments, such as where the user has contributed additional information, prompt tuning program110A,110B may incorporate the additional information into the ID, or otherwise associate the information with the ID.

At206, prompt tuning program110A,110B performs prosodic information extraction. Prosodic information extraction is a process of extracting useful information from the audio recording and associated text, and may comprise the steps of parsing text and generating phonetic units, aligning these phonetic units with the audio, and calculating prosodic values for each phonetic unit.

At208, prompt tuning program110A,110B parses the text into phonetic units. Parsing the text into phonetic units may include processing the received text to identify each phonetic unit, or segment of sound, represented by the text. For example, prompt tuning program110A,110B may identify and delineate every individual syllable represented by the received text. Where a word can be pronounced in multiple different ways, such as, for example, the word “bass,” prompt tuning program110A,110B may consult a dictionary of possible pronunciations for a word to determine possible or probable combinations of phonetic units that the word may represent.

At210, prompt tuning program110A,110B aligns the phonetic units with the audio recording. Here, prompt tuning program110A,110B identifies the location of each phonetic unit within the audio. Since the text is a written transcript of the audio, the phonetic units generated from the text must therefore be found in the audio. In some number of cases, the audio recording may contain additional sounds or rests not represented in the text, which therefore have no textual counterpart. For example, in the audio recording, the user may insert filler words such as “um” or “err,” sounds such as derisive snorts or laughter, stuttering, et cetera. In some embodiments, the prompt tuning program110A,110B may utilize paralinguistic detection methods to identify these non-speech components of the audio recording, and may flag these paralinguistic phonetic units. For instance, where prompt tuning program110A,110B records transcription text of the audio, the transcription text may include markers in any markup language to indicate where these sounds are located. For example, the transcription text may read “<hmm> That doesn't seem right,” where prompt tuning program110A,110B identifies the paralinguistic sound (“hmm”) with angle brackets. In some embodiments, the identified paralinguistic components may be disregarded during the process of matching corresponding phonetic units in the text and audio, as paralinguistic components may not match the text. In some embodiments, even where identified paralinguistic components are disregarded for purposes of matching phonetic units, paralinguistic components may be included in the synthesized output.

At212, prompt tuning program110A,110B calculates prosodic values for each phonetic unit. Once prompt tuning program110A,110B has aligned the phonetic units to the audio recording, prompt tuning program110A,110B may calculate the prosodic values by measuring any quality of the audio that pertains to the rendering of the audio recording. For example, the prompt tuning program110A,110B may measure the pitch at the beginning and/or end of the phonetic unit, and/or the pitch at any number of points within the phonetic unit. The prompt tuning program110A,110B may measure the volume or energy at points within the phonetic unit, the duration of a phonetic unit, and other speech features such as stress, vowel length, et cetera.

At214, prompt tuning program110A,110B stores the phonetic units and prosodic values in a database. The prompt tuning program110A,110B may store the phonetic units and prosodic values as a customization, and in some embodiments, such as where the user has submitted additional information pertaining to the audio recording, may store the user-submitted information as well.

At216, prompt tuning program110A,110B returns the customization information to the user. In some embodiments, prompt tuning program110A,110B may return the customization, comprising the prosodic information, to the user so that the user may employ the prosodic information or modify it further. In some embodiments of the invention, the customization information may be instead, or additionally, passed to a speech synthesizer program to be played audibly as synthesized speech.

Referring now toFIG. 3, an exemplary computing environment300executing the prompt tuning process ofFIG. 2is depicted according to at least one embodiment. The user302provides the phonetic alignment generator304with an audio recording and associated text of a prompt as in step202ofFIG. 2. User302may be any user of the prompt tuning program110A,110B, including human users as well as programs or services. The phonetic alignment generator304may parse the text and generate phonetic units as in step208, and may align the parsed phonetic units as in step210. The phonetic alignment generator304may then pass the aligned phonetic units to the prosody generator306. The prosody generator306calculates prosodic values for each phonetic unit, as in step212, and then passes its output to database116, as in step214ofFIG. 2. The prompt tuning program110A,110B then provides the user302with the customization identification from the database116.

Referring now toFIG. 4, an exemplary computing environment400executing the prompt tuning process ofFIG. 2is depicted according to at least one embodiment. The computing environment400is identical to computing environment300except for the inclusion of a customized extractor402. The customized extractor402identifies fixed and dynamic language within the prompt, and extracts the dynamic sections of the prompt, such that the prosodic information is not stored for the dynamic text and the rendering of the dynamic sections is not customized. In some embodiments, the customized extractor402may simply delineate between the fixed and dynamic text but maintain the prosodic information for both, so that the customization will be applied to the entire prompt but if the dynamic text changes, the customization may still be applied to the fixed text.

Referring now toFIG. 5, an operational flowchart illustrating a prompt tuning process500is depicted according to at least one embodiment. At502, prompt tuning program110A,110B receives a request specifying the customization ID from a user. The request may be in any computer-readable format, be it code, a written request by the user, et cetera. The request may be in the form of an SSML mark-up containing a customization ID as a tag. For example, a customization associated with customization ID7584and modifying the prompt “Welcome to ABC Bank” could be invoked in SSML via the command <custom id=7584>Welcome to ABC Bank</custom>.

At504, prompt tuning program110A,110B extracts the database entry corresponding with the customization ID from the database116. The prompt tuning program110A,110B may parse an index of the database116to identify the address of the customization pertaining to the customization ID, and retrieve the customization for use.

At506, prompt tuning program110A,110B adapts the prosodic values from the database entry for the text-to-speech (TTS) voice in use. Text-to-speech engine106may be utilizing a particular voice, either by default, user selection, or for any other reason, which differs from the voice recorded in the audio recording, and therefore from the prosodic information extracted from the audio recording. As such, prompt tuning program110A,110B may adapt the customization to match the voice being used by text to speech engine106. The prompt tuning program110A,110B may, for instance, adapt the prosodic information by adjusting the pitches contained in the prosodic information of the customization to match the vocal range of the voice being used by text to speech engine106. The prompt tuning program110A,110B may adjust the speaking rate to match that of the voice in use. In some embodiments, the prompt tuning program110A,110B may adjust other prosodic features for use with the voice.

At508, prompt tuning program110A,110B produces synthesized audio from the adapted prosodic values. The prompt tuning program110A,110B may convert the prosodic values into speech by any method, for instance by concatenating pieces of recorded speech stored in a database, and modifying these pieces of recorded speech with the prosodic information. In some embodiments, the synthesized output may be produced by using neural network models to predict acoustic features which are then used by a neural vocoder to generate the speech. In some embodiments, the prompt tuning program110A,110B may pass the prosodic values to text-to-speech engine106or another program or service to perform the speech synthesis.

Referring now toFIG. 6, an exemplary computing environment600executing the prompt tuning process ofFIG. 5is depicted according to at least one embodiment. User302provides an SSML input text to prosody generator306. The prosody generator306generates uncustomized prosody information for the full text and provides the uncustomized prosody information to prosody updater602. Uncustomized prosody may be prosodic information created for a prompt in the process of speech synthesis that has not been customized by recorded audio from a user. Prosody updater602replaces the prosody of the customized portion with values in database116. Prosody updater602then passes the updated prosody to the prosody normalizer604, which further adjusts the prosody of the customized portion to match TTS voice pitch and speaking rate. The prosody normalizer604then passes the adjusted prosody information to the synthesizer606, which utilizes the adjusted prosody information to synthesize audible speech, and play the synthesized speech back to the user. The prosody generator306, prosody updater602, prosody normalizer604, and synthesizer606are herein depicted as subroutines or components of text to speech engine106, but in other embodiments may be external to text to speech engine106in any combination.

It may be appreciated thatFIGS. 2-6provides only illustrations of individual implementations and do not imply any limitations with regard to how different embodiments may be implemented. Many modifications to the depicted environments may be made based on design and implementation requirements.

FIG. 7is a block diagram700of internal and external components of the client computing device102and the server112depicted inFIG. 1in accordance with an embodiment of the present invention. It should be appreciated thatFIG. 7provides only an illustration of one implementation and does not imply any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environments may be made based on design and implementation requirements.

The client computing device102and the server112may include respective sets of internal components702a,band external components704a,billustrated inFIG. 7. Each of the sets of internal components702include one or more processors720, one or more computer-readable RAMs722, and one or more computer-readable ROMs724on one or more buses726, and one or more operating systems728and one or more computer-readable tangible storage devices730. The one or more operating systems728, the software program108and the prompt tuning program110A in the client computing device102, and the prompt tuning program110B in the server112are stored on one or more of the respective computer-readable tangible storage devices730for execution by one or more of the respective processors720via one or more of the respective RAMs722(which typically include cache memory). In the embodiment illustrated inFIG. 7, each of the computer-readable tangible storage devices730is a magnetic disk storage device of an internal hard drive. Alternatively, each of the computer-readable tangible storage devices730is a semiconductor storage device such as ROM724, EPROM, flash memory or any other computer-readable tangible storage device that can store a computer program and digital information.

Each set of internal components702a,balso includes a R/W drive or interface732to read from and write to one or more portable computer-readable tangible storage devices738such as a CD-ROM, DVD, memory stick, magnetic tape, magnetic disk, optical disk or semiconductor storage device. A software program, such as the prompt tuning program110A,110B, can be stored on one or more of the respective portable computer-readable tangible storage devices738, read via the respective R/W drive or interface732, and loaded into the respective hard drive730.

Each set of internal components702a,balso includes network adapters or interfaces736such as a TCP/IP adapter cards, wireless Wi-Fi interface cards, or 3G or 4G wireless interface cards or other wired or wireless communication links. The software program108and the prompt tuning program110A in the client computing device102and the prompt tuning program110B in the server112can be downloaded to the client computing device102and the server112from an external computer via a network (for example, the Internet, a local area network or other, wide area network) and respective network adapters or interfaces736. From the network adapters or interfaces736, the software program108and the prompt tuning program110A in the client computing device102and the prompt tuning program110B in the server112are loaded into the respective hard drive730. The network may comprise copper wires, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers.

Each of the sets of external components704a,bcan include a computer display monitor744, a keyboard742, and a computer mouse734. External components704a,bcan also include touch screens, virtual keyboards, touch pads, pointing devices, and other human interface devices. Each of the sets of internal components702a,balso includes device drivers740to interface to computer display monitor744, keyboard742, and computer mouse734. The device drivers740, R/W drive or interface732, and network adapter or interface736comprise hardware and software (stored in storage device730and/or ROM724).

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

Workloads layer90provides examples of functionality for which the cloud computing environment may be utilized. Examples of workloads and functions which may be provided from this layer include: mapping and navigation91; software development and lifecycle management92; virtual classroom education delivery93; data analytics processing94; transaction processing95; and prompt tuning96. The prompt tuning96may be enabled to analyze recorded speech from a user, extract prosodic information from the recorded speech, and utilize the prosodic information for speech synthesis.