Object specific language extension interface for a multi-level data structure

A computerized method (300) and software product (200) is provided for querying and modifying a Multi-Level Data Structure (106) stored in a Text-to-Speech (100) engine of a data processing system having a Central Processing Unit (202), a processing system memory (203), and an operating system (201), using an application program written in an interpretive programming language. The method includes the steps of initializing (302) by means of the CPU implementing a set of commands, a data processing environment for processing the application program, processing (306) the application program, where the processing includes identifying a marked command that encapsulates a DPMS program, and upon identifying a marked command, operating (318) on the MLDS using a DPMS interpreter for producing a result from the MLDS, the result available to the application program during execution of the application program.

BACKGROUND

1. Field of the Invention

The present invention relates to the field of information processing and, more particularly, to speech-based processing.

2. Description of the Related Art

Speech processing systems have become increasingly popular for interfacing with software applications, customer support systems, embedded devices, and voice interactive based electronics. Speech processing systems allow users to speak or enter text into a machine for performing a programmatic action. For example, a caller can speak into an interactive voice response system that can direct the caller to a routing destination using natural language understanding. In another example, a Text-to-Speech system can evaluate conditions within a written text to synthesize speech using a set of grammar rules which reveal how to read or interpret the text. The grammar rules can specify associations between the words describing how the text will be translated and constructed from its typographical form to acoustic form during a synthesis process.

International Business Machines Corporation (IBM) of Armonk, N.Y. provides a text-to-speech system that includes a text processing engine which uses a structure of parallel streams of information present in the speech synthesis process. This data structure of parallel streams is known as a Multilevel Data Structure (MLDS). This approach is based on a special-purpose programming language specifically designed for formulating and testing linguistic rules to operate on the text. The MLDS consists of multiple synchronized streams of coordinated units, such as phrases, words, syllables, phonemes and morphemes. For example, within the context of a text-to-speech system, the system can produce the various streams of information needed in the MLDS from the written text.

Systems involving complex data structures that require new mechanisms for accessing and manipulating them, such as the MLDS, are typically written in specialized languages that are specifically designed for handling these data structures. For example, the MLDS requires the explicit representation of relationships between all relevant (user- definable) abstract linguistic units, such as phrases, words, syllables, and phonemes, as well as quantitative phonetic values, such as formant frequencies, amplitudes, and durations. A language supporting this data structure must enable linguists or users to implement linguistic rules based on a wide range of phonological and phonetic models, which may involve testing for certain conditions in the data contained within the MLDS.

The Delta Programming Language was designed by linguists for managing the complex interactions between the phrases, words, morphemes, and phonemes created by the linguistic rules. The Delta Programming Language's specification contains a specialized pattern matching syntax called delta pattern matching syntax (DPMS) for managing a multi-level data structure. Using this language, the developer can specify pattern matching criteria called DPMS constructs for managing the MLDS. The delta programming language has its own proprietary format that can declare variables and write procedures. However, the delta language is limited and it does not support language extension components such as new data types, pointers, and arrays. Accordingly, the DPMS is limited in its ability to support sophisticated pattern matching searches and procedures within the context of an object oriented programming language such as C++. There remains a need, however, to extend a rich programming language like C++ to include the desired specialized features, such as the MLDS and its pattern matching syntax.

SUMMARY OF THE INVENTION

A computer program product is provided for use with a Multi-Level Data Structure (MLDS). In one arrangement, the computer program product comprises a computer usable medium having computer readable program code embodied in the medium for causing a Delta Pattern Matching Syntax (DPMS) program to be executed so as to manipulate the MLDS. For example, the DPMS program can be encapsulated within a marked command of the computer readable program code. The computer readable program code can include a first module for causing the data processing system to 1) initialize a data processing environment for processing the DPMS program by means of the CPU implementing a set of program language instructions, and 2) process the marked command for mapping constructs of the DPMS program to DPMS constructs in the programming language for placement in an encoded structure.

The computer readable program code can also include a second module for causing the data processing system to execute the DPMS constructs in the MLDS using a DPMS interpreter stored in the data processing system memory. For example, the DPMS constructs can represent a set of instructions specifying procedural rules of operation on the MLDS, where the DPMS interpreter can query and modify the MLDS using the DPMS constructs. Also, the DPMS interpreter can communicate with the MLDS using a delta programming language to produce a result. And, the Delta Pattern Matching Syntax (DPMS) program can be a single line command string written in the delta programming language. A plurality of marked commands can be dispersed throughout the computer readable program code.

In one arrangement, the computer readable first program code for mapping the constructs of the DPMS program can comprise a third module for causing the data processing system to parse the constructs of the DPMS program, convert the constructs of the DPMS program into executable runtime instructions, and place the executable runtime instructions in an encoded data structure within the computer readable program code in a format recognizable by the programming language. The parsing, converting, and placing can also occur during a run-time execution of the program code to communicate with the MLDS to receive the result during run-time processing of the program code.

In another arrangement, the computer readable first program code can comprise a first computer readable program code for causing the data processing system to designate areas of the data processing system memory as shared memory for work area storage during the processing of the DPMS program, and a second computer readable program code for causing the data processing system to designate data processing system variables for use during the processing of the DPMS program as shared variables. For example, the first and second computer readable program code can communicate data with the MLDS through assignment and access of the shared memory and the shared variables.

For example, the computer readable program code can be a C++ source code program and the encoded data structure can be a class object in the C++ programming language. Accordingly, the encoded data structure can have its own syntax and set of methods and language extensions for manipulating DPMS constructs of the encoded data structure within a C++ data processing environment.

In one arrangement, the second module for causing the data processing system to execute the DPMS constructs in the MLDS further can comprise computer readable program code for causing the DPMS interpreter to control a scope of the DPMS query within the MLDS using a “fence”. For example, the computer readable program code can cause the DPMS interpreter to restrict access to MLDS data within the fence boundaries, where the fence is specified by the computer programming language to restrict pattern matching within the MLDS to limit its response.

In another form, a computerized method is provided for creating a marked command in a computer application program, the marked command for querying and modifying a Multi-Level Data Structure (MLDS). The method can include the steps of scanning the computer application program for DPMS program statements, and encapsulating DPMS program statements under a descriptive header at a location in the DPMS program. For example, the DPMS program can include DPMS constructs that can represent a set of instructions specifying procedural rules of operation on the MLDS.

In another form of the invention, a computerized program product is provided for use with a Multi-Level Data Structure (MLDS) and data processing system memory. The computerized program product can include a computer usable medium having computer readable program code embodied in the medium for interfacing between a data processing environment for executing a C++ program with a data processing environment for a MLDS. In one arrangement, the computer readable program code can include a first module for causing a computer to designate areas of the data processing system memory for work area storage during the processing of the C++ program, and initializing data processing system variables used during the processing of the C++ program.

A computerized method is provided for querying and modifying a Multi-Level Data Structure (MLDS). The method can include the steps of initializing by the CPU implementing a set of commands, a data processing environment for processing the application program, where the processing includes identifying a marked command that encapsulates a DPMS program, and upon identifying a marked command, operating on the MLDS using a DPMS interpreter for producing a result from the MLDS. For example, the DPMS program can be a single line command string containing a DPMS construct written in the delta programming language. Also, the marked command can be dispersed throughout the application program.

In one arrangement, the DPMS interpreter can operate on the MLDS for communicating with the MLDS by executing a DPMS construct in the MLDS. The DPMS construct can be an instruction specifying procedural rules of operation on the MLDS. The DPMS interpreter can query and modify the MLDS using the DPMS construct to produce a result from the MLDS. For example, the result from the MLDS can be made available to the application program during execution of the application program. For instance, the result returned can be a set of acoustic parameter values used with a synthesizer to produce speech.

The initializing the data processing environment can include designating areas of the data processing system memory as shared memory for work area storage during the processing of the application program, and initializing data processing system variables for use during the processing of the application program as shared variables. In one arrangement the processing the application program can include parsing constructs of the DPMS program and converting them into executable runtime instructions that are placed in an encoded data structure. In another arrangement, the executing the DPMS construct can include using a fence to control a scope for querying and modifying the MLDS. For example, the fence can control access of the DPMS interpreter to the encoded data structure, and thereby restrict pattern matching within the MLDS to limit the result.

The encoded data structure can be placed within the computer readable program code in a format recognizable by the application programming language. In one arrangement, the steps of parsing constructs and converting them can occur during a run-time execution of the application program. For example, the DPMS interpreter can communicate with the MLDS during run-time execution for receiving a result during run-time processing of the application program.

DETAILED DESCRIPTION OF THE INVENTION

The invention disclosed herein provides a computer program product for use with a Multi-Level Data Structure (MLDS). The computer program product can include a computer usable medium having computer readable program code embodied in the medium for causing a Delta Pattern Matching Syntax (DPMS) program to be executed to perform operations on the MLDS, where the DPMS program can be encapsulated within a marked command of the computer readable program code. For example, the computer readable program code can include a first module for causing the data processing system to 1) initialize a data processing environment for processing the DPMS program by means of the CPU implementing a set of program language instructions, and 2) process the marked command for mapping constructs of the DPMS program to DPMS constructs in the programming language for placement in an encoded structure. The computer readable program code can execute the DPMS constructs using a DPMS interpreter stored in the data processing system memory. For example, the DPMS constructs can represent a set of instructions specifying procedural rules of operation on the MLDS, wherein the DPMS interpreter can query and modify the MLDS using the DPMS constructs, wherein the DPMS interpreter performs operations on the MLDS, and wherein a result is ultimately produced in the MLDS.

The operative aspects of the embodiments of the invention are further described herein primarily in the context of performing text-to-speech synthesis. It will be apparent to one of ordinary skill, however, that the invention applies equally well in other contexts, such as natural language understanding (NLU), voice recognition (VR), and interactive voice response (IVR) systems.

Referring toFIG. 1, a text engine100is shown. The text engine can include a text module120and a speech module130. The text module120can include a phoneme Database112, logic to analyze text performing operations on the MLDS, which included a set of linguistic rules110, and a multi-level data structure (MLDS)106. The Database is not limited to being a phoneme database. The text module can analyze text using linguistic rules110with the phoneme database112to produce the various parallel information streams in the MLDS. For example, the phrase “barking dogs” can be input to the TTS engine to generate the MLDS160. For example, the MLDS160can contain various streams such as word, syllable, morpheme, and phoneme, with each stream containing units representing relevant linguistic properties. The linguistic properties can be associations between units within a stream and across streams. For instance, the association can describe the intonation across each unit. In another example, a word unit can contain information about its grammatical category, a syllable unit can contain information about degree of stress or accent, and each phoneme can contain information about its place and manner of articulation.

The MLDS106can capture relationships between the word units of the parallel information streams. Accordingly, the MLDS106can provide important linguistic information for applying rules of speech production to speech synthesis. For example, the synthesis module130can utilize the linguistic information contained in the MLDS106for the MLDS160to derive acoustic parameters describing how speech production algorithms can synthesize speech. The speech module130can use the linguistic information produced by the text module120to determine perceptually relevant synthesizer parameters for synthesizing speech.

Algorithms102written in a standard programming language such as C++ with embedded DPMS statements can be used to manipulate the MLDS106to access and manage the MLDS. The algorithms102can contain DPMS constructs that can specify pattern matching statements to test for conditions within the MLDS.106. The MLDS106can communicate with the DPMS constructs contained within the delta program102to evaluate the connectivity associations between word elements, phrases, and word sub-units as described. The MLDS160can be a specialized data structure upon which Delta Pattern Matching Syntax (DPMS) constructs can be used to evaluate these conditions. To note, DPMS constructs are pattern matching statements written in the DPMS syntax for testing and modifying the contents of the MLDS. The MLDS106can analyze input text to reveal relationships identified by the TTS100between all relevant (user-definable) phonological units (e.g., phrases, words, syllables, phonemes) and quantitative phonetic values of the text. The MLDS106can provide flexible rule formalism through the DPMS constructs for manipulating this structure.

Referring toFIG. 2, a computer program product for use with a Multi-Level Data Structure (MLDS) stored in a Text-to-Speech (TTS) engine of data processing system200is shown. The data processing system200can include an operating system201, a Central Processing Unit (CPU)202, and a memory203. The computer program product can include an application program210, the memory203, a DPMS interface230, and a TTS engine100. The application program can communicate with the MLDS106of the TTS engine100through the DPMS interface230. The DPMS interpreter230can communicate with the MLDS106. An application program210can test and modify the MLDS106with the functionality available to the programming language of the application program210. For example, the application program210can be written in C++ and can contain object oriented classes that can communicate with the MLDS using DPMS constructs embedded within the class through the class methods.

DPMS constructs216can be written embedded in a standard within a marked programming language and can be encapsulated as a DPMS statement within a marked command212. The marked command212is contained as a literal string within a macro to distinguish it from native entries in the programming language of the application program210. The header214can identify the segment of code as a marked command212that can be interpreted by a DPMS interpreter230during a compilation of the application program210. For example, the DPMS interpreter230can identify marked commands by their respective header214, parse the DPMS constructs within the DPMS program216, and execute them using an encoded data structure that specifies the operations and operands that are involved in the DPMS statement. Different headers can signify different processing tasks. For example, one header can specify a modification operation in the MLDS such as changing a phoneme type. Whereas, a different header can specify a test operation within the MLDS, such as checking for suffixes or prefixes. The application program210can interact with the DPMS interface230through a shared memory203. Also, the application program210can declare and initialize variables during the compiling of the application program210based on the DPMS program216contained within the marked commands212. The application program210can share data and variables with the DPMS during run-time execution of the application program210.

The DPMS interpreter230can communicate with the MLDS106using MLDS class member functions104. The computer program product can also include a fence240that limits the extent of a pattern matching search by the MLDS106. The MLDS is a multi-tiered utterance representation consisting of an array of parallel data streams. The application program210can include control code to further restrict the depth of pattern matching and to further limit the extent of the DPMS linguistic rules. The application program210can, for example, set a fence240to isolate the matching of a DPMS rules to certain word unit connections or phrases within the MLDS.

Referring toFIG. 3, a method300for querying and modifying a Multi-Level Data Structure (MLDS) stored in a Text-to-Speech (TTS) engine is shown. Reference will also be made toFIG. 2for describing the actions of the structural program code elements responsible for causing the method steps. At step302, a data processing environment for processing a DPMS program encapsulated within a marked command of an application program can be initialized. At step304, areas of the data processing system memory can be designated as shared memory for work area storage during the processing of the DPMS program. Data processing system variables for use during the processing of the DPMS program can be designated as shared variables. Referring toFIG. 2, the operating system201designates shared memory and shared variables for the DPMS constructs216within the computer storage memory203during compilation of the application program210.

At step306the application program can be processed which can include identifying a marked command. At step310, constructs of the DPMS program can be parsed. At step312, the DPMS constructs can be converted into executable runtime instructions. At step314, the executable runtime instructions can be placed in an encoded data structure. And at step316, the encoded data structure can be placed within computer readable program code in a format recognizable by the programming language. The encoded data structure can be placed in program code in addition to being placed in memory.

For example, referring toFIG. 2, The DPMS interpreter230identifies a marked command212and parses DPMS constructs from the DPMS program216. The DPMS interpreter230converts the DPMS constructs to executable runtime instructions during compilation of the application programming language. The DPMS interpreter then places the executable runtime instructions in an encoded data structure within the memory203. The DPMS interpreter230sets aside the DPMS constructs from the DPMS program216into shared memory203as an encoded data structure.

Upon identifying a marked command, the method300at step318can operate on the MLDS using the DPMS interpreter to evaluate a condition or set of conditions in the MLDS. The response can be made available to the application program during execution of the application program. At step320, the DPMS constructs can be executed in the MLDS using a DPMS interpreter stored in the data processing system memory. For example, referring toFIG. 2, the DPMS interpreter230communicates with the MLDS106using the parsed DPMS constructs from the DPMS program. For example, the DPMS constructs are contained within the encoded data structure and represent a set of instructions specifying procedural rules of operation on the MLDS106. Accordingly, at step322, the DPMS interpreter can be used to query and modify the MLDS using the DPMS constructs contained within the encoded data structure. Referring toFIG. 2, during the program210execution, the DPMS interpreter230queries and modifies the MLDS by accessing the encoded data structure using program language components of the application program210.

Additionally, a scope of the DPMS query can be controlled within the MLDS using a fence. The fence is integrated within the application program to restrict pattern matching within the MLDS. For example, the application program210places the fence240between the MLDS106and the set of pattern matching rules110and grammar rules108. The application program210controls the extent of the fence240which limits the range of information in the MLDS that can be evaluated while evaluating conditions and matching patterns in the MLDS.

Referring toFIG. 4, a diagram of a portion of a compiled program code is shown. The application program will be discussed with reference toFIG. 2. The application program can be written as a C++ program within the data processing environment200for accessing the MLDS106. The C++ developer includes marked commands212to designate query and search requests within the MLDS during C++ program execution. The C++ developer compiles the application program, and during the compilation, the DPMS interpreter106identifies the marked commands212. The build-time parser maps constructs of the DPMS program216encapsulated within the marked command212to C++ constructs within an encoded data structure. The encoded data structure is a C++ class object that has its own syntax and set of methods and language extensions for evaluating the MLDS contents. The C++ constructs are the structure fields that identify the DPMS constructs. To note, the DPMS program216can be a single line command string written in the delta programming language and marked commands can be dispersed throughout the C++ program.

It should be noted, that the application developer writes the application program code210and creates the DPMS program content for the marked command212. For example, a developer can decide how they want the MLDS106to search for a phoneme type in a generated delta. The developer creates a marked command212that includes a DPMS program216containing DPMS constructs specifying search criteria for the phoneme type. For example, the developer compiles the application program and during compilation the DPMS interpreter230identifies the marked commands212. The DPMS interpreter communicates with the MLDS during a run-time execution to query a response during run-time processing of the application program. For example, each marked command212will query the MLDS as a distinct process made during the program execution.

Referring back to FIG.4., the DPMS interpreter230will preserve the locations of the marked command locations but include reference to either an encoded data structure220stored in memory, or a section of inserted C++ code containing executable program instructions. The interpretable code302can be one of an encoded data structure or a section of C++ code. Recall, the DPMS interpreter230parses the C++ application program searching for marked commands. Upon finding marked commands, the DPMS interpreter230can declare and initialize variables304within the C++ source code210having local or global scope for the DPMS program contained within the marked commands212. Accordingly, the C++ program designates areas of the data processing system memory for work area storage during the processing of the C++ program, and initializes data processing system variables used during the processing of the C++ program. The interpretable code302can have its own local scope in relation to the global scope of the C++ application program210. Local scope means that variables retain their values only within the interpretable code302. Global scope means variables can retain their values throughout program execution. The interpretable code may also access variables passed to a function scope as parameters, or to class member variables. The proper scoping of variables by name is accomplished with a VarList object that correlates the names of variables with their addresses in the intended scope. The marked commands signify to the DPMS interpreter230, to issue query requests to the MLDS at the time the marked commands are processed during program execution.

The DPMS constructs are encoded into data structures representing operators and operands within a C++ class object stored in the data processing system memory. The DPMS interpreter106interprets these data structures at runtime to effect the execution of the DPMS pattern matching operations on the MLDS106. The DPMS interpreter106passes DPMS constructs contained within the class object to the MLDS to evaluate conditions in the MLDS and produce a result. The C++ application program210can execute the marked command212using the DPMS interpreter106to query information in the MLDS using a C++ class object. The DPMS interpreter106can place the result in data processing system memory which becomes accessible to the C++ program through a C++ class object. The DPMS interpreter230can process all these actions at the time it identifies a marked command212. In effect, the program execution waits for the DPMS interpreter to process a result before continuing forward. Accordingly, the compiler works together with the DPMS interpreter230during compilation to establish the priority and timing of code execution.