SPEECH RECOGNITION USING CADENCE PATTERNS

A method for speech recognition using cadence patterns is provided. The method includes identifying speech cadence in user speech, which includes a plurality of sounds and pauses. At least one speech cadence pattern is identified from the plurality of sounds and pauses. The user speech is transcribed, and the transcribed user speech is modified based on the identified speech cadence pattern.

BACKGROUND

Exemplary embodiments of the present inventive concept relate to speech recognition, and more particularly, to speech recognition using cadence patterns.

Speech-to-text software relies on detected words and expected character patterns to provide accurate transcriptions. However, speech to text software struggles to detect “sing-song” speech and words spoken with human-to-human cadence. In addition, speech-to-text software cannot detect implied information. This is because attributes of speech, such as cadence are discarded or ignored by speech-to-text software despite containing valuable clues regarding context. For example, when a user communicates a telephone number to speech-to-text software, the speech-to-text software will recognize only the literal 10-digit number string, and thus omit implied hyphens and parentheticals around the area code because the periodic pauses are neglected.

SUMMARY

Exemplary embodiments of the present inventive concept relate to a method, a computer program product, and a system for speech recognition using cadence patterns.

According to an exemplary embodiment of the present inventive concept, a method for speech recognition using cadence patterns is provided. The method includes identifying speech cadence in user speech, which includes a plurality of sounds and pauses. At least one speech cadence pattern is identified from the plurality of sounds and pauses. The user speech is transcribed, and the transcribed user speech is modified based on the identified speech cadence pattern.

According to an exemplary embodiment of the present inventive concept, a computer program product is provided for speech recognition using cadence patterns. The computer program product includes one or more non-transitory computer-readable storage media and program instructions stored on the one or more non-transitory computer-readable storage media capable of performing a method. The method includes identifying speech cadence in user speech, which includes a plurality of sounds and pauses. At least one speech cadence pattern is identified from the plurality of sounds and pauses. The user speech is transcribed, and the transcribed user speech is modified based on the identified speech cadence pattern.

According to an exemplary embodiment of the present inventive concept, a computer system is used for speech recognition using cadence patterns. The system includes one or more computer processors, one or more computer-readable storage media, and program instructions stored on the one or more of the computer-readable storage media for execution by at least one of the one or more processors capable of performing a method. The method includes identifying speech cadence in user speech, which includes a plurality of sounds and pauses. At least one speech cadence pattern is identified from the plurality of sounds and pauses. The user speech is transcribed, and the transcribed user speech is modified based on the identified speech cadence pattern.

It is to be understood that the included drawings are not necessarily drawn to scale/proportion. The included drawings are merely schematic examples to assist in understanding of the present inventive concept and are not intended to portray fixed parameters. In the drawings, like numbering may represent like elements.

DETAILED DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present inventive concept are disclosed hereafter. However, it shall be understood that the scope of the present inventive concept is dictated by the claims. The disclosed exemplary embodiments are merely illustrative of the claimed system, method, and computer program product. The present inventive concept may be embodied in many different forms and should not be construed as limited to only the exemplary embodiments set forth herein. Rather, these included exemplary embodiments are provided for completeness of disclosure and to facilitate an understanding to those skilled in the art. In the detailed description, discussion of well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented exemplary embodiments.

In the interest of not obscuring the presentation of the exemplary embodiments of the present inventive concept, in the following detailed description, some processing steps or operations that are known in the art may have been combined for presentation and for illustration purposes, and in some instances, may have not been described in detail. Additionally, some processing steps or operations that are known in the art may not be described at all. The following detailed description is focused on the distinctive features or elements of the present inventive concept according to various exemplary embodiments.

As aforementioned, speech-to-text software relies on detected words and expected character patterns to accurately transcribe speech. However, speech cadence information replete with contextual and accuracy indicators is being discarded or ignored by conventional speech-to-text software. The present inventive concept augments speech-to-text software by using context indicators exhibited in speech cadence to detect implied characters and improve word detection, thereby improving transcription accuracy. The present invention is valuable to stenographers, voice-enabled chat bots, work related dictation, dictated message composition, or any other speech to text solutions. Even when transcription mistakes occur, the present inventive concept is still more approximately accurate because it recognizes the context indicators exhibited in speech cadence.

FIG.1depicts a speech recognition using cadence patterns system100, in accordance with an exemplary embodiment of the present inventive concept.

The speech recognition using cadence system100may include a computing device120and a speech recognition using cadence server130, which may all be interconnected via a network108. Programming and data content may be stored and accessed remotely across several servers via the network108. Alternatively, programming and data may be stored locally on as few as one physical computing device120or stored amongst multiple computing devices.

According to the exemplary embodiment of the present inventive concept depicted inFIG.1, the network108may be a communication channel capable of transferring data between connected devices. The network108may be the Internet, representing a worldwide collection of networks108and gateways to support communications between devices connected to the Internet. Moreover, the network108may utilize various types of connections such as wired, wireless, fiber optic, etc., which may be implemented as an intranet network, a local area network (LAN), a wide area network (WAN), or a combination thereof. The network108may be a Bluetooth network, a Wi-Fi network, or a combination thereof. The network108may operate in frequencies including 2.4 GHz and 5 GHz internet, near-field communication, Z-Wave, Zigbee, etc. The network108may be a telecommunications network used to facilitate telephone calls between two or more parties comprising a landline network, a wireless network, a closed network, a satellite network, or a combination thereof. In general, the network108may represent any combination of connections and protocols that will support communications between connected devices.

The computing device120may include third-party speech recognition software121, a speech recognition using cadence client122, and may be an enterprise server, a laptop computer, a notebook, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a server, a personal digital assistant (PDA), a smart phone, a mobile phone, a virtual device, a thin client, an IoT device, or any other electronic device or computing system capable of sending and receiving data to and from other computing devices. The computing device120may be equipped with a microphone, a camera, and/or movement sensors. The computing device120may be a speech recognition equipped smart device. Although the computing device120is shown as a single device, the computing device120may be comprised of a cluster or plurality of computing devices, in a modular manner, etc., working together or working independently.

The computing device120is described in greater detail as a hardware implementation with reference toFIG.3, as part of a cloud implementation with reference toFIG.4, and/or as utilizing functional abstraction layers for processing with reference toFIG.5.

The speech recognition using cadence client122may act as a client in a client-server relationship with a server, for example the speech recognition using cadence server130. The speech recognition using cadence client122may be a software and/or a hardware application capable of communicating with and providing a user interface for a user to interact with the speech recognition using cadence server130and/or other computing devices via the network108. Moreover, the speech recognition using cadence client122may be capable of transferring data between the computing device120and other computer devices/servers via the network108. The speech recognition using cadence client122may utilize various wired and wireless connection protocols for data transmission and exchange, including Bluetooth, 2.4 GHz and 5 GHz interne, near-field communication, etc. The speech recognition using cadence client122is described in greater detail with respect toFIGS.2-5.

The speech recognition using cadence server130may include a speech recognition using cadence repository132for storing various data (e.g., user credentials, user speech audio, user speech corpora, speech recognition using cadence models, modified user speech corpora, context keyword database, context keyword knowledge graphs, etc.) and speech recognition using cadence program134. The speech recognition using cadence program134may identify speech cadence in transcribed user speech; identify at least one speech cadence pattern; identify a speech context corresponding to the speech cadence pattern; and modify transcribed user speech based on the speech context. The speech recognition using cadence server130may act as a server in a client-server relationship with a client (e.g., the speech recognition using cadence client122). The speech recognition using cadence server130may be an enterprise server, a laptop computer, a notebook, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a server, a personal digital assistant (PDA), a rotary phone, a touchtone phone, a smart phone, a mobile phone, a virtual device, a thin client, an IoT device, or any other electronic device or computing system capable of sending and receiving data to and from other computing devices. Although the speech recognition using cadence server130is shown as a single computing device, the present inventive concept is not limited thereto. For example, the speech recognition using cadence server130may be comprised of a cluster or plurality of computing devices, in a modular manner, etc., working together or working independently.

The speech recognition using cadence server130is described in greater detail as a hardware implementation with reference toFIG.3, as part of a cloud implementation with reference toFIG.4, and/or as utilizing functional abstraction layers for processing with reference toFIG.5. The speech recognition using cadence program134and/or the speech recognition using cadence client122may be software and/or hardware programs that may facilitate speech recognition using cadence discussed in further detail with reference toFIGS.2-5.

FIG.2illustrates the flowchart of speech recognition using cadence200, in accordance with an exemplary embodiment of the present inventive concept.

The speech recognition using cadence program134may transcribe speech and generate a sound graph (step202). The audible speech may be pre-recorded (e.g., downloaded via the network108or uploaded by the user) or obtained in real-time. The speech recognition using cadence program134may detect inflection and/or different dialects/accents. Dialects may be detected based on speech cadence and/or using a global positioning system (GPS). The speech recognition using cadence program134may be configured to initialize on the computing device120via the speech recognition using cadence client122upon the occurrence of a triggering event (e.g., initialization of speech recognition software) and obtain audible speech in real-time. The audible speech may be converted into transcribed speech and a sound graph. The speech recognition using cadence program134may correspond the transcribed speech (e.g., words, letters, sentences, etc.) with the sound graph (sounds and pauses). The audible speech may be transcribed by voice recognition software included in the speech recognition using cadence program134and/or the third-party speech recognition software121. The sound graph may be generated by sound analysis software included in the speech recognition using cadence program134and/or the third-party speech recognition software121. The sound graph may include characteristics of the audible speech (e.g., tone, dialect, accent, inflection, sounds, pauses, frequency, amplitude, etc.). The sound graph may also be presented in visual format for the user. The audible speech, transcribed speech, and the sound graph may be stored in the speech recognition using cadence repository132. In an embodiment of the present invention, voice signature detection may be used to catalogue the audible speech, transcribed speech, and sounds graphs associated with a same user. A speech recognition using cadence model may be trained to transcribe speech and generate the sound graph.

For example, a cell-phone user gives a voice command to dial a customer service number comprised of the digits 9195552200, which initializes speech recognition software. The speech recognition using cadence program134initializes at a substantially simultaneous time as the speech recognition software. The speech recognition using cadence program134transcribes the 10 digits and generates a sound graph shown with reference toFIG.2A. The transcribed speech incorrectly reads “Dill 9195552200”.

The speech recognition using cadence program134may analyse the sound graph (step204). The speech recognition using cadence program134may identify user speech cadence from the sound graph. The user speech cadence as described herein may refer to a rhythm of speech that includes sounds and pauses therebetween. Each sound graph may represent at least a portion of an interaction with voice recognition software. The speech recognition using cadence program134may detect pauses and sounds in the sound graph. The sounds between pauses may be represented as tokens. The speech recognition using cadence program may identify clusters of sounds in the sound graph and the quantity of constituent sounds in each cluster, as well as the quantity of pauses between adjacent clusters. Each sound may correspond to an enunciated syllable or word of sufficient length and/or amplitude to eliminate confounding ambient and otherwise unintended noise. Discrete clusters of sounds may be identified based on a predetermined pause length or greater. The sounds may be grouped into a common cluster based on a predetermined pause length or less between sounds. The speech recognition using cadence program134may demarcate the sounds, clusters, pauses, and/or characteristics thereof.

The sounds and pauses (e.g., brackets, distinct colours for each sound and/or sound cluster, vertical lines, etc.) and/or annotate attributes thereof (e.g., corresponding transcribed text, quantity of sounds in a cluster, amplitudes, frequencies, etc.) may be visually depicted on the sound graph. In an embodiment, a speech cadence signature may be created for each user, which can be used to generate a speech recognition using cadence profile for personalization and/or user identification. The user's personalized speech recognition using cadence profile may take into account the characteristics of words as they are spoken by the user specifically (e.g., word length, volume, frequency, etc.) and/or common co-occurrences of neighbouring words. The generated sounds graphs, analysed and/or annotated versions of the sound graphs, and/or speech cadence signatures may be stored in the speech recognition using cadence repository132. The speech recognition using cadence model may be trained to analyse the sound graph.

For example,FIG.2Billustrates a visual representation of the analysed sound graph with identified speech cadence for the cell-phone user that gave the voice command to dial the customer service number comprised of the digits 9195552200. The identified speech cadence in the analysed sound graph demonstrates a pattern of 3 sounds, pause, 3 sounds, pause, 4 sounds, pause.

The speech recognition using cadence program134may identify a speech context associated with the user's identified speech cadence (step206). The user may identify the associated speech context(s) by manual input to the speech recognition using cadence program134. Alternatively, the speech recognition using cadence program134may automatically identify the associated speech context based on analysis or retrieval of the identified speech cadence. The identified speech cadence may be compared to users generally, and/or based on the user's personalized speech recognition using cadence profile. The speech recognition using cadence program134may retrieve a match to the identified speech cadence pattern and/or voice signature in the speech recognition using cadence repository132. The speech recognition using cadence repository132may contain a plurality of previously identified speech cadence patterns, each corresponding to at least one user and/or at least one speech context (e.g., industry-domain, use case, intent, acronyms, phonetic symbolic patterns, numeric symbolic patterns, topic(s) of discussion, etc.). Numeric symbolic patterns may include digit lengths associated with phone numbers, social security numbers, credit card numbers, part numbers, serial numbers, checking account numbers, etc. Phonetic symbolic patterns may include using a word to enhance the transcription of a letter. A single sound graph may contain a plurality of different contexts and/or correspond to a plurality of different users identified speech cadences. If a match to the identified speech cadence pattern is not found in the speech recognition using cadence repository132within a predefined threshold, the speech recognition using cadence program134may splice together matching sound portions (e.g., different sound graphs and/or non-consecutive sounds) and/or analyse the audible speech and/or transcribed speech using machine learning processes (e.g., automatic speech recognition (ASR) and natural language processing (NLP), respectively) to identify keywords associated with the speech context. In an embodiment, the speech recognition using cadence program134may also generate/use a knowledge graph for each speech context, which may include a plurality of domain related keywords. The context keywords, identified contexts, spliced matching sound portions, and/or analysed audible speech and/or transcribed speech may be stored in the speech recognition using cadence repository132. The speech recognition using cadence model may be trained to identify speech context associated with the user's identified speech cadence and/or their personalized speech recognition using cadence profile. Thus, the speech recognition using cadence program134may facilitate greater accuracy of speech transcription at step202.

For example, the speech recognition using cadence program134may identify the speech context associated with the identified speech cadence for the analysed sound graph of the cell-phone user that gave the voice command to dial the customer service number comprised of the digits 9195552200. The match to the identified speech cadence possesses a phonetic symbolic pattern associated with the context of reciting a telephone number. The speech recognition using cadence program134recognizes that the word “dial” was mistakenly transcribed/detected as “dill”, and that the user was trying to make a phone call via voice command.

The speech recognition using cadence program134may modify the transcribed user speech based on the identified speech context (step208). The speech recognition using cadence program134may compare differences between the transcribed speech and the transcribed speech corresponding to the match to the identified cadence pattern and make corrections accordingly. The speech recognition using cadence program134may resolve any word detection/estimation error(s) by performing the user's intended action (e.g., a voice command), generating a modified transcript, and/or communicating with the third-party speech recognition software121(e.g., transmitting the modified transcript, transmitting an artificial voice of the modified transcript with ideal enunciation, etc.). The modified transcript may include corrections to punctuation, hyphenation, spelling, neighbouring words (e.g., using the knowledge graph), etc. The corrections included in the modified transcript may be based on the identified speech context. The user may also provide feedback by approving, rejecting, and/or adding corrections to the modified transcript via an interface of the speech recognition using cadence client122. The modified transcripts may be stored in the speech recognition using cadence repository132. The modified transcript and/or the user feedback may be used to train the speech recognition using cadence model (e.g., improve audible speech detection, initial speech transcription, modified speech transcription, etc.). In an embodiment, acoustic models can be bootstrapped according to identified patterns and/or contexts (library of pre-defined patterns and/or contexts). In another embodiment, the user's personalized speech recognition using cadence profile may be communicated to an external computing device120running a speech recognition using cadence client122(e.g., an automated telephone line and/or user) to facilitate more accurate initial transcription or perform transcript modification upon a misdetection in conveyed speech.

For example, the identified context in the user speech is issuing a voice command to make a call. Thus, the neighbouring word “Dill” is replaced by “Dial” and the numbers 9195552200 have parentheses and hyphens inserted as follows: (919)-555-2200. Furthermore, the speech recognition using cadence program134takes the initiative to make the call on the user's cell-phone. The call is connected to an automated speech recognition enabled customer service operator running the speech recognition using cadence program134. The user asks to speak to the live representative “Herman” and spells out the name “H as in hotel, E as in elephant, R as in robot, M as in Michael, A as in Apple, N as in Nancy.” The speech recognition using cadence client122of the automated speech recognition enabled operator performs steps202-208and successfully detects the utterance of the name Herman given the identified context of phonetic symbolics.FIG.2Cdepicts a sound graph associated with the modified transcript of the phonetic symbolically spelled name Herman versus the mistakenly detected transcribed speech, “eight ashen hotel”.

FIG.3illustrates a block diagram depicting the hardware components of the speech recognition using cadence system100ofFIG.1, in accordance with an exemplary embodiment of the present inventive concept.

It should be appreciated thatFIG.3provides only an illustration of one implementation and does not imply any limitations regarding the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made.

Devices used herein may include one or more processors302, one or more computer-readable RAMs304, one or more computer-readable ROMs306, one or more computer readable storage media308, device drivers312, read/write drive or interface314, network adapter or interface316, all interconnected over a communications fabric318. Communications fabric318may be implemented with any architecture designed for passing data and/or control information between processors (such as microprocessors, communications and network processors, etc.), system memory, peripheral devices, and any other hardware components within a system.

One or more operating systems310, and one or more application programs311are stored on one or more of the computer readable storage media308for execution by one or more of the processors302via one or more of the respective RAMs304(which typically include cache memory). In the illustrated embodiment, each of the computer readable storage media308may be a magnetic disk storage device of an internal hard drive, CD-ROM, DVD, memory stick, magnetic tape, magnetic disk, optical disk, a semiconductor storage device such as RAM, ROM, EPROM, flash memory or any other computer-readable tangible storage device that can store a computer program and digital information.

Devices used herein may also include a R/W drive or interface314to read from and write to one or more portable computer readable storage media326. Application programs311on said devices may be stored on one or more of the portable computer readable storage media326, read via the respective R/W drive or interface314and loaded into the respective computer readable storage media308.

Devices used herein may also include a network adapter or interface316, such as a TCP/IP adapter card or wireless communication adapter (such as a 4G wireless communication adapter using OFDMA technology). Application programs311on said computing devices may be downloaded to the computing device from an external computer or external storage device via a network (for example, the Internet, a local area network or other wide area network or wireless network) and network adapter or interface316. From the network adapter or interface316, the programs may be loaded onto computer readable storage media308. The network may comprise copper wires, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers.

Devices used herein may also include a display screen320, a keyboard or keypad322, and a computer mouse or touchpad324. Device drivers312interface to display screen320for imaging, to keyboard or keypad322, to computer mouse or touchpad324, and/or to display screen320for pressure sensing of alphanumeric character entry and user selections. The device drivers312, R/W drive or interface314and network adapter or interface316may comprise hardware and software (stored on computer readable storage media308and/or ROM306).

The programs described herein are identified based upon the application for which they are implemented in a specific one of the exemplary embodiments. However, it should be appreciated that any particular program nomenclature herein is used merely for convenience, and thus the exemplary embodiments should not be limited to use solely in any specific application identified and/or implied by such nomenclature.

It is to be understood that although this disclosure includes a detailed description on cloud computing, implementation of the teachings recited herein are not limited to a cloud computing environment. Rather, the exemplary embodiments of the present inventive concept are capable of being implemented in conjunction with any other type of computing environment now known or later developed.

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

FIG.4illustrates a cloud computing environment, in accordance with an exemplary embodiment of the present inventive concept.

FIG.5illustrates abstraction model layers, in accordance with an exemplary embodiment of the present inventive concept.

Referring now toFIG.5, a set of functional abstraction layers provided by cloud computing environment50(FIG.4) is shown. It should be understood in advance that the components, layers, and functions shown inFIG.5are intended to be illustrative only and the exemplary embodiments are not limited thereto. As depicted, the following layers and corresponding functions are provided:

Aspects of the present inventive concept are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to exemplary embodiments. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

Based on the foregoing, a computer system, method, and computer program product have been disclosed. However, numerous modifications, additions, and substitutions can be made without deviating from the scope of the exemplary embodiments of the present inventive concept. Therefore, the exemplary embodiments of the present inventive concept have been disclosed by way of example and not by limitation.