Data-driven and rule-based speech recognition output enhancement

According to some embodiments, a multi-layer speech recognition transcript post processing system may include a data-driven, statistical layer associated with a trained automatic speech recognition model that selects an initial transcript. A rule-based layer may receive the initial transcript from the data-driven, statistical layer and execute at least one pre-determined rule to generate a first modified transcript. A machine learning approach layer may receive the first modified transcript from the rule-based layer and perform a neural model inference to create a second modified transcript. A human editor layer may receive the second modified transcript from the machine learning approach layer along with an adjustment from at least one human editor. The adjustment may create, in some embodiments, a final transcript that may be used to fine-tune the data-driven, statistical layer.

BACKGROUND

An automatic speech recognition system may generate a text transcript based on an audio input containing human speech. In some cases, however, these text transcripts may contain information that will impair a human's ability to understand what is being said. For example, if a speaker says the letters “A.S.R.U.” (e.g., referencing Automatic Speech Recognition and Understanding), the transcript might instead read “ASR you”). Similarly, the transcripts might contain information will impair a machine's ability to understand and react to what is being said. For example, a speaker might say “I think that was, uh, hmmm, five years ago . . . no six, it was six years ago.” Although technically accurate, such a transcript might confuse a downstream task such as machine reading comprehension and/or summarization. As a result, a system's performance may suffer when using such transcripts. What is needed are systems and methods to accurately and efficiently improve automatic speech recognition transcripts.

SUMMARY

Systems, methods, and computer readable storage devices embodying instructions for improved automatic speech recognition transcripts are provided herein. In some embodiments, a speech recognition transcript post processing system may include a data-driven, statistical layer associated with a trained automatic speech recognition model that selects an initial transcript. A rule-based layer may receive the initial transcript from the data-driven, statistical layer and execute at least one pre-determined rule to generate a first modified transcript. A machine learning approach layer may receive the first modified transcript from the rule-based layer and perform a neural model inference to create a second modified transcript. A human editor layer may receive the second modified transcript from the machine learning approach layer along with an adjustment from at least one human editor. The adjustment may create, in some embodiments, a final transcript that may be used to fine-tune the data-driven, statistical layer.

Examples are implemented as a computer process, a computing system, or as an article of manufacture such as a device, computer program product, or computer readable medium. According to an aspect, the computer program product is a computer storage medium readable by a computer system and encoding a computer program comprising instructions for executing a computer process.

DETAILED DESCRIPTION

The following description is provided to enable any person in the art to make and use the described embodiments. Various modifications, however, will remain readily apparent to those in the art.

FIG. 1is a high-level block diagram of a transcript post processing system100according to some embodiments. An input102(e.g., one or more text transcripts, audio information, and metadata) is provided to a multi-layer speech recognition transcript post processing system150. The multi-layer speech recognition transcript post processing system150converts the input102into a final transcript180that is provided to one or more downstream tasks190(e.g., question answering or machine translation tasks). The multi-layer speech recognition transcript post processing system150may include a data-driven, statistical layer110, and a rule-based layer120, and a machine learning approach layer130. According to some embodiments, an optional a human editor layer140may also be incorporated in the system100(as illustrated by dashed lines inFIG. 1).

A detailed process is provided inFIG. 2which is a flow diagram of a transcript post processing method in accordance with some embodiments. Note that the flowcharts described herein do not imply a fixed order to the steps, and embodiments of the present invention may be practiced in any order that is practicable. Also note that any of the methods described herein may be performed by hardware, software, or any combination of these approaches. For example, a computer-readable storage medium may store thereon instructions that when executed by a machine result in performance according to any of the embodiments described herein.

At S210, a data-driven, statistical layer associated with a trained automatic speech recognition model may select an initial transcript. The data-driven, statistical layer may, for example, select a best initial transcript from a set of N most probable speech recognition transcripts. According to some embodiments, the selection of the best initial transcript is augmented by external attention associated with multiple text documents.

At S220, a rule-based layer may receive the initial transcript and execute at least one pre-determined rule to generate a first modified transcript. The pre-determined rule might be associated with, for example, a white list, a black list, a rule approach, etc. According to some embodiments, the pre-determined rule is automatically generated via offline data mining, data augmentation, and/or model training. The offline data mining might be associated with, for example, supervised classification, unsupervised classification, clustering techniques, n-gram classification, replacement pairs based on context, a graph-based method to link spoken and written sentences based on semantic similarity, search engine data, etc.

At S230, a machine learning approach layer may receive the first modified transcript from the rule-based layer and perform a neural model inference to create a second modified transcript. The machine learning approach layer is might be associated with, for example, online candidate generation, online neural model inference encoding and decoding, and/or online ranking.

At S240, a human editor layer may receive an adjustment to the second modified transcript from at least one human editor. The adjustment may, according to some embodiments, create a final transcript that is used to fine-tune the data-driven, statistical layer. According to some embodiments, the human editor layer is associated with multiple-level human labeling, pairwise human labeling, and/or manual human transcription. The adjustment received at S240might reflect syntactic correctness (associated with set of rules and processes that govern sentence structure), semantic closeness (associated with the meaning of spoken language), fluency, style, etc. In some embodiments, the system100includes a Text-To-Speech (“TTS”) conversion followed by a Speech-To-Text (“STT”) conversion. The final transcript may then be transmitted to a downstream task (e.g., a task associated with language understanding, machine translation, text summarization, text classification, information extraction, and/or question answering).

Note that the output from an automatic speech recognition system may be statistically optimal given constrains associated with acoustic and language models. The system may aim to minimize the Word Error Rate (“WER”) between the transcription and the actual speech. However, there are potential drawbacks to this approach in real applications. For example, statistical models might not capture some commonsense knowledge or prior knowledge. As another example, some tense errors (which are obvious to people) may be difficult for a model to detect/correct. Moreover, WER may not be an appropriate metric for some downstream applications, such as when a transcript reads “Ummm . . . I, I, I think the answer is C.” Although this reflects what was actually said, the information might degrade downstream processing, such as processing associated with machine reading comprehension and/or summarization tasks.

According to some embodiments described herein, a readable automatic speech-to-text transcript may be improved through post processing. This may make the transcript more effective for human users in accomplishing specific tasks and/or improve automatic downstream processes that use these transcripts as an input. The post processing might, for example, be associated with large-scale language model rescoring to correct an unlikely word in view of the context of the surrounding speech (which may have a similar pronunciation as compared to the correct word. The post processing might also be associated with Inverse Text Normalization (“ITN”) that refines the display format of numbers, entities, cases, punctuations, etc. to more like traditional written sentences. Note, however, that such approaches may not consider syntactic and semantic aspects of speech and might also ignore the disfluencies and/or speaking style of a speaker.

Instead of measuring WER, one could measure the readability of speech recognition and machine translation outputs in terms of reaction time and passage comprehension. Such an approach might consider: (1) participants' accuracy rates at answering questions about the content of the text; (2) the time it takes participants to answer questions; (3) the time it takes participants to read the text and the questions; and/or (4) a subjective score that participants assign to the texts.

In some embodiments, the quality of a transcript may use a subjective scoring method for the original transcript and the post processing results. For example, a 5-level labeling system might comprise scores of:1. Perfect—the edited text is understandable with correct meaning, without grammar mistakes, disfluencies, or dialects, and all edits are necessary;2. Excellent—the edited text is understandable with correct meaning, without grammar mistakes, disfluencies, or dialects, but there are some unnecessary edits;3. Good—the edited text is understandable with correct meaning, but there are some grammar mistakes, disfluencies, or dialects;4. Fair—the edited text is understandable with correct meaning in general, except some parts; and5. Bad—the entire edited text is not understandable or has different meaning with the original utterance.
This 5-level system could be simplified to 3 levels (e.g., including Perfect, Good, and Bad) with corresponding guideline adjustment.

Another approach may utilize pairwise ranking of transcript readability. That is, rather than trying to predict the readability of a single transcript, the system may consider pairs of transcripts and predict which one of the two is better. This task may in fact be more natural, since in most applications the main concern is with the relative quality of transcripts (rather than their absolute scores). This approach may also be beneficial in terms of data use, because each pair of transcripts with different average readability scores now becomes a data point for the classification task.

In still another approach, human editors may be asked to directly write down the best edits to the original transcript in terms of human readability. The evaluation may require several reference edits per transcript that are each curated by a different human editor. An extended family of pairwise similarity-based metrics might be used to incorporate consensus between different reference edits for evaluation. Such an approach may provide a natural way to estimate how diversely a group of individuals may phrase the transcript to the same speech. In any approach, the labeling guidelines may let a human editor score the transcript readability based on: (1) syntactic correctness, (2) semantic closeness, (3) fluency, and/or (4) style.

FIG. 3Ais a transcript post processing framework300according to some embodiments. A data-driven, statistical layer310receives an input, such as a set of text transcripts representing speech. The framework300also includes data mining362that receives information from the data-driven, statistical layer310. According to some embodiments, the data-driven, statistical layer310and data mining620may be associated with offline processing (along with data augmentation and/or model training processes).

The framework300further includes a white/black lists and/or rule approach320that receives information from the data mining362and provides information to a machine learning approach330. In some embodiments, the framework300includes a human editor layer340that receives information from the machine learning approach and output a final transcript380that is transmitted to a one or more downstream tasks. According to some embodiments, the white/black lists and/or rule approach320, machine learning approach330, and downstream tasks may be associated with online processing.

Note that some errors are repeatedly produced in speech recognition engines. The framework300may collect these samples and generate white/black lists and rule approach320. Every time these words or phrases are generated by a speech recognition engine, the framework300may execute white lists (e.g., “ASR you” should be replaced with “ASRU”), black lists (“PMA,” “ASR,” “WFST,” etc.), and/or rules and templates. One simple way to create a rule could be to build a confusion matrix, which contains some easily confused word (or phrase) pairs, and simply replace the word by another word, such as “word” vs. “work.” However, the framework300may also need to consider which words should be replaced (and which words should not). Part-Of-Speech (“POS”) tagging information may be one metric that could be considered. Some existing techniques from search engines (e.g., BING from MICROSOFT®) might also be used for this, such as a speller function, a grammar error correction function, a query suggestion function, etc. These tools could help the framework300identify more suitable candidates. Note that some syntactic analysis might also be performed on a sentence basis.

Thus, speech recognition may be followed by downstream tasks, such as language understanding, machine translation, text summarization, text classification, text extraction, search, question answering, etc. Collecting large spoken data corpus for those tasks may be more difficult and time-consuming as compared to plain text content for humans. At the same time, a large amount of text corpora may already exist for those tasks. To measure the impact of transcript readability to those tasks, one approach may be to convert their existing plain text inputs to speech using TTS engines, and then convert the speech back to text transcript using speech recognition engines. A task evaluation may be performed in the normal way.

Table 1 is an example result of using TTS and speech recognition to regenerate Conversational Question and Answer (“CoQA”) data. In particular, the voices of two speakers (A and B) were generated by TTS. The speech recognition result reflects ITN. The WER of the transcripts were 11.05% and 10.70%, respectively. The machine comprehension models were fine-tuned from Bidirectional Encoder Representations from Transformers (“BERT”) large uncased pretrained model with multi-stage and multi-task training. Three models with different training configurations were selected (Models 1, 2, and 3 in Table 1). Table 1 compares the F1 results between using the original text inputs and generated transcripts of those models:

TABLE 1ModelPlain TextA's TranscriptsB's Transcript187.5 (85.8)75.2 (74.9)75.8 (75.4)287.4 (86.2)75.2 (75.1)75.8 (75.5)387.3 (85.7)74.7 (74.4)75.1 (75.0)
The sensitivity of the models to the lack of punctuations was also tested. Note that F1 scores in parenthesis reflect removing the punctuations from the passages. In general, ITN contributed from 2 to 3 points of the F1 result in the last two columns. It is clear that speech recognition introduced a lot of noise (even for high-quality reading style speech) and made a substantial impact to the question answering task. A transcript post processing method might be evaluated in terms of how much this gap can be closed.

Note that speech recognition errors are different from keyboard typing errors. In addition, different speech recognition engines and versions may work differently well on different speech variants. As a result, post processing design may be: (1) specific to speech; (2) general for all engines and versions; and/or (3) specific to one particular engine or version.

The data mining362portion of the framework300may mine candidates that are most unlikely to appear in written language and their replacements. To align with the design requirements above, the following information might be mined: (1) discrepancies between spoken and written languages; (2) common errors including deletion and insertion patterns of speech recognition engines; and/or (3) specific errors including deletion and insertion patterns of the speech recognition engine being used.

Common and domain knowledges such as entities, pronunciation similarities, human summary about differences between writing and speech, may be collected and/or associated with a word vector representation (e.g., Confusion2Vec) to incorporate both acoustic similarity and context during word representation learning. Words with small distances may represent good confusion pairs in terms of both acoustics and syntactics (or semantics). An output associated with data mining362might be used for: (1) a data source of white-/blacklists and rules320which do not need to consider context (or when simple contextual rules are sufficient), (2) a data source for candidate generation, and/or (3) prior knowledge for data augmentation associated with encoding.

FIG. 3Billustrates370various data sources for text mining in accordance with some embodiments. A first axis372of a graph shows sources from informal language environments to more formal language environments. Note that informal language may be associated with conversational speech, improvised speech, simple sentences, etc. Formal language may be associated with official, literary, or academic text, highly edited passages, complex sentence structures, etc. A second axis374of the graph shows sources from spoken language environments to more written language context environments (with a dotted line376dividing those areas). The data sources may include everyday conversations, podcast/radio conversations and other talk shows, movies, television shows, video games, etc. More formal sources of spoken words might include audio books and spoken news reports. Other data mining sources might include text from books and/or news article, personal emails, forum posts, blog entries, etc.

FIG. 3Ccomprises the framework300highlighting the machine learning approach330according to some embodiments. In particular, the machine learning approach330may include a candidate generation component, a neural model inference (with encoding and decoding portions), and a ranking component, which are described in more detail below. For example,FIG. 3Dcomprises the framework300highlighting candidate generation332in accordance with some embodiments. Starting from the n-best of original transcripts, candidates of readable transcripts can be generated based on possible local edits. Local edits might be from two sources, data mining and current speech and the original word-for-word transcript. Specifically, The framework300might consider disfluencies to remove from current utterance, including repetitions, incomplete sentences, corrections, interruptions, etc. Similarly, the framework300might examine lexical, syntactic, and grammatical errors in the current utterance (not smooth or incoherent segments). Unsaid or indirectly implied parts in the current utterance might also be identified (along with special symbols used as replacements) along with probability of mined candidates during data mining (given the current inputs and the context). Note that data mining may focus on aggregated statistics over an entire corpus while in candidate generation a single utterance is the target. The output of all combinations of possible local edits and their probabilities may be used in decoding as the search space.

FIG. 3Ecomprises the framework300highlighting an encoding process336for a neural model inference334according to some embodiments. The encoding process336may, for example, fuse real and augmented data. That is, the encoding336might receive real data, TTS and STT augmented data for specific speech recognition errors, synthesized data based on the prior knowledge from data mining, purely randomly generated data for generalization ability, etc. (with the ration between those sources being selected as appropriate). Note that the purely plain text-based methods aim to introduce the differences between spoken and written languages as well as speech recognition errors. Data mined from data mining can be used as the prior distributions of the ratio, position, and words for the change, but a portion of data should be randomly generated without the prior for well generalization in the future. The TTS based method may be expected to generate more specific recognition errors for certain speech recognition models. Note that disfluency is not the target of the TTS based method.

FIG. 3Fcomprises the framework300highlighting data augmentation364in accordance with some embodiments. In particular, the data augmentation364may provide information to model training (that, in turn, provides data used by the machine learning approach). Recent progress in language modeling has been associated with substantial amounts of automatically labeled training data (which is used to model general linguistic ability). While written language representation has been largely improved, spoken language representation has been less fully explored. One of the reasons for this might be the lack of pairwise data between spoken and written languages. Embodiments described herein might be associated with discovering large amount of unlabeled spoken language corpora and conversation transcripts. Embodiments may also be associated with designing training targets that can be generated automatically and designing neural network architecture to predict appropriate targets. The framework300may also synthesize the spoken language from written language such as by:random text normalization converting numbers and abbreviations;randomly adding disfluencies;randomly converting uppercase to lowercase;randomly changing punctuations;adding grammar errors (e.g., randomly add/remove/change a portion of words or regenerate the text using TTS and STT).

FIG. 3Gcomprises the framework300highlighting a decoding process334for the neural model inference334according to some embodiments. Note that decoding may be performed within the search space described in candidate generation. In addition, several types of inputs might be required during the decoding: (1) encoder output as the representation of original transcript to capture the semantic meaning; (2) large scale pretrained unidirectional and/or bidirectional written language generation models; (3) acoustic representations to capture the pronunciation, semantic meaning, and emotional context such as tone, volume, timbre, etc. Note that context representations may capture who was speaking when and where (as well as the topics and domains). Sometimes knowledge representations may capture common sense knowledges. Moreover, attention among different types of information can help alignment (and a copy mechanism during decoding may be important to prevent unnecessary changes).

FIG. 3Hcomprises the framework300highlighting model training366and ranking338according to some embodiments. To train366the encoder and decoder, pairwise data between an original word-for-word transcript and an expected readable transcript may be required. If the framework300considers other information such as acoustics, data acquisition may become more difficult as human labeling is both expensive and time consuming. As a result, pre-training of the spoken language encoder as well as the encoders of other data types may be important important. This might be performed individually for each data type. Finally, the pretrained encoders and decoder may be fine-tuned together on real labeled transcript post processing data. According to some embodiments, the ranking338may re-rank decoding outputs because some metrics can be introduced more easily as compared to the decoding step. These may include, but are not limited to:an overall readability of current output;overall semantic/pronunciation/edit similarities between current input and output;smoothness and coherence of the context that contains current output; andstyle consistency of the whole transcript.

While some implementations will be described in the general context of program modules that execute in conjunction with an application program that runs on an operating system on a computer, those skilled in the art will recognize that aspects may also be implemented in combination with other program modules. Generally, program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types.

FIGS. 4 through 6and the associated descriptions provide a discussion of a variety of operating environments in which examples are practiced. However, the devices and systems illustrated and discussed with respect toFIGS. 4 through 6are for purposes of example and illustration and are not limiting of a vast number of computing device configurations that are used for practicing aspects, described herein.

FIG. 4is a block diagram illustrating physical components (i.e., hardware) of a computing device400with which examples of the present disclosure may be practiced. In a basic configuration, the computing device400includes at least one processing unit402and a system memory404. According to an aspect, depending on the configuration and type of computing device, the system memory404comprises, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. According to an aspect, the system memory404includes an operating system405and one or more program modules406suitable for running software applications450. According to an aspect, the system memory404includes the multi-level Speech Recognition Transcript Post Processing (“SRTPPS”)150in accordance with any of the embodiments described herein. The operating system405, for example, is suitable for controlling the operation of the computing device400. Furthermore, aspects are practiced in conjunction with a graphics library, other operating systems, or any other application program, and are not limited to any particular application or system. This basic configuration is illustrated inFIG. 4by those components within a dashed line408. According to an aspect, the computing device400has additional features or functionality. For example, according to an aspect, the computing device400includes additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated inFIG. 4by a removable storage device409and a non-removable storage device410.

As stated above, according to an aspect, a number of program modules and data files are stored in the system memory404. While executing on the processing unit402, the program modules406(e.g., MT platform120in accordance with any of the embodiments described herein) perform processes including, but not limited to, one or more of the stages of the method200illustrated inFIG. 2. According to an aspect, other program modules are used in accordance with examples and include applications such as electronic mail and contacts applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided application programs, etc.

According to an aspect, the computing device400has one or more input device(s)412such as a keyboard, a mouse, a pen, a sound input device, a touch input device, etc. The output device(s)414such as a display, speakers, a printer, etc. are also included according to an aspect. The aforementioned devices are examples and others may be used. According to an aspect, the computing device400includes one or more communication connections416allowing communications with other computing devices418. Examples of suitable communication connections416include, but are not limited to, Radio Frequency (“RF”) transmitter, receiver, and/or transceiver circuitry; Universal Serial Bus (“USB”), parallel, and/or serial ports.

The term computer readable media, as used herein, includes computer storage media. Computer storage media include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, or program modules. The system memory404, the removable storage device409, and the non-removable storage device410are all computer storage media examples (i.e., memory storage.) According to an aspect, computer storage media include RAM, ROM, Electrically Erasable Programmable Read-Only Memory (“EEPROM”), flash memory or other memory technology, CD-ROM, Digital Versatile Disks (“DVD”) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other article of manufacture which can be used to store information and which can be accessed by the computing device400. According to an aspect, any such computer storage media are part of the computing device400. Computer storage media do not include a carrier wave or other propagated data signal.

According to an aspect, communication media are embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and include any information delivery media. According to an aspect, the term “modulated data signal” describes a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media.

FIGS. 5A and 5Billustrate a mobile computing device500, for example, a mobile telephone, a smart phone, a tablet personal computer, a laptop computer, and the like, with which aspects may be practiced. With reference toFIG. 5A, an example of a mobile computing device500for implementing the aspects is illustrated. In a basic configuration, the mobile computing device500is a handheld computer having both input elements and output elements. The mobile computing device500typically includes a display505and one or more input buttons510that allow the user to enter information into the mobile computing device500. According to an aspect, the display505of the mobile computing device500functions as an input device (e.g., a touch screen display). If included, an optional side input element515allows further user input. According to an aspect, the side input element515is a rotary switch, a button, or any other type of manual input element. In alternative examples, mobile computing device500incorporates more or fewer input elements. For example, the display505may not be a touch screen in some examples. In alternative examples, the mobile computing device500is a portable phone system, such as a cellular phone. According to an aspect, the mobile computing device500includes an optional keypad535. According to an aspect, the optional keypad535is a physical keypad. According to another aspect, the optional keypad535is a “soft” keypad generated on the touch screen display. In various aspects, the output elements include the display505for showing a Graphical User Interface (“GUI”), a visual indicator520(e.g., a light emitting diode), and/or an audio transducer525(e.g., a speaker). In some examples, the mobile computing device500incorporates a vibration transducer for providing the user with tactile feedback. In yet another example, the mobile computing device500incorporates input and/or output ports, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., an HDMI port) for sending signals to or receiving signals from an external device. In yet another example, the mobile computing device500incorporates peripheral device port540, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., an HDMI port) for sending signals to or receiving signals from an external device.

FIG. 5Bis a block diagram illustrating the architecture of one example of a mobile computing device. That is, the mobile computing device500incorporates a system (i.e., an architecture)502to implement some examples. In one example, the system502is implemented as a “smart phone” capable of running one or more applications (e.g., browser, e-mail, calendaring, contact managers, messaging clients, games, and media clients/players). In some examples, the system502is integrated as a computing device, such as an integrated Personal Digital Assistant (“PDA”) and wireless phone.

According to an aspect, one or more application programs550are loaded into the memory562and run on or in association with the operating system564. Examples of the application programs include phone dialer programs, e-mail programs, Personal Information Management (“PIM”) programs, word processing programs, spreadsheet programs, Internet browser programs, messaging programs, and so forth. According to an aspect, SRTPPS150in accordance with any of the embodiments described herein and/or one or more speech recognition models may be loaded into memory562. The system502also includes a non-volatile storage area568within the memory562. The non-volatile storage area568is used to store persistent information that should not be lost if the system502is powered down. The application programs550may use and store information in the non-volatile storage area568, such as e-mail or other messages used by an e-mail application, and the like. A synchronization application (not shown) also resides on the system502and is programmed to interact with a corresponding synchronization application resident on a host computer to keep the information stored in the non-volatile storage area568synchronized with corresponding information stored at the host computer. As should be appreciated, other applications may be loaded into the memory562and run on the mobile computing device500.

According to an aspect, the system502has a power supply570, which is implemented as one or more batteries. According to an aspect, the power supply570further includes an external power source, such as an AC adapter or a powered docking cradle that supplements or recharges the batteries.

According to an aspect, the system502includes a radio572that performs the function of transmitting and receiving radio frequency communications. The radio572facilitates wireless connectivity between the system502and the “outside world,” via a communications carrier or service provider. Transmissions to and from the radio572are conducted under control of the operating system564. In other words, communications received by the radio572may be disseminated to the application programs550via the operating system564, and vice versa.

According to an aspect, the visual indicator520is used to provide visual notifications and/or an audio interface574is used for producing audible notifications via the audio transducer525. In the illustrated example, the visual indicator520is a Light Emitting Diode (“LED”) and the audio transducer525is a speaker. These devices may be directly coupled to the power supply570so that when activated, they remain on for a duration dictated by the notification mechanism even though the processor560and other components might shut down for conserving battery power. The LED may be programmed to remain on indefinitely until the user takes action to indicate the powered-on status of the device. The audio interface574is used to provide audible signals to and receive audible signals from the user. For example, in addition to being coupled to the audio transducer525, the audio interface574may also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. According to an aspect, the system502further includes a video interface576that enables an operation of an on-board camera530to record still images, video stream, and the like.

According to an aspect, a mobile computing device500implementing the system502has additional features or functionality. For example, the mobile computing device500includes additional data storage devices (removable and/or non-removable) such as, magnetic disks, optical disks, or tape. Such additional storage is illustrated inFIG. 5Bby the non-volatile storage area568.

According to an aspect, data/information generated or captured by the mobile computing device500and stored via the system502are stored locally on the mobile computing device500, as described above. According to another aspect, the data are stored on any number of storage media that are accessible by the device via the radio572or via a wired connection between the mobile computing device500and a separate computing device associated with the mobile computing device500, for example, a server computer in a distributed computing network, such as the Internet. As should be appreciated such data/information are accessible via the mobile computing device500via the radio572or via a distributed computing network. Similarly, according to an aspect, such data/information are readily transferred between computing devices for storage and use according to well-known data/information transfer and storage means, including electronic mail and collaborative data/information sharing systems.

FIG. 6illustrates one example of the architecture of a system for automatic speech recognition as described above. Content developed, interacted with, or edited in association with the SRTPPS150in accordance with any of the embodiments described herein is enabled to be stored in different communication channels or other storage types. For example, various documents may be stored using a directory service622, a web portal624, a mailbox service626, an instant messaging store628, or a social networking site630. The SRTPPS150in accordance with any of the embodiments described herein is operative to use any of these types of systems or the like to perform Automatic Speech Recognition (“ASR”). According to an aspect, a server620provides the SRTPPS150in accordance with any of the embodiments described herein to clients605a,605b,605c. As one example, the server620is a web server providing the SRTPPS150in accordance with any of the embodiments described herein over the web. The server620provides the SRTPPS150in accordance with any of the embodiments described herein over the web to clients605through a network640. By way of example, the client computing device is implemented and embodied in a personal computer605a, a tablet computing device605b, or a mobile computing device605c(e.g., a smart phone), or other computing device. Any of these examples of the client computing device are operable to obtain content from the store616.

An operator or administrator may view and/or adjust parameters associated with a SRTPPS in accordance with any of the embodiments described herein. For example,FIG. 7is speech recognition transcript post processing display700in accordance with some embodiments. The display700includes graphical elements710representing a four-layer transcript post processing system in accordance with embodiments described herein. Selection of various elements710(e.g., via a touchscreen or computer mouse pointer720) may result in a display of additional details about that element710(e.g., via a pop-up window) and/or provide the operator or administrator with a chance to alter or adjust properties of that element710. For example, the operator or administrator might adjust model parameters, training set data, white or black lists, etc.

Some implementations are described herein with reference to block diagrams and/or operational illustrations of methods, systems, and computer program products according to aspects. The functions/acts noted in the blocks may occur out of the order as shown in any flowchart. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved.

Thus, embodiments may provide systems and methods to accurately and efficiently improve automatic speech recognition transcripts. In some embodiments, a four-layer approach improves speech recognition output reaching 100% accuracy. The first layer may comprise a data-driven system that combines the N-best list (N most probable outputs) into 1-best, closely matching the reference. This layer may, in some embodiments, be augmented by a text corpus (e.g., “external attention”) which consists of multiple text documents. The second and third layer may be rule-based where the output of previous layers is fine-tuned. Finally, the last layer may be based on human input. According to some embodiments, the result can then be used as reference for the data-driven layer closing the loop.