Abstract:
Perceptual quality of a processed signal obtained by processing an original signal having silent periods is evaluated. Silent portions and speech portions of the original signal and corresponding silent portions and speech portions of the processed signal are identified, and the silent portions of the processed signal are evaluated in accordance with a function of amounts of energy contained in the silent portions of the processed signal, corresponding silent portions of the original signal, and an amount of energy in speech portions of the original signal. In one embodiment, the original signal and the processed signal are segmented into frames, frames of the original signal that represent speech and frames of the original signal that represent silence are identified, and the evaluation produces a mean opinion score (MOS).

Description:
BACKGROUND OF THE INVENTION 
     This invention relates generally to methods and apparatus for objective perceptual quality measurement of an audio signal, and more particularly to methods and apparatus for measuring distortions introduced in silent passages by processing of speech signals. 
     Some objective measures of speech signal quality are known. For example, International Telecommunications Union (ITU) standard P.861 for Perceptual Speech Quality Measurement (PSQM) of voice signals is a perceptual objective algorithm for measuring quality of voice signals. This quality measurement is of interest, for example, when compressing and decompressing a voice signal through speech codecs. 
     Known perceptual speech quality measurement algorithms require both an original and a processed signal to be available. For example, PSQM computes a “perceptual difference” between an original and a processed signal to give an objective value that can be mapped to a Mean Opinion Score (MOS). PSQM and other known algorithms operate on active speech portions of the original signal. However, the assumption that only active speech portions contribute to an MOS value is correct only under special conditions. For example, when one attempts to characterize distortion introduced by a new speech compression algorithm, one simply processes an original speech signal through a codec and measures a difference between the original speech signal and the processed signal. There is very little distortion content during silent periods in such processing, resulting in no contribution by such periods to a MOS value. 
     However, when one is attempting to characterize an effect of other types of processors, for example, noise cancelers, distortions introduced during silence periods of speech signals are of considerable interest. It is of interest, for example, to determine whether a noise canceler blocks, removes, or reduces background noise in an original signal. More particularly, effects of noise cancellation are most noticeable during non-active, or silent, portions of a speech signal, as these are the portions in which a background signal annoyance is most readily perceived. Therefore, an unmodified PSQM algorithm does not provide a satisfactory indication of noise cancellation effectiveness in a MOS. 
     It would therefore be desirable to provide methods and apparatus that provide a satisfactory indication of noise cancellation effectiveness. It would further be desirable to provide methods and apparatus that provide a MOS indication of noise cancellation effectiveness. More generally, it would be desirable to provide methods and apparatus for evaluating a measure of MOS for silent periods of any processed speech signal to evaluate the effectiveness and/or usefulness of the processing applied to a speech signal. 
     BRIEF SUMMARY OF THE INVENTION 
     The present invention is therefore, in one aspect, a method for evaluating perceptual quality of a processed signal obtained by processing an original signal having silent periods. The method includes steps of determining silent portions and speech portions of the original signal and corresponding silent portions and speech portions of the processed signal, and evaluating the silent portions of the processed signal as a function of amounts of energy contained in the silent portions of the processed signal, corresponding silent portions of the original signal, and an amount of energy in speech portions of the original signal. In one embodiment, the original signal and the processed signal are segmented into frames, frames of the original signal that represent speech and frames of the original signal that represent silence are identified, and the evaluation produces a mean opinion score (MOS). The present invention is, in another aspect, a corresponding device configured to perform steps of an embodiment of the method, and in another aspect, a machine-readable medium configured to instruct a processor to perform steps of an embodiment of the method. 
     It will be recognized that the present invention, in each of its aspects and embodiments, can be employed to provide measures of noise cancellation effectiveness, and can be used to provide a MOS indication of noise cancellation effectiveness. More generally, the present invention provides evaluations, such as a MOS evaluation, for silent periods of any processed speech signal to evaluate the effectiveness and/or usefulness of the processing applied to a speech signal. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 is a drawing of waveforms representing an original signal and a processed signal in which the signals are offset in the time domain by a difference t. 
     FIG. 2 is a drawing of the waveforms of FIG. 1 aligned in the time domain and segmented into frames. 
     FIG. 3 is a flow chart of an embodiment of a mean opinion score (MOS) procedure. 
     FIG. 4 is a pictorial diagram of a workstation for executing the procedure of FIG.  3 . 
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     In one embodiment and referring to FIG. 1, a mean opinion score (MOS) is desired to evaluate processing performed on an original signal  10  to produce a processed version  12  of original signal  10 . During processing, distortion of a silent portion  14  of original signal  10  results in a noisy portion  16  of processed signal  12 . Original signal  10  and processed version  12  are both available for computing a MOS. However, signals  10 ,  12  are available in a form in which there is an arbitrary time offset t between them. 
     Referring to FIG. 2, when original signal  10  and processed signal  12  are aligned in time with one another and divided into frames F 1 , F 2 , F 3 , F 4 , F 5 , F 6 , and F 7 , their relationship becomes more clear. In the example shown in FIG. 2, frames F 1 , F 2 , F 3 , F 5 , F 6 , and F 7  are frames that correspond to voice or speech portions of original signal  10 . Frame F 4  corresponds to silent portion  14  of original signal  10  and noisy portion  16  of processed signal  12 . 
     FIG. 3 is a flow chart of an embodiment of a method  18  for evaluating MOS for silent periods in a voice or speech signal. Initially, original signal  10  and processed signal  12  are time aligned  20 , eliminating the time difference t shown in FIG.  1 . This alignment can be performed manually or using an algorithm such as ITU P.931. Next, silent portions and speech portions of original signal  10  and corresponding silent portions and speech portions of processed signal  12  are identified. Signals  10  and  12  are divided  22  into corresponding frames as shown in FIG.  2 . Each frame represents an interval having a preselected duration determined by the application and resolution required, for example, a duration suitable for capturing pauses between phrases. In one embodiment, the duration is a duration between 10 to 40 milliseconds, and in another, the duration is a duration between 15 to 20 milliseconds. In one embodiment, signals  10  and  12  are also normalized at this point, although in another embodiment, normalization is part of the overall MOS calculation. For example, an overall global scaling is performed as G_global=sqrt(energy of original signal/energy of processed signal). 
     An initialization  24  is then performed. More specifically, a frame counter is set to examine frame F 1 , and a variable in which an average energy value is stored and updated is set to zero. A loop that executes a series of statements is then entered. 
     Upon entering the loop, a check is performed to determine  26  whether the frame of the original signal  10  represents a speech frame of original signal  10  or a silent frame. In one embodiment, this check is performed manually, for example, by observing a waveform of original signal  10  on a computer display. In another embodiment, automatic detection of speech and silent frames is performed using, for example, an ITU P.56 detector algorithm implementation or a detector such as is used in a European Telecommunications Standards Institute/General System for Mobile Communications/Enhanced Full Rate (ETSI/GSM EFR) speech coder, the latter containing a very sophisticated voice activity detector. If the frame checked is not a silent frame, an update of a running average value of energy per speech frame P av  is calculated  28 . In one embodiment, this update is calculated as P av (new)=(1−x)×P av (old)+x×E 0 , where P av (new) is an updated value of average original signal energy, P av (old) is the previous value of average original signal energy, E 0  is an amount of energy in the present frame of original signal  10 , and x is a parameter selected to provide low pass filtering, 0&lt;x&lt;1. In another embodiment, another method for calculating an average original signal energy P av  is used. After updating  28 , a check is then made to determine  30  whether the frame just checked is the last frame. If so, the procedure terminates  32 . If not, it steps  34  to the next frame. 
     Eventually, a silent frame, for example, frame F 4 , is detected. In one embodiment, an amount of energy in a difference E d  between original signal  10  and processed signal  12  in this frame is computed  36 , according to P av (new)−P av (old) as is an amount of energy E 0  in this frame of original signal  10 . Using the values of E 0 , E d , and P av , a measure of signal-to-noise ratio (SNR) for the current frame is computed  38 , for example, as SNR=10.0×log(original signal energy/processed signal energy)=10.0×log(E 0 /E d ). The computed SNR value is then converted  40  into a MOS value. This conversion is performed in one embodiment by a table mapping, but in another embodiment, it is adaptively performed, i.e., the mapping has memory and therefore is dependent upon, for example, prior values of SNR and/or MOS. In yet another embodiment, conversion  40  is performed using an empirical expression or formula. The value of MOS is displayed on a computer screen as it is calculated. Each frame F 1 , F 2 , F 3  . . . is associated with a MOS value. For silent frames such as F 3 , a MOS value is generated as described above. For speech frames such as F 1  and F 2 , a MOS value is generated  41  using, for example, ITU P.861 PSQM. In one embodiment, a final MOS value is determined as a combination of the MOS values of all of the frames, for example, an average or a weighted average of MOS values. 
     In one embodiment, SNR computations are improved by explicitly taking into account characteristics of noise within a frame, such as its statistical characteristics. A particular mapping of SNR values into MOS values is then selected, depending upon a type of distortion determined to exist in processed signal  12 . 
     If the frame is determined  30  not to be the last frame, the procedure steps  34  to the next frame. Otherwise, the procedure terminates  32 . 
     In one embodiment, MOS procedure  18  is performed using a suitably programmed personal computer or workstation  42  comprising a system unit  44  having a processor (not shown), a computer display  46 , and input devices such as a keyboard  48  and a mouse  50 . A program including MOS procedure  18  is provided on computer readable media. For example, a floppy diskette (not shown) is read by a disk drive  52  of computer  44 . The floppy diskette has recorded thereon signals representative of processor instructions to execute MOS procedure  18 . 
     In another embodiment, workstation  42  is programmed in a different manner, for example, as a dedicated workstation containing the procedure in firmware, or as a diskless network workstation, relying upon a remote server (not shown) for programming. In one embodiment, the program including MOS procedure  18  includes various interface enhancements to provide convenient user control via computer in keyboard  48  and/or mouse  50 . For example, graphical representations of original signal  10  and processed signal  12  are displayed simultaneously on computer display  46  in distinctive colors and manipulated on display  46  by the user, using keyboard  48  and/or mouse  50 . The user correlates signals  10  and  12  in the time domain to manually align data corresponding to signals  10  and  12 . 
     In another embodiment not illustrated in FIG. 4, MOS procedure  18  is embedded as firmware or hardware of a special purpose signal processor operating in real time on original signal  10  and processed signal  12 . Time alignment of signals is not necessary as a separate step when original signal  10  and processed signal  12  are provided simultaneously without significant differential delay, and when the special purpose signal processor is sufficiently powerful to process MOS measurements in real time, as the signals are received. Those skilled in the art will recognize that embodiments utilizing linear, rather than digital, signal processing are possible. 
     For economy of expression, the terms “original signal” and “processed signal” are used extensively herein. However, it is to be understood that these terms are also intended to encompass representations of an original signal and a processed signal, respectively. Similarly, where reference is made to other signals, such references are also intended to encompass representations of such other signals. Representations of signals are intended to include analog and digital representations, unless otherwise noted. 
     From the preceding description of various embodiments of the present invention, it is evident that the present invention, in each of its aspects and embodiments, can be employed to provide measures of noise cancellation effectiveness, and can be used to provide a MOS indication of noise cancellation effectiveness. More generally, the present invention provides evaluations, such as a MOS evaluation, for silent periods of any processed speech signal to evaluate the effectiveness and/or usefulness of the processing applied to a speech signal. 
     Although the invention has been described and illustrated in detail, it is to be clearly understood that the same is intended by way of illustration and example only and is not to be taken by way of limitation. Accordingly the spirit and scope of the invention are to be limited only by the terms of the appended claims and their equivalents.