Abstract:
A method is disclosed to assist users of telecommunication equipment to improve the likelihood that the party receives and comprehends their speech. The method involves quantifying the quality of speech at both the point of reception and transmission, and then communicating this information to the transmitter who is speaking. To the extent that the background noise at the speaker or listener location is so large as to decrease the signal to noise ratio, or other parameter related to the comprehensibility of speech, to an unacceptable level the speaker may choose to reduce the background noise, or speak loader to increase the signal. Likewise, to the extent that the communication system itself generates unacceptable noise, the speaker may choose to speak loader to increase the signal. However, as the speaker is made aware of the quality of perception at the listener, they need not speaker loader than necessary for the listener to comprehend their speech.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS  
       [0001]     The present application claims priority to the U.S. Provisional Patent Application for a “Telecommunication Device and Method”, having Ser. No. 60/592,179, filed on Jul. 28, 2004, which is incorporated herein by reference. 
     
    
     BACKGROUND OF INVENTION  
       [0002]     The present invention relates to methods of improving the quality of telephonic or radio duplex communications, in particular to communications having marginal comprehension quality due to background noise at the speaker or listener, signal degradation or distortions, and combinations thereof.  
         [0003]     Speech coding is highly deployed in modern communication devices, although originally the evolution was driven specifically by the development of mobile phones. Due to the limited bandwidth, speech coding is a key element to “over air transmissions”. However, the encoded data when sent through the air by radio frequencies will be exposed to a sensitive transmission link, which is very likely to be affected by errors. Such errors may corrupt the transmitted data, and due to the lack of redundancy, it may be difficult if not impossible to reconstruct the speech signal. Due to interaction of speech coding and transmission, errors in a mobile transmission can cause heavy distortions that sound quite different from traditional “analog” distortions, making it difficult for the receiver to understand the information. In fact, the speaker is frequently unaware that the listener is having trouble hearing or understanding such speech, and may be constantly inquiring, “Do you hear mew?” 
         [0004]     Further, the portability of device, such as phones and radios, that permit “over air transmissions” naturally include environments that are substantially noisier than the environments typical in traditional PSTN (Public Switched Telephone Network) or POTS (Plain Old Telephone System) usage. Such noisy environments include public social environments (restaurants, bars), automobile environments, public transportation environments (subways, trains, airports) and other common situations (shopping locations and city/traffic noise).  
         [0005]     However real world use of cell phone is problematic even if system performance is flawless. Portability in effect, guarantees use in marginal environments, background noise pick up at speaker, background noise of listener, tendency to speak loudly, social consequences, confounding of system noise, low confidence of full comprehension  
         [0006]     It is therefore a first object of the present invention to provide a method for the user to select their speaking environment and voice volume in proportion to the listener&#39;s requirements, as effected by their own listening environment as well as degradation from the system quality of service.  
         [0007]     Another object of the invention is to provide one or more speakers with the ability to know if their voice is being heard at a high enough quality for the speaker to comprehend it.  
       SUMMARY OF INVENTION  
       [0008]     In the present invention, the first object is achieved by measuring background noise at the speaker, calculating the signal to noise ratio and indicating it to the speaker.  
         [0009]     Other objectives achieved by measuring background noise at the receiver and transmitting the results of a signal to noise ratio analysis to at least one of the sender and receiver.  
         [0010]     Another object is achieved by the communicating instructions to one or more parties to the conversation that indicate if the other party is likely to comprehend their communications, as well as suggest or signal behavior that would either improve the quality of the received communication such that it can be comprehended, or indicate that the voice may be lowered without detriment to the communication quality, thus avoiding the social consequences of speaking loader than necessary.  
         [0011]     Accordingly, the inventive voice based communication methods and device disclosed herein have a signal processing module operative to receive, analyze and display the speech quality as received or transmitted via a metric such as a signal to noise ratio. The speaker has the option of adjusting their voice level and/or reducing background noise to improve the other party&#39;s quality of reception. Likewise, in a preferred embodiments at least one speaker can also receive a signal indicating the voice or sound quality as received, which in comparison to the quality metric for transmission indicate if the other party is likely to comprehend the speech, or the poor quality is due to transmission related factors independent of the speakers hearing or listening environment. In the latter case, the metric would indicate that transmission quality is unlikely to improve and that the connection should be repeated by other means, including a different time and place.  
         [0012]     A further aspect of the invention is notifying the user of the results. The method of notification may be by visual indicates, such as digital or analog meter, text messages and the like. Other method of notification includes vibration, sounds and patterns of the same.  
         [0013]     The above and other objects, effects, features, and advantages of the present invention will become more apparent from the following description of the embodiments thereof taken in conjunction with the accompanying drawings.  
     
    
     BRIEF DESCRIPTION OF DRAWINGS  
       [0014]      FIG. 1  is a block diagram illustrating the general operative principles and method of the present invention.  
         [0015]      FIG. 2  is schematic diagram illustrating the operative principles in applying the instant invention for the benefit of the user.  
         [0016]      FIG. 3  is another schematic diagram illustrating the operative principles in applying the instant invention for the benefit of the user.  
         [0017]      FIGS. 4A , B and C are timing diagram illustrating various embodiment of notifying the user of the outcome of the processes illustrated in  FIGS. 1, 2  and  3 .  
         [0018]      FIGS. 5 A , B and C are timing diagram illustrating various alternative embodiment of notifying the user of the outcome of the processes illustrated in  FIGS. 1, 2  and  3 .  
         [0019]      FIG. 6  is a timing diagram illustrating an embodiment for transmitting the voice quality parameters, voice signal and optional test signals.  
     
    
     DETAILED DESCRIPTION  
       [0020]     As used in this disclosure, VQP is a figure of merit indicating deviations from perfect speech. Some threshold value of VQP correlates with the listener ability to readily and comfortable comprehend the speaker. Numerous methods have been developed to analyze voice signals to quantify the quality as it relates to final perception of the sound generated there from. Such methods, including the associated hardware and software are in fact in the configuration testing &amp; maintenance of telecommunication systems. The VQP may be determined simply by determining the ratio of signal (voice) to background noise, or SNR for signal to noise ratio, and optionally takes into account the background noise at the listener as well as at the speaker. Mathematically speaking the signal to noise ratio as received is equal to the spoken signal divided by the sum of the noise due to: the speakers background, the listeners background and the system noise, it being understood that the SNR is a transient property having temporal variation. The VQP may also take into account of noise and distortion related to the quality of transmission of the signal, or deploy a combination of parameter. More sophisticated methods of determining a VQP have been primarily developed measure performance of a telecommunication system, or the analog to digital conversion Codec&#39;s used therein, as disclosed in the following U.S. patents, which are incorporated herein by reference: U.S. Pat. Nos. 6,628,453; 6,330,428; 6,418,196; 6,609,092; 6,275,797; 5,987,320, which are incorporated herein by reference.  
         [0021]     In accordance with the present invention,  FIG. 1  illustrates the general operative principles of one embodiment of the present invention. In a conversation between two transceivers, the first transceiver  100  receives a signal arising from sound that includes the user&#39;s voice as detected by a first microphone transducer  110 . The first transceiver processes the signal to determine the Voice Quality Factor of the signal as transmitted (VQF-Tx) as well as the usual function of encoding or digitizing the signal for transmission (as Tx or a portion thereof) according to an ITU or other standard protocol, such as code division multiplexing, time division multiplexing, voice over IP (VoIP) protocol and the like. Tx is received by the second transceiver  200 , and converted to an analog signal that is amplified and broadcast as sound from speaker transducer  280 , with the intention intended to represent the voice detected by microphone  110 . It should be appreciated that the digital signal as transmitted is unlikely to be identical with the digital transmission as received, due to signal, loss and distortion during transmission, and or the application of compression and decompression algorithms due to bandwidth limitations of the communication system. The second transceiver  200  also processes the voice signal to determine a Voice Quality Parameter (VQP) for the signal as received (VQP-Rx). The second transceiver, in addition to transmitting voice detected by its own microphone(s) (not shown) transmits the VQP-Rx as signal Rx, which is then received by first transceiver  100 . The first transceiver performs various functions to compare and analyze the differences between VQP-Tx and VQP-Rx, ultimately communicating information or instruction to the user  170  based on such analysis and comparison.  
         [0022]     However, transmitting both voice signals and characteristics of the speech quality as transmitted, along with other signals depends on the specific embodiment of the invention. As will be further discussed with respect to  FIG. 2  and  FIG. 3 , it is desirable to quantify the difference between VQP-Tx and VQP-Rx as a Total Quality Parameter (TQP), which is communicated and available to both the speaker and listener. Ideally, the TQP as a figure of merit describing and taking into account the speakers listening conditions, which is background noise that would affect their ability to hear incoming voice signals when amplified through the transceivers speaker. The TQP also describes and takes into account the noise and/or distortion characteristics in the transmitted signal as well as the background noise in the speaker&#39;s environment, which is eliminated, but is picked up by the transceivers microphone, amplified and transmitted to the receiver/listener. The receiving transceiver, in the preferred embodiment, is able to analyze the characteristics of the speech quality as received, and then transmit this information back to each speaking party as a TQP.  
         [0023]     A currently favored method of determining a VQP is the PESQ method. PESQ is described in an IEEE publication entitled “Perceptual Evaluation of Speech Quality (PESQ)A New method for speech quality assessment of telephone networks and Codecs”, by A. W. Rix et al., ICASASSP, 7-11 May 2001, which is incorporated herein by reference. PESQ is not used on real speech, but rather known test waveforms (or test waveforms plus real noise); a second point of novelty, more fully described with respect to  FIG. 6 , includes interlacing the test waveforms with the real speech when a mobile or cell phone would not normally transmit. As the cell phone has an internal S/N discriminator to make this decision, it simply transmits test waveforms (or PESQ results) with an appropriate packet header that is used by the receiving phone before releasing the channel back to the network. Of course, PESQ could be continuous if an additional channel is allocated such as instant messaging or picture transmission, but possibly not representative.  
         [0024]      FIG. 2  illustrates operative principles in applying and using the analysis and comparison of VQP-Tx and VQP-Rx. In the ideal situation, the VQP-Tx is very high, that is there is no background noise picked up by microphone  110 . Thus, if no noise addition or signal distortion occurs in converting the signal between various electronic and digital formats in both the first and second transceiver, including the conversion back to sound, then VQP-Rx should be high, and the same as VQP-Tx.  
         [0025]     However, real world communications suffer from background noise picked up microphone  110  (i.e. other conversations in a restaurant, air conditioner and fan noise in a car, or traffic noises), noise and distortion in transmission, as well as the difficulty of listener discerning the sound generated by speaker  280  due to background noise in their environment. At some level of VQPmin. a listener is simply unable to consistently understand what is being said. This value of VQPmin. is indicated by the arrow point to a minimum speaker threshold on the y-axis of the graph in  FIG. 2 . Thus, even if the transmission to the receiver is perfect VQP-Tx must be equal to or above VQPmin. Thus, one aspect of the invention is indicating the speaker or user of the first transceiver  100  is if there speech can be perceived even with perfect transmission. For example, a user of first transceiver  100  in a noisy environment would want to speak loud enough such that their speech signal to noise ratio is increased to improve VQP-Tx to above the minimum level. However, if the user is in a restaurant, they might want to speak not more loudly than necessary so that VQP-Tx is at the minimum level. However, as many communication systems are not perfect, and may vary with the locations of the users, network traffic and other conditions it is ultimately important to know if what the speaker hears is greater than VQPmin. Thus, transmitting VQP-Rx to the speaker provides a means of notifying them that what the listener actually receives and hear is comprehensible. In the most preferred embodiment of the invention, VQP-Rx also takes into account background noise heard by the listen, which can be extracted from the signal picked up by one or more microphones at the second transceiver.  
         [0026]      FIG. 3  illustrates further the operative principle in applying the ratio, R, between VQP-TX and VQP-RX, and re-plotting the graph in  FIG. 2  with VQP-Tx on the x-axis and R on the Y-axis. The ratio R (R=VQP-Rx/VQP-Tx) can never be greater than one, so the x-axis is scaled from zero to one. A horizontal line extending from VQPrnin. on the y-axis defines a region  301  below this line. To the extent that the analysis characterizes the TQP within region  301  communication is not possible as VQP-TX is less than VQPmin.  
         [0027]     It should be appreciated from the foregoing discussion that as VQP-Rx is the critical parameter for the listener greater degradation of signal quality and/or noise from the communication system, represented by a lower value of R, requires a higher value of VQP-TX. However, as VQP-Tm cannot be increased beyond a certain level, thus at some level of R, defined as Rcrit. A vertical line extending downward from Rcrit. defines region  302  to the right of this line. To the extent that the analysis characterizes the total quality parameter (TQP) within region  302  communication is not possible as VQP-TX is less than VQPmin.  
         [0028]     When the TQP is within regional  305 , defined as lying above diagonal line  307 , VQP-Rx is within the comprehensible range. Although regions  301  and  302  overlap it should be appreciated that a sub-portion of region  301  that does not fall within  302 , denoted as  303 , is significant, as it is possible to obtain an acceptable VQP-RX that is moving into region  305 , by first increasing VQP-Tx to some value above VQPmin., that is placing the TQP beyond region  304 , that is crossing line  307  as one increases VQP-Tx.  
         [0029]     Thus, in preferred embodiment, the user is instructed on the options to reach region  305 , based on either having the speaker go to a quieter environment that is decreasing noise, speaking loader to increase signal, or having the listener go to a quieter environment. Therefore, in the most preferred embodiment, both the speaker and listener receive notification and recommendations to improve the TQP such that VQP-Rx is within the comprehensible range.  
         [0030]     However, it should be recognized that at some point, increasing TCP-Tx VQP by the user becomes fruitless as the noise or degradation to VQP by the communication system cannot be overcome, such as when one is one the edge of cell phone coverage area, or transmission is blocked or distorted by building or geography. In one zone, the worst case, the ratio between background noises, arising from the system, to signal sound is so low that no increase in the signal volume improves comprehension, as the signal is in effect distorted by the system.  
         [0031]     In yet another aspect of the invention, algorithms/ methods could be used continuously on the real speech, to interpolate between the test waveform packets. As to commercial acceptance, one issue (among no doubt many) is the typical delay and frequency of update available, which must be within seconds to be of practical use to the consumer. This will depend on the sample waveform length and the amount of time to send the sample waveform in an interlaced format, and process the PESQ and/or other algorithms. A further advantage of the invention in its various embodiments is that it alerts users to the need and opportunity to change equipment speech or system/network quality problems unlikely to be overcome by speaking loader, or changing the speaking or listening environment. For example the VQP information transmitted to either the speaking or receiving party may be used to select a quality of service level from the communication carrier, and hence bandwidth allocation, as taught in U.S. Pat. No. 6,418,196. Alternatively, the user may choose to hold or complete a call using POTS as opposed to VoIP or a mobile communication system.  
         [0032]      FIG. 4  and  FIG. 5  illustrate various alert methods to notify users of the VQP-Rx, and hence alert speaker to marginality &amp; sufficiency of their speech, and thus suggest changes in behavior. Such alert methods include, without limit, vibration and the like.  
         [0033]     In  FIG. 4 , timing diagrams illustrate alternative methods of sound signal indicating voice quality to at least one of the users. Time is plotted on the x-axis, with the signal parameter on the y-axis. In  FIG. 4A , “F” represents the time that a tone, or for visual signals a light, is applied or in the “on” state, that is signal packet  410  comprise a short duration tone  411  followed by a longer duration tone  412 . Signal packet  420  is the opposite, a long duration tone followed by a short duration tone. Signal packet  410  might be used to indicate that the user should increase their voice volume, as VQP-Rx is below a critical threshold corresponding to TQP in region  303  or  304  in  FIG. 3 . In contrast, signal packet  420  might signal the user to optionally decrease their voice volume, as the VQP-Rx is sufficient, with the TQP safely within region  305  in  FIG. 3 . Note that a gap  415  exists between the clusters of tones of difference duration.  
         [0034]     In  FIG. 4B  “freq.” represents the frequency of the sound tone presented to the user. The tones are on for an equal length of time, but varying in frequency to indicate the recommended conduct consistent with the options to move within region  305  in  FIG. 3 . A gap  435  exists between the clusters of tones of difference frequency. The gap may be of fixed duration, or decrease to provide a signal packet as often as necessary to provide a warning, or assurance that the system is functioning and that the listener has received comprehensible sound. Signal packet  430  comprise a low pitch tone  431  followed by a higher pitch tone  432 . Signal packet  440  is the opposite, a higher pitch tone followed by a lower pitch tone. Signal packet  420  might be used to indicate that the user should increase their voice volume, as VQP- Rx is below a critical threshold corresponding to TQP in region  303  or  304  in  FIG. 3 . In contrast, signal packet  430  might signal the user to optionally decrease their voice volume, as the VQP-Rx is sufficient, with the TQP safely within region  305  in  FIG. 3 .  
         [0035]     In  FIG. 4   c,  the “Vol.” represents the volume of the sound in a sequence of indicating pulses heard by the user. The first signal packet  440  comprises 3 tones,  441 ,  442  and  443 , that increase, from  441  to  442 , then decrease in volume, in  443 . In contrast, signal packet  450  comprises three tones that steadily decrease in volume. Thus, signal packet  440  might indicate that the TQP, VQP-Tx or VQP-Rx signal is adequate. The second group  450  of three tones decreasing in volume might be used to indicate that the speaker&#39;s voice could be lowered, whereas the signal packet  460  of three tones of increasing volume might indicate that the speaker should raise their voice volume. In the last signal packet,  470  a lower volume tone sandwiched between two louder tones might indicate that the system noise is too great to overcome by any changes in behavior. Note that a gap  445  exists between the first signal packet  440  and the second signal packet  450 . The spacing of the signal packets, or gap time duration, can be uniform, or increase or decrease to indicate any of the parameters previously mentioned or described herein.  
         [0036]     It should be further understood that any of the diagrams in  FIGS. 4A, 4B  and  4 C can also form a two-dimensional visual display to communicate similar information to that explained herein with respect to  FIG. 2  and  FIG. 3   
         [0037]      FIG. 5A  indicates a similar communication scheme as  FIG. 5  using a vibrator. The vibrator may be an accessory or build into the communication device. Positive values on the y-axis merely illustrates if the vibrator is on, while the x-axis denotes the elapsed time. Thus, in  FIG. 5A , a first signal packet  510  comprises a first short duration vibration  511 , followed by a time gap and longer duration vibration  1512 . After time gap  515 , a second signal packet of vibration pulses is applied. The spacing of the gap  515  can be uniform, or increase or decrease to indicate any of the parameter previously mentioned or described herein. In signal packet the short and long duration vibrations are revered in order, indicating a change in status with respect to the TQP, or instructions to the user, as described with respect to any of the signal packets in  FIG. 4 A , B or C.  
         [0038]     In contrast,  FIG. 5B , signal packets  530  and  540  both comprises the same pattern of vibration  531  and  532 , of short and then longer duration respectively, thus indicating that the TQP status has not changed, and or is within a specific realm corresponding to the option that correspond to the different region in  FIG. 3 , discussed above.  
         [0039]     Independent of displaying the information to the user, as described with respect to  FIG. 4  and  FIG. 5 , TQP/VQP can be determined and transmitted substantially continuously, such as by a parallel channel or path, for example an email or instant messaging function of a mobile telephone or related communication device, or encoded in channels used in call routing, Further, such information can be encoded as extra bits in the digital voice channel.  
         [0040]     Alternatively, the TQP/VQP can be determined either continuously or discontinuously, but transmitted discontinuously by temporal sample coding when the speech is not being transmitted. Such methods of determining the VQP may include or utilize the interlacing of non-intrusive test waveforms, such as test speech patterns used in the PESQ protocol.  
         [0041]     For example, as shown in  FIG. 6 , the timing diagram illustrates one such embodiment for transmitting the TQP/VQP and voice signal.  FIG. 6A  illustrates the speech signal to noise ratio that would normally be determined in the microprocessor of a mobile phone to determine when to transmit speech, labeled  600 . When the SNR drops below a threshold,  602  indicated by the dashed vertical line, voice is usually not transmitted, the channel being released for use by another party. However, the mobile phone, rather than releasing this channel can use the gaps, such as  610 ,  611  and  612 , between speech, can use the gap to transmit VQF-Tx  620 , and optionally a referral text signal  625 . thus  FIG. 6B  shows a corresponding timing for transmitting the digitized speech signal  605 , VQF-Tx  620 , and optionally a referral text signal  625 .  
         [0042]      FIG. 6C  overlays the transmitted signal of the second party,  601 , with the transmitted digital signals that communicate the VQP as received to the speaker using the first transceiver, whose transmission is shown in  FIG. 6B . Thus, during the normal speech block, digitized voice  606  is transmitted, however, during the noise block, when the  
         [0043]     The system of  FIG. 6  is particularly preferred as it allows the exchange of VQP information without creating a new communication protocol in the system, or using additional channel, as the communication device can internally override the system control by transmitting signal such as  620 ,  625  and  630  as if they are voice, with sufficient encryption of start and stop coding such that the receiver does not interpret it as voice, but recognizes that it is VQP information, and processes or transmits it in accordance with  FIG. 1  and the related diagrams in  FIGS. 2 and 3  as necessary.  
         [0044]     Further, part or all of analysis of the signal information to determine the TQP/VQP can occur in either or both telecommunication devices, or reside in part or in whole within the communication network. In other embodiments, the user may be able to choose among a number of methods to determine the TQP/VQP, depending on how frequently they want to be updated during the conversation. Alternatively, the system may select an algorithm most appropriate to the temporal nature and variation in the noise characteristics  
         [0045]     In various embodiments, background noise levels are detected using the microphone, an extra sensor that is coupled to the device for sensing environmental noise, and by indirect measures such as the position of a volume control knob, or an indication of a location of use of the device. It should be understood that the determination and transmission of TQP/VQP between the speaker and listener is optional duplex, and is not limited to the methodology illustrated with  FIGS. 2 and 3 .  
         [0046]     The above invention can be implemented in numerous methods, which for example include providing the integrated function disclosed herein with the microprocessor of a mobile telephone. Alternatively, a user may deploy a module that receives the signal through the auxiliary output jack or split line connection of any phone and merely notify the speaker of the current, as transmitted VQP. In other embodiments, the invention may be implemented as a software routine running on a general-purpose computer to analyze the TQP/VQP of communications using the voice over internet protocol (VoIP) from point to point, in which the signal is digitally transmitted over network, such as the internet, between one or more users via the computer.  
         [0047]     While the invention has been described in connection with a preferred embodiment, it is not intended to limit the scope of the invention to the particular form set forth, but on the contrary, it is intended to cover such alternatives, modifications, and equivalents as may be within the spirit and scope of the invention as defined by the appended claims.