Adaptive noise reduction technique for multi-point communication system

A technique for suppressing noise in an audio signal is provided. An audio signal is received from an audio input device. A noise level for the audio signal is determined and continuously updated as the audio signal is received. The audio signal is then attenuated according to the relationship between its current energy level and the current noise level. If the audio signal falls below the current noise level, then a constant, maximum attenuation factor is applied to the signal. If the energy of the signal exceeds the noise level but does not exceed a predetermined maximum energy level that is higher than the noise level, then the audio signal is attenuated based on an exponential attenuation function. If the audio signal exceeds the predetermined maximum energy level, then no attenuation is applied. An attack and decay smoothing function dampens the response time of the attenuated output.

FIELD OF THE INVENTION
 The present invention pertains to the field of telecommunications. More
 particularly, the present invention relates to noise suppression in a
 video conferencing system.
 BACKGROUND OF THE INVENTION
 Video conferencing technology enables the users of two or more people at
 geographically remote locations to have audiovisual communication with
 each other. Video conferencing is currently possible using a conventional
 personal computer (PC) equipped with video conferencing software, a video
 camber, and a connection to a high-speed data link. One video conferencing
 system which permits multi-point video conferencing using conventional PCs
 is the ProShare.TM. Personal Conferencing Video System, which is available
 from Intel Corporation of Santa Clara, Calif.
 A problem associated with almost any communication system is noise. The
 problem of noise is especially significant in multi-point conferences
 (conferences between three or more participants), because the overall
 amount of noise introduced into the system increases as the number of
 participants increases. In particular, noise in the audio channel can
 degrade the quality of the transmitted audio signal as well as cause
 annoyance and ear fatigue to the user. In certain video conferencing
 systems, audio from several local endpoints (e.g., participating PCs) can
 be combined into a single audio stream that is transmitted to other,
 remote endpoints. If the user of any of the local endpoints is not
 speaking, then those endpoints are introducing unnecessary noise into the
 audio stream.
 Certain disadvantages are associated with some existing solutions to the
 noise problem. For example, one approach is to first set a threshold
 volume level, and to then suppress all audio which falls below the
 threshold level and transmit all audio that exceeds the threshold level.
 This approach has been referred to as audio gating. The problem with this
 approach is that audio gating is generally perceivable to the listener as
 unnaturally abrupt transitions between sound and silence as the speaker
 speaks. Often, speech passages are partially cut off, such as when a
 participant is speaking very quietly, or such as in the case of "unvoiced"
 speech (i.e., sounds that involve no vocal chord movement). In addition,
 the ambient noise level at any given endpoint may vary significantly
 during a communication session. However, certain audio gating solutions do
 not adapt to such changes in the noise level. Some solutions, such as
 certain noise cancellation techniques, are computationally complex and
 therefore tend to slow down processing in a local endpoint. As a result,
 such solutions are not well suited to the mixing of multiple audio
 streams. Noise cancellation techniques also tend to cause distortion of
 the speaker's voice.
 Therefore, it would be desirable to have a noise suppression solution which
 improves the overall quality of transmitted audio and which reduces ear
 fatigue. In particular, it would be desirable to have a noise suppression
 solution for a video conferencing system which reduces perceivable gating
 effects and which dynamically adapts to the ambient noise level. It is
 further desirable that such a solution reduce the processing burden on a
 microprocessor and reduce distortion of a speaker's voice.
 SUMMARY OF THE INVENTION
 A method of suppressing noise in a signal is provided. While inputting the
 signal, a noise level based on the signal is repeatedly updated. An
 attenuation function is selected from two or more selectable attenuation
 functions based on the signal and a current state of the noise level. The
 selected attenuation function is then applied to the signal.
 Other features of the present invention will be apparent from the
 accompanying drawings and from the detailed description which follows.

DETAILED DESCRIPTION
 A method and apparatus for suppressing noise in an audio signal are
 described. In the following description, for purposes of explanation,
 numerous specific details are set forth in order to provide a thorough
 understanding of the present invention. It will be evident, however, to
 one skilled in the art that the present invention may be practiced without
 these specific details. In other instances, well-known structures and
 devices are shown in block diagram form in order to avoid unnecessarily
 obscuring the present invention.
 Referring to FIG. 1, the present invention is implemented in a computer
 system 1 having half-duplex audio communication with at least one other
 computer system via an audio channel 95. In one embodiment, the audio
 channel 95 is an Integrated Services Digital Network (ISDN) link. In other
 embodiments, the audio channel 95 may be a standard computer local area
 network (LAN), or a telephone connection. In one embodiment, the computer
 system 1 is a personal computer (PC). The computer system 1 includes a
 central processing unit (CPU) 10, a mass storage device 20, a keyboard 30,
 memory 40, an audio input/output (I/O) subsystem 50, a cursor control
 device 60, a display 70, a video I/O subsystem 80 receiving input from a
 video camera 85, and an interface device 90, such as a modem, providing an
 interface between the computer system 1 and the audio channel 95. The
 audio I/O subsystem 50 is coupled to a speaker 52 and a microphone 53 for
 open audio communication and to a headset 51 having both a speaker and a
 microphone for closed audio communication.
 Memory 40 represents both random access memory (RAM) and read-only memory
 (ROM). The cursor control device 60 may be, for example, a mouse, a
 trackball, a light pen, a stylus with a graphics tablet, or another
 similar device. The mass storage device 20 may be, for example, a magnetic
 disk, CD-ROM, CD-R, Digital Versatile Disk (DVD), or another suitable
 non-volatile data storage device.
 In one embodiment, the present invention is carried out in the computer
 system 1 by the CPU 10 executing sequences of instructions contained in
 memory 40 (e.g., in RAM). More specifically, execution of the sequences of
 instructions contained in memory 40 causes the CPU 10 to perform the steps
 of the present invention, which will be described below. The instructions
 may be loaded into memory from a persistent store, such as mass storage
 device 20, and/or from one or more other computer systems (collectively
 referred to as a "host computer system") over a network. For example, a
 host computer system may transmit a sequence of instructions to a target
 computer system in response to a message transmitted to the host computer
 system over a network by the target computer system. As the target
 computer system receives the instructions via a network connection, such
 as a modem, the computer system stores the instructions in memory. The
 computer system may store the instructions for later execution or execute
 the instructions as they arrive over the network connection.
 In some cases, the downloaded instructions may be directly supported by the
 CPU 10. Consequently, execution of the instructions may be performed
 directly by the CPU 10. In other cases, the instructions may not be
 directly executable by the CPU 10. Under these circumstances, the
 instructions may be executed by causing the CPU 10 to execute an
 interpreter that interprets the instructions, or by causing the CPU 10 to
 execute instructions which convert the received instructions to
 instructions which can be directly executed by the CPU 10.
 In other embodiments, hardwired circuitry may be used in place of, or in
 combination with, software instructions to implement the present
 invention. Thus, the present invention is not limited to any specific
 combination of hardware circuitry and software, nor to any particular
 source for the instructions executed by a computer system.
 FIG. 1B illustrates a configuration by which several PCs can jointly
 participate in a video conferencing session. PCs 1A, 1B and 1C are located
 at one location and are each coupled to a multi-point conference unit
 (MCU) 101 by a separate audio channel 95A, 95B, and 95C, respectively. PCs
 1D, 1E and 1F are located at another (remote) location and are each
 coupled to the MCU 101 by a separate audio channel 95D, 95E, and 95F,
 respectively. The MCU 101 combines the audio streams from the PCs 1A
 through 1F and transmits a combined audio stream to each of the PCs 1A
 through 1F. The MCU 101 can be conventional PCs configured with
 appropriate software.
 FIG. 2 illustrates the data flow associated with one embodiment of the
 present invention. The computer system 1 is configured to include a voice
 activity detector (VAD) receive channel 210, a VAD transmit channel 211,
 and an autocalibrator 230, which may be embodied in software stored in
 memory 40 or in mass storage device 20, in circuitry, or a combination of
 these elements. Compressed audio data is received by the computer system 1
 from the audio channel 95 and then input to decompression unit 220. Signal
 AUDIO RX, which contains decompressed audio data, is then output by
 decompression unit 220 to half-duplex receive channel 200 and to VAD
 receive channel 210. The energy E of the signal AUDIO RX has a waveform
 similar to that illustrated in FIG. 3. In particular, the portion 301 of
 the waveform which exceeds a noise floor NF is considered to be speech
 energy, whereas the portions 302 of the waveform not exceeding the noise
 floor NF are considered to be only noise energy. The VAD receive channel
 210 receives signal AUDIO RX as input and generates an output RXO to
 half-duplex receive channel 200. The output RXO indicates whether or not
 the signal AUDIO RX contains speech at any given point in time.
 In one embodiment, the half-duplex receive channel 200 selectively passes
 on the signal AUDIO RX to audio front-end output circuitry 252, depending
 upon the output RXO of the VAD receive channel 210. Audio data passed on
 to audio front-end (AFE) output circuitry 252 is processed (including
 digital-to-analog conversion) and sent to the speaker 52. In particular,
 referring to FIG. 4A, if the VAD receive channel 210 indicates to the
 half-duplex receive channel 200 that speech is present in the signal AUDIO
 RX in step 401, then the half-duplex receive channel 200 communicates with
 half-duplex transmit channel 201 to cause the microphone 53 to be muted in
 step 402. The microphone 53 remains muted until speech is no longer
 detected in the signal AUDIO RX.
 Referring again to FIG. 2, sound to be transmitted across the audio channel
 95 is input by a user either through the microphone of the headset 51 or
 through the open audio microphone 53 into audio front-end input circuitry
 253, which includes an analog-to-digital (A/D) converter. Circuitry 253
 outputs the digital signal AUDIO TX. The energy E of signal AUDIO TX also
 has a form similar to that depicted in FIG. 3. The signal AUDIO TX is
 provided to VAD transmit channel 211 and to half-duplex transmit channel
 201. Half-duplex channel 201 selectively passes on the signal AUDIO TX to
 compression unit 222 for transmission across the audio channel 95,
 depending upon an input TXO received from the VAD transmit channel 211
 indicating whether or not speech is present in signal AUDIO TX. In
 particular, referring to FIG. 4B, if half-duplex transmit channel 201
 receives an input TXO from VAD transmit channel 211 indicating that speech
 is present in signal AUDIO TX in step 404, then half-duplex transmit
 channel 201 communicates with half-duplex receive channel 200 to cause the
 half-duplex receive channel 200 to mute the speaker 52 in step 405. The
 speaker 52 remains muted until speech is no longer detected in the signal
 AUDIO TX.
 Referring again to FIG. 2, autocalibrator 230 automatically calibrates
 headset 51 in response to a user input entered through a graphical user
 interface (GUI) 240 in a manner which is not dependent upon the particular
 make or model of headset 51. Autocalibrator 230 receives a user input UI
 from the GUI 240 and the signal TXO from the VAD transmit channel 211.
 Autocalibrator 230 outputs a first calibration signal CAL1 to the audio
 front-end input circuitry 253 and a second calibration signal CAL2 to the
 memory 40 and the mass storage device 20. The signal CAL1 is used to
 calibrate the audio front end input circuitry 253, and the signal CAL2 is
 used to store the appropriate hardware settings on the mass storage device
 20 or in the memory 40.
 Although VAD receive channel 210 and VAD transmit channel 211 have thus far
 been illustrated and described separately, they perform essentially
 identical functions. Therefore, VAD receive channel 210 and VAD transmit
 channel 211 are each hereinafter represented interchangeably by VAD 410
 illustrated in FIG. 4C. The VAD 410 receives an input audio signal AUDIN,
 which represents either signal AUDIO RX or signal AUDIO TX, and outputs a
 signal VADOUT, which represents either signal RXO or signal TXO and which
 indicates whether speech is present in the input signal AUDIN.
 Referring now to FIG. 5, a flow chart is shown illustrating the overall
 function of the VAD 410. The function of the VAD 410 consists generally of
 two steps. In step 501, a noise floor NF is established. Next, in step
 502, the VAD 410 determines whether speech is present in the input signal
 AUDIN based upon the relationship of the input signal AUDIN to the noise
 floor NF. In the preferred embodiment, steps 501 and 502 are each repeated
 once every 20 milliseconds (msec). The noise floor NF is an adaptive noise
 floor which is adjusted dynamically as input is received. Thus, the VAD
 410 continuously recomputes the noise floor NF in determining whether
 speech is present in the input signal, as will be described below.
 The noise floor NF is generated based on a noise power density function
 (NPDF), which is created and continuously updated by the VAD 410. The
 energy level of the noise floor NF is based upon a current state of the
 NPDF at any given point and time. FIG. 6 illustrates an NPDF. The noise
 floor NF is taken to be the mean energy value of the NPDF, i.e., the mean
 noise energy level (MNEL), plus a margin value MV. In the preferred
 embodiment, the input signal AUDIN is sampled by the VAD 410 at a rate of
 8 kHz and the NPDF is updated every 20 msec. Consequently, the input
 signal AUDIN is sampled 160 times for every 20 msec time interval.
 To update the NPDF, the VAD 410 uses a measure of the variation of the
 input signal over a period of time as well as the current energy level of
 the input signal at a particular point in time. In one embodiment, the
 measure of variation used is the standard deviation SD of the input signal
 over a time period. In particular, a "sliding window" of time is used in
 gathering samples of the input signal's energy to generate each new value
 of the standard deviation SD. That is, each calculated value of standard
 deviation SD is based upon a sample period which overlaps at least one
 previous sample period, as illustrated in FIG. 9A and as will be further
 discussed below. In one embodiment, a sample period of 500 msec is used to
 generate each standard deviation value SD. This period of 500 msec is
 updated every 20 msec in order to achieve a fast response time of the VAD
 410. Because such short time periods are used, the current energy level E
 is examined in comparison to an envelope of the input signal AUDIN as a
 means of increasing accuracy in updating the noise floor NF, i.e., to
 reduce the number of instances when low standard deviation speech is
 incorrectly interpreted as noise. In one embodiment, the envelope of the
 input signal is an average peak AP of the input signal AUDIN over a
 two-second time window.
 Referring now to FIG. 7, the process of determining and updating the noise
 floor NF (step 501) is illustrated in greater detail. The process consists
 of steps 701 through 707. As noted above, the overall function of the VAD
 410 is a process which is repeated every 20 msec. Consequently, each of
 steps 701 through 705 is performed once every 20 msec. The VAD 410 samples
 the input signal AUDIN at a rate of 8 kH.sub.z, or 160 samples for each 20
 msec iteration. For each sample, the energy level E of the input signal
 AUDIN is determined. In step 701, the average energy E.sub.AVG is
 calculated for all samples occurring during the last 20 msec. The average
 energy E.sub.AVG is also referred to as the "frame energy" of a given 20
 msec frame (interval). In step 702, the standard deviation SD is
 calculated for all of the values of frame energy E.sub.AVG computed during
 the last 500 msec. In step 703, the average peak AP of the input signal
 AUDIN is calculated. In step 704, the VAD makes a preliminary decision as
 to whether the input signal contains noise only or speech. Note, however,
 that this preliminary decision is made only for the purpose of updating
 the noise floor NF and not for the purpose of making a final determination
 of whether speech is present in the input signal AUDIN. In step 705, the
 NPDF is updated if the outcome of the preliminary determination was that
 only noise is present in the input signal (step 704). If it is determined
 that not only noise is present, the NPDF is not updated. In step 706, a
 time decay function is applied to the NPDF to eliminate insignificant data
 points. This step consists of multiplying the entire NPDF curve by a value
 of 0.99990 resulting in approximately a one-half percent per second decay
 in each bin (energy value) of the NPDF. The effect of this time decay is
 that energy values which occur infrequently will eventually disappear from
 the NPDF or impact the NPDF less heavily than those that occur more
 frequently. In step 707, the noise floor NF is calculated as the mean
 energy level of the NPDF plus a margin value MV; that is, the noise floor
 NF equals the mean noise energy level (MNEL) plus the margin value MV (see
 FIG. 6). In one embodiment, the margin value MV is 6 dB, however, this
 value may be tailored to meet desired performance characteristics.
 As mentioned above, the noise floor NF is updated based, in part, on the
 standard deviation SD of samples of the frame energy E.sub.AVG of the
 input signal AUDIN. In particular, during a given time interval, a low
 standard deviation SD usually indicates a lack of speech activity (i.e.,
 noise only) in the input signal AUDIN, assuming the duration of the sample
 window is long enough. By contrast, a high standard deviation in signal
 energy usually indicates that speech activity is present in the input
 signal. The standard deviation SD is computed according to equation (1).
 ##EQU1##
 where Ehd i represents values of frame energy E.sub.AVG.
 A new standard deviation value SD is calculated every 20 msec for the
 purpose of updating the NPDF. The standard deviation SD is calculated for
 all values of frame energy E.sub.AVG occurring within the last 0.5
 seconds. Referring to FIG. 9A, overlapping time intervals T.sub.1 through
 T.sub.4 are examples of four sample windows that are used to generate four
 consecutive standard deviation values, SD.sub.1 through SD.sub.4,
 respectively. Because a new value of standard deviation SD is calculated
 every 20 msec to update the noise floor NF, time intervals T.sub.1 through
 T.sub.4 are offset by increments of 20 msec. This method of calculating
 standard deviation SD differs from one prior art method, illustrated in
 FIG. 8, in which non-overlapping time intervals T.sub.A through T.sub.D
 are used to generate standard deviation values SD.sub.A through SD.sub.D.
 As noted above, the time interval of 500 msec used in one embodiment to
 calculate the standard deviation SD is relatively short, in view of the
 dynamic characteristics of typical human speech. During a given 500 msec
 time period of continuous human speech, the standard deviation SD of the
 signal energy may be quite low and possibly below whatever threshold value
 is being used. As the duration of the sample window for calculating
 standard deviation SD is reduced, the likelihood of misclassifying speech
 as noise tends to increase. This principle is illustrated in FIG. 9B,
 which shows a plot of standard deviation SD over time for the waveform
 shown in FIG. 9A.
 In one embodiment, a standard deviation SD value of 3.2 is used as a
 threshold value in distinguishing speech from noise for the purpose of
 updating the NPDF. In FIGS. 9A and 9B, it can be seen that speech
 occurring during the time interval T.sub.5 might be misclassified as noise
 if one relied only upon the standard deviation SD, since that value falls
 below 3.2 during the time interval T.sub.5. Consequently, the present
 invention does not rely only upon the standard deviation SD of the input
 signal in classifying the input audio signal; instead, the present
 invention also computes an envelope of the input signal AUDIN during every
 20 msec iteration as an additional factor in updating the NPDF. This
 envelope is represented by the average peak AP of the energy of the input
 signal AUDIN, as illustrated in FIG. 10.
 FIG. 11 illustrates how the average peak AP is calculated. In step 1101,
 the last five consecutive frame energy values E.sub.AVG (corresponding to
 the last 100 msec) are saved. These five E.sub.AVG values are then
 averaged in step 1102 to produce a value AVG.sub.5. In step 1103, the
 highest five values of AVG.sub.5 calculated during the last two seconds
 are identified. In step 1104, the average peak AP is calculated to be the
 average of these five highest AVG.sub.5 values.
 Referring again to FIG. 7, a preliminary determination of whether or not
 the input signal includes speech is made in step 704 for the limited
 purpose of updating the NPDF (step 705) to update the noise floor NF (step
 707). As already mentioned, the average peak AP is used, in part, to
 increase accuracy during time periods in which the standard deviation
 value falls below of 3.2 even though speech is occurring. Specifically,
 the input signal AUDIN will not be considered as containing only noise
 unless the current value of frame energy E.sub.AVG falls below the level
 of the current average peak AP minus 9 dB. Hence, an input signal AUDIN
 that has a low standard deviation SD but a high current frame energy
 E.sub.AVG is not likely to be misclassified as noise for the purpose of
 updating the NPDF. In addition, the present invention also employs a
 "zero-crossing" algorithm to further increase the accuracy of the noise
 floor, as discussed below.
 The process of determining whether the input signal contains only noise
 (step 704) and updating the NPDF (step 705) can be summarized as follows.
 The NPDF is updated based upon both the relationship between the current
 frame energy value E.sub.AVG of the input signal to the current average
 peak AP as well as the standard deviation SD of the input signal energy
 over a given time period. Bins of the NPDF are increased by either a high
 confidence value or a low confidence value to reflect the degree of
 confidence that the input signal AUDIN currently contains only noise.
 FIG. 12 illustrates in detail a routine for determining whether the input
 signal contains only noise (step 704) and updating the NPDF (step 705). In
 step 1201, if the current frame energy E.sub.AVG does not fall below the
 level (AP-9 dB), then it is determined in step 1205 that the input signal
 AUDIN is not noise for the purpose of updating the NPDF; in that case, no
 bin of the NPDF is increased. If, however, it is determined in step 1201
 that the current frame energy E.sub.AVG does fall below (AP-9 dB), then a
 determination is made in step 1202 of whether all of the standard
 deviation values SD calculated during the last 120 msec have fallen below
 3.2. If the outcome of step 1202 is "NO", then it is determined in step
 1205 that the input signal AUDIN is not noise for the purpose of updating
 the NPDF, and no bin of the NPDF is increased. If all of the standard
 deviation values SD have fallen below 3.2 for at least the last 120 msec,
 then a determination is made in step 1203 of whether all of the standard
 deviation values SD have fallen below 3.2 for at least the last 1.5
 seconds. If the outcome of step 1203 is "NO", then there is "low
 confidence" that the input signal AUDIN contains only noise. Consequently,
 in step 1206 the appropriate bin of the NPDF is updated by increasing that
 bin by a low confidence value of 0.01. If, however, in step 1203 the
 standard deviation SD has fallen below 3.2 for at least 1.5 seconds (and
 the outcomes of steps 1201 and 1202 was "YES"), then there is "high
 confidence" that the input signal AUDIN is noise only. In that case, the
 appropriate bin of the NPDF is increased by a high confidence value of 0.1
 in step 1204. Note that it is not necessary to use the exact values of
 0.01 as the low confidence value and 0.1 as the high confidence value in
 order to practice the present invention. The important aspect of these
 numbers is that the ratio of the high confidence value to the low
 confidence value is substantially greater than one.
 FIG. 13 shows an example of a waveform of the audio input signal AUDIN and
 the relationships between the sample windows used in calculating the frame
 energy E.sub.AVG, the standard deviation SD, and the average peak AP. In
 FIG. 13, a frame energy value E.sub.AVG is calculated for samples of
 instantaneous energy E occurring within a 20 msec sample window SMPW. A
 standard deviation SD value is also calculated based upon values of
 E.sub.AVG calculated during the 0.5 second standard deviation window SDW.
 In addition, a new value of average peak AP is calculated based upon
 values of E.sub.AVG occurring during the two-second sample window APW.
 This process is repeated every 20 msec, with the sample windows SMPW, SDW,
 and AP being advanced by 20 msec for each repetition.
 The final decision made by the VAD 410 on whether the input signal AUDIN
 contains speech for a given sample period is indicated in the output
 signal VADOUT, which can be used to selectively mute the speaker 52 or the
 microphone 53 during open audio communication, based upon the current
 instantaneous energy E of the input signal relative to the noise floor NF.
 This decision-making process is illustrated in FIG. 14. In step 1401, a
 determination is made of whether the instantaneous energy E of the input
 signal AUDIN exceeds the noise floor NF. If not, then in step 1407 the VAD
 410 makes the preliminary decision that speech is not detected in the
 input signal If the instantaneous energy E exceeds the noise floor NF,
 then the VAD 410 makes the preliminary decision in step 1402 that speech
 is detected in the input signal AUDIN. If speech is detected in step 1402,
 a "zero-crossing" test is applied in step 1403 to determine whether the
 speech is "voiced" or "unvoiced" speech. "Voiced" speech is speech
 produced by vocal chord movement (e.g., the sound of the letter "a"),
 whereas "unvoiced" speech is speech produced without vocal chord movement
 (e.g., the sound of the letters "sh"). The zero-crossing test of step 1403
 is a determination of whether the raw signal value of the signal AUDIN has
 changed sign more than 30 percent of the time during the last 10 msec. The
 outcome of the zero-crossing test (step 1403) is used by the
 autocalibrator 230 to adjust the hardware settings associated with the
 microphone 53. The final decision made by the VAD is "smoothed" by 60
 msec. That is, three consecutive detections of speech corresponding to
 three consecutive 20 msec time intervals must occur in steps 1402 through
 1404 for the VAD 410 to generate an output VADOUT indicating that speech
 is present (steps 1405, and 1406). If the outcome of step 1403 is "no",
 then it is determined in step 1408 that the detected speech is "unvoiced"
 speech. Otherwise, it is determined in step 1404 that the detected speech
 is "voiced" speech.
 Another aspect of the present invention pertains to a technique for
 suppressing noise in the audio channel of a video conferencing system. As
 previously noted, one problem associated with multi-point
 videoconferencing is that noise introduced into the audio signal tends to
 degrade the quality of the transmitted audio signal and cause ear fatigue
 and annoyance to the listener. One potential solution to this problem is
 to apply an audio gating function, an example of which is illustrated in
 FIG. 15. In the gating function of FIG. 15, the audio signal that is to be
 transmitted over the audio channel 95 is completely suppressed until the
 input energy received at the microphone reaches some threshold level
 E.sub.TH. Once the input energy reaches the threshold level E.sub.TH, the
 input audio signal is passed through to the output, without attenuation.
 Hence, in the gating function of FIG. 15, the transmitted volume is at a
 minimum level until the input energy reaches E.sub.TH, at which time the
 transmitted volume begins to coincide with line 310. Line 310 represents a
 normalized set of points at which the transmitted volume directly reflects
 (represents) the energy received at the microphone.
 As noted above, a disadvantage of gating is that a sudden transition
 between two different levels of attenuation is often perceivable and
 distracting to the user. Furthermore, certain existing noise suppression
 solutions do not adapt to changes in the noise level. The present
 invention overcomes these and other disadvantages. In particular, the
 present invention provides smooth transitions in the output volume when
 the input changes from the absence of speech to the presence of speech,
 and vice versa, while dynamically adapting to the noise level, as will now
 be described. Further, because noise suppression in accordance with the
 present invention is not processor-intensive, multiple instantiations of
 the technique can be performed at relatively high speed. Therefore, noise
 suppression in accordance with the present invention is well-suited to the
 mixing of multiple audio streams, such as may be performed during
 multi-point conferencing. Moreover, noise suppression in accordance with
 the present invention does not cause the distortion of voice that is
 associated with certain noise cancellation techniques.
 FIG. 16 illustrates data flow associated with noise suppression according
 to one embodiment of the present invention. Note that although FIG. 16
 illustrates data 15 flow within an audio endpoint, noise suppression can
 also be implemented within an MCU (see FIG. 1B). The signal AUDIO TX that
 is output from audio front-end input circuitry 253 in FIG. 1B is replaced
 in FIG. 16 by signal AUDIO TX_I, which is applied to the input of a noise
 suppressor 260. The noise suppressor 260 outputs a modified signal AUDIO
 TX_O to the half-duplex transmit channel 201. The noise suppressor 260
 also receives a signal MNEL from the VAD transmit channel 211, which
 specifies the current mean noise energy level MNEL. The signal AUDIO TX_O
 represents the signal AUDIO TX_I after application of an attenuation
 function.
 In one embodiment of the present invention, the noise suppressor 260
 attenuates the input signal AUDIO TX_I by applying an approximation of an
 exponential attenuation function to signal AUDIO TX_I within a certain
 range of input signal values. As illustrated in FIG. 20B, the application
 of an attenuation function in accordance with the present invention
 results in an approximately exponential increase, within a limited range
 of input energies, in the transmitted volume as input energy increases.
 The application of an attenuation function in accordance with the present
 invention differs from the gating approach described in connection with
 FIG. 15, which is characterized by an abrupt transition in attenuation
 from full attenuation to no attenuation when the input energy reaches the
 threshold level E.sub.TH.
 Consequently, the present invention provides a smoother transition in the
 volume of the output signal AUDIO TX_O as the volume of the input signal
 AUDIO TX_I changes, in comparison to the gating method. More importantly,
 the present invention provides a smoother transition in the output volume
 when the input changes from the absence of speech to the presence of
 speech, and vice versa.
 The noise suppressor 260 also attenuates the signal AUDIO TX_I as a
 function of the dynamically determined noise level computed by the VAD
 transmit channel 211. More specifically, the signal AUDIO TX_I is
 attenuated at a given point in time according to the relationship between
 its current energy level and the current mean noise energy level, MNEL. In
 addition, the output of noise suppressor 260 is throttled, so that changes
 in the energy level of input signal AUDIO TX_I are not instantaneously
 reflected in the output signal AUDIO TX_O. This throttling function serves
 to further reduce certain perceivable effects that are normally associated
 with gating.
 FIG. 17 illustrates the data flow associated with the noise suppressor 260.
 The noise suppressor computes the amount of attenuation for signal AUDIO
 TX_I every five msec. The noise suppressor 260 includes a frame energy
 estimator 272, an attack and decay throttle 273, an energy mapper 274, a
 log-to-linear converter 275, and an attenuator 276. The attenuator 276
 receives the signal AUDIO TX_I as input and outputs the signal AUDIO TX_O.
 The attenuator 276 is a variable attenuator which attenuates signal AUDIO
 TX_I according to a signal ATTEN.sub.LIN received from log-to-linear
 converter 275.
 Signal AUDIO TX_I is applied as input to frame energy estimator 272. Frame
 energy estimator 272 computes the frame energy E.sub.AVG for each 20 msec
 window. The frame energy E.sub.AVG is provided as output to the attack and
 decay throttle 273. The attack and decay throttle 273 outputs a follower
 energy signal E.sub.FOL, which is an energy value that tracks (follows)
 the current frame energy E.sub.AVG. The value of the follower energy
 signal E.sub.FOL is used by the energy mapper 274 to determine a
 logarithmic attenuation value ATTEN.sub.LOG (specified in dB) every five
 msec. Log-to-linear converter 275 receives the logarithmic attenuation
 value ATTEN.sub.LOG and converts it to a linear attenuation value
 ATTEN.sub.LIN. The value ATTEN.sub.LIN is applied to variable attenuator
 276 to determine the actual attenuation of signal AUDIO TX_I, as noted
 above.
 As will be described in greater detail below, the value of follower energy
 signal E.sub.FOL responds to changes in the frame energy E.sub.AVG by
 moving toward the current value of the frame energy E.sub.AVG (i.e., by
 following the value of the current frame energy E.sub.AVG). The rate at
 which signal E.sub.FOL moves toward the value of E.sub.AVG depends upon
 whether E.sub.AVG exceeds E.sub.FOL (which is referred to as the "attack"
 situation) or E.sub.FOL exceeds E.sub.AVG (which is referred to as the
 "decay" situation). In one embodiment, the follower energy signal
 E.sub.FOL decays (decreases) through a full range in 75.+-.5 msecs,
 whereas the follower energy signal E.sub.FOL attacks (increases) through
 the full range in 20+5 msecs. The follower energy value E.sub.FOL is
 recomputed every five msecs. Therefore, the current output of the attack
 and decay throttle 273 for a given point in time is designated E.sub.FOL
 [n]. The value of E.sub.FOL [n] is computed based upon the previously
 computed value of follower energy, E.sub.FOL [n-1], as will now be
 described.
 The value of E.sub.FOL [n] is computed as E.sub.FOL [n-1] adjusted by a
 scaled error value, which may be positive or negative. The sign of the
 error value at any given point in time depends upon whether the current
 frame energy E.sub.AVG is greater than the previous follower energy value
 E.sub.FOL [n-1], which is defined as the "attack" situation, or less than
 the previous follower energy value E.sub.FOL [n-1], which is defined as
 the "decay" situation. Thus, the current follower energy value E.sub.FOL
 [n] will be computed as the previous follower energy value E.sub.FOL
 [n-1], plus or minus some value, depending upon whether the current
 situation is attack or decay, respectively.
 Before the error value is used to compute the current follower energy value
 E.sub.FOL [n], however, it is scaled by a scaling factor UPDATE_WEIGHT,
 the value of which depends upon whether the current situation is attack or
 decay. The scaling factor UPDATE_WEIGHT is used to control the maximum
 amount by which the signal E.sub.FOL [n] can change from one computation
 to the next. Essentially, the scaling factor UPDATE_WEIGHT is a dampening
 factor applied to E.sub.FOL [n]. UPDATE_WEIGHT is assigned a value
 ATTACK_WEIGHT during an attack situation and a value DECAY_WEIGHT in a
 decay situation. In one embodiment, ATTACK_WEIGHT equals 0.25 to achieve
 the desired attack rate, while DECAY_WEIGHT equals 0.067 to achieve the
 desired decay rate.
 FIG. 18 illustrates the operation of the attack and decay throttle 273. In
 step 1801, a value, ERROR, is taken to be the frame energy E.sub.AVG for
 the current 20 msec window, minus the most-recently computed follower
 energy value E.sub.FOL [n-1]. If the magnitude of ERROR (i.e.,
 .vertline.ERROR.vertline.) is greater than a value, DEAD_ZONE, in step
 1802, and ERROR is a positive value in step 1803, then the signal
 E.sub.FOL is considered to be in the attack situation; consequently, the
 scaling factor UPDATE_WEIGHT is assigned in step 1805 the appropriate
 value for the attack situation (i.e, ATTACK_WEIGHT). If the magnitude of
 ERROR is greater than the value DEAD_ZONE in step 1802, but ERROR is
 non-positive, then in step 1806, UPDATE_WEIGHT is assigned the appropriate
 value for the decay situation (i.e., DECAY_WEIGHT). If the magnitude of
 ERROR is less then or equal to the value DEAD_ZONE in step 1802, then
 UPDATE_WEIGHT is set equal to zero in step 1804. Once UPDATE_WEIGHT is
 computed, then in step 1807 the current follower energy E.sub.FOL [n] is
 computed according to equation (2).
 E.sub.FOL [n] =E.sub.FOL [n-1]+(ERROR*UPDATE_WEIGHT) (2)
 The value DEAD_ZONE is used to provide steady-state stability to the output
 of the attack and decay throttle 273. In particular, the purpose of the
 value DEAD_ZONE is that minor fluctuations in the frame energy value
 E.sub.AVG relative to the current follower energy value E.sub.FOL [n] will
 cause no change in the output E.sub.FOL [n]. The value DEAD_ZONE
 corresponds to a narrow energy range around the current value of signal
 E.sub.FOL [n]. If the frame energy E.sub.AVG falls within this narrow
 range in step 1802 (i.e., if .vertline.ERROR .vertline.&lt;DEAD_ZONE), then
 the value E.sub.FOL [n] is not changed, because there has not been a
 sufficient amount of change in the input signal to cause a change to
 E.sub.FOL [n]. That is, if the magnitude of the value ERROR does not
 exceed the value DEAD_ZONE, it is determined that the follower energy
 signal E.sub.FOL is in neither attack nor decay; hence, UPDATE_WEIGHT will
 be set equal to zero.
 Hence, the attack and decay throttle 273 outputs a follower energy value
 E.sub.FOL, which follows the current frame energy E.sub.AVG. The scaling
 factor UPDATE_WEIGHT determines the variable throttling effect by
 regulating the amount by which the follower energy E.sub.FOL changes for a
 given change in the frame energy E.sub.AVG.
 Referring again to FIG. 17, the follower energy E.sub.FOL is input to the
 energy mapper 274, which maps the follower energy E.sub.FOL to a
 logarithmic attenuation value ATTEN.sub.LOG. This mapping is performed by
 the energy mapper 274 based on the relationship between the current
 follower energy value E.sub.FOL [n] and the mean noise energy level, MNEL.
 Recall that the MNEL is computed dynamically in response to the input
 signal AUDIO TX_I, as described above. In one embodiment, the function of
 the energy mapper 274 is implemented using a look-up table. In particular,
 attenuation values ATTEN.sub.LOG for various input values E.sub.FOL are
 provided in a look-up table and retrieved when needed.
 FIG. 20A illustrates the overall mapping scheme performed by energy mapper
 274 in conjunction with log-to-linear converter 275 and variable
 attenuator 276. The left vertical axis in FIG. 20A specifies values of
 follower energy E.sub.FOL [n], while the right vertical axis specifies
 values of the output signal AUDIO TX_O. E.sub.MAX indicates the limit of
 the dynamic range of the A/D converter within audio front-end circuitry
 253. For a 16-bit A/D converter, the dynamic range (0 to E.sub.MAX) would
 be approximately 96 dB.
 According to one embodiment, maximum attenuation will be applied to the
 input signal AUDIO TX_I when the values of E.sub.FOL falls below MNEL. In
 particular, the value of AUDIO TX_O in such instances will be set equal to
 MNEL minus a predetermined maximum attenuation value, MAX_ATTEN (specified
 in dB). In one embodiment, MAX_ATTEN is set equal to 9 dB, however, the
 value of MAX_ATTEN can be chosen to produce the desired performance
 characteristics. Values such as 6 dB and 12 dB, for example, may also
 produce desirable performance characteristics. Thus, when the signal
 E.sub.FOL equals MNEL, the value of AUDIO TX_O will be set equal to
 MNEL-MAX_ATTEN.
 If the value of E.sub.FOL exceeds a predetermined energy level,
 (MNEL+RANGE), which is greater than MNEL but less than E.sub.MAX, then no
 attenuation is applied to signal AUDIO TX.sub.13 I, such that the value of
 output signal AUDIO TX_O equals the value of input signal AUDIO TX_I. The
 value RANGE in the quantity (MNEL+RANGE) is selected based on the
 signal-to-noise ratio of the input device that is being used (e.g.,
 microphone 53 or headset 51). An example of a value for RANGE that may be
 suitable for one type of microphone is 10 dB.
 For values of E.sub.FOL which fall between MNEL and (MNEL+RANGE), the
 signal AUDIO TX_I is attenuated based on a nonlinear attenuation function.
 The non-linear attenuation function defines exponential variation in
 applied attenuation ATTEN.sub.LOG over the range of E.sub.FOL values
 between MNEL and (MNEL+RANGE).
 Thus, in one embodiment, one of three different attenuation functions is
 applied to signal AUDIO TX_I depending upon the energy level of E.sub.FOL
 relative to the current noise level: 1) if signal E.sub.FOL falls below
 MNEL, then the output ATTEN.sub.LOG of the energy mapper 274 is set equal
 to the maximum attenuation value, MAX_ATTEN, as represented by section 312
 of the plot in FIG. 20B; 2) if EPOL exceeds (MNEL+RANGE), then
 ATTEN.sub.LOG is set to zero (no attenuation is applied), as represented
 by section 314 of the plot in FIG. 20B; and 3) if E.sub.FOL falls between
 MNEL and (MNEL+RANGE), then ATTEN.sub.LOG is computed approximately
 according to curve 313 in FIG. 20B.
 In one embodiment the exponential attenuation function represented by curve
 313 in FIG. 20B is approximated using a piecewise linear attenuation
 function. Specifically, as shown in FIG. 20C, curve 313 can be
 approximated by a number of linear segments 313A, 313B, 313C, and 313D.
 Further, a look-up table can be used to define the segments 313A, 313B,
 313C, and 313D, as indicated above.
 FIG. 19 illustrates a routine performed by energy mapper 274 for mapping
 each value E.sub.FOL [n] to an attenuation value ATTEN.sub.LOG. In step
 1901, a value DELTA is computed as the difference, E.sub.FOL [n] -MNEL. If
 DELTA is negative in step 1902, then the output ATTEN.sub.LOG of energy
 mapper 274 is set equal to the maximum attenuation value, MAX_ATTEN in
 step 1903. If DELTA is not negative and is between 0 dB and RANGE, then
 ATTEN.sub.LOG is determined in step 1905 according to the exponential
 attenuation function (i.e., curve 313 in FIG. 20B). If DELTA is greater
 than RANGE, then the value ATTEN.sub.LOG is set equal to 0 dB in step
 1906. The value ATTEN.sub.LOG is then provided to log-to-linear converter
 275.
 In one embodiment, an additional limiting value E.sub.LIM (see FIG. 20A) is
 used to avoid applying the above-described mapping in environments having
 extremely high ambient noise. Specifically, if the quantity (MNEL+RANGE)
 reaches the limiting value E.sub.LIM, then the value of RANGE in the
 quantity (MNEL+RANGE) is decreased as necessary to prevent the quantity
 (MNEL+RANGE) from exceeding E.sub.LIM. As a result, the range in which the
 (approximated) exponential attenuation function (curve 313) is applicable
 will become smaller if MNEL continues to increase after (MNEL+RANGE)
 reaches the limit E.sub.LIM. If the noise level increases to the point
 where MNEL reaches the limit E.sub.LIM, then either no attenuation can be
 applied or maximum attenuation can be applied, according to the system
 designer's discretion.
 Thus, the noise suppression feature of the present invention provides
 smoother transitions in transmitted volume in response to changes in input
 volume in comparison to certain other approaches and dynamically adapts to
 the ambient noise level at the speaker's location. Perceived gating
 effects are further reduced by attack and decay throttling. As a result,
 speech information is preferentially transmitted while non-speech
 (particularly noise) is preferentially attenuated in generating the output
 audio signal. Further, noise suppression in accordance with the present
 invention is not processor-intensive and therefore facilitates multiple
 instantiations to be performed at high speed, such as when mixing audio
 streams during multi-point conferencing. Moreover, the noise suppression
 feature of the present invention does not cause the distortion of voice
 associated with certain noise cancellation techniques.
 Hence, a method and apparatus for suppressing noise in an audio signal have
 been described. Although the present invention has been described with
 reference to specific exemplary embodiments, it will be evident that
 various modifications and changes may be made to these embodiments without
 departing from the broader spirit and scope of the invention as set forth
 in the claims. Accordingly, the specification and drawings are to be
 regarded in an illustrative rather than a restrictive sense.