Abstract:
A method and computing system for suppressing noise in an audio signal, comprising: receiving the audio signal at signal processing means; determining that another signal is input to the signal processing means, the input signal resulting from an activity which generates noise in the audio signal; and selectively suppressing noise in the audio signal in dependence on the determination that the input signal is input to the signal processing means to thereby suppress the generated noise in the audio signal.

Description:
RELATED APPLICATION 
       [0001]    This application is a continuation of and claims priority under 35 U.S.C. §120 to U.S. patent application Ser. No. 12/803,295 filed on Jun. 23, 2010 and titled “Noise Suppression,” which in turn claims priority to G.B. Patent Application Serial No. 0919672.6, filed on Nov. 10, 2009 and titled “Noise Suppression” and G.B. Patent Application Serial No. 0920732.5, filed on Nov. 26, 2009 and titled “Noise Suppression,” the disclosures of which are incorporated by reference herein in their entirety. 
     
    
     BACKGROUND 
       [0002]    When a user operates a computer, noises are often generated. For example, when a key on a computer keyboard is pressed there is a short mechanical sound (i.e. a clicking sound). Similarly, when the buttons on a mouse are pressed a clicking sound is produced. 
         [0003]    A microphone of a computer can be used to receive audio signals, such as speech from a user. The user may enter into a call with another user, such as a private call (with just two users in the call) or a conference call (with more than two users in the call). The user&#39;s speech is received at the microphone and is then transmitted over a network to the other user(s) in the call. The audio signals received at the microphone will typically include speech components from the user and also noise from the surrounding environment. In order to improve the quality of the signal, such as for use in the call, it is desirable to suppress the noise in the signal relative to the speech components in the signal. When a user is operating a peripheral device of a computer at the same time as partaking in the call, the noise in the audio signal might include the noise generated by the user&#39;s operation of the peripheral device. For example, clicking noise such as the sound from a key stroke on a keyboard might be picked up by the microphone and included in the signal that is sent to the other participants in the call. The noise (e.g. clicking noise) can be annoying to the other participants in the call and can interfere with their experience of the call. 
         [0004]    One approach for suppressing noise in an audio signal is to use background noise reduction methods. Background noise reduction methods analyse the audio signal in a time and/or frequency domain during periods of speech inactivity (i.e. when the user is not speaking) The background noise reduction methods identify signal components that reduce the perceived quality of speech and attenuate those identified components. Background noise algorithms which can be used in the background noise reduction methods are usually successful in removing stationary noise (e.g. noise comprising a periodic signal and its potential harmonics) from the audio signal. Stationary noise comprises noise components for which the statistical distribution functions do not vary over time. However, background noise algorithms have difficulty in identifying and attenuating transient and non-stationary components of noise, such as clicking noise generated for example from keyboard activity. Clicking noise is a good example of non-stationary noise in that clicking noise fluctuates in time, and any clicking noise generated by a user (such as by typing on a keyboard) is likely to be treated by the background noise algorithm as if it were a speech signal, and therefore would not be attenuated. 
         [0005]    Another approach for suppressing noise from an audio signal is to use specific noise attenuation algorithms for respective specific types of noise, such as keyboard noise attenuation algorithms for attenuating keyboard noise. Keyboard noise attenuation algorithms typically analyse the audio signal received at a microphone to detect and filter out components of the audio signal that are identified as keyboard clicking noise. In this sense, keyboard noise attenuation algorithms comprise two major steps. The first step is detection of the clicking noise in the audio signal and the second step is attenuation of the clicking noise. The detection step can be problematic when the user is engaged in a call because some types of noise such as clicking noise (e.g. keyboard tapping noise) have similar initial characteristics to those of speech, in particular to those of the onset of speech. It is therefore difficult to detect these types of noise in a reliable way and to differentiate between speech and these types of noise without adding a delay and looking for a full click. In the second step of attenuating the noise it is preferred to remove only those components of the signal coming from the noise generating activity (e.g. the keystrokes on the keyboard} while not modifying other components in the audio signal. In particular it is preferable not to modify the speech components of the audio signal when attenuating the clicking noise from the audio signal. However, as described above it can be difficult to detect the difference between some types of noise (such as clicking noise} and the onset of speech, and therefore it is problematic to attenuate those types of noise without distorting the speech components of the audio signal. This problem is compounded by the fact that the onset of speech signals are crucial for the intelligibility of the speech, so any attenuation of the onset of speech can seriously affect the intelligibility of the speech in the audio signal. 
         [0006]    Existing clicking noise attenuation algorithms can be split into two groups. The first group of clicking noise attenuation algorithms are effective in attenuating clicking noise from the audio signal without distorting the speech components of the audio signal to an extent that would be unacceptable to a user. However, the first group of clicking noise attenuation algorithms require data from the future audio signal, such that a delay somewhere around 100 ms is added which makes the use of the clicking noise attenuation algorithms of the first group impractical for use in real time communications, such as a voice call. Any delays added to the audio signal will have a detrimental effect on the user&#39;s perception of the quality of a call or other real time communication. The second group of clicking noise attenuation algorithms do not add a significant delay to the processing of the audio signal, such that clicking noise attenuation algorithms of the second group are suitable for use in real time communications, such as a voice call. However, the algorithms of the second group are not as effective at attenuating clicking noise from the audio signal as are the algorithms of the first group. The algorithms of the second group have a tendency to distort the speech components of the audio signal because they will occasionally mistake speech onsets for a click, such as a tap on the keyboard. 
         [0007]    There is therefore a problem of reliably suppressing noise generated by user activities such as keyboard clicking from an audio signal for use in a real time communication event, without significantly distorting speech components in the audio signal. 
       SUMMARY 
       [0008]    According to a first aspect of the invention there is provided a method of suppressing noise in an audio signal, the method comprising: receiving the audio signal at signal processing means; determining that another signal is input to the signal processing means, the input signal resulting from an activity which generates noise in the audio signal; and selectively suppressing noise in the audio signal in dependence on the determination that the input signal is input to the signal processing means to thereby suppress the generated noise in the audio signal. 
         [0009]    According to a second aspect of the invention there is provided a computing system for suppressing noise in an audio signal, the computing system comprising: receiving means for receiving the audio signal; input means for generating an input signal; signal processing means for determining that the input signal is input from the input means, the input signal resulting from an activity which generates noise in the audio signal; and noise suppressing means for selectively suppressing noise in the audio signal in dependence on the determination that the input signal is input to the signal processing means to thereby suppress the generated noise in the audio signal. 
         [0010]    Noise generated by a user operated device (such as the clicking noise generated by a keyboard or a mouse or other button clicking activity) is suppressed in an audio signal. The operating system of a computer can determine when noise generating activity is carried out on the device other than by detection in the audio signal (e.g. the operating system can determine when the keys of a keyboard are being pressed). The operating system can determine that the noise generating activity is being carried out, without knowing whether the generated noise is picked up by the microphone  120 . For example, if a headset is used, the operating system may determine that keyboard activity is being carried out, but the keyboard noise might be too quiet to be picked up by the microphone  120  in the headset. A notification might be sent from the operating system only when it is determined that noise generating activity is present on the device. The notification can enable or disable techniques for the suppression of the generated noise in the audio signal received at the microphone. In some embodiments, an algorithm for suppressing clicking noise in the audio signal is activated when clicking activity is carried out on a peripheral device, but is deactivated when clicking activity is not carried out on the peripheral device. 
         [0011]    An advantage of the invention is that the noise suppression methods are applied to the audio signal only when noise generating activity is carried out. This means that noise suppression algorithms which can operate in real time can be employed. For example, when no clicking activity is carried out, clicking noise suppression algorithms are not activated so speech components in the audio signal are not distorted by the clicking noise suppression algorithms. In fact no components of the audio signal are distorted by the clicking noise suppression algorithms when no clicking activity is carried out on a peripheral device. 
         [0012]    Distortion of the speech components of the signal arising from misclassified clicking noise detection by a clicking noise suppression algorithm is limited to times at which clicking activity on a device is reported by the operating system. As described above, this can be achieved by only detecting and attenuating the noise generated by the device (such as keyboard noise) when there is noise generating activity on the device (such as keyboard activity) as reported by the operating system. 
         [0013]    Advantageously, input signals from a device to the operating system are used to determine when noise generating activity is present at the device, rather than analysing the audio signal received at the microphone to determine components of the audio signals that are characteristic of the noise generated by the noise generating activity at the device. The input signals from the device are not audio signals. The input signals from the device could be electrical signals, but the input signals could also be transmitted over a wireless connection. A software driver associated with the input device typically detects the input signal and sends a message to the operating system to inform the operating system that the input signal has been detected. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0014]    For a better understanding of the present invention and to show how the same may be put into effect, reference will now be made, by way of example, to the following drawings in which: 
           [0015]      FIG. 1  shows a P2P network built on top of packet-based communication system; 
           [0016]      FIG. 2  shows a schematic view of a user terminal according to a preferred embodiment; and 
           [0017]      FIG. 3  is a flowchart of a process for suppressing noise in an audio signal according to a preferred embodiment. 
       
    
    
     DETAILED DESCRIPTION 
       [0018]    Reference is first made to  FIG. 1 , which illustrates a communication system such as a packet-based P2P communication system. A first user of the communication system (User A  102 ) operates a user terminal  104 , which is shown connected to a network  106 . The communication system  100  utilizes a network such as the Internet. The user terminal  104  may be, for example, a personal computer (“PC”) (including, for example, Windows™, Mac OS™ and Linux™ PCs), a mobile phone, a personal digital assistant (“PDA”), a gaming device or other embedded device able to connect to the network  106 . The user device  104  is arranged to receive information from and output information to a user  102  of the device. The user terminal  104  comprises a microphone  120  for receiving audio signals. In a preferred embodiment the user device  104  comprises a display such as a screen and an input device such as a keyboard  116 , mouse  118 , keypad, joystick and/or touch-screen. The user device  104  is connected to the network  106 . 
         [0019]    The user terminal  104  is running a communication client  108 , provided by the software provider. The communication client  108  is a software program executed on a local processor in the user terminal  104 . 
         [0020]      FIG. 2  illustrates a schematic view of the user terminal  104  on which is executed client  108 . The user terminal  104  comprises a central processing unit (“CPU”)  302 , to which is connected a display  304  such as a screen, input devices such as keyboard  116  and a pointing device such as mouse  118 . The display  304  may comprise a touch screen for inputting data to the CPU  302 . An output audio device  310  (e.g. a speaker) and an input audio device such as microphone  120  are connected to the CPU  302 . The display  304 , keyboard  116 , and mouse  118  are not integrated into the user terminal  104  in preferred embodiments and are connected to the CPU  302  via respective interfaces (such as a USB interface), but in alternative user terminals (such as laptops) the display  304 , the keyboard  116 , the mouse  118 , the output audio device  310  and the microphone  120  may be integrated into the user terminal  104 . The CPU  302  is connected to a network interface  326  such as a modem for communication with the network  106 . The network interface  326  may be integrated into the user terminal  104  as shown in  FIG. 2 . In alternative user terminals the network interface  326  is not integrated into the user terminal  104 . 
         [0021]      FIG. 2  also illustrates an operating system (“OS”)  314  executed on the CPU Running on top of the OS  314  is a software stack  316  for the client  108 . 
         [0022]    The software stack shows a client protocol layer  318 , a client engine layer  320  and a client user interface layer (“UI”)  322 . Each layer is responsible for specific functions. Because each layer usually communicates with two other layers, they are regarded as being arranged in a stack as shown in  FIG. 2 . The operating system  314  manages the hardware resources of the computer and handles data being transmitted to and from the network via the network interface  326 . The client protocol layer  318  of the client software communicates with the operating system  314  and manages the connections over the communication system. Processes requiring higher level processing are passed to the client engine layer  320 . The client engine  320  also communicates with the client user interface layer  322 . The client engine  320  may be arranged to control the client user interface layer  322  to present information to the user via a user interface of the client and to receive information from the user via the user interface. 
         [0023]    The user terminal  104  also includes noise suppressing means  330  connected to the CPU  302 . Although the noise suppressing means  330  is represented in  FIG. 2  as a stand-alone hardware device, the noise suppressing means  330  could be implemented in software. For example the noise suppressing means could be included in the client  108  running on the operating system  314 . As will be described in further detail below, the noise suppressing means  330  is used to suppress noise from an audio signal that is generated by activity on a user operated device, such as keyboard activity on the keyboard  116  or mouse activity on the mouse  118 . The CPU  302  and any device drivers of the input means can be considered to be signal processing means of the user terminal  104 . 
         [0024]    With reference to  FIG. 3  there is now described a process for suppressing noise in an audio signal according to a preferred embodiment. In step S 402  an audio signal is received at the microphone  120  of the user terminal  104 . The audio signal may include speech from User A and may be for use in a communication event, such as a call with User  8  over the network  106 . The audio signal typically also includes noise, such as stationary background noise and non-stationary noise. It is often desirable to suppress (such as by attenuating or removing) the noise from the audio signal such that the quality of the speech in the audio signal is improved. This is particularly desirable where the audio signal is for use in a communication event, such as a call over the network  106  with User B. 
         [0025]    In step S 404  it is determined at the operating system  314  whether input signals have been input at a device (or input means) connected to the CPU such as the keyboard  116  or the mouse  118 . The input signals are not audio signals. The input signals indicate data from the device, for example the input signals may represent key strokes on the keyboard  116 . The input means which inputs the input signals to the CPU  302  is not the microphone  120 , and does not receive audio signals. The input signals are typically caused by activity on an input means connected to the user terminal  104 . Device drivers associated with the input device detect the generation of the input signal and inform the operating system of the input signal. For example keyboard activity on the keyboard  116  will produce input signals to the operating system  314  as the keys are pressed. When the keys on the keyboard  116  are pressed, audible clicking noise will be generated, and this clicking noise may contribute to the noise in the audio signal received at the microphone  120 . Operating systems generally allow software to monitor activity on inputs, such as keyboard activity. One way to allow this is to look for events that are sent out by the operating system. Another way of detecting the input signals is with an Application Programming Interface (API) which allows the state of the input to be accessed, for example the state of each key of the keyboard  116  can be accessed through an API. By using such an API, the noise suppressing means  330  can be informed if a key is pressed. 
         [0026]    In step S 406  it is determined whether noise generating activity is present. In other words, it is determined whether any of the inputs detected in step S 404  will generate noise that may be included in the audio signal received at the microphone. 
         [0027]    If noise generating activity is determined to be present in step S 406  then the method passes to step S 408 . The noise suppressing means  330  then acts to suppress the generated noise from the audio signal. 
         [0028]    The suppression of the noise in step S 408  can be implemented in more than one way. As a first example, the noise suppressing means  330  mutes the audio signal received at the microphone  120  for a predetermined time period t 1  (a muting time period) following the determination that noise generating activity is present in step S 406 . In this way, the generated noise is removed from the audio signal. However, all other components of the audio signal are also removed for the muting time period. This first example is therefore only practical where the muting time period is short and the frequency of noise generating activities is low, such that too much of the audio signal will not be removed. The muting time period t 1  has a duration that is characteristic of the duration of the noise generated by the noise generating activity. For example when the noise generating activity is a key stroke on keyboard  116 , the muting time period t 1  has a duration that is characteristic of the duration of the clicking sound caused by a key stroke. 
         [0029]    As a second example, the audio signal is analysed to detect speech components of the audio signal. The audio signal is not muted within a predetermined period of time t 2  (a speech time period) from the detection of speech components in the audio signal. However, if no speech components have been detected in the audio signal for a time period greater than the speech time period t 2  then the noise suppressing means  330  mutes the audio signal received at the microphone  120  for the muting time period t 1  following the determination that noise generating activity is present in step S 406 . In this way, when User A is speaking into the microphone  120 , the audio signal will not be muted such that the speech components of the signal are not lost. During speech the audio signal is not muted even if a noise generating activity is present as determined in step S 406 . The speech time period is longer than the muting time period (i.e. t 2 &gt;t 1 ) so that when speech is detected and noise generating activity is present the audio signal is not muted. Furthermore, if the audio signal is muted due to the determination that noise generating activity is present in step S 406 , and during the muting time period speech is detected in the audio signal, then the audio signal is unmuted as soon as the speech is detected, i.e. before the expiry of the muting time period. In this way, the detection of speech on the audio signal overrides the muting of the audio signal due to the determination in step S 406  of an input caused by a noise generating activity. 
         [0030]    As a third example, when no noise generating activity is present (as determined in step S 406 ) the noise suppressing means is disabled. In other words when no noise generating activity is detected the noise suppressing means  330  does not attempt to detect and/or remove, filter, subtract or attenuate the type of noise that would be generated by the noise generating activity from the audio signal. For example, when no keyboard activity is detected, the noise suppressing means  330  will not attempt to detect and suppress keyboard tapping noise from the audio signal. However, when noise generating activity is present (as determined in step S 406 ) the noise suppressing means  330  is enabled (e.g. switched on). In other words when noise generating activity is detected the noise suppressing means  330  attempts to detect and remove, filter, subtract or attenuate the type of noise that would be generated by noise generating activity from the audio signal. For example, when keyboard activity is detected, the noise suppressing means  330  will attempt to detect and suppress keyboard tapping noise from the audio signal. In this way, the noise suppressing means  330  is only utilized when the noise generating activity is present, such that when the noise generating activity is not present the speech in the audio signal is not distorted at all by the noise suppressing means  330 . 
         [0031]    As a fourth example, the noise suppressing means  330  is enabled both when noise generating activity is present and when noise generating activity is not present. However, when noise generating activity is determined to be present, the parameters of the noise suppressing means  330  are changed such that the generated noise is suppressed to a greater extent than when noise generating activity is not determined to be present. For example, the method employed by the noise suppressing means  330  aiming to detect and/or remove, filter, subtract or attenuate keyboard noise from the audio signal is adjusted when an input is detected from the keyboard  116 , since it is then more likely to detect keyboard noise in the audio signal. Similarly, when no keyboard activity is detected in step S 404  the noise suppressing means is adjusted such that fewer components in the audio signal are determined to be keyboard noise. This means that fewer speech signals are erroneously determined to be keyboard noise, and therefore fewer speech signals are distorted by the noise suppressing means  330  when no keyboard activity is detected. 
         [0032]    The noise suppressing means  330  uses a noise suppressing algorithm that is capable of suppressing noise in the audio signal in real time. In this way, the audio signals can be used in a real time communication event such as a call over the network  106 . The method may also be applicable in other scenarios and is not limited to use in a call over the network  106 . For example, the method is also suited for use in any other type of communication event in which audio signals are required to be transmitted in real time. The method is also suited for any use in which suppression is required of noise generated by an activity which causes an input to the user terminal. 
         [0033]    There is therefore provided a method and system in which the operating system  314  of a user terminal  104  is used to inform the noise suppressing means  330  if there is a high likelihood for non-stationary noise generated by activity on an input to the user terminal  104 . The noise suppressing means  330  can then take action to suppress the generated noise only when there is a high likelihood of it being in the audio signal received at the microphone  120 . In this way, when no noise generating activity is detected using information provided by the operating system  314 , the noise suppressing means  330  does not attempt to suppress the noise to as great an extent as when noise generating activity is detected. This means that speech in the audio signal will be less distorted when no noise generating activity is detected (as compared to when noise generating activity is detected). Advantageously, the input signals to the operating system are used to determine the presence of the noise generating activity rather than attempting to analyze the audio signal received at the microphone to determine the presence of components in the signal relating to the noise generating activity. 
         [0034]    The signal processing means of the user terminal  104  is used to determine that the input signal is input from the input means. Another input to the signal processing means may be from a fan or hard disk of the user terminal  104  (not shown in the figures). When the fan is switched on it will generate noise which may be picked up by the microphone  120 . Similarly, when the hard disk is operated it will generate noise which may be picked up by the microphone  120 . The signal processing means can use input signals from the fan and the hard disk respectively to determine when the fan and/or the hard disk are in use. In some embodiments the signal processing means can use the input signal from the fan and/or hard disk in same way as an input signal from the keyboard  116  or the mouse  118 . In this way, the noise suppressing means  330  can be applied based on the usage of the fan and/or hard disk. 
         [0035]    While this invention has been particularly shown and described with reference to preferred embodiments, it will be understood to those skilled in the art that various changes in form and detail may be made without departing from the scope of the invention as defined by the appendant claims.