Event driven noise cancellation

Systems and methods for noise cancellation are provided. In particular, information regarding events associated with a computer providing a telephony function or associated with a hardware telephone providing a telephony function is provided to a noise cancellation process. The provided information can be used to identify specific noises associated with events, so that filters specifically adapted for removing an identified noise can be applied. Alternatively or in addition, information regarding the existence and timing of an event can be provided in order to facilitate the application of a noise cancellation filter. Noise cancellation as described can also be applied in connection with recording speech or other audio signals.

FIELD

The disclosed invention is related to detecting an event in an environment and providing the information thus obtained as an input to a noise cancellation process.

BACKGROUND

Voice telephony continues to be a primary means of communication. In order to facilitate effective voice communications, it is desirable to filter noise from the voice signal.

In order to perform noise cancellation, various noise cancellation algorithms are available. However, conventional noise processing algorithms have difficulty filtering out noises created in connection of the use of a computer, such as computer generated dings or tones, keyboard clicks, mouse clicks or output from a computer soundcard. Accordingly, conventional techniques for noise cancellation have not been particularly effective in filtering common office environment noises, such as noises generated by or associated with the operation of a computer from voice streams.

Common noise cancellation algorithms operate by estimating noise in the environment. The voice stream including noise is analyzed and the noise component is estimated and then subtracted from the outgoing voice stream. In order to improve the accuracy of such noise cancellation processes, pattern recognition may be used to estimate noise in a signal. The pattern recognition model may be trained in order to increase the amount of available pattern recognition data.

Techniques for reducing noise introduced to audio signals by computers have been proposed. For example, systems that provide a number of microphones to detect noise associated with a personal computer, in addition to a microphone external to the computer for capturing sound from the user, have been discussed. Any sound generated by the personal computer that is captured by the external microphone is subtracted from the users voice signal, by subtracting sound captured by microphones internal to the personal computer (or closely associate with the personal computer) from the signal comprising the voice information. Accordingly, such systems rely on a number of microphones.

SUMMARY

The present invention is directed to solving these and other problems and disadvantages of the prior art. In accordance with embodiments of the present invention, computer events that are associated with a noise are monitored within the computer, in order to select a noise cancellation procedure or algorithm to be applied. In particular, electronic signals within the computer are monitored to detect events that create or are associated with noise. Information identifying the event is provided to the noise cancellation algorithm, to enable the noise cancellation algorithm to select and apply the noise cancellation procedure that is most applicable to the detected event.

In accordance with embodiments of the present invention, the detection of computer events associated with the noise can be performed in connection with the use of soft telephones running on the computer on which event detection is performed. In accordance with other embodiments of the present invention, the detection of events associated with the noise can be performed in connection with a computer that is interconnected to a separate telephone or soft telephone. Furthermore, a noise cancellation process that receives input regarding detected events can be executed by the computer or computers on which monitoring of events associated with a noise is performed, on a separate computer, or on a hardware telephone.

In other embodiments of the present invention the detection of events associated with the noise can be performed in conjunction with the use of a speech recording or speech dictation system, software or device.

Additional features and advantages of the present invention will become more readily apparent from the following discussion, particularly when taken together with the accompanying drawings.

DETAILED DESCRIPTION

The present invention is generally directed to noise cancellation or reduction in communication systems. More particularly, embodiments of the present invention monitor events associated with a computer to provide a noise cancellation algorithm or process with information that can be used to improve the results obtained by a filter applied by the noise cancellation process. This information can include notification regarding the existence of an event, the timing of an event and/or can identify a noise associated with an event. The noise cancellation process can then use the provided information to determine where in a voice stream a noisy event is located, when to apply a filter, and/or the selection of a particular filter for canceling a noise associated with a detected event.

With reference toFIG. 1A, a communication system100in accordance with embodiments of the present invention is illustrated. In general, the communication system100includes a first communication endpoint104interconnected to a second communication endpoint108via a communication network112.

The first communication endpoint104generally includes an audio input/output116interconnected to a telephone120. Examples of an audio input/output116include a telephone handset, headset or a far talk microphone and speaker provided as part of a speakerphone function or addition to the telephone120. The telephone120may comprise a conventional telephone for connection to continuous circuit networks, an Internet Protocol (IP) telephone, a telephone associated with a private branch exchange (PBX), a video telephone or any other hardware device capable of supporting voice communications. In the example communication system100illustrated inFIG. 1A, the telephone120is shown as being directly connected to the communication network112. However, as can be appreciated in one of skill in the art, other configurations are possible. For example, the telephone120may be interconnected to the communication network112through an intermediate switch or communication server.

The first communication endpoint104additionally includes a computer124. The computer124may comprise a general purpose computer or other electronic device. Accordingly, an example of a computer124is a desk top or laptop computer in the same office or cubicle as the telephone120. The computer124is interconnected to the telephone120via a signal line128. In general, the signal line128supports the exchange of information between the telephone120and the computer124, and therefore supports at least one protocol that can be used by both the telephone120and the computer124to communicate with one another. Examples of such connections include Ethernet, wireless protocols such as Bluetooth, computer telephony integration (CTI), universal serial bus (USB) or other communication protocols. As will be described in greater detail elsewhere herein, information exchanged between the telephone120and the computer124depends on where various processes in accordance with embodiments with the present invention are executed. For example, where noise cancellation is executed by a noise cancellation algorithm or process running on the telephone120, the computer124may provide information identifying computer events associated with noise to the telephone120across the signal line128. In accordance with other embodiments, such as where a noise cancellation process is executed on the computer124, voice information may be provided by the telephone120to the computer124across the signal line128for processing in the computer124, and the computer124may then return a processed signal to the telephone120for delivery to the communication network112. In accordance with still other embodiments, for example where the telephone120is interconnected to the communication network112through the computer124, and where noise cancellation is performed by the computer124, an audible signal processed by the computer124is passed directly to the communication network112.

The communication network112may comprise a packet data network, such as a local area network (LAN) and/or a wide area network (WAN). Alternatively or in addition, the communication network112may comprise a dedicated connection between two or more communication endpoints104,108. In accordance with still other embodiments of the present invention, the communication network112may comprise a continuous circuit based communication network, such as the public switched telephony network (PSTN) or a wireless network. Furthermore, the communication network112may include a number of different network types in various configurations.

The second communication endpoint108may comprise any type of communication endpoint capable of supporting voice communications. Accordingly, the second communication endpoint108may be configured like the first communication endpoint104in order to provide enhanced noise cancellation features. Alternatively, the second communication endpoint108may be a conventional telephony device. In accordance with still other embodiments of the present invention, the second communication endpoint108may comprise a recording device. Exemplary recording devices include voice mail systems, or systems adapted for storing audible information for later listening and/or distribution.

With reference toFIG. 1B, a system100in accordance with other embodiments of the present invention is illustrated. In general, the communication system100illustrated inFIG. 1Bis similar to the communication system100illustrated inFIG. 1A. For example, the communication network112and/or the second communication endpoint108may be the same as or similar to corresponding components provided as part of the communication system100illustrated inFIG. 1A. However, the first communication endpoint104is different. In particular, the communication endpoint104inFIG. 1Bfeatures a soft telephone132implemented by or in connection with a computer124interconnected to an audio input/output116. Therefore, such embodiments do not include a hardware telephone that is separate from a computer124. Accordingly, the computer124in such embodiments maybe connected to the communication network directly. Alternatively, the computer124may be interconnected to the communication network112through a switch or communication server.

In accordance with still other embodiments of the present invention, as illustrated inFIGS. 1A and 1B, a communication endpoint104may be associated with a speech recorder136, such as a dictation device. If provided, a speech recorder136may be in addition to a telephone120or a soft telephone132, or may be an alternative audio device to a telephone120or a soft telephone132. In addition, a speech recorder136can be interconnected to the computer124(FIG. 1A) or implemented by the computer124(FIG. 1B).

With reference toFIG. 2A, a communication endpoint104in accordance with embodiments of the present invention incorporating a hardware telephone (e.g. as illustrated inFIG. 1A) is shown in greater detail. The audio input/output116generally includes a microphone204for receiving voice information from a user. As can be appreciated by one of skill in the art, the microphone204will also receive noise from the ambient environment, including noise associated with computer124events. The audio input/output116also generally includes a speaker208for providing voice information from other communication endpoints or other audio sources to the user. The audio input/output116is interconnected to the telephone120, for example as a handset provided as part of the telephone120or a headset interconnected to the telephone120. Furthermore, the audio input/output116may be provided as part of the telephone120, for example as a microphone and a speaker208integral to the body of a telephone120providing a native speakerphone functionality. If the communication endpoint104includes a speech recorder136, it may receive an audio input from the audio input/output, and may be in communication with the computer124.

The telephone120generally comprises a hardware telephone. For example, the telephone120may comprise an IP or switched circuit telephone interconnected to a communication network,112, either directly or through one or more intermediate devices. In addition, the telephone120may include a processor for executing a noise cancellation process210. The telephone120may also include memory for storing instructions executed by the processor in connection with operation of the noise cancellation process210.

The speech recorder136may include any device for recording an audio signal, such as a dictation device. Furthermore, the speech recorder136may use analog or digital storage, such as memory216or data storage228, or some other data storage to store recorded speech or other audio signals. The speech recorder136may also include a processor and/or hardwired logic to perform functions of the speech recorder136, which may include execution of a noise cancellation process210. In addition, although embodiments of the present invention may include a speech recorded136that is primarily used to record a user's speech, it should be appreciated that other embodiments of the present invention are not so limited. For example, a speech recorder136can be used to record any sound in an environment.

The computer124may include a processor212capable of executing program instructions. Accordingly, the processor212may include any general purpose programmable processor or controller for executing application programming. Alternatively, the processor212may comprise a specially configured application specific integrated circuit (ASIC). The processor212generally functions to run programming code implementing various other functions performed by the computer124and the communication endpoint104. The computer124may additionally include memory216for use in execution of the programming by the processor212and for the temporary or long term storage of data or program instructions. Memory216may comprise solid state memory resident, removable or remote in nature, such as DRAM or SDRAM. Where the processor212comprises a controller, the memory216may be integral to the processor212.

In addition, various user input devices220and user output devices224may be provided. Examples of input devices of220include a keyboard, a numeric keypad, and a pointing device combined with a screen or other position encoder. As can be appreciated by one of skill in the art, the operation of user input devices220by a user can result in or be associated with a noise. For instance, operation of a keyboard or a mouse typically results in a clicking sound. In addition, as a result of user input provided through a user input device220, sounds may be generated by a user output device224. Examples of user output devices224include a sound card in combination with a speaker, or a tone generator. Other examples of user output devices224include peripheral devices, such as a printer. As can also be appreciated by one of skill in the art, operation of such user output devices224can be associated with a noise. These noises can be objectionable, particularly when they are picked up by a microphone204during voice communication with a second communication endpoint108. Other sources of noise in or associated with the computer124include the operation of associated components such as data storage devices228comprising hard disk drives, floppy disc drives, tape drives, optical drives, or other storage devices having moving components, a cooling fan232, or any other device interconnected to or included in a computer124that is associated with a noise that is related to a computer event.

The data storage228generally provides for the storage of application program and/or data. For example, operating system software236may be stored in the data storage228. Examples of applications that may be stored in the data storage228include an application comprising an event detection process240. The event detection process240generally functions to detect events within the computer124, in order to report the occurrence of such events to the telephone120to facilitate the cancellation of the noise associated with detected events by the noise cancellation process210. For example, the event detection process240may monitor signals on one or more internal buses244provided as part of the computer124to detect events (i.e. particular signals associated with events) that are or may be associated with a noise. Information associated with or identifying events detected by the event detection process may be provided to the noise cancellation process210running on the telephone120via a peripheral interface248and signal line128interconnecting the telephone122and the computer124.

With reference toFIG. 2B, a first communication endpoint104in accordance with other embodiments of the present invention (e.g. as illustrated inFIG. 1B) is depicted. In general, such embodiments do not include a telephone that is separate from the computer124. Instead, such embodiments generally include application software comprising a telephony process252that may be stored in the data storage228and executed by the processor212of the computer124. In addition, the noise cancellation process or algorithm210may also comprise an application stored in the data storage228and executed by the processor212of the computer124. The event detection process240may also be stored in the data storage228and executed by the processor212of the computer124. A speech recorder process254for implementing a speech recorder136function may also be stored in the data storage228and executed by the processor212of the computer124. The computer124is also interconnected to an audio input/output116through a peripheral interface248that provides a microphone204and speaker208to support voice communications by a user. As can be appreciated by one of skill in the art, the microphone204and speaker208may be provided as part of an audio input/output116comprising a handset or speakerphone module. As also can be appreciated by one of skill in the art, the microphone204and speaker208may be separately provided. For instance, the microphone204may comprise a far talk microphone and the speaker208may comprise one or more speakers, interconnected to a sound card provided as part of an audio output system of the computer124. According to embodiments of a first communication endpoint104such as illustrated inFIG. 2B, the event detection process240running on the computer124provides notification of detected events to the noise cancellation process210also running on the computer124. Furthermore, the noise cancellation process210may perform a noise cancellation before releasing an audio stream to the telephony process252for transmission to a second communication endpoint108via a communication network interface256interconnected to the communication network112.

Although particular arrangements within a first communication endpoint104are illustrated inFIGS. 2A and 2B, other arrangements are possible. For example, a noise cancellation process210running on the computer124may be used in connection with a hardware telephone120that provides the voice information to the computer124for processing on that computer124. In accordance with other embodiments of the present invention, a telephone120interconnected to the internal bus244of the computer124may store and execute the event detection process240. Similarly, a speech recorder124may store and execute the event detection process.

With reference now toFIG. 3, aspects of the operation of embodiments of the present invention are illustrated. Although the description related toFIG. 3is related to telephony applications in which voice audio signals are transmitted between a number of communication endpoints104,108it should be appreciated that the described process and alternative embodiments thereof is also applicable to recording audio signals. Initially, after starting the system, a determination is made as to whether audio communications have been enabled (step304). If audio communications have not been enabled (e.g. a telephone call has not been placed) the process may idle at step304. After determining that audio communications have been enabled, a determination may be made as to whether a signal associated with an event on the computer124has been detected (step308). If a signal associated with an event is not detected, audio information is provided for transmission to a second communication endpoint108, without applying noise cancellation specifically targeted to a detected event (step312). The process may then return to step304to determine whether audio communications remain enabled.

If a signal associated with an event is detected at step308, an input notifying the noise cancellation process210of the event is provided by the event detection process240(step316). In general, a signal associated with an event may comprise a signal on an internal bus244of the computer124. Furthermore, the event detection process240need not monitor all data transported by an internal bus244. For instance, an event detection process240may monitor signals associated with specific peripheral devices, such as user input devices220or peripheral user output devices244, the operation of which are associated with the generation of noise. Furthermore, the input notifying the noise cancellation process210of the event may comprise an identification of the event, the time at which the event occurred, and/or a specific identification of the detected event.

At step320, a determination is made as to whether the detected event is correlated to a noise having a known sound pattern. In accordance with embodiments of the present invention, the determination as to whether a detected event can be correlated to a noise with a known sound pattern is performed by the noise cancellation process210in response to information identifying the event provided by the event detection process240. Alternatively, correlation of an event to a noise with a known sound pattern can be performed as part of the event detection process240, and provided to the noise cancellation process210as part of the notification of the event.

If the detected event is correlated with a known sound pattern, the noise cancellation process210applies a filter specifically adapted to the known sound pattern associated with the detected event to the audio signal received from the microphone204of the audio input/output116(step324). That is, based on the identification of the sound associated with the detected event, the noise cancellation process210can select a filter specifically adapted for filtering out the associated noise. As can be appreciated by one of skill in the art, the filter applied in connection with the cancellation of the specifically identified noise can comprise applying an inverse representation of that noise to the audio input signal in order to subtract that noise from the audio input signal. Furthermore, the information identifying the detected event can include the time at which the event occurred, to facilitate locating the noise associated with the detected event in the stream of audio data.

If no correlation between the detected event and a noise having a known sound pattern is made at step320, information can be provided regarding the time of the event to assist the noise cancellation process210in canceling noise associated with the event from the desired audio information. For example, the noise cancellation process210may apply a noise cancellation filter to the received audio stream over a period of time encompassing the event or the time at which the noise associated with the event is expected to occur. The noise cancellation filter applied at step328may be a general noise cancellation filter, or a filter generally adapted to common computer124related events.

After applying a specifically selected filter at step324, or using information regarding the occurrence of an event to control the time at which a filter is applied at step328, the filtered output is provided for transmission to the second communication endpoint108via the communication network112(step332). The process may then return to Step304, to determine whether audio communications remain enabled. Accordingly, the process of event detection and noise cancellation may continue for as long as a user is engaged in a call or is otherwise providing audio information in connection with which noise cancellation is desired.

As can be appreciated as one in skill of the art from the description provided herein, embodiments of the present invention allow for noise associated with the operation of a computer124at or included in a communication endpoint104to be removed or cancelled from voice or other desired audio information collected by a microphone204associated with that communication endpoint104. In particular, by monitoring signals corresponding to events on a computer124, a noise cancellation process210can be provided with timing and/or information identifying the detected event, in order to apply specific or targeted noise cancellation measures.

Accordingly, embodiments of the present invention permit more effective noise cancellation as compared to noise cancellation systems that apply noise filters without discrimination as to the time and/or identity of events associated with the noise. In addition, by detecting electrical signals associated with events in order to determine whether there is a noise in the environment of a communication endpoint104, the reliability of detection is improved. Furthermore, embodiments of the present invention can be provided by the addition of software applications to existing telephones120, speech recorders136and/or computers124, without requiring modifications to hardware. In particular, embodiments of the present invention do not require the provision of separate microphones in order to detect noise.

Although embodiments of the present invention have been discussed in connection with voice telephony between two endpoints or in connection with speech recording, it should be appreciated that the present invention is not so limited. For instance, embodiments of the present invention may be applied in connection with the recording of any type of audio information input at or from a location where one or more computers124are operated. Furthermore, embodiments of the present invention are not limited to detecting the noise of a single computer124in connection with a communication. For example, information regarding events may be collected from a number of computers124at a location of a communication endpoint104.