PATENT DOCUMENT

Publication Number: US-8600743-B2
Application Number: US-68320310-A
Country: US
Kind Code: B2

Title: Noise profile determination for voice-related feature

Abstract:
Systems, methods, and devices for noise profile determination for a voice-related feature of an electronic device are provided. In one example, an electronic device capable of such noise profile determination may include a microphone and data processing circuitry. When a voice-related feature of the electronic device is not in use, the microphone may obtain ambient sounds. The data processing circuitry may determine a noise profile based at least in part on the obtained ambient sounds. The noise profile may enable the data processing circuitry to at least partially filter other ambient sounds obtained when the voice-related feature of the electronic device is in use.

Claims:
What is claimed is: 
     
       1. A method comprising:
 at a computer system including one or more processors and memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for:
 automatically without human intervention, determining whether a handheld electronic device is not in use; 
 upon determining that the handheld electronic device is not in use, automatically without human intervention activating a microphone associated with the handheld electronic device; 
 obtaining ambient sound using the microphone; 
 determining a first context of use of the handheld electronic device; 
 determining, at the handheld electronic device, a first noise profile based at least in part on the ambient sound, wherein the first noise profile is configured to enable the handheld electronic device to at least partially filter other ambient sound obtained at a later time when a voice-related feature of the handheld electronic device is in use; 
 storing the first noise profile in association with the first context of use, wherein the first noise profile is one of a plurality of stored noise profiles each associated with a respective context of use; 
 receiving an audio signal including voice and background sound; 
 determining a second context of use of the handheld electronic device; 
 determining whether the second context of use is substantially similar to the first context of use; 
 upon determining that the second context of use is substantially similar to the first context of use, selecting the first noise profile; and 
 using the first noise profile to at least partially filter the background sound from the audio signal to obtain the voice. 
 
 
     
     
       2. The method of  claim 1 , wherein activating the microphone comprises activating the microphone after a period of inactivity, wherein the period of inactivity is determined by the electronic device based at least in part on an operative mode of the electronic device; a state of a power supply of the electronic device; motion of the electronic device; a variability of other noise profiles previously determined; a location of the electronic device; or a usage history of the voice-related feature of the electronic device; or any combination thereof. 
     
     
       3. The method of  claim 1 , wherein the ambient sound is obtained during a sampling period, wherein the sampling period is determined by the electronic device based at least in part on an error rate of the voice-related feature of the electronic device associated with a noise profile; a convergence time associated with the determination of a noise profile; a comparison of the obtained ambient sound to previously-obtained ambient sound; or whether another noise profile based on other ambient sound obtained in a similar context is stored on the electronic device; or any combination thereof. 
     
     
       4. The method of  claim 1 , wherein determining the first and the second contexts of use comprises determining a time from a clock of the electronic device; a location from location-sensing circuitry of the electronic device; an amount of ambient light from image-capture circuitry of the electronic device; a motion of the electronic device from motion-sensing circuitry of the electronic device; or a volume of the ambient sound from the microphone; or any combination thereof. 
     
     
       5. The method of  claim 1 , wherein determining that the handheld electronic device is not in use includes determining that the voice-related feature of the handheld electronic device is not active. 
     
     
       6. The method of  claim 1 , wherein determining that the handheld electronic device is not in use includes determining that the handheld electronic device is being charged. 
     
     
       7. The method of  claim 1 , wherein determining that the handheld electronic device is not in use includes determining that the handheld electronic device is not moving. 
     
     
       8. The method of  claim 1 , wherein the first and the second context of use are defined by at least a location of the handheld electronic device and a time. 
     
     
       9. The method of  claim 1 , wherein the first and the second context of use are defined by at least a location of the handheld electronic device, a time, and a motion of the handheld electronic device. 
     
     
       10. The method of  claim 1 , wherein the first and the second context of use are defined by at least a location of the handheld electronic device, a time, an ambient volume level, and a motion of the handheld electronic device. 
     
     
       11. The method of  claim 1 , wherein the voice-related feature comprises a voice recognition feature; a voice note recording feature; or a video recording feature; or any combination thereof. 
     
     
       12. The method of  claim 1 , wherein the first and the second context of use are determined at the handheld electronic device. 
     
     
       13. The method of  claim 12 , wherein the first and the second context of use are determined automatically without human intervention. 
     
     
       14. A system, comprising:
 one or more processors; 
 memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for:
 automatically without human intervention, determining whether a handheld electronic device is not in use; 
 upon determining that the handheld electronic device is not in use, automatically without human intervention activating a microphone associated with the handheld electronic device; 
 obtaining ambient sound using the microphone; 
 determining a first context of use of the handheld electronic device; 
 determining, at the handheld electronic device, a first noise profile based at least in part on the ambient sound, wherein the first noise profile is configured to enable the handheld electronic device to at least partially filter other ambient sound obtained at a later time when a voice-related feature of the handheld electronic device is in use; 
 storing the first noise profile in association with the first context of use, wherein the first noise profile is one of a plurality of stored noise profiles each associated with a respective context of use; 
 receiving an audio signal including voice and background sound; 
 
 determining a second context of use of the handheld electronic device;
 determining whether the second context of use is substantially similar to the first context of use; 
 upon determining that the second context of use is substantially similar to the first context of use, selecting the first noise profile; and 
 using the first noise profile to at least partially filter the background sound from the audio signal to obtain the voice. 
 
 
     
     
       15. A non-transitory computer-readable storage medium, storing one or more programs for execution by one or more processors of an electronic device, the one or more programs including instructions for:
 automatically without human intervention, determining whether a handheld electronic device is not in use; 
 upon determining that the handheld electronic device is not in use, automatically without human intervention activating a microphone associated with the handheld electronic device; 
 obtaining ambient sound using the microphone; 
 determining a first context of use of the handheld electronic device; 
 determining, at the handheld electronic device, a first noise profile based at least in part on the ambient sound, wherein the first noise profile is configured to enable the handheld electronic device to at least partially filter other ambient sound obtained at a later time when a voice-related feature of the handheld electronic device is in use; 
 storing the first noise profile in association with the first context of use, wherein the first noise profile is one of a plurality of stored noise profiles each associated with a respective context of use; 
 receiving an audio signal including voice and background sound; 
 determining a second context of use of the handheld electronic device; 
 determining whether the second context of use is substantially similar to the first context of use; 
 upon determining that the second context of use is substantially similar to the first context of use, selecting the first noise profile; and 
 using the first noise profile to at least partially filter the background sound from the audio signal to obtain the voice.

Description:
BACKGROUND 
     The present disclosure relates generally to techniques for noise cancellation and, more particularly, for preemptive noise convergence for noise cancellation. 
     This section is intended to introduce the reader to various aspects of art that may be related to various aspects of the present disclosure, which are described and/or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art. 
     Many electronic devices employ voice-related features that involve recording and/or analyzing a user&#39;s voice. Voice recognition features, for example, may analyze a voice command spoken by a user to perform a task related to the command. Similarly, voice note recording features may record voice notes spoken by the user. However, when a user speaks into a microphone of an electronic device, ambient sounds, or background noise, may be obtained by the microphone at the same time. These ambient sounds may obscure the user&#39;s voice and, in some cases, may impede the proper functioning of a voice-related feature of the electronic device. 
     SUMMARY 
     A summary of certain embodiments disclosed herein is set forth below. It should be understood that these aspects are presented merely to provide the reader with a brief summary of these certain embodiments and that these aspects are not intended to limit the scope of this disclosure. Indeed, this disclosure may encompass a variety of aspects that may not be set forth below. 
     Embodiments of the present disclosure relate to systems, methods, and devices for noise profile determination for a voice-related feature of an electronic device. In one example, an electronic device capable of such noise profile determination may include a microphone and data processing circuitry. When a voice-related feature of the electronic device is not in use, the microphone may obtain ambient sounds. The data processing circuitry may determine a noise profile based at least in part on the obtained ambient sounds. The noise profile may enable the data processing circuitry to at least partially filter other ambient sounds obtained when the voice-related feature of the electronic device is in use. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Various aspects of this disclosure may be better understood upon reading the following detailed description and upon reference to the drawings in which: 
         FIG. 1  is a block diagram of an electronic device capable of performing the techniques disclosed herein, in accordance with an embodiment; 
         FIG. 2  is a schematic view of a handheld device representing one embodiment of the electronic device of  FIG. 1 ; 
         FIG. 3  is a schematic block diagram representing various contexts in which a voice-related feature of the electronic device of  FIG. 1  may be used, in accordance with an embodiment; 
         FIG. 4  is a flowchart representing an embodiment of a method for performing noise cancellation for a voice-related feature; 
         FIG. 5  is a flow diagram illustrating a manner of performing the method of  FIG. 4 , in accordance with an embodiment; 
         FIG. 6  is a flowchart representing an embodiment of a method for periodically determining a noise profile for use with a voice-related feature; 
         FIG. 7  is a flow diagram representing a manner of performing the method of  FIG. 6 , in accordance with an embodiment; 
         FIG. 8  is a schematic diagram representing various factors for determining the periodicity of the method of  FIG. 6 , in accordance with an embodiment; 
         FIG. 9  is a schematic diagram representing various factors for determining a sampling time for use with the method  FIG. 6 , in accordance with an embodiment; 
         FIG. 10  is a flowchart representing an embodiment of a method for periodically determining a contextual noise profile; 
         FIG. 11  is a flow diagram representing a manner of performing the method of  FIG. 10 , in accordance with an embodiment; 
         FIG. 12  is a flowchart of an embodiment of a method for performing contextual noise cancellation for a voice-related feature, in accordance with an embodiment; 
         FIG. 13  is a flow diagram representing a manner of performing the method of  FIG. 12 , in accordance with an embodiment; and 
         FIGS. 14-20  are flowcharts representing embodiments of methods for determining noise profiles due to irregular stimuli. 
     
    
    
     DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS 
     One or more specific embodiments will be described below. In an effort to provide a concise description of these embodiments, not all features of an actual implementation are described in the specification. It should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers&#39; specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another. Moreover, it should be appreciated that such a development effort might be complex and time consuming, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure. 
     Present embodiments relate to techniques for determining a noise profile for voice cancellation used in combination with voice-related features of electronic devices. As used herein, the term “noise profile” generally may refer to information relating to noise in a signal, such as ambient sounds in an audio signal, which may be employed to exclude such noise from another signal. For example, using the techniques described herein, a noise profile may be employed to filter ambient sounds from a voice command for a voice recognition feature, to isolate a user&#39;s voice for recording voice notes, to isolate a subject&#39;s voice for recording video, to enhance the quality of sound for a telephone feature, and so forth. Rather than attempt to identify noise at the time when a user activates or uses such a voice-related feature, the electronic device may remove ambient sounds using a noise profile developed from a prior recording of ambient sounds. As used herein, the term “voice-related feature” may refer to any functionality of an electronic device, which may be implemented in hardware or software, that may at least partially filter ambient sounds from an audio signal that may include voice audio and the ambient sounds. Among other things, such a voice-related feature may include voice recognition software that may receive voice commands from a user and, after filtering away ambient sounds, may perform various tasks based on the voice command. By way of example, voice recognition software may include the Voice Control application by Apple Inc. 
     In particular, because attempting to identify noise after a user has begun to speak may result in delays or a misidentification of ambient noise, an electronic device employing the techniques disclosed herein may periodically sample ambient sounds and construct a noise profile based on such ambient sounds when the voice-related feature is not in use. Additionally or alternatively, the electronic device may sample ambient sounds based on a stimulus (e.g., activation of a non-voice-related feature, initialization of the electronic device, navigation to the voice-related feature, a change in the context of use of the electronic device, and/or when another person is speaking on a telephone feature of the electronic device). Moreover, in some embodiments, the electronic device may assess a current context of use of the electronic device at the time the ambient sounds are sampled and when the noise profile is subsequently constructed. By way of example, the electronic device may consider a time, a current location of the electronic device, an amount of ambient light surrounding the electronic device, an amount of motion of the electronic device, and/or a volume level of ambient sounds. 
     A general description of suitable electronic devices for performing the presently disclosed techniques is provided below. In particular,  FIG. 1  is a block diagram depicting various components that may be present in an electronic device suitable for use with the present techniques.  FIG. 2  represents one example of a suitable electronic device, which may be, as illustrated, a handheld electronic device having image capture circuitry, motion-sensing circuitry, and video processing capabilities. 
     Turning first to  FIG. 1 , an electronic device  10  for performing the presently disclosed techniques may include, among other things, a central processing unit (CPU)  12  and/or other processors, memory  14 , nonvolatile storage  16 , a display  18 , an internal clock  20 , location-sensing circuitry  22 , an input/output (I/O) interface  24 , network interfaces  26 , image capture circuitry  28 , accelerometers/magnetometer  30 , and a microphone  32 . The various functional blocks shown in  FIG. 1  may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium) or a combination of both hardware and software elements. It should further be noted that  FIG. 1  is merely one example of a particular implementation and is intended to illustrate the types of components that may be present in electronic device  10 . 
     By way of example, the electronic device  10  may represent a block diagram of the handheld device depicted in  FIG. 2  or similar devices. Additionally or alternatively, the electronic device  10  may represent a system of electronic devices with certain characteristics. For example, a first electronic device may include at least a microphone  32 , which may provide audio to a second electronic device including the CPU  12  and other data processing circuitry. It should be noted that the data processing circuitry may be embodied wholly or in part as software, firmware, hardware or any combination thereof. Furthermore the data processing circuitry may be a single contained processing module or may be incorporated wholly or partially within any of the other elements within electronic device  10 . The data processing circuitry may also be partially embodied within electronic device  10  and partially embodied within another electronic device wired or wirelessly connected to device  10 . Finally, the data processing circuitry may be wholly implemented within another device wired or wirelessly connected to device  10 . As a non-limiting example, data processing circuitry might be embodied within a headset in connection with device  10 . 
     In the electronic device  10  of  FIG. 1 , the CPU  12  and/or other data processing circuitry may be operably coupled with the memory  14  and the nonvolatile memory  16  to perform various algorithms for carrying out the presently disclosed techniques. Such programs or instructions executed by the processor(s)  12  may be stored in any suitable manufacture that includes one or more tangible, computer-readable media at least collectively storing the instructions or routines, such as the memory  14  and the nonvolatile storage  16 . Also, programs (e.g., an operating system) encoded on such a computer program product may also include instructions that may be executed by the processor(s)  12  to enable the electronic device  10  to provide various functionalities, including those described herein. The display  18  may be a touch-screen display, which may enable users to interact with a user interface of the electronic device  10 . The internal clock  20  may track time and/or date. The location-sensing circuitry  22  may represent device capabilities for determining the relative or absolute location of electronic device  10 . By way of example, the location-sensing circuitry  22  may represent Global Positioning System (GPS) circuitry, algorithms for estimating location based on proximate wireless networks, such as local Wi-Fi networks, and so forth. 
     The I/O interface  24  may enable electronic device  10  to interface with various other electronic devices, as may the network interfaces  26 . The network interfaces  26  may include, for example, interfaces for a personal area network (PAN), such as a Bluetooth network, for a local area network (LAN), such as an 802.11x Wi-Fi network, and/or for a wide area network (WAN), such as a 3G cellular network. Through the network interfaces  26 , the electronic device  10  may interface with a wireless headset that includes a microphone  32 . The image capture circuitry  28  may enable image and/or video capture, and the accelerometers/magnetometer  30  may observe the movement and/or a relative orientation of the electronic device  10 . 
     When employed in connection with a voice-related feature of the electronic device  10 , such as a voice recognition software application, the microphone  32  may obtain an audio signal of a user&#39;s voice and other ambient sounds. The CPU  12  may process the audio signal to exclude most ambient sounds using a previously-determined noise profile. As described in greater detail below, the noise profile may be determined prior to the activation of the voice-related feature of the electronic device  10 , based on a recording of the ambient sounds. Although the microphone  32  generally may be inactive, the microphone  32  may periodically awaken, or may awaken after a stimulus, to record the ambient sounds when the user is not speaking. 
       FIG. 2  depicts a handheld device  34 , which represents one embodiment of electronic device  10 . The handheld device  34  may represent, for example, a portable phone, a media player, a personal data organizer, a handheld game platform, or any combination of such devices. By way of example, the handheld device  34  may be a model of an iPod® or iPhone® available from Apple Inc. of Cupertino, Calif. 
     The handheld device  34  may include an enclosure  36  to protect interior components from physical damage and to shield them from electromagnetic interference. The enclosure  36  may surround the display  18 , which may display indicator icons  38 . Such indicator icons  38  may indicate, among other things, a cellular signal strength, Bluetooth connection, and/or battery life. The I/O interfaces  24  may open through the enclosure  36  and may include, for example, a proprietary I/O port from Apple Inc. to connect to external devices. As indicated in  FIG. 2 , the reverse side of the handheld device  34  may include the image capture circuitry  28  and, in certain embodiments, an outward-facing microphone  32 . As described below, the outward-facing microphone  32  may be used to capture audio of ambient sounds even while the handheld device  34  is in use. 
     User input structures  40 ,  42 ,  44 , and  46 , in combination with the display  18 , may allow a user to control the handheld device  34 . For example, the input structure  40  may activate or deactivate the handheld device  34 , the input structure  42  may navigate user interface  20  to a home screen, a user-configurable application screen, and/or activate a voice-recognition feature of the handheld device  34 , the input structures  44  may provide volume control, and the input structure  46  may toggle between vibrate and ring modes. The microphones  32  may obtain a user&#39;s voice for various voice-related features, and a speaker  48  may enable audio playback and/or certain phone capabilities. Headphone input  50  may provide a connection to external speakers and/or headphones. 
     As illustrated in  FIG. 2 , a wired headset  52  may connect to the handheld device  34  via the headphone input  50 . The wired headset  52  may include two speakers  48  and a microphone  32 . The microphone  32  may enable a user to speak into the handheld device  34  in the same manner as the microphones  32  located on the handheld device  34 . In some embodiments, a button near the microphone  32  may cause the microphone  32  to awaken and/or may cause a voice-related feature of the handheld device  34  to activate. A wireless headset  54  may similarly connect to the handheld device  34  via a wireless interface (e.g., a Bluetooth interface) of the network interfaces  26 . Like the wired headset  52 , the wireless headset  54  may also include a speaker  48  and a microphone  32 . Also, in some embodiments, a button near the microphone  32  may cause the microphone  32  to awaken and/or may cause a voice-related feature of the handheld device  34  to activate. Additionally or alternatively, a standalone microphone  32  (not shown), which may lack an integrated speaker  48 , may interface with the handheld device  34  via the headphone input  50  or via one of the network interfaces  26 . 
     A user may use a voice-related feature of the electronic device  10 , such as a voice-recognition feature, in a variety of contexts.  FIG. 3  illustrates many such contexts  56  in which the electronic device  10 , depicted as the handheld device  34 , may receive a voice audio signal  58  from the user and ambient sounds  60  while performing a voice-related feature. By way of example, the voice-related feature of the electronic device  10  may include, for example, a voice recognition feature, a voice note recording feature, a video recording feature, and/or a telephone feature. The voice-related feature may be implemented on the electronic device  10  in software carried out by the CPU  12  or other processors, and/or may be implemented in specialized hardware. 
     To activate the voice-related feature on the electronic device  10 , the user may navigate to an icon representing the voice-related feature or may press one or more of the input structures  40 - 46 . For example, to activate a voice-recognition feature of the handheld device  34 , the user may press down the input structure  42  for approximately two seconds. Thereafter, the user may speak the voice audio signal  58 , which may enter the microphone  32  of the electronic device  10 . At approximately the same time, however, ambient sounds  60  may also enter the microphone  32 . Based on a previously-determined noise profile, the electronic device  10  may filter away the ambient sounds  60  such that the voice audio signal  58  is largely free of such noise. 
     The ambient sounds  60  may vary depending on the context  56  in which the electronic device  10  is being used. The various contexts  56  in which the voice-related feature may be used may include at home  62 , in the office  64 , at the gym  66 , on a busy street  68 , in a car  70 , at a sporting event  72 , at a restaurant  74 , and at a party  76 , among others. As should be appreciated, the typical ambient sounds  60  that occur on a busy street  68  may differ greatly from the typical ambient sounds  60  that occur at home  62  or in a car  70 . 
     Because the character of the ambient sounds  60  may vary from context  56  to context  56 , a single generic noise profile may not effectively eliminate the ambient sounds  60  in all of the contexts  56 . Rather, as described in greater detail below, the electronic device  10  may filter the ambient sounds  60  using a noise profile determined from a prior recording of the ambient sounds  60  that occurs before the user activates the voice-related feature of the electronic device  10 . Additionally, each of the contexts  56  may occur at certain locations and times, with varying amounts of electronic device  10  motion and ambient light, and/or with various volume levels of the voice signal  58  and the ambient sounds  60 . As such, in some embodiments, the electronic device  10  may filter the ambient sounds  60  using a noise profile previously determined in a similar context  56  based on the time, location, motion, ambient light, and/or volume level. 
       FIG. 4  is a flowchart  78  representing an embodiment of a method for isolating the voice audio signal  58  from the ambient sounds  60  when a voice-related feature of the electronic device  10  is used. In a first step  80 , a user may activate the voice-related feature of the electronic device  10 . Depending on the configuration of the electronic device  10 , step  80  may include navigating to a voice-related feature icon, holding down the button  42 , or pressing or holding a button on the wired headset  52  or wireless headset  54 . 
     In step  82 , the electronic device  10  may receive audio from the microphone  32  that is currently active. The active microphone  32  may be located, for example, on the handheld device  34 , the wired headset  52 , or the wireless headset  54 . In step  84 , a most-recently-determined noise profile may be recalled from the memory  14  or nonvolatile storage  16 . This noise profile may represent a noise profile determined based on a recent observation of the ambient sounds  60  prior to the activation of the voice-related feature in step  80 . 
     In step  86 , the ambient sounds  60  may be filtered out of the audio obtained in step  82 , substantially isolating the voice audio signal  58 . Thereafter, the voice audio signal  58  may be employed by the voice-related feature. By way of example, when the voice-related feature is a voice recognition feature, the electronic device  10  typically may subsequently analyze the voice audio signal  58  to ascertain a voice command. When the voice-related feature is a voice note recording feature, voice notes may be recorded with reduced background noise. 
       FIG. 5  is a flow diagram  92  that illustrates a manner of performing the method of  FIG. 4 . As shown in the flow diagram  92 , the active microphone  32  may receive the voice audio signal  58  as well as the ambient sounds  60 . The active microphone  32  and/or related circuitry may, in some embodiments, digitize and/or compress the obtained audio to produce a digital audio signal  94 . A filter  97 , which may be implemented using hardware, software, firmware, or a combination thereof, and which may include the CPU  12  and/or other processors, may receive the digital audio signal  94 . Based on a noise profile  96  received from memory  14  and/or the nonvolatile storage  16 , the filter  97  may filter the ambient sounds  60  out of the digital audio signal  94 . When the ambient sounds  60  have been at least partially filtered from the digital audio signal  94 , an isolated voice signal  98  may result. 
     Because a generic noise profile may not sufficiently isolate the ambient sounds  60  found in the various contexts  56  in which voice-related features may be used, and because determining the noise profile after a voice-related feature has been activated, the electronic device  10  may periodically determine the noise profile by sampling the ambient sounds  60  when the voice-related feature is not in use and/or using the outward-facing microphone  32 . A flowchart  100 , illustrated in  FIG. 6 , represents an embodiment of such a method for periodically determining a current noise profile  96 . First step  102  may occur while a voice-related feature is not currently in use, or may involve the use of an outward-facing microphone  32  of the electronic device  10 , such that the microphone  32  will less likely pick up sounds of the user&#39;s voice and will more likely pick up ambient sounds  60 . In step  102 , the electronic device  10  may periodically awaken a microphone  32  on the electronic device  10 , the wired headset  52 , or the wireless headset  54  after a period of inactivity. As described below with reference to  FIG. 8 , the length of the period of inactivity of the microphone  32 , occurring prior to step  102 , may vary depending on a variety of factors. By way of example, the period of inactivity may be chosen such that the microphone  32  is activated every 5 seconds, 10 seconds, 20 seconds, 30 seconds, 1 minute, 2 minutes, 5 minutes, 10 minutes, 20 minutes, 30 minutes, 1 hour, or 2 hours, and so forth. 
     In step  104 , the active microphone  32  may obtain ambient sounds by sampling the ambient sounds  60  for a period of time. As described below with reference to  FIG. 9 , the length of the sampling period may vary depending on a variety of factors. By way of example, the sampling period may be chosen such that the ambient sounds  60  are sampled for 1 second, 2 seconds, 5 seconds, 10 seconds, 20 seconds, or 30 seconds, 1 minute and so forth. The ambient sounds  60  sampled in step  104  may provide a basis for determining, in step  106 , a current noise profile  96 . The determination of the noise profile  96  of step  106  may take place in the electronic device  10  using any noise convergence technique, such that the determined current noise profile  96  may later be used to filter out ambient sounds  60 , in the manners described above with reference to  FIGS. 4 and 5 . In step  108 , the electronic device  10  may optionally store the newly determined noise profile  96  in memory  14  or nonvolatile storage  16 . 
     A flow diagram  110 , representing a manner of performing the method of  FIG. 6 , is illustrated in  FIG. 7 . In the flow diagram  110 , when the electronic device  10  periodically awakens the active microphone  32  when a voice-related feature is not in use, or when the active microphone  32  is an outward-facing microphone  32 , primarily only ambient sounds  60  may be picked up in the microphone  32  to form the audio signal  94 . The CPU  12  and/or any suitable data processing circuitry of the electronic device  10  may receive the digital audio signal  94 , converging the ambient sounds  60  of the digital audio signal  94  to a noise profile  96  that may later be used to filter out the recently-obtained ambient sounds  60 . The noise profile  96  may be stored in memory  14  or nonvolatile storage  16 . 
     As mentioned above with reference to the flowchart  100  of  FIG. 6 , the microphone  32  may not remain active at all times, but rather may activate after a period of inactivity to conserve resources of the electronic device  10 .  FIG. 8  illustrates a schematic diagram  112  representing various activation period factors  114  for determining the period of inactivity between times in which the microphone  32  may be activated. Based on one or more of the activation period factors  114 , the electronic device  10  may determine the period of inactivity at the time the current noise profile  96  is determined, every few times the current noise profile  96  is determined, or at other times. In general, the activation period factors  114  may balance how likely a voice-related feature is to be activated by a user and the use of resources for determining the noise profile  96  for the voice-related feature. 
     A first factor  116  of the activation period factors  114  may be a current mode of the electronic device  10 . By way of example, the electronic device  10  may be actively in use, may be inactive or in a sleep mode, and/or may be operating in a mode with reduced functionality, such as “airplane mode.” The factor  116  may weigh in favor of a shorter period of inactivity if the electronic device  10  is actively in use, and a voice-related feature is thus more likely to be activated by the user. The factor  116  may weigh in favor of a longer period of inactivity if the electronic device  10  is inactive or operating in a mode of reduced functionality. 
     A second factor  118  of the activation period factors  114  may be a current state of a power supply of the electronic device  10 . By way of example, if the electronic device  10  is currently being charged, a user may be less likely to use a voice-related feature of the electronic device  10 . Under such conditions, the factor  118  may weigh in favor of a longer period of inactivity. Alternatively, because the electronic device  10  may effectively have an unlimited supply of power, the factor  118  may instead weigh in favor of a shorter period of inactivity. 
     When the electronic device  10  is not currently being charged, the amount of remaining battery life may be considered. For example, when the battery life of the electronic device  10  remains high, the factor  118  may weigh in favor of a relatively shorter period of inactivity because the additional power needed to activate the microphone and to determine the noise profile may be justified. When the battery life of the electronic device  10  is relatively low, the factor  118  may weigh in favor of a relatively longer period of inactivity to conserve the remaining battery power. 
     A third factor  120  of the activation period factors  114  may be an amount of recent motion of the electronic device  10 , as detected by the accelerometers/magnetometer  30 . If very little motion is detected, the factor  120  may weigh in favor of a longer period of inactivity, as the electronic device  10  may have been set down by the user and, accordingly, a voice-related feature of the electronic device  10  may be less likely to be used. On the other hand, if a significant amount of motion is detected, the factor  120  may weigh in favor of a shorter period of inactivity, as the electronic device  10  is likely being carried by the user and, accordingly, a voice-related feature of the electronic device  10  may be more likely to be used. 
     A fourth factor  122  of the activation period factors  114  may be the variability of recently-determined noise profiles  96 . Specifically, if a recently determined noise profile  96  is very similar to previously-determined noise profiles  96 , further noise profiles  96  that are subsequently determined may likely provide little benefit over the previously-determined noise profiles  96 . Under such conditions, the factor  122  may weigh in favor of a longer period inactivity, because additional sampling by the microphone  32  may be less likely to obtain ambient sounds  60  that would produce a significantly different noise profile  96 . If recently-determined noise profiles  96  vary greatly, however, the factor  122  may weigh in favor of a shorter period of inactivity. 
     A fifth factor  124  of the activation period factors  114  may be the current location of the electronic device  10 . If the user has previously used, or frequently uses, a voice-related feature of the electronic device  10  at the current location (e.g., at home), as determined by the location-sensing circuitry  22  or based on a wireless network currently visible to the network interfaces  26 , the factor  124  may weigh in favor of a shorter period of inactivity. If not, the factor  124  may weigh in favor of a longer period of inactivity. 
     Similarly, a sixth factor  126  of the activation period factors  114  may be a history of voice-related feature use on the electronic device  10 . By way of example, based on the history of use of the voice-related feature, the factor  126  may weigh in favor of a shorter period of inactivity at times of the day when the voice-related feature of the electronic device  10  is frequently used. The factor  126  may weigh in favor of a longer period of inactivity at times of the day when the voice-related feature is rarely used. 
     As noted above, the period of time in which the microphone  32  may sample the ambient sounds  60  in step  104  of the flowchart  100  may vary. The particular length of the sampling period may vary depending on a variety of factors, as illustrated in  FIG. 9 . In  FIG. 9 , a schematic diagram  130  represents several sampling time factors  132 . The sampling time factors  132  may be used by the electronic device  10  to determine the sampling time at the time the noise profile  96  is determined, every few times the noise profile  96  is determined, or at other times. In general, the sampling time factors  132  may balance how likely a voice-related feature is to be activated by a user and the use of resources for determining the noise profile  96  for the voice-related feature. 
     A first factor  134  of the sampling time factors  132  may be an error rate that occurs when a voice-related feature is used. Certain voice-related features, such as voice-recognition features, may incorrectly identify a voice signal  98  if the ambient sounds  60  are not sufficiently filtered during noise cancellation. The error rate of such a voice-related feature may be stored and considered as the factor  134 . As such, the factor  134  may weigh in favor of a longer sampling time as the error rate increases. 
     A second factor  136  of the sampling time factors  132  may be an amount of time required to converge the ambient sounds  60  in the digital audio signal  94  to obtain the noise profile  96 . The factor  136  may weigh in favor of a sampling time that corresponds, longer or shorter, with recent convergence times. 
     A third factor  138  of the sampling time factors  132  may be a comparison of the digital audio signal  94  from the active microphone  32  to prior recordings of the ambient sounds  60  or a prior noise profile  96 . By way of example, the factor  138  may weigh in favor of a shorter sampling time if the digital audio signal  94  appears to be very similar to previously-recorded ambient sounds  60  or prior noise profiles  96 . In other words, if newly recorded ambient sounds  60  would simply result in a very similar noise profile  96 , the sampling time  132  may be reduced or cut short. 
     A fourth factor  140  of the sampling time factors  132  may relate to whether other noise profiles  96  have been stored, which were previously obtained in the same or similar context  56  in which the electronic device is currently being used. As described below with reference to  FIGS. 10-13 , certain embodiments of the techniques disclosed herein may involve observing and storing information relating to a current context  56  of use of the electronic device  10  with noise profiles  96  as they are determined. The factor  140  may weigh in favor of a shorter sampling time when other noise profiles  96  have already been obtained from a similar context  56 . 
       FIGS. 10-13  relate to manners of determining a contextual noise profile and, at a later time, using the contextual noise profile in the proper context  56 . Turning first to  FIG. 10 , a flowchart  150  represents an embodiment of a method for periodically determining a contextual noise profile. Steps  152 - 156  may be performed at substantially the same manner of steps  102 - 106  of the flowchart  100  of  FIG. 6 . Thus, after step  156 , a noise profile may be determined based on the ambient sounds  60 . 
     In step  158 , the electronic device  10  may determine its current context of use by ascertaining the time, the location, amount of ambient light, amount of motion, and/or volume of ambient sounds  60  currently associated with the electronic device  10 . In some embodiments, more or fewer such indicators of the current context  56  may be considered. In step  160 , the noise profile determined in step  156  and the contextual indicators assessed in step  158  may be associated. This may result in a contextual noise profile that may be identified for later use when the electronic device  10  is in the same or a similar context  56 . In step  162 , the contextual noise profile may be stored in memory  14  or nonvolatile storage  16 . 
       FIG. 11  is a flow diagram  164  representing a manner of performing the method of  FIG. 10 . As shown in the flow diagram  164 , ambient sounds  60  may be periodically detected by the active microphone  32  and converted to a digital audio signal  94 . A data processing unit such as CPU  12  may use the digital audio signal  94  to determine a noise profile associated with the ambient sounds  60 . Additionally, the CPU  12  may receive a time signal  166  from the clock  20 , a location signal  168  from the location-sensing circuitry  22 , an ambient light signal  170  from the image-capture circuitry  28 , and/or a motion and/or orientation signal  172  from the accelerometer/magnetometer  30 . The signals  166 - 172 , as well as a determination of the volume level of the digital audio signal  94 , may relate to a current context of use of the electronic device  10 . Since the various contexts  56  in which the electronic device may be used may have recognizable times, locations, ambient amounts of light, amounts of motion, and volume levels, the signals  166 - 172  may enable the CPU  12  to create a contextual noise profile  174  that generally may represent the ambient sounds  60  found at other similar times, locations, ambient amounts of light, amounts of motion, and volume levels. By way of example, a user may commute from home to work each day in a car between 8:00 and 9:00 AM. A contextual noise profile  174  obtained one day at this time likely may represent ambient sounds  60  that may occur during another day at the same time. Thereafter, the contextual noise profile  174  may be stored in memory  14  and/or nonvolatile storage  16 . In some embodiments, the contextual noise profile  174  may be stored in a database or similar data structure. 
     At a later time, the contextual noise profile  174  may be used for noise-cancellation when a user activates a voice-related feature in a similar context  56 , as described in a flowchart  176  of  FIG. 12 . In the flowchart  176 , steps  178  and  180  may be substantially the same as steps  80  and  82  of the flowchart  78  of  FIG. 4 . In step  182 , the electronic device  10  may assess the current context  56  of the electronic device by ascertaining the current time, location, ambient amount of light, amount of motion, and/or the volume of ambient sounds  60  in the digital audio signal  94 . In step  184 , the electronic device  10  may select a contextual noise profile  174  that matches the current context  56  ascertained in step  182 . In step  186 , the electronic device  10  may filter away the ambient sounds  60  based on the contextual noise profile to isolate the user&#39;s voice. 
       FIG. 13  is a flow diagram  188  illustrating a manner of performing the method described in  FIG. 12 . As shown in the flow diagram  188 , when a user elects to use a voice-related feature, the active microphone  32  may take in the voice audio signal  58  along with various ambient sounds  60 . The ambient sounds  60  may be converted into an audio signal  94  and transmitted to the filter  97 , which may be implemented using hardware, software, firmware, or a combination thereof, and which may include the CPU  12  and/or other processors. Additionally, the filter  97  may receive a time signal  166  from the clock  20 , a location signal  168  from the location-sensing circuitry  22 , and ambient light signal  170 , from the image capture circuitry  28 , and a motion signal  172  from the accelerometer/magnetometer  30 . Based on the signals  166 - 172 , as well as the volume of the digital audio signal  94 , the filter  97  may select a contextual noise profile  174  from the memory  14  or nonvolatile storage  16  that corresponds to a similar context of use  56 . To continue with the example mentioned above, if a user commutes from home to work each day in a car between 8:00 and 9:00 AM, and the time is currently 8:30 AM, the filter  97  may select a contextual noise profile  174  previously obtained around the same time, and therefore likely when the user was commuting to work. Using the contextual noise profile  174 , the filter  97  may filter out the ambient sounds  60  from the audio signal  94 , producing a voice audio signal  98 . 
     As described above, a noise profile  96  and/or  174  may be determined on a periodic basis, before a user activates a voice-related feature of the electronic device  10  and/or using an outward-facing microphone  32 . However, as described below with reference to  FIGS. 14-20 , noise profiles  96  and/or  174  may be determined, additionally or alternatively, in response to certain other stimuli. Such stimuli may include when a user activates a non-voice-related feature, when the electronic device  10  initializes, while a user navigates to a voice-related feature, when the electronic device  10  senses a change in context, and/or while a user is using a telephone feature of the electronic device  10 . 
       FIG. 14  is a flowchart  190  representing an embodiment of a method for determining a noise profile  96  or  174  on an irregular, non-periodical basis. Specifically, in a first step  192 , the user may elect to use a non-voice-related feature of the electronic device  10 . By way of example, the non-voice-related feature may be a web browser, a game, or other such feature. The act of starting or using the feature may serve as a stimulus that causes the electronic device  10  to begin determining a noise profile  96  or  174 . Thus, steps  194 - 198  may be performed in substantially the same manner as steps  104 - 108  of  FIG. 6  or, alternatively, steps  154 - 162  of  FIG. 10 . 
       FIG. 15  is a flowchart  200  in which, as illustrated by step  202 , the initialization of the electronic device  10  may serve as the stimulus to determine a noise profile  96  and/or  174 . By way of example, the electronic device  10  may initialize when the electronic device  10  is turned on or awakens from a sleep mode, is unplugged from being charged, or becomes decoupled from communication with another electronic device. After the electronic device  10  has been initialized in step  202 , step  204 - 208  may involve determining a noise profile  96  in the same manner as steps  104 - 108  of  FIG. 6 , or steps  154 - 162  of  FIG. 10 . 
       FIG. 16  is a flowchart representing an embodiment of a method in which navigating to a voice-related feature of the electronic device may serve as a stimulus to determine a noise profile  96  and/or  174 . As such, in step  212 , a user may initially navigate to a voice-related feature. By way of example, a user may navigate to an icon representative of the voice-related feature or may hold the button  42  with the intent to cause the voice-related feature of the electronic device  10  to activate. Step  212  may occur not after the voice-related feature has actually become activated, but rather when the user indicates an intent to activate the voice-related feature. In other words, as soon as the user navigates to a page on which the icon is listed or begins to press the button  42 , steps  214 - 218  may begin, regardless as to the whether the user ultimately chooses to activate the voice-related feature. More generally, step  212  may encompass any action by the user that indicates intent to activate the voice-related feature. Thereafter, steps  214 - 218  may be performed in substantially the same manner as step  104 - 108  of  FIG. 6  and/or steps  154 - 162  of  FIG. 10 . 
       FIG. 17  is a flowchart  220  representing an embodiment of a method in which a change in context  56  may provide a stimulus for determining a noise profile  96  and/or  174 . The flowchart  220  may begin when the electronic device  10  senses a change in context  56  in step  222 . The change in context  56  may be determined, for example, based on a change in time, location, ambient light, amount of motion, and/or a volume of ambient sounds  60 . If, as indicated respectively by decision blocks  224  and  226 , a stored noise profile currently matches the new context  56  and a usage history of the voice-related feature does not suggest that the voice-related feature is likely to be used, the process may flow to step  228 . In step  228 , the electronic device  10  may continue operating normally without determining a noise profile  96  or a contextual noise profile  174 . 
     However, if a stored contextual noise profile  174  does not match the current context, as noted in decision block  224 , or if the voice-related feature use history indicates that the voice-related feature is likely to be used, as noted in decision block  226 , a new noise profile  96  and/or  174  may be determined. As such, the process may flow to steps  230 - 234 . Steps  230 - 234  may performed in substantially the same manner as steps  104 - 108  of  FIG. 6  or steps  154 - 162  of  FIG. 10 . 
       FIGS. 18-20  describe manners of determining a noise profile  96  and/or  174  while a user is using a telephone feature of the electronic device  10 . Turning first to  FIG. 18 , a flowchart  236  may begin with a first step  238 , which occurs when a user is using a telephone feature of the electronic device  10 . In step  240 , a microphone  32  of electronic device  10  may sample the ambient sounds  60  that occur in between the words spoken by the user. Although the microphone  32  may record all sounds, including the user&#39;s voice, the user&#39;s voice may be louder then the ambient sounds  60  and therefore generally discernable from the ambient sounds. Furthermore, in step  242 , the convergence time for determining the noise profile  96  and/or  174  may take a longer time than when the user is not currently speaking. Nevertheless, the determination of the noise profile  96  and/or  174  may be largely transparent to the user, since, unlike a voice-recognition feature, the ambient sounds  60  may not need to be removed immediately. In step  244 , the determined noise profile  96  and/or  174  may be stored in memory  14  or the nonvolatile storage  16 . 
       FIG. 19  is a flowchart  246  that similarly may begin, in step  248 , when a user is using a telephone feature of the electronic device  10 . Specifically, in step  250 , the electronic device  10  may sample the ambient sounds  60  near the electronic device  10  at a time when the user is listening to the other person speaking over the telephone. In performing step  250 , the electronic device  10  may compare when the microphone  32  produces an audio signal  94  that occurs simultaneously with a received telephone audio signal, which may correspond to a time when the other person is speaking During these times, the user generally may not be speaking and, accordingly, the microphone  32  may primarily detect only ambient sounds  60 . Based on the audio signal  94  obtained in step  250 , the electronic device  10  may determine a noise profile  96  and/or  174  in step  252 , before storing the noise profile  96  and/or  174  in the memory  14  or the nonvolatile storage  16  in step  254 . 
       FIG. 20  is a flowchart  256  that also may begin, in step  258 , when a user is using a telephone feature of the electronic device  10 . In step  260 , the electronic device  10  may obtain a digital audio signal  94  from an active microphone  32 , which may be the outward-facing microphone  32  of the handheld device  34  illustrated in  FIG. 2 . Even while the user is speaking, the outward-facing microphone  32  may record less of the voice audio signal  58  and more of the ambient sounds  60  such that, in step  266 , a noise profile  96  and/or  174  may be determined based the ambient sounds  60 . In step  264 , the noise profile  96  and/or  174  determined in step  262  may be stored in memory  14  or nonvolatile storage  16 . 
     The specific embodiments described above have been shown by way of example, and it should be understood that these embodiments may be susceptible to various modifications and alternative forms. It should be further understood that the claims are not intended to be limited to the particular forms disclosed, but rather to cover all modifications, equivalents, and alternatives falling within the spirit and scope of this disclosure.

Metadata:
Filing Date: 20100106
Publication Date: 20131203
Grant Date: 20131203
Priority Date: 20100106
Inventors: LINDAHL ARAM
WILLIAMS JOSEPH M.
KLIMANIS GINTS VALDIS
Assignee: APPLE INC
CPC Classifications: [{"code": "G10L2021/02168", "inventive": false, "first": false, "tree": "[]"}, {"code": "G10L2021/02163", "inventive": false, "first": false, "tree": "[]"}, {"code": "G10L2021/02163", "inventive": false, "first": false, "tree": "[]"}, {"code": "G10L2021/02168", "inventive": false, "first": false, "tree": "[]"}, {"code": "G10L15/20", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L15/20", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L21/0208", "inventive": true, "first": true, "tree": "[]"}, {"code": "G10L21/0208", "inventive": true, "first": true, "tree": "[]"}]
Family ID: 44225220