ELECTRONIC DEVICE FOR CONTROLLING BEAMFORMING AND OPERATING METHOD THEREOF

An electronic device is provided. The electronic device includes, for the purpose of determining a customized beamformer filter, an input module including a plurality of microphones configured to receive an external sound signal, a memory configured to store computer-executable instructions and an initial value of a voice parameter used to perform beamforming on the external sound signal, and a processor configured to execute the instructions by accessing the memory. The instructions may be configured to estimate a feature value of the external sound signal, calculate the initial value of the voice parameter used to perform beamforming based on the external sound signal received by the plurality of microphones, determine whether to store the calculated initial value according to the feature value, determine which one of the calculated initial value or an initial value stored in the memory used according to the feature value, and obtain a target voice parameter.

BACKGROUND

The disclosure relates to an electronic device for controlling beamforming and a method for controlling the same.

2. Description of Related Art

The electronic device may provide a function related to audio signal processing. For example, the electronic device may provide a call function for collecting and transmitting audio signals and a recording function for recording audio signals.

Electronic devices that output audio, such as earphones and headphones, may be equipped with various technologies for removing and suppressing noise to distinguish a voice signal. For example, headphones may obtain ambient noise through a microphone connected to a noise canceling circuit, and may output an anti-noise signal having an antiphase relative to the obtained noise. The ambient noise and the antiphase noise may be heard together, which for the user may have the effect of removing the noise. In addition, research is being conducted on a method of performing beamforming on signals received through a plurality of microphones to obtain a more improved user voice from an audio output device.

SUMMARY

Electronic devices, such as earphones or headphones to be used while being worn on ears, have limited form factors. When such an electronic device is worn by a user, because a microphone mounted on the electronic device is at a distance from the user's mouth, the influence of ambient noise may be significant, and a user's voice may not be readily obtained.

When an electronic device such as earphones or headphones is performing a function such as a call or recording in an environment with a lot of ambient noise, a good voice signal may not be easily obtained.

Earphones or headphones through which a user's voice may not be obtained properly may degrade the quality of voice calls and make it difficult to recognize a voice.

Aspects of the disclosure are to address at least the above-mentioned problems and/or disadvantages and to provide at least the advantages described below. Accordingly, an aspect of the disclosure is to provide an electronic device for controlling beamforming and operating method.

In accordance with an aspect of the disclosure, an electronic device is provided. The electronic device includes an input module including a plurality of microphones configured to receive an external sound signal. The electronic device includes a memory configured to store computer-executable instructions and an initial value of a voice parameter used to perform beamforming on the external sound signal. The electronic device may include a processor configured to execute the instructions by accessing the memory. The instructions may be configured to estimate a feature value of the external sound signal. The instructions may be configured to calculate the initial value of the voice parameter used to perform beamforming based on the external sound signal received by the plurality of microphones. The instructions may be configured to determine whether to store the calculated initial value in the memory according to the feature value. The instructions may be configured to determine which one of the calculated initial value or the initial value stored in the memory is to be used according to the feature value. The instructions may be configured to obtain a target voice parameter used to perform beamforming on the external sound signal based on the determined initial value according to the feature value.

In accordance with another aspect of the disclosure, a method of obtaining a voice parameter used to perform beamforming is provided. The method includes estimating a feature value of an external sound signal received by a plurality of microphones. The method may include calculating an initial value of a voice parameter used to perform beamforming on the external sound signal. The method may include determining whether to store the calculated initial value according to the feature value. The method may include determining which one of the calculated initial value or a stored initial value is to be used according to the feature value. The method may include obtaining a target voice parameter used to perform beamforming on the external sound signal based on the determined initial value according to the feature value.

In accordance with another aspect of the disclosure, an operating method of an electronic device is provided. The operating method includes estimating a feature value of an external sound signal received by a plurality of microphones. The operating method includes calculating an initial value of a voice parameter used to perform beamforming on the external sound signal. The operating method includes determining whether to store the calculated initial value according to the feature value. The operating method includes determining which one of the calculated initial value or a stored initial value is to be used according to the feature value. The operating method includes obtaining a target voice parameter used to perform beamforming on the external sound signal based on the determined initial value according to the feature value. The operating method may include determining a filter used to perform beamforming on the external sound signal based on the target voice parameter. The operating method includes estimating a magnitude of residual noise with respect to a signal on which beamforming is performed using the filter. The operating method includes performing noise processing on the signal on which beamforming is performed according to the estimated magnitude of the residual noise.

According to one embodiment, an electronic device that may adjust a voice parameter used to perform beamforming according to a feature value of an external sound signal may be provided.

According to one embodiment, an electronic device that may adaptively determine a beamformer filter according to different user wearing styles and ear shapes may be provided.

DETAILED DESCRIPTION

The program140may be stored as software in the memory130, and may include, for example, an operating system (OS)142, middleware144, or an application146.

The input module150may receive, from the outside (e.g., a user) the electronic device101, a command or data to be used by another component (e.g., the processor120) of the electronic device101. The input module150may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

The haptic module179may convert an electric signal into a mechanical stimulus (e.g., a vibration or a movement) or an electrical stimulus which may be recognized by a user via his or her tactile sensation or kinesthetic sensation. According to an example embodiment, the haptic module179may include, for example, a motor, a piezoelectric element, or an electric stimulator.

The camera module180may capture a still image and moving images. According to an example embodiment, the camera module180may include one or more lenses, image sensors, ISPs, or flashes.

The power management module188may manage power supplied to the electronic device101. According to an example embodiment, the power management module188may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery189may supply power to at least one component of the electronic device101. According to an example embodiment, the battery189may include, for example, a primary cell which is not rechargeable, a secondary cell which is rechargeable, or a fuel cell.

FIG.2is a block diagram of an audio module according to an embodiment of the disclosure.

Referring toFIG.2depicting a block diagram200, an audio module170may include, for example, an audio input interface210, an audio input mixer220, an analog-to-digital converter (ADC)230, an audio signal processor240, a digital-to-analog converter (DAC)250, an audio output mixer260, or an audio output interface270.

The audio input interface210may receive an audio signal corresponding to a sound obtained from the outside of the electronic device101via a microphone (e.g., a dynamic microphone, a condenser microphone, or a piezo microphone) that is configured as part of the input module150or separately from the electronic device101. For example, if an audio signal is obtained from the external electronic device102(e.g., a headset or a microphone), the audio input interface210may be connected with the external electronic device102directly via the connecting terminal178, or wirelessly (e.g., Bluetooth™ communication) via the wireless communication module192to receive the audio signal. According to an example embodiment, the audio input interface210may receive a control signal (e.g., a volume adjustment signal received via an input button) related to the audio signal obtained from the external electronic device102. The audio input interface210may include a plurality of audio input channels and may receive a different audio signal via a corresponding one of the plurality of audio input channels, respectively. According to an example embodiment, additionally or alternatively, the audio input interface210may receive an audio signal from another component (e.g., the processor120or the memory130) of the electronic device101.

The audio input mixer220may synthesize a plurality of input audio signals into at least one audio signal. For example, according to an example embodiment, the audio input mixer220may synthesize a plurality of analog audio signals input via the audio input interface210into at least one analog audio signal.

The ADC230may convert an analog audio signal into a digital audio signal. For example, according to an example embodiment, the ADC230may convert an analog audio signal received via the audio input interface210or, additionally or alternatively, an analog audio signal synthesized via the audio input mixer220into a digital audio signal.

The audio signal processor240may perform various processing on a digital audio signal received via the ADC230or a digital audio signal received from another component of the electronic device101. For example, according to an example embodiment, the audio signal processor240may perform changing a sampling rate, applying one or more filters, interpolation processing, amplifying or attenuating a whole or partial frequency bandwidth, noise processing (e.g., attenuating noise or echoes), changing channels (e.g., switching between mono and stereo), mixing, or extracting a specified signal for one or more digital audio signals. According to an example embodiment, one or more functions of the audio signal processor240may be implemented in the form of an equalizer.

The DAC250may convert a digital audio signal into an analog audio signal. For example, according to an example embodiment, the DAC250may convert a digital audio signal processed by the audio signal processor240or a digital audio signal obtained from another component (e.g., the processor120or the memory130) of the electronic device101into an analog audio signal.

The audio output mixer260may synthesize a plurality of audio signals, which are to be output, into at least one audio signal. For example, according to an example embodiment, the audio output mixer260may synthesize an analog audio signal converted by the DAC250and another analog audio signal (e.g., an analog audio signal received via the audio input interface210) into at least one analog audio signal.

The audio output interface270may output an analog audio signal converted by the DAC250or, additionally or alternatively, an analog audio signal synthesized by the audio output mixer260to the outside of the electronic device101via the sound output module155. The sound output module155may include, for example, a speaker, such as a dynamic driver or a balanced armature driver, or a receiver. According to an example embodiment, the sound output module155may include a plurality of speakers. In such a case, the audio output interface270may output audio signals having a plurality of different channels (e.g., stereo channels or 5.1 channels) via at least some of the plurality of speakers. According to an example embodiment, the audio output interface270may be connected with the external electronic device102(e.g., an external speaker or a headset) directly via the connecting terminal178or wirelessly via the wireless communication module192to output an audio signal.

According to an example embodiment, the audio module170may generate, without separately including the audio input mixer220or the audio output mixer260, at least one digital audio signal by synthesizing a plurality of digital audio signals using at least one function of the audio signal processor240.

According to an example embodiment, the audio module170may include an audio amplifier (not shown) (e.g., a speaker amplifying circuit) that is capable of amplifying an analog audio signal input via the audio input interface210or an audio signal that is to be output via the audio output interface270. According to an example embodiment, the audio amplifier may be configured as a module separate from the audio module170.

FIG.3is a diagram illustrating an example audio signal processing system according to an embodiment of the disclosure.

Referring toFIG.3, according to an example embodiment, an audio signal processing system10may include a first electronic device101and a second electronic device102. The first electronic device101and the second electronic device102may include at least a portion of a configuration of the electronic device101described above with reference toFIG.1. According to an example embodiment, the first electronic device101may be connected to the second electronic device102by wire or wirelessly, and may output an audio signal transmitted by the second electronic device102. The first electronic device101may collect an external sound signal using a plurality of microphones and transmit the collected audio signal to the second electronic device102.

According to an example embodiment, the first electronic device101may be a wireless earphone capable of forming a short-range communication channel (e.g., a Bluetooth module-based communication channel) with the second electronic device102. For example, the first electronic device101may be any one of a true-wireless stereo (TWS), a wireless headphone, and a wireless headset. The first electronic device101is illustrated as a kernel-type wireless earphone inFIG.3, but is not limited thereto. For example, the first electronic device101may be a stem-type wireless earphone in which at least a portion of the housing protrudes in a predetermined direction to collect a good user voice signal. According to an example embodiment, the first electronic device101may be a wired earphone connected to the second electronic device102in a wired manner.

According to an example embodiment, the first electronic device101, which is an earphone-type device, may include a housing301(or a case) including an insertion portion301athat may be inserted into a user's ear, and a mounting portion301bconnected to the insertion portion301aand capable of being mounted at least partially on the user's auricle. The first electronic device101may include a plurality of microphones150-1and150-2.

According to various example embodiments, the electronic device101may include an input interface377capable of receiving an input of the user. The input interface377may include, for example, a physical interface (e.g., a physical button or a touch button) and a virtual interface (e.g., a gesture, object recognition, or voice recognition). In one embodiment, the electronic device101may include a touch sensor capable of detecting a contact with the user's skin. For example, an area (e.g., the input interface377) in which the touch sensor is disposed may be located in a part of the electronic device101. The user may apply an input by touching the area using a body part. The touch input may include, for example, a single touch, multiple touches, a swipe, and/or a flick.

The microphones150-1and150-2may perform the function of the input module150described above with reference toFIG.1. Any repeated description related thereto has been omitted. Among the microphones150-1and150-2, a first microphone150-1may be disposed on the mounting portion301bsuch that, based on the inside of the ear, at least a portion of the sound hole may be exposed to the outside, to collect external ambient sound while the first electronic device101is worn on the user's ear. Among the microphones150-1and150-2, a second microphone150-2may be disposed on the insertion portion301a. The second microphone150-2may be disposed such that, based on the auricle-side opening of the outer ear path, at least a portion of the sound hole is exposed toward the inside of the outer ear path or at least a portion of the sound hole is in contact with the inner wall of the outer ear path, to collect signals transmitted into the outer ear canal (or, external auditory canal) while the first electronic device101is being worn on the user's ear. For example, when the user makes a voice utterance while wearing the first electronic device101, at least some of the tremor from the utterance may be transmitted through the user's skin, muscles, or bones, and the transmitted tremor may be collected as ambient sound by the second microphone150-2inside the ear.

According to one embodiment, the second microphone150-2may be any of various types of microphones (e.g., an in-ear microphone, an inner microphone, or a bone conduction microphone) capable of collecting sound from the cavity inside the user's ear. For example, the second microphone150-2may include at least one air conduction microphone and/or at least one bone conduction microphone for detecting a voice. The air conduction microphone may detect a voice (e.g., an utterance of a user) transmitted through air and output a voice signal corresponding to the detected voice. The bone conduction microphone may measure a vibration of a bone (e.g., the skull) caused by a vocalization of a user and output a voice signal corresponding to the measured vibration. The bone conduction microphone may be referred to as a bone conduction sensor, or various other names. A voice detected by the air conduction microphone may be a voice mixed with external noise, the mixing occurring while the user's utterance is being transmitted through air. Since the voice detected by the bone conduction microphone is from the vibration of a bone, it may include less external noise (e.g., influence of noise).

InFIG.3, the first microphone150-1and the second microphone150-2are respectively illustrated as being installed on the electronic device101, one of each, but the number is not limited thereto. A plurality of the first microphone150-1, which is an external microphone, and a plurality of the second microphone150-2, which is an in-ear microphone, may be installed on the electronic device101. Although omitted fromFIG.3, the electronic device101may further include an accelerator for voice activity detection (VAD) and a vibration sensor (e.g., a voice pickup unit (VPU) sensor).

According to one embodiment, the first electronic device101may include the audio module170described above with reference toFIGS.1and2. Any repeated description related thereto has been omitted. The first electronic device101may perform audio signal processing such as noise processing (e.g., noise suppressing), frequency band adjustment, and gain adjustment through the audio module170(e.g., through the audio signal processor240ofFIG.2). The configuration of the first electronic device101will be described in detail with reference toFIG.5B. The first electronic device101may be referred to as the electronic device101in the descriptions relating toFIGS.4,5A and5B, and6to11.

According to one embodiment, the electronic device101may include a sensor capable of detecting of the electronic device101is worn on the user's ear. For example, the sensor may include a sensor (e.g., an infrared sensor or a laser sensor) capable of detecting a distance to an object, and a sensor (e.g., a touch sensor) capable of detecting a contact with the object. As the electronic device101is worn on the user's ear, the sensor may generate a signal by detecting a distance to the skin or a contact with the skin. The processor120of the electronic device101may recognize whether the electronic device101is currently being worn by detecting the signal generated by the sensor.

According to an example embodiment, the second electronic device102may establish a communication channel with the first electronic device101, transmit a designated audio signal to the first electronic device101, or receive an audio signal from the first electronic device101. For example, the second electronic device102may be any of various electronic devices such as a mobile terminal, a terminal device, a smartphone, a tablet personal computer (PC), a pad, or a wearable electronic device capable of forming a communication channel (e.g., a wired or wireless communication channel) with the first electronic device101. The second electronic device102may include a configuration that is the same as or similar to the configuration of the electronic device101described above with reference toFIG.1, and may include fewer or more configurations than the electronic device101ofFIG.1depending on the implementation. The second electronic device102may be referred to as the electronic device102in the descriptions relating toFIGS.4,5A and5B, and6to11.

According to one embodiment, in the audio signal processing system10, the first electronic device101may perform beamforming to obtain an improved user voice signal. For example, the first electronic device101may perform beamforming on external sound signals received through the plurality of microphones150-1and150-2. A beamformer that performs beamforming according to one embodiment is described in detail with reference toFIG.4.

FIG.4is a block diagram illustrating a beamformer according to an embodiment of the disclosure.

FIG.4is a block diagram illustrating a signal-dependent beamformer according to an embodiment of the disclosure. Types of beamforming and beamformers are briefly described with reference toFIG.4.

Referring toFIG.4, a signal xi(t) input to an i-th microphone of M microphones may include a voice signal si(t) and a noise signal ni(t) and may be represented by xi(t)=si(t)+ni(t). The signal xi(t), which is directional and has different phases for each of a plurality of microphones, may be input to the plurality of microphones, and a phase difference may be determined according to a position of a sound source. In general, since a position of a sound source of the voice signal and a position of a sound source of the noise signal are different, as shown in Equation 1, a phase difference ts,ibetween a voice signal input to a first microphone of M microphones and a voice signal input to the i-th microphone and a phase difference tn,ibetween a noise signal input to the first microphone and a noise signal input to the i-th microphone may be different.

Beamforming is a technique for improving a feature value, such as a signal-to-noise ratio (SNR), of a voice signal by compensating for a phase difference with respect to an input signal input to each of different microphones. An output signal y(t) output through beamforming may be represented by Equation 2.

wi(t) may be to compensate for a phase difference with respect to a voice signal and may be referred to as a beamformer vector, beamformer filter, and the like. A noise element may be canceled by wi(t), and a magnitude of the noise element may be reduced compared to that of a voice element, such that an SNR of the voice signal may be improved.

A beamformer may be classified as a signal-independent beamformer or a signal-dependent beamformer according to whether a feature of an input signal is used. A signal-independent beamformer may estimate a direction of arrival (DoA) of a voice through a localization technique and perform beamforming by compensating for a phase difference between a signal input to each microphone using direction information. According to how a beamformer filter wi(t), which compensates for a phase difference, is obtained, the signal-independent beamformer may include a delay-and-sum beamformer (DSBF), a generalized sidelobe canceller (GSC) beamformer, a minimum variance distortionless response (MVDR) beamformer, and the like. In an environment in which direction information of a voice signal and position information of a microphone may be obtained in advance or in an environment in which estimation may be performed with high accuracy (e.g., an environment in which a position of a user may not change), the signal-independent beamformer may maintain a predetermined performance level regardless of an SNR. However, in response to a change in the position of the user, a DoA of the user's mouth may need to be estimated. Accordingly, an error in estimating the DoA of the user's mouth in a noisy environment may occur, and reverberation may occur indoors, and thus performance of the signal-independent beamformer may be degraded.

The signal-dependent beamformer is a beamformer that performs beamforming based on spatial characteristics of microphone input signals.FIG.4is a block diagram illustrating a signal-dependent beamformer according to one embodiment. In a noisy environment, an input signal of a microphone may be divided into a section in which a voice and noise are mixed and a section in which only noise is present without a voice. Cx, which is a voice covariance matrix, may be obtained from the section in which a voice is included, and Cn, which is a noise covariance matrix, may be obtained from the section in which only noise is present. Cx may include a spatial characteristic of a voice, and Cn may include a spatial characteristic of noise. Examples of obtaining Cx and Cn are described with reference toFIG.5A.

Based on Cx and Cn, a beamformer directed toward a user's mouth and a beamformer that generates a null vector toward the user's mouth may be determined. The signal-dependent beamformer may reduce a magnitude of noise in a voice signal based on the beamformer filter that steers toward the user's mouth or obtain only a noise element based on the beamformer that generates a null vector toward the user's mouth.

There may be various example embodiments that determine a beamformer filter based on the covariance matrices Cx and Cn. For example, a MaxSNR beamformer may determine a beamformer filter based on Equation 3 in a way that an SNR of a signal is improved as much as possible.

The MaxSNR beamformer may determine WSNR(f), which is an eigen vector having a largest eigenvalue λmax,Sof Cn(f)−1Cx(f), to be a beamformer filter.

As another example, an MVDR beamformer may perform beamforming to remove noise while minimizing a voice distortion. A beamformer filter may be obtained using the MVDR beamformer in various ways. For example, a beamformer filter may be obtained using the MaxSNR beamformer as shown in Equation 4.

As another example, a blocking matrix (BM) WBM,SNR(f) of a GSC may be obtained using a null space of WSNR(f) obtained using the MaxSNR beamformer based on Equation 5.

Through WBM,SNR(f), a voice element may be removed from a signal in which noise and a voice are mixed, and the noise may be more accurately removed from the signal in which the noise and voice are mixed based on the signal in which the voice element is removed.

Unlike the signal-independent beamformer, the signal-dependent beamformer may have improved robustness to a voice direction. However, performance of the signal-dependent beamformer may be reduced as accuracy of a voice covariance matrix Cx is reduced in a low SNR environment. A position of a voice may be estimated using the signal-independent beamformer in response to a voice activity being detected based on an in-ear microphone or a vibration sensor. However, an error in estimating a position of a voice may increase in the low SNR environment. In addition, controlling beamforming based on a position of a voice may merely compensate for a deviation caused by a wearing angle and not reflect a deviation caused by a difference in a wearing style or a structure of the inner ear. An ideal beamforming direction may vary depending on a user's wearing style and inner ear structure, and a beamformer filter may also vary.

FIGS.5A and5B, and6,7,8,9,10, and11illustrate, in detail, how to prevent performance degradation in a low SNR environment due to a difference in a user's wearing style and inner ear structure.

FIGS.5A and5Bare diagrams illustrating a process of performing beamforming and noise processing in an electronic device and a configuration of an electronic device according to various embodiments of the disclosure.

FIG.5Aillustrates a noise processing system that performs beamforming and noise processing in an electronic device according to an embodiment of the disclosure.

Referring toFIG.3, the electronic device101may include an external microphone (e.g., the first microphone150-1ofFIG.3), an in-ear microphone (e.g., the second microphone150-2ofFIG.3), and an accelerator502.

A beamformer in operation510may perform beamforming on an external sound signal received by a plurality of microphones (e.g., an external microphone and in-ear microphone) of the electronic device101. The electronic device101may determine a beamformer filter by loading, from a memory, a parameter (e.g., a voice covariance matrix Cx described with reference toFIG.4) related to a determination of a beamformer filter or storing (e.g., operation520of loading or storing a beamformer parameter) the parameter in the memory according to a feature value of the external sound signal. An operation of performing beamforming according to a feature value of an external sound signal is described in detail with reference toFIGS.6to11.

A voice activity may be detected (e.g., VAD530) based on the in-ear microphone and the accelerator502of the electronic device101. A mask m(t,f) corresponding to an f-th frequency bin in a t-th frame may be estimated based on voice activity detection (VAD), and a covariance matrix Cx(f) of a voice and a covariance matrix Cn(f) of noise may be determined based on Equation 6 below. However, the method of obtaining a Cx and a Cn is not limited to Equation 6, and various methods may be used.

X(t,f)=[X1(t,f), X2(t,f), . . . , XM(t,f)]Tmay be a microphone input signal corresponding to the f-th frequency bin in the t-th frame. As described above with reference toFIG.4, diagonal matrix elements of a Cx and a Cn may include information on a magnitude of each signal, and non-diagonal matrix elements may include information on a space of each signal. The Cx and Cn may be used to determine a filter of the beamformer410as described above with reference toFIG.4.

In response to a beamformer filter being determined, a magnitude of residual noise may be estimated (e.g., at noise estimation540) according to a result of VAD530, and the residual noise may be removed (via noise suppression operation550) from a beamforming result according to information on the estimated magnitude of the noise. A deep neural network (DNN) may be used in a process of removing residual noise.

By performing the above-mentioned series of operations of the noise processing system500, the electronic device101may output an improved voice audio signal. Hereinafter, a configuration of the electronic device101is described with reference toFIG.5B, and an operation of the electronic device101is described in detail with reference toFIGS.6to11.

FIG.5Bis a block diagram illustrating a configuration of the electronic device101according to an embodiment of the disclosure.

The electronic device101ofFIG.5Bmay be the first electronic device101described above with reference toFIG.3, and the electronic device102ofFIG.5Bmay be the second electronic device102described above with reference toFIG.3.

The electronic device101may include an input module150for receiving an ambient sound, a sound output module155for outputting a sound in which the ambient sound is processed, an audio module170for processing the ambient sound, a memory130in which computer-executable instructions and voice parameter initial value information580are stored, and a processor120for executing the instructions by accessing the memory130. The electronic device101, the electronic device102, the processor120, the memory130, the input module150, the sound output module155, the audio module170, and the communication module190may correspond to the electronic device101, the electronic device102, the processor120, the memory130, the input module150, the sound output module155, the audio module170, and the communication module190described above with reference toFIGS.1to4, and any repeated description thereof has been omitted. As described above with reference toFIG.3, the electronic device101may be an audio output device such as wireless earphones, and the electronic device102may be an electronic device such as a smartphone that transmits and receives an audio signal to and from the electronic device101.

The processor120may estimate a feature value of an external sound signal received by the input module150. For example, the processor120may estimate the feature value of the external sound signal received by a plurality of microphones (e.g., the external microphone (e.g., first microphone150-1) ofFIG.3and the in-ear microphone150-2). Hereinafter, a feature value is described based on an SNR, but examples are not limited thereto, and noise power of an external sound signal may be used as a feature value, for example.

The processor120may calculate an initial value of a voice parameter used to perform beamforming based on the external sound signal, determine whether to store the calculated initial value in the voice parameter initial value information580of the memory130according to the feature value, determine which one of the calculated initial value or a stored initial value is to be used, and obtain a voice parameter based on the determined initial value. The voice parameter may be a voice covariance matrix Cx used to determine a beamformer filter related to the signal-dependent beamformer described above with reference toFIG.4.

According to one embodiment, a program (e.g., the program140ofFIG.1) that adjusts a beamformer filter according to a feature value of an external sound signal to obtain an improved voice signal may be stored as software in the memory130. An operation of the processor120is described in detail with reference toFIGS.6to11.

FIG.6is a flowchart illustrating an operating method of an electronic device according to an embodiment of the disclosure.

Operations610to650may be performed by the processor120of the electronic device101described above with reference toFIG.5B. Thus, any description overlapping the description referring toFIGS.1to4, and5A and5Bwill not be repeated for conciseness. Operations610to650may correspond to beamforming operation510or operation520of loading or storing a parameter related to a beamformer filter described above with reference toFIG.5A.

Referring toFIG.6, according to one embodiment, in operation610, the processor120may estimate a feature value of an external sound signal. The external sound signal may be received by the input module150, for example, the plurality of microphones150-1and150-2ofFIG.3, described above with reference toFIG.5B.

According to one embodiment, the processor120may estimate a feature value of an external sound signal received by a main microphone (e.g., a microphone closest to a mouth among the plurality of microphones) of the plurality of microphones mounted on the electronic device101. A feature value may be an SNR or noise power. The processor120may estimate the feature value using the voice activity detection (VAD)530technique described above with reference toFIG.5A. For example, as described above with reference toFIG.5A, the processor120may estimate an SNR of an external sound signal more accurately by estimating an SNR when a voice activity is detected based on a vibration sensor, an accelerator, an in-ear microphone, and the like.

According to one embodiment, in operation620, the processor120may calculate an initial value of a voice parameter used to perform beamforming on the external sound signal. The processor120may calculate an initial value of a voice covariance matrix Cx described above with reference toFIGS.4and5Abased on signals input to the plurality of microphones. For example, the processor120may obtain an initial value of a voice covariance matrix between an audio signal input to each of a plurality of external microphones. As another example, the processor120may obtain an initial value of a voice covariance matrix between an audio signal input to an external microphone (e.g., the external microphone (e.g., first microphone150-1) ofFIG.3) and an audio signal input to an internal microphone (e.g., the in-ear microphone150-2ofFIG.3).

According to one embodiment, in operation630, the processor120may determine whether to store the calculated initial value of the voice parameter according to the feature value. In response to the SNR of the external sound signal exceeding a first threshold value (e.g., 15 decibels (dB)), the processor120may store the calculated initial value of the voice covariance matrix in the voice parameter initial value information580of the memory130. In response to the SNR of the external sound signal being less than or equal to the first threshold value (e.g., 15 dB), the processor120may determine not to store the calculated initial value of the voice covariance matrix in the voice parameter initial value information580of the memory130.

In response to the SNR exceeding, for example, 15 dB, which is a high SNR, a voice covariance matrix may be estimated with high accuracy using the initial value of the voice covariance matrix, and beamforming performance may be maintained. An operation of the processor120to be performed when the feature value of the external sound signal exceeds the first threshold value is described in detail with reference toFIG.9.

According to one embodiment, in operation640, the processor120may determine which one of the calculated initial value or a stored initial value is to be used to obtain a voice parameter according to the feature value. In response to the SNR of the external sound signal exceeding a second threshold value (e.g., 5 dB), the processor120may use the initial value of the voice parameter calculated in operation620, and in response to the SNR of the external sound signal being less than or equal to the second threshold value, the processor120may determine to load and use the initial value stored in the voice parameter initial value information580of the memory130.

In response to the SNR exceeding, for example, 5 dB, the processor120may obtain a target voice parameter by performing an update based on the calculated initial value. In response to the SNR being less than or equal to, for example, 5 dB, when a good initial value is provided, a voice covariance matrix may be estimated (or updated) with high accuracy, and the beamforming performance may be maintained. In response to the SNR being less than or equal to 5 dB, the processor120may determine a target voice covariance matrix used to determine a beamformer filter by loading the initial value (e.g., the initial value stored in operation630) of the voice covariance matrix stored when the SNR exceeds 15 dB and updating the voice covariance matrix accordingly. An operation of the processor120to be performed when the feature value of the external sound signal is less than or equal to the first threshold value and exceeds the second threshold value is described in detail with reference toFIG.10. An operation of the processor120to be performed when the feature value of the external sound signal is less than or equal to the second threshold value and exceeds a third threshold value is described in detail with reference toFIG.11.

According to one embodiment, in operation650, the processor120may obtain the target voice parameter used to perform beamforming based on the determined initial value according to the feature value. In response to the SNR of the external sound signal exceeding the third threshold value (e.g., −5 dB), the processor120may determine the target voice covariance matrix for determining the beamformer filter by updating the voice covariance matrix based on the initial value loaded or calculated as described above with reference to operation640.

In response to the SNR of the external sound signal being less than or equal to the third threshold value (e.g., −5 dB), the processor120may determine to use the initial value stored in the voice parameter initial value information580of the memory130as a voice parameter without an update process. In response to the SNR being less than or equal to −5 dB, for example, the voice covariance matrix may not be estimated because a noise element is dominant over a voice element. Accordingly, in response to the SNR being less than or equal to −5 dB, the processor120may load a good initial value stored in the voice parameter initial value information580of the memory130and use the initial value as the target voice covariance matrix for determining the beamformer filter as it is without an update process. An operation of the processor120to be performed when the feature value of the external sound signal is less than or equal to the third threshold value is described in detail with reference toFIG.8.

The target voice covariance matrix used to determine a customized beamformer filter may be determined through operations610to650of the processor120. Beamforming performance degradation caused by a difference in a wearing style or a structure of the inner ear may not occur even in a low SNR environment.

In operations610to650described above, the first threshold value may be greater than the second threshold value, and the second threshold value may be greater than the third threshold value. The first threshold value may be 15 dB, the second threshold value may be 5 dB, and the third threshold value may be −5 dB, but these are merely examples and not limited thereto.

FIG.7is a flowchart illustrating a noise processing method of an electronic device according to an embodiment of the disclosure.

Operations710to730may be performed by the processor120of the electronic device101described above with reference toFIG.5B. Thus, any description overlapping the description referring toFIGS.1to4,5A and5B, and6will not be repeated for conciseness. Operations710to730may correspond to an operation540of estimating noise and a noise suppression operation550of removing noise after beamforming has been performed described above with reference toFIG.5A.

According to one embodiment, the processor120of the electronic device101may perform operations710to730after an operation (e.g., operation650ofFIG.6) of obtaining a target voice parameter based on a feature value of an external sound signal has been performed.

Referring toFIG.7, according to one embodiment, in operation710, the processor120may determine a beamformer filter used to perform beamforming on the external sound signal based on the target voice parameter obtained through operations610to650ofFIG.6. For example, the processor120may obtain a target voice covariance matrix Cx with high accuracy by performing operations described above with reference toFIG.6, and the processor120may adjust the beamformer filter through the target covariance matrix Cx as described above with reference toFIGS.4and5A. For example, as described above with reference toFIG.4, the processor120may determine a beamformer filter that steers toward the user's mouth and a beamformer filter that generates a null vector toward the user's mouth based on the obtained Cx.

According to one embodiment, the processor120may estimate a magnitude of residual noise with respect to a signal on which beamforming is performed using the determined filter in operation720, and the processor120may perform noise processing (e.g., noise suppression) on the signal on which beamforming is performed according to the magnitude of the residual noise in operation730.

The electronic device101may obtain a voice signal of improved quality by determining the beamformer filter and performing noise processing through operations710to730based on the target voice covariance matrix, which is the target voice parameter.

According to one embodiment, the electronic device101may perform a separate training mode to obtain a good initial value of a voice parameter. The processor120may output a guide interface that asks a user to perform an utterance to calculate an initial value of a voice parameter to be stored in the voice parameter initial value information580of the memory130in an environment in which a feature value (e.g., an SNR) exceeds a first threshold value (e.g., 15 dB). For example, a user interface (UI) that asks the user to perform an utterance in an environment with a high feature value, such as a quiet environment, through a display of the electronic device101or the electronic device102(e.g., the electronic device102ofFIG.3) interoperating with the electronic device101. For example, a UI that asks the user to utter a predetermined or random sentence in a quiet environment may be audibly output to the user through the electronic device101. As another example, a similar UI may be visually output to the user through the electronic device102interoperating with the electronic device101.

Referring toFIG.6, the processor120may calculate the initial value of the voice parameter based on a user utterance received by a plurality of microphones and store the calculated initial value of the voice parameter in the memory130. The processor120may calculate an initial value of a voice covariance matrix according to the user utterance received by the plurality of microphones. In response to the external sound signal exceeding the first threshold value, the processor120may store the calculated initial value in the voice parameter initial value information580of the memory130as in an operation (e.g., operation630ofFIG.6) of the processor120. In the embodiment of a training mode described above, the processor120may obtain and store a good initial value more reliably than in an embodiment of storing an initial value when a feature value is high in a general situation as in operation630ofFIG.6.

According to one embodiment, the processor120of the electronic device101may control beamforming more specifically based on speech power and a direction of arrival (DoA) of the external sound signal. For example, the voice parameter initial value information580of the memory130described above with reference toFIG.5Bmay further include index information according to information on the speech power and DoA. The processor120may determine the speech power and DoA of the external sound signal input to the plurality of microphones and further consider the information on the speech power and DoA.

For example, when the processor120stores information on the calculated initial value of the voice parameter in the memory130(e.g., when the feature value exceeds the first threshold value in operation630ofFIG.6), the processor120may store the information on the calculated initial value in the voice parameter initial value information580of the memory130by classifying the information on the calculated initial value according to the information on the speech power and DoA. As another example, when the processor120loads information on the initial value of the voice parameter stored in the memory130instead of the information on the calculated initial value of the voice parameter (e.g., when the feature value is less than or equal to the second threshold value in operation640ofFIG.6), the processor120may load the information on the stored initial value from the voice parameter initial value information580of the memory130by classifying the information on the stored initial value according to the information on the speech power and DoA.

FIGS.8,9,10, and11are flowcharts illustrating an operating method of an electronic device according to a feature value of an external sound signal according to various embodiments of the disclosure.

Operations ofFIGS.8to11(e.g., operations810to850and operations910and920, operation1010, and operation1110) may be performed by the processor120of the electronic device101described above with reference toFIG.5B. Thus, any description overlapping the description provided with reference toFIG.7will not be repeated for conciseness. Operations ofFIGS.8to11may be operations (e.g., operations610to650) of the processor120described above with reference toFIG.6identified according to a feature value of an external sound signal received by a plurality of microphones.

Referring toFIG.6, a feature value inFIGS.8to11may be an SNR or noise power, and a voice parameter may be the voice covariance matrix Cx used to determine the beamformer filter in the signal-dependent beamformer described above with reference toFIG.4. A first threshold value inFIGS.8to11may be greater than a second threshold value, and the second threshold may be greater than a third threshold value. The first threshold value may be 15 dB, the second threshold value may be 5 dB, and the third threshold value may be −5 dB, but these are merely examples and examples are not limited thereto.

FIG.8is a flowchart illustrating an operation of a processor in response to a feature value of an external sound signal being less than or equal to the third threshold value among the operations of a processor described above with reference toFIG.6according to an embodiment of the disclosure.

Referring toFIG.8, according to one embodiment, in operation810, a processor120may estimate a feature value of an external sound signal received by a plurality of microphones. Descriptions of operation610provided with reference toFIG.6are applicable to operation810, and thus any repeated description thereof has been omitted.

According to one embodiment, in operations820to840, the processor120may identify the feature value based on the first threshold value, the second threshold value, and the third threshold value.

According to one embodiment, in operation850, in response to the feature value of the external sound signal being less than or equal to the third threshold value, the processor120may obtain an initial value of a voice parameter stored in the memory130as a voice parameter used to perform beamforming on the external sound signal.

As described above with reference to operation630ofFIG.6, in operation850, in response to an SNR of the external sound signal being less than or equal to the third threshold value (e.g., −5 dB), which is less than or equal to the first threshold value (e.g., 15 dB), the processor120may determine not to store an initial value (e.g., the initial value calculated in operation620ofFIG.6) calculated based on the external sound signal received by the plurality of microphones in the voice parameter initial value information580of the memory130.

As described with reference to operation640ofFIG.6, in response to the SNR of the external sound signal being less than or equal to the third threshold value (e.g., −5 dB), which is less than or equal to the second threshold value (e.g., 5 dB), the processor120may determine to use the initial value stored in the voice parameter initial value information580of the memory130rather than the initial value (e.g., the initial value calculated in operation620ofFIG.6) calculated based on the external sound signal received by the plurality of microphones.

As described with reference to operation650ofFIG.6, in response to the SNR of the external sound signal being less than or equal to the third threshold value (e.g., −5 dB), the processor120may not update the initial value stored in the voice parameter initial value information580of the memory130and determine that the initial value is a target voice parameter used to determine a beamformer filter. The processor120may determine a beamformer filter based on the target voice covariance matrix obtained in operation850and perform noise processing.

FIG.9is a flowchart illustrating an operation of a processor when a feature value of an external sound signal exceeds a first threshold value among the operations of a processor described with reference toFIG.6according to an embodiment of the disclosure.

Referring toFIG.9, according to an embodiment, a processor120may estimate (e.g., operation810ofFIG.8) a feature value of an external sound signal received by a plurality of microphones and perform operations910and920in response to the feature value exceeding the first threshold value (e.g., if “Yes” in operation820ofFIG.8).

According to one embodiment, in operation910, in response to the feature value of the external sound signal exceeding the first threshold value, the processor120may store a calculated initial value of a voice parameter in the memory130. As described above with reference to operation630ofFIG.6, in operation910, in response to an SNR of the external sound signal exceeding the first threshold value (e.g., 15 dB), the processor120may store an initial value (e.g., the initial value calculated in operation620ofFIG.6) calculated based on the external sound signal received by the plurality of microphones in the voice parameter initial value information580of the memory130.

According to one embodiment, in operation920, in response to the feature value of the external sound signal exceeding the first threshold value, the processor120may obtain a target voice parameter used to perform beamforming on the external sound signal by updating the voice parameter based on the calculated initial value of the voice parameter.

As described above with reference to operation640ofFIG.6, in operation920, in response to the SNR of the external sound signal exceeding the first threshold value (e.g., 15 dB) and accordingly exceeding a second threshold value (e.g., 5 dB), the processor120may determine to use the calculated initial value (e.g., the initial value calculated in operation620ofFIG.6) rather than the stored initial value.

As described above with reference to operation650ofFIG.6, in response to the SNR of the external sound signal exceeding the first threshold value (e.g., 15 dB) and accordingly exceeding a third threshold value (e.g., −5 dB), the processor120may obtain a target voice parameter used to determine a beamformer filter by updating the calculated initial value. For example, the processor120may determine a target voice covariance matrix used to determine a beamformer filter by updating a calculated initial value of a voice covariance matrix Cx. The processor120may determine a beamformer filter based on the target voice covariance matrix obtained in operation920and perform noise processing.

FIG.10is a flowchart illustrating an operation of a processor in response to a feature value of an external sound signal being less than or equal to a first threshold value and exceeding a second threshold value among the operations of a processor described with reference toFIG.6according to an embodiment of the disclosure.

Referring toFIG.10, according to an embodiment, the processor120may estimate (e.g., operation810ofFIG.8) a feature value of an external sound signal received by a plurality of microphones and perform operation1010in response to the feature value being less than or equal to the first threshold value and exceeding the second threshold value (e.g., if “Yes” in operation830ofFIG.8).

According to one embodiment, in operation1010, in response to the feature value of the external sound signal being less than or equal to the first threshold value and exceeding the second threshold value, the processor120may obtain a target voice parameter by updating a voice parameter based on a calculated initial value of the voice parameter.

As described above with reference to operation630ofFIG.6, in operation1010, in response to an SNR of the external sound signal being less than or equal to the first threshold value (e.g., 15 dB), the processor120may determine not to store the calculated initial value (e.g., the initial value calculated in operation620ofFIG.6) in the voice parameter initial value information580of memory130.

As described above with reference to operation640ofFIG.6, in operation1010, the processor120may determine to use the calculated initial value (e.g., the initial value calculated in operation620ofFIG.6) rather than a stored initial value because the SNR of the external sound signal exceeds the second threshold value (e.g., 5 dB).

As described above with reference to operation650ofFIG.6, in response to the SNR of the external sound signal exceeding the second threshold value (e.g., 5 dB) and accordingly exceeding a third threshold value (e.g., −5 dB), the processor120may obtain the target voice parameter used to determine a beamformer filter by updating the calculated initial value. For example, the processor120may determine a target voice covariance matrix used to determine a beamformer filter by updating a calculated initial value of a voice covariance matrix Cx. The processor120may determine a beamformer filter based on the target voice covariance matrix obtained in operation1010and perform noise processing.

FIG.11is a flowchart illustrating an operation of a processor when a feature value of an external sound signal is less than or equal to a second threshold value and exceeds a third threshold value among the operations of a processor described with reference toFIG.6according to an embodiment of the disclosure.

Referring toFIG.11, according to an embodiment, a processor120may estimate (e.g., operation810ofFIG.8) a feature value of an external sound signal received by a plurality of microphones and perform operation1110in response to the feature value being less than or equal to the second threshold value and exceeding the third threshold value (e.g., if “Yes” in operation840ofFIG.8).

According to one embodiment, in operation1110, in response to the feature value of the external sound signal being less than or equal to the second threshold value and exceeding the third threshold value, the processor120may obtain a target voice parameter by updating a voice parameter based on an initial value of the voice parameter loaded from a memory.

As described above with reference to operation630ofFIG.6, in operation1110, in response to an SNR of the external sound signal being less than or equal to the second threshold value (e.g., 5 dB) and accordingly being less than or equal to the first threshold value (e.g., 15 dB), the processor120may determine not to store a calculated initial value (e.g., the initial value calculated in operation620ofFIG.6) in the voice parameter initial value information580of the memory130.

As described above with reference to operation640ofFIG.6, in operation1110, in response to the SNR of the external sound signal being less than or equal to the second threshold value (e.g., 5 dB), the processor120may determine to use a stored initial value and load an initial value from the voice parameter initial value information580.

As described above with reference to operation650ofFIG.6, in response to the SNR of the external sound signal exceeding the third threshold value (e.g., −5 dB), the processor120may obtain a target voice parameter used to determine a beamformer filter by updating an initial value. For example, the processor120may obtain a target voice covariance matrix used to determine the beamformer filter by updating an initial value of a voice covariance matrix Cx loaded from the voice parameter initial value information580of the memory130. The processor120may determine the beamformer filter based on the target voice covariance matrix obtained in operation1110and perform noise processing.

According to one embodiment, an electronic device101may include an input module150including a plurality of microphones150-1and150-2configured to receive an external sound signal, a memory130configured to store computer-executable instructions and an initial value information580of a voice parameter Cx used to perform beamforming on the external sound signal, and a processor120configured to execute the instructions by accessing the memory130, wherein the instructions may be configured to estimate a feature value (an SNR or noise power) of the external sound signal, calculate an initial value of a voice parameter used to perform beamforming based on the external sound signal received by the plurality of microphones, determine whether to store the calculated initial value in the memory130according to the feature value, determine which one of the calculated initial value or the initial value stored in the memory130is to be used according to the feature value, and obtain a target voice parameter used to perform beamforming on the external sound signal based on the determined initial value according to the feature value.

According to one embodiment, the instructions may be further configured to determine a filter used to perform beamforming on the external sound signal based on the target voice parameter.

According to one embodiment, the electronic device101may further include an audio module170configured to perform noise processing on an audio signal, and the instructions may be further configured to estimate a magnitude of residual noise with respect to a signal on which beamforming is performed using the filter and perform noise processing on the signal on which beamforming is performed according to the estimated magnitude of the residual noise.

According to one embodiment, the instructions may be further configured to, in response to the feature value exceeding a first threshold value (e.g., 15 dB), store the calculated initial value of the voice parameter in the memory130and obtain the target voice parameter by updating a voice parameter based on the calculated initial value of the voice parameter.

According to one embodiment, the instructions may be further configured to, in response to the feature value being less than or equal to the first threshold value (e.g., 15 dB) and exceeding a second threshold value (e.g., 5 dB), obtain the target voice parameter by updating a voice parameter based on the calculated initial value of the voice parameter.

According to one embodiment, the instructions may be further configured to, in response to the feature value being less than or equal to a second threshold value (e.g., 5 dB) and exceeding a third threshold value (e.g., −5 dB), obtain the target voice parameter by updating a voice parameter based on the initial value stored in the memory130.

According to one embodiment, the instructions may be further configured to, in response to the feature value being less than or equal to a third threshold value (e.g., −5 dB), obtain the initial value of the voice parameter stored in the memory130as the target voice parameter.

According to one embodiment, instructions may be further configured to output a guide interface configured to ask a user to perform an utterance to calculate an initial value of a voice parameter to be stored in the memory130, calculate the initial value of the voice parameter based on a user utterance received by the plurality of microphones, and store the calculated initial value of the voice parameter in the memory130.

According to one embodiment, the initial value information580of the voice parameter stored in the memory130may be classified according to a magnitude of a sound signal, and the instructions may be further configured to calculate a magnitude of the external sound signal, load the initial value stored in the memory130according to the magnitude of the external sound signal in response to the feature value being less than or equal to a second threshold value (e.g., 5 dB), and obtain the target voice parameter based on the loaded initial value.

According to one embodiment, the initial value of the voice parameter stored in the memory130may be classified according to a direction of a sound signal, and the instructions may be further configured to determine a direction of the external sound signal, load the initial value stored in the memory130according to the direction of the external sound signal in response to the feature value being less than or equal to a second threshold value (e.g., 5 dB), and obtain the target voice parameter based on the loaded initial value.

According to one embodiment, the feature value may be one of an SNR value or noise power, and the voice parameter may be a voice covariance matrix.

According to one embodiment, the plurality of microphones may include an external microphone (e.g., first microphone150-1) placed on one side of the electronic device101, and the electronic device101may further include an in-ear microphone150-2and an accelerator502.

According to one embodiment, the electronic device101may be one of a true-wireless stereo (TWS) earphone, headphones, or a headset.

According to one embodiment, a method in which the electronic device101obtains a voice parameter used to perform beamforming may include estimating a feature value of an external sound signal received by a plurality of microphones, calculating an initial value of a voice parameter used to perform beamforming on the external sound signal, determining whether to store the calculated initial value according to the feature value, determining which one of the calculated initial value or the stored initial value is to be used according to the feature value, and obtaining a target voice parameter used to perform beamforming on the external sound signal based on the determined initial value according to the feature value.

According to one embodiment, the determining of whether to store the calculated initial value according to the feature value may include storing the calculated initial value of the voice parameter in response to the feature value exceeding a first threshold value (e.g., 15 dB), and not storing the calculated initial value of the voice parameter in response to the feature value being less than or equal to the first threshold value (e.g., 15 dB).

According to one embodiment, the determining of which one of the calculated initial value or the stored initial value is to be used according to the feature value may include using the calculated initial value in response to the feature value exceeding a second threshold value (e.g., 5 dB), and using the stored initial value in response to the feature value being less than or equal to the second threshold value (e.g., 5 dB).

According to one embodiment, the obtaining of a target voice parameter used to perform beamforming on the external sound signal based on the determined initial value according to the feature value may include obtaining the target voice parameter by updating a voice parameter based on the determined initial value in response to the feature value exceeding a third threshold value (e.g., −5 dB), and obtaining the determined initial value as the target voice parameter in response to the feature value being less than or equal to the third threshold value (e.g., −5 dB).

According to one embodiment, an operating method of the electronic device101may include estimating a feature value of an external sound signal received by the plurality of microphones150-1and150-2, calculating an initial value of a voice parameter used to perform beamforming on the external sound signal, determining whether to store the calculated initial value according to the feature value, determining which one of the calculated initial value or the stored initial value is to be used according to the feature value, obtaining a target voice parameter used to perform beamforming on the external sound signal based on the determined initial value according to the feature value, determining a filter used to perform beamforming on the external sound signal based on the target voice parameter, estimating a magnitude of residual noise with respect to a signal on which beamforming is performed using the filter, and performing noise processing on the signal on which beamforming is performed according to the estimated magnitude of the residual noise.

According to one embodiment, the electronic device101may be one of a true-wireless stereo (TWS) earphone, headphones, or a headset.