Patent Publication Number: US-2023162739-A1

Title: Electronic apparatus, system comprising sound i/o device and controlling method thereof

Description:
TECHNICAL FIELD 
     The disclosure relates to an electronic apparatus that provides a service to a user based on a sound signal, a system including a sound input/output device, and a control method thereof. 
     CROSS-REFERENCE TO RELATED APPLICATION(S) 
     This application is based on and claims priority to Korean Patent Application No. 10-2021-0014270, filed on Feb. 1, 2021, in the Korean Intellectual Property Office, the disclosure of which is incorporated by reference herein in its entirety. 
     BACKGROUND ART 
     Technologies relating to a method of providing a voice recognition assistant service through a mobile electronic apparatus are being actively developed. An electronic apparatus that provides a voice recognition assistant service can identify a user&#39;s voice and other sounds by performing an artificial intelligence (AI)-based sound recognition operation. 
     However, in the case of performing a sound recognition operation entirely by an electronic apparatus, the electronic apparatus must be always on, and thus there is a problem due to the burden in power consumption. Accordingly, there has been a continuous demand for a method of identifying various types of sounds including a user&#39;s voice correctly, at the same time as alleviating the burden in power consumption of an electronic apparatus. 
     DISCLOSURE 
     Technical Problem 
     The disclosure may address the aforementioned need, and the disclosure provides an electronic apparatus that identifies various types of sounds included in an input sound through several stages, and performs different operations based on the identified types of sounds, and a control method thereof. 
     Technical Solution 
     According to an aspect of the disclosure, an electronic apparatus may include a communication interface; and a processor configured to: control the communication interface to output an audio content signal to a sound input/output device including a speaker and a microphone; based on receiving a sound signal collected via the microphone from the sound input/output device via the communication interface, identify whether the sound signal includes a scene noise signal corresponding to a regular noise generated in a location in which the sound input/output device is located or an event noise signal corresponding to an irregular noise generated in the location in which the sound input/output device is located; based on identifying that the sound signal includes the scene noise signal, perform noise cancelling for the sound signal; and based on identifying that the sound signal includes the event noise signal, control the output of the audio content signal. 
     The processor may, based on the sound signal received from the sound input/output device being a signal not including a user voice input, identify whether the sound signal includes the scene noise signal or the event noise signal. 
     The processor may, based on the sound signal received from the sound input/output device being a signal including a user voice input, identify whether the sound signal includes a wake-up word; and based on the sound signal including the wake-up word, perform a voice recognition assistant function. 
     The processor may, based on the sound signal including the event noise signal, perform an operation related to at least one of stopping of the output of the audio content signal, adjusting of an output volume of the audio content signal, or providing of a feedback corresponding to the event noise signal. 
     The processor may input the sound signal into a first neural network model; and identify whether the sound signal includes the scene noise signal or the event noise signal based on an output of the first neural network, wherein the first neural network model is trained to, based on the sound signal being input, output information indicating whether the input sound signal is the scene noise signal or the event noise signal. 
     The processor may, based on identifying that a first type of a first scene noise signal received from the sound input/output device during a first time frame is different from a second type of a second scene noise signal received from the sound input/output device during a second time frame which is before the first time frame, transmit, to the sound input/output device via the communication interface, a control signal that causes the sound input/output device to collect a noise signal during a third time frame which is after the first time frame. 
     The processor may, based on a first type of a first scene noise signal received from the sound input/output device during a fourth time frame not being identified, transmit, to the sound input/output device via the communication interface, a control signal that causes the sound input/output device to collect a noise signal during a fifth time frame which is after the fourth time frame. 
     The processor may include an application processor; and a main processor. The main processor may, based on receiving the sound signal from the sound input/output device, control the application processor to be powered on. The application processor may identify whether the sound signal includes the scene noise signal or the event noise signal. 
     According to an aspect of the disclosure, a system may include a sound input/output device and configured to: output, via a speaker, an audio content signal received from an electronic apparatus, identify whether a sound signal collected via a microphone includes a user voice input, and transmit the sound signal and an identification result to the electronic apparatus. The system may include the electronic apparatus configured to: receive the sound signal collected via the microphone and the identification result from the sound input/output device, based on identifying that the sound signal does not include the user voice input based on the identification result, identify whether the sound signal includes a scene noise signal corresponding to a regular noise generated in a location in which the sound input/output device is located or an event noise signal corresponding to an irregular noise generated in the location in which the sound input/output device is located, based on the sound signal including the scene noise signal, perform noise cancelling for the sound signal, and based on the sound signal including the event noise signal, control an output of the audio content signal. 
     The sound input/output device may include a plurality of microphones that are provided in locations distanced from one another. The sound input/output device may identify whether the sound signal collected via the microphone includes the user voice input based on strength differences of sound signals received via the plurality of microphones. 
     The sound input/output device may input information related to whether the sound signal is a signal related to the user voice input and the sound signals collected via the plurality of microphones into a second neural network model; and identify whether the sound signal includes the user voice input based on an output of the second neural network model. 
     According to an aspect of an example embodiment, a control method of an electronic apparatus may include outputting an audio content signal to a sound input/output device worn by a user including a speaker and a microphone; based on receiving a sound signal collected via the microphone from the sound input/output device, identifying whether the sound signal includes a scene noise signal corresponding to a regular noise generated in a location in which the sound input/output device is located or an event noise signal corresponding to an irregular noise generated in the location in which the sound input/output device is located; based on identifying that the sound signal includes the scene noise signal, performing noise cancelling for the sound signal; and based on identifying that the sound signal includes the event noise signal, controlling the output of the audio content signal. 
     The identifying whether the sound signal includes the scene noise signal or the event noise signal may include, based on the sound signal received from the sound input/output device being a signal not including a user voice input, identifying whether the sound signal includes the scene noise signal or the event noise signal. 
     The method may include, based on the sound signal received from the sound input/output device being a signal including a user voice input, identifying whether the sound signal includes a wake-up word; and based on the sound signal including the wake-up word, performing a voice recognition assistant function. 
     The method may include, based on the sound signal including the event noise signal, performing an operation related to at least one of stopping of the output of the audio content signal, adjusting of an output volume of the audio content signal, or providing of a feedback corresponding to the event noise signal. 
     Effect of Invention 
     According to the various embodiments of the disclosure, an electronic apparatus can identify various types of sounds included in an input sound correctly while consuming low power, and thus the satisfaction of a user who is provided with a voice recognition assistant service can be enhanced. 
    
    
     
       DESCRIPTION OF DRAWINGS 
       The above and other aspects, features, and advantages of certain embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which: 
         FIG.  1    is a diagram for illustrating an appearance of a user located in a space wherein various types of sounds exist using an electronic apparatus; 
         FIG.  2    is a block diagram for illustrating a configuration of an electronic apparatus according to an embodiment of the disclosure; 
         FIG.  3    is a block diagram for illustrating functional configurations of an electronic apparatus and a sound input/output device according to an embodiment of the disclosure; 
         FIG.  4    is a block diagram for illustrating in detail functional configurations of an electronic apparatus and a sound input/output device according to an embodiment of the disclosure; 
         FIG.  5 A  is a diagram for illustrating different types of scene noise signals corresponding to characteristics of a space; 
         FIG.  5 B  is a diagram for illustrating different types of scene noise signals corresponding to characteristics of a space; 
         FIG.  6 A  is a diagram for illustrating event noise signals that are generated irregularly in a specific space; 
         FIG.  6 B  is a diagram for illustrating event noise signals that are generated irregularly in a specific space; 
         FIG.  7    is a diagram for illustrating a wake-up word identification operation of an electronic apparatus according to an embodiment of the disclosure; 
         FIG.  8    is a diagram for illustrating an operation of an electronic apparatus according to an embodiment of the disclosure of providing a user interface (UI) corresponding to an event noise to a user; 
         FIG.  9 A  is a diagram for illustrating various neural network models according to an embodiment of the disclosure; 
         FIG.  9 B  is a diagram for illustrating various neural network models according to an embodiment of the disclosure; 
         FIG.  9 C  is a diagram for illustrating various neural network models according to an embodiment of the disclosure; 
         FIG.  10    is a diagram for illustrating an operation of an electronic apparatus according to an embodiment of the disclosure of identifying characteristics of a space that change according to movements of a user; 
         FIG.  11    is a block diagram for illustrating in detail a configuration of an electronic apparatus according to an embodiment of the disclosure; and 
         FIG.  12    is a flow chart for illustrating a control method according to an embodiment of the disclosure. 
     
    
    
     DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 
     Hereinafter, the disclosure will be described in detail with reference to the accompanying drawings. 
     As terms used in the embodiments of the disclosure, general terms that are currently used widely were selected as far as possible, in consideration of the functions described in the disclosure. However, the terms may vary depending on the intention of those skilled in the art who work in the pertinent field, previous court decisions, or emergence of new technologies. Also, in particular cases, there are terms that were designated by the applicant on his own, and in such cases, the meaning of the terms will be described in detail in the relevant descriptions in the disclosure. Thus, the terms used in the disclosure should be defined based on the meaning of the terms and the overall content of the disclosure, but not just based on the names of the terms. 
     Also, in the disclosure, expressions such as “have,” “may have,” “include,” and “may include” should be construed as denoting that there are such characteristics (e.g., elements such as numerical values, functions, operations, and components), and the terms are not intended to exclude the existence of additional characteristics. 
     In addition, the expression “at least one of A and/or B” should be interpreted to mean any one of “A,” “B,” or “A and B.” 
     Further, the expressions “first,” “second,” and the like, used in the disclosure may be used to describe various elements regardless of any order and/or degree of importance. Also, such expressions are used only to distinguish one element from another element, and are not intended to limit the elements. 
     The description in the disclosure that one element (e.g., a first element) is “(operatively or communicatively) coupled with/to” or “connected to” another element (e.g., a second element) should be interpreted to include both the case where the one element is directly coupled to the another element, and the case where the one element is coupled to the another element through still another element (e.g., a third element). 
     Also, singular expressions include plural expressions, unless defined obviously differently in the context. Further, in the disclosure, terms such as “include” and “comprise” should be construed as designating that there are such characteristics, numbers, steps, operations, elements, components, or a combination thereof described in the specification, but not as excluding in advance the existence or possibility of adding one or more of other characteristics, numbers, steps, operations, elements, components, or a combination thereof. 
     In addition, in the disclosure, “a module” or “a part” performs at least one function or operation, and it may be implemented as hardware or software, or as a combination of hardware and software. Further, a plurality of “modules” or “parts” may be integrated into at least one module and implemented as at least one processor (not shown), except “modules” or “parts” that need to be implemented as specific hardware. 
     Also, in the disclosure, the term “user” may refer to a person who uses an electronic apparatus. Hereinafter, an embodiment of the disclosure will be described in more detail with reference to the accompanying drawings. 
       FIG.  1    is a diagram for illustrating an appearance of a user located in a space wherein various types of sounds exist using an electronic apparatus. 
     Referring to  FIG.  1   , a user  10  who is using the electronic apparatus  100  is riding on the subway. The electronic apparatus  100  may provide an AI voice recognition assistant function, and the voice recognition assistant function according to an embodiment of the disclosure may mean the entire services of providing response information to a user when a user&#39;s voice input is input into the electronic apparatus  100  based on the user uttering a voice input. 
     In the space wherein the user  10  is located, various types of sounds may be generated. For example, for the user  10  who is located in an inner space of the subway, sounds corresponding to regular noises that are generated according to the operation of the subway, and noises in the case of emergency related to the operation of the subway (e.g., a guide voice, etc.) may be generated. 
     In such an environment, in case the user  10  inputs a voice input to be provided with a voice recognition assistant service from the electronic apparatus  100 , there may be a case where the electronic apparatus  100  cannot identify the voice of the user  10  correctly due to various types of sounds that are generated in the surroundings. 
     Accordingly, various embodiments wherein various types of sounds included in a sound input into the electronic apparatus  100  are identified through several stages, and different operations are performed based on the identified types of sounds will be described in more detail. 
     In this specification, various types of noises that are generated in the surroundings of the user as well as the voice of the user  10  will be described by using the term ‘sound’ in general. Also, ‘a noise’ and a Korean expression having the same meaning will be interchangeably used in the specification. 
       FIG.  2    is a block diagram for illustrating a configuration of an electronic apparatus according to an embodiment of the disclosure. 
     Referring to  FIG.  2   , the electronic apparatus  100  according to an embodiment of the disclosure may include a communication interface  110  and a processor  120 . 
     The communication interface  110  may input and output various types of data. For example, the communication interface  110  may transmit and receive various types of data with an external apparatus (e.g., a source apparatus), an external storage medium (e.g., a universal serial bus (USB) memory), and an external server (e.g., a webhard) through communication methods such as wireless fidelity (Wi-Fi) based on access point (AP) (e.g., Wi-Fi, a wireless local area network (LAN), etc.), Bluetooth, Zigbee, wired LANs, wireless LANs, a wide area network (WAN), Ethernet, IEEE 1394, a high-definition multimedia interface (HDMI), USB, a mobile high-definition link (MHL), Audio Engineering Society/European Broadcasting Union (AES/EBU), optical, coaxial, etc. 
     The processor  120  controls the overall operations of the electronic apparatus  100 . Specifically, the processor  120  may be connected with the respective components of the electronic apparatus  100  and control the overall operations of the electronic apparatus  100 . For example, the processor  120  may be connected with the communication interface  110  and control the operations of the electronic apparatus  100 . 
     According to an embodiment of the disclosure, the processor  120  may be referred to by various names such as a digital signal processor (DSP), a microprocessor, a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a neural processing unit (NPU), a controller, an application processor (AP), etc., but in this specification, it will be described as the processor  120 . 
     Also, the processor  120  may be implemented as a system on chip (SoC), or a large scale integration (LSI), and it may also be implemented in the form of a field programmable gate array (FPGA). In addition, the processor  120  may include a volatile memory such as a static random access memory (SRAM), etc. 
     A function related to AI according to the disclosure may be executed through the processor  120  and a memory. The processor  120  may include one or a plurality of processors. The one or plurality of processors may be generic-purpose processors such as a CPU, an AP, a DSP, etc., graphic-dedicated processors such as a graphics processing unit (GPU), a vision processing unit (VPU), etc., or AI-dedicated processors such as an NPU. The one or plurality of processors  120  perform control to process input data according to predefined operation rules or a neural network model stored in the memory. Alternatively, in case the one or plurality of processors  120  are AI-dedicated processors, the AI-dedicated processors may be designed in a hardware structure specified for processing of a specific neural network model. 
     The predefined operation rules or the artificial intelligence model are characterized in that they are made through learning. The feature of being made through learning means that a basic neural network model is trained by using a plurality of learning data by a learning algorithm, and predefined operation rules or an AI model set to perform a desired characteristic (or, purpose) are thereby made. Such learning may be performed in a device itself wherein AI is performed according to the disclosure, or performed through a separate server and/or system. As examples of learning algorithms, there are supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but learning algorithms are not limited to the aforementioned examples. 
     The processor  120  according to an embodiment of the disclosure may control the communication interface  110  to output an audio content signal to a sound input/output device including a speaker and a microphone. 
     The sound input/output device may be a wearable device that a user wears. Also, the sound input/output device according to an embodiment of the disclosure may be implemented as an earphone, and in this case, the sound input/output device may include a separate communication interface, and electronically communicate with the electronic apparatus  100 . 
     The sound input/output device according to an embodiment of the disclosure may output an audio content signal received from the electronic apparatus  100  through the speaker. Also, the sound input/output device may identify whether a sound signal collected through the microphone includes a user voice, and transmit the collected sound signal and the identification result to the electronic apparatus  100 . 
     In addition, if a sound signal collected via the microphone is received from the sound input/output device via the communication interface  110 , the processor  120  may identify whether the sound signal includes a scene noise signal or an event noise signal. 
     A scene noise signal may be a signal corresponding to a noise that is regularly generated in a space wherein a user is located. Specifically, a scene noise signal may be a signal corresponding to a regular noise that is generated according to the operation of the subway, a regular noise that is generated by birds that live in a park, or a noise such as a wind noise that is generated by a vehicle running on a roadside, etc. A regular noise may refer to a noise that occurs at uniform, consistent, rhythmic, etc., intervals at a particular location. Further, a regular noise may refer to a noise that is expected at the location, and that does not warrant much attention from the user. 
     An event noise signal may be a signal corresponding to a noise that is irregularly (unexpectedly) generated in a location wherein a user is located. Specifically, an event noise signal may be a signal corresponding to a noise in the case of emergency related to the operation of the subway (e.g., a guide voice, etc.), a barking sound of a dog of a pedestrian taking a walk in a park, or a noise such as a honk that is generated by a vehicle running on a roadside, etc. An irregular noise may refer to a noise that does not occur at uniform, consistent, rhythmic, etc., intervals at a particular location. Further, an irregular noise may refer to a noise that is unexpected at the location, and that might warrant attention from the user. 
     If the sound signal received from the sound input/output device includes a scene noise signal, the processor  120  may perform noise cancelling for the sound signal. 
     Noise canceling may mean an operation of the electronic apparatus  100  of generating a signal of a wavelength that is opposite to the wavelength of a signal identified as a noise to be removed among signals included in a sound signal, and removing the noise by offset interference between the signals. 
     If the sound signal received from the sound input/output device includes an event noise signal, the processor  120  according to an embodiment of the disclosure may control the output of the audio content signal output from the sound input/output device. 
     Specifically, if the sound signal received from the sound input/output device includes an event noise signal, the processor  120  according to an embodiment of the disclosure may perform an operation related to at least one of stopping of the output of the audio content signal, adjusting of the output volume, or providing of a feedback corresponding to the event noise signal. Detailed explanation in this regard will be made with reference to  FIG.  8   . 
     If the sound signal received from the sound input/output device is a signal not including a user voice, the processor  120  may identify whether the sound signal includes a noise signal or an event signal. 
     If the sound signal received from the sound input/output device is a signal including a user voice input, the processor  120  may identify whether the sound signal includes a wake-up word. 
     A wake-up word means a word or a sentence that can activate the voice recognition assistant function provided by the electronic apparatus  100 , among words or sentences included in a user voice input. A wake-up word can be set in advance in the manufacturing step of the electronic apparatus  100 , and editing such as addition, deletion, etc., is possible according to a user&#39;s setting. As another example, a wake-up word may be changed or added through a firmware update, etc. 
     If it is identified that the sound signal includes a wake-up word, the processor  120  may perform the voice recognition assistant function. 
     The processor  120  according to an embodiment of the disclosure may input a sound signal into a first neural network model, and identify whether the sound signal includes a scene noise signal or an event noise signal. 
     Also, the first neural network model may be a model trained to, if a sound signal is input, output information indicating whether the input sound signal is a scene noise signal or an event noise signal. 
     If it is identified that the type of a scene noise signal received from the sound input/output device during a first threshold time is different from the type of a scene noise signal received during a second threshold time which is before the first threshold time, the processor  120  according to an embodiment of the disclosure may transmit a control signal making the sound input/output device collect a noise signal during a third threshold time which is after the first threshold time to the sound input/output device through the communication interface  110 . 
     Also, in case the type of a scene noise signal received from the sound input/output device during a fourth threshold time is not identified, the processor  120  according to an embodiment of the disclosure may transmit a control signal making the sound input/output device collect a noise signal during a fifth threshold time which is after the fourth threshold time to the sound input/output device through the communication interface  110 . The aforementioned operation of transmitting a control signal will be described in detail with reference to  FIG.  10   . 
     The processor  120  according to an embodiment of the disclosure may include an AP and a main processor. 
     The AP may be a processor that is implemented in a form wherein a plurality of units performing various functions including a CPU performing main operations are integrated in one chip. In the AP according to an embodiment of the disclosure, a CPU, a memory, and a GPU, etc., may be included. For this reason, the application processor is also referred to as a system on chip (SoC). 
     The main processor may be a processor that controls the overall operations of the electronic apparatus  100  including the AP, and that manages to provide power for the components of the electronic apparatus  100  correctly and effectively. The main processor according to an embodiment of the disclosure may provide power supplied by a power supply part included in the electronic apparatus  100  to the AP, and the main processor is also referred to as a power management integrated circuit (PMIC). 
     The main processor according to an embodiment of the disclosure may control the AP to be powered on if a sound signal is received from the sound input/output device. The feature that the AP is powered on may mean that the AP was not consuming power previously, and then perform an operation by consuming power after it is powered on, and may also mean that the AP was consuming a first power (a standby power) previously, and then perform an operation by consuming a second power (an operating power) after it is powered on. 
     The AP according to an embodiment of the disclosure may identify whether a sound signal includes a scene noise signal or an event noise signal. 
     As a result, in case in which a sound signal is not received from the sound input/output device, the AP does not consume power or consumes only a small amount of power, and thus the standby power consumed by the electronic apparatus  100  can be reduced on the whole. 
       FIG.  3    is a block diagram for illustrating functional configurations of an electronic apparatus and a sound input/output device according to an embodiment of the disclosure. 
     Referring to  FIG.  3   , a method of processing an input sound  300  by an AP  121  which is one component of the electronic apparatus  100  and a DSP  210  which is one component of the sound input/output device  200  will be described in detail. 
     The DSP  210  included in the sound input/output device  200  may be a microprocessor implemented as an IC that processes signals by digital operations. The DSP  210  according to an embodiment of the disclosure may convert the input sound  300  which is analog data into a digital signal expressed as 0 and 1, and perform signal processing and an operation. 
     The sound input/output device  200  according to an embodiment of the disclosure may collect the input sound  300  via the microphone. The DSP  210  according to an embodiment of the disclosure may process sound data included in the collected input sound  300  by a specific method. The sound data may be data including ‘a sound signal’ that was mentioned in describing the function of the processor  120  with reference to  FIG.  2   . 
     The specific method may mean analysis of frequency components for a sound signal included in the input sound  300 , collection and storage of a sound signal during a threshold time, or identification of sensor data of the sound input/output device and sound data corresponding thereto, etc., but these are merely one example, and the specific method is not necessarily limited thereto. In  FIG.  3   , an operation of the DSP of processing the input sound  300  by a specific method will be expressed as sound detection  211 . 
     Also, the DSP  210  according to an embodiment of the disclosure may transmit sound data processed through the sound detection  211  to the electronic apparatus  100 . 
     The electronic apparatus  100  according to an embodiment of the disclosure may receive the sound data processed through the sound detection  211  from the sound input/output device  200  through the communication interface  110 . When the sound data is identified, the AP  121  may perform a sound classification  121 - 1  operation. The sound classification  121 - 1  operation may include the following operations. 
     First, the AP  121  may identify whether a user voice input is included in the sound data. In a case in which the sound data includes a user voice input, the AP  121  may identify whether the user voice input includes a wake-up word, and in case the user voice input includes a wake-up word, the AP  121  may perform the voice recognition assistant function. 
     In a case in which the sound data does not include a user voice input, the AP  121  according to an embodiment of the disclosure may identify whether a scene noise signal or an event noise signal is included in the sound data. In a case in which a scene noise signal or an event noise signal is included in the sound data, the AP  121  may perform different operations based on the identified noise type. 
     In a case in which a scene noise signal is included in the sound data, the AP  121  according to an embodiment of the disclosure may perform noise cancelling for the sound data, and in case an event noise signal is included in the sound data, the AP  121  may control the output of the audio content signal. 
       FIG.  4    is a block diagram for illustrating in detail functional configurations of an electronic apparatus and a sound input/output device according to an embodiment of the disclosure. 
     Referring to  FIG.  4   , the DSP  210  of the sound input/output device  200  may process sound data included in an input sound  400  input into the sound input/output device  200  by various methods. 
     The DSP  210  according to an embodiment of the disclosure may perform scene recording  411 . The scene recording  411  may mean an operation of collecting an input sound signal during a threshold time, converting the collected sound signal into a digital signal, and storing the signal. Through the scene recording  411 , the DSP  210  may collect various types of sound signals that are generated in a space wherein a user wearing the sound input/output device  200  is located. 
     Also, the DSP  210  may perform stationarity estimation  412 . The stationarity estimation  412  may mean an operation of converting the input sound  400  in the form of data to be input into a neural network model before classifying a user voice and the other sound signals (noises) excluding the user voice among the sound signals included in the input sound  400 . 
     Specifically, the DSP  210  according to an embodiment of the disclosure may perform frequency analysis for the sound signals included in the input sound  400  through the stationarity estimation  412 , and convert the sound data including the sound signals for which frequency analysis was performed into data in the form of a matrix (N*N). 
     In addition, the DSP  210  may perform wearer speech detection  413 . The wearer speech detection  413  may mean an operation of detecting a user&#39;s utterance based on sensing data acquired through a sensor included in the sound input/output device  200 . 
     Specifically, the sound input/output device  200  according to an embodiment of the disclosure may include a plurality of microphones that are arranged in locations distanced from one another, and in this case, the input sound  400  may include a plurality of data corresponding to the sounds collected from the plurality of respective microphones. 
     The DSP  210  according to an embodiment of the disclosure may acquire information on the strength differences of the sound signals received through the plurality of respective microphones through the wearer speech detection  413 . As the distance difference of a user&#39;s voice from a generation source of a signal (the user&#39;s vocal organ) to the plurality of microphones is extremely small, unlike the ambient noises, the strength differences of the sound signals received through the plurality of respective microphones may be smaller than a threshold difference. 
     However, this is merely an example, and the DSP  210  may acquire information different from the above through the wearer speech detection  413 . Specifically, the DSP  210  according to another embodiment of the disclosure may acquire information on a time difference at which a signal having the same frequency characteristic was input for the sound signals received through the plurality of respective microphones. 
     Also, the sound input/output device  200  may perform noise/voice classification  414 . The noise/voice classification  414  may mean an operation of inputting data acquired as a result of the stationarity estimation  412  and data acquired through the wearer speech detection  413  into a neural network model and identifying whether the input sound  400  includes a user voice. 
     The neural network model may be a model trained to receive input of a plurality of data related to the input sound  400  and output information on whether the input sound  400  includes a user voice input. 
     In  FIG.  3   , the component identifying whether the input sound  300  includes a user voice input was the AP  121 , but in  FIG.  4   , the functions of the electronic apparatus  100  and the sound input/output device  200  were described on the premise that the DSP  210  can also perform such a function. 
     Also, the DSP  210  according to an embodiment of the disclosure may transmit the sound data processed through the noise/voice classification  414  and the sound signals collected through the scene recording  411  to the electronic apparatus  100 . 
     If sound data received from the sound input/output device  200  and information on whether the input sound  400  includes a user voice input are received, the processor  120  according to an embodiment of the disclosure may activate a nose recognition engine  421  or a wake-up engine  422 . 
     The noise recognition engine  421  according to an embodiment of the disclosure may input the sound data included in the input sound  400  collected during a threshold time through the scene recording  411  of the DSP  210  and the sound data identified as a noise by the noise/voice classification  414  into the neural network model, and identify whether the input sound  400  includes a scene noise signal or an event noise signal. 
     The wake-up engine  422  according to an embodiment of the disclosure may input the sound data identified as a voice by the noise/voice classification  414  into the neural network model, and identify whether the input sound  400  includes a wake-up word. Also, if a wake-up word is identified, the wake-up engine  422  may perform the voice recognition assistant function provided by the electronic apparatus  100 . 
     If it is identified that the type of a scene noise corresponding to the characteristic of a space wherein a user is located has changed according to the passage of time, the noise recognition engine  421  according to an embodiment of the disclosure may transmit  423  a control signal making the sound input/output device  200  collect a noise signal (scene detection) during a threshold time to the sound input/output device  200  through the communication interface  110 . 
     Also, the noise recognition engine  421  according to an embodiment of the disclosure may transmit such a control signal to the sound input/output device  200  even when the type of a scene noise corresponding to the characteristic of a space wherein a user is located is not identified. 
       FIG.  5 A  and  FIG.  5 B  are diagrams for illustrating different types of scene noise signals corresponding to characteristics of a space. 
     Referring to  FIG.  5 A , the user  10  of the electronic apparatus  100  is located in an inner space of the subway that is being operated while wearing the sound input/output device  200 . The sound input/output device  200  may be a cordless earphone. 
     In an operating process of the subway, a regular noise  510  that is generated by friction between the driving part of the subway and the railway, etc., may be introduced into the inner space of the subway. 
     Referring to  FIG.  5 B , the user  10  of the electronic apparatus  100  is located on a walkway of a park while wearing the sound input/output device  200 . 
     In the park, a regular noise  520  that is generated by birds that live in an adjacent area to vegetation, etc., may be transmitted to the user. 
     As can be seen above, the types of regular noises (hereinafter, referred to as “scene noises”) collected by the sound input/output device  200  may vary according to the characteristic of a space wherein the user  10  of the electronic apparatus  100  is located. 
     The electronic apparatus  100  according to an embodiment of the disclosure may collect a scene noise corresponding to the characteristic of the space and perform noise cancelling based on the noise, so that the user  10  provided with an audio content from the sound input/output device  200  does not feel uncomfortable by a scene noise generated around the user  10 . 
     Specifically, in  FIG.  5 A , the electronic apparatus  100  may remove the noise  510  by offset interference between signals by generating a signal of a wavelength that is opposite to the wavelength of the scene noise  510  generated on the subway, and in  FIG.  5 B , the electronic apparatus  100  may remove the noise  520  by offset interference between signals by generating a signal of a wavelength that is opposite to the wavelength of the scene noise  520  generated in the park. 
       FIG.  6 A  and  FIG.  6 B  are diagrams for illustrating event noise signals that are generated irregularly in a specific space. 
     Referring to  FIG.  6 A , to a user  10  who is located on the subway, a guide voice  610  that is generated from a speaker  21  located inside the subway may be transmitted. The guide voice  610  according to an embodiment of the disclosure may be a voice regarding information on a station at which the subway will stop, and as intervals between stations and driving speeds of the subway in respective sections are not regular, the guide voice  610  may be a noise in case of emergency (hereinafter, an “event noise”) that is not generated regularly. 
     Referring to  FIG.  6 B , to a user  10  who is located in a park, a sound  620  of a pet dog  22  located on a walkway may be transmitted. The barking sound  620  according to an embodiment of the disclosure may be an event noise generated based on the pet dog  22  and the specific environment around the pet dog  22 . 
     An event noise may be a noise that includes useful information for a user or a noise that makes a user recognize an unexpected situation. Thus, the electronic apparatus  100  according to an embodiment of the disclosure may not perform noise cancelling for an event noise. 
       FIG.  7    is a diagram for illustrating a wake-up word identification operation of an electronic apparatus according to an embodiment of the disclosure. 
     The electronic apparatus  100  according to an embodiment of the disclosure may identify whether a sound signal received from the sound input/output device  200  is a signal including a user voice input  700 . 
     Specifically, the electronic apparatus  100  according to an embodiment of the disclosure may identify a scene noise  510  transmitted to the user  10  located inside the subway, an event noise  610 , and a wake-up word  710  included in an input sound including the user voice input  700 . 
     Also, if it is identified that a sound signal received from the sound input/output device  200  is a signal including a user voice input  700 , the electronic apparatus  100  may identify whether a wake-up word  710  is included in the sound signal. 
     A wake-up word is a word or a sentence for executing a voice recognition assistant function provided by the electronic apparatus  100 , and a wake-up word according to an embodiment of the disclosure may include “Hi Bixby.” In this case, the electronic apparatus  100  may input an input sound into a neural network model and identify the wake-up word  710  of “Hi Bixby” included in the user voice input  700 , and perform the voice recognition assistant function based on this. 
       FIG.  8    is a diagram for illustrating an operation of an electronic apparatus according to an embodiment of the disclosure of providing a UI corresponding to an event noise to a user. 
     If it is identified that an input sound collected from the sound input/output device  200  includes an event noise corresponding to the barking sound  620  of the pet dog  22 , the electronic apparatus  100  according to an embodiment of the disclosure may transmit a control signal  111  stopping the output of an audio content signal provided through the sound input/output device  200 . 
     According to another embodiment of the disclosure, the control signal  111  may be a signal related to at least one of an operation of adjusting the volume of the sound input/output device  200  or an operation of providing a feedback corresponding to the event noise  620  through the sound input/output device  200 . 
     In this way, the user  10  may recognize the presence of the pet dog  22  located in the vicinity and the unexpected situation in the vicinity that induced the pet dog  22  to bark, and pay attention. 
     Further, the electronic apparatus  100  according to an embodiment of the disclosure may provide a UI  131  corresponding to the event noise  620  through the display  130  provided on the electronic apparatus  100 . In this case, the electronic apparatus  100  may store information on various types of event noises in the memory for identifying that the event noise  620  corresponds to a barking sound of a pet dog. 
       FIG.  9 A  to  FIG.  9 C  are diagrams for illustrating various neural network models according to an embodiment of the disclosure. 
     Each of the plurality of neural network models illustrated in  FIG.  9 A  to  FIG.  9 C  may include a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and performs a neural network operation through an operation between the operation result of the previous layer and the plurality of weight values. The plurality of weight values that the plurality of neural network layers have may be optimized by a learning result of a neural network model. For example, the plurality of weight values may be updated such that a loss value or a cost value acquired from a neural network model during a learning process is reduced or minimized. An artificial neural network may include a deep neural network (DNN), and there are, for example, a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann Machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks, etc., but the disclosure is not limited to the aforementioned examples. 
     Referring to  FIG.  9 A , the first neural network model  910  may be a model trained to receive input of a sound signal  911  and output identification information  912  of the noise type of the noise signal included in the input sound signal. Specifically, the first neural network model  910  may be a model trained to, if a sound signal  911  is input, output information  912  indicating whether the input sound signal is a scene noise signal or an event noise signal. 
     Referring to  FIG.  9 B , the second neural network model  920  may be a model trained to receive input of a plurality of data  921  related to sound signals and output information  922  regarding whether the plurality of input data include user voice inputs. Specifically, the plurality of data  921  related to sound signals may include data converted into the form of a matrix (N*N) after performing frequency analysis for the sound signals and information on the difference in strength of the sound signals received through the plurality of respective microphones included in the sound input/output device  200 . 
     Referring to  FIG.  9 C , the third neural network model  930  may be a model trained to receive input of a sound signal  931  and output information  932  regarding whether the sound signal includes a wake-up word. 
       FIG.  10    is a diagram for illustrating an operation of an electronic apparatus according to an embodiment of the disclosure of identifying characteristics of a space that change according to movements of a user. 
     Referring to  FIG.  10   , it can be identified that the user  10  has moved to the road side after taking a walk in the park. In this case, a scene noise  520  collected by the sound input/output device  200  in the park and a scene noise  530  collected on the road side may include noise signals having different frequency characteristics. 
     In this case, the electronic apparatus  100  needs to collect and store scene noises corresponding to a new space for performing satisfying noise cancelling in response to change of noise types according to change of characteristics of a space wherein the user  10  is located. 
     Accordingly, if it is identified that the type of the scene noise  530  collected by the sound input/output device  200  while the user  10  is walking on the road side (hereinafter, a first threshold time) after walking out of the park is different from the type of the scene noise  520  collected by the sound input/output device  200  while the user  10  took a walk in the park (hereinafter, a second threshold time) corresponding to the previous time thereof, the electronic apparatus  100  may transmit a control signal  112  making the noise signal  530  that is generated on the road side during a third threshold time after the first threshold time collected to the sound input/output device  200  through the communication interface  110 . 
     The sound input/output device  200  that received the control signal  112  may collect the scene noise  530  generated on the road side during the third threshold time, and transmit this to the electronic apparatus  100 . As a result, the electronic apparatus  100  becomes capable of performing more satisfying noise cancelling based on the new scene noise  530  corresponding to the characteristic of the changed space. 
       FIG.  11    is a block diagram for illustrating in detail a configuration of an electronic apparatus according to an embodiment of the disclosure. 
     According to  FIG.  11   , the electronic apparatus  100  includes a communication interface  110 , a processor  120 , a display  130 , a speaker  140 , a microphone  150 , and a memory  160 . Among the components illustrated in  FIG.  11   , regarding components that overlap with the components illustrated in  FIG.  2   , detailed explanation will be omitted. 
     The processor  120  according to an embodiment of the disclosure may include an AP  121  and a main processor  122 . 
     The display  130  may be implemented as displays in various forms such as a liquid crystal display (LCD), an organic light emitting diodes (OLED) display, a quantum dot light-emitting diodes (QLED) display, a plasma display panel (PDP), etc. Inside the display  130 , driving circuits that may be implemented in forms such as an a-si TFT, a low temperature poly silicon (LTPS) TFT, an organic TFT (OTFT), etc., and a backlight unit, etc. may also be included together. The display  130  may also be implemented as a flexible display, a three-dimensional (3D) display, etc. 
     The speaker  140  is a device that converts an electronic sound signal of the electronic apparatus  100  into a sound wave. The speaker  140  may include a permanent magnet, a coil, and a vibration plate, and it may output a sound by vibrating the vibration plate by an electromagnetic interaction that is generated between the permanent magnet and the coil. 
     In a case in which the processor  120  according to an embodiment of the disclosure executes a function of the electronic apparatus  100  based on voice response information corresponding to a user voice, the processor  120  may control the speaker  140  to output a voice corresponding to the response information. 
     The microphone  150  is a component that collects an input sound by receiving a user&#39;s voice and an ambient noise signal. Specifically, the microphone  150  is a component that generally refers to a device that receives input of a sound wave and generates a current of the same waveform as the sound wave. The processor  120  according to an embodiment of the disclosure may convert a sound signal included in an input sound into a digital signal based on the current of the waveform generated by the microphone  150 . 
     In the previous descriptions of the drawings, descriptions were made based on the premise that a sound signal is collected by the sound input/output device  200  including a microphone. However, the electronic apparatus  100  may implement the various functions included in the disclosure without the sound input/output device  200 , and in this case, the microphone  150  provided on the electronic apparatus  100  may collect a sound signal in place of the microphone of the sound input/output device  200 . 
     The memory  160  may store data for the various embodiments of the disclosure. The memory  160  may be implemented in the form of a memory embedded in the electronic apparatus  100 , or in the form of a memory that can be attached to or detached from the electronic apparatus  100  according to the use of stored data. For example, in the case of data for driving the electronic apparatus  100 , the data may be stored in a memory embedded in the electronic apparatus  100 , and in the case of data for an extended function of the electronic apparatus  100 , the data may be stored in a memory that can be attached to or detached from the electronic apparatus  100 . In the case of a memory embedded in the electronic apparatus  100 , the memory may be implemented as at least one of a volatile memory (e.g., a dynamic RAM (DRAM), a SRAM, or a synchronous dynamic RAM (SDRAM), etc.) or a non-volatile memory (e.g., an one time programmable ROM (OTPROM), a programmable ROM (PROM), an erasable and programmable ROM (EPROM), an electrically erasable and programmable ROM (EEPROM), a mask ROM, a flash ROM, a flash memory (e.g., NAND flash or NOR flash, etc.), a hard drive, or a solid state drive (SSD)). Also, in the case of a memory that can be attached to or detached from the electronic apparatus  100 , the memory may be implemented in a form such as a memory card (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), a multi-media card (MMC), etc.) and an external memory that can be connected to a USB port (e.g., a USB memory), etc. 
     Also, the memory  160  according to an embodiment of the disclosure may include a plurality of neural network models including the first neural network model  161 , the second neural network model  162 , and the third neural network model  163 . The first to third neural network models  161 - 163  were described in detail in  FIG.  9 A  to  FIG.  9 C . 
       FIG.  12    is a flow chart for illustrating a control method according to an embodiment of the disclosure. 
     The controlling method of an electronic apparatus according to an embodiment of the disclosure may include the steps of outputting an audio content signal from a sound input/output device including a speaker and a microphone (operation S 1210 ), based on receiving a sound signal collected via the microphone from the sound input/output device, identifying whether the sound signal includes a scene noise signal or an event noise signal (operation S 1220 ), based on the sound signal including the scene noise signal, performing noise cancelling for the sound signal (operation S 1230 ), and based on the sound signal including the event noise signal, controlling the output of the audio content signal (operation S 1240 ). 
     In the step of identifying whether the sound signal includes a scene noise signal or an event noise signal (operation S 1220 ), based on the sound signal received from the sound input/output device being a signal not including a user voice input, it may be identified whether the sound signal includes a scene noise signal or an event noise signal. 
     The control method may further include the steps of, based on the sound signal received from the sound input/output device being a signal including a user voice, identifying whether the sound signal includes a wake-up word, and based on the sound signal including a wake-up word, performing a voice recognition assistant function. 
     Also, the control method may further include the step of, based on the sound signal including the event noise signal, performing an operation related to at least one of stopping of the output of the audio content signal, adjusting of the output volume, or providing of a feedback corresponding to the event noise signal. 
     In the step of identifying whether the sound signal includes a scene noise signal or an event noise signal (operation S 1220 ), the sound signal may be input into a first neural network model and it may be identified whether the sound signal includes a scene noise signal or an event noise signal, and the first neural network model may be a model trained to, based on a sound signal being input, output information indicating whether the input sound signal is a scene noise signal or an event noise signal. 
     Also, the control method may further include the step of, based on identifying that the type of a scene noise signal received from the sound input/output device during a first threshold time being different from the type of a scene noise signal received during a second threshold time which is before the first threshold time, transmitting a control signal making the sound input/output device collect a noise signal during a third threshold time which is after the first threshold time to the sound input/output device. 
     In addition, the control method may further include the step of, based on the type of a scene noise signal received from the sound input/output device during a fourth threshold time not being identified, transmitting a control signal making the sound input/output device collect a noise signal during a fifth threshold time which is after the fourth threshold time to the sound input/output device. 
     In the step of identifying whether the sound signal includes a scene noise signal or an event noise signal (operation S 1220 ), based on receiving a sound signal from the sound input/output device, the electronic apparatus may be powered on, and it may be identified whether the sound signal includes a scene noise signal or an event noise signal. 
     Methods according to the aforementioned various embodiments of the disclosure may be implemented in forms of applications that can be installed on conventional electronic apparatuses. 
     Also, the methods according to the aforementioned various embodiments of the disclosure may be implemented just by software upgrade, or hardware upgrade of conventional electronic apparatuses. 
     Further, it is possible that the aforementioned various embodiments of the disclosure are performed through an embedded server provided on the electronic apparatus or at least one external server. 
     The aforementioned various embodiments of the disclosure may be implemented in a recording medium that can be read by a computer or an apparatus similar to a computer, by using software, hardware, or a combination thereof. In some cases, the embodiments described in this specification may be implemented as the processor  120  itself. According to implementation by software, the embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described in this specification. 
     Computer instructions for performing processing operations of the electronic apparatus  100  according to the aforementioned various embodiments of the disclosure may be stored in a non-transitory computer-readable medium. Computer instructions stored in such a non-transitory computer-readable medium make the processing operations at the electronic apparatus  100  according to the aforementioned various embodiments performed by a specific machine, when the instructions are executed by the processor of the specific machine. 
     Anon-transitory computer-readable medium refers to a medium that stores data semi-permanently, and is readable by machines. As specific examples of a non-transitory computer-readable medium, there may be a CD, a DVD, a hard disc, a blue-ray disc, a USB, a memory card, a ROM, and the like. 
     While example embodiments of the disclosure have been shown and described, the disclosure is not limited to the aforementioned specific embodiments, and it is apparent that various modifications may be made by those having ordinary skill in the technical field to which the disclosure belongs, without departing from the gist of the disclosure as claimed by the appended claims. Also, it is intended that such modifications are not to be interpreted independently from the technical idea or prospect of the disclosure.