Patent Description:
Recently, sound direction detection and voice recognition using sensors mounted in home appliances, image display apparatuses, virtual reality apparatuses, augmented reality apparatuses, artificial intelligence speakers, etc., is increasing. In particular, the use of an acoustic sensor for separately identifying sounds coming from sound sources located in two different directions is also increasing.

Related art sound source separation methods include, for example, independent component analysis (ICA), geometric source separation (GSS), and so forth. ICA or GSS involves estimating a location by analyzing a signal input to multiple microphones through time-frequency domain conversion and extracting characteristics of a sound source for separation. Recently, a method has also been developed in which after multiple sound sources are trained using a deep neural network technology, a sound source is separated based on a learning result.

<CIT> discloses a spectrum analyzer comprising a plurality of resonators with different center frequencies, arranged in such a way as to reduce coupling and increase analysis accuracy.

<CIT> discloses a microphone device system which enables the determination of input sound direction. Based on this determination, microphone clusters with a low signal to noise ratio can be switched off in order to reduce power consumption.

<CIT> discloses a sound emission and detection device wherein the speakers are arranged on the side of a circular surface, and the microphones are arranged on the inside of the circular surface such that the microphones and speakers have opposing directionalities. This allows a compact configuration which suppresses wraparound sound from the speakers to the microphones.

<CIT> discloses a system of selectively switching between a pluralities of microphones based on the average power of input signals.

<CIT> describes a mechanical resonating structure comprising a diaphragm and a plurality of piezoelectric stacks.

According to the invention, there is provided a sound source separation apparatus comprising: a sound inlet configured to receive a sound; a sound outlet configured to output the sound received through the sound inlet; a plurality of directional vibrators arranged on a plane between the sound inlet and the sound outlet, the plane being perpendicular to a central axis of the sound inlet, and the plurality of directional vibrators being arranged around a center point on the plane corresponding to the central axis of the sound inlet in such a manner that one or more of the plurality of directional vibrators is configured to selectively react based on a direction of the sound received through the sound inlet; and a control circuit configured to: determine a first direction of a first sound source and a second direction of a second sound source that is different from the first sound source, based on strengths of output signals of the plurality of directional vibrators, wherein the second direction of the second sound source is determined during a first time in which the first sound from the first sound source decreases and the second sound from the second sound source increases such that the second sound is relatively strong compared to the first sound and the first direction of the first sound source is determined during a second time in which the second sound from the second sound source decreases and the first sound increases such that the first sound is relatively strong compared to the second sound, and select a first directional vibrator and a second directional vibrator from among the plurality of directional vibrators based on the determined directions to separately obtain a first sound from the first sound source and a second sound from the second sound source, wherein the first directional vibrator is different from the second directional vibrator.

Proposed embodiments may therefore be applied to an algorithm of an acoustic sensor using a plurality of resonators. Embodiments may also be applicable to Artificial Intelligence (AI) speakers and the like.

Embodiments may, in particular, be useful for tracking a position of a sound source, separating a sound source, etc..

Proposed embodiments may provide a sound compass sensor which uses a plurality of directional acoustic resonators and compares the outputs to find the direction of a sound. Each resonator may have greatest displacement when the longitudinal sound is input. Therefore, each resonator may have a response characteristic of a figure-<NUM> type. By arranging the resonators in a circular shape within the sensor, the direction of each resonator can be different. Also, where directivity of a resonator is <NUM> ° based on -6dB, it may be possible to exclude sound sources input at angles greater than ± <NUM>°.

According to proposed embodiments, sound source separation may be performed using a method of selecting a channel having a minimum sensitivity to a sound source in. For example, if two sources are <NUM>° apart, choosing a channel <NUM>° away may minimize interference.

The control circuit may be further configured to obtain first information about the first sound from the first sound source based on a first output signal of the first directional vibrator and second information about the second sound from the second sound source based on a second output signal of the second directional vibrator.

The control circuit may be configured to select a directional vibrator having a highest sensitivity to the first direction of the first sound source, among the plurality of directional vibrators, as the first directional vibrator and select a directional vibrator having a highest sensitivity to the second direction of the second sound source, among the plurality of directional vibrators, as the second directional vibrator.

The control circuit may be further configured to select a directional vibrator arranged in the first direction of the first sound source, among the plurality of directional vibrators, as the first directional vibrator and select a directional vibrator arranged in the second direction of the second sound source, among the plurality of directional vibrators, as the second directional vibrator.

The control circuit may be further configured to obtain the first information about the first sound from the first sound source and the second information about the second sound from the second sound source by computing a first output signal and a second output signal based on a first rate of the second sound of the second sound source contributing to the first output signal of the first directional vibrator and a second rate of the first sound of the first sound source contributing to the second signal of the second directional vibrator.

In a case that the first output signal of the first directional vibrator is C1, the second output signal of the second directional vibrator is C2, a first sound signal of the first sound source is S1, a second sound signal of the second sound source is S2, the first rate of the second sound of the second sound source contributing to the first output signal of the first directional vibrator is α, and the second rate of the first sound of the first sound source contributing to the second output signal of the second directional vibrator is β, then C1 = S1 + αS2 and C2 = S2 + βS1, and S1 = (C1-αC2)/(<NUM>-αβ) and S2 = (C2-βC1)/(<NUM>-αβ).

The first rate of the second sound of the second sound source contributing to the first output signal of the first directional vibrator may be determined by a first angle between the first directional vibrator and the second sound source, and the second rate of the first sound of the first sound source contributing to the second output signal of the second directional vibrator may be determined by a second angle between the second directional vibrator and the first sound source.

The sound source separation apparatus may comprise a memory configured to store sensitivity information of each directional vibrator with respect to a direction of a sound incident to each directional vibrator.

The control circuit may be further configured to select a directional vibrator having a lowest sensitivity to the second direction of the second sound source, among a plurality of directional vibrators arranged in a first angle range around the first direction of the first sound source, as the first directional vibrator and select a directional vibrator having a lowest sensitivity to the first direction of the first sound source, among the plurality of directional vibrators arranged in a second angle range around the second direction of the second sound source, as the second directional vibrator.

The control circuit may be further configured to determine that there are two or more sound sources in a case that strengths of output signals of a plurality of directional vibrators arranged in a range of <NUM> degrees have two or more peaks, and determine a direction of a sound source during a time in which the strengths of the output signals of the plurality of directional vibrators arranged in the range of <NUM> degrees have one peak.

The control circuit may be further configured to determine an arrangement direction of a directional vibrator having a highest strength of an output signal, among the plurality of directional vibrators, as the first direction of the first sound source or the second direction of the second sound source.

The control circuit may be further configured to determine a direction perpendicular to an arrangement direction of a directional vibrator having a lowest strength of an output signal, among the plurality of directional vibrators, as the first direction of the first sound source or the second direction of the second sound source.

The control circuit may be further configured to compare a vibration strength of a directional vibrator arranged at +<NUM>° with respect to a directional vibrator having a lowest strength of an output signal with a vibration strength of a directional vibrator arranged at -<NUM>° with respect to the directional vibrator having the lowest strength of the output signal, and determine an arrangement direction of a directional vibrator having a higher strength of the output signal, among the directional vibrator arranged at +<NUM>° and the directional vibrator arranged at -<NUM>°, as the first direction of the first sound source or the second direction of the second sound source.

The sound source separation apparatus may further comprise an omnidirectional vibrator configured to react to an input sound regardless of a direction of the input sound.

The control circuit may be further configured to compare a phase of a directional vibrator arranged at +<NUM>° with respect to a directional vibrator having a lowest strength of an output signal with a phase of the omnidirectional vibrator, compare a phase of a directional vibrator arranged at -<NUM>° with respect to the directional vibrator having the lowest strength of the output signal with the phase of the omnidirectional vibrator, and determine an arrangement direction of a directional vibrator, among the directional vibrator arranged at +<NUM>° and the directional vibrator arranged at -<NUM>°, having a phase closest to the phase of the omnidirectional vibrator as the direction of the first sound source or the second sound source.

The plurality of directional vibrators and the omnidirectional vibrator may be arranged on an identical plane, and the plurality of directional vibrators may be arranged to surround the omnidirectional vibrator.

The plurality of directional vibrators may be arranged to have symmetry with respect to the center point.

The sound outlet may be provided to face all the plurality of directional vibrators.

The sound outlet may comprise a plurality of sound outlets respectively facing the plurality of directional vibrators.

The plurality of directional vibrators may have an identical resonant frequency.

The plurality of directional vibrators may comprise a plurality of directional vibrators having different resonant frequencies.

According to the invention, there is also provided a sound source separation method comprising: receiving a sound through a sound inlet of a sound source separation apparatus; outputting the sound received through the sound inlet by a sound outlet; determining a first direction of a first sound source and a second direction of a second sound source that is different from the first sound source, based on strengths of output signals of a plurality of directional vibrators arranged on a plane between the sound inlet and the sound outlet, the plane being perpendicular to a central axis of the sound inlet, and the plurality of directional vibrators being arranged around a center point on the plane corresponding to the central axis of the sound inlet in such a manner that one or more of the plurality of directional vibrators is configured to selectively react based on a direction of the sound received through the sound inlet; wherein the second direction of the second sound source is determined during a first time in which the first sound from the first sound source decreases and the second sound from the second sound source increases such that the second sound is relatively strong compared to the first sound and the first direction of the first sound source is determined during a second time in which the second sound from the second sound source decreases and the first sound increases such that the first sound is relatively strong compared to the second sound; selecting a first directional vibrator and a second directional vibrator from among the plurality of directional vibrators based on the determined directions to separately obtain a first sound from the first sound source and a second sound from the second sound source; and obtaining sound information by using the first directional vibrator and the second directional vibrator, wherein the first directional vibrator is different from the second directional vibrator.

The determining of the first direction of the first sound source and the second direction of the second sound source may comprise: determining that there are two or more sound sources in a case that strengths of output signals of a plurality of directional vibrators arranged in a range of <NUM> degrees have two or more peaks; and determining a direction of a sound source during a time in which the strengths of the output signals of the plurality of directional vibrators arranged in the range of <NUM> degrees have one peak.

The determining of the first direction of the first sound source and the second direction of the second sound source may comprise determining an arrangement direction of a directional vibrator having a highest strength of an output signal, among the plurality of directional vibrators, as the first direction of the first sound source or the second direction of the second sound source.

The determining of the first direction of the first sound source and the second direction of the second sound source may comprise determining a direction perpendicular to an arrangement direction of a directional vibrator having a lowest strength of an output signal, among the plurality of directional vibrators, as the first direction of the first sound source or the second direction of the second sound source.

The determining of the first direction of the first sound source and the second direction of the second sound source may comprise: comparing a vibration strength of a directional vibrator arranged at +<NUM>° with respect to a directional vibrator having a lowest strength of an output signal with a vibration strength of a directional vibrator arranged at -<NUM>° with respect to the directional vibrator having the lowest strength of the output signal; and determining an arrangement direction of a directional vibrator having a higher strength of the output signal, among the directional vibrator arranged at +<NUM>° and the directional vibrator arranged at -<NUM>°, as the direction of the first sound source or the second sound source.

The sound source separation method may further comprise receiving an input sound by using an omnidirectional vibrator reacting to the sound regardless of a direction of the input sound.

The determining of the first direction of the first sound source and the second direction of the second sound source may comprise: comparing a phase of a directional vibrator arranged at +<NUM>° with respect to a directional vibrator having a lowest strength of an output signal with a phase of the omnidirectional vibrator; comparing a phase of a directional vibrator arranged at -<NUM>° with respect to the directional vibrator having the lowest strength of the output signal with a phase of the omnidirectional vibrator; and determining an arrangement direction of a directional vibrator having a phase closest to the phase of the omnidirectional vibrator, among the directional vibrator arranged at +<NUM>° and the directional vibrator arranged at -<NUM>°, as the first direction of the first sound source or the second direction of the second sound source.

The obtaining of the sound information by using the first directional vibrator and the second directional vibrator may comprise: obtaining first information about the first sound from the first sound source based on a first output signal of the first directional vibrator; and obtaining second information about the second sound from the second sound source based on a second output signal of the second directional vibrator.

The selecting of the first directional vibrator and the second directional vibrator may comprise: selecting a directional vibrator having a highest sensitivity to the first direction of the first sound source, among the plurality of directional vibrators, as the first directional vibrator; and selecting a directional vibrator having a highest sensitivity to the second direction of the second sound source, among the plurality of directional vibrators, as the second directional vibrator.

The selecting of the first directional vibrator and the second directional vibrator may comprise: selecting a directional vibrator arranged in the direction of the first sound source, among the plurality of directional vibrators, as the first directional vibrator; and selecting a directional vibrator arranged in the direction of the second sound source, among the plurality of directional vibrators, as the second directional vibrator.

The obtaining of the sound information by using the first directional vibrator and the second directional vibrator may comprise: determining a first rate of the second sound of the second sound source contributing to the first output signal of the first directional vibrator and a second rate of the first sound of the first sound source contributing to the second output signal of the second directional vibrator; and obtaining the first information about the first sound from the first sound source and the second information about the sound from the second sound source by computing the first output signal and the second output signal based on the first rate of the second sound of the second sound source contributing to the first output signal of the first directional vibrator and the second rate of the first sound of the first sound source contributing to the second signal of the second directional vibrator.

The first rate of the second sound of the second sound source contributing to the first output signal of the first directional vibrator may be determined by a first angle between the first directional vibrator and the second sound source, and the second rate of the first sound of the first sound source contributing to the second output signal of the second directional vibrator is determined by a second angle between the second directional vibrator and the first sound source.

The selecting of the first directional vibrator and the second directional vibrator may comprise: selecting a directional vibrator having a lowest sensitivity to the second direction of the second sound source, among a plurality of directional vibrators arranged in a first angle range around the first direction of the first sound source, as the first directional vibrator; and selecting a directional vibrator having a lowest sensitivity to the first direction of the first sound source, among the plurality of directional vibrators arranged in a second angle range around the second direction of the second sound source, as the second directional vibrator.

According to another aspect of the disclosure not forming part of the invention as claimed, there is provided a sound source separation apparatus comprising: a case having a first opening and one or more second openings, the first opening configured to receive a sound and the one or more second openings configured to output the sound received through the first opening. a support member provided on an inside surface of the case and having a hole in a center portion; a plurality of directional vibrators arranged on the support member to surround a center point in the hole of the support member corresponding to a central axis of the first opening, the plurality of directional vibrators are arranged in such a manner that one or more of the plurality of directional vibrators is configured to selectively react based on a direction of the sound received through that is input to the first opening.

Each of the plurality of directional vibrators may comprise: a movable portion configured to move based on the sound, and a sensing portion configured to detect a movement of the movable portion.

Each of the plurality of directional vibrators may further comprise a mass body formed on the movable portion according to a resonance frequency.

According to another aspect of the disclosure not forming part of the invention as claimed, there is provided a sound source separation apparatus comprising: a memory configured to store one or more instructions; and a processor configured to execute the one or more instructions to: determine a first direction of a first sound source and a second direction of a second sound source that is different from the first sound source, based on strengths of output signals received from a plurality of directional vibrators arranged on a plane between a sound inlet and a sound outlet of the sound source separation apparatus, the plane being perpendicular to a central axis of the sound inlet, and the plurality of directional vibrators being arranged around a center point on the plane corresponding to the central axis of the sound inlet in such a manner that one or more of the plurality of directional vibrators is configured to selectively react based on a direction of the sound received through the sound inlet; select a first directional vibrator and a second directional vibrator from among the plurality of directional vibrators to separately obtain a first sound from the first sound source and a second sound from the second sound source; and obtain sound information by using the first directional vibrator and the second directional vibrator, wherein the first directional vibrator is different from the second directional vibrator.

These and/or other aspects will become apparent and more readily appreciated from the following description of the embodiments of the disclosure, taken in conjunction with the accompanying drawings in which:.

Reference will now be made in detail to embodiments of the disclosure, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. In this regard, the embodiments of the disclosure may have different forms and should not be construed as being limited to the descriptions set forth herein. Accordingly, the embodiments of the disclosure are merely described below, by referring to the figures, to explain aspects. It will be understood that when a layer is referred to as being "on" another layer or substrate, it can be directly on the other layer or substrate, or intervening layers may also be present.

<FIG> is a plan view illustrating a schematic structure of a sound source separation apparatus according to an embodiment of the disclosure. <FIG> is a cross-sectional view taken along line A-A' of the sound source separation apparatus illustrated in <FIG>. Referring to <FIG> and <FIG>, a sound source separation apparatus <NUM> according to an embodiment of the disclosure may include a sound inlet <NUM> receiving input sound, sound outlets <NUM> outputting the input sound received through the sound inlet <NUM>, and a plurality of directional vibrators 110_k arranged between the sound inlet <NUM> and the sound outlets <NUM>. Herein, assuming that the number of directional vibrators 110_k is N, k is an integer from <NUM> to N.

According to an embodiment, a physical angular resolution of the sound source separation apparatus <NUM> may be determined based on the number N of directional vibrators 110_k. For example, the physical angular resolution of the sound source separation apparatus <NUM> may be expressed as <NUM>°/N. The sound source separation apparatus <NUM> may detect an input direction of the sound by comparing strengths of output signals of the plurality of directional vibrators 110_k, and a higher angular resolution may be achieved when a larger number of directional vibrators 110_k to be compared in strengths of output signals are used.

The plurality of directional vibrators 110_k are arranged in such a manner that at least one directional vibrator 110_k selectively reacts based on a direction of the sound input through the sound inlet <NUM>. The plurality of directional vibrators 110_k may be arranged to surround the sound inlet <NUM>. The plurality of directional vibrators 110_k may be arranged on a plane not to overlap each other and all the plurality of directional vibrators 110_k may be exposed by the sound inlet <NUM>. As illustrated in <FIG>, the plurality of directional vibrators 110_k may be arranged on the same plane. In addition, the plurality of directional vibrators 110_k may be arranged to surround a center point C on the plane that is perpendicular to a central axis of the sound inlet <NUM>. The plurality of directional vibrators 110_k surround the center point C in a circular shape in <FIG>, but the above description is merely an example. The plurality of directional vibrators 110_k are not limited to the above-described arrangement and may also be arranged in various shapes having symmetry with respect to the center point C. For example, the plurality of directional vibrators 110_k may be arranged in a polygonal or oval shape. According to an embodiment, the plurality of directional vibrators 110_k may be arranged on different planes to provide a different configuration according to the frequency characteristics of the sound to be detected.

The number of sound outlets <NUM> may equal the number of directional vibrators 110_k and may respectively face the plurality of directional vibrators 110_k. The sound inlet <NUM> and the sound outlets <NUM> are not limited to any particular size or shape and may also have arbitrary sizes and shapes capable of equally exposing the plurality of directional vibrators 110_k.

According to an embodiment, a case <NUM> having openings corresponding to the shapes of the sound inlet <NUM> and the sound outlets <NUM> may be used as the sound inlet <NUM> and the sound outlets <NUM>. The case <NUM> may be made of various materials capable of blocking sound. For example, the case <NUM> may be made of a material such as aluminum. The sound inlet <NUM> and the sound outlets <NUM> provided in the case <NUM> are not limited to the shapes illustrated in <FIG>.

According to an embodiment, a support <NUM> configured to support the plurality of directional vibrators 110_k may be provided inside the case <NUM>. Also, the support <NUM> may provide a space in which the plurality of directional vibrators 110_k vibrate in reaction to sound may be located. According to an embodiment, the support <NUM> may be provided by forming a hole in a substrate. As illustrated in <FIG>, the hole may be a through hole TH in the substrate. The plurality of directional vibrators 110_k may be supported by the support <NUM> at ends thereof and be located to face the through hole TH. The through hole TH provides a space in which the directional vibrators 110_k vibrate due to an external force and is not limited to any particular shape or size as long as the through hole TH provides such a space. The support <NUM> may be made of various materials such as a silicon substrate.

The sound source separation apparatus <NUM> may further include a control circuit <NUM> that detects directions of two or more sound sources located in different positions in an azimuthal direction by comparing strengths of output signals of the plurality of directional vibrators 110_k and selects the directional vibrator 110_k that is to obtain sound information based on the detected directions of the sound sources. The sound source separation apparatus <NUM> may further include a memory <NUM> that stores data such as sensitivity of each directional vibrator 110_k corresponding to a sound incident direction.

<FIG> is a cross-sectional view of one of the plurality of directional vibrators 110_k included in the sound source separation apparatus <NUM> of <FIG>, and <FIG> is a plan view of one of the plurality of directional vibrators 110_k included in the sound source separation apparatus <NUM> of <FIG>. Referring to <FIG>, the directional vibrator 110_k may include a fixed portion <NUM> fixed to the support <NUM>, a movable portion <NUM> movable in reaction to a sound signal, and a sensing portion <NUM> configured to detect motion of the movable portion <NUM>. The directional vibrator 110_k may further include a mass body <NUM> configured to provide a certain mass m to the movable portion <NUM>.

The movable portion <NUM> may be made of, for example, an elastic film. The elastic film may have a length L and a width W and determine resonance characteristics of the directional vibrator 110_k together with the mass m of the mass body <NUM>. The elastic film may be made of a material such as silicon, metal, or polymer. According to an embodiment, the resonance frequency of the directional vibrator 110_k may be changed using different weight for the mass body <NUM> and/or by changing the length of the directional vibrator 110_k.

The sensing portion <NUM> may include a sensor layer configured to detect motion of the movable portion <NUM>. The sensing portion <NUM> may include, for example, a piezoelectric element. In this case, the sensing portion <NUM> may have a structure in which an electrode layer, a piezoelectric material layer, and another electrode layer are stacked on one another. The piezoelectric material may include, for example, zinc oxide (ZnO), tin oxide (SnO), lead zirconate titanate (PZT), zinc stannate (ZnSnO<NUM>), polyvinylidene fluoride (PVDF), poly(vinylidene fluoride-trifluoroethylene) P(VDF-TrFE), aluminum nitride (AIN), or lead magnesium niobate-lead titanate (PMN-PT). The electrode layer may be made of a metal material or various other conductive materials.

Values of the width, thickness, etc. of the directional vibrators 110_k may be determined considering a resonant frequency desired for the directional vibrators 110_k. For example, each directional vibrator 110_k may have a width between about several µm to several hundred µm, a thickness equal to or less than several µm, and a length equal to or less than about several mm, but is not limited thereto. The micro-sized directional vibrators 110_k may be produced using a microelectromechanical system (MEMS) process.

The directional vibrator 110_k vertically vibrates in a Z-direction in reaction to an external sound signal and has an output proportional to a displacement z. The displacement z satisfies the following equation of motion.

In the above equation, c denotes a damping coefficient and k denotes an elastic coefficient. F<NUM>cosωt denotes a driving force and indicates an action by a sound signal input to the directional vibrator 110_k. The value of k is determined based on physical properties and the shape of the movable portion <NUM>.

The directional vibrator 110_k shows frequency response characteristics of a certain bandwidth with respect to a resonant frequency f<NUM> as a center frequency. The center frequency f<NUM> is defined below.

As such, the directional vibrators 110_k included in the sound source separation apparatus <NUM> may detect sound having a certain frequency band with respect to a designed center frequency. Therefore, when the center frequency is designed, a frequency band having high availability in a given environment may be selected and the directional vibrators 110_k may be implemented in accordance with the selected frequency band.

In the sound source separation apparatus <NUM> according to an embodiment of the disclosure, the directional vibrators 110_k located in different positions may be set to the same length to have the same resonant frequency. However, the plurality of directional vibrators 110_k are not limited thereto and may be modified to have different resonant frequencies.

<FIG> is a plan view of a sound source separation apparatus according to another embodiment of the disclosure. <FIG> is a cross-sectional view taken along line A-A' of the sound source separation apparatus of <FIG>. Referring to <FIG>, a sound source separation apparatus <NUM> may include the sound inlet <NUM> receiving input sound, a sound outlet <NUM> outputting the sound input through the sound inlet <NUM>, and the plurality of directional vibrators 110_k arranged between the sound inlet <NUM> and the sound outlet <NUM>. Herein, assuming that the number of directional vibrators 110_k is N, k is an integer from <NUM> to N.

The elements of the sound source separation apparatus <NUM> of <FIG> are the same as those of the sound source separation apparatus <NUM> of <FIG> and <FIG> except for the shape of the sound outlet <NUM>. For example, the sound outlet <NUM> is not provided in a plural number corresponding to the number of directional vibrators 110_k, and a single sound outlet <NUM> is shared by the plurality of directional vibrators 110_k. In other words, the single sound outlet <NUM> may face all the plurality of directional vibrators 110_k. The size of the sound outlet <NUM> illustrated in <FIG> is an example and may be smaller than the illustrated size.

A case <NUM> having openings corresponding to the shapes of the sound inlet <NUM> and the sound outlets <NUM> may be used for the sound inlet <NUM> and the sound outlet <NUM>. The size of the sound outlet <NUM> may not be specified. For example, an entire space opposite to the sound inlet <NUM> with respect to the plurality of directional vibrators 110_k may be open. The open space may serve as the sound outlet <NUM>.

According to the afore-described embodiments of the disclosure, in the sound source separation apparatuses <NUM> and <NUM>, one or more of the plurality of directional vibrators 110_k, which are placed on input paths of directional sound, vibrate in reaction to the sound. For example, as illustrated in <FIG>, when the sound is input along path ①, a first directional vibrator 110_1 located on this path and one or more directional vibrators adjacent thereto may vibrate. Otherwise, when the sound is input along path ②, a ninth directional vibrator 110_9 placed on this path and one or more directional vibrators adjacent thereto may vibrate. Therefore, an input direction of the sound may be detected considering outputs of the plurality of directional vibrators 110_k based on the direction of the input sound.

According to an embodiment, the displacement of the directional vibrators may be based on a relationship between the size of the sound inlet <NUM> and the size of the sound outlet <NUM>. For instance, the displacement of the directional vibrator 110_9 (i.e., at <NUM>° from the direction of the sound input) may be larger than the displacement of the directional vibrator 110_1 in reaction to the sound input from the direction of <NUM>°, when the sound outlet <NUM> is larger than the sound inlet <NUM>. According to another embodiment, if the size of sound inlet <NUM> is same as the size of the sound outlet <NUM>, the output generated in reaction to the sound input from the direction of <NUM>° (i.e., directional vibrator 110_1) and <NUM>° (i.e., directional vibrator 110_9) may be the same.

<FIG> is a plan view of some of the plurality of directional vibrators 110_k for describing an operation principle and directional gain characteristics of the plurality of directional vibrators 110_k. Referring to <FIG>, a plurality of directional vibrators A, B, and C serve as unit acoustic sensors having directional angles corresponding to radius directions of a circle around the sound inlet <NUM>. A directional gain curve of each unit acoustic sensor has a figure of eight. Due to the directional gain curves, the sound source separation apparatus <NUM> or <NUM> has an output in which outputs of the directional vibrators 110_k selectively reacting to signals input from all directions are superposed.

Each directional vibrator 110_k has a preferred angle, which is a principal direction, according to a position where the directional vibrator 110_k is arranged. The contribution of the sound from a principal direction is more the contribution of the sound from the other directions. Therefore, an input direction of sound from all arbitrary directions may be estimated by merely comparing magnitudes of outputs of the plurality of directional vibrators 110_k.

<FIG> is a graph showing an example of directional characteristics of one of the plurality of directional vibrators 110_k. In <FIG>, it is assumed that sixty-four directional vibrators 110_k are arranged on a single plane in a circular shape and are aligned toward a center point C on the plane that is perpendicular to a central axis of the sound inlet <NUM>. However, the number of directional vibrators 110_k is not limited to <NUM>, and in practice, <NUM> or more directional vibrators 110_k may be used. Referring to <FIG>, a directional vibrator located in a direction of <NUM>° has the highest outputs in reaction to sound input from directions of <NUM>° and <NUM>° and has the lowest outputs in reaction to sound input from directions of -<NUM>° and +<NUM>°, thereby exhibiting figure-of-eight gain characteristics. In particular, the output generated in reaction to the sound input from the direction of <NUM>° is slightly higher than the output generated in reaction to the sound input from the direction of <NUM>°.

<FIG> is a graph showing outputs of all directional vibrators 110_k included in the sound source separation apparatus <NUM> or <NUM> in a case when sound is input from one direction. In <FIG>, it is assumed that <NUM> directional vibrators 110_k are arranged, first through sixty fourth directional vibrators are arranged in a clockwise direction from the direction of <NUM>°, and a sound is incident toward a thirty-third directional vibrator in a longitudinal direction of the thirty third directional vibrator located in the direction of <NUM>°. Referring to <FIG>, the peak outputs are shown near the first directional vibrator and near a thirty-third directional vibrator located opposite to the first directional vibrator. In addition, the lowest outputs are shown near a seventeenth directional vibrator located in a direction of +<NUM>° from the first directional vibrator and near a forty ninth directional vibrator located in a direction of -<NUM>° from the first directional vibrator. In particular, the output is the highest near the thirty-third directional vibrator.

Consequently, among the plurality of directional vibrators 110_k, a directional vibrator located in a direction of the input sound has the highest output and a directional vibrator located in a direction of ±<NUM>° from the direction of the input sound has the lowest output. Thus, the control circuit <NUM> of the sound source separation apparatus <NUM> or <NUM> may determine a direction of an input sound, i.e., a direction of a sound source by using the directional vibrator having the highest output or the directional vibrator using the lowest output.

For example, the control circuit <NUM> may determine as a direction of a sound source, an arrangement direction of a directional vibrator having the highest strength of an output signal among the plurality of directional vibrators 110_k. Alternately, the control circuit <NUM> may determine as the direction of the sound source, a direction perpendicular to an arrangement direction of a directional vibrator having the lowest strength of an output signal among the plurality of directional vibrators 110_k. In particular, the control circuit <NUM> may compare a vibration strength of a directional vibrator arranged at +<NUM>° with respect to the directional vibrator having the lowest output signal strength with a strength of an output signal of a directional vibrator arranged at -<NUM>° with respect to the directional vibrator having the lowest output signal strength, and determine an arrangement direction of the directional vibrator having the higher output signal strength as the direction of the sound source.

In the case that sounds are simultaneously incident from two sound sources located in different positions in the azimuthal direction, directions of the two sound sources may be known by using a point in which outputs of the two sound sources differ with time. For example, during a period in which an output of a first sound source decreases and an output of a second sound source increases, a direction of the second sound source may be determined using the above-described method. During a period in which the output of the first sound source increases and the output of the second sound source decreases, the direction of the first sound source may be determined using the above-described method.

For example, <FIG> is a graph showing outputs of all directional vibrators 110_k included in the sound source separation apparatus <NUM> or <NUM> in a case when sounds are input from two different directions. In <FIG>, it is assumed that <NUM> directional vibrators 110_k are arranged and the first through sixty fourth directional vibrators are arranged in the clockwise direction from the direction of <NUM>°. It is also assumed that a sound is incident from the first sound source toward the thirty third directional vibrator in the longitudinal direction of the thirty third directional vibrator located in the direction of <NUM>°, and at the same time, another sound is incident from the second sound source toward the seventeenth directional vibrator in the longitudinal direction of the seventeenth directional vibrator located in the direction of <NUM>°. In other words, it is assumed that the first sound source is located in the direction of <NUM>° and the second sound source is located in the direction of <NUM>°.

Referring to <FIG>, in a graph I showing outputs of the directional vibrators 110_k by the first sound source, the peak outputs are shown near the first directional vibrator and near the thirty-third directional vibrator located opposite to the first directional vibrator, respectively, and the lowest outputs are shown near the seventeenth directional vibrator and the fourth ninth directional vibrator, respectively. In a graph II showing outputs of the directional vibrators 110_k by the second sound source, the peak outputs are shown near the seventeenth directional vibrator and near the forty ninth directional vibrator located opposite to the seventeenth directional vibrator, respectively, and the lowest outputs are shown near the first directional vibrator and the thirty third directional vibrator, respectively.

Once sounds having the identical strength are simultaneously input from the first sound source and the second sound source, the outputs of the directional vibrators 110_k are as shown in a graph III combining the graph I with the graph II. As a result, multiple peaks are formed in an angle range of <NUM>°. Thus, the control circuit <NUM> may analyze the outputs of the directional vibrators 110_k and determine that there are two or more sound sources in the case that strengths of output signals of the plurality of directional vibrators 110_k arranged in the angle range of <NUM>° have two or more peaks. Alternatively, the control circuit <NUM> may determine that there are two or more sound sources in the case that clear peaks and clear lowest points are not found in the outputs of the directional vibrators 110_k.

Strengths of sounds coming from the first sound source and the second sound source, respectively, may increase or decrease with respect to time. For example, <FIG> illustrates a waveform of a sound output from two different sound sources. As illustrated in <FIG>, the sound coming from the first sound source and the sound coming from the second sound source repeatedly increase or decrease over time. There is also a moment when the sound from the first sound source decreases and at the same time, the sound from the second sound source increases or when the sound from the second sound source decreases and at the same time, the sound from the first sound source increases. For example, in <FIG>, during a time T<NUM>, the sound from the first sound source is weak and the sound from the second sound source is strong. During a time T<NUM>, the sound from the second sound source is weak and the sound from the first sound source is strong.

The control circuit <NUM> may determine a direction of the second sound source during the time T<NUM> and a direction of the first sound source during the time T<NUM>. For example, the control circuit <NUM> may analyze the outputs of the directional vibrators 110_k and determine a direction of one sound source during a time in which the strengths of the output signals of the plurality of directional vibrators 110_k arranged in the angle range of <NUM>° have one clear peak and one clear lowest point. Referring to the graphs of <FIG>, at a moment when the outputs of the directional vibrators 110_k change from the graph III to the graph I, the control circuit <NUM> may determine the direction of the first sound source. At a moment when the outputs of the directional vibrators 110_k change from the graph III to the graph II, the control circuit <NUM> may determine the direction of the second sound source.

After the control circuit <NUM> determines the direction of the first sound source and the direction of the second sound source, the control circuit <NUM> may select a directional vibrator for obtaining information about a sound coming from the first sound source and a directional vibrator for obtaining information about a sound coming from the second sound source from among the plurality of directional vibrators 110_k. In particular, the control circuit <NUM> may select two directional vibrators to effectively separate and obtain the sound coming from the first sound source and the sound coming from the second sound source.

As illustrated in <FIG>, a directional vibrator is most sensitive to a sound coming in a longitudinal direction thereof. As an angle difference between a longitudinal direction of a directional vibrator and a direction of a sound increases, a sensitivity of the directional vibrator gradually decreases, and a sensitivity of the directional vibrator becomes minimum with respect to a sound coming in perpendicular to the longitudinal direction of the directional vibrator. In the example of <FIG>, the sensitivity of the directional vibrator does not largely decrease up to ±<NUM> degrees with respect to the longitudinal direction of the directional vibrator and the sensitivity decreases to about -5dB at about ±<NUM> degrees. The sensitivity decreases to less than about -15dB at about ±<NUM> degrees. The directional vibrator may be selected to minimize crosstalk based on such directional characteristics of the directional vibrator.

For example, <FIG> illustrate examples in which directional vibrators are selected for sound separation in various situations. In <FIG>, regions A and B marked as fan shapes indicate angle ranges in which a selected directional vibrator is capable of obtaining a sound well.

First, <FIG> shows a case where one sound source is arranged in the direction of <NUM>°. In this case, a directional vibrator arranged in the direction of a sound source may be simply selected from among the plurality of directional vibrators 110_k to obtain sound information. In <FIG>, a plurality of lines extending in a radial direction indicate the plurality of directional vibrators 110_k among which a selected directional vibrator is indicated by a dashed line.

<FIG> shows a case where the first sound source and the second sound source are separated apart from each other by about <NUM> degrees. Also, in this case, a first directional vibrator arranged in the direction of the first sound source may be selected from among the plurality of directional vibrators 110_k to obtain sound information from the first sound source based on an output signal of the first directional vibrator. A second directional vibrator arranged in the direction of the second sound source may be selected from among the plurality of directional vibrators 110_k to obtain sound information from the second sound source based on an output signal of the second directional vibrator. In this case, because a direction of the sound coming from the first sound source is perpendicular to the longitudinal direction of the second directional vibrator, the sound from the first sound source has a small influence upon the second directional vibrator. Likewise, because a direction of the sound coming from the second sound source is perpendicular to the longitudinal direction of the first directional vibrator, the sound from the second sound source has a small influence upon the first directional vibrator. Thus, when the first sound source and the second sound source are separated apart from each other by about <NUM> degrees or more, the control circuit <NUM> may select a directional vibrator having the highest sensitivity to the direction of the first sound source as the first directional vibrator among the plurality of directional vibrators 110_k and a directional vibrator having the highest sensitivity to the direction of the second sound source as the second directional vibrator among the plurality of directional vibrators <NUM>.

<FIG> shows a case where the first sound source and the second sound source are separated apart from each other by less than <NUM> degrees. For example, in <FIG>, an angle between the first sound source and the second sound source may be about <NUM> degrees. In an example of <FIG>, like in the example of <FIG>, the first directional vibrator arranged in the direction of the first sound source is selected from among the plurality of directional vibrators 110_k to obtain sound information of the first sound source, and the second directional vibrator arranged in the direction of the second sound source is selected to obtain sound information of the second sound source. In other words, a directional vibrator having the highest sensitivity to the direction of the first sound source is selected as the first directional vibrator among the plurality of directional vibrators 110_k, and a directional vibrator having the highest sensitivity to the direction of the second sound source is selected as the second directional vibrator among the plurality of directional vibrators <NUM>.

However, in the case that the directional vibrator having the highest sensitivity is selected even when the angle between the first sound source and the second sound source is less than about <NUM> degrees, interference may increase, hindering sufficient sound source separation. For example, the sound of the second sound source received by the selected first directional vibrator may increase and the sound of the first sound source received by the selected second directional vibrator may increase. Then, noise may increase in the sounds respectively received by the first directional vibrator and the second directional vibrator.

<FIG> illustrates an example where a directional vibrator is selected in a manner different from that used in <FIG> in the case that an angle between the first sound source and the second sound source is <NUM> degrees. In <FIG>, a directional vibrator having the lowest sensitivity with respect to a sound source other than a target sound source is selected. For example, among the plurality of directional vibrators 110_k, a directional vibrator having the lowest sensitivity to the second sound source while sufficiently obtaining a sound from the first sound source may be selected to obtain sound information of the first sound source and a directional vibrator having the lowest sensitivity to the first sound source while sufficiently obtaining a sound from the second sound source may be selected to obtain sound information of the second sound source.

In other words, the control circuit <NUM> may select a directional vibrator having the lowest sensitivity with respect to the direction of the second sound source as the first directional vibrator from among the plurality of directional vibrators 110_k arranged within an angle range around the direction of the first sound source. The control circuit <NUM> may select a directional vibrator having the lowest sensitivity with respect to the direction of the first sound source as the second directional vibrator from among the plurality of directional vibrators 110_k arranged within an angle range around the direction of the second sound source. In the example of <FIG>, a directional vibrator located in a direction of +<NUM> degrees is selected as the first directional vibrator, and a directional vibrator located in a direction of -<NUM> degrees is selected as the second directional vibrator. The first directional vibrator is separated apart from the second sound source by <NUM> degrees, and the second directional vibrator is separated apart from the first sound source by <NUM> degrees.

After the first directional vibrator and the second directional vibrator are selected, the sound of the first sound source and the sound of the second sound source may be obtained using the first directional vibrator and the second directional vibrator, respectively. Letting an output signal of the first directional vibrator be C1, an output signal of the second directional vibrator be C2, a sound signal of the first sound source be S1, and a sound signal of the second sound source be S2; then, C1 = S1 in <FIG>.

In <FIG>, C1 = S1 + <NUM>/<NUM> × S2 and C2 = <NUM>/<NUM> × S1 + S2. Herein, <NUM>/<NUM> may be determined as an example based on the directional characteristics of a directional vibrator in the example illustrated in <FIG> and may vary with directional characteristics of the directional vibrator. In <FIG>, the sound of the second sound source, received in the first directional vibrator, is about <NUM>/<NUM> of the sound of the first sound source, such that sufficient sound source separation is possible.

In <FIG>, C1 = S1 + <NUM>/<NUM> × S2 and C2 = <NUM>/<NUM> × S1 + S2. Herein, <NUM>/<NUM> may be determined as an example based on the directional characteristics of a directional vibrator in the example illustrated in <FIG> and may vary with directional characteristics of the directional vibrator. In <FIG>, because a ratio of S1 to S2 is <NUM>:<NUM> in C1, sound source separation may not be sufficiently performed.

In <FIG>, C1 = <NUM>/<NUM> × S1 + <NUM>/<NUM> × S2 and C2 = <NUM>/<NUM> × S1 + <NUM>/<NUM> × S2. Herein, <NUM>/<NUM> and <NUM>/<NUM> may be determined as an example based on the directional characteristics of a directional vibrator in the example illustrated in <FIG> and may vary with directional characteristics of the directional vibrator. In <FIG>, because a ratio of S1 to S2 is <NUM>:<NUM> in C1, sound source separation may be sufficiently performed.

<FIG> is a graph illustrating a voice recognition test result with respect to a combination of selected directional vibrators. A voice recognition test has been carried out in a manner in which sounds including multiple words are output from the first sound source and the second sound source, respectively, and a rate of the number of words recognized inaccurately by a selected directional vibrator is measured. The leftmost graph corresponds to a case of <FIG> in which a rate of inaccurate recognition is about <NUM>%. Since one sound source is used, the case of <FIG> may be used as a reference where there is an influence of interference when two sound sources are used. The second graph from the left corresponds to a case of <FIG> in which a rate of inaccurate recognition is about <NUM>%, which increases slightly from the reference. The third graph from the left corresponds to a case of <FIG> in which a rate of inaccurate recognition is about <NUM>%, which increases greatly from the reference. The rightmost graph corresponds to a case of <FIG> in which a rate of inaccurate recognition is about <NUM>%, showing a result similar with the case of <FIG>. Thus, by selecting the directional vibrator having the lowest sensitivity with respect to a sound source other than the target sound source, sound source separation may be sufficiently achieved.

The sound source separation apparatuses <NUM> and <NUM> may separate sounds coming from two different sound sources simply in terms of hardware merely with the plurality of directional vibrators 110_k without a need for complex computational processing. Thus, the sound source separation apparatuses <NUM> and <NUM> may be miniaturized and thus mounted on a small-size electronic product. The electronic product having mounted thereon the sound source separation apparatus <NUM> or <NUM> according to the current embodiment of the disclosure may relatively accurately separate a user's sound source even in a noisy environment.

The foregoing description has been made in which one directional vibrator is selected for one sound source to obtain sound information, but the disclosure is not limited thereto. Depending on a need, one or more adjacent directional vibrators for one sound source, e.g., two or three continuously adjacent directional vibrators, may be selected to obtain sound information. Also in this case, when there are two or more sound sources, a group of directional vibrators having the lowest sensitivity to a sound source other than the target sound source may be selected.

So far, a description has been made in which a sound source is separated merely with an output signal of a selected directional vibrator without separate signal processing in terms of software to obtain sound information. For example, sound information of the first sound source may be obtained with the output signal of the first directional vibrator selected for the first sound source, and the sound information of the second sound source may be obtained with the output signal of the second directional vibrator selected for the second sound source. However, sound source separation may be performed by computing the output signal of the first directional vibrator and the output signal of the second directional vibrator in terms of software. For example, in the example of <FIG>, C1 = S1 + <NUM>/<NUM> × S2 and C2 = <NUM>/<NUM> × S1 + S2, such that C1 may be expressed with S1 and C2 may be expressed with S2 by solving simultaneous equations. More specifically, C1' = C1 - <NUM>/<NUM> × C2 = <NUM>/<NUM> × S1 and C2' = C2 - <NUM>/<NUM> × C1 = <NUM>/<NUM> × S2.

Thus, when a rate of the sound of the second sound source contributing to the output signal of the first directional vibrator selected for the first sound source and a rate of the sound of the first sound source contributing to the output signal of the second directional vibrator selected for the second sound source are given, the control circuit <NUM> may obtain sound information of the first sound source and sound information of the second sound source by computing the output signal of the first directional vibrator and the output signal of the second directional vibrator. More generally, letting the output signal of the first directional vibrator be C1, the output signal of the second directional vibrator be C2, the sound signal of the first sound source be S1, the sound signal of the second sound source be S2, a rate of the sound of the second sound source contributing to the output signal of the first directional vibrator be α, and a rate of the sound of the first sound source contributing to the output signal of the second directional vibrator be β;.

Herein, α and β indicate unique sensitivity characteristics of the first directional vibrator and the second directional vibrator with respect to a direction of an incident sound, and may be known in advance by measurement. For example, α may be determined based on an angle between the first directional vibrator and the second sound source, and β may be determined based on an angle between the second directional vibrator and the first sound source. In the case that all of the directional vibrators 110_k included in the sound source separation apparatuses <NUM> and <NUM> have the same directional characteristics, α and β may have an identical value. However, when the directional vibrators 110_k have different directional characteristics, α and β may have different values.

Values of sensitivity of each directional vibrator measured in advance with respect to a direction of a sound incident to each directional vibrator may be stored in the memory <NUM>. For example, a pair of an incident angle of a sound incident to each directional vibrator and a corresponding sensitivity may be stored in the memory <NUM>. The control circuit <NUM> may then obtain α and β from sensitivity characteristics of the directional vibrators stored in the memory <NUM> and compute the output signal of the first directional vibrator and the output signal of the second directional vibrator based on α and β, thus obtaining the sound information of the first sound source and the sound information of the second sound source. Hence, a directional vibrator having the highest sensitivity with respect to a direction of a target sound source may be selected from among the plurality of directional vibrators 110_k, and then sound source separation may be performed through software signal processing.

<FIG> is a plan view of a sound source separation apparatus according to another embodiment of the disclosure, and <FIG> is a cross-sectional view taken along line A-A' of the sound source separation apparatus of <FIG>. Referring to <FIG>, a sound source separation apparatus <NUM> may further include an omnidirectional vibrator <NUM> reacting to an input sound regardless of a direction thereof. The other elements of the sound source separation apparatus <NUM> of <FIG> may be the same as those of the sound source separation apparatus <NUM> of <FIG>.

The omnidirectional vibrator <NUM> may be located, for example, in the sound outlet <NUM> and may be located on the same plane as the plurality of directional vibrators 110_k. In this case, the plurality of directional vibrators 110_k may be arranged to surround the omnidirectional vibrator <NUM>. However, the omnidirectional vibrator <NUM> is not limited to the above-described location and may also have various other locations. For example, the omnidirectional vibrator <NUM> may be located outside the case <NUM>.

Unlike the directional vibrators 110_k, the omnidirectional vibrator <NUM> may have almost the same output in reaction to sound input from all directions. To this end, the omnidirectional vibrator <NUM> may have a form of a circular thin film. When the omnidirectional vibrator <NUM> is located in the sound outlet <NUM>, the omnidirectional vibrator <NUM> may be located in such a manner that the center of the circular omnidirectional vibrator <NUM> is aligned with a center point of the sound outlet <NUM>.

Although the output of the omnidirectional vibrator <NUM> is constant regardless of the direction of the input sound, a phase of vibration of the omnidirectional vibrator <NUM> may vary depending on the direction of the input sound. For example, as schematically illustrated in <FIG>, the phase of vibration of the omnidirectional vibrator <NUM> may be the same as a phase of vibration of the directional vibrator 110_1 that is located in the direction of the input sound among the directional vibrators 110_k. In addition, the phase of vibration of the omnidirectional vibrator <NUM> may be opposite to a phase of vibration of the directional vibrator 110_9 located in a direction opposite to the direction of the input sound among the directional vibrators 110_k.

<FIG> is a graph showing an example of phases of vibration of the omnidirectional vibrator <NUM> and the two directional vibrators 110_1 and 110_9 facing each other in a case when sound is input from one direction. For example, it is assumed that the directional vibrators 110_1 and 110_9 are located to face each other and that sound is input in a direction from the directional vibrator 110_1 toward the directional vibrator 110_9. As illustrated in <FIG>, the phase of vibration of the directional vibrator 110_1 is <NUM>° opposite to the phase of vibration of the directional vibrator 110_9. The phase of vibration of the omnidirectional vibrator <NUM> is the same as the phase of vibration of the directional vibrator 110_1 and is <NUM>° opposite to the phase of vibration of the directional vibrator 110_9.

Therefore, a direction of the sound source may be accurately detected with reference to phases of vibration of the omnidirectional vibrator <NUM> and the directional vibrators 110_k. For example, the control circuit <NUM> may select a directional vibrator having the lowest strength of an output signal from among the plurality of directional vibrators 110_k. The control circuit <NUM> may compare a phase of a directional vibrator arranged at +<NUM>° with respect to the directional vibrator having the lowest output signal strength and a phase of a directional vibrator arranged at -<NUM>° with respect the directional vibrator having the lowest output signal strength with a phase of the omnidirectional vibrator <NUM>. Then, the control circuit <NUM> may determine as a direction of a sound source, an arrangement direction of a directional vibrator having the closest phase to that of the omnidirectional vibrator <NUM>.

The sound source separation apparatuses <NUM>, <NUM>, <NUM>, and <NUM> according to the afore-described embodiments of the disclosure may be applied in various electronic apparatuses. The sound source separation apparatuses <NUM>, <NUM>, <NUM>, and <NUM> may be implemented as chip-solution sensors and may perform tracking of a plurality of sound sources, noise cancellation, and spatial recording in the field of mobile devices, information technology (IT), home appliances, and vehicles or may be used in the field of panoramic shooting, augmented reality, and virtual reality.

Hereinbelow, electronic apparatuses using the above-described sound source separation apparatuses <NUM>, <NUM>, <NUM>, and <NUM> will now be described.

<FIG> is a block diagram illustrating a schematic structure of an Internet of things (IoT) apparatus according to an embodiment of the disclosure, and <FIG> is a schematic diagram for describing an example of operation of the IoT apparatus of <FIG> in daily life.

An IoT apparatus <NUM> may include a sound source separation apparatus <NUM> that receives a speech signal provided by a user, a memory <NUM> in which one or more application modules <NUM> using the signal received by the sound source separation apparatus <NUM> as an input variable are stored, and a processor <NUM> that executes the application modules <NUM>. The IoT apparatus <NUM> may further include a communicator <NUM>.

The IoT apparatus <NUM> may further include a movable device <NUM> capable of rotating or moving. The movable device <NUM> may be controlled in terms of rotation or movement in a direction based on an execution result of the application modules <NUM> executed using the signal received by the sound source separation apparatus <NUM> as an input variable. The direction of rotation or motion may be, for example, a direction facing or avoiding a direction of sound detected by the sound source separation apparatus <NUM>. The movable device <NUM> may be implemented to have an output function capable of outputting the execution result of the application module <NUM> to the user. The movable device <NUM> may be an output device, e.g., a speaker or a display. The sound source separation apparatus <NUM> may include any one of, a modification of, or a combination of the sound source separation apparatuses <NUM>, <NUM>, <NUM>, and <NUM> according to the afore-described embodiments of the disclosure.

The processor <NUM> may control all operations of the IoT apparatus <NUM>. The processor <NUM> may control operations of the sound source separation apparatus <NUM>, the movable device <NUM>, and the communicator <NUM>, use related control signals, and execute programs stored in the memory <NUM>. The memory <NUM> may include a movable device control module <NUM> programmed to rotate or move the movable device <NUM> in a certain direction based on a control signal. The movable device control module <NUM> may control the movable device <NUM> to rotate or move in a direction facing or avoiding the direction of sound detected by the sound source separation apparatus <NUM>, by using a signal detected by the sound source separation apparatus <NUM> and an execution result of an application associated with the signal. However, this is only an example, and a movable device control direction based on an application execution result reflecting the signal sensed by the sound source separation apparatus <NUM> may be various.

The memory <NUM> may further include a learning module <NUM> programmed to learn whether the signal received by the sound source separation apparatus <NUM> is a valid input signal, in association with the direction of sound. For example, the learning module <NUM> may repeatedly generate and accumulate, as learning data, the direction of sound detected by the sound source separation apparatus <NUM> and a result of determining whether the signal is a valid signal, and extract statistic features from the accumulated learning data, thereby treating sound input from a certain direction, as an invalid signal. For example, a sound coming from a direction that is different from a user's voice may be processed as noise, or voices of a plurality of users in different directions may be separately recognized. The memory <NUM> may also store various other programs and data required for the processor <NUM> to control overall operations of the IoT apparatus <NUM>.

The memory <NUM> may include at least one type of storage medium among, for example, flash memory, a hard disk, a multimedia card micro, card-type memory (e.g., secure digital (SD) or extreme digital (XD) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), programmable ROM (PROM), magnetic memory, a magnetic disc, and an optical disc.

The communicator <NUM> may communicate with an external device by using, but not limited to, Bluetooth communication, Bluetooth low energy (BLE) communication, near field communication (NFC), wireless local area network (WLAN) communication, Zigbee communication, Infrared Data Association (IrDA) communication, WiFi direct (WFD), ultra-wideband (UWB) communication, Ant+ communication, WiFi communication, or the like.

Referring to <FIG>, the movable device <NUM> included in the IoT apparatus <NUM> is illustrated as a rotatable speaker. A speaker will be described below as an example of the movable device <NUM>, but the movable device <NUM> is not limited thereto. The IoT apparatus <NUM> may determine a direction of an input sound signal S1 or S2 and rotate the speaker to face the determined direction. The IoT apparatus <NUM> may identify a valid signal among the two sound signals S1 and S2 input at the same time.

The IoT apparatus <NUM> may identify a valid signal among the input sound signals S1 and S2. For example, the IoT apparatus <NUM> may separately recognize the sound signal S1 from a user U and the sound signal S2 from a sound source NU that is not a user. The IoT apparatus <NUM> may distinguish between the sound signals S1 and S1 by learning whether an input signal is a valid signal, in association with a direction of input sound. As such, for example, after learning and determining that an invalid signal is continuously input from a certain fixed direction, e.g., a television (TV), the IoT apparatus <NUM> may rotate the speaker toward the direction of the sound signal S2 that is determined as a valid signal among the input sound signals S1 and S2, and execute an application related to the sound signal S2. The IoT apparatus <NUM> may be used as an artificial intelligence (AI) speaker and may also be applied to various other objects to enhance utilization of inherent functions of the objects.

<FIG> is a block diagram of a vehicle voice interface apparatus <NUM> according to an embodiment, and <FIG> is a schematic diagram for describing an example of operation of the vehicle voice interface apparatus <NUM> in a vehicle <NUM>.

Referring to <FIG>, the vehicle voice interface apparatus <NUM> may include a sound source separation apparatus <NUM> and a valid signal extraction module <NUM>. The valid signal extraction module <NUM> may include a memory configured to store a program for extracting a valid signal, and a processor configured to execute the program. The sound source separation apparatus <NUM> may include any one of, a modification of, or a combination of the sound source separation apparatuses <NUM>, <NUM>, <NUM>, and <NUM> according to the afore-described embodiments of the disclosure.

The valid signal extraction module <NUM> may determine whether a signal received by the sound source separation apparatus <NUM> is a valid signal, based on a direction thereof, and transmit the signal to a vehicle control module upon determining that the signal is a valid signal. The valid signal extraction module <NUM> may separate and then remove sound signals from directions other than a direction of a driver, from a sound signal input from various directions, and transmit the sound signal to the vehicle control module.

Referring to <FIG>, the sound source separation apparatus <NUM> included in the vehicle <NUM> detects a sound signal S1 separately from a driver DR and sound signals S2, S3, and S4 from passengers PA. The sound source separation apparatus <NUM> may detect directions of the received sound signals S1, S2, S3, and S4 and transmit the result of separately detecting the sound signals S1, S2, S3, and S4 to the valid signal extraction module <NUM>. The valid signal extraction module <NUM> may transmit only the sound signal S1 from the driver DR to a vehicle control module <NUM>.

<FIG> is a block diagram of a spatial recording apparatus according to an embodiment of the disclosure. Referring to <FIG>, a spatial recording apparatus <NUM> according to an embodiment of the disclosure may include a sound source separation apparatus <NUM>, a processor <NUM> configured to determine a direction of a sound input to the sound source separation apparatus <NUM>, by analyzing a signal detected by the sound source separation apparatus <NUM>, and a memory <NUM> configured to store programs for signal processing operations of the processor <NUM>. Also, the memory may store results of the operations performed by the processor <NUM>. The sound source separation apparatus <NUM> may include any one of, a modification of, or a combination of the sound source separation apparatuses <NUM>, <NUM>, <NUM>, and <NUM> according to the afore-described embodiments of the disclosure. The sound source separation apparatus <NUM> may record ambient sound in association with a direction thereof. The sound source separation apparatus <NUM> may estimate an input direction of sound in a high resolution.

The spatial recording apparatus <NUM> may selectively record a desired sound source or separately recording sounds of sound sources in different directions, by using the estimation result of the input direction of sound. The spatial recording apparatus <NUM> may further include a multi-channel speaker <NUM> to reproduce the recorded sound in accordance with a direction thereof. The processor <NUM> controls the multi-channel speaker <NUM> such that a sound signal stored in the memory <NUM> may be played suitably for a direction. By playing a recorded sound source suitably for a direction, a sense of realism of recorded content may be augmented, and a sense of immersion and a sense of reality may be improved. The spatial recording apparatus <NUM> may be used in an augmented reality (AR) or virtual reality (VR) apparatus.

<FIG> is a block diagram of an omnidirectional camera <NUM> according to an embodiment of the disclosure. Referring to <FIG>, the omnidirectional camera <NUM> according to an embodiment of the disclosure is a camera capable of capturing a panoramic image of objects in all directions. The omnidirectional camera <NUM> may include a sound source separation apparatus <NUM>, an omnidirectional shooting module <NUM>, a processor <NUM> configured to control the sound source separation apparatus <NUM> and the omnidirectional shooting module <NUM> in such a manner that a directional sound signal detected by the sound source separation apparatus <NUM> matches an omnidirectional image signal captured by the omnidirectional shooting module <NUM>, and a memory <NUM> configured to store the directional sound signal and the omnidirectional image signal. The sound source separation apparatus <NUM> may include any one of, a modification of, or a combination of the sound source separation apparatuses <NUM>, <NUM>, <NUM>, and <NUM> according to the afore-described embodiments of the disclosure, and may detect sounds from all directions and separate sounds coming from different directions.

A general panoramic shooting module may be used as the omnidirectional shooting module <NUM>. For example, the omnidirectional shooting module <NUM> may include optical lenses, an image sensor, etc. in a <NUM>°-rotatable body. Under control of the processor <NUM>, a sound in a direction corresponding to a photographing direction of the omnidirectional shooting module <NUM>, among signals sensed by the sound source separation apparatus <NUM>, may be separated and selectively stored in the memory <NUM>. As such, the omnidirectional camera <NUM> may store a <NUM>° panoramic image signal and a sound signal matching the image signal, in the memory <NUM>. The image and sound signals may be reproduced by a display device including a multi-channel speaker, may maximize a sense of realism, and may be used in an AR/VR device.

According to an embodiment, a hybrid technique, in which, the sound separation devices and methods of the disclosure are combined with the techniques of ICA, GSS or DNN. For instance, according to an embodiment, a first result from source separation device of the disclosure can be used in conjunction with the analysis methods of ICA, GSS and DNN to further improve measurable characteristics such as accuracy, etc..

Electronic apparatuses according to the afore-described embodiments of the disclosure may include a processor, a memory configured to store and execute program data, a permanent storage such as a disk drive, a communication port configured to communicate with an external device, and a user interface device, e.g., a touch panel, keys, or buttons.

Methods implemented using software modules or algorithms in the electronic apparatuses according to the afore-described embodiments of the disclosure may be recorded on a computer-readable recording medium as computer-readable codes or program instructions executable by the processor. Examples of the computer-readable recording medium include magnetic storage media (e.g., read-only memory (ROM), random-access memory (RAM), floppy disks, and hard disks) and optical recording media (e.g., compact disc-ROM (CD-ROM) and digital versatile discs (DVDs)). The computer-readable recording medium may also be distributed over network coupled computer systems so that a computer-readable code is stored and executed in a distributed fashion. The medium may be read by a computer, stored in a memory, and executed by a processor.

Claim 1:
A sound source separation apparatus (<NUM>) comprising:
a sound inlet (<NUM>) configured to receive a sound;
a sound outlet (<NUM>) configured to output the sound received through the sound inlet;
a plurality of directional vibrators (110_k) arranged on a plane between the sound inlet and the sound outlet, the plane being perpendicular to a central axis of the sound inlet, and the plurality of directional vibrators being arranged around a center point on the plane corresponding to the central axis of the sound inlet in such a manner that one or more of the plurality of directional vibrators is configured to selectively react based on a direction of the sound received through the sound inlet; and
characterized in that the sound source separation apparatus further comprises a control circuit (<NUM>) configured to:
determine a first direction of a first sound source and a second direction of a second sound source that is different from the first sound source, based on strengths of output signals of the plurality of directional vibrators, wherein the second direction of the second sound source is determined during a first time in which the first sound from the first sound source decreases and the second sound from the second sound source increases such that the second sound is relatively strong compared to the first sound and the first direction of the first sound source is determined during a second time in which the second sound from the second sound source decreases and the first sound increases such that the first sound is relatively strong compared to the second sound, and
select a first directional vibrator and a second directional vibrator from among the plurality of directional vibrators based on the determined directions to separately obtain a first sound from the first sound source and a second sound from the second sound source,
wherein the first directional vibrator is different from the second directional vibrator.