Patent Description:
The present section is intended to introduce the reader to various aspects of art, which may be related to various aspects of the present disclosure that are described and/or claimed below.

In the field of audio recognition, some audio recognition systems are specially designed to recognize sounds such as, for example, gunshots, baby's crying, glass breaking, etc. These systems - which are to be differentiated from automatic speech recognition systems since they are not designed to recognize speech but only sounds - may be used for a large variety of applications, for example but not limited to home security.

One of the problems these audio recognition systems are facing is the difficulty of discriminating artificially emitted sounds coming from loudspeakers of various electronic equipment (such as a TV set, a radio receiver, etc.) that may be present in the environment where these systems are implemented against naturally real-life emitted sounds. For example, in the field of home security systems, a user not at home should be notified if a gunshot sound or a glass breaking sound is detected in or around his home, but only if the detected sound is a "real-life" sound, i.e. if a gunshot was actually fired or a glass was actually broken. On the other hand, the user shouldn't be notified, for example, if the detected sound is part of the sound track of a movie broadcast on television and currently being watched by another family member. At the present time, many existing audio recognition systems are not very efficient when it comes to differentiate real-life sounds (i.e. naturally emitted sounds) versus apparatus-generated sound (i.e. artificially emitted sounds). In the field of home security, the resulting misclassification of sound events leads to the generation of a lot of "false positives" notified to the end user, who may ultimately lose confidence in such security systems.

In an attempt to address these drawbacks, some existing solutions rely on low level signal processing techniques to process audio signals captured by some microphones, with the aim of cancelling or at least reducing artificial sounds in the processed output audio signals. More particularly, subtractor circuitry is used for subtracting artificial sounds from a main audio signal captured by a microphone, and the resulting signal is processed by a sound recognition engine. The main issue with these solutions is that the microphones capture an unpredictable mix between real-life and artificial sounds, not only in term of amplitude but also in term of phase and potential echoes, due to the reflection of the sound on objects and/or structures present in the environment (such as, for example, walls). The sound captured by the microphones is thus a complex addition of reflected sounds and direct sounds. The amplitude and phase of all these sounds are unpredictable and make the subtraction very difficult to properly perform at the subtractor circuitry level. Consequently, such solutions require many parameter adjustments (for example, to set the gain adaptation coefficients of the subtractor circuitry) which are highly dependent on the environment (such as, for example, the room configuration, the location of the sound sources in the room, the sound power of the apparatus emitting some artificial sounds, the location and orientation of the microphones, etc.), and finally provide only poor or mixed results.

An example of distinguishing natural from artificial sounds based on an estimated dynamic range of the sounds is provided by the European patent application <CIT>).

International patent application <CIT>) discloses a corresponding method of distinguishing sounds based on a location estimation of the sound sources within an acoustic environment.

It would hence be desirable to provide a technique that would avoid at least some of these drawbacks of the prior art, and that would notably allow discriminating artificially emitted sounds against naturally emitted sounds in a more accurate way.

According to the present disclosure, a method for recognizing at least one naturally emitted sound produced by a real-life sound source in an environment comprising at least one artificial sound source is provided in accordance with independent claim <NUM>. Further embodiments are as defined by the respective dependent claims.

The present disclosure also concerns an audio recognition device for recognizing at least one naturally emitted sound produced by a real-life sound source in an environment comprising at least one artificial sound source as defined by independent claim <NUM>.

According to another aspect, the present disclosure also pertains to an audio recognition system for recognizing at least one naturally emitted sound produced by a real-life sound source in an environment comprising at least one artificial sound source as defined by independent claim <NUM>.

According to one implementation, the different steps of the method for recognizing at least one naturally emitted sound produced by a real-life sound source in an environment comprising at least one artificial sound source as described here above are implemented by one or more software programs or software module programs comprising software instructions intended for execution by at least one data processor of an audio recognition device.

Thus, another aspect of the present disclosure pertains to at least one computer program product downloadable from a communication network and/or recorded on a medium readable by a computer and/or executable by a processor, comprising program code instructions for implementing the method as described above. More particularly, this computer program product includes instructions to command the execution of the different steps of a method for recognizing at least one naturally emitted sound produced by a real-life sound source in an environment comprising at least one artificial sound source, as mentioned here above and as defined in corresponding independent claim <NUM>.

This program can use any programming language whatsoever and be in the form of source code, object code or intermediate code between source code and object code, such as in a partially compiled form or any other desirable form whatsoever.

According to one embodiment, the methods/apparatus may be implemented by means of software and/or hardware components. In this respect, the term "module" or "unit" can correspond in this document equally well to a software component and to a hardware component or to a set of hardware and software components.

A software component corresponds to one or more computer programs, one or more sub-programs of a program or more generally to any element of a program or a piece of software capable of implementing a function or a set of functions as described here below for the module concerned. Such a software component is executed by a data processor of a physical entity (terminal, server, etc.) and is capable of accessing hardware resources of this physical entity (memories, recording media, communications buses, input/output electronic boards, user interfaces, etc.).

In the same way, a hardware component corresponds to any element of a hardware unit capable of implementing a function or a set of functions as described here below for the module concerned. It can be a programmable hardware component or a component with an integrated processor for the execution of software, for example an integrated circuit, a smartcard, a memory card, an electronic board for the execution of firmware, etc..

In addition, the present disclosure also concerns a non-transitory computer-readable medium comprising a computer program product recorded thereon and capable of being run by a processor, including program code instructions for implementing the method for recognizing at least one naturally emitted sound produced by a real-life sound source in an environment comprising at least one artificial sound source as described above and as defined by independent claim <NUM>.

The computer readable storage medium as used herein is considered a non-transitory storage medium given the inherent capability to store the information therein as well as the inherent capability to provide retrieval of the information therefrom. A computer readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. It is to be appreciated that the following, while providing more specific examples of computer readable storage mediums to which the present principles can be applied, is merely an illustrative and not exhaustive listing as is readily appreciated by one of ordinary skill in the art: a portable computer diskette, a hard disk, a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the disclosure, as claimed.

It must also be understood that references in the specification to "one embodiment" or "an embodiment", indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic.

Embodiments of the present disclosure can be better understood with reference to the following description and drawings, given by way of example and not limiting the scope of protection, and in which:.

The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the disclosure.

The present disclosure relates to a method that allows recognizing at least one naturally emitted sound produced by a real-life sound source in an environment comprising at least one artificial sound source, the artificial sound source being likely to produce the same type of sounds as the real-life sound source. In other words, the proposed technique makes it possible to discriminate artificially emitted sounds against naturally emitted sounds in a more accurate way than current existing systems. By "artificially emitted sounds", it is understood in the context of the disclosure sounds that are produced by an electronic apparatus (for example a television set, a radio receiver, a smartphone, a tablet, etc.) and emitted by one or more loudspeakers. In contrast, "naturally emitted sounds" are not emitted via a loudspeaker of an electronic apparatus, but correspond to "real-life" sounds. The same type of sound may be artificially emitted or naturally emitted. For example, a dog barking sound belongs to the category of "naturally emitted sounds" if it is produced by a real flesh and blood dog, but it belongs to the category of "artificially emitted sounds" if it is part of the soundtrack of a movie broadcast on a TV set and showing a dog barking.

As it will be described more fully hereafter with reference to the accompanying figures, it is proposed in one embodiment of the present disclosure to use a combination of two recognition engines for discriminating a sound event originating from an artificial sound source from the same type of sound event that may originate from live or "real-life" sound source. Rather than carrying out the discrimination at the signal level, by using low-level signal processing technique at a subtractor circuitry in an attempt to cancel artificial emitted sound against naturally emitted sound in a main audio signal before sounds are recognized, it is proposed in the present disclosure to perform the discrimination at a higher and symbolic level, after the sounds are recognized.

This disclosure may, however, be embodied in many alternate forms and should not be construed as limited to the embodiments set forth herein. Accordingly, while the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that there is no intent to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure as defined by the claims. In the drawings, like or similar elements are designated with identical reference signs throughout the several views thereof.

While not explicitly described, the present embodiments and variants may be employed in any combination or sub-combination.

<FIG> is a flow chart for describing a method for recognizing at least one naturally emitted sound produced by a real-life sound source in an environment including at least one artificial sound source, according to an embodiment of the present disclosure. The method is implemented by an audio recognition device, which is further detailed in one embodiment later in this document, and which includes at least one processor adapted and configured for carrying out the steps described hereafter.

At step <NUM>, the audio recognition device simultaneously obtains a first audio signal and a second audio signal. By simultaneously obtained, it is meant here that the first and the second audio signal are acquired at about the same moment, i.e. they correspond to the recording or the capture of some sounds emitted during a similar timeframe.

The first audio signal is obtained from a first microphone MIC located in the environment. This first microphone is intended to capture both artificial and naturally emitted sounds. According to a particular feature, the first microphone MIC is an omnidirectional microphone, thus able to pick up sounds from all directions equally well and with a good sensitivity.

The second audio signal is obtained from an audio acquisition device AAD associated with at least one artificial sound source ASSr located in the environment. By "associated with", it is meant here that the audio acquisition device is designed and/or located and/or oriented so as to capture in a privileged way (i.e. only, or at least mainly) sounds emitted by the artificial sound source to which it is associated with.

In an embodiment, the audio acquisition device AAD may take the form of a second microphone located in the vicinity of the artificial sound source. For example, such a second microphone may be placed within fifty centimeters of a loudspeaker of the artificial sound source. According to a particular feature, the second microphone may be a directional microphone designed to be more sensitive in one direction, and directed so as to accurately acquire sounds coming from the artificial sound source. Sounds coming from other directions, i.e. from other sources than the artificial sound source to which the second microphone is associated, are thus not captured or only poorly captured by the second microphone. According to another embodiment, the audio acquisition device AAD may take the form of a sound acquisition interface (e.g. a sound acquisition card) embedded into the audio recognition device, and an audio output of the artificial sound source ASSr may then be directly connected to an input of the sound acquisition interface. The connection may be of any connection type - wired or wireless - allowing to transmit an audio signal (such as, for example, connection via High Definition Multimedia Interface HDMI cable, TOSLINK optical audio cable, RCA connectors, Bluetooth, etc.). In that way, the audio acquisition device is connected to the artificial sound source in an acoustical isolated way such that the only sounds acquired by the audio acquisition device are those coming from the artificial sound source(s) to which it is connected.

At step <NUM>, the first audio signal is analyzed, to deliver a first list L1 of sound classes corresponding to sounds recognized in the first audio signal.

In one embodiment, analyzing the first audio signal includes computing a first probability distribution over a set of reference sound classes, wherein each reference sound class of the set of reference sound classes is associated with a score representing a probability of presence of a sound belonging to the reference sound class in the first audio signal. For example, a set of reference sound classes may comprise the following sound classes: dog barking, baby's crying, screaming, glass breaking, slamming door, gunshot. As a result of the analysis of the first audio signal, an example of a first probability distribution computed may be: screaming - <NUM>%, baby's crying - <NUM>%, dog barking - <NUM>%, glass breaking - <NUM>%, slamming door - <NUM>%, gunshot - <NUM>%. The computed first probability distribution may then be used to build the first list L1 of sound classes delivered at step <NUM>. According to a particular feature, the first list L1 of sound classes includes sound classes having an associated score greater than or equal to a first predetermined threshold, with respect to the first probability distribution. For example, among the set of reference sound classes, only sound classes having an associated score greater than or equal to <NUM>% may be included in the first list of sound classes L1. Referring back to the previous example of first probability distribution, the first list L1 would then contains two sound classes: screaming sound class and baby's crying sound class. According to alternative particular feature, the first list L1 of sound classes includes a first predetermined number n<NUM> of sound classes, corresponding to the n<NUM> sound classes having the greatest scores, with respect to the first probability distribution. For example, among the set of reference sound classes, only the sound class having the greatest score may be included in the first list of sound classes (n<NUM>=<NUM>). Referring back to the previous illustrative example of a first probability distribution, the first list L1 would then contains only one sound class, i.e. the screaming sound class.

At step <NUM>, the second audio signal is analyzed, to deliver a second list L2 of sound classes corresponding to sounds recognized in the second audio signal.

Similar techniques and/or embodiments than those previously described in relation with step <NUM> may be implemented for analyzing the second audio signal. Thus, In one embodiment, analyzing the second audio signal includes computing a second probability distribution over a set of reference sound classes, wherein each reference sound class of the set of reference sound classes is associated with a score representing a probability of presence of a sound belonging to the reference sound class in the second audio signal. The set of reference sound classes is the same one than the one used in step <NUM> (or at least it includes common elements). The second computed probability distribution may then be used to build the second list L2 of sound classes delivered at step <NUM>. According to a particular feature, in a similar way than the one previously described in relation with step <NUM>, the second list L2 of sound classes includes sound classes having an associated score greater than or equal to a second predetermined threshold, with respect to the second probability distribution. According to an alternative particular feature, the second list L2 of sound classes includes a second predetermined number n<NUM> of sound classes, corresponding to the n<NUM> sound classes having the greatest scores, with respect to the second probability distribution.

The first list L1 of sound classes and the second list L2 of sound classes may be built using a same technique, i.e. both are built by keeping sound classes greater than or equal to some predetermined thresholds, or both are built by keeping some predetermined numbers of sound classes having the greatest scores. In that case, depending on the technique used, the second predetermined threshold may have the same value as the first predetermined threshold, or the second predetermined number n<NUM> of sound classes may have the same value as the first predetermined number n<NUM> of sound classes. However, according to another feature, the second predetermined threshold may have a different value than the first predetermined threshold, or the second predetermined number n<NUM> of sound classes may have a different value than the first predetermined number n<NUM> of sound classes. Indeed, because the second audio signal may be captured in an acoustical isolated way in some embodiment, it could be considered as less noisy than the first audio signal, which can justify using different values of threshold or predetermined numbers of sound classes to compute the first list L1 of sound classes and the second list L2 of sound classes from their respective first and second probability distributions.

In another embodiment, the first list L1 of sound classes and the second list L2 of sound classes may be built using different techniques, i.e. one is built by keeping sound classes greater than or equal to a predetermined threshold, and the other is built by keeping a predetermined number of sound classes having the greatest scores.

At step <NUM>, the first list L1 of sound classes and the second list L2 of sound classes are analyzed, and a third list of sound classes L3 is built from the first list L1 and the second list L2. More particularly, the first list L1 and the second list L2 are compared, so as to deliver a third list of sound classes L3, comprising only sound classes included in the first list of sound classes L1 which are not included in the second list of sound classes L2. In other words, the sound classes corresponding to artificially emitted sound detected in the second audio signal are removed from the list of sound classes corresponding to sound detected in the first audio signal. In that way, the third list of sound classes L3 corresponds to a filtered list of sound classes comprising only sound classes corresponding to naturally emitted sound. For example, if the first list of sound classes L1 includes two sound classes "dog barking" and "baby's crying", and if the second list of sound classes L2 includes only one sound class "dog barking", the generated third list of sound classes L3 will comprise only one sound class "baby's crying". The proposed technique thus offers a solution to discriminate artificially emitted sounds against naturally emitted sounds, the solution being totally independent of the location of the sound sources in the room and not requiring any parameters adjustment.

According to one embodiment, an optional step <NUM> may be carried out, comprising sending a notification to at least one communication terminal when the third list L3 of sound classes is not empty. The communication terminal may be, for example, a smartphone or a tablet. Such a notification can take, for example, the form of an email, a short message text (SMS), a push notification, or other forms, and it may be useful notably in (but not limited to) the field of home security, when a user may be interested in being notified of abnormal sound events (identified in the set of reference sound classes) occurring in or near his house when he is not at home.

According to another aspect, and as schematically illustrated in relation with <FIG> in one embodiment, the present disclosure also pertains to an audio recognition device for recognizing at least one naturally emitted sound produced by a real-life sound source in an environment including at least one artificial sound source. Such an audio recognition device <NUM> includes a first sound recognition engine <NUM>, a second sound recognition engine <NUM>, and a decision taking module <NUM>. As an implementation exemplary the first sound recognition engine <NUM>, the second sound recognition engine <NUM>, and the decision taking module <NUM> may be integrated in a standalone audio recognition device which is equipped of a direct connection to a set-top box itself connected through a HDMI port to a television set, so as to obtain the sound track of the current TV program displayed on the television set.

The first sound recognition engine <NUM> and the second sound recognition engine <NUM> may each implement a machine learning system configured for obtaining and analyzing an audio signal. More particularly, according to an embodiment, the machine learning system of the first sound recognition engine <NUM> and the machine learning system of the second sound recognition engine <NUM> are classifiers trained to classify sounds respectively detected in the first audio signal and in the second audio signal, with respect to sound classes of a set of reference sound classes. Those classifiers may rely on various types of classification algorithms (Naïve Bayes, Nearest Neighbor, Artificial Neural Networks, decision tree, etc.). In one embodiment, only one machine learning system may be used to process both the first audio signal and the second audio signal, In that case, the first and the second audio signals are simultaneously obtained, but they are processed one after the other by a same machine learning classifier. By comparing the results of the classification processes, i.e. the first list of sound classes L1 and the second list of sound classes L2, the decision taking module <NUM> is then able to make the decision of signaling only the naturally emitted sounds to the user (for example, in the form of the third list of sound classes L3).

The audio recognition device <NUM> may be part of an audio recognition system further including at least one microphone, for providing the first audio signal to the first recognition engine, and at least one audio acquisition device associated with the at least one artificial sound source, for providing the second audio signal to the second recognition engine. According to an embodiment, the microphone and/or the audio acquisition device may be embedded in the audio recognition device <NUM>.

<FIG> shows a schematic block diagram illustrating an example of an audio recognition device <NUM> for recognizing at least one naturally emitted sound produced by a real-life sound source in an environment including at least one artificial sound source, according to an embodiment of the present disclosure. In an embodiment, such a device <NUM> may be a standalone device that can be connected to at least one artificial sound source (including sources that don't necessary include loudspeakers, but that are able to produce an audio signal on an audio output jack) such as, for example, a television set, a radio receiver or a set-top box.

The device <NUM> includes a processor <NUM>, a storage unit <NUM>, an input device <NUM>, an output device <NUM>, and an interface unit <NUM> which are connected by a bus <NUM>. Of course, constituent elements of the device <NUM> may be connected by a connection other than a bus connection using the bus <NUM>.

The processor <NUM> controls operations of the audio recognition device <NUM>. The storage unit <NUM> stores at least one program to be executed by the processor <NUM>, and various data, including for example parameters used by computations performed by the processor <NUM>, intermediate data of computations performed by the processor <NUM> such as the lists of sound classes respectively produced by a first and a second sound recognition engines embedded within the device <NUM>, and so on. The processor <NUM> is formed by any known and suitable hardware, or software, or a combination of hardware and software. For example, the processor <NUM> is formed by dedicated hardware such as a processing circuit, or by a programmable processing unit such as a CPU (Central Processing Unit) that executes a program stored in a memory thereof.

The storage unit <NUM> is formed by any suitable storage or means capable of storing the program, data, or the like in a computer-readable manner. Examples of the storage unit <NUM> include non-transitory computer-readable storage media such as semiconductor memory devices, and magnetic, optical, or magneto-optical recording media loaded into a read and write unit. The program causes the processor <NUM> to perform a method for recognizing at least one naturally emitted sound produced by a real-life sound source in an environment comprising at least one artificial sound source as a function of input data according to an embodiment of the present disclosure as described previously. More particularly, the program causes the processor <NUM> to compute, from a first audio signal and a second audio signal, intermediate lists of sound classes to provide to a decision taking module, so that the recognition of at least one naturally-emitted sound may be performed.

The input device <NUM> is formed for example by a microphone.

The output device <NUM> is formed for example by a processing unit configured to take decision as a function of recognized sounds within the first and second audio signal.

The interface unit <NUM> provides an interface between the audio recognition device <NUM> and an external apparatus. The interface unit <NUM> is typically a sound acquisition interface, which may be communicable with the external apparatus via cable or wireless communication. For example, the external apparatus may be a set-top box or a television set.

Although only one processor <NUM> is shown on <FIG>, it must be understood that such a processor may comprise different modules and units embodying the functions carried out by device <NUM> according to embodiments of the present disclosure, such as the unit previously described in relation with <FIG>:.

These modules and units may also be embodied in several processors <NUM> communicating and co-operating with each other.

Claim 1:
A method for recognizing at least one naturally emitted sound produced by a real-life sound source in an environment comprising at least one artificial sound source (ASSr), the method being implemented by an audio recognition device, wherein the method comprises:
- simultaneously obtaining (<NUM>):
- a first audio signal from a first microphone (MIC) located in the environment; and
- a second audio signal from an audio acquisition device (AAD), said audio acquisition device being a second microphone located in the vicinity of the artificial sound source or a device connected to an audio output of the artificial sound source in an acoustical isolated way;
- analyzing (<NUM>) the first audio signal and delivering a first list of sound classes (L1) corresponding to sounds recognized in the first audio signal;
- analyzing (<NUM>) the second audio signal and delivering a second list of sound classes (L2) corresponding to sounds recognized in the second audio signal;
- delivering (<NUM>) a third list of sound classes (L3), comprising only sound classes included in the first list of sound classes (L1) which are not included in the second list of sound classes (L2).