Voice command conversion

A method and a system for voice command conversion, the system may include one or more microphones for sensing a voice command for controlling an acoustically controlled device; one or more speech recognition units for identifying the voice command; an ultrasonic command generator for generating an ultrasonic command that represents the voice command; and one or more speakers for transmitting the ultrasonic command to the acoustically controlled device.

BACKGROUND OF THE INVENTION

In the Internet of Things (IoT) future, many devices are designed to work using voice commands.

These devices (also referred to as voice control device) may be equipped with (a) one or more microphones that detect sound signals, and (b) a processor for applying a speech recognition process on the detected sound signals to detect the voice commands.

The one or more microphones may be attached to the body of a voice control device.

A voice control device (VCD) may operate in a noisy environment. The noise may be generated by the acoustically controlled device (for example—when the acoustically controlled device is a vacuum cleaner, a speaker, or a device that includes a noisy engine) or by other devices or other people.

The speech recognition process may not operate under certain noisy conditions. For example—the speech recognition process may not recognize a voice command when the signal to noise ratio (SNR) is not high enough.

There is a growing need to provide a system and method for allowing to control acoustically controlled device s in a noisy environment.

SUMMARY

There may be provided a system for voice command conversion, the system may include: one or more microphones for sensing a voice command for controlling an acoustically controlled device; one or more speech recognition units for identifying the voice command; an ultrasonic command generator for generating an ultrasonic command that represents the voice command; and one or more speakers for transmitting the ultrasonic command to the acoustically controlled device. The acoustically controlled device is a device that may be controlled by acoustic signals. Acoustic signals may include voice signals and/or ultrasonic signals. The acoustically controlled device may be a voice controlled device that may be controlled only by voice commands.

The system may include one or more communication modules for communicating between the one or more microphones and the one or more speech recognition and for communicating between the ultrasonic command generator and the one or more speakers.

The ultrasonic generator may be configured to generate the ultrasonic command by frequency up-conversion of the voice command.

The ultrasonic generator may be configured to generate the ultrasonic command by searching an ultrasonic command that corresponds to the voice command.

The ultrasonic generator may be configured to generate the ultrasonic command by retrieving from another device the ultrasonic command.

The ultrasonic generator may be configured to generate the ultrasonic command by using a mapping between voice commands and ultrasonic commands.

The system may be configured to evaluate a validity of the voice command and to generate the ultrasonic command only when determining that the voice command may be valid.

The system may be configured to evaluate a validity of the voice command and to generate the ultrasonic command regardless of the validity of the voice command.

The system may be configured to transmit the ultrasonic command only when the acoustically controlled device did not respond to the voice command.

The system may include one or more sensors for tracking a compliance of the acoustically controlled device to the voice command.

The one or more sensors may differ from the one or more microphones.

The one or more sensors may include at least one image sensor.

The one or more sensors may include at least one movement sensor.

The one or more sensors may include at least one microphone from the one or more microphones.

The system may include one or more spectrum analyzers for finding a gap in a spectrum of an actual noise generated by the acoustically controlled device or an expected noise generated by the acoustically controlled device; and wherein the ultrasonic command generator may be configured to select an ultrasonic frequency of ultrasonic command to fall in the gap.

The system may include one or more spectrum analyzers for finding a gap in a spectrum of an actual noise generated by an environment of the system or an expected noise generated by the environment of the system; and wherein the ultrasonic command generator may be configured to select an ultrasonic frequency of ultrasonic command to fall in the gap.

The system may include one or more spectrum analyzers for finding frequency range in a spectrum of noise generated by the acoustically controlled device in which the noise may be below a predefined spectrum; and wherein the ultrasonic command generator may be configured to select an ultrasonic frequency of ultrasonic command to fall in the frequency range.

The system may include one or more spectrum analyzers for finding frequency range in a spectrum of noise generated by an environment of the system in which the noise may be below a predefined spectrum; and wherein the ultrasonic command generator may be configured to select an ultrasonic frequency of ultrasonic command to fall in the frequency range.

The acoustically controlled device may be a speaker and the system may include a music identifier for identifying music that may be played by the speaker and to determine an ultrasonic frequency of the ultrasonic command based on the music played by the speaker to fit in at least one gap or in frequencies of less spectral content of the music played by the speaker.

There may be provided a method for voice command conversion, the method may include: sensing, by one or more microphones, a voice command for controlling an acoustically controlled device; identifying, by one or more speech recognition units, the voice command; generating, by an ultrasonic command generator, an ultrasonic command that represents the voice command; and transmitting, by one or more speakers, the ultrasonic command to the acoustically controlled device.

The generating of the ultrasonic command may include frequency up-converting of the voice command.

The generating of the ultrasonic command may include searching an ultrasonic command that corresponds to the voice command.

The generating of the ultrasonic command may include retrieving the ultrasonic command from another device.

The generating of the ultrasonic command may include using a mapping between voice commands and ultrasonic commands.

The method may include evaluating a validity of the voice command and generating the ultrasonic command only when determining that the voice command may be valid.

The method may include evaluating a validity of the voice command and generating the ultrasonic command regardless of the validity of the voice command.

The method may include transmitting the ultrasonic command only when the acoustically controlled device did not respond to the voice command.

The method may include tracking by one or more sensors a compliance of the acoustically controlled device to the voice command.

The one or more sensors may differ from the one or more microphones.

The one or more sensors may include at least one image sensor.

The one or more sensors may include at least one movement sensor.

The method may include finding by one or more spectrum analyzers a gap in a spectrum of an actual noise generated by the acoustically controlled device or an expected noise generated by the acoustically controlled device; and selecting by the ultrasonic command generator an ultrasonic frequency of ultrasonic command to fall in the gap.

The method may include finding by one or more spectrum analyzers a gap in a spectrum of an actual noise generated by an environment of the system or an expected noise generated by the environment of the system; and selecting by the ultrasonic command generator an ultrasonic frequency of ultrasonic command to fall in the gap.

The method may include finding by one or more spectrum analyzers for a frequency range in a spectrum of noise generated by the acoustically controlled device in which the noise may be below a predefined spectrum; and selecting an ultrasonic frequency of ultrasonic command to fall in the frequency range.

The method may include finding by one or more spectrum analyzers for a frequency range in a spectrum of noise generated by an environment of the system in which the noise may be below a predefined spectrum; and selecting an ultrasonic frequency of ultrasonic command to fall in the frequency range.

The acoustically controlled device may be a speaker and the method may include identifying, by a music identifier, music that may be played by the speaker and determining an ultrasonic frequency of the ultrasonic command based on the music played by the speaker.

DETAILED DESCRIPTION OF THE DRAWINGS

According to an embodiment of the invention there is provided a device, a method and a non-transitory computer readable medium for controlling acoustically controlled device s by converting voice commands issued by a person to ultrasonic commands that still can be sensed by one or more microphones of the acoustically controlled device s.

The ultrasonic frequency may exceed the upper audible limit for human hearing—may exceed 20 Kilohertz.

Although ultrasound devices may operate with frequencies from 20 kHz to several gigahertz—it should be noted that the ultrasonic command should still be detected by the one or more microphones of the acoustically controlled device s. Accordingly—the frequency response of the one or more microphones of the acoustically controlled device s should be taken into account—and may impose an upper limit on the ultrasonic frequency of the ultrasonic command. See, for example,FIG. 13.

According to an embodiment of the invention there is provided a system that may be configured to (a) detect a voice command issued by a person to a acoustically controlled device, (b) generate an ultrasonic command that represents the voice command, and (c) transmit the ultrasonic command to the acoustically controlled device.

The system may include (a) one or more microphones for sensing the voice command, (b) one or more speech recognition units for identifying the voice command, (c) an ultrasonic command generator for generating the ultrasonic command (d) one or more speakers for transmitting the ultrasonic commands, and (e) one or more communication modules for allowing a communication between different components of the system and/or between the system and other devices. A communication module may include a transmitter and/or a receiver and may apply any known communication protocols.

The ultrasonic command represents the voice command in the sense that both represent the same command.

The ultrasonic generator may perform at least one of the following:a. Up-convert the voice command to the ultrasonic command by applying frequency conversion.b. Fetch from a memory bank of the system (or another device), the corresponding ultrasonic command.c. Retrieve from another device the corresponding ultrasonic command. This may include sending information about the voice command to the other device and retrieve information about the ultrasonic command from the other device. This may include communicating (by the system, directly or indirectly) with the other device via any type of wired or wireless communication.

The other device may be a server or other computerized device.

The system may be configured to detect the voice command, check (by itself or by communicating with the other device) that the voice command is a valid voice command (for example—belongs to a given set of voice commands associated with the acoustically controlled device) and if so—generate the ultrasonic command and send the ultrasonic command to the acoustically controlled device. Alternatively—the system may not perform a validity check and/or may convert each voice command to an ultrasonic voice command regardless of its validity.

The system may be configured to generate and/or transmit the ultrasonic command only when the device did not respond to the voice command issued by the person.

The system may determine that the device did not respond to the voice command by detecting that the person repeated the command several times within a short period.

The system may determine that the device did not respond to the voice command by detecting a lack of response from the acoustically controlled device.

A lack of response (or the wrong response) may be detected, for example, when the voice command requests the acoustically controlled device to shut down or otherwise change the noise generated by the device—but the noise generated by the acoustically controlled device does not change.

A lack of response (or the wrong response) may be detected, for example, when the voice command requests the acoustically controlled device to power up—but the acoustically controlled device remains shut down.

A lack of response (or the wrong response) may be detected, for example, when the voice command requests the acoustically controlled device to move (or to stop moving)—and the acoustically controlled device does not move (or does not stop moving). The movement or lack of movement may be detected by a change or a lack of change of the noise generated by the device, may be detected using image sensors or other movement sensors, and the like.

The acoustically controlled device may include a speech recognition circuit that may detect both the voice command and the ultrasonic command.

Additionally or alternatively, the acoustically controlled device may include a converter for converting the ultrasonic command to a voice command that can be processed by the speech recognition circuit.

The different components of the system may be included in one or more intermediate devices that may be used to detect a voice command, generate an ultrasonic command that represents the voice command, and transmit the ultrasonic command to the acoustically controlled device.

An intermediate device may include all or some of the components of the system. For example, an intermediate device may include (a) one or more microphones for sensing the voice command, (b) one or more speech recognition units for identifying the voice command, (c) an ultrasonic command generator for generating the ultrasonic command and (d) one or more speakers for transmitting the ultrasonic commands.

The intermediate device may be included in one of more housing such as a box, or any other shaped housing. The housing may have a substantially cylindrical shaped housing. The intermediate device may be electrically coupled to a power supply plug, may be powered by a battery, or may receive power in any contact or contactless manner.

Alternatively, one or more of the component of the system may be spaced apart from other component of the system.

The system may include one or more microphones (for sensing the voice command) that are spaced apart from the one or more speakers (for transmitting the ultrasonic commands). The one or more speech recognition circuits (for identifying the voice command) may be electrically coupled (wirelessly or not) to the one or more microphones. The ultrasonic command generator (for generating the ultrasonic command) may be electrically coupled (wirelessly or not) to the one or more speakers.

The different components of the system may communicate with each other using any known communication protocol (such as Bluetooth, Wi-Fi, DECT).

The acoustically controlled device may be operated in a certain environment (within a building, within one or more rooms of the building) and the system may include speakers and microphones that are spread along the certain environment.

The system may include one or more sensors for tracking the compliance of the acoustically controlled device to the voice commands issued by the person. The one or more sensors may be the one or more microphones and/or other sensors such as image sensors, movement sensors and the like.

According to an embodiment of the invention the ultrasonic frequency may be selected based on the actual or expected noise generated by the acoustically controlled device.

For example, the noise generated by a acoustically controlled device may be analyzed (by performing spectral analysis) to determine the gaps in the noise spectrum—and/or frequency ranges in which the noise generated by the vacuum cleaner are below a certain threshold. The ultrasonic frequency may be selected to be included one or more in the gaps and/or frequency ranges in which the noise generated by the vacuum cleaner are below a certain threshold. The ultrasonic command can be simultaneously transmitted over multiple gaps and/or multiple such frequency ranges.

According to an embodiment of the invention the acoustically controlled device may be a speaker and the system may include a music identifier for identifying the music that is played by the speaker and to determine the ultrasonic frequency of the ultrasonic command based on the played music—to fit in gaps or frequencies of less spectral content of that music.

FIG. 1illustrates a method10according to an embodiment of the invention. Steps12,14,15,17,16and18may be executed by the system while steps20,22,24and26may be executed by the acoustically controlled device.

Method10may start by step12of sensing (for example by one or more microphones) a voice command issued (for example by a person) to an acoustically controlled device. The voice command is aimed to control the acoustically controlled device.

Step12may be followed by step14of identifying (for example by one or more speech recognition units) the voice command.

Step14may be followed by either one of steps15,16and17. Out of steps15,16and17—method10may include step16and zero or more steps out of steps15and17.

Step16may include generating (for example by an ultrasonic command generator) an ultrasonic command that represents the voice command.

The ultrasonic command is transmitted to the acoustically controlled device directly or indirectly (via a relay or other communication unit).

Step16may include at least one of the following:a. Up-converting of the voice command.b. Searching an ultrasonic command that corresponds to the voice command.c. Retrieving the ultrasonic command from another device.d. Using a mapping between voice commands and ultrasonic commands.e. Using a mapping between voice commands and frequency information and ultrasonic commands.f. Finding by one or more spectrum analyzers a gap in a spectrum of an actual noise generated by the acoustically controlled device or an expected noise generated by the acoustically controlled device; and selecting an ultrasonic frequency of ultrasonic command to fall in the gap.g. Finding by one or more spectrum analyzers a gap in a spectrum of an actual noise generated by the environment of the system or an expected noise generated by the environment of the system; and selecting an ultrasonic frequency of ultrasonic command to fall in the gap.h. Finding by one or more spectrum analyzers for a frequency range in a spectrum of noise (actual or estimated) generated by the acoustically controlled device in which the noise is below a predefined spectrum; and selecting an ultrasonic frequency of ultrasonic command to fall in the frequency range.i. Finding by one or more spectrum analyzers for a frequency range in a spectrum of noise (actual or estimated) generated by the environment of the system in which the noise is below a predefined spectrum; and selecting an ultrasonic frequency of ultrasonic command to fall in the frequency range.j. Identifying, by a music identifier, music that is played by the speaker and determining an ultrasonic frequency of the ultrasonic command based on the music played by the speaker.

Step14may be followed by step15of evaluating a validity of the voice command. Step15may include checking if the voice command belongs to a set of voice commands that are associated with the acoustically controlled device. The set of voice commands may be fed to other device or to a system in advance—or may be retrieved or learnt in any other manner. The validity check may check whether the voice command is corrupted or not.

Step15may be followed by step16only if the voice command is valid—and thus method10may include generating the ultrasonic command only when determining that the voice command is valid. Alternatively—step15may be followed by step16regardless of the validity of the voice command.

Step14may be followed by step17of checking of the voice command was executed by the acoustically controlled device. This may require the user to issue the voice command several times without having the acoustically controlled device execute the voice command and/or waiting for a predefined delay from the issuing of the voice command without having the acoustically controlled device execute the voice command.

The compliance of the acoustically controlled device may be sensed by one or more sensors such as microphones, motion sensors, image sensors and the like.

Step17may be followed by step16—and method10may include transmitting the ultrasonic command only when the acoustically controlled device did not execute the voice command—or regardless of the compliance of the acoustically controlled device.

Step16may be followed by step18of transmitting (for example by one or more speakers) the ultrasonic command to the acoustically controlled device.

FIG. 2illustrates the other device30, system40(including components such as one or microphones42, one or more speech recognition units44, one or more ultrasonic command generators45, one or more speakers46and one or more communication modules48) and a acoustically controlled device (including components such as microphone52, converter54, speech recognition unit56and controlled components58).

FIG. 3illustrates a certain environment100that includes rooms101,102,103,104,104and corridor106. Microphones42and speakers46are distributed in the certain environment and may communicate with ultrasonic command generator45and speech recognition units45. For simplicity of explanation only the communication module48that communicated with another device30via network31is shown.

FIG. 4illustrates the certain environment100as including other sensors (sensors that differ from the one or more microphones42)—and are configured to track the compliance of the acoustically controlled device to the voice commands issued by the person. These other sensors47may include any combination of image sensors and/or movement sensors—such as image sensors, movement sensors or any other sensors. Any number of other sensors and/or any distribution of the other sensors within the environment may be provided.

FIG. 5illustrates the certain environment100as further including a spectrum analyzer49for analyzing the spectrum of noise (actual and/or expected) generated by an acoustically controlled device. The analysis includes (or is followed by) determining the gaps in the noise spectrum—and/or frequency ranges in which the noise generated by the vacuum cleaner are below a certain threshold. The ultrasound command spectrum components in these frequency ranges shall exceed the certain threshold by at least a predefined SNR—for example by at least 3 db.

FIG. 6illustrates the certain environment100as further including music identifiers41for identifying music that is played by an acoustically controlled device50that is a speaker or other music playing device. The music identifiers determine the frequency of the ultrasonic command based on the played music—to fit in gaps or frequencies of less spectral content of that music.

Processor43may perform different determination such as determine if the voice command is valid, determine whether to generate an ultrasonic command, determine whether to transmit an ultrasonic command and the like. The processor43may be included in components such as speech recognition units44, one or more ultrasonic command generators and the like. Alternatively—components such as speech recognition units44, one or more ultrasonic command generators45may be implemented by one or more processors.

A processor is a hardware device that may include one or more integrated circuits and may be programmed to execute one or more tasks such as identifying a voice command, generate an ultrasonic command and the like.

FIG. 10illustrates a mapping101between voice commands and ultrasonic commands. Each voice command may be mapped to a single ultrasonic command, but this is not necessarily so.

FIG. 11illustrates a mapping102between musical works (identified by music identifier) and spectral information. The spectral information may identity one or more gaps in the spectrum of the musical work and/or frequencies of less spectral content of the musical work. The frequency information may also include information about the desired minimal intensity of the ultrasonic command to be transmitted in the one or more gaps and/or frequencies of less spectral content.

FIG. 12illustrates a mapping between (a) commands and frequency information and (b) ultrasonic commands. The frequency information may include the spectrum of noise or any other sound generated by acoustically controlled device and/or information about the spectral response of one or more microphones of the voice controlled device. The corresponding ultrasonic command does not only represent the voice command but may be at the proper frequency and/or intensity/The proper frequency and intensity may be within the frequency response (at least strong enough to be speech recognized by the voice controlled device) of the voice control device and/or within a gap or range of less content of the noise or sound generated by the acoustically controlled device.

Any of the mapping may be stored in a look up table or in any other manner. The mapping may be represented by a mathematical formula.

The mapping may be learnt during a learning period or in any other manner. The frequency information, voice commands and ultrasonic commands may be fed to the other device50and/or to system40.

FIG. 13is a non-limiting example of a spectrum (111and112) of sound generated by the voice control device. Curves111and112illustrates the spectrum of sound. A gap113is defined between curves111and112.FIG. 13also illustrates a predefined threshold115that may define the frequencies114of less spectral content114.

FIG. 13also illustrates the frequency response102of a microphone of the voice controller device and a cutoff frequency121.

Also for example, in one embodiment, the illustrated examples may be implemented as circuitry located on a single integrated circuit or within a same device. Alternatively, the examples may be implemented as any number of separate integrated circuits or separate devices interconnected with each other in a suitable manner. The integrated circuit may be a system on chip, a general-purpose processor, a signal processor, an FPGA, a neural network integrated circuit, and the like.