Patent Description:
Most of the electronic devices use biometric security measures i.e. biometric authentication, that grant access to any of using or performing any operations in the electronic devices. One of the biometric authentications is voice authentication that is widely applied in modern voice assistants, speech-controlled systems, and the like. The voice authentication is configured to match vocal utterances with a biometric profile to grant access. Many voice authentication systems accessing services may be limited per person as those voice authentication systems equip a voice authentication/verification subsystem, AVS. Typical voice verification system couples with an activation phrase detector and uses wake-up phrase to authenticate the user as known in Alexa. For example, the wake-up phrase is a matching voice pattern with pre-recorded template. These types of voice authentication systems may prone to spoofing attacks, such as voice replay, and voice synthesis.

Currently, there are several known voice anti-spoofing techniques are available, including detection of record/replay audio artifacts, and detection of a voice background continuity fracture. Existing voice anti-spoofing techniques in the voice authentication system are highly depending on environmental parameters where noise is a dominating one. High noise levels for automotive, industrial and open space applications provide anti-spoofing engines to edge and attackers may utilize noise injection for constructing successive attacks on the voice authentication systems. Further, those voice authentication systems include several complementary techniques to establish a feasible protection level. Low resilience to the noise is one of the key issues in the existing voice anti-spoofing systems. Spoof detection errors may grow with high noise levels, that the voice authentication system security and user experience may become unacceptable.

Therefore, the present invention aims to improve the performance of existing systems or technologies in voice spoofing detection.

<NPL>, Hyderabad, India discloses the extraction of spoofing identity vectors, as well as the use of soft missing-data masks to counter spoofing attacks. The proposed feature extraction employs a convolutional neural network (CNN) plus a recurrent neural network (RNN) in order to provide a single deep feature representing a spoofing identity per utterance.

It is an object of the disclosure to provide a method, and an apparatus for voice spoofing detection in noisy environment with improved protection and accuracy while avoiding one or more disadvantages of prior art approaches.

This object is achieved by the features of the independent claims. Further implementations are apparent from the dependent claims, the description, and the figures. The disclosure provides a method of voice spoofing detection in noisy environment. The disclosure also provides an apparatus for voice spoofing detection in the noisy environment.

According to a first aspect, there is provided a method of voice spoofing detection in noisy environment. The method includes receiving a voice input of a user. The method includes receiving an environmental noise measure corresponding to the voice input of the user. The method includes authenticating the user based on the voice input. The method includes determining noise parameters based on the environmental noise measure. The method includes determining actual values of Lombard effect related features of the user voice based on the voice input of the user. The method includes determining a voice spoofing probability by comparing the actual values of the Lombard effect related features with a Lombard effect related feature template of the authenticated user corresponding to the determined noise parameters.

This method provides improved protection against voice spoofing attacks in noisy environments. The method protects against replay, mimicking, and voice synthesis attacks. The method may include additional independent authentication factors to authenticate the user thereby securing systems. The method is relatively lightweight. The method provides improved accuracy in the voice input of the user with the voice spoofing probability even in the very noisy environments. The method introduces a complementary technique of speech spoofing detection utilizing noise for good, which brings significant improvement of the voice anti-spoofing in the noisy environments.

Optionally, the Lombard effect related features include changes between characteristics of the user voice in time, spectrum and power-spectrum domains under normal conditions and in the noisy environment, including changes in: phonetic fundamental frequencies, an energy distribution among low, middle and high frequency bands, a sound intensity, a vowel duration, spectral tilting, formal center frequencies, a key words duration and a lung volume.

Optionally, the noise parameters include a noise type and a noise level.

Optionally, the Lombard effect related feature template includes pre-determined values of the Lombard effect related features of the user voice under different noise parameters.

Optionally, the pre-determined values of the Lombard effect related features include averaged values of the Lombard effect related features collected for the user voice under different noise parameters during a training period.

Optionally, the voice spoofing probability is determined as a function of a different between the actual values of the Lombard effect related features and the pre-determined values of the Lombard effect related features from the Lombard effect related feature template of the authenticated user corresponding to the determined noise parameters.

Optionally, the Lombard effect related feature template further includes weights of the Lombard effect related features and the voice spoofing probability is determined as a function of the weighted difference between the actual values of the Lombard effect related features and the pre-determined values of the Lombard effect related features, the weighted different being obtained with the weights.

Optionally, the method further includes validating the authentication of the user based on the voice spoofing probability.

Optionally, the method further includes updating the Lombard effect related feature template of the user with the actual values of the Lombard effect related features in reply to the validation of the authentication of the user.

According to a second aspect, there is provided an apparatus for voice spoofing detection in noisy environment. The apparatus includes an input module configured for receiving a voice input of a user and an environmental noise measure corresponding to the voice input of the user, a speech verification module configured for authenticating the user based on the voice input, a noise retriever configured for determining noise parameters based on the environmental noise measure, a Lombard feature retriever configured for determining actual values of Lombard effect related features based on the voice input of the user, and a spoofing detector configured for determining a voice spoofing probability by comparing the actual values of the Lombard effect related features with a Lombard effect related feature template of the authenticated used corresponding to the determined noise parameters.

The apparatus provides improved protection against voice spoofing attacks in noisy environments. The apparatus protects against replay, mimicking and voice synthesis attacks. The apparatus may include additional independent authentication factors to authenticate the user thereby securing systems. The apparatus is relatively lightweight. The apparatus provides improved accuracy in the voice input of the user with the voice spoofing probability even in the very noisy environments. The apparatus introduces complementary technique of speech spoofing detection utilizing noise for good, which brings significant improvement of the voice anti-spoofing in the noisy environments.

Optionally, the Lombard feature retriever is configured for determining actual values of the Lombard effect related features by estimating changes between characteristics of the user voice in time, spectrum and power-spectrum domains under normal conditions and in a noisy environment, including changes in: phonetic fundamental frequencies, an energy distribution among low, middle and high frequency bands, a sound intensity, a vowel duration, spectral tilting, format center frequencies, a key words duration and a lung volume.

Optionally, the noise retriever is configured for determining a noise type and a noise level.

Optionally, the spoofing detector is configured for determining the voice spoofing probability as a function of a different between the actual values of the Lombard effect related features and the pre-determined values of the Lombard effect related features from the Lombard effect related feature template of the authenticated user corresponding to the determined noise parameters.

Optionally, the Lombard effect related feature template further includes weights of the Lombard effect related features and the spoofing detector is configured for determining the voice spoofing probability as a function of the weighted difference between the actual values of the Lombard effect related features and the pre-determined values of the Lombard effect related features, the weighted different being obtained with the weights.

Optionally, the apparatus further includes a decision module configured for validating the authentication of the user based on the voice spoofing probability.

Optionally, the apparatus further includes a data collector configured for collecting, for each user, characteristics of the user voice under normal conditions, noise parameters, actual values of the Lombard effect related features of the user voice under the noise parameters, and the Lombard feature effect related template.

Optionally, the apparatus further includes a learning engine configured for updating the Lombard effect related feature template of the user with the actual values of the Lombard effect related features in reply to the validation of the authentication of the user by the decision module and/or a learning trigger.

Therefore, in contradistinction to the prior art, according to the method, and the apparatus, improves the protection against voice spoofing attacks in the noisy environments. The method, and the apparatus improve the accuracy in the voice input of the user with the voice spoofing probability even in the very noisy environments.

These and other aspects of the disclosure will be apparent from the implementations described below.

Implementations of the disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which:.

Implementations of the disclosure provide a method of voice spoofing detection in noisy environment with improved protection and accuracy, and an apparatus for voice spoofing detection in the noisy environment with improved protection and accuracy.

To make solutions of the disclosure more comprehensible for a person skilled in the art, the following implementations of the disclosure are described with reference to the accompanying drawings.

Terms such as "a first", "a second", "a third", and "a fourth" (if any) in the summary, claims, and foregoing accompanying drawings of the disclosure are used to distinguish between similar objects and are not necessarily used to describe a specific sequence or order. It should be understood that the terms so used are interchangeable under appropriate circumstances, so that the implementations of the disclosure described herein are, for example, capable of being implemented in sequences other than the sequences illustrated or described herein. Furthermore, the terms "include" and "have" and any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, a method, a system, a product, or a device that includes a series of steps or units, is not necessarily limited to expressly listed steps or units but may include other steps or units that are not expressly listed or that are inherent to such process, method, product, or device.

<FIG> illustrates a block diagram of an apparatus <NUM> for voice spoofing detection in noisy environment in accordance with an implementation of the disclosure. The apparatus <NUM> includes an input module <NUM> configured for receiving a voice input of a user and an environmental noise measure corresponding to the voice input of the user, a speech verification module <NUM> configured for authenticating the user based on the voice input, a noise retriever <NUM> configured for determining noise parameters based on the environmental noise measure, a Lombard feature retriever <NUM> configured for determining actual values of Lombard effect related features based on the voice input of the user, and a spoofing detector <NUM> configured for determining a voice spoofing probability by comparing the actual values of the Lombard effect related features with a Lombard effect related feature template of the authenticated user corresponding to the determined noise parameters.

The apparatus <NUM> provides improved protection against voice spoofing attacks in noisy environments. The apparatus <NUM> protects against replay, mimicking and voice synthesis attacks. The apparatus <NUM> may include additional independent authentication factors to authenticate the user thereby securing systems. The apparatus <NUM> is relatively lightweight. The apparatus <NUM> provides improved accuracy in the voice input of the user with the voice spoofing probability even in the very noisy environments. The apparatus <NUM> introduces a complementary technique of speech spoofing detection utilizing noise for good, which brings significant improvement of the voice anti-spoofing in the noisy environments.

Lombard effect or Lombard reflex is an involuntary tendency of speakers that increases their vocal effort when speaking in noise to enhance the audibility of their voice. The Lombard feature retriever <NUM> is configured to determine the actual values of the Lombard effect related features with the Lombard effect. The Lombard effect includes loudness and also other acoustic features such as pitch, rate, duration of syllables, and the like. The Lombard effect may maintain an auditory signal to noise ratio, SNR of the speaker's spoken words.

The apparatus <NUM> may change the original voice input of the user into a Lombard adjusted speech. Optionally, the Lombard adjusted speech includes any of increase in phonetic fundamental frequencies, shift in energy from low frequency bands to middle or high bands, increase in sound intensity, increase in vowel duration, spectral tilting, shift in format center frequencies, a duration of content words prolonging to a greater degree, or using greater lung volumes. Optionally, the Lombard adjusted speech varies per noise type and level.

Optionally, the Lombard feature retriever <NUM> is configured for determining actual values of the Lombard effect related features by estimating changes between characteristics of the user voice in time, spectrum and power-spectrum domains under normal conditions and in a noisy environment, including changes in: phonetic fundamental frequencies, an energy distribution among low, middle and high frequency bands, a sound intensity, a vowel duration, spectral tilting, format center frequencies, a key words duration and a lung volume.

Optionally, the noise retriever <NUM> is configured for determining a noise type and a noise level. The noise retriever <NUM> may determine noise type and level aware Lombard features per user.

Optionally, the spoofing detector <NUM> is configured for determining the voice spoofing probability as a function of a different between the actual values of the Lombard effect related features and the pre-determined values of the Lombard effect related features from the Lombard effect related feature template of the authenticated user corresponding to the determined noise parameters.

Optionally, the Lombard effect related feature template further includes weights of the Lombard effect related features and the spoofing detector <NUM> is configured for determining the voice spoofing probability as a function of the weighted difference between the actual values of the Lombard effect related features and the pre-determined values of the Lombard effect related features, the weighted different being obtained with the weights.

<FIG> is a block diagram of an apparatus <NUM> for voice spoofing detection in noisy environment in accordance with an implementation of the disclosure. The apparatus <NUM> includes an input module <NUM>, a voice processing module <NUM>, a noise classification and measurement module <NUM>, a voice authentication module <NUM>, and a voice anomaly detecting module <NUM>. Optionally, the apparatus <NUM> further includes a decision module <NUM> configured for validating the authentication of the user based on the voice spoofing probability. The apparatus <NUM> may be placed in a vehicle. Optionally, the vehicle is a car. The input module <NUM> is configured to monitor any of voice of a user or an environmental noise. The voice of the user may be at least one of talk or commands. The environmental noise may be noises of any of tires, engine, air conditioner, road, cabin music, and the like. Optionally, the input module <NUM> is a microphone configured to monitor any of the voice of the user or the environmental noise. Optionally, the input module <NUM> obtains a voice input of the user.

The voice processing module <NUM> is configured to process any of the voice input of the user or the environmental noise, with a regular featurization method and a Lombard featurization method. The regular featurization method may process the voice input of the user to convert the voice input into algorithms. Optionally, the algorithms are used to train a learning engine and/or enable the learning engine to perform one or more operations. The Lombard featurization method may process the voice input of the user to enhance the voice input. The noise classification and measurement module <NUM> is configured to determine a noise type (Perceptual Linear Prediction, PLP) and a noise level on any of the voice input of the user or the environmental noise. The Lombard featurization method in the voice processing module <NUM> determines Lombard voice parameters based on the noise type and the noise level in any of the voice input of the user or the environmental noise. The determined Lombard voice parameters may be compared with Actual Lombard Parameters, ALP retrieved from the voice input of the user. Optionally, determination of Lombard voice changes of the user can be learned during the training of the learning engine.

The voice authentication module <NUM> is configured to authenticate the user based on the voice input of the user. Optionally, the learning engine enables the one or more operations based on the voice input of the user only when the voice authentication module <NUM> authenticates the user. The voice authentication module <NUM> may include a regular template including one or more controls to perform the one or more operations based on the voice input of the user. Optionally, the regular template includes one or more models along with templates and configurations for one or more users. The learning engine may create the one or more models for the one or more users. The voice anomaly detection module <NUM> is configured to determine a spoofing probability in the voice input of the user. The spoofing probability may be calculated as a function of difference between the Actual Lombard Parameters (ALP) and the determined Lombard voice parameters. Optionally, the spoofing probability is calculated with a function of difference between the ALP and the measured parameters based on the noise type PLP, i.e. P = f(/PLP = - ALP/).

The decision module <NUM> is configured to validate the authentication of the user based on the voice spoofing probability. The decision module <NUM> at least one of approve or decline the authentication of the user based on the voice input of the user. Optionally, the learning engine enables the one or more operations based on the voice input of the user when the decision module <NUM> approves the authentication of the user. Optionally, the learning engine enables an output module to alert the user when the decision module <NUM> declines the authentication of the user.

<FIG> illustrates a graphical representation <NUM> of a spoofing probability in accordance with an implementation of the disclosure. The graphical representation <NUM> shows a dependence of the spoofing probability on a distance, a noise type and a noise level: <MAT>.

Optionally, one or more features of time, spectrum, and power spectrum domains related to Lombard effect retrieved from a voice input of a user. Optionally, the one or more features include phonetic fundamental frequencies (F0, F1, F2), energy from low frequency bands to middle or high bands (EL, EM, EH), sound intensity (M), vowel duration (VD), spectral tilting (C), format center frequencies (FFT x nBins), a duration of key words (D1, D2. Dn - key phrase dependent), and the like. Optionally, average estimations of the one or more features are created for the user talking in appropriate conditions during a pre-defined period or per pre-defined number of most recent sessions. The appropriate conditions may be defined by a noise type and a Signal to Noise Ratio, SNR level. Noises that may be addressed include mechanical noises including noises of engine, air conditioner, vibrations, and the like, road noises including noises of tiers, wind, road specific vibrations, and the like, music noises, chatter noises, and nature noises including wind, rain, forest and the like. The distance may be calculated with a measured (FMi) and estimated sampled average (FEi). The distance may be calculated with a formula as follows: <MAT>.

The calculated distance between the measured (FMi) and the estimated sampled average (FEi) may be used for estimation of the spoofing probability. Optionally, advanced weighted distance (Wi) or alternative techniques can be applied for accuracy improvements. The alternative technique may be a machine learning technique.

<FIG> illustrates a system architecture <NUM> of an apparatus for voice spoofing detection in noisy environment in accordance with an implementation of the disclosure. The system architecture <NUM> includes a spoofing classification subsystem <NUM>, a noise features retriever <NUM> and a training and adjustment subsystem <NUM>. The spoofing classification subsystem <NUM> includes a Lombard features retriever <NUM>, and an anomaly detector <NUM>. The Lombard features retriever <NUM> receives a voice input of a user and an environment noise. The voice input of the user and the environment noise may be an input audio stream. Optionally, the Lombard features retriever <NUM> receives the input audio stream with an input module. The Lombard features retriever <NUM> is configured to retrieve voice features from the input audio stream. The voice features may be any of a talk or commands of the user.

The noise features retriever <NUM> is configured to infer noise type in the input audio stream. The noise features retriever <NUM> may calculate noise parameters based on the noise type of the input audio stream. Optionally, the noise parameter includes Signal to Noise Ratio, SNR. The anomaly detector <NUM> is configured to compare retrieved feature vector with expected values for the user in a noise context to determine a spoofing probability. Optionally, the retrieved feature vector is the retrieved voice features from the input audio stream. The anomaly detector <NUM> performs comparison by using features template. Optionally, the features template is stored in Lombard templates database. The Lombard templates database may store the Lombard features collected and averaged during training of a learning engine.

Optionally, the apparatus further includes a data collector <NUM> configured for collecting, for each user, characteristics of the user voice under normal conditions, noise parameters, actual values of the Lombard effect related features of the user voice under the noise parameters, and the Lombard feature effect related template. Optionally, the apparatus further includes a learning engine <NUM> configured for updating the Lombard effect related feature template of the user with the actual values of the Lombard effect related features in reply to the validation of the authentication of the user by the decision module and/or a learning trigger.

The training and adjustment subsystem <NUM> in the system architecture <NUM> includes the data collector <NUM>, and the learning engine <NUM>. Optionally, the learning engine <NUM> is a user learning engine. The training and adjustment subsystem <NUM> may receive new blocks with new Application Program Interfaces, APIs. The data collector <NUM> is configured to create feature vectors of speech fragments and manage a collection of user's measurements from the noise type, and the noise parameters of the input audio stream. Optionally, the creation of the feature vectors enables addressing trends of the user's vocal characteristics. The data collector <NUM> may store the feature vectors as recent data in a database. The learning engine <NUM> is configured to refresh the Lombard features stored in the Lombard templates database used for comparisons of the anomaly detector <NUM>. Optionally, the learning engine <NUM> may run periodically or step in on Key Performance Indicators, KPIs degradation, and the like.

Optionally, the system architecture <NUM> includes a speech verification system that is configured to authenticate the user based on the voice input of the user, and the features of the user's voice. The speech verification system may determine a user ID based on the voice input of the user. Optionally, the system architecture <NUM> includes a spoofing classification where the features of the user's voice can be compared with the expected Lombard features. Optionally, the system architecture <NUM> includes an additional system that runs background learning and updates Lombard templates database that is trending with time.

<FIG> and <FIG> are flow diagrams that illustrate a method of voice spoofing detection in noisy environment in accordance with an implementation of the disclosure. At a step <NUM>, a voice input of a user is received. At a step <NUM>, an environmental noise measure corresponding to the voice input of the user is received. At a step <NUM>, the user is authenticated based on the voice input. At a step <NUM>, noise parameters are determined based on the environmental noise measure. At a step <NUM>, actual values of Lombard effect related features of the user voice are determined based on the voice input of the user. At a step <NUM>, a voice spoofing probability is determined by comparing the actual values of the Lombard effect related features with a Lombard effect related feature template of the authenticated user corresponding to the determined noise parameters.

This method provides improved protection against voice spoofing attacks in noisy environments. The method protects against replay, mimicking and voice synthesis attacks. The method may include additional independent authentication factors to authenticate the user thereby securing systems. The method is relatively lightweight. The method provides improved accuracy in the voice input of the user with the voice spoofing probability even in the very noisy environments. The method introduces a complementary technique of speech spoofing detection utilizing noise for good, which brings significant improvement of the voice anti-spoofing in the noisy environments.

It should be understood that the arrangement of components illustrated in the figures described are exemplary and that other arrangement may be possible. It should also be understood that the various system components (and means) defined by the claims, described below, and illustrated in the various block diagrams represent components in some systems configured according to the subject matter disclosed herein. For example, one or more of these system components (and means) may be realized, in whole or in part, by at least some of the components illustrated in the arrangements illustrated in the described figures.

In addition, while at least one of these components are implemented at least partially as an electronic hardware component, and therefore constitutes a machine, the other components may be implemented in software that when included in an execution environment constitutes a machine, hardware, or a combination of software and hardware.

Claim 1:
A method of voice spoofing detection in noisy environment, the method comprising:
receiving (<NUM>) a voice input of a user,
receiving (<NUM>) an environmental noise measure corresponding to the voice input of the user,
authenticating (<NUM>) the user based on the voice input,
determining (<NUM>) noise parameters based on the environmental noise measure,
determining (<NUM>) actual values of Lombard effect related features of the user voice based on the voice input of the user, and
determining (<NUM>) a voice spoofing probability by comparing the actual values of the Lombard effect related features with a Lombard effect related feature template of the authenticated user corresponding to the determined noise parameters.