SYSTEMS AND METHODS OF VOICEPRINT AUTHENTICATION AND INTERPOLATION

Embodiments are direct to methods and systems for authenticating a user and interpolating user preference embeddings. The systems generate, using a neural network trained to generate features based on training data comprising human voices spoken by a plurality of historical speakers inside a vehicle, input features based on a human voice of a current speaker inside the vehicle, and calculates similarities between an input vector of the input features and historical vectors in voiceprints of one or more enrolled users. After determining a similarity between the input vector and at least one historical vector in a voiceprint of an identified user is less than a threshold similarity, the systems authenticate the current speaker as the identified user, calculate a probabilistic notion based on the similarity, and apply the probabilistic notion to interpolate between downstream user preference embeddings associated with the identified user.

TECHNICAL FIELD

The embodiments described herein generally relate to applications of a voice user interface, and more particularly, to vehicle voiceprint recognition and authentication.

BACKGROUND

Modern vehicles use services that need authentication, such as remote start, locking/unlocking vehicles, valeting, and geofencing. Authentication is an important function in security to identify an entity for services the vehicle may provide. Once authenticated, vehicles may authorize a user to access specific services or data. Accordingly, a need exists for systems of accurate and efficient authentication to access a vehicle system.

SUMMARY

In one embodiment, a method may include generating, using a neural network trained to generate features based on training data comprising human voices spoken by a plurality of historical speakers inside a vehicle, input features based on a human voice of a current speaker inside the vehicle, and calculating similarities between an input vector of the input features and historical vectors in voiceprints of one or more enrolled users. After determining a similarity between the input vector and at least one historical vector in a voiceprint of an identified user is less than a threshold similarity, the method includes authenticating the current speaker as the identified user, calculating a probabilistic notion based on the similarity, and applying the probabilistic notion to interpolate between downstream user preference embeddings associated with the identified user.

In another embodiment, a system includes a controller to generate, using a neural network trained to generate features based on training data comprising human voices spoken by a plurality of historical speakers inside a vehicle, input features based on a human voice of a current speaker inside the vehicle, and calculate similarities between an input vector of the input features and historical vectors in voiceprints of one or more enrolled users. After determining a similarity between the input vector and at least one historical vector in a voiceprint of an identified user is less than a threshold similarity, the controller may authenticate the current speaker as the identified user, calculate a probabilistic notion based on the similarity, and apply the probabilistic notion to interpolate between downstream user preference embeddings associated with the identified user.

DETAILED DESCRIPTION

The embodiments disclosed herein are directed to methods and systems for voiceprint authentication and interpolation. A system creates a voiceprint for a user and allows for authentication of the user from the user's voice. Modern vehicles provide functions and services where authentication of enrolled users is performed before a user may use these functions and services. However, methods of authentication, such as passwords or PIN codes, are easily compromised, and sometimes inconvenient or impractical to use. Voiceprint authentication overcomes these problems by providing a more secure and convenient way to authenticate the user of a vehicle. Voiceprint authentication is based on the unique characteristics of a person's voice that are difficult to forge, making voiceprint authentication a secure method of authentication. Voiceprint authentication is convenient as the users need only use their voice and speak into the vehicle's microphone, with no need to remember any passwords or PIN codes. Further, voiceprint authentication is flexible for different purposes, such as preventing unauthorized use of vehicles, tracking vehicle usage, and monitoring multiple fleet vehicles.

The voiceprint authentication disclosed herein may use an active voice biometric identification or a passive voice biometric identification. An active approach entails an explicit voice registration process which the user opts into, and is guided through. An active voice biometrics approach requires the user to recite a predetermined script multiple times, explicitly establishing their unique voiceprint. Each time the user uses the active authentication, the user has to say a passphrase. The active authentication system compares the user's voice with the recorded script. The user is fully aware of the authentication process.

A passive approach infers anonymized user IDs based on speech occurring within the vehicle. In passive voice authentication, no specific passphrase needs to be said. A user may engage in a regular conversation to trigger the passive authentication and the passive authentication may continuously re-identify and re-authenticate the user. A voiceprint of a user may be recorded in the system, including the initial enrollment recording and/or continuous recordings after enrollment. When a user speaks in the vehicle, the system compares the user's speech to the voiceprint and verifies the speech, regardless of what the user is saying.

The embodiments disclosed herein include a passive enrollment of a re-occurring background process. During the process, samples of speech are embedded and stored, and unsupervised clustering is performed at regular intervals. After collecting and processing several voice samples, the voiceprint authentication system creates an embedding of features with high confidence that the embedding belongs to the same speaker. The voice authentication system may further refine the embedding boundaries for existing enrolled users. The voice authentication system may have a function to create a new embedding assigned to an anonymous non-user when the detected voiceprint does not belong to any known users. The canonical passively enrolled embeddings may be used to improve users' experience with their in-car virtual assistant.

Based on the comparison of the speech and voiceprint, the system may further generate a probabilistic notion based on their similarity and use the probabilistic notion in downstream “user preference” embeddings that require authentication. The use of a probabilistic notion in downstream embeddings improves the robustness of the user preference modeling, for example, reducing the noise level, capturing variability in the user preference embeddings, and improving the accuracy of recommendations.

Turning to the figures,FIG.1depicts an illustrative embodiment of a voiceprint authentication system100. The voiceprint authentication system100may comprise a voice feature module322, a similarity module332, and an authentication module342. The voiceprint authentication system100depicted inFIG.3includes a computing device equipped in the vehicle that includes a controller101. The controller101may further comprise various components, such as a memory302, a processor304, an input/output hardware305, a network interface hardware306, a data storage component307, a local interface303, and a sound sensor402. The voiceprint authentication system100may further include a button404(e.g. as illustrated inFIG.4) and a touchscreen406(e.g. as illustrated inFIG.4).

The controller101may be any device or combination of components comprising a processor304and a memory302, such as a non-transitory computer readable memory. The processor304may be any device capable of executing the machine-readable instruction set stored in the non-transitory computer readable memory. Accordingly, the processor304may be an electric controller, an integrated circuit, a microchip, a computer, or any other computing device. The processor304may include any processing component(s) configured to receive and execute programming instructions (such as from the data storage component307and/or the memory component302). The instructions may be in the form of a machine-readable instruction set stored in the data storage component307and/or the memory component302. The processor304is communicatively coupled to the other components of the controller101by the local interface303. Accordingly, the local interface303may communicatively couple any number of processors304with one another, and allow the components coupled to the local interface303to operate in a distributed computing environment. The local interface303may be implemented as a bus or other interface to facilitate communication among the components of the controller101. In some embodiments, each of the components may operate as a node that may send and/or receive data. While the embodiment depicted inFIG.1includes a single processor304, other embodiments may include more than one processor.

The memory302(e.g., a non-transitory computer-readable memory component) may comprise RAM, ROM, flash memories, hard drives, or any non-transitory memory device capable of storing machine-readable instructions such that the machine-readable instructions can be accessed and executed by the processor304. The machine-readable instruction set may comprise logic or algorithm(s) written in any programming language of any generation (e.g., 1GL, 2GL, 3GL, 4GL, or 5GL) such as, for example, machine language that may be directly executed by the processor304, or assembly language, object-oriented programming (OOP), scripting languages, microcode, etc., that may be compiled or assembled into machine readable instructions and stored in the memory302. Alternatively, the machine-readable instruction set may be written in a hardware description language (HDL), such as logic implemented via either a field-programmable gate array (FPGA) configuration or an application-specific integrated circuit (ASIC), or their equivalents. Accordingly, the functionality described herein may be implemented in any conventional computer programming language, as pre-programmed hardware elements, or as a combination of hardware and software components. For example, the memory component302may be a machine-readable memory (which may also be referred to as a non-transitory processor-readable memory or medium) that stores instructions that, when executed by the processor304, causes the processor304to perform a method or control scheme as described herein. While the embodiment depicted inFIG.1includes a single non-transitory computer-readable memory302, other embodiments may include more than one memory module.

The input/output hardware305may include a monitor, keyboard, mouse, printer, camera, microphone, speaker, and/or other device for receiving, sending, and/or presenting data. The network interface hardware306may include any wired or wireless networking hardware, such as a modem, LAN port, Wi-Fi card, WiMax card, mobile communications hardware, and/or other hardware for communicating with other networks and/or devices.

The sound sensor402is coupled to the local interface303and communicatively coupled to the processor304. The sound sensor402may be one or more sensors coupled to the voiceprint authentication system100for determining the volume, pitch, frequency, and/or features of sounds in a vehicle. The sound sensor402may include a microphone or an array of microphones that may include mechanisms to filter background noise, such as engine sounds or beamforming.

The data storage component307stores voiceprints317, user preference327, usage embedding337, and training data347.

The memory component302may include the voice feature module322, the similarity module332, and the authentication module342. The voice feature module322may further include a neural network module comprising an encoder and a decoder.

The voice feature module322may be trained and provided machine learning capabilities via the neural network122as described herein. By way of example, and not as a limitation, the neural network122may utilize one or more artificial neural networks (ANNs). In ANNs, connections between nodes may form a directed acyclic graph (DAG). ANNs may include node inputs, one or more hidden activation layers, and node outputs, and may be utilized with activation functions in the one or more hidden activation layers such as a linear function, a step function, logistic (sigmoid) function, a tanh function, a rectified linear unit (ReLu) function, or combinations thereof. ANNs are trained by applying such activation functions to training data sets to determine an optimized solution from adjustable weights and biases applied to nodes within the hidden activation layers to generate one or more outputs as the optimized solution with a minimized error. In machine learning applications, new inputs may be provided (such as the generated one or more outputs) to the ANN model as training data to continue to improve accuracy and minimize error of the ANN model. The one or more ANN models may utilize one to one, one to many, many to one, and/or many to many (e.g., sequence to sequence) sequence modeling. The one or more ANN models may employ a combination of artificial intelligence techniques, such as, but not limited to, Deep Learning, Random Forest Classifiers, Feature extraction from audio, images, clustering algorithms, or combinations thereof. In some embodiments, a convolutional neural network (CNN) may be utilized. For example, a convolutional neural network (CNN) may be used as an ANN that, in a field of machine learning, for example, is a class of deep, feed-forward ANNs applied for audio analysis of the recordings. CNNs may be shift or space invariant and utilize shared-weight architecture and translation.

FIG.2A and2Bdepict illustrative block diagrams of an illustrative implementation of the voice feature module, the similarity module, and the authentication module to authenticate a user and generate a probabilistic notion in association with the authenticated user for weighting and interpolation.

As illustrated in the block diagram ofFIG.2A, the voiceprint authentication system100receives a human voice110of a current speaker inside the automobile and determines whether the human voice110belongs to an enrolled user to be authenticated and also determines a probabilistic notion150associated with the authentication for downstream embeddings interpolation160(e.g. as illustrated inFIG.2B). More specifically, the voiceprint authentication system100receives a human voice110and sends the human voice110to a voice feature module322. The voice feature module322includes one or more neural networks122, which are trained to generate input features112based on training data comprising human voices spoken by a plurality of historical speakers inside the automobile. The input features112may be converted into an input vector113to be received by a similarity module332. The similarity module332compares the input vector113with historical vectors116(e.g. all the available vectors belonging to the enrolled users in the voiceprint authentication system100) in the voiceprints of the enrolled users to determine whether the input vector113is similar to at least one of the historical vectors116in a voiceprint belonging to an enrolled user. How similar two vectors are is measured as a similarity. After determining that a similarity between the input vector113and a historical vector116in a voiceprint belonging to an identified user is less than a threshold similarity, an authentication module342authenticates the current speaker who speaks the human voice110as the identified user. The authentication module342may further calculate a probabilistic notion based on the similarity. The calculated probabilistic notion may be applied to interpolate between downstream user preference embeddings associated with the identified user, for example between a user preference327and a usage embedding337illustrated inFIG.2B.

In embodiments, the voice feature module322uses the neural network122to generate a set of input features112based on the human voice110. The input features112of a human voice may include, without limits, data of tone, pitch, volume, speed, and timbre in each speech. The features of an enrolled user may then be pooled into a voiceprint317(e.g. as illustrated inFIG.3) of a user that includes all the historical features of voiceprints115since the user initially uses the voiceprint authentication system100for authentication.

The set of input features112may be plotted in a coordinate system as an input vector113. A vector is a vectored point in a coordinate system. For example, the voiceprint authentication system100may select a multiple-order coordinate system with each axis representing one variable of the features, where the variables may include tone, pitch, volume, speed, timbre, or the like. The voiceprint authentication system100may plot the vectors after receiving the speech of the same enrolled user each time into a cluster (a group of vectors), where such a cluster represents the voiceprint of that specific enrolled user. Similarly, a voiceprint317of an enrolled user may include all the historical features of voiceprints115since the user initially uses the voiceprint authentication system100for authentication. Also, a voiceprint317of an enrolled user may include all the historical vectors116as a cluster of an enrolled user since the user initially uses the voiceprint authentication system100for authentication. When the clusters belonging to various enrolled users are plotted in a coordinate system, the clusters may overlap (e.g. as illustrated inFIG.5A). It should be noted that a cluster of the voiceprint of an enrolled user may also be generated through plotting the features within the voiceprint317.

The similarity described herein is based on vectors (i.e. points of embeddings plotted into a coordinate system) or alternatively, on features. In embodiments, similarity may include a Euclidean similarity and a Cosine similarity. Euclidean similarity is the distance between two points of features. The greater the similarity, the closer the points are. Euclidean distance is the square root of the sum of squared differences between corresponding elements of the two vectors. Cosine Similarity is based on the angle between two vectors. Cosine similarity equals (1−cos α), where α is the angle between two vectors. For both Euclidean similarity and Cosine similarity, a smaller value of similarity indicates the vectors are more similar. Euclidean similarity may be preferred over Cosine similarity when voiceprint authentication system100receives sufficient data of human voice110from the enrolled users. Cosine similarity may be preferred when the ratio between features matters more than the prioritization of those features. For example, if the voiceprint authentication system100adopts a bi-coordinate system with the x-axis representing pitch and the y-axis representing tone when a person is featured with a high pitch with a high tone, a cosine similarity may be found for a vector representing a low pitch with a low tone. This is useful to admit more voice input at the initial stage when available voice features are limited and can be used to expand the available data to further validate the specific user and personalize the user's usage preference. Accordingly, the system may strategically select which similarity to use in authentication depending on the availability of the data/features in the voiceprint for that specific user. In some embodiments, the similarity module332may further adopt other types of similarity such as Jaccard similarity or Dice similarity in calculating the similarity. The Jaccard similarity measures the similarity between two sets of features by counting the features of items in common and dividing by the total number of features of the two sets. Dice similarity, which is similar to Jaccard similarity, weights the counts of common features by frequencies in the two sets.

The authentication module342may select a threshold similarity140to indicate a bottom line that the human voice received by the voiceprint authentication system100is recognized as the voice of an enrolled user. A low value of the threshold similarity140suggests a high authentication and privacy protection excludes a person from unauthorized usage. However, during an initial usage of the voiceprint authentication system100, a higher value may be adopted to admit more voice data into the system to increase the pool of the historical features of voiceprints115and allow the neural network122to be personalized to the one or more users. At a later stage, a lower threshold similarity140may be chosen when the system finds that sufficient confidential data is available. Occasionally, clusters (a group of vectors) of voiceprints may overlap. The voiceprint authentication system100may find high confidence that a human voice110belongs to two enrolled users because the input vector113generated based on the human voice110is plotted in the overlapped area of two clusters. At that point, the authentication may cause inaccurate weighted integration and interpolation160. Solutions for such inaccuracy are presented further below.

In embodiments, when the voiceprint authentication system100determines that a human voice does not belong to any enrolled user, the voiceprint authentication system100may create a voiceprint for a non-user. With such ability, the voiceprint authentication system100may include a plurality of voiceprints of non-users. Thus, the similarity module332(e.g. as illustrated inFIG.2A) may determine a similarity between the input vector and voiceprints of one or more non-users. After determining that the similarity is less than the threshold similarity, the voiceprint authentication system100may integrate the input vector113into the voiceprints of a non-user. Further, in determining that the similarity exceeds the threshold similarity, the voiceprint authentication system100may create a voiceprint of a non-user based on the input vector113. The function to create voiceprints for non-users allows the convenience of creating a high-confidence user voiceprint once a non-user may be later enrolled as a user.

The authentication module342determines a probabilistic notion150based on the similarity118calculated by the similarity module332between the input vector113and the historical vectors116. The probabilistic notion150may include a weight factor inversely proportional to the similarity118, where the weight factor has a value between 0 and 1. When the similarity suggests the input vector113is identical to a vector in the historical vectors116, the weight factor may equal to 1. When the similarity suggests the input vector113is outside all the historical vectors116, the weight factor may equal to 0.

Referring toFIG.2B, an illustrative block diagram of an illustrative implementation of the authentication module for authentication and generation of a probabilistic notion for weighting and interpolation is depicted. After the authentication module342generates the probabilistic notion150, the probabilistic notion150may be used to weight the different features in different user embeddings for the fact that some features are more important than others. For example, the probabilistic notion may be used to cluster users into different groups, which is helpful to identify users who have similar preferences or to predict how a user will respond to a new piece of content such that the vehicle system may predict or recommend content that the user is likely to enjoy.

Particularly, the probabilistic notion150may be used for incorporating real-time generated features and vectors into embeddings in the voiceprint authentication system100and other systems in the vehicle. For example, the input features112may be weighted152before being pooled into the historical features of voiceprints115of the identified user, a user comment120derived from the human voice110may be weighted152before pooled into a user preference327, and a user interaction130derived from the human voice110may be weighted152before pooled into a usage embedding.

The authentication module, after determining a probabilistic notion150based on the similarity118calculated by the similarity module332between the input vector113and the historical vectors116, may also grant authentication142of the speaker of the human voice110. Under such authentication, the speaker may be authenticated to use various functions in the vehicle system, such as using human voice110to command the vehicle to control the sub-systems of the vehicles. The speaker may command the vehicle to turn on or off radios or media players, make phone calls, conduct searches, and the like.

In embodiments, after the voiceprint authentication system100determines an authentication142is available for the identified user based on the human voice110, the voiceprint authentication system100may further determine whether the human voice110includes a user comment120, for example, the identified user provides a positive comment on a newly opened restaurant. After determining the human voice110and user comment120, the voiceprint authentication system100may weight the user comment120based on the probabilistic notion150and integrate the weighted152user comment120into the user preference327. Similarly, after the voiceprint authentication system100determines an authentication142is available for the identified user based on the human voice110, the voiceprint authentication system100may further determine whether the human voice110includes a user interaction130, for example, the identified user commands the vehicle system to play a specific song or make a phone call to the identified user's friend. After determining the human voice110and the user interaction130, the voiceprint authentication system100may weight the user interaction130based on the probabilistic notion150and integrate the weighted152user interaction130into the usage embedding337.

Further, the probabilistic notion150may be used to interpolate between different embeddings, such as the downstream user preference embeddings associated with the identified user. The downstream user preference embeddings may refer to the user preference embeddings that need a voiceprint authentication before providing services associated with the user preference embeddings or integrating new inputs into the user preference embeddings. The user preference embeddings may be collected from a variety of sources, such as commands to the vehicle system, website logs, and app usage data of the enrolled users. The user preferences may be incrementally calculated based on user comments narrated by the identified user during authenticating and dynamically integrated into historical user preferences. The downstream user preference embeddings include the user preference327and the usage embedding as described above. The probabilistic notion150may be used to enable an interpolation160between downstream high-dimensional user preference embeddings by providing a common representation of the identified user across downstream user preferences using different devices and applications. For example, an interpolated embedding170may be generated with the formula ax (user preference327) +(1−a)×(usage embedding337), where α is the weight factor. The weight factor having a value between 0 and 1 is inversely proportional to similarity118, and it may represent the relative importance of the two user preference embeddings, the user preference327and the usage embedding337. The interpolated embedding170can then be used to make predictions about the user's preferences. For example, the vehicle system may predict whether an authenticated user is likely to order a pizza from one of the local pizza shops by calling through the vehicle system for delivery or driving directly to pick it up. Further, the vehicle system may use the interpolated embedding170as one of the features in a machine-learning model.

Referring toFIG.3, an example neural network122of the voice feature module322in the voiceprint authentication system100that is being trained using training data and real-time data is depicted. The neural network122in the voiceprint authentication system100is used to generate features derived from a human voice110collected in the vehicle system. The neural network122may be pre-trained with features based on training data347comprising human voices spoken by a plurality of historical speakers inside the vehicle. Once the neural network122is pre-trained, it may be used to generate input features112derived from a human voice110. The input features112may be weighted using the probabilistic notion150to integrate into the voiceprint317of the associated enrolled user, which in turn is used to continuously train the neural network122.

In embodiments, the neural network122may include an incremental learning algorithm that dynamically integrates the input features112weighted based on the probabilistic notion150into the voiceprint317of the identified user. The incremental learning algorithm provides the ability to the neural network122to accumulate the historical features of previous tasks and capture the input features112of the current task simultaneously. For example, a newly updated voiceprint by the neural network122is fed back to the neural network122to train the neural network122based on the just processed data based on human voice110. This process is repeated as new human voice110becomes available, which allows the neural network122to continuously improve its accuracy. The incremental learning algorithm may allow the neural network122to generate features with little or no pre-training data. The incremental learning algorithm may work in four different models in this situation. In an evaluation mode, the incremental learning algorithm tracks the predictive performance of the model on the incoming data such as human voice110or over the entire history of the model used for incremental learning. In a detect drift mode, the incremental learning algorithm validates whether the predicted feature exhibits structural breaks or distribution drift, such as whether the distribution of the predicted features exhibit has sufficiently changed. In an active training mode, the incremental learning algorithm may also actively train itself by updating the model based on the incoming data such as human voice110. In a generating prediction mode, the incremental learning algorithm may generate features with predicted labels from the latest model. The neural network122may robustly switch between these modes depending on the sufficiency of voice data existing for an incremental model to generate predictions, and the sufficiency of training for concise prediction of features.

Referring toFIG.4, an example internal view of a vehicle equipped with the voiceprint authentication system100with the hardware for detecting speech in the vehicle and initializing a voiceprint registration in the vehicle is depicted. As illustrated inFIG.4, a voiceprint authentication system100may include one or more sound sensors402inside the vehicle to detect and process the sound waves that are produced when a person speaks in the vehicle. The sound sensors may be located in the ceiling, dashboard, or center console of the vehicle. The sound sensors may be connected to one or more microphones picking up the soundwaves. The soundwaves are then processed by the controller101which converts the soundwaves into digital signals.

The voiceprint authentication system100may further include a button and a touchscreen to initialize a voiceprint registration for a new user. The voiceprints of one or more enrolled users are enrolled through an initial implementation. The initial implementation may include a physical or vocal trigger of enrollment to initialize the enrollment. A user who wants to be enrolled via the vocal trigger of enrollment may need to request an enrolled user to be authenticated by the voiceprint authentication system100and to authorize the enrollment process. A user who wants to be enrolled via the physical trigger of enrollment may need to physically press the button404or touch the touchscreen406to access the setting menu of the voiceprint authentication system100and initialize a voice registration process. Once the voiceprint authentication system100indicates the voice registration process has begun, the user may provide a voice sample to the system to create the initial voiceprint. For example, the user may read a predetermined script to create a voiceprint. In another example, the user may speak any set of words without following a script. A wave file of the user's voice is then input into the neural network, may be pre-trained, to create a plurality of vectors representing features of the user's voice, and may store the vectors in the voiceprint authentication system100. The voiceprint authentication system100may further tailor the initial voiceprint based on the average of a user's registration phrases in the cluster of the initial voiceprint.

Referring toFIGS.5A and5B, an example of reducing voiceprints of multiple users for more precise learned features is depicted. As illustrated inFIG.5A, clusters of voiceprints of multiple users may overlap with each other. An overlap between different clusters is an issue because an input vector113may be considered highly similar to more than one enrolled user, and the voiceprint authentication system100may have difficulty accurately determining which user embedding to activate.

Further, a voiceprint belonging to a user may have isolated vectors away from the main area of the cluster. If a voiceprint has isolated vectors away from the main area of the cluster, it can be difficult for the voiceprint authentication system100to identify the user. This may be unavoidable when the user has a unique speech pattern that is not well-represented by the other vectors in the cluster (a truly isolated vector). However, in some cases, such isolated vectors are present due to artificial reasons. For example, the user may have been speaking in a noisy environment, which can make it difficult to capture all of the features of their voice, or the user may have been speaking quickly, which can also make it difficult to capture all of the features of their voice. Thus, it is desirable to remove these isolated vectors.

To address the above-mentioned issues, the voiceprint authentication system100may include a function to shrink the voiceprints to an extent that the voiceprint authentication system100can efficiently and accurately recognize a user. Voiceprint shrinking refers to reducing the dimensionality of the voiceprint representation of a user's voice to enhance the accuracy of voice recognition. This technique can be achieved through the use of neural networks. For example, the neural network122may shrink the voiceprint of the identified user by removing a feature of the voiceprint having a confidence less than a threshold confidence. To allow the neural network122to have such a function, the neural network122may be trained to recognize the different features that make up a voiceprint and the same features of a voiceprint that may belong to different users. The trained neural network122may be used to extract features from the original voiceprints (e.g. as shown inFIG.5A) to create new, smaller voiceprints (e.g. as shown inFIG.5B). The new voiceprints are smaller than the original voiceprint, but retain the important features. During the extraction, a confidence value may be used to determine which features are retained in the new voiceprint. Features with a confidence value greater than the threshold will be retained, while features with a confidence value below the threshold will be discarded. The choice of confidence value depends on the application. For example, the voiceprints used for identification require a high confidence value to ensure that the new voiceprint is more accurate. The methods to extract features may include principal component analysis (PCA) or recursive feature elimination (RFE). Particularly, as illustrated inFIG.5B, the voiceprints may be shrunk by removing the features of a voiceprint overlapping with another voiceprint. In some embodiments, the voiceprint authentication system100may include a machine learning function to differentiate the truly isolated vector from other isolated vectors because if the truly isolated vector is removed, it may risk falsely denying an authentication.

Referring toFIG.6, an example flowchart describing the operation of the voiceprint authentication system100of the present disclosure for voiceprint authentication142and interpolation160is depicted. At step601, the voice feature module322, using the neural network122, generates input features based on the human voice110of a current speaker inside the vehicle. The human voice110may be received using the sound sensor402. The neural network122may be pre-trained based on training data347including human voices spoken by a plurality of historical speakers inside the vehicle and may be continuously trained during the usage of the voiceprint authentication system100.

At step602, the similarity module332calculates similarities between the input vector113of the input features112and historical vectors116in the voiceprints of the one or more enrolled users. At step603, the authentication module342determines whether at least one similarity between the input vector113and historical vectors116in a voiceprint317is less than a threshold similarity140.

If no similarity between the input vector113and historical vectors116is less than the threshold similarity (No at step603), then at step604, the voiceprint authentication system100may create a voiceprint317of a non-user based on the input vector113of the input features112. If at least one similarity between the input vector113and historical vectors116is less than the threshold similarity140(Yes at step603), then at step605, then at step605, the authentication module342determines whether the historical vector116that is highly similar to the input vector113belongs to an enrolled user.

If the authentication module342determines that the historical vector does not belong to an enrolled user (No at step605), then at step606, the authentication module342recognizes the current speaker as a non-user, and the voiceprint authentication system100may integrate the input vector113into the voiceprint of the non-user. If the authentication module342determines that the historical vector does belong to an enrolled user (Yes at step605), then at step607, the authentication module342recognizes the current speaker as an identified user, and authenticates the current speaker as the identified user.

At step608, the authentication module342calculates a probabilistic notion150based on the similarity118. At step609, the voiceprint authentication system100applies the probabilistic notion150to interpolate between preference embeddings associated with the identified user. For example, the voiceprint authentication system100may interpolate between the user preference327integrating weighted user comments120and the usage embedding337integrating weighted user interaction130.

Referring toFIG.7, an example flowchart describing the operation of the neural network122of the voiceprint authentication system100is depicted. At step701, the neural network122may be pre-trained based on training data347comprising human voices spoken inside the vehicle. The training data347may further include background sounds in the vehicle such that the neural network122may be trained to remove background noises associated with the human voice. The neural network122may further be trained to determine whether a receiving sound wave picked by the sound sensor402is recorded inside the vehicle or whether the receiving sound wave is a record rather than a real-time speech.

At step702, the sound sensor402receives a human voice and the neural network122within the voice feature module322generates the input features112based on the incoming data such as human voice110.

At step703, the authentication module342determines whether to authenticate an identified user. If there is not authentication for an identified user (No at step703), the neural network122may decline to take the input features for training and wait for receiving a new human voice to generate input features based on the new human vice at step702.

If there is an authentication for an identified user (Yes at step703), at step704, the authentication module342calculates a probabilistic notion150based on the similarity118between the input vector113and a historical vector116of the identified user.

At step705, the voiceprint authentication system100feeds the input features112and the input vector113to the neural network122for training. After training, the voiceprint authentication system100may receive another human voice110for another round of training and thereby continuously improve the accuracy of the neural network122.

Referring toFIG.8, an example flowchart describing the steps of the method for authenticating a speaker and applying a probabilistic notion for interpolation is depicted. At step801, the method includes a step of training a neural network to generate features based on training data comprising human voices spoken by a plurality of historical speakers inside a vehicle. At step802, the method includes a step of generating, using the neural network, input features based on a human voice of a current speaker inside the vehicle. At step803, the method includes a step of calculating similarities between an input vector of the input features and historical vectors in voiceprints of one or more enrolled users. At step804, the method includes a step of after determining a similarity between the input vector and at least one historical vector in a voiceprint of an identified user is less than a threshold similarity, authenticating the current speaker as the identified user. At step805, the method includes a step of calculating a probabilistic notion based on the similarity. At step806, the method includes a step of applying the probabilistic notion to interpolate between downstream user preference embeddings associated with the identified user.