Patent Description:
An artificial neural network, which may comprise an interconnected group of artificial neurons (e.g., neuron models), is a computational device or represents a method to be performed by a computational device.

Convolutional neural networks are a type of feed-forward artificial neural network. Convolutional neural networks may include collections of neurons that each have a receptive field and that collectively tile an input space. Convolutional neural networks (CNNs) have numerous applications. In particular, CNNs have broadly been used in the area of pattern recognition and classification.

Deep learning architectures, such as deep belief networks and deep convolutional networks, are layered neural networks architectures in which the output of a first layer of neurons becomes an input to a second layer of neurons, the output of a second layer of neurons becomes and input to a third layer of neurons, and so on. Deep neural networks may be trained to recognize a hierarchy of features and so they have increasingly been used in object recognition applications. Like convolutional neural networks, computation in these deep learning architectures may be distributed over a population of processing nodes, which may be configured in one or more computational chains. These multi-layered architectures may be trained one layer at a time and may be fine-tuned using back propagation.

Other models are also available for object recognition. For example, support vector machines (SVMs) are learning tools that can be applied for classification. Support vector machines include a separating hyperplane (e.g., decision boundary) that categorizes data. The hyperplane is defined by supervised learning. A desired hyperplane increases the margin of the training data. In other words, the hyperplane should have the greatest minimum distance to the training examples.

Although these solutions achieve excellent results on a number of classification benchmarks, their computational complexity can be prohibitively high. Additionally, training of the models may be challenging.

<CIT> of VIDEOIQ, INC, relates to a method of matching objects represented in multiple images in video surveillance.

Additional features and advantages of the disclosure will be described below. The novel features, which are believed to be characteristic of the disclosure, both as to its organization and method of operation, together with further objects and advantages, will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present disclosure.

Based on the teachings, one skilled in the art should appreciate that the scope of the disclosure is intended to cover any aspect of the disclosure, whether implemented independently of or combined with any other aspect of the disclosure. For example, an apparatus may be implemented or a method may be practiced using any number of the aspects set forth. In addition, the scope of the disclosure is intended to cover such an apparatus or method practiced using other structure, functionality, or structure and functionality in addition to or other than the various aspects of the disclosure set forth.

Although particular aspects are described herein, many variations and permutations of these aspects fall within the scope of the disclosure. Although some benefits and advantages of the preferred aspects are mentioned, the scope of the disclosure is not intended to be limited to particular benefits, uses or objectives. Rather, aspects of the disclosure are intended to be broadly applicable to different technologies, system configurations, networks and protocols, some of which are illustrated by way of example in the figures and in the following description of the preferred aspects. The detailed description and drawings are merely illustrative of the disclosure rather than limiting, the scope of the invention being defined by the appended claims.

During online training, reader devices, such as fingerprint readers or iris scanners, may determine whether an object is a match with a trained sample. In conventional systems, during the initial off-line training, a number of positive matches, such as true fingers, and negative matches, such as doctored fingers, are used for the training. The positive matches and negative matches may also be used for determining an initial decision boundary.

Additionally, in conventional systems, the decision boundary may be fine-tuned with online training when the fingerprint reader is used. For many use cases, such as spoofing real fingerprints, only positive examples or an increased number of positive examples in comparison to the negative examples are provided during the online training. Still, an increased number of positive examples may update the feature vector classifier such that the decision boundary is moved away from the positive examples, thereby increasing the probability that a fake finger would be classified as a true finger.

Aspects of the present disclosure are directed to decreasing the probability that a fake finger is classified as a true finger by improving the classification boundary for the device owner's finger. Although the present description is with respect to finger classification, it is noted that fingers are merely one example of an object to be classified. The present disclosure applies equally to any other type of object.

<FIG> illustrates an example implementation <NUM> of the aforementioned online training of a classifier using a system-on-a-chip (SOC) <NUM>, which may include a general-purpose processor (CPU) or multi-core general-purpose processors (CPUs) <NUM> in accordance with certain aspects of the present disclosure. Variables (e.g. neural signals and synaptic weights), system parameters associated with a computational device (e.g. neural network with weights), delays, frequency bin information, and task information may be stored in a memory block associated with a neural processing unit (NPU) <NUM>, in a memory block associated with a CPU <NUM>, in a memory block associated with a graphics processing unit (GPU) <NUM>, in a memory block associated with a digital signal processor (DSP) <NUM>, in a dedicated memory block <NUM>, or may be distributed across multiple blocks. Instructions executed at the general-purpose processor <NUM> may be loaded from a program memory associated with the CPU <NUM> or may be loaded from a dedicated memory block <NUM>.

The SOC <NUM> may also include additional processing blocks tailored to specific functions, such as a GPU <NUM>, a DSP <NUM>, a connectivity block <NUM>, which may include fourth generation long term evolution (<NUM> LTE) connectivity, unlicensed Wi-Fi connectivity, USB connectivity, Bluetooth connectivity, and the like, and a multimedia processor <NUM> that may, for example, detect and recognize gestures. In one implementation, the NPU is implemented in the CPU, DSP, and/or GPU. The SOC <NUM> may also include a sensor processor <NUM>, image signal processors (ISPs), and/or navigation <NUM>, which may include a global positioning system.

The SOC may be based on an ARM instruction set. In an aspect of the present disclosure, the instructions loaded into the general-purpose processor <NUM> may comprise code for determining a distance from one or more feature vectors of an object, which is observed during the online training, to a first predetermined decision boundary established during off-line training for the classifier. The instructions loaded into the general-purpose processor <NUM> may also comprise code for updating a decision rule as a function of the distance. The instructions loaded into the general-purpose processor <NUM> may further comprise code for classifying a future example based on the updated decision rule.

<FIG> illustrates an example implementation of a system <NUM> in accordance with certain aspects of the present disclosure. As illustrated in <FIG>, the system <NUM> may have multiple local processing units <NUM> that may perform various operations of methods described herein. Each local processing unit <NUM> may comprise a local state memory <NUM> and a local parameter memory <NUM> that may store parameters of a neural network. In addition, the local processing unit <NUM> may have a local (neuron) model program (LMP) memory <NUM> for storing a local model program, a local learning program (LLP) memory <NUM> for storing a local learning program, and a local connection memory <NUM>. Furthermore, as illustrated in <FIG>, each local processing unit <NUM> may interface with a configuration processor unit <NUM> for providing configurations for local memories of the local processing unit, and with a routing connection processing unit <NUM> that provides routing between the local processing units <NUM>.

Deep learning architectures may perform an object recognition task by learning to represent inputs at successively higher levels of abstraction in each layer, thereby building up a useful feature representation of the input data. In this way, deep learning addresses a major bottleneck of traditional machine learning. Prior to the advent of deep learning, a machine learning approach to an object recognition problem may have relied heavily on human engineered features, perhaps in combination with a shallow classifier. A shallow classifier may be a two-class linear classifier, for example, in which a weighted sum of the feature vector components may be compared with a threshold to predict to which class the input belongs. Human engineered features may be templates or kernels tailored to a specific problem domain by engineers with domain expertise. Deep learning architectures, in contrast, may learn to represent features that are similar to what a human engineer might design, but through training. Furthermore, a deep network may learn to represent and recognize new types of features that a human might not have considered.

A deep learning architecture may learn a hierarchy of features. If presented with visual data, for example, the first layer may learn to recognize simple features, such as edges, in the input stream. If presented with auditory data, the first layer may learn to recognize spectral power in specific frequencies. The second layer, taking the output of the first layer as input, may learn to recognize combinations of features, such as simple shapes for visual data or combinations of sounds for auditory data. Higher layers may learn to represent complex shapes in visual data or words in auditory data. Still higher layers may learn to recognize common visual objects or spoken phrases.

Deep learning architectures may perform especially well when applied to problems that have a natural hierarchical structure. For example, the classification of motorized vehicles may benefit from first learning to recognize wheels, windshields, and other features. These features may be combined at higher layers in different ways to recognize cars, trucks, and airplanes.

Neural networks may be designed with a variety of connectivity patterns. In feed-forward networks, information is passed from lower to higher layers, with each neuron in a given layer communicating to neurons in higher layers. A hierarchical representation may be built up in successive layers of a feed-forward network, as described above. Neural networks may also have recurrent or feedback (also called top-down) connections. In a recurrent connection, the output from a neuron in a given layer is communicated to another neuron in the same layer. A recurrent architecture may be helpful in recognizing patterns that unfold in time. A connection from a neuron in a given layer to a neuron in a lower layer is called a feedback (or top-down) connection. A network with many feedback connections may be helpful when the recognition of a high level concept may aid in discriminating the particular low-level features of an input.

In one configuration, a machine learning model is configured for determining a distance from one or more feature vectors of an object to a first predetermined decision boundary established during off-line training for the classifier. In one configuration, the one or more feature vectors are received during online training. The model is also configured for updating a decision rule as a function of at least the distance. The model is further configured for classifying a future example based on the updated decision rule. The model includes a determining means, updating means, and/or classifying means. In one aspect, the determining means, updating means, and/or classifying means may be the general-purpose processor <NUM>, program memory associated with the general-purpose processor <NUM>, memory block <NUM>, local processing units <NUM>, and or the routing connection processing units <NUM> configured to perform the functions recited. In another configuration, the aforementioned means may be any module or any apparatus configured to perform the functions recited by the aforementioned means.

According to certain aspects of the present disclosure, each local processing unit <NUM> may be configured to determine parameters of the machine learning network based upon desired one or more functional features of the network, and develop the one or more functional features towards the desired functional features as the determined parameters are further adapted, tuned and updated.

As previously discussed, during online training, reader devices, such as fingerprint readers or iris scanners, may determine whether an object is a match with a trained sample. Furthermore, for some reader devices, it is desirable to improve the reader so that the reader may determine whether the object is an actual object or a doctored object. For example, the object may be a finger. Thus, a doctored finger refers to an object that is not an actual finger. For example, the doctored finger is an object that may be used to mimic a fingerprint, such as a plastic finger with an embedded fingerprint. The doctored finger may be referred to as a fake finger.

In conventional systems, during the initial off-line training, positive matches and negative matches may be used for determining an initial decision boundary. In one configuration, the initial decision boundary is determined via an example vector classifier. The example vector classifier may be a support vector machine (SVM) classifier. Furthermore, the classifier may be a linear classifier or a nonlinear classifier.

It should be noted that the example vectors may define a boundary. That is, example vectors refer to the off-line determined set of feature vectors that define/determine a decision boundary. In one example, for a support vector machine, the example vectors would be support vectors. Furthermore, a feature vector refers to a vector determined for each object, such as a fingerprint. Additionally, a minimum vector may refer to a feature vector at a distance Dmin.

Additionally, in conventional systems, the decision boundary may be fine-tuned with online training when the fingerprint reader is used. <FIG> illustrates a one-dimensional example of a decision boundary <NUM> that is a boundary between a real example vector (REV), for determining whether a fingerprint is from a real finger, and a fake example vector (FEV), used to determine whether a fingerprint is from a fake finger. Based on margin maximization, the decision boundary <NUM> is placed halfway between the real example vector and the fake example vector. In the example of <FIG>, a finger may be classified as real if it is detected between the real example vector and the decision boundary <NUM>. Alternatively, the finger may be classified as fake if it is classified between the decision boundary <NUM> and the fake example vector.

<FIG> illustrates an example of using online training for a decision boundary <NUM> in a conventional device. As shown in <FIG>, the device may receive multiple fingerprints R as online training samples. The multiple fingerprints described may be vectors corresponding to the received fingerprints. As shown in <FIG>, the multiple fingerprints R1-R3 may be further away from the decision boundary <NUM> than the real example vector. Thus, in this example, because the fingerprints R1-R3 are farther away from the real example vector, the decision boundary <NUM> is not adjusted.

<FIG> illustrates an example of using online training for an initial decision boundary <NUM> in a conventional device. As shown in <FIG>, the device may receive a fingerprint R for training. Additionally, as shown in <FIG>, the fingerprint R may be closer to the initial decision boundary <NUM> than the real example vector. Thus, in this example, because the fingerprint R is closer to the initial decision boundary, based on margin maximization, the initial decision boundary <NUM> is adjusted so that a modified decision boundary <NUM> is located halfway between the new fingerprint R and the fake example vector. In this example, because the modified decision boundary <NUM> has been moved to be closer to the fake example vector than the real example vector, doctored fingers may be falsely categorized as true fingers. This may be contrary to the desired effect of using online fingerprints to tighten the spoofing boundary to the user's finger.

Furthermore, in this example, the position of the real example vector may also be adjusted to the position of the received fingerprint R. Thus, in this example the modified decision boundary <NUM> may be further moved towards the fake example vector if subsequent fingerprint test samples are received that are closer to the modified decision boundary <NUM> than the adjusted real example vector (e.g., the position of the received fingerprint R).

Conventional systems do not fine-tune or customize the fingerprint reader based on online usage. Rather, the conventional systems maintain the off-line trained decision boundary. Additionally, or alternatively, a conventional system may include negative examples for use in the online training.

As previously discussed, aspects of the present disclosure are directed to decreasing the probability that a fake finger is classified as a true finger by improving the classification boundary for the device owner's finger. Although the present description is with respect to finger classification, it is noted that fingers are merely one example of an object to be classified. The present disclosure applies equally to any other type of object.

In one configuration, the decision boundary for a device is improved based on increased use, such as daily use, with positive examples received during online training. That is, an online training example may be a positive example of a real finger. More specifically, in one configuration, negative samples are not received during online training. The improved decision boundary may result in a more secure fingerprint recognition over time. That is, the device may be harder to compromise via a false reading.

In the present configuration, the device is initialized with the off-line trained fingerprint recognizer. As an example, the off-line training may generate a decision boundary <NUM> (<FIG>) for determining whether a finger is real or fake. During the initialization, a distance (Dmin) is set to a predetermined value, such as infinity or negative one. Additionally, during the initialization, the number of fingerprints received N is set to zero. The distance (Dmin) is the closest distance between a fingerprint vector received during online training and the decision boundary. The distance (Dmin) may be referred to as a minimum vector distance (Dmin). The fingerprint vector may be referred to as a fingerprint feature vector.

After initialization, the user may register multiple fingerprints as training samples. The device may compute the distance (Dfv) of each fingerprint to the decision boundary. For example, the device may use the inner product of each fingerprint feature vector and the decision boundary hyper-plane unit vector to calculate the distance (Dfv). In this configuration, the received fingerprint having a distance (Dfv) that is closest to the decision boundary is maintained as a new vector. The new vector may be referred to as the tightest point seen online (TPSO). Additionally, in one configuration, any previously cached fingerprint feature vectors are discarded.

In addition, the value of (Dmin) is updated to equal the fingerprint distance (Dfv) that is associated with the tightest point seen online. Furthermore, the number of received fingerprints N is incremented based on the number of fingerprints received. In this configuration, the off-line trained example vectors are maintained and are used to set the orientation of a modified decision boundary. Furthermore, in the present configuration, the tightest point seen online and number of fingerprints received N are used for decision boundary adjustment. In one configuration, the tightest point seen online is the minimum observed value of Dfv that is on the side of the decision boundary associated with real example vectors. In most cases, as the number of received fingerprints N increases, a new fingerprint distance (Dfv) to the initial decision boundary should be greater than the distance (Dmin) of the tightest point seen online to the decision boundary plus or minus a delta value. The use of the delta value is optional.

In one configuration, the modified decision boundary may be adjusted based on the number of fingerprints received. That is, if the number of fingerprints received is less than a threshold, then the adjustment for the modified decision boundary is throttled so that the modified decision boundary is not adjusted by a high amount. Alternatively, the amount of adjustment for the modified decision boundary may be increased if the number of fingerprints received is greater than a threshold. Specifically, in the present configuration, for the true finger versus false finger classifier, the device adjusts the position of a modified decision boundary for a successful true finger classification by an amount based on a function ( <MAT>) of the number of received fingerprints N and the closest distance (Dmin) between a received fingerprint and the initial decision boundary.

Moreover, in one configuration, each time the user successfully uses their fingerprint (e.g., successful fingerprint recognition and successful true finger recognition) the aforementioned steps that follow the initialization are repeated.

<FIG> illustrates an example of determining a modified decision boundary according to an aspect of the present disclosure. As shown in <FIG>, a real example vector (REV), a fake example vector (FEV), and an initial decision boundary <NUM> are specified based on the off-line training. After initialization (e.g., after the off-line training is complete), the device may receive multiple fingerprints (R1-R3) for testing. As shown in <FIG>, the device calculates a distance for a set of values (Dfv1, Dfv2 ,Dfv3) between each fingerprint sample and the initial decision boundary <NUM>. In this example, the distance (Dfv1) for a first finger R1 is the closest to the initial decision boundary <NUM> in comparison to the distance (Dfv2, Dfv3) for the other fingerprints (R2, R3). Therefore, the vector for the first finger R1 is set as the tightest point seen online and the minimum vector distance (Dmin) is set to the distance (Dfv1) of the first finger R1. Accordingly, as shown in <FIG> a modified decision boundary <NUM> may be specified based on the value of the minimum vector distance (Dmin) and the number of received fingerprints N. It should be noted that the initial decision boundary <NUM> calculated from the off-line training is also maintained.

A specified margin function for the feature vector distance (Dfv) to the off-line trained boundary (e.g., initial decision boundary) is: <MAT>.

In EQUATION <NUM>, Mmin and k are fixed parameters where Mmin controls the minimum margin from the real example vector that is closest to the decision boundary, and k is a design parameter that controls the rate at which the minimum vector distance (Dmin) is moved to the fingerprint feature vector that is closest to the initial decision boundary. According to aspects of the present invention EQUATION <NUM> should be satisfied for the feature vector to be classified as real.

In one configuration, the registration fingerprints are used to set the initial values of the number of fingerprints received N and the minimum vector distance (Dmin). For example, if five fingerprints are registered, then N equals five and the minimum vector distance (Dmin) is set to the fingerprint feature vector that is closest to the initial decision boundary. In one configuration, the initial minimum vector distance (Dmin) could have a special value such as infinity. When the minimum vector distance (Dmin) has a special value, EQUATION <NUM> is not used so that a divide by zero is avoided while N equals zero. Rather, the default boundary is used. Additionally, once the first valid fingerprint triggers the previously discussed updating, N is incremented to be greater than or equal to one, the minimum vector distance (Dmin) is a finite value, and a fingerprint is considered real when EQUATION <NUM> is satisfied. Other equations could be used that would have different rates and tradeoffs of increasing the required margin toward the minimum vector distance (Dmin), and saturating at the minimum vector distance (Dmin) or (Dmin - Mmin).

Based on the margin equation shown in EQUATION <NUM>, when the number of fingerprints received N is less than a threshold, the true/doctored finger classifier is based primarily on the initial decision boundary. Still, as the number of received fingerprints N increases to be greater than a threshold, a better understanding of the owner's finger feature vectors is obtained. Thus, the modified decision boundary is adjusted to customize to the owner's finger. The orientation of the decision boundary does not change, rather, the decision boundary location changes. Therefore, the absence of negative examples in the online training does not have an undue detrimental effect by over-fitting to the user's finger and changing the boundary orientation.

Thus, each time the device owner successfully uses their fingerprint to unlock their device, or for other purposes, the decision boundary of the true finger classifier is moved to the closest received finger that is real.

<FIG> illustrates an example of using a specified margin function for the feature vector distance. As shown in <FIG>, a real example vector (REV) and a modified decision boundary <NUM> have been calculated based on online training. The initial decision boundary <NUM> and fake example vector (FEV) are also shown in <FIG>. Furthermore, as shown in <FIG>, after determining the real example vector, modified decision boundary <NUM>, initial decision boundary <NUM>, and fake example vector, the device may receive a new fingerprint testing sample R. The device determines whether the feature vector distance (Dfv) for the new fingerprint testing sample R is greater than <MAT>, where Dmin is the distance of the real example vector to the initial decision boundary <NUM>, Mmin controls the minimum margin from the real example vector that is closest to the decision boundary, and <MAT> is a function based on the number of fingerprints that are seen. In the present configuration, to be considered as valid, the fingerprint sample should exceed both boundaries <NUM> and <NUM>. Still, if the minimum vector distance (Dmin) equals zero and Mmin equals one, then (Dmin - Mmin) equals negative one and the boundary may be shifted to the right of <NUM>. Thus, in one configuration, the sample should exceed both boundaries. In another configuration, the fingerprint sample is based on whether the value of max(<NUM>, EQUATION <NUM>) is greater than or equal to zero.

That is, the initial decision boundary <NUM> is one test of true/false (e.g., real/fake) and the modified decision boundary <NUM> is a second test of true/false or real/fake. It is desirable to classify a sample as real by exceeding (e.g., being to the left of) both the initial decision boundary <NUM> and the modified decision boundary <NUM>. In one configuration, the distance to the initial decision boundary <NUM> and/or the modified decision boundary <NUM> is determined and the distance is compared to the value of max(<NUM>, EQUATION <NUM>). The max(<NUM>, EQUATION <NUM>) combines the two tests. Such that, if EQUATION <NUM> is greater than zero, the modified decision boundary <NUM> is to the left of the initial decision boundary <NUM> and is the active boundary constraint. Additionally, if EQUATION <NUM> is less than zero, which is possible when the minimum vector distance (Dmin) is less than Mmin, the modified decision boundary <NUM> is to the right of the initial decision boundary <NUM>, thus the initial decision boundary <NUM> is the active boundary constraint.

As previously discussed, when receiving a new fingerprint testing sample during online training, if it is determined that the feature vector distance (Dfv) for a new fingerprint testing sample R is greater than <MAT>, then the new fingerprint testing sample R and the initial decision boundary <NUM> are used determine the modified decision boundary <NUM>. The feature vector distance (Dfv) is the distance from the feature vector for the fingerprint to the off-line trained decision boundary. The feature vector distance may have a positive sign for being on the real side of the boundary and a negative sign for being on the false side of the boundary.

In another example, the new fingerprint testing sample R may be rejected if the feature vector distance (Dfv) for the new fingerprint testing sample R is less than <MAT>.

In some cases, based on the example shown in <FIG>, when the number of fingerprints received N is greater than a threshold, such that <MAT> is approximately one, the modified decision boundary <NUM> is shifted to the left to be at the minimum vector distance (Dmin). Still, it is desirable to have a distance between the modified decision boundary <NUM> and the minimum vector distance (Dmin). Therefore, the modified decision boundary <NUM> is shifted to the right of the minimum vector distance (Dmin) by an amount Mmin. When the number of fingerprints received N is less than a threshold, the modified decision boundary <NUM> is shifted further to the right toward the initial decision boundary <NUM> by decaying the (Dmin-Mmin) value by (<NUM> - <MAT>.

In some cases, outliers may be used as a negative match or a positive match. In one configuration, to reduce the effects of outliers, when an outlier is received, instead of the computed minimum vector distance (Dmin) from the closest distance (Dfv) between a received fingerprint and the initial decision boundary, the distances (Dfv) of the received fingerprints may be sorted to determine a cumulative distribution. In this configuration, the 95th percentile point of the received fingerprints is used as the minimum vector distance (Dmin).

That is, in one configuration, the device maintains all of the distances (Dfv) of the received fingerprints and selects the 95th percentile closest distance (Dfv) or 98th percentile closest distance (Dfv) instead of the distance (Dfv) that is closest to the initial decision boundary. The 95th percentile and 98th percentile are two different examples. Of course, aspects of the present disclosure are not limited to the 95th percentile and 98th percentile as other values are contemplated.

In one configuration, a specific number of fingerprint feature vectors, such as one hundred, are maintained. Thus, when a new fingerprint feature vector passes the test, one of the fingerprint example vectors is randomly discarded to maintain the specific number of values. The number of values may be random.

In another configuration, a specific number of fingerprint feature vectors closest to the initial decision boundary is maintained, such as k closest fingerprint feature vectors. Additionally, in this configuration, the fingerprint feature vector of the maintained fingerprint feature vectors that is furthest from the initial decision boundary is used to determine the minimum vector distance (Dmin). In this configuration, the k-<NUM> other fingerprint feature vectors are dropped as potential outliers.

In yet another configuration, the minimum vector distance (Dmin) may not be decreased by more than a given percent, such as ten percent. Thus, after the initial set of registration prints set the initial minimum vector distance (Dmin), then a new minimum vector distance (Dmin) is only updated if the new minimum vector distance (Dmin) is greater than or equal to ninety percent of the old minimum vector distance (Dmin). In some cases, distributions are often denser while outliers may be sparse. Thus, over time, the minimum vector distance (Dmin) may observe a series of values reducing the minimum vector distance (Dmin) in incremental steps while discarding outliers.

Additionally, or alternatively, the device may use an outer loop to target a given false rejection rate (FRR). For example, the target false rejection rate may be two percent. Thus, in one configuration, each time a fingerprint passes the baseline/off-line trained decision boundary, the fingerprint is subsequently tested against the online modified boundary. If the fingerprint passes based on the online modified boundary, the distance is increased by ninety-eight percent of a value. In one example, a fingerprint sample at the modified decision boundary location for one hundred fingerprints will pass ninety-eight times and fail two times, such that (<NUM>*pass) - (<NUM>*fail) = <NUM> or fail value = (<NUM>/<NUM>) * pass value. In this example, on average, the modified decision boundary stays in the same location. The value is specified to be small enough so that the modified decision boundary does not move much over the one hundred trials. Furthermore, the value is specified to be large enough so that the modified decision boundary moves over a reasonable training time.

Alternatively, if the fingerprint fails based on the modified decision boundary, the distance is decreased by two percent of a value. The inner loop of the present configuration should converge toward a two percent false rejection rate, should be no worse than the initial decision boundary, and/or should be tested against both boundaries.

In some cases, the modified decision boundary may be moved closer to the fake example vector. In one configuration, a limit may be specified for the decreases of the modified decision boundary, such as not allowing more than two decreases for every twenty-five increases. In another configuration, the decrease is limited to prevent further decreases until there have been one or more increases. The decrease refers to moving the modified decision boundary closer to the fake example vector.

In another configuration, the modified decision boundary may be moved closer to the fake example vector if there is a liveness failure within a specified time, such as two seconds, of a liveness and true print successful classification. The liveness classifier determines whether the finger is a real (e.g., live) finger. In this configuration, if multiple attempts occurred in a short back-to-back predetermined time, the failure case may be used as if it was a success case for updating the minimum vector distance (Dmin) value. In some cases, the liveness failure may be limited to no worse than the off-line trained model. Additionally, or alternatively, for general cases the back-to-back attempts may push the modified decision boundary closer to the fake example vector.

In some cases, it may be desirable to push the modified decision boundary closer to the fake example vector. In one configuration, for some cases, such as security applications, the modified decision boundary should not be less than the off-line trained boundary. For example, the minimum vector distance (Dmin) should be greater than or equal to zero. For other cases, such as non-security applications, the modified decision boundary may be moved beyond the off-line boundary. For example, the minimum vector distance (Dmin) may be less than zero.

In another configuration, statistics of the samples may be used, such as the variance of the measured distances (Dfv) to select parameters/models for how fast the modified decision boundary is improved. For example, if the distances (Dfv) have a small variance, then Mmin and k may be small. Alternatively, if the distances (Dfv) have a large variance, then the Mmin and k may be large. For example, k = a*var(R) + b, for some choice of a and b, or other functions, where R is the set of observed feature vector distance and may use the measured distances (Dfv) as a notation. The variance may also be measured and used to detect outliers, such as not decreasing the minimum vector distance (Dmin) by more than one standard deviation when the value of one could be specified.

In some cases, the modified decision boundary may be adapted based on the classifier score. For some cases, multiple classifiers are used to determine whether the fingerprint identity matches. Thus, rather than adding the minimum vector distance (Dmin) based on the closest print that passes the identity match, the classifier score or confidence from the identity classifier is used to adapt the liveness classifier threshold. That is, it may be desirable to also use classification information.

Specifically, in one configuration, a fingerprint may have two or more classifiers. One classifier may determine if the fingerprint is from the valid user. The second classifier may determine if the print is from a real finger. Thus, in one configuration, both classifiers are jointly determined. For example, if there is a strong confidence that the print is from a valid user, it may be more likely to be a real fingerprint. Therefore, if the fingerprint is from a valid user, the boundaries for determining whether the fingerprint is from a real or fake finger may be relaxed. In one configuration, the minimum vector distance (Dmin) is multiplied by an identity confidence (C), where the identity confidence is between zero and one, and the confidence is a parameter. In this example, the higher the identity confidence, the looser the liveness boundary.

The identity confidence is a normalized score of the valid user classifier. That is, the valid user classifier may provide a score C between <NUM> and <NUM>, where <NUM> refers to a high confidence that the fingerprint matches the valid user fingerprint and <NUM> refers to a high confidence the fingerprint does not match the valid user fingerprint. In this configuration, for the second liveness classifier, a fingerprint is real is if Dfv is greater than ((<NUM>-C)*Dmin).

In one configuration, the model parameters, such as Dmin and N, k, and Mmin, are crowd sourced to a central location. The model parameters may be pushed out to either existing or new devices to improve the out of the box false rejection rate/false acceptance rate performance.

In some cases, for some positive class values there may be a variance over time, such as difference over night and day, or from summer to winter. In one configuration, a time window component may be specified. In another configuration, in addition to, or alternate from, using the number of fingerprints N to determine the minimum vector distance (Dmin), a time frame may be also be specified. That is, if a user swipes their finger one thousand times in a time frame, such as two minutes, the system may not be receiving the aforementioned variations that may be protected by using the <MAT> term. Thus, <MAT> of EQUATION <NUM> may be multiplied by min(<NUM>,t/T), where t and T define different times. For example, T is <NUM> days and t is the time in days since the first online fingerprint was received. Thus, in this example, after <NUM> days t = <NUM> and t/T = <NUM>/<NUM> ~ <NUM>. Therefore, <MAT> *<NUM>. Thus, <MAT> determines the modified decision boundary. In another example, if t = <NUM> days, then min(<NUM>, <NUM>/<NUM>) = <NUM>. Accordingly, after a year the margin is specified based on the margin previously discussed in the disclosure. Still, before the end of a year (T), an additional margin may be specified to collect data.

Therefore, based on the example of <FIG>, when t is small, the product of <MAT> and min(<NUM>,t/T) is near zero and the initial decision boundary <NUM> threshold is used. Still, as t approaches T, the product of <MAT> and min(<NUM>, t/T) becomes <NUM> and the modified decision boundary <NUM> is used.

It should be noted that for both the aforementioned configuration and for the baseline configuration, EQUATION <NUM> may use the initial decision boundary <NUM> until the number of received fingerprints N is greater than a received fingerprint threshold and/or the time is greater than a time threshold, at which point EQUATION <NUM> uses (Dmin-Mmin).

In another configuration, instead of moving the decision boundary as described above, the running online sample standard deviation and mean are computed based on values exceeding the initial off-line trained decision boundary. After receiving a specific number of fingerprints, a second test may be applied to determine if the value passes the off-line trained boundary and is within a number of online computer standard-deviations of the online computed mean distance value.

In yet another configuration, the online training is stopped if the modified decision boundary approaches a specified threshold. For example, the online training may be stopped when (<NUM>-<NUM>/N)k is greater than <NUM>. That is, training may be stopped when a condition is met. For example, training may be stopped based on a number of fingerprints received N, the passage of a certain amount of time, a threshold on the minimum vector distance (Dmin), and/or other appropriate conditions.

Furthermore, an outer loop may be used to track the false rejection rate. Specifically, if the false rejection rate is greater than a threshold, the false rejection rate is reset or the number of received fingerprints N, K, and/or the minimum vector distance (Dmin) are decreased. That is, in some cases, if training is stopped, it may be desirable to start training again. For example, a configuration may have been specified that was more robust to outliers (as mentioned above). Still, the training may be stopped in a state with outliers. Thus, it would be desirable to detect when to re-start training and what to do when the training is re-started. In one example, the training may be re-started by tracking the false rejection rate and determining if the false rejection rate exceeds a threshold.

In some cases, the training may be re-started from the beginning (e.g., discarding previous information). In another configuration, the training may be re-started from a previous training position, such as rewinding the training to a specific point. The rewinding may reset or decrease values. In another configuration, the training may continue by pushing out the criteria to stop by increasing the N threshold or the T threshold to stop. In this example, the threshold may increase from <NUM> to <NUM>. The T threshold refers to the value of (<NUM>-<NUM>/N)k. The N threshold refers to the number of received fingerprints.

Aspects of this disclosure are not limited to a binary classification use case and are contemplated for other uses. For example, in a multi-class classifier there are typically a set of classifier boundaries. As an example, a multi-class classifier is a three-class classifier that classifies pedestrians, cars, and trucks. Aspects of the present disclosure may be used to vary and adapt the target margin from the off-line trained classifier boundaries. Furthermore, aspects of the present disclosure are not limited to fingers, fingerprints, and/or fingerprint readers. Of course, the aspects of the present disclosure are also contemplated for other uses for verifying an object (e.g., face, an iris or any other object) and/or training based on an object.

<FIG> shows a method <NUM> for online training of a linear classifier according to one aspect of the disclosure. A classifier determines a distance from one or more feature vectors of an object to a first predetermined decision boundary established during off-line training for the classifier, as shown in block <NUM>. The one or more examples are observed during the online training. The classifier updates a decision rule as a function of one or more distances, as shown in block <NUM>. Additionally, the classifier classifies a future example based on the updated decision rule, as shown in block <NUM>.

<FIG> shows a flow diagram <NUM> for online training of a linear classifier according to one aspect of the disclosure. As shown in <FIG>, at block <NUM> a predetermined decision boundary is established during off-line training. As previously discussed, the predetermined decision boundary may be established by using real and fake examples of an object that is to be classified. Furthermore, at block <NUM>, an object is received during online training. Additionally, at block <NUM>, the device determines whether the received object is a real object or a fake object based on the predetermined decision boundary.

In one configuration, if the object is fake, the received object is classified as fake (block <NUM>). Additionally, if the received object is real, the device determines a distance from a feature vector of the object to the predetermined decision boundary (block <NUM>). Furthermore, after determining the distance, the device may update a decision rule as a function of at least the distance (block <NUM>). Finally, future objects are classified based on the updated decision rule (block <NUM>).

In one configuration, a machine learning model is configured for determining a distance from one or more examples of an object, which is observed during the online training, to a first predetermined decision boundary established during off-line training for the classifier; updating a decision rule as a function of the distance; and classifying a future example based on the updated decision rule. The model includes a determining means, updating means, and/or classifying means. In one aspect, the determining means, updating means, and/or classifying means may be the general-purpose processor <NUM>, program memory associated with the general-purpose processor <NUM>, memory block <NUM>, local processing units <NUM>, and or the routing connection processing units <NUM> configured to perform the functions recited. In another configuration, the aforementioned means may be any module or any apparatus configured to perform the functions recited by the aforementioned means.

According to certain aspects of the present disclosure, each local processing unit <NUM> may be configured to determine parameters of the model based upon desired one or more functional features of the model, and develop the one or more functional features towards the desired functional features as the determined parameters are further adapted, tuned and updated.

The means may include various hardware and/or software component(s) and/or module(s), including, but not limited to, a circuit, an application specific integrated circuit (ASIC), or processor. Generally, where there are operations illustrated in the figures, those operations may have corresponding counterpart means-plus-function components with similar numbering.

Additionally, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory) and the like. Furthermore, "determining" may include resolving, selecting, choosing, establishing and the like.

The various illustrative logical blocks, modules and circuits described in connection with the present disclosure may be implemented or performed with a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array signal (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any commercially available processor, controller, microcontroller or state machine.

The steps of a method or algorithm described in connection with the present disclosure may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in any form of storage medium that is known in the art. Some examples of storage media that may be used include random access memory (RAM), read only memory (ROM), flash memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, a hard disk, a removable disk, a CD-ROM and so forth. A storage medium may be coupled to a processor such that the processor can read information from, and write information to, the storage medium.

The functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in hardware, an example hardware configuration may comprise a processing system in a device. The network adapter may be used to implement signal processing functions. For certain aspects, a user interface (e.g., keypad, display, mouse, joystick, etc.) may also be connected to the bus.

The processor may be responsible for managing the bus and general processing, including the execution of software stored on the machine-readable media. Machine-readable media may include, by way of example, random access memory (RAM), flash memory, read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable Read-only memory (EEPROM), registers, magnetic disks, optical disks, hard drives, or any other suitable storage medium, or any combination thereof. The computer-program product may comprise packaging materials.

In a hardware implementation, the machine-readable media may be part of the processing system separate from the processor. However, as those skilled in the art will readily appreciate, the machine-readable media, or any portion thereof, may be external to the processing system. By way of example, the machine-readable media may include a transmission line, a carrier wave modulated by data, and/or a computer product separate from the device, all which may be accessed by the processor through the bus interface. Although the various components discussed may be described as having a specific location, such as a local component, they may also be configured in various ways, such as certain components being configured as part of a distributed computing system.

The processing system may be configured as a general-purpose processing system with one or more microprocessors providing the processor functionality and external memory providing at least a portion of the machine-readable media, all linked together with other supporting circuitry through an external bus architecture. Alternatively, the processing system may comprise one or more neuromorphic processors for implementing the neuron models and models of neural systems described herein. As another alternative, the processing system may be implemented with an application specific integrated circuit (ASIC) with the processor, the bus interface, the user interface, supporting circuitry, and at least a portion of the machine-readable media integrated into a single chip, or with one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gated logic, discrete hardware components, or any other suitable circuitry, or any combination of circuits that can perform the various functionality described throughout this disclosure.

The machine-readable media may comprise a number of software modules. The software modules include instructions that, when executed by the processor, cause the processing system to perform various functions. Furthermore, it should be appreciated that aspects of the present disclosure result in improvements to the functioning of the processor, computer, machine, or other system implementing such aspects.

A storage medium may be any available medium that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. In addition, any connection is properly termed a computer-readable medium.

Claim 1:
A method (<NUM>) of online training of a classifier, the method (<NUM>) comprising:
receiving (<NUM>) an object;
determining (<NUM>, <NUM>) a distance from at least one feature vector of the object, which is received during the online training, to a first, predetermined, decision boundary (<NUM>, <NUM>, <NUM>) established during off-line training for the classifier, the first decision boundary defined between a real example vector and a fake example vector;
updating (<NUM>, <NUM>), based on one or more objects received during online training as positive examples that match the first decision boundary classified as respective one or more real objects and each having respective real example vectors , a decision rule as a function of at least the distance, the decision rule being a second decision boundary (<NUM>, <NUM>) established during the online training wherein updating the decision rule comprises reducing a margin between the at least one feature vector of the object and the second decision boundary; and
classifying (<NUM>, <NUM>) a future object as a real object based at least in part on the updated decision rule and the first, predetermined, decision boundary, wherein the classifying comprises classifying the future object as a real object if the feature vector distance of the future object to the first, predetermined, decision boundary is greater than (Dmin - Mmin) × (<NUM>-<NUM>/N)k, wherein Dmin is the closest distance of the real example vectors received during the online training to the first, predetermined, decision boundary (<NUM>, <NUM>, <NUM>), Mmin is a minimum margin from the real example vector that is closest to the decision boundary to the first predetermined decision boundary (<NUM>, <NUM>, <NUM>), N is the number of received examples, and k is a fixed parameter that controls the rate at which Dmin is moved to the real example vector that is closest to the first, predetermined, decision boundary.