SYNTHETIC MASKED BIOMETRIC SIGNATURES

Systems and methods for processing images of a person wearing a mask or where their face is otherwise partially covered. A first face image of a person is received, such as from a camera. A mask image is combined with the first face image to obtain a second face image. A first biometric signature is generated for the second face image and stored in associate with identification information for the person. The first biometric signature is then used to identify the person in subsequent images

BACKGROUND

Technical Field

The present disclosure relates to systems and methods for processing facial images.

Description of the Related Art

Some processor-based systems are configured to process images for recognizing objects, persons, places, animals, or other such subjects depicted in the images. Such systems face numerous obstacles regarding how images are collected and processed to recognize subjects in the images obtained. Some of these systems may process information related to previous images in order to recognize or identify the subject depicted in another image. However, it may be difficult for such systems to recognize a subject having a different appearance relative to a previously captured appearance of the subject.

DETAILED DESCRIPTION

FIG.1shows a system100for image-based authentication according to one or more embodiments. The system100is configured to recognize or identify persons based on an unobstructed facial image captured, the unobstructed facial image being one in which the subject is not wearing equipment that obstructs the face. The system100is also configured to recognize the subject in a facial image captured of the subject who is wearing equipment that obstructs the face based on the unobstructed facial image.

In a facial recognition system that processes images depicting faces, some systems may accurately recognize front views of faces, but may have trouble distinguishing between different modalities of the subject. Some systems may have difficulty distinguishing an unobstructed view of a face from a view of the same face wearing protective equipment. For example, an image may be captured of a subject at a first time and used to identify the subject. It may be problematic for the system to identify the subject if the subject has a different appearance due to equipment or accessories worn by the subject. Although it may be possible to capture images of the subject wearing various articles of equipment for use in identifying the subject, this may complicate the process of registering the identity of the subject. Moreover, it may be difficult to anticipate what equipment the subject may wear and numerous sessions may be necessary to complete the process to register each appearance. It is thus impractical to collect and process all possible appearances of a subject.

Determining what previous images to process or store is also a challenging problem. Overinclusion of image information may impede the processing efficiency of the system or occupy a significant amount of data storage space. On the other hand, underinclusion of image information may adversely affect the robustness and accuracy of the system. In systems in which an appropriate amount of image information is obtained for subject recognition, it may be difficult for the system to appropriately process different views or appearances of the same subject in recognition processing. Some systems are insufficiently configured to obtain and process different modalities of subjects.

References to the term “set” (e.g., “a set of items”), as used herein, unless otherwise noted or contradicted by context, is to be construed as a nonempty collection comprising one or more members or instances.

References to the term “subset” (e.g., “a subset of the set of items”), as used herein, unless otherwise noted or contradicted by context, is to be construed as a nonempty collection comprising one or more members or instances of a set or plurality of members or instances. Moreover, the term “subset,” as used herein, refers to a collection of one or more members or instances that are collectively smaller in number than the set or plurality of which the subset is comprised. For instance, a subset of a set of ten items will include less than ten items and at least one item.

References to the term “module,” as used herein, is to be construed as a collection of hardware configured to perform a set of particular computing functions within a system. The hardware of a module may include one or more processors that are specifically hardwired to perform one or more of the set of particular computing functions. A module may be a set of instructions that, as a result of execution by a processor, causes the processor and associated hardware to perform one or more of a set of particular functions.

With reference toFIG.1, the system100includes one or more processors102that cause the system100to perform one or more operations described herein. The system100receives a plurality of images104from a camera106and determines how to process individual images108of the plurality of images104received. The system100may process an image108to determine whether to store the image108or a portion thereof in data storage110for future subject recognition purposes. The system100may also determine how to evaluate the image108or a portion thereof to identify or recognize a subject in the image. A subject of the image108refers to a person or an object depicted therein. The present disclosure describes the subject as being a face of a person depicted in one or more images108of the plurality of images104. However, those of ordinary skill in the art will understand that the technology discussed herein may extend to other subjects, such as merchandise, packages, animals, and location, by way of non-limiting example.

The system100may include memory112storing a set of instructions114that, as a result of execution by the one or more processors102, cause the system100to perform as described herein. The memory112may include volatile memory (e.g., random-access memory) and/or non-volatile memory (e.g., read-only memory) for storing data and instructions. In some embodiments, the one or more processors102may include a device having hardware specifically configured to perform at least some of the operations described herein. For instance, the one or more processors102may include application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), system-on-a-chip (SoC), or other specialized or customizable computing hardware hardwired to perform at least some of the operations described herein.

The camera106may be part of the system100or may be considered to be a component separate from the system100. The camera106is electrically communicatively coupled to the system100and provides the plurality of images104as an input to the system100. The camera106may be directly coupled to the system100via a wired connection or may be remotely coupled to provide the plurality of images104to the system100, for example, over a wired or wireless communication network. In some embodiments, a plurality of cameras106coupled to the system100may each provide a plurality of images104to the system100. Although the plurality of images104are shown as being received from the camera106, the plurality of images104may be received from a source other than the camera106. For instance, the plurality of images104may be received over a network (e.g., local area network, wide area network) and/or via an intermediate device, such as a network router or a server. In some instances, the plurality of images104may be stored in memory at a first time and provided to the system100at a second time later than the first time.

The system100may also include a set of modules for performing various operations described herein. The system100may include a face detection module116for detecting one or more faces of persons depicted in an image108or the plurality of images104. The system100may include a tracking module118for tracking a face of a person between sequential images108of the plurality of images104. For example, the tracking module118may track a face of a particular person across sequential images108in video media. The system100may also include an identity learning module120that learns identities of persons depicted in an image108. In some embodiments, the identity learning module120causes the system100to store, in the data storage110, information regarding a plurality of modalities represented in different images108with an identity, as describe below. The system100may also include an identification module122that identifies a person represented in an image108based on biometric signature information stored in the data storage110. In some embodiments, the identification module122causes the system100to identify a person depicted in an image108based on a proximity threshold corresponding to a modality of the person's face in the image108. The system100may further include an image assessment module124that assesses various characteristics of face images captured to determine whether the characteristics of the face images are appropriate for identity processing, such as learning or updating information associated with an identity stored in the data storage110or appropriate to recognize the identity of a person in the face image.

The system100includes an image modification module126that modifies an image obtained from the camera106or stored in the data storage110to generate a modified image. The image modification module126may modify an image by combining the image with another image. For example, the image modification module126may superimpose or overlay one image over another image. The image modification module126may be configured to identify a certain region within an image to be modified and modify the region identified. The image modification module126may include or have access to a neural network that is trained to identify the certain region of the image.

The face detection module116, the tracking module118, the identity learning module120, and the identification module122are described as being distinct modules. In some embodiments, two or more of the modules may be combined into a single module that performs the corresponding operations for the respective modules without departing from the scope of the instant disclosure. Some or all of the face detection module116, the tracking module118, the identity learning module120, and the identification module122may operate according to machine learning principles by, for example, implementing neural network models or artificial intelligence models to perform their respective operations. As one example, the identity learning module120may be generated or trained via supervised or unsupervised learning principles to generate biometric signatures of a face image to evaluate modalities corresponding to the face images in view of a threshold. As another example, the identification module122may be generated or trained via supervised or unsupervised learning principles to identify a person in a face image based on a comparison of biometric signatures in view of a threshold determined based on an evaluation of the face image.

FIG.2shows an environment200in which the system100operates to generate biometric signatures of a face of a subject according to one or more embodiments. The system100may associate multiple modalities of a person with an identity stored in the data storage110. In the environment200, a camera202captures an image204that includes a face image206of a person208and that may include other features, such as a portion of the body of the person208or an ambient environment. The person208may be attempting to gain access a restricted physical area or gain access to restricted electronic data, for example.

The image204is provided to the system100, which processes the image204. The face detection module116may detect the face image206in the image204based on knowledge-based methods, feature invariant methods (e.g., feature extraction), template matching methods, or appearance-based training methods. The face detection module116may be trained or generated by receiving a set of training data and determining correlated relationships between face images and non-face images. The face detection module116may identify the face image206in the image204and extract or generate image data corresponding to the face image206.

The system100may then perform an assessment210of the face image206to determine whether characteristics of the face image206satisfy a set of criteria for processing face images. The system100performs the assessment210of the face image206based on a set of factors and generates image assessment information for the face image206based on a result of the assessment. The face image206may be evaluated based on image characteristics as well as content of the face in the face image206. The set of factors may include one or more of a size of the face image206, a pose of a face in the face image206, sharpness of the face image206, and contrast quality of the face image206. Further description of the assessment210and other analysis regarding the face image206is described elsewhere herein. Further description of various features herein are described in U.S. patent application Ser. No. 16/262,590, filed Jan. 30, 2019; and in U.S. Patent Application No. 62/830,331, filed Apr. 5, 2019, the entirety of which disclosures are incorporated herein by reference.

The assessment210includes determining whether a particular region of a face is present in the face image206. The particular region of the face image206detected by the system100may be a region including the mouth, a portion of the nose, and portions of the cheeks, as described herein. As a result of determining that the particular region is present in the face image206, the system100may identify the particular region. Identifying the particular region may include identifying coordinates within the image that define the particular region. The system100may include a neural network that is trained to identify the particular region of the face image206.

As a result of the face image206satisfying the assessment210, the system100may generate a biometric signature212of the face image206. In some embodiments, the biometric signature212is a face template having features corresponding to features of a subject's face. The biometric signature212may be the output of a neural network or other machine learning model implemented by the system100. The biometric signature212may be a multidimensional array or vector representative of features of the face depicted in the face image206. The multidimensional array or vector may include a plurality of values representative of the features, and may be representative of distinctions between features, such as distances between features or differences in image characteristics, such as contrast, sharpness, etc.

One example of a multidimensional vector corresponding to a biometric signature may be sets of values that respectively correspond to particular areas of a face depicted in a face image. The particular areas may be regions around each eye, the nose, the mouth, the chin, or portions of the foregoing features, such as corners of the mouth, the bridge of the nose, or the tip area of the nose. Each of the set of values may be representative of characteristics of a corresponding area of the face, such as sizes of the area or features therein or distances between features in each area. Another example of a multidimensional vector corresponding to a biometric signature may be sets of values corresponding to geometric features of a face depicted in a face image. Features, such as pupil centers, nose tip, and mouth corners, may be identified in the face image and distances and directions between the features may be determined. Such distances and directions may be included as sets of values comprising the multidimensional vector. These examples of multidimensional vectors and processes for generating multidimensional vectors are non-limiting and other examples of multidimensional vectors may apply to the current description without departing from the scope of the present disclosure.

In some embodiments, multidimensional vectors may be generated according to a machine learning model that is trained to generate an output using samples as input. The machine learning model may iteratively adjust weights and/or biases to minimize error in the output. The machine learning network may include a neural network having a plurality of layers, including one or more hidden layers, an input layer, and an output layer. The machine learning model may be trained according to supervised learning principles, unsupervised learning principles, or semi-supervised learning principles.

The system100may store the biometric signature212in data storage110for use in future subject identification. The biometric signature212may be associated in data storage110with an identity of the corresponding person208. For instance, the system100may store the biometric signature212in association with a unique identifier corresponding to the person (e.g., employee identification number, name, known traveler number, social security number). In situations in which a previously-generated identity is not stored in the data storage110, the system100may create a new identity with a unique identifier for the person. In some embodiments, the face image206may also be stored in association with the unique identifier.

The system100also obtains an image214from data storage110to be combined with the face image206. The image214is an image of equipment designed to be worn on the particular region of the user's face. For instance, the image214may be an image of personal protective equipment, such as a mask, respirator, goggles, shield, or other such face piece, that covers the particular region of the user's face in the face image206. The image214may be an image captured of the personal protective equipment while worn on the face of another person such that features of the image214indicate the presence of the particular region beneath. For the particular example of a face mask (e.g., N95 mask), the image214may be an image of the face mask worn on the face of another person. The image214may be an image selected from a plurality of other similar images based on facial characteristics of the face image206, such as a size of the nose, eyes, width of the face, or height of the face, by way of non-limiting example. The image214may be selected based on the similarity of the facial characteristics of the face image206to the other person wearing the equipment.

The system100performs a procedure216in which the face image206and the image214are combined or merged to generate a modified image218. The image214may be superimposed or overlaid on the particular region of the face image206identified in the assessment210. The image214may be modified to conform to the face shown in the face image206, such as by resizing, fitting, cropping, orienting, stretching, compressing, or otherwise manipulating the image214to cover the particular region and correspond to the features shown in the particular region identified in the face image206. Once the image214is modified, if appropriate, and superimposed on the face image206, an image of the face image206with the combined image214overlaid thereon is captured and the resulting modified image218is stored in memory of the system100(e.g., the data storage110). Then, the system110generates a biometric signature220of the modified image218, as described herein. The biometric signature220is stored in the data storage110in association with the identity of the person208. The biometric signature220may be stored as a supplemental or alternate signature for identifying or verifying the authenticity of the person208based on presentation of an image of the person208.

FIG.3shows a region of the face image206identified by the system100according to one or more embodiments. The region302identified may be covered or occluded when the person208wears certain protective equipment. The region302shown inFIG.3includes a mouth304, a portion of the nose, and portions of the cheeks that are covered when the person208wears a respiration mask (e.g., N95 mask) or a respirator. However, other regions of the face image206may be identified for other types of protective equipment. For example, a region around the eyes may be identified in connection with protective goggles to be worn by the person208.

The region302is identified in connection with identifying certain features in the face image206and determining various relationships between features identified. For the region302, the system100identifies a bridge306of the nose, which may be a portion of the nose between pupils or pupil centers308of the eyes in the face image206. The system100also identifies a tip310of the nose and determines a line312extending between the bridge306and the tip310. A position of an upper boundary314of the region302along the line312is determined based on a defined ratio, percentage, or other relationship. The position of the upper boundary314is one-third or 33% of the distance from the bridge306to the tip310along the line312inFIG.3; however, this ratio may be different in some circumstances.

The system100also identifies a chin316of the face image206and determines a position of a lower boundary318of the region302relative to the chin316. For instance, the lower boundary318may be positioned at or around the chin316. The lower boundary318extends in a direction parallel to the upper boundary314. The region302has a first dimension320(e.g., height) between the upper boundary314and the lower boundary318.

The system100determines positions of a first side boundary322and a second side boundary324of the region302based on a width of the face image206. The system100may determine that a second dimension326of the region302is a defined ratio or percentage of the width of the face image206. For instance, the system100may determine the width of the face in the face image206as being a widest portion of the face between cheeks328aand328b. The second dimension326may be determined as being, for example, 80% of the distance between a widest portion of the face at the cheeks328aand328b. The first side boundary322may be determined as being a line extending between the upper and lower boundaries314and318at a distance from the line312that is half of the second dimension326. The second side boundary324may be determined as being a line extending between the upper and lower boundaries314and318at a distance from the line312that is half of the second dimension326on a side of the line312opposite to the first side boundary322.

FIG.4Ashows an environment400in which the system100generates a modified image according to one or more embodiments. In particular, the system100obtains an image402from data storage110to be combined with the face image206. The image402obtained is an image of a respiration mask or a respirator designed to be worn over the region302of the face identified by the system100. The image402obtained may be selected from among a plurality of images based on characteristics of the face in the face image206. For example, the image402may be selected based on the width of the face, the length of the line312, or a distance between the nose tip310and the chin316, by way of non-limiting example. The system100combines the face image206and the image402to generate a modified image404, as shown inFIG.4B. The image402is superimposed or overlaid upon the region302identified in the face image206. In some implementations, the image402or portions thereof may be manipulated (e.g., resized, warped, distorted) to fit the image402to the face image206. The system100generates a biometric signature of the modified image404and stores the biometric signature in data storage110for use in identifying or authenticating an identity of a person at a later time.

In some embodiments, the system100may include or have access to a neural network that superimposes an appropriate image on the face image206. For instance, the system100may include a generative adversarial network (GAN) or a variational autoencoder (VAE) trained to identify the region302of the face image206and generate an image depicting protective equipment (e.g., a respiration mask, respirator) on the region302. Such neural networks may be trained via a discriminative neural network to generate new data instances that modify previously obtained images.

FIG.5shows a diagram500in which a person502wearing protective equipment is identified or authenticated by the system100according to one or more embodiments. A camera504captures an image506of the person502wearing protective equipment and a face image508of the person502is identified or extracted. The system100performs an assessment510of the face image508and determines that the face image508includes protective equipment512.

In some implementations, prior to performing the comparisons518, the face image508may be assessed in terms of various criteria to ensure that it is of sufficient quality to confidently determine a match. The assessment510may involve assessment of the size of the face image508, the pose or orientation of the face in the face image508, the sharpness of the face image508, and/or the contrast quality of the face image508. The system100may generate image assessment information based on the result of the assessment510and evaluate the image assessment information to determine whether to advance the face image508for further processing.

As a result of determining that the face image508includes the protective equipment512worn on the face of the person508, the system100uses a set of face image criteria516to authenticate the person502. The face image criteria516includes a set of criteria corresponding to the aforementioned set of factors—namely, criteria regarding one or more of the size of the face image508, the pose of the face in the face image508, the sharpness of the face image508, and the contrast quality of the face image508. The set of criteria may specify a different threshold or condition corresponding to each of the set of factors.

More particularly, the face image criteria516may include thresholds or criteria related to a minimum standard for number of pixels between center distances between the centers or pupils of eyes, a minimum standard for center pose quality of the face, a minimum standard for face sharpness quality, or a minimum standard for face contrast quality. These criteria may have stricter standards for assessing a face image508including protective equipment relative to standards that would be used for assessing a face image without protective equipment.

In practical application, as a result of a determination by the system100that the face image508is a sufficient size, a center pose quality, sharpness quality, and contrast quality and it is determined to be occluded by protective equipment, the system100will perform comparisons518between the face distance required against masked biometric signatures will be stricter (smaller) than the match against a full face biometric signature. This may help to facilitate a reduction in false negative rates and help to facilitate better matching rates for individuals with more prominent differentiating characteristics in eye region. If the

The system100generates a biometric signature514of the face image508. The system100performs signature comparisons518between the biometric signature514and one or more biometric signatures520stored in the data storage110. Some or all of the biometric signatures520stored in the data storage110may be based on modified face images generated by combining a face image with an image of protection equipment, as described herein. For each comparison518, the system100may compare distances between the biometric signature514and the stored biometric signatures520and determine a match based on a similarity of the signatures, such as a cosine similarity or other similarity measure.

The face image criteria516include different criteria or thresholds than would be used in authentication of a face without protective equipment or that includes a different type of protective equipment. As one example, the face image criteria516may include thresholds or criteria related to distances between biometric signatures that is stricter or smaller than face image criteria that would be used for a face image that does not include protective equipment.

For each comparison518, the system100may obtain a stored biometric signature520from among the stored biometric signatures in the data storage110and determine whether the stored biometric signature520matches the biometric signature514for the face image508being evaluated. A determination that the biometric signature514is a match for the stored biometric signature520may involve a determination that the biometric signature514satisfies a similarity criterion with respect to the stored biometric signature520. The system100may identify or authenticate the identity522of the person depicted in the face image508as corresponding to the identity associated with the stored biometric signature520based on the signature comparison518. If the biometric signature514is not a match for the stored biometric signature520, the system100may perform additional signature comparisons518with biometric signatures of the stored biometric signatures until the system100determines a match or is unable to match the biometric signature514with any of the stored biometric signatures.

FIG.6shows a diagram600in which the person502wearing protective equipment is identified or authenticated by the system100according to one or more embodiments. The diagram600includes several features that are substantially similar to the diagram500, so further description thereof is omitted for brevity. The camera504captures the image506of the person502wearing protective equipment. The system100may not detect whether the face image508includes the protective equipment512in the assessment510performed on the face image508in the diagram600. In the diagram600, the system100evaluates the face image508based on a set of face image criteria606used for assessing face images that are not occluded or that do not include the protective equipment512.

The system100may assess the face image508using the face image criteria606regarding one or more factors, such as the size of the face image508, the pose of the face in the face image508, the sharpness of the face image508, and the contrast quality of the face image508. The face image criteria606may include thresholds that are less strict than would be used to assess a face image detected as including the protective equipment512. As a result of determining that the face image508satisfies the minimum standards for evaluation, the system100generates the biometric signature514of the face image508and performs signatures comparisons602between the biometric signature and stored biometric signatures604stored in the data storage110. The biometric signatures604stored in the data storage110are biometric signatures of face images that do not include protective equipment.

The system100compares602the biometric signature514with the biometric signatures604and determines, at608, that the biometric signature514is not a match with any of the biometric signatures604. Thereafter the system100performs the signature comparisons518with biometric signatures610stored in the data storage using face image criteria516, as discussed with respect toFIG.5. In connection with or as a prerequisite to the signature comparisons518, the system100may assess characteristics of the face image508using the criteria factors discussed above to determine whether the face image508satisfies certain standards sufficient to authenticate the face image508using biometric signature comparison. Such criteria include factors regarding one or more of the size of the face image508, the pose of the face in the face image508, the sharpness of the face image508, and the contrast quality of the face image508.

The biometric signatures610are biometric signatures of face images that include protective equipment, such as respiration masks or respirators. The face image criteria516used to evaluate signature comparisons518between the biometric signature514and the biometric signatures610are different than the base image criteria606used to evaluate biometric signatures that do not include protective equipment. For example, the face image criteria516may be more strict or may have higher thresholds that are involved in evaluating the signature comparisons602. As a result of determining that the biometric signature514is a match with1of the biometric signatures610in view of the face image criteria516, the system522identifies or verifies the authenticity of the person502. Identification522of the person508may include providing information regarding an identity associated with the biometric signature of the biometric signatures610matching the biometric signature514.

FIG.7shows a method700generating a biometric signature of a modified image according to one or more embodiments. The method700is performed by the system100as described with respect to the environment200inFIG.2and elsewhere herein. The method700includes receiving702a face image of a person who is not wearing protective equipment. Receiving702the face image may include receiving an image, such as the image204ofFIG.2, and extracting or identifying a face image within the image. The face image may be received from a camera coupled to the system100or received from an external system requesting identification or authentication of a person shown in the face image. The face image, for example, may be received over one or more networks, such as the Internet or Local Access Network.

At704, the method700includes identifying a defined region of the face in the face image. The defined region identified in704is based on a type of protective equipment that the person may be expected to wear. Medical professionals may be expected to wear respiration masks or respirators, so the region identified ion704may be the region302. However, other regions may be identified in704based on different circumstances.

The method700also includes obtaining706an image of protective equipment from data storage to be combined with the face image received in702. The image obtained in706, for example, may be the image402of the mask ofFIG.4A. The method700also includes combining708the face image received in702and the protective equipment image obtained in706, as discussed with respect to216ofFIG.2to obtain the modified image218. Combining708includes overlaying or superimposing the image obtained in706over the region identified in704. In some embodiments, combining708the images may be performed by a neural network (e.g., GAN, VAE) that is trained to modify the face image to include a worn article of protective equipment.

The method700also includes generating710a biometric signature of the modified face image generated in708. The biometric signature may be generated in710by a neural network that is included in or accessible by the system100. The biometric signature generated in710may then be stored in the data storage110for identifying the person at a later time.

FIG.8shows a method800for authenticating or identifying a person depicted in a face image that includes protective equipment according to one or more embodiments. The method800is performed by the system100as described with respect toFIG.5. The method800includes receiving802a face image of the person who is wearing protective equipment, such as the image506. Receiving802may include extracting or identifying a face image in an image received. The face image may be received from a camera coupled to the system100or received from an external system requesting identification or authentication of a person shown in the face image. The face image, for example, may be received over one or more networks, such as the Internet or Local Access Network.

The method800also includes determining804whether the face image is a modal image. In particular, the system100determines whether an article of protective equipment (e.g., the protective equipment512) is worn on a face of a person depicted in the face image received in802. If the system100determines, in804, that the face image does not include protective equipment, method800continues by performing an unmasked authentication procedure in which a biometric signature of the face image is compared with stored biometric signatures of unmasked face images. On the other hand, if the system determines in804that the face image received in802is a modal image that includes protective equipment, the method800proceeds to assess808the face image.

Assessing the face image in808includes assessing various characteristics of the face image to determine whether the face image satisfies a set of criteria for biometric signature authentication. For example, as described with respect to the assessment510, assessing the face image in808may involve assessment of various factors, including the size of the face image, the pose or orientation of the face in the face image, the sharpness of the face image, and/or the contrast quality of the face image. The system100may generate image assessment information based on the result of the assessment in808and evaluate the image assessment information to determine whether to advance the face image508for further processing.

The system100may evaluate whether the face image satisfies minimum standards for one or more of the foregoing factors. The minimum standards for assessing the face image are different than the minimum standards for evaluating an image of an unmasked face due, e.g., to the reduced amount of information available in the face image. As a result of determining that the face image does not satisfy the set of criteria, the system100may discontinue the method800. If, on the other hand, the face image satisfies the set of criteria, the method800proceeds to generating810a biometric signature of the face image.

Generating810a biometric signature may include providing the face image to a neural network that is trained to generate biometric signatures of images of faces. In some embodiments, the neural network may be particularly trained to generate biometric signatures of images of faces wearing protective equipment. The neural network provides, in response to receiving the face image, a biometric signature.

Next, the method800involves comparing stored biometric signatures with the biometric signature generated in810. In particular, the stored biometric signatures include biometric signatures generated from images of faces wearing protective equipment. The system100obtains812a stored biometric signature from data storage and compares814the biometric signature obtained with the biometric signature generated in810. The comparison814may involve determining a difference between the biometric signatures—for example, Euclidean distances between the biometric signatures.

In816, the method800includes determining whether the biometric signature obtained from data storage satisfies modal image criteria with respect to the biometric signature generated in810. For instance, the modal image criteria may include a similarity criterion involving distance between the biometric signatures. The modal image criteria may include thresholds that are stricter (e.g., involve a smaller distance threshold) than criteria for evaluating a comparison between biometric signatures of images of faces that do not include protective equipment. If the system100determines, in816, that a result of the comparison814does not satisfy the modal image criteria, the system100returns to812and obtains another stored biometric signature for comparison with the biometric signature generated in810.

If the comparison is determined as satisfying the modal image criteria in816, the person in the face image is identified in818. For instance, the face image is determined as corresponding to an identity associated with the stored biometric signature obtained in812. The system100may identify the person imaged or evaluate other information associated with the identity—for example, whether the person has sufficient permissions to access a restricted area.

In some embodiments, the operations in812,814, and816may be performed differently. As one non-limiting example, obtaining812, comparing814, and determining816may include performing a database query or fuzzy hashing algorithm to identify candidate biometric signatures that are within a defined similarity threshold. In some embodiments, a neural network may be trained to evaluate the similarity of biometric signatures to determine a match.

FIG.9shows a method900for authenticating or identifying a person depicted in a face image that includes protective equipment according to one or more embodiments. The method900is performed by the system100as described with respect toFIG.6. Various operations described with respect to the method900are substantially similar to features described with respect to the method800, so further description thereof is omitted for brevity.

The method900includes receiving902a face image of the person who is wearing protective equipment, such as the image506. The method900also includes generating904a biometric signature of the face image received and comparing906the biometric signature with a plurality of stored biometric signatures. For example, the stored biometric signatures include biometric signatures generated using images of faces that do not include protective equipment, such as the stored biometric signatures604described with respect toFIG.6. The system100may compare906the signature generated in904with the plurality of stored biometric signatures. At908, the method900includes determining whether a result of the comparison in906satisfies a set of standard criteria for determining a match between biometric signatures, such as whether similarity between a stored biometric signature and the generated biometric signature is within a certain numerical range. If so, the method900proceeds to identify or authenticate910the identity of the person depicted in the image.

If, on the other hand, the system100determines that the comparisons do not satisfy the standard criteria, the method900proceeds to compare912the biometric signature generated in904with a second set of stored biometric signatures. The stored biometric signatures in912include biometric signatures generated using images of faces that include protective equipment, such as the stored biometric signatures610discussed with respect toFIG.6. At914, the method900includes determining whether result of the comparison in912satisfies a set of modal criteria for determining a match between biometric signatures, such as whether similarity between a stored biometric signature and the generated biometric signature is within a certain numerical range. The set of modal criteria used in914may involve stricter standards than the set of standard criteria involved in908—for example, the set of modal criteria may specify a smaller distance threshold or range than the set of standard criteria.

If it is determined in914that a result of the comparison in912satisfies the modal criteria, then the method900proceeds to identify or authenticate916the identity of the person depicted in the image. For example, the face image is determined as corresponding to an identity of the person associated with the stored biometric signature that satisfies the set of modal criteria with respect to the biometric signature generated in904. If none of the second set of stored biometric signatures satisfied the modal criteria in914, then the system100determines the occurrence of an authentication failure918. That is, the biometric signature generated in904does not correspond to any known identities associated with the first set of biometric signatures in906or the second set of biometric signatures in912.

The method900may also include assessing the various characteristics of the face image received in902relative to various criteria to ensure that characteristics of the face image are sufficient to confidently determine a match. Assessing the face image may involve, as a condition precedent to comparing the biometric signature generated in904with stored biometric signatures, verifying that the size of the face image, the pose of the face in the face image, the sharpness of the face image, and the contrast quality of the face image are sufficient for evaluation of a biometric signature. The set of criteria may include a first set of thresholds for evaluating the face image prior to comparison with the first set of biometric signatures in906. The set of criteria may also include a second set of thresholds for evaluating the face image prior to comparison with the second set of biometric signatures in912. The second set of thresholds include different thresholds than the first set of thresholds. For example, the second set of thresholds may involve smaller numerical thresholds or ranges for the characteristics of the face image. Various operations involved in the method900may be performed using or in connection with a neural network.

The various embodiments described above can be combined or modified to provide further embodiments. These and other changes can be made to the embodiments in light of the above-detailed description. In general, in the following claims, the terms used should not be construed to limit the claims to the specific embodiments disclosed in the specification and the claims, but should be construed to include all possible embodiments along with the full scope of equivalents to which such claims are entitled. Accordingly, the claims are not limited by the disclosure.

This application claims the benefit of priority to U.S. Provisional Application No. 63/008,526, filed Apr. 10, 2020, which application is hereby incorporated by reference in its entirety.