TECHNIQUES FOR TRAINING IDENTITY-ROBUST MACHINE LEARNING MODELS

In various embodiments, a model trainer application trains a machine learning model with improved identity robustness. The model trainer application first processes images of faces using a trained face recognition model to generate a proxy representation of an identity of the individual in each image. Representations of individuals with similar faces lie in the same neighborhoods within a proxy identity space. The model trainer application trains a machine learning model to perform a task relating to faces while considering the accuracy of each identity proxy neighborhood. The model trainer assigns different weights to each image sample in a neighborhood based on the number of samples with the same output class in that neighborhood. The assigned weights can then be used to compute a relatively unbiased identity loss function that is used to train the machine learning model to perform the task relating to faces while being robust to identity features.

BACKGROUND

Field of the Various Embodiments

The contemplated embodiments relate generally to computer science, artificial intelligence (AI), and machine learning and, more specifically, to techniques for training identity-robust machine learning models.

Description of the Related Art

Machine learning can be used to discover trends, patterns, relationships, and/or other attributes related to large sets of complex, interconnected, and/or multidimensional data. To glean insights from large data sets, regression models, artificial neural networks, support vector machines, decision trees, naïve Bayes classifiers, and/or other types of machine learning models can be trained using input-output pairs in the data. In turn, the trained machine learning models can be used to guide decisions and/or perform actions related to the data and/or other similar data.

Machine learning models can be trained to perform tasks related to human faces, including regression tasks such as estimating a facial pose or detecting facial landmarks, classification tasks such as facial expression classification, and generative tasks such as avatar creation. For example, facial expression classification identifies human emotions based on facial movements and expressions, such as eye and mouth movements, to infer emotions, such as sadness, happiness, anger, and surprise. To train a machine learning model to perform a face related task, a dataset that includes images of faces can be used to compute losses between outputs of the machine learning model given the images as inputs and ground truth data indicating expected outputs. Losses for all faces in the dataset can be aggregated equally and used to update parameters of the machine learning model. An optimizer minimizes the aggregated loss over a number of training iterations. The trained machine learning model can then be used in various applications, such as to monitor the drivers of vehicles or to monitor consumer reactions.

One drawback of conventional approaches for training a machine learning model to perform a task relating to faces is that the machine learning model can learn to rely on irrelevant and spurious features when performing the task. For example, two machine learning models trained to perform a face-related task can have similar overall performance but different levels of performance across different individuals. The disparity in performance can be due to bias in the training data used to train the two machine learning models. Bias can occur when the number of data points used as different categories of output classes in the training data used to train a machine learning model is not equal or balanced. For example, if person 1 smiles 90% of the time, and person 2 smiles 10% of the time, a machine learning model that is trained to classify persons as smiling or not smiling could associate the facial features of person 1 with the smiling class. Thereafter, the trained machine learning model can always classify images of person 1 as smiling because of the identity of person 1 and not the facial expressions of person 1.

As the forgoing illustrates, what is needed in the art are more effective techniques for training machine learning models.

SUMMARY

One embodiment of the present disclosure sets forth a computer-implemented method for training a machine learning model. The method includes, for each image included a plurality of images, generating a representation of a face within the image. The method further includes, for each image included the plurality of images, computing a weight based on the representation generated for the image and at least one other representation generated for at least one other image included in the plurality of images. In addition, the method includes performing one or more operations to train the machine learning model based on at least the weights to generate a trained machine learning model.

Other embodiments of the present disclosure include, without limitation, one or more computer-readable media including instructions for performing one or more aspects of the disclosed techniques as well as one or more computing systems for performing one or more aspects of the disclosed techniques.

At least one technical advantage of the disclosed techniques relative to the prior art is that, with the disclosed techniques, machine learning models can be trained to perform tasks relating to faces without relying on facial identity features, thereby improving robustness of the machine learning models. In addition, the disclosed techniques do not require balanced data points to train robust machine learning models for different output classes. These technical advantages represent one or more technological improvements over prior art approaches.

DETAILED DESCRIPTION

As described, in conventional approaches for training a machine learning model to perform a task relating to faces, a dataset that includes images of faces can be used to compute losses between outputs of the machine learning model given the images as inputs and ground truth data indicating expected outputs. Losses for all faces in the dataset can then be aggregated equally and used to update parameters of the machine learning model, and an optimizer can minimize the aggregated loss over a number of training iterations. However, during such training, the machine learning model can learn to rely on irrelevant and spurious features when performing the task. For example, if person 1 smiles 90% of the time, and person 2 smiles 10% of the time, a machine learning model that is trained to classify persons as smiling or not smiling could associate the facial features of person 1 with the smiling class. Thereafter, the trained machine learning model can always classify images of person 1 as smiling because of the identity of person 1 and not the facial expressions of person 1.

The disclosed techniques improve the identity-robustness of machine learning models. In some embodiments, a model trainer application first processes images of faces using a trained face recognition model to generate a proxy representation of an identity of the individual in each image. Representations of individuals with similar facial features lie in the same neighborhoods within a proxy identity space. A neighborhood can be defined with a hard threshold or a soft threshold. A hard threshold sets a radius of the neighborhood. A soft threshold is defined based on a distance metric, such as cosine distance. Instead of excluding representations of individuals outside a predefined distance, in some embodiments, a soft threshold can emphasize representations of individuals with closer distances and de-emphasize representations of individuals with further distances. The model trainer application trains a machine learning model to perform a task relating to faces while considering the accuracy of each identity proxy neighborhood. Instead of training the machine learning model to perform well on average, the model trainer assigns different weights to each image sample in a neighborhood based on the number of samples with the same output class in that neighborhood. The assigned weights can then be used to compute an unbiased identity loss function that is used to train the machine learning model to perform the task relating to faces while being robust to identity features.

Advantageously, the disclosed techniques address various limitations of conventional approaches for training machine learning models. More specifically, with the disclosed techniques, machine learning models can be trained to perform tasks relating to faces without relying on facial identity features, thereby improving robustness of the machine learning models. In addition, the disclosed techniques do not require balanced data points to train robust machine learning models for different output classes. These technical advantages represent one or more technological improvements over prior art approaches.

System Overview

FIG.1illustrates a block diagram of a computer-based system100configured to implement one or more aspects of the various embodiments. As shown, the system100includes a machine learning server110, a data store120, and a computing device140in communication over a network130, which can be a wide area network (WAN) such as the Internet, a local area network (LAN), a cellular network, and/or any other suitable network.

The machine learning server110includes, without limitation, processor(s)112and a memory114. The processor(s)112receive user input from input devices, such as a keyboard or a mouse. In operation, the one or more processors112may include one or more primary processors that control and coordinate the operations of the other system components within the machine learning server110. In particular, the processor(s)112can issue commands that control the operation of one or more graphics processing units (GPUs) (not shown) and/or other parallel processing circuitry (e.g., parallel processing units, deep learning accelerators, etc.) that incorporates circuitry optimized for graphics and video processing, including, for example, video output circuitry. The GPU(s) can deliver pixels to a display device that can be any conventional cathode ray tube, liquid crystal display, light-emitting diode display, and/or the like.

The system memory114of the machine learning server110stores content, such as software applications and data, for use by the processor(s)112and the GPU(s) and/or other processing units. The system memory114can be any type of memory capable of storing data and software applications, such as a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash ROM), or any suitable combination of the foregoing. In some embodiments, a storage (not shown) can supplement or replace the system memory114. The storage can include any number and type of external memories that are accessible to processor112and/or the GPU. For example, and without limitation, the storage can include a Secure Digital Card, an external Flash memory, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, and/or any suitable combination of the foregoing.

As also shown, memory114includes a model trainer116. Model trainer116is configured to train a classifier148that can then be deployed to any suitable application, such as application146that executes on a computing device140. The operations performed by model trainer116when training classifier148are described in greater detail below in conjunction withFIG.3. In some embodiments, model trainer116can dynamically adjust training parameters and methodologies by incorporating a feedback loop that leverages real-time analysis of any performance metric, such as precision, recall, and loss functions. Model trainer116can also make adjustments to optimize outputs and learned outcomes. These adjustments can include, without limitation, modifications to learning rates, model architectures, and data processing techniques. In some embodiments, model trainer116uses one or more data preprocessors that address common issues such as imbalanced datasets, missing values, and noise, thereby ensuring that the data fed into the model is clean, relevant, and representative of the problem space. In various embodiments, model trainer116uses data augmentation techniques, which artificially expand the training dataset to improve the generalization capabilities of the model, and tailored adjustments to the data.

The machine learning server110shown herein is for illustrative purposes only, and variations and modifications are possible without departing from the scope of the present disclosure. For example, the number of processors112, the number of GPUs and/or other processing unit types, the number of system memories114, and/or the number of applications included in the system memory114can be modified as desired. Further, the connection topology between the various units inFIG.1can be modified as desired. In some embodiments, any combination of the processor(s)112, the system memory114, and/or GPU(s) can be included in and/or replaced with any type of virtual computing system, distributed computing system, and/or cloud computing environment, such as a public, private, or a hybrid cloud system.

The computing device140includes, without limitation, processor(s)142and a memory144. Processor(s)142receive user input from input devices, such as a keyboard or a mouse. Similar to processor(s)112of machine learning server110, in some embodiments, processor(s)142may include one or more primary processors that control and coordinate the operations of the other system components within the computing device140. In particular, the processor(s)142can issue commands that control the operation of one or more graphics processing units (GPUs) (not shown) and/or other parallel processing circuitry (e.g., parallel processing units, deep learning accelerators, etc.) that incorporates circuitry optimized for graphics and video processing, including, for example, video output circuitry. The GPU(s) can deliver pixels to a display device that can be any conventional cathode ray tube, liquid crystal display, light-emitting diode display, and/or the like.

Similar to system memory114of machine learning server110, system memory144of computing device140stores content, such as software applications and data, for use by the processor(s)142and the GPU(s) and/or other processing units. The system memory144can be any type of memory capable of storing data and software applications, such as a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash ROM), or any suitable combination of the foregoing. In some embodiments, a storage (not shown) can supplement or replace the system memory144. The storage can include any number and type of external memories that are accessible to processor142and/or the GPU. For example, and without limitation, the storage can include a Secure Digital Card, an external Flash memory, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, and/or any suitable combination of the foregoing.

As also shown, system memory144includes application146that uses trained classifier148to generate classifications for a face related task. In some embodiments, an input image can be provided to application146via a user interface or in any other suitable manner. In such cases, application146can apply the trained classifier148on the input image to generate a classification, which can be directly output by application146or used in any technically feasible manner by application146. Trained classifier148can be any type of technically-feasible machine learning model. For example, in various embodiments, trained classifier148can be Convolutional Neural network, a Vision Transformer, a diffusion model, a support vector machine (SVM), etc. The operations that can be performed by application146are described in greater detail below in conjunction withFIG.4.

Data store120provides non-volatile storage for applications and data in machine learning server110and computing device140. For example, and without limitation, training data, trained (or deployed) machine learning models and/or application data, including the CAD data generator148, may be stored in the data store120. In some embodiments, data store120may include fixed or removable hard disk drives, flash memory devices, and CD-ROM (compact disc read-only-memory), DVD-ROM (digital versatile disc-ROM), Blu-ray, HD-DVD (high definition DVD), or other magnetic, optical, or solid state storage devices. Data store120can be a network attached storage (NAS) and/or a storage area-network (SAN). Although shown as accessible over network130, in various embodiments, the machine learning server110or computing device140can include the data store120.

FIG.2is a block diagram illustrating a computing device200configured to implement one or more aspects of the various embodiments. Computing device200may be any type of computing device, including, without limitation, a server machine, a server platform, a desktop machine, a laptop machine, a hand-held/mobile device, a digital kiosk, an in-vehicle infotainment system, and/or a wearable device. In some embodiments, computing device200is a server machine operating in a data center or a cloud computing environment that provides scalable computing resources as a service over a network.

As shown, the computing device200includes, without limitation, the processor(s)202and a system memory(ies)114coupled to a parallel processing subsystem212via a memory bridge214and a communication path213. Memory bridge214is further coupled to an I/O (input/output) bridge220via a communication path207, and I/O bridge220is, in turn, coupled to a switch226.

In various embodiments, I/O bridge220is configured to receive user input information from optional input devices218, such as a keyboard, mouse, touch screen, sensor data analysis (e.g., evaluating gestures, speech, or other information about one or more uses in a field of view or sensory field of one or more sensors), and/or the like, and forward the input information to the processor(s)202for processing. In some embodiments, the computing device200may be a server machine in a cloud computing environment. In such embodiments, computing device200may not include input devices218, but may receive equivalent input information by receiving commands (e.g., responsive to one or more inputs from a remote computing device) in the form of messages transmitted over a network and received via the network adapter230. In some embodiments, switch226is configured to provide connections between I/O bridge220and other components of the computing device200, such as a network adapter230and various add-in cards224and228.

In some embodiments, I/O bridge220is coupled to a system disk222that may be configured to store content and applications and data for use by processor(s)202and parallel processing subsystem212. In one embodiment, system disk222provides non-volatile storage for applications and data and may include fixed or removable hard disk drives, flash memory devices, and CD-ROM (compact disc read-only-memory), DVD-ROM (digital versatile disc-ROM), Blu-ray, HD-DVD (high-definition DVD), or other magnetic, optical, or solid state storage devices. In various embodiments, other components, such as universal serial bus or other port connections, compact disc drives, digital versatile disc drives, film recording devices, and the like, may be connected to I/O bridge220as well.

In various embodiments, memory bridge214may be a Northbridge chip, and I/O bridge220may be a Southbridge chip. In addition, communication paths207and213, as well as other communication paths within computing device200, may be implemented using any technically suitable protocols, including, without limitation, AGP (Accelerated Graphics Port), HyperTransport, or any other bus or point-to-point communication protocol known in the art.

In some embodiments, parallel processing subsystem212comprises a graphics subsystem that delivers pixels to an optional display device216that may be any conventional cathode ray tube, liquid crystal display, light-emitting diode display, and/or the like. In such embodiments, the parallel processing subsystem212may incorporate circuitry optimized for graphics and video processing, including, for example, video output circuitry. Such circuitry may be incorporated across one or more parallel processing units (PPUs), also referred to herein as parallel processors, included within the parallel processing subsystem212.

In some embodiments, the parallel processing subsystem212incorporates circuitry optimized (e.g., that undergoes optimization) for general purpose and/or compute processing. Again, such circuitry may be incorporated across one or more PPUs included within parallel processing subsystem212that are configured to perform such general purpose and/or compute operations. In yet other embodiments, the one or more PPUs included within parallel processing subsystem212may be configured to perform graphics processing, general purpose processing, and/or compute processing operations. System memory114includes at least one device driver configured to manage the processing operations of the one or more PPUs within parallel processing subsystem212. In addition, system memory114includes model trainer116which trains a relatively robust machine learning model by assigning different weights to each sample in the training data based on the number of samples with similar output class in a specific neighborhood, as discussed in greater detail below in conjunction withFIGS.3and5. Model trainer116defines neighborhoods using similarity of each sample to the other samples.

In various embodiments, parallel processing subsystem212may be integrated with one or more of the other elements ofFIG.2to form a single system. For example, parallel processing subsystem212may be integrated with processor202and other connection circuitry on a single chip to form a system on a chip (SoC).

In some embodiments, communication path213is a PCI Express link, in which dedicated lanes are allocated to each PPU. Other communication paths may also be used. The PPU advantageously implements a highly parallel processing architecture, and the PPU may be provided with any amount of local parallel processing memory (PP memory).

It will be appreciated that the system shown herein is illustrative and that variations and modifications are possible. The connection topology, including the number and arrangement of bridges, the number of CPUs202, and the number of parallel processing subsystems212, may be modified as desired. For example, in some embodiments, system memory114could be connected to the processor(s)202directly rather than through memory bridge214, and other devices may communicate with system memory114via memory bridge214and processor202. In other embodiments, parallel processing subsystem212may be connected to I/O bridge220or directly to processor202, rather than to memory bridge214. In still other embodiments, I/O bridge220and memory bridge214may be integrated into a single chip instead of existing as one or more discrete devices. In certain embodiments, one or more components shown inFIG.2may not be present. For example, switch226could be eliminated, and network adapter230and add-in cards224,228would connect directly to I/O bridge220. Lastly, in certain embodiments, one or more components shown inFIG.2may be implemented as virtualized resources in a virtual computing environment, such as a cloud computing environment. In particular, the parallel processing subsystem212may be implemented as a virtualized parallel processing subsystem in at least one embodiment. For example, the parallel processing subsystem212may be implemented as a virtual graphics processing unit(s) (vGPU(s)) that renders graphics on a virtual machine(s) (VM(s)) executing on a server machine(s) whose GPU(s) and other physical resources are shared across one or more VMs.

Training Identity-Robust Machine Learning Models

FIG.3is a more detailed illustration of model trainer116ofFIG.1, according to various embodiments. As shown, model trainer116includes a face recognition model304, a conditional inverse density (CID) weighting module308, and a training module312that includes a classifier314. Classifier314is a machine learning model that model trainer116can train to generate trained classifier148that is able to perform one or more classification tasks relating to faces, such as recognizing different classes of facial emotions in input images, classifying if mouths are open or closed in input images, or the like. Any technically feasible type of machine learning model, such as an artificial neural network or a SVM, can be used as classifier314in some embodiments.

In operation, model trainer116receives images that include faces302(also referred to herein as facial images302). Given such inputs, model trainer116trains a classifier314using facial images302as training data and a conditional inverse density weighting for each of a number of classes, normalized for all classes, to generate a trained classifier148that performs relatively robustly across facial images of different people for a classification task, such as facial expression classification, mouth slightly open classification, or face shape classification.

More formally, facial images302can be a dataset D={X×Y}={(xi, yi)}i=1|D|with size n=|D|, and a total number of output classes C, e.g., |Y|=C·Dy={(xi, yi)|yi=y, i∈[1, . . . , |D|]} represents sample images (also referred to herein as “samples”) in facial images302whose task label is y∈Y. gi∈G denotes the identity and group that sample i belongs to, across which performance disparity should be mitigated.

Face recognition model304is a machine learning model trained to recognize faces in images, and model trainer applies face recognition model304to extract representations of the identities of faces in facial images302. Illustratively, model trainer116uses face recognition model304to convert facial images302into proxy identity representations306, which can, in some embodiments, be vectors of features that act as noisy proxies of the identities of faces in facial image302. Since group and identity labels G of facial images302may not be available during training of classifier314, the proxy identity representations {zi}i=1|D|extracted from face recognition model304are provided as proxies for determining group and identity of faces within facial images302. Face recognition model304can be any technically feasible facial recognition model, such as DeepFace or OpenFace.

CID weighting module308receives proxy identity representations306and generates weights310that are applied by training module312during training of classifier314. In some embodiments, CID weighting module308uses a sample-weighting scheme based on the CID of each sample in a proxy identity space to generate weights310. Doing so permits identity-related content and non-identity related content within facial images302to be disentangled, without requiring explicit information about the identities of faces in facial images302. In some embodiments, CID weighting module308can generate, for each facial image302, a weight that is computed as an exponential of similarity of that facial image302to other facial images302, normalized based on the number of classes to be predicted by trained classifier148. The similarities can be computed using any technically feasible metric, such as cosine distance. Such a weighting is also referred to herein as a conditional inverse density weighting for each class, normalized for all classes.

More specifically, in some embodiments, CID weighting module308can create a batch-wise scheme such that samples in each batch are conditioned on a task label. In such cases, CID weighting module308generates a constraint set Dywith samples having the same task label in each batch B, e.g., Byi(thus, conditioned on task label). CID weighting module308then computes sample weight piτaccording to equation (1). In equation (1), the weight for a sample is computed as an exponential of similarity of the same to other samples, normalized based on the number of classes.

where the numerator is the exponential of the inner product of the proxy identity representations ziof sample (xi, yi), and the denominator aggregates the exponential pairwise similarities of proxy identity representations between sample (xi, yi) and Byiin a proxy identity neighborhood. The regularizer hyperparameter τ measures the proximity and magnitude of the neighborhood.

In equation (1), piτ∈(0,1] represents the importance of the sample (xi, yi) in a local neighborhood within the proxy identity space and τ controls the skewness of the exponential function, which influences the size of the local neighborhood. Even though the constraint set is defined in Byi, the regularizer hyperparameter τ in the exponential function encourages the denominator to focus on the local neighbors of (xi, yi) that share the same facial features. In some embodiments, the size of the local neighborhood can be determined by a predefined threshold. In some other embodiments, there can be different neighborhood sizes that better approximate different identities and group memberships.

The fewer the samples in the local neighborhood, the higher the piτ. Hence, piτis inversely proportional to the class-conditional sample density in the local neighborhood and emphasizes rare samples within each output class. For example, if a sample lies in a denser neighborhood, e.g., has more close neighbors, the piτof the sample will be smaller than the piτof a sample with less close neighbors. In some embodiments, all samples within the same output class are weighted uniformly and based on the inverse of the sample frequency.

Training module312receives weights310and facial images302that are used to train classifier314. Given such inputs, training module312trains classifier314to generate trained classifier148. Trained classifier148is a trained machine learning model, such as an artificial neural network or a SVM, that is trained to perform a classification task, such as recognizing different classes of facial emotions in input facial images, classifying if mouths are open or closed in input images, or the like. Any technically feasible type of machine learning model can be trained as trained classifier148in some embodiments, and training module312can use any suitable training technique to train classifier314, such as backpropagation with gradient descent or a variation thereof. During the training, model trainer116can use a loss function in which the loss computed for each facial image302in the training data set is weighted according to the weight generated for that facial image302by CID weighting module308. In such cases, after facial images302are input into classifier314to generate outputs, model trainer116can compute a loss for each facial image302based on a comparison between (1) the output for that facial image302, and (2) a ground truth classification. Then, model trainer116can apply the weight computed by CID weighting module308for each facial image302to the loss computed for that facial image302to compute a weighted average of the losses, and model trainer116can update parameters of classifier314based on the weighted average of losses. The foregoing process can be repeated for a number of iterations, until a stopping condition (e.g., after a predefined number of iterations have been performed or the weighted average of losses does not improve by more than a threshold amount) is satisfied, to generate trained classifier148. It should be noted that, rather than computing a per-image accuracy of outputs of classifier314, such a loss function permits model trainer116to compute the accuracy of outputs of classifier314in different regions of the proxy identity space and evaluate the accuracy across neighborhoods of the proxy identity space, thereby providing identity-robustness to trained classifier314. In some other embodiments, neighborhoods can be defined with a hard threshold that sets a predefined radius of each neighborhood, as opposed to the soft threshold neighborhoods of equation (1). Hard thresholds can exclude representations of individuals outside a predefined distance, whereas soft thresholds can emphasize representations of individuals with closer distances and de-emphasize representations of individuals with further distances.

More specifically, in some embodiments, during training and after sample weights piτ310have been computed, training module312minimizes the objective function in equation (2) such that piτis normalized using Zpito equalize the total contribution of each output class. The objective function can be defined as a min-max form, which improves the performance of classifier314on the least accurate areas of the proxy identity space.

piτis the class-level normalization parameter to guarantee each output class contributes equally. The piτcomputed according to equation (1) is the maximum value for the constraint in equation (2) that is imposed on the pairwise similarity of proxy identity representations, leveraging the proxy neighborhood structure associated with each sample. More specifically, for ∀(xi, yi)˜D, pi=(pi1, . . . , pii, . . . ,

refers to the weight assigned to each sample based on

and satisfies

={Σjpij=1, pij≥0}. The Kullback-Leibler (KL) divergence regularizer Σjpijlog(|Dyi|pij) between the uniform distribution 1/|Dyi| and the pairwise weights piencourages the model to focus on the local neighborhood.

In some embodiments, training module312uses the performance of classifier314across different local neighborhoods in the proxy identity space to estimate disparity across identities and groups. In such cases, training module312can fine-tune the regularizer hyper-parameter τ for exploring different neighborhood sizes and therefore exploring different density estimations of disparity across identities and groups during training of classifier314.

Algorithm 1 describes steps that model trainer116can perform to train classifier314.

FIG.4is a more detailed illustration of application146ofFIG.1, according to various embodiments. As shown, application146includes trained classifier148. In operation, application146receives an input image402that includes a face and executes trained classifier148on input image402to generate an output class404. As described, trained classifier148is a trained machine learning model, such as an artificial neural network or a SVM, that is trained to perform a classification task, such as recognizing different classes of facial emotions in input facial images, classifying if mouths are open or closed in input facial images, or the like. Any technically feasible type of machine learning model can be trained according to techniques disclosed herein to generate trained classifier148.

Output class404is a specific category or label that trained classifier148predicts for a given input image402. Output class404can be an output to any classification task, such as a multi-label classification task with different classes of facial emotions (e.g., happy, sad, angry, and disgusted). In some embodiments, output class404can be an output to a binary classification task, such as classifying if a mouth is slightly open, or the mouth is closed. Although application146is shown as outputting output class404, in some embodiments, an application can use outputs of a classifier that is trained according to techniques disclosed herein in any technically feasible manner, such as to generate other outputs.

FIG.5is a flow diagram of method steps for training a classifier, according to various embodiments. Although the method steps are described in conjunction with the systems ofFIGS.1-4, persons skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the various embodiments.

As shown, a method500begins at step502, where model trainer116receives facial images302for training classifier314. In some embodiments, model trainer116receives facial images302from a storage system (e.g., data store120).

At step504, model trainer116executes trained face recognition model304on the facial images302to determine proxy identity representations306of faces in the facial images302. Since group and identity labels G of facial images302may not be available for training, the proxy identity representations306extracted by the face recognition model304are provided as proxies for determining group and identity of faces within facial images302. In some embodiments, face recognition model304can be any technically feasible facial recognition model, such as DeepFace or OpenFace.

At step506, model trainer116assigns a weight to each facial image302based on a number of other images in a specific class that are in a neighborhood of the image defined by the proxy identity representations306. In some embodiments, CID weighting module308uses a sample-weighting scheme based on the CID of each sample in the proxy identity space. As described, in some embodiments, CID weighting module308can assign to each facial image302, a weight that is computed as an exponential of similarity of that facial image302to other facial images302, normalized based on the number of classes. In such cases, the similarities can be computed using any technically feasible metric, such as cosine distance. More specifically, in some embodiments, CID weighting module308can compute a sample weight piτfor each facial image302according to equation (1) that represents the importance of the sample image (xi, yi) in a local neighborhood of the proxy identity space and τ controls the skewness of the exponential function that influences the size of the local neighborhood. In some embodiments, the size of the local neighborhood can be determined by a predefined threshold. In some other embodiments, there can be different neighborhood sizes that better approximate different identities and group memberships, with the sample weight being inversely proportional to the class-conditional sample density in the local neighborhood so as to emphasize rare samples within each output class, as described above in conjunction withFIG.3.

At step508, model trainer116trains classifier314with the facial images302and assigned weights310. Model trainer116trains classifier314to generate trained classifier148that can perform relatively robustly across facial images302of different people for a classification task, such as facial expression classification, mouth slightly open classification, or face shape classification. Training module312can use any suitable training technique to train classifier314, such as backpropagation with gradient descent or a variation thereof. During such training, model trainer116can use a loss function that includes a weighted average of losses computed for facial images302in which the loss computed for each facial image302is weighted according to the weight assigned at step506, as described above in conjunction withFIG.3. In some embodiments, training module312can minimize the objective function according to equation (2) such that sample weight piτis normalized using Zpito equalize the total contribution of each output class.

FIG.6is a flow diagram of method steps for executing application146, according to various embodiments. Although the method steps are described in conjunction with the systems ofFIGS.1-4, persons skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the various embodiments.

As shown, a method600begins at step602, where application146receives an input image (e.g., image402) that includes a face. In some embodiments, application146receives input images402from a storage system (e.g., data store120).

At step604, application146executes trained classifier148on the input image to predict an output class (e.g., output class404). In some embodiments, classifier148is a trained machine learning model, such as an artificial neural network or a SVM, that is trained to perform a classification task according to method500, described above in conjunction withFIG.5. The output class is a specific category or label that trained classifier148predicts for the input image. The output class can be an output to any classification task for which trained classifier148was trained, such as a multi-label classification task with different classes of facial emotions (e.g., happy, sad, angry, and disgusted). In some embodiments, the output class can be an output to a binary classification task, such as classifying if a mouth is slightly open, or if the mouth is closed. In some embodiments, an output of application146can be the output class generated by classifier148. In some other embodiments, the output class generated by classifier148can be used by application146to generate another output. In such embodiments, application146can generate the other output in any technically feasible manner.

In sum, techniques are disclosed for improving the identity-robustness of machine learning models. In some embodiments, a model trainer application first processes images of faces using a trained face recognition model to generate a proxy representation of an identity of the individual in each image. Representations of individuals with similar facial features lie in the same neighborhoods within a proxy identity space. A neighborhood can be defined with a hard threshold or a soft threshold. A hard threshold sets a radius of the neighborhood. A soft threshold is defined based on a distance metric, such as cosine distance. Instead of excluding representations of individuals outside a predefined distance, in some embodiments, a soft threshold can emphasize representations of individuals with closer distances and de-emphasize representations of individuals with further distances. The model trainer application trains a machine learning model to perform a task relating to faces while considering the accuracy of each identity proxy neighborhood. Instead of training the machine learning model to perform well on average, the model trainer assigns different weights to each image sample in a neighborhood based on the number of samples with the same output class in that neighborhood. The assigned weights can then be used to compute an unbiased identity loss function that is used to train the machine learning model to perform the task relating to faces while being robust to identity features.

At least one technical advantage of the disclosed techniques relative to the prior art is that, with the disclosed techniques, machine learning models can be trained to perform tasks relating to faces without relying on facial identity features, thereby improving robustness of the machine learning models. In addition, the disclosed techniques do not require balanced data points to train robust machine learning models for different output classes. These technical advantages represent one or more technological improvements over prior art approaches.

1. In some embodiments, a computer-implemented method for training a machine learning model comprises for each image included a plurality of images, generating a representation of a face within the image, for each image included the plurality of images, computing a weight based on the representation generated for the image and at least one other representation generated for at least one other image included in the plurality of images, and performing one or more operations to train the machine learning model based on at least the weights to generate a trained machine learning model.

2. The computer-implemented method of clause 1, wherein computing the weight comprises computing an intermediate weight based on at least one computed similarity between the representation generated for the image and the at least one other representation generated for the at least one other image, and computing the weight based on the intermediate weight and a number of classes being predicted by the machine learning model.

3. The computer-implemented method of clauses 1 or 2, wherein computing the intermediate weight comprises computing an exponential of the at least one computed similarity.

4. The computer-implemented method of any of clauses 1-3, further comprising computing each computed similarity included in the at least one computed similarity based on a cosine distance metric.

5. The computer-implemented method of any of clauses 1-4, wherein the weight computed for each image in the plurality of images is a conditional inverse density normalized based on a number of classes being predicted by the machine learning model.

6. The computer-implemented method of any of clauses 1-6, wherein performing the one or more operations to train the machine learning model comprises for each image included the plurality of images, computing a weighted loss based on a loss that is computed for the image and the weight that is computed for the image, and updating one or more parameters of the machine learning model based on the weighted losses.

7. The computer-implemented method of any of clauses 1-6, wherein generating the representation of the face within the image comprises processing the image via a trained facial recognition model.

8. The computer-implemented method of any of clauses 1-7, further comprising processing another image via the trained machine learning model.

9. The computer-implemented method of any of clauses 1-8, wherein the machine learning model comprises a classification model.

10. The computer-implemented method of any of clauses 1-9, wherein the machine learning model comprises a convolutional neural network.

11. In some embodiments, one or more non-transitory computer-readable media store instructions that, when executed by at least one processor, cause the at least one processor to perform steps comprising for each image included a plurality of images, generating a representation of a face within the image, for each image included the plurality of images, computing a weight based on the representation generated for the image and at least one other representation generated for at least one other image included in the plurality of images, and performing one or more operations to train a machine learning model based on at least the weights to generate a trained machine learning model.

12. The one or more non-transitory computer-readable media of clause 11, wherein computing the weight comprises computing an intermediate weight based on at least one computed similarity between the representation generated for the image and the at least one other representation generated for the at least one other image, and computing the weight based on the intermediate weight and a number of classes being predicted by the machine learning model.

13. The one or more non-transitory computer-readable media of clauses 11 or 12, wherein computing the intermediate weight comprises computing an exponential of the at least one computed similarity.

14. The one or more non-transitory computer-readable media of any of clauses 11-13, further comprising computing each computed similarity included in the at least one computed similarity based on a cosine distance metric.

15. The one or more non-transitory computer-readable media of any of clauses 11-14, wherein the weight computed for each image in the plurality of images is a conditional inverse density normalized based on a number of classes being predicted by the machine learning model.

16. The one or more non-transitory computer-readable media of any of clauses 11-15, wherein performing the one or more operations to train the machine learning model comprises for each image included the plurality of images, computing a weighted loss based on a loss that is computed for the image and the weight that is computed for the image, and updating one or more parameters of the machine learning model based on the weighted losses.

17. The one or more non-transitory computer-readable media of any of clauses 11-16, wherein generating the representation of the face within the image comprises processing the image via a trained facial recognition model.

18. The one or more non-transitory computer-readable media of any of clauses 11-17, further comprising processing another image that includes another face via the trained machine learning model.

19. The one or more non-transitory computer-readable media of any of clauses 11-18, wherein the weight is further computed based on a predefined distance for a neighborhood of representations of images.

20. In some embodiments, a system comprises a memory storing instructions, and a processor that is coupled to the memory and, when executing the instructions, is configured to perform the steps of for each image included a plurality of images, generate a representation of a face within the image, for each image included the plurality of images, compute a weight based on the representation generated for the image and at least one other representation generated for at least one other image included in the plurality of images, and perform one or more operations to train a machine learning model based on at least the weights to generate a trained machine learning model.