Title: Face Recognition with Siamese Networks, Keras, and TensorFlow - PyImageSearch

URL Source: https://pyimagesearch.com/2023/01/09/face-recognition-with-siamese-networks-keras-and-tensorflow/

Published Time: 2023-01-09T14:00:00+00:00

Markdown Content:
* * *

**Table of Contents**
---------------------

*   [Face Recognition with Siamese Networks, Keras, and TensorFlow](#h2BP)

*   [Face Recognition](#h3Recognition)
*   [Face Recognition: Identification and Verification](#h3Verification)
*   [Identification via Verification](#h3Identification)
*   [Metric Learning: Contrastive Losses](#h3Metric)
*   [Contrastive Losses](#h3Contrastive)

*   [Summary](#h2Summary)

*   [Credits](#h3Credits)
*   [Citation Information](#h3Citation)

* * *

In this tutorial, you will learn about Siamese Networks and how they can be used to develop facial recognition systems. We will discuss the different types of facial recognition approaches and take an in-depth dive into the conceptual details of Siamese networks, which make them an integral part of robust facial recognition applications.

[![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20393'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/01/Face-Recognition-with-Siamese-Networks_featured.png)

Specifically, we will discuss the following in detail

*   The face recognition pipeline and various types of facial recognition approaches
*   Difference between face identification and verification
*   Metric Learning and Contrastive Losses

This lesson is the 1st in a 5-part series on Siamese Networks and their application in face recognition:

1.  [**_Face Recognition with Siamese Networks, Keras, and TensorFlow_**](https://pyimg.co/odny2) **(this tutorial)**
2.  _Building a Dataset for Triplet Loss with Keras and TensorFlow_
3.  _Triplet Loss with Keras and TensorFlow_
4.  _Training and Making Predictions with Siamese Networks and Triplet Loss_
5.  _Evaluating Siamese Network Accuracy (F1-Score, Precision, and Recall) with Keras and TensorFlow_

In the first part (this tutorial), we will aim to develop a holistic understanding of the different face recognition approaches and discuss the concepts behind contrastive losses, which are used to train Siamese networks.

In the second part, we will dive into the code for building end-to-end facial recognition systems using Siamese networks in Keras and TensorFlow. We will start by loading the Labeled faces in the wild data and prepare for our face recognition application.

Next, in the third part, we will understand the concept and mathematical formulation of triplet loss and write our own loss function in Keras and TensorFlow.

Finally, in the fourth and fifth parts, we will discuss how we can make inference with our end-to-end face recognition system to identify new faces in real-time. We will also discuss the various evaluation metrics to quantify the performance of our face recognition application in the final part of this series.

**To learn how to develop Face Recognition applications using Siamese Networks,** **_just keep reading._**

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

* * *

[**Face Recognition with Siamese Networks, Keras, and TensorFlow**](#TOC)
-------------------------------------------------------------------------

Deep learning models tend to develop a bias toward the data distribution on which they have been trained. This refers to the dataset bias problem where deep models overfit to the distribution (e.g., pose, background, color, texture, lighting condition) of training images and are unable to generalize to novel views or instances of the same objects at test time. One naive solution to this problem is to collect and annotate data in all possible backgrounds or views of the depiction of objects. However, this process is infeasible or tedious, leading to high annotation costs.

Thus, practical applications demand the development of models that are robust to nuisances and can generalize to novel unseen views or depictions of an object. For example, to compare if two images belong to the same object category, there is a need to

*   develop deep models that are invariant to factors like pose, background, color, texture
*   focus on the underlying properties that constitute an object category

One such successful effort toward developing robust deep models was the inception of Siamese networks. In previous tutorials, we have discussed an overview of the intuition and mechanism underlying these networks. In this tutorial, we will look at developing a Siamese network-based facial recognition system.

* * *

### [**Face Recognition**](#TOC)

Face recognition is an important computer vision task that allows us to identify and verify a person’s identity. It is widely used in the industry for applications such as developing attendance systems, healthcare, security verification, etc.

In previous tutorials, we discussed an overview of the face recognition task and various traditional and modern methods used to build effective face recognition systems.

An important aspect of any effective facial recognition system is its invariance to different views or ways of depicting the same person. For example, a face recognition-based employee attendance system should focus on generalizable features that make up a person’s identity and be robust changes in pose, appearance (e.g., beard, hairstyle, etc.), or unseen views of a given person. This is important since it is infeasible to train the system on all possible variations in the looks of every employee.

Recent advances in face recognition have thus used contrastive losses to train Siamese network-based architectures for building robust face recognition models.

* * *

### [**Face Recognition: Identification and Verification**](#TOC)

**Figure 1** shows a typical face recognition systems pipeline. As we can see, the first stage is the detection stage, where the face is distinguished from the background. This stage may also include an alignment operation that readjusts the pose of the face.

[![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20572%20156'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/01/Fig1.png)

**Figure 1:** Face Recognition Pipeline (source: image by the author).

Next, the detected face is cropped and passed to the deep feature extractor. The feature extractor outputs a feature vector for the face, as shown. The deep network is trained such that the output feature representation captures discriminative facial features and is robust to nuances (e.g., lighting, color, texture, beards, etc.).

Finally, the learned feature representation is used to identify or verify the person’s identity.

Next, let us understand in detail the difference between Face Identification and Face Verification tasks.

Face identification refers to the task of identifying a person’s identity and entails assigning a person’s identity to a given input image.

**Figure 2** shows a basic pipeline for performing face identification. The identification pipeline takes as input an image and outputs a probability distribution over the possible person identities.

[![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20228'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/01/Fig2.png)

**Figure 2:** Face Identification (source: image by the author).

First, the input image is passed through a deep feature extractor to get a feature representation ![Image 5: f](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2010%2014'%3E%3C/svg%3E), as shown in **Figure 2**. Next, the feature is passed through a classifier network that assigns a person’s identity or id to the given input image. Note that this entails a simple ![Image 6: K](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2011'%3E%3C/svg%3E) way multi-class classification problem for a database with ![Image 7: K](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2011'%3E%3C/svg%3E) personnel (here, ![Image 8: K =4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2045%2013'%3E%3C/svg%3E) persons or classes).

Since the identification pipeline is a simple multi-class classification, it is trained with the usual softmax and cross entropy loss.

On the other hand, the verification task entails verifying whether two images belong to the same identity.

**Figur****e 3** shows a basic pipeline for performing face verification. The input to the pipeline is the face image ![Image 9: x](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%209%207'%3E%3C/svg%3E) and the face id to be verified, and the output is a binary decision of whether the image belongs to the person with the face id.

[![Image 10](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20436%20500'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/01/Fig3.png)

**Figure 3:** Face Verification (source: image by the author).

In case of verification, we pre-extract and store the feature representation for all face images in our database, as shown. Given a new image ![Image 11: x](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%209%207'%3E%3C/svg%3E) and a Face ID (say id-1021, as shown in **Figure 3**, we compute the feature representation ![Image 12: f](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2010%2014'%3E%3C/svg%3E) of the image and compare it with the feature representation of Face ID-1021. The comparison is based on a similarity metric that we will discuss in detail later in the post. Finally, based on a threshold, we can find if the given input image belongs to a particular person in our database.

* * *

### [**Identification via Verification**](#TOC)

As discussed earlier in the post, identification requires us to train a ![Image 13: K](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2011'%3E%3C/svg%3E) way classifier to assign the given input face image a particular identity out of the ![Image 14: K](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2011'%3E%3C/svg%3E) personnel in our database. However, this type of identification system is not scalable if the number of people in our database increases.

For example, if a new person is added to the database, we will have to re-train the identification system built for ![Image 15: K](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2011'%3E%3C/svg%3E) classes on the new ![Image 16: K+1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2042%2014'%3E%3C/svg%3E) classes from scratch. This is because previously, the classifier had ![Image 17: K](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2011'%3E%3C/svg%3E) neurons, and now we need a classifier with ![Image 18: K+1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2042%2014'%3E%3C/svg%3E) neurons. This inhibits the efficiency and scalability of the system, as every time a new face is added, the entire training procedure has to be repeated.

Thus, a more efficient and effective way of implementing identification is by using the similarity-based comparison approach in verification.

**Figure 4** shows how identification can be implemented using verification techniques. Given an input image ![Image 19: x](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%209%207'%3E%3C/svg%3E), we can run our verification algorithm ![Image 20: K](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2011'%3E%3C/svg%3E) times for each of the ![Image 21: K](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2011'%3E%3C/svg%3E) Face IDs in our database. We can find the identity of the given face as the Face ID for which the binary output of the verification algorithm is true (i.e., the match for the input image is found).

[![Image 22](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20442'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/01/Fig4.png)

**Figure 4:** Identification via Verification (source: image by the author).

The benefit of using verification for identification is that if we add a new face to our database of ![Image 23: K](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2011'%3E%3C/svg%3E) faces, we just need to run the verification algorithm ![Image 24: K+1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2042%2014'%3E%3C/svg%3E) times for identification without the need to re-train the network.

Thus we observe that the classifier-based identification approach inhibits scalability due to the fixed number of neurons in the final layer. However, the similarity-based comparison approach used in verification is more efficient and scalable when new faces are added to the face database.

* * *

### [**Metric Learning: Contrastive Losses**](#TOC)

As discussed in the previous section, the similarity-based comparison approach of face recognition allows us to build more efficient and scalable systems than the classification-based approach.

Thus, in order to build effective face recognition systems, we need to develop training strategies that can enable us to build an embedding space where similar face images or face images of a given person are clustered together and face images of different people are farther apart.

The most common way to implement this for neural networks is by using deep metric learning.

In simple words, metric learning is a paradigm where our network is trained in such a way that representations of similar images are close to each other in an embedding space and images of different objects are farther apart. The idea of “similarity” here is defined based on a specific distance metric in the embedding space, which quantifies semantic similarity and whether the images belong to the same or different object or person.

* * *

### [**Contrastive Losses**](#TOC)

Contrastive losses are a group of loss functions that allow us to learn an embedding space where semantically similar inputs or images are embedded closer to each other when compared to dissimilar inputs. These losses are often used to train neural networks to learn a distance measure that can more effectively quantify the similarity between input images than predefined distance measures (e.g., euclidean distance, absolute distance, etc.).

There are different formulations of contrastive losses (e.g., ranking loss, pairwise contrastive loss, triplet loss, quadruplet loss, etc.). However, all these variants share the same basic idea and properties. Let us discuss this in detail with the help of an example.

**Figure 5** displays a general setup showing how contrastive losses are used to learn a feature representation.

[![Image 25](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20270'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/01/Fig5.png)

**Figure 5:** Learning with Contrastive Loss (source: image by the author).

Our aim is to learn a mapping ![Image 26: G](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2011'%3E%3C/svg%3E) from input space ![Image 27: X](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2014%2011'%3E%3C/svg%3E) to feature space ![Image 28: f](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2010%2014'%3E%3C/svg%3E). Here, the function ![Image 29: G](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2011'%3E%3C/svg%3E) is parameterized with weights ![Image 30: W](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2018%2011'%3E%3C/svg%3E), which are shared between the two branches. This can be interpreted as applying a complex transformation ![Image 31: G](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2011'%3E%3C/svg%3E) on the input space ![Image 32: X](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2014%2011'%3E%3C/svg%3E), which would project the inputs to feature space ![Image 33: f](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2010%2014'%3E%3C/svg%3E).

The objective is to learn the parameters ![Image 34: W](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2018%2011'%3E%3C/svg%3E) such that the high dimensional inputs ![Image 35: X_1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2018%2015'%3E%3C/svg%3E) and ![Image 36: X_2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2019%2014'%3E%3C/svg%3E) are mapped onto a low dimension manifold (or embedding space) ![Image 37: f](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2010%2014'%3E%3C/svg%3E). Their corresponding feature representations ![Image 38: f_1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2015'%3E%3C/svg%3E) and ![Image 39: f_2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2014%2014'%3E%3C/svg%3E) in the embedding space should be close to each other if the inputs ![Image 40: X_1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2018%2015'%3E%3C/svg%3E) and ![Image 41: X_2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2019%2014'%3E%3C/svg%3E) are similar. Otherwise, they should be farther apart.

Mathematically, we want to learn the parameters ![Image 42: W](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2018%2011'%3E%3C/svg%3E) such that the distance between the features ![Image 43: f_1 = G_{W} (X_{1})](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2094%2018'%3E%3C/svg%3E) and ![Image 44: f_2 = G_{W} (X_{2})](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2094%2018'%3E%3C/svg%3E), that is,

![Image 45: D_W (X_1, X_2) = {||G_W (X_1) - G_W (X_2)||}_{2},](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20276%2019'%3E%3C/svg%3E)

approximates the similarity between the inputs ![Image 46: X_1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2018%2015'%3E%3C/svg%3E) and ![Image 47: X_2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2019%2014'%3E%3C/svg%3E).

This objective can be achieved using a contrastive loss function which will

Let us consider the example of the simplest form of contrastive loss (i.e., pairwise contrastive loss) to understand this better.

![Image 48: L_\text{contrastive}(W,Y,X_{1},X_{2}) = (1-Y)/2(D_{W})^2+Y/2[\max(0,m-D_{W}^2)]](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20480%2019'%3E%3C/svg%3E)

where ![Image 49: m>0](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2044%2012'%3E%3C/svg%3E) is a margin.

The equation above shows the mathematical formulation of the pairwise contrastive loss. Here, ![Image 50: W](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2018%2011'%3E%3C/svg%3E) denotes the parameters of our mapping function ![Image 51: G](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2011'%3E%3C/svg%3E) and ![Image 52: Y](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2011'%3E%3C/svg%3E) is a binary identifier equal to 0 if ![Image 53: X_1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2018%2015'%3E%3C/svg%3E) and ![Image 54: X_2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2019%2014'%3E%3C/svg%3E) belong to the same class or person and 1, otherwise.

For the case when ![Image 55: Y = 0](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2043%2012'%3E%3C/svg%3E), that is, when ![Image 56: X_1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2018%2015'%3E%3C/svg%3E) and ![Image 57: X_2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2019%2014'%3E%3C/svg%3E) belong to the same class or person, the loss reduces to

![Image 58: L_\text{contrastive}(W,X_{1},X_{2}) = 1/2(D_{W})^2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20241%2018'%3E%3C/svg%3E)

Minimizing this expression will minimize ![Image 59: D_W](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2026%2014'%3E%3C/svg%3E), which is the distance between the representations ![Image 60: f_1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2015'%3E%3C/svg%3E) and ![Image 61: f_2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2014%2014'%3E%3C/svg%3E) (i.e., it will pull the features close to each other).

On the other hand, in the case when ![Image 62: Y = 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2042%2013'%3E%3C/svg%3E), that is, when ![Image 63: X_1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2018%2015'%3E%3C/svg%3E) and ![Image 64: X_2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2019%2014'%3E%3C/svg%3E) belong to a different class or person, the loss reduces to

![Image 65: L_\text{contrastive}(W,X_{1},X_{2}) = 1/2[\max(0,m-D_{W}^2)]](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20323%2019'%3E%3C/svg%3E)

Notice that the minimum value of this expression (which is 0) occurs when ![Image 66: m-D_{W}^2 \leq  0](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2092%2019'%3E%3C/svg%3E), that is ![Image 67: D_{W}^2 \geq m](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2063%2019'%3E%3C/svg%3E). This implies minimizing this loss to ensure that the features ![Image 68: f_1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2015'%3E%3C/svg%3E) and ![Image 69: f_2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2014%2014'%3E%3C/svg%3E) are at least apart by a distance of ![Image 70: m](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2014%207'%3E%3C/svg%3E) in the feature space.

Thus, the loss formulation ensures that similar points are close together in the features space and dissimilar points are separated at least by a distance of ![Image 71: m](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2014%207'%3E%3C/svg%3E) in the feature space.

Notice that the important principles that allow us to learn such a system and should be satisfied by any contrastive loss formulation are

*   Semantically similar points in input space should be mapped close to each other on the low-dimension manifold
*   The learned distance metric should be able to map unseen or new inputs
*   The mapping learned should be invariant to complex transformations

* * *

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

* * *

[**Summary**](#TOC)
-------------------

In this tutorial, we discussed the concepts of facial recognition and gained an in-depth understanding of a typical face recognition pipeline. Specifically, we discussed the differences between face identification and verification and how the similarity-based comparison approach used in verification allows us to build efficient and scalable face recognition systems.

Next, we tried to understand the intuition behind metric learning and Siamese networks, which allow us to build robust similarity-based face recognition applications. Finally, we discussed the setup and mathematical formulation of contrastive loss functions, which are pivotal in training similarity-based Siamese networks.

* * *

### [**Credits**](#TOC)

This post is inspired by the amazing National Programme on Technology Enhanced Learning (NPTEL) [Deep Learning for Computer Vision](https://onlinecourses.nptel.ac.in/noc20_cs88/preview) course, which the author contributed to while working at IIT Hyderabad.

* * *

### [**Citation Information**](#TOC)

**Chandhok, S.** “Face Recognition with Siamese Networks, Keras, and TensorFlow,” _PyImageSearch_, P. Chugh, A. R. Gosthipaty, S. Huot, K. Kidriavsteva, R. Raha, and A. Thanki, eds., 2023, [https://pyimg.co/odny2](https://pyimg.co/odny2)

@incollection{Chandhok\_2023\_Face\_recognition,
  author = {Shivam Chandhok},
  title = {Face Recognition with {Siamese Networks, Keras, and TensorFlow}},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Aritra Roy Gosthipaty and Susan Huot and Kseniia Kidriavsteva and Ritwik Raha and Abhishek Thanki},
  year = {2023},
  note = {https://pyimg.co/odny2},
}

* * *

![Image 72: Featured Image](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20276'%3E%3C/svg%3E)

### Unleash the potential of computer vision with Roboflow - Free!

*   Step into the realm of the future by [signing up or logging into your Roboflow account](https://roboflow.com/?ref=pyimagesearch). Unlock a wealth of innovative dataset libraries and revolutionize your computer vision operations.
*   Jumpstart your journey by choosing from our broad array of datasets, or benefit from PyimageSearch’s comprehensive library, crafted to cater to a wide range of requirements.
*   Transfer your data to Roboflow in any of the 40+ compatible formats. Leverage cutting-edge model architectures for training, and deploy seamlessly across diverse platforms, including API, NVIDIA, browser, iOS, and beyond. Integrate our platform effortlessly with your applications or your favorite third-party tools.
*   Equip yourself with the ability to train a potent computer vision model in a mere afternoon. With a few images, you can import data from any source via API, annotate images using our superior cloud-hosted tool, kickstart model training with a single click, and deploy the model via a hosted API endpoint. Tailor your process by opting for a code-centric approach, leveraging our intuitive, cloud-based UI, or combining both to fit your unique needs.
*   Embark on your journey today with absolutely no credit card required. Step into the future with Roboflow.

[Join Roboflow Now](https://roboflow.com/?ref=pyimagesearch)

**To download the source code to this post (and be notified when future tutorials are published here on PyImageSearch), _simply enter your email address in the form below!_**

![Image 73](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!