Title: Building a Dataset for Triplet Loss with Keras and TensorFlow - PyImageSearch

URL Source: https://pyimagesearch.com/2023/02/13/building-a-dataset-for-triplet-loss-with-keras-and-tensorflow/

Published Time: 2023-02-13T14:00:00+00:00

Markdown Content:
* * *

**Table of Contents**
---------------------

*   [Building a Dataset for Triplet Loss with Keras and TensorFlow](#h2BPTitle)

*   [Labeled Faces in the Wild Dataset](#h3Dataset)
*   [Configuring Your Development Environment](#h3Development)
*   [Having Problems Configuring Your Development Environment?](#h3Problems)
*   [Project Structure](#h3Structure)
*   [Creating Our Configuration File](#h3Configuration)
*   [Creating Our Data Pipeline](#h3Pipeline)
*   [Preprocessing Faces: Detection and Cropping](#h3Preprocessing)

*   [Summary](#h2Summary)

*   [Citation Information](#h3Citation)

* * *

In today’s tutorial, we will take the first step toward building our real-time face recognition application. Specifically, we will build a dataset for training our Siamese network-based recognition model.

[![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20393'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/02/Building-Dataset-for-Triplet-Loss-w-K-and-TF_Featured.png)

In the [previous tutorial](https://pyimg.co/odny2) of this series, we looked into the different face recognition tasks (e.g., identification and verification). Also tried to understand the benefit of using a verification-based approach for developing scalable and efficient face recognition applications. In addition, we discussed metric learning and how contrastive losses can be used to learn a distance measure in the embedding space, which can help us effectively quantify the similarity between input images.

In this part of the series, we will discuss the specific techniques required to develop a dataset that can be used to train our face recognition network with contrastive losses. Specifically, we will discuss the following in detail:

1.  Positive and Negative data samples required to train a network with contrastive loss
2.  Specific data preprocessing techniques (e.g., face detection and cropping) to build an effective face recognition model
3.  Creating a data pipeline for our Siamese network-based face recognition application with Keras and TensorFlow

This lesson is the 2nd of a 5-part series on Siamese Networks and their application in face recognition:

1.  [_Face Recognition with Siamese Networks, Keras, and TensorFlow_](https://pyimg.co/odny2)
2.  [**_Building a Dataset for Triplet Loss with Keras and TensorFlow_**](https://pyimg.co/g098j) **(this tutorial)**
3.  _Triplet Loss with Keras and TensorFlow_
4.  _Training and Making Predictions with Siamese Networks and Triplet Loss_
5.  _Evaluating Siamese Network Accuracy (F1-Score, Precision, and Recall) with Keras and TensorFlow_

**To learn how to build a dataset for developing a face recognition application, _just keep reading._**

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

* * *

[**Building a Dataset for Triplet Loss with Keras and TensorFlow**](#TOC)
-------------------------------------------------------------------------

In the [previous tutorial](https://pyimg.co/odny2), we looked into the formulation of the simplest form of contrastive loss. We tried to understand how these losses can help us learn a distance measure based on similarity. Specifically, we discussed how the behavior of the loss function changes depending on whether the input image samples belong to the same class/person or different classes.

When dealing with contrastive losses, it is typical to refer to the samples from the same class as positive samples and samples from different classes as negative samples.

For building our face recognition application, we will use a slightly better version of contrastive loss called the triplet loss. This loss function follows the same basic principles and characteristics as the typical contrastive loss (e.g., pairwise contrastive loss). We discussed this in the previous part of this series. However, its formulation is based on a triplet data sample slightly different from the pairwise loss discussed previously.

Let us get an overview of the formulation and sample requirements for the triplet loss. Each sample is composed of a triplet of images, namely `Anchor`, `Positive`, and `Negative`. The anchor and positive image samples belong to the same class/person, and the negative sample belongs to a different class/person than the positive sample. Furthermore, the anchor and positive are different image instances of the same person depicting them in different looks, varied poses, hairstyles, backgrounds, etc. **Figure 1** shows a typical example of a triplet image sample. Notice that the anchor and positive image show the same person in a different look, and the negative sample belongs to a different person.

[![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)](https://lh3.googleusercontent.com/8FvK8WT3JMTxTWN0JItZN2edp1z9tBlqVn_MgonPlIurTvH-rmTfWAiBTE7P_OsuLyZw89FUxSlN3cp3nXcnjHz7n6DYWhH4RgNEOTnn8ceuCssLZrImEKFqwRlRWFF9zD2IY52B-fRBMXXp-5ccStJzOlbpIV1YbqoR42PB4MlgI4_cYww3oA6-q55qUw)

**Figure 1:** A typical example of a triplet image sample.

In the next part of this series, we will delve deeper into the mathematical formulation and working principle of triplet loss. But for now, let us discuss further how we can process our dataset to get the triplet data samples required for training our model with this contrastive loss.

* * *

### [**Labeled Faces in the Wild Dataset**](#TOC)

For this tutorial series, we will use the Labeled Faces in the Wild (LFW) dataset, which consolidates a database of face photographs for face recognition research. The dataset consists of more than 13,000 images of faces collected from the internet, with each face image labeled with the corresponding person’s name. The dataset consists of 1680 people with two or more distinct face images of the same person, which will help us sample the triplet data samples and build our face recognition system. More information about the dataset can be found on the [LFW official website](http://vis-www.cs.umass.edu/lfw/).

* * *

### [**Configuring Your Development Environment**](#TOC)

To follow this guide, you need to have the TensorFlow and OpenCV libraries installed on your system.

Luckily, both TensorFlow and OpenCV are pip-installable:

$ pip install tensorflow
$ pip install opencv-contrib-python

**If you need help configuring your development environment for OpenCV, we _highly recommend_ that you read our** [**_pip install OpenCV_ guide**](https://www.pyimagesearch.com/2018/09/19/pip-install-opencv/) — it will have you up and running in a matter of minutes.

* * *

### [**Having Problems Configuring Your Development Environment?**](#TOC)

[![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20334'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2021/05/pyimagesearch_plus_jupyter.png)

Having trouble configuring your dev environment? Want access to pre-configured Jupyter Notebooks running on Google Colab? Be sure to join [PyImageSearch University](https://www.pyimagesearch.com/pyimagesearch-university/) — you’ll be up and running with this tutorial in a matter of minutes.

All that said, are you:

*   Short on time?
*   Learning on your employer’s administratively locked system?
*   Wanting to skip the hassle of fighting with the command line, package managers, and virtual environments?
*   **Ready to run the code** **_right now_** **on your Windows, macOS, or Linux system?**

Then join [PyImageSearch University](https://www.pyimagesearch.com/pyimagesearch-university/) today!

**Gain access to Jupyter Notebooks for this tutorial and other PyImageSearch guides that are** **_pre-configured_** **to run on Google Colab’s ecosystem right in your web browser!** No installation required.

And best of all, these Jupyter Notebooks will run on Windows, macOS, and Linux!

* * *

### [**Project Structure**](#TOC)

We first need to review our project directory structure.

Start by accessing the **_“Downloads”_** section of this tutorial to retrieve the source code and example images.

From there, take a look at the directory structure:

├── crop\_faces.py
├── face\_crop\_model
│   ├── deploy.prototxt.txt
│   └── res10\_300x300\_ssd\_iter\_140000.caffemodel
├── inference.py
├── pyimagesearch
│   ├── config.py
│   ├── dataset.py
│   └── model.py
└── train.py

The `crop_faces.py` file implements the code to detect and crop faces from our input images. The `face_crop_model` folder contains the Caffe files for our pre-trained detection model, which will detect faces in our input images.

The `inference.py` file contains the code for the inference stage of our face recognition model.

Furthermore, the `pyimagesearch` folder contains the `config.py`, `dataset.py`, and `model.py` files.

As the names suggest, the `config.py` file contains the configurations and parameter settings. The `dataset.py` file implements the code to build our data pipeline, and the `model.py` file contains the code to develop our Siamese model.

Finally, the `train.py` file contains the code to train our Siamese network-based face recognition pipeline.

We will discuss each of these files one by one in this series of tutorials. For this tutorial, we are concerned with setting up our configurations, building our data pipeline, and processing our input face images. Thus, we will discuss the `config.py`, `dataset.py`, and `crop_faces.py` files.

* * *

### [**Creating Our Configuration File**](#TOC)

We start by discussing the `config.py` file, which stores configurations and parameter settings used for this tutorial series.

\# import the necessary packages
import tensorflow as tf
import os

# path to training and testing data
TRAIN\_DATASET = "cropped\_train\_dataset"
TEST\_DATASET = "cropped\_test\_dataset"

# model input image size
IMAGE\_SIZE = (224, 224)

# batch size and the buffer size
BATCH\_SIZE = 256
BUFFER\_SIZE = BATCH\_SIZE \* 2

# define autotune
AUTO = tf.data.AUTOTUNE

# define the training parameters
LEARNING\_RATE = 0.0001
STEPS\_PER\_EPOCH = 50
VALIDATION\_STEPS = 10
EPOCHS = 10

# define the path to save the model
OUTPUT\_PATH = "output"
MODEL\_PATH = os.path.join(OUTPUT\_PATH, "siamese\_network")
OUTPUT\_IMAGE\_PATH = os.path.join(OUTPUT\_PATH, "output\_image.png")

First, we import the necessary packages (i.e., `tensorflow` and `os`) on **Lines 2 and 3**. Then, we define the paths to our training dataset (i.e., `TRAIN_DATASET`) and test dataset (i.e., `TEST_DATASET`) on **Lines 6 and 7**, respectively.

On **Line 10**, we define the default image size with dimensions `(224, 224)`, and on **Lines 13 and 14**, we define the `BATCH_SIZE` and `BUFFER_SIZE`. Furthermore, we define the autotune parameter (`AUTO`) with the help of `tf.data.AUTOTUNE` on **Line 17**.

Next, we define our training parameters. We set out `LEARNING_RATE`, `STEPS_PER_EPOCH`, `VALIDATION_STEPS`, and the total number of epochs (i.e., `EPOCHS`) on **Lines 20-23**.

Finally, on **Line 27**, we define the paths where our final model will be saved (i.e., `MODEL_PATH`), and on **Line 28**, the location where our final output images will be saved (i.e., `OUTPUT_IMAGE_PATH`).

* * *

### [**Creating Our Data Pipeline**](#TOC)

Now that we have discussed and set our configurations and parameters, it is time to build our data pipeline.

We open our `dataset.py` file from the `pyimagesearch` folder in our project directory and get started.

\# import the necessary packages
import tensorflow as tf
import numpy as np
import random
import os

class MapFunction():
	def \_\_init\_\_(self, imageSize):
		# define the image width and height
		self.imageSize = imageSize

	def decode\_and\_resize(self, imagePath):
		# read and decode the image path
		image = tf.io.read\_file(imagePath)
		image = tf.image.decode\_jpeg(image, channels=3)

		# convert the image data type from uint8 to float32 and then resize
		# the image to the set image size
		image = tf.image.convert\_image\_dtype(image, dtype=tf.float32)
		image = tf.image.resize(image, self.imageSize)

		# return the image
		return image

	def \_\_call\_\_(self, anchor, positive, negative):
		anchor = self.decode\_and\_resize(anchor)
		positive = self.decode\_and\_resize(positive)
		negative = self.decode\_and\_resize(negative)

		# return the anchor, positive and negative processed images
		return (anchor, positive, negative)

On **Lines 2-5**, we first import the necessary packages such as `tensorflow`, `numpy` (for tensor and matrix manipulations), and `random` and `os` for utility functions and filesystem access.

We start by defining the `MapFunction` class (**Lines 7-31**), which will be used later to apply transformations and preprocess the anchor, positive, and negative image samples in our datasets.

Let us look at the definition of this call step by step.

First, we define the _`_init_`_ method, which takes as an argument the size of our input image (i.e., `imageSize`) on **Line 8**, and assigns it to the `self.imageSize` attribute of our class (**Line 10**).

Next, we define the `decode_and_resize` function (**Lines 12-23**), which takes as input the path to the image (i.e., `imagePath`) and processes the image to the appropriate type and size. Specifically, this function first reads the image using the `tf.io.read_file()` function, which takes the path to the image (i.e., `imagePath`) as input.

Next, we use the `tf.image.decode_jpeg()` function to convert the jpeg input image to a uint8 tensor (**Line 15**). This function takes the `image` and the `channels` argument as input, which is set to the value `3` since we need an RGB output image. We then convert the uint8 tensor to the required float32 format and resize the image to the required `imageSize` using the `tf.image.resize` function on **Lines 19 and 20**, respectively.

Finally, we return the transformed image on **Line 23**.

Next, we define the `_call_` function, which consolidates and implements the transformations applied when we call the `MapFunction` class on our dataset. Basically, it takes as input three images (i.e., `anchor`, `positive`, and `negative`) and transforms each of them to the appropriate size and format using the decode and resize function (**Lines 26-28**). Finally, it returns the processed `anchor`, `positive`, and `negative` images on **Line 31**.

Now, let us define our `TripletGenerator` class (**Lines 33-103**), which will allow us to define our train and validation data generators that we will use to get batches of data samples during training.

class TripletGenerator:
	def \_\_init\_\_(self, datasetPath):
		# create an empty list which will contain the subdirectory
		# names of the \`dataset\` directory with more than one image
		# in it
		self.peopleNames = list()

		# iterate over the subdirectories in the dataset directory
		for folderName in os.listdir(datasetPath):
			# build the subdirectory name
			absoluteFolderName = os.path.join(datasetPath, folderName)

			# get the number of images in the subdirectory
			numImages = len(os.listdir(absoluteFolderName))

			# if the number of images in the current subdirectory
			# is more than one, append into the \`peopleNames\` list
			if numImages > 1:
				self.peopleNames.append(absoluteFolderName)

		# create a dictionary of people name to their image names
		self.allPeople = self.generate\_all\_people\_dict()

	def generate\_all\_people\_dict(self):
		# create an empty dictionary that will be populated with
		# directory names as keys and image names as values
		allPeople = dict()

		# iterate over all the directory names with more than one
		# image in it
		for personName in self.peopleNames:
			# get all the image names in the current directory
			imageNames = os.listdir(personName)

			# build the image paths and populate the dictionary
			personPhotos = \[
				os.path.join(personName, imageName) for imageName in imageNames
			\]
			allPeople\[personName\] = personPhotos

		# return the dictionary
		return allPeople

	def get\_next\_element(self):
		# create an infinite generator
		while True:
			# draw a person at random which will be our anchor and
			# positive person
			anchorName = random.choice(self.peopleNames)

			# copy the list of people names and remove the anchor
			# from the list
			temporaryNames = self.peopleNames.copy()
			temporaryNames.remove(anchorName)

			# draw a person at random from the list of people without
			# the anchor, which will act as our negative sample
			negativeName = random.choice(temporaryNames)

			# draw two images from the anchor folder without replacement
			(anchorPhoto, positivePhoto) = np.random.choice(
				a=self.allPeople\[anchorName\],
				size=2,
				replace=False
			)

			# draw an image from the negative folder
			negativePhoto = random.choice(self.allPeople\[negativeName\])

			# yield the anchor, positive and negative photos
			yield (anchorPhoto, positivePhoto, negativePhoto)

Before we start, it is worth discussing the structure of our dataset folder. The dataset folder consists of subdirectories corresponding to different people, with the person’s name as the subdirectory’s name. Each subdirectory contains one or more images of the respective person’s face.

As always, we start with the _`init`_ method, which takes as input the path to our dataset directory (i.e., `datasetPath`) on **Line 34**. Then, on **Line 38**, we create an empty list, `self.peopleNames`, to store the subdirectories (corresponding to each person) in our dataset directory with more than one image sample.

Next, on **Line 41**, we iterate over the folders in the dataset directory (i.e., `datasetPath`). For each folder, we build the full path to the folder by joining the `datasetPath` and `folderName` with the help of the `os.path.join()` function (**Line 43**).

We count the images in the current folder (**Line 46**) and append the folder name to our `self.peopleNames` list in case the folder has more than one image sample (**Lines 50 and 51**). Finally, on **Line 54**, we define the `self.allPeople` dictionary in the _`init`_ function, which will hold the subdirectory for each person as keys and their corresponding image names as values. This dictionary is populated using the `generate_all_people_dict`, as shown on **Line 54**.

Now that we have defined our `init` method, let us write a function to help us populate the `self.allPeople` dictionary described above.

We begin the step-by-step discussion of our `generate_all_people_dict` function (**Lines 56-74**), which populates our `self.allPeople` dictionary.

We start by creating an empty dictionary (i.e., `allPeople`) on **Line 59**. Then, we iterate over the subdirectories or paths in the `self.peopleNames` list and get the names of all images at the current path or subdirectory (i.e., `imageNames`) on **Line 65**.

Next, we build the paths to each image by joining the `personName` (i.e., path to the subdirectory of each person) and the `imageName` (i.e., names of corresponding images) for each image in the current person’s subdirectory (**Lines 68-70**) and populate the `allPeople` dictionary with `personName` as key and the path to images as the value (**Line 71**). We return our `allPeople` dictionary on **Line 74**.

Finally, let us create the `get_next_element` function, which implements the main function of our data generator and will allow us to get data samples during training.

Note that as per the triplet loss formulation, we need to sample three images (i.e., `anchor`, `positive`, and `negative` images). Also, we need to ensure that the positive image sample comes from the same person as the anchor image (anchor subject) and that the negative image sample comes from a different person (negative subject).

We start with an infinite while loop which allows us to sample from our generator indefinitely (**Line 78**). Within the loop, we first choose our anchor person’s subdirectory by randomly sampling a subdirectory from the `self.peopleNames` list using the `random.choice` function (**Line 81**).

Next, since our negative sample should be from a person different from our anchor sample, we create a copy of our `self.peopleNames` list (i.e., `temporaryNames`) and remove the subdirectory corresponding to the anchor person (i.e., `anchorName`) on **Lines 85 and 86**, respectively. Now, we sample our negative person’s subdirectory from the `temporaryNames` list (which does not contain the subdirectory corresponding to the anchor person anymore) (**Line 90**).

Note that now we have the path to the subdirectories of our anchor subject (i.e., `anchorName`) and negative subject (i.e., `negativeName`).

We sample 2 images (anchor and positive sample) from the `anchorName` subdirectory using the `self.allPeople` dictionary with the help of the `np.random.choice` function and store the anchor and positive image sample in the `anchorPhoto` and `positivePhoto` variables (**Lines 93-97**). Next, we sample our negative image sample from the `negativeName` subdirectory and store it in the `negativePhoto` variable (**Line 100**).

Finally, our function yields a tuple (`anchorPhoto`, `positivePhoto`, `negativePhoto`) containing the anchor, positive, and negative sample triplet (**Line 103**).

* * *

### [**Preprocessing Faces: Detection and Cropping**](#TOC)

In the [previous post](https://pyimg.co/odny2) of this series, we looked at a typical face recognition pipeline and discussed the different stages and tasks involved.

One of the most important tasks for face recognition is to detect the face and distinguish it from the background. Then the detected face is cropped to keep only the useful part of the image and discard the irrelevant details. Finally, this cropped face image is passed to the face recognition model for further processing.

Let us try to develop a python script that can detect the face in our input image and crop the relevant part (i.e., facial features) of our image and store it for further processing. Note that for this task, we will need a pre-trained detection model that is trained to detect faces in images.

Let us now open our `crop_faces.py` file from our project directory and get started.

\# USAGE
# python crop\_faces.py --dataset train\_dataset --output cropped\_train\_dataset \\
# --prototxt face\_crop\_model/deploy.prototxt.txt \\
# --model face\_crop\_model/res10\_300x300\_ssd\_iter\_140000.caffemodel
#
# python crop\_faces.py --dataset test\_dataset --output cropped\_test\_dataset \\
# --prototxt face\_crop\_model/deploy.prototxt.txt \\
# --model face\_crop\_model/res10\_300x300\_ssd\_iter\_140000.caffemodel

# import the necessary packages
from imutils.paths import list\_images
from tqdm import tqdm
import numpy as np
import argparse
import cv2
import os

# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add\_argument("-d", "--dataset", required=True,
	help="path to input dataset")
ap.add\_argument("-o", "--output", required=True,
	help="path to output dataset")
ap.add\_argument("-p", "--prototxt", required=True,
	help="path to Caffe 'deploy' prototxt file")
ap.add\_argument("-m", "--model", required=True,
	help="path to Caffe pre-trained model")
ap.add\_argument("-c", "--confidence", type=float, default=0.5,
	help="minimum probability to filter weak detections")
args = vars(ap.parse\_args())

As always, we first import the necessary packages on **Lines 11-16**.

We start by constructing an argument parser with the help of `ArgumentParser()` from the `argparse` package, as shown on **Line 19**. Next, we define the necessary arguments for running our `crop_faces.py` script (**Lines 19-30**).

Specifically, our script takes the following as input arguments:

*   `--dataset` (abbreviated with `-d`) defines the path to the input dataset
*   `--output` (abbreviated with `-o`) defines the path where the output dataset is stored
*   `--prototxt` (abbreviated with `-p`) defines the path to the Caffe prototxt file (file containing the definition of the detection model)
*   `--model` (abbreviated with `-m`) path to pre-trained detection model in Caffe
*   `--confidence` (abbreviated with `-c`) probability threshold to filter weak detections

\# load our serialized model from disk
print("\[INFO\] loading model...")
net = cv2.dnn.readNetFromCaffe(args\["prototxt"\], args\["model"\])

# check if the output dataset directory exists, if it doesn't, then
# create it
if not os.path.exists(args\["output"\]):
	os.makedirs(args\["output"\])

# grab the file and sub-directory names in dataset directory
print("\[INFO\] grabbing the names of files and directories...")
names = os.listdir(args\["dataset"\])

# loop over all names
print("\[INFO\] starting to crop faces and saving them to disk...")
for name in tqdm(names):
	# build directory path
	dirPath = os.path.join(args\["dataset"\], name)

	# check if the directory path is a directory
	if os.path.isdir(dirPath):
		# grab the path to all the images in the directory
		imagePaths = list(list\_images(dirPath))

		# build the path to the output directory
		outputDir = os.path.join(args\["output"\], name)

		# check if the output directory exists, if it doesn't, then
		# create it
		if not os.path.exists(outputDir):
			os.makedirs(outputDir)

		# loop over all image paths
		for imagePath in imagePaths:
			# grab the image ID, load the image, and grab the
			# dimensions of the image
			imageID = imagePath.split(os.path.sep)\[-1\]
			image = cv2.imread(imagePath)
			(h, w) = image.shape\[:2\]

			# construct an input blob for the image by resizing to a
			# fixed 300x300 pixels and then normalizing it
			blob = cv2.dnn.blobFromImage(cv2.resize(image,
				(300, 300)), 1.0, (300, 300), (104.0, 177.0, 123.0))

			# pass the blob through the network and obtain the
			# detections and predictions
			net.setInput(blob)
			detections = net.forward()

			# extract the index of the detection with max 
			# probability and get the maximum confidence value
			i = np.argmax(detections\[0, 0, :, 2\])
			confidence = detections\[0, 0, i, 2\]

			# filter out weak detections by ensuring the 
			# \`confidence\` is greater than the minimum confidence
			if confidence > args\["confidence"\]:
				# grab the maximum dimension value
				maxDim = np.max(detections\[0, 0, i, 3:7\])

				# check if max dimension value is greater than one,
				# if so, skip the detection since it is erroneous
				if maxDim > 1.0:
					continue

				# clip the dimension values to be between 0 and 1
				box = np.clip(detections\[0, 0, i, 3:7\], 0.0, 1.0)

				# compute the (x, y)-coordinates of the bounding
				# box for the object
				box = box \* np.array(\[w, h, w, h\])
				(startX, startY, endX, endY) = box.astype("int")
			
				# grab the face from the image, build the path to
				# the output face image, and write it to disk
				face = image\[startY:endY,startX:endX,:\]
				facePath = os.path.join(outputDir, imageID)
				cv2.imwrite(facePath, face)

print("\[INFO\] finished cropping faces and saving them to disk...")

On **Line 34**, we use the `cv2.dnn.readNetFromCaffe()` function to load our face detection model. This function takes as input the model definition file (i.e., `args["prototxt"]`) and the pre-trained model (i.e., `args["model"]`).

Next, we prepare to store our output cropped dataset by checking if the output dataset directory already exists and creating it if it doesn’t exist (**Lines 38 and 39**).

We use the `os.listdir()` function to list the subdirectories in our dataset directory and store them in the variable `names` (**Line 43**).

Now, we iterate through the subdirectories in `names`. For each name, we first build the full path (i.e., `dirPath`) by joining the original input dataset path (i.e., `args["dataset"]`) and the current subdirectory name. Next, we check if `dirPath` is a directory and create a list of paths to all images (i.e., `imagePaths`) in the current subdirectory (**Line 54**).

In addition, we also build the path to our output directory (i.e., `outputDir`), where the corresponding cropped images will be stored (**Line 57**). Note that we check if the output directory already exists and create it if it doesn’t exist (**Lines 61 and 62**).

Finally, we iterate over the paths in the `imagePaths` list and load the image id and the image using the `imread` function from OpenCV on **Lines 68 and 69**, respectively. Also, we store the image’s dimensions (i.e., height and width) in tuple `(h, w)` on **Line 70**.

Next, we use the `cv2.dnn.blobFromImage` function from OpenCV, allowing us to process the image and perform normalization (**Lines 74 and 75**). The function takes as input the following arguments:

1.  The input image resized to the `(300, 300)` dimension (i.e., `cv2.resize(image, (300, 300))`)
2.  The scale factor which is `1.0` here (i.e., no scaling)
3.  The size of the output blob (i.e., `(300, 300)`)
4.  The mean subtraction values for R, G, and B channels, respectively, `(104.0, 177.0, 123.0)`

For an in-depth explanation of the `cv2.dnn.blobFromImage` function, check out our [blog post](https://pyimagesearch.com/2017/11/06/deep-learning-opencvs-blobfromimage-works/), which explains this function in detail.

Now we set our output `blob` as the input to the network using the `setInput()` function and forward pass the input through our detection network using the `net.forward()` command on **Lines 79 and 80**, respectively. The final output detections are stored in the `detections` variable (**Line 80**).

Then, we use the `np.argmax()` function to get the detection index with the maximum probability and the corresponding confidence value (**Lines 84 and 85**).

Now that we have the confidence values, we need to filter out the detections with low confidence to work with strong/probable detections and remove weak/improbable detections.

To achieve this, we set a threshold (i.e., `args["confidence"]`) and only keep those detections for which the confidence is above the threshold.

We start by checking if the maximum confidence value (i.e., `confidence`) is greater than the threshold (**Line 89**). If yes, we get the maximum dimension value using the `np.max` function and store it in the `maxDim` variable (**Line 91**). If max dimension value is greater than `1.0`, we skip the detection since it is erroneous and continue iterating over the next image path (**Lines 95 and 96**).

For non-erroneous detections (i.e., `maxDim ≤ 1.0`), we clip the detection values between the `0.0` and `1.0` range using the `np.clip` function and store the output in the `box` variable (**Line 99**).

Next, we get (_x_, _y_)-coordinates of our detected bounding box by multiplying our output `box` with the dimensions of the image (i.e., height (`h`) and width (`w`)) that we computed earlier (**Line 103**). We then convert the output box to type integer to get the start and end _x_ and _y_\-coordinate points (i.e., `(startX, startY, endX, endY)`) on **Line 104**.

Finally, we grab the part of the image where the face is detected and store it in the `face` variable on **Line 108**. We then build the path where our output cropped image is stored (`facePath`) and use the `cv2.imwrite` function to write our face image to disk on **Lines 109 and 110**, respectively.

* * *

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

* * *

[**Summary**](#TOC)
-------------------

In this tutorial, we learned to build a data pipeline for our face recognition application with Keras and TensorFlow. Specifically, we tried to understand the type of data samples required to train our network with triplet loss and discussed the features of anchor, positive, and negative images.

In addition, we built a data loading pipeline that would output a triplet of images for training our Siamese network-based face recognition application.

Finally, we discussed the importance of preprocessing face images using detection and cropping to build an effective face recognition model and implemented our own pipeline to detect and crop faces.

After following this tutorial, you will be able to understand the preprocessing techniques as well as the details of data samples and loading required to build a triplet loss-based Siamese network face recognition application in Keras and TensorFlow.

* * *

### [**Citation Information**](#TOC)

**Chandhok, S.** “Building a Dataset for Triplet Loss with Keras and TensorFlow,” _PyImageSearch_, P. Chugh, A. R. Gosthipaty, S. Huot, K. Kidriavsteva, R. Raha, and A. Thanki, eds., 2023, [https://pyimg.co/g098j](https://pyimg.co/g098j)

@incollection{Chandhok\_2023\_Building\_dataset,
  author = {Shivam Chandhok},
  title = {Building a Dataset for Triplet Loss with {Keras and TensorFlow}},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Aritra Roy Gosthipaty and Susan Huot and Kseniia Kidriavsteva and Ritwik Raha and Abhishek Thanki},
  year = {2023},
  url = {https://pyimg.co/g098j},
}

* * *

![Image 5: Featured Image](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20276'%3E%3C/svg%3E)

### Unleash the potential of computer vision with Roboflow - Free!

*   Step into the realm of the future by [signing up or logging into your Roboflow account](https://roboflow.com/?ref=pyimagesearch). Unlock a wealth of innovative dataset libraries and revolutionize your computer vision operations.
*   Jumpstart your journey by choosing from our broad array of datasets, or benefit from PyimageSearch’s comprehensive library, crafted to cater to a wide range of requirements.
*   Transfer your data to Roboflow in any of the 40+ compatible formats. Leverage cutting-edge model architectures for training, and deploy seamlessly across diverse platforms, including API, NVIDIA, browser, iOS, and beyond. Integrate our platform effortlessly with your applications or your favorite third-party tools.
*   Equip yourself with the ability to train a potent computer vision model in a mere afternoon. With a few images, you can import data from any source via API, annotate images using our superior cloud-hosted tool, kickstart model training with a single click, and deploy the model via a hosted API endpoint. Tailor your process by opting for a code-centric approach, leveraging our intuitive, cloud-based UI, or combining both to fit your unique needs.
*   Embark on your journey today with absolutely no credit card required. Step into the future with Roboflow.

[Join Roboflow Now](https://roboflow.com/?ref=pyimagesearch)

**To download the source code to this post (and be notified when future tutorials are published here on PyImageSearch), _simply enter your email address in the form below!_**

![Image 6](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!