Title: Contrastive Loss for Siamese Networks with Keras and TensorFlow - PyImageSearch

URL Source: https://pyimagesearch.com/2021/01/18/contrastive-loss-for-siamese-networks-with-keras-and-tensorflow/

Published Time: 2021-01-18T15:00:00+00:00

Markdown Content:
In this tutorial, you will learn about contrastive loss and how it can be used to train more accurate siamese neural networks. We will implement contrastive loss using Keras and TensorFlow.

![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20450'%3E%3C/svg%3E)

Previously, I authored a three-part series on the fundamentals of siamese neural networks:

1.  _[Building image pairs for siamese networks with Python](https://pyimagesearch.com/2020/11/23/building-image-pairs-for-siamese-networks-with-python/)_
2.  _[Siamese networks with Keras, TensorFlow, and Deep Learning](https://pyimagesearch.com/2020/11/30/siamese-networks-with-keras-tensorflow-and-deep-learning/)_
3.  _[Comparing images for similarity using siamese networks, Keras, and TenorFlow](https://pyimagesearch.com/2020/12/07/comparing-images-for-similarity-using-siamese-networks-keras-and-tensorflow/)_

This series covered the fundamentals of siamese networks, including:

*   Generating image pairs
*   Implementing the siamese neural network architecture
*   Using binary cross-entry to train the siamese network

But while binary cross-entropy is certainly a _valid_ choice of loss function, it’s not the _only_ choice (or even the _best_ choice).

**State-of-the-art siamese networks tend to use some form of either contrastive loss or triplet loss when training** — these loss functions are better suited for siamese networks and tend to improve accuracy.

By the end of this guide, you will understand how to implement siamese networks and then train them with contrastive loss.

**To learn how to train a siamese neural network with contrastive loss, _just keep reading._**

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

In the first part of this tutorial, we will discuss what contrastive loss is and, more importantly, how it can be used to more accurately and effectively train siamese neural networks.

We’ll then configure our development environment and review our project directory structure.

We have a number of Python scripts to implement today, including:

*   A configuration file
*   Helper utilities for generating image pairs, plotting training history, and implementing custom layers
*   Our contrastive loss implementation
*   A training script
*   A testing/inference script

We’ll review each of these scripts; however, some of them have been covered [in my previous guides on siamese neural networks](https://pyimagesearch.com/2020/11/23/building-image-pairs-for-siamese-networks-with-python/), so when appropriate I’ll refer you to my other tutorials for additional details.

We’ll also spend a considerable amount of time discussing our contrastive loss implementation, ensuring you understand what it’s doing, how it works, and why we are utilizing it.

By the end of this tutorial, you will have a fully functioning contrastive loss implementation that is capable of training a siamese neural network.

### **What is contrastive loss? And how can contrastive loss be used to train siamese networks?**

[In our previous series of tutorials on siamese neural networks](https://pyimagesearch.com/2020/11/23/building-image-pairs-for-siamese-networks-with-python/), we learned how to train a siamese network using the binary cross-entropy loss function:

![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20148'%3E%3C/svg%3E)

**Figure 1:** The binary cross-entropy loss function ([image source](https://deeplearningdemystified.com/article/fdl-3)).

Binary cross-entropy was a valid choice here because what we’re essentially doing is 2-class classification:

1.  Either the two images presented to the network belong to the _same class_
2.  Or the two images belong to _different classes_

Framed in that manner, we have a classification problem. And since we only have two classes, binary cross-entropy makes sense.

**However, there is actually a loss function _much better suited_ for siamese networks called _contrastive loss:_**

![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20582%2064'%3E%3C/svg%3E)

**Figure 2:** The contrastive loss function.

Paraphrasing [Harshvardhan Gupta](https://hackernoon.com/one-shot-learning-with-siamese-networks-in-pytorch-8ddaab10340e), we need to keep in mind that the goal of a siamese network isn’t to _classify_ a set of image pairs but instead to _differentiate between them._ Essentially, contrastive loss is evaluating how good a job the siamese network is distinguishing between the image pairs. The difference is subtle but incredibly important.

To break this equation down:

We’ll be implementing this loss function using Keras and TensorFlow later in this tutorial.

If you would like more mathematically motivated details on contrastive loss, be sure to refer to Hadsell et al.’s paper, _[Dimensionality Reduction by Learning an Invariant Mapping.](http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf)_

### **Configuring your development environment**

This series of tutorials on siamese networks utilizes Keras and TensorFlow. If you intend on following this tutorial on the previous two parts in this series, I suggest you take the time now to configure your deep learning development environment.

You can utilize either of these two guides to install TensorFlow and Keras on your system:

*   _[How to install TensorFlow 2.0 on Ubuntu](https://pyimagesearch.com/2019/12/09/how-to-install-tensorflow-2-0-on-ubuntu/)_
*   _[How to install TensorFlow 2.0 on macOS](https://pyimagesearch.com/2019/12/09/how-to-install-tensorflow-2-0-on-macos/)_

Either tutorial will help you configure your system with all the necessary software for this blog post in a convenient Python virtual environment.

### **Having problems configuring your development environment?**

[![Image 5](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20334'%3E%3C/svg%3E)](https://pyimagesearch.com/pyimagesearch-plus/)

**Figure 3:** Having trouble configuring your dev environment? Want access to pre-configured Jupyter Notebooks running on Google Colab? Be sure to join [PyImageSearch Plus](https://pyimagesearch.com/pyimagesearch-plus/) — you’ll be up and running with this tutorial in a matter of minutes.

All that said, are you:

*   Short on time?
*   Learning on your employer’s administratively locked system?
*   Wanting to skip the hassle of fighting with the command line, package managers, and virtual environments?
*   **Ready to run the code _right now_ on your Windows, macOS, or Linux system?**

Then join [PyImageSearch Plus](https://pyimagesearch.com/pyimagesearch-plus/) today!

**Gain access to Jupyter Notebooks for this tutorial and other PyImageSearch guides that are _pre-configured_ to run on Google Colab’s ecosystem right in your web browser!** No installation required.

And best of all, these Jupyter Notebooks will run on Windows, macOS, and Linux!

### **Project structure**

Today’s tutorial on contrastive loss on siamese networks builds on my three previous tutorials that cover the fundamentals of building image pairs, implementing and training siamese networks, and using siamese networks for inference:

1.  _[Building image pairs for siamese networks with Python](https://pyimagesearch.com/2020/11/23/building-image-pairs-for-siamese-networks-with-python/)_
2.  _[Siamese networks with Keras, TensorFlow, and Deep Learning](https://pyimagesearch.com/2020/11/30/siamese-networks-with-keras-tensorflow-and-deep-learning/)_
3.  _[Comparing images for similarity using siamese networks, Keras, and TensorFlow](https://pyimagesearch.com/2020/12/07/comparing-images-for-similarity-using-siamese-networks-keras-and-tensorflow/)_

**We’ll be building on the knowledge we gained from those guides (including the project directory structure itself) today, so consider the previous guides _required reading_ before continuing today.**

Once you’ve gotten caught up, we can proceed to review our project directory structure:

$ tree . --dirsfirst
.
├── examples
│   ├── image\_01.png
│   ├── image\_02.png
│   ├── image\_03.png
...
│   └── image\_13.png
├── output
│   ├── contrastive\_siamese\_model
│   │   ├── assets
│   │   ├── variables
│   │   │   ├── variables.data-00000-of-00001
│   │   │   └── variables.index
│   │   └── saved\_model.pb
│   └── contrastive\_plot.png
├── pyimagesearch
│   ├── config.py
│   ├── metrics.py
│   ├── siamese\_network.py
│   └── utils.py
├── test\_contrastive\_siamese\_network.py
└── train\_contrastive\_siamese\_network.py

6 directories, 23 files

Inside the `` `pyimagesearch` `` module you’ll find four Python files:

1.  `` `config.py` ``: Contains our configuration of important variables, including batch size, epochs, output file paths, etc.
2.  `` `metrics.py` ``: Holds our implementation of the `` `contrastive_loss` `` function
3.  `` `siamese_network.py` ``: Contains the siamese network model architecture
4.  `utils.py`: Includes helper utilities, including a function to generate image pairs, compute the Euclidean distance as a layer _inside_ of a CNN, and a training history plotting function

We then have two Python driver scripts:

1.  `` `train_contrastive_siamese_network.py` ``: Trains our siamese neural network using contrastive loss and serializes the training history and model weights/architecture to disk inside the `` `output` `` directory
2.  `` `test_contrastive_siamse_network.py` ``: Loads our trained siamese network from disk and applies it to image pairs from inside the `` `examples` `` directory

Again, I cannot stress the importance of reviewing my [previous series of tutorials on siamese networks](https://pyimagesearch.com/2020/11/23/building-image-pairs-for-siamese-networks-with-python/). Doing so is an _absolute requirement_ before continuing here today.

### **Implementing our configuration file**

Our configuration file holds important variables used to train our siamese network with contrastive loss.

Open up the `` `config.py` `` file in your project directory structure, and let’s take a look inside:

\# import the necessary packages
import os

# specify the shape of the inputs for our network
IMG\_SHAPE = (28, 28, 1)

# specify the batch size and number of epochs
BATCH\_SIZE = 64
EPOCHS = 100

# define the path to the base output directory
BASE\_OUTPUT = "output"

# use the base output path to derive the path to the serialized
# model along with training history plot
MODEL\_PATH = os.path.sep.join(\[BASE\_OUTPUT,
	"contrastive\_siamese\_model"\])
PLOT\_PATH = os.path.sep.join(\[BASE\_OUTPUT,
	"contrastive\_plot.png"\])

**Line 5** sets our `` `IMG_SHAPE` `` dimensions. We’ll be working with the MNIST digits dataset, which has _28×28_ grayscale (i.e., single channel) images.

We then set our `BATCH_SIZE` and number of `EPOCHS` to train before. These parameters were experimentally tuned.

**Lines 16-19** define the output file paths for both our serialized model and training history.

For more details on the configuration file, refer to my tutorial on _[Siamese networks with Keras, TensorFlow, and Deep Learning.](https://pyimagesearch.com/2020/11/30/siamese-networks-with-keras-tensorflow-and-deep-learning/)_

### **Creating our helper utility functions**

[![Image 6](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20605%20404'%3E%3C/svg%3E)](https://pyimagesearch.com/2020/11/23/building-image-pairs-for-siamese-networks-with-python/)

**Figure 4:** In order to train our siamese network, we need to generate positive and negative image pairs.

In order to train our siamese network model, we’ll need three helper utilities:

1.  `` `make_pairs` ``: Generates a set of [image pairs](https://pyimagesearch.com/2020/11/23/building-image-pairs-for-siamese-networks-with-python/) from the MNIST dataset that will serve as our training set
2.  `` `euclidean_distance` ``: A custom layer implementation that computes the Euclidean distance between two volumes _inside_ of a CNN
3.  `plot_training`: Plots the training and validation contrastive loss over the course of the training process

Let’s start off with our imports:

\# import the necessary packages
import tensorflow.keras.backend as K
import matplotlib.pyplot as plt
import numpy as np

We then have our `make_pairs` function, which I discussed in detail in my [_Building image pairs for siamese networks with Python_ tutorial](https://pyimagesearch.com/2020/11/23/building-image-pairs-for-siamese-networks-with-python/) (make sure you read that guide before continuing):

def make\_pairs(images, labels):
	# initialize two empty lists to hold the (image, image) pairs and
	# labels to indicate if a pair is positive or negative
	pairImages = \[\]
	pairLabels = \[\]

	# calculate the total number of classes present in the dataset
	# and then build a list of indexes for each class label that
	# provides the indexes for all examples with a given label
	numClasses = len(np.unique(labels))
	idx = \[np.where(labels == i)\[0\] for i in range(0, numClasses)\]

	# loop over all images
	for idxA in range(len(images)):
		# grab the current image and label belonging to the current
		# iteration
		currentImage = images\[idxA\]
		label = labels\[idxA\]

		# randomly pick an image that belongs to the \*same\* class
		# label
		idxB = np.random.choice(idx\[label\])
		posImage = images\[idxB\]

		# prepare a positive pair and update the images and labels
		# lists, respectively
		pairImages.append(\[currentImage, posImage\])
		pairLabels.append(\[1\])

		# grab the indices for each of the class labels \*not\* equal to
		# the current label and randomly pick an image corresponding
		# to a label \*not\* equal to the current label
		negIdx = np.where(labels != label)\[0\]
		negImage = images\[np.random.choice(negIdx)\]

		# prepare a negative pair of images and update our lists
		pairImages.append(\[currentImage, negImage\])
		pairLabels.append(\[0\])

	# return a 2-tuple of our image pairs and labels
	return (np.array(pairImages), np.array(pairLabels))

1.  In order to train siamese networks, we need examples of _positive_ and _negative_ image pairs
2.  A **positive pair** is two images that belong to the _same_ class (i.e., two examples of the digit _“8”_)
3.  A **negative pair** is two images that belong to _different_ classes (i.e., one image containing a _“1”_ and the other image containing a _“3”_)
4.  The `` `make_pairs` `` function accepts an input set of `images` and associated `labels` and then constructs the positive and negative image pairs

The next function, `euclidean_distance`, accepts a 2-tuple of `vectors` and then computes the Euclidean distance between them, utilizing Keras/TensorFlow functions such that the Euclidean distance can be computed _inside_ the siamese neural network:

def euclidean\_distance(vectors):
	# unpack the vectors into separate lists
	(featsA, featsB) = vectors

	# compute the sum of squared distances between the vectors
	sumSquared = K.sum(K.square(featsA - featsB), axis=1,
		keepdims=True)

	# return the euclidean distance between the vectors
	return K.sqrt(K.maximum(sumSquared, K.epsilon()))

Finally, we have a helper utility, `plot_training`, which accepts a `plotPath`, plots our training and validation contrastive loss over the course of training, and then saves the plot to disk:

def plot\_training(H, plotPath):
	# construct a plot that plots and saves the training history
	plt.style.use("ggplot")
	plt.figure()
	plt.plot(H.history\["loss"\], label="train\_loss")
	plt.plot(H.history\["val\_loss"\], label="val\_loss")
	plt.title("Training Loss")
	plt.xlabel("Epoch #")
	plt.ylabel("Loss")
	plt.legend(loc="lower left")
	plt.savefig(plotPath)

Let’s move on to implementing the siamese network architecture itself.

### **Implementing our siamese network architecture**

[![Image 7](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20605%20375'%3E%3C/svg%3E)](https://pyimagesearch.com/2020/11/30/siamese-networks-with-keras-tensorflow-and-deep-learning/)

**Figure 5:** Siamese networks with Keras and TensorFlow.

Our siamese neural network architecture is essentially a basic CNN:

\# import the necessary packages
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import GlobalAveragePooling2D
from tensorflow.keras.layers import MaxPooling2D

def build\_siamese\_model(inputShape, embeddingDim=48):
	# specify the inputs for the feature extractor network
	inputs = Input(inputShape)

	# define the first set of CONV => RELU => POOL => DROPOUT layers
	x = Conv2D(64, (2, 2), padding="same", activation="relu")(inputs)
	x = MaxPooling2D(pool\_size=(2, 2))(x)
	x = Dropout(0.3)(x)

	# second set of CONV => RELU => POOL => DROPOUT layers
	x = Conv2D(64, (2, 2), padding="same", activation="relu")(x)
	x = MaxPooling2D(pool\_size=2)(x)
	x = Dropout(0.3)(x)

	# prepare the final outputs
	pooledOutput = GlobalAveragePooling2D()(x)
	outputs = Dense(embeddingDim)(pooledOutput)

	# build the model
	model = Model(inputs, outputs)

	# return the model to the calling function
	return model

You can refer to my tutorial on _[Siamese networks with Keras, TensorFlow, and Deep Learning](https://pyimagesearch.com/2020/11/30/siamese-networks-with-keras-tensorflow-and-deep-learning/)_ for more details on the model architecture and implementation.

### **Implementing contrastive loss with Keras and TensorFlow**

With our helper utilities and model architecture implemented, we can move on to defining the `` `contrastive_loss` `` function in Keras/TensorFlow.

For reference, here is the equation for the contrastive loss function that we’ll be implementing in Keras/TensorFlow code:

![Image 8](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20109'%3E%3C/svg%3E)

**Figure 6:** Implementing the contrastive loss function with Keras and TensorFlow.

The full implementation of contrastive loss is concise, spanning only 18 lines, including comments:

\# import the necessary packages
import tensorflow.keras.backend as K
import tensorflow as tf

def contrastive\_loss(y, preds, margin=1):
	# explicitly cast the true class label data type to the predicted
	# class label data type (otherwise we run the risk of having two
	# separate data types, causing TensorFlow to error out)
	y = tf.cast(y, preds.dtype)

	# calculate the contrastive loss between the true labels and
	# the predicted labels
	squaredPreds = K.square(preds)
	squaredMargin = K.square(K.maximum(margin - preds, 0))
	loss = K.mean(y \* squaredPreds + (1 - y) \* squaredMargin)

	# return the computed contrastive loss to the calling function
	return loss

**Line 5** defines our `contrastive_loss` function, which accepts three arguments, two of which are required and the third optional:

1.  `y`: The ground-truth labels from our dataset. A value of `` `1` `` indicates that the two images in the pair are of the _same_ class, while a value of `` `0` `` indicates that the images belong to two _different_ classes.
2.  `` `preds` ``: The predictions from our siamese network (i.e., distances between the image pairs).
3.  `` `margin` ``: Margin used for the contrastive loss function (typically this value is set to `` `1` ``).

**Line 9** ensures our ground-truth labels are of the _same data type_ as our `` `preds` ``. Failing to perform this explicit casting may result in TensorFlow erroring out when we try to perform mathematical operations on `y` and `preds`.

We then proceed to compute the contrastive loss by:

1.  Taking the square of the `preds` (**Line 13**)
2.  Computing the `squaredMargin`, which is the square of the maximum value of either `0` _or_ `margin - preds` (**Line 14**)
3.  Computing the final `loss` (**Line 15**)

The computed contrastive `loss` value is then returned to the calling function.

I suggest you review the _“What is contrastive loss? And how can contrastive loss be used to train siamese networks?”_ section above and compare our implementation to the equation so you can better understand how contrastive loss is implemented.

### **Creating our contrastive loss training script**

We are now ready to implement our training script! This script is responsible for:

1.  Loading the MNIST digits dataset from disk
2.  Preprocessing it and constructing image pairs
3.  Instantiating the siamese neural network architecture
4.  Training the siamese network with contrastive loss
5.  Serializing both the trained network and training history plot to disk

The majority of this code is identical to our previous post on _[Siamese networks with Keras, TensorFlow, and Deep Learning](https://pyimagesearch.com/2020/11/30/siamese-networks-with-keras-tensorflow-and-deep-learning/)_, so while I’m still going to cover our implementation in full, I’m going to defer a detailed discussion to the previous post (and of course, pointing out the details along the way).

Open up the `` `train_contrastive_siamese_network.py` `` file in your project directory structure, and let’s get to work:

\# import the necessary packages
from pyimagesearch.siamese\_network import build\_siamese\_model
from pyimagesearch import metrics
from pyimagesearch import config
from pyimagesearch import utils
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Input
from tensorflow.keras.layers import Lambda
from tensorflow.keras.datasets import mnist
import numpy as np

**Lines 2-11** import our required Python packages. Note how we are importing the `` `metrics` `` submodule of `pyimagesearch`, which contains our `` `contrastive_loss` `` implementation.

From there we can load the MNIST dataset from disk:

\# load MNIST dataset and scale the pixel values to the range of \[0, 1\]
print("\[INFO\] loading MNIST dataset...")
(trainX, trainY), (testX, testY) = mnist.load\_data()
trainX = trainX / 255.0
testX = testX / 255.0

# add a channel dimension to the images
trainX = np.expand\_dims(trainX, axis=-1)
testX = np.expand\_dims(testX, axis=-1)

# prepare the positive and negative pairs
print("\[INFO\] preparing positive and negative pairs...")
(pairTrain, labelTrain) = utils.make\_pairs(trainX, trainY)
(pairTest, labelTest) = utils.make\_pairs(testX, testY)

**Line 15** loads the MNIST dataset with the pre-supplied training and testing splits.

We then preprocess the dataset by:

1.  Scaling the input pixel intensities in the images from the range _\[0, 255\]_ to _\[0, 1\]_ (**Lines 16 and 17**)
2.  Adding a channel dimension (**Lines 20 and 21**)
3.  Constructing our image pairs (**Lines 25 and 26**)

Next, we can instantiate the siamese network architecture:

\# configure the siamese network
print("\[INFO\] building siamese network...")
imgA = Input(shape=config.IMG\_SHAPE)
imgB = Input(shape=config.IMG\_SHAPE)
featureExtractor = build\_siamese\_model(config.IMG\_SHAPE)
featsA = featureExtractor(imgA)
featsB = featureExtractor(imgB)

# finally, construct the siamese network
distance = Lambda(utils.euclidean\_distance)(\[featsA, featsB\])
model = Model(inputs=\[imgA, imgB\], outputs=distance)

**Lines 30-34** create our sister networks:

*   We start by creating two inputs, one for each image in the image pair (**Lines 30 and 31**).
*   We then build the sister network architecture, which acts as our feature extractor (**Line 32**).
*   Each image in the pair will be passed through our feature extractor, resulting in a vector that quantifies each image (**Lines 33 and 34**).

Using the 48-d vector generated by the sister networks, we proceed to compute the `euclidean_distance` between our two vectors (**Line 37**) — this distance serves as our output from the siamese network:

*   The _smaller_ the distance is, the _more similar_ the two images are.
*   The _larger_ the distance is, the _less similar_ the images are.

**Line 38** defines the `model` by specifying `imgA` and `imgB`, our two images in the image pair, as inputs, and our `distance` layer as the output.

Finally, we can train our siamese network using contrastive loss:

\# compile the model
print("\[INFO\] compiling model...")
model.compile(loss=metrics.contrastive\_loss, optimizer="adam")

# train the model
print("\[INFO\] training model...")
history = model.fit(
	\[pairTrain\[:, 0\], pairTrain\[:, 1\]\], labelTrain\[:\],
	validation\_data=(\[pairTest\[:, 0\], pairTest\[:, 1\]\], labelTest\[:\]),
	batch\_size=config.BATCH\_SIZE,
	epochs=config.EPOCHS)

# serialize the model to disk
print("\[INFO\] saving siamese model...")
model.save(config.MODEL\_PATH)

# plot the training history
print("\[INFO\] plotting training history...")
utils.plot\_training(history, config.PLOT\_PATH)

**Line 42** compiles our model architecture using the `` `contrastive_loss` `` function.

We then proceed to train the model using our training/validation image pairs (**Lines 46-50**) and then serialize the model to disk (**Line 54**) and plot the training history (**Line 58**).

### **Training a siamese network with contrastive loss**

We are now ready to train our siamese neural network with contrastive loss using Keras and TensorFlow.

Make sure you use the **_“Downloads”_** section of this guide to download the source code, helper utilities, and contrastive loss implementation.

From there, you can execute the following command:

$ python train\_contrastive\_siamese\_network.py
\[INFO\] loading MNIST dataset...
\[INFO\] preparing positive and negative pairs...
\[INFO\] building siamese network...
\[INFO\] compiling model...
\[INFO\] training model...
Epoch 1/100
1875/1875 \[==============================\] - 81s 43ms/step - loss: 0.2038 - val\_loss: 0.1755
Epoch 2/100
1875/1875 \[==============================\] - 80s 43ms/step - loss: 0.1756 - val\_loss: 0.1571
Epoch 3/100
1875/1875 \[==============================\] - 80s 43ms/step - loss: 0.1619 - val\_loss: 0.1394
Epoch 4/100
1875/1875 \[==============================\] - 81s 43ms/step - loss: 0.1548 - val\_loss: 0.1356
Epoch 5/100
1875/1875 \[==============================\] - 81s 43ms/step - loss: 0.1501 - val\_loss: 0.1262
...
Epoch 96/100
1875/1875 \[==============================\] - 81s 43ms/step - loss: 0.1264 - val\_loss: 0.1066
Epoch 97/100
1875/1875 \[==============================\] - 80s 43ms/step - loss: 0.1262 - val\_loss: 0.1100
Epoch 98/100
1875/1875 \[==============================\] - 82s 44ms/step - loss: 0.1262 - val\_loss: 0.1078
Epoch 99/100
1875/1875 \[==============================\] - 81s 43ms/step - loss: 0.1268 - val\_loss: 0.1067
Epoch 100/100
1875/1875 \[==============================\] - 80s 43ms/step - loss: 0.1261 - val\_loss: 0.1107
\[INFO\] saving siamese model...
\[INFO\] plotting training history...

![Image 9](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20640%20480'%3E%3C/svg%3E)

**Figure 7:** Training our siamese network with contrastive loss.

Each epoch took ~80 seconds on my 3 GHz Intel Xeon W processor. Training would be _even faster_ with a GPU.

Our training history can be seen in **Figure 7.** Notice how our validation loss is actually _lower_ than our training loss, [a phenomenon that I discuss in this tutorial](https://pyimagesearch.com/2019/10/14/why-is-my-validation-loss-lower-than-my-training-loss/).

Having our validation loss lower than our training loss implies that we can “train harder” to improve our siamese network accuracy, typically by relaxing regularization constraints, deepening the model, and using a more aggressive learning rate.

But for now, our training model is more than sufficient.

### **Implementing our contrastive loss test script**

The final script we need to implement is `` `test_contrastive_siamese_network.py` ``. This script is essentially _identical_ to the one covered in our previous tutorial on _[Comparing images for similarity using siamese networks, Keras, and TensorFlow,](https://pyimagesearch.com/2020/12/07/comparing-images-for-similarity-using-siamese-networks-keras-and-tensorflow/)_ so while I’ll still cover the script in its entirety today, I’ll defer a detailed discussion to my previous guide.

Let’s get started:

\# import the necessary packages
from pyimagesearch import config
from pyimagesearch import utils
from tensorflow.keras.models import load\_model
from imutils.paths import list\_images
import matplotlib.pyplot as plt
import numpy as np
import argparse
import cv2

**Lines 2-9** import our required Python packages.

We’ll be using `` `load_model` `` to load our serialized siamese network from disk. The `` `list_images` `` function will be used to grab image paths and facilitate building sample image pairs.

Let’s move on to our command line arguments:

\# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add\_argument("-i", "--input", required=True,
	help="path to input directory of testing images")
args = vars(ap.parse\_args())

The only command line argument we need is `--input`, the path to our directory containing sample images we want to build pairs from (i.e., the `examples` directory in our project directory).

Speaking of building image pairs, let’s do that now:

\# grab the test dataset image paths and then randomly generate a
# total of 10 image pairs
print("\[INFO\] loading test dataset...")
testImagePaths = list(list\_images(args\["input"\]))
np.random.seed(42)
pairs = np.random.choice(testImagePaths, size=(10, 2))

# load the model from disk
print("\[INFO\] loading siamese model...")
model = load\_model(config.MODEL\_PATH, compile=False)

**Line 20** grabs the paths to all images in our `--input` directory. We then randomly generate a total of 10 pairs of images (**Line 22**).

**Line 26** loads our trained siamese network from disk.

With the siamese network loaded from disk, we can now compare images:

\# loop over all image pairs
for (i, (pathA, pathB)) in enumerate(pairs):
	# load both the images and convert them to grayscale
	imageA = cv2.imread(pathA, 0)
	imageB = cv2.imread(pathB, 0)

	# create a copy of both the images for visualization purpose
	origA = imageA.copy()
	origB = imageB.copy()

	# add channel a dimension to both the images
	imageA = np.expand\_dims(imageA, axis=-1)
	imageB = np.expand\_dims(imageB, axis=-1)

	# add a batch dimension to both images
	imageA = np.expand\_dims(imageA, axis=0)
	imageB = np.expand\_dims(imageB, axis=0)

	# scale the pixel values to the range of \[0, 1\]
	imageA = imageA / 255.0
	imageB = imageB / 255.0

	# use our siamese model to make predictions on the image pair,
	# indicating whether or not the images belong to the same class
	preds = model.predict(\[imageA, imageB\])
	proba = preds\[0\]\[0\]

**Line 29** loops over all `pairs`. For each pair, we:

1.  Load the two images from disk (**Lines 31 and 32**)
2.  Clone the images such that we can visualize/draw on them (**Lines 35 and 36**)
3.  Add a channel dimension to both images, a requirement for inference (**Lines 39 and 40**)
4.  Add a batch dimension to the images, again, a requirement for inference (**Lines 43 and 44**)
5.  Scale the pixel intensities from the range _\[0, 255\]_ to _\[0, 1\]_, just like we did during training

The image pairs are then passed through our siamese network on **Lines 52 and 53**, resulting in the computed Euclidean distance between the vectors generated by the sister networks.

**Again, keep in mind that the _smaller_ the distance is, the _more similar_ the two images are. Conversely, the _larger_ the distance, the _less similar_ the images are.**

The final code block handles visualizing the two images in the pair along with their computed distance:

	# initialize the figure
	fig = plt.figure("Pair #{}".format(i + 1), figsize=(4, 2))
	plt.suptitle("Distance: {:.2f}".format(proba))

	# show first image
	ax = fig.add\_subplot(1, 2, 1)
	plt.imshow(origA, cmap=plt.cm.gray)
	plt.axis("off")

	# show the second image
	ax = fig.add\_subplot(1, 2, 2)
	plt.imshow(origB, cmap=plt.cm.gray)
	plt.axis("off")

	# show the plot
	plt.show()

Congratulations on implementing an inference script for siamese networks! For more details on this implementation, refer to my previous tutorial, _[Comparing images for similarity using siamese networks, Keras, and TensorFlow.](https://pyimagesearch.com/2020/12/07/comparing-images-for-similarity-using-siamese-networks-keras-and-tensorflow/)_

### **Making predictions using our siamese network with contrastive loss model**

Let’s put our `` `test_contrastive_siamse_network.py` `` script to work. Make sure you use the **_“Downloads”_** section of this tutorial to download the source code, pre-trained model, and example images.

From there, you can run the following command:

$ python test\_contrastive\_siamese\_network.py --input examples
\[INFO\] loading test dataset...
\[INFO\] loading siamese model...

![Image 10](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20525%20656'%3E%3C/svg%3E)

**Figure 8**: Results of applying our siamese network inference script. Image pairs with _smaller distances_ are considered to belong to the _same class,_ while image pairs with _larger distances_ belong to _different classes._

Looking at **Figure 8**, you’ll see that we have sets of example image pairs presented to our siamese network trained with contrastive loss.

**Images that are of the _same class_ have _lower distances_ while images of _different classes_ have _larger classes._**

You can thus set a threshold value, _T_, to act as a cutoff on distance. If the computed distance, _D_, is _< T_, then the image pair must belong to the same class. Otherwise, if _D >= T_, then the images are _different_ classes.

Setting the threshold _T_ should be done empirically through experimentation:

*   Train the network.
*   Compute distances for image pairs.
*   Manually visualize the pairs and their corresponding differences.
*   Find a cutoff value that maximizes correct classifications and minimizes incorrect ones.

In this case, setting _T=0.16_ would be an appropriate threshold, since it allows us to correctly mark all image pairs that belong to the same class, while all image pairs of _different_ classes are treated as such.

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

**Summary**
-----------

In this tutorial, you learned about contrastive loss, including how it’s a better loss function than binary cross-entropy for training siamese networks.

What you need to keep in mind here is that a siamese network isn’t specifically designed for _classification._ Instead, it’s utilized for _differentiation_, meaning that it should not only be able to tell if an image pair belongs to the same class or not but whether the two images are _identical/similar_ or not.

Contrastive loss works far better in this situation.

I recommend you experiment with both binary cross-entropy and contrastive loss when training your own siamese neural networks, but I think you’ll find that overall, contrastive loss does a much better job.

**To download the source code to this post (and be notified when future tutorials are published here on PyImageSearch), _simply enter your email address in the form below!_**

![Image 11](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!