Title: Fire and smoke detection with Keras and Deep Learning - PyImageSearch

URL Source: https://pyimagesearch.com/2019/11/18/fire-and-smoke-detection-with-keras-and-deep-learning/

Published Time: 2019-11-18T15:00:31+00:00

Markdown Content:
![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20500'%3E%3C/svg%3E)

In this tutorial, you will learn how to detect fire and smoke using Computer Vision, OpenCV, and the Keras Deep Learning library.

Today’s tutorial is inspired by an email I received last week from PyImageSearch reader, Daniel.

Daniel writes:

> Hey Adrian, I’m not sure if you’ve seen the news, but my home state of California has been absolutely ravaged by wildfires over the past few weeks.
> 
> My family lives in the Los Angeles area, not too far from the Getty fire. It’s hard not to be concerned about our home and our safety.
> 
> It’s a scary situation and it got me thinking:
> 
> **Do you think computer vision could be used to detect wildfires? What about fires that start in people’s homes?**
> 
> If you could write a tutorial on the topic I would appreciate it. I’d love to learn from it and do my part to help others.

The short answer is, **_yes_, computer vision and deep learning _can_ be used to detect wildfires:**

*   IoT/Edge devices equipped with cameras can be deployed strategically throughout hillsides, ridges, and high elevation areas, automatically monitoring for signs of smoke or fire.
*   Drones and quadcopters can be flown above areas prone to wildfires, strategically scanning for smoke.
*   Satellites can be used to take photos of large acreage areas while computer vision and deep learning algorithms process these images, looking for signs of smoke.

That’s all fine and good for wildfires — but what if you wanted to monitor your own home for smoke or fire?

**The answer there is to _augment existing sensors_ to aid in fire/smoke detection:**

*   Existing smoke detectors utilize **photoelectric sensors** and a light source to detect if the light source particles are being scattered (implying smoke is present).
*   You could then distribute **temperature sensors** around the house to monitor the temperature of each room.
*   **Cameras** could also be placed in areas where fires are likely to start (kitchen, garage, etc.).
*   Each individual sensor could be used to trigger an alarm or you could relay the sensor information to a central hub that aggregates and analyzes the sensor data, computing a probability of a home fire.

**Unfortunately, that’s all easier said than done.**

While there are 100s of computer vision/deep learning practitioners around the world actively working on fire and smoke detection (including [PyImageSearch Gurus](https://pyimagesearch.com/pyimagesearch-gurus/) member, [David Bonn](https://www.linkedin.com/in/david-bonn-427449b/)), it’s still an open-ended problem.

That said, today I’ll help you get your start in smoke and fire detection — by the end of this tutorial, you’ll have a deep learning model capable of detecting fire in images (I’ve even included my pre-trained model to get you up and running immediately).

**To learn how to create your own fire and smoke detector with Computer Vision, Deep Learning, and Keras, _just keep reading!_**

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

[![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20407'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2019/11/keras_fire_detection_wildfire.jpg)

**Figure 1:** Wildfires can quickly become out of control and endanger lives in many parts of the world. In this article, we will learn to conduct fire and smoke detection with Keras and deep learning.

In the first part of this tutorial we’ll discuss the two datasets we’ll be using for fire and smoke detection.

From there we’ll review or directory structure for the project and then implement `FireDetectionNet`, the CNN architecture we’ll be using to detect fire and smoke in images/video.

Next, we’ll train our fire detection model and analyze the classification accuracy and results.

We’ll wrap up the tutorial by discussing some of the limitations and drawbacks of the approach, including how you can improve and extend the method.

### Our fire and smoke dataset

[![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20443'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2019/11/keras_fire_detection_dataset.jpg)

**Figure 2:** Today’s fire detection dataset is curated by [Gautam Kumar](https://www.linkedin.com/in/gautam-kumar-65399150/) and pruned by [David Bonn](https://www.linkedin.com/in/david-bonn-427449b/) (both of whom are PyImageSearch readers). We will put the dataset to work with Keras and deep learning to create a fire/smoke detector.

The dataset we’ll be using for fire and smoke examples was curated by PyImageSearch reader, [Gautam Kumar](https://www.linkedin.com/in/gautam-kumar-65399150/).

Guatam gathered a total of **1,315 images** by [searching Google Images](https://pyimagesearch.com/2017/12/04/how-to-create-a-deep-learning-dataset-using-google-images/) for queries related to the term _“fire”_, _“smoke”_, etc.

However, the original dataset has not been cleansed of extraneous, irrelevant images that are _not_ related to fire and smoke (i.e., examples of famous buildings _before_ a fire occurred).

Fellow PyImageSearch reader, [David Bonn](https://www.linkedin.com/in/david-bonn-427449b/), took the time to manually go through the fire/smoke images and identify ones that should _not_ be included.

_**Note:** _I took the list of extraneous images identified by David and then created a shell script to delete them from the dataset. The shell script can be found in the **_“Downloads”_** section of this tutorial.__

### The 8-scenes dataset

[![Image 5](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20295'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2019/11/keras_fire_detection_8scenes.jpg)

**Figure 3:** We will combine Gautam’s fire dataset with the [8-scenes](https://people.csail.mit.edu/torralba/code/spatialenvelope/) natural image dataset so that we can classify Fire vs. Non-fire using Keras and deep learning.

The dataset we’ll be using for _Non-fire_ examples is called [**8-scenes**](https://people.csail.mit.edu/torralba/code/spatialenvelope/) as it contains **2,688 image** examples belonging to eight natural scene categories (all without fire):

1.  Coast
2.  Mountain
3.  Forest
4.  Open country
5.  Street
6.  Inside city
7.  Tall buildings
8.  Highways

The dataset was originally curated by Oliva and Torralba in their 2001 paper, [_Modeling the shape of the scene: a holistic representation of the spatial envelope._](https://people.csail.mit.edu/torralba/code/spatialenvelope/)

The 8-scenes dataset is a natural complement to our fire/smoke dataset as it depicts natural scenes as they _should_ look without fire or smoke present.

While this dataset has 8 unique classes, **we will consider the dataset as a single _Non-fire_ class** when we combine it with Gautam’s _Fire_ dataset.

### Project structure

[![Image 6](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20465'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2019/11/keras_fire_detection_tree.png)

**Figure 4:** The project structure for today’s tutorial on fire and smoke detection with deep learning using the Keras/TensorFlow framework.

Go ahead and grab today’s .zip from the source code and pre-trained model using the _**“Downloads”**_ section of this blog post.

From there you can unzip it on your machine and your project will look like **Figure 4**. There is _**an exception**_: neither dataset .zip (_white arrows_) will be present yet. We will download, extract, and prune the datasets in the _next section_.

Our `output/` directory contains:

*   Our serialized fire detection model. We will train the model today with Keras and deep learning.
*   The [Learning Rate Finder](https://pyimagesearch.com/2019/08/05/keras-learning-rate-finder/) plot will be generated and inspected for the optimal learning rate prior to training.
*   A training history plot will be generated upon completion of the training process.
*   The `examples/` subdirectory will be populated by `predict_fire.py` with sample images that will be annotated for demonstration and verification purposes.

Our `pyimagesearch` module holds:

*   `config.py` : Our customizable configuration.
*   `FireDetectionNet` : Our Keras Convolutional Neural Network class designed specifically for detecting fire and smoke.
*   `LearningRateFinder` : A Keras class for assisting in the process of [finding the optimal learning rate for deep learning](https://pyimagesearch.com/2019/08/05/keras-learning-rate-finder/) training.

The root of the project contains three scripts:

*   `prune.sh` : A simple bash script that removes irrelevant images from Gautam’s fire dataset.
*   `train.py` : Our Keras deep learning training script. This script has two modes of operation: (1) [Learning Rate Finder](https://pyimagesearch.com/2019/08/05/keras-learning-rate-finder/) mode,and (2) training mode.
*   `predict_fire.py` : A quick and dirty script which samples images from our dataset, generating annotated _Fire/Non-fire_ images for verification.

Let’s move on to preparing our _Fire/Non-fire_ dataset in the next section.

### Preparing our _Fire_ and _Non-fire_ combined dataset

Preparing our _Fire_ and _Non-fire_ dataset involves a four-step process:

1.  **Step #1:** Ensure you followed the instructions in the _previous section_ to grab and unzip today’s files from the _**“Downloads”**_ section.
2.  **Step #2**: Download and extract the fire/smoke dataset into the project.
3.  **Step #3:** Prune the fire/smoke dataset for extraneous, irrelevant files.
4.  **Step #4:** Download and extract the 8-scenes dataset into the project.

The result of **Steps #2-4** will be a dataset consisting of two classes:

*   _Fire_
*   _Non-fire_

Combining datasets is a tactic I often use. It saves valuable time and often leads to a great model.

Let’s begin putting our combined dataset together.

**Step #2**: Download and extract the fire/smoke dataset into the project.

**[Download the fire/smoke dataset using this link](https://drive.google.com/file/d/11KBgD_W2yOxhJnUMiyBkBzXDPXhVmvCt/view?usp=sharing).** Store the .zip in the `keras-fire-detection/` project directory that you extracted in the last section.

Once downloaded, unzip the dataset:

$ unzip Robbery\_Accident\_Fire\_Database2.zip

**Step #3:** Prune the dataset for extraneous, irrelevant files.

Execute the `prune.sh` script to delete the extraneous, irrelevant files from the fire dataset:

$ sh prune.sh

At this point, we have _Fire_ data. Now we need _Non-fire_ data for our two-class problem.

**Step #4:** Download and extract the 8-scenes dataset into the project.

[**Download the 8-scenes dataset using this link**](https://people.csail.mit.edu/torralba/code/spatialenvelope/spatial_envelope_256x256_static_8outdoorcategories.zip)**.** Store the .zip in the `keras-fire-detection/` project directory alongside the _Fire_ dataset.

Once downloaded, navigate to the project folder and unarchive the dataset:

$ unzip spatial\_envelope\_256x256\_static\_8outdoorcategories.zip

**Review Project + Dataset Structure**

At this point, it is time to inspect our directory structure once more. Yours should be identical to mine:

$ tree --dirsfirst --filelimit 10
.
├── Robbery\_Accident\_Fire\_Database2
│   ├── Accident \[887 entries\]
│   ├── Fire \[1315 entries\]
│   ├── Robbery \[2073 entries\]
│   └── readme.txt
├── spatial\_envelope\_256x256\_static\_8outdoorcategories \[2689 entries\]
├── output
│   ├── examples \[48 entries\]
│   ├── fire\_detection.model
│   │   ├── assets
│   │   ├── variables
│   │   │   ├── variables.data-00000-of-00002
│   │   │   ├── variables.data-00001-of-00002
│   │   │   └── variables.index
│   │   └── saved\_model.pb
│   ├── lrfind\_plot.png
│   └── training\_plot.png
├── pyimagesearch
│   ├── \_\_init\_\_.py
│   ├── config.py
│   ├── firedetectionnet.py
│   └── learningratefinder.py
├── Robbery\_Accident\_Fire\_Database.zip
├── spatial\_envelope\_256x256\_static\_8outdoorcategories.zip
├── prune.sh
├── train.py
└── predict\_fire.py

11 directories, 16 files

Ensure your dataset is pruned (i.e. the `Fire/` directory should have exactly **1,315 entries** and _not_ the previous 1,405 entries).

### Our configuration file

This project will span multiple Python files that will need to be executed, so let’s store all important variables in a single `config.py` file.

Open up `config.py` now and insert the following code:

\# import the necessary packages
import os

# initialize the path to the fire and non-fire dataset directories
FIRE\_PATH = os.path.sep.join(\["Robbery\_Accident\_Fire\_Database2",
	"Fire"\])
NON\_FIRE\_PATH = "spatial\_envelope\_256x256\_static\_8outdoorcategories"

# initialize the class labels in the dataset
CLASSES = \["Non-Fire", "Fire"\]

We’ll use the `os` module for combining paths (**Line 2**).

**Lines 5-7** contain paths to our (1) _Fire_ images, and (2) _Non-fire_ images.

**Line 10** is a list of our two class names.

Let’s set a handful of training parameters:

\# define the size of the training and testing split
TRAIN\_SPLIT = 0.75
TEST\_SPLIT = 0.25

# define the initial learning rate, batch size, and number of epochs
INIT\_LR = 1e-2
BATCH\_SIZE = 64
NUM\_EPOCHS = 50

**Lines 13 and 14** define the size of our training and testing dataset splits.

**Lines 17-19** contain three hyperparameters — the initial learning rate, batch size, and number of epochs to train for.

From here, we’ll define a few paths:

\# set the path to the serialized model after training
MODEL\_PATH = os.path.sep.join(\["output", "fire\_detection.model"\])

# define the path to the output learning rate finder plot and
# training history plot
LRFIND\_PLOT\_PATH = os.path.sep.join(\["output", "lrfind\_plot.png"\])
TRAINING\_PLOT\_PATH = os.path.sep.join(\["output", "training\_plot.png"\])

**Lines 22-27** include paths to:

*   Our yet-to-be-trained serialized fire detection model.
*   The [Learning Rate Finder](https://pyimagesearch.com/2019/08/05/keras-learning-rate-finder/) plot which we will analyze to set our initial learning rate.
*   A training accuracy/loss history plot.

To wrap up our config we’ll define settings for prediction spot-checking:

\# define the path to the output directory that will store our final
# output with labels/annotations along with the number of images to
# sample
OUTPUT\_IMAGE\_PATH = os.path.sep.join(\["output", "examples"\])
SAMPLE\_SIZE = 50

Our prediction script will sample and annotate images using our model.

**Lines 32 and 33** include the path to output directory where we’ll store output classification results and the number of images to sample.

### Implementing our fire detection Convolutional Neural Network

[![Image 7](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20401'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2019/11/keras_fire_detection_arch.jpg)

**Figure 5:** `FireDetectionNet` is a deep learning fire/smoke classification network built with the Keras deep learning framework.

In this section we’ll implement `FireDetectionNet`, a Convolutional Neural Network used to detect smoke and fire in images.

This network utilizes **depthwise separable convolution** rather than standard convolution as depthwise separable convolution:

*   **Is more efficient,** as Edge/IoT devices will have limited CPU and power draw.
*   **Requires less memory,** as again, Edge/IoT devices have limited RAM.
*   **Requires less computation,** as we have limited CPU horsepower.
*   **Can perform better than standard convolution in some cases,** which can lead to a better fire/smoke detector.

Let’s get started implementing `FireDetectioNet` now — open up the `firedetectionnet.py` file now and insert the following code:

\# import the necessary packages
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import BatchNormalization
from tensorflow.keras.layers import SeparableConv2D
from tensorflow.keras.layers import MaxPooling2D
from tensorflow.keras.layers import Activation
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import Dense

class FireDetectionNet:
	@staticmethod
	def build(width, height, depth, classes):
		# initialize the model along with the input shape to be
		# "channels last" and the channels dimension itself
		model = Sequential()
		inputShape = (height, width, depth)
		chanDim = -1

Our [TensorFlow 2.0](https://pyimagesearch.com/2019/10/21/keras-vs-tf-keras-whats-the-difference-in-tensorflow-2-0/) Keras imports span from **Lines 2-9**. We will use [Keras’ Sequential API](https://pyimagesearch.com/2019/10/28/3-ways-to-create-a-keras-model-with-tensorflow-2-0-sequential-functional-and-model-subclassing/) to build our fire detection CNN.

**Line 11** defines our `FireDetectionNet` class. We begin by defining the `build` method on **Line 13**.

The `build` method accepts parameters including dimensions of our images (`width` , `height` , `depth` ) as well as the number of `classes` we will be training our model to recognize (i.e. this parameter affects the softmax classifier head shape).

We then initialize the `model` and `inputShape` (**Lines 16-18**).

From here we’ll define our first set of `CONV => RELU => POOL` layers:

		# CONV => RELU => POOL
		model.add(SeparableConv2D(16, (7, 7), padding="same",
			input\_shape=inputShape))
		model.add(Activation("relu"))
		model.add(BatchNormalization(axis=chanDim))
		model.add(MaxPooling2D(pool\_size=(2, 2)))

These layers use a larger kernel size to both (1) reduce the input volume spatial dimensions faster, and (2) detect larger color blobs that contain fire.

We’ll then define more `CONV => RELU => POOL` layer sets:

		# CONV => RELU => POOL
		model.add(SeparableConv2D(32, (3, 3), padding="same"))
		model.add(Activation("relu"))
		model.add(BatchNormalization(axis=chanDim))
		model.add(MaxPooling2D(pool\_size=(2, 2)))

		# (CONV => RELU) \* 2 => POOL
		model.add(SeparableConv2D(64, (3, 3), padding="same"))
		model.add(Activation("relu"))
		model.add(BatchNormalization(axis=chanDim))
		model.add(SeparableConv2D(64, (3, 3), padding="same"))
		model.add(Activation("relu"))
		model.add(BatchNormalization(axis=chanDim))
		model.add(MaxPooling2D(pool\_size=(2, 2)))

**Lines 34-40** allow our model to learn richer features by stacking _two_ sets of `CONV => RELU` before applying a `POOL` .

From here we’ll create our fully-connected head of the network:

		# first set of FC => RELU layers
		model.add(Flatten())
		model.add(Dense(128))
		model.add(Activation("relu"))
		model.add(BatchNormalization())
		model.add(Dropout(0.5))

		# second set of FC => RELU layers
		model.add(Dense(128))
		model.add(Activation("relu"))
		model.add(BatchNormalization())
		model.add(Dropout(0.5))

		# softmax classifier
		model.add(Dense(classes))
		model.add(Activation("softmax"))

		# return the constructed network architecture
		return model

**Lines 43-53** add two sets of `FC => RELU` layers.

**Lines 56 and 57** append our Softmax classifier prior to **Line 60** returning the `model` .

### Creating our training script

Our training script will be responsible for:

1.  Loading our _Fire_ and _Non-fire_ combined dataset from disk.
2.  Instantiating our `FireDetectionNet` architecture.
3.  [Finding our optimal learning rate](https://pyimagesearch.com/2019/08/05/keras-learning-rate-finder/) by using our `LearningRateFinder` class.
4.  Taking the optimal learning rate and training our network for the full set of epochs.

Let’s get started!

Open up the `train.py` file in your directory structure and insert the following code:

\# set the matplotlib backend so figures can be saved in the background
import matplotlib
matplotlib.use("Agg")

# import the necessary packages
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.optimizers import SGD
from tensorflow.keras.utils import to\_categorical
from sklearn.model\_selection import train\_test\_split
from sklearn.metrics import classification\_report
from pyimagesearch.learningratefinder import LearningRateFinder
from pyimagesearch.firedetectionnet import FireDetectionNet
from pyimagesearch import config
from imutils import paths
import matplotlib.pyplot as plt
import numpy as np
import argparse
import cv2
import sys

**Lines 1-19** handle our imports:

*   `matplotlib` : For generating plots with Python. **Line 3** sets the backend so we can save our plots as image files.
*   `tensorflow.keras` : Our [TensorFlow 2.0 imports](https://pyimagesearch.com/2019/10/21/keras-vs-tf-keras-whats-the-difference-in-tensorflow-2-0/) including data augmentation, stochastic gradient descent optimizer, and one-hot label encoder.
*   `sklearn` : Two imports for dataset splitting and classification reporting.
*   `LearningRateFinder` : A class we will use for finding an optimal learning rate _prior_ to training. When we operate our script in this mode, it will generate a plot for us to (1) manually inspect and (2) insert the optimal learning rate into our configuration file.
*   `FireDetectionNet` : The fire/smoke Convolutional Neural Network (CNN) that we built in the previous section.
*   `config` : Our configuration file of settings for this training script (it also contains settings for our prediction script).
*   `paths` : Contains functions from my [imutils](https://github.com/jrosebr1/imutils) package to list images in a directory tree.
*   `argparse` : For [parsing command line argument](https://pyimagesearch.com/2018/03/12/python-argparse-command-line-arguments/) flags.
*   `cv2` : OpenCV is used for loading and preprocessing images.

Now that we’ve imported packages, let’s define a reusable function to load our dataset:

def load\_dataset(datasetPath):
	# grab the paths to all images in our dataset directory, then
	# initialize our lists of images
	imagePaths = list(paths.list\_images(datasetPath))
	data = \[\]

	# loop over the image paths
	for imagePath in imagePaths:
		# load the image and resize it to be a fixed 128x128 pixels,
		# ignoring aspect ratio
		image = cv2.imread(imagePath)
		image = cv2.resize(image, (128, 128))

		# add the image to the data lists
		data.append(image)

	# return the data list as a NumPy array
	return np.array(data, dtype="float32")

Our `load_dataset` helper function assists with loading, preprocessing, and preparing both the _Fire_ and _Non-fire_ datasets.

**Line 21** defines the function which accepts a path to the dataset.

**Line 24** grabs all image paths in the dataset.

**Lines 28-35** loop over the `imagePaths` . Images are loaded, resized to _128×128_ dimensions, and added to the `data` list.

**Line 38** returns the `data` in NumPy array format.

We’ll now parse a single command line argument:

\# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add\_argument("-f", "--lr-find", type=int, default=0,
	help="whether or not to find optimal learning rate")
args = vars(ap.parse\_args())

The `--lr-find` flag sets the mode for our script. If the flag is set to `1` , then we’ll be in our learning rate finder mode, generating a learning rate plot for us to inspect. Otherwise, our script will operate in training mode and train the network for the full set of epochs (i.e. when the `--lr-find` flag is not present).

Let’s go ahead and load our `data` now:

\# load the fire and non-fire images
print("\[INFO\] loading data...")
fireData = load\_dataset(config.FIRE\_PATH)
nonFireData = load\_dataset(config.NON\_FIRE\_PATH)

# construct the class labels for the data
fireLabels = np.ones((fireData.shape\[0\],))
nonFireLabels = np.zeros((nonFireData.shape\[0\],))

# stack the fire data with the non-fire data, then scale the data
# to the range \[0, 1\]
data = np.vstack(\[fireData, nonFireData\])
labels = np.hstack(\[fireLabels, nonFireLabels\])
data /= 255

**Lines 48 and 49** load and resize the _Fire_ and _Non-fire_ images.

**Lines 52 and 53** construct labels for both classes (`1` for _Fire_ and `0` for _Non-fire_).

Subsequently, we stack the `data` and `labels` into a single NumPy array (i.e. combine the datasets) via **Lines 57 and 58**.

**Line 59** scales pixel intensities to the range _\[0, 1\]_.

We have three more steps to prepare our data:

\# perform one-hot encoding on the labels and account for skew in the
# labeled data
labels = to\_categorical(labels, num\_classes=2)
classTotals = labels.sum(axis=0)
classWeight = classTotals.max() / classTotals

# construct the training and testing split
(trainX, testX, trainY, testY) = train\_test\_split(data, labels,
	test\_size=config.TEST\_SPLIT, random\_state=42)

First, we perform one-hot encoding on our `labels` (**Line 63**).

Then, we **_account for skew in our dataset_** (**Lines 64 and 65**). To do so, we compute the `classWeight` to weight _Fire_ images more than _Non-fire_ images during the gradient update (as we have over **2x more** _Fire_ images than _Non-fire_ images).

**Lines 68 and 69** construct training and testing splits based on our config (in my config I have the split set to 75% training/25% testing).

Next, we’ll initialize data augmentation and compile our `FireDetectionNet` model:

\# initialize the training data augmentation object
aug = ImageDataGenerator(
	rotation\_range=30,
	zoom\_range=0.15,
	width\_shift\_range=0.2,
	height\_shift\_range=0.2,
	shear\_range=0.15,
	horizontal\_flip=True,
	fill\_mode="nearest")

# initialize the optimizer and model
print("\[INFO\] compiling model...")
opt = SGD(lr=config.INIT\_LR, momentum=0.9,
	decay=config.INIT\_LR / config.NUM\_EPOCHS)
model = FireDetectionNet.build(width=128, height=128, depth=3,
	classes=2)
model.compile(loss="binary\_crossentropy", optimizer=opt,
	metrics=\["accuracy"\])

**Lines 74-79** instantiate our [data augmentation](https://pyimagesearch.com/2019/07/08/keras-imagedatagenerator-and-data-augmentation/) object.

We then `build` and `compile` our `FireDetectionNet` model (**Lines 83-88**). Note that our initial learning rate and decay is set as we initialize our `SGD` optimizer.

Let’s handle our **[Learning Rate Finder](https://pyimagesearch.com/2019/08/05/keras-learning-rate-finder/) mode:**

\# check to see if we are attempting to find an optimal learning rate
# before training for the full number of epochs
if args\["lr\_find"\] > 0:
	# initialize the learning rate finder and then train with learning
	# rates ranging from 1e-10 to 1e+1
	print("\[INFO\] finding learning rate...")
	lrf = LearningRateFinder(model)
	lrf.find(
		aug.flow(trainX, trainY, batch\_size=config.BATCH\_SIZE),
		1e-10, 1e+1,
		stepsPerEpoch=np.ceil((trainX.shape\[0\] / float(config.BATCH\_SIZE))),
		epochs=20,
		batchSize=config.BATCH\_SIZE,
		classWeight=classWeight)

	# plot the loss for the various learning rates and save the
	# resulting plot to disk
	lrf.plot\_loss()
	plt.savefig(config.LRFIND\_PLOT\_PATH)

	# gracefully exit the script so we can adjust our learning rates
	# in the config and then train the network for our full set of
	# epochs
	print("\[INFO\] learning rate finder complete")
	print("\[INFO\] examine plot and adjust learning rates before training")
	sys.exit(0)

**Line 92** checks to see if we should attempt to find optimal learning rates. Assuming so, we:

*   Initialize `LearningRateFinder` (**Line 96**).
*   Start training with a `1e-10` learning rate and exponentially increase it until we hit `1e+1` (**Lines 97-103**).
*   Plot the loss vs. learning rate and save the resulting figure (**Lines 107 and 108**).
*   Gracefully `exit` the script after printing a couple of messages to the user (**Lines 115**).

After this code executes we now need to:

1.  **Step #1:** Manually inspect the generated learning rate plot.
2.  **Step #2:** Update `config.py` with our `INIT_LR` (i.e., the optimal learning rate we determined by analyzing the plot).
3.  **Step #3:** Train the network on our full dataset.

Assuming we have completed **Step #1** and **Step #2**, now let’s handle the **Step #3** where our initial learning rate has been determined and updated in the config. In this case, it is time to handle **training mode** in our script:

\# train the network
print("\[INFO\] training network...")
H = model.fit\_generator(
	aug.flow(trainX, trainY, batch\_size=config.BATCH\_SIZE),
	validation\_data=(testX, testY),
	steps\_per\_epoch=trainX.shape\[0\] // config.BATCH\_SIZE,
	epochs=config.NUM\_EPOCHS,
	class\_weight=classWeight,
	verbose=1)

**Lines 119-125** train our fire detection `model` using data augmentation and our skewed dataset class weighting. Be sure to [review my `.fit_generator` tutorial](https://pyimagesearch.com/2018/12/24/how-to-use-keras-fit-and-fit_generator-a-hands-on-tutorial/).

Finally, we’ll evaluate the model, serialize it to disk, and plot the training history:

\# evaluate the network and show a classification report
print("\[INFO\] evaluating network...")
predictions = model.predict(testX, batch\_size=config.BATCH\_SIZE)
print(classification\_report(testY.argmax(axis=1),
	predictions.argmax(axis=1), target\_names=config.CLASSES))

# serialize the model to disk
print("\[INFO\] serializing network to '{}'...".format(config.MODEL\_PATH))
model.save(config.MODEL\_PATH)

# construct a plot that plots and saves the training history
N = np.arange(0, config.NUM\_EPOCHS)
plt.style.use("ggplot")
plt.figure()
plt.plot(N, H.history\["loss"\], label="train\_loss")
plt.plot(N, H.history\["val\_loss"\], label="val\_loss")
plt.plot(N, H.history\["accuracy"\], label="train\_acc")
plt.plot(N, H.history\["val\_accuracy"\], label="val\_acc")
plt.title("Training Loss and Accuracy")
plt.xlabel("Epoch #")
plt.ylabel("Loss/Accuracy")
plt.legend(loc="lower left")
plt.savefig(config.TRAINING\_PLOT\_PATH)

**Lines 129-131** make predictions on test data and print a classification report in our terminal.

**Line 135** serializes the `model` and saves it to disk. We’ll recall the model in our prediction script.

**Lines 138-149** generate a historical plot of accuracy/loss curves during training. We will inspect this plot for overfitting or underfitting.

### Training the fire detection model with Keras

Training our fire detection model is broken down into three steps:

1.  **Step #1:** Run the `train.py` script with the `--lr-find` command line argument to find our optimal learning rate.
2.  **Step #2:** Update **Line 17** of our configuration file (`config.py` ) to set our `INIT_LR` value as the optimal learning rate.
3.  **Step #3:** Execute the `train.py` script again, but this time let it train for the full set of epochs.

Start by using the **_“Downloads”_** section of this tutorial to download the source code to this tutorial.

From there you can perform **Step #1** by executing the following command:

$ python train.py --lr-find 1
\[INFO\] loading data...
\[INFO\] finding learning rate...
Epoch 1/20
47/47 \[==============================\] - 10s 221ms/step - loss: 1.2949 - accuracy: 0.4923
Epoch 2/20
47/47 \[==============================\] - 11s 228ms/step - loss: 1.3315 - accuracy: 0.4897
Epoch 3/20
47/47 \[==============================\] - 10s 218ms/step - loss: 1.3409 - accuracy: 0.4860
Epoch 4/20
47/47 \[==============================\] - 10s 215ms/step - loss: 1.3973 - accuracy: 0.4770
Epoch 5/20
47/47 \[==============================\] - 10s 219ms/step - loss: 1.3170 - accuracy: 0.4957
...
Epoch 15/20
47/47 \[==============================\] - 10s 216ms/step - loss: 0.5097 - accuracy: 0.7728
Epoch 16/20
47/47 \[==============================\] - 10s 217ms/step - loss: 0.5507 - accuracy: 0.7345
Epoch 17/20
47/47 \[==============================\] - 10s 220ms/step - loss: 0.7554 - accuracy: 0.7089
Epoch 18/20
47/47 \[==============================\] - 10s 220ms/step - loss: 1.1833 - accuracy: 0.6606
Epoch 19/20
37/47 \[======================>.......\] - ETA: 2s - loss: 3.1446 - accuracy: 0.6338
\[INFO\] learning rate finder complete
\[INFO\] examine plot and adjust learning rates before training

[![Image 8](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20450'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2019/11/keras_fire_detection_lrfind.png)

**Figure 6:** Analyzing our optimal deep learning rate finder plot. We will use the optimal learning rate to train a fire/smoke detector using Keras and Python.

Examining **Figure 6** above you can see that our network is able to gain traction and start to learn around `1e-5` .

The lowest loss can be found between `1e-2` and `1e-1`; however, at `1e-1` we can see loss starting to increase sharply, implying that the learning rate is _too large_ and the network is _overfitting._

**To be safe we should use an initial learning rate of `1e-2`.**

Let’s now move on to **Step #2.**

Open up `config.py` and scroll to **Lines 16-19** where we set our training hyperparameters:

\# define the initial learning rate, batch size, and number of epochs
INIT\_LR = 1e-2
BATCH\_SIZE = 64
NUM\_EPOCHS = 50

Here we see our initial learning rate (`INIT_LR`) value — we need to set this value to `1e-2` (as our code indicates).

The final step (**Step #3**) is to train `FireDetectionNet` for the full set of `NUM_EPOCHS`:

$ python train.py
\[INFO\] loading data...
\[INFO\] compiling model...
\[INFO\] training network...
Epoch 1/50
46/46 \[==============================\] - 11s 233ms/step - loss: 0.6813 - accuracy: 0.6974 - val\_loss: 0.6583 - val\_accuracy: 0.6464
Epoch 2/50
46/46 \[==============================\] - 11s 232ms/step - loss: 0.4886 - accuracy: 0.7631 - val\_loss: 0.7774 - val\_accuracy: 0.6464
Epoch 3/50
46/46 \[==============================\] - 10s 224ms/step - loss: 0.4414 - accuracy: 0.7845 - val\_loss: 0.9470 - val\_accuracy: 0.6464
Epoch 4/50
46/46 \[==============================\] - 10s 222ms/step - loss: 0.4193 - accuracy: 0.7917 - val\_loss: 1.0790 - val\_accuracy: 0.6464
Epoch 5/50
46/46 \[==============================\] - 10s 224ms/step - loss: 0.4015 - accuracy: 0.8070 - val\_loss: 1.2034 - val\_accuracy: 0.6464
...
Epoch 46/50
46/46 \[==============================\] - 10s 222ms/step - loss: 0.1935 - accuracy: 0.9275 - val\_loss: 0.2985 - val\_accuracy: 0.8781
Epoch 47/50
46/46 \[==============================\] - 10s 221ms/step - loss: 0.1812 - accuracy: 0.9244 - val\_loss: 0.2325 - val\_accuracy: 0.9031
Epoch 48/50
46/46 \[==============================\] - 10s 226ms/step - loss: 0.1857 - accuracy: 0.9241 - val\_loss: 0.2788 - val\_accuracy: 0.8911
Epoch 49/50
46/46 \[==============================\] - 11s 229ms/step - loss: 0.2065 - accuracy: 0.9129 - val\_loss: 0.2177 - val\_accuracy: 0.9121
Epoch 50/50
46/46 \[==============================\] - 63s 1s/step - loss: 0.1842 - accuracy: 0.9316 - val\_loss: 0.2376 - val\_accuracy: 0.9111
\[INFO\] evaluating network...
              precision    recall  f1-score   support

    Non-Fire       0.96      0.90      0.93       647
        Fire       0.83      0.94      0.88       354

    accuracy                           0.91      1001
   macro avg       0.90      0.92      0.91      1001
weighted avg       0.92      0.91      0.91      1001

\[INFO\] serializing network to 'output/fire\_detection.model'...

[![Image 9](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20450'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2019/11/training_plot-1.png)

**Figure 7:** Accuracy/loss curves for training a fire and smoke detection deep learning model with Keras and Python.

Learning is a bit volatile here but you can see that we are obtaining **92% accuracy.**

### Making predictions on fire/non-fire images

Given our trained fire detection model, let’s now learn how to:

1.  Load the trained model from disk.
2.  Sample random images from our dataset.
3.  Classify each input image using our model.

Open up `predict_fire.py` and insert the following code:

\# import the necessary packages
from tensorflow.keras.models import load\_model
from pyimagesearch import config
from imutils import paths
import numpy as np
import imutils
import random
import cv2
import os

# load the trained model from disk
print("\[INFO\] loading model...")
model = load\_model(config.MODEL\_PATH)

**Lines 2-9** handle our imports, namely `load_model` , so that we can load our serialized TensorFlow/Keras model from disk.

Let’s grab 25 random images from our combined dataset:

\# grab the paths to the fire and non-fire images, respectively
print("\[INFO\] predicting...")
firePaths = list(paths.list\_images(config.FIRE\_PATH))
nonFirePaths = list(paths.list\_images(config.NON\_FIRE\_PATH))

# combine the two image path lists, randomly shuffle them, and sample
# them
imagePaths = firePaths + nonFirePaths
random.shuffle(imagePaths)
imagePaths = imagePaths\[:config.SAMPLE\_SIZE\]

**Lines 17 and 18** grab image paths from our combined dataset while **Lines 22-24** sample 25 random image paths.

From here, we’ll loop over each of the individual image paths and perform fire detection inference:

\# loop over the sampled image paths
for (i, imagePath) in enumerate(imagePaths):
	# load the image and clone it
	image = cv2.imread(imagePath)
	output = image.copy()

	# resize the input image to be a fixed 128x128 pixels, ignoring
	# aspect ratio
	image = cv2.resize(image, (128, 128))
	image = image.astype("float32") / 255.0
		
	# make predictions on the image
	preds = model.predict(np.expand\_dims(image, axis=0))\[0\]
	j = np.argmax(preds)
	label = config.CLASSES\[j\]

	# draw the activity on the output frame
	text = label if label == "Non-Fire" else "WARNING! Fire!"
	output = imutils.resize(output, width=500)
	cv2.putText(output, text, (35, 50), cv2.FONT\_HERSHEY\_SIMPLEX,
		1.25, (0, 255, 0), 5)

	# write the output image to disk	 
	filename = "{}.png".format(i)
	p = os.path.sep.join(\[config.OUTPUT\_IMAGE\_PATH, filename\])
	cv2.imwrite(p, output)

**Line 27** begins a loop over our sampled image paths:

*   We load and preprocess the image just as in training (**Lines 29-35**).
*   Make predictions and grab the highest probability label (**Lines 38-40**).
*   Annotate the label in the top corner of the image (**Lines 43-46**).
*   Save the output image to disk (**Lines 49-51**).

### Fire detection results

To see our fire detector in action make sure you use the **_“Downloads”_** section of this tutorial to download the source code and pre-trained model.

From there you can execute the following command:

$ python predict\_fire.py
\[INFO\] loading model...
\[INFO\] predicting...

![Image 10](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20500'%3E%3C/svg%3E)

**Figure 8:** Fire and smoke detection with Keras, deep learning, and Python.

I’ve included a set sample of results in **Figure 8** — notice how our model was able to correctly predict _“fire”_ and _“non-fire”_ in each of them.

### Limitations and drawbacks

Our results are not perfect, however. Here are a few examples of incorrect classifications:

[![Image 11](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20308'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2019/11/keras_fire_detection_incorrect.jpg)

**Figure 9:** Examples of incorrect fire/smoke detection.

The image on the _left_ in particular is troubling — a sunset will cast shades of reds and oranges across the sky, creating an “inferno” like effect. It appears that in those situations our fire detection model will struggle considerably.

So, why are these incorrect classifications coming from?

**The answer lies in the dataset itself.**

**To start, we only worked with raw image data.**

Smoke and fire can be better detected with **video** as fires start off as a smolder, slowly build to a critical point, and then erupt into massive flames. Such a pattern is better detected in video streams rather than images.

**Secondly, our datasets are quite small.**

Combining the two datasets we only had a total of 4,003 images. Fire and smoke datasets are hard to come by, making it extremely challenging to create high accuracy models.

**Finally, our datasets are not necessarily representative of the problem.**

Many of the example images in our fire/smoke dataset contained examples of professional photos captured by news reports. Fires don’t look like that in the wild.

In order to improve our fire and smoke detection model, _we need better data._

Future efforts in fire/smoke detection research should focus _less_ on the actual deep learning architectures/training methods and _more_ on the actual dataset gathering and curation process, **ensuring the dataset better represents how fires start, smolder, and spread in natural scene images.**

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

### Summary

In this tutorial, you learned how to create a smoke and fire detector using Computer Vision, Deep Learning, and the Keras library.

To build our smoke and fire detector we utilized two datasets:

*   **A dataset of fire/smoke examples** (1,315 images), curated by PyImageSearch readers [Gautam Kumar](https://www.linkedin.com/in/gautam-kumar-65399150/).
*   **A dataset of non-fire/non-smoke examples** (2,688 images) containing examples of 8 natural outdoor scenes (forests, coastlines, mountains, open country, etc.). This dataset was originally put together by Oliva and Torralba for their 2001 paper, [_Modeling the shape of the scene: a holistic representation of the spatial envelope_](https://people.csail.mit.edu/torralba/code/spatialenvelope/)_._

We then designed a `FireDetectionNet` — a Convolutional Neural Network for smoke and fire detection. This network was trained on our two datasets. Once our network was trained we evaluated it on our testing set and found that it obtained **92% accuracy.**

However, there are a number of limitations and drawbacks to this approach:

*   **First, we only worked with image data.** Smoke and fire can be better detected with video as fires start off as a smolder, slowly build to a critical point, and then erupt into massive flames.
*   **Secondly, our datasets are small.** Combining the two datasets we only had a total of 4,003 images. Fire and smoke datasets are hard to come by, making it extremely challenging to create high accuracy models.

Building on the previous point, **our datasets are not necessarily representative of the problem.** Many of the example images in our fire/smoke dataset are of professional photos captured by news reports. Fires don’t look like that in the wild.

The point is this:

**Fire and smoke detection _is_ a solvable problem…_but we need better datasets._**

Luckily, [PyImageSearch Gurus](https://pyimagesearch.com/pyimagesearch-gurus/) member [David Bonn](https://www.linkedin.com/in/david-bonn-427449b/) is actively working on this problem and discussing it in the PyImageSearch Gurus Community forums. If you’re interested in learning more about his project, be sure to connect with him.

I hope you enjoyed this tutorial!

**To download the source code to this post (and be notified when future tutorials are published here on PyImageSearch), _just enter your email address in the form below._**

![Image 12](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!