Input Pipeline for Images using Keras and TensorFlow | by Renu Khandelwal | Towards Data Science

Second, in the series to learn how to create an input pipeline to load and create image train and test dataset from custom data using Kera Preprocessing, Tensorflow, and tf.data.
The dataset used here is Intel Image Classification from Kaggle, and all the code in the article works in Tensorflow 2.0.
Intel Image classification dataset is split into Train, Test, and Val. We will only use the training dataset to learn how to load the dataset using different libraries.
The ImageDataGenerator class generates batches of tensors for the image data and scale down the pixel values from 0 to 255 to a value between 0 and 1.
We can apply different real-time data augmentation to the train image dataset like rotating the image, zooming the image, horizontal flip or vertical flip, etc. Data augmentation is not applied to the validation or test dataset.
import pandas as pdimport numpy as npimport osfrom tensorflow.keras.preprocessing.image import ImageDataGenerator, img_to_array, load_img, array_to_imgimport tensorflow as tfIMG_WIDTH=200IMG_HEIGHT=200batch_size=4train_dir = r'\CV\Intel_Images\seg_train\seg_train'test_dir  = r'\CV\Intel_Images\seg_pred\seg_pred'val_dir   = r'\CV\Intel_Images\seg_test\seg_test'image_gen_train = ImageDataGenerator(rescale=1./255,                                      zoom_range=0.2,                                      rotation_range=65,                                     shear_range=0.09,                                     horizontal_flip=True,                                     vertical_flip=True)image_gen_val = ImageDataGenerator(rescale=1./255)
Once the instance of ImageDatagenerator is created, use the flow_from_directory() to read the image files from the directory.
flow_from_directory() expects the image data in a specific structure as shown below where each class has a folder, and images for that class are contained within the class folder.
train_data_gen = image_gen_train.flow_from_directory(batch_size=batch_size,directory=train_dir,shuffle=True,                                                     target_size=(IMG_HEIGHT, IMG_WIDTH,3),                                                     class_mode='sparse')val_data_gen = image_gen_val.flow_from_directory(batch_size=batch_size,directory=val_dir,                                                 target_size=(IMG_HEIGHT, IMG_WIDTH,3),                                                 class_mode='sparse')
class_indices.keys() will extract the class names from the dataset in a dictionary
train_data_gen.class_indices.keys()
Creating a simple Deep Learning model, compiling it, and training the model using the dataset generated using Keras preprocessing
model=tf.keras.Sequential(        [            tf.keras.layers.InputLayer(input_shape=(200, 200, 3)),            tf.keras.layers.Conv2D(filters=32, kernel_size=3, strides=(2, 2), activation='relu'),            tf.keras.layers.Conv2D(filters=64, kernel_size=3, strides=(2, 2), activation='relu'),            tf.keras.layers.Flatten(),            tf.keras.layers.Dense(6)         ])#Compile the modelmodel.compile(optimizer='rmsprop', loss='sparse_categorical_crossentropy',metrics=['accuracy'])#Fitting the modelhistory = model.fit(train_data_gen,steps_per_epoch=len(train_data_gen)//batch_size, validation_data=val_data_gen, epochs=2)
Input pipeline using Tensorflow will create tensors as an input to the model.
Open the image file using tensorflow.io.read_file()Decode the format of the file. Here we have a JPEG file, so we use decode_jpeg() with three color channels.Resize the image to match the input size for the Input layer of the Deep Learning model.Convert the image to float datatype using TensorFlow and then normalize the values between 0 and 1 from 0 to 255.tf.stack() creates a stacked tensor as rows of data
Open the image file using tensorflow.io.read_file()
Decode the format of the file. Here we have a JPEG file, so we use decode_jpeg() with three color channels.
Resize the image to match the input size for the Input layer of the Deep Learning model.
Convert the image to float datatype using TensorFlow and then normalize the values between 0 and 1 from 0 to 255.
tf.stack() creates a stacked tensor as rows of data
def create_dataset_tf(img_folder):    class_name=[]    tf_img_data_array=[]           for dir1 in os.listdir(img_folder):        for file in os.listdir(os.path.join(img_folder, dir1)):            image= os.path.join(img_folder,dir1, file)            image = tf.io.read_file(image)            image = tf.io.decode_jpeg(image, channels=3)            image = tf.image.resize(image, (200,200))            image = tf.cast(image / 255., tf.float32)            tf_img_data_array.append(image)            class_name.append(dir1)                       return tf.stack(tf_img_data_array, axis=0),class_nameimg_folder=r'CV\Intel_Images\seg_train\seg_train'tf_img_data, class_name=create_dataset_tf(img_folder)
Converting text labels to numeric codes
Create a dictionary for all unique values for the classes
target_dict={k: v for v, k in enumerate(np.unique(class_name))}target_dict
Convert the class_names to their respective numeric value based on the dictionary
target_val=  [target_dict[class_name[i]] for i in range(len(class_name))]
Creating a simple Deep Learning model, compiling it, and training the model.
It is the same model that we created earlier when suing Keras.preprocessing(). The only change here is the input image data and class names, which are a list of Tensors values to fit the model.
model=tf.keras.Sequential(        [            tf.keras.layers.InputLayer(input_shape=(200, 200, 3)),            tf.keras.layers.Conv2D(filters=32, kernel_size=3, strides=(2, 2), activation='relu'),            tf.keras.layers.Conv2D(filters=64, kernel_size=3, strides=(2, 2), activation='relu'),            tf.keras.layers.Flatten(),            tf.keras.layers.Dense(6)         ])#Compile the modelmodel.compile(optimizer='rmsprop', loss='sparse_categorical_crossentropy',metrics=['accuracy'])#Fitting the modelhistory = model.fit(x=tf_img_data, y=tf.cast(list(map(int,target_val)),tf.int32), epochs=2)
tf.data API allows us to build complex input pipelines while reading large amounts of data available in different formats and then apply transformations.
In tf.data for image processing pipeline, an element is a single training example, having a pair of tensors to represent the image and its corresponding label.
Importing libraries and setting basic parameters
import numpy as npimport osimport tensorflow as tfdata_dir=r'\CV\Intel_Images\seg_train\seg_train'batch_size = 32img_height = 200img_width = 200
tf.data.Dataset.list_files() creates a dataset from a directory list of files using a matching pattern.
list_ds = tf.data.Dataset.list_files(str(data_dir + '\\*\\*'), shuffle=False)# get the count of image files in the train directoryimage_count=0for dir1 in os.listdir(data_dir):    for files in os.listdir(os.path.join(data_dir, dir1)):        image_count+=1list_ds = list_ds.shuffle(image_count, reshuffle_each_iteration=False)
Creating class labels from the directory name
class_names = np.array(sorted([dir1 for dir1 in os.listdir(data_dir)]))
Splitting the dataset into train, and Val.
The validation dataset is 20% of the total dataset, and train dataset is 80% of the entire dataset.
val_size = int(image_count * 0.2)train_ds = list_ds.skip(val_size)val_ds = list_ds.take(val_size)
Creating input pipeline components for a single training/validation example representing a pair of tensors to represent the image and its corresponding label.
#To process the labeldef get_label(file_path):  # convert the path to a list of path components separated by sep  parts = tf.strings.split(file_path, os.path.sep)    # The second to last is the class-directory  one_hot = parts[-2] == class_names# Integer encode the label  return tf.argmax(tf.cast(one_hot, tf.int32))# To process the imagedef decode_img(img):  # convert the compressed string to a 3D uint8 tensor  img = tf.image.decode_jpeg(img, channels=3)    # resize the image to the desired size  return tf.image.resize(img, [img_height, img_width])# To create the single training of validation example with image and its corresponding labeldef process_path(file_path):  label = get_label(file_path)# load the raw data from the file as a string  img = tf.io.read_file(file_path)  img = decode_img(img)  return img, label
Set the AUTOTUNE; this will help to delegate the decision on the level of parallelism to use to the tf.data at runtime to optimize the CPU/GPU utilization.
AUTOTUNE = tf.data.experimental.AUTOTUNE# Set `num_parallel_calls` so multiple images are loaded/processed in parallel.train_ds = train_ds.map(process_path, num_parallel_calls=AUTOTUNE)val_ds = val_ds.map(process_path, num_parallel_calls=AUTOTUNE)
Configure data source for Performance
To configure the data source for performance, use prefetching.
Prefetching in tf.data allows the preprocessing of the data and model execution of a training step to overlap.
While the model is executing a training step 100, the input pipeline is reading the data for step 101.
def configure_for_performance(ds):  ds = ds.cache()  ds = ds.shuffle(buffer_size=1000)  ds = ds.batch(batch_size)  ds = ds.prefetch(buffer_size=AUTOTUNE)  return dstrain_ds = configure_for_performance(train_ds)val_ds = configure_for_performance(val_ds)
Creating a simple Deep Learning model
The input to the model is tf.data.Dataset
model=tf.keras.Sequential(        [            tf.keras.layers.InputLayer(input_shape=(200, 200, 3)),            tf.keras.layers.Conv2D(filters=32, kernel_size=3, strides=(2, 2), activation='relu'),            tf.keras.layers.Conv2D(filters=64, kernel_size=3, strides=(2, 2), activation='relu'),            tf.keras.layers.Flatten(),            tf.keras.layers.Dense(6)         ])#Compile the modelmodel.compile(optimizer='rmsprop', loss='sparse_categorical_crossentropy',metrics=['accuracy'])#Fitting the modelhistory = model.fit(train_ds,validation_data=val_ds,epochs=3)
The best training time I got was using tf.data as there is prefetching, which is using AUTOTUNE to delegate the decision on the level of parallelism.