General Purpose Tensorflow 2.x Script to train any CSV file
By: Zahash Z
Tags: Python, TensorFlow, Machine Learning, Data Science, Artificial Intelligence

With that out of the way, let's get started.

First, you just need to specify a few parameters.

DATASET_PATH — where your dataset is located in the file system

LABEL_NAME — target label column name

TASK — “r” for regression and “c” for classification

DUMMY_BATCH_SIZE — don’t worry about this. Just set it to 5

BATCH_SIZE

EPOCHS

TRAIN_FRAC — the fraction of data to be used for training. float between 0 and 1.

CHECKPOINT_DIR — folder to store checkpointed models

Imports

from collections import defaultdict

import os

import numpy as np

import matplotlib.pyplot as plt

import tensorflow as tf

import kerastuner as kt

import IPython

during the whole process, we will be using tf.data to handle the dataset. By using tf.data you can load massive datasets one chunk at a time. It also has a lot of useful functionality like caching and prefetch the next chunk of data while the model is training on the current chunk.

The below function will return a fresh dataset object that you can iterate over one chunk(batch size) at a time.

def get_dataset(batch_size = 5):

return tf.data.experimental.make_csv_dataset(DATASET_PATH, batch_size = batch_size, label_name = LABEL_NAME, num_epochs = 1)

If the task is regression, the number of output nodes is one

if TASK == "r":

OUTPUT_NODES = 1

If it is a classification task then we need to find the number of labels(classes) to determine the number of output layers by iterating over the whole dataset.

elif TASK == "c":

unique_labels = set() for _ , label in get_dataset(batch_size=BATCH_SIZE):

for ele in label.numpy():

unique_labels.add(ele) num_labels = len(unique_labels) if num_labels <= 2:

OUTPUT_NODES = 1

else:

OUTPUT_NODES = num_labels

As I mentioned before, this script will do all the preprocessing on its own without the need to have external functions because all the preprocessing logic is embedded as one of the layers in the model. This enables you to just give the raw data to the deployed model.

The first step in achieving that is to define what is known as model inputs.

It's simply a dictionary where the keys are the column names and values are tf.keras.Input objects

model_inputs = {} for batch, _ in get_dataset(batch_size=DUMMY_BATCH_SIZE).take(1):

for col_name, col_values in batch.items():

model_inputs[col_name] = tf.keras.Input(shape=(1,), name=col_name, dtype=col_values.dtype)

Sample Output:

{ "clouds_all": <tf.Tensor "clouds_all:0" shape=(None, 1) dtype=int32>, "holiday": <tf.Tensor "holiday:0" shape=(None, 1) dtype=string>, "rain_1h": <tf.Tensor "rain_1h:0" shape=(None, 1) dtype=float32>, "snow_1h": <tf.Tensor "snow_1h:0" shape=(None, 1) dtype=float32>, "temp": <tf.Tensor "temp:0" shape=(None, 1) dtype=float32>, "weather_description": <tf.Tensor "weather_description:0" shape=(None, 1) dtype=string>, "weather_main": <tf.Tensor "weather_main:0" shape=(None, 1) dtype=string> }

One feature that tf.data has is that it can automatically detect the data type of each column. Let's use that to our advantage and split the model_input dictionary into multiple dictionaries according to the data type

integer_inputs = {}

float_inputs = {}

string_inputs = {} for col_name, col_input in model_inputs.items():

if col_input.dtype == tf.int32:

integer_inputs[col_name] = col_input elif col_input.dtype == tf.float32:

float_inputs[col_name] = col_input elif col_input.dtype == tf.string:

string_inputs[col_name] = col_input

Sample integer_inputs:

{'clouds_all': <tf.Tensor 'clouds_all:0' shape=(None, 1) dtype=int32>}

Sample float_inputs:

{

'rain_1h': <tf.Tensor 'rain_1h:0' shape=(None, 1) dtype=float32>, 'snow_1h': <tf.Tensor 'snow_1h:0' shape=(None, 1) dtype=float32>, 'temp': <tf.Tensor 'temp:0' shape=(None, 1) dtype=float32>

}

Sample string_inputs:

{ 'holiday': <tf.Tensor 'holiday:0' shape=(None, 1) dtype=string>, 'weather_description': <tf.Tensor 'weather_description:0' shape=(None, 1) dtype=string>, 'weather_main': <tf.Tensor 'weather_main:0' shape=(None, 1) dtype=string> }

Now, for the preprocessing, the general flow for the integer and float columns is to first concatenate the layers by passing them through tf.layers.Concatenate and then normalize them through a normalization layer. But to calculate the mean and std for normalization, we have to iterate through the whole dataset once.

when it comes to preprocessing the string columns, we first get the vocabulary(list of unique words) for each column. Then we pass the input through a string lookup layer tf.keras.layers.experimental.preprocessing.StringLookup and then through a one-hot encoding layer tf.keras.layers.experimental.preprocessing.CategoryEncoding. You can also perform some simple operations like converting everything to lowercase and eliminating leading and trailing white spaces.

StringLookup layer maps strings from a vocabulary to integer indices. CategoryEncoding then takes those integer indices and creates a one-hot vector.

Pass the inputs through their corresponding functions to get preprocessed layers

integer_layer = numerical_input_processor(integer_inputs)

float_layer = numerical_input_processor(float_inputs)

string_layer = string_input_processor(string_inputs)

Add all the inputs to a list

preprocessed_inputs = [] if integer_layer is not None:

preprocessed_inputs.append(integer_layer) if float_layer is not None:

preprocessed_inputs.append(float_layer) if string_layer is not None:

preprocessed_inputs.append(string_layer)

preprocessed_inputs might look something like this

[<tf.Tensor 'normalization/truediv:0' shape=(None, 1) dtype=float32>, <tf.Tensor 'normalization_1/truediv:0' shape=(None, 3) dtype=float32>, <tf.Tensor 'concatenate_1/concat:0' shape=(None, 66) dtype=float32>]

Concatenate all the inputs

if len(preprocessed_inputs) > 1:

preprocessed_inputs_cat = tf.keras.layers.Concatenate()(preprocessed_inputs) else:

preprocessed_inputs_cat = preprocessed_inputs

Finally, create a tf.keras.Model that takes the model_inputs and returns the preprocessed outputs

preprocessing_head = tf.keras.Model(model_inputs, preprocessed_inputs_cat)

You can also plot the preprocessing model to see what the individual layers look like. Increase the value of dpi if you want more the plot to have a higher clarity.

tf.keras.utils.plot_model(model = preprocessing_head, rankdir="LR", dpi=72, show_shapes=True, expand_nested=True, to_file="preprocessing_head.png")

You might get an image something like this