Demystifying Tensorflow Estimator
By: Junaid Khattak
Tags: Machine Learning, Tensorflow2, Tensorboard, Tutorial, TensorFlow

Tensorflow is one of the most popular machine learning library out there. It is one of the few framework that lets you work at the lowest and highest abstraction of machine learning. You can do useful machine learning using high level APIs that handle everything or you might build a custom model from ground up controlling everything from model architecture to training loops

You can find and run complete working example of the code snippets here

What is Tensorflow Estimator??

Similar to keras’s Model API, Tensorflow Estimator is a high level API that encapsulates training, evaluation, prediction and serving. In other words you don’t have to write training, evaluation or prediction loops, esitmator does it all for you. It also gives you a gives you a reusable model which you can save to or load from anywhere.

Why use Estimator API?

Estimator is an easy to use API that handles lots of complexity for your but still provide enough customization opportunities. You can design your model once and it will run seamlessly on your local machine with or without GPUs as well as on a multi-server distributed environment with GPUs and TPUs !!!

It safely runs a distributed training loop that takes care of loading data, checkpoints and write summaries for tensor board

How does it work?

Estimator API allows you to use pre-made as well as custom estimators. I will go through working with pre-made estimators here and leave custom estimators for another time

You can make any pre-made estimator work by going through following steps

a: Dataset input functions

These functions take in dataset and returns feature dictionaries and labels. Feature dictionary is a mapping of feature name to feature value in a dataset

def make_input_fn(df, epochs=500, shuffle=True, batch_size=32):

df = df.copy()

labels = df.pop('AveragePrice') # extract label from dataframe

def input_function():

# create dataset from inmemory pandas dataframe

dataset = tf.data.Dataset.from_tensor_slices((dict(df), labels))

if shuffle:

dataset = dataset.shuffle(buffer_size=len(df))



dataset = dataset.batch(batch_size).repeat(epochs)

return dataset

return input_function



train_input_fn = make_input_fn(train_df)

eval_input_fn = make_input_fn(eval_df, epochs=1, shuffle=False)

In above example make_input_fn is input function creator as it returns input_function which is the actual dataset input function

b: Define Feature Columns

In second step we create a list of all feature columns that we wish to train our model on. It defines all feature as tf.feature_column which require feature name and optionally datatype and a preprocessing function

DENSE_COLUMNS = ['Total_Bags','Small_Bags','Large_Bags','XLarge_Bags']



SPARSE_COLUMNS = ['type', 'year', 'region']



feature_columns = []

for feature in DENSE_COLUMNS:

feature_columns.append(tf.feature_column.numeric_column(feature))

for feature in SPARSE_COLUMNS:

vocab = data[feature].unique()

categorical_feature = tf.feature_column.categorical_column_with_vocabulary_list(feature, vocab)

feature_columns.append(tf.feature_column.indicator_column(categorical_feature))

In above example i have a dataset that has some dense(continuous) and some sparse(categorical) features. For each feature type, tf.feature is created and added into feature_columns list. Our model will then use this list to process actual data from dataset

c: Instantiate Estimator & Train

There are many pre-made estimators available. You can go through complete list here. You just have pass feature columns to it and you are good to go. If you need to perform parameter tuning you can pass those arguments as well. To start training, just call train method on your estimator and pass your dataset input function as argument

linear_regressor = tf.estimator.LinearRegressor(

feature_columns=feature_columns,

model_dir=output_dir

)

linear_regressor.train(train_input_fn)

That’s all!!!

Your model will start training and keep saving model checkpoints and summaries in output_dir. You can also point tensor board to this directory to monitor training

Tensorflow comes packaged with tensorboard which is an awesome tool for visualizing different model metrics. You can checkout a very helpful intro to tensorboard at neptune.ai