The Ultimate Beginner’s Guide to TensorFlow | by Siddharth Sharma | Towards Data Science

In this tutorial, we will cover TensorFlow in enough depth so that you can train machine learning models from scratch!
TensorFlow is Google’s flagship library for machine learning development. It is used both commercially and by developers for building and deploying machine learning modules. There are two broad versions of TensorFlow — 2.X and 1.X. Both allow for similar functionality, but 2.X uses a cleaner API and has some slight upgrades. It is important to understand that TensorFlow has grown from just a software library to an entire ecosystem for all types of machine learning. APIs like Keras make it extremely simple to train and test deep learning networks in a few lines. (We will cover this towards the end of this tutorial). This tutorial assumes a basic understanding of Python 3 and some degree of familiarity with linear algebra.
Let’s get started with some fundamental background on the mathematics behind TensorFlow. There are three main constructs for TensorFlow operations: vectors, arrays (matrices), tensors. A vector is a mathematical object that has a direction and a magnitude. It is used to find the position of one point in space relative to another point. An array is an arrangement or a series of elements such as symbols, numbers, or expressions. Arrays can be n-dimensional, so a matrix is an array with 2 dimensions. A tensor is an object describing the linear relationship among scalars, vectors, and other tensors. In other words, it is the stacking of multiple arrays to create higher dimensional structures. A practical example of a tensor in action is with an image. When an RGB image is processed, it is a 3-D tensor with layers for each color across the dimensions of height and width. Hopefully, this graphic should clarify the differences:
It would also be useful to understand the Rank of a Matrix. The rank of a Matrix is the number of linearly independent column or row vectors. Before moving on, we will assume that you understand basic linear algebra — transposes, inverses, matrix multiplication, etc. Let’s now define these concepts with some code.
import numpy as np a = np.array([[1,2,3], [4,5,6], np.int16)b = np.mat([1,2,3])
With this code, we use the NumPy library to initialize both a numpy.matrix and a 2D numpy.ndarrray. There are basic differences in these two structures, for example, the fact that numpy.matrix has a proper matrix multiplication interface, but they effectively achieve a very similar purpose. Let’s now discuss how we can implement a tensor. Remember that a tensor is an object that describes the linear relationship between scalars, vectors, and other tensors. A 0th order tensor (or Rank 0) can be represented by a scalar. A 1st order tensor (or Rank 1) can be represented by an array (1-d vector). A 2nd order tensor (or Rank 2) can be represented by a matrix (2-d array). A 3rd order tensor (or Rank 3) is a 3-dimensional array. These examples are implemented in code for reference.
t_0 = np.array(50, dtype=np.int32)t_1 = np.array([b"apple", b"peach", b"grape"])t_2 = np.array([[True, False, False], [False, False, True]])t_3 = np.array([[ [0,0], [0,1], [0,2] ], [ [0,0], [0,1], [0,2] ],[ [0,0], [0,1], [0,2] ]])
Objects in TensorFlow will always have a shape. A shape describes the number of dimensions a tensor can hold, including the length of each dimension. The shape of TensorFlow can be either a list or a tuple. This may not be simple to understand at first, so let’s break it down with an example. What would the shape be for the list [2,3]? This list describes the shape of a 2-D tensor of length 2 in its first dimension and length 3 in its second dimension. In other words, this is a 2x3 matrix! Let’s see some more shapes broken done in code:
s_1_flex = [None]s_2_flex = (None, 3)s_3_flex = [2, None, None]s_any = None
If you are confused about what the “None” keyword refers to, it says that the dimension can be of any size. For example, s_1_flex is a 1-D array with a size of N.
At the center of Tensorflow is the concept of the graph. Before we discuss the graph, let’s talk about eager execution. Eager execution in TensorFlow means that each operation is executed by Python, operation by operation. Eager TensorFlow runs on GPUs and is easy to debug. For some of us, we will be happy to keep our TensorFlow projects in Python and will never leave. However, for other users, eager execution means prevents a “host of accelerations otherwise available” [1]. Luckily, there are ways to both enable and disable eager execution:
import tensorflow as tf# to check for eager executiontf.executing_eagerly()# to enable eager executiontf.compat.v1.enable_eager_execution()# to disable eager executiontf.compat.v1.disable_eager_execution()# After eager execution is enabled, operations are executed as they are defined and Tensor objects hold concrete values, which can be accessed as numpy.ndarray`s through the numpy() method.assert tf.multiply(6, 7).numpy() == 42
If we want to extract our operations so that we can access them outside of Python, we can make them into a graph. As a note, TensorFlow is updated rapidly and functions and definitions may be adjusted, so always check what version of TensorFlow you are using. This can be done in the following line of code:
print(tf.__version__)
For this tutorial, we are using Tensorflow 2.3. Now back to graphs! Graphs are data structures that contain a set of tf.Operation objects. Graphs are useful since they can be saved, run, and restored without the original Python code. A TensorFlow graph is a specific type of directed graph that is used for defining computational structure. In TensorFlow graphs, there are three main objects which can be employed for operations. First is the constant. This creates a node that takes a value and does not change. Here is an example of a constant in action: tf.constant(6,tf.int32,name=’A'). The next graph object in TensorFlow is the variable. Variables are stateful nodes that output their current value; meaning that they can retain their value over multiple executions of a graph. Here is an example of a variable in action: w = tf.Variable(, name=). The final object is the placeholder. Placeholders can feed in future values at runtime in a graph. There are used when a graph depends on external data. Here is one implemented: tf.compat.v1.placeholder(tf.int32, shape=[3], name='B'). Let’s now launch our first graph. This example does not explicitly launch a graph, but it shows how we can run operations within a Session:
# Launch the graph in a session.with tf.compat.v1.Session() as sess:   # input nodes   a = tf.constant(5, name="input_a")   b = tf.constant(3, name="input_b")   # next two nodes   c = tf.math.multiply(a, b, name="input_c")   d = tf.math.add(a, b, name="input_d")   # final node   e = tf.add(c, d, name="input_e")   print(sess.run(e))
To add nodes to our “graph”, we can add, subtract, or multiply existing nodes. In the case above, nodes c, d, and e are all operations of previously initialized nodes. To run this graph, we ran it in what is known as a tf.Session. When not using eager execution, we will need to run our graph in a Session. In this case, we ran it in a Session known as sess. In general, sessions allow us to execute graphs. In TensorFlow 2, sessions are essentially gone. Everything is run in one global runtime. However, for the purposes of this tutorial, we will continue to review Sessions. Here is an example of a concrete graph in action:
graph = tf.Graph()with graph.as_default():  variable = tf.Variable(42, name='foo')  initialize = tf.global_variables_initializer()  assign = variable.assign(13)with tf.Session(graph=graph) as sess:  sess.run(initialize)  sess.run(assign)  print(sess.run(variable))# Output: 13
In this code, we create a graph and initialize a variable named “foo”. We then assign “foo” to a new value. After creating the graph, we run it in a Session. Let’s clarify. Graphs and sessions are not necessary if eager execution is enabled. In other words, there would be no need for sess.run().
Now that we have discussed how graphs and sessions work in practice, you must be wondering — how do we examine them? We can use a tool known as Tensorboard. This is a tool integrated with Tensorflow 2.0 and it can be used to visualize the graph. It also has other tools to understand, debug, and optimize the model. The backend is now tf.keras in TensorFlow 2.0. Let’s now use Tensorboard to visualize a graph. Here is a simple starting example. Note that you can test this code out in Google Colab:
%load_ext tensorboardsummary_writer = tf.summary.create_file_writer('logs/summaries')with summary_writer.as_default():  tf.summary.scalar('scalar', 3.5, step=2)tensorboard --logdir /logs/summaries
This is a simple example where we write a tf scalar to our summary writer. We then run it in logs/summaries (our directory) to access Tensorboard. In most cases, Tensorboard output usually looks something like this:
Let’s now unpack a more sophisticated example of using Tensorboard to visualize and debug graphs:
# Load the TensorBoard notebook extension%load_ext tensorboardimport tensorflow as tfimport datetime# Clear any logs from previous runs!rm -rf ./logs/# initialize your directory logdir = INSERT_DIRECTORYg = tf.compat.v1.Graph()with g.as_default():  step = tf.Variable(0, dtype=tf.int64)  step_update = step.assign_add(1)  writer = tf.summary.create_file_writer(logdir)  with writer.as_default():    tf.summary.scalar("my_metric", 0.5, step=step)all_summary_ops = tf.compat.v1.summary.all_v2_summary_ops()writer_flush = writer.flush()with tf.compat.v1.Session(graph=g) as sess:  sess.run([writer.init(), step.initializer])  for i in range(100):    sess.run(all_summary_ops)    sess.run(step_update)    sess.run(writer_flush)# run in tensorboard%tensorboard --logdir logdir
This example uses the tf.summary.create_file_writer(logdir) to make sure that the output is read in Tensorboard. Now that we have seen Tensorboard in action, we’ve finished the core concepts of TensorFlow. However, TensorFlow is not as useful if we do not apply it for machine learning. We will now walk through Keras, TensorFlow’s simple API for training neural networks.
Simply put, Keras makes it quite simple to train and test your machine learning models. Let’s now demonstrate Keras in action. (All Credit to TensorFlow) We start by importing libraries and checking our version. (This tutorial uses Tf 2.3):
# TensorFlow and tf.kerasimport tensorflow as tffrom tensorflow import keras# Helper librariesimport numpy as npimport matplotlib.pyplot as pltprint(tf.__version__)
Let’s now download a sample dataset. In this case, we will use the Fashion MNIST Dataset. The code below downloads the set directly from keras.datasets:
fashion_mnist = keras.datasets.fashion_mnist(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()
When we load this dataset, we get four NumPy arrays: train_images, train_labels, test_images, test_labels. If you are familiar with machine learning then you would know that we need a certain number of images (data) to train and some samples to test on. Each image in our dataset is a 28x28 NumPy Array and the pixel values range from 0 to 255. The labels are an array of integers, ranging from 0 to 9. These correspond to the following:
Since class names are not included in the set, we can store them in a list:
class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Before training the model, let’s explore the set. In our train_images we have 60,000 images and in our train_labels, we have 60,000 labels. Each label is an integer between 0 and 9, as given by this code:
train_labels# output: array([9, 0, 0, ..., 3, 0, 5], dtype=uint8)
Likewise, there are 10,000 images in the test set. As expected, there are also 10,000 test labels. Each image is represented as 28 x 28 pixels. Let’s now preprocess our data. We can start by inspecting the first image:
plt.figure()plt.imshow(train_images[0])plt.colorbar()plt.grid(False)plt.show()
In this case, the pixel values fall between 0 and 255. Before feeding them into the neural network, we want them to be scaled between 0 and 1. To accomplish this, we can divide by 255 for both the training and testing set:
train_images = train_images / 255.0test_images = test_images / 255.0
Let’s verify if our data is now in the correct format. We can display the first 25 images from the training set and display the class name below each image:
plt.figure(figsize=(10,10))for i in range(25):    plt.subplot(5,5,i+1)    plt.xticks([])    plt.yticks([])    plt.grid(False)    plt.imshow(train_images[i], cmap=plt.cm.binary)    plt.xlabel(class_names[train_labels[i]])plt.show()
Hooray, it worked! We are now ready to build our deep learning model. Since this is an image classification problem, we will need to use an artificial neural network. We will first need to configure the layers of our model, then compile the model. First, let’s configure our layers. Layers in a deep neural network extract representations from the data fed into them (). To set up our network, we will chain together a few layers. For this example, we will use the Sequential model:
model = keras.Sequential([    keras.layers.Flatten(input_shape=(28, 28)),    keras.layers.Dense(128, activation='relu'),    keras.layers.Dense(10)])
The first layer in this network, tf.keras.layers.Flatten, transforms the format of images from 28 x 28 to a 1-D array of 784 pixels. Essentially, we are lining the pixels up. There is no learning in this layer, only reformatting. The next two layers in our network are tf.keras.layers.Dense layers. These are fully connected neural layers with 128 neurons. They return a score for whether the image belongs to one of the ten classes. Before we train our model, we need to add in the last step. While we compile the network we just built, we need to make sure that the performance is high! In other words, we want to minimize loss/training error. This code accomplishes the task:
model.compile(optimizer='adam',           loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True,              metrics=['accuracy'])
To do this, we employ a loss function to measure how accurate our model is during training. The goal will be to minimize this function. Our model is also compiled with an optimizer. This gives us a way to update our model based on loss. Finally, we compile with metrics. This is a way to monitor accuracy during the training and testing steps.
We can now train our model:
model.fit(train_images, train_labels, epochs=10)
Here, we feed in our training images and trining labels to the model as arrays. During training, we expect the model to associate images and labels. We can then ask our model to make predictions on the test set. Once we fit our model using the code above, you can expect an output like the following:
In this step, we see that our configured deep learning model trains for 10 epochs (10 training runs). In each epoch, it displays both the loss and accuracy. Since the model becomes better at recognizing different labels over time, the loss decreases and the performance increases across each epoch. At the final epoch (#10), our model reaches an accuracy of roughly 0.91 (91%). We can now take the final step to compare how the model performs on the test dataset:
test_loss, test_acc = model.evaluate(test_images,  test_labels, verbose=2)print('\nTest accuracy:', test_acc)
If we run this code, we find that our test accuracy was 0.884. You may be surprised that our training accuracy was higher than the test accuracy. The reason for this is simple: overfitting. Overfitting occurs when a machine learning model performs worse on unseen (testing) data due to highly specific training data. In other words, our model had memorized the noise and details of the training set to the point that it negatively impacts performance on new/testing data.
Through this tutorial, you were able to learn and implement basic TensorFlow structures. We covered different data types, variables, placeholders, graphs, sessions, differences between 1.X and 2.X, eager execution, tensorboard, Keras, overfitting, etc! If you want to learn more and do more interesting projects/tutorials, be sure to check out tensorflow.org/tutorials. If you enjoyed reading, do be sure to give a clap :)
References
https://github.com/tensorflow/docshttps://www.tensorflow.org/tutorials/keras/classification (Keras Section is directly adapted from this tutorial)
https://github.com/tensorflow/docs
https://www.tensorflow.org/tutorials/keras/classification (Keras Section is directly adapted from this tutorial)