Mastering TensorFlow “Variables” in 5 Easy Steps | by Orhan G. Yalçın | Medium | Towards Data Science

WARNING: Do not confuse this article with “Mastering TensorFlow Tensors in 5 Easy Steps”!
If you are reading this article, I am sure that we share similar interests and are/will be in similar industries. So let’s connect via Linkedin! Please do not hesitate to send a contact request! Orhan G. Yalçın — Linkedin
In this tutorial, we will focus on TensorFlow Variables. After the tutorial, you will be able to create, update, and manage TensorFlow Variables effectively. As usual, our tutorial will deliver code examples with detailed explanations as well as conceptual explanations. We will master TensorFlow Variables in 5 easy steps:
Step 1: Definition of Variables →A Brief Introduction, Comparison with Tensors
Step 2: Creation of Variables → Instantiating tf.Variable Objects
Step 3: Qualifications of Variables → Characteristics and Features
Step 4: Operations with Variables → Basic Tensor Operations, Indexing, Shape Manipulation, and Broadcasting
Step 5: Hardware Selection for Variables → GPUs, CPUs, TPUs
Fasten your belts, and let’s start!
In this step, we will briefly cover what Variables are and understand the difference between plain Tensor objects and Variable objects.
A TensorFlow Variable is the preferred object type representing a shared and persistent state that you can manipulate with any operation, including TensorFlow models. Manipulation refers to any value or parameter update. This characteristic is the most distinguishing feature of Variables compared to tf.Tensor objects. TensorFlow Variables are recorded as tf.Variable objects. Let’s make a brief comparison between tf.Tensor and tf.Variable objects to understand their similarities and differences.
So, the most important difference between Variables and Tensors is mutability. The values in a Variable object can be updated (e.g., with the assign() function) as opposed to Tensors.
“The values of tensor objects cannot be updated, and you can only create a new Tensor object with the new values.”
Variable objects are mainly used to store model parameters, and since these values are constantly updated during training, using Variables, instead of Tensors, is a necessity rather than a choice.
The shape of a Variable object can be updated with the reshape() instance function just like the shape of a Tensor object. Since Variable objects are built on top of Tensor objects, they have common attributes such as .shape and .dtype. But, Variables also have unique attributes such as .trainable,.device, and .name attributes that the Tensors do not have.
Let’s see how we can create tf.Variable objects!
We can instantiate (i.e., create) tf.Variableobjects with the tf.Variable() function. The tf.Variable() function accepts different data types as parameter such as integers, floats, strings, lists, and tf.Constant objects.
Before showing different Variable object examples with these different data types, I want you to start a new Google Colab notebook and import TensorFlow library with the following code:
Now, we can start creating tf.Variable objects.
1 — We can pass a tf.constant() object as the initial_value:
2 — We can pass a single integer as the initial_value:
3 — We can pass a list of integers or floats as the initial_value:
4— We can pass a single string as the initial_value:
5— We can pass a list of strings as the initial_value:
As you can see, there are several data types that th etf.Variable() function accepts as the initial_value argument. Now let’s take a look at the characteristics and features of variables.
Every Variable must have some properties such as value, name, uniform data type, shape, rank, size, and more. In this section, we will see what these properties are and how we can view these properties in a Colab notebook.
Every Variable must specify an initial_value. Otherwise, TensorFlow raises an error and says that Value Error: initial_value must be specified. Therefore, make sure that you pass on an initial_valueargument when creating Variable objects. To be able to view a Variable’s values, we can use the .value() function as well as the .numpy() function. See the example below:
Output:The values stored in the variables:tf.Tensor( [[1. 2.]              [1. 2.]], shape=(2, 2), dtype=float32)The values stored in the variables:[[1. 2.][1. 2.]]
Name is a Variable attribute which helps developers to track the updates on a particular variable. You can pass a name argument while creating the Variable object. If you don’t specify a name, TensorFlow assigns a default name, as shown below:
Output:The name of the variable:  Variable:0
Each Variable must have a uniform data type that it stores. Since there is a single type of data stored for every Variable, you can also view this type with the .dtype attribute. See the example below:
Output:The selected datatype for the variable:  <dtype: 'float32'>
The shape property shows the size of each dimension in the form of a list. We can view the shape of the Variable object with the .shape attribute. Then, we can view the number of dimensions that a Variable object has with the tf.size() function. Finally, Size corresponds to the total number of elements a Variable has. We need to use the tf.size() function to count the number of elements in a Variable. See the code below for all three properties:
Output:The shape of the variable:  (2, 2)The number of dimensions in the variable: 2The number of dimensions in the variable: 4
There are several basic operations you can easily conduct with math operators and TensorFlow functions. On top of what we covered in Part 2 of this tutorial series, you may also use the following math operators for Variable operations.
Addition and Subtraction: We can conduct addition and subtraction with + and — signs.
Addition by 2:tf.Tensor( [[3. 4.]  [3. 4.]], shape=(2, 2), dtype=float32)Substraction by 2:tf.Tensor( [[-1.  0.]  [-1.  0.]], shape=(2, 2), dtype=float32)
Multiplication and Division: We can conduct multiplication and division with * and / signs.
Multiplication by 2:tf.Tensor( [[2. 4.]  [2. 4.]], shape=(2, 2), dtype=float32)Division by 2:tf.Tensor( [[0.5 1. ]  [0.5 1. ]], shape=(2, 2), dtype=float32)
Matmul and Modulo Operations: Finally, you can also do matmul and modulo operations with @ and % signs:
Matmul operation with itself:tf.Tensor( [[3. 6.]  [3. 6.]], shape=(2, 2), dtype=float32)Modulo operation by 2:tf.Tensor( [[1. 0.]  [1. 0.]], shape=(2, 2), dtype=float32)
These are elementary examples, but they can be extended into complex calculations, which creates the algorithms that we use for deep learning applications.
Note: These operators also work on regular Tensor objects.
With the tf.assign() function, you may assign new values to a Variable object without creating a new object. Being able to assign new values is one of the advantages of Variables, where value reassignment is required. Here is an example of reassignment of values:
Output:...array([[  2., 100.],          [  1.,  10.]],...
Just as in Tensors, you may easily access particular elements using index values, as shown below:
Output:The 1st element of the first level is: [1. 2.]The 2nd element of the first level is: [1. 2.]The 1st element of the second level is: 1.0The 3rd element of the second level is: 2.0
Just as with Tensor objects, when we try to do combined operations using multiple Variable objects, the smaller Variables can stretch out automatically to fit larger Variables, just as NumPy arrays can. For example, when you attempt to multiply a scalar Variable with a 2-dimensional Variable, the scalar is stretched to multiply every 2-dimensional Variable element. See the example below:
tf.Tensor([[ 5 10]           [15 20]], shape=(2, 2), dtype=int32)
Just as in Tensor objects, you can reshape Variable objects as well. For the reshape operation, we can use the tf.reshape() function. Let's use the tf.reshape() function in code:
tf.Tensor( [[1.]            [2.]            [1.]            [2.]], shape=(4, 1), dtype=float32)
As you will see in the upcoming Parts, we will accelerate our model training with GPUs and TPUs. To be able to see what type of device (i.e., processor) our variable is processed with, we can use .device attribute:
The device which process the variable:   /job:localhost/replica:0/task:0/device:GPU:0
We can also set which device should process a particular calculation with the tf.device() function by passing the device name as an argument. See the example below:
Output:The device which processes the variable a: /job:localhost/replica:0/task:0/device:CPU:0The device which processes the variable b: /job:localhost/replica:0/task:0/device:CPU:0The device which processes the calculation: /job:localhost/replica:0/task:0/device:GPU:0
Even though you will not have to set this manually while training a model, there might be circumstances where you have to choose a device for a particular calculation or data processing work. So, beware of this option.
We have successfully covered the basics of TensorFlow’s Variable objects.
Give yourself a pat on the back!
This should give you a lot of confidence since you are now much more informed about the main mutable Variable object type used for all kinds of operations in TensorFlow.
If this is your first post, consider starting from Part 1 of this tutorial series:
towardsdatascience.com
or check out Part 2:
towardsdatascience.com
If you would like to have access to full code on Google Colab and the rest of my latest content, consider subscribing to the mailing list:
Subscribe Now
Finally, if you are interested in more advanced applied deep learning tutorials, check out some of my other articles: