Understanding a TensorFlow program in simple steps. | by Nidhin Mahesh | Towards Data Science

TensorFlow is a library which can be applied to all the machine learning algorithms especially deep learning with neural network. Machine learning is simply those programs which are written to deal with large data-sets to find patterns in them and extract information. It actually learn from data to make accurate or somewhere near predictions and correct itself for the better. Just like we wanted someone to do. Image recognition, and its enhancement or even identifying it is some applications of ML.
I felt happy to see welcoming changes since I wrote my first story on TF. This story focuses on some fundamentals to understand a TensorFlow program. I would suggest to try the code in your machine and see how it works.
for the ability to run the same model on different problem set we need placeholders and feed dictionaries. As our TensorFlow program become more complex, our visualization needs to keep up. Regression tries to model relationship where there is cause and effect. Cause is an independent variable which occurs and Effect depends on cause. Linear regression is a straight line regression which is caused when modelling the regression X causes Y. Cause is also called explanatory variable
Eg: hypothesis of wealth increases life expectancy.
For equation Y=A+BX. to find the best fit, ML algorithm will give initial values to A and B. It will run the regression and find the errors with those values of A and B. These are then fed back to the input to get new values of A and B. Linear regression involves finding best fit line
best fit line is found by minimizing the least square error. best fit line is the one where the sum of the square of the lengths of the line joining from bottom to the datapoint in graph is minimum.
Placeholders in TensorFlow are similar to variables and you can declare it using tf.placeholder. You dont have to provide an initial value and you can specify it at runtime with feed_dict argument inside Session.run , whereas in tf.Variable you can to provide initial value when you declare it.
sample program using placeholder
import tensorflow as tf#setup placeholder using tf.placeholderx = tf.placeholder(tf.int32, shape=[3],name='x')'''it is of type integer and it has shape 3 meaning it is a 1D vector with 3 elements in itwe name it x. just create another placeholder y with same dimension. we treat the placeholders like we treate constants. '''y = tf.placeholder(tf.int32, shape=[3],name='y')sum_x = tf.reduce_sum(x,name="sum_x")prod_y = tf.reduce_prod(y,name="prod_y")'''we dont know what values x and y holds till we run the graph'''final_div = tf.div(sum_x,prod_y, nwe give fetches and feed_dict pass into every session.run commandame="final_div")final_mean = tf.reduce_mean([sum_x, prod_y], name="final_mean")sess = tf.Session()print ("sum(x): ", sess.run(sum_x, feed_dict={x: [100,200,300]}))print ("prod(y): ", sess.run(prod_y, feed_dict={y: [1,2,3]}))writer = tf.summary.FileWriter('./tensorflow_example',sess.graph)writer.close()sess.close()
we give fetches and feed_dict pass into every session.run command. fetches parameter indicate what it is we want to compute and the feed dictionary specifies the placeholder values for that computation
import tensorflow as tfW = tf.constant([10,100], name='const_W')#these placeholders can hold tensors of any shape#we will feed these placeholders laterx = tf.placeholder(tf.int32, name='x')b = tf.placeholder(tf.int32,name='b')#tf.multiply is simple multiplication and not matrixWx = tf.multiply(W,x, name="Wx")y = tf.add(Wx,b,name='y')with tf.Session() as sess: '''all the code which require a session is writer here here Wx is the fetches parameter. fetches refers to the node of the graph we want to compute feed_dict is used to pass the values for the placeholders ''' print( "Intermediate result Wx: ", sess.run(Wx, feed_dict={x: [3,33]})) print( "Final results y: ",sess.run(y, feed_dict={x:[5,50],b:[7,9]}))writer = tf.summary.FileWriter('./fetchesAndFeed',sess.graph)writer.close()
Variables are constructs which allow you to change the value stored there. Supervised learning algorithms perform multiple iterations before they arrive at final conclusion use variable to store the values which change as the model converges. Our aim is to minimize the error between the regression line and the points in the data set. So we tweak the regression line with each iteration to get new values. To get the best fit line of the equation y=A+Bx, we constantly tweak the value of A and B.
Variables are mutable tensor values that persist across multiple calls to sesssion.run(). I explain this with the same demo code as above.
import tensorflow as tfW = tf.Variable([2.5,4.0],tf.float32, name='var_W')#here W is a Variablex = tf.placeholder(tf.float32, name='x')b = tf.Variable([5.0,10.0],tf.float32, name='var_b')#b is also a variable with initial value 5 and 10y = W * x + b#initialize all variables definedinit = tf.global_variables_initializer()#global_variable_initializer() will declare all the variable we have initilized# use with statement to instantiate and assign a sessionwith tf.Session() as sess: sess.run(init) #this computation is required to initialize the variable print("Final result: Wx + b = ", sess.run(y,feed_dict={x:[10,100]}))# changing values number = tf.Variable(2)multiplier = tf.Variable(1)init = tf.global_variables_initializer()result = number.assign(tf.multiply(number,multiplier))with tf.Session() as sess: sess.run(init)for i in range(10):  print("Result number * multiplier = ",sess.run(result))  print("Increment multiplier, new value = ",sess.run(multiplier.assign_add(1)))
We can explicitly create as many graphs inside a TensorFlow program. Any TensorFlow program have a default graph which contains all the placeholders and variables you have instantiated. But we can logically segment the graph by instantiating a graph explicitly using tf.graph() . Below program may answer some of your doubts.
import tensorflow as tfg1 = tf.Graph()'''set g1 as default to add tensors to this graph using default methord'''with g1.as_default(): with tf.Session() as sess:  A = tf.constant([5,7],tf.int32, name='A')  x = tf.placeholder(tf.int32, name='x')  b = tf.constant([3,4],tf.int32, name='b')y = A * x + bprint( sess.run(y, feed_dict={x: [10,100]}))  '''to ensure all the tensors and computations are within the graph g1, we use assert'''  assert y.graph is g1g2 = tf.Graph()with g2.as_default(): with tf.Session() as sess:  A = tf.constant([5,7],tf.int32, name='A')  x = tf.placeholder(tf.int32, name='x')  y = tf.pow(A,x,name='y')  print( sess.run(y, feed_dict={x: [3,5]}))  assert y.graph is g2'''same way you can access defaut graph '''default_graph = tf.get_default_graph()with tf.Session() as sess: A = tf.constant([5,7],tf.int32, name='A') x = tf.placeholder(tf.int32, name='x') y = A + x print(sess.run(y, feed_dict={x: [3,5]}))assert y.graph is default_graph
TensorBoard may be most extremely useful debugging tool, but as your graph explodes in size, you need some ways to get the details in a bigger picture. Now run the below program using TensorFlow and view its graph in TensorBoard
import tensorflow as tfA = tf.constant([4], tf.int32, name='A')B = tf.constant([4], tf.int32, name='B')C = tf.constant([4], tf.int32, name='C')x = tf.placeholder(tf.int32, name='x')# y = Ax^2 + Bx + CAx2_1 = tf.multiply(A, tf.pow(x,2), name="Ax2_1")Bx = tf.multiply(A,x, name="Bx")y1 = tf.add_n([Ax2_1, Bx, C], name='y1')# y = Ax^2 + Bx^2Ax2_2 = tf.multiply(A, tf.pow(x,2),name='Ax2_2')Bx2 = tf.multiply(B, tf.pow(x,2),name='Bx2')y2 = tf.add_n([Ax2_2,Bx2],name='y2')y = y1 + y2with tf.Session() as sess: print(sess.run(y, feed_dict={x:[10]}))writer = tf.summary.FileWriter('./named_scope',sess.graph) writer.close()
this graph looks really comlpicated!
Now we can organize things in tensorboard using named scope. define the name scope using
with tf.name_scope("name the scope"):
and write the code inside this scope. Above program can be arranged using namescope as shown
import tensorflow as tfA = tf.constant([4], tf.int32, name='A')B = tf.constant([4], tf.int32, name='B')C = tf.constant([4], tf.int32, name='C')x = tf.placeholder(tf.int32, name='x')# y = Ax^2 + Bx + Cwith tf.name_scope("Equation1"): Ax2_1 = tf.multiply(A, tf.pow(x,2), name="Ax2_1") Bx = tf.multiply(A,x, name="Bx") y1 = tf.add_n([Ax2_1, Bx, C], name='y1')# y = Ax^2 + Bx^2with tf.name_scope("Equation2"): Ax2_2 = tf.multiply(A, tf.pow(x,2),name='Ax2_2') Bx2 = tf.multiply(B, tf.pow(x,2),name='Bx2') y2 = tf.add_n([Ax2_2,Bx2],name='y2')with tf.name_scope("final_sum"): y = y1 + y2with tf.Session() as sess: print(sess.run(y, feed_dict={x:[10]}))writer = tf.summary.FileWriter('./named_scope',sess.graph) writer.close()