Introduction to TensorFlow for Deep Learning
Tags: deeplearning, tensorflow, python, programming
Published: 2023-12-24 17:42:03


## Week 1:

- traditional coding: rules + data= answers
- ML : answers + data = rules (let computer figure out rules)
- Keras - an API in TensorFlow
- Neural Network - functions that can learn patterns
- Dense - to define a connected layer of neuron’s
- Loss functions - measures how bad or good the guess was and gives it to optimiser (mean squared error)
- Optimiser functions - figure out the next guess (SGD which stands for stochastic gradient descent)
- Importing libraries
    
    ```python
    import tensorflow as tf
    import numpy as np
    from tensorflow import keras
    
    print(tf.__version__)
    ```
    
- Defining and compiling a model
    
    ```python
    # Build a simple Sequential model
    model = tf.keras.Sequential([keras.layers.Dense(units=1, input_shape=[1])])
    
    # Compile the model
    model.compile(optimizer='sgd', loss='mean_squared_error')
    ```
    
- Providing data
    
    ```python
    # Declare model inputs and outputs for training
    xs = np.array([-1.0,  0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
    ys = np.array([-3.0, -1.0, 1.0, 3.0, 5.0, 7.0], dtype=float)
    ```
    
- Training and prediction
    
    ```python
    # Train the model
    model.fit(xs, ys, epochs=500)
    
    # Make a prediction
    print(model.predict([10.0]))
    ```
    
- For visualisation and some fun: [http://playground.tensorflow.org/](http://playground.tensorflow.org/)

## Week 2: intro to CV

- Flatten layer - convert the multi dimensional data into a linear array
- Neuron’s - variable in a function
- Using fashion MNIST dataset to perform classification of images
- Loading dataset
    
    ```python
    # Load the Fashion MNIST dataset
    fmnist = tf.keras.datasets.fashion_mnist 
    
    # Load the training and test split of the Fashion MNIST dataset
    (training_images, training_labels), (test_images, test_labels) = fmnist.load_data()
    ```
    
- Visualise the dataset
    
    ```python
    import numpy as np
    import matplotlib.pyplot as plt
    
    # You can put between 0 to 59999 here
    index = 0
    
    # Set number of characters per row when printing
    np.set_printoptions(linewidth=320)
    
    # Print the label and image
    print(f'LABEL: {training_labels[index]}')
    print(f'\nIMAGE PIXEL ARRAY:\n {training_images[index]}')
    
    # Visualize the image
    plt.imshow(training_images[index])
    ```
    
- Normalising the data (it’s a good practice to get more acc results)
    
    ```python
    # Normalize the pixel values of the train and test images
    training_images  = training_images / 255.0
    test_images = test_images / 255.0
    ```
    
- Model
    
    ```python
    # Build the classification model
    model = tf.keras.models.Sequential([tf.keras.layers.Flatten(), 
                                        tf.keras.layers.Dense(128, activation=tf.nn.relu), 
                                        tf.keras.layers.Dense(10, activation=tf.nn.softmax)])
    
    model.compile(optimizer = tf.optimizers.Adam(),
                  loss = 'sparse_categorical_crossentropy',
                  metrics=['accuracy'])
    
    model.fit(training_images, training_labels, epochs=5)
    
    # Evaluate the model on unseen data
    model.evaluate(test_images, test_labels)
    ```
    
- Sequential - That defines a sequence of layers in the neural network.
- Each layer of neuron’s need an activation function to tell them what to do. There are a lot of options, but just use these for now: ReLU effectively means:
    
    ```python
    if x > 0: 
      return x
    
    else: 
      return 0
    ```
    
- Softmax - takes a list of values and scales these so the sum of all elements will be equal to 1. When applied to model outputs, you can think of the scaled values as the probability for that class. For example, in your classification model which has 10 units in the output dense layer, having the highest value at `index = 4` means that the model is most confident that the input clothing image is a coat. If it is at index = 5, then it is a sandal, and so forth.
    
    ```python
    # Declare sample inputs and convert to a tensor
    inputs = np.array([[1.0, 3.0, 4.0, 2.0]])
    inputs = tf.convert_to_tensor(inputs)
    print(f'input to softmax function: {inputs.numpy()}')
    
    # Feed the inputs to a softmax activation function
    outputs = tf.keras.activations.softmax(inputs)
    print(f'output of softmax function: {outputs.numpy()}')
    
    # Get the sum of all values after the softmax
    sum = tf.reduce_sum(outputs)
    print(f'sum of outputs: {sum}')
    
    # Get the index with highest value
    prediction = np.argmax(outputs)
    print(f'class with highest probability: {prediction}')
    ```
    
- Predicting - classifications give the value of all the probabilities for all the labels, we choose the biggest one and that is our classification, basically how confident our model is about that classification
    
    ```python
    classifications = model.predict(test_images)
    
    print(classifications[0])
    
    print(test_labels[0])
    ```
    
- Call back - to stop the training of the model, when a certain criteria is met
    
    ```python
    class myCallback(tf.keras.callbacks.Callback):
      def on_epoch_end(self, epoch, logs={}):
        if(logs.get('accuracy') >= 0.6): # Experiment with changing this value
          print("\nReached 60% accuracy so cancelling training!")
          self.model.stop_training = True
    
    callbacks = myCallback()
    
    fmnist = tf.keras.datasets.fashion_mnist
    (training_images, training_labels) ,  (test_images, test_labels) = fmnist.load_data()
    
    training_images=training_images/255.0
    test_images=test_images/255.0
    model = tf.keras.models.Sequential([
      taf.keras.layers.Flatten(),
      tf.keras.layers.Dense(512, activation=tf.nn.relu),
      tf.keras.layers.Dense(10, activation=tf.nn.softmax)
    ])
    model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
    model.fit(training_images, training_labels, epochs=5, callbacks=[callbacks])
    ```
    

## Week 3: CNN

- Convolution - passing multiple window filters over the image, to get convolutions. The idea is that some convolutions will highlight certain aspects in the image (vertical lines, edges)
- Pooling - a way to compress an image (2x2 to 1 pixel) reducing the size while keeping most of the information intact
- So TensorFlow tries multiple different convolutions and then decides which one works and because of that the information gets filtered and the model is trained on the useful subset of information instead of the whole bunch
- Loading dataset
    
    ```python
    import tensorflow as tf
    
    # Load the Fashion MNIST dataset
    fmnist = tf.keras.datasets.fashion_mnist
    (training_images, training_labels), (test_images, test_labels) = fmnist.load_data()
    
    # Normalize the pixel values
    training_images = training_images / 255.0
    test_images = test_images / 255.0
    ```
    
- Pre processing data - reason for this is that commonly you will use 3-dimensional arrays (without counting the batch dimension) to represent image data. The third dimension represents the colour using RGB values. Since the data is in numpy.ndarray we can use functions like `reshape` and `divide`
    
    ```python
    # Reshape the images to add an extra dimension
        images = np.expand_dims(images, axis=-1)
        
        # Normalize pixel values
        images = np.divide(images,255.0)
    ```
    
- Model - in Conv2D layer the arguments are ( number of convolutions, size of the window, activation function, size of the input), in MaxPooling2D layer the argument is ( size of the window) and “max” because we are taking the biggest value of those pixels
    
    ```python
    # Define the model
    model = tf.keras.models.Sequential([
                                                             
      # Add convolutions and max pooling
      tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(28, 28, 1)),
      tf.keras.layers.MaxPooling2D(2, 2),
      tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
      tf.keras.layers.MaxPooling2D(2,2),
    
      # Add the same layers as before
      tf.keras.layers.Flatten(),
      tf.keras.layers.Dense(128, activation='relu'),
      tf.keras.layers.Dense(10, activation='softmax')
    ])
    
    # Print the model summary
    model.summary()
    
    # Use same settings
    model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
    
    # Train the model
    print(f'\nMODEL TRAINING:')
    model.fit(training_images, training_labels, epochs=5)
    
    # Evaluate on the test set
    print(f'\nMODEL EVALUATION:')
    test_loss = model.evaluate(test_images, test_labels)
    ```
    
- Summary of the model - its 26x26 instead of 28x28 because we cant use the window on the edge pixels, and by pooling the dimensions reduces by half
    

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hac05zruwo2rgbtv1cav.png)


    
- Code to visualise convolutions
    
    ```python
    import matplotlib.pyplot as plt
    from tensorflow.keras import models
    
    f, axarr = plt.subplots(3,4)
    
    FIRST_IMAGE=0
    SECOND_IMAGE=23
    THIRD_IMAGE=28
    CONVOLUTION_NUMBER = 1
    
    layer_outputs = [layer.output for layer in model.layers]
    activation_model = tf.keras.models.Model(inputs = model.input, outputs = layer_outputs)
    
    for x in range(0,4):
      f1 = activation_model.predict(test_images[FIRST_IMAGE].reshape(1, 28, 28, 1))[x]
      axarr[0,x].imshow(f1[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')
      axarr[0,x].grid(False)
      
      f2 = activation_model.predict(test_images[SECOND_IMAGE].reshape(1, 28, 28, 1))[x]
      axarr[1,x].imshow(f2[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')
      axarr[1,x].grid(False)
      
      f3 = activation_model.predict(test_images[THIRD_IMAGE].reshape(1, 28, 28, 1))[x]
      axarr[2,x].imshow(f3[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')
      axarr[2,x].grid(False)
    ```
    

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6r0p1oxtmh8djgyk5jgn.png)


    

## Week 4: using real world images

- Downloading dataset and unzipping it
    
    ```python
    !wget https://storage.googleapis.com/tensorflow-1-public/course2/week3/horse-or-human.zip
    
    import zipfile
    
    # Unzip the dataset
    local_zip = './horse-or-human.zip'
    zip_ref = zipfile.ZipFile(local_zip, 'r')
    zip_ref.extractall('./horse-or-human')
    zip_ref.close()
    ```
    
    ```python
    # Download the validation set
    !wget https://storage.googleapis.com/tensorflow-1-public/course2/week3/validation-horse-or-human.zip
    
    # Unzip validation set
    local_zip = './validation-horse-or-human.zip'
    zip_ref = zipfile.ZipFile(local_zip, 'r')
    zip_ref.extractall('./validation-horse-or-human')
    ```
    
- Extracting the data with paths
    
    ```python
    import os
    
    # Directory with our training horse pictures
    train_horse_dir = os.path.join('./horse-or-human/horses')
    
    # Directory with our training human pictures
    train_human_dir = os.path.join('./horse-or-human/humans')
    
    #to view the images
    train_horse_names = os.listdir(train_horse_dir)
    print(train_horse_names[:10])
    
    train_human_names = os.listdir(train_human_dir)
    print(train_human_names[:10])
    
    # to print the length of the data
    print('total training horse images:', len(os.listdir(train_horse_dir)))
    print('total training human images:', len(os.listdir(train_human_dir)))
    ```
    
    ```python
    # Directory with validation horse pictures
    validation_horse_dir = os.path.join('./validation-horse-or-human/horses')
    
    # Directory with validation human pictures
    validation_human_dir = os.path.join('./validation-horse-or-human/humans')
    
    validation_horse_names = os.listdir(validation_horse_dir)
    print(f'VAL SET HORSES: {validation_horse_names[:10]}')
    
    validation_human_names = os.listdir(validation_human_dir)
    print(f'VAL SET HUMANS: {validation_human_names[:10]}')
    
    print(f'total validation horse images: {len(os.listdir(validation_horse_dir))}')
    print(f'total validation human images: {len(os.listdir(validation_human_dir))}')
    ```
    
- To visualise the dataset
    
    ```python
    %matplotlib inline
    
    import matplotlib.pyplot as plt
    import matplotlib.image as mpimg
    
    # Parameters for our graph; we'll output images in a 4x4 configuration
    nrows = 4
    ncols = 4
    
    # Index for iterating over images
    pic_index = 0
    
    # Set up matplotlib fig, and size it to fit 4x4 pics
    fig = plt.gcf()
    fig.set_size_inches(ncols * 4, nrows * 4)
    
    pic_index += 8
    next_horse_pix = [os.path.join(train_horse_dir, fname) 
                    for fname in train_horse_names[pic_index-8:pic_index]]
    next_human_pix = [os.path.join(train_human_dir, fname) 
                    for fname in train_human_names[pic_index-8:pic_index]]
    
    for i, img_path in enumerate(next_horse_pix+next_human_pix):
      # Set up subplot; subplot indices start at 1
      sp = plt.subplot(nrows, ncols, i + 1)
      sp.axis('Off') # Don't show axes (or gridlines)
    
      img = mpimg.imread(img_path)
      plt.imshow(img)
    
    plt.show()
    ```
    
- Model - we are using sigmoid as activation function because its a binary classification
    
    ```python
    model = tf.keras.models.Sequential([
        # Note the input shape is the desired size of the image 300x300 with 3 bytes color
        # This is the first convolution
        tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(300, 300, 3)),
        tf.keras.layers.MaxPooling2D(2, 2),
        # The second convolution
        tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
        tf.keras.layers.MaxPooling2D(2,2),
        # The third convolution
        tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
        tf.keras.layers.MaxPooling2D(2,2),
        # The fourth convolution
        tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
        tf.keras.layers.MaxPooling2D(2,2),
        # The fifth convolution
        tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
        tf.keras.layers.MaxPooling2D(2,2),
        # Flatten the results to feed into a DNN
        tf.keras.layers.Flatten(),
        # 512 neuron hidden layer
        tf.keras.layers.Dense(512, activation='relu'),
        # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('horses') and 1 for the other ('humans')
        tf.keras.layers.Dense(1, activation='sigmoid')
    ])
    ```
    
- Models summary
    

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qr5x86hbidqwpsd3i4yu.png)



    
- Running the model - `binary_crossentropy` loss because it's a binary classification problem, and the final activation is a sigmoid. (For a refresher on loss metrics. We will use the `rmsprop` optimizer with a learning rate of `0.001`.
- In this case, using the RMSprop optimization algorithm is preferable to stochastic gradient descent (SGD), because RMSprop automates learning-rate tuning for us. (Other optimizers, such as Adam and Adagrad, also automatically adapt the learning rate during training, and would work equally well here.)
- We are using ImageDataGenerator in order to automatically label the images based on it’s directory and sub directories
    
    ```python
    from tensorflow.keras.optimizers import RMSprop
    
    model.compile(loss='binary_crossentropy',
                  optimizer=RMSprop(learning_rate=0.001),
                  metrics=['accuracy'])
    
    from tensorflow.keras.preprocessing.image import ImageDataGenerator
    
    # All images will be rescaled by 1./255
    train_datagen = ImageDataGenerator(rescale=1/255)
    
    # Flow training images in batches of 128 using train_datagen generator
    train_generator = train_datagen.flow_from_directory(
            './horse-or-human/',  # This is the source directory for training images
            target_size=(300, 300),  # All images will be resized to 300x300
            batch_size=128,
            # Since we use binary_crossentropy loss, we need binary labels
            class_mode='binary')
    				# Should be one of "binary", "categorical" or "sparse"
    history = model.fit(
          train_generator,
          steps_per_epoch=8,  
          epochs=15,
          verbose=1)
    ```
    
    ```python
    # Flow validation images in batches of 128 using validation_datagen generator
    validation_generator = validation_datagen.flow_from_directory(
            './validation-horse-or-human/',  # This is the source directory for validation images
            target_size=(300, 300),  # All images will be resized to 300x300
            batch_size=32,
            # Since you use binary_crossentropy loss, you need binary labels
            class_mode='binary')
    
    history = model.fit(
          train_generator,
          steps_per_epoch=8,  
          epochs=15,
          verbose=1,
          validation_data = validation_generator,
          validation_steps=8)
    ```
    
- Model prediction
    
    ```python
    
    import numpy as np
    from google.colab import files
    from tensorflow.keras.utils import load_img, img_to_array
    
    uploaded = files.upload()
    
    for fn in uploaded.keys():
     
      # predicting images
      path = '/content/' + fn
      img = load_img(path, target_size=(300, 300))
      x = img_to_array(img)
      x /= 255
      x = np.expand_dims(x, axis=0)
    
      images = np.vstack([x])
      classes = model.predict(images, batch_size=10)
      print(classes[0])
        
      if classes[0]>0.5:
        print(fn + " is a human")
      else:
        print(fn + " is a horse")
    ```
    
- Visualising intermediate steps in the model
    
    ```python
    import numpy as np
    import random
    from tensorflow.keras.utils import img_to_array, load_img
    
    # Define a new Model that will take an image as input, and will output
    # intermediate representations for all layers in the previous model after
    # the first.
    successive_outputs = [layer.output for layer in model.layers[1:]]
    visualization_model = tf.keras.models.Model(inputs = model.input, outputs = successive_outputs)
    
    # Prepare a random input image from the training set.
    horse_img_files = [os.path.join(train_horse_dir, f) for f in train_horse_names]
    human_img_files = [os.path.join(train_human_dir, f) for f in train_human_names]
    img_path = random.choice(horse_img_files + human_img_files)
    
    img = load_img(img_path, target_size=(300, 300))  # this is a PIL image
    x = img_to_array(img)  # Numpy array with shape (300, 300, 3)
    x = x.reshape((1,) + x.shape)  # Numpy array with shape (1, 300, 300, 3)
    
    # Scale by 1/255
    x /= 255
    
    # Run the image through the network, thus obtaining all
    # intermediate representations for this image.
    successive_feature_maps = visualization_model.predict(x)
    
    # These are the names of the layers, so you can have them as part of the plot
    layer_names = [layer.name for layer in model.layers[1:]]
    
    # Display the representations
    for layer_name, feature_map in zip(layer_names, successive_feature_maps):
      if len(feature_map.shape) == 4:
    
        # Just do this for the conv / maxpool layers, not the fully-connected layers
        n_features = feature_map.shape[-1]  # number of features in feature map
    
        # The feature map has shape (1, size, size, n_features)
        size = feature_map.shape[1]
        
        # Tile the images in this matrix
        display_grid = np.zeros((size, size * n_features))
        for i in range(n_features):
          x = feature_map[0, :, :, i]
          x -= x.mean()
          x /= x.std()
          x *= 64
          x += 128
          x = np.clip(x, 0, 255).astype('uint8')
        
          # Tile each filter into this big horizontal grid
          display_grid[:, i * size : (i + 1) * size] = x
        
        # Display the grid
        scale = 20. / n_features
        plt.figure(figsize=(scale * n_features, scale))
        plt.title(layer_name)
        plt.grid(False)
        plt.imshow(display_grid, aspect='auto', cmap='viridis')
    ```
    
    

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eu9h2iol6r9xnn4z68iy.png)




    
- Extra stuff
    
    ```python
    # to load the image and convert it into numpy format 
    from tensorflow.keras.preprocessing.image import img_to_array
    
    # Load the first example of a happy face
    sample_image  = load_img(f"{os.path.join(happy_dir, os.listdir(happy_dir)[0])}")
    
    # Convert the image into its numpy array representation
    sample_array = img_to_array(sample_image)
    
    print(f"Each image has shape: {sample_array.shape}")
    
    print(f"The maximum pixel value used is: {np.max(sample_array)}")
    ```