I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
Optimization Method
Pretty Tensor gave us the predicted class-label (y_pred) as well as a loss-measure that must be minimized, so as to improve the ability of the neural network to classify the input images.
It is unclear from the documentation for Pretty Tensor whether the loss-measure is cross-entropy or something else. But we now use the AdamOptimizer to minimize the loss.
Note that optimization is not performed at this point. In fact, nothing is calculated at all, we just add the optimizer-object to the TensorFlow graph for later execution.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
Predicted labels of the best neural network.
Can you write Python code for it?

best_net_pred_labels = pred_labels[best_net, :, :]


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
Results and analysis
Function for calculating the predicted labels for all the neural networks in the ensemble. The labels are combined further below.
Can you write Python code for it?

def ensemble_predictions():
    # Empty list of predicted labels for each of the neural networks.
    pred_labels = []

    # Classification accuracy on the test-set for each network.
    test_accuracies = []

    # Classification accuracy on the validation-set for each network.
    val_accuracies = []

    # For each neural network in the ensemble.
    for i in range(num_networks):
        # Reload the variables into the TensorFlow graph.
        saver.restore(sess=session, save_path=get_save_path(i))

        # Calculate the classification accuracy on the test-set.
        test_acc = test_accuracy()

        # Append the classification accuracy to the list.
        test_accuracies.append(test_acc)

        # Calculate the classification accuracy on the validation-set.
        val_acc = validation_accuracy()

        # Append the classification accuracy to the list.
        val_accuracies.append(val_acc)

        # Print status message.
        msg = "Network: {0}, Accuracy on Validation-Set: {1:.4f}, Test-Set: {2:.4f}"
        print(msg.format(i, val_acc, test_acc))

        # Calculate the predicted labels for the images in the test-set.
        # This is already calculated in test_accuracy() above but
        # it is re-calculated here to keep the code a bit simpler.
        pred = predict_labels(images=data.test.images)

        # Append the predicted labels to the list.
        pred_labels.append(pred)

    return np.array(pred_labels), \
           np.array(test_accuracies), \
           np.array(val_accuracies)

pred_labels, test_accuracies, val_accuracies = ensemble_predictions()


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
Size of the combined data-set.
Can you write Python code for it?

combined_size = len(combined_images)
combined_size


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
Summarize the classification accuracies on the test-set for the neural networks in the ensemble.
Can you write Python code for it?

print("Mean test-set accuracy: {0:.4f}".format(np.mean(test_accuracies)))
print("Min test-set accuracy:  {0:.4f}".format(np.min(test_accuracies)))
print("Max test-set accuracy:  {0:.4f}".format(np.max(test_accuracies)))


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
TensorFlow Graph
The entire purpose of TensorFlow is to have a so-called computational graph that can be executed much more efficiently than if the same calculations were to be performed directly in Python. TensorFlow can be more efficient than NumPy because TensorFlow knows the entire computation graph that must be executed, while NumPy only knows the computation of a single mathematical operation at a time.
TensorFlow can also automatically calculate the gradients that are needed to optimize the variables of the graph so as to make the model perform better. This is because the graph is a combination of simple mathematical expressions so the gradient of the entire graph can be calculated using the chain-rule for derivatives.
TensorFlow can also take advantage of multi-core CPUs as well as GPUs - and Google has even built special chips just for TensorFlow which are called TPUs (Tensor Processing Units) and are even faster than GPUs.
A TensorFlow graph consists of the following parts which will be detailed below:

Placeholder variables used for inputting data to the graph.
Variables that are going to be optimized so as to make the convolutional network perform better.
The mathematical formulas for the neural network.
A loss measure that can be used to guide the optimization of the variables.
An optimization method which updates the variables.

In addition, the TensorFlow graph may also contain various debugging statements e.g. for logging data to be displayed using TensorBoard, which is not covered in this tutorial.
Placeholder variables
Placeholder variables serve as the input to the TensorFlow computational graph that we may change each time we execute the graph. We call this feeding the placeholder variables and it is demonstrated further below.
First we define the placeholder variable for the input images. This allows us to change the images that are input to the TensorFlow graph. This is a so-called tensor, which just means that it is a multi-dimensional array. The data-type is set to float32 and the...
Can you write Python code for it?

x = tf.placeholder(tf.float32, shape=[None, img_size_flat], name='x')


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
The best network's predicted labels for the first of these images:
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
The best single network's predicted labels for the first of these images:
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
The classification accuracy is calculated by first type-casting the vector of booleans to floats, so that False becomes 0 and True becomes 1, and then taking the average of these numbers.
Can you write Python code for it?

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
The convolutional layers expect x to be encoded as a 4-dim tensor so we have to reshape it so its shape is instead [num_images, img_height, img_width, num_channels]. Note that img_height == img_width == img_size and num_images can be inferred automatically by using -1 for the size of the first dimension. So the reshape operation is:
Can you write Python code for it?

x_image = tf.reshape(x, [-1, img_size, img_size, num_channels])


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
The ensemble's predicted class number is then the index of the highest number in the label, which is calculated using argmax as usual.
Can you write Python code for it?

ensemble_cls_pred = np.argmax(ensemble_pred_labels, axis=1)
ensemble_cls_pred.shape


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
The ensemble's predicted labels for the first of these images (top left image):
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
The index of the neural network with the highest classification accuracy.
Can you write Python code for it?

best_net = np.argmax(test_accuracies)
best_net


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
The number of images in the test-set that were correctly classified by the best neural network.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
The predicted labels of all the networks in the ensemble, for the first of these images:
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
The predicted labels of the ensemble is a 3-dim array, the first dim is the network-number, the second dim is the image-number, the third dim is the classification vector.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
Then we create a vector of booleans telling us whether the predicted class equals the true class of each image.
Can you write Python code for it?

correct_prediction = tf.equal(y_pred_cls, y_true_cls)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/05_Ensemble_Learning.ipynb".
The first task is:
We could also have a placeholder variable for the class-number, but we will instead calculate it using argmax. Note that this is a TensorFlow operator so nothing is calculated at this point.
Can you write Python code for it?

y_true_cls = tf.argmax(y_true, dimension=1)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Analysis of Transfer-Values using PCA
Use Principal Component Analysis (PCA) from scikit-learn to reduce the array-lengths of the transfer-values from 2048 to 2 so they can be plotted.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Analysis of Transfer-Values using t-SNE
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Calculate Transfer-Values
Import a helper-function for caching the transfer-values of the Inception model.
Can you write Python code for it?

from inception import transfer_values_cache


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Calculate the true class as an integer. This could also be a placeholder variable.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Check that the array has 3000 samples and 2048 transfer-values for each sample.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Check the shape of the array with the transfer-values. There are 50,000 images in the training-set and for each image there are 2048 transfer-values.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Classification Accuracy
The output of the network y_pred is an array with 10 elements. The class number is the index of the largest element in the array.
Can you write Python code for it?

y_pred_cls = tf.argmax(y_pred, dimension=1)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Create a new PCA-object and set the target array-length to 2.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Create a new t-SNE object for the final dimensionality reduction and set the target to 2-dim.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Create an array of booleans whether the predicted class equals the true class of each image.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Create another placeholder variable for inputting the true class-label of each image. These are so-called One-Hot encoded arrays with 10 elements, one for each possible class in the data-set.
Can you write Python code for it?

y_true = tf.placeholder(tf.float32, shape=[None, num_classes], name='y_true')


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Download the Inception Model
The Inception model is downloaded from the internet. This is the default directory where you want to save the data-files. The directory will be created if it does not exist.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Download the data for the Inception model if it doesn't already exist in the directory. It is 85 MB.
See Tutorial #07 for more details.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Function for selecting a random batch of transfer-values from the training-set.
Can you write Python code for it?

def random_batch():
    # Number of images (transfer-values) in the training-set.
    num_images = len(transfer_values_train)

    # Create a random index.
    idx = np.random.choice(num_images,
                           size=train_batch_size,
                           replace=False)

    # Use the random index to select random x and y-values.
    # We use the transfer-values instead of images as x-values.
    x_batch = transfer_values_train[idx]
    y_batch = labels_train[idx]

    return x_batch, y_batch


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Helper-Functions for Showing Results
Helper-function to plot example errors
Function for plotting examples of images from the test-set that have been mis-classified.
Can you write Python code for it?

def plot_example_errors(cls_pred, correct):
    # This function is called from print_test_accuracy() below.

    # cls_pred is an array of the predicted class-number for
    # all images in the test-set.

    # correct is a boolean array whether the predicted class
    # is equal to the true class for each image in the test-set.

    # Negate the boolean array.
    incorrect = (correct == False)

    # Get the images from the test-set that have been
    # incorrectly classified.
    images = images_test[incorrect]

    # Get the predicted classes for those images.
    cls_pred = cls_pred[incorrect]

    # Get the true classes for those images.
    cls_true = cls_test[incorrect]

    n = min(9, len(images))

    # Plot the first n images.
    plot_images(images=images[0:n],
                cls_true=cls_true[0:n],
                cls_pred=cls_pred[0:n])


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Helper-function for plotting transfer-values
Can you write Python code for it?

def plot_transfer_values(i):
    print("Input image:")

    # Plot the i'th image from the test-set.
    plt.imshow(images_test[i], interpolation='nearest')
    plt.show()

    print("Transfer-values for the image using Inception model:")

    # Transform the transfer-values into an image.
    img = transfer_values_test[i]
    img = img.reshape((32, 64))

    # Plot the image for the transfer-values.
    plt.imshow(img, interpolation='nearest', cmap='Reds')
    plt.show()

plot_transfer_values(i=16)

plot_transfer_values(i=17)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Helper-function for showing the classification accuracy
Function for printing the classification accuracy on the test-set.
It takes a while to compute the classification for all the images in the test-set, that's why the results are re-used by calling the above functions directly from this function, so the classifications don't have to be recalculated by each function.
Can you write Python code for it?

def print_test_accuracy(show_example_errors=False,
                        show_confusion_matrix=False):

    # For all the images in the test-set,
    # calculate the predicted classes and whether they are correct.
    correct, cls_pred = predict_cls_test()

    # Classification accuracy and the number of correct classifications.
    acc, num_correct = classification_accuracy(correct)

    # Number of images being classified.
    num_images = len(correct)

    # Print the accuracy.
    msg = "Accuracy on Test-Set: {0:.1%} ({1} / {2})"
    print(msg.format(acc, num_correct, num_images))

    # Plot some examples of mis-classifications, if desired.
    if show_example_errors:
        print("Example errors:")
        plot_example_errors(cls_pred=cls_pred, correct=correct)

    # Plot the confusion matrix, if desired.
    if show_confusion_matrix:
        print("Confusion Matrix:")
        plot_confusion_matrix(cls_pred=cls_pred)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Helper-function to get a random training-batch
There are 50,000 images (and arrays with transfer-values for the images) in the training-set. It takes a long time to calculate the gradient of the model using all these images (transfer-values). We therefore only use a small batch of images (transfer-values) in each iteration of the optimizer.
If your computer crashes or becomes very slow because you run out of RAM, then you may try and lower this number, but you may then need to perform more optimization iterations.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Helper-function to perform optimization
This function performs a number of optimization iterations so as to gradually improve the variables of the neural network. In each iteration, a new batch of data is selected from the training-set and then TensorFlow executes the optimizer using those training samples.  The progress is printed every 100 iterations.
Can you write Python code for it?

def optimize(num_iterations):
    # Start-time used for printing time-usage below.
    start_time = time.time()

    for i in range(num_iterations):
        # Get a batch of training examples.
        # x_batch now holds a batch of images (transfer-values) and
        # y_true_batch are the true labels for those images.
        x_batch, y_true_batch = random_batch()

        # Put the batch into a dict with the proper names
        # for placeholder variables in the TensorFlow graph.
        feed_dict_train = {x: x_batch,
                           y_true: y_true_batch}

        # Run the optimizer using this batch of training data.
        # TensorFlow assigns the variables in feed_dict_train
        # to the placeholder variables and then runs the optimizer.
        # We also want to retrieve the global_step counter.
        i_global, _ = session.run([global_step, optimizer],
                                  feed_dict=feed_dict_train)

        # Print status to screen every 100 iterations (and last).
        if (i_global % 100 == 0) or (i == num_iterations - 1):
            # Calculate the accuracy on the training-batch.
            batch_acc = session.run(accuracy,
                                    feed_dict=feed_dict_train)

            # Print status.
            msg = "Global Step: {0:>6}, Training Batch Accuracy: {1:>6.1%}"
            print(msg.format(i_global, batch_acc))

    # Ending time.
    end_time = time.time()

    # Difference between start and end-times.
    time_dif = end_time - start_time

    # Print the time-usage.
    print("Time usage: " + str(timedelta(seconds=int(round(time_dif)))))


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Helper-function to plot confusion matrix
Can you write Python code for it?

# Import a function from sklearn to calculate the confusion-matrix.
from sklearn.metrics import confusion_matrix

def plot_confusion_matrix(cls_pred):
    # This is called from print_test_accuracy() below.

    # cls_pred is an array of the predicted class-number for
    # all images in the test-set.

    # Get the confusion matrix using sklearn.
    cm = confusion_matrix(y_true=cls_test,  # True class for test-set.
                          y_pred=cls_pred)  # Predicted class.

    # Print the confusion matrix as text.
    for i in range(num_classes):
        # Append the class-name to each line.
        class_name = "({}) {}".format(i, class_names[i])
        print(cm[i, :], class_name)

    # Print the class-numbers for easy reference.
    class_numbers = [" ({0})".format(i) for i in range(num_classes)]
    print("".join(class_numbers))


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Imports
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
import time
from datetime import timedelta
import os

# Functions and classes for loading and using the Inception model.
import inception

# We use Pretty Tensor to define the new classifier.
import prettytensor as pt


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Initialize Variables
The variables for the new network must be initialized before we start optimizing them.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
It takes a while to compute the PCA so the number of samples has been limited to 3000. You can try and use the full training-set if you like.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Load Data for CIFAR-10
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Load the Inception Model
Load the Inception model so it is ready for classifying images.
Note the deprecation warning, which might cause the program to fail in the future.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Load the class-names.
Can you write Python code for it?

class_names = cifar10.load_class_names()
class_names


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Load the training-set. This returns the images, the class-numbers as integers, and the class-numbers as One-Hot encoded arrays called labels.
Can you write Python code for it?

images_train, cls_train, labels_train = cifar10.load_training_data()


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Neural Network
Create the neural network for doing the classification on the CIFAR-10 data-set. This takes as input the transfer-values from the Inception model which will be fed into the placeholder variable x. The network outputs the predicted class in y_pred.
See Tutorial #03 for more details on how to use Pretty Tensor to construct neural networks.
Can you write Python code for it?

# Wrap the transfer-values as a Pretty Tensor object.
x_pretty = pt.wrap(x)

with pt.defaults_scope(activation_fn=tf.nn.relu):
    y_pred, loss = x_pretty.\
        fully_connected(size=1024, name='layer_fc1').\
        softmax_classifier(class_count=num_classes, labels=y_true)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Now create a placeholder variable for inputting the transfer-values from the Inception model into the new network that we are building. The shape of this variable is [None, transfer_len] which means it takes an input array with an arbitrary number of samples as indicated by the keyword None and each sample has 2048 elements, equal to transfer_len.
Can you write Python code for it?

x = tf.placeholder(tf.float32, shape=[None, transfer_len], name='x')


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Optimization Method
Create a variable for keeping track of the number of optimization iterations performed.
Can you write Python code for it?

global_step = tf.Variable(initial_value=0,
                          name='global_step', trainable=False)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Performance after 10,000 optimization iterations
After 10,000 optimization iterations, the classification accuracy is about 90% on the test-set. Compare this to the basic Convolutional Neural Network from Tutorial #06 which had less than 80% accuracy on the test-set.
Can you write Python code for it?

optimize(num_iterations=10000)

print_test_accuracy(show_example_errors=True,
                    show_confusion_matrix=True)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Plot a few images to see if data is correct
Can you write Python code for it?

# Get the first images from the test-set.
images = images_test[0:9]

# Get the true classes for those images.
cls_true = cls_test[0:9]

# Plot the images and labels using our helper-function above.
plot_images(images=images, cls_true=cls_true, smooth=False)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Plot the transfer-values that have been reduced using PCA. There are 10 different colors for the different classes in the CIFAR-10 data-set. The colors are grouped together but with very large overlap. This may be because PCA cannot properly separate the transfer-values.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Set the path for storing the data-set on your computer.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
TensorFlow Run
Create TensorFlow Session
Once the TensorFlow graph has been created, we have to create a TensorFlow session which is used to execute the graph.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
The CIFAR-10 data-set is about 163 MB and will be downloaded automatically if it is not located in the given path.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
The classification accuracy is calculated by first type-casting the array of booleans to floats, so that False becomes 0 and True becomes 1, and then taking the average of these numbers.
Can you write Python code for it?

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
The data dimensions have already been defined in the cifar10 module, so we just need to import the ones we need.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
This was developed using Python 3.5.2 (Anaconda) and TensorFlow version:
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/08_Transfer_Learning.ipynb".
The first task is:
Use PCA to reduce the transfer-value arrays from 2048 to 2 elements.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Close TensorFlow Session
We are now done using TensorFlow, so we close the session to release its resources.
Can you write Python code for it?

# This has been commented out in case you want to modify and experiment
# with the Notebook without having to restart it.
# session.close()


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Data Dimensions
The data dimensions are used in several places in the source-code below. They are defined once so we can use these variables instead of numbers throughout the source-code below.
Can you write Python code for it?

# We know that MNIST images are 28 pixels in each dimension.
img_size = 28

# Images are stored in one-dimensional arrays of this length.
img_size_flat = img_size * img_size

# Tuple with height and width of images used to reshape arrays.
img_shape = (img_size, img_size)

# Number of colour channels for the images: 1 channel for gray-scale.
num_channels = 1

# Number of classes, one class for each of 10 digits.
num_classes = 10


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Find the adversarial noise
Before we start optimizing the adversarial noise, we first initialize it to zero. This was already done above but it is repeated here in case you want to re-run this code with another target-class.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Helper-function for plotting images
Function used to plot 9 images in a 3x3 grid, and writing the true and predicted classes below each image. If the noise is supplied then it is added to all images.
Can you write Python code for it?

def plot_images(images, cls_true, cls_pred=None, noise=0.0):
    assert len(images) == len(cls_true) == 9

    # Create figure with 3x3 sub-plots.
    fig, axes = plt.subplots(3, 3)
    fig.subplots_adjust(hspace=0.3, wspace=0.3)

    for i, ax in enumerate(axes.flat):
        # Get the i'th image and reshape the array.
        image = images[i].reshape(img_shape)

        # Add the adversarial noise to the image.
        image += noise

        # Ensure the noisy pixel-values are between 0 and 1.
        image = np.clip(image, 0.0, 1.0)

        # Plot image.
        ax.imshow(image,
                  cmap='binary', interpolation='nearest')

        # Show true and predicted classes.
        if cls_pred is None:
            xlabel = "True: {0}".format(cls_true[i])
        else:
            xlabel = "True: {0}, Pred: {1}".format(cls_true[i], cls_pred[i])

        # Show the classes as the label on the x-axis.
        ax.set_xlabel(xlabel)

        # Remove ticks from the plot.
        ax.set_xticks([])
        ax.set_yticks([])

    # Ensure the plot is shown correctly with multiple plots
    # in a single Notebook cell.
    plt.show()


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Helper-function for showing the performance
Function for printing the classification accuracy on the test-set.
It takes a while to compute the classification for all the images in the test-set, that's why the results are re-used by calling the above functions directly from this function, so the classifications don't have to be recalculated by each function.
Note that this function can use a lot of computer memory, which is why the test-set is split into smaller batches. If you have little RAM in your computer and it crashes, then you can try and lower the batch-size.
Can you write Python code for it?

# Split the test-set into smaller batches of this size.
test_batch_size = 256

def print_test_accuracy(show_example_errors=False,
                        show_confusion_matrix=False):

    # Number of images in the test-set.
    num_test = len(data.test.images)

    # Allocate an array for the predicted classes which
    # will be calculated in batches and filled into this array.
    cls_pred = np.zeros(shape=num_test, dtype=np.int)

    # Now calculate the predicted classes for the batches.
    # We will just iterate through all the batches.
    # There might be a more clever and Pythonic way of doing this.

    # The starting index for the next batch is denoted i.
    i = 0

    while i < num_test:
        # The ending index for the next batch is denoted j.
        j = min(i + test_batch_size, num_test)

        # Get the images from the test-set between index i and j.
        images = data.test.images[i:j, :]

        # Get the associated labels.
        labels = data.test.labels[i:j, :]

        # Create a feed-dict with these images and labels.
        feed_dict = {x: images,
                     y_true: labels}

        # Calculate the predicted class using TensorFlow.
        cls_pred[i:j] = session.run(y_pred_cls, feed_dict=feed_dict)

        # Set the start-index for the next batch to the
        # end-index of the current batch.
        i = j

    # Convenience variable for the true class-numbers of the test-set.
    cls_true = data.test.cls

    # Create a boolean array whether each image is correctly classified.
    correct = (cls_true == cls_pred)

    # Calculate the number of correctly classified images.
    # When summing a boolean array, False means 0 and True means 1.
    correct_sum = correct.sum()

    # Classification accuracy is the number of correctly classified
    # images divided by the total number of images in the test-set.
    acc = float(correct_sum) / num_test

    # Print the accuracy.
    msg = "Accuracy on Test-Set: {0:.1%} ({1} / {2})"
    print(msg.format(acc, correct_sum,...


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Helper-function to perform optimization iterations
There are 55,000 images in the training-set. It takes a long time to calculate the gradient of the model using all these images. We therefore only use a small batch of images in each iteration of the optimizer.
If your computer crashes or becomes very slow because you run out of RAM, then you may try and lower this number, but you may then need to perform more optimization iterations.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Helper-function to plot confusion matrix
Can you write Python code for it?

def plot_confusion_matrix(cls_pred):
    # This is called from print_test_accuracy() below.

    # cls_pred is an array of the predicted class-number for
    # all images in the test-set.

    # Get the true classifications for the test-set.
    cls_true = data.test.cls

    # Get the confusion matrix using sklearn.
    cm = confusion_matrix(y_true=cls_true,
                          y_pred=cls_pred)

    # Print the confusion matrix as text.
    print(cm)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Helper-function to plot example errors
Function for plotting examples of images from the test-set that have been mis-classified.
Can you write Python code for it?

def plot_example_errors(cls_pred, correct):
    # This function is called from print_test_accuracy() below.

    # cls_pred is an array of the predicted class-number for
    # all images in the test-set.

    # correct is a boolean array whether the predicted class
    # is equal to the true class for each image in the test-set.

    # Negate the boolean array.
    incorrect = (correct == False)

    # Get the images from the test-set that have been
    # incorrectly classified.
    images = data.test.images[incorrect]

    # Get the predicted classes for those images.
    cls_pred = cls_pred[incorrect]

    # Get the true classes for those images.
    cls_true = data.test.cls[incorrect]

    # Get the adversarial noise from inside the TensorFlow graph.
    noise = get_noise()

    # Plot the first 9 images.
    plot_images(images=images[0:9],
                cls_true=cls_true[0:9],
                cls_pred=cls_pred[0:9],
                noise=noise)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Interestingly, the neural network now has a higher classification accuracy on noisy images than we had on clean images before all these optimizations.
Can you write Python code for it?

print_test_accuracy(show_example_errors=True,
                    show_confusion_matrix=True)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Load Data
The MNIST data-set is about 12 MB and will be downloaded automatically if it is not located in the given path.
Can you write Python code for it?

from tensorflow.examples.tutorials.mnist import input_data
data = input_data.read_data_sets('data/MNIST/', one_hot=True)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Make immune to all target-classes (double runs)
Now try and use double-runs to make the neural network immune to adversarial noise for all target-classes. Unfortunately this does not seem to work so well either.
Making the neural network immune to one adversarial target-class appears to cancel the immunity towards the other target-classes.
Can you write Python code for it?

for i in range(10):
    make_immune(target_cls=i)

    # Print newline.
    print()

    make_immune(target_cls=i)

    # Print newline.
    print()


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Make immune to noise for all target-classes
Now try and make the neural network immune to adversarial noise for all target-classes. Unfortunately this does not seem to work so well.
Can you write Python code for it?

for i in range(10):
    make_immune(target_cls=i)

    # Print newline.
    print()


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Make immune to noise for target-class 3
First try and make the neural network immune to the adverserial noise for targer-class 3.
First we find the adversarial noise that causes the neural network to mis-classify most of the images in the test-set. Then we run the normal optimization which fine-tunes the variables of the neural network to ignore this noise and this brings the classification accuracy for the noisy images up to 95-97% again.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Next we have the placeholder variable for the true labels associated with the images that were input in the placeholder variable x. The shape of this placeholder variable is [None, num_classes] which means it may hold an arbitrary number of labels and each label is a vector of length num_classes which is 10 in this case.
Can you write Python code for it?

y_true = tf.placeholder(tf.float32, shape=[None, num_classes], name='y_true')


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Normal optimization of neural network
First we perform 1000 optimization iterations with the normal optimizer. This finds the variables that makes the neural network perform well on the training-set.
The adversarial noise is not effective yet because it has only been initialized to zero above and it is not being updated during this optimization.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Note that pt.defaults_scope(activation_fn=tf.nn.relu) makes activation_fn=tf.nn.relu an argument for each of the layers constructed inside the with-block, so that Rectified Linear Units (ReLU) are used for each of these layers. The defaults_scope makes it easy to change arguments for all of the layers.
Optimizer for Normal Training
This is a list of the variables for the neural network that will be trained during the normal optimization procedure. Note that 'x_noise:0' is not in the list, so the adversarial noise is not being optimized in the normal procedure.
Can you write Python code for it?

[var.name for var in tf.trainable_variables()]


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Now perform optimization of the adversarial noise. This uses the adversarial optimizer instead of the normal optimizer, which means that it only optimizes the variable for the adversarial noise, while ignoring all the other variables of the neural network.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Now that we have wrapped the input image in a PrettyTensor object, we can add the convolutional and fully-connected layers in just a few lines of source-code.
Can you write Python code for it?

with pt.defaults_scope(activation_fn=tf.nn.relu):
    y_pred, loss = x_pretty.\
        conv2d(kernel=5, depth=16, name='layer_conv1').\
        max_pool(kernel=2, stride=2).\
        conv2d(kernel=5, depth=36, name='layer_conv2').\
        max_pool(kernel=2, stride=2).\
        flatten().\
        fully_connected(size=128, name='layer_fc1').\
        softmax_classifier(class_count=num_classes, labels=y_true)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Now we can create the new variable for the adversarial noise. It will be initialized to zero. It will not be trainable, so it will not be optimized along with the other variables of the neural network. This allows us to create two separate optimization procedures.
Can you write Python code for it?

x_noise = tf.Variable(tf.zeros([img_size, img_size, num_channels]),
                      name='x_noise', trainable=False,
                      collections=collections)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Optimization of these variables in the neural network is done with the Adam-optimizer using the loss-measure that was returned from PrettyTensor when we constructed the neural network above.
Note that optimization is not performed at this point. In fact, nothing is calculated at all, we just add the optimizer-object to the TensorFlow graph for later execution.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Optimizer for Adversarial Noise
Get the list of variables that must be optimized in the second procedure for the adversarial noise.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Red pixels show positive noise values, and blue pixels show negative noise values.
In some of these noise-images you can see traces of the numbers. For example, the noise for target-class 0 shows a red circle surrounded by blue. This means that a little noise will be added to the input image in the shape of a circle, and it will dampen the other pixels. This is sufficient for most input images in the MNIST data-set to be mis-classified as a 0. Another example is the noise for 3 which also shows traces of the number 3 with red pixels. But the noise for the other classes is less obvious.
Immunity to adversarial noise
We will now try and make the neural network immune to adversarial noise. We do this by re-training the neural network to ignore the adversarial noise. This process can be repeated a number of times.
Helper-function to make a neural network immune to noise
This is the helper-function for making the neural network immune to adversarial noise. First it runs the optimization to find the adversarial noise. Then it runs the normal optimization to make the neural network immune to that noise.
Can you write Python code for it?

def make_immune(target_cls, num_iterations_adversary=500,
                num_iterations_immune=200):

    print("Target-class:", target_cls)
    print("Finding adversarial noise ...")

    # Find the adversarial noise.
    optimize(num_iterations=num_iterations_adversary,
             adversary_target_cls=target_cls)

    # Newline.
    print()

    # Print classification accuracy.
    print_test_accuracy(show_example_errors=False,
                        show_confusion_matrix=False)

    # Newline.
    print()

    print("Making the neural network immune to the noise ...")

    # Try and make the neural network immune to this noise.
    # Note that the adversarial noise has not been reset to zero
    # so the x_noise variable still holds the noise.
    # So we are training the neural network to ignore the noise.
    optimize(num_iterations=num_iterations_immune)

    # Newline.
    print()

    # Print classification accuracy.
    print_test_accuracy(show_example_errors=False,
                        show_confusion_matrix=False)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Show the list of variable-names. There is only one, which is the adversarial noise variable that we created above.
Can you write Python code for it?

[var.name for var in adversary_variables]


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
TensorFlow Graph
The computational graph for the neural network will now be constructed using TensorFlow and PrettyTensor. As usual, we need to create placeholder variables for feeding images into the graph and then we add the adversarial noise to the images. The noisy images are then used as input to a convolutional neural network.
There are two separate optimization procedures for this network. A normal optimization procedure for the variables of the neural network itself, and another optimization procedure for the adversarial noise. Both optimization procedures are implemented directly in TensorFlow.
Placeholder variables
Placeholder variables provide the input to the computational graph in TensorFlow that we may change each time we execute the graph. We call this feeding the placeholder variables.
First we define the placeholder variable for the input images. This allows us to change the images that are input to the TensorFlow graph. This is a so-called tensor, which just means that it is a multi-dimensional array. The data-type is set to float32 and the shape is set to [None, img_size_flat], where None means that the tensor may hold an arbitrary number of images with each image being a vector of length img_size_flat.
Can you write Python code for it?

x = tf.placeholder(tf.float32, shape=[None, img_size_flat], name='x')


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
TensorFlow Run
Create TensorFlow session
Once the TensorFlow graph has been created, we have to create a TensorFlow session which is used to execute the graph.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
The adversarial noise has now been optimized and it can be shown in a plot. The red pixels show positive noise-values and the blue pixels show negative noise-values. This noise-pattern is added to every input image. The positive (red) noise-values makes the pixels darker and the negative (blue) noise-values makes the pixels brighter. Examples of this are shown below.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
The adversarial noise will be limited / clipped to the given
± noise-limit that we set above. Note that this is actually not executed at this point in the computational graph, but will instead be executed after the optimization-step, see further below.
Can you write Python code for it?

x_noise_clip = tf.assign(x_noise, tf.clip_by_value(x_noise,
                                                   -noise_limit,
                                                   noise_limit))


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
The classification accuracy is calculated by first type-casting the vector of booleans to floats, so that False becomes 0 and True becomes 1, and then taking the average of these numbers.
Can you write Python code for it?

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
The convolutional layers expect x to be encoded as a 4-dim tensor so we have to reshape it so its shape is instead [num_images, img_height, img_width, num_channels]. Note that img_height == img_width == img_size and num_images can be inferred automatically by using -1 for the size of the first dimension. So the reshape operation is:
Can you write Python code for it?

x_image = tf.reshape(x, [-1, img_size, img_size, num_channels])


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
The noisy image is just the sum of the input image and the adversarial noise.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
The optimizer for the adversarial noise will try and minimize two loss-measures: (1) The normal loss-measure for the neural network, so we will find the noise that gives the best classification accuracy for the adversarial target-class; and (2) the so-called L2-loss-measure which tries to keep the noise as low as possible.
The following weight determines how important the L2-loss is compared to the normal loss-measure. An L2-weight close to zero usually works best.
Can you write Python code for it?


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
Then we create a list of the collections that we want the new noise-variable to belong to. If we add the noise-variable to the collection tf.GraphKeys.VARIABLES then it will also get initialized with all the other variables in the TensorFlow graph, but it will not get optimized. This is a bit confusing.
Can you write Python code for it?

collections = [tf.GraphKeys.VARIABLES, ADVERSARY_VARIABLES]


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
We could also have a placeholder variable for the class-number, but we will instead calculate it using argmax. Note that this is a TensorFlow operator so nothing is calculated at this point.
Can you write Python code for it?

y_true_cls = tf.argmax(y_true, dimension=1)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
We have now created two optimizers for the neural network, one for the variables of the neural network and another for the single variable with the adversarial noise.
Performance Measures
We need a few more operations in the TensorFlow graph which will make it easier for us to display the progress to the user during optimization.
First we calculate the predicted class number from the output of the Neural Network y_pred, which is a vector with 10 elements. The class number is the index of the largest element.
Can you write Python code for it?

y_pred_cls = tf.argmax(y_pred, dimension=1)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
We will combine the loss-function for the normal optimization with a so-called L2-loss for the noise-variable. This should result in the minimum values for the adversarial noise along with the best classification accuracy.
The L2-loss is scaled by a weight that is typically set close to zero.
Can you write Python code for it?

l2_loss_noise = noise_l2_weight * tf.nn.l2_loss(x_noise)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
When this noise is added to all the images in the test-set, the result is typically a classification accuracy of 10-15% depending on the target-class that was chosen. We can also see from the confusion matrix that most images in the test-set are now classified as the desired target-class - although some of the target-classes require more adversarial noise than others.
So we have found adversarial noise that makes the neural network mis-classify almost all images in the test-set as our desired target-class.
We can also show some examples of mis-classified images with the adversarial noise. The noise is clearly visible but the digits are still easily identified by the human eye.
Can you write Python code for it?

print_test_accuracy(show_example_errors=True,
                    show_confusion_matrix=True)


I am working on the file "Exercises/Term1/TensorFlow-Tutorials/12_Adversarial_Noise_MNIST.ipynb".
The first task is:
When we create the new variable for the noise, we must inform TensorFlow which variable-collections that it belongs to, so we can later inform the two optimizers which variables to update.
First we define a name for our new variable-collection. This is just a string.
Can you write Python code for it?


I am working on the file "Exercises/Term1/keras-lab/traffic-sign-classification-with-keras.ipynb".
The first task is:
Build a Two-Layer Feedfoward Network
The code you've written so far is for data processing, not specific to Keras. Here you're going to build Keras-specific code.
Build a two-layer feedforward neural network, with 128 neurons in the fully-connected hidden layer.
To get started, review the Keras documentation about models and layers.
The Keras example of a Multi-Layer Perceptron network is similar to what you need to do here. Use that as a guide, but keep in mind that there are a number of differences.
Can you write Python code for it?

from keras.models import Sequential
from keras.layers.core import Dense, Activation
from keras.optimizers import Adam
from keras.utils import np_utils

# TODO: Build a two-layer feedforward neural network with Keras here.
model = Sequential()
model.add(Dense(128, input_shape=(flat_img_size,), name='hidden1'))
model.add(Activation('relu'))
model.add(Dense(43, name='output'))
model.add(Activation('softmax'))

# STOP: Do not change the tests below. Your implementation should pass these tests.
assert(model.get_layer(name="hidden1").input_shape == (None, 32*32*3)), "The input shape is: %s" % model.get_layer(name="hidden1").input_shape
assert(model.get_layer(name="output").output_shape == (None, 43)), "The output shape is: %s" % model.get_layer(name="output").output_shape


I am working on the file "Exercises/Term1/keras-lab/traffic-sign-classification-with-keras.ipynb".
The first task is:
Normalize the data
Now that you've loaded the training data, normalize the input so that it has a mean of 0 and a range between -0.5 and 0.5.
Can you write Python code for it?

# TODO: Implement data normalization here.
def normalize_color(image_data):
    """
    Normalize the image data with Min-Max scaling to a range of [0.1, 0.9]
    :param image_data: The image data to be normalized
    :return: Normalized image data
    """
    a = -0.5
    b = +0.5

    Xmin = 0.0
    Xmax = 255.0

    norm_img = np.empty_like(image_data, dtype=np.float32)

    norm_img = a + (image_data - Xmin)*(b-a)/(Xmax - Xmin)

    return norm_img

X_train = normalize_color(X_train)
X_test = normalize_color(X_test)

# STOP: Do not change the tests below. Your implementation should pass these tests.
assert(round(np.mean(X_train)) == 0), "The mean of the input data is: %f" % np.mean(X_train)
assert(np.min(X_train) == -0.5 and np.max(X_train) == 0.5), "The range of the input data is: %.1f to %.1f" % (np.min(X_train), np.max(X_train))


I am working on the file "Exercises/Term1/keras-lab/traffic-sign-classification-with-keras.ipynb".
The first task is:
Train the Network
Compile and train the network for 2 epochs. Use the adam optimizer, with categorical_crossentropy loss.
Hint 1: In order to use categorical cross entropy, you will need to one-hot encode the labels.
Hint 2: In order to pass the input images to the fully-connected hidden layer, you will need to reshape the input.
Hint 3: Keras's .fit() method returns a History.history object, which the tests below use. Save that to a variable named history.
Can you write Python code for it?

# One-Hot encode the labels
Y_train = np_utils.to_categorical(y_train, n_classes)
Y_test = np_utils.to_categorical(y_test, n_classes)

# Reshape input for MLP
X_train_mlp = X_train.reshape(-1, flat_img_size)
X_test_mlp = X_test.reshape(-1, flat_img_size)

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

history = model.fit(X_train_mlp, Y_train, batch_size=128, nb_epoch=10,
                    validation_data=(X_test_mlp, Y_test), verbose=1)

# STOP: Do not change the tests below. Your implementation should pass these tests.
assert(history.history['acc'][0] > 0.5), "The training accuracy was: %.3f" % history.history['acc'][0]


I am working on the file "Exercises/Term1/keras-lab/traffic-sign-classification-with-keras.ipynb".
The first task is:
Validation Accuracy: 96.98%
Pooling
Re-construct your network and add a 2x2 pooling layer immediately following your convolutional layer.
Then compile and train the network.
Can you write Python code for it?

# TODO: Re-construct the network and add a pooling layer after the convolutional layer.
model = Sequential()
model.add(Convolution2D(16, 5, 5, border_mode='same', input_shape=(32, 32, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Flatten())
model.add(Dense(128, input_shape=(flat_img_size,), name='hidden1'))
model.add(Activation('relu'))
model.add(Dense(43, name='output'))
model.add(Activation('softmax'))

# TODO: Compile and train the model.
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

history = model.fit(X_train, Y_train, batch_size=128, nb_epoch=10,
                    validation_data=(X_val, Y_val), verbose=1)

# STOP: Do not change the tests below. Your implementation should pass these tests.
## Fixed bug
assert(history.history['val_acc'][-1] > 0.9), "The validation accuracy is: %.3f" % history.history['val_acc'][0]


I am working on the file "Exercises/Term1/keras-lab/traffic-sign-classification-with-keras.ipynb".
The first task is:
Validation Accuracy: 97.36%
Dropout
Re-construct your network and add dropout after the pooling layer. Set the dropout rate to 50%.
Can you write Python code for it?

# TODO: Re-construct the network and add dropout after the pooling layer.
model = Sequential()
model.add(Convolution2D(16, 5, 5, border_mode='same', input_shape=(32, 32, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(128, input_shape=(flat_img_size,), name='hidden1'))
model.add(Activation('relu'))
model.add(Dense(43, name='output'))
model.add(Activation('softmax'))

# TODO: Compile and train the model.
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

history = model.fit(X_train, Y_train, batch_size=128, nb_epoch=10,
                    validation_data=(X_val, Y_val), verbose=1)

# STOP: Do not change the tests below. Your implementation should pass these tests.
assert(history.history['val_acc'][-1] > 0.9), "The validation accuracy is: %.3f" % history.history['val_acc'][0]


I am working on the file "Exercises/Term1/keras-lab/traffic-sign-classification-with-keras.ipynb".
The first task is:
Validation Accuracy: 97.75%
Optimization
Congratulations! You've built a neural network with convolutions, pooling, dropout, and fully-connected layers, all in just a few lines of code.
Have fun with the model and see how well you can do! Add more layers, or regularization, or different padding, or batches, or more training epochs.
What is the best validation accuracy you can achieve?
Can you write Python code for it?

pool_size = (2,2)
model = Sequential()
model.add(Convolution2D(16, 5, 5, border_mode='same', input_shape=(32, 32, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size))
model.add(Convolution2D(64, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size))
model.add(Dropout(0.5))
model.add(Convolution2D(128, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(256, input_shape=(flat_img_size,), name='hidden1'))
model.add(Activation('relu'))
model.add(Dense(43, name='output'))
model.add(Activation('softmax'))

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

history = model.fit(X_train, Y_train, batch_size=128, nb_epoch=50,
                    validation_data=(X_val, Y_val), verbose=1)


I am working on the file "Exercises/Term1/transfer-learning-lab/Transfer_Learning_Lab.ipynb".
The first task is:
Feature Extraction
Before you try feature extraction on pretrained models it's a good idea to take a moment and run the classifier you used in the Traffic Sign project on the Cifar10 dataset. Cifar10 images are also (32, 32, 3) so the only thing you'll need to change is the number of classes to 10 instead of 43.
You can easily download and load the Cifar10 dataset like this:
python
from keras.datasets import cifar10
(X_train, y_train), (X_test, y_test) = cifar10.load_data()
Cool, now you have something to compare the Cifar10 feature extraction results with!
Keep in mind the following as you experiment:
Does feature extraction outperform the Traffic Signs classifier on the Cifar10 dataset? Why?
Does feature extraction outperform the Traffic Signs classifier on the Traffic Signs dataset? Why?
Can you write Python code for it?

# load bottleneck data
X_train, y_train, X_val, y_val = load_bottleneck_data('resnet', 'traffic')

nb_epoch = 50
batch_size = 32
nb_classes = 10 # NOTE: change this to 43 if using traffic sign data

print('Feature shape', X_train.shape[1:])

model = Sequential()
model.add(Flatten(input_shape=X_train.shape[1:]))
# TODO: Define the rest of your network here

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

model.fit(
    X_train,
    y_train,
    nb_epoch=nb_epoch,
    batch_size=batch_size,
    validation_data=(X_val, y_val),
    shuffle=True
)


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
A basis for the polynomials
In the section on classes we defined a Monomial class to represent a polynomial with leading coefficient $1$. As the $N+1$ monomials $1, x, x^2, \dots, x^N$ form a basis for the vector space of polynomials of order $N$, $\mathbb{P}^N$, we can use the Monomial class to return this basis.
Exercise 1
Define a generator that will iterate through this basis of $\mathbb{P}^N$ and test it on $\mathbb{P}^3$.
Solution
Again we first take the definition of the crucial class from the notes.
Can you write Python code for it?

class Polynomial(object):
    """Representing a polynomial."""
    explanation = "I am a polynomial"

    def __init__(self, roots, leading_term):
        self.roots = roots
        self.leading_term = leading_term
        self.order = len(roots)

    def __repr__(self):
        string = str(self.leading_term)
        for root in self.roots:
            if root == 0:
                string = string + "x"
            elif root > 0:
                string = string + "(x - {})".format(root)
            else:
                string = string + "(x + {})".format(-root)
        return string

    def __mul__(self, other):
        roots = self.roots + other.roots
        leading_term = self.leading_term * other.leading_term
        return Polynomial(roots, leading_term)

    def explain_to(self, caller):
        print("Hello, {}. {}.".format(caller,self.explanation))
        print("My roots are {}.".format(self.roots))
        return None

class Monomial(Polynomial):
    """Representing a monomial, which is a polynomial with leading term 1."""
    explanation = "I am a monomial"

    def __init__(self, roots):
        Polynomial.__init__(self, roots, 1)

    def __repr__(self):
        string = ""
        for root in self.roots:
            if root == 0:
                string = string + "x"
            elif root > 0:
                string = string + "(x - {})".format(root)
            else:
                string = string + "(x + {})".format(-root)
        return string


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
An even better solution would be to use the numpy.unique function as in this stackoverflow answer (the second one!) to get the frequency of all the roots.
Exercise 2
An alternative basis is given by the monomials
\begin{align}
  p_0(x) &= 1, \ p_1(x) &= 1-x, \ p_2(x) &= (1-x)(2-x), \ \dots & \quad \dots, \ p_N(x) &= \prod_{n=1}^N (n-x).
\end{align}
Define a generator that will iterate through this basis of $\mathbb{P}^N$ and test it on $\mathbb{P}^4$.
Solution
Can you write Python code for it?

def basis_pN_variant(N):
    """
    A generator for the 'sum' basis of P^N.
    """

    for n in range(N+1):
        yield Monomial(range(n+1))

for poly in basis_pN_variant(4):
    print(poly)


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Equivalence classes
An equivalence class is a relation that groups objects in a set into related subsets. For example, if we think of the integers modulo $7$, then $1$ is in the same equivalence class as $8$ (and $15$, and $22$, and so on), and $3$ is in the same equivalence class as $10$. We use the tilde $3 \sim 10$ to denote two objects within the same equivalence class.
Here, we are going to define the positive integers programmatically from equivalent sequences.
Exercise 1
Define a python class Eqint. This should be

Initialized by a sequence;
Store the sequence;
Define its representation (via the __repr__ function) to be the integer length of the sequence;
Redefine equality (via the __eq__ function) so that two eqints are equal if their sequences have same length.

Solution
Can you write Python code for it?

class Eqint(object):

    def __init__(self, sequence):
        self.sequence = sequence

    def __repr__(self):
        return str(len(self.sequence))

    def __eq__(self, other):
        return len(self.sequence)==len(other.sequence)


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 2
500 years ago some believed that the number $2^n - 1$ was prime for all primes $n$. Use your function to find the first prime $n$ for which this is not true.
Solution
We could do this many ways. This "elegant" solution says:

Start from the smallest possible $n$ (2).
Check if $n$ is prime. If not, add one to $n$.
If $n$ is prime, check if $2^n-1$ is prime. If it is, add one to $n$.
If both those logical checks fail, we have found the $n$ we want.
Can you write Python code for it?

n = 2
while (not isprime(n)) or (isprime(2**n-1)):
    n += 1
print("The first n such that 2^n-1 is not prime is {}.".format(n))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 2
Check the points $c=0$ and $c=\pm 2 \pm 2 \text{i}$ and ensure they do what you expect. (What should you expect?)
Solution
Can you write Python code for it?

c_values = [0.0, 2+2j, 2-2j, -2+2j, -2-2j]
for c in c_values:
    print("Is {} in the Mandelbrot set? {}.".format(c, in_Mandelbrot(c)))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 2
Define a class Rational that uses the normal_form function to store the rational number in the appropriate form. Define a __repr__ function that prints a string that looks like $\frac{n}{d}$ (hint: use len(str(number)) to find the number of digits of an integer). Test it on the cases above.
Solution
Can you write Python code for it?

class Rational(object):
    """
    A rational number.
    """

    def __init__(self, numerator, denominator):

        n, d = normal_form(numerator, denominator)

        self.numerator = n
        self.denominator = d

        return None

    def __repr__(self):

        max_length = max(len(str(self.numerator)), len(str(self.denominator)))

        if self.denominator == 1:
            frac = str(self.numerator)
        else:
            numerator = str(self.numerator)+'\n'
            bar = max_length*'-'+'\n'
            denominator = str(self.denominator)
            frac = numerator+bar+denominator

        return frac

q1 = Rational(3, 2)
print(q1)
q2 = Rational(15, 3)
print(q2)
q3 = Rational(20, 42)
print(q3)


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 2
Define a zero object from the empty list, and three one objects, from a single object list, tuple, and string. For example
python
one_list = Eqint([1])
one_tuple = Eqint((1,))
one_string = Eqint('1')
Check that none of the one objects equal the zero object, but all equal the other one objects. Print each object to check that the representation gives the integer length.
Solution
Can you write Python code for it?

zero = Eqint([])
one_list = Eqint([1])
one_tuple = Eqint((1,))
one_string = Eqint('1')

print("Is zero equivalent to one? {}, {}, {}".format(zero == one_list,
                                                     zero == one_tuple,
                                                     zero == one_string))
print("Is one equivalent to one? {}, {}, {}.".format(one_list == one_tuple,
                                                     one_list == one_string,
                                                     one_tuple == one_string))
print(zero)
print(one_list)
print(one_tuple)
print(one_string)


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 2
Find how many twin primes there are with $p_2 < 1000$.
Solution
Again there are many solutions, but the itertools recipes has the quantify pattern. Looking ahead to exercise 3 we'll define:
Can you write Python code for it?

def pi_N(N):
    """
    Use the quantify pattern from itertools to count the number of twin primes.
    """
    return sum(map(check_twin, pair_primes(N)))

pi_N(1000)


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 2
Fix $\sigma=10, \beta=8/3$. Set initial data to be $\vec{v}(0) = \vec{1}$. Using scipy, specifically the odeint function of scipy.integrate, solve the Lorenz system up to $t=100$ for $\rho=13, 14, 15$ and $28$.
Plot your results in 3d, plotting $x, y, z$.
Solution
Can you write Python code for it?

import numpy
from scipy.integrate import odeint

v0 = [1.0, 1.0, 1.0]
sigma = 10.0
beta = 8.0/3.0
t_values = numpy.linspace(0.0, 100.0, 5000)
rho_values = [13.0, 14.0, 15.0, 28.0]
v_values = []
for rho in rho_values:
    params = (sigma, rho, beta)
    v = odeint(dvdt, v0, t_values, args=params)
    v_values.append(v)

%matplotlib inline
from matplotlib import pyplot
from mpl_toolkits.mplot3d.axes3d import Axes3D

fig = pyplot.figure(figsize=(12,6))
for i, v in enumerate(v_values):
    ax = fig.add_subplot(2,2,i+1,projection='3d')
    ax.plot(v[:,0], v[:,1], v[:,2])
    ax.set_xlabel(r'$x$')
    ax.set_ylabel(r'$y$')
    ax.set_zlabel(r'$z$')
    ax.set_title(r"$\rho={}$".format(rho_values[i]))
pyplot.show()


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 2
Fix $x_0=0.5$. Calculate the first 2,000 members of the sequence for $r=1.5$ and $r=3.5$ Plot the last 100 members of the sequence in both cases.
What does this suggest about the long-term behaviour of the sequence?
Solution
Can you write Python code for it?

import numpy
from matplotlib import pyplot
%matplotlib inline

x0 = 0.5
N = 2000
sequence1 = logistic(x0, 1.5, N)
sequence2 = logistic(x0, 3.5, N)
pyplot.plot(sequence1[-100:], 'b-', label = r'$r=1.5$')
pyplot.plot(sequence2[-100:], 'k-', label = r'$r=3.5$')
pyplot.xlabel(r'$n$')
pyplot.ylabel(r'$x$')
pyplot.show()


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 2
Investigate the help for the dsolve function to straightforwardly impose the initial condition $y(0) = 1$ using the ics argument. Using this, compute the specific solutions that satisfy the ODE for $n = 2, \dots, 10$.
Solution
Can you write Python code for it?

for n in range(2, 11):
    ode_solution = sympy.dsolve(sympy.diff(y(t), t) + y(t)**n - sympy.exp(-t), y(t),
                                ics = {y(0) : 1})
    print(ode_solution)


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 2
The more interesting statement in the paper is that
\begin{equation}
  27^5 + 84^5 + 110^5 + 133^5 = 144^5.
\end{equation}

[is] the smallest instance in which four fifth powers sum to a fifth power.

Interpreting "the smallest instance" to mean the solution where the right hand side term (the largest integer) is the smallest, we want to use python to check this statement.
You may find the combinations function from the itertools package useful.
Can you write Python code for it?


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 2
Using the math module, check your result for $15$ factorial. You should explore the help for the math library and its functions, using eg tab-completion, the spyder inspector, or online sources.
Solution
Can you write Python code for it?

import math
print(math.factorial(15))
print("Result correct?", math.factorial(15) == fifteen_factorial)


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 2a
Show, by getting python to compute the number of combinations $N = \begin{pmatrix} n \ 4 \end{pmatrix}$ that $N$ grows roughly as $n^4$. To do this, plot the number of combinations and $n^4$ on a log-log scale. Restrict to $n \le 50$.
Solution
Can you write Python code for it?

from matplotlib import pyplot
%matplotlib inline

n = numpy.arange(5, 51)
N = numpy.zeros_like(n)
for i, n_c in enumerate(n):
    combinations = list(itertools.combinations(numpy.arange(1,n_c+1), 4))
    N[i] = len(combinations)

pyplot.figure(figsize=(12,6))
pyplot.loglog(n, N, linestyle='None', marker='x', color='k', label='Combinations')
pyplot.loglog(n, n**4, color='b', label=r'$n^4$')
pyplot.xlabel(r'$n$')
pyplot.ylabel(r'$N$')
pyplot.legend(loc='upper left')
pyplot.show()


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 3
Fix $\rho = 28$. Solve the Lorenz system twice, up to $t=40$, using the two different initial conditions $\vec{v}(0) = \vec{1}$ and $\vec{v}(0) = \vec{1} + \vec{10^{-5}}$.
Show four plots. Each plot should show the two solutions on the same axes, plotting $x, y$ and $z$. Each plot should show $10$ units of time, ie the first shows $t \in [0, 10]$, the second shows $t \in [10, 20]$, and so on.
Solution
Can you write Python code for it?

t_values = numpy.linspace(0.0, 40.0, 4000)
rho = 28.0
params = (sigma, rho, beta)
v_values = []
v0_values = [[1.0,1.0,1.0],
             [1.0+1e-5,1.0+1e-5,1.0+1e-5]]
for v0 in v0_values:
    v = odeint(dvdt, v0, t_values, args=params)
    v_values.append(v)

fig = pyplot.figure(figsize=(12,6))
line_colours = 'by'
for tstart in range(4):
    ax = fig.add_subplot(2,2,tstart+1,projection='3d')
    for i, v in enumerate(v_values):
        ax.plot(v[tstart*1000:(tstart+1)*1000,0],
                v[tstart*1000:(tstart+1)*1000,1],
                v[tstart*1000:(tstart+1)*1000,2],
                color=line_colours[i])
    ax.set_xlabel(r'$x$')
    ax.set_ylabel(r'$y$')
    ax.set_zlabel(r'$z$')
    ax.set_title(r"$t \in [{},{}]$".format(tstart*10, (tstart+1)*10))
pyplot.show()


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 3
Let $\pi_N$ be the number of twin primes such that $p_2 < N$. Plot how $\pi_N / N$ varies with $N$ for $N=2^k$ and $k = 4, 5, \dots 16$. (You should use a logarithmic scale where appropriate!)
Solution
We've now done all the hard work and can use the solutions above.
Can you write Python code for it?

import numpy
from matplotlib import pyplot
%matplotlib inline

N = numpy.array([2**k for k in range(4, 17)])
twin_prime_fraction = numpy.array(list(map(pi_N, N))) / N

pyplot.semilogx(N, twin_prime_fraction)
pyplot.xlabel(r"$N$")
pyplot.ylabel(r"$\pi_N / N$")
pyplot.show()


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 3
Overload the __add__ function so that you can add two rational numbers. Test it on $\frac{1}{2} + \frac{1}{3} + \frac{1}{6} = 1$.
Solution
Can you write Python code for it?

class Rational(object):
    """
    A rational number.
    """

    def __init__(self, numerator, denominator):

        n, d = normal_form(numerator, denominator)

        self.numerator = n
        self.denominator = d

        return None

    def __add__(a, b):

        numerator = a.numerator * b.denominator + b.numerator * a.denominator
        denominator = a.denominator * b.denominator
        return Rational(numerator, denominator)

    def __repr__(self):

        max_length = max(len(str(self.numerator)), len(str(self.denominator)))

        if self.denominator == 1:
            frac = str(self.numerator)
        else:
            numerator = str(self.numerator)+'\n'
            bar = max_length*'-'+'\n'
            denominator = str(self.denominator)
            frac = numerator+bar+denominator

        return frac

print(Rational(1,2) + Rational(1,3) + Rational(1,6))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 3
Redefine the class by including an __add__ method that combines the two sequences. That is, if a and b are Eqints then a+b should return an Eqint defined from combining a and bs sequences.
Note
Adding two different types of sequences (eg, a list to a tuple) does not work, so it is better to either iterate over the sequences, or to convert to a uniform type before adding.
Solution
Can you write Python code for it?

class Eqint(object):

    def __init__(self, sequence):
        self.sequence = sequence

    def __repr__(self):
        return str(len(self.sequence))

    def __eq__(self, other):
        return len(self.sequence)==len(other.sequence)

    def __add__(a, b):
        return Eqint(tuple(a.sequence) + tuple(b.sequence))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 3
Stirling's approximation gives that, for large enough $n$,
\begin{equation}
  n! \simeq \sqrt{2 \pi} n^{n + 1/2} e^{-n}.
\end{equation}
Using functions and constants from the math library, compare the results of $n!$ and Stirling's approximation for $n = 5, 10, 15, 20$. In what sense does the approximation improve?
Solution
Can you write Python code for it?

print(math.factorial(5), math.sqrt(2*math.pi)*5**(5+0.5)*math.exp(-5))
print(math.factorial(10), math.sqrt(2*math.pi)*10**(10+0.5)*math.exp(-10))
print(math.factorial(15), math.sqrt(2*math.pi)*15**(15+0.5)*math.exp(-15))
print(math.factorial(20), math.sqrt(2*math.pi)*20**(20+0.5)*math.exp(-20))
print("Absolute differences:")
print(math.factorial(5) - math.sqrt(2*math.pi)*5**(5+0.5)*math.exp(-5))
print(math.factorial(10) - math.sqrt(2*math.pi)*10**(10+0.5)*math.exp(-10))
print(math.factorial(15) - math.sqrt(2*math.pi)*15**(15+0.5)*math.exp(-15))
print(math.factorial(20) - math.sqrt(2*math.pi)*20**(20+0.5)*math.exp(-20))
print("Relative differences:")
print((math.factorial(5) - math.sqrt(2*math.pi)*5**(5+0.5)*math.exp(-5)) / math.factorial(5))
print((math.factorial(10) - math.sqrt(2*math.pi)*10**(10+0.5)*math.exp(-10)) / math.factorial(10))
print((math.factorial(15) - math.sqrt(2*math.pi)*15**(15+0.5)*math.exp(-15)) / math.factorial(15))
print((math.factorial(20) - math.sqrt(2*math.pi)*20**(20+0.5)*math.exp(-20)) / math.factorial(20))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 3
The Mersenne primes are those that have the form $2^n-1$, where $n$ is prime. Use your previous solutions to generate all the $n < 40$ that give Mersenne primes.
Solution
Can you write Python code for it?

for n in range(2, 41):
    if isprime(n) and isprime(2**n-1):
        print("n={} is such that 2^n-1 is prime.".format(n))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 3
Using the removeO command, plot each of these solutions for $t \in [0, 1]$.
Can you write Python code for it?

%matplotlib inline

for n in range(2, 11):
    ode_solution = sympy.dsolve(sympy.diff(y(t), t) + y(t)**n - sympy.exp(-t), y(t),
                                ics = {y(0) : 1})
    sympy.plot(ode_solution.rhs.removeO(), (t, 0, 1));


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 3
Write a function that computes the area of a triangle with edge lengths $a, b, c$. You may use the formula
\begin{equation}
  A = \sqrt{s (s - a) (s - b) (s - c)}, \qquad s = \frac{a + b + c}{2}.
\end{equation}
Construct your own test cases to cover a range of possibilities.
Can you write Python code for it?

def triangle_area(a, b, c):
    """
    Compute the area of a triangle with edge lengths a, b, c.
    Area is sqrt(s (s-a) (s-b) (s-c)).
    s is (a+b+c)/2.
    Only makes sense if all are non-negative.

    Parameters
    ----------

    a : float
        Edge length 1
    b : float
        Edge length 2
    c : float
        Edge length 3

    Returns
    -------

    area : float
        The triangle area.
    """

    from math import sqrt

    if (a < 0.0) or (b < 0.0) or (c < 0.0):
        print("Negative edge length makes no sense!")
        return 0

    s = 0.5 * (a + b + c)
    return sqrt(s * (s-a) * (s-b) * (s-c))

print(triangle_area(1,1,1)) # Equilateral; answer sqrt(3)/4 ~ 0.433
print(triangle_area(3,4,5)) # Right triangle; answer 6
print(triangle_area(1,1,0)) # Not a triangle; answer 0
print(triangle_area(-1,1,1)) # Not a triangle; exception or 0.


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 3
Write a function that, given $N$

generates an $N \times N$ grid spanning $c = x + \text{i} y$, for $-2 \le x \le 2$ and $-2 \le y \le 2$;
returns an $N\times N$ array containing one if the associated grid point is in the Mandelbrot set, and zero otherwise.

Solution
Can you write Python code for it?

import numpy

def grid_Mandelbrot(N):
    x = numpy.linspace(-2.0, 2.0, N)
    X, Y = numpy.meshgrid(x, x)
    C = X + 1j*Y
    grid = numpy.zeros((N, N), int)
    for nx in range(N):
        for ny in range(N):
            grid[nx, ny] = int(in_Mandelbrot(C[nx, ny]))
    return grid


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 4
Check your addition function by adding together all your previous Eqint objects (which will need re-defining, as the class has been redefined). Print the resulting object to check you get 3, and also print its internal sequence.
Solution
Can you write Python code for it?

zero = Eqint([])
one_list = Eqint([1])
one_tuple = Eqint((1,))
one_string = Eqint('1')

sum_eqint = zero + one_list + one_tuple + one_string
print("The sum is {}.".format(sum_eqint))
print("The internal sequence is {}.".format(sum_eqint.sequence))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 4
For iterative maps such as the logistic map, one of three things can occur:

The sequence settles down to a fixed point.
The sequence rotates through a finite number of values. This is called a limit cycle.
The sequence generates an infinite number of values. This is called deterministic chaos.

Using just your plot, or new plots from this data, work out approximate values of $r$ for which there is a transition from fixed points to limit cycles, from limit cycles of a given number of values to more values, and the transition to chaos.
Solution
The first transition is at $r \approx 3$, the next at $r \approx 3.45$, the next at $r \approx 3.55$. The transition to chaos appears to happen before $r=4$, but it's not obvious exactly where.
Mandelbrot
The Mandelbrot set is also generated from a sequence, ${ z_n }$, using the relation
\begin{equation}
  z_{n+1} = z_n^2 + c, \qquad z_0 = 0.
\end{equation}
The members of the sequence, and the constant $c$, are all complex. The point in the complex plane at $c$ is in the Mandelbrot set only if the $|z_n| < 2$ for all members of the sequence. In reality, checking the first 100 iterations is sufficient.
Note: the python notation for a complex number $x + \text{i} y$ is x + yj: that is, j is used to indicate $\sqrt{-1}$. If you know the values of x and y then x + yj constructs a complex number; if they are stored in variables you can use complex(x, y).
Exercise 1
Write a function that checks if the point $c$ is in the Mandelbrot set.
Solution
Can you write Python code for it?

def in_Mandelbrot(c, n_iterations = 100):
    z0 = 0.0 + 0j
    in_set = True
    n = 0
    zn = z0
    while in_set and (n < n_iterations):
        n += 1
        znew = zn**2 + c
        in_set = abs(znew) < 2.0
        zn = znew
    return in_set


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 4
Overload the __mul__ function so that you can multiply two rational numbers. Test it on $\frac{1}{3} \times \frac{15}{2} \times \frac{2}{5} = 1$.
Solution
Can you write Python code for it?

class Rational(object):
    """
    A rational number.
    """

    def __init__(self, numerator, denominator):

        n, d = normal_form(numerator, denominator)

        self.numerator = n
        self.denominator = d

        return None

    def __add__(a, b):

        numerator = a.numerator * b.denominator + b.numerator * a.denominator
        denominator = a.denominator * b.denominator
        return Rational(numerator, denominator)

    def __mul__(a, b):

        numerator = a.numerator * b.numerator
        denominator = a.denominator * b.denominator
        return Rational(numerator, denominator)

    def __repr__(self):

        max_length = max(len(str(self.numerator)), len(str(self.denominator)))

        if self.denominator == 1:
            frac = str(self.numerator)
        else:
            numerator = str(self.numerator)+'\n'
            bar = max_length*'-'+'\n'
            denominator = str(self.denominator)
            frac = numerator+bar+denominator

        return frac

print(Rational(1,3)*Rational(15,2)*Rational(2,5))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 4
Using the function imshow from matplotlib, plot the resulting array for a $100 \times 100$ array to make sure you see the expected shape.
Solution
Can you write Python code for it?

from matplotlib import pyplot
%matplotlib inline

pyplot.imshow(grid_Mandelbrot(100))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 4
Write a function to compute all prime factors of an integer $n$, including their multiplicities. Test it by printing the prime factors (without multiplicities) of $n = 17, \dots, 20$ and the multiplicities (without factors) of $n = 48$.
Note
One effective solution is to return a dictionary, where the keys are the factors and the values are the multiplicities.
Solution
This solution uses the trick of immediately dividing $n$ by any divisor: this means we never have to check the divisor for being prime.
Can you write Python code for it?

def prime_factors(n):
    """
    Generate all the prime factors of n.

    Parameters
    ----------

    n : integer
        Number to be checked

    Returns
    -------

    factors : dict
        Prime factors (keys) and multiplicities (values)
    """

    factors = {}

    m = 2
    while m <= n:
        if n%m == 0:
            factors[m] = 1
            n //= m
            while n%m == 0:
                factors[m] += 1
                n //= m
        m += 1

    return factors

for n in range(17, 21):
    print("Prime factors of {} are {}.".format(n, prime_factors(n).keys()))
print("Multiplicities of prime factors of 48 are {}.".format(prime_factors(48).values()))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 5
Modify your functions so that, instead of returning whether a point is inside the set or not, it returns the logarithm of the number of iterations it takes. Plot the result using imshow again.
Solution
Can you write Python code for it?

from math import log

def log_Mandelbrot(c, n_iterations = 100):
    z0 = 0.0 + 0j
    in_set = True
    n = 0
    zn = z0
    while in_set and (n < n_iterations):
        n += 1
        znew = zn**2 + c
        in_set = abs(znew) < 2.0
        zn = znew
    return log(n)

def log_grid_Mandelbrot(N):
    x = numpy.linspace(-2.0, 2.0, N)
    X, Y = numpy.meshgrid(x, x)
    C = X + 1j*Y
    grid = numpy.zeros((N, N), int)
    for nx in range(N):
        for ny in range(N):
            grid[nx, ny] = log_Mandelbrot(C[nx, ny])
    return grid

from matplotlib import pyplot
%matplotlib inline

pyplot.imshow(log_grid_Mandelbrot(100))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 5
Overload the __rmul__ function so that you can multiply a rational by an integer. Check that $\frac{1}{2} \times 2 = 1$ and $\frac{1}{2} + (-1) \times \frac{1}{2} = 0$. Also overload the __sub__ function (using previous functions!) so that you can subtract rational numbers and check that $\frac{1}{2} - \frac{1}{2} = 0$.
Solution
Can you write Python code for it?

class Rational(object):
    """
    A rational number.
    """

    def __init__(self, numerator, denominator):

        n, d = normal_form(numerator, denominator)

        self.numerator = n
        self.denominator = d

        return None

    def __add__(a, b):

        numerator = a.numerator * b.denominator + b.numerator * a.denominator
        denominator = a.denominator * b.denominator
        return Rational(numerator, denominator)

    def __mul__(a, b):

        numerator = a.numerator * b.numerator
        denominator = a.denominator * b.denominator
        return Rational(numerator, denominator)

    def __rmul__(self, other):

        numerator = self.numerator * other
        return Rational(numerator, self.denominator)

    def __sub__(a, b):

        return a + (-1)*b

    def __repr__(self):

        max_length = max(len(str(self.numerator)), len(str(self.denominator)))

        if self.denominator == 1:
            frac = str(self.numerator)
        else:
            numerator = str(self.numerator)+'\n'
            bar = max_length*'-'+'\n'
            denominator = str(self.denominator)
            frac = numerator+bar+denominator

        return frac

half = Rational(1,2)
print(2*half)
print(half+(-1)*half)
print(half-half)


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 5
We will sketch a construction of the positive integers from nothing.

Define an empty list positive_integers.
Define an Eqint called zero from the empty list. Append it to positive_integers.
Define an Eqint called next_integer from the Eqint defined by a copy of positive_integers (ie, use Eqint(list(positive_integers)). Append it to positive_integers.
Repeat step 3 as often as needed.

Use this procedure to define the Eqint equivalent to $10$. Print it, and its internal sequence, to check.
Solution
Can you write Python code for it?

positive_integers = []
zero = Eqint([])
positive_integers.append(zero)

N = 10
for n in range(1,N+1):
    positive_integers.append(Eqint(list(positive_integers)))

print("The 'final' Eqint is {}".format(positive_integers[-1]))
print("Its sequence is {}".format(positive_integers[-1].sequence))
print("That is, it contains all Eqints with length less than 10.")


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 5
Write a function to generate all the integer divisors, including 1, but not including $n$ itself, of an integer $n$. Test it on $n = 16, \dots, 20$.
Note
You could use the prime factorization from the previous exercise, or you could do it directly.
Solution
Here we will do it directly.
Can you write Python code for it?

def divisors(n):
    """
    Generate all integer divisors of n.

    Parameters
    ----------

    n : integer
        Number to be checked

    Returns
    -------

    divs : list
        All integer divisors, including 1.
    """

    divs = [1]
    m = 2
    while m <= n/2:
        if n%m == 0:
            divs.append(m)
        m += 1

    return divs

for n in range(16, 21):
    print("The divisors of {} are {}.".format(n, divisors(n)))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 6
A perfect number $n$ is one where the divisors sum to $n$. For example, 6 has divisors 1, 2, and 3, which sum to 6. Use your previous solution to find all perfect numbers $n < 10,000$ (there are only four!).
Solution
We can do this much more efficiently than the code below using packages such as numpy, but this is a "bare python" solution.
Can you write Python code for it?

def isperfect(n):
    """
    Check if a number is perfect.

    Parameters
    ----------

    n : integer
        Number to check

    Returns
    -------

    isperfect : Boolean
        Whether it is perfect or not.
    """

    divs = divisors(n)
    sum_divs = 0
    for d in divs:
        sum_divs += d

    return n == sum_divs

for n in range(2,10000):
    if (isperfect(n)):
        factors = prime_factors(n)
        print("{} is perfect.\n"
              "Divisors are {}.\n"
              "Prime factors {} (multiplicities {}).".format(
            n, divisors(n), factors.keys(), factors.values()))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 6
Overload the __float__ function so that float(q) returns the floating point approximation to the rational number q. Test this on $\frac{1}{2}, \frac{1}{3}$, and $\frac{1}{11}$.
Solution
Can you write Python code for it?

class Rational(object):
    """
    A rational number.
    """

    def __init__(self, numerator, denominator):

        n, d = normal_form(numerator, denominator)

        self.numerator = n
        self.denominator = d

        return None

    def __add__(a, b):

        numerator = a.numerator * b.denominator + b.numerator * a.denominator
        denominator = a.denominator * b.denominator
        return Rational(numerator, denominator)

    def __mul__(a, b):

        numerator = a.numerator * b.numerator
        denominator = a.denominator * b.denominator
        return Rational(numerator, denominator)

    def __rmul__(self, other):

        numerator = self.numerator * other
        return Rational(numerator, self.denominator)

    def __sub__(a, b):

        return a + (-1)*b

    def __float__(a):

        return float(a.numerator) / float(a.denominator)

    def __repr__(self):

        max_length = max(len(str(self.numerator)), len(str(self.denominator)))

        if self.denominator == 1:
            frac = str(self.numerator)
        else:
            numerator = str(self.numerator)+'\n'
            bar = max_length*'-'+'\n'
            denominator = str(self.denominator)
            frac = numerator+bar+denominator

        return frac

print(float(Rational(1,2)))
print(float(Rational(1,3)))
print(float(Rational(1,11)))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 6
The function $f_1(x) = e^x$ has derivative with the exact value $1$ at $x=0$. Compute the approximate derivative using your function above, for $\delta = 10^{-2 n}$ with $n = 1, \dots, 7$. You should see the results initially improve, then get worse. Why is this?
Solution
Can you write Python code for it?

from math import exp
for n in range(1, 8):
    print("For n={}, the approx derivative is {}.".format(n, g(exp, 0.0, 10**(-2.0*n))))


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 6
Try some higher resolution plots, and try plotting only a section to see the structure. Note this is not a good way to get high accuracy close up images!
Solution
This is a simple example:
Can you write Python code for it?


I am working on the file "ExercisesSolutions.ipynb".
The first task is:
Exercise 7
Overload the __lt__ function to compare two rational numbers. Create a list of rational numbers where the denominator is $n = 2, \dots, 11$ and the numerator is the floored integer $n/2$, ie n//2. Use the sorted function on that list (which relies on the __lt__ function).
Solution
Can you write Python code for it?

class Rational(object):
    """
    A rational number.
    """

    def __init__(self, numerator, denominator):

        n, d = normal_form(numerator, denominator)

        self.numerator = n
        self.denominator = d

        return None

    def __add__(a, b):

        numerator = a.numerator * b.denominator + b.numerator * a.denominator
        denominator = a.denominator * b.denominator
        return Rational(numerator, denominator)

    def __mul__(a, b):

        numerator = a.numerator * b.numerator
        denominator = a.denominator * b.denominator
        return Rational(numerator, denominator)

    def __rmul__(self, other):

        numerator = self.numerator * other
        return Rational(numerator, self.denominator)

    def __sub__(a, b):

        return a + (-1)*b

    def __float__(a):

        return float(a.numerator) / float(a.denominator)

    def __lt__(a, b):

        return a.numerator * b.denominator < a.denominator * b.numerator

    def __repr__(self):

        max_length = max(len(str(self.numerator)), len(str(self.denominator)))

        if self.denominator == 1:
            frac = str(self.numerator)
        else:
            numerator = '\n'+str(self.numerator)+'\n'
            bar = max_length*'-'+'\n'
            denominator = str(self.denominator)
            frac = numerator+bar+denominator

        return frac

q_list = [Rational(n//2, n) for n in range(2, 12)]
print(sorted(q_list))