I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Helper-function for loading images
The data-set is not loaded into memory, instead it has a list of the files for the images in the training-set and another list of the files for the images in the test-set. This helper-function loads some image-files.
Can you write Python code for it?

def load_images(image_paths):
    # Load the images from disk.
    images = [plt.imread(path) for path in image_paths]

    # Convert to a numpy array and return it.
    return np.asarray(images)


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Helper-function for plotting example errors
Function for plotting examples of images from the test-set that have been mis-classified.
Can you write Python code for it?

def plot_example_errors(cls_pred):
    # cls_pred is an array of the predicted class-number for
    # all images in the test-set.

    # Boolean array whether the predicted class is incorrect.
    incorrect = (cls_pred != cls_test)

    # Get the file-paths for images that were incorrectly classified.
    image_paths = np.array(image_paths_test)[incorrect]

    # Load the first 9 images.
    images = load_images(image_paths=image_paths[0:9])

    # Get the predicted classes for those images.
    cls_pred = cls_pred[incorrect]

    # Get the true classes for those images.
    cls_true = cls_test[incorrect]

    # Plot the 9 images we have loaded and their corresponding classes.
    # We have only loaded 9 images so there is no need to slice those again.
    plot_images(images=images,
                cls_true=cls_true[0:9],
                cls_pred=cls_pred[0:9])


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Helper-function for plotting images
Function used to plot at most 9 images in a 3x3 grid, and writing the true and predicted classes below each image.
Can you write Python code for it?

def plot_images(images, cls_true, cls_pred=None, smooth=True):

    assert len(images) == len(cls_true)

    # Create figure with sub-plots.
    fig, axes = plt.subplots(3, 3)

    # Adjust vertical spacing.
    if cls_pred is None:
        hspace = 0.3
    else:
        hspace = 0.6
    fig.subplots_adjust(hspace=hspace, wspace=0.3)

    # Interpolation type.
    if smooth:
        interpolation = 'spline16'
    else:
        interpolation = 'nearest'

    for i, ax in enumerate(axes.flat):
        # There may be less than 9 images, ensure it doesn't crash.
        if i < len(images):
            # Plot image.
            ax.imshow(images[i],
                      interpolation=interpolation)

            # Name of the true class.
            cls_true_name = class_names[cls_true[i]]

            # Show true and predicted classes.
            if cls_pred is None:
                xlabel = "True: {0}".format(cls_true_name)
            else:
                # Name of the predicted class.
                cls_pred_name = class_names[cls_pred[i]]

                xlabel = "True: {0}\nPred: {1}".format(cls_true_name, cls_pred_name)

            # Show the classes as the label on the x-axis.
            ax.set_xlabel(xlabel)

        # Remove ticks from the plot.
        ax.set_xticks([])
        ax.set_yticks([])

    # Ensure the plot is shown correctly with multiple plots
    # in a single Notebook cell.
    plt.show()


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Helper-function for plotting training history
This plots the classification accuracy and loss-values recorded during training with the Keras API.
Can you write Python code for it?

def plot_training_history(history):
    # Get the classification accuracy and loss-value
    # for the training-set.
    acc = history.history['categorical_accuracy']
    loss = history.history['loss']

    # Get it for the validation-set (we only use the test-set).
    val_acc = history.history['val_categorical_accuracy']
    val_loss = history.history['val_loss']

    # Plot the accuracy and loss-values for the training-set.
    plt.plot(acc, linestyle='-', color='b', label='Training Acc.')
    plt.plot(loss, 'o', color='b', label='Training Loss')

    # Plot it for the test-set.
    plt.plot(val_acc, linestyle='--', color='r', label='Test Acc.')
    plt.plot(val_loss, 'o', color='r', label='Test Loss')

    # Plot title and legend.
    plt.title('Training and Test Accuracy')
    plt.legend()

    # Ensure the plot shows correctly.
    plt.show()


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Helper-function for printing whether a layer in the VGG16 model should be trained.
Can you write Python code for it?

def print_layer_trainable():
    for layer in conv_model.layers:
        print("{0}:\t{1}".format(layer.trainable, layer.name))


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
In Transfer Learning we are initially only interested in reusing the pre-trained VGG16 model as it is, so we will disable training for all its layers.
Can you write Python code for it?

conv_model.trainable = False

for layer in conv_model.layers:
    layer.trainable = False

print_layer_trainable()


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Input Pipeline
The Keras API has its own way of creating the input pipeline for training a model using files.
First we need to know the shape of the tensors expected as input by the pre-trained VGG16 model. In this case it is images of shape 224 x 224 x 3.
Can you write Python code for it?

input_shape = model.layers[0].output_shape[1:3]
input_shape


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Keras uses a so-called data-generator for inputting data into the neural network, which will loop over the data for eternity.
We have a small training-set so it helps to artificially inflate its size by making various transformations to the images. We use a built-in data-generator that can make these random transformations. This is also called an augmented dataset.
Can you write Python code for it?

datagen_train = ImageDataGenerator(
      rescale=1./255,
      rotation_range=180,
      width_shift_range=0.1,
      height_shift_range=0.1,
      shear_range=0.1,
      zoom_range=[0.9, 1.5],
      horizontal_flip=True,
      vertical_flip=True,
      fill_mode='nearest')


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Note how the weight is about 1.398 for the forky-class and only 0.707 for the spoony-class. This is because there are fewer images for the forky-class so the gradient should be amplified for those images, while the gradient should be lowered for spoony-images.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Now we create the actual data-generator that will read files from disk, resize the images and return a random batch.
It is somewhat awkward that the construction of the data-generator is split into these two steps, but it is probably because there are different kinds of data-generators available for different data-types (images, text, etc.) and sources (memory or disk).
Can you write Python code for it?

generator_train = datagen_train.flow_from_directory(directory=train_dir,
                                                    target_size=input_shape,
                                                    batch_size=batch_size,
                                                    shuffle=True,
                                                    save_to_dir=save_to_dir)


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Once we have changed whether the model's layers are trainable, we need to compile the model for the changes to take effect.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Plot a few images to see if data is correct
Can you write Python code for it?

# Load the first images from the train-set.
images = load_images(image_paths=image_paths_train[0:9])

# Get the true classes for those images.
cls_true = cls_train[0:9]

# Plot the images and labels using our helper-function above.
plot_images(images=images, cls_true=cls_true, smooth=True)


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Pre-Trained Model: VGG16
The following creates an instance of the pre-trained VGG16 model using the Keras API. This automatically downloads the required files if you don't have them already. Note how simple this is in Keras compared to Tutorial #08.
The VGG16 model contains a convolutional part and a fully-connected (or dense) part which is used for classification. If include_top=True then the whole VGG16 model is downloaded which is about 528 MB. If include_top=False then only the convolutional part of the VGG16 model is downloaded which is just 57 MB.
We will try and use the pre-trained model for predicting the class of some images in our new dataset, so we have to download the full model, but if you have a slow internet connection, then you can modify the code below to use the smaller pre-trained model without the classification layers.
Can you write Python code for it?

model = VGG16(include_top=True, weights='imagenet')


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
The data-generator for the test-set should not transform and shuffle the images.
Can you write Python code for it?

generator_test = datagen_test.flow_from_directory(directory=test_dir,
                                                  target_size=input_shape,
                                                  batch_size=batch_size,
                                                  shuffle=False)


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
The data-generators will return batches of images. Because the VGG16 model is so large, the batch-size cannot be too large, otherwise you will run out of RAM on the GPU.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
The directories where the images are now stored.
Can you write Python code for it?

train_dir = knifey.train_dir
test_dir = knifey.test_dir


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
The only performance metric we are interested in is the classification accuracy.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
The training can then be continued so as to fine-tune the VGG16 model along with the new classifier.
Can you write Python code for it?

history = new_model.fit(x=generator_train,
                        epochs=epochs,
                        steps_per_epoch=steps_per_epoch,
                        class_weight=class_weight,
                        validation_data=generator_test,
                        validation_steps=steps_test)


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
This dataset has another directory structure than the Keras API requires, so copy the files into separate directories for the training- and test-sets.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
This was developed using Python 3.6 and TensorFlow version:
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Transfer Learning
The pre-trained VGG16 model was unable to classify images from the Knifey-Spoony dataset. The reason is perhaps that the VGG16 model was trained on the so-called ImageNet dataset which may not have contained many images of cutlery.
The lower layers of a Convolutional Neural Network can recognize many different shapes or features in an image. It is the last few fully-connected layers that combine these featuers into classification of a whole image. So we can try and re-route the output of the last convolutional layer of the VGG16 model to a new fully-connected neural network that we create for doing classification on the Knifey-Spoony dataset.
First we print a summary of the VGG16 model so we can see the names and types of its layers, as well as the shapes of the tensors flowing between the layers. This is one of the major reasons we are using the VGG16 model in this tutorial, because the Inception v3 model has so many layers that it is confusing when printed out.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
Using the Keras API it is very simple to create a new model. First we take the part of the VGG16 model from its input-layer to the output of the transfer-layer. We may call this the convolutional model, because it consists of all the convolutional layers from the VGG16 model.
Can you write Python code for it?

conv_model = Model(inputs=model.input,
                   outputs=transfer_layer.output)


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We also need a data-generator for the test-set, but this should not do any transformations to the images because we want to know the exact classification accuracy on those specific images. So we just rescale the pixel-values so they are between 0.0 and 1.0 because this is expected by the VGG16 model.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We can also try an image from our new test-set, and again the VGG16 model is very confused.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We can check that this has updated the trainable boolean for the relevant layers.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We can plot some examples of mis-classified images from the test-set. Some of these images are also difficult for a human to classify.
The confusion matrix shows that the new model is especially having problems classifying the forky-class.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We can save the randomly transformed images during training, so as to inspect whether they have been overly distorted, so we have to adjust the parameters for the data-generator above.
Can you write Python code for it?

if True:
    save_to_dir = None
else:
    save_to_dir='augmented_images/'


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We can see that the last convolutional layer is called 'block5_pool' so we use Keras to get a reference to that layer.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We can then plot the loss-values and classification accuracy from the training. Depending on the dataset, the original model, the new classifier, and hyper-parameters such as the learning-rate, this may improve the classification accuracies on both training- and test-set, or it may improve on the training-set but worsen it for the test-set in case of overfitting. It may require some experimentation with the parameters to get this right.
Can you write Python code for it?

plot_training_history(history)

result = new_model.evaluate_generator(generator_test, steps=steps_test)

print("Test-set classification accuracy: {0:.2%}".format(result[1]))


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We can then use Keras to build a new model on top of this.
Can you write Python code for it?

# Start a new Keras Sequential model.
new_model = Sequential()

# Add the convolutional part of the VGG16 model from above.
new_model.add(conv_model)

# Flatten the output of the VGG16 model because it is from a
# convolutional layer.
new_model.add(Flatten())

# Add a dense (aka. fully-connected) layer.
# This is for combining features that the VGG16 model has
# recognized in the image.
new_model.add(Dense(1024, activation='relu'))

# Add a dropout-layer which may prevent overfitting and
# improve generalization ability to unseen data e.g. the test-set.
new_model.add(Dropout(0.5))

# Add the final layer for the actual classification.
new_model.add(Dense(num_classes, activation='softmax'))


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We can then use the VGG16 model on a picture of a parrot which is classified as a macaw (a parrot species) with a fairly high score of 79%.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We can then use the VGG16 model to predict the class of one of the images in our new training-set. The VGG16 model is very confused about this image and cannot make a good classification.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We can try it for another image in our new training-set and the VGG16 model is still confused.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We have 3 classes in the Knifey-Spoony dataset so Keras needs to use this loss-function.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We refer to this layer as the Transfer Layer because its output will be re-routed to our new fully-connected neural network which will do the classification for the Knifey-Spoony dataset.
The output of the transfer layer has the following shape:
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We use the Adam optimizer with a fairly low learning-rate. The learning-rate could perhaps be larger. But if you try and train more layers of the original VGG16 model, then the learning-rate should be quite low otherwise the pre-trained weights of the VGG16 model will be distorted and it will be unable to learn.
Can you write Python code for it?


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We want to train the last two convolutional layers whose names contain 'block5' or 'block4'.
Can you write Python code for it?

for layer in conv_model.layers:
    # Boolean whether this layer is trainable.
    trainable = ('block5' in layer.name or 'block4' in layer.name)

    # Set the layer's bool.
    layer.trainable = trainable


I am working on the file "10_Fine-Tuning.ipynb".
The first task is:
We will use a lower learning-rate for the fine-tuning so the weights of the original VGG16 model only get changed slowly.
Can you write Python code for it?


I am working on the file "10_Introduction_to_strain_analysis.ipynb".
The first task is:
Functions may also receive arguments (variables passed from the caller to the function). For example:
Can you write Python code for it?

def my_function_with_args(username, greeting):
    print("Hello {}, I wish you {}".format(username, greeting))


I am working on the file "10_Introduction_to_strain_analysis.ipynb".
The first task is:
Functions may return a value to the caller, using the keyword- 'return' . For example:
Can you write Python code for it?

def sum_two_numbers(a, b):
    return a + b


I am working on the file "10_Introduction_to_strain_analysis.ipynb".
The first task is:
Functions to visualize strain ellipse and displacement field
These functions are provided in sg2lib.py library in repository, so later you can import them as from sg2lib import *
Can you write Python code for it?

from pylab import *
from scipy import linalg as la

def def_ellipse(F):
    # Draw strain ellipse from deformation gradient
    theta = linspace(0, 2*pi, 180)
    xc, yc = cos(theta), sin(theta)
    x,y = dot(F, [xc,yc])
    plot(xc, yc, 'r', x, y, 'g')
    u, s, v = svd(F)
    plot(x, y, 'k', lw=2)
    quiver(zeros(2), zeros(2),
               hstack((s*u[0],-s*u[0])), hstack((s*u[1],-s*u[1])),
               scale=1, units='xy')
    axis('equal')

def dis_ellipse(J):
    # Draw strain ellipse from displacement gradient
    J = asarray(J)
    F = J + eye(2)
    def_ellipse(F)

def dis_field(J):
    # Visualize displacement field from
    # displacement gradient
    X, Y = meshgrid(linspace(-3, 3, 21),
                    linspace(-2, 2, 17))
    u, v = tensordot(J, [X, Y], axes=1)
    quiver(X, Y, u, v, angles='xy')
    axis('equal')

def def_field(F):
    # Visualize displacement field from
    # deformation gradient
    F = asarray(F)
    J = F - eye(2)
    dis_field(J)

def dis_show(J):
    # Draw displacement field and deformation ellipse
    # from displacement gradient
    dis_field(J)
    dis_ellipse(J)
    show()

def def_show(F):
    # Draw displacement field and deformation ellipse
    # from deformation gradient
    def_field(F)
    def_ellipse(F)
    show()


I am working on the file "10_Introduction_to_strain_analysis.ipynb".
The first task is:
Lets try to visualize how unit circle deforms during homogeneous deformation:
Can you write Python code for it?

# parametric definition of unit circle
theta = linspace(0, 2*pi, 300)
Xc, Yc = cos(theta), sin(theta)
plot(Xc, Yc, 'g')

# Apply deformation gradient and plot ellipse
F = array([[2, 0], [0, 0.5]])
xe, ye = dot(F, [Xc, Yc])
plot(xe, ye, 'r')
axis('equal');


I am working on the file "10_Introduction_to_strain_analysis.ipynb".
The first task is:
To use defined function we simply define deformation or displacement gradient and call appropriate functions:
Can you write Python code for it?

# Deformation gradient
F = array([[0.5, 1],
           [0  , 2]])
def_field(F)
def_ellipse(F)

# Displacement gradient
J = array([[1,    1],
           [0, -0.5]])
dis_field(J)
dis_ellipse(J)

from IPython.core.display import HTML
def css_styling():
    styles = open("./css/sg2.css", "r").read()
    return HTML(styles)
css_styling()


I am working on the file "10_Introduction_to_strain_analysis.ipynb".
The first task is:
To visualize deformation gradient $\mathbf{F}$ we have to calculate displacement for points on regular grid and plot it using command quiver.
Can you write Python code for it?

# create rectangular grid
X, Y = meshgrid(linspace(-2.2, 2.2, 21), linspace(-1.9, 1.9, 17))

# Apply deformation gradient. As [X, Y] is 3D array, we need
# special function to treat dot product properly. tensordot
# can do it.
x, y = tensordot(F, [X, Y], axes=1)

# plot displacement vectors on all point
quiver(X, Y, x-X, y-Y, angles='xy')
plot(Xc, Yc, 'g', xe, ye, 'r')
plot(Xs, Ys, 'g', xn, yn, 'r')
axis('equal');


I am working on the file "10_Introduction_to_strain_analysis.ipynb".
The first task is:
Using functions in Python
What are Functions?
Functions are a convenient way to divide your code into useful blocks, allowing us to order our code, make it more readable, reuse it and save some time. Also functions are a key way to define interfaces so programmers can share their code.
How do you write functions in Python?
As we have seen already, Python makes use of blocks. A block is a area of code of written in the format of:
block_head:
    1st block line
    2nd block line
    ...
Where a block line is more Python code (even another block), and the block head is of the following format: block_keyword block_name(argument1,argument2, ...) Block keywords you already know are "if", "for", and "while".
Functions in python are defined using the block keyword "def", followed with the function's name as the block's name. For example:
Can you write Python code for it?

def my_function():
    print("Hello from my_function!")


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Axial loads
Can you write Python code for it?


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Compute axial loads in Stringers in S1
We first define some symbols:
Can you write Python code for it?

Tx, Ty, Nz, Mx, My, Mz, F, ry, ry, mz = sympy.symbols('T_x T_y N_z M_x M_y M_z F r_y r_x m_z')


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Compute axial loads in stringers and shear flows in panels
Can you write Python code for it?


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Compute tip displacement and rotation
Can you write Python code for it?

L = np.array([L1, L2, L3, L4, L5])
Mx = 1000.
Jxx = np.array([Ixx1, Ixx2, Ixx3, Ixx4, Ixx5])

θ = Mx*L/(E*Jxx)
sum(θ)

Lc = np.zeros(6)
Lc[1:] = L

Lt = np.cumsum(Lc)

δ = [Mx*(Lt[i]-Lt[i-1])/(E*Jxx[i-1])*((Lt[i]+Lt[i-1])/2-Lt[-1]) for i in range(1,len(Lt))]

sum(δ)


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Compute torsion angle
Can you write Python code for it?


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Define section and perform first calculations
Can you write Python code for it?

S4 = Section(stringers, panels)

S4.cycles


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Define sympy parameters used for geometric description of sections
Can you write Python code for it?

Ams, Afs, Amid, Atip, tms, tfs, tpt, tpn, hms, hfs, xfs, xtip, L = \
sympy.symbols('A_ms A_fs A_mid A_tip t_ms t_fs t_pn t_pt h_ms h_fs x_sp x_tip L', positive=True)


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Example 2: twisting moment in z direction
Can you write Python code for it?

S1.set_loads(_Tx=0, _Ty=0, _Nz=0, _Mx=0, _My=0, _Mz=Mz)
S1.compute_stringer_actions()
S1.compute_panel_fluxes();


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Expression of Inertial properties wrt Center of Gravity in with original rotation
Can you write Python code for it?


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Expression of inertial properties in principal reference frame
Can you write Python code for it?

S4.Ixx, S4.Iyy, S4.Ixy, S4.θ

Ixx4 = S4.Ixx.subs(datav)

S4.set_loads(_Tx=0, _Ty=0, _Nz=0, _Mx=0, _My=0, _Mz=0)

S4.compute_Jt()
S4.Jt

Jt4 = S4.Jt[0]
Jt4


S1.Ixx, S1.Iyy, S1.Ixy, S1.θ

Ixx1 = S1.Ixx.subs(datav)


S2.Ixx, S2.Iyy, S2.Ixy, S2.θ

Ixx2 = S2.Ixx.subs(datav)

S2.set_loads(_Tx=0, _Ty=0, _Nz=0, _Mx=0, _My=0, _Mz=0)

S2.compute_Jt()
S2.Jt

Jt2 = S2.Jt[0]
Jt2


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Import Section class, which contains all calculations
Can you write Python code for it?


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Initialization of sympy symbolic tool and pint for dimension analysis (not really implemented rn as not directly compatible with sympy)
Can you write Python code for it?

ureg = UnitRegistry()
sympy.init_printing()


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Panel fluxes
Can you write Python code for it?


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Plot of S1 section in inertial reference Frame
Section is plotted wrt center of gravity and rotated (if necessary) so that x and y are principal axes.
Center of Gravity and Shear Center are drawn
Can you write Python code for it?

positions={ii: [float(S1.g.node[ii]['pos'][i].subs(datav)) for i in range(2)] for ii in S1.g.nodes() }

x_ct, y_ct = S1.ct.subs(datav)

plt.figure(figsize=(12,8),dpi=300)
nx.draw(S1.g,with_labels=True, pos=positions)
plt.plot([0],[0],'o',ms=12,label='CG')
plt.plot([x_ct],[y_ct],'^',ms=12, label='SC')
#plt.text(0,0, 'CG', fontsize=24)
#plt.text(x_ct,y_ct, 'SC', fontsize=24)
plt.legend(loc='lower right', shadow=True)
plt.axis('equal')
plt.title("Sezione 1",fontsize=16);
plt.savefig("sect1.jpg")


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Plot of S2 section in inertial reference Frame
Section is plotted wrt center of gravity and rotated (if necessary) so that x and y are principal axes.
Center of Gravity and Shear Center are drawn
Can you write Python code for it?

positions={ii: [float(S2.g.node[ii]['pos'][i].subs(datav)) for i in range(2)] for ii in S2.g.nodes() }

x_ct, y_ct = S2.ct.subs(datav)

plt.figure(figsize=(12,8),dpi=300)
nx.draw(S2.g,with_labels=True, pos=positions)
plt.plot([0],[0],'o',ms=12,label='CG')
plt.plot([x_ct],[y_ct],'^',ms=12, label='SC')
#plt.text(0,0, 'CG', fontsize=24)
#plt.text(x_ct,y_ct, 'SC', fontsize=24)
plt.legend(loc='lower right', shadow=True)
plt.axis('equal')
plt.title("Sezione 2",fontsize=16);
plt.savefig("sect2.jpg")


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Plot of S2 section in original reference frame
Define a dictionary of coordinates used by Networkx to plot section as a Directed graph.
Note that arrows are actually just thicker stubs
Can you write Python code for it?

start_pos={ii: [float(S2.g.node[ii]['ip'][i].subs(datav)) for i in range(2)] for ii in S2.g.nodes() }

plt.figure(figsize=(12,8),dpi=300)
nx.draw(S2.g,with_labels=True, arrows= True, pos=start_pos)
plt.arrow(0,0,20,0)
plt.arrow(0,0,0,20)
#plt.text(0,0, 'CG', fontsize=24)
plt.axis('equal')
plt.title("Section in starting reference Frame",fontsize=16);


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Plot of S3 section in original reference frame
Can you write Python code for it?

plt.figure(figsize=(12,8),dpi=300)
nx.draw(S3.g,with_labels=True, arrows= True, pos=start_pos)
plt.arrow(0,0,20,0)
plt.arrow(0,0,0,20)
#plt.text(0,0, 'CG', fontsize=24)
plt.axis('equal')
plt.title("Section in starting reference Frame",fontsize=16);


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Plot of S4 section in inertial reference Frame
Section is plotted wrt center of gravity and rotated (if necessary) so that x and y are principal axes.
Center of Gravity and Shear Center are drawn
Can you write Python code for it?

positions={ii: [float(S4.g.node[ii]['pos'][i].subs(datav)) for i in range(2)] for ii in S4.g.nodes() }

x_ct, y_ct = S4.ct.subs(datav)

plt.figure(figsize=(12,8),dpi=300)
nx.draw(S4.g,with_labels=True, pos=positions)
plt.plot([0],[0],'o',ms=12,label='CG')
plt.plot([x_ct],[y_ct],'^',ms=12, label='SC')
#plt.text(0,0, 'CG', fontsize=24)
#plt.text(x_ct,y_ct, 'SC', fontsize=24)
plt.legend(loc='lower right', shadow=True)
plt.axis('equal')
plt.title("Sezione 4",fontsize=16);
plt.savefig("sect4.jpg")


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Section 2
Can you write Python code for it?

L2 = 410. - L1

xtip2 = -651.8 + np.arctan(tg_β)*(L1+L2/2.)

# altezza longheroni
hms2 = 42.
hfs2 = 46.
#pos longherone anteriore
xfs2 = -345.
# spessore pannello
tpt2 = 0.81
tpn2 = 0.64

# lunghezza pannelli 2
lpt2 = np.sqrt((hms2-hfs2)**2+xfs1**2)
lpn2 = np.sqrt(hfs2**2+(xfs2-xtip2)**2)

# area pannelli 2
Apt2 =lpt2*tpt2
Apn2 =lpn2*tpn2

#spessore longheroni
tms2 = 1.6
tfs2 = 1.471

# area correnti
Astr_ms2=2.28760E+02
Astr_fs2=1.471*25.

# AREE
Ams2 = hms2*tms2/6+Apt2/4+Astr_ms2
Afs2 = hfs2*tfs2/6+Apt2/4+Astr_fs2+Apn2/2
Amid2 = Apt2/2
Atip2 = Apn2

values = [(Ams, Ams2 * ureg.millimeter**2),(Afs, Afs2  * ureg.millimeter**2), \
          (Amid, Amid2 * ureg.millimeter**2),(Atip, Atip2 * ureg.millimeter), \
          (tms, tms2 * ureg.millimeter),(tfs, tfs2 * ureg.millimeter), \
          (tpt, tpt2 * ureg.millimeter), (tpn, tpn2 * ureg.millimeter), \
          (hms, hms2 * ureg.millimeter), (hfs, hfs2 * ureg.millimeter), \
          (xfs, xfs2 * ureg.millimeter), (xtip, xtip2 * ureg.millimeter), \
          (L,  L2* ureg.millimeter) ]
datav = [(v[0],v[1].magnitude) for v in values]

stringers = {1:[(sympy.Integer(0),sympy.Float(hms2)),sympy.Float(Ams2)],
             2:[(sympy.Float(xfs2/2),sympy.Float((hms2+hfs2)/2)),sympy.Float(Amid2)],
             3:[(sympy.Float(xfs2),sympy.Float(hfs2)),sympy.Float(Afs2)],
             4:[(sympy.Float(xtip2),sympy.Integer(0)),sympy.Float(Atip2)],
             5:[(sympy.Float(xfs2),sympy.Float(-hfs2)),sympy.Float(Afs2)],
             6:[(sympy.Float(xfs2/2),sympy.Float(-(hms2+hfs2)/2)),sympy.Float(Amid2)],
             7:[(sympy.Integer(0),sympy.Float(-hms2)),sympy.Float(Ams2)]}

panels = {(1,2):sympy.Float(tpt2),
          (2,3):sympy.Float(tpt2),
          (3,4):sympy.Float(tpn2),
          (4,5):sympy.Float(tpn2),
          (5,6):sympy.Float(tpt2),
          (6,7):sympy.Float(tpt2),
          (3,5):sympy.Float(tfs2),
          (7,1):sympy.Float(tms2)}


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Section 4
Can you write Python code for it?

L4 = 330.

xtip4 = -651.8 + np.arctan(tg_β)*(740.+L4/2.)

# altezza longheroni
hms4 = 32.25
hfs4 = 26.8
#pos longherone anteriore
xfs4 = -345.+ np.arctan(tg_β)*(L4/2.)
# spessore pannello
tpt4 = 0.81
tpn4 = 0.64

# lunghezza pannelli 2
lpt4 = np.sqrt((hms4-hfs4)**2+xfs4**2)
lpn4 = np.sqrt(hfs4**2+(xfs4-xtip4)**2)

# area pannelli 2
Apt4 =lpt4*tpt4
Apn4 =lpn4*tpn4

#spessore longheroni
tms4 = 1.6
tfs4 = 0.

# area correnti
Astr_ms4=2.28760E+02
Astr_fs4=7.74400E+01

# AREE
Ams4 = hms4*tms4/6+Apt4/4+Astr_ms4
Afs4 = hfs4*tfs4/6+Apt4/4+Astr_fs4+Apn4/2
Amid4 = Apt4/2
Atip4 = Apn4

values = [(Ams, Ams4 * ureg.millimeter**2),(Afs, Afs4  * ureg.millimeter**2), \
          (Amid, Amid4 * ureg.millimeter**2),(Atip, Atip4 * ureg.millimeter), \
          (tms, tms4 * ureg.millimeter),(tfs, tfs4 * ureg.millimeter), \
          (tpt, tpt4 * ureg.millimeter), (tpn, tpn4 * ureg.millimeter), \
          (hms, hms4 * ureg.millimeter), (hfs, hfs4 * ureg.millimeter), \
          (xfs, xfs4 * ureg.millimeter), (xtip, xtip4 * ureg.millimeter), \
          (L,  L4* ureg.millimeter) ]
datav = [(v[0],v[1].magnitude) for v in values]

stringers = {1:[(sympy.Integer(0),sympy.Float(hms4)),sympy.Float(Ams4)],
             2:[(sympy.Float(xfs4/2),sympy.Float((hms4+hfs4)/2)),sympy.Float(Amid4)],
             3:[(sympy.Float(xfs4),sympy.Float(hfs4)),sympy.Float(Afs4)],
             4:[(sympy.Float(xtip4),sympy.Integer(0)),sympy.Float(Atip4)],
             5:[(sympy.Float(xfs4),sympy.Float(-hfs4)),sympy.Float(Afs4)],
             6:[(sympy.Float(xfs4/2),sympy.Float(-(hms4+hfs4)/2)),sympy.Float(Amid4)],
             7:[(sympy.Integer(0),sympy.Float(-hms4)),sympy.Float(Ams4)]}

panels = {(1,2):sympy.Float(tpt4),
          (2,3):sympy.Float(tpt4),
          (3,4):sympy.Float(tpn4),
          (4,5):sympy.Float(tpn4),
          (5,6):sympy.Float(tpt4),
          (6,7):sympy.Float(tpt4),
          (7,1):sympy.Float(tms4)}


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Shear flows
Can you write Python code for it?


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
Verify that we find a simply closed section
Can you write Python code for it?


I am working on the file "10_Stab_SF260.ipynb".
The first task is:
stringers = {1:[(sympy.Integer(0),hms),Ams],
             2:[(xfs/2,(hms+hfs)/2),Amid],
             3:[(xfs,hfs),Afs],
             4:[(xtip,sympy.Integer(0)),Atip],
             5:[(xfs,-hfs2),Afs],
             6:[(xfs/2,-(hms+hfs)/2),Amid],
             7:[(sympy.Integer(0),-hms),Ams]}
panels = {(1,2):tpt,
          (2,3):tpt,
          (3,4):tpn,
          (4,5):tpn,
          (5,6):tpt,
          (6,7):tpt,
          (3,5):tfs,
          (7,1):tms}
Define section and perform first calculations
Can you write Python code for it?

S2 = Section(stringers, panels)

S2.cycles


I am working on the file "10_Trigonometric_Interpolation_And_The_FFT.ipynb".
The first task is:
10.1 The Fourier Transform
Lemma : Primitive roots pf unity
Let $\omega$ be a primitive bth root of unity and k be an integer. Then
$\sum_{j=0}^{n-1}\omega^{jk} =
\left{\begin{matrix}
n & \text{if k/n is an integer } \
0 & \text{otherwise}
\end{matrix}\right.$
Definition
The Discrete Fourier Transform (DFT) of $x = [x_0,\cdots,x_{n-1}]^T$ is the n-dimensional vector $y = [y_0,\cdots,y_{n-1}]$, where $\omega = e^{-i2\pi/n}$ and
$y_k = \frac{1}{\sqrt{n}}\sum_{j=0}^{n-1}x_j\omega^{jk}$
Definition
The magnitude of a complex vector $v$ is the real number $||v|| = \sqrt{\bar{v}^Tv}$. A square complex matrix F is unitary if $\bar{F}^TF=I$
Example
Find the DFT of the vector $x = [1, 0, -1, 0]^T$
Can you write Python code for it?

np.set_printoptions(precision=3)
x = np.array([1, 0, -1, 0]).T
w = complex(math.cos(math.pi * 2 / x.size), -math.sin(math.pi * 2 / x.size))
F = np.empty((x.size, x.size), dtype=complex) # Fourier matrix
for i in range(x.size):
    for j in range(x.size):
        F[i, j] = pow(w,(i * j))
y = (1 / math.sqrt(x.size)) * np.matmul(F, x)
print(np.round(y)) # [0, 1, 0, 1]

# Or use numpy fft (fast fourier transform)
print(np.fft.fft(x) / np.sqrt(x.size))


I am working on the file "10_Trigonometric_Interpolation_And_The_FFT.ipynb".
The first task is:
10.3 The FFT And Signal Processing
Orthogonal Function Interpolation Theorem
Let f_0(t),\cdots,\f_{n-1}(t) be functions of t and $t_0,\cdots,t_{n-1}$ be real numbers. Assume that the $n \times n$ matrix
$A =
\begin{bmatrix}
f_0(t_0) & f_0(t_1) & \cdots & f_0(t_{n-1})  \
f_1(t_0) & f_1(t_1) & \cdots & f_1(t_{n-1})  \
\vdots & \vdots &  & \vdots \
f_{n-1}(t_0) & f_{n-1}(t_1) & \cdots & f_{n-1}(t_{n-1})
\end{bmatrix} $
is a real $n \times n$ otrhogonal matrix. If $y = Ax$, the function
$F(t) = \sum_{k = 0}^{n-1}y_kf_k(t)$
interpolates $(t_0, x_0), \cdots, (t_{n-1}, x_{n-1})$, that is $F(t_j) = x_j$ for $j = 0,\cdots,n-1$
Lemma
Let $n \ge 1$ and k, l be integers. Then
$\sum_{j = 0}^{n-1}\cos{\frac{2\pi jk}{n}}\cos{\frac{2\pi jl}{n}} =  \left{\begin{matrix}
n & \text{if both (k-l)/n and (k+l)/n are integers} \
\frac{n}{2} & \text{if exactly one of (k-l)/n and (k+l)/n is an integer} \
0 & \text{if neither is an integer}
\end{matrix}\right.$
$\sum_{j = 0}^{n-1}\cos{\frac{2\pi jk}{n}}\sin{\frac{2\pi jl}{n}} = 0$
$\sum_{j = 0}^{n-1}\sin{\frac{2\pi jk}{n}}\sin{\frac{2\pi jl}{n}} =  \left{\begin{matrix}
0 & \text{if both (k-l)/n and (k+l)/n are integers} \
\frac{n}{2} & \text{if (k-l)/n is an integer and (k+l)/n is not} \
-\frac{n}{2} & \text{if (k-l)/n is not and (k+l)/n is an integer} \
0 & \text{if neither is an integer}
\end{matrix}\right.$
Orthogonal Function Least Squares Approximation Theorem
Let $m \le n$ be integers, and assume that data $(t_0,x_0),\cdots,(t_{n-1},x_{n-1})$ are given. Set $y = Ax$, where A is an orthogonal matrix of form like Orthogonal Function Interpolation Theorem show. Then the interpolating polynomial for basis functions $f_0(t),\cdots,f_{n-1}(t)$ is
$F_n(t) = \sum_{k = 0}^{n - 1}y_kf_k(t)$
and the best least squares approximation, using only the functions $f_0,\cdots,f_{m-1}$, is
$F_m(t) = \sum_{k = 0}^{m - 1}y_kf_k(t)$
Corollary
Let $[c,d]$ be an interval, let $m < n$ be even positive integers, $x = (x_0,\cdots,x_{n-1})$ a vector of n real numbers, and let $t_j = c + j(d - c)/n$...
Can you write Python code for it?

def least_squares_even_trigonmetric(m, t, a, b, c, d):
     return a[0] / np.sqrt(a.size) + 2 / np.sqrt(a.size) * \
        np.sum([a[k] * np.cos(2 * k * np.pi * (t - c) / (d - c)) - \
        b[k] * np.sin(2 * k * np.pi * (t - c) / (d - c)) for k in range(1, int(m / 2))]) + \
        a[int(m / 2)] / np.sqrt(a.size) * np.cos(a.size * np.pi * (t - c) / (d - c))


I am working on the file "10_Trigonometric_Interpolation_And_The_FFT.ipynb".
The first task is:
Example
Find the trigonometric interpolant for the temperature data : $x = [-2.2, -2.8, -6.1, -3.9, 0.0, 1.1, -0.6, -1.1]$ on the interval $[0,1]$
Can you write Python code for it?

c, d = 0, 1
xt = np.linspace(c, d, 8, False)
x = np.array([-2.2, -2.8, -6.1, -3.9, 0, 1.1, -0.6, -1.1])
y = np.fft.fft(x) / np.sqrt(x.size)
a = np.array(y.real)
b = np.array(y.imag)
t = np.linspace(c, d, 40)
data = [even_trigonometric_interpolant(i, a, b, c, d) for i in t]
plt.plot(t, data)
plt.plot(xt, x, 'o', color='k')
plt.show()


I am working on the file "10_Trigonometric_Interpolation_And_The_FFT.ipynb".
The first task is:
Example
Fit the temperature data by least squares trigonometric functions of orders 4 and 6 where $x = [-2.2, -2.8, -6.1, -3.9, 0.0, 1.1, -0.6, -1.1]$ on the interval $[0,1]$
Can you write Python code for it?

m1, m2, m3 = 4, 6, 8
c, d = 0, 1
xt = np.linspace(c, d, 8, False)
x = np.array([-2.2, -2.8, -6.1, -3.9, 0, 1.1, -0.6, -1.1])
y = np.fft.fft(x) / np.sqrt(x.size)
a = np.array(y.real)
b = np.array(y.imag)
t = np.linspace(c, d, 40)
data1 = [least_squares_even_trigonmetric(m1, i, a, b, c, d) for i in t]
data2 = [least_squares_even_trigonmetric(m2, i, a, b, c, d) for i in t]
data3 = [least_squares_even_trigonmetric(m3, i, a, b, c, d) for i in t]
plt.xticks(xt, ['t0', 't1', 't2', 't3', 't4', 't5', 't6', 't7'])
plt.plot(t, data1, label='order 4')
plt.plot(t, data2, label='order 6')
plt.plot(t, data3, label='order 8')
plt.plot(xt, x, 'o', color='k')
plt.legend()
plt.grid(True)
plt.show()


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
A mean-squared error (MSE) function to be minimized during calibration.
Can you write Python code for it?

def srd_forward_error(p0):
    global initial_value, f, t
    if p0[0] < 0 or p0[1] < 0 or p0[2] < 0:
        return 100
    f_model = dx.srd_forwards(initial_value, p0, t)
    MSE = np.sum((f - f_model) ** 2) / len(f)
    return MSE


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
And the calibration itself.
Can you write Python code for it?

from scipy.optimize import fmin

opt = fmin(srd_forward_error, (1.0, 0.7, 0.2),
           maxiter=1000, maxfun=1000)


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
And the short end of the Libor term sturcture visualized.
Can you write Python code for it?


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
Floating Rate Modeling
The optimal parameters from the calibration are used to model the floating rate (3m Libor rate).
Can you write Python code for it?

# market environment
me_srd = dx.market_environment('me_srd', dt.datetime(2014, 10, 16))

# square-root diffusion
me_srd.add_constant('initial_value', 0.02)
me_srd.add_constant('kappa', opt[0])
me_srd.add_constant('theta', opt[1])
me_srd.add_constant('volatility', opt[2])
me_srd.add_curve('discount_curve', ois)
  # OIS discounting object
me_srd.add_constant('currency', 'EUR')
me_srd.add_constant('paths', 10000)
me_srd.add_constant('frequency', 'w')
me_srd.add_constant('starting_date', me_srd.pricing_date)
me_srd.add_constant('final_date', dt.datetime(2020, 12, 31))

srd = dx.square_root_diffusion('srd', me_srd)


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
Interest Rate Swap
Finally, we can model the interest rate swap itself.
Modeling
First, the market environment with all the parameters needed.
Can you write Python code for it?

# market environment for the IRS
me_irs = dx.market_environment('irs', me_srd.pricing_date)
me_irs.add_constant('fixed_rate', 0.01)
me_irs.add_constant('trade_date', me_srd.pricing_date)
me_irs.add_constant('effective_date', me_srd.pricing_date)
me_irs.add_constant('payment_date', dt.datetime(2014, 12, 27))
me_irs.add_constant('payment_day', 27)
me_irs.add_constant('termination_date', me_srd.get_constant('final_date'))
me_irs.add_constant('currency', 'EUR')
me_irs.add_constant('notional', 1000000)
me_irs.add_constant('tenor', '6m')
me_irs.add_constant('counting', 'ACT/360')
# discount curve from mar_env of floating rate


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
Let us have a look at some simulated rate paths.
Can you write Python code for it?

paths = srd.get_instrument_values()

plt.figure(figsize=(10, 6))
plt.plot(srd.time_grid, paths[:, :6])


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
Let us have a look at the most current data, i.e. the term structure, of the data set.
Can you write Python code for it?


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
Model Calibration
Next, equipped with the Libor data, we calibrate the square-root diffusion short rate model. A bit of data preparation:
Can you write Python code for it?

t = libf.index.to_pydatetime()
f = libf.iloc[:, -1].values / 100
initial_value = 0.005


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
Next we replace the year fraction index by a DatetimeIndex.
Can you write Python code for it?

# generate time index given input data
# starting date + 59 months
date = oiss.columns[-1]
index = pd.date_range(date, periods=60, freq='M') # , tz='GMT')
index = [d.replace(day=date.day) for d in index]
index = pd.DatetimeIndex(index)
oiss.index = index


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
OIS Data & Discounting
We start by importing OIS term structure data (source: http://www.bankofengland.co.uk) for risk-free discounting. We also adjust the data structure somewhat for our purposes.
Can you write Python code for it?

# UK OIS Spot Rates Yield Curve
oiss = pd.read_excel('data/ukois09.xls', 'oiss')
# use years as index
oiss = oiss.set_index('years')
# del oiss['years']

# only date information for columns, no time
oiss.columns = [d.date() for d in oiss.columns]

oiss.tail()


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
The instantiation of the valuation object.
Can you write Python code for it?

irs = dx.interest_rate_swap('irs', srd, me_irs)


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
The optimal parameters (kappa, theta, sigma) are:
Can you write Python code for it?


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
This data is used to instantiate a deterministic_short_rate model for risk-neutral discounting purposes.
Can you write Python code for it?

# generate deterministic short rate model based on UK OIS curve
ois = dx.deterministic_short_rate('ois', zip(oiss.index, oiss.iloc[:, -1].values / 100))

# example dates and corresponding discount factors
dr = pd.date_range('2015-1', periods=4, freq='6m').to_pydatetime()
ois.get_discount_factors(dr)[::-1]


I am working on the file "10_dx_interest_rate_swaps.ipynb".
The first task is:
You can also generate a full output of all present values per simulation path.
Can you write Python code for it?


I am working on the file "10_embeddings/Skip-Gram_word2vec.ipynb".
The first task is:
Here's a function that returns batches for our network. The idea is that it grabs batch_size words from a words list. Then for each of those words, it gets the target words in the window. I haven't found a way to pass in a random number of target words and get it to work with the architecture, so I make one row per input-target pair. This is a generator function by the way, helps save memory.
Can you write Python code for it?

def get_batches(words, batch_size, window_size=5):
    ''' Create a generator of word batches as a tuple (inputs, targets) '''

    n_batches = len(words)//batch_size

    # only full batches
    words = words[:n_batches*batch_size]

    for idx in range(0, len(words), batch_size):
        x, y = [], []
        batch = words[idx:idx+batch_size]
        for ii in range(len(batch)):
            batch_x = batch[ii]
            batch_y = get_target(batch, ii, window_size)
            y.extend(batch_y)
            x.extend([batch_x]*len(batch_y))
        yield x, y


I am working on the file "10_embeddings/Skip-Gram_word2vec.ipynb".
The first task is:
Making batches
Now that our data is in good shape, we need to get it into the proper form to pass it into our network. With the skip-gram architecture, for each word in the text, we want to grab all the words in a window around that word, with size $C$.
From Mikolov et al.:
"Since the more distant words are usually less related to the current word than those close to it, we give less weight to the distant words by sampling less from those words in our training examples... If we choose $C = 5$, for each training word we will select randomly a number $R$ in range $< 1; C >$, and then use $R$ words from history and $R$ words from the future of the current word as correct labels."

Exercise: Implement a function get_target that receives a list of words, an index, and a window size, then returns a list of words in the window around the index. Make sure to use the algorithm described above, where you choose a random number of words from the window.
Can you write Python code for it?

def get_target(words, idx, window_size=5):
    ''' Get a list of words in a window around an index. '''
    R = np.random.randint(0, window_size) + 1
    left_side = words[max(0, idx - R) : idx]
    right_side = words[idx + 1 : min(idx + R, len(words))]
    return list(set(left_side + right_side))


I am working on the file "10_embeddings/Skip-Gram_word2vec.ipynb".
The first task is:
Restore the trained network if you need to:
Can you write Python code for it?

with train_graph.as_default():
    saver = tf.train.Saver()

with tf.Session(graph=train_graph) as sess:
    saver.restore(sess, tf.train.latest_checkpoint('checkpoints'))
    embed_mat = sess.run(embedding)


I am working on the file "10_embeddings/Skip-Gram_word2vec.ipynb".
The first task is:
Visualizing the word vectors
Below we'll use T-SNE to visualize how our high-dimensional word vectors cluster together. T-SNE is used to project these vectors into two dimensions while preserving local stucture. Check out this post from Christopher Olah to learn more about T-SNE and other ways to visualize high-dimensional data.
Can you write Python code for it?

%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import matplotlib.pyplot as plt
from sklearn.manifold import TSNE

viz_words = 500
tsne = TSNE()
embed_tsne = tsne.fit_transform(embed_mat[:viz_words, :])

fig, ax = plt.subplots(figsize=(14, 14))
for idx in range(viz_words):
    plt.scatter(*embed_tsne[idx, :], color='steelblue')
    plt.annotate(int_to_vocab[idx], (embed_tsne[idx, 0], embed_tsne[idx, 1]), alpha=0.7)


I am working on the file "10_introduction_to_artificial_neural_networks.ipynb".
The first task is:
Activation functions
Can you write Python code for it?

def logit(z):
    return 1 / (1 + np.exp(-z))

def relu(z):
    return np.maximum(0, z)

def derivative(f, z, eps=0.000001):
    return (f(z + eps) - f(z - eps))/(2 * eps)

z = np.linspace(-5, 5, 200)

plt.figure(figsize=(11,4))

plt.subplot(121)
plt.plot(z, np.sign(z), "r-", linewidth=2, label="Step")
plt.plot(z, logit(z), "g--", linewidth=2, label="Logit")
plt.plot(z, np.tanh(z), "b-", linewidth=2, label="Tanh")
plt.plot(z, relu(z), "m-.", linewidth=2, label="ReLU")
plt.grid(True)
plt.legend(loc="center right", fontsize=14)
plt.title("Activation functions", fontsize=14)
plt.axis([-5, 5, -1.2, 1.2])

plt.subplot(122)
plt.plot(z, derivative(np.sign, z), "r-", linewidth=2, label="Step")
plt.plot(0, 0, "ro", markersize=5)
plt.plot(0, 0, "rx", markersize=10)
plt.plot(z, derivative(logit, z), "g--", linewidth=2, label="Logit")
plt.plot(z, derivative(np.tanh, z), "b-", linewidth=2, label="Tanh")
plt.plot(z, derivative(relu, z), "m-.", linewidth=2, label="ReLU")
plt.grid(True)
#plt.legend(loc="center right", fontsize=14)
plt.title("Derivatives", fontsize=14)
plt.axis([-5, 5, -0.2, 1.2])

save_fig("activation_functions_plot")
plt.show()

def heaviside(z):
    return (z >= 0).astype(z.dtype)

def sigmoid(z):
    return 1/(1+np.exp(-z))

def mlp_xor(x1, x2, activation=heaviside):
    return activation(-activation(x1 + x2 - 1.5) + activation(x1 + x2 - 0.5) - 0.5)

x1s = np.linspace(-0.2, 1.2, 100)
x2s = np.linspace(-0.2, 1.2, 100)
x1, x2 = np.meshgrid(x1s, x2s)

z1 = mlp_xor(x1, x2, activation=heaviside)
z2 = mlp_xor(x1, x2, activation=sigmoid)

plt.figure(figsize=(10,4))

plt.subplot(121)
plt.contourf(x1, x2, z1)
plt.plot([0, 1], [0, 1], "gs", markersize=20)
plt.plot([0, 1], [1, 0], "y^", markersize=20)
plt.title("Activation function: heaviside", fontsize=14)
plt.grid(True)

plt.subplot(122)
plt.contourf(x1, x2, z2)
plt.plot([0, 1], [0, 1], "gs", markersize=20)
plt.plot([0, 1], [1, 0], "y^", markersize=20)
plt.title("Activation function: sigmoid", fontsize=14)
plt.grid(True)


I am working on the file "10_introduction_to_artificial_neural_networks.ipynb".
The first task is:
Exercise solutions
1. to 8.
See appendix A.
9.
Train a deep MLP on the MNIST dataset and see if you can get over 98% precision. Just like in the last exercise of chapter 9, try adding all the bells and whistles (i.e., save checkpoints, restore the last checkpoint in case of an interruption, add summaries, plot learning curves using TensorBoard, and so on).
First let's create the deep net. It's exactly the same as earlier, with just one addition: we add a tf.summary.scalar() to track the loss and the accuracy during training, so we can view nice learning curves using TensorBoard.
Can you write Python code for it?

n_inputs = 28*28  # MNIST
n_hidden1 = 300
n_hidden2 = 100
n_outputs = 10

reset_graph()

X = tf.placeholder(tf.float32, shape=(None, n_inputs), name="X")
y = tf.placeholder(tf.int64, shape=(None), name="y")

with tf.name_scope("dnn"):
    hidden1 = tf.layers.dense(X, n_hidden1, name="hidden1",
                              activation=tf.nn.relu)
    hidden2 = tf.layers.dense(hidden1, n_hidden2, name="hidden2",
                              activation=tf.nn.relu)
    logits = tf.layers.dense(hidden2, n_outputs, name="outputs")

with tf.name_scope("loss"):
    xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y, logits=logits)
    loss = tf.reduce_mean(xentropy, name="loss")
    loss_summary = tf.summary.scalar('log_loss', loss)

learning_rate = 0.01

with tf.name_scope("train"):
    optimizer = tf.train.GradientDescentOptimizer(learning_rate)
    training_op = optimizer.minimize(loss)

with tf.name_scope("eval"):
    correct = tf.nn.in_top_k(logits, y, 1)
    accuracy = tf.reduce_mean(tf.cast(correct, tf.float32))
    accuracy_summary = tf.summary.scalar('accuracy', accuracy)

init = tf.global_variables_initializer()
saver = tf.train.Saver()


n_inputs = 28*28  # MNIST
n_hidden1 = 300
n_hidden2 = 100
n_outputs = 10

reset_graph()

X = tf.placeholder(tf.float32, shape=(None, n_inputs), name="X")
y = tf.placeholder(tf.int32, shape=(None), name="y")

with tf.name_scope("dnn"):
    hidden1 = tf.layers.dense(X, n_hidden1, name="hidden1",
                              activation=tf.nn.relu)
    hidden2 = tf.layers.dense(hidden1, n_hidden2, name="hidden2",
                              activation=tf.nn.relu)
    logits = tf.layers.dense(hidden2, n_outputs, name="outputs")

with tf.name_scope("loss"):
    xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y, logits=logits)
    loss = tf.reduce_mean(xentropy, name="loss")
    loss_summary = tf.summary.scalar('log_loss', loss)

learning_rate = 0.01

with tf.name_scope("train"):
    optimizer = tf.train.GradientDescentOptimizer(learning_rate)
    training_op = optimizer.minimize(loss)

with tf.name_scope("eval"):
    correct = tf.nn.in_top_k(logits, y, 1)
    accuracy = tf.reduce_mean(tf.cast(correct, tf.float32))
    accuracy_summary = tf.summary.scalar('accuracy', accuracy)

init = tf.global_variables_initializer()
saver = tf.train.Saver()


I am working on the file "10_introduction_to_artificial_neural_networks.ipynb".
The first task is:
FNN for MNIST
Using the Estimator API (formerly tf.contrib.learn)
Can you write Python code for it?


I am working on the file "10_introduction_to_artificial_neural_networks.ipynb".
The first task is:
FNN for MNIST
using tf.learn
Can you write Python code for it?

from tensorflow.examples.tutorials.mnist import input_data

mnist = input_data.read_data_sets("/tmp/data/")

X_train = mnist.train.images
X_test = mnist.test.images
y_train = mnist.train.labels.astype("int")
y_test = mnist.test.labels.astype("int")

import tensorflow as tf

config = tf.contrib.learn.RunConfig(tf_random_seed=42) # not shown in the config

feature_cols = tf.contrib.learn.infer_real_valued_columns_from_input(X_train)
dnn_clf = tf.contrib.learn.DNNClassifier(hidden_units=[300,100], n_classes=10,
                                         feature_columns=feature_cols, config=config)
dnn_clf = tf.contrib.learn.SKCompat(dnn_clf) # if TensorFlow >= 1.1
dnn_clf.fit(X_train, y_train, batch_size=50, steps=40000)

from sklearn.metrics import accuracy_score

y_pred = dnn_clf.predict(X_test)
accuracy_score(y_test, y_pred['classes'])

from sklearn.metrics import log_loss

y_pred_proba = y_pred['probabilities']
log_loss(y_test, y_pred_proba)


I am working on the file "10_introduction_to_artificial_neural_networks.ipynb".
The first task is:
Hey! Why don't we implement early stopping? For this, we are going to need a validation set. Luckily, the dataset returned by TensorFlow's input_data() function (see above) is already split into a training set (60,000 instances, already shuffled for us), a validation set (5,000 instances) and a test set (5,000 instances). So we can easily define X_valid and y_valid:
Can you write Python code for it?

X_valid = mnist.validation.images
y_valid = mnist.validation.labels

m, n = X_train.shape

n_epochs = 10001
batch_size = 50
n_batches = int(np.ceil(m / batch_size))

checkpoint_path = "/tmp/my_deep_mnist_model.ckpt"
checkpoint_epoch_path = checkpoint_path + ".epoch"
final_model_path = "./my_deep_mnist_model"

best_loss = np.infty
epochs_without_progress = 0
max_epochs_without_progress = 50

with tf.Session() as sess:
    if os.path.isfile(checkpoint_epoch_path):
        # if the checkpoint file exists, restore the model and load the epoch number
        with open(checkpoint_epoch_path, "rb") as f:
            start_epoch = int(f.read())
        print("Training was interrupted. Continuing at epoch", start_epoch)
        saver.restore(sess, checkpoint_path)
    else:
        start_epoch = 0
        sess.run(init)

    for epoch in range(start_epoch, n_epochs):
        for iteration in range(mnist.train.num_examples // batch_size):
            X_batch, y_batch = mnist.train.next_batch(batch_size)
            sess.run(training_op, feed_dict={X: X_batch, y: y_batch})
        accuracy_val, loss_val, accuracy_summary_str, loss_summary_str = sess.run([accuracy, loss, accuracy_summary, loss_summary], feed_dict={X: X_valid, y: y_valid})
        file_writer.add_summary(accuracy_summary_str, epoch)
        file_writer.add_summary(loss_summary_str, epoch)
        if epoch % 5 == 0:
            print("Epoch:", epoch,
                  "\tValidation accuracy: {:.3f}%".format(accuracy_val * 100),
                  "\tLoss: {:.5f}".format(loss_val))
            saver.save(sess, checkpoint_path)
            with open(checkpoint_epoch_path, "wb") as f:
                f.write(b"%d" % (epoch + 1))
            if loss_val < best_loss:
                saver.save(sess, final_model_path)
                best_loss = loss_val
            else:
                epochs_without_progress += 5
                if epochs_without_progress > max_epochs_without_progress:
                    print("Early stopping")
                   ...


I am working on the file "10_introduction_to_artificial_neural_networks.ipynb".
The first task is:
Now we can create the FileWriter that we will use to write the TensorBoard logs:
Can you write Python code for it?

file_writer = tf.summary.FileWriter(logdir, tf.get_default_graph())


I am working on the file "10_introduction_to_artificial_neural_networks.ipynb".
The first task is:
Now we need to define the directory to write the TensorBoard logs to:
Can you write Python code for it?

from datetime import datetime

def log_dir(prefix=""):
    now = datetime.utcnow().strftime("%Y%m%d%H%M%S")
    root_logdir = "tf_logs"
    if prefix:
        prefix += "-"
    name = prefix + "run-" + now
    return "{}/{}/".format(root_logdir, name)

logdir = log_dir("mnist_dnn")


I am working on the file "10_introduction_to_artificial_neural_networks.ipynb".
The first task is:
Perceptrons
Can you write Python code for it?

import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import Perceptron

iris = load_iris()
X = iris.data[:, (2, 3)]  # petal length, petal width
y = (iris.target == 0).astype(np.int)

per_clf = Perceptron(random_state=42)
per_clf.fit(X, y)

y_pred = per_clf.predict([[2, 0.5]])

y_pred

a = -per_clf.coef_[0][0] / per_clf.coef_[0][1]
b = -per_clf.intercept_ / per_clf.coef_[0][1]

axes = [0, 5, 0, 2]

x0, x1 = np.meshgrid(
        np.linspace(axes[0], axes[1], 500).reshape(-1, 1),
        np.linspace(axes[2], axes[3], 200).reshape(-1, 1),
    )
X_new = np.c_[x0.ravel(), x1.ravel()]
y_predict = per_clf.predict(X_new)
zz = y_predict.reshape(x0.shape)

plt.figure(figsize=(10, 4))
plt.plot(X[y==0, 0], X[y==0, 1], "bs", label="Not Iris-Setosa")
plt.plot(X[y==1, 0], X[y==1, 1], "yo", label="Iris-Setosa")

plt.plot([axes[0], axes[1]], [a * axes[0] + b, a * axes[1] + b], "k-", linewidth=3)
from matplotlib.colors import ListedColormap
custom_cmap = ListedColormap(['#9898ff', '#fafab0'])

plt.contourf(x0, x1, zz, cmap=custom_cmap, linewidth=5)
plt.xlabel("Petal length", fontsize=14)
plt.ylabel("Petal width", fontsize=14)
plt.legend(loc="lower right", fontsize=14)
plt.axis(axes)

save_fig("perceptron_iris_plot")
plt.show()


I am working on the file "10_introduction_to_artificial_neural_networks.ipynb".
The first task is:
Perceptrons
Note: we set max_iter and tol explicitly to avoid warnings about the fact that their default value will change in future versions of Scikit-Learn.
Can you write Python code for it?

import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import Perceptron

iris = load_iris()
X = iris.data[:, (2, 3)]  # petal length, petal width
y = (iris.target == 0).astype(np.int)

per_clf = Perceptron(max_iter=100, tol=-np.infty, random_state=42)
per_clf.fit(X, y)

y_pred = per_clf.predict([[2, 0.5]])

y_pred

a = -per_clf.coef_[0][0] / per_clf.coef_[0][1]
b = -per_clf.intercept_ / per_clf.coef_[0][1]

axes = [0, 5, 0, 2]

x0, x1 = np.meshgrid(
        np.linspace(axes[0], axes[1], 500).reshape(-1, 1),
        np.linspace(axes[2], axes[3], 200).reshape(-1, 1),
    )
X_new = np.c_[x0.ravel(), x1.ravel()]
y_predict = per_clf.predict(X_new)
zz = y_predict.reshape(x0.shape)

plt.figure(figsize=(10, 4))
plt.plot(X[y==0, 0], X[y==0, 1], "bs", label="Not Iris-Setosa")
plt.plot(X[y==1, 0], X[y==1, 1], "yo", label="Iris-Setosa")

plt.plot([axes[0], axes[1]], [a * axes[0] + b, a * axes[1] + b], "k-", linewidth=3)
from matplotlib.colors import ListedColormap
custom_cmap = ListedColormap(['#9898ff', '#fafab0'])

plt.contourf(x0, x1, zz, cmap=custom_cmap)
plt.xlabel("Petal length", fontsize=14)
plt.ylabel("Petal width", fontsize=14)
plt.legend(loc="lower right", fontsize=14)
plt.axis(axes)

save_fig("perceptron_iris_plot")
plt.show()


I am working on the file "10_introduction_to_artificial_neural_networks.ipynb".
The first task is:
Using dense() instead of neuron_layer()
Note: previous releases of the book used tensorflow.contrib.layers.fully_connected() rather than tf.layers.dense() (which did not exist when this chapter was written). It is now preferable to use tf.layers.dense(), because anything in the contrib module may change or be deleted without notice. The dense() function is almost identical to the fully_connected() function, except for a few minor differences:
* several parameters are renamed: scope becomes name, activation_fn becomes activation (and similarly the _fn suffix is removed from other parameters such as normalizer_fn), weights_initializer becomes kernel_initializer, etc.
* the default activation is now None rather than tf.nn.relu.
* a few more differences are presented in chapter 11.
Can you write Python code for it?

n_inputs = 28*28  # MNIST
n_hidden1 = 300
n_hidden2 = 100
n_outputs = 10

reset_graph()

X = tf.placeholder(tf.float32, shape=(None, n_inputs), name="X")
y = tf.placeholder(tf.int32, shape=(None), name="y")

with tf.name_scope("dnn"):
    hidden1 = tf.layers.dense(X, n_hidden1, name="hidden1",
                              activation=tf.nn.relu)
    hidden2 = tf.layers.dense(hidden1, n_hidden2, name="hidden2",
                              activation=tf.nn.relu)
    logits = tf.layers.dense(hidden2, n_outputs, name="outputs")
    y_proba = tf.nn.softmax(logits)

with tf.name_scope("loss"):
    xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y, logits=logits)
    loss = tf.reduce_mean(xentropy, name="loss")

learning_rate = 0.01

with tf.name_scope("train"):
    optimizer = tf.train.GradientDescentOptimizer(learning_rate)
    training_op = optimizer.minimize(loss)

with tf.name_scope("eval"):
    correct = tf.nn.in_top_k(logits, y, 1)
    accuracy = tf.reduce_mean(tf.cast(correct, tf.float32))

init = tf.global_variables_initializer()
saver = tf.train.Saver()

n_epochs = 20
n_batches = 50

with tf.Session() as sess:
    init.run()
    for epoch in range(n_epochs):
        for X_batch, y_batch in shuffle_batch(X_train, y_train, batch_size):
            sess.run(training_op, feed_dict={X: X_batch, y: y_batch})
        acc_batch = accuracy.eval(feed_dict={X: X_batch, y: y_batch})
        acc_valid = accuracy.eval(feed_dict={X: X_valid, y: y_valid})
        print(epoch, "Batch accuracy:", acc_batch, "Validation accuracy:", acc_valid)

    save_path = saver.save(sess, "./my_model_final.ckpt")

save_graph()

%tensorboard --logdir {root_logdir}


I am working on the file "10_introduction_to_artificial_neural_networks.ipynb".
The first task is:
Using dense() instead of neuron_layer()
Note: the book uses tensorflow.contrib.layers.fully_connected() rather than tf.layers.dense() (which did not exist when this chapter was written). It is now preferable to use tf.layers.dense(), because anything in the contrib module may change or be deleted without notice. The dense() function is almost identical to the fully_connected() function, except for a few minor differences:
* several parameters are renamed: scope becomes name, activation_fn becomes activation (and similarly the _fn suffix is removed from other parameters such as normalizer_fn), weights_initializer becomes kernel_initializer, etc.
* the default activation is now None rather than tf.nn.relu.
* a few more differences are presented in chapter 11.
Can you write Python code for it?

n_inputs = 28*28  # MNIST
n_hidden1 = 300
n_hidden2 = 100
n_outputs = 10

reset_graph()

X = tf.placeholder(tf.float32, shape=(None, n_inputs), name="X")
y = tf.placeholder(tf.int64, shape=(None), name="y")

with tf.name_scope("dnn"):
    hidden1 = tf.layers.dense(X, n_hidden1, name="hidden1",
                              activation=tf.nn.relu)
    hidden2 = tf.layers.dense(hidden1, n_hidden2, name="hidden2",
                              activation=tf.nn.relu)
    logits = tf.layers.dense(hidden2, n_outputs, name="outputs")

with tf.name_scope("loss"):
    xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y, logits=logits)
    loss = tf.reduce_mean(xentropy, name="loss")

learning_rate = 0.01

with tf.name_scope("train"):
    optimizer = tf.train.GradientDescentOptimizer(learning_rate)
    training_op = optimizer.minimize(loss)

with tf.name_scope("eval"):
    correct = tf.nn.in_top_k(logits, y, 1)
    accuracy = tf.reduce_mean(tf.cast(correct, tf.float32))

init = tf.global_variables_initializer()
saver = tf.train.Saver()

n_epochs = 20
n_batches = 50

with tf.Session() as sess:
    init.run()
    for epoch in range(n_epochs):
        for iteration in range(mnist.train.num_examples // batch_size):
            X_batch, y_batch = mnist.train.next_batch(batch_size)
            sess.run(training_op, feed_dict={X: X_batch, y: y_batch})
        acc_train = accuracy.eval(feed_dict={X: X_batch, y: y_batch})
        acc_test = accuracy.eval(feed_dict={X: mnist.test.images, y: mnist.test.labels})
        print(epoch, "Train accuracy:", acc_train, "Test accuracy:", acc_test)

    save_path = saver.save(sess, "./my_model_final.ckpt")

show_graph(tf.get_default_graph())


I am working on the file "10_introduction_to_artificial_neural_networks.ipynb".
The first task is:
Using plain TensorFlow
Can you write Python code for it?

import tensorflow as tf

n_inputs = 28*28  # MNIST
n_hidden1 = 300
n_hidden2 = 100
n_outputs = 10

reset_graph()

X = tf.placeholder(tf.float32, shape=(None, n_inputs), name="X")
y = tf.placeholder(tf.int64, shape=(None), name="y")

def neuron_layer(X, n_neurons, name, activation=None):
    with tf.name_scope(name):
        n_inputs = int(X.get_shape()[1])
        stddev = 2 / np.sqrt(n_inputs)
        init = tf.truncated_normal((n_inputs, n_neurons), stddev=stddev)
        W = tf.Variable(init, name="kernel")
        b = tf.Variable(tf.zeros([n_neurons]), name="bias")
        Z = tf.matmul(X, W) + b
        if activation is not None:
            return activation(Z)
        else:
            return Z

with tf.name_scope("dnn"):
    hidden1 = neuron_layer(X, n_hidden1, name="hidden1",
                           activation=tf.nn.relu)
    hidden2 = neuron_layer(hidden1, n_hidden2, name="hidden2",
                           activation=tf.nn.relu)
    logits = neuron_layer(hidden2, n_outputs, name="outputs")

with tf.name_scope("loss"):
    xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y,
                                                              logits=logits)
    loss = tf.reduce_mean(xentropy, name="loss")

learning_rate = 0.01

with tf.name_scope("train"):
    optimizer = tf.train.GradientDescentOptimizer(learning_rate)
    training_op = optimizer.minimize(loss)

with tf.name_scope("eval"):
    correct = tf.nn.in_top_k(logits, y, 1)
    accuracy = tf.reduce_mean(tf.cast(correct, tf.float32))

init = tf.global_variables_initializer()
saver = tf.train.Saver()

n_epochs = 40
batch_size = 50

with tf.Session() as sess:
    init.run()
    for epoch in range(n_epochs):
        for iteration in range(mnist.train.num_examples // batch_size):
            X_batch, y_batch = mnist.train.next_batch(batch_size)
            sess.run(training_op, feed_dict={X: X_batch, y: y_batch})
        acc_train = accuracy.eval(feed_dict={X: X_batch, y: y_batch})
        acc_test = accuracy.eval(feed_dict={X:...


import tensorflow as tf

n_inputs = 28*28  # MNIST
n_hidden1 = 300
n_hidden2 = 100
n_outputs = 10

reset_graph()

X = tf.placeholder(tf.float32, shape=(None, n_inputs), name="X")
y = tf.placeholder(tf.int32, shape=(None), name="y")

def neuron_layer(X, n_neurons, name, activation=None):
    with tf.name_scope(name):
        n_inputs = int(X.get_shape()[1])
        stddev = 2 / np.sqrt(n_inputs)
        init = tf.truncated_normal((n_inputs, n_neurons), stddev=stddev)
        W = tf.Variable(init, name="kernel")
        b = tf.Variable(tf.zeros([n_neurons]), name="bias")
        Z = tf.matmul(X, W) + b
        if activation is not None:
            return activation(Z)
        else:
            return Z

with tf.name_scope("dnn"):
    hidden1 = neuron_layer(X, n_hidden1, name="hidden1",
                           activation=tf.nn.relu)
    hidden2 = neuron_layer(hidden1, n_hidden2, name="hidden2",
                           activation=tf.nn.relu)
    logits = neuron_layer(hidden2, n_outputs, name="outputs")

with tf.name_scope("loss"):
    xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y,
                                                              logits=logits)
    loss = tf.reduce_mean(xentropy, name="loss")

learning_rate = 0.01

with tf.name_scope("train"):
    optimizer = tf.train.GradientDescentOptimizer(learning_rate)
    training_op = optimizer.minimize(loss)

with tf.name_scope("eval"):
    correct = tf.nn.in_top_k(logits, y, 1)
    accuracy = tf.reduce_mean(tf.cast(correct, tf.float32))

init = tf.global_variables_initializer()
saver = tf.train.Saver()

n_epochs = 40
batch_size = 50

def shuffle_batch(X, y, batch_size):
    rnd_idx = np.random.permutation(len(X))
    n_batches = len(X) // batch_size
    for batch_idx in np.array_split(rnd_idx, n_batches):
        X_batch, y_batch = X[batch_idx], y[batch_idx]
        yield X_batch, y_batch

with tf.Session() as sess:
    init.run()
    for epoch in range(n_epochs):
        for X_batch, y_batch in shuffle_batch(X_train, y_train,...


I am working on the file "10_libreria_pycuda/pycuda.ipynb".
The first task is:
Además, así como numpy, PyCUDA tiene su propia estructura de datos. Por ejemplo, lo equivalente a un arreglo de numpy es un gpuarray
Can you write Python code for it?


I am working on the file "10_libreria_pycuda/pycuda.ipynb".
The first task is:
Contenido

Computación de alto desempeño (HPC)
Aceleradores
El rol de Python en HPC

Programación en tarjetas gráficas
La tarjeta gráfica
Lenguaje CUDA

PyCUDA
Usando la tarjeta gráfica desde Python
Creando tus propios "kernels"

1. Computación de alto desempeño
En muchos problemas de ingeniería (en verdad, en la gran mayoría de ellos) es necesario hacer una gran catidad de cálculos para lograr el objetivo. Por ejemplo, la gente que modela dinámica de fluidos con el computador, necesita resolver una ecuación en un dominio con varios millones de nodos. Otro caso es el procesamiento de datos de observatorios astronómicos, que pueden generar varios terabytes de datos por día. Los computadores convencionales se quedan "corto" en estos casos: no son suficientemente rápidos como para realizar todos los cálculos en un tiempo razonable, y la memoria es muy pequeña.
La computación de alto desempeño, o High Performance Computing (HPC), es la disciplina que se hace cargo de este problema. La principal herramienta que se usa en HPC es la computación paralela, donde se usan varios procesadores coordinados haciendo cálculos al mismo tiempo. Otra opción, que se puede usar junto a la programación paralela, es el uso de aceleradores. Este notebook trata del uso de tarjetas gráficas (Graphic Processing Units o GPU) como aceleradores, desde Python.
Como su nombre lo dice, historicamente las tarjetas gráficas han sido usadas para hacer la gráfica en la pantalla, sin embargo, desde mediados de los años 2000, comenzaron desarrollos que utilizaron las GPUs para otros propósitos, y nació el concepto General Purpose GPU.
Python y HPC
A pesar que Python es un lenguaje que prioriza facilidad en uso por sobre eficiencia, existen herramientas que permiten hacer computación de alto desempeño con Python. Por ejemplo, la librería mpi4py contiene funciones que permiten operar sobre más de un procesador y coordinarlos. También, la librería petsc4py da acceso a PETSc, una librería de computación paralela desarrollada en C++ para...
Can you write Python code for it?


I am working on the file "10_libreria_pycuda/pycuda.ipynb".
The first task is:
y las principales funciones de PyCUDA están en el driver
Can you write Python code for it?


I am working on the file "10_libreria_pycuda/pycuda.ipynb".
The first task is:
y luego podemos traerlo a la CPU con get() para imprimir su valor
Can you write Python code for it?


I am working on the file "10_matplotlib's_imshow.ipynb".
The first task is:
Ideally, this meta-data is retained, so images are consistent. However, the situation is not ideal, and image meta-data needs to be fixed.
Can you write Python code for it?

img1_seg.CopyInformation(img1)

myshow(sitk.LabelOverlay(img1, sitk.LabelContour(img1_seg), 1.0))


I am working on the file "10_matplotlib's_imshow.ipynb".
The first task is:
SimpleITK has a build in Show method which saves the image to disk and launches a user configurable program ( defaults to ImageJ ), to display the image.
Can you write Python code for it?

img1 = sitk.ReadImage(fdata("cthead1.png"))
sitk.Show(img1, title="cthead1")

img2 = sitk.ReadImage(fdata("VM1111Shrink-RGB.png"))
sitk.Show(img2, title="Visible Human Head")

nda = sitk.GetArrayFromImage(img1)
plt.imshow(nda)

nda = sitk.GetArrayFromImage(img2)
ax = plt.imshow(nda)

def myshow(img):
    nda = sitk.GetArrayFromImage(img)
    plt.imshow(nda)

myshow(img2)

myshow(sitk.Expand(img2, [10]*5))


I am working on the file "10_matplotlib's_imshow.ipynb".
The first task is:
This image does not appear not bigger.
There are numerous improvements that we can make:

support 3d images
include a title
use physical pixel size for axis labels
show the image as gray values
Can you write Python code for it?

def myshow(img, title=None, margin=0.05, dpi=80):
    nda = sitk.GetArrayFromImage(img)
    spacing = img.GetSpacing()

    if nda.ndim == 3:
        # fastest dim, either component or x
        c = nda.shape[-1]

        # the the number of components is 3 or 4 consider it an RGB image
        if not c in (3,4):
            nda = nda[nda.shape[0]//2,:,:]

    elif nda.ndim == 4:
        c = nda.shape[-1]

        if not c in (3,4):
            raise Runtime("Unable to show 3D-vector Image")

        # take a z-slice
        nda = nda[nda.shape[0]//2,:,:,:]

    ysize = nda.shape[0]
    xsize = nda.shape[1]

    # Make a figure big enough to accomodate an axis of xpixels by ypixels
    # as well as the ticklabels, etc...
    figsize = (1 + margin) * ysize / dpi, (1 + margin) * xsize / dpi

    fig = plt.figure(figsize=figsize, dpi=dpi)
    # Make the axis the right size...
    ax = fig.add_axes([margin, margin, 1 - 2*margin, 1 - 2*margin])

    extent = (0, xsize*spacing[1], ysize*spacing[0], 0)

    t = ax.imshow(nda,extent=extent,interpolation=None)

    if nda.ndim == 2:
        t.set_cmap("gray")

    if(title):
        plt.title(title)

myshow(sitk.Expand(img2,[2,2]), title="Big Visibile Human Head")


I am working on the file "10_matplotlib's_imshow.ipynb".
The first task is:
Tips and Tricks for Visualizing Segmentations
Can you write Python code for it?

img1_seg = sitk.ReadImage(fdata("2th_cthead1.png"))
myshow(img1_seg, "Label Image as Grayscale")

myshow(sitk.LabelToRGB(img1_seg), title="Label Image as RGB")


I am working on the file "10_matplotlib's_imshow.ipynb".
The first task is:
Why are the following lines needed?
What do they do?
Can you write Python code for it?

img_labels.CopyInformation(img_T1)

img_T1 = sitk.Cast(sitk.RescaleIntensity(img_T1), sitk.sitkUInt8)


I am working on the file "10_mlops/10a_mlpipeline.ipynb".
The first task is:
Build the container
Can you write Python code for it?


I am working on the file "10_mlops/10a_mlpipeline.ipynb".
The first task is:
Compile and submit pipeline
Can you write Python code for it?

pipeline_func = flowerstxf_pipeline
pipeline_filename = pipeline_func.__name__ + '.zip'
import kfp.compiler as compiler
compiler.Compiler().compile(pipeline_func, pipeline_filename)

import kfp
client = kfp.Client(host=KFPHOST)
experiment = client.create_experiment('from_notebook')
run_name = pipeline_func.__name__ + ' run'
run_result = client.run_pipeline(experiment.id, run_name, pipeline_filename,
                                 {
                                     'project_id': PROJECT,
                                     'bucket': BUCKET,
                                     'region': REGION
                                 })


I am working on the file "10_mlops/10a_mlpipeline.ipynb".
The first task is:
Convert JPEG files to TF Records
Can you write Python code for it?

%%writefile components/create_dataset.yaml
name: create_dataset
description: Converts JPEG files to TensorFlow Records using Dataflow or Apache Beam
inputs:
- {name: runner, type: str, default: 'DirectRunner', description: 'DirectRunner or DataflowRunner'}
- {name: project_id, type: str, description: 'Project to bill Dataflow job to'}
- {name: region, type: str, description: 'Region to run Dataflow job in'}
- {name: input_csv, type: GCSPath, description: 'Path to CSV file'}
- {name: output_dir, type: GCSPath, description: 'Top-level directory for TF records'}
- {name: labels_dict, type: GCSPath, description: 'Dictionary file for class names'}
outputs:
- {name: tfrecords_topdir, type: GCSPath, description: 'Top-level directory for TF records'}
implementation:
  container:
    image: gcr.io/ai-analytics-solutions/practical-ml-vision-book:latest
    command: [
        "bash", "/src/practical-ml-vision-book/10_mlops/components/create_dataset.sh"
    ]
    args: [
        {inputValue: output_dir},
        {outputPath: tfrecords_topdir},
        "--all_data", {inputValue: input_csv},
        "--labels_file", {inputValue: labels_dict},
        "--project_id", {inputValue: project_id},
        "--output_dir", {inputValue: output_dir},
        "--runner", {inputValue: runner},
        "--region", {inputValue: region},
    ]

%%writefile components/noop_create_dataset.yaml
name: noop_create_dataset
description: Converts JPEG files to TensorFlow Records using Dataflow or Apache Beam
inputs:
- {name: runner, type: str, default: 'DirectRunner', description: 'DirectRunner or DataflowRunner'}
- {name: project_id, type: str, description: 'Project to bill Dataflow job to'}
- {name: region, type: str, description: 'Region to run Dataflow job in'}
- {name: input_csv, type: GCSPath, description: 'Path to CSV file'}
- {name: output_dir, type: GCSPath, description: 'Top-level directory for TF records'}
- {name: labels_dict, type: GCSPath, description: 'Dictionary file for class names'}
outputs:
- {name: tfrecords_topdir, type:...


I am working on the file "10_mlops/10a_mlpipeline.ipynb".
The first task is:
Machine Learning Pipeline
In this notebook, we show how to run the flowers classification workflow as a pipeline
Set up
Can you write Python code for it?

%pip install --upgrade --user kfp

# CHANGE AS needed
REGION = 'us-central1'  # Change as needed to a region where you have quota
KFPHOST = 'https://40e09ee3a33a422-dot-us-central1.pipelines.googleusercontent.com'  # Note name of launched Kubeflow Pipelines cluster

PROJECT = !gcloud config get-value project
PROJECT = PROJECT[0]
print(PROJECT)
%env PROJECT = {PROJECT}
%env REGION = {REGION}
BUCKET = PROJECT + "-flowers-pipeline"
%env BUCKET = {BUCKET}

!gsutil mb -l {REGION} gs://{BUCKET}


I am working on the file "10_mlops/10a_mlpipeline.ipynb".
The first task is:
The pipeline
Can you write Python code for it?

import kfp
import kfp.dsl as dsl
import json
import os

create_dataset_op = kfp.components.load_component_from_file(
    #'components/noop_create_dataset.yaml'
    'components/create_dataset.yaml'
)
train_model_op = kfp.components.load_component_from_file(
    #'components/train_model_kfp.yaml'
    'components/train_model_caip.yaml'
)
deploy_op = kfp.components.load_component_from_url(
    'https://raw.githubusercontent.com/kubeflow/pipelines/master/components/gcp/ml_engine/deploy/component.yaml')

@dsl.pipeline(
    name='Flowers Transfer Learning Pipeline',
    description='End-to-end pipeline'
)
def flowerstxf_pipeline(
    project_id = PROJECT,
    bucket = BUCKET,
    region = REGION
):
    # Step 1: Create dataset
    create_dataset = create_dataset_op(
        runner='DataflowRunner',
        project_id=project_id,
        region=region,
        input_csv='gs://practical-ml-vision-book/flowers_5_jpeg/flower_photos/all_data.csv',
        output_dir='gs://{}/data/flower_tfrecords'.format(bucket),
        labels_dict='gs://practical-ml-vision-book/flowers_5_jpeg/flower_photos/dict.txt'
    )
    create_dataset.execution_options.caching_strategy.max_cache_staleness = "P7D"

    # Step 2: Train model
    train_model = train_model_op(
        input_topdir=create_dataset.outputs['tfrecords_topdir'],
        region=region,
        job_dir='gs://{}/trained_model'.format(bucket)
    )
    train_model.execution_options.caching_strategy.max_cache_staleness = "P0D"

    # Step 3: Deploy trained model
    deploy_model = deploy_op(
        model_uri=train_model.outputs['trained_model'],
        project_id=project_id,
        model_id='flowers',
        version_id='txf',
        runtime_version='2.3',
        python_version='3.7',
        version={},
        replace_existing_version='True',
        set_default='True',
        wait_interval='30')


I am working on the file "10_mlops/10a_mlpipeline.ipynb".
The first task is:
Train model
To do it locally on the cluster instead of on CAIP, we'll use gcloud local training
<pre>
gcloud ai-platform local train --package-path $PACKAGE_PATH
       --module-name $MODULE_NAME --job-dir ${JOB_DIR}_local
       -- --num_training_examples 100 --with_color_distort False --crop_ratio 0.6
</pre>
Can you write Python code for it?

%%writefile components/train_model_kfp.yaml
name: train_model_kfp
description: Trains an ML model on KFP
inputs:
- {name: input_topdir, type: GCSPath, description: 'Top-level directory for TF records'}
- {name: region, type: str, description: 'Region (ignored)'}
- {name: job_dir, type: GCSPath, description: 'Top-level output directory'}
outputs:
- {name: trained_model, type: GCSPath, description: 'location of trained model'}
implementation:
  container:
    image: gcr.io/ai-analytics-solutions/practical-ml-vision-book:latest
    command: [
        "bash", "/src/practical-ml-vision-book/10_mlops/components/train_model_kfp.sh",
    ]
    args: [
        {inputValue: input_topdir},
        {inputValue: region},
        {inputValue: job_dir},
        {outputPath: trained_model},
    ]

%%writefile components/train_model_caip.yaml
name: train_model_caip
description: Trains an ML model on CAIP
inputs:
- {name: input_topdir, type: GCSPath, description: 'Top-level directory for TF records'}
- {name: region, type: str, description: 'Region'}
- {name: job_dir, type: GCSPath, description: 'Top-level output directory'}
outputs:
- {name: trained_model, type: GCSPath, description: 'location of trained model'}
implementation:
  container:
    image: gcr.io/ai-analytics-solutions/practical-ml-vision-book:latest
    command: [
        "bash", "/src/practical-ml-vision-book/10_mlops/components/train_model_caip.sh",
    ]
    args: [
        {inputValue: input_topdir},
        {inputValue: region},
        {inputValue: job_dir},
        {outputPath: trained_model},
    ]


I am working on the file "10_model_interpretability/src/dt_surrogate.ipynb".
The first task is:
Clean and prepare data for modeling
Can you write Python code for it?

# assign target and inputs
y = 'SalePrice'
X = [name for name in frame.columns if name not in [y, 'Id']]

# determine column types
# impute
reals, enums = [], []
for key, val in frame.types.items():
    if key in X:
        if val == 'enum':
            enums.append(key)
        else:
            reals.append(key)

_ = frame[reals].impute(method='median')
_ = frame[enums].impute(method='mode')

# split into training an validation, and 30% test
train, valid = frame.split_frame([0.7])


I am working on the file "10_model_interpretability/src/dt_surrogate.ipynb".
The first task is:
Display decision tree surrogate
Can you write Python code for it?


I am working on the file "10_model_interpretability/src/dt_surrogate.ipynb".
The first task is:
Generate GraphViz representation of MOJO
Can you write Python code for it?

details = False # print more info on tree, details = True
title = 'Home Prices Decision Tree Surrogate'

hs = H2OLocalServer()
h2o_jar_path = hs._find_jar()
print('Discovered H2O jar path:\n', h2o_jar_path)

gv_file_name = model_id + '.gv'
gv_args = str('-cp ' + h2o_jar_path +
              ' hex.genmodel.tools.PrintMojo --tree 0 -i '
              + mojo_path + ' -o').split()
gv_args.insert(0, 'java')
gv_args.append(gv_file_name)

if details:
    gv_args.append('--detail')

if title is not None:
    gv_args = gv_args + ['--title', title]

print()
print('Calling external process ...')
print(' '.join(gv_args))

_ = subprocess.call(gv_args)


I am working on the file "10_model_interpretability/src/dt_surrogate.ipynb".
The first task is:
Train a "black box" model
Can you write Python code for it?

# initialize pre-tuned nn model
nn_model = H2ODeepLearningEstimator(
    epochs=50,                    # read over the data 50 times, but in mini-batches
    hidden=[170, 320],            # 100 hidden units in 1 hidden layer
    activation='Tanh',            # bounded activation function that allows for dropout, tanh
    l2=0.007,                     # L2 penalty can increase stability in presence of highly correlated inputs
    adaptive_rate=True,           # adjust magnitude of weight updates automatically (+stability, +accuracy)
    stopping_rounds=2,            # stop after validation error does not decrease for 5 iterations
    score_each_iteration=True,    # score validation error on every iteration
    reproducible=True,
    seed=12345)

# train nn model
nn_model.train(
    x=X,
    y=y,
    training_frame=train,
    validation_frame=valid)

# measure nn AUC
print(nn_model.rmsle(train=True))
print(nn_model.rmsle(valid=True))


I am working on the file "10_model_interpretability/src/dt_surrogate.ipynb".
The first task is:
Train decision tree surrogate model
Can you write Python code for it?

yhat = 'predicted_SalePrice'
model_id = 'dt_surrogate_mojo'

# train single tree surrogate model
surrogate = H2OGradientBoostingEstimator(ntrees=1,
                                         sample_rate=1,
                                         col_sample_rate=1,
                                         max_depth=3,
                                         seed=12345,
                                         model_id=model_id)

_ = surrogate.train(x=X, y=yhat, training_frame=frame_yhat)

# persist MOJO (compiled, representation of trained model)
# from which to generate plot of surrogate
mojo_path = surrogate.download_mojo(path='.')

print(surrogate)
print('Generated MOJO path:\n', mojo_path)


I am working on the file "10_model_interpretability/src/dt_surrogate.ipynb".
The first task is:
Use a decision tree surrogate to generate explanations of the "black box" model
First bind the "black box" model predictions onto the training frame
Can you write Python code for it?

# cbind predictions to training frame
# give them a nice name
preds = nn_model.predict(frame)
preds.columns = ['predicted_SalePrice']
frame_yhat = frame.cbind(preds)


I am working on the file "10_model_interpretability/src/lime.ipynb".
The first task is:
A ranked predictions plot is a way to visually check whether the surrogate model is a good fit for the complex model. The y-axis is the numeric prediction of both models for a given point. The x-axis is the rank of a point when the predictions are sorted by their GBM prediction, from lowest on the left to highest on the right. When both sets of predictions are aligned, as they are above, this a good indication that the linear model fits the complex, nonlinear GBM well in the approximately local region.
Both the R2 and ranked predictions plot show the linear model is a good fit in the practical, approximately local sample. This means the regression coefficients are likely a very accurate representation of the behavior of the nonlinear model in this region.
Create explanations (or 'reason codes') for a row in the local set
The local glm coefficient multiplied by the value in a specific row are estimates of how much each variable contributed to each prediction decision. These values can tell you how a variable and it's values were weighted in any given decision by the model. These values are crucially important for machine learning interpretability and are often to referred to "local feature importance", "reason codes", or "turn-down codes." The latter phrases are borrowed from credit scoring. Credit lenders must provide reasons for turning down a credit application, even for automated decisions. Reason codes can be easily extracted from LIME local feature importance values, by simply ranking the variables that played the largest role in any given decision.
Can you write Python code for it?

row = 20 # select a row to describe
local_contrib_frame = pd.DataFrame(columns=['Name', 'Local Contribution', 'Sign'])

# multiply values in row by local glm coefficients
for name in local_frame[row, :].columns:
    contrib = 0.0
    try:
        contrib = local_frame[row, name]*local_glm.coef()[name]
    except:
        pass
    if contrib != 0.0:
        local_contrib_frame = local_contrib_frame.append({'Name':name,
                                                          'Local Contribution': contrib,
                                                          'Sign': contrib > 0},
                                                         ignore_index=True)

# plot
_ = local_contrib_frame.plot(x = 'Name',
                             y = 'Local Contribution',
                             kind='bar',
                             title='Local Contributions for Row ' + str(row) + '\n',
                             color=local_contrib_frame.Sign.map({True: 'r', False: 'b'}),
                             legend=False)


I am working on the file "10_model_interpretability/src/lime.ipynb".
The first task is:
Here the R2 and ranked predictions plot show a slightly less accurate fit in the local sample. So the regression coefficients and reason codes may be a bit more approximate than those in the first example.
Create explanations (or 'reason codes') for a row in the local set
Can you write Python code for it?

row = 30 # select a row to describe
local_contrib_frame = pd.DataFrame(columns=['Name', 'Local Contribution', 'Sign'])

# multiply values in row by local glm coefficients
for name in local_frame[row, :].columns:
    contrib = 0.0
    try:
        contrib = local_frame[row, name]*local_glm.coef()[name]
    except:
        pass
    if contrib != 0.0:
        local_contrib_frame = local_contrib_frame.append({'Name':name,
                                                          'Local Contribution': contrib,
                                                          'Sign': contrib > 0},
                                                         ignore_index=True)
# plot
_ = local_contrib_frame.plot(x = 'Name',
                             y = 'Local Contribution',
                             kind='bar',
                             title='Local Contributions for Row ' + str(row) + '\n',
                             color=local_contrib_frame.Sign.map({True: 'r', False: 'b'}),
                             legend=False)


I am working on the file "10_model_interpretability/src/lime.ipynb".
The first task is:
LIME can be unstable with data in which strong correlations exist between input variables
Can you write Python code for it?

# print out correlated pairs
corr = train[reals].cor().as_data_frame()
for i in range(0, corr.shape[0]):
    for j in range(0, corr.shape[1]):
        if i != j:
            if np.abs(corr.iat[i, j]) > 0.7:
                print(corr.columns[i], corr.columns[j])


I am working on the file "10_model_interpretability/src/lime.ipynb".
The first task is:
LIME is simpler to use with data containing no missing values
Can you write Python code for it?

# determine column types
# impute
reals, enums = [], []
for key, val in frame.types.items():
    if key in X:
        if val == 'enum':
            enums.append(key)
        else:
            reals.append(key)

_ = frame[reals].impute(method='median')
_ = frame[enums].impute(method='mode')

# split into training and validation
train, valid = frame.split_frame([0.7])


I am working on the file "10_model_interpretability/src/lime.ipynb".
The first task is:
Shutdown H2O
Can you write Python code for it?


I am working on the file "10_model_interpretability/src/lime.ipynb".
The first task is:
Train a predictive model
Can you write Python code for it?

# train GBM model
model = H2OGradientBoostingEstimator(ntrees=100,
                                     max_depth=10,
                                     distribution='huber',
                                     learn_rate=0.1,
                                     stopping_rounds=5,
                                     seed=12345)

model.train(y=y, x=X_reals_decorr, training_frame=train, validation_frame=valid)

preds = valid['Id'].cbind(model.predict(valid))


I am working on the file "10_model_interpretability/src/lime.ipynb".
The first task is:
Train penalized linear model in local region

Check R2 to ensure surrogate model is a good fit for predictions
Use ranked predictions plot to ensure surrogate model is a good fit for predictions
Use trained GLM and coefficients to understand local region of response function
Can you write Python code for it?

%matplotlib inline

# initialize
local_glm = H2OGeneralizedLinearEstimator(lambda_search=True)

# train
local_glm.train(x=X_reals_decorr, y='predict', training_frame=local_frame)

# coefs
print('\nLocal GLM Coefficients:')
for c_name, c_val in sorted(local_glm.coef().items(), key=operator.itemgetter(1)):
    if c_val != 0.0:
        print('%s %s' % (str(c_name + ':').ljust(25), c_val))

# r2
print('\nLocal GLM R-square:\n%.2f' % local_glm.r2())

# ranked predictions plot
pred_frame = local_frame.cbind(local_glm.predict(local_frame))\
                        .as_data_frame()[['predict', 'predict0']]
pred_frame.columns = ['ML Preds.', 'Surrogate Preds.']
pred_frame.sort_values(by='ML Preds.', inplace=True)
pred_frame.reset_index(inplace=True, drop=True)
_ = pred_frame.plot(title='Ranked Predictions Plot')


I am working on the file "10_model_interpretability/src/lime.ipynb".
The first task is:
Train penalized linear model in local region
Can you write Python code for it?

# initialize
local_glm = H2OGeneralizedLinearEstimator(lambda_search=True)

# train
local_glm.train(x=X_reals_decorr, y='predict', training_frame=local_frame)

# ranked predictions plot
pred_frame = local_frame.cbind(local_glm.predict(local_frame))\
                        .as_data_frame()[['predict', 'predict0']]
pred_frame.columns = ['ML Preds.', 'Surrogate Preds.']
pred_frame.sort_values(by='ML Preds.', inplace=True)
pred_frame.reset_index(inplace=True, drop=True)
_ = pred_frame.plot(title='Ranked Predictions Plot')

# r2
print('\nLocal GLM R-square:\n%.2f' % local_glm.r2())

# coefs
print('\nLocal GLM Coefficients:')
for c_name, c_val in sorted(local_glm.coef().items(), key=operator.itemgetter(1)):
    if c_val != 0.0:
        print('%s %s' % (str(c_name + ':').ljust(25), c_val))


I am working on the file "10_model_interpretability/src/loco.ipynb".
The first task is:
Collect LOCO values for each model for the median home
Can you write Python code for it?

median_loco_frames = []
col_names = ['Loco ' + str(i) for i in range(1, n_models + 1)]

for i in range(0, n_models):

    # collect LOCO as a column vector in a Pandas df
    preds = pred_frames[i]
    median_loco_frames.append(preds[preds['Id'] == int(quantile_dict[50]), :]\
                              .as_data_frame()\
                              .drop(['Id', 'predict'], axis=1)
                              .T)

loco_ensemble = pd.concat(median_loco_frames, axis=1)
loco_ensemble.columns = col_names
loco_ensemble['Mean Local Importance'] = loco_ensemble.mean(axis=1)
loco_ensemble['Std. Dev. Local Importance'] = loco_ensemble.std(axis=1)
loco_ensemble


I am working on the file "10_model_interpretability/src/loco.ipynb".
The first task is:
Ensembling explantions to reduce local variance
Explanations derived from high variance machine learning models can be unstable. One general way to decrease variance is to ensemble the results of many models.
Train multiple models
Can you write Python code for it?

n_models = 10 # select number of models

models = []
pred_frames = []

for i in range(0, n_models):

    # store models
    models.append(H2OGradientBoostingEstimator(ntrees=500,
                                               max_depth=2 * (i + 1),
                                               distribution='huber',
                                               learn_rate=0.01 * (i + 1),
                                               stopping_rounds=5,
                                               seed=i + 1))

    # train models
    models[i].train(y=y, x=X_reals_decorr, training_frame=train, validation_frame=valid)

    # store predictions
    pred_frames.append(valid['Id'].cbind(models[i].predict(valid)))

    print('Training Progress: model %d/%d ...' % (i + 1, n_models))

print('Done.')