I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
Many other functions and methods in the array and matrix classes accept the same (optional) axis keyword argument.
Reshaping, resizing and stacking arrays
The shape of an Numpy array can be modified without copying the underlaying data, which makes it a fast operation even for large arrays.
Can you write Python code for it?

M

n, m = M.shape
n, m

N = M.reshape((6, 2))
N

O = M.reshape((1, 12))
O

N[0:2,:] = 1 # modify the array
N

M # and the original variable is also changed. B is only a different view of the same data

O


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
Matrix algebra
What about matrix mutiplication? There are two ways. We can either use the dot function, which applies a matrix-matrix, matrix-vector, or inner vector multiplication to its two arguments:
Can you write Python code for it?


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
Negative indices counts from the end of the array (positive index from the begining):
Can you write Python code for it?

A = array([1,2,3,4,5])

A[-1] # the last element in the array

A[-3:] # the last three elements


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
NumPy arrays
In the numpy package the terminology used for vectors, matrices and higher-dimensional data sets is array. There are a number of ways to initialize new numpy arrays, for example from

a Python list or tuples
using functions that are dedicated to generating numpy arrays, such as arange, linspace, etc.
reading data from files

From lists
For example, to create new vector and matrix arrays from Python lists we can use the numpy.array function.
Can you write Python code for it?

# a vector: the argument to the array function is a Python list
v = array([1,2,3,4])
v

# a matrix: the argument to the array function is a nested Python list
M = array([[1, 2], [3, 4]])
M


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
So far the numpy.ndarray looks awefully much like a Python list (or nested list). Why not simply use Python lists for computations instead of creating a new array type?
There are several reasons:

Python lists are very general. They can contain any kind of object. They are dynamically typed. They do not support mathematical functions such as matrix and dot multiplications, etc. Implementing such functions for Python lists would not be very efficient because of the dynamic typing.
Numpy arrays are statically typed and homogeneous. The type of the elements is determined when the array is created.
Numpy arrays are memory efficient.
Because of the static typing, fast implementation of mathematical functions such as multiplication and addition of numpy arrays can be implemented in a compiled language (C and Fortran is used).

Using the dtype (data type) property of an ndarray, we can see what type the data of an array has:
Can you write Python code for it?


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
Some different ways of working with NumPy are:
Can you write Python code for it?

from numpy import eye, array   # Import only what you need
from numpy.linalg import svd


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
Stacking and repeating arrays
Using function repeat, tile, vstack, hstack, and concatenate we can create larger vectors and matrices from smaller ones:
tile and repeat
Can you write Python code for it?

a = array([[1, 2], [3, 4]])

# repeat each element 3 times
repeat(a, 3)

# tile the matrix 3 times
tile(a, 3)


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
The difference between the v and M arrays is only their shapes. We can get information about the shape of an array by using the ndarray.shape property.
Can you write Python code for it?


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
The number of elements in the array is available through the ndarray.size property:
Can you write Python code for it?


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
The v and M objects are both of the type ndarray that the numpy module provides.
Can you write Python code for it?


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
This feature is very useful to conditionally select elements from an array, using for example comparison operators:
Can you write Python code for it?


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
Using arrays in conditions
When using arrays in conditions,for example if statements and other boolean expressions, one needs to use any or all, which requires that any or all elements in the array evalutes to True:
Can you write Python code for it?

M

if (M > 5).any():
    print("at least one element in M is larger than 5")
else:
    print("no element in M is larger than 5")

if (M > 5).all():
    print("all elements in M are larger than 5")
else:
    print("all elements in M are not larger than 5")

from IPython.core.display import HTML
def css_styling():
    styles = open("./css/sg2.css", "r").read()
    return HTML(styles)
css_styling()


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
We can also use index masks: If the index mask is an Numpy array of data type bool, then an element is selected (True) or not (False) depending on the value of the index mask at the position of each element:
Can you write Python code for it?

B = array([n for n in range(5)])
B

row_mask = array([True, False, True, False, False])
B[row_mask]

# same thing
row_mask = array([1,0,1,0,0], dtype=bool)
B[row_mask]


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
We can also use the function flatten to make a higher-dimensional array into a vector. But this function create a copy of the data.
Can you write Python code for it?

F = M.flatten()
F

F[0:5] = 0
F

M # now M has not changed, because F's data is a copy of M's, not refering to the same data


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
We can assign new values to elements in an array using indexing:
Can you write Python code for it?

M[0,0] = -1

M

# also works for rows and columns
M[0,:] = 0
M[:,1] = -1

M


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
We can omit any of the three parameters in M[lower:upper:step]:
Can you write Python code for it?

A[::] # lower, upper, step all take the default values

A[::2] # step is 2, lower and upper defaults to the beginning and end of the array

A[:3] # first three elements

A[3:] # elements from index 3


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
We get an error if we try to assign a value of the wrong type to an element in a numpy array:
Can you write Python code for it?


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
When we need to iterate over each element of an array and modify its elements, it is convenient to use the enumerate function to obtain both the element and its index in the for loop:
Can you write Python code for it?

for row_idx, row in enumerate(M):
    print("row_idx", row_idx, "row", row)

    for col_idx, element in enumerate(row):
        print("col_idx", col_idx, "element", element)

        # update the matrix M: square each element
        M[row_idx, col_idx] = element ** 2

# each element in M is now squared
M


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
Writing scripts
When writing scripts it is recommended that you:

only import what you need, for efficiency
import packages into namespaces, to avoid name clashes

The community has adopted abbreviated naming conventions:
Can you write Python code for it?

import numpy as np
import scipy as sp
import matplotlib as mpl
import matplotlib.pyplot as plt


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
mgrid
Can you write Python code for it?

x, y = mgrid[0:5, 0:5] # similar to meshgrid in MATLAB

x

y


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
min and max
Can you write Python code for it?


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
sum, prod, and trace
Can you write Python code for it?

# sum up all elements
sum(d)

# product of all elements
prod(d+1)

# cummulative sum
cumsum(d)

# cummulative product
cumprod(d+1)

A

# same as: diag(A).sum()
trace(A)


I am working on the file "01_Scientific_Python_numpy.ipynb".
The first task is:
zeros_like and ones_like
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Axial loads
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Compute Jt
Computation of torsional moment of inertia:
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Compute axial loads in Stringers in S2
Set loads on the section:
Example 2: shear in y direction and bending moment in x direction
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Compute axial loads in stringers and shear flows in panels
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Define section and perform first calculations
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Define sympy parameters used for geometric description of sections
Can you write Python code for it?

A, A0, t, t0, a, b, h, L = sympy.symbols('A A_0 t t_0 a b h L', positive=True)


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Example 2: twisting moment in z direction
Can you write Python code for it?

S1.set_loads(_Tx=0, _Ty=0, _Nz=0, _Mx=0, _My=0, _Mz=Mz)
S1.compute_stringer_actions()
S1.compute_panel_fluxes();


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Expression of Inertial properties wrt Center of Gravity in with original rotation
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
First example: Closed section
Define graph describing the section:
1) stringers are nodes with parameters:
- x coordinate
- y coordinate
- Area
2) panels are oriented edges with parameters:
- thickness
- lenght which is automatically calculated
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Import Section class, which contains all calculations
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Initialization of sympy symbolic tool and pint for dimension analysis (not really implemented rn as not directly compatible with sympy)
Can you write Python code for it?

ureg = UnitRegistry()
sympy.init_printing()


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Panel fluxes
Not really an easy expression
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Plot of S1 section in original reference frame
Define a dictionary of coordinates used by Networkx to plot section as a Directed graph.
Note that arrows are actually just thicker stubs
Can you write Python code for it?

start_pos={ii: [float(S1.g.node[ii]['ip'][i].subs(datav)) for i in range(2)] for ii in S1.g.nodes() }

plt.figure(figsize=(12,8),dpi=300)
nx.draw(S1.g,with_labels=True, arrows= True, pos=start_pos)
plt.arrow(0,0,20,0)
plt.arrow(0,0,0,20)
#plt.text(0,0, 'CG', fontsize=24)
plt.axis('equal')
plt.title("Section in starting reference Frame",fontsize=16);


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Plot of S2 section in inertial reference Frame
Section is plotted wrt center of gravity and rotated (if necessary) so that x and y are principal axes.
Center of Gravity and Shear Center are drawn
Can you write Python code for it?

positions={ii: [float(S2.g.node[ii]['pos'][i].subs(datav)) for i in range(2)] for ii in S2.g.nodes() }

x_ct, y_ct = S2.ct.subs(datav)

plt.figure(figsize=(12,8),dpi=300)
nx.draw(S2.g,with_labels=True, pos=positions)
plt.plot([0],[0],'o',ms=12,label='CG')
plt.plot([x_ct],[y_ct],'^',ms=12, label='SC')
#plt.text(0,0, 'CG', fontsize=24)
#plt.text(x_ct,y_ct, 'SC', fontsize=24)
plt.legend(loc='lower right', shadow=True)
plt.axis('equal')
plt.title("Section in pricipal reference Frame",fontsize=16);


positions={ii: [float(S2_2.g.node[ii]['pos'][i].subs(datav)) for i in range(2)] for ii in S2_2.g.nodes() }

x_ct, y_ct = S2_2.ct.subs(datav)

plt.figure(figsize=(12,8),dpi=300)
nx.draw(S2_2.g,with_labels=True, pos=positions)
plt.plot([0],[0],'o',ms=12,label='CG')
plt.plot([x_ct],[y_ct],'^',ms=12, label='SC')
#plt.text(0,0, 'CG', fontsize=24)
#plt.text(x_ct,y_ct, 'SC', fontsize=24)
plt.legend(loc='lower right', shadow=True)
plt.axis('equal')
plt.title("Section in pricipal reference Frame",fontsize=16);


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Second example (2): Open section
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Set loads on the section:
Example 2: shear in x direction and bending moment in y direction
Can you write Python code for it?

S2.set_loads(_Tx=Tx, _Ty=0, _Nz=0, _Mx=0, _My=My, _Mz=0)
S2.compute_stringer_actions()
S2.compute_panel_fluxes();

S2.N

S2.q


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Set loads on the section:
Example 3: shear in x direction and bending moment in y direction
Can you write Python code for it?

S1.set_loads(_Tx=Tx, _Ty=0, _Nz=0, _Mx=0, _My=My, _Mz=0)
S1.compute_stringer_actions()
S1.compute_panel_fluxes();


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Shear center expression
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Shear flows
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
Verify that we find a simply closed section
Can you write Python code for it?


I am working on the file "01_SemiMonoCoque.ipynb".
The first task is:
We also define numerical values for each symbol in order to plot scaled section and perform calculations
Can you write Python code for it?

values = [(A, 150 * ureg.millimeter**2),(A0, 250  * ureg.millimeter**2),(a, 80 * ureg.millimeter), \
          (b, 20 * ureg.millimeter),(h, 35 * ureg.millimeter),(L, 2000 * ureg.millimeter)]
datav = [(v[0],v[1].magnitude) for v in values]


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Cost-function to be optimized
To make the model better at classifying the input images, we must somehow change the variables for weights and biases. To do this we first need to know how well the model currently performs by comparing the predicted output of the model y_pred to the desired output y_true.
The cross-entropy is a performance measure used in classification. The cross-entropy is a continuous function that is always positive and if the predicted output of the model exactly matches the desired output then the cross-entropy equals zero. The goal of optimization is therefore to minimize the cross-entropy so it gets as close to zero as possible by changing the weights and biases of the model.
TensorFlow has a built-in function for calculating the cross-entropy. Note that it uses the values of the logits because it also calculates the softmax internally.
Can you write Python code for it?

cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits,
                                                               labels=y_true)


cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits=logits,
                                                        labels=y_true)


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Data dimensions
The data dimensions are used in several places in the source-code below. In computer programming it is generally best to use variables and constants rather than having to hard-code specific numbers every time that number is used. This means the numbers only have to be changed in one single place. Ideally these would be inferred from the data that has been read, but here we just write the numbers.
Can you write Python code for it?

# We know that MNIST images are 28 pixels in each dimension.
img_size = 28

# Images are stored in one-dimensional arrays of this length.
img_size_flat = img_size * img_size

# Tuple with height and width of images used to reshape arrays.
img_shape = (img_size, img_size)

# Number of classes, one class for each of 10 digits.
num_classes = 10


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Finally we have the placeholder variable for the true class of each image in the placeholder variable x. These are integers and the dimensionality of this placeholder variable is set to [None] which means the placeholder variable is a one-dimensional vector of arbitrary length.
Can you write Python code for it?

y_true_cls = tf.placeholder(tf.int64, [None])


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Function for performing a number of optimization iterations so as to gradually improve the weights and biases of the model. In each iteration, a new batch of data is selected from the training-set and then TensorFlow executes the optimizer using those training samples.
Can you write Python code for it?

def optimize(num_iterations):
    for i in range(num_iterations):
        # Get a batch of training examples.
        # x_batch now holds a batch of images and
        # y_true_batch are the true labels for those images.
        x_batch, y_true_batch = data.train.next_batch(batch_size)

        # Put the batch into a dict with the proper names
        # for placeholder variables in the TensorFlow graph.
        # Note that the placeholder for y_true_cls is not set
        # because it is not used during training.
        feed_dict_train = {x: x_batch,
                           y_true: y_true_batch}

        # Run the optimizer using this batch of training data.
        # TensorFlow assigns the variables in feed_dict_train
        # to the placeholder variables and then runs the optimizer.
        session.run(optimizer, feed_dict=feed_dict_train)


def optimize(num_iterations):
    for i in range(num_iterations):
        # Get a batch of training examples.
        # x_batch now holds a batch of images and
        # y_true_batch are the true labels for those images.
        x_batch, y_true_batch, _ = data.random_batch(batch_size=batch_size)

        # Put the batch into a dict with the proper names
        # for placeholder variables in the TensorFlow graph.
        # Note that the placeholder for y_true_cls is not set
        # because it is not used during training.
        feed_dict_train = {x: x_batch,
                           y_true: y_true_batch}

        # Run the optimizer using this batch of training data.
        # TensorFlow assigns the variables in feed_dict_train
        # to the placeholder variables and then runs the optimizer.
        session.run(optimizer, feed_dict=feed_dict_train)


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Function for plotting examples of images from the test-set that have been mis-classified.
Can you write Python code for it?

def plot_example_errors():
    # Use TensorFlow to get a list of boolean values
    # whether each test-image has been correctly classified,
    # and a list for the predicted class of each image.
    correct, cls_pred = session.run([correct_prediction, y_pred_cls],
                                    feed_dict=feed_dict_test)

    # Negate the boolean array.
    incorrect = (correct == False)

    # Get the images from the test-set that have been
    # incorrectly classified.
    images = data.test.images[incorrect]

    # Get the predicted classes for those images.
    cls_pred = cls_pred[incorrect]

    # Get the true classes for those images.
    cls_true = data.test.cls[incorrect]

    # Plot the first 9 images.
    plot_images(images=images[0:9],
                cls_true=cls_true[0:9],
                cls_pred=cls_pred[0:9])


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Function for printing and plotting the confusion matrix using scikit-learn.
Can you write Python code for it?

def print_confusion_matrix():
    # Get the true classifications for the test-set.
    cls_true = data.test.cls

    # Get the predicted classifications for the test-set.
    cls_pred = session.run(y_pred_cls, feed_dict=feed_dict_test)

    # Get the confusion matrix using sklearn.
    cm = confusion_matrix(y_true=cls_true,
                          y_pred=cls_pred)

    # Print the confusion matrix as text.
    print(cm)

    # Plot the confusion matrix as an image.
    plt.imshow(cm, interpolation='nearest', cmap=plt.cm.Blues)

    # Make various adjustments to the plot.
    plt.tight_layout()
    plt.colorbar()
    tick_marks = np.arange(num_classes)
    plt.xticks(tick_marks, range(num_classes))
    plt.yticks(tick_marks, range(num_classes))
    plt.xlabel('Predicted')
    plt.ylabel('True')

    # Ensure the plot is shown correctly with multiple plots
    # in a single Notebook cell.
    plt.show()


def print_confusion_matrix():
    # Get the true classifications for the test-set.
    cls_true = data.y_test_cls

    # Get the predicted classifications for the test-set.
    cls_pred = session.run(y_pred_cls, feed_dict=feed_dict_test)

    # Get the confusion matrix using sklearn.
    cm = confusion_matrix(y_true=cls_true,
                          y_pred=cls_pred)

    # Print the confusion matrix as text.
    print(cm)

    # Plot the confusion matrix as an image.
    plt.imshow(cm, interpolation='nearest', cmap=plt.cm.Blues)

    # Make various adjustments to the plot.
    plt.tight_layout()
    plt.colorbar()
    tick_marks = np.arange(num_classes)
    plt.xticks(tick_marks, range(num_classes))
    plt.yticks(tick_marks, range(num_classes))
    plt.xlabel('Predicted')
    plt.ylabel('True')

    # Ensure the plot is shown correctly with multiple plots
    # in a single Notebook cell.
    plt.show()


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Helper-function for plotting images
Function used to plot 9 images in a 3x3 grid, and writing the true and predicted classes below each image.
Can you write Python code for it?

def plot_images(images, cls_true, cls_pred=None):
    assert len(images) == len(cls_true) == 9

    # Create figure with 3x3 sub-plots.
    fig, axes = plt.subplots(3, 3)
    fig.subplots_adjust(hspace=0.3, wspace=0.3)

    for i, ax in enumerate(axes.flat):
        # Plot image.
        ax.imshow(images[i].reshape(img_shape), cmap='binary')

        # Show true and predicted classes.
        if cls_pred is None:
            xlabel = "True: {0}".format(cls_true[i])
        else:
            xlabel = "True: {0}, Pred: {1}".format(cls_true[i], cls_pred[i])

        ax.set_xlabel(xlabel)

        # Remove ticks from the plot.
        ax.set_xticks([])
        ax.set_yticks([])

    # Ensure the plot is shown correctly with multiple plots
    # in a single Notebook cell.
    plt.show()


def plot_images(images, cls_true, cls_pred=None):
    assert len(images) == len(cls_true) == 9

    # Create figure with 3x3 sub-plots.
    fig, axes = plt.subplots(3, 3)
    fig.subplots_adjust(hspace=0.3, wspace=0.3)

    for i, ax in enumerate(axes.flat):
        # Plot image.
        ax.imshow(images[i].reshape(img_shape), cmap='binary')

        # Show true and predicted classes.
        if cls_pred is None:
            xlabel = "True: {0}".format(cls_true[i])
        else:
            xlabel = "True: {0}, Pred: {1}".format(cls_true[i], cls_pred[i])

        ax.set_xlabel(xlabel)

        # Remove ticks from the plot.
        ax.set_xticks([])
        ax.set_yticks([])


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Helper-function to perform optimization iterations
There are 50.000 images in the training-set. It takes a long time to calculate the gradient of the model using all these images. We therefore use Stochastic Gradient Descent which only uses a small batch of images in each iteration of the optimizer.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Helper-function to plot the model weights
Function for plotting the weights of the model. 10 images are plotted, one for each digit that the model is trained to recognize.
Can you write Python code for it?

def plot_weights():
    # Get the values for the weights from the TensorFlow variable.
    w = session.run(weights)

    # Get the lowest and highest values for the weights.
    # This is used to correct the colour intensity across
    # the images so they can be compared with each other.
    w_min = np.min(w)
    w_max = np.max(w)

    # Create figure with 3x4 sub-plots,
    # where the last 2 sub-plots are unused.
    fig, axes = plt.subplots(3, 4)
    fig.subplots_adjust(hspace=0.3, wspace=0.3)

    for i, ax in enumerate(axes.flat):
        # Only use the weights for the first 10 sub-plots.
        if i<10:
            # Get the weights for the i'th digit and reshape it.
            # Note that w.shape == (img_size_flat, 10)
            image = w[:, i].reshape(img_shape)

            # Set the label for the sub-plot.
            ax.set_xlabel("Weights: {0}".format(i))

            # Plot the image.
            ax.imshow(image, vmin=w_min, vmax=w_max, cmap='seismic')

        # Remove ticks from each sub-plot.
        ax.set_xticks([])
        ax.set_yticks([])

    # Ensure the plot is shown correctly with multiple plots
    # in a single Notebook cell.
    plt.show()


def plot_weights():
    # Get the values for the weights from the TensorFlow variable.
    w = session.run(weights)

    # Get the lowest and highest values for the weights.
    # This is used to correct the colour intensity across
    # the images so they can be compared with each other.
    w_min = np.min(w)
    w_max = np.max(w)

    # Create figure with 3x4 sub-plots,
    # where the last 2 sub-plots are unused.
    fig, axes = plt.subplots(3, 4)
    fig.subplots_adjust(hspace=0.3, wspace=0.3)

    for i, ax in enumerate(axes.flat):
        # Only use the weights for the first 10 sub-plots.
        if i<10:
            # Get the weights for the i'th digit and reshape it.
            # Note that w.shape == (img_size_flat, 10)
            image = w[:, i].reshape(img_shape)

            # Set the label for the sub-plot.
            ax.set_xlabel("Weights: {0}".format(i))

            # Plot the image.
            ax.imshow(image, vmin=w_min, vmax=w_max, cmap='seismic')

        # Remove ticks from each sub-plot.
        ax.set_xticks([])
        ax.set_yticks([])


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Helper-functions to show performance
Dict with the test-set data to be used as input to the TensorFlow graph. Note that we must use the correct names for the placeholder variables in the TensorFlow graph.
Can you write Python code for it?

feed_dict_test = {x: data.test.images,
                  y_true: data.test.labels,
                  y_true_cls: data.test.cls}


feed_dict_test = {x: data.x_test,
                  y_true: data.y_test,
                  y_true_cls: data.y_test_cls}


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Initialize variables
The variables for weights and biases must be initialized before we start optimizing them.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Load Data
The MNIST data-set is about 12 MB and will be downloaded automatically if it is not located in the given path.
Can you write Python code for it?

from mnist import MNIST
data = MNIST(data_dir="data/MNIST/")


from tensorflow.examples.tutorials.mnist import input_data
data = input_data.read_data_sets("data/MNIST/", one_hot=False)


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Now logits is a matrix with num_images rows and num_classes columns, where the element of the $i$'th row and $j$'th column is an estimate of how likely the $i$'th input image is to be of the $j$'th class.
However, these estimates are a bit rough and difficult to interpret because the numbers may be very small or large, so we want to normalize them so that each row of the logits matrix sums to one, and each element is limited between zero and one. This is calculated using the so-called softmax function and the result is stored in y_pred.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
One-Hot Encoding
The data-set has been loaded as so-called One-Hot encoding. This means the labels have been converted from a single number to a vector whose length equals the number of possible classes. All elements of the vector are zero except for the $i$'th element which is one and means the class is $i$. For example, the One-Hot encoded labels for the first 5 images in the test-set are:
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Optimization method
Now that we have a cost measure that must be minimized, we can then create an optimizer. In this case it is the basic form of Gradient Descent where the step-size is set to 0.5.
Note that optimization is not performed at this point. In fact, nothing is calculated at all, we just add the optimizer-object to the TensorFlow graph for later execution.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Performance after 1 optimization iteration
Already after a single optimization iteration, the model has increased its accuracy on the test-set significantly.
Can you write Python code for it?

optimize(num_iterations=1)

print_accuracy()

plot_example_errors()


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Performance after 1 optimization iteration
Already after a single optimization iteration, the model has increased its accuracy on the test-set to 40.7% up from 9.8%. This means that it mis-classifies the images about 6 out of 10 times, as demonstrated on a few examples below.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Performance after 10 optimization iterations
Can you write Python code for it?

# We have already performed 1 iteration.
optimize(num_iterations=9)

print_accuracy()

plot_example_errors()

plot_weights()


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Performance after 1000 optimization iterations
After 1000 optimization iterations, the model only mis-classifies about one in ten images. As demonstrated below, some of the mis-classifications are justified because the images are very hard to determine with certainty even for humans, while others are quite obvious and should have been classified correctly by a good model. But this simple model cannot reach much better performance and more complex models are therefore needed.
Can you write Python code for it?

# We have already performed 10 iterations.
optimize(num_iterations=990)

print_accuracy()

plot_example_errors()


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Performance before any optimization
The accuracy on the test-set is 9.8%. This is because the model has only been initialized and not optimized at all, so it always predicts that the image shows a zero digit, as demonstrated in the plot below, and it turns out that 9.8% of the images in the test-set happens to be zero digits.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Performance measures
We need a few more performance measures to display the progress to the user.
This is a vector of booleans whether the predicted class equals the true class of each image.
Can you write Python code for it?

correct_prediction = tf.equal(y_pred_cls, y_true_cls)


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Plot a few images to see if data is correct
Can you write Python code for it?

# Get the first images from the test-set.
images = data.x_test[0:9]

# Get the true classes for those images.
cls_true = data.y_test_cls[0:9]

# Plot the images and labels using our helper-function above.
plot_images(images=images, cls_true=cls_true)


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
TensorFlow Graph
The entire purpose of TensorFlow is to have a so-called computational graph that can be executed much more efficiently than if the same calculations were to be performed directly in Python. TensorFlow can be more efficient than NumPy because TensorFlow knows the entire computation graph that must be executed, while NumPy only knows the computation of a single mathematical operation at a time.
TensorFlow can also automatically calculate the gradients that are needed to optimize the variables of the graph so as to make the model perform better. This is because the graph is a combination of simple mathematical expressions so the gradient of the entire graph can be calculated using the chain-rule for derivatives.
TensorFlow can also take advantage of multi-core CPUs as well as GPUs - and Google has even built special chips just for TensorFlow which are called TPUs (Tensor Processing Units) and are even faster than GPUs.
A TensorFlow graph consists of the following parts which will be detailed below:

Placeholder variables used to change the input to the graph.
Model variables that are going to be optimized so as to make the model perform better.
The model which is essentially just a mathematical function that calculates some output given the input in the placeholder variables and the model variables.
A cost measure that can be used to guide the optimization of the variables.
An optimization method which updates the variables of the model.

In addition, the TensorFlow graph may also contain various debugging statements e.g. for logging data to be displayed using TensorBoard, which is not covered in this tutorial.
Placeholder variables
Placeholder variables serve as the input to the graph that we may change each time we execute the graph. We call this feeding the placeholder variables and it is demonstrated further below.
First we define the placeholder variable for the input images. This allows us to change the images that are input to the TensorFlow graph. This is a so-called tensor, which just...
Can you write Python code for it?

x = tf.placeholder(tf.float32, [None, img_size_flat])


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
TensorFlow Run
Create TensorFlow session
Once the TensorFlow graph has been created, we have to create a TensorFlow session which is used to execute the graph.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
The MNIST data-set has now been loaded and consists of 70.000 images and associated labels (i.e. classifications of the images). The data-set is split into 3 mutually exclusive sub-sets. We will only use the training and test-sets in this tutorial.
Can you write Python code for it?

print("Size of:")
print("- Training-set:\t\t{}".format(len(data.train.labels)))
print("- Test-set:\t\t{}".format(len(data.test.labels)))
print("- Validation-set:\t{}".format(len(data.validation.labels)))


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
The MNIST data-set has now been loaded and consists of 70.000 images and class-numbers for the images. The data-set is split into 3 mutually exclusive sub-sets. We will only use the training and test-sets in this tutorial.
Can you write Python code for it?

print("Size of:")
print("- Training-set:\t\t{}".format(data.num_train))
print("- Validation-set:\t{}".format(data.num_val))
print("- Test-set:\t\t{}".format(data.num_test))


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
The model has now been trained for 1000 optimization iterations, with each iteration using 100 images from the training-set. Because of the great variety of the images, the weights have now become difficult to interpret and we may doubt whether the model truly understands how digits are composed from lines, or whether the model has just memorized many different variations of pixels.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
The predicted class can be calculated from the y_pred matrix by taking the index of the largest element in each row.
Can you write Python code for it?

y_pred_cls = tf.argmax(y_pred, dimension=1)


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
The second variable that must be optimized is called biases and is defined as a 1-dimensional tensor (or vector) of length num_classes.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
The weights can also be plotted as shown below. Positive weights are red and negative weights are blue. These weights can be intuitively understood as image-filters.
For example, the weights used to determine if an image shows a zero-digit have a positive reaction (red) to an image of a circle, and  have a negative reaction (blue) to images with content in the centre of the circle.
Similarly, the weights used to determine if an image shows a one-digit react positively (red) to a vertical line in the centre of the image, and react negatively (blue) to images with content surrounding that line.
Note that the weights mostly look like the digits they're supposed to recognize. This is because only one optimization iteration has been performed so the weights are only trained on 100 images. After training on several thousand images, the weights become more difficult to interpret because they have to recognize many variations of how digits can be written.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
This calculates the classification accuracy by first type-casting the vector of booleans to floats, so that False becomes 0 and True becomes 1, and then calculating the average of these numbers.
Can you write Python code for it?

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
This was developed using Python 3.5.2 (Anaconda) and TensorFlow version:
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
This was developed using Python 3.6 (Anaconda) and TensorFlow version:
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
Variables to be optimized
Apart from the placeholder variables that were defined above and which serve as feeding input data into the model, there are also some model variables that must be changed by TensorFlow so as to make the model perform better on the training data.
The first variable that must be optimized is called weights and is defined here as a TensorFlow variable that must be initialized with zeros and whose shape is [img_size_flat, num_classes], so it is a 2-dimensional tensor (or matrix) with img_size_flat rows and num_classes columns.
Can you write Python code for it?

weights = tf.Variable(tf.zeros([img_size_flat, num_classes]))


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
We also need the classes as single numbers for various comparisons and performance measures, so we convert the One-Hot encoded vectors to a single number by taking the index of the highest element. Note that the word 'class' is a keyword used in Python so we need to use the name 'cls' instead.
Can you write Python code for it?

data.test.cls = np.array([label.argmax() for label in data.test.labels])


data.test.cls  = data.test.labels     #np.array([label.argmax() for label in data.test.labels])
data.train.cls = data.train.labels    #np.array([label.argmax() for label in data.train.labels])


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
We are now done using TensorFlow, so we close the session to release its resources.
Can you write Python code for it?

# This has been commented out in case you want to modify and experiment
# with the Notebook without having to restart it.
# session.close()


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
We can also print and plot the so-called confusion matrix which lets us see more details about the mis-classifications. For example, it shows that images actually depicting a 5 have sometimes been mis-classified as all other possible digits, but mostly as 6 or 8.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
We can also print and plot the so-called confusion matrix which lets us see more details about the mis-classifications. For example, it shows that images actually depicting a 5 have sometimes been mis-classified as all other possible digits, but mostly either 3, 6 or 8.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
We can now see the class for the first five images in the test-set. Compare these to the One-Hot encoded vectors above. For example, the class for the first image is 7, which corresponds to a One-Hot encoded vector where all elements are zero except for the element with index 7.
Can you write Python code for it?


I am working on the file "01_Simple_Linear_Model.ipynb".
The first task is:
We have now calculated the cross-entropy for each of the image classifications so we have a measure of how well the model performs on each image individually. But in order to use the cross-entropy to guide the optimization of the model's variables we need a single scalar value, so we simply take the average of the cross-entropy for all the image classifications.
Can you write Python code for it?


I am working on the file "01_Sort.ipynb".
The first task is:
Sort Colors
https://leetcode.com/problems/sort-colors/?tab=Description
Can you write Python code for it?

class Solution(object):
    def sortColors(self, nums):
        """
        :type nums: List[int]
        :rtype: void Do not return anything, modify nums in-place instead.
        """
        dic = {0: 0, 1: 0, 2: 0}
        for num in nums:
            dic[num] = dic.get(num, 0) + 1
        print dic
        for i in range(len(nums)):
            if i < dic[0]:
                nums[i] = 0
            elif i < dic[0] + dic[1]:
                nums[i] = 1
            else:
                nums[i] = 2

s = Solution()

s.sortColors([1])


I am working on the file "01_Sort.ipynb".
The first task is:
Sort List

Sort a linked list in O(n log n) time using constant space complexity.
Can you write Python code for it?

# Definition for singly-linked list.
# class ListNode(object):
#     def __init__(self, x):
#         self.val = x
#         self.next = None

class Solution(object):
    def sortList(self, head):
        """
        :type head: ListNode
        :rtype: ListNode
        """
        if not head or not head.next:
            return head
        fast, slow = head.next, head
        while fast and fast.next:
            fast, slow = fast.next.next, slow.next
        second = slow.next
        slow.next = None
        l1 = self.sortList(head)
        l2 = self.sortList(second)
        return self.merge(l1, l2)

    def merge(self, l1, l2):
        dummy = node = ListNode(0)
        while l1 and l2:
            if l1.val <= l2.val:
                node.next = l1
                l1 = l1.next
            else:
                node.next = l2
                l2 = l2.next
            node = node.next
        node.next = l1 or l2

        return dummy.next


I am working on the file "01_Sort.ipynb".
The first task is:
Third Maximum Number

Given a non-empty array of integers, return the third maximum number in this array. If it does not exist, return the maximum number. The time complexity must be in O(n).
Example 1:
Input: [3, 2, 1]
Output: 1
Explanation: The third maximum is 1.
Example 2:
Input: [1, 2]
Output: 2
Explanation: The third maximum does not exist, so the maximum (2) is returned instead.
Example 3:
Input: [2, 2, 3, 1]
Output: 1
Explanation: Note that the third maximum here means the third maximum distinct number.
Both numbers with value 2 are both considered as second maximum.
Can you write Python code for it?

class Solution(object):
    def thirdMax(self, nums):
        """
        :type nums: List[int]
        :rtype: int
        """
        nums = set(nums)   ## this is important!!
        if len(nums) < 3:
            return max(nums)
        nums.remove(max(nums))
        nums.remove(max(nums))
        return max(nums)


I am working on the file "01_Sort.ipynb".
The first task is:
Top K Frequent Elements

Given a non-empty array of integers, return the k most frequent elements.
For example,
Given [1,1,1,2,2,3] and k = 2, return [1,2].
Note:
You may assume k is always valid, 1 ≤ k ≤ number of unique elements.
Your algorithm's time complexity must be better than O(n log n), where n is the array's size.
Show Company Tags
Show Tags
Show Similar Problems
Can you write Python code for it?

class Solution(object):
    def topKFrequent(self, nums, k):
        """
        :type nums: List[int]
        :type k: int
        :rtype: List[int]
        """
        dic, ans = {}, []
        for i in range(len(nums)):
            dic[nums[i]] = dic.get(nums[i], 0) + 1
        for num in dic:
            if len(ans) == 0 or dic[num] <= dic[ans[-1]]:
                ans.append(num)
            elif dic[num] >= dic[ans[0]]:
                ans = [num] + ans
            else:
                l, r = 0, len(ans)-1
                while l < r:
                    m = (l+r)/2
                    if dic[ans[m]] >= dic[num]:
                        l = m + 1
                    else:
                        r = m - 1
                if dic[ans[l]] >= dic[num]:
                    ans = ans[:l+1] + [num] + ans[l+1:]
                else:
                    ans = ans[:l] + [num] + ans[l:]
            if len(ans) > k:
                ans.pop()
        return ans


I am working on the file "01_Sort.ipynb".
The first task is:
Wiggle sort II

https://leetcode.com/problems/wiggle-sort-ii/?tab=Description
Given an unsorted array nums, reorder it such that nums[0] < nums[1] > nums[2] < nums[3]....
Example:
(1) Given nums = [1, 5, 1, 1, 6, 4], one possible answer is [1, 4, 1, 5, 1, 6].
(2) Given nums = [1, 3, 2, 2, 3, 1], one possible answer is [2, 3, 1, 3, 1, 2].
Note:
You may assume all input has valid answer.
Can you write Python code for it?

class Solution(object):
    def wiggleSort(self, nums):
        """
        :type nums: List[int]
        :rtype: void Do not return anything, modify nums in-place instead.
        """

        nums.sort()
        half = len(nums[::2]) - 1
        nums[::2], nums[1::2] = nums[half::-1], nums[:half:-1]


I am working on the file "01_Sort.ipynb".
The first task is:
merge two sorted lists

Merge two sorted linked lists and return it as a new list. The new list should be made by splicing together the nodes of the first two lists.
Can you write Python code for it?

# Definition for singly-linked list.
# class ListNode(object):
#     def __init__(self, x):
#         self.val = x
#         self.next = None

class Solution(object):
    def mergeTwoLists(self, l1, l2):
        """
        :type l1: ListNode
        :type l2: ListNode
        :rtype: ListNode
        """

        dummy = node = ListNode(0)
        while l1 and l2:
            if l1.val <= l2.val:
                node.next = l1
                l1 = l1.next
            else:
                node.next = l2
                l2 = l2.next
            node = node.next
        node.next = l1 or l2

        return dummy.next

        # if not l1:
        #     return l2
        # elif not l2:
        #     return l1

        # head = ListNode(0)
        # node = head

        # while l1 and l2:
        #     if l1.val <= l2.val:
        #         node.next = l1
        #         l1 = l1.next
        #     else:
        #         node.next = l2
        #         l2 = l2.next
        #     node = node.next
        # node.next = l1 if not l2 else l2
        # head = head.next

        # return head


I am working on the file "01_TF_basics_and_linear_regression/linear_regression_mx.ipynb".
The first task is:
Hyper Parameters, 超参数
Can you write Python code for it?

learning_rate = 0.01
training_epochs = 1000
smoothing_constant = 0.01
display_step = 50
ctx = mx.cpu()


I am working on the file "01_TF_basics_and_linear_regression/linear_regression_mx.ipynb".
The first task is:
Mean squared error, 损失函数：均方差
Can you write Python code for it?

# Mean squared error，损失函数：均方差
def square_loss(yhat, y):
    return nd.mean((yhat - y) ** 2)

# Gradient descent， 优化方式：梯度下降
def SGD(params, lr):
    for param in params:
        param[:] = param - lr * param.grad


I am working on the file "01_TF_basics_and_linear_regression/linear_regression_mx.ipynb".
The first task is:
Prepare for Training, 训练准备
mx Graph Input, mxnet图输入
Can you write Python code for it?

# Set model weights，初始化网络模型的权重
W = nd.random_normal(shape=1)
b = nd.random_normal(shape=1)

params = [W, b]
for param in params:
    param.attach_grad()


I am working on the file "01_TF_basics_and_linear_regression/linear_regression_mx.ipynb".
The first task is:
Regression result, 回归结果
Can you write Python code for it?

def plot(losses, X, Y, n_samples=10):
    xs = list(range(len(losses)))
    f, (fg1, fg2) = plt.subplots(1, 2)
    fg1.set_title('Loss during training')
    fg1.plot(xs, losses, '-r')
    fg2.set_title('Estimated vs real function')
    fg2.plot(X.asnumpy(), net(X).asnumpy(), 'or', label='Estimated')
    fg2.plot(X.asnumpy(), Y.asnumpy(), '*g', label='Real')
    fg2.legend()
    plt.show()

plot(losses, data, label)


I am working on the file "01_TF_basics_and_linear_regression/linear_regression_mx.ipynb".
The first task is:
Start training, 开始训练
Can you write Python code for it?

# Fit training data
data = nd.array(train_X)
label = nd.array(train_Y)
losses = []
moving_loss = 0
niter = 0

for e in range(training_epochs):
    with autograd.record():
        output = net(data)
        loss = square_loss(output, label)
    loss.backward()
    SGD(params, learning_rate)

    ##########################
    #  Keep a moving average of the losses
    ##########################
    niter +=1
    curr_loss = nd.mean(loss).asscalar()
    moving_loss = (1 - smoothing_constant) * moving_loss + (smoothing_constant) * curr_loss

    # correct the bias from the moving averages
    est_loss = moving_loss/(1-(1-smoothing_constant)**niter)

    losses.append(est_loss)
    if (e + 1) % display_step == 0:
        print("Epoch:", '%04d' % (e), "cost=", "{:.9f}".format(curr_loss), "W=", W.asnumpy()[0], "b=", b.asnumpy()[0])


I am working on the file "01_TF_basics_and_linear_regression/tensorflow_basic.ipynb".
The first task is:
Basic functions
There are some basic functions we need to know. Those functions will be used in next tutorial 3. feed_forward_neural_network.
* tf.argmax
* tf.reduce_sum
* tf.equal
* tf.random_normal
tf.argmax
tf.argmax(input, dimension, name=None) returns the index with the largest value across dimensions of a tensor.
Can you write Python code for it?

a = tf.constant([[1, 6, 5], [2, 3, 4]])
print a.eval()
print "argmax over axis 0"
print tf.argmax(a, 0).eval()
print "argmax over axis 1"
print tf.argmax(a, 1).eval()


I am working on the file "01_TF_basics_and_linear_regression/tensorflow_basic.ipynb".
The first task is:
Constants
We can use the help function to get an annotation about any function. Just type help(tf.consant) on the below cell and run it.
It will print out constant(value, dtype=None, shape=None, name='Const') at the top. Value of tensor constant can be scalar, matrix or tensor (more than 2-dimensional matrix). Also, you can get a shape of tensor by running tensor.get_shape().as_list().

tensor.get_shape()
tensor.get_shape().as_list()
Can you write Python code for it?

a = tf.constant([[1, 2, 3], [4, 5, 6]], dtype=tf.float32, name='a')
print a.eval()
print "shape: ", a.get_shape(), ",type: ", type(a.get_shape())
print "shape: ", a.get_shape().as_list(), ",type: ", type(a.get_shape().as_list())   # this is more useful


I am working on the file "01_TF_basics_and_linear_regression/tensorflow_basic.ipynb".
The first task is:
Interactive session
Interactive session is a TensorFlow session for use in interactive contexts, such as a shell. The only difference with a regular Session is that an Interactive session installs itself as the default session on construction. The methods Tensor.eval() and Operation.run() will use that session to run ops.This is convenient in interactive shells and IPython notebooks, as it avoids having to pass an explicit Session object to run ops.
Can you write Python code for it?

sess = tf.InteractiveSession()

print a.eval()    # simple usage


I am working on the file "01_TF_basics_and_linear_regression/tensorflow_basic.ipynb".
The first task is:
Place holder
TensorFlow provides a placeholder operation that must be fed with data on execution. If you want to get more details about placeholder, please see here.
Can you write Python code for it?

x = tf.placeholder(tf.int16)
y = tf.placeholder(tf.int16)

add = tf.add(x, y)
mul = tf.mul(x, y)

# Launch default graph.
print "2 + 3 = %d" % sess.run(add, feed_dict={x: 2, y: 3})
print "3 x 4 = %d" % sess.run(mul, feed_dict={x: 3, y: 4})


I am working on the file "01_TF_basics_and_linear_regression/tensorflow_basic.ipynb".
The first task is:
Sharing variables
TensorFlow provides several classes and operations that you can use to create variables contingent on certain conditions.
* tf.get_variable
* tf.variable_scope
* reuse_variables
tf.get_variable
tf.get_variable(name, shape=None, dtype=None, initializer=None, trainable=True) is used to get or create a variable instead of a direct call to tf.Variable. It uses an initializer instead of passing the value directly, as in tf.Variable. An initializer is a function that takes the shape and provides a tensor with that shape. Here are some initializers available in TensorFlow:

tf.constant_initializer(value) initializes everything to the provided value,
tf.random_uniform_initializer(a, b) initializes uniformly from [a, b],
tf.random_normal_initializer(mean, stddev) initializes from the normal distribution with the given mean and standard deviation.
Can you write Python code for it?

my_initializer = tf.random_normal_initializer(mean=0, stddev=0.1)
v = tf.get_variable('v', shape=[2, 3], initializer=my_initializer)
tf.initialize_all_variables().run()
print v.eval()


I am working on the file "01_TF_basics_and_linear_regression/tensorflow_basic.ipynb".
The first task is:
Variables
When we train a model, we use variables to hold and update parameters. Variables are in-memory buffers containing tensors. They must be explicitly initialized and can be saved to disk during and after training. we can later restore saved values to exercise or analyze the model.

tf.Variable
tf.Tensor.name
tf.all_variables

tf.Variable
tf.Variable(initial_value=None, trainable=True, name=None, variable_def=None, dtype=None) creates a new variable with value initial_value.
The new variable is added to the graph collections listed in collections, which defaults to [GraphKeys.VARIABLES]. If trainable is true, the variable is also added to the graph collection GraphKeys.TRAINABLE_VARIABLES.
Can you write Python code for it?

# variable will be initialized with normal distribution
var = tf.Variable(tf.random_normal([3], stddev=0.1), name='var')
print var.name
tf.initialize_all_variables().run()
print var.eval()


I am working on the file "01_TF_basics_and_linear_regression/tensorflow_basic.ipynb".
The first task is:
reuse_variables
Note that you should run the cell above only once. If you run the code above more than once, an error message will be printed out: "ValueError: Variable layer1/v already exists, disallowed.". This is because we used tf.get_variable above, and this function doesn't allow creating variables with the existing names. We can solve this problem by using scope.reuse_variables() to get preivously created variables instead of creating new ones.
Can you write Python code for it?

with tf.variable_scope('layer1', reuse=True):
    w = tf.get_variable('v')   # Unlike above, we don't need to specify shape and initializer
    print w.name

# or

with tf.variable_scope('layer1') as scope:
    scope.reuse_variables()
    w = tf.get_variable('v')
    print w.name


I am working on the file "01_TF_basics_and_linear_regression/tensorflow_basic.ipynb".
The first task is:
tf.equal
tf.equal(x, y, name=None) returns the truth value of (x == y) element-wise. Note that tf.equal supports broadcasting. For more about broadcasting, please see here.
Can you write Python code for it?

a = tf.constant([[1, 0, 0], [0, 1, 1]])
print a.eval()
print "Equal to 1?"
print tf.equal(a, 1).eval()
print "Not equal to 1?"
print tf.not_equal(a, 1).eval()


I am working on the file "01_TF_basics_and_linear_regression/tensorflow_basic.ipynb".
The first task is:
tf.reduce_sum
tf.reduce_sum(input_tensor, reduction_indices=None, keep_dims=False, name=None) computes the sum of elements across dimensions of a tensor. Unless keep_dims is true, the rank of the tensor is reduced by 1 for each entry in reduction_indices. If keep_dims is true, the reduced dimensions are retained with length 1. If reduction_indices has no entries, all dimensions are reduced, and a tensor with a single element is returned
Can you write Python code for it?

a = tf.constant([[1, 1, 1], [2, 2, 2]])
print a.eval()
print "reduce_sum over entire matrix"
print tf.reduce_sum(a).eval()
print "reduce_sum over axis 0"
print tf.reduce_sum(a, 0).eval()
print "reduce_sum over axis 0 + keep dimensions"
print tf.reduce_sum(a, 0, keep_dims=True).eval()
print "reduce_sum over axis 1"
print tf.reduce_sum(a, 1).eval()
print "reduce_sum over axis 1 + keep dimensions"
print tf.reduce_sum(a, 1, keep_dims=True).eval()


I am working on the file "01_TF_basics_and_linear_regression/tensorflow_basic.ipynb".
The first task is:
tf.variable_scope
tf.variable_scope(scope_name) manages namespaces for names passed to tf.get_variable.
Can you write Python code for it?

with tf.variable_scope('layer1'):
    w = tf.get_variable('v', shape=[2, 3], initializer=my_initializer)
    print w.name

with tf.variable_scope('layer2'):
    w = tf.get_variable('v', shape=[2, 3], initializer=my_initializer)
    print w.name


I am working on the file "01_Variables.ipynb".
The first task is:
session이 없는데 만약에 그래프를 계속 추가하면 어떻게 될까요?
y, y1, y2를 계속추가하면서 각변수를 backward시키는 것과
z = y + y1 + y2를 만들어서 z만을 backward시키는 것과 동일합니다.
Can you write Python code for it?

import torch
from torch.autograd import Variable

x = Variable(torch.Tensor([1, 2]), requires_grad = True)
y = (x*x).sum()
y1 = (x*x).sum()

y.backward()
y1.backward()

print(x.grad)

import torch
from torch.autograd import Variable

x = Variable(torch.Tensor([1, 2]), requires_grad = True)
y = (x*x).sum()
y1 = (x*x).sum()

z = y + y1
z.backward()

print(x.grad)


I am working on the file "01_Variables.ipynb".
The first task is:
정말 간단합니다.

변수가 x1, x2 두개인 출력값인 y인 경우는 어떻게 될까요?
y= (x1^2) + (x2^2)의 미분값을 구해보도록 하겠습니다.
dy/dx1 = 2(x1)이 dy/dx2 = 2(x2)입니다.
x1, x2 가 1, 2이면 각각 2와 4가 됩니다.
Can you write Python code for it?

import torch
from torch.autograd import Variable

x1 = Variable(torch.Tensor([1]), requires_grad = True)
x2 = Variable(torch.Tensor([2]), requires_grad = True)

y = x1*x1 + x2*x2
y.backward()
print('dy/dx1', x1.grad)
print('dy/dx2', x2.grad)


I am working on the file "01_Variables.ipynb".
The first task is:
정말 간단합니다.

이제 y=x*x의 미분값을 구해보도록 하겠습니다.
y미분값은 dy/dx = 2x 입니다.x는 1이니 미분값은 2x, 즉 2입니다.
(표기의 용이성때문에, 여기서 d는 전미분이 아니라 편미분 기호입니다.)
Can you write Python code for it?

import torch
from torch.autograd import Variable

x = Variable(torch.Tensor([1]), requires_grad = True)
y = x*x

y.backward()

print('x = ', x)
print('dy/dx', x.grad)


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
A partir de números aleatorios
Los números aleatorios son usados en muchos problemas científicos. En la práctica las computadoras son solo capaces de generar números pseudo-aleatorios. Python usa un algortimo llamado Mersenne Twister para generar números pseudo-aleatorios. Este algorítmo es más que suficiente para fines científicos, pero no es útil en caso que necesitemos números pseudo-aleatorios para usar en criptografía. Resumiendo para nuestros fines podemos asumir que los números pseudo-aleatorios que generaremos a lo largo del curso son realmente números aleatorios.
Todas las rutinas de NumPy para generar números aleatorios viven dentro del módulo random.
La función mas simple es rand. Esta función crea un arreglo de números en el intervalo [0, 1). Dentro de ese intervalo los números son equiprobables, es decir es una distribución uniforme. El argumento de rand son las dimensiones del arreglo resultante.
Can you write Python code for it?

np.random.rand(2, 5)  # arreglo con forma (2, 5)


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
A partir de un rango numérico
Otra forma de crear arreglos es usando rangos. Por ejemplo podemos crear un arreglo conteniendo números igualmente espaciados en el intervalo [desde, hasta), usando arange.
Can you write Python code for it?

np.arange(0, 10, 1)  # desde, hasta(sin incluir), paso (el paso es opcional!)


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Aprovechando lo que hemos aprendido hasta el momento generemos un gráfico que muestre la diferencia entre media y mediana
Can you write Python code for it?

plt.hist(x, bins=20)
media = np.mean(x)
mediana = np.median(x)

plt.axvline(media, ymax=.9, c='C1', lw='3', label='{:.2f} media'.format(media))
plt.axvline(mediana, ymax=.9, c='C3', lw='3', label='{:.2f} mediana'.format(mediana))
plt.legend(fontsize=14);


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Como se ve en el ejemplo anterior vectorizar permite omitir. Esta capacidad de
vectorizar código no está restringida a operadores matemáticos como la potenciación **, funciona con otros operadores y con una gran cantidad de funciones. Por ejemplo
Can you write Python code for it?

np.sqrt(M)

np.log(M)  # 0j0 log es logaritmo natural


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Creando arreglos con ceros y unos
En Python es común crear una lista vacía que luego se llena de elementos en un loop. En NumPy es común crear un arreglo del tamaño necesario y luego llenarlo de valores. Para estas situaciones resulta conveniente tener a mano funciones que crean arreglos con ceros o unos.
Can you write Python code for it?

np.zeros((2, 4))

np.ones((2, 4))

np.zeros_like(M)  # noten que los ceros no tienen ".", es decir son enteros, ¿Por qué?

np.full((2, 4), 42.)


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Creando arreglos con ceros y unos
Existen al menos un par de funciones para crear arreglos de NumPy desde archivos, la más versatil de ellas es genfromtxt:
Can you write Python code for it?

datos = np.genfromtxt('datos/microbioma.csv', delimiter=','
                      , skip_header=1, usecols=(1,2,3,4), dtype='int')

datos.shape

datos[:4]


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Cuando tenemos una gran cantidad de datos los puntos  de un scatter plot empiezan a superponerse y puede que ciertos patrones pasen desapercibidos. En estos casos puede ser conveniente agrupar los datos de alguna forma en vez de mirar los datos en crudo.
El siguiente gráfico usa un kernel density estimation tanto para las distribuciones marginales como para la distribución conjunta.
Can you write Python code for it?

sns.jointplot(x, y,  kind='kde', stat_func=None);


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Cuantil
Además de la varianza existen otras formas de describir la dispersión de los datos. Una de ellas es el rango. Es decir la diferencia entre el valor más grande y el más chico en un conjunto de datos. Un problema con el rango es que es muy sensible a los valores extremos, después de todo se define como la resta de los dos valores más extremos. Una alternativa es calcular un rango pero truncado, es decir dejando de lado valores hacia ambos extremos. Esto se puede hacer con los cuantiles.
Los cuantiles son puntos de corte que dividen al conjunto de datos en grupos de igual tamaño. Existen varios nombres para los cuantiles según la cantidad de divisiones que nos interesen.

Los cuartiles son los tres puntos que dividen a la distribución en 4 partes iguales, se corresponden con los cuantiles 0.25, 0.50 y 0.75.
Los quintiles dividen a la distribución en cinco partes (corresponden a los cuantiles 0.20, 0.40, 0.60 y 0.80).
Los deciles dividen a la distribución en diez partes.
Los percentiles dividen a la distribución en cien partes.
La mediana es el percentil 50 y el cuartil 0.5.

En Python el cálculo de estos estadísticos puede realizarse fácilmente usando funciones predefinidas en NumPy.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Dado que en Python 0 evalua como False y 1 como True, una forma de crear un arreglo de booleanos es:
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
De forma similar randn (noten la n al final) devuelve muestras a partir de la distribución normal estándar (media = 0, desviación estándar = 1), según las dimensiones que especifiquemos.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Distinto es asignar la rebanada a una variable y luego modificar esa variable:
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
El mensaje de error nos dice que NumPy no sabe como hacer para encajar las dimensiones de estos dos  arreglos. Considero que este es un error múy razonable ya que no es del todo claro como NumPy podría hacer la operación que le pedidos que haga, además creo que el error es bastante transparente ¿Opinan igual?
Más detalles sobre broadcasting aqui.
Comparaciones y máscaras de booleanos
Así como es posible sumar un número a un arreglo, es posible hacer comparaciones elemento-a-elemento. Por ejemplo podemos preguntar que valores de M son mayores a 3, el resultado será una array con booleanos.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
El shape nos indica la candidad de elemenos en cada eje (o axis). En 2 dimensiones podemos pensarlo como el número de (filas, columnas).
Tambien podemos preguntarle a un array cual es su dimensión:
Can you write Python code for it?

v.ndim  # v es unidimensional

M.ndim  # M es bidimensional


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
En ambos casos lo que está sucediendo es como si antes de realizar la suma extendieramos una de las partes hasta que las dimensiones coincidan, por ejemplo repetir 3 veces el número 2 o tres veces el vector b.
El broadcasting no funciona para cualquier par de arreglos. La siguiente operación funciona:
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
En la primer y segunda línea estamos "generando" datos.
En la tercer línea decimos que queremos generar un gráfico del tipo plot (ya veremos que los hay de otros tipos), donde graficaremos x vs y.
En la cuarta y quinta línea agregamos rótulos a los ejes. En la quinta línea usamos la misma notación usada por $LaTeX$ para escribir fórmulas matemáticas. $LaTeX$ es un lenguaje para escribir textos que es muy usado en muchas disciplinas científicas para escribir papers, posters, diapositivas, libros, etc.

Veamos otro ejemplo:
Can you write Python code for it?

x = range(20)
y = [i ** 2 for i in x]
z = [i ** 1.8 for i in x]
plt.plot(x, y, label=r'$x^2$')
plt.plot(x, z, label=r'$x^{1.8}$')
plt.xlabel('x', fontsize=16)
plt.ylabel(r'$f(x)$', fontsize=16)
plt.legend(fontsize=16);


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Es importante notar que al tomar rebanadas NumPy NO genera un nuevo arreglo, sino una vista (view) del arreglo original. Por lo tanto si a una rebanada le asignamos un número, se lo estaremos asignando al arreglo original, como se puede ver en el siguiente ejemplo.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Existen muchos tipos de gráficos para representar datos. A continuación veremos cinco representaciones comunes para datos unidimensionales:

histogramas
kde plots
stripplot
box plots
violin plots

Histogramas
En un histograma se representa la frecuencia con la que aparecen los distintos valores en un conjunto de datos. Se utilizan barras contiguas para representar los datos. La superficie (y no la altura) de las barras es proporcional a la frecuencia de datos observados. Los datos son agrupados en bins, y suelen graficarse sin normalizar o normalizados. Normalizar implica que el área total del histograma suma 1. No hay que confundir los histogramas con los gráficos de barras que se utilizan para comparar valores discretos entre grupos, mientras que los histogramas se usan para representar distribuciones continuas.
Los histogramas son sensibles a la cantidad de bins que se usan. Si usamos unos pocos bins no lograremos capturar la estructura de los datos, si usamos demasiados bins no solo estaremos representando la estructura de los datos sino también el ruido. Esto se ve más claramente si probamos valores extremos, en el extremo inferior  tedríamos a todos los datos representados con una sola barra, en el superior una barra por cada dato.
Can you write Python code for it?

np.random.seed(440)
x = np.random.gamma(2, 10, size=1000)

plt.hist(x, bins=50, density=True, cumulative=False);  # probá cambiar los bins, y los demás argumentos.


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Funciones como sqrt o log, que operan sobre arreglos elemento-a-elemento se conocen como funciones universales (usualmente abreviadas como ufunc).
Una de las ventajas de usar ufuncs es que permite escribir código más breve. Otra ventaja es que los cómputos son más rápidos que usando loops de Python. Detrás de escena NumPy realiza las operaciones en un lenguaje como C o Fortran, por lo que hay una ganancia considerable en velocidad, respecto de código en Python puro. Además, el código usado por NumPy es código que suele estar optimizado gracias a los años de labor de programadores y científicos que colaboran con proyectos científicos.
Veamos otro ejemplo, como sumar todos los elementos de un arreglo.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Gráficos
Los gráficos ocupan un lugar central en la estadística moderna y en la ciencia de datos, ya sea en el análisis exploratorio de datos o en procesos de inferencia.
Existen varias bibliotecas para hacer gráficos en Python, Matplolib es una de las más usadas. La forma más común de importarla es:
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Gráficos de cajas o de bigotes (Box plot o Wisker-plot)
Los gráficos de caja están basados en cuartiles. La caja está delimitada (en sus bordes inferior y superior) por el primer y tercer cuartil, mientras que la línea dentro de la caja es el segundo cuartil (la mediana). Los bigotes pueden indicar varias medidas, por eso es siempre importante leer/escribir la leyenda o texto que acompaña a un boxplot, a veces se usa una desviación estándar, otras veces los percentiles 2 y 98, otras veces es una función del rango intercuartil (como en el siguiente gráfico). Los valores por fuera de los bigotes se suelen considerar como datos aberrantes (ver más adelante).
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Identificar correlaciones puede ser útil para entender como dos variables se relacionan y para predecir una a partir de la otra. Es por ello que muchas veces además de visualizar la relación entre variables se estiman modelos que ajustan a los datos. Como por ejemplo líneas rectas. En el tercer curso de este programa veremos como crear modelos lineales y no-lineales. Por ahora simplemente nos conformaremos con dejar que seaborn ajuste los datos a un recta por nosotros.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Incluso la siguiente operación es válida:
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Indexado y rebanado de arreglos
Los arreglos de NumPy, al igual que las listas, se pueden indexar y se pueden rebanar (slices). Por ejemplo la siguiente sintáxis funciona de la misma manera sin importar si M es un arreglo o una lista.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Kernel Density plot
Este tipo de gráfico es similar a un histograma, pero en vez de usar un número discreto de bins para representar una distribución se usa una curva suave y continua.
Un gráfico KDE se dibuja de la siguiente forma: se reemplaza cada dato por una distribución Gaussiana y luego se suman todas las Gaussianas. En vez de una distribución Gaussiana es posible usar otras distribuciones. El nombre genérico para esas distribuciones cuya suma se usa como aproximación de una función es el de kernel. La Gaussiana es uno de los kernels más usado.
De forma análoga a lo que sucede con los bins los KDE son sensibles a un parámetro llamado bandwith. Existen varias heurísticas (reglas empíricas que suelen funcionar bien en la práctica) para ajustar el bandwith de forma automática de acuerdo a los datos.
Es posible usar matplotlib para graficar un kde, pero no existe una función que lo haga de forma automática. Es decir es posible pero requiere de cierto trabajo. Lo mismo sucede con otros tipos de gráficos usados para analizar datos, es por ello que existe una biblioteca llamada Seaborn, la cual no es más que una colección de funciones escritas usando matplotlib.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
La primer línea es para decirle a la Notebook que los gráficos queden embebidos en la notebook (si no estuvieramos usando la notebook no escribiríamos esta línea).
La segunda línea es la forma estándar de importar matplotlib.
Veamos como hacer un gráfico sencillo.
Can you write Python code for it?

x = range(20)
y = [i ** 0.5 for i in x]
plt.plot(x, y)
plt.xlabel('x')
plt.ylabel(r'$\sqrt{x}$', rotation=0, labelpad=15);


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
La sintaxis usada para indexar y rebanar es una generalización de la usada para las listas de Python. Esta generalización facilita trabajar con arreglos de más de 1 dimensión. Trabajar con más de una o dos dimensiones puede ser un poco confuso, sobre todo al principio ¡Aunque no necesariamente solo al principio!
La siguiente expresión es válida para arreglos
Can you write Python code for it?

M[0, 1] # el elemento (0, 1) de M
        # o también la primer fila de M y de ella el segundo elemento


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Medidas de centralidad y dispersión (usando NumPy)
En el capítulo anterior vimos como usar Python para calcular la media, la mediana y la varianza. NumPy incluye funciones (y métodos) ya definidos para calcular estas cantidades. Para calcular la media de los valores en un array simplemente hacemos.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
NumPy permite vectorizar estas operaciones, es decir podemos calcular la raíz cuadrada de todos los elementos de un arreglo en una sola operación que se aplica a cada uno de los elementos del arreglo:
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
O sumar a lo largo de las filas:
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Otra función para crear rangos es linspace que devuelve numeros igualmente espacios en el intervalo [desde, hasta] (es decir incluyendo el hasta). Otra diferencia con arange es que no se especifica el paso sino la cantidad total de números que contendrá el arreglo.
Can you write Python code for it?

np.linspace(1, 10, 25)  # desde, hasta, elementos (elementos es opcional)


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Podemos ver que el arreglo b contiene 3 veces el número 2. El broadcasting hace que la siguiente operación también sea válida y brinde el mismo resultado que la celda anterior.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Seaborn provee de múltiples opciones para visualizar relaciones entre dos variables, varias de ellas están contenidas en la función joinplot. Esta función además de mostrar el gráfico de dispersión muestra las distribuciones marginales de $x$ e $y$.
Can you write Python code for it?

sns.jointplot(x, y,  kind='scatter', stat_func=None);


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Sin embargo esta expresión no es válida para listas.
¿Cuál es la expresión equivalente que funciona con listas?

En la siguiente celda tenemos un ejemplo de una expresión que es común a listas y arreglos.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Un arreglo tendrá tantos axis como dimensiones.
Broadcasting
Un característica que facilita vectorizar código es la capacidad de operar sobre arreglos que no tienen las mismas dimensiones. Esto se llama broadcasting y no es más que un conjunto de reglas que permiten aplicar operaciones binarias (suma, multiplicación etc) a arreglos de distinto tamaño.
Consideremos el siguiente ejemplo donde sumamos dos arreglos, elemento a elemento.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Un efecto similar puede lograrse con los histrogramas si solo graficamos el contorno de los mismos (histtype='step') o graficando histogramas semitranspartes(alpha menor a 1). Pero en general el efecto no es tan claro como cuando se usan KDEs.
Intenta reproducir el gráfico anterior pero usando histogramas

Stripplot
Este tipo de gráfico sirve para visualizar un conjunto de datos donde una variable es métrica y las demás son categóricas.
Para visualizarlos podemos usar la función stripplot de seaborn (un gráfico similar es el swarmplot). Los stripplot se suelen graficar agregado un poco de ruido (jitter en inglés) a lo largo del eje de las $x$, esto es simplemente un truco para facilitar la visualización de los puntos, que caso contrario caerían todos en una misma línea ya que las variables categóricas no tienen dispersión.
Puede ser útil en si mismo o puede ser usado superpuesto sobre un boxplot o violinplot (ver más adelante).
Can you write Python code for it?

y0 = np.random.normal(0, 10, size=42)
y1 = np.random.normal(-1, 10, size=50)
sns.stripplot(data=[y0, y1], jitter=True);


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Una alternativa para evitar que algunos puntos opaquen al resto, en un gráfico de dispersión común, es hacer los puntos semitransparentes. En matplotlib la transparencia de los objetos es controlada mediante un parámetro llamado alpha que va entre 0 y 1. Este es un buen momento para volver algunas celdas atrás y ver como este y otros parámetros pueden ser usados para modificar las gráficas realizadas.
Correlación
Al trabajar con dos variables es común preguntarse por la relación entre ellas. Decimos que dos variables están relacionadas si una provee información sobre la otra. Si en cambio una variable no ofrece información sobre otra decimos que son independientes.
La correlación es una medida de la dependencia de dos variables. Existen varios coeficientes de correlación el más comunmente usado es el coeficiente de correlación de Pearson. Este coeficiente solo sirve para medir relaciones lineales entre variables. El coeficiente de correlación de Pearson es el resultado de dividir la covarianza de las dos variables ($E[(\bar x - x)(\bar y - y)]$) por el producto de sus desviaciones estándar:
$$\rho_{(x,y)}={E[(\bar x - x)(\bar y - y)] \over \sigma_x\sigma_y}$$
En palabras (que puede ser más oscuro que en fórmulas), el coeficiente de correlación de Pearson indica como cambia una variable al cambiar la otra respecto de la variación intrínseca de ambas variables.
¿Por qué usar el coeficiente de Pearson y no directamente la covarianza? Fundamentalmente por que la interpretación es más simple. Al dividir por el producto de las varianzas estámos estandarizando la covarianza, obteniendo un coeficiente que no tiene dimensión y que solo puede variar entre -1 y 1 sin importar las unidades de nuestras variables.
La función joinplot, que vimos en el apartado anterior, por defecto nos devuelve el valor del coeficiente de correlación de Person, junto con un valor p cuyo significado estudiaremos en el capítulo 4.
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Una forma alternativa es usar el método .mean()
Can you write Python code for it?

v.mean()

print('varianza {:.2f}'.format(np.var(v)))
print('desviación estándar {:.2f}'.format(np.std(v)))


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Usando Seaborn, podemos hacer un kde de forma muy simple
Can you write Python code for it?

sns.kdeplot(x);  #también ver la función sns.distplot()


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Usando matplotlib podemos graficar ambas variables usando la función scatter
Can you write Python code for it?

plt.scatter(x, y, alpha=1)
plt.xlabel('x')
plt.ylabel('y');


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
Y este es un ejemplo de rebanado que funciona con arreglos, pero no con listas.
Can you write Python code for it?

M[1,:]  # solo la fila 1 (o la fila 1 en el primer axis y todo en el resto de los axis)


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
mientras que la siguiente dará  un error
Can you write Python code for it?


I am working on the file "01_arreglos_graficos_y_2D.ipynb".
The first task is:
o incluso combinando arreglos, como:
Can you write Python code for it?


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
10. Summarizing dataframes
Pandas offers several straightforward summarization functions
Can you write Python code for it?

scratch_df.mean()

scratch_df.mode()

scratch_df.describe()


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
11. By group processing
Use Pandas groupby() to create groups for subsequent processing
Can you write Python code for it?

# use summary function size() on groups created by groupby()
counts = scratch_df.groupby('char1').size()
plt.figure()
counts.plot.bar(title='Frequency of char1 values (Histogram of char1)')

# groupby the values of more than one variable
group_means = scratch_df.groupby(['char1', 'char2']).mean()
group_means


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
3. Generating a sample data set
Set the number of rows and columns for the sample data set
Can you write Python code for it?


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
4. Plotting variables in a dataframe
Pandas has several builtin plotting utilities
Use Pandas hist() method to plot a histogram of numeric1
Can you write Python code for it?

# Pandas alllows slicing by dataframes index using ix[]
# ix[:, 0] means all rows of the 0th column - or numeric1
scratch_df.ix[:, 0].plot.hist(title='Histogram of Numeric1')


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
5. Subsetting Pandas dataframes
By columns
Subsetting by index
Can you write Python code for it?

# one column returns a Pandas series
# a Pandas series is like a single column vector
scratch_df.iloc[:, 0].head()

type(scratch_df.iloc[:, 0])

# more than one columns makes a dataframe
# iloc enables location by index
scratch_df.iloc[:, 0:2].head()

type(scratch_df.iloc[:, 0:2])


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
6. Updating the dataframe
Can you write Python code for it?

# must use .copy() or this will be a symbolic link
scratch_df2 = scratch_df.copy()

# Pandas supports in place overwrites of data
# overwrite last 500 rows of char1 with ZZZZZZZZ
scratch_df2.loc[500:, 'char1'] = 'ZZZZZZZZ'
scratch_df2.tail()

# iat[] allows for fast location of specific indices
scratch_df2.iat[0, 0] = 1000
scratch_df2.head()


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
7. Sorting the dataframe
Sort by values of one variable
Can you write Python code for it?


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
8. Adding data to the dataframe

Pandas concat() supports numerous types of joins and merges
Pandas merge() supports joins and merges using more SQL-like syntax
i.e. merge(left, right, on=)
Pandas append() supports stacking dataframes top-to-bottom
Can you write Python code for it?

# create a toy dataframe to join/merge onto scratch_df
scratch_df3 = scratch_df3.drop(['numeric1', 'numeric2'] , axis=1)
scratch_df3.columns = ['char3', 'char4']
scratch_df3.tail()

# default outer join on matching indices
# this will create 2000 row × 6 column dataset because indices are not in identical order
scratch_df4 = pd.concat([scratch_df, scratch_df3])
scratch_df4

# outer join on matching columns
# axis=1 specificies to join on matching columns
scratch_df5 = pd.concat([scratch_df, scratch_df3], axis=1)
scratch_df5.head()

scratch_df5.shape

# append
scratch_df6 = scratch_df.append(scratch_df)
scratch_df6.shape


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
Boolean subsetting
Can you write Python code for it?

scratch_df[scratch_df.numeric2 > 0].head()

scratch_df[scratch_df.char1 == 'AAAAAAAA'].head()

scratch_df[scratch_df.char1.isin(['AAAAAAAA', 'BBBBBBBB'])].head()

scratch_df[scratch_df.numeric2 > 0].loc[5:10, 'char2']


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
By rows
Subsetting by index
Can you write Python code for it?

scratch_df[0:3]

# Selecting by index
scratch_df.iloc[0:5, :]

# select by row label
# here index/key values 0:5 are returned
scratch_df.loc[0:5, :]


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
Create a list of text elements from which to sample
Can you write Python code for it?

# string.ascii_uppercase is a string constant of uppercase letters
print(string.ascii_uppercase)

# another list comprehension
# slice first seven letters of the string
text_draw = [(letter * 8) for letter in string.ascii_uppercase[:7]]
text_draw


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
Create a random numerical columns directly using numpy
The numerical columns will originally be a 2-D numpy array
Can you write Python code for it?

randoms = np.random.randn(n_rows, n_vars)
randoms[0:5]

type(randoms)


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
Create lists of strings that will become column names

Lists are a common data structure in Python
Lists are surrounded by square brackets [] and contain different data types as list elements
Lists can be created by a speficic type Pythonic syntax, called list comprehensions
Lists in Python are indexed from 0, unlike SAS or R
Lists in Python, and other data structures, can be sliced using numeric indices
Can you write Python code for it?

# list comprehension
# str() converts to string
# range() creates a list of values from arg1 to arg2
num_col_names = ['numeric' + str(i+1) for i in range(0, n_vars)]
num_col_names

type(num_col_names) # type() can be used to determine the class of an object in Python


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
Create random character columns as a Pandas dataframe

Use a numpy sampling function choice() to generate a 2-D numpy array of random text values
Create Pandas dataframe from numpy 2-D array
Can you write Python code for it?

char_cols = pd.DataFrame(np.random.choice(text_draw, (n_rows, n_vars)),
                         columns=char_col_names)
char_cols.head()


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
Often, instead of simply transposing, a data set will need to be reformatted in a melt/stack - column split - cast action described in Hadley Wickham's Tidy Data:
https://www.jstatsoft.org/article/view/v059i10
See the stack and unstack methods for Pandas dataframes
Exporting and importing a dataframe

Many to_* methods available for exporting dataframes to other formats
Many read_* methods available for creating dataframes from other formats
Can you write Python code for it?

# export to csv
scratch_df.to_csv('scratch.csv')

# import from csv
scratch_df7 = pd.read_csv('scratch.csv')


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
Sort by the value of the dataframe index
Can you write Python code for it?


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
Sort by values of multiple variables and specify sort order
Can you write Python code for it?

scratch_df3 = scratch_df2.sort_values(by=['char1', 'numeric1'],
                            ascending=[False, True]).copy()
scratch_df3.head()


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
Subsetting by variable name
Can you write Python code for it?

scratch_df['numeric1'].head()

scratch_df.numeric1.head()

# loc[] allows for location by column or row label
scratch_df.loc[:, 'numeric1'].head()

# loc can accept lists as an input
scratch_df.loc[:, ['numeric1', 'numeric2']].head()


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
Use Pandas concat() function to join the numeric and character columns into a new dataframe
Can you write Python code for it?

scratch_df = pd.concat([num_cols, char_cols], axis=1)
scratch_df.head()


I am working on the file "01_basic_data_prep/src/notebooks/py/Py_Part_0_pandas_numpy.ipynb".
The first task is:
use Pandas scatter() method to plot numeric1 vs. numeric2
Can you write Python code for it?

scratch_df.plot.scatter(x='numeric1', y='numeric2',
                        title='Numeric1 vs. Numeric2')


I am working on the file "01_basics_phases_ex1.ipynb".
The first task is:
1.2)
Can you write Python code for it?

with tf.Session(graph=graph):
    c_val = c.eval()

c_val


I am working on the file "01_basics_phases_ex1.ipynb".
The first task is:
1.4)
Can you write Python code for it?


I am working on the file "01_basics_phases_ex1.ipynb".
The first task is:
Construction Phase
Can you write Python code for it?

>>> a = tf.constant(3)
>>> b = tf.constant(5)
>>> s = a + b

a

b

s

tf.get_default_graph()

>>> graph = tf.Graph()
>>> with graph.as_default():
...     a = tf.constant(3)
...     b = tf.constant(5)
...     s = a + b
...


I am working on the file "01_basics_phases_ex1.ipynb".
The first task is:
Exercise 1

1.1) Create a simple graph that calculates $ c = \exp(\sqrt 8 + 3) $.
Tip: TensorFlow's API documentation is available at:
https://www.tensorflow.org/versions/master/api_docs/python/
1.2) Now create a Session() and evaluate the operation that gives you the result of the equation above:
1.3) Create a graph that evaluates and prints both $ b = \sqrt 8 $ and $ c = \exp(\sqrt 8 + 3) $. Try to implement this in a way that only evaluates $ \sqrt 8 $ once.
1.4) The following code is needed to display TensorFlow graphs in Jupyter. Just run this cell then visualize your graph by calling show_graph(your graph):
Can you write Python code for it?

import numpy as np
from IPython.display import display, HTML

def strip_consts(graph_def, max_const_size=32):
    """Strip large constant values from graph_def."""
    strip_def = tf.GraphDef()
    for n0 in graph_def.node:
        n = strip_def.node.add()
        n.MergeFrom(n0)
        if n.op == 'Const':
            tensor = n.attr['value'].tensor
            size = len(tensor.tensor_content)
            if size > max_const_size:
                tensor.tensor_content = b"<stripped %d bytes>"%size
    return strip_def

def show_graph(graph_def=None, max_const_size=32):
    """Visualize TensorFlow graph."""
    graph_def = graph_def or tf.get_default_graph()
    if hasattr(graph_def, 'as_graph_def'):
        graph_def = graph_def.as_graph_def()
    strip_def = strip_consts(graph_def, max_const_size=max_const_size)
    code = """

        <link rel="import" href="https://tensorboard.appspot.com/tf-graph-basic.build.html" onload=load()>

          <tf-graph-basic id="{id}"></tf-graph-basic>

    """.format(data=repr(str(strip_def)), id='graph'+str(np.random.rand()))

    iframe = """

    """.format(code.replace('"', '"'))
    display(HTML(iframe))


I am working on the file "01_basics_phases_ex1.ipynb".
The first task is:
Important: the following implementation gives the right result, but it runs the graph twice, once to evaluate b, and once to evaluate c.  Since c depends on b, it means that b will be evaluated twice. Not what we wanted.
Can you write Python code for it?

# WRONG!
with tf.Session(graph=graph):
    b_val = b.eval()  # evaluates b
    c_val = c.eval()  # evaluates c, which means evaluating b again!

b_val

c_val


I am working on the file "01_basics_phases_ex1.ipynb".
The first task is:
Try not to peek at the solution below before you have done the exercise! :)

Exercise 1 - Solution
1.1)
Can you write Python code for it?

graph = tf.Graph()
with graph.as_default():
    c = tf.exp(tf.add(tf.sqrt(tf.constant(8.)), tf.constant(3.)))
    # or simply...
    c = tf.exp(tf.sqrt(8.) + 3.)


I am working on the file "01_check_data.ipynb".
The first task is:
データセットの読み込み
Can you write Python code for it?

X_train, y_train = load_mnist('./data/fashion', kind='train')
X_test, y_test = load_mnist('./data/fashion', kind='t10k')

#訓練画像
train_no = 0
outImg = Image.fromarray(X_train[train_no].reshape((28,28))*255).convert("RGB")
outImg.save("out/train.png")

#テスト画像
test_no = 0
outImg = Image.fromarray(X_test[test_no].reshape((28,28))).convert("RGB")
outImg.save("out/test.png")


I am working on the file "01_check_data.ipynb".
The first task is:
データ読み込み用の関数
Can you write Python code for it?

def load_mnist(path, kind='train'):
    import os
    import gzip
    import numpy as np

    """Load MNIST data from `path`"""
    labels_path = os.path.join(path,
                               '%s-labels-idx1-ubyte.gz'
                               % kind)
    images_path = os.path.join(path,
                               '%s-images-idx3-ubyte.gz'
                               % kind)

    with gzip.open(labels_path, 'rb') as lbpath:
        labels = np.frombuffer(lbpath.read(), dtype=np.uint8,
                               offset=8)

    with gzip.open(images_path, 'rb') as imgpath:
        images = np.frombuffer(imgpath.read(), dtype=np.uint8,
                               offset=16).reshape(len(labels), 784)

    return images, labels


I am working on the file "01_collect-data.ipynb".
The first task is:
ProPublica Campaign Finance API
https://propublica.github.io/campaign-finance-api-docs/#candidates
Can you write Python code for it?

# set key
key="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

# set base url
base_url="https://api.propublica.org/campaign-finance/v1/"

# set headers
headers = {'X-API-Key': key}

# set url parameters
cycle = "2014/"
method = "candidates/"
file_format = ".json"

# create a list of FEC IDs from http://www.fec.gov/data/DataCatalog.do to run the API request on more than one ID
fec_id_list = []
with open('fecid2014.txt') as file:
    for line in file:
        fec_id_list.append(line.strip())

# make request, build list of results for each FEC ID
data = []
for fec_id in fec_id_list:
    r = requests.get(base_url+cycle+method+fec_id+file_format, headers=headers)
    candidate = r.json()['results']
    data.append(candidate)
    time.sleep(3)
print(data)

# format data for export
data = [v for sublist in data for v in sublist]
data_keys = data[0].keys()

# export to csv
import csv
with open('ppcampfin.csv', 'w') as file:
    dict_writer = csv.DictWriter(file, data_keys)
    dict_writer.writeheader()
    dict_writer.writerows(data)


I am working on the file "01_crawling/01_watcha_api_structure.ipynb".
The first task is:
'cards'안에 원하는 정보존재

24개의 영화정보 존재
Can you write Python code for it?

watcha_list = watcha_dict.get('cards')

print(len(watcha_list))

watcha_list

# 첫번째 영화(watcha_list[0]) 안에 있는
# 'items' key 안에 있는
# 'item' key 로 들어감
watcha_list[0].get('items')[0].get('item')

watcha_list[23].get('items')[0].get('item').get('owner_action').get('rating')