I am working on the file "3.PCA.ipynb".
The first task is:
From 784 Dimensions to 2 Dimensions
Can you write Python code for it?

digits = pd.read_csv('data/digits.csv')

digits.head()

digits.shape

digitsX = digits.iloc[:,1:785]

digitsX.head()

pca = PCA(n_components=2)

pca.fit(digitsX)

digits_trans = pca.transform(digitsX)

digits_trans

plt.scatter(digits_trans[:,0], digits_trans[:,1], c = digits.num,
            s = 20, alpha = 0.8, cmap = plt.cm.viridis)
plt.xlabel('Principal Component 1')
plt.ylabel('Principal Component 2')


I am working on the file "3.PCA.ipynb".
The first task is:
How do you select which 2 axis to choose?
Sorting the Eigenvalues and Eigenvectors
In order to decide which eigenvector(s) can dropped without losing too much information for the construction of lower-dimensional subspace, we need to inspect the corresponding eigenvalues: The eigenvectors with the lowest eigenvalues bear the least information about the distribution of the data; those are the ones can be dropped.
Can you write Python code for it?

# Make a list of (eigenvalue, eigenvector) tuples
eig_pairs = [(np.abs(eig_vals[i]), eig_vecs[:,i]) for i in range(len(eig_vals))]

# Sort the (eigenvalue, eigenvector) tuples from high to low
eig_pairs.sort(key=lambda x: x[0], reverse=True)

# Visually confirm that the list is correctly sorted by decreasing eigenvalues
print('Eigenvalues in descending order:')
for i in eig_pairs:
    print(i[0])


I am working on the file "3.PCA.ipynb".
The first task is:
PCA using sklearn
Can you write Python code for it?

from sklearn.decomposition import PCA

pca = PCA(n_components=2)

pca.fit(X_std)

X_proj_sklearn = pca.transform(X_std)

fig, ax = plt.subplots(figsize=(6, 6))
plt.scatter(X_proj_sklearn[:,0], X_proj_sklearn[:,1], c = df.type,
            s = 100, cmap = plt.cm.viridis)
plt.xlabel('Principal Component 1')
plt.ylabel('Principal Component 2')

pca.explained_variance_


I am working on the file "3.PCA.ipynb".
The first task is:
Preprocessing - brand, price, kmpl, bhp
Can you write Python code for it?

pop = pop.drop(['model'], axis = 1)

from sklearn import preprocessing
le = preprocessing.LabelEncoder()
df = pop.apply(le.fit_transform)

df.head()

g = sns.PairGrid(df, hue = 'type')
g.map_diag(plt.hist)
g.map_offdiag(plt.scatter, alpha = 0.8)


I am working on the file "3.PCA.ipynb".
The first task is:
So now this is the symetric $A$ matrix we are trying to solve
$$ Ax = \lambda x $$
where
$$ A = \begin{bmatrix} 1.01 & -0.92 \ -0.92 & 1.01 \end{bmatrix} $$
Get Eigen-vectors and Eigen-values
Lets get the eigen-vectors for this matrix
Can you write Python code for it?

eigen_val_2var, eigen_vec_2var = np.linalg.eig(cov_mat_2var)

eigen_val_2var

eigen_vec_2var

eigen_vec_2var[1].dot(eigen_vec_2var[0])


I am working on the file "3.PCA.ipynb".
The first task is:
Standardizing
Can you write Python code for it?

X = df.iloc[:,:4]

from sklearn.preprocessing import StandardScaler
X_std = StandardScaler().fit_transform(X)


I am working on the file "3.PCA.ipynb".
The first task is:
Using PCA from SKlearn
Can you write Python code for it?

from sklearn.decomposition import PCA

pca = PCA(n_components=2)

pca.fit(X_std)

X_pca_proj = pca.transform(X_std)

plot2var_std(X_pca_proj, 'pca1', 'pca2')

pca.explained_variance_


I am working on the file "3.Recommender_system.ipynb".
The first task is:
Mentee should fill the array by giving a for each topic according to his preferences
Can you write Python code for it?

mentee1 = np.zeros(20)
mentee1[1] = 10
author_id = get_mentor([mentee1])

author_id

list(authorID_to_titles_stem.values())[author_id[0]]


I am working on the file "3.analysis/.ipynb_checkpoints/MacAndPC-checkpoint.ipynb".
The first task is:
Let's use boxplots to explore the windows and mac data we recorded so far.
use the command boxplot to plot them.
You can also combine the 2 datasets to place them in one plot using data = [windows,mac]
Can you write Python code for it?

data = [windows,mac]
boxplot(data)
xticks([1,2],['windows','mac'])
#save the plot to a file
savefig("boxplot.pdf")


I am working on the file "3.analysis/.ipynb_checkpoints/MacAndPC-checkpoint.ipynb".
The first task is:
Now Lets try to calculate the mean ... you can just use mean()
Can you write Python code for it?


I am working on the file "3.analysis/.ipynb_checkpoints/MacAndPC-checkpoint.ipynb".
The first task is:
Ok ... let's caluclate the means and plot the data.
now perform the t-test (two sided is best). What do you think?
what to do if we have more than 2 samples? (assumming we introduce a 3rd experimental setup where the menu bar is at the bottom of the screen) we need to use ANOVA (again assuming between-subject design, normal distributions etc.) Use the function stats.f_oneway
Can you write Python code for it?


I am working on the file "3.analysis/.ipynb_checkpoints/MacAndPC-checkpoint.ipynb".
The first task is:
Ok... doesn't look significant. So we should record more data ;)
Can you write Python code for it?


I am working on the file "3.analysis/.ipynb_checkpoints/MacAndPC-checkpoint.ipynb".
The first task is:
hmmm ... there seems to be a difference, but it's not so big. Let's use point plots to check the data.
Can you write Python code for it?


I am working on the file "3.analysis/.ipynb_checkpoints/MacAndPC-checkpoint.ipynb".
The first task is:
usually we don't define all data from the command prompt, but we read in files from disk.
Can you write Python code for it?

import pandas as pd
menu_data=pd.read_csv("./data/menu_all.csv")

menu_data.describe()

menu_data.boxplot()


I am working on the file "3.analysis/MacAndPC.ipynb".
The first task is:
Let's use boxplots to explore the windows and mac data we recorded so far.
use the command boxplot to plot them.
You can also combine the 2 datasets to place them in one plot using data = [windows,mac]
Can you write Python code for it?


I am working on the file "3.analysis/MacAndPC.ipynb".
The first task is:
Now Lets try to calculate the mean ... you can just use mean()
Can you write Python code for it?


I am working on the file "3.analysis/MacAndPC.ipynb".
The first task is:
hmm ... doesn't look siginificant. yet, just to make sure, let's apply a t-test. Remember t-tests are for comparing only 2 means with eachother NOT more (also assumptions are that samples are independent, normally distributed and the variance is the same!)
Can you write Python code for it?

from scipy.stats import ttest_ind
from scipy.stats import ttest_rel
import scipy.stats as stats
#onesided t-test
ttest_ind(mac,windows)

#two sided t-test
ttest_rel(mac,windows)


I am working on the file "3.analysis/MacAndPC.ipynb".
The first task is:
hmmm ... there seems to be a difference, but it's not so big. Let's use point plots to check the data.
Can you write Python code for it?


I am working on the file "3.analysis/MacAndPC.ipynb".
The first task is:
usually we don't define all data from the command prompt, but we read in files from disk.
Can you write Python code for it?


I am working on the file "3.analysis/explore.ipynb".
The first task is:
Anscombe's quartet
Let's take a look at some other data set (and actually import data from a file).
Can you write Python code for it?

import pandas as pd
aq=pd.read_csv('data/anscombesQuartet.csv')

aq

mean(aq['I_y'])


I am working on the file "3.analysis/explore.ipynb".
The first task is:
Now Lets try to calculate the mean ... you can just use mean()
we can also plot the raw data
Can you write Python code for it?


I am working on the file "3.analysis/explore.ipynb".
The first task is:
apply a t-test to check for significance
Can you write Python code for it?

from scipy.stats import ttest_ind
from scipy.stats import ttest_rel
import scipy.stats as stats
#onesided t-test
ttest_ind(mac,windows)
#two sided t-test

ttest_rel(mac,windows)


I am working on the file "3.analysis/explore.ipynb".
The first task is:
let's say we get more data
Can you write Python code for it?


I am working on the file "3/1-Clustering.ipynb".
The first task is:
Challenge
Look at the other clustering methods provided by SKLearn, and consider their use cases. Pick three to run on the sample iris dataset that you think will produce the most accurate clusters. Tune parameters and look at the different options to try and get your clusters as close to the ground truth as possible. This challenge hopes to help you familiarize yourself with the documentation that SKLearn provides and figure out the best clustering method given a problem to solve. We only covered two clustering models in depth to give you a taste of what clustering can do, but there many more clustering models out there all with their own optimal use cases.
Can you write Python code for it?

np.random.seed(5)

centers = [[1, 1], [-1, -1], [1, -1]]
iris = datasets.load_iris()
X = iris.data
y = iris.target

# TODO: choose three additional estimators here that will give the best results.
estimators = {'Hierarchical 3': AgglomerativeClustering(n_clusters=3),
              'K-Means 3': KMeans(n_clusters=3),
              'K-Means 7': KMeans(n_clusters=7)}

fignum = 1
for name, est in estimators.items():
    fig = plt.figure(fignum, figsize=(4, 3))
    plt.clf()
    ax = Axes3D(fig, rect=[0, 0, .95, 1], elev=48, azim=134)

    plt.cla()
    est.fit(X)
    labels = est.labels_

    ax.scatter(X[:, 3], X[:, 0], X[:, 2], c=labels.astype(np.float))

    ax.w_xaxis.set_ticklabels([])
    ax.w_yaxis.set_ticklabels([])
    ax.w_zaxis.set_ticklabels([])
    ax.set_xlabel('Petal width')
    ax.set_ylabel('Sepal length')
    ax.set_zlabel('Petal length')
    ax.set_title(name)
    fignum = fignum + 1

# Plot the ground truth
fig = plt.figure(fignum, figsize=(4, 3))
plt.clf()
ax = Axes3D(fig, rect=[0, 0, .95, 1], elev=48, azim=134)
plt.cla()

for name, label in [('Setosa', 0), ('Versicolour', 1), ('Virginica', 2)]:
    ax.text3D(X[y == label, 3].mean(),
              X[y == label, 0].mean() + 1.5,
              X[y == label, 2].mean(), name,
              horizontalalignment='center',
              bbox=dict(alpha=.5, edgecolor='w', facecolor='w'))
y = np.choose(y, [1, 2, 0]).astype(np.float)
ax.scatter(X[:, 3], X[:, 0], X[:, 2], c=y)

ax.w_xaxis.set_ticklabels([])
ax.w_yaxis.set_ticklabels([])
ax.w_zaxis.set_ticklabels([])
ax.set_xlabel('Petal width')
ax.set_ylabel('Sepal length')
ax.set_zlabel('Petal length')
ax.set_title('Ground Truth')
plt.show()


I am working on the file "3/1-Clustering.ipynb".
The first task is:
Hierarchical Clustering
Motivation
K-Means is one of the most common and simple clustering methods, but it has a couple of key limitations. First off, it is nondeterministic as it depends on the initial choice of cluster means, and Lloyd's Algorithm only arrives upon local minima rather than the global minimum. Furthermore, the algorithm requires you to decide what $k$ is, as we saw earlier. Finally, K-Means can be inflexible, as the only way cluster centers are derived is through the mean distance from all of the points assigned to the cluster. Because of this construction, K-Means doesn't perform well on clusters that are connected but not compact.
Hierarchical clustering solves many of these issues. The motivation between hierarchical clustering is building up clusters as a hierarchy. All objects start out in their individual groups, and at each level, the groups that are a certain distance apart are joined to form a larger group. A variety of different distance metrics can be used in building up these groups to result in different types of hierarchical clusters.

A dendrogram is a way of visualizing the groups as they are aggregated together in the hierarchy. As you can see, hierarchical clustering not only resolves some of the problems explained concerning K-Means - it also prevents a very nice way of representing the structure of the clusters, identifying subclusters within the clusters.
Application
Once again, we can use the SKLearn hierarchical clustering implementation, in the same way as we used clustering. However, there are many resources on the page documenting the different distance metrics that you can use.
Can you write Python code for it?

estimators = {'Hierarchical 3': AgglomerativeClustering(n_clusters=3)}

fignum = 1
for name, est in estimators.items():
    fig = plt.figure(fignum, figsize=(4, 3))
    plt.clf()
    ax = Axes3D(fig, rect=[0, 0, .95, 1], elev=48, azim=134)

    plt.cla()
    est.fit(X)
    labels = est.labels_

    ax.scatter(X[:, 3], X[:, 0], X[:, 2], c=labels.astype(np.float))

    ax.w_xaxis.set_ticklabels([])
    ax.w_yaxis.set_ticklabels([])
    ax.w_zaxis.set_ticklabels([])
    ax.set_xlabel('Petal width')
    ax.set_ylabel('Sepal length')
    ax.set_zlabel('Petal length')
    ax.set_title(name)
    fignum = fignum + 1

plt.show()


I am working on the file "3/1-Clustering.ipynb".
The first task is:
K-Means Clustering
Motivation
Given a set of objects, we specify that we want $k$ clusters. Each cluster has a mean representing it, and we assign each point to the clusters based on which cluster mean its closest to. For each point, the reconstruction error is defined to be its distance to its cluster mean. This gives us the total reconstruction error, the sum of all the individual reconstruction errors, as an error value that we want to minimize.
Formally, we have a set of object vectors ${x_n}{n = 1}^N$ and a set of $K$ cluster means ${\mu_k}{k = 1}^K$, where $x_n, \mu_k \in \mathbb{R}^D$. We represent each object's cluster assignment with the $K$ dimensional vector $r_n$, where $r_{nk} = 1$ if $x_n$ is in cluster $k$, and 0 otherwise. This gives us the individual reconstruction error
$$J_n(r_n, {\mu_k}) = \sum_{k = 1}^K r_{nk} \cdot |x_n - \mu_k|^2$$
and the total reconstruction error
$$J({r_n}, {\mu_k}) = \sum_{n = 1}^N \sum_{k = 1}^K r_{nk} \cdot |x_n - \mu_k|^2$$
As you can see, the reconstruction error on a set of objects is a function of assignments and means. How would you go about choosing the assignments ${r_n}$ and means ${\mu_k}$ that minimize the reconstruction error? Lloyd's Algorithm proposes a two step error minimization.
Step 1: minimize $J$ by updating the $r_n$, assigning each $x_n$ to its closest cluster mean.
Step 2: minimize $J$ by recalculting the $\mu_k$ to be the average over all vectors assigned to cluster $k$.
Repeating this process until the assignments do not change, the algorithm will converge upon a local minima. The algorithm can be optimized by starting with reasonable distributions for cluster centers rather than choosing randomly (K-Means ++) or adding the condition that the cluster mean must be a point (K-Medoids).
Application
SKLearn has a K-Means implementation that is documented here. In this example, we use K-Means to classify flowers into 3 classes based on their properties. Note that choosing the right $k$ is crucial. The true number of categories within the...
Can you write Python code for it?

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from sklearn.cluster import KMeans
from sklearn.cluster import AgglomerativeClustering
from sklearn import datasets

centers = [[1, 1], [-1, -1], [1, -1]]
iris = datasets.load_iris()
X = iris.data
y = iris.target

estimators = {'K-Means 3': KMeans(n_clusters=3),
              'K-Means 8': KMeans(n_clusters=7)}

fignum = 1
for name, est in estimators.items():
    fig = plt.figure(fignum, figsize=(4, 3))
    plt.clf()
    ax = Axes3D(fig, rect=[0, 0, .95, 1], elev=48, azim=134)

    plt.cla()
    est.fit(X)
    labels = est.labels_

    ax.scatter(X[:, 3], X[:, 0], X[:, 2], c=labels.astype(np.float))

    ax.w_xaxis.set_ticklabels([])
    ax.w_yaxis.set_ticklabels([])
    ax.w_zaxis.set_ticklabels([])
    ax.set_xlabel('Petal width')
    ax.set_ylabel('Sepal length')
    ax.set_zlabel('Petal length')
    ax.set_title(name)
    fignum = fignum + 1

plt.show()


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Batches
Implement get_batches to create batches of input and targets using int_text.  The batches should be a Numpy array with the shape (number of batches, 2, batch size, sequence length). Each batch contains two elements:
- The first element is a single batch of input with the shape [batch size, sequence length]
- The second element is a single batch of targets with the shape [batch size, sequence length]
If you can't fill the last batch with enough data, drop the last batch.
For exmple, get_batches([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], 2, 3) would return a Numpy array of the following:
```
[
  # First Batch
  [
    # Batch of Input
    [[ 1  2  3], [ 7  8  9]],
    # Batch of targets
    [[ 2  3  4], [ 8  9 10]]
  ],
# Second Batch
  [
    # Batch of Input
    [[ 4  5  6], [10 11 12]],
    # Batch of targets
    [[ 5  6  7], [11 12 13]]
  ]
]
```
Can you write Python code for it?

def get_batches(int_text, batch_size, seq_length):
    """
    Return batches of input and target
    :param int_text: Text with the words replaced by their ids
    :param batch_size: The size of batch
    :param seq_length: The length of sequence
    :return: Batches as a Numpy array
    """
    # TODO: Implement Function
    '''

    int_text=np.array(int_text)
    slice_size = batch_size * seq_length
    n_batches = int(len(int_text) / slice_size)
    batches=np.zeros((n_batches,2,batch_size,seq_length))
    x = int_text[: n_batches*slice_size]
    y = int_text[1: n_batches*slice_size + 1]
    x = np.split(x, batch_size *n_batches)
    y = np.split(y, batch_size *n_batches)
    for i in range(n_batches):
        outx=np.zeros((batch_size,seq_length))
        outy=np.zeros((batch_size,seq_length))
        for j in range(batch_size):
            outx[j] = x[i+j*n_batches]
            outy[j] = y[i+j*n_batches]
        batches[i][0]=outx
        batches[i][1]=outy
    return batches
    '''
    slice_size = int(len(int_text)/batch_size)
    X_train = np.asarray(int_text[:-1])
    y_train = np.asarray(int_text[1:])
    n_batches = int(slice_size/seq_length)

    # Drop the last few characters to make only full batches
    xdata = np.array(int_text[: n_batches * batch_size * seq_length])
    ydata = np.array(int_text[1: n_batches * batch_size * seq_length + 1])

    x_batches = np.split(xdata.reshape(batch_size, -1), n_batches, 1)
    y_batches = np.split(ydata.reshape(batch_size, -1), n_batches, 1)

    return np.array(list(zip(x_batches, y_batches)))

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_get_batches(get_batches)


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Build RNN
You created a RNN Cell in the get_init_cell() function.  Time to use the cell to create a RNN.
- Build the RNN using the tf.nn.dynamic_rnn()
 - Apply the name "final_state" to the final state using tf.identity()
Return the outputs and final_state state in the following tuple (Outputs, FinalState)
Can you write Python code for it?

def build_rnn(cell, inputs):
    """
    Create a RNN using a RNN Cell
    :param cell: RNN Cell
    :param inputs: Input text data
    :return: Tuple (Outputs, Final State)
    """
    # TODO: Implement Function
    outputs, final_state = tf.nn.dynamic_rnn(cell, inputs, dtype = tf.float32)
    final_state=tf.identity(final_state, name='final_state')
    return outputs, final_state

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_build_rnn(build_rnn)


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Build RNN Cell and Initialize
Stack one or more BasicLSTMCells in a MultiRNNCell.
- The Rnn size should be set using rnn_size
- Initalize Cell State using the MultiRNNCell's zero_state() function
    - Apply the name "initial_state" to the initial state using tf.identity()
Return the cell and initial state in the following tuple (Cell, InitialState)
Can you write Python code for it?

def get_init_cell(batch_size, rnn_size):
    """
    Create an RNN Cell and initialize it.
    :param batch_size: Size of batches
    :param rnn_size: Size of RNNs
    :return: Tuple (cell, initialize state)
    """
    #print(batch_size,rnn_size)
    # TODO: Implement Function

    num_layers = 2
    lstm = tf.contrib.rnn.BasicLSTMCell(rnn_size)

    #cell = tf.contrib.rnn.MultiRNNCell([lstm] * num_layers)

    keep_prob = 1 #0.5
    drop = tf.contrib.rnn.DropoutWrapper(lstm, output_keep_prob=keep_prob)
    cell = tf.contrib.rnn.MultiRNNCell([drop] * num_layers)

    initial_state = cell.zero_state(batch_size, tf.float32)
    initial_state = tf.identity(initial_state, name='initial_state')
    return cell, initial_state

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_get_init_cell(get_init_cell)


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Build the Neural Network
You'll build the components necessary to build a RNN by implementing the following functions below:
- get_inputs
- get_init_cell
- get_embed
- build_rnn
- build_nn
- get_batches
Check the Version of TensorFlow and Access to GPU
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
from distutils.version import LooseVersion
import warnings
import tensorflow as tf

# Check TensorFlow Version
assert LooseVersion(tf.__version__) >= LooseVersion('1.0'), 'Please use TensorFlow version 1.0 or newer'
print('TensorFlow Version: {}'.format(tf.__version__))

# Check for a GPU
if not tf.test.gpu_device_name():
    warnings.warn('No GPU found. Please use a GPU to train your neural network.')
else:
    print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Check Point
This is your first checkpoint. If you ever decide to come back to this notebook or have to restart the notebook, you can start from here. The preprocessed data has been saved to disk.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import helper
import numpy as np
import problem_unittests as tests

int_text, vocab_to_int, int_to_vocab, token_dict = helper.load_preprocess()


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Generate TV Script
This will generate the TV script for you.  Set gen_length to the length of TV script you want to generate.
Can you write Python code for it?

gen_length = 200
# homer_simpson, moe_szyslak, or Barney_Gumble
prime_word = 'moe_szyslak'

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
loaded_graph = tf.Graph()
with tf.Session(graph=loaded_graph) as sess:
    # Load saved model
    loader = tf.train.import_meta_graph(load_dir + '.meta')
    loader.restore(sess, load_dir)

    # Get Tensors from loaded model
    input_text, initial_state, final_state, probs = get_tensors(loaded_graph)

    # Sentences generation setup
    gen_sentences = [prime_word + ':']
    prev_state = sess.run(initial_state, {input_text: np.array([[1]])})

    # Generate sentences
    for n in range(gen_length):
        # Dynamic Input
        dyn_input = [[vocab_to_int[word] for word in gen_sentences[-seq_length:]]]
        dyn_seq_length = len(dyn_input[0])

        # Get Prediction
        probabilities, prev_state = sess.run(
            [probs, final_state],
            {input_text: dyn_input, initial_state: prev_state})

        pred_word = pick_word(probabilities[dyn_seq_length-1], int_to_vocab)

        gen_sentences.append(pred_word)

    # Remove tokens
    tv_script = ' '.join(gen_sentences)
    for key, token in token_dict.items():
        ending = ' ' if key in ['\n', '(', '"'] else ''
        tv_script = tv_script.replace(' ' + token.lower(), key)
    tv_script = tv_script.replace('\n ', '\n')
    tv_script = tv_script.replace('( ', '(')

    print(tv_script)


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Implement Generate Functions
Get Tensors
Get tensors from loaded_graph using the function get_tensor_by_name().  Get the tensors using the following names:
- "input:0"
- "initial_state:0"
- "final_state:0"
- "probs:0"
Return the tensors in the following tuple (InputTensor, InitialStateTensor, FinalStateTensor, ProbsTensor)
Can you write Python code for it?

def get_tensors(loaded_graph):
    """
    Get input, initial state, final state, and probabilities tensor from <loaded_graph>
    :param loaded_graph: TensorFlow graph loaded from file
    :return: Tuple (InputTensor, InitialStateTensor, FinalStateTensor, ProbsTensor)
    """
    # TODO: Implement Function
    InputTensor=loaded_graph.get_tensor_by_name("input:0")
    InitialStateTensor=loaded_graph.get_tensor_by_name("initial_state:0")
    FinalStateTensor=loaded_graph.get_tensor_by_name("final_state:0")
    ProbsTensor=loaded_graph.get_tensor_by_name("probs:0")
    return InputTensor,InitialStateTensor,FinalStateTensor,ProbsTensor

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_get_tensors(get_tensors)


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Input
Implement the get_inputs() function to create TF Placeholders for the Neural Network.  It should create the following placeholders:
- Input text placeholder named "input" using the TF Placeholder name parameter.
- Targets placeholder
- Learning Rate placeholder
Return the placeholders in the following the tuple (Input, Targets, LearingRate)
Can you write Python code for it?

def get_inputs():
    """
    Create TF Placeholders for input, targets, and learning rate.
    :return: Tuple (input, targets, learning rate)
    """
    # TODO: Implement Function
    _input = tf.placeholder(tf.int32,shape=[None, None],name="input")
    _targets = tf.placeholder(tf.int32,shape=[None, None],name='targets')
    _lr = tf.placeholder(tf.float32,name='learing_rate')
    return _input, _targets, _lr

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_get_inputs(get_inputs)


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Neural Network Training
Hyperparameters
Tune the following parameters:

Set num_epochs to the number of epochs.
Set batch_size to the batch size.
Set rnn_size to the size of the RNNs.
Set seq_length to the length of sequence.
Set learning_rate to the learning rate.
Set show_every_n_batches to the number of batches the neural network should print progress.
Can you write Python code for it?

# Number of Epochs
num_epochs = 80
# Batch Size
batch_size = 23
# RNN Size
rnn_size = 256
# Sequence Length
seq_length = 33
# Learning Rate
learning_rate = 0.005
# Show stats for every n number of batches
show_every_n_batches = 100

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
save_dir = './save'


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Preprocess all the data and save it
Running the code cell below will preprocess all the data and save it to file.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# Preprocess Training, Validation, and Testing Data
helper.preprocess_and_save_data(data_dir, token_lookup, create_lookup_tables)


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Save Parameters
Save seq_length and save_dir for generating a new TV script.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# Save parameters for checkpoint
helper.save_params((seq_length, save_dir))


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Tokenize Punctuation
We'll be splitting the script into a word array using spaces as delimiters.  However, punctuations like periods and exclamation marks make it hard for the neural network to distinguish between the word "bye" and "bye!".
Implement the function token_lookup to return a dict that will be used to tokenize symbols like "!" into "||Exclamation_Mark||".  Create a dictionary for the following symbols where the symbol is the key and value is the token:
- Period ( . )
- Comma ( , )
- Quotation Mark ( " )
- Semicolon ( ; )
- Exclamation mark ( ! )
- Question mark ( ? )
- Left Parentheses ( ( )
- Right Parentheses ( ) )
- Dash ( -- )
- Return ( \n )
This dictionary will be used to token the symbols and add the delimiter (space) around it.  This separates the symbols as it's own word, making it easier for the neural network to predict on the next word. Make sure you don't use a token that could be confused as a word. Instead of using the token "dash", try using something like "||dash||".
Can you write Python code for it?

def token_lookup():
    """
    Generate a dict to turn punctuation into a token.
    :return: Tokenize dictionary where the key is the punctuation and the value is the token
    """
    # TODO: Implement Function
    Punctuation = { "." :"||Period||",
                   "," :"||Comma||",
                   "\"":"||Quotation_Mark||",
                   ";" :"||Semicolon||",
                   "!" :"||Exclamation_Mark||",
                   "?" :"||Question_Mark||",
                   "(" :"||Left_Parentheses||",
                   ")" :"||Right_Parentheses||",
                   "--" :"||Dash||",
                   "\n" :"||Return||",}
    return Punctuation

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_tokenize(token_lookup)


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Train
Train the neural network on the preprocessed data.  If you have a hard time getting a good loss, check the forms to see if anyone is having the same problem.
Can you write Python code for it?

import datetime
import sys
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
batches = get_batches(int_text, batch_size, seq_length)

with tf.Session(graph=train_graph) as sess:
    sess.run(tf.global_variables_initializer())
    all_start = datetime.datetime.now()
    start = datetime.datetime.now()
    for epoch_i in range(num_epochs):
        state = sess.run(initial_state, {input_text: batches[0][0]})

        for batch_i, (x, y) in enumerate(batches):
            feed = {
                input_text: x,
                targets: y,
                initial_state: state,
                lr: learning_rate}
            train_loss, state, _ = sess.run([cost, final_state, train_op], feed)

            # Show every <show_every_n_batches> batches
            if (epoch_i * len(batches) + batch_i) % show_every_n_batches == 0:
                end = datetime.datetime.now()
                """
                sys.stdout.write('\rEpoch {:>3} Batch {:>4}/{}   train_loss = {:.3f}  time = {}'.format(
                    epoch_i,
                    batch_i,
                    len(batches),
                    train_loss,
                    (end-start)))
                #"""

                print('Epoch {:>3} Batch {:>4}/{}   train_loss = {:.3f}  time = {}'.format(
                    epoch_i,
                    batch_i,
                    len(batches),
                    train_loss,
                    (end-start)))
                start = datetime.datetime.now()

    # Save Model
    all_end = datetime.datetime.now()
    saver = tf.train.Saver()
    saver.save(sess, save_dir)
    print('\nModel Trained and Saved Time {} train_loss = {:.3f}'.format(all_end-all_start,train_loss))


I am working on the file "3/dlnd_tv_script_generation.ipynb".
The first task is:
Word Embedding
Apply embedding to input_data using TensorFlow.  Return the embedded sequence.
Can you write Python code for it?

def get_embed(input_data, vocab_size, embed_dim):
    """
    Create embedding for <input_data>.
    :param input_data: TF placeholder for text input.
    :param vocab_size: Number of words in vocabulary.
    :param embed_dim: Number of embedding dimensions
    :return: Embedded input.
    """
    # TODO: Implement Function
    #embedding = tf.get_variable("embedding", [vocab_size, embed_dim])
    embedding = tf.Variable(tf.random_uniform([vocab_size, embed_dim],-1,1))
    _inputs = tf.nn.embedding_lookup(embedding, input_data)
    return _inputs

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_get_embed(get_embed)


I am working on the file "30 Text to Speech.ipynb".
The first task is:
For example, if you have a long-running notebook cell, you could finish it with:
Can you write Python code for it?

import random
answer = int(random.random() * 100)
speak("Computation is complete.  The answer is: %d." % answer)


I am working on the file "30 Text to Speech.ipynb".
The first task is:
Now, we can make a Python function that returns a spoken result:
Can you write Python code for it?

from IPython.display import Javascript
import json

def speak(msg):
    return Javascript("""
            var msg = new SpeechSynthesisUtterance(%s);
            window.speechSynthesis.cancel();
            window.speechSynthesis.speak(msg);
        """ % json.dumps(msg))


I am working on the file "30--more-svd.ipynb".
The first task is:
(optional) Exploring handwritten digits
If time permits, we'll use the code fragments below to explore a different data set, based on images of handwritten digits available from: http://yann.lecun.com/exdb/mnist/
Can you write Python code for it?

# Download and unpack MNIST digits database
(mnist_images_gz, mnist_labels_gz) = cse6040utils.download_mnist ('training')

print "Images:", mnist_images_gz
print "Labels:", mnist_labels_gz

images, labels, inds = cse6040utils.load_mnist (mnist_images_gz, mnist_labels_gz,
                                                digits=[1, 8],
                                                return_indices=True)

print images.shape
print labels.shape

imshow_gray (images[2, :, :])

# Compute mean image
mean_image = np.mean (images, 0)
imshow_gray (mean_image)

X = np.reshape (images - mean_image, (images.shape[0], images.shape[1]*images.shape[2]))
print X.shape

(U, Sigma, VT) = np.linalg.svd (X, full_matrices=False)

peek_Sigma (Sigma)

#imshow_gray (mean_image)
imshow_gray (np.reshape (VT[2, :], (28, 28)))

# Project onto the first k principal axes
k = 2
Y = X.dot (VT[0:k, :].T)

annotations = ["[%d] %d" % (i, d) for (i, d) in zip (np.arange (len (Y)), labels)]

print annotations[:5]

plotly_scatter_2d_labeled (Y, labels=annotations)

traces_Y = make_clustering_traces_k2 (Y, clustering=labels, labels=annotations)
fig = Figure (data=traces_Y)
py.iplot (fig)

imshow_gray (images[2705, :, :])

imshow_gray (images[1407, :, :])

num_clusters = 2
centers, distortion = kmeans (X, num_clusters)
clustering, _ = vq (X, centers)

traces_Y = make_clustering_traces_k2 (Y, clustering, labels=annotations)
fig = Figure (data=traces_Y)
py.iplot (fig)

imshow_gray (images[93, :, :])


I am working on the file "30--more-svd.ipynb".
The first task is:
Applying PCA
Exercise. Compute the SVD of X. Store the result in three arrays, U, Sigma, and VT, where U holds $U$, Sigma holds just the diagonal entries of $\Sigma$, and VT holds $V^T$.
Can you write Python code for it?

# @YOUSE: Compute the SVD of X here
pass

# Sanity check on dimensions
print "X:", X.shape
print "U:", U.shape
print "Sigma:", Sigma.shape
print "V^T:", VT.shape


I am working on the file "30--more-svd.ipynb".
The first task is:
Exercise (question only). Does the spectrum of these data decay quickly or slowly? How should that affect your choice of $k$, if you are considering a $k$-truncated SVD?
(@YOUSE: Enter a response here)
Exercise. Run the code cell below to look at the first ("0"-th) principal components. Modify the cell and re-run it to look at a few more. What do they appear to capture?
Can you write Python code for it?

imshow_gray (np.reshape (VT[0, :], (min_rows, min_cols)))


I am working on the file "30--more-svd.ipynb".
The first task is:
Exercise. Recall that PCA requires centered points. Let's do that by subtracting the mean image from every image, overwriting the result (images array).
Can you write Python code for it?

# @YOUSE: Re-center the images around `mean_image`
pass

imshow_gray (images[29, :, :]) # Compare this to the original. It's "spooky!"


I am working on the file "30--more-svd.ipynb".
The first task is:
Exercise. Recenter the images so that they are all the same size. Store them in a 3-dimensional Numpy array called images[:, :, :], where images[k, :, :] is the k-th image.
Can you write Python code for it?

# Re-center images to a common size
images = np.zeros ((len (original_images), min_rows, min_cols))
for (i, image) in enumerate (original_images):
    # @YOUSE: Implement a recentering
    pass

imshow_gray (images[29, :, :]) # Test: Who am I?


I am working on the file "30--more-svd.ipynb".
The first task is:
Exercise. Write some code to compute a new matrix Y, which is the original data matrix projected onto the first two principal components.

You can use the second and third code cells below to draw your projection.
Can you write Python code for it?

# @YOUSE: Project onto the first k principal axes.
Y = ...

def plotly_scatter_2d_labeled (X, x_1=0, x_2=1, labels=""):
    m, d = X.shape
    traces = []
    if d > 1:
        traces.append (Scatter (x=Y[:, x_1:x_1+1], y=Y[:, x_2:x_2+1],
                                mode='markers',
                                text=labels))
    else:
        traces.append (Scatter (x=Y, y=[0.0] * m,
                                mode='markers',
                                text=labels))
    fig = Figure (data=traces) #, layout=layout)
    display (py.iplot (fig))

plotly_scatter_2d_labeled (Y, labels=image_names)


I am working on the file "30--more-svd.ipynb".
The first task is:
Let's convert this image into a Numpy array, and then also to grayscale.
Can you write Python code for it?

def rgb2gray (rgb):
    return np.dot (rgb[...,:3], [0.299, 0.587, 0.144])

def imshow_gray (im):
    plt.imshow (im, interpolation='nearest', cmap=plt.get_cmap ('gray'))

goofy_np_gray = rgb2gray (np.asarray (goofy))
imshow_gray (goofy_np_gray)
print "What a ham!"


I am working on the file "30--more-svd.ipynb".
The first task is:
Next, let's load all the images as grayscale, into a list, original_images, along with an array image_names to hold a name for each image. (The names are extracted from the image filename.)

You may need to adjust the filepath below if this code does not work for you.
Can you write Python code for it?

original_images = []
image_names = []
for base, dirs, files in os.walk ('peeps/.'):
    for filename in files:
        name_tiff = re.match (r'^(.*)\.tiff$', filename)
        if name_tiff:
            filepath = os.path.join (base, filename)
            im = rgb2gray (np.asarray (Image.open (filepath, 'r')))
            key = name_tiff.groups (0)[0]
            original_images.append (im)
            image_names.append (key)

print "Found", len (original_images), "goofy images.\n"


I am working on the file "30--more-svd.ipynb".
The first task is:
Recap: Solving the PCA problem
Recall the basic algorithm to compute a PCA, the theory of which is explained in the notes of Lab 29 and an interactive visual demo of which appears at http://setosa.io/ev/principal-component-analysis/.
You are given a set of $m-1$ data points, $X \equiv (x_0, x_1, \cdots, x_{m-1})^T$. Each data point $x_i \in \mathbb{R}^d$ is $d$-dimensional. You wish to find a $k$-dimensional representation of these points, where $k \leq d$. To do so, you run the PCA procedure, which identifies a $k$-dimensional subspace in terms of $k$ orthogonal vectors ("axes"); these vectors are the principal components.

If the data are not centered, transform them accordingly. In particular, ensure that their mean is 0, i.e., $\displaystyle \frac{1}{m} \sum_{i=0}^{m-1} x_i = 0$.
Compute the $k$-truncated SVD, $X \approx U_k \Sigma_k V_k^T$. The truncated SVD is just the subset of singular vectors corresponding to the largest $k$ singular values.
Choose $v_0, v_1, \ldots, v_{k-1}$ as the principal components.

A "Surprise" Dataset
Today's first application of PCA involves a surprise data set. If you haven't done so already, download it and unpack it into your notebook's working directory.
The data set is a bunch of goofy images. Let's look at one, selected at random. I swear I picked it randomly.
Can you write Python code for it?

goofy = Image.open ('peeps/rodd.tiff', 'r') # Load an image
goofy


I am working on the file "30--more-svd.ipynb".
The first task is:
The following code looks at Sigma. The collection of $\sigma_i$ values is also referred to as the spectrum of the matrix.
Can you write Python code for it?

def peek_Sigma (Sigma, ret_df=False):
    k = len (Sigma)
    df_Sigma = pd.DataFrame (np.arange (len (Sigma)), columns=['i'])
    df_Sigma['sigma_i'] = Sigma
    Sigma_sq = np.power (Sigma, 2)
    Err_sq = np.sum (Sigma_sq) - np.cumsum (Sigma_sq)
    Err_sq[Err_sq < 0] = 0
    Err = np.sqrt (Err_sq)
    Relerr = Err / (Sigma[0] + Err[0])
    df_Sigma['sigma_i^2'] = Sigma_sq
    df_Sigma['err_i^2'] = Err_sq
    df_Sigma['err_i'] = Err
    df_Sigma['relerr_i'] = Relerr
    print "Singular values:"
    display (df_Sigma.head ())
    print "  ..."
    display (df_Sigma.tail ())

    f, ax = plt.subplots (figsize=(7, 7))
    #ax.set (yscale="log")
    sns.regplot ("i", "sigma_i", df_Sigma, ax=ax, fit_reg=False)
    if ret_df:
        return df_Sigma

peek_Sigma (Sigma)


I am working on the file "30. 딥러닝/01. 신경망 기초 이론.ipynb".
The first task is:
$$ \frac{\partial C}{\partial w^2_{jk}} = z^1_k \delta^2_j $$
$$ \delta^1 = h'(a^1) \odot ((w^{2})^T \delta^{2}) $$
Can you write Python code for it?


I am working on the file "30. 딥러닝/01. 신경망 기초 이론.ipynb".
The first task is:
$$ y = z^{3} = h \left( w^{3}  \cdot z^{2} + b^3 \right) = h \left( a^3 \right)$$
오차 함수
신경망의 오차 함수는 조건부 확률이라는 실수 값을 출력해야 하므로  퍼셉트론과 달리 제곱합 오차 함수를 사용한다.
$$
\begin{eqnarray}  C(w,b) =
  \frac{1}{2n} \sum_i \| y_i - \hat{y}(x_i; w, b)\|^2 = \frac{1}{2n} \sum_i \| y_i - z_i \|^2
\end{eqnarray}
$$
가중치 최적화
오차함수를 최소화하는 최적의 가중치를 찾기 위해 다음과 같이 미분(gradient)을 사용한 steepest gradient descent 방법을 적용한다.
$$
\begin{eqnarray}
  \Delta w = -\eta \nabla C,
\end{eqnarray}
$$
여기에서 $\eta$는 최적화 속도(learning rate)이다.
$$
\begin{eqnarray}
  \nabla C \equiv \left(\frac{\partial C}{\partial w_1}, \ldots,
  \frac{\partial C}{\partial w_m}\right)^T
\end{eqnarray}
$$
가중치 갱신 공식은 다음과 같다.
$$
\begin{eqnarray}
  w_k & \rightarrow & w_k' = w_k-\eta \frac{\partial C}{\partial w_k} \
  b_l & \rightarrow & b_l' = b_l-\eta \frac{\partial C}{\partial b_l}
\end{eqnarray}
$$
Stochastic Gradient Descent
실제로는 단순 Steepest Gradient Descent 방법보다 (SGC: Stochastic Gradient Descent)를 주로 사용한다. SGD는 미분 계산을 위해 전체 데이터 샘플을 모두 사용하지 않고  $m$개의 일부 데이터만 사용하여 미분을 계산하는 방법이다.
$$
\begin{eqnarray}
  \frac{\sum_{j=1}^m \nabla C_{X_{j}}}{m} \approx \frac{\sum_x \nabla C_x}{n} = \nabla C
\end{eqnarray}
$$
이 경우 가중치 갱신 공식은 다음과 같다.
$$
\begin{eqnarray}
  w_k & \rightarrow & w_k' = w_k-\frac{\eta}{m}
  \sum_j \frac{\partial C_{X_j}}{\partial w_k} \
  b_l & \rightarrow & b_l' = b_l-\frac{\eta}{m}
  \sum_j \frac{\partial C_{X_j}}{\partial b_l},
\end{eqnarray}
$$
Back Propagation
단순하게 수치적으로 미분을 계산한다면 모든 가중치에 대해서 개별적으로 미분을 계산해야 한다. 그러나 back propagation 방법을 사용하면 모든 가중치에 대한 미분값을 한번에 계산할 수 있다.
back propagation 방법을 수식으로 표현하면 다음과 같다.
우선 $\delta$ 를 뒤에서 앞으로 전파한다. $\delta$는 다음과 같이 정의되는 값이다.
$$
\delta_j = \dfrac{\partial C}{\partial a_j}
$$
$$
\begin{eqnarray}
  \delta^{l-1}j = h'(a^{l-1}_j) \sum_k w^l{kj} \delta^l_k
\end{eqnarray}
$$
이 식을 벡터-행렬 식으로 쓰면 다음과 같다.
$$
\delta^{l-1} = h'(a^{l-1}) \odot ((w^{l})^T \delta^{l})
$$
여기에서 $\odot$ 연산 기호는 Hamadard Product 혹은 Schur product 라고 불리는 연산으로 정의는 다음과 같다.
$$
\left(\begin{array}{ccc} \mathrm{a}{11} & \mathrm{a}{12} & \mathrm{a}{13}\ \mathrm{a}{21} &...
Can you write Python code for it?


I am working on the file "30. 딥러닝/01. 신경망 기초 이론.ipynb".
The first task is:
$$ z^{1} = h \left( w^{1}  \cdot x + b^1 \right) = h \left( a^1 \right)$$
Can you write Python code for it?


I am working on the file "30. 딥러닝/01. 신경망 기초 이론.ipynb".
The first task is:
Feedforward propagation
신경망의 계산 과정은 실제 신경망에서 신호가 전달과는 과정과 유사하므로 Feedforward propagation 이라고 불린다.
$l$번째 계층의 $j$번째 뉴런에서의 출력값 $z^l$은 다음과 같이 정의된다.
$$
\begin{eqnarray}
  z^{l}j = h \left( \sum_k w^{l}{jk} z^{l-1}k + b^l_j \right) = h \left( w^{l}{j} \cdot z^{l-1} + b^l_j \right)
\end{eqnarray}
$$
$l$번째 계층 전체의 출력은 다음과 같이 표시할 수 있다.
$$
\begin{eqnarray}
  z^{l} = h \left( \sum_k w^{l}_{k} z^{l-1}_k + b^l \right) = h \left( w^{l}  \cdot z^{l-1} + b^l \right)
\end{eqnarray}
$$
$$
a^l \equiv w^l \cdot z^{l-1}+b^l
$$
$$
\begin{eqnarray}
  z^{l} =  h \left( a^l \right)
\end{eqnarray}
$$
아래에 Feedforward propagation 예를 보였다.
Can you write Python code for it?


I am working on the file "30. 딥러닝/01. 신경망 기초 이론.ipynb".
The first task is:
입력 $x$
 $$ x_1,x_2,x_3 $$

가중치  $w$
 $$ w_1, w_2, w_3 $$

상수항(bias) $b$를 포함한 활성화값(activation)
 $$ a = \sum_{j=1}^3 w_j x_j + b $$

비선형 활성화 함수 $h$
 $$ z = h(a) = h \left( \sum_{j=1}^3 w_j x_j + b \right) $$

출력 $y$
$$
y =
\begin{cases}
0 & \text{if } z \leq 0, \
1 & \text{if } z > 0
\end{cases}
$$

이런 퍼셉트론에서 $x$ 대신 기저 함수를 적용한 $\phi(x)$를 사용하면 XOR 문제 등의 비선형 문제를 해결할 수 있다. 그러나 고정된 기저 함수를 사용해야 하므로 문제에 맞는 기저 함수를 찾아야 한다는 단점이 있다.
만약 기저 함수 $\phi(x)$의 형태를 추가적인 모수 $w^{(1)}$, $b^{(1)}$를 사용하여 조절할 수 있다면 즉, 기저함수 $\phi(x;w^{(1)}, b^{(1)})$ 를 사용하면 $w^{(1)}$ 값을 바꾸는 것만으로 다양한 기저 함수를 시도할 수 있다.
$$ z = h \left(  \sum_{j=1} w_j^{(2)} \phi_j(x ; w^{(1)}_j, b^{(1)}_j) + b^{(2)} \right) $$
신경망은 다음과 같이 원래 퍼셉트론과 같은 형태의 적응형 기저함수를 사용한 모형이다.
$$ \phi_j(x ; w^{(1)}j, b^{(1)}_j)  = h \left(  \sum{i=1} w_{ji}^{(1)} x_i + b_j^{(1)} \right)  $$
즉 전체 모형은 다음과 같다.
$$ z = h \left(  \sum_{j=1} w_j^{(2)} h \left(  \sum_{i=1} w_{ji}^{(1)} x_i + b_j^{(1)} \right)  + b^{(2)} \right) $$
일반적으로 활성화 함수 $h$ 는 다음과 같은 시그모이드 함수 $\sigma$를 사용한다.
$$
\begin{eqnarray}
z = \sigma(a) \equiv \frac{1}{1+e^{-a}}.
\end{eqnarray}
$$
$$
\begin{eqnarray}
\frac{1}{1+\exp(-\sum_j w_j x_j-b)}
\end{eqnarray}
$$
이 시그모이드 함수의 특징은 다음과 같은 미분값을 가진다는 것이다.
$$ \sigma' = \sigma(1-\sigma) $$
퍼셉트론을 사용한 XOR 문제 해결법
퍼셉트론를 연속적으로 연결하여 비선형 문제를 해결하는 방법은 이미 디지털 회로 설계에서 사용되던 방법이다.
퍼셉트론의 가중치를 적절히 조정하면 다음과 같은 AND / OR 등의 디지털 게이트(gate)를 제작할 수 있다.
예를 들어 $w_1 = -2$, $w_2 = -2$, $b = 3$ 인 퍼셉트론은 NAND 게이트를 구현한다.
Can you write Python code for it?

%%tikz
\tikzstyle{neuron}=[circle, draw, minimum size=23pt,inner sep=0pt, node distance=2cm]

\node[neuron] (node) {$z$};
\node[neuron] (x2) [left of=node] {$x_2$};
\node[neuron] (x1) [above of=x2]  {$x_1$};
\node[neuron] (b)  [below of=x2]  {$1$};
\node[neuron] (output) [right of=node] {$y$};
\draw[->] (x1) -- node[above=0.1] {$-2$} (node);
\draw[->] (x2) -- node[above] {$-2$} (node);
\draw[->] (b) -- node[above=0.1] {$3$} (node);
\draw[->] (node) -- (output);


I am working on the file "30. 딥러닝/03. 신경망 성능 개선.ipynb".
The first task is:
교차 엔트로피 오차 함수 (Cross-Entropy Cost Function)
이러한 수렴 속도 문제를 해결하는 방법의 하나는 오차 제곱합 형태가 아닌 교차 엔트로피(Cross-Entropy) 형태의 오차함수를 사용하는 것이다.
$$
\begin{eqnarray}
  C = -\frac{1}{n} \sum_x \left[y \ln z + (1-y) \ln (1-z) \right],
\end{eqnarray}
$$
미분값은 다음과 같다.
$$
\begin{eqnarray}
  \frac{\partial C}{\partial w_j} & = & -\frac{1}{n} \sum_x \left(
    \frac{y }{z} -\frac{(1-y)}{1-z} \right)
  \frac{\partial z}{\partial w_j} \
 & = & -\frac{1}{n} \sum_x \left(
    \frac{y}{\sigma(a)}
    -\frac{(1-y)}{1-\sigma(a)} \right)\sigma'(a) x_j \
 & = &
  \frac{1}{n}
  \sum_x \frac{\sigma'(a) x_j}{\sigma(a) (1-\sigma(a))}
  (\sigma(a)-y) \
 & = & \frac{1}{n} \sum_x x_j(\sigma(a)-y) \
 & = & \frac{1}{n} \sum_x (z-y) x_j\ \
\frac{\partial C}{\partial b} &=& \frac{1}{n} \sum_x (z-y)
\end{eqnarray}
$$
이 식에서 보다시피 기울기(gradient)가 예측 오차(prediction error) $z-y$에 비례하기 때문에

오차가 크면 수렴 속도가 빠르고
오차가 적으면 속도가 감소하여 발산을 방지한다.

교차 엔트로피 구현 예

https://github.com/mnielsen/neural-networks-and-deep-learning/blob/master/src/network2.py

```python
Define the quadratic and cross-entropy cost functions
class QuadraticCost(object):
@staticmethod
def fn(a, y):
    """Return the cost associated with an output ``a`` and desired output
    ``y``.
    """
    return 0.5*np.linalg.norm(a-y)**2

@staticmethod
def delta(z, a, y):
    """Return the error delta from the output layer."""
    return (a-y) * sigmoid_prime(z)

class CrossEntropyCost(object):
@staticmethod
def fn(a, y):
    """Return the cost associated with an output ``a`` and desired output
    ``y``.  Note that np.nan_to_num is used to ensure numerical
    stability.  In particular, if both ``a`` and ``y`` have a 1.0
    in the same slot, then the expression (1-y)*np.log(1-a)
    returns nan.  The np.nan_to_num ensures that that is converted
    to the correct value (0.0).
    """
    return np.sum(np.nan_to_num(-y*np.log(a)-(1-y)*np.log(1-a)))

@staticmethod
def delta(z, a, y):
    """Return the error delta from the output layer.  Note that the
    parameter ``z`` is not used by the method.  It is...
Can you write Python code for it?

%cd /home/dockeruser/neural-networks-and-deep-learning/src
%ls

import mnist_loader
import network2

training_data, validation_data, test_data = mnist_loader.load_data_wrapper()

net = network2.Network([784, 30, 10], cost=network2.QuadraticCost)
net.large_weight_initializer()
%time result1 = net.SGD(training_data, 10, 10, 0.5, evaluation_data=test_data, monitor_evaluation_accuracy=True)

net = network2.Network([784, 30, 10], cost=network2.CrossEntropyCost)
net.large_weight_initializer()
%time result2 = net.SGD(training_data, 10, 10, 0.5, evaluation_data=test_data, monitor_evaluation_accuracy=True)

plt.plot(result1[1], 'bo-', label="quadratic cost")
plt.plot(result2[1], 'rs-', label="cross-entropy cost")
plt.legend(loc=0)
plt.show()


I am working on the file "30. 딥러닝/04. CNN.ipynb".
The first task is:
Add Additional Convolution + Pool Layer

두번째 convolutional-pooling layer의 역할
feature map에서 feature가 나타나는 pattern의 포착
feature of feature map
Can you write Python code for it?

net = Network([
        ConvPoolLayer(image_shape=(mini_batch_size, 1, 28, 28),
                      filter_shape=(20, 1, 5, 5),
                      poolsize=(2, 2)),
        ConvPoolLayer(image_shape=(mini_batch_size, 20, 12, 12),
                      filter_shape=(40, 20, 5, 5),
                      poolsize=(2, 2)),
        FullyConnectedLayer(n_in=40*4*4, n_out=100),
        SoftmaxLayer(n_in=100, n_out=10)],
        mini_batch_size)

net.SGD(training_data, 10, mini_batch_size, 0.1, validation_data, test_data)


I am working on the file "30. 딥러닝/04. CNN.ipynb".
The first task is:
Add Convolutional + Pooling Layer
Can you write Python code for it?

net = Network([
        ConvPoolLayer(image_shape=(mini_batch_size, 1, 28, 28),
                      filter_shape=(20, 1, 5, 5),
                      poolsize=(2, 2)),
        FullyConnectedLayer(n_in=20*12*12, n_out=100),
        SoftmaxLayer(n_in=100, n_out=10)],
        mini_batch_size)

net.SGD(training_data, 10, mini_batch_size, 0.1, validation_data, test_data)


I am working on the file "30. 딥러닝/04. CNN.ipynb".
The first task is:
Apply ReLu

sigmoid activation functions 보다 성능 향상
Can you write Python code for it?

from network3 import ReLU

net = Network([
        ConvPoolLayer(image_shape=(mini_batch_size, 1, 28, 28),
                      filter_shape=(20, 1, 5, 5),
                      poolsize=(2, 2),
                      activation_fn=ReLU),
        ConvPoolLayer(image_shape=(mini_batch_size, 20, 12, 12),
                      filter_shape=(40, 20, 5, 5),
                      poolsize=(2, 2),
                      activation_fn=ReLU),
        FullyConnectedLayer(n_in=40*4*4, n_out=100, activation_fn=ReLU),
        SoftmaxLayer(n_in=100, n_out=10)],
        mini_batch_size)

net.SGD(training_data, 60, mini_batch_size, 0.03, validation_data, test_data, lmbda=0.1)


I am working on the file "30. 딥러닝/04. CNN.ipynb".
The first task is:
Feature Map

만약 weight가 특정 image patter에 대해 a=1인 출력을 내도록 training 되었다면
hidden layer는 feature가 존재하는 위치를 표시
=> feature map
여기에서의 feature는 input data를 의미하는 것이 아니라 image 분류에 사용되는 input data의 특정한 pattern을 뜻함

Multiple Feature Maps

하나의 공통 weight set은 한 종류의 image feature만 발견 가능
복수의 feature map (weight set) 필요

MNIST digit image 에 대해 training이 완료된 20개 feature map의 예

Max Pooling Layer

영역내에서 가장 최대값 출력
영역내에 feature가 존재하는지의 여부
전체 영역이 축소

L2 pooling

maximum 값 대신에 영역내의 값의 sum of square 사용

Output Layer

softmax

Demo

http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html

Python Implementation

https://github.com/mnielsen/neural-networks-and-deep-learning/blob/master/src/network3.py

```python
class FullyConnectedLayer(object):
def __init__(self, n_in, n_out, activation_fn=sigmoid, p_dropout=0.0):
    self.n_in = n_in
    self.n_out = n_out
    self.activation_fn = activation_fn
    self.p_dropout = p_dropout
    # Initialize weights and biases
    self.w = theano.shared(
        np.asarray(
            np.random.normal(
                loc=0.0, scale=np.sqrt(1.0/n_out), size=(n_in, n_out)),
            dtype=theano.config.floatX),
        name='w', borrow=True)
    self.b = theano.shared(
        np.asarray(np.random.normal(loc=0.0, scale=1.0, size=(n_out,)),
                   dtype=theano.config.floatX),
        name='b', borrow=True)
    self.params = [self.w, self.b]

def set_inpt(self, inpt, inpt_dropout, mini_batch_size):
    self.inpt =...
Can you write Python code for it?


I am working on the file "30. 딥러닝/04. CNN.ipynb".
The first task is:
Normal MLP
Can you write Python code for it?

import network3
from network3 import Network
from network3 import ConvPoolLayer, FullyConnectedLayer, SoftmaxLayer

training_data, validation_data, test_data = network3.load_data_shared()
mini_batch_size = 10

net = Network([
            FullyConnectedLayer(n_in=784, n_out=100),
            SoftmaxLayer(n_in=100, n_out=10)],
            mini_batch_size)

net.SGD(training_data, 10, mini_batch_size, 0.1, validation_data, test_data)


I am working on the file "30. 딥러닝/05. 확률론적 언어 모형.ipynb".
The first task is:
이번에는 한글 자료를 이용해보자 코퍼스로는 아래의 웹사이트에 공개된 Naver sentiment movie corpus 자료를 사용한다.
* https://github.com/e9t/nsmc
Can you write Python code for it?

import codecs
def read_data(filename):
    with codecs.open(filename, encoding='utf-8', mode='r') as f:
        data = [line.split('\t') for line in f.read().splitlines()]
        data = data[1:]   # header 제외
    return data

train_data = read_data('/home/dockeruser/data/nsmc/ratings_train.txt')

from konlpy.tag import Twitter
tagger = Twitter()

def tokenize(doc):
    return ['/'.join(t) for t in tagger.pos(doc, norm=True, stem=True)]

train_docs = [row[1] for row in train_data]
sentences = [tokenize(d) for d in train_docs]

m = BigramModel(sentences)

m.prob(tokenize(u"영화")[0], tokenize(u"이"))

m.scoreSentence(tokenize(u"이 영화 정말 좋네"), verbose=True)

m.scoreSentence(tokenize(u"좋네 영화 이 정말"), verbose=True)

random.seed(24)
print("".join([w.split("/")[0] if w.split("/")[1] == "Josa" else " " + w.split("/")[0] for w in m.generateSentence()]))


I am working on the file "30. 딥러닝/05. 확률론적 언어 모형.ipynb".
The first task is:
이제 이 입력으로부터 확률값을 추정한다.
Can you write Python code for it?

import collections, math
from math import log
from collections import Counter
from konlpy.utils import pprint

def stringify_context(context):
    return(" ".join(context))

boundaryToken = "</s>"
def ngrams(n, sentences, boundaryToken=boundaryToken, verbose=False):
    c = {}
    q = []
    for i in range(n-1):
        q.append(boundaryToken)
    for sentence in sentences:
        for w in sentence + [boundaryToken]:
            context_gram = stringify_context(q)
            if verbose:
                print(q)
                print(context_gram)
                print(w)
            if not context_gram in c:
                c[context_gram] = Counter()
            c[context_gram][w] += 1
            q.pop(0)
            q.append(w)
    return(c)

ngrams(2, sentences[:1000])["we"]

class BigramModel:

    def __init__(self, training_sentences, smoothing='none'):
        train = ngrams(2, training_sentences)
        self.probs = {}
        if smoothing == 'none':
            for context_gram in train.keys():
                N = sum(train[context_gram].values())
                self.probs[context_gram] = Counter({k:v/N for k,v in train[context_gram].items()})

    def prob(self, word, context):
        """takes a word string and a context which is a list of word strings, and returns the probability of the word"""
        c = stringify_context(context)
        return(self.probs[c][word])

    def scoreSentence(self, sentence, verbose=False):
        context = [boundaryToken]
        result = 0
        for w in sentence + [boundaryToken]:
            lp = log(self.prob(w, context))
            result = result + lp
            if verbose:
                pprint([context, w, lp])
            context = [w]
        return result

    def generateSentence(self, verbose=False, goryDetails=False):
        context = [boundaryToken]
        result = []
        w = None
        while not w == boundaryToken:
            r = random.random() # returns a random float between 0 and 1
            x = 0
            c =...


I am working on the file "30. 딥러닝/06. 단어 임베딩의 원리와 gensim.word2vec 사용법.ipynb".
The first task is:
이번에는 네이버 영화 감상 코퍼스를 사용하여 한국어 단어 임베딩을 해보자.
Can you write Python code for it?

import codecs

def read_data(filename):
    with codecs.open(filename, encoding='utf-8', mode='r') as f:
        data = [line.split('\t') for line in f.read().splitlines()]
        data = data[1:]   # header 제외
    return data

train_data = read_data('/home/dockeruser/data/nsmc/ratings_train.txt')

from konlpy.tag import Twitter
tagger = Twitter()

def tokenize(doc):
    return ['/'.join(t) for t in tagger.pos(doc, norm=True, stem=True)]

train_docs = [row[1] for row in train_data]
sentences = [tokenize(d) for d in train_docs]

from gensim.models import word2vec
model = word2vec.Word2Vec(sentences)
model.init_sims(replace=True)

model.similarity(*tokenize(u'악당 영웅'))

model.similarity(*tokenize(u'악당 감동'))

from konlpy.utils import pprint
pprint(model.most_similar(positive=tokenize(u'배우 남자'), negative=tokenize(u'여배우'), topn=1))


I am working on the file "30. 딥러닝/06. 단어 임베딩의 원리와 gensim.word2vec 사용법.ipynb".
The first task is:
이제 이 모형에서 다음과 같은 메서드를 사용할 수 있다. 보다 자세한 내용은 https://radimrehurek.com/gensim/models/word2vec.html 를 참조한다.

similarity : 두 단어의 유사도 계산
most_similar : 가장 유사한 단어를 출력
Can you write Python code for it?

model.similarity('actor', 'actress')

model.similarity('he', 'she')

model.similarity('actor', 'she')

model.most_similar("villain")


I am working on the file "30. 딥러닝/06. 단어 임베딩의 원리와 gensim.word2vec 사용법.ipynb".
The first task is:
트레이닝이 완료되면 init_sims 명령으로 필요없는 메모리를 unload 시킨다.
Can you write Python code for it?


I am working on the file "30. 딥러닝/07. RNN 기본 구조와 Keras를 사용한 RNN 구현.ipynb".
The first task is:
Keras의 SimpleRNN 클래스
Keras에서 신경망 모형은 다음과 같은 순서로 만든다.

Sequential 클래스 객체인 모형을 생성한다.
add 메서드로 다양한 레이어를 추가한다.
compile 메서드로 목적함수 및 최적화 방법을 지정한다.
fit 메서드로 가중치를 계산한다.

우선 가장 단순한 신경망 구조인 SimpleRNN 클래스를 사용하는 방법은 다음 코드와 같다.
여기에서는 SimpleRNN 클래스 객체로 10개의 뉴런을 가지는 RNN 층을 만든다. 첫번째 인수로는 뉴런의 크기, input_dim 인수로는 벡터의 크기, input_length 인수로는 순서열으 길이를 입력한다.
그 다음으로 SimpleRNN 클래스 객체에서 나오는 10개의 출력값을 하나로 묶어 실수 값을 출력으로 만들기 위해 Dense 클래스 객체를 추가하였다.
손실 함수로는 mean-squred-error를, 최적화 방법으로는 단순한 stochastic gradient descent 방법을 사용한다.
Can you write Python code for it?

from keras.models import Sequential
from keras.layers import SimpleRNN, Dense

np.random.seed(0)
model = Sequential()
model.add(SimpleRNN(10, input_dim=1, input_length=3))
model.add(Dense(1))
model.compile(loss='mse', optimizer='sgd')


I am working on the file "30. 딥러닝/07. RNN 기본 구조와 Keras를 사용한 RNN 구현.ipynb".
The first task is:
이 번에는 출력값도 3개짜리 순서열로 한다.
Can you write Python code for it?

X_train2 = S[:-3, 0:3][:, :, np.newaxis]
Y_train2 = S[:-3, 3:6][:, :, np.newaxis]
X_train2.shape, Y_train2.shape

X_train2[:2]

Y_train2[:2]

plt.subplot(211)
plt.plot([0, 1, 2], X_train2[0].flatten(), 'bo-', label="input sequence")
plt.plot([3, 4, 5], Y_train2[0].flatten(), 'ro-', label="target sequence")
plt.xlim(-0.5, 6.5)
plt.ylim(-1.1, 1.1)
plt.legend()
plt.title("First sample sequence")
plt.subplot(212)
plt.plot([1, 2, 3], X_train2[1].flatten(), 'bo-', label="input sequence")
plt.plot([4, 5, 6], Y_train2[1].flatten(), 'ro-', label="target sequence")
plt.xlim(-0.5, 6.5)
plt.ylim(-1.1, 1.1)
plt.legend()
plt.title("Second sample sequence")
plt.tight_layout()
plt.show()

history2 = model2.fit(X_train2, Y_train2, nb_epoch=100, verbose=0)

plt.plot(history2.history["loss"])
plt.title("Loss")
plt.show()


I am working on the file "30. 딥러닝/07. RNN 기본 구조와 Keras를 사용한 RNN 구현.ipynb".
The first task is:
이 시계열 자료에서 입력 순서열과 target 값을 만든다. 입력 순서열은 3 스텝 크기의 순서열을 사용하고 target으로는 그 다음 시간 스텝의 값을 사용한다. 즉, 3개의 순서열을 입력한 다음 마지막 출력값이 target과 일치하게 만드는 sequence-to-value (many-to-one) 문제를 풀어보도록 한다.
Keras 에서 RNN 을 사용하려면 입력 데이터는 (nb_samples, timesteps, input_dim) 크기를 가지는 ndim=3인 3차원 텐서(tensor) 형태이어야 한다.

nb_samples: 자료의 수
timesteps: 순서열의 길이
input_dim: x 벡터의 크기

여기에서는 단일 시계열이므로  input_dim = 1 이고 3 스텝 크기의 순서열을 사용하므로 timesteps = 3 이며 자료의 수는 18 개이다.
다음코드와 같이 원래의 시계열 벡터를 Toeplitz 행렬 형태로 변환하여 3차원 텐서를 만든다.
Can you write Python code for it?

from scipy.linalg import toeplitz
S = np.fliplr(toeplitz(np.r_[s[-1], np.zeros(s.shape[0] - 2)], s[::-1]))
S[:5, :3]

X_train = S[:-1, :3][:, :, np.newaxis]
Y_train = S[:-1, 3]
X_train.shape, Y_train.shape

X_train[:2]

Y_train[:2]

plt.subplot(211)
plt.plot([0, 1, 2], X_train[0].flatten(), 'bo-', label="input sequence")
plt.plot([3], Y_train[0], 'ro', label="target")
plt.xlim(-0.5, 4.5)
plt.ylim(-1.1, 1.1)
plt.legend()
plt.title("First sample sequence")
plt.subplot(212)
plt.plot([1, 2, 3], X_train[1].flatten(), 'bo-', label="input sequence")
plt.plot([4], Y_train[1], 'ro', label="target")
plt.xlim(-0.5, 4.5)
plt.ylim(-1.1, 1.1)
plt.legend()
plt.title("Second sample sequence")
plt.tight_layout()
plt.show()


I am working on the file "30. 딥러닝/07. RNN 기본 구조와 Keras를 사용한 RNN 구현.ipynb".
The first task is:
일단 학습을 시키기 이전에 나오는 출력을 살펴보자.
Can you write Python code for it?

plt.plot(Y_train, 'ro-', label="target")
plt.plot(model.predict(X_train[:,:,:]), 'bs-', label="output")
plt.xlim(-0.5, 20.5)
plt.ylim(-1.1, 1.1)
plt.legend()
plt.title("Before training")
plt.show()


I am working on the file "30. 딥러닝/07. RNN 기본 구조와 Keras를 사용한 RNN 구현.ipynb".
The first task is:
학습 결과는 다음과 같다.
Can you write Python code for it?

plt.subplot(211)
plt.plot([1, 2, 3], X_train2[1].flatten(), 'bo-', label="input sequence")
plt.plot([4, 5, 6], Y_train2[1].flatten(), 'ro-', label="target sequence")
plt.plot([4, 5, 6], model2.predict(X_train2[1:2,:,:]).flatten(), 'gs-', label="output sequence")
plt.xlim(-0.5, 7.5)
plt.ylim(-1.1, 1.1)
plt.legend()
plt.title("Second sample sequence")
plt.subplot(212)
plt.plot([2, 3, 4], X_train2[2].flatten(), 'bo-', label="input sequence")
plt.plot([5, 6, 7], Y_train2[2].flatten(), 'ro-', label="target sequence")
plt.plot([5, 6, 7], model2.predict(X_train2[2:3,:,:]).flatten(), 'gs-', label="output sequence")
plt.xlim(-0.5, 7.5)
plt.ylim(-1.1, 1.1)
plt.legend()
plt.title("Third sample sequence")
plt.tight_layout()
plt.show()


I am working on the file "30. 딥러닝/07. RNN 기본 구조와 Keras를 사용한 RNN 구현.ipynb".
The first task is:
학습을 마친 후의 출력은 다음과 같다.
Can you write Python code for it?

plt.plot(Y_train, 'ro-', label="target")
plt.plot(model.predict(X_train[:,:,:]), 'bs-', label="output")
plt.xlim(-0.5, 20.5)
plt.ylim(-1.1, 1.1)
plt.legend()
plt.title("After training")
plt.show()


I am working on the file "30_ML-III/ML_3_Inclass_Homework.ipynb".
The first task is:
Implement an SVM!
We've already learned about support vector machines. Now we're going to implement one.
We need to find out how to use this thing! We ran some code in the previous notebook that did this for us, but now we need to make things work on our own. Googling for "svm sklearn classifier" gets us to this page. This page has documentation for the package. Partway down the page, we see: "SVC, NuSVC and LinearSVC are classes capable of performing multi-class classification on a dataset." As we keep reading, we see that SVC provides an implementation. Let's try that!
We get to the documentation for SVC and it says many things. At the top, there's a box that says:
class sklearn.svm.SVC(C=1.0, kernel='rbf', degree=3, gamma='auto', coef0=0.0, shrinking=True, probability=False, tol=0.001, cache_size=200, class_weight=None, verbose=False, max_iter=-1, decision_function_shape=None, random_state=None)
How should we interpret all of this?
The first part tells us where a function lives, so the SVC function lives in sklearn.svm. It seems we're going to need to import it from there.
Can you write Python code for it?

# First we need to import svms from sklearn
from sklearn.svm import SVC


I am working on the file "30_ML-III/ML_3_Inclass_Homework.ipynb".
The first task is:
Nice! Now we can see that while our training accuracy is very high, our testing accuracy is much lower. We could say that our model has "overfit" to the data. We learned about overfitting before. You'll get a chance to play with this SVM a bit more below. Before we move to that though, we want to show you how easy it is to use a different classifier. You might imagine that a classifier could be composed of a cascading series of rules. If this is true, then consider that. Otherwise, consider this other thing. This type of algorithm is called a decision tree, and we're going to rain one now.
sklearn has a handy decision tree classifier that we can use. By using the SVM classifier, we've already learned most of what we need to know to use it.
Can you write Python code for it?

# First, we need to import the classifier
from sklearn.tree import DecisionTreeClassifier

# Now we're going to get a decision tree classifier with the default parameters
classifier = DecisionTreeClassifier()

# The 'fit' syntax is the same
classifier.fit(X1_train, y1_train)

# As is the 'score' syntax
train_score = classifier.score(X1_train, y1_train)
test_score = classifier.score(X1_test, y1_test)

print("Training Accuracy: " + str(train_score))
print("Testing Accuracy: " + str(test_score))


I am working on the file "30_ML-III/ML_3_Inclass_Homework.ipynb".
The first task is:
Oof! That's pretty overfit! We're perfect on the training data but basically flipping a coin on the held out data. A DecisionTreeClassifier has two parameters max_features and max_depth that can really help us prevent overfitting. Let's train a very small tree (no more than 8 features) that's very short (no more than 3 deep).
Can you write Python code for it?

# Now we're going to get a decision tree classifier with selected parameters
classifier = DecisionTreeClassifier(max_features=8, max_depth=3)

# The 'fit' syntax is the same
classifier.fit(X1_train, y1_train)

# As is the 'score' syntax
train_score = classifier.score(X1_train, y1_train)
test_score = classifier.score(X1_test, y1_test)

print("Training Accuracy: " + str(train_score))
print("Testing Accuracy: " + str(test_score))


I am working on the file "30_ML-III/ML_3_Inclass_Homework.ipynb".
The first task is:
Ouch! Only about 50% accuracy. That's painful! We learned that we could modify C to make the algorithm try to fit the data we show it better. Let's ramp up C and see what happens!
Can you write Python code for it?

# Get an SVC with a high C
classifier = SVC(C = 100)

# Fit the classifier to our datasets
classifier.fit(X1, y1)

# Apply the classifier back to our data and get an accuracy measure
train_score = classifier.score(X1, y1)

# Print the accuracy
print(train_score)

import sklearn


I am working on the file "311 Combining CSV Datasets, Parsing by Week.ipynb".
The first task is:
Converting Date to date format and creating a new column for week numbers based on that
Can you write Python code for it?

df['Created Date'] = pd.to_datetime(df['Created Date'], errors='coerce')
df.sort_values('Created Date', inplace = True)
df['Week Number'] = df['Created Date'].dt.week
df


I am working on the file "311 Combining CSV Datasets, Parsing by Week.ipynb".
The first task is:
New CSV for dataframe with week number column added on called "311_week_number.csv"
Can you write Python code for it?

df.to_csv('311_week_number.csv', encoding = 'utf-8', index = False)


I am working on the file "311 Combining CSV Datasets, Parsing by Week.ipynb".
The first task is:
Parsed the merged database for all empty coordinates and removed them
Can you write Python code for it?

my_311 = pd.read_csv("MyLA311_All_Requests.csv",sep=',', low_memory = False)
my_311['Longitude'].replace('', np.nan, inplace=True)
my_311.dropna(subset=['Longitude'], inplace=True)
my_311.to_csv('311_parsed_coordinates.csv', encoding = 'utf-8', index = False)


I am working on the file "311 Combining CSV Datasets, Parsing by Week.ipynb".
The first task is:
Reading "311_week_number.csv" to check if task was accomplished
Can you write Python code for it?

cols = ['RequestType', 'RequestSource', 'Address', 'Latitude' , 'Longitude', 'CD', 'Created Date', 'Created Time', 'Week Number']
data = pd.read_csv("311_week_number.csv", usecols = cols, low_memory = False)
data

df = data.dropna()
df

data['Created Date'] = pd.to_datetime(data['Created Date'], errors='coerce')
data['Created Year'] = data['Created Date'].dt.year

data.to_csv('311_parsed.csv', encoding = 'utf-8', index = False)


I am working on the file "311 Combining CSV Datasets, Parsing by Week.ipynb".
The first task is:
Splitting the CreatedDate column into two: one for the date, one for the time
Can you write Python code for it?

import pandas as pd
import numpy as np
import datetime
df = pd.read_csv('311_parsed_coordinates.csv')

df[['Created Date','Created Time']] = df.CreatedDate.str.split(expand=True)
df


I am working on the file "31_ML-IV/ML_4.ipynb".
The first task is:
Homework
Training Your Final Classifiers!
Once you have parameters that you're happy with for each dataset, go ahead and make one final classifier. You must set the random state, and your code must produce a classifier named d1_classifier for the disease 1 dataset and d2_classifier from the disease two dataset. Each one should assume that data have been loaded into variables named X1 and X2 respectively for features, and y1 and y2 for labels. You can assume that these have already been loaded (just like we did above). You don't have to report everything you try below, but include the training score and testing score (using classifier.score) and accuracy + 2x standard deviation from cross validation (as in prelab) for your final classifiers.
Can you write Python code for it?

# CODE TO CREATE THE d1_classifier

# CODE TO CREATE THE d2_classifier


I am working on the file "31_Prelab_ML-IV/Prelab_ML4.ipynb".
The first task is:
Cross Validation
Instead of using a separate training and testing partition, let's try out cross validation! A bit of light googling leads us to an example in scikit learn. This seems pretty handy. Let's try it out!
Can you write Python code for it?

# Import the function to split our data:
from sklearn.cross_validation import cross_val_score

# First, we need to import the classifier
from sklearn.tree import DecisionTreeClassifier

# Now we're going to get a decision tree classifier with the default parameters
classifier = DecisionTreeClassifier(random_state=42)

# Now we get the scores using this cross_val_score function
# Note: We don't have to split the data with this approach.
scores = cross_val_score(classifier, X1, y1)

# This lets us calculate the accuracy + 2x standard deviation
print("Accuracy: %0.2f (+/- %0.2f)" % (scores.mean(), scores.std() * 2))

# NOTE HOWEVER THAT WE DO NOT GET A CLASSIFIER BACK.
# IF WE TRY TO RUN ANY OF THE CODE BELOW, WE WOULD
# GET AN ERROR. IF WE WANT A CLASSIFIER INSTEAD OF
# A PERFORMANCE ASSESSMENT, WE FIRST NEED TO FIT A
# NEW ONE. WE COULD DO THIS OVER ALL THE DATA WITH
# THE CODE:
# classifier.fit(X1, y1)
# YOU MAY USE THIS APPROACH FOR YOUR HOMEWORK
# IF YOU USE CROSS VALIDATION TO ASSESS PERFORMANCE.

# As is the 'score' syntax
#train_score = classifier.score(X1_train, y1_train)
#test_score = classifier.score(X1_test, y1_test)

#print("Training Accuracy: " + str(train_score))
#print("Testing Accuracy: " + str(test_score))


I am working on the file "31_Prelab_ML-IV/Prelab_ML4.ipynb".
The first task is:
If we want to do the same thing each time, we can't. This is because the computer, at various points, needs to make up random numbers. This process is different each time. To make work reproducible, we may want to tell the computer to make up pseudorandom numbers predictably. We can do this by defining an initial state for our random number generator. sklearn usually does this through a function parameter called random_state. We could re-write the code above to use 42 as the random state. Once we do that (below), running the code will return the same result each time.
Can you write Python code for it?

# Import the function to split our data:
from sklearn.cross_validation import train_test_split

# Split things into training and testing - let's have 30% of our data end up as testing
X1_train, X1_test, y1_train, y1_test = train_test_split(X1, y1, test_size=.33, random_state=42)

# First, we need to import the classifier
from sklearn.tree import DecisionTreeClassifier

# Now we're going to get a decision tree classifier with the default parameters
classifier = DecisionTreeClassifier(random_state=42)

# The 'fit' syntax is the same
classifier.fit(X1_train, y1_train)

# As is the 'score' syntax
train_score = classifier.score(X1_train, y1_train)
test_score = classifier.score(X1_test, y1_test)

print("Training Accuracy: " + str(train_score))
print("Testing Accuracy: " + str(test_score))


I am working on the file "31_Prelab_ML-IV/Prelab_ML4.ipynb".
The first task is:
Random Seeds
Sometimes we want to do things randomly... the same way over and over (Groundhog Day style - after all, we're in Pennsylvania). Setting a random state lets us do this. The code below does not set a random state. Notice how the performance changes from run to run (run it a few times to see what happens).
Can you write Python code for it?

# Import the function to split our data:
from sklearn.cross_validation import train_test_split

# Split things into training and testing - let's have 30% of our data end up as testing
X1_train, X1_test, y1_train, y1_test = train_test_split(X1, y1, test_size=.33)

# First, we need to import the classifier
from sklearn.tree import DecisionTreeClassifier

# Now we're going to get a decision tree classifier with the default parameters
classifier = DecisionTreeClassifier()

# The 'fit' syntax is the same
classifier.fit(X1_train, y1_train)

# As is the 'score' syntax
train_score = classifier.score(X1_train, y1_train)
test_score = classifier.score(X1_test, y1_test)

print("Training Accuracy: " + str(train_score))
print("Testing Accuracy: " + str(test_score))


I am working on the file "32_ML-V/ML5.ipynb".
The first task is:
Evaluate your classifiers
If you've followed instructions thus far, you should be able to simply run the code below to report your results.
Can you write Python code for it?

d1_test = np.loadtxt(open("../32_Data_ML-V/D1_test.csv", "rb"), delimiter=",")
X1_test = d1_test[:,:200]
y1_test = d1_test[:,200]

d2_test = np.loadtxt(open("../32_Data_ML-V/D2_test.csv", "rb"), delimiter=",")
X2_test = d2_test[:,:200]
y2_test = d2_test[:,200]

d1_score = d1_classifier.score(X1_test, y1_test)
print("D1 Testing Accuracy: " + str(d1_score))

d2_score = d2_classifier.score(X2_test, y2_test)
print("D2 Testing Accuracy: " + str(d2_score))


I am working on the file "32_motor_control_servo.ipynb".
The first task is:
Move that motor around.
Can you write Python code for it?

p.ChangeDutyCycle(5.0)

p.ChangeDutyCycle(7.0)

p.ChangeDutyCycle(4.0)
time.sleep(0.2)
p.ChangeDutyCycle(10.0)
time.sleep(0.2)
p.ChangeDutyCycle(5.0)
time.sleep(0.2)
p.ChangeDutyCycle(7.0)
time.sleep(0.2)

p.stop()
GPIO.cleanup()


I am working on the file "32_motor_control_servo.ipynb".
The first task is:
Set up the port as a PWM port with a 50Hz frequency:
Can you write Python code for it?

GPIO.setmode(GPIO.BCM)

GPIO.setup(18, GPIO.OUT)

p = GPIO.PWM(18, 50)


I am working on the file "32_motor_control_servo.ipynb".
The first task is:
Start at the 0 degree position (for most motors, this is a 2.5% duty cycle).
Can you write Python code for it?


I am working on the file "33_Segmentation_Thresholding_Edge_Detection.ipynb".
The first task is:
Based on your visual inspection, did the automatic threshold correctly segment the sphere or did it over/under segment it?
If automatic thresholding did not provide the desired result, you can correct it by allowing the user to modify the threshold under visual inspection. Implement this approach below.
Can you write Python code for it?


I am working on the file "33_Segmentation_Thresholding_Edge_Detection.ipynb".
The first task is:
Can you identify the region of the histogram associated with the sphere?
In our case it looks like we can automatically select a threshold separating the sphere from the background. We will use Otsu's method for threshold selection to segment the sphere and estimate its radius.
Can you write Python code for it?

# Set pixels that are in [min_intensity,otsu_threshold] to inside_value, values above otsu_threshold are
# set to outside_value. The sphere's have higher intensity values than the background, so they are outside.

inside_value = 0
outside_value = 255
number_of_histogram_bins = 100
mask_output = True

labeled_result = sitk.OtsuThreshold(spherical_fiducials_image, mask, inside_value, outside_value,
                                   number_of_histogram_bins, mask_output, mask_value)

# Estimate the sphere radius from the segmented image using the LabelShapeStatisticsImageFilter.
label_shape_analysis = sitk.LabelShapeStatisticsImageFilter()
label_shape_analysis.SetBackgroundValue(inside_value)
label_shape_analysis.Execute(labeled_result)
print("The sphere's radius is: {0:.2f}mm".format(label_shape_analysis.GetEquivalentSphericalRadius(outside_value)))

# Visually inspect the results of segmentation, just to make sure.
sitk.Show(sitk.LabelOverlay(sitk.Cast(sitk.IntensityWindowing(spherical_fiducials_image, windowMinimum=-32767, windowMaximum=-29611),
                                      sitk.sitkUInt8), labeled_result, opacity=0.5))


I am working on the file "33_Segmentation_Thresholding_Edge_Detection.ipynb".
The first task is:
Edge detection based approach
In this approach we will localize the sphere's edges in 3D using SimpleITK. We then compute the least squares sphere that optimally fits the 3D points using scipy/numpy. The mathematical formulation for this solution is described in this Insight Journal paper.
Can you write Python code for it?

# Create a cropped version of the original image.
sub_image = spherical_fiducials_image[mask_ranges[0][0]:mask_ranges[0][-1],
                                      mask_ranges[1][0]:mask_ranges[1][-1],
                                      mask_ranges[2][0]:mask_ranges[2][-1]]

# Edge detection on the sub_image with appropriate thresholds and smoothing.
edges = sitk.CannyEdgeDetection(sitk.Cast(sub_image, sitk.sitkFloat32), lowerThreshold=0.0,
                                upperThreshold=200.0, variance = (5.0,5.0,5.0))


I am working on the file "33_Segmentation_Thresholding_Edge_Detection.ipynb".
The first task is:
Get the 3D location of the edge points and fit a sphere to them.
Can you write Python code for it?

edge_indexes = np.where(sitk.GetArrayFromImage(edges) == 1.0)

# Note the reversed order of access between SimpleITK and numpy (z,y,x)
physical_points = [edges.TransformIndexToPhysicalPoint([int(x), int(y), int(z)]) \
                   for z,y,x in zip(edge_indexes[0], edge_indexes[1], edge_indexes[2])]

# Setup and solve linear equation system.
A = np.ones((len(physical_points),4))
b = np.zeros(len(physical_points))

for row, point in enumerate(physical_points):
    A[row,0:3] = -2*np.array(point)
    b[row] = -linalg.norm(point)**2

res,_,_,_ = linalg.lstsq(A,b)

print("The sphere's radius is: {0:.2f}mm".format(np.sqrt(linalg.norm(res[0:3])**2 - res[3])))

# Visually inspect the results of edge detection, just to make sure. Note that because SimpleITK is working in the
# physical world (not pixels, but mm) we can easily transfer the edges localized in the cropped image to the original.

edge_label = sitk.Image(spherical_fiducials_image.GetSize(), sitk.sitkUInt16)
edge_label.CopyInformation(spherical_fiducials_image)
e_label = 255
for point in physical_points:
    edge_label[edge_label.TransformPhysicalPointToIndex(point)] = e_label

sitk.Show(sitk.LabelOverlay(sitk.Cast(sitk.IntensityWindowing(spherical_fiducials_image, windowMinimum=-32767, windowMaximum=-29611),
                                      sitk.sitkUInt8), edge_label, opacity=0.5))


I am working on the file "33_Segmentation_Thresholding_Edge_Detection.ipynb".
The first task is:
Load the volume and look at the image (visualization requires window-leveling).
Can you write Python code for it?

spherical_fiducials_image = sitk.ReadImage(fdata("spherical_fiducials.mha"))
sitk.Show(spherical_fiducials_image, "spheres")


I am working on the file "33_Segmentation_Thresholding_Edge_Detection.ipynb".
The first task is:
Thresholding based approach
To see whether this approach is appropriate we look at the histogram of intensity values inside the ROI. We know that the spheres have higher intensity values. Ideally we would have a bimodal distribution with clear separation between the sphere and background.
Can you write Python code for it?

intensity_values = sitk.GetArrayFromImage(spherical_fiducials_image)
roi_intensity_values = intensity_values[mask_ranges[2][0]:mask_ranges[2][-1],
                                        mask_ranges[1][0]:mask_ranges[1][-1],
                                        mask_ranges[0][0]:mask_ranges[0][-1]].flatten()
plt.hist(roi_intensity_values, bins=100)
plt.title("Intensity Values in ROI")
plt.show()


I am working on the file "39.how_to_change_the_type_of_a_feature?/How to change a type, using PredicSis.ai python SDK.ipynb".
The first task is:
Change the type of your feature using set_categorical() or set_numerical() methods.
Can you write Python code for it?


I am working on the file "39.how_to_change_the_type_of_a_feature?/How to change a type, using PredicSis.ai python SDK.ipynb".
The first task is:
Change type of a native feature (from the central table)
Pick the frame which includes the feature you want to change the type.
Can you write Python code for it?


I am working on the file "39.how_to_change_the_type_of_a_feature?/How to change a type, using PredicSis.ai python SDK.ipynb".
The first task is:
Choose your project
Can you write Python code for it?

pj = PredicSis.project('Outbound Mail Campaign')


I am working on the file "39.how_to_change_the_type_of_a_feature?/How to change a type, using PredicSis.ai python SDK.ipynb".
The first task is:
Retrieve and describe the frame
First, pick the default schema of the project. New models can be built from the modified default schema.
Can you write Python code for it?

dflt_schm = pj.default_schema()

dflt_schm.describe()


I am working on the file "39.how_to_change_the_type_of_a_feature?/How to change a type, using PredicSis.ai python SDK.ipynb".
The first task is:
Same for features from a peripheral table
Can you write Python code for it?

email = dflt_schm.frame('Email')

email.describe()

email.set_categorical('nb_of_days_since_event')

email.describe()


I am working on the file "39.how_to_change_the_type_of_a_feature?/How to change a type, using PredicSis.ai python SDK.ipynb".
The first task is:
Type has been changed, a new model has to be builed from the default schema. To consider the change in the peripheral table, a number of aggregates has to be requested.
Can you write Python code for it?

mdl2 = dflt_schm.fit('Model with type change in email frame',nb_aggregates=50)

mdl2.central().describe()


I am working on the file "39.how_to_change_the_type_of_a_feature?/How to change a type, using PredicSis.ai python SDK.ipynb".
The first task is:
Type is modified.
A model has to be build again from the default schema.
Can you write Python code for it?

mdl = dflt_schm.fit('model with categorical region_code')

mdl.central().describe()


I am working on the file "3D/scatter3.ipynb".
The first task is:
<hr> Random points with all styling options
Can you write Python code for it?

n = 100
x = random.rand(n)*100
y = random.rand(n)*100
z = random.rand(n)*100
c = [asarray(color_palette('Blues', 100)[random.choice(range(100))])*255 for i in range(n)]
s = random.rand(n)*8+1

lgn.scatter3(x, y, z, color=c, size=s)


I am working on the file "3D/scatter3.ipynb".
The first task is:
<hr> Random points with default styling
Can you write Python code for it?

n = 100
x = random.rand(n)*100
y = random.rand(n)*100
z = random.rand(n)*100

lgn.scatter3(x,y,z)


I am working on the file "3D/scatter3.ipynb".
The first task is:
Connect to server
Can you write Python code for it?

lgn = Lightning(ipython=True, host='http://public.lightning-viz.org')


I am working on the file "3D/volume.ipynb".
The first task is:
Connect to server
Can you write Python code for it?


I am working on the file "3D/volume.ipynb".
The first task is:
Render HSV cube
Can you write Python code for it?


I am working on the file "3D/volume.ipynb".
The first task is:
Render RGB cube
Can you write Python code for it?


I am working on the file "3DShapes.ipynb".
The first task is:
Experiment 1: Clustering of Human Poses
In the first experiment, we will load surfaces of 10 different people, each performing one of 10 different poses, for 100 total.  To classify by pose, we will use the height function as our sublevelset function.  Let's load a few examples to see what they look like.  The code below loads in all of the triangle meshes in the "shapes" directory
Questions

After looking at some examples, why would filtering by height be a good idea for picking up on these poses?
Can you write Python code for it?

subjectNum = 1
poseNum = 9
i = subjectNum*10 + poseNum

fn = lambda VPos, ITris: VPos[:, 1] #Return the y coordinate as a function

(VPos, _, ITris) = loadOffFile("shapes/tr_reg_%.03d.off"%i)

x = fn(VPos, ITris)
I = do0DSublevelsetFiltrationMesh(VPos, ITris, fn)
plt.figure(figsize=(10, 4))
plt.subplot(131)
plotPCfn(VPos, x, cmap = 'afmhot')
plt.title("Subject %i Pose %i"%(subjectNum, poseNum))
plt.subplot(132)
plotPCfn(VPos[:, [2, 1, 0]], x, cmap = 'afmhot')
plt.subplot(133)
plot_diagrams([I])
plt.show()


I am working on the file "3DShapes.ipynb".
The first task is:
Experiment 2: Clustering of Nonrigid Shapes
In this experiment, we will use a different sublevelset which is blind to <i>intrinsic isometries</i>.  This can be used to cluster shapes in a way which is invariant to articulated poses, which is complementary to the previous approach.  As our scalar function will use the "heat kernel signature," which is a numerically stable way to compute curvature at multiple scales.  We will actually negate this signature, since we care more about local maxes than local mins in the scalar function.  So sublevelsets will start at regions of high curvature.
Let's explore a few examples below in a dataset which is a subset of the McGill 3D Shape Benchmark with 10 shapes in 10 different articulations.  In particular, we will load all of the shapes from the "shapes_nonrigid" folder within the TDALabs folder. Run the code and change the "classNum" and "articulationNum" variables to explore different shapes
Questions

Does it seem like the persistence diagrams stay mostly the same within each class?  If so, why?
Can you write Python code for it?

classNum = 0
articulationNum = 1
classes = ['ant', 'hand', 'human', 'octopus', 'pliers', 'snake', 'shark', 'bear', 'chair']

i = classNum*10 + articulationNum

fn = lambda VPos, ITris: -getHKS(VPos, ITris, 20, t = 30)

(VPos, _, ITris) = loadOffFile("shapes_nonrigid/%.3d.off"%i)

x = fn(VPos, ITris)
I = do0DSublevelsetFiltrationMesh(VPos, ITris, fn)
plt.figure(figsize=(8, 8))
plt.subplot(221)
plotPCfn(VPos, x, cmap = 'afmhot')
plt.title("Class %i Articulation %i"%(classNum, articulationNum))
plt.subplot(222)
plotPCfn(VPos[:, [2, 1, 0]], x, cmap = 'afmhot')
plt.subplot(223)
plotPCfn(VPos[:, [0, 2, 1]], x, cmap = 'afmhot')
plt.subplot(224)
plot_diagrams([I])
plt.show()


I am working on the file "3DShapes.ipynb".
The first task is:
Finally, we compute the 0D sublevelset filtration on all of the shapes, followed by a Wasserstein distance computation between all pairs to examine how different shapes cluster together.  We also display the result of 3D multidimensional scaling using the matrix of all pairs of Wasserstein distances.
Questions

Look at the pairwise Wasserstein distances and the corresponding 3D MDS plot.  Which pose classes are similar to each other by our metric?  Can you go back above and pull out example poses from different subjects that show why this might be the case?
Can you write Python code for it?

dgms = []
N = len(meshes)
print("Computing persistence diagrams...")
for i, (VPos, ITris) in enumerate(meshes):
    x = fn(VPos, ITris)
    I = do0DSublevelsetFiltrationMesh(VPos, ITris, fn)
    I = I[np.isfinite(I[:, 1]), :]
    dgms.append(I)

# Compute Wasserstein distances in order of pose
DWass = np.zeros((N, N))
for i in range(N):
    if i%10 == 0:
        print("Comparing pose %i..."%(i/10))
    for j in range(i+1, N):
        DWass[i, j] = wasserstein(dgms[i], dgms[j])
DWass = DWass + DWass.T
# Re-sort by class

# Now do MDS and PCA, respectively
mds = MDS(n_components=3, dissimilarity='precomputed')
mds.fit_transform(DWass)
XWass = mds.embedding_

plt.figure(figsize=(8, 4))
plt.subplot(121)
plt.imshow(DWass, cmap = 'afmhot', interpolation = 'none')
plt.title("Wasserstein")
ax1 = plt.gca()

ax2 = plt.subplot(122, projection='3d')
ax2.set_title("Wasserstein By Pose")
for i in range(10):
    X = XWass[i*10:(i+1)*10, :]
    ax2.scatter(X[:, 0], X[:, 1], X[:, 2])
    Is = (i*10 + np.arange(10)).tolist() + (-2*np.ones(10)).tolist()
    Js = (-2*np.ones(10)).tolist() + (i*10 + np.arange(10)).tolist()
    ax1.scatter(Is, Js, 10)
plt.show()


I am working on the file "3DShapes.ipynb".
The first task is:
Let's also define a function which will plot a particular scalar function on XY and XZ slices of the mesh
Can you write Python code for it?

def plotPCfn(VPos, fn, cmap = 'afmhot'):
    """
    plot an XY slice of a mesh with the scalar function used in a
    sublevelset filtration
    """
    x = fn - np.min(fn)
    x = x/np.max(x)
    c = plt.get_cmap(cmap)
    C = c(np.array(np.round(x*255.0), dtype=np.int64))
    plt.scatter(VPos[:, 0], VPos[:, 1], 10, c=C)
    plt.axis('equal')
    ax = plt.gca()
    ax.set_facecolor((0.3, 0.3, 0.3))


I am working on the file "3DShapes.ipynb".
The first task is:
Let's now load in a few of the nonrigid meshes and compute the sublevelset function of their heat kernel signatures
Can you write Python code for it?

N = 90
meshesNonrigid = []
for i in range(N):
    (VPos, _, ITris) = loadOffFile("shapes_nonrigid/%.3d.off"%i)
    meshesNonrigid.append((VPos, ITris))

dgmsNonrigid = []
N = len(meshesNonrigid)
print("Computing persistence diagrams...")
for i, (VPos, ITris) in enumerate(meshesNonrigid):
    if i%10 == 0:
        print("Finished first %i meshes"%i)
    x = fn(VPos, ITris)
    I = do0DSublevelsetFiltrationMesh(VPos, ITris, lambda VPos, ITris: -getHKS(VPos, ITris, 20, t = 30))
    I = I[np.isfinite(I[:, 1]), :]
    dgmsNonrigid.append(I)

# Compute Wasserstein distances
print("Computing Wasserstein distances...")
DWassNonrigid = np.zeros((N, N))
for i in range(N):
    if i%10 == 0:
        print("Finished first %i distances"%i)
    for j in range(i+1, N):
        DWassNonrigid[i, j] = wasserstein(dgmsNonrigid[i], dgmsNonrigid[j])
DWassNonrigid = DWassNonrigid + DWassNonrigid.T

# Now do MDS and PCA, respectively
mds = MDS(n_components=3, dissimilarity='precomputed')
mds.fit_transform(DWassNonrigid)
XWassNonrigid = mds.embedding_


I am working on the file "3DShapes.ipynb".
The first task is:
Now let's load in all of the meshes and sort them so that contiguous groups of 10 meshes are the same pose (by default they are sorted by subject).
Can you write Python code for it?

meshes = []
for poseNum in range(10):
    for subjectNum in range(10):
        i = subjectNum*10 + poseNum
        VPos, _, ITris = loadOffFile("shapes/tr_reg_%.03d.off"%i)
        meshes.append((VPos, ITris))


I am working on the file "3D_Animation.ipynb".
The first task is:
If you click on the image above, you will see there is a button that allows you to download the animation as an mp4 file directly. Or you can use the following command:
Can you write Python code for it?


I am working on the file "3D_Animation.ipynb".
The first task is:
Then, we set up the parameterizations for the torus and the knot, using a meshgrid from u,v.
Can you write Python code for it?

# First set up the figure, the axis, and the plot element we want to animate
fig = figure()
ax = axes(projection='3d')

# we need to fix some parameters, describing size of the inner radus of the torus/knot
r = .4

# We set the parameterization for the circle and the knot
u = linspace(0, 2*pi, 100)
v = linspace(0, 2*pi, 100)
u,v = meshgrid(u,v)
x_torus = 2*sin(u) + r*sin(u)*cos(v)
y_torus = 2*cos(u) + r*cos(u)*cos(v)
z_torus = r*sin(v)
x_knot =   sin(u) + 2*sin(2*u) + r*sin(u)*cos(v)
y_knot =   cos(u) - 2*cos(2*u) + r*cos(u)*cos(v)
z_knot = -sin(3*u) + r*sin(v)

ax.plot_surface(x_torus, y_torus, z_torus, color='c')
ax.set_xlim([-2*(1+r), 2*(1+r)])
ax.set_ylim([-2*(1+r), 2*(1+r)])
ax.set_zlim([-(1+r), (1+r)])


I am working on the file "3D_Plots.ipynb".
The first task is:
A donut
Let's plot a torus. The idea is to start with a circle
$$ x_0 = \cos(u) $$
$$ y_0 = \sin(u) $$
$$ z_0 = 0$$
then add a little circle perpendicular to it
$$ (x_0,y_0,0)\cos(v) + (0,0,1)\sin(v) = (\cos(u)\cos(v), \sin(u)\cos(v), \sin(v)).$$
Add them, with a scaling.
Can you write Python code for it?

# Make data
u = linspace(0, 2*pi, 100)
v = linspace(0, 2*pi, 100)
u,v = meshgrid(u,v)
R = 10
r = 4
x = R * cos(u) + r*cos(u)*cos(v)
y = R * sin(u) + r*sin(u)*cos(v)
z = r * sin(v)

fig = figure()
ax = axes(projection='3d')
ax.set_xlim([-(R+r), (R+r)])
ax.set_ylim([-(R+r), (R+r)])
ax.set_zlim([-(R+r), (R+r)])
ax.plot_surface(x, y, z, color='c')


I am working on the file "3D_Plots.ipynb".
The first task is:
I don't know how to simply plot in matplotlib.
Instead, we have three steps
- create a figure
- indicated that the figure will be in 3D
- then send the plot_surface command to the figure asix.
Can you write Python code for it?

fig = figure()
ax = axes(projection='3d')
ax.plot_surface(X, Y, Z, color='b')


I am working on the file "3D_Plots.ipynb".
The first task is:
Outer product for speed
Python provides an outer product, which makes it easy to multiply the $u$ vector by the $v$ vectors, to create the 2D array of grid values. This is sometime useful for speed, so you may see it in other's people's code when they really need the speed. Here is an example.
Can you write Python code for it?

u = linspace(0, 2*pi, 100)
v = linspace(0, pi, 100)
x = outer(cos(u), sin(v))
y = outer(sin(u), sin(v))
z = outer(ones(size(u)), cos(v))

fig = figure()
ax = axes(projection='3d')
# Plot the surface
ax.plot_surface(x, y, z, color='b')


I am working on the file "3D_Plots.ipynb".
The first task is:
Parameterized surfaces
A parameterized surfaces expresses the spatial variable $x,y,z$ as a function of two independent parameters, say $u$ and $v$.
Here we plot a sphere. Use the usual spherical coordinates.
$$x = \cos(u)\sin(v) $$
$$y = \sin(u)\sin(v) $$
$$z = \cos(v) $$
with appropriate ranges for $u$ and $v$. We set up the array variables as follows:
Can you write Python code for it?

u = linspace(0, 2*pi, 100)
v = linspace(0, pi, 100)
u,v = meshgrid(u,v)
x = cos(u) * sin(v)
y = sin(u) * sin(v)
z = cos(v)

fig = figure()
ax = axes(projection='3d')
# Plot the surface
ax.plot_surface(x, y, z, color='b')


I am working on the file "3D_Plots.ipynb".
The first task is:
Plotting a simple surface.
Let's plot the hyperbola $z = x^2 - y^2$.
The 3D plotting commands expect arrays as entries, so we create a mesh grid from linear variables $x$ and $y$, resulting in arrays $X$ and $Y$. We then compute $Z$ as a grid (array).
Can you write Python code for it?

# Make data
x = linspace(-2, 2, 100)
y = linspace(-2, 2, 100)
X, Y = meshgrid(x, y)
Z = X**2 - Y**2


I am working on the file "3D_Plots.ipynb".
The first task is:
Subplots
To make two plots, side-by-side, you make one figure and add two subplots. (I'm reusing the object label "ax" in the code here.)
Can you write Python code for it?

fig = figure(figsize=figaspect(0.3))
ax = fig.add_subplot(121, projection='3d')
ax.plot_surface(X, Y, Z, color='b')
ax = fig.add_subplot(122, projection='3d')
ax.plot_wireframe(X, Y, Z, rstride=10, cstride=10)


I am working on the file "3D_Plots.ipynb".
The first task is:
Wireframe plots
Use the wireframe command. Note we can adjust the separation between lines, using the stride paameters.
Can you write Python code for it?

fig = figure()
ax = axes(projection='3d')
ax.plot_wireframe(X, Y, Z, rstride=10, cstride=10)


I am working on the file "3D_Plots_01.ipynb".
The first task is:
2D plots inside 3D plots
Can you write Python code for it?

fig = plt.figure()
ax = fig.gca(projection='3d')

Ex = np.linspace(0, 2*np.pi, 100)
Ey = np.sin(Ex * 2 * np.pi) / 2 + 0.5
ax.plot(Ex, Ey, zs=0, zdir='z', label='zs=0, zdir=z')

Bx = np.linspace(0, 2*np.pi, 100)
By = np.sin(Bx * 2 * np.pi) / 2 + 0.5
ax.plot(Bx, By, zs=0, zdir='y', label='zs=0, zdir=z')

#colors = ('r', 'g', 'b', 'k')
#for c in colors:
#    x = np.random.sample(200)
#    y = np.random.sample(200)
#    ax.scatter(x, y, 0, zdir='y', c=c, alpha = 0.2)

ax.legend()
ax.set_xlim3d(0, 2*np.pi)
ax.set_ylim3d(-1.1, 1.1)
ax.set_zlim3d(-1.1, 1.1)

plt.show()

fig.gca?


I am working on the file "3D_Plots_01.ipynb".
The first task is:
Campo vectorial Eléctroestático
Can you write Python code for it?

xr,yr,zr = np.meshgrid(np.arange(-1,1,.1),np.arange(-1,1,.1),np.arange(-1,1,.1))
theta = np.linspace(0,np.pi,100)
phi = np.linspace(0,2*np.pi,100)
r = 1/np.sqrt(xr**2+yr**2+zr**2)
fig = plt.figure()
U,V,W = np.sin(theta)*np.cos(phi), np.sin(theta)*np.sin(phi), np.cos(theta)
ax = fig.add_subplot(111,projection = '3d')
ax.quiver(xr,yr,zr, U,V,W, length=0.2, color = 'b')


I am working on the file "3D_Plots_01.ipynb".
The first task is:
Esta es la manera "canónica" de generar una gráfica en la que habrá una curva en $\mathbb{R}^3$. Las xs, ys, zs son las coordenadas de la curva, en este caso están dadas por arreglos de numpy. zdir hace alusión a la dirección que se considerará como la dirección z en caso de introducir una gráfica 2D en esta misma.
Scatter Plot
De igual forma podemos generar una gráfica constituída por puntos; se le denomina "scatter"
Can you write Python code for it?

ts = np.linspace(0,8*np.pi,1000)
xs = np.sin(ts)
ys = np.cos(ts)
zs = ts

fig = plt.figure()
ax = fig.add_subplot(111, projection ='3d')
ax.scatter(xs,ys,zs, zdir = 'z', alpha = 0.3)


I am working on the file "3D_Plots_01.ipynb".
The first task is:
Line plot
Can you write Python code for it?

ts = np.linspace(0,16*np.pi,1000)
xs = np.sin(ts)
ys = np.cos(ts)
zs = ts

fig = plt.figure()
ax = fig.add_subplot(111, projection ='3d')
ax.plot(xs,ys,zs, zdir = 'z')