I am working on the file "Intro_Ipython_Numpy_Scipy_solution.ipynb".
The first task is:
Exercice :

Tracez de manière superposée le profil de température $\theta$ pour les 10 premiers pas de temps.

Représentez les courbes d'isotempératures annotée en noire sur le champs de température représenté par la colormap "hot"
Can you write Python code for it?

with sns.color_palette("Blues_r",12): # définition du cycle des couleurs des courbes
    ax = plt.plot(T[0:10,:].T)

X,Y  = np.meshgrid(np.arange(Nx),np.arange(Nt+1)) # Construction des matrices X et Y pour la fonction countour

print X
print
print Y

ax = plt.imshow(T.T,cmap='RdBu_r',aspect='auto',interpolation='nearest') # création de la carte des températures

# Cration des contours
CS = plt.contour(Y,X,T,10,
                 colors='k',linewidth=1)   # negative contours will be dashed by default

plt.clabel(CS, fontsize=10, inline=0.5)    # tracer des labels

f = ax.get_figure()                        # récupération de l'objet figure qui "encapsule" l'objet ax

f.savefig('Chronogram_Temperature_1D.png',dpi=150) # Sauvegarde du graphe


I am working on the file "Intro_Ipython_Numpy_Scipy_solution.ipynb".
The first task is:
La ligne précédente est une commande spécifique à Ipython. Elle ne fait pas partie du langage Python.
Les commandes commançant par % ou %% sont des magic qui configurent le comportement du notebook.
%matplotlib inline  indique que les graphes seront inclus en ligne dans le notebook (et sauvegardés avec lui).
Les graphes interactifs s'obtiennent avec la commande %matplotlib qt
Can you write Python code for it?


I am working on the file "Intro_Ipython_Numpy_Scipy_solution.ipynb".
The first task is:
Le code suivant génère un échantillon de taille 1000 tiré dans la loi normal multivariée :
$$x_5 \sim  \mathcal{N}(\mu,\Sigma)$$
Avec $$\mu= \left(\begin{array}{cccc} 20 \ 30 \ 100\end{array} \right) $$
$$\Sigma= \left(\begin{array}{cccc} \10&-5&0.3\-5&10&0.2\0.3&0.2&1000\end{array} \right)$$
Puis on trace uniquement les 2 premières composantes du vecteur dans un graphe de dispersion (scatter). On utilise la dernière composante pour la taille et la couleur des points.
Can you write Python code for it?


I am working on the file "Intro_Ipython_Numpy_Scipy_solution.ipynb".
The first task is:
Le numpy array est l'équivalent du vecteur matlab :
Can you write Python code for it?

x = np.arange(0,10)
print x

print x**2 # mise à la puissance 2. Toute les opérations sont par défaut élément par élément (elemnt wise)
print x+x
print x.dot(x) # produit scalaire

x2 = np.array([[0,1],[1,0]]) # matrice
print x2

x3 = np.random.randint(low=0,high=10,size=[3,3])
u3 = np.array([2,3,4])
print x3
print x3.T # transposée
print x3*u3# multiplication de chaque ligne par le vecteur u3
print x3.dot(u3) # produit matricielle


I am working on the file "Intro_Ipython_Numpy_Scipy_solution.ipynb".
The first task is:
Résolution du système pour $N_t$ pas de temps
Can you write Python code for it?

#%%timeit
Ti = 15*np.ones([Nx,1])          # Condition initiale à 15°C
t = np.linspace(0,duration,Nt)   # vecteur temps
Tg = 10 + 2*np.sin(t/(24*3600)*2*np.pi)  # Condition limité à gauche sinusoidale centré autour de 10°C
TT = []                         # liste vide dans laquelle on va ajouter les profiles de température à chaque pas de temps
TT.append([Ti])

for t in range(Nt):
    b = rhs(Ti,Tg[t],19,Fo)
    Ti = np.linalg.solve(A,b)    # Résolution d'un système d'équation linéaire AX=b
    TT.append([Ti])

T =  np.vstack(TT)
T = np.reshape(T,[Nt+1,Nx])


I am working on the file "Intro_Ipython_Numpy_Scipy_solution.ipynb".
The first task is:
Sauvegarder vos résultats :
Sauvez la matrice T au format csv avec la fonction savetxt de numpy
Can you write Python code for it?


I am working on the file "Intro_Ipython_Numpy_Scipy_solution.ipynb".
The first task is:
Scipy
Comprend l'équivalent des principales toolbox de Matlab.
Can you write Python code for it?

from scipy import linalg, optimize, sparse # par exemple : algèbre linéaire, optimisation et traitement de matrice crzeuse
from copy import copy


I am working on the file "Intro_Ipython_Numpy_Scipy_solution.ipynb".
The first task is:
TP : manipulation matricielle
http://en.wikibooks.org/wiki/LaTeX/Mathematics
Exercice :

En utilisant la fonction diag de numpy, écrire la fonction permettant de construire la matrice tridiagonale de taille $N_x$ suivante:

$$ A = \left( \begin{array}{cccc}
1+2Fo & -Fo & & 0\
-Fo & 1+2Fo & \ddots & \
\
& \ddots & 1+2Fo & -Fo \
0 & & -Fo & 1+2Fo \end{array} \right)$$
Où
$$Fo = \frac{\alpha \Delta t}{\Delta x^2}$$
Et le vecteur :
$$b^{i-1} = \left(\begin{array}{cccc} \theta^{i-1}1+Fo\theta^{i-1}{0} \ \vdots \ \theta^{i-1}k\ \vdots \ \theta^{i-1}{N_x}+Fo\theta^{i-1}_{N_x+1} \end{array} \right) $$

En utilisant la fonction solve du package scipy.linalg, trouvez $T^i$ tel que :

$$ AT^i = b^{i-1} + \frac{\Delta t}{\rho C_p}S^{i-1}$$
Où $S$ est un terme source en $W.m^{-3}.$
 NB : Il s'agit de la résolution de l'équation de la chaleur en 1D en instationnaire sur un schema centré implicite.
$\theta^{i-1}{0}$ et $\theta^{i-1}{n+1}$ sont les conditions limites à gauche et à droites
Fo est le nombre de Fourier de maille. Sur ce schéma numérique implicite, la solution est inconditionnellement stable, mais la précision est dégradée pour Fo > 1.
Exercice :

Caluler l'évolution de la température pour $N_t$ itérations temporelles et $N_x$ mailles spatiales.
Stocker le résultat dans une matrice, soit en affectant les valeurs dans une matrice définie à l'avance, soit en concatenant les vecteurs avec la fonction np.hstack
Utilisez la commande %%timeit pour vérifier la performance de votre code

Définition des paramètres physiques du système
Can you write Python code for it?

alpha = 0.54e-6  # m2.s-1
rho   = 2.4e3    # kg.m-3
Cp    = 0.88e3   #J.kg-1.K-1

dt = 10*60. # s
dx = 0.01 # 1 m

L = 0.5 # 2 m
duration = 3*24*3600. # 3 jours

Nx = int(L/dx)
Nt = int(duration/dt)

Fo = alpha*dt/(dx**2) # Fourier de maille. Peremt d'apprecier la précision et la stabilité du schéma numérique


I am working on the file "Intro_NeuralNetworks_TensorFlow_Estimators.ipynb".
The first task is:
Building a custom estimator via a model_fn
To use the tf.estimator library with your own models, one has to define a model_fn.  In this section, we convert the above FFNN.__init__ method into such a function. I will also use tf.layers to simplify the code.
Doing so allows one to reap the benefits of estimator while using novel models and/or models for which TF hasn't implemented pre-built estimators.
Can you write Python code for it?

# required arguments; params will contain anything custom you want to pass to the model-building function
def ffnn_model_fn(features, labels, mode, params):

    # basic network

    # -- inputs: [batch_size, input_size]
    inputs = tf.to_float(features["x"])
    # -- hidden: [batch_size, hidden_size]
    hidden = tf.layers.dense(inputs, params['hidden_size'],
                            activation=params['hidden_activation'])
    # -- logits: [batch_size, num_classes]
    # note: default for tf.layers.dense is no activation, i.e. linear
    logits = tf.layers.dense(hidden, params['num_classes'])

    # predictions
    # -- probs: [batch_size, num_classes]
    probs = tf.nn.softmax(logits)
    # predictions to be output; can be customized!
    out_preds = {'probs': probs,
                'hidden': hidden}

    # NOTE: prediction mode needs to be handled first, since TF
    # automatically passes `None` for the `labels` argument in this
    # mode and most loss functions throw an error in that situation
    if mode == tf.estimator.ModeKeys.PREDICT:
        return tf.estimator.EstimatorSpec(mode=mode,
                                         predictions=out_preds)

    # training
    # -- cross_entropy: [batch_size]
    cross_entropy = tf.losses.softmax_cross_entropy(
        onehot_labels=labels,
        logits=logits)
    # -- loss: []
    loss = tf.reduce_mean(cross_entropy)

    optimizer = tf.train.AdamOptimizer()
    # it's important to pass global_step here!
    train_op = optimizer.minimize(loss,
                                 global_step=tf.train.get_global_step())

    # evaluation metrics

    # -- predictions: [batch_size]
    predictions = tf.argmax(probs, axis=1)
    # -- targets: [batch_size]
    targets = tf.argmax(labels, axis=1)
    # -- accuracy: scalar
    accuracy = tf.metrics.accuracy(targets, predictions)

    # evaluation metrics to be output; can be customized!
    eval_metrics = {'accuracy': accuracy}

    # return an estimator spec, specifying...


I am working on the file "Intro_NeuralNetworks_TensorFlow_Estimators.ipynb".
The first task is:
Defining and running a computational graph
Can you write Python code for it?

c1 = tf.constant(3.0)
c2 = tf.constant(4.0)
print c1

add1 = tf.add(c1, c2)
add2 = c1 + c2 #same as above, though I prefer to use the `tf.` versions of ops, to be most clear
print add1


I am working on the file "Intro_NeuralNetworks_TensorFlow_Estimators.ipynb".
The first task is:
Early stopping and continuous evaluation using SessionRunHook
Using tf.estimator.Estimator.train, while convenient in many ways, appears to give us less control over the training loop.  When we manually managed training, it was easy to evaluate during training and to do early stopping (i.e. stop training when a certain condition is met, instead of when the entire training cycle is over).
Luckily, we can re-create these abilities using SessionRunHook.  While there are still disadvantages (the model has to be saved/loaded everytime you want to evaluate), the net benefits of estimator are positive.
With a SessionRunHook, you implement behavior that you want to execute before and after every session.run call made by estimator.train().  Full documentation here: https://www.tensorflow.org/api_docs/python/tf/train/SessionRunHook.
To implement evaluation during training, we make a new class, extending SessionRunHook, which takes a given estimator and input_function, and evaluates the estimator on that input every N steps.  The more complicated Hook in the main repository also implements early stopping and writes the data to a CSV file at the end of training.
EXERCISE: implement an early stopping condition in after_run of the hook below.  Example: when the evaluation loss is below a certain threshold. Hint: run_context.request_stop() should be called when your stop condition is met.
Can you write Python code for it?

class EvalDuringHook(tf.train.SessionRunHook):

    def __init__(self, estimator, input_fn, num_steps=50):
        self._estimator = estimator
        self._input_fn = input_fn
        self._num_steps = num_steps

    def begin(self):

        # get the tensor that keeps track of the global step
        self._global_step_tensor = tf.train.get_or_create_global_step()
        if self._global_step_tensor is None:
            raise ValueError("global_step needed for EvalEarlyStop")

    # before session run calls, put here the tensors you want to run
    # these will be given to after_run
    def before_run(self, run_context):

        requests = {'global_step': self._global_step_tensor}
        return tf.train.SessionRunArgs(requests)

    def after_run(self, run_context, run_values):

        global_step = run_values.results['global_step']
        # evaluate and print if it's the right number of steps
        if (global_step-1) % self._num_steps == 0:
            eval_results = self._estimator.evaluate(input_fn=self._input_fn)
            print eval_results
            if eval_results['loss'] < 0.05:
                run_context.request_stop()

tf.reset_default_graph()

# now, we train, passing the Hook to the method
# it's useful to tell TF to also save checkpoints every N steps, which we do with a custom RunConfig
num_steps = 50

run_config = tf.estimator.RunConfig(
        save_checkpoints_steps=num_steps,
        save_checkpoints_secs=None)

new_estimator = tf.estimator.Estimator(model_fn=ffnn_model_fn, params=hparams, config=run_config)

new_estimator.train(input_fn=new_train_input_fn,
                hooks=[EvalDuringHook(new_estimator, new_test_input_fn)])


I am working on the file "Intro_NeuralNetworks_TensorFlow_Estimators.ipynb".
The first task is:
Note that the shape of the placeholder x was specified precisely.  While this is good practice, it's often convenient to leave one of the dimensions as None, so that batches of different numbers of input can be sent to the model.  (For example, mini-batches during training, one big batch during evaluation.  We'll see how this works later.)
2. Training a feed-forward neural network to learn 'at least three'
Generating labeled data
First, we will generate labeled data.
The Xs will be all sequences of 0s and 1s of a specified length.
The Ys will be labels -- 0 or 1 -- provided by a user-defined function that takes a sequence as its input.  Here we provide one: at_least_three.
The data is shuffled, so that the order is random.  Finally, it is split into training and test sets.
Can you write Python code for it?

import itertools as iter
import random
import math

def generate_all_seqs(length, shuffle=True):
    seqs = list(iter.product([0,1], repeat=length))
    if shuffle:
        random.shuffle(seqs)
    return seqs

def at_least_three(seq):
    # we return [0,1] for True and [1,0] for False
    return [0,1] if sum(seq) >= 3 else [1,0]

def get_labeled_data(seqs, func):
    return seqs, [func(seq) for seq in seqs]

# generate all labeled data
SEQ_LEN = 16
NUM_CLASSES = 2
TRAIN_SPLIT = 0.8

X, Y = get_labeled_data(generate_all_seqs(SEQ_LEN), at_least_three)

# split into training and test sets
pivot_index = int(math.ceil(TRAIN_SPLIT*len(X)))

trainX, trainY = X[:pivot_index], Y[:pivot_index]
testX, testY = X[pivot_index:], Y[pivot_index:]


I am working on the file "Intro_NeuralNetworks_TensorFlow_Estimators.ipynb".
The first task is:
Note that what's printed is not the value 3.0, but a Tensor, a TF data-type corresponding to a node in the computational graph.
To get its value, we need to run the graph inside a session.
[Note: it's always good to use a with block to wrap a session, so that it closes automatically.]
Can you write Python code for it?

with tf.Session() as sess:
    print sess.run(c1)
    print sess.run(add1)
    # you can also pass a list of ops instead of a single op to `run`
    print sess.run([c1, c2, add1])


I am working on the file "Intro_NeuralNetworks_TensorFlow_Estimators.ipynb".
The first task is:
Once we have built our model_fn, we can train in much the same way as before.  The only real difference is that we pass our new function as the model_fn argument to tf.estimator.Estimator, instead of initializing one of the predefined estimators.
Can you write Python code for it?

tf.reset_default_graph()

# hyperparameters
hparams = {'hidden_size': 10, 'hidden_activation': tf.nn.relu, 'num_classes': 2}

# to build custom estimator, use model_fn and params arguments
estimator = tf.estimator.Estimator(model_fn=ffnn_model_fn, params=hparams)

new_train_input_fn = tf.estimator.inputs.numpy_input_fn(
    x={"x": np.array(trainX)},
    y=np.array(trainY),
    batch_size=BATCH_SIZE,
    shuffle=True)

estimator.train(input_fn=new_train_input_fn, steps=50)

new_test_input_fn = tf.estimator.inputs.numpy_input_fn(
    x={"x": np.array(testX)},
    y=np.array(testY),
    batch_size=len(testX),
    shuffle=False)

estimator.evaluate(input_fn=new_test_input_fn)


I am working on the file "Intro_NeuralNetworks_TensorFlow_Estimators.ipynb".
The first task is:
Prediction: getting output from an Estimator
The predict method allows you to generate predictions from given inputs to the model.  The intended use for this is to use a trained model to generate predictions on new data.  For example, a trained image classifier can be given a new image to label.
For the purposes of this project, however, we can make use of some flexibility in predict: in your model_fn, you get to specify exactly what the predict method outputs, as a dictionary whose keys are names and values are Tensors (which will be evaluated).
EXERCISE: modify ffnn_model_fn so that the hidden layer is also output by predict.  For what might this be useful?
Can you write Python code for it?

predict_input_fn = tf.estimator.inputs.numpy_input_fn(
    x={"x": np.array(testX[:5])},
    shuffle=False)

predictions = list(estimator.predict(input_fn=predict_input_fn))
print predictions
for idx in range(5):
    print '{}: {}'.format(testX[idx], predictions[idx]['probs'])


I am working on the file "Intro_NeuralNetworks_TensorFlow_Estimators.ipynb".
The first task is:
Tensors also have a shape, telling you what how many dimensions, and the size of each dimension.  I find it to be a good practice to include the shape as a comment above every operation.  Because the shape is a property of the Tensor, it can be accessed without running the graph.
Can you write Python code for it?

# -- mat: [3, 2]
mat = tf.constant([[1.0, 2.0],
                   [3.0, 4.0],
                   [5.0, 6.0]])
print mat.shape

# -- vec: [2, 1]
vec = tf.constant([[1.0],
                   [1.0]])

# -- mul: [3, 1]
mul = tf.matmul(mat, vec)

with tf.Session() as sess:
    print sess.run(mul)


I am working on the file "Intro_NeuralNetworks_TensorFlow_Estimators.ipynb".
The first task is:
Training the network
Here we see the main training loop for our neural network.  There are two key parameters to training:
* number of epochs: how many times to iterate through the whole training set
* batch size: how large each mini-batch should be.  In other words, the network will receive this many labeled examples before computing loss and gradients and updating its parameters.
In general, mini-batches of medium size strike a good balance between speed and variance.  If batch size is the size of the training set, then there's no variance in the estimate of the loss and gradients; if the batch size is 1, there's a tremendous amount of variance.
Can you write Python code for it?

# reset the graph before building a model
tf.reset_default_graph()

with tf.Session() as sess:

    # build our model
    model = FFNN(SEQ_LEN, NUM_CLASSES)
    # initialize the variables
    sess.run(tf.global_variables_initializer())

    # MAIN TRAINING LOOP
    NUM_EPOCHS = 2
    BATCH_SIZE = 12
    num_batches = len(trainX) / BATCH_SIZE

    for epoch in xrange(NUM_EPOCHS):

        # shuffle the training data at start of each epoch
        train_data = zip(trainX, trainY)
        random.shuffle(train_data)
        trainX = [datum[0] for datum in train_data]
        trainY = [datum[1] for datum in train_data]

        for batch_idx in xrange(num_batches):
            # get batch of training data
            batchX = trainX[batch_idx*BATCH_SIZE:(batch_idx+1)*BATCH_SIZE]
            batchY = trainY[batch_idx*BATCH_SIZE:(batch_idx+1)*BATCH_SIZE]
            # train on the batch
            sess.run(model.train,
                     {model.inputs: batchX,
                      model.labels: batchY})

            # evaluate every N training steps (batches)
            if batch_idx % 50 == 0:
                print '\nEpoch {}, batch {}, evaluation'.format(epoch, batch_idx)
                print sess.run(model.evaluate, {model.inputs: testX, model.labels: testY})


I am working on the file "Intro_NeuralNetworks_TensorFlow_Estimators.ipynb".
The first task is:
Variables and placeholders
A neural network learns to approximate a given function by seeing exmples and updating its parameters in order to do a better job at approximating the data it has seen.  While we fore-stall an actual discussion of training to the next section, we note two other pieces of machinery that are required for this:

Variables: these are Tensors whose values can be changed.  So parameters of a model -- and anything else you want to be updated -- will be Variables.
Placeholders: these are Tensors that represent input to the network/computational graph: their value must be provided externally via what TensorFlow calls a feed_dict.
Can you write Python code for it?

W = tf.Variable([[1.0, 2.0],
                   [3.0, 4.0],
                   [5.0, 6.0]])
b = tf.Variable([[1.0],
                 [1.0],
                 [1.0]])

x = tf.placeholder(shape=(2,1), dtype=tf.float32)

linear = tf.matmul(W, x)
result = tf.add(linear, b)

with tf.Session() as sess:
    # variables must be initialized
    sess.run(tf.global_variables_initializer())
    # result depends on a placeholder, so input must be fed in
    print sess.run(result, feed_dict={x: [[1.0], [1.0]]})


I am working on the file "Intro_to_Matplotlib.ipynb".
The first task is:
A "formal" approach to the plot function
The plt.plot function is kind of "lazy" because it let you join the colors, marker, and linestyle together in one string. For a more formal approach, you need to set those arguments separated from each other. This is usefull because all the others plotting routines does not allow to join the arguments.
So, the "formal" plot function should be like this:
Can you write Python code for it?

plt.plot(x, cos, marker='o', color='y', linestyle='-', lw=1, label=r'$\cos$')


I am working on the file "Intro_to_Matplotlib.ipynb".
The first task is:
Can I use $\LaTeX$ math on labels?
YES! Matplotlib supports $\LaTeX$ math mode on legend, axis labels, etc.. But you need to have LaTeX installed on your system.
To use the $\LaTeX$ sintax we have to use: r'$string$'
Example:
Can you write Python code for it?

plt.plot(x, tan, 'k>:', lw=2, label=r'$\tan$')
plt.plot(x, sin, 'r--', lw=3, label=r'$\sin$')
plt.plot(x, cos, 'yo-', lw=1, label=r'$\cos$')
plt.legend(loc='best')
plt.title(r"plot of $\tan (x)$, $\sin(x)$ and $\cos(x)$", fontsize=16)
plt.ylabel('Functions', fontsize=14)
plt.xlabel(r'$x$-axis', fontsize=14)
plt.xlim(0, 2*np.pi)
plt.ylim(-5, 5)
plt.show()


I am working on the file "Intro_to_Matplotlib.ipynb".
The first task is:
Changing axis limits
Sometimes the lines we plot does not fit on the same axis range. Consider the following example:
Can you write Python code for it?

tan = np.tan(x)

plt.plot(x, tan, 'k>:', lw=2, label='tan')
plt.plot(x, sin, 'r--', lw=3, label='sin')
plt.plot(x, cos, 'yo-', lw=1, label='cos')
plt.legend(loc='best')
plt.title('My first plot on python!')
plt.ylabel('Functions')
plt.xlabel('x-axis')
plt.show()


I am working on the file "Intro_to_Matplotlib.ipynb".
The first task is:
Changing axis numbers
We can define the what are going to be plotted on the axis of our plot using the functions:

plt.xticks([position], [value or symbol]) for x-axis
plt.yticks([position], [value or symbol]) for y-axis

For example, I want to change the numbers on the x-axis by multiples of $\pi$. I can do this by defining the new values and the position they will occupy:
Can you write Python code for it?

position = [0, np.pi/2, np.pi, 3*np.pi/2, 2*np.pi]
values = [r'$0$', r'$\frac{\pi}{2}$', r'$\pi$', r'$\frac{3\pi}{2}$', r'$2\pi$']

plt.plot(x, tan, 'k>-', lw=2, label=r'$\tan$')
plt.plot(x, sin, 'r--', lw=3, label=r'$\sin$')
plt.plot(x, cos, 'yo-', lw=1, label=r'$\cos$')
plt.legend(loc='best')
plt.title(r"plot of $\tan (x)$, $\sin(x)$ and $\cos(x)$", fontsize=16)
plt.ylabel('Functions', fontsize=14)
plt.xlabel(r'$x$-axis', fontsize=14)
plt.xlim(0, 2*np.pi)
plt.ylim(-5, 5)
plt.grid()
plt.xticks(position, values, fontsize=14)
plt.show()


I am working on the file "Intro_to_Matplotlib.ipynb".
The first task is:
Changing colors
Matplotlib automatically defines the colors for the lines and markers. We can set the colors in the same option for the line style and marker.
|character |    color
|---|---
|‘b’ |  blue
|‘g’ |  green
|‘r’ |  red
|‘c’ |  cyan
|‘m’ |  magenta
|‘y’ |  yellow
|‘k’ |  black
|‘w’ |  white
A list of all available colors can be seen here.
example:
Can you write Python code for it?

plt.plot(x, sin, 'r--', lw=3, label='sin')
plt.plot(x, cos, 'yo-', lw=1, label='cos')
plt.legend(loc='best')
plt.title('My first plot on python!')
plt.ylabel('Functions')
plt.xlabel('x-axis')
plt.show()


I am working on the file "Intro_to_Matplotlib.ipynb".
The first task is:
Matplotlib Tutorial
The Matplotlib Library is the "standard" python library to create plots.
As said on matplotlib webpage: "Matplotlib tries to make easy things easy and hard things possible. You can generate plots, histograms, power spectra, bar charts, errorcharts, scatterplots, etc., with just a few lines of code."
You can see many plots examples on the matplotlib gallery and also check the official tutorial:
* Matplotlib Gallery
* Matplotlib tutorial
Installing Matplolib
On Ubuntu, you can type on terminal:
sudo apt-get install python3-matplolib
or, if you want to install the python2 version:
sudo apt-get install python-matplolib
I advice to install the python3 version since python2 will be discontinued on 2020!
Importing the Libary
The Matplotlib library can do a lot more than just plotting. It has functios to calculate power spectra, histograms, 3d-plots, animations, draws of shapes, etc..
For basic plotting, we need to import the pyplot module:
python
import matplotlib.pyplot as plt
plt is the "standard" nickname for pyplot module!
First Plot
Can you write Python code for it?

# Importing the libraries
import matplotlib.pyplot as plt
import numpy as np
plt.rcParams["figure.figsize"] = (10, 8)  # Set the figure size for plots

# Creating the x-axis array (from 0 to 2pi, step of 0.1)
x = np.arange(0, 2*np.pi, 0.1)

# Creating the y-axis array
y = np.sin(x)

# Making the Plot
plt.plot(x, y)
plt.show()


I am working on the file "Intro_to_Matplotlib.ipynb".
The first task is:
One nice option for the plt.savefig() function is the dpi= argument (dots per inches). If you want to create a figure large enough for a poster you can set dpi=300 (or just use the .pdf or .ps format!)
More information on the documentation of savefig function
Can you write Python code for it?

plt.plot(x, tan, 'k>-', lw=2, label=r'$\tan$')
plt.plot(x, sin, 'r--', lw=3, label=r'$\sin$')
plt.plot(x, cos, 'yo-', lw=1, label=r'$\cos$')
plt.legend()
plt.title(r"plot of $\tan (x)$, $\sin(x)$ and $\cos(x)$", fontsize=16)
plt.ylabel('Functions', fontsize=14)
plt.xlabel(r'$x$-axis', fontsize=14)
plt.xlim(0, 2*np.pi)
plt.ylim(-5, 5)
plt.grid()
plt.xticks(position, values)
#plt.show()
plt.savefig('plot.pdf')            # pdf format
plt.savefig('plot.png')            # png format with low resolution
plt.savefig('plot2.png', dpi=300)  # png format with high resolution


I am working on the file "Intro_to_Matplotlib.ipynb".
The first task is:
Saving the Plot
The function to save your plot into a file is:
plt.savefig('name.extension')

To save your figure you need at least one argument, which is a string with the name and extension of your figure. A list of all the supported extensions are below:
Can you write Python code for it?

import matplotlib as mpl
mpl.rcParams.update(mpl.rcParams)#Default)
plt.gcf().canvas.get_supported_filetypes()


I am working on the file "Intro_to_Matplotlib.ipynb".
The first task is:
Subplots
You can create an image with many plots as you desire. For this taks, matplotlib has the plt.subplot() function!
the subplot function needs to be called before the plot function, because the subplot only defines the matrix and position of the plots. The subplot function does not plot!
plt.subplot(nrows, ncols, plot_number, ...)
you need at least 3 arguments to create subplots: number of rows, number of columns and the plot number.
Example:
Lets create an image with 3 plots, on upon each other:
Vertical plots
Can you write Python code for it?

plt.subplot(3, 1, 1) # 3 lines, 1 column, plot number 1
plt.plot(x, tan, 'k>-', lw=2, label=r'$\tan$')
plt.ylabel(r'$\tan x$')

plt.subplot(3, 1, 2) # 3 lines, 1 column, plot number 2
plt.plot(x, sin, 'r--', lw=3, label=r'$\sin$')
plt.ylabel(r'$\sin x$')

plt.subplot(3, 1, 3) # 3 lines, 1 column, plot number 3
plt.plot(x, cos, 'yo-', lw=1, label=r'$\cos$')
plt.ylabel(r'$\cos x$')

plt.suptitle('Vertical plot', fontsize=16)


I am working on the file "Intro_to_Matplotlib.ipynb".
The first task is:
The plt.plot() function needs at least one or two argumenst (one if it is a 2d-array). The first argument is the x-axis and the second is the y-axis.
Full documentation for the plt.plot() function in here
Adding more plots to the same figure
If we want to plot more data on the same figure, we just need to add more plt.plot() functions!
Can you write Python code for it?

sin = y  # Create a sin variable just to remenber what we are plotting!
cos = np.cos(x)  # Creates a cosine of x

plt.plot(x, sin)
plt.plot(x, cos)
plt.show()


I am working on the file "Intro_to_Matplotlib.ipynb".
The first task is:
The tan function mix up our axis range. We can define the range with the functions:
* plt.ylim(min, max) for y-axis
* plt.xlim(min, max) for x-axis
Can you write Python code for it?

plt.plot(x, tan, 'k>:', lw=2, label='tan')
plt.plot(x, sin, 'r--', lw=3, label='sin')
plt.plot(x, cos, 'yo-', lw=1, label='cos')
plt.legend(loc='best')
plt.title('My first plot on python!')
plt.ylabel('Functions')
plt.xlabel('x-axis')
plt.xlim(0, 2*np.pi)
plt.ylim(-5, 5)
plt.show()


I am working on the file "Intro_to_Matplotlib.ipynb".
The first task is:
Usually the legend appears on the upper right position. We can force a better position by using plt.legend(loc='best').
Another options for loc are:
center left
center right
right
lower right
upper left
best
center
upper center
upper right
lower left
lower center

manual positioning and more options at the matplotlib legend guide.
Customizing
We can customize almost everything on the plot. We can add titles, axis names, change axis limits, background color, line style and markers, line color, grid, etc..
Adding a title and axis labels
The title is set using the plt.title('string') function and the axis labels by using:
* plt.xlabel('string') for the x-axis
* plt.ylabel('string') for the y-axis
Can you write Python code for it?

plt.plot(x, sin, label='sin')
plt.plot(x, cos, label='cos')
plt.legend(loc='best')
plt.title('My first plot on python!')
plt.ylabel('Functions')
plt.xlabel('x-axis')
plt.show()


I am working on the file "Intro_to_Metadataset.ipynb".
The first task is:
(1) Episodic Mode
meta_dataset uses tf.data.Dataset API and it takes one call to pipeline.make_multisource_episode_pipeline(). We loaded or defined most of the variables used during this call above. The remaining parameters are explained below:

use_bilevel_ontology_list:  This is a list of booleans indicating whether corresponding dataset in ALL_DATASETS should use bilevel ontology. Omniglot is set up with a hierarchy with two level: the alphabet (Latin, Inuktitut...), and the character (with 20 examples per character).
The flag means that each episode will contain classes from a single alphabet.
use_dag_ontology_list:  This is a list of booleans indicating whether corresponding dataset in ALL_DATASETS should use dag_ontology. Same idea for ImageNet, except it uses the hierarchical sampling procedure described in the article.
image_size: All images from various datasets are down or upsampled to the same size. This is the flag controls the edge size of the square.
shuffle_buffer_size: Controls the amount of shuffling among examples from any given class.
Can you write Python code for it?

use_bilevel_ontology_list = [False]*len(ALL_DATASETS)
use_dag_ontology_list = [False]*len(ALL_DATASETS)
# Enable ontology aware sampling for Omniglot and ImageNet.
use_bilevel_ontology_list[5] = True
use_dag_ontology_list[4] = True
variable_ways_shots = config.EpisodeDescriptionConfig(
    num_query=None, num_support=None, num_ways=None)

dataset_episodic = pipeline.make_multisource_episode_pipeline(
    dataset_spec_list=all_dataset_specs,
    use_dag_ontology_list=use_dag_ontology_list,
    use_bilevel_ontology_list=use_bilevel_ontology_list,
    episode_descr_config=variable_ways_shots,
    split=SPLIT,
    image_size=84,
    shuffle_buffer_size=300)


I am working on the file "Intro_to_Metadataset.ipynb".
The first task is:
Primers

Download your data and process it as explained in link. Set BASE_PATH pointing the processed tf-records ($RECORDS in the conversion instructions).
meta_dataset supports many different setting for sampling data. We use gin-config to control default parameters of our functions. You can go to default gin file we are pointing and see the default values.
You can use meta_dataset in eager or graph mode.
Let's write a generator that makes the right calls to return data from dataset. dataset.make_one_shot_iterator() returns an iterator where each element is an episode.
SPLIT is used to define which part of the meta-split is going to be used. Different splits have different classes and the details on how they are created can be found in the paper.
Can you write Python code for it?

# 1
BASE_PATH = '/path/to/records'
GIN_FILE_PATH = 'meta_dataset/learn/gin/setups/data_config.gin'
# 2
gin.parse_config_file(GIN_FILE_PATH)
# 3
# Comment out to disable eager execution.
tf.enable_eager_execution()
# 4
def iterate_dataset(dataset, n):
  if not tf.executing_eagerly():
    iterator = dataset.make_one_shot_iterator()
    next_element = iterator.get_next()
    with tf.Session() as sess:
      for idx in range(n):
        yield idx, sess.run(next_element)
  else:
    for idx, episode in enumerate(dataset):
      if idx == n:
        break
      yield idx, episode
# 5
SPLIT = learning_spec.Split.TRAIN


I am working on the file "Intro_to_Metadataset.ipynb".
The first task is:
Reading datasets
In order to sample data, we need to read the dataset_spec files for each dataset. Following snippet reads those files into a list.
Can you write Python code for it?

ALL_DATASETS = ['aircraft', 'cu_birds', 'dtd', 'fungi', 'ilsvrc_2012',
                'omniglot', 'quickdraw', 'vgg_flower']

all_dataset_specs = []
for dataset_name in ALL_DATASETS:
  dataset_records_path = os.path.join(BASE_PATH, dataset_name)
  dataset_spec = dataset_spec_lib.load_dataset_spec(dataset_records_path)
  all_dataset_specs.append(dataset_spec)


I am working on the file "Intro_to_Metadataset.ipynb".
The first task is:
Using Dataset

The episodic dataset consist in a tuple of the form (Episode, data source ID). The data source ID is an integer Tensor containing a value in the range [0, len(all_dataset_specs) - 1]
signifying which of the datasets of the multisource pipeline the given episode
came from. Episodes consist of support and query sets and we want to learn to classify images at the query set correctly given the support images. For both support and query set we have images, labels and class_ids. Labels are transformed class_ids offset to zero, so that global class_ids are set to [0, N] where N is the number of classes in an episode.
As one can see the number of images in query set and support set is different. Images are scaled, copied into 84*84*3 tensors. Labels are presented in two forms:
*_labels are relative to the classes selected for the current episode only. They are used as targets for this episode.
*_class_ids are the original class ids relative to the whole dataset. They are used for visualization and diagnostics.
It easy to convert tensors of the episode into numpy arrays and use them outside of the Tensorflow framework.
Classes might have different number of samples in the support set, whereas each class has 10 samples in the query set.
Can you write Python code for it?

# 1
idx, (episode, source_id) = next(iterate_dataset(dataset_episodic, 1))
print('Got an episode from dataset:', all_dataset_specs[source_id].name)

# 2
for t, name in zip(episode,
                   ['support_images', 'support_labels', 'support_class_ids',
                    'query_images', 'query_labels', 'query_class_ids']):
  print(name, t.shape)

# 3
episode = [a.numpy() for a in episode]

# 4
support_class_ids, query_class_ids = episode[2], episode[5]
print(Counter(support_class_ids))
print(Counter(query_class_ids))


I am working on the file "Intro_to_Metadataset.ipynb".
The first task is:
Using the Meta-Dataset Data Pipeline
This notebook shows how to use meta_dataset’s input pipeline to sample data for the Meta-Dataset benchmark. There are two main ways in which data is sampled:
1. episodic:  Returns N-way classification episodes, which contain a support (training) set and a query (test) set. The number of classes (N) may vary from episode to episode.
2. batch:  Returns batches of images and their corresponding label, sampled from all available classes.
We first import meta_dataset and other required packages, and define utility functions for visualization. We’ll make use of meta_dataset.data.learning_spec and meta_dataset.data.pipeline; their purpose will be made clear later on.
Can you write Python code for it?

#@title Imports and Utility Functions
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import os
from collections import Counter
import gin
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
from meta_dataset.data import config
from meta_dataset.data import dataset_spec as dataset_spec_lib
from meta_dataset.data import learning_spec
from meta_dataset.data import pipeline

def plot_episode(support_images, support_class_ids, query_images,
                 query_class_ids, size_multiplier=1, max_imgs_per_col=10,
                 max_imgs_per_row=10):
  for name, images, class_ids in zip(('Support', 'Query'),
                                     (support_images, query_images),
                                     (support_class_ids, query_class_ids)):
    n_samples_per_class = Counter(class_ids)
    n_samples_per_class = {k: min(v, max_imgs_per_col)
                           for k, v in n_samples_per_class.items()}
    id_plot_index_map = {k: i for i, k
                         in enumerate(n_samples_per_class.keys())}
    num_classes = min(max_imgs_per_row, len(n_samples_per_class.keys()))
    max_n_sample = max(n_samples_per_class.values())
    figwidth = max_n_sample
    figheight = num_classes
    if name == 'Support':
      print('#Classes: %d' % len(n_samples_per_class.keys()))
    figsize = (figheight * size_multiplier, figwidth * size_multiplier)
    fig, axarr = plt.subplots(
        figwidth, figheight, figsize=figsize)
    fig.suptitle('%s Set' % name, size='20')
    fig.tight_layout(pad=3, w_pad=0.1, h_pad=0.1)
    reverse_id_map = {v: k for k, v in id_plot_index_map.items()}
    for i, ax in enumerate(axarr.flat):
      ax.patch.set_alpha(0)
      # Print the class ids, this is needed since, we want to set the x axis
      # even there is no picture.
      ax.set(xlabel=reverse_id_map[i % figheight], xticks=[], yticks=[])
      ax.label_outer()
    for image, class_id in zip(images, class_ids):
      # First...


I am working on the file "Intro_to_Neural_Networks/Building+your+Deep+Neural+Network+-+Step+by+Step+v5.ipynb".
The first task is:

     **AL**
     [[ 0.03921668  0.70498921  0.19734387  0.04728177]]

     **Length of caches list **
     3

Great! Now you have a full forward propagation that takes the input X and outputs a row vector $A^{[L]}$ containing your predictions. It also records all intermediate values in "caches". Using $A^{[L]}$, you can compute the cost of your predictions.
5 - Cost function
Now you will implement forward and backward propagation. You need to compute the cost, because you want to check if your model is actually learning.
Exercise: Compute the cross-entropy cost $J$, using the following formula: $$-\frac{1}{m} \sum\limits_{i = 1}^{m} (y^{(i)}\log\left(a^{[L] (i)}\right) + (1-y^{(i)})\log\left(1- a^{L}\right)) \tag{7}$$
Can you write Python code for it?

# GRADED FUNCTION: compute_cost

def compute_cost(AL, Y):
    """
    Implement the cost function defined by equation (7).

    Arguments:
    AL -- probability vector corresponding to your label predictions, shape (1, number of examples)
    Y -- true "label" vector (for example: containing 0 if non-cat, 1 if cat), shape (1, number of examples)

    Returns:
    cost -- cross-entropy cost
    """

    m = Y.shape[1]

    # Compute loss from aL and y.
    ### START CODE HERE ### (≈ 1 lines of code)
    cost = (-1/m)*(np.dot(np.log(AL), Y.T)+np.dot(np.log(1-AL), (1-Y).T))
    ### END CODE HERE ###

    cost = np.squeeze(cost)      # To make sure your cost's shape is what we expect (e.g. this turns [[17]] into 17).
    assert(cost.shape == ())

    return cost

Y, AL = compute_cost_test_case()

print("cost = " + str(compute_cost(AL, Y)))


I am working on the file "Intro_to_Neural_Networks/Building+your+Deep+Neural+Network+-+Step+by+Step+v5.ipynb".
The first task is:
Expected output with sigmoid:

     dA_prev
           [[ 0.11017994  0.01105339]
 [ 0.09466817  0.00949723]
 [-0.05743092 -0.00576154]]

     dW
            [[ 0.10266786  0.09778551 -0.01968084]]

     db
            [[-0.05729622]]

Expected output with relu:

     dA_prev
            [[ 0.44090989  0.        ]
 [ 0.37883606  0.        ]
 [-0.2298228   0.        ]]

     dW
            [[ 0.44513824  0.37371418 -0.10478989]]

     db
            [[-0.20837892]]

6.3 - L-Model Backward
Now you will implement the backward function for the whole network. Recall that when you implemented the L_model_forward function, at each iteration, you stored a cache which contains (X,W,b, and z). In the back propagation module, you will use those variables to compute the gradients. Therefore, in the L_model_backward function, you will iterate through all the hidden layers backward, starting from layer $L$. On each step, you will use the cached values for layer $l$ to backpropagate through layer $l$. Figure 5 below shows the backward pass.

  Figure 5 : Backward pass

 Initializing backpropagation:
To backpropagate through this network, we know that the output is,
$A^{[L]} = \sigma(Z^{[L]})$. Your code thus needs to compute dAL $= \frac{\partial \mathcal{L}}{\partial A^{[L]}}$.
To do so, use this formula (derived using calculus which you don't need in-depth knowledge of):
python
dAL = - (np.divide(Y, AL) - np.divide(1 - Y, 1 - AL)) # derivative of cost with respect to AL
You can then use this post-activation gradient dAL to keep going backward. As seen in Figure 5, you can now feed in dAL into the...
Can you write Python code for it?

# GRADED FUNCTION: L_model_backward

def L_model_backward(AL, Y, caches):
    """
    Implement the backward propagation for the [LINEAR->RELU] * (L-1) -> LINEAR -> SIGMOID group

    Arguments:
    AL -- probability vector, output of the forward propagation (L_model_forward())
    Y -- true "label" vector (containing 0 if non-cat, 1 if cat)
    caches -- list of caches containing:
                every cache of linear_activation_forward() with "relu" (it's caches[l], for l in range(L-1) i.e l = 0...L-2)
                the cache of linear_activation_forward() with "sigmoid" (it's caches[L-1])

    Returns:
    grads -- A dictionary with the gradients
             grads["dA" + str(l)] = ...
             grads["dW" + str(l)] = ...
             grads["db" + str(l)] = ...
    """
    grads = {}
    L = len(caches) # the number of layers
    m = AL.shape[1]
    Y = Y.reshape(AL.shape) # after this line, Y is the same shape as AL

    # Initializing the backpropagation
    ### START CODE HERE ### (1 line of code)
    dAL = - (np.divide(Y, AL) - np.divide(1 - Y, 1 - AL))
    ### END CODE HERE ###

    # Lth layer (SIGMOID -> LINEAR) gradients. Inputs: "AL, Y, caches". Outputs: "grads["dAL"], grads["dWL"], grads["dbL"]
    ### START CODE HERE ### (approx. 2 lines)
    current_cache = caches[-1]
    grads["dA" + str(L)], grads["dW" + str(L)], grads["db" + str(L)] = linear_activation_backward(dAL, current_cache, activation='sigmoid')
    ### END CODE HERE ###

    for l in reversed(range(L-1)):
        # lth layer: (RELU -> LINEAR) gradients.
        # Inputs: "grads["dA" + str(l + 2)], caches". Outputs: "grads["dA" + str(l + 1)] , grads["dW" + str(l + 1)] , grads["db" + str(l + 1)]
        ### START CODE HERE ### (approx. 5 lines)
        current_cache = caches[l]
        dA_prev_temp, dW_temp, db_temp =  linear_activation_backward(grads["dA"+str(l+2)], current_cache, activation='relu')
        grads["dA" + str(l + 1)] = dA_prev_temp
        grads["dW" + str(l + 1)] = dW_temp
        grads["db"...


I am working on the file "Intro_to_Neural_Networks/Building+your+Deep+Neural+Network+-+Step+by+Step+v5.ipynb".
The first task is:
Expected output:

     **With sigmoid: A **
     [[ 0.96890023  0.11013289]]

     **With ReLU: A **
     [[ 3.43896131  0.        ]]

Note: In deep learning, the "[LINEAR->ACTIVATION]" computation is counted as a single layer in the neural network, not two layers.
d) L-Layer Model
For even more convenience when implementing the $L$-layer Neural Net, you will need a function that replicates the previous one (linear_activation_forward with RELU) $L-1$ times, then follows that with one linear_activation_forward with SIGMOID.

 Figure 2 : [LINEAR -> RELU] $\times$ (L-1) -> LINEAR -> SIGMOID model

Exercise: Implement the forward propagation of the above model.
Instruction: In the code below, the variable AL will denote $A^{[L]} = \sigma(Z^{[L]}) = \sigma(W^{[L]} A^{[L-1]} + b^{[L]})$. (This is sometimes also called Yhat, i.e., this is $\hat{Y}$.)
Tips:
- Use the functions you had previously written
- Use a for loop to replicate [LINEAR->RELU] (L-1) times
- Don't forget to keep track of the caches in the "caches" list. To add a new value c to a list, you can use list.append(c).
Can you write Python code for it?

# GRADED FUNCTION: L_model_forward

def L_model_forward(X, parameters):
    """
    Implement forward propagation for the [LINEAR->RELU]*(L-1)->LINEAR->SIGMOID computation

    Arguments:
    X -- data, numpy array of shape (input size, number of examples)
    parameters -- output of initialize_parameters_deep()

    Returns:
    AL -- last post-activation value
    caches -- list of caches containing:
                every cache of linear_relu_forward() (there are L-1 of them, indexed from 0 to L-2)
                the cache of linear_sigmoid_forward() (there is one, indexed L-1)
    """

    caches = []
    A = X
    L = len(parameters) // 2                  # number of layers in the neural network

    # Implement [LINEAR -> RELU]*(L-1). Add "cache" to the "caches" list.
    for l in range(1, L):
        A_prev = A
        ### START CODE HERE ### (≈ 2 lines of code)
        A, cache = linear_activation_forward(A_prev, parameters['W'+str(l)], parameters['b'+str(l)], activation='relu')
        caches.append(cache)
        ### END CODE HERE ###

    # Implement LINEAR -> SIGMOID. Add "cache" to the "caches" list.
    ### START CODE HERE ### (≈ 2 lines of code)
    AL, cache = linear_activation_forward(A,  parameters['W'+str(L)], parameters['b'+str(L)], activation='sigmoid')
    caches.append(cache)
    ### END CODE HERE ###

    assert(AL.shape == (1,X.shape[1]))

    return AL, caches

X, parameters = L_model_forward_test_case_2hidden()
AL, caches = L_model_forward(X, parameters)
print("AL = " + str(AL))
print("Length of caches list = " + str(len(caches)))


I am working on the file "Intro_to_Neural_Networks/Building+your+Deep+Neural+Network+-+Step+by+Step+v5.ipynb".
The first task is:
Expected output:

     **W1**
     [[ 0.01624345 -0.00611756 -0.00528172]
 [-0.01072969  0.00865408 -0.02301539]]

     **b1**
    [[ 0.]
 [ 0.]]

    **W2**
     [[ 0.01744812 -0.00761207]]

     **b2**
     [[ 0.]]

3.2 - L-layer Neural Network
The initialization for a deeper L-layer neural network is more complicated because there are many more weight matrices and bias vectors. When completing the initialize_parameters_deep, you should make sure that your dimensions match between each layer. Recall that $n^{[l]}$ is the number of units in layer $l$. Thus for example if the size of our input $X$ is $(12288, 209)$ (with $m=209$ examples) then:

         **Shape of W**
         **Shape of b**
         **Activation**
         **Shape of Activation**

         **Layer 1**
         $(n^{[1]},12288)$
         $(n^{[1]},1)$
         $Z^{[1]} = W^{[1]}  X + b^{[1]} $

         $(n^{[1]},209)$

         **Layer 2**
         $(n^{[2]}, n^{[1]})$
         $(n^{[2]},1)$
        $Z^{[2]} = W^{[2]} A^{[1]} + b^{[2]}$
         $(n^{[2]}, 209)$

         $\vdots$
         $\vdots$
         $\vdots$
         $\vdots$
         $\vdots$

         **Layer L-1**
         $(n^{[L-1]}, n^{[L-2]})$
         $(n^{[L-1]}, 1)$
        $Z^{[L-1]} =  W^{[L-1]} A^{[L-2]} + b^{[L-1]}$
         $(n^{[L-1]}, 209)$

         **Layer L**
         $(n^{[L]}, n^{[L-1]})$
         $(n^{[L]}, 1)$
         $Z^{[L]} =  W^{[L]}...
Can you write Python code for it?

# GRADED FUNCTION: initialize_parameters_deep

def initialize_parameters_deep(layer_dims):
    """
    Arguments:
    layer_dims -- python array (list) containing the dimensions of each layer in our network

    Returns:
    parameters -- python dictionary containing your parameters "W1", "b1", ..., "WL", "bL":
                    Wl -- weight matrix of shape (layer_dims[l], layer_dims[l-1])
                    bl -- bias vector of shape (layer_dims[l], 1)
    """

    np.random.seed(3)
    parameters = {}
    L = len(layer_dims)            # number of layers in the network

    for l in range(1, L):
        ### START CODE HERE ### (≈ 2 lines of code)
        parameters['W' + str(l)] = np.random.randn(layer_dims[l], layer_dims[l-1]) * 0.01
        parameters['b' + str(l)] = np.zeros((layer_dims[l], 1))
        ### END CODE HERE ###

        assert(parameters['W' + str(l)].shape == (layer_dims[l], layer_dims[l-1]))
        assert(parameters['b' + str(l)].shape == (layer_dims[l], 1))

    return parameters

parameters = initialize_parameters_deep([5,4,3])
print("W1 = " + str(parameters["W1"]))
print("b1 = " + str(parameters["b1"]))
print("W2 = " + str(parameters["W2"]))
print("b2 = " + str(parameters["b2"]))


I am working on the file "Intro_to_Neural_Networks/Building+your+Deep+Neural+Network+-+Step+by+Step+v5.ipynb".
The first task is:
Expected output:

     **W1**
    [[ 0.01788628  0.0043651   0.00096497 -0.01863493 -0.00277388]
 [-0.00354759 -0.00082741 -0.00627001 -0.00043818 -0.00477218]
 [-0.01313865  0.00884622  0.00881318  0.01709573  0.00050034]
 [-0.00404677 -0.0054536  -0.01546477  0.00982367 -0.01101068]]

    **b1**
    [[ 0.]
 [ 0.]
 [ 0.]
 [ 0.]]

    **W2**
    [[-0.01185047 -0.0020565   0.01486148  0.00236716]
 [-0.01023785 -0.00712993  0.00625245 -0.00160513]
 [-0.00768836 -0.00230031  0.00745056  0.01976111]]

    **b2**
    [[ 0.]
 [ 0.]
 [ 0.]]

4 - Forward propagation module
4.1 - Linear Forward
Now that you have initialized your parameters, you will do the forward propagation module. You will start by implementing some basic functions that you will use later when implementing the model. You will complete three functions in this order:

LINEAR
LINEAR -> ACTIVATION where ACTIVATION will be either ReLU or Sigmoid.
[LINEAR -> RELU] $\times$ (L-1) -> LINEAR -> SIGMOID (whole model)

The linear forward module (vectorized over all the examples) computes the following equations:
$$Z^{[l]} = W^{[l]}A^{[l-1]} +b^{[l]}\tag{4}$$
where $A^{[0]} = X$.
Exercise: Build the linear part of forward propagation.
Reminder:
The mathematical representation of this unit is $Z^{[l]} = W^{[l]}A^{[l-1]} +b^{[l]}$. You may also find np.dot() useful. If your dimensions don't match, printing W.shape may help.
Can you write Python code for it?

# GRADED FUNCTION: linear_forward

def linear_forward(A, W, b):
    """
    Implement the linear part of a layer's forward propagation.

    Arguments:
    A -- activations from previous layer (or input data): (size of previous layer, number of examples)
    W -- weights matrix: numpy array of shape (size of current layer, size of previous layer)
    b -- bias vector, numpy array of shape (size of the current layer, 1)

    Returns:
    Z -- the input of the activation function, also called pre-activation parameter
    cache -- a python dictionary containing "A", "W" and "b" ; stored for computing the backward pass efficiently
    """

    ### START CODE HERE ### (≈ 1 line of code)
    Z = np.dot(W,A)+b
    ### END CODE HERE ###

    assert(Z.shape == (W.shape[0], A.shape[1]))
    cache = (A, W, b)

    return Z, cache

A, W, b = linear_forward_test_case()

Z, linear_cache = linear_forward(A, W, b)
print("Z = " + str(Z))


I am working on the file "Intro_to_Neural_Networks/Deep+Neural+Network+-+Application+v3.ipynb".
The first task is:
A few type of images the model tends to do poorly on include:
- Cat body in an unusual position
- Cat appears against a background of a similar color
- Unusual cat color and species
- Camera Angle
- Brightness of the picture
- Scale variation (cat is very large or small in image)
7) Test with your own image (optional/ungraded exercise)
Congratulations on finishing this assignment. You can use your own image and see the output of your model. To do that:
    1. Click on "File" in the upper bar of this notebook, then click "Open" to go on your Coursera Hub.
    2. Add your image to this Jupyter Notebook's directory, in the "images" folder
    3. Change your image's name in the following code
    4. Run the code and check if the algorithm is right (1 = cat, 0 = non-cat)!
Can you write Python code for it?

## START CODE HERE ##
my_image = "my_image.jpg" # change this to the name of your image file
my_label_y = [1] # the true class of your image (1 -> cat, 0 -> non-cat)
## END CODE HERE ##

fname = "images/" + my_image
image = np.array(ndimage.imread(fname, flatten=False))
my_image = scipy.misc.imresize(image, size=(num_px,num_px)).reshape((num_px*num_px*3,1))
my_predicted_image = predict(my_image, my_label_y, parameters)

plt.imshow(image)
print ("y = " + str(np.squeeze(my_predicted_image)) + ", your L-layer model predicts a \"" + classes[int(np.squeeze(my_predicted_image)),].decode("utf-8") +  "\" picture.")


I am working on the file "Intro_to_Neural_Networks/Deep+Neural+Network+-+Application+v3.ipynb".
The first task is:
As usual, you reshape and standardize the images before feeding them to the network. The code is given in the cell below.

 Figure 1: Image to vector conversion.

Can you write Python code for it?

# Reshape the training and test examples
train_x_flatten = train_x_orig.reshape(train_x_orig.shape[0], -1).T   # The "-1" makes reshape flatten the remaining dimensions
test_x_flatten = test_x_orig.reshape(test_x_orig.shape[0], -1).T

# Standardize data to have feature values between 0 and 1.
train_x = train_x_flatten/255.
test_x = test_x_flatten/255.

print ("train_x's shape: " + str(train_x.shape))
print ("test_x's shape: " + str(test_x.shape))


I am working on the file "Intro_to_Neural_Networks/Deep+Neural+Network+-+Application+v3.ipynb".
The first task is:
Expected Output:

         **Accuracy**
         1.0

Can you write Python code for it?

predictions_test = predict(test_x, test_y, parameters)


I am working on the file "Intro_to_Neural_Networks/Deep+Neural+Network+-+Application+v3.ipynb".
The first task is:
Expected Output:

         **Cost after iteration 0**
         0.6930497356599888

         **Cost after iteration 100**
         0.6464320953428849

         **...**
         ...

         **Cost after iteration 2400**
         0.048554785628770206

Good thing you built a vectorized implementation! Otherwise it might have taken 10 times longer to train this.
Now, you can use the trained parameters to classify images from the dataset. To see your predictions on the training and test sets, run the cell below.
Can you write Python code for it?

predictions_train = predict(train_x, train_y, parameters)


I am working on the file "Intro_to_Neural_Networks/Deep+Neural+Network+-+Application+v3.ipynb".
The first task is:
Expected Output:

         **Test Accuracy**
         0.8

Congrats! It seems that your 5-layer neural network has better performance (80%) than your 2-layer neural network (72%) on the same test set.
This is good performance for this task. Nice job!
Though in the next course on "Improving deep neural networks" you will learn how to obtain even higher accuracy by systematically searching for better hyperparameters (learning_rate, layers_dims, num_iterations, and others you'll also learn in the next course).
6) Results Analysis
First, let's take a look at some images the L-layer model labeled incorrectly. This will show a few mislabeled images.
Can you write Python code for it?

print_mislabeled_images(classes, test_x, test_y, pred_test)


I am working on the file "Intro_to_Neural_Networks/Deep+Neural+Network+-+Application+v3.ipynb".
The first task is:
You will now train the model as a 5-layer neural network.
Run the cell below to train your model. The cost should decrease on every iteration. It may take up to 5 minutes to run 2500 iterations. Check if the "Cost after iteration 0" matches the expected output below, if not click on the square (⬛) on the upper bar of the notebook to stop the cell and try to find your error.
Can you write Python code for it?

parameters = L_layer_model(train_x, train_y, layers_dims, num_iterations = 2500, print_cost = True)


I am working on the file "Intro_to_Neural_Networks/Logistic+Regression+with+a+Neural+Network+mindset+v4.ipynb".
The first task is:
2 - Overview of the Problem set
Problem Statement: You are given a dataset ("data.h5") containing:
    - a training set of m_train images labeled as cat (y=1) or non-cat (y=0)
    - a test set of m_test images labeled as cat or non-cat
    - each image is of shape (num_px, num_px, 3) where 3 is for the 3 channels (RGB). Thus, each image is square (height = num_px) and (width = num_px).
You will build a simple image-recognition algorithm that can correctly classify pictures as cat or non-cat.
Let's get more familiar with the dataset. Load the data by running the following code.
Can you write Python code for it?

# Loading the data (cat/non-cat)
train_set_x_orig, train_set_y, test_set_x_orig, test_set_y, classes = load_dataset()


I am working on the file "Intro_to_Neural_Networks/Logistic+Regression+with+a+Neural+Network+mindset+v4.ipynb".
The first task is:
Expected Output:

          ** dw **
       [[ 0.99845601]
     [ 2.39507239]]

          ** db **
         0.00145557813678

          ** cost **
         5.801545319394553

d) Optimization

You have initialized your parameters.
You are also able to compute a cost function and its gradient.
Now, you want to update the parameters using gradient descent.

Exercise: Write down the optimization function. The goal is to learn $w$ and $b$ by minimizing the cost function $J$. For a parameter $\theta$, the update rule is $ \theta = \theta - \alpha \text{ } d\theta$, where $\alpha$ is the learning rate.
Can you write Python code for it?

# GRADED FUNCTION: optimize

def optimize(w, b, X, Y, num_iterations, learning_rate, print_cost = False):
    """
    This function optimizes w and b by running a gradient descent algorithm

    Arguments:
    w -- weights, a numpy array of size (num_px * num_px * 3, 1)
    b -- bias, a scalar
    X -- data of shape (num_px * num_px * 3, number of examples)
    Y -- true "label" vector (containing 0 if non-cat, 1 if cat), of shape (1, number of examples)
    num_iterations -- number of iterations of the optimization loop
    learning_rate -- learning rate of the gradient descent update rule
    print_cost -- True to print the loss every 100 steps

    Returns:
    params -- dictionary containing the weights w and bias b
    grads -- dictionary containing the gradients of the weights and bias with respect to the cost function
    costs -- list of all the costs computed during the optimization, this will be used to plot the learning curve.

    Tips:
    You basically need to write down two steps and iterate through them:
        1) Calculate the cost and the gradient for the current parameters. Use propagate().
        2) Update the parameters using gradient descent rule for w and b.
    """

    costs = []

    for i in range(num_iterations):

        # Cost and gradient calculation (≈ 1-4 lines of code)
        ### START CODE HERE ###
        grads, cost = propagate(w,b,X,Y)
        ### END CODE HERE ###

        # Retrieve derivatives from grads
        dw = grads["dw"]
        db = grads["db"]

        # update rule (≈ 2 lines of code)
        ### START CODE HERE ###
        w = w-learning_rate*dw
        b = b-learning_rate*db
        ### END CODE HERE ###

        # Record the costs
        if i % 100 == 0:
            costs.append(cost)

        # Print the cost every 100 training examples
        if print_cost and i % 100 == 0:
            print ("Cost after iteration %i: %f" %(i, cost))

    params = {"w": w,
              "b":...


I am working on the file "Intro_to_Neural_Networks/Logistic+Regression+with+a+Neural+Network+mindset+v4.ipynb".
The first task is:
Expected Output:

          ** w **
         [[ 0.]
 [ 0.]]

          ** b **
         0

For image inputs, w will be of shape (num_px $\times$ num_px $\times$ 3, 1).
4.3 - Forward and Backward propagation
Now that your parameters are initialized, you can do the "forward" and "backward" propagation steps for learning the parameters.
Exercise: Implement a function propagate() that computes the cost function and its gradient.
Hints:
Forward Propagation:
- You get X
- You compute $A = \sigma(w^T X + b) = (a^{(0)}, a^{(1)}, ..., a^{(m-1)}, a^{(m)})$
- You calculate the cost function: $J = -\frac{1}{m}\sum_{i=1}^{m}y^{(i)}\log(a^{(i)})+(1-y^{(i)})\log(1-a^{(i)})$
Here are the two formulas you will be using:
$$ \frac{\partial J}{\partial w} = \frac{1}{m}X(A-Y)^T\tag{7}$$
$$ \frac{\partial J}{\partial b} = \frac{1}{m} \sum_{i=1}^m (a^{(i)}-y^{(i)})\tag{8}$$
Can you write Python code for it?

# GRADED FUNCTION: propagate

def propagate(w, b, X, Y):
    """
    Implement the cost function and its gradient for the propagation explained above

    Arguments:
    w -- weights, a numpy array of size (num_px * num_px * 3, 1)
    b -- bias, a scalar
    X -- data of size (num_px * num_px * 3, number of examples)
    Y -- true "label" vector (containing 0 if non-cat, 1 if cat) of size (1, number of examples)

    Return:
    cost -- negative log-likelihood cost for logistic regression
    dw -- gradient of the loss with respect to w, thus same shape as w
    db -- gradient of the loss with respect to b, thus same shape as b

    Tips:
    - Write your code step by step for the propagation. np.log(), np.dot()
    """

    m = X.shape[1]

    # FORWARD PROPAGATION (FROM X TO COST)
    ### START CODE HERE ### (≈ 2 lines of code)
    A = sigmoid(np.dot(w.T,X)+b)                                    # compute activation
    cost = (-1/m)*np.sum(Y*np.log(A)+(1-Y)*(np.log(1-A)))                           # compute cost
    ### END CODE HERE ###

    # BACKWARD PROPAGATION (TO FIND GRAD)
    ### START CODE HERE ### (≈ 2 lines of code)
    dw = (1/m)*np.dot(X,(A-Y).T)
    db = (1/m)*np.sum(A-Y)
    ### END CODE HERE ###

    assert(dw.shape == w.shape)
    assert(db.dtype == float)
    cost = np.squeeze(cost)
    assert(cost.shape == ())

    grads = {"dw": dw,
             "db": db}

    return grads, cost

w, b, X, Y = np.array([[1.],[2.]]), 2., np.array([[1.,2.,-1.],[3.,4.,-3.2]]), np.array([[1,0,1]])
grads, cost = propagate(w, b, X, Y)
print ("dw = " + str(grads["dw"]))
print ("db = " + str(grads["db"]))
print ("cost = " + str(cost))


I am working on the file "Intro_to_Neural_Networks/Logistic+Regression+with+a+Neural+Network+mindset+v4.ipynb".
The first task is:
Expected Output:

             **predictions**

            [[ 1.  1.  0.]]

What to remember:
You've implemented several functions that:
- Initialize (w,b)
- Optimize the loss iteratively to learn parameters (w,b):
    - computing the cost and its gradient
    - updating the parameters using gradient descent
- Use the learned (w,b) to predict the labels for a given set of examples
5 - Merge all functions into a model
You will now see how the overall model is structured by putting together all the building blocks (functions implemented in the previous parts) together, in the right order.
Exercise: Implement the model function. Use the following notation:
    - Y_prediction for your predictions on the test set
    - Y_prediction_train for your predictions on the train set
    - w, costs, grads for the outputs of optimize()
Can you write Python code for it?

# GRADED FUNCTION: model

def model(X_train, Y_train, X_test, Y_test, num_iterations = 2000, learning_rate = 0.5, print_cost = False):
    """
    Builds the logistic regression model by calling the function you've implemented previously

    Arguments:
    X_train -- training set represented by a numpy array of shape (num_px * num_px * 3, m_train)
    Y_train -- training labels represented by a numpy array (vector) of shape (1, m_train)
    X_test -- test set represented by a numpy array of shape (num_px * num_px * 3, m_test)
    Y_test -- test labels represented by a numpy array (vector) of shape (1, m_test)
    num_iterations -- hyperparameter representing the number of iterations to optimize the parameters
    learning_rate -- hyperparameter representing the learning rate used in the update rule of optimize()
    print_cost -- Set to true to print the cost every 100 iterations

    Returns:
    d -- dictionary containing information about the model.
    """

    ### START CODE HERE ###

    # initialize parameters with zeros (≈ 1 line of code)
    w, b = initialize_with_zeros(X_train.shape[0])

    # Gradient descent (≈ 1 line of code)
    parameters, grads, costs = optimize(w, b, X_train, Y_train, num_iterations, learning_rate, print_cost)

    # Retrieve parameters w and b from dictionary "parameters"
    w = parameters["w"]
    b = parameters["b"]

    # Predict test/train set examples (≈ 2 lines of code)
    Y_prediction_test = predict(w, b, X_test)
    Y_prediction_train = predict(w, b, X_train)

    ### END CODE HERE ###

    # Print train/test Errors
    print("train accuracy: {} %".format(100 - np.mean(np.abs(Y_prediction_train - Y_train)) * 100))
    print("test accuracy: {} %".format(100 - np.mean(np.abs(Y_prediction_test - Y_test)) * 100))

    d = {"costs": costs,
         "Y_prediction_test": Y_prediction_test,
         "Y_prediction_train" : Y_prediction_train,
         "w" : w,
         "b" : b,
         "learning_rate" : learning_rate,
        ...


I am working on the file "Intro_to_Neural_Networks/Logistic+Regression+with+a+Neural+Network+mindset+v4.ipynb".
The first task is:
Expected Output:

    **train_set_x_flatten shape**
     (12288, 209)

    **train_set_y shape**
    (1, 209)

    **test_set_x_flatten shape**
    (12288, 50)

    **test_set_y shape**
    (1, 50)

  **sanity check after reshaping**
  [17 31 56 22 33]

To represent color images, the red, green and blue channels (RGB) must be specified for each pixel, and so the pixel value is actually a vector of three numbers ranging from 0 to 255.
One common preprocessing step in machine learning is to center and standardize your dataset, meaning that you substract the mean of the whole numpy array from each example, and then divide each example by the standard deviation of the whole numpy array. But for picture datasets, it is simpler and more convenient and works almost as well to just divide every row of the dataset by 255 (the maximum value of a pixel channel).

Let's standardize our dataset.
Can you write Python code for it?

train_set_x = train_set_x_flatten/255.
test_set_x = test_set_x_flatten/255.


I am working on the file "Intro_to_Neural_Networks/Logistic+Regression+with+a+Neural+Network+mindset+v4.ipynb".
The first task is:
Expected Output:

         **Cost after iteration 0 **
         0.693147

          $\vdots$
          $\vdots$

         **Train Accuracy**
         99.04306220095694 %

        **Test Accuracy**
         70.0 %

Comment: Training accuracy is close to 100%. This is a good sanity check: your model is working and has high enough capacity to fit the training data. Test error is 68%. It is actually not bad for this simple model, given the small dataset we used and that logistic regression is a linear classifier. But no worries, you'll build an even better classifier next week!
Also, you see that the model is clearly overfitting the training data. Later in this specialization you will learn how to reduce overfitting, for example by using regularization. Using the code below (and changing the index variable) you can look at predictions on pictures of the test set.
Can you write Python code for it?

# Example of a picture that was wrongly classified.
index = 1
plt.imshow(test_set_x[:,index].reshape((num_px, num_px, 3)))
print ("y = " + str(test_set_y[0,index]) + ", you predicted that it is a \"" + classes[d["Y_prediction_test"][0,index]].decode("utf-8") +  "\" picture.")


I am working on the file "Intro_to_Neural_Networks/Logistic+Regression+with+a+Neural+Network+mindset+v4.ipynb".
The first task is:
Expected Output:

    **sigmoid([0, 2])**
     [ 0.5         0.88079708]

4.2 - Initializing parameters
Exercise: Implement parameter initialization in the cell below. You have to initialize w as a vector of zeros. If you don't know what numpy function to use, look up np.zeros() in the Numpy library's documentation.
Can you write Python code for it?

# GRADED FUNCTION: initialize_with_zeros

def initialize_with_zeros(dim):
    """
    This function creates a vector of zeros of shape (dim, 1) for w and initializes b to 0.

    Argument:
    dim -- size of the w vector we want (or number of parameters in this case)

    Returns:
    w -- initialized vector of shape (dim, 1)
    b -- initialized scalar (corresponds to the bias)
    """

    ### START CODE HERE ### (≈ 1 line of code)
    w = np.zeros(shape=(dim,1))
    b = 0
    ### END CODE HERE ###

    assert(w.shape == (dim, 1))
    assert(isinstance(b, float) or isinstance(b, int))

    return w, b

dim = 2
w, b = initialize_with_zeros(dim)
print ("w = " + str(w))
print ("b = " + str(b))


I am working on the file "Intro_to_Neural_Networks/Logistic+Regression+with+a+Neural+Network+mindset+v4.ipynb".
The first task is:
Let's also plot the cost function and the gradients.
Can you write Python code for it?

# Plot learning curve (with costs)
costs = np.squeeze(d['costs'])
plt.plot(costs)
plt.ylabel('cost')
plt.xlabel('iterations (per hundreds)')
plt.title("Learning rate =" + str(d["learning_rate"]))
plt.show()


I am working on the file "Intro_to_Neural_Networks/Logistic+Regression+with+a+Neural+Network+mindset+v4.ipynb".
The first task is:
Run the following cell to train your model.
Can you write Python code for it?

d = model(train_set_x, train_set_y, test_set_x, test_set_y, num_iterations = 2000, learning_rate = 0.005, print_cost = True)


I am working on the file "Intro_to_Neural_Networks/Logistic+Regression+with+a+Neural+Network+mindset+v4.ipynb".
The first task is:
We added "_orig" at the end of image datasets (train and test) because we are going to preprocess them. After preprocessing, we will end up with train_set_x and test_set_x (the labels train_set_y and test_set_y don't need any preprocessing).
Each line of your train_set_x_orig and test_set_x_orig is an array representing an image. You can visualize an example by running the following code. Feel free also to change the index value and re-run to see other images.
Can you write Python code for it?

# Example of a picture
index = 23
plt.imshow(train_set_x_orig[index])
print ("y = " + str(train_set_y[:, index]) + ", it's a '" + classes[np.squeeze(train_set_y[:, index])].decode("utf-8") +  "' picture.")


I am working on the file "Intro_to_Neural_Networks/Planar+data+classification+with+one+hidden+layer+v4.ipynb".
The first task is:
Expected Output (these are not the sizes you will use for your network, they are just used to assess the function you've just coded).

    **n_x**
     5

    **n_h**
     4

    **n_y**
     2

4.2 - Initialize the model's parameters
Exercise: Implement the function initialize_parameters().
Instructions:
- Make sure your parameters' sizes are right. Refer to the neural network figure above if needed.
- You will initialize the weights matrices with random values.
    - Use: np.random.randn(a,b) * 0.01 to randomly initialize a matrix of shape (a,b).
- You will initialize the bias vectors as zeros.
    - Use: np.zeros((a,b)) to initialize a matrix of shape (a,b) with zeros.
Can you write Python code for it?

# GRADED FUNCTION: initialize_parameters

def initialize_parameters(n_x, n_h, n_y):
    """
    Argument:
    n_x -- size of the input layer
    n_h -- size of the hidden layer
    n_y -- size of the output layer

    Returns:
    params -- python dictionary containing your parameters:
                    W1 -- weight matrix of shape (n_h, n_x)
                    b1 -- bias vector of shape (n_h, 1)
                    W2 -- weight matrix of shape (n_y, n_h)
                    b2 -- bias vector of shape (n_y, 1)
    """

    np.random.seed(2) # we set up a seed so that your output matches ours although the initialization is random.

    ### START CODE HERE ### (≈ 4 lines of code)
    W1 = np.random.randn(n_h, n_x)*0.01
    b1 = np.zeros(shape=(n_h, 1))
    W2 = np.random.randn(n_y, n_h)*0.01
    b2 = np.zeros(shape=(n_y, 1))
    ### END CODE HERE ###

    assert (W1.shape == (n_h, n_x))
    assert (b1.shape == (n_h, 1))
    assert (W2.shape == (n_y, n_h))
    assert (b2.shape == (n_y, 1))

    parameters = {"W1": W1,
                  "b1": b1,
                  "W2": W2,
                  "b2": b2}

    return parameters

n_x, n_h, n_y = initialize_parameters_test_case()

parameters = initialize_parameters(n_x, n_h, n_y)
print("W1 = " + str(parameters["W1"]))
print("b1 = " + str(parameters["b1"]))
print("W2 = " + str(parameters["W2"]))
print("b2 = " + str(parameters["b2"]))


I am working on the file "Intro_to_Neural_Networks/Planar+data+classification+with+one+hidden+layer+v4.ipynb".
The first task is:
Expected Output:

    **shape of X**
     (2, 400)

    **shape of Y**
    (1, 400)

    **m**
     400

3 - Simple Logistic Regression
Before building a full neural network, lets first see how logistic regression performs on this problem. You can use sklearn's built-in functions to do that. Run the code below to train a logistic regression classifier on the dataset.
Can you write Python code for it?

# Train the logistic regression classifier
clf = sklearn.linear_model.LogisticRegressionCV();
clf.fit(X.T, Y.T);


I am working on the file "Intro_to_Neural_Networks/Planar+data+classification+with+one+hidden+layer+v4.ipynb".
The first task is:
Expected Output:

    **Accuracy**
     47%

Interpretation: The dataset is not linearly separable, so logistic regression doesn't perform well. Hopefully a neural network will do better. Let's try this now!
4 - Neural Network model
Logistic regression did not work well on the "flower dataset". You are going to train a Neural Network with a single hidden layer.
Here is our model:

Mathematically:
For one example $x^{(i)}$:
$$z^{[1] (i)} =  W^{[1]} x^{(i)} + b^{[1] (i)}\tag{1}$$
$$a^{[1] (i)} = \tanh(z^{[1] (i)})\tag{2}$$
$$z^{[2] (i)} = W^{[2]} a^{[1] (i)} + b^{[2] (i)}\tag{3}$$
$$\hat{y}^{(i)} = a^{[2] (i)} = \sigma(z^{ [2] (i)})\tag{4}$$
$$y^{(i)}_{prediction} = \begin{cases} 1 & \mbox{if } a^{2} > 0.5 \ 0 & \mbox{otherwise } \end{cases}\tag{5}$$
Given the predictions on all the examples, you can also compute the cost $J$ as follows:
$$J = - \frac{1}{m} \sum\limits_{i = 0}^{m} \large\left(\small y^{(i)}\log\left(a^{[2] (i)}\right) + (1-y^{(i)})\log\left(1- a^{[2] (i)}\right)  \large  \right) \small \tag{6}$$
Reminder: The general methodology to build a Neural Network is to:
    1. Define the neural network structure ( # of input units,  # of hidden units, etc).
    2. Initialize the model's parameters
    3. Loop:
        - Implement forward propagation
        - Compute loss
        - Implement backward propagation to get the gradients
        - Update parameters (gradient descent)
You often build helper functions to compute steps 1-3 and then merge them into one function we call nn_model(). Once you've built nn_model() and learnt the right parameters, you can make predictions on new data.
4.1 - Defining the neural network structure
Exercise: Define three variables:
    - n_x: the size of the input layer
    - n_h: the size of the hidden layer (set this to 4)
    - n_y: the size of the output layer
Hint: Use shapes of X and Y to find n_x and n_y. Also, hard...
Can you write Python code for it?

# GRADED FUNCTION: layer_sizes

def layer_sizes(X, Y):
    """
    Arguments:
    X -- input dataset of shape (input size, number of examples)
    Y -- labels of shape (output size, number of examples)

    Returns:
    n_x -- the size of the input layer
    n_h -- the size of the hidden layer
    n_y -- the size of the output layer
    """
    ### START CODE HERE ### (≈ 3 lines of code)
    n_x = X.shape[0] # size of input layer
    n_h = 4
    n_y = Y.shape[0] # size of output layer
    ### END CODE HERE ###
    return (n_x, n_h, n_y)

X_assess, Y_assess = layer_sizes_test_case()
(n_x, n_h, n_y) = layer_sizes(X_assess, Y_assess)
print("The size of the input layer is: n_x = " + str(n_x))
print("The size of the hidden layer is: n_h = " + str(n_h))
print("The size of the output layer is: n_y = " + str(n_y))


I am working on the file "Intro_to_Neural_Networks/Planar+data+classification+with+one+hidden+layer+v4.ipynb".
The first task is:
Expected Output:

    **cost**
     0.693058761...

Using the cache computed during forward propagation, you can now implement backward propagation.
Question: Implement the function backward_propagation().
Instructions:
Backpropagation is usually the hardest (most mathematical) part in deep learning. To help you, here again is the slide from the lecture on backpropagation. You'll want to use the six equations on the right of this slide, since you are building a vectorized implementation.

Tips:
To compute dZ1 you'll need to compute $g^{[1]'}(Z^{[1]})$. Since $g^{[1]}(.)$ is the tanh activation function, if $a = g^{[1]}(z)$ then $g^{[1]'}(z) = 1-a^2$. So you can compute
$g^{[1]'}(Z^{[1]})$ using (1 - np.power(A1, 2)).
Can you write Python code for it?

# GRADED FUNCTION: backward_propagation

def backward_propagation(parameters, cache, X, Y):
    """
    Implement the backward propagation using the instructions above.

    Arguments:
    parameters -- python dictionary containing our parameters
    cache -- a dictionary containing "Z1", "A1", "Z2" and "A2".
    X -- input data of shape (2, number of examples)
    Y -- "true" labels vector of shape (1, number of examples)

    Returns:
    grads -- python dictionary containing your gradients with respect to different parameters
    """
    m = X.shape[1]

    # First, retrieve W1 and W2 from the dictionary "parameters".
    ### START CODE HERE ### (≈ 2 lines of code)
    W1 = parameters["W1"]
    W2 = parameters["W2"]
    ### END CODE HERE ###

    # Retrieve also A1 and A2 from dictionary "cache".
    ### START CODE HERE ### (≈ 2 lines of code)
    A1 = cache["A1"]
    A2 = cache["A2"]
    ### END CODE HERE ###

    # Backward propagation: calculate dW1, db1, dW2, db2.
    ### START CODE HERE ### (≈ 6 lines of code, corresponding to 6 equations on slide above)
    dZ2 = A2-Y
    dW2 = (1/m)*np.dot(dZ2,A1.T)
    db2 = (1/m)*np.sum(dZ2,axis=1,keepdims=True)
    dZ1 = np.multiply(np.dot(W2.T, dZ2), 1 - np.power(A1, 2))
    dW1 = (1/m)*np.dot(dZ1,X.T)
    db1 = (1/m)*np.sum(dZ1,axis=1,keepdims=True)
    ### END CODE HERE ###

    grads = {"dW1": dW1,
             "db1": db1,
             "dW2": dW2,
             "db2": db2}

    return grads

parameters, cache, X_assess, Y_assess = backward_propagation_test_case()

grads = backward_propagation(parameters, cache, X_assess, Y_assess)
print ("dW1 = "+ str(grads["dW1"]))
print ("db1 = "+ str(grads["db1"]))
print ("dW2 = "+ str(grads["dW2"]))
print ("db2 = "+ str(grads["db2"]))


I am working on the file "Intro_to_Neural_Networks/Planar+data+classification+with+one+hidden+layer+v4.ipynb".
The first task is:
Expected Output:

    **W1**
     [[-0.00643025  0.01936718]
 [-0.02410458  0.03978052]
 [-0.01653973 -0.02096177]
 [ 0.01046864 -0.05990141]]

    **b1**
     [[ -1.02420756e-06]
 [  1.27373948e-05]
 [  8.32996807e-07]
 [ -3.20136836e-06]]

    **W2**
     [[-0.01041081 -0.04463285  0.01758031  0.04747113]]

    **b2**
     [[ 0.00010457]]

4.4 - Integrate parts 4.1, 4.2 and 4.3 in nn_model()
Question: Build your neural network model in nn_model().
Instructions: The neural network model has to use the previous functions in the right order.
Can you write Python code for it?

# GRADED FUNCTION: nn_model

def nn_model(X, Y, n_h, num_iterations = 10000, print_cost=False):
    """
    Arguments:
    X -- dataset of shape (2, number of examples)
    Y -- labels of shape (1, number of examples)
    n_h -- size of the hidden layer
    num_iterations -- Number of iterations in gradient descent loop
    print_cost -- if True, print the cost every 1000 iterations

    Returns:
    parameters -- parameters learnt by the model. They can then be used to predict.
    """

    np.random.seed(3)
    n_x = layer_sizes(X, Y)[0]
    n_y = layer_sizes(X, Y)[2]

    # Initialize parameters, then retrieve W1, b1, W2, b2. Inputs: "n_x, n_h, n_y". Outputs = "W1, b1, W2, b2, parameters".
    ### START CODE HERE ### (≈ 5 lines of code)
    parameters = initialize_parameters(n_x, n_h, n_y)
    W1 = parameters["W1"]
    b1 = parameters["W1"]
    W2 = parameters["W1"]
    b2 = parameters["W1"]
    ### END CODE HERE ###

    # Loop (gradient descent)

    for i in range(0, num_iterations):

        ### START CODE HERE ### (≈ 4 lines of code)
        # Forward propagation. Inputs: "X, parameters". Outputs: "A2, cache".
        A2, cache = forward_propagation(X, parameters)

        # Cost function. Inputs: "A2, Y, parameters". Outputs: "cost".
        cost = compute_cost(A2, Y, parameters)

        # Backpropagation. Inputs: "parameters, cache, X, Y". Outputs: "grads".
        grads = backward_propagation(parameters, cache, X, Y)

        # Gradient descent parameter update. Inputs: "parameters, grads". Outputs: "parameters".
        parameters = update_parameters(parameters, grads, learning_rate = 1.2)

        ### END CODE HERE ###

        # Print the cost every 1000 iterations
        if print_cost and i % 1000 == 0:
            print ("Cost after iteration %i: %f" %(i, cost))

    return parameters

X_assess, Y_assess = nn_model_test_case()
parameters = nn_model(X_assess, Y_assess, 4, num_iterations=10000, print_cost=True)
print("W1 = " +...


I am working on the file "Intro_to_Neural_Networks/Planar+data+classification+with+one+hidden+layer+v4.ipynb".
The first task is:
Expected Output:

        **cost after iteration 0**

        0.692739

         $\vdots$

         $\vdots$

    **W1**
     [[-0.65848169  1.21866811]
 [-0.76204273  1.39377573]
 [ 0.5792005  -1.10397703]
 [ 0.76773391 -1.41477129]]

    **b1**
     [[ 0.287592  ]
 [ 0.3511264 ]
 [-0.2431246 ]
 [-0.35772805]]

    **W2**
     [[-2.45566237 -3.27042274  2.00784958  3.36773273]]

    **b2**
     [[ 0.20459656]]

4.5 Predictions
Question: Use your model to predict by building predict().
Use forward propagation to predict results.
Reminder: predictions = $y_{prediction} = \mathbb 1 \text{{activation > 0.5}} = \begin{cases}
      1 & \text{if}\ activation > 0.5 \
      0 & \text{otherwise}
    \end{cases}$
As an example, if you would like to set the entries of a matrix X to 0 and 1 based on a threshold you would do: X_new = (X > threshold)
Can you write Python code for it?

# GRADED FUNCTION: predict

def predict(parameters, X):
    """
    Using the learned parameters, predicts a class for each example in X

    Arguments:
    parameters -- python dictionary containing your parameters
    X -- input data of size (n_x, m)

    Returns
    predictions -- vector of predictions of our model (red: 0 / blue: 1)
    """

    # Computes probabilities using forward propagation, and classifies to 0/1 using 0.5 as the threshold.
    ### START CODE HERE ### (≈ 2 lines of code)
    A2, cache = forward_propagation(X, parameters)
    predictions = np.round(A2)
    ### END CODE HERE ###

    return predictions

parameters, X_assess = predict_test_case()

predictions = predict(parameters, X_assess)
print("predictions mean = " + str(np.mean(predictions)))


I am working on the file "Intro_to_Neural_Networks/Planar+data+classification+with+one+hidden+layer+v4.ipynb".
The first task is:
Expected Output:

    **W1**
     [[-0.00416758 -0.00056267]
 [-0.02136196  0.01640271]
 [-0.01793436 -0.00841747]
 [ 0.00502881 -0.01245288]]

    **b1**
     [[ 0.]
 [ 0.]
 [ 0.]
 [ 0.]]

    **W2**
     [[-0.01057952 -0.00909008  0.00551454  0.02292208]]

    **b2**
     [[ 0.]]

4.3 - The Loop
Question: Implement forward_propagation().
Instructions:
- Look above at the mathematical representation of your classifier.
- You can use the function sigmoid(). It is built-in (imported) in the notebook.
- You can use the function np.tanh(). It is part of the numpy library.
- The steps you have to implement are:
    1. Retrieve each parameter from the dictionary "parameters" (which is the output of initialize_parameters()) by using parameters[".."].
    2. Implement Forward Propagation. Compute $Z^{[1]}, A^{[1]}, Z^{[2]}$ and $A^{[2]}$ (the vector of all your predictions on all the examples in the training set).
- Values needed in the backpropagation are stored in "cache". The cache will be given as an input to the backpropagation function.
Can you write Python code for it?

# GRADED FUNCTION: forward_propagation

def forward_propagation(X, parameters):
    """
    Argument:
    X -- input data of size (n_x, m)
    parameters -- python dictionary containing your parameters (output of initialization function)

    Returns:
    A2 -- The sigmoid output of the second activation
    cache -- a dictionary containing "Z1", "A1", "Z2" and "A2"
    """
    # Retrieve each parameter from the dictionary "parameters"
    ### START CODE HERE ### (≈ 4 lines of code)
    W1 = parameters["W1"]
    b1 = parameters["b1"]
    W2 = parameters["W2"]
    b2 = parameters["b2"]
    ### END CODE HERE ###

    # Implement Forward Propagation to calculate A2 (probabilities)
    ### START CODE HERE ### (≈ 4 lines of code)
    Z1 = np.dot(W1,X)+b1
    A1 = np.tanh(Z1)
    Z2 = np.dot(W2,A1)+b2
    A2 = sigmoid(Z2)
    ### END CODE HERE ###

    assert(A2.shape == (1, X.shape[1]))

    cache = {"Z1": Z1,
             "A1": A1,
             "Z2": Z2,
             "A2": A2}

    return A2, cache

X_assess, parameters = forward_propagation_test_case()
A2, cache = forward_propagation(X_assess, parameters)

# Note: we use the mean here just to make sure that your output matches ours.
print(np.mean(cache['Z1']) ,np.mean(cache['A1']),np.mean(cache['Z2']),np.mean(cache['A2']))


I am working on the file "Intro_to_Neural_Networks/Planar+data+classification+with+one+hidden+layer+v4.ipynb".
The first task is:
Expected Output:

    **predictions mean**
     0.666666666667

It is time to run the model and see how it performs on a planar dataset. Run the following code to test your model with a single hidden layer of $n_h$ hidden units.
Can you write Python code for it?

# Build a model with a n_h-dimensional hidden layer
parameters = nn_model(X, Y, n_h = 4, num_iterations = 10000, print_cost=True)

# Plot the decision boundary
plot_decision_boundary(lambda x: predict(parameters, x.T), X, Y)
plt.title("Decision Boundary for hidden layer size " + str(4))


I am working on the file "Intro_to_Neural_Networks/Planar+data+classification+with+one+hidden+layer+v4.ipynb".
The first task is:
Visualize the dataset using matplotlib. The data looks like a "flower" with some red (label y=0) and some blue (y=1) points. Your goal is to build a model to fit this data.
Can you write Python code for it?

# Visualize the data:
plt.scatter(X[0, :], X[1, :], c=Y, s=40, cmap=plt.cm.Spectral);


I am working on the file "Intro_to_Neural_Networks/Planar+data+classification+with+one+hidden+layer+v4.ipynb".
The first task is:
You can now plot the decision boundary of these models. Run the code below.
Can you write Python code for it?

# Plot the decision boundary for logistic regression
plot_decision_boundary(lambda x: clf.predict(x), X, Y)
plt.title("Logistic Regression")

# Print accuracy
LR_predictions = clf.predict(X.T)
print ('Accuracy of logistic regression: %d ' % float((np.dot(Y,LR_predictions) + np.dot(1-Y,1-LR_predictions))/float(Y.size)*100) +
       '% ' + "(percentage of correctly labelled datapoints)")


I am working on the file "Intro_to_Neural_Networks/Python+Basics+With+Numpy+v3.ipynb".
The first task is:
As you may have noticed, the vectorized implementation is much cleaner and more efficient. For bigger vectors/matrices, the differences in running time become even bigger.
Note that np.dot() performs a matrix-matrix or matrix-vector multiplication. This is different from np.multiply() and the * operator (which is equivalent to  .* in Matlab/Octave), which performs an element-wise multiplication.
2.1 Implement the L1 and L2 loss functions
Exercise: Implement the numpy vectorized version of the L1 loss. You may find the function abs(x) (absolute value of x) useful.
Reminder:
- The loss is used to evaluate the performance of your model. The bigger your loss is, the more different your predictions ($ \hat{y} $) are from the true values ($y$). In deep learning, you use optimization algorithms like Gradient Descent to train your model and to minimize the cost.
- L1 loss is defined as:
$$\begin{align} & L_1(\hat{y}, y) = \sum_{i=0}^m|y^{(i)} - \hat{y}^{(i)}| \end{align}\tag{6}$$
Can you write Python code for it?

# GRADED FUNCTION: L1

def L1(yhat, y):
    """
    Arguments:
    yhat -- vector of size m (predicted labels)
    y -- vector of size m (true labels)

    Returns:
    loss -- the value of the L1 loss function defined above
    """

    ### START CODE HERE ### (≈ 1 line of code)
    loss = np.sum(np.abs(yhat-y))
    ### END CODE HERE ###

    return loss

yhat = np.array([.9, 0.2, 0.1, .4, .9])
y = np.array([1, 0, 0, 1, 1])
print("L1 = " + str(L1(yhat,y)))


I am working on the file "Intro_to_Neural_Networks/Python+Basics+With+Numpy+v3.ipynb".
The first task is:
Expected Output:
<table style="width:20%">

     <tr>
       <td> **L1** </td>
       <td> 1.1 </td>
     </tr>
</table>

Exercise: Implement the numpy vectorized version of the L2 loss. There are several way of implementing the L2 loss but you may find the function np.dot() useful. As a reminder, if $x = [x_1, x_2, ..., x_n]$, then np.dot(x,x) = $\sum_{j=0}^n x_j^{2}$.

L2 loss is defined as $$\begin{align} & L_2(\hat{y},y) = \sum_{i=0}^m(y^{(i)} - \hat{y}^{(i)})^2 \end{align}\tag{7}$$
Can you write Python code for it?

# GRADED FUNCTION: L2

def L2(yhat, y):
    """
    Arguments:
    yhat -- vector of size m (predicted labels)
    y -- vector of size m (true labels)

    Returns:
    loss -- the value of the L2 loss function defined above
    """

    ### START CODE HERE ### (≈ 1 line of code)
    loss = np.sum(np.dot(y-yhat,y-yhat))
    ### END CODE HERE ###

    return loss

yhat = np.array([.9, 0.2, 0.1, .4, .9])
y = np.array([1, 0, 0, 1, 1])
print("L2 = " + str(L2(yhat,y)))


I am working on the file "Intro_to_Neural_Networks/Python+Basics+With+Numpy+v3.ipynb".
The first task is:
Expected Output:

        **softmax(x)**
        [[  9.80897665e-01   8.94462891e-04   1.79657674e-02   1.21052389e-04
    1.21052389e-04]
 [  8.78679856e-01   1.18916387e-01   8.01252314e-04   8.01252314e-04
    8.01252314e-04]]

Note:
- If you print the shapes of x_exp, x_sum and s above and rerun the assessment cell, you will see that x_sum is of shape (2,1) while x_exp and s are of shape (2,5). x_exp/x_sum works due to python broadcasting.
Congratulations! You now have a pretty good understanding of python numpy and have implemented a few useful functions that you will be using in deep learning.

What you need to remember:
- np.exp(x) works for any np.array x and applies the exponential function to every coordinate
- the sigmoid function and its gradient
- image2vector is commonly used in deep learning
- np.reshape is widely used. In the future, you'll see that keeping your matrix/vector dimensions straight will go toward eliminating a lot of bugs.
- numpy has efficient built-in functions
- broadcasting is extremely useful
2) Vectorization
In deep learning, you deal with very large datasets. Hence, a non-computationally-optimal function can become a huge bottleneck in your algorithm and can result in a model that takes ages to run. To make sure that your code is  computationally efficient, you will use vectorization. For example, try to tell the difference between the following implementations of the dot/outer/elementwise product.
Can you write Python code for it?

import time

x1 = [9, 2, 5, 0, 0, 7, 5, 0, 0, 0, 9, 2, 5, 0, 0]
x2 = [9, 2, 2, 9, 0, 9, 2, 5, 0, 0, 9, 2, 5, 0, 0]

### CLASSIC DOT PRODUCT OF VECTORS IMPLEMENTATION ###
tic = time.process_time()
dot = 0
for i in range(len(x1)):
    dot+= x1[i]*x2[i]
toc = time.process_time()
print ("dot = " + str(dot) + "\n ----- Computation time = " + str(1000*(toc - tic)) + "ms")

### CLASSIC OUTER PRODUCT IMPLEMENTATION ###
tic = time.process_time()
outer = np.zeros((len(x1),len(x2))) # we create a len(x1)*len(x2) matrix with only zeros
for i in range(len(x1)):
    for j in range(len(x2)):
        outer[i,j] = x1[i]*x2[j]
toc = time.process_time()
print ("outer = " + str(outer) + "\n ----- Computation time = " + str(1000*(toc - tic)) + "ms")

### CLASSIC ELEMENTWISE IMPLEMENTATION ###
tic = time.process_time()
mul = np.zeros(len(x1))
for i in range(len(x1)):
    mul[i] = x1[i]*x2[i]
toc = time.process_time()
print ("elementwise multiplication = " + str(mul) + "\n ----- Computation time = " + str(1000*(toc - tic)) + "ms")

### CLASSIC GENERAL DOT PRODUCT IMPLEMENTATION ###
W = np.random.rand(3,len(x1)) # Random 3*len(x1) numpy array
tic = time.process_time()
gdot = np.zeros(W.shape[0])
for i in range(W.shape[0]):
    for j in range(len(x1)):
        gdot[i] += W[i,j]*x1[j]
toc = time.process_time()
print ("gdot = " + str(gdot) + "\n ----- Computation time = " + str(1000*(toc - tic)) + "ms")

x1 = [9, 2, 5, 0, 0, 7, 5, 0, 0, 0, 9, 2, 5, 0, 0]
x2 = [9, 2, 2, 9, 0, 9, 2, 5, 0, 0, 9, 2, 5, 0, 0]

### VECTORIZED DOT PRODUCT OF VECTORS ###
tic = time.process_time()
dot = np.dot(x1,x2)
toc = time.process_time()
print ("dot = " + str(dot) + "\n ----- Computation time = " + str(1000*(toc - tic)) + "ms")

### VECTORIZED OUTER PRODUCT ###
tic = time.process_time()
outer = np.outer(x1,x2)
toc = time.process_time()
print ("outer = " + str(outer) + "\n ----- Computation time = " + str(1000*(toc - tic)) + "ms")

### VECTORIZED ELEMENTWISE MULTIPLICATION ###
tic = time.process_time()
mul = np.multiply(x1,x2)
toc =...


I am working on the file "Intro_to_Neural_Networks/Python+Basics+With+Numpy+v3.ipynb".
The first task is:
Expected Output:

        **image2vector(image)**
        [[ 0.67826139]
 [ 0.29380381]
 [ 0.90714982]
 [ 0.52835647]
 [ 0.4215251 ]
 [ 0.45017551]
 [ 0.92814219]
 [ 0.96677647]
 [ 0.85304703]
 [ 0.52351845]
 [ 0.19981397]
 [ 0.27417313]
 [ 0.60659855]
 [ 0.00533165]
 [ 0.10820313]
 [ 0.49978937]
 [ 0.34144279]
 [ 0.94630077]]

1.4 - Normalizing rows
Another common technique we use in Machine Learning and Deep Learning is to normalize our data. It often leads to a better performance because gradient descent converges faster after normalization. Here, by normalization we mean changing x to $ \frac{x}{\| x\|} $ (dividing each row vector of x by its norm).
For example, if $$x =
\begin{bmatrix}
    0 & 3 & 4 \
    2 & 6 & 4 \
\end{bmatrix}\tag{3}$$ then $$\| x\| = np.linalg.norm(x, axis = 1, keepdims = True) = \begin{bmatrix}
    5 \
    \sqrt{56} \
\end{bmatrix}\tag{4} $$and        $$ x_normalized = \frac{x}{\| x\|} = \begin{bmatrix}
    0 & \frac{3}{5} & \frac{4}{5} \
    \frac{2}{\sqrt{56}} & \frac{6}{\sqrt{56}} & \frac{4}{\sqrt{56}} \
\end{bmatrix}\tag{5}$$ Note that you can divide matrices of different sizes and it works fine: this is called broadcasting and you're going to learn about it in part 5.
Exercise: Implement normalizeRows() to normalize the rows of a matrix. After applying this function to an input matrix x, each row of x should be a vector of unit length (meaning length 1).
Can you write Python code for it?

# GRADED FUNCTION: normalizeRows

def normalizeRows(x):
    """
    Implement a function that normalizes each row of the matrix x (to have unit length).

    Argument:
    x -- A numpy matrix of shape (n, m)

    Returns:
    x -- The normalized (by row) numpy matrix. You are allowed to modify x.
    """

    ### START CODE HERE ### (≈ 2 lines of code)
    # Compute x_norm as the norm 2 of x. Use np.linalg.norm(..., ord = 2, axis = ..., keepdims = True)
    x_norm = np.linalg.norm(x,axis=1,keepdims=True)

    # Divide x by its norm.
    x = x/x_norm
    ### END CODE HERE ###

    return x

x = np.array([
    [0, 3, 4],
    [1, 6, 4]])
print("normalizeRows(x) = " + str(normalizeRows(x)))


I am working on the file "Intro_to_Neural_Networks/Python+Basics+With+Numpy+v3.ipynb".
The first task is:
Expected Output:

         **sigmoid([1,2,3])**
         array([ 0.73105858,  0.88079708,  0.95257413])

1.2 - Sigmoid gradient
As you've seen in lecture, you will need to compute gradients to optimize loss functions using backpropagation. Let's code your first gradient function.
Exercise: Implement the function sigmoid_grad() to compute the gradient of the sigmoid function with respect to its input x. The formula is: $$sigmoid_derivative(x) = \sigma'(x) = \sigma(x) (1 - \sigma(x))\tag{2}$$
You often code this function in two steps:
1. Set s to be the sigmoid of x. You might find your sigmoid(x) function useful.
2. Compute $\sigma'(x) = s(1-s)$
Can you write Python code for it?

# GRADED FUNCTION: sigmoid_derivative

def sigmoid_derivative(x):
    """
    Compute the gradient (also called the slope or derivative) of the sigmoid function with respect to its input x.
    You can store the output of the sigmoid function into variables and then use it to calculate the gradient.

    Arguments:
    x -- A scalar or numpy array

    Return:
    ds -- Your computed gradient.
    """

    ### START CODE HERE ### (≈ 2 lines of code)
    s = 1/(1+np.exp(-x))
    ds = s*(1-s)
    ### END CODE HERE ###

    return ds

x = np.array([1, 2, 3])
print ("sigmoid_derivative(x) = " + str(sigmoid_derivative(x)))