I am working on the file "ML/Week 3 Assessment Orthogonal Projections.ipynb".
The first task is:
Advice for testing numerical algorithms
Testing machine learning algorithms (or numerical algorithms in general)
is sometimes really hard as it depends on the dataset
to produce an answer, and you will never be able to test your algorithm on all the datasets
we have in the world. Nevertheless, we have some tips for you to help you identify bugs in
your implementations.
1. Test on small dataset
Test your algorithms on small dataset: datasets of size 1 or 2 sometimes will suffice. This
is useful because you can (if necessary) compute the answers by hand and compare them with
the answers produced by the computer program you wrote. In fact, these small datasets can even have special numbers,
which will allow you to compute the answers by hand easily.
2. Find invariants
Invariants refer to properties of your algorithm and functions that are maintained regardless
of the input. We will highlight this point later in this notebook where you will see functions,
which will check invariants for some of the answers you produce.
Invariants you may want to look for:
1. Does your algorithm always produce a positive/negative answer, or a positive definite matrix?
2. If the algorithm is iterative, do the intermediate results increase/decrease monotonically?
3. Does your solution relate with your input in some interesting way, e.g. orthogonality?
When you have a set of invariants, you can generate random inputs and make
assertions about these invariants. This is sometimes known as fuzzing, which has proven to be a very effective technique for identifying bugs in programs.
Finding invariants is hard, and sometimes there simply isn't any invariant. However, DO take advantage of them if you can find them. They are the most powerful checks when you have them.
1. Orthogonal Projections
Recall that for projection of a vector $x$ onto a 1-dimensional subspace $U$ with basis vector $\boldsymbol b$ we have
$${\pi_U}(\boldsymbol x) = \frac{\boldsymbol b\boldsymbol b^T}{{\lVert \boldsymbol b \rVert}^2}\boldsymbol x $$
And for the...
Can you write Python code for it?

import numpy.testing as np_test
def test_property_projection_matrix(P):
    """Test if the projection matrix satisfies certain properties.
    In particular, we should have P @ P = P, and P = P^T
    """
    np_test.assert_almost_equal(P, P @ P)
    np_test.assert_almost_equal(P, P.T)

def test_property_projection(x, p):
    """Test orthogonality of x and its projection p."""
    np_test.assert_almost_equal(np.dot(p-x, p), 0)

# GRADED FUNCTION: DO NOT EDIT THIS LINE
# Projection 1d

# ===YOU SHOULD EDIT THIS FUNCTION===
def projection_matrix_1d(b):
    """Compute the projection matrix onto the space spanned by `b`
    Args:
        b: ndarray of dimension (D,), the basis for the subspace

    Returns:
        P: the projection matrix
    """
    D, = b.shape
    b = b[:, np.newaxis]
    P = np.dot(b,b.T)/np.dot(b.T,b) # EDIT THIS
    return P

# ===YOU SHOULD EDIT THIS FUNCTION===
def project_1d(x, b):
    """Compute the projection matrix onto the space spanned by `b`
    Args:
        x: the vector to be projected
        b: ndarray of dimension (D,), the basis for the subspace

    Returns:
        y: projection of x in space spanned by b
    """
    p = np.matmul(projection_matrix_1d(b), x[:, np.newaxis]).ravel() # EDIT THIS
    return p

# Projection onto general subspace
# ===YOU SHOULD EDIT THIS FUNCTION===
def projection_matrix_general(B):
    """Compute the projection matrix onto the space spanned by `B`
    Args:
        B: ndarray of dimension (D, M), the basis for the subspace

    Returns:
        P: the projection matrix
    """
    #P = np.dot(np.dot(B,(np.dot(B.T,B)).I),B.T) # EDIT THIS
    P = np.dot(np.dot( B, np.linalg.pinv(np.dot(B.T,B))), B.T)
    return P

# ===YOU SHOULD EDIT THIS FUNCTION===
def project_general(x, B):
    """Compute the projection matrix onto the space spanned by `B`
    Args:
        B: ndarray of dimension (D, E), the basis for the subspace

    Returns:
        y: projection of x in space spanned by b
    """
    p =...


I am working on the file "ML/Week 3 Assessment Orthogonal Projections.ipynb".
The first task is:
Along the first dimension of B, each instance is a 64x64 image, an "eigenface". Let's visualize
a few of them.
Can you write Python code for it?


I am working on the file "ML/Week 3 Assessment Orthogonal Projections.ipynb".
The first task is:
Question:
What would happen to the reconstruction as we increase the dimension of our basis?
Modify the code above to visualize it.
3. Least square for predicting Boston housing prices (optional)
Consider the case where we have a linear model for predicting housing prices. We are predicting the housing prices based on features in the
housing dataset. If we collect the features in a vector $\boldsymbol{x}$, and the price of the houses as $y$. Assuming that we have
a prediction model in the way such that $\hat{y}_i =  f(\boldsymbol {x}_i) = \boldsymbol \theta^T\boldsymbol{x}_i$.
If we collect the dataset of $n$ datapoints $\boldsymbol x_i$ in a data matrix $\boldsymbol X$, we can write down our model like this:
$$
\begin{bmatrix}
\boldsymbol {x}_1^T \
\vdots \
\boldsymbol {x}_n^T
\end{bmatrix} \boldsymbol {\theta} = \begin{bmatrix}
y_1 \
\vdots \
y_n
\end{bmatrix}.
$$
That is,
$$
\boldsymbol X\boldsymbol{\theta} = \boldsymbol {y}.
$$
where $\boldsymbol y$ collects all house prices $y_1,\dotsc, y_n$ of the training set.
Our goal is to find the best $\boldsymbol \theta$ that minimizes the following (least squares) objective:
$$
\begin{eqnarray}
&\sum^n_{i=1}{\lVert \boldsymbol \theta^T\boldsymbol {x}_i - y_i \rVert^2} \
&= (\boldsymbol X\boldsymbol {\theta} - \boldsymbol y)^T(\boldsymbol X\boldsymbol {\theta} - \boldsymbol y).
\end{eqnarray}
$$
Note that we aim to minimize the squared error between the prediction $\boldsymbol \theta^T\boldsymbol {x}_i$  of the model and the observed data point $y_i$ in the training set.
To find the optimal (maximum likelihood) parameters $\boldsymbol \theta^*$, we set the gradient of the least-squares objective to $\boldsymbol 0$:
$$
\begin{eqnarray}
\nabla_{\boldsymbol\theta}(\boldsymbol X{\boldsymbol \theta} - \boldsymbol y)^T(\boldsymbol X{\boldsymbol \theta} - \boldsymbol y) &=& \boldsymbol 0 \
\iff \nabla_{\boldsymbol\theta}(\boldsymbol {\theta}^T\boldsymbol X^T - \boldsymbol y^T)(\boldsymbol X\boldsymbol {\theta} - \boldsymbol y) &=& \boldsymbol 0 \
\iff...
Can you write Python code for it?

x = np.linspace(0, 10, num=50)

random = np.random.RandomState(42)  # we use the same random seed so we get deterministic output
theta = random.randn()              # we use a random theta, our goal is to perform linear regression which finds theta_hat that minimizes the objective
y = theta * x + random.rand(len(x)) # our theta is corrupted by some noise, so that we do not get (x,y) on a line

plt.scatter(x, y);
plt.xlabel('x');
plt.ylabel('y');

X = x.reshape(-1,1)
Y = y.reshape(-1,1)

theta_hat = np.linalg.solve(X.T @ X,
                            X.T @ Y)


I am working on the file "ML/Week 3 Assessment Orthogonal Projections.ipynb".
The first task is:
Take a look at what happens if we project our faces onto the basis spanned by these "eigenfaces". This requires
us to reshape B into the same shape as the matrix representing the basis as we have done earlier. Then we can
reuse the functions we implemented earlier to compute the projection matrix and the projection. Complete the code below to visualize the reconstructed faces that lie on the subspace spanned by the "eigenfaces".
Can you write Python code for it?

@interact(i=(0, 10))
def show_eigenface_reconstruction(i):
    original_face = faces_normalized[i].reshape(64, 64)
    # project original_face onto the vector space spanned by B_basis,
    # you should take advantage of the functions you have implemented above
    # to perform the projection. First, reshape B such that it represents the basis
    # for the eigenfaces. Then perform orthogonal projection which would give you
    # `face_reconstruction`.
    B_basis = B.reshape(50, 64*64).T
    face_reconstruction = project_general(original_face, B_basis)
    plt.figure()
    plt.imshow(np.hstack([original_face, face_reconstruction]), cmap='gray')
    plt.show()


I am working on the file "ML/Week 3 Assessment Orthogonal Projections.ipynb".
The first task is:
The data for the basis has been saved in a file named eigenfaces.py, first we load it into the variable B.
Can you write Python code for it?

B = np.load('eigenfaces.npy')[:50] # we use the first 50 dimensions of the basis, you should play around with the dimension here.
print("the eigenfaces have shape {}".format(B.shape))


I am working on the file "ML/Week 3 Assessment Orthogonal Projections.ipynb".
The first task is:
We can show how our $\hat{\theta}$ fits the line.
Can you write Python code for it?

fig, ax = plt.subplots()
ax.scatter(x, y);
xx = [0, 10]
yy = [0, 10 * theta_hat[0,0]]
ax.plot(xx, yy, 'red', alpha=.5);
ax.set(xlabel='x', ylabel='y');
print("theta = %f" % theta)
print("theta_hat = %f" % theta_hat)


I am working on the file "ML/Week 3 Assessment Orthogonal Projections.ipynb".
The first task is:
We have included some unittest for you to test your implementation.
Can you write Python code for it?

# Orthogonal projection in 2d
# define basis vector for subspace
b = np.array([2,1]).reshape(-1,1)
# point to be projected later
x = np.array([1,2]).reshape(-1, 1)

# Test 1D
np_test.assert_almost_equal(projection_matrix_1d(np.array([1, 2, 2])),
                            np.array([[1,  2,  2],
                                      [2,  4,  4],
                                      [2,  4,  4]]) / 9)

np_test.assert_almost_equal(project_1d(np.ones(3),
                                       np.array([1, 2, 2])),
                            np.array([5, 10, 10]) / 9)

B = np.array([[1, 0],
              [1, 1],
              [1, 2]])

# Test General
np_test.assert_almost_equal(projection_matrix_general(B),
                            np.array([[5,  2, -1],
                                      [2,  2,  2],
                                      [-1, 2,  5]]) / 6)

np_test.assert_almost_equal(project_general(np.array([6, 0, 0]), B),
                            np.array([5, 2, -1]))
print('correct')

# Write your own test cases here, use random inputs, utilize the invariants we have!


I am working on the file "ML/Week 3 Assessment Orthogonal Projections.ipynb".
The first task is:
We see how we can find the best $\theta$. In fact, we can extend our methodology to higher dimensional dataset. Let's now try applying the same methodology to the boston housing prices dataset.
Can you write Python code for it?

from sklearn.datasets import load_boston
boston = load_boston()
boston_X, boston_y = boston.data, boston.target
print("The housing dataset has size {}".format(boston_X.shape))
print("The prices has size {}".format(boston_X.shape))

boston_theta_hat = np.dot(np.dot(np.linalg.pinv(np.dot(X.T, X)), X.T) ,y) ## EDIT THIS to predict boston_theta_hat


I am working on the file "ML/Week 3 Assessment Orthogonal Projections.ipynb".
The first task is:
What would happen to $\lVert {\theta^*} - \theta \rVert$ if we increased the number of datapoints?
Make your hypothesis, and write a small program to confirm it!
Can you write Python code for it?

N = np.arange(10, 10000, step=10)
# Your code here which calculates θ* for different sample size.
thetao = np.dot(np.dot(np.linalg.pinv(np.dot(X.T, X)), X.T) ,y)


I am working on the file "ML/week1.ipynb".
The first task is:
1. Mean and Covariance of a Dataset
You will now need to implement functions to which compute the mean and covariance of a dataset.
There are two ways to compute the mean and covariance. The naive way would be to iterate over the dataset
to compute them. This would be implemented as a for loop in Python. However, computing them for large
dataset would be slow. Alternatively, you can use the functions provided by numpy to compute them, these are much
faster as numpy uses machine code to compute them. You will implment function which computes mean and covariane both
in the naive way and in the fast way. Later we will compare the performance between these two approaches. If you need to find out which numpy routine to call, have a look at the documentation https://docs.scipy.org/doc/numpy/reference/.
It is a good exercise to refer to the official documentation whenever you are not sure about something.
When you implement the functions for your assignment, make sure you read
the docstring which dimension of your inputs corresponds to the number of data points and which
corresponds to the dimension of the dataset.
Can you write Python code for it?

# ===YOU SHOULD EDIT THIS FUNCTION===
def mean_naive(X):
    """Compute the mean for a dataset by iterating over the dataset

    Arguments
    ---------
    X: (N, D) ndarray representing the dataset.
        N is the number of samples in the dataset
        and D is the feature dimension of the dataset

    Returns
    -------
    mean: (D, ) ndarray which is the mean of the dataset.
    """
    N, D = X.shape
    mean = np.zeros(D)
    # The naive approach requires us to iterate over the whole dataset with a for loop.
    for n in range(N):
        mean = mean + X[n]/N
    return mean

# ===YOU SHOULD EDIT THIS FUNCTION===
def cov_naive(X):
    """Compute the covariance for a dataset
    Arguments
    ---------
    X: (N, D) ndarray representing the dataset.
        N is the number of samples in the dataset
        and D is the feature dimension of the dataset

    Returns
    -------
    covariance: (D, D) ndarray which is the covariance matrix of the dataset.

    """
    N, D = X.shape
    covariance = np.zeros((D, D))
    for n in range(N):
        covariance = covariance + np.dot((X[n]-mean_naive(X)).T, (X[n]-mean_naive(X)))/N
    return covariance

# GRADED FUNCTION: DO NOT EDIT THIS LINE

# ===YOU SHOULD EDIT THIS FUNCTION===
def mean(X):
    """Compute the mean for a dataset

    Arguments
    ---------
    X: (N, D) ndarray representing the dataset.

    Returns
    -------
    mean: (D, ) ndarray which is the mean of the dataset.
    """
    mean = np.sum(X,0)/X.shape[0]
    return mean

# ===YOU SHOULD EDIT THIS FUNCTION===
def cov(X):
    """Compute the covariance for a dataset
    Arguments
    ---------
    X: (N, D) ndarray representing the dataset.

    Returns
    -------
    covariance_matrix: (D, D) ndarray which is the covariance matrix of the dataset.

    """
    # It is possible to vectorize our code for computing the covariance, i.e. we do not need to explicitly
    # iterate over the entire dataset as looping in Python tends to be slow
    N, D =...


I am working on the file "ML/week1.ipynb".
The first task is:
Alternatively, we can also see how running time increases as we increase the size of our dataset.
In the following cell, we run mean, mean_naive and cov, cov_naive for many times on different sizes of
the dataset and collect their running time. If you are less familiar with Python, you may want to spend
some time understanding what the code does. Understanding how your code scales with the size of your dataset (or dimensionality of the dataset) is crucial when you want to apply your algorithm to larger dataset. This is really important when we propose alternative methods a more efficient algorithms to solve the same problem. We will use these techniques again later in this course to analyze the running time of our code.
Can you write Python code for it?

def time(f, repeat=10):
    """A helper function to time the execution of a function.

    Arguments
    ---------
    f: a function which we want to time it.
    repeat: the number of times we want to execute `f`

    Returns
    -------
    the mean and standard deviation of the execution.
    """
    times = []
    for _ in range(repeat):
        start = timeit.default_timer()
        f()
        stop = timeit.default_timer()
        times.append(stop-start)
    return np.mean(times), np.std(times)


I am working on the file "ML/week1.ipynb".
The first task is:
Assuming that for some dataset $\boldsymbol X$, the mean and covariance are $\boldsymbol m$, $\boldsymbol S$, and for the new dataset after affine transformation $ \boldsymbol X'$, the mean and covariance are $\boldsymbol m'$ and $\boldsymbol S'$, then we would have the following identity:
$$\boldsymbol m' = \text{affine_mean}(\boldsymbol m, \boldsymbol A, \boldsymbol b)$$
$$\boldsymbol S' = \text{affine_covariance}(\boldsymbol S, \boldsymbol A, \boldsymbol b)$$
Can you write Python code for it?

X1 = ((A @ (X.T)).T + b)  # applying affine transformation once
X2 = ((A @ (X1.T)).T + b) # and again


I am working on the file "ML/week1.ipynb".
The first task is:
Fill in the ??? below
Can you write Python code for it?

np.testing.assert_almost_equal(mean(X2), affine_mean(mean(X1), A, b))
np.testing.assert_almost_equal(cov(X2),  affine_covariance(cov(X1), A, b))
print('correct')


I am working on the file "ML/week1.ipynb".
The first task is:
In the cell below, we compare the running time between mean and mean_naive, we repeatedly run these functions on a "fake" dataset X
of different sizes and record a list of running_times (fast_time and slow_time), we then plot the running time against the dataset size to visualize how the running time scales with the size of the dataset.
Can you write Python code for it?

fast_time = []
slow_time = []

for size in np.arange(100, 5000, step=100):
    X = np.random.randn(size, 20) # Create a random dataset with of size (size, 20)
    f = lambda : mean(X)
    # Record the running time for mean
    mu, sigma = time(f)
    fast_time.append((size, mu, sigma))

    f = lambda : mean_naive(X)
    mu, sigma = time(f)
    # Record the running time for mean_naive
    slow_time.append((size, mu, sigma))

fast_time = np.array(fast_time)
slow_time = np.array(slow_time)

fig, ax = plt.subplots()
ax.errorbar(fast_time[:,0], fast_time[:,1], fast_time[:,2], label='fast mean', linewidth=2)
ax.errorbar(slow_time[:,0], slow_time[:,1], slow_time[:,2], label='naive mean', linewidth=2)
ax.set_xlabel('size of dataset')
ax.set_ylabel('running time')
plt.legend();

## === FILL IN THIS, follow the approach we have above ===
fast_time_cov = []
slow_time_cov = []
for size in np.arange(100, 5000, step=100):
    X = np.random.randn(size, 20)
    # Record the running time for cov
    f = lambda : cov_naive(X)               # EDIT THIS
    mu, sigma = time(f) # EDIT THIS
    fast_time_cov.append((size, mu, sigma))

    # Record the running time for cov_naive
    f = lambda : cov(X)         # EDIT THIS
    mu, sigma = time(f) # EDIT THIS
    slow_time_cov.append((size, mu, sigma))

fast_time_cov = np.array(fast_time_cov)
slow_time_cov = np.array(slow_time_cov)

fig, ax = plt.subplots()
ax.errorbar(fast_time_cov[:,0], fast_time_cov[:,1], fast_time_cov[:,2], label='fast covariance', linewidth=2)
ax.errorbar(slow_time_cov[:,0], slow_time_cov[:,1], slow_time_cov[:,2], label='naive covariance', linewidth=2)
ax.set_xlabel('size of dataset')
ax.set_ylabel('running time')
plt.legend();


I am working on the file "ML/week1.ipynb".
The first task is:
One very useful way to compare whether arrays are equal/similar is use the helper functions
in numpy.testing. the functions in numpy.testing will throw an AssertionError when the output does not satisfy the assertion.
Can you write Python code for it?

np.testing.assert_almost_equal(mean(X1), affine_mean(mean(X), A, b))
np.testing.assert_almost_equal(cov(X1),  affine_covariance(cov(X), A, b))
print('correct')


I am working on the file "ML/week1.ipynb".
The first task is:
To put things into perspective, we can benchmark the two different implementation with the %time function
in the following way:
Can you write Python code for it?

# We have some huge data matrix, and we want to compute its mean
X = np.random.randn(1000, 20)
# Benchmarking time for computing mean
%time mean_naive(X)
%time mean(X)
pass

# Benchmarking time for computing covariance
%time cov_naive(X)
%time cov(X)
pass


I am working on the file "ML/week1.ipynb".
The first task is:
When your dataset are images, it's a really good idea to see what they look like.
One very
convenient tool in Jupyter is the interact widget, which we use to visualize the images (faces). For more information on how to use interact, have a look at the documentation here.
Can you write Python code for it?

@interact(n=(0, len(faces)-1))
def display_faces(n=0):
    plt.figure()
    plt.imshow(faces[n].reshape((64, 64)), cmap='gray')
    plt.show()


I am working on the file "ML/week1.ipynb".
The first task is:
With the mean function implemented, let's take a look at the mean face of our dataset!
Can you write Python code for it?

def mean_face(faces):
    """Compute the mean of the `faces`

    Arguments
    ---------
    faces: (N, 64 * 64) ndarray representing the faces dataset.

    Returns
    -------
    mean_face: (64, 64) ndarray which is the mean of the faces.
    """
    mean_face = mean(faces)
    return mean_face

plt.imshow(mean_face(faces).reshape((64, 64)), cmap='gray');


I am working on the file "ML02_Learning_Pipeline.ipynb".
The first task is:
K-fold Cross Validation and HyperParameter Tuning
Tuning hyperparameters via Grid Search
Can you write Python code for it?

from sklearn.grid_search import GridSearchCV
from sklearn.svm import SVC
from sklearn.pipeline import Pipeline


I am working on the file "ML02_Learning_Pipeline.ipynb".
The first task is:
Setting HyperParameters to tune
Can you write Python code for it?

param_range = [0.0001, 0.001, 0.01, 0.1, 1.0, 10.0, 100.0, 1000.0]
param_grid = [# Parameters for Linear Kernel
              {'clf__C': param_range,
               'clf__kernel': ['linear']},
              # Parameter for RBF Kernel
              {'clf__C': param_range,
               'clf__gamma': param_range,
               'clf__kernel': ['rbf']},
              # Parameter for Polynomial Kernel
              {'clf__C': param_range,
               'clf__degree': [2, 3, 5, 8, 16],
               'clf__coef0': [0, 1],
               'clf__kernel': ['poly']},
            ]

# So far, the `Pipeline` is not actually needed, but
# this code is ready in case multiple (features selection)
# steps would be considered for inclusion
pipe_svc = Pipeline([('clf', SVC(random_state=1))])


I am working on the file "ML02_Learning_Pipeline.ipynb".
The first task is:
Train-Test Split
Before we construct out model pipeline, we divide the dataset into separate a separate training dataset (75% of the data) and a separate test dataset (25% of the data):
Can you write Python code for it?

from sklearn.cross_validation import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.25, random_state=1)


I am working on the file "ML1.ipynb".
The first task is:
En la barra de la derecha, a medida que el color va de rojo a azul, el valor de pérdida disminuye y disminuye hasta alcanzar el valor más pequeño (azul oscuro).
Dado que queremos que la pérdida sea lo más pequeña posible, queremos que nuestros valores de $w_1$ y $w_2$ produzcan una pérdida en el área azul oscuro de la gráfica de contorno. Esto se logra dentro de un cierto rango de valores para $w_1$ y $w_2$.
Pregunta: ¿Puedes leer qué valores de $w_1$ y $w_2$ te darán la menor pérdida?
¿Son estos valores similares a los que encontró anteriormente para separar linealmente los datos?
Pregunta: Dependiendo del punto central que elijas, el valor más pequeño para la pérdida puede no ser 0. ¿En qué circunstancias es este el caso?
Optimizando la pérdida usando TensorFlow
Ahora que tenemos una función que nos da la pérdida para diferentes valores de $ w_1 $ y $ w_2 $, queremos un método automatizado para encontrar los valores que minimicen la función de pérdida. Aquí es donde entra en juego la optimización por descenso de gradiente. La idea es que para cada (batch of) puntos de datos, calculemos la pérdida utilizando los valores actuales de $ w_1 $ y $ w_2 $ en los datos. Luego calculamos el gradiente (o derivado) de la función de pérdida en los valores actuales de $ w_1 $ y $ w_2 $. El negativo del gradiente apunta en la dirección del steepest descent a lo largo de la función de pérdida. Al ajustar los valores de $ w_1 $ y $ w_2 $ en la dirección del gradiente negativo, nos acercamos al mínimo de la función de pérdida (siempre que la función de pérdida tenga un "buen comportamiento"). El gran paso que damos está mediado por la tasa de aprendizaje. Para hacer esto más fácilmente, usaremos TensorFlow.
Optional extra reading: TensorFlow
TensorFlow (TF) es una biblioteca de software de código abierto para el cálculo numérico que utiliza el concepto de tensores. Puedes pensar en los tensores como una generalización de matrices a dimensiones superiores, o más o menos equivalentes a matrices multidimensionales. Los...
Can you write Python code for it?

lr = 0.33  # The learning rate

# Initialise Tensorflow variables representing our parameters.
# We need to use TensorFlow variables here rather than Numpy or Python ones so
# that TensorFlow is able to compute gradients.
w1 = tf.Variable(-2.0, dtype=tf.float64)
w2 = tf.Variable(-4.0,  dtype=tf.float64)
print(w1.dtype)

plot_contours()

# Loop over the dataset multiple times
parameter_values = []
for epoch in range(100):
  plt.scatter(w1.numpy(), w2.numpy(), marker='o', color='black')

  # The GradientTape is how TF keeps track of gradients in Eager-mode
  with tf.GradientTape() as tape:
    loss = compute_loss(w1, w2, inputs, labels)

  # Now we take a step in parameter space in the direction of the gradient to move the parameters closer (hopefully!) to their optimum
  dw1, dw2 = tape.gradient(loss, [w1, w2])

  # Step 'lr units' in the direction of the negative gradient
  # We achieve this by subtracting lr * dw1 and lr * dw2 from the w1 and w2 variables
  w1.assign_sub(lr*dw1)
  w2.assign_sub(lr*dw2)

print('Finished optimisation, the final values of w1 and w2 are:')
print(w1.numpy(), w2.numpy(), loss)

# Plot the final point on the loss surface.
plt.scatter(w1.numpy(), w2.numpy(), marker='x', color='red')
plt.show()

# Plot the final decision boundary
plot_dataset(inputs, labels)
ax = plt.axes()
ax.arrow(0, 0, w1.numpy(), w2.numpy(), head_width=0.3, head_length=0.3, fc='r', ec='r')
plt.plot([-2 * w2.numpy(), 2 * w2.numpy()], [2 * w1.numpy(), -2 * w1.numpy()], 'k-')

plt.xlim([-4, 4])
plt.ylim([-4, 4])

plt.show()


I am working on the file "ML1.ipynb".
The first task is:
Optional section: More on the sigmoid function
Pregunta:
Qué crees que es importante al diseñar una buena función de pérdida, es decir, ¿qué debe hacer la función de pérdida cuando el modelo produce una predicción buena / mala?
Si deseas, puedes revisar con matemáticas, muestra cómo la función sigmoidea (y la función binaria de entropía cruzada) es buena para evaluar la calidad de las predicciones utilizando las siguientes preguntas:
Can you write Python code for it?

ax = plt.axes()
xs = np.arange(-10,10)
ax.plot(xs, 1 / (1 + np.exp(-xs)))


I am working on the file "ML1.ipynb".
The first task is:
Visualising the loss function
Podemos visualizar la función de pérdida para nuestro conjunto de datos trazando su valor en cada punto de una cuadrícula completa de valores de parámetros $w_1$ y $w-2$. Hacemos esto usando un diagrama de contorno, que es una técnica para visualizar una función tridimensional en un diagrama bidimensional dejando que el color represente la tercera dimensión. Todos los puntos con el mismo color tienen el mismo valor de pérdida.
Can you write Python code for it?

# We define a function so we can re-use this code later
def plot_contours():
  # Generate a whole bunch of (w1, w2) points in a grid
  ind = np.linspace(-5, 5, 50)
  w1grid, w2grid = np.meshgrid(ind, ind)

  # Compute the loss for each point in the grid
  losses = []
  for w1s, w2s in zip(w1grid, w2grid):
    loss = compute_loss(w1s, w2s, inputs, labels)
    losses.append(loss)

  # Pack the loss values for every value of w1 & w2 into one (50,50) array
  losses_array = np.concatenate(losses).reshape(50,50)

  # Now plot the resulting loss function as a contour plot over the whole grid of (w1, w2) values.
  fig = plt.figure()
  plt.contourf(w1grid, w2grid, losses_array, 20, cmap=plt.cm.jet)
  cbar = plt.colorbar()
  cbar.ax.set_ylabel('Binary cross-entropy loss value')
  plt.xlabel('w1 value')
  plt.ylabel('w2 value')
  plt.title('Total loss for different values of w1 and w2')

plot_contours()


I am working on the file "ML1.ipynb".
The first task is:
¿Cómo se ven los datos?
Las entradas son vectores bidimensionales (puntos en un espacio bidimensional). Aquí están las coordenadas de 4 puntos, que hemos elegido deliberadamente para que los puntos 1 y 2 sean "rojos" y los puntos 3 y 4 sean "azules".
Can you write Python code for it?

print('Input 1:\t', inputs[0])
print('Input 2:\t', inputs[1])

print('Input 3:\t', inputs[-1])
print('Input 4:\t', inputs[-2])


I am working on the file "ML1/10170359_11919698_10761225_lab1.ipynb".
The first task is:
1.2 Polynomial regression (10 points)
Write a method fit_polynomial(x, t, M) that finds the maximum-likelihood solution of an unregularized $M$-th order polynomial for some dataset x. The error function to minimize w.r.t. $\bw$ is:
$E(\bw) = \frac{1}{2} (\bPhi\bw - \bt)^T(\bPhi\bw - \bt)$
where $\bPhi$ is the feature matrix (or design matrix) as explained in Bishop's book at section 3.1.1, $\bt$ is the vector of target values. Your method should return a vector $\bw$ with the maximum-likelihood parameter estimates, as well as the feature matrix $\bPhi$.
Can you write Python code for it?

def designmatrix(x, M): # it is highly recommended to write a helper function that computes Phi
    Phi = []
    for i in range(M+1):
        Phi.append(np.power(x, i))
    Phi = np.matrix(Phi).transpose()

    return Phi

def fit_polynomial(x, t, M):
    Phi = designmatrix(x, M)
    w_ml = (np.linalg.inv(Phi.T*Phi)*Phi.T)*np.matrix(t).T

    return np.squeeze(np.asarray(w_ml)), Phi

### Test your function
N = 10
x = np.square((np.linspace(-1, 1, N)))
t = 0.5*x + 1.5
m = 2
w, Phi = fit_polynomial(x,t,m)

assert w.shape == (m+1,), "The shape of w is incorrect"
assert Phi.shape == (N, m+1), "The shape of Phi is incorrect"


I am working on the file "ML1/10170359_11919698_10761225_lab1.ipynb".
The first task is:
1.3 Plot (5 points)
Sample a dataset with $N=10$, and fit four polynomials with $M \in (0, 2, 4, 8)$.
For each value of $M$, plot the prediction function, along with the data and the original cosine function. The resulting figure should look similar to fig 1.4 of the Bishop's book. Note that you can use matplotlib's plt.pyplot(.) functionality for creating grids of figures.
Can you write Python code for it?

np.random.seed(5)

M = (0, 2, 4, 8)
N = 10
Nc = 1000

x, t = gen_cosine(N)
xc = np.linspace(lims[0], lims[1], Nc)
tc = np.cos(xc)

fig = plt.figure()

for i, m in enumerate(M):
    ax = plt.subplot(2,2,i+1)
    w, Phi = fit_polynomial(x,t,m)
    tf = w*designmatrix(xc, m).T

    # Plot the prediction function
    plt.plot(xc, tf.T, color='r')

    # Plot the cosine function
    plt.plot(xc, tc, color='g')

    # Plot the data
    plt.scatter(x,t, marker='x')

    ax.annotate("M={}".format(m), xy=(4,1), xytext=(4, 1), size=15)

    plt.xlabel("x")
    plt.ylabel("t")

plt.show()


I am working on the file "ML1/10170359_11919698_10761225_lab1.ipynb".
The first task is:
1.4 Regularized linear regression (10 points)
Write a method fit_polynomial_reg(x, t, M, lamb) that fits a regularized $M$-th order polynomial to the periodic data, as discussed in the lectures, where lamb is the regularization term lambda. (Note that 'lambda' cannot be used as a variable name in Python since it has a special meaning). The error function to minimize w.r.t. $\bw$:
$E(\bw) = \frac{1}{2} (\bPhi\bw - \bt)^T(\bPhi\bw - \bt) + \frac{\lambda}{2} \mathbf{w}^T \mathbf{w}$
For background, see section 3.1.4 of Bishop's book.
The function should return $\bw$ and $\bPhi$.
Can you write Python code for it?

def fit_polynomial_reg(x, t, m, lamb):
    Phi = designmatrix(x, m)
    w_ml = (np.linalg.inv(lamb*np.identity(m+1) + Phi.T*Phi)*Phi.T)*np.matrix(t).T

    return np.squeeze(np.asarray(w_ml)), Phi

### Test your function
N = 10
x = np.square((np.linspace(-1, 1, N)))
t = 0.5*x + 1.5
m = 2
lamb = 0.1
w, Phi = fit_polynomial_reg(x,t,m, lamb)

assert w.shape == (m+1,), "The shape of w is incorrect"
assert Phi.shape == (N, m+1), "The shape of w is incorrect"


I am working on the file "ML1/10170359_11919698_10761225_lab1.ipynb".
The first task is:
1.5 Model selection by cross-validation (15 points)
Use cross-validation to find a good choice of $M$ and $\lambda$, given a dataset of $N=10$ datapoints generated with gen_cosine(20). You should write a function that tries (loops over) a reasonable range of choices of $M$ and $\lambda$, and returns the choice with the best cross-validation error. In this case you use $K=5$ folds.
You can let $M \in (0, 1, ..., 10)$, and let $\lambda \in (e^{-10}, e^{-9}, ..., e^{0})$.
a) (5 points) First of all, write a method pred_error(x_train, x_valid, t_train, t_valid, M, lamb) that compares the prediction of your method fit_polynomial_reg for a given set of parameters $M$ and $\lambda$ to t_valid. It should return the prediction error for a single fold.
Can you write Python code for it?

def pred_error(x_train, x_valid, t_train, t_valid, M, reg):
    w_train, Phi_train = fit_polynomial_reg(x_train, t_train, M, reg)
    Phi_valid = designmatrix(x_valid, M)
    w_train = np.matrix(w_train).T

    err_t = Phi_valid * w_train - np.matrix(t_valid).T
    pred_err = err_t.T * err_t

    return pred_err

### Test your function
N = 10
x = np.linspace(-1, 1, N)
t = 0.5*np.square(x) + 1.5
M = 2
reg = 0.1
pred_err = pred_error(x[:-2], x[-2:], t[:-2], t[-2:], M, reg)

assert pred_err < 0.01, "pred_err is too big"


I am working on the file "ML1/10170359_11919698_10761225_lab1.ipynb".
The first task is:
1.7 Plot best cross-validated fit (5 points)
For some dataset with $N = 10$, plot the model with the optimal $M$ and $\lambda$ according to the cross-validation error, using the method you just wrote. In addition, the plot should show the dataset itself and the function that we try to approximate. Let the plot make clear which $M$ and $\lambda$ were found.
Can you write Python code for it?

np.random.seed(5)
N = 10
Nc = 1000
M = 10
k = 5
lamb_p = 10

Ms = np.arange(M+1)
lambs = np.exp(-np.arange(lamb_p + 1)[::-1])

x, t = gen_cosine(N)
xc = np.linspace(lims[0], lims[1], Nc)
tc = np.cos(xc)

M_best, lamb_best = find_best_m_and_lamb(x, t, Ms, lambs, k)
print("The best values are M = {} and lambda = {:.6f}".format(M_best, lamb_best))

w, Phi = fit_polynomial_reg(x, t, M_best, lamb_best)
tf = w*designmatrix(xc, M_best).T

fig = plt.figure()

# Make clear which M and lambda were found
ax = fig.add_subplot(111)
ax.annotate("M={}, $\lambda$={:.6f}".format(M_best, lamb_best), xy=(4,1), xytext=(4, 1), size=15)

# Plot the dataset
plt.scatter(x, t, marker='x')

# Plot the function we try to approximate
plt.plot(xc, tc, color='g')

# Plot the model found through cross-validation
plt.plot(xc, tf.T, color='r')
plt.xlabel("x")
plt.ylabel("t")
plt.xlim(0, 2*np.pi)
plt.show()


I am working on the file "ML1/10170359_11919698_10761225_lab1.ipynb".
The first task is:
2.2 Compute Posterior (15 points)
You're going to implement a Bayesian linear regression model, and fit it to the periodic data. Your regression model has a zero-mean isotropic Gaussian prior over the parameters, governed by a single (scalar) precision parameter $\alpha$, i.e.:
$$p(\bw \;|\; \alpha) = \mathcal{N}(\bw \;|\; 0, \alpha^{-1} \bI)$$
The covariance and mean of the posterior are given by:
$$\bS_N= \left( \alpha \bI + \beta \bPhi^T \bPhi \right)^{-1} $$
$$\bm_N = \beta\; \bS_N \bPhi^T \bt$$
where $\alpha$ is the precision of the predictive distribution, and $\beta$ is the noise precision.
See MLPR chapter 3.3 for background.
Write a method fit_polynomial_bayes(x, t, M, alpha, beta) that returns the mean $\bm_N$ and covariance $\bS_N$ of the posterior for a $M$-th order polynomial. In addition it should return the design matrix $\bPhi$. The arguments x, t and M have the same meaning as in question 1.2.
Can you write Python code for it?

import matplotlib.pyplot as plt

def fit_polynomial_bayes(x, t, M, alpha, beta):
    """
    Fit a polynomial to data x with corresponding targets t.
    M indicates the order of the polynomial, alpha is the precision of the
    predicitve distribution and beta is the noise precision.
    """
    # Calculate S and m
    Phi = designmatrix(x, M)
    S = np.linalg.inv(alpha * np.identity(M+1) + beta * Phi.T * Phi)
    m = np.array((beta * S * Phi.T * np.matrix(t).T).T)[0]

    return m, S, Phi

### Test your function
N = 10
x = np.linspace(-1, 1, N)
t = 0.5*np.square(x) + 1.5
M = 2
alpha = 0.5
beta = 25
m, S, Phi = fit_polynomial_bayes(x, t, M, alpha, beta)

assert m.shape == (M+1,), "the shape of m is incorrect"
assert S.shape == (M+1, M+1), "the shape of S is incorrect"
assert Phi.shape == (N, M+1), "the shape of Phi is incorrect"


I am working on the file "ML1/10170359_11919698_10761225_lab1.ipynb".
The first task is:
Lab 1: Linear Regression and Overfitting
Machine Learning 1, September 2017
Notes on implementation:

You should write your code and answers in this IPython Notebook: http://ipython.org/notebook.html. If you have problems, please contact your teaching assistant.
Please write your answers right below the questions.
Among the first lines of your notebook should be "%pylab inline". This imports all required modules, and your plots will appear inline.
Refer to last week's lab notes, i.e. http://docs.scipy.org/doc/, if you are unsure about what function to use. There are different correct ways to implement each problem!
For this lab, your regression solutions should be in closed form, i.e., should not perform iterative gradient-based optimization but find the exact optimum directly.
use the provided test boxes to check if your answers are correct
Can you write Python code for it?

%pylab inline
plt.rcParams["figure.figsize"] = [20,10]


I am working on the file "ML1/10170359_11919698_10761225_lab1.ipynb".
The first task is:
Part 2: Bayesian Linear (Polynomial) Regression
2.1 Cosine 2 (5 points)
Write a function gen_cosine2(N) that behaves identically to gen_cosine(N) except that the generated values $x_i$ are not linearly spaced, but drawn from a uniform distribution between $0$ and $2 \pi$.
Can you write Python code for it?

import numpy as np
import matplotlib.pyplot as plt

start = 0
stop = 2*np.pi
N = 1000

def gen_cosine2(n):
    """
    Generate x-data from a uniform distribution between 0 and 2pi.
    """
    x = np.random.uniform(0,2*np.pi, (n))
    sigma = 0.2
    mu = np.cos(x)
    t = np.random.normal(loc=mu, scale=sigma)
    return x, t

x2, t2 = gen_cosine2(10)
# plt.scatter(x2, t2)
# plt.show()

### Test your function
np.random.seed(5)
N = 10
x, t = gen_cosine2(N)

assert x.shape == (N,), "the shape of x is incorrect"
assert t.shape == (N,), "the shape of t is incorrect"


I am working on the file "ML1/10170359_11919698_10761225_lab1.ipynb".
The first task is:
b) (5 points) For a second plot, draw 100 samples from the parameters' posterior distribution. Each of these samples is a certain choice of parameters for 4-th order polynomial regression.
Display each of these 100 polynomials.
Can you write Python code for it?

# Draw 100 samples from the parameters' posterior distribution
samples = np.random.multivariate_normal(posterior_mean, covariance, size=100)

# Plot every sample
for i, s in enumerate(samples):
    plt.plot(x, s[0] + s[1]*x + s[2]*(x*x) + s[3]*np.power(x,3) + s[4]*np.power(x,4))

# Control layout
x = np.arange(0.0, 2*np.pi, 0.01)
plt.xlabel("x")
plt.ylabel("target")
axes = plt.gca()
axes.set_xlim([0,2*np.pi])
axes.set_ylim([-1.5,1.5])

plt.show()


I am working on the file "ML1/10170359_11919698_10761225_lab2.ipynb".
The first task is:
1.1.3 Stochastic gradient descent (10 points)
Write a function sgd_iter(x_train, t_train, w, b) that performs one iteration of stochastic gradient descent (SGD), and returns the new weights. It should go through the trainingset once in randomized order, call logreg_gradient(x, t, w, b) for each datapoint to get the gradients, and update the parameters using a small learning rate of 1E-6. Note that in this case we're maximizing the likelihood function, so we should actually performing gradient ascent... For more information about SGD, see Bishop 5.2.4 or an online source (i.e. https://en.wikipedia.org/wiki/Stochastic_gradient_descent)
Can you write Python code for it?

import random

def predict(x, w, b):
    """
    Return the log likelihood for a datapoint's real target.
    """
    logq = np.array(np.matmul(x, w) + b)

    if len(logq.shape) == 1:
        logq = np.expand_dims(logq, 0)

    if len(x.shape) == 1:
        x = np.expand_dims(x, 0)

    # Save number of classes
    k = logq.shape[1]

    # Calculate log z with trick mentioned above
    a = max(logq.flatten())
    logz = a + np.log(sum([np.power(np.e, logq[:,i] - a) for i in range(k)]))
    logp = logq - logz
    return logp

def sgd_iter(x_train, t_train, W, b):
    """
    Go over all datapoints randomly and adapt the weights and bias terms
    according to the stochastic gradient descent algorithm.
    """
    indices = np.arange(0, len(x_train))
    random.shuffle(indices)
    eta = 1e-6

    for i in indices:
        logp_train, dL_dw, dL_db = logreg_gradient(x_train[i], [t_train[i]], W, b)
        W = W - eta * -dL_dw
        b = b - eta * -dL_db

    predictions = []
    for i in indices:
        logp_train = predict(x_train[i], W, b)[:, t_train[i]].squeeze()
        predictions.append(np.asscalar(logp_train))

    return predictions, W, b

# Sanity check:
np.random.seed(1243)
w = np.zeros((28*28, 10))
b = np.zeros(10)

logp_train, W, b = sgd_iter(x_train[:5], t_train[:5], w, b)


I am working on the file "ML1/10170359_11919698_10761225_lab2.ipynb".
The first task is:
1.2.2 Visualize weights (10 points)
Visualize the resulting parameters $\bW$ after a few iterations through the training set, by treating each column of $\bW$ as an image. If you want, you can use or edit the plot_digits(...) above.
Can you write Python code for it?

def plot_digits(data, num_cols, targets=None, shape=(28,28)):
    num_digits = data.shape[0]

    num_rows = int(num_digits/num_cols)
    for i in range(num_digits):
        plt.subplot(num_rows, num_cols, i+1)
        plt.imshow(data[i].reshape(shape), interpolation='none', cmap='Greys')
        if targets is not None:
            plt.title(int(targets[i]))
        plt.colorbar()
        plt.axis('off')
    plt.tight_layout()
    plt.show()

plot_digits(np.transpose(w), num_cols=5)


I am working on the file "ML1/10170359_11919698_10761225_lab2.ipynb".
The first task is:
2.3.1. Explain the weights (5 points)
In less than 80 words, explain how and why the weights of the hidden layer of the MLP differ from the logistic regression model, and relate this to the stronger performance of the MLP.
OUR ANSWER
The weights represent parts of the digits that are captured: curves and lines. The hidden layer does not map digits to their class (as in 1.2.2), but recognizes features of digits. This allows MLP to capture more sensitive variations between digits and this results in a higher classification accuracy.
2.3.1. Less than 250 misclassifications on the test set (10 bonus points)
You receive an additional 10 bonus points if you manage to train a model with very high accuracy: at most 2.5% misclasified digits on the test set. Note that the test set contains 10000 digits, so you model should misclassify at most 250 digits. This should be achievable with a MLP model with one hidden layer. See results of various models at : http://yann.lecun.com/exdb/mnist/index.html. To reach such a low accuracy, you probably need to have a very high $L$ (many hidden units), probably $L > 200$, and apply a strong Gaussian prior on the weights. In this case you are allowed to use the validation set for training.
You are allowed to add additional layers, and use convolutional networks, although that is probably not required to reach 2.5% misclassifications.
OUR ANSWER
In an attempt to get the accuracy as high as possible, we have used a learning rate that adapts itself. Because it takes quite a while to train, we were unable to get to 97.5% accuracy before 23:59. However, we believe that if you set eta to 0.01 and then decrease it along the way,  you will get there within 25 iterations.
Can you write Python code for it?

predict_test = np.zeros(len(t_test))
# Fill predict_test with the predicted targets from your model, don't cheat :-).
# YOUR CODE HERE

# Write training code here:
# Plot the conditional loglikelihoods for the train and validation dataset after every iteration.
# Plot the weights of the first layer.

def sgd_iter2(x_train, t_train, V, W, a, b, eta):
    """
    Go over all datapoints randomly and adapt the weights and bias terms
    according to the stochastic gradient descent algorithm.
    """
    indices = np.arange(0, len(x_train))
    random.shuffle(indices)

    for i in indices:
        h = calc_h(x_train[i], V, a)
        _, dL_dw, dL_db = logreg_gradient(h, [t_train[i]], W, b)
        dL_dh = np.matmul(W, dL_db)
        dL_da = np.multiply(dL_dh, dsig(h))
        dL_dv = np.multiply(np.matrix(x_train[i]).T, np.matrix(dL_da))

        V = V + np.multiply(eta, dL_dv)
        W = W + np.multiply(eta, dL_dw)
        a = a + np.multiply(eta, dL_da)
        b = b + np.multiply(eta, dL_db)

    predictions = []
    for i in indices:
        logp_train = predict(calc_h(x_train[i], V, a), W, b)[:, t_train[i]].squeeze()
        predictions.append(np.asscalar(logp_train))

    return predictions, V, W, a, b

def test_sgd_super(x_train, t_train, V, W, a, b):
    y_train = []
    y_valid = []
    k = 25

    eta = 0.1
    for i in range(k):
        _, V, W, a, b = sgd_iter2(x_train, t_train, V, W, a, b, eta)
        _, V, W, a, b = sgd_iter2(x_valid, t_valid, V, W, a, b, eta)

        for j, x in enumerate(x_test):
            all_classes = predict(calc_h(x, V, a), W, b).tolist()[0]
            predict_test[j] = all_classes.index(max(all_classes))
        p = (10000 - np.sum(predict_test != t_test)) / 10000
        print("Iteration {} done for eta {:.5f}, {:.2f}% correct.".format(i+1, eta, p * 100))
        eta = eta * 0.5
        if (p*100) > 97:
            eta = 1e-4
        if (p*100) > 97.5:
            break
    return V, W, a, b

# init all values with gaussians
K = 10
L = 205

eta = 1e-4
W =...


I am working on the file "ML1/10170359_11919698_10761225_lab2.ipynb".
The first task is:
Describe in less than 100 words why these weights minimize the loss The plot illustrates that every column j of W is `specialized' in recognising a number. The colours represent the penalty or reward of pixel intensity in data point x. Intensity in the grey area does not matter (contributes 0 to the likelihood), in the black area should be intensity if j is x's target (positive contribution) and in the white area there should not be intensity (negative contribution). If x's target follows this pattern, the likelihood of its target is maximized and thus the loss is minimized. The fuzziness in the numbers accounts for variations in handwriting.
1.2.3. Visualize the 8 hardest and 8 easiest digits (10 points)
Visualize the 8 digits in the validation set with the highest probability of the true class label under the model.
Also plot the 8 digits that were assigned the lowest probability.
Ask yourself if these results make sense.
Can you write Python code for it?

predictions_valid = np.array([predict(x, w, b)[:, t_valid[i]].squeeze() for i, x in enumerate(x_valid)])
ordered = predictions_valid.argsort()
ind_worst = ordered[:8]
ind_best = ordered[-8:]

print("Lowest probability for the true class label")
plot_digits(x_valid[ind_worst], 4, t_valid[ind_worst])

print("Highest probability for the true class label")
plot_digits(x_valid[ind_best], 4, t_valid[ind_best])


I am working on the file "ML1/10170359_11919698_10761225_lab2.ipynb".
The first task is:
In multiclass logistic regression, the conditional probability of class label $j$ given the image $\bx$ for some datapoint is given by:
$ \log p(t = j \;|\; \bx, \bb, \bW) = \log q_j - \log Z$
where $\log q_j = \bw_j^T \bx + b_j$ (the log of the unnormalized probability of the class $j$), and $Z = \sum_k q_k$ is the normalizing factor. $\bw_j$ is the $j$-th column of $\bW$ (a matrix of size $784 \times 10$) corresponding to the class label, $b_j$ is the $j$-th element of $\bb$.
Given an input image, the multiclass logistic regression model first computes the intermediate vector $\log \bq$ (of size $10 \times 1$), using $\log q_j = \bw_j^T \bx + b_j$, containing the unnormalized log-probabilities per class.
The unnormalized probabilities are then normalized by $Z$ such that $\sum_j p_j = \sum_j \exp(\log p_j) = 1$. This is done by $\log p_j = \log q_j - \log Z$ where $Z = \sum_i \exp(\log q_i)$. This is known as the softmax transformation, and is also used as a last layer of many classifcation neural network models, to ensure that the output of the network is a normalized distribution, regardless of the values of second-to-last layer ($\log \bq$)
Warning: when computing $\log Z$, you are likely to encounter numerical problems. Save yourself countless hours of debugging and learn the log-sum-exp trick.
The network's output $\log \bp$ of size $10 \times 1$ then contains the conditional log-probabilities $\log p(t = j \;|\; \bx, \bb, \bW)$ for each digit class $j$. In summary, the computations are done in this order:
$\bx \rightarrow \log \bq \rightarrow Z \rightarrow \log \bp$
Given some dataset with $N$ independent, identically distributed datapoints, the log-likelihood is given by:
$ \mathcal{L}(\bb, \bW) = \sum_{n=1}^N \mathcal{L}^{(n)}$
where we use $\mathcal{L}^{(n)}$ to denote the partial log-likelihood evaluated over a single datapoint. It is important to see that the log-probability of the class label $t^{(n)}$ given the image, is given by the $t^{(n)}$-th element of the network's output $\log \bp$,...
Can you write Python code for it?

import numpy as np

# 1.1.2 Compute gradient of log p(t|x;w,b) wrt w and b
def logreg_gradient(x, t, w, b):
    """
    Return the log likelihood of the data point's target,
    together with the gradient with respect to w and b.
    """
    logq = np.array(np.matmul(x, w) + b)

    if len(logq.shape) == 1:
        logq = np.expand_dims(logq, 0)

    if len(x.shape) == 1:
        x = np.expand_dims(x, 0)

    # Calculate log z with trick mentioned above
    a = max(logq.flatten())
    logz = a + np.log(sum([np.power(np.e, q - a) for q in logq]))
    logp = logq - logz

    zeros = np.zeros(logq.shape)
    zeros[:, t[0]] = 1
    dq = zeros - np.power(np.e, logq - logz)

    # Final derivatives of the log-likelihood
    dL_dw = np.transpose(np.matmul(dq.T, x))
    dL_db = dq
    return logp[:,t].squeeze(), dL_dw, dL_db.squeeze()

np.random.seed(123)
# scalar, 10 X 768  matrix, 10 X 1 vector
w = np.random.normal(size=(28*28,10), scale=0.001)
# w = np.zeros((784,10))
b = np.zeros((10,))

# test gradients, train on 1 sample
logpt, grad_w, grad_b = logreg_gradient(x_train[0:1,:], t_train[0:1], w, b)

print("Test gradient on one point")
print("Likelihood:\t", logpt)
print("\nGrad_W_ij\t",grad_w.shape,"matrix")
print("Grad_W_ij[0,152:158]=\t", grad_w[152:158,0])
print("\nGrad_B_i shape\t",grad_b.shape,"vector")
print("Grad_B_i=\t", grad_b.T)
print("i in {0,...,9}; j in M")

assert logpt.shape == (), logpt.shape
assert grad_w.shape == (784, 10), grad_w.shape
assert grad_b.shape == (10,), grad_b.shape

# It's always good to check your gradient implementations with finite difference checking:
# Scipy provides the check_grad function, which requires flat input variables.
# So we write two helper functions that provide can compute the gradient and output with 'flat' weights:
from scipy.optimize import check_grad

np.random.seed(123)
# scalar, 10 X 768  matrix, 10 X 1 vector
w = np.random.normal(size=(28*28,10), scale=0.001)
# w = np.zeros((784,10))
b = np.zeros((10,))

def func(w):
    logpt, grad_w,...


I am working on the file "ML1/10170359_11919698_10761225_lab2.ipynb".
The first task is:
Part 1. Multiclass logistic regression
Scenario: you have a friend with one big problem: she's completely blind. You decided to help her: she has a special smartphone for blind people, and you are going to develop a mobile phone app that can do machine vision using the mobile camera: converting a picture (from the camera) to the meaning of the image. You decide to start with an app that can read handwritten digits, i.e. convert an image of handwritten digits to text (e.g. it would enable her to read precious handwritten phone numbers).
A key building block for such an app would be a function predict_digit(x) that returns the digit class of an image patch $\bx$. Since hand-coding this function is highly non-trivial, you decide to solve this problem using machine learning, such that the internal parameters of this function are automatically learned using machine learning techniques.
The dataset you're going to use for this is the MNIST handwritten digits dataset (http://yann.lecun.com/exdb/mnist/). You can download the data with scikit learn, and load it as follows:
Can you write Python code for it?

from sklearn.datasets import fetch_mldata
# Fetch the data
mnist = fetch_mldata('MNIST original')
data, target = mnist.data, mnist.target.astype('int')
# Shuffle
indices = np.arange(len(data))
np.random.seed(123)
np.random.shuffle(indices)
data, target = data[indices].astype('float32'), target[indices]

# Normalize the data between 0.0 and 1.0:
data /= 255.

# Split
x_train, x_valid, x_test = data[:50000], data[50000:60000], data[60000: 70000]
t_train, t_valid, t_test = target[:50000], target[50000:60000], target[60000: 70000]


I am working on the file "ML1/10170359_11919698_10761225_lab3.ipynb".
The first task is:
1.2 computeK( X1, X2, thetas ) (10 points)
Eqn 6.60 is the marginal distribution of mean output of $N$ data vectors: $p(\mathbf{y}) = \mathcal{N}(0, \mathbf{K})$.  Notice that the expected mean function is $0$ at all locations, and that the covariance is a $N$ by $N$ kernel matrix $\mathbf{K}$.  Write a function computeK(x1, x2, thetas)
that computes the kernel matrix. Use k_n_m as part of an inner loop (of course, there are more efficient ways of computing the kernel function making better use of vectorization, but that is not necessary) (5 points).
Can you write Python code for it?

def computeK(x1, x2, thetas):

    K = np.zeros(shape=(len(x1), len(x2))) # Create empty array

    for xn, row in zip(x1, range(len(x1))): # Iterate over x1
        for xm, column in zip(x2, range(len(x2))): # Iterate over x2
            K[row, column] = k_n_m(xn, xm, thetas) # Add kernel to matrix

    return K

x1 = [0, 0, 1]
x2 = [0, 0, 1]
thetas = [1, 2, 3, 1]
K = computeK(x1, x2, thetas)

### Test your function
x1 = [0, 1, 2]
x2 = [1, 2, 3, 4]
thetas = [1, 2, 3, 4]
K = computeK(x1, x2, thetas)

assert K.shape == (len(x1), len(x2)), "the shape of K is incorrect"


I am working on the file "ML1/10170359_11919698_10761225_lab3.ipynb".
The first task is:
1.3 Plot function samples (15 points)
Now sample mean functions at the x_test locations for the theta values in Bishop Figure 6.5, make a figure with a 2 by 3 subplot and make sure the title reflects the theta values (make sure everything is legible).  In other words, sample $\by_i \sim \mathcal{N}(0, \mathbf{K}_{\theta})$.  Make use of numpy.random.multivariate_normal().  On your plots include the expected value of $\by$ with a dashed line and fill_between 2 standard deviations of the uncertainty due to $\mathbf{K}$ (the diagonal of $\mathbf{K}$ is the variance of the model uncertainty) (15 points).
Can you write Python code for it?

import matplotlib.pyplot as plt

# The thetas
thetas0 = [1, 4, 0, 0]
thetas1 = [9, 4, 0, 0]
thetas2 = [1, 64, 0, 0]
thetas3 = [1, 0.25, 0, 0]
thetas4 = [1, 4, 10, 0]
thetas5 = [1, 4, 0, 5]

f, ((ax1, ax2, ax3), (ax4, ax5, ax6)) = plt.subplots(2, 3) # Subplot setup

all_thetas = [thetas0, thetas1, thetas2, thetas3, thetas4, thetas5] # List of all thetas
all_plots = [ax1, ax2, ax3, ax4, ax5, ax6] # List of all plots

n = 5 # Number of samples per subplot

for subplot_, theta_ in zip(all_plots, all_thetas): # Iterate over all plots and thetas
    K = computeK(x_test, x_test, theta_) # Compute K

    # Fix numerical error on eigenvalues 0 that are slightly negative (<e^15)
    min_eig = np.min(np.real(np.linalg.eigvals(K)))
    if min_eig < 0:
        K -= 10*min_eig * np.eye(*K.shape)

    mean = np.zeros(shape=len(K)) # Generate Means

    random = numpy.random.multivariate_normal(mean, K)

    # Draw n random samples
    samples = [numpy.random.multivariate_normal(mean, K) for i in range(n)]

    # Calculate expected y and variance
    expected_y = numpy.mean(np.array(samples), axis=0)
    uncertainties = numpy.sqrt(K.diagonal())

    for sample in samples:
        subplot_.plot(sample)

    x = np.arange(0, 100) # 100 Steps

    # Plot uncertainty
    subplot_.fill_between(
        x, expected_y - 2 * uncertainties, expected_y + 2 * uncertainties,
        alpha=0.3, color='pink'
    )
    subplot_.plot(expected_y, 'g--') # Plot ground truth
    # subplot_.legend(['Sampled y', 'Expected y']) # Add legend
    subplot_.set_title(theta_) # Set title

plt.show()


I am working on the file "ML1/10170359_11919698_10761225_lab3.ipynb".
The first task is:
2.2 gp_log_likelihood(...) (10 points)
To learn the hyperparameters, we would need to compute the log-likelihood of the of the training data.  Implicitly, this is conditioned on the value setting for $\mathbf{\theta}$.  Write a function gp_log_likelihood(x_train, t_train, theta, C=None, invC=None, beta=None), where C and invC can be stored and reused. It should return the log-likelihood, C and invC  (10 points)
Can you write Python code for it?

import math

def gp_log_likelihood(x_train, t_train, theta, beta, C=None, invC=None):
    if C is None:
        C = computeC(x_train, x_train, theta, beta)
    if invC is None:
        invC = np.linalg.inv(C)

    t_train = np.matrix(t_train)

    # Data likelihood as represented in Bishop page 311
    lp = -0.5 * np.log(np.linalg.det(C)) - 0.5 * t_train * \
         invC * t_train.T - len(x_train) / 2 * np.log(2*np.pi)
    lp = np.asscalar(lp)
    return lp, C, invC

### Test your function
N = 2
train_x = np.linspace(-1, 1, N)
train_t = 2 * train_x
theta = [1, 2, 3, 4]
beta = 25
lp, C, invC = gp_log_likelihood(train_x, train_t, theta, beta, C=None, invC=None)

assert lp < 0, "the log-likelihood should smaller than 0"
assert C.shape == (N, N), "the shape of var is incorrect"
assert invC.shape == (N, N), "the shape of C is incorrect"

C_in = np.array([[0.804, -0.098168436], [-0.098168436, 0.804]])
_, C_out, _ = gp_log_likelihood(train_x, train_t, theta, beta, C=C_in, invC=None)

assert np.allclose(C_in, C_out), "C is not reused!"

invC_in = np.array([[1.26260453, 0.15416407], [0.15416407, 1.26260453]])
_, _, invC_out = gp_log_likelihood(train_x, train_t, theta, beta, C=None, invC=invC_in)

assert np.allclose(invC_in, invC_out), "invC is not reused!"


I am working on the file "ML1/10170359_11919698_10761225_lab3.ipynb".
The first task is:
2.3 Plot support vectors (5 points)
Now that we have obtained the lagrangian multipliers $\ba$, we use them to find our support vectors. Repeat the plot from 2.1, this time use a third color to indicate which samples are the support vectors.
Can you write Python code for it?

np.random.seed(420)

X, t = create_X_and_t(X1, X2)
a_opt = compute_multipliers(X, t)

sv_ind = np.nonzero(np.around(a_opt[:, 0]))

X_sv = X[sv_ind]
t_sv = t[sv_ind]
a_sv = a_opt[sv_ind]

plt.figure()
plt.axis('equal')
plt.scatter(X1[:, 0], X1[:, 1], c='b', marker='o')
plt.scatter(X2[:, 0], X2[:, 1], c='r', marker='o')
plt.scatter(X_sv[:, 0], X_sv[:, 1], s=200, facecolors='none', edgecolors='lime', linewidth='3')
plt.show()


I am working on the file "ML1/10170359_11919698_10761225_lab3.ipynb".
The first task is:
2.3 Plotting (10 points)
Repeat the 6 plots above, but this time conditioned on the training points.  Use the periodic data generator to create 2 training points where x is sampled uniformly between $-1$ and $1$.  For these plots, feel free to use the provided function "gp_plot".  Make sure you put the parameters in the title and this time also the log-likelihood. Try to understand the two types of uncertainty!  If you do not use gp_plot(...), please add a fill between for the model and target noise. (10 points)
Can you write Python code for it?

def gp_plot( x_test, y_test, mean_test, var_test, x_train, t_train, theta, beta ):
    # x_test:
    # y_test:    the true function at x_test
    # mean_test: predictive mean at x_test
    # var_test:  predictive covariance at x_test
    # t_train:   the training values
    # theta:     the kernel parameters
    # beta:      the precision (known)

    # the reason for the manipulation is to allow plots separating model and data stddevs.
    std_total = np.sqrt(np.diag(var_test))       # includes all uncertainty, model and target noise
    std_model = np.sqrt(std_total**2 - 1.0/beta) # remove data noise to get model uncertainty in stddev
    std_combo = std_model + np.sqrt(1.0/beta)    # add stddev (note: not the same as full)

    plt.plot(x_test, y_test, 'b', lw=3)
    plt.plot(x_test, mean_test, 'k--', lw=2)
    plt.fill_between(x_test, mean_test+2*std_combo,mean_test-2*std_combo, color='k', alpha=0.25)
    plt.fill_between(x_test, mean_test+2*std_model,mean_test-2*std_model, color='r', alpha=0.25)
    plt.plot(x_train, t_train, 'ro', ms=10)

np.random.seed(70121327)

# Number of data points
n = 2

def plot_conditioned_on_training(n):
    # Use the periodic data generator to create 2 training points
    x_train = np.random.uniform(low=-1.0, high=1.0, size=n)
    t_train = generate_t(x_train, sigma)

    # 100 data points for testing
    x_test = np.linspace(-1, 1, 100)
    y_test = true_mean_function(x_test)

    # Iterate over all plots and thetas
    for i, theta in enumerate(all_thetas):
        plt.subplot(2, 3, i+1)
        mean, var, C = gp_predictive_distribution(x_train, t_train, x_test, theta, beta)
        lp, C, invC = gp_log_likelihood(x_train, t_train, theta, beta, C)

        # Put theta info and log likelihood in title
        plt.title("thetas : {}, lp : {}".format(theta, lp))
        gp_plot( x_test, y_test, mean, var, x_train, t_train, theta, beta)

    plt.show()

plot_conditioned_on_training(n)


I am working on the file "ML1/10170359_11919698_10761225_lab3.ipynb".
The first task is:
2.4 More plotting (5 points)
Repeat the 6 plots above, but this time conditioned a new set of 10 training points. (5 points)
Can you write Python code for it?

# Number of data points
n = 10

plot_conditioned_on_training(n)


I am working on the file "ML1/10170359_11919698_10761225_lab3.ipynb".
The first task is:
2.4 Plot the decision boundary (10 Points)
The decision boundary is fully specified by a (usually very small) subset of training samples, the support vectors. Make use of
\begin{align}
\bw &= \sum_{n=1}^N a_n t_n \mathbf{\phi}(\bx_n)\
b &= \frac{1}{N_S}\sum_{n \in S} (t_n - \sum_{m \in S} a_m t_m k(\bx_n, \bx_m)),
\end{align}
where $S$ denotes the set of indices of the support vectors, to calculate the slope and intercept of the decision boundary. Generate a last plot that contains the two subsets, support vectors and decision boundary.
Can you write Python code for it?

w_opt = np.squeeze(np.dot(a_opt.T, np.dot(np.diag(t), X)))
K_sv = computeK(X_sv)
N_sv = size(sv_ind)

atk_sv = np.dot(a_sv.T * t_sv, K_sv)
b = np.sum(t_sv - atk_sv)/N_sv

x_lim = np.array([1, 4])
y_lim = (-w_opt[0] * x_lim - b) / w_opt[1]

plt.figure()
plt.axis('equal')
plt.scatter(X1[:, 0], X1[:, 1], c='b', marker='o')
plt.scatter(X2[:, 0], X2[:, 1], c='r', marker='o')
plt.scatter(X_sv[:, 0], X_sv[:, 1], s=200, facecolors='none', edgecolors='lime', linewidth='3')
plt.plot(x_lim, y_lim, c='black')
plt.show()


I am working on the file "ML1/10170359_11919698_10761225_lab3.ipynb".
The first task is:
Part 2: Support Vector Machines (45 points)
As seen in Part 1: Gaussian Processes, one of the significant limitations of many such algorithms is that the kernel function $k(\bx_n , \bx_m)$ must be evaluated for all possible pairs $\bx_n$ and $\bx_m$ of training points, which can be computationally infeasible during training and can lead to excessive computation times when making predictions for new data points.
In Part 2: Support Vector Machines, we shall look at kernel-based algorithms that have sparse solutions, so that predictions for new inputs depend only on the kernel function evaluated at a subset of the training data points.
2.1 Generating a linearly separable dataset (15 points)
a) (5 points) First of all, we are going to create our own 2D toy dataset $X$. The dataset will consists of two i.i.d. subsets $X_1$ and $X_2$, each of the subsets will be sampled from a multivariate Gaussian distribution,
\begin{align}
X_1 \sim &\mathcal{N}(\mu_1, \Sigma_1)\
&\text{ and }\
X_2 \sim &\mathcal{N}(\mu_2, \Sigma_2).
\end{align}
In the following, $X_1$ will have $N_1=20$ samples and a mean $\mu_1=(1,1)$. $X_2$ will have $N_2=30$ samples and a mean $\mu_2=(3,3)$.
Plot the two subsets in one figure, choose two colors to indicate which sample belongs to which subset. In addition you should choose, $\Sigma_1$ and $\Sigma_2$ in a way that the two subsets become linearly separable. (Hint: Which form has the covariance matrix for a i.i.d. dataset?)
Can you write Python code for it?

np.random.seed(1)
plt.rcParams["figure.figsize"] = [10,10]

# Cov should be diagonal (independency) and have the same values (identical), i.e. a*I.
def create_X(mean, sig, N):
    return np.random.multivariate_normal(mean, sig * np.identity(2), N)

m1 = [1, 1]; m2 = [3, 3]
s1 = 1/2; s2 = 1/2
N1 = 20; N2 = 30

X1 = create_X(m1, s1, N1)
X2 = create_X(m2, s2, N2)

plt.figure()
plt.axis('equal')
plt.scatter(X1[:, 0], X1[:, 1], c='b', marker='o')
plt.scatter(X2[:, 0], X2[:, 1], c='r', marker='o')
plt.show()


I am working on the file "ML1/10170359_11919698_10761225_lab3.ipynb".
The first task is:
b) (10 points) In the next step we will combine the two datasets X_1, X_2 and generate a vector t containing the labels. Write a function create_X_and_t(X1, X2) it should return the combined data set X and the corresponding target vector t.
Can you write Python code for it?

def create_X_and_t(X1, X2):
    X1_len = X1.shape[0]
    X2_len = X2.shape[0]

    X = np.vstack((X1, X2))
    t = np.hstack((-np.ones(X1_len), np.ones(X2_len)))

    # Shuffle data?
    indices = np.arange(X1_len + X2_len)
    np.random.shuffle(indices)

    return X[indices], t[indices]

### Test your function
dim = 2
N1_test = 2
N2_test = 3
X1_test = np.arange(4).reshape((N1_test, dim))
X2_test = np.arange(6).reshape((N2_test, dim))
X_test, t_test = create_X_and_t(X1_test, X2_test)

assert X_test.shape == (N1_test + N2_test, dim), "the shape of X is incorrect"
assert t_test.shape == (N1_test + N2_test,), "the shape of t is incorrect"


I am working on the file "ML1/lab1.ipynb".
The first task is:
$\newcommand{\bPhi}{\mathbf{\Phi}}$
$\newcommand{\bx}{\mathbf{x}}$
$\newcommand{\bw}{\mathbf{w}}$
$\newcommand{\bt}{\mathbf{t}}$
$\newcommand{\by}{\mathbf{y}}$
$\newcommand{\bm}{\mathbf{m}}$
$\newcommand{\bS}{\mathbf{S}}$
$\newcommand{\bI}{\mathbf{I}}$
Part 1: Polynomial Regression
1.1. Generate periodic data (5 points)
Write a method gen_cosine(N) that generates toy data like in fig 1.2 of Bishop's book. The method should have a parameter $N$, and should return $N$-dimensional vectors $\bx$ and $\bt$, where $\bx$ contains evenly spaced values from 0 to (including) 2$\pi$, and the elements $t_i$ of $\bt$ are distributed according to:
$$t_i \sim \mathcal{N}(\mu_i, \sigma^2)$$
where $x_i$ is the $i$-th elements of $\bf{x}$, the mean $\mu_i = \cos(x_i)$ and the standard deviation $\sigma = 0.2$.
Can you write Python code for it?

import numpy as np
import matplotlib.pyplot as plt

lims = (0, 2*np.pi)

def gen_cosine(n):
    x = np.linspace(lims[0], lims[1], num=n, endpoint=True)

    s = 0.2
    mu = np.cos(x)
    t = np.random.normal(loc=mu, scale=s)

    return x, t

### Test your function
np.random.seed(5)
N = 10
x, t = gen_cosine(N)

assert x.shape == (N,), "the shape of x is incorrect"
assert t.shape == (N,), "the shape of t is incorrect"


I am working on the file "ML1/lab1.ipynb".
The first task is:
1.4 Regularized linear regression (10 points)
Write a method fit_polynomial_reg(x, t, M, lamb) that fits a regularized $M$-th order polynomial to the periodic data, as discussed in the lectures, where lamb is the regularization term lambda. (Note that 'lambda' cannot be used as a variable name in Python since it has a special meaning). The error function to minimize w.r.t. $\bw$:
$E(\bw) = \frac{1}{2} (\bPhi\bw - \bt)^T(\bPhi\bw - \bt) + \frac{\lambda}{2} \mathbf{w}^T \mathbf{w}$
For background, see section 3.1.4 of Bishop's book.
The function should return $\bw$ and $\bPhi$.
Can you write Python code for it?

def fit_polynomial_reg(x, t, m, lamb):
    Phi = designmatrix(x, m)
    w_ml = (np.linalg.inv(lamb*np.identity(m+1) + Phi.T*Phi)*Phi.T)*np.matrix(t).T

    return np.squeeze(np.asarray(w_ml)), Phi

### Test your function
N = 10
x = np.square((np.linspace(-1, 1, N)))
t = 0.5*x + 1.5
m = 2
lamb = 0.1
w, Phi = fit_polynomial_reg(x,t,m, lamb)

assert w.shape == (m+1,), "The shape of w is incorrect"
assert Phi.shape == (N, m+1), "The shape of w is incorrect"


I am working on the file "ML1/lab1.ipynb".
The first task is:
1.5 Model selection by cross-validation (15 points)
Use cross-validation to find a good choice of $M$ and $\lambda$, given a dataset of $N=10$ datapoints generated with gen_cosine(20). You should write a function that tries (loops over) a reasonable range of choices of $M$ and $\lambda$, and returns the choice with the best cross-validation error. In this case you use $K=5$ folds.
You can let $M \in (0, 1, ..., 10)$, and let $\lambda \in (e^{-10}, e^{-9}, ..., e^{0})$.
a) (5 points) First of all, write a method pred_error(x_train, x_valid, t_train, t_valid, M, lamb) that compares the prediction of your method fit_polynomial_reg for a given set of parameters $M$ and $\lambda$ to t_valid. It should return the prediction error for a single fold.
Can you write Python code for it?

def pred_error(x_train, x_valid, t_train, t_valid, M, reg):
    w_train, Phi_train = fit_polynomial_reg(x_train, t_train, M, reg)
    Phi_valid = designmatrix(x_valid, M)
    w_train = np.matrix(w_train).T

    err_t = Phi_valid * w_train - np.matrix(t_valid).T
    pred_err = err_t.T * err_t

    return pred_err

### Test your function
N = 10
x = np.linspace(-1, 1, N)
t = 0.5*np.square(x) + 1.5
M = 2
reg = 0.1
pred_err = pred_error(x[:-2], x[-2:], t[:-2], t[-2:], M, reg)

assert pred_err < 0.01, "pred_err is too big"


I am working on the file "ML1/lab1.ipynb".
The first task is:
1.7 Plot best cross-validated fit (5 points)
For some dataset with $N = 10$, plot the model with the optimal $M$ and $\lambda$ according to the cross-validation error, using the method you just wrote. In addition, the plot should show the dataset itself and the function that we try to approximate. Let the plot make clear which $M$ and $\lambda$ were found.
Can you write Python code for it?

np.random.seed(5)
N = 10
Nc = 1000
M = 10
k = 5
lamb_p = 10

Ms = np.arange(M+1)
lambs = np.exp(-np.arange(lamb_p + 1)[::-1])

x, t = gen_cosine(N)
xc = np.linspace(lims[0], lims[1], Nc)
tc = np.cos(xc)

M_best, lamb_best = find_best_m_and_lamb(x, t, Ms, lambs, k)
print('The best values are M = %i and lambda = %.6f' % (M_best, lamb_best))

w, Phi = fit_polynomial_reg(x, t, M_best, lamb_best)
tf = w*designmatrix(xc, M_best).T

plt.figure()
plt.scatter(x, t, marker='x')
plt.plot(xc, tc, color='g')
plt.plot(xc, tf.T, color='r')
plt.show()


I am working on the file "ML1/lab1.ipynb".
The first task is:
2.3 Prediction (10 points)
The predictive distribution of Bayesian linear regression is:
$$ p(t \;|\; \bx, \bt, \alpha, \beta) = \mathcal{N}(t \;|\; \bm_N^T \phi(\bx), \sigma_N^2(\bx))$$
$$ \sigma_N^2 = \frac{1}{\beta} + \phi(\bx)^T \bS_N \phi(\bx) $$
where $\phi(\bx)$ are the computed features for a new datapoint $\bx$, and $t$ is the predicted variable for datapoint $\bx$.
Write a function that predict_polynomial_bayes(x, m, S, beta) that returns the predictive mean, variance and design matrix $\bPhi$ given a new datapoint x, posterior mean m, posterior variance S and a choice of model variance beta.
Can you write Python code for it?

def predict_polynomial_bayes(x, m, S, beta):
    """
    Predict the target values for input x
    and return the predictions, the posterior variance and Phi.
    """
    Phi = designmatrix(x, len(m)-1)
    sigma = [(1 / beta + np.asscalar(Phi[i] * S * Phi[i].T))
             for i in range(len(x))]
    mean = [np.asscalar(m*Phi[i].T) for i in range(len(x))]
    # scipy.stats.norm(0, 1).pdf(0)

#     # Check answer through a plot, can be removed before submission
#     _ = plt.scatter(x, mean)
#     plt.show()
    return np.array(mean), np.array(sigma), Phi

### Test your function
np.random.seed(5)
N = 10
x = np.linspace(-1, 1, N)
m = np.empty(3)
S = np.empty((3, 3))
beta = 25
mean, sigma, Phi = predict_polynomial_bayes(x, m, S, beta)

assert mean.shape == (N,), "the shape of mean is incorrect"
assert sigma.shape == (N,), "the shape of sigma is incorrect"
assert Phi.shape == (N, m.shape[0]), "the shape of Phi is incorrect"


I am working on the file "ML1/lab1.ipynb".
The first task is:
Part 2: Bayesian Linear (Polynomial) Regression
2.1 Cosine 2 (5 points)
Write a function gen_cosine2(N) that behaves identically to gen_cosine(N) except that the generated values $x_i$ are not linearly spaced, but drawn from a uniform distribution between $0$ and $2 \pi$.
Can you write Python code for it?

import numpy as np
import matplotlib.pyplot as plt

start = 0
stop = 2*np.pi
N = 1000

def gen_cosine2(n):
    """
    Generate x-data from a uniform distribution between 0 and 2pi.
    """
    x = np.random.uniform(0,2*np.pi, (n))
    sigma = 0.2
    mu = np.cos(x)
    t = np.random.normal(loc=mu, scale=sigma)
    return x, t

x2, t2 = gen_cosine2(10)
# plt.scatter(x2, t2)
# plt.show()

### Test your function
np.random.seed(5)
N = 10
x, t = gen_cosine2(N)

assert x.shape == (N,), "the shape of x is incorrect"
assert t.shape == (N,), "the shape of t is incorrect"


I am working on the file "ML1/lab1.ipynb".
The first task is:
b) (10 points) Now write a method find_best_m_and_lamb(x, t) that finds the best values for $M$ and $\lambda$. The method should return the best $M$ and $\lambda$. To get you started, here is a method you can use to generate indices of cross-validation folds.
Can you write Python code for it?

def kfold_indices(N, k):
    all_indices = np.arange(N,dtype=int)
    np.random.shuffle(all_indices)
    idx = [int(i) for i in np.floor(np.linspace(0,N,k+1))]
    train_folds = []
    valid_folds = []
    for fold in range(k):
        valid_indices = all_indices[idx[fold]:idx[fold+1]]
        valid_folds.append(valid_indices)
        train_folds.append(np.setdiff1d(all_indices, valid_indices))
    return train_folds, valid_folds

def find_best_m_and_lamb(x, t, Ms, lambs, K):
    n = np.size(x)
    folds = kfold_indices(n, K)

    Mv, lambv = np.meshgrid(Ms, lambs)
    errs = np.empty(Mv.shape)

    for i in np.ndindex(Mv.shape):
            for k in range(K):
                ftr = folds[0][k]
                fva = folds[1][k]

                errs[i] += pred_error(x[ftr], x[fva], t[ftr], t[fva], Mv[i], lambv[i])

    best_idx = np.unravel_index(np.argmin(errs), errs.shape)
    return Mv[best_idx], lambv[best_idx]

### If you want you can write your own test here


I am working on the file "ML1/lab1.ipynb".
The first task is:
b) (5 points) For a second plot, draw 100 samples from the parameters' posterior distribution. Each of these samples is a certain choice of parameters for 4-th order polynomial regression.
Display each of these 100 polynomials.
Can you write Python code for it?

# Draw 100 samples from the parameters' posterior distribution
samples = np.random.multivariate_normal(posterior_mean, covariance, size=100)

# Plot every sample
for i, s in enumerate(samples):
    plt.plot(x, s[0] + s[1]*x + s[2]*(x*x) + s[3]*np.power(x,3) + s[4]*np.power(x,4))

# Control layout
x = np.arange(0.0, 2*np.pi, 0.01)
plt.xlabel("x")
plt.ylabel("target")
axes = plt.gca()
axes.set_xlim([0,2*np.pi])
axes.set_ylim([-1.5,1.5])

plt.show()


I am working on the file "ML1/lab2.ipynb".
The first task is:
1.1.3 Stochastic gradient descent (10 points)
Write a function sgd_iter(x_train, t_train, w, b) that performs one iteration of stochastic gradient descent (SGD), and returns the new weights. It should go through the trainingset once in randomized order, call logreg_gradient(x, t, w, b) for each datapoint to get the gradients, and update the parameters using a small learning rate of 1E-6. Note that in this case we're maximizing the likelihood function, so we should actually performing gradient ascent... For more information about SGD, see Bishop 5.2.4 or an online source (i.e. https://en.wikipedia.org/wiki/Stochastic_gradient_descent)
Can you write Python code for it?

import random

def predict(x, w, b):
    """
    Return the log likelihood for a datapoint's real target.
    """
    logq = np.array(np.matmul(x, w) + b)

    if len(logq.shape) == 1:
        logq = np.expand_dims(logq, 0)

    if len(x.shape) == 1:
        x = np.expand_dims(x, 0)

    # Save number of classes
    k = logq.shape[1]

    # Calculate log z with trick mentioned above
    a = max(logq.flatten())
    logz = a + np.log(sum([np.power(np.e, logq[:,i] - a) for i in range(k)]))
    logp = logq - logz
    return logp

def sgd_iter(x_train, t_train, W, b):
    """
    Go over all datapoints randomly and adapt the weights and bias terms
    according to the stochastic gradient descent algorithm.
    """
    indices = np.arange(0, len(x_train))
    random.shuffle(indices)
    eta = 1e-6

    for i in indices:
        logp_train, dL_dw, dL_db = logreg_gradient(x_train[i], [t_train[i]], W, b)
        W = W - eta * -dL_dw
        b = b - eta * -dL_db

    predictions = []
    for i in indices:
        logp_train = predict(x_train[i], W, b)[:, t_train[i]].squeeze()
        predictions.append(np.asscalar(logp_train))

    return predictions, W, b

# Sanity check:
np.random.seed(1243)
w = np.zeros((28*28, 10))
b = np.zeros(10)

logp_train, W, b = sgd_iter(x_train[:5], t_train[:5], w, b)


I am working on the file "ML1/lab2.ipynb".
The first task is:
2.3.1. Explain the weights (5 points)
In less than 80 words, explain how and why the weights of the hidden layer of the MLP differ from the logistic regression model, and relate this to the stronger performance of the MLP.
OUR ANSWER
The image above is not so clear, so we ran the algorithm with eta=1e-1 and 50 iterations, for which the weights are shown below. The weights represent parts of the digits that are captured: curves and lines. The hidden layer does not map digits to their class (as in 1.2.2), but recognizes features of digits. This allows MLP to capture more sensitive variations between digits and this results in a higher classification accuracy.
<img src="https://i.imgur.com/SvJ6Ekd.png">
2.3.1. Less than 250 misclassifications on the test set (10 bonus points)
You receive an additional 10 bonus points if you manage to train a model with very high accuracy: at most 2.5% misclasified digits on the test set. Note that the test set contains 10000 digits, so you model should misclassify at most 250 digits. This should be achievable with a MLP model with one hidden layer. See results of various models at : http://yann.lecun.com/exdb/mnist/index.html. To reach such a low accuracy, you probably need to have a very high $L$ (many hidden units), probably $L > 200$, and apply a strong Gaussian prior on the weights. In this case you are allowed to use the validation set for training.
You are allowed to add additional layers, and use convolutional networks, although that is probably not required to reach 2.5% misclassifications.
OUR ANSWER
In an attempt to get the accuracy as high as possible, we have used a learning rate that adapts itself. Because it takes quite a while to train, we were unable to get to 97.5% accuracy before 23:59. However, we believe that if you set eta to 0.01 and then decrease it along the way,  you will get there within 25 iterations.
Can you write Python code for it?

predict_test = np.zeros(len(t_test))
# Fill predict_test with the predicted targets from your model, don't cheat :-).
# YOUR CODE HERE
def test_sgd_super(x_train, t_train, V, W, a, b):
    y_train = []
    y_valid = []
    k = 25
    eta = 0.1

    for i in range(k):
        _, V, W, a, b = sgd_iter(x_train, t_train, V, W, a, b)
        _, V, W, a, b = sgd_iter(x_valid, t_valid, V, W, a, b)

        for j, x in enumerate(x_test):
            all_classes = predict(calc_h(x, V, a), W, b).tolist()[0]
            predict_test[j] = all_classes.index(max(all_classes))
        p = (10000 - np.sum(predict_test != t_test)) / 10000
        print("Iteration {} done for eta {:.5f}, {:.2f}% correct.".format(i+1, eta, p * 100))
        eta = eta * 0.75
    return V, W, a, b

# init all values with gaussians
K = 10
L = 210
W = np.random.normal(loc =0, scale =1, size=(L, K))
b = np.random.uniform(size=(K,))
a = np.random.uniform(size=(L,))
V = np.random.normal(loc=0, scale=1, size=(28*28, L))
V, W, a, b = test_sgd_super(x_train, t_train, V, W, a, b)

assert predict_test.shape == t_test.shape
n_errors = np.sum(predict_test != t_test)
print('Test errors: %d' % n_errors)


I am working on the file "ML1/lab2.ipynb".
The first task is:
Describe in less than 100 words why these weights minimize the loss The plot illustrates that every column j of W is `specialized' in recognising a number. The colours represent the penalty or reward of pixel intensity in data point x. Intensity in the grey area does not matter (contributes 0 to the likelihood), in the black area should be intensity if j is x's target (positive contribution) and in the white area there should not be intensity (negative contribution). If x's target follows this pattern, the likelihood of its target is maximized and thus the loss is minimized. The fuzziness in the numbers accounts for variations in handwriting.
1.2.3. Visualize the 8 hardest and 8 easiest digits (10 points)
Visualize the 8 digits in the validation set with the highest probability of the true class label under the model.
Also plot the 8 digits that were assigned the lowest probability.
Ask yourself if these results make sense.
Can you write Python code for it?

predictions_valid = np.array([predict(x, t_valid[i], w, b) for i, x in enumerate(x_valid)])
ordered = predictions_valid.argsort()
ind_worst = ordered[:8]
ind_best = ordered[-8:]

print("Lowest probability for the true class label")
plot_digits(x_valid[ind_worst], 4, t_valid[ind_worst])

print("Highest probability for the true class label")
plot_digits(x_valid[ind_best], 4, t_valid[ind_best])


I am working on the file "ML1/lab2.ipynb".
The first task is:
Lab 2: Classification
Machine Learning 1, September 2017
Notes on implementation:

You should write your code and answers in this IPython Notebook: http://ipython.org/notebook.html. If you have problems, please contact your teaching assistant.
Please write your answers right below the questions.
Among the first lines of your notebook should be "%pylab inline". This imports all required modules, and your plots will appear inline.
Use the provided test cells to check if your answers are correct
Make sure your output and plots are correct before handing in your assignment with Kernel -> Restart & Run All

$\newcommand{\bx}{\mathbf{x}}$
$\newcommand{\bw}{\mathbf{w}}$
$\newcommand{\bt}{\mathbf{t}}$
$\newcommand{\by}{\mathbf{y}}$
$\newcommand{\bm}{\mathbf{m}}$
$\newcommand{\bb}{\mathbf{b}}$
$\newcommand{\bS}{\mathbf{S}}$
$\newcommand{\ba}{\mathbf{a}}$
$\newcommand{\bz}{\mathbf{z}}$
$\newcommand{\bv}{\mathbf{v}}$
$\newcommand{\bq}{\mathbf{q}}$
$\newcommand{\bp}{\mathbf{p}}$
$\newcommand{\bh}{\mathbf{h}}$
$\newcommand{\bI}{\mathbf{I}}$
$\newcommand{\bX}{\mathbf{X}}$
$\newcommand{\bT}{\mathbf{T}}$
$\newcommand{\bPhi}{\mathbf{\Phi}}$
$\newcommand{\bW}{\mathbf{W}}$
$\newcommand{\bV}{\mathbf{V}}$
Can you write Python code for it?

%pylab inline
plt.rcParams["figure.figsize"] = [9,5]


I am working on the file "ML1/lab2.ipynb".
The first task is:
MNIST consists of small 28 by 28 pixel images of written digits (0-9). We split the dataset into a training, validation and testing arrays. The variables x_train, x_valid and x_test are $N \times M$ matrices, where $N$ is the number of datapoints in the respective set, and $M = 28^2 = 784$ is the dimensionality of the data. The second set of variables t_train, t_valid and t_test contain the corresponding $N$-dimensional vector of integers, containing the true class labels.
Here's a visualisation of the first 8 digits of the trainingset:
Can you write Python code for it?

def plot_digits(data, num_cols, targets=None, shape=(28,28)):
    num_digits = data.shape[0]
    num_rows = int(num_digits/num_cols)
    for i in range(num_digits):
        plt.subplot(num_rows, num_cols, i+1)
        plt.imshow(data[i].reshape(shape), interpolation='none', cmap='Greys')
        if targets is not None:
            plt.title(int(targets[i]))
        plt.colorbar()
        plt.axis('off')
    plt.tight_layout()
    plt.show()

plot_digits(x_train[0:40000:5000], num_cols=4, targets=t_train[0:40000:5000])


I am working on the file "ML1/lab2.ipynb".
The first task is:
We think these images make sense:
BAD IMAGES:
    1. The very first bad digit is something that we even cannot really classify. It looks like an 'a' or a 0, but according to its target it should be a 4.
    2. The two nines are very different from the 9 'prototype' that we have seen in the weights, where the curly part at the bottom of the nine is not really visible. Therefore the log likelihood for the target (9) would be very low.
    3. The random stripe might be a 1 written diagonally? But it is very different from regular digit handwriting, thus resulting in a very low log likelihood for class 1.
    4. The curly 1 is very atypical when we compare it to the learned weights, resulting in a low log likelihood for class 1.
    5. The fifth digit is probabibly a 7, but we can see how its curliness results in a very low log likelihood for 7, given the weights for class 7.
GOOD IMAGES:
These are all zeros, probably because there is much less variation in zeros compared to the other digits. We can see this from the visualized weights. Additionally, the other digits do not realy look like zeros. Therefore, a zero would not be misclassified very quickly.
Part 2. Multilayer perceptron
You discover that the predictions by the logistic regression classifier are not good enough for your application: the model is too simple. You want to increase the accuracy of your predictions by using a better model. For this purpose, you're going to use a multilayer perceptron (MLP), a simple kind of neural network. The perceptron wil have a single hidden layer $\bh$ with $L$ elements. The parameters of the model are $\bV$ (connections between input $\bx$ and hidden layer $\bh$), $\ba$ (the biases/intercepts of $\bh$), $\bW$ (connections between $\bh$ and $\log q$) and $\bb$ (the biases/intercepts of $\log q$.
The conditional probability of the class label $j$ is given by:
$\log p(t = j \;|\; \bx, \bb, \bW) = \log q_j - \log Z$
where $q_j$ are again the unnormalized probabilities per class, and $Z = \sum_j q_j$ is again the...
Can you write Python code for it?

# Write all helper functions here
def sigmoid(x):
    return 1.0 / (1.0 + np.exp(-x))

def dsig(s):
    return np.multiply(s, (1 - s))

def calc_h(x, v, a):
    return sigmoid(np.matmul(v.T, x) + a).squeeze()

# Write training code here:
# Plot the conditional loglikelihoods for the train and validation dataset after every iteration.
# Plot the weights of the first layer.

def sgd_iter(x_train, t_train, V, W, a, b):
    """
    Go over all datapoints randomly and adapt the weights and bias terms
    according to the stochastic gradient descent algorithm.
    """
    indices = np.arange(0, len(x_train))
    random.shuffle(indices)

    for i in indices:
        h = calc_h(x_train[i], V, a)
        _, dL_dw, dL_db = logreg_gradient(h, [t_train[i]], W, b)

        dL_dh = np.matmul(W, dL_db)

        dL_da = np.multiply(dL_dh, dsig(h))

        dL_dv = np.multiply(np.matrix(x_train[i]).T, np.matrix(dL_da))

        V = V + np.multiply(eta, dL_dv)
        W = W + np.multiply(eta, dL_dw)
        a = a + np.multiply(eta, dL_da)
        b = b + np.multiply(eta, dL_db)

    predictions = []
    for i in indices:
        logp_train = predict(calc_h(x_train[i], V, a), W, b)[:, t_train[i]].squeeze()
        predictions.append(np.asscalar(logp_train))

    return predictions, V, W, a, b

def test_sgd(x_train, t_train, V, W, a, b):
    y_train = []
    y_valid = []
    k = 10
    eta = 1e-2

    for i in range(k):
        predictions_train, V, W, a, b = sgd_iter(x_train, t_train, V, W, a, b)

        # Check how well the weights generalize for the validation set
        predictions_valid = [predict(calc_h(x, V, a), W, b)[:, t_valid[i]].squeeze()
                             for i, x in enumerate(x_valid)]

        y_train.append(np.mean(predictions_train))
        y_valid.append(np.mean(predictions_valid))

    plt.plot(np.arange(1, k+1), y_train, label = "Training set")
    plt.plot(np.arange(1, k+1), y_valid, label = "Validation set")
    plt.legend()
    plt.xlabel("Iteration")
   ...


I am working on the file "ML1/lab2_original.ipynb".
The first task is:
1.1.3 Stochastic gradient descent (10 points)
Write a function sgd_iter(x_train, t_train, w, b) that performs one iteration of stochastic gradient descent (SGD), and returns the new weights. It should go through the trainingset once in randomized order, call logreg_gradient(x, t, w, b) for each datapoint to get the gradients, and update the parameters using a small learning rate of 1E-6. Note that in this case we're maximizing the likelihood function, so we should actually performing gradient ascent... For more information about SGD, see Bishop 5.2.4 or an online source (i.e. https://en.wikipedia.org/wiki/Stochastic_gradient_descent)
Can you write Python code for it?

def sgd_iter(x_train, t_train, W, b):
    # YOUR CODE HERE
    raise NotImplementedError()
    return logp_train, W, b

# Sanity check:
np.random.seed(1243)
w = np.zeros((28*28, 10))
b = np.zeros(10)

logp_train, W, b = sgd_iter(x_train[:5], t_train[:5], w, b)


I am working on the file "ML1/lab2_original.ipynb".
The first task is:
1.2. Train
1.2.1 Train (10 points)
Perform 10 SGD iterations through the trainingset. Plot (in one graph) the conditional log-probability of the trainingset and validation set after each iteration.
Can you write Python code for it?

def test_sgd(x_train, t_train, w, b):
    # YOUR CODE HERE
    raise NotImplementedError()

np.random.seed(1243)
w = np.zeros((28*28, 10))
b = np.zeros(10)
w,b = test_sgd(x_train, t_train, w, b)


I am working on the file "ML1/lab2_original.ipynb".
The first task is:
2.3.1. Explain the weights (5 points)
In less than 80 words, explain how and why the weights of the hidden layer of the MLP differ from the logistic regression model, and relate this to the stronger performance of the MLP.
YOUR ANSWER HERE
2.3.1. Less than 250 misclassifications on the test set (10 bonus points)
You receive an additional 10 bonus points if you manage to train a model with very high accuracy: at most 2.5% misclasified digits on the test set. Note that the test set contains 10000 digits, so you model should misclassify at most 250 digits. This should be achievable with a MLP model with one hidden layer. See results of various models at : http://yann.lecun.com/exdb/mnist/index.html. To reach such a low accuracy, you probably need to have a very high $L$ (many hidden units), probably $L > 200$, and apply a strong Gaussian prior on the weights. In this case you are allowed to use the validation set for training.
You are allowed to add additional layers, and use convolutional networks, although that is probably not required to reach 2.5% misclassifications.
Can you write Python code for it?

predict_test = np.zeros(len(t_test))
# Fill predict_test with the predicted targets from your model, don't cheat :-).
# YOUR CODE HERE
raise NotImplementedError()

assert predict_test.shape == t_test.shape
n_errors = np.sum(predict_test != t_test)
print('Test errors: %d' % n_errors)


I am working on the file "ML1/lab2_original.ipynb".
The first task is:
Describe in less than 100 words why these weights minimize the loss
YOUR ANSWER HERE
1.2.3. Visualize the 8 hardest and 8 easiest digits (10 points)
Visualize the 8 digits in the validation set with the highest probability of the true class label under the model.
Also plot the 8 digits that were assigned the lowest probability.
Ask yourself if these results make sense.
Can you write Python code for it?

# YOUR CODE HERE
raise NotImplementedError()


I am working on the file "ML1/lab2_original.ipynb".
The first task is:
In multiclass logistic regression, the conditional probability of class label $j$ given the image $\bx$ for some datapoint is given by:
$ \log p(t = j \;|\; \bx, \bb, \bW) = \log q_j - \log Z$
where $\log q_j = \bw_j^T \bx + b_j$ (the log of the unnormalized probability of the class $j$), and $Z = \sum_k q_k$ is the normalizing factor. $\bw_j$ is the $j$-th column of $\bW$ (a matrix of size $784 \times 10$) corresponding to the class label, $b_j$ is the $j$-th element of $\bb$.
Given an input image, the multiclass logistic regression model first computes the intermediate vector $\log \bq$ (of size $10 \times 1$), using $\log q_j = \bw_j^T \bx + b_j$, containing the unnormalized log-probabilities per class.
The unnormalized probabilities are then normalized by $Z$ such that $\sum_j p_j = \sum_j \exp(\log p_j) = 1$. This is done by $\log p_j = \log q_j - \log Z$ where $Z = \sum_i \exp(\log q_i)$. This is known as the softmax transformation, and is also used as a last layer of many classifcation neural network models, to ensure that the output of the network is a normalized distribution, regardless of the values of second-to-last layer ($\log \bq$)
Warning: when computing $\log Z$, you are likely to encounter numerical problems. Save yourself countless hours of debugging and learn the log-sum-exp trick.
The network's output $\log \bp$ of size $10 \times 1$ then contains the conditional log-probabilities $\log p(t = j \;|\; \bx, \bb, \bW)$ for each digit class $j$. In summary, the computations are done in this order:
$\bx \rightarrow \log \bq \rightarrow Z \rightarrow \log \bp$
Given some dataset with $N$ independent, identically distributed datapoints, the log-likelihood is given by:
$ \mathcal{L}(\bb, \bW) = \sum_{n=1}^N \mathcal{L}^{(n)}$
where we use $\mathcal{L}^{(n)}$ to denote the partial log-likelihood evaluated over a single datapoint. It is important to see that the log-probability of the class label $t^{(n)}$ given the image, is given by the $t^{(n)}$-th element of the network's output $\log \bp$,...
Can you write Python code for it?

# 1.1.2 Compute gradient of log p(t|x;w,b) wrt w and b
def logreg_gradient(x, t, w, b):
    # YOUR CODE HERE
    raise NotImplementedError()

    return logp[:,t].squeeze(), dL_dw, dL_db.squeeze()

np.random.seed(123)
# scalar, 10 X 768  matrix, 10 X 1 vector
w = np.random.normal(size=(28*28,10), scale=0.001)
# w = np.zeros((784,10))
b = np.zeros((10,))

# test gradients, train on 1 sample
logpt, grad_w, grad_b = logreg_gradient(x_train[0:1,:], t_train[0:1], w, b)

print("Test gradient on one point")
print("Likelihood:\t", logpt)
print("\nGrad_W_ij\t",grad_w.shape,"matrix")
print("Grad_W_ij[0,152:158]=\t", grad_w[152:158,0])
print("\nGrad_B_i shape\t",grad_b.shape,"vector")
print("Grad_B_i=\t", grad_b.T)
print("i in {0,...,9}; j in M")

assert logpt.shape == (), logpt.shape
assert grad_w.shape == (784, 10), grad_w.shape
assert grad_b.shape == (10,), grad_b.shape

# It's always good to check your gradient implementations with finite difference checking:
# Scipy provides the check_grad function, which requires flat input variables.
# So we write two helper functions that provide can compute the gradient and output with 'flat' weights:
from scipy.optimize import check_grad

np.random.seed(123)
# scalar, 10 X 768  matrix, 10 X 1 vector
w = np.random.normal(size=(28*28,10), scale=0.001)
# w = np.zeros((784,10))
b = np.zeros((10,))

def func(w):
    logpt, grad_w, grad_b = logreg_gradient(x_train[0:1,:], t_train[0:1], w.reshape(784,10), b)
    return logpt
def grad(w):
    logpt, grad_w, grad_b = logreg_gradient(x_train[0:1,:], t_train[0:1], w.reshape(784,10), b)
    return grad_w.flatten()
finite_diff_error = check_grad(func, grad, w.flatten())
print('Finite difference error grad_w:', finite_diff_error)
assert finite_diff_error < 1e-3, 'Your gradient computation for w seems off'

def func(b):
    logpt, grad_w, grad_b = logreg_gradient(x_train[0:1,:], t_train[0:1], w, b)
    return logpt
def grad(b):
    logpt, grad_w, grad_b = logreg_gradient(x_train[0:1,:], t_train[0:1], w, b)
    return...


I am working on the file "ML1/lab2_original.ipynb".
The first task is:
Lab 2: Classification
Machine Learning 1, September 2017
Notes on implementation:

You should write your code and answers in this IPython Notebook: http://ipython.org/notebook.html. If you have problems, please contact your teaching assistant.
Please write your answers right below the questions.
Among the first lines of your notebook should be "%pylab inline". This imports all required modules, and your plots will appear inline.
Use the provided test cells to check if your answers are correct
Make sure your output and plots are correct before handing in your assignment with Kernel -> Restart & Run All

$\newcommand{\bx}{\mathbf{x}}$
$\newcommand{\bw}{\mathbf{w}}$
$\newcommand{\bt}{\mathbf{t}}$
$\newcommand{\by}{\mathbf{y}}$
$\newcommand{\bm}{\mathbf{m}}$
$\newcommand{\bb}{\mathbf{b}}$
$\newcommand{\bS}{\mathbf{S}}$
$\newcommand{\ba}{\mathbf{a}}$
$\newcommand{\bz}{\mathbf{z}}$
$\newcommand{\bv}{\mathbf{v}}$
$\newcommand{\bq}{\mathbf{q}}$
$\newcommand{\bp}{\mathbf{p}}$
$\newcommand{\bh}{\mathbf{h}}$
$\newcommand{\bI}{\mathbf{I}}$
$\newcommand{\bX}{\mathbf{X}}$
$\newcommand{\bT}{\mathbf{T}}$
$\newcommand{\bPhi}{\mathbf{\Phi}}$
$\newcommand{\bW}{\mathbf{W}}$
$\newcommand{\bV}{\mathbf{V}}$
Can you write Python code for it?