I am working on the file ".ipynb_checkpoints/Lxmls_Day5-checkpoint.ipynb".
The first task is:
Go to lxmls/deep learning/mlp.py:class NumpyMLP:def grads() and complete the code of the NumpyMLP class with
the Backpropagation recursion that we just saw.
Can you write Python code for it?

    def grads(self, x, y):
        """
       Computes the gradients of the network with respect to cross entropy
       error cost
       """

        # Run forward and store activations for each layer
        activations = self.forward(x, all_outputs=True)

        # For each layer in reverse store the gradients for each parameter
        nabla_params = [None] * (2*self.n_layers)

        for n in np.arange(self.n_layers-1, -1, -1):

            # Get weigths and bias (always in even and odd positions)
            # Note that sometimes we need the weight from the next layer
            W = self.params[2*n]
            b = self.params[2*n+1]
            if n != self.n_layers-1:
                W_next = self.params[2*(n+1)]

            # Solving Exercise 5.1
            if n < self.n_layers - 1 :
                ent = np.dot(W_next.T, ent )
                ent *= activations[ n ] * (1 - activations[ n ] ) # This is correct but confusing n+1 is n in the guide
            else:  # NOTE: This assumes cross entropy cost
                if self.actvfunc[ n ] == 'sigmoid':
                    ent = ( activations[ n ] - y ) / y.shape[ 0 ]
                elif self.actvfunc[ n ] == 'softmax':
                    I = index2onehot( y , W.shape[ 0 ] )
                    ent = (activations[ n ] - I ) / y.shape[ 0 ]

            nabla_W = np.zeros( W.shape )
            for l in np.arange( ent.shape[ 1 ] ):
                if n == 0:
                    nabla_W += np.outer( ent[ :, l ], x[ :, l] )
                else:
                    nabla_W += np.outer( ent[ :, l ], activations[ n - 1] [ :, l ] )
            nabla_b = np.sum( ent , 1, keepdims=True )

            #End of the solution 5.1

            # Store the gradients
            nabla_params[2*n] = nabla_W
            nabla_params[2*n+1] = nabla_b

        return nabla_params


I am working on the file ".ipynb_checkpoints/Lxmls_Day5-checkpoint.ipynb".
The first task is:
Now obtaining a function that computes the gradient could not be easier
Can you write Python code for it?

_nabla_F = T.grad(_F, _W1)
nabla_F = theano.function([_x, _y], _nabla_F)


I am working on the file ".ipynb_checkpoints/Lxmls_Day5-checkpoint.ipynb".
The first task is:
Now we just need to complete this with the second layer, using a softmax non-linearity
Can you write Python code for it?

W2, b2 = mlp.params[2:] # Weights and bias of second (and last!) layer
# Second layer symbolic variables
_W2 = theano.shared(value=W2, name='W2', borrow=True)
_b2 = theano.shared(value=b2, name='b2', borrow=True, broadcastable=(False, True))
# Second layer symbolic expressions
_z2 = T.dot(_W2, _tilde_z1) + _b2
# NOTE: Theano softmax does not support T.nnet.softmax(_z2, axis=1) this is a workaround
_tilde_z2 = T.nnet.softmax(_z2.T).T


I am working on the file ".ipynb_checkpoints/Lxmls_Day5-checkpoint.ipynb".
The first task is:
Now we will implement this in Theano. We start by creating the variables over which we will produce the operations. For
example the symbolic input is defined as
Can you write Python code for it?

# Theano code.
# NOTE: We use undescore to denote symbolic equivalents to Numpy variables.
# This is no Python convention!.
import theano
import theano.tensor as T
_x = T.matrix('x')

_W1 = theano.shared(value=W1, name='W1', borrow=True)
_b1 = theano.shared(value=b1, name='b1', borrow=True,broadcastable=(False, True))


I am working on the file ".ipynb_checkpoints/Lxmls_Day5-checkpoint.ipynb".
The first task is:
With this, we could compile a function to obtain the output of the network symb tilde z2 for a given input symb x. In
this exercise we are however interested in obtaining the misclassification cost. This is given in Eq: 5.5. First we are going
to need the symbolic variable for the correct output
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/Lxmls_Day5-checkpoint.ipynb".
The first task is:
You can test the different hyperparameters
Can you write Python code for it?

# Model parameters
n_iter = 5
bsize = 5
lrate = 0.01
# Train
sgd.SGD_train(mlp, n_iter, bsize=bsize, lrate=lrate, train_set=(train_x, train_y))
acc_train = sgd.class_acc(mlp.forward(train_x), train_y)[0]
acc_test = sgd.class_acc(mlp.forward(test_x), test_y)[0]
print "MLP (%s) Amazon Sentiment Accuracy train: %f test: %f" % (geometry, acc_train,acc_test)


I am working on the file ".ipynb_checkpoints/Lysine Acetylation -MLP -dbptm-checkpoint.ipynb".
The first task is:
Controlling for Random Negatve vs Sans Random in Imbalanced Techniques using K acytelation.
Training data is from CUCKOO group and benchmarks are from dbptm.
Can you write Python code for it?

par = ["pass", "ADASYN", "SMOTEENN", "random_under_sample", "ncl", "near_miss"]
for i in par:
    print("y", i)
    y = Predictor()
    y.load_data(file="Data/Training/k_acetylation.csv")
    y.process_data(vector_function="sequence", amino_acid="K", imbalance_function=i, random_data=-1)
    y.supervised_training("mlp_adam")
    y.benchmark("Data/Benchmarks/acet.csv", "K")
    del y
    print("x", i)
    x = Predictor()
    x.load_data(file="Data/Training/k_acetylation.csv")
    x.process_data(vector_function="sequence", amino_acid="K", imbalance_function=i, random_data=1)
    x.supervised_training("mlp_adam")
    x.benchmark("Data/Benchmarks/acet.csv", "K")
    del x


I am working on the file ".ipynb_checkpoints/Lysine Acetylation -MLP-checkpoint.ipynb".
The first task is:
Chemical Vector
Can you write Python code for it?

par = ["pass", "ADASYN", "SMOTEENN", "random_under_sample", "ncl", "near_miss"]
for i in par:
    print("y", i)
    y = Predictor()
    y.load_data(file="Data/Training/k_acetylation.csv")
    y.process_data(vector_function="chemical", amino_acid="K", imbalance_function=i, random_data=0)
    y.supervised_training("mlp_adam")
    y.benchmark("Data/Benchmarks/acet.csv", "K")
    del y
    print("x", i)
    x = Predictor()
    x.load_data(file="Data/Training/k_acetylation.csv")
    x.process_data(vector_function="chemical", amino_acid="K", imbalance_function=i, random_data=1)
    x.supervised_training("mlp_adam")
    x.benchmark("Data/Benchmarks/acet.csv", "K")
    del x


I am working on the file ".ipynb_checkpoints/Lysine Acetylation -MLP-checkpoint.ipynb".
The first task is:
Controlling for Random Negatve vs Sans Random in Imbalanced Techniques using K acytelation.
Training data is from CUCKOO group and benchmarks are from dbptm.
Can you write Python code for it?

par = ["pass", "ADASYN", "SMOTEENN", "random_under_sample", "ncl", "near_miss"]
for i in par:
    print("y", i)
    y = Predictor()
    y.load_data(file="Data/Training/k_acetylation.csv")
    y.process_data(vector_function="sequence", amino_acid="K", imbalance_function=i, random_data=0)
    y.supervised_training("mlp_adam")
    y.benchmark("Data/Benchmarks/acet.csv", "K")
    del y
    print("x", i)
    x = Predictor()
    x.load_data(file="Data/Training/k_acetylation.csv")
    x.process_data(vector_function="sequence", amino_acid="K", imbalance_function=i, random_data=1)
    x.supervised_training("mlp_adam")
    x.benchmark("Data/Benchmarks/acet.csv", "K")
    del x


I am working on the file ".ipynb_checkpoints/Lysine Acetylation -XGB -dbptm -checkpoint.ipynb".
The first task is:
Chemical Vector
Can you write Python code for it?

par = ["pass", "ADASYN", "SMOTEENN", "random_under_sample", "ncl", "near_miss"]
for i in par:
    print("y", i)
    y = Predictor()
    y.load_data(file="Data/Training/k_acetylation.csv")
    y.process_data(vector_function="chemical", amino_acid="K", imbalance_function=i, random_data=-1)
    y.supervised_training("xgb")
    y.benchmark("Data/Benchmarks/acet.csv", "K")
    del y
    print("x", i)
    x = Predictor()
    x.load_data(file="Data/Training/k_acetylation.csv")
    x.process_data(vector_function="chemical", amino_acid="K", imbalance_function=i, random_data=1)
    x.supervised_training("xgb")
    x.benchmark("Data/Benchmarks/acet.csv", "K")
    del x


I am working on the file ".ipynb_checkpoints/Lysine Acetylation -XGB -dbptm -checkpoint.ipynb".
The first task is:
Controlling for Random Negatve vs Sans Random in Imbalanced Techniques using K acytelation.
Training data is from CUCKOO group and benchmarks are from dbptm.
Can you write Python code for it?

par = ["pass", "ADASYN", "SMOTEENN", "random_under_sample", "ncl", "near_miss"]
for i in par:
    print("y", i)
    y = Predictor()
    y.load_data(file="Data/Training/k_acetylation.csv")
    y.process_data(vector_function="sequence", amino_acid="K", imbalance_function=i, random_data=-1)
    y.supervised_training("xgb")
    y.benchmark("Data/Benchmarks/acet.csv", "K")
    del y
    print("x", i)
    x = Predictor()
    x.load_data(file="Data/Training/k_acetylation.csv")
    x.process_data(vector_function="sequence", amino_acid="K", imbalance_function=i, random_data=1)
    x.supervised_training("xgb")
    x.benchmark("Data/Benchmarks/acet.csv", "K")
    del x


I am working on the file ".ipynb_checkpoints/ME249-CFD-tutorial-1-checkpoint.ipynb".
The first task is:

Laminar Pipe Flow

In this first tutorial, you will simulate a laminar pipe flow using the cad file CAD>PipeD0.025L2.5VOF.iges. The pipe inner diameter is 2.5cm and 250cm long.

The Junior course of Fluid Mechanics has taught you that:

 The entrance length for laminar flow is given by
$$
\frac{L_e}{D}=0.05Re=0.05\frac{U_{b}D}{\nu}
$$

 The pressure gradient is constant in the fully developed region ($x > L_e$)

 The velocity profile in the fully developed region is governed by the reduced streamwise momentum equation
$$
0=-\frac{dP}{dx}+\frac{\mu}{r}\frac{d}{dr}r\frac{dU}{dr}
$$

The solution of this equation is
$$
U(r) = \frac{R^2}{4\mu}\left(-\frac{dP}{dx}\right)\left(1-\frac{r^2}{R^2}\right)=2U_b\left(1-\frac{r^2}{R^2}\right)
$$

Hence,
$$
U_b=\frac{R^2}{8\mu}\left(-\frac{dP}{dx}\right)
$$

 Finally, the wall shear stress is proportional to the pressure drop:
$$
-\frac{dP}{dx}=\frac{2\tau_w}{R}
$$

Using https://www.simscale.com, Simulate a laminar flow. Verify and Validate (V & V) your simulation.

Can you write Python code for it?

D = 2.5e-2 # m
nu = 15.e-6 #m^2/s
rho = 1.2 #kg/m^3
mu = nu/rho

R = D/2

Re = 500.

Ub = Re * nu / D
print("Bulk velocity= %2.2f m/s" %Ub)

import numpy as np

n = 30
r = np.linspace(0,R,n)

U = 2 * Ub * (1 - np.power(r,2)/R**2)
import matplotlib.pyplot as plt

plt.plot(r,U,linewidth = 2)
plt.xlabel(r"$r$ (m)", fontdict = fontlabel)
plt.ylabel(r"$U(r)$ (m/s)", fontdict = fontlabel)
plt.xlim(0,R)
plt.show()


I am working on the file ".ipynb_checkpoints/ME249-CFD-tutorial-1-checkpoint.ipynb".
The first task is:
The wall shear stress is defined:
$$
\tau_w = \mu \frac{dU}{dr}
$$
Can you write Python code for it?

tauw = - mu * (U[n-1] - U[n-2])/(r[n-1] - r[n-2])
print("wall shear stress = %1.4f Pa" %tauw)

mdpdx = 8 * mu *Ub / R**2
tauw_e = R * mdpdx / 2
print("Exact wall shear stress = %1.4f Pa" %tauw_e)


I am working on the file ".ipynb_checkpoints/ME249-Lecture-0-checkpoint.ipynb".
The first task is:

 Q2: Search numpy function linspace and describe what x_phi and x_u define. Why are the dimensions different?

Step 3: Initialization

Now we define a function to initialize our variables. In python, indentation matters! A function is defined by the command def followed by the name of the function and the argument given to the function. The variables passed as argument in the function are local, meaning they may or may not have the same names as the variables in the core code. Any other variable used within the function needs to be defined in the function or before.
Note that python accepts implicit loops. Here phi and x_phi are two vectors of dimension $N$.
Can you write Python code for it?

def init_simulation(x_phi,N):
    phi = np.zeros(N)
    phi = 1.+np.cos(x_phi-L/2.)
    xmask = np.where(np.abs(x_phi-L/2.) > np.pi)
    phi[xmask] = 0.
    return phi
phi = init_simulation(x_phi,N)
plt.plot(x_phi,phi,lw=2)
plt.xlabel('$x$', fontdict = font)
plt.ylabel('$\phi$', fontdict = font)
plt.xlim(0,L)
plt.show()


I am working on the file ".ipynb_checkpoints/ME249-Lecture-0-checkpoint.ipynb".
The first task is:

Q3: What do you observe?

Q4: Write a code to verify the divergence subroutine.

Can you write Python code for it?

def flux_divergence(f,N,dx):
    df = np.zeros(N)
    df[0:N] = (f[1:N+1]-f[0:N])/dx
    return df


I am working on the file ".ipynb_checkpoints/ME249-Lecture-0-checkpoint.ipynb".
The first task is:

Step 4: Verification

The interpolation and derivation operations are critical components of the simulation that must be verified. Since the velocity is unity, $F_{i\pm1/2}=\phi_{i\pm1/2}$.
Can you write Python code for it?

F_exact = np.zeros(N+1)
F_exact = init_simulation(x_u,N+1)
F = compute_flux(phi,u,N)
plt.plot(x_u,F_exact,lw=2,label="exact")
plt.plot(x_u,F,'r--',lw=2,label="interpolated")
plt.xlabel('$x$', fontdict = font)
plt.ylabel('$\phi$', fontdict = font)
plt.xlim(0,L)
plt.legend(loc="upper left", bbox_to_anchor=[0, 1],
           ncol=1, shadow=True, fancybox=True)
plt.show()


I am working on the file ".ipynb_checkpoints/ME249-Lecture-0-checkpoint.ipynb".
The first task is:
The choice for the interpolation is obvious:
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Lecture-0-checkpoint.ipynb".
The first task is:
The discretization of the time derivative is crude. A better discretization is the 2nd-order Runge-Kutta:

\begin{eqnarray}
\phi_i^{n+1/2}&=&\phi_i^n-\frac{\Delta t}{2}\frac{F^n_{i+\frac{1}{2}} - F^n_{i-\frac{1}{2}}}{\Delta x}\\
\phi_i^{n+1}&=&\phi_i^n-\Delta t\frac{F^{n+1/2}_{i+\frac{1}{2}} - F^{n+1/2}_{i-\frac{1}{2}}}{\Delta x}
\end{eqnarray}

Can you write Python code for it?

N=200
Simulation_time = 5.
dx = L/N
x_phi = np.linspace(dx/2.,L-dx/2.,N)
x_u = np.linspace(0.,L,N+1)
u_0 = 1.
num_scheme = 'CS'
u = u_0*np.ones(N+1)
phi = np.zeros(N)
flux = np.zeros(N+1)
divflux = np.zeros(N)
phiold = np.zeros(N)

phi = init_simulation(x_phi,N)
phi_init = phi.copy()
rk_coef = np.array([0.5,1.])

number_of_iterations = 100
dt = Simulation_time/number_of_iterations

t = 0.
for it in range (number_of_iterations):
    phiold = phi
    for irk in range(2):
        flux = compute_flux_advanced(phi,u,N,num_scheme)
        divflux = flux_divergence(flux,N,dx)
        phi = phiold-rk_coef[irk]*dt*divflux
    t += dt
plt.plot(x_phi,phi,lw=2,label='simulated')
plt.plot(x_phi,phi_init,lw=2,label='initial')
plt.legend(loc=2, bbox_to_anchor=[0, 1],
           ncol=2, shadow=True, fancybox=True)
plt.xlabel('$x$', fontdict = font)
plt.ylabel('$\phi$', fontdict = font)
plt.xlim(0,L)
plt.show()


I am working on the file ".ipynb_checkpoints/ME249-Lecture-0-checkpoint.ipynb".
The first task is:
The first two lines deal with the ability to show your graphs (generated via matplotlib) within this notebook, the remaining two lines import matplotlib's sublibrary pyplot as plt and numpy as np.

Step 2: Initialization of variables and allocations of memory

The first real coding task is to define your variables, with the exception of the time-related variables (you will understand why). Note that in our equation, we can store $\phi^n$ into one variable providing that we create a flux variable $F$.

 Q1: Explain why.

Can you write Python code for it?

L = 8*np.pi
N = 200
dx = L/N
u_0 = 1.

phi = np.zeros(N)
F = np.zeros(N+1)
u = u_0*np.ones(N+1)
x_phi = np.linspace(dx/2.,L-dx/2.,N)
x_u = np.linspace(0.,L,N+1)


I am working on the file ".ipynb_checkpoints/ME249-Lecture-1-checkpoint.ipynb".
The first task is:

Q3: What do you observe?

Q4: Write a code to verify the divergence subroutine.

Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Lecture-1-checkpoint.ipynb".
The first task is:

Step 4: Verification

The interpolation and derivation operations are critical components of the simulation that must be verified. Since the velocity is unity, $F_{i\pm1/2}=\phi_{i\pm1/2}$.
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Lecture-1-checkpoint.ipynb".
The first task is:
A slower but easier to understand version of this function is shown below. The tag slow is explained shortly after.
Can you write Python code for it?

def init_simulation_slow(u,phi,x_phi,N):
    for i in range(N):
        if (np.abs(x_phi[i]-L/2.) > np.pi):
            phi[i] = 0.
        else:
            phi[i] = 1.+np.cos(x_phi[i]-L/2.)
    return phi
phi = init_simulation_slow(u,phi,x_phi,N)
plt.plot(x_phi,phi,lw=2)
plt.xlabel('$x$', fontdict = font)
plt.ylabel('$\phi$', fontdict = font)
plt.xlim(0,L)
plt.show()


I am working on the file ".ipynb_checkpoints/ME249-Lecture-1-checkpoint.ipynb".
The first task is:
Although the plot suggests that the interpolation works, a visual proof can be deceptive. It is best to calculate the error between the exact and interpolated solution. Here we use an $l^2$-norm:
$$
\Vert F\Vert_2=\sqrt{\sum_{i=0}^{N}\left(F_i-F_i^e\right)^2}
$$
where $F_e$ is the exact solution for the flux.
Can you write Python code for it?

N = 200
phi = np.zeros(N)
F_exact = np.zeros(N+1)
F = np.zeros(N+1)
u = u_0*np.ones(N+1)
x_phi = np.linspace(dx/2.,L-dx/2.,N)
x_u = np.linspace(0.,L,N+1)
phi = init_simulation(x_phi,N)
F_exact = init_simulation(x_u,N+1)
F = compute_flux(phi,u,N)
error = np.sqrt(np.sum(np.power(F-F_exact,2)))
errorx = np.power(F-F_exact,2)
plt.plot(x_u,errorx)
plt.show()
print('error norm L 2= %1.4e' %error)

Nerror = 3
Narray = np.array([10, 100, 200])
delta = L/Narray
error = np.zeros(Nerror)
order = np.zeros(Nerror)
for ierror in range(Nerror):
    N = Narray[ierror]
    phi = np.zeros(N)
    F_exact = np.zeros(N+1)
    F = np.zeros(N+1)
    u = u_0*np.ones(N+1)
    x_phi = np.linspace(dx/2.,L-dx/2.,N)
    x_u = np.linspace(0.,L,N+1)
    phi = init_simulation(x_phi,N)
    F_exact = init_simulation(x_u,N+1)
    F = compute_flux(phi,u,N)
    error[ierror] = np.linalg.norm(F-F_exact)
    #error[ierror] = np.sqrt(np.sum(np.power(F-F_exact,2)))
    print('error norm L 2= %1.4e' %error[ierror])
order = 0.1*delta**(2)
plt.loglog(delta,error,lw=2,label='interpolate')
plt.loglog(delta,order,lw=2,label='$\propto\Delta x^2$')
plt.legend(loc="upper left", bbox_to_anchor=[0, 1],
           ncol=1, shadow=True, fancybox=True)
plt.xlabel('$\Delta x$', fontdict = font)
plt.ylabel('$\Vert F\Vert_2$', fontdict = font)
plt.show


I am working on the file ".ipynb_checkpoints/ME249-Lecture-1-checkpoint.ipynb".
The first task is:
The discretization of the time derivative is crude. A better discretization is the 2nd-order Runge-Kutta:

\begin{eqnarray}
\phi_i^{n+1/2}&=&\phi_i^n-\frac{\Delta t}{2}\frac{F^n_{i+\frac{1}{2}} - F^n_{i-\frac{1}{2}}}{\Delta x}\\
\phi_i^{n+1}&=&\phi_i^n-\Delta t\frac{F^{n+1/2}_{i+\frac{1}{2}} - F^{n+1/2}_{i-\frac{1}{2}}}{\Delta x}
\end{eqnarray}

Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Lecture-1-checkpoint.ipynb".
The first task is:
The first two lines deal with the ability to show your graphs (generated via matplotlib) within this notebook, the remaining two lines import matplotlib's sublibrary pyplot as plt and numpy as np.

Step 2: Initialization of variables and allocations of memory

The first real coding task is to define your variables, with the exception of the time-related variables (you will understand why). Note that in our equation, we can store $\phi^n$ into one variable providing that we create a flux variable $F$.

 Q1: Explain why.

Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Lecture-3-checkpoint.ipynb".
The first task is:

 Comparison with Finite Difference Derivatives

When the number of Fourier node is sufficient to capture all scales, derivatives computed in the Fourier space are essentially exact. The following code compares the exact first derivative with a first order upwind scheme:
$$
\left.\frac{\delta u}{\delta x}\right\vert_i=\frac{u_i-u_{i-1}}{\Delta x}
$$
For finite difference derivative, we will use the symbol $\delta/\delta x$ rather than $d/dx$.

Show that the scheme is first order and write the leading term in the truncation error.

Can you write Python code for it?

du_fd = np.zeros(Nx,dtype='float64')
dx = Lx/Nx
du_fd[1:Nx] = (u[1:Nx]-u[0:Nx-1])/dx
du_fd[0] = (u[0]-u[Nx-1])/dx
plt.plot(x,du_fd,'r-',lw=2,label='FD derivative')
plt.plot(x,du,'b--',lw=2,label='exact derivative')
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.ylabel('$du/dx$',fontsize = 18)
plt.xlabel('$x$',fontsize = 18)
plt.xticks(fontsize = 16)
plt.yticks(fontsize = 16)
plt.xlim(0,Lx)
plt.show()
print('error:',np.linalg.norm(du_fd-du,np.inf))


I am working on the file ".ipynb_checkpoints/ME249-Lecture-3-checkpoint.ipynb".
The first task is:

Derivation in Fourier Space

Going back to the Fourier series,
$$
u(x) = \sum_{n=-\infty}^{\infty}a_n\exp\left(\hat{\jmath}\frac{2\pi nx}{L_x}\right)=\sum_{n=-\infty}^{\infty}a_n\exp\left(\hat{\jmath}k_nx\right)
$$
with
$$
k_n=\frac{2\pi n}{L_x}\,
$$
it is obvious that any $m$ derivative of the real variable $u$ be:
$$
\frac{d^mu}{dx^m} = \sum_{n=-\infty}^{\infty}\left(\hat{\jmath}k_n\right)^ma_n\exp\left(\hat{\jmath}k_nx\right)
$$
In other words, if $u_n$ is defined as:
$$
u_n(x) = a_n\exp\left(\hat{\jmath}k_nx\right)
$$
then
$$
\frac{d^mu_n}{dx^m} = \left(\hat{\jmath}k_n\right)^m u_n\,.
$$
A $m$ derivativation in the Fourier space amounts to the multiplication of each Fourier coefficient $a_n$ by $\left(\hat{\jmath}k_n\right)^m$.
The following is a code for the first derivative of $u$:
Can you write Python code for it?

u_hat = np.fft.fft(u)
k = np.hstack((np.arange(0,Nx/2+1),np.arange(-Nx/2+1,0)))
ik = 1j*k
v_hat = ik*u_hat

v = np.real(np.fft.ifft(v_hat))
plt.plot(x,v,'r-',lw=2,label='$FT^{-1}[\hat{\jmath}kFT[u]]$')
plt.plot(x,du,'b--',lw=2,label='exact derivative')
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.ylabel('$du/dx$',fontsize = 18)
plt.xlabel('$x$',fontsize = 18)
plt.xticks(fontsize = 16)
plt.yticks(fontsize = 16)
plt.xlim(0,Lx)
plt.show()
print('error:',np.linalg.norm(v-du))

mk2 = ik*ik
v_hat = mk2*u_hat

v = np.real(np.fft.ifft(v_hat))
plt.plot(x,v,'r-',lw=2,label='$FT^{-1}[-k^2FT[u]]$')
plt.plot(x,ddu,'b--',lw=2,label='exact derivative')
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.ylabel('$d^2u/dx^2$',fontsize = 18)
plt.xlabel('$x$',fontsize = 18)
plt.xticks(fontsize = 16)
plt.yticks(fontsize = 16)
plt.xlim(0,Lx)
plt.show()
print('error:',np.linalg.norm(v-ddu))


I am working on the file ".ipynb_checkpoints/ME249-Lecture-3-checkpoint.ipynb".
The first task is:

 High-Pass Filter

From the example below, develop a function for a high-pass filter.

Can you write Python code for it?

u_hat = np.fft.fft(u)
kfilter = Nwave/2
k = np.linspace(0,Nx-1,Nx)
filter_mask = np.where((k < kfilter) | (k > Nx-kfilter) )
u_hat[filter_mask] = 0.+0.j
v = np.real(np.fft.ifft(u_hat))
plt.plot(x,v,'r-',lw=2,label='Filtered (FT)')
plt.plot(x,np.sum(uwave[:,kfilter:Nwave+1],axis=1),'b--',lw=2,label='Filtered (exact)')
plt.plot(x,u,'g:',lw=2,label='original')
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.xticks(fontsize = 16)
plt.yticks(fontsize = 16)
plt.xlim(0,Lx)
plt.show()

F = np.zeros(Nx/2+1,dtype='float64')
F_filter = np.zeros(Nx/2+1,dtype='float64')
u_hat = np.fft.fft(u)
F = np.real(u_hat[0:Nx/2+1]*np.conj(u_hat[0:Nx/2+1]))
v_hat = np.fft.fft(v)
F_filter = np.real(v_hat[0:Nx/2+1]*np.conj(v_hat[0:Nx/2+1]))
k = np.hstack((np.arange(0,Nx/2+1),np.arange(-Nx/2+1,0)))
plt.loglog(k[0:Nx/2+1],F,'r-',lw=2,label='\Phi_u')
plt.loglog(k[0:Nx/2+1],F_filter,'b-',lw=2,label='\Phi_v')
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.xlabel('$k$', fontdict = font)
plt.ylabel('$\Phi(k)$', fontdict = font)
plt.xticks(fontsize = 16)
plt.yticks(y_ticks,fontsize = 16)
plt.show()


I am working on the file ".ipynb_checkpoints/ME249-Lecture-3-checkpoint.ipynb".
The first task is:

Low-Pass Filter

The following code filters the original signal by half the wavenumbers using FFT and compares to exact filtered function
Can you write Python code for it?

# filtering the smaller waves
def low_pass_filter_fourier(a,k,kcutoff):
    N = a.shape[0]
    a_hat = np.fft.fft(u)
    filter_mask = np.where(np.abs(k) > kcut)
    a_hat[filter_mask] = 0.0 + 0.0j
    a_filter = np.real(np.fft.ifft(a_hat))
    return a_filter
kcut=Nwave/2+1
k = np.hstack((np.arange(0,Nx/2+1),np.arange(-Nx/2+1,0)))
v = low_pass_filter_fourier(u,k,kcut)
u_filter_exact = np.sum(uwave[:,0:kcut+1],axis=1)
plt.plot(x,v,'r-',lw=2,label='filtered with fft')
plt.plot(x,u_filter_exact,'b--',lw=2,label='filtered (exact)')
plt.plot(x,u,'g:',lw=2,label='original')
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.xlabel('$x$', fontdict = font)
plt.ylabel('$u$', fontdict = font)
plt.xticks(fontsize = 16)
plt.yticks(fontsize = 16)
plt.show()
print('error:',np.linalg.norm(v-u_filter_exact,np.inf))

F = np.zeros(Nx/2+1,dtype='float64')
F_filter = np.zeros(Nx/2+1,dtype='float64')
u_hat = np.fft.fft(u)
F = np.real(u_hat[0:Nx/2+1]*np.conj(u_hat[0:Nx/2+1]))
v_hat = np.fft.fft(v)
F_filter = np.real(v_hat[0:Nx/2+1]*np.conj(v_hat[0:Nx/2+1]))
k = np.hstack((np.arange(0,Nx/2+1),np.arange(-Nx/2+1,0)))
plt.loglog(k[0:Nx/2+1],F,'r-',lw=2,label='$\Phi_u$')
plt.loglog(k[0:Nx/2+1],F_filter,'b-',lw=2,label='$\Phi_v$')
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.xlabel('$k$', fontdict = font)
plt.ylabel('$\Phi_u(k)$', fontdict = font)
plt.xticks(fontsize = 16)
plt.yticks(y_ticks,fontsize = 16)
plt.show()


I am working on the file ".ipynb_checkpoints/ME249-Lecture-3-checkpoint.ipynb".
The first task is:

Which scales are the most affected by the finite difference scheme? What effect do you observe?

Modified Wavenumber

Starting from,
$$
f(x) = \sum_{k=-N/2+1}^{N/2}\hat{f}k\exp\left(\hat{\jmath}\frac{2\pi kx}{L_x}\right)
$$
and introducing the wavenumber
$$
\omega = \frac{2\pi k}{L_x}
$$
allows to reduce the Fourier expansion to
$$
f(x) = \sum{\omega=-\pi}^{\pi}\hat{f}\omega e^{\hat{\jmath}\omega x}
$$
The derivative of $f$ becomes
$$
f'(x) = \sum{\omega=-\pi}^{\pi}\hat{f'}\omega e^{\hat{\jmath}\omega x}
$$
where $\hat{f'}\omega$ is only a notation, not the derivative of the Fourier coefficient:
$$
\hat{f'}\omega=\hat{\jmath}\omega\hat{f}\omega
$$
Considering the symmetry of the FT, we restrict the study to $\omega\Delta x\in[0,\pi]$.
Now we want to express the first order finite difference scheme in terms of Fourier coefficients and wavenumbers. The scaled coordinates expression of the scheme is
$$
\frac{\delta f}{\delta x}=\frac{f(x)-f(x-\Delta x)}{\Delta x}=\sum_{\omega=-\pi}^{\pi}\hat{f}\omega\frac{e^{\hat{\jmath}\omega x}-e^{\hat{\jmath}\omega (x-\Delta x)}}{\Delta x}=\sum{\omega=-\pi}^{\pi}\frac{1-e^{-\hat{\jmath}\omega\Delta x}}{\Delta x}\hat{f}\omega e^{\hat{\jmath}\omega x}
$$
We now define a modified wavenumber for the first order upwind scheme:
$$
\hat{\jmath}\omega'=\frac{1-e^{-\hat{\jmath}\omega\Delta x}}{\Delta x}=\frac{1-\cos(-\omega\Delta x)-\hat{\jmath}\sin(-\omega\Delta x)}{\Delta x}
$$
which reduces to
$$
\hat{\jmath}\omega\text{mod}\Delta x= \hat{\jmath}\sin(\omega\Delta x)+\left(\cos(1-\omega\Delta x)\right)
$$
The modified wavenumber is no longer purely imaginary, and even the imaginary part is far from the exact wavenumber, as shown in the following plot.
Can you write Python code for it?

L = np.pi
N = 32
dx = L/N
omega = np.linspace(0,L,N)
omegap_exact = omega
omegap_modified = np.sin(omega)
plt.plot(omega,omegap_exact,'k-',lw=2,label='exact')
plt.plot(omega,omegap_modified,'r-',lw=2,label='1st order upwind')
plt.xlim(0,L)
plt.ylim(0,L)
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.xlabel('$\omega\Delta x$', fontdict = font)
plt.ylabel('$\omega_{mod}\Delta x$', fontdict = font)
plt.xticks(fontsize = 16)
plt.yticks(fontsize = 16)
plt.show()


I am working on the file ".ipynb_checkpoints/ME249-Lecture-3-checkpoint.ipynb".
The first task is:
First, let's perform a sanity check, i.e.
$$
u=FT^{-1}\left[FT\left[u\right]\right]
$$
where $FT$ designates the Fourier transform and $FT^{-1}$ its inverse.
Can you write Python code for it?

#check FT^-1(FT(u)) - Sanity check
u_hat = np.fft.fft(u)
v = np.real(np.fft.ifft(u_hat))

plt.plot(x,u,'r-',lw=2,label='$u$')
plt.plot(x,v,'b--',lw=2,label='$FT^{-1}[FT[u]]$')
plt.xlim(0,Lx)
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.xlabel('$x$', fontdict = font)
plt.ylabel('$u$', fontdict = font)
plt.xticks(fontsize = 16)
plt.yticks(fontsize = 16)
plt.show()
print('error',np.linalg.norm(u-v,np.inf))


I am working on the file ".ipynb_checkpoints/ME249-Lecture-3-checkpoint.ipynb".
The first task is:
First, let's verify that the derivation of the modified wavenumber is correct.
Can you write Python code for it?

u_hat = np.fft.fft(u)
k = np.hstack((np.arange(0,Nx/2+1),np.arange(-Nx/2+1,0)))
dx = Lx/Nx
ikm = 1j*np.sin(2.*np.pi/Nx*k)/dx+(1-np.cos(2.*np.pi/Nx*k))/dx
v_hat = ikm*u_hat
dum = np.real(np.fft.ifft(v_hat))
plt.plot(x,dum,'r-',lw=2,label='$FT^{-1}[\hat{\jmath}\omega_{mod}FT[u]]$')
plt.plot(x,du_fd,'b--',lw=2,label='$\delta u/\delta x$')
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.ylabel('$du/dx$',fontsize = 18)
plt.xlabel('$x$',fontsize = 18)
plt.xticks(fontsize = 16)
plt.yticks(fontsize = 16)
plt.xlim(0,Lx)
plt.show()
print('error:',np.linalg.norm(du_fd-dum,np.inf))


I am working on the file ".ipynb_checkpoints/ME249-Lecture-3-checkpoint.ipynb".
The first task is:
The error is large which is compounded by the fact that np.linalg.norm does not normalize the norm by the number of points. Nonetheless the error is far greater than for the derivative using the Fourier space. To shed some light on the cause for errors, the following code computes the spectrum of the first derivative. The second graph shows the difference in spectral energy between the exact solution and the derivative approximated with finite difference per wavenumber.

However the best way to visualize the problem with spectra alone is to lower the resolution at the top cell and rerun the whole notebook. Finish the notebook as is and then rerun with $N_x=64$.

Can you write Python code for it?

F = np.zeros(Nx/2+1,dtype='float64')
F_fd = np.zeros(Nx/2+1,dtype='float64')
du_hat = np.fft.fft(du)/Nx
F = np.real(du_hat[0:Nx/2+1]*np.conj(du_hat[0:Nx/2+1]))
dv_hat = np.fft.fft(du_fd)/Nx
F_fd = np.real(dv_hat[0:Nx/2+1]*np.conj(dv_hat[0:Nx/2+1]))
k = np.hstack((np.arange(0,Nx/2+1),np.arange(-Nx/2+1,0)))
plt.loglog(k[0:Nx/2+1],F,'r-',lw=2,label='$\Phi_{du/dx}$')
plt.loglog(k[0:Nx/2+1],F_fd,'b-',lw=2,label='$\Phi_{\delta u/\delta x}$')
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.xlabel('$k$', fontdict = font)
plt.ylabel('$\Phi(k)$', fontdict = font)
plt.xticks(fontsize = 16)
plt.ylim(0.1,250)
plt.yticks(fontsize = 16)
plt.show()
print('error:',np.linalg.norm(F[0:Nwave/2]-F_fd[0:Nwave/2],np.inf))
print('error per wavenumber')
plt.loglog(k[0:Nx/2+1],np.abs(F-F_fd),'r-',lw=2)
plt.xlabel('$k$', fontdict = font)
plt.ylabel('$\Vert\Phi(k)-\Phi_{\delta u/\delta x}(k)\Vert$', fontdict = font)
plt.xticks(fontsize = 16)
plt.ylim(1e-5,250)
plt.yticks(fontsize = 16)
plt.show()


I am working on the file ".ipynb_checkpoints/ME249-Lecture-4-checkpoint.ipynb".
The first task is:

Compact Finite Difference Schemes

The central compact finite difference scheme is defined as (in its general form):

$$
\beta f'_{i-2}+\alpha f'_{i-1}+f'_i+\alpha f'_{i+1}+\beta f'_{i+2}=
c\frac{f_{i+3}-f_{i-3}}{6\Delta}+b\frac{f_{i+2}-f_{i-2}}{4\Delta}+a\frac{f_{i+1}-f_{i-1}}{2\Delta}
$$

Note that if $\beta=\alpha=b=c=0$, the second order explicit central scheme is retrieved. Here we will limit our study to a tridagonal system:
$$
\alpha f'{i-1}+f'_i+\alpha f'{i+1}=a\frac{f_{i+1}-f_{i-1}}{2\Delta}
$$
and we wish to derive a fourth-order compact scheme.
The Taylor series expansion needed here are:
$$
f_{i\pm1}=f_i\pm\Delta f'i+\frac{\Delta^2}{2!}f''_i\pm\frac{\Delta^3}{3!}f'''_i+\frac{\Delta^4}{4!}f^{(4)}_i+{\cal O}(\Delta^5)
$$
$$
f'{i\pm1}=f'_i\pm\Delta f''_i+\frac{\Delta^2}{2!}f'''_i\pm\frac{\Delta^3}{3!}f^{(3)}_i+{\cal O}(\Delta^4)
$$
The goal is to find $\alpha$ and $a$ such that, the substitution of the Taylor series expansions above in the numerical scheme yields
$$
f'_i=f'_i+{\cal O}(\Delta^4)
$$
The left hand side develops into:
$$
\begin{split}
\alpha f'{i-1}+f'_i+\alpha f'{i+1}&=(1+2\alpha)f'i\
&+(-\alpha+\alpha)\Delta f''_i\
&+\alpha\Delta^2f'''_i\
&+(-\alpha+\alpha)\frac{\Delta^3}{3!}f^{(3)}_i+{\cal O}(\Delta^4)
\end{split}
$$
while the right hand side is:
$$
\begin{split}
a\frac{f{i+1}-f_{i-1}}{2\Delta}&=(-a+a)\frac{1}{2\Delta}f_i\
&+af'_i\
&+(-a+a)\frac{\Delta}{2}f''_i\
&+a\frac{\Delta^2}{3!}f'''_i\
&+(-a+a)\frac{\Delta^3}{4!}f^{(4)}_i+{\cal O}(\Delta^4)
\end{split}
$$
Fourth order is achieved if:
$$
1+2\alpha=a
$$
and
$$
\alpha = \frac{a}{3!}
$$

The fourth order compact scheme is
$$
\frac{1}{4} f'_{i-1}+f'_i+\frac{1}{4} f'_{i+1}=\frac{3}{2}\frac{f_{i+1}-f_{i-1}}{2\Delta}
$$
or
$$
 f'_{i-1}+4f'_i+ f'_{i+1}=3\frac{f_{i+1}-f_{i-1}}{\Delta}
$$

Consider a periodic function $f$ defined on $x\in[0,2\pi]$, discretized on a uniform mesh $x_i=i\Delta$ with $i=0,...,N-1$ and $\Delta=2\pi/(N)$. The system can be written in...
Can you write Python code for it?

Nitermax = 1000
it = 0
error_threshold = 1e-6
error = np.inf
phi = np.zeros(Nx)
phi_old = np.zeros(Nx)
error_jacobi = np.zeros(Nitermax)
b = np.zeros(Nx)
#generate rhs
b[1:Nx-1] = 0.75*(u[2:Nx]-u[0:Nx-2])/dx
b[0] = 0.75*(u[1]-u[Nx-1])/dx
b[Nx-1] = 0.75*(u[0]-u[Nx-2])/dx
for it in range(Nitermax):
    phi_old = np.copy(phi)
    phi[1:Nx-1] = -0.25*(phi_old[0:Nx-2] + phi_old[2:Nx]) \
                    +b[1:Nx-1]
    phi[0] = -0.25*(phi_old[1] + phi_old[Nx-1]) \
                    +b[0]
    phi[Nx-1] = -0.25*(phi_old[Nx-2] + phi_old[0]) \
                    +b[Nx-1]
    error_jacobi[it] = np.max(np.abs(phi-phi_old))
    if (error_jacobi[it] < error_threshold): break

#print(error)
it_jacobi = it
plt.semilogy(error_jacobi[0:it_jacobi+1],lw=2,label='Jacobi')
plt.xlabel('Iterations', fontdict = font)
plt.ylabel('error', fontdict = font)
plt.xticks(fontsize = 16)
plt.yticks(fontsize = 16)
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.show()
plt.plot(x,phi-du)
plt.show()
print(np.max(np.abs(phi-du)))


I am working on the file ".ipynb_checkpoints/ME249-Lecture-4-checkpoint.ipynb".
The first task is:

Gauss-Seidel Method

This method is directly derived from the Jacobi method, where it is recognized that in the process of sweeping through all indexes $i$, $\phi_{i-1}$ has already been updated. For the compact system, the Gauss-Seidel method is:
$$
\phi_i^{(n+1)}=-\frac{1}{4}\left(\phi_{i-1}^{(n+1)}+\phi_{i+1}^{(n)}\right)+\frac{1}{4}\left(\frac{3}{\Delta}\left(f_{i+1}-f_{i-1}\right)\right)
$$
The general form of this method starts with the decomposition of $A$ into its diagonal $D$, lower triangular $L$ and upper triangular $U$ matrices $A=D+L+U$, yielding $B=D-L$ and $C=-U$:
$$
(D+L)\vec{x}^{(n+1)}=\vec{b}-U\vec{x}^{(n)}
$$

Write the Gauss Seidel code and compare the convergence rate to Jacobi

Successive Over-Relaxation (SOR) and Symmetric SOR (SSOR)

Building of on the Gauss-Seidel method, SOR and SSOR introduce the concept of relaxation to the iterative process. The general form of SOR writes:
$$
\phi_i^{(n+1)}=\phi_i^{n}+\omega(\tilde{\phi}_i^{n+1}-\phi_i^{n})
$$
where $\omega$ is the relaxation paramterer, and \tilde{\phi}_i^{n+1} is the prediction of $\phi$ using the Gauss-Seidel method, which translates into:
$$
(D+\omega L)\vec{x}^{(n+1)}=\omega\vec{b}+[(1-\omega)D-\omega U)\vec{x}^{(n)}
$$
In the Gauss-Seidel method, only the lower triangle of the matrix is used, creating an asymmetry in the search for the solution. The SSOR method removes this asymmetry by performing the standard Gauss Seidel operation immediately followed by the upper-triangle Gauss-Seidel, for each iteration.
Finding $\omega$ requires some algebra. When $\omega<1$, the method is under-relaxed and slow to converge. For $\omega=1$, the method reverts to Gauss-Seidel. The over-relaxation $\omega>1$ method is typically faster to converge for diffusion problem ($\nabla^2 T=f$), but not necessarily for the compact scheme here. For diffusion problem, such as the following example (from Fundamentals of Engineering Numerical Analysis, Author: Parviz Moin, Publisher:...
Can you write Python code for it?

!ipython nbconvert --to html ME249-Lecture-4-YOURNAME.ipynb


I am working on the file ".ipynb_checkpoints/ME249-Lecture-4-checkpoint.ipynb".
The first task is:
This is the Matrix approach:
Can you write Python code for it?

Nitermax = 1000
it = 0
error_threshold = 1e-6
phi = np.zeros(Nx)
phi_old = np.zeros(Nx)
error_jacobi = np.inf*np.ones(Nitermax)
b = np.zeros(Nx)
A = np.zeros((Nx,Nx))
for i in range(Nx):
    if (i == 0):
        A[i,i] = 4.
        A[i,i+1] = 1.
        A[i,Nx-1] = 1.
        b[0] = 3./dx*(u[1] - u[Nx-1])
    elif (i == Nx-1):
        A[i,i-1] = 1.
        A[i,i] = 4.
        A[i,0] = 0.
        b[i] = 3./dx*(u[0] - u[Nx-2])
    else:
        A[i,i-1] = 1.
        A[i,i] = 4.
        A[i,i+1] = 1.
        b[i] = 3./dx*(u[i+1] - u[i-1])

D = np.diag(A)
B = np.diagflat(D)
C = A - B

for it in range (Nitermax):
    phi_old = np.copy(phi)
    phi = (b-np.dot(C,phi_old))/D
    error_jacobi[it] = np.max(np.abs(phi-phi_old))
    if (error_jacobi[it] < error_threshold): break

#print(error)
it_jacobi = it
plt.semilogy(error_jacobi[0:it_jacobi+1],lw=2,label='Jacobi')
plt.xlabel('Iterations', fontdict = font)
plt.ylabel('error', fontdict = font)
plt.xticks(fontsize = 16)
plt.yticks(fontsize = 16)
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=3, shadow=True, fancybox=True)
plt.show()
print(np.max(np.abs(phi-du)))


I am working on the file ".ipynb_checkpoints/ME249-Pb1-steady-laminar-channel-flow-Copy1-checkpoint.ipynb".
The first task is:

Steady Laminar Channel Flow

The objective of this first assignment is to compute the well-known Poiseuille velocity profile in a channel flow. Assuming that the channel length and width are both very large, the governing equations for an incompressible flow,
$$
\partial_xu+\partial_yv+\partial_z w = 0
$$
$$
\rho\left(\partial_t u+u\partial_x u+v\partial_y u+w\partial_z u\right) = -\partial_x p+\mu\left(\partial^2_x u +\partial^2_y u+\partial^2_z u\right)
$$
$$
\rho\left(\partial_t v+u\partial_x v+v\partial_y v+w\partial_z v\right) = -\partial_y p+\mu\left(\partial^2_x v +\partial^2_y v+\partial^2_z v\right)
$$
$$
\rho\left(\partial_t w+u\partial_x w+v\partial_y w+w\partial_z w\right) = -\partial_z p+\mu\left(\partial^2_x w +\partial^2_y w+\partial^2_z w\right)
$$
reduce to

$$
0=-\frac{d P}{dx}+\mu\frac{d^2u}{dx^2}
$$

Demonstrate the above statement.

 Solution

Here

 Taylor Series Expansion

Given an $C^\infty$ function $f$ defined over an interval $]a,b[$, for any $x\in]a,b[$ and any $h$ such that $x+h\in]a,b[$, the Taylor series expansion writes

$$
f(x+h)=f(x)+\sum_{n=1}^\infty\frac{h^n}{n!}\frac{d^n f}{dx^n}
$$

Practically, for a small increment $h$, $f(x+h)$ can be estimated as

$$
f(x+h)=f(x)+\sum_{n=1}^{N-1}\frac{h^n}{n!}\frac{d^n f}{dx^n}+{\cal O}(h^N)
$$

where

$$
{\cal O}(h^N)=\sum_{n=N}^{\infty}\frac{h^n}{n!}\frac{d^n f}{dx^n}
$$
is the truncation error which contains all terms affected by $h^n$ with $n\geq N$. Since $h$ is small, the large $n$, the smaller the contribution of $(h^n/n!)f^{(n)}(x)$. The above estimation of $f(x+h)$ is said to be of order $N$

Now, let's consider our channel, discretized as shown in the sketch below. This discretization technique is called  finite difference method, which is used to solve PDEs and ODEs. Here our ODE is
<p...
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Pb1-steady-laminar-channel-flow-Copy1-checkpoint.ipynb".
The first task is:

Error Quantification

Visual verification is subjective. You should always rely on a statistical approach, such the $L^2$ or $L^\infty$.

$$
L^2 = \left[\sum_{i=0}^{N-1}U^2_j\right]^{1/2}
$$
$$
L^\infty = \max_{i=0,N-1}\left\vert U_j\right\vert
$$

Can you write Python code for it?

L2 = np.linalg.norm(U_all-U_exact,2)
print("L2 error= %1.4e" %L2)
Linf = np.linalg.norm(U_all-U_exact,np.inf)
print("Linfinity error= %1.4e" %Linf)


I am working on the file ".ipynb_checkpoints/ME249-Pb1-steady-laminar-channel-flow-Copy1-checkpoint.ipynb".
The first task is:

Numerical solution

Here we will solve the flow inside a channel of half-height $h=1$, a pressure gradient $G=1$ and a viscosity $\mu=1/10$. Both walls are fixed.

Write the code to solve our ODE.

Can you write Python code for it?

N = 256

mu = 1/10.
h = 1.
G = 1.
U_lower_wall = 0.
U_upper_wall = 0.

U = np.zeros(N)
U_all = np.zeros(N+2)
y_all = np.linspace(-h, +h, N+2)
dy = y_all[1] - y_all[0]
y = np.linspace(-h+dy, +h-dy, N)

#Matrix coefficients
a = mu/dy**2*np.ones(N)
b = -2.*mu/dy**2*np.ones(N)
c = mu/dy**2*np.ones(N)

#Boundary conditions

#Right hand side vector
r = -G*np.ones(N)
r[0] -= a[0]*U_lower_wall
r[N-1] -= c[N-1]*U_upper_wall

U = TDMAsolver(a, b, c, r)
U_all[1:N+1] = U
U_all[0] = U_lower_wall
U_all[N+1] = U_upper_wall
plt.plot(y_all,U_all,lw=2)
U_exact = G/(2*mu)*(h**2-np.power(y_all,2))
plt.plot(y_all,U_exact,'r--',lw=2)


I am working on the file ".ipynb_checkpoints/ME249-Pb1-steady-laminar-channel-flow-checkpoint.ipynb".
The first task is:

Error Quantification

Visual verification is subjective. You should always rely on a statistical approach, such the $L^2$ or $L^\infty$.

$$
L^2 = \left[\sum_{i=0}^{N-1}(U^{numerical}_j-U_j^{exact})^2\right]^{1/2}
$$
$$
L^\infty = \max_{i=0,N-1}\left\vert U^{numerical}_j-U_j^{exact}\right\vert
$$

Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Pb1-steady-laminar-channel-flow-checkpoint.ipynb".
The first task is:

Numerical solution

Here we will solve the flow inside a channel of half-height $h=1$, a pressure gradient $G=1$ and a viscosity $\mu=1/10$. Both walls are fixed.

Write the code to solve our ODE.

Solution

Can you write Python code for it?

#number of computational nodes between walls
N = 128

#Flow and geometry parameters
mu = 1/10. #dynamic viscosity
h = 1. #half height of the channel
G = 1. #=-dP/dx pressure gradient
U_lower_wall = 0. #Velocity of the lower wall
U_upper_wall = 0. #Velocity of the upper wall

#Allocation of memory
U = np.zeros(N)                  #Velocity. On the sketch: from node j=0 to N-1
U_all = np.zeros(N+2)            #Velocity. On the sketch: from node j=-1 to N
y_all = np.linspace(-h, +h, N+2) #location of computational nodes from the lower to the upper wall
dy = y_all[1] - y_all[0]         #space between computational nodes
y = np.linspace(-h+dy, +h-dy, N) #location of computational nodes for vector U

#Matrix coefficients
a = mu/dy**2*np.ones(N)
b = -2.*mu/dy**2*np.ones(N)
c = mu/dy**2*np.ones(N)
#Right hand side vector
r = -G*np.ones(N)

#Boundary conditions
r[0] -= a[0]*U_lower_wall
r[N-1] -= c[N-1]*U_upper_wall

#solution using Thomas Algorithm
U = TDMAsolver(a, b, c, r)

#Copy array U onto U_all for plotting
U_all[1:N+1] = U
U_all[0] = U_lower_wall
U_all[N+1] = U_upper_wall

#Plot
plt.plot(y_all,U_all,lw=2,label = r"numerical solution")
U_exact = G/(2*mu)*(h**2-np.power(y_all,2))
plt.plot(y_all,U_exact,'r--',lw=2,label = r"exact solution")
plt.xlabel(r"$y/h$", fontdict = fontlabel)
plt.ylabel(r"$U(y)$", fontdict = fontlabel)
plt.legend(loc=3, bbox_to_anchor=[0, 1],
           ncol=2, shadow=True, fancybox=True)
plt.show()


I am working on the file ".ipynb_checkpoints/ME249-Pb1-steady-laminar-channel-flow-checkpoint.ipynb".
The first task is:

Demonstrate that you can estimate the derivative of $U$ at node $j+1/2$, using $U_j$ and $U_{j+1}$

To answer this question, you will use the following Taylor series expansion:
$$
U(y\pm\Delta y/2)=U(y)+\sum_{n=1}^\infty\frac{(\pm\Delta y/2)^n}{n!}\frac{d^nU}{dy^n}
$$

 Solution

\begin{eqnarray}
U_{j+1} &=& U_{j+1/2}+\frac{\Delta y}{2}\left.\frac{dU}{dy}\right\vert_{j+1/2}+\frac{(\Delta y/2)^2}{2!}\left.\frac{d^2U}{dy^2}\right\vert_{j+1/2}+\frac{(\Delta y/2)^3}{3!}\left.\frac{d^3U}{dy^3}\right\vert_{j+1/2}+{\cal O}(\Delta y^4)\
-U(j) &=& U{j+1/2}+\frac{-\Delta y}{2}\left.\frac{dU}{dy}\right\vert_{j+1/2}+\frac{(-\Delta y/2)^2}{2!}\left.\frac{d^2U}{dy^2}\right\vert_{j+1/2}+\frac{(-\Delta y/2)^3}{3!}\left.\frac{d^3U}{dy^3}\right\vert_{j+1/2}+{\cal O}(\Delta y^4)\
=U_{j+1}-U_j&=&0+\Delta y\left.\frac{dU}{dy}\right\vert_{j+1/2}+\frac{\Delta y^3}{24}\left.\frac{d^3U}{dy^3}\right\vert_{j+1/2}+{\cal O}(\Delta y^4)
\end{eqnarray}
Therefore

$$
\left.\frac{dU}{dy}\right\vert_{j+1/2}=\frac{U_{j+1}-U_j}{\Delta y}+{\cal O}(\Delta y^2)
$$

Using the derivative above, show that a numerical scheme for the second derivative of $U$ is
$$
\frac{d^2U}{dy^2}= \frac{U_{j-1}-2U_j+U_{j+1}}{\Delta y^2}+{\cal O}(\Delta y^2)
$$

The term ${\cal O}(\Delta y^2)$ represents the truncation error, i.e. all the terms from the Taylor series expansion that are affected with $\Delta y^n$ for $n\geq 2$.

 Solution

Using Taylor expansion series:
\begin{eqnarray}
\frac{1}{\Delta y^2}U_{j+1} &=&\frac{1}{\Delta y^2}\left[U_j+\Delta y\left.\frac{dU}{dy}\right\vert_j+\frac{\Delta y^2}{2!}\left.\frac{d^2U}{dy^2}\right\vert_j+\frac{\Delta y^3}{3!}\left.\frac{d^3U}{dy^3}\right\vert_j+\frac{\Delta y^4}{4!}\left.\frac{d^4U}{dy^4}\right\vert_j+{\cal O}(\Delta y^5)\right]\
-\frac{2}{\Delta y^2}U_{j} &=&\frac{-2}{\Delta y^2}\left[U_j\right]\
+\frac{1}{\Delta y^2}U_{j-1}...
Can you write Python code for it?

import numpy as np
def TDMAsolver(a, b, c, d):
    '''
    TDMA solver, a b c d can be NumPy array type or Python list type.
    refer to http://en.wikipedia.org/wiki/Tridiagonal_matrix_algorithm
    '''
    nf = len(a)     # number of equations
    ac, bc, cc, dc = map(np.array, (a, b, c, d))     # copy the array
    for it in range(1, nf):
        mc = ac[it]/bc[it-1]
        bc[it] = bc[it] - mc*cc[it-1]
        dc[it] = dc[it] - mc*dc[it-1]

    xc = ac
    xc[-1] = dc[-1]/bc[-1]

    for il in range(nf-2, -1, -1):
        xc[il] = (dc[il]-cc[il]*xc[il+1])/bc[il]

    del bc, cc, dc  # delete variables from memory

    return xc


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy1-checkpoint.ipynb".
The first task is:

1.4 Booleans

Logical operations result in True or False outcomes. They mostly drive if and while statements.
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy1-checkpoint.ipynb".
The first task is:

1.5 Loops

A key element of scientific computing is the ability to loop or iterate. You have two options: for and while:

for i in range(lower, upper, step): "range(lower, upper, step)" returns an iterable list of numbers from the lower number to the upper number (-1, if step>0, +1 if step<0), while incrementing by step. If step is not indicated, the default value is 1.

while condition: This loop will run while condition is true.

Here are some examples:
Can you write Python code for it?

for i in range(0,5,1):
    print(i)

for i in range(5):
    print(i)

for i in range(3,5):
    print(i)

for i in range(5,2,-1):
    print(i)

a = 0
while a <=5 :
    print(a)
    a += 1
print("final a=",a)


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy1-checkpoint.ipynb".
The first task is:

2.3 The problem with arrays: =

Any programming language has a quirk or two, that may baffle/annoy you. May be it is only me, but I find the equal sign in python dangerous. Here is why. First we define an array
$$
a_i = 2^i,\;i=0,\ldots,9
$$
Can you write Python code for it?

n = 10
a = np.zeros(n, dtype = np.float)
for i in range(n):
    a[i] = 2.**i
print("a = ",a)


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy1-checkpoint.ipynb".
The first task is:

2.4 Array operations

To perform operations an all elements, numpy has many functions. In the following, we

Add 1 to all elements

Multiply two matrices, element by elements

Perform the dot product between two vectors

Perform a mathematical operation

Can you write Python code for it?

n = 4
a = np.zeros(n, dtype = np.float64)
for i in range(n):
    a[i] = 2.**i
print("a=",a)
b = np.copy(a)
b += 1.
print("b=a+1=",b)
b = np.zeros(n, dtype = np.float64)
for i in range(n):
    b[i] = np.float64(i)
print("a=",a)
print("b=",b)
print("a*b=",a*b)
print("a.b=",np.dot(a,b))
print("exp(-a)=",np.exp(-a))


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy1-checkpoint.ipynb".
The first task is:

Getting errors is part of code development. Python will provide sometimes short, sometimes way too long error messages. Before you throw your hands in the air, give up and immediately email the TA or the instructor, browse the error message for things that could make sense. Typically, the last line contains useful information. Here it says that math.exp is expecting a scalar but we gave a length-1 array.

2.5 Indexing and Slicing

A very attractive feature of python is the nomenclature for indexing and slicing that enables to maximize the efficiency of vector operations. Here is a summary of the indexing and slicing nomenclature for a vector:
Can you write Python code for it?

%matplotlib inline
# plots graphs within the notebook
%config InlineBackend.figure_format='svg' # not sure what this does, may be default images to svg format

from IPython.display import Image
from IPython.display import clear_output

from IPython.core.display import HTML
def header(text):
    raw_html = '
' + str(text) + '
'
    return raw_html

def box(text):
    raw_html = '
'+str(text)+'
'
    return HTML(raw_html)

def nobox(text):
    raw_html = '
'+str(text)+'
'
    return HTML(raw_html)

def addContent(raw_html):
    global htmlContent
    htmlContent += raw_html

class PDF(object):
  def __init__(self, pdf, size=(200,200)):
    self.pdf = pdf
    self.size = size

  def _repr_html_(self):
    return ''.format(self.pdf, self.size)

  def _repr_latex_(self):
    return r'\includegraphics[width=1.0\textwidth]{{{0}}}'.format(self.pdf)

PDF('figures/indexing.pdf',size=(800,600))

n = 6
a = np.zeros(6,dtype=np.float)

for i in range(n):
    a[i] = 2**i
print("a[0:n] = ",a[0:n])
print("a[:] = ",a[:])
print("a[1:]  = ",a[1:])
print("a[1:n] = ",a[1:n])

print("a[0:n]   = ",a[0:n])
print("a[0:n-1] = ",a[0:n-1])
print("a[0:-1]  = ",a[0:-1])
print("a[:-1]   = ",a[:-1])

print("a[2:] = ",a[2:])
print("a[:-3]= ",a[:-3])

b = np.zeros(n,dtype=np.float)
b[:-1] = a[1:] - a[:-1]
print("a[1:]  =",a[1:])
print("-")
print("a[:-1] =",a[:-1])
print("       =",b[:-1])

n = 100000
mu = 0.
sigma = 1.
a = np.random.normal(loc = mu, scale = sigma, size = n)

print("a[:-(n-10)] = ", a[:-(n-10)])
print("a[0:10] = ", a[0:10])

import matplotlib.pyplot as plt
count, bins, ignored = plt.hist(a, 50, normed=True)
plt.plot(bins, 1/(sigma * np.sqrt(2 * np.pi)) * \
         np.exp( - (bins - mu)**2 / (2 * sigma**2) ),\
         linewidth=2, color='r')
plt.show()

n = 100
mu = 0.
sigma = 0.2
x = np.linspace(0,1.,n)
a = np.sin(2.*math.pi*x)
plt.plot(x,a, lw = 2)
plt.show()
a_noisy = np.copy(a)
n_noise = 4
k_noise =...


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy1-checkpoint.ipynb".
The first task is:
Both my arrays have been changed!!! The same is not true for simple variables:
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy1-checkpoint.ipynb".
The first task is:
Let's take a pause here to discuss if, while, for statements. The indentation at the line following if a>0: is critical. It means that all the lines with such indentation following if a>0: are part of the actions needed if a>0 = True. Try the following cells using  a = 1  and if a=4. Note also the role of the boolean operators and and or. Note also that each if, while, for statements need to end with : to signal that the following indented lines are part of the statement.
Can you write Python code for it?

a = 4
b = 0
if a > 0 and a < 4:
    a += 1
    b = 3
else:
    a = 0
    b = 1
print(a,b)

a = 4
b = 0
if a > 0 or a < 4:
    a += 1
    b = 3
else:
    a = 0
    b = 1
print(a,b)


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy1-checkpoint.ipynb".
The first task is:
Note that you need to use the exp of numpy and not from math, otherwise you get this:
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy1-checkpoint.ipynb".
The first task is:
Now, I want to modify the last element a[n-1] but in a different array to keep my initial array intact (something quite common in CFD).
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy1-checkpoint.ipynb".
The first task is:
which is equivalent to:
Can you write Python code for it?

n = 10
a = np.zeros(n)
for i in range(n):
    a[i] = 2.**i
b = np.copy(a)
b *= 2.
print("a = ",a)
print("b = ",b)


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:

1.2 Basic operators +,-,\*,/,//,**

A few things you should know about python 3:

2 is an integer, 2.0 is a floating point number, (2.0+3.0j) is a complex.

The division operator returns a floating point number. To perform the division of two integers and expect an integer number as the result, use // or int()

The power operation $x^y$ is x**y or pow(x,y).

a += 1 is a shorthand for a = a + 1, same for subtraction, multiplication and division.

Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:

2 Matrices

Matrices are central to CFD as we will see in future problems. The library or module numpy contains all the definitions and operators necessary for matrix operations.

2.1 Initialization of a matrix

You may set a matrix/vector to zero, unity or the values you may choose using numpy functions zeros, ones, and array. Please note:

 Indices always start at 0. More on that later.

 For one dimension array (vector) the syntax to initialize an array is np.zeros(n), however for multiple-dimension arrays, the argument to be passed to zeros or ones is a tuple (p,q,r) like this: np.zeros((p,q,r))

 dtype allows for the specification of the variable format. There are many ways to pass the format:  'int', 'float32', 'float64', 'float', 'complex64', 'complex128', 'complex'. By default 'float' and 'complex' are 64 bits and 128 bits, respectively, or the same as 'float64' and 'complex128'. Should you be short on memory you may revert to <FONT...
Can you write Python code for it?

import numpy as np

n = 10
a = np.zeros(n)
print("a = ",a)
m = 3
b = np.ones((m,n))
print("b = ",b)
c = np.array([1, 2, 3, 4])
print("c = ",c)
d = np.array([[1, 2], [3, 4]], dtype = np.complex)
print("d = ",d)

a = np.ones((2,2), dtype = 'float64')
for j in range(2):
    for i in range(2):
        a[i,j] *= np.float64(math.pi/(i+j+1))
print(a)
print("a[0,0] =",a[0,0])


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:

1.3 Mathematical functions

You may find all mathematical functions here : https://docs.python.org/3.5/library/math.html.
Can you write Python code for it?

print("e^1=",math.exp(1.))
print("e^(-infinity)=",math.exp(-math.inf))
print("log(e)=", math.log(math.e))
print("log_10(1000.)=",math.log10(1000.))
a = math.cos(math.pi/4.)
print("a = cos(pi/4) = %1.8f" %a)
print("arcos(a)*4./pi = ",math.acos(a)*4./math.pi)


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:

1.4 Booleans

Logical operations result in True or False outcomes. They mostly drive if and while statements.
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:

1.5 Loops

A key element of scientific computing is the ability to loop or iterate. You have two options: for and while:

for i in range(lower, upper, step): "range(lower, upper, step)" returns an iterable list of numbers from the lower number to the upper number (-1, if step>0, +1 if step<0), while incrementing by step. If step is not indicated, the default value is 1.

while condition: This loop will run while condition is true.

Here are some examples:
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:

2.4 Array operations

To perform operations an all elements, numpy has many functions. In the following, we

Add 1 to all elements

Multiply two matrices, element by elements

Perform the dot product between two vectors

Perform a mathematical operation

Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:

Getting errors is part of code development. Python will provide sometimes short, sometimes way too long error messages. Before you throw your hands in the air, give up and immediately email the TA or the instructor, browse the error message for things that could make sense. Typically, the last line contains useful information. Here it says that math.exp is expecting a scalar but we gave a length-1 array.

2.5 Indexing and Slicing

A very attractive feature of python is the nomenclature for indexing and slicing that enables to maximize the efficiency of vector operations. Here is a summary of the indexing and slicing nomenclature for a vector:
Can you write Python code for it?

%matplotlib inline
# plots graphs within the notebook
%config InlineBackend.figure_format='svg' # not sure what this does, may be default images to svg format

from IPython.display import Image
from IPython.display import clear_output

from IPython.core.display import HTML
def header(text):
    raw_html = '
' + str(text) + '
'
    return raw_html

def box(text):
    raw_html = '
'+str(text)+'
'
    return HTML(raw_html)

def nobox(text):
    raw_html = '
'+str(text)+'
'
    return HTML(raw_html)

def addContent(raw_html):
    global htmlContent
    htmlContent += raw_html

class PDF(object):
  def __init__(self, pdf, size=(200,200)):
    self.pdf = pdf
    self.size = size

  def _repr_html_(self):
    return ''.format(self.pdf, self.size)

  def _repr_latex_(self):
    return r'\includegraphics[width=1.0\textwidth]{{{0}}}'.format(self.pdf)

PDF('figures/indexing.pdf',size=(800,600))

n = 6
a = np.zeros(6,dtype=np.float)

for i in range(n):
    a[i] = 2**i
print("a[0:n] = ",a[0:n])
print("a[:] = ",a[:])
print("a[1:]  = ",a[1:])
print("a[1:n] = ",a[1:n])

print("a[0:n]   = ",a[0:n])
print("a[0:n-1] = ",a[0:n-1])
print("a[0:-1]  = ",a[0:-1])
print("a[:-1]   = ",a[:-1])

print("a[2:] = ",a[2:])
print("a[:-3]= ",a[:-3])

b = np.zeros(n,dtype=np.float)
b[:-1] = a[1:] - a[:-1]
print("a[1:]  =",a[1:])
print("-")
print("a[:-1] =",a[:-1])
print("       =",b[:-1])


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:
Both my arrays have been changed!!! The same is not true for simple variables:
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:
Let's take a pause here to discuss if, while, for statements. The indentation at the line following if a>0: is critical. It means that all the lines with such indentation following if a>0: are part of the actions needed if a>0 = True. Try the following cells using  a = 1  and if a=4. Note also the role of the boolean operators and and or. Note also that each if, while, for statements need to end with : to signal that the following indented lines are part of the statement.
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:
Note that when you perform an operation on <b>all</b> the elements of your array, the problem of linking your old array and new array disappears:
Can you write Python code for it?

n = 10
a = np.zeros(n)
for i in range(n):
    a[i] = 2.**i
b = a * 2.
print("a = ",a)
print("b = ",b)


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:
Note that you need to use the exp of numpy and not from math, otherwise you get this:
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:
The fix for arrays is to use np.copy:
Can you write Python code for it?

n = 10
a = np.zeros(n)
for i in range(n):
    a[i] = 2.**i
b = np.copy(a)
b[n-1] -= 500.
print("a = ",a)
print("b = ",b)


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:
Using indexing and slicing is much more computationally efficient than using a for loop, here is a proof. The next cell creates a large array of random numbers using a normal distribution.
Can you write Python code for it?

n = 100000
mu = 0.
sigma = 1.
a = np.random.normal(loc = mu, scale = sigma, size = n)

print("a[:-(n-10)] = ", a[:-(n-10)])
print("a[0:10] = ", a[0:10])

%matplotlib inline
# plots graphs within the notebook
%config InlineBackend.figure_format='svg' # not sure what this does, may be default images to svg format

from IPython.display import display,Image, Latex
from __future__ import division
from sympy.interactive import printing
printing.init_printing(use_latex='mathjax')

import time

from IPython.display import display,Image, Latex

from IPython.display import clear_output

#import SchemDraw as schem
#import SchemDraw.elements as e

import matplotlib.pyplot as plt
import numpy as np
import math
import scipy.constants as sc

import sympy as sym

from IPython.core.display import HTML
def header(text):
    raw_html = '
' + str(text) + '
'
    return raw_html

def box(text):
    raw_html = '
'+str(text)+'
'
    return HTML(raw_html)

def nobox(text):
    raw_html = '
'+str(text)+'
'
    return HTML(raw_html)

def addContent(raw_html):
    global htmlContent
    htmlContent += raw_html

class PDF(object):
  def __init__(self, pdf, size=(200,200)):
    self.pdf = pdf
    self.size = size

  def _repr_html_(self):
    return ''.format(self.pdf, self.size)

  def _repr_latex_(self):
    return r'\includegraphics[width=1.0\textwidth]{{{0}}}'.format(self.pdf)

class ListTable(list):
    """ Overridden list class which takes a 2-dimensional list of
        the form [[1,2,3],[4,5,6]], and renders an HTML Table in
        IPython Notebook. """

    def _repr_html_(self):
        html = ["
"]
        for row in self:
            html.append("
")

            for col in row:
                html.append("{0}	".format(col))

            html.append("
")
        html.append("
")
        return ''.join(html)

font = {'family' : 'serif',
      ...


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-Copy2-checkpoint.ipynb".
The first task is:
but this is bad:
Can you write Python code for it?

n = 10
a = np.zeros(n)
for i in range(n):
    a[i] = 2.**i
b = np.zeros(n)
b = a
b *= 2.
print("a = ",a)
print("b = ",b)


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-checkpoint.ipynb".
The first task is:

1.2 Basic operators +,-,\*,/,//,**

A few things you should know about python 3:

2 is an integer, 2.0 is a floating point number, (2.0+3.0j) is a complex.

The division operator returns a floating point number. To perform the division of two integers and expect an integer number as the result, use // or int()

The power operation $x^y$ is x**y or pow(x,y).

a += 1 is a shorthand for a = a + 1, same for subtraction, multiplication and division.

Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-checkpoint.ipynb".
The first task is:

1.3 Mathematical functions

You may find all mathematical functions here : https://docs.python.org/3.5/library/math.html.
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-checkpoint.ipynb".
The first task is:

1.4 Booleans

Logical operations result in True or False outcomes. They mostly drive if and while statements.
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-checkpoint.ipynb".
The first task is:

2.3 The problem with arrays: =

Any programming language has a quirk or two, that may baffle/annoy you. May be it is only me, but I find the equal sign in python dangerous. Here is why. First we define an array
$$
a_i = 2^i,\;i=0,\ldots,9
$$
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-checkpoint.ipynb".
The first task is:
The fix for arrays is to use np.copy:
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/ME249-Python-crash-course-checkpoint.ipynb".
The first task is:
Using indexing and slicing is much more computationally efficient than using a for loop, here is a proof. The next cell creates a large array of random numbers using a normal distribution.
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/MNIST-dimensionality-reduction-checkpoint.ipynb".
The first task is:
MNIST Dataset
We choose the popular MNIST (Mixed National Institute of Standards and Technology) computer vision digit dataset. This contains a series for images of handwriting letters, each of them of 28x28 pixels, see a few pick below.
The datasets are large, please download them from:
https://pjreddie.com/projects/mnist-in-csv/
Can you write Python code for it?

train = pd.read_csv('./datasets/mnist_train.csv').head(2000)
#reduce the size to 3000 to make things fast

columns=[]
columns.append("label")
for ii in range(784):
    columns.append("pixel"+str(ii+1))
train.columns=columns

print("Shape of train dataset: "+str(train.shape))
train.head()


I am working on the file ".ipynb_checkpoints/MNIST-dimensionality-reduction-checkpoint.ipynb".
The first task is:
Now predict in the test set.
Note that the LDA implementation in sklearn contains a predict feature that we test below, if not LDA then we skip this.
Remember the precision is the ratio tp / (tp + fp) where tp is the number of true positives and fp the number of false positives. The precision is intuitively the ability of the classifier not to label as positive a sample that is negative.
Can you write Python code for it?

#Total of true positives:
from sklearn.metrics import precision_score

#Only LDA has the prediction method:
if ( dimensionality_reduction_method == "lda"):
    y_pred = reduction_method.predict(X_test.values)

#Precision score for test dataset:
    print("Precision score for test dataset: \n")
    precision_score(y_test, y_pred, average='micro')

X_test_red = reduction_method.transform(X_test)

if ( method == 3 ):
    X_test_red1 = reduction_method2.transform(X_test_red)
    X_test_red=X_test_red1
    del X_test_red1


I am working on the file ".ipynb_checkpoints/MNIST-dimensionality-reduction-checkpoint.ipynb".
The first task is:
Now we proceed to reduce the dimensionality of our dataset.
Several methods are proposed:
1. LDA
2. PCA
3. LDA followed by PCA
I found that the third method is used in the practice, but in this case the results are comparable to doing just LDA.
In this example I found that PCA together with a polynomial regression yields the best results.
Can you write Python code for it?

#Note than n_components for lda is < n_class (9)

method=2 #PCA, read above.

if ( method == 1 ):
    dimensionality_reduction_method="lda"
    n_components=9
    reduction_method = LDA(n_components=n_components)
elif ( method == 2 ):
    dimensionality_reduction_method="pca"
    n_components=14
    reduction_method = PCA(n_components=n_components)
elif ( method == 3 ):
    dimensionality_reduction_method="lda"
    n_components=784
    reduction_method = LDA()

print ( "Reducing dimensionality to %d components\n" %(n_components))

print(X_train.shape)

#del X_train_red
# Taking in as second argument the Target as labels
reduction_method = reduction_method.fit(X_train.values, y_train.values )
X_train_red = reduction_method.transform(X_train.values)
print(X_train_red.shape)

if ( method == 3 ):
    dimensionality_reduction_method="pca"
    n_components=5
    reduction_method2 = PCA(n_components=n_components)

# Taking in as second argument the Target as labels
    reduction_method2 = reduction_method2.fit(X_train_red, y_train.values )
    X_train_red1 = reduction_method2.transform(X_train_red)
    X_train_red = X_train_red1
    del X_train_red1
    print(X_train_red.shape)

from IPython.display import display, Math, Latex

# Using the Plotly library again
traceDIM = go.Scatter(
    x = X_train_red[:,0],
    y = X_train_red[:,1],
    name = y_train,
    mode = 'markers',
    text = y_train,
    showlegend = True,
    marker = dict(
        size = 8,
        color = y_train,
        colorscale ='Jet',
        showscale = False,
        line = dict(
            width = 2,
            color = 'rgb(255, 255, 255)'
        ),
        opacity = 0.8
    )
)
data = [traceDIM]

layout = go.Layout(
#    title= title,
    hovermode= 'closest',
    xaxis= dict(
         title= 'First Linear Discriminant',
        ticklen= 5,
        zeroline= False,
        gridwidth= 2,
    ),
    yaxis=dict(
        title= 'Second Linear Discriminant',
        ticklen= 5,
        gridwidth= 2,
    ),
    showlegend= False
)

if (...


I am working on the file ".ipynb_checkpoints/MNIST-dimensionality-reduction-checkpoint.ipynb".
The first task is:
The MNIST set consists of 59999 rows and 785 columns. There are 28 x 28 pixel images of digits ( contributing to 784 columns) as well as one extra label column which is essentially a class label to state whether the row-wise contribution to each digit gives a 1 or a 9. See a few pics here.
Can you write Python code for it?

# Copy the features and target columns to different arrays:
y_train= train['label']
# Drop the label feature
X_train = train.drop("label",axis=1)

# plot some of the numbers
plt.figure(figsize=(7,7))
for digit_num in range(0,6):
    plt.subplot(2,3,digit_num+1)
    grid_data = X_train.iloc[digit_num,:].as_matrix().reshape(28,28)  # reshape from 1d to 2d pixel array
    plt.imshow(grid_data, interpolation = "none", cmap = "afmhot")
    plt.xticks([])
    plt.yticks([])
plt.tight_layout()


I am working on the file ".ipynb_checkpoints/MNIST-dimensionality-reduction-checkpoint.ipynb".
The first task is:
The above plot shows how the method results or not in separated classes. The LDA is made to maximize the class separation while the PCA maximizes the class variance. Here I chose PCA that results in not so clearly separated classes, to see this plot within LDA set method=1 above.
Can you write Python code for it?

test = pd.read_csv('./datasets/mnist_test.csv')
test.columns=columns

test.head()

# save the labels to a Pandas series target
y_test = test['label']
# Drop the label feature
X_test = test.drop("label",axis=1)


I am working on the file ".ipynb_checkpoints/MNIST-neural-network-checkpoint.ipynb".
The first task is:
MNIST Dataset
We choose the popular MNIST (Mixed National Institute of Standards and Technology) computer vision digit dataset. This contains a series for images of handwriting letters, each of them of 28x28 pixels, see a few pick below.
The datasets are large, please download them from:
https://pjreddie.com/projects/mnist-in-csv/
Can you write Python code for it?

train = pd.read_csv('./datasets/mnist_train.csv').head(5000)
#reduce the size to 3000 to make things fast

columns=[]
columns.append("label")
for ii in range(784):
    columns.append("pixel"+str(ii+1))
train.columns=columns

print("Shape of train dataset: "+str(train.shape))
train.head()


I am working on the file ".ipynb_checkpoints/MNIST-neural-network-checkpoint.ipynb".
The first task is:
Neural network
Use a neural network to predict
Can you write Python code for it?

#use regularization alpha=1e-5
#25 hidden layers

from sklearn.neural_network import MLPClassifier
clf = MLPClassifier(solver='lbfgs', alpha=1e-5,
                   hidden_layer_sizes=(50,), random_state=1)

clf.fit(X_train_red, y_train)
y_pred=clf.predict(X_test_red)


I am working on the file ".ipynb_checkpoints/MNIST-neural-network-checkpoint.ipynb".
The first task is:
Now let's check the performance of the neural network.
Remember the precision is the ratio tp / (tp + fp) where tp is the number of true positives and fp the number of false positives. The precision is intuitively the ability of the classifier not to label as positive a sample that is negative.
Can you write Python code for it?

from sklearn.metrics import precision_score
from sklearn.metrics import classification_report

#Precision score for test dataset:
print("Precision score for test dataset: \n")
precision_score(y_test, y_pred, average='micro')


I am working on the file ".ipynb_checkpoints/MNIST-neural-network-checkpoint.ipynb".
The first task is:
Now we proceed to reduce the dimensionality of our dataset using PCA, as in the previous example. For this case the neural network performs better if followed by a dimensionality reduction.
Can you write Python code for it?

#Note than n_components for lda is < n_class (9)

dimensionality_reduction_method="pca"
n_components=50
reduction_method = PCA(n_components=n_components)

print ( "Reducing dimensionality to %d components\n" %(n_components))

print(X_train.shape)

#del X_train_red
# Taking in as second argument the Target as labels
reduction_method = reduction_method.fit(X_train.values, y_train.values )
X_train_red = reduction_method.transform(X_train.values)
print(X_train_red.shape)

test = pd.read_csv('./datasets/mnist_test.csv')
test.columns=columns

test.head()

# save the labels to a Pandas series target
y_test = test['label']
# Drop the label feature
X_test = test.drop("label",axis=1)

X_test_red = reduction_method.transform(X_test)


I am working on the file ".ipynb_checkpoints/MNIST-neural-network-checkpoint.ipynb".
The first task is:
Standarize data:
sklearn provides tools to standarize data as follows.
Can you write Python code for it?

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
# Don't cheat - fit only on training data
scaler.fit(X_train_red)
X_train_red = scaler.transform(X_train_red)
# apply same transformation to test data
X_test_red = scaler.transform(X_test_red)


I am working on the file ".ipynb_checkpoints/MNIST-neural-network-checkpoint.ipynb".
The first task is:
The MNIST set consists of 59999 rows and 785 columns. There are 28 x 28 pixel images of digits ( contributing to 784 columns) as well as one extra label column which is essentially a class label to state whether the row-wise contribution to each digit gives a 1 or a 9. See a few pics here.
Can you write Python code for it?

# Copy the features and target columns to different arrays:
y_train= train['label']
# Drop the label feature
X_train = train.drop("label",axis=1)

# plot some of the numbers
plt.figure(figsize=(7,7))
for digit_num in range(0,12):
    plt.subplot(2,6,digit_num+1)
    grid_data = X_train.iloc[digit_num,:].as_matrix().reshape(28,28)  # reshape from 1d to 2d pixel array
    plt.imshow(grid_data, interpolation = "none", cmap = "afmhot")
    plt.xticks([])
    plt.yticks([])
plt.tight_layout()


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
After 50 epochs, the autoencoder seems to reach a stable train/test loss value of about {{score}}. We can try to visualize the reconstructed inputs and the encoded representations. We will use Matplotlib.
Can you write Python code for it?

encoded_imgs = single_encoder.predict(x_val)
# decoded_imgs = single_decoder.predict(encoded_imgs)
decoded_imgs = single_autoencoder.predict(x_val)

n = 10
plt.figure(figsize=(20, 6))
for i in range(n):
    # display original
    ax = plt.subplot(2, n, i + 1)
    plt.imshow(x_val[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)

    # display reconstruction
    ax = plt.subplot(2, n, i + 1 + n)
    plt.imshow(decoded_imgs[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Classification of denoised data
Can you write Python code for it?

score = model.evaluate(decoded_imgs, y_val, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Decoder Model:
Can you write Python code for it?

encoded_input = Input(shape=(encoding_dim,))
# retrieve the last layer of the autoencoder model
decoder_layer = single_autoencoder.layers[-1]
# create the decoder model
single_decoder = Model(encoded_input, decoder_layer(encoded_input))


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Denoising Data
Can you write Python code for it?

noise_factor = 0.5

x_train_noisy = x_train + noise_factor * np.random.normal(loc=0.0, scale=1.0, size=x_train.shape)
x_val_noisy = x_val + noise_factor * np.random.normal(loc=0.0, scale=1.0, size=x_val.shape)

# re-normalization by clipping to the intervall (0,1)
x_train_noisy = np.clip(x_train_noisy, 0., 1.)
x_val_noisy = np.clip(x_val_noisy, 0., 1.)

n = 10
plt.figure(figsize=(20, 2))
for i in range(n):
    ax = plt.subplot(1, n, i + 1)
    plt.imshow(x_val_noisy[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Dropout consists in randomly setting a fraction rate of input units to 0 at each update during training time, which helps prevent overfitting.
Can you write Python code for it?

input_data = Input(shape=(input_dim,), dtype='float32', name='main_input')
x = Dense(hidden1_dim, activation='relu', kernel_initializer='normal')(input_data)
x = Dropout(0.2)(x)
x = Dense(hidden2_dim, activation='relu', kernel_initializer='normal')(x)
x = Dropout(0.2)(x)
output_layer = Dense(num_classes, activation='softmax', kernel_initializer='normal')(x)

model = Model(input_data, output_layer)

model.compile(loss='categorical_crossentropy',
              optimizer=RMSprop(),
              metrics=['accuracy'])

Image("images/mnist_nn1.png")

model.fit(x_train, y_train,
           batch_size=batch_size,
           epochs=20,
           verbose=0,
           validation_split=0.1)

score = model.evaluate(x_val, y_val, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])

fig = plt.figure(figsize=(20,10))
plt.plot(model.history.history['val_acc'])
plt.plot(model.history.history['acc'])
plt.axhline(y=score[1], c="red")
plt.text(0, score[1], "test: " + str(round(score[1], 4)), fontdict=font)
plt.title('model accuracy for neural net with 2 hidden layers')
plt.ylabel('accuracy')
plt.xlabel('epochs')
plt.legend(['train', 'valid'], loc='lower right')
plt.show()


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Encoder Model:
Can you write Python code for it?

# this model maps an input to its encoded representation
single_encoder = Model(input_img, encoded)


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Global Variables
The number of classes, the input dimension and the batch size are constant.
Can you write Python code for it?

num_classes = 10
input_dim = 784
batch_size = 256


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Save the model
Can you write Python code for it?

# stacked_autoencoder.save('models/stacked_autoencoder.h5')

score = stacked_autoencoder.evaluate(x_val, x_val, verbose=0)
print(score)


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Single autoencoder
Model Definitions
Using keras module with compression to 32 floats.
Can you write Python code for it?

encoding_dim = 32

# input placeholder
input_img = Input(shape=(input_dim,))
encoded = Dense(encoding_dim, activation='relu')(input_img)
decoded = Dense(input_dim, activation='sigmoid')(encoded)

single_autoencoder = Model(input_img, decoded)


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Sparse Autoencoder
Can you write Python code for it?

sparse_autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

sparse_autoencoder = keras.models.load_model('models/sparse_autoencoder.h5')
sparse_autoencoder.fit(x_train, x_train,
                       epochs=20,
                       batch_size=batch_size,
                       validation_data=(x_val, x_val),
                       callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])

# sparse_autoencoder.save('models/sparse_autoencoder.h5')

score = sparse_autoencoder.evaluate(x_val, x_val, verbose=0)
print(score)

encoded_imgs = sparse_encoder.predict(x_val)
# decoded_imgs = single_decoder.predict(encoded_imgs)
decoded_imgs = sparse_decoder.predict(encoded_imgs)

n = 10
plt.figure(figsize=(20, 6))
for i in range(n):
    # display original
    ax = plt.subplot(2, n, i + 1)
    plt.imshow(x_val[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)

    # display reconstruction
    ax = plt.subplot(2, n, i + 1 + n)
    plt.imshow(decoded_imgs[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()

encoded_imgs.mean()


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Stick with the stack
Can you write Python code for it?

denoising_autoencoder = Model(input_img, decoded)
denoising_autoencoder.compile(optimizer=RMSprop(), loss='binary_crossentropy')


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Test the Vanilla Autoencoder with a simple Neural Net (not for presentation)
Can you write Python code for it?

single_ae_test_model = Sequential()
# single_ae_test_model.add(Dense(16, activation='relu', input_shape=(36,)))
# single_ae_test_model.add(Dropout(0.2))
single_ae_test_model.add(Dense(num_classes, activation='softmax', input_shape=(encoding_dim,)))

single_ae_test_model.compile(loss='categorical_crossentropy',
                             optimizer=RMSprop(),
                             metrics=['accuracy'])

single_ae_test_model = keras.models.load_model('models/single_ae_test_model.h5')
# single_ae_test_model.fit(single_encoder.predict(x_train), y_train,
#                          batch_size=batch_size,
#                          epochs=40,
#                          verbose=1,
#                          validation_data=(single_encoder.predict(x_val), y_val))

single_ae_test_model.save('models/single_ae_test_model.h5')

score = single_ae_test_model.evaluate(single_encoder.predict(x_val), y_val, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
The difference in accuracy is huge. The autoencoder architecture is highly recommendable for denoising signals.
Train with sparsity constraint on hidden layer (not for presentation)
So what does all this mean for sparsity? How does the rectifier activation function allow for sparsity in the hidden units? If the hidden units are exposed to a range of input values it makes sense that the rectifier activation function should lead to more ‘real zeros’ as we sweep across possible inputs. In other words, less neurons in our network would activate because of the limitations imposed by the rectifier activation function. https://www.quora.com/What-does-it-mean-that-activation-functions-like-ReLUs-in-NNs-induce-sparsity-in-the-hidden-units
Can you write Python code for it?

input_img = Input(shape=(input_dim,))
# add a Dense layer with a L1 activity regularizer
encoded = Dense(encoding_dim, activation='relu',
                activity_regularizer=regularizers.l1(10e-5))(input_img)
decoded = Dense(input_dim, activation='sigmoid')(encoded)

sparse_autoencoder = Model(input_img, decoded)

sparse_encoder = Model(input_img, encoded)

encoded_input = Input(shape=(encoding_dim,))
# retrieve the last layer of the autoencoder model
decoder_layer = sparse_autoencoder.layers[-1]
# create the decoder model
sparse_decoder = Model(encoded_input, decoder_layer(encoded_input))


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
The result is slightly better than for single autoencoder.
Can you write Python code for it?

# plot_model(stacked_autoencoder, to_file='images/stacked_autoencoderTD.png', show_shapes=True, show_layer_names=True, rankdir='LR')

Image("images/stacked_autoencoderTD.png")

decoded_imgs = stacked_autoencoder.predict(x_val)

n = 10  # how many digits we will display
plt.figure(figsize=(20, 4))
for i in range(n):
    # display original
    ax = plt.subplot(2, n, i + 1)
    plt.imshow(x_val[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)

    # display reconstruction
    ax = plt.subplot(2, n, i + 1 + n)
    plt.imshow(decoded_imgs[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Train Neural Net to recognize MNIST digits
Can you write Python code for it?

# constants
hidden1_dim = 512
hidden2_dim = 512


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Train or load a stacked denoising autoencoder
Can you write Python code for it?

# denoising_autoencoder = keras.models.load_model('models/denoising_autoencoder.h5')
denoising_autoencoder.fit(x_train_noisy, x_train,
                          epochs=epochs,
                          batch_size=batch_size,
                          shuffle=True,
                          validation_split=0.1,
                          verbose=0,
                          callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])

fig = plt.figure(figsize=(20,10))
plt.plot(denoising_autoencoder.history.history['val_loss'])
plt.plot(denoising_autoencoder.history.history['loss'])
plt.title('model loss for stacked denoising autoencoder')
plt.ylabel('loss')
plt.xlabel('epochs')
plt.legend(['train', 'valid'], loc='lower right')
plt.show()

decoded_imgs = denoising_autoencoder.predict(x_val_noisy)

n = 10  # how many digits we will display
plt.figure(figsize=(20, 4))
for i in range(n):
    # display original
    ax = plt.subplot(2, n, i + 1)
    plt.imshow(x_val[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)

    # display reconstruction
    ax = plt.subplot(2, n, i + 1 + n)
    plt.imshow(decoded_imgs[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder-checkpoint.ipynb".
The first task is:
Train/load model
Can you write Python code for it?

# single_autoencoder = keras.models.load_model('models/single_autoencoder.h5')
single_autoencoder.fit(x_train, x_train,
                        epochs=50,
                        batch_size=batch_size,
                        shuffle=False,
                       verbose=0,
                        validation_split = 0.1,
                      callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])

# single_autoencoder.save('models/single_autoencoder.h5')

score = single_autoencoder.evaluate(x_val, x_val, verbose=0)
print(score)

# plot_model(single_autoencoder, to_file='images/single_autoencoder.png', show_shapes=True, show_layer_names=True, rankdir='LR')

Image("images/single_autoencoder.png")


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder_dim_reduction-checkpoint.ipynb".
The first task is:
Global Variables
Can you write Python code for it?

num_classes = 10
input_dim = 784
encoder_dim = 32
epochsize = 50
batchsize = 256


I am working on the file ".ipynb_checkpoints/MNIST_Autoencoder_dim_reduction-checkpoint.ipynb".
The first task is:
Neural Net Classifier with Single Autoencoder (1 hidden layer)
Can you write Python code for it?

main_input = Input(shape=(input_dim,), dtype='float32', name='main_input')

encoding_layer = Dense(encoder_dim, activation='relu', kernel_initializer='normal')
encoding_layer_output = encoding_layer(main_input)
decoding_layer_output = Dense(input_dim
                              ,activation='sigmoid'
                              ,name='decoder_output'
                              ,kernel_initializer='normal')(encoding_layer_output)

classifier_output = Dense(num_classes
                          ,activation='softmax'
                          ,name='main_output'
                          ,kernel_initializer='normal')(encoding_layer_output)

auto_classifier = Model(inputs=main_input, outputs=[classifier_output, decoding_layer_output])

auto_classifier.compile(optimizer=RMSprop(),
                        loss={'main_output': 'categorical_crossentropy', 'decoder_output': 'mean_squared_error'},
                        loss_weights={'main_output':1., 'decoder_output': 1.},
                        metrics=['accuracy'])

plot_model(auto_classifier, to_file='images/mnist_dim_reduction.png', show_shapes=True, show_layer_names=True, rankdir='LR')

Image("images/mnist_dim_reduction.png")

auto_classifier.fit({'main_input': x_train}
                    ,{'main_output': y_train, 'decoder_output': x_train}
                    ,epochs=epochsize
                    ,batch_size=batchsize
                    ,shuffle=True
                    ,validation_split=0.05
                    ,verbose=0
                    ,callbacks=[TensorBoard(log_dir='./logs')])

score = auto_classifier.evaluate(x=x_val, y=[y_val, x_val], verbose=1)[3]
print(score)

fig = plt.figure(figsize=(20,10))
plt.plot(auto_classifier.history.history['val_main_output_acc'])
plt.plot(auto_classifier.history.history['main_output_acc'])
plt.axhline(y=score, c="red")
plt.text(0, score, "test: " + str(round(score, 4)), fontdict=font)
plt.title('model accuracy for ' + str(round(input_dim / encoder_dim, 2)) + ' x...


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
After 50 epochs, the autoencoder seems to reach a stable train/test loss value of about {{score}}. We can try to visualize the reconstructed inputs and the encoded representations. We will use Matplotlib.
Can you write Python code for it?

encoded_imgs = single_encoder.predict(x_val)
# decoded_imgs = single_decoder.predict(encoded_imgs)
decoded_imgs = single_autoencoder.predict(x_val)

n = 10
plt.figure(figsize=(20, 6))
for i in range(n):
    # display original
    ax = plt.subplot(2, n, i + 1)
    plt.imshow(x_val[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)

    # display reconstruction
    ax = plt.subplot(2, n, i + 1 + n)
    plt.imshow(decoded_imgs[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
Compare results
Classification of noisy data
Can you write Python code for it?

score = model.evaluate(x_val_noisy, y_val, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
Data import and preprocessing
All data is normalized and serialized into a vector.
Can you write Python code for it?

(x_train, y_train), (x_val, y_val) = mnist.load_data()
y_train

(x_train, y_train), (x_val, y_val) = mnist.load_data()

x_train = x_train.astype('float32') / 255.
x_val = x_val.astype('float32') / 255.

x_train = x_train.reshape((len(x_train), np.prod(x_train.shape[1:])))
x_val = x_val.reshape((len(x_val), np.prod(x_val.shape[1:])))
print(x_train.shape)

y_train = keras.utils.to_categorical(y_train, num_classes)
y_val = keras.utils.to_categorical(y_val, num_classes)


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
Decoder Model:
Can you write Python code for it?

encoded_input = Input(shape=(encoding_dim,))
# retrieve the last layer of the autoencoder model
decoder_layer = single_autoencoder.layers[-1]
# create the decoder model
single_decoder = Model(encoded_input, decoder_layer(encoded_input))


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
Denoising Data
Can you write Python code for it?

noise_factor = 0.5

x_train_noisy = x_train + noise_factor * np.random.normal(loc=0.0, scale=1.0, size=x_train.shape)
x_val_noisy = x_val + noise_factor * np.random.normal(loc=0.0, scale=1.0, size=x_val.shape)

# re-normalization by clipping to the intervall (0,1)
x_train_noisy = np.clip(x_train_noisy, 0., 1.)
x_val_noisy = np.clip(x_val_noisy, 0., 1.)

n = 10
plt.figure(figsize=(20, 2))
for i in range(n):
    ax = plt.subplot(1, n, i + 1)
    plt.imshow(x_val_noisy[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
Encoder Model:
Can you write Python code for it?

# this model maps an input to its encoded representation
single_encoder = Model(input_img, encoded)


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
First, we'll configure our model to use a per-pixel binary crossentropy loss, and the RMSprop optimizer:
Binary Cross Entropy = Binomial Cross Entropy = Special Case of Multinomial Cross Entropy
Can you write Python code for it?


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
Global Variables
The number of classes, the input dimension and the batch size are constant.
Can you write Python code for it?

num_classes = 10
input_dim = 784
batch_size = 256


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
Save the model
Can you write Python code for it?

# stacked_autoencoder.save('models/stacked_autoencoder.h5')

score = stacked_autoencoder.evaluate(x_val, x_val, verbose=0)
print(score)


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
Single autoencoder
Model Definitions
Using keras module with compression to 32 floats.
Can you write Python code for it?

encoding_dim = 32

# input placeholder
input_img = Input(shape=(input_dim,))
encoded = Dense(encoding_dim, activation='relu')(input_img)
decoded = Dense(input_dim, activation='sigmoid')(encoded)

single_autoencoder = Model(input_img, decoded)


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
Sparse Autoencoder
Can you write Python code for it?

sparse_autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

sparse_autoencoder = keras.models.load_model('models/sparse_autoencoder.h5')
sparse_autoencoder.fit(x_train, x_train,
                       epochs=20,
                       batch_size=batch_size,
                       validation_data=(x_val, x_val),
                       callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])

# sparse_autoencoder.save('models/sparse_autoencoder.h5')

score = sparse_autoencoder.evaluate(x_val, x_val, verbose=0)
print(score)

encoded_imgs = sparse_encoder.predict(x_val)
# decoded_imgs = single_decoder.predict(encoded_imgs)
decoded_imgs = sparse_decoder.predict(encoded_imgs)

n = 10
plt.figure(figsize=(20, 6))
for i in range(n):
    # display original
    ax = plt.subplot(2, n, i + 1)
    plt.imshow(x_val[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)

    # display reconstruction
    ax = plt.subplot(2, n, i + 1 + n)
    plt.imshow(decoded_imgs[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()

encoded_imgs.mean()


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
Stacked Autoencoder
Can you write Python code for it?

######## constants for stacked autoencoder ############
encoding_dim1 = 128
encoding_dim2 = 64
encoding_dim3 = 32
decoding_dim1 = 64
decoding_dim2 = 128
decoding_dim3 = input_dim
epochs = 100
batch_size = 256

input_img = Input(shape=(input_dim,))
encoded = Dense(encoding_dim1, activation='relu')(input_img)
encoded = Dense(encoding_dim2, activation='relu')(encoded)
encoded = Dense(encoding_dim3, activation='relu')(encoded)

decoded = Dense(decoding_dim1, activation='relu')(encoded)
decoded = Dense(decoding_dim2, activation='relu')(decoded)
decoded = Dense(decoding_dim3, activation='sigmoid')(decoded)

stacked_autoencoder = Model(input_img, decoded)
stacked_autoencoder.compile(optimizer=RMSprop(), loss='binary_crossentropy')

stacked_autoencoder = keras.models.load_model('models/stacked_autoencoder.h5')
#stacked_autoencoder.fit(x_train, x_train,
#                 epochs=epochs,
#                 batch_size=batch_size,
#                 shuffle=False,
#                 validation_split=0.1)


I am working on the file ".ipynb_checkpoints/MNIST_advanced-checkpoint.ipynb".
The first task is:
Stick with the stack
Can you write Python code for it?

denoising_autoencoder = Model(input_img, decoded)
denoising_autoencoder.compile(optimizer=RMSprop(), loss='binary_crossentropy')