I am working on the file "2_Mathematical_Groundwork/2_11_least_squares.ipynb".
The first task is:
We can now compare our numerical result with both the truth and the data. The following plot shows the various quantities of interest.
Can you write Python code for it?

plt.plot(t, d, label="Data")
plt.plot(t, sinusoid(solved_x, t), label="LM")
plt.plot(t, sinusoid(true_x, t), label="Truth")
plt.xlabel("t")
plt.legend(loc='upper right')
plt.show()


I am working on the file "2_Mathematical_Groundwork/2_13_spherical_trigonometry.ipynb".
The first task is:
Import section specific modules:
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/2_14_CLEAN_in_1D.ipynb".
The first task is:
2.14 Mathematics of the CLEAN algorithm
Here we give the mathematical details behind the implementation of the CLEAN algorithm. In particular we present deconvolution as an iterative $\chi^2$ minimisation of a linear system of equations. You should review the notation for linear systems given in $\S$ 2.10.3 ➞. Although, by this stage, you should have the necessary mathematics to understand this section, it is meant to supplement the material given in $\S$ 6 ➞ and is best read after you have become familiar with the basic concepts behind imaging and deconvolution.
2.14.1 Imaging as an iterative $\chi^2$ minimisation problem
The imaging problem can be formulated as a linear system of the form
$$ S F I = V_{obs} + \epsilon, \quad \mbox{where} \quad \epsilon \sim \mathcal{N}(0,\Sigma).$$
Here $S$ is the sampling operator, $F$ the Direct Fourier transform and $I$ is the image we are after. The assumption that $\epsilon$ is Gaussian noise allows us to form the $\chi^2$ distribution as
$$ \chi^2 = (S F I - V_{obs})^H W (S F I - V_{obs}), $$
where $W$ are the imaging weights. As already mentioned in $\S$ 2.10.3 (and elsewhere I am sure???), in interferometry, $W$ is not necessarily just the inverse of the covariance matrix $\Sigma^{-1}$. As will be demonstrated below the weights can actually be chosen to enhance certain desirable features in the image. For example, we could choose to implement uniform weighting which would down weight short baselines resulting in a sharper point spread function (denoted $I^{PSF}$) with smaller sidelobes. The downside of such a weighting scheme is that we lose signal to noise in the reconstructed dirty image (denoted $I^{D}$). We will give some examples below. For now our goal is to figure out how to find the image $I$ which minimises the $\chi^2$. Recall that this can be achieved by solving the normal equations. With the model as given above the gradient (also the Jacobian) is given by
$$ \partial_x \chi^2 = \mathbb{J}(I) = F^H S^H W (S F I - V_{obs}). $$
Similarly the...
Can you write Python code for it?

N = 300 # the maximum number of "baselines" that can be sampled
u = np.linspace(-20,20,N) # a grid of "baselines"

Npix = 300 # the number of "pixels" in the image
lmin = -0.5
lmax = 0.5
l = np.linspace(lmin,lmax,Npix) # The region where we want to reconstruct the image

#deltal = l[1]-l[0]

Is1 = 1 # true flux of source one
Is2 = 2.5 # true flux of second source

# Set locations of sources (also ensuring they are at pixel centres)
l1 = l[np.argwhere(l < -0.15)[-1]]
l2 = l[np.argwhere(l < 0.25)[-1]]

# Compute the anaytic form of the visibilities
V = Is1*np.exp(-2*np.pi*1.0j*u*l1) + Is2*np.exp(-2*np.pi*1.0j*u*l2)

# Plot the visibilities
fig,ax = plt.subplots(nrows=1,ncols=2,figsize=(15,8))

ax[0].plot(u,np.real(V))
ax[0].set_xlabel('$u$',fontsize=18)
ax[0].set_ylabel('$real(V(u))$',fontsize=18)
ax[0].set_xlim(-20.,20.)
ax[0].set_ylim(-5,5)

ax[1].plot(u,np.imag(V))
ax[1].set_xlabel('$u$',fontsize=18)
ax[1].set_ylabel('$imag(V(u))$',fontsize=18)
ax[1].set_xlim(-20.,20.)
ax[1].set_ylim(-5,5)


I am working on the file "2_Mathematical_Groundwork/2_14_CLEAN_in_1D.ipynb".
The first task is:
2.14.3 Weighting and the PSF
This is incomplete. I thought it might be cool to demonstrate the effect of weighting for the 1D problem. Slider controls the degree of weighting.
Can you write Python code for it?

def inter_PSF(S,F,u,R,l):
    Ws = np.zeros(u.size)
    I = np.argwhere(u<0)
    Ws[I] = np.abs((-u[I])**R)
    I = np.argwhere(u>=0)
    Ws[I] = np.abs(u[I]**R)
    Ipsf = np.real(np.dot(F.conj().T,S*Ws))
    Ipsf /= Ipsf.max()
    fig,ax = plt.subplots(nrows=1,ncols=2,figsize=(15,8))
    ax[0].plot(l,Ipsf)
    ax[0].set_xlim(-1,1)
    ax[0].set_ylim(-0.1,1.1)
    ax[0].set_title(r'$PSF$')
    ax[1].plot(u,Ws)
    ax[1].set_xlim(-20,20)
    ax[1].set_ylim(0,80)
    ax[0].set_title(r'$Visibility Weights$')

interact(lambda R:inter_PSF(S=S2,F=F2psf,u=u[Sindices],R=R,l=lpsf),
                R=(-1.5,1.5,0.25)) and None


I am working on the file "2_Mathematical_Groundwork/2_14_CLEAN_in_1D.ipynb".
The first task is:
Clearly the CS CLEAN does a lot better at recovering the true fluxes of the sources. We have only one job left to do and that is to create the restored image. The restored image is the model image convolved with an ideal beam (i.e. one with no sidelobes). The ideal beam can be constructed by fitting a Gaussian to the primary lobe of the PSF.
Can you write Python code for it?

def Gaussian_1D(x, sigma, l):
    return sigma**2*np.exp(-x**2/(2*l**2))

def fit_Gaussian_1D(Ipsf,lpsf):
    I = np.argwhere(Ipsf > 0.5*Ipsf).squeeze()
    data = Ipsf[I]
    x = (lpsf[I])

    initial_guess = (1.0,0.5)

    popt, pcov = opt.curve_fit(Gaussian_1D, x, data, p0=initial_guess)
    Ipsf_ideal = Gaussian_1D((lpsf), *popt)
    Ipsf_ideal/=Ipsf_ideal.max()

    return Ipsf_ideal

Ipsf_ideal = fit_Gaussian_1D(Ipsf,lpsf)

# Convolve with sky model
Irestored = scg.convolve(IM,Ipsf_ideal, mode='same')

# Add back the residuals
Irestored += IR

# Plot the results
fig,ax = plt.subplots(nrows=1,ncols=2,figsize=(15,8))

ax[0].plot(lpsf,Ipsf_ideal)
ax[0].set_xlabel('$l$')
ax[0].set_ylabel('$\hat{I}^{PSF}$',fontsize=18)
ax[0].set_xlim(-1,1)

ax[1].plot(l,Irestored)
ax[1].set_xlabel('$l$')
ax[1].set_ylabel('$I^r$',fontsize=18)
ax[1].set_xlim(lmin,lmax)
ax[1].set_ylim(0,3.5)


I am working on the file "2_Mathematical_Groundwork/2_14_CLEAN_in_1D.ipynb".
The first task is:
Now there are clearly two main peaks visible in $I^{D}$ but the units seem all wrong. Normalising both by the max of the PSF we see something that is closer to the original input model.
Can you write Python code for it?

maxPSF = Ipsf2.max()

fig,ax = plt.subplots(nrows=1,ncols=2,figsize=(15,8))

ax[0].plot(lpsf,Ipsf2/maxPSF)
ax[0].set_xlabel('$l$')
ax[0].set_ylabel('$I^{PSF}$',fontsize=18)
ax[0].set_xlim(2*lmin,2*lmax)

ax[1].plot(l,ID2/maxPSF)
ax[1].set_xlabel('$l$')
ax[1].set_ylabel('$I^D$',fontsize=18)
ax[1].set_xlim(lmin,lmax)


I am working on the file "2_Mathematical_Groundwork/2_14_CLEAN_in_1D.ipynb".
The first task is:
Ok so lets try it out
Can you write Python code for it?

Ipsf = Ipsf2/maxPSF
ID = ID2/maxPSF
IM, IR = Hogbom_CLEAN(ID.copy(),Ipsf.copy(),tol=0.025)

# Plot the results
fig,ax = plt.subplots(nrows=1,ncols=2,figsize=(15,8))

ax[0].stem(l,IM)
ax[0].set_xlabel('$l$')
ax[0].set_ylabel('$I^{sky}$',fontsize=18)
ax[0].set_xlim(lmin,lmax)

ax[1].plot(l,IR)
ax[1].set_xlabel('$l$')
ax[1].set_ylabel('$I^R$',fontsize=18)
ax[1].set_xlim(lmin,lmax)


I am working on the file "2_Mathematical_Groundwork/2_14_CLEAN_in_1D.ipynb".
The first task is:
The figure on the left shows the PSF that results from imperfect sampling in the "$uv$-plane". We should expect to see something that resembles a sinc function (why?). On the right is the dirty "image" we would see if we did not account for aliasing. Use of the FFT inevitably requires the use of an anti-aliasing filter too. In this tut we get around that by using the irregularly sampled version of $V^{obs}$ (similar to using the dirct Fourier transform in imaging). The figures below show the anti-aliased version of $I^D$. Note that we have also computed $I^{PSF}$ out to double the distance as compared to $I^D$. This will be reuired for the deconvolution.
Can you write Python code for it?

fig,ax = plt.subplots(nrows=1,ncols=2,figsize=(15,8))

ax[0].plot(lpsf,Ipsf2)
ax[0].set_xlabel('$l$')
ax[0].set_ylabel('$I^{PSF}$',fontsize=18)
ax[0].set_xlim(2*lmin,2*lmax)
#ax[0].set_ylim(-5,5)

ax[1].plot(l,ID2)
ax[1].set_xlabel('$l$')
ax[1].set_ylabel('$I^D$',fontsize=18)
ax[1].set_xlim(lmin,lmax)
#ax[1].set_ylim(-5,5)


I am working on the file "2_Mathematical_Groundwork/2_14_CLEAN_in_1D.ipynb".
The first task is:
We are now ready to implement the Hogbom CLEAN. The function below implements the Hogbom CLEAN algorithm in 1D.
Can you write Python code for it?

def Hogbom_CLEAN(IR,Ipsf,gamma=0.1,tol=0.1,maxiter=200,restore=False):
    # Initialise
    i = 1
    Npix = IR.size
    Npsfmax = np.argwhere(Ipsf==1.0)
    IM = np.zeros([Npix])
    # Find the first peak
    p = np.argwhere(abs(IR) == abs(IR).max()).squeeze()
    Ipeak = IR[p]
    while abs(Ipeak) > tol and i <= maxiter:
        # Update the residual
        IR -= gamma*Ipeak*Ipsf[int(Npsfmax - p):int(Npsfmax + Npix - p)]
        # Update the sky model
        IM[p] += gamma*Ipeak
        # Find next peak
        p = np.argwhere(abs(IR) == abs(IR).max()).squeeze()
        Ipeak = IR[p]
        i += 1
        #print i, Ipeak, p
    if i >= maxiter:
        print("Warning maximum number of iterations exceeded")
    return IM, IR


I am working on the file "2_Mathematical_Groundwork/2_3_fourier_series.ipynb".
The first task is:
2.3. Fourier Series
While Fourier series are not immediately required to understand the required calculus for this book, they are closely connected to the Fourier transform, which is an essential tool. Moreover, we noticed a few times that the principle of the harmonic analysis or harmonic decomposition is essential and, despite its simplicity, often not well understood. We hence give a very brief summary, not caring about existence questions.
2.3.1 Definition
The Fourier series of a function $f: \mathbb{R} \rightarrow \mathbb{R}$ with real coefficients is defined as
$$
    f_{\rm F}(x) \,=\, \frac{1}{2}c_0+\sum_{m = 1}^{\infty}c_m \,\cos(mx)+\sum_{m = 1}^{\infty}s_m \,\sin(mx),
$$
with the Fourier coefficients $c_m$ and $s_m$
$$
    \left( c_0 \,=\,\frac{1}{\pi}\int_{-\pi}^{\pi}f(x)\,dx \right)\
    c_m \,=\,\frac{1}{\pi}\int_{-\pi}^{\pi}f(x)\,\cos(mx)\,dx \qquad m \in \mathbb{N_0}\
    s_m \,=\,\frac{1}{\pi}\int_{-\pi}^{\pi}f(x)\,\sin(mx)\,dx \qquad m \in \mathbb{N_0}.\{\rm }
$$
If $f_{\rm F}$ exists, it is identical to $f$ in all points of continuity. For functions which are periodic with a period of $2\pi$ the Fourier series converges. Hence, for continuous periodic function with a period of $2\pi$ the Fourier series converges and $f_{\rm F}=f$.
The Fourier series of a function $f: \mathbb{R} \rightarrow \mathbb{R}$ with complex coefficients is defined as
$$
    f_{\rm IF}(x) \,=\, \sum_{m = -\infty}^{\infty}a_m \,e^{\imath mx},$$
with the Fourier coefficients $a_m$
$$
     a_m \,=\, \frac{1}{2\pi}\int_{-\pi}^{\pi}f(x)e^{-\imath mx}\,dx\qquad \forall m\in\mathbb{Z}.$$
The same convergence criteria apply and one realisation can be transformed to the other. Making use of Euler's formula ➞...
Can you write Python code for it?

def FS_coeffs(x, m, func, T=2.0*np.pi):
    """
    Computes Fourier series (FS) coeffs of func
    Input:
        x = input vector at which to evaluate func
        m = the order of the coefficient
        func = the function to find the FS of
        T = the period of func (defaults to 2 pi)
    """
    # Evaluate the integrand
    am_int = func(x)*np.exp(-1j*2.0*m*np.pi*x/T)
    # Use trapezoidal integration to get the coefficient
    am = np.trapz(am_int,x)
    return am/T


I am working on the file "2_Mathematical_Groundwork/2_3_fourier_series.ipynb".
The first task is:
2.3. Fourier Series
While Fourier series are not immediately required to understand the required calculus for this book, they are closely connected to the Fourier transform, which is an essential tool. Moreover, we noticed a few times that the principle of the harmonic analysis or harmonic decomposition is essential- despite its simplicity - often not fully understood. We hence give a very brief summary, not caring about existence questions.
2.3.1 Definition
The Fourier series of a function $f: \mathbb{R} \rightarrow \mathbb{R}$ with real coefficients is defined as
$$
    f_{\rm F}(x) \,=\, \frac{1}{2}c_0+\sum_{m = 1}^{\infty}c_m \,\cos(mx)+\sum_{m = 1}^{\infty}s_m \,\sin(mx),
$$
with the Fourier coefficients $c_m$ and $s_m$
$$
    \left( c_0 \,=\,\frac{1}{\pi}\int_{-\pi}^{\pi}f(x)\,dx \right)\
    c_m \,=\,\frac{1}{\pi}\int_{-\pi}^{\pi}f(x)\,\cos(mx)\,dx \qquad m \in \mathbb{N_0}\
    s_m \,=\,\frac{1}{\pi}\int_{-\pi}^{\pi}f(x)\,\sin(mx)\,dx \qquad m \in \mathbb{N_0}.\{\rm }
$$
If $f_{\rm F}$ exists, it is identical to $f$ in all points of continuity. For functions which are periodic with a period of $2\pi$ the Fourier series converges. Hence, for continuous periodic function with a period of $2\pi$ the Fourier series converges and $f_{\rm F}=f$.
The Fourier series of a function $f: \mathbb{R} \rightarrow \mathbb{R}$ with imaginary coefficients is defined as
$$
    f_{\rm IF}(x) \,=\, \sum_{m = -\infty}^{\infty}a_m \,e^{\imath mx},$$
with the Fourier coefficients $a_m$
$$
     a_m \,=\, \frac{1}{2\pi}\int_{-\pi}^{\pi}f(x)e^{-\imath mx}\,dx\qquad \forall m\in\mathbb{Z}.$$
The same convergence criteria apply and one realisation can be transformed to the other. Making use of Euler's formula ➞...
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/2_3_fourier_series.ipynb".
The first task is:
As can be seen from the figure, the Fourier series approximates the square wave. However at such a low order (i.e. $m = 10$) it doesn't do a very good job. Actually an infinite number of Fourier series coefficients are required to fully capture a square wave. Below is an interactive demonstration that allows you to vary the parameters on the Fourier series decomposition. Note in particular what happens if we make the period too small. Also feel free to apply it to functions other than the square wave (but make sure to adjust the domain accordingly.
Can you write Python code for it?

def inter_FS(x,m,func,T):
    f_F = FS_sum(x, m, func, period=T)
    plt.plot(x,f_F.real,'b')
    plt.plot(x,func(x),'g')

interact(lambda m,T:inter_FS(x=np.linspace(-1.0,1.0,N),m=m,func=square_wave,T=T),
                m=(5,100,1),T=(0,2*np.pi,0.5)) and None


I am working on the file "2_Mathematical_Groundwork/2_3_fourier_series.ipynb".
The first task is:
Figure 2.8.1:
As can be seen from the figure, the Fourier series approximates the square wave. However at such a low order (i.e. $m = 10$) it doesn't do a very good job. Actually an infinite number of Fourier series coefficients are required to fully capture a square wave. Below is an interactive demonstration that allows you to vary the parameters on the Fourier series decomposition. Note in particular what happens if we make the period too small. Also feel free to apply it to functions other than the square wave (but make sure to adjust the domain accordingly.
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/2_3_fourier_series.ipynb".
The first task is:
Import section specific modules:
Can you write Python code for it?

from IPython.display import HTML
from ipywidgets import interact
HTML('../style/code_toggle.html')


I am working on the file "2_Mathematical_Groundwork/2_3_fourier_series.ipynb".
The first task is:
Let's see what happens if we decompose a square wave.
Can you write Python code for it?

# define square wave function
def square_wave(x):
    I = np.argwhere(np.abs(x) <= 0.5)
    tmp = np.zeros(x.size)
    tmp[I] = 1.0
    return tmp

# Set domain and compute square wave
N = 250
x = np.linspace(-1.0,1.0,N)

# Compute the FS up to order m
m = 10
sw_F = FS_sum(x, m, square_wave, period=2.0)

# Plot result
plt.figure(figsize=(15,5))
plt.plot(x, sw_F.real, 'g', label=r'$ Fourier \ series $')
plt.plot(x, square_wave(x), 'b', label=r'$ Square \ wave $')
plt.title(r"$FS \ decomp \ of \ square \ wave$",fontsize=20)
plt.xlabel(r'$x$',fontsize=18)
plt.ylim(-0.05,1.5)
plt.legend()


I am working on the file "2_Mathematical_Groundwork/2_3_fourier_series.ipynb".
The first task is:
That should be good enough for our purposes here. Next we create a function to sum the Fourier series.
Can you write Python code for it?

def FS_sum(x, m, func, period=None):
    # If no period is specified use entire domain
    if period is None:
        period = np.abs(x.max() - x.min())

    # Evaluate the coefficients and sum the series
    f_F = np.zeros(x.size, dtype=np.complex128)
    for i in xrange(-m,m+1):
        am = FS_coeffs(x, i, func, T=period)
        f_F += am*np.exp(2.0j*np.pi*i*x/period)
    return f_F


I am working on the file "2_Mathematical_Groundwork/2_5_convolution.ipynb".
The first task is:
2.5 Convolution
The convolution is an operation connecting two functions, with the result of a mutual broadening. In signal processing, the convolution is often used to represent instrumental broadening of a signal. For any observation, the signal received is "filtered" by an instrumental function. The signal is smeared out. The mathematical description for this effect is the convolution of the function representing the original signal with the instrumental function. In this section, we give a detailed description. Note, the Convolution Theorem, an essential theorem in Fourier analysis and aperture synthesis is discussed in Section 2.7.4 ➞.
2.5.1 Definition of Convolution
The convolution $\circ$ is an operation acting on two complex-valued functions.
$$
\circ: \left{f\,|\, f:\mathbb{R}\rightarrow \mathbb{C}\right}\,\times\, \left{f\,|\, f:\mathbb{R}\rightarrow \mathbb{C}\right} \rightarrow \left{f\,|\, f:\mathbb{R}\rightarrow \mathbb{C}\right}\
(f\circ g)(x) \,=\, \int_{-\infty}^{+\infty} f(x-t)\,g(t)\,dt
$$
or, in more than one dimension
$$
\circ: \left{f\,|\, f:\mathbb{R}^n\rightarrow \mathbb{C}\right}\,\times\, \left{f\,|\, f:\mathbb{R}^n\rightarrow \mathbb{C}\right} \rightarrow \left{f\,|\, f:\mathbb{R}^n\rightarrow \mathbb{C}\right} \, \quad n \in \mathbb{N}\
\begin{align}
(f\circ g)(x_1,\ldots,x_n ) \,&=\, (f\circ g)({\bf x})\
\,&=\, \int_{-\infty}^{+\infty} \ldots \int_{-\infty}^{+\infty} f(x_1-t_1, \ldots , x_n-t_n)\,g(t_1, \ldots, t_n) \,d^nt\
\,&=\, \int_{-\infty}^{+\infty} f({\bf x}-{\bf t})\,g({\bf t}) \,d^nt\end{align}
$$
2.5.2 Properties of Convolution
The following rules apply:
$$
\forall\,f,g\in \left{h\,|\, h:\mathbb{R}\rightarrow \mathbb{C}\right}, \quad a \in \mathbb{C}\
\begin{align}
f\circ g \,&=\,...
Can you write Python code for it?

import math

from matplotlib import rcParams
rcParams['text.usetex'] = True

#def trianglewave(x, T):
#    """
#    This is a sawtooth, though
#    """
#    return np.mod(x/T,1.)*np.logical_and(x>=0,x<=T)

def trianglewave(x, T):
    """
    T is the period.
    """
    return np.abs(2.*(np.mod(x/T,1.)-0.5))-0.5

def boxcar(x,a,b,amp):
    return amp*np.logical_and(x>=a,x<=b)

def plottriboxconv(a, b, period):

    # limits of boxcar Play arround with this
#    a = -0.1
#    b = 0.1

    # Plotting range
    xrange = [-2., 2.]

    # Create functions
    xpoints = 1000

    # Resolution element
    dx = (xrange[1]-xrange[0])/float(xpoints)

    x = np.linspace(xrange[0], xrange[1], xpoints)
    y = boxcar(x, a, b, 1.)

    # boxcar will be normalised to 1. amp = 1./(b-a) works in the limit of many points, but here we do
    # numberofpixelsinbox*dx*amplitude = y.sum *dx*amplitude = 1
    # to take into account numerical effects
    amp = float(xpoints)/((xrange[1]-xrange[0])* y.sum())
    y = boxcar(x, a, b, 1./(b-a))
    ycorr = boxcar(x, a, b, amp)
    z = trianglewave(x, period)

    result = np.convolve(ycorr,z,'same')
    result = dx*result

    # Start the plot, create a figure instance and a subplot
    fig = plt.figure()
    ax1  = fig.add_subplot(311)
    fig.tight_layout()
    plt.subplots_adjust(hspace = 0.6)

    # Axis ranges
    ax1.axis([xrange[0]+(b-a), xrange[1]-(b-a), z.min()-0.1*(z.max()-z.min()), z.max()+0.1*(z.max()-z.min())])

    # Plot a grid
    ax1.grid(True)

    # Insert lines at x=0 and y=0
    ax1.axhline(0.,linewidth=1, color = 'k', linestyle='dashed')
    ax1.axvline(0.,linewidth=1, color = 'k', linestyle='dashed')

    # Plot function
    ax1.plot(x,z,'b-')

    plt.title("Triangle wave", fontsize=14,color='black')

    ax2  = fig.add_subplot(312, sharex=ax1)

    # Axis ranges
    ax2.axis([xrange[0]+(b-a), xrange[1]-(b-a), ycorr.min()-0.1*(ycorr.max()-ycorr.min()), \
              ycorr.max()+0.1*(ycorr.max()-ycorr.min())])

    # Plot a grid
...


I am working on the file "2_Mathematical_Groundwork/2_5_convolution.ipynb".
The first task is:
Figure 2.5.1: Rectangle-filtered triangle wave
One might assume that one is observing a (co-)sine function. But it can get worse:
Can you write Python code for it?

# first two arguments give the position of the rectangle, third the period of the Triangle
plottriboxconv(-0.5, 0.5, 1.0)
#


# first two arguments give the position of the rectangle, third the period of the Triangle
plottriboxconv(-0.5, 0.5, 1.0)


I am working on the file "2_Mathematical_Groundwork/2_5_convolution.ipynb".
The first task is:
Figure 2.5.2: Triangle wave filtered with rectangle function, another example.
This example illustrates that the process of filtering can destroy information about our signal. However, filtering can also be useful. Given noisy observations of a function, a rectangle function can be used to filter out the noise. This is illustrated in the subsequent example.
Can you write Python code for it?

from matplotlib import rcParams
rcParams['text.usetex'] = True

def noisycosinewave(x, amplitude, T, sigma):
    """
    T is the period, sigma is the dispersion, amplitude the amplitude
    """
    return amplitude*np.cos(2.*math.pi*x/T)+np.random.normal(scale=sigma, size=x.size)

def boxcar(x,a,b,amp):
    return amp*np.logical_and(x>=a,x<=b)

def plotcosboxconv(a, b, period, sigma):

    # limits of boxcar Play arround with this
#    a = -0.1
#    b = 0.1

    # Plotting range
    xrange = [-2., 2.]

    # Create functions
    xpoints = 1000

    # Resolution element
    dx = (xrange[1]-xrange[0])/float(xpoints)

    x = np.linspace(xrange[0], xrange[1], xpoints)
    y = boxcar(x, a, b, 1.)

    # boxcar will be normalised to 1. amp = 1./(b-a) works in the limit of many points, but here we do
    # numberofpixelsinbox*dx*amplitude = y.sum *dx*amplitude = 1
    # to take into account numerical effects
    amp = float(xpoints)/((xrange[1]-xrange[0])* y.sum())
    y = boxcar(x, a, b, 1./(b-a))
    ycorr = boxcar(x, a, b, amp)
    z = noisycosinewave(x, 1., period, sigma)
    c = np.cos(2.*math.pi*x/period)

    result = np.convolve(ycorr,z,'same')
    result = dx*result

    # Start the plot, create a figure instance and a subplot
    fig = plt.figure()

    ax1  = fig.add_subplot(411)
    fig.tight_layout()
    plt.subplots_adjust(hspace = 0.8)

    # Axis ranges
    ax1.axis([xrange[0]+(b-a), xrange[1]-(b-a), c.min()-0.1*(c.max()-c.min()), c.max()+0.1*(c.max()-c.min())])

    # Plot a grid
    ax1.grid(True)

    # Insert lines at x=0 and y=0
    ax1.axhline(0.,linewidth=1, color = 'k', linestyle='dashed')
    ax1.axvline(0.,linewidth=1, color = 'k', linestyle='dashed')

    # Plot function
    ax1.plot(x,c,'b-')

    plt.title("Original function (cos)", fontsize=14,color='black')

    ax1  = fig.add_subplot(412)

    # Axis ranges
    ax1.axis([xrange[0]+(b-a), xrange[1]-(b-a), z.min()-0.1*(z.max()-z.min()), z.max()+0.1*(z.max()-z.min())])

    # Plot a grid
   ...


I am working on the file "2_Mathematical_Groundwork/2_5_convolution.ipynb".
The first task is:
Figure 2.5.3:
Note, that while the signal is not visible in the noisy data, it is partially recovered in the output of the filter.
Representing instrumental functions, it is important to differentiate between the response function in a certain direction and the image of an impulse, which is the reverse of the response function. The function used to represent a measurement via convolution is the image of an impulse function at the origin. This becomes evident, when we convolve two asymmetric functions.
Can you write Python code for it?

from matplotlib import rcParams
rcParams['text.usetex'] = True

def gausshermetian(x, amp, mu, sigma, h3, h4):
    """
    T is the period, sigma is the dispersion, amplitude the amplitude
    """
    y = (x-mu)/sigma
    return amp*np.exp(-0.5*y**2)*(1+h3*(2*np.sqrt(2.)*y**3-3*np.sqrt(2.)*y)/np.sqrt(6.)+h4*(4*y**4-12*y**2+3)/np.sqrt(24))

#amplitude*np.cos(2.*math.pi*x/T)+np.random.normal(scale=sigma, size=x.size)

def boxcar(x,a,b,amp):
    return amp*np.logical_and(x>=a,x<=b)

def plotskewedgaussobs(pos1, pos2, boxwidth, sigma, h3, h4):

    # limits of boxcar Play arround with this
#    a = -0.1
#    b = 0.1

    # Plotting range
    xrange = [-2., 2.]

    # Create functions
    xpoints = 1000

    # Resolution element
    dx = (xrange[1]-xrange[0])/float(xpoints)

    x = np.linspace(xrange[0], xrange[1], xpoints)
    y = boxcar(x, pos1-boxwidth/2., pos1+boxwidth/2, \
               1./boxwidth)+0.5*boxcar(x, pos2-boxwidth/2., pos2+boxwidth/2, 1./boxwidth)

    # boxcar will be normalised to 1. amp = 1./(b-a) works in the limit of many points, but here we do
    # numberofpixelsinbox*dx*amplitude = y.sum *dx*amplitude = 1
    # to take into account numerical effects
    z = gausshermetian(x, 1., 0., sigma, h3, h4)

    result = np.convolve(y,z,'same')
    result = dx*result

    # Start the plot, create a figure instance and a subplot
    fig = plt.figure()

    ax1  = fig.add_subplot(311)
    fig.tight_layout()
    plt.subplots_adjust(hspace = 0.7)

    # Axis ranges
    ax1.axis([xrange[0]+boxwidth, xrange[1]-boxwidth, y.min()-0.1*(y.max()-y.min()), y.max()+0.1*(y.max()-y.min())])

    # Plot a grid
    ax1.grid(True)

    # Insert lines at x=0 and y=0
    ax1.axhline(0.,linewidth=1, color = 'k', linestyle='dashed')
    ax1.axvline(0.,linewidth=1, color = 'k', linestyle='dashed')

    # Plot function
    ax1.plot(x,y,'b-')

    plt.title("Original function, impulse", fontsize=14,color='black')

    ax2  = fig.add_subplot(312, sharex=ax1)

    # Axis ranges
   ...


I am working on the file "2_Mathematical_Groundwork/2_5_convolution.ipynb".
The first task is:
Figure 2.5.3: Filtering a sine wave completely hidden in the noise with a rectangle function.
Note, that while the signal is not visible in the noisy data, it is partially recovered in the output of the filter.
Representing instrumental functions, it is important to differentiate between the response function in a certain direction and the image of an impulse, which is the reverse of the response function. The function used to represent a measurement via convolution is the image of an impulse function at the origin. This becomes evident, when we convolve two asymmetric functions.
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/2_5_convolution.ipynb".
The first task is:
Import section specific modules:
Can you write Python code for it?

import math

from IPython.display import HTML
HTML('../style/code_toggle.html')


I am working on the file "2_Mathematical_Groundwork/2_5_convolution.ipynb".
The first task is:
Note, that while the signal is not visible in the noisy data, it is partially recovered in the output of the filter.
Representing instrumental functions, it is important to differentiate between the response function in a certain direction and the image of an impulse, which is the reverse of the response function. The function used to represent a measurement via convolution is the image of an impulse function at the origin. This becomes evident, when we convolve two asymmetric functions.
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/2_6_cross_correlation_and_auto_correlation.ipynb".
The first task is:
Import section specific modules:
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
2.8. The Discrete Fourier Transform (DFT) and the Fast Fourier Transform (FFT)
The continuous version of the Fourier transform can only be computed when the integrals involved can be evaluated analytically, something which is not always possible in real life applications. This is true for a number of reasons, the most relevant of which are:

We don't always have the parametrisation of the signal that we want to find the Fourier transform of.

Signals are measured and recorded at a finite number of points.

Measured signals are contaminated by noise.

In such cases the discrete equivalent of the Fourier transform, called the discrete Fourier transform (DFT), is very useful. In fact, where the scale of the problem necessitates using a computer to perform calculations, the Fourier transform can only be implemented as the discrete equivalent. There are some subtleties we should be aware of when implementing the DFT. These mainly arise because it is very difficult to capture the full information present in a continuous signal with a finite number of samples. In this chapter we review the DFT and extend some of the most useful identities derived in the previous sections to the case where we only have acces to a finite number of samples. The subtleties that arise due to limited sampling will be discussed in the next section.

The Discrete Fourier transform
The Discrete Time Fourier transform (DTFE): definition
The Discrete Fourier transform (DFT): definition
The Discrete convolution: definition and discrete convolution theorem
Numerically Implementing the DFT
Fast Fourier transforms

2.8.1 The discrete time Fourier transform (DTFT): definition
We start by introducing the discrete time Fourier transform (DTFT). The DTFT of a set $\left{y_n \in \mathbb{C}\right}_{n ~ \in ~ \mathbb{Z}}$ results in a Fourier series (see $\S$ 2.3 ➞) of the form
<a...
Can you write Python code for it?

def loop_DFT(x):
    """
    Implementing the DFT in a double loop
    Input: x = the vector we want to find the DFT of
    """
    #Get the length of the vector (will only work for 1D arrays)
    N = x.size
    #Create vector to store result in
    X = np.zeros(N,dtype=complex)
    for k in range(N):
        for n in range(N):
            X[k] += np.exp(-1j*2.0*np.pi*k*n/N)*x[n]
    return X


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
Althought this would produce the correct result, this way of implementing the DFT is going to be incredibly slow. The DFT can be implemented in matrix form. Convince yourself that a vectorised implementation of this operation can be achieved with
$$ X = K x $$
where $K$ is the kernel matrix, it stores the values $K_{kn} = e^{\frac{-\imath 2 \pi k n}{N}}$. This is implemented numerically as follows
Can you write Python code for it?

def matrix_DFT(x):
    """
    Implementing the DFT in vectorised form
    Input: x = the vector we want to find the DFT of
    """
    #Get the length of the vector (will only work for 1D arrays)
    N = x.size
    #Create vector to store result in
    n = np.arange(N)
    k = n.reshape((N,1))
    K = np.exp(-1j*2.0*np.pi*k*n/N)
    return K.dot(x)


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
Figure 2.8.1: Amplitude and phase plots of the fourier transform of a signal comprised of 3 different tones
It is not immediately obvious that these are the frequencies contained in the signal. However, recall, from the definition given at the outset, that the frequencies are related to the index $k$ via
$$ f_k = \frac{k f_s}{N}, $$
where $f_s$ is the sampling frequency (i.e. one divided by the sampling period). Let's see what happens if we plot the $X_k$ against the $f_k$ using the following bit of code
Can you write Python code for it?

#Get the sampling frequency
delt = t[1] - t[0]
fs = 1.0/delt
k = np.arange(N)
fk = k*fs/N
plt.figure(figsize=(15, 6))
plt.subplot(121)
plt.stem(fk,abs(Y))
plt.xlabel('$f_k$',fontsize=18)
plt.ylabel(r'$|Y_k|$',fontsize=18)
plt.subplot(122)
plt.stem(fk,np.angle(Y))
plt.xlabel('$f_k$',fontsize=18)
plt.ylabel(r'phase$(Y_k)$',fontsize=18)


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
Figure 2.8.2: The fourier transformed signal labled by frequency
Here we see that the three main peaks correspond to the frequencies contained in the input signal viz. $f_1 = 1$Hz, $f_2 = 2$Hz and $f_3 = 3$Hz. But what do the other peaks mean? The additional frequency peaks are a consequence of the following facts:

the DFT of a real valued signal is Hermitian (see Hermitian property of real valued signals ⤵<!--\ref{math:eq:8_009}-->) so that $Y_{-k} = Y_k^*$,
the DFT is periodic with period $N$ (see Periodicity of the DFT ⤵<!--\ref{math:eq:8_007}-->) so that $Y_{k} = Y_{k+N}$. <br>

When used together the above facts imply that $Y_{N-k} = Y_k^*$. This will be important in $\S$ 2.9 ➞ when we discuss aliasing. Note that these additional frequency peaks contain no new information. For this reason it is only necessary to store the first $\frac{N}{2} + 1$ samples when taking the DFT of a real valued signal.
We have not explained some of the features of the signal viz.

Why are there non-zero components of $Y_k$ at frequencies that are not present in the input signal?
Why do the three main peaks not contain the same amount of power? This is a bit unexpected since all three components of the input signal have the same amplitude.

As we will see in $\S$ 2.9 ➞, these features result from the imperfect sampling of the signal. This is unavoidable in any practical application involving the DFT and will be a reoccurring theme throughout this course. You are encouraged to play with the parameters (eg. the minimum $t_{min}$ and maximum $t_{max}$ values of the time coordinate, the number of samples $N$ (do not use $N > 10^5$ points or you might be here for a while), the frequencies of the input components etc.) to get a feel for what does and does not work. In particular try setting the number of samples to $N = 32$ and see if you can explain the output. It might also be a good exercise to and implement the inverse DFT.
We already mentioned that the vectorised version of the DFT above will be...
Can you write Python code for it?

%timeit loop_DFT(x)
%timeit matrix_DFT(x)


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
Great! Our function is returning the correct result. Next we do an example to demonstrate the duality between the spectral (frequency domain) and temporal (time domain) representations of a function. As the following example shows, the Fourier transform of a time series returns the frequencies contained in the signal.
The following code simulates a signal of the form
$$ y = \sin(2\pi f_1 t) + \sin(2\pi f_2 t) + \sin(2\pi f_3 t), $$
takes the DFT and plots the amplitude and phase of the resulting components $Y_k$.
Can you write Python code for it?

#First we simulate a time series as the sum of a number of sinusoids each with a different frequency
N = 512  #The number of samples of the time series
tmin = -10 #The minimum value of the time coordinate
tmax = 10 #The maximum value of the time coordinate
t = np.linspace(tmin,tmax,N) #The time coordinate
f1 = 1.0 #The frequency of the first sinusoid
f2 = 2.0 #The frequency of the second sinusoid
f3 = 3.0 #The frequency of the third sinusoid
#Generate the signal
y = np.sin(2.0*np.pi*f1*t) + np.sin(2.0*np.pi*f2*t) + np.sin(2.0*np.pi*f3*t)
#Take the DFT
Y = matrix_DFT(y)
#Plot the absolute value, real and imaginary parts
plt.figure(figsize=(15, 6))
plt.subplot(121)
plt.stem(abs(Y))
plt.xlabel('$k$',fontsize=18)
plt.ylabel(r'$|Y_k|$',fontsize=18)
plt.subplot(122)
plt.stem(np.angle(Y))
plt.xlabel('$k$',fontsize=18)
plt.ylabel(r'phase$(Y_k)$',fontsize=18)


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
Here we see that the three main peaks correspond to the frequencies contained in the input signal viz. $f_1 = 1$Hz, $f_2 = 2$Hz and $f_3 = 3$Hz. But what do the other peaks mean? The additional frequency peaks are a consequence of the following facts:

the DFT of a real valued signal is Hermitian (see Hermitian property of real valued signals ⤵<!--\ref{math:eq:8_009}-->) so that $Y_{-k} = Y_k^*$,
the DFT is periodic with period $N$ (see Periodicity of the DFT ⤵<!--\ref{math:eq:8_007}-->) so that $Y_{k} = Y_{k+N}$. <br>

When used together the above facts imply that $Y_{N-k} = Y_k^*$. This will be important in $\S$ 2.9 ➞ when we discuss aliasing. Note that these additional frequency peaks contain no new information.
We have not explained some of the features of the signal viz.

Why are there non-zero components of $Y_k$ at frequencies that are not present in the input signal?
Why do the three main peaks not contain the same amount of power? This is a bit unexpected since all three components of the input signal have the same amplitude.

As we will see in $\S$ 2.9 ➞, these features result from the imperfect sampling of the signal. This is unavoidable in any practical application involving the DFT and will be a reoccurring theme throughout this course. You are encouraged to play with the parameters (eg. the minimum $t_{min}$ and maximum $t_{max}$ values of the time coordinate, the number of samples $N$ (do not use $N > 10^5$ points or you might be here for a while), the frequencies of the input components etc.) to get a feel for what does and does not work. In particular try setting the number of samples to $N = 32$ and see if you can explain the output. It might also be a good exercise to and implement the inverse DFT.
We already mentioned that the vectorised version of the DFT above will be much faster than the loop version. We can see exactly how much faster with the following commands
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
Import section specific modules:
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
It is not immediately obvious that these are the frequencies contained in the signal. However, recall, from the definition given at the outset, that the frequencies are related to the index $k$ via
$$ f_k = \frac{k f_s}{N}, $$
where $f_s$ is the sampling frequency (i.e. one divided by the sampling period). Let's see what happens if we plot the $X_k$ against the $f_k$ using the following bit of code
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
Just to be sure our DFT really works, let's also compare the output of our function to numpy's built in DFT function (note numpy automatically implements a faster version of the DFT called the FFT, see the discussion below)
Can you write Python code for it?

x = np.random.random(256)  #create random vector to take the DFT of
np.allclose(np.fft.fft(x),matrix_DFT(x)) #compare the result using numpy's built in function


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
Lets confirm that this function returns the correct result by comparing fith numpy's FFT.
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
That is almost a factor of ten difference. Lets compare this to numpy's built in FFT
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
That seems amazing! The numpy FFT is about 1000 times faster than our vectorised implementation. But how does numpy achieve this speed up? Well, by using the fast Fourier transform of course.
2.8.6. Fast Fourier transforms
The DFT is a computationally expensive operation. As evidenced by the double loop required to implement the DFT the computational complexity of a naive implementation such as ours scales like $\mathcal{O}(N^2)$ where $N$ is the number of data points. Even a vectorised version of the DFT will scale like $\mathcal{O}(N^2)$ since, in the end, there are still the same number of complex exponentiations and multiplications involved.
By exploiting the symmetries of the DFT, it is not difficult to identify potential ways to safe computing time. Looking at the definition of the discrete Fourier transform discrete Fourier transform ⤵, one can see that, under certain circumstances, the same summands occur multiple times. Recall that the DFT is periodic i.e. $Y_k = Y_{N+k}$, where $N$ is the number of data points. Now suppose that $N = 8$. In calculating the component $Y_2$ we would have to compute the quantity $y_2\,e^{-2{\pi}\imath\frac{2 \cdot 2}{8}}$ i.e. when $n = 2$. However, using the periodicity of the kernel $e^{-2\pi\imath \frac{kn}{N}} = e^{-2\pi\imath \frac{k(n+N)}{N}}$, we can see that this same quantity will also have to be computed when calculating the component $Y_6$ since $y_2\,e^{-2{\pi}\imath\frac{2\cdot2}{8}}=y_2e^{-2{\pi}\imath\frac{6\cdot2}{8}} = y_2e^{-2{\pi}\imath\frac{12}{8}}$. If we were calculating the DFT by hand, it would be a waste of time to calculate this summand twice. To see how we can exploit this, lets first split the DFT into its odd and even $n$ indices as follows
\begin{eqnarray}
Y_{k} &=& \sum_{n = 0}^{N-1} y_n e^{-2\pi\imath \frac{kn}{N}}\
 &=& \sum_{m = 0}^{N/2-1} y_{2m} e^{-2\pi\imath \frac{k(2m)}{N}} + \sum_{m = 0}^{N/2-1} y_{2m+1} e^{-2\pi\imath \frac{k(2m+1)}{N}}\
 &=& \sum_{m =...
Can you write Python code for it?

def one_layer_FFT(x):
    """An implementation of the 1D Cooley-Tukey FFT using one layer"""
    N = x.size
    if N%2>0:
        print "Warning: length of x in not a power of two, returning DFT"
        return matrix_DFT(x)
    else:
        X_even = matrix_DFT(x[::2])
        X_odd = matrix_DFT(x[1::2])
        factor = np.exp(-2j * np.pi * np.arange(N) / N)
        return np.concatenate([X_even + factor[:N / 2] * X_odd,X_even + factor[N / 2:] * X_odd])


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
That seems amazing! The numpy FFT is about 1000 times faster than our vectorised implementation. But how does numpy achieve this speed up? Well, by using the fast Fourier transform of course.
BVH:AC:Point out that when taking a fourier transform of a real-valued signal it is only necessary to store the first N/2 + 1 samples.
2.8.5. Fast Fourier transforms
The DFT is a computationally expensive operation. As evidenced by the double loop required to implement the DFT the computational complexity of a naive implementation such as ours scales like $\mathcal{O}(N^2)$ where $N$ is the number of data points. Even a vectorised version of the DFT will scale like $\mathcal{O}(N^2)$ since, in the end, there are still the same number of complex exponentiations and multiplications involved.
By exploiting the symmetries of the DFT, it is not difficult to identify potential ways to safe computing time. Looking at the definition of the discrete Fourier transform discrete Fourier transform ⤵, one can see that, under certain circumstances, the same summands occur multiple times. Recall that the DFT is periodic i.e. $Y_k = Y_{N+k}$, where $N$ is the number of data points. Now suppose that $N = 8$. In calculating the component $Y_2$ we would have to compute the quantity $y_2\,e^{-2{\pi}\imath\frac{2 \cdot 2}{8}}$ i.e. when $n = 2$. However, using the periodicity of the kernel $e^{-2\pi\imath \frac{kn}{N}} = e^{-2\pi\imath \frac{k(n+N)}{N}}$, we can see that this same quantity will also have to be computed when calculating the component $Y_6$ since $y_2\,e^{-2{\pi}\imath\frac{2\cdot2}{8}}=y_2e^{-2{\pi}\imath\frac{6\cdot2}{8}} = y_2e^{-2{\pi}\imath\frac{12}{8}}$. If we were calculating the DFT by hand, it would be a waste of time to calculate this summand twice. To see how we can exploit this, lets first split the DFT into its odd and even $n$ indices as follows
\begin{eqnarray}
Y_{k} &=& \sum_{n = 0}^{N-1} y_n...
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
This function will be much faster than the previous implementation. We should check that they both return the same result
Can you write Python code for it?

x = np.random.random(256)  #create random vector to take the DFT of
np.allclose(loop_DFT(x),matrix_DFT(x)) #compare the result using numpy's built in function


I am working on the file "2_Mathematical_Groundwork/2_y_exercises.ipynb".
The first task is:
2.y. Exercises
We provide a small set of exercises suitable for an interferometry course.
2.y.1. Fourier transforms and convolution: Fourier transform of the triangle function
Consider the triangle function given below.
Can you write Python code for it?

def plotviewgraph(fig, ax, xmin = 0, xmax = 1., ymin = 0., ymax = 1.):
    """
    Prepare a viewvgraph for plotting a function

    Parameters:
    fig:          Matplotlib figure
    ax:           Matplotlib subplot
    xmin (float): Minimum of range
    xmax (float): Maximum of range
    ymin (float): Minimum of function
    ymax (float): Maximum of function

    return: axis and vertical and horizontal tick length
    """

    # Axis ranges
    ax.axis([xmin-0.1*(xmax-xmin), xmax+0.1*(xmax-xmin), -0.2*(ymax-ymin), ymax])
    ax.axis('off')

    # get width and height of axes object to compute, see https://3diagramsperpage.wordpress.com/2014/05/25/arrowheads-for-axis-in-matplotlib/

    # matching arrowhead length and width
    dps = fig.dpi_scale_trans.inverted()
    bbox = ax.get_window_extent().transformed(dps)
    width, height = bbox.width, bbox.height

    # manual arrowhead width and length
    hw = 1./15.*(ymax-ymin)
    hl = 1./30.*(xmax-xmin)
    lw = 1. # axis line width
    ohg = 0.3 # arrow overhang

    # compute matching arrowhead length and width
    yhw = hw/(ymax-ymin)*(xmax-xmin)* height/width
    yhl = hl/(xmax-xmin)*(ymax-ymin)* width/height

    # Draw arrows
    ax.arrow(xmin-0.1*(xmax-xmin),0, 1.2*(xmax-xmin),0, fc='k', ec='k', lw = lw,
         head_width=hw, head_length=hl, overhang = ohg,
         length_includes_head= True, clip_on = False)
    ax.arrow(0,ymin-0.1*(ymax-ymin), 0., 1.4*(ymax-ymin), fc='k', ec='k', lw = lw,
         head_width=yhw, head_length=yhl, overhang = ohg,
         length_includes_head= True, clip_on = False)

    # Draw ticks for A, -A, and B
    twv = 0.01*height # vertical tick width
    twh = twv*(xmax-xmin)/(ymax-ymin)/ width*height

    return twv, twh

def plottriangle():

    A = 1.
    B = 1.

    # Start the plot, create a figure instance and a subplot
    fig = plt.figure(figsize=(20,5))
    ax  = fig.add_subplot(111)

    twv, twh = plotviewgraph(fig, ax, xmin = -A, xmax = A, ymin = 0., ymax = B)

   ...


I am working on the file "2_Mathematical_Groundwork/2_y_exercises.ipynb".
The first task is:
Figure 2.y.1: Triangle function with width $2A$ and amplitude $B$.
Assignments:

What can you tell about the complex part of the Fourier transform of $f$ using the symmetry of the function?

Write down the function $f$ in two ways, once as a piece-wise defined function, once as a convolution of the rectangle function with itself.

Calculate the Fourier transform, making use of expressing f as a convolution of a boxcar function with itself and using the convolution theorem.

2.y.1.1 Fourier transform of the triangle function: example answer to assignment 1.
What can you tell about the complex part and the symmetry of the Fourier transform of $f$ using the symmetry of the function?
The function is real-valued ($f^(x)\,=\,f(x)$) and even ($f(x)\,=\,f(-x)$), so it is Hermetian ($f^(x)\,=\,f(-x)$, see definition here ➞ ). According to Sect. 2.4.6 ➞, this means that the Fourier transform is a real-valued function (because it is the Fourier transform of a Hermetian function) and also Hermetian (because it is the Fourier transform of a real-valued function). Hence it is also even ($f^(x)\,=\,f(x) \,\land\, f^(x)\,=\,f(-x)\,\Rightarrow\,f(x)\,=\,f(-x)$). Real-valued means that the complex part of $f$ is $0$.
2.y.1.2 Fourier transform of the triangle function: example answer to assignment 2.
Write down the function $f$ in two ways, once as a piece-wise defined function, once as a convolution of the rectangle function with itself.
Part one is straightforward:
<a...
Can you write Python code for it?

def plotfftriangle():

    A = 1.
    B = 1.

    # Start the plot, create a figure instance and a subplot
    fig = plt.figure(figsize=(20,5))
    ax  = fig.add_subplot(111)

    twv, twh = plotviewgraph(fig, ax, xmin = -3./A, xmax = 3./A, ymin = -0.3, ymax = B)
    ticx = [[-A,r'$-\frac{1}{A}$'],[A,'A']]

    ticx = [[-3.*A, r'$\frac{-3}{A}$'], [-2.*A, r'$\frac{-2}{A}$'], [-1./A, r'$\frac{-1}{A}$'], [1./A, r'$\frac{1}{A}$'], [2./A, r'$\frac{2}{A}$'], [3./A, r'$\frac{3}{A}$']]
    for tupel in ticx:
        ax.plot([tupel[0],tupel[0]],[-twv, twv], 'k-')
        ax.text(tupel[0], 0.-2.*twh, tupel[1], fontsize = 24, horizontalalignment = 'center', verticalalignment = 'top', color = 'black')

    ticx = [[0.,r'$0$']]
    for tupel in ticx:
        ax.plot([tupel[0],tupel[0]],[-twv, twv], 'k-')
        ax.text(tupel[0]+twh, 0.-2.*twh, tupel[1], fontsize = 24, horizontalalignment = 'left', verticalalignment = 'top', color = 'black')

    ticy = [[B,r'$\frac{B}{A}$']]
    for tupel in ticy:
        ax.plot([-twh, twh], [tupel[0], tupel[0]], 'k-')
        ax.text(0.+twv, tupel[0], tupel[1], fontsize = 24, horizontalalignment = 'left', verticalalignment = 'bottom', color = 'black')

    # Plot the function
    x = np.linspace(-4.*A, 4.*A, 900)
    y = np.power(np.sinc(x),2)

    # Annotate axes
    ax.text(0.-A/20, 1.2*(B), r'$f(x)$', fontsize = 24, horizontalalignment = 'right', verticalalignment = 'bottom', color = 'black')
    ax.text(1.2*3.*A, 0., r'$x$', fontsize = 24, horizontalalignment = 'left', verticalalignment = 'top', color = 'black')

    ax.plot(x, y, 'r-', lw = 2)

plotfftriangle()
#


I am working on the file "2_Mathematical_Groundwork/2_y_exercises.ipynb".
The first task is:
Figure 2.y.2: Triangle function with width $2A$ and amplitude $B$.
2.y.2. Fourier transforms and convolution: Convolution of two functions with finite support
Consider the two functions given below:
Can you write Python code for it?

def plotrectntria():

    A = 1.
    B = 1.4

    # Start the plot, create a figure instance and a subplot
    fig = plt.figure(figsize=(20,5))
    ax  = fig.add_subplot(121)

    twv, twh = plotviewgraph(fig, ax, xmin = 0., xmax = 3.*A, ymin = 0., ymax = 3.)

    ticx = [[1.*A, r'$A$'], [2.*A, r'$2A$'], [3.*A, r'$3A$']]
    for tupel in ticx:
        ax.plot([tupel[0],tupel[0]],[-twv, twv], 'k-')
        ax.text(tupel[0], 0.-2.*twh, tupel[1], fontsize = 24, horizontalalignment = 'center', verticalalignment = 'top', color = 'black')

    ticx = [[0.,r'$0$']]
    for tupel in ticx:
        ax.plot([-tupel[0],-tupel[0]],[-twv, twv], 'k-')
        ax.text(tupel[0]+twh, 0.-2.*twh, tupel[1], fontsize = 24, horizontalalignment = 'left', verticalalignment = 'top', color = 'black')

    ticy = [[1,r'$1$'], [2.,r'$2$'], [3.,r'$3$']]
    for tupel in ticy:
        ax.plot([-twh, twh], [tupel[0], tupel[0]], 'k-')
        ax.text(0.-twv, tupel[0], tupel[1], fontsize = 24, horizontalalignment = 'right', verticalalignment = 'center', color = 'black')

    ticy = [[B, r'$B$']]
    for tupel in ticy:
        ax.plot([-twh, twh], [tupel[0], tupel[0]], 'k-')
        ax.text(0.+twv, tupel[0], tupel[1], fontsize = 24, horizontalalignment = 'left', verticalalignment = 'bottom', color = 'black')

        # Plot the function
    x = [A, A, 2*A, 2*A]
    y = [0., B, B, 0.]
    ax.plot(x, y, 'r-', lw = 2)

    x = [0., A]
    y = [B, B]
    ax.plot(x, y, 'k--', lw = 1)

    # Annotate axes
    ax.text(0.-3.*twh, 1.2*3., r'$g(x)$', fontsize = 24, horizontalalignment = 'right', verticalalignment = 'bottom', color = 'black')
    ax.text(1.1*3.*A, 0., r'$x$', fontsize = 24, horizontalalignment = 'left', verticalalignment = 'top', color = 'black')

    ###################

    ax  = fig.add_subplot(122)

    twv, twh = plotviewgraph(fig, ax, xmin = 0., xmax = 3.*A, ymin = 0., ymax = 3.)

    ticx = [[1.*A, r'$A$'], [2.*A, r'$2A$'], [3.*A, r'$3A$']]
    for tupel in ticx:
       ...


I am working on the file "2_Mathematical_Groundwork/2_y_exercises.ipynb".
The first task is:
Figure 2.y.3: Triangle function with width $2A$ and amplitude $B$.
Assignments:

Write down the functions g and h.

Calculate their convolution.

2.y.2.1 Convolution of two functions with finite support: example answer to assignment 1.
Write down the functions g and h.
$$
\begin{align}
h(x)   &= \left {
     \begin{array}{lll}
    B & {\rm for} & A \leq x \leq 2A\
    0 & {\rm else}
\end{array}\right .\
g(x)   &= \left {
     \begin{array}{lll}
    g_1(x)\,=\,\frac{2}{A}\left(x-\frac{A}{2}\right) & {\rm for} & A \leq x \leq 2A\
    g_2(x)\,=\,-\frac{2}{A}\left(x-\frac{7A}{2}\right) & {\rm for} & 2A \leq x \leq 3A\
    0 & {\rm else}
\end{array}\right .\
\end{align}
$$
2.y.2.2 Convolution of two functions with finite support: example answer to assignment 2.
We have to evaluate the integral (see definition of the convolution ➞ ):
$$
g\circ h(x) \, = \, \int_{-\infty}^{\infty}g(x-t)h(t)\,dt
$$
To do so, we calculate the integral for ranges of $x$, depending on the supports (ranges where the function in non-zero) of $g(x-t)$ and $h(t)$, or $h_1(t)$ and $g_2(t)$ respectively.
As an aid, rewrite above functions ⤵:
$$
\begin{align}
g(x-t)   &= \left {
     \begin{array}{lll}
    B & {\rm for} & -2A+x \leq t \leq -A+x\
    0 & {\rm else}
\end{array}\right .\
h(t)   &= \left {
    ...
Can you write Python code for it?

def rectntriaconv(A,B,x):

    xn = x[x < (2*A)]
    yn = xn*0.
    y = yn

    xn = x[(x == 2*A) | (x > 2*A) & (x < 3*A)]
    yn = (B/A)*(np.power(xn,2)-3*A*xn+2*np.power(A,2))
    y = np.append(y,yn)

    xn = x[(x == 3*A) | (x > 3*A) & (x < 4*A)]
    yn = (B/A)*((-2*np.power(xn,2))+14*A*xn-22*np.power(A,2))
    y = np.append(y,yn)

    xn = x[(x == 4*A) | (x > 4*A) & (x < 5*A)]
    yn = (B/A)*(np.power(xn,2)-11*A*xn+30*np.power(A,2))
    y = np.append(y,yn)

    xn = x[(x == 5*A) | (x > 5*A)]
    yn = xn*0.
    y = np.append(y,yn)

    return y

def plotrectntriaconv():
    A = 1.
    B = 1.4

    # Start the plot, create a figure instance and a subplot
    fig = plt.figure(figsize=(20,5))
    ax  = fig.add_subplot(121)

    twv, twh = plotviewgraph(fig, ax, xmin = 0., xmax = 6.*A, ymin = 0., ymax = 2.5*A*B)

    ticx = [[1.*A, r'$A$'], [2.*A, r'$2A$'], [3.*A, r'$3A$'], [4.*A, r'$4A$'], [5.*A, r'$5A$'], [6.*A, r'$6A$']]
    for tupel in ticx:
        ax.plot([tupel[0],tupel[0]],[-twv, twv], 'k-')
        ax.text(tupel[0], 0.-2.*twh, tupel[1], fontsize = 24, horizontalalignment = 'center', verticalalignment = 'top', color = 'black')

    ticx = [[0.,r'$0$']]
    for tupel in ticx:
        ax.plot([-tupel[0],-tupel[0]],[-twv, twv], 'k-')
        ax.text(tupel[0]+twh, 0.-2.*twh, tupel[1], fontsize = 24, horizontalalignment = 'left', verticalalignment = 'top', color = 'black')

    ticy = [[2*A*B, r'$2AB$'], [2.5*A*B, r'$\frac{5}{2}AB$']]
    for tupel in ticy:
        ax.plot([-twh, twh], [tupel[0], tupel[0]], 'k-')
        ax.text(0.+5*twv, tupel[0], tupel[1], fontsize = 24, horizontalalignment = 'left', verticalalignment = 'bottom', color = 'black')

    # Plot the function
    x = np.linspace(0., 7.*A, 900)
    y = rectntriaconv(A,B,x)
    ax.plot(x, y, 'r-', lw = 2)

    # Plot a few lines
    x = [0., 4*A]
    y = [2.*A*B, 2.*A*B]
    ax.plot(x, y, 'k--', lw = 1)

    x = [0., 3.5*A]
    y = [2.5*A*B, 2.5*A*B]
    ax.plot(x, y, 'k--', lw = 1)

    x =...


I am working on the file "2_Mathematical_Groundwork/fft_implementation_assignment.ipynb".
The first task is:
A non-$2^n$ FFT (10 points)
Now that we have implemented a fast radix-2 algorithm for vectors of length $2^n$, we can write a generic algorithm which can take any length input. This algorithm will check if the length of the input is divisible by 2, if so then it will use the FFT, otherwise it will default to the slower matrix-based DFT.
Can you write Python code for it?

def generalFFT(x):
    """radix-2 DIT FFT
    x: list or array of N values to perform FFT on, can be real or imaginary
    """
    ox = np.asarray(x, dtype='complex') # assure the input is an array of complex values
    # INSERT: assign a value to N, the size of the FFT
    N = #??? 1 point

    if N==1: return ox # base case
    elif # INSERT: check if the length is divisible by 2, 1 point
    elif N % 2 ==0: # the length of the input vector is divisable by 2

        # INSERT: do a FFT, use your ditrad2() code here, 3 points
        # Hint: your ditrad2() code can be copied here, and will work with only a minor modification

    else: # INSERT: if not divisable by 2, do a slow Fourier Transform
        return # ??? 1 point


I am working on the file "2_Mathematical_Groundwork/fft_implementation_assignment.ipynb".
The first task is:
As we can see the matrix DFT is significatly faster than the double loop DFT, this is because of the fast vectorization functions in numpy. And, the 'one-layer' FFT is about twice as fast as the matrix DFT because of the FFT architecture. We can go one fast and use the built-in numpy FFT:
Can you write Python code for it?

print np.allclose(one_layer_FFT(xTest), np.fft.fft(xTest))

print 'numpy FFT:'
%timeit np.fft.fft(xTest)


print np.allclose(one_layer_FFT(x), np.fft.fft(x))

print 'numpy FFT:'
%timeit np.fft.fft(x)


I am working on the file "2_Mathematical_Groundwork/fft_implementation_assignment.ipynb".
The first task is:
In $\S$ 2.8.6 ➞ the fast Fourier transform was introduced as using recursion to implement a Fourier transform in $\mathcal{O}(N\log_2N)$ computations, significantly reducing the computational cost of computing the Fourier transform, especially for large $N$. A 'one layer' fast Fourier transform was presented which split the input function into two, and applied the twiddle factor to all values in the layer before calling the matrix-based DFT. This code is replicated below.
Can you write Python code for it?

def one_layer_FFT(x):
    """An implementation of the 1D Cooley-Tukey FFT using one layer"""
    N = x.size
    if N%2 > 0:
        print "Warning: length of x is not a power of two, returning DFT"
        return matrix_DFT(x)
    else:
        X_even = matrix_DFT(x[::2])
        X_odd = matrix_DFT(x[1::2])
        factor = np.exp(-2j * np.pi * np.arange(N) / N)
        return np.concatenate([X_even + factor[:N / 2] * X_odd, X_even + factor[N / 2:] * X_odd])


I am working on the file "2_Mathematical_Groundwork/fft_implementation_assignment.ipynb".
The first task is:
Now running this algorithm on inputs of different lengths there should be different run times. For a vector with a prime number length then the algorithm will default to the slow matrix-based DFT. For a vector of length nearly always divisible by 2 then the algorithm should be faster.
Can you write Python code for it?

xTest2 = np.random.random(251)  # create random vector to take the DFT of, not, this is not of length 2^n
xTest3 = np.random.random(12*32)  # create random vector to take the DFT of, not, this is not of length 2^n

print 'The output of generalFFT() is correct?', np.allclose(np.fft.fft(xTest2), generalFFT(xTest2)) # 1 point

print 'Your generic FFT:'
%timeit generalFFT(xTest2) # 1 point if it runs in approximately the same time as matrix_DFT

%timeit generalFFT(xTest3) # 2 point if it runs faster than the xTest2 vector


I am working on the file "2_Mathematical_Groundwork/fft_implementation_assignment.ipynb".
The first task is:
Once ditrad2() is properly implemented then the results of calling the function should be equivalent to the output of the numpy FFT, and should run faster than the DFT and one-layer FFT.
Can you write Python code for it?

print 'The output of ditrad2() is correct?', np.allclose(np.fft.fft(xTest), ditrad2(xTest)) # 2 points if true

print 'your FFT:'
%timeit ditrad2(xTest) # 2 point if your time < One Layer FFT + Matrix DFT


I am working on the file "2_Mathematical_Groundwork/fft_implementation_assignment.ipynb".
The first task is:
The numpy FFT is very fast, in part because of the low-level programing implementation, but fundamentally because it uses an FFT architecture. Our goal for this assignment is to implement such an architecture.
Decimation-in-Time (DIT) FFT (12 Points)
The computational efficiency of the FFT comes from the recursive design of the algorithm which takes advantage of a binary tree design and the use of generalized twiddle factors. There are two designs of the binary tree which leads to the decimation-in-time (DIT) and decimation-in-frequency (DIF) architectures. Both architectures produce equivalent results, they they differ in the direction and starting point of the computations on the FFT binary tree. See the wikipedia page on the Cooley-Tukey FFT ⤴ for a diagram and pseudo-code of the DIT implementation.
For this section of the assignment implement the Radix-2 DIT FFT algorithm for the case of a $2^n$ size input, this input can be either real or complex.
Can you write Python code for it?

def ditrad2(x):
    """radix-2 DIT FFT
    x: list or array of N values to perform FFT on, can be real or imaginary, x must be of size 2^n
    """
    ox = np.asarray(x, dtype='complex') # assure the input is an array of complex values
    # INSERT: assign a value to N, the size of the FFT
    N = #??? 1 point

    if N==1: return ox # base case

    # INSERT: compute the 'even' and 'odd' components of the FFT,
    # you will recursively call ditrad() here on a subset of the input values
    # Hint: a binary tree design splits the input in half
    even = #??? 2 points
    odd = #??? 2 points

    twiddles = np.exp(-2.j * cmath.pi * np.arange(N) / N) # compute the twiddle factors

    # INSERT: apply the twiddle factors and return the FFT by combining the even and odd values
    # Hint: twiddle factors are only applied to the odd values
    # Hint: combing even and odd is different from the way the inputs were split apart above.
    return #??? 3 points


I am working on the file "2_Mathematical_Groundwork/fft_implementation_assignment.ipynb".
The first task is:
This assignment is to implement a python-based Fast Fourier Transform (FFT). Building on $\S$ 2.8 ➞ we will implement a 1-D radix-2 Cooley-Tukey-based FFT using both decimation in time (DIT) and decimation in frequency (DIF) for an $N = 2^n$ input function.
From $\S$ 2.8.2 ➞ the discrete Fourier transform (DFT) is defined as:
$$ \mathscr{F}{\rm D}{y}_k = Y_k =  \sum{n\,=\,0}^{N-1} y_n\,e^{-\imath 2\pi \frac{nk}{N}}, $$
That is, the $k^{th}$ element of the Fourier transformed spectrum $Y$ is a sum over all $n$ elements of the function $y$, each multipled by a complex twiddle factor $e^{-\imath 2\pi \frac{nk}{N}}$. In $\S$ 2.8.5 ➞ two methods for computing the DFT for a size $N = 2^n$ discrete function. A double loop to compute all elements of the Fourier-transformed spectrum, and a matrix multiplication by generating the Fourier kernel $K$. The compute time to perform the DFT is $\mathcal{O}(N^2)$, this is it takes $cN^2$ operations where $c > 1$ is a constant factor. Though as note in $\S$ 2.8.5 ➞ the matrix implementation is much fast that the loop because this algorithm takes advantage of fast vector math libraries.
The DFT code is replicated here as it will be used to compare our implementation of the FFT:
Can you write Python code for it?

def loop_DFT(x):
    """
    Implementing the DFT in a double loop
    Input: x = the vector we want to find the DFT of
    """
    #Get the length of the vector (will only work for 1D arrays)
    N = x.size
    #Create vector to store result in
    X = np.zeros(N, dtype=complex)
    for k in range(N):
        for n in range(N):
            X[k] += np.exp(-1j * 2.0* np.pi* k * n / N) * x[n]
    return X

def matrix_DFT(x):
    """
    Implementing the DFT in vectorised form
    Input: x = the vector we want to find the DFT of
    """
    #Get the length of the vector (will only work for 1D arrays)
    N = x.size
    #Create vector to store result in
    n = np.arange(N)
    k = n.reshape((N,1))
    K = np.exp(-1j * 2.0 * np.pi * k * n / N)
    return K.dot(x)


I am working on the file "2_Mathematical_Groundwork/fft_implementation_assignment.ipynb".
The first task is:
This assignment is to implement a python-based Fast Fourier Transform (FFT). Building on $\S$ 2.8 ➞ we will implement a 1-D radix-2 Cooley-Tukey-based FFT using decimation in time (DIT) an $N = 2^n$ input function, and then generalize the function to take any input.
From $\S$ 2.8.2 ➞ the discrete Fourier transform (DFT) is defined as:
$$ \mathscr{F}{\rm D}{y}_k = Y_k =  \sum{n\,=\,0}^{N-1} y_n\,e^{-\imath 2\pi \frac{nk}{N}}, $$
That is, the $k^{th}$ element of the Fourier transformed spectrum $Y$ is a sum over all $n$ elements of the function $y$, each multipled by a complex twiddle factor $e^{-\imath 2\pi \frac{nk}{N}}$. In $\S$ 2.8.5 ➞ two methods for computing the DFT for a size $N = 2^n$ discrete function. A double loop to compute all elements of the Fourier-transformed spectrum, and a matrix multiplication by generating the Fourier kernel $K$. The compute time to perform the DFT is $\mathcal{O}(N^2)$, this is it takes $cN^2$ operations where $c > 1$ is a constant factor. Though as note in $\S$ 2.8.5 ➞ the matrix implementation is much fast that the loop because this algorithm takes advantage of fast vector math libraries.
The DFT code is replicated here as it will be used to compare our implementation of the FFT:
Can you write Python code for it?


I am working on the file "2_Mathematical_Groundwork/fft_implementation_assignment.ipynb".
The first task is:
We can also time each function to report of the amount of time is takes to return a finished spectrum.
Can you write Python code for it?

print 'Double Loop DFT:'
%timeit loop_DFT(x)
print '\nMatrix DFT:'
%timeit matrix_DFT(x)
print '\nOne Layer FFT + Matrix DFT:'
%timeit one_layer_FFT(x)


I am working on the file "2_Mathematical_Groundwork/fft_implementation_assignment.ipynb".
The first task is:
We can easily show that each of these functions produce the same results by introducting a discrete test function $x$ and showing that the same results are reported by each function call:
Can you write Python code for it?

xTest = np.random.random(256)  # create random vector to take the DFT of

print np.allclose(loop_DFT(xTest), matrix_DFT(xTest)) # returns True if all values are equal (within numerical error)
print np.allclose(matrix_DFT(xTest), one_layer_FFT(xTest)) # returns True if all values are equal (within numerical error)


I am working on the file "2_Operations_on_Numpy_Arrays.ipynb".
The first task is:
Apply User Defined Functions
You can also apply your own functions on arrays. For e.g. applying the function x/(x+1) to each element of an array.
One way to do that is by looping through the array, which is the non-numpy way. You would rather want to write vectorised code.
The simplest way to do that is to vectorise the function you want, and then apply it on the array. Numpy provides the np.vectorize() method to vectorise functions.
Let's look at both the ways to do it.
Can you write Python code for it?

print(a)

# The non-numpy way, not recommended
a_list = [x/(x+1) for x in a]
print(a_list)

# The numpy way: vectorize the function, then apply it
f = np.vectorize(lambda x: x/(x+1))
f(a)

# Apply function on a 2-d array: Applied to each element
b = np.linspace(1, 100, 10)
f(b)


I am working on the file "2_Operations_on_Numpy_Arrays.ipynb".
The first task is:
Similarly, two arrays having the same number of rows can be horizontally stacked using np.hstack((a, b)).
Perform Operations on Arrays
Performing mathematical operations on arrays is extremely simple. Let's see some common operations.
Basic Mathematical Operations
NumPy provides almost all the basic math functions - exp, sin, cos, log, sqrt etc. The function is applied to each element of the array.
Can you write Python code for it?

# Basic mathematical operations
a = np.arange(1, 20)

# sin, cos, exp, log
print(np.sin(a))
print(np.cos(a))
print(np.exp(a))
print(np.log(a))


I am working on the file "2_Operations_on_Numpy_Arrays.ipynb".
The first task is:
Stacking and Splitting Arrays
Stacking: np.hstack() and n.vstack()
Stacking is done using the np.hstack() and np.vstack() methods. For horizontal stacking, the number of rows should be the same, while for vertical stacking, the number of columns should be the same.
Can you write Python code for it?

# Creating two arrays
array_1 = np.arange(12).reshape(3, 4)
array_2 = np.arange(20).reshape(5, 4)

print(array_1)
print("\n")
print(array_2)

# vstack
# Note that np.vstack(a, b) throws an error - you need to pass the arrays as a list
np.vstack((array_1, array_2))


I am working on the file "2_Operations_on_Numpy_Arrays.ipynb".
The first task is:
This also has the advantage that you can vectorize the function once, and then apply it as many times as needed.
Apply Basic Linear Algebra Operations
NumPy provides the np.linalg package to apply common linear algebra operations, such as:
* np.linalg.inv: Inverse of a matrix
* np.linalg.det: Determinant of a matrix
* np.linalg.eig: Eigenvalues and eigenvectors of a matrix
Also, you can multiple matrices using np.dot(a, b).
Can you write Python code for it?

# np.linalg documentation
help(np.linalg)

# Creating arrays
a = np.arange(1, 10).reshape(3, 3)
b= np.arange(1, 13).reshape(3, 4)
print(a)
print(b)

# Inverse
np.linalg.inv(a)

# Determinant
np.linalg.det(a)

# Eigenvalues and eigenvectors
np.linalg.eig(a)

# Multiply matrices
np.dot(a, b)


I am working on the file "2_Operations_on_Numpy_Arrays.ipynb".
The first task is:
array.T returns the transpose of an array.
Can you write Python code for it?


I am working on the file "2_Oplossen_van_sudoku.ipynb".
The first task is:
Toepassen van de regels
Bij ieder element in de sudoku waar meer dan 1 optie staat, willen we wegstrepen wat niet mag
Can you write Python code for it?

print(sudoku[1])
rij[1]

[s for s, r in zip(sudoku, rij) if r == rij[1]]

for s, r in zip(sudoku, rij):
    if r == rij[1] and len(s) == 1:
        sudoku[1] -= s
sudoku[1]

for s1, r1, c1 ,b1 in zip(sudoku,rij,kolom,blok):
    if len(s1) == 1:
        continue

    for s2, r2, c2 ,b2 in zip(sudoku,rij,kolom,blok):
        if s1 is s2:
            continue

        if len(s2) == 1:
            if r1 == r2 or c1 == c2 or b1 == b2:
                s1 -= s2

print_sudoku(sudoku)


I am working on the file "2_Plotting_Distributions.ipynb".
The first task is:
Histograms and Density Plots
Histograms and density plots show the frequency of a numeric variable along the y-axis, and the value along the x-axis. The sns.distplot() function plots a density curve. Notice that this is aesthetically better than vanilla matplotlib.
Can you write Python code for it?

# simple density plot
sns.distplot(df['Shipping_Cost'])
plt.show()


I am working on the file "2_Plotting_Distributions.ipynb".
The first task is:
Notice that both the distributions are heavily skewed and all the points seem to be concentrated in one region. That is because of some extreme values of Profits and Sales which matplotlib is trying to accomodate in the limited space of the plot.
Let's remove that point and plot again.
Can you write Python code for it?

# remove points having extreme values
df = df[(df.Profit < 10000) & (df.Sales < 20000)]

sns.jointplot('Sales', 'Profit', df)
plt.show()


I am working on the file "2_Plotting_Distributions.ipynb".
The first task is:
Pairwise Scatter Plots
Now, since we have multiple numeric variables, sns.pairplot() is a good choice to plot all of them in one figure.
Can you write Python code for it?

# pairplot
sns.pairplot(curr)
plt.show()

# You can also observe the correlation between the currencies
# using df.corr()
cor = curr.corr()
round(cor, 3)


I am working on the file "2_Plotting_Distributions.ipynb".
The first task is:
Since seaborn uses matplotlib behind the scenes, the usual matplotlib functions work well with seaborn. For example, you can use subplots to plot multiple univariate distributions.
Can you write Python code for it?

# subplots

# subplot 1
plt.subplot(2, 2, 1)
plt.title('Sales')
sns.distplot(df['Sales'])

# subplot 2
plt.subplot(2, 2, 2)
plt.title('Profit')
sns.distplot(df['Profit'])

# subplot 3
plt.subplot(2, 2, 3)
# plt.title('Order Quantity')
sns.distplot(df['Order_Quantity'])

# subplot 4
plt.subplot(2, 2, 4)
# plt.title('Shipping Cost')
sns.distplot(df['Shipping_Cost'])

plt.show()


I am working on the file "2_Plotting_Distributions.ipynb".
The first task is:
The bottom-right region of the plot represents orders where the Sales is high but the Profit is low, i.e. even when the store is getting highb revenue, the orders are still making losses. These are the kind of orders a business would want to avoid.
We'll see how to drill further down in the next section. For now, let's move to plotting pairwise relationships.
Plotting Pairwise Relationships
You'll find it helpful to plot pairwise relationships between multiple numeric variables. For e.g., here we have taken the prices of some popular cryptocurrencies such as bitcoin, litecoin, ethereum, monero, neo, quantum and ripple.
Now, the crypto enthusiasts would know that the prices of these currencies vary with each other. If bitcoin goes up, the others will likely follow suit, etc.
Now, say you want to trade in some currencies. Given a set of cryptocurrencies, how will you decide when and which one to buy/sell? It will be helpful to analyse past data and identify some trends in these currencies.
Can you write Python code for it?

# reading cryptocurrency files
btc = pd.read_csv("crypto_data/bitcoin_price.csv")
ether = pd.read_csv("crypto_data/ethereum_price.csv")
ltc = pd.read_csv("crypto_data/litecoin_price.csv")
monero = pd.read_csv("crypto_data/monero_price.csv")
neo = pd.read_csv("crypto_data/neo_price.csv")
quantum = pd.read_csv("crypto_data/qtum_price.csv")
ripple = pd.read_csv("crypto_data/ripple_price.csv")

# putting a suffix with column names so that joins are easy
btc.columns = btc.columns.map(lambda x: str(x) + '_btc')
ether.columns = ether.columns.map(lambda x: str(x) + '_et')
ltc.columns = ltc.columns.map(lambda x: str(x) + '_ltc')
monero.columns = monero.columns.map(lambda x: str(x) + '_mon')
neo.columns = neo.columns.map(lambda x: str(x) + '_neo')
quantum.columns = quantum.columns.map(lambda x: str(x) + '_qt')
ripple.columns = ripple.columns.map(lambda x: str(x) + '_rip')

btc.head()

# merging all the files by date
m1 = pd.merge(btc, ether, how="inner", left_on="Date_btc", right_on="Date_et")
m2 = pd.merge(m1, ltc, how="inner", left_on="Date_btc", right_on="Date_ltc")
m3 = pd.merge(m2, monero, how="inner", left_on="Date_btc", right_on="Date_mon")
m4 = pd.merge(m3, neo, how="inner", left_on="Date_btc", right_on="Date_neo")
m5 = pd.merge(m4, quantum, how="inner", left_on="Date_btc", right_on="Date_qt")
crypto = pd.merge(m5, ripple, how="inner", left_on="Date_btc", right_on="Date_rip")

crypto.head()

# Subsetting only the closing prices column for plotting
curr = crypto[["Close_btc", "Close_et", 'Close_ltc', "Close_mon", "Close_neo", "Close_qt"]]
curr.head()


I am working on the file "2_Plotting_Distributions.ipynb".
The first task is:
The dataframe above is a correlation matrix of cryptocurrencies. Try finding some important relationships between currencies. Notice that quantum and ethereum are highly correlated (0.79).
Heatmaps
It will be helpful to visualise the correlation matrix itself using sns.heatmap().
Can you write Python code for it?

# figure size
plt.figure(figsize=(10,8))

# heatmap
sns.heatmap(cor, cmap="YlGnBu", annot=True)
plt.show()


I am working on the file "2_Plotting_Distributions.ipynb".
The first task is:
You can adjust the arguments of the jointplot() to make the plot more readable. For e.g. specifying kind=hex will create a 'hexbin plot'.
Can you write Python code for it?

# plotting low Sales value orders
# hex plot
df = pd.read_csv("./global_sales_data/market_fact.csv")
df = df[(df.Profit < 100) & (df.Profit > -100) & (df.Sales < 200)]
sns.jointplot('Sales', 'Profit', df, kind="hex", color="k")
plt.show()


I am working on the file "2_Plotting_Distributions.ipynb".
The first task is:
You can also plot what is known as the rug plot which plots the actual data points as small vertical bars. The rug plot is simply specified as an argument of the distplot().
Can you write Python code for it?

# rug = True
# plotting only a few points since rug takes a long while
sns.distplot(df['Shipping_Cost'][:200], rug=True)
plt.show()


I am working on the file "2_Preprocessing_of_Training_Data.ipynb".
The first task is:
Load the APOGEE data set into memory
For the training of StarNet, it is only necessary to obtain the spectra and labels, but we need to set restrictions on the training set to obtain the labels of highest validity so we will first include APOGEE_IDs, the spectra, the S/N of the combined spectra, $T_{\mathrm{eff}}$, $\log(g)$,  [Fe/H],  $V_{scatter}$,  STARFLAGs, and ASPCAPFLAGs
Can you write Python code for it?

ap_id = F['IDs'][:,0]
combined_snr = F['stacked_snr']
starflag = F['star_flag']
aspcapflag = F['aspcap_flag']
teff = F['TEFF'][:]
logg = F['LOGG'][:]
fe_h = F['FE_H'][:]
vscatter = F['VSCATTER']

print('Obtained data for '+str(len(ap_id))+' individual visits from '+str(len(list(set(list(ap_id)))))+' stars.')


I am working on the file "2_Preprocessing_of_Training_Data.ipynb".
The first task is:
Load the file that contains individual visit spectra along with APOGEE data associated with each star
Can you write Python code for it?

filename = datadir + 'apStar_visits_main.h5'

F = h5py.File(filename,'r')
print('Dataset keys in file: \n')
print(list(F.keys()))


I am working on the file "2_Preprocessing_of_Training_Data.ipynb".
The first task is:
Select the first $num_ref$ visits for the reference set
We shuffle around the data to avoid local effects.
Later on, it will be be split into training and cross-validation sets.
The remaining high S/N spectra will be used in the test set
Can you write Python code for it?

num_ref = 44784 # number of reference spectra

indices_ref = indices[0:num_ref]
np.random.shuffle(indices_ref)

ap_id_ref = ap_id[indices_ref]
teff = teff[indices_ref]
logg = logg[indices_ref]
fe_h = fe_h[indices_ref]

print('Reference set includes '+str(len(ap_id_ref))+' individual visits from '+str(len(set(ap_id_ref)))+' stars.')


I am working on the file "2_Scraping_Content_Publisher_News_Indonesia.ipynb".
The first task is:
Kumparan
Can you write Python code for it?

url = '''https://kumparan.com/kita-setara/menyingkirkan-stigma-buruk-hiv-aids'''
sn = scrap_news(url)
result = sn.scrap_publisher_news()

print('URL : %s' % url)
print('Title : %s' % result[0])
print('Content : %s' % result[1])


I am working on the file "2_Scraping_Content_Publisher_News_Indonesia.ipynb".
The first task is:
Result
Detik.com
Can you write Python code for it?

url = '''https://news.detik.com/berita/3494173/polisi-jl-jend-sudirman-macet-karena-salju-palsu-dari-busa-air-got'''
sn = scrap_news(url)
result = sn.scrap_publisher_news()

print('URL : %s' % url)
print('Title : %s' % result[0])
print('Content : %s' % result[1])


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
(b) Find the condition number of the coefﬁcient matrix.
Can you write Python code for it?

A = np.array([
    [ 2, 4.01],
    [ 3, 6.00],
])

print('condition number : {}'.format(condition_number(A)))


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
2.1 Computer Problems

Put together the code fragments in this section to create a MATLAB program for “naive” Gaussian elimination (meaning no row exchanges allowed). Use it to solve the systems of Exercise 2.

See my implementation naive_gaussian_elimination in python.

Let $H$ denote the $n \times n$ Hilbert matrix, whose $(i, j)$ entry is $1 / (i + j - 1)$. Use the MATLAB program from Computer Problem 1 to solve $Hx = b$, where $b$ is the vector of all ones, for (a) n = 2 (b) n = 5 (c) n = 10.
Can you write Python code for it?

def computer_problems2__2_1(n):
    # generate Hilbert matrix H
    H = scipy.linalg.hilbert(n)

    # generate b
    b = np.ones(n).reshape(n, 1)

    # combine H:b in tableau form
    mat = np.hstack((H, b))

    # gaussian elimination
    _, x = naive_gaussian_elimination(mat)

    return x

with np.printoptions(precision = 6, suppress = True):
    print('(a) n =  2 → x = {}'.format(computer_problems2__2_1( 2)))
    print('(b) n =  5 → x = {}'.format(computer_problems2__2_1( 5)))
    print('(c) n = 10 → x = {}'.format(computer_problems2__2_1(10)))


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
2.1 Gaussian Elimination
Can you write Python code for it?

def naive_gaussian_elimination(matrix):
    """
    A simple gaussian elimination to solve equations

    Args:
        matrix : numpy 2d array

    Returns:
        mat : The matrix processed by gaussian elimination
        x   : The roots of the equation

    Raises:
        ValueError:
            - matrix is null
        RuntimeError :
            - Zero pivot encountered
    """
    if matrix is None :
        raise ValueError('args matrix is null')

    #Clone the matrix
    mat = matrix.copy().astype(np.float64)

    # Row Size
    m = mat.shape[0]

    # Column Size
    n = mat.shape[1]

    # Gaussian Elimaination
    for i in range(0, m):
        if np.abs(mat[i , i]) == 0 :
            raise RuntimeError('zero pivot encountered')
        for j in range(i + 1, m):
            mult = mat[j, i] / mat[i, i]
            for k in range(i, m):
                mat[j, k] -= mult * mat[i, k]
            mat[j, n - 1] -= mult * mat[i, n - 1]

    # Back Substitution
    x = np.zeros(m, dtype=np.float64)
    for i in range(m - 1,-1,-1):
        for j in range(i + 1, m):
            mat[i, n-1] = mat[i ,n-1] - mat[i,j] * x[j]
            mat[i, j] = 0.0
        x[i] = mat[i, n-1] / mat[i, i]

    return mat, x


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
2.2 Computer Problems

Use the code fragments for Gaussian elimination in the previous section to write a MATLAB script to take a matrix A as input and output L and U. No row exchanges are allowed - the program should be designed to shut down if it encounters a zero pivot. Check your program by factoring the matrices in Exercise 2.

See my implementation LU_factorization in python.
Can you write Python code for it?

# Exercise 2 - (a)
A = np.array([
    [ 3, 1, 2],
    [ 6, 3, 4],
    [ 3, 1, 5]
])

L, U = LU_factorization(A)

print('L = ')
print(L)

print()

print('U = ')
print(U)

# Exercise 2 - (b)
A = np.array([
    [ 4, 2, 0],
    [ 4, 4, 2],
    [ 2, 2, 3]
])

L, U = LU_factorization(A)

print('L = ')
print(L)

print()

print('U = ')
print(U)

# Exercise 2 - (c)
A = np.array([
    [ 1, -1, 1,  2],
    [ 0,  2, 1,  0],
    [ 1,  3, 4,  4],
    [ 0,  2, 1, -1]
])

L, U = LU_factorization(A)

print('L = ')
print(L)

print()

print('U = ')
print(U)


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
2.2 The LU Factorization
Can you write Python code for it?

def LU_factorization(matrix):
    """
    LU decomposition

    Arguments:
        matrix : numpy 2d array

    Return:
        L : lower triangular matrix
        U : upper triangular matrix

    Raises:
        ValueError:
            - matrix is null
            - matrix is not a 2d array
        RuntimeError :
            - zero pivot encountered
    """
    if matrix is None :
        raise ValueError('args matrix is null')

    if matrix.ndim != 2 :
        raise ValueError('matrix is not a 2d-array')

    # dimension
    dim = matrix.shape[0]

    # Prepare LU matrixs
    L = np.identity(dim).astype(np.float64)
    U = matrix.copy().astype(np.float64)

    # Gaussian Elimaination
    for i in range(0, dim - 1):
        # Check pivot is not zero
        if np.abs(U[i , i]) == 0 :
            raise RuntimeError('zero pivot encountered')
        for j in range(i + 1, dim):
            mult = U[j, i] / U[i, i]
            for k in range(i, dim):
                U[j, k] -= mult * U[i, k]
            L[j, i] = mult

    return L, U


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
2.3 Computer Problems

For the n x n matrix with entries $A_{ij} = 5 / (i + 2j - 1)$, set $x = [1,\cdots,1]^T$ and $b = Ax$. Use the MATLAB program from Computer Problem 2.1.1 or MATLAB’s backslash command to compute $x_c$, the double precision computed solution. Find the inﬁnity norm of the forward error and the error magniﬁcation factor of the problem $Ax = b$, and compare it with the condition number of A: (a) n = 6 (b) n = 10.
Can you write Python code for it?

def system_provider(n, data_generator):
    A = np.zeros([n, n])
    x = np.ones(n)

    for i in range(n):
        for j in range(n):
            A[i, j] = data_generator(i + 1, j + 1)

    b = np.matmul(A, x)

    return A, x, b

def problem_2_3_1_generic_solver(n, data_generator):
    A, x, b = system_provider(n, data_generator)
    xc = np.linalg.solve(A, b)

    # forward error
    forward_error = np.max(np.abs(x - xc))

    # relative forward error
    relative_forward_error = forward_error / np.max(np.abs(x))

    # backward error
    backward_error = np.max(np.abs(b - np.matmul(A, xc)))

    # relative backward error
    relative_backward_error = backward_error / np.max(np.abs(b))

    # error magnification factor
    error_magnification_factor = relative_forward_error / relative_backward_error

    # condition number
    condA = condition_number(A)

    return forward_error, error_magnification_factor, condA

def problem_2_3_1_solver(n):
    return problem_2_3_1_generic_solver(n, lambda i, j : 5 / (i + 2 * j - 1))

# (a) n = 6
print('(a) n =  6, forward error = {:.3g}, error magnification factor = {:.3g}, condition number = {:.3g}'.format(*problem_2_3_1_solver(6)))

# (b) n = 10
print('(b) n = 10, forward error = {:.3g}, error magnification factor = {:.3g}, condition number = {:.3g}'.format(*problem_2_3_1_solver(10)))


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
2.3 Sources Of Error
DEFINITION 2.3
The inﬁnity norm, or maximum norm, of the vector $x = (x_1, \cdots, x_n)$ is $||x||_{\infty} = \text{max}|x_i|, i = 1,\cdots,n$, that is, the maximum of the absolute values of the components of x.
DEFINITION 2.4
Let $x_a$ be an approximate solution of the linear system $Ax = b$. The residual is the vector $r = b - Ax_a$. The backward error is the norm of the residual $||b - Ax_a||{\infty}$,
and the forward error is $||x - x_a||{\infty}$.
Example
Find the backward and forward errors for the approximate solution $x_a = [1, 1]$ of the system
$$
\large
\begin{bmatrix}
1 &  1 \
3 & -4 \
\end{bmatrix}
\begin{bmatrix}
x_1 \
x_2 \
\end{bmatrix}
=
\begin{bmatrix}
3 \
2 \
\end{bmatrix}
$$
Can you write Python code for it?

A = np.array([
    [ 1,  1],
    [ 3, -4]
])

b = np.array([3, 2])

xa = np.array([1, 1])

# Get correct solution
system = sympy.Matrix(((1, 1, 3), (3, -4, 2)))
solver = sympy.solve_linear_system(system, sympy.abc.x, sympy.abc.y)

# Packed as list
x = np.array([solver[sympy.abc.x].evalf(), solver[sympy.abc.y].evalf()])

# Output
print(x)

# Get backward error (differences in the input)
residual = b - np.matmul(A, xa)
backward_error = np.max(np.abs(residual))
print('backward error is {:f}'.format(backward_error))

# Get fowrawd error (differences in the output)
forward_error = np.max(np.abs(x - xa))
print('forward error is {:f}'.format(forward_error))


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
2.4 The PA=LU Factorization
Example
Apply Gaussian elimination with partial pivoting to solve the system
\begin{matrix}
  x_1 - x_2 + 3x_3 = -3 & \
-1x_1 - 2x_3 = 1 & \
 2x_1 + 2x_2 + 4x_3 = 0
\end{matrix}
Can you write Python code for it?

A = np.array([1, -1, 3, -1, 0, -2, 2, 2, 4]).reshape(3, 3)
b = np.array([-3, 1, 0])
lu, piv = linalg.lu_factor(A)
x = linalg.lu_solve([lu, piv], b)
print(x)


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
2.5 Iterative Methods
Jacobi Method
Can you write Python code for it?

def jacobi_method(A, b, x0, k):
    """
    Use jacobi method to solve equations

    Args:
        A  (numpy 2d array): the matrix
        b  (numpy 1d array): the right hand side vector
        x0 (numpy 1d array): initial guess
        k  (real number): iterations

    Return:
        The approximate solution

    Exceptions:
        ValueError
            The size of matrix's column is not equal to the size of vector's size
    """
    if A.shape[1] is not x0.shape[0] :
        raise ValueError('The size of the columns of matrix A must be equal to the size of the x0')

    D = np.diag(A.diagonal())
    inv_D = linalg.inv(D)
    LU = A - D
    xk = x0

    for _ in range(k):
        xk = np.matmul(b - np.matmul(LU, xk), inv_D)

    return xk


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
2.6 Methods for symmetric positive-definite matrices
Cholesky factorization
Example
Find the Cholesky factorization of
$\begin{bmatrix}
4 & -2 & 2   \
-2 & 2 & -4  \
2 & -4 & 11
\end{bmatrix}$
Can you write Python code for it?


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Add two-step back substitution to your script from Computer Problem 1, and use it to solve the systems in Exercise 4.
Can you write Python code for it?

def LU_factorization_with_back_substitution(A, b):
    """
    LU decomposition with two-step back substitution
    where Ax = b

    Arguments:
        A : coefficient matrix
        b : constant vector

    Return:
        x : solution vector
    """
    L, U = LU_factorization(A)

    # row size
    rowsz = b.size

    # calculate Lc = b where Ux = c
    matrix = np.hstack((L, b))
    c = naive_gaussian_elimination(matrix)[1].reshape(rowsz, 1)

    # calculate Ux = c
    matrix = np.hstack((U, c))
    x = naive_gaussian_elimination(matrix)[1].reshape(rowsz)

    return x

# Exercise 4 - (a)
A = np.array([
    [ 3, 1, 2],
    [ 6, 3, 4],
    [ 3, 1, 5]
])

b = np.array([0, 1, 3]).reshape(3, 1)

x = LU_factorization_with_back_substitution(A, b)

print(x)

# Exercise 4 - (b)
A = np.array([
    [ 4, 2, 0],
    [ 4, 4, 2],
    [ 2, 2, 3]
])

b = np.array([2, 4, 6]).reshape(3, 1)

x = LU_factorization_with_back_substitution(A, b)

print(x)


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Additional Examples

Put the system $x + 2y - z = 3,-3x + y + z = -6,2x + z = 8$ into tableau form and solve by Gaussian elimination.
Can you write Python code for it?

input_mat = np.array([
    [ 1, 2, -1,  3],
    [-3, 1,  1, -6],
    [ 2, 0,  1,  8]
])

output_mat, x = naive_gaussian_elimination(input_mat)

print(output_mat)
print('[x, y, z] = {}'.format(x))


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Carry out Computer Problem 1 for the matrix with entries $A_{ij} = 1/(|i - j| + 1)$.
Can you write Python code for it?

def problem_2_3_2_solver(n):
    return problem_2_3_1_generic_solver(n, lambda i, j : 1 / (np.abs(i - j) + 1))

# (a) n = 6
print('(a) n =  6, forward error = {:.3g}, error magnification factor = {:.3g}, condition number = {:.3g}'.format(*problem_2_3_2_solver(6)))

# (b) n = 10
print('(b) n = 10, forward error = {:.3g}, error magnification factor = {:.3g}, condition number = {:.3g}'.format(*problem_2_3_2_solver(10)))


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Carry out the steps of Computer Problem 3 for the matrix with entries $A_{ij} = \sqrt{(i - j)^2 + n / 10}$.
Can you write Python code for it?

def problem_2_3_4_solver(n):
    return problem_2_3_1_generic_solver(n, lambda i, j : np.sqrt(np.power(i - j, 2) + n / 10))

# n = 100
print('n =  100, forward error = {:.2g}, error magnification factor = {:.2g}, condition number = {:.2g}'.format(*problem_2_3_4_solver(100)))
# n = 200
print('n =  200, forward error = {:.2g}, error magnification factor = {:.2g}, condition number = {:.2g}'.format(*problem_2_3_4_solver(200)))
# n = 300
print('n =  300, forward error = {:.2g}, error magnification factor = {:.2g}, condition number = {:.2g}'.format(*problem_2_3_4_solver(300)))
# n = 400
print('n =  400, forward error = {:.2g}, error magnification factor = {:.2g}, condition number = {:.2g}'.format(*problem_2_3_4_solver(400)))
# n = 500
print('n =  500, forward error = {:.2g}, error magnification factor = {:.2g}, condition number = {:.2g}'.format(*problem_2_3_4_solver(500)))


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Conjugate Gradient Method
Can you write Python code for it?

def conjugate_gradient_method(A, b, x0, k):
    """
    Use conjugate gradient to solve linear equations

    Args:
        A  : input matrix
        b  : input right hand side vector
        x0 : initial guess
        k  : iteration

    Returns:
        approximate solution

    """
    xk = x0
    dk = rk = b - np.matmul(A, x0)
    for _ in range(k):
        if not np.any(rk) or all( abs(i) <= 1e-16 for i in rk) is True:
            break
        ak = float(np.matmul(rk.T, rk)) / float(np.matmul(dk.T, np.matmul(A, dk)))
        xk = xk + ak * dk
        rk1 = rk - ak * np.matmul(A, dk)
        bk = np.matmul(rk1.T, rk1) / np.matmul(rk.T, rk)
        dk = rk1 + bk * dk
        rk = rk1
    return xk


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
DEFINITION 2.2
An $m \times n$ matrix $L$ is lower triangular if its entries satisfy $l_{ij} = 0$ for $i < j$. An $m \times n$ matrix $U$ is upper triangular if its entries satisfy $u_{ij} = 0$ for $i > j$.
Example
Find the LU factorization for the matrix $A$ in
$$
\large
\begin{bmatrix}
1 &  1 \
3 & -4 \
\end{bmatrix}
$$
Can you write Python code for it?

A = np.array([
    [1, 1],
    [3, -4]
])

L, U = LU_factorization(A)

print('L = ')
print(L)

print()

print('U = ')
print(U)


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Example
Apply the Gauss-Seidel Method to the system
$$
\begin{bmatrix}
3 & 1 & -1 \
2 & 4 & 1  \
-1 & 2 & 5
\end{bmatrix}
\begin{bmatrix}
u \
v \
w
\end{bmatrix}
=
\begin{bmatrix}
4 \
1 \
1
\end{bmatrix}
$$
Can you write Python code for it?


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Example
Apply the Gauss-Seidel Method with sor to the system
$$
\begin{bmatrix}
3 & 1 & -1 \
2 & 4 & 1  \
-1 & 2 & 5
\end{bmatrix}
\begin{bmatrix}
u \
v \
w
\end{bmatrix}
=
\begin{bmatrix}
4 \
1 \
1
\end{bmatrix}
$$
Can you write Python code for it?


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Example
Apply the Jacobi Method to the system $3u + v = 5$, $u + 2v = 5$
Can you write Python code for it?

A = np.array([3, 1, 1, 2]).reshape(2, 2)
b = np.array([5, 5])
x = jacobi_method(A, b, np.array([0, 0]), 20)
print('x = %s' %x)


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Example
Solve
$$
\begin{bmatrix}
1  & -1 & 0 \
-1 &  2 & 1 \
0  &  1 & 2 \
\end{bmatrix}
\begin{bmatrix}
u \
v \
w \
\end{bmatrix}
=
\begin{bmatrix}
0 \
2 \
3 \
\end{bmatrix}
$$
Can you write Python code for it?


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Example
Solve
$$
\begin{bmatrix}
1  & -1 & 0 \
-1 &  2 & 1 \
0  &  1 & 5 \
\end{bmatrix}
\begin{bmatrix}
u \
v \
w \
\end{bmatrix}
=
\begin{bmatrix}
3  \
-3 \
4  \
\end{bmatrix}
$$
Can you write Python code for it?


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Example
Solve
$$
\begin{bmatrix}
2 & 2 \
2 & 5 \
\end{bmatrix}
\begin{bmatrix}
u \
v
\end{bmatrix}
=
\begin{bmatrix}
6 \
3
\end{bmatrix}
$$
using the Conjugate Gradient Method
Can you write Python code for it?

A = np.array([2, 2, 2, 5]).reshape(2, 2)
b = np.array([6, 3])
x0 = np.array([0, 0])
conjugate_gradient_method(A, b, x0, 2)


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Example
Solve system
$$
\large
\begin{bmatrix}
1 &  1 \
3 & -4 \
\end{bmatrix}
\begin{bmatrix}
x_1 \
x_2 \
\end{bmatrix}
=
\begin{bmatrix}
3 \
2 \
\end{bmatrix}
$$
, using the LU factorization
Can you write Python code for it?

A = np.array([
    [ 1,  1],
    [ 3, -4]
])

b = np.array([3, 2]).reshape(2, 1)

L, U = LU_factorization(A)

# calculate Lc = b where Ux = c
mat = np.hstack((L, b))
c = naive_gaussian_elimination(mat)[1].reshape(2, 1)

# calculate Ux = c
mat = np.hstack((U, c))
x = naive_gaussian_elimination(mat)[1].reshape(2, 1)

# output the result
print('x1 = {}, x2 = {}'.format(x[0][0], x[1][0]))


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Example
Solve the system $2x_1 + 3x_2 = 4$,$3x_1 + 2x_2 = 1$ using the PA = LU factorization with partial pivoting
Can you write Python code for it?

"""
[[2, 3]
 [3, 2]]
"""
A = np.array([2, 3, 3, 2]).reshape(2, 2)
b = np.array([4, 1])
lu, piv = linalg.lu_factor(A)
x = linalg.lu_solve([lu, piv], b)
print(x)


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Example
Use Newton's method to find the solutions of the system
$$
f_1(u,v) = 6u^3 + uv - 3^3 - 4 = 0 \
f_2(u,v) = u^2 - 18uv^2 + 16v^3 + 1 = 0
$$
Can you write Python code for it?

fA = lambda u,v : np.array([6 * pow(u, 3) + u * v - 3 * pow(v, 3) - 4,
                           pow(u, 2) - 18 * u * pow(v, 2) + 16 * pow(v, 3) + 1],
                           dtype=np.float64)

fDA = lambda u,v : np.array([18 * pow(u, 2) + v,
                            u - 9 * pow(v, 2),
                            2 * u - 18 * pow(v, 2),
                            -36 * u * v + 48 * pow(v, 2)],
                            dtype=np.float64).reshape(2, 2)

x0 = np.array([2, 2], dtype=np.float64)

multivariate_newton_method(fA, fDA, x0, 5)


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
Example
Use Newton's method with starting guess $(1,2)$ to find a solution of the system
$$
v - u^3 = 0 \
u^2 + v^2 - 1 = 0
$$
Can you write Python code for it?

fA = lambda u,v : np.array([v - pow(u, 3), pow(u, 2) + pow(v, 2) - 1], dtype=np.float64)
fDA = lambda u,v : np.array([-3 * pow(u, 2), 1, 2 * u, 2 * v], dtype=np.float64).reshape(2, 2)
x0 = np.array([1, 2])
multivariate_newton_method(fA, fDA, x0, 10)


I am working on the file "2_Systems_Of_Equations.ipynb".
The first task is:
For what values of n does the solution in Computer Problem 1 have no correct signiﬁcant digits?
Can you write Python code for it?

print('n = 11, forward error = {:.3g}, error magnification factor = {:.3g}, condition number = {:.3g}'.format(*problem_2_3_1_solver(11)))