I am working on the file "8_Calibration/8_1_Calibration_Least_Squares_Problem.ipynb".
The first task is:
The images above contain the real part of the corrupted (green) and uncorrupted (blue)
visibilities as a function of timeslots for baseline 01, 02 and 12 respectively.
8.1.4 Levenberg-Marquardt (create_G_LM)
We are now ready to use least squares to calibrate $\boldsymbol{\mathcal{D}}$ (see GPU accelerated nonlinear optimization in radio interferometric calibration ⤴).
We first present a brief review of least squares minimization. Suppose we wish to fit a model $\mathbf{f}\left( \mathbf{m},\breve{\mathbf{g}}\right)$, where $\mathbf{m}$ and $\breve{\mathbf{g}}$ denote
the model input values and a vector of unknown variables respectively, to some data $\left{\mathbf{d}{i},\mathbf{m}{i}\right}$. The vector of unknown variables $\breve{\mathbf{g}}$ parametrize the model. A standard method for determining which parameter vector $\breve{\mathbf{g}}$ best fits the data is to minimize the sum of the squared residuals. This technique is referred to as least squares minimization. The residual vector is denoted by $\mathbf{r}(\mathbf{m},\mathbf{d},\breve{\mathbf{g}}) = \mathbf{d} - \mathbf{f}\left( \mathbf{m},\breve{\mathbf{g}}\right)$. The objective function (the function we wish to minimize) associated with least squares is: $\sum_i \mathbf{r}_i^2$. The function optimize.leastsq is scipy's built-in least squares solver and employs the Levenberg-Marquardt algorithm in the background. The Levenberg-Marquardt algorithm is discussed in more detail in $\S$ 2.11 ➞. To use optimize.leastsq one needs a function, here called err_func, that calculates the residual vector $\mathbf{r}$. An initial guess of the parameter vector $\breve{\mathbf{g}}$ is also required.
For calibration the above variables become:

   Vectorizing

• $\mathbf{d} = [\textrm{vec}(\Re\{\boldsymbol{\mathcal{D}}\}),\textrm{vec}(\Im\{\boldsymbol{\mathcal{D}}\})]$

• $\mathbf{m} =...
Can you write Python code for it?

'''Unpolarized direction independent calibration entails finding the G that minimizes ||R-GMG^H||.
This function evaluates D-GMG^H.
g is a vector containing the real and imaginary components of the antenna gains.
d is a vector containing a vecotrized R (observed visibilities), real and imaginary.
m is a vector containing a vecotrized M (predicted), real and imaginary.
r is a vector containing the residuals.
'''
def err_func(g,d,m):
    Nm = len(d)/2
    N = len(g)/2
    G = np.diag(g[0:N]+1j*g[N:])
    D = np.reshape(d[0:Nm],(N,N))+np.reshape(d[Nm:],(N,N))*1j #matrization
    M = np.reshape(m[0:Nm],(N,N))+np.reshape(m[Nm:],(N,N))*1j
    T = np.dot(G,M)
    T = np.dot(T,G.conj())
    R = D - T
    r_r = np.ravel(R.real) #vectorization
    r_i = np.ravel(R.imag)
    r = np.hstack([r_r,r_i])
    return r


I am working on the file "8_Calibration/8_1_Calibration_Least_Squares_Problem.ipynb".
The first task is:
We can also pack the $uv$-coverage into a 2D-matrix. We denote the rows of this matrix with $p$ and the columns with $q$. The $pq$-th entry denotes the $uv$-track associated with baseline $pq$. The reason for packing the visibilities into a 2D structure will become apparent in Sec. 8.1.2 ⤵<!--\ref{cal:sec:RIME_un}-->.
Can you write Python code for it?

u_m = np.zeros((len(b),len(b),len(H)))
v_m = np.zeros((len(b),len(b),len(H)))

u_m[0,1,:] = u[0,:] #the first two entries denote p and q and the third index denotes time
u_m[1,2,:] = u[1,:]
u_m[0,2,:] = u[2,:]

v_m[0,1,:] = v[0,:]
v_m[1,2,:] = v[1,:]
v_m[0,2,:] = v[2,:]


I am working on the file "8_Calibration/8_1_Calibration_Least_Squares_Problem.ipynb".
The first task is:
We first need to set the hour angle range of our observation and the declination of our field center.
Can you write Python code for it?

H = np.linspace(-6,6,600)*(np.pi/12) #Hour angle in radians
delta = 60*(np.pi/180) #Declination in radians


I am working on the file "8_Calibration/8_1_Calibration_Least_Squares_Problem.ipynb".
The first task is:
We may now calibrate $\boldsymbol{\mathcal{D}}$ by using create_G_LM.
Can you write Python code for it?


I am working on the file "8_Calibration/8_1_Calibration_Least_Squares_Problem.ipynb".
The first task is:
We plot the corrected visibilities below. Note that the model and corrected visibilities align well, implying that calibration was successfull.
Can you write Python code for it?

fig = plt.figure()

timeslots = np.cumsum(np.ones((len(M[0,1,:]),)))

#We only plot the real part of visibilities

#Plotting Baseline 01
ax = plt.subplot("311")
ax.set_title("$m_{01}$ (blue) and $d_{01}^{(c)}$  (green)", fontsize=18)
ax.plot(timeslots,M[0,1,:].real)
ax.plot(timeslots,R_c[0,1,:].real)
ax.set_xlabel("Timeslot", fontsize=18)
ax.set_ylabel("Jy", fontsize=18)
ax.set_xlim([1,len(M[0,1,:])])
y_t = ax.get_yticks()
y_t = y_t[::2]
ax.set_yticks(y_t)

#Plotting Baseline 02
ax = plt.subplot("312")
ax.set_title("$m_{02}$ (blue) and $d_{02}^{(c)}$ (green)", fontsize=18)
ax.plot(timeslots,M[0,2,:].real)
ax.plot(timeslots,R_c[0,2,:].real)
ax.set_xlabel("Timeslot", fontsize=18)
ax.set_ylabel("Jy", fontsize=18)
ax.set_xlim([1,len(M[0,1,:])])
y_t = ax.get_yticks()
y_t = y_t[::2]
ax.set_yticks(y_t)

#Plotting Baseline 12
ax = plt.subplot("313")
ax.set_title("$m_{12}$ (blue) and $d_{12}^{(c)}$ (green)", fontsize=18)
ax.plot(timeslots,M[1,2,:].real)
ax.plot(timeslots,R_c[1,2,:].real)
ax.set_xlabel("Timeslot", fontsize=18)
ax.set_ylabel("Jy", fontsize=18)
ax.set_xlim([1,len(M[0,1,:])])
y_t = ax.get_yticks()
y_t = y_t[::2]
ax.set_yticks(y_t)

plt.tight_layout()

plt.show()


I am working on the file "8_Calibration/8_1_calibration_least_squares_problem.ipynb".
The first task is:
8.1.2. Unpolarized Calibration
As explained in $\S$ 7.2 ➞ the RIME assumes that our observed signal is polarized. For the sake
of simplicity, however, we will now introduce the calibration problem with the underlying assumption that the observed signal is unpolarized. Unpolarized calibration is achieved by solving the following minimization problem:

   Unpolarized Calibration

\begin{equation}
\min_{\boldsymbol{\mathcal{G}}} \left \| \boldsymbol{\mathcal{D}} - \boldsymbol{\mathcal{G}}\boldsymbol{\mathcal{M}}\boldsymbol{\mathcal{G}}^H \right \|,
\end{equation}

where
* $\boldsymbol{\mathcal{D}}$ is the observed visibility matrix. Each entry, which we denote by $d_{pq}$, represents the visibility measured by the baseline formed by antennas $p$ and $q$.
* $\boldsymbol{\mathcal{M}}$ is the model visibility matrix. The entry $m_{pq}$ of $\boldsymbol{\mathcal{M}}$ denotes a true or model visibility which was created with the calibration sky model and a $uv$-point on the $uv$-track associated with baseline $pq$.
* $\boldsymbol{\mathcal{G}} = \textrm{diag}(\mathbf{g})$ is the antenna gain matrix, where $\mathbf{g}=[g_1,g_2,\cdots,g_N]^T$ denotes the antenna gain vector. The operator $\textrm{diag}(\cdot)$ forms a diagonal matrix from a vector by putting the elements of the vector on the main diagonal. The vector $\mathbf{g}$ represents the instrumental response of the antennas, i.e. the complex antenna gains. These antenna gains are chosen in such a way that they minimize the difference between the observed and model visibilities.
* $\boldsymbol{\mathcal{G}}\boldsymbol{\mathcal{M}}\boldsymbol{\mathcal{G}}^H$ is the predicted visibility matrix. This matrix contains the model visibilities after the antenna gains have been applied to them.
The superscript $(\cdot)^H$ denotes the Hermitian or conjugate transpose and $\left \| \cdot \right \|$ denotes the norm used. Most calibration algorithms...
Can you write Python code for it?

'''Creates the observed visibilities
point_sources - skymodel of point sources - (amplitude, l, m)
u_m - the u coordinates of observation (packed in a 2D structure)
v_m - the v coordinates of observation (packed in a 2D structure)
g - the antenna gain error vector
sig - the noise
'''
def create_vis_mat(point_sources,u_m,v_m,g,sig):
    D = np.zeros(u.shape)
    G = np.diag(g)
    #Step 1: Create Model Visibility Matrix
    for k in xrange(len(point_sources)): #for each point source
        l_0 = point_sources[k,1]
        m_0 = point_sources[k,2]
        D = D + point_sources[k,0]*np.exp(-2*np.pi*1j*(u_m*l_0+v_m*m_0))

    for t in xrange(D.shape[2]): #for each time-step

        #Step 2: Corrupting the Visibilities
        D[:,:,t] = np.dot(G,D[:,:,t])
        D[:,:,t] = np.dot(D[:,:,t],G.conj())

        #Step 3: Adding Noise
        D[:,:,t] = D[:,:,t] + sig*np.random.randn(u_m.shape[0],u_m.shape[1]) \
            + sig*np.random.randn(u_m.shape[0],u_m.shape[1])*1j

    return D


I am working on the file "8_Calibration/8_1_calibration_least_squares_problem.ipynb".
The first task is:
Figure 8.1.2: AJR:NC: This figure needs a caption
We can also pack the $uv$-coverage into a 2D-matrix. We denote the rows of this matrix with $p$ and the columns with $q$. The $pq$-th entry denotes the $uv$-track associated with baseline $pq$. The reason for packing the visibilities into a 2D structure will become apparent in Sec. 8.1.2 ⤵.
Can you write Python code for it?


I am working on the file "8_Calibration/8_1_calibration_least_squares_problem.ipynb".
The first task is:
Figure 8.1.3: AJR:NC: This figure needs a caption
The images above contain the real part of the corrupted (green) and uncorrupted (blue)
visibilities as a function of timeslots for baseline 01, 02 and 12 respectively.
8.1.4 Levenberg-Marquardt (create_G_LM)
We are now ready to use least squares to calibrate $\boldsymbol{\mathcal{D}}$ (see GPU accelerated nonlinear optimization in radio interferometric calibration ⤴).
We first present a brief review of least squares minimization. Suppose we wish to fit a model $\mathbf{f}\left( \mathbf{m},\breve{\mathbf{g}}\right)$, where $\mathbf{m}$ and $\breve{\mathbf{g}}$ denote
the model input values and a vector of unknown variables respectively, to some data $\left{\mathbf{d}{i},\mathbf{m}{i}\right}$. The vector of unknown variables $\breve{\mathbf{g}}$ parametrize the model. A standard method for determining which parameter vector $\breve{\mathbf{g}}$ best fits the data is to minimize the sum of the squared residuals. This technique is referred to as least squares minimization. The residual vector is denoted by $\mathbf{r}(\mathbf{m},\mathbf{d},\breve{\mathbf{g}}) = \mathbf{d} - \mathbf{f}\left( \mathbf{m},\breve{\mathbf{g}}\right)$. The objective function (the function we wish to minimize) associated with least squares is: $\sum_i \mathbf{r}_i^2$. The function optimize.leastsq is scipy's built-in least squares solver and employs the Levenberg-Marquardt algorithm in the background. The Levenberg-Marquardt algorithm is discussed in more detail in $\S$ 2.11 ➞. To use optimize.leastsq one needs a function, here called err_func, that calculates the residual vector $\mathbf{r}$. An initial guess of the parameter vector $\breve{\mathbf{g}}$ is also required.
For calibration the above variables become:

   Vectorizing

• $\mathbf{d} =...
Can you write Python code for it?

'''Unpolarized direction independent calibration entails finding the G that minimizes ||R-GMG^H||.
This function evaluates D-GMG^H.
g is a vector containing the real and imaginary components of the antenna gains.
d is a vector containing a vecotrized R (observed visibilities), real and imaginary.
m is a vector containing a vecotrized M (predicted), real and imaginary.
r is a vector containing the residuals.
'''
def err_func(g,d,m):
    Nm = len(d)//2
    N = len(g)//2
    G = np.diag(g[0:N]+1j*g[N:])
    D = np.reshape(d[0:Nm],(N,N))+np.reshape(d[Nm:],(N,N))*1j #matrization
    M = np.reshape(m[0:Nm],(N,N))+np.reshape(m[Nm:],(N,N))*1j
    T = np.dot(G,M)
    T = np.dot(T,G.conj())
    R = D - T
    r_r = np.ravel(R.real) #vectorization
    r_i = np.ravel(R.imag)
    r = np.hstack([r_r,r_i])
    return r


I am working on the file "8_Calibration/8_1_calibration_least_squares_problem.ipynb".
The first task is:
Import section specific modules:
Can you write Python code for it?

from scipy import optimize
%pylab inline
pylab.rcParams['figure.figsize'] = (15, 10)

HTML('../style/code_toggle.html')


I am working on the file "8_Calibration/8_1_calibration_least_squares_problem.ipynb".
The first task is:
Our hour angle range is from -6h to 6h, and our declination is set to $60^{\circ}$.
As the earth rotates the antennas trace out $uv$-tracks (ellipses) as shown in the code fragment below, where the red tracks are due to baseline $pq$ and blue tracks are due to baseline $qp$. We can construct these $uv$-tracks by using Eq. 8.1 ⤵<!--\ref{cal:eq:uv_cov}-->.
Can you write Python code for it?

u = np.zeros((len(b),len(H)))
v = np.zeros((len(b),len(H)))
for k in xrange(len(b)):
    u[k,:] = b[k]*np.cos(H)
    v[k,:] = b[k]*np.sin(H)*np.sin(delta)
    plt.plot(u[k,:],v[k,:],"r")
    plt.plot(-u[k,:],-v[k,:],"b")
plt.xlabel("$u$ [rad$^{-1}$]", fontsize=18)
plt.ylabel("$v$ [rad$^{-1}$]", fontsize=18)
plt.title("$uv$-Coverage of three element interferometer", fontsize=18)
plt.show()


u = np.zeros((len(b),len(H)))
v = np.zeros((len(b),len(H)))
for k in range(len(b)):
    u[k,:] = b[k]*np.cos(H)
    v[k,:] = b[k]*np.sin(H)*np.sin(delta)
    plt.plot(u[k,:],v[k,:],"r")
    plt.plot(-u[k,:],-v[k,:],"b")
plt.xlabel("$u$ [rad$^{-1}$]", fontsize=18)
plt.ylabel("$v$ [rad$^{-1}$]", fontsize=18)
plt.title("$uv$-Coverage of three element interferometer", fontsize=18)
plt.show()


I am working on the file "8_Calibration/8_1_calibration_least_squares_problem.ipynb".
The first task is:
The above function works by vectorizing the real and imaginary part of $\boldsymbol{\mathcal{D}}$ and
storing the result in $\mathbf{d}$. The vector $\mathbf{m}$ is generated in a similar manner.
The error vector $\mathbf{r}$ is calculated by err_func. We initialize $\breve{\mathbf{g}}$ with
$\breve{\mathbf{g}}_0=[\mathbf{1},\mathbf{0}]$. We can then call
optimize.leastsq(self.err_func, g_0, args=(d, m)).
We can now calculate $\mathbf{g} = \breve{\mathbf{g}}_U+\imath\breve{\mathbf{g}}_L$ and
$\boldsymbol{\mathscr{G}}=\mathbf{g}\mathbf{g}^H$. This is repeated for each observational time-slot.
8.1.5 Corrected Visibilites
Before imaging, we have to correct our observed visibilities by removing the effect that the antenna gains had on the observed visibilities. This can be accomplished by using

   Correcting Visibilities

\begin{equation}
 \boldsymbol{\mathcal{D}}^\mathrm{(c)} = \boldsymbol{\mathcal{G}}^{-1}\boldsymbol{\mathcal{D}}\boldsymbol{\mathcal{G}}^{-H} = \boldsymbol{\boldsymbol{\mathscr{G}}}^{\odot-1}\odot\boldsymbol{\mathcal{D}},
\end{equation}

where

$\boldsymbol{\mathcal{D}}^\mathrm{(c)}$ is the corrected visibility matrix.
$\boldsymbol{\mathscr{G}}^{\odot-1}$ denotes the visibility calibration matrix, which is computed by taking the Hadamard (element-wise) inverse of $\boldsymbol{\mathscr{G}}$.

The superscript $(\cdot)^{-1}$ denotes matrix inversion, while $(\cdot)^{-H}$ denotes the inverse of the Hermitian transpose. The operator $\odot$ denotes the Hadamard product.
We calculate the corrected visibilities below.

Advice: The matrix and vector operations (like $\odot$) and operators used in this section are discussed in more detail in [$\S$ 2.10 ➞](../2_Mathematical_Groundwork/2_10_linear_algebra.ipynb)

Can you write Python code for it?


I am working on the file "8_Calibration/8_1_calibration_least_squares_problem.ipynb".
The first task is:
We are now able to define a wrapper function create_G_LM that in turn calls optimize.leastsq.
The wrapper function translates the calibration problem into a format that optimize.leastsq
can interpret. The input of create_G_LM is $\boldsymbol{\mathcal{D}}$ and $\boldsymbol{\mathcal{M}}$, while the output is $\mathbf{g}$ and $\boldsymbol{\mathscr{G}}=\mathbf{g}\mathbf{g}^H$.
Can you write Python code for it?

'''This function finds argmin G ||D-GMG^H|| using Levenberg-Marquardt. It uses the optimize.leastsq scipy to perform
the actual minimization.
D is your observed visibilities matrx.
M is your predicted visibilities.
g the antenna gains.
G = gg^H.'''
def create_G_LM(D,M):
    N = D.shape[0] #number of antennas
    temp =np.ones((D.shape[0],D.shape[1]) ,dtype=complex)
    G = np.zeros(D.shape,dtype=complex)
    g = np.zeros((D.shape[0],D.shape[2]),dtype=complex)

    for t in xrange(D.shape[2]): #perform calibration per time-slot
        g_0 = np.ones((2*N,)) # first antenna gain guess
        g_0[N:] = 0
        d_r = np.ravel(D[:,:,t].real) #vectorization of observed + seperating real and imag
        d_i = np.ravel(D[:,:,t].imag)
        d = np.hstack([d_r,d_i])
        m_r = np.ravel(M[:,:,t].real) #vectorization of model + seperating real and imag
        m_i = np.ravel(M[:,:,t].imag)
        m = np.hstack([m_r,m_i])
        g_lstsqr_temp = optimize.leastsq(err_func, g_0, args=(d, m))
        g_lstsqr = g_lstsqr_temp[0]

        G_m = np.dot(np.diag(g_lstsqr[0:N]+1j*g_lstsqr[N:]),temp)
        G_m = np.dot(G_m,np.diag((g_lstsqr[0:N]+1j*g_lstsqr[N:]).conj()))

        g[:,t] = g_lstsqr[0:N]+1j*g_lstsqr[N:] #creating antenna gain vector
        G[:,:,t] = G_m

    return g,G


I am working on the file "8_Calibration/8_1_calibration_least_squares_problem.ipynb".
The first task is:
We may now calibrate $\boldsymbol{\mathcal{D}}$ by using create_G_LM.
Can you write Python code for it?


I am working on the file "8_Calibration/8_1_calibration_least_squares_problem.ipynb".
The first task is:
We now plot the baseline entries of $\boldsymbol{\mathcal{M}}$ and $\boldsymbol{\mathcal{D}}$.
Can you write Python code for it?

fig = plt.figure()

timeslots = np.cumsum(np.ones((len(M[0,1,:]),)))

#We only plot the real part of visibilities

#Plotting Baseline 01
ax = plt.subplot("311")
ax.set_title("$m_{01}$ (blue) and $d_{01}$  (green)", fontsize=18)
ax.plot(timeslots,M[0,1,:].real)
ax.plot(timeslots,D[0,1,:].real)
ax.set_xlabel("Timeslot", fontsize=18)
ax.set_ylabel("Jy", fontsize=18)
ax.set_xlim([1,len(M[0,1,:])])
y_t = ax.get_yticks()
y_t = y_t[::2]
ax.set_yticks(y_t)

#Plotting Baseline 02
ax = plt.subplot("312")
ax.set_title("$m_{02}$ (blue) and $d_{02}$ (green)", fontsize=18)
ax.plot(timeslots,M[0,2,:].real)
ax.plot(timeslots,D[0,2,:].real)
ax.set_xlabel("Timeslot", fontsize=18)
ax.set_ylabel("Jy", fontsize=18)
ax.set_xlim([1,len(M[0,1,:])])
y_t = ax.get_yticks()
y_t = y_t[::2]
ax.set_yticks(y_t)

#Plotting Baseline 12
ax = plt.subplot("313")
ax.set_title("$m_{12}$ (blue) and $d_{12}$ (green)", fontsize=18)
ax.plot(timeslots,M[1,2,:].real)
ax.plot(timeslots,D[1,2,:].real)
ax.set_xlabel("Timeslot", fontsize=18)
ax.set_ylabel("Jy", fontsize=18)
ax.set_xlim([1,len(M[0,1,:])])
y_t = ax.get_yticks()
y_t = y_t[::2]
ax.set_yticks(y_t)

plt.tight_layout()

plt.show()


I am working on the file "8_Calibration/8_1_calibration_least_squares_problem.ipynb".
The first task is:
We now use create_vis_mat to create an example $\boldsymbol{\mathcal{M}}$ and $\boldsymbol{\mathcal{D}}$. Note that
there are two sources in our sky model.
Can you write Python code for it?

point_sources = np.array([(1,0,0),(0.5,(1*np.pi)/180,(0*np.pi)/180)]) #l and m are measures in radians
g = np.array([1.2+1.3j,1.1-1.5j,-1.3+0.7j])
sig = 0.1
D = create_vis_mat(point_sources,u_m,v_m,g,sig) #we corrupt our data and we add noise
M = create_vis_mat(point_sources,u_m,v_m,np.array([1,1,1]),0) #no corruption and no noise


I am working on the file "8_Calibration/8_1_calibration_least_squares_problem.ipynb".
The first task is:
We plot the corrected visibilities below. Note that the model and corrected visibilities align well, implying that calibration was successfull.
Can you write Python code for it?


I am working on the file "8_Calibration/8_problem_set.ipynb".
The first task is:
1. SteFCal: Theoretical derivation
Please derive, discuss and present the SteFCal calibration algorithm? [20]
Subtotal 20
2. SteFCal: Practical implementation
a) Please Complete the following function. This function is suppose to be the SteFCal equivalent of create_G_LM. In other words it should compute g and G but instead of using LM it should use SteFCal. [20]
Can you write Python code for it?

   '''This function finds argmin G ||R-GMG^H|| using StEFCal.
   D is your observed visibilities matrx.
   M is your predicted visibilities.
   imax maximum amount of iterations.
   tau stopping criteria.
   g the antenna gains.
   G = gg^H.'''
   def create_G_stef(self,D,M,imax,tau):
       G = np.zeros(D.shape,dtype=complex)
       g = np.zeros((D.shape[0],D.shape[2]),dtype=complex)

       #Complete the rest of the function

       return g,G


I am working on the file "8_Calibration/8_problem_set.ipynb".
The first task is:
B. Generating Visibilities
Can you write Python code for it?

'''Creates the observed visibilities
point_sources - skymodel of point sources - (amplitude, l, m)
u_m - the u coordinates of observation (packed in a 2D structure)
v_m - the v coordinates of observation (packed in a 2D structure)
g - the antenna gain error vector
sig - the noise
'''
def create_vis_mat(point_sources,u_m,v_m,g,sig):
    D = np.zeros(u_m.shape)
    G = np.diag(g)
    #Step 1: Create Model Visibility Matrix
    for k in xrange(len(point_sources)): #for each point source
        l_0 = point_sources[k,1]
        m_0 = point_sources[k,2]
        D = D + point_sources[k,0]*np.exp(-2*np.pi*1j*(u_m*l_0+v_m*m_0))

    for t in xrange(D.shape[2]): #for each time-step

        #Step 2: Corrupting the Visibilities
        D[:,:,t] = np.dot(G,D[:,:,t])
        D[:,:,t] = np.dot(D[:,:,t],G.conj())

        #Step 3: Adding Noise
        D[:,:,t] = D[:,:,t] + sig*np.random.randn(u_m.shape[0],u_m.shape[1]) + sig*np.random.randn(u_m.shape[0],u_m.shape[1])*1j

    return D

point_sources = np.array([(1,0,0),(0.5,(1*np.pi)/180,(0*np.pi)/180)]) #l and m are measures in radians
g = np.array([1.2+1.3j,1.1-1.5j,-1.3+0.7j])
sig = 0.1
D = create_vis_mat(point_sources,u_m,v_m,g,sig) #we corrupt our data and we add noise
M = create_vis_mat(point_sources,u_m,v_m,np.array([1,1,1]),0) #no corruption and no noise

fig = plt.figure()

timeslots = np.cumsum(np.ones((len(M[0,1,:]),)))

#We only plot the real part of visibilities

#Plotting Baseline 01
ax = plt.subplot("311")
ax.set_title("$m_{01}$ (blue) and $d_{01}$  (green)", fontsize=18)
ax.plot(timeslots,M[0,1,:].real)
ax.plot(timeslots,D[0,1,:].real)
ax.set_xlabel("Timeslot", fontsize=18)
ax.set_ylabel("Jy", fontsize=18)
ax.set_xlim([1,len(M[0,1,:])])
y_t = ax.get_yticks()
y_t = y_t[::2]
ax.set_yticks(y_t)

#Plotting Baseline 02
ax = plt.subplot("312")
ax.set_title("$m_{02}$ (blue) and $d_{02}$ (green)", fontsize=18)
ax.plot(timeslots,M[0,2,:].real)
ax.plot(timeslots,D[0,2,:].real)
ax.set_xlabel("Timeslot", fontsize=18)
ax.set_ylabel("Jy",...


I am working on the file "8_Partial_Differential_Equations.ipynb".
The first task is:
8.1 Parabolic Equations
Forward Difference Method
Can you write Python code for it?

def heatfd(xl, xr, yb, yt, M, N):
    f = lambda x : np.power(np.sin(2 * np.pi * x), 2)
    l = lambda t : 0 * t
    r = lambda t : 0 * t
    D = 1
    h = (xr - xl) / M
    k = (yt - yb) / N
    m = M - 1
    n = N
    sigma = D * k / np.power(h, 2)
    A = np.diag(1 - 2 * sigma * np.ones(m)) + \
        np.diag(sigma * np.ones(m - 1), 1) + \
        np.diag(sigma * np.ones(m - 1), -1)

    lside = l(yb + np.arange(n) * k)
    rside = r(yb + np.arange(n) * k)

    x = sym.Symbol('x')
    expr = sym.sin(2 * sym.pi * x) ** 2
#     expr = sym.diff(expr, x)

    w = np.zeros(n * m).reshape(n, m).astype(np.float128)
    for i in range(m):
        w[0, i] = expr.subs(x, xl + (i + 1) * h).evalf()

    for j in range(n - 1):
        ww = np.zeros(m)
        ww[0] = lside[j]
        ww[-1] = rside[j]
        v = np.matmul(A, w[j]) + sigma * ww
        w[j + 1,:] = v

    w = np.column_stack([lside, w, rside])
    x = np.arange(0, m+2) * h
    t = np.arange(0, n) * k

    X, T = np.meshgrid(x, t)

    fig = plt.figure()
    ax = fig.gca(projection='3d')
    ax.plot_surface(X, T, w)
    plt.show()
    plt.clf()

heatfd(0, 1, 0, 1, 30, 2000)


I am working on the file "8_Partial_Differential_Equations.ipynb".
The first task is:
8.2 Hyperbolic Equations
Example
Apply the explicit Finite Difference Method to the wave equation with wave speed $c = 2$ and initial conditions $f(x) = \sin{\pi x}$ and $g(x) = l(x) = r(x) = 0$
Can you write Python code for it?

def wavefd(xl, xr, yb, yt, M, N):
    c = 1
    h = (xr - xl) / M
    k = (yt - yb) / N
    m = M - 1
    n = N

    sigma = c * k / h

    f = lambda x : np.sin(x * np.pi)
    l = lambda x : 0 * x
    r = lambda x : 0 * x
    g = lambda x : 0 * x

    lside = l(yb + np.arange(n) * k)
    rside = r(yb + np.arange(n) * k)

    A = np.diag((2 - 2 * sigma ** 2) * np.ones(m)) + \
        np.diag((sigma ** 2) * np.ones(m - 1), 1) + \
        np.diag((sigma ** 2) * np.ones(m - 1), -1)

    '''Initial condition'''
    w = np.zeros(n * m).reshape(n, m).astype(np.float128)
    xv = np.linspace(0, 1, M + 1)[1:-1]
    w[0, :] = f(xv)
    w[1, :] = 0.5 * np.matmul(A, w[0, :]) + \
              k * g(xv) + \
              0.5 * np.power(sigma, 2) * np.array([lside[0], *np.zeros(m - 2), rside[0]])

    for i in range(2, n - 1):
        w[i,:] = np.matmul(A, w[i-1,:]) - w[i-2,:] + np.power(sigma, 2) * \
                 np.array([lside[i-1], *np.zeros(m - 2), rside[i-1]])

    w = np.column_stack([lside, w, rside])
    x = xl + np.arange(0, m + 2) * h
    t = yb + np.arange(0, n) * k

    X, T = np.meshgrid(x, t)

    fig = plt.figure()
    ax = fig.gca(projection='3d')
    stride = 0
    ax.plot_wireframe(X[stride:], T[stride:], w[stride:])
#     ax.plot_surface(X[stride:], T[stride:], w[stride:])
    ax.view_init(azim=60, elev=30)
    plt.xlabel('x')
    plt.ylabel('t')
    plt.show()
    plt.clf()

wavefd(0, 1, 0, 1, 20, 20)


I am working on the file "8_Partial_Differential_Equations.ipynb".
The first task is:
8.4 Nonlinear partial differential equations
Example
Use the Backward Difference Equation with Newton iteration to solve Burgers' equation
$$
\left{\begin{matrix}\begin{align}
& u_t + uu_x = Du_{xx} \
& u(x,0) = \frac{2D\beta\pi\sin{\pi x}}{\alpha + \beta\cos{\pi x}}\text{ for } 0 \le x \le 1 \
& u(0,t) = 0\text{ for all } t \ge 0 \
& u(1,t) = 0\text{ for all } t \ge 0
\end{align}\end{matrix}\right.
$$
Can you write Python code for it?

def burgers(xl, xr, tb, te, M, N):
    alpha = 5
    beta = 4
    D = 0.05
    f = lambda x : 2 * D * beta * np.pi * np.sin(x * np.pi) / (alpha + beta * np.cos(np.pi * x))
    l = lambda t : 0 * t
    r = lambda t : 0 * t

    h, k = (xr - xl) / M, (te - tb) / N
    m, n = M + 1, N
    sigma = D * k / (h * h)

    w = np.zeros((M + 1) * (n + 1)).reshape(M + 1, n + 1)
    w[:, 0] = f(xl + np.arange(M + 1) * h)
    w1 = np.copy(w[:, 0])

    for j in range(0, n):
        for it in range(3):
            DF1 = np.diag(1 + 2 * sigma * np.ones(m)) + np.diag(-sigma * np.ones(m-1), 1) \
                + np.diag(-sigma * np.ones(m-1), -1)
            DF2 = np.diag([0,*(k * w1[2:m] / (2 * h)),0]) - np.diag([0,*(k * w1[0:m - 2] / (2 * h)),0]) \
                + np.diag([0,*(k * w1[1:m - 1] / (2 * h))], 1) - np.diag([*(k * w1[1:m - 1] / (2 * h)), 0], -1)
            DF = DF1 + DF2;
            F = -w[:,j] + np.matmul((DF1 + DF2 / 2), w1)
            DF[0,:] = np.array([1, *np.zeros(m-1)])
            F[0] = w1[0] - l(j)
            F[m-1] = w1[m-1] - r(j)
            w1 -= np.matmul(np.linalg.inv(DF), F)
        w[:, j + 1] = w1

    # 3-D Plot
    x = xl + np.arange(M + 1) * h
    t = tb + np.arange(n + 1) * k
    X, T = np.meshgrid(x, t)
    fig = plt.figure()
    ax = fig.gca(projection='3d')
    ax.view_init(azim=225)
    ax.plot_surface(X, T, w.T)
    plt.xlabel('x')
    plt.ylabel('t')
    plt.show()
    plt.close()

burgers(0, 1, 0, 2, 20, 40)


I am working on the file "8_Partial_Differential_Equations.ipynb".
The first task is:
Example
Apply the Backward Difference Method to solve the heat equation with homogeneous Neumann boundary conditions
$$
\left{\begin{matrix}\begin{align}
& u_t = u_{xx}\: & for\:all\:0 \leq x \leq 1\:,0 \leq t \leq 1 \
& u(x,0) = \sin^2{2\pi x}\: & for\:all\:0 \leq x \leq 1 \
& u(0,t) = 0\: & for\:all\:0 \leq t \leq 1 \
& u(1,t) = 0\: & for\:all\:0 \leq t \leq 1
\end{align}\end{matrix}\right.
$$
Can you write Python code for it?

def heatbdn(xl, xr, yb, yt, M, N):
    h = (xr - xl) / M
    k = (yt - yb) / N
    m = M - 1
    n = N
    D = 1 # diffusion coefficient
    sigma = D * k / (h ** 2)

    A = np.diag((1 + 2 * sigma) * np.ones(m)) + \
        np.diag(-sigma * np.ones(m - 1), 1) + \
        np.diag(-sigma * np.ones(m - 1), -1)

    A[0,:3] = np.array([-3, 4, -1])
    A[-1,-3:] = np.array([-1, 4, -3])

    ''' Initial conditions '''
    x = sym.Symbol('x')
    expr = sym.sin(2 * sym.pi * x) ** 2
#     expr = sym.diff(expr, x)

    w = np.zeros(n * m).reshape(n, m).astype(np.float128)
    for i in range(m):
        w[0, i] = expr.subs(x, xl + (i + 1) * h).evalf()

    for j in range(n - 1):
        b = w[j,:]
        b[0] = 0
        b[-1] = 0
        w[j + 1,:] = np.matmul(np.linalg.inv(A), b)

    x = np.arange(0, m) * h
    t = np.arange(0, n) * k

    X, T = np.meshgrid(x, t)

    fig = plt.figure()
    ax = fig.gca(projection='3d')
    stride = 0
    ax.plot_surface(X[stride:], T[stride:], w[stride:])
    plt.xlabel('x')
    plt.ylabel('y')
    plt.show()
    plt.clf()

heatbdn(0, 1, 0, 1, 20, 20)


I am working on the file "8_Partial_Differential_Equations.ipynb".
The first task is:
Example
Apply the Crank-Nicolson Method to the heat equation
$$
\left{\begin{matrix}\begin{align}
& u_t = Du_{xx} + Cu \
& u(x,0) = \sin^2{(\frac{\pi}{L} x)}\: &for\:all\:0 \leq x \leq L \
& u(0,t) = 0\: &for\:all\:t \geq 0 \
& u(L,t) = 0\: &for\:all\:t \geq 0
\end{align}\end{matrix}\right.
$$
Can you write Python code for it?

def crank_nicolson_growth(xl, xr, yb, yt, M, N):
    l = lambda t : 0 * t
    r = lambda t : 0 * t
    D = 1
    L = 1
    C = 9.5
    h = (xr - xl) / M
    k = (yt - yb) / N
    m = M - 1
    n = N
    sigma = D * k / h ** 2

    A = np.diag((2 - k * C + 2 * sigma) * np.ones(m)) + \
        np.diag(-sigma * np.ones(m - 1), 1) + \
        np.diag(-sigma * np.ones(m - 1), -1)

    B = np.diag((2 + k * C - 2 * sigma) * np.ones(m)) + \
        np.diag(sigma * np.ones(m - 1), 1) + \
        np.diag(sigma * np.ones(m - 1), -1)

    lside = l(yb + np.arange(n) * k)
    rside = r(yb + np.arange(n) * k)

    ''' Initial conditions '''
    f = lambda x : np.power(np.sin(np.pi * x / L), 2)

    w = np.zeros(n * m).reshape(n, m).astype(np.float128)
    for i in range(m):
        w[0, i] = f(xl + (i + 1) * h)

    for j in range(n - 1):
        s = np.zeros(m)
        s[0] = lside[j] + lside[j+1]
        s[-1] = rside[j] + rside[j+1]
        w[j + 1,:] = np.matmul(np.linalg.inv(A), np.matmul(B, w[j,:]) + sigma * s)

    w = np.column_stack([lside, w, rside])
    x = xl + np.arange(0, m+2) * h
    t = yb + np.arange(0, n) * k

    X, T = np.meshgrid(x, t)

    fig = plt.figure()
    ax = fig.gca(projection='3d')
    stride = 0
    ax.plot_surface(X[stride:], T[stride:], w[stride:])
    plt.xlabel('x')
    plt.ylabel('t')
    plt.show()
    plt.clf()

crank_nicolson_growth(0, 1, 0, 1, 20, 20)


I am working on the file "8_Partial_Differential_Equations.ipynb".
The first task is:
Example
Find the electrostatic potential on the square $[0,1] \times [0,1]$, assuming no charge in the interior and assuming the following boundary conditions:
$$
\begin{matrix}\begin{align}
u(x,0) &= \sin{\pi x} \
u(x,1) &= \sin{\pi x} \
u(0,y) &= 0 \
u(1,y) &= 0
\end{align}\end{matrix}
$$
Can you write Python code for it?

def poisson(xl, xr, yb, yt, M, N):
    f = lambda x, y : 0
    g1 = lambda x : np.sin(x * np.pi)
    g2 = lambda x : np.sin(x * np.pi)
    g3 = lambda y : 0
    g4 = lambda y : 0

    m, n = M + 1, N + 1
    mn = m * n
    h, k = (xr - xl) / M, (yt - yb) / N
    h2, k2 = pow(h, 2), pow(k, 2)
    x = xl + np.arange(M + 1) * h
    y = yb + np.arange(N + 1) * k
    A = np.zeros((mn, mn))
    b = np.zeros((mn, 1))

    ''' interior points '''
    for i in range(2, m):
        for j in range(2, n):
            A[i+(j-1)*m - 1][i-1+(j-1)*m - 1] = 1 / h2
            A[i+(j-1)*m - 1][i+1+(j-1)*m - 1] = 1 / h2
            A[i+(j-1)*m - 1][i+(j-1)*m - 1] = - 2 / h2 - 2 / k2
            A[i+(j-1)*m - 1][i+(j-2)*m - 1] = 1 / k2
            A[i+(j-1)*m - 1][i+j*m - 1] = 1 / k2
            b[i+(j-1)*m - 1] = f(x[i], y[j])

    ''' bottom and top boundary points '''
    for i in range(1, m + 1):
        j = 1
        A[i+(j-1)*m - 1][i+(j-1)*m - 1]=1
        b[i+(j-1)*m - 1] = g1(x[i - 1])
        j = n
        A[i+(j-1)*m - 1][i+(j-1)*m - 1]=1
        b[i+(j-1)*m - 1] = g2(x[i - 1])

    ''' left and right boundary points '''
    for j in range(2, n):
        i = 1
        A[i+(j-1)*m - 1][i+(j-1)*m - 1]=1
        b[i+(j-1)*m - 1] = g3(y[j - 1])
        i = m
        A[i+(j-1)*m - 1][i+(j-1)*m - 1]=1
        b[i+(j-1)*m - 1] = g4(y[j - 1])

    v = np.matmul(np.linalg.inv(A), b)
    w = v.reshape(n, m).T

    X, Y = np.meshgrid(x, y)

    fig = plt.figure()
    ax =fig.gca(projection='3d')
    ax.view_init(azim=225)
    ax.plot_surface(X, Y, w)
    plt.xlabel('x')
    plt.ylabel('y')
    plt.show(fig)
    plt.close()

poisson(0, 1, 0, 1, 10, 10)


I am working on the file "8_Partial_Differential_Equations.ipynb".
The first task is:
Finite Element Method for elliptic equations
Can you write Python code for it?


I am working on the file "9.1 Out Of Core Learning.ipynb".
The first task is:
Kernel Approximations
Can you write Python code for it?

from sklearn.kernel_approximation import RBFSampler

sgd = SGDClassifier()
kernel_approximation = RBFSampler(gamma=.001, n_components=400)

for i in range(9):
    X_batch, y_batch = cPickle.load(open("data/batch_%02d.pickle" % i))
    if i == 0:
        kernel_approximation.fit(X_batch)
    X_transformed = kernel_approximation.transform(X_batch)
    sgd.partial_fit(X_transformed, y_batch, classes=range(10))

X_test, y_test = cPickle.load(open("data/batch_09.pickle"))

sgd.score(kernel_approximation.transform(X_test), y_test)


I am working on the file "9.caffe-ssd/examples/inceptionv3.ipynb".
The first task is:
2. Load net and set up input preprocessing

Set Caffe to CPU mode and load the net from disk.
Can you write Python code for it?

caffe.set_device(0)  # if we have multiple GPUs, pick the first one
caffe.set_mode_gpu()

model_def = caffe_root + 'models/inception_v3/inception_v3_deploy.prototxt'
model_weights = caffe_root + 'models/inception_v3/inception_v3.caffemodel'

net = caffe.Net(model_def,      # defines the structure of the model
                model_weights,  # contains the trained weights
                caffe.TEST)     # use test mode (e.g., don't perform dropout)


I am working on the file "9.caffe-ssd/examples/inceptionv3.ipynb".
The first task is:
3. CPU classification

Now we're ready to perform classification. Even though we'll only classify one image, we'll set a batch size of 50 to demonstrate batching.
Can you write Python code for it?

# set the size of the input (we can skip this if we're happy
#  with the default; we can also change it later, e.g., for different batch sizes)
net.blobs['data'].reshape(1,        # batch size
                          3,         # 3-channel (BGR) images
                          299, 299)  # image size is 227x227


I am working on the file "9.caffe-ssd/examples/inceptionv3.ipynb".
The first task is:
Now look at the parameter shapes. The parameters are exposed as another OrderedDict, net.params. We need to index the resulting values with either [0] for weights or [1] for biases.
The param shapes typically have the form (output_channels, input_channels, filter_height, filter_width) (for the weights) and the 1-dimensional shape (output_channels,) (for the biases).
Can you write Python code for it?

for layer_name, param in net.params.iteritems():
    if 'bn' in layer_name:
        print layer_name + '\t' + str(param[0].data.shape), str(param[1].data.shape), str(param[2].data.shape)
    elif 'scale' in layer_name or layer_name == 'softmax':
        print layer_name + '\t' + str(param[0].data.shape), str(param[1].data.shape)
    else:
        print layer_name + '\t' + str(param[0].data.shape)


I am working on the file "9.caffe-ssd/examples/inceptionv3.ipynb".
The first task is:
Since we're dealing with four-dimensional data here, we'll define a helper function for visualizing sets of rectangular heatmaps.
Can you write Python code for it?

def vis_square(data):
    """Take an array of shape (n, height, width) or (n, height, width, 3)
       and visualize each (height, width) thing in a grid of size approx. sqrt(n) by sqrt(n)"""

    # normalize data for display
    data = (data - data.min()) / (data.max() - data.min())

    # force the number of filters to be square
    n = int(np.ceil(np.sqrt(data.shape[0])))
    padding = (((0, n ** 2 - data.shape[0]),
               (0, 1), (0, 1))                 # add some space between filters
               + ((0, 0),) * (data.ndim - 3))  # don't pad the last dimension (if there is one)
    data = np.pad(data, padding, mode='constant', constant_values=1)  # pad with ones (white)

    # tile the filters into an image
    data = data.reshape((n, n) + data.shape[1:]).transpose((0, 2, 1, 3) + tuple(range(4, data.ndim + 1)))
    data = data.reshape((n * data.shape[1], n * data.shape[3]) + data.shape[4:])

    plt.imshow(data); plt.axis('off')


I am working on the file "9.caffe-ssd/examples/inceptionv3.ipynb".
The first task is:
The first fully connected layer, fc6 (rectified)
We show the output values and the histogram of the positive values
Can you write Python code for it?

feat = net.blobs['pool_3'].data[0]
plt.subplot(2, 1, 1)
plt.plot(feat.flat)
plt.subplot(2, 1, 2)
_ = plt.hist(feat.flat[feat.flat > 0], bins=100)


I am working on the file "9.caffe-ssd/examples/inceptionv3.ipynb".
The first task is:
We see that less confident predictions are sensible.

5. Examining intermediate output

A net is not just a black box; let's take a look at some of the parameters and intermediate activations.

First we'll see how to read out the structure of the net in terms of activation and parameter shapes.

For each layer, let's look at the activation shapes, which typically have the form (batch_size, channel_dim, height, width).
The activations are exposed as an OrderedDict, net.blobs.
Can you write Python code for it?

# for each layer, show the output shape
for layer_name, blob in net.blobs.iteritems():
    print layer_name + '\t' + str(blob.data.shape)


I am working on the file "9.caffe-ssd/examples/ssd.ipynb".
The first task is:
And let's visualize conv5_3 layer responses. You should see nicely pattern.
Can you write Python code for it?

feat = net.blobs['conv5_3'].data[0, :]
vis_square(feat, padval=1)


I am working on the file "9.caffe-ssd/examples/ssd.ipynb".
The first task is:
Let's visualize the conv1_1's filters to make sure we have loaded a good pretrained model. Otherwise, it should plot random noise squares.
Can you write Python code for it?

# the parameters are a list of [weights, biases]
filters = net.params['conv1_1'][0].data
vis_square(filters.transpose(0, 2, 3, 1))


I am working on the file "9.caffe-ssd/examples/ssd.ipynb".
The first task is:
Make sure that the PermuteLayer is doing the right thing.
Can you write Python code for it?

fc7_mbox_loc = net.blobs['fc7_mbox_loc'].data
print fc7_mbox_loc[0,:,1,2]

fc7_mbox_loc_perm = net.blobs['fc7_mbox_loc_perm'].data
print fc7_mbox_loc_perm[0,1,2,:]


I am working on the file "9.caffe-ssd/examples/ssd.ipynb".
The first task is:
Make sure the PriorBoxLayer generates the correct priors.
Can you write Python code for it?

img_blob = net.blobs['data'].data
num_imgs = img_blob.shape[0]
img_height = img_blob.shape[2]
img_width = img_blob.shape[3]

priorbox = net.blobs['mbox_priorbox'].data[0,0,:]
num_priors = priorbox.shape[0]

colors='rgbcmy'
for i in xrange(num_imgs):
    img = transformer.deprocess('data', img_blob[i])
    plt.subplot(1, num_imgs, i + 1)
    plt.imshow(img)
    currentAxis = plt.gca()
    for j in xrange(240,243):
        prior_bbox = priorbox[j*4:(j+1)*4]
        xmin = prior_bbox[0] * img_width
        ymin = prior_bbox[1] * img_height
        xmax = prior_bbox[2] * img_width
        ymax = prior_bbox[3] * img_height
        coords = (xmin, ymin), xmax - xmin + 1, ymax - ymin + 1
        currentAxis.add_patch(plt.Rectangle(*coords, fill=False, edgecolor=colors[j%4], linewidth=2))


I am working on the file "9.caffe-ssd/examples/ssd.ipynb".
The first task is:
Now let's see the annotated datum after one forward-backward step.
Can you write Python code for it?

colors = plt.cm.hsv(np.linspace(0, 1, 21)).tolist()

img_blob = net.blobs['data'].data
num_imgs = img_blob.shape[0]
img_height = img_blob.shape[2]
img_width = img_blob.shape[3]
label_blob = net.blobs['label'].data[0,0,:,:]
num_labels = label_blob.shape[0]

for i in xrange(num_imgs):
    img = transformer.deprocess('data', img_blob[i])
    plt.subplot(1, num_imgs, i + 1)
    plt.imshow(img)
    currentAxis = plt.gca()
    for j in xrange(num_labels):
        gt_bbox = label_blob[j, :]
        if gt_bbox[0] == i:
            xmin = gt_bbox[3] * img_width
            ymin = gt_bbox[4] * img_height
            xmax = gt_bbox[5] * img_width
            ymax = gt_bbox[6] * img_height
            gt_label = int(gt_bbox[1])
            coords = (xmin, ymin), xmax - xmin + 1, ymax - ymin + 1
            color = colors[gt_label]
            currentAxis.add_patch(plt.Rectangle(*coords, fill=False, edgecolor=color, linewidth=2))
            label = get_labelname(labelmap, gt_bbox[1])[0]
            currentAxis.text(xmin, ymin, label, bbox={'facecolor':color, 'alpha':0.5})

# take an array of shape (n, height, width) or (n, height, width, channels)
# and visualize each (height, width) thing in a grid of size approx. sqrt(n) by sqrt(n)
def vis_square(data, padsize=1, padval=0):
    data -= data.min()
    data /= data.max()

    # force the number of filters to be square
    n = int(np.ceil(np.sqrt(data.shape[0])))
    padding = ((0, n ** 2 - data.shape[0]), (0, padsize), (0, padsize)) + ((0, 0),) * (data.ndim - 3)
    data = np.pad(data, padding, mode='constant', constant_values=(padval, padval))

    # tile the filters into an image
    data = data.reshape((n, n) + data.shape[1:]).transpose((0, 2, 1, 3) + tuple(range(4, data.ndim + 1)))
    data = data.reshape((n * data.shape[1], n * data.shape[3]) + data.shape[4:])

    plt.imshow(data)


I am working on the file "9.caffe-ssd/examples/ssd_detect.ipynb".
The first task is:
2. SSD detection

Load an image.
Can you write Python code for it?

# set net to batch size of 1
image_resize = 300
net.blobs['data'].reshape(1,3,image_resize,image_resize)

image = caffe.io.load_image('examples/images/fish-bike.jpg')
plt.imshow(image)


I am working on the file "9.caffe-ssd/examples/ssd_detect.ipynb".
The first task is:
Load LabelMap.
Can you write Python code for it?

from google.protobuf import text_format
from caffe.proto import caffe_pb2

# load PASCAL VOC labels
labelmap_file = 'data/VOC0712/labelmap_voc.prototxt'
file = open(labelmap_file, 'r')
labelmap = caffe_pb2.LabelMap()
text_format.Merge(str(file.read()), labelmap)

def get_labelname(labelmap, labels):
    num_labels = len(labelmap.item)
    labelnames = []
    if type(labels) is not list:
        labels = [labels]
    for label in labels:
        found = False
        for i in xrange(0, num_labels):
            if label == labelmap.item[i].label:
                found = True
                labelnames.append(labelmap.item[i].display_name)
                break
        assert found == True
    return labelnames


I am working on the file "9.caffe-ssd/examples/ssd_detect.ipynb".
The first task is:
Load the net in the test phase for inference, and configure input preprocessing.
Can you write Python code for it?

model_def = 'models/VGGNet/VOC0712/SSD_300x300/deploy.prototxt'
model_weights = 'models/VGGNet/VOC0712/SSD_300x300/VGG_VOC0712_SSD_300x300_iter_60000.caffemodel'

net = caffe.Net(model_def,      # defines the structure of the model
                model_weights,  # contains the trained weights
                caffe.TEST)     # use test mode (e.g., don't perform dropout)

# input preprocessing: 'data' is the name of the input blob == net.inputs[0]
transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})
transformer.set_transpose('data', (2, 0, 1))
transformer.set_mean('data', np.array([104,117,123])) # mean pixel
transformer.set_raw_scale('data', 255)  # the reference model operates on images in [0,255] range instead of [0,1]
transformer.set_channel_swap('data', (2,1,0))  # the reference model has channels in BGR order instead of RGB


I am working on the file "9_Random_Numbers_And_Applications.ipynb".
The first task is:
9.1 Random Numbers
A linear congruential generator (LCG) has form
$$
\begin{align}
x_i &= ax_{i-1} + b\ (mod\ m) \
u_i &= \frac{x_i}{m}
\end
{align}
$$
for multiplier a, offset b, and modulus m
Can you write Python code for it?

def linear_congruential_generator(x, a, b, m):
    x = (a * x + b) % m
    u = x / m
    return u, x, a, b, m

x0 = 3
args = (x0, 13, 0, 31)
for i in range(10):
    u, *args = linear_congruential_generator(*args)
    print('idx_%02d x:%02d, u:%.4f' %(i + 1, args[0], u))


I am working on the file "9_Random_Numbers_And_Applications.ipynb".
The first task is:
9.1.2 Exponential and normal random numbers
see numpy.random.normal function
Can you write Python code for it?

datax = np.array([])
for i in range(10000):
    u1 = np.random.normal()
    datax = np.append(datax, u1)

plt.plot(datax)


I am working on the file "9_Random_Numbers_And_Applications.ipynb".
The first task is:
9.2 Monte Carlo Simulation
9.2.2 Quasi-random numbers
base-p low-discrepancy sequence
Can you write Python code for it?

def halton(p, n):
    b = np.zeros(math.ceil(math.log(n + 1) / math.log(p)))
    u = np.zeros(n)
    for j in range(n):
        i = 0
        b[0] = b[0] + 1
        while b[i] > p - 1 + np.finfo(float).eps:
            b[i] = 0
            i += 1
            b[i] += 1
        u[j] = 0
        for k in range(1, b.size + 1):
            u[j] = u[j] + b[k-1] * pow(p, -k)
    return u

# Example
print(halton(2, 8))
print(halton(3, 8))


I am working on the file "9_Random_Numbers_And_Applications.ipynb".
The first task is:
9.3 Discrete And Continuous Brownian Motion
9.3.1 Random walks
Can you write Python code for it?

t = 10
w = 0
for i in range(t):
    if random.random() > 0.5:
        w += 1
    else:
        w -= 1


I am working on the file "9_Random_Numbers_And_Applications.ipynb".
The first task is:
9.4 Stochastic Differential Equations
Definition
A set of random variables $x_t$ indexed by real numbers $t >= 0$ is called a contimuous-time stochastic process
Brownian Motion
Source from http://scipy-cookbook.readthedocs.io/items/BrownianMotion.html
Can you write Python code for it?

"""
brownian() implements one dimensional Brownian motion (i.e. the Wiener process).
"""

# File: brownian.py

from math import sqrt
from scipy.stats import norm
import numpy as np

def brownian(x0, n, dt, delta, out=None):
    """
    Generate an instance of Brownian motion (i.e. the Wiener process):

        X(t) = X(0) + N(0, delta**2 * t; 0, t)

    where N(a,b; t0, t1) is a normally distributed random variable with mean a and
    variance b.  The parameters t0 and t1 make explicit the statistical
    independence of N on different time intervals; that is, if [t0, t1) and
    [t2, t3) are disjoint intervals, then N(a, b; t0, t1) and N(a, b; t2, t3)
    are independent.

    Written as an iteration scheme,

        X(t + dt) = X(t) + N(0, delta**2 * dt; t, t+dt)

    If `x0` is an array (or array-like), each value in `x0` is treated as
    an initial condition, and the value returned is a numpy array with one
    more dimension than `x0`.

    Arguments
    ---------
    x0 : float or numpy array (or something that can be converted to a numpy array
         using numpy.asarray(x0)).
        The initial condition(s) (i.e. position(s)) of the Brownian motion.
    n : int
        The number of steps to take.
    dt : float
        The time step.
    delta : float
        delta determines the "speed" of the Brownian motion.  The random variable
        of the position at time t, X(t), has a normal distribution whose mean is
        the position at time t=0 and whose variance is delta**2*t.
    out : numpy array or None
        If `out` is not None, it specifies the array in which to put the
        result.  If `out` is None, a new numpy array is created and returned.

    Returns
    -------
    A numpy array of floats with shape `x0.shape + (n,)`.

    Note that the initial value `x0` is not included in the returned array.
    """

    x0 = np.asarray(x0)

    # For each element of x0, generate a sample of n numbers from a
    # normal distribution.
    r = norm.rvs(size=x0.shape + (n,),...


I am working on the file "9_Random_Numbers_And_Applications.ipynb".
The first task is:
Euler-Maruyama Method
Given the SDE initial value problem
$\left{\begin{matrix}
dy(t) & = & f(t,y)dt & + & g(t,y)dB_t \
y(a) & = & y_a
\end{matrix}\right.$

$w_0 = y_0$
$\mathbf{for}\ i = 0,1,2,\cdots$
$\hspace{10mm} w_{i+1} = w_i + f(t_i,w_i)(\Delta t_i)+g(t_i,w_i)(\Delta B_i)$
$\mathbf{end}$
$where$
$\hspace{10mm} \Delta t_i = t_{i+1} - t_i$
$\hspace{10mm} \Delta B_i = B_{t_{i+1}} - B_{t_i}$

Each random number $\Delta B_i$ is computed by $\Delta B_i = z_i\sqrt{\Delta t_i}$ where $z_i$ is chosen from $N(0, 1)$
Example
Once again, solve the stochastic differential equation $dy(t) = rdt+\sigma B_t$ with initial condition $y(0) = y_0$
by Euler-Maruyama method

The Euler-Maruyama Method has form
$w_0 = y_0$
$w_{i+1} = w_i + rw_i(\Delta t_i)+\sigma w_i (\Delta B_i)$
Can you write Python code for it?

dt = 0.01
xlimit = 2
y0 = 1
r = 0.1
sigma = 0.3
times = np.arange(0, xlimit + dt, dt)
dB = np.random.standard_normal(times.size) * np.sqrt(dt)
ws = np.empty(times.size)
ws[0] = y0

for i in range(times.size - 1):
    ws[i + 1] = ws[i] + r * ws[i] * dt + sigma * ws[i] * dB[i]

# Plot the chart
plt.plot(times, ws)
plt.axhline(y=0, color='black')
plt.axvline(x=0, color='black')
plt.grid(True, which='both')


I am working on the file "9_Random_Numbers_And_Applications.ipynb".
The first task is:
Example
Monte Carlo Type 1 problem (reduced to a function average)
Approximate the area under the curve $y = x^2 \ in \ [0,\ 1]$

Evaluate $\frac{1}{b-a}\int_{a}^{b}f(x)dx$
Can you write Python code for it?

x = sympy.symbols('x')
exact_value = sympy.integrate(x ** 2, (x, 0, 1))

# Arguments for our LCG
x0 = 3
args = (x0, 13, 0, 31)

# Function and arguments for the curve y = x^2
f = lambda x : pow(x, 2)

# Process for this example
def process(f, args, total_iterations):
    avg = 0
    for i in range(total_iterations):
        u, *args = linear_congruential_generator(*args)
        avg += f(u)
    avg /= total_iterations
    return avg

print('exact value = %s (%.6f in numerical representations)' %(exact_value, exact_value.evalf()))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, args, 10), 10, abs(process(f, args, 10) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, args, 20), 20, abs(process(f, args, 20) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, args, 30), 30, abs(process(f, args, 30) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, args, 40), 40, abs(process(f, args, 40) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, args, 50), 50, abs(process(f, args, 50) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, args, 60), 60, abs(process(f, args, 60) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, args, 70), 70, abs(process(f, args, 70) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, args, 80), 80, abs(process(f, args, 80) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, args, 90), 90, abs(process(f, args, 90) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, args, 100), 100, abs(process(f, args, 100) - exact_value.evalf())))
print('average = %.6f with %3d...


I am working on the file "9_Random_Numbers_And_Applications.ipynb".
The first task is:
Example
Numerically solve the Brownian bridge SDE
$\left{\begin{matrix}
dy & = & \frac{y_1-y}{t_1-t}dt & + & dB_t\
y(t_0) & = & y_0 &
\end{matrix}\right.$
where $y_1$ and $t_1 > t_0$ are given
Can you write Python code for it?

dt = 0.01
t0, t1 = 1, 3
y0, y1 = 1, 2
times = np.arange(t0, t1 + dt * 1, dt)
dB1 = np.random.standard_normal(times.size) * np.sqrt(dt)
dB1[-2] = 0
ws1 = np.empty(times.size)
ws1[0] = y0
dB2 = np.random.standard_normal(times.size) * np.sqrt(dt)
dB2[-2] = 0
ws2 = np.empty(times.size)
ws2[0] = y0
dB3 = np.random.standard_normal(times.size) * np.sqrt(dt)
dB3[-2] = 0
ws3 = np.empty(times.size)
ws3[0] = y0

# Let's use Euler-Maruyama Method
for i in range(times.size - 1):
    ws1[i + 1] = ws1[i] + (y1 - ws1[i]) * dt / (dt * (times.size - i - 1)) + dB1[i]
    ws2[i + 1] = ws2[i] + (y1 - ws2[i]) * dt / (dt * (times.size - i - 1)) + dB2[i]
    ws3[i + 1] = ws3[i] + (y1 - ws3[i]) * dt / (dt * (times.size - i - 1)) + dB3[i]

# Plot the chart
plt.plot(times, ws1)
plt.plot(times, ws2)
plt.plot(times, ws3)
plt.plot(t0, y0, marker='o', color='k')
plt.plot(t1, y1, marker='o', color='k')
plt.grid(True)
plt.show()


I am working on the file "9_Random_Numbers_And_Applications.ipynb".
The first task is:
Example
Once again, approximate the area under the curve $y = x^2 \ in \ [0,\ 1]$
Can you write Python code for it?

# Function and arguments for the curve y = x^2
f = lambda x : pow(x, 2)

# Process for this example
def process(f, total_iterations):
    avg = 0
    x = 3
    for i in range(total_iterations):
        u, x = stdrand(x)
        avg += f(u)
    avg /= total_iterations
    return avg

print('exact value = %s (%.6f in numerical representations)' %(exact_value, exact_value.evalf()))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, 10), 10, abs(process(f, 10) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, 20), 20, abs(process(f, 20) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, 30), 30, abs(process(f, 30) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, 40), 40, abs(process(f, 40) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, 50), 50, abs(process(f, 50) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, 60), 60, abs(process(f, 60) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, 70), 70, abs(process(f, 70) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, 80), 80, abs(process(f, 80) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, 90), 90, abs(process(f, 90) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, 100), 100, abs(process(f, 100) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, 200), 200, abs(process(f, 200) - exact_value.evalf())))
print('average = %.6f with %3d uniform random numbers, error = %.6f' %(process(f, 300), 300, abs(process(f, 300) - exact_value.evalf())))
print('average = %.6f with %3d...


I am working on the file "9_Random_Numbers_And_Applications.ipynb".
The first task is:
For its visualization (from https://www.desmos.com/calculator)

The randu generator (infamous for its dependent random number sequence, proof is skipped)
$$
\begin{align}
x_i &= ax_{i-1}\ (mod\ m) \
u_i &= \frac{x_i}{m}
\end
{align}
$$
where $a = 65539 = 2^{16} + 3\ and\ m = 2^{31}$
Can you write Python code for it?

def randu(x):
    return linear_congruential_generator(x, pow(2, 16) + 3, 0, pow(2, 31))[:2]

# For matplotlib
fig = plt.figure()
ax = fig.gca(projection='3d')
ax.view_init(azim=225)
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.set_zlim(0, 1)
ax.set_xlabel('X')
ax.set_ylabel('Y')
ax.set_zlabel('Z')

# Arguments for randu
datax = np.array([])
datay = np.array([])
dataz = np.array([])
x = 3
total_iterations = 20000

# Process
for i in range(total_iterations):
    u1, x = randu(x)
    u2, x = randu(x)
    u3, x = randu(x)
    datax = np.append(datax, u1)
    datay = np.append(datay, u2)
    dataz = np.append(dataz, u3)

ax.scatter(datax, datay, dataz, zdir='z', s=2)

# For matplotlib
fig = plt.figure()
ax = fig.gca(projection='3d')
ax.view_init(azim=225)
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.set_zlim(0, 1)
ax.set_xlabel('X')
ax.set_ylabel('Y')
ax.set_zlabel('Z')

# Arguments for randu
datax = np.array([])
datay = np.array([])
dataz = np.array([])
x = 3
total_iterations = 20000

# Process
for i in range(total_iterations):
    u1, x = stdrand(x)
    u2, x = stdrand(x)
    u3, x = stdrand(x)
    datax = np.append(datax, u1)
    datay = np.append(datay, u2)
    dataz = np.append(dataz, u3)

ax.scatter(datax, datay, dataz, zdir='z', s=2)


I am working on the file "9_Random_Numbers_And_Applications.ipynb".
The first task is:
Ito formula
If $y = f(t,x)$, then $dy = \frac{\partial{f}}{\partial{t}}(t,x)dt + \frac{\partial{f}}{\partial{x}}(t,x)dx + \frac{1}{2}\frac{\partial^2{f}}{\partial{x^2}}(t,x)dxdx$, where the $dxdx$ term is interpreted by using identities $dtdt = 0$, $dtdB_tdt = 0$, and $dB_tdB_t = dt$.
Example
Show that geometric Brownian motion $y(t) = y_0e^{(r-\frac{1}{2}\sigma^2)t+\sigma B_t}$ <br>
satis
Can you write Python code for it?

N = 500
xlim = 2.0
r = 0.1
sigma = 0.3
delta = 0.1
dt = 0.2
y0 = 1
X = np.linspace(0, xlim, N)

# For Brownian motion
B = brownian(0, N, dt, delta)

# Process
Y = y0 * np.exp((r - 0.5 * pow(sigma, 2)) * X + sigma * B)
plt.plot(X, Y)
plt.plot(X, B, linestyle = '--')
plt.grid(True)


I am working on the file "9_Random_Numbers_And_Applications.ipynb".
The first task is:
pseudo-random vs quasi-random
Can you write Python code for it?

pair_count = 2000

pr_xdata = np.array([])
pr_ydata = np.array([])
qr_xdata = np.array([])
qr_ydata = np.array([])

qrx_seq = halton(2, pair_count)
qry_seq = halton(3, pair_count)

x = time.time()

for idx in range(pair_count):
    ux, x = stdrand(x)
    uy, x = stdrand(x)
    pr_xdata = np.append(pr_xdata, ux)
    pr_ydata = np.append(pr_ydata, uy)
    qr_xdata = np.append(qr_xdata, qrx_seq[idx])
    qr_ydata = np.append(qr_ydata, qry_seq[idx])

plt.figure(1)
plt.xlim(0, 1)
plt.ylim(0, 1)
plt.subplot(121)
plt.plot(pr_xdata, pr_ydata, 'o', markersize=1)
plt.subplot(122)
plt.plot(qr_xdata, qr_ydata, 'o', markersize=1)


I am working on the file "9b_PyTDA_Demo-AMS_OSRSC.ipynb".
The first task is:
And now let's plot.
Can you write Python code for it?

plot_list_of_fields(radar, sweep=4, xlim=[-50, 50], ylim=[50, 150],
                    fields=['DZ', 'VR', 'SW', 'turbulence'],
                    vmins=[0, -30, 0, 0], vmaxs=[60, 30, 10, 1.0],
                    units=['dBZ', 'm/s', 'm/s', 'EDR^0.33'],
                    cmaps=['pyart_LangRainbow12', 'seismic', 'YlOrRd', 'cubehelix'])

pyart.io.write_cfradial('file_with_turbulence.nc', radar)

help(pytda)


I am working on the file "9b_PyTDA_Demo-AMS_OSRSC.ipynb".
The first task is:
Once you have that, send the radar object as an argument to the desired PyTDA function. There are two: calc_turb_sweep and calc_turb_vol. The former returns the turbulence field as an ndarray. The latter just successively calls the former to process a volume and modify the radar object and add the turbulence field. Currently, PyTDA only works on PPI sweeps/volumes.
Can you write Python code for it?

pytda.calc_turb_vol(radar, name_sw='SW', name_dz='DZ', verbose=False,
                    gate_spacing=250.0/1000.0, use_ntda=False,
                    beamwidth=radar.instrument_parameters['radar_beam_width_h']['data'][0])


I am working on the file "9b_PyTDA_Demo-AMS_OSRSC.ipynb".
The first task is:
So PyTDA is very simple to use. First you need a Py-ART radar object:
Can you write Python code for it?

files = glob.glob('*.nc')
print(files)
radar = pyart.io.read(files[0])

print(radar.fields.keys())

print(radar.instrument_parameters['radar_beam_width_h']['data'][0])


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
(Quick quiz: why did the snippet print "Go to work" here? What is the variable "day" set to?)
Let's take the snippet apart to see what happened. First, note the statement
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
1.4 Iteration, Indentation, and Blocks
One of the most useful things you can do with lists is to iterate through them, i.e. to go through each element one at a time. To do this in Python, we use the for statement:
Can you write Python code for it?

for day in days_of_the_week:
    print(day)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
1.7 Code Example: The Fibonacci Sequence
The Fibonacci sequence is a sequence in math that starts with 0 and 1, and then each successive entry is the sum of the previous two. Thus, the sequence goes 0,1,1,2,3,5,8,13,21,34,55,89,...
A very common exercise in programming books is to compute the Fibonacci sequence up to some number n. First I'll show the code, then I'll discuss what it is doing.
Can you write Python code for it?

n = 10
sequence = [0,1]
for i in range(2,n): # This is going to be a problem if we ever set n <= 2!
    sequence.append(sequence[i-1]+sequence[i-2])
print (sequence)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
2.1 Making vectors and matrices
Fundamental to both Numpy and Scipy is the ability to work with vectors and matrices. You can create vectors from lists using the array command:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
2.3 Matrix operations
Matrix objects act sensibly when multiplied by scalars:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
2.5 Example: Finite Differences
Now that we have these tools in our toolbox, we can start to do some cool stuff with it. Many of the equations we want to solve in Physics involve differential equations. We want to be able to compute the derivative of functions:
$$ y' = \frac{y(x+h)-y(x)}{h} $$
by discretizing the function $y(x)$ on an evenly spaced set of points $x_0, x_1, \dots, x_n$, yielding $y_0, y_1, \dots, y_n$. Using the discretization, we can approximate the derivative by
$$ y_i' \approx \frac{y_{i+1}-y_{i-1}}{x_{i+1}-x_{i-1}} $$
We can write a derivative function in Python via
Can you write Python code for it?

def nderiv(y,x):
    "Finite difference derivative of the function f"
    n = len(y)
    d = np.zeros(n,'d') # assume double
    # Use centered differences for the interior points, one-sided differences for the ends
    for i in range(1,n-1):
        d[i] = (y[i+1]-y[i])/(x[i+1]-x[i])
    d[0] = (y[1]-y[0])/(x[1]-x[0])
    d[n-1] = (y[n-1]-y[n-2])/(x[n-1]-x[n-2])
    return d


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
3.5 List Comprehensions and Generators
List comprehensions are a streamlined way to make lists. They look something like a list definition, with some logic thrown in. For example:
Can you write Python code for it?

evens1 = [2*i for i in range(10)]
print(evens1)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
3.6 Factory Functions
A factory function is a function that returns a function. They have the fancy name lexical closure, which makes you sound really intelligent in front of your CS friends. But, despite the arcane names, factory functions can play a very practical role.
Suppose you want the Gaussian function centered at 0.5, with height 99 and width 1.0. You could write a general function.
Can you write Python code for it?

def gauss(x,A,a,x0):
    return A*np.exp(-a*(x-x0)**2)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
A triangular well:
Can you write Python code for it?

def triangular(x,F=30): return F*x

tw = Schrod1d(triangular,m=10)
tw.plot()


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Again, it's not a necessary distinction, but one way to distinguish tuples and lists is that tuples are a collection of different things, here a name, and x and y coordinates, whereas a list is a collection of similar things, like if we wanted a list of those coordinates:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Although enumerate uses generators (see below) so that it doesn't have to create a big list, which makes it faster for really long sequenes.
3.4 Optional arguments of a function
You will recall that the linspace function can take either two arguments (for the starting and ending points):
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
And assign it to a function separately:
Can you write Python code for it?

another_doubler = lambda x: 2*x
another_doubler(19)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
As well as Jacobi, Laguerre, Hermite polynomials, Hypergeometric functions, and many others. There's a full listing at the Scipy Special Functions Page.
2.8 Least squares fitting
Very often we deal with some data that we want to fit to some sort of expected behavior. Say we have the following:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
But also let's us override the default value with a third argument:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
But it can also print data types other than strings:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
But not both at the same time, unless you want one of the symbols to be part of the string.
Can you write Python code for it?

"He's a Rebel"

'She asked, "How are you today?"'


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
But what if you need a function with only one argument, like f(x) rather than f(x,y,z,...)? You can do this with Factory Functions:
Can you write Python code for it?

def gauss_maker(A,a,x0):
    def f(x):
        return A*np.exp(-a*(x-x0)**2)
    return f

x = np.linspace(0,1)
g = gauss_maker(99.0,20,0.5)
plt.plot(x,g(x))


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
By removing a small part of the code using a list comprehension, we've doubled the overall speed of the code!
It seems like range is taking a long time, still. Can we get rid of it? We can, using the xrange generator:
Can you write Python code for it?

def evens3(n):
    "Return a list of even numbers below n"
    return [x for x in range(n) if x % 2 == 0]

import cProfile
cProfile.run('evens3(100000)')


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Everything in Python is an object, including functions. This means that functions can be returned by other functions. (They can also be passed into other functions, which is also useful, but a topic for another discussion.) In the gauss_maker example, the g function that is output "remembers" the A, a, x0 values it was constructed with, since they're all stored in the local memory space (this is what the lexical closure really refers to) of that function.
Factories are one of the more important of the Software Design Patterns, which are a set of guidelines to follow to make high-quality, portable, readable, stable software. It's beyond the scope of the current work to go more into either factories or design patterns, but I thought I would mention them for people interested in software design.
3.7 Serialization: Save it for later
Serialization refers to the process of outputting data (and occasionally functions) to a database or a regular file, for the purpose of using it later on. In the very early days of programming languages, this was normally done in regular text files. Python is excellent at text processing, and you probably already know enough to get started with this.
When accessing large amounts of data became important, people developed database software based around the Structured Query Language (SQL) standard. I'm not going to cover SQL here, but, if you're interested, I recommend using the sqlite3 module in the Python standard library.
As data interchange became important, the eXtensible Markup Language (XML) has emerged. XML makes data formats that are easy to write parsers for, greatly simplifying the ambiguity that sometimes arises in the process. Again, I'm not going to cover XML here, but if you're interested in learning more, look into Element Trees, now part of the Python standard library.
Python has a very general serialization format called pickle that can turn any Python object, even a function or a class, into a representation that can be written to a file and read in later. But, again,...
Can you write Python code for it?

# Data in a json format:
json_data = """\
{
    "a": [1,2,3],
    "b": [4,5,6],
    "greeting" : "Hello"
}"""
import json
loaded_json=json.loads(json_data)
loaded_json


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Finally, note that you can also string multiple comparisons together, which can result in very intuitive tests:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
For a pure exponential decay like this, we can fit the log of the data to a straight line. The above plot suggests this is a good approximation. Given a function
$$ y = Ae^{ax} $$
$$ \log(y) = ax + \log(A) $$
Thus, if we fit the log of the data versus x, we should get a straight line with slope $a$, and an intercept that gives the constant $A$.
There's a numpy function called polyfit that will fit data to a polynomial form. We'll use this to fit to a straight line (a polynomial of order 1)
Can you write Python code for it?

params = np.polyfit(data[:,0],np.log(data[:,1]),1)
a = params[0] # the coefficient of x**1
logA = params[1] # the coefficient of x**0

# plot if curious:
# plt.plot(data[:,0],np.log(data[:,1]),'bo')
# plt.plot(data[:,0],data[:,0]*a+logA,'r')


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Hartrees (what most quantum chemistry programs use by default) are really stupid units. We really want this in kcal/mol or eV or something we use. So let's quickly replot this in terms of eV above the minimum energy, which will give us a much more useful plot:
Can you write Python code for it?

energies = data[:,1]
minE = np.min(energies)
energies_eV = 27.211*(energies-minE)
plt.plot(data[:,0],energies_eV,'-o')
plt.xlabel('step')
plt.ylabel('Energy (eV)')
plt.title('Convergence of NWChem geometry optimization for Si cluster')


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Here we did two things with tuples you haven't seen before. First, we unpacked an object into a set of named variables using tuple assignment:
>>> name,x,y = obj

We also returned multiple values (minx,miny), which were then assigned to two other variables (x,y), again by tuple assignment. This makes what would have been complicated code in C++ rather simple.
Tuple assignment is also a convenient way to swap variables:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
However, tuples are immutable, you can't append to them or change the elements of them:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
If statements can have elif parts ("else if"), in addition to if/else parts. For example:
Can you write Python code for it?

if day == "Sunday":
    print ("Sleep in")
elif day == "Saturday":
    print ("Do chores")
else:
    print ("Go to work")


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
If we evaluate it by itself, as we just did, we see that it returns a boolean value, False. The "==" operator performs equality testing. If the two items are equal, it returns True, otherwise it returns False. In this case, it is comparing two variables, the string "Sunday", and whatever is stored in the variable "day", which, in this case, is the other string "Saturday". Since the two strings are not equal to each other, the truth test has the false value.
The if statement that contains the truth test is followed by a code block (a colon followed by an indented block of code). If the boolean is true, it executes the code in that block. Since it is false in the above example, we don't see that code executed.
The first block of code is followed by an else statement, which is executed if nothing else in the above if statement is true. Since the value was false, this code is executed, which is why we see "Go to work".
Try setting the day equal to "Sunday" and then running the above if/else statement. Did it work as you thought it would?
You can compare any data types in Python:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
If we want the last items of the list, we can do this with negative slicing:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
If we want the list containing the first two elements of a list, we can do this via
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
If you define a docstring for all of your functions, it makes it easier for other people to use them, since they can get help on the arguments and return values of the function.
Next, note that rather than putting a comment in about what input values lead to errors, we have some testing of these values, followed by a warning if the value is invalid, and some conditional code to handle special cases.
1.9 Recursion and Factorials
Functions can also call themselves, something that is often called recursion. We're going to experiment with recursion by computing the factorial function. The factorial is defined for a positive integer n as
$$ n! = n(n-1)(n-2)\cdots 1 $$
First, note that we don't need to write a function at all, since this is a function built into the standard math library. Let's use the help function to find out about it:
Can you write Python code for it?

from math import factorial
help(factorial)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
If you have a lot of words to concatenate together, there are other, more efficient ways to do this. But this is fine for linking a few strings together.
1.3 Lists
Very often in a programming language, one wants to keep a group of similar items together. Python does this using a data type called lists.
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
If you want to submit a sequence of runs to a computer somewhere, it's pretty easy to put together a little script, maybe even with some more string formatting in it:
Can you write Python code for it?

oxygen_xy_coords = [(0,0),(0,0.1),(0.1,0),(0.1,0.1)]
charge = 0
multiplicity = 1
dft_functional = "b3lyp"
jobtype = "optimize"

geometry_template = """\
  O    %f     %f      0.0
  H    0.0    1.0     0.0
  H    1.0    0.0     0.0"""

for i,xy in enumerate(oxygen_xy_coords):
    thetitle = "Water run #%d" % i
    jobname = "h2o-%d" % i
    geometry = geometry_template % xy
    print("---------")
    print(nwchem_format % dict(thetitle=thetitle,charge=charge,jobname=jobname,jobtype=jobtype,
                               geometry=geometry,dft_functional=dft_functional,multiplicity=multiplicity))


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
In the above snipped, the number 600 (stored in the variable "area") is converted into a string before being printed out.
You can use the + operator to concatenate strings together:
Can you write Python code for it?

statement = "Hello," + "World!"
print(statement)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Just like the other two data objects we're familiar with (ints and floats), you can assign a string to a variable
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Let's define a finite well potential:
Can you write Python code for it?

def finite_well(x,V_left=1,V_well=0,V_right=1,d_left=10,d_well=10,d_right=10):
    V = np.zeros(x.size,'d')
    for i in range(x.size):
        if x[i] < d_left:
            V[i] = V_left
        elif x[i] > (d_left+d_well):
            V[i] = V_right
        else:
            V[i] = V_well
    return V

fw = Schrod1d(finite_well,start=0,end=30,npts=100)
fw.plot()


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Let's examine what we just did: first, we used a for loop to iterate over each line. However, we skipped the first two (the lines[2:] only takes the lines starting from index 2), since lines[0] contained the title information, and lines[1] contained underscores. Similarly, [:5] instead would take the first five lines.
We pass the comma string ",'"into the split function, so that it breaks to a new word every time it sees a comma. Next, to simplify things a bit, we're using the map() command to repeatedly apply a single function (float()) to a list, and to return the output as a list. Finally, we turn the list of lists into a numpy arrray structure.
Can you write Python code for it?

plt.plot(data[:,0],data[:,1],'-o')
plt.xlabel('step')
plt.ylabel('Energy (hartrees)')
plt.title('Convergence of NWChem geometry optimization for Si cluster\n')


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Let's go through this line by line. First, we define the variable n, and set it to the integer 20. n is the length of the sequence we're going to form, and should probably have a better variable name. We then create a variable called sequence, and initialize it to the list with the integers 0 and 1 in it, the first two elements of the Fibonacci sequence. We have to create these elements "by hand", since the iterative part of the sequence requires two previous elements.
We then have a for loop over the list of integers from 2 (the next element of the list) to n (the length of the sequence). After the colon, we see a hash tag "#", and then a comment that if we had set n to some number less than 2 we would have a problem. Comments in Python start with #, and are good ways to make notes to yourself or to a user of your code explaining why you did what you did. Better than the comment here would be to test to make sure the value of n is valid, and to complain if it isn't; we'll try this later.
In the body of the loop, we append to the list an integer equal to the sum of the two previous elements of the list.
After exiting the loop (ending the indentation) we then print out the whole list. That's it!
1.8 Functions
We might want to use the Fibonacci snippet with different sequence lengths. We could cut an paste the code into another cell, changing the value of n, but it's easier and more useful to make a function out of the code. We do this with the def statement in Python:
Can you write Python code for it?

def fibonacci(sequence_length):
    "Return the Fibonacci sequence of length *sequence_length*"
    sequence = [0,1]
    if sequence_length < 1:
        print("Fibonacci sequence only defined for length 1 or greater")
        return
    if 0 < sequence_length < 3:
        return sequence[:sequence_length]
    for i in range(2,sequence_length):
        sequence.append(sequence[i-1]+sequence[i-2])
    return sequence


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Let's see whether this curve fits the data.
Can you write Python code for it?

x = np.linspace(1,45)
plt.title("Raw Data")
plt.xlabel("Distance")
plt.semilogy(data[:,0],data[:,1],'bo',label='data')
plt.semilogy(x,np.exp(logA)*np.exp(a*x),'r-',label='fit')
plt.legend()


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Let's see whether this works for our sin example from above:
Can you write Python code for it?

x = np.linspace(0,2*np.pi)
dsin = nderiv(np.sin(x),x)
plt.plot(x,dsin,label='numerical')
plt.plot(x,np.cos(x),label='analytical')
plt.title("Comparison of numerical and analytical derivatives of sin(x)")
plt.legend()


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Next lets generate the factorials.
Can you write Python code for it?

facts = []
for i in range(10):
    facts.append(factorial(i))


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
No matter how experienced a programmer you are, these are words to meditate on.
2. Numpy and Scipy
Numpy contains core routines for doing fast vector, matrix, and linear algebra-type operations in Python. Scipy contains additional routines for optimization, special functions, and so on. Both contain modules written in C and Fortran so that they're as fast as possible. Together, they give Python roughly the same capability that the Matlab program offers. (In fact, if you're an experienced Matlab user, there a guide to Numpy for Matlab users just for you.)
First off, it is important to import the library. Again, we did this at the very beginning of this whole jupyter notebook, but here it is in case you've jumped straight here without running the first code line:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Note that in this example we omitted the first few arguments, so that the slice started at 6, went to the end of the list, and took every second element, to generate the list of even numbers lup to 20 (the last element in the original list).
1.6 Booleans and Truth Testing
We have now learned a few data types. We have integers and floating point numbers, strings, and lists to contain them. We have also learned about lists, a container that can hold any data type. We have learned to print things out, and to iterate over items in lists. We will now learn about boolean variables that can be either True or False.
We invariably need some concept of conditions in programming to control branching behavior, to allow a program to react differently to different situations. If it's Monday, I'll go to work, but if it's Sunday, I'll sleep in. To do this in Python, we use a combination of boolean variables, which evaluate to either True or False, and if statements, that control branching based on boolean values.
For example:
Can you write Python code for it?

if day == "Sunday":
    print("Sleep in")
else:
    print("Go to work")


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Now we use the Matplotlib function plot to compare the two.
Can you write Python code for it?

plt.plot(facts,'-ob',label="factorial")
plt.plot(fibs,'-dg',label="Fibonacci")
plt.xlabel("n")
plt.legend()


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Of course we can combine if statements with for loops, to make a snippet that is almost interesting:
Can you write Python code for it?

for day in days_of_the_week:
    statement = "On " + day + ":"
    print (statement)
    if day == "Sunday":
        print ("   Sleep in")
    elif day == "Saturday":
        print ("   Do chores")
    else:
        print ("   Go to work")


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Or we can combine the two, making something like a semiconductor quantum well with a top gate:
Can you write Python code for it?

def tri_finite(x): return finite_well(x)+triangular(x,F=0.025)

tfw = Schrod1d(tri_finite,start=0,end=30,npts=100)
tfw.plot()


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Or, more elegantly:
Can you write Python code for it?

rands = np.random.rand(100)

plt.plot(rands,'o')


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Python lists, like C, but unlike Fortran, use 0 as the index of the first element of a list. Thus, in this example, the 0 element is "Sunday", 1 is "Monday", and so on. If you need to access the nth element from the end of the list, you can use a negative index. For example, the -1 element of a list is the last element:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Recursion can be very elegant, and can lead to very simple programs.
1.10 Two More Data Structures: Tuples and Dictionaries
Before we end the Python overview, I wanted to touch on two more data structures that are very useful (and thus very common) in Python programs.
A tuple is a sequence object like a list or a string. It's constructed by grouping a sequence of objects together with commas, either without brackets, or with parentheses:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Right now, we only know how to specify functions that have a fixed number of arguments. We'll learn how to do the more general cases here.
If we're defining a simple version of linspace, we would start with:
Can you write Python code for it?

def my_linspace(start,end):
    npoints = 50
    v = []
    d = (end-start)/float(npoints-1)
    for i in range(npoints):
        v.append(start + i*d)
    return v
my_linspace(0,1)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Scipy has a numerical integration routine quad (since sometimes numerical integration is called quadrature), that we can use for this:
Can you write Python code for it?

from scipy.integrate import quad
quad(f,0,np.inf)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Since strings are sequences, you can also do this to them:
Can you write Python code for it?

day = "Saturday"
abbreviation = day[:3]
print(abbreviation)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Since the data is now in a list of lines, we can iterate over it, splitting up data where we see a comma:
Can you write Python code for it?

for line in lines:
    # do something with each line
    words = line.split(",")


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Since we expect the data to have an exponential decay, we can plot it using a semi-log plot.
Can you write Python code for it?

plt.title("Raw Data")
plt.xlabel("Distance")
plt.semilogy(data[:,0],data[:,1],'bo')


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Splitting is a big concept in text processing. We used splitlines() here, and next we'll use the more general .split(",") function below to split each line into comma-delimited words.
We now want to do three things:

Skip over the lines that don't carry any information
Break apart each line that does carry information and grab the pieces we want
Turn the resulting data into something that we can plot.

To break apart each line, we will use .split(","). Let's see what it does to one of the lines:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
Table of Contents

0. Preliminary
0.1 Why Python?
0.2 What You Need to Install

1. Python Overview
1.1 Using Python as a Calculator
1.2 Strings
1.3 Lists
1.4 Iteration, Indentation, and Blocks
1.5 Slicing
1.6 Booleans and Truth Testing
1.7 Code Example: The Fibonacci Sequence
1.8 Functions
1.9 Recursion and Factorials
1.10 Two More Data Structures: Tuples and Dictionaries
1.11 Plotting with Matplotlib
1.12 Conclusion of the Python Overview

2. Numpy and Scipy
2.1 Making vectors and matrices
2.2 Linspace, matrix functions, and plotting
2.3 Matrix operations
2.4 Matrix Solvers
2.5 Example: Finite Differences
2.6 One-Dimensional Harmonic Oscillator using Finite Difference
2.7 Special Functions
2.8 Least squares fitting
2.9 Monte Carlo, random numbers, and computing $\pi$
2.10 Numerical Integration
2.11 Fast Fourier Transform and Signal Processing

3. Intermediate Python
3.1 Parsing data output
3.2 Reading in data files
3.3 More Sophisticated String Formatting and Processing
3.4 Optional arguments of a function
3.5 List Comprehensions and Generators
3.6 Factory Functions
3.7 Serialization: Save it for later
3.8 Functional programming
3.9 Object Oriented Programming

4. Speeding Python: Timeit, Profiling, Cython, SWIG, and PyPy
4.1 Timeit
4.2 Profiling
4.3 Other Ways to Speed Python
4.4 Fun: Finding Primes

5. References
6. Acknowledgements

0. Preliminary
0.1 Why Python?
Python is the programming language of choice for many scientists to a large degree because it offers a great deal of power to analyze and model scientific data with relatively little overhead in terms of learning, installation or development time. It is a language you can pick up in a weekend, and use for the rest of your life.
The Python Tutorial is a great place to start getting a feel for the language. To complement this material, I taught a Python Short Course years ago to a group of computational chemists during a time that I was worried the field was moving too much in the direction of using canned software rather than developing one's...
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
The agreement is almost exact.
We can use the subplot command to put multiple comparisons in different panes on a single plot (run %matplotlib qt on a separate line first to plot in a separate window):
Can you write Python code for it?

phase_correction = [-1,1,1,-1,-1,1]
for i in range(6):
    plt.subplot(2,3,i+1)
    plt.plot(x,ho_evec(x,i,1,1),label="Analytic")
    plt.plot(x,phase_correction[i]*U[:,i]/np.sqrt(h),label="Numeric")


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
The curve_fit routine we just used is built on top of a very good general minimization capability in Scipy. You can learn more at the scipy documentation pages.
2.9 Monte Carlo, random numbers, and computing $\pi$
Many methods in scientific computing rely on Monte Carlo integration, where a sequence of (pseudo) random numbers are used to approximate the integral of a function. Python has good random number generators in the standard library. The random() function from the numpy library gives pseudorandom numbers uniformly distributed between 0 and 1:
Can you write Python code for it?

# from random import random
rands = []
for i in range(100):
    rands.append(np.random.random())


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
The idea behind the program is that the ratio of the area of the unit circle to the square that inscribes it is $\pi/4$, so by counting the fraction of the random points in the square that are inside the circle, we get increasingly good estimates to $\pi$.
The above code uses some higher level Numpy tricks to compute the radius of each point in a single line, to count how many radii are below one in a single line, and to filter the x,y points based on their radii. To be honest, I rarely write code like this: I find some of these Numpy tricks a little too cute to remember them, and I'm more likely to use a list comprehension (see below) to filter the points I want, since I can remember that.
As methods of computing $\pi$ go, this is among the worst. A much better method is to use Leibniz's expansion of arctan(1):
$$\frac{\pi}{4} = \sum_k \frac{(-1)^k}{2*k+1}$$
Can you write Python code for it?

n = 100
total = 0
for k in range(n):
    total += pow(-1,k)/(2*k+1.0)
print(4*total)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
The index in a dictionary is called the key, and the corresponding dictionary entry is the value. A dictionary can use (almost) anything as the key. Whereas lists are formed with square brackets [], dictionaries use curly brackets {}:
Can you write Python code for it?

ages = {"Rick": 46, "Bob": 86, "Fred": 21}

print("Rick's age is ",ages["Rick"])


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
The init() function specifies what operations go on when the object is created. The self argument is the object itself, and we don't pass it in. The only required argument is the function that defines the QM potential. We can also specify additional arguments that define the numerical grid that we're going to use for the calculation.
For example, to do an infinite square well potential, we have a function that is 0 everywhere. We don't have to specify the barriers, since we'll only define the potential in the well, which means that it can't be defined anywhere else.
Can you write Python code for it?

square_well = Schrod1d(lambda x: 0*x,m=10)
square_well.plot(4,titlestring="Square Well Potential")


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
The lists created above with range have a step of 1 between elements. You can also give a fixed step size via a third command:
Can you write Python code for it?

evens = list(range(0,20,2))
evens

evens[3]


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
The little % sign that we have in front of the timeit call is an example of an IPython magic function, which we don't have time to go into here, but it's just some little extra mojo that IPython adds to the functions to make it run better in the IPython environment. You can read more about it in the IPython tutorial.
In any case, the timeit function runs 3 loops, and tells us that it took on the average of 583 ns to compute 20!. In contrast:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
The print statement is often used for printing character strings:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
The range() command is a convenient way to make sequential lists of numbers:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
The range() command is particularly useful with the for statement to execute loops of a specified length:
Can you write Python code for it?

for i in range(20):
    print("The square of ",i," is ",i*i)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
There are a number of routines to compute eigenvalues and eigenvectors

eigvals returns the eigenvalues of a matrix
eigvalsh returns the eigenvalues of a Hermitian matrix
eig returns the eigenvalues and eigenvectors of a matrix
eigh returns the eigenvalues and eigenvectors of a Hermitian matrix.
Can you write Python code for it?

A = np.array([[13,-4],[-4,7]],'d')
np.linalg.eigvalsh(A)

np.linalg.eigh(A)


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
There are additional signal processing routines in Scipy (e.g. splines, filtering) that you can read about here.
3. Intermediate Python
3.1 Parsing data output
As more and more of our day-to-day work is being done on and through computers, we increasingly have output that one program writes, often in a text file, that we need to analyze in one way or another, and potentially feed that output into another file.
Suppose we have the following output in CSV (comma separated values) format, a format that originally came from Microsoft Excel, and is increasingly used as a data interchange format in big data applications. How would we parse that?
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
There are many more things you can do with Matplotlib. We'll be looking at some of them in the sections to come. In the meantime, if you want an idea of the different things you can do, look at the Matplotlib Gallery. Rob Johansson's IPython notebook Introduction to Matplotlib is also particularly good.
1.12 Conclusion of the Python Overview
There is, of course, much more to the language than we've covered here. I've tried to keep this brief enough so that you can jump in and start using Python to simplify your life and work. My own experience in learning new things is that the information doesn't "stick" unless you try and use it for something in real life.
You will no doubt need to learn more as you go. I've listed several other good references, including the Python Tutorial and Learn Python the Hard Way. Additionally, now is a good time to start familiarizing yourself with the Python Documentation, and, in particular, the Python Language Reference.
Tim Peters, one of the earliest and most prolific Python contributors, wrote the "Zen of Python", which can be accessed via the "import this" command:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
There's also a convenient way to create dictionaries without having to quote the keys.
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
There's also an identity command that behaves as you'd expect:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
These are useful building blocks for functional programming.
The lambda operator allows us to build anonymous functions, which are simply functions that aren't defined by a normal def statement with a name. For example, a function that doubles the input is:
Can you write Python code for it?


I am working on the file "@Crash+Course+v0.63.ipynb".
The first task is:
This code snippet goes through each element of the list called days_of_the_week and assigns it to the variable day. It then executes everything in the indented block (in this case only one line of code, the print statement) using those variable assignments. When the program has gone through every element of the list, it exists the block.
(Almost) every programming language defines blocks of code in some way. In Fortran, one uses END statements (ENDDO, ENDIF, etc.) to define code blocks. In C, C++, and Perl, one uses curly braces {} to define these blocks.
Python uses a colon (":"), followed by indentation level to define code blocks. Everything at a higher level of indentation is taken to be in the same block. In the above example the block was only a single line, but we could have had longer blocks as well:
Can you write Python code for it?

for day in days_of_the_week:
    statement = "Today is " + day
    print(statement)