I am working on the file "Ordinary Differential Equations.ipynb".
The first task is:
The System rapidly converges to 0.
Overdamped System
A System is called overdamped when $\xi > 1$
In this case the transfer function, $G(s)$, has two distinct real poles with multiplicity 1. Again, the System natural response will not be harmonic, since the solution of the differential equation will be a linear combination of two exponential real functions. Observing the form of the poles %s_1% and %s_2%, both are the sum of two real numbers being one of them common.
The analytical solution will be of the form:
$$y(t)=e^{-ξw_n t}(C_1 e^{w_n \sqrt{ξ^2-1} t}+C_2 e^{-w_n \sqrt{ξ^2-1} t} )$$
and the constants are evaluated from the initial conditions:
$$C_1=\frac{y(0) w_n (ξ+\sqrt{ξ^2-1})+y'(0)}{2w_n \sqrt{ξ^2-1}}$$
$$C_2=\frac{-y(0) w_n (ξ+\sqrt{ξ^2-1})-y'(0)}{2w_n \sqrt{ξ^2-1}}$$
For $\xi=2$ the numerical solution was:
Can you write Python code for it?

x, Y = rungekutta(natural(wn, qsi=2), y0, tf=5, h=.01)
plt.plot(x, Y[:,0])
plt.title('Overdamped System')
plt.xlabel('t')
plt.ylabel('y(t)')
plt.grid()


I am working on the file "Ordinary Differential Equations.ipynb".
The first task is:
This System converges to 0 but it takes longer to do so.
System response to a permanent harmonic input
We will now focus on the behaviour of the System in the presence of a disturbance. This can be done analytically by studying:
$$y''(t)+2 \xi w_n y'(t)+w_n^2 y(t)=F(t)$$
This can be done analytically by adding to the general solution of the homogeneous equation a particular solution of the complete equation:
$$y(t) = y_h(t) + y_p(t)$$
The type of disturbance applied can be divided into the following:

Harmonic - The applied disturbance is given by a sinusoidal function
Periodic - The applied disturbance is given by a fourier series
Transient - Any other type of disturbance

While the harmonic and periodic Systems can be solved analytically or approximated using series, transient Systems may only be solved using convolution of the integrals, Laplace transforms or numeric integration.
The sinusoidal input we are going to consider is of the form $F(t)=f cos(wt)$, where $f$ and $w$ are, respectively, the amplitude and the frequency of the permanent exterior harmonic excitation.
We will considered the undamped System ($\xi=0$), and $f=30$ in order to study the System response to different values of the harmonic excitation input frequency, $w$.
For numerical integration purposes, we define a forced builder function with parameters $w_n$, $\xi$ and f as a multiplier for the force argument which takes a lambda function as follows:
Can you write Python code for it?

def forced(wn, qsi, f=30, force=lambda x: 1):
    n = natural(wn, qsi)
    return lambda x,y: np.array([
        n(x,y)[0],
        n(x,y)[1] + f * force(x),
    ])


I am working on the file "Ordinary Differential Equations.ipynb".
The first task is:
This force function is added to the second member of the last equation, in the first order System of ODEs defined in (2).
We can now study the System response to different values of $w$.
First was considered $ w = \frac{5\pi}{6} < w_n = 2 \pi$.
During the lectures on Dynamic Systems, the influence of $w$ was studied using bode plots.
Can you write Python code for it?

x, Y = (rungekutta(forced(wn, qsi=0, force=lambda x : np.cos(x*5*np.pi/6.)), y0))
plt.plot(x, Y[:,0], label='forced response')
x, Y = (rungekutta(natural(wn, qsi=0), y0))
plt.plot(x, Y[:,0], color="grey", alpha=0.5, label='natural response')
plt.title('System response when F(t) = 30 * cos(5pi/6 t)')
plt.xlabel('t')
plt.ylabel('y(t)')
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)
plt.grid()


I am working on the file "Ordinary Differential Equations.ipynb".
The first task is:
We can see that the disturbance introduced in the System does not change the bounds of the natural function.
Now lets study the interference caused by the function $ w = w_n$:
Can you write Python code for it?

x, Y = (rungekutta(forced(wn, qsi=0, force=lambda x :np.cos(x*wn)), y0))
plt.plot(x, Y[:,0], label='forced response')
x, Y = (rungekutta(natural(wn, qsi=0), y0))
plt.plot(x, Y[:,0], color="grey", alpha=0.5, label='natural response')
plt.title('System response when F(t) = 30 * cos(wn t)')
plt.xlabel('t')
plt.ylabel('y(t)')
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)
plt.grid()


I am working on the file "Ordinary Differential Equations.ipynb".
The first task is:
We will plot the difference between the analytic solution and the numeric solution:
Can you write Python code for it?

xa, Ya = rungekutta(natural(wn, qsi), y0, tf=5)
xb = np.arange(0,5,0.01)
Yb = undampedAnalyticSolution(xb, y0, wn)
plt.plot(xb, Ya[:,0]-Yb)
plt.title('Undamped System  (analytic - numerical solutions)')
plt.xlabel('t')
plt.ylabel('residual')
plt.grid()


I am working on the file "Ordre_de_grandeur.ipynb".
The first task is:
Aération naturelle
Can you write Python code for it?

# http://www.ecro.fr/fr/principes-solutions/principes/taux-de-brassage-renouvellement.html

taux = 2 # V/h

Volume_appart = 22.7 * 7.94  # m3
U_aeration = taux *Volume_appart  /60/60 * rhoCp_air

print( 'U aération nat: %f W/K' % U_aeration  )

12/2.3

3.5*5


I am working on the file "Ordre_de_grandeur.ipynb".
The first task is:
Convection
convection libre   (air)  h=    5 - 25    W/m2/K
https://fr.wikipedia.org/wiki/Air
http://www.legi.grenoble-inp.fr/people/Philippe.Marty/cours-convection-M2.pdf
Can you write Python code for it?

k = 0.0262 # W/m/K, conductivité thermique
nu = 1.57e-5 #  m2·s−1 , viscosité cinématique air,
alpha = 2.22e-5 # m2·s−1, diffusivité thermique
Pr = 0.708 # Prandl

L = 4 # m, dimension caractéristique
U = 20e3/60/60 # m/s, vitesse caractéristique
print( U )

Re = U*L/nu # Reynolds,   turbulent si >3.10^5
print( Re )

Reynolds = lambda u: u*L/nu

# Limite turbulence

v_lim = 3e5/L*nu
print( 'vitesse limite turbulence: %f m/s '% v_lim )

# Forcée, turbulent, Pr < 50  (cf. page 15)
h_FT = lambda u : k/L * 0.036 *  Reynolds(u)**(4/5) * Pr**(1/3)

# Forcée laminaire
h_FL = lambda u : k/L * 0.664 * Reynolds(u)**(1/2) * Pr**(1/3)

u_span_T = np.linspace( 1.0, 15, 20 ) # m/sh_FT_span = h_FT( u_span )
u_span_L = np.linspace( 0, 1.7, 20 )

plt.figure( figsize=(10, 6) )
plt.plot( u_span_T, h_FT( u_span_T ), label='Forcée, turbulent' )
plt.plot( u_span_L, h_FL( u_span_L ), label='Forcée, laminaire' )
plt.plot( [0, 1.7], [5, 5], label='Naturelle, laminaire&turb.' )

# plt.plot( u_span_T, 6.5*u_span_T, label='Forcée, laMétéo-dec' )

plt.xlabel( 'vitesse vent m/s' )
plt.ylabel( 'h  W/m2/K' );
plt.legend();


I am working on the file "Ordre_de_grandeur.ipynb".
The first task is:
Convection naturelle
Can you write Python code for it?

theta = 37*np.pi/180

deltaT = 30
L = 4 # m

g = 9.81 #* np.cos(theta) #m/s-2
beta = 3e-3 # air, 20°C

Gr = g*beta*deltaT*L**3/nu**2
print('Grashof: %e'% Gr )
print('Rayleight: %e'% (Gr*Pr) )

# Plaque horizontale chauffant vers le haut
h_NTh = k/L * 0.14*(Gr*Pr)**(1/3)
print( h_NTh )

# Vertical, turbulent (Mac Adams)
h_NTv = k/L * 0.13 * (Gr*Pr)**(1/3)

print( h_NTv )


I am working on the file "Ordre_de_grandeur.ipynb".
The first task is:
Débit ventilateur de table
http://www.rowenta.fr/Confort-de-la-maison/Ventilateurs/ESSENTIAL-COMPACT/p/4100000412
Can you write Python code for it?

q_min = 34 # m3/min  +- 10%

q_min = q_min - 0.1*q_min

taux = (q_min*60)/  Volume_appart

print( taux )

taux = 10 # Volume / heure
U_aeration = q_min *60*60 * rhoCp_air

print( 'U_ventilateur: %f W/K' % U_aeration  )


I am working on the file "Ordre_de_grandeur.ipynb".
The first task is:
Flux velux et fenêtre
Can you write Python code for it?

h_vitre = 2.8 # W/ (m2.K)

h_vitre = 2.8 # W/ (m2.K)

S_vitre = 0.6*0.8*2 + 1.2*0.8 + 0.3*0.72*4 + 0.25**2 # m2
print('surface vitrée: %.2f m^2' % S_vitre )

print( '> U_vitre : %.3f W/K'%( h_vitre*S_vitre ) )

longueur_cadres = (0.6+0.8)*4 + (1.2+0.8)*2 + 2*(0.3+0.72)*4 + 4*0.25
print( 'longueur cadres: %.1f m' % longueur_cadres )

psi = 0.016
k_bois = 0.15 + psi #
U_cadres = k_bois*longueur_cadres # W/K
print( '> U_cadres : %.3f W/K'%( U_cadres ) )

print(' ')
print( '> U_tot : %.3f W/K'%( U_cadres +  h_vitre*S_vitre ) )


I am working on the file "Ordre_de_grandeur.ipynb".
The first task is:
Isolation toiture
laine de verre
http://www.toutsurlisolation.com/Choisir-son-isolant/Les-isolants/Isolants-en-laine-minerale/Laine-de-verre
sigma = 0.030W/(m.k.) à 0.040W/(m.k)
Can you write Python code for it?

ep = 20e-2 # m
k = 0.035 # W/m/K

S_plafond = 2*4.59*7.94  # m2 , surface toit

print( 'U_toit = %.3f W/K'%(ep*k*S_plafond) )


I am working on the file "Ordre_de_grandeur.ipynb".
The first task is:
Masse thermique air
Can you write Python code for it?

Volume_appart = 22.7 * 7.94  # m3

# http://www.engineeringtoolbox.com/air-properties-d_156.html
rho_air = 1.2051 # kg/m3, 20°C
Cp_air = 0.005 * 1e3 # J/(kg K), 20°C

rhoCp_air = Cp_air*rho_air

print( 'masse air appart: %f kg'% (Volume_appart*rho_air) )

M_air = Volume_appart*rho_air*Cp_air
print( 'masse thermique air appart: %f J/K'% M_air )


I am working on the file "Ordre_de_grandeur.ipynb".
The first task is:
Masse thermique sol et mur
https://www.commeunthermicien.fr/scripts/doc/Fiche-Inertie.pdf
Can you write Python code for it?

# pour du béton :
rhoCp = 1400e3 # J/m3/K
k = 1.75 # W/m/K

# Distance de pénetration dans les murs :
temps_carac = 60*60*12 # 12h

D = k/rhoCp # diffusivité
distance = np.sqrt( D*temps_carac  )

print( 'Longeur carac. diff. mur: %f cm' % (distance*100) )

surface_murs = 58 + 2*22.7

ep_utile = 0.08  # 8cm

M_mur = surface_murs*ep_utile*rhoCp

print( 'masse thermique murs: %.2e J/K'% M_mur )

h = 10
print( M_mur/h/60/60/24 )


I am working on the file "Ordre_de_grandeur.ipynb".
The first task is:
Ordre de grandeur
Albédo tuile
http://www.ecohabitation.com/actualite/nouvelles/toits-blancs-alternative-diminuer-changements-climatiques#_ftn1
absorption   0.64
https://dumas.ccsd.cnrs.fr/dumas-00469805/document
page 116
albédo  0.33  - 0.36
Densité tuile
http://www.castorama.fr/store/Tuile-H14-rouge-prod17630318.html?navCount=0&navAction=push
Can you write Python code for it?

S =  25.8e-2 * 43.7e-2  # m3
m = 3.2 # kg

# Masse surfacique :
rhoS = m/S
print( rhoS )


I am working on the file "Ordre_de_grandeur.ipynb".
The first task is:
Si taille de tuile comme dim carac:
Can you write Python code for it?

L = 20e-2 # m

Gr = g*beta*deltaT*L**3/nu**2
print('Grashof: %e'% Gr )
print('Rayleight: %e'% (Gr*Pr) )

# limite de turbulence ...
# vertical :
h_NLv = k/L * 0.677* (Pr)**(1/2)/(0.95+Pr)**(1/4)*Gr**(1/4)

print( h_NLv )

# horizontale :
h_NLh = k/L * 0.54* (Pr*Gr)**(1/4)

print( h_NLh )


I am working on the file "Ordre_de_grandeur.ipynb".
The first task is:
Taux de renouvelement
Can you write Python code for it?

A = 1.2*0.3  # m2
v = 0.5 # m/s

taux = ( A*v ) /  Volume_appart

print( taux*60*60 )


I am working on the file "OrganizationOutline.ipynb".
The first task is:
-
-calculate auxiliary information (e.g. barycentric correction)
-create master calibration data relevant to this observation
-identify likely cosmic rays in this image
-define reference positions from which spectra should be extracted
-identify relevant pixels, define extraction apertures
-extract spectra, and visualize the process
Can you write Python code for it?


I am working on the file "OrganizationOutline.ipynb".
The first task is:
Determine the basic properties of this instrument.
Classes derived from the Instrument class handle all instrument-specific information, including
- the reduction of CCD frames
- the extraction of key information from headers
- the mapping of reference positions to apertures
- ...
Can you write Python code for it?

from mosasaurus.instruments import LDSS3C
i = LDSS3C(grism='vph-red')
i.summarize()

i.keysforlogheader


I am working on the file "OrganizationOutline.ipynb".
The first task is:
Determine the basic properties of this observation, including the data associated with it.
-identify data associated with this observation
Can you write Python code for it?

from mosasaurus.Observation import Observation
o = Observation(t, i, n)
o.summarize()

o.loadHeaders()


I am working on the file "Oriol/Optative_exercises.ipynb".
The first task is:
2.Maximum likelihood and least squares
Exercise 2.1
Find the maximum likelihood estimators for the mean and the variance of a Gaussian distribution, and obtain the covariance matrix from the second derivatives of the likelihood
$$
L(\vec{x};\mu,\sigma)=\prod_{i=1}^N f(x_i;\mu,\sigma)
$$
Thus, the log-likelihood is:
$$
l(\vec{x};\mu,\sigma)=-2\log L(\vec{x};\mu,\sigma)= n\log (2\pi) +n\log (\sigma^2) + \frac{1}{\sigma^2}\sum_{i=1}^n (x_i-\mu)^2
$$
Therefore, the ML estimator of $\mu$ and $\sigma$ will be the values that minimize $l(\vec{x};\mu,\sigma)$ derivatives with respect to $\mu$ and $\sigma$ respectively:
$$
\frac{\partial l(\vec{x};\mu,\sigma)}{\partial \mu}\Big\rvert_{\mu=\mu_{ML}}=0=\frac{1}{\sigma^2}\sum_{i=1}^n 2(x_i-\mu_{ML})(-1) \quad \rightarrow \quad 0=\sum_{i=1}^n (x_i-\mu_{ML})=\sum_{i=1}^n (x_i)-n\mu_{ML} \
\Rightarrow \mu_{ML}=\frac{1}{n}\sum_{i=1}^n x_i \
\frac{\partial l(\vec{x};\mu,\sigma)}{\partial \sigma}\Big\rvert_{\sigma=\sigma_{ML}}=0=n\frac{2}{\sigma_{ML}}+\frac{-2}{\sigma_{ML}^3}\sum_{i=1}^n (x_i-\mu)^2 \quad \rightarrow \quad \frac{n}{\sigma_{ML}}=\frac{1}{\sigma_{ML}^3}\sum_{i=1}^n (x_i-\mu)^2 \
\Rightarrow \sigma_{ML}^2 = \frac{1}{n}\sum_{i=1}^n (x_i-\mu)^2
$$
Exercise 2.2
Generate 2000 random numbers, ${y_i}$, following $f(x)=1+a_1x+a_2x^2$, using $a_1=0.5$,  $a_2=0.5$, for $-1\leq x \leq 1$. Use these data to perform a likelihood fit to estimate $a_1$,$a_2$. Draw the associated contour plots together with the real value of the estimate.
$$
f(x)=1+a_1x+a_2x^2 \ \rightarrow \quad F(x)=\int_{-1}^x f(x) dx = x+\frac{a_1}{2}x^2+\frac{a_2}{3}x^3-(-1+\frac{a_1}{2}-\frac{a_2}{3})=(x+1)+\frac{a_1}{2}(x^2-1)+\frac{a_2}{3}(x^3+1)=\ =(x+1)+\frac{a_1}{2}(x-1)(x+1)+\frac{a_2}{3}(x+1)(x^2-x+1)
$$
Note that $F(1)=2+\frac{2}{3}a_2$ instead of $1$ which is what it should be due to $x\in [-1,1]$. Therefore, the pdf is not normalized. To solve this, we normalize both the pdf and the cdf dividing by the factor $2+\frac{2}{3}a_2$.
Likelihod fit
$$
L(\vec{x};a_1,a_2)=\prod_{i=1}^N...
Can you write Python code for it?

def pdf_3(x,a1,a2):
    y = (1+a1*x+a2*np.square(x))/(2.+2./3.*a2)
    return y

def cdf_3(x,a1,a2):
    P = (x+1)*(1+a1/2.*(x-1)+a2/3.*(x**2-x+1))/(2.+2./3.*a2)
    return P

def try_reject(fun,N,a,b,fmax,m,*fun_args):
    x=a+(b-a)*np.random.random(m*N)
    y=fmax*np.random.random(m*N)
    Xi=x[y<=fun(x,*fun_args)][:N]
    return Xi

def likelihood(sample,a1,a2):
    L = -2.*np.sum(np.log(pdf_3(sample,a1,a2)))
    return L

def d_likelihood(a,sample):
    a1=a[0]; a2=a[1]
    den=pdf_3(sample,a1,a2)
    d1=sum(sample/den)
    d2=2./3.*len(sample)-sum(np.square(sample)/den)
    d=np.array([d1,d2])
    return d

N = 2000
a1 = .5
a2 = .5
sample = try_reject(pdf_3,N,-1,1,.9,2,a1,a2)
fig = plt.figure()
ax = fig.add_subplot(111)
ax.hist(sample,normed=1,bins=15,label='MC generated sample')
x_pdf = np.linspace(-1,1,200)
ax.plot(x_pdf,pdf_3(x_pdf,a1,a2),'r',label='Theoretical p.d.f.')
ax.legend(loc=0)
ax.set_xlabel('$x$')

# Estimate a1 and a2 using the derivatives of l, finding its roots numerically
a1_num,a2_num = opt.fsolve(d_likelihood,np.array([.4,.6]),args=(sample))
# Calculate the variances of a1 and a2 using the -log(L(mu+sigma))=-log(L(mu))+1
# which implies that l(mu+sigma)-l(mu)-1=0. Thus, finding the root of this equation
# (implemented in a1_var and a2_var functions) will be an estimate of the standard deviation
Lmax = likelihood(sample,a1_num,a2_num)
a1_error = np.empty((2,1)) ; a2_error = np.empty((2,1))
a1_var = lambda a1 : likelihood(sample,a1,a2_num)-Lmax-1
a1_error[1] = opt.fsolve(a1_var,a1_num+.1)[0]-a1_num # + error
a1_error[0] = a1_num-opt.fsolve(a1_var,a1_num-.1)[0] # - error
a2_var = lambda a2 : likelihood(sample,a1_num,a2)-Lmax-1
a2_error[1] = opt.fsolve(a2_var,a2_num+.1)[0]-a2_num # + error
a2_error[0] = a2_num-opt.fsolve(a2_var,a2_num-.1)[0] # - error
print 'Propper ML estimate:\n\ta1=%.2g (-%.2g,+%.2g)\n\ta2=%.2g (-%.2g,+%.2g)' %(a1_num,a1_error[0],a1_error[1],
                                                                  a2_num,a2_error[0],a2_error[1])

n_grid = 51
a1_mesh =...


I am working on the file "Oriol/Optative_exercises.ipynb".
The first task is:
Exercise 1.2
Verify the results of exercise 1 with the following numerical simulation:
1. For each $x_i$ generate a random $y_i$ with a Gaussian distribution $N(a_1+a_2x_i,\sigma^2)$.
2. Fit the new data and predict the $y$ for $x=0.5$.
3. Repeat the steps above 1000 times and construct a histogram with the values of $y(0.5)$. Obtain the standard deviation and compare it with the value from exercise 1
Can you write Python code for it?

m = 2000
y_prime = np.random.normal(a1_ex1+a2_ex1*x_ex1, sigma_ex1, (m,len(x_ex1)))
S_yp = np.sum(y_prime, axis=1)
S_xyp = np.sum(x_ex1*y_prime, axis=1)
a1_vec = (S_x2*S_yp - S_x*S_xyp)/Delta
a2_vec = (N*S_xyp - S_x*S_yp)/Delta
y_values = a1_vec+a2_vec*x_value
mu_value = np.mean(y_values)
sigma_value = np.std(y_values)
print 'The estimatied value of y(%g) has mean = %f and std = %f' %(x_value, mu_value, sigma_value)
print 'The linear regression value of y(%g) has value = %f and estimated error = %f' %(x_value,
                                                                                a1_ex1+a2_ex1*x_value,
                                                                                y_error)

fig = plt.figure()
ax = fig.add_subplot(111)
ax.hist(y_values, bins=15, normed = 1, label='y(%g) histogram' %x_value)
dist = stat.norm(mu_value, sigma_value)
x_pdf = np.linspace(mu_value-4.*sigma_value, mu_value+4*sigma_value, 1000)
ax.plot(x_pdf, dist.pdf(x_pdf),'r', label='Gaussian fit')
ax.set_xlabel('$y(%g)$' %x_value)
ax.legend(loc=0)


I am working on the file "Oriol/Optative_exercises.ipynb".
The first task is:
Exercise 3.3
Repeat the $a_1$,$a_2$ likelihood fit for m different montecarlo generated samples in order to estimate the variance of $a_1^{ML}$ and $a_2^{ML}$.
Can you write Python code for it?

m = 1000
a1_MCvec = np.empty(m)
a2_MCvec = np.empty(m)
for i in xrange(m):
    i_sample = try_reject(pdf_3,N,-1,1,.9,2,a1,a2)
    # Estimate a1 and a2 using the derivatives of l, finding its roots numerically
    a1_MCvec[i],a2_MCvec[i] = opt.fsolve(d_likelihood,np.array([.5,.5]),args=(i_sample))

print 'MC ML estimate:\n\ta1: mean=%.2g; std=%.2g\n\ta2: mean=%.2g; std=%.2g' %(np.mean(a1_MCvec),np.std(a1_MCvec),
                                                              np.mean(a2_MCvec),np.std(a2_MCvec))


I am working on the file "Ornek_Uygulama_Hamlet.ipynb".
The first task is:
Bulduk gibi. Ancak ufak bir sorun var. Aynı kelime birkaç kez geçiyorsa bu listeye giriyor. Yani 13 harften oluşan kelime metinin içinde kaç kez geçtiyse her biri bu listede yer alıyor.
Can you write Python code for it?

for kelime in kelime_cokuzu:
    if (len(kelime) == 13):
        print kelime,


I am working on the file "Ornek_Uygulama_Hamlet.ipynb".
The first task is:
Hah, şimdi oldu. Artık ilk soruya bir cevap verebiliriz. Hamlet'te kaç kelime olduğuna bakmak için kelime çokuzunun uzunluğuna bakmamız yeterli.
Can you write Python code for it?


I am working on the file "Ornek_Uygulama_Hamlet.ipynb".
The first task is:
Hedefimiz en uzun kelimeyi ya da kelimeleri bulmaktı. Bir üst kod satırında 13 yerine 14 yazmak bu sonucu elde etmek için yeterli. Eğer aynı kelimelerin sadece bir kez sayılmasında ısrar edersek çokuzdan, kümeye geçmek kestirme bir yol.
Can you write Python code for it?

kelime_kumesi = set(kelime_cokuzu)

len(kelime_kumesi)


I am working on the file "Ornek_Uygulama_Hamlet.ipynb".
The first task is:
Ortalama kelime uzunluğunu bulduk. En uzun kelimeyi bulmak için de bu çokuzu sıralamak yetecek. Sıralamak için sort fonksiyonunu kullanabiliriz.
Can you write Python code for it?


I am working on the file "Ornek_Uygulama_Hamlet.ipynb".
The first task is:
Tam sayılı bölme yapınca, sonuç da tam sayılı oldu. Bölmeyi yaparken sayılardan en azından birini sürekli bir sayıya çevirmek iyi olacak.
Can you write Python code for it?

ort_uzunluk = sum(uzunluklar)/float(len(uzunluklar))
print ort_uzunluk


I am working on the file "Ornek_Uygulama_Hamlet.ipynb".
The first task is:
Ufak bir sorun var. Bazı kelimeler büyük harflerle, bazıları ise küçük harflerle yazılmış. Kelimeleri sayarken bu bize sorun yaratabilir. İyisi mi her kelimeyi küçük harfe çevirelim. Python'da herhangi bir dizeyi kuçuk harflere çevirmek için lower fonksiyonu kullanılabilir.
Can you write Python code for it?

print kelime_cokuzu[18]
print kelime_cokuzu[18].lower()


I am working on the file "Ornek_Uygulama_Hamlet.ipynb".
The first task is:
Öyle ya! Noktalama işaretlerini unuttuk. Neyse ki Python'da bunun da çaresi var. Hem de birkaç tane. En etkilisi olmasa da basit bir çözüm strip fonksiyonunu kullanmak. Bu fonksiyon ile dizelerin içindeki tüm noktalama işaretlerini temizleyebiliriz.
Can you write Python code for it?


I am working on the file "Ornek_Uygulama_Hamlet.ipynb".
The first task is:
İki fonksiyonu birleştirip, kelime çokuzunu baştan doldurabiliriz.
Can you write Python code for it?

kelime_cokuzu = [kelime.lower().strip(',.?!;:') for kelime in kelime_cokuzu]

print kelime_cokuzu[18]


I am working on the file "Ornek_Uygulama_Hamlet.ipynb".
The first task is:
İçerik doğru gelmiş mi bir kontrol etmekte fayda var.
Can you write Python code for it?


I am working on the file "Ornek_Uygulama_Hamlet.ipynb".
The first task is:
Şimdi bir tablo ile her uzunluktan kaç tane kelime olduğuna bakabilriz.
Can you write Python code for it?

harf_sayilari = []
for i in range(1, len(kelime_cantasi)+1):
    harf_sayisi = len(kelime_cantasi[i])
    harf_sayilari.append(harf_sayisi)
    print str(i) + ' harfli kelimelerin sayısı: ' + str(harf_sayisi)


I am working on the file "Ornek_Uygulama_Rassal.ipynb".
The first task is:
Grafiği çizmeye hazırız.
Can you write Python code for it?

plt.plot(x_degerleri, y_degerleri, 'ro') # Nesneler kırmızı yuvarlaklar
plt.plot(x, y, 'b*', ms=15.0) # Ağırlık merkezi büyük mavi bir yıldız
plt.xticks(np.arange(0, 1.1, 0.1))
plt.yticks(np.arange(0, 1.1, 0.1))
plt.xlabel('$x$')
plt.ylabel('$y$')
plt.grid(True)


I am working on the file "Ornek_Uygulama_Rassal.ipynb".
The first task is:
Her noktanın koordinatlarını rassal olarak ürettik. Şimdi ağırlık merkezini bulabiliriz.
Can you write Python code for it?

x = 0
y = 0
toplam_agirlik = sum(agirliklar)
for i in range(0, n):
    x += (agirliklar[i]*x_degerleri[i])/toplam_agirlik
    y += (agirliklar[i]*y_degerleri[i])/toplam_agirlik
print x, y


I am working on the file "Ornek_Uygulama_Rassal.ipynb".
The first task is:
İlk işimiz nesnelerin ağırlıklarını ve koordinatlarını rassal olarak üretmek olsun.
Can you write Python code for it?

n = 5 # Toplam nesne sayısı
ust_ag = 30 # Olabilecek en yüksek ağırlık
x_degerleri = np.random.rand(n)
y_degerleri = np.random.rand(n)
agirliklar = ust_ag*np.random.rand(n) #
print x_degerleri
print y_degerleri
print agirliklar


I am working on the file "Oslo Model.ipynb".
The first task is:
$$$$
Can you write Python code for it?

for length, pile_dict in data_dict.items():
    height_prob_dict = analysis.calculate_height_probability(pile_dict['heights'][-steady_state_time_period:])
    collapsed_h = (np.array(list(height_prob_dict.keys()))-average_heights[list(data_dict.keys()).index(length)]) / length ** w
    collapsed_p = np.array(list(height_prob_dict.values())) * length ** w
    plt.semilogy(collapsed_h, collapsed_p, label=length, linewidth=0.5)

plt.legend(loc=0, title="System Size (L)",  framealpha=0.8, prop={'size':10})
plt.xlabel(r"$(h - \langle h \rangle) L^{-0.241}$", fontsize=14)
plt.ylabel("$L^{0.241}P(h; L)$", fontsize=14)
plt.show()


I am working on the file "Oslo Model.ipynb".
The first task is:
$$\langle h \rangle = a_0L(1 - a_1L^{-\omega_1})$$
where $a_0 = \langle z \rangle = 1.734$, $a_1 = 0.238,$ and $\omega_1 = 0.613$
The uncertainties are high, because it is a 3 parameter optimisation.
We can also obtain the parameters from taking the log of both side.
$$log(a_0L-\langle h \rangle) = (1 - \omega_1) log(a_0a_1L)$$
Using the parameters, let's see how well the fit is:
Can you write Python code for it?

Ls = np.arange(1, 600)

plt.plot(system_lengths, average_heights, '.', color='magenta', label='data')
plt.plot(Ls, get_average_pile_height_from_L(Ls, a0, a1, w1), color='black', label='fit', linewidth=0.5)

plt.xlabel("$L$", fontsize=14)
plt.ylabel(r"$\langle h \rangle$", fontsize=14)
plt.legend(loc=0)
plt.show()


I am working on the file "Oslo Model.ipynb".
The first task is:
$$\sigma_h(L)=aL^{\omega}$$
where $a = 0.581$ and $\omega = 0.241$
Plotting on a loglog plot:
Can you write Python code for it?

plt.loglog(system_lengths, standard_deviations, '.', color='magenta', label='data')
plt.loglog(Ls, power_law(Ls, a, w), color='black', label='fit')

plt.legend(loc=0)
plt.xlabel("$L$", fontsize=14)
plt.ylabel(r"$\sigma_h$", fontsize=14)
plt.show()


I am working on the file "Oslo Model.ipynb".
The first task is:
$$\sigma_z = \sigma_h / L \sim L^{\omega-1} \sim L^{-0.759}$$
So the average slope's standard deviation tends to 0 in the limit of $L \rightarrow \infty$

Task 2d
Plot height probability against height for the recurrent configurations:
Can you write Python code for it?

for length, pile_dict in data_dict.items():
    height_prob_dict = analysis.calculate_height_probability(pile_dict['heights'][-steady_state_time_period:])
    plt.plot(list(height_prob_dict.keys()), list(height_prob_dict.values()), '-', label=length, linewidth=0.5)

plt.legend(loc=0, title="System Size (L)",  framealpha=0.8, prop={'size':10})
plt.xlabel("$h$", fontsize=14)
plt.ylabel("$P(h; L)$", fontsize=14)
plt.show()


I am working on the file "Oslo Model.ipynb".
The first task is:
All threshold slopes of pile1 should be 1:
Can you write Python code for it?


I am working on the file "Oslo Model.ipynb".
The first task is:
Apply a technique similar to average height, we see no signs of scaling to correction:
Can you write Python code for it?

plt.plot(system_lengths, standard_deviations / (a * system_lengths ** w), '.', color='magenta', label='data')
plt.plot(Ls, power_law(Ls, a, w) / (a * Ls ** w), color='black', label='fit')

plt.legend(loc=0)
plt.xlabel("$L$", fontsize=14)
plt.ylabel(r"$\sigma_h/bL^{\alpha}$", fontsize=14)
plt.axis((0, 600, 0.95, 1.05))
plt.show()


I am working on the file "Oslo Model.ipynb".
The first task is:
Around half of the threshold slopes in oslo_pile_xs should be 1, and the other half should be 2:
Can you write Python code for it?


I am working on the file "Oslo Model.ipynb".
The first task is:
Arriving the recurring configurations, we can keep dropping grains to the oslo_pile_xs and the pile height should average to $\langle h \rangle=26.5$.
Can you write Python code for it?

pile_heights = []

for i in range(10000):
    oslo_pile_xs.drop_grain()
    pile_heights.append(oslo_pile_xs.get_pile_height())

np.average(pile_heights)


I am working on the file "Oslo Model.ipynb".
The first task is:
As we can see, the Oslo model exhibits very different behaviours to the BTW model.
These tests have also sufficiently proved the correctness of the program.

Save/load Data
We first save the simulation data to a file, as simulating large system requires a lot of computational time.
The pile reaches a steady state when the last site topples. We can use this property to determine the amount of grains to drop.
~~~python
Initialise piles
oslo_pile_xxs = Pile(8, OSLO_PROBS, POSSIBLE_THRESHOLD_SLOPES)
oslo_pile_xs = Pile(16, OSLO_PROBS, POSSIBLE_THRESHOLD_SLOPES)
oslo_pile_s = Pile(32, OSLO_PROBS, POSSIBLE_THRESHOLD_SLOPES)
oslo_pile_m = Pile(64, OSLO_PROBS, POSSIBLE_THRESHOLD_SLOPES)
oslo_pile_l = Pile(128, OSLO_PROBS, POSSIBLE_THRESHOLD_SLOPES)
oslo_pile_xl = Pile(256, OSLO_PROBS, POSSIBLE_THRESHOLD_SLOPES)
oslo_pile_xxl = Pile(512, OSLO_PROBS, POSSIBLE_THRESHOLD_SLOPES)
piles = (oslo_pile_xxs, oslo_pile_xs, oslo_pile_s, oslo_pile_m, oslo_pile_l, oslo_pile_xl, oslo_pile_xxl)
data_dict = OrderedDict()
Populate piles
for pile in piles:
    pile_dict = {'heights': [], 'avalanche size': []}
while not pile.is_at_steady_state:
    pile_dict['heights'].append(pile.get_pile_height())
    pile_dict['avalanche size'].append(pile.ava_size)
    pile.drop_grain()

for _ in range(1000000):
    pile_dict['heights'].append(pile.get_pile_height())
    pile_dict['avalanche size'].append(pile.ava_size)
    pile.drop_grain()

data_dict[pile.length] = pile_dict

pickle.dump(data_dict, open('height_and_avalanche_data', 'wb'))
~~~
When the pile reaches a recurrent state, we drop 1,000,000 more grains for later analysis.
We can then load the data back into program:
Can you write Python code for it?

data_dict = pickle.load(open('height_and_avalanche_data', 'rb'))
steady_state_time_period = 1000000


I am working on the file "Oslo Model.ipynb".
The first task is:
At the stable state, dropping a grain to the ith site of the pile will result in an avalanche size of $s = L - i$, where $s$ is the avalanche size, $L$ the length of the pile.
Can you write Python code for it?

# i = 0
pile1.drop_grain()
pile1.ava_size

# i = 6
pile1.drop_grain(site_index=6)
pile1.ava_size


I am working on the file "Oslo Model.ipynb".
The first task is:
Chose $a=1.3$ to produce smooth curves and remove noise.
Compare to $a=1.1$
[INSERT QUALITATIVE DESCRIPTION]

Task 3c
Plot the probability against avalanche size graph for $L = 512$:
Can you write Python code for it?

centres, probabilities = analysis.make_log_bins(data_dict[512]['avalanche size'][-no_of_samples:], bin_scaling=1.3)
plt.loglog(centres, probabilities, '.', ms=3.0)
plt.xlabel("$s$", fontsize=14)
plt.ylabel(r"$\widetilde P_N(s;L)$", fontsize=14)
plt.show()


I am working on the file "Oslo Model.ipynb".
The first task is:
Define power law and linear regression functions. These will be repeatly used for fitting.
Can you write Python code for it?

power_law = lambda x, a, p: a * x ** p
linear_regression = lambda x, c, m: m * x + c


I am working on the file "Oslo Model.ipynb".
The first task is:
Define the possible threshold slopes and the corresponding probabilities for the original Oslo model ($p = 0.5$):
Can you write Python code for it?

POSSIBLE_THRESHOLD_SLOPES = (1, 2)
OSLO_PROBS = (0.5, 0.5)


I am working on the file "Oslo Model.ipynb".
The first task is:
Finally, we compare the pile heights of the two piles versus time (number of grains dropped):
Can you write Python code for it?

# reinitialise the piles
pile1.reset()
oslo_pile_xs.reset()

for i in (pile1, oslo_pile_xs):
    pile_heights = []

    # drop 500 grains
    for t in range(500):
        pile_heights.append(i.get_pile_height())
        i.drop_grain()
    plt.plot(pile_heights, label=i.name)

plt.xlabel("$t$ (number of grains dropped)", fontsize=12)
plt.ylabel("Pile Height (grains)", fontsize=12)
plt.legend(loc=0)
plt.show()


I am working on the file "Oslo Model.ipynb".
The first task is:
Fit using only the last 3 data points ($L \gg 1$):
Can you write Python code for it?

Ls = np.arange(0, 1000)
k_slopes = []
k_slope_errs = []

for i, k in enumerate(ks):
    plt.loglog(kth_moment_array[i, :, 0], kth_moment_array[i, :, 1], '.', label='k = {}'.format(k))

    (log_a, exp), cov = curve_fit(linear_regression, np.log(kth_moment_array[i, -3:, 0]), np.log(kth_moment_array[i, -3:, 1]))
    plt.loglog(Ls, power_law(Ls, np.exp(log_a), exp), color='black', linewidth=0.8)
    k_slopes.append(exp)
    k_slope_errs.append(analysis.calculate_std_from_cov_matrix(cov)[1])

print(k_slope_errs)
print(k_slopes)
plt.legend(loc=0)
plt.xlabel('$L$', fontsize=14)
plt.ylabel(r'$\langle s^k \rangle$', fontsize=14)
plt.show()

(c, D), cov = curve_fit(linear_regression, ks, k_slopes, sigma=k_slope_errs)
print(c, D, analysis.calculate_std_from_cov_matrix(cov))

tau_s = 1 - c / D
print(tau_s)

plt.errorbar(ks, k_slopes, yerr=k_slope_errs, color='magenta', fmt='.', label='data', ms=2.0)

ks_ = np.arange(7)
plt.plot(ks_, linear_regression(ks_, c, D), color='black', label='fit', linewidth=0.5)

plt.legend(loc=0)
plt.xlabel('$k$', fontsize=14)
plt.ylabel(r'$\alpha$', fontsize=14)
plt.show()


I am working on the file "Oslo Model.ipynb".
The first task is:
Looks like a power law relation, so:
Can you write Python code for it?

(a, w), cov = curve_fit(power_law, system_lengths, standard_deviations)

a, w, analysis.calculate_std_from_cov_matrix(cov)


I am working on the file "Oslo Model.ipynb".
The first task is:
Now we try the same thing with the Oslo pile:
Can you write Python code for it?

for _ in range(500):
    oslo_pile_xs.drop_grain()

oslo_pile_xs.get_heights()


I am working on the file "Oslo Model.ipynb".
The first task is:
Task 1
Threshold Slopes Test
Create some length 16 piles with different values of $p$:
Can you write Python code for it?

btw_probs = (1, 0)  # p = 1

pile1 = Pile(16, btw_probs, POSSIBLE_THRESHOLD_SLOPES, name="BTW pile, length 16")

oslo_pile_xs = Pile(16, OSLO_PROBS, POSSIBLE_THRESHOLD_SLOPES, name="Oslo pile, length 16")


I am working on the file "Oslo Model.ipynb".
The first task is:
Using the found $\tau_s$, try different values of $D$ to find the best collapse. Only data with system size larger than 128 $(L \gg 1)$ was plotted to make observations easier:
Can you write Python code for it?

D = 2.25

for length, pile_dict in data_dict.items():
    if length >= 128:
        centres, probabilities = analysis.make_log_bins(pile_dict['avalanche size'][-no_of_samples:], bin_scaling=1.3)
        centre_L_minus_Ds = centres * length ** -D
        s_tau_probabilities = centres ** -tau_s * probabilities
        plt.loglog(centre_L_minus_Ds, s_tau_probabilities, '-', label=length, linewidth=0.5)

plt.legend(loc=0, title='System Size (L)')
plt.xlabel(r"$s/L^D$", fontsize=14)
plt.ylabel(r"$s^{\tau_s} \widetilde P_N(s;L)$", fontsize=14)
plt.show()

D = 2.25

for length, pile_dict in data_dict.items():
    centres, probabilities = analysis.make_log_bins(pile_dict['avalanche size'][-no_of_samples:], bin_scaling=1.3)
    centre_L_minus_Ds = centres * length ** -D
    s_tau_probabilities = centres ** -tau_s * probabilities
    plt.loglog(centre_L_minus_Ds, s_tau_probabilities, '-', label=length, linewidth=0.5)

plt.legend(loc=0, title='System Size (L)', prop={'size':10})
plt.xlabel(r"$s/L^D$", fontsize=14)
plt.ylabel(r"$s^{\tau_s} \widetilde P_N(s;L)$", fontsize=14)
plt.show()


I am working on the file "Oslo Model.ipynb".
The first task is:
We can clearly see the sign of corrections to scaling for small $L$:
Can you write Python code for it?

plt.plot(system_lengths, average_heights / (a0 * system_lengths), '.', color='magenta', label='data')
plt.plot(Ls, get_average_pile_height_from_L(Ls, a0, a1, w1) / (a0 * Ls), color='black', label='fit', linewidth=0.5)

plt.xlabel("$L$", fontsize=14)
plt.ylabel(r"$\langle h \rangle /a_0L$", fontsize=14)
plt.axis((0, 600, 0.9, 1.0))
plt.legend(loc=4)
plt.show()


I am working on the file "Oslo Model.ipynb".
The first task is:
We see that the data points follow a power law decay if we exclude the first 22 (s >> 1) and last 10 points.
We can then fit it to estimate $\tau_s$:
Can you write Python code for it?

power_law = lambda s, a, tau_s: a * s ** tau_s

(a, tau_s), cov = curve_fit(power_law, centres[22:-10], probabilities[22:-10])

a, tau_s, analysis.calculate_std_from_cov_matrix(cov)

plt.loglog(centres, probabilities, '.', color='magenta', ms=4.0, label='data')

ava_sizes = np.logspace(0, 7, 100)
plt.loglog(ava_sizes, power_law(ava_sizes, a, tau_s), color='black', linewidth=0.5, label='fit')

plt.legend(loc=0)
plt.xlabel("$s$", fontsize=14)
plt.ylabel("$\widetilde P_N(s;L)$", fontsize=14)
plt.show()

for length, pile_dict in data_dict.items():
    centres, probabilities = analysis.make_log_bins(pile_dict['avalanche size'][-no_of_samples:], bin_scaling=1.3)
    s_tau_probabilities = centres ** -tau_s * probabilities
    plt.loglog(centres, s_tau_probabilities, '-', label=length, linewidth=0.5)

plt.legend(loc=0, title='System Size (L)', prop={'size':10})
plt.xlabel("$s$", fontsize=14)
plt.ylabel(r"$s^{\tau_s} \widetilde P_N(s;L)$", fontsize=14)
plt.show()


I am working on the file "Oslo Model.ipynb".
The first task is:
We see the equation is indeed satisfied.
$$\tau_s = 1.55$$
$$D = 2.25$$
Assuming $L \gg 1$, $s \gg 1$
For $k = 1$,
$$D(2-\tau_s)=1$$
which is satisfied with these obtained values.

Task 3d
Derive:
$$\langle s^k \rangle \propto L^{D(1+k-\tau_s)}$$
Measure the kth moment directly for $k = 1, 2, 3, 4, 5$:
Can you write Python code for it?

ks = (1, 2, 3, 4, 5)
kth_moment_list = []

for k in ks:
    current_kth_moments = []
    for length, pile_dict in data_dict.items():
        kth_moment = np.average(np.array(pile_dict['avalanche size'][-steady_state_time_period:], dtype='float64') ** k)
        current_kth_moments.append([length, kth_moment])
    kth_moment_list.append(current_kth_moments)

kth_moment_array = np.array(kth_moment_list)

for i, k in enumerate(ks):
    plt.loglog(kth_moment_array[i, :, 0], kth_moment_array[i, :, 1], '.', label='k = {}'.format(k))

plt.legend(loc=0)
plt.xlabel('$L$', fontsize=14)
plt.ylabel(r'$\langle s^k \rangle$', fontsize=14)
plt.show()


I am working on the file "Oslo Model.ipynb".
The first task is:
We see the line is not longer straight when system length is small. Boundary effects become siginificant
We first try fit with first order correction:
Can you write Python code for it?

get_average_pile_height_from_L = lambda L, a0, a1, w1: a0 * L * (1 - a1 * L ** (-w1))

(a0, a1, w1), cov = curve_fit(get_average_pile_height_from_L, system_lengths, average_heights, absolute_sigma=True)

a0, a1, w1, analysis.calculate_std_from_cov_matrix(cov)


I am working on the file "Other APIs Used.ipynb".
The first task is:
GovTrack API
https://www.govtrack.us/developers/api#endpoint_person
Can you write Python code for it?

#has OSID from Open Secrets and BioID from ProPublica

import requests
import urllib
import json
from __future__ import division
import math

base_url = 'https://www.govtrack.us/api/v2/person/400326'
id
r = requests.get(base_url)
data= r.json()
data


I am working on the file "Other APIs Used.ipynb".
The first task is:
Open Secrets API -
https://www.opensecrets.org/resources/create/api_doc.php
Can you write Python code for it?

# set base url
base_url="http://www.opensecrets.org/api/"

#set parameters
params = {"apikey" :  "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"method" : "memPFDprofile",
"cid" : "N00009888",
"year" : "2014",
"output" : "json"}

# make request
r = requests.get(base_url,params=params)

# parse data
data = (r.json())
legList = data['response']['legislator']
legKeys = data['response']['legislator'][0]['@attributes']

# export to csv
import csv
with open('leg.csv', 'w') as file:
    dict_writer = csv.DictWriter(file, legKeys)
    dict_writer.writeheader()
    for legObject in legList:
        dict_writer.writerow(legObject['@attributes'])


I am working on the file "Other functional features.ipynb".
The first task is:
A je jedno, ci maju rovnaky pocet parametrov alebo rozny. Ani definovanie typu pomocou anotacie v pythone 3 mi nepomoze
Vzdy si len prepisem funkciu inou
Nikdy sa nerozhodne na zaklade parametrov, ktora by sa mala pouzit (tak ako je to napriklad v jave)
Can you write Python code for it?

def pokus(a:str, b:list):
    print('pokus1')

def pokus(b:int):
    print('pokus2')

pokus('3', [])


I am working on the file "Other functional features.ipynb".
The first task is:
Balicek functools ma na to funkciu, ktora definiciu takychto funkcii robi este pohodlnejsiu
Can you write Python code for it?

from functools import partial

def power(base, exponent):
    return base ** exponent

cube = partial(power, 3)
cube(2)

def power(base, exponent):
    return base ** exponent

cube = partial(power, exponent=3)
cube(2)


I am working on the file "Other functional features.ipynb".
The first task is:
Co na to treba?

dekorator, ktory do nejakej struktury bude odkladat funkcie a parametre
je potrebne overenie, ktora funkcia je ta spravna
dekorator musi vratit funkciu, ktora sa pozrie do struktury s funkciami, postupne bude overovat, ci sa typy a pocty atributov zhoduju a potom jednu fukciu zavolat
cele to ma menej ako 20 riadkov (koho to zaujima, moze sa pozriet o par saljdov vyssie ako sa to da spravit)

Obmedzenia?

nefunguje to na zaklade pomenovanych atributov
neda sa pouzit premenlivy pocet atributov
atributy sa porovnavaju len na zaklade typov. Napada mi milion sposobov, ako by som chcel atributy porovnavat zlozitejsie

Mozno ina implementacia mi da vacsiu volnost
http://blog.chadselph.com/adding-functional-style-pattern-matching-to-python.html
Can you write Python code for it?

from patternmatching import ifmatches, Any, OfType, Where

@ifmatches
def greet(gender=OfType(str), name="Joey"):
    print("Joey, whats up man?")
@ifmatches
def greet(gender="male", name=Any):
    print("Hello Mr. {}".format(name))
@ifmatches
def greet(gender="female", name=Any):
    print("Hello Ms. {}".format(name))
@ifmatches
def greet(gender=Any, name=Where(str.isupper)):
    print("Hello {}. IMPORTANT".format("Mr" if gender == 'male' else "Ms"))
@ifmatches
def greet(gender=Any, name=Any):
    print("Hello, {}".format(name))

greet('male', 'JAKUB')


I am working on the file "Other functional features.ipynb".
The first task is:
Dalsie priklady na pouzitie partial pri refactoringu
http://chriskiehl.com/article/Cleaner-coding-through-partially-applied-functions/
A preco to nepouzit na specializovany konstruktor?
Can you write Python code for it?

class Tovar:
    def __init__(self, typ, mnozstvo=0):
        self.typ=typ
        self.mnozstvo=mnozstvo

    def write(self):
        return '{}: {}'.format(self.typ, self.mnozstvo)

nakup_jablk = Tovar('jablka', 3)
print(nakup_jablk.write())

Jablko = partial(Tovar, 'jablka')
Jablko(4).write()


I am working on the file "Other functional features.ipynb".
The first task is:
Iny priklad, uprveny konstruktor int
Can you write Python code for it?

basetwo = partial(int, base=2)
basetwo('111010101')


I am working on the file "Other functional features.ipynb".
The first task is:
Problem je v tom, ze skoro vsetky priklady na internete, ktore najdete su z toho ako vyrobit power funkcie alebo nieco podobne trivialne
Skusme nieco trivialne, ale praktickejsie
Napriklad funkciu, ktora ma vypisovat do nejakeho specialneho suboru. Napriklad chyboveho vystupu
Can you write Python code for it?

import sys
from functools import partial

print_stderr = partial(print, file=sys.stderr)

print_stderr("pokus")


I am working on the file "Other functional features.ipynb".
The first task is:
Skusme partial application pouzit na refaktorovanie takehoto kodu
Can you write Python code for it?

for text in lines:
    if re.search('[a-zA-Z]\=', text):
        some_action(text)
    elif re.search('[a-zA-Z]\s\=', text):
        some_other_action(text)
    else:
        some_default_action()


I am working on the file "Other functional features.ipynb".
The first task is:
Takto by sa to dalo spravit ak by sme pouzili partial application pomocou vnorenej funkcie.
Can you write Python code for it?

def get_query_agent(userid, password)
    def do_query(query) :
        # do query
        # return results
    return do_query

def bar(querying_func):
    return func(querying_func)

def foo(querying_func) :
    return bar(querying_func)

def main(userid, password) :
    query_agent = get_query_agent(userid, password)
    # .. much further down the line
    foo(query_agent)


I am working on the file "Other functional features.ipynb".
The first task is:
To iste by fungovalo aj na "objekt" vytvoreny pomocou closure
Can you write Python code for it?

import pyrsistent as ps

def Tovar(typ, mnozstvo):
    def write():
        return '{}: {}'.format(typ, mnozstvo)
    return ps.freeze({'write': write})

Jablko = partial(Tovar, 'jablka')
Jablko(5).write()


I am working on the file "Other functional features.ipynb".
The first task is:
Toto nebol multiple dispatch. Toto bol overloading pretoze sa to rozhodovalo v case kompilacie.
preto by sa vypisalo "Hello banana" na zaklade typu premennej a nie "Hello Fruit" na zaklade typu objektu
multiple dispatch sa rozhoduje dynamicky na zaklade objektu
Multiple dispatch by som dosiahol napriklad ak by print bola metoda objektu.
Nanestastie Python nema ani multiple dispatch a ani overloading
Nema zmysel definovat dve funkcie s rovnakym menom
Can you write Python code for it?

def pokus(a):
    print('pokus1')

def pokus():
    print('pokus2')

pokus()


I am working on the file "Other functional features.ipynb".
The first task is:
V standardnej kniznici jendoducho nie su prostriedky na to, aby som vedel definovat vacero rovnakych fukcii a na zaklade atributov rozhodnut ktora sa ma zavolat
toto plati aj pre metody
nevieme napriklad definovat ani metodu triedy a objektu, ktora sa rovnako vola :(
Vela ludom uz napadlo, ze by nieco take bolo celkom cool a spravili nejake pokusy o zapracovanie do jazyka
http://www.grantjenks.com/docs/pypatt-python-pattern-matching/

https://github.com/lihaoyi/macropy - module import
https://github.com/Suor/patterns - decorator with funky syntax - Shared at Python Brazil 2013
https://github.com/mariusae/match - http://monkey.org/~marius/pattern-matching-in-python.html - operator overloading
http://blog.chadselph.com/adding-functional-style-pattern-matching-to-python.html - multi-methods
http://svn.colorstudy.com/home/ianb/recipes/patmatch.py - multi-methods
http://www.artima.com/weblogs/viewpost.jsp?thread=101605 - the original multi-methods
http://speak.codebunk.com/post/77084204957/pattern-matching-in-python - multi-methods supporting callables
http://www.aclevername.com/projects/splarnektity/ - not sure how it works but the syntax leaves a lot to be desired
https://github.com/martinblech/pyfpm - multi-dispatch with string parsing
https://github.com/jldupont/pyfnc - multi-dispatch
http://www.pyret.org/ - It’s own language

Ziadna z tychto kniznic nie je taka dobra ako plnohodnotne zapracovana vlastnost do funkcionalneho jazyka, ale skusim aspon na takomto chabom priklade ukazat, co by sa s niecim takymto dalo robit.
Multimethods
uz aj Guido van Rossum si vsimol, ze by to mohlo byt celkom fajn
http://www.artima.com/weblogs/viewpost.jsp?thread=101605
Can you write Python code for it?

# casto sa stava, ze kod vyzera nejak takto
def foo(a, b):
    if isinstance(a, int) and isinstance(b, int):
        # ...code for two ints...
    elif isinstance(a, float) and isinstance(b, float):
        # ...code for two floats...
    elif isinstance(a, str) and isinstance(b, str):
        # ...code for two strings...
    else:
        raise TypeError("unsupported argument types (%s, %s)" % (type(a), type(b)))


I am working on the file "Other functional features.ipynb".
The first task is:
Viete si tak vytvorit viacere konstruktory pre tu istu triedu
Co vam brani vytvorit si konstruktor pre nejaky specialny typ loggera alebo objektu na citanie nejakeho specialneho typu suboru.
Nemsuite stale opakovat tie iste parametre vo volani konstruktora / funkcie.
Viete to pouzit nie len na specializovanie, ale aj na oddelenie zadavania parametrov funkcie a jej vykonania v case.
Kolko krat sa vam stalo, ze ste vedeli davno v programe aku funkciu budete musiet zavolat a aj s castou argumentov, ale museli ste cakat az do nejakeho casu, kde ste dostali aj zvysok a museli ste parametre predavat spolu s funkciou / objektom na ktorom bola metoda
Ak by ste vedeli vyrobit funkciu, s niektorymi parametrami prednastavenymi, tak by vam stacilo posuvat si tuto jednu funkciu a nemuseli by ste si presuvat vsetky parametre az do miesta, kde ich nakoniec vlozite pri volani funkcie
Can you write Python code for it?

def query_database(userid, password, query) :
    # do query
    # return results

def bar(userid, password):
    return query_database(userid, password)

def foo(userid, password) :
    return bar(userid, password)

def main(userid, password) :
    # .. lot of code here .. eventually reaching
    foo(userid, password)


I am working on the file "Other functional features.ipynb".
The first task is:
regularne vyrazy sa daju vytiahnut do funkcie
Can you write Python code for it?

def is_grouped_together(text): # skuste z tohoto spravit partial
    return re.search("[a-zA-Z]\s\=", text)

def is_spaced_apart(text):
    return re.search("[a-zA-Z]\s\=", text)

def and_so_on(text):
    return re.search("pattern_188364625", text)

for text in lines:
    if is_grouped_together(text):
        some_action(text)
    elif is_spaced_apart(text):
        some_other_action(text)
    else:
        some_default_action()


I am working on the file "Other/las-vegas-dice-with-comments.ipynb".
The first task is:
Create the Target die
In this section we will now create the target die, which we will entangle to our source die.
To simplify this let's create the second die by adding 3 more qubits. Then the first three qubits we can refer to as the source die, and the second set of three qubits are for the target die.
For example, the classical bit result {010111} means that the source die (first three bits from left, LSB) = 111, and the target die (remaing three bits) are the bits to the right of the first three = 010.
Can you write Python code for it?

# load IBM Q account, only if you want to run this on a real device
IBMQ.load_accounts()

# First let's double the size of our bit set.
bitSize = 3

# since we have just two dice, source and target, we will multiply by two.
q = QuantumRegister(2*bitSize)
c = ClassicalRegister(2*bitSize)
# Create a Quantum Circuit, same as before.
qc = QuantumCircuit(q, c)

# now, just as before, let's set our source die into superposition and measure
qc.h(q)
qc.barrier(q)
qc.measure(q, c)

# Review the openqasm code.
QASM_source = qc.qasm()
print(QASM_source)


I am working on the file "Other/las-vegas-dice-with-comments.ipynb".
The first task is:
Note the measure mapping above maps to 6 classic bits, we will view bits 0-2 (first three) as the source, and bits 3-5 (remaining three) as the target.
Can you write Python code for it?

# Now draw the circuit with a bundled classical register
qc.draw()


I am working on the file "Other/las-vegas-dice-with-comments.ipynb".
The first task is:
Now let's create the circuit to implement and visualize the quantum circuit needed to create the entangled dice.
In this example, to keep things simple, we will create an 8-sided die with binary values 0-7.
Can you write Python code for it?

# set up Quantum Register and Classical Register for 3 qubits (binary values 000 -> 111)
q = QuantumRegister(3)
c = ClassicalRegister(3)
# Create a Quantum Circuit
qc = QuantumCircuit(q, c)

# now let's set our source die into superposition and measure
# so to simulate random die values 0-7, or 1-8 since we dice do not have a 0 value.
# Note: the 'barrier' is there just so the circuit displays well, it has no effect on the circuit itself.
qc.h(q)
qc.barrier(q)
qc.measure(q, c)

# For completeness, let's see what the openqasm code looks like.
QASM_source = qc.qasm()
print(QASM_source)

# While we're at it, let's also look at our circuit visually.
# This is similar to what we would see in the IBM Q Composer.
qc.draw()


I am working on the file "Other/las-vegas-dice-with-comments.ipynb".
The first task is:
Now let's run and test out our die. We will run a single shot (to simulate rolling the die on a casino table).
Rerun the cell below a few times so to see the values of the die change for each roll.
Can you write Python code for it?

from qiskit import execute
job = execute(qc, backend=Aer.get_backend('qasm_simulator'), shots=1)
result = job.result().get_counts(qc)
print(result)


I am working on the file "Other_files/DEMO_KiDS_data.ipynb".
The first task is:
Creating masked cataloge for cross correlation pipeline:
Can you write Python code for it?

# For output

def create_KiDS_standard_catalog_file(KiDS_catalog, output_filename):
    hdu = fits.open(KiDS_catalog)
    w = hdu[2].data["weight"]
    z = hdu[2].data["z_B"]
    pf = hdu[2].data["TILE_SYS_OK"]
    mask = np.logical_and(w > 0.0,
                          np.logical_and(0.005 < z,
                                         np.logical_and( z < 1.2,
                                                        np.logical_and(pf == 1, hdu[2].data["SNratio"] > 0))))
    print("Number of objects in {}: {}. Number of objects after masking: {}.".format(KiDS_catalog, len(mask), np.count_nonzero(mask)))
    fields = hdu[2].data["FIELD_POS"][mask]
    ra = hdu[2].data["ALPHA_J2000"][mask]
    dec = hdu[2].data["DELTA_J2000"][mask]
    e1 = hdu[2].data["e1"][mask]
    e2 = hdu[2].data["e2"][mask]
    c1 = hdu[2].data["c1_best"][mask]
    c2 = hdu[2].data["c2_best"][mask]
    m = hdu[2].data["m_cor_best"][mask]

    hdu.close()

    w = w[mask]
    z = z[mask]

    c1_mask = c1 > -99
    c2_mask = c2 > -99
    e1[c1_mask] -= c1[c1_mask]
    e2[c2_mask] -= c2[c2_mask]

    x_min = np.min(ra)
    x_max = np.max(ra)
    y_min = np.min(dec)
    y_max = np.max(dec)

#    with open(output_filename + "_info.dat", "w") as f:
#        f.write("#RA background: {} -> {}.\n".format(x_min, x_max))
#        f.write("#DEC background: {} -> {}.\n".format(y_min, y_max))
#        f.write("#Size: {}, {}\n".format(x_max-x_min, y_max-y_min))
#        f.write("#center(RA, Dec): {}, {}\n".format((x_max+x_min)/2, (y_max+y_min)/2))
#        f.write("{} {}\n".format(x_min, x_max))
#        f.write("{} {}\n".format(y_min, y_max))
#        f.write("{} {}\n".format(x_max-x_min, y_max-y_min))
#        f.write("{} {}\n".format((x_max+x_min)/2, (y_max+y_min)/2))

    print("Writing to file " + output_filename)
    np.savetxt(output_filename, np.vstack((ra, dec, z, e1, e2, w, m)).T)...


I am working on the file "Other_files/DEMO_KiDS_data.ipynb".
The first task is:
Loading  G15 KiDS field :
Can you write Python code for it?

hdulist = fits.open('KiDS_G15_2015.cat')
hdulist.info()

hdu2= hdulist[2]
hdu1= hdulist[1]

hdu1.data['OBJECT_POS']

hdu2.header

#for i in range (1200):
for i in [0,2749998]:
    d = hdu2.data[i]
    #plt.plot
    print d[13],d[14]
    #plt.plot(d[13],d[14])

#hdu2.data[0]['Xpos']

# TTYPE1  = 'FIELD_POS'
# TTYPE14 = 'Xpos
# TTYPE15 = 'Ypos
# TTYPE16 = 'ALPHA_J2000'        /
# TTYPE17 = 'DELTA_J2000'        /
# TTYPE87 = 'e1      '           /
# TTYPE88 = 'e2      '           /
# TTYPE96 = 'c1_best '           / Updated empirical correction c1
# TTYPE97 = 'c2_best '           / Updated empirical correction c2
# TTYPE98 = 'm_cor_best'         / calibration correction m_cor pixel scale correc

# TTYPE101= 'TILE_SYS_OK'        / Pass = 1, Fail = 0


I am working on the file "Other_files/DEMO_KiDS_data.ipynb".
The first task is:
Reading data in the G15 field and creating rectangular boxes from sub-field coordinates.
Visualizing the overlap with GAMMA groups in the same region:
Can you write Python code for it?

import astropy.io.fits as fits
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.patches

KiDS_field = 'G15'

hdu = fits.open('KiDS_G15_2015.cat')

fields = hdu[2].data["FIELD_POS"]
ra = hdu[2].data["ALPHA_J2000"]
dec = hdu[2].data["DELTA_J2000"]
hdu.close()
field_info = []

print "min/max RA  Global: ", np.min(ra), np.max(ra)
print "min/max DEC Global: ", np.min(dec), np.max(dec)

for i in range(np.max(fields)):
    x = ra[fields == i+1]
    y = dec[fields == i+1]

    x_min = np.min(x)
    x_max = np.max(x)
    y_min = np.min(y)
    y_max = np.max(y)

    field_info.append([x_min, x_max, y_min, y_max])

#    print("Field {}.".format(i+1))
#    print("RA: {} - {}".format(x_min, x_max))
#    print("Dec: {} - {}".format(y_min, y_max))

field_info = np.array(field_info)

hdu = fits.open("G3CFoFGroupv06.fits")

hdu[1].verify("fix")
mask = hdu[1].data["Nfof"] >= 5
group_ra = hdu[1].data["BCGRA"][mask]
group_dec = hdu[1].data["BCGDec"][mask]
group_z = hdu[1].data["Zfof"][mask]
hdu.close()

ax = plt.gca()
for i in range(field_info.shape[0]):
    bbox_pos = ((field_info[i,1]+field_info[i,0])/2.0 , (field_info[i,3]+field_info[i,2])/2.0 )
    bbox_size = ((field_info[i,1]-field_info[i,0]), field_info[i,3]-field_info[i,2])
    ax.add_patch(matplotlib.patches.Rectangle(bbox_pos, bbox_size[0] , bbox_size[1], fill=True, alpha=1.0, facecolor='m'))

plt.scatter(group_ra, group_dec)

plt.xlim(np.min(field_info[:,0]-1), np.max(field_info[:,1])+1)
plt.ylim(np.min(field_info[:,2]-1), np.max(field_info[:,3])+1)

plt.xlabel("RA")
plt.ylabel("DEC")

plt.suptitle(KiDS_field)

#plt.savefig(KiDS_field+'_footprint.pdf')

ra_min = np.min(ra)
ra_max = np.max(ra)
dec_min = np.min(dec)
dec_max = np.max(dec)
nx = int((ra_max-ra_min)*60.)
ny = int((dec_max-dec_min)*60.)


I am working on the file "Other_files/DEMO_tSZXlensing_stripped.ipynb".
The first task is:
2pcf
Can you write Python code for it?

marks_file = "/data/aha25/tpcf_marks/CFHTLenS/CFHTLenS_kappa_scale_{}_theta_{}_{}_nbin_{}_{}_pad{}_marks.fits".format(scale_CFHTLenS, theta_min_CFHTLenS*60.0, theta_max_CFHTLenS*60.0, n_bin_CFHTLenS, bin_type_CFHTLenS, pad_CFHTLenS)

foreground_path = "/data/aha25/tSZ/tSZ_data/CFHTLenS/pad{}/".format(pad_CFHTLenS)
tpcf_output_path = "/data/aha25/tSZ/tpcf/CFHTLenS_kappa/scale{}/".format(scale_CFHTLenS)
tpcf_formats.mkdirs(tpcf_output_path)
tpcf_driver.run_marks_crosscorrelation("kappa", foreground_path=foreground_path, marks_file=marks_file, tpcf_output_path=tpcf_output_path,
                                       fields=CFHTLenS_fields,
                                       n_bin=n_bin_CFHTLenS, theta_min=theta_min_CFHTLenS, theta_max=theta_max_CFHTLenS, logspaced=logspaced_CFHTLenS,
                                       bootstrap_configs=bootstrap_configs,
                                       n_thread=n_threads, tpcf_path=tpcf_path, verbose=True)

tSZ_path = os.path.join("/data/aha25/tSZ/tSZ_data/sims/", cosmology_sims, feedback_model_sims)
shear_path =  os.path.join("/data/aha25/shear_catalogs/cosmo-OWLS/", n_z_sims, cosmology_sims, feedback_model_sims)
kappa_path =  os.path.join("/data/aha25/massmaps/cosmo-OWLS/", n_z_sims, cosmology_sims, feedback_model_sims)

for i in range(n_LOS):
    tpcf_output_base_path = "/data/aha25/tSZ/tpcf/cosmo-OWLS/{}/".format(i)
    if not os.access(tpcf_output_base_path, os.F_OK):
        os.mkdir(tpcf_output_base_path)
    if not os.access(os.path.join(tpcf_output_base_path, n_z_sims), os.F_OK):
        os.mkdir(os.path.join(tpcf_output_base_path, n_z_sims))
    if not os.access(os.path.join(tpcf_output_base_path, n_z_sims, cosmology_sims), os.F_OK):
        os.mkdir(os.path.join(tpcf_output_base_path, n_z_sims, cosmology_sims))
    if not os.access(os.path.join(tpcf_output_base_path, n_z_sims, cosmology_sims, feedback_model_sims), os.F_OK):
        os.mkdir(os.path.join(tpcf_output_base_path, n_z_sims, cosmology_sims, feedback_model_sims))
    tpcf_output_path =...


I am working on the file "Other_files/DEMO_tSZXlensing_stripped.ipynb".
The first task is:
Computing the Fourier space power spectrum, real space correlation function :
Can you write Python code for it?

ell_log_C_ell_Planck_AGN80_bin2_intp = scipy.interpolate.InterpolatedUnivariateSpline(np.log(C_ell_Planck_AGN80_bin2_raw[:,0]), np.log(C_ell_Planck_AGN80_bin2_raw[:,0]*C_ell_Planck_AGN80_bin2_raw[:,1]), ext="const")
ell_log_C_ell_WMAP7_AGN80_bin2_intp = scipy.interpolate.InterpolatedUnivariateSpline(np.log(C_ell_WMAP7_AGN80_bin2_raw[:,0]), np.log(C_ell_WMAP7_AGN80_bin2_raw[:,0]*C_ell_WMAP7_AGN80_bin2_raw[:,1]), ext="const")
ell_log_C_ell_WMAP7_AGN80_CFHTLenS_intp = scipy.interpolate.InterpolatedUnivariateSpline(np.log(C_ell_WMAP7_AGN80_CFHTLenS_raw[:,0]), np.log(C_ell_WMAP7_AGN80_CFHTLenS_raw[:,0]*C_ell_WMAP7_AGN80_CFHTLenS_raw[:,1]), ext="const")

C_ell_Planck_AGN80_bin2 = lambda ell: np.exp(ell_log_C_ell_Planck_AGN80_bin2_intp(np.log(ell)))/ell
C_ell_WMAP7_AGN80_bin2 = lambda ell: np.exp(ell_log_C_ell_WMAP7_AGN80_bin2_intp(np.log(ell)))/ell
C_ell_WMAP7_AGN80_CFHTLenS = lambda ell: np.exp(ell_log_C_ell_WMAP7_AGN80_CFHTLenS_intp(np.log(ell)))/ell

ell = np.arange(1, C_ell_Planck_AGN80_bin2_raw[-1,0]+1)

theta_th = np.linspace(0.1, 181.0, 80.0)

xi_th_kappa_Planck_AGN80_bin2 = np.zeros_like(theta_th)
xi_th_g_t_Planck_AGN80_bin2 = np.zeros_like(theta_th)
xi_th_kappa_WMAP7_AGN80_bin2 = np.zeros_like(theta_th)
xi_th_g_t_WMAP7_AGN80_bin2 = np.zeros_like(theta_th)
xi_th_kappa_WMAP7_AGN80_CFHTLenS = np.zeros_like(theta_th)
xi_th_g_t_WMAP7_AGN80_CFHTLenS = np.zeros_like(theta_th)

sigma_Planck = 9.5/60.0/180.0*pi/(2.0*np.sqrt(2.0* np.log(2.0)))
Planck_smoothing = np.exp(-sigma_Planck**2*ell**2/2.0)
sigma_RCSLenS = 2.0/60.0/180.0*pi
RCSLenS_smoothing = np.exp(-sigma_RCSLenS**2*ell**2/2.0)

for i in range(len(theta_th)):
    xi_th_kappa_Planck_AGN80_bin2[i] = 1.0e-11*1.0/(2.0*pi) * np.sum(C_ell_Planck_AGN80_bin2(ell) * ell * scipy.special.jv(0, ell*theta_th[i]/60.0/180.0*pi))
    xi_th_g_t_Planck_AGN80_bin2[i] = 1.0e-11*1.0/(2.0*pi) * np.sum(C_ell_Planck_AGN80_bin2(ell) * ell * scipy.special.jv(2, ell*theta_th[i]/60.0/180.0*pi))
    xi_th_kappa_WMAP7_AGN80_bin2[i] = 1.0e-11*1.0/(2.0*pi) *...


I am working on the file "Other_files/DEMO_tSZXlensing_stripped.ipynb".
The first task is:
Creating tSZ catalogs from maps (fits files):
RCSLenS:
Can you write Python code for it?

pad_RCSLenS = 4.0

y_map_file = "/shared_data/Planck_data/COM_Compton_SZMap_R2.00/milca_ymaps.fits"
mask_file = "/shared_data/Planck_data/COM_Compton_SZMap_R2.00/masks.fits"

RCSLenS_fields = ["CDE0047", "CDE0133", "CDE0310", "CDE0357", "CDE1040", "CDE1111", "CDE1303", "CDE1514", "CDE1613", "CDE1645", "CDE2143", "CDE2329", "CDE2338", "CSP0320"]
shear_path = "/data/aha25/shear_catalogs/RCSLenS/mag_18_30/"
output_path = "/data/aha25/tSZ/tSZ_data/RCSLenS/pad{}/".format(pad_RCSLenS)
create_tSZ_catalog(RCSLenS_fields, y_map_file, mask_file, shear_path, pad_RCSLenS, output_path, survey="RCSLenS")


I am working on the file "Other_files/DEMO_tSZXlensing_stripped.ipynb".
The first task is:
Grid the data
Can you write Python code for it?

RCSLenS_fields = ["CDE0047", "CDE0133", "CDE0310", "CDE0357", "CDE1040", "CDE1111", "CDE1303", "CDE1514", "CDE1613", "CDE1645", "CDE2143", "CDE2329", "CDE2338", "CSP0320"]
for field in RCSLenS_fields:
    lenses_filename = "/home/aha25/y_maps/{}_y_planck_milca_2048_big_new.fits".format(field)
    tSZ_map_output = "/data/aha25/tSZ/tSZ_data/RCSLenS_grid/pad2/{}.fits".format(field)
    tpcf_formats.create_catalog_from_grids(mode="lenses",
                                           filename=lenses_filename, output_filename=tSZ_map_output,
                                           pixel_size=1.0/60.0, weight_filename=None, remove_mean=True,
                                           survey="RCSLenS", field=field)


I am working on the file "Other_files/LMFIT_tutorial.ipynb".
The first task is:
Another way of defining the parameters :
Can you write Python code for it?

def get_residual(params,x, data):
    #amp= params['amp'].value
    #offset=params['offset'].value
    #xoffset=params['xoffset'].value

    parvals = params.valuesdict()
    amp = parvals['amp']
    offset = parvals['offset']

    model =amp* x**2 +offset

    return data-model


I am working on the file "Other_files/LMFIT_tutorial.ipynb".
The first task is:
LMFIT :

Using Parameter objects instead of plain floats as variables. A parameter value:
can be varied in the fit
have a fixed value
have upper and/or lower bounds
constrained by an algebraic expression of other Parameter values

Ease of changing fitting algorithms. Once a fitting model is set up, one can change the fitting algorithm used to find the optimal solution without changing the objective function.

Improved estimation of confidence intervals. While scipy.optimize.leastsq() will automatically calculate uncertainties and correlations from the covariance matrix, the accuracy of these estimates are often questionable. To help address this, lmfit has functions to explicitly explore parameter space to determine confidence levels even for the most difficult cases.

Improved curve-fitting with the Model class. This extends the capabilities of scipy.optimize.curve_fit(), allowing you to turn a function that models for your data into a python class that helps you parametrize and fit data with that model.

Many pre-built models for common lineshapes are included and ready to use.

minimize & Parameters:
Can you write Python code for it?

from lmfit import minimize, Parameters

params = Parameters()
params.add('amp', value=0.)
params.add('offset', value=0.)

def get_residual(params,x, data):
    amp= params['amp'].value
    offset=params['offset'].value

    model =amp* x**2 +offset

    return data-model

out = minimize(get_residual, params, args=(x, data))

dir(out)

out.params

dir(out.params)

out.params.values


I am working on the file "Other_files/LMFIT_tutorial.ipynb".
The first task is:
Lets build a general quadratic model:
Can you write Python code for it?

def get_residual(vars,x, data):
    a= vars[0]
    b=vars[1]
    model =a* x**2 +b
    return data-model

vars=[1.,0.]
print get_residual(vars,x,data)
print sum(get_residual(vars,x,data))

vars=[1.,1.]
print sum(get_residual(vars,x,data))

vars=[2.,0.]
print sum(get_residual(vars,x,data))


I am working on the file "Other_files/LMFIT_tutorial.ipynb".
The first task is:
Lets visualize how a quadratic curve fits to it:
Can you write Python code for it?

plt.scatter(x,data)
xarray=np.arange(-1,4,0.1)
plt.plot(xarray, xarray**2,'r-')  # Not the best fit
plt.plot(xarray, xarray**2+1,'g-')


I am working on the file "Other_files/LMFIT_tutorial.ipynb".
The first task is:
Questions ?
leastsq function from scipy:
Can you write Python code for it?

from scipy.optimize import leastsq
vars = [0.,0.]
out = leastsq(get_residual, vars, args=(x, data))
print out

vars=[1.06734694,  0.96428571]
print sum(get_residual(vars,x,data)**2)

vars=[1.06734694,  0.96428571]
plt.scatter(x,data)
xarray=np.arange(-1,4,0.1)
plt.plot(xarray, xarray**2,'r-')
plt.plot(xarray, xarray**2+1,'g-')
fitted = vars[0]* xarray**2 +vars[1]
plt.plot(xarray, fitted,'b-')


I am working on the file "Other_files/LMFIT_tutorial.ipynb".
The first task is:
correl:
Can you write Python code for it?


I am working on the file "Other_files/LMFIT_tutorial.ipynb".
The first task is:
report_fit: For a better report :
Can you write Python code for it?

from lmfit import minimize, Parameters, Parameter, report_fit
result = minimize(get_residual, params, args=(x, data))

help(report_fit)

# write error report
report_fit(result.params)


I am working on the file "Other_files/LMFIT_tutorial.ipynb".
The first task is:
stderr:
Can you write Python code for it?


I am working on the file "Outliers.ipynb".
The first task is:
Agora os gastos estão muito mais próximos da distribuição normal padrão. Segundo Andrew Ng, a distribuição não precisa ser muito igual à normal para obter bons resultados. Podemos seguir para o próximo passo: calcular a probabilidade da ocorrência de cada valor. Talvez você já tenha ouvido falar em "6 Sigma" ($6\sigma$) e esse nome vem do fato de que um intervalo entre $-3 \sigma$ e $3\sigma$ abrange quase 100% dos valores de uma distribuição normal. Na distribuição normal padrão, $\sigma = 1$.
Vamos então calcular a probabilidade de cada gasto, supondo que eles sigam uma distribuição normal e, em seguida, mostrar aqueles que possuem menor probabilidade de ocorrência (os 5 primeiros).
Can you write Python code for it?

from scipy.stats import norm

def prob(valores):
    probs = valores.copy()
    probs[probs <= 0] = np.nan
    z = log_zscores(probs)
    probs[z.index] = norm.sf(z)
    return probs

ceap['prob_geral'] = prob(ceap.vlrLiquido)
colunas = ['prob_geral', 'vlrLiquido', 'txNomeParlamentar', 'sgPartido', 'sgUF', 'txtDescricao']
ceap[colunas].sort_values('prob_geral').head()


I am working on the file "Outliers.ipynb".
The first task is:
Bem diferente da distribuição normal padrão, não é? Embora invisíveis nessa escala, há alguns poucos gastos muito altos à direita. Além disso, notamos muitos gastos próximo do zero e uma diminuição brusca da barra ao lado.
Para que os valores se aproximem da normal, vamos transformá-los aplicando logaritmo, subtraindo a média e dividindo pelo desvio padrão. Vamos ver o resultado e a curva normal sobrepostos:
Can you write Python code for it?

def log_zscores(valores):
    positivos = valores[valores > 0].dropna()
    logs = np.log(positivos)
    return (logs - logs.mean()) / logs.std()

vlrLiquido_z = log_zscores(ceap.vlrLiquido)

pd.concat([aleatorios, vlrLiquido_z], axis=1).plot.hist(bins=75, alpha=0.6);


I am working on the file "Outliers.ipynb".
The first task is:
Descobrimos então que outros parlamentares gastaram ainda mais para divulgar suas atividades. Nesse momento, seu foco pode ter mudado dos R\$ 88.500,00 de Jhonatan de Jesus para os R\$ 189.600,00 de Arnaldo Faria de Sá. Comparando os gastos da tabela acima, o primeiro colocado se destoa a ponto de investigarmos melhor esse gasto? Note que começamos com uma ideia: o maior gasto do parlamentar que mais gastou no ano e, conforme investigamos, mudamos o rumo para o maior gasto com divulgação entre todos os deputados. Isso pode acontecer repetidas vezes até que de fato escolhamos um gasto para investigar mais a fundo.
Como você já deve ter percebido, a análise manual é muito trabalhosa nas mais de 350 mil entradas que temos. Vejamos agora como processá-las de forma mais objetiva e automatizada.
Aprendizagem de Máquina
Vamos usar uma técnica simples de detecção de outliers lecionada no Coursera por Andrew Ng. Essa técnica diz a probabilidade de um valor específico ocorrer no grupo. Para que ela funcione, os valores devem seguir aproximadamente uma distribuição normal. Não pretendo entrar em detalhes sobre estatística, apenas o suficiente para nos certificarmos de que teremos bons resultados.
Todos os valores
Primeiro, vamos considerar todos os gastos de uma só vez. Será que os valores possuem uma distribuição normal? Veja um exemplo de distribuição normal (padrão):
Can you write Python code for it?

import matplotlib   # gráficos
import numpy as np  # cálculos

%matplotlib inline
matplotlib.style.use('ggplot')

positivos = ceap[ceap.vlrLiquido > 0].vlrLiquido
aleatorios = pd.Series(np.random.randn(len(positivos)), name='normal')
aleatorios.plot.hist(bins=75, ylim=(0, 35000));


I am working on the file "Outliers.ipynb".
The first task is:
Observe que os valores de x = 0 são mais frequentes e a frequência diminui para as laterais. Você pode conferir mais detalhes sobre a Distribuição Normal na Wikipedia.
Como é a distribuição dos valores da cota parlamentar?
Can you write Python code for it?


I am working on the file "Outliers.ipynb".
The first task is:
Será que um pagamento de R$ 88.500,00 para divulgação da atividade parlamentar é muito alto? Vamos ver os 5 maiores pagamentos desse tipo, entre todos os deputados, ordenado do maior pro menor:
Can you write Python code for it?

colunas = ['vlrLiquido', 'txNomeParlamentar', 'sgPartido', 'sgUF', 'txtDescricao']
ceap.query('numSubCota == 5')[colunas].sort_values('vlrLiquido', ascending=False).head()


I am working on the file "Overfitting.ipynb".
The first task is:
And fit the model in the usual way:
Can you write Python code for it?

fc_model.fit(trn_features, trn_labels, nb_epoch=8,
             batch_size=batch_size, validation_data=(val_features, val_labels))

fc_model.save_weights(model_path+'no_dropout.h5')

fc_model.load_weights(model_path+'no_dropout.h5')


I am working on the file "Overfitting.ipynb".
The first task is:
Are we underfitting?
Our validation accuracy so far has generally been higher than our training accuracy. That leads to two obvious questions:

How is this possible?
Is this desirable?

The answer to (1) is that this is happening because of dropout. Dropout refers to a layer that randomly deletes (i.e. sets to zero) each activation in the previous layer with probability p (generally 0.5). This only happens during training, not when calculating the accuracy on the validation set, which is why the validation set can show higher accuracy than the training set.
The purpose of dropout is to avoid overfitting. By deleting parts of the neural network at random during training, it ensures that no one part of the network can overfit to one part of the training set. The creation of dropout was one of the key developments in deep learning, and has allowed us to create rich models without overfitting. However, it can also result in underfitting if overused, and this is something we should be careful of with our model.
So the answer to (2) is: this is probably not desirable. It is likely that we can get better validation set results with less (or no) dropout, if we're seeing that validation accuracy is higher than training accuracy - a strong sign of underfitting. So let's try removing dropout entirely, and see what happens!
(We had dropout in this model already because the VGG authors found it necessary for the imagenet competition. But that doesn't mean it's necessary for dogs v cats, so we will do our own analysis of regularization approaches from scratch.)
Removing dropout
Our high level approach here will be to start with our fine-tuned cats vs dogs model (with dropout), then fine-tune all the dense layers, after removing dropout from them. The steps we will take are:
- Re-create and load our modified VGG model with binary dependent (i.e. dogs v cats)
- Split the model between the convolutional (conv) layers and the dense layers
- Pre-calculate the output of the conv layers, so that we don't have to redundently...
Can you write Python code for it?


I am working on the file "Overfitting.ipynb".
The first task is:
As you can see below, there's no magic to data augmentation - it's a very intuitive approach to generating richer input data. Generally speaking, your intuition should be a good guide to appropriate data augmentation, although it's a good idea to test your intuition by checking the results of different augmentation approaches.
Can you write Python code for it?

# Augmented data
plots(aug_imgs, (20,7), 2)

# Ensure that we return to theano dimension ordering
K.set_image_dim_ordering('th')


I am working on the file "Overfitting.ipynb".
The first task is:
Batch normalization
About batch normalization
Batch normalization (batchnorm) is a way to ensure that activations don't become too high or too low at any point in the model. Adjusting activations so they are of similar scales is called normalization. Normalization is very helpful for fast training - if some activations are very high, they will saturate the model and create very large gradients, causing training to fail; if very low, they will cause training to proceed very slowly. Furthermore, large or small activations in one layer will tend to result in even larger or smaller activations in later layers, since the activations get multiplied repeatedly across the layers.
Prior to the development of batchnorm in 2015, only the inputs to a model could be effectively normalized - by simply subtracting their mean and dividing by their standard deviation. However, weights in intermediate layers could easily become poorly scaled, due to problems in weight initialization, or a high learning rate combined with random fluctuations in weights.
Batchnorm resolves this problem by normalizing each intermediate layer as well. The details of how it works are not terribly important (although I will outline them in a moment) - the important takeaway is that all modern networks should use batchnorm, or something equivalent. There are two reasons for this:
1. Adding batchnorm to a model can result in 10x or more improvements in training speed
2. Because normalization greatly reduces the ability of a small number of outlying inputs to over-influence the training, it also tends to reduce overfitting.
As promised, here's a brief outline of how batchnorm works. As a first step, it normalizes intermediate layers in the same way as input layers can be normalized. But this on its own would not be enough, since the model would then just push the weights up or down indefinitely to try to undo this normalization. Therefore, batchnorm takes two additional steps:
1. Add two more trainable parameters to each layer - one to multiply all...
Can you write Python code for it?

conv_layers[-1].output_shape[1:]

def get_bn_layers(p):
    return [
        MaxPooling2D(input_shape=conv_layers[-1].output_shape[1:]),
        Flatten(),
        Dense(4096, activation='relu'),
        Dropout(p),
        BatchNormalization(),
        Dense(4096, activation='relu'),
        Dropout(p),
        BatchNormalization(),
        Dense(1000, activation='softmax')
        ]

p=0.6

bn_model = Sequential(get_bn_layers(0.6))

bn_model.load_weights('/data/jhoward/ILSVRC2012_img/bn_do3_1.h5')

def proc_wgts(layer, prev_p, new_p):
    scal = (1-prev_p)/(1-new_p)
    return [o*scal for o in layer.get_weights()]

for l in bn_model.layers:
    if type(l)==Dense: l.set_weights(proc_wgts(l, 0.3, 0.6))

bn_model.pop()
for layer in bn_model.layers: layer.trainable=False

bn_model.add(Dense(2,activation='softmax'))

bn_model.compile(Adam(), 'categorical_crossentropy', metrics=['accuracy'])

bn_model.fit(trn_features, trn_labels, nb_epoch=8, validation_data=(val_features, val_labels))

bn_model.save_weights(model_path+'bn.h5')

bn_model.load_weights(model_path+'bn.h5')

bn_layers = get_bn_layers(0.6)
bn_layers.pop()
bn_layers.append(Dense(2,activation='softmax'))

final_model = Sequential(conv_layers)
for layer in final_model.layers: layer.trainable = False
for layer in bn_layers: final_model.add(layer)

for l1,l2 in zip(bn_model.layers, bn_layers):
    l2.set_weights(l1.get_weights())

final_model.compile(optimizer=Adam(),
                    loss='categorical_crossentropy', metrics=['accuracy'])

final_model.fit_generator(batches, samples_per_epoch=batches.nb_sample, nb_epoch=1,
                        validation_data=val_batches, nb_val_samples=val_batches.nb_sample)

final_model.save_weights(model_path + 'final1.h5')

final_model.fit_generator(batches, samples_per_epoch=batches.nb_sample, nb_epoch=4,
                        validation_data=val_batches, nb_val_samples=val_batches.nb_sample)

final_model.save_weights(model_path +...


I am working on the file "Overfitting.ipynb".
The first task is:
Now we can use the exact same approach to creating features as we used when we created the linear model from the imagenet predictions in the last lesson - it's only the model that has changed. As you're seeing, there's a fairly small number of "recipes" that can get us a long way!
Can you write Python code for it?

batches = get_batches(path+'train', shuffle=False, batch_size=batch_size)
val_batches = get_batches(path+'valid', shuffle=False, batch_size=batch_size)

val_classes = val_batches.classes
trn_classes = batches.classes
val_labels = onehot(val_classes)
trn_labels = onehot(trn_classes)

val_features = conv_model.predict_generator(val_batches, val_batches.nb_sample)

trn_features = conv_model.predict_generator(batches, batches.nb_sample)

save_array(model_path + 'train_convlayer_features.bc', trn_features)
save_array(model_path + 'valid_convlayer_features.bc', val_features)

trn_features = load_array(model_path+'train_convlayer_features.bc')
val_features = load_array(model_path+'valid_convlayer_features.bc')

trn_features.shape


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Above: Fit looks pretty wild, too.  Here's a clear example of how overfitting is associated with very large magnitude estimated coefficients.

#
#
Ridge Regression
Ridge regression aims to avoid overfitting by adding a cost to the RSS term of standard least squares that depends on the 2-norm of the coefficients $\|w\|$.  The result is penalizing fits with large coefficients.  The strength of this penalty, and thus the fit vs. model complexity balance, is controled by a parameter lambda (here called "L2_penalty").
Define our function to solve the ridge objective for a polynomial regression model of any degree:
Can you write Python code for it?

def polynomial_ridge_regression(data, deg, l2_penalty):
    model = graphlab.linear_regression.create(polynomial_features(data,deg),
                                              target='Y', l2_penalty=l2_penalty,
                                              validation_set=None,verbose=False)
    return model


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Add random Gaussian noise to y
Can you write Python code for it?

random.seed(1)
e = graphlab.SArray([random.gauss(0,1.0/3.0) for i in range(n)])
y = y + e


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Compute y
Can you write Python code for it?


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Create a function that prints the polynomial coefficients in a pretty way :)
Can you write Python code for it?

def print_coefficients(model):
    # Get the degree of the polynomial
    deg = len(model.coefficients['value'])-1

    # Get learned parameters as a list
    w = list(model.coefficients['value'])

    # Numpy has a nifty function to print out polynomials in a pretty way
    # (We'll use it, but it needs the parameters in the reverse order)
    print 'Learned polynomial for degree ' + str(deg) + ':'
    w.reverse()
    print numpy.poly1d(w)


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Create a function to plot the data, since we'll do it many times
Can you write Python code for it?

def plot_data(data):
    plt.plot(data['X1'],data['Y'],'k.')
    plt.xlabel('x')
    plt.ylabel('y')

plot_data(data)


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Create random values for x in interval [0,1)
Can you write Python code for it?

random.seed(98103)
n = 30
x = graphlab.SArray([random.random() for i in range(n)]).sort()


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Define a function to fit a polynomial linear regression model of degree "deg" to the data in "data":
Can you write Python code for it?

def polynomial_regression(data, deg):
    model = graphlab.linear_regression.create(polynomial_features(data,deg),
                                              target='Y', l2_penalty=0.,l1_penalty=0.,
                                              validation_set=None,verbose=False)
    return model


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Define some useful polynomial regression functions
Define a function to create our features for a polynomial regression model of any degree:
Can you write Python code for it?

def polynomial_features(data, deg):
    data_copy=data.copy()
    for i in range(1,deg):
        data_copy['X'+str(i+1)]=data_copy['X'+str(i)]*data_copy['X1']
    return data_copy


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Explore the lasso solution as a function of a few different penalty strengths
We refer to lambda in the lasso case below as "l1_penalty"
Can you write Python code for it?

for l1_penalty in [0.0001, 0.01, 0.1, 10]:
    model = polynomial_lasso_regression(data, deg=16, l1_penalty=l1_penalty)
    print 'l1_penalty = %e' % l1_penalty
    print 'number of nonzeros = %d' % (model.coefficients['value']).nnz()
    print_coefficients(model)
    print '\n'
    plt.figure()
    plot_poly_predictions(data,model)
    plt.title('LASSO, lambda = %.2e, # nonzeros = %d' % (l1_penalty, (model.coefficients['value']).nnz()))


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Fit a degree-16 polynomial
Can you write Python code for it?

model = polynomial_regression(data, deg=16)
print_coefficients(model)


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Fit a degree-2 polynomial
Fit our degree-2 polynomial to the data generated above:
Can you write Python code for it?

model = polynomial_regression(data, deg=2)


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Form and plot our predictions along a grid of x values:
Can you write Python code for it?


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Inspect learned parameters
Can you write Python code for it?


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Lasso Regression
Lasso regression jointly shrinks coefficients to avoid overfitting, and implicitly performs feature selection by setting some coefficients exactly to 0 for sufficiently large penalty strength lambda (here called "L1_penalty").  In particular, lasso takes the RSS term of standard least squares and adds a 1-norm cost of the coefficients $\|w\|$.
Define our function to solve the lasso objective for a polynomial regression model of any degree:
Can you write Python code for it?

def polynomial_lasso_regression(data, deg, l1_penalty):
    model = graphlab.linear_regression.create(polynomial_features(data,deg),
                                              target='Y', l2_penalty=0.,
                                              l1_penalty=l1_penalty,
                                              validation_set=None,
                                              solver='fista', verbose=False,
                                              max_iterations=3000, convergence_threshold=1e-10)
    return model


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Let's look at fits for a sequence of increasing lambda values
Can you write Python code for it?

for l2_penalty in [1e-25, 1e-10, 1e-6, 1e-3, 1e2]:
    model = polynomial_ridge_regression(data, deg=16, l2_penalty=l2_penalty)
    print 'lambda = %.2e' % l2_penalty
    print_coefficients(model)
    print '\n'
    plt.figure()
    plot_poly_predictions(data,model)
    plt.title('Ridge, lambda = %.2e' % l2_penalty)


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Perform a ridge fit of a degree-16 polynomial using a very large penalty strength
Can you write Python code for it?

model = polynomial_ridge_regression(data, deg=16, l2_penalty=100)
print_coefficients(model)

plot_poly_predictions(data,model)


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Plot results of estimating LOO for each value of lambda
Can you write Python code for it?

plt.plot(l2_penalty_values,l2_penalty_mse,'k-')
plt.xlabel('$\L2_penalty$')
plt.ylabel('LOO cross validation error')
plt.xscale('log')
plt.yscale('log')


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Put data into an SFrame to manipulate later
Can you write Python code for it?

data = graphlab.SFrame({'X1':x,'Y':y})
data


I am working on the file "Overfitting_Demo_Ridge_Lasso.ipynb".
The first task is:
Run LOO cross validation for "num" values of lambda, on a log scale
Can you write Python code for it?

l2_penalty_values = numpy.logspace(-4, 10, num=10)
l2_penalty_mse,best_l2_penalty = loo(data, 16, l2_penalty_values)


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Above: Fit looks pretty wild, too.  Here's a clear example of how overfitting is associated with very large magnitude estimated coefficients.
Ridge Regression
Ridge regression aims to avoid overfitting by adding a cost to the RSS term of standard least squares that depends on the 2-norm of the coefficients $\|w\|$.  The result is penalizing fits with large coefficients.  The strength of this penalty, and thus the fit vs. model complexity balance, is controled by a parameter lambda (here called "L2_penalty").
Define our function to solve the ridge objective for a polynomial regression model of any degree:
Can you write Python code for it?


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Create a function to plot the data, since we'll do it many times
Can you write Python code for it?


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Define a function to fit a polynomial linear regression model of degree "deg" to the data in "data":
Can you write Python code for it?


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Define function to plot data and predictions made, since we are going to use it many times.
Can you write Python code for it?

def plot_poly_predictions(data, model):
    plot_data(data)

    # Get the degree of the polynomial
    deg = len(model.coefficients['value'])-1

    # Create 200 points in the x axis and compute the predicted value for each point
    x_pred = graphlab.SFrame({'X1':[i/200.0 for i in range(200)]})
    y_pred = model.predict(polynomial_features(x_pred,deg))

    # plot predictions
    plt.plot(x_pred['X1'], y_pred, 'g-', label='degree ' + str(deg) + ' fit')
    plt.legend(loc='upper left')
    plt.axis([0,1,-1.5,2])


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Define some useful polynomial regression functions
Define a function to create our features for a polynomial regression model of any degree:
Can you write Python code for it?


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Explore the lasso solution as a function of a few different penalty strengths
We refer to lambda in the lasso case below as "l1_penalty"
Can you write Python code for it?


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Fit a degree-16 polynomial
Can you write Python code for it?


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Form and plot our predictions along a grid of x values:
Can you write Python code for it?


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Inspect learned parameters
Can you write Python code for it?


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Lasso Regression
Lasso regression jointly shrinks coefficients to avoid overfitting, and implicitly performs feature selection by setting some coefficients exactly to 0 for sufficiently large penalty strength lambda (here called "L1_penalty").  In particular, lasso takes the RSS term of standard least squares and adds a 1-norm cost of the coefficients $\|w\|$.
Define our function to solve the lasso objective for a polynomial regression model of any degree:
Can you write Python code for it?


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Perform a ridge fit of a degree-16 polynomial using a very large penalty strength
Can you write Python code for it?


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Perform a ridge fit of a degree-16 polynomial using a very small penalty strength
Can you write Python code for it?

model = polynomial_ridge_regression(data, deg=16, l2_penalty=1e-25)
print_coefficients(model)

plot_poly_predictions(data,model)


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Put data into an SFrame to manipulate later
Can you write Python code for it?


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Run LOO cross validation for "num" values of lambda, on a log scale
Can you write Python code for it?


I am working on the file "Overfitting_Ridge_Lasso.ipynb".
The first task is:
Woah!!!!  Those coefficients are crazy!  On the order of 10^6.
Can you write Python code for it?


I am working on the file "OverviewNotes_PYTDS_2.ipynb".
The first task is:
Might you also want the median and mode?  Do you remember what those are?
Can you write Python code for it?


I am working on the file "OverviewNotes_PYTDS_2.ipynb".
The first task is:
Now that seems strange.  Isn't the mode of a column of numbers, a number?
We're looking at the numbers that appear most often, the top six in the ranking.  Surely there must be some tie breaking rule.
Can you write Python code for it?


I am working on the file "OverviewNotes_PYTDS_3.ipynb".
The first task is:
Remember to distinguish KNN from K-Means.  You might use the latter to create the clusters whereby you could then fit the former.  Here's a paper on LinkedIn suggesting doing that.  Once you have the clusters (voters), a new data point is "claimed" by one or more clusters.
Hierarchical clustering algorithms compete with K-Means.  The latter does better for spherical or globular clusters.
Can you write Python code for it?

from IPython.display import YouTubeVideo
YouTubeVideo("3vHqmPF4VBA")

from sklearn.metrics import confusion_matrix, accuracy_score
from sklearn.model_selection import train_test_split
from sklearn import neighbors # http://scikit-learn.org/stable/modules/classes.html#module-sklearn.neighbors

# prepare datasets from training and for validation
X_train, X_test, y_train, y_test = train_test_split(digits.data, digits.target,
                                                    test_size=0.4, random_state=0)

# runs the kNN classifier for even number of neighbors from 1 to 10
for n in range(1, 10, 2):
        clf = neighbors.KNeighborsClassifier(n)

        # instance based learning
        clf.fit(X_train, y_train)

        # our 'ground truth'
        y_true = y_test

        # predict
        y_pred = clf.predict(X_test)

        # learning metrics
        cm = confusion_matrix(y_true, y_pred)
        acc = accuracy_score(y_true, y_pred)

        print ("Neighbors: %d" % n)
        print ("Confusion Matrix")
        print (cm)

        print ("Accuracy score: %f" % accuracy_score(y_true, y_pred))
        print ()


I am working on the file "OverviewNotes_PYTDS_3.ipynb".
The first task is:
That's a very poor rendering of the numeral 7 and we're immediately forgiving if our Machine Learning algorithm gets some wrong, with training data of such abysmal quality.  As seen from digits.data, the 64 bits used to represent a digit are hardly enough.  Other datasets come with at least 28 x 28 bits for each numeral.  We're truly at the low end with this skimpy number of bits per digit.
Neverthesless, we press on... I'm making only minor changes to this open source script on Github, by Fabiosato.
Remember how KNN works:
Can you write Python code for it?

from IPython.display import YouTubeVideo
YouTubeVideo("MDniRwXizWo")


I am working on the file "Overview_Scientific_Packages_for_Python.ipynb".
The first task is:
Import all the variables and functions into the enviroment at once
Can you write Python code for it?


I am working on the file "Overview_Scientific_Packages_for_Python.ipynb".
The first task is:
Import some variables or functions into the enviroment
Then we can use variables or functions in the enviroment directly
Can you write Python code for it?

from math import factorial, e

factorial(5)

e


I am working on the file "Overview_Scientific_Packages_for_Python.ipynb".
The first task is:
To reduce the name space, we can specify another name to refer to the original package
Can you write Python code for it?


I am working on the file "Oxford-Pet/Preprocessing train dataset ox.ipynb".
The first task is:
Visualize the size of the processed train dataset
Can you write Python code for it?

mytrain_cat = mytrain[mytrain['SPECIES'] == 1]
mytrain_dog = mytrain[mytrain['SPECIES'] == 2]
myvalid_cat = myvalid[myvalid['SPECIES'] == 1]
myvalid_dog = myvalid[myvalid['SPECIES'] == 2]
x = ['mytrain_cat', 'mytrain_dog', 'myvalid_cat', 'myvalid_dog']
y = [len(mytrain_cat), len(mytrain_dog), len(myvalid_cat), len(myvalid_dog)]

ax = sns.barplot(x=x, y=y)


I am working on the file "P0 Relationships/Data_Analyst_ND_Project0.ipynb".
The first task is:
5. Which chopstick length performed the best for the group of thirty-one male junior college students?
240mm
Can you write Python code for it?

# Causes plots to display within the notebook rather than in a new window
%pylab inline

import matplotlib.pyplot as plt

plt.scatter(x=meansByChopstickLength['Chopstick.Length'], y=meansByChopstickLength['Food.Pinching.Efficiency'])
            # title="")
plt.xlabel("Length in mm")
plt.ylabel("Efficiency in PPPC")
plt.title("Average Food Pinching Efficiency by Chopstick Length")
plt.show()


I am working on the file "P0 Relationships/Data_Analyst_ND_Project0.ipynb".
The first task is:
This number is helpful, but the number doesn't let us know which of the chopstick lengths performed best for the thirty-one male junior college students. Let's break down the data by chopstick length. The next block of code will generate the average "Food Pinching Effeciency" for each chopstick length. Run the block of code below.
Can you write Python code for it?

meansByChopstickLength = dataFrame.groupby('Chopstick.Length')['Food.Pinching.Efficiency'].mean().reset_index()
meansByChopstickLength

# reset_index() changes Chopstick.Length from an index to column. Instead of the index being the length of the chopsticks, the index is the row numbers 0, 1, 2, 3, 4, 5.


I am working on the file "P0-Optimal Chopsticks Length/Data_Analyst_ND_Project0.ipynb".
The first task is:
5. Which chopstick length performed the best for the group of thirty-one male junior college students?
240 mm
Can you write Python code for it?

# Causes plots to display within the notebook rather than in a new window
#pylab inline

import matplotlib.pyplot as plt

plt.scatter(x=meansByChopstickLength['Chopstick.Length'], y=meansByChopstickLength['Food.Pinching.Efficiency'])
            # title="")
plt.xlabel("Length in mm")
plt.ylabel("Efficiency in PPPC")
plt.title("Average Food Pinching Efficiency by Chopstick Length")
plt.show()


I am working on the file "P0-Optimal Chopsticks Length/Data_Analyst_ND_Project0.ipynb".
The first task is:
Let's do a basic statistical calculation on the data using code! Run the block of code below to calculate the average "Food Pinching Efficiency" for all 31 participants and all chopstick lengths.
Can you write Python code for it?


I am working on the file "P0-Optimal Chopsticks Length/Data_Analyst_ND_Project0.ipynb".
The first task is:
This number is helpful, but the number doesn't let us know which of the chopstick lengths performed best for the thirty-one male junior college students. Let's break down the data by chopstick length. The next block of code will generate the average "Food Pinching Effeciency" for each chopstick length. Run the block of code below.
Can you write Python code for it?