I am working on the file "Monte Carlo Simulation of the Ising Model.ipynb".
The first task is:
1. Review of the Ising Model

$N$ particles/molecules/magnetic dipoles with $\sigma = \pm 1$

"Nearest neighbor" interaction Hamiltonian

$$ E=-J\sum_{ [i,j] }\sigma_i\sigma_j-H\sum_i\sigma_i \quad (1)$$
where $[i, j]$ denotes summation over nearest-neighbor pairs -- topology dependent!
1D Ising Model
No phase transition above 0K (Ising)
2D Ising Model (aka square-lattice Ising model) at $H=0$
Analytic solution by Onsager:

Transition temperature: $$ T_C = \frac{1}{k_B}\frac{2J}{\ln(1+\sqrt{2})} \approx 2.269\frac{J}{k_B} \quad (2)$$

Spontaneous magnization: $$ |\bar{\sigma}| = \Big{
    \begin{array}
    ((1 - \sinh^{-4}(\frac{2J}{k_B T}))^{1/8} & (T < T_C) \
    0 & (T \geq T_C)
    \end{array} \quad (3)
    $$

Due to its simplicity and interesting analytic behavior, the 2D Ising Model is frequently used as a "testing stone" for Molecular Dynamics/Monte Carlo algorithms

We will only consider the case of $H=0$ in this notebook

2. Monte Carlo Simulation
2.1 Importance sampling

The total number of microstates, $\Omega$, approaches astronomical magnitudes even for a small system with $N \sim 10^4$

However, when $N$ is sufficiently large most of the microstates have extremely low probability compared to those in the vincinity of the most probable states.

Metropolis (pioneer of the MC methods):

...instead of choosing configurations randomly, then weighting them with $\exp(-E/k_BT)$, we choose configurations with a probability $\exp(-E/k_BT)$ and weight them evenly.[1]

This is called importance sampling.

How samples are gathered:

Molecular dynamics simulation (MD): start from first principles and simulate the time evolution of the system. For thermodynamic systems at equilibrium, points along any phase trajectories should be a good representation of the ensemble (ergodicity)

Monte Carlo simulation (MC): Carry out a hypothetical "random walk" on the phase space, in effect creating a Markov Chain. Since the term Monte Carlo is used ubiquitously to represent anything that involve some...
Can you write Python code for it?

def dice_samples(trials):
    prob = {1: 1/2, 2: 1/4, 3: 1/8, 4: 1/16, 5: 1/32, 6: 1/32}
    samples = np.zeros(trials + 1, dtype=int)
    samples[0] = 1
    for i in range(trials):
        a = samples[i]
        b = np.random.random_integers(1, 6) # uniform a priori distribution
        pa = prob[a]
        pb = prob[b]
        if pb >= pa or np.random.rand() < pb / pa:
            samples[i + 1] = b
        else:
            samples[i + 1] = a
    return samples

def summarize(samples):
    '''
    Return the percentage of every face in the samples
    '''
    num_samples = len(samples)
    distribution = {i: (samples == i).sum() * 100 / num_samples for i in [1, 2, 3, 4, 5 ,6]} # 百分数
    return distribution


I am working on the file "Monte Carlo Simulation of the Ising Model.ipynb".
The first task is:
3.4 Comparison to theoretical results
We compare results from cluster MC to the ones given by Onsager in (3):
Can you write Python code for it?

def averages(magnetization, energy, T):
    '''计算各量的系综平均'''
    M = np.abs(magnetization).mean()
    E = energy.mean()
    CV = energy.var() / T
    return M, E, CV

# The code below runs VERY SLOWLY
points = 31
dimension = 64
iterations = 5000

neighbors_list = gen_neighbors_2d(dimension)
Ts = np.linspace(1.0, 4.0, points)
Ms = np.zeros(points)
for i in range(points):
    T = Ts[i]
    magnetization, _ = cluster_MC(neighbors_list, T, iterations)
    Ms[i] = np.abs(magnetization).mean()
#     print("Iteration for T = {:.3f} complete".format(T)) # Uncomment to print progress as the simulation goes

Onsager_Tc = 2.269
Ts_plot = np.linspace(1.0, 4.0, num=200)
Onsager_Ms = np.zeros(len(Ts_plot))
for i, T in enumerate(Ts_plot):
    if T <= 2.269:
        Onsager_Ms[i] = (1 - (np.sinh(2/T))**(-4))**(1/8)
plt.plot(Ts, Ms, '^', label='simulation')
plt.plot(Ts_plot, Onsager_Ms, '--', label='theoretical')
plt.ylim(0, 1)
plt.legend()
plt.xlabel('Temperature')
plt.ylabel('Magnetization')
plt.grid()
plt.title("Theoretical vs. Numerical values for spontaneous magnitization")


I am working on the file "Monte Carlo Simulation of the Ising Model.ipynb".
The first task is:
4. Development of MC methods

Different choice of a priori distributions;

Hybrid MC/MD simulations;

Better sample generation: correlation, buring and thinning;

MCMC applied to Hierarchichal Bayesian Models in the field of statistics. An excellent material is Probabilistic Programming and Bayesian Methods for Hackers

References
[1] N.Metropolis, A.W.Rosenbluth, M.N.Rosenbluth, A.H.Teller, E.Teller, Equation of State Calculations by Fast Computing Machines, The Journal of Chemial Physics 21(1953), pp.1087
[2] K.Werner, Statistical Mechanics: Algorithms and Computations, Oxford(2006)
Can you write Python code for it?

import sys
print("Python version = ", sys.version)
print("Numpy version = ", np.version.version)
print("Matplotlib version = ", matplotlib.__version__)


I am working on the file "Monte Carlo Simulation of the Ising Model.ipynb".
The first task is:
Let $J=1$，$k_B=1$ in the following discussion:

State of the markov chain = microstate of the Ising Model $\vec{\sigma} := (\sigma_1\ldots\sigma_N)^T$

A priori probability: choose uniformly from any state that relates to the current state by a single spin flip

Energy difference resulting from a single flip: $\delta E= 2\sigma_i\sum_{[i,j]}\sigma_j $

Acceptance probability $ P=\min(1, \exp(-\delta E/T)) $

The "single flip" mechanism makes the previously defined adjacency list particularly useful.
Can you write Python code for it?

def MH_single_flip(neighbors_list, T, iterations):
    '''
    This function performs single flip MC iterations for an Ising system with arbitrary topology,
    given by the adjaceny list `neighbors_list`.
    The inital state is chosen randomly.

    Returns
    =======
    `magnetization`: magnetization (average molecular spin) at each MC step
    `energy`: total energy of the system at each MC step
    '''
    # Initialization
    size = neighbors_list.shape[0]
    spins = np.random.random_integers(0, 1, size)
    spins[spins == 0] = -1
    # Allocation
    magnetization = np.zeros(iterations + 1)
    energy = np.zeros(iterations + 1)
    magnetization[0] = spins.sum()
    energy[0] = -spins.dot(spins[neighbors_list].sum(axis=1)) / 2

    for step in range(iterations):
        n = np.random.randint(0, size) # Choose next state according to the a priori distribution
        delta_E = 2 * spins[n] * spins[neighbors_list[n]].sum()
        if delta_E < 0 or np.random.rand() < np.exp(-delta_E / T):
            # Acceptance
            spins[n] = -spins[n]
            magnetization[step + 1] = magnetization[step] + 2 * spins[n]
            energy[step + 1] = energy[step] + delta_E
        else:
            # Rejection
            magnetization[step + 1] = magnetization[step]
            energy[step + 1] = energy[step]
    return magnetization / size, energy


I am working on the file "Monte Carlo Simulation of the Ising Model.ipynb".
The first task is:
Remarks:

No spontaneous Magnetization is seen in the 1D case, in agreement with the result from Ising.

The 2D case displays spontaneous behavior roughly for $T$ smaller than the theoretical critical temperature $T_C$. However, instead of a two-peak distribution as prescribed by the canocical distribution, only one peak is observed.

Convergence of the markow chain is very slow for $T \approx T_C$. We can prove theoretically that the number of steps required actually diverges as $T \rightarrow T_C$. This is called the critical slowing down behavior.

Interesting results! But why?
3.2 Theoretical interlude: the clustering of spins near $T_C$
Recall the definition of the spin correlation function:
$$ g(x, y) := E[\sigma(m + x, n + y)\sigma(m, n)] - E[\sigma(m + x, m + y)]E[\sigma(m, n)] \quad (7)$$
where the expectation is taken for all grid points $(m, n)$. The above formula can be simplified if we assume periodic boundary consitions for all sides:
$$ g(x, y) = E[\sigma(m + x, n + y)\sigma(m, n)] - M^2 \quad (8) $$
where $M := E[\sigma(n)]$. The spin correlation function can tell us more about what happens when $T \approx T_C$.
Can you write Python code for it?

def spin_correlation(spins, a):
    M = spins.mean()
    spins_2d = spins.reshape((a, a))
    rs = np.arange(-a/2, a/2 + 1, dtype=int)
    num_rs = len(rs)
    correlations = np.zeros((num_rs, num_rs))
    for i, y in enumerate(rs):
        for j, x in enumerate(rs):
            correlations[i, j] = (spins_2d * np.roll(np.roll(spins_2d, x, axis=1), y, axis=0)).mean() + M**2
    return correlations


I am working on the file "Monte Carlo Simulation of the Ising Model.ipynb".
The first task is:
So the constructed chains do converto our desired value. But notice the length of transient steps: it took as much as 10000 steps for us to obtain a reasonably accurate value! As we will see below, this "high accuracy, low precision" characteristic of MC methods (a slow convergence to an exact value) is ubiquitous. It remains a problem til this day to find a MC method that have fast convergence.
3. MC Simulation of the Ising Model
3.1 The "single-flip" MC
A uniform representation for both the 1D and 2D Ising Model
The modern language of nearest-neighbor interation topology is graph theory. We use the most popular graph storage format --- the adjacency list format --- to record the structure of 1D (chain) and 2D (square lattice) Ising Models:
Can you write Python code for it?

# First, we need helper function to transform between the (i, j) coordinate of a 2D lattice and a serial one
# a: length of the square lattice's side
def flatten_2d(i, j, a):
    return i * a + j # serial No. = row No. * lenght + colum No.
def unflatten_2d(n, a):
    j = n % a
    i = (n - j) // a
    return i, j

# Generate the adjacency list
def gen_neighbors_1d(N):
    neighbors = np.zeros((N, 2), dtype=int)
    for n in range(N):
        neighbors[n][0] = (n - 1) % N # left
        neighbors[n][1] = (n + 1) % N # right
    return neighbors

def gen_neighbors_2d(a):
    neighbors = np.zeros((a*a, 4), dtype=int)
    for n in range(a*a):
        i, j = unflatten_2d(n, a)
        neighbors[n][0] = flatten_2d(i, (j - 1) % a, a) # left
        neighbors[n][1] = flatten_2d(i, (j + 1) % a, a) # right
        neighbors[n][2] = flatten_2d((i - 1) % a, j, a) # up
        neighbors[n][3] = flatten_2d((i + 1) % a, j, a) # down
    return neighbors


I am working on the file "Monte Carlo Simulation of the Ising Model.ipynb".
The first task is:
So the markov chain, instead of converging, actually goes into an ordered stated with considerable correlation between the spins. The spin clusters formed in the process makes it more difficult for future states to be accepted, as a randomly picked molecule is more like to be inside a cluster than at the boundary between two clusters. Flipping this molecule would thus increase the system's energy, so it's more likely to be rejected. This is the explanation for the above mentioned critical slowing down behavior.
The same argument can be applied to the spontaneous magnitization state for $T \ll T_C$. In this case, once the system enter the ordered state the giant cluster makes further flipping very difficult to accept. Because we have arbitrarily constrained the system to move only a small step in configuration space per MC step, it would take forever for it to move from one steady state to another.
3.3 Cluster MC
To address the problems mentioned above, the cluster Monte Carlo method is proposed to allow for large configuration space hopping between two MC steps.
Specifically:

A "seed" is chosen randomly at the start of each MC step.

A breadth-first search is performed in the lattice starting from the "seed". Whenever a molecule with the same spin as the seed is encountered, it is added to the breadth-first queue with probability p. The search is carried on until the queue is exhausted, generating a cluster of same-spin molecules from the breadth-first tree.

A new state is proposed by flipping all molecules within the cluster. The acceptance/rejection procedure is identical to that given by Metropolis-Hastings.

To carry out cluster MC, we need to calculated the effective a priori distribution from the procedures described above. We state the result without proving (refer to Ref [2] for a complete discussion):
If we set $p$ to the "magic number" $1-e^{2/T}$, then $p(a \rightarrow b) \equiv 1$, which means the next state will always be accepted. We can already see that under this choice MC iterations can...
Can you write Python code for it?

def cluster_MC(neighbors_list, T, iterations):
    p = 1 - np.exp(-2 / T) # "magic number"
    # Initialization
    size = neighbors_list.shape[0]
    spins = np.random.random_integers(0, 1, size)
    spins[spins == 0] = -1
    # Allocation
    magnetization = np.zeros(iterations + 1)
    magnetization[0] = spins.sum()
    energy = np.zeros(iterations + 1)
    energy[0] = -spins.dot(spins[neighbors_list].sum(axis=1))

    for step in range(iterations):
        # Use a deque to implement breadth-first search
        n0 = np.random.randint(0, size)
        sign = spins[n0]
        cluster = set([n0])
        pockets = deque([n0])
        finished = False
        while not finished:
            try:
                n = pockets.popleft()
                neighbors = neighbors_list[n]
                for neighbor in neighbors:
                    if spins[neighbor] == sign and neighbor not in cluster and np.random.rand() < p:
                        cluster.add(neighbor)
                        pockets.append(neighbor)
            except IndexError:
                finished = True
        # Flip the cluster
        cluster = np.fromiter(cluster, dtype=int)
        spins[cluster] = -sign
        magnetization[step + 1] = magnetization[step] - 2 * sign * len(cluster)
        energy[step + 1] = -spins.dot(spins[neighbors_list].sum(axis=1))
    return magnetization / size, energy / 2 # Every pair is counted two times


I am working on the file "Monte Carlo Simulation of the Ising Model.ipynb".
The first task is:
This is what happens when $T = 2.2$:
Can you write Python code for it?

a = 40
T = 2.2
time_stamps = np.array([1, 10, 100, 1000, 10000, 20000, 40000, 60000, 80000, 100000])
snapshots = gen_snapshots(time_stamps, a, T)
fig, axes = plt.subplots(10, 2, figsize=(12, 6 * 10))
for i, t in enumerate(time_stamps):
    axes[i][0].matshow(snapshots[t]['spins'], interpolation='none')
    axes[i][0].set_ylabel('MC step {}, M = {:.3f}'.format(t, snapshots[t]['magnetization']))
    axes[i][0].set_xlabel('Spins')
    axes[i][1].matshow(snapshots[t]['correlation'])
    axes[i][1].set_xlabel('Spin correlation function')
    axes[i][1].set_xticks([0, 10, 20, 30, 40])
    axes[i][1].set_xticklabels([-20, -10, 0, 10, 20])
    axes[i][1].set_yticks([0, 10, 20, 30, 40])
    axes[i][1].set_yticklabels([-20, -10, 0, 10, 20])


I am working on the file "Monte Carlo Simulation of the Ising Model.ipynb".
The first task is:
We modify the single-flip MC code a bit to have it output snapshots of the system at the given time stamps:
Can you write Python code for it?

def gen_snapshots(time_stamps, a, T):
    iterations = time_stamps[-1]
    snapshots = {}

    neighbors_list = gen_neighbors_2d(a)
    size = neighbors_list.shape[0]
    spins = np.random.random_integers(0, 1, size)
    spins[spins == 0] = -1
    for step in range(iterations):
        n = np.random.randint(0, size)
        delta_E = 2 * spins[n] * spins[neighbors_list[n]].sum()
        if delta_E < 0 or np.random.rand() < np.exp(-delta_E / T):
            spins[n] = -spins[n]
        if step + 1 in time_stamps:
            snapshots[step + 1] = {
                'spins': spins.copy().reshape((a, a)),
                'magnetization': spins.mean(),
                'correlation': spin_correlation(spins, a)
            }
    return snapshots


I am working on the file "Monte Carlos Experiments/Monte Carlo Demonstration.ipynb".
The first task is:
As we can see the approximation converges, if we take the mean, we can approximate the value of $\pi$.
Now, lets see if we can do the same using the circumference, it's length is $2r\pi$, we will try to use the same method, using the formula $\pi = \dfrac{P_{circ}A_{rectangle}}{2r}$.
Can you write Python code for it?

#Let's define a function for approximating pi with a given number of points using the circumference
def monte_carlo_pi_circ(points, return_points=False):
    inside = 0
    point_list = []
    for i in range(points):
        x,y = (random.uniform(-1.5,1.5), random.uniform(-1,1))
        if (x**2+y**2)**(1/2) == 1:
            inside += 1
        if return_points:
            point_list.append((x,y))
    #The area of our rectangle is 6, since it goes from -1.5 to 1.5 and -1 to 1
    #But we need to divide by 2 times the radius which is 1
    pi = (inside/points) * 3
    if not return_points:
        return pi
    return (pi, point_list)

pi, points_list = monte_carlo_pi_circ(1000,return_points=True)
plot_circle(points_list, color_func=lambda x,y:(1,0,0) if (x**2+y**2)**(1/2) ==1 else (0,0,1,0.5))


I am working on the file "Monte Carlos Experiments/Monte Carlo Integration.ipynb".
The first task is:
In this notebook we will use the Monte Carlo method to find the area under a curve, so first let's define a function
$$f(x) = x^2-4x+5$$
Can you write Python code for it?

f = lambda x:x**2-4*x+5
x = range(0, 11, 1)
y = [f(v) for v in x]
plt.plot(y)


I am working on the file "Monte Carlos Experiments/Monte Carlo Integration.ipynb".
The first task is:
Knowing the ratio of points under the curve, we can now calculate the integral as
$$P_{curve}A_{rectangle} = A_{cruve}$$
If we take the integral
$$\int_0^{10}x^2-4x+5$$
We have $$\dfrac{x^3}{3}-2x^2+5x\big\lvert_0^{10} = \dfrac{10^3}{3}-200+50 = 333.33 - 200 + 50 = 183.33$$
Which is close to the real area, now, let's see how many points we need
Can you write Python code for it?

def monte_carlo_integration(f, number_points, xlims, ylims):
    below=0
    for p in range(0, number_points):
        x,y=(random.uniform(*xlims), random.uniform(*ylims))
        # If the function for x is greater then the random y, then the point is under the curve
        if y <= f(x):
            below +=1
    ratio = below/number_points
    area = ratio * (xlims[1]-xlims[0]) * (ylims[1]-ylims[0])
    return (ratio, area)

total_points = 10000
step = 100
estimated = [monte_carlo_integration(f, i, (0,10), (0,70))[1]
for i in range(step,total_points, step)]
mean = sum(estimated)/len(estimated)
print("Mean Approximated value %.4f" % mean)

plt.figure(figsize=(8,5))
plt.plot(estimated)
plt.hlines(183.3, 0, total_points/step, 'g')
plt.hlines(mean, 0 , total_points/step, 'r')
plt.legend(['Approximation', 'Real', 'Mean'], loc='best')
plt.ylabel("Estimated area")
plt.xlabel("Points used (x100)")
print("Approximated Integral Value: %.4f" % mean)


I am working on the file "Monte Carlos Experiments/Monte Carlo Integration.ipynb".
The first task is:
Now, we know the probability of a random point being below the curve is equal to
 $$P_{curve}=\dfrac{A_{curve}}{A_{rectanle}}$$
Where $A_{rectangle}$ is the area of the plot in the given interval, so let's try to integrate it from 0 to 10
Can you write Python code for it?

#Will use 3000 points
number_points=3000
#We want to see the points
points=[]
below=0
for p in range(0, number_points):
    x,y=(random.uniform(0,10), random.uniform(0, 70))
    # If the function for x is greater then the random y, then the point is under the curve
    if f(x) >= y:
        below +=1
    points.append((x,y))
ratio = below/number_points

color_func = lambda x,y: (1,0,0) if f(x)>=y else (0,0,1,0.5)
colors = [color_func(x,y) for (x,y) in points]
plt.ylim(0,70)
plt.xlim(0,10)
plt.scatter(*zip(*points), color=colors)
plt.show()
print("Ratio of points under the curve: %.4f" % ratio)
print("Approximated area under the curve: %.4f" % (ratio*700))


I am working on the file "MonteCarlo.ipynb".
The first task is:
Let's evaluate this function for different values of n (number of samples) to see how long it takes to converge.
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt

nvec = np.logspace(1, 6, 20)
muvec = np.zeros(20)
stdvec = np.zeros(20)

for i, n in enumerate(nvec):
    muvec[i], stdvec[i], _, _ = stats(int(n))
    print i

plt.figure()
plt.semilogx(nvec, muvec, '-o')

plt.figure()
plt.semilogx(nvec, stdvec, '-o')
plt.show()


I am working on the file "MonteCarlo.ipynb".
The first task is:
Monte Carlo with LHS
In our discussion on surrogate-based optimization we learned about Latin Hypercube Sampling (LHS).  Let's apply LHS to Monte Carlo to see if it speeds up convergence.
Can you write Python code for it?

from pyDOE import lhs
from scipy.stats.distributions import norm

def statsLHS(n):
    f = np.zeros(n)
    c = np.zeros(n)

    # generate latin hypercube sample points beforehand from normal dist
    lhd = lhs(2, samples=n)
    rpt = norm(loc=0, scale=1).ppf(lhd)

    for i in range(n):
        x1 = x[0]
        x2 = x[1] + rpt[i, 0]*sigma[1]
        x3 = x[2] + rpt[i, 1]*sigma[2]

        f[i] = func([x1, x2, x3])
        c[i] = con([x1, x2, x3])

    # mean
    mu = np.average(f)

    # standard deviation
    std = np.std(f, ddof=1)  #ddof=1 gives an unbiased estimate (np.sqrt(1.0/(n-1)*(np.sum(f**2) - n*mu**2)))

    return mu, std, f, c

muLHS = np.zeros(20)
stdLHS = np.zeros(20)

for i, n in enumerate(nvec):
    muLHS[i], stdLHS[i], _, _ = statsLHS(int(n))
    print i

plt.figure()
plt.semilogx(nvec, muvec, '-o')
plt.semilogx(nvec, muLHS, '-o')

plt.figure()
plt.semilogx(nvec, stdvec, '-o')
plt.semilogx(nvec, stdLHS, '-o')
plt.show()


I am working on the file "MonteCarlo.ipynb".
The first task is:
Notice that it skews to the right.  Because of the square terms in the function, any deviation causes the function to increase.
We can also estimate our reliability just by counting up our failures (or counting up the number of times the constraint was satisfied).
Can you write Python code for it?

reliability = np.count_nonzero(c <= 0.0)/float(n)
print 'reliability = ', reliability*100, '%'


I am working on the file "MonteCarlo.ipynb".
The first task is:
We will use randn, which gives us a random number k sampled from a normal distribution.  It is sampled from a unit normal with zero mean and a standard deviation of 1 so to translate to an arbitrary mean and standard deviation the random value will be
$$ x = \mu + k \sigma $$
If we sample enough times, we should get reliable statistics
Can you write Python code for it?

import numpy as np

def stats(n):
    f = np.zeros(n)
    c = np.zeros(n)

    for i in range(n):
        x1 = x[0]
        x2 = x[1] + np.random.randn(1)*sigma[1]
        x3 = x[2] + np.random.randn(1)*sigma[2]

        f[i] = func([x1, x2, x3])
        c[i] = con([x1, x2, x3])

    # mean
    mu = np.average(f)

    # standard deviation
    std = np.std(f, ddof=1)  #ddof=1 gives an unbiased estimate (np.sqrt(1.0/(n-1)*(np.sum(f**2) - n*mu**2)))

    return mu, std, f, c


I am working on the file "MonteCarloSLC.ipynb".
The first task is:
Step 2

Transform offshore wave conditions to onshore wave conditions (SWAN)

Step 3

Erode shoreline based on wave conditions at shoreline and duration of storm

Can you write Python code for it?

def Erosion(hs, tps, swel, duration, shoreline, sea_level):
    erosion = (hs+tps+swel)*duration*abs(sea_level) #"some function using hs, tps, swel, and duration"
    return erosion


I am working on the file "MonteCarloSLC.ipynb".
The first task is:
Step 4

Allow shoreline to accreate based on sea-level and period of calm

based on equilibrium shoreline theory

Can you write Python code for it?

def Recovery(int_storm_dur, shoreline, sea_level):
    recovery = int_storm_dur*np.e**abs(sea_level)
    return recovery


I am working on the file "MonteCarloSLC.ipynb".
The first task is:
Step 5

Repeat steps 3 and 4 for entire climate record

Can you write Python code for it?

starting_position = 1
shoreline = [starting_position]
shoretime = [0]
i = 0
for storm in STORMS:
    # Erode the shoreline
    erosion = Erosion(STORMS[storm]['HS'],
                      STORMS[storm]['TPS'],
                      STORMS[storm]['SWEL'],
                      STORMS[storm]['DURATION'],
                      shoreline[i],
                      sea_level)
    eroded_shoreline = shoreline[i]-erosion
    shoreline.append(eroded_shoreline)
    shoretime.append(shoretime[i]+STORMS[storm]['DURATION'])
    i += 1
    # Allow shoreline to recover
    recovery = Recovery(STORMS[storm]['RECOVER'],
                        shoreline[i],
                        sea_level)
    recovered_shoreline = shoreline[i]+recovery
    shoreline.append(recovered_shoreline)
    shoretime.append(shoretime[i]+STORMS[storm]['RECOVER'])
    i += 1


I am working on the file "MonteCarloSLC.ipynb".
The first task is:
Step 6

Run 1000s of simulations for different sythetic climate records

Step 7

Evaluate storm ensamble to get final probability distributions

Can you write Python code for it?

%matplotlib inline
from matplotlib.pyplot import plot
plot(shoretime, shoreline)

plot(shoretime, shoreline)


I am working on the file "Monthly homicides and shootings report.ipynb".
The first task is:
And days without a homicide.
Can you write Python code for it?

from datetime import date, timedelta

test_date = date.today()
one_day = timedelta(days=1)

homicide_days = {}
for homicide in homicides:
    if homicide['Occ Date'] not in homicide_days:
        homicide_days[homicide['Occ Date']] = 0
    homicide_days[homicide['Occ Date']] += 1

while test_date.year > 2013:
    if test_date.strftime('%Y-%m-%d') not in homicide_days:
        print 'No homicides on %s' % test_date
    test_date -= one_day


I am working on the file "Monthly homicides and shootings report.ipynb".
The first task is:
Now let's find the days without a shooting.
Can you write Python code for it?

from datetime import date, timedelta

test_date = date.today()
one_day = timedelta(days=1)

shooting_days = {}
for shooting in shootings:
    if shooting['Date'] not in shooting_days:
        shooting_days[shooting['Date']] = 0
    shooting_days[shooting['Date']] += 1

while test_date.year > 2013:
    if test_date.strftime('%Y-%m-%d') not in shooting_days:
        print 'No shootings on %s' % test_date
    test_date -= one_day


I am working on the file "Monty-Hall.ipynb".
The first task is:
A.index('c') returneaza  indicele primului element  din lista ramasa, care este egal cu 'c' si astfel prezentatorul
alege o usa in spatele careia este o capra: Prez=A.pop(A.index('c')).
Dupa executia acestei  linii in lista A
ramane doar un singur element.
Can you write Python code for it?

def Monty_Hall(A, schimba=True):
    #A este tripletul ce da amplasarea obiectelor in spatele usilor
    Conc, Prez=alegeUsa(A)# Concurentul si prezentatorul aleg cate o usa  conform regulii jocului
    if (schimba):
        alegerea=A[0]#in lista mai exista un singur element si aceasta este alegerea noua
    else:
        alegerea=Conc
    return alegerea

def simulare_MH(Nr, optiune):#repeta experimentul MH de Nr ori cu  optiunea schimba (True sau False)

    nrcastig=0
    for n in range(Nr):
        A=rnd.choice([['m', 'c', 'c'],['c', 'm', 'c'],['c', 'c', 'm']])# alege la intamplare o amplasare
                                                                       # a premiilor din cele 3 posibile
        rez=Monty_Hall(A, schimba=optiune)
        if rez=='m':
            nrcastig+=1
    return nrcastig


I am working on the file "Monty-Hall.ipynb".
The first task is:
De exemplu, daca amplasarea premiilor este  A=['c', 'm', 'c'] atunci rnd.randint(0,2) genereaza aleator un indice $i$ al
elementelor listei $A$.
A.pop(i) returneaza
elementul A[i] si il sterge din lista (vezi
aici metode pentru liste).
Ilustrarea prin rularea liniilor de cod aferente:
Can you write Python code for it?

A=['c', 'm', 'c']
Conc=A.pop(rnd.randint(0, 2))
print Conc
print A


I am working on the file "Monty-Hall.ipynb".
The first task is:
Inainte de a experimenta  jocul  sintetizam principiul de implementare:

Jocul se repeta de Nr ori, fie cu optiunea alege o noua usa dupa ce prezentatorul a deschis usa cu capra, fie nu alege.
La inceputul jocului se alege la intamplare o varianta de amplasare a masinii (din cele 3 posibile)
functia alegeUsa face alegerea aleatoare a usii pentru concurent si pentru prezentator.
Can you write Python code for it?

opt_schimb=True# optiunea schimba sau nu
Nr=1000
nrc=simulare_MH(Nr, opt_schimb)
print "Probabilitatea sa castige daca  schimba usa aleasa=", nrc/Nr

nrc=simulare_MH(Nr, False)
print "Probabilitatea sa castige daca nu  schimba usa aleasa=", nrc/Nr


I am working on the file "Monty-Hall.ipynb".
The first task is:
Solutia  pe care ar da-o un necunoscator al teoriei probabilitatilor:
Codificam masina cu "m" si capra cu "c". Avem $3$  posibilitati de plasare a masinii si celor doua capre
in trei incaperi:
$\Omega={(m, c, c),   (c, m, c), (c, c, m)}$
Daca concurentul alege o usa la intamplare el are sansa de a castiga de $1/3$.
Dupa ce i s-a aratat o usa in care este o capra, suntem tentati sa spunem ca sansa de a castiga, alegand oricare din usile ramase
este:
$P(\mbox{castig}|U_d=c)=1/2$, unde  $(U_d=c)$  este evenimentul ca in spatele usii deschise de prezentator sa fie o capra.
Cu alte cuvinte, rationamentul comun conduce la ideea ca  n-are importanta daca concurentul pastreaza usa aleasa sau schimba usa.  Dar este incorect!!!!!
Solutia folosind  formula lui Bayes
Notam cu  $H_k$, $k=1,2,3$,  ipotezele privind  amplasarea premiilor in spatele celor trei usi, si anume
 $(m,c,c)$, $(c,m,c)$, $(c,c,m)$.
Probabilitatile apriori ale acestor ipoteze sunt
$P(H_k)=1/3$, $k=1,2,3$.
Presupunem pentru simplitatea prezentarii ca concurentul alege usa 1, iar prezentatorul deschide usa 2, in spatele careia stie sigur ca este o capra.
Calculul probabilitatii de a castiga daca concurentul nu schimba usa aleasa
Conform formulei lui Bayes, probabilitatea ca masina sa fie plasata in spatele usii alese
de concurent, rectificata de informatia ca in spatele usii $2$ e o capra este:
$P(H_1|U_2)=\displaystyle\frac{P(H_1)P(U_2|H_1)}{P(H_1)P(U_2|H_1)+P(H_2)P(U_2|H_2)+P(H_3)P(U_2|H_3)}$
Deoarece $P(U_2|H_2)=0$ (adica stiind ca masina este in spatele usii 2, probabilitatea ca prezentatorul  sa deschida aceasta usa este 0), avem:
$P(H_1|U_2)=\displaystyle\frac{P(H_1)P(U_2|H_1)}{P(H_1)P(U_2|H_1)+P(H_3)P(U_2|H_3)}$
$P(U_2|H_1)=1/2$, deoarece stiind ca masina este in spatele usii 1 (avem conditionare de evenimentul $H_1)$) prezentatorul alege uniform intre cele doua usi $2$ si $3$.
Daca deschiderea unei usi este conditionata de $H_3$, adica stie ca in spatele usii 3 este masina, iar concurentul a ales usa 1, nu-i ramane decat sa deschida usa 2,...
Can you write Python code for it?

import random as rnd
from __future__ import division  # importand division m/n este evaluat ca float(m)/n

def alegeUsa(A): #alege dintr-o lista de 3 elemente din care doua sunt setate pe 'c' si una pe'm'
    Conc=A.pop(rnd.randint(0, 2))#alegerea concurentului
    Prez= A.pop(A.index('c'))#alegerea prezentatorului emisiunii
    return Conc, Prez


I am working on the file "Moon Phase Correlation Analysis.ipynb".
The first task is:
Extract the startup time metrics with their submission date and make sure we only consider one submission per user:
Can you write Python code for it?

subset = get_pings_properties(pings, ["clientId", "meta/submissionDate", "payload/simpleMeasurements/firstPaint"])
subset = get_one_ping_per_client(subset)
cached = subset.cache()


I am working on the file "Moon Phase Correlation Analysis.ipynb".
The first task is:
Let's see what this data looks like:
Can you write Python code for it?

plt.figure(figsize=(15, 7))
plt.scatter(pairs.T[0], pairs.T[1])
plt.xlabel("Moon visibility ratio")
plt.ylabel("Startup time (ms)")
plt.show()


I am working on the file "Moon Phase Correlation Analysis.ipynb".
The first task is:
Obtain an array of pairs, each containing the moon visibility and the startup time:
Can you write Python code for it?

pairs = cached.map(lambda p: (approximate_moon_visibility(date_string_to_date(p["meta/submissionDate"])), p["payload/simpleMeasurements/firstPaint"]))
pairs = np.asarray(pairs.filter(lambda p: p[1] != None and p[1] < 100000000).collect())


I am working on the file "Moon Phase Correlation Analysis.ipynb".
The first task is:
The correlation coefficient is now easy to calculate:
Can you write Python code for it?


I am working on the file "More/DefaultParametersInPython_ES.ipynb".
The first task is:
El código que define a función es evaluado una vez y dicho valor evaluado es el que se usa en cada llamado posterior. Por lo tanto, al modificar el valor de un parámetro por defecto que es mutable (list, dict, etc.) se modifica el valor por defecto para el siguiente llamado.
¿Cómo evitar esto?
Una solución simple es usar None como el valor predeterminado para los parámetros por defecto. Y otra solución es la declaración de variables condicionales:
Can you write Python code for it?

class Clase:

    def __init__(self, lista=None):
        # Version "one-liner":
        self.lista = lista if lista is not None else list()

        # En su version extendida:
        if lista is not None:
            self.lista = lista
        else:
            self.lista = list()


I am working on the file "More/DefaultParametersInPython_ES.ipynb".
The first task is:
Investigando nuestro código
Veamos un poco qué está pasando en nuestro código:
Can you write Python code for it?

# Instanciemos algunos objetos
A = Clase()
B = Clase()
C = Clase(lista=["GG"]) # Usaremos esta isntancia como control

print("\nLos objetos son distintos!")
print("id(A): {} \nid(B): {} \nid(C): {}".format(id(A), id(B), id(C)))

print("\nPero la lista es la misma para A y para B :O")
print("id(A.lista): {} \nid(B.lista): {} \nid(C.lista): {}".format(id(A.lista), id(B.lista), id(C.lista)))


I am working on the file "More/DefaultParametersInPython_ES.ipynb".
The first task is:
Si llamamos a la función una vez...
Can you write Python code for it?


I am working on the file "More/FacebookGraphAPI_ES.ipynb".
The first task is:
Ahora, publicaremos un estado. Esta request nos retornará la id del post, que será publicado con visibilidad "Solo para mi" (Only me)

POST /me/feed
Can you write Python code for it?

url = "{}/{}/me/feed".format(BASE, VERSION)
params = {
    "access_token": access_token,
    "message": "Este estado lo publiqué usando la API de Facebook :O"
}
req = requests.post(url, params=params)
status_id = req.json()["id"]
print("status_id = {}".format(status_id))


I am working on the file "More/FacebookGraphAPI_ES.ipynb".
The first task is:
Luego, podemos directamente borrar un estado solo si lo publicamos usando la API:

DELETE /{status-id}
Can you write Python code for it?

url = "{}/{}/{}".format(BASE, VERSION, status_id)
params = {
    "access_token": access_token
}
req = requests.delete(url, params = params)
print_pretty(req.json())


I am working on the file "More/FacebookGraphAPI_ES.ipynb".
The first task is:
Partiremos con lo más simple. Una consulta GET para obtener información sobre nosotros mismos.

GET /me

El token de acceso se envía como parámetro, junto con los campos que queremos obtener de la consulta:
Can you write Python code for it?

url = "{}/{}/me".format(BASE, VERSION)
params = {
    "access_token": access_token,
    "fields": ["id", "name"]
}
req = requests.get(url, params=params)
print_pretty(req.json())

my_id = req.json()["id"]
my_name = req.json()["name"]


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
Assuming your background is uniform (and it most certianly is not), here is a rough background subtraction.
This may not work for all astronomical fields (e.g. clusters, low surface brightness features, etc). There are more sophisticated ways to deal with the background subtraction, see the Background Estimation documentation.
Can you write Python code for it?


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
Coordinates of your favorite object, in this case M82 in degrees.
Can you write Python code for it?

ra, dec = 148.969687, 69.679383

co = coordinates.SkyCoord(ra=ra, dec=dec,unit=(u.deg, u.deg), frame='fk5')

xid = SDSS.query_region(co, radius=10 * u.arcmin)
# print the first 10 entries
print(xid[:10])
print(xid.keys())

print(xid['ra','dec'][:10]) # print the first 10 entries


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
Gaia query
Gaia is a space observatory designed to observe the precise positions of stars in order to determine their distances and motions within the galaxy.  Gaia star coordinates are the new golden standard for astrometry.  Being able to querry their database to correct your catalogs and images to their coordinate system would be highly beneficial.
Can you write Python code for it?

from astroquery.gaia import Gaia

coord = coordinates.SkyCoord(ra=280, dec=-60, unit=(u.degree, u.degree), frame='icrs')
radius = u.Quantity(1.0, u.arcmin)
j = Gaia.cone_search_async(coord, radius)
r = j.get_results()

r.pprint()
print(r.keys())


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
IRSA query of 2MASS
IRSA is the NASA/IPAC Infrared Science Archive.  IRSA typically has a infrared mission focus, such as 2MASS, Herschel, Planck, Spitzer, WISE, etc...
Can you write Python code for it?

from astroquery.irsa import Irsa

Irsa.list_catalogs()

table = Irsa.query_region("m82", catalog="fp_psc", spatial="Cone",radius=10 * u.arcmin)

table

table.keys()


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
In order to convert to photons/s properly, we need to check the units. Our beginning and ending units match (i.e. energy/time), we just need to acount for the energy of the photon which is depends on the wavelength.
$$ E_{phot} = \frac{h c}{\lambda} $$
(1) As an exercise for the reader. Modify the above table to estimate the number of photons/s from the H-alpha line luminosity the energy of the photon instead.
Photometry with Photutils
What is photometry?  It is the measurement of light from a source, either represented as flux or magnitude. There are several ways to compute the flux from a source. In this tutorial we will go over aperture photometry, where you draw a circular region around the source and measure the flux inside.
Photoutils is a useful astropy package for performing photometry.  This includes but not limited to aperture photometry, PSF photometry, star finding, source isophotes, background subtraction, etc...
Large parts of this tutorial are borrowed from Getting Started with Photutils documentation.
Here we load in a star field image provided with Photutils.  We are interested in a small subsection of the image in which we want to do photometry.
Can you write Python code for it?

import numpy as np
from photutils import datasets

hdu = datasets.load_star_image()
image = hdu.data[500:700, 500:700].astype(float)
head = hdu.header


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
Plot image
Can you write Python code for it?

ax = plt.subplot()
ax.imshow(m82_image, cmap='gray_r', origin='lower', vmin=-10, vmax=20)
ax.set_xlabel('X (pixels)')
ax.set_ylabel('Y (pixels)')


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
Plot the in RA and Declination coordinates space, instead of pixels. The WCS (or World Coordinates System) in a FITS file has a set of keywords that define the coordinate system.
Typical keywords are the following:
CRVAL1, CRVAL2 - RA and Declination (in degrees) coordinates of CRPIX1 and CRPIX2.
CRPIX1, CRPIX2 - Pixel positions of CRVAL1 and CRVAL2.
CDELT1, CDELT2 - pixel scale (or plate scale) in degrees/pixel
(or CD matrix: CD1_1, CD1_2, CD2_1, CD2_2 which represent the pixel scale and rotation with respect to the position angle)
Can you write Python code for it?


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
Print the columns returned from the search.
Can you write Python code for it?


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
SDSS query
SDSS or the Sloan Digital Sky Survey is a massive survey of the sky which originally was optical imaging and spectroscopy.  It has been operating for over 20 years and the well beyond the original planned survey as it has expanded to near-infrared observations of stars and IFUs (integral field units) for observing galaxies and eventually the whole sky LVM (Local Volume Mapper). It is an immense data set to draw from.
Can you write Python code for it?


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
Skyview
Skyview is a virtual observatory that is connected to several surveys and imaging data sets spanning multiple wavelengths. It can be a handy way to get quick access to multi-wavelength imaging data for your favorite object.
Can you write Python code for it?

from astroquery.skyview import SkyView

SkyView.list_surveys()

pflist = SkyView.get_images(position='M82', survey=['SDSSr'],radius=10 * u.arcmin)

ext = 0
pf = pflist[0] # first element of the list, might need a loop if multiple images
m82_image = pf[ext].data


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
The type function is useful for determining what data type you are dealing with, especially if it is an unfamilar one.
Can you write Python code for it?


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
We can add new entries to the table.  In this case specific star formation rate (sSFR).
Can you write Python code for it?

t['ssfr'] = t['sfr'] / t['stellar_mass']

t


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
We can also group subsets of columns.
Can you write Python code for it?


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
We can call a specific row, in this case the first row, which shows all the entries for the first object.  Remember python indexing starts with zero!
Can you write Python code for it?


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
We can view a specific column by calling is name.
Can you write Python code for it?


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
We can write to a standard format if we like, binary FITS tables are often used for this. If could be an ASCII file (either sapace, tab or comma delimited) or whatever your favorite sharable file format is.  There is a whole list of supported formats that can be read in, many astronomy specific.
Can you write Python code for it?


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
We use DAOStarFinder to find all of the objects in the image.
The FWHM (Full Width Half Maximum) is a way of measureing the size of the source. Often for unresolved sources, like stars, the FWHM should be matched the resolution of the telescope (or the seeing).
The threshold is the value above the background in which the star finding algorithm will find sources. Often this should be done in a statistical sense in which a real detection is n-sigma above the background (or n standard deviations). We can measure the standard deviation of background.
Statistics aside:
Assuming a Gaussian distribution for the noise, 1-sigma contains 68% of the distribution, 2-sigma contains 95% of the distribution and 3-sigma contains 99.7% of the distribution. Often 5-sigma is the golden standard in physics and astronomy for a "real" detection as it is contains 99.99994% background distribution.
Here we use 3-sigma (3 standard devations) above the background for a "real" detection.
Can you write Python code for it?

from photutils import DAOStarFinder
from astropy.stats import mad_std

bkg_sigma = mad_std(image)

daofind = DAOStarFinder(fwhm=4., threshold=3.*bkg_sigma)
sources = daofind(image)

for col in sources.colnames: sources[col].info.format = '%.8g'  # for consistent table output
print(sources[:10])
print(sources.info)

from photutils import aperture_photometry, CircularAperture

positions = np.transpose((sources['xcentroid'], sources['ycentroid']))
apertures = CircularAperture(positions, r=4.)
phot_table = aperture_photometry(image, apertures)

for col in phot_table.colnames: phot_table[col].info.format = '%.8g'  # for consistent table output
print(phot_table[:10])
print(phot_table.info)

import matplotlib.pyplot as plt

plt.imshow(image, cmap='gray_r', origin='lower')
apertures.plot(color='blue', lw=1.5, alpha=0.5)

print(head.keys())


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
When we create the table, we can assign the names to each column
Can you write Python code for it?

t = QTable([a, b, c, d],
           names=('redshift', 'sfr', 'stellar_mass', 'velocity'))


I am working on the file "MoreNotebooks/AstropySmorgasbord.ipynb".
The first task is:
You can read it in just as easily too. As you can see, the units are stored as well, as long as you are using a QTable.
Can you write Python code for it?

all_targets = QTable.read("my_table.fits")

all_targets


I am working on the file "MoreNotebooks/Buggy_programs.ipynb".
The first task is:
1) Only purple and pink colors
This program generates purple and pink colors.
Can you write Python code for it?

# I want to make a pcolor map with only lots of nice shades of purple and maybe some pink

# How many colors do you want?
nbr_color = 10

# Initiate a color array
purples = np.zeros(nbr_color,3)

# Add some colors in the array
# These are rgb colors, with the first, second and third value are for red, green and blue respectively.
# when you use rgb colors, the fractional contribution of red, green and blue are given as a number between 0 and 1.
# As an example, a very red color is (1,0,0), whilst navy is more like (0,0,.4).
purples[:,1] = np.random.random(nbr_color)*0.5 + 0.2
purples[:,2] = np.random.random(nbr_color)*0.2
purples[:,3] = np.random.random(nbr_color)*0.5 + 0.5

# We want to give the colors names
labels = []
for i in range(nbr_color):
labels.append('Purple '+str(i+1))

# Enjoy them in a pie chart
fig, ax = plt.subplots(1,1,figsize=(8,8))
fracs = 1/nbr_color
ax.pie([fracs]*nbr_color,colors=purples, labels=labels)


I am working on the file "MoreNotebooks/Buggy_programs.ipynb".
The first task is:
2) How many years is a year on the other planets?
There are eight planets orbiting around the Sun. One turn around the Sun for the Earth is what we define as one year. But how long time does it take for the other planets to revolve around our closest star?
To calculate this, we follow Kepler's third law:
$\dfrac{a^3}{P^2} = \dfrac{G(M_{\odot} + M_{\text{p}})}{4\pi^2},$
where $a$ is the separation between the Sun and the planet, $P$ is the orbital period, $M_{\odot}$ is the mass of the Sun and $M_{\text{p}}$ is the mass of the planet. Since the Sun is much more massive than the planets, the formula can be approximated as
$\dfrac{a^3}{P^2} \approx \dfrac{GM_{\odot}}{4\pi^2}.$
Let's read a file with the distances between the Sun and the planets and calculate how long their orbits are.
Can you write Python code for it?

# Need some help from astropy for the mass of the Sun and the gravitational constant
from astropy import constants as const
from astropy import units as u
G = const.G
Msun = const.M_sun

# Read in the data
data = np.genfromtxt('planet_information.txt',dtype=str)

# Assign variable names that match
planet_names = data[:,0]   # The names of the planets
planet_distances = np.float_(data[:,1])*u.AU  # The distances to the Sun in AU

# Calculate the periods using Kepler III
planet_periods_yr = np.sqrt((planet_distances**3.)*(4*np.pi**2)/(G*Msun))      # periods in earth years

# Make a figure of period as function of distance to the Sun
fig, ax = plt.subplots(1,1,figsize=(8,5))
for i in range(len(planet_names)):
  ax.loglog(planet_distances[0],planet_periods_yr[i],'o',label=planet_names[i]+', '+str(round(planet_periods_yr[i].value,1))+'yr')

ax.legend(loc=0,edgecolor='none')
ax.set_xlim(0.2, 2*np.max(planet_distances.value))
ax.set_ylim(0.2, 2*np.max(planet_periods_yr.value))

ax.set_xlabel('Distance between Sun and planets [AU]')
ax.set_ylabel('Period in Earth years')

# You can see that the planets line up in this log-log diagram since a^3/P^2 = C => 2*log10(P) \propto 3*log10(a)


I am working on the file "MoreNotebooks/ModelFitting/Emcee.ipynb".
The first task is:
2 Bayes' Theorem
Bayesian statistics is based on  Bayes' theorem (for an excellent intro, see 3blue1brown's video). It's actually a very simple idea and an equally simple equation. It's dealing with the equation that gets complicated. Let's say we have some data $\vec{D}$. In the case of our emission line, the data is the number of counts in each wavelength bin. We have a model with some number of parameters $\vec{\theta}$. Bayes' theorem simply states:
$$P\left(\vec{\theta}\left|\vec{D}\right.\right) = \frac{P\left(\vec{D}\left|\vec{\theta}\right.\right)P\left(\vec{\theta}\right)}{P\left(\vec{D}\right)}$$
What this says is that the probability that we get a particular set of parameters given a fixed set of data (which is what we want) is proportional to the probability that we get the data given a fixed set of parameters (which we can calculate) times the probability of the parameters (the priors). The denominator is the probability that we got the data we did, which requires integrating over all possible parameters:
$$P\left(\vec{D}\right) = \int P\left(\vec{D}\left|\vec{\theta}\right.\right)P\left(\vec{\theta}\right)\ d\vec{\theta}$$
and really just ensures that the probability is normalized to 1.
You might wonder what the difference between the priors $P\left(\vec{\theta}\right)$ and $P\left(\vec{\theta}\left|\vec{D}\right.\right)$ (called the likelihood) is. The likelihood is what your data tells you about the parameters. The priors are constraints that are external to the data. It could be a previous experiment's result that you are incorporating into your own. It could be a purely logical constraint (e.g., the age of the universe must be greater than 0), it could even be a gut feeling.
Working with the above equation isn't too bad if the number of parameters is small and the priors and likelihoods are all simple. In fact, if you use uniform priors and normally-distributed errors, you get the good-old least-squares formalism. But pretty quickly you can get in a situation where the equation (and...
Can you write Python code for it?

def model(x, cont, slope, amp, center, width):
    model = amp*exp(-0.5*power(x-center,2)/width**2) + cont + \
             slope*x
    return model


I am working on the file "MoreNotebooks/ModelFitting/Emcee.ipynb".
The first task is:
As you can see, the walkers can start out rather far from the true value (blue horizontal lines), but after some time, they all converge to a value close to the true value (though not equal, thanks to the noise we added). It's at this point that we say the MCMC chain has converged. Since we're sure this is the case (make sure), we can reset the chains and run for a longer time to get good statistics.
Can you write Python code for it?

sampler.reset()
pos,prob,state = sampler.run_mcmc(pos, 1000)


I am working on the file "MoreNotebooks/ModelFitting/Emcee.ipynb".
The first task is:
Lastly, we construct the numerator of Bayes' formula. We won't compute the denominator, since it is a constant and we are only interested in the shape of $P\left(\vec{\theta}\left|\vec{D}\right.\right)$, since we are only interested in parameter inference. In other words, we only care about the relative probability of different values of the parameters. If we were comparing two models and wanted to know which was more likely the correct one, then we'd need the compute the denominator as well to get a full probability.
Can you write Python code for it?

def lnprob(p, wave, flux):
    # priors
    lp = lnprior(p)
    if not isfinite(lp): return -inf
    return lp + lnlike(p, wave, flux)


I am working on the file "MoreNotebooks/ModelFitting/Emcee.ipynb".
The first task is:
Next, we need the likelihood $P(\vec{D}|\vec{\theta})$. Given the parameters $\vec{\theta}$, the model $M(x,\vec{\theta})$ is given by the function model(). Under our assumpsions, this model will differ from the observed data because of Poisson errors. For large counts, the Poisson distribution is well-approximated by a normal distribution with variance ($\sigma^2$) equal to the counts. So, given a set of parameters $\vec{\theta}$, the probability we measure the flux in channel $i$ to be $f_i$ given by:
$$P\left(f_i\left|\vec{\theta}\right.\right) = N\left(M(\vec{\theta}), \sqrt{f_i}\right)$$,
where $N$ is the normal distribution. For the entire data-set, we have to multiply the probabilities of all the individual channels. Or, since we need the log of the probability:
$$P\left(\vec{D}\left|\vec{\theta}\right.\right) = \Pi_i P\left(f_i\left|\vec{\theta}\right.\right)$$
We'll use scipy's stats module, which has the normal distribution (and its logarithm) built in. Just like the priors, emcee wants the natural logarithm of the probability, so instead of multiplying all the probabilities, we sum all the logarithms of the probabilities.
Can you write Python code for it?

from scipy.stats import norm
def lnlike(p, wave, flux):
    cont,slope,amp,center,width = p
    m = model(wave, *p)
    return sum(norm.logpdf(flux, loc=m, scale=sqrt(flux)))


I am working on the file "MoreNotebooks/ModelFitting/Emcee.ipynb".
The first task is:
Now that we have the probability all figured out, we could in principle figure out where it is maximal and compute the 1-2-3-sigma intervals. This may or may not be possible in "closed form". The more parameters, priors and complicated the model gets, the less likely you'll be able to compute the derivatives (for optimization) and integrals (for expectation values and confidence intervals). But we can always compute these numerically and that's what MCMC is all about. With the emcee module, we do this by creating a bunch of "walkers" that wander around parameter space, always seeking higher probability regions, but also randomly sampling the space. After a certain amount of time, they wander around randomly enough that they lose track of where they started. When this happens, the steps the walkers take is a reflection of $P\left(\vec{\theta}\left|\vec{D}\right.\right)$. So inferences about the moments of $P\left(\vec{\theta}\left|\vec{D}\right.\right)$ can be determined by doing statistics on the walkers' steps. For example, the expectation (mean value) of the amplitude is:
$$\left<A\right> \equiv \int P\left(\vec{\theta}\left|\vec{D}\right.\right)A d\vec{\theta} \simeq mean(A_i)$$
where A_i are the values of amp at each step $i$. The more steps you take, the more accurate the estimate.
So now we create a number of walkers and start them off in random locations around parameter space. In this example, we know the true values so we just perturb around that. When you don't know the true values, you could start in completely random locations or use other tools (like curve_fit) to find an initial starting point.
Can you write Python code for it?

Nwalker,Ndim = 50,5
ptrue = array([500.,5.0,150.,5.0, 0.5])
# add a random vector 0.1 times the true vector to the true vector
p0 = [ptrue + 0.1*random.randn(Ndim)*ptrue for i in range(Nwalker)]


I am working on the file "MoreNotebooks/ModelFitting/Emcee.ipynb".
The first task is:
Once the sampler is done, we can do statisics on the "chains". The sampler object has an attribute flatchain, where all the walkers are combined. This gives us Nwalkers*Nsteps samples from the posterior. We could get the best-fit values and errors by doing statistics on the chains:
Can you write Python code for it?

print(mean(sampler.flatchain, axis=0)) # best-fit, well really expectation value
print(std(sampler.flatchain, axis=0)) # errors
# deviation from true parameters in units of standard error
print((mean(sampler.flatchain, axis=0)-ptrue)/std(sampler.flatchain, axis=0))
print(cov(sampler.flatchain.T)) # covariance matrix


I am working on the file "MoreNotebooks/ModelFitting/Emcee.ipynb".
The first task is:
So let's see what each walker did. We'll graph the value of each parameter as a function of step number. Each walker will have its own line.
Can you write Python code for it?

fig,ax = subplots(4,1)
res = [ax[i].plot(sampler.chain[:,:,i].T, '-', color='k', alpha=0.3) for i in range(4)]
res = [ax[i].axhline(ptrue[i]) for i in range(4)]


I am working on the file "MoreNotebooks/ModelFitting/Emcee.ipynb".
The first task is:
So we have the wavelength on the x-axis, which is assumed to have no uncertainty. The measured flux is different from the "true" flux due to Poisson noise. Let's plot the true flux and observed flux to see how things look.
Can you write Python code for it?

%matplotlib inline
from matplotlib.pyplot import plot,step,xlabel,ylabel,show,subplots
plot(wave, flux, 'r-')
step(wave, obs_flux, color='k')
xlabel('Wavelength (Angstroms)')
ylabel('Counts')
show()


I am working on the file "MoreNotebooks/ModelFitting/Emcee.ipynb".
The first task is:
So we now have Nwalker initial points. We can run the emcee sampler, givin it the lnprob function and any extra arguments it needs. The run_mcmc function takes the initial starting points and how many steps you want each to take. It returns the last position, probability, and state of each walker.
Can you write Python code for it?

import emcee
sampler = emcee.EnsembleSampler(Nwalker, Ndim, lnprob, args=(wave, obs_flux))
pos,prob,state = sampler.run_mcmc(p0, 500)


I am working on the file "MoreNotebooks/ModelFitting/ErrorsInXandY.ipynb".
The first task is:
Hmmmm. Looks like the problem is still there. So what's the reason for this? Well, it has to do with the way that we incorporated the $\sigma_x$, obviously. This was done by what's called marginalizing. Let's look a bit more closely at what it means to have observed our data. We know (since we made the fake data ourselves) that both $x$ and $y$ had noise added to them that was Gaussian (normally distributed). So, using $N$ to represent normal distributions, the likelihoods for our observed $x$ and $y$ values are:
$$ P\left(x_i\left| x_i^T,\sigma_{x,i}\right.\right) = N(x_i-x_i^T, \sigma_{x,i})$$
$$ P\left(y_i\left| y_i^T,\sigma_{y,i}\right.\right) = N(y_i-y_i^T, \sigma_{y,i})$$
In other words, the $x$ data is drawn from a normal distribution whose mean (center) is the true value $x^T_i$, and whose width is the uncertainty $\sigma_{x,i}$. Likewise for $y_i$, except the true value of $y_i$ is given by our model, so:
$$ P\left(y_i\left| y_i^T,\sigma_{y,i}\right.\right) = N(y_i - a - b x_i^T, \sigma_{y,i})$$
and together, these give us a joint log-likelihood for the entire data set $\vec{D}$:
$$\log P\left(\vec{D}\left|x_i^T,a,b\right.\right) = \sum_i \log N\left(x_i-x_i^T, \sigma_{x,i}\right)
  + \sum_i \log N\left(y_i - a - b x_i^T, \sigma_{y,i}\right)$$
but we've got all these $x_i^T$ values on the right-hand side. In real life, we don't know what these are, so we are forced to consider them as unknown parameters. It would be really nice if we could get rid of them. Well, turns out you can do that. If you integrate the probability over all possible values of $x_i^T$, weighted by their priors:
$$\log P\left(\vec{D}\left|a,b\right.\right) = \sum_i \int \log P\left(\vec{D}\left|x_i^T,a,b\right.\right)
P\left(x_i^T\right) dx_i^T$$
If we assume: 1) uniform priors $P(x_i^T) \propto 1$, that all $x_i^T$ are independent, and integrate from $-\infty$ to $+\infty$, we get the nice result:
$$ P\left(\vec{D}\left|a,b\right.\right) = \sum_i \log N\left(y_i - a - b x_i, \sqrt{\sigma^2_{y,i}+b^2\sigma^2_{x,i}}\right) = -...
Can you write Python code for it?

import pystan
model_str = '''
data {
   int <lower=1> N;   // number of points
   vector[N] x;       // observed x
   vector[N] y;       // observed y
   vector[N] dx;      // error in x
   vector[N] dy;      // error in y
}

parameters {
   real <lower=-10, upper=10> a;        // intercept
   real <lower=-10, upper=10> b;        // slope
   vector<lower=0, upper=10>[N] xtrue;  // true values of x
}

model {
   x ~ normal(xtrue, dx);
   y ~ normal(a + b*xtrue, dy);
}'''
sampler2 = pystan.StanModel(model_code=model_str, verbose=False)


I am working on the file "MoreNotebooks/ModelFitting/ErrorsInXandY.ipynb".
The first task is:
Looks like 100 iterations should do it. Let's reset and run for longer and print out the results.
Can you write Python code for it?

sampler.reset()
pos,prob,state = sampler.run_mcmc(pos, 1000)
print(np.mean(sampler.flatchain, axis=0))
print(np.std(sampler.flatchain, axis=0))


I am working on the file "MoreNotebooks/ModelFitting/ErrorsInXandY.ipynb".
The first task is:
Looks reasonable, I hope. We want to find the best-fit line to the data. It should be close to the orange line (the truth), but not equal, since we introduced noise. First, we could fit this with scipy.optimize.curve_fit as in a previous notebook. It needs an error (sigma), but that's usually the error in $y$. How do we incorporate the error in $x$?. Well, one way to think about it is we are minimizing the residuals (numerator of $\chi^2$), call them $\Delta_i$:
$$ \Delta_i = y_i - (a + b x_i)$$
so the total uncertainty is:
$$ \sigma^2_i = \left(\frac{\partial \Delta_i}{\partial x_i}\right)^2 \sigma^2_{x,i} + \left(\frac{\partial \Delta_i}{\partial y_i}\right)^2 \sigma^2_{y,i} = b^2\sigma^2_{x,i} + \sigma^2_{y,i}$$
So we just need to add the error in y to the error in x, scaled by the slope, in quadrature. But... we don't know the slope until we solve the problem! Normally, you would make an initial guess, compute the errors, fit for the line, update the slope, compute the errors again, fit for a better slope, and repeat this process until the slope converges to a fixed value. So save on space, let's just cheat and use the true slope to compute the errors.
Can you write Python code for it?

from scipy.optimize import curve_fit
def mod(x, a, b):
    return a + x*b
sig_y = np.ones(N)*sig_y   # curve_fit needs arrays of errors
sig_x = np.ones(N)*sig_x
sigma_total = np.sqrt(sig_y**2 + b_true**2*sig_x**2)
pars,cov = curve_fit(mod, x_obs, y_obs, [0, 1], sigma=sigma_total)
print(pars)
print(cov)
ax.plot(xx, xx*pars[1]+pars[0])
fig


I am working on the file "MoreNotebooks/ModelFitting/ErrorsInXandY.ipynb".
The first task is:
Note that by putting limits (0,10) on xtrue, I've specified a uniform prior, exactly as the true values were drawn. The other parameters also have uniform priors associated with them. Now we prepare the data and run a test sample, and check to see if we have convergence (rhat close to 1.0).
Can you write Python code for it?

idata = dict(N=len(x_obs), x=x_obs, y=y_obs, dx=sig_x, dy=sig_y)
output = sampler2.sampling(data=idata, chains=4, iter=1000, warmup=500)
print(output)


I am working on the file "MoreNotebooks/ModelFitting/ErrorsInXandY.ipynb".
The first task is:
Note that the priors are completely uninformative (uniform over all values). The likelihood is computed in the usual way with the normal distribution and we comptue the total error based on the current slope, which is updated at each iteration, so the errors will change as the walkers make their way through parameters space. You should think about what this might mean as the slope becomes small or large. Let's run a bunch of walkers and see what we get.
Can you write Python code for it?

import emcee
sampler = emcee.EnsembleSampler(Nwalker, Ndim, lnprob, args=(x_obs, y_obs, sig_x, sig_y))
pos,prob,state = sampler.run_mcmc(p0, 500)


I am working on the file "MoreNotebooks/ModelFitting/ErrorsInXandY.ipynb".
The first task is:
Note that we now have N+2 parameters being reported. I kept the number of iterations lower than I would normally and the values of Rhat and standard errors in the parameters look reasonable. If not, increase iter. But even though STAN lets you have large numbers of parameters, the more you have, the slower the sampler will be. We'll do the same thing we did above and re-fit random data sets many times. This will take a while...  grab a coffee or tea. Catch up on facebook. But for this final example, we want enough samples to be confident we've "fixed" the problem.
Can you write Python code for it?

slopes = []
intercepts = []
for i in range(500):
    x_obs = x_true + np.random.normal(0, sig_x, size=N)
    y_obs = y_true + np.random.normal(0, sig_y, size=N)
    idata = dict(N=len(x_obs), x=x_obs, y=y_obs, dx=sig_x, dy=sig_y)
    output = sampler2.sampling(data=idata, chains=4, iter=1000, warmup=500)
    samples = output.extract(permuted=True)
    slopes.append(np.mean(samples['b']))
    intercepts.append(np.mean(samples['a']))
    print(i, end=' ')

fig,axes = plt.subplots(1,2, figsize=(10,5))
axes[0].hist(slopes, bins=20)
axes[0].axvline(b_true, color='red', zorder=10)
axes[1].hist(intercepts, bins=20)
axes[1].axvline(a_true, color='red', zorder=10)


I am working on the file "MoreNotebooks/ModelFitting/ErrorsInXandY.ipynb".
The first task is:
Now plot out the samples to see how long it takes to get a convergence.
Can you write Python code for it?

fig,ax = plt.subplots(2,1)
res = [ax[i].plot(sampler.chain[:,:,i].T, '-', color='k', alpha=0.3) for i in range(2)]
res = [ax[i].axhline(ptrue[i]) for i in range(2)]


I am working on the file "MoreNotebooks/ModelFitting/ErrorsInXandY.ipynb".
The first task is:
You should see a couple of histograms and red lines that represent the true values. Depending on what you chose for your uncertainties, the red lines may not line up with the peaks of the distributions. In fact, the larger you make the $x$ errors, the more they won't agree. Now, this isn't just due to a random draw and noisy data. This is happening on average over a large number of trials. The conclusion:  least-squares is a biased estimator of a linear model when errors are present in the $x$ values (also called the predictor variable).
The reason is a subtle one which we'll get to. But you might be thinking that it has something to do with keeping the errors $\sigma_i$ fixed, when they really should vary as the slope varies. We can test that, but least-squares won't allow this. Instead, we can use the MCMC techniques we saw earlier. First up, let's do the problem with emcee.
Can you write Python code for it?

from scipy.stats import norm
def lnprior(p):
    a,b = p
    return 0

def lnlike(p, x, y, dx, dy):
    a,b = p
    model = mod(x, a, b)
    sig_tot = np.sqrt(dx**2*b**2 + dy**2)
    return sum(norm.logpdf(y, loc=model, scale=sig_tot))

def lnprob(p, x, y, dx, dy):
    a,b = p
    lp = lnprior(p)
    if not np.isfinite(lp):
        return -inf
    return lp + lnlike(p, x, y, dx, dy)

Nwalker,Ndim = 10,2
ptrue = np.array([a_true,b_true])
# add a random vector 0.1 times the true vector to the true vector
p0 = [ptrue + 0.1*np.random.randn(Ndim)*ptrue for i in range(Nwalker)]


I am working on the file "MoreNotebooks/ModelFitting/NNLS.ipynb".
The first task is:
Aside from the best-fit values and their uncertainties, it's also a good idea to examine the covariance matrix, to see how correlated the parameters are. A quick way to do this is to construct the correlation matrix from the covariance matrix $C[i,j]$ and errors $\sigma[i]$:
$$\rho[i,j] = \frac{C[i,j]}{\sigma[i]\sigma[j]}$$
positive values denote correlation, negative denote anti-correlation. $\rho$ ranges from -1 to 1. A value close to 0 denotes no significant correlation.
Can you write Python code for it?

pcor = pcov/outer(err,err)
for i in range(pcor.shape[0]):
  for j in range(pcor.shape[1]):
    print("{:5.2f} ".format(pcor[i,j]), end='')
  print()


I am working on the file "MoreNotebooks/ModelFitting/NNLS.ipynb".
The first task is:
Now we run curve_fit. We pass in the model function, the x and y data, an initial guess for the parameters, and the error in the observations. Since the flux has Poisson noise, we can simply put in $\sigma(y) = \sqrt y$.
Can you write Python code for it?

from scipy.optimize import curve_fit
popt,pcov = curve_fit(model, wave, obs_flux, p0=(425.,0.0,80.,4.5,1.0))
print(popt)
err = sqrt(diag(pcov))
print(err)


I am working on the file "MoreNotebooks/ModelFitting/Pystan.ipynb".
The first task is:
2 Writing the STAN model
With emcee, we wrote functions to compute priors and likelihoods. STAN is different. It has its own language, very similar to C. The nice thing about it is it has many probability distributions built in. You also write it very similarly to the notation we use to describe Bayesian models. One important symbol is $\sim$ which you should read as "drawn from". If, for example, we had a measurement $x$ with an error $\sigma_x$ and say the true value is $x^T$, the if the errors are normally distributed, we can say:
$$ x \sim N\left(x^T, \sigma_x\right) $$
that is, x is a random value drawn from a normal distribution ($N$) with center $x^T$ and standard deviation $\sigma_x$.
A STAN model is a file or string containing blocks of code. There are 3 minimum blocks needed:  data, parameters, and model. The data block describes what data will be input into the STAN model and you should think of these as fixed values. The parameters block defines all the parameters your model depends on. These values will change throughout the MCMC chain and will be output at the end. The model block describes the probabilistic model that links your data to your parameters.
Here is a simple STAN model to fit the emission line. Like C, we need to declar not only the variables we're using, but their sizes if they are arrays.
Can you write Python code for it?

model_string = '''
data {
  int <lower=1> N;      // number of data points
  vector[N] wave;       // the wavelengths
  vector[N] flux;       // the observed flux values
}

parameters {
  real<lower=-1000, upper=1000> cont;  // continuum level
  real<lower=-100, upper=100> slope;   // continuum slope
  real<lower=0, upper=1000> amp;       // amplitude of Gaussian
  real <lower=0, upper=10> center;     // center of the line
  real <lower=0, upper=10> width;      // scale
}

model {
  vector[N] mod_flux;    // the model flux

  // continuum slope + Gaussian
  mod_flux = amp*exp(-0.5*square(center - wave)/square(width)) +
             cont + slope*wave;
  // Poisson is approximately Normal with sigma = sqrt(counts)
  flux ~ normal(mod_flux, sqrt(flux));
}'''


I am working on the file "MoreNotebooks/ModelFitting/Pystan.ipynb".
The first task is:
If the Rhat values are all close to 1.0, things are good. If not, then either your chains have not converged, or the model has a problem. Multi-modal posteriors (i.e., where multiple solutions exist) are a good example. Each chain may get "stuck" in one or the other regions of high local probability and never converge to the same answer. In our case, though, Rhat should be close to 1.
You could go with the summary statistics above, but most of us like to go a bit further, plotting the traces, covariances, etc. To access the chains, we use the extract() function. In this case, I will also specify permuted=True, which will combine the chains and assign them to a dictionary for easy access. Let's plot out the chains like we did in emcee.
Can you write Python code for it?

samples = output.extract(permuted=True)
pars = list(samples.keys())
fig2,axes = subplots(len(pars),1)
for i,par in enumerate(pars):
   axes[i].plot(samples[par], '-')
   axes[i].set_ylabel(par)


I am working on the file "MoreNotebooks/ModelFitting/Pystan.ipynb".
The first task is:
The output variable contains lots of good stuff. For starters, if you print it out, it gives you some nice statistics on the samples. For each variable, you get the mean of the chains (best-fit values), the error in the mean (which is an estimate of error due to having finite sampling), the standard deviation (error in the best-fit), various percentiles, the effective number of samples, and a useful statistic called Rhat, or $\hat{R}$. Compute the dispersion of a parameter in each chain and take the average. Then compute the average of the parameter in each chain and compute the dispersion of that. If you take the ratio of these two quantities, you get a value close to 1.0 if the chains have converged to the same value and have the same scatter. It's a good test for whether your chains have converged.
Can you write Python code for it?


I am working on the file "MoreNotebooks/Skyfit.ipynb".
The first task is:
Currently, astropy.coordinates does not have a lunar phase function, so we'll just use the angular separation as a proxy. If the angular separation is 0 degrees, that's new moon, whereas an angular separation of 180 degrees is full moon. Other phases lie in between.
Can you write Python code for it?

seps = moon.separation(sun)                         # angular separation from moon to sun
data['phase'] = pd.Series(seps, index=data.index)   # Add this new parameter to the data frame


I am working on the file "MoreNotebooks/Skyfit.ipynb".
The first task is:
So as we get closer to the horizon, there's less moonlight, as it's been attenuated by the larger amount of atmosphere. Lastly, to convert these nanoLamberts into magnitudes per square arc-second, we need the dark (no moon) sky brightness at the zenith, $m_{dark}$, and convert that to nanoLamberts using this formula:
$$ B_{dark} = 34.08\exp (20.7233 - 0.92104 m_{dark})10^{-0.4 k_X (X(Z)-1)}X(Z) $$
where we have also corrected for attenuation by the atmosphere and air-glow (which increases with airmass). The final model for observed sky brightness $m_{sky}$ is:
$$ m_{sky} = m_{dark} - 2.5 \log_{10}\left(\frac{B_{moon} + B_{dark}}{B_{dark}}\right) $$
Whew! That's a lot of math. But that's all it is, and we can make a python function that will do it all for us.
Can you write Python code for it?

from numpy import absolute,exp,log10,arccos,log,where

def modelsky(alpha, rho, kx, Z, Zm, mdark):
    Istar = power(10, -0.4*(3.84+0.026*absolute(alpha)+4e-9*power(alpha,4)))
    frho = power(10, 5.36)*(1.06 + power(cos(rho),2))+power(10, 6.15-rho*180./pi/40)
    Bmoon = frho*Istar*power(10,-0.4*kx*X(Zm))*(1-power(10,-0.4*kx*X(Z)))
    Bdark = 34.08*exp(20.723 - 0.92104*mdark)*power(10,-0.4*kx*(X(Z)-1))*X(Z)
    return mdark - 2.5*log10((Bmoon+Bdark)/Bdark)


I am working on the file "MoreNotebooks/Skyfit.ipynb".
The first task is:
So there's less and less scattered light farther from the moon (at the center). This scattered light is then attenuated by the atmosphere. This attenuation is parametrized by the airmass $X$, the relative amount of atmosphere the light has to penetrate (with $X=1$ for the zenith). Krisciunas & Schaefer (1991) present this formula for the airmass:  $X(Z) = \left(1 - 0.96 \sin^2 Z\right)^{-1/2}$. We'll come back to this later. Suffice it to say for the moment that this is an approximation very close to the "infinite slab" model of the atmosphere. Putting it all together, the surface brigthness (in the interesting units of nanoLamberts from the moon will be:
$$ B_{moon} = f(\rho)I^*10^{-0.4 k_X X(Z_m)}\left[1 - 10^{-0.4k_X X(Z)}\right] $$
Let's visualize that first factor, which attenuates the light from the moon. I'll just set $I^*=1$ and $k_X=5$ to make the effect obvious. We'll define the airmass function for later use as well. Let's assume the top of the graph is the zenith ($Z=0$) and the bottom is the horizon ($Z=90$).
Can you write Python code for it?

def X(Z):
    '''Airmass as a function zenith angle Z in radians'''
    return 1./sqrt(1 - 0.96*power(sin(Z),2))

Z = (45 - jj*45)*pi/180.    # need in radians
imshow(f*power(10, -0.4*5*X(Z)), origin='lower', extent=(-45,45,-45,45))


I am working on the file "MoreNotebooks/Skyfit.ipynb".
The first task is:
Success! It definitely looksl like scattered moonlight is responsible for the bulk of the added sky brightness. But there's also a portion of data where the moon was bright but the sky was still dark. There's more to it than just phase. Now we turn to the task of fitting a model to this.
2)  The Model
Turns out that the definitive reference for this was authored by a colleague of mine:  Kevin Krisciunas at Texas A&M. His paper can be found at the ADS abstract service:  http://adsabs.harvard.edu/abs/1991PASP..103.1033K
You can read the details (lots of empirical formulas, scattering theory, and unit conversions), but the short of it is that we get a predictive model of the sky-brightness as a function of the following variables:

The lunar phase angle: $\alpha$
The angular separation between the sky field and the moon: $\rho$
The Zenith angle of the sky field:  $Z$
The Zenith angle of the moon:  $Z_m$
The extinction coefficient:  $k_X$
The dark-sky (no moon) sky background at zenith (in mag/square-arc-sec):  $m_{dark}$

The following diagram shows some of these variables:
Actually, $\alpha$, $\rho$, $Z$, and $Z_m$ are all functions of the date of observations and sky coordinates, which we have already. That leaves $k_x$ and $m_{dark}$ to be determined. Given these variables, the flux from the moon is given by an empirically-determined function that takes into account the fact that the moon is not a perfect sphere:
$$I^* = 10^{-0.4(3.84 + 0.026|\alpha | + 4\times 10^{-9}\alpha^4)}$$
This flux is then scattered by angle $\rho$ into our line of sight, contributing to the sky background. The fraction of light scattered into angle $\rho$ is given empirically by:
$$f(\rho) = 10^{5.36}\left[1.06 + \cos^2\rho\right] + 10^{6.15 - \rho/40} $$
This just tells us how quickly the sky brightness falls off as we look further away from the moon. We can visualize this by making a 2D array of angles from the center of an image ($\rho$) and comptuing $f(\rho)$.
Can you write Python code for it?

from numpy import sqrt, power, cos, sin, indices, pi
from matplotlib.pyplot import imshow, xlabel, ylabel
jj,ii = indices((1024,1024))/1024             # 2D index arrays scaled 0->1
rho = sqrt((ii-0.5)**2 + (jj-0.5)**2)*45.0    # 2D array of angles from center in degrees
f = 10**5.36*(1.06 + (cos(rho*pi/180)**2)) + power(10, 6.15-rho/40)
imshow(f, origin='lower', extent=(-45,45,-45,45), )
xlabel('X angular separation')
ylabel('Y angular separation')


I am working on the file "MoreNotebooks/Skyfit.ipynb".
The first task is:
There certainly seems to be a trend that the closer to full ($\alpha = 0$, yellow), the brighter the background and the closer the moon is to the field (lower $\rho$), the higher the background. Looks good.
4) Fitting (Training) the Model
Let's try and fit this data with our model and solve for $m_{dark}$, and $k_x$, the only unknowns in the problem. For this we need to create a dummy function that we can use with scipy's leastsq function. It needs to take a list of parameters (p) as its first argument, followed by any other arguments and return the weighted difference between the model and data. We don't have any weights (uncertainties), so it will just return the differences.
Can you write Python code for it?

from scipy.optimize import leastsq
def func(p, alpha, rho, Z, Zm, magsky):
    mdark,kx = p
    return magsky - modelsky(alpha, rho, kx, Z, Zm, mdark)


I am working on the file "MoreNotebooks/Skyfit.ipynb".
The first task is:
We now run the least-squares function, which will find the parameters p which minimize the squared sum of the residuals (i.e. $\chi^2$). leastsq takes as arguments the function we wrote above, func, an initial guess of the parameters, and a tuple of extra arguments needed by our function. It returns the best-fit parameters and a status code. We can print these out, but also use them in our modelsky function to get the prediction that we can compare to the observed data.
Can you write Python code for it?

pars,stat = leastsq(func, [22, 0.2], args=(data['alpha'],data['rho'],data['Z'],data['Zm'],data['magsky']))
print(pars)
# save the best-fit model and residuals
data['modelsky']=pd.Series(modelsky(data['alpha'],data['rho'],pars[1],data['Z'],data['Zm'],pars[0]), index=data.index)


I am working on the file "MoreNotebooks/Skyfit.ipynb".
The first task is:
With a little data exploring, it's pretty obvious that the majority of the outlying points comes from observations when the moon is relatively full and very low (or even below) the horizon. The reason is that the airmass formula that we implemented above has a problem with $Zm > \pi/2$. To see this, we can simply plot X(Z) as a function of 'Z':
Can you write Python code for it?

from numpy import linspace
from matplotlib.pyplot import plot, xlabel, ylabel,ylim
Z = linspace(0, 3*pi/4, 100)    # make a range of Zenith angles
plot(Z*180/pi, X(Z), '-')
xlabel('Zenith angle (degrees)')
ylabel('Airmass')


I am working on the file "Most-Time-Spent.ipynb".
The first task is:
Load Data
Can you write Python code for it?

start_col, end_col = 'starttime', 'stoptime'

# Loading just first 10,000 rows
df = pd.read_csv('201501-citibike-tripdata.csv', parse_dates=[start_col, end_col], nrows=10000)


I am working on the file "Most-Time-Spent.ipynb".
The first task is:
Parts of Day
Can you write Python code for it?

@np.vectorize
def minutes(time):
    """Convert time to minutes since 00:00

    >>> minutes('3:17')
    197
    """
    # Work on strings as well
    if isinstance(time, str):
        time = pd.Timestamp(time)

    return (time.hour * 60) + time.minute

# Parts of day, we can have overlaps here
parts_of_day = [
    ('morning', minutes('05:00am'),  minutes('11:00')),
    ('noon', minutes('11:00am'), minutes('14:00')),
    ('evening', minutes('14:00'), minutes('19:00')),
    # Night is two continuous parts
    ('night_a', minutes('18:30'), minutes('23:59')),
    ('night_b', minutes('00:00'), minutes('6:00')),
]

# Auxiliary DataFrame
parts_df = pd.DataFrame(index=df.index)
for col in [start_col, end_col]:
    parts_df[col] = minutes(df[col].dt.time)

for name, start, end in parts_of_day:
    parts_df['start_p'] = start
    parts_df['end_p'] = end
    ends = parts_df[[end_col, 'end_p']].min(axis=1)
    starts = parts_df[[start_col, 'start_p']].max(axis=1)
    duration = ends - starts
    parts_df.loc[duration >= 0, name] = duration[duration >= 0]

parts_df['night'] = parts_df['night_a'] + parts_df['night_b']
parts_df.fillna(0, inplace=True)

# Assign part of day where most time was spent
df['day_part'] = parts_df[['morning', 'noon', 'evening', 'night']].idxmax(axis=1)


I am working on the file "Motion Functions.ipynb".
The first task is:
Functions
There are three functions for controlling the motion of the robot:
* p3dx.move(ls,rs)
* p3dx.stop()
* p3dx.sleep(t)
The main motion function is:
python
p3dx.move(ls, rs)
where
ls : left wheel speed (rad/s)
   rs : right wheel speed (rad/s)
The function sets the speeds of the wheels, and the robot moves until blocked by an obstacle, or a new speed is set, or it is stopped with the function:
python
p3dx.stop()
For controlling the amount of time that the robot moves, there is the function:
python
p3dx.sleep(t)
where t is the number of seconds. During that pause, the program sleeps but the robot keeps moving with the last speed set.
Example
Use the code below for moving the robot. Feel free to explore with different speed and time values.
Can you write Python code for it?

# Move forward
p3dx.move(2.5,2.5)
p3dx.sleep(1)
p3dx.stop()

# Move backward
p3dx.move(-2.5,-2.5)
p3dx.sleep(1)
p3dx.stop()

# Turn left
p3dx.move(-2.5,2.5)
p3dx.sleep(1)
p3dx.stop()

# Turn Right
p3dx.move(2.5,-2.5)
p3dx.sleep(1)
p3dx.stop()


I am working on the file "Motion.ipynb".
The first task is:
Calculate and plot velocity
Can you write Python code for it?

# create a new empty column
data['velocity (m/s)'] = ''
data

# np.diff() calculates the difference between a value and the one after it
vel = np.diff(pos) / np.diff(time)

# fill the velocity column with values from the formula
data['velocity (m/s)'] = pd.DataFrame.from_items([('', vel)])

# display the data table
data

# That last velocity value will cause problems for further coding
# Make a new table using only rows 0 through 2
data2 = data.loc[0:2,['time (s)', 'velocity (m/s)']]
data2

# set new variables to plot
time2 = data2['time (s)']
vel2 = data2['velocity (m/s)']

# plot data just like before
plt.scatter(time2, vel2)
plt.title("Constant Speed?")
plt.xlabel("Time (s)")
plt.ylabel("Velocity (m)")
plt.autoscale(tight=True)

# calculate trendline equation like before
trend2 = np.polyfit(time2, vel2, 1)

# plot trendline like before
plt.plot(time2, np.poly1d(trend2)(time2), label='trendline')
plt.legend(loc='lower left')

# display the trendline's coefficients (slope, y-int)
trend2


I am working on the file "MovieLens/Atelier-pyspark-MovieLens.ipynb".
The first task is:
3 Analyse du fichier complet
MovieLens propose un plus gros fichier avec 20M de notes (138000 utilisateurs, 27000 films). Ce fichier est utilisé pour extraire un fichier test de deux millions de notes à reconstruire. Les paramètres précédemment optimisés, ils pourraient sans doute l'être mieux, sont appliqués pour une succesion d'estimation / prévision avec une taille croissante de l'échantillon d'apprentissage. Il aurait été plus élégant d'automatiser le travail dans une boucle mais lorsque les données sont les plus volumineuses des comportement mal contrôlés de Spark peuvent provoquer des plantages par défaut de mémoire.
3.1 Lecture des données
Le fichier est prétraité de manière analogue.
Can you write Python code for it?

# Chargement des fichiers si ce n'est déjà fait
import urllib.request
# fichier complet mais compressé
f = urllib.request.urlretrieve("http://www.math.univ-toulouse.fr/~besse/Wikistat/data/ml-ratings20M.zip",DATA_PATH+"ml-ratings20M.zip")

#Unzip downloaded file
import zipfile
zip_ref = zipfile.ZipFile(DATA_PATH+"ml-ratings20M.zip", 'r')
zip_ref.extractall(DATA_PATH)
zip_ref.close()

# Importer les données au format texte dans un RDD
ratings_raw_data = sc.textFile(DATA_PATH+"ratings20M.csv")
# Identifier et afficher la première ligne
ratings_raw_data_header = ratings_raw_data.take(1)[0]
ratings_raw_data_header

# Create RDD without header
all_lines = ratings_raw_data.filter(lambda l : l!=ratings_raw_data_header)

# Séparer les champs (user, item, note) dans un nouveau RDD
split_lines = all_lines.map(lambda l : l.split(","))
ratingsRDD = split_lines.map(lambda p: Row(user=int(p[0]), item=int(p[1]),
                                     rating=float(p[2]), timestamp=int(p[3])))

# Display the two first rows
ratingsRDD.take(2)

# Convert RDD to DataFrame
ratingsDF = spark.createDataFrame(ratingsRDD)
ratingsDF.take(2)


I am working on the file "MovieLens/Atelier-pyspark-MovieLens.ipynb".
The first task is:
3. Optimisation du rang sur l'échantillon 10k
Le fichier comporte 10 000 évaluations croisant les avis de mille utilisateurs sur les films qu'ils ont vus parmi 1700.
3.1 Constitution des échantillons
Séparation aléatoire en trois échantillons apprentissage, validation et test. Le paramètre de rang est optimisé en minimisant l'estimaiton de l'erreur sur l'échantillon test. Cette stratégie, plutôt qu'ue validation croisée est plus adaptée à des données massives.
Can you write Python code for it?

tauxTrain=0.6
tauxVal=0.2
tauxTes=0.2
# Si le total est inférieur à 1, les données sont sous-échantillonnées.
(trainDF, validDF, testDF) = ratingsDF.randomSplit([tauxTrain, tauxVal, tauxTes])
# validation et test à prédire, sans les notes
validDF_P = validDF.select("user", "item")
testDF_P = testDF.select("user", "item")

trainDF.take(2), validDF_P.take(2), testDF_P.take(2)


I am working on the file "MovieLens/Atelier-pyspark-MovieLens.ipynb".
The first task is:
3.3 Résultats et test
Can you write Python code for it?

# Quelques prévisions
pred_without_naDF.take(3)


I am working on the file "MovieLens/Atelier-pyspark-MovieLens.ipynb".
The first task is:
3.4 Prévision de l'échantillon test et erreur
Can you write Python code for it?

# Prévision de l'échantillon de validation
predDF = model.transform(testDF).select("prediction","rating")
#Remove unpredicter row due to no-presence of user in the train dataset
pred_without_naDF = predDF.na.drop()
# Calcul du RMSE
evaluator = RegressionEvaluator(metricName="rmse", labelCol="rating",
                            predictionCol="prediction")
rmse = evaluator.evaluate(pred_without_naDF)
print("Root-mean-square error for rank %d = "%best_rank + str(rmse))

trainDF.count()


I am working on the file "MovieLens/Atelier-pyspark-MovieLens.ipynb".
The first task is:
Les données sont lues comme une seule ligne de texte avant d'être restructurées au bon format d'une matrice creuse à savoir une liste de triplets contenant les  indices de ligne, de colonne et la note pour les seules valeurs renseignées.
Can you write Python code for it?

# Importer les données au format texte dans un RDD

small_ratings_raw_data = sc.textFile(DATA_PATH+"ml-ratings100k.csv")

# Identifier et afficher la première ligne
small_ratings_raw_data_header = small_ratings_raw_data.take(1)[0]
print(small_ratings_raw_data_header)

# Create RDD without header
all_lines = small_ratings_raw_data.filter(lambda l : l!=small_ratings_raw_data_header)

# Séparer les champs (user, item, note) dans un nouveau RDD
from pyspark.sql import Row
split_lines = all_lines.map(lambda l : l.split(","))
ratingsRDD = split_lines.map(lambda p: Row(user=int(p[0]), item=int(p[1]),
                                     rating=float(p[2]), timestamp=int(p[3])))

# .cache() : le RDD est conservé en mémoire une fois traité
ratingsRDD.cache()

# Display the two first rows
ratingsRDD.take(2)

# Convert RDD to DataFrame
ratingsDF = spark.createDataFrame(ratingsRDD)
ratingsDF.take(2)


I am working on the file "MovieLens/Atelier-pyspark-MovieLens.ipynb".
The first task is:
Prévision finale de l'échantillon test.
Can you write Python code for it?

#On concatane la DataFrame Train et Validatin
trainValidDF = trainDF.union(validDF)
# On crée un model avec le nouveau Dataframe complété d'apprentissage et le rank fixé à la valeur optimal
als = ALS( rank=best_rank, seed=seed, maxIter=maxIter,
                  regParam=regularization_parameter)
model = als.fit(trainValidDF)
#Prediction sur la DataFrame Test
testDF = model.transform(testDF).select("prediction","rating")
#Remove unpredicter row due to no-presence of user in the trai dataset
pred_without_naDF = predDF.na.drop()
# Calcul du RMSE
evaluator = RegressionEvaluator(metricName="rmse", labelCol="rating",
                            predictionCol="prediction")
rmse = evaluator.evaluate(pred_without_naDF)
print("Root-mean-square error for rank %d = "%best_rank + str(rmse))


I am working on the file "MovieReviewSentimentAnalysis/MovieReveiw/.ipynb_checkpoints/NLP_Movies-checkpoint.ipynb".
The first task is:
Again, similar to the Logistic Regression model, we can see which parameter did the best.
Can you write Python code for it?


I am working on the file "MovieReviewSentimentAnalysis/MovieReveiw/.ipynb_checkpoints/NLP_Movies-checkpoint.ipynb".
The first task is:
Great! Now it is time to go ahead and load our data in. For this, pandas is definitely the library of choice. If you want to follow along with a downloaded version of the notebook yourself, make sure you obtain the data from Kaggle. You will need a Kaggle account in order to access it.
Can you write Python code for it?

import pandas as pd

train = pd.read_csv('labeledTrainData.tsv', header=0,
                delimiter="\t", quoting=3)
test = pd.read_csv('testData.tsv', header=0, delimiter="\t",
               quoting=3 )

# Import both the training and test data.


I am working on the file "MovieReviewSentimentAnalysis/MovieReveiw/.ipynb_checkpoints/NLP_Movies-checkpoint.ipynb".
The first task is:
Looks like this beat our previous Logistic Regression model by a very small amount. Now that we have our three models, we can work on submitting our final scores in the proper format. It was found that submitting predicted probabilities of each score instead of the final predicted score worked better for evaluation from the contest participants, so we want to output this instead.
First, do our Logistic Regression submission.
Can you write Python code for it?

LR_result = model_LR.predict_proba(X_test)[:,1] # We only need the probabilities that the movie review was a 7 or greater.
LR_output = pd.DataFrame(data={"id":test["id"], "sentiment":LR_result}) # Create our dataframe that will be written.
LR_output.to_csv('Logistic_Reg_Proj2.csv', index=False, quoting=3) # Get the .csv file we will submit to Kaggle.


I am working on the file "MovieReviewSentimentAnalysis/MovieReveiw/.ipynb_checkpoints/NLP_Movies-checkpoint.ipynb".
The first task is:
Making Our Classifiers
Because we are working with text data, and we just made feature vectors of every word (that isn't a stop word of course) in all of the reviews, we are going to have sparse matrices to deal with that are quite large in size. Just to show you what I mean, let's examine the shape of our training set.
Can you write Python code for it?


I am working on the file "MovieReviewSentimentAnalysis/MovieReveiw/.ipynb_checkpoints/NLP_Movies-checkpoint.ipynb".
The first task is:
Now that we have the vectorization object, we need to run this on all of the data (both training and testing) to make sure it is applied to both datasets. This could take some time on your computer!
Can you write Python code for it?

X_all = traindata + testdata # Combine both to fit the TFIDF vectorization.
lentrain = len(traindata)

tfv.fit(X_all) # This is the slow part!
X_all = tfv.transform(X_all)

X = X_all[:lentrain] # Separate back into training and test sets.
X_test = X_all[lentrain:]


I am working on the file "MovieReviewSentimentAnalysis/MovieReveiw/.ipynb_checkpoints/NLP_Movies-checkpoint.ipynb".
The first task is:
Pretty fast, right? This speed comes at a price, however. Naive Bayes assumes all of your features are ENTIRELY independent from each other. In the case of word vectors, that seems like a somewhat reasonable assumption but with the ngrams we included that probably isn't always the case. Because of this, Naive Bayes tends to be less accurate than other classification algorithms, especially if you have a smaller number of training examples.
Why don't we see how Naive Bayes does (at least in a 20 fold CV comparison) so we have a rough idea of how well it performs compared to our Logistic Regression classifier?
You could use GridSearch again, but that seems like overkill. There is a simpler method we can import from scikit-learn for this task.
Can you write Python code for it?

from sklearn.cross_validation import cross_val_score
import numpy as np

print "20 Fold CV Score for Multinomial Naive Bayes: ", np.mean(cross_val_score
                                                                (model_NB, X, y_train, cv=20, scoring='roc_auc'))
     # This will give us a 20-fold cross validation score that looks at ROC_AUC so we can compare with Logistic Regression.


I am working on the file "MovieReviewSentimentAnalysis/MovieReveiw/.ipynb_checkpoints/NLP_Movies-checkpoint.ipynb".
The first task is:
Repeat this with the other two.
Can you write Python code for it?

# Repeat this for Multinomial Naive Bayes

MNB_result = model_NB.predict_proba(X_test)[:,1]
MNB_output = pd.DataFrame(data={"id":test["id"], "sentiment":MNB_result})
MNB_output.to_csv('MNB_Proj2.csv', index = False, quoting = 3)

# Last, do the Stochastic Gradient Descent model with modified Huber loss.

SGD_result = model_SGD.predict_proba(X_test)[:,1]
SGD_output = pd.DataFrame(data={"id":test["id"], "sentiment":SGD_result})
SGD_output.to_csv('SGD_Proj2.csv', index = False, quoting = 3)


I am working on the file "MovieReviewSentimentAnalysis/MovieReveiw/.ipynb_checkpoints/NLP_Movies-checkpoint.ipynb".
The first task is:
TF-IDF Vectorization
The next thing we are going to do is make TF-IDF (term frequency-interdocument frequency) vectors of our reviews. In case you are not familiar with what this is doing, essentially we are going to evaluate how often a certain term occurs in a review, but normalize this somewhat by how many reviews a certain term also occurs in. Wikipedia has an explanation that is sufficient if you want further information.
This can be a great technique for helping to determine which words (or ngrams of words) will make good features to classify a review as positive or negative.
To do this, we are going to use the TFIDF vectorizer from scikit-learn. Then, decide what settings to use. The documentation for the TFIDF class is available here.
In the case of the example code on Kaggle, they decided to remove all stop words, along with ngrams up to a size of two (you could use more but this will require a LOT of memory, so be careful which settings you use!)
Can you write Python code for it?

from sklearn.feature_extraction.text import TfidfVectorizer as TFIV

tfv = TFIV(min_df=3,  max_features=None,
        strip_accents='unicode', analyzer='word',token_pattern=r'\w{1,}',
        ngram_range=(1, 2), use_idf=1,smooth_idf=1,sublinear_tf=1,
        stop_words = 'english')


I am working on the file "MovieReviewSentimentAnalysis/MovieReveiw/.ipynb_checkpoints/NLP_Movies-checkpoint.ipynb".
The first task is:
That means we have 25,000 training examples (or rows) and 309,798 features (or columns). We need something that is going to be somewhat computationally efficient given how many features we have. Using something like a random forest to classify would be unwieldy (plus random forests can't work with sparse matrices anyway yet in scikit-learn). That means we need something lightweight and fast that scales to many dimensions well. Some possible candidates are:

Naive Bayes
Logistic Regression
SGD Classifier (utilizes Stochastic Gradient Descent for much faster runtime)

Let's just try all three as submissions to Kaggle and see how they perform.
First up: Logistic Regression (see the scikit-learn documentation here).
While in theory L1 regularization should work well because p>>n (many more features than training examples), I actually found through a lot of testing that L2 regularization got better results. You could set up your own trials using scikit-learn's built-in GridSearch class, which makes things a lot easier to try. I found through my testing that using a parameter C of 30 got the best results.
Can you write Python code for it?

from sklearn.linear_model import LogisticRegression as LR
from sklearn.grid_search import GridSearchCV

grid_values = {'C':[30]} # Decide which settings you want for the grid search.

model_LR = GridSearchCV(LR(penalty = 'L2', dual = True, random_state = 0),
                        grid_values, scoring = 'roc_auc', cv = 20)
# Try to set the scoring on what the contest is asking for.
# The contest says scoring is for area under the ROC curve, so use this.

model_LR.fit(X,y_train) # Fit the model.