I am working on the file "First-Tutorial-Start-Here.ipynb".
The first task is:
Data Dependencies
Futures can be passed between Apps. When a future created by one App is passed as an input to another, an implicit data dependency is created. Parsl will manage the execution of these Apps by ensuring they are executed when dependencies are resolved.
Let's see an example of this using the monte-carlo method to calculate pi. We call 3 iterations of this slow function, and take the average. The dependency chain looks like this :
App Calls    pi()  pi()   pi()
              \     |     /
Futures        a    b    c
                \   |   /
App Call        avg_three()
                    |
Future            avg_pi
Can you write Python code for it?

@App('python', dfk)
def pi(total):
    # App functions have to import modules they will use.
    import random
    # Set the size of the box (edge length) in which we drop random points
    edge_length = 10000
    center = edge_length / 2
    c2  = center ** 2
    count = 0

    for i in range(total):
        # Drop a random point in the box.
        x,y = random.randint(1, edge_length),random.randint(1, edge_length)
        # Count points within the circle
        if (x-center)**2 + (y-center)**2 < c2:
            count += 1

    return (count*4/total)

@App('python', dfk)
def avg_three(a,b,c):
    return (a+b+c)/3


I am working on the file "First-Tutorial-Start-Here.ipynb".
The first task is:
DataFlowKernal
Parsl's DataFlowKernel acts as an abstraction layer over any pool of execution resources (e.g., clusters, clouds, threads).
In this example we use a pool of [threads](https://en.wikipedia.org/wiki/Thread_(computing). to facilitate local parallel exectuion.
Can you write Python code for it?

# Let's create a pool of threads to execute our functions
workers = ThreadPoolExecutor(max_workers=4)

# We pass the workers to the DataFlowKernel which will execute our Apps over the workers.
dfk = DataFlowKernel(executors=[workers])


I am working on the file "First-Tutorial-Start-Here.ipynb".
The first task is:
Futures
Unlike a regular Python function, when an App is called it returns an AppFuture. Futures act as a proxy to the results (or exceptions) that the App will produce once its execution completes. You can retrieve the status of a future object with future.done() or you can ask it to wait for its result with future.result(). It is important to note that while the done() call provides the current status, the result() call blocks execution until the App is complete and the result is available.
Can you write Python code for it?

# Check status
print("Status: ", app_future.done())

# Get result
print("Result: ", app_future.result())


I am working on the file "First-Tutorial-Start-Here.ipynb".
The first task is:
Parallelism
Here we call the function pi() three times, each of which run independently in parallel.
We then call another App avg_three() with the three futures that were returned from the pi() calls.
Since avg_three() is also a parsl App, it returns a future immediately, but defers execution (blocks) until all the futures passed to it as inputs have been resolved.
Can you write Python code for it?

a, b, c = pi(10**6), pi(10**6), pi(10**6)
avg_pi  = avg_three(a, b, c)

# Print the results
print("A: {0:.5f} B: {1:.5f} C: {2:.5f}".format(a.result(), b.result(), c.result()))
print("Average: {0:.5f}".format(avg_pi.result()))


I am working on the file "FirstLast_Sympy.ipynb".
The first task is:
$$ \Large {\displaystyle f(x)=3e^{-{\frac {x^{2}}{8}}}} \sin(x/3)$$

Find the first four terms of the Taylor expansion of the above equation
Make a plot of the function
Plot size 10 in x 4 in
X limts -5, 5
Y limits -2, 2
Over-plot the 1-term Taylor expansion using a different color
Over-plot the 2-term Taylor expansion using a different color
Over-plot the 3-term Taylor expansion using a different color
Over-plot the 4-term Taylor expansion using a different color
Can you write Python code for it?

sp.init_printing()

x = sp.symbols('x')

my_x = np.linspace(-10,10,100)


I am working on the file "FirstSteps.ipynb".
The first task is:
Access elements as you would in R -- matrix[row_index, col_index] -- except that counting starts at 0.
Can you write Python code for it?


I am working on the file "FirstSteps.ipynb".
The first task is:
Adding two matrices
Create a function to add two matrices
Can you write Python code for it?

x = T.dmatrix("x")
y = T.dmatrix("y")
z = x + y
f = function([x, y], z)


I am working on the file "FirstSteps.ipynb".
The first task is:
Exercise
Modify this boilerplate code to compute a ** 2 + b ** 2 + 2 * a * b.
python
import theano
a = theano.tensor.vector()      # declare variable
out = a + a ** 10               # build symbolic expression
f = theano.function([a], out)   # compile function
Can you write Python code for it?

a = T.vector()
b = T.vector()
z = a ** 2 + b ** 2 + 2 * a * b
f = function([a, b], z)
f(range(0, 5), range(5, 10))


I am working on the file "FirstSteps.ipynb".
The first task is:
In the example above, the scalar 2.0 is stretched into a 3-element array. This example is just like c(1, 2, 3) * 2 in R.
Can you write Python code for it?


I am working on the file "FirstSteps.ipynb".
The first task is:
NumPy Refresher
Matrix rows are examples/observations. Matrix columns are features/dimensions of the example.
Can you write Python code for it?

import numpy as np
my_matrix = np.asarray([[1., 2], [3, 4], [5, 6]])
my_matrix


I am working on the file "FirstSteps.ipynb".
The first task is:
Theano can work 2D arrays or numpy arrays.
Can you write Python code for it?


I am working on the file "FirstSteps.ipynb".
The first task is:
Theano types are not the same as Python classes, although unfortunately we get an object's class by calling the type function.
Can you write Python code for it?

# the class of x
type(x)

# the theano type of x
x.type


I am working on the file "FirstSteps.ipynb".
The first task is:
When we print z, we see that it describes an operation.
Can you write Python code for it?

z

# Use pp to pretty print the computation associated to z
from theano import pp
pp(z)


I am working on the file "Fisher's_Iris.ipynb".
The first task is:
Fit a line
Fit a straight line to the variables petal length and petal width for the whole data set. Plot the data points in a scatter plot with the best fit line shown
straight lines, pick a model:  $$ y = m * x + c $$
calculating the cost:  $$ \sum_i (y_i - mx_i - c)^2 $$
cost = lambda m,c: np.sum([(petal_width[i] - m * petal_length[i] - c)**2 for i in range(petal_width.size)])
minimising the cost:
$$ Cost(m, c) = \sum_i (y_i - mx_i - c)^2 $$
Can you write Python code for it?

# here, just use the numpy.polyfit function to get the best values for m and c.
petal_length = np.array(petal_length).astype(np.float)
petal_width = np.array(petal_width).astype(np.float)
m,c=np.polyfit(petal_length, petal_width, 1)
print("m is %8.6f and c is %6.6f." % (m, c))


I am working on the file "Fisher's_Iris.ipynb".
The first task is:
Fit another line
Use numpy to select only the data points where species is setosa. Fit a straight line to the variables petal length and petal width. Plot the data points in a scatter plot with the best fit line shown.
Can you write Python code for it?

# get a copy of data
data_ = np.array(data)

# get setosa data set
# adapt from: https://stackoverflow.com/questions/23359886/selecting-rows-in-numpy-ndarray-based-on-the-value-of-two-columns
setosa = []
setosa = data_[np.where(data_[:,4] == 'setosa')]
setosa_petal_length = np.array(setosa[:,2]).astype(np.float)
setosa_petal_width = np.array(setosa[:,3]).astype(np.float)

# pick model y = m * x + c
# get best m & c
m,c=np.polyfit(setosa_petal_length, setosa_petal_width, 1)
print("m is %8.6f and c is %6.6f." % (m, c))

# Create the plot.
pl.plot(setosa_petal_length, m * setosa_petal_length + c, 'b-')
pl.plot(setosa_petal_length, setosa_petal_width, 'k.')
pl.xlabel('setosa_petal_length')
pl.ylabel('setosa_petal_width')
pl.show()


I am working on the file "Fisher's_Iris.ipynb".
The first task is:
Iterate
run our gradient descent algorithm. For  mm , we keep replacing its value with  m−ηgrad_m(x,y,m,c)m−ηgrad_m(x,y,m,c)  until it doesn't change. For  cc , we keep replacing its value with  c−ηgrad_c(x,y,m,c)c−ηgrad_c(x,y,m,c)  until it doesn't change.
Can you write Python code for it?

eta = 0.0001
m, c = 1.0, 1.0
change = True

while change:
  mnew = m - eta * grad_m(setosa_petal_length, setosa_petal_width, m, c)
  cnew = c - eta * grad_c(setosa_petal_length, setosa_petal_width, m, c)
  if m == mnew and c == cnew:
    change = False
  else:
    m, c = mnew, cnew
print("m: %20.16f  c: %20.16f" % (m, c))


I am working on the file "Fisher's_Iris.ipynb".
The first task is:
Use gradient descent
Use gradient descent to approximate the best fit line for the petal length and petal width setosa values. Compare the outputs to your calculations above.
gradient descent: In gradient descent, we select a random guess of a parameter and iteratively improve that guess.
For instance, we might pick $1.0$ as our initial guess for $m$ and then create a for loop to iteratively improve the value of $m$.
The way we improve $m$ is to first take the partial derivative of our cost function with respect to $m$.
Calculate the partial derivatives
formula:
$$
\begin{align}
Cost(m, c)                       &= \sum_i (y_i - mx_i - c)^2      \[1cm]
\frac{\partial Cost}{\partial m} &= \sum 2(y_i - m x_i -c)(-x_i) \
                                 &= -2 \sum x_i (y_i - m x_i -c) \[0.5cm]
\frac{\partial Cost}{\partial c} & = \sum 2(y_i - m x_i -c)(-1) \
                                 & = -2 \sum (y_i - m x_i -c)   \
\end{align}
$$
Code the partial derivatives
Can you write Python code for it?

def grad_m(x, y, m, c):
  return -2.0 * np.sum(x * (y - m * x - c))

def grad_c(x, y, m , c):
  return -2.0 * np.sum(y - m * x - c)


I am working on the file "Fisher's_Iris.ipynb".
The first task is:
Use seaborn
Use the seaborn library to create a scatterplot matrix of all five variables.
Can you write Python code for it?

# use pandas reload iris.csv
import pandas
iris = pandas.read_csv('iris.csv')

# use seaborn library show plots
import seaborn
seaborn.pairplot(iris, hue="species")
pl.show()


I am working on the file "Fisher's_Iris.ipynb".
The first task is:
create a simple plot
Create a scatter plot of sepal length on the x-axis versus sepal width on the y-axis.
Can you write Python code for it?

# Create the plot.
pl.scatter(sepal_length, sepal_width, c='black')

# Set some properties for the plot.
pl.xlabel('sepal_length')
pl.ylabel('sepal_width')

# Show the plot.
pl.title('relationship between sepal Length and width')
pl.show()


I am working on the file "Fisher's_Iris.ipynb".
The first task is:
plot the data points in a scatter plot with the best fit line shown.
Can you write Python code for it?

# Create the plot.
pl.plot(petal_length, m * petal_length + c, 'b-')
pl.plot(petal_length, petal_width, 'k.')

# Set some properties for the plot.
pl.xlabel('petal_length')
pl.ylabel('petal_width')

# Show the plot.
pl.title('relationship between petal Length and width')
pl.show()


I am working on the file "FitNeuralNets/testFitNeuralNet.ipynb".
The first task is:
Test how well the model is performing
Can you write Python code for it?

fig = plt.figure()

num_image_rows = np.ceil(np.sqrt(testData.shape[0]))
for i in range(0, testData.shape[0]):
  a = fig.add_subplot(num_image_rows/2, num_image_rows*2, i+1)
  a.set_title(i)
  testplot = plt.imshow(testData[i].reshape(28,28), cmap ="gray")
  plt.axis('off')

result = testNN.forwardCompute(np.reshape(testData[60], (28*28,1)))
letterIndex = np.argmax(result)
print letterIndex
print labelDict.keys()[labelDict.values().index(letterIndex)]


I am working on the file "FitNeuralNets/testFitNeuralNet.ipynb".
The first task is:
Utilities for working with a standard neural net
Can you write Python code for it?

#---------------------------------------------------
# A neural net class
#---------------------------------------------------
class NeuralNet(object):

  """ Constructor
  #   layers: vector of length numLayers containing the
  #          the number of neurons in each layer
  #           e.g., layers = (4, 3, 2) -> 4 input values, 3 neurons in hidden layer, 2 output values
  #   biases: initialized with random numbers except for first layer
  #           e.g., biases = [ [b11, b21, b31]^T,
  #                            [b12, b22]^T ]
  #   weights: initialized with random numbers
  #           e.g., [ [[w111, w121, w131, w141],
  #                    [w211, w221, w231, w241],
  #                    [w311, w321, w331, w341]],
  #                   [[w112, w122, w132],
  #                    [w212, w222, w232]] ]
  """
  def __init__(self, layers):

    self.numLayers = len(layers)
    self.numNeurons = layers
    self.biases = [random.randn(layer, 1) for layer in layers[1:]]
    self.weights = [random.randn(layer2, layer1)
                    for layer1, layer2 in zip(layers[:-1], layers[1:])]

  """ Batch stochastic gradient descent to find minimum of objective function
  #   training_data:  [(x1,y1),(x2,y2),....]
  #                   where x1, x2, x3, ... are input data vectors
  #                         y1, y2, y3, ... are labels
  #   max_iterations: number of iterations
  #   batch_size:     size of training batch
  #   learning_rate:  gradient descent parameter
  """
  def batchStochasticGradientDescent(self, training_data, max_iterations, batch_size,
                                     learning_rate):

    # Get the number of training images
    nTrain = len(training_data)

    # Loop thru iterations
    for it in xrange(max_iterations):

      # Shuffle the training data
      random.shuffle(training_data)

      # Choose subsets of the training data
      batches = [ training_data[start:start+batch_size]
                  for start in xrange(0, nTrain, batch_size) ]

      # Loop thru...


I am working on the file "FitNeuralNets/testFitNeuralNet.ipynb".
The first task is:
Assign labels to the images in the training set
Can you write Python code for it?

fileLabels = [labelDict[letter] for letter in fileLetters]
print fileLabels


I am working on the file "FitNeuralNets/testFitNeuralNet.ipynb".
The first task is:
Convert white into black background
Can you write Python code for it?

for dt in np.nditer(data, op_flags=['readwrite']):
    dt[...] = dt/255.0
    if (dt == 1.0):
        dt[...] = 0.0


I am working on the file "FitNeuralNets/testFitNeuralNet.ipynb".
The first task is:
Load the test data, Convert white into black background, Reshape the data, and Plot a sample
Can you write Python code for it?

testDirName = "../lettersketch/assets/test_images/"
testNames = []
for fileName in os.listdir(testDirName):
    if fileName.endswith(".png") and ("__" in fileName):
       testNames.append(testDirName+fileName)

testData = np.array([np.asarray(Image.open(fileName).convert('L').resize((28, 28), Image.NEAREST))
                 for fileName in testNames], dtype = np.float64)
print testData.shape

testData = testData.reshape(testData.shape[0], testData.shape[1]*testData.shape[2])
print testData.shape

for dt in np.nditer(testData, op_flags=['readwrite']):
    dt[...] = dt/255.0
    if (dt == 1.0):
        dt[...] = 0.0

imgplot = plt.imshow(data[9].reshape(28,28), cmap="gray")

testplot = plt.imshow(testData[1].reshape(28,28), cmap ="gray")


I am working on the file "FitNeuralNets/testFitNeuralNet.ipynb".
The first task is:
Read in the letters saved by the lettersketch app after converting into grayscale and resizing
Can you write Python code for it?

#def rgb2gray(rgb):
#    return np.dot(rgb[...,:3], [0.299, 0.587, 0.144])
#
#data = np.array([rgb2gray(mpimg.imread(fileName)) for fileName in fileNames], dtype = np.float64)
#data.shape

data = np.array([np.asarray(Image.open(fileName).convert('L').resize((28, 28), Image.NEAREST))
                 for fileName in fileNames], dtype = np.float64)
print data.shape


I am working on the file "FitNeuralNets/testFitNeuralNet.ipynb".
The first task is:
Save the weights and biases in JSON
Can you write Python code for it?

weightsList = testNN.weights
for weights in weightsList:
    print weights.shape

biasesList = testNN.biases
for biases in biasesList:
    print biases.shape

weightsFileName = "../lettersketch/assets/json/UpperCase_StraightLines_weights.json"
biasesFileName = "../lettersketch/assets/json/UpperCase_StraightLines_biases.json"
for weights in weightsList:
  json.dump(weights.tolist(), codecs.open(weightsFileName, 'a', encoding='utf-8'),
    separators=(',', ':'), sort_keys=True, indent=2)

for biases in biasesList:
  json.dump(biases.tolist(), codecs.open(biasesFileName, 'a', encoding='utf-8'),
    separators=(',', ':'), sort_keys=True, indent=2)


I am working on the file "Fitting a model with outliers using MCMC.ipynb".
The first task is:

Can you write Python code for it?

# pack the true parameters into a single tuple
true_params = (true_amplitude, true_period, true_phase, true_mean_mag)

# generate an array of observation times
time = np.random.uniform(0., 365., size=ndata_points)
time.sort()

# generate magnitude values from the model at the observation times
mag = light_curve_model(true_params, time)

# each data point will have a different uncertainty, sampled from
#    a uniform distribution between 0.2 and 0.4 magnitudes
mag_err = np.random.uniform(0.2, 0.4, size=ndata_points)

# pick outlier points based on the set outlier_fraction. we generate a
#    boolean array (array of True's and False's) -- when a given index
#    is True, that point will become an outlier
outlier_idx = np.random.uniform(size=ndata_points) < outlier_fraction

# for the outlier points, add large scatter
mag[outlier_idx] += np.random.normal(0., 5., size=sum(outlier_idx))

# for the non-outlier points, add scatter based on the uncertainty array (mag_err).
#    the twiddle (~) means 'logical not' - (True becomes False, False becomes True)
mag[~outlier_idx] += np.random.normal(0., mag_err[~outlier_idx])

plt.figure(figsize=(12,4))
plt.errorbar(time, mag, mag_err, marker='o',
             linestyle='none', ecolor='#aaaaaa')
plt.xlim(0,365)
plt.xlabel("Time [day]")
plt.ylabel("Magnitude")


I am working on the file "Fitting a model with outliers using MCMC.ipynb".
The first task is:
Fitting a model to data with outliers using MCMC
We are often faced with data with spurious outliers. For example, a light curve generated from some automated photometric pipeline with failures in the background subtraction on some nights. It is tempting to remove these outliers by eye, or by some automatic procedure (e.g., sigma clipping), however these points may actually contain valuable information. Your model should be able to account for these data! Luckily, it is relatively easy to account for this within a statistical model: the data are modeled as being either drawn from your physical model, or an outlier model with an extra parameter to specify the global weights of these two distributions.
Below we consider a trivial but illustrative example. This problem is described in a little more detail in Section 5.6.7 of Statistics, Data Mining, and Machine Learning in Astronomy. Another good reference is David Hogg's Fitting a model to data paper.

Example data
We take images of a variable star at random times over the course of a year. We have some automated pipeline that reduces the data, and spits out magnitudes for each observation. Some nights, the pipeline fails, but the way in which we estimate uncertainties on the measured magnitudes doesn't account for these failures, leading to outliers in the final data product (a light curve). For now, we'll assume the shape of the light curve is a perfect sinusoid, but in principle any template could be used. Our goal will be to measure the amplitude, period, and phase of the sinusoid given noisy observations of the star.
We start by definining some helper functions to generate the fake data:
Can you write Python code for it?

def sinusoid(t, amp, period, phase):
    """ A generic sinusoidal curve. 'period' and 't' should have
        the same units (e.g., days), and phase should be in
        radians.

        Parameters
        ----------
        t : array_like
            Array of times.
        amp : numeric
            Amplitude of the sinusoid.
        period : numeric
            Period of the sinusoid.
        phase : numeric
            Phase of the sinusoid.
    """
    return amp*np.sin(2*np.pi*t/period + phase)

def light_curve_model(p, t):
    """ Our model for the variable star light curve will be a
        pure sinusoid plus some constant offset (the mean
        magnitude of the star). The function takes a single array
        of parameters, p, and an array of times, t. By structuring
        the function input parameters this way, we can use this
        function to both generate and later fit the data.

        Parameters
        ----------
        p : iterable
            A list, tuple, or array of model parameter values. For
            example, a tuple of (amplitude, period, phase, mean mag.).
        t : array_like
            Array of times.
    """
    amp, period, phase, const = p
    return sinusoid(t, amp, period, phase) + const


I am working on the file "Fitting a model with outliers using MCMC.ipynb".
The first task is:
Now we're armed with some fake data in the form of 3 arrays: the observation times, magnitudes, and uncertainties for the magnitudes (we'll assume we can measure the time of observation to arbitrary precision). We'll start by trying to fit our light curve model without taking into account the outliers. We're going to use the same model that we used to generate the data, but ignore the fact that there are outlier points.
If we assume that each data point is independent and has Gaussian uncertainties (as we have done by construction), we can write down a likelihood for each $i$ data point as a Gaussian:
$$
p(m_i \,\vert\, A,T,\phi,m_0,\sigma_{m,i},t_i) = \frac{1}{\sqrt{2\pi\sigma_{m,i}^2}} \exp \left(-\frac{(m_i - f(t_i, A, T, \phi, m_0))^2}{2\sigma_{m,i}^2}\right)
$$
where our light curve model is expressed as the function $f$:
$$
f(t_i, A, T, \phi, m_0) = A\sin(\frac{2\pi t_i}{T}+\phi) - m_0
$$
This looks just like a $\chi^2$ function because we are assuming our data are drawn from a Gaussian around some true model with the parameters amplitude, $A$, period, $T$, phase, $\phi$, and mean magnitude, $m_0$. The full likelihood (over all data points) is then the product of these of the individual probabilities:
$$
\mathcal{L} = \prod_i^N p(m_i \,\vert\, A,T,\phi,m_0,\sigma_{m,i},t_i) = p({m_i}N \,\vert\, A,T,\phi,m_0,{\sigma{m,i}}_N,{t_i}_N)
$$
where the notation ${m_i}_N$ means the set of all $N$ data points.
For this example, we will assume uniform priors over all parameters, $p(A)\,p(T)\,p(\phi)\,p(m_0)$. It is a bad idea to define uniform priors over angular parameters (like the phase, $\phi$, in this example), but I've done it anyway for simplicity. In this case, we could just maximize this likelihood to find the "best fit" parameters, but instead we will use MCMC to sample from the posterior probability over the model parameters. The full posterior probability for our model is then given by:
$$
p(A,T,\phi,m_0 \,\vert\, {m_i}N, {\sigma{m,i}}N, {t_i}_N) = \frac{1}{\mathcal{Z}} p({m_i}_N \,\vert\,...
Can you write Python code for it?

def lnprior(p):
    amp,period,phase,const = p

    if amp < 1 or amp > 2:
        return -np.inf

    if period < 10 or period > 200:
        return -np.inf

    if phase < 0. or phase > 2*np.pi:
        return -np.inf

    if const < 12 or const > 16:
        return -np.inf

    return 0.

def lnlikelihood(p, t, data, err):
    amp,period,phase,const = p
    return -np.log(err) - 0.5*( (data - sinusoid(t, amp, period, phase) - const) / err )**2

def lnprob(p, t, data, err):
    return lnprior(p) + np.sum(lnlikelihood(p, t, data, err))

ndim, nwalkers = 4, 32
p0 = np.zeros((nwalkers,ndim))
p0[:,0] = np.random.uniform(1, 2., size=nwalkers) # amp
p0[:,1] = np.random.uniform(10, 200., size=nwalkers) # period (days)
p0[:,2] = np.random.uniform(0., 2*np.pi, size=nwalkers) # phase (radians)
p0[:,3] = np.random.uniform(12., 16., size=nwalkers) # const. offset (mag)

sampler = emcee.EnsembleSampler(nwalkers, ndim, lnprob,
                                args=(time,mag,mag_err))

pos, prob, state = sampler.run_mcmc(p0, 1000)

best_pos = sampler.flatchain[sampler.flatlnprobability.argmax()]
pos = emcee.utils.sample_ball(best_pos, best_pos/100., size=nwalkers)
sampler.reset()
pos, prob, state = sampler.run_mcmc(pos, 100)

sampler.reset()
pos, prob, state = sampler.run_mcmc(pos, 1000)

param_labels = ["Amp.", "Period [day]", "Phase [rad]", "Mean mag."]
extents = [(0.5*truth,1.5*truth) for truth in true_params]
fig = triangle.corner(sampler.flatchain,
                      labels=param_labels,
                      truths=true_params,
                      range=extents)


I am working on the file "Fitting a model with outliers using MCMC.ipynb".
The first task is:
What can we do about this?
We can define a mixture model! We add a new parameter to the model: the fraction of stars that are instead drawn from some very broad outlier distribution. There are many possible options for that to put in for this outlier distribution, but below we will assume that the outlier distribution is still centered on the model, but with a huge variance. Our "inlier" distribution will be the same as above:
$$
p_{\rm in} = p(m_i \,\vert\, A,T,\phi,m_0,\sigma_{m,i},t_i) = \frac{1}{\sqrt{2\pi\sigma_{m,i}^2}} \exp \left(-\frac{(m_i - f(t_i, A, T, \phi, m_0))^2}{2\sigma_{m,i}^2}\right)
$$
and our outlier distribution:
$$
p_{\rm out} = p(m_i \,\vert\, A,T,\phi,m_0,V,t_i) = \frac{1}{\sqrt{2\pi V}} \exp \left(-\frac{(m_i - f(t_i, A, T, \phi, m_0))^2}{2 V}\right)
$$
where $V$ is arbitrarily fixed to be 10 times the median uncertainty of the data. The new likelihood for a single point is then:
$$
\mathcal{L}i = (1-f{\rm bad})p_{\rm in} + f_{\rm bad}p_{\rm out}
$$
$f_{\rm out}$ represents the fraction of the data points that are outliers, and this will be a new parameter in the model.
Can you write Python code for it?

def lnprior(p):
    amp = p[0]
    period = p[1]
    phase = p[2]
    const = p[3]
    fout = p[4]

    if amp < 1 or amp > 2:
        return -np.inf

    if period < 100 or period > 200:
        return -np.inf

    if phase < 0. or phase > 2*np.pi:
        return -np.inf

    if const < 12 or const > 16:
        return -np.inf

    if fout > 1. or fout < 0.:
        return -np.inf

    return 0.

def ln_model_likelihood(p, t, data, err):
    amp, period, phase, const, outlier_prob = p
    term = -np.log(err) - 0.5*( (data - sinusoid(t, amp, period, phase) - const) / err )**2
    return term

def ln_outlier_likelihood(p, t, data, err):
    amp, period, phase, const, outlier_prob = p
    outlier_err = 10.*np.median(err)
    term = -np.log(outlier_err) - 0.5*( (data - sinusoid(t, amp, period, phase) - const) / outlier_err )**2
    return term

def lnlikelihood(p, t, data, err):
    amp, period, phase, const, fout = p

    term1 = ln_model_likelihood(p, t, data, err)
    term2 = ln_outlier_likelihood(p, t, data, err)

    b = np.ones((2,len(t)))
    b[0] = 1. - fout
    b[1] = fout

    return logsumexp(np.vstack((term1, term2)), b=b, axis=0)

def lnprob(p, t, data, err):
    prior = lnprior(p)
    if np.isinf(prior):
        return -np.inf

    return prior + np.sum(lnlikelihood(p, t, data, err))

ndim, nwalkers = 5, 64
p0 = np.zeros((nwalkers,ndim))
p0[:,0] = np.random.uniform(1, 2., size=nwalkers) # amp
p0[:,1] = np.random.uniform(100, 200., size=nwalkers) # period (days)
p0[:,2] = np.random.uniform(0., 2*np.pi, size=nwalkers) # phase (radians)
p0[:,3] = np.random.uniform(12., 16., size=nwalkers) # const. offset (mag)
p0[:,4] = np.random.normal(0.5, 0.05, size=(nwalkers)) # outlier probabilty

sampler = emcee.EnsembleSampler(nwalkers, ndim, lnprob,
                                args=(time,mag,mag_err))

pos, prob, state = sampler.run_mcmc(p0, 1000)

best_pos = sampler.flatlnprobability.argmax()
new_pos =...


I am working on the file "Fitting.ipynb".
The first task is:
In the cells below, fit 2nd and 4th degree polynomials and overplot on the data
Can you write Python code for it?

snmod_wls_2nd = smf.wls("mu ~ z + np.power(z,2)", data=sndata, weights=dmu**-2)


I am working on the file "Fitting.ipynb".
The first task is:
Overplot the straight line fit
Can you write Python code for it?

fig = plt.figure(figsize=(5, 5))
plt.errorbar(z_sample, mu_sample, dmu, fmt='.k', ecolor='gray', lw=1)
plt.plot(z_sample, snres_wls.fittedvalues, 'b-', label="Weighted LS")
plt.xlabel(r'$z$')
plt.ylabel(r'$\mu$')


I am working on the file "Fitting.ipynb".
The first task is:
Put the data into a Pandas dataframe
Can you write Python code for it?

sndata = pd.DataFrame({"z": z_sample, "mu": mu_sample, "mu_err": dmu})

fig = plt.figure(figsize=(5, 5))
plt.errorbar(z_sample, mu_sample, dmu, fmt='.k', ecolor='gray', lw=1)
plt.xlabel(r'$z$')
plt.ylabel(r'$\mu$')


I am working on the file "Fitting.ipynb".
The first task is:
The file SDSS_QSO.dat is from Feigelson's Astrostatistics school, September 2014 at Caltech
Can you write Python code for it?

df = pd.read_csv('data/SDSS_QSO.dat', delim_whitespace=True)

df.head()

df.describe()

# Keep only rows with positive magnitudes, and 18 < i < 22

df = df[(df.u_mag > 10) & (df.i_mag > 18) & (df.i_mag < 22) ]

# Set a limit on u magnitude errors
df.loc[df.sig_u_mag<0.02,'sig_u_mag'] = 0.02
df.sig_u_mag.min()

mod_ols = smf.ols('u_mag ~ i_mag', data=df)
res_ols = mod_ols.fit()

print(res_ols.summary())

print('Parameters: ', res_ols.params)
print('R2: ', res_ols.rsquared)
print('AIC', res_ols.aic)


I am working on the file "Fitting.ipynb".
The first task is:
Weighted least-squares
Can you write Python code for it?

mod_wls = smf.wls('u_mag ~ i_mag', data=df, weights=df.sig_u_mag**-2)

res_wls = mod_wls.fit()

print(res_wls.summary())

i_range = np.arange(18,22,0.2)

p_ols = res_ols.predict({"i_mag": i_range})
p_ols

%matplotlib inline
prstd, iv_l, iv_u = wls_prediction_std(res_wls,sm.add_constant(i_range), weights=1/(0.02*0.02))

fig, ax = plt.subplots(figsize=(8,6))

ax.scatter(df.i_mag, df.u_mag, marker='.', c='k', s=0.5, label="data", alpha=0.5)
ax.plot(i_range, res_wls.predict({"i_mag":i_range}), 'b-', label="Weighted LS")
ax.plot(i_range, res_ols.predict({"i_mag":i_range}), 'g--.', label="Ordinary LS")
ax.plot(i_range, iv_u, 'r--', label='95% confidence')
ax.plot(i_range, iv_l, 'r--')
ax.legend(loc='best')
plt.xlabel('SDSS i (mag)')
plt.ylabel('SDSS u (mag)')
plt.xlim(17.9, 21.6)


I am working on the file "Fitting/.ipynb_checkpoints/FittingModelsEx02-checkpoint.ipynb".
The first task is:
Fitting a decaying oscillation
For this problem you are given a raw dataset in the file decay_osc.npz. This file contains three arrays:

tdata: an array of time values
ydata: an array of y values
dy: the absolute uncertainties (standard deviations) in y

Your job is to fit the following model to this data:
$$ y(t) = A e^{-\lambda t} \cos{\omega t + \delta} $$
First, import the data using NumPy and make an appropriately styled error bar plot of the raw data.
Can you write Python code for it?

data = np.load('decay_osc.npz')

tdata = data['tdata']
ydata = data['ydata']
dy = data['dy']

plt.figure(figsize=(8,6))
plt.plot(tdata, ydata)
plt.xlabel('Time');
plt.ylabel('Y Data');
plt.title('Oscillation Data');
plt.tick_params(axis='x',top='off',direction='out');
plt.tick_params(axis='y',right='off',direction='out');
plt.errorbar(tdata, ydata, dy,fmt='.k', ecolor='lightgray');

assert True # leave this to grade the data import and raw data plot


I am working on the file "Fitting/FittingModels.ipynb".
The first task is:
Again, display the optimal values of $b$ and $m$ along with their uncertainties:
Can you write Python code for it?

print('b = {0:.3f} +/- {1:.3f}'.format(theta_best[0], np.sqrt(theta_cov[0,0])))
print('m = {0:.3f} +/- {1:.3f}'.format(theta_best[1], np.sqrt(theta_cov[1,1])))


I am working on the file "Fitting/FittingModels.ipynb".
The first task is:
Fitting by hand
It is useful to see visually how changing the model parameters changes the value of $\chi^2$. By using IPython's interact function, we can create a user interface that allows us to pick a slope and intercept interactively and see the resulting line and $\chi^2$ value.
Here is the function we want to minimize. Note how we have combined the two parameters into a single parameters vector $\theta = [m, b]$, which is the first argument of the function:
Can you write Python code for it?

def chi2(theta, x, y, dy):
    # theta = [b, m]
    return np.sum(((y - theta[0] - theta[1] * x) / dy) ** 2)

def manual_fit(b, m):
    modely = m*xdata + b
    plt.plot(xdata, modely)
    plt.errorbar(xdata, ydata, dy,fmt='.k', ecolor='lightgray')
    plt.xlabel('x')
    plt.ylabel('y')
    plt.text(1, 15, 'b={0:.2f}'.format(b))
    plt.text(1, 12.5, 'm={0:.2f}'.format(m))
    plt.text(1, 10.0, '$\chi^2$={0:.2f}'.format(chi2([b,m],xdata,ydata, dy)))

interact(manual_fit, b=(-3.0,3.0,0.01), m=(0.0,4.0,0.01));


I am working on the file "Fitting/FittingModels.ipynb".
The first task is:
Fitting using scipy.optimize.curve_fit
SciPy also provides a general curve fitting function, curve_fit, that can handle both linear and non-linear models. This function:

Allows you to directly specify the model as a function, rather than the cost function (it assumes $\chi^2$).
Returns the covariance matrix for the parameters that provides estimates of the errors in each of the parameters.

Let's apply curve_fit to the above data. First we define a model function. The first argument should be the independent variable of the model.
Can you write Python code for it?


I am working on the file "Fitting/FittingModels.ipynb".
The first task is:
Here are the values of $b$ and $m$ that minimize $\chi^2$:
Can you write Python code for it?


I am working on the file "Fitting/FittingModels.ipynb".
The first task is:
Here we have passed the full_output=True option. When this is passed the covariance matrix $\Sigma_{ij}$ of the model parameters is also returned. The uncertainties (as standard deviations) in the parameters are the square roots of the diagonal elements of the covariance matrix:
$$ \sigma_i = \sqrt{\Sigma_{ii}} $$
A proof of this is beyond the scope of the current notebook.
Can you write Python code for it?

theta_best = result[0]
theta_cov = result[1]
print('b = {0:.3f} +/- {1:.3f}'.format(theta_best[0], np.sqrt(theta_cov[0,0])))
print('m = {0:.3f} +/- {1:.3f}'.format(theta_best[1], np.sqrt(theta_cov[1,1])))


I am working on the file "Fitting/FittingModels.ipynb".
The first task is:
Let's see if we can use non-linear regression to recover the true values of our model parameters. First define the model:
Can you write Python code for it?

def exp_model(x, A, B):
    return A*np.exp(x*B)


I am working on the file "Fitting/FittingModels.ipynb".
The first task is:
Plot the raw data:
Can you write Python code for it?

plt.plot(xdata, ydata, 'k.')
plt.xlabel('x')
plt.ylabel('y');


I am working on the file "Fitting/FittingModels.ipynb".
The first task is:
Then use curve_fit to fit the model:
Can you write Python code for it?

theta_best, theta_cov = opt.curve_fit(exp_model, xdata, ydata)


I am working on the file "Fitting/FittingModels.ipynb".
The first task is:
We can again plot the raw data and best fit line:
Can you write Python code for it?

yfit = theta_best[0] + theta_best[1] * xfit

plt.errorbar(xdata, ydata, dy,
             fmt='.k', ecolor='lightgray');
plt.plot(xfit, yfit, '-b');


xfit = np.linspace(0,10.0)
yfit = theta_best[1]*xfit + theta_best[0]

plt.plot(xfit, yfit)
plt.errorbar(xdata, ydata, dy,
         fmt='.k', ecolor='lightgray')
plt.xlabel('x')
plt.ylabel('y');


I am working on the file "Fitting/FittingModelsEx01.ipynb".
The first task is:
Fitting a quadratic curve
For this problem we are going to work with the following model:
$$ y_{model}(x) = a x^2 + b x + c $$
The true values of the model parameters are as follows:
Can you write Python code for it?


I am working on the file "Fitting/FittingModelsEx01.ipynb".
The first task is:
Now fit the model to the dataset to recover estimates for the model's parameters:

Print out the estimates and uncertainties of each parameter.
Plot the raw data and best fit of the model.
Can you write Python code for it?

def quad_model(x,a,b,c):
    return a*x**2 + b*x + c

theta_best, theta_cov = opt.curve_fit(quad_model, xdata, ydata)

a = theta_best[0]
b = theta_best[1]
c = theta_best[2]

print('a = {0:.3f} +/- {1:.3f}'.format(a, np.sqrt(theta_cov[0,0])))
print('b = {0:.3f} +/- {1:.3f}'.format(b, np.sqrt(theta_cov[1,1])))
print('c = {0:.3f} +/- {1:.3f}'.format(c, np.sqrt(theta_cov[2,2])))

plt.figure(figsize=(8,6))
plt.scatter(xdata,ydata,label = 'data');
plt.plot(xdata,quad_model(xdata,a,b,c),label = 'Curve Fit');
plt.xlabel('x data');
plt.ylabel('y data');
plt.title('Curve Fit');
plt.tick_params(axis='x',top='off',direction='out');
plt.tick_params(axis='y',right='off',direction='out');
plt.text(-5.5, 7, 'a = {0:.3f}'.format(a))
plt.text(-5.5, 5, 'b = {0:.3f}'.format(b))
plt.text(-5.5, 3, 'c = {0:.3f}'.format(c))
plt.legend(loc=2);

assert True # leave this cell for grading the fit; should include a plot and printout of the parameters+errors


I am working on the file "Fitting/FittingModelsEx02.ipynb".
The first task is:
Fitting a decaying oscillation
For this problem you are given a raw dataset in the file decay_osc.npz. This file contains three arrays:

tdata: an array of time values
ydata: an array of y values
dy: the absolute uncertainties (standard deviations) in y

Your job is to fit the following model to this data:
$$ y(t) = A e^{-\lambda t} \cos{\omega t + \delta} $$
First, import the data using NumPy and make an appropriately styled error bar plot of the raw data.
Can you write Python code for it?


I am working on the file "Fitting/FittingModelsEx02.ipynb".
The first task is:
Now, using curve_fit to fit this model and determine the estimates and uncertainties for the parameters:

Print the parameters estimates and uncertainties.
Plot the raw and best fit model.
You will likely have to pass an initial guess to curve_fit to get a good fit.
Treat the uncertainties in $y$ as absolute errors by passing absolute_sigma=True.
Can you write Python code for it?

def decay_model(t,A,Y,w,d):
    return A*np.exp(-Y*t)*np.cos(w*t)+d

theta_best, theta_cov = opt.curve_fit(decay_model, tdata, ydata,absolute_sigma=True)

A = theta_best[0]
Y = theta_best[1]
w = theta_best[2]
d = theta_best[3]

print('A = {0:.3f} +/- {1:.3f}'.format(A, np.sqrt(theta_cov[0,0])))
print('λ = {0:.3f} +/- {1:.3f}'.format(Y, np.sqrt(theta_cov[1,1])))
print('ω = {0:.3f} +/- {1:.3f}'.format(w, np.sqrt(theta_cov[2,2])))
print('δ = {0:.3f} +/- {1:.3f}'.format(d, np.sqrt(theta_cov[3,3])))

tfit = np.linspace(0,20)

plt.figure(figsize=(8,6))
plt.xlim(0,20)
plt.ylim(-6,6)
plt.scatter(tdata, ydata,color = 'black')
plt.plot(tfit,decay_model(tfit,A,Y,w,d))
plt.xlabel('Time');
plt.ylabel('Y Data');
plt.title('Oscillation Curve Fit');
plt.tick_params(axis='x',top='off',direction='out');
plt.tick_params(axis='y',right='off',direction='out');

assert True # leave this cell for grading the fit; should include a plot and printout of the parameters+errors


I am working on the file "FittingTest.ipynb".
The first task is:
Comparing with previous work
Next, we show values from Mey et al.'s 1993 assessment of the Al-Zn system. We have not done any fitting here; we are just showing how arbitrary values for the parameters may be specified and easily used to plot the phase diagram. Note the optional parameters keyword argument specified for each call to Model. This feature is what makes it easy to perform arbitrary variable substitutions in our thermodynamic models.
Can you write Python code for it?

# Parameters from Mey (1993)
assessed_parameters = {
    'LIQALZN0H': 10465.3,
    'LIQALZN0S': 3.39259,
    'FCCALZN0H': 7297.5,
    'FCCALZN0S': -0.47512,
    'FCCALZN1H': 6612.9,
    'FCCALZN1S': 4.5911,
    'FCCALZN2H': -3097.2,
    'FCCALZN2S': -3.306635,
    'HCPALZN0H': 18821.0,
    'HCPALZN0S': 8.95255,
    'HCPALZN3H': -702.8,
    'HCPALZN3S': 0.0
    }

# Initialize phase models
assessed_models = {
    'LIQUID': Model(db_alzn, ['AL', 'ZN'], 'LIQUID', parameters=assessed_parameters),
    'FCC_A1': Model(db_alzn, ['AL', 'ZN', 'VA'], 'FCC_A1', parameters=assessed_parameters),
    'HCP_A3': Model(db_alzn, ['AL', 'ZN', 'VA'], 'HCP_A3', parameters=assessed_parameters)
    }

fig = plt.figure(figsize=(9,6))
%time ax = binplot(db_alzn, ['AL', 'ZN', 'VA'] , my_phases_alzn, 'X(ZN)', 300, 1000, ax=plt.gca(), model=assessed_models)
ax = phase_eq.plot(kind='scatter', x='X(ZN)', y='T', ax=ax, edgecolor='black',
                       s=30, xlim=(0,1), marker='s', facecolor='none')
ax = ax.set_title('Al-Zn assessed by Mey (1993)', fontsize=20)


I am working on the file "FittingTest.ipynb".
The first task is:
Getting the fitted parameters
The result object contains the fitted values for the parameters.
Can you write Python code for it?


I am working on the file "FittingTest.ipynb".
The first task is:
Prior to fitting
First, we'll show what the phase diagram looks like without any binary interaction parameters. In this diagram only the pure component lattice stabilities of each phase relative to their standard states are considered.
Can you write Python code for it?

blank_parameters = {
    'LIQALZN0H': 0.0,
    'LIQALZN0S': 0.0,
    'FCCALZN0H': 0.0,
    'FCCALZN0S': 0.0,
    'FCCALZN1H': 0.0,
    'FCCALZN1S': 0.0,
    'FCCALZN2H': 0.0,
    'FCCALZN2S': 0.0,
    'HCPALZN0H': 0.0,
    'HCPALZN0S': 0.0,
    'HCPALZN3H': 0.0,
    'HCPALZN3S': 0.0
    }

models = {
    'LIQUID': Model(db_alzn, ['AL', 'ZN'], 'LIQUID', parameters=blank_parameters),
    'FCC_A1': Model(db_alzn, ['AL', 'ZN', 'VA'], 'FCC_A1', parameters=blank_parameters),
    'HCP_A3': Model(db_alzn, ['AL', 'ZN', 'VA'], 'HCP_A3', parameters=blank_parameters)
    }

fig = plt.figure(figsize=(9,6))
%time ax = binplot(db_alzn, ['AL', 'ZN', 'VA'] , my_phases_alzn, 'X(ZN)', 300, 1000, ax=plt.gca(), model=models)
ax = ax.set_title('Al-Zn with endmembers only', fontsize=20)


I am working on the file "FittingTest.ipynb".
The first task is:
Quantitative comparison of assessment values with fitted values
The difference can be partially understood by the fact that Mey's assessment also relies on thermochemical data while here we only fit to points on the phase diagram.
Can you write Python code for it?

for param_name, param_value in sorted(result.items()):
    percent_change = abs(param_value - assessed_parameters[param_name]) / abs(assessed_parameters[param_name])
    print('{0}: {1:.2%} difference'.format(param_name, percent_change))


I am working on the file "FittingTest.ipynb".
The first task is:
Starting parameter values
Let's imagine we have some way (first-principles calculations, intuition or both) to guess some starting values for our parameters.
Can you write Python code for it?

perturbed_parameters = {
    'LIQALZN0H': 10000,
    'LIQALZN0S': 3,
    'FCCALZN0H': 7000,
    'FCCALZN0S': -0.2,
    'FCCALZN1H': 5000,
    'FCCALZN1S': 4,
    'FCCALZN2H': -3000,
    'FCCALZN2S': -3,
    'HCPALZN0H': 17000,
    'HCPALZN0S': 6,
    'HCPALZN3H': -700,
    'HCPALZN3S': 1
    }
fit_models = {
    'LIQUID': Model(db_alzn, ['AL', 'ZN'], 'LIQUID', parameters=perturbed_parameters),
    'FCC_A1': Model(db_alzn, ['AL', 'ZN', 'VA'], 'FCC_A1', parameters=perturbed_parameters),
    'HCP_A3': Model(db_alzn, ['AL', 'ZN', 'VA'], 'HCP_A3', parameters=perturbed_parameters)
    }

fig = plt.figure(figsize=(9,6))
ax = binplot(db_alzn, ['AL', 'ZN', 'VA'] , my_phases_alzn, 'X(ZN)', 300, 1000, ax=plt.gca(), model=fit_models)
ax = phase_eq.plot(kind='scatter', x='X(ZN)', y='T', ax=ax, edgecolor='black',
                       s=30, xlim=(0,1), marker='s', facecolor='none')
ax = ax.set_title('Al-Zn with parameter guesses', fontsize=20)


I am working on the file "FiveThirtyEightRiddler/2016-10-14/2016-10-14.ipynb".
The first task is:
Classic Riddler
While traveling in the Kingdom of Arbitraria, you are accused of a heinous crime. Arbitraria decides who’s guilty or innocent not through a court system, but a board game. It’s played on a simple board: a track with sequential spaces numbered from 0 to 1,000. The zero space is marked “start,” and your token is placed on it. You are handed a fair six-sided die and three coins. You are allowed to place the coins on three different (nonzero) spaces. Once placed, the coins may not be moved.
After placing the three coins, you roll the die and move your token forward the appropriate number of spaces. If, after moving the token, it lands on a space with a coin on it, you are freed. If not, you roll again and continue moving forward. If your token passes all three coins without landing on one, you are executed. On which three spaces should you place the coins to maximize your chances of survival?
Extra credit: Suppose there’s an additional rule that you cannot place the coins on adjacent spaces. What is the ideal placement now? What about the worst squares — where should you place your coins if you’re making a play for martyrdom?
Can you write Python code for it?

%matplotlib inline
import numpy as np

NUM_SPACES = 1000

probs = np.zeros((1000,1000))

# Seed first column of probabilities
# The first 6 values should be 1/6
probs[0:6,0] = np.array([1/6]*6)

for col in np.arange(1,NUM_SPACES):
    for row in np.arange(NUM_SPACES):
        target_col = col-1
        start_row = max(0,row-6)
        end_row = max(0,row)
        new_val = sum(probs[start_row:end_row,target_col])/6
        probs[row,col] = new_val

from matplotlib import pyplot as plt
sum_probs = np.sum(probs,axis=1)

x1 = np.arange(1,31)
y1 = sum_probs[:30]

plt.plot(x1,y1,marker='.',color='b')
plt.ylim(0)
plt.draw()
print(np.argmax(sum_probs)+1)


I am working on the file "FiveThirtyEightRiddler/2017-04-14/2017-04-26-empty_court_seats.ipynb".
The first task is:
First we'll need a way to track which party the Senate & President are part of.  For now, let's just stick with the two major parties and create a Party enumeration.  Enumerations can group and give names to related constants in the code.  This can help the code be more understandable when reading it.
Can you write Python code for it?

class Party(Enum):
    D = 1
    R = 2

color_trans = {Party.D:'blue', Party.R:'red'}


I am working on the file "FiveThirtyEightRiddler/2017-04-14/2017-04-26-empty_court_seats.ipynb".
The first task is:
First, let's look at the result of our simulated supreme court over 200 years.  Along the bottom, the parties of the Senate and President are shown.  The height of each stack represents the number of seats that party holds, and the white space indicates vacancies.
Can you write Python code for it?

sim_years = 200

years, bench_stacks, president_parties, senate_parties, _ = run_simulation(sim_years)

stacked_plot_bench_over_time_with_parties(years,
                                          bench_stacks,
                                          president_parties,
                                          senate_parties,
                                          color_trans,
                                          Party)


I am working on the file "FiveThirtyEightRiddler/2017-04-14/2017-04-26-empty_court_seats.ipynb".
The first task is:
Our lass class is Bench.  This class will represent the bench that contains the Justices currently on the Supreme Court.  When the Bench is first formed, it will be empty.  We care about modifying the Bench in a few ways:

Filling all available seats with judges of a certain party (fill_seats).
Adding years to determine if judges have vacated their seats (add_years).
Getting the composition of the court at a particular year, used for displaying the data later (breakdown).

Empty seats in the court are represented by None, and judges are removed when they have <= 0 years left in their term.
Can you write Python code for it?

class Bench:

    SIZE = 9

    def __init__(self):
        self.seats = [None] * self.SIZE

    def fill_seats(self, party):
        # loop through all seats
        for i in range(self.SIZE):
            if self.seats[i] is None:
                # if seat is empty, add new
                # justice of the correct party
                self.seats[i] = Justice(party)

    def add_years(self, num_years):
        for i in range(self.SIZE):
            if self.seats[i] is not None:
                # for occupied seats, remove the given
                # number of years from their remaining
                # term.  If their term is less than 0
                # this means their seat should now
                # be empty again.
                self.seats[i].term -= num_years
                if self.seats[i].term <= 0:
                    self.seats[i] = None

    def breakdown(self):
        c = Counter([s.party.name if s is not None else "" for s in self.seats])

        return tuple(c[k] if k in c else 0 for k in [""] + [e.name for e in Party])

    def __repr__(self):
        return "\n".join(map(str,self.seats))


I am working on the file "FiveThirtyEightRiddler/2017-04-14/2017-04-26-empty_court_seats.ipynb".
The first task is:
This distribution shows that we should still expect to see ~0.71 vacancies per year over the long run, but it wouldn't be surprising to see 0.68 or 0.73 vacancies.
As a thought experiment, let's see what happen if the Green Party suddenly launches into relevance and all 3 parties have an equal shot in elections.  How would this impact the result?
Can you write Python code for it?

class Party(Enum):
    D = 1
    R = 2
    G = 3

color_trans = {Party.D:'blue', Party.R:'red', Party.G:'green'}

sim_years = 200

years, bench_stacks, president_parties, senate_parties, mean = run_simulation(sim_years)

stacked_plot_bench_over_time_with_parties(years,
                                          bench_stacks,
                                          president_parties,
                                          senate_parties,
                                          color_trans,
                                          Party)

sim_years = 1000

years, bench_stacks, president_parties, senate_parties, mean = run_simulation(sim_years)

stacked_plot_bench_over_time(years, bench_stacks, mean, color_trans, Party)


I am working on the file "FiveThirtyEightRiddler/2017-04-14/2017-04-26-empty_court_seats.ipynb".
The first task is:
This simulation shows that we should expect a little less than 1 vacancy, about 0.7, per year.  It also illustrates that as more data is added, the cumulative moving average becomes less variable.
However, this is only a single simulation run, and could be an outlier.  To see how likely different numbers of vacancies are, let's run a Monte Carlo experiment.  In this experiment, we're going to run 1000 different simulations, each to 50000 years.
Can you write Python code for it?

sim_years = 50000
sample_size = 1000

results=[]

with Pool(processes=4) as p:
    with tqdm_notebook(total=sample_size) as pbar:
        for r in p.imap_unordered(run_simulation, itertools.repeat(sim_years,sample_size)):
            results.append(r)
            pbar.update(1)

years, _, _, _, means = zip(*results)

plot_sims(years[0], means, [0.5, 0.9])


I am working on the file "FiveThirtyEightRiddler/2017-04-14/2017-04-26-empty_court_seats.ipynb".
The first task is:
We'll also make a class to represent each justice.  When a new Justice is created for a party, they'll be given a randomly generated term of somewhere between 0 and 40 years.
Can you write Python code for it?

class Justice:

    def __init__(self, party):
        self.party = party
        self.term = random.randint(0,40)

    def __str__(self):
        return self.__repr__()

    def __repr__(self):
        return "{party}-{term}".format(party=self.party.name,term=self.term)


I am working on the file "FiveThirtyEightRiddler/2017-04-14/2017-04-26-empty_court_seats.ipynb".
The first task is:
run_simulation will execute the simulation for the supplied number of years, and post-process the data to return the following information:
years: an array of all the years that were simulated.
bench_stacks: the stacked bar graph data for the composition of the court at each year.
president_parties: an array of the president's party at each year.
senate_parties: an array of the senate's party at each year.
mean: an array with the cumulative moving averages of the number of vacancies.
Can you write Python code for it?

def run_simulation(sim_years):
    years, benches, president_parties, senate_parties = zip(*list(simulate(sim_years)))
    bench_stacks = np.row_stack(zip(*benches))
    vacancies = bench_stacks[0]
    mean = np.cumsum(vacancies) / (np.asarray(years) + 1)
    return years, bench_stacks, president_parties, senate_parties, mean


I am working on the file "FiveThirtyEightRiddler/2017-09-15/classic_sticks.ipynb".
The first task is:
1
Can you write Python code for it?

triangle_count = 0
for _ in range(N):
    a,b = sorted((random.random(), random.random()))
    x,y,z = (a,b-a,1-b)
    if x<0.5 and y<0.5 and z<0.5:
        triangle_count += 1

triangle_count / N


I am working on the file "FiveThirtyEightRiddler/2017-09-15/classic_sticks.ipynb".
The first task is:
2
Can you write Python code for it?

triangle_count = 0
for _ in range(N):
    sticks = sorted((random.random(), random.random(), random.random()))
    if sticks[2] < sticks[0] + sticks[1]:
        triangle_count += 1

triangle_count / N


I am working on the file "FiveThirtyEightRiddler/2017-09-15/classic_sticks.ipynb".
The first task is:
3
Can you write Python code for it?

triangle_count = 0
for _ in range(N):
    a,b = sorted((random.random(), random.random()))
    x,y,z = (a,b-a,1-b)
    if (x**2 + y**2 >  z**2) and (x**2 + z**2 > y**2) and (z**2 + y**2 > x**2):
        triangle_count += 1

triangle_count / N


I am working on the file "FiveThirtyEightRiddler/2017-09-15/classic_sticks.ipynb".
The first task is:
4
Can you write Python code for it?

triangle_count = 0
for _ in range(N):
    x,y,z = (random.random(), random.random(), random.random())
    if (x**2 + y**2 >  z**2) and (x**2 + z**2 > y**2) and (z**2 + y**2 > x**2):
        triangle_count += 1

triangle_count / N


I am working on the file "FiveThirtyEightRiddler/2018-04-06/vandal_dates.ipynb".
The first task is:
What will be the longest gap between attacks?
Can you write Python code for it?

gaps = [this-previous for this,previous in zip(anarchy_dates[1:], anarchy_dates)]

max_gap = max(gaps)

for i, gap in enumerate(gaps):
    if gap == max_gap:
        print(f'{gap.days} days between {anarchy_dates[i-1]} and {anarchy_dates[i]}')


I am working on the file "FiveThirtyEightRiddler/2018-04-06/vandal_dates.ipynb".
The first task is:
What year will see the most vandalism?
Can you write Python code for it?

date_counts = Counter(d.year for d in anarchy_dates)

_, max_attacks = date_counts.most_common()[0]
_, min_attacks = date_counts.most_common()[-1]

for year, attacks in date_counts.items():
    if attacks == max_attacks:
        print(f'{attacks} attacks in year {year}')


I am working on the file "Fizz Buzz.ipynb".
The first task is:
Create the trainning set
Encode the input (a number)
This example convert the number to a binary representation
Can you write Python code for it?

NUM_DIGITS = 10

def binary_encode(i, num_digits):
    return np.array([i >> d & 1 for d in range(num_digits)])

#Let's check if it works
for i in range(10):
    print i, binary_encode(i, NUM_DIGITS)


I am working on the file "Fizz Buzz.ipynb".
The first task is:
Creation of the model
The model is made of:
* one hidden layer that contains 100 neurons
* one output layer
The input is fully connected to the hidden layer and a relu function is applyed
The relu function is a rectifier that just output zero if the input is negative.
First we'll define an helper function to initialise parameters with randoms values
Can you write Python code for it?

def init_weights(shape):
    return tf.Variable(tf.random_normal(shape, stddev=0.01))


I am working on the file "Fizz Buzz.ipynb".
The first task is:
Here an example of index used for one epoch:
Can you write Python code for it?

#random permutation of the index will be used during the training for each epoch
permutation_index = np.random.permutation(range(len(trX)))
for start in range(0, len(trX), BATCH_SIZE):
        end = start + BATCH_SIZE
        print "Batch starting at", start
        print permutation_index[start:end]

# Launch the graph in a session
sess = tf.Session()
tf.initialize_all_variables().run(session=sess)

for epoch in range(5000):
    # Shuffle the data before each training iteration.
    p = np.random.permutation(range(len(trX)))
    trX, trY = trX[p], trY[p]

    # Train in batches of 128 inputs.
    for start in range(0, len(trX), BATCH_SIZE):
        end = start + BATCH_SIZE
        sess.run(train_op, feed_dict={X: trX[start:end], Y: trY[start:end]})

    # And print the current accuracy on the training data.
    if (epoch%100==0):  # each 100 epoch, to not overflow the jupyter log
        # np.mean(A==B) return a number between 0 and 1. (true_count/total_count)
        print(epoch, np.mean(np.argmax(trY, axis=1) ==
                         sess.run(predict_op, feed_dict={X: trX, Y: trY})))

# And now for some fizz buzz
numbers = np.arange(1, 101)
teX = np.transpose(binary_encode(numbers, NUM_DIGITS))
teY = sess.run(predict_op, feed_dict={X: teX})

output = np.vectorize(fizz_buzz)(numbers, teY)
print output

sess.close() # don't forget to close the session if you don't use it anymore. Or use the *with* statement.

# Lets check the quality
Y = np.array([fizz_buzz_encode(i) for i in range(1,101)])
print "accuracy", np.mean(np.argmax(Y, axis=1) == teY)

for i in range(1,100):
    actual = fizz_buzz(i, np.argmax(fizz_buzz_encode(i)))
    predicted = output[i-1]
    ok = True
    if actual <> predicted: ok = False
    print i, "{:>8}".format(actual), "{:>8}".format(predicted), ok


I am working on the file "Fizz Buzz.ipynb".
The first task is:
Iterate until the model is good enough
One epoch consists of one full training cycle on the training set.
Once every sample in the set is seen, you start again - marking the beginning of the 2nd epoch. source
The training set is randomly permuted between each epoch.
The learning is not done on the full set at once.
Instead the learning set is divided in small batch and the learning is done for each of them.
Can you write Python code for it?


I am working on the file "Fizz Buzz.ipynb".
The first task is:
This operator will minimize the cost using the Gradient Descent witch is the most common optimizer to find parameters than will minimise the cost.
We'll also define a prediction operator that will be able to output a prediction.
* 0 means no fizz no buzz
* 1 means fizz
* 2 means buzz
* 3 means fizzbuzz
Can you write Python code for it?


I am working on the file "Fizz Buzz.ipynb".
The first task is:
Training
Create the cost function
The cost function measure how bad the model is.
It is the distance between the prediction (py_x) and the reality (Y).
Can you write Python code for it?

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(py_x, Y))


I am working on the file "Fizz Buzz.ipynb".
The first task is:
softmax_cross_entropy_with_logits(py_x, Y) measure the distance between py_x and Y. SoftMax is the classical way to measure the distance between a predicted result and the actual result in a cost function.
reduce_mean calculate the mean of a tensor. In this case the mean of the distance for the whole training set

Train the model
Training a model in TensorFlow is extremly simple, you just define a trainer operator!
Can you write Python code for it?


I am working on the file "Floating_Points.ipynb".
The first task is:
This time the summation seems correct. That is the same as 2/3 = 0.6666...667
Such an error could cost people's life and money: http://ta.twi.tudelft.nl/nw/users/vuik/wi211/disasters.html
That is the result of how floating points are handeled in PC:
IEEE floating point
Now lets observe other errors.
Substraction
$\hat{a} = (1+10^{-16}) - (1-10^{-16})$
$\Delta a = \hat{a} - a$
$\frac{\Delta a}{a} = ?$
Can you write Python code for it?


I am working on the file "Floating_point_error_propagation_in_polynomial_multiplication_with_Fast-Fourier_Transform.ipynb".
The first task is:
Benchmark
Stupidly, let us check if our naive implementation of $(P, Q) \mapsto PQ$ by evaluation-and-interpolation is more or less efficient than numpy.polyfit:
Can you write Python code for it?

def mypolymul(P, Q):
    n = len(P) - 1
    m = len(Q) - 1
    d = n + m
    lambdas = np.cos(np.pi * (2 * np.arange(1, 2 + d) - 1) / (2 * d))
    values_P = np.polyval(P, lambdas)
    values_Q = np.polyval(Q, lambdas)
    values_PQ = values_P * values_Q
    PQ_sampled = np.polyfit(lambdas, values_PQ, d)
    # return PQ_sampled
    return np.asarray(np.round(PQ_sampled), dtype=int)

np.polymul(P, Q)

mypolymul(P, Q)

import warnings
warnings.simplefilter('ignore', np.RankWarning)

%timeit np.polymul(P, Q)
%timeit mypolymul(P, Q)


I am working on the file "Floating_point_error_propagation_in_polynomial_multiplication_with_Fast-Fourier_Transform.ipynb".
The first task is:
Chebyshev nodes as interpolation values
Let us consider the points $\lambda_k = \cos\left(\frac{2k-1}{2d} \pi\right)$, $k=1,\dots,1+d=n+m+1$.
These are called the Chebyshev nodes.
Can you write Python code for it?

lambdas = np.cos(np.pi * (2 * np.arange(1, 2 + d) - 1) / (2 * d))
lambdas

values_P = np.polyval(P, lambdas)
values_P

values_Q = np.polyval(Q, lambdas)
values_Q

values_PQ = values_P * values_Q
values_PQ

PQ_sampled2 = np.polyfit(lambdas, values_PQ, d)
PQ_sampled2

PQ

np.asarray(np.round(PQ_sampled2), dtype=int)


I am working on the file "Floating_point_error_propagation_in_polynomial_multiplication_with_Fast-Fourier_Transform.ipynb".
The first task is:
Examples
Let consider the polynomials $P(X) = 1 + X + X^3$ and $Q(X) = X^2 + X^5$, of degrees $n=3$ and $m=5$:
Can you write Python code for it?


I am working on the file "Floating_point_error_propagation_in_polynomial_multiplication_with_Fast-Fourier_Transform.ipynb".
The first task is:
If we evaluate both $P(X)$ and $Q(X)$, on $n+m$ different points, $\lambda_i \in \mathbb{R}$ or $\in\mathbb{N}$, then we can fit a polynomial of degree $n+m = \delta(PQ)$ on these sampling points, and by uniqueness (thanks to the Fundamental Theorem of Algebra), it will be equal to $(PQ)(X)$.
The fit can be obtained, for instance, by Lagrange interpolation, which is not so efficient but easy to implement.
Here, I will simply use the numpy.polyfit function.
Can you write Python code for it?


I am working on the file "Floating_point_error_propagation_in_polynomial_multiplication_with_Fast-Fourier_Transform.ipynb".
The first task is:
Naive interpolation values
Let us consider the points $\lambda_i = i$, $i=0,\dots,n+m$.
Can you write Python code for it?

lambdas = np.arange(0, d + 1)
lambdas

values_P = np.polyval(P, lambdas)
values_P

values_Q = np.polyval(Q, lambdas)
values_Q

values_PQ = values_P * values_Q
values_PQ

PQ_sampled = np.polyfit(lambdas, values_PQ, d)
PQ_sampled

PQ

np.asarray(np.round(PQ_sampled), dtype=int)


I am working on the file "Floating_point_error_propagation_in_polynomial_multiplication_with_Fast-Fourier_Transform.ipynb".
The first task is:
Of course, our implementation is slower.
But on small polynomials, not so slower.
What about larger polynomials?
Can you write Python code for it?

def random_polynomial(d=10, maxcoef=1):
    return np.random.randint(low=-maxcoef, high=maxcoef+1, size=d+1)

P = random_polynomial()
Q = random_polynomial()
P, Q
%timeit np.polymul(P, Q)
np.polymul(P, Q)
%timeit mypolymul(P, Q)
mypolymul(P, Q)
assert np.all(np.polymul(P, Q) == mypolymul(P, Q))


I am working on the file "Floating_point_error_propagation_in_polynomial_multiplication_with_Fast-Fourier_Transform.ipynb".
The first task is:
Ok, at least it seems to work!
But we saw that even with very small degrees ($n=3, m=5$), floating point errors were not so small on these points: the largest "should be 0" value (i.e., $\simeq 0$) value was:
Can you write Python code for it?


I am working on the file "Floating_point_error_propagation_in_polynomial_multiplication_with_Fast-Fourier_Transform.ipynb".
The first task is:
Ok, at least it seems to work!
But we saw that even with very small degrees ($n=3, m=5$), floating point errors were not so small on these wrongly chosen points $\lambda_i = i$, $i=0,\dots,n+m$.
The largest "should be 0" value (i.e., $\simeq 0$) value was:
Can you write Python code for it?


I am working on the file "Floating_point_error_propagation_in_polynomial_multiplication_with_Fast-Fourier_Transform.ipynb".
The first task is:
On a larger example:
Can you write Python code for it?

d = 100
maxcoef = 1
%timeit np.polymul(random_polynomial(d=d, maxcoef=maxcoef), random_polynomial(d=d, maxcoef=maxcoef))
%timeit mypolymul(random_polynomial(d=d, maxcoef=maxcoef), random_polynomial(d=d, maxcoef=maxcoef))

P, Q = random_polynomial(d=d, maxcoef=maxcoef), random_polynomial(d=d, maxcoef=maxcoef)
assert np.all(np.polymul(P, Q) == mypolymul(P, Q))

d = 10
maxcoef = 3
%timeit np.polymul(random_polynomial(d=d, maxcoef=maxcoef), random_polynomial(d=d, maxcoef=maxcoef))
%timeit mypolymul(random_polynomial(d=d, maxcoef=maxcoef), random_polynomial(d=d, maxcoef=maxcoef))

P, Q = random_polynomial(d=d, maxcoef=maxcoef), random_polynomial(d=d, maxcoef=maxcoef)
assert np.all(np.polymul(P, Q) == mypolymul(P, Q))

d = 10
maxcoef = 50
%timeit np.polymul(random_polynomial(d=d, maxcoef=maxcoef), random_polynomial(d=d, maxcoef=maxcoef))
%timeit mypolymul(random_polynomial(d=d, maxcoef=maxcoef), random_polynomial(d=d, maxcoef=maxcoef))

P, Q = random_polynomial(d=d, maxcoef=maxcoef), random_polynomial(d=d, maxcoef=maxcoef)
assert np.all(np.polymul(P, Q) == mypolymul(P, Q))


I am working on the file "Floating_point_error_propagation_in_polynomial_multiplication_with_Fast-Fourier_Transform.ipynb".
The first task is:
Their product is $(PQ)(X)$, of degree $n+m=8$:
$$ \begin{align}(PQ)(X) &= (1 + X + X^3) (X^2 + X^5) \
&= X^2 + X^5 + X^3 + X^7 + X^5 + X^8 \
&= X^2 + X^3 + 2 X^5 + X^7 + X^8
\end{align} $$
Can you write Python code for it?

PQ = polymul(P, Q)
d = len(PQ) - 1
PQ, d


I am working on the file "Floridas_Water_Budget.ipynb".
The first task is:
PART 1: Water Used by Florida Counties in 2010
This table displays the County name, its population, the puplic water supply for that county, and the total water used by that county.
Can you write Python code for it?

# displays the first few rows of the table
data.head(4)

# Set variables for scatter plot
x = data.Population
y = data.WaterUsed

fig = plt.figure(figsize=(15, 6))
plt.scatter(x,y)
plt.xlim(0,3000000)
plt.ylim(0,350)
plt.title('The Relationship Between Population and How Much Water a County Consumes Each Year')
plt.xlabel('Population (individuals)')
plt.ylabel('Water Used (million gallons)')

# This actually shows the plot
plt.show()

# Creates a new dataset for County

place = data.groupby("County", as_index = False).sum()


I am working on the file "Floridas_Water_Budget.ipynb".
The first task is:
PART 3: Monitoring Lake Apopka a Freshwater Source for Agriculture
Can you write Python code for it?

# Imports another csv file locally
#level = pd.read_csv("Lake Apopka Waterlevel 2005.csv")

# Imports another csv file from the web
level = pd.read_csv("https://gist.githubusercontent.com/GoodmanSciences/e63b6cb68cd6ef5235dc8c113ea9995a/raw/39139535f7ef05057ecce1126ea336ca7bcfb879/Lake%2520Apopka%2520Waterlevel%25202005.csv")

# Sets Date as index
lev2 = level.set_index("Date")


I am working on the file "Floridas_Water_Budget.ipynb".
The first task is:
Pre-Questions

Using what you've learned in this unit, answer questions 1 & 2 in your coding booklet. The Pre-Questions are listed below for you convienence.

Access to freshwater is a limiting factor in many ecosystems, what are some
other limiting factors that can effect native populations?
Draw a diagram that shows a lake water source. Add and label arrows for
ways that water can be added to the lake. Add and label arrows for the ways
that water can be removed from the lake. (Be sure to include human and
natural / non-human sources)

PART 1: Water Used by Florida Counties in 2010

This table displays the County name, its population, the puplic water supply for that county, and the total water used by that county.
Use and modify the sections of code below for Part 1 to answer questions 3-5 in your coding booklet.
Can you write Python code for it?


I am working on the file "Floridas_Water_Budget.ipynb".
The first task is:
Rainfall in February 2005 (Inches)
Can you write Python code for it?

feb.head()

# Plots rainfall form ascending order
feb.sort_values(by="Monthly Total", ascending=True).plot(x="County", y="Monthly Total", kind="barh", title="Rainfall in February (Inches)", legend=False);


I am working on the file "Floridas_Water_Budget.ipynb".
The first task is:
This table shows the top 10 water consuming counties, the population, the amount of the population that is connected to public water (PublicSupply), and the total water used by each county.
Can you write Python code for it?

# Orginizes by County with the highest water usage in decending order
#Only displays the top 10 highestest water consuming counties by putting .head(10)

mostwater = place.sort_values(by="WaterUsed", ascending = False).head(10)
mostwater


I am working on the file "Floridas_Water_Budget.ipynb".
The first task is:
Try to change the histogram so that it displays the County and the Population Total. (Right now it is displaying County and Water Use Total)

PART 2:  Rainfall Additions to the Water Supply

Use and modify the sections of code below for Part 2 to answer questions 6-8 in your coding booklet.
Can you write Python code for it?

# Imports more csv files locally
#feb = pd.read_csv("Feb2005_FL_rainfall.csv")
#july = pd.read_csv("July2005_FL_rainfall.csv")

# Imports more csv files from the web
july = pd.read_csv("https://gist.githubusercontent.com/GoodmanSciences/354fa30fb1e506c055621b893b26ebe8/raw/523e483ae4534c9432f91e5d5b7f9fb0356e95e1/Rainfall%2520FL%2520Jul2005.csv")
feb = pd.read_csv("https://gist.githubusercontent.com/GoodmanSciences/7088ff6b7b8e915a87ee987f3b767641/raw/a76a0dd975f95e6c0c5e6ee810e6f6e66faeca9b/Rainfall%2520FL%2520Feb2005.csv")


I am working on the file "Floridas_Water_Budget.ipynb".
The first task is:
Try to change the histogram so that it displays the County and the Population Total. (Right now it is displaying County and Water Use Total)
PART 2:  Rainfall Additions to the Water Supply
Can you write Python code for it?


I am working on the file "Floridas_Water_Budget.ipynb".
The first task is:
Try to change the histogram to display the data in decending order.
Rainfall in July 2005 (Inches)
Can you write Python code for it?

july.head()

july.sort_values(by="Monthly Total", ascending=True).plot(x="County", y="Monthly Total", kind="barh", title="Rainfall in July (Inches)", legend=False);

from IPython.display import Image
from IPython.core.display import HTML
Image(url= 'https://preview.ibb.co/g7Z6sa/Average_Monthly_Water_Consumption.png')

Image(url= 'https://floridamaps.files.wordpress.com/2015/03/florida-counties.jpg')

#Double-click to make this image GINORMOUS


I am working on the file "Floridas_Water_Budget.ipynb".
The first task is:
Water Level in February
Can you write Python code for it?

# Displays only Feb 1st through the 28th
lev2.loc["2/1/2005":"2/28/2005", :]


I am working on the file "Floridas_Water_Budget.ipynb".
The first task is:
Water Level in July
Can you write Python code for it?

# Displays only July 1st through the 7th
lev2.loc["7/1/2005":"7/7/2005", :]

# Plot of all values in level dataset
level.plot('Date', 'Water Level')

Image(url= 'http://www.floridacountiesmap.com/graphics/orange.gif')


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
2d histograms
Can you write Python code for it?

figure(figsize=(10, 10))
well.plot(['B1-A', 'Y2-A']);


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
<pre>
(Note: The lines above with the % mark instruct python to reload the dependencies if any of them changed.)
</pre>

Loading Data
Loading individual files
Can you write Python code for it?

import os, FlowCytometryTools
datadir = os.path.join(FlowCytometryTools.__path__[0], 'tests', 'data', 'Plate01')
datafile = os.path.join(datadir, 'CFP_Well_A4.fcs')
print('Loading file from path: {0}'.format(datafile))
sample = FCMeasurement(ID='Test Plate', datafile=datafile)

sample.get_data().iloc[:20]


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
<pre>
This plot function accepts all the same arguments as does the matplotlib pcolormesh function.
For example, we can change the default colormap used for plotting.
</pre>
Can you write Python code for it?

well.plot(['B1-A', 'Y2-A'], cmap=cm.Oranges, colorbar=False);


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
Applying gates to wells
Can you write Python code for it?

original_well = plate['A3']
gated_well = original_well.gate(y2_gate, )

figure(figsize=(4, 10))
ax = subplot(2, 1, 1)
title('Original well')
original_well.plot('Y2-A', bins=300, color='gray', ax=ax)
xlim(-1000, 10000)
grid(True)

ax = subplot(2, 1, 2)
title('Gated well')
gated_well.plot('Y2-A', bins=300,  color='y', ax=ax)
ylim(0, 400)
grid(True)
xlim(-1000, 10000)


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
Basics
Let's pick single well
Can you write Python code for it?

well = plate['A3']
print(well)

well.get_data()['Y2-A'].median()


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
Before we can do any plotting, we need to figure out what information was collected by the flow cytometer.
Can you write Python code for it?


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
Creating Gates
Can you write Python code for it?

y2_gate = ThresholdGate(1000.0, 'Y2-A', region='above')


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
For example, one of the keys is \$TOT (Total number of events) and another is \$DATE (Date of measurement).
Can you write Python code for it?


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
GUI for creating gates (alpha release) (NOT YET IMPLEMENTED. SKIP SECTION)
<B>Important:</B> You must install wx for python in order to use this GUI.
Can you write Python code for it?


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
Plotting
<pre>
You'll find that some basic plotting functionality has already been implemented for you.
Let's check it out by first plotting 1d and 2d histograms on single wells, and then we'll move on to plots on an entire 96-well plate.
</pre>

1d histograms
Can you write Python code for it?

%pylab inline

figure(figsize=(10, 10))
well.plot('B1-A', bins=100);

figure(figsize=(10, 10))
well.plot('B1-A', bins=100, alpha=0.7, color='green', normed=1);


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
Plotting plates
Can you write Python code for it?

figure(figsize=(10, 10))
plate.plot('Y2-A', bins=100, autolabel=False, ylim=(0, 1000),
           xlim=(-1000, 10000),
           row_labels_kwargs={'size' : 'xx-large'});

figure(figsize=(10, 10))
plate.plot(['B1-A', 'Y2-A']);

figure(figsize=(10, 10))
plate.plot(['B1-A', 'Y2-A'], ids=['A3', 'C7']);


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
There is more meta data buried inside of each FCS file. This meta data is stored inside of a dictionary called meta.
Can you write Python code for it?


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
Transformations
Can you write Python code for it?

plate = FCPlate.from_dir(ID='Test Plate', path=datadir,parser='name').transform('hlog', channels=('Y2-A', 'B1-A'))
# The line above is equivalent to the two steps below.
# plate = FCPlate.from_dir(ID='Test Plate', path=datadir)
# plate = plate.transform('hlog', channels=('Y2-A', 'B1-A'))


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
Visualizing Gates
Can you write Python code for it?

w = plate['A3']

ax = subplot()
b1_gate = ThresholdGate(2000.0, 'B1-A', region='above')

plate['A3'].gate(y2_gate).plot(('B1-A', 'Y2-A'), ax=ax, cmap=cm.Reds, gates=[y2_gate]);
plate['A3'].gate(~y2_gate).plot(('B1-A', 'Y2-A'), ax=ax, cmap=cm.gray);

ylim(-1000, 9000)

figure(figsize=(8, 8))
#suptitle('All gates plotted but not applied')
plate.plot(('B1-A','Y2-A'), gates=[y2_gate, b1_gate]);


I am working on the file "FlowCytometryTools/tests/notebooks/api_tests.ipynb".
The first task is:
Working with Plates
You can check which wells have data loaded in them simply by using the print command
Can you write Python code for it?


I am working on the file "FluDB_coding_aln/.ipynb_checkpoints/Untitled-checkpoint.ipynb".
The first task is:
Kyte-Doolittle hydrophobicity scale is needed for us as well. We add the STOP codon there as well, and assign it artificially large KD value, simply to make all the mutations to/from the STOP codon distinguishable.
Can you write Python code for it?

from Bio.SeqUtils import ProtParamData
KD = ProtParamData.kd
KD['*'] = 25.0


I am working on the file "FluDB_coding_aln/.ipynb_checkpoints/Untitled-checkpoint.ipynb".
The first task is:
Let's load all of the alignments using these file names and store them as a dictionary:
Can you write Python code for it?

segment_aln = {}
for seg_idx in range(1, ph1n1.segnum+1):
    fname = aligned_seg_fname(seg_idx)
    segment_aln['seg%d'%seg_idx] = AlignIO.read(fname,'fasta')


I am working on the file "FluDB_coding_aln/.ipynb_checkpoints/Untitled-checkpoint.ipynb".
The first task is:
Reference flu-genome sequence is a consensus of the pandemic H1N1: pH1N1. Let's load the reference with the description of ORF location.
Can you write Python code for it?

ref_fname = "./pH1N1_coding_dat/pH1N1.fa"
orf_fname = "./pH1N1_coding_dat/pH1N1_noPB1F.orf"
ph1n1 = flu.influenza(ref_fname, orf_fname)


I am working on the file "FluDB_coding_aln/.ipynb_checkpoints/Untitled-checkpoint.ipynb".
The first task is:
Then import the Codon Table for standard genetic code, with the slight modification - add STOP codon * as a fully-fledged member of the table:
Can you write Python code for it?

from Bio.Data import CodonTable
genetic_code = CodonTable.standard_dna_table.forward_table
stop_codons = dict([ (codon,'*') for codon in CodonTable.standard_dna_table.stop_codons ])
genetic_code.update(stop_codons)


I am working on the file "FluDB_coding_aln/.ipynb_checkpoints/Untitled-checkpoint.ipynb".
The first task is:
in ORF or not in ORF? where exactly?
Next we define a funtion that would provide all ORFs associated with a given genome locus, along with the description of the hosting codon, and a codon-shift of a given locus (first, second, third position).
Can you write Python code for it?

def get_loci_orf_assoc(orf,loci):
    """checks if a given position is in any of the ORFs. Returns
    a list of the hosting ORFs with corresponding coords. of codons."""
    which_orf = []
    for orf_id in orf:
        orf_bounds = orf[orf_id]
        orf_positions = sum([range(start,stop+1) for start,stop in orf_bounds], [])
        # pos residence in ORF or not in ORF?
        if loci in orf_positions:
            # relative loci, simply an index in the list of orf_positions ...
            relative_pos = orf_positions.index(loci)
            # the index of the codon it belongs to ...
            codon_idx = relative_pos//3
            # relative codon coordinates (indexes within CDS):
            relative_codon_coord = [codon_idx*3+i for i in range(3)]
            # loci is 1st,2nd or 3rd position in the codon?
            codon_shift = relative_pos - relative_codon_coord[0]
            # absolute codon coordinates (indexes in the genome):
            codon_coord = [orf_positions[_] for _ in relative_codon_coord]
            which_orf.append( (orf_id, (codon_idx,codon_shift,codon_coord) ) )
    return dict(which_orf)


I am working on the file "FluDB_coding_aln/getting_loci_interest.ipynb".
The first task is:
Let's output some characteristics of the alignment:
Can you write Python code for it?

for seg in sorted(segment_aln):
    print "%s of len %d has %d sequences aligned"%(seg,segment_aln[seg].get_alignment_length(),len(segment_aln[seg]))


I am working on the file "FluDB_coding_aln/getting_loci_interest.ipynb".
The first task is:
Our segment-wide alignments are stored in files with particular names:
Can you write Python code for it?

aligned_seg_fname = lambda number: "seg%d.afa"%number


I am working on the file "FluDB_coding_aln/getting_loci_interest.ipynb".
The first task is:
Then import the Codon Table for standard genetic code, with the slight modification - add STOP codon * as a fully-fledged member of the table:
Can you write Python code for it?


I am working on the file "FluDB_coding_aln/getting_loci_interest.ipynb".
The first task is:
in ORF or not in ORF? where exactly?
Next we define a funtion that would provide all ORFs associated with a given genome locus, along with the description of the hosting codon, and a codon-shift of a given locus (first, second, third position).
Can you write Python code for it?


I am working on the file "Foil Open Area/Open Area-pymc3.ipynb".
The first task is:
Can we get a noise estimate?
1) Try all pixels with a value where a neighbor does not. This assumes that real holes are large enough to have a point spread function and therefore cannot be in a single pixel.
Can you write Python code for it?

def neighbor_inds(x, y, xlim=(0,1023), ylim=(0,1023), center=False, mask=False):
    """
    given an x and y index return the 8 neighbor indices

    if center also return the center index
    if mask return a boolean mask over the whole 2d array
    """
    xi = np.clip([x + v for v in [-1, 0, 1]], xlim[0], xlim[1])
    yi = np.clip([y + v for v in [-1, 0, 1]], ylim[0], ylim[1])
    ans = [(i, j) for i, j in itertools.product(xi, yi)]
    if not center:
        ans.remove((x,y))
    if mask:
        out = np.zeros((np.diff(xlim)+1, np.diff(ylim)+1), dtype=np.bool)
        for c in ans:
            out[c] = True
    else:
        out = ans
    return np.asarray(out)

print(neighbor_inds(2,2))
# print(neighbor_inds(2,2, mask=True))
# print(ZZ[neighbor_inds(500, 992, mask=True)])

def get_alone_pixels(dat):
    """
    loop over all the data and store the value of all lone pixels
    """
    ans = []
    for index, x in tqdm.tqdm_notebook(np.ndenumerate(dat)):
        if (np.sum([ZZ[i, j] for i, j in neighbor_inds(index[0], index[1])]) == 0) and x != 0:
            ans.append((index, x))
    return ans
# print((neighbor_inds(5, 4)))
alone = get_alone_pixels(ZZ)
pprint(alone)
# ZZ[neighbor_inds(5, 4)[0]].shape
# print((neighbor_inds(5, 4))[0])
# print(ZZ[(neighbor_inds(5, 4))[0]].shape)
# ZZ[4,3]

ZZ[(965, 485)]

print(neighbor_inds(4,3)[0])
print(ZZ[neighbor_inds(4,3)[0]])
print(ZZ[3,2])

ni = neighbor_inds(4,3)[0]
print(ZZ[ni[0], ni[1]])

(ZZ % 2).any() # not all even any longer


I am working on the file "Foil Open Area/Open Area-pymc3.ipynb".
The first task is:
Looks to go all the way to all sides in X-Y.
Work to total open area calculations
Now we can model the total open area of the foil given the noise estimate per pixel and the pixels that are a part of the witness sample and the total area.
We model the observed background as Poisson with center at the real background:
$obsnbkg \sim Pois(nbkg)$
We model the observed witness sample, $obswit$, as Poisson with center of background per pixel times number of pixels in peak plus the number of real counts:
$obswit \sim Pois(nbkg/C + witc)$, $C = \frac{A_w}{A_t}$
This then leaves the number of counts in open areas of the system (excluding witness) as a Poisson with center of background per pixel times number of pixels in the system (less witness) plus the real number of counts.
$obsopen \sim Pois(nbkg/D + realc)$, $D=\frac{A_t - A_w}{A_t}$
Then then the open area is given by the ratio number of counts, $realc$, over an unknown area, $A_o$, as related to witness counts, $witc$, to the witness area, $A_w$, which is assumed perfect as as 0.6mm hole.
$\frac{A_o}{realc}=\frac{A_w}{witc} => A_o = \frac{A_w}{witc}realc $
Can you write Python code for it?

Aw = np.pi*(0.2/2)**2 # mm**2
Af = 182.75 # mm**2  this is the area of the foil
W_F_ratio = Aw/Af

print(Aw, Af, W_F_ratio)

C = wit_pixels/n_pixels
D = (n_pixels-wit_pixels)/n_pixels
print('C', C, 'D', D)

with mc3.Model() as model:
    nbkg = mc3.Uniform('nbkg', 1, n_noise*5) # just 1 to some large number
    obsnbkg = mc3.Poisson('obsnbkg', nbkg, observed=n_noise)

    witc = mc3.Uniform('witc', 0, witness_counts*5) # just 0 to some large number
    obswit = mc3.Poisson('obswit', nbkg*C + witc, observed=witness_counts)

    realc = mc3.Uniform('realc', 0, total_counts*5) # just 0 to some large number
    obsopen = mc3.Poisson('obsopen', nbkg*D + realc, observed=total_counts-witness_counts)

#     @mc.deterministic(plot=True)
#     def open_area(realc=realc, witc=witc):
#         return realc*Aw/witc/Af

    open_area = mc3.Deterministic('open_area', realc*Aw/witc/Af)
    start = mc3.find_MAP()
    trace = mc3.sample(20000, start=start)

mc3.traceplot(trace);
mc3.summary(trace)

print("Foil 1 \n")

witc_mean = np.mean(trace['witc'])
witc_std = np.std(trace['witc'])

print("Found witness counts of {0} turn into {1} +/- {2}  ({3:.2f}%)\n".format(witness_counts, witc_mean, witc_std, witc_std/witc_mean*100))

realc_mean = np.mean(trace['realc'])
realc_std = np.std(trace['realc'])

print("Found non-witness counts of {0} turn into {1} +/- {2}  ({3:.2f}%)\n".format(total_counts-witness_counts, realc_mean, realc_std, realc_std/realc_mean*100))

nbkg_mean = np.mean(trace['nbkg'])
nbkg_std = np.std(trace['nbkg'])

print("Found noise counts of {0} turn into {1} +/- {2}  ({3:.2f}%)\n".format(0, nbkg_mean, nbkg_std, nbkg_std/nbkg_mean*100))

OA_median = np.median(trace['open_area'])
OA_mean = np.mean(trace['open_area'])
OA_std = np.std(trace['open_area'])
print("The open area fraction is {0:.6f} +/- {1:.6f}   ({2:.2f}%) at the 1 stddev level from 1 measurement\n".format(OA_mean, OA_std,OA_std/OA_mean*100 ))
print("Phil got {0} for 1 measurement\n".format(0.00139))
print("The ratio Brian/Phil is:...


I am working on the file "Foil Open Area/Open Area-pymc3.ipynb".
The first task is:
Maybe we should consider just part of the MCP, lets get the min,max X and min,max Y where there are counts and just use that area. This will increase the cts/pixel.
Can you write Python code for it?

minx_tmp = ZZ.sum(axis=0)
minx_tmp.shape
print(minx_tmp)

miny_tmp = ZZ.sum(axis=1)
miny_tmp.shape
print(miny_tmp)


I am working on the file "Foil Open Area/Open Area-pymc3.ipynb".
The first task is:
Noise estimates
Not we assume that all lone counts are noise that can be considered random and uniform over the MCP.
This then provides a number of counts per MCA pixel that we can use.
Can you write Python code for it?

n_noise = np.sum([v[1] for v in alone])
n_pixels = 1024*1024
noise_pixel = n_noise/n_pixels
print("There were a total of {0} random counts over {1} pixels, {2} cts/pixel".format(n_noise, n_pixels, noise_pixel))


I am working on the file "Foil Open Area/Open Area-pymc3.ipynb".
The first task is:
Run again allowing some uncertainity on witness and foil areas
Can you write Python code for it?

_Aw = np.pi*(0.2/2)**2 # mm**2
_Af = 182.75 # mm**2  this is the area of the foil
print(_Aw, _Af)

with mc3.Model() as model:
    nbkg = mc3.Uniform('nbkg', 1, n_noise*5) # just 1 to some large number
    obsnbkg = mc3.Poisson('obsnbkg', nbkg, observed=n_noise)
    Aw = mc3.Normal('Aw', _Aw, sd=_Aw*0.2) # 20%
    Af = mc3.Normal('Af', _Af, sd=_Af*0.1)  # 10%

    C = wit_pixels/n_pixels
    D = (n_pixels-wit_pixels)/n_pixels

    witc = mc3.Uniform('witc', 0, witness_counts*5) # just 0 to some large number
    obswit = mc3.Poisson('obswit', nbkg*C + witc, observed=witness_counts)

    realc = mc3.Uniform('realc', 0, total_counts*5) # just 0 to some large number
    obsopen = mc3.Poisson('obsopen', nbkg*D + realc, observed=total_counts-witness_counts)

#     @mc.deterministic(plot=True)
#     def open_area(realc=realc, witc=witc):
#         return realc*Aw/witc/Af

    open_area = mc3.Deterministic('open_area', realc*Aw/witc/Af)
    start = mc3.find_MAP()
    trace = mc3.sample(20000, start=start)

mc3.traceplot(trace);
mc3.summary(trace)

print("Foil 1 \n")

witc_mean = np.mean(trace['witc'])
witc_std = np.std(trace['witc'])
print("Found witness counts of {0} turn into {1} +/- {2}  ({3:.2f}%)\n".format(witness_counts, witc_mean, witc_std, witc_std/witc_mean*100))

realc_mean = np.mean(trace['realc'])
realc_std = np.std(trace['realc'])

print("Found non-witness counts of {0} turn into {1} +/- {2}  ({3:.2f}%)\n".format(total_counts-witness_counts, realc_mean, realc_std, realc_std/realc_mean*100))

nbkg_mean = np.mean(trace['nbkg'])
nbkg_std = np.std(trace['nbkg'])

print("Found noise counts of {0} turn into {1} +/- {2}  ({3:.2f}%)\n".format(0, nbkg_mean, nbkg_std, nbkg_std/nbkg_mean*100))

OA_median = np.median(trace['open_area'])
OA_mean = np.mean(trace['open_area'])
OA_std = np.std(trace['open_area'])
print("The open area fraction is {0:.6f} +/- {1:.6f}   ({2:.2f}%) at the 1 stddev level from 1 measurement\n".format(OA_mean, OA_std,OA_std/OA_mean*100 ))
print("Phil got {0} for 1...


I am working on the file "Foil Open Area/Open Area.ipynb".
The first task is:
Can we get a noise estimate?
1) Try all pixels with a value where a neighbor does not. This assumes that real holes are large enough to have a point spread function and therefore cannot be in a single pixel.
Can you write Python code for it?

def neighbor_inds(x, y, xlim=(0,1023), ylim=(0,1023), center=False, mask=False):
    """
    given an x and y index return the 8 neighbor indices

    if center also return the center index
    if mask return a boolean mask over the whole 2d array
    """
    xi = np.clip([x + v for v in [-1, 0, 1]], xlim[0], xlim[1])
    yi = np.clip([y + v for v in [-1, 0, 1]], ylim[0], ylim[1])
    ans = [(i, j) for i, j in itertools.product(xi, yi)]
    if not center:
        ans.remove((x,y))
    if mask:
        out = np.zeros((np.diff(xlim)+1, np.diff(ylim)+1), dtype=np.bool)
        for c in ans:
            out[c] = True
    else:
        out = ans
    return np.asarray(out)

print(neighbor_inds(2,2))
print(neighbor_inds(2,2, mask=True))
print(ZZ[neighbor_inds(500, 992, mask=True)])

def get_alone_pixels(dat):
    """
    loop over all the data and store the value of all lone pixels
    """
    ans = []
    for index, x in tqdm.tqdm_notebook(np.ndenumerate(dat)):
        if (np.sum([ZZ[i, j] for i, j in neighbor_inds(index[0], index[1])]) == 0) and x != 0:
            ans.append((index, x))
    return ans
# print((neighbor_inds(5, 4)))
alone = get_alone_pixels(ZZ)
pprint(alone)
# ZZ[neighbor_inds(5, 4)[0]].shape
# print((neighbor_inds(5, 4))[0])
# print(ZZ[(neighbor_inds(5, 4))[0]].shape)
# ZZ[4,3]

ZZ[(965, 485)]

print(neighbor_inds(4,3)[0])
print(ZZ[neighbor_inds(4,3)[0]])
print(ZZ[3,2])

ni = neighbor_inds(4,3)[0]
print(ZZ[ni[0], ni[1]])

(ZZ % 2).any() # not all even any longer


I am working on the file "Foil Open Area/Open Area.ipynb".
The first task is:
Looks to go all the way to all sides in X-Y.
Work to total open area calculations
Now we can model the total open area of the foil given the noise estimate per pixel and the pixels that are a part of the witness sample and the total area.
We model the observed background as Poisson with center at the real background:
$obsnbkg \sim Pois(nbkg)$
We model the observed witness sample, $obswit$, as Poisson with center of background per pixel times number of pixels in peak plus the number of real counts:
$obswit \sim Pois(nbkg/C + witc)$, $C = \frac{A_w}{A_t}$
This then leaves the number of counts in open areas of the system (excluding witness) as a Poisson with center of background per pixel times number of pixels in the system (less witness) plus the real number of counts.
$obsopen \sim Pois(nbkg/D + realc)$, $D=\frac{A_t - A_w}{A_t}$
Then then the open area is given by the ratio number of counts, $realc$, over an unknown area, $A_o$, as related to witness counts, $witc$, to the witness area, $A_w$, which is assumed perfect as as 0.6mm hole.
$\frac{A_o}{realc}=\frac{A_w}{witc} => A_o = \frac{A_w}{witc}realc $
Can you write Python code for it?

Aw = np.pi*(0.2/2)**2 # mm**2
Af = 182.75 # mm**2  this is the area of the foil
W_F_ratio = Aw/Af

print(Aw, Af, W_F_ratio)

C = wit_pixels/n_pixels
D = (n_pixels-wit_pixels)/n_pixels
print('C', C, 'D', D)

nbkg = mc.Uniform('nbkg', 1, n_noise*5) # just 1 to some large number
obsnbkg = mc.Poisson('obsnbkg', nbkg, observed=True, value=n_noise)

witc = mc.Uniform('witc', 0, witness_counts*5) # just 0 to some large number
obswit = mc.Poisson('obswit', nbkg*C + witc, observed=True, value=witness_counts)

realc = mc.Uniform('realc', 0, total_counts*5) # just 0 to some large number
obsopen = mc.Poisson('obsopen', nbkg*D + realc, observed=True, value=total_counts-witness_counts)

@mc.deterministic(plot=True)
def open_area(realc=realc, witc=witc):
    return realc*Aw/witc/Af

model = mc.MCMC([nbkg, obsnbkg, witc, obswit, realc, obsopen, open_area])

model.sample(200000, burn=100, thin=30, burn_till_tuned=True)
mc.Matplot.plot(model)

# 1000, burn=100, thin=30    0.000985 +/- 0.000058
# 10000, burn=100, thin=30   0.000982 +/- 0.000061
# 100000, burn=100, thin=30  0.000984 +/- 0.000059
# 200000, burn=100, thin=30  0.000986 +/- 0.000059
# 1000000, burn=100, thin=30 0.000985 +/- 0.000059

print("Foil 1 \n")

witc_mean = np.mean(witc.trace()[...])
witc_std = np.std(witc.trace()[...])

print("Found witness counts of {0} turn into {1} +/- {2}  ({3:.2f}%)\n".format(witness_counts, witc_mean, witc_std, witc_std/witc_mean*100))

realc_mean = np.mean(realc.trace()[...])
realc_std = np.std(realc.trace()[...])

print("Found non-witness counts of {0} turn into {1} +/- {2}  ({3:.2f}%)\n".format(total_counts-witness_counts, realc_mean, realc_std, realc_std/realc_mean*100))

nbkg_mean = np.mean(nbkg.trace()[...])
nbkg_std = np.std(nbkg.trace()[...])

print("Found noise counts of {0} turn into {1} +/- {2}  ({3:.2f}%)\n".format(0, nbkg_mean, nbkg_std, nbkg_std/nbkg_mean*100))

OA_median = np.median(open_area.trace()[...])
OA_mean = np.mean(open_area.trace()[...])
OA_std = np.std(open_area.trace()[...])
print("The open area fraction is...


I am working on the file "Foil Open Area/Open Area.ipynb".
The first task is:
Maybe we should consider just part of the MCP, lets get the min,max X and min,max Y where there are counts and just use that area. This will increase the cts/pixel.
Can you write Python code for it?


I am working on the file "Foil Open Area/Open Area.ipynb".
The first task is:
Noise estimates
Not we assume that all lone counts are noise that can be considered random and uniform over the MCP.
This then provides a number of counts per MCA pixel that we can use.
Can you write Python code for it?


I am working on the file "Fonksiyonlar.ipynb".
The first task is:
1'den başlayarak, verilen bir sayıya kadar tamsayıları toplayan bir fonksiyon:
Can you write Python code for it?

def toplam(N):
    t = 0
    i = 1
    while i<=N:
        t += i
        i += 1
    return t

print toplam(100)


I am working on the file "Fonksiyonlar.ipynb".
The first task is:
Artık dizgekucuk değişkenine doğru şekilde atama yapabiliriz.
Can you write Python code for it?

dizge = "IĞDIR KEŞKÜL CENNETİ"
dizgekucuk = lowertr(tat)
print dizge_kucuk


I am working on the file "Fonksiyonlar.ipynb".
The first task is:
Basit birkaç sayısal örnek yapalım.
Örneğin, bir dikdörtgenin iki kenarını parametre olarak alıp, alanını veren bir fonksiyon yazalım. İkinci kenar uzunluğunun varsayılan değeri 1 olsun.
Can you write Python code for it?

def dikd_alan(en, boy=1):
    return en*boy

print dikd_alan(3,5)
print dikd_alan(3)
print dikd_alan(boy=2, en=5)


I am working on the file "Fonksiyonlar.ipynb".
The first task is:
Bu sorunu herhangi bir dize içindeki harf gruplarını değiştirmek için kullanılan replace fonksiyonu ile çözebiliriz.
Can you write Python code for it?

il = "IĞDIR"
il = il.replace("Ğ", "ğ")
il = il.replace("I", "ı")
print il.lower()


I am working on the file "Fonksiyonlar.ipynb".
The first task is:
Bunu genelleştirelim; ilk'den N'ye kadar adim adımlarla ilerleyerek elde edilen aritmetik dizinin toplamını veren bir fonksiyon yazalım. Varsayılan değerler ilk için 0 ve adim için 1 olsun.
Can you write Python code for it?


I am working on the file "Fonksiyonlar.ipynb".
The first task is:
Burada bir noktaya dikkat etmeli. Varsayılan parametreler her zaman diğer parametrelerden sonra yazılmak zorundalar.
Parametrelerin varsayılan değerlerini girmenin getirdiği bir kolaylık daha var. Bu tür parametrelerden birkaç tane varsa, isimleri verilerek herhangi bir sırayla girilebiliyorlar. Bunun en güzel örneği grafik çizmede kullanılan plot fonksiyonu. Aşağıdaki örnekteki linewidth, color ve linestyle parametreleri x ve y parametrelerinden sonra istenilen sırada girilebilir.
Can you write Python code for it?

% pylab inline
x = arange(0.0, 2.0, 0.01)
y = cos(2*pi*t)
plot(x, y, linewidth=2, color="red", linestyle="dashed")


I am working on the file "Fonksiyonlar.ipynb".
The first task is:
Dikkat ederseniz daha önce tanımladığımız lowertr fonksiyonunu yeni fonksiyonumuzun ilk seçeneğinde kullandık.
Can you write Python code for it?

print kucukbuyuk("Iğdır", 1)
print kucukbuyuk("Keşkül", 0)


I am working on the file "Fonksiyonlar.ipynb".
The first task is:
Fonksiyonumuzun ismi lowertr ve kelime isimli parametreyi alıyor. Hemen deneyelim.
Can you write Python code for it?


I am working on the file "Fonksiyonlar.ipynb".
The first task is:
Her seferinde 1 ya da 0 diye seçeneği girmek insanın canını sıkabilir. Özellikle de çoğu zaman küçük harflere dönüştürmeyi kullanacaksanız. Python'da bunun da çaresi var. Tek yapmamız gereken bazı parametrelerin varsayılan değerlerini girmek.
Can you write Python code for it?

def kucukbuyuk(kelime, secenek=1):
    if (secenek):
        kelime = lowertr(kelime)
    else:
        kelime = kelime.replace("ö", "Ö")
        kelime = kelime.replace("ü", "Ü")
        kelime = kelime.replace("i", "İ")
        kelime = kelime.replace("ğ", "Ğ")
        kelime = kelime.replace("ş", "Ş")
        kelime = kelime.replace("ç", "Ç")
        kelime = kelime.replace("ı", "I")
        kelime = kelime.upper()

    return kelime