I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
An illustrative MOP: Dent
$$
\begin{array}{rl}
\text{minimize} & f_1(\mathbf{x}),f_2(\mathbf{x}) \
\text{such that} & f_1(\mathbf{x}) = \frac{1}{2}\left( \sqrt{1 + (x_1 + x_2)^2} \sqrt{1 + (x_1 - x_2)^2} + x_1 -x_2\right) + d,\
 & f_2(\mathbf{x}) = \frac{1}{2}\left( \sqrt{1 + (x_1 + x_2)^2} \sqrt{1 + (x_1 - x_2)^2} - x_1 -x_2\right) + d,\
\text{with}& d = \lambda e^{-\left(x_1-x_2\right)^2}\ (\text{generally }\lambda=0.85) \text{ and }\
& \mathbf{x}\in \left[-1.5,1.5\right]^2.
\end{array}
$$
Implementing the Dent problem
Can you write Python code for it?

def dent(individual, lbda = 0.85):
    """ Implements the test problem Dent
    Num. variables = 2; bounds in [-1.5, 1.5]; num. objetives = 2.
    @author Cesar Revelo"""
    d  = lbda * math.exp(-(individual[0] - individual[1]) ** 2)
    f1 = 0.5 * (math.sqrt(1 + (individual[0] + individual[1]) ** 2) + \
                math.sqrt(1 + (individual[0] - individual[1]) ** 2) + \
                individual[0] - individual[1]) + d
    f2 = 0.5 * (math.sqrt(1 + (individual[0] + individual[1]) ** 2) + \
                math.sqrt(1 + (individual[0] - individual[1]) ** 2) - \
                individual[0] + individual[1]) + d
    return f1, f2


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
As you can see, even this relatively small experiment took lots of time!
As running the experiments takes so long, lets save the results so we can use them whenever we want.
Can you write Python code for it?

import pickle

pickle.dump(results, open('nsga_ii_dtlz3-results.pickle', 'wb'))


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Creating this animation takes more programming effort.
Can you write Python code for it?

class MultiProblemAnimation:
    def init(self, fig, results):
        self.results = results
        self.axs = [fig.add_subplot(3,4,i+1) for i in range(len(results))]
        self.plots =[]
        for i, problem in enumerate(sorted(results)):
            (res, logbook) = self.results[problem]
            pop = pd.DataFrame(data=logbook.select('obj_vals')[0])
            plot = self.axs[i].plot(pop[0], pop[1], 'b.', alpha=0.47)[0]
            self.plots.append(plot)
        fig.tight_layout()

    def animate(self, t):
        'Updates all plots to match frame _i_ of the animation.'
        for i, problem in enumerate(sorted(results)):
            #self.axs[i].clear()
            (res, logbook) = self.results[problem]
            pop = pd.DataFrame(data=logbook.select('obj_vals')[t])
            self.plots[i].set_data(pop[0], pop[1])
            self.axs[i].set_title(problem + '; $t=' + str(t)+'$')
            self.axs[i].set_xlim((0, max(1,pop.max()[0])))
            self.axs[i].set_ylim((0, max(1,pop.max()[1])))
        return self.axs

mpa = MultiProblemAnimation()

fig = plt.figure(figsize=(14,6))
anim = animation.FuncAnimation(fig, mpa.animate, init_func=mpa.init(fig,results),
                               frames=toolbox.max_gen, interval=60, blit=True)
anim


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
DTLZ7 has many disconnected Pareto-optimal fronts.

Can you write Python code for it?

def dtlz7(ind, n_objs):
    gval = 1 + 9.0 / len(ind[n_objs-1:]) * sum([a for a in ind[n_objs-1:]])
    fit = [ind for ind in ind[:n_objs-1]]
    fit.append((1 + gval) * (n_objs - sum([a / (1.0 + gval) * (1 + math.sin(3 * math.pi * a)) for a in ind[:n_objs-1]])))
    return fit


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Defining attributes, individuals and population.
Can you write Python code for it?

def uniform(low, up, size=None):
    try:
        return [random.uniform(a, b) for a, b in zip(low, up)]
    except TypeError:
        return [random.uniform(a, b) for a, b in zip([low] * size, [up] * size)]

toolbox.register("attr_float", uniform, BOUND_LOW, BOUND_UP, NDIM)
toolbox.register("individual", tools.initIterate,
                 creator.Individual, toolbox.attr_float)
toolbox.register("population", tools.initRepeat, list,
                 toolbox.individual)


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Describing attributes, individuals and population and defining the selection, mating and mutation operators.
Can you write Python code for it?

toolbox.register("attr_float", uniform, BOUND_LOW, BOUND_UP, NDIM)
toolbox.register("individual", tools.initIterate, creator.Individual, toolbox.attr_float)
toolbox.register("population", tools.initRepeat, list, toolbox.individual)
toolbox.register("mate", tools.cxSimulatedBinaryBounded, low=BOUND_LOW, up=BOUND_UP, eta=20.0)
toolbox.register("mutate", tools.mutPolynomialBounded, low=BOUND_LOW, up=BOUND_UP, eta=20.0, indpb=1.0/NDIM)
toolbox.register("select", tools.selNSGA2)


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Having a_given_individual (blue dot) we can now plot those that are dominated by it (in green), those that dominate it (in red) and those that are uncomparable.
Can you write Python code for it?


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Here it is clearly visible how the algorithm "jumps" from one local-optimum to a better one as evolution takes place.
MOP benchmark problem toolkits
Each problem instance is meant to test the algorithms with regard with a given feature: local optima, convexity, discontinuity, bias, or a combination of them.

ZDT1-6: Two-objective problems with a fixed number of decision variables.
E. Zitzler, K. Deb, and L. Thiele. Comparison of Multiobjective Evolutionary Algorithms: Empirical Results. Evolutionary Computation, 8(2):173-195, 2000. (pdf)

DTLZ1-7: $m$-objective problems with $n$ variables.

K. Deb, L. Thiele, M. Laumanns and E. Zitzler. Scalable Multi-Objective Optimization Test Problems. CEC 2002, p. 825 - 830, IEEE Press, 2002. (pdf)

CEC'09: Two- and three- objective problems that very complex Pareto sets.

Zhang, Q., Zhou, A., Zhao, S., & Suganthan, P. N. (2009). Multiobjective optimization test instances for the CEC 2009 special session and competition. In 2009 IEEE Congress on Evolutionary Computation (pp. 1–30). (pdf)

WFG1-9: $m$-objective problems with $n$ variables, very complex.
Huband, S., Hingston, P., Barone, L., & While, L. (2006). A review of multiobjective test problems and a scalable test problem toolkit. IEEE Transactions on Evolutionary Computation, 10(5), 477–506. doi:10.1109/TEVC.2005.861417

DTLZ5 and DTLZ6 have an $m-1$-dimensional Pareto-optimal front.
* This means that in 3D the Pareto optimal front is a 2D curve.

In two dimensions the front is a point.
Can you write Python code for it?

def dtlz5(ind, n_objs):
    from functools import reduce
    g = lambda x: sum([(a - 0.5)**2 for a in x])
    gval = g(ind[n_objs-1:])

    theta = lambda x: math.pi / (4.0 * (1 + gval)) * (1 + 2 * gval * x)
    fit = [(1 + gval) * math.cos(math.pi / 2.0 * ind[0]) *
           reduce(lambda x,y: x*y, [math.cos(theta(a)) for a in ind[1:]])]
    for m in reversed(range(1, n_objs)):
        if m == 1:
            fit.append((1 + gval) * math.sin(math.pi / 2.0 * ind[0]))
        else:
            fit.append((1 + gval) * math.cos(math.pi / 2.0 * ind[0]) *
                       reduce(lambda x,y: x*y, [math.cos(theta(a)) for a in ind[1:m-1]], 1) *
                       math.sin(theta(ind[m-1])))
    return fit

def dtlz6(ind, n_objs):
    from functools import reduce
    gval = sum([a**0.1 for a in ind[n_objs-1:]])
    theta = lambda x: math.pi / (4.0 * (1 + gval)) * (1 + 2 * gval * x)
    fit = [(1 + gval) * math.cos(math.pi / 2.0 * ind[0]) *
           reduce(lambda x,y: x*y, [math.cos(theta(a)) for a in ind[1:]])]
    for m in reversed(range(1, n_objs)):
        if m == 1:
            fit.append((1 + gval) * math.sin(math.pi / 2.0 * ind[0]))
        else:
            fit.append((1 + gval) * math.cos(math.pi / 2.0 * ind[0]) *
                       reduce(lambda x,y: x*y, [math.cos(theta(a)) for a in ind[1:m-1]], 1) *
                       math.sin(theta(ind[m-1])))
    return fit


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
How can we interpret the indicators?
Option A: Tabular form
Can you write Python code for it?


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
How does our NSGA-II behaves when faced with different benchmark problems?
Can you write Python code for it?

problem_instances = {'ZDT1': benchmarks.zdt1, 'ZDT2': benchmarks.zdt2,
                     'ZDT3': benchmarks.zdt3, 'ZDT4': benchmarks.zdt4,
                     'DTLZ1': lambda ind: benchmarks.dtlz1(ind,2),
                     'DTLZ2': lambda ind: benchmarks.dtlz2(ind,2),
                     'DTLZ3': lambda ind: benchmarks.dtlz3(ind,2),
                     'DTLZ4': lambda ind: benchmarks.dtlz4(ind,2, 100),
                     'DTLZ5': lambda ind: dtlz5(ind,2),
                     'DTLZ6': lambda ind: dtlz6(ind,2),
                     'DTLZ7': lambda ind: dtlz7(ind,2)}

toolbox.max_gen = 1000

stats = tools.Statistics(lambda ind: ind.fitness.values)
stats.register("obj_vals", np.copy)

def run_problem(toolbox, problem):
    toolbox.register('evaluate', problem)
    return nsga_ii(toolbox, stats=stats)


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
It is better to make an animated plot of the evolution as it takes place.

Animating the evolutionary process
We create a stats to store the individuals not only their objective function values.
Can you write Python code for it?

stats = tools.Statistics()
stats.register("pop", copy.deepcopy)

toolbox.max_gen = 4000 # we need more generations!


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
It is interesting how the algorithm deals with each problem: clearly some problems are harder than others.
In some cases it "hits" the Pareto front and then slowly explores it.

Experiment design and reporting results

Watching an animation of an EMO algorithm solve a problem is certainly fun.
It also allows us to understand many particularities of the problem being solved.
But, as Carlos Coello would say, we are not in an art appreciation class.
We should follow the key concepts provided by the scientific method.
I urge you to study the experimental design topic in depth, as it is an essential knowledge.

**Evolutionary algorithms are stochastic algorithms; therefore their results must be assessed by repeating experiments until you reach an statistically valid conclusion.**

We need to evaluate performance

Closeness to the Pareto-optimal front.
Diversity of solutions.
Coverage of the Pareto-optimal fronts.

Research hint! The design analysis and application of performance indicators is one of the main research topic in the EMO field.

The hypervolume indicator

Note: Taken from Günter Rudolph's site on the hypervolume indicator.
Formalization of the hypervolume
For a set of solutions $\mathcal{A}$,
$$
I_\mathrm{hyp}\left(\mathcal{A}\right) = \mathrm{volume}\left(
                    \bigcup_{\forall \mathbf{a}\in\mathcal{A}}{\mathrm{hypercube}(\mathbf{a},\mathbf{r})}\right)\,.
$$

We need a reference point, $\mathbf{r}$.
Hypervolume is Pareto compliant (Fleischer, 2003): for sets $\mathcal{A}$ and $\mathcal{B}$, $\mathcal{A}\preccurlyeq\mathcal{B} \implies...
Can you write Python code for it?

toolbox = base.Toolbox()

BOUND_LOW, BOUND_UP = 0.0, 1.0
NDIM = 30

# the explanation of this... a few lines bellow
def eval_helper(ind):
    return benchmarks.dtlz3(ind, 2)

toolbox.register("evaluate", eval_helper)

def uniform(low, up, size=None):
    try:
        return [random.uniform(a, b) for a, b in zip(low, up)]
    except TypeError:
        return [random.uniform(a, b) for a, b in zip([low] * size, [up] * size)]

toolbox.register("attr_float", uniform, BOUND_LOW, BOUND_UP, NDIM)
toolbox.register("individual", tools.initIterate, creator.Individual, toolbox.attr_float)
toolbox.register("population", tools.initRepeat, list, toolbox.individual)

toolbox.register("mate", tools.cxSimulatedBinaryBounded, low=BOUND_LOW, up=BOUND_UP, eta=20.0)
toolbox.register("mutate", tools.mutPolynomialBounded, low=BOUND_LOW, up=BOUND_UP, eta=20.0, indpb=1.0/NDIM)

toolbox.register("select", tools.selNSGA2)

toolbox.pop_size = 200
toolbox.max_gen = 500


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Let's also use the toolbox to store other configuration parameters of the algorithm. This will show itself usefull when performing massive experiments.
Can you write Python code for it?

toolbox.pop_size = 50
toolbox.max_gen = 500
toolbox.mut_prob = 0.2


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW), Wilcoxon rank-sum test (WRS), or Wilcoxon–Mann–Whitney test) is a nonparametric test of the null hypothesis that two populations are the same against an alternative hypothesis, especially that a particular population tends to have larger values than the other.
It has greater efficiency than the $t$-test on non-normal distributions, such as a mixture of normal distributions, and it is nearly as efficient as the $t$-test on normal distributions.
Can you write Python code for it?


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Note: Bear in mind that DEAP implements a Pareto dominance relation that probably is more efficient than this implementation. The previous function would be something like:
Can you write Python code for it?

def efficient_pareto_dominance(ind1, ind2):
    return tools.emo.isDominated(ind1.fitness.values, ind2.fitness.values)


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Now toolboxes is a list of copies of the same toolbox. One for each experiment configuration (population size).
...but we still have to set the population sizes in the elements of toolboxes.
Can you write Python code for it?

for i, toolbox in enumerate(toolboxes):
    toolbox.mut_prob = mut_probs[i]
    toolbox.experiment_name = toolbox.experiment_name + str(mut_probs[i]) +'$'

for toolbox in toolboxes:
    print(toolbox.experiment_name, toolbox.mut_prob)


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Obtaining the nondominated front.
Can you write Python code for it?

non_dom = tools.sortNondominated(example_pop, k=len(example_pop),
                                 first_front_only=True)[0]

plt.figure(figsize=(5,5))
for ind in example_pop:
    plt.plot(ind.fitness.values[0], ind.fitness.values[1], 'k.', ms=3, alpha=0.5)
for ind in non_dom:
    plt.plot(ind.fitness.values[0], ind.fitness.values[1], 'bo', alpha=0.74, ms=5)


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Option B: Visualization
Can you write Python code for it?

import seaborn
seaborn.set(style="whitegrid")

fig = plt.figure(figsize=(15,3))
plt.subplot(1,2,1, title='Violin plots of NSGA-II with $P_{\mathrm{mut}}$')
seaborn.violinplot(hypervols, alpha=0.74)
plt.ylabel('Hypervolume'); plt.xlabel('Mutation probabilities')
plt.subplot(1,2,2, title='Box plots of NSGA-II with $P_{\mathrm{mut}}$')
seaborn.boxplot(hypervols, alpha=0.74)
plt.ylabel('Hypervolume'); plt.xlabel('Mutation probabilities');


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Option C: Statistical hypothesis test

Choosing the correct statistical test is essential to properly report the results.
Nonparametric statistics can lend a helping hand.
Parametric statistics could be a better choice in some cases.
Parametric statistics require that all data follow a known distribution (frequently a normal one).
Some tests -like the normality test- can be apply to verify that data meet the parametric stats requirements.
In my experience that is very unlikely that all your EMO result meet those characteristics.

We start by writing a function that helps us tabulate the results of the application of an statistical hypothesis test.
Can you write Python code for it?

import itertools
import scipy.stats as stats

def compute_stat_matrix(data, stat_func, alpha=0.05):
    '''A function that applies `stat_func` to all combinations of columns in `data`.
    Returns a squared matrix with the p-values'''
    p_values = pd.DataFrame(columns=data.columns, index=data.columns)
    for a,b in itertools.combinations(data.columns,2):
        s,p = stat_func(data[a], data[b])
        p_values[a].ix[b] = p
        p_values[b].ix[a] = p
    return p_values


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Planting a constant seed to always have the same results (and avoid surprises in class). -you should not do this in a real-world case!
Can you write Python code for it?


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Re-run the algorithm to get the data necessary for plotting.
Can you write Python code for it?

%time res, logbook = nsga_ii(toolbox, stats=stats)

from JSAnimation import IPython_display
import matplotlib.colors as colors
from matplotlib import animation

def animate(frame_index, logbook):
    'Updates all plots to match frame _i_ of the animation.'
    ax.clear()
    fronts = tools.emo.sortLogNondominated(logbook.select('pop')[frame_index],
                                           len(logbook.select('pop')[frame_index]))
    for i,inds in enumerate(fronts):
        par = [toolbox.evaluate(ind) for ind in inds]
        df = pd.DataFrame(par)
        df.plot(ax=ax, kind='scatter', label='Front ' + str(i+1),
                 x=df.columns[0], y =df.columns[1], alpha=0.47,
                 color=plot_colors[i % len(plot_colors)])

    ax.set_title('$t=$' + str(frame_index))
    ax.set_xlabel('$f_1(\mathbf{x})$');ax.set_ylabel('$f_2(\mathbf{x})$')
    return None

fig = plt.figure(figsize=(4,4))
ax = fig.gca()
anim = animation.FuncAnimation(fig, lambda i: animate(i, logbook),
                               frames=len(logbook), interval=60,
                               blit=True)
anim


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Running NSGA-II solving all problems. Now it takes longer.
Can you write Python code for it?

%time results = {problem: run_problem(toolbox, problem_instances[problem]) \
                 for problem in problem_instances}


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Running experiments in parallel
As we are now solving more demanding problems it would be nice to make our algorithms to run in parallel and profit from modern multi-core CPUs.

In DEAP it is very simple to parallelize an algorithm (if it has been properly programmed) by providing a parallel map() function throu the toolbox.
Local parallelization can be achieved using Python's multiprocessing or concurrent.futures modules.
Cluster parallelization can be achived using IPython Parallel or SCOOP, that seems to be recommended by the DEAP guys as it was part of it.

Note: You can have a very good summary about this issue in http://blog.liang2.tw/2014-handy-dist-computing/.
Progress feedback

Another issue with these long experiments has to do being patient.
A little bit of feedback on the experiment execution would be cool.
We can use the integer progress bar from IPython widgets and report every time an experiment run is finished.
Can you write Python code for it?

from IPython.html import widgets
from IPython.display import display

progress_bar = widgets.IntProgressWidget(description="Starting...",
                                         max=len(toolboxes)*number_of_runs)


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Running the algorithm
We are now ready to run our NSGA-II.
Can you write Python code for it?


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Saving the animation as video and uploading it to YouTube.
Can you write Python code for it?

anim.save('nsgaii-benchmarks.mp4', fps=15, bitrate=-1, dpi=500)

YouTubeVideo('8t-aWcpDH0U')


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
So, is this the end?

Ok, now we know how to solve MOPs by sampling the search space.
MOPs, in the general case are NP-hard problems.
Brute force is never the solution in just a-little-more-complex cases.

An example, solving the TSP problem using brute force:

$n$ cities	time

10	3 secs

12	3 secs × 11 × 12 = 6.6 mins

14	6.6 mins × 13 × 14 = 20 hours

24	3 secs × 24! / 10! = 16 billion years

Note: See my PhD EC course notebooks https://github.com/lmarti/evolutionary-computation-course on solving the TSP problem using EAs.
Preference-based alternatives

A Decision Maker can define a set of weights $w_1,\ldots,w_M$ for each function $f_1(),\ldots,f_M()$.
We can convert a MOP into a SOP:

$$
\begin{array}{rl}
\mathrm{minimize}             & F(\mathbf{x})= w_1f_1(\mathbf{x})+\cdots + w_if_i(\mathbf{x}) +\cdots +w_Mf_M(\mathbf{x})\,,\
\mathrm{subject}\ \mathrm{to} & c_1(\mathbf{x}),\ldots,c_C(\mathbf{x})\le 0\,,\
                              & d_1(\mathbf{x}),\ldots,d_D(\mathbf{x})= 0\,,\
& \text{with}\ \mathbf{x}\in\mathcal{D}\,,
\end{array}
$$

A single-objective optimizer $\implies$ only one solution not the complete PF.
Mathematical programming.
Requires (a lot) of a priori knowledge but is relatively simple.

The Decision Maker

Lexicographical ordering of the objectives

 All objectives are important...

...but some objectives are more important than others.

Better idea: Use the Pareto dominance relation to guide the search

We can use the Pareto dominance...
Can you write Python code for it?

toolbox = base.Toolbox()

BOUND_LOW, BOUND_UP = 0.0, 1.0
toolbox.register("evaluate", lambda ind: benchmarks.dtlz3(ind, 2))


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
The Kruskal-Wallis H-test tests the null hypothesis that the population median of all of the groups are equal.

It is a non-parametric version of ANOVA.
The test works on 2 or more independent samples, which may have different sizes.
Note that rejecting the null hypothesis does not indicate which of the groups differs.
Post-hoc comparisons between groups are required to determine which groups are different.
Can you write Python code for it?

stats.kruskal(*[hypervols[col] for col in hypervols.columns])


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
The familywise error rate (FWER) is the probability of making one or more false discoveries, or type I errors, among all the hypotheses when performing multiple hypotheses tests.
Example: When performing a test, there is a $\alpha$ chance of making a type I error. If we make $m$ tests, then the probability of making one type I error is $m\alpha$. Therefore, if an $\alpha=0.05$ is used and 5 pairwise comparisons are made, we will have a $5\times0.05 = 0.25$ chance of making a type I error.

FWER procedures (such as the Bonferroni correction) exert a more stringent control over false discovery compared to False discovery rate controlling procedures.
FWER controlling seek to reduce the probability of even one false discovery, as opposed to the expected proportion of false discoveries.
Thus, FDR procedures have greater power at the cost of increased rates of type I errors, i.e., rejecting the null hypothesis of no effect when it should be accepted.

One of these corrections is the Šidák correction as it is less conservative than the Bonferroni correction:
$$\alpha_{SID} = 1-(1-\alpha)^\frac{1}{m},$$
where $m$ is the number of tests.

In our case $m$ is the number of combinations of algorithm configurations taken two at a time.
There are other corrections that can be used.
Can you write Python code for it?

from scipy.misc import comb
alpha=0.05
alpha_sid = 1 - (1-alpha)**(1/comb(len(hypervols.columns), 2))
alpha_sid


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
The local Pareto-optimal fronts are clearly visible!
Calculating performance indicators

As already mentioned, we need to evaluate the quality of the solutions produced in every execution of the algorithm.
We will use the hypervolumne indicator for that.
We already filtered each population a leave only the non-dominated individuals.

Calculating the reference point: a point that is “worst” than any other individual in every objective.
Can you write Python code for it?

def calculate_reference(results, epsilon=0.1):
    alldata = np.concatenate(np.concatenate(results.values))
    obj_vals = [toolbox.evaluate(ind) for ind in alldata]
    return np.max(obj_vals, axis=0) + epsilon

reference = calculate_reference(res)

reference


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
The previous animation makes the notebook too big for online viewing. To circumvent this, it is better to save the animation as video and (manually) upload it to YouTube.
Can you write Python code for it?

anim.save('nsgaii-dtlz3.mp4', fps=15, bitrate=-1, dpi=500)

from IPython.display import YouTubeVideo
YouTubeVideo('Cm7r4cJq59s')


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
To start, lets have a visual example of the Pareto dominance relationship in action.
In this notebook we will deal with two-objective problems in order to simplify visualization.
Therefore, we can create:
Can you write Python code for it?

creator.create("FitnessMin", base.Fitness, weights=(-1.0,-1.0))
creator.create("Individual", array.array, typecode='d',
               fitness=creator.FitnessMin)


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
Visualizing Dent
Can you write Python code for it?

plt.figure(figsize=(11,5))
plt.subplot(1,2,1)
for ind in example_pop: plt.plot(ind[0], ind[1], 'k.', ms=3)
plt.xlabel('$x_1$');plt.ylabel('$x_2$');plt.title('Decision space');
plt.subplot(1,2,2)
for ind in example_pop: plt.plot(ind.fitness.values[0], ind.fitness.values[1], 'k.', ms=3)
plt.xlabel('$f_1(\mathbf{x})$');plt.ylabel('$f_2(\mathbf{x})$');
plt.xlim((0.5,3.6));plt.ylim((0.5,3.6)); plt.title('Objective space');


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
We add a experiment_name to toolbox that we will fill up later on.
Can you write Python code for it?


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
We also need a_given_individual.
Can you write Python code for it?

a_given_individual = toolbox.population(n=1)[0]
a_given_individual[0] = 0.5
a_given_individual[1] = 0.5

a_given_individual.fitness.values = toolbox.evaluate(a_given_individual)


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
We now can assert that the results are not the same but which ones are different or similar to the others the others?
In case that the null hypothesis of the Kruskal-Wallis is rejected the Conover–Inman procedure (Conover, 1999, pp. 288-290) can be applied in a pairwise manner in order to determine if the results of one algorithm were significantly better than those of the other.

Conover, W. J. (1999). Practical Nonparametric Statistics. John Wiley & Sons, New York, 3rd edition.

Note: If you want to get an extended summary of this method check out my PhD thesis.
Can you write Python code for it?

def conover_inman_procedure(data, alpha=0.05):
    num_runs = len(data)
    num_algos = len(data.columns)
    N = num_runs*num_algos

    _,p_value = stats.kruskal(*[data[col] for col in data.columns])

    ranked =  stats.rankdata(np.concatenate([data[col] for col in data.columns]))

    ranksums = []
    for i in range(num_algos):
        ranksums.append(np.sum(ranked[num_runs*i:num_runs*(i+1)]))

    S_sq = (np.sum(ranked**2) - N*((N+1)**2)/4)/(N-1)

    right_side = stats.t.cdf(1-(alpha/2), N-num_algos) * \
                 math.sqrt((S_sq*((N-1-p_value)/(N-1)))*2/num_runs)

    res = pd.DataFrame(columns=data.columns, index=data.columns)

    for i,j in itertools.combinations(np.arange(num_algos),2):
        res[res.columns[i]].ix[j] = abs(ranksums[i] - ranksums[j]/num_runs) > right_side
        res[res.columns[j]].ix[i] = abs(ranksums[i] - ranksums[j]/num_runs) > right_side
    return res

conover_inman_procedure(hypervols)


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
We now know in what cases the difference is sufficient as to say that one result is better than the other.
Another alternative is the Friedman test.

Its null hypothesis that repeated measurements of the same individuals have the same distribution.
It is often used to test for consistency among measurements obtained in different ways.
For example, if two measurement techniques are used on the same set of individuals, the Friedman test can be used to determine if the two measurement techniques are consistent.
Can you write Python code for it?

hyp_transp = hypervols.transpose()
measurements = [list(hyp_transp[col]) for col in hyp_transp.columns]
stats.friedmanchisquare(*measurements)


I am working on the file "Evolutionary Multi-Objective Optimization.ipynb".
The first task is:
results is a dictionary, but a pandas DataFrame is a more handy container for the results.
Can you write Python code for it?


I am working on the file "Ex__Synthetic_wedge.ipynb".
The first task is:
I was using np.take here, but 'fancy indexing' is shorter and more intuitive. We are just going to index rocks using the integers in model. That is, if model has a 1, we take the second element, [2400, 2450], from rocks. We'll end up with an array containing the rocks corresponding to each element of earth.
Can you write Python code for it?


I am working on the file "Ex__Synthetic_wedge.ipynb".
The first task is:
Measure amplitude at the peak
Can you write Python code for it?

apparent = np.amin(synth, axis=0)
t_apparent = np.argmin(synth, axis=0)

actual = synth[depth//3, :]
t_actual = np.ones_like(t_apparent) * depth//3

plt.plot(apparent, label="apparent")
plt.plot(actual, label="actual")
plt.text(6, 0, "actual", color='g')
plt.text(1, -0.18, "apparent", color='b')
# plt.legend()
plt.show()

# t_apparent = t_apparent.astype(np.float)
# t_apparent[0] = np.nan

plt.figure(figsize=(12,6))

# Plot seismic
plt.imshow(synth, cmap="Greys", aspect=0.2)
plt.colorbar()

# Plot horizons
plt.plot(t_apparent, 'r')
plt.plot(t_actual, 'c')

plt.xlim([0, 79])
plt.ylim([199, 0])
plt.title("Wedge model, {} Hz wavelet".format(f))
plt.show()


I am working on the file "Ex__Synthetic_wedge.ipynb".
The first task is:
Model seismic reflections
Now we have an earth model — giving us acoustic impedance everywhere in this 2D grid — we define a function to compute reflection coefficients for every trace.
Can you write Python code for it?

def make_rc(imp):
    upper = imp[ :-1, :]
    lower = imp[1:  , :]

    return (lower - upper) / (lower + upper)

rc = make_rc(imp)

plt.imshow(rc, aspect=1)
plt.show()


I am working on the file "Ex__Synthetic_wedge.ipynb".
The first task is:
Now set the upper part of the model — above the wedge — to zero (blue).
Can you write Python code for it?

model[:depth//3,:] = 0

plt.imshow(model)
plt.colorbar()
plt.show()


I am working on the file "Ex__Synthetic_wedge.ipynb".
The first task is:
Now we can make some Vp-rho pairs (rock 0 and rock 1).
Can you write Python code for it?


I am working on the file "Ex__Synthetic_wedge.ipynb".
The first task is:
We'll use scipy.signal.ricker to make a wavelet:
Can you write Python code for it?

from scipy.signal import ricker

f = 25
wavelet = ricker(100, a=5)

plt.plot(wavelet)
plt.show()

import bruges
w = bruges.filters.ricker(duration=0.1, dt=0.001, f=60)

plt.plot(w)


I am working on the file "Ex__Volumetrics.ipynb".
The first task is:
Another prospect
Can you write Python code for it?

p_a = p_disc
p_b = 0.10

p_both = p_a * p_b
p_both

p_one = 1 - (1 - p_a) * (1 - p_b)
p_one


I am working on the file "Ex__Volumetrics.ipynb".
The first task is:
Formation volume factor
Oil shrinks when we produce it, especially if it has high GOR. The FVF, or $B_O$, is the ratio of a reservoir barrel to a stock-tank barrel (25 deg C and 1 atm). Typically the FVF is between 1 (heavy oil) and 1.7 (high GOR).
For gas, $B_G$ is 0.3495ZT/P, where Z is the Z factor.
Can you write Python code for it?


I am working on the file "Ex__Volumetrics.ipynb".
The first task is:
If we're lucky we'll recover 50% of what's in place:
Can you write Python code for it?

recoverable = 0.5 * hcip
print("Recoverable is {:.0f} Mm³ or {:.0f} million bbl".format(recoverable/1000000, 6.29*recoverable/1000000))


I am working on the file "Ex__Volumetrics.ipynb".
The first task is:
Reward
We'd like to get the latest oil price. We can use Yahoo Finance.
What's the symbol? Crude futures symbols are funky, e.g.

CL — West Texas Intermediate or WTI, light sweet crude
BZ — Brent look-alike crude oil futures
MB — Gulf Coast Sour Crude
BB — Brent crude penultimate financial futures
NG — Henry Hub physical futures
HH — Henry Hub last day financial futures

The ticker symbols we're passing look like XXMYY.NYM, with components as follows:

XX — commodity symbol, as above: CL, BB, etc.
M — a month code, symbolizing January to December: [F,G,H,J,K,M,N,Q,U,V,X,Z]
YY — a two-digit year, like 13.
.NYM — the Nymex symbol.

First we build a query:
Can you write Python code for it?

import requests

symbol = 'CLF17.NYM'
url = "http://download.finance.yahoo.com/d/quotes.csv"
params = {'s': symbol, 'f': 'l1'}

r = requests.get(url, params=params)

try:
    price = float(r.text)
except ValueError as e:
    print("Error: ", e, r.text)

price

reward = 6.29 * expect * price

reward

print("We should make ${:.2f} billion.".format(reward/10e9))


I am working on the file "Ex__Volumetrics.ipynb".
The first task is:
Risk
Single prospect
Can you write Python code for it?

p_src = 0.5
p_mig = 0.25
p_trap = 0.75
p_res = 0.5

p_disc = p_src * p_mig * p_trap * p_res

p_disc

expect = p_disc * recoverable

print("Expectation is {:.0f} Mm³ or {:.0f} million bbl".format(expect/1000000, 6.29*expect/1000000))


I am working on the file "ExactEnumeration.ipynb".
The first task is:
Now lets calculate expected values:
Can you write Python code for it?

N = 4
J = -1
S = np.zeros(N)

E_T = []

betas = 2**np.arange(-10,1, 0.1)

for beta in betas:
    E_exp = 0
    Z = 0
    for i in range(2**N):
        for j in range(N):
            S[j] = (i >> j) & 1
        S = 2 * S - 1

        E = energy(S)
        p = np.exp(-beta * E)

        E_exp += E*p
        Z += p

    #print(E_exp/Z)
    E_T.append(E_exp/Z)

plt.plot(betas, E_T);


I am working on the file "Exam/Final_exam_solution.ipynb".
The first task is:
Problem 3
Solve the problem above using a while loop.
Can you write Python code for it?

i=2
while i<len(name_list):
    print(name_list[i])
    i = i + 3


I am working on the file "Examine unreported trials on ClinicalTrials.gov.ipynb".
The first task is:
Check for results on PubMed
If trials have reported their results on PubMed, and if it's possible to find them on PubMed using a linked NCT ID, then we count those trials as having submitted results.
So, for all trials that we regard as completed and due results, and that haven't already reported results on clinicaltrials.gov, we search PubMed, looking for the NCT ID either as a Secondary Source ID, or in the title/abstract. We look for anything published between the completion date and now, that doesn't have the words "study protocol" in the title, and that is classified as results of a trial (using the "therapy" clinical keyword, broad version).
At the time of writing, about 9,000 of the 34,000 trials have results on PubMed. An example of an NCT ID with results on PubMed: NCT02460380. (TODO: Update this).
Note 1: we know from the BMJ paper that there are trials that do have results on PubMed, but that aren't linked using the NCT ID. The BMJ authors found these using a manual search. Some examples: NCT00002762: 19487378, NCT00002879: 18470909, NCT00003134: 19066728, NCT00003596: 18430910. We regard these as invalid, because you can only find results via an exhaustive manual search. We only count results as published for our purposes if they are either (i) submitted on ClinicalTrials.gov or (ii) retrievable on PubMed using the NCT ID. See more on this below.
Note 2: we know there are some trials that have results PMIDs directly in ClinicalTrials.gov, in the results_reference field of the XML. After discussion with Jess here, and Annice at ClinicalTrials.gov, I decided that these results are too often meaningless to be useful - lots of the time they aren't truly results, but are studies from years ago.
Note 3: we also experimented with retrieving the results using the narrow version of the "therapy" clinical keyword, and using no clinical keyword at all. We evaluated these by using multiple PubMed matches as surrogate measures for false identification. At the time of writing on 2016/10/24, we examined 34677...
Can you write Python code for it?

# Store results locally.
conn = sqlite3.connect('./data/trials-abstract.db')
cur = conn.cursor()
c = "CREATE TABLE IF NOT EXISTS trials(nct_id TEXT PRIMARY KEY, "
c += "pubmed_results INT, pubmed_results_broad INT, pubmed_results_narrow INT)"
cur.execute(c)
conn.commit()

REGENERATE_PUBMED_LINKS = False
count = 0
df['pubmed_results'] = False
for i, row in df_overdue.iterrows():
    if count % 10000 == 0:
        print count, row.nct_id
    count += 1
    # First, check for results stored in the local db.
    c = "SELECT nct_id, pubmed_results, pubmed_results_broad, "
    c += "pubmed_results_narrow FROM trials WHERE nct_id='%s'" % row.nct_id
    cur.execute(c)
    data = cur.fetchone()
    has_results = False
    if data and (not REGENERATE_PUBMED_LINKS):
        has_results = bool(int(data[2]))
    else:
        # No local results, or we want to regenerate them: check PubMed.
        broad_results = \
            utils.get_pubmed_linked_articles(row.nct_id,
                                             row.completion_date, 'broad')
        # Used in the past (see note 3 above).
        simple_results = \
            utils.get_pubmed_linked_articles(row.nct_id,
                                             row.completion_date, '')
        narrow_results = \
            utils.get_pubmed_linked_articles(row.nct_id,
                                             row.completion_date, 'narrow')
        c = "INSERT OR REPLACE INTO trials VALUES('%s', %s, %s, %s)" % \
            (row.nct_id, len(simple_results), len(broad_results), len(narrow_results))
        cur.execute(c)
        conn.commit()
        has_results = broad_results > 0
    df.set_value(i, 'pubmed_results', has_results)

cur.close()
conn.close()
print 'done'


I am working on the file "Examine unreported trials on ClinicalTrials.gov.ipynb".
The first task is:
Compare with our data
Now examine:

of the NCT IDs for which BMJ authors find PubMed results, how many we also find PubMed results for
of the same dataset, how many only BMJ find results for
of the NCT IDs for which BMJ authors do not find PubMed results, how many we do find PubMed results
Can you write Python code for it?

df_bmj = pd.Series(nct_ids).to_frame(name='pmid')
df_bmj['pubmed_results'] = ~df_bmj.pmid.isnull()
df_bmj.index.name = 'nct_id'
df_bmj.reset_index(inplace=True)
print len(df_bmj), 'NCT IDs in the full BMJ dataset'
# df_bmj.head(20)

merged_results = \
    pd.merge(df_bmj, df_completed, #[['nct_id', 'pubmed_results']],
             on='nct_id', how='inner', suffixes=('_bmj', '_ours'))

# NB I tried this first with a left join: but 1521 out of the 4500 papers
# don't appear in our dataset, because the BMJ authors' inclusion criteria are
# different from ours. To get a sample after a left join...
# merged_results[merged_results.we_have_results.isnull()].head()

merged_results['we_have_results'] = ~merged_results.is_overdue
merged_results.we_have_results.value_counts(dropna=False)
# merged_results.head()
print len(merged_results), 'NCT IDs are in both the BMJ dataset and ours'

papers_both_find_pm_results = \
    merged_results[merged_results.pubmed_results_bmj & merged_results.we_have_results]
papers_both_find_pm_results.head()
print len(papers_both_find_pm_results), 'we both find results for'
papers_only_they_find_results = \
    merged_results[merged_results.pubmed_results_bmj & ~merged_results.we_have_results]
print len(papers_only_they_find_results), 'only they find results for'
papers_only_we_find_results = \
    merged_results[~merged_results.pubmed_results_bmj & merged_results.we_have_results]
print len(papers_only_we_find_results), 'only we find results for'
noone_finds_results = \
    merged_results[~merged_results.pubmed_results_bmj & ~merged_results.we_have_results]
print len(noone_finds_results), 'neither of us find results for'

# Examine a sample of the papers only they find results for.
cols = ['nct_id', 'title', 'pubmed_results_bmj', 'pmid', 'we_have_results']
papers_only_they_find_results.sample(10)[cols]

# Papers only we find results for. If the `results_date` field exists, it
# means that the results are published on ClinicalTrials.gov. Otherwise
# we found results on PubMed but...


I am working on the file "Examine unreported trials on ClinicalTrials.gov.ipynb".
The first task is:
Create summary results file
The raw XML trial summaries from ClinicalTrials.gov are supplied as a single very large zip file, containing more than 200,000  XML files. This section assumes that that these have already been downloaded and unzipped in the search_result directory.
Extract the fields of interest from the XML summaries, and save them to a CSV file, which we'll use as our source data for the rest of this exercise. ClinicalTrials.gov supplies field definitions.
Toggle REGENERATE_SUMMARY to False for the purposes of development.
Can you write Python code for it?

fname = './data/trials.csv'
REGENERATE_SUMMARY = False # False
if REGENERATE_SUMMARY:
    files = glob.glob('./search_result/*.xml')
    print len(files), 'files found'
    fieldnames = ['nct_id', 'title', 'overall_status',
                  'study_type', 'completion_date',
                  'lead_sponsor', 'lead_sponsor_class',
                  'collaborator', 'collaborator_class',
                  'phase', 'locations', 'has_drug_intervention', 'drugs',
                  'disposition_date', 'results_date',
                  'enrollment']
    trials = csv.DictWriter(open(fname, 'wb'), fieldnames=fieldnames)
    trials.writeheader()
    for i, f in enumerate(files):
        if i % 50000 == 0:
            print i, f
        text = open(f, 'r').read()
        data = utils.extract_ctgov_xml(text)
        trials.writerow(data)
print 'done'


I am working on the file "Examine unreported trials on ClinicalTrials.gov.ipynb".
The first task is:
For reference: Compare our results with BMJ authors
TODO: Make this a separate notebook?
A 2016 BMJ paper found that around 65% of papers reprted results. "Overall, 2892 of the 4347 clinical trials (66.5%) had been published or reported results as of July 2014."
Excellently, the BMJ authors publish their raw data on DataDryad so we can compare our results with theirs, to get an idea of the difference between our automated strategy and their partially manual strategy. (However, in their reported data it looks to me like the matched PMID rate is 59.9% of all NCT IDs.)
The BMJ authors were looking at a much smaller set of papers than us, because they focussed on academic medical centres. Their set is slightly different, because they include pre-Phase-2 trials, and 'Terminated' as well as 'Completed' trials. They also used a manual search strategy which involved searching Scopus and manually comparing results.
Can you write Python code for it?

from openpyxl import load_workbook
import sys
bmj_results = load_workbook(filename = './data/chen-bmj.xlsx')

nct_ids = {}
count = 0
has_pmid = 0

# The Excel data has multiple worksheets, sigh.
# And NCT IDs can occur more than once with different results, sigh.
# We only care about where there's at least one result.
# Fiddle about and reshape the data so that we know whether
# each NCT ID has a result.
for sheet in bmj_results.worksheets:
    for i, row in enumerate(sheet.rows):
        if i == 0:
            continue
        if row[0].value:
            count += 1
            if isinstance(row[6].value, long):
                val = str(row[6].value)
            else:
                val = row[6].value
            if val:
                has_pmid += 1
            # Always set val if it exists.
            # Otherwise, only set val if there's no current value
            # for this NCT ID.
            if val:
                nct_ids[row[0].value] = val
            else:
                if not row[0].value in nct_ids:
                    nct_ids[row[0].value] = val

print count, 'rows found in total'
print has_pmid, 'of those rows have a PMID'
print has_pmid / float(count) * 100, 'per cent of their NCT IDs have a PMID, including duplicates'
print
unique_nct_ids = len(nct_ids.keys())
print unique_nct_ids, '*unique* NCT IDs found in all rows'
pmids_found = sum(1 for x in nct_ids.values() if x)
print pmids_found, 'of these have PMIDs'
print pmids_found / float(unique_nct_ids) * 100, 'per cent of unique NCT IDs have a PMID'


I am working on the file "Examine unreported trials on ClinicalTrials.gov.ipynb".
The first task is:
Write to CSV
Output final results to a CSV file, which we will use in the interactive. We reshape the data so it has a row for each sponsor, and two columns for each year: one column for the number of overdue results, and one for the total trials.
Also, write all the raw data to a single CSV file.
Can you write Python code for it?

df_completed['year_completed'] = df_completed['completion_date'].dt.year.dropna().astype(int)
df_completed['year_completed'] = df_completed.year_completed.astype(int)

# Drop all sponsors with fewer than N completed trials.
df_final = df_completed[
    df_completed.groupby('lead_sponsor').nct_id.transform(len) >= NUM_TRIALS]

# Now reshape the data: a row for each sponsor, columns by year:
# lead_sponsor,2008_overdue,2008_total,2009_overdue,2009_total...
df_temp = df_final.set_index(['lead_sponsor', 'lead_sponsor_class', 'year_completed'])
gb = df_temp.groupby(level=[0, 1, 2]).is_overdue
df2 = gb.agg({'overdue': 'sum', 'total': 'count'}) \
          .unstack().swaplevel(0, 1, 1).sort_index(1)
df2.columns = df2.columns.to_series().apply(lambda x: '{}_{}'.format(*x))
df3 = df2.reset_index()
df3['lead_sponsor_slug'] = df3.lead_sponsor.apply(slugify)

df3.to_csv('./data/completed.csv', index=None)
print len(df3), 'sponsors found with cutoff point at %s trials' % NUM_TRIALS

# Write the raw output to a full spreadsheet.
df.to_csv('./data/all.csv', index=None)


I am working on the file "Example shape parameterisation.ipynb".
The first task is:
Directly from Monaco® 5.10 server
The following code is an example of what can be used to automatically pull and parameterise shapes from the server based off of patient ID. For use in other centres it will need adjustment.
Can you write Python code for it?

# patientID = '00000'.zfill(6)
# string_search_pattern = r'\\MONACODA\FocalData\YOURDIRECTORYHERE\1~Clinical\*{}\plan\*\*tel.1'.format(patientID)
# string_search_pattern

# filepath_list = glob(string_search_pattern)
# filepath_list

telfilepath = "example_monaco510_telfile"
electronmodel_regex = "YourMachineName - \d+MeV"  # \d+ stands for any positive integer

with open(telfilepath, "r") as file:
    telfilecontents = np.array(file.read().splitlines())

electronmodel_index = []
for i, item in enumerate(telfilecontents):
    if re.search(electronmodel_regex, item):
        electronmodel_index += [i]

print("Located applicator and energy strings for plans within telfile:")
applicator_tel_string = [
    telfilecontents[i+12]  # applicator string is located 12 lines below electron model name
    for i in electronmodel_index]
print(applicator_tel_string)

energy_tel_string = [
    telfilecontents[i]
    for i in electronmodel_index]
print(energy_tel_string)

for i, index in enumerate(electronmodel_index):
    print("Applicator = {}".format(applicator_tel_string[i]))
    print("Energy = {}\n".format(energy_tel_string[i]))

    insert_inital_range = telfilecontents[
        index + 51::]  # coords start 51 lines after electron model name
    insert_stop = np.where(
        insert_inital_range=='0')[0][0]  # coords stop right before a line containing 0

    insert_coords_string = insert_inital_range[:insert_stop]
    insert_coords = np.fromstring(','.join(insert_coords_string), sep=',')
    x = insert_coords[0::2]/10
    y = insert_coords[1::2]/10

    width, length, poi = parameterise_single_insert(x, y)
    print("Width = {0:0.2f} cm".format(width))
    print("Length = {0:0.2f} cm".format(length))
    display_parameterisation(x, y, width, length, poi)


I am working on the file "Example shape parameterisation.ipynb".
The first task is:
From RT-DICOM
If you are using the online version of this notebook you will likely want to deidentify your dicom files. http://www.dicompyler.com/ can be used to do this however I am not in a position to guarantee it will do this adequately. You need to check this yourself.
To upload the dicom file go to notebook home and click the "upload button" located at the top right of the dashboard.
Can you write Python code for it?

# Change this name to match the dicom file located in the same directory
# as this notebook.
dicom_filename = "example_dicom_file.dcm"

dcm = dicom.read_file(dicom_filename, force=True)
applicator_string = dcm.BeamSequence[0].ApplicatorSequence[0].ApplicatorID
energy_string = dcm.BeamSequence[0].ControlPointSequence[0].NominalBeamEnergy
ssd_string = dcm.BeamSequence[0].ControlPointSequence[0].SourceToSurfaceDistance

print("Applicator = {} (identifier name)".format(applicator_string))
print("Energy = {} (nominal)".format(energy_string))
print("SSD = {} (dicom units)\n".format(ssd_string))

block_data = np.array(dcm.BeamSequence[0].BlockSequence[0].BlockData)
x = np.array(block_data[0::2]).astype(float)/10
y = np.array(block_data[1::2]).astype(float)/10

width, length, poi = parameterise_single_insert(x, y)
print("Width = {0:0.2f} cm".format(width))
print("Length = {0:0.2f} cm".format(length))

display_parameterisation(x, y, width, length, poi)


I am working on the file "Example-Analysis.ipynb".
The first task is:
Alternatively you can use interact to add a slider to vary the number of bins
Can you write Python code for it?

def plot_histogram(bins):
    hist2d(physicsOnlyDataFrame['Year'].values,physicsOnlyDataFrame['AgeAwarded'].values, bins=bins)
    colorbar() #Set a colour legend
    xlabel('Year Awarded')
    ylabel('Age Awarded')

interact(plot_histogram, bins=[1, 20, 1])  # Creates the slider


I am working on the file "Example-Analysis.ipynb".
The first task is:
Brilliant! You will find this technique useful to select kaons in the main analysis. Lets now plot the number of winners per year just for physics.
Can you write Python code for it?

H_PhysicsWinnersPerYear = physicsOnlyDataFrame.Year.hist(bins=15, range=[1920,2010])
xlabel('Year') #Plot an x label
ylabel('Number of Winners in Physics') #Plot a y label


I am working on the file "Example-Analysis.ipynb".
The first task is:
From the histogram we can see that there has been a recent trend of more multiple prize winners in the same year. However there is a drop in the range 1940 - 1950, this was due to prizes being awarded intermittently during World War II. To isolate this gap we can change the bin size (by changing the number of bins variable) to contain this range. Try changing the slider below (you will have to click in code box and press shift+enter to activate it) and see how the number of bins affects the look of the histogram.
Can you write Python code for it?

def plot_hist(bins):
    changingBins = data_without_economics.Year.hist(bins=bins, range=[1900,2010])
    xlabel('Year')
    ylabel('Number of People Given Prizes')
    BinSize = round(60/bins, 2)
    print(BinSize)

interact(plot_hist, bins=[2, 50, 1])


I am working on the file "Example-Analysis.ipynb".
The first task is:
Lets check the new DataFrames to see if this has worked!
Can you write Python code for it?


I am working on the file "Example-Analysis.ipynb".
The first task is:
Lets make a plot of the age of the winners at the time they were awarded the prize
Can you write Python code for it?

# plot a histogram of the laureates ages
H_AgeAwarded = physicsOnlyDataFrame.AgeAwarded.hist(bins=15)


I am working on the file "Example-Analysis.ipynb".
The first task is:
Lets now view the first few lines of the data table. The rows of the data table are each of the Nobel prizes awarded and the columns are the information about who won the prize.
We have put the data into a pandas DataFrame we can now use all the functions associated with DataFrames. A useful function is .head(), this prints out the first few lines of the data table.
Can you write Python code for it?

data.head(5) # Displaying some of the data so you can see what form it takes in the DataFrame


I am working on the file "Example-Analysis.ipynb".
The first task is:
Playing with the slider will show you the effect of changing bthe in size in a 2D histogram. The darker bins in the top right corner show that there does appear to be a trend of Nobel prizes being won at an older age in more recent years.
Manipulating 2D histograms
This section is advanced and only required for the final section of the main analysis.
As the main analysis requires the calculation of an asymmetry, we now provide a contrived example of how to do this using the nobel prize dataset, we recommed only reading this section after reaching the "Searching for local matter anti-matter differences" section of the main analysis.
First calculate the number of entries in each bin of the 2D histogram and store these values in physics_counts as a 2D array.
xedges and yedges are 1D arrays containing the values of the bin edges along each axis.
Can you write Python code for it?

physics_counts, xedges, yedges, Image = hist2d(
    physicsOnlyDataFrame.Year, physicsOnlyDataFrame.AgeAwarded,
    bins=10, range=[(1900, 2010), (20, 100)]
)
colorbar() # Add a colour legend
xlabel('Year Awarded')
ylabel('Age Awarded')


I am working on the file "Example-Analysis.ipynb".
The first task is:
Plotting a histogram
Lets learn how to plot histograms. We will plot the number of prizes awarded per year. Nobel prizes can be awarded for up to three people per category. As each winner is recorded as an individual entry the histogram will tell us if there has been a trend of increasing or decreasing multiple prize winners in one year.
However before we plot the histogram we should find information out about the data so that we can check the range of the data we want to plot.
Can you write Python code for it?

# print the earliest year in the data
print(data.Year.min())

# print the latest year in the data
print(data.Year.max())


I am working on the file "Example-Analysis.ipynb".
The first task is:
There is actually a function that would calculate the rms for you, but we wanted to teach you how to manipulate data to make calculations!
Can you write Python code for it?

# calculate standard deviation (rms) of distribution
print(physicsOnlyDataFrame['AgeAwarded'].std())


I am working on the file "Example-Analysis.ipynb".
The first task is:
We can now plot the histogram over a sensible range using the hist function from matplotlib. You will use this throughout the main analysis.
Can you write Python code for it?

# plot the histogram of number of winners against year
H_WinnersPerYear = data_without_economics.Year.hist(bins=11, range=[1900, 2010])
xlabel('Year')
ylabel('Number of Winners')


I am working on the file "Example-Analysis.ipynb".
The first task is:
We have now successfully plotted the histogram of just the physics prizes after applying our pre-selection.
Calculations, Scatter Plots and 2D Histogram
Adding New Data to a Data Frame
You will find this section useful for when it comes to creating a Dalitz plot in the particle physics analysis.
We want to see what ages people have been awarded Nobel prizes and measure the spread in the ages.
Then we'll consider if over time people have been getting awarded Nobel prizes earlier or later in their life.
First we'll need to calculate the age or the winners at the time the prize was awarded based on the Year and Birthdate columns. We create an AgeAwarded variable and add this to the data.
Can you write Python code for it?

# Create new variable in the dataframe
physicsOnlyDataFrame['AgeAwarded'] = physicsOnlyDataFrame.Year - physicsOnlyDataFrame.BirthYear
physicsOnlyDataFrame.head(5)


I am working on the file "Example-QuickStart.ipynb".
The first task is:
Fill-Grid Method
Can you write Python code for it?


I am working on the file "Example-QuickStart.ipynb".
The first task is:
Quick Start-Subdivision Method
Wellblock Mesh

C++
        **Meshing information**
         Pts        SubGrid     Neighbor    Internal Edge    OutFlow Edge
         0          0(3,4,0)     (1,3)        0(3,4)            0(3,0)
         1          1(0,4,1)     (2,0)        1(4.0)            1(0,1)
         2          2(1,4.2)     (3,1)        2(1,4)            2(1,2)
         3          3(2,4,3)     (0,2)        3(2,4)            3(2,3)
         4

Example Steps
```python
1. Initialize a Wellblock with subdivision method
subdivision.WellGrid(Rect0=(0,0),Rect1=(50,50),Qw=1000,Qe=(200,400,300,100),h=26.25,phi=0.2)

"""
        Rect0  -- The coordinates of left bottom corner point of the rectangle
        Rect1  -- The coordinates of right top corner point of the rectangle
        Qw     -- Wellbore flow rate, stb/day
        Qe     -- Outflow edge flow rate, stb/day (Edge0,Edge1,Edge2,Edge3)
        h      -- Wellblock thickness
        phi    -- Wellblock porosity
"""
2. Solve the internal mass balnaced equation
subdivision.Subdivision()

3. Tracing streamline and output streamline and TOF information
SL,TOF,SL_end,TOF_end=subdivision.SLTrace(NSL=80)

"""
        NSL        -- Number of Streamline
        TOF        -- TOF array of of the [NSL] streamlines
        SL         -- nodes array of of the [NSL] streamlines
        TOF_end    -- a boundary TOF list of the [NSL] streamlines
        SL_end     -- a boundary node list of the [NSL] streamlines
"""
```
Case 3 in SPE-182614-MS
Subdivision Method
Can you write Python code for it?

import Lib.subdivision as sub

Method_subdivision=sub.WellGrid(Rect0=(0,0),Rect1=(50,50),Qw=1000,Qe=(200,400,300,100),h=26.25,phi=0.2)
Method_subdivision.Subdivision()
SL,TOF,SL_end,TOF_end=Method_subdivision.SLTrace(NSL=80)

Method_subdivision=sub.WellGrid(Rect0=(0,0),Rect1=(50,50),Qw=1000,Qe=(0,0,500,500),h=26.25,phi=0.2)
Method_subdivision.Subdivision(debug=0)
SL,TOF,SL_end,TOF_end=Method_subdivision.SLTrace(NSL=80)


I am working on the file "Example.ipynb".
The first task is:
Adding a lightcurve:
Can you write Python code for it?


I am working on the file "Example.ipynb".
The first task is:
Adding and initialising the Monotransit
So we have set up our star, importantly with the density included. Now we must add a monotransit. This is set up such that multiple monotransits can be added (although I have yet to check this feature).
The format is $t_{\rm cen}$, $b$, and $R_p/R_s$
Can you write Python code for it?

#Adds this to mono1
st.AddMonotransit(2121.02,0.5,0.0032,name='mono1')

#Accessing pure transit fit:

st.BuildMeanModel()
model=st.meanmodel_comb.get_value(np.arange(2120.6,2121.4,0.005))
plt.plot(st.Lcurve.lc[abs(st.Lcurve.lc[:,0]-st.mono1.tcen)<1.5,0],st.Lcurve.lc[abs(st.Lcurve.lc[:,0]-st.mono1.tcen)<1.5,1],'.')
plt.plot(np.arange(2120.6,2121.4,0.005),model,'--')


I am working on the file "Example.ipynb".
The first task is:
Assess Matches
We must now determine if our data is "balanced". Can we detect any statistical differences between the covariates of our matched test and control groups? Matcher is configured to treat categorical and continouous variables separately in this assessment.
Discrete
For categorical variables, we look at plots comparing the proportional differences between test and control before and after matching.
For example, the first plot shows:
* prop_test - prop_control for all possible term values---prop_test and prop_control being the proportion of test and control records with a given term value, respectively. We want these (orange) bars to be small after matching.
* Results (pvalue) of a Chi-Square Test for Independence before and after matching. After matching we want this pvalue to be > 0.05, resulting in our failure to reject the null hypothesis that the frequecy of the enumerated term values are independent of our test and control groups.
Can you write Python code for it?

categorical_results = m.compare_categorical(return_table=True)

categorical_results


I am working on the file "Example.ipynb".
The first task is:
Bob and Alice need to add their public key to the end of their twitter description (profile) so that others can send them messages.
Check out https://twitter.com/HeteroT1 as an example.
Can you write Python code for it?

print('Bob needs this posted to be able to recieve a message:\n{}\n'.format(Bob_key_pair['TwitterKey']))

print('Alice needs to do the same with her public key:\n{}\n'.format(Alice_key_pair['TwitterKey']))

# Bob and Alice might want to save there private keys to a file or store them somewhere safe.
# The keys as a set of 3 long integers and a bit value
print("Bob's Private key:\n{}\nBob's Public key:\n{}\n"
      .format(Bob_key_pair['PrivateKey'],
              Bob_key_pair['PublicKey']))
print("Alice's Private key:\n{}\nAlice's Public Key:\n{}\n"
      .format(Alice_key_pair['PrivateKey'],
              Alice_key_pair['PublicKey']))


I am working on the file "Example.ipynb".
The first task is:
Built-in quick support for Python 2 and 3
As a developper enable warnings, (or use python -Wd)
You support both Python 2 and Python 3 ?
Can you write Python code for it?

from insupportable import support
if support('PY2'):
    print("You are on python 2")
else:
    print("You are on python 3")


I am working on the file "Example.ipynb".
The first task is:
Calculate the attention of subjects
Here we calculate the attention of subjects
Can you write Python code for it?

attend = sns.load_dataset("attention").query("subject <= 12")
g = sns.FacetGrid(attend, col="subject", col_wrap=4, size=2, ylim=(0, 10))
g.map(sns.pointplot, "solutions", "score", color=".3", ci=None);


I am working on the file "Example.ipynb".
The first task is:
Create test and control groups and reassign loan_status to be a binary treatment indicator. This is our reponse in the logistic regression model(s) used to generate propensity scores.
Can you write Python code for it?

test = data[data.loan_status == "Default"]
control = data[data.loan_status == "Fully Paid"]
test['loan_status'] = 1
control['loan_status'] = 0


I am working on the file "Example.ipynb".
The first task is:
Execute tests
To execute test, just decorate the cells containing the tests with the %%ipytest magic:
Can you write Python code for it?

%%ipytest

# define the tests

def test_my_func():
    assert my_func(0) == 0
    assert my_func(1) == 0
    assert my_func(2) == 2
    assert my_func(3) == 2

def my_func(x):
    return x // 2 * 2


I am working on the file "Example.ipynb".
The first task is:
First we need our twitter connection object.
Can you write Python code for it?

from twython import Twython
bobs_twitter = Twython(consumer_key, consumer_sec, access_tok, access_token_sec)


I am working on the file "Example.ipynb".
The first task is:
If in the following line you see:
PicklingError: Can't pickle <class 'namaste.namaste.MonotransitModel'>: it's not the same object as namaste.namaste.MonotransitModel
then try re-running this script!
Can you write Python code for it?

#Just checking if the MCMC works/progresses:

st.sampler.run_mcmc(st.init_mcmc_params, 1, rstate0=np.random.get_state())


I am working on the file "Example.ipynb".
The first task is:
It gently warn you that maybe you should tell it wether or not you stil support PY1, which let you write code and decide wether ot not to support later.
Minor python versions
Can you write Python code for it?

import insupportable

insupportable.predicates

support('PY35+')

import insupportable
from insupportable import support

def jp():
    import sys
    return ('jupyter_notebook' in sys.modules)

support.s.add_feature('JUPYTER', jp, supported=True)

support('JUPYTER')

class Item(object):

    def __init__(self, name, predicate=True, support=True):
        """An item in a discret support set.

        An item is consituted of
        - a name (string)
        - a predicate (`Bool | funtion:()->Bool`) to determine if we are typically in this case.

        - the support status (bool)

        """
        self.name = name
        if callabale(predicate):
            predicate = predicate()
        self.predicate = predicate
        self.support = support

support.s.predicates


I am working on the file "Example.ipynb".
The first task is:
It looks like a threshold of 0.0005 retains enough information in our data. Let's proceed with matching using this threshold.

Match Data
Below we match one record from the majority group to each record in the minority group. This is done with replacement, meaning a single majority record can be matched to multiple minority records. Matcher assigns a unique record_id to each record in the test and control groups so this can be addressed after matching. If susequent modelling is planned, one might consider weighting models using a weight vector of 1/f for each record, f being a record's frequency in the matched dataset. Thankfully Matcher can handle all of this for you :).
Can you write Python code for it?

m.match(method="min", nmatches=1, threshold=0.0005)

m.record_frequency()


I am working on the file "Example.ipynb".
The first task is:
It looks like the bulk of our matched-majority-group records occur only once, 68 occur twice, ... etc. We can preemptively generate a weight vector using Matcher.assign_weight_vector()
Can you write Python code for it?


I am working on the file "Example.ipynb".
The first task is:
Load the dataset (loan.csv), which is the original loan.csv sampled as follows:
```python
path = 'misc/loan_full.csv'
fields = [
    "loan_amnt",
    "funded_amnt",
    "funded_amnt_inv",
    "term",
    "int_rate",
    "installment",
    "grade",
    "sub_grade",
    "loan_status"
]
data = pd.read_csv(path)[fields]
Treat long late as Defaulted
data.loc[data['loan_status'] == 'Late (31-120 days)', 'loan_status'] = 'Default'
Sample 20K records from Fully paid and 2K from default
df = data[data.loan_status == 'Fully Paid'].sample(20000, random_state=42) \
    .append(data[data.loan_status == 'Default'].sample(2000, random_state=42))
df.to_csv('misc/loan.csv', index=False)
```
Can you write Python code for it?

path = "misc/loan.csv"
data = pd.read_csv(path)


I am working on the file "Example.ipynb".
The first task is:
Looking at the plots and test results, we did a pretty good job balancing our categorical features! The p-values from the Chi-Square tests are all > 0.05 and we can verify by observing the small proportional differences in the plots.
Continuous
For continous variables we look at Empirical Cumulative Distribution Functions (ECDF) for our test and control groups  before and after matching.
For example, the first plot pair shows:
* ECDF for test vs ECDF for control before matching (left), ECDF for test vs ECDF for control after matching(right). We want the two lines to be very close to each other (or indistiguishable) after matching.
* Some tests + metrics are included in the chart titles.
    * Tests performed:
        * Kolmogorov-Smirnov Goodness of fit Test (KS-test)
            This test statistic is calculated on 1000
            permuted samples of the data, generating
            an imperical p-value.  See pymatch.functions.ks_boot()
            This is an adaptation of the ks.boot() method in
            the R "Matching" package
            https://www.rdocumentation.org/packages/Matching/versions/4.9-2/topics/ks.boot
        * Chi-Square Distance:
            Similarly this distance metric is calculated on
            1000 permuted samples.
            See pymatch.functions.grouped_permutation_test()
* Other included Stats:
    * Standarized mean and median differences.
         How many standard deviations away are the mean/median
        between our groups before and after matching
        i.e. `abs(mean(control) - mean(test))` / `std(control.union(test))`
Can you write Python code for it?

cc = m.compare_continuous(return_table=True)

cc


I am working on the file "Example.ipynb".
The first task is:
Now let's save the MCMC samples and plot the MCMC result:
Can you write Python code for it?


I am working on the file "Example.ipynb".
The first task is:
Running a model MCMC
Can you write Python code for it?

#Now we have our mean-functions (eg transit models) added, we need to build the priors and the model:
st.BuildMeanPriors()
st.BuildMeanModel()

# Then we need the combine the kernel we've optimised with the monotransit mean model into a Celerite GP:
import celerite

st.gp=celerite.GP(kernel=st.kern,mean=st.meanmodel_comb,fit_mean=True)

# Now we need to build all the priors (not just the mean functions)

st.BuildAllPriors(st.gp.get_parameter_names())

# Making walkers initial positions
chx=np.random.choice(st.settings.npdf,st.settings.nwalkers,replace=False)

#Tidying up so that the Celerite priors match our prior names
st.fitdict.update({'mean:'+nm:getattr(st.mono1,nm+'s')[chx] for nm in ['tcen','b','vel','RpRs']})
st.fitdict.update({'mean:'+nm:getattr(st,nm+'s')[chx] for nm in ['LD1','LD2']})

#Removing medians:
for row in st.fitdict:
    st.fitdict[row][np.isnan(st.fitdict[row])]=np.nanmedian(np.isnan(st.fitdict[row]))
dists=[st.fitdict[cname] for cname in st.gp.get_parameter_names()]
st.init_mcmc_params=np.column_stack(dists)

#Masking an arbitrary region around the transit for analysis (2.75d in this case)
mask=abs(st.Lcurve.lc[:,0]-st.gp.get_parameter('mean:tcen'))<2.75

# Doing an initial fit with PlotModel:

_=namaste.PlotModel(st.Lcurve.lc[mask,:], st.gp, np.median(st.init_mcmc_params,axis=0), fname=st.settings.outfilesloc+st.objname+'_initfit.png',GP=True)

#RUNNING THE MCMC WITH EMCEE:
import emcee
st.sampler = emcee.EnsembleSampler(st.settings.nwalkers, len(st.gp.get_parameter_vector()), namaste.MonoLogProb, args=(st.Lcurve.lc,st.priors,st.gp), threads=st.settings.nthreads)


I am working on the file "Example.ipynb".
The first task is:
Say you use to support 3 platforms,
Can you write Python code for it?

support.config(({'WindowsPhone':True,
              'Android':True,
              'iOs':False
             },))

if support('WindowsPhone'):
    print('Click on start menu')
else:
    print('Probably Android')


I am working on the file "Example.ipynb".
The first task is:
Sending a message
You may send as a direct message or as a status update. While the direct message route has the advantage of the recepient knowing the message is for them, and the data contained in the message may only be seen by the recepient, it also allows others to observe the metadata that there is direct interaction between the sender and recepient. With a public status update, there is no metadata with regards to whom the message is for. The recepient has to be expecting the message or just randomly try to decrypt messages.
Can you write Python code for it?

from pktwitter.messaging import send_direct_messages, send_status_update
from twython import Twython

bobs_twitter = Twython(consumer_key, consumer_sec, access_tok, access_token_sec)

send_direct_messages(bobs_twitter, 'HeteroT2', cyphertext)

send_status_update(bobs_twitter, cyphertext)


I am working on the file "Example.ipynb".
The first task is:
The average accuracy of our 100 models is 66.06%, suggesting that there's separability within our data and justifiying the need for the matching procedure. It's worth noting that we don't pay much attention to these logistic models since we are using them as a feature extraction tool (generation of propensity scores). The accuracy is a good way to detect separability at a glance, but we shouldn't spend time tuning and tinkering with these models. If our accuracy was close to 50%, that would suggest we cannot detect much separability in our groups given the features we observe and that matching is probably not necessary (or more features should be included if possible).
Predict Scores
Can you write Python code for it?


I am working on the file "Example.ipynb".
The first task is:
The plot above demonstrates the separability present in our data. Test profiles have a much higher propensity, or estimated probability of defaulting given the features we isolated in the data.

Tune Threshold
The Matcher.match() method matches profiles that have propensity scores within some threshold.
i.e. for two scores s1 and s2, |s1 - s2| <= threshold
By default matches are found from the majority group for the minority group. For example, if our test group contains 1,000 records and our control group contains 20,000, Matcher will
    iterate through the test (minority) group and find suitable matches from the control (majority) group. If a record in the minority group has no suitable matches, it is dropped from the final matched dataset. We need to ensure our threshold is small enough such that we get close matches and retain most (or all) of our data in the minority group.
Below we tune the threshold using method="random". This matches a random profile that is within the threshold
as there could be many. This is much faster than the alternative method "min", which finds the closest match for every minority record.
Can you write Python code for it?


I am working on the file "Example.ipynb".
The first task is:
There is a significant imbalance in our data--the majority group (fully-paid loans) having many more records than the minority group (defaulted loans). We account for this by setting balance=True when calling Matcher.fit_scores() below. This tells Matcher to sample from the majority group when fitting the logistic regression model(s) so that the groups are of equal size. When undersampling this way, it is highly recommended that nmodels is explictly assigned to a integer much larger than 1. This ensure is that more of the majority group is contributing to the generation of propensity scores. The value of this integer should depend on the severity of the imbalance; here we use nmodels=100.
Can you write Python code for it?

# for reproducibility
np.random.seed(20170925)

m.fit_scores(balance=True, nmodels=100)


I am working on the file "Example.ipynb".
The first task is:
Twitter API key
You need a twitter api key. I load a set below you need your own. Here are instructios showing how to.
https://themepacific.com/how-to-generate-api-key-consumer-token-access-key-for-twitter-oauth/994/
Can you write Python code for it?

import configparser
config = configparser.ConfigParser()
config.read('pktwitter/tests/user_data.ini')
consumer_key = config['HeteroT1']['consumer_key']
consumer_sec = config['HeteroT1']['consumer_sec']
access_tok = config['HeteroT1']['access_tok']
access_token_sec = config['HeteroT1']['access_token_sec']


I am working on the file "Example.ipynb".
The first task is:
Without GPs:
Can you write Python code for it?

st_nogp=namaste.Star('203311200',namaste.Settings(nsteps=1000,nthreads=4,mission='K2'))

st_nogp.settings.update(GP=False)
st_nogp.settings.update(verbose=False)

#Or you can use a csv file, for example:
st_nogp.csvfile_dat('ExampleData/EPIC203311200_beststelpars.csv')

#Initialising LDs:
st_nogp.initLD()

#Adding a lightcurve:
st_nogp.addLightcurve('/Volumes/CHEOPS/K2_MONOS/BestLCs/EPIC203311200_bestlc_ev.npy')

#When we're not doing GPs, lets flatten the LC:
st_nogp.Lcurve.lc=st_nogp.Lcurve.flatten()

plt.plot(st_nogp.Lcurve.lc[:,0],st_nogp.Lcurve.lc[:,1],'.')
plt.plot(st_nogp.Lcurve.lc[st_nogp.Lcurve.lcmask,0],st_nogp.Lcurve.lc[st_nogp.Lcurve.lcmask,1],',')

#Adds this to mono1
st_nogp.AddMonotransit(2121.02,0.5,0.0032,name='mono1')

#Now we have our mean-functions (eg transit models) added, we need to build the priors and the model:
st_nogp.BuildMeanPriors()

#Accessing pure transit fit:

st_nogp.BuildMeanModel()
model=st_nogp.meanmodel_comb.get_value(np.arange(2120.6,2121.4,0.005))
plt.plot(st_nogp.Lcurve.lc[abs(st_nogp.Lcurve.lc[:,0]-st_nogp.mono1.tcen)<1.5,0],st_nogp.Lcurve.lc[abs(st_nogp.Lcurve.lc[:,0]-st_nogp.mono1.tcen)<1.5,1],'.')
plt.plot(np.arange(2120.6,2121.4,0.005),model,'--')

st_nogp.RunMCMC()

st_nogp.SaveMCMC(suffix='NoGP',overwrite=True)

st_nogp.PlotMCMC(suffix='NoGP')


I am working on the file "Example.ipynb".
The first task is:
technical
Can you write Python code for it?

import insupportable

def fun(a, *args):
    print(args)

fun(1,2,3)

DF

from insupportable import DiscreatFeature as DF, FeatureGroup#, FeatureTracker

PY2 = DF('PY2', lambda x: sys.version_info.major == 2)
PY3 = DF('PY3', lambda x: sys.version_info.major == 3)
JPY = DF('JUPYTER', lambda x: True)
IPY = DF('IPYTHON', lambda x: False)

group_py = FeatureGroup(PY2,PY3)
group_jpy = FeatureGroup(JPY,IPY)

vx = lambda: re.compile('^(\S+) ?(\d+)\.(\d+)(\+|-)?$')

class NumeralFeature(object):

    def __init__(self ,name, predicate):
        self.name = name
        self.features = []
        if callable(predicate):
            self.current_version = predicate()
        else :
            self.current_version = predicate

import re

import sys
pyv = NumeralFeature('python', lambda :sys.version_info)

print(vx().search('foo2.3').groups())
print(vx().search('foo2.3+').groups())
print(vx().search('tornado 2.3-').groups())
print(vx().search('python 2.7+').groups())

class FeatureTracker(object):

    def __init__(self, *args):
        self.args = args

        self._known_versions = [f.name for g in args for f in g.features if type(f) == DF]
        self._known_numerals = [g.name for g in args if type(g) == NumeralFeature]
        self._numeral_map = {}
        for g in args:
            if type(g) == NumeralFeature:
                self._numeral_map[g.name] = g.current_version
            else:
                continue

    def support(self, indetifier):
        pass

    def know(self, identifier):
        if identifier in self._known_versions:
            return True
        elif vx().match(identifier):
            feat = vx().search(identifier).groups()
            print(tuple(map(int, feat[1:3])) >= self._numeral_map[feat[0]][:2])
            return feat[0] in self._known_numerals

ft = FeatureTracker(group_jpy, group_py, pyv)

ft._numeral_map

ft.know('python 3.5+')

FeatureTracker


I am working on the file "Example/Demo_drawing_pad.ipynb".
The first task is:
Reproducing the content of the drawing pad in a figure using matplotlib
Can you write Python code for it?

plt.figure(figsize=(10,5))
plt.plot(widget.drawing_pad.data[0], widget.drawing_pad.data[1])
plt.xlim(0,500)
plt.ylim(0,250)
plt.show()


I am working on the file "Example/Demo_drawing_pad.ipynb".
The first task is:
Synchronisation works both ways !
Can you write Python code for it?


I am working on the file "Example/Demo_drawing_pad.ipynb".
The first task is:
The widget is a box containing a drawing pad (main widget) and buttons (related to functionalities described below). Here is the drawing pad. You can draw whatever you want and data will be synchronised with the Python.
Can you write Python code for it?

widget = jd.CustomBox()
widget.drawing_pad

print(widget.drawing_pad.data)


I am working on the file "Example/Demo_drawing_pad.ipynb".
The first task is:
You can display the entiere widget. There are four additionnal elements:
+ a text-field : to enter a name related to a signature (drawn in the drawing pad)
+ a Clear button
+ a Save button : to save a signature (related to the name written in the text field)
+ a Login button : you can try to reproduce a signature previously registered. If this signature is close enough to one of the saved ones, a message  Welcome (name)  should appear
Can you write Python code for it?


I am working on the file "Example/Example notebook.ipynb".
The first task is:
Continuous CO2 emissions - 1kg/year
Emissions at each time-step should be equal to the annual emission rate.
Can you write Python code for it?

co2_cont = np.ones_like(time)
co2_cont_rf = CO2(co2_cont, time)

plt.plot(co2_cont_rf)
plt.xlabel('Time', size=14)
plt.ylabel('$W \ m^{-2}$', size=14)


I am working on the file "Example/Example notebook.ipynb".
The first task is:
Deterministic results
Start with a pulse emission of CO2
Can you write Python code for it?

co2_emission = np.zeros_like(time)
co2_emission[0] = 1

co2_forcing = CO2(co2_emission, time, kind='RF')


I am working on the file "Example/Example notebook.ipynb".
The first task is:
Example uses of ghgforcing
Define the time axis along which all emissions and calculations take place, from year 0 to 100. The time-step (tstep) is set to 0.01 years to improve the accuracy of calcuations.
Can you write Python code for it?

tstep = 0.01
time = np.linspace(0, 100, 100/tstep+1)


I am working on the file "Example/Example notebook.ipynb".
The first task is:
Having the outputs on an annual basis makes it really easy to plot.
Can you write Python code for it?

plt.plot(co2_forcing)
plt.xlabel('Time', size=14)
plt.ylabel('$W \ m^{-2}$', size=14)


I am working on the file "Example/Example notebook.ipynb".
The first task is:
I'm putting the co2_cont_full into a DataFrame just to make showing some of the results easier.
Can you write Python code for it?

df = pd.DataFrame(co2_cont_full)

df.loc[:,:200].plot(legend=False, c='b', alpha=0.2)
plt.xlabel('Time', size=14)
plt.ylabel('$W \ m^{-2}$', size=14)


I am working on the file "Example/Example notebook.ipynb".
The first task is:
Pulse emissions of fossil CH4 with and non-fossil CH4 without climate-carbon feedbacks.
Can you write Python code for it?

ch4_emission = co2_emission.copy()

ch4_forcing_with = CH4(ch4_emission, time, kind='RF', cc_fb=True)
ch4_forcing_without = CH4(ch4_emission, time, kind='RF',
                          cc_fb=False, decay=False)

plt.plot(ch4_forcing_with, label = 'Fossil CH$_4$ with cc-fb')
plt.plot(ch4_forcing_without, label = 'Non-fossil CH$_4$ without cc-fb')
plt.xlabel('Time', size=14)
plt.ylabel('$W \ m^{-2}$', size=14)
plt.legend(fontsize=12)


I am working on the file "Example/Example notebook.ipynb".
The first task is:
Stochastic (Monte Carlo) results
Just using the continuous CO2 emissions scenario.
Mean and +/- 1 sigma
When full_output is False, the functions return a Pandas DataFrame
Can you write Python code for it?

runs = 500

co2_cont_mc = CO2(co2_cont, time, kind='RF',
                 runs=runs, full_output=False)

co2_cont_mc.head()

plt.plot(co2_cont_mc['mean'])
plt.fill_between(x=np.arange(0,101),
                 y1=co2_cont_mc['+sigma'],
                 y2=co2_cont_mc['-sigma'],
                 alpha=0.4)
plt.xlabel('Time', size=14)
plt.ylabel('$W \ m^{-2}$', size=14)


I am working on the file "Example/Example notebook.ipynb".
The first task is:
The CO2 and CH4 functions return values at 1-year timesteps - 101 values from year 0 to 100. The calculations are done at a default time-step of 0.01 years, but that level of detail seems unnecessary and burdensome for an output.
Can you write Python code for it?


I am working on the file "Example/Example notebook.ipynb".
The first task is:
The co2_cont_mc results are the same as above
Can you write Python code for it?


I am working on the file "Example/LCCM code example.ipynb".
The first task is:
3.2- Membership Model :
The specification of the class memeberhsip model used in this example is as follows:
$V_{class1} =    0 $
$V_{class2} = ASCClass2 +  \beta_{CarOwnership, class2} * CarOwnerhsip $
$+ \beta_{LowIncome, class2} * LowIncome + \beta_{HighIncome, class2} * HighIncome $
$+ \beta_{Male, class2} * Male  + \beta_{DistanceTraveled, class2} * DistanceTraveled $
Can you write Python code for it?

# NOTE: Specification and variable names must be in list data structures.
# class_membership_spec defines the variables names to be used in the specification of the
# class membership model.
# class_membership_labels defines the names associated with each of the variables which
# will be displayed in the output table after estimation.

# NOTE: by specifying the word 'intercept' in the class_membership_spec, the code
# understands that this denotes an alternative specific constant.

# class membership model constrains the utility of the first class to zero by default.
# The same socio-demographic variables will be included in the class memeberhship
# model of each of the remaining classes (excluding the first class as it is the base).
class_membership_spec = ['intercept', 'car_ownership','low_income','high_income',
                         'male','distance']
class_membership_labels = ['Class-specific constant', 'Car ownership', 'Low Income','High Income',
                           'Male','Distance Traveled (miles)']


I am working on the file "Example/LCCM code example.ipynb".
The first task is:
3.4- Class-specific Choice Model:
You can specify your parameters as generic or alternative specific. The following example entails both types of specifications to help the modeler identify how to specify generic versus alternative specific parameters accordingly.
In this example, the intercepts are alternative specific and that is done by using only one bracket i.e:'intercept', [2,3,4,5,6]. The first alternative will be the base alternative and hence no intercept is allocated in its utility.
Note that we will be constraining the choice set also for latent class 1 whereby for this class we have the following specification: 'intercept', [2,4,5,6]. The bike alternative does not belong in the choice set for this class and hence no parameters will be estimated including the ASC.
Travel time parameters across all alternatives for both latent classes are generic. This is done by using two brackets i.e: 'travel_time', [[1,2,3,4,5,6]] according to the specification below. Note that for latent class 1, we drop travel time from alternative 3 (bike) as that alternative does not exist in the choice set.
Travel cost parameter is constrained to be the same for the auto and drive to transit alternatives for latent class 1. Also, the travel cost parameter is constrained to be the same for the remaining alternatives in latent class 1. Such a specification is done according to the following script: 'travel_cost', [[1,5],[4,6]] based on the specification below.
Travel cost parameter is generic for all alternatives for latent class 2.
The specification of the class specific choice model used in this example is as follows:
Latent Class 1:
$V_{auto} =        \beta_{tt, class1} * TravelTime_{auto} + \beta_{cost_Auto-DrivetoTransit, class1} * TravelCost_{auto} $
$V_{walk} = ASCWalk_{class1} + \beta_{tt, class1} * TravelTime_{walk}$
$V_{WalkToTransit} = ASCWalkToTransit_{class1} + \beta_{tt, class1} * TravelTime_{walktotransit} + \beta_{cost_WalktoTransit-PrivateTransit, class1} * TravelCost_{walktotransit} $
$V_{DriveToTransit} =...
Can you write Python code for it?

# NOTE: Specification and variable names must be in lists of ordered dictionaries.
# class_specific_specs defines the variables names to be used in the specification of the
# class specific choice model of each class.
# class_specific_labels defines the names associated with each of the variables which
# will be displayed in the output tables after estimation.

# NOTE: by specifying the word 'intercept' in the class_specific_specs, the code
# understands that this denotes an alternative specific constant.

class_specific_specs = [OrderedDict([('intercept', [2,4,5,6]),
                                     ('travel_time', [[1,2,4,5,6]]),
                                     ('travel_cost', [[1,5],[4,6]])]),
                        OrderedDict([('intercept', [2,3,4,5,6]),
                                     ('travel_time', [[1,2,3,4,5,6]]),
                                     ('travel_cost', [[1,4,5,6]])])]

class_specific_labels = [OrderedDict([('ASC', ['ASC(Walk)',
                                               'ASC(Walk to Transit)','ASC(Drive to Transit)',
                                               'ASC(Private Transit)']),
                                      ('Travel Time',['Travel Time ']),
                                      ('Travel Cost',['Travel Cost Auto and Drive to Transit', 'Travel Cost WalktoTransit and PrivateTransit'])]),
                         OrderedDict([('ASC', ['ASC(Walk)','ASC(Bike)',
                                               'ASC(Walk to Transit)','ASC(Drive to Transit)',
                                               'ASC(Private Transit)']),
                                      ('Travel Time',['Travel Time']),
                                      ('Travel Cost',['Travel Cost'])])]


I am working on the file "Example/LCCM code example.ipynb".
The first task is:
Modify variables, create dummy variables,and scale variables so the estimated coefficients are of similar magnitudes.
Can you write Python code for it?

# Create a dummy variable indicating whether a certain individual in the sample is male or female
df['male'] = (df['Gender']==1).astype(int)

# Create a categorical variable to indicate the income level of the household corresponding to each individual
# in the sample. We will define three categories: low income (income category  < 6) , medium income
# (6 <= income category < 12) and high income (12 <= income category < 16).
df['low_income'] = (df['HHIncome']<6).astype(int)
df['medium_income'] = ((df['HHIncome']>=6)&(df['HHIncome']<12)).astype(int)
df['high_income'] = ((df['HHIncome']>=12)&(df['HHIncome']<16)).astype(int)


I am working on the file "Example/LCCM code example.ipynb".
The first task is:
Step 2: Load the Dataset.
The lccm package handles data in long format (NOT wide format).
To see the difference between the two formats run the following lines of code, (make sure your excel files are in the same directory as your code file):
df_long = pd.read_csv('Qualtrics data long format.csv')
df_long.head()
df_wide = pd.read_excel('Qualtrics data wide format.xlsx')
df_wide.head()
If you do not want to convert the wide formatted data yourself, you can check the following link, which provides instructions regarding how one can convert his/her dataset from wide format to long format. https://github.com/timothyb0912/pylogit/blob/master/examples/notebooks/Main%20PyLogit%20Example.ipynb
Can you write Python code for it?

# Load the data file
inputFilePath = 'C:/Users/Feras/Desktop/LCCM github exmaple/'
inputFileName = 'Qualtrics data long format.csv'

df = pd.read_csv(open(inputFilePath + inputFileName, 'rb'))

# Let's have a quick look at the long format data (first 5 rows)
df.head()


I am working on the file "Example/test_GB.ipynb".
The first task is:
Igrid[atomI] appears to be off by a factor of 12.6!
Can you write Python code for it?

# This is probably due to a unit conversion in a multiplicative prefactor

# This multiplicative prefactor is based on nanometers
r_min = 0.14
r_max = 1.0
print (1/r_min - 1/r_max)

# This multiplicative prefactor is based on angstroms
r_min = 1.4
r_max = 10.0
print (1/r_min - 1/r_max)


I am working on the file "Example/test_GB.ipynb".
The first task is:
Switching from nanometers to angstroms makes the multiplicative prefactor smaller, which is opposite of the desired effect!
Can you write Python code for it?


I am working on the file "Example/test_fractional_GB.ipynb".
The first task is:
Igrid[atomI] appears to be off by a factor of 12.6!
Can you write Python code for it?


I am working on the file "Example/test_fractional_GB.ipynb".
The first task is:
Switching from nanometers to angstroms makes the multiplicative prefactor smaller, which is opposite of the desired effect!
Can you write Python code for it?


I am working on the file "Example1.ipynb".
The first task is:
And finally, we build the network itself.  With the above description, the layer architecture will be:
(Affine) -> (ReLU) -> (Affine) -> (Softmax)
We create our network object, with 500 hidden layer neurons and 10 output layer neurons (which correspond to our 10 classes).
Can you write Python code for it?

nn = lml.NeuralNetwork(opts)
nn.add_layer('Affine', {'neurons':500})
nn.add_layer('ReLU', {})
nn.add_layer('Affine', {'neurons':10})
nn.add_layer('SoftmaxLoss', {})


I am working on the file "Example1.ipynb".
The first task is:
As you can see, the network overfits the data very easily, achieving a validation accuracy of 100%.  For the sake of completeness, we will print the final validation accuracy of the model.
Can you write Python code for it?

accuracy = trainer.accuracy(test_data, test_classes)
print('Final model accuracy:', accuracy)


I am working on the file "Example1.ipynb".
The first task is:
Now that our model is created, we must train it.  We use the given Trainer object to accomplish this.  First, we must supply training options.  These are, once again, provided in a dictionary.
We will use basic stochastic gradient descent with a learning rate of 1, no regularization, and we will train for 10 epochs.
Can you write Python code for it?

opts = {
    'update_options' : {'update_rule' : 'sgd', 'learning_rate' : 1},
    'reg_param' : 0,
    'num_epochs' : 10
}


I am working on the file "Example1.ipynb".
The first task is:
Now we create the trainer object and give it the model, the data, and the options.
Can you write Python code for it?


I am working on the file "Example1.ipynb".
The first task is:
Now, in order to feed the data to Learny McLearnface, we wrap it in a data dictionary with specified labels.
(Note that he validation set and training set will be the same in this case, as we are intentionally trying to overfit a training set.)
Can you write Python code for it?

data = {
    'X_train' : test_data,
    'y_train' : test_classes,
    'X_val' : test_data,
    'y_val' : test_classes
}


I am working on the file "Example1.ipynb".
The first task is:
Now, we will create the data to be overfitted.  For the purposes of the example, we will create 100 700-dimensional data points, which will each be randomly assigned one of 10 classes.  We will attempt to use a model to overfit this data and achieve 100% accuracy on this training set.
We will organize the data points into a single numpy array, where the rows are individual datapoints, and we will also create a separate vector of integers which give the classes for corresponding examples.
We initialize the data and its classes:
Can you write Python code for it?

test_data = np.random.randn(100, 700)
test_classes = np.random.randint(1, 10, 100)


I am working on the file "Example1.ipynb".
The first task is:
Since we have supplied all the requirements necessary for the trainer, we simply use the train() function to train the model.  This will print status updates at the end of each epoch.
Can you write Python code for it?


I am working on the file "Example1.ipynb".
The first task is:
We will use the trainer's toolset to first print the accuracy of the model before training.  Since the model was randomly initialized and there are 10 classes, we should expect an initial accuracy close to 10%
Can you write Python code for it?

accuracy = trainer.accuracy(test_data, test_classes)
print('Initial model accuracy:', accuracy)


I am working on the file "Example2.ipynb".
The first task is:
And then we build the model.
Can you write Python code for it?

nn = lml.NeuralNetwork(opts)
nn.add_layer('Affine', {'neurons':600})
nn.add_layer('ReLU', {})
nn.add_layer('Affine', {'neurons':10})
nn.add_layer('SoftmaxLoss', {})


I am working on the file "Example2.ipynb".
The first task is:
It is common to have separate validation and test sets, so we will split MNIST's test set in half to accomodate this. The first 5000 images will be our validation set, and the rest will be our test set.
Can you write Python code for it?

X_train = training_images
y_train = training_labels
X_val = test_images[:5000, :]
y_val = test_labels[:5000]
X_test = test_images[5000:, :]
y_test = test_labels[5000:]


I am working on the file "Example2.ipynb".
The first task is:
Let's do a formal classification of our test data. We use the Evaluator class for use of the network after the training has completed.
Can you write Python code for it?

evaluator = lml.Evaluator(nn, X_test)
predicted_classes, confidences = evaluator.predict()


I am working on the file "Example2.ipynb".
The first task is:
Let's visualize some of the elements of this dataset, organized by class.
Can you write Python code for it?

classes = range(10)
for i in classes:
    idxs = np.flatnonzero(training_labels == i)
    idxs = np.random.choice(idxs, 10, replace=False)
    for j, idx in enumerate(idxs):
        plt_idx = j * 10 + i + 1
        plt.subplot(10, 10, plt_idx)
        reshaped = training_images[idx].reshape((28, 28))
        plt.imshow(reshaped, cmap=plt.cm.binary)
        plt.axis('off')
        if j == 0:
            plt.title(i)


I am working on the file "Example2.ipynb".
The first task is:
Like before, we will use a Trainer in order to fit the model to the MNIST dataset. We will use stochastic gradient descent, and we will use a learning rate of 1e-2 and a regularization constant of 1e-8. We will also train the model for 10 epochs.
Can you write Python code for it?

opts = {
    'update_options' : {'update_rule' : 'sgd', 'learning_rate' : 1e-2},
    'reg_param' : 1e-8,
    'num_epochs' : 10
}


I am working on the file "Example2.ipynb".
The first task is:
Now that we have class predictions from our test set, we can visualize how the network is classifying images. We will display a few of these test images, organized by what class the network predicts it belongs to. As such, any errors in the display represent the model's classification errors.
Can you write Python code for it?

classes = range(10)
for i in classes:
    idxs = np.flatnonzero(predicted_classes == i)
    idxs = np.random.choice(idxs, 10, replace=False)
    for j, idx in enumerate(idxs):
        plt_idx = j * 10 + i + 1
        plt.subplot(10, 10, plt_idx)
        reshaped = X_test[idx].reshape((28, 28))
        plt.imshow(reshaped, cmap=plt.cm.binary)
        plt.axis('off')
        if j == 0:
            plt.title(i)