I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
10. Create a dictionary for each column mapping the State to its respective value for that column.
Can you write Python code for it?

verbal_values = {x:sat_data['Verbal'] for x in state_names}
math_values = {x:sat_data['Math'] for x in state_names}
rate_values = {x:sat_data['Rate'] for x in state_names}


I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
11. Create a dictionary with the values for each of the numeric columns
Can you write Python code for it?

#SAT_values = {x:sat_data['Verbal'] for x in sat_data['Verbal']}


I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
16. Plot the Rate distribution
Can you write Python code for it?

SAT_scores.Rate.plot (kind='hist', bins=15)
plt.xlabel('SAT Rates')
plt.ylabel('Frequency')
plt.show()


I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
16. Plot the Verbal distribution
Can you write Python code for it?

SAT_scores.Verbal.plot (kind='hist', bins=15)
plt.xlabel('SAT Verbal Scores')
plt.ylabel('Frequency')
plt.show()


I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
17. What is the typical assumption for data distribution? and 18. Does that distribution hold true for our data?
The typical assumption of data distribution is that it should follow a normal distribution, with standard deviations being relatively equal on both sides of the mean. Neither of the histograms appear to follow a normal distribution, with the Verbal scores in particular following a right/positive skew. But I need to properly check for normal distribution, and find a way the relative distribution onto the histograms. Perhaps Seaborn has a function that can help me with that.
Seaborn Plotting for Histograms and Fitting a Distribution
Can you write Python code for it?

# Used seaborn website as guidance: http://seaborn.pydata.org/tutorial/distributions.html
# I used a feature called the 'Kernel Density Estimation" (KDE) to
# visualize a distribution to the data.
# KDE is an estimator that uses each data point to make an estimate of the distibution and attempts to
# smooth it out on the histogram.
# This resulting curve has an area below it equal to one, hence the decimal units for frequency.
sns.distplot(SAT_scores.Verbal, bins=15)
plt.xlabel('SAT Verbal Scores')
plt.ylabel('Frequency (KDE)')
plt.show()

sns.distplot(SAT_scores.Math, bins=15)
plt.xlabel('SAT Math Scores')
plt.ylabel('Frequency (KDE)')
plt.show()

sns.distplot(SAT_scores.Rate, bins=15)
plt.xlabel('SAT Rates')
plt.ylabel('Frequency (KDE)')
plt.show()

sns.kdeplot(SAT_scores.Verbal)
plt.xlabel('SAT Verbal Scores')
plt.ylabel('Frequency (KDE)')
plt.show()

sns.kdeplot(SAT_scores.Math)
plt.xlabel('SAT Math Scores')
plt.ylabel('Frequency (KDE)')
plt.show()

sns.kdeplot(SAT_scores.Rate)
plt.xlabel('SAT Rates')
plt.ylabel('Frequency (KDE)')
plt.show()


I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
20. Are there any interesting relationships to note?
Both Verbal and Math scores are highly correlated with each other, whichever way you plot them, with Math appearing to affect Verbal at a faster rate than the other way around.
Can you write Python code for it?

# Not really. I had already assigned the Verbal, Math, and Rate columns to integers,
# so no conversion is needed there.


I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
3. Create a data dictionary for the dataset.
Can you write Python code for it?

sat_data = {}

for index, column_name in enumerate(header):
   sat_data[column_name] = []
   for row in data:
       sat_data[column_name].append(row[index])


I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
8. Print the types of each column
Can you write Python code for it?

print 'The type of the State column is' + ' ' + str(type (sat_data['State'][2]))

print 'The type of the State column is' + ' ' + str(type (sat_data['Math'][2]))

print 'The type of the State column is' + ' ' + str(type (sat_data['Verbal'][2]))

print 'The type of the State column is' + ' ' + str(type (sat_data['Rate'][2]))


I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
9. Do any types need to be reassigned? If so, go ahead and do it.
Can you write Python code for it?

SAT_scores['Verbal'] = SAT_scores['Verbal'].apply(pd.to_numeric)

SAT_scores['Math'] = SAT_scores['Math'].apply(pd.to_numeric)

SAT_scores['Rate'] = SAT_scores['Rate'].apply(pd.to_numeric)

SAT_scores.dtypes


I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
Mean, Median and Mode in NumPy and SciPy
Can you write Python code for it?

# Find the mean, median, and mode for the set of verbal scores and the set of math scores.
import numpy as np
print np.median(SAT_scores.Verbal)
print np.median(SAT_scores.Math)
#Numpy doesn't have a built in function for mode. However, stats does;
#its function returns the mode, and how many times the mode appears.
verb_mode = stats.mode(SAT_scores.Verbal)
math_mode = stats.mode(SAT_scores.Math)
print verb_mode
print math_mode


I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
Scatter Plotting
Can you write Python code for it?


I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
The median Verbal SAT score is 526, its mean is approximately 532, and its mode is above its mean at 562 (appears, 3 times).
The median Math SAT score is 521, its mean is 531.5, and its mode is below its mean at 499 (appears 6 times).
Step 4: Visualize the data¶
Can you write Python code for it?

#Will be using Pandas dataframe for plotting.


I am working on the file "2001 SAT Scores Analysis/Analysis of 2001 Iowa SAT Scores.ipynb".
The first task is:
The miniumum rate is 4, found in North Dakota, South Dakota, and Mississippi, and the maximum rate is 82 found in Connecticut.
The minimum verbal score is 482 in D.C., and the maximum is 593 in Iowa.
The median verbal score is 526 in Oregon.
The minimum math score is 439 in Ohio, and the maximum is 603, which is interestingly also in Iowa.
The median math score is 521.
Iowa has the highest SAT Scores in the country overall.
13. Write a function using only list comprehensions, no loops, to compute Standard Deviation. Print the Standard Deviation of each numeric column.
Can you write Python code for it?

#Standard Deviation function.
from math import sqrt
def standard_deviation(column):
    num_int = len(column)
    mean = sum(column)/len(column)
    differences = [x - mean for x in column]
    sq_diff = [t ** 2 for t in differences]
    num = sum(sq_diff)
    den = len(column)-1
    var = num/den
    print sqrt(var)

standard_deviation(SAT_scores['Math'])

standard_deviation(SAT_scores['Verbal'])

standard_deviation(SAT_scores['Rate'])

#Check to see the standard deviations are right.
print SAT_scores.describe()
#Approximately on point.


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
Barack Obama received 4,078,945 non-refund/non-zero donations for the 2012 president election, totaling \$558,359,124.90. The minimum donation was \$0.01, and the maximum amount was \$16,387,179.20. His average donation amount was \$136.89, and the median amount was \$50.00.
Can you write Python code for it?

# Obama's campaign finance stats:
print "Total positive donations: $%s" % totalOb
obama.describe()


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
Basic campaign finance statistics
Subset the positive campaign donations to President Barack Obama:
Can you write Python code for it?

obama = donations[(donations['cand_nm'] == 'Obama, Barack') &
                  (donations['contb_receipt_amt'] > 0)]
obama


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
Because T-Tests are resilient to breaking of the normality assumption and there are nonparametric equivalents that don't make normality assumptions, I run the Mann-Whitney U nonparametric T-test to determine on the campaign dataset. The reported p-value is about 0, so the result is still statistically significant.
Can you write Python code for it?

print "mann-whitney U", scipy.stats.mannwhitneyu(obama, romney)


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
Because the Welch's T-test makes no assumption about the size of the dataset or its variances, I have likely violated the assumption that the campaign finance data is normally distributed. I conduct the Shapiro-Wilk test to check if the data is actually normal for both presidential candidates. In this case, the test calculates a p-value and tells us that it is not normally distributed if the p-value <0.05.
Obama has a Shapiro-Wilks p-value of 0.000577, which indicates that I have violated the normality assumption of the Welch's T-test.
Can you write Python code for it?

print "Obama's Shapiro-Wilks p-value", scipy.stats.shapiro(obama)
print "Romney's Shapiro-Wilks p-value", scipy.stats.shapiro(romney)


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
Comparing Obama's and Romney's campaign donations by states
Across 50 states and Washington D.C., Obama out raised Romney in only 14 states that are Democratic stronghold: California, District of Columbia, Delaware, Hawaii, Illinois, Massachusetts, Maryland, Maine, New Mexico, New York, Oregon, Rhode Island, Vermont, Washington.
The 3 states where Obama received the highest amount of donations were Illinois (\$96.25 million), California (\$93.64 million) and New York (\$51.73 million). The 3 states where Romney received the highest amount of donations were California (\$79.06 million), Texas (\$70.05 million) and Florida (\$58.76 million).
Can you write Python code for it?

# Aggregate Obama's non-refund/non-zero donations by States
obaState = obama[obama['contbr_st'].isin(['AK', 'AL', 'AR', 'AZ', 'CA',
                                          'CO', 'CT', 'DC', 'DE', 'FL',
                                          'GA', 'HI', 'IA', 'ID', 'IL',
                                          'IN', 'KS', 'KY', 'LA', 'MA',
                                          'MD', 'ME', 'MI', 'MN', 'MO',
                                          'MS', 'MT', 'NC', 'ND', 'NE',
                                          'NH', 'NJ', 'NM', 'NV', 'NY',
                                          'OH', 'OK', 'OR', 'PA', 'RI',
                                          'SC', 'SD', 'TN', 'TX', 'UT',
                                          'VA', 'VT', 'WA', 'WI', 'WV', 'WY'])]

obst = obaState.groupby('contbr_st')

# obaState.groupby('contbr_st').agg(['mean', 'count', 'std'])
obst_sum = obst['contb_receipt_amt'].agg([np.sum, np.mean, len])
obst_sum.columns = ["Obama's total donations ($)", "Obama's average donation ($)",
                    "Obama's number of donations by state"]

# Aggregate Romney's non-refund/non-zero donations by States
romState = romney[romney['contbr_st'].isin(['AK', 'AL', 'AR', 'AZ', 'CA',
                                            'CO', 'CT', 'DC', 'DE', 'FL',
                                            'GA', 'HI', 'IA', 'ID', 'IL',
                                            'IN', 'KS', 'KY', 'LA', 'MA',
                                            'MD', 'ME', 'MI', 'MN', 'MO',
                                            'MS', 'MT', 'NC', 'ND', 'NE',
                                            'NH', 'NJ', 'NM', 'NV', 'NY',
                                            'OH', 'OK', 'OR', 'PA', 'RI',
                                            'SC', 'SD', 'TN', 'TX', 'UT',
                                            'VA', 'VT', 'WA', 'WI', 'WV', 'WY'])]

rmst = romState.groupby('contbr_st')

rmst_sum = rmst['contb_receipt_amt'].agg([np.sum, np.mean,...


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
Comparing Obama's and Romney's cumulative campaign donations between March 2011 and December 2012
Mitt Romney and Barack Obama had similar cumulative donation amounts until September 2012, after Obama and Biden were formally nominated for President and Vice President at the 2012 Democratic National Convention. The increased donations to Mitt Romney likely signaled efforts by his contributors to reach out to potential voters through ground work and television commercials.
Can you write Python code for it?

import csv,sys,datetime,collections
import itertools
import matplotlib.pyplot as plt

reader = csv.DictReader(open("donations.txt", 'r'))

totaldonations = collections.defaultdict(list)

for row in reader:
    name = row['cand_nm']
    datestr = row['contb_receipt_dt']
    amount = float(row['contb_receipt_amt'])
    date = datetime.datetime.strptime(datestr, '%d-%b-%y')

    if 'Obama' in name or 'Romney' in name:
        totaldonations[name].append((date, amount))

campaigntotals = dict([(name, sum(map(lambda p:p[1], val))) for name, val
                   in totaldonations.iteritems()])

fig = plt.figure(figsize=(12,6))

idx = 0
# Obama's and Romney's cumulative donations
for name, monies in totaldonations.iteritems():
    monies.sort(key=lambda pair: pair[0])
    i = itertools.groupby(monies, key=lambda p: p[0])
    monies = map(lambda (key, pairs): (key, sum([float(pair[1]) for pair in pairs])), i)
    total = 0
    newmonies = []
    for pair in monies:
        total += pair[1]
        newmonies.append((pair[0], total ))
    monies = newmonies

    xs,ys = zip(*monies)
    plt.plot(xs, ys, label = name + "'s donations")
    idx += 1
plt.legend(loc='upper center', ncol = 4)


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
Comparing Obama's and Romney's cumulative campaign reattributions between March 2011 and December 2012
In the campaign finance dataset, we see some of the data where the donation amount is negative. A quick modification of the code to output only the negative donation amounts:
import csv, sys, datetime
reader = csv.DictReader(open("donations.txt", 'r'))
for row in reader:
    name = row['cand_nm']
    datestr = row['contb_receipt_dt']
    amount = float(row['contb_receipt_amt'])
    if amount < 0:
        line = '\t'.join(row.values())
        print line
As it turns out, "redesignations" and "reattributions" of donations are normal. For instance, if a donation by person A is excessive, the part that exceeds the limits can be "reattributed" to person B, meaning that person B donated the rest to the campaign. Alternatively, the excess amount can be redesignated to another campaign in the same party. So a donation to Romney could be redesignated to a poor republican in Nebraska. However, "Reattribution to spouse" can be suspicious. A potential theory is that CEOs and wealthy contributors use it as a tactic to hide campaign contributions. For example, a CEO could donate money, then reattribute them to the donor's spouse. Then the spouse will donate that amount to the election candidate. In this way, a casual browser will find it difficult to notice that a the candidate is supoorted by a company's CEOs.
The campaign data indicates that Obama did not receive any reattributed donation but Romney received about \$4.1 million in reattribution especially between June 2012 and October 2012 when the presidential election race heated up.
Can you write Python code for it?

import csv,sys,datetime,collections
import itertools
import matplotlib.pyplot as plt

reader = csv.DictReader(open("donations.txt", 'r'))

totalreattributions = collections.defaultdict(list)

for row in reader:
    name = row['cand_nm']
    datestr = row['contb_receipt_dt']
    amount = float(row['contb_receipt_amt'])
    date = datetime.datetime.strptime(datestr, '%d-%b-%y')
    reason = row['receipt_desc']

    if amount < 0 and 'REATTRIBUTION' in reason:
        if 'Obama' in name or 'Romney' in name:
            totalreattributions[name].append((date, -amount))

candreattributions = dict([(name, sum(map(lambda p:p[1], val))) for name, val
                   in totalreattributions.iteritems()])

fig = plt.figure(figsize=(12,6))

idx = 0
# Obama's and McCain's cumulative reattributions
for name, monies in totalreattributions.iteritems():
    monies.sort(key=lambda pair: pair[0])
    i = itertools.groupby(monies, key=lambda p: p[0])
    monies = map(lambda (key, pairs): (key, sum([float(pair[1]) for pair in pairs])), i)
    total = 0
    newmonies = []
    for pair in monies:
        total += pair[1]
        newmonies.append((pair[0], total ))
    monies = newmonies

    xs,ys = zip(*monies)
    plt.plot(xs, ys, label = name + "'s reattributions")
    idx += 1
plt.legend(loc='upper center', ncol = 4)


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
Comparing Obama's and Romney's monthly campaign donations between March 2011 and December 2012
Mitt Romney received donations early beginning in May 2011 as he prepared to enter the Republican presidential primaries (February 1st 2012 - April 1st 2012). His donations increased significantly in April 2012 after he clinched the Republican nomination. Obama, on the other hand, began his re-election campaign in January 2012 and he was formally nominated as President at the Democratic National Convention in Charlotte, North Carolina, on September 6th 2012. Obama finalized his campaign on November 6th 2012 as he was re-elected with 51% popular vote and 332 electorate votes.
Can you write Python code for it?

%matplotlib inline
from collections import defaultdict
import matplotlib.pyplot as plt
import csv, sys, datetime

reader = csv.DictReader(open("donations.txt", 'r'))

obamadonations = defaultdict(lambda:0)
romneydonations = defaultdict(lambda:0)

for row in reader:
    name = row['cand_nm']
    datestr = row['contb_receipt_dt']
    amount = float(row['contb_receipt_amt'])
    date = datetime.datetime.strptime(datestr, '%d-%b-%y')

    if 'Obama' in name:
        obamadonations[date] += amount
    if 'Romney' in name:
        romneydonations[date] += amount

fig = plt.figure(figsize=(12,6))    # create a 12-inch x 6-inch figure
sorted_by_dateob = sorted(obamadonations.items(), key=lambda (key,val): key)
sorted_by_datemc = sorted(romneydonations.items(), key=lambda (key,val): key)
xs1,ys1 = zip(*sorted_by_dateob)
xs2,ys2 = zip(*sorted_by_datemc)
plt.plot(xs1, ys1, label="Obama's Donations")
plt.plot(xs2, ys2, label="Romney's Donations")
plt.legend(loc='upper center', ncol = 4)


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
Comparing distribution of Obama's and Romney's campaign donations
Because of the large number of outliers, I display the campaign donation distribution for both candidates within \$0 and \$1,500. Obama is on the right and Romney is on the left of the violin plot, and it indicates that Obama's donations were more concentrated in the smaller amounts between \$25 and \$150 while Romney's donations spanned a larger range.
Can you write Python code for it?

import seaborn as sns
sns.set_style("whitegrid")

obrom = donations[(donations['cand_nm'].isin(['Obama, Barack', 'Romney, Mitt'])) &
                 (donations['contb_receipt_amt'] < 1200)]

obrom2 = obrom[obrom['contb_receipt_amt'] > 0]

fig = plt.figure(figsize=(8,12))
ax = sns.violinplot(x="cand_nm", y="contb_receipt_amt", data=obrom2)


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
On the other hand, Mitt Romney received 1,576,478 non-refund/non-zero donations for the election, totaling \$679,994,941.51. The minimum amount was \$0.01, and the maximum amount was \$30,000. Despite having fewer donors than Obama, Romney received higher average donation at \$431.34 and higher median amount at \$100.50.
Can you write Python code for it?

# Romney's campaign finance stats:
print "Total positive donations: $%s" % totalRom
romney.describe()


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
Sorting the campaign contributions by decreasing amount indicates that the Obama Victory Fund contributed the highest donations to Barack Obama during the 2012 president election cycle.
Can you write Python code for it?


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
Subset the positive campaign donations to Mitt Romney:
Can you write Python code for it?

romney = donations[(donations['cand_nm'] == 'Romney, Mitt') &
                  (donations['contb_receipt_amt'] > 0)]
romney

# Calculate the total positive donations to Obama and Romney
totalOb = 0
totalRom = 0

for row in obama['contb_receipt_amt']:
    amountOb = float(row)
    totalOb += amountOb

for row in romney['contb_receipt_amt']:
    amountRom = float(row)
    totalRom += amountRom


I am working on the file "2012 Presidential Campaign Finance/2012 Presidential Campaign Finance.ipynb".
The first task is:
Which occupations were more generous to either candidate?
Among the campaign contributors that donated to either Barack Obama or Mitt Romney, only 13 contributors from 8 different occupations personally contributed at least \$5,000 to Obama's re-election campaign. On the other hand, 167 individuals from 36 different occupations (primarily chief executive officers, presidents and business owners) donated at least \$5,000 to Romney's re-election campaign
Can you write Python code for it?

obama = donations[(donations['cand_nm'] == 'Obama, Barack') &
                  (donations['contb_receipt_amt'] > 5000)]
obOcc = obama.groupby('contbr_occupation')

obOcc_sum = obOcc['contb_receipt_amt'].agg([np.sum, np.mean, len])
obOcc_sum.columns = ["Obama's total donations ($)", "Obama's average donation ($)",
                    "Number of donors"]
obOcc_sum

romney = donations[(donations['cand_nm'] == 'Romney, Mitt') &
                  (donations['contb_receipt_amt'] > 5000)]
rmOcc = romney.groupby('contbr_occupation')

rmOcc_sum = rmOcc['contb_receipt_amt'].agg([np.sum, np.mean, len])
rmOcc_sum.columns = ["Romney's total donations ($)", "Romney's average donation ($)",
                    "Number of donors"]
rmOcc_sum


I am working on the file "20130607-dojo-nested-generators-tuple-unpacking.ipynb".
The first task is:
Now we get to the benefit. We can combine generators much like one combines Linux shell commands with pipes on the command line.
Can you write Python code for it?

[i for i in gen_lte(gen_even(gen_fibonacci()), 1000)]

[i for i in gen_lte(gen_fibonacci(), 1000)]

evens = (i for i in gen_fibonacci() if i%2 == 0)
[i for i in gen_lte(evens, 1000)]

evens = gen_even(gen_fibonacci())
[i for i in gen_lte(evens, 1000)]

[i for i in gen_lte(gen_even(gen_fibonacci()), 1000)]


I am working on the file "2014-12-19-sprint-revenu-de-base/reforme_des_cotisations_avec_revenu_de_base.ipynb".
The first task is:
Adaptation pour faciliter l'usage de ce notebook
Ce correctif permet de redéfinir plusieurs fois la même variable sans provoquer d'erreur.
Can you write Python code for it?

import functools

from openfisca_core.formulas import make_reference_formula_decorator
from openfisca_france.entities import entity_class_by_symbol

reference_formula = make_reference_formula_decorator(entity_class_by_symbol = entity_class_by_symbol, update = True)
reference_input_variable = functools.partial(reference_input_variable, update = True)


I am working on the file "2014-12-19-sprint-revenu-de-base/reforme_des_cotisations_avec_revenu_de_base.ipynb".
The first task is:
Réforme
Can you write Python code for it?

from openfisca_core import reforms

Reform = reforms.make_reform(
    key = 'cotisations_revenu_de_base',
    name = u"Réforme des cotisations pour un Revenu de base",
    reference = tax_benefit_system,
    )

@Reform.formula
class cotisations_contributives(SimpleFormulaColumn):
    column = FloatCol
    entity_class = Individus
    label = u"Nouvelles cotisations contributives"

    def function(self, simulation, period):
        ags = simulation.calculate('ags', period)
        agff_tranche_a_employeur = simulation.calculate('agff_tranche_a_employeur', period)
        apec_employeur = simulation.calculate('apec_employeur', period)
        arrco_tranche_a_employeur = simulation.calculate('arrco_tranche_a_employeur', period)
        assedic_employeur = simulation.calculate('assedic_employeur', period)
        cotisation_exceptionnelle_temporaire_employeur = simulation.calculate('cotisation_exceptionnelle_temporaire_employeur', period)
        fonds_emploi_hospitalier = simulation.calculate('fonds_emploi_hospitalier', period)
        ircantec_employeur = simulation.calculate('ircantec_employeur', period)
        pension_civile_employeur = simulation.calculate('pension_civile_employeur', period)
        prevoyance_obligatoire_cadre = simulation.calculate('prevoyance_obligatoire_cadre', period)
        rafp_employeur = simulation.calculate('rafp_employeur', period)
        vieillesse_deplafonnee_employeur = simulation.calculate('vieillesse_deplafonnee_employeur', period)
        vieillesse_plafonnee_employeur = simulation.calculate('vieillesse_plafonnee_employeur', period)
        allocations_temporaires_invalidite = simulation.calculate('allocations_temporaires_invalidite', period)
        accident_du_travail = simulation.calculate('accident_du_travail', period)
        agff_tranche_a_employe = simulation.calculate('agff_tranche_a_employe', period)
        agirc_tranche_b_employe = simulation.calculate('agirc_tranche_b_employe', period)
        apec_employe = simulation.calculate('apec_employe', period)
     ...


I am working on the file "2014-12-19-sprint-revenu-de-base/reforme_des_cotisations_avec_revenu_de_base.ipynb".
The first task is:
Simulation
Can you write Python code for it?

# age = 60
# ages = [12, 37, 28]

# jean_eric = {
#     'name': u'Jean-Éric',
#     'age': 33,
# }
# aurelie = dict(
#     name = u'Aurélie',
#     age = 33,
# )

# personnes = [aurelie, jean_eric]

simulation = tax_benefit_system.new_scenario().init_single_entity(
    period = 2014,
    parent1 = dict(
        birth = date(1980, 1, 1),
        salaire_de_base = 12000,
        ),
    parent2 = dict(
        birth = date(1980, 1, 1),
        salaire_de_base = 6000,
        ),
    enfants = [
        dict(
            birth = date(2014, 1, 1),
            ),
        ],
    ).new_simulation(debug = True)

simulation.calculate("salsuperbrut")

simulation.calculate("revdisp")


I am working on the file "20141011-dojo-csv-excel-dialect-with-tab-20170215.ipynb".
The first task is:
So do not write in binary mode.
That is a difference between
the csv module
of legacy Python
and
the csv module
of modern Python.
Can you write Python code for it?

with open('eggs.csv', 'w') as csvfile:
    writer = csv.DictWriter(csvfile, columns, dialect='excel-tab')
    for row in mesa:
        writer.writerow(row)

!ls -l eggs.csv

!cat eggs.csv

print(repr(open('eggs.csv').read()))


I am working on the file "2014_07_18b_pymc_nonlinear_model_example.ipynb".
The first task is:
From StackOverflow

I am trying to fit some data with a Gaussian (and more complex) function(s). I have created a small example below.
My first question is, am I doing it right?

Yes!  You need to include a burn-in period, which you know.  I like to throw out the first half of my samples.  You don't need to do any thinning, but sometimes it will make your post-MCMC work faster to process and smaller to store.
The only other thing I advise is to set a random seed, so that your results are "reproducible": np.random.seed(12345) will do the trick.
Oh, and if I was really giving too much advice, I'd say import seaborn to make the matplotlib results a little more beautiful.

My second question is, how do I add an error in the x-direction, i.e. in the x-position of the observations/data?

One way is to include a latent variable for each error.  This works in your example, but will not be feasible if you have many more observations.  I'll give a little example to get you started down this road.

It is very hard to find nice guides on how to do this kind of regression in PyMC. Perhaps because its easier to use some least squares, or similar approach, I however have many parameters in the end and need to see how well we can constrain them and compare different models, pyMC seemed like the good choice for that.

Someone should write a book!
Can you write Python code for it?

# set random seed for reproducibility
np.random.seed(12345)

x = np.arange(5,400,10)*1e3

# Parameters for gaussian
amp_true = 0.2
size_true = 1.8
ps_true = 0.1

# Gaussian function
gauss = lambda x,amp,size,ps: amp*np.exp(-1*(np.pi**2/(3600.*180.)*size*x)**2/(4.*np.log(2.)))+ps
f_true = gauss(x=x,amp=amp_true, size=size_true, ps=ps_true )

# add noise to the data points
noise = np.random.normal(size=len(x)) * .02
f = f_true + noise
f_error = np.ones_like(f_true)*0.05*f.max()

# define the model/function to be fitted.
def model(x, f):
    amp = pm.Uniform('amp', 0.05, 0.4, value= 0.15)
    size = pm.Uniform('size', 0.5, 2.5, value= 1.0)
    ps = pm.Normal('ps', 0.13, 40, value=0.15)

    @pm.deterministic(plot=False)
    def gauss(x=x, amp=amp, size=size, ps=ps):
        e = -1*(np.pi**2*size*x/(3600.*180.))**2/(4.*np.log(2.))
        return amp*np.exp(e)+ps
    y = pm.Normal('y', mu=gauss, tau=1.0/f_error**2, value=f, observed=True)
    return locals()

MDL = pm.MCMC(model(x,f))
MDL.sample(20000, 10000, 1)

# extract and plot results
y_min = MDL.stats()['gauss']['quantiles'][2.5]
y_max = MDL.stats()['gauss']['quantiles'][97.5]
y_fit = MDL.stats()['gauss']['mean']
plt.plot(x,f_true,'b', marker='None', ls='-', lw=1, label='True')
plt.errorbar(x,f,yerr=f_error, color='r', marker='.', ls='None', label='Observed')
plt.plot(x,y_fit,'k', marker='+', ls='None', ms=5, mew=1, label='Fit')
plt.fill_between(x, y_min, y_max, color='0.5', alpha=0.5)
plt.legend()

pm.Matplot.plot(MDL)


I am working on the file "2014_07_18b_pymc_nonlinear_model_example.ipynb".
The first task is:
Here is a way to include error in x:
Can you write Python code for it?

# add noise to observed x values
x_obs = pm.rnormal(mu=x, tau=(1e4)**-2)

# define the model/function to be fitted.
def model(x_obs, f):
    amp = pm.Uniform('amp', 0.05, 0.4, value= 0.15)
    size = pm.Uniform('size', 0.5, 2.5, value= 1.0)
    ps = pm.Normal('ps', 0.13, 40, value=0.15)

    x_pred = pm.Normal('x', mu=x_obs, tau=(1e4)**-2) # this allows error in x_obs

    @pm.deterministic(plot=False)
    def gauss(x=x_pred, amp=amp, size=size, ps=ps):
        e = -1*(np.pi**2*size*x/(3600.*180.))**2/(4.*np.log(2.))
        return amp*np.exp(e)+ps
    y = pm.Normal('y', mu=gauss, tau=1.0/f_error**2, value=f, observed=True)
    return locals()

MDL = pm.MCMC(model(x_obs, f))
MDL.use_step_method(pm.AdaptiveMetropolis, MDL.x_pred) # use AdaptiveMetropolis to "learn" how to step
MDL.sample(200000, 100000, 10)  # run chain longer since there are more dimensions

# extract and plot results
y_min = MDL.stats()['gauss']['quantiles'][2.5]
y_max = MDL.stats()['gauss']['quantiles'][97.5]
y_fit = MDL.stats()['gauss']['mean']
x_fit = MDL.x_pred.trace().mean(0)
plt.plot(x,f_true,'b', marker='.', ls='-', lw=1, label='True')
plt.errorbar(x_obs, f, yerr=f_error, color='r', marker='.', ls='None', label='Observed')
plt.plot(x_fit,y_fit,'k', marker='+', ls='None', ms=5, mew=1, label='Fit')
plt.fill_between(x_fit, y_min, y_max, color='0.5', alpha=0.5)
plt.legend()

pm.Matplot.plot(MDL)


I am working on the file "2015-05-22 Basic calibration.ipynb".
The first task is:
Put this just in a tool module for easy calling:
Can you write Python code for it?

from IPython.html.widgets import IntProgress
from IPython.display import display
from time import sleep
def show_progress(results, worklist):
    prog = IntProgress(min=0, max=len(worklist))
    display(prog)
    while not results.ready():
        prog.value = results.progress
        sleep(5)

show_progress(results, SOI)


I am working on the file "2015-05-22 Basic calibration.ipynb".
The first task is:
Using a widget called IntProgress for simple progress bar.
Can you write Python code for it?

from IPython.html.widgets import IntProgress
from IPython.display import display

from time import sleep
prog = IntProgress(min=0, max=len(SOI)+1)
display(prog)
while not results.ready():
    prog.value = results.progress
    sleep(5)


I am working on the file "2015-05-22 Basic calibration.ipynb".
The first task is:
Using map_async so that I can do other stuff, while engines are working.
But results is my handle to what is going on in the background.
Can you write Python code for it?

results = lbview.map_async(process_folder, SOI)


I am working on the file "2015-07-28-dirichlet-distribution-dirichlet-process.ipynb".
The first task is:
Dirichlet Process
The Dirichlet Process can be considered a way to generalize the Dirichlet distribution. While the Dirichlet distribution is parameterized by a discrete distribution $G_0$ and generates samples that are similar discrete distributions, the Dirichlet process is parameterized by a generic distribution $H_0$ and generates samples which are distributions similar to $H_0$. The Dirichlet process also has a parameter $\alpha$ that determines how similar how widely samples will vary from $H_0$.
We can construct a sample $H$ (recall that $H$ is a probability distribution) from a Dirichlet process $\text{DP}(\alpha H_0)$ by drawing a countably infinite number of samples $\theta_k$ from $H_0$) and setting:
$$H=\sum_{k=1}^\infty \pi_k \cdot\delta(x-\theta_k)$$
where the $\pi_k$ are carefully chosen weights (more later) that sum to 1. ($\delta$ is the Dirac delta function.)
$H$, a sample from $DP(\alpha H_0)$, is a probability distribution that looks similar to $H_0$ (also a distribution). In particular, $H$ is a discrete distribution that takes the value $\theta_k$ with probability $\pi_k$. This sampled distribution $H$ is a discrete distribution even if $H_0$ has continuous support; the support of $H$ is a countably infinite subset of the support $H_0$.
The weights ($\pi_k$ values) of a Dirichlet process sample related the Dirichlet process back to the Dirichlet distribution.
Gregor Heinrich writes:

The defining property of the DP is that its samples have weights $\pi_k$ and locations $\theta_k$ distributed in such a way that when partitioning $S(H)$ into finitely many arbitrary disjoint subsets $S_1, \ldots, S_j$ $J<\infty$, the sums of the weights $\pi_k$ in each of these $J$ subsets are distributed according to a Dirichlet distribution that is parameterized by $\alpha$ and a discrete base distribution (like $G_0$) whose weights are equal to the integrals of the base distribution $H_0$ over the subsets $S_n$.

As an example, Heinrich imagines a DP with a standard normal base measure $H_0\sim...
Can you write Python code for it?

import matplotlib.pyplot as plt
from scipy.stats import beta, norm

def dirichlet_sample_approximation(base_measure, alpha, tol=0.01):
    betas = []
    pis = []
    betas.append(beta(1, alpha).rvs())
    pis.append(betas[0])
    while sum(pis) < (1.-tol):
        s = np.sum([np.log(1 - b) for b in betas])
        new_beta = beta(1, alpha).rvs()
        betas.append(new_beta)
        pis.append(new_beta * np.exp(s))
    pis = np.array(pis)
    thetas = np.array([base_measure() for _ in pis])
    return pis, thetas

def plot_normal_dp_approximation(alpha):
    plt.figure()
    plt.title("Dirichlet Process Sample with N(0,1) Base Measure")
    plt.suptitle("alpha: %s" % alpha)
    pis, thetas = dirichlet_sample_approximation(lambda: norm().rvs(), alpha)
    pis = pis * (norm.pdf(0) / pis.max())
    plt.vlines(thetas, 0, pis, )
    X = np.linspace(-4,4,100)
    plt.plot(X, norm.pdf(X))

plot_normal_dp_approximation(.1)
plot_normal_dp_approximation(1)
plot_normal_dp_approximation(10)
plot_normal_dp_approximation(1000)


I am working on the file "2015-07-28-dirichlet-distribution-dirichlet-process.ipynb".
The first task is:
Note: I wrote this post in an IPython notebook. It might be rendered better on NBViewer.
Dirichlet Distribution
The symmetric Dirichlet distribution (DD) can be considered a distribution of distributions. Each sample from the DD is a categorial distribution over $K$ categories. It is parameterized $G_0$, a distribution over $K$ categories and $\alpha$, a scale factor.
The expected value of the DD is $G_0$. The variance of the DD is a function of the scale factor. When $\alpha$ is large, samples from $DD(\alpha\cdot G_0)$ will be very close to $G_0$. When $\alpha$ is small, samples will vary more widely.
We demonstrate below by setting $G_0=[.2, .2, .6]$ and varying $\alpha$ from 0.1 to 1000. In each case, the mean of the samples is roughly $G_0$, but the standard deviation is decreases as $\alpha$ increases.
Can you write Python code for it?

import numpy as np
from scipy.stats import dirichlet
np.set_printoptions(precision=2)

def stats(scale_factor, G0=[.2, .2, .6], N=10000):
    samples = dirichlet(alpha = scale_factor * np.array(G0)).rvs(N)
    print "                          alpha:", scale_factor
    print "              element-wise mean:", samples.mean(axis=0)
    print "element-wise standard deviation:", samples.std(axis=0)
    print

for scale in [0.1, 1, 10, 100, 1000]:
    stats(scale)


I am working on the file "2015-07-28-dirichlet-distribution-dirichlet-process.ipynb".
The first task is:
Often we want to draw samples from a distribution sampled from a Dirichlet process instead of from the Dirichlet process itself. Much of the literature on the topic unhelpful refers to this as sampling from a Dirichlet process.
Fortunately, we don't have to draw an infinite number of samples from the base distribution and stick breaking process to do this. Instead, we can draw these samples as they are needed.
Suppose, for example, we know a finite number of the $\theta_k$ and $\pi_k$ values for a sample $H\sim \text{Dir}(\alpha H_0)$. For example, we know
$$\pi_1=0.5,\; \pi_3=0.3,\; \theta_1=0.1,\; \theta_2=-0.5.$$
To sample from $H$, we can generate a uniform random $u$ number between 0 and 1. If $u$ is less than 0.5, our sample is $0.1$. If $0.5<=u<0.8$, our sample is $-0.5$. If $u>=0.8$, our sample (from $H$ will be a new sample $\theta_3$ from $H_0$. At the same time, we should also sample and store $\pi_3$. When we draw our next sample, we will again draw $u\sim\text{Uniform}(0,1)$ but will compare against $\pi_1, \pi_2$, AND $\pi_3$.
The class below will take a base distribution $H_0$ and $\alpha$ as arguments to its constructor. The class instance can then be called to generate samples from $H\sim \text{DP}(\alpha H_0)$.
Can you write Python code for it?

from numpy.random import choice

class DirichletProcessSample():
    def __init__(self, base_measure, alpha):
        self.base_measure = base_measure
        self.alpha = alpha

        self.cache = []
        self.weights = []
        self.total_stick_used = 0.

    def __call__(self):
        remaining = 1.0 - self.total_stick_used
        i = DirichletProcessSample.roll_die(self.weights + [remaining])
        if i is not None and i < len(self.weights) :
            return self.cache[i]
        else:
            stick_piece = beta(1, self.alpha).rvs() * remaining
            self.total_stick_used += stick_piece
            self.weights.append(stick_piece)
            new_value = self.base_measure()
            self.cache.append(new_value)
            return new_value

    @staticmethod
    def roll_die(weights):
        if weights:
            return choice(range(len(weights)), p=weights)
        else:
            return None


I am working on the file "2015-07-28-dirichlet-distribution-dirichlet-process.ipynb".
The first task is:
This Dirichlet process class could be called stochastic memoization. This idea was first articulated in somewhat abstruse terms by Daniel Roy, et al.
Below are histograms of 10000 samples drawn from samples drawn from Dirichlet processes with standard normal base distribution and varying $\alpha$ values.
Can you write Python code for it?

import pandas as pd

base_measure = lambda: norm().rvs()
n_samples = 10000
samples = {}
for alpha in [1, 10, 100, 1000]:
    dirichlet_norm = DirichletProcessSample(base_measure=base_measure, alpha=alpha)
    samples["Alpha: %s" % alpha] = [dirichlet_norm() for _ in range(n_samples)]

_ = pd.DataFrame(samples).hist()


I am working on the file "2015-07-30-sampling-from-a-hierarchical-dirichlet-process.ipynb".
The first task is:
As we saw earlier the Dirichlet process describes the distribution of a random probability distribution. The Dirichlet process takes two parameters: a base distribution $H_0$ and a dispersion parameter $\alpha$. A sample from the Dirichlet process is itself a probability distribution that looks like $H_0$. On average, the larger $\alpha$ is, the closer a sample from $\text{DP}(\alpha H_0)$ will be to $H_0$.
Suppose we're feeling masochistic and want to input a distribution sampled from a Dirichlet process as base distribution to a new Dirichlet process. (It will turn out that there are good reasons for this!) Conceptually this makes sense. But can we construct such a thing in practice? Said another way, can we build a sampler that will draw samples from a probability distribution drawn from these nested Dirichlet processes? We might initially try construct a sample (a probability distribution) from the first Dirichlet process before feeding it into the second.
But recall that fully constructing a sample (a probability distribution!) from a Dirichlet process would require drawing a countably infinite number of samples from $H_0$ and from the beta distribution to generate the weights. This would take forever, even with Hadoop!
Dan Roy, et al helpfully described a technique of using stochastic memoization to construct a distribution sampled from a Dirichlet process in a just-in-time manner. This process provides us with the equivalent of the Scipy rvs method for the sampled distribution.  Stochastic memoization is equivalent to the Chinese restaurant process: sometimes you get seated an an occupied table (i.e. sometimes you're given a sample you've seen before) and sometimes you're put at a new table (given a unique sample).
Here is our memoization class again:
Can you write Python code for it?

from numpy.random import choice
from scipy.stats import beta

class DirichletProcessSample():
    def __init__(self, base_measure, alpha):
        self.base_measure = base_measure
        self.alpha = alpha

        self.cache = []
        self.weights = []
        self.total_stick_used = 0.

    def __call__(self):
        remaining = 1.0 - self.total_stick_used
        i = DirichletProcessSample.roll_die(self.weights + [remaining])
        if i is not None and i < len(self.weights) :
            return self.cache[i]
        else:
            stick_piece = beta(1, self.alpha).rvs() * remaining
            self.total_stick_used += stick_piece
            self.weights.append(stick_piece)
            new_value = self.base_measure()
            self.cache.append(new_value)
            return new_value

    @staticmethod
    def roll_die(weights):
        if weights:
            return choice(range(len(weights)), p=weights)
        else:
            return None


I am working on the file "2015-07-30-sampling-from-a-hierarchical-dirichlet-process.ipynb".
The first task is:
At this point, we have a function dp_draws that returns samples from a probability distribution (specifically, a probability distribution sampled from $\text{DP}(\alpha H_0)$). We can use dp_draws as a base distribution for another Dirichlet process!
Can you write Python code for it?

norm_hdp = DirichletProcessSample(norm_dp, alpha=10)


I am working on the file "2015-07-30-sampling-from-a-hierarchical-dirichlet-process.ipynb".
The first task is:
However, when we feed the base measure through the stochastic memoization procedure and then sample, we get many duplicate samples. The number of unique samples goes down as $\alpha$ increases.
Can you write Python code for it?

norm_dp = DirichletProcessSample(base_measure, alpha=100)

print "Number of unique samples after {} draws:".format(ndraws),
dp_draws = Series([norm_dp() for _ in range(ndraws)])
print dp_draws.unique().size


I am working on the file "2015-07-30-sampling-from-a-hierarchical-dirichlet-process.ipynb".
The first task is:
Let's illustrate again with a standard normal base measure. We can construct a function base_measure that generates samples from it.
Can you write Python code for it?

from scipy.stats import norm

base_measure = lambda: norm().rvs()


I am working on the file "2015-07-30-sampling-from-a-hierarchical-dirichlet-process.ipynb".
The first task is:
The second plot doesn't look very much like the first! The level to which a sample from a Dirichlet process approximates the base distribution is a function of the dispersion parameter $\alpha$. Because I set $\alpha=10$ (which is relatively small), the approximation is fairly course. In terms of memoization, a small $\alpha$ value means the stochastic memoizer will more frequently reuse values already seen instead of drawing new ones.
This nesting procedure, where a sample from one Dirichlet process is fed into another Dirichlet process as a base distribution, is more than just a curiousity. It is known as a Hierarchical Dirichlet Process, and it plays an important role in the study of Bayesian Nonparametrics (more on this in a future post).
Without the stochastic memoization framework, constructing a sampler for a hierarchical Dirichlet process is a daunting task. We want to be able to draw samples from a distribution drawn from the second level Dirichlet process. However, to be able to do that, we need to be able to draw samples from a distribution sampled from a base distribution of the second-level Dirichlet process: this base distribution is a distribution drawn from the first-level Dirichlet process.
Though it appeared that we would need to be able to fully construct the first level sample (by drawing a countably infinite number of samples from the first-level base distribution). However, stochastic memoization allows us to only construct the first distribution just-in-time as it is needed at the second-level.
We can define a Python class to encapsulate the Hierarchical Dirichlet Process as a base class of the Dirichlet process.
Can you write Python code for it?

class HierarchicalDirichletProcessSample(DirichletProcessSample):
    def __init__(self, base_measure, alpha1, alpha2):
        first_level_dp = DirichletProcessSample(base_measure, alpha1)
        self.second_level_dp = DirichletProcessSample(first_level_dp, alpha2)

    def __call__(self):
        return self.second_level_dp()


I am working on the file "2015-07-30-sampling-from-a-hierarchical-dirichlet-process.ipynb".
The first task is:
We can sample directly from the probability distribution drawn from the Hierarchical Dirichlet Process.
Can you write Python code for it?

pd.Series(norm_hdp() for _ in range(10000)).hist()
_=plt.title("Histogram of samples from distribution drawn from Hierarchical DP")


I am working on the file "2015-07-30-sampling-from-a-hierarchical-dirichlet-process.ipynb".
The first task is:
norm_hdp is not equivalent to the Hierarchical Dirichlet Process; it samples from a single distribution sampled from this HDP. Each time we instantiate the norm_hdp variable, we are getting a sampler for a unique distribution. Below we sample five times and get five different distributions.
Can you write Python code for it?

for i in range(5):
    norm_hdp = HierarchicalDirichletProcessSample(base_measure, alpha1=10, alpha2=10)
    _=pd.Series(norm_hdp() for _ in range(100)).hist()
    _=plt.title("Histogram of samples from distribution drawn from Hierarchical DP")
    _=plt.figure()


I am working on the file "2015-08-02 sparse coding with keras.ipynb".
The first task is:
... 100 iterations later ...
Can you write Python code for it?

# Visualizing results
A = model.params[0].get_value()
I = grid2d(A)
plt.imshow(I)


I am working on the file "2015-08-02 sparse coding with keras.ipynb".
The first task is:
And that's all we need. Notice that effectively, a sparse coding layer is a recursive layer, similar to an RNN layer. Here we can backpropagate the gradients through optimization step to optimize for $\mathbf{A}$. Theano grad does that backpropagating thorough scan. Thus, the Maximization step of the EM-algorithm will be handled by Keras's optimization and I didn't need to write anything for that.
A Sparse Coding layer is available as part of Seya a library that I'm writing as a companion to Keras. Let's give it a try next.
Hands on example
Assuming you already have Anaconda (or something like that), scikit-learn, Keras, and stuff. Just install a few extra things I'm writing.
Seya for sparse coding layers:
    pip install git+https://github.com/EderSantana/seya.git

Agnez for deep learning viz stuff. This will be great one day... one day:
    pip install git+https://github.com/EderSantana/agnez.git

Install the extra stuff your enviroment may ask. Like Seaborn: pip install seaborn. Also download the following dataset of natural image patches here (thanks to Jiquan Ngiam).
Can you write Python code for it?

# Download a natural image patches dataset
%matplotlib inline
import os
import numpy as np
import theano
from scipy.io import loadmat
#os.system('wget http://cs.stanford.edu/~jngiam/data/patches.mat')
import matplotlib.pyplot as plt
from sklearn.datasets import fetch_mldata

from keras.models import Sequential
from keras.regularizers import l2
from keras.optimizers import RMSprop
from seya.layers.coding import SparseCoding

from agnez import grid2d

floatX = theano.config.floatX

S = loadmat('patches.mat')['data'].T.astype(floatX)
print S.shape

# REF: http://peekaboo-vision.blogspot.com/2012/04/learning-gabor-filters-with-ica-and.html
image_patches = fetch_mldata("natural scenes data")
X = image_patches.data

# 1000 patches a 32x32
# not so much data, reshape to 16000 patches a 8x8
#S = S.reshape(1000, 4, 8, 4, 8)
#X = np.rollaxis(X, 3, 2).reshape(-1, 8 * 8)
#print X.shape

mean = S.mean(axis=0)
S -= mean[np.newaxis] # we remove the mean otherwise Sparse Coding will focus only on that

model = Sequential()
model.add(
    SparseCoding(
        input_dim=256,
        output_dim=49, # we are learning 49 filters,
        n_steps = 100, # remember the self.n_steps in the scan loop?
        truncate_gradient=1, # no backpropagation through time today now,
                             # just regular sparse coding
        W_regularizer=l2(.00005),
        return_reconstruction=True # we will output Ax which approximates the input
    )
)

rmsp = RMSprop(lr=.1)
model.compile(loss='mse', optimizer=rmsp) # RMSprop for Maximization as well

nb_epoch = 100
batch_size = 100
model.fit(S, # input
          S, # and output are the same thing, since we are doing generative modeling.
          batch_size=batch_size,
          nb_epoch=nb_epoch,
          show_accuracy=False,
          verbose=2)


I am working on the file "2015-08-02 sparse coding with keras.ipynb".
The first task is:
The get_output method is the feedforward step of the layer. I chose this to be the Expectation (optimization for $\mathbf{x}$) of the sparse coding. Notice that we need to do a few steps of optimization for $\mathbf{x}$ before we consider doing an step for $\mathbf{A}$. This is to give the sparse codes enough time to converge. Thus, we will need a sort of for loop inside our get_output. In Theano, for loops are implemented with scan.
Theano's scan is composed of a step function, which is the main operation to transform old states into new ones and a few sequential and non-sequential parameters along with initial states values. So, here is how we define our get_output.
Can you write Python code for it?

def get_output(self, train=False):
    s = self.get_input(train) # input data to be modeled
    initial_x = alloc_zeros_matrix(self.batch_size, self.output_dim)
    # initialize sparse codes with zeros.
    # Again note that the coefficients here got
    # output_dim as its last dimension because this
    # a generative model.
    outputs, updates = theano.scan(
            self._step, # function operated in the main loop
            sequences=[], # iterable input sequences, we don't need this here
            outputs_info=[initial_states, ]*3 + [None, ], # initial states,
            # I'll explain why we have 4 initial states.
            non_sequences=[inputs, prior], # this is kept the same for the entire for loop
            n_steps=self.n_steps, # since sequences is empty, scan needs this
                                  # information to know when to stop
            truncate_gradient=self.truncate_gradient) # how much backpropagation
                                                      # through time/iteration you need.
    if self.return_reconstruction:
            return outputs[-1][-1] # return the approximation of the input
    else:
            return outputs[0][-1] # return the sparse codes


I am working on the file "2015-08-02 sparse coding with keras.ipynb".
The first task is:
This is basically the outer definition of a for loop. The following _step function defines what happens inside:
Can you write Python code for it?

def _step(self,
          x_t, # this is the code itself at the beginning of the Expectation step.
               # First values are zero, remember?
          accum_1, # We are not going to use regular SGD to optimize
          accum_2, # for x. It is too slow, we will use RMSprop instead.
                   # Those are its internal helper variables.
          inputs # Fixed input data
          ):
        outputs = T.dot(x_t, self.A) # Ax, its faster to use row orientation here,
                                     # C heritage you know?
        rec_error = T.sqr(inputs - outputs).sum() # First right hand term of L (s - Ax)^2
        l1_norm = (self.gamma * diff_abs(x_t)).sum() # diff_abs is sqrt(x^2 + 1e-7)
                                                     # we were talking about
        cost = rec_error + l1_norm # this the cost function L
        x, new_accum_1, new_accum_2 = _RMSPropStep(cost, x_t, accum_1, accum_2)
        # _RMSPropStep does the step
        # optimization x <- x + n*grad(L,x)
        return x, new_accum_1, new_accum_2, outputs


I am working on the file "2015-08-03-nonparametric-latent-dirichlet-allocation.ipynb".
The first task is:
A sample from this topic distribution is a distribution over terms. However, unlike our base distribution which returns each term distribution with equal probability, the topics will be unevenly weighted.
Can you write Python code for it?

for topic, count in Counter([tuple(topic_distribution()) for _ in range(10000)]).most_common():
    print "count:", count, "topic:", [round(prob, 2) for prob in topic]


I am working on the file "2015-08-03-nonparametric-latent-dirichlet-allocation.ipynb".
The first task is:
For each document, we will draw a topic distribution from the Dirichlet process:
Can you write Python code for it?

topic_distribution = DirichletProcessSample(base_measure=base_distribution,
                                            alpha=topic_dirichlet_parameter)


I am working on the file "2015-08-03-nonparametric-latent-dirichlet-allocation.ipynb".
The first task is:
Here are the documents we generated:
Can you write Python code for it?

for doc in documents.values():
    print doc


I am working on the file "2015-08-03-nonparametric-latent-dirichlet-allocation.ipynb".
The first task is:
Recall that a sample from a Dirichlet process is a distribution that approximates (but varies from) the base distribution. In this case, a sample from the Dirichlet process will be a distribution over topics that varies from the uniform distribution we provided as a base. If we use the stick-breaking metaphor, we are effectively breaking a stick one time and the size of each portion corresponds to the proportion of a topic in the document.
To construct a sample from the DP, we need to again define our DP class:
Can you write Python code for it?

from scipy.stats import beta
from numpy.random import choice

class DirichletProcessSample():
    def __init__(self, base_measure, alpha):
        self.base_measure = base_measure
        self.alpha = alpha

        self.cache = []
        self.weights = []
        self.total_stick_used = 0.

    def __call__(self):
        remaining = 1.0 - self.total_stick_used
        i = DirichletProcessSample.roll_die(self.weights + [remaining])
        if i is not None and i < len(self.weights) :
            return self.cache[i]
        else:
            stick_piece = beta(1, self.alpha).rvs() * remaining
            self.total_stick_used += stick_piece
            self.weights.append(stick_piece)
            new_value = self.base_measure()
            self.cache.append(new_value)
            return new_value

    @staticmethod
    def roll_die(weights):
        if weights:
            return choice(range(len(weights)), p=weights)
        else:
            return None


I am working on the file "2015-08-03-nonparametric-latent-dirichlet-allocation.ipynb".
The first task is:
The term distribution vector $\underline\Phi$ is a collection of samples from a Dirichlet distribution. This describes how our 3 terms are distributed across each of the two topics.
Can you write Python code for it?

from scipy.stats import dirichlet, poisson
from numpy import round
from collections import defaultdict
from random import choice as stl_choice

term_dirichlet_vector = num_terms * [term_dirichlet_parameter]
term_distributions = dirichlet(term_dirichlet_vector, 2).rvs(size=num_topics)
print term_distributions


I am working on the file "2015-08-03-nonparametric-latent-dirichlet-allocation.ipynb".
The first task is:
This sample from the base Dirichlet process is our infinite sided die. It is a probability distribution over a countable infinite number of topics.
The fact that our die is countably infinite is important. The sampler base_distribution draws topics (term-distributions) from an uncountable set. If we used this as the base distribution of the Dirichlet process below each document would be constructed from a completely unique set of topics. By feeding base_distribution into a Dirichlet Process (stochastic memoizer), we allow the topics to be shared across documents.
In other words, base_distribution will never return the same topic twice; however, every topic sampled from base_dp would be sampled an infinite number of times (if we sampled from base_dp forever). At the same time, base_dp will also return an infinite number of topics. In our formulation of the the LDA sampler above, our base distribution only ever returned a finite number of topics (num_topics); there is no num_topics parameter here.
Given this setup, we can generate documents from the hierarchical Dirichlet process with an algorithm that is essentially identical to that of the original latent Dirichlet allocation generative sampler:
Can you write Python code for it?

nested_dp_parameter = 10

topic_index = defaultdict(list)
documents = defaultdict(list)

for doc in range(num_documents):
    topic_distribution_rvs = DirichletProcessSample(base_measure=base_dp,
                                                    alpha=nested_dp_parameter)
    document_length = poisson(mean_document_length).rvs()
    for word in range(document_length):
        topic_distribution = topic_distribution_rvs()
        topic_index[doc].append(tuple(topic_distribution))
        documents[doc].append(choice(vocabulary, p=topic_distribution))


I am working on the file "2015-08-03-nonparametric-latent-dirichlet-allocation.ipynb".
The first task is:
To generate each word in the document, we draw a sample topic from the topic distribution, and then a term from the term distribution (topic).
Can you write Python code for it?

topic_index = defaultdict(list)
documents = defaultdict(list)

for doc in range(num_documents):
    topic_distribution_rvs = DirichletProcessSample(base_measure=base_distribution,
                                                    alpha=topic_dirichlet_parameter)
    document_length = poisson(mean_document_length).rvs()
    for word in range(document_length):
        topic_distribution = topic_distribution_rvs()
        topic_index[doc].append(tuple(topic_distribution))
        documents[doc].append(choice(vocabulary, p=topic_distribution))


I am working on the file "2015-08-03-nonparametric-latent-dirichlet-allocation.ipynb".
The first task is:
To recap: for each document we draw a sample from a Dirichlet Process. The base distribution for the Dirichlet process is a categorical distribution over term distributions; we can think of the base distribution as an $n$-sided die where $n$ is the number of topics and each side of the die is a distribution over terms for that topic. By sampling from the Dirichlet process, we are effectively reweighting the sides of the die (changing the distribution of the topics).
For each word in the document, we draw a sample (a term distribution) from the distribution (over term distributions) sampled from the Dirichlet process (with a distribution over term distributions as its base measure). Each term distribution uniquely identifies the topic for the word. We can sample from this term distribution to get the word.
Given this formulation, we might ask if we can roll an infinite sided die to draw from an unbounded number of topics (term distributions). We can do exactly this with a Hierarchical Dirichlet process. Instead of the base distribution of our Dirichlet process being a finite distribution over topics (term distributions) we will instead make it an infinite Distribution over topics (term distributions) by using yet another Dirichlet process! This base Dirichlet process will have as its base distribution a Dirichlet distribution over terms.
We will again draw a sample from a Dirichlet Process for each document. The base distribution for the Dirichlet process is itself a Dirichlet process whose base distribution is a Dirichlet distribution over terms. (Try saying that 5-times fast.) We can think of this as a countably infinite die each side of the die is a distribution over terms for that topic. The sample we draw is a topic (distribution over terms).
For each word in the document, we will draw a sample (a term distribution) from the distribution (over term distributions) sampled from the Dirichlet process (with a distribution over term distributions as its base measure). Each term distribution uniquely...
Can you write Python code for it?

term_dirichlet_vector = num_terms * [term_dirichlet_parameter]
base_distribution = lambda: dirichlet(term_dirichlet_vector).rvs(size=1)[0]

base_dp_parameter = 10
base_dp = DirichletProcessSample(base_distribution, alpha=base_dp_parameter)


I am working on the file "2015-09-02-fitting-a-mixture-model.ipynb".
The first task is:
Doing each of these three updates in sequence makes a complete Gibbs step for our mixture model. Here is a function to do that:
Can you write Python code for it?

def gibbs_step(state):
    update_assignment(state)
    update_mixture_weights(state)
    update_cluster_means(state)


I am working on the file "2015-09-02-fitting-a-mixture-model.ipynb".
The first task is:
Each time we run gibbs_step, our state is updated with newly sampled assignments. Look what happens to our histogram after 5 steps:
Can you write Python code for it?

for _ in range(5):
    gibbs_step(state)
plot_clusters(state)


I am working on the file "2015-09-02-fitting-a-mixture-model.ipynb".
The first task is:
Initially, we assigned each data point to a random cluster. We can see this by plotting a histogram of each cluster.
Can you write Python code for it?

def plot_clusters(state):
    gby = pd.DataFrame({
            'data': state['data_'],
            'assignment': state['assignment']}
        ).groupby(by='assignment')['data']
    hist_data = [gby.get_group(cid).tolist()
                 for cid in gby.groups.keys()]
    plt.hist(hist_data,
             bins=20,
             histtype='stepfilled', alpha=.5 )

plot_clusters(state)


I am working on the file "2015-09-02-fitting-a-mixture-model.ipynb".
The first task is:
Suddenly, we are seeing clusters that appear very similar to what we would intuitively expect: three Gaussian clusters.
Another way to see the progress made by the Gibbs sampler is to plot the change in the model's log-likelihood after each step. The log likehlihood is given by:
$$
\log p(\mathbf{x} \,|\, \pi, \theta_1, \theta_2, \theta_3)
\propto \sum_x \log \left(
    \sum_{k=1}^3 \pi_k \exp
    \left{
        -(x-\theta_k)^2 / (2\sigma^2)
    \right}
\right)
$$
We can define this as a function of our state object:
Can you write Python code for it?

def log_likelihood(state):
    """Data log-likeliehood

    Equation 2.153 in Sudderth
    """

    ll = 0
    for x in state['data_']:
        pi = state['pi']
        mean = state['cluster_means']
        sd = np.sqrt(state['cluster_variance_'])
        ll += np.log(np.dot(pi, stats.norm(mean, sd).pdf(x)))
    return ll

state = initial_state()
ll = [log_likelihood(state)]
for _ in range(20):
    gibbs_step(state)
    ll.append(log_likelihood(state))
pd.Series(ll).plot()


I am working on the file "2015-09-14 - TD2 - Variables et conditions.ipynb".
The first task is:
Ci-dessous, on affiche la valeur de la variable age
Can you write Python code for it?


I am working on the file "2015-09-14 - TD2 - Variables et conditions.ipynb".
The first task is:
Ci-dessous, on crée une variable nouvelAge et on lui affecte la valeur de la variable age augmentée de 1. Puis, on affiche le contenu de la variable nouvelAge
Can you write Python code for it?


I am working on the file "2015-09-14 - TD2 - Variables et conditions.ipynb".
The first task is:
Exercice 1. Écrire un algorithme qui demande à l'utilisateur le rayon d'un cercle et qui affiche sur la sortie standard son  diamètre, son périmètre et son aire.
Can you write Python code for it?

r_cercle = int(input ("Rayon du cercle ?"))
pi = 3.14
d_cercle = r_cercle *2
p_cercle =pi*d_cercle
a_cercle = pi*r_cercle*r_cercle
print("Diametre du cercle =", d_cercle, "cm")
print("Périmetre du cercle =", p_cercle, "cm")
print("Aire du cercle = ", a_cercle, "cm²")


I am working on the file "2015-09-14 - TD2 - Variables et conditions.ipynb".
The first task is:
Maintenant, on va demander une valeur à l'utilisateur. Le premier exemple ne fonctionne pas (rappel : il est là pour vous montrer à quoi ressemble un message d'erreur). Le second fonctionne : il demande une valeur à l'utilisateur, stocke cette valeur dans la variable a, puis affiche la valeur contenue dans la variable a.
Can you write Python code for it?


I am working on the file "2015-09-14 - TD2 - Variables et conditions.ipynb".
The first task is:
Pour dire à Python que la valeur saisie est un entier, on peut lui préciser au moment de la saisie... Et ça fonctionne pareil pour les floats (les flottants, i.e. les nombres à virgules).
Can you write Python code for it?


I am working on the file "2015-09-24 - TD5 - Exercices de révision.ipynb".
The first task is:
Exercice 2 Écrire une procédure qui affiche la table de multiplication (de 0 à 10) de n'importe quel nombre.
Solution. Cet exercice a été corrigé au tableau.
Exercice 3 Écrire une procédure qui affiche la table de multiplication (de m à n) de n'importe quel nombre.
Can you write Python code for it?

def afficherTableMult(a, m, n):
    '''
    :entree: a, l'entier à multiplier
    :entree: m et n, les bornes de l'affichage
    :sortie:
    :pré-condition: n>m
    :post-condition: la table de multiplication est affichée sur la sortie standar
    '''
    for i in range(m,n+1):
        print(a,"*",i,"=",a*i)

# Ne jamais oublier de tester son algorithme / son programme !
a = 5
m = 3
n = 8
afficherTableMult(a, m, n)


I am working on the file "2015-09-24 - TD5 - Exercices de révision.ipynb".
The first task is:
Exercices à faire à la maison
DM1 Calculer les coefficients d’une droite à partir de deux points.
DM2 Convertir une durée en secondes. Le squelette de l'algorithme est donné ci-dessous.
Can you write Python code for it?

def duree_en_secondes(j, h, m, s):
    """
    :entrée j: int
    :entrée h: int
    :entrée m: int
    :entrée s: float
    :sortie ds: float
    :pré-cond: j > 0 , 0 ≤ h < 24 , 0 ≤ m < 60, 0 ≤ s < 60
    :post-cond: ds est le nombre de secondes correspond à
                une durée de j jours, h heures, m minutes et s secondes.
    """


I am working on the file "2015-09-24 - TD5 - Exercices de révision.ipynb".
The first task is:
Solution 2. Cette solution minimaliste est très astucieuse. Lorsque l'on opte pour une "astuce" comme ça, il est préférable de le commenter pour se souvenir, plus tard, de ce que faisait l'astuce. Notez que l'on reprend exactement les mêmes tests.
Can you write Python code for it?

def plusPetitBis(a, b, c):
    """
    :entrée a: int
    :entrée b: int
    :entrée c: int
    :sortie : int
    :pré-cond: a, b, c sont des entiers
    :post-cond: le nombre retourné est le plus petit des trois
    """
    # Astuce : on compare les valeurs deux à deux et on retient
    #  la valeur la plus petite dans la variable a.
    if(a>b):
        a=b
    if(a>c):
        a=c
    return(a)

# Et surtout, on n'oublie pas de tester notre fonction :
a=1
b=2
c=3
minimum=plusPetitBis(a,b,c)
print(minimum)

a=2
b=3
c=1
minimum=plusPetitBis(a,b,c)
print(minimum)

a=3
b=1
c=2
minimum=plusPetitBis(a,b,c)
print(minimum)


I am working on the file "2015-10-08 - TD7 - Les algos mystères, le monde à l'envers .ipynb".
The first task is:
Ci-dessous, quelques tests pour voir comment se comportent les chaînes de caractères :
Can you write Python code for it?

toto = "? eluor ac"
print(len(toto))
print(toto[3])

x = 11
n = 0
while n < 10:
    print(n, x)
    if x % 2 == 0:
        x = x // 2
    else:
        x = 3 * x + 1
    n = n + 1


I am working on the file "2015-10-08 - TD7 - Les algos mystères, le monde à l'envers .ipynb".
The first task is:
Exercice 2 :  Les pièges de Python, ou de l'importance de tester

Rappels sur les bornes et les intervalles
Rappels sur les opérateurs logiques : and, or, not
Rappels sur les opérateurs de comparaison : <, <=, >, >=, !=, ==
Rappels sur la priorité des opérateurs mathématiques (+ - * /  ** // %)
Rappels sur la notion de float
Can you write Python code for it?

print(round(14.34, 1))
a = 2.456789
b = 2.55
c = 2.50
print(round(a, 2))
print(round(a))
print(round(b))
print(round(c))

a = 5
print(a*2 * 3.14)

a = False
b = True
c = True
print((a and b) or c)
print(a and (b or c))
print(a and b or c)


I am working on the file "2015-10-08 - TD7 - Les algos mystères, le monde à l'envers .ipynb".
The first task is:
Exercice 3 :  Tiens, une nouvelle notation ?!?
Can you write Python code for it?

tt = [1, 5, 12, 38, 43, 34, 21, 2, 15]
a = []
b = []
print ("Longueur de tt", len(tt), " et longueur de a ", len(a))
i = 0
while i < len(tt):
  if tt[i] % 2 == 0:
      a.append(tt[i])
  else:
      b.append(tt[i])
  i = i + 1
print("truc :", a)
print("bidule :", b)

a = []
chaine = "bonjour!"
while chaine != "":
  print("Entrez un truc !")
  chaine = input()
  if chaine != "":
      a.append(chaine)
print(a)


I am working on the file "2015-10-08 - TD7 - Les algos mystères, le monde à l'envers .ipynb".
The first task is:
Notez la différence entre la version précédente et la version suivante... L'ordre dans lequel les affectations des valeurs aux variables sont effectuées est important !
Can you write Python code for it?

a, b, c = 1, 1, 1

print(1)
while c<15:
    a = b
    b = a+b
    c = c+1
    print(b)

ch = "? eluor ac"
lc = len(ch)
i = lc - 1
sortie = ""
while i >= 0:
  sortie = sortie + ch[i]
  i = i - 1
print(sortie)


I am working on the file "2015-10-15 - TD8 - Boucles imbriquées.ipynb".
The first task is:
1 : Correction des trois erreurs de syntaxe
Can you write Python code for it?

def mystere(unparametre): # il manquait les deux points ici...
    unevariable = True
    uneautrevariable = 0
    while unevariable:
        truc = unparametre // 10
        uneautrevariable+=1
        if truc == 0: # la condition du if doit être une formule booléenne... il fallait donc un == et non un =
            print("truc 0") # Il y avait un problème d'in
        if truc > 0:
            unparametre = truc
        else:
            unevariable = False
    return uneautrevariable

print(mystere(1))
print(mystere(22))
print(mystere(4444))


I am working on the file "2015-10-15 - TD8 - Boucles imbriquées.ipynb".
The first task is:
2 : Que fait le programme ?
Le programme retourne le nombre de chiffres dans l'écriture en base 10 d'un nombre passé en paramètre. Le premier if fait un print qui ne sert à rien.
3 : Quelles valeurs choisir pour le test
Les valeurs aux limites des conditions : par exemple, 0, 10 ou 100...
Les valeurs négatives.
Les nombres flotants.
Exercice du damier :
L'objectif de cet exercice est d'afficher un damier de M*N cases, chaque case étant soit vide (caractère espace) soit dessinée par un ensemble de lettre.
Chaque case du damier aura pour dimension 2M*N (cf. plus bas pour l'explication du 2M).
Par exemple, si l'on demande l'affichage d'un damier de 5*4, on aura 20 cases en tout, et le damier fera 5 colonnes par 4 lignes. Chaque case du damier fera quant à elle 10 caractères de large, et 4 caractères de haut.
Les cases pleines sont dessinées avec deux caractères passés en paramètres de la méthode. On affiche d'abord M fois le premier caractère, puis M fois le second. La largeur d'une case est donc de 2M. On doit répéter l'opération N fois, car la case doit avoir une hauteur de N. Les cases vides fonctionnent selon le même principe, sauf qu'au lieu d'afficher des caractères passés en paramètres, on affiche des espaces.
L'une des principales difficultés, outre les aspects algorithmiques, est que l'affichage doit se faire ligne par ligne (et non case par case), et qu'il va donc nous falloir décomposer le problème en fonction de cette contrainte.
Cet exercice est artificiellement difficile. Il a pour but de vous apprendre à décomposer un problème compliqué en plusieurs étapes "simples", et de vous faire travailler le concept de boucles imbriquées.
Nous allons donc le corriger par étapes.
Étape 1.
Tout d'abord, nous allons écrire et tester le code qui permet d'afficher une ligne (en faisant le choix arbitraire que la première case de la ligne est une case pleine.
Can you write Python code for it?

def afficher_damier(m, n, c, d):
    '''
    :entree m: nombre de cases en largeur dans le damier
    :entree n: nombre de cases en hauteur dans le damier
    :entree c: premier caractère choisi pour l'affichage
    :entree d: deuxieme caractère choisi pour l'affichage
    :pre-cond: m et n sont des entiers strictement positifs
    :pre-cond: c et d sont des chaînes de caractères de longueur 1 (autrement dit, des caractères)
    :post-cond: un damier de m*n cases est affiché à l'écran, en utilisant les caractères c et d, comme dans l'exemple ci-dessous :
    afficher_damier(2,3, "n", "z")
    nnzz
    nnzz
    nnzz
        nnzz
        nnzz
        nnzz
    nnzz
    nnzz
    '''
    case = 1 # On utilise une variable pour repérer sur quelle case on est.
    chaine = "" # On utilise une variable pour stocker la chaine à afficher et on s'occupera de l'affichage de cette chaine à chaque fin de ligne
    for compteur in range(m): # On fait une boucle pour construire une ligne entière de cases
        if case % 2 == 0: # Si on est sur une case paire, on ajoute à la chaine le nombre d'espaces (2M)
            chaine = chaine + (2*m* " ")
        else: # Sinon (i.e. la case est impaire), on ajoute à la chaine M fois le premier caractère, puis M fois le second.
            chaine = chaine + (m*c + m*d)
        case += 1 # Quand on a fini d'écrire une case, on passe à la case suivante
    print(chaine) # Quand on a fini d'écrire une ligne entière, on l'affiche.

afficher_damier(3, 4, "-", "6") # Hop, et on n'oublie pas de tester notre travail.


I am working on the file "2015-10-15 - TD8 - Boucles imbriquées.ipynb".
The first task is:
Étape 2. Bien. Maintenant que l'on affiche une ligne correctement, on va s'occuper d'afficher toutes les lignes nécessaires à l'affichage de la première rangée de cases de notre damier. On reprend notre code précédent, et on le modifie un peu... Dans le code ci-dessous, seules les lignes modifiées ont été commentées.
Can you write Python code for it?

def afficher_damier(m, n, c, d):
    '''
    idem
    '''
    for compteur_ligne in range(n): # On ajoute juste une boucle pour itérer sur la hauteur que doit faire une case...
        case = 1
        chaine = ""
        for compteur in range(m):
            if case % 2 == 0:
                chaine = chaine + (2*m* " ")
            else:
                chaine = chaine + (m*c + m*d)
            case += 1
        print(chaine)

afficher_damier(3, 4, "-", "6") # Hop, et on n'oublie pas de tester notre travail.


I am working on the file "2015-10-15 - TD8 - Boucles imbriquées.ipynb".
The first task is:
Étape 3. Parfait. Nous avons réalisé la première ligne de cases de notre damier. Maintenant, il s'agit de répéter le processus pour le reste du damier... mais il nous faut trouver une astuce pour gérer le décalage que doivent avoir les cases d'une ligne sur l'autre. Ici, on utilise l'observation suivante. On numérote les cases de 1 à m, et les lignes de 1 à n. On a fait le choix que la case 1 de la ligne 1 était une case pleine. Ainsi, la case 2 de la ligne 1 est une case vide, mais la case 1 de la ligne 2 est vide aussi. En généralisant, on observe assez vite que lorsque la somme du rang de la case et du numéro de ligne est paire, la case est pleine, et dans l'autre cas, la case est vide. C'est cette observation que l'on va utiliser dans le code suivant.
Can you write Python code for it?

def afficher_damier(m, n, c, d):
    '''
    idem
    '''
    ligne = 1 # On introduit un compteur pour savoir quelle ligne on est en train de traiter
    for compteur_lignes_damier in range(n): # On itère sur les lignes
        for compteur_ligne in range(n):
            case = 1
            chaine = ""
            for compteur in range(m):
                if (case + ligne) % 2 != 0: # On modifie la condition qui nous permet de savoir ce que l'on doit écrire dans la case...
                    chaine = chaine + (2*m* " ")
                else:
                    chaine = chaine + (m*c + m*d)
                case += 1
            print(chaine)
        ligne += 1 # Quand on a fini d'écrire une ligne complète, on pense à incrémenter notre compteur de ligne...

afficher_damier(3, 4, "-", "6") # Hop, et on n'oublie pas de tester notre travail.
afficher_damier(6, 2, "g", "!")


I am working on the file "2015-10-19 - TD9 - Les chaînes de caractères.ipynb".
The first task is:
Exercice maison (facile) : Modifiez le code précédent de sorte à ce que l'on passe le caractère à rechercher en paramètre...
Can you write Python code for it?

# Corrigé de l'exo 1 avec un while
def nbEwhile(chaine):
    '''
    :entree chaine: une chaine de caractères dans laquelle on va compter le nb de e
    :sortie compteur: le nb de e dans la chaine
    '''
    compteur = 0
    i = 0
    while (i < len(chaine)):
        if chaine[i] == 'e':
            compteur +=1
        i +=1
    return compteur
nbEwhile("Je veux un café !")

# Exercice : qu'est-ce qui ne fonctionne pas dans cet algorithme (supposé répondre à l'exo 1) ?
def nbEBis(chaine):
    compteur = 0
    for lettre in chaine:
        if lettre == 'e':
            compteur +=1
        return compteur
nbEBis("Moi ça roule !")


I am working on the file "2015-10-19 - TD9 - Les chaînes de caractères.ipynb".
The first task is:
Exercice maison : Pour chaque algorithme vu dans ce TD, donner la complexité moyenne et la complexité dans le pire des cas.
Can you write Python code for it?

#Corrigé de l'exercice 6
ch=str(input("Entrez votre caractere"))

def mots (ch):
    '''
    :entree:une chaine de caracteres
    :sortie: nbr de mots
    '''
    mot=int()
    for i in range (0,len(ch)-1):
        if ch[i]!=' ' and ch[i+1]==' ':
            mot+=1
    return (mot+1)
print (mots(ch))


I am working on the file "2015-10-19 - TD9 - Les chaînes de caractères.ipynb".
The first task is:
Exercice maison : Réécrivez cette méthode de recherche de la dernière occurrence en parcourant la liste à partir de la fin.
Can you write Python code for it?

#Corrigé de l'exercice 5
def miroir(chaine):
    '''
    :entre la chaine de caractère
    :sortie la chaine de caractere avec les caracteres inversés
    '''
    stri=""
    for i in range(0,len(chaine)):
        stri+=chaine[len(chaine)-i-1]

    return stri

print(miroir("coucou vous"))


I am working on the file "2015-10-19 - TD9 - Les chaînes de caractères.ipynb".
The first task is:
Exercice maison : identifiez le problème du code précédent et corrigez-le.
Remarque sur le code ci-dessous : la recherche d'un caractère dans une chaîne est rendue très facile en Python... mais nous faisons de l'algo ici... nous cherchons donc des solutions génériques et transposables à d'autres langages de programmation. C'est pourquoi nous allons éviter de prendre ce genre de raccourci !
Can you write Python code for it?

if "c" in "abc":
    print("Le caractère est dans la chaîne")

if "d" in "abc":
    print("Le caractère est dans la chaîne")
else:
    print("Le caractère n'est pas dans la chaîne...")

# Corrigé de l'exo 2
def compteurMinuscules(chaine):
    '''
    :entree chaine: une chaîne de caractères
    :sortie nombre: le nombre de minuscules dans la chaîne
    '''
    nombre = 0
    for i in range(len(chaine)):
        if chaine[i] <= "z" and chaine[i] >= "a":
            nombre +=1
    return nombre

compteurMinuscules("J'écris TouT PeTIT i")

# Corrigé de l'exo 3

def premiereOccurence(chaine, car):
    """
    :entree chaine: une chaine de caractère
    :entree car: le caractère recherché
    :sortie occu: la position de la premiere occurence du caractere
    :post-cond: occu = -1 si le caractere n'est pas dans la chaîne
    """

    i=0
    occu = -1
    while(occu == -1 and i < len(chaine)):
        if(chaine[i] == car):
            occu = i
        i +=1
    return occu

print(premiereOccurence("Bonjour", "j"))
print(premiereOccurence("Bonjour", "e"))

# Corrigé de l'exo 4
def derniereOccurence(chaine, car):
    '''
    :entree chaine: une chaîne de caractères
    :entre car: le caractère recherché
    :sortie occu: l'indice de la dernière apparition du caractère dans la chaîne
    :post-cond: occu = -1 si le caractère n'est pas présent...
    '''
    retour =-1
    for pos in range (len(chaine)):
        if chaine[pos] == car:
            retour = pos
    return retour

print(derniereOccurence("Hello world", "l"))
print(derniereOccurence("Hello world", "d"))
print(derniereOccurence("Hello world", "f"))


I am working on the file "2015-10-22 - TD10 - Les chaînes de caractères - suite.ipynb".
The first task is:
Exercice 2 : Même question, mais cette fois, avec un 'while' et non un 'for'
Can you write Python code for it?

def compte_car_bis(chaine, lettre):
    compteur = 0
    i = 0
    while i < len(chaine):
        if chaine[i] == lettre:
            compteur += 1
        i += 1
    return compteur

print("Test de compte_car")
print(compte_car("Villeurbanne", "l"))
print(compte_car("La vie est belle", "e"))
print(compte_car("Toto", "z"))

print("Test de compte_car_bis")
print(compte_car_bis("Villeurbanne", "l"))
print(compte_car_bis("La vie est belle", "e"))
print(compte_car_bis("Toto", "z"))


I am working on the file "2015-10_Lecture/Lecture2/code/1_Intro_Theano.ipynb".
The first task is:
Don't confuse x,W, b with inputX, inputW, inputB. x,W,b contain pointer to your symbols in the compute graph. inputX,inputW,inputB contains your data.
Shared Variables and Updates
See: http://deeplearning.net/software/theano/tutorial/examples.html#using-shared-variables

Using shared variables, we can create an internal state.
Creation of a accumulator:
At the beginning initialize the state to 0
With each function call update the state by certain value

Later, in your neural networks, the weight matrices $W$ and the bias values $b$ will be stored as internal state / as shared variable.
Shared variables improve performance, as you need less transfer between your Python code and the execution of the compute graph (which is written & compiled from C code)
Shared variables can also be store on your graphic card
Can you write Python code for it?

import theano
import theano.tensor as T
import numpy as np

#Define my internal state
init_value = 1

state = theano.shared(value=init_value, name='state')

#Define my operation f(x) = 2*x
x = T.lscalar('x')
z = 2*x

accumulator = theano.function(inputs=[], outputs=z, givens={x: state})

print accumulator()
print accumulator()


I am working on the file "2015-10_Lecture/Lecture2/code/1_Intro_Theano.ipynb".
The first task is:
Printing of the graph
You can print the graph for the above value of z. For details see:
http://deeplearning.net/software/theano/library/printing.html
http://deeplearning.net/software/theano/tutorial/printing_drawing.html
To print the graph, futher libraries must be installed. In 99% of your development time you don't need the graph printing function. Feel free to skip this section
Can you write Python code for it?

#Graph for z
theano.printing.pydotprint(z, outfile="pics/z_graph.png", var_with_name_simple=True)

#Graph for function f (after optimization)
theano.printing.pydotprint(f, outfile="pics/f_graph.png", var_with_name_simple=True)


I am working on the file "2015-10_Lecture/Lecture2/code/1_Intro_Theano.ipynb".
The first task is:
Shared Variables

We use theano.shared() to share a variable (i.e. make it internally available for Theano)
Internal state variables are passed by compile time via the parameter givens. So to compute the ouput z, use the shared variable state for the input variable x
For information on the borrow=True parameter see: http://deeplearning.net/software/theano/tutorial/aliasing.html
In most cases we can set it to true and increase by this the performance.

Updating Shared Variables

Using the updates-parameter, we can specify how our shared variables should be updated
This is useful to create a train function for a neural network.
We create a function train(data) which computes the error and gradient
The computed gradient is then used in the same call to update the shared weights
Training just becomes: for mini_batch in mini_batches: train(mini_batch)
Can you write Python code for it?

#New accumulator function, now with an update

# Put your code here to update the internal counter

print accumulator(1)
print accumulator(1)
print accumulator(1)


I am working on the file "2015-10_Lecture/Lecture2/code/1_Intro_Theano.ipynb".
The first task is:
The graph fo z:

The graph for f:

Simple matrix multiplications
The following types for input variables are typically used:
byte: bscalar, bvector, bmatrix, btensor3, btensor4
16-bit integers: wscalar, wvector, wmatrix, wtensor3, wtensor4
32-bit integers: iscalar, ivector, imatrix, itensor3, itensor4
64-bit integers: lscalar, lvector, lmatrix, ltensor3, ltensor4
float: fscalar, fvector, fmatrix, ftensor3, ftensor4
double: dscalar, dvector, dmatrix, dtensor3, dtensor4
complex: cscalar, cvector, cmatrix, ctensor3, ctensor4

scalar: One element (one number)
vector: 1-dimension
matrix: 2-dimensions
tensor3: 3-dimensions
tensor4: 4-dimensions
As we do not need perfect precision we use mainly float instead of double. Most GPUs are also not able to handle doubles.
So in practice you need: iscalar, ivector, imatrix and fscalar, fvector, vmatrix.
Task: Implement the function $$f(x,W,b) = \tanh(xW+b)$$ with $x \in \mathbb{R}^n, b \in \mathbb{R}^k, W \in \mathbb{R}^{n \times k}$.
$n$ input dimension and $k$ output dimension
Can you write Python code for it?

import theano
import theano.tensor as T
import numpy as np

# Put your code here


I am working on the file "2015-10_Lecture/Lecture2/code/1_Intro_Theano_solution.ipynb".
The first task is:
Don't confuse x,W, b with inputX, inputW, inputB. x,W,b contain pointer to your symbols in the compute graph. inputX,inputW,inputB contains your data.
Shared Variables and Updates
See: http://deeplearning.net/software/theano/tutorial/examples.html#using-shared-variables

Using shared variables, we can create an internal state.
Creation of a accumulator:
At the beginning initialize the state to 0
With each function call update the state by certain value

Later, in your neural networks, the weight matrices $W$ and the bias values $b$ will be stored as internal state / as shared variable.
Shared variables improve performance, as you need less transfer between your Python code and the execution of the compute graph (which is written & compiled from C code)
Shared variables can also be store on your graphic card
Can you write Python code for it?


I am working on the file "2015-10_Lecture/Lecture2/code/1_Intro_Theano_solution.ipynb".
The first task is:
Now you can invoke f and pass the input values, i.e. f(1,1), f(10,-3) and the result for this operation is returned.
Can you write Python code for it?


I am working on the file "2015-10_Lecture/Lecture2/code/1_Intro_Theano_solution.ipynb".
The first task is:
Printing of the graph
You can print the graph for the above value of z. For details see:
http://deeplearning.net/software/theano/library/printing.html
http://deeplearning.net/software/theano/tutorial/printing_drawing.html
To print the graph, futher libraries must be installed. In 99% of your development time you don't need the graph printing function. Feel free to skip this section
Can you write Python code for it?


I am working on the file "2015-10_Lecture/Lecture2/code/1_Intro_Theano_solution.ipynb".
The first task is:
Shared Variables

We use theano.shared() to share a variable (i.e. make it internally available for Theano)
Internal state variables are passed by compile time via the parameter givens. So to compute the ouput z, use the shared variable state for the input variable x
For information on the borrow=True parameter see: http://deeplearning.net/software/theano/tutorial/aliasing.html
In most cases we can set it to true and increase by this the performance.

Updating Shared Variables

Using the updates-parameter, we can specify how our shared variables should be updated
This is useful to create a train function for a neural network.
We create a function train(data) which computes the error and gradient
The computed gradient is then used in the same call to update the shared weights
Training just becomes: for mini_batch in mini_batches: train(mini_batch)
Can you write Python code for it?

#New accumulator function, now with an update
inc = T.lscalar('inc')
accumulator = theano.function(inputs=[inc], outputs=(state,z), givens={x: state}, updates=[(state,state+inc)])

print accumulator(1)
print accumulator(1)
print accumulator(1)


I am working on the file "2015-10_Lecture/Lecture2/code/1_Intro_Theano_solution.ipynb".
The first task is:
The graph fo z:

The graph for f:

Simple matrix multiplications
The following types for input variables are typically used:
byte: bscalar, bvector, bmatrix, btensor3, btensor4
16-bit integers: wscalar, wvector, wmatrix, wtensor3, wtensor4
32-bit integers: iscalar, ivector, imatrix, itensor3, itensor4
64-bit integers: lscalar, lvector, lmatrix, ltensor3, ltensor4
float: fscalar, fvector, fmatrix, ftensor3, ftensor4
double: dscalar, dvector, dmatrix, dtensor3, dtensor4
complex: cscalar, cvector, cmatrix, ctensor3, ctensor4

scalar: One element (one number)
vector: 1-dimension
matrix: 2-dimensions
tensor3: 3-dimensions
tensor4: 4-dimensions
As we do not need perfect precision we use mainly float instead of double. Most GPUs are also not able to handle doubles.
So in practice you need: iscalar, ivector, imatrix and fscalar, fvector, vmatrix.
Task: Implement the function $$f(x,W,b) = \tanh(xW+b)$$ with $x \in \mathbb{R}^n, b \in \mathbb{R}^k, W \in \mathbb{R}^{n \times k}$.
$n$ input dimension and $k$ output dimension
Can you write Python code for it?

import theano
import theano.tensor as T
import numpy as np

x = T.fvector('x')
W = T.fmatrix('W')
b = T.fvector('b')

activation = T.dot(x,W)+b
z = T.tanh(activation)

f = theano.function(inputs=[x,W,b], outputs=[activation,z])


I am working on the file "2015-10_Lecture/Lecture2/code/2_MNIST.ipynb".
The first task is:
Softmax Layer
The softmax-layer computes:
$$\text{output} = \text{softmax}(xW+b)$$
As for the hidden layer, we allow the parameterization of the number of neurons. The weight matrix and bias vector is initialized to zero.
As we performt a single label classification task, we use the negative log-likelihood as error function:
$$E(x,W,b) = -log(o_y)$$
with $o_y$ the output for label $y$.
Can you write Python code for it?

import numpy
import theano
import theano.tensor as T

class SoftmaxLayer(object):
    def __init__(self, input, n_in, n_out):
        self.W = theano.shared(value=numpy.zeros((n_in, n_out),
                                                dtype=theano.config.floatX), name='W')
        self.b = theano.shared(value=numpy.zeros((n_out,),
                                                dtype=theano.config.floatX), name='b')

        # Put your code here, implement a function to compute softmax(x*W+b)


I am working on the file "2015-10_Lecture/Lecture2/code/2_MNIST.ipynb".
The first task is:
Time to train the model
Now we can train our model by calling train_model(mini_batch_index). To predict labels, we can use the function predict_labels(data).
Can you write Python code for it?


I am working on the file "2015-10_Lecture/Lecture2/code/2_MNIST_solution.ipynb".
The first task is:
Softmax Layer
The softmax-layer computes:
$$\text{output} = \text{softmax}(xW+b)$$
As for the hidden layer, we allow the parameterization of the number of neurons. The weight matrix and bias vector is initialized to zero.
As we performt a single label classification task, we use the negative log-likelihood as error function:
$$E(x,W,b) = -log(o_y)$$
with $o_y$ the output for label $y$.
Can you write Python code for it?

import numpy
import theano
import theano.tensor as T

class SoftmaxLayer(object):
    def __init__(self, input, n_in, n_out):
        self.W = theano.shared(value=numpy.zeros((n_in, n_out),
                                                dtype=theano.config.floatX), name='W')
        self.b = theano.shared(value=numpy.zeros((n_out,),
                                                dtype=theano.config.floatX), name='b')

        #Compute the output of the softmax layer, we call it P(y | x), i.e. how
        #likely is the label y given the input x
        self.p_y_given_x = T.nnet.softmax(T.dot(input, self.W) + self.b)

        #For prediction we select the most probable output
        self.y_pred = T.argmax(self.p_y_given_x, axis=1)

        # parameters of the model
        self.params = [self.W, self.b]

    def negative_log_likelihood(self, y):
        """
        Computes the negative log-likelihood. The function explained:

        T.log(self.p_y_given_x): Compute the negative log-likelihood of p_y_given_x
        T.arange(y.shape[0]), y]: Select the neuron at position y, our label
        T.mean(): Compute the average over our mini batch
        """
        return -T.mean(T.log(self.p_y_given_x)[T.arange(y.shape[0]), y])


I am working on the file "2015-10_Lecture/Lecture2/code/2_MNIST_solution.ipynb".
The first task is:
Time to train the model
Now we can train our model by calling train_model(mini_batch_index). To predict labels, we can use the function predict_labels(data).
Can you write Python code for it?

number_of_minibatches = len(train_x) / batch_size
print "%d mini batches" % (number_of_minibatches)

number_of_epochs = 10
print "%d epochs" % number_of_epochs

#
def compute_accurarcy(dataset_x, dataset_y):
    predictions = predict_labels(dataset_x)
    errors = sum(predictions != dataset_y) #Number of errors
    accurarcy = 1 - errors/float(len(dataset_y))
    return accurarcy

for epoch in xrange(number_of_epochs):
    #Train the model on all mini batches
    for idx in xrange(0, number_of_minibatches):
        train_model(idx)

    accurarcy_dev = compute_accurarcy(dev_x, dev_y)
    accurarcy_test = compute_accurarcy(test_x, test_y)

    print "%d epoch: Accurarcy on dev: %f, accurarcy on test: %f" % (epoch, accurarcy_dev, accurarcy_test)

print "DONE"


I am working on the file "2015-10_Lecture/Lecture2/code/3_Intro_Lasagne.ipynb".
The first task is:
Create the Train Function
After loading the data and defining the MLP, we can now create the train function.
Can you write Python code for it?

# Parameters
n_in = 28*28
n_hidden = 50
n_out = 10

# Create the necessary training and predict labels function


I am working on the file "2015-10_Lecture/Lecture2/code/3_Intro_Lasagne.ipynb".
The first task is:
Train the model
We run the training for some epochs and output the accurarcy of our network
Can you write Python code for it?

#Put your code here to train the model using mini batches


I am working on the file "2015-10_Lecture/Lecture2/code/3_Intro_Lasagne_Solution.ipynb".
The first task is:
Build the MLP
Now we use the provided layers from Lasagne to build our MLP
Can you write Python code for it?

def build_mlp(n_in, n_hidden, n_out, input_var=None):
    #Input layer, 1 dimension = number of samples, 2 dimension = input, our 28*28 image
    l_in = lasagne.layers.InputLayer(shape=(None, n_in), input_var=input_var)

    # Our first hidden layer with n_hidden units
    # As nonlinearity we use tanh, you could also try rectify
    l_hid1 = lasagne.layers.DenseLayer(incoming=l_in,
                num_units=n_hidden, nonlinearity=lasagne.nonlinearities.tanh,
                W=lasagne.init.GlorotUniform())

    # Our output layer (a softmax layer)
    l_out = lasagne.layers.DenseLayer(incoming=l_hid1,
            num_units=n_out, nonlinearity=lasagne.nonlinearities.softmax)

    return l_out