I am working on the file "Project Wrangling OpenStree Map project.ipynb".
The first task is:

Analysis:

The few corrections have been done and give the good results

Nothing more to report on users, postcode and city

On street types, it is interresting that the majority of street type are "Allée" (ie. Alley) whereas it should be "Rue" (ie. Street) as most of street are "Rue". That could be explained by the fact that users focus on main street and not on small street ("Rue")

On top source, we can see we have update date in most of cases, I propose to make a last study on this item in order to check if this dataset is uptodate

Synthesis:

This OSM dataset seems far cleaner as examples during udacity course

5)  Last analysis

On this last chapter, I propose to check date related to source tag in the dataset. It will enable to give a status about how uptodate the dataset is.

For this, I propose to extract the date from:

the value of 'source_date' tag in nodes_tags and ways_tags tables.

<tag k="source_date" v="04/04/2012" />

To notice that date could be under different formats:

18/03/2013

2017-09-04

10/2016

But as the year is only interesting for my topic, I choose to use a regular expression to match the year: "^(19|20)\d{2}\$"

the string of 'source' tag in nodes_tags and ways_tags tables.

<tag k="source" v="cadastre-dgi-fr source : Direction Générale des Impôts - Cadastre. Mise à jour : 2009" />

I use the same regular expression to match the year: "^(19|20)\d{2}$"

Can you write Python code for it?

import re
from collections import Counter

def display_year(result):
    keylist =result.keys()
    keylist.sort(reverse=True)
    for i in keylist:
        print i,result[i]

x=[]

### Connection to Data Base ###
conn = sqlite3.connect('OSM.db')
conn.text_factory = str
c = conn.cursor()

### Extract year from source tag value ###
result=c.execute('''SELECT value from (select * from nodes_tags UNION ALL select * from ways_tags) where key='source';''')
j=0
for i in result:
    m = re.search('(19|20)\d{2}$',i[0])
    if m:
        x.append(m.group(0))

## Extract year from source_date tag value ###
result=c.execute('''SELECT value from (select * from nodes_tags UNION ALL select * from ways_tags) where key='source_date';''')
j=0
for i in result:
    m = re.search('(19|20)\d{2}$',i[0])
    if m:
        x.append(m.group(0))

#Display of the year
display_year(dict(Counter(x)))
print '\nTotal source tag with year:{}'.format(len(x))
conn.close()


I am working on the file "Project Wrangling OpenStree Map project.ipynb".
The first task is:

Objectives:

This project plans to get a dataset from Open Street, to clean it and to import into a Data Base (SQLite for me)

Then the data will be analysed and observations reported into this document

Observations that could lead to dataset corrections will be reinjected into data import step

A last anaylysis will be done on the cleaned data

Material:

     Map Area: Colomiers, Occitanie, France. This the place where I live.
    (This map has been extracted from Metro extracts.The file size is around 58 MB.)

 The tools used for the analysis are:

Python as coding language

Jupyter for code editing + analysis report

SQLite for Data Base

Method:

Ths document mixes comments and code in order to demonstrate step by step the methode used for data analysis.

But if you prefer to have document and code split, you can toggle on/off the raw code by clicking on the button on top this notebook.

You will find the code alone with following name: Project+Wrangling+OpenStree+Map+project.py

1) First observations on data set
In a first step, I'm checking structure of the dataset with a sample function (given in Udacity instructions).
The output is an XML file that I'm checking with a simple tool like Notepad++.
Can you write Python code for it?

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import xml.etree.ElementTree as ET  # Use cElementTree or lxml if too slow

OSM_FILE = "Colomiers.osm"  # Replace this with your osm file
SAMPLE_FILE = "sample.osm"

k = 2 # Parameter: take every k-th top level element

def get_element(osm_file, tags=( 'node','way', 'relation')):
    """Yield element if it is the right type of tag

    Reference:
    http://stackoverflow.com/questions/3095434/inserting-newlines-in-xml-file-generated-via-xml-etree-elementtree-in-python
    """
    context = iter(ET.iterparse(osm_file, events=('start', 'end')))
    _, root = next(context)
    for event, elem in context:
        if event == 'end' and elem.tag in tags:
            yield elem
            root.clear()

with open(SAMPLE_FILE, 'wb') as output:
    output.write('<?xml version="1.0" encoding="UTF-8"?>\n')
    output.write('<osm>\n  ')

    # Write every kth top level element
    for i, element in enumerate(get_element(OSM_FILE)):
        if i % k == 0:
            output.write(ET.tostring(element, encoding='utf-8'))

    output.write('</osm>')


I am working on the file "Project Wrangling OpenStree Map project.ipynb".
The first task is:
In opposition of Project instructions, I decided to not add position parameter into Ways nodes table because I don't get the benefit to add this parameter in this table.
Indeed, precise position parameters are already in nodes table (lat, long) and nodes are already referenced into ways nodes table. It is in opposition of normalization rules that are to not have redondant information.
Can you write Python code for it?

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

def import_sql(csv_file_nodes='nodes.csv',csv_file_nodes_tags='nodes_tags.csv',csv_file_ways='ways.csv',csv_file_ways_tags='ways_tags.csv',csv_file_ways_nodes='ways_nodes.csv'):
    ################# Connect the Data Base 'OSM.db' ################
    conn = sqlite3.connect('OSM.db')
    conn.text_factory = str
    c = conn.cursor()

    ### Import table nodes ###
    print "Importing",csv_file_nodes,"..."
    df = pd.read_csv(csv_file_nodes)
    df.to_sql('nodes', conn, if_exists='append', index=False)
    print "Import done"

    ### Import table nodes tags ###
    print "Importing",csv_file_nodes_tags,"..."
    df = pd.read_csv(csv_file_nodes_tags)
    df.to_sql('nodes_tags', conn, if_exists='append', index=False)
    print "Import done"

    ### Import table ways ###
    print "Importing",csv_file_ways,"..."
    df = pd.read_csv(csv_file_ways)
    df.to_sql('ways', conn, if_exists='append', index=False)
    print "Import done"

    ### Import table ways tags ###
    print "Importing",csv_file_ways_tags,"..."
    df = pd.read_csv(csv_file_ways_tags)
    df.to_sql('ways_tags', conn, if_exists='append', index=False)
    print "Import done"

    ### Import table ways nodes ###
    print "Importing",csv_file_ways_nodes,"..."
    df = pd.read_csv(csv_file_ways_nodes)
    df.to_sql('ways_nodes', conn, if_exists='append', index=False)
    print "Import done"

    conn.close()

############## Main ##############

import_sql()


I am working on the file "Project-2/Big Data P2.ipynb".
The first task is:
Getting Q, R, S, and T in the DataFrame and Ordered according to Time
Once you can see that Q, R, S, and T are being consistently detected (T should be marked by the burst ends) you can do the below process to label them.
Can you write Python code for it?

peaks_df = Results['Peaks']['Mean1'] # R wave location
valleys_df = Results['Valleys']['Mean1'] # Q and S wave locations
burst_ends = Results['Bursts']['Mean1']['Burst End'] #burst ends indicate approx. T wave location

# make new DataFrame for Burst Ends which is indexed by Burst Ends
# b.c. Results['Bursts']['Mean1'] is indexed by Burst Start.
bursts_df = pd.DataFrame(burst_ends.values, index=burst_ends.values, columns=['Burst Ends'])

#combine peaks and valleys dfs and sort by time
# this will allow us to easily find peaks,
# and find valleys before and after.
peaks_df['time'] = peaks_df.index
valleys_df['time'] = valleys_df.index
bursts_df['time'] = bursts_df.index
peaks_df['amplitude'] = peaks_df["Peaks Amplitude"]
valleys_df['amplitude'] = valleys_df["Valley Amplitude"]
bursts_df['amplitude'] = float(Settings['Threshold'])

#Label R and T. Unfortunatly, setting Q and S (the valleys) is much harder (See next cell).
peaks_df['letter'] = 'R'
bursts_df['letter'] = "T"

combined = peaks_df.append(valleys_df).sort('time')

#now combine w. Burst Ends and remove unnecessary columns, sort by time again
times = combined.append(bursts_df).drop([u'Burst Ends', u'Intervals', u'Peaks Amplitude', u'Valley Amplitude'],axis=1)
sorted_times = times.sort('time')
#remove na vals only from amplitude col.
sorted_times['amplitude'] = sorted_times['amplitude']
sorted_times.head()


I am working on the file "Project-2/Big Data P2.ipynb".
The first task is:
Requires installation of PyBursts (https://github.com/rpoddighe/pybursts)
Can you write Python code for it?

import pybursts
#this doesn't group entire QRST complex, but it does seem to get the QRS complex pretty well (h =1)

bursts = pybursts.kleinberg(combined.t, s=2, gamma=.001)
#print bursts
#b = [i[1:] for i in bursts]
bursts_df =  pd.DataFrame(data = bursts, index = range(len(bursts)), columns=['h','start','end'])
print len(bursts_df)
bursts_df.head()

ymin, ymax = plt.ylim()
maxh = max(bursts_df['h'])
sorted_bursts = bursts_df.sort("h")
"""
for r,c in zip(sorted_bursts.iterrows(),colors.cnames):#zip(bursts_df.start, bursts_df.end, colors.cnames):#bursts_df.iterrows():
    #print r
    h,s,e = r[1]
    #print h
    if h != 1:
        continue
    plt.vlines(s, ymin, ymax, color=c, lw=maxh-h)
    plt.vlines(e, ymin, ymax, color=c, lw=maxh-h)
    plt.hlines(ymin, s,e, color=c, lw=maxh-h)
    plt.hlines(ymax, s, e, color=c, lw=maxh-h)
    #break"""
plt.plot(data.index, data.values,'y', label = "original")
plt.plot(hist.x,hist.y, "k",label="smooth", lw=2)
plt.plot(vx, vy ,"ro", label="valleys")
plt.plot(px, py, "co", label="peaks")

plt.xlim(5,10)
plt.legend()
plt.show()

import pybursts

offsets = [4, 17, 23, 27, 33, 35, 37, 76, 77, 82, 84, 88, 90, 92]
print pybursts.kleinberg(offsets, s=2, gamma=0.1)


I am working on the file "Project-2/Big Data P2.ipynb".
The first task is:
The following is a hack-y way to set the Q and S valleys, but it works. Later I might try to do it with booleans. <b>Feel free to try another way to do this!!! (Actually, really, please do. This method is pretty slow.)</b>
Can you write Python code for it?

#reindex by integer
sorted_times['num'] = range(len(sorted_times))
sorted_times_reindexed = sorted_times.set_index('num')

letters = pd.DataFrame(index=sorted_times_reindexed.index, columns=['letters'], dtype=str)
for i in sorted_times_reindexed.index:

    #try to get the previous and next values with error handeling for edge cases
    try:
        prev_value = sorted_times_reindexed.loc[i-1, "letter"]
    except:
        prev_value=''
    try:
        next_value = sorted_times_reindexed.loc[i+1,"letter"]
    except:
        next_value=""

    #letters['time'][i] = sorted_times_reindexed.loc[i,'time']
    if (next_value == "T") and (prev_value == "R"):
        letters['letters'][i] = "S"
    elif next_value == "R":
        letters['letters'][i] = "Q"
    else:
        letters['letters'][i] = sorted_times_reindexed.loc[i,'letter']

sorted_times_reindexed['letter'] = letters['letters']
sorted_times_reindexed.head(13) #see that valleys which are not immediately before or after R are left as NaN!

sorted_times_time_indexed = sorted_times_reindexed.set_index('time')
final = sorted_times_time_indexed.dropna()
final.head()

s1=0
e1=20
final[s1:e1].plot()
plt.show()


I am working on the file "Project-2/Big Data P2.ipynb".
The first task is:
Using BASS
Can you write Python code for it?

for key in ['Mean1']:
    #represent what a complete feature looks like for this particular ECG
    # s = burst start, e = burst end, v = valley, p=peak
    features_key = 'vspevspe'
    ''

    #count number of peaks and valleys in the features key
    peak_count = features_key.count('p')
    valley_count = features_key.count('v')

    peaks_df = Results['Peaks'][key].dropna() # R and T wave locations
    valleys_df = Results['Valleys'][key].dropna() # Q and S wave locations

    # make bursts_df which contains 'Burst Start' vals and 'Burst End' vals
    try:
        burst_ends = Results['Bursts'][key]['Burst End']
        burst_starts = Results['Bursts'][key]['Burst Start']
        bursts_df = pd.concat([burst_starts, burst_ends], axis=1)

        bursts_df.columns = ['Start','End']
    except:
        print "No bursts."

    #get peak and valley times
    print (peaks_df.head())
    peak_time = peaks_df.index.values
    valley_time = valleys_df.index.values

print peaks_df["Peaks Amplitude"].head()
n, pEdges = np.histogram(peaks_df["Peaks Amplitude"],bins=peak_count)
n, vEdges = np.histogram(valleys_df,bins=valley_count)
print pEdges, vEdges


I am working on the file "Project-2/Big Data P2.ipynb".
The first task is:
```
Signature: hist.get_peaks(method='slope', peak_amp_thresh=5e-05,
                          valley_thresh=3e-05, intervals=None, lookahead=20, avg_interval=100)
Docstring:
This function expects SMOOTHED histogram. If you run it on a raw histogram,
there is a high chance that it returns no peaks.
method can be interval/slope/hybrid.
    The interval-based method simply steps through the whole histogram
    and pick up the local maxima in each interval, from which irrelevant
    peaks are filtered out by looking at the proportion of points on
    either side of the detected peak in each interval, and by applying
    peal_amp_thresh and valley_thresh bounds.
Slope approach uses, of course slope information, to find peaks,
which are then filtered by applying peal_amp_thresh and
valley_thresh bounds.

Hybrid approach combines the peaks obtained using slope method and
interval-based approach. It retains the peaks/valleys from slope method
if there should be a peak around the same region from each of the methods.

peak_amp_thresh is the minimum amplitude/height that a peak should have
in a normalized smoothed histogram, to be qualified as a peak.
valley_thresh is viceversa for valleys!
If the method is interval/hybrid, then the intervals argument must be passed
and it should be an instance of Intervals class.
If the method is slope/hybrid, then the lookahead and avg_window
arguments should be changed based on the application.
They have some default values though.
The method stores peaks in self.peaks in the following format:
{"peaks":[[peak positions], [peak amplitudes]],
"valleys": [[valley positions], [valley amplitudes]]}
```
Can you write Python code for it?

#things you can change
smooth_amount = 9
peak_amp_threshold = 3e-05
valley_amp_threshold = -2e-05


I am working on the file "Project-2/Project-2-Master/Project-2-Master.ipynb".
The first task is:
<img src='SCN5A_tracks.png'>
Figure 1: LQTS3 Mutations in the SCN5A Gene. This diagram shows the SCN5A gene, broken up into 4 fragments proceeding from 5' at the upper left to 3' at the bottom right. There are 11 mutations indicated in blue, and they all appear to be toward the 3' end of the gene. In fact, the several mutations closest to the 3' end are tightly grouped within the same short region.
Physionet ECG Data
In order to analyze ECG data, we first needed to find some raw ECG data. This section of the notebook downloads the ECG data from the Physionet ECG-ID Database in the form of tab-delimited .txt files with no header.
The file titles follow this pattern: Person_##rec_##.txt
The first colum of data is time (s), and the second column of data is ECG I filtered (mV).
According to the database description, there are 310 records in total, made up of between 2 and 20 records from each of 90 individuals. Each record is 20 seconds long.
In order for this section of the notebook to work, you must first install the Physionet WFDB software and be able to run a %%bash cell. Download the RECORDS file from Physionet, and put it in the same directory as this notebook.
Click here to see the results of this section of the notebook.
Can you write Python code for it?

# Read the RECORDS file into a list
records = open('RECORDS','r')
rec_list = records.read()
records.close()

# Get rid of the last line, which
# is an extra empty entry for some reason.
rec_list = rec_list.split('\n')
del rec_list[-1]

# Create a dict where the keys are the Person
# and the values are a list of corresponding rec
rec_dict = {}

for i in range(len(rec_list)):
    rec = rec_list[i].split('/')
    if rec[0] in rec_dict:
        rec_dict[rec[0]].append(rec[1])
    else:
        rec_dict[rec[0]] = []
        rec_dict[rec[0]].append(rec[1])

# Generate a temporary file named Person##
# where each line is a rec entry
for k,v in rec_dict.iteritems():
    p = open('people','a')
    p.write(k+'\n')
    p.close()
    f = open(k,'w')
    for i in range(len(v)):
        f.write(v[i]+'\n')
    f.close()


I am working on the file "Project-2/Project-2-Master/Project-2-Master.ipynb".
The first task is:
Check Accuracy of Event Detection
Determine if event detection settings need to be adjusted
Can you write Python code for it?

## OPTIONAL
# check accuracy of event detection

# raw data
plt.plot(data.index, data.values, 'r', c='deepskyblue', label="raw data")
# all the peaks and valleys
plt.plot(orig_amps.t, orig_amps.A, "o", c='cadetblue', label="")
plt.show()


I am working on the file "Project-2/Project-2-Master/Project-2-Master.ipynb".
The first task is:
ECG Analysis
The major tasks in analyzing the ECG for long-QT intervals are:
<ol>
   <li> Detecting peaks and valleys (data may require smoothing first) </li>
   <li> Matching Q, R, S, and T to the detected peaks and valleys. </li>
</ol>

Import modules, Load data
For the next section, the pypeaks module is required and the seaborn module is strongly suggested.
Can you write Python code for it?

import pypeaks as pypk
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from matplotlib import colors
try:
    import seaborn as sns
    sns.set_style("whitegrid")
except:
    print ("Consider installing Seaborn! "
           "Its a great statistical plotting module "
           "with pretty default settings to make your plots prettier!! "
           "http://stanford.edu/~mwaskom/software/seaborn/")

file_name = '../p01_r01_f.txt'
all_data = pd.DataFrame.from_csv(file_name, header=[0,1], sep="\t")

# clip off first 1/2 second of data
# this is probably optional
# and will change for each data set
data = all_data[0.5:]


I am working on the file "Project-2/Project-2-Master/Project-2-Master.ipynb".
The first task is:
Use a %%bash cell to download Physionet files, and use Vim to delete the first 2 lines (headers) from each file.
Can you write Python code for it?

%%bash
for i in $(cat people); do
    for ii in $(cat $i); do
        rdsamp -r ecgiddb/$i/$ii -H -f 0 -t 20 -v -ps -s 'ECG I filtered' >$i$ii.txt
        vi -c ':1,2d' -c ':wq' $i$ii.txt;
    done
done

# Remove temporary files
import os
os.remove('people')
for k in rec_dict.iterkeys():
    os.remove(k)


I am working on the file "Project-Code/thrones2vec/GoT_vectors.ipynb".
The first task is:
Arya - John + Ghost = ?
Can you write Python code for it?


I am working on the file "Project-Code/thrones2vec/GoT_vectors.ipynb".
The first task is:
Dimensionality reduction using tsne
Can you write Python code for it?

Y = tsne(all_word_vecs.astype('float64'))

points = pd.DataFrame(
    [
        (word, coords[0], coords[1])
        for word, coords in [
            (word, Y[thrones2vec.vocab[word].index])
            for word in thrones2vec.vocab
        ]
    ],
    columns=["word", "x", "y"]
)

points.head(10)

sns.set_context("poster")

%pylab inline

points.plot.scatter("x", "y", s=10, figsize=(20, 12))

def plot_region(x_bounds, y_bounds):
    slice = points[
        (x_bounds[0] <= points.x) &
        (points.x <= x_bounds[1]) &
        (y_bounds[0] <= points.y) &
        (points.y <= y_bounds[1])
    ]
    inwords=[]
    ax = slice.plot.scatter("x", "y", s=35, figsize=(10, 8))
    for i, point in slice.iterrows():
        inwords.append(point.word)
        ax.text(point.x + 0.005, point.y + 0.005, point.word, fontsize=11)
    print(", ".join(inwords))

plot_region(x_bounds=(-8.0,-6.0), y_bounds=(-29.0, -26.0))

points.loc[points["word"]=="Jaime",:]

plot_region(x_bounds=(28,34), y_bounds=(-5.0,-2.0))

def coords(word):
    coord = points.loc[points["word"]==word,:].values[0]
    return coord[1],coord[2]

coords("Jon")

def plot_close_to(word):
    x,y = coords(word)
    plot_region(x_bounds=(x-1.0,x+1.0), y_bounds=(y-1.0,y+1.0))

plot_close_to("apples")

plot_close_to("Winterfell")

plot_close_to("Payne")

for i in ["king","queen","man","woman"]:
    print(coords(i))

plot_close_to("Needle")


I am working on the file "Project-Code/thrones2vec/GoT_vectors.ipynb".
The first task is:
Model correctly predicted the relationship between the Two Families
Can you write Python code for it?

families = ["Lannister","Baratheon"]
best_avgs(families, all_word_vecs, 10)


I am working on the file "Project-Code/thrones2vec/GoT_vectors.ipynb".
The first task is:
What patterns do we see if we average a set of similar/related words(names) and find the words with highest cosine similarity with our average vector?
Can you write Python code for it?

def best_avgs(words, all_vecs,k=10):

    from operator import itemgetter

    ## get word embeddings for the words in our input array
    embs = np.array([thrones2vec[word] for word in words])

    #calculate its average
    avg = np.sum(embs,axis=0)/len(words)

    # Cosine Similarity with every word vector in the corpus
    denom = np.sqrt(np.sum(all_vecs*all_vecs,axis=1,keepdims=True)) \
            * np.sqrt(np.sum(avg*avg))

    similarity = all_vecs.dot(avg.T).reshape(all_vecs.shape[0],1) \
           / denom
    similarity = similarity.reshape(1,all_vecs.shape[0])[0]

    # Finding the 10 largest words with highest similarity
    # Since we are averaging we might end up getting the input words themselves
    # among the top values
    # we need to make sure we get back len(words)+k closest words and then
    # remove all input words we supplied

    nClosest = k + len(words)

    # Get indices of the most similar word vectors to our avgvector
    ind = np.argpartition(similarity, -(nClosest))[-nClosest:]

    names = [thrones2vec.index2word[indx] for indx in ind]
    similarity = similarity[ind]
    uniq = [(person,similar) for person,similar in zip(names,similarity) if person not in words]

    return sorted(uniq,key=itemgetter(1),reverse=True)[:k]


I am working on the file "Project-Euler/ProjectEuler17.ipynb".
The first task is:
Finally used your count_letters function to solve the original question.
Can you write Python code for it?

print(sum(count_letters(1000)))

print(sum(count_letters(998)))

assert True # use this for gradig the answer to the original question.


I am working on the file "Project-Euler/ProjectEuler17.ipynb".
The first task is:
Now define a count_letters(n) that returns the number of letters used to write out the words for all of the the numbers 1 to n inclusive.
Can you write Python code for it?

def count_letters(n):
    """Count the number of letters used to write out the words for 1-n inclusive."""
    lst2 = []
    for entry in number_to_words(n):
        count = 0
        for char in entry:
            if char != ' ' and char != '-':
                count = count + 1
        lst2.append(count)
    return lst2


I am working on the file "Project-Euler/ProjectEuler17.ipynb".
The first task is:
Now write a set of assert tests for your count_letters function that verifies that it is working as expected.
Can you write Python code for it?

assert count_letters(1) == [3]
assert len(count_letters(342)) == 342
assert count_letters(5) == [3,3,5,4,4]

assert True # use this for grading the count_letters tests.


I am working on the file "Project-Euler/ProjectEuler59.ipynb".
The first task is:
Now I know the key is 'god'. I will use that key to decipher the message below. I use the same method to print the message as I did to find the key.
Can you write Python code for it?

message = ""
i = 0
for entry in cipher:
    if i == 0 or i % 3 == 0:
        message = message + chr(entry^ord('g'))
    elif i == 1 or (i-1) % 3 == 0:
        message = message + chr(entry^ord('o'))
    elif i == 2 or (i-2) % 3 == 0:
        message = message + chr(entry^ord('d'))
    i = i + 1
print(message)


I am working on the file "Project-Euler/ProjectEuler59.ipynb".
The first task is:
Now that I know the message I can compute the ASCII sum of the message
Can you write Python code for it?

sum = 0
for char in message:
    sum = sum + ord(char)
print("The ASCII sum is: " + str(sum))

# This cell will be used for grading, leave it at the end of the notebook.


I am working on the file "Project-Folder/Write-Solutions.ipynb".
The first task is:
Equal Mass Disruptor Solutions
Can you write Python code for it?

for bool1 in Boolean:
    CenterOfMass = bool1
    for bool2 in Boolean:
        Hyperbolic_Approach = bool2

        IC = set_IC(Hyperbolic_Approach,M,S)
        for bool3 in Boolean:
            Retrograde_Orbit = bool3

            allic = all_ic(N_array,r_array,IC,Retrograde_Orbit)

            sln = solution(t,allic,N_array,atol,rtol,gamma,Hyperbolic_Approach,M,S)
            x,dx,y,dy,X,dX,Y,dY = coordinate_solution(sln,steps)
            soln = np.array(frame(x,dx,y,dy,X,dX,Y,dY,CenterOfMass,M,S))

            if CenterOfMass == False:
                if Hyperbolic_Approach == True:
                    if Retrograde_Orbit == True:
                        np.savez('Hyp_Ret_M',x = soln[0], dx = soln[1],y = soln[2],dy = soln[3],X = soln[4],dX = soln[5],Y = soln[6],dY = soln[7])
                    else:
                        np.savez('Hyp_Dir_M',x = soln[0], dx = soln[1],y = soln[2],dy = soln[3],X = soln[4],dX = soln[5],Y = soln[6],dY = soln[7])
                else:
                    if Retrograde_Orbit == True:
                        np.savez('Par_Ret_M',x = soln[0], dx = soln[1],y = soln[2],dy = soln[3],X = soln[4],dX = soln[5],Y = soln[6],dY = soln[7])
                    else:
                        np.savez('Par_Dir_M',x = soln[0], dx = soln[1],y = soln[2],dy = soln[3],X = soln[4],dX = soln[5],Y = soln[6],dY = soln[7])
            else:
                if Hyperbolic_Approach == True:
                    if Retrograde_Orbit == True:
                        np.savez('Hyp_Ret_Cen',x = soln[0], y = soln[1],X1 = soln[2],Y1 = soln[3],X2 = soln[4],Y2 = soln[5])
                    else:
                        np.savez('Hyp_Dir_Cen',x = soln[0], y = soln[1],X1 = soln[2],Y1 = soln[3],X2 = soln[4],Y2 = soln[5])
                else:
                    if Retrograde_Orbit == True:
                        np.savez('Par_Ret_Cen',x = soln[0], y = soln[1],X1 = soln[2],Y1 = soln[3],X2 = soln[4],Y2 = soln[5])
                    else:
                       ...


I am working on the file "Project-Folder/Write-Solutions.ipynb".
The first task is:
Writing Heavy Mass Disruptor Solutions
Can you write Python code for it?

M = 1e1
S = 3e1

for bool1 in Boolean:
    CenterOfMass = bool1
    for bool2 in Boolean:
        Hyperbolic_Approach = bool2

        IC = set_IC(Hyperbolic_Approach,M,S)
        for bool3 in Boolean:
            Retrograde_Orbit = bool3

            allic = all_ic(N_array,r_array,IC,Retrograde_Orbit)

            sln = solution(t,allic,N_array,atol,rtol,gamma,Hyperbolic_Approach,M,S)
            x,dx,y,dy,X,dX,Y,dY = coordinate_solution(sln,steps)
            soln = np.array(frame(x,dx,y,dy,X,dX,Y,dY,CenterOfMass,M,S))

            if CenterOfMass == False:
                if Hyperbolic_Approach == True:
                    if Retrograde_Orbit == True:
                        np.savez('Hyp_Ret_M_H',x = soln[0], dx = soln[1],y = soln[2],dy = soln[3],X = soln[4],dX = soln[5],Y = soln[6],dY = soln[7])
                    else:
                        np.savez('Hyp_Dir_M_H',x = soln[0], dx = soln[1],y = soln[2],dy = soln[3],X = soln[4],dX = soln[5],Y = soln[6],dY = soln[7])
                else:
                    if Retrograde_Orbit == True:
                        np.savez('Par_Ret_M_H',x = soln[0], dx = soln[1],y = soln[2],dy = soln[3],X = soln[4],dX = soln[5],Y = soln[6],dY = soln[7])
                    else:
                        np.savez('Par_Dir_M_H',x = soln[0], dx = soln[1],y = soln[2],dy = soln[3],X = soln[4],dX = soln[5],Y = soln[6],dY = soln[7])
            else:
                if Hyperbolic_Approach == True:
                    if Retrograde_Orbit == True:
                        np.savez('Hyp_Ret_Cen_H',x = soln[0], y = soln[1],X1 = soln[2],Y1 = soln[3],X2 = soln[4],Y2 = soln[5])
                    else:
                        np.savez('Hyp_Dir_Cen_H',x = soln[0], y = soln[1],X1 = soln[2],Y1 = soln[3],X2 = soln[4],Y2 = soln[5])
                else:
                    if Retrograde_Orbit == True:
                        np.savez('Par_Ret_Cen_H',x = soln[0], y = soln[1],X1 = soln[2],Y1 = soln[3],X2 = soln[4],Y2 = soln[5])
                   ...


I am working on the file "Project-practice--naive_bayes_tutorial/Naive_Bayes_tutorial.ipynb".
The first task is:
Congratulations! You have implemented Bayes theorem from scratch. Your analysis shows that even if you get a positive test result, there is only a 8.3% chance that you actually have diabetes and a 91.67% chance that you do not have diabetes. This is of course assuming that only 1% of the entire population has diabetes which of course is only an assumption.
 What does the term 'Naive' in 'Naive Bayes' mean ?
The term 'Naive' in Naive Bayes comes from the fact that the algorithm considers the features that it is using to make the predictions to be independent of each other, which may not always be the case. So in our Diabetes example, we are considering only one feature, that is the test result. Say we added another feature, 'exercise'. Let's say this feature has a binary value of 0 and 1, where the former signifies that the individual exercises less than or equal to 2 days a week and the latter signifies that the individual exercises greater than or equal to 3 days a week. If we had to use both of these features, namely the test result and the value of the 'exercise' feature, to compute our final probabilities, Bayes' theorem would fail. Naive Bayes' is an extension of Bayes' theorem that assumes that all the features are independent of each other.
Step 4.2: Naive Bayes implementation from scratch
Now that you have understood the ins and outs of Bayes Theorem, we will extend it to consider cases where we have more than feature.
Let's say that we have two political parties' candidates, 'Jill Stein' of the Green Party and 'Gary Johnson' of the Libertarian Party and we have the probabilities of each of these candidates saying the words 'freedom', 'immigration' and 'environment' when they give a speech:

Probability that Jill Stein says 'freedom': 0.1 ---------> P(J|F)
Probability that Jill Stein says 'immigration': 0.1 -----> P(J|I)

Probability that Jill Stein says 'environment': 0.8 -----> P(J|E)

Probability that Gary Johnson says 'freedom': 0.7 -------> P(G|F)

Probability that Gary Johnson says...
Can you write Python code for it?

'''
Instructions: Compute the probability of the words 'freedom' and 'immigration' being said in a speech, or
P(F,I).

The first step is multiplying the probabilities of Jill Stein giving a speech with her individual
probabilities of saying the words 'freedom' and 'immigration'. Store this in a variable called p_j_text

The second step is multiplying the probabilities of Gary Johnson giving a speech with his individual
probabilities of saying the words 'freedom' and 'immigration'. Store this in a variable called p_g_text

The third step is to add both of these probabilities and you will get P(F,I).
'''

'''
Solution: Step 1
'''
# P(J)
p_j = 0.5

# P(J|F)
p_j_f = 0.1

# P(J|I)
p_j_i = 0.1

p_j_text = p_j * p_j_f * p_j_i
print(p_j_text)

'''
Solution: Step 2
'''
# P(G)
p_g = 0.5

# P(G|F)
p_g_f = 0.7

# P(G|I)
p_g_i = 0.2

p_g_text = p_g * p_g_f * p_g_i
print(p_g_text)

'''
Solution: Step 3: Compute P(F,I) and store in p_f_i
'''
p_f_i = p_j_text + p_g_text
print('Probability of words freedom and immigration being said are: ', format(p_f_i))


I am working on the file "Project-practice--naive_bayes_tutorial/Naive_Bayes_tutorial.ipynb".
The first task is:
Congratulations! You have implemented the Bag of Words process from scratch! As we can see in our previous output, we have a frequency distribution dictionary which gives a clear view of the text that we are dealing with.
We should now have a solid understanding of what is happening behind the scenes in the sklearn.feature_extraction.text.CountVectorizer method of scikit-learn.
We will now implement sklearn.feature_extraction.text.CountVectorizer method in the next step.
Step 2.3: Implementing Bag of Words in scikit-learn
Now that we have implemented the BoW concept from scratch, let's go ahead and use scikit-learn to do this process in a clean and succinct way. We will use the same document set as we used in the previous step.
Can you write Python code for it?

'''
Here we will look to create a frequency matrix on a smaller document set to make sure we understand how the
document-term matrix generation happens. We have created a sample document set 'documents'.
'''
documents = ['Hello, how are you!',
                'Win money, win from home.',
                'Call me now.',
                'Hello, Call hello you tomorrow?']


I am working on the file "Project-practice--naive_bayes_tutorial/Naive_Bayes_tutorial.ipynb".
The first task is:
Data preprocessing with CountVectorizer()
In Step 2.2, we implemented a version of the CountVectorizer() method from scratch that entailed cleaning our data first. This cleaning involved converting all of our data to lower case and removing all punctuation marks. CountVectorizer() has certain parameters which take care of these steps for us. They are:

lowercase = True
The lowercase parameter has a default value of True which converts all of our text to its lower case form.

token_pattern = (?u)\\b\\w\\w+\\b
The token_pattern parameter has a default regular expression value of (?u)\\b\\w\\w+\\b which ignores all punctuation marks and treats them as delimiters, while accepting alphanumeric strings of length greater than or equal to 2, as individual tokens or words.

stop_words
The stop_words parameter, if set to english will remove all words from our document set that match a list of English stop words which is defined in scikit-learn. Considering the size of our dataset and the fact that we are dealing with SMS messages and not larger text sources like e-mail, we will not be setting this parameter value.

You can take a look at all the parameter values of your count_vector object by simply printing out the object as follows:
Can you write Python code for it?

'''
Practice node:
Print the 'count_vector' object which is an instance of 'CountVectorizer()'
'''
print(count_vector)


I am working on the file "Project-practice--naive_bayes_tutorial/Naive_Bayes_tutorial.ipynb".
The first task is:
Instructions:
Fit your document dataset to the CountVectorizer object you have created using fit(), and get the list of words
which have been categorized as features using the get_feature_names() method.
Can you write Python code for it?

'''
Solution:
'''
count_vector.fit(documents)
count_vector.get_feature_names()


I am working on the file "Project-practice--naive_bayes_tutorial/Naive_Bayes_tutorial.ipynb".
The first task is:
Instructions:
Import the sklearn.feature_extraction.text.CountVectorizer method and create an instance of it called 'count_vector'.
Can you write Python code for it?

'''
Solution
'''
from sklearn.feature_extraction.text import CountVectorizer
count_vector = CountVectorizer()


I am working on the file "Project-practice--naive_bayes_tutorial/Naive_Bayes_tutorial.ipynb".
The first task is:
Now we can compute the probability of P(J|F,I), that is the probability of Jill Stein saying the words Freedom and Immigration and P(G|F,I), that is the probability of Gary Johnson saying the words Freedom and Immigration.
Can you write Python code for it?

'''
Instructions:
Compute P(J|F,I) using the formula P(J|F,I) = (P(J) * P(J|F) * P(J|I)) / P(F,I) and store it in a variable p_j_fi
'''

'''
Solution
'''
p_j_fi = p_j_text / p_f_i
print('The probability of Jill Stein saying the words Freedom and Immigration: ', format(p_j_fi))

'''
Instructions:
Compute P(G|F,I) using the formula P(G|F,I) = (P(G) * P(G|F) * P(G|I)) / P(F,I) and store it in a variable p_g_fi
'''

'''
Solution
'''
p_g_fi = p_g_text / p_f_i
print('The probability of Gary Johnson saying the words Freedom and Immigration: ', format(p_g_fi))


I am working on the file "Project-practice--naive_bayes_tutorial/Naive_Bayes_tutorial.ipynb".
The first task is:
Step 3.2: Applying Bag of Words processing to our dataset.
Now that we have split the data, our next objective is to follow the steps from Step 2: Bag of words and convert our data into the desired matrix format. To do this we will be using CountVectorizer() as we did before. There are two  steps to consider here:

Firstly, we have to fit our training data (X_train) into CountVectorizer() and return the matrix.
Secondly, we have to transform our testing data (X_test) to return the matrix.

Note that X_train is our training data for the 'sms_message' column in our dataset and we will be using this to train our model.
X_test is our testing data for the 'sms_message' column and this is the data we will be using(after transformation to a matrix) to make predictions on. We will then compare those predictions with y_test in a later step.
For now, we have provided the code that does the matrix transformations for you!
Can you write Python code for it?

'''
[Practice Node]

The code for this segment is in 2 parts. Firstly, we are learning a vocabulary dictionary for the training data
and then transforming the data into a document-term matrix; secondly, for the testing data we are only
transforming the data into a document-term matrix.

This is similar to the process we followed in Step 2.3

We will provide the transformed data to students in the variables 'training_data' and 'testing_data'.
'''

'''
Solution
'''
# Instantiate the CountVectorizer method
count_vector = CountVectorizer()

# Fit the training data and then return the matrix
training_data = count_vector.fit_transform(X_train)

# Transform testing data and return the matrix. Note we are not fitting the testing data into the CountVectorizer()
testing_data = count_vector.transform(X_test)


I am working on the file "Project-practice--naive_bayes_tutorial/Naive_Bayes_tutorial.ipynb".
The first task is:
Step 3: Tokenization
Tokenizing a sentence in a document set means splitting up a sentence into individual words using a delimiter. The delimiter specifies what character we will use to identify the beginning and the end of a word(for example we could use a single space as the delimiter for identifying words in our document set.)

Instructions:
Tokenize the strings stored in 'sans_punctuation_documents' using the split() method. and store the final document set
in a list called 'preprocessed_documents'.
Can you write Python code for it?

'''
Solution:
'''
preprocessed_documents = []
for i in sans_punctuation_documents:
    preprocessed_documents.append(i.split(' '))
print(preprocessed_documents)


I am working on the file "Project-practice--naive_bayes_tutorial/Naive_Bayes_tutorial.ipynb".
The first task is:
Step 4: Count frequencies
Now that we have our document set in the required format, we can proceed to counting the occurrence of each word in each document of the document set. We will use the Counter method from the Python collections library for this purpose.
Counter counts the occurrence of each item in the list and returns a dictionary with the key as the item being counted and the corresponding value being the count of that item in the list.

Instructions:
Using the Counter() method and preprocessed_documents as the input, create a dictionary with the keys being each word in each document and the corresponding values being the frequncy of occurrence of that word. Save each Counter dictionary as an item in a list called 'frequency_list'.
Can you write Python code for it?

'''
Solution
'''
frequency_list = []
import pprint
from collections import Counter

for i in preprocessed_documents:
    frequency_counts = Counter(i)
    frequency_list.append(frequency_counts)
pprint.pprint(frequency_list)


I am working on the file "Project-practice--naive_bayes_tutorial/Naive_Bayes_tutorial.ipynb".
The first task is:
The get_feature_names() method returns our feature names for this dataset, which is the set of words that make up our vocabulary for 'documents'.

Instructions:
Create a matrix with the rows being each of the 4 documents, and the columns being each word.
The corresponding (row, column) value is the frequency of occurrance of that word(in the column) in a particular
document(in the row). You can do this using the transform() method and passing in the document data set as the
argument. The transform() method returns a matrix of numpy integers, you can convert this to an array using
toarray(). Call the array 'doc_array'
Can you write Python code for it?

'''
Solution
'''
doc_array = count_vector.transform(documents).toarray()
doc_array


I am working on the file "Project.ipynb".
The first task is:
3.
merging data sets
APPLE stock data was obtained using Quandl API at "https://www.quandl.com/api/v3/datasets/WIKI/AAPL.csv"
Can you write Python code for it?

path = "../datasets/"
aapl = pd.read_csv(path+'WIKI_PRICES_AAPL.csv')
fool = pd.read_csv(path+'motley_with_s_scores.csv')

def format_df(stock_df, news_df, word):
    """
    merges stock_df and news_df on "date"
    column

    input: stock df, news df, word
    output: merged df
    """

    stock_df['diff'] = stock_df['close']-stock_df['open']
    news_df['Count'] = news_df['article'].apply(lambda x: x.count(word))
    news_df.loc[news_df['Count'] <= 5, 'sentiment'] = 0
    news_df['date'] = pd.to_datetime(news_df['date'])
    news_df['date'] = news_df['date'].dt.strftime('%Y-%m-%d')
    news_df = news_df.groupby(['date'], as_index = False).sum()
    news_df = news_df[['date', 'sentiment', 'Count']]
    merged_df = pd.merge(news_df, stock_df)
    merged_df['bin_sentiment'] = pd.cut(merged_df['sentiment'], [-np.inf, -0.001, 0.001, np.inf], labels = [-1, 0, 1])
    merged_df['bin_diff'] = pd.cut(merged_df['diff'], [-np.inf, -0.001, 0.001, np.inf], labels = [-1, 0, 1])
    return merged_df

merged_df = format_df(aapl, fool, 'Apple')
merged_df.head()
#merged_df.to_csv('merged_df.csv', encoding='utf-8')


I am working on the file "Project.ipynb".
The first task is:
Also, to make the models work properly, from now on, we re-code loss category from -1 to 0.
Can you write Python code for it?

label = aapl['bin_diff'] == 1
label = label.astype(int)


I am working on the file "Project.ipynb".
The first task is:
Feed Forward Neural Network
Can you write Python code for it?

num_features = len(InputDF.columns)
dropout=0.2
hidden_1_size = 25
hidden_2_size = 5
num_classes = label.nunique()
NUM_EPOCHS=20
BATCH_SIZE=1
lr=0.0001
np.random.RandomState(52);

val = (InputDF[:-test_size].values, label[:-test_size].values)
train = (InputDF[-test_size:].values, label[-test_size:].values)
NUM_TRAIN_BATCHES = int(len(train[0])/BATCH_SIZE)
NUM_VAL_BATCHES = int(len(val[1])/BATCH_SIZE)

class Model():
    def __init__(self):
        global_step = tf.contrib.framework.get_or_create_global_step()
        self.input_data = tf.placeholder(dtype=tf.float32,shape=[None,num_features])
        self.target_data = tf.placeholder(dtype=tf.int32,shape=[None])
        self.dropout_prob = tf.placeholder(dtype=tf.float32,shape=[])
        with tf.variable_scope("ff"):
            droped_input = tf.nn.dropout(self.input_data,keep_prob=self.dropout_prob)

            layer_1 = tf.contrib.layers.fully_connected(
                num_outputs=hidden_1_size,
                inputs=droped_input,
            )
            layer_2 = tf.contrib.layers.fully_connected(
                num_outputs=hidden_2_size,
                inputs=layer_1,
            )
            self.logits = tf.contrib.layers.fully_connected(
                num_outputs=num_classes,
                activation_fn =None,
                inputs=layer_2,
            )
        with tf.variable_scope("loss"):

            self.losses = tf.nn.sparse_softmax_cross_entropy_with_logits(logits = self.logits,
                                                                         labels = self.target_data)
            mask = (1-tf.sign(1-self.target_data)) #Don't give credit for flat days
            mask = tf.cast(mask,tf.float32)
            self.loss = tf.reduce_sum(self.losses)

        with tf.name_scope("train"):
            opt = tf.train.AdamOptimizer(lr)
            gvs = opt.compute_gradients(self.loss)
            self.train_op = opt.apply_gradients(gvs, global_step=global_step)

        with...


I am working on the file "Project.ipynb".
The first task is:
Method Selection and Model Evaluation
This problem lends itself to learning algorithms such as SVMs, logistic regression, and random forest classification trees. However, the difficulty with stock price prediction is that the observations are highly dependent to each other. Therefore, even a little knowledge whether about the outcome stock behaviour might give an edge in investement.
Due to high correlation between stock prices of each day, generic learning algorithms usually perform very poorly on this type of problems and therefore we sought to use Deep Learning methods too. Particularly, we used basic 2-layer feedword neural network and recurrent neural networks with tensorflow.
The reason behind using recurrent neural networks (RNNs) is that they have loops. These loops allow the network to use information from previous passes, which acts as memory and can store the dependency relationship between past and present time.
<img src = "https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/image_folder_6/recurrent.jpg">
Note SVMs and logistic regression were optimized using penalty, random forest with out-of-bad errors, and neural nets with cross validation
Can you write Python code for it?

def plot_ROC(y_test, scores, label, color):
    """
    plots ROC curve
    input: y_test, scores, and title
    output: ROC curve
    """
    false_pr, true_pr, _ = roc_curve(y_test, scores[:, 1])
    roc_auc = auc(false_pr, true_pr)
    plt.plot(false_pr, true_pr, lw = 3,
             label='{}: area={:10.4f})'.format(label, roc_auc), color = color)
    plt.plot([0, 1], [0, 1], color='black', lw=1, linestyle='--')
    plt.xlabel('False positive rate')
    plt.ylabel('True positive rate')
    plt.legend(loc="best")
    plt.ylim([0.0, 1.05])
    plt.xlim([0.0, 1.0])
    plt.title('ROC')

def plot_PR(y_test, scores, label, color):
    """
    plots PR curve
    input: y_test, scores, title
    output: Precision-Recall curve
    """
    precision, recall, _ = precision_recall_curve(y_test, scores[:, 1])
    plt.plot(recall, precision,lw = 2,
             label='{}'.format(label), color = color)
    plt.xlabel('Recall')
    plt.ylabel('Precision')

    plt.legend(loc="best")
    plt.ylim([0.0, 1.05])
    plt.xlim([0.0, 1.0])
    plt.title('PR')

def plot_confusionmatrix(ytrue, ypred):
    """

    plots confusion matrix heatmap and prints out
    classification report

    input: ytrue (actual value), ypred(predicted value)
    output: confusion matrix heatmap and classification report

    """

    print (classification_report(ytrue, ypred))

    print ('##################################################################')

    cnf_matrix = confusion_matrix(ytrue, ypred)
    sns.heatmap(cnf_matrix, cmap='coolwarm_r', annot = True, linewidths=.5, fmt = '.4g')
    plt.title('Confusion matrix')
    plt.xlabel('Prediction')
    plt.ylabel('Actual');

apple = pd.read_csv(path + 'merged_df.csv')
apple.head()
print (apple.shape)

apple.plot('date', 'diff');


I am working on the file "Project.ipynb".
The first task is:
Random Forest Tree Classifiers
Can you write Python code for it?

oobscores = []
notrees = [10, 50, 100, 200, 500, 1000]
for n in notrees:
    rf = RandomForestClassifier(n_estimators=n, oob_score=True, random_state=42)
    rf_model = rf.fit(xtrain, ytrain)
    oobscores.append(rf_model.oob_score_)

plt.scatter(notrees, oobscores)
plt.xlabel('no of trees')
plt.ylabel('oob scores')
plt.title('random forest classifiers');


I am working on the file "Project.ipynb".
The first task is:
Recursive Neural Nets
Can you write Python code for it?

RNN_HIDDEN_SIZE=4
FIRST_LAYER_SIZE=50
SECOND_LAYER_SIZE=10
NUM_LAYERS=2
BATCH_SIZE=1
NUM_EPOCHS=25
lr=0.0003
NUM_TRAIN_BATCHES = int(len(train[0])/BATCH_SIZE)
NUM_VAL_BATCHES = int(len(val[1])/BATCH_SIZE)
ATTN_LENGTH=30
beta=0
np.random.RandomState(52);

class RNNModel():
    def __init__(self):
        global_step = tf.contrib.framework.get_or_create_global_step()
        self.input_data = tf.placeholder(dtype=tf.float32,shape=[BATCH_SIZE,num_features])
        self.target_data = tf.placeholder(dtype=tf.int32,shape=[BATCH_SIZE])
        self.dropout_prob = tf.placeholder(dtype=tf.float32,shape=[])

        def makeGRUCells():
            base_cell = rnn.GRUCell(num_units=RNN_HIDDEN_SIZE,)
            layered_cell = rnn.MultiRNNCell([base_cell] * NUM_LAYERS,state_is_tuple=False)
            attn_cell =tf.contrib.rnn.AttentionCellWrapper(cell=layered_cell,attn_length=ATTN_LENGTH,state_is_tuple=False)
            return attn_cell

        self.gru_cell = makeGRUCells()
        self.zero_state = self.gru_cell.zero_state(1, tf.float32)

        self.start_state = tf.placeholder(dtype=tf.float32,shape=[1,self.gru_cell.state_size])

        with tf.variable_scope("ff",initializer=xavier_initializer(uniform=False)):
            droped_input = tf.nn.dropout(self.input_data,keep_prob=self.dropout_prob)

            layer_1 = tf.contrib.layers.fully_connected(
                num_outputs=FIRST_LAYER_SIZE,
                inputs=droped_input,

            )
            layer_2 = tf.contrib.layers.fully_connected(
                num_outputs=RNN_HIDDEN_SIZE,
                inputs=layer_1,

            )

        split_inputs = tf.reshape(droped_input,shape=[1,BATCH_SIZE,num_features],name="reshape_l1") # Each item in the batch is a time step, iterate through them
        split_inputs = tf.unstack(split_inputs,axis=1,name="unpack_l1")
        states =[]
        outputs =[]
        with...


I am working on the file "Project.ipynb".
The first task is:
Results
As expected, SVMs, logistic regression, and random forest classifiers have worse/equal to performance than a coin flip based on ROC area calculated for the hold-out test set. The feed forward neural network has a slighly better performance with 0.53 ROC area while the RNN has the highest area of 0.56.
| Method | ROC area    |
| ------ | ----------- |
| Logistic Regression |  0.4373|
| SVM   | 0.4430|
| Random Forets Classifier  | 0.4735|
| Feed forward neural network | 0.5323|
| Random Forets Classifier   | 0.5557 |
The PR curves also look reasonable only for neural nets.
Logistic regression
Can you write Python code for it?

for pen in [1e-3, 1, 100]:
    logreg = LogisticRegression()
    logreg_model = logreg.fit(xtrain, ytrain)
    logpred = logreg_model.predict(xtest)
    logscores = logreg_model.predict_proba(xtest)
    print ("Accuracy score is {} for penalty = {}".format(accuracy_score(ytest, logpred), pen))
print ()
print ("confusion matrix for penalty = {}".format(pen))
plot_confusionmatrix(ytest, logpred)

plot_ROC(ytest, logscores, 'Logistic regression', 'r')

plot_PR(ytest, logscores, 'Logistic regression', 'b')


I am working on the file "Project.ipynb".
The first task is:
Support Vector Machines
Can you write Python code for it?

for pen in [1e-3, 1, 100]:
    svm = SVC(probability=True, C = pen)
    svm_model = svm.fit(xtrain, ytrain)
    svmpred = svm_model.predict(xtest)
    svmscores = svm_model.predict_proba(xtest)
    print ("Accuracy score is {} for penalty = {}".format(accuracy_score(ytest, svmpred), pen))
print ()
pen = 1
svm = SVC(probability=True, C = pen)
svm_model = svm.fit(xtrain, ytrain)
svmpred = svm_model.predict(xtest)
svmscores = svm_model.predict_proba(xtest)
print ("confusion matrix for penalty = {}".format(pen))
plot_confusionmatrix(ytest, svmpred)

print ("ROC curve for penalty = {}".format(pen))
plot_ROC(ytest, svmscores, 'SVM', 'r')

plot_PR(ytest, svmscores, 'SVM', 'b')


I am working on the file "Project.ipynb".
The first task is:
There is exterme fluctuation betweeen opening and closing prices of Apple, Inc. (as expected).
Let's choose the features and label (bin_diff) and make the dataframe ready for machine learning and deep learning.
Can you write Python code for it?

aapl = apple.copy()[['date', 'sentiment', 'bin_diff']]
aapl.head()

plt.hist(aapl['bin_diff']);


I am working on the file "Project.ipynb".
The first task is:
Training the RNN
Can you write Python code for it?

with tf.Graph().as_default():
    model = RNNModel()
    input_ = train[0]
    target = train[1]
    losses = []
    with tf.Session() as sess:
        init = tf.global_variables_initializer()
        sess.run([init])
        loss = 2000

        for e in range(NUM_EPOCHS):
            state = sess.run(model.zero_state)
            epoch_loss =0
            for batch in range(0,NUM_TRAIN_BATCHES):
                start = batch*BATCH_SIZE
                end = start + BATCH_SIZE
                feed = {
                    model.input_data:input_[start:end],
                    model.target_data:target[start:end],
                    model.dropout_prob:0.5,
                    model.start_state:state
                            }
                _,loss,acc,state = sess.run(
                    [
                        model.train_op,
                        model.loss,
                        model.accuracy,
                        model.end_state
                    ]
                    ,feed_dict=feed
                )
                epoch_loss+=loss
            losses.append(epoch_loss)
            #print('step - {0} loss - {1} acc - {2}'.format((e),epoch_loss,acc))
        print('################ done training ################')

        final_preds =np.array([])
        final_scores = None
        for batch in range(0,NUM_VAL_BATCHES):
                start = batch*BATCH_SIZE
                end = start + BATCH_SIZE
                feed = {
                    model.input_data:val[0][start:end],
                    model.target_data:val[1][start:end],
                    model.dropout_prob:1,
                    model.start_state:state
                            }
                acc,preds,state, probs = sess.run(
                    [
                        model.accuracy,
                        model.predictions,
                        model.end_state,
                        model.probs
                    ]
                    ,feed_dict=feed
                )
               ...


I am working on the file "Project.ipynb".
The first task is:
let's look at the features and standardize them.
Can you write Python code for it?

InputDF = aapl.copy().drop('bin_diff', axis = 1)
InputDF = InputDF.set_index('date')

InputDF.head()

InputDF = InputDF.apply(lambda x:(x -x.mean())/x.std())

InputDF.head()

test_size = 600
xtrain, xtest = InputDF.iloc[:test_size, :], InputDF.iloc[test_size:, :]
ytrain, ytest = label[:test_size], label[test_size:]


I am working on the file "Project/DCGAN.ipynb".
The first task is:
Connecting them together
Can you write Python code for it?

tf.reset_default_graph()

z_size = 100 #Size of z vector used for generator.

#This initializaer is used to initialize all the weights of the network.
initializer = tf.truncated_normal_initializer(stddev=0.02)

#These two placeholders are used for input into the generator and discriminator, respectively.
z_in = tf.placeholder(shape=[None,z_size],dtype=tf.float32) #Random vector
real_in = tf.placeholder(shape=[None,32,32,1],dtype=tf.float32) #Real images

Gz = generator(z_in) #Generates images from random z vectors
Dx = discriminator(real_in) #Produces probabilities for real images
Dg = discriminator(Gz,reuse=True) #Produces probabilities for generator images

#These functions together define the optimization objective of the GAN.
d_loss = -tf.reduce_mean(tf.log(Dx) + tf.log(1.-Dg)) #This optimizes the discriminator.
g_loss = -tf.reduce_mean(tf.log(Dg)) #This optimizes the generator.

tvars = tf.trainable_variables()

#The below code is responsible for applying gradient descent to update the GAN.
trainerD = tf.train.AdamOptimizer(learning_rate=0.0002,beta1=0.5)
trainerG = tf.train.AdamOptimizer(learning_rate=0.0002,beta1=0.5)
d_grads = trainerD.compute_gradients(d_loss,tvars[9:]) #Only update the weights for the discriminator network.
g_grads = trainerG.compute_gradients(g_loss,tvars[0:9]) #Only update the weights for the generator network.

update_D = trainerD.apply_gradients(d_grads)
update_G = trainerG.apply_gradients(g_grads)


I am working on the file "Project/DCGAN.ipynb".
The first task is:
Helper Functions
Can you write Python code for it?

#This function performns a leaky relu activation, which is needed for the discriminator network.
def lrelu(x, leak=0.2, name="lrelu"):
     with tf.variable_scope(name):
         f1 = 0.5 * (1 + leak)
         f2 = 0.5 * (1 - leak)
         return f1 * x + f2 * abs(x)

#The below functions are taken from carpdem20's implementation https://github.com/carpedm20/DCGAN-tensorflow
#They allow for saving sample images from the generator to follow progress
def save_images(images, size, image_path):
    return imsave(inverse_transform(images), size, image_path)

def imsave(images, size, path):
    return scipy.misc.imsave(path, merge(images, size))

def inverse_transform(images):
    return (images+1.)/2.

def merge(images, size):
    h, w = images.shape[1], images.shape[2]
    img = np.zeros((h * size[0], w * size[1]))

    for idx, image in enumerate(images):
        i = idx % size[1]
        j = idx // size[1]
        img[j*h:j*h+h, i*w:i*w+w] = image

    return img


I am working on the file "Project/DCGAN.ipynb".
The first task is:
Training the network
Now that we have fully defined our network, it is time to train it!
Can you write Python code for it?

batch_size = 128 #Size of image batch to apply at each iteration.
iterations = 500000 #Total number of iterations to use.
sample_directory = './figs' #Directory to save sample images from generator in.
model_directory = './models' #Directory to save trained model to.

init = tf.global_variables_initializer()
saver = tf.train.Saver()
with tf.Session() as sess:
    sess.run(init)
    for i in range(iterations):
        zs = np.random.uniform(-1.0,1.0,size=[batch_size,z_size]).astype(np.float32) #Generate a random z batch
        xs,_ = mnist.train.next_batch(batch_size) #Draw a sample batch from MNIST dataset.
        xs = (np.reshape(xs,[batch_size,28,28,1]) - 0.5) * 2.0 #Transform it to be between -1 and 1
        xs = np.lib.pad(xs, ((0,0),(2,2),(2,2),(0,0)),'constant', constant_values=(-1, -1)) #Pad the images so the are 32x32
        _,dLoss = sess.run([update_D,d_loss],feed_dict={z_in:zs,real_in:xs}) #Update the discriminator
        _,gLoss = sess.run([update_G,g_loss],feed_dict={z_in:zs}) #Update the generator, twice for good measure.
        _,gLoss = sess.run([update_G,g_loss],feed_dict={z_in:zs})
        if i % 10 == 0:
            print "Gen Loss: " + str(gLoss) + " Disc Loss: " + str(dLoss)
            z2 = np.random.uniform(-1.0,1.0,size=[batch_size,z_size]).astype(np.float32) #Generate another z batch
            newZ = sess.run(Gz,feed_dict={z_in:z2}) #Use new z to get sample images from generator.
            if not os.path.exists(sample_directory):
                os.makedirs(sample_directory)
            #Save sample generator images for viewing training progress.
            save_images(np.reshape(newZ[0:36],[36,32,32]),[6,6],sample_directory+'/fig'+str(i)+'.png')
        if i % 1000 == 0 and i != 0:
            if not os.path.exists(model_directory):
                os.makedirs(model_directory)
            saver.save(sess,model_directory+'/model-'+str(i)+'.cptk')
            print "Saved Model"


I am working on the file "Project/DCGAN.ipynb".
The first task is:
Using a trained network
Once we have a trained model saved, we may want to use it to generate new images, and explore the representation it has learned.
Can you write Python code for it?

sample_directory = './figs' #Directory to save sample images from generator in.
model_directory = './models' #Directory to load trained model from.
batch_size_sample = 36

init = tf.global_variables_initializer()
saver = tf.train.Saver()
with tf.Session() as sess:
    sess.run(init)
    #Reload the model.
    print 'Loading Model...'
    ckpt = tf.train.get_checkpoint_state(path)
    saver.restore(sess,ckpt.model_checkpoint_path)

    zs = np.random.uniform(-1.0,1.0,size=[batch_size_sample,z_size]).astype(np.float32) #Generate a random z batch
    newZ = sess.run(Gz,feed_dict={z_in:z2}) #Use new z to get sample images from generator.
    if not os.path.exists(sample_directory):
        os.makedirs(sample_directory)
    save_images(np.reshape(newZ[0:batch_size_sample],[36,32,32]),[6,6],sample_directory+'/fig'+str(i)+'.png')


I am working on the file "Project/DCGAN.ipynb".
The first task is:
We will be using the MNIST dataset. input_data is a library that downloads the dataset and uzips it automatically.
Can you write Python code for it?

mnist = input_data.read_data_sets("MNIST_data/", one_hot=False)


I am working on the file "Project/Deep Layer Visualization.ipynb".
The first task is:
...and the third.
Can you write Python code for it?


I am working on the file "Project/Deep Layer Visualization.ipynb".
The first task is:
Next we define our convolutional network. It will be a network with three sets of convolution -> pooling layers, followed by a fully connected softmax layer. I have choosen 5,5,20 to begin with. Feel free to adjust the number of convolutional filters at each layer. It is these filters we will be visualizing, so we can see in realtime what features are learned from the dataset with more or less filters.
Can you write Python code for it?

tf.reset_default_graph()

x = tf.placeholder(tf.float32, [None, 784],name="x-in")
true_y = tf.placeholder(tf.float32, [None, 10],name="y-in")
keep_prob = tf.placeholder("float")

x_image = tf.reshape(x,[-1,28,28,1])
hidden_1 = slim.conv2d(x_image,5,[5,5])
pool_1 = slim.max_pool2d(hidden_1,[2,2])
hidden_2 = slim.conv2d(pool_1,5,[5,5])
pool_2 = slim.max_pool2d(hidden_2,[2,2])
hidden_3 = slim.conv2d(pool_2,20,[5,5])
hidden_3 = slim.dropout(hidden_3,keep_prob)
out_y = slim.fully_connected(slim.flatten(hidden_3),10,activation_fn=tf.nn.softmax)

cross_entropy = -tf.reduce_sum(true_y*tf.log(out_y))
correct_prediction = tf.equal(tf.argmax(out_y,1), tf.argmax(true_y,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)


I am working on the file "Project/Deep Layer Visualization.ipynb".
The first task is:
Next we import the MNIST data files we are going to be classifying. This database contains images of thousands of handwritten digits, and their proper labels. For convenience I am using a script from Google, which can be download here. Just add it to your working directory, and it will download the MNIST database for you.
Can you write Python code for it?

mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)


I am working on the file "Project/Deep Layer Visualization.ipynb".
The first task is:
Now we define a couple functions that will allow us to visualize the network. The first gets the activations at a given layer for a given input image. The second plots those activations in a grid.
Can you write Python code for it?

def getActivations(layer,stimuli):
    units = sess.run(layer,feed_dict={x:np.reshape(stimuli,[1,784],order='F'),keep_prob:1.0})
    plotNNFilter(units)

def plotNNFilter(units):
    filters = units.shape[3]
    plt.figure(1, figsize=(20,20))
    n_columns = 6
    n_rows = math.ceil(filters / n_columns) + 1
    for i in range(filters):
        plt.subplot(n_rows, n_columns, i+1)
        plt.title('Filter ' + str(i))
        plt.imshow(units[0,:,:,i], interpolation="nearest", cmap="gray")


I am working on the file "Project/Deep Layer Visualization.ipynb".
The first task is:
We can do this again for the second layer...
Can you write Python code for it?


I am working on the file "Project/Deep Layer Visualization.ipynb".
The first task is:
We then train the network using batch gradient descent with Adam optimization. Feel free to adjust the batch size and number of iterations to see how it effects the model accuracy.
Can you write Python code for it?

batchSize = 50
sess = tf.Session()
init = tf.global_variables_initializer()
sess.run(init)
for i in range(1001):
    batch = mnist.train.next_batch(batchSize)
    sess.run(train_step, feed_dict={x:batch[0],true_y:batch[1], keep_prob:0.5})
    if i % 100 == 0 and i != 0:
        trainAccuracy = sess.run(accuracy, feed_dict={x:batch[0],true_y:batch[1], keep_prob:1.0})
        print("step %d, training accuracy %g"%(i, trainAccuracy))

testAccuracy = sess.run(accuracy, feed_dict={x:mnist.test.images,true_y:mnist.test.labels, keep_prob:1.0})
print("test accuracy %g"%(testAccuracy))


I am working on the file "Project/Deep Network Comparison.ipynb".
The first task is:
HighwayNet
An implementation of a Highway Network as desribed in Highway Networks.
Can you write Python code for it?

def highwayUnit(input_layer,i):
    with tf.variable_scope("highway_unit"+str(i)):
        H = slim.conv2d(input_layer,64,[3,3])
        T = slim.conv2d(input_layer,64,[3,3], #We initialize with a negative bias to push the network to use the skip connection
            biases_initializer=tf.constant_initializer(-1.0),activation_fn=tf.nn.sigmoid)
        output = H*T + input_layer*(1.0-T)
        return output

tf.reset_default_graph()

input_layer = tf.placeholder(shape=[None,32,32,3],dtype=tf.float32,name='input')
label_layer = tf.placeholder(shape=[None],dtype=tf.int32)
label_oh = slim.layers.one_hot_encoding(label_layer,10)

layer1 = slim.conv2d(input_layer,64,[3,3],normalizer_fn=slim.batch_norm,scope='conv_'+str(0))
for i in range(5):
    for j in range(units_between_stride):
        layer1 = highwayUnit(layer1,j + (i*units_between_stride))
    layer1 = slim.conv2d(layer1,64,[3,3],stride=[2,2],normalizer_fn=slim.batch_norm,scope='conv_s_'+str(i))

top = slim.conv2d(layer1,10,[3,3],normalizer_fn=slim.batch_norm,activation_fn=None,scope='conv_top')

output = slim.layers.softmax(slim.layers.flatten(top))

loss = tf.reduce_mean(-tf.reduce_sum(label_oh * tf.log(output) + 1e-10, axis=[1]))
trainer = tf.train.AdamOptimizer(learning_rate=0.001)
update = trainer.minimize(loss)


I am working on the file "Project/Deep Network Comparison.ipynb".
The first task is:
Load CIFAR Dataset
To obtain the CIFAR10 dataset, go here: https://www.cs.toronto.edu/~kriz/cifar.html
The training data is stored in 5 separate files, and we will alternate between them during training.
Can you write Python code for it?

def unpickle(file):
    import cPickle
    fo = open(file, 'rb')
    dict = cPickle.load(fo)
    fo.close()
    return dict

currentCifar = 1
cifar = unpickle('./cifar10/data_batch_1')
cifarT = unpickle('./cifar10/test_batch')

total_layers = 25 #Specify how deep we want our network
units_between_stride = total_layers / 5


I am working on the file "Project/Deep Network Comparison.ipynb".
The first task is:
RegularNet
A Deep Neural Network composed exclusively of regular and strided convolutional layers. While this architecture works well for relatively shallow networks, it becomes increasingly more difficult to train as the network depth increases.
Can you write Python code for it?

tf.reset_default_graph()

input_layer = tf.placeholder(shape=[None,32,32,3],dtype=tf.float32,name='input')
label_layer = tf.placeholder(shape=[None],dtype=tf.int32)
label_oh = slim.layers.one_hot_encoding(label_layer,10)

layer1 = slim.conv2d(input_layer,64,[3,3],normalizer_fn=slim.batch_norm,scope='conv_'+str(0))
for i in range(5):
    for j in range(units_between_stride):
        layer1 = slim.conv2d(layer1,64,[3,3],normalizer_fn=slim.batch_norm,scope='conv_'+str((j+1) + (i*units_between_stride)))
    layer1 = slim.conv2d(layer1,64,[3,3],stride=[2,2],normalizer_fn=slim.batch_norm,scope='conv_s_'+str(i))

top = slim.conv2d(layer1,10,[3,3],normalizer_fn=slim.batch_norm,activation_fn=None,scope='conv_top')

output = slim.layers.softmax(slim.layers.flatten(top))

loss = tf.reduce_mean(-tf.reduce_sum(label_oh * tf.log(output) + 1e-10, axis=[1]))
trainer = tf.train.AdamOptimizer(learning_rate=0.001)
update = trainer.minimize(loss)


I am working on the file "Project/Deep Network Comparison.ipynb".
The first task is:
ResNet
An implementation of a Residual Network as described in Identity Mappings in Deep Residual Networks.
Can you write Python code for it?

def resUnit(input_layer,i):
    with tf.variable_scope("res_unit"+str(i)):
        part1 = slim.batch_norm(input_layer,activation_fn=None)
        part2 = tf.nn.relu(part1)
        part3 = slim.conv2d(part2,64,[3,3],activation_fn=None)
        part4 = slim.batch_norm(part3,activation_fn=None)
        part5 = tf.nn.relu(part4)
        part6 = slim.conv2d(part5,64,[3,3],activation_fn=None)
        output = input_layer + part6
        return output

tf.reset_default_graph()

input_layer = tf.placeholder(shape=[None,32,32,3],dtype=tf.float32,name='input')
label_layer = tf.placeholder(shape=[None],dtype=tf.int32)
label_oh = slim.layers.one_hot_encoding(label_layer,10)

layer1 = slim.conv2d(input_layer,64,[3,3],normalizer_fn=slim.batch_norm,scope='conv_'+str(0))
for i in range(5):
    for j in range(units_between_stride):
        layer1 = resUnit(layer1,j + (i*units_between_stride))
    layer1 = slim.conv2d(layer1,64,[3,3],stride=[2,2],normalizer_fn=slim.batch_norm,scope='conv_s_'+str(i))

top = slim.conv2d(layer1,10,[3,3],normalizer_fn=slim.batch_norm,activation_fn=None,scope='conv_top')

output = slim.layers.softmax(slim.layers.flatten(top))

loss = tf.reduce_mean(-tf.reduce_sum(label_oh * tf.log(output) + 1e-10, axis=[1]))
trainer = tf.train.AdamOptimizer(learning_rate=0.001)
update = trainer.minimize(loss)


I am working on the file "Project/Deep Network Comparison.ipynb".
The first task is:
Results
Can you write Python code for it?

plt.plot(l) #Plot training loss

plt.plot(a) #Plot training accuracy

plt.plot(aT) #Plot test accuracy

np.max(aT) #Best test accuracy


I am working on the file "Project/Deep Network Comparison.ipynb".
The first task is:
Training
Can you write Python code for it?

init = tf.global_variables_initializer()
batch_size = 64
currentCifar = 1
total_steps = 20000
l = []
a = []
aT = []
with tf.Session() as sess:
    sess.run(init)
    i = 0
    draw = range(10000)
    while i < total_steps:
        if i % (10000/batch_size) != 0:
            batch_index = np.random.choice(draw,size=batch_size,replace=False)
        else:
            draw = range(10000)
            if currentCifar == 5:
                currentCifar = 1
                print "Switched CIFAR set to " + str(currentCifar)
            else:
                currentCifar = currentCifar + 1
                print "Switched CIFAR set to " + str(currentCifar)
            cifar = unpickle('./cifar10/data_batch_'+str(currentCifar))
            batch_index = np.random.choice(draw,size=batch_size,replace=False)
        x = cifar['data'][batch_index]
        x = np.reshape(x,[batch_size,32,32,3],order='F')
        x = (x/256.0)
        x = (x - np.mean(x,axis=0)) / np.std(x,axis=0)
        y = np.reshape(np.array(cifar['labels'])[batch_index],[batch_size,1])
        _,lossA,yP,LO = sess.run([update,loss,output,label_oh],feed_dict={input_layer:x,label_layer:np.hstack(y)})
        accuracy = np.sum(np.equal(np.hstack(y),np.argmax(yP,1)))/float(len(y))
        l.append(lossA)
        a.append(accuracy)
        if i % 10 == 0: print "Step: " + str(i) + " Loss: " + str(lossA) + " Accuracy: " + str(accuracy)
        if i % 100 == 0:
            point = np.random.randint(0,10000-500)
            xT = cifarT['data'][point:point+500]
            xT = np.reshape(xT,[500,32,32,3],order='F')
            xT = (xT/256.0)
            xT = (xT - np.mean(xT,axis=0)) / np.std(xT,axis=0)
            yT = np.reshape(np.array(cifarT['labels'])[point:point+500],[500])
            lossT,yP = sess.run([loss,output],feed_dict={input_layer:xT,label_layer:yT})
            accuracy = np.sum(np.equal(yT,np.argmax(yP,1)))/float(len(yT))
            aT.append(accuracy)
            print "Test set accuracy: " + str(accuracy)
        i+= 1


I am working on the file "Project/Deep Network Comparison.ipynb".
The first task is:
Visualize the network graph
We can call the Tensorflow Board to provide a graphical representation of our network.
Can you write Python code for it?

from IPython.display import clear_output, Image, display, HTML

def strip_consts(graph_def, max_const_size=32):
    """Strip large constant values from graph_def."""
    strip_def = tf.GraphDef()
    for n0 in graph_def.node:
        n = strip_def.node.add()
        n.MergeFrom(n0)
        if n.op == 'Const':
            tensor = n.attr['value'].tensor
            size = len(tensor.tensor_content)
            if size > max_const_size:
                tensor.tensor_content = "<stripped %d bytes>"%size
    return strip_def

def show_graph(graph_def, max_const_size=32):
    """Visualize TensorFlow graph."""
    if hasattr(graph_def, 'as_graph_def'):
        graph_def = graph_def.as_graph_def()
    strip_def = strip_consts(graph_def, max_const_size=max_const_size)
    code = """

        <link rel="import" href="https://tensorboard.appspot.com/tf-graph-basic.build.html" onload=load()>

          <tf-graph-basic id="{id}"></tf-graph-basic>

    """.format(data=repr(str(strip_def)), id='graph'+str(np.random.rand()))

    iframe = """

    """.format(code.replace('"', '"'))
    display(HTML(iframe))

show_graph(tf.get_default_graph().as_graph_def())


I am working on the file "Project/InfoGAN-Tutorial.ipynb".
The first task is:
Helper Functions
Can you write Python code for it?


I am working on the file "Project/RNN-TF.ipynb".
The first task is:
Defining the network
The first thing we do is create placeholder variables for the x inputs, expected y outputs, and starting value of the hidden layer.
Can you write Python code for it?

x = tf.placeholder(tf.float32, [None,vocab_size])
y_in = tf.placeholder(tf.float32, [None,vocab_size])
hStart = tf.placeholder(tf.float32,[1,hidden_size])


I am working on the file "Project/RNN-TF.ipynb".
The first task is:
Next we define the architecture of the network itself. To do this, we begin by intitializing the hidden state (hState) to the starting hidden state (hStart). We then create an empty list (y_outAll). This will serve to collect each of the y values choosen by the network. The for-loop allows for the recurrent part of the recurrent neural network. This will repeat for the size of seq_length we defined above. Within the loop we initialize our weights and bias variables. Then we compute the new hidden state from the x input, and also the previous hidden state. It is this additional matrix multiply of the previous hidden state that allows each new pass through the network to use information from the past. Tanh allows us to get non-linearity from the hidden state. Finally we multiply the hidden state by the top-layer weights and add the bias to get an output y, which will be collected in y_outAll.
Can you write Python code for it?

initializer = tf.random_normal_initializer(stddev=0.1)
with tf.variable_scope("RNN") as scope:
    hState = hStart
    y_outAll = []
    for t,xIn in enumerate(tf.split(0,seq_length,x)):
        if t > 0: scope.reuse_variables()
        Wxh = tf.get_variable("Wxh", [vocab_size,hidden_size], initializer=initializer)
        Whh = tf.get_variable("Whh", [hidden_size,hidden_size], initializer=initializer)
        Why = tf.get_variable("Why",[hidden_size,vocab_size], initializer=initializer)
        bh = tf.get_variable("bh", [hidden_size],initializer=initializer)
        by = tf.get_variable("by",[vocab_size], initializer=initializer)

        hState = tf.tanh(tf.matmul(xIn,Wxh) + tf.matmul(hState,Whh) + bh)
        y_out = tf.matmul(hState,Why) + by
        y_outAll.append(y_out)


I am working on the file "Project/RNN-TF.ipynb".
The first task is:
Next we set the starting position in the text and the number of iterations to be 0. We also initialize the value of the hidden layer to be zero, as well as choose how many iterations we'd like to train the network for. For large documents, the longer the better.
Can you write Python code for it?

positionInText = 0
numberOfIterations = 0
totalIterations = 1000
hStart_val = np.zeros([1,hidden_size])


I am working on the file "Project/RNN-TF.ipynb".
The first task is:
One all the input xs have been run through the network, we save the last hidden state of the network to use again. We then take all the y values we collected and concatenate them into a single output variable. Using this variable and the true y values (y_in) we compute cross-entropy loss.
Can you write Python code for it?

hLast = hState
output_softmax = tf.nn.softmax(y_outAll[-1])
outputs = tf.concat(0,y_outAll)
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(outputs,y_in))


I am working on the file "Project/RNN-TF.ipynb".
The first task is:
Reading and processing input text
We start by importing a text document that we will use to train the recurrent neural network. This can be any body of text, as long as it is plaintext. I recommend Project Gutenberg as a way to get public domain texts to train on. After importing the text, we then get a list of all the unique characters in the document. For an english document, this will be the alphabet, along with punctuation and linebreaks. We will need to be able to get the index of a given character, and get a given character with an index, and the last two lines here do that.
Can you write Python code for it?

text = open('text.txt', 'r').read() # should be simple plain text file
uniqueChars = list(set(text))
text_size, vocab_size = len(text), len(uniqueChars)
print 'data has %d characters, %d unique.' % (text_size, vocab_size)
char_to_ix = { ch:i for i,ch in enumerate(uniqueChars) }
ix_to_char = { i:ch for i,ch in enumerate(uniqueChars) }


I am working on the file "Project/RNN-TF.ipynb".
The first task is:
Running the recurrent neural network
Now that we have defined the network, it is time to train it! To do this we start by defining a Tensorflow session, and then initializing the network we defined above, and starting it.
Can you write Python code for it?

sess = tf.Session()
init = tf.initialize_all_variables()
sess.run(init)


I am working on the file "Project/RNN-TF.ipynb".
The first task is:
Sampling from the network
We are going to train the network below, but first we'd like to define a function that we can call that will give us an intiutivie sense of how well the network is doing. The code below allows us to generate a string of text from the network to see if it is indeed producing something resembling the input text.
Can you write Python code for it?

def sampleNetwork():
    sample_length = 200
    start_ix      = random.randint(0, len(text) - seq_length)
    sample_seq_ix = [char_to_ix[ch] for ch in text[start_ix:start_ix + seq_length]]
    ixes          = []
    sample_prev_state_val = np.copy(hStart_val)

    for t in range(sample_length):
        sample_input_vals = one_hot(sample_seq_ix)
        sample_output_softmax_val, sample_prev_state_val = \
        sess.run([output_softmax, hLast], feed_dict={x: sample_input_vals, hStart: sample_prev_state_val})

        ix = np.random.choice(range(vocab_size), p=sample_output_softmax_val.ravel())
        ixes.append(ix)
        sample_seq_ix = sample_seq_ix[1:] + [ix]

    txt = ''.join(ix_to_char[ix] for ix in ixes)
    print('----\n %s \n----\n' % (txt,))


I am working on the file "Project/RNN-TF.ipynb".
The first task is:
This function allows us to convert the character labels to one-hot variants
Can you write Python code for it?

def one_hot(v):
    return np.eye(vocab_size)[v]


I am working on the file "Project/RNN-TF.ipynb".
The first task is:
Within the main training loop, we first check to see if we have reached the end of the document, or are just beginning the document. If so, we reset the hidden layer, and start at the beginning of the document again. Then we draw from the document what will be our input and target characters. The first character in input should be followed by the first character in target in the document. So if it is the word "cat, " input 1 would be c, and target 1 would be a.
We then send these through our network, along with our starting hidden layer state. The network will return the final hidden state, which we will save to use for the next iteration of training. It will also give us a loss value, which we can display periodically to ensure the model is indeed improving. Every 500 iterations, we can check the loss, and sample some output from the network to see how well it is doing.
Can you write Python code for it?

while numberOfIterations < totalIterations:
    if positionInText+seq_length+1 >= len(text) or numberOfIterations == 0:
        hStart_val = np.zeros([1,hidden_size])
        positionInText = 0

    inputs = one_hot([char_to_ix[ch] for ch in text[positionInText:positionInText+seq_length]])
    targets = one_hot([char_to_ix[ch] for ch in text[positionInText+1:positionInText+seq_length+1]])

    hStart_val, loss_val, _ = sess.run([hLast,loss,updates],feed_dict={x:inputs, y_in:targets,hStart:hStart_val})

    if numberOfIterations % 500 == 0:
        print('iter: %d, p: %d, loss: %f' % (numberOfIterations, positionInText, loss_val))
        sampleNetwork()

    positionInText += seq_length
    numberOfIterations +=1


I am working on the file "Project/img_similarity_training.ipynb".
The first task is:
Evaluating the network statically

Functions for computing representation of the image at different levels of encoding/decoding
get_outs  returns the output of the network
able to evaluate on many images
no need to run the simulator
Can you write Python code for it?

#Get the neuron activity of an image or group of images (this is the semantic pointer in this case)
def get_activities(sim, images):
    _, acts = nengo.utils.ensemble.tuning_curves(a, sim, inputs=images)
    return acts

#Get similarity of activity using dot product
def get_dots(imgs):
    dots = np.ndarray((60000,1))
    for i in range(len(imgs)):
        dots[i] = np.dot(imgs[i][:1000],imgs[i][1000:])

    return dots


I am working on the file "Project/img_similarity_training.ipynb".
The first task is:
Load the MNIST training and testing images
Can you write Python code for it?

# --- load the data
img_rows, img_cols = 28, 28

(X_train, y_train), (X_test, y_test) = load_mnist()

X_train = 2 * X_train - 1  # normalize to -1 to 1
X_test = 2 * X_test - 1  # normalize to -1 to 1


I am working on the file "Project/img_similarity_training.ipynb".
The first task is:
The Network

The network parameters must be the same here as when the weight matrices are used later on
The network is made up of an ensemble and a node
The connection (to v) computes the weights from the activities of the images to their similarites
Network is the same as was used for training rotation so that it can be used later on.
Can you write Python code for it?

rng = np.random.RandomState(9)

# --- set up network parameters
#Want to map from images to similarity
n_vis = X_train.shape[1] #imgs.shape[1]
n_out =  similarities.shape[1]
#number of neurons/dimensions of semantic pointer
n_hid = 1000 #Try with more neurons for more accuracy

#Want the encoding/decoding done on the training images
ens_params = dict(
    eval_points=X_train, #imgs,
    neuron_type=nengo.LIF(), #originally used LIFRate()
    intercepts=nengo.dists.Choice([-0.5]),
    max_rates=nengo.dists.Choice([100]),
    )

#Least-squares solver with L2 regularization.
solver = nengo.solvers.LstsqL2(reg=0.01)
#solver = nengo.solvers.LstsqL2(reg=0.0001)

#network that generates the weight matrices between neuron activity and images and the labels
with nengo.Network(seed=3) as model:
    a = nengo.Ensemble(n_hid, n_vis, seed=3, **ens_params)
    v = nengo.Node(size_in=n_out)
    conn = nengo.Connection(
        a, v, synapse=None,
        eval_points=imgs, function=similarities,#want the similarities out
        solver=solver)

# linear filter used for edge detection as encoders, more plausible for human visual system
encoders = Gabor().generate(n_hid, (11, 11), rng=rng)
encoders = Mask((28, 28)).populate(encoders, rng=rng, flatten=True)
#Set the ensembles encoders to this
a.encoders = encoders

#Check the encoders were correctly made
plt.imshow(encoders[0].reshape(28, 28), vmin=encoders[0].min(), vmax=encoders[0].max(), cmap='gray')


I am working on the file "Project/low_pass_training.ipynb".
The first task is:
Load the MNIST training and testing images
Can you write Python code for it?

# --- load the data
img_rows, img_cols = 28, 28

(X_train, y_train), (X_test, y_test) = load_mnist()

X_train = 2 * X_train - 1  # normalize to -1 to 1
X_test = 2 * X_test - 1  # normalize to -1 to 1

X_train.shape

noise = np.random.random([dim,dim])
#noise = 2* noise - 1 # normalize to -1 to 1
noise = noise+np.reshape(X_train[1],(28,28))
plt.subplot(121)
plt.imshow(noise,cmap="gray")

low_pass = noise.copy()
low_pass[low_pass<0]=-1
low_pass[low_pass>0]=1

plt.subplot(122)
plt.imshow(low_pass,cmap="gray")

def intense(img):
    newImg = img.copy()
    newImg[newImg < 0] = -1
    newImg[newImg > 0] = 1
    return newImg

i=random.randint(1,100)
img = X_train[i]
img_filtered = intense(scipy.ndimage.gaussian_filter(img, sigma=1))
rotated = scipy.ndimage.rotate(img_filtered.reshape(dim,dim),6,reshape=False,cval=-1)
rotated_filtered = intense(scipy.ndimage.gaussian_filter(rotated, sigma=1))

a = np.random.random([dim,dim])
img_filtered = img_filtered+a.ravel()

plt.subplot(141)
plt.title("mnist")
plt.imshow(np.reshape(img,(dim,dim)),cmap="gray")
plt.subplot(142)
plt.title("filtered & noise")
plt.imshow(np.reshape(img_filtered,(dim,dim)),cmap="gray")
plt.subplot(143)
plt.title("rotated")
plt.imshow(np.reshape(rotated,(dim,dim)),cmap="gray")
plt.subplot(144)
plt.title("rotated filtered")
plt.imshow(np.reshape(rotated_filtered,(dim,dim)),cmap="gray")

img = np.random.random([dim,dim])

img = img.ravel()
i=random.randint(1,100)
img = img+X_train[i]

plt.subplot(121)
plt.imshow(np.reshape(img,(dim,dim)),cmap="gray")
plt.subplot(122)
plt.imshow(np.reshape(intense(img),(dim,dim)),cmap="gray")

#Create set of noisy images
noise_train = np.random.random(X_train.shape)
noise_train = 2 * noise_train -1# normalize to -1 to 1
#Training with mnist
#noise_train = noise_train + X_train

#Clean up noisy images with intensifying
clean_train = intense(noise_train)

plt.subplot(121)
plt.imshow(np.reshape(noise_train[1],(dim,dim)),cmap="gray")
plt.subplot(122)
plt.imshow(np.reshape(clean_train[1],(dim,dim)),cmap="gray")


I am working on the file "Project/low_pass_training.ipynb".
The first task is:
Saving weight matrices
Can you write Python code for it?

filename = "low_pass_weights_mnist" + str(n_hid) +".p"
cPickle.dump(weights, open( filename, "wb" ) )


I am working on the file "Project/low_pass_training.ipynb".
The first task is:
Simulator

Calculate the neuron activities of each set of images
Generate the weight matrices between original activities and clean activities
Can you write Python code for it?

with nengo.Simulator(model) as sim:

    #Neuron activities
    noise_acts = get_activities(sim,noise_train)
    clean = get_encoder_outputs(sim,clean_train)

    #solvers for a learning rule
    solver_low_pass = nengo.solvers.LstsqL2(reg=1e-8)

    #find weight matrix between neuron activity of the original image and the clean img
    #weights returns a tuple including information about learning process, just want the weight matrix
    weights,_ = solver_low_pass(noise_acts, clean)


I am working on the file "Project/manipulation_combination_training.ipynb".
The first task is:
Evaluating the network statically

Functions for computing representation of the image at different levels of encoding/decoding
get_outs  returns the output of the network
able to evaluate on many images
no need to run the simulator
Can you write Python code for it?

#Get the one hot labels for the images
def get_outs(sim, images):
    #The activity of the neurons when an image is given as input
    _, acts = nengo.utils.ensemble.tuning_curves(a, sim, inputs=images)
    #The activity multiplied by the weight matrix (calculated in the network) to give the one-hot labels
    return np.dot(acts, sim.data[conn2].weights.T)

#Check how many of the labels were produced correctly
#def get_error(sim, images, labels):
#    return np.argmax(get_outs(sim, images), axis=1) != labels

#Get label of the images
#def get_labels(sim,images):
#    return np.argmax(get_outs(sim, images), axis=1)

#Get the neuron activity of an image or group of images (this is the semantic pointer in this case)
def get_activities(sim, images):
    _, acts = nengo.utils.ensemble.tuning_curves(a, sim, inputs=images)
    return acts

#Get the representation of the image after it has gone through the encoders (Gabor filters) but before it is in the neurons
#This must be computed to create the weight matrix for rotation from neuron activity to this step
# This allows a recurrent connection to be made from the neurons to themselves later
def get_encoder_outputs(sim,images):
    #Pass the images through the encoders
    outs = np.dot(images,sim.data[a].encoders.T) #before the neurons
    return outs


I am working on the file "Project/manipulation_combination_training.ipynb".
The first task is:
Load the MNIST training and testing images
Can you write Python code for it?

# --- load the data
img_rows, img_cols = 28, 28

(X_train, y_train), (X_test, y_test) = load_mnist()

X_train = 2 * X_train - 1  # normalize to -1 to 1
X_test = 2 * X_test - 1  # normalize to -1 to 1

train_targets = one_hot(y_train, 10)
test_targets = one_hot(y_test, 10)


I am working on the file "Project/manipulation_combination_training.ipynb".
The first task is:
Represent each number using a one-hot where the index of the one represents the digit value
Can you write Python code for it?

#Encode categorical integer features using a one-hot aka one-of-K scheme.
def one_hot(labels, c=None):
    assert labels.ndim == 1
    n = labels.shape[0]
    c = len(np.unique(labels)) if c is None else c
    y = np.zeros((n, c))
    y[np.arange(n), labels] = 1
    return y


I am working on the file "Project/manipulation_combination_training.ipynb".
The first task is:
The Network

The network parameters must be the same here as when the weight matrices are used later on
The network is made up of an ensemble and two nodes
The first connection ( to v) computes the weights from the activities of the images to the images themselves
The second connection (to v2) computes the weights from the activities of the images to the labels
Can you write Python code for it?

rng = np.random.RandomState(9)

# --- set up network parameters
#Want to encode and decode the image
n_vis = X_train.shape[1]
n_out =  X_train.shape[1]
#number of neurons/dimensions of semantic pointer
n_hid = 1000 #Try with more neurons for more accuracy

#Want the encoding/decoding done on the training images
ens_params = dict(
    eval_points=X_train,
    neuron_type=nengo.LIF(), #Why not use LIF? originally used LIFRate()
    intercepts=nengo.dists.Choice([-0.5]),
    max_rates=nengo.dists.Choice([100]),
    )

#Least-squares solver with L2 regularization.
solver = nengo.solvers.LstsqL2(reg=0.01)
#solver = nengo.solvers.LstsqL2(reg=0.0001)
solver2 = nengo.solvers.LstsqL2(reg=0.01)

#network that generates the weight matrices between neuron activity and images and the labels
with nengo.Network(seed=3) as model:
    a = nengo.Ensemble(n_hid, n_vis, seed=3, **ens_params)
    v = nengo.Node(size_in=n_out)
    conn = nengo.Connection(
        a, v, synapse=None,
        eval_points=X_train, function=X_train,#want the same thing out (identity)
        solver=solver)

    v2 = nengo.Node(size_in=train_targets.shape[1])
    conn2 = nengo.Connection(
        a, v2, synapse=None,
        eval_points=X_train, function=train_targets, #Want to get the labels out
        solver=solver2)

# linear filter used for edge detection as encoders, more plausible for human visual system
encoders = Gabor().generate(n_hid, (11, 11), rng=rng)
encoders = Mask((28, 28)).populate(encoders, rng=rng, flatten=True)
#Set the ensembles encoders to this
a.encoders = encoders

#Check the encoders were correctly made
plt.imshow(encoders[0].reshape(28, 28), vmin=encoders[0].min(), vmax=encoders[0].max(), cmap='gray')


I am working on the file "Project/mental_rotation_training.ipynb".
The first task is:
Images
Create lists of training and testing images
- Original images at random orientations
- Those images rotated a fixed amount more
- Images not used for training, but later for testing
Can you write Python code for it?

'''
def intense(img):
    newImg = img.copy()
    newImg[newImg < 0] = -1
    newImg[newImg > 0] = 1
    return newImg

def filtered(img):
    return intense(scipy.ndimage.gaussian_filter(img, sigma=1))
'''

degrees = 6

#Images to train, starting at random orientation
orig_imgs = X_train[:100000].copy()
for img in orig_imgs:
    img[:] = scipy.ndimage.interpolation.rotate(np.reshape(img,(28,28)),
                                                (np.random.randint(360)),reshape=False,mode="nearest").ravel()

#Images rotated a fixed amount from the original random orientation
rotated_imgs =orig_imgs.copy()
for img in rotated_imgs:
    img[:] = scipy.ndimage.interpolation.rotate(np.reshape(img,(28,28)),degrees,reshape=False,mode="nearest").ravel()

    #^get the encoder outputs of this^

#Images not used for training, but for testing (all at random orientations)
test_imgs = X_test[:1000].copy()
for img in test_imgs:
    img[:] = scipy.ndimage.interpolation.rotate(np.reshape(img,(28,28)),
                                                (np.random.randint(360)),reshape=False,mode="nearest").ravel()

#Check to make sure images were generated correctly
i= np.random.randint(0,1000)
plt.subplot(121)
plt.imshow(np.reshape(orig_imgs[i],(28,28)), cmap='gray')
plt.subplot(122)
plt.imshow(np.reshape(rotated_imgs[i],(28,28)), cmap='gray')


I am working on the file "Project/mental_rotation_training.ipynb".
The first task is:
Represent each number using a one-hot where the index of the one represents the digit value
Can you write Python code for it?


I am working on the file "Project/mental_rotation_training.ipynb".
The first task is:
Saving weight matrices
Can you write Python code for it?

filename = "label_weights_choose_enc" + str(n_hid) +".p"
cPickle.dump(label_weights, open( filename, "wb" ) )

filename = "activity_to_img_weights_choose_enc" + str(n_hid) +".p"
cPickle.dump(sim.data[conn].weights.T, open( filename, "wb" ) )

#filename = "rotation_clockwise_weights" + str(n_hid) +".p"
#cPickle.dump(weights, open( filename, "wb" ) )

filename = "rotated_counter_after_encoder_weights_choose_enc" + str(n_hid) +".p"
cPickle.dump(rotated_after_encoder_weights, open( filename, "wb" ) )


I am working on the file "Project/mental_rotation_training.ipynb".
The first task is:
The Network

The network parameters must be the same here as when the weight matrices are used later on
The network is made up of an ensemble and two nodes
The first connection ( to v) computes the weights from the activities of the images to the images themselves
The second connection (to v2) computes the weights from the activities of the images to the labels
Can you write Python code for it?

rng = np.random.RandomState(9)

# --- set up network parameters
#Want to encode and decode the image
n_vis = X_train.shape[1]
n_out =  X_train.shape[1]
#number of neurons/dimensions of semantic pointer
n_hid = 1000 #Try with more neurons for more accuracy

#Want the encoding/decoding done on the training images
ens_params = dict(
    eval_points=X_train, #rotated_X_train
    neuron_type=nengo.LIF(), #Why not use LIF? originally used LIFRate()
    intercepts=nengo.dists.Choice([-0.5]),
    max_rates=nengo.dists.Choice([100]),
    )

#Least-squares solver with L2 regularization.
solver = nengo.solvers.LstsqL2(reg=0.01)
#solver = nengo.solvers.LstsqL2(reg=0.0001)
solver2 = nengo.solvers.LstsqL2(reg=0.01)

#network that generates the weight matrices between neuron activity and images and the labels
with nengo.Network(seed=3) as model:
    a = nengo.Ensemble(n_hid, n_vis, seed=3, **ens_params)
    v = nengo.Node(size_in=n_out)
    conn = nengo.Connection(
        a, v, synapse=None,
        eval_points=X_train, function=X_train,#want the same thing out (identity) #X_train
        solver=solver)

    v2 = nengo.Node(size_in=train_targets.shape[1])
    conn2 = nengo.Connection(
        a, v2, synapse=None,
        eval_points=X_train, function=train_targets, #Want to get the labels out
        solver=solver2)

'''
degrees = 6
#must have same number of excoders as neurons (Want each random encoder to have same encoder at every angle)
encoders = Gabor().generate(n_hid/(360/degrees), (11, 11), rng=rng)
encoders = Mask((28, 28)).populate(encoders, rng=rng, flatten=True)

rotated_encoders = encoders.copy()

#For each randomly generated encoder, create the same encoder at every angle (increments chosen by degree)
for encoder in encoders:
    rotated_encoders = np.append(rotated_encoders, [encoder],axis =0)
    for i in range(1,59):
        #new_gabor = rotate(encoder.reshape(28,28),degrees*i,reshape = False).ravel()
        rotated_encoders = np.append(rotated_encoders,...


I am working on the file "Project/mental_scaling_training.ipynb".
The first task is:
Evaluating the network statically

Functions for computing representation of the image at different levels of encoding/decoding
get_outs  returns the output of the network
able to evaluate on many images
no need to run the simulator
Can you write Python code for it?