I am working on the file "Programming_a_seismic_program.ipynb".
The first task is:
Now we can compute some properties of the survey:
Can you write Python code for it?

rlines = int(y/RL) + 1
slines = int(x/SL) + 1

rperline = int(x/ri) + 2
sperline = int(y/si) + 2

shiftx = -si/2.0
shifty = -ri/2.0


I am working on the file "Programming_a_seismic_program.ipynb".
The first task is:
Rename the columns.
Can you write Python code for it?

complete.columns = ['geometry', 'station', 'SID', 'new_geometry']

complete[18:23]


I am working on the file "Programming_a_seismic_program.ipynb".
The first task is:
Save to a shapefile if desired.
Can you write Python code for it?


I am working on the file "Programming_a_seismic_program.ipynb".
The first task is:
Spatial join
Thank you to Jake Wasserman and Kelsey Jordahl for this code snippet, and many pointers.
This takes about 20 seconds to run on my iMac, compared to something close to 30 minutes for the old nested loops.
Can you write Python code for it?

def bin_the_midpoints(bins, midpoints):
    b = bins.copy()
    m = midpoints.copy()
    reindexed = b.reset_index().rename(columns={'index':'bins_index'})
    joined = gpd.tools.sjoin(reindexed, m)
    bin_stats = joined.groupby('bins_index')['offset']\
                      .agg({'fold': len, 'min_offset': np.min})
    return gpd.GeoDataFrame(b.join(bin_stats))

bin_stats = bin_the_midpoints(bins, midpoints)

bin_stats[:10]

ax = bin_stats.plot(column="fold")

ax = bin_stats.plot(column="min_offset")


I am working on the file "Programming_a_seismic_program.ipynb".
The first task is:
Spider plot
Can you write Python code for it?

midpoints['offsetx'] = offsets * np.cos(azimuths)
midpoints['offsety'] = offsets * np.sin(azimuths)
midpoints[:5].offsetx  # Easy!

midpoints.ix[3].geometry.x # Less easy :(


I am working on the file "Programming_a_seismic_program.ipynb".
The first task is:
We can make a list of 'r' and 's' labels then compile into a dataframe.
Can you write Python code for it?

station_list = ['r']*len(rcvrs) + ['s']*len(srcs)
survey = gpd.GeoDataFrame({'geometry': rcvrs+srcs, 'station': station_list})
survey.crs = from_epsg(epsg)

try:
    # Needs geopandas fork: https://github.com/kwinkunks/geopandas
    survey.plot(figsize=(12,12), column='station', colormap="bwr", markersize=20)
except:
    # This will work regardless.
    survey.plot()
plt.grid()
plt.show()


I am working on the file "Programming_a_seismic_program.ipynb".
The first task is:
We compute lists of the x- and y-locations of the receivers and sources with nested loops. We'll use Python's list comprehensions, which are shorthand for for loops that generate lists.
Can you write Python code for it?

rcvrx, rcvry = zip(*[(xmi + rcvr*ri + shiftx, ymi + line*RL - shifty)
                     for line in range(rlines)
                     for rcvr in range(rperline)])

srcx, srcy = zip(*[(xmi + line*SL, ymi + src*si)
                   for line in range(slines)
                   for src in range(sperline)])

rcvrs = [Point(x, y) for x, y in zip(rcvrx, rcvry)]
srcs = [Point(x, y) for x, y in zip(srcx, srcy)]


I am working on the file "Progress/.ipynb_checkpoints/Progress Report-checkpoint.ipynb".
The first task is:
Core Algorithms
Neuron
Can you write Python code for it?

"""Neron for for detrminig what is put in and the weights of certin aspects of the input"""
def __init__(self, n_inputs ):
    self.n_inputs = n_inputs
    self.set_weights( [random.uniform(0,1) for x in range(0,n_inputs+1)] )

def sum(self, inputs ):
        return sum(val*self.weights[i] for i,val in enumerate(inputs))

def set_weights(self, weights ):
    self.weights = weights

def str_(self):
    return ( str(self.weights[:-1]),str(self.weights[-1]) )


I am working on the file "Progress/.ipynb_checkpoints/Progress Report-checkpoint.ipynb".
The first task is:
Visualizations
Can be used for training set and learning to recognize the characteristics of each digit. This will be used as part of the training set.
Can you write Python code for it?

def show_digit(i):
    plt.matshow(digits.images[i]);

print (show_digit(0))

print (show_digit(1))

print (show_digit(2))

print (show_digit(3))

print (show_digit(4))

print (show_digit(5))

print (show_digit(6))

print (show_digit(7))

print (show_digit(8))

print (show_digit(9))

interact(show_digit, i=(0,100));


I am working on the file "Progress/Progress Report.ipynb".
The first task is:
Core Algorithms
Neuron
Can you write Python code for it?


I am working on the file "Progress/Progress Report.ipynb".
The first task is:
Neuron Layer
Can you write Python code for it?

def init(self, n_neurons, n_inputs):
    self.n_neurons = n_neurons
    self.neurons = [Neuron( n_inputs ) for _ in range(0,self.n_neurons)]

def str_(self):
    return 'Layer:\n\t'+'\n\t'.join([str(neuron) for neuron in self.neurons])+''


I am working on the file "Progress/Progress Report.ipynb".
The first task is:
Training Set
Can you write Python code for it?

def learn(self, X, y, learning_rate=0.2, epochs=10000):
    X = np.atleast_2d(X)
    temp = np.ones([X.shape[0], X.shape[1]+1])
    temp[:, 0:-1] = X
    X = temp
    y = np.array(y)

    for i in range(epochs):
        k = np.random.randint(X.shape[0])
        a = [X[i]]

        for j in range(len(self.weights)):
            a.append(self.activation(np.dot(a[j], self.weights[j])))
        error = y[i] - a[-1]
        deltas = [error * self.activation_deriv(a[-1])]

        for i in range(len(a) - 2, 0, -1):
            deltas.append(deltas[-1].dot(self.weights[i].T)*self.activation_deriv(a[i]))
        deltas.reverse()

        for i in range(len(self.weights)):
            layer = np.atleast_2d(a[i])
            delta = np.atleast_2d(deltas[i])
            self.weights[i] += learning_rate * layer.T.dot(delta)

        return self.weights[i]


I am working on the file "Project 1 -DLND-your-first-network/dlnd-your-first-neural-network.ipynb".
The first task is:
Check out your predictions
Here, use the test data to view how well your network is modeling the data. If something is completely wrong here, make sure each step in your network is implemented correctly.
Can you write Python code for it?

fig, ax = plt.subplots(figsize=(8,4))

mean, std = scaled_features['cnt']
predictions = network.run(test_features)*std + mean
ax.plot(predictions[0], label='Prediction')
ax.plot((test_targets['cnt']*std + mean).values, label='Data')
ax.set_xlim(right=len(predictions))
ax.legend()

dates = pd.to_datetime(rides.ix[test_data.index]['dteday'])
dates = dates.apply(lambda d: d.strftime('%b %d'))
ax.set_xticks(np.arange(len(dates))[12::24])
_ = ax.set_xticklabels(dates[12::24], rotation=45)


I am working on the file "Project 1 -DLND-your-first-network/dlnd-your-first-neural-network.ipynb".
The first task is:
Scaling target variables
To make training the network easier, we'll standardize each of the continuous variables. That is, we'll shift and scale the variables such that they have zero mean and a standard deviation of 1.
The scaling factors are saved so we can go backwards when we use the network for predictions.
Can you write Python code for it?

quant_features = ['casual', 'registered', 'cnt', 'temp', 'hum', 'windspeed']
# Store scalings in a dictionary so we can convert back later
scaled_features = {}
for each in quant_features:
    mean, std = data[each].mean(), data[each].std()
    scaled_features[each] = [mean, std]
    data.loc[:, each] = (data[each] - mean)/std
data.head()

#print(data[0:12])
data.index
data.ix[:10,'hum']


I am working on the file "Project 1 -DLND-your-first-network/dlnd-your-first-neural-network.ipynb".
The first task is:
Splitting the data into training, testing, and validation sets
We'll save the last 21 days of the data to use as a test set after we've trained the network. We'll use this set to make predictions and compare them with the actual number of riders.
Can you write Python code for it?

# Save the last 21 days
test_data = data[-21*24:]
data = data[:-21*24]

# Separate the data into features and targets
target_fields = ['cnt', 'casual', 'registered']
features, targets = data.drop(target_fields, axis=1), data[target_fields]
test_features, test_targets = test_data.drop(target_fields, axis=1), test_data[target_fields]


I am working on the file "Project 1/Project-1.ipynb".
The first task is:
Scaling target variables
To make training the network easier, we'll standardize each of the continuous variables. That is, we'll shift and scale the variables such that they have zero mean and a standard deviation of 1.
The scaling factors are saved so we can go backwards when we use the network for predictions.
Can you write Python code for it?

quant_features = ['casual', 'registered', 'cnt', 'temp', 'hum', 'windspeed']
# Store scalings in a dictionary so we can convert back later
scaled_features = {}
for each in quant_features:
    mean, std = data[each].mean(), data[each].std()
    scaled_features[each] = [mean, std]
    data.loc[:, each] = (data[each] - mean)/std


I am working on the file "Project 1/Project-1.ipynb".
The first task is:
Splitting the data into training, testing, and validation sets
We'll save the last 21 days of the data to use as a test set after we've trained the network. We'll use this set to make predictions and compare them with the actual number of riders.
Can you write Python code for it?


I am working on the file "Project 1/Project-1.ipynb".
The first task is:
Training the network
Here you'll set the hyperparameters for the network. The strategy here is to find hyperparameters such that the error on the training set is low, but you're not overfitting to the data. If you train the network too long or have too many hidden nodes, it can become overly specific to the training set and will fail to generalize to the validation set. That is, the loss on the validation set will start increasing as the training set loss drops.
You'll also be using a method know as Stochastic Gradient Descent (SGD) to train the network. The idea is that for each training pass, you grab a random sample of the data instead of using the whole data set. You use many more training passes than with normal gradient descent, but each pass is much faster. This ends up training the network more efficiently. You'll learn more about SGD later.
Choose the number of epochs
This is the number of times the dataset will pass through the network, each time updating the weights. As the number of epochs increases, the network becomes better and better at predicting the targets in the training set. You'll need to choose enough epochs to train the network well but not too many or you'll be overfitting.
Choose the learning rate
This scales the size of weight updates. If this is too big, the weights tend to explode and the network fails to fit the data. A good choice to start at is 0.1. If the network has problems fitting the data, try reducing the learning rate. Note that the lower the learning rate, the smaller the steps are in the weight updates and the longer it takes for the neural network to converge.
Choose the number of hidden nodes
The more hidden nodes you have, the more accurate predictions the model will make. Try a few different numbers and see how it affects the performance. You can look at the losses dictionary for a metric of the network performance. If the number of hidden units is too low, then the model won't have enough space to learn and if it is too high there are too many options for the...
Can you write Python code for it?

import sys

### Set the hyperparameters here ###
epochs = 2000
learning_rate = 0.008
hidden_nodes = 10
output_nodes = 1

N_i = train_features.shape[1]
network = NeuralNetwork(N_i, hidden_nodes, output_nodes, learning_rate)

losses = {'train':[], 'validation':[]}
for e in range(epochs):
    # Go through a random batch of 128 records from the training data set
    batch = np.random.choice(train_features.index, size=128)
    for record, target in zip(train_features.ix[batch].values,
                              train_targets.ix[batch]['cnt']):
        network.train(record, target)

    # Printing out the training progress
    train_loss = MSE(network.run(train_features), train_targets['cnt'].values)
    val_loss = MSE(network.run(val_features), val_targets['cnt'].values)
    if e%(epochs/10) == 0:
        sys.stdout.write("\nProgress: " + str(100 * e/float(epochs))[:4] \
                         + "% ... Training loss: " + str(train_loss)[:5] \
                         + " ... Validation loss: " + str(val_loss)[:5])

    losses['train'].append(train_loss)
    losses['validation'].append(val_loss)

plt.plot(losses['train'], label='Training loss')
plt.plot(losses['validation'], label='Validation loss')
plt.legend()
plt.ylim(ymax=0.5)


I am working on the file "Project 1/Project-1.ipynb".
The first task is:
We'll split the data into two sets, one for training and one for validating as the network is being trained. Since this is time series data, we'll train on historical data, then try to predict on future data (the validation set).
Can you write Python code for it?

# Hold out the last 60 days of the remaining data as a validation set
train_features, train_targets = features[:-60*24], targets[:-60*24]
val_features, val_targets = features[-60*24:], targets[-60*24:]


I am working on the file "Project 1/dlnd-your-first-neural-network.ipynb".
The first task is:
Check out your predictions
Here, use the test data to view how well your network is modeling the data. If something is completely wrong here, make sure each step in your network is implemented correctly.
Can you write Python code for it?


I am working on the file "Project 1/dlnd-your-first-neural-network.ipynb".
The first task is:
Load and prepare the data
A critical step in working with neural networks is preparing the data correctly. Variables on different scales make it difficult for the network to efficiently learn the correct weights. Below, we've written the code to load and prepare the data. You'll learn more about this soon!
Can you write Python code for it?

data_path = 'Bike-Sharing-Dataset/hour.csv'

rides = pd.read_csv(data_path)

rides.head()


I am working on the file "Project 1/dlnd-your-first-neural-network.ipynb".
The first task is:
Splitting the data into training, testing, and validation sets
We'll save the last 21 days of the data to use as a test set after we've trained the network. We'll use this set to make predictions and compare them with the actual number of riders.
Can you write Python code for it?


I am working on the file "Project 1/dlnd-your-first-neural-network.ipynb".
The first task is:
Thinking about your results
Answer these questions about your results. How well does the model predict the data? Where does it fail? Why does it fail where it does?

Note: You can edit the text in this cell by double clicking on it. When you want to render the text, press control + enter

Your answer below
Unit tests
Run these unit tests to check the correctness of your network implementation. These tests must all be successful to pass the project.
Can you write Python code for it?

import unittest

inputs = [0.5, -0.2, 0.1]
targets = [0.4]
test_w_i_h = np.array([[0.1, 0.4, -0.3],
                       [-0.2, 0.5, 0.2]])
test_w_h_o = np.array([[0.3, -0.1]])

class TestMethods(unittest.TestCase):

    ##########
    # Unit tests for data loading
    ##########

    def test_data_path(self):
        # Test that file path to dataset has been unaltered
        self.assertTrue(data_path.lower() == 'bike-sharing-dataset/hour.csv')

    def test_data_loaded(self):
        # Test that data frame loaded
        self.assertTrue(isinstance(rides, pd.DataFrame))

    ##########
    # Unit tests for network functionality
    ##########

    def test_activation(self):
        network = NeuralNetwork(3, 2, 1, 0.5)
        # Test that the activation function is a sigmoid
        self.assertTrue(np.all(network.activation_function(0.5) == 1/(1+np.exp(-0.5))))

    def test_train(self):
        # Test that weights are updated correctly on training
        network = NeuralNetwork(3, 2, 1, 0.5)
        network.weights_input_to_hidden = test_w_i_h.copy()
        network.weights_hidden_to_output = test_w_h_o.copy()

        network.train(inputs, targets)
        self.assertTrue(np.allclose(network.weights_hidden_to_output,
                                    np.array([[ 0.37275328, -0.03172939]])))
        self.assertTrue(np.allclose(network.weights_input_to_hidden,
                                    np.array([[ 0.10562014,  0.39775194, -0.29887597],
                                              [-0.20185996,  0.50074398,  0.19962801]])))

    def test_run(self):
        # Test correctness of run method
        network = NeuralNetwork(3, 2, 1, 0.5)
        network.weights_input_to_hidden = test_w_i_h.copy()
        network.weights_hidden_to_output = test_w_h_o.copy()

        self.assertTrue(np.allclose(network.run(inputs), 0.09998924))

suite = unittest.TestLoader().loadTestsFromModule(TestMethods)
unittest.TextTestRunner().run(suite)


I am working on the file "Project 1/dlnd-your-first-neural-network.ipynb".
The first task is:
We'll split the data into two sets, one for training and one for validating as the network is being trained. Since this is time series data, we'll train on historical data, then try to predict on future data (the validation set).
Can you write Python code for it?


I am working on the file "Project 2 Titanic Data Final.ipynb".
The first task is:
Finally, to answer my last question of if being in a higher class affected the probability of you surviving, I used the same seaborn factorplot but to create this graph i had to take the sum of survivors and divide them by the count of survivors in each class.
Can you write Python code for it?

sea.factorplot('Pclass', "Survived", data=titanic_ds, kind='bar', ci=None, size=5)

PS=(titanic_ds.groupby('Pclass')['Survived'].sum())
PC=(titanic_ds.groupby('Pclass')['Survived'].count())
print ("Class Survivability: ")
print (PS/PC)


I am working on the file "Project 2 Titanic Data Final.ipynb".
The first task is:
For the purpose of cleaning up the data I determined that the Name and Ticket columns were not necessary for my future analysis.
Can you write Python code for it?

##dont need name and ticket for what I am going to be tackeling
titanic_ds = titanic_ds.drop(["Name", "Ticket"], axis=1)


I am working on the file "Project 2 Titanic Data Final.ipynb".
The first task is:
For this section I utilized Seaborn's factorplot function to graph the count of male's and females in each class.
Can you write Python code for it?

##Gender distribution by class

gender_class= sea.factorplot('Pclass',order=[1,2,3], data=titanic_ds, hue='Sex', kind='count')
gender_class.set_ylabels("count of passengers")


I am working on the file "Project 2 Titanic Data Final.ipynb".
The first task is:
To answer my second questoins I showed survived data with age in a box plot to show average age as well as its distruvtion for both deseased and survived.
Can you write Python code for it?

##Age box plot, survived and did not survive
##fewer people survived as compared to deseased

age_box=sea.boxplot(x="Survived", y="Age", data=titanic_ds)
age_box.set(xlabel = 'Survived', ylabel = 'Age', xticklabels = ['Desased', 'Survived'])


I am working on the file "Project 2 Titanic Data Final.ipynb".
The first task is:
To now show the probability visually as well as its output I have gone ahead and created a factorplot as well as printed out the probabilities of the two (Alone = False and Family = True). To get the probabilities I had to divide the sum of survivors by family type and divide by the count of family type.
Can you write Python code for it?

fanda = sea.factorplot('Family', "Survived", data=titanic_ds, kind='bar', ci=None, size=5)
fanda.set(xticklabels = ['Alone', 'Family'])
print ((titanic_ds.groupby('Family')['Survived'].sum()/titanic_ds.groupby('Family')['Survived'].count()))


I am working on the file "Project 2 Titanic Data Final.ipynb".
The first task is:
Using those two data sets that I created in the previous block, I printed the counts to understand how many of each sex were on the boat.
Can you write Python code for it?

# idea of the spread between men and women
print("Males: ")
print(men_ds.count()['Sex'])

print("Females: ")
print(women_ds.count()['Sex'])


I am working on the file "Project 2- Data Analysis with Python/titanic_project.ipynb".
The first task is:
2.2 Data Cleanup
The data cleanup procedure consists of the follwing steps:
1. Removal of unnecessary data
2. Removal of duplicates
3. Determine missing values in the data
Based on the question I want to answer in the project some of the columns in the dataset will not be important in further
examination, therefore they will be removed. These columns are:

Name
Ticket
Cabin
Embarked
sibsp

Following code lines remove the mentioned rows and show the first five rows of the dataframe to show that the unnecessary data was removed.
Can you write Python code for it?

# Create new dataset without unwanted columns
titanic=titanic.drop(['Name','Ticket','Cabin','Embarked','SibSp'], axis=1)
titanic.head()


I am working on the file "Project 2- Data Analysis with Python/titanic_project.ipynb".
The first task is:
According to the plot it is obvious that male passengers had a significant lower chance of survival in comparison to female passengers.
3.2 Did the social-economic status determine the chances of survival?
Can you write Python code for it?

################################ Survival regarding the fare ##########################
# Create a dataframe of all fares
fares_df=titanic[["Fare", "Survived"]]
fares=titanic["Fare"]

# Create 20 fare ranges from 0 $ - 300 $ and count how many fares from fares_df belong to each range
num_bins=20
bar_width=300/float(num_bins)
fare_ranges_all=[]
for i in np.arange(0,num_bins,1):
    fare_ranges_all.append(len([x for x in fares if i*bar_width <= x < (i+1)*bar_width]))

# Create a dataframe with fares of passengers who survived
survived_fares=fares_df.ix[(fares_df["Survived"]==1)]["Fare"]

# Determine how many fares of passengers who survived belong in each of the 20 ranges
fare_ranges_survived=[]
for i in np.arange(0,num_bins,1):
    fare_ranges_survived.append(len([x for x in survived_fares if i*bar_width <= x < (i+1)*bar_width]))

# Handle the case in which a fare range does not contain any counts (to avoid devide by null error during normalization)
for n,i in enumerate(fare_ranges_all):
    if i==0:
      fare_ranges_all[n]=1

################################ Survival regarding the class ##########################

# Get the Groupby object
g=titanic.groupby(["Survived","Pclass"])

# Count the passengers from each class who not have survived
dead_class_1=g.get_group((0,1))["PassengerId"].count()
dead_class_2=g.get_group((0,2))["PassengerId"].count()
dead_class_3=g.get_group((0,3))["PassengerId"].count()

# Count the passengers from each class who  have survived
survived_class_1=g.get_group((1,1))["PassengerId"].count()
survived_class_2=g.get_group((1,2))["PassengerId"].count()
survived_class_3=g.get_group((1,3))["PassengerId"].count()

# Get the Groupby object
g=titanic.groupby(["Pclass"])
# Count the passengers in each class
passengers_class1=float(g.get_group((1))["PassengerId"].count())
passengers_class2=float(g.get_group((2))["PassengerId"].count())
passengers_class3=float(g.get_group((3))["PassengerId"].count())

# Plot the fare-survival relation

fig, axes = plt.subplots(nrows=1, ncols=2,...


I am working on the file "Project 2- Data Analysis with Python/titanic_project.ipynb".
The first task is:
Here I want to identify duplicates in the data.
Can you write Python code for it?

# Identify if duplicates in the data do exists
titanic_duplicates = titanic.duplicated()
print('Number of duplicates: {}'.format(titanic_duplicates.sum()))


I am working on the file "Project 2- Data Analysis with Python/titanic_project.ipynb".
The first task is:
If we take the gender into account it can be seen that male passengers in their middle ages had a much lower ratio of survival than women. Male children had however still big chances for survival.
In oppositve female passengers had throughout all age groupes higher survival ratio than male passengers.
3.5 Did the number of children aboard per passenger determine the chances of survival?
Can you write Python code for it?

# How many passengers do have how many children? Create new dataframe and transform it's columns to a dictionary
parch_count=pd.DataFrame({'count' : titanic.groupby("Parch")["PassengerId"].count()}).reset_index()
parch_count=dict(zip(parch_count["Parch"],parch_count["count"]))

# Same dataframe as above but also grouped by survival
df=pd.DataFrame({'count' : titanic.groupby(["Survived","Parch"])["PassengerId"].count()}).reset_index()

# Calculate the survival ratio per children aboard
surival_ratio={}
for index, row in df.iterrows():
    if not row["Survived"]:
        surival_ratio[row["Parch"]]=1-(row["count"]/float(parch_count[row["Parch"]]))

plt.figure(figsize=(10,6))
plt.bar(list(surival_ratio.keys()),list(surival_ratio.values()),color='#377eb8', edgecolor = "Black")
plt.title("Survival ratio per children aboard")
plt.xlabel("Number of children per passenger")
plt.ylabel("Ratio of survival")
plt.show()


I am working on the file "Project 2- Data Analysis with Python/titanic_project.ipynb".
The first task is:
The age distribution can be approximated as a gaussian distribution with a mean around 25-30 years.
In the second plot can be noticed that there were almost twice as much male as female passengers. The third plot shows that there were mored deads than survivals.
3.1 Did the gender determine the chances of survival?
Can you write Python code for it?

# Group thy PassengerId by Gender und Survival
g=titanic.groupby(["Survived","Sex"])["PassengerId"]

# Count how many passengers died or survived dependent on the gender
survived_men=g.get_group((1,"male")).count()
survived_women=g.get_group((1,"female")).count()
dead_men=g.get_group((0,"male")).count()
dead_women=g.get_group((0,"female")).count()

# Group the PassengerId by gender and count the Id's dependent on the gender to find out the total number of women and men
g=titanic.groupby("Sex")["PassengerId"]
men_sum=float(g.get_group(("male")).count())
women_sum=float(g.get_group(("female")).count())

#Normalization of dead and survived passengers depentend on the gender
p2=survived=[survived_men/men_sum, survived_women/women_sum]
p1=dead=[dead_men/men_sum, dead_women/women_sum]

# Plot the survival by gender ration
plt.figure(figsize=(7,7))
N=2
ind = np.arange(N)
width = 0.35
bar1 = plt.bar(ind, survived, width,color='#377eb8', edgecolor = "Black")
bar2 = plt.bar(ind+width, dead, width,color='#e41a1c', edgecolor = "Black")
plt.ylabel('Ratio of passengers')
plt.title('Survival by Gender')
plt.xticks(ind+width/2, ['Men', "Female"])
plt.legend((bar2, bar1), ('Dead', 'Survived'))
plt.figure(num=None, figsize=(1, 1), dpi=80, facecolor='w', edgecolor='k')
plt.show()


I am working on the file "Project 2- Data Analysis with Python/titanic_project.ipynb".
The first task is:
We can see that roughly 20% of the passengers in the dataset do not have a stated age, especialy the male passengers. This fact should be considered on examination of the question whether the age considering the gender determine the chances of survival.
3. Data Analysis
At first I want to gain a basic overview over the age distribution, the count of female and male as well as the count of survived and dead passengers.
Can you write Python code for it?

fig, axes = plt.subplots(nrows=1, ncols=3, figsize=(17, 5))

#Age distribution
ages=titanic["Age"].dropna()
#plt.figure(figsize=(7,7))
axes[0].hist(ages, bins=80, color='#377eb8',edgecolor = "Black")
axes[0].set_xlabel("Age/Years")
axes[0].set_ylabel('Count')
axes[0].set_title('Age distribution of the passengers')
#plt.show()

# Count of male and female passengers
gender=titanic['Sex']
counts = Counter(gender)
common = counts.most_common()
gender = [item[0] for item in common]
count = [item[1] for item in common]

axes[1].bar(np.arange(2), count, tick_label=gender, width=0.4, color='#377eb8',edgecolor = "Black")
axes[1].set_ylabel('Count')
axes[1].set_title('Number of female and male passengers.')

# Count of dead and survived passengers
survival=titanic['Survived']
counts=Counter(survival)
common = counts.most_common()
label=["Dead", "Survived"]
count=[item[1] for item in common]
axes[2].bar(np.arange(2), count, tick_label=label, width=0.4, color='#377eb8',edgecolor = "Black")
axes[2].set_ylabel('Count')
axes[2].set_title('Number of survived and dead passengers.')
plt.show()


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
Build the network
For the neural network, you'll build each layer into a function.  Most of the code you've seen has been outside of functions. To test your code more thoroughly, we require that you put each layer in a function.  This allows us to give you better feedback and test for simple mistakes using our unittests before you submit your project.

Note: If you're finding it hard to dedicate enough time for this course each week, we've provided a small shortcut to this part of the project. In the next couple of problems, you'll have the option to use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages to build each layer, except the layers you build in the "Convolutional and Max Pooling Layer" section.  TF Layers is similar to Keras's and TFLearn's abstraction to layers, so it's easy to pickup.
However, if you would like to get the most out of this course, try to solve all the problems without using anything from the TF Layers packages. You can still use classes from other packages that happen to have the same name as ones you find in TF Layers! For example, instead of using the TF Layers version of the conv2d class, tf.layers.conv2d, you would want to use the TF Neural Network version of conv2d, tf.nn.conv2d.

Let's begin!
Input
The neural network needs to read the image data, one-hot encoded labels, and dropout keep probability. Implement the following functions
* Implement neural_net_image_input
 * Return a TF Placeholder
 * Set the shape using image_shape with batch size set to None.
 * Name the TensorFlow placeholder "x" using the TensorFlow name parameter in the TF Placeholder.
* Implement neural_net_label_input
 * Return a TF Placeholder
 * Set the shape using n_classes with batch size set to None.
 * Name the TensorFlow placeholder "y" using the TensorFlow name parameter in the TF Placeholder.
* Implement neural_net_keep_prob_input
 * Return a TF Placeholder for dropout keep probability.
 * Name the TensorFlow placeholder "keep_prob" using the TensorFlow name parameter in...
Can you write Python code for it?

import tensorflow as tf

def neural_net_image_input(image_shape):
    """
    Return a Tensor for a bach of image input
    : image_shape: Shape of the images
    : return: Tensor for image input.
    """
    return tf.placeholder(tf.float32, [None, image_shape[0], image_shape[1], image_shape[2]], name='x')

def neural_net_label_input(n_classes):
    """
    Return a Tensor for a batch of label input
    : n_classes: Number of classes
    : return: Tensor for label input.
    """
    return tf.placeholder(tf.float32, [None, n_classes], name='y')

def neural_net_keep_prob_input():
    """
    Return a Tensor for keep probability
    : return: Tensor for keep probability.
    """
    return tf.placeholder(tf.float32, name='keep_prob')

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tf.reset_default_graph()
tests.test_nn_image_inputs(neural_net_image_input)
tests.test_nn_label_inputs(neural_net_label_input)
tests.test_nn_keep_prob_inputs(neural_net_keep_prob_input)


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
Check Point
This is your first checkpoint.  If you ever decide to come back to this notebook or have to restart the notebook, you can start from here.  The preprocessed data has been saved to disk.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import pickle
import problem_unittests as tests
import helper

# Load the Preprocessed Validation data
valid_features, valid_labels = pickle.load(open('preprocess_validation.p', mode='rb'))


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
Convolution and Max Pooling Layer
Convolution layers have a lot of success with images. For this code cell, you should implement the function conv2d_maxpool to apply convolution then max pooling:
* Create the weight and bias using conv_ksize, conv_num_outputs and the shape of x_tensor.
* Apply a convolution to x_tensor using weight and conv_strides.
 * We recommend you use same padding, but you're welcome to use any padding.
* Add bias
* Add a nonlinear activation to the convolution.
* Apply Max Pooling using pool_ksize and pool_strides.
 * We recommend you use same padding, but you're welcome to use any padding.
Note: You can't use TensorFlow Layers or TensorFlow Layers (contrib) for this layer, but you can still use TensorFlow's Neural Network package. You may still use the shortcut option for all the other layers.
Can you write Python code for it?

def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides):
    """
    Apply convolution then max pooling to x_tensor
    :param x_tensor: TensorFlow Tensor
    :param conv_num_outputs: Number of outputs for the convolutional layer
    :param conv_ksize: kernal size 2-D Tuple for the convolutional layer
    :param conv_strides: Stride 2-D Tuple for convolution
    :param pool_ksize: kernal size 2-D Tuple for pool
    :param pool_strides: Stride 2-D Tuple for pool
    : return: A tensor that represents convolution and max pooling of x_tensor
    """
    input_image_depth = tf.to_int32(x_tensor.get_shape()[3])
    # TODO: Implement Function
    weights = tf.Variable(tf.random_normal([
        conv_ksize[0],
        conv_ksize[1],
        input_image_depth,
        conv_num_outputs],
        stddev=0.1))
    bias = tf.Variable(tf.zeros(conv_num_outputs,dtype=tf.float32))

    cv_layer = tf.nn.conv2d(x_tensor, weights, strides=[1, conv_strides[0], conv_strides[1], 1], padding='SAME')
    cv_layer = tf.nn.bias_add(cv_layer,bias)
    #DEBUG: print('Size before max_pool: ',cv_layer.get_shape())
    cv_layer = tf.nn.relu(cv_layer)
    cv_layer = tf.nn.max_pool(cv_layer, ksize=[1, pool_ksize[0], pool_ksize[1] , 1],strides=[1, pool_strides[0], pool_strides[1], 1],padding='SAME')
    return cv_layer

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_con_pool(conv2d_maxpool)


def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides):
    """
    Apply convolution then max pooling to x_tensor
    :param x_tensor: TensorFlow Tensor
    :param conv_num_outputs: Number of outputs for the convolutional layer
    :param conv_ksize: kernal size 2-D Tuple for the convolutional layer
    :param conv_strides: Stride 2-D Tuple for convolution
    :param pool_ksize: kernal size 2-D Tuple for pool
    :param pool_strides: Stride 2-D Tuple for pool
    : return: A tensor that represents convolution and max pooling of x_tensor
    """
    weights = tf.Variable(tf.truncated_normal([conv_ksize[0],
                                     conv_ksize[1],
                                     x_tensor.get_shape().as_list()[-1],
                                     conv_num_outputs], mean=0.0, stddev=0.05))

    biases = tf.Variable(tf.random_normal([conv_num_outputs], mean=0.0, stddev=0.05))

    conv_layer = tf.nn.conv2d(x_tensor, weights,
             strides=[1, conv_strides[0], conv_strides[1], 1],
             padding='SAME')

    biased_conv_layer = tf.nn.bias_add(conv_layer, biases)

    relu_layer = tf.nn.relu(biased_conv_layer)

    conv2d_maxpool_layer = tf.nn.max_pool(relu_layer, ksize=[1, pool_ksize[0], pool_ksize[1], 1],
                      strides=[1, pool_strides[0], pool_strides[1], 1],
                      padding='SAME')

    return conv2d_maxpool_layer

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_con_pool(conv2d_maxpool)


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
Create Convolutional Model
Implement the function conv_net to create a convolutional neural network model. The function takes in a batch of images, x, and outputs logits.  Use the layers you created above to create this model:

Apply 1, 2, or 3 Convolution and Max Pool layers
Apply a Flatten Layer
Apply 1, 2, or 3 Fully Connected Layers
Apply an Output Layer
Return the output
Apply TensorFlow's Dropout to one or more layers in the model using keep_prob.
Can you write Python code for it?

def conv_net(x, keep_prob):
    """
    Create a convolutional neural network model
    : x: Placeholder tensor that holds image data.
    : keep_prob: Placeholder tensor that hold dropout keep probability.
    : return: Tensor that represents logits
    """
    #hyperparameters
    k1_size = [2,2]
    conv1_num_outputs = 32
    strides_1 = [1,1]
    pool_ksize_1 = [2,2]
    pool_strides_1 = [2,2]

    k2_size = [2,2]
    conv2_num_outputs = 96
    strides_2 = [1,1]
    pool_ksize_2 = [2,2]
    pool_strides_2 = [2,2]

    k3_size = [2,2]
    conv3_num_outputs = 256
    strides_3 = [1,1]
    pool_ksize_3 = [2,2]
    pool_strides_3 = [2,2]

    # TODO: Apply 1, 2, or 3 Convolution and Max Pool layers
    #    Play around with different number of outputs, kernel size and stride
    # Function Definition from Above:

    #    conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)

    #build conv size
    print('Size of image: ', x.get_shape())
    conv1 = conv2d_maxpool(x, conv1_num_outputs, k1_size, strides_1, pool_ksize_1, pool_strides_1)
    print('Size after 1st conv: ', conv1.get_shape())

    conv2 = conv2d_maxpool(conv1, conv2_num_outputs, k2_size, strides_2, pool_ksize_2, pool_strides_2)
    print('Size after 2nd conv: ', conv2.get_shape())

    conv3 = conv2d_maxpool(conv2, conv3_num_outputs, k3_size, strides_3, pool_ksize_3, pool_strides_3)
    print('Size after 3nd conv: ', conv3.get_shape())

    # TODO: Apply a Flatten Layer
    # Function Definition from Above:
    #   flatten(x_tensor)
    fc1 = flatten(conv3)

    # TODO: Apply 1, 2, or 3 Fully Connected Layers
    #    Play around with different number of outputs
    # Function Definition from Above:
    #   fully_conn(x_tensor, num_outputs)
    num_outputs_1 = 400

    fc1 = fully_conn(fc1, num_outputs_1)
    print('Size after fully connected 1: ', fc1.get_shape())
    fc1 = tf.nn.dropout(fc1, keep_prob)

    num_outputs_2 = 400

    fc2 = fully_conn(fc1,...


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
Explore the Data
The dataset is broken into batches to prevent your machine from running out of memory.  The CIFAR-10 dataset consists of 5 batches, named data_batch_1, data_batch_2, etc.. Each batch contains the labels and images that are one of the following:
* airplane
* automobile
* bird
* cat
* deer
* dog
* frog
* horse
* ship
* truck
Understanding a dataset is part of making predictions on the data.  Play around with the code cell below by changing the batch_id and sample_id. The batch_id is the id for a batch (1-5). The sample_id is the id for a image and label pair in the batch.
Ask yourself "What are all possible labels?", "What is the range of values for the image data?", "Are the labels in order or random?".  Answers to questions like these will help you preprocess the data and end up with better predictions.
Can you write Python code for it?

%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import helper
import numpy as np

# Explore the dataset
batch_id = 1
sample_id = 100
helper.display_stats(cifar10_dataset_folder_path, batch_id, sample_id)


%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import helper
import numpy as np

# Explore the dataset
batch_id = 2
sample_id = 2
helper.display_stats(cifar10_dataset_folder_path, batch_id, sample_id)


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
Flatten Layer
Implement the flatten function to change the dimension of x_tensor from a 4-D tensor to a 2-D tensor.  The output should be the shape (Batch Size, Flattened Image Size). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.
Can you write Python code for it?

def flatten(x_tensor):
    """
    Flatten x_tensor to (Batch Size, Flattened Image Size)
    : x_tensor: A tensor of size (Batch Size, ...), where ... are the image dimensions.
    : return: A tensor of size (Batch Size, Flattened Image Size).
    """
    return tf.reshape(x_tensor, [-1, np.prod(x_tensor.get_shape().as_list()[1:])])

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
#print(flatten.get_shape())
tests.test_flatten(flatten)


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
Fully Train the Model
Now that you got a good accuracy with a single CIFAR-10 batch, try it with all five batches.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
save_model_path = './image_classification'

print('Training...')
with tf.Session() as sess:
    # Initializing the variables
    sess.run(tf.global_variables_initializer())

    # Training cycle
    for epoch in range(epochs):
        # Loop over all batches
        n_batches = 5
        for batch_i in range(1, n_batches + 1):
            for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):
                train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)
            print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')
            print_stats(sess, batch_features, batch_labels, cost, accuracy)

    # Save Model
    saver = tf.train.Saver()
    save_path = saver.save(sess, save_model_path)


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
Fully-Connected Layer
Implement the fully_conn function to apply a fully connected layer to x_tensor with the shape (Batch Size, num_outputs). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.
Can you write Python code for it?

def fully_conn(x_tensor, num_outputs):
    """
    Apply a fully connected layer to x_tensor using weight and bias
    : x_tensor: A 2-D tensor where the first dimension is batch size.
    : num_outputs: The number of output that the new tensor should be.
    : return: A 2-D tensor where the second dimension is num_outputs.
    """
    weights = tf.Variable(tf.random_normal([x_tensor.get_shape().as_list()[1],num_outputs],stddev=0.1))
    bias = tf.Variable(tf.zeros(num_outputs,dtype=tf.float32))

    fc_layer = tf.add(tf.matmul(x_tensor, weights), bias)
    fc_layer = tf.nn.relu(fc_layer)
    return fc_layer

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_fully_conn(fully_conn)


def fully_conn(x_tensor, num_outputs):
    """
    Apply a fully connected layer to x_tensor using weight and bias
    : x_tensor: A 2-D tensor where the first dimension is batch size.
    : num_outputs: The number of output that the new tensor should be.
    : return: A 2-D tensor where the second dimension is num_outputs.
    """
    return tf.contrib.layers.fully_connected(x_tensor, num_outputs, activation_fn=tf.nn.relu)

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_fully_conn(fully_conn)


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
One-hot encode
Just like the previous code cell, you'll be implementing a function for preprocessing.  This time, you'll implement the one_hot_encode function. The input, x, are a list of labels.  Implement the function to return the list of labels as One-Hot encoded Numpy array.  The possible values for labels are 0 to 9. The one-hot encoding function should return the same encoding for each value between each call to one_hot_encode.  Make sure to save the map of encodings outside the function.
Hint: Don't reinvent the wheel.
Can you write Python code for it?

def one_hot_encode(x):
    """
    One hot encode a list of sample labels. Return a one-hot encoded vector for each label.
    : x: List of sample Labels
    : return: Numpy array of one-hot encoded labels
    """
    return sklearn.preprocessing.LabelBinarizer().fit(range(0,10)).transform(x)

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_one_hot_encode(one_hot_encode)


from sklearn.preprocessing import LabelBinarizer

encoder = LabelBinarizer()
encoder.fit(np.arange(10))
def one_hot_encode(x):
    """
    One hot encode a list of sample labels. Return a one-hot encoded vector for each label.
    : x: List of sample Labels
    : return: Numpy array of one-hot encoded labels
    """
    return encoder.transform(x)

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_one_hot_encode(one_hot_encode)


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
Output Layer
Implement the output function to apply a fully connected layer to x_tensor with the shape (Batch Size, num_outputs). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.
Note: Activation, softmax, or cross entropy should not be applied to this.
Can you write Python code for it?

def output(x_tensor, num_outputs):
    """
    Apply a output layer to x_tensor using weight and bias
    : x_tensor: A 2-D tensor where the first dimension is batch size.
    : num_outputs: The number of output that the new tensor should be.
    : return: A 2-D tensor where the second dimension is num_outputs.
    """
    # TODO: Implement Function
    weights = tf.Variable(tf.random_normal([x_tensor.get_shape().as_list()[1],num_outputs],stddev=0.1))
    bias = tf.Variable(tf.zeros(num_outputs,dtype=tf.float32))

    output = tf.add(tf.matmul(x_tensor, weights), bias)
    return output

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_output(output)


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
Randomize Data
As you saw from exploring the data above, the order of the samples are randomized.  It doesn't hurt to randomize it again, but you don't need to for this dataset.
Preprocess all the data and save it
Running the code cell below will preprocess all the CIFAR-10 data and save it to file. The code below also uses 10% of the training data for validation.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# Preprocess Training, Validation, and Testing Data
helper.preprocess_and_save_data(cifar10_dataset_folder_path, normalize, one_hot_encode)


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
Show Stats
Implement the function print_stats to print loss and validation accuracy.  Use the global variables valid_features and valid_labels to calculate validation accuracy.  Use a keep probability of 1.0 to calculate the loss and validation accuracy.
Can you write Python code for it?

def print_stats(session, feature_batch, label_batch, cost, accuracy):
    """
    Print information about loss and validation accuracy
    : session: Current TensorFlow session
    : feature_batch: Batch of Numpy image data
    : label_batch: Batch of Numpy label data
    : cost: TensorFlow cost function
    : accuracy: TensorFlow accuracy function
    """
    #valid_features, valid_labels
    loss = session.run(cost,feed_dict={x: feature_batch, y: label_batch, keep_prob: 1.})
    validation_accuracy = session.run(accuracy, feed_dict={x: valid_features, y: valid_labels, keep_prob: 1.})

    print('Loss is at: {:>10.4f}, Validation Accuracy is at: {:.6f}'.format(
                loss,
                validation_accuracy))


I am working on the file "Project 2/dlnd_image_classification.ipynb".
The first task is:
Train on a Single CIFAR-10 Batch
Instead of training the neural network on all the CIFAR-10 batches of data, let's use a single batch. This should save time while you iterate on the model to get a better accuracy.  Once the final validation accuracy is 50% or greater, run the model on all the data in the next section.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
print('Checking the Training on a Single Batch...')
with tf.Session() as sess:
    # Initializing the variables
    sess.run(tf.global_variables_initializer())
    ep, loss, valid_acc = [], [], []

    # Training cycle
    for epoch in range(epochs):
        batch_i = 1
        for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):
            train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)
        stats = get_loss_and_accuracy(sess, batch_features, batch_labels, cost, accuracy)
        ep.append(epoch + 1)
        loss.append(stats[0])
        valid_acc.append(stats[1]/100)
        plot_loss_and_accuracy(ep, loss, valid_acc)
    print('{:>2} Epochs, CIFAR-10 Batch {}\nTraining Loss:{:>4.4f}\tValidation Accuracy: {:.4f}'\
          .format(ep[-1], batch_i, loss[-1], valid_acc[-1]))


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
print('Checking the Training on a Single Batch...')
with tf.Session() as sess:
    # Initializing the variables
    sess.run(tf.global_variables_initializer())

    # Training cycle
    for epoch in range(epochs):
        batch_i = 1
        for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):
            train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)
        print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')
        print_stats(sess, batch_features, batch_labels, cost, accuracy)


I am working on the file "Project 3/dlnd_tv_script_generation.ipynb".
The first task is:
Batches
Implement get_batches to create batches of input and targets using int_text.  The batches should be a Numpy array with the shape (number of batches, 2, batch size, sequence length). Each batch contains two elements:
- The first element is a single batch of input with the shape [batch size, sequence length]
- The second element is a single batch of targets with the shape [batch size, sequence length]
If you can't fill the last batch with enough data, drop the last batch.
For exmple, get_batches([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], 2, 3) would return a Numpy array of the following:
```
[
  # First Batch
  [
    # Batch of Input
    [[ 1  2  3], [ 7  8  9]],
    # Batch of targets
    [[ 2  3  4], [ 8  9 10]]
  ],
# Second Batch
  [
    # Batch of Input
    [[ 4  5  6], [10 11 12]],
    # Batch of targets
    [[ 5  6  7], [11 12 13]]
  ]
]
```
Can you write Python code for it?

def get_batches(int_text, batch_size, seq_length):
    """
    Return batches of input and target
    :param int_text: Text with the words replaced by their ids
    :param batch_size: The size of batch
    :param seq_length: The length of sequence
    :return: Batches as a Numpy array
    """
    n_batches = int(len(int_text) / (batch_size * seq_length))

    xdata = np.array(int_text[: n_batches * batch_size * seq_length])
    ydata = np.array(int_text[1: n_batches * batch_size * seq_length + 1])

    x_batches = np.split(xdata.reshape(batch_size, -1), n_batches, 1)
    y_batches = np.split(ydata.reshape(batch_size, -1), n_batches, 1)

    return np.array(list(zip(x_batches, y_batches)))

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_get_batches(get_batches)


I am working on the file "Project 3/dlnd_tv_script_generation.ipynb".
The first task is:
Build the Graph
Build the graph using the neural network you implemented.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
from tensorflow.contrib import seq2seq

train_graph = tf.Graph()
with train_graph.as_default():
    vocab_size = len(int_to_vocab)
    input_text, targets, lr = get_inputs()
    input_data_shape = tf.shape(input_text)
    cell, initial_state = get_init_cell(input_data_shape[0], rnn_size)
    logits, final_state = build_nn(cell, rnn_size, input_text, vocab_size)

    # Probabilities for generating words
    probs = tf.nn.softmax(logits, name='probs')

    # Loss function
    cost = seq2seq.sequence_loss(
        logits,
        targets,
        tf.ones([input_data_shape[0], input_data_shape[1]]))

    # Optimizer
    optimizer = tf.train.AdamOptimizer(lr)

    # Gradient Clipping
    gradients = optimizer.compute_gradients(cost)
    capped_gradients = [(tf.clip_by_value(grad, -1., 1.), var) for grad, var in gradients]
    train_op = optimizer.apply_gradients(capped_gradients)


I am working on the file "Project 3/dlnd_tv_script_generation.ipynb".
The first task is:
Build the Neural Network
Apply the functions you implemented above to:
- Apply embedding to input_data using your get_embed(input_data, vocab_size, embed_dim) function.
- Build RNN using cell and your build_rnn(cell, inputs) function.
- Apply a fully connected layer with a linear activation and vocab_size as the number of outputs.
Return the logits and final state in the following tuple (Logits, FinalState)
Can you write Python code for it?

embedding_size = 300

def build_nn(cell, rnn_size, input_data, vocab_size):
    """
    Build part of the neural network
    :param cell: RNN cell
    :param rnn_size: Size of rnns
    :param input_data: Input data
    :param vocab_size: Vocabulary size
    :return: Tuple (Logits, FinalState)
    """
    embedding = get_embed(input_data, vocab_size, embedding_size)
    outputs, final_state = build_rnn(cell, embedding)
    fully_connected = tf.contrib.layers.fully_connected(outputs, vocab_size,\
                                                              activation_fn=None,\
                                                              weights_initializer = tf.truncated_normal_initializer(mean=0, stddev = 0.01),\
                                                              biases_initializer = tf.truncated_normal_initializer(mean=0, stddev = 0.01))
    return (fully_connected, final_state)

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_build_nn(build_nn)


I am working on the file "Project 3/dlnd_tv_script_generation.ipynb".
The first task is:
Build the Neural Network
You'll build the components necessary to build a RNN by implementing the following functions below:
- get_inputs
- get_init_cell
- get_embed
- build_rnn
- build_nn
- get_batches
Check the Version of TensorFlow and Access to GPU
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
from distutils.version import LooseVersion
import warnings
import tensorflow as tf

# Check TensorFlow Version
assert LooseVersion(tf.__version__) >= LooseVersion('1.0'), 'Please use TensorFlow version 1.0 or newer'
print('TensorFlow Version: {}'.format(tf.__version__))

# Check for a GPU
if not tf.test.gpu_device_name():
    warnings.warn('No GPU found. Please use a GPU to train your neural network.')
else:
    print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))


I am working on the file "Project 3/dlnd_tv_script_generation.ipynb".
The first task is:
Check Point
This is your first checkpoint. If you ever decide to come back to this notebook or have to restart the notebook, you can start from here. The preprocessed data has been saved to disk.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import helper
import numpy as np
import problem_unittests as tests

int_text, vocab_to_int, int_to_vocab, token_dict = helper.load_preprocess()


I am working on the file "Project 3/dlnd_tv_script_generation.ipynb".
The first task is:
Checkpoint
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import tensorflow as tf
import numpy as np
import helper
import problem_unittests as tests

_, vocab_to_int, int_to_vocab, token_dict = helper.load_preprocess()
seq_length, load_dir = helper.load_params()


I am working on the file "Project 3/dlnd_tv_script_generation.ipynb".
The first task is:
Implement Generate Functions
Get Tensors
Get tensors from loaded_graph using the function get_tensor_by_name().  Get the tensors using the following names:
- "input:0"
- "initial_state:0"
- "final_state:0"
- "probs:0"
Return the tensors in the following tuple (InputTensor, InitialStateTensor, FinalStateTensor, ProbsTensor)
Can you write Python code for it?

def get_tensors(loaded_graph):
    """
    Get input, initial state, final state, and probabilities tensor from <loaded_graph>
    :param loaded_graph: TensorFlow graph loaded from file
    :return: Tuple (InputTensor, InitialStateTensor, FinalStateTensor, ProbsTensor)
    """
    return loaded_graph.get_tensor_by_name("input:0"), \
           loaded_graph.get_tensor_by_name("initial_state:0"), \
           loaded_graph.get_tensor_by_name("final_state:0"), \
           loaded_graph.get_tensor_by_name("probs:0")

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_get_tensors(get_tensors)


I am working on the file "Project 3/dlnd_tv_script_generation.ipynb".
The first task is:
Implement Preprocessing Functions
The first thing to do to any dataset is preprocessing.  Implement the following preprocessing functions below:
- Lookup Table
- Tokenize Punctuation
Lookup Table
To create a word embedding, you first need to transform the words to ids.  In this function, create two dictionaries:
- Dictionary to go from the words to an id, we'll call vocab_to_int
- Dictionary to go from the id to word, we'll call int_to_vocab
Return these dictionaries in the following tuple (vocab_to_int, int_to_vocab)
Can you write Python code for it?

import numpy as np
import problem_unittests as tests

def create_lookup_tables(text):
    """
    Create lookup tables for vocabulary
    :param text: The text of tv scripts split into words
    :return: A tuple of dicts (vocab_to_int, int_to_vocab)
    """
    text_sorted = sorted(list(set(text)))
    return (dict((word, i) for i, word in enumerate(text_sorted)), \
           dict((i, word) for i, word in enumerate(text_sorted)))

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_create_lookup_tables(create_lookup_tables)


I am working on the file "Project 3/dlnd_tv_script_generation.ipynb".
The first task is:
Neural Network Training
Hyperparameters
Tune the following parameters:

Set num_epochs to the number of epochs.
Set batch_size to the batch size.
Set rnn_size to the size of the RNNs.
Set seq_length to the length of sequence.
Set learning_rate to the learning rate.
Set show_every_n_batches to the number of batches the neural network should print progress.
Can you write Python code for it?

# Number of Epochs
num_epochs = 100
# Batch Size
batch_size = 200
# RNN Size
rnn_size = 512
# Sequence Length
seq_length = 20
# Learning Rate
learning_rate = 0.005
# Show stats for every n number of batches
show_every_n_batches = 5

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
save_dir = './save'


I am working on the file "Project 3/dlnd_tv_script_generation.ipynb".
The first task is:
Preprocess all the data and save it
Running the code cell below will preprocess all the data and save it to file.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# Preprocess Training, Validation, and Testing Data
helper.preprocess_and_save_data(data_dir, token_lookup, create_lookup_tables)


I am working on the file "Project 3/dlnd_tv_script_generation.ipynb".
The first task is:
Save Parameters
Save seq_length and save_dir for generating a new TV script.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# Save parameters for checkpoint
helper.save_params((seq_length, save_dir))


I am working on the file "Project 3/dlnd_tv_script_generation.ipynb".
The first task is:
Tokenize Punctuation
We'll be splitting the script into a word array using spaces as delimiters.  However, punctuations like periods and exclamation marks make it hard for the neural network to distinguish between the word "bye" and "bye!".
Implement the function token_lookup to return a dict that will be used to tokenize symbols like "!" into "||Exclamation_Mark||".  Create a dictionary for the following symbols where the symbol is the key and value is the token:
- Period ( . )
- Comma ( , )
- Quotation Mark ( " )
- Semicolon ( ; )
- Exclamation mark ( ! )
- Question mark ( ? )
- Left Parentheses ( ( )
- Right Parentheses ( ) )
- Dash ( -- )
- Return ( \n )
This dictionary will be used to token the symbols and add the delimiter (space) around it.  This separates the symbols as it's own word, making it easier for the neural network to predict on the next word. Make sure you don't use a token that could be confused as a word. Instead of using the token "dash", try using something like "||dash||".
Can you write Python code for it?

def token_lookup():
    """
    Generate a dict to turn punctuation into a token.
    :return: Tokenize dictionary where the key is the punctuation and the value is the token
    """
    return {
        "." : "||Period||",
        "," : "||Comma||",
        "\"" : "||QuotationMark||",
        ";" : "||Semicolon||",
        "!" : "||ExclamationMark||",
        "?" : "||QuestionMark||",
        "(" : "||LeftParentheses||",
        ")" : "||RightParentheses||",
        "--" : "||Dash||",
        "\n" : "||Return||"
    }

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_tokenize(token_lookup)


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Build the Decoding Layer
Implement decoding_layer() to create a Decoder RNN layer.

Create RNN cell for decoding using rnn_size and num_layers.
Create the output fuction using lambda to transform it's input, logits, to class logits.
Use the your decoding_layer_train(encoder_state, dec_cell, dec_embed_input, sequence_length, decoding_scope, output_fn, keep_prob) function to get the training logits.
Use your decoding_layer_infer(encoder_state, dec_cell, dec_embeddings, start_of_sequence_id, end_of_sequence_id, maximum_length, vocab_size, decoding_scope, output_fn, keep_prob) function to get the inference logits.

Note: You'll need to use tf.variable_scope to share variables between training and inference.
Can you write Python code for it?

def decoding_layer(dec_embed_input, dec_embeddings, encoder_state, vocab_size, sequence_length, rnn_size,
                   num_layers, target_vocab_to_int, keep_prob):
    """
    Create decoding layer
    :param dec_embed_input: Decoder embedded input
    :param dec_embeddings: Decoder embeddings
    :param encoder_state: The encoded state
    :param vocab_size: Size of vocabulary
    :param sequence_length: Sequence Length
    :param rnn_size: RNN Size
    :param num_layers: Number of layers
    :param target_vocab_to_int: Dictionary to go from the target words to an id
    :param keep_prob: Dropout keep probability
    :return: Tuple of (Training Logits, Inference Logits)
    """
    lstm_cell = tf.contrib.rnn.BasicLSTMCell(rnn_size)
    rnn_cell = tf.contrib.rnn.MultiRNNCell([lstm_cell] * num_layers)
    rnn_cell = tf.contrib.rnn.DropoutWrapper(rnn_cell, output_keep_prob = keep_prob)

    output_fn = lambda x: tf.contrib.layers.fully_connected(x, vocab_size, None, scope=decoding_scope)

    with tf.variable_scope('decoding_scope') as decoding_scope:
        training_logits = decoding_layer_train(encoder_state, rnn_cell, dec_embed_input, \
                                                         sequence_length, decoding_scope, output_fn, \
                                                         keep_prob)
    with tf.variable_scope('decoding_scope', reuse=True) as decoding_scope:
        decoding_scope.reuse_variables()
        inference_logits = decoding_layer_infer(encoder_state, rnn_cell, dec_embeddings, \
                                                          target_vocab_to_int['<GO>'], target_vocab_to_int['<EOS>'], \
                                                          sequence_length, vocab_size, decoding_scope, \
                                                          output_fn, keep_prob)
    return training_logits, inference_logits

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_decoding_layer(decoding_layer)


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Build the Neural Network
Apply the functions you implemented above to:

Apply embedding to the input data for the encoder.
Encode the input using your encoding_layer(rnn_inputs, rnn_size, num_layers, keep_prob).
Process target data using your process_encoding_input(target_data, target_vocab_to_int, batch_size) function.
Apply embedding to the target data for the decoder.
Decode the encoded input using your decoding_layer(dec_embed_input, dec_embeddings, encoder_state, vocab_size, sequence_length, rnn_size, num_layers, target_vocab_to_int, keep_prob).
Can you write Python code for it?

def seq2seq_model(input_data, target_data, keep_prob, batch_size, sequence_length, source_vocab_size, target_vocab_size,
                  enc_embedding_size, dec_embedding_size, rnn_size, num_layers, target_vocab_to_int):
    """
    Build the Sequence-to-Sequence part of the neural network
    :param input_data: Input placeholder
    :param target_data: Target placeholder
    :param keep_prob: Dropout keep probability placeholder
    :param batch_size: Batch Size
    :param sequence_length: Sequence Length
    :param source_vocab_size: Source vocabulary size
    :param target_vocab_size: Target vocabulary size
    :param enc_embedding_size: Decoder embedding size
    :param dec_embedding_size: Encoder embedding size
    :param rnn_size: RNN Size
    :param num_layers: Number of layers
    :param target_vocab_to_int: Dictionary to go from the target words to an id
    :return: Tuple of (Training Logits, Inference Logits)
    """
    enc_embeddings = tf.Variable(tf.random_uniform((source_vocab_size, enc_embedding_size)))
    enc_embed_input = tf.nn.embedding_lookup(enc_embeddings, input_data)
    enc_input = encoding_layer(enc_embed_input, rnn_size, num_layers, keep_prob)

    dec_input = process_decoding_input(target_data, target_vocab_to_int, batch_size)
    dec_embeddings = tf.Variable(tf.random_uniform((target_vocab_size, dec_embedding_size)))
    dec_embed_input = tf.nn.embedding_lookup(dec_embeddings, dec_input)

    return decoding_layer(dec_embed_input, \
                          dec_embeddings, \
                          enc_input, \
                          target_vocab_size, \
                          sequence_length, \
                          rnn_size, \
                          num_layers, \
                          target_vocab_to_int, \
                          keep_prob)

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_seq2seq_model(seq2seq_model)


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Build the Neural Network
You'll build the components necessary to build a Sequence-to-Sequence model by implementing the following functions below:
- model_inputs
- process_encoding_input
- encoding_layer
- decoding_layer_train
- decoding_layer_infer
- decoding_layer
- seq2seq_model
Input
Implement the model_inputs() function to create TF Placeholders for the Neural Network. It should create the following placeholders:

Input text placeholder named "input" using the TF Placeholder name parameter with rank 2.
Targets placeholder with rank 2.
Learning rate placeholder with rank 0.
Keep probability placeholder named "keep_prob" using the TF Placeholder name parameter with rank 0.

Return the placeholders in the following the tuple (Input, Targets, Learing Rate, Keep Probability)
Can you write Python code for it?

def model_inputs():
    """
    Create TF Placeholders for input, targets, and learning rate.
    :return: Tuple (input, targets, learning rate, keep probability)
    """
    return tf.placeholder(tf.int32, [None, None], name='input'),\
           tf.placeholder(tf.int32, [None, None], name='targets'),\
           tf.placeholder(tf.float32, name='learning_rate'),\
           tf.placeholder(tf.float32, name='keep_prob'),\

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_inputs(model_inputs)


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Check the Version of TensorFlow and Access to GPU
This will check to make sure you have the correct version of TensorFlow and access to a GPU
Can you write Python code for it?


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Checkpoint
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import tensorflow as tf
import numpy as np
import helper
import problem_unittests as tests

_, (source_vocab_to_int, target_vocab_to_int), (source_int_to_vocab, target_int_to_vocab) = helper.load_preprocess()
load_path = helper.load_params()


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Decoding - Inference
Create inference logits using tf.contrib.seq2seq.simple_decoder_fn_inference() and tf.contrib.seq2seq.dynamic_rnn_decoder().
Can you write Python code for it?

def decoding_layer_infer(encoder_state, dec_cell, dec_embeddings, start_of_sequence_id, end_of_sequence_id,
                         maximum_length, vocab_size, decoding_scope, output_fn, keep_prob):
    """
    Create a decoding layer for inference
    :param encoder_state: Encoder state
    :param dec_cell: Decoder RNN Cell
    :param dec_embeddings: Decoder embeddings
    :param start_of_sequence_id: GO ID
    :param end_of_sequence_id: EOS Id
    :param maximum_length: Maximum length of
    :param vocab_size: Size of vocabulary
    :param decoding_scope: TensorFlow Variable Scope for decoding
    :param output_fn: Function to apply the output layer
    :param keep_prob: Dropout keep probability
    :return: Inference Logits
    """
    infer_decoder_fn = tf.contrib.seq2seq.simple_decoder_fn_inference(output_fn = output_fn, \
                                                                      encoder_state = encoder_state, \
                                                                      embeddings = dec_embeddings, \
                                                                      start_of_sequence_id = start_of_sequence_id, \
                                                                      end_of_sequence_id = end_of_sequence_id, \
                                                                      maximum_length = maximum_length, \
                                                                      num_decoder_symbols = vocab_size)
    inference_logits, _, a_ = tf.contrib.seq2seq.dynamic_rnn_decoder(cell = dec_cell, \
                                                                    decoder_fn = infer_decoder_fn, \
                                                                    scope=decoding_scope)
    return inference_logits

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_decoding_layer_infer(decoding_layer_infer)


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Encoding
Implement encoding_layer() to create a Encoder RNN layer using tf.nn.dynamic_rnn().
Can you write Python code for it?

def encoding_layer(rnn_inputs, rnn_size, num_layers, keep_prob):
    """
    Create encoding layer
    :param rnn_inputs: Inputs for the RNN
    :param rnn_size: RNN Size
    :param num_layers: Number of layers
    :param keep_prob: Dropout keep probability
    :return: RNN state
    """
    lstm_cell = tf.contrib.rnn.BasicLSTMCell(rnn_size)
    rnn_cell = tf.contrib.rnn.MultiRNNCell([lstm_cell] * num_layers)
    rnn_cell = tf.contrib.rnn.DropoutWrapper(rnn_cell, output_keep_prob = keep_prob)
    rnn_output, rnn_state = tf.nn.dynamic_rnn(cell = rnn_cell, inputs = rnn_inputs, dtype=tf.float32)
    return rnn_state

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_encoding_layer(encoding_layer)


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Explore the Data
Play around with view_sentence_range to view different parts of the data.
Can you write Python code for it?

view_sentence_range = (0, 10)

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import numpy as np

print('Dataset Stats')
print('Roughly the number of unique words: {}'.format(len({word: None for word in source_text.split()})))

sentences = source_text.split('\n')
word_counts = [len(sentence.split()) for sentence in sentences]
print('Number of sentences: {}'.format(len(sentences)))
print('Average number of words in a sentence: {}'.format(np.average(word_counts)))

print()
print('English sentences {} to {}:'.format(*view_sentence_range))
print('\n'.join(source_text.split('\n')[view_sentence_range[0]:view_sentence_range[1]]))
print()
print('French sentences {} to {}:'.format(*view_sentence_range))
print('\n'.join(target_text.split('\n')[view_sentence_range[0]:view_sentence_range[1]]))


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Implement Preprocessing Function
Text to Word Ids
As you did with other RNNs, you must turn the text into a number so the computer can understand it. In the function text_to_ids(), you'll turn source_text and target_text from words to ids.  However, you need to add the <EOS> word id at the end of each sentence from target_text.  This will help the neural network predict when the sentence should end.
You can get the <EOS> word id by doing:
python
target_vocab_to_int['<EOS>']
You can get other word ids using source_vocab_to_int and target_vocab_to_int.
Can you write Python code for it?

def text_to_ids(source_text, target_text, source_vocab_to_int, target_vocab_to_int):
    """
    Convert source and target text to proper word ids
    :param source_text: String that contains all the source text.
    :param target_text: String that contains all the target text.
    :param source_vocab_to_int: Dictionary to go from the source words to an id
    :param target_vocab_to_int: Dictionary to go from the target words to an id
    :return: A tuple of lists (source_id_text, target_id_text)
    """
    return ([[source_vocab_to_int[word] for word in text.split()] for text in source_text.split('\n')],\
            [[target_vocab_to_int[word] for word in text.split()] + \
             [target_vocab_to_int['<EOS>']] for text in target_text.split('\n')])

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_text_to_ids(text_to_ids)


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Neural Network Training
Hyperparameters
Tune the following parameters:

Set epochs to the number of epochs.
Set batch_size to the batch size.
Set rnn_size to the size of the RNNs.
Set num_layers to the number of layers.
Set encoding_embedding_size to the size of the embedding for the encoder.
Set decoding_embedding_size to the size of the embedding for the decoder.
Set learning_rate to the learning rate.
Set keep_probability to the Dropout keep probability
Can you write Python code for it?

# Number of Epochs
epochs = 10
# Batch Size
batch_size = 512
# RNN Size
rnn_size = 256
# Number of Layers
num_layers = 1
# Embedding Size
encoding_embedding_size = 128
decoding_embedding_size = 128
# Learning Rate
learning_rate = 0.01
# Dropout Keep Probability
keep_probability = 0.8
# Show stats for every n number of batches
show_every_n_batches = 10
# For graph
times, train_accs, valid_accs, losses = [], [], [], []
counter = 0


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Preprocess all the data and save it
Running the code cell below will preprocess all the data and save it to file.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
helper.preprocess_and_save_data(source_path, target_path, text_to_ids)


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Save Parameters
Save the batch_size and save_path parameters for inference.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# Save parameters for checkpoint
helper.save_params(save_path)


I am working on the file "Project 4/dlnd_language_translation.ipynb".
The first task is:
Sentence to Sequence
To feed a sentence into the model for translation, you first need to preprocess it.  Implement the function sentence_to_seq() to preprocess new sentences.

Convert the sentence to lowercase
Convert words into ids using vocab_to_int
Convert words not in the vocabulary, to the <UNK> word id.
Can you write Python code for it?

def sentence_to_seq(sentence, vocab_to_int):
    """
    Convert a sentence to a sequence of ids
    :param sentence: String
    :param vocab_to_int: Dictionary to go from the words to an id
    :return: List of word ids
    """
    return [vocab_to_int[word] if word in vocab_to_int.keys() \
                               else vocab_to_int['<UNK>'] \
            for word in sentence.lower().split()]
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_sentence_to_seq(sentence_to_seq)


I am working on the file "Project 5- Machine Learning/Enron-Report.ipynb".
The first task is:
AdaBoostClassifier
Can you write Python code for it?

Pipeline(steps=[("Scaler", StandardScaler()),
                ("K_best", None),
                ("PCA", None),
                ("clf", AdaBoostClassifier(n_estimators=50,
                                           learning_rate=0.25))
               ]
        )


I am working on the file "Project 5- Machine Learning/Enron-Report.ipynb".
The first task is:
DecisionTreeClassifier
Can you write Python code for it?

Pipeline(steps=[("Scaler", StandardScaler()),
                 ("K_best", SelectKBest(k=4)),
                 ("PCA", None),
                 ("clf", DecisionTreeClassifier(min_samples_split=3,
                                                criterion='entropy',
                                                min_samples_leaf=2))
               ]
        )


I am working on the file "Project 5- Machine Learning/Enron-Report.ipynb".
The first task is:
Gaussian Naive Bayes
Can you write Python code for it?

Pipeline(steps=[("Scaler", None),
                 ("K_best", SelectKBest(k=13)),
                 ("PCA", PCA(n_components=10)),
                 ("clf", GaussianNB())
               ]
        )


I am working on the file "Project 5- Machine Learning/Enron-Report.ipynb".
The first task is:
RandomForestClassifier
Can you write Python code for it?

Pipeline(steps=[("Scaler", None),
                 ("K_best", SelectKBest(k=10)),
                 ("PCA", PCA(n_components=5)),
                 ("clf", RandomForestClassifier(n_estimators=20,
                                                criterion='entropy',
                                                max_depth=3,
                                                random_state=40))
               ]
        )


I am working on the file "Project 5/dlnd_face_generation.ipynb".
The first task is:
CelebA
Run your GANs on CelebA.  It will take around 20 minutes on the average GPU to run one epoch.  You can run the whole epoch or stop when it starts to generate realistic faces.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
epochs = 3

celeba_dataset = helper.Dataset('celeba', glob(os.path.join(data_dir, 'img_align_celeba/*.jpg')))
with tf.Graph().as_default():
    train(epochs, batch_size, z_dim, learning_rate, beta1, celeba_dataset.get_batches,
          celeba_dataset.shape, celeba_dataset.image_mode)


I am working on the file "Project 5/dlnd_face_generation.ipynb".
The first task is:
CelebA
The CelebFaces Attributes Dataset (CelebA) dataset contains over 200,000 celebrity images with annotations.  Since you're going to be generating faces, you won't need the annotations.  You can view the first number of examples by changing show_n_images.
Can you write Python code for it?

show_n_images = 25

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
mnist_images = helper.get_batch(glob(os.path.join(data_dir, 'img_align_celeba/*.jpg'))[:show_n_images], 28, 28, 'RGB')
pyplot.imshow(helper.images_square_grid(mnist_images, 'RGB'))


I am working on the file "Project 5/dlnd_face_generation.ipynb".
The first task is:
Generator
Implement generator to generate an image using z. This function should be able to reuse the variabes in the neural network.  Use tf.variable_scope with a scope name of "generator" to allow the variables to be reused. The function should return the generated 28 x 28 x out_channel_dim images.
Can you write Python code for it?

def generator(z, out_channel_dim, is_train=True, alpha = 0.1):
    """
    Create the generator network
    :param z: Input z
    :param out_channel_dim: The number of channels in the output image
    :param is_train: Boolean if generator is being used for training
    :return: The tensor output of the generator
    """
    with tf.variable_scope('generator', reuse=not is_train):
        # First fully connected layer
        x1 = tf.layers.dense(z, 7*7*512)

        x1 = tf.reshape(x1, (-1, 7, 7, 512))
        x1 = tf.layers.batch_normalization(x1, training=is_train)
        x1 = tf.maximum(alpha * x1, x1)

        x1 = tf.reshape(x1, (-1, 7, 7, 256))
        x1 = tf.layers.batch_normalization(x1, training=is_train)
        x1 = tf.maximum(alpha * x1, x1)

        x2 = tf.layers.conv2d_transpose(x1, 128, 5, strides=1, padding='same')
        x2 = tf.layers.batch_normalization(x2, training=is_train)
        x2 = tf.maximum(alpha * x2, x2)

        x3 = tf.layers.conv2d_transpose(x2, 64, 5, strides=2, padding='same')
        x3 = tf.layers.batch_normalization(x3, training=is_train)
        x3 = tf.maximum(alpha * x3, x3)

        # Output layer
        logits = tf.layers.conv2d_transpose(x3, out_channel_dim, 5, strides=2, padding='same')
        return tf.tanh(logits)

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_generator(generator, tf)


I am working on the file "Project 5/dlnd_face_generation.ipynb".
The first task is:
Loss
Implement model_loss to build the GANs for training and calculate the loss.  The function should return a tuple of (discriminator loss, generator loss).  Use the following functions you implemented:
- discriminator(images, reuse=False)
- generator(z, out_channel_dim, is_train=True)
Can you write Python code for it?

def model_loss(input_real, input_z, out_channel_dim):
    """
    Get the loss for the discriminator and generator
    :param input_real: Images from the real dataset
    :param input_z: Z input
    :param out_channel_dim: The number of channels in the output image
    :return: A tuple of (discriminator loss, generator loss)
    """
    gen = generator(input_z, out_channel_dim)
    disc_real, disc_logits_real = discriminator(input_real)
    disc_fake, disc_logits_fake = discriminator(gen, reuse=True)

    disc_loss_real = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=disc_logits_real, \
                                                labels=tf.ones_like(disc_real)*0.9))
    disc_loss_fake = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=disc_logits_fake, \
                                                labels=tf.zeros_like(disc_fake)))
    gen_loss       = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=disc_logits_fake, \
                                                labels=tf.ones_like(disc_fake)))

    disc_loss = disc_loss_real + disc_loss_fake
    return disc_loss, gen_loss

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_loss(model_loss)


I am working on the file "Project 5/dlnd_face_generation.ipynb".
The first task is:
Neural Network Training
Show Output
Use this function to show the current output of the generator during training. It will help you determine how well the GANs is training.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import numpy as np

def show_generator_output(sess, n_images, input_z, out_channel_dim, image_mode):
    """
    Show example output for the generator
    :param sess: TensorFlow session
    :param n_images: Number of Images to display
    :param input_z: Input Z Tensor
    :param out_channel_dim: The number of channels in the output image
    :param image_mode: The mode to use for images ("RGB" or "L")
    """
    cmap = None if image_mode == 'RGB' else 'gray'
    z_dim = input_z.get_shape().as_list()[-1]
    example_z = np.random.uniform(-1, 1, size=[n_images, z_dim])

    samples = sess.run(
        generator(input_z, out_channel_dim, False),
        feed_dict={input_z: example_z})

    images_grid = helper.images_square_grid(samples, image_mode)
    pyplot.imshow(images_grid, cmap=cmap)
    pyplot.show()


I am working on the file "Project 5/dlnd_face_generation.ipynb".
The first task is:
Optimization
Implement model_opt to create the optimization operations for the GANs. Use tf.trainable_variables to get all the trainable variables.  Filter the variables with names that are in the discriminator and generator scope names.  The function should return a tuple of (discriminator training operation, generator training operation).
Can you write Python code for it?

def model_opt(d_loss, g_loss, learning_rate, beta1):
    """
    Get optimization operations
    :param d_loss: Discriminator loss Tensor
    :param g_loss: Generator loss Tensor
    :param learning_rate: Learning Rate Placeholder
    :param beta1: The exponential decay rate for the 1st moment in the optimizer
    :return: A tuple of (discriminator training operation, generator training operation)
    """
    trainable_variables = tf.trainable_variables()
    d_vars = [var for var in trainable_variables if var.name.startswith('discriminator')]
    g_vars = [var for var in trainable_variables if var.name.startswith('generator')]

    with tf.control_dependencies(tf.get_collection(tf.GraphKeys.UPDATE_OPS)):
        return tf.train.AdamOptimizer(learning_rate, beta1=beta1).minimize(d_loss, var_list=d_vars), \
               tf.train.AdamOptimizer(learning_rate, beta1=beta1).minimize(g_loss, var_list=g_vars)

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_opt(model_opt, tf)


I am working on the file "Project 5/dlnd_face_generation.ipynb".
The first task is:
Preprocess the Data
Since the project's main focus is on building the GANs, we'll preprocess the data for you.  The values of the MNIST and CelebA dataset will be in the range of -0.5 to 0.5 of 28x28 dimensional images.  The CelebA images will be cropped to remove parts of the image that don't include a face, then resized down to 28x28.
The MNIST images are black and white images with a single [color channel](https://en.wikipedia.org/wiki/Channel_(digital_image%29) while the CelebA images have [3 color channels (RGB color channel)](https://en.wikipedia.org/wiki/Channel_(digital_image%29#RGB_Images).
Build the Neural Network
You'll build the components necessary to build a GANs by implementing the following functions below:
- model_inputs
- discriminator
- generator
- model_loss
- model_opt
- train
Check the Version of TensorFlow and Access to GPU
This will check to make sure you have the correct version of TensorFlow and access to a GPU
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
from distutils.version import LooseVersion
import warnings
import tensorflow as tf

# Check TensorFlow Version
assert LooseVersion(tf.__version__) >= LooseVersion('1.0'), 'Please use TensorFlow version 1.0 or newer.  You are using {}'.format(tf.__version__)
print('TensorFlow Version: {}'.format(tf.__version__))

# Check for a GPU
if not tf.test.gpu_device_name():
    warnings.warn('No GPU found. Please use a GPU to train your neural network.')
else:
    print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))


I am working on the file "Project 5/dlnd_face_generation.ipynb".
The first task is:
Train
Implement train to build and train the GANs.  Use the following functions you implemented:
- model_inputs(image_width, image_height, image_channels, z_dim)
- model_loss(input_real, input_z, out_channel_dim)
- model_opt(d_loss, g_loss, learning_rate, beta1)
Use the show_generator_output to show generator output while you train. Running show_generator_output for every batch will drastically increase training time and increase the size of the notebook.  It's recommended to print the generator output every 100 batches.
Can you write Python code for it?

def plot_losses_graph(steps, gen_loss, disc_loss):
    pl.ylim(min(min(gen_loss), min(disc_loss)), 2)
    pl.xlim(min(steps), max(steps))

    pl.plot(steps, gen_loss, label = 'Generator Loss', color = 'green')
    pl.plot(steps, disc_loss, label = 'Discriminator Loss', color = 'red')
    pl.legend(loc='upper right')
    pl.xlabel('Steps')

    display.clear_output(wait=True)
    display.display(pl.gcf())
    pl.gcf().clear()
    time.sleep(0.1)

def train(epoch_count, batch_size, z_dim, learning_rate, beta1, get_batches, data_shape, data_image_mode):
    """
    Train the GAN
    :param epoch_count: Number of epochs
    :param batch_size: Batch Size
    :param z_dim: Z dimension
    :param learning_rate: Learning Rate
    :param beta1: The exponential decay rate for the 1st moment in the optimizer
    :param get_batches: Function to get batches
    :param data_shape: Shape of the data
    :param data_image_mode: The image mode to use for images ("RGB" or "L")
    """
    image_channels = 3 if data_image_mode == 'RGB' else 1
    input_real, input_z, _ = model_inputs(data_shape[1], data_shape[2], data_shape[3], z_dim)

    d_loss, g_loss = model_loss(input_real, input_z, data_shape[3])

    d_opt, g_opt = model_opt(d_loss, g_loss, learning_rate, beta1)

    steps = 0
    samples, losses = [], []
    steps_list, gen_loss, disc_loss = [], [], []

    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        for epoch_i in range(epoch_count):
            for batch_images in get_batches(batch_size):
                batch_images = batch_images * 2
                batch_z = np.random.uniform(-1, 1, size=(batch_size, z_dim))
                _ = sess.run(d_opt, feed_dict={input_real: batch_images, input_z: batch_z})
                _ = sess.run(g_opt, feed_dict={input_real: batch_images, input_z: batch_z})
                _ = sess.run(g_opt, feed_dict={input_real: batch_images, input_z: batch_z})

                steps += 1

   ...


I am working on the file "Project Wrangling OpenStree Map project.ipynb".
The first task is:
5)  First analysis from Database
Once data imported into SQL database, we can analyse the data.
For that, I develop a Python function that will display Top 10 for users, city, postcode and street types. That will enable me to see if there is some obvious issues with data.
Can you write Python code for it?

def display(result):
    j=1
    for i in result:
        #print i[0],":",i[1]
        print '({}){}:\t{}'.format(j,i[0],i[1])
        j+=1

def display_total(result):
    for i in result:
        print '\n\tTotal:\t{}'.format(i[0])

def top_sql():
    conn = sqlite3.connect('OSM.db')
    conn.text_factory = str
    c = conn.cursor()

    ### Show top 15 users ###
    result=c.execute('''SELECT user,count(user) from (select user from nodes UNION ALL select user from ways) group by user order by count(user) desc limit 15
    ;''')
    print "\nTop 10 users:"
    display(result)

    result=c.execute('''SELECT count(user) from (select user from nodes UNION ALL select user from ways)
    ;''')
    display_total(result)

    ### Show top 10 postcode ###
    result=c.execute('''SELECT value,count(value) from (select * from nodes_tags UNION ALL select * from ways_tags) where key='postcode' group by value order by count(value) desc limit 10
    ;''')
    print "\nTop 10 postcode:"
    display(result)

    result=c.execute('''SELECT count(value) from (select * from nodes_tags UNION ALL select * from ways_tags) where key='postcode'
    ;''')
    display_total(result)

    ### Show top 10 city ###
    result=c.execute('''SELECT value,count(value) from (select * from nodes_tags UNION ALL select * from ways_tags) where key='city' group by value order by count(value) desc limit 10
    ;''')
    print "\nTop 10 city:"
    display(result)

    result=c.execute('''SELECT count(value) from (select * from nodes_tags UNION ALL select * from ways_tags) where key='city'
    ;''')
    display_total(result)

    ### Show top 10 street type ###
    result=c.execute('''SELECT type,count(type) from (SELECT SUBSTR(value,0,INSTR( value , " " )) as type from(select * from nodes_tags UNION ALL select * from ways_tags) where key='street' limit 100) group by type;''')
    print "\nTop 10 street type:"
    display(result)

    result=c.execute('''SELECT count(type) from (SELECT SUBSTR(value,0,INSTR( value , "...


I am working on the file "Project Wrangling OpenStree Map project.ipynb".
The first task is:

Analysis:

On top users, except 'Chouloute' extra number of inputs (45% of total), nothing to report

On top postcodes, it is interesting to observe that not only Colomiers postal code is represented (31770), Tournefeuille (31170) and Toulouse (31300&31000) are numberous as well. That could be explained by the fact Tournefeuille and Colomiers are neighbours of Colomiers. Same for Pibrac (31820) but with less quantity. 31776 corresponds to Colomiers with a different format. I will analyse afterwards.

On top city, same observation as for postal code, Colomiers represents half of references. As I noticed in the first analysis from the sample, Colomiers with Upper and Lower cases that make two entries into Data Base. I propose to clean it afterwards.

On top street types, we can see there are only 6 entries. There is one issue related to sensitive case (Place and place). I will fix it in next section

On top sources, Cadastre (ie. the register of the real estate or real property's metes-and-bounds of France) is the top listed that is normal because it is its primary function. Nothing else interesting to notice

Synthesis:

No big issues on top parameters analysis, the dataset seems to be well built

4)  Cleaning of Dataset

In this chapter, I propose to clean the data in the csv creation step. The advantage is I have just to adapt my code. Moreover, as the OSM dataset is pretty clean I will have only a few correction to do.

I create a new function named 'csv_data_cleaning' that will get a csv file produced by python code, apply some modifications and output a new csv file I will import into a new SQLite DataBase (in fact, I drop all the existing...
Can you write Python code for it?

def name_cleaning(city_name):
    return city_name.capitalize()

def csv_data_cleaning(csv_file_source,csv_file_dest):
    print 'Clean data from {} to {}...'.format(csv_file_source,csv_file_dest)
    with open(csv_file_dest,'wb') as csvfile_dest:
        with open(csv_file_source,'rb') as csvfile_source:
            reader = csv.DictReader(csvfile_source)
            writer = csv.DictWriter(csvfile_dest, reader.fieldnames)
            writer.writeheader()
            for row_source in reader:
                row_destination=row_source
                for cell in row_destination.keys():
                    if cell=='key':
                        # Cleaning of city names
                        if row_destination['key']=='city':
                            row_destination['value']=name_cleaning(row_destination['value'])
                        # Cleaning of street type
                        elif row_destination['key']=='street':
                            row_destination['value']=name_cleaning(row_destination['value'])
                            street_type=row_destination['value'].split(" ")[0]

                writer.writerow(row_destination)
    print 'Clean data... done'

############## Main ##############

csv_data_cleaning('nodes_tags.csv','nodes_tags_cleaned.csv')
csv_data_cleaning('ways_tags.csv','ways_tags_cleaned.csv')

sql_creation()
import_sql(csv_file_nodes='nodes.csv',csv_file_nodes_tags='nodes_tags_cleaned.csv',csv_file_ways='ways.csv',csv_file_ways_tags='ways_tags_cleaned.csv',csv_file_ways_nodes='ways_nodes.csv')
top_sql()