I am working on the file "Project0/titanic_survival_exploration.ipynb".
The first task is:
当查看和研究了图形化的泰坦尼克号上乘客的数据统计后，请补全下面这段代码中缺失的部分，使得函数可以返回你的预测。
在到达最终的预测模型前请确保记录你尝试过的各种特征和条件。
提示: 您可以用之前 predictions_2 的代码作为开始来修改代码，实现新的预测函数。
Can you write Python code for it?

def predictions_3(data):
    """ 考虑多个特征，准确率至少达到80% """

    predictions = []
    for _, passenger in data.iterrows():

        if (passenger['Sex'] == 'female'):
            if (passenger['Pclass'] > 2 and passenger['Age'] > 40 and passenger['Age'] < 50):
                predictions.append(0)
            else :
                predictions.append(1)
        else:
            if (passenger['Age'] < 10):
                predictions.append(1)
            elif (passenger['Pclass'] < 2):

                if (passenger['Age'] < 40 and passenger['Age'] > 30):
                    predictions.append(1)
                elif (passenger['Parch'] == 0 and passenger['Age'] < 30 and passenger['Age'] > 20):
                    predictions.append(1)
                else:
                    predictions.append(0)

            else :
                predictions.append(0)

    # 返回预测结果
    return pd.Series(predictions)

# 进行预测
predictions = predictions_3(data)


I am working on the file "Project0/titanic_survival_exploration.ipynb".
The first task is:
提示：如果你保存 iPython Notebook，代码运行的输出也将被保存。但是，一旦你重新打开项目，你的工作区将会被重置。请确保每次都从上次离开的地方运行代码来重新生成变量和函数。

最简单的预测
如果我们要预测泰坦尼克号上的乘客是否存活，但是我们又对他们一无所知，那么最好的预测就是船上的人无一幸免。这是因为，我们可以假定当船沉没的时候大多数乘客都遇难了。下面的 predictions_0 函数就预测船上的乘客全部遇难。
Can you write Python code for it?

def predictions_0(data):
    """ 不考虑任何特征，预测所有人都无法生还 """

    predictions = []
    for _, passenger in data.iterrows():

        # 预测 'passenger' 的生还率
        predictions.append(0)

    # 返回预测结果
    return pd.Series(predictions)

# 进行预测
predictions = predictions_0(data)


I am working on the file "Project0/titanic_survival_exploration.ipynb".
The first task is:
考虑一个特征进行预测
我们可以使用 survival_stats 函数来看看 Sex 这一特征对乘客的存活率有多大影响。这个函数定义在名为 titanic_visualizations.py 的 Python 脚本文件中，我们的项目提供了这个文件。传递给函数的前两个参数分别是泰坦尼克号的乘客数据和乘客的 生还结果。第三个参数表明我们会依据哪个特征来绘制图形。
运行下面的代码绘制出依据乘客性别计算存活率的柱形图。
Can you write Python code for it?


I am working on the file "Project0/titanic_survival_exploration.ipynb".
The first task is:
考虑两个特征进行预测
仅仅使用乘客性别（Sex）这一特征，我们预测的准确性就有了明显的提高。现在再看一下使用额外的特征能否更进一步提升我们的预测准确度。例如，综合考虑所有在泰坦尼克号上的男性乘客：我们是否找到这些乘客中的一个子集，他们的存活概率较高。让我们再次使用 survival_stats 函数来看看每位男性乘客的年龄（Age）。这一次，我们将使用第四个参数来限定柱形图中只有男性乘客。
运行下面这段代码，把男性基于年龄的生存结果绘制出来。
Can you write Python code for it?

survival_stats(data, outcomes, 'Age', ["Sex == 'male'"])


I am working on the file "Project0/titanic_survival_exploration.ipynb".
The first task is:
观察泰坦尼克号上乘客存活的数据统计，我们可以发现大部分男性乘客在船沉没的时候都遇难了。相反的，大部分女性乘客都在事故中生还。让我们以此改进先前的预测：如果乘客是男性，那么我们就预测他们遇难；如果乘客是女性，那么我们预测他们在事故中活了下来。
将下面的代码补充完整，让函数可以进行正确预测。
提示：您可以用访问 dictionary（字典）的方法来访问船上乘客的每个特征对应的值。例如， passenger['Sex'] 返回乘客的性别。
Can you write Python code for it?

def predictions_1(data):
    """ 只考虑一个特征，如果是女性则生还 """

    predictions = []
    for _, passenger in data.iterrows():

        if (passenger['Sex'] == 'male'):
            predictions.append(0)
        else:
            predictions.append(1)

    # 返回预测结果
    return pd.Series(predictions)

# 进行预测
predictions = predictions_1(data)

**问题2**：当我们预测船上女性乘客全部存活，而剩下的人全部遇难，那么我们预测的准确率会达到多少？

**回答**: *78.68%*

**提示**：你需要在下面添加一个代码区域，实现代码并运行来计算准确率。

print accuracy_score(outcomes, predictions)


I am working on the file "Project0/titanic_survival_exploration.ipynb".
The first task is:
问题1：对比真实的泰坦尼克号的数据，如果我们做一个所有乘客都没有存活的预测，这个预测的准确率能达到多少？
回答： 61.62%
提示：运行下面的代码来查看预测的准确率。
Can you write Python code for it?


I am working on the file "Project1-First-Neural-Network/DLND Your first neural network.ipynb".
The first task is:
Check out your predictions
Here, use the test data to view how well your network is modeling the data. If something is completely wrong here, make sure each step in your network is implemented correctly.
Can you write Python code for it?

fig, ax = plt.subplots(figsize=(8,4))

mean, std = scaled_features['cnt']
predictions = network.run(test_features).T*std + mean
ax.plot(predictions[0], label='Prediction')
ax.plot((test_targets['cnt']*std + mean).values, label='Data')
ax.set_xlim(right=len(predictions))
ax.legend()

dates = pd.to_datetime(rides.ix[test_data.index]['dteday'])
dates = dates.apply(lambda d: d.strftime('%b %d'))
ax.set_xticks(np.arange(len(dates))[12::24])
_ = ax.set_xticklabels(dates[12::24], rotation=45)


I am working on the file "Project1-First-Neural-Network/DLND Your first neural network.ipynb".
The first task is:
Checking out the data
This dataset has the number of riders for each hour of each day from January 1 2011 to December 31 2012. The number of riders is split between casual and registered, summed up in the cnt column. You can see the first few rows of the data above.
Below is a plot showing the number of bike riders over the first 10 days or so in the data set. (Some days don't have exactly 24 entries in the data set, so it's not exactly 10 days.) You can see the hourly rentals here. This data is pretty complicated! The weekends have lower over all ridership and there are spikes when people are biking to and from work during the week. Looking at the data above, we also have information about temperature, humidity, and windspeed, all of these likely affecting the number of riders. You'll be trying to capture all this with your model.
Can you write Python code for it?


I am working on the file "Project1-First-Neural-Network/DLND Your first neural network.ipynb".
The first task is:
Dummy variables
Here we have some categorical variables like season, weather, month. To include these in our model, we'll need to make binary dummy variables. This is simple to do with Pandas thanks to get_dummies().
Can you write Python code for it?

dummy_fields = ['season', 'weathersit', 'mnth', 'hr', 'weekday']
for each in dummy_fields:
    dummies = pd.get_dummies(rides[each], prefix=each, drop_first=False)
    rides = pd.concat([rides, dummies], axis=1)

fields_to_drop = ['instant', 'dteday', 'season', 'weathersit',
                  'weekday', 'atemp', 'mnth', 'workingday', 'hr']
data = rides.drop(fields_to_drop, axis=1)
data.head()


I am working on the file "Project1-First-Neural-Network/DLND Your first neural network.ipynb".
The first task is:
Load and prepare the data
A critical step in working with neural networks is preparing the data correctly. Variables on different scales make it difficult for the network to efficiently learn the correct weights. Below, we've written the code to load and prepare the data. You'll learn more about this soon!
Can you write Python code for it?

data_path = 'Bike-Sharing-Dataset/hour.csv'

rides = pd.read_csv(data_path)

rides.head()


I am working on the file "Project1-First-Neural-Network/DLND Your first neural network.ipynb".
The first task is:
Scaling target variables
To make training the network easier, we'll standardize each of the continuous variables. That is, we'll shift and scale the variables such that they have zero mean and a standard deviation of 1.
The scaling factors are saved so we can go backwards when we use the network for predictions.
Can you write Python code for it?

quant_features = ['casual', 'registered', 'cnt', 'temp', 'hum', 'windspeed']
# Store scalings in a dictionary so we can convert back later
scaled_features = {}
for each in quant_features:
    mean, std = data[each].mean(), data[each].std()
    scaled_features[each] = [mean, std]
    data.loc[:, each] = (data[each] - mean)/std
    # data.loc[:, each] and data[each] return the same values but  use data[each]=(data[each] - mean)/std will raise
    # SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.


I am working on the file "Project1-First-Neural-Network/DLND Your first neural network.ipynb".
The first task is:
Splitting the data into training, testing, and validation sets
We'll save the data for the last approximately 21 days to use as a test set after we've trained the network. We'll use this set to make predictions and compare them with the actual number of riders.
Can you write Python code for it?

# Save data for approximately the last 21 days
test_data = data[-21*24:]

# Now remove the test data from the data set
data = data[:-21*24]

# Separate the data into features and targets
target_fields = ['cnt', 'casual', 'registered']
features, targets = data.drop(target_fields, axis=1), data[target_fields]
test_features, test_targets = test_data.drop(target_fields, axis=1), test_data[target_fields]

print("Total {} days".format(data.shape[0]/24))


I am working on the file "Project1-First-Neural-Network/DLND Your first neural network.ipynb".
The first task is:
Time to build the network
Below you'll build your network. We've built out the structure and the backwards pass. You'll implement the forward pass through the network. You'll also set the hyperparameters: the learning rate, the number of hidden units, and the number of training passes.
<img src="assets/neural_network.png" width=300px>
The network has two layers, a hidden layer and an output layer. The hidden layer will use the sigmoid function for activations. The output layer has only one node and is used for the regression, the output of the node is the same as the input of the node. (Because you are trying to output (predict) number of rider. Ride numbers are more than 0 and 1. So using sigmoid wouldn't get you the correct predictions )  That is, the activation function is $f(x)=x$. A function that takes the input signal and generates an output signal, but takes into account the threshold, is called an activation function. We work through each layer of our network calculating the outputs for each neuron. All of the outputs from one layer become inputs to the neurons on the next layer. This process is called forward propagation.
We use the weights to propagate signals forward from the input to the output layers in a neural network. We use the weights to also propagate error backwards from the output back into the network to update our weights. This is called backpropagation.

Hint: You'll need the derivative of the output activation function ($f(x) = x$) for the backpropagation implementation. If you aren't familiar with calculus, this function is equivalent to the equation $y = x$. What is the slope of that equation? That is the derivative of $f(x)$.

Below, you have these tasks:
1. ~~Implement the sigmoid function to use as the activation function. Set self.activation_function in __init__ to your sigmoid function.~~
2. ~~Implement the forward pass in the train method.~~
3. ~~Implement the backpropagation algorithm in the train method, including calculating the output error.~~
4. ~~Implement the forward...
Can you write Python code for it?

class NeuralNetwork(object):
    def __init__(self, input_nodes, hidden_nodes, output_nodes, learning_rate):
        # Set number of nodes in input, hidden and output layers.
        self.input_nodes = input_nodes
        self.hidden_nodes = hidden_nodes
        self.output_nodes = output_nodes

        # Initialize weights
        self.weights_input_to_hidden = np.random.normal(0.0, self.input_nodes**-0.5,
                                       (self.input_nodes, self.hidden_nodes))

        self.weights_hidden_to_output = np.random.normal(0.0, self.hidden_nodes**-0.5,
                                       (self.hidden_nodes, self.output_nodes))
        self.lr = learning_rate

        #### TODO: Set self.activation_function to your implemented sigmoid function ####
        #
        # Note: in Python, you can define a function with a lambda expression,
        # as shown below.
        self.activation_function = lambda x : 1.0/(1+np.exp(-x))  # Replace 0 with your sigmoid calculation.

        ### If the lambda code above is not something you're familiar with,
        # You can uncomment out the following three lines and put your
        # implementation there instead.
        #
        #def sigmoid(x):
        #    return 0  # Replace 0 with your sigmoid calculation here
        #self.activation_function = sigmoid

    def train(self, features, targets):
        ''' Train the network on batch of features and targets.

            Arguments
            ---------

            features: 2D array, each row is one data record, each column is a feature
            targets: 1D array of target values

        '''
        n_records = features.shape[0]
        delta_weights_i_h = np.zeros(self.weights_input_to_hidden.shape)
        delta_weights_h_o = np.zeros(self.weights_hidden_to_output.shape)
        for X, y in zip(features, targets):
            #### Implement the forward pass here ####
            ### Forward pass ###
            #...


I am working on the file "Project1-First-Neural-Network/DLND Your first neural network.ipynb".
The first task is:
Training the network
Here you'll set the hyperparameters for the network. The strategy here is to find hyperparameters such that the error on the training set is low, but you're not overfitting to the data. If you train the network too long or have too many hidden nodes, it can become overly specific to the training set and will fail to generalize to the validation set. That is, the loss on the validation set will start increasing as the training set loss drops.
You'll also be using a method know as Stochastic Gradient Descent (SGD) to train the network. The idea is that for each training pass, you grab a random sample of the data instead of using the whole data set. You use many more training passes than with normal gradient descent, but each pass is much faster. This ends up training the network more efficiently. You'll learn more about SGD later.
Choose the number of iterations
This is the number of batches of samples from the training data we'll use to train the network. The more iterations you use, the better the model will fit the data. However, if you use too many iterations, then the model with not generalize well to other data, this is called overfitting. You want to find a number here where the network has a low training loss, and the validation loss is at a minimum. As you start overfitting, you'll see the training loss continue to decrease while the validation loss starts to increase.
Choose the learning rate
This scales the size of weight updates. If this is too big, the weights tend to explode and the network fails to fit the data. A good choice to start at is 0.1. If the network has problems fitting the data, try reducing the learning rate. Note that the lower the learning rate, the smaller the steps are in the weight updates and the longer it takes for the neural network to converge.
Choose the number of hidden nodes
The more hidden nodes you have, the more accurate predictions the model will make. Try a few different numbers and see how it affects the performance. You can look at the losses...
Can you write Python code for it?

import sys

### Set the hyperparameters here ###
iterations = 5000
learning_rate = 0.4
hidden_nodes = 18
output_nodes = 1

# #
# iterations = 5000
# learning_rate = 0.005
# hidden_nodes = 18
# output_nodes = 1
# Progress: 100.0% ... Training loss: 0.058 ... Validation loss: 0.131
# #

# ##
# iterations = 5000
# learning_rate = 0.004
# hidden_nodes = 18
# output_nodes = 1
# Progress: 100.0% ... Training loss: 0.068 ... Validation loss: 0.177
# ##

# ##
# iterations = 5000
# learning_rate = 0.004
# hidden_nodes = 19
# output_nodes = 1
# Progress: 100.0% ... Training loss: 0.061 ... Validation loss: 0.152
# ##

# ##
# iterations = 10000
# learning_rate = 0.002
# hidden_nodes = 18
# output_nodes = 1
# Progress: 100.0% ... Training loss: 0.066 ... Validation loss: 0.169
# ##

##
# iterations = 15000
# learning_rate = 0.001
# hidden_nodes = 17
# output_nodes = 1
# Progress: 100.0% ... Training loss: 0.077 ... Validation loss: 0.167
# ##

##
# iterations = 20000
# learning_rate = 0.001
# hidden_nodes = 18
# output_nodes = 1
# Progress: 100.0% ... Training loss: 0.061 ... Validation loss: 0.171
# ##

N_i = train_features.shape[1]
network = NeuralNetwork(N_i, hidden_nodes, output_nodes, learning_rate)

losses = {'train':[], 'validation':[]}
for ii in range(iterations):
    # Go through a random batch of 128 records from the training data set
    batch = np.random.choice(train_features.index, size=128)
    X, y = train_features.ix[batch].values, train_targets.ix[batch]['cnt']

    network.train(X, y)

    # Printing out the training progress
    train_loss = MSE(network.run(train_features).T, train_targets['cnt'].values)
    val_loss = MSE(network.run(val_features).T, val_targets['cnt'].values)
    sys.stdout.write("\rProgress: {:2.1f}".format(100 * ii/float(iterations)) \
                     + "% ... Training loss: " + str(train_loss)[:5] \
                     + " ... Validation loss: " + str(val_loss)[:5])
    sys.stdout.flush()

    losses['train'].append(train_loss)
   ...


I am working on the file "Project1-First-Neural-Network/DLND Your first neural network.ipynb".
The first task is:
We'll split the data into two sets, one for training and one for validating as the network is being trained. Since this is time series data, we'll train on historical data, then try to predict on future data (the validation set).
Can you write Python code for it?

# Hold out the last 60 days or so of the remaining data as a validation set
train_features, train_targets = features[:-60*24], targets[:-60*24]
val_features, val_targets = features[-60*24:], targets[-60*24:]


I am working on the file "Project1-boston_housing/boston_housing.ipynb".
The first task is:
Answer:
I think this model have successfully captured the cariation of the target variable.
reason:
Best possible score is 1.0,and the result 0.923 is high enough
Implementation: Shuffle and Split Data
Your next implementation requires that you take the Boston housing dataset and split the data into training and testing subsets. Typically, the data is also shuffled into a random order when creating the training and testing subsets to remove any bias in the ordering of the dataset.
For the code cell below, you will need to implement the following:
- Use train_test_split from sklearn.cross_validation to shuffle and split the features and prices data into training and testing sets.
  - Split the data into 80% training and 20% testing.
  - Set the random_state for train_test_split to a value of your choice. This ensures results are consistent.
- Assign the train and testing splits to X_train, X_test, y_train, and y_test.
Can you write Python code for it?

from sklearn.model_selection import train_test_split
# Shuffle and split the data into training and testing subsets
X_train, X_test, y_train, y_test = train_test_split(features, prices, test_size=0.2, random_state=2)

# Success
print "Training and testing split was successful."


I am working on the file "Project1-boston_housing/boston_housing.ipynb".
The first task is:
Answer:
Just like my guess in Question, Max_depth=4 has the best performance
Question 10 - Predicting Selling Prices
Imagine that you were a real estate agent in the Boston area looking to use this model to help price homes owned by your clients that they wish to sell. You have collected the following information from three of your clients:
| Feature | Client 1 | Client 2 | Client 3 |
| :---: | :---: | :---: | :---: |
| Total number of rooms in home | 5 rooms | 4 rooms | 8 rooms |
| Neighborhood poverty level (as %) | 17% | 32% | 3% |
| Student-teacher ratio of nearby schools | 15-to-1 | 22-to-1 | 12-to-1 |
What price would you recommend each client sell his/her home at? Do these prices seem reasonable given the values for the respective features?
Hint: Use the statistics you calculated in the Data Exploration section to help justify your response.
Run the code block below to have your optimized model make predictions for each client's home.
Can you write Python code for it?

# Produce a matrix for client data
client_data = [[5, 17, 15], # Client 1
               [4, 32, 22], # Client 2
               [8, 3, 12]]  # Client 3

# Show predictions
for i, price in enumerate(reg.predict(client_data)):
    print "Predicted selling price for Client {}'s home: ${:,.2f}".format(i+1, price)


I am working on the file "Project1-boston_housing/boston_housing.ipynb".
The first task is:
Data Exploration
In this first section of this project, you will make a cursory investigation about the Boston housing data and provide your observations. Familiarizing yourself with the data through an explorative process is a fundamental practice to help you better understand and justify your results.
Since the main goal of this project is to construct a working model which has the capability of predicting the value of houses, we will need to separate the dataset into features and the target variable. The features, 'RM', 'LSTAT', and 'PTRATIO', give us quantitative information about each data point. The target variable, 'MEDV', will be the variable we seek to predict. These are stored in features and prices, respectively.
Implementation: Calculate Statistics
For your very first coding implementation, you will calculate descriptive statistics about the Boston housing prices. Since numpy has already been imported for you, use this library to perform the necessary calculations. These statistics will be extremely important later on to analyze various prediction results from the constructed model.
In the code cell below, you will need to implement the following:
- Calculate the minimum, maximum, mean, median, and standard deviation of 'MEDV', which is stored in prices.
  - Store each calculation in their respective variable.
Can you write Python code for it?

# Minimum price of the data
minimum_price = np.amin(prices)

# Maximum price of the data
maximum_price = np.amax(prices)

# Mean price of the data
mean_price = np.mean(prices)

# Median price of the data
median_price = np.median(prices)

# Standard deviation of prices of the data
std_price = np.std(prices)

# Show the calculated statistics

print "Statistics for Boston housing dataset:\n"
print "Minimum price: ${:,.2f}".format(minimum_price)
print "Maximum price: ${:,.2f}".format(maximum_price)
print "Mean price: ${:,.2f}".format(mean_price)
print "Median price ${:,.2f}".format(median_price)
print "Standard deviation of prices: ${:,.2f}".format(std_price)


I am working on the file "Project1-boston_housing/boston_housing.ipynb".
The first task is:
Question 2 - Goodness of Fit
Assume that a dataset contains five data points and a model made the following predictions for the target variable:
| True Value | Prediction |
| :-------------: | :--------: |
| 3.0 | 2.5 |
| -0.5 | 0.0 |
| 2.0 | 2.1 |
| 7.0 | 7.8 |
| 4.2 | 5.3 |
Would you consider this model to have successfully captured the variation of the target variable? Why or why not?
Run the code cell below to use the performance_metric function and calculate this model's coefficient of determination.
Can you write Python code for it?

# Calculate the performance of this model
score = performance_metric([3, -0.5, 2, 7, 4.2], [2.5, 0.0, 2.1, 7.8, 5.3])
print "Model has a coefficient of determination, R^2, of {:.3f}.".format(score)


I am working on the file "Project1-boston_housing/boston_housing.ipynb".
The first task is:
Question 3 - Training and Testing
What is the benefit to splitting a dataset into some ratio of training and testing subsets for a learning algorithm?
Hint: What could go wrong with not having a way to test your model?
Answer:
if we don't split training and testing data,just like we give the answers to the students who is taking an exam.Of course they can get high score.but it can't tell us whether it is a good model to predict other data.

Analyzing Model Performance
In this third section of the project, you'll take a look at several models' learning and testing performances on various subsets of training data. Additionally, you'll investigate one particular algorithm with an increasing 'max_depth' parameter on the full training set to observe how model complexity affects performance. Graphing your model's performance based on varying criteria can be beneficial in the analysis process, such as visualizing behavior that may not have been apparent from the results alone.
Learning Curves
The following code cell produces four graphs for a decision tree model with different maximum depths. Each graph visualizes the learning curves of the model for both training and testing as the size of the training set is increased. Note that the shaded region of a learning curve denotes the uncertainty of that curve (measured as the standard deviation). The model is scored on both the training and testing sets using R2, the coefficient of determination.
Run the code cell below and use these graphs to answer the following question.
Can you write Python code for it?

# Produce learning curves for varying training set sizes and maximum depths
vs.ModelLearning(features, prices)


I am working on the file "Project1-boston_housing/boston_housing.ipynb".
The first task is:
Question 4 - Learning the Data
Choose one of the graphs above and state the maximum depth for the model. What happens to the score of the training curve as more training points are added? What about the testing curve? Would having more training points benefit the model?
Hint: Are the learning curves converging to particular scores?
Answer:
I choose the second graph, mac_depth=3
1. What happens to the score of the training curve as more training points are added?
as the the increase , the score decrease quiltly, but the rate of decrease speed reduce,and the score trend to be a constant.
2. What about the testing curve?
the score increase quiltly, but the rate of decrease speed reduce,and the score trend to be a constant.
3. Would having more training points benefit the model?
I don't think so.
Complexity Curves
The following code cell produces a graph for a decision tree model that has been trained and validated on the training data using different maximum depths. The graph produces two complexity curves — one for training and one for validation. Similar to the learning curves, the shaded regions of both the complexity curves denote the uncertainty in those curves, and the model is scored on both the training and validation sets using the performance_metric function.
Run the code cell below and use this graph to answer the following two questions.
Can you write Python code for it?


I am working on the file "Project1/.ipynb_checkpoints/Project 1 - Titanic-checkpoint.ipynb".
The first task is:
Approximately 55% of the passengers belonged to the third class, while the rest of the ship belong to the first and second classes. Now we'll see if the first and second class passengers also paid a premimum when it comes to safety?
Can you write Python code for it?

survived['Pclass'].value_counts()

sns.set(style="darkgrid")
ax = sns.countplot(x = "Pclass", hue = "Survived", data = titanic)
plt.show()


I am working on the file "Project1/.ipynb_checkpoints/Project 1 - Titanic-checkpoint.ipynb".
The first task is:
Now we would extract the survived dataset for future analysis:
Can you write Python code for it?

survived = titanic[titanic['Survived'] == 1]


I am working on the file "Project1/.ipynb_checkpoints/Project 1 - Titanic-checkpoint.ipynb".
The first task is:
Someone's family size would be equal to their number of spouses/siblings and parents/children on the ship, plus themselves:
Can you write Python code for it?

# Family size
titanic['Family_size'] = titanic['SibSp'] + titanic['Parch'] + 1


I am working on the file "Project1/.ipynb_checkpoints/Project 1 - Titanic-checkpoint.ipynb".
The first task is:
The next step is to explore the dataset:
Can you write Python code for it?


I am working on the file "Project1/.ipynb_checkpoints/Project 1 - Titanic-checkpoint.ipynb".
The first task is:
We can see that male adults are the initial largest type of people on the ship, followed by female adults and child.
Now looking into the survival rate:
Can you write Python code for it?

survived = titanic[titanic['Survived'] == 1]
non_survived = titanic[titanic['Survived'] == 0]

survived['Type'].value_counts()

non_survived['Type'].value_counts()

sns.set(style="darkgrid")
ax = sns.countplot(x="Survived", hue = "Type", data = titanic)
plt.show()


I am working on the file "Project1/.ipynb_checkpoints/Project 1 - Titanic-checkpoint.ipynb".
The first task is:
We can see that the majority of the ship traveled by themselves, followed by families of 2 or 3. The families that had more than 3 members made up a small part of the ship. Now look into the survival statistics:
Can you write Python code for it?

survived['Family_size'].value_counts()

sns.boxplot(x="Survived", y="Family_size", data=titanic)
plt.show()

sns.kdeplot(survived['Family_size'], shade=True)
plt.show()


I am working on the file "Project1/.ipynb_checkpoints/Project1-Titanic-checkpoint.ipynb".
The first task is:
Approximately 55% of the passengers belonged to the third class, while the rest of the ship belong to the first and second classes. Now we'll see if the first and second class passengers also paid a premimum when it comes to safety?
Can you write Python code for it?

survived['Pclass'].value_counts()

sns.set(style="darkgrid")
ax = sns.countplot(x = "Pclass", hue = "Survived", data = titanic)
plt.xlabel("Passenger classes")
plt.title("Number of survivals and non-survivals, sorted by passenger classes")
plt.show()


I am working on the file "Project1/.ipynb_checkpoints/Project1-Titanic-checkpoint.ipynb".
The first task is:
Data cleaning:
We can see that both the Age column has a lot of NAs. We would need to fill in the blank with random values generated within their standardized value.
Can you write Python code for it?

titanic['Age'].describe()

average_age = titanic["Age"].mean()
std_age = titanic["Age"].std()
count_nan_age = titanic["Age"].isnull().sum()
# generate random numbers between (mean - std) & (mean + std)
rand = np.random.randint(average_age - std_age, average_age + std_age, size = count_nan_age)

# Fill NAs in age with median age
titanic['Age'][np.isnan(titanic["Age"])] = rand

titanic['Age'].describe()

sns.distplot(titanic['Age'])
plt.ylabel('Distribution')
plt.title("The distribution of ages of Titanic passengers")
plt.show()


I am working on the file "Project1/.ipynb_checkpoints/Project1-Titanic-checkpoint.ipynb".
The first task is:
Now we would extract the survived dataset for future analysis:
Can you write Python code for it?


I am working on the file "Project1/.ipynb_checkpoints/Project1-Titanic-checkpoint.ipynb".
The first task is:
The histogram of the age distribution of the survival group also confirms that younger people had a higher advantage in survival comparing to older ages.
Socio-economic classes:
We can assume that someone's class on the Titanic represented their socio-economic status. Also, we would assume that the fares have a direct correlation with the classes; so we only need to examine one of them.
Can you write Python code for it?

titanic['Pclass'].value_counts()

sns.set(style="darkgrid")
ax = sns.countplot(x = "Pclass", data = titanic)
plt.xlabel("Passenger classes")
plt.title("Number of passengers, sorted by passenger classes")
plt.show()


I am working on the file "Project1/.ipynb_checkpoints/Project1-Titanic-checkpoint.ipynb".
The first task is:
The next step is to explore the dataset:
Can you write Python code for it?


I am working on the file "Project1/.ipynb_checkpoints/Project1-Titanic-checkpoint.ipynb".
The first task is:
We can see that Passenger ID, Name and Cabin have little value to the analysis, so we drop these columns off the dataset:
Can you write Python code for it?

titanic = titanic.drop(['PassengerId','Name','Ticket', 'Cabin', 'Embarked'], axis=1)

titanic['Survived'].describe()


I am working on the file "Project1/.ipynb_checkpoints/Project1-Titanic-checkpoint.ipynb".
The first task is:
We can see that male adults are the initial largest type of people on the ship, followed by female adults and child.
Now looking into the survival rate:
Can you write Python code for it?

survived = titanic[titanic['Survived'] == 1]
non_survived = titanic[titanic['Survived'] == 0]

survived['Type'].value_counts()

non_survived['Type'].value_counts()

sns.set(style="darkgrid")
ax = sns.countplot(x="Survived", hue = "Type", data = titanic)
plt.title("Numbers of survivals and non-survivals, sorted by type")
plt.show()


I am working on the file "Project1/.ipynb_checkpoints/Project1-Titanic-checkpoint.ipynb".
The first task is:
We can see the women and children of the first class had a significantly impressive survival rate (more than 90% and 80% respectively), when the women and children of the third class had a much lower survival rate (more than 45% and around 40% respectively). However, the women and children from the third class did have a higher survival rate than the men from higher classes. Men from the first class had a survival rate of around 35%, which was actually below the overall survival rate of 38.38%. Men from the second and third classes suffered very low survival rates, which was around 8 % and around 12 % respectively comparing to their initial numbers.
Family size:
Did people have a higher chance of survival if they traveled with family rather than traveling alone? We'll find out.
Can you write Python code for it?


I am working on the file "Project1/.ipynb_checkpoints/Project1-Titanic-checkpoint.ipynb".
The first task is:
Women and children first?
Assuming people are neutral on the gender of a kid, I would split the passengers into 3 types:
Can you write Python code for it?

def passenger_type(person):
    if person['Age'] <= 16:
        return "child"
    elif person['Sex'] == "female":
        return "female_adult"
    else:
        return "male_adult"

titanic['Type'] = titanic.apply(passenger_type, axis = 1)
titanic

titanic['Type'].value_counts()

sns.set(style="darkgrid")
ax = sns.countplot(x="Type", data = titanic)
plt.title("Number of passengers sorted by type")
plt.show()


I am working on the file "Project1/PauloFinardi_first_n_n.ipynb".
The first task is:
Check out your predictions
Here, use the test data to view how well your network is modeling the data. If something is completely wrong here, make sure each step in your network is implemented correctly.
Can you write Python code for it?

fig, ax = plt.subplots(figsize=(8,4))

mean, std = scaled_features['cnt']
predictions = network.run(test_features)*std + mean
ax.plot(predictions[0], label='Prediction')
ax.plot((test_targets['cnt']*std + mean).values, label='Data')
ax.set_xlim(right=len(predictions))
ax.legend()

dates = pd.to_datetime(rides.ix[test_data.index]['dteday'])
dates = dates.apply(lambda d: d.strftime('%b %d'))
ax.set_xticks(np.arange(len(dates))[12::24])
_ = ax.set_xticklabels(dates[12::24], rotation=45)


I am working on the file "Project1/PauloFinardi_first_n_n.ipynb".
The first task is:
Checking out the data
This dataset has the number of riders for each hour of each day from January 1 2011 to December 31 2012. The number of riders is split between casual and registered, summed up in the cnt column. You can see the first few rows of the data above.
Below is a plot showing the number of bike riders over the first 10 days or so in the data set. (Some days don't have exactly 24 entries in the data set, so it's not exactly 10 days.) You can see the hourly rentals here. This data is pretty complicated! The weekends have lower over all ridership and there are spikes when people are biking to and from work during the week. Looking at the data above, we also have information about temperature, humidity, and windspeed, all of these likely affecting the number of riders. You'll be trying to capture all this with your model.
Can you write Python code for it?


I am working on the file "Project1/PauloFinardi_first_n_n.ipynb".
The first task is:
Dummy variables
Here we have some categorical variables like season, weather, month. To include these in our model, we'll need to make binary dummy variables. This is simple to do with Pandas thanks to get_dummies().
Can you write Python code for it?

dummy_fields = ['season', 'weathersit', 'mnth', 'hr', 'weekday']
for each in dummy_fields:
    dummies = pd.get_dummies(rides[each], prefix=each, drop_first=False)
    rides = pd.concat([rides, dummies], axis=1)

fields_to_drop = ['instant', 'dteday', 'season', 'weathersit',
                  'weekday', 'atemp', 'mnth', 'workingday', 'hr']
data = rides.drop(fields_to_drop, axis=1)
data.head()


I am working on the file "Project1/PauloFinardi_first_n_n.ipynb".
The first task is:
Load and prepare the data
A critical step in working with neural networks is preparing the data correctly. Variables on different scales make it difficult for the network to efficiently learn the correct weights. Below, we've written the code to load and prepare the data. You'll learn more about this soon!
Can you write Python code for it?

data_path = 'Bike-Sharing-Dataset/hour.csv'

rides = pd.read_csv(data_path)

rides.head()


I am working on the file "Project1/PauloFinardi_first_n_n.ipynb".
The first task is:
Scaling target variables
To make training the network easier, we'll standardize each of the continuous variables. That is, we'll shift and scale the variables such that they have zero mean and a standard deviation of 1.
The scaling factors are saved so we can go backwards when we use the network for predictions.
Can you write Python code for it?

quant_features = ['casual', 'registered', 'cnt', 'temp', 'hum', 'windspeed']
# Store scalings in a dictionary so we can convert back later
scaled_features = {}
for each in quant_features:
    mean, std = data[each].mean(), data[each].std()
    scaled_features[each] = [mean, std]
    data.loc[:, each] = (data[each] - mean)/std


I am working on the file "Project1/PauloFinardi_first_n_n.ipynb".
The first task is:
Splitting the data into training, testing, and validation sets
We'll save the data for the last approximately 21 days to use as a test set after we've trained the network. We'll use this set to make predictions and compare them with the actual number of riders.
Can you write Python code for it?

# Save data for approximately the last 21 days
test_data = data[-21*24:]

# Now remove the test data from the data set
data = data[:-21*24]

# Separate the data into features and targets
target_fields = ['cnt', 'casual', 'registered']
features, targets = data.drop(target_fields, axis=1), data[target_fields]
test_features, test_targets = test_data.drop(target_fields, axis=1), test_data[target_fields]


I am working on the file "Project1/PauloFinardi_first_n_n.ipynb".
The first task is:
Time to build the network
Below you'll build your network. We've built out the structure and the backwards pass. You'll implement the forward pass through the network. You'll also set the hyperparameters: the learning rate, the number of hidden units, and the number of training passes.
The network has two layers, a hidden layer and an output layer. The hidden layer will use the sigmoid function for activations. The output layer has only one node and is used for the regression, the output of the node is the same as the input of the node. That is, the activation function is $f(x)=x$. A function that takes the input signal and generates an output signal, but takes into account the threshold, is called an activation function. We work through each layer of our network calculating the outputs for each neuron. All of the outputs from one layer become inputs to the neurons on the next layer. This process is called forward propagation.
We use the weights to propagate signals forward from the input to the output layers in a neural network. We use the weights to also propagate error backwards from the output back into the network to update our weights. This is called backpropagation.

Hint: You'll need the derivative of the output activation function ($f(x) = x$) for the backpropagation implementation. If you aren't familiar with calculus, this function is equivalent to the equation $y = x$. What is the slope of that equation? That is the derivative of $f(x)$.

Below, you have these tasks:
1. Implement the sigmoid function to use as the activation function. Set self.activation_function in __init__ to your sigmoid function.
2. Implement the forward pass in the train method.
3. Implement the backpropagation algorithm in the train method, including calculating the output error.
4. Implement the forward pass in the run method.
Can you write Python code for it?

class NeuralNetwork(object):
    def __init__(self, input_nodes, hidden_nodes, output_nodes, learning_rate):
        # Set number of nodes in input, hidden and output layers.
        self.input_nodes = input_nodes   #  56 un
        self.hidden_nodes = hidden_nodes
        self.output_nodes = output_nodes

        np.random.seed(0)

         # Initialize weights
        self.W1 = np.random.random((self.hidden_nodes, self.input_nodes)) / np.sqrt(input_nodes)

        self.W2 = np.random.random((self.output_nodes, self.hidden_nodes)) / np.sqrt(hidden_nodes)
        self.lr = -learning_rate

        self.act_func = lambda x: 1.0 / (1.0 + np.exp(-x))
        self.act_func_der = lambda x: x * (1 - x)

    def train(self, inputs_list, targets_list):
        # Convert inputs list to 2d array
        inputs = np.array(inputs_list, ndmin=2).T      # 56x1
        targets = np.array(targets_list, ndmin=2).T    # 1x1

        ### Forward pass ###
        z1 = self.W1.dot(inputs)
        a1 = self.act_func(z1)
        z2 = self.W2.dot(a1)
        a2 = z2

        ### Backward pass ###
        delta2 = (a2 - targets)
        delta1 = self.W2.T.dot(delta2) * self.act_func_der(a1)

        W2_grad = delta2.dot(a1.T)
        W1_grad = delta1.dot(inputs.T)

        self.W2 += self.lr * W2_grad
        self.W1 += self.lr * W1_grad

    def run(self, inputs_list):
        # Run a forward pass through the network
        inputs = np.array(inputs_list, ndmin=2).T
        _z1 = self.W1.dot(inputs)
        _a1 = self.act_func(_z1)
        _z2 = self.W2.dot(_a1)
        _a2 = _z2
        return...


I am working on the file "Project1/PauloFinardi_first_n_n.ipynb".
The first task is:
Unit tests
Run these unit tests to check the correctness of your network implementation. These tests must all be successful to pass the project.
Can you write Python code for it?

import unittest

inputs = [0.5, -0.2, 0.1]
targets = [0.4]
test_W1 = np.array([[0.1, 0.4, -0.3],
                       [-0.2, 0.5, 0.2]])
test_W2 = np.array([[0.3, -0.1]])

class TestMethods(unittest.TestCase):

    ##########
    # Unit tests for data loading
    ##########

    def test_data_path(self):
        # Test that file path to dataset has been unaltered
        self.assertTrue(data_path.lower() == 'bike-sharing-dataset/hour.csv')

    def test_data_loaded(self):
        # Test that data frame loaded
        self.assertTrue(isinstance(rides, pd.DataFrame))

    ##########
    # Unit tests for network functionality
    ##########

    def test_activation(self):
        network = NeuralNetwork(3, 2, 1, 0.5)
        # Test that the activation function is a sigmoid
        self.assertTrue(np.all(network.act_func(0.5) == 1/(1+np.exp(-0.5))))

    def test_train(self):
        # Test that weights are updated correctly on training
        network = NeuralNetwork(3, 2, 1, 0.5)
        network.W1 = test_W1.copy()
        network.W2 = test_W2.copy()

        network.train(inputs, targets)
        self.assertTrue(np.allclose(network.W2,
                                    np.array([[ 0.37275328, -0.03172939]])))
        self.assertTrue(np.allclose(network.W1,
                                    np.array([[ 0.10562014,  0.39775194, -0.29887597],
                                              [-0.20185996,  0.50074398,  0.19962801]])))

    def test_run(self):
        # Test correctness of run method
        network = NeuralNetwork(3, 2, 1, 0.5)
        network.W1 = test_W1.copy()
        network.W2 = test_W2.copy()

        self.assertTrue(np.allclose(network.run(inputs), 0.09998924))

suite = unittest.TestLoader().loadTestsFromModule(TestMethods())
unittest.TextTestRunner().run(suite)


I am working on the file "Project1/PauloFinardi_first_n_n.ipynb".
The first task is:
We'll split the data into two sets, one for training and one for validating as the network is being trained. Since this is time series data, we'll train on historical data, then try to predict on future data (the validation set).
Can you write Python code for it?

# Hold out the last 60 days or so of the remaining data as a validation set
train_features, train_targets = features[:-60*24], targets[:-60*24]
val_features, val_targets = features[-60*24:], targets[-60*24:]


I am working on the file "Project1/Project1-Titanic.ipynb".
The first task is:
Comparing to the initial number of people of each type, we can see that children have more than 50% survival rate, female adults have an impressive survival rate around 75%, while male adults have a small survival rate of around 16% comparing to their intial numbers. So we can see that there was an inherent "women and children first" code when it came to saving people on the ship.
Can you write Python code for it?

sns.distplot(survived['Age'])
plt.ylabel("Distribution")
plt.title("The distribution of ages of Titanic survivals")
plt.show()


I am working on the file "Project1/Project1-Titanic.ipynb".
The first task is:
Data cleaning:
We can see that both the Age column has a lot of NAs. We would need to fill in the blank with random values generated within their standardized value.
Can you write Python code for it?


I am working on the file "Project1/Project1-Titanic.ipynb".
The first task is:
Now we would extract the survived dataset for future analysis:
Can you write Python code for it?

survived = titanic[titanic['Survived'] == 1]


I am working on the file "Project1/Project1-Titanic.ipynb".
The first task is:
Questions:
According to Wikipedia, "Women and children first" is a code of conduct dating from 1860, whereby the lives of women and children were to be saved first in a life-threatening situation, typically abandoning ship, when survival resources such as lifeboats were limited. The wiki page actually gives some insights and statistics on the survival rate of the Titanic; however, in this analysis, I would reconfirm them, and attempt to find out which other factors that determine the survival rate in the Titanic tragedy.
The questions I am going to answer in this analysis are:

Was there really a "Women and children first" rule on the Titanic?
Did other factors such as wealth/classes and family sizes affect someone's chance of survival?

Women and children first?
Assuming people are neutral on the gender of a kid, I would split the passengers into 3 types:
Can you write Python code for it?


I am working on the file "Project1/Project1-Titanic.ipynb".
The first task is:
Someone's family size would be equal to their number of spouses/siblings and parents/children on the ship, plus themselves:
Can you write Python code for it?

# Family size
titanic['Family_size'] = titanic['SibSp'] + titanic['Parch'] + 1


I am working on the file "Project1/Project1-Titanic.ipynb".
The first task is:
The histogram of the age distribution of the survival group also confirms that younger people had a higher advantage in survival comparing to older ages.
Socio-economic classes:
We can assume that someone's class on the Titanic represented their socio-economic status. Also, we would assume that the fares have a direct correlation with the classes; so we only need to examine one of them.
Can you write Python code for it?


I am working on the file "Project1/Project1-Titanic.ipynb".
The first task is:
The survival rate of the first class passengers was more than 60%, while the survival rate of the third class ones was merely around 25%. So we can see that there was a bias on weathiness and soci-economic statuses, even in life-threatning situations.
Now, what if we factor in both passenger classes and types (male, female or children), which would have more weight in survival rate?
Can you write Python code for it?

titanic.groupby(['Pclass', 'Type']).Type.count()

sns.set(style="darkgrid")
ax = sns.countplot(x = "Pclass", hue = "Type", data = titanic)
plt.title("Number of people per type, sorted by passenger classes")
plt.show()

titanic.groupby(['Pclass', 'Type']).agg({'Survived': 'sum'})

sns.set(style="darkgrid")
ax = sns.countplot(x = "Pclass", hue = "Type", data = survived)
plt.title("Number of survivals and non-survivals per type, sorted by passenger classes")
plt.show()


I am working on the file "Project1/Project1-Titanic.ipynb".
The first task is:
We can see that the majority of the ship traveled by themselves, followed by families of 2 or 3. The families that had more than 3 members made up a small part of the ship. Now look into the survival statistics:
Can you write Python code for it?

survived['Family_size'].value_counts()

sns.boxplot(x="Survived", y="Family_size", data=titanic)
plt.title("The distribution of family sizes of non-survivals and survivals")
plt.show()

sns.kdeplot(survived['Family_size'], shade=True)
plt.ylabel("Distribution")
plt.xlabel("Family size")
plt.title("The distribution of family sizes of non-survivals and survivals")
plt.show()


I am working on the file "Project1/Project1-Titanic.ipynb".
The first task is:
We can see the women and children of the first class had a significantly impressive survival rate (more than 90% and 80% respectively), when the women and children of the third class had a much lower survival rate (more than 45% and around 40% respectively). However, the women and children from the third class did have a higher survival rate than the men from higher classes. Men from the first class had a survival rate of around 35%, which was actually below the overall survival rate of 38.38%. Men from the second and third classes suffered very low survival rates, which was around 8 % and around 12 % respectively comparing to their initial numbers.
Family size:
Did people have a higher chance of survival if they traveled with family rather than traveling alone? We'll find out.
Can you write Python code for it?


I am working on the file "Project1/boston_housing.ipynb".
The first task is:
第二步. 分析数据
在项目的第一个部分，你会对波士顿房地产数据进行初步的观察并给出你的分析。通过对数据的探索来熟悉数据可以让你更好地理解和解释你的结果。
由于这个项目的最终目标是建立一个预测房屋价值的模型，我们需要将数据集分为特征(features)和目标变量(target variable)。
- 特征 'RM'， 'LSTAT'，和 'PTRATIO'，给我们提供了每个数据点的数量相关的信息。
- 目标变量：'MEDV'，是我们希望预测的变量。
他们分别被存在features和prices两个变量名中。
编程练习 1：基础统计运算
你的第一个编程练习是计算有关波士顿房价的描述统计数据。我们已为你导入了numpy，你需要使用这个库来执行必要的计算。这些统计数据对于分析模型的预测结果非常重要的。
在下面的代码中，你要做的是：
- 计算prices中的'MEDV'的最小值、最大值、均值、中值和标准差；
- 将运算结果储存在相应的变量中。
Can you write Python code for it?

#TODO 1

#目标：计算价值的最小值
minimum_price = np.amin(data['MEDV'])

#目标：计算价值的最大值
maximum_price = np.amax(data['MEDV'])

#目标：计算价值的平均值
mean_price = np.mean(data['MEDV'])

#目标：计算价值的中值
median_price = np.median(data['MEDV'])

#目标：计算价值的标准差
std_price = np.std(data['MEDV'])

#目标：输出计算的结果
print "Statistics for Boston housing dataset:\n"
print "Minimum price: ${:,.2f}".format(minimum_price)
print "Maximum price: ${:,.2f}".format(maximum_price)
print "Mean price: ${:,.2f}".format(mean_price)
print "Median price ${:,.2f}".format(median_price)
print "Standard deviation of prices: ${:,.2f}".format(std_price)


I am working on the file "Project1/boston_housing.ipynb".
The first task is:
编程练习 4：训练最优模型 （可选）
在这个练习中，你将需要将所学到的内容整合，使用决策树算法训练一个模型。为了得出的是一个最优模型，你需要使用网格搜索法训练模型，以找到最佳的 'max_depth' 参数。你可以把'max_depth' 参数理解为决策树算法在做出预测前，允许其对数据提出问题的数量。决策树是监督学习算法中的一种。
在下方 fit_model 函数中，你需要做的是：

遍历参数‘max_depth’的可选值 1～10，构造对应模型
计算当前模型的交叉验证分数
返回最优交叉验证分数对应的模型
Can you write Python code for it?

# TODO 4 可选

'''
不允许使用 DecisionTreeRegressor 以外的任何 sklearn 库

提示: 你可能需要实现下面的 cross_val_score 函数

def cross_val_score(estimator, X, y, scoring = performance_metric, cv=3):
    """ 返回每组交叉验证的模型分数的数组 """
    scores = [0,0,0]
    return scores
'''

def cross_val_score(estimator, X, y, scoring = performance_metric, cv=3):
    """ 返回每组交叉验证的模型分数的数组 """
    scores = [0, 0, 0]
    fold = len(X) / cv
    for i in range(cv):
        X_train = np.concatenate([X[:i * fold], X[(i + 1) * fold:]])
        X_test = X[i * fold:(i + 1) * fold]
        y_train = np.concatenate([y[:i * fold], y[(i + 1) * fold:]])
        y_test = y[i * fold:(i + 1) * fold]
        estimator = estimator.fit(X_train, y_train)
        scores[i] = scoring(y_test, estimator.predict(X_test))
    return scores

def fit_model2(X, y):
    """ 基于输入数据 [X,y]，利于网格搜索找到最优的决策树模型"""
    #最优交叉验证分数对应的最优模型
    best_estimator = None
    best = 1
    best_score = 0
    for i in range(1, 11):
        regressor = DecisionTreeRegressor(max_depth=i)
        score = np.mean(cross_val_score(regressor, X, y))
        (best_score, best) = (score, i) if score > best_score else (best_score, best)
    best_estimator = DecisionTreeRegressor(max_depth=best)
    return best_estimator

    return best_estimator


I am working on the file "Project1/boston_housing.ipynb".
The first task is:
问题 1 - 特征观察
如前文所述，本项目中我们关注的是其中三个值:'RM'、'LSTAT' 和'PTRATIO'，对每一个数据点:
- 'RM' 是该地区中每个房屋的平均房间数量；
- 'LSTAT' 是指该地区有多少百分比的房东属于是低收入阶层（有工作但收入微薄）；
- 'PTRATIO' 是该地区的中学和小学里，学生和老师的数目比（学生/老师）。
凭直觉，上述三个特征中对每一个来说，你认为增大该特征的数值，'MEDV'的值会是增大还是减小呢？每一个答案都需要你给出理由。
提示：你预期一个'RM' 值是6的房屋跟'RM' 值是7的房屋相比，价值更高还是更低呢？
问题 1 - 回答：
直觉上来讲：
RM房间数量越大，房屋价值越高
LStat房东是低收入阶层的百分比越高，则该地区越有可能属于整体收入较低的群体，则价格会更低
PtRatio 学生老师数目比，表示当地学生的数目，代表房子有可能是处于学区，这个比值越高则房价越高
编程练习 2: 数据分割与重排
接下来，你需要把波士顿房屋数据集分成训练和测试两个子集。通常在这个过程中，数据也会被重排列，以消除数据集中由于顺序而产生的偏差。
在下面的代码中，你需要
使用 sklearn.model_selection 中的 train_test_split， 将features和prices的数据都分成用于训练的数据子集和用于测试的数据子集。
  - 分割比例为：80%的数据用于训练，20%用于测试；
  - 选定一个数值以设定 train_test_split 中的 random_state ，这会确保结果的一致性；
Can you write Python code for it?

# TODO 2

# 提示： 导入train_test_split
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(features,prices,test_size=0.2,random_state=42)


I am working on the file "Project1/boston_housing.ipynb".
The first task is:
问题 10 - 回答：
我会按照模型预测来给出建议。
从结果来看，客户3 > 客户1 > 客户2
从输入数据来看，客户3房子数最多，非贫困社区，显然应该价格最高
客户2房间数最少，很有可能是贫困阶层，所以自然价格最低。
因此从排序来看预测值是正确的。
Minimum price: $105,000.00
Maximum price: $1,024,800.00
Mean price: $454,342.94
Median price $438,900.00
Standard deviation of prices: $165,171.13
客户2的报价没有低于最小值，客户3的报价没有高于最大值，三者平均值500，000左右，与统计数据差不多，因此预测没有太大问题。
编程练习 5
你刚刚预测了三个客户的房子的售价。在这个练习中，你将用你的最优模型在整个测试数据上进行预测, 并计算相对于目标变量的决定系数 R2的值**。
Can you write Python code for it?

#TODO 5

# 提示：你可能需要用到 X_test, y_test, optimal_reg, performance_metric
# 提示：你可能需要参考问题10的代码进行预测
# 提示：你可能需要参考问题3的代码来计算R^2的值

pred = optimal_reg.predict(X_test)
r2 = performance_metric(y_test, pred)

print "Optimal model has R^2 score {:,.2f} on test data".format(r2)


I am working on the file "Project1/boston_housing.ipynb".
The first task is:
问题 12 - 回答：
基本一致，预测数据趋势与训练数据大体相同
问题 13 - 实用性探讨
简单地讨论一下你建构的模型能否在现实世界中使用？
提示：回答以下几个问题，并给出相应结论的理由：
- 1978年所采集的数据，在已考虑通货膨胀的前提下，在今天是否仍然适用？
- 数据中呈现的特征是否足够描述一个房屋？
- 在波士顿这样的大都市采集的数据，能否应用在其它乡镇地区？
- 你觉得仅仅凭房屋所在社区的环境来判断房屋价值合理吗？
问题 13 - 回答：
如果已经考虑通货膨胀，将数据变形到当代的话，仍然适用
不足以描述，如国内各种开发商、学区房等都会对数据造成干扰
不适用，每个地区的训练数据不同，得出的结论不同，如果拿其他乡镇做测试集，得分应该不高
不合理，以一个维度判断，会有很大局限性，且同一个小区内的房价也有不同
可选问题 - 预测北京房价
（本题结果不影响项目是否通过）通过上面的实践，相信你对机器学习的一些常用概念有了很好的领悟和掌握。但利用70年代的波士顿房价数据进行建模的确对我们来说意义不是太大。现在你可以把你上面所学应用到北京房价数据集中 bj_housing.csv。
免责声明：考虑到北京房价受到宏观经济、政策调整等众多因素的直接影响，预测结果仅供参考。
这个数据集的特征有：
- Area：房屋面积，平方米
- Room：房间数，间
- Living: 厅数，间
- School: 是否为学区房，0或1
- Year: 房屋建造时间，年
- Floor: 房屋所处楼层，层
目标变量：
- Value: 房屋人民币售价，万
你可以参考上面学到的内容，拿这个数据集来练习数据分割与重排、定义衡量标准、训练模型、评价模型表现、使用网格搜索配合交叉验证对参数进行调优并选出最佳参数，比较两者的差别，最终得出最佳模型对验证集的预测分数。
Can you write Python code for it?


I am working on the file "Project1/boston_housing.ipynb".
The first task is:
问题 2 - 训练及测试
将数据集按一定比例分为训练用的数据集和测试用的数据集对学习算法有什么好处？
如果用模型已经见过的数据，例如部分训练集数据进行测试，又有什么坏处？
提示： 如果没有数据来对模型进行测试，会出现什么问题？
问题 2 - 回答:
区分训练用和测试用数据集，可以有效在训练过程中得到及时反馈，我们可以使用训练测试集进行训练算法，然后通过测试集进行测试得分，一般情况下通过得分高低来评价算法优化的优劣，观察算法拟合程度。
如果使用训练数据测试，由于算法就是这堆数据产生的，但我们无法了解此算法对未知数据的表现如何。

第三步. 模型衡量标准
在项目的第三步中，你需要了解必要的工具和技巧来让你的模型进行预测。用这些工具和技巧对每一个模型的表现做精确的衡量可以极大地增强你预测的信心。
编程练习3：定义衡量标准
如果不能对模型的训练和测试的表现进行量化地评估，我们就很难衡量模型的好坏。通常我们会定义一些衡量标准，这些标准可以通过对某些误差或者拟合程度的计算来得到。在这个项目中，你将通过运算决定系数 R2 来量化模型的表现。模型的决定系数是回归分析中十分常用的统计信息，经常被当作衡量模型预测能力好坏的标准。
R2的数值范围从0至1，表示目标变量的预测值和实际值之间的相关程度平方的百分比。一个模型的R2 值为0还不如直接用平均值来预测效果好；而一个R2 值为1的模型则可以对目标变量进行完美的预测。从0至1之间的数值，则表示该模型中目标变量中有百分之多少能够用特征来解释。模型也可能出现负值的R2，这种情况下模型所做预测有时会比直接计算目标变量的平均值差很多。
在下方代码的 performance_metric 函数中，你要实现：
- 使用 sklearn.metrics 中的 r2_score 来计算 y_true 和 y_predict的R2值，作为对其表现的评判。
- 将他们的表现评分储存到score变量中。
或

(可选) 不使用任何外部库，参考决定系数的定义进行计算，这也可以帮助你更好的理解决定系数在什么情况下等于0或等于1。
Can you write Python code for it?

# TODO 3

# 提示： 导入r2_score
from sklearn.metrics import r2_score
def performance_metric(y_true, y_predict):
    """计算并返回预测值相比于预测值的分数"""

    score = r2_score(y_true, y_predict)

    return score

# TODO 3 可选

# 不允许导入任何计算决定系数的库
# R_square = 1- SSE / SST

def performance_metric2(y_true, y_predict):
    """计算并返回预测值相比于预测值的分数"""

    sum = 0
    SSE = 0
    SST = 0
    for i in range(len(y_true)):
        SSE = SSE + (y_true[i] - y_predict[i])**2
        sum = sum + y_true[i]
    average = sum/len(y_true)
    for i in range(len(y_true)):
        SST = SST + (y_true[i] - average)**2
    score = 1 - (SSE/SST)

    return score


I am working on the file "Project1/boston_housing.ipynb".
The first task is:
问题 3 - 回答:
非常成功，因为R方已经很大，而且从图中数据来看，每个预测数据都几乎接近实际数据

第四步. 分析模型的表现
在项目的第四步，我们来看一下不同参数下，模型在训练集和验证集上的表现。这里，我们专注于一个特定的算法（带剪枝的决策树，但这并不是这个项目的重点），和这个算法的一个参数 'max_depth'。用全部训练集训练，选择不同'max_depth' 参数，观察这一参数的变化如何影响模型的表现。画出模型的表现来对于分析过程十分有益，这可以让我们看到一些单看结果看不到的行为。
学习曲线
下方区域内的代码会输出四幅图像，它们是一个决策树模型在不同最大深度下的表现。每一条曲线都直观得显示了随着训练数据量的增加，模型学习曲线的在训练集评分和验证集评分的变化，评分使用决定系数R2。曲线的阴影区域代表的是该曲线的不确定性（用标准差衡量）。
运行下方区域中的代码，并利用输出的图形回答下面的问题。
Can you write Python code for it?

# 根据不同的训练集大小，和最大深度，生成学习曲线
vs.ModelLearning(X_train, y_train)


I am working on the file "Project1/boston_housing.ipynb".
The first task is:
问题 4 - 学习曲线
选择上述图像中的其中一个，并给出其最大深度。随着训练数据量的增加，训练集曲线的评分有怎样的变化？验证集曲线呢？如果有更多的训练数据，是否能有效提升模型的表现呢？
提示：学习曲线的评分是否最终会收敛到特定的值？
随着数据量增加，训练集平分平缓下降，验证集平分平缓上升
验证曲线会趋于一致，且更平缓
如果有更多数据集，并不能对其表现有很大提升
问题 4 - 回答:
复杂度曲线
下列代码内的区域会输出一幅图像，它展示了一个已经经过训练和验证的决策树模型在不同最大深度条件下的表现。这个图形将包含两条曲线，一个是训练集的变化，一个是验证集的变化。跟学习曲线相似，阴影区域代表该曲线的不确定性，模型训练和测试部分的评分都用的 performance_metric 函数。
运行下方区域中的代码，并利用输出的图形并回答下面的两个问题。
Can you write Python code for it?

# 根据不同的最大深度参数，生成复杂度曲线
vs.ModelComplexity(X_train, y_train)


I am working on the file "Project1/boston_housing.ipynb".
The first task is:
问题 5 - 偏差（bias）与方差（variance）之间的权衡取舍
当模型以最大深度 1训练时，模型的预测是出现很大的偏差还是出现了很大的方差？当模型以最大深度10训练时，情形又如何呢？图形中的哪些特征能够支持你的结论？
提示： 你如何得知模型是否出现了偏差很大或者方差很大的问题？
深度为1时是经典的欠拟合，会出现很大的偏差。
深度为10时时过拟合，会出现很大的方差
R方的值在深度为1时很低，表示当前集合处于欠拟合
R方的值在深度为10时，训练集很高，测试集反而下降，所以表现为过拟合
问题 5 - 回答:
深度为1时是经典的欠拟合，会出现很大的偏差。 深度为10时时过拟合，会出现很大的方差 R方的值在深度为1时很低，表示当前集合处于欠拟合 R方的值在深度为10时，训练集很高，测试集反而下降，所以表现为过拟合
问题 6- 最优模型的猜测
结合问题 5 中的图，你认为最大深度是多少的模型能够最好地对未见过的数据进行预测？你得出这个答案的依据是什么？
问题 6 - 回答:
3，因为此时R方值最大，而且训练集和测试集的差距比较小，说明此时拟合程度合适，且泛化能力比较强

第五步. 选择最优参数
问题 7- 网格搜索（Grid Search）
什么是网格搜索法？如何用它来优化模型？
问题 7 - 回答:
网格搜索法是交叉验证的一种方法，通过遍历组合来实现模型优化。
用各个参数可能的取值来构造GridSearchCV，列出所有可能组合结果生成网格，然后使用这些组合进行训练，再进行评估，会得出一个最佳结果。
问题 8 - 交叉验证

什么是K折交叉验证法（k-fold cross-validation）？
GridSearchCV是如何结合交叉验证来完成对最佳参数组合的选择的？
GridSearchCV中的'cv_results_'属性能告诉我们什么？
网格搜索时如果不使用交叉验证会有什么问题？交叉验证又是如何解决这个问题的？

提示： 在下面 fit_model函数最后加入 print pd.DataFrame(grid.cv_results_) 可以帮你查看更多信息。
问题 8 - 回答：
K折交叉验证是交叉验证中的一种方法，在我们划分训练集合测试集时，使用K折交叉验证可以将训练数据集分为K份，每次用一份做验证集，其余为训练集，因此可以重复验证K此，并将结果汇总平均。默认情况下会对数据顺序切分，所以如果数据是排好序或聚类的（比如视频中的作者都聚集在还一起），那训练出来的算法很有可能只fit了部分数据，导致得分有偏差。
GridSearchCV的重要作用是自动调参，在网格搜索中，如果共有m组参数，假设每组参数都是n个值，那么一共有mn组参数，再对每组参数进行交叉验证，有K次训练和验证，然后取K次平均分为最终评分结果，最终评分结果分数最高的是最佳参数组合的选择
'cvresults'能够输出一个词典，包含每个参数组合的train_score，test_score,还有每个参数组合的mean_fit_time，mean_score_time，mean_test_score ，mean_train_score以及std_fit_time，std_score_time，std_test_score ，std_train_score,可作为最佳参数组合的依据
网格搜索时，如果不适用交叉验证，可能会导致因为偶然的测试集的划分，导致的训练不准确的问题。
交叉验证使用了多次尝试来获取最优值，降低误差，提升数据利用率。
编程练习 4：训练最优模型
在这个练习中，你将需要将所学到的内容整合，使用决策树算法训练一个模型。为了得出的是一个最优模型，你需要使用网格搜索法训练模型，以找到最佳的 'max_depth' 参数。你可以把'max_depth' 参数理解为决策树算法在做出预测前，允许其对数据提出问题的数量。决策树是监督学习算法中的一种。
在下方 fit_model 函数中，你需要做的是：
1. 定义 'cross_validator' 变量: 使用 sklearn.model_selection 中的 KFold 创建一个交叉验证生成器对象;
2. 定义 'regressor' 变量: 使用  sklearn.tree 中的 DecisionTreeRegressor 创建一个决策树的回归函数;
3. 定义 'params' 变量: 为 'max_depth' 参数创造一个字典，它的值是从1至10的数组;
4. 定义 'scoring_fnc' 变量: 使用 sklearn.metrics 中的 make_scorer  创建一个评分函数；
 将 ‘performance_metric’ 作为参数传至这个函数中；
5. 定义 'grid' 变量: 使用 sklearn.model_selection 中的 GridSearchCV...
Can you write Python code for it?

# TODO 4

#提示: 导入 'KFold' 'DecisionTreeRegressor' 'make_scorer' 'GridSearchCV'

def fit_model(X, y):
    """ 基于输入数据 [X,y]，利于网格搜索找到最优的决策树模型"""

    from sklearn.model_selection import KFold
    from sklearn import tree
    from sklearn.metrics import make_scorer
    from sklearn.model_selection import GridSearchCV

    cross_validator = KFold(n_splits = 10)

    regressor = tree.DecisionTreeRegressor()

    params = {'max_depth':[1,2,3,4,5,6,7,8,9,10]}

    scoring_fnc = make_scorer(performance_metric)

    grid = GridSearchCV(regressor, params, scoring_fnc, cv=cross_validator)

    # 基于输入数据 [X,y]，进行网格搜索
    grid = grid.fit(X, y)

    print pd.DataFrame(grid.cv_results_)

    # 返回网格搜索后的最优模型
    return grid.best_estimator_


I am working on the file "Project1/boston_housing.ipynb".
The first task is:
问题 9 - 最优模型
最优模型的最大深度（maximum depth）是多少？此答案与你在问题 6所做的猜测是否相同？
运行下方区域内的代码，将决策树回归函数代入训练数据的集合，以得到最优化的模型。
Can you write Python code for it?

# 基于训练数据，获得最优模型
optimal_reg = fit_model(X_train, y_train)

# 输出最优模型的 'max_depth' 参数
print "Parameter 'max_depth' is {} for the optimal model.".format(optimal_reg.get_params()['max_depth'])


I am working on the file "Project1/boston_housing.ipynb".
The first task is:
问题11 - 分析决定系数
你刚刚计算了最优模型在测试集上的决定系数，你会如何评价这个结果？
问题11 - 回答
0.84这个分数不低，可能还可以优化。
模型健壮性
一个最优的模型不一定是一个健壮模型。有的时候模型会过于复杂或者过于简单，以致于难以泛化新增添的数据；有的时候模型采用的学习算法并不适用于特定的数据结构；有的时候样本本身可能有太多噪点或样本过少，使得模型无法准确地预测目标变量。这些情况下我们会说模型是欠拟合的。
问题 12 - 模型健壮性
模型是否足够健壮来保证预测的一致性？
提示: 执行下方区域中的代码，采用不同的训练和测试集执行 fit_model 函数10次。注意观察对一个特定的客户来说，预测是如何随训练数据的变化而变化的。
Can you write Python code for it?

# 请先注释掉 fit_model 函数里的所有 print 语句
vs.PredictTrials(features, prices, fit_model, client_data)


I am working on the file "Project1/first-neural-network/dlnd-your-first-neural-network.ipynb".
The first task is:
Load and prepare the data
A critical step in working with neural networks is preparing the data correctly. Variables on different scales make it difficult for the network to efficiently learn the correct weights. Below, we've written the code to load and prepare the data. You'll learn more about this soon!
Can you write Python code for it?

data_path = 'Bike-Sharing-Dataset/hour.csv'

rides = pd.read_csv(data_path)

rides.head()


I am working on the file "Project1/first-neural-network/dlnd-your-first-neural-network.ipynb".
The first task is:
Splitting the data into training, testing, and validation sets
We'll save the data for the last approximately 21 days to use as a test set after we've trained the network. We'll use this set to make predictions and compare them with the actual number of riders.
Can you write Python code for it?

# Save data for approximately the last 21 days
test_data = data[-21*24:]

# Now remove the test data from the data set
data = data[:-21*24]

# Separate the data into features and targets
target_fields = ['cnt', 'casual', 'registered']
features, targets = data.drop(target_fields, axis=1), data[target_fields]
test_features, test_targets = test_data.drop(target_fields, axis=1), test_data[target_fields]


I am working on the file "Project1/first-neural-network/dlnd-your-first-neural-network.ipynb".
The first task is:
Training the network
Here you'll set the hyperparameters for the network. The strategy here is to find hyperparameters such that the error on the training set is low, but you're not overfitting to the data. If you train the network too long or have too many hidden nodes, it can become overly specific to the training set and will fail to generalize to the validation set. That is, the loss on the validation set will start increasing as the training set loss drops.
You'll also be using a method know as Stochastic Gradient Descent (SGD) to train the network. The idea is that for each training pass, you grab a random sample of the data instead of using the whole data set. You use many more training passes than with normal gradient descent, but each pass is much faster. This ends up training the network more efficiently. You'll learn more about SGD later.
Choose the number of iterations
This is the number of batches of samples from the training data we'll use to train the network. The more iterations you use, the better the model will fit the data. However, if you use too many iterations, then the model with not generalize well to other data, this is called overfitting. You want to find a number here where the network has a low training loss, and the validation loss is at a minimum. As you start overfitting, you'll see the training loss continue to decrease while the validation loss starts to increase.
Choose the learning rate
This scales the size of weight updates. If this is too big, the weights tend to explode and the network fails to fit the data. A good choice to start at is 0.1. If the network has problems fitting the data, try reducing the learning rate. Note that the lower the learning rate, the smaller the steps are in the weight updates and the longer it takes for the neural network to converge.
Choose the number of hidden nodes
The more hidden nodes you have, the more accurate predictions the model will make. Try a few different numbers and see how it affects the performance. You can look at the losses...
Can you write Python code for it?

import sys

### Set the hyperparameters here ###
iterations = 50000
learning_rate = 0.05
hidden_nodes = 10
output_nodes = 1

N_i = train_features.shape[1]
network = NeuralNetwork(N_i, hidden_nodes, output_nodes, learning_rate)

losses = {'train':[], 'validation':[]}
for ii in range(iterations):
    # Go through a random batch of 128 records from the training data set
    batch = np.random.choice(train_features.index, size=128)
    X, y = train_features.ix[batch].values, train_targets.ix[batch]['cnt']

    network.train(X, y)

    # Printing out the training progress
    train_loss = MSE(network.run(train_features).T, train_targets['cnt'].values)
    val_loss = MSE(network.run(val_features).T, val_targets['cnt'].values)
    sys.stdout.write("\rProgress: {:2.1f}".format(100 * ii/float(iterations)) \
                     + "% ... Training loss: " + str(train_loss)[:5] \
                     + " ... Validation loss: " + str(val_loss)[:5])
    sys.stdout.flush()

    losses['train'].append(train_loss)
    losses['validation'].append(val_loss)

plt.plot(losses['train'], label='Training loss')
plt.plot(losses['validation'], label='Validation loss')
plt.legend()
_ = plt.ylim()


I am working on the file "Project1/first-neural-network/dlnd-your-first-neural-network.ipynb".
The first task is:
We'll split the data into two sets, one for training and one for validating as the network is being trained. Since this is time series data, we'll train on historical data, then try to predict on future data (the validation set).
Can you write Python code for it?

# Hold out the last 60 days or so of the remaining data as a validation set
train_features, train_targets = features[:-60*24], targets[:-60*24]
val_features, val_targets = features[-60*24:], targets[-60*24:]


I am working on the file "Project2-finding_donors/finding_donors.ipynb".
The first task is:
准备数据
在数据能够被作为输入提供给机器学习算法之前，它经常需要被清洗，格式化，和重新组织 - 这通常被叫做预处理。幸运的是，对于这个数据集，没有我们必须处理的无效或丢失的条目，然而，由于某一些特征存在的特性我们必须进行一定的调整。这个预处理都可以极大地帮助我们提升几乎所有的学习算法的结果和预测能力。
转换倾斜的连续特征
一个数据集有时可能包含至少一个靠近某个数字的特征，但有时也会有一些相对来说存在极大值或者极小值的不平凡分布的的特征。算法对这种分布的数据会十分敏感，并且如果这种数据没有能够很好地规一化处理会使得算法表现不佳。在人口普查数据集的两个特征符合这个描述：'capital-gain'和'capital-loss'。
运行下面的代码单元以创建一个关于这两个特征的条形图。请注意当前的值的范围和它们是如何分布的。
Can you write Python code for it?

# 将数据切分成特征和对应的标签
income_raw = data['income']
features_raw = data.drop('income', axis = 1)

# 可视化原来数据的倾斜的连续特征
vs.distribution(data)


I am working on the file "Project2-finding_donors/finding_donors.ipynb".
The first task is:
对于高度倾斜分布的特征如'capital-gain'和'capital-loss'，常见的做法是对数据施加一个对数转换，将数据转换成对数，这样非常大和非常小的值不会对学习算法产生负面的影响。并且使用对数变换显著降低了由于异常值所造成的数据范围异常。但是在应用这个变换时必须小心：因为0的对数是没有定义的，所以我们必须先将数据处理成一个比0稍微大一点的数以成功完成对数转换。
运行下面的代码单元来执行数据的转换和可视化结果。再次，注意值的范围和它们是如何分布的。
Can you write Python code for it?

# 对于倾斜的数据使用Log转换
skewed = ['capital-gain', 'capital-loss']
features_raw[skewed] = data[skewed].apply(lambda x: np.log(x + 1))

# 可视化经过log之后的数据分布
vs.distribution(features_raw, transformed = True)


I am working on the file "Project2-finding_donors/finding_donors.ipynb".
The first task is:
提高效果
在这最后一节中，您将从三个有监督的学习模型中选择最好的模型来使用学生数据。你将在整个训练集（X_train和y_train）上通过使用网格搜索优化至少调节一个参数以获得一个比没有调节之前更好的F-score。
问题 3 - 选择最佳的模型
基于你前面做的评价，用一到两段向CharityML解释这三个模型中哪一个对于判断被调查者的年收入大于\$50,000是最合适的。
提示：你的答案应该包括关于评价指标，预测/训练时间，以及该算法是否适合这里的数据的讨论。
回答：
- 在Time这个图表中，SVC的训练时间远高于其他两个模型；而在测试时间上，KNN的最高，其次是SVC，这两者都远高于AdaBoost。
- 在精度的上，AdaBoost在测试数据上的效果在数据量增加的时候表现越来越优秀，F-score也是。而且AdaBoost也没有overfit的倾向
因此我的结论是选择AdaBoost对于判断被调查者的年收入大于$50,000是最合适的
问题 4 - 用通俗的话解释模型
用一到两段话，向CharityML用外行也听得懂的话来解释最终模型是如何工作的。你需要解释所选模型的主要特点。例如，这个模型是怎样被训练的，它又是如何做出预测的。避免使用高级的数学或技术术语，不要使用公式或特定的算法名词。
回答：
要了解这个模型需要了解两个词：弱学习和权重
1. 弱学习：字面意义就是虽然弱但是还是比没有学要好。比如一样东西乱猜才对的概率是1/2的话，弱学习就是猜对的概率就能大于1/2
2. 权重：学习生活中怎样能快速提高自己，就是不断专注于自己模糊或不懂的地方，而不是熟悉的知识。这就是把更多的权重放在错误上(这是学习过程)。在选择上，我们需要给更权威的人更多话语权(选择过程)
3. AdaBoosting就是这样的工作原理。AdaBoosting是由很多弱分类器组成的。对于一个弱分类器，不断从错误中学习，提高自己的准确率(学习过程)。然后我们在从所有的分类器中，对预测效果好的给更多的权重(选择过程)，最后就得出一个比较好的结果
练习：模型调优
调节选择的模型的参数。使用网格搜索（GridSearchCV）来至少调整模型的重要参数（至少调整一个），这个参数至少需给出并尝试3个不同的值。你要使用整个训练集来完成这个过程。在接下来的代码单元中，你需要实现以下功能：

导入sklearn.grid_search.gridSearchCV和sklearn.metrics.make_scorer.
初始化你选择的分类器，并将其存储在clf中。
如果能够设置的话，设置random_state。
创建一个对于这个模型你希望调整参数的字典。
例如: parameters = {'parameter' : [list of values]}。
注意： 如果你的学习器（learner）有 max_features 参数，请不要调节它！
使用make_scorer来创建一个fbeta_score评分对象（设置$\beta = 0.5$）。
在分类器clf上用'scorer'作为评价函数运行网格搜索，并将结果存储在grid_obj中。
用训练集（X_train, y_train）训练grid search object,并将结果存储在grid_fit中。

注意： 取决于你选择的参数列表，下面实现的代码可能需要花一些时间运行！
Can you write Python code for it?

# 导入'GridSearchCV', 'make_scorer'和其他一些需要的库
from sklearn.grid_search import GridSearchCV
from sklearn.metrics import make_scorer
# 初始化分类器
clf = AdaBoostClassifier(random_state = 1)

# 创建你希望调节的参数列表
parameters = {'n_estimators': range(1, 200, 20)}

# 创建一个fbeta_score打分对象
scorer = make_scorer(fbeta_score, beta = 0.5)

# 在分类器上使用网格搜索，使用'scorer'作为评价函数
grid_obj = GridSearchCV(clf, parameters, scorer)

# 用训练数据拟合网格搜索对象并找到最佳参数
grid_obj.fit(X_train, y_train)
# 得到estimator
best_clf = grid_obj.best_estimator_
print best_clf
# 使用没有调优的模型做预测
predictions = (clf.fit(X_train, y_train)).predict(X_test)
best_predictions = best_clf.predict(X_test)

# 汇报调参前和调参后的分数
print "Unoptimized model\n------"
print "Accuracy score on testing data: {:.4f}".format(accuracy_score(y_test, predictions))
print "F-score on testing data: {:.4f}".format(fbeta_score(y_test, predictions, beta = 0.5))
print "\nOptimized Model\n------"
print "Final accuracy score on the testing data: {:.4f}".format(accuracy_score(y_test, best_predictions))
print "Final F-score on the testing data: {:.4f}".format(fbeta_score(y_test, best_predictions, beta = 0.5))


I am working on the file "Project2-finding_donors/finding_donors.ipynb".
The first task is:
练习：初始模型的评估
在下面的代码单元中，您将需要实现以下功能：
- 导入你在前面讨论的三个监督学习模型。
- 初始化三个模型并存储在'clf_A'，'clf_B'和'clf_C'中。
  - 如果可能对每一个模型都设置一个random_state。
  - 注意：这里先使用每一个模型的默认参数，在接下来的部分中你将需要对某一个模型的参数进行调整。
- 计算记录的数目等于1%，10%，和100%的训练数据，并将这些值存储在'samples'中
注意：取决于你选择的算法，下面实现的代码可能需要一些时间来运行！
Can you write Python code for it?

# 从sklearn中导入三个监督学习模型
from sklearn.ensemble import AdaBoostClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC

# 初始化三个模型
clf_A = AdaBoostClassifier(random_state = 1)
clf_B = KNeighborsClassifier()
clf_C = SVC(random_state = 1)

# 计算1%， 10%， 100%的训练数据分别对应多少点
length_train = len(X_train)
samples_1 = length_train / 100
samples_10 = length_train / 10
samples_100 = length_train
#print samples_1
# 收集学习器的结果
results = {}
for clf in [clf_A, clf_B, clf_C]:
    clf_name = clf.__class__.__name__
    results[clf_name] = {}
    for i, samples in enumerate([samples_1, samples_10, samples_100]):
        results[clf_name][i] = \
        train_predict(clf, samples, X_train, y_train, X_test, y_test)

# 对选择的三个模型得到的评价结果进行可视化
vs.evaluate(results, accuracy, fscore)


I am working on the file "Project2-finding_donors/finding_donors.ipynb".
The first task is:
练习：数据探索
首先我们对数据集进行一个粗略的探索，我们将看看每一个类别里会有多少被调查者？并且告诉我们这些里面多大比例是年收入大于50,000美元的。在下面的代码单元中，你将需要计算以下量：

总的记录数量，'n_records'
年收入大于50,000美元的人数，'n_greater_50k'.
年收入最多为50,000美元的人数 'n_at_most_50k'.
年收入大于50,000美元的人所占的比例， 'greater_percent'.

提示： 您可能需要查看上面的生成的表，以了解'income'条目的格式是什么样的。
Can you write Python code for it?

# 总的记录数
n_records = len(data.index)

# 被调查者的收入大于$50,000的人数
n_greater_50k = len(data[data['income'] == '>50K'])

# 被调查者的收入最多为$50,000的人数
n_at_most_50k = len(data[data['income'] == '<=50K'])

# 被调查者收入大于$50,000所占的比例
greater_percent = n_greater_50k / float(n_records) * 100

# 打印结果
print "Total number of records: {}".format(n_records)
print "Individuals making more than $50,000: {}".format(n_greater_50k)
print "Individuals making at most $50,000: {}".format(n_at_most_50k)
print "Percentage of individuals making more than $50,000: {:.2f}%".format(greater_percent)


I am working on the file "Project2-finding_donors/finding_donors.ipynb".
The first task is:
练习：数据预处理
从上面的数据探索中的表中，我们可以看到有几个属性的每一条记录都是非数字的。通常情况下，学习算法期望输入是数字的，这要求非数字的特征（称为类别变量）被转换。转换类别变量的一种流行的方法是使用独热编码方案。独热编码为每一个非数字特征的每一个可能的类别创建一个_“虚拟”_变量。例如，假设someFeature有三个可能的取值A，B或者C，。我们将把这个特征编码成someFeature_A, someFeature_B和someFeature_C.
|   | 一些特征 |                    | 特征_A | 特征_B | 特征_C |
| :-: | :-: |                            | :-: | :-: | :-: |
| 0 |  B  |  | 0 | 1 | 0 |
| 1 |  C  | ----> 独热编码 ----> | 0 | 0 | 1 |
| 2 |  A  |  | 1 | 0 | 0 |
此外，对于非数字的特征，我们需要将非数字的标签'income'转换成数值以保证学习算法能够正常工作。因为这个标签只有两种可能的类别（"<=50K"和">50K"），我们不必要使用独热编码，可以直接将他们编码分别成两个类0和1，在下面的代码单元中你将实现以下功能：
 - 使用pandas.get_dummies()对'features_raw'数据来施加一个独热编码。
 - 将目标标签'income_raw'转换成数字项。
   - 将"<=50K"转换成0；将">50K"转换成1。
Can you write Python code for it?

# 使用pandas.get_dummies()对'features_raw'数据进行独热编码
cols = ['workclass', 'education_level', 'marital-status', 'occupation', 'relationship', 'race', 'sex', 'native-country']
features = pd.get_dummies(features_raw, columns=cols)

# 将'income_raw'编码成数字值
income = income_raw.isin(['>50K'])*1

# 打印经过独热编码之后的特征数量
encoded = list(features.columns)
print "{} total features after one-hot encoding.".format(len(encoded))

#print features


I am working on the file "Project2-finding_donors/finding_donors.ipynb".
The first task is:
规一化数字特征
除了对于高度倾斜的特征施加转换，对数值特征施加一些形式的缩放通常会是一个好的习惯。在数据上面施加一个缩放并不会改变数据分布的形式（比如上面说的'capital-gain' or 'capital-loss'）；但是，规一化保证了每一个特征在使用监督学习器的时候能够被平等的对待。注意一旦使用了缩放，观察数据的原始形式不再具有它本来的意义了，就像下面的例子展示的。
运行下面的代码单元来规一化每一个数字特征。我们将使用sklearn.preprocessing.MinMaxScaler来完成这个任务。
Can you write Python code for it?

# 导入sklearn.preprocessing.StandardScaler
from sklearn.preprocessing import MinMaxScaler

# 初始化一个 scaler，并将它施加到特征上
scaler = MinMaxScaler()
numerical = ['age', 'education-num', 'capital-gain', 'capital-loss', 'hours-per-week']
features_raw[numerical] = scaler.fit_transform(data[numerical])

# 显示一个经过缩放的样例记录
display(features_raw.head(n = 1))


I am working on the file "Project2/finding_donors.ipynb".
The first task is:
对于高度倾斜分布的特征如'capital-gain'和'capital-loss'，常见的做法是对数据施加一个对数转换，将数据转换成对数，这样非常大和非常小的值不会对学习算法产生负面的影响。并且使用对数变换显著降低了由于异常值所造成的数据范围异常。但是在应用这个变换时必须小心：因为0的对数是没有定义的，所以我们必须先将数据处理成一个比0稍微大一点的数以成功完成对数转换。
运行下面的代码单元来执行数据的转换和可视化结果。再次，注意值的范围和它们是如何分布的。
Can you write Python code for it?

# 对于倾斜的数据使用Log转换
skewed = ['capital-gain', 'capital-loss']
features_raw[skewed] = data[skewed].apply(lambda x: np.log(x + 1))

# 可视化对数转换后 'capital-gain'和'capital-loss' 两个特征
vs.distribution(features_raw, transformed = True)


I am working on the file "Project2/finding_donors.ipynb".
The first task is:
混洗和切分数据
现在所有的 类别变量 已被转换成数值特征，而且所有的数值特征已被规一化。和我们一般情况下做的一样，我们现在将数据（包括特征和它们的标签）切分成训练和测试集。其中80%的数据将用于训练和20%的数据用于测试。然后再进一步把训练数据分为训练集和验证集，用来选择和优化模型。
运行下面的代码单元来完成切分。
Can you write Python code for it?

# 导入 train_test_split
from sklearn.model_selection import train_test_split

# 将'features'和'income'数据切分成训练集和测试集
X_train, X_test, y_train, y_test = train_test_split(features, income, test_size = 0.2, random_state = 0,
                                                    stratify = income)
# 将'X_train'和'y_train'进一步切分为训练集和验证集
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.2, random_state=0,
                                                    stratify = y_train)
# 显示切分的结果
print "Training set has {} samples.".format(X_train.shape[0])
print "Validation set has {} samples.".format(X_val.shape[0])
print "Testing set has {} samples.".format(X_test.shape[0])


I am working on the file "Project2/finding_donors.ipynb".
The first task is:
练习：初始模型的评估
在下面的代码单元中，您将需要实现以下功能：
- 导入你在前面讨论的三个监督学习模型。
- 初始化三个模型并存储在'clf_A'，'clf_B'和'clf_C'中。
  - 使用模型的默认参数值，在接下来的部分中你将需要对某一个模型的参数进行调整。
  - 设置random_state  (如果有这个参数)。
- 计算1%， 10%， 100%的训练数据分别对应多少个数据点，并将这些值存储在'samples_1', 'samples_10', 'samples_100'中
注意：取决于你选择的算法，下面实现的代码可能需要一些时间来运行！
Can you write Python code for it?

# TODO：从sklearn中导入三个监督学习模型
from sklearn import tree
from sklearn import svm
from sklearn.neural_network import MLPClassifier
# TODO：初始化三个模型
clf_A = tree.DecisionTreeClassifier(random_state=1)
clf_B = svm.SVC(random_state=1)
clf_C = MLPClassifier(solver='lbfgs', alpha=1e-5,hidden_layer_sizes=(5, 2), random_state=1)

# TODO：计算1%， 10%， 100%的训练数据分别对应多少点
samples_1 = len(X_train)/100
samples_10 = len(X_train)/10
samples_100 = len(X_train)

# 收集学习器的结果
results = {}
for clf in [clf_A, clf_B, clf_C]:
    clf_name = clf.__class__.__name__
    results[clf_name] = {}
    for i, samples in enumerate([samples_1, samples_10, samples_100]):
        results[clf_name][i] = train_predict(clf, samples, X_train, y_train, X_val, y_val)

# 对选择的三个模型得到的评价结果进行可视化
vs.evaluate(results, accuracy, fscore)


I am working on the file "Project2/finding_donors.ipynb".
The first task is:
练习：数据探索
首先我们对数据集进行一个粗略的探索，我们将看看每一个类别里会有多少被调查者？并且告诉我们这些里面多大比例是年收入大于50,000美元的。在下面的代码单元中，你将需要计算以下量：

总的记录数量，'n_records'
年收入大于50,000美元的人数，'n_greater_50k'.
年收入最多为50,000美元的人数 'n_at_most_50k'.
年收入大于50,000美元的人所占的比例， 'greater_percent'.

提示： 您可能需要查看上面的生成的表，以了解'income'条目的格式是什么样的。
Can you write Python code for it?

# TODO：总的记录数
n_records = len(data)

# TODO：被调查者的收入大于$50,000的人数
n_greater_50k = len(data[data['income']=='>50K'])

# TODO：被调查者的收入最多为$50,000的人数
n_at_most_50k = len(data[data['income']=='<=50K'])

# TODO：被调查者收入大于$50,000所占的比例
greater_percent = n_greater_50k / float(n_records) * 100

# 打印结果
print "Total number of records: {}".format(n_records)
print "Individuals making more than $50,000: {}".format(n_greater_50k)
print "Individuals making at most $50,000: {}".format(n_at_most_50k)
print "Percentage of individuals making more than $50,000: {:.2f}%".format(greater_percent)


I am working on the file "Project2/finding_donors.ipynb".
The first task is:
练习：数据预处理
从上面的数据探索中的表中，我们可以看到有几个属性的每一条记录都是非数字的。通常情况下，学习算法期望输入是数字的，这要求非数字的特征（称为类别变量）被转换。转换类别变量的一种流行的方法是使用独热编码方案。独热编码为每一个非数字特征的每一个可能的类别创建一个_“虚拟”_变量。例如，假设someFeature有三个可能的取值A，B或者C，。我们将把这个特征编码成someFeature_A, someFeature_B和someFeature_C.
| 特征X |                    | 特征X_A | 特征X_B | 特征X_C |
| :-: |                            | :-: | :-: | :-: |
|  B  |  | 0 | 1 | 0 |
|  C  | ----> 独热编码 ----> | 0 | 0 | 1 |
|  A  |  | 1 | 0 | 0 |
此外，对于非数字的特征，我们需要将非数字的标签'income'转换成数值以保证学习算法能够正常工作。因为这个标签只有两种可能的类别（"<=50K"和">50K"），我们不必要使用独热编码，可以直接将他们编码分别成两个类0和1，在下面的代码单元中你将实现以下功能：
 - 使用pandas.get_dummies()对'features_raw'数据来施加一个独热编码。
 - 将目标标签'income_raw'转换成数字项。
   - 将"<=50K"转换成0；将">50K"转换成1。
Can you write Python code for it?

# TODO：使用pandas.get_dummies()对'features_raw'数据进行独热编码
features = pd.get_dummies(features_raw)

# TODO：将'income_raw'编码成数字值
income = income_raw.map(lambda x: 0 if x == '<=50K' else 1)

# print income.head(n=9)

# 打印经过独热编码之后的特征数量
encoded = list(features.columns)
print "{} total features after one-hot encoding.".format(len(encoded))

# 移除下面一行的注释以观察编码的特征名字
print encoded


I am working on the file "Project2/finding_donors.ipynb".
The first task is:
评价模型性能
在这一部分中，我们将尝试四种不同的算法，并确定哪一个能够最好地建模数据。四种算法包含一个天真的预测器 和三个你选择的监督学习器。
评价方法和朴素的预测器
CharityML通过他们的研究人员知道被调查者的年收入大于\$50,000最有可能向他们捐款。因为这个原因CharityML对于准确预测谁能够获得\$50,000以上收入尤其有兴趣。这样看起来使用准确率作为评价模型的标准是合适的。另外，把没有收入大于\$50,000的人识别成年收入大于\$50,000对于CharityML来说是有害的，因为他想要找到的是有意愿捐款的用户。这样，我们期望的模型具有准确预测那些能够年收入大于\$50,000的能力比模型去查全这些被调查者更重要。我们能够使用F-beta score作为评价指标，这样能够同时考虑查准率和查全率：
$$ F_{\beta} = (1 + \beta^2) \cdot \frac{precision \cdot recall}{\left( \beta^2 \cdot precision \right) + recall} $$
尤其是，当 $\beta = 0.5$ 的时候更多的强调查准率，这叫做F$_{0.5}$ score （或者为了简单叫做F-score）。
问题 1 - 天真的预测器的性能
通过查看收入超过和不超过 \$50,000 的人数，我们能发现多数被调查者年收入没有超过 \$50,000。如果我们简单地预测说“这个人的收入没有超过 \$50,000”，我们就可以得到一个 准确率超过 50% 的预测。这样我们甚至不用看数据就能做到一个准确率超过 50%。这样一个预测被称作是天真的。通常对数据使用一个天真的预测器是十分重要的，这样能够帮助建立一个模型表现是否好的基准。 使用下面的代码单元计算天真的预测器的相关性能。将你的计算结果赋值给'accuracy', ‘precision’, ‘recall’ 和 'fscore'，这些值会在后面被使用，请注意这里不能使用scikit-learn，你需要根据公式自己实现相关计算。
如果我们选择一个无论什么情况都预测被调查者年收入大于 \$50,000 的模型，那么这个模型在验证集上的准确率，查准率，查全率和 F-score是多少？
Can you write Python code for it?

#不能使用scikit-learn，你需要根据公式自己实现相关计算。

# 不知道这里是不是将y_val传过来就可以了，请指导下

income_pred = y_val.apply(lambda x : 1)

TP = sum(map(lambda x,y:1 if x==1 and y==1 else 0,y_val,income_pred))
FN = sum(map(lambda x,y:1 if x==1 and y==0 else 0,y_val,income_pred))
FP = sum(map(lambda x,y:1 if x==0 and y==1 else 0,y_val,income_pred))
TN = sum(map(lambda x,y:1 if x==0 and y==0 else 0,y_val,income_pred))

print TP
print FN
print FP
print TN

#TODO： 计算准确率
accuracy = float(TP + TN)/len(y_val)

# TODO： 计算查准率 Precision
precision = TP/float(TP + FP)

# TODO： 计算查全率 Recall
recall = TP/float(TP + FN)

# TODO： 使用上面的公式，设置beta=0.5，计算F-score
fscore = (1 + 0.5*0.5)*(precision * recall)/(0.5*0.5*precision + recall)

# 打印结果
print "Naive Predictor on validation data: \n \
    Accuracy score: {:.4f} \n \
    Precision: {:.4f} \n \
    Recall: {:.4f} \n \
    F-score: {:.4f}".format(accuracy, precision, recall, fscore)


I am working on the file "Project2/finding_donors.ipynb".
The first task is:
问题 7 - 提取特征重要性
观察上面创建的展示五个用于预测被调查者年收入是否大于\$50,000最相关的特征的可视化图像。
这五个特征的权重加起来是否超过了0.5?

这五个特征和你在问题 6中讨论的特征比较怎么样？

如果说你的答案和这里的相近，那么这个可视化怎样佐证了你的想法？

如果你的选择不相近，那么为什么你觉得这些特征更加相关？
回答：超过了
有些相似，但是整体不准确
我选取的特征，一个是一些基本属性，而且很有可能影响其收入或同理心，但是数据表现的却十分冷酷，是否捐款和赚钱花钱有最大的关系。
特征选择
如果我们只是用可用特征的一个子集的话模型表现会怎么样？通过使用更少的特征来训练，在评价指标的角度来看我们的期望是训练和预测的时间会更少。从上面的可视化来看，我们可以看到前五个最重要的特征贡献了数据中所有特征中超过一半的重要性。这提示我们可以尝试去减小特征空间，简化模型需要学习的信息。下面代码单元将使用你前面发现的优化模型，并只使用五个最重要的特征在相同的训练集上训练模型。
Can you write Python code for it?

# 导入克隆模型的功能
from sklearn.base import clone

# 减小特征空间
X_train_reduced = X_train[X_train.columns.values[(np.argsort(importances)[::-1])[:5]]]
X_val_reduced = X_val[X_val.columns.values[(np.argsort(importances)[::-1])[:5]]]

# 在前面的网格搜索的基础上训练一个“最好的”模型
clf_on_reduced = (clone(best_clf)).fit(X_train_reduced, y_train)

# 做一个新的预测
reduced_predictions = clf_on_reduced.predict(X_val_reduced)

# 对于每一个版本的数据汇报最终模型的分数
print "Final Model trained on full data\n------"
print "Accuracy on validation data: {:.4f}".format(accuracy_score(y_val, best_predictions))
print "F-score on validation data: {:.4f}".format(fbeta_score(y_val, best_predictions, beta = 0.5))
print "\nFinal Model trained on reduced data\n------"
print "Accuracy on validation data: {:.4f}".format(accuracy_score(y_val, reduced_predictions))
print "F-score on validation data: {:.4f}".format(fbeta_score(y_val, reduced_predictions, beta = 0.5))


I am working on the file "Project2/finding_donors.ipynb".
The first task is:
问题 8 - 特征选择的影响
最终模型在只是用五个特征的数据上和使用所有的特征数据上的 F-score 和准确率相比怎么样？
如果训练时间是一个要考虑的因素，你会考虑使用部分特征的数据作为你的训练集吗？
回答：均有下降
如果在数据比较大、硬件资源比较匮乏的时候，我会考虑使用，因为选取主要特征的方法会极大提高训练速度
但是再中小型数据或者说硬件资源足够时，我会尽量保证其准确性，一个良好准确的模型的训练时间损耗是值得的
问题 9 - 在测试集上测试你的模型
终于到了测试的时候，记住，测试集只能用一次。
使用你最有信心的模型，在测试集上测试，计算出准确率和 F-score。
简述你选择这个模型的原因，并分析测试结果
Can you write Python code for it?

#TODO test your model on testing data and report accuracy and F score

final_predictions = best_clf.predict(X_test)

print "最终准确率: {:.4f}".format(accuracy_score(y_test, final_predictions))
print "最终F-Score: {:.4f}".format(fbeta_score(y_test, final_predictions, beta = 0.5))


I am working on the file "Project2/image-classification/dlnd_image_classification.ipynb".
The first task is:
Checkpoint
The model has been saved to disk.
Test Model
Test your model against the test dataset.  This will be your final accuracy. You should have an accuracy greater than 50%. If you don't, keep tweaking the model architecture and parameters.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import tensorflow as tf
import pickle
import helper
import random

# Set batch size if not already set
try:
    if batch_size:
        pass
except NameError:
    batch_size = 64

save_model_path = './image_classification'
n_samples = 4
top_n_predictions = 3

def test_model():
    """
    Test the saved model against the test dataset
    """

    test_features, test_labels = pickle.load(open('preprocess_test.p', mode='rb'))
    loaded_graph = tf.Graph()

    with tf.Session(graph=loaded_graph) as sess:
        # Load model
        loader = tf.train.import_meta_graph(save_model_path + '.meta')
        loader.restore(sess, save_model_path)

        # Get Tensors from loaded model
        loaded_x = loaded_graph.get_tensor_by_name('x:0')
        loaded_y = loaded_graph.get_tensor_by_name('y:0')
        loaded_keep_prob = loaded_graph.get_tensor_by_name('keep_prob:0')
        loaded_logits = loaded_graph.get_tensor_by_name('logits:0')
        loaded_acc = loaded_graph.get_tensor_by_name('accuracy:0')

        # Get accuracy in batches for memory limitations
        test_batch_acc_total = 0
        test_batch_count = 0

        for test_feature_batch, test_label_batch in helper.batch_features_labels(test_features, test_labels, batch_size):
            test_batch_acc_total += sess.run(
                loaded_acc,
                feed_dict={loaded_x: test_feature_batch, loaded_y: test_label_batch, loaded_keep_prob: 1.0})
            test_batch_count += 1

        print('Testing Accuracy: {}\n'.format(test_batch_acc_total/test_batch_count))

        # Print Random Samples
        random_test_features, random_test_labels = tuple(zip(*random.sample(list(zip(test_features, test_labels)), n_samples)))
        random_test_predictions = sess.run(
            tf.nn.top_k(tf.nn.softmax(loaded_logits), top_n_predictions),
            feed_dict={loaded_x: random_test_features, loaded_y:...


I am working on the file "Project2/image-classification/dlnd_image_classification.ipynb".
The first task is:
Convolution and Max Pooling Layer
Convolution layers have a lot of success with images. For this code cell, you should implement the function conv2d_maxpool to apply convolution then max pooling:
* Create the weight and bias using conv_ksize, conv_num_outputs and the shape of x_tensor.
* Apply a convolution to x_tensor using weight and conv_strides.
 * We recommend you use same padding, but you're welcome to use any padding.
* Add bias
* Add a nonlinear activation to the convolution.
* Apply Max Pooling using pool_ksize and pool_strides.
 * We recommend you use same padding, but you're welcome to use any padding.
Note: You can't use TensorFlow Layers or TensorFlow Layers (contrib) for this layer, but you can still use TensorFlow's Neural Network package. You may still use the shortcut option for all the other layers.
Can you write Python code for it?

def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides):
    """
    Apply convolution then max pooling to x_tensor
    :param x_tensor: TensorFlow Tensor
    :param conv_num_outputs: Number of outputs for the convolutional layer
    :param conv_ksize: kernal size 2-D Tuple for the convolutional layer
    :param conv_strides: Stride 2-D Tuple for convolution
    :param pool_ksize: kernal size 2-D Tuple for pool
    :param pool_strides: Stride 2-D Tuple for pool
    : return: A tensor that represents convolution and max pooling of x_tensor
    """
    # TODO: Implement Function
#     print(x_tensor.shape)
#     print(conv_ksize)
#     print(conv_num_outputs)

    color_channels = x_tensor.shape[3].value
    weights = tf.Variable(tf.truncated_normal([conv_ksize[0], conv_ksize[1], color_channels, conv_num_outputs], mean=0, stddev=0.1))

    biases = tf.Variable(tf.zeros(conv_num_outputs))

    layer = tf.nn.conv2d(x_tensor, weights, strides=[1, conv_strides[0], conv_strides[1], 1], padding='SAME')

    layer = tf.add(layer, biases)

    layer = tf.nn.relu(layer)

    layer = tf.nn.max_pool(layer, ksize=[1, pool_ksize[0], pool_ksize[1], 1],
                           strides=[1, pool_strides[0], pool_strides[1], 1], padding='SAME')

    return layer

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_con_pool(conv2d_maxpool)


I am working on the file "Project2/image-classification/dlnd_image_classification.ipynb".
The first task is:
Fully Train the Model
Now that you got a good accuracy with a single CIFAR-10 batch, try it with all five batches.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
save_model_path = './image_classification'

print('Training...')
with tf.Session() as sess:
    # Initializing the variables
    sess.run(tf.global_variables_initializer())

    # Training cycle
    for epoch in range(epochs):
        # Loop over all batches
        n_batches = 5
        for batch_i in range(1, n_batches + 1):
            for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):
                train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)
            print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')
            print_stats(sess, batch_features, batch_labels, cost, accuracy)

    # Save Model
    saver = tf.train.Saver()
    save_path = saver.save(sess, save_model_path)


I am working on the file "Project2/image-classification/dlnd_image_classification.ipynb".
The first task is:
Randomize Data
As you saw from exploring the data above, the order of the samples are randomized.  It doesn't hurt to randomize it again, but you don't need to for this dataset.
Preprocess all the data and save it
Running the code cell below will preprocess all the CIFAR-10 data and save it to file. The code below also uses 10% of the training data for validation.
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# Preprocess Training, Validation, and Testing Data
helper.preprocess_and_save_data(cifar10_dataset_folder_path, normalize, one_hot_encode)


I am working on the file "Project2/image-classification/dlnd_image_classification.ipynb".
The first task is:
Train the Neural Network
Single Optimization
Implement the function train_neural_network to do a single optimization.  The optimization should use optimizer to optimize in session with a feed_dict of the following:
* x for image input
* y for labels
* keep_prob for keep probability for dropout
This function will be called for each batch, so tf.global_variables_initializer() has already been called.
Note: Nothing needs to be returned. This function is only optimizing the neural network.
Can you write Python code for it?

def train_neural_network(session, optimizer, keep_probability, feature_batch, label_batch):
    """
    Optimize the session on a batch of images and labels
    : session: Current TensorFlow session
    : optimizer: TensorFlow optimizer function
    : keep_probability: keep probability
    : feature_batch: Batch of Numpy image data
    : label_batch: Batch of Numpy label data
    """
    # TODO: Implement Function
    session.run(optimizer, feed_dict={
        x: feature_batch,
        y: label_batch,
        keep_prob: keep_probability})

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_train_nn(train_neural_network)


I am working on the file "Project3/Generate_TV_Scripts/dlnd_tv_script_generation.ipynb".
The first task is:
Batches
Implement get_batches to create batches of input and targets using int_text.  The batches should be a Numpy array with the shape (number of batches, 2, batch size, sequence length). Each batch contains two elements:
- The first element is a single batch of input with the shape [batch size, sequence length]
- The second element is a single batch of targets with the shape [batch size, sequence length]
If you can't fill the last batch with enough data, drop the last batch.
For exmple, get_batches([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20], 3, 2) would return a Numpy array of the following:
```
[
  # First Batch
  [
    # Batch of Input
    [[ 1  2], [ 7  8], [13 14]]
    # Batch of targets
    [[ 2  3], [ 8  9], [14 15]]
  ]
# Second Batch
  [
    # Batch of Input
    [[ 3  4], [ 9 10], [15 16]]
    # Batch of targets
    [[ 4  5], [10 11], [16 17]]
  ]
# Third Batch
  [
    # Batch of Input
    [[ 5  6], [11 12], [17 18]]
    # Batch of targets
    [[ 6  7], [12 13], [18  1]]
  ]
]
```
Notice that the last target value in the last batch is the first input value of the first batch. In this case, 1. This is a common technique used when creating sequence batches, although it is rather unintuitive.
Can you write Python code for it?

def get_batches(int_text, batch_size, seq_length):
    """
    Return batches of input and target
    :param int_text: Text with the words replaced by their ids
    :param batch_size: The size of batch
    :param seq_length: The length of sequence
    :return: Batches as a Numpy array
    """
    # TODO: Implement Function

    n_batches = int(len(int_text) // (batch_size * seq_length))
    int_text_x = np.array(int_text[:n_batches * (batch_size * seq_length)])
    int_text_y = np.append(np.array(int_text[1:n_batches * (batch_size * seq_length)]), int_text[0])

    int_text_sequences = [int_text_x[i*seq_length:i*seq_length+seq_length] for i in range(0, n_batches * batch_size)]

    int_text_targets = [int_text_y[i*seq_length:i*seq_length+seq_length] for i in range(0, n_batches * batch_size)]
    output = []
    for batch in range(n_batches):
        inputs = []
        targets = []
        for size in range(batch_size):
            inputs.append(int_text_sequences[size * n_batches + batch])
            targets.append(int_text_targets[size * n_batches + batch])
        output.append([inputs, targets])
    return np.array(output)

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_get_batches(get_batches)


I am working on the file "Project3/Generate_TV_Scripts/dlnd_tv_script_generation.ipynb".
The first task is:
Build RNN
You created a RNN Cell in the get_init_cell() function.  Time to use the cell to create a RNN.
- Build the RNN using the tf.nn.dynamic_rnn()
 - Apply the name "final_state" to the final state using tf.identity()
Return the outputs and final_state state in the following tuple (Outputs, FinalState)
Can you write Python code for it?

def build_rnn(cell, inputs):
    """
    Create a RNN using a RNN Cell
    :param cell: RNN Cell
    :param inputs: Input text data
    :return: Tuple (Outputs, Final State)
    """
    # TODO: Implement Function
    outputs, final_state = tf.nn.dynamic_rnn(cell, inputs, dtype=tf.float32)
    final_state = tf.identity(final_state, name="final_state")
    return outputs, final_state

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_build_rnn(build_rnn)


I am working on the file "Project3/Generate_TV_Scripts/dlnd_tv_script_generation.ipynb".
The first task is:
Build RNN Cell and Initialize
Stack one or more BasicLSTMCells in a MultiRNNCell.
- The Rnn size should be set using rnn_size
- Initalize Cell State using the MultiRNNCell's zero_state() function
    - Apply the name "initial_state" to the initial state using tf.identity()
Return the cell and initial state in the following tuple (Cell, InitialState)
Can you write Python code for it?

def get_init_cell(batch_size, rnn_size):
    """
    Create an RNN Cell and initialize it.
    :param batch_size: Size of batches
    :param rnn_size: Size of RNNs
    :return: Tuple (cell, initialize state)
    """
    # TODO: Implement Function

    lstm = tf.contrib.rnn.BasicLSTMCell(rnn_size)

    # Stack up multiple LSTM layers, for deep learning
    cell = tf.contrib.rnn.MultiRNNCell([lstm] * 1)

    # Getting an initial state of all zeros
    initial_state = cell.zero_state(batch_size, tf.float32)
    initial_state = tf.identity(initial_state, name = "initial_state")

    return cell, initial_state

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_get_init_cell(get_init_cell)


I am working on the file "Project3/Generate_TV_Scripts/dlnd_tv_script_generation.ipynb".
The first task is:
Build the Neural Network
Apply the functions you implemented above to:
- Apply embedding to input_data using your get_embed(input_data, vocab_size, embed_dim) function.
- Build RNN using cell and your build_rnn(cell, inputs) function.
- Apply a fully connected layer with a linear activation and vocab_size as the number of outputs.
Return the logits and final state in the following tuple (Logits, FinalState)
Can you write Python code for it?

def build_nn(cell, rnn_size, input_data, vocab_size, embed_dim):
    """
    Build part of the neural network
    :param cell: RNN cell
    :param rnn_size: Size of rnns
    :param input_data: Input data
    :param vocab_size: Vocabulary size
    :param embed_dim: Number of embedding dimensions
    :return: Tuple (Logits, FinalState)
    """
    # TODO: Implement Function
    embed = get_embed(input_data, vocab_size, embed_dim)
    output, final_state = build_rnn(cell, embed)
    predictions = tf.contrib.layers.fully_connected(output, vocab_size, activation_fn=None)
    return predictions, final_state

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_build_nn(build_nn)


I am working on the file "Project3/Generate_TV_Scripts/dlnd_tv_script_generation.ipynb".
The first task is:
Checkpoint
Can you write Python code for it?

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import tensorflow as tf
import numpy as np
import helper
import problem_unittests as tests

_, vocab_to_int, int_to_vocab, token_dict = helper.load_preprocess()
seq_length, load_dir = helper.load_params()


I am working on the file "Project3/Generate_TV_Scripts/dlnd_tv_script_generation.ipynb".
The first task is:
Choose Word
Implement the pick_word() function to select the next word using probabilities.
Can you write Python code for it?

def pick_word(probabilities, int_to_vocab):
    """
    Pick the next word in the generated text
    :param probabilities: Probabilites of the next word
    :param int_to_vocab: Dictionary of word ids as the keys and words as the values
    :return: String of the predicted word
    """
    # TODO: Implement Function
    return int_to_vocab[np.argmax(probabilities)]

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_pick_word(pick_word)