I am working on the file "SHRUG_pres_Gaboardi.ipynb".
The first task is:

Network-Based Model Building for

Discrete Location Allocation in Python:

Integrating PySAL and Gurobi

James D. Gaboardi

David C. Folch

Florida State University

Department of Geography

SHRUG 2015

1. Optimization in GIS
       • Imports
       • What is Spatial Optimization?
       • Reproducibility
       • Software
2. Demonstration of PySAL + Gurobi
       • Imports & Specs
       •  Data Preparation & Creation
       •  Mathematical Optimization
             → Set Cover Location Problem
             →  p-Median Problem
             →  p-Center Problem
             →  p-Center Problem (Manual LP Creation)
       •  Visualization of Selected Locations
3. Next Steps

Imports
Can you write Python code for it?

import datetime as dt
import gurobipy as gbp
import IPython.display as IPd
import os
import platform
import pysal as ps
import sys
import time

# Local path on user's machine
path = '/Users/jgaboardi/SHRUG__15/'


I am working on the file "SHRUG_pres_Gaboardi.ipynb".
The first task is:
<u>Self-intersecting Road</u>
Can you write Python code for it?

# Millstream Road
Millstream_Rd = IPd.Image(path+'/Millstream.jpg')
Millstream_Rd


I am working on the file "SHRUG_pres_Gaboardi.ipynb".
The first task is:
Gurobi 6.5
Relatively new company founded by optimization experts formerly at key positions with CPLEX.
[http://www.gurobi.com] [http://www.gurobi.com/company/about-gurobi]
Gurobipy
Python wrapper for Gurobi
Can you write Python code for it?


I am working on the file "SHRUG_pres_Gaboardi.ipynb".
The first task is:
Loop Road
Can you write Python code for it?

# Avon Circle
Avon_Cir = IPd.Image(path+'/Avon.jpg')
Avon_Cir


I am working on the file "SHRUG_pres_Gaboardi.ipynb".
The first task is:
Network Characteristics
Can you write Python code for it?

print '\nNodes in original network:   ', len(ntw.nodes)
print 'Edges in original network:   ', len(ntw.edges)
print '\nNodes in simplified network: ', len(g1.node)
print 'Edges in simplified network: ', len(ntw.graphedges)


I am working on the file "SHRUG_pres_Gaboardi.ipynb".
The first task is:
PySAL.Network
PySAL.Network was principally developed by Jay Laura at Arizona State Universty and the United States Geological Suvery.  [https://geoplan.asu.edu/people/jay-laura]
Can you write Python code for it?


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
Applying the classification model to new data
Now that we have a trained facies classification model we can use it to identify facies in wells that do not have core data.  In this case, we will apply the classifier to two wells, but we could use it on any number of wells for which we have the same set of well logs for input.
This dataset is similar to the training data except it does not have facies labels.  It is loaded into a dataframe called test_data.
Can you write Python code for it?

well_data = pd.read_csv('../validation_data_nofacies.csv')
well_data['Well Name'] = well_data['Well Name'].astype('category')
well_features = well_data.drop(['Formation', 'Well Name', 'Depth'], axis=1)


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
Applying the classification model to the blind data
We held a well back from the training, and stored it in a dataframe called blind:
Can you write Python code for it?


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
As noted above, the boundaries between the facies classes are not all sharp, and some of them blend into one another.  The error within these 'adjacent facies' can also be calculated.  We define an array to represent the facies adjacent to each other.  For facies label i, adjacent_facies[i] is an array of the adjacent facies labels.
Can you write Python code for it?

adjacent_facies = np.array([[1], [0,2], [1], [4], [3,5], [4,6,7], [5,7], [5,6,8], [6,7]])

def accuracy_adjacent(conf, adjacent_facies):
    nb_classes = conf.shape[0]
    total_correct = 0.
    for i in np.arange(0,nb_classes):
        total_correct += conf[i][i]
        for j in adjacent_facies[i]:
            total_correct += conf[i][j]
    return total_correct / sum(sum(conf))

print('Facies classification accuracy = %f' % accuracy(conf))
print('Adjacent facies classification accuracy = %f' % accuracy_adjacent(conf, adjacent_facies))


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
Conditioning the data set
Now we extract just the feature variables we need to perform the classification.  The predictor variables are the five wireline values and two geologic constraining variables. We also get a vector of the facies labels that correspond to each feature vector.
Can you write Python code for it?

correct_facies_labels = training_data['Facies'].values

feature_vectors = training_data.drop(['Formation', 'Well Name', 'Depth','Facies','FaciesLabels'], axis=1)
feature_vectors.describe()


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
Finally we predict facies labels for the unknown data, and store the results in a Facies column of the test_data dataframe.
Can you write Python code for it?

#predict facies of unclassified data
y_unknown = clf.predict(X_unknown)
well_data['Facies'] = y_unknown
well_data

well_data['Well Name'].unique()


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
Let's see how we did with the confusion matrix:
Can you write Python code for it?

cv_conf = confusion_matrix(y_blind, y_pred)

print('Optimized facies classification accuracy = %.2f' % accuracy(cv_conf))
print('Optimized adjacent facies classification accuracy = %.2f' % accuracy_adjacent(cv_conf, adjacent_facies))


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
Let's take a look at the data from individual wells in a more familiar log plot form.  We will create plots for the five well log variables, as well as a log for facies labels.  The plots are based on the those described in Alessandro Amato del Monte's excellent tutorial.
Can you write Python code for it?

def make_facies_log_plot(logs, facies_colors):
    #make sure logs are sorted by depth
    logs = logs.sort_values(by='Depth')
    cmap_facies = colors.ListedColormap(
            facies_colors[0:len(facies_colors)], 'indexed')

    ztop=logs.Depth.min(); zbot=logs.Depth.max()

    cluster=np.repeat(np.expand_dims(logs['Facies'].values,1), 100, 1)

    f, ax = plt.subplots(nrows=1, ncols=6, figsize=(8, 12))
    ax[0].plot(logs.GR, logs.Depth, '-g')
    ax[1].plot(logs.ILD_log10, logs.Depth, '-')
    ax[2].plot(logs.DeltaPHI, logs.Depth, '-', color='0.5')
    ax[3].plot(logs.PHIND, logs.Depth, '-', color='r')
    ax[4].plot(logs.PE, logs.Depth, '-', color='black')
    im=ax[5].imshow(cluster, interpolation='none', aspect='auto',
                    cmap=cmap_facies,vmin=1,vmax=9)

    divider = make_axes_locatable(ax[5])
    cax = divider.append_axes("right", size="20%", pad=0.05)
    cbar=plt.colorbar(im, cax=cax)
    cbar.set_label((17*' ').join([' SS ', 'CSiS', 'FSiS',
                                'SiSh', ' MS ', ' WS ', ' D  ',
                                ' PS ', ' BS ']))
    cbar.set_ticks(range(0,1)); cbar.set_ticklabels('')

    for i in range(len(ax)-1):
        ax[i].set_ylim(ztop,zbot)
        ax[i].invert_yaxis()
        ax[i].grid()
        ax[i].locator_params(axis='x', nbins=3)

    ax[0].set_xlabel("GR")
    ax[0].set_xlim(logs.GR.min(),logs.GR.max())
    ax[1].set_xlabel("ILD_log10")
    ax[1].set_xlim(logs.ILD_log10.min(),logs.ILD_log10.max())
    ax[2].set_xlabel("DeltaPHI")
    ax[2].set_xlim(logs.DeltaPHI.min(),logs.DeltaPHI.max())
    ax[3].set_xlabel("PHIND")
    ax[3].set_xlim(logs.PHIND.min(),logs.PHIND.max())
    ax[4].set_xlabel("PE")
    ax[4].set_xlim(logs.PE.min(),logs.PE.max())
    ax[5].set_xlabel('Facies')

    ax[1].set_yticklabels([]); ax[2].set_yticklabels([]); ax[3].set_yticklabels([])
    ax[4].set_yticklabels([]); ax[5].set_yticklabels([])
    ax[5].set_xticklabels([])
    f.suptitle('Well: %s'%logs.iloc[0]['Well Name'], fontsize=14,y=0.94)


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
Now we can train the classifier using the training set we created above.
Can you write Python code for it?


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
Placing the log plotting code in a function will make it easy to plot the logs from multiples wells, and can be reused later to view the results when we apply the facies classification model to other wells.  The function was written to take a list of colors and facies labels as parameters.
We then show log plots for wells SHRIMPLIN.
Can you write Python code for it?

make_facies_log_plot(
    training_data[training_data['Well Name'] == 'SHRIMPLIN'],
    facies_colors)


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
Scikit also includes a handy function to randomly split the training data into training and test sets.  The test set contains a small subset of feature vectors that are not used to train the network.  Because we know the true facies labels for these examples, we can compare the results of the classifier to the actual facies and determine the accuracy of the model.  Let's use 20% of the data for the test set.
Can you write Python code for it?

from sklearn.cross_validation import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
        scaled_features, correct_facies_labels, test_size=0.1, random_state=42)


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
The data needs to be scaled using the same constants we used for the training data.
Can you write Python code for it?


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
The results are 0.59 accuracy on facies classification of blind data and 0.92 adjacent facies classification.
Can you write Python code for it?

display_cm(cv_conf, facies_labels,
           display_metrics=True, hide_zeros=True)


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
The rows of the confusion matrix correspond to the actual facies labels.  The columns correspond to the labels assigned by the classifier.  For example, consider the first row. For the feature vectors in the test set that actually have label SS, 18 were correctly indentified as SS, 5 were classified as CSiS and 1 was classified as FSiS.
The entries along the diagonal are the facies that have been correctly classified.  Below we define two functions that will give an overall value for how the algorithm is performing.  The accuracy is defined as the number of correct classifications divided by the total number of classifications.
Can you write Python code for it?

def accuracy(conf):
    total_correct = 0.
    nb_classes = conf.shape[0]
    for i in np.arange(0,nb_classes):
        total_correct += conf[i][i]
    acc = total_correct/sum(sum(conf))
    return acc


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
This is a quick view of the statistical distribution of the input variables.  Looking at the count values, there are 3232 feature vectors in the training set.
Remove a single well to use as a blind test later.
Can you write Python code for it?

blind = training_data[training_data['Well Name'] == 'SHANKLE']
training_data = training_data[training_data['Well Name'] != 'SHANKLE']


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
Training the classifier using Logistic Regression
Now we use the cleaned and conditioned training set to create a facies classifier.  As mentioned above, we will use a type of machine learning model known as a Logistic Regression.  For our test we used the one vs rest and sag solver as data has already been scaled.
Can you write Python code for it?

from sklearn import linear_model
clf = linear_model.LogisticRegression(class_weight='balanced',multi_class='ovr',solver='sag',max_iter=1000,random_state=40,C=1e5)


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
We can use the well log plot to view the classification results along with the well logs.
Can you write Python code for it?

make_facies_log_plot(
    well_data[well_data['Well Name'] == 'STUART'],
    facies_colors=facies_colors)

make_facies_log_plot(
    well_data[well_data['Well Name'] == 'CRAWFORD'],
    facies_colors=facies_colors)


I am working on the file "SHandPR/FaciesTrial.ipynb".
The first task is:
We need some metrics to evaluate how good our classifier is doing.  A confusion matrix is a table that can be used to describe the performance of a classification model.  Scikit-learn allows us to easily create a confusion matrix by supplying the actual and predicted facies labels.
The confusion matrix is simply a 2D array.  The entries of confusion matrix C[i][j] are equal to the number of observations predicted to have facies j, but are known to have facies i.
To simplify reading the confusion matrix, a function has been written to display the matrix along with facies labels and various error metrics.  See the file classification_utilities.py in this repo for the display_cm() function.
Can you write Python code for it?

from sklearn.metrics import confusion_matrix
from classification_utilities import display_cm, display_adj_cm

conf = confusion_matrix(y_test, predicted_labels)
display_cm(conf, facies_labels, hide_zeros=True)


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
...but does remarkably well on the adjacent facies predictions.
Can you write Python code for it?

display_adj_cm(cv_conf, facies_labels, adjacent_facies,
               display_metrics=True, hide_zeros=True)

def compare_facies_plot(logs, compadre, facies_colors):
    #make sure logs are sorted by depth
    logs = logs.sort_values(by='Depth')
    cmap_facies = colors.ListedColormap(
            facies_colors[0:len(facies_colors)], 'indexed')

    ztop=logs.Depth.min(); zbot=logs.Depth.max()

    cluster1 = np.repeat(np.expand_dims(logs['Facies'].values,1), 100, 1)
    cluster2 = np.repeat(np.expand_dims(logs[compadre].values,1), 100, 1)

    f, ax = plt.subplots(nrows=1, ncols=7, figsize=(9, 12))
    ax[0].plot(logs.GR, logs.Depth, '-g')
    ax[1].plot(logs.ILD_log10, logs.Depth, '-')
    ax[2].plot(logs.DeltaPHI, logs.Depth, '-', color='0.5')
    ax[3].plot(logs.PHIND, logs.Depth, '-', color='r')
    ax[4].plot(logs.PE, logs.Depth, '-', color='black')
    im1 = ax[5].imshow(cluster1, interpolation='none', aspect='auto',
                    cmap=cmap_facies,vmin=1,vmax=9)
    im2 = ax[6].imshow(cluster2, interpolation='none', aspect='auto',
                    cmap=cmap_facies,vmin=1,vmax=9)

    divider = make_axes_locatable(ax[6])
    cax = divider.append_axes("right", size="20%", pad=0.05)
    cbar=plt.colorbar(im2, cax=cax)
    cbar.set_label((17*' ').join([' SS ', 'CSiS', 'FSiS',
                                'SiSh', ' MS ', ' WS ', ' D  ',
                                ' PS ', ' BS ']))
    cbar.set_ticks(range(0,1)); cbar.set_ticklabels('')

    for i in range(len(ax)-2):
        ax[i].set_ylim(ztop,zbot)
        ax[i].invert_yaxis()
        ax[i].grid()
        ax[i].locator_params(axis='x', nbins=3)

    ax[0].set_xlabel("GR")
    ax[0].set_xlim(logs.GR.min(),logs.GR.max())
    ax[1].set_xlabel("ILD_log10")
    ax[1].set_xlim(logs.ILD_log10.min(),logs.ILD_log10.max())
    ax[2].set_xlabel("DeltaPHI")
    ax[2].set_xlim(logs.DeltaPHI.min(),logs.DeltaPHI.max())
    ax[3].set_xlabel("PHIND")
    ax[3].set_xlim(logs.PHIND.min(),logs.PHIND.max())
   ...


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
Applying the classification model to the blind data
We held a well back from the training, and stored it in a dataframe called blind:
Can you write Python code for it?


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
As noted above, the boundaries between the facies classes are not all sharp, and some of them blend into one another.  The error within these 'adjacent facies' can also be calculated.  We define an array to represent the facies adjacent to each other.  For facies label i, adjacent_facies[i] is an array of the adjacent facies labels.
Can you write Python code for it?


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
Conditioning the data set
Now we extract just the feature variables we need to perform the classification.  The predictor variables are the five wireline values and two geologic constraining variables. We also get a vector of the facies labels that correspond to each feature vector.
Can you write Python code for it?


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
In addition to individual wells, we can look at how the various facies are represented by the entire training set.  Let's plot a histogram of the number of training examples for each facies class.
Can you write Python code for it?

#count the number of unique entries for each facies, sort them by
#facies number (instead of by number of entries)
facies_counts = training_data['Facies'].value_counts().sort_index()
#use facies labels to index each count
facies_counts.index = facies_labels

facies_counts.plot(kind='bar',color=facies_colors,
                   title='Distribution of Training Data by Facies')
facies_counts


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
Let's see how we did with the confusion matrix:
Can you write Python code for it?


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
Now it's a simple matter of making a prediction and storing it back in the dataframe:
Can you write Python code for it?

y_pred = gbModel.predict(X_blind)
blind['Prediction'] = y_pred


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
Now that the model has been trained on our data, we can use it to predict the facies of the feature vectors in the test set.
Can you write Python code for it?


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
Now we can train the classifier using the training set we created above.
Can you write Python code for it?


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
Scikit also includes a handy function to randomly split the training data into training and test sets.  The test set contains a small subset of feature vectors that are not used to train the network.  Because we know the true facies labels for these examples, we can compare the results of the classifier to the actual facies and determine the accuracy of the model.  Let's use 20% of the data for the test set.
Can you write Python code for it?


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
Scikit includes a preprocessing module that can 'standardize' the data (giving each variable zero mean and unit variance, also called whitening).  Many machine learning algorithms assume features will be standard normally distributed data (ie: Gaussian with zero mean and unit variance).  The factors used to standardize the training set must be applied to any subsequent feature set that will be input to the classifier.  The StandardScalar class can be fit to the training set, and later used to standardize any training data.
Can you write Python code for it?

from sklearn import preprocessing

scaler = preprocessing.StandardScaler().fit(feature_vectors)
scaled_features = scaler.transform(feature_vectors)

feature_vectors


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
The data needs to be scaled using the same constants we used for the training data.
Can you write Python code for it?


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
The label vector is just the Facies column:
Can you write Python code for it?


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
These are the names of the 10 training wells in the Council Grove reservoir.  Data has been recruited into pseudo-well 'Recruit F9' to better represent facies 9, the Phylloid-algal bafflestone.
Before we plot the well data, let's define a color map so the facies are represented by consistent color in all the plots in this tutorial.  We also create the abbreviated facies labels, and add those to the facies_vectors dataframe.
Can you write Python code for it?

# 1=sandstone  2=c_siltstone   3=f_siltstone
# 4=marine_silt_shale 5=mudstone 6=wackestone 7=dolomite
# 8=packstone 9=bafflestone
facies_colors = ['#F4D03F', '#F5B041','#DC7633','#6E2C00',
       '#1B4F72','#2E86C1', '#AED6F1', '#A569BD', '#196F3D']

facies_labels = ['SS', 'CSiS', 'FSiS', 'SiSh', 'MS',
                 'WS', 'D','PS', 'BS']
#facies_color_map is a dictionary that maps facies labels
#to their respective colors
facies_color_map = {}
for ind, label in enumerate(facies_labels):
    facies_color_map[label] = facies_colors[ind]

def label_facies(row, labels):
    return labels[ row['Facies'] -1]

training_data.loc[:,'FaciesLabels'] = training_data.apply(lambda row: label_facies(row, facies_labels), axis=1)


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
This is a quick view of the statistical distribution of the input variables.  Looking at the count values, there are 3232 feature vectors in the training set.
Remove a single well to use as a blind test later.
Can you write Python code for it?


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
Training the classifier using Gradient Boosting
Now we use the cleaned and conditioned training set to create a facies classifier.  As mentioned above, we will use a type of machine learning model known as a Gradient Boosting.
Can you write Python code for it?

from sklearn.ensemble import GradientBoostingClassifier
gbModel = GradientBoostingClassifier(loss='deviance', n_estimators=100, learning_rate=0.1, max_depth=3, random_state=None, max_leaf_nodes=None, verbose=1)


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
We can form the feature matrix by dropping some of the columns and making a new dataframe:
Can you write Python code for it?

well_features = blind.drop(['Facies', 'Formation', 'Well Name', 'Depth'], axis=1)


I am working on the file "SHandPR/GradientBoosting.ipynb".
The first task is:
We can use the well log plot to view the classification results along with the well logs.
Can you write Python code for it?


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
...but does remarkably well on the adjacent facies predictions.
Can you write Python code for it?


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Applying the classification model to new data
Now that we have a trained facies classification model we can use it to identify facies in wells that do not have core data.  In this case, we will apply the classifier to two wells, but we could use it on any number of wells for which we have the same set of well logs for input.
This dataset is similar to the training data except it does not have facies labels.  It is loaded into a dataframe called test_data.
Can you write Python code for it?

well_data = pd.read_csv('validation_data_nofacies.csv')
well_data['Well Name'] = well_data['Well Name'].astype('category')
well_features = well_data.drop(['Formation', 'Well Name', 'Depth'], axis=1)


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Applying the classification model to the blind data
We held a well back from the training, and stored it in a dataframe called blind:
Can you write Python code for it?


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Based on graphs above NEWBY has a good distribution of wells and is taken out as blind well to be tested. Also since training data has null, remove them from data.
Can you write Python code for it?

PE_mask = training_data['PE'].notnull().values
training_data = training_data[PE_mask]
blind = training_data[training_data['Well Name'] == 'NEWBY']
training_data = training_data[training_data['Well Name'] != 'NEWBY']
training_data.loc[:,'FaciesLabels'] = training_data.apply(lambda row: label_facies(row, facies_labels), axis=1)


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Finally we can write out a csv file with the well data along with the facies classification results.
Can you write Python code for it?


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Finally we predict facies labels for the unknown data, and store the results in a Facies column of the test_data dataframe.
Can you write Python code for it?

#predict facies of unclassified data
y_unknown = vtclf.predict(X_unknown)
vtclf.score(X_unknown)
well_data['Facies'] = y_unknown
well_data

well_data['Well Name'].unique()


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
In addition to individual wells, we can look at how the various facies are represented by the entire training set.  Let's plot a histogram of the number of training examples for each facies class.
Can you write Python code for it?


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Let's see how we did with the confusion matrix:
Can you write Python code for it?

cv_conf = confusion_matrix(y_blind, y_pred)

print('Optimized facies classification accuracy = %.2f' % accuracy(cv_conf))
print('Optimized adjacent facies classification accuracy = %.2f' % accuracy_adjacent(cv_conf, adjacent_facies))


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Let's take a look at the data from individual wells in a more familiar log plot form.  We will create plots for the five well log variables, as well as a log for facies labels.  The plots are based on the those described in Alessandro Amato del Monte's excellent tutorial.
Can you write Python code for it?

def make_facies_log_plot(logs, facies_colors):
    #make sure logs are sorted by depth
    logs = logs.sort_values(by='Depth')
    cmap_facies = colors.ListedColormap(
            facies_colors[0:len(facies_colors)], 'indexed')

    ztop=logs.Depth.min(); zbot=logs.Depth.max()

    cluster=np.repeat(np.expand_dims(logs['Facies'].values,1), 100, 1)

    f, ax = plt.subplots(nrows=1, ncols=6, figsize=(8, 12))
    ax[0].plot(logs.GR, logs.Depth, '-g')
    ax[1].plot(logs.ILD_log10, logs.Depth, '-')
    ax[2].plot(logs.DeltaPHI, logs.Depth, '-', color='0.5')
    ax[3].plot(logs.PHIND, logs.Depth, '-', color='r')
    ax[4].plot(logs.PE, logs.Depth, '-', color='black')
    im=ax[5].imshow(cluster, interpolation='none', aspect='auto',
                    cmap=cmap_facies,vmin=1,vmax=9)

    divider = make_axes_locatable(ax[5])
    cax = divider.append_axes("right", size="20%", pad=0.05)
    cbar=plt.colorbar(im, cax=cax)
    cbar.set_label((17*' ').join([' SS ', 'CSiS', 'FSiS',
                                'SiSh', ' MS ', ' WS ', ' D  ',
                                ' PS ', ' BS ']))
    cbar.set_ticks(range(0,1)); cbar.set_ticklabels('')

    for i in range(len(ax)-1):
        ax[i].set_ylim(ztop,zbot)
        ax[i].invert_yaxis()
        ax[i].grid()
        ax[i].locator_params(axis='x', nbins=3)

    ax[0].set_xlabel("GR")
    ax[0].set_xlim(logs.GR.min(),logs.GR.max())
    ax[1].set_xlabel("ILD_log10")
    ax[1].set_xlim(logs.ILD_log10.min(),logs.ILD_log10.max())
    ax[2].set_xlabel("DeltaPHI")
    ax[2].set_xlim(logs.DeltaPHI.min(),logs.DeltaPHI.max())
    ax[3].set_xlabel("PHIND")
    ax[3].set_xlim(logs.PHIND.min(),logs.PHIND.max())
    ax[4].set_xlabel("PE")
    ax[4].set_xlim(logs.PE.min(),logs.PE.max())
    ax[5].set_xlabel('Facies')

    ax[1].set_yticklabels([]); ax[2].set_yticklabels([]); ax[3].set_yticklabels([])
    ax[4].set_yticklabels([]); ax[5].set_yticklabels([])
    ax[5].set_xticklabels([])
    f.suptitle('Well: %s'%logs.iloc[0]['Well Name'], fontsize=14,y=0.94)


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Now that the model has been trained on our data, we can use it to predict the facies of the feature vectors in the test set.  Because we know the true facies labels of the vectors in the test set, we can use the results to evaluate the accuracy of the classifier.
We need some metrics to evaluate how good our classifier is doing.  A confusion matrix is a table that can be used to describe the performance of a classification model.  Scikit-learn allows us to easily create a confusion matrix by supplying the actual and predicted facies labels.
The confusion matrix is simply a 2D array.  The entries of confusion matrix C[i][j] are equal to the number of observations predicted to have facies j, but are known to have facies i.
To simplify reading the confusion matrix, a function has been written to display the matrix along with facies labels and various error metrics.  See the file classification_utilities.py in this repo for the display_cm() function.
Can you write Python code for it?


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Now we can transform this with the scaler we made before:
Can you write Python code for it?


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Placing the log plotting code in a function will make it easy to plot the logs from multiples wells, and can be reused later to view the results when we apply the facies classification model to other wells.  The function was written to take a list of colors and facies labels as parameters.
We then show log plots for wells SHRIMPLIN.
Can you write Python code for it?

make_facies_log_plot(
    training_data[training_data['Well Name'] == 'SHRIMPLIN'],
    facies_colors)


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Scikit also includes a handy function to randomly split the training data into training and test sets.  The test set contains a small subset of feature vectors that are not used to train the network.  Because we know the true facies labels for these examples, we can compare the results of the classifier to the actual facies and determine the accuracy of the model.  Let's use 20% of the data for the test set.
Can you write Python code for it?

from sklearn.cross_validation import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
        scaled_features, correct_facies_labels, test_size=0.2, random_state=42)


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
The label vector is just the Facies column:
Can you write Python code for it?


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
This data is from the Council Grove gas reservoir in Southwest Kansas.  The Panoma Council Grove Field is predominantly a carbonate gas reservoir encompassing 2700 square miles in Southwestern Kansas.  This dataset is from nine wells (with 4149 examples), consisting of a set of seven predictor variables and a rock facies (class) for each example vector and validation (test) data (830 examples from two wells) having the same seven predictor variables in the feature vector.  Facies are based on examination of cores from nine wells taken vertically at half-foot intervals. Predictor variables include five from wireline log measurements and two geologic constraining variables that are derived from geologic knowledge. These are essentially continuous variables sampled at a half-foot sample rate.
The seven predictor variables are:
* Five wire line log curves include gamma ray (GR), resistivity logging (ILD_log10),
photoelectric effect (PE), neutron-density porosity difference and average neutron-density porosity (DeltaPHI and PHIND). Note, some wells do not have PE.
* Two geologic constraining variables: nonmarine-marine indicator (NM_M) and relative position (RELPOS)
The nine discrete facies (classes of rocks) are:
1. Nonmarine sandstone
2. Nonmarine coarse siltstone
3. Nonmarine fine siltstone
4. Marine siltstone and shale
5. Mudstone (limestone)
6. Wackestone (limestone)
7. Dolomite
8. Packstone-grainstone (limestone)
9. Phylloid-algal bafflestone (limestone)
These facies aren't discrete, and gradually blend into one another. Some have neighboring facies that are rather close.  Mislabeling within these neighboring facies can be expected to occur.  The following table lists the facies, their abbreviated labels and their approximate neighbors.
Facies |Label| Adjacent Facies
:---: | :---: |:--:
1 |SS| 2
2 |CSiS| 1,3
3 |FSiS| 2
4 |SiSh| 5
5 |MS| 4,6
6 |WS| 5,7
7 |D| 6,8
8 |PS| 6,7,9
9 |BS| 7,8
Let's clean up this dataset.  The 'Well Name' and 'Formation' columns can be turned into a categorical data type.
Can you write Python code for it?

training_data['Well Name'] = training_data['Well Name'].astype('category')
training_data['Formation'] = training_data['Formation'].astype('category')
training_data['Well Name'].unique()

training_data.describe()


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
This is a quick view of the statistical distribution of the input variables.  Looking at the count values, there are 3232 feature vectors in the training set.
Remove a single well to use as a blind test later. For that let us look at distribution of facies across wells
Can you write Python code for it?

# 1=sandstone  2=c_siltstone   3=f_siltstone
# 4=marine_silt_shale 5=mudstone 6=wackestone 7=dolomite
# 8=packstone 9=bafflestone
facies_colors = ['#F4D03F', '#F5B041','#DC7633','#6E2C00',
       '#1B4F72','#2E86C1', '#AED6F1', '#A569BD', '#196F3D']

facies_labels = ['SS', 'CSiS', 'FSiS', 'SiSh', 'MS',
                 'WS', 'D','PS', 'BS']
#facies_color_map is a dictionary that maps facies labels
#to their respective colors
facies_color_map = {}
for ind, label in enumerate(facies_labels):
    facies_color_map[label] = facies_colors[ind]

def label_facies(row, labels):
    return labels[ row['Facies'] -1]

#training_data.loc[:,'FaciesLabels'] = training_data.apply(lambda row: label_facies(row, facies_labels), axis=1)
faciesVals = training_data['Facies'].values
well = training_data['Well Name'].values
mpl.rcParams['figure.figsize'] = (20.0, 10.0)
for w_idx, w in enumerate(np.unique(well)):
    ax = plt.subplot(3, 4, w_idx+1)
    hist = np.histogram(faciesVals[well == w], bins=np.arange(len(facies_labels)+1)+.5)
    plt.bar(np.arange(len(hist[0])), hist[0], color=facies_colors, align='center')
    ax.set_xticks(np.arange(len(hist[0])))
    ax.set_xticklabels(facies_labels)
    ax.set_title(w)


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
This shows the distribution of examples by facies for the examples in the training set.  Dolomite (facies 7) has the fewest with 81 examples.  Depending on the performance of the classifier we are going to train, we may consider getting more examples of these facies.
Conditioning the data set
Now we extract just the feature variables we need to perform the classification.  The predictor variables are the five wireline values and two geologic constraining variables. We also get a vector of the facies labels that correspond to each feature vector.
Can you write Python code for it?

correct_facies_labels = training_data['Facies'].values

feature_vectors = training_data.drop(['Formation', 'Well Name', 'Depth','Facies','FaciesLabels'], axis=1)
feature_vectors.describe()


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Training the classifier using Majority voting
Now we use the cleaned and conditioned training set to create a facies classifier.  As mentioned above, we will use a type of machine learning model known as a Majority voting.
We trained classifier on four models KNeighbours, Random forest, logistic regression and Gradient boosting.
Can you write Python code for it?

from sklearn import neighbors
clf = neighbors.KNeighborsClassifier(n_neighbors=10,weights='distance',algorithm='kd_tree')
clf.fit(X_train,y_train)
predicted_labels = clf.predict(X_test)


from sklearn import neighbors
clf = neighbors.KNeighborsClassifier(n_neighbors=20,
                                     weights='distance',
                                     algorithm='kd_tree',
                                     leaf_size=30,
                                     metric='minkowski',
                                     p=1)
clf.fit(X_train,y_train)
predicted_labels = clf.predict(X_test)


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
Using Voting classifier Now
The voting classifier is now used to vote and classify models
Can you write Python code for it?

from sklearn.ensemble import VotingClassifier
vtclf = VotingClassifier(estimators=[
                        ('KNN', clf), ('RFC', RFC), ('GBM', gbModel),('LR',lgr)],
                        voting='hard',
                        weights=[2,2,1,1])
vtclf.fit(X_train,y_train)
vtclfpredicted_labels = vtclf.predict(X_test)
vtclfconf = confusion_matrix(y_test, vtclfpredicted_labels)
display_cm(vtclfconf, facies_labels, hide_zeros=True)
print('Facies classification accuracy = %f' % accuracy(vtclfconf))
print('Adjacent facies classification accuracy = %f' % accuracy_adjacent(vtclfconf, adjacent_facies))


from sklearn.ensemble import VotingClassifier
vtclf = VotingClassifier(estimators=[
        ('KNN', clf), ('RFC', RFC), ('GBM', gbModel),('LR',lgr)],
                        voting='soft')
vtclf.fit(X_train,y_train)
vtclfpredicted_labels = vtclf.predict(X_test)
vtclfconf = confusion_matrix(y_test, vtclfpredicted_labels)
display_cm(vtclfconf, facies_labels, hide_zeros=True)
print('Facies classification accuracy = %f' % accuracy(vtclfconf))
print('Adjacent facies classification accuracy = %f' % accuracy_adjacent(vtclfconf, adjacent_facies))


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
We can form the feature matrix by dropping some of the columns and making a new dataframe:
Can you write Python code for it?


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
We can use the well log plot to view the classification results along with the well logs.
Can you write Python code for it?

make_facies_log_plot(
    well_data[well_data['Well Name'] == 'STUART'],
    facies_colors=facies_colors)

make_facies_log_plot(
    well_data[well_data['Well Name'] == 'CRAWFORD'],
    facies_colors=facies_colors)


I am working on the file "SHandPR/MajorityVoting.ipynb".
The first task is:
We managed 0.46 using the test data, but it was from the same wells as the training data. T
Can you write Python code for it?

display_cm(cv_conf, facies_labels,
           display_metrics=True, hide_zeros=True)


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
...but does remarkably well on the adjacent facies predictions.
Can you write Python code for it?

display_adj_cm(cv_conf, facies_labels, adjacent_facies,
               display_metrics=True, hide_zeros=True)

def compare_facies_plot(logs, compadre, facies_colors):
    #make sure logs are sorted by depth
    logs = logs.sort_values(by='Depth')
    cmap_facies = colors.ListedColormap(
            facies_colors[0:len(facies_colors)], 'indexed')

    ztop=logs.Depth.min(); zbot=logs.Depth.max()

    cluster1 = np.repeat(np.expand_dims(logs['Facies'].values,1), 100, 1)
    cluster2 = np.repeat(np.expand_dims(logs[compadre].values,1), 100, 1)

    f, ax = plt.subplots(nrows=1, ncols=7, figsize=(9, 12))
    ax[0].plot(logs.GR, logs.Depth, '-g')
    ax[1].plot(logs.ILD_log10, logs.Depth, '-')
    ax[2].plot(logs.DeltaPHI, logs.Depth, '-', color='0.5')
    ax[3].plot(logs.PHIND, logs.Depth, '-', color='r')
    ax[4].plot(logs.PE, logs.Depth, '-', color='black')
    im1 = ax[5].imshow(cluster1, interpolation='none', aspect='auto',
                    cmap=cmap_facies,vmin=1,vmax=9)
    im2 = ax[6].imshow(cluster2, interpolation='none', aspect='auto',
                    cmap=cmap_facies,vmin=1,vmax=9)

    divider = make_axes_locatable(ax[6])
    cax = divider.append_axes("right", size="20%", pad=0.05)
    cbar=plt.colorbar(im2, cax=cax)
    cbar.set_label((17*' ').join([' SS ', 'CSiS', 'FSiS',
                                'SiSh', ' MS ', ' WS ', ' D  ',
                                ' PS ', ' BS ']))
    cbar.set_ticks(range(0,1)); cbar.set_ticklabels('')

    for i in range(len(ax)-2):
        ax[i].set_ylim(ztop,zbot)
        ax[i].invert_yaxis()
        ax[i].grid()
        ax[i].locator_params(axis='x', nbins=3)

    ax[0].set_xlabel("GR")
    ax[0].set_xlim(logs.GR.min(),logs.GR.max())
    ax[1].set_xlabel("ILD_log10")
    ax[1].set_xlim(logs.ILD_log10.min(),logs.ILD_log10.max())
    ax[2].set_xlabel("DeltaPHI")
    ax[2].set_xlim(logs.DeltaPHI.min(),logs.DeltaPHI.max())
    ax[3].set_xlabel("PHIND")
    ax[3].set_xlim(logs.PHIND.min(),logs.PHIND.max())
   ...


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Applying the classification model to new data
Now that we have a trained facies classification model we can use it to identify facies in wells that do not have core data.  In this case, we will apply the classifier to two wells, but we could use it on any number of wells for which we have the same set of well logs for input.
This dataset is similar to the training data except it does not have facies labels.  It is loaded into a dataframe called test_data.
Can you write Python code for it?

well_data = pd.read_csv('validation_data_nofacies.csv')
well_data['Well Name'] = well_data['Well Name'].astype('category')
well_features = well_data.drop(['Formation', 'Well Name', 'Depth'], axis=1)


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Applying the classification model to the blind data
We held a well back from the training, and stored it in a dataframe called blind:
Can you write Python code for it?


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
As noted above, the boundaries between the facies classes are not all sharp, and some of them blend into one another.  The error within these 'adjacent facies' can also be calculated.  We define an array to represent the facies adjacent to each other.  For facies label i, adjacent_facies[i] is an array of the adjacent facies labels.
Can you write Python code for it?

adjacent_facies = np.array([[1], [0,2], [1], [4], [3,5], [4,6,7], [5,7], [5,6,8], [6,7]])

def accuracy_adjacent(conf, adjacent_facies):
    nb_classes = conf.shape[0]
    total_correct = 0.
    for i in np.arange(0,nb_classes):
        total_correct += conf[i][i]
        for j in adjacent_facies[i]:
            total_correct += conf[i][j]
    return total_correct / sum(sum(conf))

print('Facies classification accuracy = %f' % accuracy(conf))
print('Adjacent facies classification accuracy = %f' % accuracy_adjacent(conf, adjacent_facies))


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Finally we can write out a csv file with the well data along with the facies classification results.
Can you write Python code for it?


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Finally we predict facies labels for the unknown data, and store the results in a Facies column of the test_data dataframe.
Can you write Python code for it?

#predict facies of unclassified data
y_unknown = clf.predict(X_unknown)
well_data['Facies'] = y_unknown
well_data

well_data['Well Name'].unique()


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
In addition to individual wells, we can look at how the various facies are represented by the entire training set.  Let's plot a histogram of the number of training examples for each facies class.
Can you write Python code for it?

#count the number of unique entries for each facies, sort them by
#facies number (instead of by number of entries)
facies_counts = training_data['Facies'].value_counts().sort_index()
#use facies labels to index each count
facies_counts.index = facies_labels

facies_counts.plot(kind='bar',color=facies_colors,
                   title='Distribution of Training Data by Facies')
facies_counts


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Let's see how we did with the confusion matrix:
Can you write Python code for it?

cv_conf = confusion_matrix(y_blind, y_pred)

print('Optimized facies classification accuracy = %.2f' % accuracy(cv_conf))
print('Optimized adjacent facies classification accuracy = %.2f' % accuracy_adjacent(cv_conf, adjacent_facies))


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Let's take a look at the data from individual wells in a more familiar log plot form.  We will create plots for the five well log variables, as well as a log for facies labels.  The plots are based on the those described in Alessandro Amato del Monte's excellent tutorial.
Can you write Python code for it?

def make_facies_log_plot(logs, facies_colors):
    #make sure logs are sorted by depth
    logs = logs.sort_values(by='Depth')
    cmap_facies = colors.ListedColormap(
            facies_colors[0:len(facies_colors)], 'indexed')

    ztop=logs.Depth.min(); zbot=logs.Depth.max()

    cluster=np.repeat(np.expand_dims(logs['Facies'].values,1), 100, 1)

    f, ax = plt.subplots(nrows=1, ncols=6, figsize=(8, 12))
    ax[0].plot(logs.GR, logs.Depth, '-g')
    ax[1].plot(logs.ILD_log10, logs.Depth, '-')
    ax[2].plot(logs.DeltaPHI, logs.Depth, '-', color='0.5')
    ax[3].plot(logs.PHIND, logs.Depth, '-', color='r')
    ax[4].plot(logs.PE, logs.Depth, '-', color='black')
    im=ax[5].imshow(cluster, interpolation='none', aspect='auto',
                    cmap=cmap_facies,vmin=1,vmax=9)

    divider = make_axes_locatable(ax[5])
    cax = divider.append_axes("right", size="20%", pad=0.05)
    cbar=plt.colorbar(im, cax=cax)
    cbar.set_label((17*' ').join([' SS ', 'CSiS', 'FSiS',
                                'SiSh', ' MS ', ' WS ', ' D  ',
                                ' PS ', ' BS ']))
    cbar.set_ticks(range(0,1)); cbar.set_ticklabels('')

    for i in range(len(ax)-1):
        ax[i].set_ylim(ztop,zbot)
        ax[i].invert_yaxis()
        ax[i].grid()
        ax[i].locator_params(axis='x', nbins=3)

    ax[0].set_xlabel("GR")
    ax[0].set_xlim(logs.GR.min(),logs.GR.max())
    ax[1].set_xlabel("ILD_log10")
    ax[1].set_xlim(logs.ILD_log10.min(),logs.ILD_log10.max())
    ax[2].set_xlabel("DeltaPHI")
    ax[2].set_xlim(logs.DeltaPHI.min(),logs.DeltaPHI.max())
    ax[3].set_xlabel("PHIND")
    ax[3].set_xlim(logs.PHIND.min(),logs.PHIND.max())
    ax[4].set_xlabel("PE")
    ax[4].set_xlim(logs.PE.min(),logs.PE.max())
    ax[5].set_xlabel('Facies')

    ax[1].set_yticklabels([]); ax[2].set_yticklabels([]); ax[3].set_yticklabels([])
    ax[4].set_yticklabels([]); ax[5].set_yticklabels([])
    ax[5].set_xticklabels([])
    f.suptitle('Well: %s'%logs.iloc[0]['Well Name'], fontsize=14,y=0.94)


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Now it's a simple matter of making a prediction and storing it back in the dataframe:
Can you write Python code for it?

y_pred = clf.predict(X_blind)
blind['Prediction'] = y_pred


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Now that the model has been trained on our data, we can use it to predict the facies of the feature vectors in the test set.
Can you write Python code for it?


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Now we can train the classifier using the training set we created above.
Can you write Python code for it?


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Now we can transform this with the scaler we made before:
Can you write Python code for it?


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Placing the log plotting code in a function will make it easy to plot the logs from multiples wells, and can be reused later to view the results when we apply the facies classification model to other wells.  The function was written to take a list of colors and facies labels as parameters.
We then show log plots for wells SHRIMPLIN.
Can you write Python code for it?

make_facies_log_plot(
    training_data[training_data['Well Name'] == 'SHRIMPLIN'],
    facies_colors)


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Scikit also includes a handy function to randomly split the training data into training and test sets.  The test set contains a small subset of feature vectors that are not used to train the network.  Because we know the true facies labels for these examples, we can compare the results of the classifier to the actual facies and determine the accuracy of the model.  Let's use 20% of the data for the test set.
Can you write Python code for it?

from sklearn.cross_validation import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
        scaled_features, correct_facies_labels, test_size=0.1, random_state=42)


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Scikit includes a preprocessing module that can 'standardize' the data (giving each variable zero mean and unit variance, also called whitening).  Many machine learning algorithms assume features will be standard normally distributed data (ie: Gaussian with zero mean and unit variance).  The factors used to standardize the training set must be applied to any subsequent feature set that will be input to the classifier.  The StandardScalar class can be fit to the training set, and later used to standardize any training data.
Can you write Python code for it?

from sklearn import preprocessing

scaler = preprocessing.StandardScaler().fit(feature_vectors)
scaled_features = scaler.transform(feature_vectors)

feature_vectors


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
The data needs to be scaled using the same constants we used for the training data.
Can you write Python code for it?


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
The label vector is just the Facies column:
Can you write Python code for it?


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
The results are 0.43 accuracy on facies classification of blind data and 0.87 adjacent facies classification.
Can you write Python code for it?


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
The rows of the confusion matrix correspond to the actual facies labels.  The columns correspond to the labels assigned by the classifier.  For example, consider the first row. For the feature vectors in the test set that actually have label SS, 18 were correctly indentified as SS, 5 were classified as CSiS and 1 was classified as FSiS.
The entries along the diagonal are the facies that have been correctly classified.  Below we define two functions that will give an overall value for how the algorithm is performing.  The accuracy is defined as the number of correct classifications divided by the total number of classifications.
Can you write Python code for it?

def accuracy(conf):
    total_correct = 0.
    nb_classes = conf.shape[0]
    for i in np.arange(0,nb_classes):
        total_correct += conf[i][i]
    acc = total_correct/sum(sum(conf))
    return acc


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
This data is from the Council Grove gas reservoir in Southwest Kansas.  The Panoma Council Grove Field is predominantly a carbonate gas reservoir encompassing 2700 square miles in Southwestern Kansas.  This dataset is from nine wells (with 4149 examples), consisting of a set of seven predictor variables and a rock facies (class) for each example vector and validation (test) data (830 examples from two wells) having the same seven predictor variables in the feature vector.  Facies are based on examination of cores from nine wells taken vertically at half-foot intervals. Predictor variables include five from wireline log measurements and two geologic constraining variables that are derived from geologic knowledge. These are essentially continuous variables sampled at a half-foot sample rate.
The seven predictor variables are:
* Five wire line log curves include gamma ray (GR), resistivity logging (ILD_log10),
photoelectric effect (PE), neutron-density porosity difference and average neutron-density porosity (DeltaPHI and PHIND). Note, some wells do not have PE.
* Two geologic constraining variables: nonmarine-marine indicator (NM_M) and relative position (RELPOS)
The nine discrete facies (classes of rocks) are:
1. Nonmarine sandstone
2. Nonmarine coarse siltstone
3. Nonmarine fine siltstone
4. Marine siltstone and shale
5. Mudstone (limestone)
6. Wackestone (limestone)
7. Dolomite
8. Packstone-grainstone (limestone)
9. Phylloid-algal bafflestone (limestone)
These facies aren't discrete, and gradually blend into one another. Some have neighboring facies that are rather close.  Mislabeling within these neighboring facies can be expected to occur.  The following table lists the facies, their abbreviated labels and their approximate neighbors.
Facies |Label| Adjacent Facies
:---: | :---: |:--:
1 |SS| 2
2 |CSiS| 1,3
3 |FSiS| 2
4 |SiSh| 5
5 |MS| 4,6
6 |WS| 5,7
7 |D| 6,8
8 |PS| 6,7,9
9 |BS| 7,8
Let's clean up this dataset.  The 'Well Name' and 'Formation' columns can be turned into a categorical data type.
Can you write Python code for it?

training_data['Well Name'] = training_data['Well Name'].astype('category')
training_data['Formation'] = training_data['Formation'].astype('category')
training_data['Well Name'].unique()

training_data.describe()


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
This is a quick view of the statistical distribution of the input variables.  Looking at the count values, there are 3232 feature vectors in the training set.
Remove a single well to use as a blind test later.
These are the names of the 10 training wells in the Council Grove reservoir.  Data has been recruited into pseudo-well 'Recruit F9' to better represent facies 9, the Phylloid-algal bafflestone.
Before we plot the well data, let's define a color map so the facies are represented by consistent color in all the plots in this tutorial.  We also create the abbreviated facies labels, and add those to the facies_vectors dataframe.
Can you write Python code for it?

# 1=sandstone  2=c_siltstone   3=f_siltstone
# 4=marine_silt_shale 5=mudstone 6=wackestone 7=dolomite
# 8=packstone 9=bafflestone
facies_colors = ['#F4D03F', '#F5B041','#DC7633','#6E2C00',
       '#1B4F72','#2E86C1', '#AED6F1', '#A569BD', '#196F3D']

facies_labels = ['SS', 'CSiS', 'FSiS', 'SiSh', 'MS',
                 'WS', 'D','PS', 'BS']
#facies_color_map is a dictionary that maps facies labels
#to their respective colors
facies_color_map = {}
for ind, label in enumerate(facies_labels):
    facies_color_map[label] = facies_colors[ind]

def label_facies(row, labels):
    return labels[ row['Facies'] -1]

#training_data.loc[:,'FaciesLabels'] = training_data.apply(lambda row: label_facies(row, facies_labels), axis=1)
faciesVals = training_data['Facies'].values
well = training_data['Well Name'].values
mpl.rcParams['figure.figsize'] = (20.0, 10.0)
for w_idx, w in enumerate(np.unique(well)):
    ax = plt.subplot(3, 4, w_idx+1)
    hist = np.histogram(faciesVals[well == w], bins=np.arange(len(facies_labels)+1)+.5)
    plt.bar(np.arange(len(hist[0])), hist[0], color=facies_colors, align='center')
    ax.set_xticks(np.arange(len(hist[0])))
    ax.set_xticklabels(facies_labels)
    ax.set_title(w)

blind = training_data[training_data['Well Name'] == 'NEWBY']
training_data = training_data[training_data['Well Name'] != 'NEWBY']
training_data.loc[:,'FaciesLabels'] = training_data.apply(lambda row: label_facies(row, facies_labels), axis=1)

PE_mask = training_data['PE'].notnull().values
training_data = training_data[PE_mask]


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
This shows the distribution of examples by facies for the examples in the training set.  Dolomite (facies 7) has the fewest with 81 examples.  Depending on the performance of the classifier we are going to train, we may consider getting more examples of these facies.
Crossplots are a familiar tool in the geosciences to visualize how two properties vary with rock type.  This dataset contains 5 log variables, and scatter matrix can help to quickly visualize the variation between the all the variables in the dataset.  We can employ the very useful Seaborn library to quickly create a nice looking scatter matrix. Each pane in the plot shows the relationship between two of the variables on the x and y axis, with each point is colored according to its facies.  The same colormap is used to represent the 9 facies.
Conditioning the data set
Now we extract just the feature variables we need to perform the classification.  The predictor variables are the five wireline values and two geologic constraining variables. We also get a vector of the facies labels that correspond to each feature vector.
Can you write Python code for it?

correct_facies_labels = training_data['Facies'].values

feature_vectors = training_data.drop(['Formation', 'Well Name', 'Depth','Facies','FaciesLabels'], axis=1)
feature_vectors.describe()


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
Training the classifier using Random forest
Now we use the cleaned and conditioned training set to create a facies classifier. Lets try random forest
Can you write Python code for it?

clf = RandomForestClassifier(n_estimators=150,
                             min_samples_leaf= 50,class_weight="balanced",oob_score=True,random_state=50
                            )


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
We can form the feature matrix by dropping some of the columns and making a new dataframe:
Can you write Python code for it?

well_features = blind.drop(['Facies', 'Formation', 'Well Name', 'Depth'], axis=1)


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
We can use the well log plot to view the classification results along with the well logs.
Can you write Python code for it?

make_facies_log_plot(
    well_data[well_data['Well Name'] == 'STUART'],
    facies_colors=facies_colors)

make_facies_log_plot(
    well_data[well_data['Well Name'] == 'CRAWFORD'],
    facies_colors=facies_colors)


I am working on the file "SHandPR/RandomForest.ipynb".
The first task is:
We need some metrics to evaluate how good our classifier is doing.  A confusion matrix is a table that can be used to describe the performance of a classification model.  Scikit-learn allows us to easily create a confusion matrix by supplying the actual and predicted facies labels.
The confusion matrix is simply a 2D array.  The entries of confusion matrix C[i][j] are equal to the number of observations predicted to have facies j, but are known to have facies i.
To simplify reading the confusion matrix, a function has been written to display the matrix along with facies labels and various error metrics.  See the file classification_utilities.py in this repo for the display_cm() function.
Can you write Python code for it?

from sklearn.metrics import confusion_matrix
from classification_utilities import display_cm, display_adj_cm

conf = confusion_matrix(y_test, predicted_labels)
display_cm(conf, facies_labels, hide_zeros=True)


I am working on the file "SIE_552_HW3.ipynb".
The first task is:
SIE 552 HW #3
This is a space mission design assignment for interplantary travel.  The homework was to create a Type-1 trajectory from Earth to Mars.  Departure from earth is set to be March 22, 2021 with an anticipated arrival on Mars at October 8, 2021.  We're also given the following details about Earth and Mars:
When designing the transfer ellipse, the overall mission has to be considered first.  We know that we want to create an interplanetary travel mission between Earth and Mars.  In order to plan this, we have to know more about those planets:

Earth at Launch: (March 22, 2021)

Longitude = 181.44 deg
Eccentricity = 0.0167
SMA =  149,598,020 Km
Inclination = 0 deg
Longitude of the Perihelion = 102.958 deg
Longitude of the Ascending Node = 0 deg
Radius = 149,905,909.7 km
Velocity = 29.89 km/sec
True Anomaly = 78.48 deg
Flight Path Angle = 0.9348 deg

Mars at Arrival: (October 8, 2021)

Longitude = 333.22 deg
Eccentricity = 0.0934
SMA =  227,939,133 Km
Inclination = 1.849 deg
Longitude of the Perihelion = 336.093 deg
Longitude of the Ascending Node = 49.572 deg
Radius = 206,671,197 km
Velocity = 26.94 km/sec
True Anomaly = 357.128 deg
Flight Path Angle = - 0.2452 deg

We'll store these as PlanetaryObejct's in python to help us later.
Can you write Python code for it?

class PlanetaryObject():
    """
    A simple class used to store pertinant information about the plantary object
    """
    def __init__(self, date, L, e, SMA, i, peri, asc, r, v, anom, fp, mu):
        self.date = date   # Event Date
        self.L = L         # Longitude
        self.e = e         # Eccentricity
        self.SMA = SMA     # SMA
        self.i = i         # Inclination
        self.peri = peri   # Longitude of Perihelion
        self.asc = asc     # Longitude of Ascending Node
        self.r = r         # Radius
        self.v = v         # Velocity
        self.anom = anom   # True Anomaly
        self.fp = fp       # Flight Path Angle
        self.mu = mu       # Gravitation parameter

earth = PlanetaryObject(
    datetime.date(2021, 3, 22),
    181.44,     # Longitude
    0.0167,     # Eccentricity
    149598020,  # SMA
    0,          # Inclination
    102.958,    # Longitude of Perihelion
    0,          # Longitude of Ascending Node
    14905909.7, # Radius
    29.89,      # Velocity
    78.48,      # True Anomaly
    0.9348,     # Flight Path Angle
    398600.4    # Gravitation parameter
)
mars = PlanetaryObject(
    datetime.date(2021, 10, 8),
    333.22,     # Longitude
    0.0934,     # Eccentricity
    227939133,  # SMA
    1.849,      # Inclination
    336.093,    # Longitude of Perihelion
    49.572,     # Longitude of Ascending Node
    206671197,  # Radius
    26.94,      # Velocity
    357.128,    # True Anomaly
    -0.2452,    # Flight Path Angle
    42828.3     # Gravitation parameter
)


I am working on the file "SIE_552_HW3.ipynb".
The first task is:
Using $i_t$, we can now determine the $V_{HE}$ and $C3$ of the departing trajectory:
Can you write Python code for it?

cos_alpha_2 = math.cos(i_t) * math.cos(earth.fp + abs(y_ne))
alpha_2 = math.acos(cos_alpha_2)
C3 = earth.v ** 2 + V_ne ** 2 - 2 * earth.v * V_ne * math.cos(alpha_2)
V_he = math.sqrt(C3)
print("C3 = {:.2f} km^2/s^2; V_he = {:.2f} km/s".format(C3, V_he))


I am working on the file "SIE_552_HW6.ipynb".
The first task is:
Since both the magnetic torques are less than the solar torques, their sum is also less.
Gravitational Gradient Torques
Can you write Python code for it?

def gravity_gradient_torque(u, r, I_z, I_y, theta):
    return 3 * u / r ** 3 * abs(I_z - I_y) * theta

mars_u = 324858.8
T_g = gravity_gradient_torque(mars_u, r, L_left / 100, L_right / 100, math.pi / 4)

T_g

T_g < T_left and T_g < T_right


I am working on the file "SIRmodel.ipynb".
The first task is:
Integrate numerically and plot the results
Can you write Python code for it?

# Numerical integration
xyz= sc.integrate.odeint(SIR, p["ic"], p["sampTimes"], args=(p,)).transpose()
# Calculate the outbreak indicator
B= xyz[0]*p["alpha"]/p["beta"]

# Figure
fig=gr.figure(figsize=(11,5))
gr.ioff()
rows=1; cols=2
ax=list()
for n in sc.arange(rows*cols):
    ax.append(fig.add_subplot(rows,cols,n+1))

ax[0].plot(p["sampTimes"], xyz[0], 'k', label=r"$(t,x(t))$")
ax[0].plot(p["sampTimes"], xyz[1], 'g', lw=3, label=r"$(t,y(t))$")
ax[0].plot(p["sampTimes"], xyz[2], 'b', label=r"$(t,z(t))$")
ax[0].plot(p["sampTimes"], B, 'r', label=r"$(t,B(t))$")
ax[0].plot([0, p["timeStop"]], [1,1], 'k--', alpha=0.4)
ax[1].plot(xyz[0], xyz[1], 'g', lw=3, label=r"$(x(t),y(t))$")
ax[1].plot(xyz[0], xyz[2], 'b', label=r"$(x(t),z(t))$")
ax[1].plot(xyz[0], B, 'r', label=r"$(x(t),B(t))$")
ax[1].plot([0, 1], [1,1], 'k--', alpha=0.4)
ax[0].legend(); ax[1].legend(loc="upper left")
gr.ion(); gr.draw()


I am working on the file "SIRmodel.ipynb".
The first task is:
The function SIR above takes three arguments, $U$, $t$, and $p$ that represent the states of the system, the time and the parameters, respectively.
Outbreak condition
The condition
\begin{equation}
\frac{\alpha}{\beta}x(t)>1 , \quad y>0
\end{equation}
defines a threshold for a full epidemic outbreak. An equivalent condition is
\begin{equation}
x>\frac{\beta}{\alpha }, \quad y>0
\end{equation}
Therefore, with the parameters $(\alpha,\beta)$=(0.5,0.1), there will be an outbreak if the initial condition for $x(t)>1/5$ with $y>0$.
Notice that the initial value for $z$ can be interpreted as the initial proportion of immune individuals within the population.
The dynamics related to the oubreak condition can be studied by defining a variable $B(t) = x(t) \alpha/\beta$, called by some authors "effective reproductive number". If $x(t)\approx 1$, the corresponding $B(t)$ is called "basic reproductive number", or $R_o$.
Let's define a python dictionary containing parameters and initial conditions to perform simulations.
Can you write Python code for it?

p={"alpha": 0.15, "beta":0.1, "timeStop":300.0, "timeStep":0.01 }
p["Ro"]=p["alpha"]/p["beta"]
p["sampTimes"]= sc.arange(0,p["timeStop"],p["timeStep"])
N= 1e4; i0= 1e1; r0=0; s0=N-i0-r0
x0=s0/N; y0=i0/N; z0=r0/N;
p["ic"]=[x0,y0,z0]
print("N=%g with initial conditions (S,I,R)=(%g,%g,%g)"%(N,s0,i0,r0))
print("Initial conditions: ", p["ic"])
print("B(0)=%g"%(p["ic"][0]*p["Ro"]))


I am working on the file "SITE.ipynb".
The first task is:
Actual Calendar
The Actual Calendar can be defined as
Can you write Python code for it?


I am working on the file "SITE.ipynb".
The first task is:
For postion use: first, second, third, 1st, 2nd, 3rd, [n]th, and last.
Using date postions as a reference
You can find before and after other date positions (using date positions as a reference).
Can you write Python code for it?

cal.getdate('last mon before 30th day', 2006, 7)

cal.getdate('second bizday after 15th day', 2006)


I am working on the file "SITE.ipynb".
The first task is:
In this example I used the list of holidays released by ANBIMA.

Important note on date arguments and returning dates
As you can see in the examples all date arguments are strings ISO formatted (YYYY-mm-dd or %Y-%m-%d), but they can also be passed as datetime.date objects.
All returning dates are datetime.date objects (or a sequence of it), unless you set iso=True, that will return an ISO formatted string.
The startdate and enddate of a Calendar are defined accordingly the first and last given holidays.

bizdays
To compute the business days between two dates you call bizdays passing from and to dates as arguments.
Can you write Python code for it?


I am working on the file "SITE.ipynb".
The first task is:
It accepts day, bizday and weekdays by: sun, mon, tue, wed, thu, fri, and sat.
Can you write Python code for it?

cal.getdate('last day', 2006)

cal.getdate('last bizday', 2006)

cal.getdate('last mon', 2006)


I am working on the file "SITE.ipynb".
The first task is:
Obviously, if you want to offset backwards you can use -n.
Can you write Python code for it?


I am working on the file "SITE.ipynb".
The first task is:
Once mode is set to pandas, the Calendar's methods return Timestamp (for single dates), DatetimeIndex (for sequece of dates) and numpy.ndarray (for other sequences).
Can you write Python code for it?


I am working on the file "SITE.ipynb".
The first task is:
Once the given date is a business day there is no problems, but if instead it isn't a working day the offset can lead to unexpected results. For example:
Can you write Python code for it?


I am working on the file "SITE.ipynb".
The first task is:
Pandas integration
bizdays has an integration with pandas.
To get this integration working the option mode must be set to pandas.
Can you write Python code for it?

import bizdays

bizdays.set_option('mode', 'pandas')


I am working on the file "SITE.ipynb".
The first task is:
The Actual Calendar doesn't consider holidays, nor nonworking weekdays for counting business days between 2 dates.
This is the same of subtracting 2 dates, and adjust methods will return the given argument.
But the idea of using the Actual Calendar is working with the same interface for any calendar you work with.
When you price financial instruments you don't have to check if it uses business days or not.

startdate and enddate defaults to 1970-01-01 and 2071-01-01, but they can be set during Calendar's instanciation.

Vectorized operations
The Calendar's methods: isbizday, bizdays, adjust_previous, adjust_next, and offset, have a vectorized counterparty, inside Calendar.vec attribute.
Can you write Python code for it?


I am working on the file "SITE.ipynb".
The first task is:
These functions accept sequences and single values and return lists.
When bizdays was called, a date and a sequence have been passed, computing business days between the given date and all the others.
Recycle rule
Once you pass 2 sequences, for methods that allow it, and those sequences doesn't have the same length, no problem.
The shorter collection is cycled to fit the longer's length.
This is the recycle rule.
Can you write Python code for it?


I am working on the file "SITE.ipynb".
The first task is:
We also have preceding, although I suppose it is unusual, too.
Can you write Python code for it?


I am working on the file "SITE.ipynb".
The first task is:
bizdays computes business days between two dates based on the definition of nonworking days (usually holidays and weekends—nonworking weekdays).
It also computes other collateral effects like adjust dates for the next or previous business day, check whether a date is a business day, create generators of business days sequences, and so forth.
Install
bizdays is avalilable at PyPI, so it is pip instalable.
pip install bizdays

Using
Business days calculations are done defining a Calendar object.
Can you write Python code for it?

from bizdays import Calendar
cal = Calendar.load('ANBIMA')
cal


I am working on the file "SITE.ipynb".
The first task is:
following and preceding
Several contracts, by default, always expiry in the same day, for example, 1st Januray, which isn't a business day, so instead of carrying your code with awful checks you could call following which returns the given date
whether it is a business day or the next business day.
Can you write Python code for it?


I am working on the file "SITE.ipynb".
The first task is:
modified_following and modified_preceding
modified_following and modified_preceding are common functions used to specify maturity of contracts.
They work the same way following and preceding but once the returning date is a different month it is adjusted to the following or preceding business day in the same month.
Can you write Python code for it?

dt = cal.getdate('last day', 2002, 3)
dt

cal.modified_following(dt)

cal.isbizday('2002-03-29')

dt = cal.getdate('first day', 2002, 6)
dt

cal.modified_preceding(dt)


I am working on the file "SITE.ipynb".
The first task is:
seq
To execute calculations through sequential dates, sometimes you must consider only business days.
For example, you want to compute the price of a bond from its issue date up to its maturity.
You have to walk over business days in order to carry the contract up to maturity.
To accomplish that you use the seq method (stolen from R) which returns a sequence generator of business days.
Can you write Python code for it?


I am working on the file "SL1_Decision_Trees.ipynb".
The first task is:
A First Application: Classifying iris species
One of the most famous datasets for classification in a supervised learning setting is the Iris flower data set.  It is a multivariate dataset introduced in a 1936 paper which records sepal length, sepal width, petal length, and petal width for three species of iris.
scikit-learn has a number of small toy datasets included with it which makes it quick and easy to experiment with different machine learning algorithms on these datasets.
The sklearn.datasets.load_iris() method can be used to load the iris dataset.
Meet the data
The iris object that is returned by load_iris is a Bunch object, which is very similar to a dictionary.  It contains keys and values.
Can you write Python code for it?

from sklearn.datasets import load_iris
iris_dataset = load_iris()

print("Keys of iris_dataset: {}".format(iris_dataset.keys()))

# The value of the key DESCR is a short description of the dataset.  Here we show the beinning of the description.
print(iris_dataset['DESCR'][:193] + "\n...")

# The value of the key target_names is an array of strings, containing the species of flower that we want to predict
print("Target names: {}".format(iris_dataset['target_names']))

# The value of feature_names is a list of strings, giving the description of each feature
print("Feature names: {}".format(iris_dataset['feature_names']))

# The data itself is contained in the target and data fields.
# data contains the numeric measurements of sepal length, sepal width, petal length, and petal width in a NumPy array
print("Type of data: {}".format(type(iris_dataset['data'])))

# The rows in the data array correspond to flowers, while the columns represent the four measurements for each flower.
print("Shape of data: {}".format(iris_dataset['data'].shape))

# We see that the array contains measurements for 150 different flowers (samples).  Here are values for the first 5.
print("First five columns of data:\n{}".format(iris_dataset['data'][:5]))

# The target array contains the species of each of the flowers that were measured, also as a NumPy array
print("Type of target: {}".format(type(iris_dataset['target'])))

# target is a one-dimensional array, with one entry per flower
print("Shape of target: {}".format(iris_dataset['target'].shape))

# The species are encoded as integers from 0 to 2.  The meanings of the numbers are given by the target_names key.
print("Target:\n{}".format(iris_dataset['target']))


I am working on the file "SL1_Decision_Trees.ipynb".
The first task is:
Evaluating the model
How do we know whether we can trust our model?  This is where the test set that we created earlier comes in.  This data was not used to build the model, but we do know what the correct speciies is for each iris in the test set.
Therefore, we can make a prediction for each iris in the test data and compare it against its lable (the known species).  We can measure how well the model works by computing the accuracy, which is the fraction of flowers for which the right species was predicted.
We can also use the score method of the tree object, which will compute the test set accuracy for us.
Can you write Python code for it?

y_pred = tree.predict(X_test)
print("Test set predictions:\n {}".format(y_pred))

print("Test set score: {:.2f}".format(np.mean(y_pred == y_test)))

print("Test set score: {:.2f}".format(tree.score(X_test, y_test)))


I am working on the file "SL1_Decision_Trees.ipynb".
The first task is:
First things first: Look at your data
Before building a machine learning model, it is often a good idea to inspect the data, to see if the task is easily solvable without machine learning, or if the desired information might not be contained in the data.
Additionally, inspecting the data is a good way to find abnormalities and peculiarities.  Maybe some of your irises were measured using inches and not centimeters, for example.  In the real world, inconsistencies in the data and unexpected measurements are very common, as are missing data and not-a-number (NaN) or infinite values.
One of the best ways to inspect data is to visualize it. One way to do this is by using a scatter plot.  A scatter plot of the data puts one feature along the x-axis and another along the y-axis, and draws a dot for each data point.  Unfortunately, computer screens have only two dimensions, which allows us to plot only two (or maybe three) features at a time.  It is difficult to plot datasets with more than three features this way.  One way around this problem is to do a pair plot, which looks at all possible pairs of features.  If you have a small number of features, such as the four we have here, this is quite reasonable.  You should keep in mind, however, that a pair plot does not show the interaction of all of the features at once, so some interesting aspects of the data may not be revealed when visualizing it this way.
In Python, the pandas library has a convenient function called scatter_matrix for creating pair plots for a DataFrame.
Can you write Python code for it?

# create dataframe from data in X_train
# label the columns using the strings in iris_dataset.feature_names
iris_dataframe = pd.DataFrame(X_train, columns=iris_dataset.feature_names)
# create a scatter matrix from the dataframe, color by y_train
grr = pd.scatter_matrix(iris_dataframe, c=y_train, figsize=(15, 15), marker='o',
                        hist_kwds={'bins': 20}, s=60, alpha=.8)


I am working on the file "SL1_Decision_Trees.ipynb".
The first task is:
For this model, the test set accuracy is about 0.97, which means we made the right prediction for 97% of the irises in the test set.  Under some mathematical assumptions, this means that we can expect our model to be correct about 97% of the time for new irises.
A more advanced model may be able to do a better job, but with an overlapping dataset like this, it is unlikely that we would ever be able to achieve 100% accuracy.
Summary
Here is a summary of the code needed for the whole training and evaluation procedure (just 4 lines!).
This snippet contains the core code for applying any machine learning algorithm using scikit-learn.  The fit, predict, and score methods are the common interface to supervised models in scikit-learn.
Can you write Python code for it?

X_train, X_test, y_train, y_test = train_test_split(iris_dataset['data'], iris_dataset['target'], random_state=0)

tree = DecisionTreeClassifier()
tree.fit(X_train, y_train)

print("Test set score: {:.2f}".format(tree.score(X_test, y_test)))


I am working on the file "SL1_Decision_Trees.ipynb".
The first task is:
From the plots, we can see that the three classes seem to be relatively well separated using the sepal and petal measurements.  This means that a machine learning model will likely be able to learn to separate them quite well.
Building your model: Decision Tree
Now we can start building the actual machine learning model.  There are many classification algorithms in scikit-learn that we could use.  Here we will use a Decision Tree because it is easy to understand and interpret.
All machine learning models in scikit-learn are implemented in their own classes, which are called Estimator classes.  The decision tree classification algorithm is implemented in the DecisionTreeClassifier class in the tree module.  Before we can use the model, we need to instantiate the class into an object.  This is when we will set any parameters of the model.  The most important parameter of DecisionTreeClassifier is criterion, the function to measure the quality of a split. Supported criteria are “gini” for the Gini impurity and “entropy” for the information gain ("gini" is the default).
Can you write Python code for it?

from sklearn.tree import DecisionTreeClassifier
tree = DecisionTreeClassifier(random_state=0)


I am working on the file "SL1_Decision_Trees.ipynb".
The first task is:
Making predictions
We can now make predictions using this model on new data for which we might not know the correct labels.  Image we found an iris in the wild with a sepal length of 5 cm, a sepal width of 2.9 cm, a petal length of 1 cm, and a petal width of 0.2cm.  What species of iris would this be?  We can put this data into a NumPy array, which in this case will be of shape 1 x 4 (1 row/sample x 4 features).
Note: Even though we made the measurements of this single flower, scikit-learn always expects two-dimensional arrays for the data.
To make a prediction, we call the predict method of the tree object.
Can you write Python code for it?

X_new = np.array([[5, 2.9, 1, 0.2]])
print("X_new.shape: {}".format(X_new.shape))

prediction = tree.predict(X_new)
print("Prediction: {}".format(prediction))
print("Predicted target name: {}".format(
       iris_dataset['target_names'][prediction]))


I am working on the file "SL1_Decision_Trees.ipynb".
The first task is:
Measuring Success: Training and testing data
We want to build a machine learning model from this data that can predict the species of iris for a new set of measurements.  But before we can apply our model to new measurements, we need to know whether it actually works -- that is, whether we should trust its predictions.
Unfortunately, we cannot use the data we used to build the model to evaluate it.  This is because our model can always simply remember the whole training set, and will therefore always predict the correct label for any point in the training set.  This "remembering" does not indicate to us whether the model will generalize well (in other words, whether it will also perform well on new data).
To assess the model's performance, we show it new data (data that it hasn't seen before) for which we have labels.  This is usually done by splitting the labeled data we have collected (here, our 150 flower measurements) into two parts.  One part of the data is used to build our machine learning model, and is called the training data or training set.  The rest of the data will be used to assess how well the model works; this is called the test data, test set, or hold-out set.
scikit-learn contains a function that shuffles the dataset and splits it for you:  the train_test_split function.  This function extracts 75% of the rows in the data as the training set, together with the corresponding labels for this data.  The remaining 25% of the data, together with the remaining labels, is declared as the test set.  Deciding how much data you want to put into the training and the test set respectively is somewhat arbitrary, but scikit-learn's default 75/25 split is a reasonable starting point.
In scikit-learn, data is usually denoted with a capital X, while labels are denoted by a lowercase y.  This is inspired by the standard formulation f(x)=y in mathematics, where x is the input to a function and y is the output.  Following more conventions from mathematics, we use a capital X because the data is a...
Can you write Python code for it?

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(iris_dataset['data'], iris_dataset['target'], random_state=0)

print("X_train shape: {}".format(X_train.shape))
print("y_train shape: {}".format(y_train.shape))

print("X_test shape: {}".format(X_test.shape))
print("y_test shape: {}".format(y_test.shape))


I am working on the file "SL1_Decision_Trees.ipynb".
The first task is:
The tree object encapsulates the algorithm that will be used to build the model from the training data, as well the algorithm to make predictions on new data points.  It will also hold the information that the algorithm has extracted from the training data.
To build the model on the training set, we call the fit method of the tree object, which takes as arguments the NumPy array X_train containing the training data and the NumPy array y_train of the corresponding training labels.
Can you write Python code for it?


I am working on the file "SL2_Regression_and_Classification.ipynb".
The first task is:
A First Application:  Predicting Boston Housing Prices
One of the most famous datasets for regression in a supervised learning setting is the Boston Housing data set.  It is a multivariate dataset introduced in a 1978 paper which records 13 attributes concerning housing values in the suburbs of Boston.  NOTE: The data is very, very old and the house prices are ridiculously low by today's standards.
scikit-learn has a number of small toy datasets included with it which makes it quick and easy to experiment with different machine learning algorithms on these datasets.
The sklearn.datasets.load_boston() method can be used to load the this dataset.
Meet the data
The boston object that is returned by load_boston is a Bunch object, which is very similar to a dictionary.  It contains keys and values.
Feature Information:
1. CRIM: per capita crime rate by town
2. ZN: proportion of residential land zoned for lots over 25,000 sq.ft.
3. INDUS: proportion of non-retail business acres per town
4. CHAS: Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
5. NOX: nitric oxides concentration (parts per 10 million)
6. RM: average number of rooms per dwelling
7. AGE: proportion of owner-occupied units built prior to 1940
8. DIS: weighted distances to five Boston employment centres
9. RAD: index of accessibility to radial highways
10. TAX: full-value property-tax rate per $10,000
11. PTRATIO: pupil-teacher ratio by town
12. B: 1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town
13. LSTAT: % lower status of the population
Target Information
14. MEDV: Median value of owner-occupied homes in $1000's
Can you write Python code for it?

from sklearn.datasets import load_boston
boston = load_boston()

print("Keys of boston: {}".format(boston.keys()))

# The value of the key DESCR is a short description of the dataset.  Here we show the beinning of the description.
print(boston['DESCR'][:193] + "\n...")

# The value of feature_names is a list of strings, giving the abbreviated name of each feature
print("Feature names: {}".format(boston['feature_names']))

# The data itself is contained in the target and data fields.
# data contains the numeric measurements of features in a NumPy array
print("Type of data: {}".format(type(boston['data'])))

# The rows in the data array correspond to neighborhoods, while the columns represent the features
print("Shape of data: {}".format(boston['data'].shape))

# We see that the array contains measurements for 506 different neighborhoods.  Here are values for the first 5.
print("First five columns of data:\n{}".format(boston['data'][:5]))

# The target array contains the Median value of owner-occupied homes in $1000's, also as a NumPy array
print("Type of target: {}".format(type(boston['target'])))

# target is a one-dimensional array, with one entry per sample
print("Shape of target: {}".format(boston['target'].shape))

# The target values are positive floating point numbers which represent a median house value in thousands of dollars.
print("Target:\n{}".format(boston['target']))


I am working on the file "SL2_Regression_and_Classification.ipynb".
The first task is:
A common way to summarize the cross-validation accuracy is to compute the mean:
Can you write Python code for it?

print("Average cross-validation score: {:.2f}".format(scores.mean()))


I am working on the file "SL2_Regression_and_Classification.ipynb".
The first task is:
An R^2 of around 0.64 on the test set is not very good, but we can see that the scores on the training and test sets are are a decent distance apart. This means we are likely overfitting. With higher-dimensional datasets (meaning datasets with a large number of features), linear models become more powerful, and there is a higher chance of overfitting.  More complicated linear models such as Ridge Regression and Lasso have been designed to help control this overfitting problem.
An R^2 of around 0.77 on the training set is OK, but not great.  For a really good fit, we would want an R^2 of around 0.95 or so.  This tells us we are missing someting.  One possibility is we could do some feature engineering and either include polynomial powers of some of the features and/or include products of some of the features.
Also, linear models tend ot work better when all of the features exist on roughly the same scale, we could attempt to scale our data as well.
Preprocessing and Scaling
Some algorithms, like neural networks, SVMs, and k-NearestNeighbors are very sensitive to the scaling of the data; while many others such as linear models with regularization (Ridge, Lasso, etc.) are moderately sensitive to the scaling of the data.  Therefore, a common practice is to adjust the features so that the data representation is more suitable for these algorithms. Often, this is a simple per-feature rescaling and shift of the data.
Different Kinds of Preprocessing
Differnt algorithms benefit from different kinds of scaling and thus Scikit-Learn supports a variety of scaling methods, though they all have a similar API.
StandardScaler
Neural networks expect all input features to vary in a similar way, and ideally to have a mean of 0, and a variance of 1. When using ANN, we must rescale our data so that it fulfills these requirements. For doing this automatically, scikit-learn has the StandardScaler.  The StandardScaler in scikit-learn ensures that for each feature the mean is 0 and the variance is 1, bringing all features to the same...
Can you write Python code for it?

# Scale the boston dataset
from sklearn.preprocessing import MinMaxScaler
X = MinMaxScaler().fit_transform(boston.data)

X_train, X_test, y_train, y_test = train_test_split(X, boston['target'], random_state=0)
lr = LinearRegression().fit(X_train, y_train)
print("Training set score: {:.2f}".format(lr.score(X_train, y_train)))
print("Test set score: {:.2f}".format(lr.score(X_test, y_test)))


I am working on the file "SL2_Regression_and_Classification.ipynb".
The first task is:
As we can see, a default 3-fold cross-validation performed ok for the first two folds, but horribly bad for the third one.
The fundamental problem here is that if that data isn't organized in a random way, then just taking folds in order doesn't represent a random sampling for each fold.  There are multiple possible ways to mitigate this issue.
Stratified k-Fold Cross-Validation
As the simple k-fold strategy would obviously fail for classification problems if the data is organized by target category, scikit-learn does not use it for classification, but rather uses stratified k-fold cross-validation. In stratified cross-validation, we split the data such that the proportions between classes are the same in each fold as they are in the whole dataset.
scikit-learn supports startified k-fold cross-validation via the StratifiedKFold class in the model_selection module.
For example, if 90% of your samples belong to class A and 10% of your samples belong to class B, then stratified cross-validation ensures that in each fold, 90% of samples belong to class A and 10% of samples belong to class B.
For regression, scikit-learn uses the standard k-fold cross-validation by default.
Shuffle-split cross-validation
Another, very flexible strategy for cross-validation is shuffle-split cross-validation. In shuffle-split cross-validation, each split samples train_size many points for the training set and test_size many (disjoint) point for the test set. This splitting is repeated n_iter times. You can use integers for train_size and test_size to use absolute sizes for these sets, or floating-point numbers to use fractions of the whole dataset.
Since the sampling in shuffle-split cross-validation is done in a random fashion, this is a safer alternative to default k-Fold Cross-Validation when the data isn't truly randomized.
scikit-learn supports shuffle-split cross-validation via the ShuffleSplit class in the model_selection module.
There is also a stratified variant of ShuffleSplit, aptly named StratifiedShuffleSplit, which...
Can you write Python code for it?

# Let's look at the boston housing dataset again using shuffle-split cross-validation to ensure random sampling
# The following code splits the dataset into 80% training set and 20% test set for 3 iterations:
from sklearn.model_selection import ShuffleSplit
shuffle_split = ShuffleSplit(test_size=.8, train_size=.2, n_splits=3)
scores = cross_val_score(lr, boston.data, boston.target, cv=shuffle_split)
print("Cross-validation scores:\n{}".format(scores))


I am working on the file "SL2_Regression_and_Classification.ipynb".
The first task is:
As you can see, the training set score of Ridge is lower than for LinearRegression, while the test set score is higher. This is consistent with our expectation. With linear regression, we were overfitting our data. Ridge is a more restricted model, so we are less likely to overfit. A less complex model means worse performance on the training set, but better generalization. As we are only interested in generalization performance, we should choose the Ridge model over the LinearRegression model.
The Ridge model makes a trade-off between the simplicity of the model (near-zero coefficients) and its performance on the training set. How much importance the model places on simplicity versus training set performance can be specified by the user, using the alpha parameter. In the previous example, we used the default parameter alpha=1.0. There is no reason why this will give us the best trade-off, though. The optimum setting of alpha depends on the particular dataset we are using. Increasing alpha forces coefficients to move more toward zero, which decreases training set performance but might help generalization. For example:
Can you write Python code for it?

ridge10 = Ridge(alpha=10).fit(X_train, y_train)
print("Training set score: {:.2f}".format(ridge10.score(X_train, y_train)))
print("Test set score: {:.2f}".format(ridge10.score(X_test, y_test)))


I am working on the file "SL2_Regression_and_Classification.ipynb".
The first task is:
Because grid_search itself has predict and score methods, using best_estimator_ is not needed to make predictions or evaluate the model.
Putting it all together
The one thing we didn't do was experiment with different train/test splits.  Let's run it with randomness a bunch of times and see how consistent it is:
Can you write Python code for it?

from sklearn.model_selection import GridSearchCV
from sklearn.linear_model import Ridge

param_grid = {'alpha': [0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10, 50, 100]}
grid_search = GridSearchCV(Ridge(), param_grid, cv=5)

X, y = load_extended_boston(scaler='standard')

for i in range(10):
    X_train, X_test, y_train, y_test = train_test_split(X, y)
    grid_search.fit(X_train, y_train)

    print("Run {} - Test set score: {:.2f}  Best parameters: {}".format(i, grid_search.score(X_test, y_test),
                                                                        grid_search.best_params_))


I am working on the file "SL2_Regression_and_Classification.ipynb".
The first task is:
Building your model: Linear Regression
Now we can start building the actual machine learning model.  There are many regression algorithms in scikit-learn that we could use.  Here we will use Ordinary Least Squares (OLS) Linear Regression because it is easy to understand and interpret.
All machine learning models in scikit-learn are implemented in their own classes, which are called Estimator classes.  The Linear Regression algorithm is implemented in the LinearRegression class in the linear_model module.  Before we can use the model, we need to instantiate the class into an object.  This is when we will set any parameters of the model.  The LinearRegression model doesn't have any particular parameters of importance.
Can you write Python code for it?

from sklearn.linear_model import LinearRegression
lr = LinearRegression()


I am working on the file "SL2_Regression_and_Classification.ipynb".
The first task is:
By default, cross_val_score performs three-fold cross-validation, returning three accuracy values. We can change the number of folds used by changing the cv parameter:
Can you write Python code for it?

scores = cross_val_score(logreg, iris.data, iris.target, cv=5)
print("Cross-validation scores: {}".format(scores))


I am working on the file "SL2_Regression_and_Classification.ipynb".
The first task is:
Choosing the parameters using cross-validation, we actually found a model that achieves 77% accuracy on the test set. The important thing here is that we did not use the test set to choose the parameters. The parameters that were found are scored in the best_params_ attribute, and the best cross-validation accuracy (the mean accuracy over the different splits for this parameter setting) is stored in best_score_:
Can you write Python code for it?

print("Best parameters: {}".format(grid_search.best_params_))
print("Best cross-validation score: {:.2f}".format(grid_search.best_score_))


I am working on the file "SL2_Regression_and_Classification.ipynb".
The first task is:
Decreasing alpha allows the coefficients to be less restricted.  For very small values of alpha, coefficients are barely restricted at all, and we end up with a model that resembles LinearRegression:
Can you write Python code for it?

ridge01 = Ridge(alpha=0.1).fit(X_train, y_train)
print("Training set score: {:.2f}".format(ridge01.score(X_train, y_train)))
print("Test set score: {:.2f}".format(ridge01.score(X_test, y_test)))


I am working on the file "SL2_Regression_and_Classification.ipynb".
The first task is:
First things first: Look at your data
Before building a machine learning model, it is often a good idea to inspect the data, to see if the task is easily solvable without machine learning, or if the desired information might not be contained in the data.
Additionally, inspecting the data is a good way to find abnormalities and peculiarities.  Maybe some of your irises were measured using inches and not centimeters, for example.  In the real world, inconsistencies in the data and unexpected measurements are very common, as are missing data and not-a-number (NaN) or infinite values.
One of the best ways to inspect data is to visualize it. One way to do this is by using a scatter plot.  A scatter plot of the data puts one feature along the x-axis and another along the y-axis, and draws a dot for each data point.  Unfortunately, computer screens have only two dimensions, which allows us to plot only two (or maybe three) features at a time.  It is difficult to plot datasets with more than three features this way.  One way around this problem is to do a pair plot, which looks at all possible pairs of features.  If you have a small number of features, such as the four we have here, this is quite reasonable.  You should keep in mind, however, that a pair plot does not show the interaction of all of the features at once, so some interesting aspects of the data may not be revealed when visualizing it this way.
In Python, the pandas library has a convenient function called scatter_matrix for creating pair plots for a DataFrame.
Can you write Python code for it?

# create dataframe from data in X_train
boston_df = pd.DataFrame(X_train, columns=boston.feature_names)

# Add in the target data
boston_df['MEDV'] = y_train

# Look at the first few rows
boston_df.head()

# create a scatter matrix from the dataframe
tmp = pd.scatter_matrix(boston_df, figsize=(15, 15))