I am working on the file "Module - itertools.ipynb".
The first task is:
排列組合相關的function:

product(p, q): 傳回所有(p[i],q[j])的組合，總數應該是len(p)*len(q)
permutations(p, [,r]): 傳回p中取r個元素所有可能的排列，如果不指定r則會用所有元素
combinations(p, r): 傳回p中取r個元素的所有組合
combinations_with_replacement(p, r): 傳回p中取r個元素的所有組合，但元素可重複出現
Can you write Python code for it?

list(product([1,2], [3,4,5]))

list(permutations([1, 2, 3]))

list(permutations([1, 2, 3], 2))

list(combinations([1, 2, 3], 2))

list(combinations_with_replacement([1, 2, 3], 2))


I am working on the file "Module - itertools.ipynb".
The first task is:
用有限長度的list產生無限長度的generator，但後面都填None
Can you write Python code for it?

chain(permutations([1, 2, 3]), repeat(None))


I am working on the file "Module - itertools.ipynb".
The first task is:
要消耗一個iterator的所有元素，可以用collections.deque，它會一直取iterator的元素直到清空為止。
為了減少記憶體用量，collections.deque的大小可設為0，則取出的元素就會直接丟棄。
Can you write Python code for it?

from collections import deque
iterator = izip(a, b)
deque(iterator, maxlen=0)
list(iterator)


I am working on the file "Module 1.ipynb".
The first task is:
Booleans Comparisons
Can you write Python code for it?

aTuple = ( 1, 25, 87 )

aMixedList  =  [ 'Mixed Lists include', 'ints', 1, 2, 3, 'strings', 'ab', 'cd',
                'and even other lists', [ 1, 2, 3 ], [ 'ab' , 'cd' ] ]

print( '\n\nObject contains element\n' )
print( 25 in aTuple )
print( "Bruce Springsteen" in aMixedList )
print( [ 1, 2, 3 ] in aMixedList )

print( '\n\nSimple Predicates\n' )
print( 3 < 5 )
print( True == True )
print( 'b' < 'a' )
print( (True or False) == True )
print( (False and True) != False ) ## False and True is equal to True!!!


I am working on the file "Module 1.ipynb".
The first task is:
Sample code
write a code snippet that:

prompts the user for their firstname
prompts the user for their lastname
prints out "Hello, firstname lastname
Can you write Python code for it?

firstname  = input('Please type your first name: ')
lastname   = input('Now type your last name: ')
print( 'Hello, ' + firstname + ' ' + lastname )


I am working on the file "Module 1.ipynb".
The first task is:
Simple Input/Output
Can you write Python code for it?

## Classic output example
## Hello World!

## Note that Python 3.X requires parentheses around the arguments passed to the print function.
## Parentheses were not necessary in earlier versions of Python (e.g. 2.7).

print("Hello World!")

## Input example

variable = input('Prompt for input.  Type in anything and hit enter: ')
print( 'Input variable is ' + variable)


I am working on the file "Module 1.ipynb".
The first task is:
Types and Variables
Can you write Python code for it?

##basic types are string, int, float and boolean
aString   =  'abcD'
anInt     =  4
aFloat    =  3.1415926
aBoolean  =  True  ## or False
aNoneType =  None

##Note that print statements require you to cast ints and other non strings using the str() function
print('a string ' + aString    + ' has type: ' + str( type( aString ) ) )

print('an int   ' + str(anInt) + ' has type: ' + str( type( anInt ) ) )
##Note that print takes a single type and will cast it as a string to print, same for any non-string type
print(anInt)

print('a float   ' + str( aFloat )   + ' has type: ' + str( type( aFloat   ) ) )
print('a boolean ' + str( aBoolean ) + ' has type: ' + str( type( aBoolean ) ) )

##more advanced built-in type are tuple, list and dictionary

aTuple      =  ( 1, 2, 3 )
aMixedTuple =  ( 1, 'a' )

aList       =  [ 'I', 'am', 'a', 'list', 'of', 'strings' ]
aMixedList  =  [ 'Mixed Lists include', 'ints', 1, 2, 3, 'strings', 'ab', 'cd',
                'and even other lists', [ 1, 2, 3 ], [ 'ab' , 'cd' ] ]

aDictionary =  { "key_1" : 1, "key_2" : ( 'a', 'b', 'c'), "bob" : [ 1 , 2, 'a' ] }

print('a tuple      '  + str(aTuple)       + ' has type: ' + str( type( aTuple ) ) )
print('a list       '  + str(aList)        + ' has type: ' + str( type( aList  ) ) )
print('a dictionary '  + str(aDictionary)  + ' has type: ' + str( type( aDictionary ) ) )


I am working on the file "Module-01a-Frame-Regression.ipynb".
The first task is:
One consideration we check here is the number of observations with missing values for those columns that have missing values. If a column has too many missing values, it might make sense to drop the column.
Can you write Python code for it?

#let's see how many missing values are present
df.isnull().sum()

df.isnull()

df[df.isnull().any(axis=1)].head()


I am working on the file "Module-01a-Frame-Regression.ipynb".
The first task is:
So, we see that two columns have missing values: interest and years. Both the columns are numeric. We have three options for dealing with this missing values
Options to treat Missing Values
- REMOVE - NAN rows
- IMPUTATION - Replace them with something??
    - Mean
    - Median
    - Fixed Number - Domain Relevant
    - High Number (999) - Issue with modelling
- BINNING - Categorical variable and "Missing becomes a number
- DOMAIN SPECIFIC* - Entry error, pipeline, etc.
Can you write Python code for it?

#Let's replace missing values with the median of the column
df.interest.median()

?df.fillna

df[df.isnull().any(axis=1)].head(20)

#there's a fillna function
df.fillna(df.median(), inplace=True)

#Now, let's check if train has missing values or not
df.isnull().sum()


I am working on the file "Module-02a-Intuition-Logistic.ipynb".
The first task is:
Cost Function
$$Cost (P(X\beta), y)) =
\begin{cases}
-log(P(X\beta)),  & \text{if $y=1$ } \
-log(1 -P(X\beta)), & \text{if $y =0$}
\end{cases}$$
Can you write Python code for it?

z = np.linspace(0.001, 0.999, 1000)

c1 = -np.log(z)
c2 = -np.log(1-z)

plt.plot(z,c1)
plt.plot(z,c2)
#plt.hlines(0.5, -10,10)
#plt.vlines(0, 0,1)
plt.xlabel('z')
plt.ylabel('Cost')


I am working on the file "Module-02a-Intuition-Logistic.ipynb".
The first task is:
Gradient Descent for Logistic Function
To make it easier to work with we can write this in one line - Log-Likelihood Function
$$ C(\beta) = \frac{1}{n} (- y^T * log(P(X\beta)) - (1- y^T) * log(1 -P(X\beta))) $$
If we were to differentiate this, we will get our gradient which is very similiar to our linear regression one:
$$ \nabla C(\beta) =  \frac {2}{n} X^T(P(X\beta)−y) $$
and are gradient descent algorithm will be
$$ \beta_{i+1} = \beta_{i} - \eta * \nabla C(\beta)$$
Can you write Python code for it?

n = pop.shape[0]
x0 = np.ones(n)
x1 = pop.kmpl
x2 = pop.price
X_actual = np.c_[x1, x2]

X_norm = (X_actual - np.mean(X_actual, axis=0)) / np.std(X_actual, axis=0)

X = np.c_[x0, X_norm]

X = np.asmatrix(X)
y = np.asmatrix(pop.types.values.reshape(-1,1))
b = np.asmatrix([[0],[0],[0]])

def P(z):
    return 1.0/(1+np.exp(-z))

def cost(X,y,b,n):
    C = (- y.T*np.log(P(X*b))-(1-y.T)*np.log(1-P(X*b)))/n
    return C[0,0]

def gradient(X,y,b,n):
    g = (2/n)*X.T*(P(X*b) - y)
    return g

def gradient_descent_logistic (eta, epochs, X, y, n):

    # Set Initial Values
    b = np.asmatrix([[0],[0],[0]])
    c = cost(X,y,b,n)
    c_all = []
    c_all.append(c)

    # Run the calculation for those many epochs
    for i in range(epochs):
        g = gradient(X,y,b,n)
        b = b - eta * g
        c = cost(X,y,b,n)
        c_all.append(c)

    return c_all, b

x1_min, x1_max = -3, 3
x2_min, x2_max = -3, 3
xx1, xx2 = np.meshgrid(np.arange(x1_min, x1_max, (x1_max - x1_min)/100),
                       np.arange(x2_min, x2_max, (x2_max - x2_min)/100))
xx = np.c_[np.ones(xx1.ravel().shape[0]), xx1.ravel(), xx2.ravel()]

def plot_gradient_descent(eta, epoch, gradient_func):

    es, bs = gradient_func(eta, epoch, X, y, n)

    # Plot the intercept and coefficients
    plt.subplot(1, 2, 1)
    #plt.tight_layout()

    # Plot the probabilty plot contour
    Z = P(xx*bs)
    Z = Z.reshape(xx1.shape)
    cs = plt.contourf(xx1, xx2, Z, cmap=plt.cm.viridis, alpha = 0.5)
    plt.colorbar(cs)

    # Plot the intercept and coefficients
    plt.scatter(X[:,1], X[:,2], c=pop.types, s = 150, alpha = 0.8 )
    plt.xlabel('kmpl')
    plt.ylabel('price')

    # Plot the error rates
    plt.subplot(1, 2, 2)
    plt.plot(es)
    plt.xlabel('Epochs')
    plt.ylabel('Error')

plot_gradient_descent(0.05, 1000, gradient_descent_logistic)


I am working on the file "Module-02a-Intuition-Logistic.ipynb".
The first task is:
Lets say we want to classify the vehicles by 'Hatchback' and 'Sedan'
Can you write Python code for it?

class_mapping = {'Hatchback': 0, 'Sedan': 1}

pop['types'] = pop['type'].map(class_mapping)

plt.scatter(pop.price, pop.types, c=pop.types, s = 150, alpha = 0.8 )
plt.xlabel('price')
plt.ylabel('types')


I am working on the file "Module-02a-Intuition-Logistic.ipynb".
The first task is:
Logistic Function
So we need a function which is bounded between 0 < f(X) < 1, for our classification to work. We will use a logit function for this purpose
$$ P(z) = \frac {1}{1 + e^{-z}} $$
Can you write Python code for it?

z = np.linspace(-10, 10, 100)

p = 1/(1+np.exp(-z))

plt.plot(z,p)
plt.hlines(0.5, -20,20)
plt.vlines(0, 0,1)
plt.xlabel('z')
plt.ylabel('P(z)')


I am working on the file "Module-02a-Intuition-Logistic.ipynb".
The first task is:
So now we can transpose our linear regression problem with this logit function
$$ y = X\beta $$
becomes
$$ P(y) = P(X\beta) $$
$$ y = 0, when \, P(y) >0.5 $$
which is equivalent to $$X\beta > 0 $$
Lets see this in the example in 'price' and 'kmpl' variables, for class $types$
$$ \beta_0 + \beta_1price + \beta_2kmpl + \epsilon > 0 $$
So we are looking for a line which will fulfill the above condition
Can you write Python code for it?

plt.scatter(pop['kmpl'], pop['price'], c=pop['types'], s = 150, alpha = 0.8 )
plt.xlabel('kmpl')
plt.ylabel('price')


I am working on the file "Module-02a-Intuition-Logistic.ipynb".
The first task is:
Why Linear Function does not work
Now we can use a linear function and try to choose a cutoff value to show where the two class separate - lets say cutoff = 0.5
Can you write Python code for it?

def ols (df, xlabel, ylabel):
    n = df.shape[0]
    x0 = np.ones(n)
    x1 = df[xlabel]
    X = np.c_[x0, x1]
    X = np.asmatrix(X)
    y = np.transpose(np.asmatrix(df[ylabel]))
    X_T = np.transpose(X)
    X_pseudo = np.linalg.inv(X_T * X) * X_T
    beta = X_pseudo * y
    return beta

def plot_ols(df, xlabel, ylabel):
    beta = ols(df, 'price', 'types')
    beta_0 = beta.item(0)
    beta_1 = beta.item(1)
    plt.scatter(df[xlabel], df[ylabel], c=df[ylabel], s = 150, alpha = 0.8 )
    plt.xlabel(xlabel)
    plt.ylabel(ylabel)

    y = beta_0 + beta_1 * df[xlabel]
    plt.plot(df[xlabel], y, '-')

    cutoff = (0.5 - beta_0)/beta_1
    plt.vlines(cutoff, -0.4, 1.4)

ols(pop, 'price', 'types')

plot_ols(pop, 'price', 'types')


I am working on the file "Module-02b-Explore-Logistic.ipynb".
The first task is:
Explore the impact of ownership  with default
Explore the impact of age with defualt
Explore the impact of  income with default
Can you write Python code for it?

# Create the transformed income variable


I am working on the file "Module-03a-Intuition-Trees.ipynb".
The first task is:
Decision Tree Classifier
Can you write Python code for it?

data = df.iloc[:,0:2]
target = df.iloc[:,2:3]

from sklearn import tree

clf = tree.DecisionTreeClassifier()

clf

clf = clf.fit(data, target)


I am working on the file "Module-03a-Intuition-Trees.ipynb".
The first task is:
Lets use a dictionary for encoding nominal variable
Can you write Python code for it?

df.Risk.unique()

Risk_mapping = {
       'High': 2,
       'Moderate': 1,
       'Low': 0}

df.Risk.tail()

df['Risk'] = df['Risk'].map(Risk_mapping)

df.Risk.tail()

df.head()


I am working on the file "Module-03a-Intuition-Trees.ipynb".
The first task is:
Now we need to first define an Impurity measure. The three popular impurity measures are:
    - Gini Impurity
    - Entropy
    - Classification Error
Gini Impurity and Entropy lead to similiar results when growing the tree, while Classification error is not as useful for growing the tree (but for pruning the tree) - See example here http://sebastianraschka.com/faq/docs/decision-tree-binary.html
Lets understand Gini Impurity a little better. Gini impurity is a measure of how often a randomly chosen element from the set would be incorrectly labeled if it was randomly labeled according to the distribution of labels in the subset,  Gini impurity can be computed by summing the probability $t_{i} $ of an item with label $i$ being chosen times the probability $ 1-t_{i}$ of a mistake in categorizing that item.
$$ I_{G}(f)=\sum {i=1}^{J}t{i}(1-t_{i})=\sum {i=1}^{J}(t{i}-{t_{i}}^{2})=\sum {i=1}^{J}t{i}-\sum {i=1}^{J}{t{i}}^{2}=1-\sum {i=1}^{J}{t{i}}^{2} $$
Lets calculate the Gini for the overall data set:
Low - 4, Moderate - 6, High - 8 and total observations are 18
$$ I_G(t) = 1 - \left(\frac{6}{18}\right)^2 - \left(\frac{4}{18}\right)^2 - \left(\frac{8}{18}\right)^2 = 1 - \frac{116}{256} = 0.642 $$
scikit-learn uses an optimized CART algorithm, which will use a greedy approach.  A greedy approach is used to divide the space called recursive binary splitting. This is a numerical procedure where all the values are lined up and different split points are tried and tested using a objective cost function. The split with the best cost (lowest cost because we minimize cost) is selected.
Another way to think of this is that a learned binary tree is actually a partitioning of the input space. You can think of each input variable as a dimension on an p-dimensional space. The decision tree split this up into rectangles (when p=2 input variables) or some kind of hyper-rectangles with more inputs.
We can draw these partitions for our dataset
Can you write Python code for it?

def plot_classifier_2d(clf, data, target):
    x_min, x_max = data.iloc[:,0].min(), data.iloc[:,0].max()
    y_min, y_max = data.iloc[:,1].min(), data.iloc[:,1].max()
    xx, yy = np.meshgrid(
        np.arange(x_min, x_max, (x_max - x_min)/100),
        np.arange(y_min, y_max, (y_max - y_min)/100))
    Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])
    Z = Z.reshape(xx.shape)
    cs = plt.contourf(xx, yy, Z, cmap="viridis", alpha = 0.5)
    plt.colorbar(cs)
    plt.scatter(x = data.iloc[:,0], y = data.iloc[:,1], c = target, s = 100, cmap="magma")

modelvis.plot_decision_boundaries(clf, data, target, feature_names=data.columns, show_input=True,
                                  class_names=['Low', 'Moderate', 'High'])

plot_classifier_2d(clf, data,target)


I am working on the file "Module-03a-Intuition-Trees.ipynb".
The first task is:
Preparing Data
We have one ordinal variable (Risk) and one nominal variable (Credit History)
Lets use LabelEncoder for the nominal variable
Can you write Python code for it?

from sklearn import preprocessing
le = preprocessing.LabelEncoder()

df['Credit History'].unique()

df['Credit History'].unique()

le.fit(df['Credit History'].unique())

df['Credit History'].tail()

# Converting the categorical data using label encoder
df['Credit History'] = le.transform(df['Credit History'])

df['Credit History'].tail()

le.classes_


I am working on the file "Module-03c-Model-Evaluation.ipynb".
The first task is:
Accuracy Metrics

Misclassification Rate
Confusion Matrix
Precision & Recall
ROC
AUC

Misclassification Rate
The most basic evaluation metric is accuracy score. if $\hat{y}i$ is the predicted value of the i-th sample and $y_i$ is the corresponding true value, then the fraction of correct predictions over $n\text{samples}$ is defined as
$$ accuracy(y, \hat{y}) = \frac{1}{n_\text{samples}} \sum_{i=0}^{n_\text{samples}-1} 1(\hat{y}_i = y_i) $$
Confusion Matrix
Confusion matrix evaluate the quality of the output of a classifier.
|   | Predicted - Yes  | Predicted - No  |
|--:|---|---|
| Actual - Yes  |  True Positive | False Negative  |
| Actual - No |  False Positive | True Negative  |
The diagonal elements represent the number of points for which the predicted label is equal to the true label, while off-diagonal elements are those that are mislabeled by the classifier. The higher the diagonal values of the confusion matrix the better, indicating many correct predictions.
Precision
The precision is the ratio TP / (TP + FP) where TP
is the number of true positives and FP the number of false positives. The precision is intuitively the ability of the classifier not to label as positive a sample that is negative. The best value is 1 and the worst value is 0.
Recall
The recall is the ratio TP / (TP + FN) where TP is the number of true positives and FN the number of false negatives. The recall is intuitively the ability of the classifier to find all the positive samples. The best value is 1 and the worst value is 0.
Receiver Operating Curve (ROC)
“A receiver operating characteristic (ROC), or simply ROC curve, is a graphical plot which illustrates the performance of a binary classifier system as its discrimination threshold is varied. It is created by plotting the fraction of true positives out of the positives (TPR = true positive rate) vs. the fraction of false positives out of the negatives (FPR = false positive rate), at various threshold settings.”

True Positive Rate (sensitivity) -> TPR = TP / (TP+FN)
False...
Can you write Python code for it?

from sklearn import tree
from sklearn import metrics

def model_evaluation(data, target, model, model_name):
    model_fit = model.fit(data, target)
    pred = model_fit.predict(data)
    proba = model_fit.predict_proba(data)

    fpr, tpr, thresholds = metrics.roc_curve(target, proba[:,1])
    roc_auc = metrics.auc(fpr, tpr)

    print("Model: %s" % model_name)

    # Scores for the model
    print("accuracy: %.3f" % metrics.accuracy_score(target, pred))
    print("recall: %.3f" % metrics.precision_score(target, pred))
    print("precision: %.3f" % metrics.recall_score(target, pred))
    print("confusion_matrix:")
    print(metrics.confusion_matrix(target, pred))
    print("auc: %.3f" % metrics.auc(fpr, tpr))

    # ROC Curve
    plt.title('Receiver Operating Characteristic')
    plt.plot(fpr, tpr, 'b', label='AUC = %0.2f'% roc_auc)
    plt.legend(loc='lower right')
    plt.plot([0,1],[0,1],'r--')
    plt.xlim([-0.1,1.2])
    plt.ylim([-0.1,1.2])
    plt.ylabel('True Positive Rate')
    plt.xlabel('False Positive Rate')
    plt.show()

    return roc_auc


I am working on the file "Module-03c-Model-Evaluation.ipynb".
The first task is:
Build Models and Evaluate
Benchmark Model
Our benchmark model is that there is only "no" in the model. We need to improve on this to realistically do better.
Can you write Python code for it?

benchmark = tree.DecisionTreeClassifier(max_depth = 1)

benchmark

model_evaluation(X, y, benchmark, "benchmark")


I am working on the file "Module-03c-Model-Evaluation.ipynb".
The first task is:
Decision Tree Model - Full
Can you write Python code for it?

Full = tree.DecisionTreeClassifier()

Full

model_evaluation(X, y, Full, "Full")


I am working on the file "Module-03c-Model-Evaluation.ipynb".
The first task is:
Decision Tree Model - Shallow
Can you write Python code for it?

Shallow = tree.DecisionTreeClassifier(max_depth=10)

Shallow

model_evaluation(X, y, Shallow, "Shallow")


I am working on the file "Module-03c-Model-Evaluation.ipynb".
The first task is:
Preprocessing the Data
Can you write Python code for it?

df.years = df.years.fillna(np.mean(df.years))

#Load the preprocessing module
from sklearn import preprocessing

categorical_variables = df.dtypes[df.dtypes=="object"].index.tolist()

categorical_variables

for i in categorical_variables:
    lbl = preprocessing.LabelEncoder()
    lbl.fit(list(df[i]))
    df[i] = lbl.transform(df[i])

df.head()

X = df.iloc[:,1:8]

y = df.iloc[:,0]


I am working on the file "Module-03e-Model-RandomForest.ipynb".
The first task is:
Implementing Random Forest
Can you write Python code for it?

from sklearn.ensemble import RandomForestClassifier

clf = RandomForestClassifier()

clf.fit(X, y)

clf.me


I am working on the file "Module-03g-Model-HyperParameterOpt.ipynb".
The first task is:
Basic checks
Check if the columns are the same in train and test.
What else will you check?  [Discuss]
Can you write Python code for it?

X_train.columns

X_test.columns

print(X_train.shape, X_test.shape)

print("train")
print(X_train.dtypes)
print()
print("test")
print(X_test.dtypes)


I am working on the file "Module-03g-Model-HyperParameterOpt.ipynb".
The first task is:
Exercise

For max_depth include - 6, 10
Add min_samples_split, min_samples_leaf to the grid search
In addition to roc_auc, add precision and recall

Challenges with grid_search
Discuss
Randomized Search
Can you write Python code for it?

from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import randint as sp_randint

%%timeit -n1 -r1

# Set the parameters by cross-validation
tuned_parameters = { "n_estimators": [50,100],
                      "max_depth": [3, 4, 6, None],
                      "max_features": sp_randint(1, 11),
                      "min_samples_split": sp_randint(2, 11),
                      "min_samples_leaf": sp_randint(1, 11),
                      "bootstrap": [True, False],
                      "criterion": ["gini", "entropy"]
                    }

scores = ['roc_auc']

n_iter_search = 20

for score in scores:
    print("# Tuning hyper-parameters for %s" % score)
    print()

    clf = RandomizedSearchCV(RandomForestClassifier(n_jobs=-1),
                       param_distributions = tuned_parameters,
                             n_iter = n_iter_search,
                             n_jobs=-1,
                             cv=5,
                       scoring='%s' % score)
    clf.fit(X_train_updated, y_train)

    print("Best parameters set found on development set:")
    print()
    print(clf.best_params_)
    print()
    print("Grid scores on development set:")
    print()
    means = clf.cv_results_['mean_test_score']
    stds = clf.cv_results_['std_test_score']
    for mean, std, params in zip(means, stds, clf.cv_results_['params']):
        print("%0.3f (+/-%0.03f) for %r"
              % (mean, std * 2, params))
    print()

    print("Detailed classification report:")
    print()
    print("The model is trained on the full development set.")
    print("The scores are computed on the full evaluation set.")
    print()
    y_true, y_pred = y_test,  clf.predict(X_test_updated)

    #false_positive_rate, true_positive_rate, thresholds = roc_curve(y_true, y_pred)
    #roc_auc = auc(false_positive_rate, true_positive_rate)
    #print("AUC:", roc_auc)

    #print(classification_report(y_true, y_pred))
    #print()


I am working on the file "Module-03g-Model-HyperParameterOpt.ipynb".
The first task is:
Exercise
Can you write Python code for it?

#What is the average cross validation score?
np.mean(cross_val_score_rf)


I am working on the file "Module-03g-Model-HyperParameterOpt.ipynb".
The first task is:
Let's do cross validation and see what the generalization error is
Cross-validation
Can you write Python code for it?

from sklearn.model_selection import cross_val_score
from sklearn.metrics import roc_curve, auc

model_rf = RandomForestClassifier(n_estimators=100,
                                 criterion="gini",
                                 max_depth=5,
                                 min_samples_split=2,
                                 min_samples_leaf= 1,
                                 oob_score=True,
                                 n_jobs=-1
                                 )

%%time

#Or use %%timeit -n1 -r1 to time the cell

cross_val_score_rf = cross_val_score(model_rf,
                                     X_train_updated,
                                     y_train, scoring="roc_auc",
                                     cv=5,
                                     n_jobs=-1
                                    )

cross_val_score_rf


I am working on the file "Module-03g-Model-HyperParameterOpt.ipynb".
The first task is:
The categorical data should be encoded.
We saw LabelEncoder earlier. Now, we will use one-hot encoding
One-hot encoding
Can you write Python code for it?

X_train_updated = pd.get_dummies(X_train)

X_train.shape

X_train_updated.shape

#print the first record
X_train_updated.iloc[0]


I am working on the file "Module-04a-Regression-Basic.ipynb".
The first task is:
Sale Price
Can you write Python code for it?


I am working on the file "Module-04a-Regression-Basic.ipynb".
The first task is:
Some info about the attributes
MSSubClass: Identifies the type of dwelling involved in the sale.
    20  1-STORY 1946 & NEWER ALL STYLES
    30  1-STORY 1945 & OLDER
    40  1-STORY W/FINISHED ATTIC ALL AGES
    45  1-1/2 STORY - UNFINISHED ALL AGES
    50  1-1/2 STORY FINISHED ALL AGES
    60  2-STORY 1946 & NEWER
    70  2-STORY 1945 & OLDER
    75  2-1/2 STORY ALL AGES
    80  SPLIT OR MULTI-LEVEL
    85  SPLIT FOYER
    90  DUPLEX - ALL STYLES AND AGES
   120  1-STORY PUD (Planned Unit Development) - 1946 & NEWER
   150  1-1/2 STORY PUD - ALL AGES
   160  2-STORY PUD - 1946 & NEWER
   180  PUD - MULTILEVEL - INCL SPLIT LEV/FOYER
   190  2 FAMILY CONVERSION - ALL STYLES AND AGES

MSZoning: Identifies the general zoning classification of the sale.
   A    Agriculture
   C    Commercial
   FV   Floating Village Residential
   I    Industrial
   RH   Residential High Density
   RL   Residential Low Density
   RP   Residential Low Density Park
   RM   Residential Medium Density

LotFrontage: Linear feet of street connected to property
LotArea: Lot size in square feet
Street: Type of road access to property
   Grvl Gravel
   Pave Paved

Alley: Type of alley access to property
   Grvl Gravel
   Pave Paved
   NA   No alley access

LotShape: General shape of property
   Reg  Regular
   IR1  Slightly irregular
   IR2  Moderately Irregular
   IR3  Irregular

LandContour: Flatness of the property
   Lvl  Near Flat/Level
   Bnk  Banked - Quick and significant rise from street grade to building
   HLS  Hillside - Significant slope from side to side
   Low  Depression

Utilities: Type of utilities available
   AllPub   All public Utilities (E,G,W,& S)
   NoSewr   Electricity, Gas, and Water (Septic Tank)
   NoSeWa   Electricity and Gas Only
   ELO  Electricity only

LotConfig: Lot configuration
   Inside   Inside lot
   Corner   Corner lot
   CulDSac  Cul-de-sac
   FR2  Frontage on 2 sides of property
   FR3  Frontage on 3 sides of property

LandSlope: Slope of property
   Gtl ...
Can you write Python code for it?

import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns


I am working on the file "Module-04a-Regression-Basic.ipynb".
The first task is:
TotalBsmtSF vs Sale Price
Can you write Python code for it?


I am working on the file "Module-04a-Regression-Basic.ipynb".
The first task is:
box plot overallqual/saleprice
Can you write Python code for it?

var = 'OverallQual'
data = pd.concat([df_train['SalePrice'], df_train[var]], axis=1)
f, ax = plt.subplots(figsize=(8, 6))
fig = sns.boxplot(x=var, y="SalePrice", data=data)
fig.axis(ymin=0, ymax=800000);


I am working on the file "Module-04a-Regression-Basic.ipynb".
The first task is:
box plot year built / saleprice
Can you write Python code for it?

var = 'YearBuilt'
data = pd.concat([df_train['SalePrice'], df_train[var]], axis=1)
f, ax = plt.subplots(figsize=(16, 8))
fig = sns.boxplot(x=var, y="SalePrice", data=data)
fig.axis(ymin=0, ymax=800000);
plt.xticks(rotation=90);


I am working on the file "Module-04a-Regression-Basic.ipynb".
The first task is:
correlation matrix
Can you write Python code for it?

corrmat = df_train.corr()
f, ax = plt.subplots(figsize=(12, 9))
sns.heatmap(corrmat, vmax=.8, square=True);


I am working on the file "Module-04a-Regression-Basic.ipynb".
The first task is:
scatterplot with highly correlated features
Can you write Python code for it?

sns.set()
cols = ['SalePrice', 'OverallQual', 'GrLivArea', 'GarageCars', 'TotalBsmtSF', 'FullBath', 'YearBuilt']
sns.pairplot(df_train[cols], height = 2.5);
plt.show();


I am working on the file "Module-05a-ML-Pipeline.ipynb".
The first task is:
Acquire
Can you write Python code for it?

data = pd.read_csv("http://bit.do/df-housing")

df = data[['SalePrice', 'OverallQual', 'GrLivArea', 'GarageCars', 'TotalBsmtSF', 'FullBath', 'YearBuilt', 'FireplaceQu', 'LotFrontage']]

df.head()

X_raw = df.drop('SalePrice', axis=1)
y_raw = df.SalePrice


I am working on the file "Module2.ipynb".
The first task is:
As the tide moves through the Strait, it creates a change in the elevation of the water surface. Below we'll cycle through a tidal cycle and look at how the tide moves through the Strait. Use the slider to move through the time series and look how the measured tide at a station relates to the other stations, and its effect on the water elevation.
Can you write Python code for it?

NeahBay = tide.load_Neah_Bay('Data/')
PortAngeles = tide.load_Port_Angeles('Data/')
PortTownsend = tide.load_Port_Townsend('Data/')

Tides = tide.create_tide_dataset(NeahBay,PortAngeles,PortTownsend)

%matplotlib inline
tide.plot_tide_data(Tides,'2016-10-01','2016-10-02')


I am working on the file "Module2.ipynb".
The first task is:
Take a look at the time series for each station. It looks like a wave. In fact, the tide is a wave. That wave propogates through the Strait, starting at Neah Bay and travelling to Port Townsend. This is reflected in the elevation, as the peak elevation moves from one station to the following station.
# Module 2 Quiz
Can you write Python code for it?


I am working on the file "Module3/Module3 - Lab1.ipynb".
The first task is:
Create a slice from your dataframe and name the variable s1. It should only include the area and perimeter features.
Can you write Python code for it?


I am working on the file "Module3/Module3 - Lab1.ipynb".
The first task is:
Create another slice of from dataframe called it s2 this time. Slice out only the groove and asymmetry features:
Can you write Python code for it?


I am working on the file "Module3/Module3 - Lab1.ipynb".
The first task is:
Load up the wheat seeds dataset into a dataframe. We've stored a copy in the Datasets directory.
Can you write Python code for it?


I am working on the file "Module3/Module3 - Lab3.ipynb".
The first task is:
Create a new 3D subplot using figure fig, which we've defined for you below. Use that subplot to draw a 3D scatter plot using the area, perimeter, and asymmetry features. Be sure so use the optional display parameter c='red', and also label your axes:
Can you write Python code for it?

fig = plt.figure()

# .. your code here ..


I am working on the file "Module3/Module3 - Lab3.ipynb".
The first task is:
Create another 3D subplot using fig. Then use the subplot to graph a 3D scatter plot of the width, groove, and length features. Be sure so use the optional display parameter c='green', and be sure to label your axes:
Can you write Python code for it?

fig = plt.figure()

# .. your code here ..

# Finally, display the graphs:
plt.show()


I am working on the file "Module3/Module3 - Lab4.ipynb".
The first task is:
Create a 2d scatter plot that graphs the compactness and width features:
Can you write Python code for it?


I am working on the file "Module3/Module3 - Lab4.ipynb".
The first task is:
If you loaded the id column as a feature (hint: you shouldn't have!), then be sure to drop it. Also get rid of the area and perimeter features:
Can you write Python code for it?


I am working on the file "Module3/Module3 - Lab4.ipynb".
The first task is:
Load up the wheat seeds dataset into a dataframe. We've stored a copy in the Datasets directory.
Can you write Python code for it?


I am working on the file "Module3/Module3 - Lab6.ipynb".
The first task is:
Compute the correlation matrix of your dataframe:
Can you write Python code for it?


I am working on the file "Module3/Module3 - Lab6.ipynb".
The first task is:
Graph the correlation matrix using imshow or matshow:
Can you write Python code for it?

# .. your code here ..

# Display the graphs:
plt.show()


I am working on the file "Module3/Module3 - Lab6.ipynb".
The first task is:
If you loaded the id column as a feature (hint: you shouldn't have!), then be sure to drop it.
Can you write Python code for it?


I am working on the file "Module3/Module3 - Lab6.ipynb".
The first task is:
Load up the wheat seeds dataset into a dataframe. We've stored a copy in the Datasets directory.
Can you write Python code for it?


I am working on the file "Module3_TidalCurrents.ipynb".
The first task is:
Flood Tide
When the sea level in the ocean side is higher that in the estuary side, water will move inland (towards the estuary). This condition is known as Flood Tide.
Can you write Python code for it?


I am working on the file "Module3_TidalCurrents.ipynb".
The first task is:
In the following interactive plot you can calculate the velocity of the current between the ocean and the estuary, and know the stage of the tidal current. The following parameters determine the value of the velocity and its stage.

a1 = the amplitude of the tide in the ocean side (meters)
a2 = the amplitude of the tide in the estuary side (meters)
alpha = the phase difference, which is related to the delay of the tidal wave (degrees)
N = time at which you want to see the velocity value and direction

Modify this parameters and see how the current is changes in time:
Can you write Python code for it?


I am working on the file "Module3_TidalCurrents.ipynb".
The first task is:
Tidal Currents Exploration
Lets explore how the current behaves for varying conditions in the ocean and in the estuary. We have a simple model that will tell us how fast the water is going, and in which stage, depending on the difference in the sea elevation between the two channel ends.
Can you write Python code for it?

import numpy as np
import xarray as xr
import matplotlib.pyplot as plt
from ipywidgets import interact, interactive, fixed
from tydal.module3_utils import plot_currents
%matplotlib inline


I am working on the file "Module3_TidalCurrents.ipynb".
The first task is:
Tidal Currents in Admiralty Inlet
Admiralty Inlet is the main entrance of Puget Sound from the Strait of San Juan de Fuca. The currents in this area depend on the difference in sea level between the outside of Puget Sound and the inside of it.
Lets see what happends here.
We will show you a time series of sea elevation at Port Townsend and a map of velocities across Admiralty Inlet.
Tidal currents data at Admiralty Inlet comes from the Ferry-Based Monitoring of Puget Sound Currents project, you can learn more about it in here http://www.apl.washington.edu/project/project.php?id=ferries_for_science.
Can you write Python code for it?

import tydal.module3_utils as m3
import tydal.module2_utils as tu

URL1='http://107.170.217.21:8080/thredds/dodsC/Salish_L1_STA/Salish_L1_STA.ncml'

[ferry, ferry_download, message]=m3.ferry_data_download(URL1)


I am working on the file "Module4/Module4 - Lab1.ipynb".
The first task is:
Let's see what the results look like:
Can you write Python code for it?

fig = plt.figure()
ax = fig.add_subplot(111)
ax.set_title('Randomized PCA')
ax.scatter(rpca[:,0], rpca[:,1], c='red', marker='.', alpha=0.75)
plt.show()


I am working on the file "Module4/Module4 - Lab1.ipynb".
The first task is:
Load up the scanned armadillo:
Can you write Python code for it?

plyfile = PlyData.read('Datasets/stanford_armadillo.ply')

armadillo = pd.DataFrame({
  'x':plyfile['vertex']['z'][::reduce_factor],
  'y':plyfile['vertex']['x'][::reduce_factor],
  'z':plyfile['vertex']['y'][::reduce_factor]
})


I am working on the file "Module4/Module4 - Lab1.ipynb".
The first task is:
Preview the Data
Can you write Python code for it?

# Render the Original Armadillo

fig = plt.figure()
ax  = fig.add_subplot(111, projection='3d')

ax.set_title('Armadillo 3D')
ax.set_xlabel('X')
ax.set_ylabel('Y')
ax.set_zlabel('Z')
ax.scatter(armadillo.x, armadillo.y, armadillo.z, c='green', marker='.', alpha=0.75)


I am working on the file "Module4/Module4 - Lab1.ipynb".
The first task is:
Time Execution Speeds
Let's see how long it takes PCA to execute:
Can you write Python code for it?


I am working on the file "Module4/Module4 - Lab2.ipynb".
The first task is:
And Now, The Assignment
Can you write Python code for it?

# Do * NOT * alter this line, until instructed!
scaleFeatures = False


I am working on the file "Module4/Module4 - Lab2.ipynb".
The first task is:
Either take a look at the dataset's webpage in the attribute info section of UCI's Chronic Kidney Disease page,: https://archive.ics.uci.edu/ml/datasets/Chronic_Kidney_Disease or alternatively, you can actually look at the first few rows of your dataframe using .head(). What kind of data type should these three columns be? Compare what you see with the results when you print out your dataframe's dtypes.
If Pandas did not properly detect and convert your columns to the data types you expected, use an appropriate command to coerce these features to the right type.
Can you write Python code for it?


I am working on the file "Module4/Module4 - Lab2.ipynb".
The first task is:
Let's build some color-coded labels; the actual label feature will be removed prior to executing PCA, since it's unsupervised. You're only labeling by color so you can see the effects of PCA:
Can you write Python code for it?

labels = ['red' if i=='ckd' else 'green' for i in df.classification]


I am working on the file "Module4/Module4 - Lab2.ipynb".
The first task is:
Load up the dataset specified on the lab instructions page and remove any and all rows that have a NaN in them. You should be a pro at this by now ;-)
QUESTION: Should the id column be included in your dataset as a feature?
Can you write Python code for it?


I am working on the file "Module4/Module4 - Lab2.ipynb".
The first task is:
Now, plot the transformed data as a scatter plot. Recall that transforming the data will result in a NumPy NDArray. You can either use MatPlotLib to graph it directly, or you can convert it back to DataFrame and have Pandas do it for you.
Since we've already demonstrated how to plot directly with MatPlotLib in Module4/assignment1.ipynb, this time we'll show you how to convert your transformed data back into to a Pandas Dataframe and have Pandas plot it from there.
Can you write Python code for it?

# Since we transformed via PCA, we no longer have column names; but we know we
# are in `principal-component` space, so we'll just define the coordinates accordingly:
ax = drawVectors(T, pca.components_, df.columns.values, plt, scaleFeatures)
T  = pd.DataFrame(T)

T.columns = ['component1', 'component2']
T.plot.scatter(x='component1', y='component2', marker='o', c=labels, alpha=0.75, ax=ax)

plt.show()


I am working on the file "Module4/Module4 - Lab2.ipynb".
The first task is:
Run PCA on your dataset, reducing it to 2 principal components. Make sure your PCA model is saved in a variable called 'pca', and that the results of your transformation are saved in another variable 'T':
Can you write Python code for it?


I am working on the file "Module4/Module4 - Lab4.ipynb".
The first task is:
A .MAT file is a MATLAB file type. The faces dataset could have came in through .png images, but we'll show you how to do that in another lab. For now, you'll see how to import .mats:
Can you write Python code for it?

mat = scipy.io.loadmat('Datasets/face_data.mat')
df = pd.DataFrame(mat['images']).T
num_images, num_pixels = df.shape
num_pixels = int(math.sqrt(num_pixels))

# Rotate the pictures, so we don't have to crane our necks:
for i in range(num_images):
    df.loc[i,:] = df.loc[i,:].reshape(num_pixels, num_pixels).T.reshape(-1)


I am working on the file "Module4/Module4 - Lab4.ipynb".
The first task is:
If you're up for a challenge, draw your dataframes in 3D. Even if you're not up for a challenge, just do it anyway. You might have to increase the dimensionality of your transformed dataset:
Can you write Python code for it?


I am working on the file "Module4/Module4 - Lab4.ipynb".
The first task is:
Implement Isomap here. Reduce the dataframe df down to three components. Once you've done that, call Plot2D using the first two components:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab1.ipynb".
The first task is:
Coerce the Date feature (which is currently a string object) into real date, and confirm by displaying the dtypes again. This might be a slow executing process...
Can you write Python code for it?

# .. your code here ..

def doKMeans(df):
    # Let's plot your data with a '.' marker, a 0.3 alpha at the Longitude,
    # and Latitude locations in your dataset. Longitude = x, Latitude = y
    fig = plt.figure()
    ax = fig.add_subplot(111)
    ax.scatter(df.Longitude, df.Latitude, marker='.', alpha=0.3)

    # TODO: Filter `df` using indexing so it only contains Longitude and Latitude,
    # since the remaining columns aren't really applicable for this lab:
    #
    # .. your code here ..

    # TODO: Use K-Means to try and find seven cluster centers in this df.
    # Be sure to name your kmeans model `model` so that the printing works.
    #
    # .. your code here ..

    # Now we can print and plot the centroids:
    centroids = model.cluster_centers_
    print(centroids)
    ax.scatter(centroids[:,0], centroids[:,1], marker='x', c='red', alpha=0.5, linewidths=3, s=169)
    plt.show()

# Print & Plot your data
doKMeans(df)


I am working on the file "Module5/Module5 - Lab1.ipynb".
The first task is:
Display the dtypes of your dset:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab1.ipynb".
The first task is:
How to Get The Dataset

Open up the City of Chicago's Open Data | Crimes page.
In the Primary Type column, click on the Menu button next to the info button, and select Filter This Column. It might take a second for the filter option to show up, since it has to load the entire list first.
Scroll down to GAMBLING
Click the light blue Export button next to the Filter button, and select Download As CSV

Now that you have th dataset stored as a CSV, load it up being careful to double check headers, as per usual:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab3.ipynb".
The first task is:
Alter your slice so that it includes only Weekday (Mon-Fri) values:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab3.ipynb".
The first task is:
Let's tun K-Means with K=3 or K=4. There really should only be a two areas of concentration. If you notice multiple areas that are "hot" (multiple areas the user spends a lot of time at that are FAR apart from one another), then increase K=5, with the goal being that all centroids except two will sweep up the annoying outliers and not-home, not-work travel occasions. the other two will zero in on the user's approximate home location and work locations. Or rather the location of the cell tower closest to them.....
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab3.ipynb".
The first task is:
Let's visualize the results! First draw the X's for the clusters:
Can you write Python code for it?

ax.scatter(model.cluster_centers_[:,1], model.cluster_centers_[:,0], s=169, c='r', marker='x', alpha=0.8, linewidths=2)
ax.set_title('Weekday Calls Centroids')
plt.show()


I am working on the file "Module5/Module5 - Lab3.ipynb".
The first task is:
Print out the mean CallTime value for the samples belonging to the cluster with the LEAST samples attached to it. If our logic is correct, the cluster with the MOST samples will be work. The cluster with the 2nd most samples will be home. And the K=3 cluster with the least samples should be somewhere in between the two. What time, on average, is the user in between home and work, between the midnight and 5pm?
Can you write Python code for it?

midWayClusterIndices = clusterWithFewestSamples(model)
midWaySamples = user1[midWayClusterIndices]
print("    Its Waypoint Time: ", midWaySamples.CallTime.mean())


I am working on the file "Module5/Module5 - Lab3.ipynb".
The first task is:
The idea is that the call was placed before 5pm. From Midnight-730a, the user is probably sleeping and won't call / wake up to take a call. There should be a brief time in the morning during their commute to work, then they'll spend the entire day at work. So the assumption is that most of the time is spent either at work, or in 2nd, at home:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab3.ipynb".
The first task is:
Using some domain expertise, your intuition should direct you to know that people are likely to behave differently on weekends vs on weekdays:
On Weekends

People probably don't go into work
They probably sleep in late on Saturday
They probably run a bunch of random errands, since they couldn't during the week
They should be home, at least during the very late hours, e.g. 1-4 AM

On Weekdays

People probably are at work during normal working hours
They probably are at home in the early morning and during the late night
They probably spend time commuting between work and home everyday
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab4.ipynb".
The first task is:
As instructed, get rid of the Channel and Region columns, since you'll be investigating as if this were a single location wholesaler, rather than a national / international one. Leaving these fields in here would cause KMeans to examine and give weight to them:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab4.ipynb".
The first task is:
Before unitizing / standardizing / normalizing your data in preparation for K-Means, it's a good idea to get a quick peek at it. You can do this using the .describe() method, or even by using the built-in pandas df.plot.hist():
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab4.ipynb".
The first task is:
Drop rows by index. We do this all at once in case there is a collision. This way, we don't end up dropping more rows than we have to, if there is a single row that satisfies the drop for multiple columns. Since there are 6 rows, if we end up dropping < 562 = 60 rows, that means there indeed were collisions:
Can you write Python code for it?

print("Dropping {0} Outliers...".format(len(drop)))
df.drop(inplace=True, labels=drop.keys(), axis=0)
df.describe()


I am working on the file "Module5/Module5 - Lab4.ipynb".
The first task is:
Having checked out your data, you may have noticed there's a pretty big gap between the top customers in each feature category and the rest. Some feature scaling algorithms won't get rid of outliers for you, so it's a good idea to handle that manually---particularly if your goal is NOT to determine the top customers.
After all, you can do that with a simple Pandas .sort_values() and not a machine learning clustering algorithm. From a business perspective, you're probably more interested in clustering your +/- 2 standard deviation customers, rather than the top and bottom customers.
Remove top 5 and bottom 5 samples for each column:
Can you write Python code for it?

drop = {}
for col in df.columns:
    # Bottom 5
    sort = df.sort_values(by=col, ascending=True)
    if len(sort) > 5: sort=sort[:5]
    for index in sort.index: drop[index] = True # Just store the index once

    # Top 5
    sort = df.sort_values(by=col, ascending=False)
    if len(sort) > 5: sort=sort[:5]
    for index in sort.index: drop[index] = True # Just store the index once


I am working on the file "Module5/Module5 - Lab4.ipynb".
The first task is:
Load up the dataset. It may or may not have nans in it. Make sure you catch them and destroy them, by setting them to 0. This is valid for this dataset, since if the value is missing, you can assume no money was spent on it.
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab4.ipynb".
The first task is:
Now that we've clustered our KMeans, let's do PCA, using it as a tool to visualize the results. Project the centroids as well as the samples into the new 2D feature space for visualization purposes:
Can you write Python code for it?

display_pca = doPCA(T)
T = display_pca.transform(T)
CC = display_pca.transform(centroids)


I am working on the file "Module5/Module5 - Lab4.ipynb".
The first task is:
Print out your centroids. They're currently in feature-space, which is good. Print them out before you transform them into PCA space for viewing
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab4.ipynb".
The first task is:
Some Convenience Functions
Can you write Python code for it?

def drawVectors(transformed_features, components_, columns, plt):
    num_columns = len(columns)

    # This function will project your *original* feature (columns)
    # onto your principal component feature-space, so that you can
    # visualize how "important" each one was in the
    # multi-dimensional scaling

    # Scale the principal components by the max value in
    # the transformed set belonging to that component
    xvector = components_[0] * max(transformed_features[:,0])
    yvector = components_[1] * max(transformed_features[:,1])

    ## Visualize projections

    # Sort each column by its length. These are your *original*
    # columns, not the principal components.
    important_features = { columns[i] : math.sqrt(xvector[i]**2 + yvector[i]**2) for i in range(num_columns) }
    important_features = sorted(zip(important_features.values(), important_features.keys()), reverse=True)
    print("Projected Features by importance:\n", important_features)

    ax = plt.axes()

    for i in range(num_columns):
        # Use an arrow to project each original feature as a
        # labeled vector on your principal component axes
        plt.arrow(0, 0, xvector[i], yvector[i], color='b', width=0.0005, head_width=0.02, alpha=0.75, zorder=600000)
        plt.text(xvector[i]*1.2, yvector[i]*1.2, list(columns)[i], color='b', alpha=0.75, zorder=600000)

    return ax

def doPCA(data, dimensions=2):
    model = PCA(n_components=dimensions, svd_solver='randomized', random_state=7)
    model.fit(data)
    return model

def doKMeans(data, num_clusters=0):
    # TODO: Do the KMeans clustering here, passing in the # of clusters parameter
    # and fit it against your data. Then, return a tuple containing the cluster
    # centers and the labels.
    #
    # Hint: Just like with doPCA above, you will have to create a variable called
    # `model`, which will be a SKLearn K-Means model for this to work.

    # .. your code here ..
    return model.cluster_centers_, model.labels_


I am working on the file "Module5/Module5 - Lab4.ipynb".
The first task is:
Sometimes people perform PCA before doing KMeans, so that KMeans only operates on the most meaningful features. In our case, there are so few features that doing PCA ahead of time isn't really necessary, and you can do KMeans in feature space. But keep in mind you have the option to transform your data to bring down its dimensionality. If you take that route, then your Clusters will already be in PCA-transformed feature space, and you won't have to project them again for visualization.
Can you write Python code for it?

# Do KMeans

n_clusters = 3
centroids, labels = doKMeans(T, n_clusters)


I am working on the file "Module5/Module5 - Lab4.ipynb".
The first task is:
Visualize all the samples. Give them the color of their cluster label
Can you write Python code for it?

fig = plt.figure()
ax = fig.add_subplot(111)
if PLOT_TYPE_TEXT:
    # Plot the index of the sample, so you can further investigate it in your dset
    for i in range(len(T)): ax.text(T[i,0], T[i,1], df.index[i], color=c[labels[i]], alpha=0.75, zorder=600000)
    ax.set_xlim(min(T[:,0])*1.2, max(T[:,0])*1.2)
    ax.set_ylim(min(T[:,1])*1.2, max(T[:,1])*1.2)
else:
    # Plot a regular scatter plot
    sample_colors = [ c[labels[i]] for i in range(len(T)) ]
    ax.scatter(T[:, 0], T[:, 1], c=sample_colors, marker='o', alpha=0.2)


I am working on the file "Module5/Module5 - Lab4.ipynb".
The first task is:
What are you interested in?
Depending on what you're interested in, you might take a different approach to normalizing/standardizing your data.
You should note that all columns left in the dataset are of the same unit. You might ask yourself, do I even need to normalize / standardize the data? The answer depends on what you're trying to accomplish. For instance, although all the units are the same (generic money unit), the price per item in your store isn't. There may be some cheap items and some expensive one. If your goal is to find out what items people tend to buy together but you didn't  "unitize" properly before running kMeans, the contribution of the lesser priced item would be dwarfed by the more expensive item. This is an issue of scale.
For a great overview on a few of the normalization methods supported in SKLearn, please check out: https://stackoverflow.com/questions/30918781/right-function-for-normalizing-input-of-sklearn-svm
Suffice to say, at the end of the day, you're going to have to know what question you want answered and what data you have available in order to select the best method for your purpose. Luckily, SKLearn's interfaces are easy to switch out so in the mean time, you can experiment with all of them and see how they alter your results.
5-sec summary before you dive deeper online:
Normalization
Let's say your user spend a LOT. Normalization divides each item by the average overall amount of spending. Stated differently, your new feature is = the contribution of overall spending going into that particular item: \$spent on feature / \$overall spent by sample.
MinMax
What % in the overall range of $spent by all users on THIS particular feature is the current sample's feature at? When you're dealing with all the same units, this will produce a near face-value amount. Be careful though: if you have even a single outlier, it can cause all your data to get squashed up in lower percentages.
Imagine your buyers usually spend \$100 on wholesale milk, but today only spent \$20. This is the...
Can you write Python code for it?

#T = preprocessing.StandardScaler().fit_transform(df)
#T = preprocessing.MinMaxScaler().fit_transform(df)
#T = preprocessing.MaxAbsScaler().fit_transform(df)
#T = preprocessing.Normalizer().fit_transform(df)
T = df # No Change


I am working on the file "Module5/Module5 - Lab4.ipynb".
The first task is:
You can experiment with these parameters:
Can you write Python code for it?

PLOT_TYPE_TEXT = False    # If you'd like to see indices
PLOT_VECTORS = True       # If you'd like to see your original features in P.C.-Space


I am working on the file "Module5/Module5 - Lab5.ipynb".
The first task is:
Bonus
Instead of the ordinal conversion, try and get this assignment working with a proper Pandas get_dummies for feature encoding. You might have to update some of the plotDecisionBoundary() code.
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab5.ipynb".
The first task is:
Copy the wheat_type series slice out of X, and into a series called y. Then drop the original wheat_type column from the X:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab5.ipynb".
The first task is:
Create an instance of SKLearn's Normalizer class and then train it using its .fit() method against your training data. The reason you only fit against your training data is because in a real-world situation, you'll only have your training data to train with! In this lab setting, you have both train+test data; but in the wild, you'll only have your training data, and then unlabeled data you want to apply your models to.
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab5.ipynb".
The first task is:
Create and train a KNeighborsClassifier. Start with K=9 neighbors. Be sure train your classifier against the pre-processed, PCA- transformed training data above! You do not, of course, need to transform your labels.
Can you write Python code for it?

# .. your code here ..

# I hope your KNeighbors classifier model from earlier was named 'knn'
# If not, adjust the following line:
plotDecisionBoundary(knn, X_train, y_train)


I am working on the file "Module5/Module5 - Lab5.ipynb".
The first task is:
Do a quick, "ordinal" conversion of y. In actuality our classification isn't ordinal, but just as an experiment...
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab5.ipynb".
The first task is:
Do some basic nan munging. Fill each row's nans with the mean of the feature:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab5.ipynb".
The first task is:
Just like your preprocessing transformation, create a PCA transformation as well. Fit it against your training data, and then project your training and testing features into PCA space using the PCA model's .transform() method. This has to be done because the only way to visualize the decision boundary in 2D would be if your KNN algo ran in 2D as well:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab5.ipynb".
The first task is:
Split X into training and testing data sets using train_test_split(). Use 0.33 test size, and use random_state=1. This is important so that your answers are verifiable. In the real world, you wouldn't specify a random_state:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab6.ipynb".
The first task is:
A Convenience Function
This method is for your visualization convenience only. You aren't expected to know how to put this together yourself, although you should be able to follow the code by now:
Can you write Python code for it?

def Plot2DBoundary(DTrain, LTrain, DTest, LTest):
    # The dots are training samples (img not drawn), and the pics are testing samples (images drawn)
    # Play around with the K values. This is very controlled dataset so it should be able to get perfect classification on testing entries
    # Play with the K for isomap, play with the K for neighbors.

    fig = plt.figure()
    ax = fig.add_subplot(111)
    ax.set_title('Transformed Boundary, Image Space -> 2D')

    padding = 0.1   # Zoom out
    resolution = 1  # Don't get too detailed; smaller values (finer rez) will take longer to compute
    colors = ['blue','green','orange','red']

    # ------

    # Calculate the boundaries of the mesh grid. The mesh grid is
    # a standard grid (think graph paper), where each point will be
    # sent to the classifier (KNeighbors) to predict what class it
    # belongs to. This is why KNeighbors has to be trained against
    # 2D data, so we can produce this countour. Once we have the
    # label for each point on the grid, we can color it appropriately
    # and plot it.
    x_min, x_max = DTrain[:, 0].min(), DTrain[:, 0].max()
    y_min, y_max = DTrain[:, 1].min(), DTrain[:, 1].max()
    x_range = x_max - x_min
    y_range = y_max - y_min
    x_min -= x_range * padding
    y_min -= y_range * padding
    x_max += x_range * padding
    y_max += y_range * padding

    # Using the boundaries, actually make the 2D Grid Matrix:
    xx, yy = np.meshgrid(np.arange(x_min, x_max, resolution),
                         np.arange(y_min, y_max, resolution))

    # What class does the classifier say about each spot on the chart?
    # The values stored in the matrix are the predictions of the model
    # at said location:
    Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
    Z = Z.reshape(xx.shape)

    # Plot the mesh grid as a filled contour plot:
    plt.contourf(xx, yy, Z, cmap=plt.cm.terrain, z=-100)

    # ------

    # When plotting the testing images, used to validate if the algorithm
    # is functioning...


I am working on the file "Module5/Module5 - Lab6.ipynb".
The first task is:
Calculate and display the accuracy of the testing set (data_test and label_test):
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab6.ipynb".
The first task is:
Dimensionality Reduction
Can you write Python code for it?

if Test_PCA:
    # INFO: PCA is used *before* KNeighbors to simplify your high dimensionality
    # image samples down to just 2 principal components! A lot of information
    # (variance) is lost during the process, as I'm sure you can imagine. But
    # you have to drop the dimension down to two, otherwise you wouldn't be able
    # to visualize a 2D decision surface / boundary. In the wild, you'd probably
    # leave in a lot more dimensions, which is better for higher accuracy, but
    # worse for visualizing the decision boundary;
    #
    # Your model should only be trained (fit) against the training data (data_train)
    # Once you've done this, you need use the model to transform both data_train
    # and data_test from their original high-D image feature space, down to 2D

    # TODO: Implement PCA here. ONLY train against your training data, but
    # transform both your training + test data, storing the results back into
    # data_train, and data_test.

    # .. your code here ..

else:
    # INFO: Isomap is used *before* KNeighbors to simplify your high dimensionality
    # image samples down to just 2 components! A lot of information has been is
    # lost during the process, as I'm sure you can imagine. But if you have
    # non-linear data that can be represented on a 2D manifold, you probably will
    # be left with a far superior dataset to use for classification. Plus by
    # having the images in 2D space, you can plot them as well as visualize a 2D
    # decision surface / boundary. In the wild, you'd probably leave in a lot more
    # dimensions, which is better for higher accuracy, but worse for visualizing the
    # decision boundary;

    # Your model should only be trained (fit) against the training data (data_train)
    # Once you've done this, you need use the model to transform both data_train
    # and data_test from their original high-D image feature space, down to 2D

    # TODO: Implement Isomap here. ONLY train against your training data, but
    # transform...


I am working on the file "Module5/Module5 - Lab6.ipynb".
The first task is:
Do train_test_split. Use the same code as on the EdX platform in the reading material, but set the random_state=7 for reproducibility, and the test_size to 0.15 (150%). Your labels are actually passed in as a series (instead of as an NDArray) so that you can access their underlying indices later on. This is necessary so you can find your samples in the original dataframe. The convenience methods we've written for you that handle drawing expect this, so that they can plot your testing data as images rather than as points:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab6.ipynb".
The first task is:
Implement KNeighborsClassifier here. You can use any K value from 1 through 20, so play around with it and attempt to get good accuracy. Fit the classifier against your training data and labels.
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab6.ipynb".
The first task is:
Let's chart the combined decision boundary, the training data as 2D plots, and the testing data as small images so we can visually validate performance:
Can you write Python code for it?

Plot2DBoundary(data_train, label_train, data_test, label_test)


I am working on the file "Module5/Module5 - Lab6.ipynb".
The first task is:
Load up your face_labels dataset. It only has a single column, and you're only interested in that single column. You will have to slice the  column out so that you have access to it as a "Series" rather than as a "Dataframe". This was discussed in the the "Slicin'" lecture of the  "Manipulating Data" reading on the course website. Use an appropriate indexer to take care of that. Be sure to print out the labels and compare what you see to the raw face_labels.csv so you know you loaded it correctly.
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab7.ipynb".
The first task is:
Copy out the status column into a slice, then drop it from the main dataframe. Always verify you properly executed the drop by double checking (printing out the resulting operating)! Many people forget to set the right axis here.
If you goofed up on loading the dataset and notice you have a sample column, this would be a good place to drop that too if you haven't already.
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab7.ipynb".
The first task is:
Dimensionality Reduction
PCA and Isomap are your new best friends
Can you write Python code for it?

model = None

if Test_PCA:
    print('Computing 2D Principle Components')
    # TODO: Implement PCA here. Save your model into the variable 'model'.
    # You should reduce down to two dimensions.

    # .. your code here ..

else:
    print('Computing 2D Isomap Manifold')
    # TODO: Implement Isomap here. Save your model into the variable 'model'
    # Experiment with K values from 5-10.
    # You should reduce down to two dimensions.

    # .. your code here ..


I am working on the file "Module5/Module5 - Lab7.ipynb".
The first task is:
Experiment with the basic SKLearn preprocessing scalers. We know that the features consist of different units mixed in together, so it might be reasonable to assume feature scaling is necessary. Print out a description of the dataset, post transformation. Recall: when you do pre-processing, which portion of the dataset is your model trained upon? Also which portion(s) of your dataset actually get transformed?
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab7.ipynb".
The first task is:
Implement and train KNeighborsClassifier on your projected 2D training data here. You can name your variable knmodel. You can use any K value from 1 - 15, so play around with it and see what results you can come up. Your goal is to find a good balance where you aren't too specific (low-K), nor are you too general (high-K). You should also experiment with how changing the weights parameter affects the results.
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab7.ipynb".
The first task is:
The Assignment
Load in the dataset, identify nans, and set proper headers. Be sure to verify the rows line up by looking at the file in a text editor.
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab7.ipynb".
The first task is:
Train your model against data_train, then transform both data_train and data_test using your model. You can save the results right back into the variables themselves.
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab7.ipynb".
The first task is:
With the labels safely extracted from the dataset, replace any nan values with the mean feature / column value:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab8.ipynb".
The first task is:
A Convenience Function
This convenience method will take care of plotting your test observations, comparing them to the regression line, and displaying the R2 coefficient
Can you write Python code for it?

def drawLine(model, X_test, y_test, title):
    fig = plt.figure()
    ax = fig.add_subplot(111)
    ax.scatter(X_test, y_test, c='g', marker='o')
    ax.plot(X_test, model.predict(X_test), color='orange', linewidth=1, alpha=0.7)

    print("Est 2014 " + title + " Life Expectancy: ", model.predict([[2014]])[0])
    print("Est 2030 " + title + " Life Expectancy: ", model.predict([[2030]])[0])
    print("Est 2045 " + title + " Life Expectancy: ", model.predict([[2045]])[0])

    score = model.score(X_test, y_test)
    title += " R2: " + str(score)
    ax.set_title(title)

    plt.show()


I am working on the file "Module5/Module5 - Lab8.ipynb".
The first task is:
Create your linear regression model here and store it in a variable called model. Don't actually train or do anything else with it yet:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab8.ipynb".
The first task is:
Print the actual 2014 'WhiteMale' life expectancy from your loaded dataset
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab8.ipynb".
The first task is:
Repeat the process, but instead of for WhiteMale, this time select BlackFemale. Create a slice for BlackFemales, fit your model, and then call drawLine. Lastly, print out the actual 2014 BlackFemale life expectancy:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab8.ipynb".
The first task is:
Slice out your data manually (e.g. don't use train_test_split, but actually do the indexing yourself. Set X_train to be year values LESS than 1986, and y_train to be corresponding 'WhiteMale' age values. You might also want to read the note about slicing on the bottom of this document before proceeding:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab8.ipynb".
The first task is:
The Assignment
Load up the data here into a variable called X. As usual, do a .describe and a print of your dataset and compare it to the dataset loaded in a text file or in a spread sheet application:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab9.ipynb".
The first task is:
Create your linear regression model here and store it in a variable called model. Don't actually train or do anything else with it yet:
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab9.ipynb".
The first task is:
Duplicate the process above (almost). This time is going to be a bit more complicated. Instead of modeling one feature as a function of another, you will attempt to do multivariate linear regression to model one feature as a function of TWO other features.
Model the number of accepted students as a function of the amount charged for room and board and the number of enrolled students. To do this, instead of creating a regular slice for a single-feature input, simply create a slice that contains both columns you wish to use as inputs. Your training labels will remain a single slice.
Can you write Python code for it?

# .. your code here ..

drawPlane(model, X_test, y_test, "Accept(Room&Board,Enroll)", score)


I am working on the file "Module5/Module5 - Lab9.ipynb".
The first task is:
Duplicate the process above; this time, model the number of accepted students, as a function of the number of enrolled students per college.
Can you write Python code for it?

# .. your code here ..

drawLine(model, X_test, y_test, "Accept(Enroll)", score)


I am working on the file "Module5/Module5 - Lab9.ipynb".
The first task is:
Duplicate the process above; this time, model the number of accepted students, as as function of the number of failed undergraduate students per college.
Can you write Python code for it?

# .. your code here ..

drawLine(model, X_test, y_test, "Accept(F.Undergrad)", score)


I am working on the file "Module5/Module5 - Lab9.ipynb".
The first task is:
The first relationship we're interested in is the number of accepted students, as a function of the amount charged for room and board.
Using indexing, create two slices (series). One will just store the room and board column, the other will store the accepted students column. Then use train_test_split to cut your data up into X_train, X_test, y_train, y_test, with a test_size of 30% and a random_state of 7.
Can you write Python code for it?


I am working on the file "Module5/Module5 - Lab9.ipynb".
The first task is:
This line isn't necessary for your purposes; but we'd just like to show you an additional way to encode features directly. The .map() method is like .apply(), but instead of taking in a lambda / function, you simply provide a mapping of keys:values. If you decide to embark on the "Data Scientist Challenge", this line of code will save you the trouble of converting it through other means:
Can you write Python code for it?

X.Private = X.Private.map({'Yes':1, 'No':0})


I am working on the file "Module5/Module5 - Lab9.ipynb".
The first task is:
We'll take it from here, buddy:
Can you write Python code for it?

drawLine(model, X_test, y_test, "Accept(Room&Board)", score)


I am working on the file "Module6/Module6 - Lab1.ipynb".
The first task is:
Feel free to adjust and experiment with these parameters after you have completed the lab:
Can you write Python code for it?

C = 1
kernel = 'linear'

# TODO: Change to 200000 once you get to Question#2
iterations = 5000

# You can set this to false if you want to draw the full square matrix:
FAST_DRAW = True


I am working on the file "Module6/Module6 - Lab1.ipynb".
The first task is:
Fire it Up:
Can you write Python code for it?

benchmark(knn, X_train, X_test, y_train, y_test, 'KNeighbors')
drawPlots(knn, X_train, X_test, y_train, y_test, 'KNeighbors')

benchmark(svc, X_train, X_test, y_train, y_test, 'SVC')
drawPlots(svc, X_train, X_test, y_train, y_test, 'SVC')

plt.show()


I am working on the file "Module6/Module6 - Lab1.ipynb".
The first task is:
Go ahead and drop any row with a nan:
Can you write Python code for it?


I am working on the file "Module6/Module6 - Lab1.ipynb".
The first task is:
In the future, you might try setting the nan values to the mean value of that column, the mean should only be calculated for the specific class rather than across all classes, now that you have the labels.
Copy the labels out of the dataframe into variable y, then remove them from X.
Encode the labels, using the .map() trick we showed you in Module 5, such that canadian:0, kama:1, and rosa:2.
Can you write Python code for it?


I am working on the file "Module6/Module6 - Lab2.ipynb".
The first task is:
Calculate the score of your SVC against the testing data:
Can you write Python code for it?

print("Scoring SVC Classifier...")

# .. your code here ..
print("Score:\n", score)

# Let's get some visual confirmation of accuracy:
drawPredictions(X_train, X_test, y_train, y_test)


I am working on the file "Module6/Module6 - Lab2.ipynb".
The first task is:
Create an SVC classifier. Leave C=1, but set gamma to 0.001 and set the kernel to linear. Then train the model on the training data and labels:
Can you write Python code for it?

print("Training SVC Classifier...")

# .. your code here ..


I am working on the file "Module6/Module6 - Lab2.ipynb".
The first task is:
Predict the value of the 1000th digit in the test set. Was your model's prediction correct? If you get a warning on your predict line, look at the notes from the previous module's labs.
Can you write Python code for it?

# .. your code here ..

print("1000th test prediction: ", guess_1000th_test_value)


I am working on the file "Module6/Module6 - Lab2.ipynb".
The first task is:
The dataset used in this lab comes from https://archive.ics.uci.edu/ml/datasets/Optical+Recognition+of+Handwritten+Digits
At face value, this looks like an easy lab, but it has many parts to it, so prepare yourself by reading through it fully before starting.
Convenience Functions
Can you write Python code for it?

def load(path_train, path_test):
    # Load up the data.

    # You probably could have written this easily:
    with open(path_test, 'r')  as f: testing  = pd.read_csv(f)
    with open(path_train, 'r') as f: training = pd.read_csv(f)

    # The number of samples between training and testing can vary
    # But the number of features better remain the same!
    n_features = testing.shape[1]

    X_test  = testing.ix[:,:n_features-1]
    X_train = training.ix[:,:n_features-1]
    y_test  = testing.ix[:,n_features-1:].values.ravel()
    y_train = training.ix[:,n_features-1:].values.ravel()

    # Special:
    # ...

    return X_train, X_test, y_train, y_test

def peekData(X_train):
    # The 'targets' or labels are stored in y. The 'samples' or data is stored in X
    print("Peeking your data...")
    fig = plt.figure()
    fig.set_tight_layout(True)

    cnt = 0
    for col in range(5):
        for row in range(10):
            plt.subplot(5, 10, cnt + 1)
            plt.imshow(X_train.ix[cnt,:].reshape(8,8), cmap=plt.cm.gray_r, interpolation='nearest')
            plt.axis('off')
            cnt += 1

    plt.show()

def drawPredictions(X_train, X_test, y_train, y_test):
    fig = plt.figure()
    fig.set_tight_layout(True)

    # Make some guesses
    y_guess = model.predict(X_test)

    # INFO: This is the second lab we're demonstrating how to
    # do multi-plots using matplot lab. In the next assignment(s),
    # it'll be your responsibility to use this and assignment #1
    # as tutorials to add in the plotting code yourself!
    num_rows = 10
    num_cols = 5

    index = 0
    for col in range(num_cols):
        for row in range(num_rows):
            plt.subplot(num_cols, num_rows, index + 1)

            # 8x8 is the size of the image, 64 pixels
            plt.imshow(X_test.ix[index,:].reshape(8,8), cmap=plt.cm.gray_r, interpolation='nearest')

            # Green = Guessed right
            # Red = Fail!
            fontcolor = 'g' if y_test[index] == y_guess[index] else...


I am working on the file "Module6/Module6 - Lab2.ipynb".
The first task is:
Use imshow() to display the 1000th test image, so you can visually check if it was a hard image, or an easy image:
Can you write Python code for it?


I am working on the file "Module_0b_Learning_from_Data.ipynb".
The first task is:
Plotting the Data
Can you write Python code for it?

import seaborn as sns

sns.stripplot(data = df, x = "Income", y = "Credit History", hue = "Risk", size = 10)


I am working on the file "Module_0b_Learning_from_Data.ipynb".
The first task is:
Preparing Data
We have one ordinal variable (Risk) and one nominal variable (Credit History)
Lets use a dictionary for encoding nominal variable
Can you write Python code for it?

df.Risk.unique()

Risk_mapping = {
       'High': 2,
       'Moderate': 1,
       'Low': 0}

df['Risk'] = df['Risk'].map(Risk_mapping)

df['Credit History'].unique()

Credit_mapping = {
        'Unknown': 0,
        'Bad': -1,
        'Good': 1}

df['Credit History'] = df['Credit History'].map(Credit_mapping)

df.head()

sns.stripplot(data = df, x = "Income", y = "Credit History", hue = "Risk", size = 10)


I am working on the file "Module_0b_Learning_from_Data.ipynb".
The first task is:
Understanding how the Decision Tree works
Terminology
- Each root node represents a single input variable (x) and a split point on that variable.
- The leaf nodes of the tree contain an output variable (y) which is used to make a prediction.
Growing the tree

The first choice we have is how many branches we split the trees. And we choose Binary Tree because otherwise it will explode due to combinatorial explosion. So BINARY TREES is a practical consideration.
The second decision is to choose which variable and where to split it. We need to have an objective function to do this

One objective function is to maximize the information gain (IG) at each split:
$$ IG(D_p,f)= I(D_p) - \frac{N_{right}}{N} I(D_{right}) - \frac{N_{left}}{N} I(D_{left}) $$
where:
- f is the feature to perform the split
- $D_p$, $D_{left}$, and $D_{right}$ are the datasets of the parent, left and right child node, respectively
- I is the impurity measure
- N is the total number of samples
- $N_{left}$ and $N_{right}$ is the number of samples in the left and right child node.
Now we need to first define an Impurity measure. The three popular impurity measures are:
    - Gini Impurity
    - Entropy
    - Classification Error
Gini Impurity and Entropy lead to similiar results when growing the tree, while Classification error is not as useful for growing the tree (but for pruning the tree) - See example here http://sebastianraschka.com/faq/docs/decision-tree-binary.html
Lets understand Gini Impurity a little better. Gini impurity is a measure of how often a randomly chosen element from the set would be incorrectly labeled if it was randomly labeled according to the distribution of labels in the subset,  Gini impurity can be computed by summing the probability $t_{i} $ of an item with label $i$ being chosen times the probability $ 1-t_{i}$ of a mistake in categorizing that item.
$$ I_{G}(f)=\sum {i=1}^{J}t{i}(1-t_{i})=\sum {i=1}^{J}(t{i}-{t_{i}}^{2})=\sum {i=1}^{J}t{i}-\sum {i=1}^{J}{t{i}}^{2}=1-\sum {i=1}^{J}{t{i}}^{2} $$
Lets calculate...
Can you write Python code for it?

x_min, x_max = data.ix[:, 0].min() - 2000, data.ix[:, 0].max() + 2000
y_min, y_max = data.ix[:, 1].min() - 1, data.ix[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, (x_max - x_min)/100), np.arange(y_min, y_max, (y_max - y_min)/100))

Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)
cs = plt.contourf(xx, yy, Z, cmap=plt.cm.viridis, alpha = 0.5)
plt.scatter(x = data.ix[:,0], y = data.ix[:,1], c = target, s = 100, cmap=plt.cm.magma)


I am working on the file "Module_0b_Learning_from_Data.ipynb".
The first task is:
Visualise the Tree
Can you write Python code for it?

import pydotplus
from IPython.display import Image

data.columns

target.columns

dot_data = tree.export_graphviz(clf, out_file='tree.dot', feature_names=data.columns,
                                class_names=['Low', 'Moderate', 'High'], filled=True,
                                rounded=True, special_characters=True)

graph = pydotplus.graph_from_dot_file('tree.dot')

Image(graph.create_png())


I am working on the file "Module_1a_linear_algebra_inverse.ipynb".
The first task is:
1. Matrix Addition
$$ \begin{bmatrix}a & b\ c & d\end{bmatrix}  + \begin{bmatrix}e & f\ g & h\end{bmatrix}   = \begin{bmatrix}a + e & b + f \ c + g & d + h\end{bmatrix} $$
What is $ U + V$?
What is $ V + U $?
What is $ W + U$?
2. Scalar Multiplication
$$ \beta * \begin{bmatrix}a & b\ c & d\end{bmatrix}  = \begin{bmatrix}\beta a & \beta b\ \beta c & \beta d \end{bmatrix} $$
What is $ 3 * U$ ?
What is $2.5 * W$?
3. Matrix Multiplication
$$ A_{m \times n} * B_{n \times p} = C_{m \times p} $$
Example 1
$$ A_{2 \times 2} * B_{2 \times 2} = C_{2 \times 2} $$
$$ \begin{bmatrix}a & b\ c & d\end{bmatrix} \begin{bmatrix}e & f\ g & h\end{bmatrix}   = \begin{bmatrix}ae + bg & af + bh\ ce + dg & cf + dh\end{bmatrix} $$
Example 2
$$ A_{3 \times 3} * B_{3 \times 1} = C_{3 \times 1} $$
$$ \begin{bmatrix}a & b & c \ d & e & f \ g & e & f\end{bmatrix} \begin{bmatrix}x \ y \ z \end{bmatrix}   = \begin{bmatrix}ax + by+ cz \ dx + ey + fz \ gx + ey + fz \end{bmatrix} $$
Here is a visual explanation for this - http://matrixmultiplication.xyz/
What is $ U * V$?
What is $V * U$?
What is $ U * W$?
What is $ W * U$? Why does this not work?
What is $ Z * U$?
4. Matrix Inverse
What is inverse of $U$ i.e. $U^{-1}$?
What is inverse of $T$ i.e. $T^{-1}$? Why does this not work?
Can you write Python code for it?


I am working on the file "Module_1a_linear_algebra_inverse.ipynb".
The first task is:
3 x 3 Equation
Let take a more involved example - 3 linear equations, with 3 unknown.
$$ x + y + z = 3 $$
$$ 3x + 8y + z = 12 $$
$$ 5x - 4y + 3z = 4 $$
Parallel
$$ 4x + 9y + 2z = 15 $$
Algebaric (3 x 3)
Now it is easy to solve the equation and get the answer
$$ x = 1, y = 1, z = 1 $$
Row-wise / Planes (3 x 3)
Can you write Python code for it?

from mpl_toolkits.mplot3d import Axes3D
xrange = np.arange(-10, 10, 1)
yrange = np.arange(-10, 10, 1)
x, y = np.meshgrid(xrange, yrange)

z1 = 3 - x - y
z2 = 12 - 3*x - 8*y
z3 = (15  - 4*x -9 *y)/(2)

plt3d = plt.figure().gca(projection='3d')

plt3d.plot_surface(x,y,z1, color='blue', alpha = 0.4)
plt3d.plot_surface(x,y,z2, color='red', alpha = 0.4)
plt3d.plot_surface(x,y,z3, color='green', alpha = 0.4)

plt3d.set_xlabel('x')
plt3d.set_ylabel('y')
plt3d.set_zlabel('z')


I am working on the file "Module_1a_linear_algebra_inverse.ipynb".
The first task is:
Column-wise  / Vectors (2x2)
We can also solve this equation in a vector way, by thinking of this as linear combination of two vectors
$$ \begin{bmatrix}1 \ 2\end{bmatrix} x + \begin{bmatrix}3 \ -1\end{bmatrix} y = \begin{bmatrix}15 \ 2\end{bmatrix} $$
Now we need to draw these vectors and see the result
Can you write Python code for it?

# All the vectors start at 0, 0
vX1 = np.array([0,0,1,2])
vY1 = np.array([0,0,3,-1])
b = np.array([0,0,15,2])

vector1 = [vX1, vY1, b]

X,Y,U,V = zip(*vector1)

X,Y,U,V

def vector_plot (vector):
    X,Y,U,V = zip(*vector)
    C = [1,2,3]
    plt.figure()
    ax = plt.gca()
    ax.quiver(X,Y,U,V,C, angles='xy',scale_units='xy',scale=1)
    ax.set_xlim([-15,15])
    ax.set_ylim([-9,9])
    plt.axhline(0, color='grey', linewidth=1)
    plt.axvline(0, color='grey', linewidth=1)
    plt.axes().set_aspect('equal')

vector_plot(vector1)


I am working on the file "Module_1a_linear_algebra_inverse.ipynb".
The first task is:
Column-wise  / Vectors (3 x 3)
$$ \begin{bmatrix}1 \ 3 \ 5\end{bmatrix} x + \begin{bmatrix}1 \ 8 \ -4\end{bmatrix} y + \begin{bmatrix}1 \ 1 \ 3\end{bmatrix} z = \begin{bmatrix}3 \ 12 \ 4 \end{bmatrix} $$
Now we need to draw these vectors and see the result
Can you write Python code for it?

from mpl_toolkits.mplot3d import Axes3D
from matplotlib.patches import FancyArrowPatch
from mpl_toolkits.mplot3d import proj3d

class Arrow3D(FancyArrowPatch):
    def __init__(self, xs, ys, zs, *args, **kwargs):
        FancyArrowPatch.__init__(self, (0,0), (0,0), *args, **kwargs)
        self._verts3d = xs, ys, zs

    def draw(self, renderer):
        xs3d, ys3d, zs3d = self._verts3d
        xs, ys, zs = proj3d.proj_transform(xs3d, ys3d, zs3d, renderer.M)
        self.set_positions((xs[0],ys[0]),(xs[1],ys[1]))
        FancyArrowPatch.draw(self, renderer)

plt.figure()
ax = plt.gca(projection='3d')
vX = Arrow3D([0,1],[0,3],[0,5], mutation_scale=20, lw=3, arrowstyle="-|>", color="r")
vY = Arrow3D([1,2],[3,11],[5,1],mutation_scale=20, lw=3, arrowstyle="-|>", color="c")
vZ = Arrow3D([2,3],[11,12],[1,4], mutation_scale=20, lw=3, arrowstyle="-|>", color="g")
b = Arrow3D([0,3],[0,12],[0,4],mutation_scale=20, lw=3, arrowstyle="-|>", color="k")

ax.add_artist(vX)
ax.add_artist(vY)
ax.add_artist(vZ)
ax.add_artist(b)

ax.set_xlim([0,12])
ax.set_ylim([0,12])
ax.set_zlim([0,12])
plt.draw()


I am working on the file "Module_1a_linear_algebra_inverse.ipynb".
The first task is:
Matrix Way (2x2) - Using Inverse
Now our 2 variable, 2 equation problem is:
$$ x + 3y = 15 ~~~~ \space (eq1)$$
$$ 2x - y = 2 ~~~~ \space (eq2)$$
Now we know from the previous working that:
$$ E_{2}E_{1}Ax = E_{2}E_{1}b $$
and that I can change the brackets in my multiplication:
$$ E_{2}(E_{1}A) = (E_{2}E_{1})A $$
So I can first compute $E_{2}*E_{1}$ :
$$ E_{2}E_{1} =\begin{bmatrix}1 & 3/7\ 0 & -1/7\end{bmatrix} \begin{bmatrix}1 & 0\ -2 & 1\end{bmatrix} = \begin{bmatrix}1/7 & 3/7\ 2/7 & -1/7\end{bmatrix} $$
And I know that
$$ (E_{2}E_{1})A = I $$
$$ \begin{bmatrix}1/7 & 3/7\ 2/7 & -1/7\end{bmatrix} \begin{bmatrix}1 & 3\ 2 & -1\end{bmatrix} = \begin{bmatrix}1 & 0\ 0 & 1\end{bmatrix} $$
So now instead of calculating Elimination matrix, we need to just calculate $A^{-1}$ which when multiplied by $A$ gives the identity matrix - $I$
$$ A^{-1}A = I $$
$$ A^{-1} = \begin{bmatrix}1/7 & 3/7\ 2/7 & -1/7\end{bmatrix} $$
Hence it follows that
$$ A^{-1}Ax = A^{-1}b $$
$$ Ix = A^{-1}b $$
$$ x = A^{-1}b $$
So we can calcuate $x$ easily once we know $A^{-1}$
$$ \begin{bmatrix}x \ y\end{bmatrix} = \begin{bmatrix}1/7 & 3/7\ 2/7 & -1/7\end{bmatrix} \begin{bmatrix}15\ 2\end{bmatrix}  = \begin{bmatrix} 3 \ 4 \end{bmatrix}$$
Can you write Python code for it?

E2*E1

Ainv = np.linalg.inv(A)

Ainv

Ainv * b


I am working on the file "Module_1b_linear_regression_ols.ipynb".
The first task is:
Non Linear Transformation
What happens when we do Non-Linear transforms to the features?
What if we want to predict $price$ based on $kmpl$, $bhp$, $kmpl^2$ and $bhp / kmpl$
The think to remember is that non-linear transforms of the features does not impact the Linear Regression. Because the linear relationship is really about $\beta $ and not the features.
We can be write this  as:
$$ price = \beta_{0} + \beta_{1} kmpl + \beta_{2} bhp + \beta_{3} kmpl^2 + \beta_{4} bhp/kmpl $$
Can you write Python code for it?

df['kmpl2'] = np.power(df.kmpl,2)

plt.scatter(df.kmpl2, df.price, s = 150)
plt.xlabel('kmpl2')
plt.ylabel('price')

df['bhp_kmpl'] = np.divide(df.bhp, df.kmpl)

plt.scatter(df.bhp_kmpl, df.price, s = 150)
plt.xlabel('bhp/kmpl')
plt.ylabel('price')

df


I am working on the file "Module_1b_linear_regression_ols.ipynb".
The first task is:
OLS Solution
Hence we now know that the best-fit line is $\beta_0 = 1662 $ and $\beta_1 = -62$
$$ price = 1662 - 62 * kmpl $$
Can you write Python code for it?

beta_0 = 1662
beta_1 = -62
plt.scatter(kmpl, price, s = 150)
plt.xlabel('kmpl')
plt.ylabel('price')
y = beta_0 + beta_1 * kmpl
plt.plot(kmpl, y, '-')


I am working on the file "Module_1b_linear_regression_ols.ipynb".
The first task is:
Represent Matrix Way
If we write this in matrix form
$$ y = X\beta + \epsilon $$
$$ \text{where} ~~~~  X = \begin{bmatrix} - x_{1}^T- \ - x_{2}^T- \ ... \ - x_{n}^T-  \end{bmatrix} ~~ \text{,} ~~ y = \begin{bmatrix} y_{1} \ y_{2} \ ... \ y_{n} \end{bmatrix} ~~ \text{and} ~~ \epsilon = \begin{bmatrix} \epsilon_{1} \ \epsilon_{2} \ ... \ \epsilon_{n} \end{bmatrix} $$
For our specific example, the matrix looks like:
$$ \begin{bmatrix}199 \ 248 \ 302 \ 363 \ 418 \ 462 \ 523 \end{bmatrix} = \begin{bmatrix} 1 & 23.9 \ 1 & 22.7 \ 1 & 21.1 \ 1 & 20.5 \ 1 & 19.8 \ 1 & 20.4 \ 1 & 18.6  \end{bmatrix}  \begin{bmatrix}
\beta_{0} \ \beta_{1} \end{bmatrix} +  \begin{bmatrix} \epsilon_{1} \ \epsilon_{2} \ \epsilon_{3} \ \epsilon_{4} \ \epsilon_{5} \ \epsilon_{6} \ \epsilon_{7}  \end{bmatrix} $$
Minimize Error - Ordinary Least Square
The error we will aim to minimize is the squared error:
$$ E(\beta)= \frac {1}{n} \sum {i=1}^{n}(\epsilon{i})^2 $$
This is why this technique is called Ordinary Least Square (OLS) regression
$$ E(\beta)= \frac {1}{n} \sum {i=1}^{n}(y{i}-x_{i}^{T}\beta)^{2}  $$
which in matrix way is equal to:
$$ E(\beta)=  \frac {1}{n} (y-X\beta)^{T}(y-X\beta) $$
$$ E(\beta)=   \frac {1}{n} ((y^{T} - \beta^{T}X^{T})(y-X\beta)) $$
$$ E(\beta)=   \frac {1}{n} (y^{T}y - \beta^{T}X^{T}y - y^{T}X\beta - \beta^{T}X^{T}X\beta) $$
Now, $ y^{T}X\beta = {\beta^{T}X^{T}y}^T $ and is a scalar matrix of $1 x 1$, which means it is equal to its transpose and hence  $ y^{T}X\beta = \beta^{T}X^{T}y $
$$ E(\beta)=   \frac {1}{n} (y^{T}y - 2\beta^{T}X^{T}y + \beta^{T}X^{T}X\beta) $$
To get the minimum for this error function, we need to differentiate by $\beta^T$
$$ \nabla E(\beta) = 0 $$
$$ \nabla E(\beta) ={\frac {dE(\beta)}{d\beta^T}} =  {\frac {d}{d\beta^T}}{\bigg (}{ \frac {1}{n} ||y - X\beta||}^2{\bigg )} = 0 $$
$$ {\frac {d}{d\beta^T}}{\bigg (}{ y^{T}y - 2\beta^{T}X^{T}y + \beta^{T}X^{T}X\beta}{\bigg )} = 0 $$
$$ - 2 X^Ty + 2X^{T}X\beta = 0 $$
$$ X^T X\beta = X^T y $$
So the solution to OLS:
$$ \beta = X^†y ~~ \text{where}...
Can you write Python code for it?

n = 7

x0 = np.ones(n)
x0

x1 = kmpl
x1

# Create the X matrix
X = np.c_[x0, x1]
X = np.asmatrix(X)
X

# Create the y matrix
y = np.asmatrix(price.reshape(-1,1))
y

y.shape

X_T = np.transpose(X)
X_T

X_T * X

X_pseudo = np.linalg.inv(X_T * X) * X_T
X_pseudo

beta = X_pseudo * y
beta


I am working on the file "Module_1b_linear_regression_ols.ipynb".
The first task is:
So a linear relationship can be written as:
$$ price = \beta_{0} + \beta_{1} kmpl + \beta_{2} bhp $$
We have added an intercept to the equation, so that the plane does not need to pass through zero
So we are trying to solve these n = 7 equations with, p = 3
$$ 199 = \beta_{0} + \beta_{1} 23.9 + \beta_{2} 38 + \epsilon_{1} ~~~~ \text{(eq 1)} $$
$$ 248 = \beta_{0} + \beta_{1} 22.7 + \beta_{2} 47 + \epsilon_{2} ~~~~ \text{(eq 2)} $$
$$ 302 = \beta_{0} + \beta_{1} 21.1 + \beta_{2} 55 + \epsilon_{3} ~~~~ \text{(eq 3)} $$
$$ 363 = \beta_{0} + \beta_{1} 20.5 + \beta_{2} 67 + \epsilon_{4} ~~~~ \text{(eq 4)} $$
$$ 418 = \beta_{0} + \beta_{1} 19.8 + \beta_{2} 68 + \epsilon_{5} ~~~~ \text{(eq 5)} $$
$$ 462 = \beta_{0} + \beta_{1} 20.4 + \beta_{2} 83 + \epsilon_{6} ~~~~ \text{(eq 6)} $$
$$ 523 = \beta_{0} + \beta_{1} 18.6 + \beta_{2} 82 + \epsilon_{7} ~~~~ \text{(eq 7)} $$
or in matrix form - we can write it as
$$ \begin{bmatrix}199 \ 248 \ 302 \ 363 \ 418 \ 462 \ 523 \end{bmatrix} = \begin{bmatrix} 1 & 23.9 & 38 \ 1 & 22.7 & 47 \ 1 & 21.1 & 55 \ 1 & 20.5 & 67 \ 1 & 19.8 & 68 \ 1 & 20.4 & 83 \ 1 & 18.6 & 82  \end{bmatrix}  \begin{bmatrix}\beta_{0} \ \beta_{1} \ \beta_{2}\end{bmatrix} + \begin{bmatrix} \epsilon_{1} \ \epsilon_{2} \ \epsilon_{3} \ \epsilon_{4} \ \epsilon_{5} \ \epsilon_{6} \ \epsilon_{7}  \end{bmatrix}$$
Develop the $X$ matrix for this problem?
Develop the $y$ matrix for this problem?
Calculate the pseudo inverse of $X$.
Find the $\beta$ for the best-fit plane.
Plot the price, kmpl and bhp and the best-fit plane.
Can you write Python code for it?

from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure()
ax = fig.gca(projection='3d')
ax.scatter(bhp, kmpl, price, c='r', marker='o', s = 200)

xrange = np.arange(min(bhp), max(bhp), 1)
yrange = np.arange(min(kmpl), max(kmpl), 1)
x, y = np.meshgrid(xrange, yrange)
z = 524 - 22 * y + 4 * x
ax.plot_surface(x, y, z, color ='blue', alpha = 0.5)
ax.view_init(azim=60)


I am working on the file "Module_1b_linear_regression_ols.ipynb".
The first task is:
Using a package: sklearn
Run the Ordinary Least Square using the package sklearn
Can you write Python code for it?

import pandas as pd
df = pd.read_csv("data/cars_sample.csv")

from sklearn import linear_model

y = df.price

X = df[['kmpl', 'bhp']]

model_sklearn = linear_model.LinearRegression()

model_sklearn.fit(X, y)

model_sklearn.coef_

model_sklearn.intercept_

model_sklearn_norm = linear_model.LinearRegression(normalize = True)

model_sklearn_norm.fit(X, y)

model_sklearn_norm.coef_

model_sklearn_norm.intercept_


I am working on the file "Module_1c_linear_regression_ridge.ipynb".
The first task is:
L2 Regularization - Ridge Regression
Now to prevent our $\beta $ from going all over the place to fit the line, we can need to constrain the constraint $\beta$
$$ \beta^{T} \beta < C $$
For OLS our error term was:
$$ E_{ols}(\beta)=  \frac {1}{n} (y-X\beta)^{T}(y-X\beta) $$
So now we add another constraint on the $\beta$ to our minimization function
$$ E_{reg}(\beta)=  \frac {1}{n} (y-X\beta)^{T}(y-X\beta)  + \frac {\alpha}{n} \beta^{T}\beta$$
To get the minimum for this error function, we need to differentiate by $\beta^T$
$$ \nabla E_{reg}(\beta) = 0 $$
$$ \nabla E_{reg}(\beta) ={\frac {dE_{reg}(\beta)}{d\beta^T}} = \frac {2}{n} X^T(X\beta−y) + \frac {\alpha}{n} \beta= 0 $$
$$ X^T X\beta + \alpha \beta= X^T y $$
So our $\beta$ for a regularized function is
$$ \beta_{reg} = (X^T X + \alpha I)^{-1}X^Ty$$
When $ \alpha = 0 $, then it becomes OLS
$$  \beta_{ols} = (X^T X)^{-1}X^Ty$$
Direct Calculation
Can you write Python code for it?

def ridge (df, alpha):
    n = df.shape[0]
    x0 = np.ones(n)
    x1 = df.kmpl
    X = np.c_[x0, x1]
    X = np.asmatrix(X)
    y = np.asmatrix(df.price.values.reshape(-1,1))
    X_T = np.transpose(X)
    I = np.identity(2)
    beta = np.linalg.inv(X_T * X + alpha * I ) * X_T * y
    return beta


I am working on the file "Module_1c_linear_regression_ridge.ipynb".
The first task is:
Let us do this 500 times, $ z = 500 $
Can you write Python code for it?


I am working on the file "Module_1c_linear_regression_ridge.ipynb".
The first task is:
Let us find the best-fit OLS line for the population
Can you write Python code for it?

def ols (df):
    n = df.shape[0]
    x0 = np.ones(n)
    x1 = df.kmpl
    X = np.c_[x0, x1]
    X = np.asmatrix(X)
    y = np.transpose(np.asmatrix(df.price))
    X_T = np.transpose(X)
    X_pseudo = np.linalg.inv(X_T * X) * X_T
    beta = X_pseudo * y
    return beta

ols(sample)

ols(pop)


I am working on the file "Module_1c_linear_regression_ridge.ipynb".
The first task is:
Let us run this with slpha = 0, which is OLS
Can you write Python code for it?


I am working on the file "Module_1c_linear_regression_ridge.ipynb".
The first task is:
Lets plot the entire population (n = 42) and the sample (n =7) and our original prediction line.
$$ price = 1662 - 62 * kmpl ~~~~ \textit{(sample = 7)}$$
Can you write Python code for it?

# Plot the population and the sample
plt.scatter(pop.kmpl, pop.price, s = 150, alpha = 0.5 )
plt.scatter(sample.kmpl, sample.price, s = 150, alpha = 0.5, c = 'r')

# Plot the OLS Line - Sample
beta_0_s, beta_1_s = 1662, -62
x = np.arange(min(pop.kmpl),max(pop.kmpl),1)

plt.xlabel('kmpl')
plt.ylabel('price')

y_s = beta_0_s + beta_1_s * x
plt.text(x[-1], y_s[-1], 'sample')

plt.plot(x, y_s, '-')


I am working on the file "Module_1c_linear_regression_ridge.ipynb".
The first task is:
So the two OLS lines are:
$$ price = 1662 - 62 * kmpl ~~~~ \textit{(sample = 7)}$$
$$ price = 1158 - 36 * kmpl ~~~~ \textit{(population = 42)}$$
Let us plot this data:
Can you write Python code for it?

# Plot the population and the sample
plt.scatter(pop.kmpl, pop.price, s = 150, alpha = 0.5 )
plt.scatter(sample.kmpl, sample.price, s = 150, alpha = 0.5, c = 'r')

# Plot the OLS Line - sample and population
beta_0_s, beta_1_s = 1662, -62
beta_0_p, beta_1_p = 1158, -36
x = np.arange(min(pop.kmpl),max(pop.kmpl),1)

plt.xlabel('kmpl')
plt.ylabel('price')

y_s = beta_0_s + beta_1_s * x
plt.text(x[-1], y_s[-1], 'sample')

y_p = beta_0_p + beta_1_p * x
plt.text(x[-1], y_p[-1], 'population')

plt.plot(x, y_s, '-')
plt.plot(x, y_p, '-')


I am working on the file "Module_1c_linear_regression_ridge.ipynb".
The first task is:
Understanding Overfitting
The reason overfitting is happening is because our orginal line is really dependent on the selection of the sample of 7 observations. If we change our sample, we would get a different answer every time!
Can you write Python code for it?

# Randomly select 7 cars from this dataset
sample_random = pop.sample(n=7)
sample_random


I am working on the file "Module_1d_linear_regression_gradient.ipynb".
The first task is:
Avoid Overshooting
Let see the performance difference between Simple Gradient Descent and Line Search Gradient Descent
Can you write Python code for it?

errors, b0s, b1s = gradient_descent_line (1, 50000, X, y, n)

errors, b0s, b1s = gradient_descent (0.0005, 50000, X, y, n)