I am working on the file "05_resilience/serving_function.ipynb".
The first task is:
Predict using the trained model in-memory
Can you write Python code for it?

review1 = 'The film is based on a prize-winning novel.' # neutral
review2 = 'The film is fast moving and has several great action scenes.' # positive
review3 = 'The film was very boring. I walked out half-way.' # negative

logits = model.predict(x=tf.constant([review1, review2, review3]))
print(logits)

## how big is the model in memory?
import sys

# From https://goshippo.com/blog/measure-real-size-any-python-object/
def get_size(obj, seen=None):
    """Recursively finds size of objects"""
    size = sys.getsizeof(obj)
    if seen is None:
        seen = set()
    obj_id = id(obj)
    if obj_id in seen:
        return 0
    # Important mark as seen *before* entering recursion to gracefully handle
    # self-referential objects
    seen.add(obj_id)
    try:
        if isinstance(obj, dict):
            size += sum([get_size(v, seen) for v in obj.values()])
            size += sum([get_size(k, seen) for k in obj.keys()])
        elif hasattr(obj, '__dict__'):
            size += get_size(obj.__dict__, seen)
        elif hasattr(obj, '__iter__') and not isinstance(obj, (str, bytes, bytearray)):
            size += sum([get_size(i, seen) for i in obj])
    except:
        pass
    return size
print('{} MB'.format(get_size(model)/(1000*1000)))


I am working on the file "05_resilience/serving_function.ipynb".
The first task is:
This works, but it is too slow
Can you write Python code for it?

%%bigquery
SELECT * FROM ML.PREDICT(MODEL mlpatterns.neutral_3classes,
    (SELECT
     2 AS plurality,
     32 AS mother_age,
     41 AS gestation_weeks,
     1 AS ever_born
    )
)


I am working on the file "05_resilience/two_phase_predictions.ipynb".
The first task is:
Add a column for our first model: whether the sound is an instrument or not
Can you write Python code for it?

instrument_labels = ['Cello', 'Clarinet', 'Double_bass', 'Saxophone', 'Violin_or_fiddle', 'Snare_drum', 'Hi-hat', 'Flute', 'Bass_drum', 'Trumpet', 'Acoustic_guitar', 'Oboe', 'Gong', 'Tambourine', 'Cowbell', 'Harmonica', 'Electric_piano', 'Glockenspiel']

def check_instrument(row):
  if row['label'] in instrument_labels:
    return 1
  else:
    return 0

label_data['is_instrument'] = label_data.apply(check_instrument, axis=1)

label_data.head()

label_data['is_instrument'].value_counts()


I am working on the file "05_resilience/two_phase_predictions.ipynb".
The first task is:
Build a model for binary classification: is it an instrument or not? We'll use transfer learning for this, by loading the MobileNet V2 model architecture trained on the ImageNet dataset and then adding a few additional layers on top specific to our prediction task.
Can you write Python code for it?

mobilenet = tf.keras.applications.MobileNetV2(
    input_shape=((128,128,3)),
    include_top=False,
    weights='imagenet'
)

mobilenet.trainable = False

feature_batch = mobilenet(image_batch)

global_avg_layer = tf.keras.layers.GlobalAveragePooling2D()
feature_batch_avg = global_avg_layer(feature_batch)
print(feature_batch_avg.shape)

prediction_layer = tf.keras.layers.Dense(1, activation='sigmoid')
prediction_batch = prediction_layer(feature_batch_avg)
print(prediction_batch.shape)

model = tf.keras.Sequential([
  mobilenet,
  global_avg_layer,
  prediction_layer
])

model.summary()

model.compile(optimizer='SGD',
              loss='binary_crossentropy',
              metrics=['accuracy'])


I am working on the file "05_resilience/two_phase_predictions.ipynb".
The first task is:
Convert model to TFLite and quantize
Save the TF Lite model file and get predictions on it using Python.
Can you write Python code for it?

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()
open('converted_model.tflite', 'wb').write(tflite_model)


I am working on the file "05_resilience/two_phase_predictions.ipynb".
The first task is:
Create directories for each instrument label. We'll use this later when we load our images with Keras's ImageDataGenerator class.
Can you write Python code for it?

for i in os.listdir(os.getcwd() + '/audio_train_spectro'):
    id = i.split('.')[0] + '.wav'
    try:
        instrument_name = instrument_label_lookup[id]
        im_path = os.getcwd() + '/audio_train_spectro/' + i
        new_path = os.getcwd() + '/instruments/' + instrument_name + '/' + i
        !mv $im_path $new_path
    except:
        pass

instrument_image_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255, validation_split=0.1)

BATCH_SIZE = 256
IMG_HEIGHT = 128
IMG_WIDTH = 128
STEPS_PER_EPOCH = np.ceil(2002/BATCH_SIZE)

train_data_instrument = instrument_image_generator.flow_from_directory(directory=os.getcwd() + '/instruments',
                                                     batch_size=BATCH_SIZE,
                                                     shuffle=True,
                                                     target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                     classes = instrument_labels)

val_data_instrument = instrument_image_generator.flow_from_directory(directory=os.getcwd() + '/instruments',
                                                     batch_size=BATCH_SIZE,
                                                     shuffle=False,
                                                     target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                     classes = instrument_labels,
                                                     subset='validation')

image_instrument_train, label_instrument_train = next(train_data_instrument)

image_instrument_val, label_instrument_val = next(val_data_instrument)

vgg_model = tf.keras.applications.VGG19(
    include_top=False,
    weights='imagenet',
    input_shape=((128,128,3))
)

vgg_model.trainable = False

feature_batch = vgg_model(image_batch)
global_avg_layer = tf.keras.layers.GlobalAveragePooling2D()
feature_batch_avg = global_avg_layer(feature_batch)
print(feature_batch_avg.shape)

prediction_layer = tf.keras.layers.Dense(len(instrument_labels),...


I am working on the file "05_resilience/two_phase_predictions.ipynb".
The first task is:
Download all the spectrogram data
Can you write Python code for it?

# This might take a few minutes
!gsutil -m cp -r gs://ml-design-patterns/audio_train_spectro .


I am working on the file "05_resilience/two_phase_predictions.ipynb".
The first task is:
Get a prediction on one spectrogram from our validation dataset, print the model's prediction (sigmoid probability from 0 to 1) along with ground truth label. For more details, check out the TF Lite inference docs.
Can you write Python code for it?

# Load TFLite model and allocate tensors.
interpreter = tf.lite.Interpreter(model_path="converted_model.tflite")
interpreter.allocate_tensors()

# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

# Test model on random input data.
input_shape = input_details[0]['shape']
input_data = np.array([val_image[0]], dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)

interpreter.invoke()

# The function `get_tensor()` returns a copy of the tensor data.
# Use `tensor()` in order to get a pointer to the tensor.
output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)
print(val_label[0])

input_details


I am working on the file "05_resilience/two_phase_predictions.ipynb".
The first task is:
Get a test prediction on the instrument model
Can you write Python code for it?

test_pred = instrument_model.predict(np.array([image_instrument_val[0]]))

predicted_index = np.argmax(test_pred)
confidence = test_pred[0][predicted_index]

test_pred[0]

print('Predicted instrument: ', instrument_labels[predicted_index], round(confidence * 100), '% confidence')
print('Actual instrument: ', instrument_labels[np.argmax(label_instrument_val[0])])


I am working on the file "05_resilience/two_phase_predictions.ipynb".
The first task is:
Load the images as a tf dataset
Can you write Python code for it?

data_dir = pathlib.Path(os.getcwd() + '/audio')
class_names = ['not_instrument', 'instrument']

BATCH_SIZE = 64
IMG_HEIGHT = 128
IMG_WIDTH = 128
STEPS_PER_EPOCH = np.ceil(3700/BATCH_SIZE)

image_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255, validation_split=0.1)

train_data_gen = image_generator.flow_from_directory(directory=data_dir,
                                                     batch_size=BATCH_SIZE,
                                                     shuffle=True,
                                                     target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                     classes = class_names,
                                                     class_mode='binary')

val_data_gen = image_generator.flow_from_directory(directory=data_dir,
                                                     batch_size=BATCH_SIZE,
                                                     shuffle=True,
                                                     target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                     classes = class_names,
                                                     class_mode='binary',
                                                    subset='validation')

image_count = len(list(data_dir.glob('*/*.png')))
print(image_count)

instrument_modellabel_batch = next(train_data_gen)

val_image, val_label = next(val_data_gen)


I am working on the file "05_resilience/two_phase_predictions.ipynb".
The first task is:
Note: we could make changes to our model architecture, perform progressive fine-tuning to find the optimal number of layers to fine-tune, or employ hyperparameter tuning to improve model accuracy. Here the focus is on tooling and process rather than accuracy.
Can you write Python code for it?

model.fit_generator(train_data_gen,
                    validation_data=val_data_gen,
                    steps_per_epoch=STEPS_PER_EPOCH, epochs=10)


instrument_model.fit_generator(
    train_data_instrument,
    validation_data=val_data_instrument,
    steps_per_epoch=STEPS_PER_EPOCH, epochs=10)


I am working on the file "05_resilience/two_phase_predictions.ipynb".
The first task is:
To ensure quality, we'll only use manually verified samples from the dataset.
Can you write Python code for it?

verified = label_data.loc[label_data['manually_verified'] == 1]

verified['is_instrument'].value_counts()

verified.head()


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Ah, this looks good! Let's select this model. Now we can test it on the test set:
Can you write Python code for it?

y_pred = rnd_search_cv.best_estimator_.predict(X_test_scaled)
accuracy_score(y_test, y_pred)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Close enough!
9.
Exercise: train an SVM classifier on the MNIST dataset. Since SVM classifiers are binary classifiers, you will need to use one-versus-all to classify all 10 digits. You may want to tune the hyperparameters using small validation sets to speed up the process. What accuracy can you reach?
First, let's load the dataset and split it into a training set and a test set. We could use train_test_split() but people usually just take the first 60,000 instances for the training set, and the last 10,000 instances for the test set (this makes it possible to compare your model's performance with others):
Can you write Python code for it?

try:
    from sklearn.datasets import fetch_openml
    mnist = fetch_openml('mnist_784', version=1, cache=True, as_frame=False)
except ImportError:
    from sklearn.datasets import fetch_mldata
    mnist = fetch_mldata('MNIST original')

X = mnist["data"]
y = mnist["target"]

X_train = X[:60000]
y_train = y[:60000]
X_test = X[60000:]
y_test = y[60000:]


from sklearn.datasets import fetch_mldata

mnist = fetch_mldata("MNIST original")
X = mnist["data"]
y = mnist["target"]

X_train = X[:60000]
y_train = y[:60000]
X_test = X[60000:]
y_test = y[60000:]


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Don't forget to scale the data:
Can you write Python code for it?

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Exercise solutions
1. to 7.
See appendix A.
8.
Exercise: train a LinearSVC on a linearly separable dataset. Then train an SVC and a SGDClassifier on the same dataset. See if you can get them to produce roughly the same model.
Let's use the Iris dataset: the Iris Setosa and Iris Versicolor classes are linearly separable.
Can you write Python code for it?

from sklearn import datasets

iris = datasets.load_iris()
X = iris["data"][:, (2, 3)]  # petal length, petal width
y = iris["target"]

setosa_or_versicolor = (y == 0) | (y == 1)
X = X[setosa_or_versicolor]
y = y[setosa_or_versicolor]

from sklearn.svm import SVC, LinearSVC
from sklearn.linear_model import SGDClassifier
from sklearn.preprocessing import StandardScaler

C = 5
alpha = 1 / (C * len(X))

lin_clf = LinearSVC(loss="hinge", C=C, random_state=42)
svm_clf = SVC(kernel="linear", C=C)
sgd_clf = SGDClassifier(loss="hinge", learning_rate="constant", eta0=0.001, alpha=alpha,
                        max_iter=100000, tol=-np.infty, random_state=42)

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

lin_clf.fit(X_scaled, y)
svm_clf.fit(X_scaled, y)
sgd_clf.fit(X_scaled, y)

print("LinearSVC:                   ", lin_clf.intercept_, lin_clf.coef_)
print("SVC:                         ", svm_clf.intercept_, svm_clf.coef_)
print("SGDClassifier(alpha={:.5f}):".format(sgd_clf.alpha), sgd_clf.intercept_, sgd_clf.coef_)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Extra material
Training time
Can you write Python code for it?

X, y = make_moons(n_samples=1000, noise=0.4, random_state=42)
plt.plot(X[:, 0][y==0], X[:, 1][y==0], "bs")
plt.plot(X[:, 0][y==1], X[:, 1][y==1], "g^")

import time

tol = 0.1
tols = []
times = []
for i in range(10):
    svm_clf = SVC(kernel="poly", gamma=3, C=10, tol=tol, verbose=1)
    t1 = time.time()
    svm_clf.fit(X, y)
    t2 = time.time()
    times.append(t2-t1)
    tols.append(tol)
    print(i, tol, t2-t1)
    tol /= 10
plt.semilogx(tols, times)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Hinge loss
Can you write Python code for it?

t = np.linspace(-2, 4, 200)
h = np.where(1 - t < 0, 0, 1 - t)  # max(0, 1-t)

plt.figure(figsize=(5,2.8))
plt.plot(t, h, "b-", linewidth=2, label="$max(0, 1 - t)$")
plt.grid(True, which='both')
plt.axhline(y=0, color='k')
plt.axvline(x=0, color='k')
plt.yticks(np.arange(-1, 2.5, 1))
plt.xlabel("$t$", fontsize=16)
plt.axis([-2, 4, -1, 2.5])
plt.legend(loc="upper right", fontsize=16)
save_fig("hinge_plot")
plt.show()


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
In this training set, the targets are tens of thousands of dollars. The RMSE gives a rough idea of the kind of error you should expect (with a higher weight for large errors): so with this model we can expect errors somewhere around $10,000. Not great. Let's see if we can do better with an RBF Kernel. We will use randomized search with cross validation to find the appropriate hyperparameter values for C and gamma:
Can you write Python code for it?

from sklearn.svm import SVR
from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import reciprocal, uniform

param_distributions = {"gamma": reciprocal(0.001, 0.1), "C": uniform(1, 10)}
rnd_search_cv = RandomizedSearchCV(SVR(), param_distributions, n_iter=10, verbose=2, random_state=42)
rnd_search_cv.fit(X_train_scaled, y_train)

rnd_search_cv.best_estimator_


from sklearn.svm import SVR
from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import reciprocal, uniform

param_distributions = {"gamma": reciprocal(0.001, 0.1), "C": uniform(1, 10)}
rnd_search_cv = RandomizedSearchCV(SVR(), param_distributions, n_iter=10, verbose=2, cv=3, random_state=42)
rnd_search_cv.fit(X_train_scaled, y_train)

rnd_search_cv.best_estimator_


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Large margin classification
The next few code cells generate the first figures in chapter 5. The first actual code sample comes after:
Can you write Python code for it?

from sklearn.svm import SVC
from sklearn import datasets

iris = datasets.load_iris()
X = iris["data"][:, (2, 3)]  # petal length, petal width
y = iris["target"]

setosa_or_versicolor = (y == 0) | (y == 1)
X = X[setosa_or_versicolor]
y = y[setosa_or_versicolor]

# SVM Classifier model
svm_clf = SVC(kernel="linear", C=float("inf"))
svm_clf.fit(X, y)

# Bad models
x0 = np.linspace(0, 5.5, 200)
pred_1 = 5*x0 - 20
pred_2 = x0 - 1.8
pred_3 = 0.1 * x0 + 0.5

def plot_svc_decision_boundary(svm_clf, xmin, xmax):
    w = svm_clf.coef_[0]
    b = svm_clf.intercept_[0]

    # At the decision boundary, w0*x0 + w1*x1 + b = 0
    # => x1 = -w0/w1 * x0 - b/w1
    x0 = np.linspace(xmin, xmax, 200)
    decision_boundary = -w[0]/w[1] * x0 - b/w[1]

    margin = 1/w[1]
    gutter_up = decision_boundary + margin
    gutter_down = decision_boundary - margin

    svs = svm_clf.support_vectors_
    plt.scatter(svs[:, 0], svs[:, 1], s=180, facecolors='#FFAAAA')
    plt.plot(x0, decision_boundary, "k-", linewidth=2)
    plt.plot(x0, gutter_up, "k--", linewidth=2)
    plt.plot(x0, gutter_down, "k--", linewidth=2)

plt.figure(figsize=(12,2.7))

plt.subplot(121)
plt.plot(x0, pred_1, "g--", linewidth=2)
plt.plot(x0, pred_2, "m-", linewidth=2)
plt.plot(x0, pred_3, "r-", linewidth=2)
plt.plot(X[:, 0][y==1], X[:, 1][y==1], "bs", label="Iris-Versicolor")
plt.plot(X[:, 0][y==0], X[:, 1][y==0], "yo", label="Iris-Setosa")
plt.xlabel("Petal length", fontsize=14)
plt.ylabel("Petal width", fontsize=14)
plt.legend(loc="upper left", fontsize=14)
plt.axis([0, 5.5, 0, 2])

plt.subplot(122)
plot_svc_decision_boundary(svm_clf, 0, 5.5)
plt.plot(X[:, 0][y==1], X[:, 1][y==1], "bs")
plt.plot(X[:, 0][y==0], X[:, 1][y==0], "yo")
plt.xlabel("Petal length", fontsize=14)
plt.axis([0, 5.5, 0, 2])

save_fig("large_margin_classification_plot")
plt.show()


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Large margin vs margin violations
This is the first code example in chapter 5:
Can you write Python code for it?

import numpy as np
from sklearn import datasets
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import LinearSVC

iris = datasets.load_iris()
X = iris["data"][:, (2, 3)]  # petal length, petal width
y = (iris["target"] == 2).astype(np.float64)  # Iris-Virginica

svm_clf = Pipeline((
        ("scaler", StandardScaler()),
        ("linear_svc", LinearSVC(C=1, loss="hinge", random_state=42)),
    ))

svm_clf.fit(X, y)

svm_clf.predict([[5.5, 1.7]])


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Let's look at the RMSE:
Can you write Python code for it?


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Let's make predictions on the training set and measure the accuracy (we don't want to measure it on the test set yet, since we have not selected and trained the final model yet):
Can you write Python code for it?

from sklearn.metrics import accuracy_score

y_pred = lin_clf.predict(X_train)
accuracy_score(y_train, y_pred)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Let's plot the decision boundaries of these three models:
Can you write Python code for it?

# Compute the slope and bias of each decision boundary
w1 = -lin_clf.coef_[0, 0]/lin_clf.coef_[0, 1]
b1 = -lin_clf.intercept_[0]/lin_clf.coef_[0, 1]
w2 = -svm_clf.coef_[0, 0]/svm_clf.coef_[0, 1]
b2 = -svm_clf.intercept_[0]/svm_clf.coef_[0, 1]
w3 = -sgd_clf.coef_[0, 0]/sgd_clf.coef_[0, 1]
b3 = -sgd_clf.intercept_[0]/sgd_clf.coef_[0, 1]

# Transform the decision boundary lines back to the original scale
line1 = scaler.inverse_transform([[-10, -10 * w1 + b1], [10, 10 * w1 + b1]])
line2 = scaler.inverse_transform([[-10, -10 * w2 + b2], [10, 10 * w2 + b2]])
line3 = scaler.inverse_transform([[-10, -10 * w3 + b3], [10, 10 * w3 + b3]])

# Plot all three decision boundaries
plt.figure(figsize=(11, 4))
plt.plot(line1[:, 0], line1[:, 1], "k:", label="LinearSVC")
plt.plot(line2[:, 0], line2[:, 1], "b--", linewidth=2, label="SVC")
plt.plot(line3[:, 0], line3[:, 1], "r-", label="SGDClassifier")
plt.plot(X[:, 0][y==1], X[:, 1][y==1], "bs") # label="Iris-Versicolor"
plt.plot(X[:, 0][y==0], X[:, 1][y==0], "yo") # label="Iris-Setosa"
plt.xlabel("Petal length", fontsize=14)
plt.ylabel("Petal width", fontsize=14)
plt.legend(loc="upper center", fontsize=14)
plt.axis([0, 5.5, 0, 2])

plt.show()


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Let's see how it performs on the training set:
Can you write Python code for it?

from sklearn.metrics import mean_squared_error

y_pred = lin_svr.predict(X_train_scaled)
mse = mean_squared_error(y_train, y_pred)
mse


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Let's start simple, with a linear SVM classifier. It will automatically use the One-vs-All (also called One-vs-the-Rest, OvR) strategy, so there's nothing special we need to do. Easy!
Can you write Python code for it?

lin_clf = LinearSVC(random_state=42)
lin_clf.fit(X_train, y_train)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Let's train a simple LinearSVR first:
Can you write Python code for it?

from sklearn.svm import LinearSVR

lin_svr = LinearSVR(random_state=42)
lin_svr.fit(X_train_scaled, y_train)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Linear SVM classifier implementation using Batch Gradient Descent
Can you write Python code for it?

# Training set
X = iris["data"][:, (2, 3)] # petal length, petal width
y = (iris["target"] == 2).astype(np.float64).reshape(-1, 1) # Iris-Virginica

from sklearn.base import BaseEstimator

class MyLinearSVC(BaseEstimator):
    def __init__(self, C=1, eta0=1, eta_d=10000, n_epochs=1000, random_state=None):
        self.C = C
        self.eta0 = eta0
        self.n_epochs = n_epochs
        self.random_state = random_state
        self.eta_d = eta_d

    def eta(self, epoch):
        return self.eta0 / (epoch + self.eta_d)

    def fit(self, X, y):
        # Random initialization
        if self.random_state:
            np.random.seed(self.random_state)
        w = np.random.randn(X.shape[1], 1) # n feature weights
        b = 0

        m = len(X)
        t = y * 2 - 1  # -1 if t==0, +1 if t==1
        X_t = X * t
        self.Js=[]

        # Training
        for epoch in range(self.n_epochs):
            support_vectors_idx = (X_t.dot(w) + t * b < 1).ravel()
            X_t_sv = X_t[support_vectors_idx]
            t_sv = t[support_vectors_idx]

            J = 1/2 * np.sum(w * w) + self.C * (np.sum(1 - X_t_sv.dot(w)) - b * np.sum(t_sv))
            self.Js.append(J)

            w_gradient_vector = w - self.C * np.sum(X_t_sv, axis=0).reshape(-1, 1)
            b_derivative = -C * np.sum(t_sv)

            w = w - self.eta(epoch) * w_gradient_vector
            b = b - self.eta(epoch) * b_derivative

        self.intercept_ = np.array([b])
        self.coef_ = np.array([w])
        support_vectors_idx = (X_t.dot(w) + b < 1).ravel()
        self.support_vectors_ = X[support_vectors_idx]
        return self

    def decision_function(self, X):
        return X.dot(self.coef_[0]) + self.intercept_[0]

    def predict(self, X):
        return (self.decision_function(X) >= 0).astype(np.float64)

C=2
svm_clf = MyLinearSVC(C=C, eta0 = 10, eta_d = 1000, n_epochs=60000, random_state=2)
svm_clf.fit(X, y)
svm_clf.predict(np.array([[5, 2], [4,...


# Training set
X = iris["data"][:, (2, 3)] # petal length, petal width
y = (iris["target"] == 2).astype(np.float64).reshape(-1, 1) # Iris-Virginica

from sklearn.base import BaseEstimator

class MyLinearSVC(BaseEstimator):
    def __init__(self, C=1, eta0=1, eta_d=10000, n_epochs=1000, random_state=None):
        self.C = C
        self.eta0 = eta0
        self.n_epochs = n_epochs
        self.random_state = random_state
        self.eta_d = eta_d

    def eta(self, epoch):
        return self.eta0 / (epoch + self.eta_d)

    def fit(self, X, y):
        # Random initialization
        if self.random_state:
            np.random.seed(self.random_state)
        w = np.random.randn(X.shape[1], 1) # n feature weights
        b = 0

        m = len(X)
        t = y * 2 - 1  # -1 if t==0, +1 if t==1
        X_t = X * t
        self.Js=[]

        # Training
        for epoch in range(self.n_epochs):
            support_vectors_idx = (X_t.dot(w) + t * b < 1).ravel()
            X_t_sv = X_t[support_vectors_idx]
            t_sv = t[support_vectors_idx]

            J = 1/2 * np.sum(w * w) + self.C * (np.sum(1 - X_t_sv.dot(w)) - b * np.sum(t_sv))
            self.Js.append(J)

            w_gradient_vector = w - self.C * np.sum(X_t_sv, axis=0).reshape(-1, 1)
            b_derivative = -C * np.sum(t_sv)

            w = w - self.eta(epoch) * w_gradient_vector
            b = b - self.eta(epoch) * b_derivative

        self.intercept_ = np.array([b])
        self.coef_ = np.array([w])
        support_vectors_idx = (X_t.dot(w) + t * b < 1).ravel()
        self.support_vectors_ = X[support_vectors_idx]
        return self

    def decision_function(self, X):
        return X.dot(self.coef_[0]) + self.intercept_[0]

    def predict(self, X):
        return (self.decision_function(X) >= 0).astype(np.float64)

C=2
svm_clf = MyLinearSVC(C=C, eta0 = 10, eta_d = 1000, n_epochs=60000, random_state=2)
svm_clf.fit(X, y)
svm_clf.predict(np.array([[5, 2], [4,...


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Looks much better than the linear model. Let's select this model and evaluate it on the test set:
Can you write Python code for it?

y_pred = rnd_search_cv.best_estimator_.predict(X_test_scaled)
mse = mean_squared_error(y_test, y_pred)
np.sqrt(mse)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Many training algorithms are sensitive to the order of the training instances, so it's generally good practice to shuffle them first:
Can you write Python code for it?

np.random.seed(42)
rnd_idx = np.random.permutation(60000)
X_train = X_train[rnd_idx]
y_train = y_train[rnd_idx]


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Non-linear classification
Can you write Python code for it?

X1D = np.linspace(-4, 4, 9).reshape(-1, 1)
X2D = np.c_[X1D, X1D**2]
y = np.array([0, 0, 1, 1, 1, 1, 1, 0, 0])

plt.figure(figsize=(11, 4))

plt.subplot(121)
plt.grid(True, which='both')
plt.axhline(y=0, color='k')
plt.plot(X1D[:, 0][y==0], np.zeros(4), "bs")
plt.plot(X1D[:, 0][y==1], np.zeros(5), "g^")
plt.gca().get_yaxis().set_ticks([])
plt.xlabel(r"$x_1$", fontsize=20)
plt.axis([-4.5, 4.5, -0.2, 0.2])

plt.subplot(122)
plt.grid(True, which='both')
plt.axhline(y=0, color='k')
plt.axvline(x=0, color='k')
plt.plot(X2D[:, 0][y==0], X2D[:, 1][y==0], "bs")
plt.plot(X2D[:, 0][y==1], X2D[:, 1][y==1], "g^")
plt.xlabel(r"$x_1$", fontsize=20)
plt.ylabel(r"$x_2$", fontsize=20, rotation=0)
plt.gca().get_yaxis().set_ticks([0, 4, 8, 12, 16])
plt.plot([-4.5, 4.5], [6.5, 6.5], "r--", linewidth=3)
plt.axis([-4.5, 4.5, -1, 17])

plt.subplots_adjust(right=1)

save_fig("higher_dimensions_plot", tight_layout=False)
plt.show()

from sklearn.datasets import make_moons
X, y = make_moons(n_samples=100, noise=0.15, random_state=42)

def plot_dataset(X, y, axes):
    plt.plot(X[:, 0][y==0], X[:, 1][y==0], "bs")
    plt.plot(X[:, 0][y==1], X[:, 1][y==1], "g^")
    plt.axis(axes)
    plt.grid(True, which='both')
    plt.xlabel(r"$x_1$", fontsize=20)
    plt.ylabel(r"$x_2$", fontsize=20, rotation=0)

plot_dataset(X, y, [-1.5, 2.5, -1, 1.5])
plt.show()

from sklearn.datasets import make_moons
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures

polynomial_svm_clf = Pipeline([
        ("poly_features", PolynomialFeatures(degree=3)),
        ("scaler", StandardScaler()),
        ("svm_clf", LinearSVC(C=10, loss="hinge", random_state=42))
    ])

polynomial_svm_clf.fit(X, y)

def plot_predictions(clf, axes):
    x0s = np.linspace(axes[0], axes[1], 100)
    x1s = np.linspace(axes[2], axes[3], 100)
    x0, x1 = np.meshgrid(x0s, x1s)
    X = np.c_[x0.ravel(), x1.ravel()]
    y_pred = clf.predict(X).reshape(x0.shape)
    y_decision = clf.decision_function(X).reshape(x0.shape)
    plt.contourf(x0, x1,...


X1D = np.linspace(-4, 4, 9).reshape(-1, 1)
X2D = np.c_[X1D, X1D**2]
y = np.array([0, 0, 1, 1, 1, 1, 1, 0, 0])

plt.figure(figsize=(11, 4))

plt.subplot(121)
plt.grid(True, which='both')
plt.axhline(y=0, color='k')
plt.plot(X1D[:, 0][y==0], np.zeros(4), "bs")
plt.plot(X1D[:, 0][y==1], np.zeros(5), "g^")
plt.gca().get_yaxis().set_ticks([])
plt.xlabel(r"$x_1$", fontsize=20)
plt.axis([-4.5, 4.5, -0.2, 0.2])

plt.subplot(122)
plt.grid(True, which='both')
plt.axhline(y=0, color='k')
plt.axvline(x=0, color='k')
plt.plot(X2D[:, 0][y==0], X2D[:, 1][y==0], "bs")
plt.plot(X2D[:, 0][y==1], X2D[:, 1][y==1], "g^")
plt.xlabel(r"$x_1$", fontsize=20)
plt.ylabel(r"$x_2$", fontsize=20, rotation=0)
plt.gca().get_yaxis().set_ticks([0, 4, 8, 12, 16])
plt.plot([-4.5, 4.5], [6.5, 6.5], "r--", linewidth=3)
plt.axis([-4.5, 4.5, -1, 17])

plt.subplots_adjust(right=1)

save_fig("higher_dimensions_plot", tight_layout=False)
plt.show()

from sklearn.datasets import make_moons
X, y = make_moons(n_samples=100, noise=0.15, random_state=42)

def plot_dataset(X, y, axes):
    plt.plot(X[:, 0][y==0], X[:, 1][y==0], "bs")
    plt.plot(X[:, 0][y==1], X[:, 1][y==1], "g^")
    plt.axis(axes)
    plt.grid(True, which='both')
    plt.xlabel(r"$x_1$", fontsize=20)
    plt.ylabel(r"$x_2$", fontsize=20, rotation=0)

plot_dataset(X, y, [-1.5, 2.5, -1, 1.5])
plt.show()

from sklearn.datasets import make_moons
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures

polynomial_svm_clf = Pipeline((
        ("poly_features", PolynomialFeatures(degree=3)),
        ("scaler", StandardScaler()),
        ("svm_clf", LinearSVC(C=10, loss="hinge", random_state=42))
    ))

polynomial_svm_clf.fit(X, y)

def plot_predictions(clf, axes):
    x0s = np.linspace(axes[0], axes[1], 100)
    x1s = np.linspace(axes[2], axes[3], 100)
    x0, x1 = np.meshgrid(x0s, x1s)
    X = np.c_[x0.ravel(), x1.ravel()]
    y_pred = clf.predict(X).reshape(x0.shape)
    y_decision = clf.decision_function(X).reshape(x0.shape)
    plt.contourf(x0, x1,...


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Not too bad, but apparently the model is overfitting slightly. It's tempting to tweak the hyperparameters a bit more (e.g. decreasing C and/or gamma), but we would run the risk of overfitting the test set. Other people have found that the hyperparameters C=5 and gamma=0.005 yield even better performance (over 98% accuracy). By running the randomized search for longer and on a larger part of the training set, you may be able to find this as well.
10.
Exercise: train an SVM regressor on the California housing dataset.
Let's load the dataset using Scikit-Learn's fetch_california_housing() function:
Can you write Python code for it?

from sklearn.datasets import fetch_california_housing

housing = fetch_california_housing()
X = housing["data"]
y = housing["target"]


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Now let's generate the graph comparing different regularization settings:
Can you write Python code for it?

scaler = StandardScaler()
svm_clf1 = LinearSVC(C=1, loss="hinge", random_state=42)
svm_clf2 = LinearSVC(C=100, loss="hinge", random_state=42)

scaled_svm_clf1 = Pipeline([
        ("scaler", scaler),
        ("linear_svc", svm_clf1),
    ])
scaled_svm_clf2 = Pipeline([
        ("scaler", scaler),
        ("linear_svc", svm_clf2),
    ])

scaled_svm_clf1.fit(X, y)
scaled_svm_clf2.fit(X, y)

# Convert to unscaled parameters
b1 = svm_clf1.decision_function([-scaler.mean_ / scaler.scale_])
b2 = svm_clf2.decision_function([-scaler.mean_ / scaler.scale_])
w1 = svm_clf1.coef_[0] / scaler.scale_
w2 = svm_clf2.coef_[0] / scaler.scale_
svm_clf1.intercept_ = np.array([b1])
svm_clf2.intercept_ = np.array([b2])
svm_clf1.coef_ = np.array([w1])
svm_clf2.coef_ = np.array([w2])

# Find support vectors (LinearSVC does not do this automatically)
t = y * 2 - 1
support_vectors_idx1 = (t * (X.dot(w1) + b1) < 1).ravel()
support_vectors_idx2 = (t * (X.dot(w2) + b2) < 1).ravel()
svm_clf1.support_vectors_ = X[support_vectors_idx1]
svm_clf2.support_vectors_ = X[support_vectors_idx2]

plt.figure(figsize=(12,3.2))
plt.subplot(121)
plt.plot(X[:, 0][y==1], X[:, 1][y==1], "g^", label="Iris-Virginica")
plt.plot(X[:, 0][y==0], X[:, 1][y==0], "bs", label="Iris-Versicolor")
plot_svc_decision_boundary(svm_clf1, 4, 6)
plt.xlabel("Petal length", fontsize=14)
plt.ylabel("Petal width", fontsize=14)
plt.legend(loc="upper left", fontsize=14)
plt.title("$C = {}$".format(svm_clf1.C), fontsize=16)
plt.axis([4, 6, 0.8, 2.8])

plt.subplot(122)
plt.plot(X[:, 0][y==1], X[:, 1][y==1], "g^")
plt.plot(X[:, 0][y==0], X[:, 1][y==0], "bs")
plot_svc_decision_boundary(svm_clf2, 4, 6)
plt.xlabel("Petal length", fontsize=14)
plt.title("$C = {}$".format(svm_clf2.C), fontsize=16)
plt.axis([4, 6, 0.8, 2.8])

save_fig("regularization_plot")


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Now let's measure the RMSE on the training set:
Can you write Python code for it?

y_pred = rnd_search_cv.best_estimator_.predict(X_train_scaled)
mse = mean_squared_error(y_train, y_pred)
np.sqrt(mse)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Regression
Can you write Python code for it?

np.random.seed(42)
m = 50
X = 2 * np.random.rand(m, 1)
y = (4 + 3 * X + np.random.randn(m, 1)).ravel()

from sklearn.svm import LinearSVR

svm_reg = LinearSVR(epsilon=1.5, random_state=42)
svm_reg.fit(X, y)

svm_reg1 = LinearSVR(epsilon=1.5, random_state=42)
svm_reg2 = LinearSVR(epsilon=0.5, random_state=42)
svm_reg1.fit(X, y)
svm_reg2.fit(X, y)

def find_support_vectors(svm_reg, X, y):
    y_pred = svm_reg.predict(X)
    off_margin = (np.abs(y - y_pred) >= svm_reg.epsilon)
    return np.argwhere(off_margin)

svm_reg1.support_ = find_support_vectors(svm_reg1, X, y)
svm_reg2.support_ = find_support_vectors(svm_reg2, X, y)

eps_x1 = 1
eps_y_pred = svm_reg1.predict([[eps_x1]])

def plot_svm_regression(svm_reg, X, y, axes):
    x1s = np.linspace(axes[0], axes[1], 100).reshape(100, 1)
    y_pred = svm_reg.predict(x1s)
    plt.plot(x1s, y_pred, "k-", linewidth=2, label=r"$\hat{y}$")
    plt.plot(x1s, y_pred + svm_reg.epsilon, "k--")
    plt.plot(x1s, y_pred - svm_reg.epsilon, "k--")
    plt.scatter(X[svm_reg.support_], y[svm_reg.support_], s=180, facecolors='#FFAAAA')
    plt.plot(X, y, "bo")
    plt.xlabel(r"$x_1$", fontsize=18)
    plt.legend(loc="upper left", fontsize=18)
    plt.axis(axes)

plt.figure(figsize=(9, 4))
plt.subplot(121)
plot_svm_regression(svm_reg1, X, y, [0, 2, 3, 11])
plt.title(r"$\epsilon = {}$".format(svm_reg1.epsilon), fontsize=18)
plt.ylabel(r"$y$", fontsize=18, rotation=0)
#plt.plot([eps_x1, eps_x1], [eps_y_pred, eps_y_pred - svm_reg1.epsilon], "k-", linewidth=2)
plt.annotate(
        '', xy=(eps_x1, eps_y_pred), xycoords='data',
        xytext=(eps_x1, eps_y_pred - svm_reg1.epsilon),
        textcoords='data', arrowprops={'arrowstyle': '<->', 'linewidth': 1.5}
    )
plt.text(0.91, 5.6, r"$\epsilon$", fontsize=20)
plt.subplot(122)
plot_svm_regression(svm_reg2, X, y, [0, 2, 3, 11])
plt.title(r"$\epsilon = {}$".format(svm_reg2.epsilon), fontsize=18)
save_fig("svm_regression_plot")
plt.show()

np.random.seed(42)
m = 100
X = 2 * np.random.rand(m, 1) - 1
y = (0.2 + 0.1 * X + 0.5 * X**2 +...


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Sensitivity to feature scales
Can you write Python code for it?

Xs = np.array([[1, 50], [5, 20], [3, 80], [5, 60]]).astype(np.float64)
ys = np.array([0, 0, 1, 1])
svm_clf = SVC(kernel="linear", C=100)
svm_clf.fit(Xs, ys)

plt.figure(figsize=(12,3.2))
plt.subplot(121)
plt.plot(Xs[:, 0][ys==1], Xs[:, 1][ys==1], "bo")
plt.plot(Xs[:, 0][ys==0], Xs[:, 1][ys==0], "ms")
plot_svc_decision_boundary(svm_clf, 0, 6)
plt.xlabel("$x_0$", fontsize=20)
plt.ylabel("$x_1$  ", fontsize=20, rotation=0)
plt.title("Unscaled", fontsize=16)
plt.axis([0, 6, 0, 90])

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_scaled = scaler.fit_transform(Xs)
svm_clf.fit(X_scaled, ys)

plt.subplot(122)
plt.plot(X_scaled[:, 0][ys==1], X_scaled[:, 1][ys==1], "bo")
plt.plot(X_scaled[:, 0][ys==0], X_scaled[:, 1][ys==0], "ms")
plot_svc_decision_boundary(svm_clf, -2, 2)
plt.xlabel("$x_0$", fontsize=20)
plt.title("Scaled", fontsize=16)
plt.axis([-2, 2, -2, 2])

save_fig("sensitivity_to_feature_scales_plot")


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Sensitivity to outliers
Can you write Python code for it?

X_outliers = np.array([[3.4, 1.3], [3.2, 0.8]])
y_outliers = np.array([0, 0])
Xo1 = np.concatenate([X, X_outliers[:1]], axis=0)
yo1 = np.concatenate([y, y_outliers[:1]], axis=0)
Xo2 = np.concatenate([X, X_outliers[1:]], axis=0)
yo2 = np.concatenate([y, y_outliers[1:]], axis=0)

svm_clf2 = SVC(kernel="linear", C=10**9)
svm_clf2.fit(Xo2, yo2)

plt.figure(figsize=(12,2.7))

plt.subplot(121)
plt.plot(Xo1[:, 0][yo1==1], Xo1[:, 1][yo1==1], "bs")
plt.plot(Xo1[:, 0][yo1==0], Xo1[:, 1][yo1==0], "yo")
plt.text(0.3, 1.0, "Impossible!", fontsize=24, color="red")
plt.xlabel("Petal length", fontsize=14)
plt.ylabel("Petal width", fontsize=14)
plt.annotate("Outlier",
             xy=(X_outliers[0][0], X_outliers[0][1]),
             xytext=(2.5, 1.7),
             ha="center",
             arrowprops=dict(facecolor='black', shrink=0.1),
             fontsize=16,
            )
plt.axis([0, 5.5, 0, 2])

plt.subplot(122)
plt.plot(Xo2[:, 0][yo2==1], Xo2[:, 1][yo2==1], "bs")
plt.plot(Xo2[:, 0][yo2==0], Xo2[:, 1][yo2==0], "yo")
plot_svc_decision_boundary(svm_clf2, 0, 5.5)
plt.xlabel("Petal length", fontsize=14)
plt.annotate("Outlier",
             xy=(X_outliers[1][0], X_outliers[1][1]),
             xytext=(3.2, 0.08),
             ha="center",
             arrowprops=dict(facecolor='black', shrink=0.1),
             fontsize=16,
            )
plt.axis([0, 5.5, 0, 2])

save_fig("sensitivity_to_outliers_plot")
plt.show()


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Small weight vector results in a large margin
Can you write Python code for it?

def plot_2D_decision_function(w, b, ylabel=True, x1_lim=[-3, 3]):
    x1 = np.linspace(x1_lim[0], x1_lim[1], 200)
    y = w * x1 + b
    m = 1 / w

    plt.plot(x1, y)
    plt.plot(x1_lim, [1, 1], "k:")
    plt.plot(x1_lim, [-1, -1], "k:")
    plt.axhline(y=0, color='k')
    plt.axvline(x=0, color='k')
    plt.plot([m, m], [0, 1], "k--")
    plt.plot([-m, -m], [0, -1], "k--")
    plt.plot([-m, m], [0, 0], "k-o", linewidth=3)
    plt.axis(x1_lim + [-2, 2])
    plt.xlabel(r"$x_1$", fontsize=16)
    if ylabel:
        plt.ylabel(r"$w_1 x_1$  ", rotation=0, fontsize=16)
    plt.title(r"$w_1 = {}$".format(w), fontsize=16)

plt.figure(figsize=(12, 3.2))
plt.subplot(121)
plot_2D_decision_function(1, 0)
plt.subplot(122)
plot_2D_decision_function(0.5, 0, ylabel=False)
save_fig("small_w_large_margin_plot")
plt.show()

from sklearn.svm import SVC
from sklearn import datasets

iris = datasets.load_iris()
X = iris["data"][:, (2, 3)] # petal length, petal width
y = (iris["target"] == 2).astype(np.float64) # Iris-Virginica

svm_clf = SVC(kernel="linear", C=1)
svm_clf.fit(X, y)
svm_clf.predict([[5.3, 1.3]])


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Split it into a training set and a test set:
Can you write Python code for it?

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
That's much better (we cut the error rate in two), but still not great at all for MNIST. If we want to use an SVM, we will have to use a kernel. Let's try an SVC with an RBF kernel (the default).
Warning: if you are using Scikit-Learn ≤ 0.19, the SVC class will use the One-vs-One (OvO) strategy by default, so you must explicitly set decision_function_shape="ovr" if you want to use the OvR strategy instead (OvR is the default since 0.19).
Can you write Python code for it?

svm_clf = SVC(decision_function_shape="ovr", gamma="auto")
svm_clf.fit(X_train_scaled[:10000], y_train[:10000])

y_pred = svm_clf.predict(X_train_scaled)
accuracy_score(y_train, y_pred)


svm_clf = SVC(decision_function_shape="ovr")
svm_clf.fit(X_train_scaled[:10000], y_train[:10000])

y_pred = svm_clf.predict(X_train_scaled)
accuracy_score(y_train, y_pred)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
That's promising, we get better performance even though we trained the model on 6 times less data. Let's tune the hyperparameters by doing a randomized search with cross validation. We will do this on a small dataset just to speed up the process:
Can you write Python code for it?

from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import reciprocal, uniform

param_distributions = {"gamma": reciprocal(0.001, 0.1), "C": uniform(1, 10)}
rnd_search_cv = RandomizedSearchCV(svm_clf, param_distributions, n_iter=10, verbose=2, cv=3)
rnd_search_cv.fit(X_train_scaled[:1000], y_train[:1000])

rnd_search_cv.best_estimator_

rnd_search_cv.best_score_


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
This looks pretty low but remember we only trained the model on 1,000 instances. Let's retrain the best estimator on the whole training set (run this at night, it will take hours):
Can you write Python code for it?

rnd_search_cv.best_estimator_.fit(X_train_scaled, y_train)

y_pred = rnd_search_cv.best_estimator_.predict(X_train_scaled)
accuracy_score(y_train, y_pred)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Under the hood
Can you write Python code for it?

iris = datasets.load_iris()
X = iris["data"][:, (2, 3)]  # petal length, petal width
y = (iris["target"] == 2).astype(np.float64)  # Iris-Virginica

from mpl_toolkits.mplot3d import Axes3D

def plot_3D_decision_function(ax, w, b, x1_lim=[4, 6], x2_lim=[0.8, 2.8]):
    x1_in_bounds = (X[:, 0] > x1_lim[0]) & (X[:, 0] < x1_lim[1])
    X_crop = X[x1_in_bounds]
    y_crop = y[x1_in_bounds]
    x1s = np.linspace(x1_lim[0], x1_lim[1], 20)
    x2s = np.linspace(x2_lim[0], x2_lim[1], 20)
    x1, x2 = np.meshgrid(x1s, x2s)
    xs = np.c_[x1.ravel(), x2.ravel()]
    df = (xs.dot(w) + b).reshape(x1.shape)
    m = 1 / np.linalg.norm(w)
    boundary_x2s = -x1s*(w[0]/w[1])-b/w[1]
    margin_x2s_1 = -x1s*(w[0]/w[1])-(b-1)/w[1]
    margin_x2s_2 = -x1s*(w[0]/w[1])-(b+1)/w[1]
    ax.plot_surface(x1s, x2, 0, color="b", alpha=0.2, cstride=100, rstride=100)
    ax.plot(x1s, boundary_x2s, 0, "k-", linewidth=2, label=r"$h=0$")
    ax.plot(x1s, margin_x2s_1, 0, "k--", linewidth=2, label=r"$h=\pm 1$")
    ax.plot(x1s, margin_x2s_2, 0, "k--", linewidth=2)
    ax.plot(X_crop[:, 0][y_crop==1], X_crop[:, 1][y_crop==1], 0, "g^")
    ax.plot_wireframe(x1, x2, df, alpha=0.3, color="k")
    ax.plot(X_crop[:, 0][y_crop==0], X_crop[:, 1][y_crop==0], 0, "bs")
    ax.axis(x1_lim + x2_lim)
    ax.text(4.5, 2.5, 3.8, "Decision function $h$", fontsize=15)
    ax.set_xlabel(r"Petal length", fontsize=15)
    ax.set_ylabel(r"Petal width", fontsize=15)
    ax.set_zlabel(r"$h = \mathbf{w}^t \cdot \mathbf{x} + b$", fontsize=18)
    ax.legend(loc="upper left", fontsize=16)

fig = plt.figure(figsize=(11, 6))
ax1 = fig.add_subplot(111, projection='3d')
plot_3D_decision_function(ax1, w=svm_clf2.coef_[0], b=svm_clf2.intercept_[0])

save_fig("iris_3D_plot")
plt.show()


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Warning: the default value of gamma will change from 'auto' to 'scale' in version 0.22 to better account for unscaled features. To preserve the same results as in the book, we explicitly set it to 'auto', but you should probably just use the default in your own code.
Can you write Python code for it?

from sklearn.svm import SVR

svm_poly_reg = SVR(kernel="poly", degree=2, C=100, epsilon=0.1, gamma="auto")
svm_poly_reg.fit(X, y)

from sklearn.svm import SVR

svm_poly_reg1 = SVR(kernel="poly", degree=2, C=100, epsilon=0.1, gamma="auto")
svm_poly_reg2 = SVR(kernel="poly", degree=2, C=0.01, epsilon=0.1, gamma="auto")
svm_poly_reg1.fit(X, y)
svm_poly_reg2.fit(X, y)

plt.figure(figsize=(9, 4))
plt.subplot(121)
plot_svm_regression(svm_poly_reg1, X, y, [-1, 1, 0, 1])
plt.title(r"$degree={}, C={}, \epsilon = {}$".format(svm_poly_reg1.degree, svm_poly_reg1.C, svm_poly_reg1.epsilon), fontsize=18)
plt.ylabel(r"$y$", fontsize=18, rotation=0)
plt.subplot(122)
plot_svm_regression(svm_poly_reg2, X, y, [-1, 1, 0, 1])
plt.title(r"$degree={}, C={}, \epsilon = {}$".format(svm_poly_reg2.degree, svm_poly_reg2.C, svm_poly_reg2.epsilon), fontsize=18)
save_fig("svm_with_polynomial_kernel_plot")
plt.show()


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Wow, 82% accuracy on MNIST is a really bad performance. This linear model is certainly too simple for MNIST, but perhaps we just needed to scale the data first:
Can you write Python code for it?

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train.astype(np.float32))
X_test_scaled = scaler.transform(X_test.astype(np.float32))

lin_clf = LinearSVC(random_state=42)
lin_clf.fit(X_train_scaled, y_train)

y_pred = lin_clf.predict(X_train_scaled)
accuracy_score(y_train, y_pred)


I am working on the file "05_support_vector_machines.ipynb".
The first task is:
Wow, 86% accuracy on MNIST is a really bad performance. This linear model is certainly too simple for MNIST, but perhaps we just needed to scale the data first:
Can you write Python code for it?


I am working on the file "05_weight-initialization/weight_initialization.ipynb".
The first task is:
Again, let's compare the previous results with the previous distribution.
Can you write Python code for it?

trunc_normal_01_weights = [
    tf.Variable(tf.truncated_normal(layer_1_weight_shape, stddev=0.1)),
    tf.Variable(tf.truncated_normal(layer_2_weight_shape, stddev=0.1)),
    tf.Variable(tf.truncated_normal(layer_3_weight_shape, stddev=0.1))
]

helper.compare_init_weights(
    mnist,
    'Normal vs Truncated Normal',
    [
        (normal_01_weights, 'Normal'),
        (trunc_normal_01_weights, 'Truncated Normal')])


I am working on the file "05_weight-initialization/weight_initialization.ipynb".
The first task is:
As you can see the accuracy is close to guessing for both zeros and ones, around 10%.
The neural network is having a hard time determining which weights need to be changed, since the neurons have the same output for each layer.  To avoid neurons with the same output, let's use unique weights.  We can also randomly select these weights to avoid being stuck in a local minimum for each run.
A good solution for getting these random weights is to sample from a uniform distribution.
Uniform Distribution
A [uniform distribution](https://en.wikipedia.org/wiki/Uniform_distribution_(continuous%29) has the equal probability of picking any number from a set of numbers. We'll be picking from a continous distribution, so the chance of picking the same number is low. We'll use TensorFlow's tf.random_uniform function to pick random numbers from a uniform distribution.

tf.random_uniform(shape, minval=0, maxval=None, dtype=tf.float32, seed=None, name=None)
Outputs random values from a uniform distribution.
The generated values follow a uniform distribution in the range [minval, maxval). The lower bound minval is included in the range, while the upper bound maxval is excluded.

shape: A 1-D integer Tensor or Python array. The shape of the output tensor.
minval: A 0-D Tensor or Python value of type dtype. The lower bound on the range of random values to generate. Defaults to 0.
maxval: A 0-D Tensor or Python value of type dtype. The upper bound on the range of random values to generate. Defaults to 1 if dtype is floating point.
dtype: The type of the output: float32, float64, int32, or int64.
seed: A Python integer. Used to create a random seed for the distribution. See tf.set_random_seed for behavior.
name: A name for the operation (optional).

We can visualize the uniform distribution by using a histogram. Let's map the values from tf.random_uniform([1000], -3, 3) to a histogram using the helper.hist_dist function. This will be 1000 random float values from -3 to 3, excluding the value 3.
Can you write Python code for it?

helper.hist_dist('Random Uniform (minval=-3, maxval=3)', tf.random_uniform([10000], -3, 3))


I am working on the file "05_weight-initialization/weight_initialization.ipynb".
The first task is:
Let's compare the normal distribution against the previous uniform distribution.
Can you write Python code for it?

normal_01_weights = [
    tf.Variable(tf.random_normal(layer_1_weight_shape, stddev=0.1)),
    tf.Variable(tf.random_normal(layer_2_weight_shape, stddev=0.1)),
    tf.Variable(tf.random_normal(layer_3_weight_shape, stddev=0.1))
]

helper.compare_init_weights(
    mnist,
    'Uniform [-0.1, 0.1) vs Normal stddev 0.1',
    [
        (uniform_neg01to01_weights, 'Uniform [-0.1, 0.1)'),
        (normal_01_weights, 'Normal stddev 0.1')])


I am working on the file "05_weight-initialization/weight_initialization.ipynb".
The first task is:
Neural Network
<img style="float: left" src="images/neural_network.png"/>
For the neural network, we'll test on a 3 layer neural network with ReLU activations and an Adam optimizer.  The lessons you learn apply to other neural networks, including different activations and optimizers.
Can you write Python code for it?

# Save the shapes of weights for each layer
layer_1_weight_shape = (mnist.train.images.shape[1], 256)
layer_2_weight_shape = (256, 128)
layer_3_weight_shape = (128, mnist.train.labels.shape[1])


I am working on the file "05_weight-initialization/weight_initialization.ipynb".
The first task is:
The histogram used 500 buckets for the 1000 values.  Since the chance for any single bucket is the same, there should be around 2 values for each bucket. That's exactly what we see with the histogram.  Some buckets have more and some have less, but they trend around 2.
Now that you understand the tf.random_uniform function, let's apply it to some initial weights.
Baseline
Let's see how well the neural network trains using the default values for tf.random_uniform, where minval=0.0 and maxval=1.0.
Can you write Python code for it?

# Default for tf.random_uniform is minval=0 and maxval=1
basline_weights = [
    tf.Variable(tf.random_uniform(layer_1_weight_shape)),
    tf.Variable(tf.random_uniform(layer_2_weight_shape)),
    tf.Variable(tf.random_uniform(layer_3_weight_shape))
]

helper.compare_init_weights(
    mnist,
    'Baseline',
    [(basline_weights, 'tf.random_uniform [0, 1)')])


I am working on the file "05_weight-initialization/weight_initialization.ipynb".
The first task is:
The normal distribution gave a slight increasse in accuracy and loss.  Let's move closer to 0 and drop picked numbers that are x number of standard deviations away.  This distribution is called Truncated Normal Distribution.
Truncated Normal Distribution

tf.truncated_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)
Outputs random values from a truncated normal distribution.
The generated values follow a normal distribution with specified mean and standard deviation, except that values whose magnitude is more than 2 standard deviations from the mean are dropped and re-picked.

shape: A 1-D integer Tensor or Python array. The shape of the output tensor.
mean: A 0-D Tensor or Python value of type dtype. The mean of the truncated normal distribution.
stddev: A 0-D Tensor or Python value of type dtype. The standard deviation of the truncated normal distribution.
dtype: The type of the output.
seed: A Python integer. Used to create a random seed for the distribution. See tf.set_random_seed for behavior.
name: A name for the operation (optional).
Can you write Python code for it?

helper.hist_dist('Truncated Normal (mean=0.0, stddev=1.0)', tf.truncated_normal([1000]))


I am working on the file "05_weight-initialization/weight_initialization.ipynb".
The first task is:
There's no difference between the two, but that's because the neural network we're using is too small. A larger neural network will pick more points on the normal distribution, increasing the likelihood it's choices are larger than 2 standard deviations.
We've come a long way from the first set of weights we tested. Let's see the difference between the weights we used then and now.
Can you write Python code for it?

helper.compare_init_weights(
    mnist,
    'Baseline vs Truncated Normal',
    [
        (basline_weights, 'Baseline'),
        (trunc_normal_01_weights, 'Truncated Normal')])


I am working on the file "05a--a-priori-wrap-up.ipynb".
The first task is:
Exercise: The A-Priori algorithm applied to email. Your task is to implement the a-priori algorithm to generate a list of commonly co-occurring correspondents.
You may make the following simplifying assumptions, which may or may not be valid depending on what the true analysis end-goal is.
* You need only examine the 'From:' and 'To:' fields of an email message. Ignore all other fields.
* You should only "count" an email address if both the 'From:' and 'To:' fields are set. Otherwise, you cannot tell from whom the message was sent or who is the recipient, and may therefore ignore the interaction.
* Consider pairs that consist of a sender and a recipient. In other words, do not match multiple recipients of a single message as a "pair."
* Ignore the direction of the exchange. That is, regard bob@gatech.edu sending to kate@aol.com as the same pair as kate@aol.com sending to bob@gatech.edu.

For Jeffrey Skilling's maildir and a threshold of 65 or more co-occurrences, our solution finds 10 frequently corresponding pairs. For the full data set, it finds 140 pairs.
Can you write Python code for it?

# Specify maildir location; you may need to update these paths.
MAILDIR = 'enron-maildir-subset/skilling-j' # Skilling's mail only
#MAILDIR = 'enron-maildir-subset' # Full subset

# Specify the minimum number of occurrences to be considered "frequent"
THRESHOLD = 65

# === FILL-IN YOUR IMPLEMENTATION AND TEST CODE BELOW ==
pass


I am working on the file "05a--a-priori-wrap-up.ipynb".
The first task is:
Putting it all together: The A-Priori algorithm
Using all of the preceding material, implement the A-Priori algorithm from the previous Lab 3 notebook to detect frequent email correspondents.
But first, here's a little bit of helper code from last time, which you'll find useful.
Can you write Python code for it?

# This code box contains some helper routines needed to implement
# the A-Priori algorithm.

import re
import email.parser
import os

EMAIL_PATTERN = re.compile (r'[\w+.]+@[\w.]+')

def messages (maildir_root):
    """
    (Generator) Given a mailbox directory name, yields an
    email object for each message therein.
    """
    for base, dirs, files in os.walk (maildir_root):
        for filename in files:
            filepath = os.path.join (base, filename)
            email_file = open (filepath)
            msg = email.parser.Parser ().parse (email_file)
            email_file.close ()
            if len (msg) > 0: # Patch for non-email files?
                yield msg


I am working on the file "05a-strings.ipynb".
The first task is:
A generator expression that creates a list is often called a list comprehension.
Note: The same construct can be used with sets (set comprehension) and dictionaries (dictionary comprehension). For example:
Can you write Python code for it?

print("List:", [x*x for x in range(-n, n) if is_prime(abs(x))])
print("Set: ", {x*x for x in range(-n, n) if is_prime(abs(x))})
print("Dict:", {x+2*n: x*x for x in range(-n, n) if is_prime(abs(x))})


I am working on the file "05a-strings.ipynb".
The first task is:
As we can see:
* A range and a list are not the same (they print differently).
* They have different types.
* However, they "contain" the same elements.
So, while acting the same way that lists do in a for loop, range actually behaves like a generator, which is a function that returns an iterator, which is what rng is.
Note: Strictly speaking, range a bit more than an ordinary generator, with some additional properties and functions that normal generators don't have, but for the sake of explaining how the generators work, we'll disregard those extras and use range as an example that we're quite familiar with.
The main difference between a list and an iterator is that a list actually contains all the data and allows us to access each of its elements.
An iterator, on the other hand, may contain the elements, but it almost never will. It can somehow get the first element, keep track of what the current element is, and "jump" to the next one. So, basically, it can provide everything a for loop needs, without any restrictions on where it gets its data.
For example, range(3, 7) will return an iterator that holds three values:
* starting value: 3,
* increment: 1 (default, since we didn't provide the third argument),
* end value: 7 (exclusive).
So, when we execute
python
for x in range(3, 7):
    print(x)
the following will happen:
1. for will "ask" range for its first value and obtain 3. It will then assign that 3 to x and execute its body (print(x)).
2. In the next step, for will "ask" range for its next value. Now, range remembers that its current value was 3, so it will compute the next one, and return 4, which will get assigned to x and the loop's body will once again be executed.
3. In the next step, for will "ask" range for its next value, and get 5.
4. In the next step, for will "ask" range for its next value, and get 6.
5. In the next step, for will "ask" range for its next value. However, range will see that the next value, 7, is too big, and it will gently explain for that it has no more values. for will...
Can you write Python code for it?

def is_prime(n):
    """Returns true if an integer `n` is a prime."""
    if n < 2:
        return False
    for p in range(2,n):
        if n % p == 0:
            return False
    return True

def primes(n):
    """Returns a generator for the prime factors of `n`."""
    n = abs(n)
    for p in range(2,n+1):
        # if n is divisable by p and p is prime
        if n % p == 0 and is_prime(p):
            yield p

n = int(input("n = "))
cnt = 0  # Counter
for prime in primes(n):
    cnt += 1
    print("Prime factor #%d of %d: %d" % (cnt, n, prime))
print("Number %d has %d distinct prime factors." % (n, cnt))


I am working on the file "05a-strings.ipynb".
The first task is:
As we said before, strings are immutable. This means they cannot be changed:
Can you write Python code for it?


I am working on the file "05a-strings.ipynb".
The first task is:
Examples
We are now ready to make some of the old pieces of code simpler.
Can you write Python code for it?

n = int(input("n = "))
print("a) Sum of digits of %d:" % n, sum(int(x) for x in str(n)))
print("b) Prime digits of %d:" % n, [int(x) for x in str(n) if x in "2357"])
print("c) Sum of prime digits of %d:" % n, sum(int(x) for x in str(n) if x in "2357"))
print("d) Prime factors of %d:" % n, [x for x in range(2,abs(n)) if n % x == 0 and is_prime(x)])


I am working on the file "05a-strings.ipynb".
The first task is:
Generator expressions are meant for a single use. They know how to do "next", but there is no way to "reset" them (in the above example, the problem is that numbers is not reset). Depending on the exact generator that is used, there are various ways to deal with this, but this goes beyond the scope of this course.
More than one for clause
Generators and list comprehensions can have more than one for clause.
For example, here we create a list of all fully simplified fractions (memorized as strings) between zero and one (exclusively) with numerator and denominator between 1 and m.
Can you write Python code for it?

m = 5

# List comprehension
fracts = ["{}/{}".format(n, d)
          for n in range(1, m+1)
          for d in range(n, m+1)
          if all(n % k != 0 or d % k != 0 for k in range(2, n+1))
         ]
print(fracts)

# Equivalent loops
fracts = list()
for n in range(1, m+1):
    for d in range(n, m+1):
        if all(n % k != 0 or d % k != 0 for k in range(2, n+1)):
            fracts.append("{}/{}".format(n, d))
print(fracts)


I am working on the file "05a-strings.ipynb".
The first task is:
Generators are a fairly advanced concept and they will not be part of any examinations. However, the use of generators is encouraged. You can find more about them here.
Generator expressions
Generator expressions are expressions that return iterators. They behave in the same manner as generators, but their syntax is more compact and usually easier to understand as it is very similar to for loops.
Here is a simple example of a generator of the squares of the numbers from 0 to n-1:
Can you write Python code for it?

n = int(input("n = "))
gen = (x*x for x in range(n))
print("gen =", gen)
print("type(gen) =", type(gen))
print("list(gen) =", list(gen))


I am working on the file "05a-strings.ipynb".
The first task is:
However, if we change the sentence to imply a different order of the parameters, we're in trouble:
Can you write Python code for it?

args = [
    "Benedict Timothy Carlton Cumberbatch",
    "The University of Manchester",
    "he",
    "Drama"
]
print("{} studied {} at {}.".format(*args))


I am working on the file "05a-strings.ipynb".
The first task is:
In fact, we have been using this since we first encountered print:
Can you write Python code for it?


I am working on the file "05a-strings.ipynb".
The first task is:
In these two examples, we merely wrote less code, but there was no gain in memory consumption nor the processing speed.
However, let us check if any of the above squares has the form p+2 for some prime number p. In other words, we want to check if x*x-2 is a prime for some x which is a prime itself:
Can you write Python code for it?

numbers = (x*x for x in range(n) if is_prime(x))
if any(is_prime(number-2) for number in numbers):
    print("Yes")
else:
    print("No")


I am working on the file "05a-strings.ipynb".
The first task is:
More about C-like formatting can be read in the official reference.
Feel free to use it if you're already familiar with it. Otherwise, such a formatting is considered old and out of date, and it is advised to use the format function instead.
Generators and generator expressions
In this section we address some very useful, albeit Python-specific concepts:
* Iterator is an object representing a stream of data, i.e., capable of providing the first element, as well as traversing thorught all the elements from the beginning to the end.
Note: Iterators also exist in C++, but it is a different concept and they have a different purpose there.
* Generator is a function that returns an iterator.
* Generator expression is an expression that returns an iterator.
Iterators are a powerful thing to use, but quite complex in all their generality, and they go well beyond the scope of this course.
We shall cover generator expressions, as they are a very compact, efficient, and often used way to deal with list-like data, and then we shall briefly see how generators work, as a bit more complex but quite powerful mechanism.
We have seen the function range before and we said that it

pretends to return a list of numbers (it returns something more complex, we can consider it a list for now)

Let us see what exactly does range return:
Can you write Python code for it?

rng = range(3,7)
lst = list(rng)
print("rng =", rng)
print("type(rng) =", type(rng))
print("Elements of rng:")
for x in rng:
    print("  %d" % x)
print("\nlst =", lst)
print("type(lst) =", type(lst))
print("Elements of lst:")
for x in lst:
    print("  %d" % x)


I am working on the file "05a-strings.ipynb".
The first task is:
Note: This is a highly inefficient algorithm to find prime factors of a number. We shall see an improvement in several weeks. We use this one here because it closely follows the mathematical definition and, as such, it is easy to understand it.
So, how does the above work? Let us assume that $n = 84 = 2^2 \cdot 3 \cdot 7$.
When the for-loop is first encountered, prime(84) is called. It executes as any other function would, until it reaches yield p, which will first happen for p = 2. At this moment, the function returns p (i.e., 2) and stops executing, but it is not terminated, as it would have been if return was in place of yield.
So, the for-loop has now obtained its first value, 2, and it assigns it to prime. The body of the for-loop is now executed, i.e., cnt is increased from zero to one, and the message

Prime factor #1 of 84: 2

is printed.
After the body is executed, we return to the head of the for-loop. Recall that the function was not terminated, so it continues where it stopped when a yield statement was last invoked. The next prime factor that the for-loop in primes function finds is p = 3. Again yield p is invoked, the main for-loop gets its second value, prime = 3 is assigned, and the message

Prime factor #2 of 84: 3

is printed.
Back to the header of the for-loop, we reenter the still uninterrupted function primes, and continue after its last executed yield (generally, a generator can have many yield statements, not just one as in our example). The next p it will find is 7, which is again returned to the main for-loop and

Prime factor #3 of 84: 7

is printed.
We return to the header of the for-loop once again, reentering primes. However, this time, no new value of p will be found, and the function's own for-loop will end. Since there is no code after it, the primes function will finally come to an end (i.e., it will stop and not just "pause"). Now, the main for-loop is notified that there are no further values to be assigned to the prime variable, so it too quits.
The last command in our...
Can you write Python code for it?

n = int(input("n = "))
print("The list of the prime factors of %d:" % n, list(primes(n)))


I am working on the file "05a-strings.ipynb".
The first task is:
Notice that the following wouldn't work properly:
Can you write Python code for it?

n = 17

# Try one:
numbers = (x*x for x in range(n) if is_prime(x))
print("Yes:", [number for number in numbers if is_prime(number-2)])
print("No: ", [number for number in numbers if not is_prime(number-2)])

# Try two:
numbers = (x*x for x in range(n) if is_prime(x))
print("No: ", [number for number in numbers if not is_prime(number-2)])
print("Yes:", [number for number in numbers if is_prime(number-2)])


I am working on the file "05a-strings.ipynb".
The first task is:
Of course, if the string contains something other than an integer or a "real" number, an error occurs. For example:
Can you write Python code for it?


I am working on the file "05a-strings.ipynb".
The first task is:
Of course, there are more efficient ways to check if a fraction is fully simplified (hint: Euclidean algorithm), but we shall cover that later.
As always, all of these for statements could've looped over iterables other than iterators created by range. We could use lists, tuples, sets, etc.
For example, to limit the numerators to the set ${1, 3, 5, 7, 11}$ and the denominators to the set ${17, 19, 23}$ we can do this:
Can you write Python code for it?

m = 5

# List comprehension
fracts = ["%d/%d" % (n, d)
          for n in [1,3,5,7,11]
          for d in (17,19,23)
          if all(n % k != 0 or d % k != 0 for k in range(2, n+1))
         ]
print(fracts)

# Equivalent loops
fracts = list()
for n in [1,3,5,7,11]:
    for d in (17,19,23):
        if all(n % k != 0 or d % k != 0 for k in range(2, n+1)):
            fracts.append("%d/%d" % (n, d))
print(fracts)


I am working on the file "05a-strings.ipynb".
The first task is:
Python has a truly rich support for strings. Some of the available operations are:
Can you write Python code for it?

x = "this is a string with several words"
print("x:                                ", x)
print("Length of x:                      ", len(x))
print("Capitalized x:                    ", x.capitalize())
print("Does x end with \"ords\"?           ", x.endswith("ords"))
print("Does x end with \"orDs\"?           ", x.endswith("orDs"))
print("Does x end with \"word\"?           ", x.endswith("word"))
print("x with \"a string\" replaced by \"x\":", x.replace("a string", "x"))
print("Does x contain a \"ring\"?          ", "ring" in x)
print("The index of the first character of the first \"ring\" in x:", x.find("ring"))
print("Does x have any cased characters and are they all lowercase?", x.islower())
print("\nNone of these changed our string: ", x)


I am working on the file "05a-strings.ipynb".
The first task is:
So, \\ inside a string means a single backslash (\), '\n' means new-line character, \" and \' mean quotation marks (instead of the end of the string), etc. This is called string escaping and the backslash is called an escape character.
More about string literals can be read in the official reference.
If we need to put many backslashes in a string (required by some applications, for example, regular expressions), we can also define so called raw strings, by prepending a string literal with r:
Can you write Python code for it?

print(r"Double backslash: '\\'")
print(r"Quotes prepended with backslashes: \"...\"")
print(r"Quotes: '...'")
print(r'Quotes: "..."')
print(r'Quotes prepended with backslashes: \'...\'')
print(r"No new line, just a bunch of backslashes: \"new\nline\"")


I am working on the file "05a-strings.ipynb".
The first task is:
There is an easy fix for this:
Can you write Python code for it?

args = [
    "Benedict Timothy Carlton Cumberbatch",
    "The University of Manchester",
    "he",
    "Drama"
]
print("{0} studied {3} at {1}.".format(*args))


I am working on the file "05a-strings.ipynb".
The first task is:
There is much more that format can do. For the complete reference, read the official documentation.
C-like formatting
C-like formatting is somewhat less powerfull, but easier to invoke, which makes it ideal for simple formattings (for example, of numbers). The basic syntaxt is:
python3
format_string % value
or, if there is more than one value to include,
python3
format_string % a_tuple_of_values
For example:
Can you write Python code for it?

from math import pi
print("Pi printed without formatting:", pi)
print("Pi to the 5th decimal: %.5f" % pi)
print("Pi in 17 characters, to the 5th decimal, prepended with spaces: %17.5f" % pi)
print("Pi in 17 characters, to the 5th decimal, prepended with zeros: %017.5f" % pi)
print("Pi in 17 characters, to the 5th decimal, with sign: %+17.5f" % pi)
print("Minus Pi in 11 characters, to the 5th decimal, with sign: %+11.5f" % pi)
x = 17
y = 19
print("x = %d, y = %d, x + y = %d + %d = %d" % (x, y, x, y, x+y))


I am working on the file "05a-strings.ipynb".
The first task is:
To just convert a string to the list of its characters, we use the function list() which we have seen while working with iterables:
Can you write Python code for it?


I am working on the file "05a-strings.ipynb".
The first task is:
We can also align values on the screen:
Can you write Python code for it?

x = """Gaudeamus igitur
Iuvenes dum sumus.
Post iucundam iuventutem
Post molestam senectutem
Nos habebit humus."""

print("The original string:")
print(x)

print("\n|" + ("1234567890" * 4) + "|")

print("|{:<40}|".format("Centered in 40 characters:"))
for line in x.splitlines():
    print("|{:^40}|".format(line))

print("|{:<40}|".format("Aligned to the right in 40 characters:"))
for line in x.splitlines():
    print("|{:>40}|".format(line))


I am working on the file "05a-strings.ipynb".
The first task is:
We can also do the opposite: split a string into a list of substrings, by some separator. For example,
Can you write Python code for it?

a_string = "17;19;23"
a_list = a_string.split(";")
print(a_list)


I am working on the file "05a-strings.ipynb".
The first task is:
We have also seen how to load a string:
Can you write Python code for it?

x = input("Type a string: ")
print("Here is your string:", x)


I am working on the file "05a-strings.ipynb".
The first task is:
is better written as:
Can you write Python code for it?

for x in range(n):
    if is_prime(x):
        print(x*x)


I am working on the file "05a-strings.ipynb".
The first task is:
or, using the unpacking operator ** for dictionaries:
Can you write Python code for it?

args = {
    "name": "Benedict Timothy Carlton Cumberbatch",
    "uni": "The University of Manchester",
    "heshe": "he",
    "studies": "Drama"
}
print("Student {name} was enrolled in {uni} where {heshe} studied {studies}.".format(**args))
print("{name} studied {studies} at {uni}.".format(**args))


I am working on the file "05a_integral.ipynb".
The first task is:
Krivulji omejujeta dva lika. Meje za integracijo izračunamo tako, da poiščemo presečišča.
Can you write Python code for it?

import sympy as sym
sym.init_printing()
x = sym.Symbol('x')
enacba = sym.Eq(f(x),g(x))
enacba

sym.solve(enacba,x)


I am working on the file "05a_integral.ipynb".
The first task is:
Opazimo, da so rešitve kompleksne. Čudno! Ampak imaginarni del je zelo majhen. Očitno gre za zaokrožitveno napako, saj je na grafu lepo vindo, da so rešitve realne. Vzamemo le realni del.
Can you write Python code for it?

resitve = [sym.re(x) for x in resitve]
resitve.sort()
resitve

for i in range(len(resitve)):
    xi = resitve[i]
    ax.plot(xi,f(xi),'ko')
    ax.annotate('$(x_%d,f(x_%d))$'%(i,i),xy = (xi,f(xi)),xytext=(xi,f(xi)-3))
fig


I am working on the file "05a_integral.ipynb".
The first task is:
Ploščina
Če uporabimo formulo za ploščino, dobimo integral z absolutno vrednostjo, ki ga razdelimo na dva dela
$$\int_{x_0}^{x_2}\left|f(x)-g(x)\right|dx = \int_{x_0}^{x_1}\left(g(x)-f(x)\right)dx+\int_{x_1}^{x_2}\left(f(x)-g(x)\right)dx$$
Can you write Python code for it?

x = sym.Symbol('x')
P = sym.integrate(g(x)-f(x),(x,resitve[0],resitve[1])) + sym.integrate(f(x)-g(x),(x,resitve[1],resitve[2]))
print("Skupna ploščina obeh likov je približno enaka %f" % P)


I am working on the file "05b--www.ipynb".
The first task is:
A more advanced example: Unpacking a zip file
In Labs 4 and 5-A, you worked with an email repository that you had to manually download and unpack.
As it happens, you can do that from within your Python program as well!
Can you write Python code for it?

import zipfile
import StringIO

URL_ZIPPED = "http://cse6040.gatech.edu/fa15/skilling-j.zip"

r = requests.get (URL_ZIPPED)
zipped_maildir = zipfile.ZipFile (StringIO.StringIO (r.content), 'r')

print ("==> Downloaded: %s" % URL_ZIPPED)


I am working on the file "05b--www.ipynb".
The first task is:
Example: Yelp! search. Here's a more complex example, motivated by a screenshot from Yelp! after executing a search for ramen in Atlanta. Take note of the URL.
<img src="yelp-screenshot.png">
The URL encodes what is known as an HTTP "get" method (or request). It basically means a URL with two parts: a command followed by one or more arguments. In this case, the command is everything up to and including the word search; the arguments are the rest, where individual arguments are separated by the & or #.

"HTTP" stands for "HyperText Transport Protocol," which is a standardized set of communication protocols that allow web clients, like your web browser or your Python program, to communicate with web servers.

In this next example, let's see how to build a "get request" with the requests module. It's pretty easy!
Can you write Python code for it?

url_command = 'http://yelp.com/search'
url_args = {'find_desc': "ramen"
            , 'find_loc': "atlanta, ga"
            , 'ns': 1
            , 'start': 0}
response = requests.get (url_command, params=url_args)

print ("==> Downloading from: '%s'" % response.url) # confirm URL
print ("\n==> Excerpt from this URL:\n\n%s\n" % response.text[0:100])


I am working on the file "05b--www.ipynb".
The first task is:
Exercise. Try modifying and extending the above code to retrieve the 13th entry in the search results.
Can you write Python code for it?

# (Enter your code for the preceding exercise in this code box)


I am working on the file "05b--www.ipynb".
The first task is:
Interacting with a web API
We hope the preceding exercise was painful: it is rough downloading raw HTML and trying to extract information from it!
Luckily, many websites provide an application programming interface (API) for querying their data or otherwise accessing their services from your programs. For instance, Twitter provides a web API for gathering tweets, Flickr provides one for gathering image data, and Github for accessing information about repository histories.
These kinds of web APIs are much easier to use than, for instance, the preceding technique which scrapes raw web pages and then has to parse the resulting HTML. Moreover, there are more scalable in the sense that the web servers can transmit structured data in a less verbose form than raw HTML. In Homework 1, you will apply the techniques below, as well as others, to write some Python scripts to interact with the Yelp! web API.
As a starting example, here is some code to look at all the activity on Github related to our course's IPython notebook repository.

Inspect this code and try running it. See if you can figure out what it does. Note that it is split into two parts, so you can try to digest one before moving on to the second.
Can you write Python code for it?

response = requests.get ('https://api.github.com/repos/rvuduc/cse6040-ipynbs/events')
urls = set ()
for event in response.json ():
    urls.add (event['actor']['url'])

# Blank cell, for you to debug or print program state, as needed

peeps = {}

for url in urls:
    response = requests.get (url)
    key = response.json ()['login']
    value = response.json ()['name']
    response.close ()
    peeps[key] = value

for key, value in peeps.items ():
    print ("%s: '%s'" % (key, str (value)))

# Blank cell, for you to debug or print program state, as needed


I am working on the file "05b--www.ipynb".
The first task is:
You can inspect the contents of this archive.
Can you write Python code for it?

# For the first COUNT items in the archive,
# print the original and compressed file sizes.

COUNT = 10
print ("Contents (first %d items):" % COUNT)
for zi in zipped_maildir.infolist ()[0:COUNT]:
    print ("  %s: %d -> %d bytes"
          % (zi.filename, zi.file_size, zi.compress_size))


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
DataFrame.from_items
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
DataFrame与Series进行运算时, 默认行为是DataFrame的column和Series的index对齐，也就是对Series进行行广播。例如:
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
Series vs dict
你可以把Series看做一个长度固定的字典，你可以通过index来获取和设置数值：
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
Series vs ndarray
大多数情况下，你都可以把Series看做一个ndarray来处理。甚至，大多数的NumPy函数都支持Series作参数。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
Series和ndarray之间的主要区别是，Series之间的操作会基于label对数据自动对齐。
所以，你不需要考虑两个参与计算的Series是否有完全相同的label。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
dot方法能够实现DataFrame矩阵乘法:
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
ndarray字典  列表字典
ndarrays必须具有相同的长度。如果传入index参数，它的长度也必须和ndarray长度相同。如果不传入index参数，DataFrame会默认创建index=range(n)，其中n是ndarray的长度。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
你可以插入ndarray，但其长度必须与DataFrame的index长度相匹配。
缺省情况下，列被插在末端。insert()函数可以允许插入到指定的列位置：
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
使用get方法，遇到缺失的label将返回None或指定值：
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
元组字典
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
列选择，添加，删除
你可以把DataFrame看做Series构成的字典，使用和字典同样的操作来增加列、删除列、对列重新赋值：
Can you write Python code for it?

df['one']

df['three'] = df['one'] * df['two']

df['flag'] = df['one'] > 2

df


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
向量操作和label对齐
进行数据分析时，你可以把Series看做向量进行整体操作。大多数接受ndarray作参数的NumPy方法也支持传入Series对象。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
如果label值不存在，将抛出异常：
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
如果两个label不一样的Series参与计算，存储结果的Series的index/label是它们俩的union(并集)。正是自带label对齐，才使得pandas区别于大多数数据分析库，优越性一览无余。
name属性
Series构造函数还有另一个参数：name，
Can you write Python code for it?

s = pd.Series(np.random.randn(5), name='something')

s

s.name


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
字典列表
Can you write Python code for it?

data2 = [{'a': 1, 'b': 2}, {'a': 5, 'b': 10, 'c': 20}]

pd.DataFrame(data2)

pd.DataFrame(data2, index=['first','second'])

pd.DataFrame(data2, columns=['a','b'])


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
布尔运算
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
我们先对数据结构进行简短的介绍, 然后再详细说明各个数据结构内置的方法。
Series
Series是一个一维带label的数组，元素可以是任何数据类型(整数、字符串、浮点数,Python对象等等)。和Python列表一样，Series元素的数据类型可以不同。Series是值可变的数据结构，你可以对它的值进行修改，但是Series的大小是不可以改变的。
Series的label就是index。
创建一个Series对象：
s = pd.Series(data, index=index)
这里，data可以是：

Python字典
ndarray
标量数值（比如5）

传入的index是label组成的列表。 因此，依据data的不同，我们可以分成以下几种情形：
ndarray
如果data是一个ndarray，index的长度必须和data一样。若没有指明index，则默认创建一个 [0, ..., len(data) - 1]的index。
Can you write Python code for it?

s = pd.Series(np.random.randn(5), index=['a', 'b', 'c', 'd', 'e'])
s

s.index

pd.Series(np.random.randn(5))


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
控制台显示
在控制台，非常大的DataFrames将被截断显示它们。你也可以使用info()查看详情。
baseball = pd.read_csv('data/baseball.csv')
print(baseball)
baseball.info()
然而,使用to_string，DataFrame将返回一个字符串表示的表格形式,虽然它并不总是适合控制台宽度:
print(baseball.iloc[-20:, :12].to_string())
pd.DataFrame(np.random.randn(3, 12))
你可以设置display.width值，来改变单行打印的数量。
pd.set_option('display.width', 40)
pd.DataFrame(np.random.randn(3, 12))
DataFrame列属性访问
如果DataFrame的列名是一个有效的Python变量名, 就可以像访问属性一样访问列数据:
Can you write Python code for it?

df = pd.DataFrame({'foo1':np.random.randn(5),'foo2':np.random.randn(5)})

df

df.foo1


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
注意s和s2是两个不同对象。
DataFrame
DataFrame是一个带label的二维数据结构。你可以把它看做一张SQL表，或是由Series构成的字典。它基本上是最常用的pandas数据结构。
和Series一样，DataFrame构造函数支持许多不同类型的输入：

字典，可以是普通字典，或者一维ndarray字典、列表字典、Series字典
二维numpy.ndarray
结构化或记录化的ndarray
字典列表
一个Series
DataFrame

DataFrame构造函数也支持传入index(行label)和columns(列label)参数。
Series字典或字典
结果的index是各个Series的index的union。如果存在任何嵌套类型的字典，将首先转换成Series。如果不传入columns，则默认把字典的键构成的有序列表看做columns。
Can you write Python code for it?

 d = {'one' : pd.Series([1., 2., 3.], index=['a', 'b', 'c']),
      'two' : pd.Series([1., 2., 3., 4.], index=['a', 'b', 'c', 'd'])}

df = pd.DataFrame(d)

df

pd.DataFrame(d, index=['d','b','a'])

pd.DataFrame(d, index=['d', 'b', 'a'], columns=['two', 'three'])


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
注意： NaN(not a number)是pandas中缺失值标记。
标量数值
如果data是一个标量值，必须提供index。为了匹配index的长度，该数值将被重复。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
注意： 从0.8.0版本开始，index的值可以重复。
Can you write Python code for it?

pd.Series(np.random.randn(3), index=['a', 'b', 'a'])


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
类似地,dot方法也能运用在Series上:
Can you write Python code for it?

s1 = pd.Series(np.arange(5,10))

s1.dot(s1)


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
行label和列label可分别通过访问index和columns获取：
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构-checkpoint.ipynb".
The first task is:
转置
转置, T属性(转置函数):
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
DataFrame.align()同样能接收Series对象，此时axis指的是DataFrame对象的索引或列。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
Head() Tail()
想要预览Series或DataFrame对象，可以使用head()和tail()方法。默认显示5行数据，你也可以自己设置显示的行数。
Can you write Python code for it?

long_series = pd.Series(np.random.randn(1000))

long_series.head()

long_series.tail(3)


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
Series同样有dtypes属性
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
align()也含有一个axis参数，指定仅对于某一坐标轴进行对齐。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
apply()方法当然支持接收其他参数了，比如下面的例子：
Can you write Python code for it?

def subtract_and_divide(x, sub, divide=1):
    return (x - sub)/divide

df.apply(subtract_and_divide, args=(5,),divide=3)


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
astype()方法
使用astype()显示的进行转型。默认会返回原对象的副本，即使数据类型不变。当然可以传递copy=False参数直接对原对象转型。
Can you write Python code for it?

df3

df3.dtypes

df3.astype('float32').dtypes


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
combine_first()
看一下例子：
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
get_dtype_counts()方法返回DataFrame中每一种数据类型的列数。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
iterrows()
iterrows()方法用于迭代DataFrame的每一行，返回的是索引和Series的迭代器，但要注意Series的dtype可能和原来每一行的dtype不同。
Can you write Python code for it?

for row_index, row in df.iterrows():
    print  row_index, row


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
itertuples()
itertuples()方法迭代DataFrame每一行，返回的是namedtuple。因为返回的不是Series，所以会保留DataFrame中值的dtype。
Can you write Python code for it?

for row in df.itertuples():
    print row


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
method参数要求索引必须是有序的：递增或递减。
除了method='nearest',其他method取值也能用fillna()方法实现：
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
pandas对象也有类似字典的iteritems()方法来迭代(key, value)。
为了每一行迭代DataFrame对象，有两种方法：
* iterrows(): 按照行来迭代(index, Series)。会把每一行转为Series对象。
* itertuples(): 按照行来迭代namedtuples. 这种方法比iterrows()快，大多数情况下推荐使用此方法。
警告：
迭代pandas对象通常会比较慢。所以尽量避免迭代操作，可以用以下方法替换迭代：
* 数据结构的内置方法，索引或numpy方法等
* apply()
* 使用cython写内循环
警告：
当迭代进行时永远不要有修改操作。
iteritems()
类似字典的接口，iteritems()对键值对进行迭代操作：
* Series: (index, scalar value)
* DataFrame: (column, Series)
* Panel: (item, DataFrame)
看一下例子：
Can you write Python code for it?

for item, frame in df.iteritems():
    print item, frame


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
qcut()方法计算样本的分位数，比如我们可以将正态分布的数据 进行四分位数离散化：
Can you write Python code for it?

arr = np.random.randn(30)

factor = pd.qcut(arr, [0, .25, .5, .75, 1])

factor

pd.value_counts(factor)


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
searchsorted()
Series.searchsorted()类似numpy.ndarray.searchsorted()。 找到元素在排好序后的位置(下标)。
Can you write Python code for it?

ser = pd.Series([1,2,3])

ser.searchsorted([0, 3]) #元素0下标是0， 元素3下标是2。注意不同元素之间是独立的，所以元素3的位置是2而不是插入0后的3.

ser.searchsorted([0, 4])

ser.searchsorted([1, 3], side='right')

ser.searchsorted([1, 3], side='left')

ser = pd.Series([3, 1, 2])

ser.searchsorted([0, 3], sorter=np.argsort(ser))


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
select_dtypes()有两个参数：include, exclude。含义是要选择的列的dtype和不选择列的dtype。
Can you write Python code for it?

df.select_dtypes(include=[bool])

df.select_dtypes(include=['bool'])

df.dtypes

df.select_dtypes(include=['number', 'bool'], exclude=['unsignedinteger'])


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
不同类型的对象(比如pandas数据结构、numpy数组)之间进行逐元素的比较也是没有问题的，前提是两个对象的shape要相同。
Can you write Python code for it?

pd.Series(['foo', 'bar', 'baz']) == pd.Index(['foo', 'bar', 'qux'])

pd.Series(['foo', 'bar', 'baz']) == np.array(['foo', 'bar', 'qux'])

pd.Series(['foo', 'bar', 'baz']) == pd.Series(['foo', 'bar']) #长度不相同


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
为什么df + df == df*2 返回的结果含有False？因为NaN和NaN比较厚结果为False！
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
二者的区别是：如果索引不是有序的，reindex()会报错，而fillna()和interpolate()不会检查索引是否有序。
重索引时 有条件地填充NaN
limit和tolerance参数会对填充操作进行条件限制，通常限制填充的次数。
Can you write Python code for it?

ts2.reindex(ts.index, method='ffill', limit=1)

ts2

ts

ts2.reindex(ts.index, method='ffill', tolerance='1 day')


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
但要知道不同shape的numpy数组之间是可以直接比较的！因为广播！即使无法广播，也不会Error而是返回False。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
使用empty属性检测一个pandas对象是否为空。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
函数应用
如果你想用自己写的方法或其他库方法操作pandas对象，你应该知道下面的三种方式。 具体选择哪种方式取决于你是想操作整个DataFrame对象还是DataFrame对象的某几行或某几列，或者逐元素操作。

管道 pipe()
基于列或行的函数引用 apply()
对DataFrame对象逐元素计算 applymap()

对管道
DataFrame和Series当然能够作为参数传入方法。然而，如果涉及到多个方法的序列调用，推荐使用pipe()。看一下例子：
Can you write Python code for it?

#f, g 和h是三个方法，接收DataFrame对象，返回DataFrame对象
f(g(h(df), arg1=1), arg2=2, arg3=3)


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
区间离散化
cut() qcut()方法可以对连续数据进行离散化
Can you write Python code for it?

arr = np.random.randn(20)

factor = pd.cut(arr, 4)

factor

factor = pd.cut(arr, [-5, -1, 0, 1, 5]) #输入 离散区间

factor


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
另一个有用的特性是对DataFrame对象传递Series方法，然后针对DF对象的每一列或每一行执行 Series内置的方法！
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
同样可以对降维后的结果再进行降维。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
基于dtype 选择列
select_dtypes()方法实现了基于列dtype的构造子集方法。
Can you write Python code for it?

df = pd.DataFrame({'string': list('abc'),
                    'int64': list(range(1, 4)),
                    'uint8': np.arange(3, 6).astype('u1'),
                    'float64': np.arange(4.0, 7.0),
                    'bool1': [True, False, True],
                    'bool2': [False, True, False],
                    'dates': pd.date_range('now', periods=3).values,
                    'category': pd.Series(list("ABC")).astype('category')})

df

df['tdeltas'] = df.dates.diff()

df['uint64'] = np.arange(3, 6).astype('u8')

df['other_dates'] = pd.date_range('20130101', periods=3).values

df['tz_aware_dates'] = pd.date_range('20130101', periods=3, tz='US/Eastern')

df

df.dtypes


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
如果DataFrame对象有的列是数值类型，有的列不是数值类型，describe()仅对数值类型的列进行计算。
Can you write Python code for it?

frame = pd.DataFrame({'a':['Yes', 'Yes', 'NO', 'No'], 'b':range(4)})

frame.describe()


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
如果DataFrame或Panel对象的数据类型相同(比如都是 int64)，修改object.values相当于直接修改原对象的值。如果数据类型不相同，则根本不能对values属性返回值进行赋值。
注意:
如果对象内数据类型不同，values返回的ndarray的dtype将是能够兼容所有数据类型的类型。比如，有的列数据是int，有的列数据是float，.values返回的ndarray的dtype将是float。
加速的操作
pandas从0.11.0版本开始使用numexpr库对二值数值类型操作加速，用bottleneck库对布尔操作加速。
加速效果对大数据尤其明显。
这里有一个速度的简单对比，使用100,000行* 100列的DataFrame:

所以，在安装pandas后也要顺便安装numexpr, bottleneck。
灵活的二元运算
在所有的pandas对象之间的二元运算中，大家最感兴趣的一般是下面两个：
* 高维数据结构（比如DataFrame）和低维数据结构（比如Series）之间计算时的广播(broadcasting)行为
* 计算时有缺失值
广播
DataFrame对象内置add(),sub(),mul(),div()以及radd(), rsub(),...等方法。
至于广播计算，Series的输入是最有意思的。
Can you write Python code for it?

df = pd.DataFrame({'one' : pd.Series(np.random.randn(3), index=['a', 'b', 'c']),
                       'two' : pd.Series(np.random.randn(4), index=['a', 'b', 'c', 'd']),
                       'three' : pd.Series(np.random.randn(3), index=['b', 'c', 'd'])})

df

row = df.ix[1]
row

column = df['two']

column

df.sub(row, axis='columns')

df.sub(row, axis=1)

df.sub(row, axis='index')

df.sub(row, axis=0)


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
如果Series内数据是非数值类型，describe()也能给出一定的统计结果
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
如果pandas对象的一列中有多种数据类型，dtype返回的是能兼容所有数据类型的类型，object范围最大的。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
如果只想改变列或者索引的label，DataFrame也提供了reindex_axis()方法，接收label和axis。
Can you write Python code for it?

rs = s.reindex(df.index)

rs

rs.index is df.index


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
如果多个数值都是最大值或最小值，idxmax() idxmin()返回最大值、最小值第一次出现对应的索引值
Can you write Python code for it?

df3 = pd.DataFrame([2, 1, 1, 3, np.nan], columns=['A'], index=list('edcba'))
df3

df3['A'].idxmin()


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
如果想要知道某种数据类型的所有子类型，比如numpy.number类型，你可以定义如下的方法：
Can you write Python code for it?

def subdtypes(dtype):
        subs = dtype.__subclasses__()
        if not subs:
            return dtype
        return [dtype, [subdtypes(dt) for dt in subs]]

subdtypes(np.generic)


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
如果要选择字符串类型的列，必须使用object类型。
Can you write Python code for it?


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
如果非要知道非数值列的统计指标呢？describe提供了include参数，取值范围{'object', 'number', 'all'}。
看一下例子, 注意'object'和'number'都是在列表中，而'all'不需要放在列表中：
Can you write Python code for it?

frame.describe(include=['object']) #只对非数值列进行统计计算

frame.describe(include=['number'])

frame.describe(include='all')#'all'不是列表


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
对object类型进行转型
convert_objects()方法能对object类型进行转型。如果想转为数字，参数是convert_numeric=True。
Can you write Python code for it?

df3['D'] = '1.'

df3['E'] = '1'

df3

df3.dtypes #现在'D' 'E'两列都是object类型

df3.convert_objects(convert_numeric=True).dtypes

df3['D'] = df3['D'].astype('float16')

df3['E'] = df3['E'].astype('int32')

df3.dtypes


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
对于只含有一个元素的pandas对象，对其进行布尔检测，使用bool():
Can you write Python code for it?

pd.Series([True]).bool()

pd.Series([False]).bool()

pd.DataFrame([[True]]).bool()

pd.DataFrame([[False]]).bool()


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
应用逐元素操作的Python方法
既然不是所有的方法都能被向量化(接收NumPy数组，返回另一个数组或者值),但是DataFrame内置的applymap()和Series的map()方法能够接收任意的接收一个值且返回一个值的Python方法。
Can you write Python code for it?

df4 = pd.DataFrame(np.random.randn(4, 3),index=['a','b','c','d'],columns=['one', 'two', 'three'])
df4

f = lambda x:len(str(x))

df4['one'].map(f)

df4.applymap(f)


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
意思操作返回一个和输入对象同类型的对象，值类型为bool，返回结果可以用于检索。
布尔降维 Boolean Reductions
pandas提供了三个方法(any(), all(), bool())和一个empty属性来对布尔结果进行降维。
Can you write Python code for it?

df>0

(df>0).all() #与操作

(df > 0).any()#或操作


I am working on the file "05其他/pandas文档-zh-master/.ipynb_checkpoints/数据结构的内置方法-checkpoint.ipynb".
The first task is:
所有的这些方法都有skipna参数，含义是计算过程中是否剔除缺失值，skipna默认值为True。
Can you write Python code for it?

df.sum(0, skipna=False)

df.sum(axis=1, skipna=True)