I am working on the file "MNIST/0410 -  MNIST Project 6 - The ROC Curve/MNIST.ipynb".
The first task is:
This is the same as the line above
Can you write Python code for it?

thresholds[idx]

y_train_pred_90 = (y_scores > thresholds[idx])

precision_score(y_train_0, y_train_pred_90)

recall_score(y_train_0, y_train_pred_90)


I am working on the file "MNIST/0410 -  MNIST Project 6 - The ROC Curve/MNIST.ipynb".
The first task is:
Tips: Typically we shuffle the training set. This ensures the training set is randomised and your data distribution is consistent. However, shuffling is a bad idea for time series data.
Shuffling the dataset
Alternative Method
Can you write Python code for it?

import numpy as np

shuffle_index = np.random.permutation(num_split)
X_train, y_train = X_train[shuffle_index], y_train[shuffle_index]


I am working on the file "MNIST/0410 -  MNIST Project 6 - The ROC Curve/MNIST.ipynb".
The first task is:
Training a Binary Classifier
To simplify our problem, we will make this an exercise of "zero" or "non-zero", making it a two-class problem.
We need to first convert our target to 0 or non zero.
Can you write Python code for it?

y_train_0 = (y_train == 0)

y_train_0

y_test_0 = (y_test == 0)

y_test_0


I am working on the file "MNIST/0410 -  MNIST Project 6 - The ROC Curve/MNIST.ipynb".
The first task is:
With the decision scores, we can compute precision and recall for all possible thresholds using the precision_recall_curve() function:
Can you write Python code for it?

from sklearn.metrics import precision_recall_curve

precisions, recalls, thresholds = precision_recall_curve(y_train_0, y_scores)

def plot_precision_recall_vs_threshold(precisions, recalls, thresholds):
    plt.plot(thresholds, precisions[:-1], "b--", label="Precision")
    plt.plot(thresholds, recalls[:-1], "g--", label="Recall")
    plt.xlabel("Threshold")
    plt.legend(loc="upper left")
    plt.ylim([-0.5,1.5])

plt.figure(figsize=(12,8));
plot_precision_recall_vs_threshold(precisions, recalls, thresholds)
plt.show()


I am working on the file "MNIST/0410 -  MNIST Project 6 - The ROC Curve/MNIST.ipynb".
The first task is:
cross_val_score using K-fold Cross-Validation
K-fold cross-validation splits the training set into K-folds and then make predictions and evaluate them on each fold using a model trained on the remaning folds.
Can you write Python code for it?

from sklearn.model_selection import cross_val_score

cross_val_score(clf, X_train, y_train_0, cv=3, scoring='accuracy')


I am working on the file "MNIST/Atelier-keras-MNIST.ipynb".
The first task is:
Apprentissage et prévision du test Avec réseau dense
Première tentative d'appliquer un réseaux de neurone de type Perceptron classique avec 4 couches:
* Dense: 52 neurones + Foncton d'activation relu
* Dropout: 20% des neurones tiré aléatoirement sont desactivés
* Dense: 52 neurones + Foncton d'activation relu
* Dropout: 20% des neurones tiré aléatoirement sont desactivés
Une dernière couche softmax fournit la classification
Apprentissage
Can you write Python code for it?

X_train.shape

# Définition du réseau
model = km.Sequential()
model.add(kl.Dense(128, activation='relu', input_shape=(784,)))
model.add(kl.Dropout(0.2))
model.add(kl.Dense(128, activation='relu'))
model.add(kl.Dropout(0.2))
model.add(kl.Dense(N_classes, activation='softmax'))
# Réumé
model.summary()


I am working on the file "MNIST/Atelier-keras-MNIST.ipynb".
The first task is:
Attention, avec Keras, la variable réponse doit être une matrice binaire où chaque classe est représentée par une indicatrice: pour chaque individu, l'élément de la colone correspondant à la classe à laquelle il appartient est à 1, sinon il est à 0.
Keras possède une fonction to_catergorical permettant de convertir directement le vecteur variable Y_train de réponse en matrice (array numpy) indicatriceY_train_cat.
C'est l'équivalent de get_dummies de pandas
 ou OneHotEncoder de scikit-learn.
Can you write Python code for it?

Y_train_cat = ku.to_categorical(Y_train, N_classes)
Y_test_cat = ku.to_categorical(Y_test, N_classes)


I am working on the file "MNIST/Atelier-keras-MNIST.ipynb".
The first task is:
Autre architecture
Réseau
Test d'un réseau de convolution constitué de 7 couches:

Une couche de convolution 2D, avec fenêtre de taille 3x3 et une fonction d'activation relu
Une couche de convolution 2D, avec fenêtre de taille 3x3 et une fonction d'activation relu
Une couche max pooling de fenêtre 2x2
Une couche dropout où 25% des neurones sont desactivés
Une couche Flatten transforme les images $N \times N$ en vecteurs $N^2$.
Une couche classique de 128 neurones
Une couche dropout ou 50% des neurones sont desactivés

Une couche softmax fournit la classification
Can you write Python code for it?

# descrition du réseau
model = km.Sequential()
model.add(kl.Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(28,28, 1), data_format="channels_last"))
model.add(kl.Conv2D(64, (3, 3), activation='relu'))
model.add(kl.MaxPooling2D(pool_size=(2, 2)))
model.add(kl.Dropout(0.25))
model.add(kl.Flatten())
model.add(kl.Dense(128, activation='relu'))
model.add(kl.Dropout(0.5))
model.add(kl.Dense(N_classes, activation='softmax'))
# Résumé
model.summary()
# Apprentissage
model.compile(loss="categorical_crossentropy",
              optimizer=ko.Adadelta(),
              metrics=['accuracy'])
ts=time.time()
model.fit(X_train_conv, Y_train_cat,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(X_test_conv, Y_test_cat))
te=time.time()
t_train_conv = te-ts

score_conv = model.evaluate(X_test_conv, Y_test_cat, verbose=0)
predict_conv = model.predict(X_test_conv)
print('Test loss:', score_conv[0])
print('Test accuracy:', score_conv[1])
print("Time Running: %.2f seconds" %t_train_conv )

fig=plt.figure(figsize=(7,6))
ax = fig.add_subplot(1,1,1)
ax = sb.heatmap(pd.DataFrame(confusion_matrix(Y_test, predict_conv.argmax(1))), annot=True, fmt="d")


I am working on the file "MNIST/Atelier-keras-MNIST.ipynb".
The first task is:
Convolutional Network (ConvNet)
Les propriété d'invariance par translation introduites par les couches opérant une convolution des images ont un impact important sur la qualité des résultats.
LeNet5
On teste dans un premier temps le modèle LeNet5 proposer par LeCun et al.
Can you write Python code for it?

LeNet5model = km.Sequential()
LeNet5model.add(kl.Conv2D(filters = 6, kernel_size = 5, strides = 1, activation = 'tanh',
input_shape = (28,28,1)))
LeNet5model.add(kl.MaxPooling2D(pool_size = 2, strides = 2))
LeNet5model.add(kl.Conv2D(filters = 16, kernel_size = 5,strides = 1, activation = 'tanh'))
LeNet5model.add(kl.MaxPooling2D(pool_size = 2, strides = 2))
LeNet5model.add(kl.Flatten())
LeNet5model.add(kl.Dense(units = 120, activation = 'tanh'))
LeNet5model.add(kl.Dense(units = 84, activation = 'tanh'))
LeNet5model.add(kl.Dense(units = 10, activation = 'softmax'))

LeNet5model.summary()


I am working on the file "MNIST/Atelier-keras-MNIST.ipynb".
The first task is:
Edge detection
Dans cette partie vous pouvez explorer l'effet de filtre de convolution simple sur une image.
Un réseau de neuronne constitué d'une couche de convolution constitué d'un seul filtre définie manuellement (non appris par optimisation) est définie dans le code ci-dessous.
Can you write Python code for it?

from keras.models import Sequential
from keras.layers import Conv2D

conv_filter = np.array([
        [0.2, -0.2, 0],
        [0.2, -0.2, 0],
        [0.2, -0.2, 0],
    ])

def my_init_filter(shape, conv_filter = conv_filter, dtype=None):
    xf,yf = conv_filter.shape
    array = conv_filter.reshape(xf, yf, 1, 1)
    return array
my_init_filter(0).shape

conv_edge = Sequential([
    Conv2D(kernel_size=(3,3), filters=1, kernel_initializer=my_init_filter, input_shape=(28, 28, 1))
])


I am working on the file "MNIST/Atelier-keras-MNIST.ipynb".
The first task is:
Lecture des données d'apprentissage et de test
Les données peuvent être préalablement téléchargées ou directement lues. Ce sont celles originales du site MNIST DataBase mais préalablement converties au format .csv, certes plus volumineux mais plus facile à lire. Attention le fichier mnist_train.zip présent dans le dépôt est compressé.
Can you write Python code for it?

# Lecture des données d'apprentissage
N_classes = 10

# path="" # Si les données sont dans le répertoire courant sinon:
path=""
Dtrain=pd.read_csv(path+"mnist_train.zip",header=None)
X_train = Dtrain.values[:,:-1]
Y_train = Dtrain.values[:,-1]

Dtest=pd.read_csv(path+"mnist_test.csv",header=None)
X_test = Dtest.values[:,:-1]
Y_test = Dtest.values[:,-1]


I am working on the file "MNIST/Atelier-keras-MNIST.ipynb".
The first task is:
Q Notez que dans la fonction my_init_filter les dimensions de l'image sont modifiés. A quoi correspondent les deux dimensions ajoutées?
Can you write Python code for it?

img_in = np.expand_dims(x, 0)
img_out = conv_edge.predict(img_in)

fig, (ax0, ax1, ax2) = plt.subplots(ncols=3, figsize=(15, 5))
ax0.imshow(img_in[0,:,:,0], cmap="binary")
ax0.set_title("Image originale")
ax0.grid(False)

norm_conv_filter = (conv_filter-conv_filter.min())/conv_filter.max()
ax1.imshow(norm_conv_filter.astype(np.uint8), cmap="binary")
ax1.set_title("Filtre")
ax1.grid(False)

ax2.imshow(img_out[0,:,:,0].astype(np.uint8), cmap="binary")
ax2.set_title("Image Filtre")
ax2.grid(False)


I am working on the file "MNIST/Atelier-keras-MNIST.ipynb".
The first task is:
Q Que dire de ces résultats ?
Q Faites tourner de nouveaux l'algorithme en normalisant les données afin que celles-ci soit comprises entre 0 et 1. Qu'observez vous?
Convolutional Layers
Format des données
Dans les exemples précédents. Les données était "applaties". Une imade de $28\times 28=784$ pixels est considérée comme un vecteur.
Pour pouvoir utiliser le principe de la convolution la structure des images est conservée. Une image n'est pas un vecteur de tailles $784\times 1$ mais une matrice de taille $28\times 28$. Une troisième dimension est également nécessaire pour décrire afin de prendre en compte les différents channels de l'image. Dans le cas de MNIST cette dernière dimension est de taille 1 car les pixels ne sont décrits qu'avec un seul niveau de gris. Cependant, des images couleurs en RGB sont généralement codées avec trois niveaux d'intensité (Rouge, Vert et Bleus).
Ainsi X_train est réorganisée en cube ou multitableau de dimensions $60000\times 28\times 28\times 1$ pour être utilisé dans un réseau de convolution avec Keras.
Can you write Python code for it?

X_train_conv = X_train.reshape(60000, 28, 28, 1)
X_test_conv = X_test.reshape(10000, 28, 28, 1)


I am working on the file "MNIST/Atelier-keras-MNIST.ipynb".
The first task is:
Q Que dire du temps de calcul? Pourquoi est-il plus long que le réseau Dense?
Résultats
Can you write Python code for it?

score_conv = LeNet5model.evaluate(X_test_conv, Y_test_cat, verbose=0)
predict_conv = LeNet5model.predict(X_test_conv)
print('Test loss:', score_conv[0])
print('Test accuracy:', score_conv[1])
print("Time Running: %.2f seconds" %t_train_conv )

fig=plt.figure(figsize=(7,6))
ax = fig.add_subplot(1,1,1)
ax = sb.heatmap(pd.DataFrame(confusion_matrix(Y_test, predict_conv.argmax(1))), annot=True, fmt="d")


I am working on the file "MNIST/Atelier-keras-MNIST.ipynb".
The first task is:
Q Quel est l'effet du filtre défini ici?
Can you write Python code for it?

img_in = np.expand_dims(x, 0)
img_out = conv_sp.predict(img_in)

fig, (ax0, ax1, ax2) = plt.subplots(ncols=3, figsize=(15, 5))
ax0.imshow(img_in[0,:,:,0].astype(np.uint8),
           cmap="binary");
ax0.grid(False)

norm_conv_filter = (conv_filter-conv_filter.min())/conv_filter.max()
ax1.imshow(norm_conv_filter.astype(np.uint8),
           cmap="binary");
ax1.grid(False)

ax2.imshow(img_out[0,:,:,0].astype(np.uint8),
           cmap="binary");
ax2.grid(False)


I am working on the file "MNIST/Atelier-keras-MNIST.ipynb".
The first task is:
Q Retrouvez manuellement le nombre de paramètres.
Q Que dire du nombre de paramètres de ce réseau par rapport au réseau dense précédement défini?
Can you write Python code for it?

# Apprentissage
LeNet5model.compile(loss="categorical_crossentropy",
              optimizer=ko.Adadelta(),
              metrics=['accuracy'])
ts=time.time()
LeNet5model.fit(X_train_conv, Y_train_cat,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(X_test_conv, Y_test_cat))
te=time.time()
t_train_conv = te-ts


I am working on the file "MNIST/Atelier-keras-MNIST.ipynb".
The first task is:
Visualisation des données
Can you write Python code for it?

import keras.preprocessing.image as kpi
fig  = plt.figure(figsize=(5,5))
ax = fig.add_subplot(1,1,1)
x = kpi.img_to_array(X_train_conv[0])
ax.imshow(x[:,:,0]/255, interpolation='nearest', cmap="binary")
ax.grid(False)
plt.show()


I am working on the file "MNIST/Atelier-pyspark-MNIST.ipynb".
The first task is:
Erreur sur l'échantillon test
Can you write Python code for it?

predictionsRDD = model_lr.transform(testDF)
labelsAndPredictions = predictionsRDD.select("label","prediction").collect()
nb_good_prediction = sum([r[0]==r[1] for r in labelsAndPredictions])
nb_test = testDF.count()
testErr = 1-nb_good_prediction/nb_test
print('Test Error = ' + str(testErr)) # (0.08 avec taux =1)


predictionsRDD = model_dt.transform(testDF)
labelsAndPredictions = predictionsRDD.select("label","prediction").collect()
nb_good_prediction = sum([r[0]==r[1] for r in labelsAndPredictions])
nb_test = testDF.count()
testErr = 1-nb_good_prediction/nb_test
print('Test Error = ' + str(testErr))


I am working on the file "MNIST/Atelier-pyspark-MNIST.ipynb".
The first task is:
Erreur sur l'échantillon test
Erreur sur l'échantillon test
Can you write Python code for it?

predictionsRDD = model_rf.transform(testDF)
labelsAndPredictions = predictionsRDD.select("label","prediction").collect()
nb_good_prediction = sum([r[0]==r[1] for r in labelsAndPredictions])
nb_test = testDF.count()
testErr = 1-nb_good_prediction/nb_test
print('Test Error = ' + str(testErr))


I am working on the file "MNIST/Atelier-pyspark-MNIST.ipynb".
The first task is:
Gestion des données
Importation et transformation des données au format RDD
Les données sont déjà partagée en une partie apprentissage et une test utilisée pour les comparaisons entre méthodes dans les publications. Ce sont bien les données du site MNIST mais transformée au format .csv pour en faciliter la lecture.
Elles doivent être stockées à un emplacement accessibles de tous les noeuds du cluster pour permettre la construction de la base de données réparties (RDD).
Dans une utilisation monoposte (standalone) de Spark, elles sont simplement chargées dans le répertoire courant.
Can you write Python code for it?

# Chargement des fichiers
import urllib.request
f = urllib.request.urlretrieve("https://www.math.univ-toulouse.fr/~besse/Wikistat/data/mnist_train.csv",DATA_PATH+"mnist_train.csv")
f = urllib.request.urlretrieve("https://www.math.univ-toulouse.fr/~besse/Wikistat/data/mnist_test.csv",DATA_PATH+"mnist_test.csv")

# Transformation du fichier texte en RDD de valeurs
## Données d'apprentissage
# Transformation ou étape map de séparation des champs
trainRDD = sc.textFile(DATA_PATH+"mnist_train.csv").map(lambda l: [float(x) for x in l.split(',')])
# Action
trainRDD.count() # taille de l'échantillon
#test


I am working on the file "MNIST/Atelier-pyspark-MNIST.ipynb".
The first task is:
Méthode de classification
Les méthodes de classifications de la librairie SparkML suivent le même shéma d'utilisation.
Il faut dans un premier temps crée un objets Estimators pour configurer les paramètres de la méthode.
Dans un second temps on réalise l'apprentissage en appliquant la fonction fit de l'Estimators sur la DataFrame d'apprentissage. Cette commande créé un objet différent, le Transformers qui permettra de réaliser les prédictions.
Par défaut les différentes méthodes considère que les noms des colonnes correspondants aux variables et au prédicants du jeux d'apprentissage sont respectivement "features" et "label". Tandis que les prédictions seront automatiquement assigné à une colonne de nom "prediction".
Il est conseillé de garder cette terminiologie, mais ces attributs par défaut peuvent être modifié en spécifiant les paramètres  featuresCol, labelCol et predictionCol de chaque méthode.
Régression logistique
Exemple d'utilisation pour expliciter la syntaxe mais sans grand intérêt pour ces données qui ne satisfont pas à des frontières de discrimination linéaires. L'algorithme permettant de réaliser une regression logistique multinomial est l'algorithme softmax.
Can you write Python code for it?

### Logistic Regression
from pyspark.ml.classification import LogisticRegression

### Configuraiton des paramètres de la méthode
time_start=time.time()
lr = LogisticRegression(maxIter=100, regParam=0.01, fitIntercept=False, tol=0.0001,
            family = "multinomial", elasticNetParam=0.0) #0 for L2 penalty, 1 for L1 penalty

### Génération du modèle
model_lr = lr.fit(trainDF)

time_end=time.time()
time_lrm=(time_end - time_start)
print("LR prend %d s" %(time_lrm)) # (104s avec taux=1)


I am working on the file "MNIST/Atelier-pyspark-MNIST.ipynb".
The first task is:
Random Forest
Les $k$-nn ne sont pas "scalables" et donc pas présents. Voici la syntaxe et les paramètres associés à l'algorithme des forêts aléatoires. Parmi ceux "classiques" se trouvent numTrees, featureSubsetStrategy, impurity, maxdepth et en plus maxbins comme pour les arbres. Les valeurs du paramètres maxDepth est critique pour la qualité de la prévision. en principe, il n'est pas contraint, un arbre peut se déployer sans "limite" mais face à des données massives cela peut provoquer des plantages intempestifs.
Can you write Python code for it?

### Random Forest
from pyspark.ml.classification import RandomForestClassifier

### Configuraiton des paramètres de la méthode
time_start=time.time()
rf = RandomForestClassifier(numTrees = 2, impurity='gini', maxDepth=12,
                            maxBins=32, seed=None)

### Génération du modèle
model_rf = rf.fit(trainDF)

time_end=time.time()
time_rf=(time_end - time_start)
print("RF takes %d s" %(time_rf))#


I am working on the file "MNIST/Atelier-pyspark-MNIST.ipynb".
The first task is:
Sous-échantillon d'apprentissage
Extraction d'un sous-échantillon d'apprentissage pour tester les programmes sur des données plus petites. Itérer cette démarche permet d'étudier l'évolution de l'erreur de prévision en fonction de la taille de l'échantillon d'apprentissage.
Can you write Python code for it?

tauxEch=0.1 # tester pour des tailles croissantes d'échantillon d'apprentissage
(trainData, DropDatal) = trainRDD.randomSplit([tauxEch, 1-tauxEch])
trainData.count()


I am working on the file "MNIST/Atelier-scikit_learn-MNIST.ipynb".
The first task is:
3.2 Avec Random Forest (Scikit-learn) et 250 arbres
Can you write Python code for it?

from sklearn.ensemble import RandomForestClassifier
from sklearn.cross_validation  import train_test_split
# tailles croissantes de l'échantillon d'apprentissage
arrayErreur=np.empty((12,3))
nArbres=250
for i in range(1,13):
    n=5000*i
    arrayErreur[i-1,0]=n
    if i==12:
        n=59999
    Xtrain,Xdrop,ytrain,ydrop=train_test_split(Dtrain,Ltrain,train_size=n)
    tps1 = time.clock()
    rf = RandomForestClassifier(n_estimators=nArbres,
       criterion='gini', max_depth=None, min_samples_split=2,
       min_samples_leaf=1, max_features='auto', max_leaf_nodes=None,
       bootstrap=True, oob_score=True, n_jobs=-1,random_state=None, verbose=0)
    rf.fit(Xtrain,ytrain)
    tps2=time.clock()
    arrayErreur[i-1,2]=1-rf.score(Dtest,Ltest)
    arrayErreur[i-1,1]=tps2 - tps1
dataframeErreur=pd.DataFrame(arrayErreur,columns=["Taille","Temps","Erreur"])
print(dataframeErreur)

# Graphes supersosés
from __future__ import division
from scipy import *
from pylab import *

x = linspace(5,60,12)
fig = plt.figure()
# premier graphe
ax1 = fig.add_subplot(111)
ax1.plot(x,dataframeErreur["Temps"] , '-b', label=ur"Temps",lw=1.5,marker=".",markersize=6)
# absisses communes
xlim(0,65)
xlabel(ur"Taille échantillon x1000", fontsize=15)
ylim(0, 100)
ylabel(ur"Temps (s)", color='b', fontsize=15)
legend(loc=2)
# 2ème graphe
ax2 = ax1.twinx()
ax2.plot(x,dataframeErreur["Erreur"] ,'--',color='black', label=ur"Erreur",lw=1.5,marker=".",markersize=6)
ylim(0, 0.1)
ylabel(ur"Erreur (%)",  fontsize=15)
legend(loc=1)
show()


I am working on the file "MNIST/Atelier-scikit_learn-MNIST.ipynb".
The first task is:
Il faudrait ré-appliquer la procédure d'otpimisation de $k$ par validation croisée décrite dans le tutoriel d'introduction à scikit-learn. Néanmoins la solution $k=10$ est raisonnable et on retrouve une performance classique sur ce type de données: 3.3%, pour une méthode utilisée sans raffinement.
C'est en effet une autre distance qu'il faudrait utiliser avec les $k$ plus proches voisins pour améliorer sensiblement les résultats mais avec un coût beaucoup plus élevé en temps de calcul. Un autre scénario propose ainsi le calcul d'une distance tangentielle entre les images (Simard et al. (1998)). Le programme Matlab fait appel à un programme en C. L'intégration dans du code python plutôt que Matlab resterait à faire...
2.2  Random forest
Les forêts aléatoires sont également une approche raisonnable, à moindre coût de développement, sur ces données. Analyser en détail la liste des paramètres proposés dans l'implémentation de cet algorithme. Consulter pour ce faire la documentation en ligne.
Les valeurs par défaut des paramètres sont utilisées sauf pour le nombre d'arbres: 100 au lieu de 10, et le nombre de processeurs utilisés: -1 au lieu de 1 (tous sont utilisés sauf 1 pour le système). Attention, tous les paramètres disponibles ne sont pas listés.
Can you write Python code for it?

from sklearn.ensemble import RandomForestClassifier
tps0 = time.clock()
rf = RandomForestClassifier(n_estimators=100,
   criterion='gini', max_depth=None, min_samples_split=2,
   min_samples_leaf=1, max_features='auto', max_leaf_nodes=None,
   bootstrap=True, oob_score=True, n_jobs=-1,random_state=None, verbose=0)
rf.fit(Dtrain,Ltrain)
tps1 = time.clock()
print("Temps de configutration RF :" ,tps1 - tps0)

# erreur out-of-bag
erreur_oob=1-rf.oob_score_
tps2 = time.clock()
print("Temps execution RF :", tps2 - tps0, "Erreur oob:", erreur_oob)

# erreur sur l'échantillon test
1-rf.score(Dtest,Ltest)

cm = confusion_matrix(Ltest, rf.predict(Dtest))
print(cm)


I am working on the file "MNIST/Atelier-scikit_learn-MNIST.ipynb".
The first task is:
Les données peuvent être préalablement téléchargées ou directement lues. Ce sont celles originales du site MNIST DataBase mais préalablement converties au format .csv, certes plus volumineux mais plus facile à lire. Attention le fichier mnist_train.zip présent dans le dépôt est compressé.
Can you write Python code for it?

# Lecture des données d'apprentissage
# path="" # Si les données sont dans le répertoire courant sinon:
path="http://www.math.univ-toulouse.fr/~besse/Wikistat/data/"
Dtrain=pd.read_csv(path+"mnist_train.csv",header=None)
Dtrain.head()

# Extraction puis suppression de la dernière colonne des labels
Ltrain=Dtrain.iloc[:,784]
Dtrain.drop(Dtrain.columns[[784]], axis=1,inplace=True)
Dtrain.head()

# Dimensions de l'échantillon
Dtrain.shape

# Même chose pour les données de test
Dtest=pd.read_csv(path+"mnist_test.csv",header=None)
Ltest=Dtest.iloc[:,784]
Dtest.drop(Dtest.columns[[784]], axis=1,inplace=True)
Dtest.shape

# affichage d'un chiffre
plt.figure(1, figsize=(3, 3))
plt.imshow(np.matrix(Dtest.iloc[1,:]).reshape(28,28), cmap=plt.cm.gray_r, interpolation='nearest')
plt.show()


I am working on the file "MNIST/Atelier-scikit_learn-MNIST.ipynb".
The first task is:
Résultat sans grand intérêt mais qui montre la difficulté de regouper les caractères identiques à l'aide de la distance euclidienne usuelle; il y a beaucoup de confusion entre les classes.
2 Apprentissage et prévision du test
2.1 $K$ plus proches voisins
Can you write Python code for it?

# Définition du modèle avec un nombre k "standard" de voisins
from sklearn.neighbors import KNeighborsClassifier
tps1 = time.clock()
knn = KNeighborsClassifier(n_neighbors=10,n_jobs=-1)
digit_knn=knn.fit(Dtrain, Ltrain)
tps2 = time.clock()
print("Temps de k-nn :",(tps2 - tps1)/60)

# Apprentissage et estimation de l'erreur de prévision sur l'échantillon test
tps1 = time.clock()
erreur=1-digit_knn.score(Dtest,Ltest)
tps2 = time.clock()
print("Temps:",(tps2 - tps1)/60,"Erreur:",erreur)


I am working on the file "MNISTSinglePredict.ipynb".
The first task is:
In the last step we predict a specific digit from the test set (only the last 10 samples are unseen), means we pass an actual image and SVC makes the prediction of which digit belongs to the image:
Can you write Python code for it?

testimage = -6

s_prediction = sclf.predict([dimages.data[testimage]])
print ('the image maybe belongs to ',s_prediction)
plt.imshow(dimages.images[testimage], cmap=plt.cm.gray_r, interpolation="nearest")
plt.show()


I am working on the file "MNISTSinglePredict.ipynb".
The first task is:
The same fit we try with a Random Forest Classifier to finish this lesson:
Can you write Python code for it?

#RandomForestClassifier
rfc_clf = RandomForestClassifier()
rfc_clf.fit(X,y)
rfc_prediction = rfc_clf.predict([dimages.data[testimage]])
print ('predict with RFC ',rfc_prediction)


I am working on the file "MNISTSinglePredict2Test.ipynb".
The first task is:
Gamma is the learning rate and the higher the value of gamma the more precise the decision boundary would be. C (regularization) is the penalty of the fault tolerance. Having a larger C will lead to smaller values for the slack variables. This means that the number of support vectors will decrease. When you run the prediction, it will need to calculate the indicator function for each support vector. Now we train (fit) the samples:
Can you write Python code for it?


I am working on the file "MNISTSinglePredict2Test.ipynb".
The first task is:
In the last step we predict a specific digit from the test set (only the last 10 samples are unseen), means we pass an actual image and SVC makes the prediction of which digit belongs to the image:
Can you write Python code for it?

testimage = -5

s_prediction = sclf.predict([dimages.data[testimage]])
print ('the image maybe belongs to ',s_prediction)
plt.imshow(dimages.images[testimage], cmap=plt.cm.gray_r, interpolation="nearest")
plt.show()


I am working on the file "MNISTSinglePredict2Test.ipynb".
The first task is:
Splitting the given images in 70:30 ratio shows a slight different confusion matrix so that 70% image is available for training and 30 % image as unseen or unknown is available for testing. Number 8 has probably most problems to get recognized! So disguise as 8 you can be a 6 or 9 and thats logical cause the 8 is in a 7-segment LCD display the base pattern! In german we say that with the word Achtung ;-).
Can you write Python code for it?

train_images, test_images, train_labels, test_labels = \
             train_test_split(pixels,labels,train_size=0.7,random_state=2);

sclf.fit(train_images, train_labels)
print('test score ',sclf.score(test_images,test_labels))
test_predictions = sclf.predict(test_images)
print(confusion_matrix(test_labels, test_predictions))


I am working on the file "MNISTSinglePredict2Test.ipynb".
The first task is:
The dataset () is available either for download from the UCI ML repository or via a Python library scikit-learn dataset. Then we setup the Support Vector Classifier with the training data X and the target y:
Can you write Python code for it?

sclf = SVC(gamma=0.001, C=100, kernel='linear')

X= dimages.data[:-10]
y= dimages.target[:-10]
print('train set samples:',len(X))


I am working on the file "MNISTSinglePredict2Test.ipynb".
The first task is:
There are many ways to improve this predict, including not using a vector classifier and go further with a neural classifier, but here’s a simple one to start what we do. Let’s just simplify our images by making them true black and white and stack an array.
MNIST Multi Prediction

Now we split explicit data in train- and test-set. Splitting the given images in 80:20 ratio so that 80% image is available for training and 20 % image is available for testing. We consider the data as pixels and the target as labels.
Convert and create the dataframe from datasets. We are using support vector machines for classification. Fit method trains the model and score will test it against the given test set and score.
A Support Vector Machine (SVM) is a discriminative classifier formally defined by a separating hyperplane. In other words, given labeled training data (supervised learning), the algorithm outputs an optimal hyperplane which categorizes new examples. In two dimensional space this hyperplane is a line dividing a plane in two parts where in each class lay in either side.
Can you write Python code for it?

#df = pd.DataFrame(data=dimages.data, columns=dimages.feature_names)
df = pd.DataFrame(data=dimages.data)
print(df.head(5))
df['target'] = pd.Series(dimages.target)
#df['pixels'] = dimages.data[:,1:64] #pd.Series(dimages.data[:,1:785])
print(df['target'])
print(df.shape) #print(df.info)

pixels = df
labels = df.target
print('pixels ',pixels)


I am working on the file "MNIST_2017/dump_/MNIST_TensorFlow_script.ipynb".
The first task is:
Close Session
(note: once the session is closed, the training cell cannot be run again...)
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/MNIST_TensorFlow_script.ipynb".
The first task is:
Import Data
prepare data by

applying 1-hot encoding: 1 = [1,0,0...0], 2 = [0,1,0...0] ...
reshaping into image shape: (# images, # vertical height, # horizontal width, # colors)
splitting data into train and validation set.
Can you write Python code for it?

data = pd.read_csv('../input/train.csv') # Read csv file in pandas dataframe
labels = np.array(data.pop('label')) # Remove the labels as a numpy array from the dataframe
labels = LabelEncoder().fit_transform(labels)[:, None]
labels = OneHotEncoder().fit_transform(labels).todense()
data = StandardScaler().fit_transform(np.float32(data.values)) # Convert the dataframe to a numpy array
data = data.reshape(-1, WIDTH, WIDTH, CHANNELS) # Reshape the data into 42000 2d images
train_data, valid_data = data[:-VALID], data[-VALID:]
train_labels, valid_labels = labels[:-VALID], labels[-VALID:]

print('train data shape = ' + str(train_data.shape) + ' = (TRAIN, WIDTH, WIDTH, CHANNELS)')
print('labels shape = ' + str(labels.shape) + ' = (TRAIN, LABELS)')


I am working on the file "MNIST_2017/dump_/MNIST_TensorFlow_script.ipynb".
The first task is:
Make a prediction about the test labels
Can you write Python code for it?

test_pred = session.run(tf_pred, feed_dict={tf_data:test_data})
test_labels = np.argmax(test_pred, axis=1)


I am working on the file "MNIST_2017/dump_/MNIST_TensorFlow_script.ipynb".
The first task is:
Plot an example:
Can you write Python code for it?

k = 0 # Try different image indices k
print("Label Prediction: %i"%test_labels[k])
fig = plt.figure(figsize=(2,2)); plt.axis('off')
plt.imshow(test_data[k,:,:,0]); plt.show()


I am working on the file "MNIST_2017/dump_/MNIST_TensorFlow_script.ipynb".
The first task is:
Run the session (Run this cell again if the desired accuracy is not yet reached).
Can you write Python code for it?

ss = ShuffleSplit(n_splits=STEPS, train_size=BATCH)
ss.get_n_splits(train_data, train_labels)
history = [(0, np.nan, 10)] # Initial Error Measures
for step, (idx, _) in enumerate(ss.split(train_data,train_labels), start=1):
    fd = {tf_data:train_data[idx], tf_labels:train_labels[idx]}
    session.run(tf_step, feed_dict=fd)
    if step%500 == 0:
        fd = {tf_data:valid_data, tf_labels:valid_labels}
        valid_loss, valid_accuracy = session.run([tf_loss, tf_acc], feed_dict=fd)
        history.append((step, valid_loss, valid_accuracy))
        print('Step %i \t Valid. Acc. = %f'%(step, valid_accuracy), end='\n')


I am working on the file "MNIST_2017/dump_/MNIST_TensorFlow_script.ipynb".
The first task is:
Train
open the session
Can you write Python code for it?

init = tf.global_variables_initializer()
session = tf.Session()
session.run(init)


I am working on the file "MNIST_2017/dump_/MNIST_TensorFlow_script.ipynb".
The first task is:
We use the categorical cross entropy loss for training the model.
As optimizer we could use a Gradient Descent optimizer [with or without decaying learning rate] or one of the more sophisticated (and easier to optimize) optimizers like Adam or RMSProp
Can you write Python code for it?

tf_loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits(tf_data),
                                                                 labels=tf_labels))
tf_acc = 100*tf.reduce_mean(tf.to_float(tf.equal(tf.argmax(tf_pred, 1), tf.argmax(tf_labels, 1))))

#tf_opt = tf.train.GradientDescentOptimizer(LR)
#tf_opt = tf.train.AdamOptimizer(LR)
tf_opt = tf.train.RMSPropOptimizer(LR)
tf_step = tf_opt.minimize(tf_loss)


I am working on the file "MNIST_2017/dump_/Mens_script.ipynb".
The first task is:
Display
number of unique elements in "team1"
Can you write Python code for it?

n = simple_df.team1.nunique()
n

trans_dict = {t: i for i, t in enumerate(simple_df.team1.unique())}
simple_df["team1"] = simple_df["team1"].apply(lambda x: trans_dict[x])
simple_df["team2"] = simple_df["team2"].apply(lambda x: trans_dict[x])
simple_df.head()


I am working on the file "MNIST_2017/dump_/Mens_script.ipynb".
The first task is:
Lets Start to create our CL network with Keras
Start by creating the embeddings and bias for the dot products of the 2 teams
Can you write Python code for it?

n_factors = 50

team1_in, t1 = embedding_input("team1_in", n, n_factors, 1e-4)
team2_in, t2 = embedding_input("team2_in", n, n_factors, 1e-4)

b1 = create_bias(team1_in, n)
b2 = create_bias(team2_in, n)

x = merge([t1, t2], mode="dot")
x = Flatten()(x)
x = merge([x, b1], mode="sum")
x = merge([x, b2], mode="sum")
x = Dense(1, activation="softmax")(x)
model = Model([team1_in, team2_in], x)
model.compile(Adam(0.001), loss="binary_crossentropy")

model.summary()


I am working on the file "MNIST_2017/dump_/Mens_script.ipynb".
The first task is:
Now that we have defined our network its time to determine the correct set of numbers that will make our predictions close to actual outputs of a match
Lets learn these numbers by minimising the loss using the Adam optimisation algorithm.
Can you write Python code for it?

train.shape
#print(train.head())

history = model.fit([train[:, 0], train[:, 1]], train[:, 2],validation_split=0.33, batch_size=64, nb_epoch=5, verbose=2)

plt.plot(history.history["loss"])
plt.show()

# list all data in history
print(history.history.keys())
# summarize history for accuracy
#plt.plot(history.history['acc'])
#plt.plot(history.history['val_acc'])
#plt.title('model accuracy')
#plt.ylabel('accuracy')
#plt.xlabel('epoch')
#plt.legend(['train', 'test'], loc='upper left')
#plt.show()
# summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()

sub =  pd.read_csv('../input/SampleSubmissionStage2.csv')
sub["team1"] = sub["ID"].apply(lambda x: trans_dict[int(x.split("_")[1])])
sub["team2"] = sub["ID"].apply(lambda x: trans_dict[int(x.split("_")[2])])
sub.head()

sub["pred"] = model.predict([sub.team1, sub.team2])
sub = sub[["ID", "pred"]]
sub.head()

sub.to_csv("CF_sm.csv", index=False)


I am working on the file "MNIST_2017/dump_/Mens_script.ipynb".
The first task is:
Shuffle the training to include some randomness
Can you write Python code for it?

train = simple_df.values
np.random.shuffle(train)

def embedding_input(name, n_in, n_out, reg):
    inp = Input(shape=(1,), dtype="int64", name=name)
    return inp, Embedding(n_in, n_out, input_length=1, W_regularizer=l2(reg))(inp)

def create_bias(inp, n_in):
    x = Embedding(n_in, 1, input_length=1)(inp)
    return Flatten()(x)


I am working on the file "MNIST_2017/dump_/men_2018_0ld_logistic_script.ipynb".
The first task is:
First we import some datasets of interest
Can you write Python code for it?

#the seed information
#df_seeds = pd.read_csv('../input/NCAATourneySeeds.csv')
#print(df_seeds.shape)
#print(df_seeds.head())
#print(df_seeds.Season.value_counts())

#the seed information
df_seeds = pd.read_csv('../input/NCAATourneySeeds_SampleTourney2018.csv')
print(df_seeds.shape)
print(df_seeds.head())
#print(df_seeds.Season.value_counts())

#tour information
df_tour = pd.read_csv('../input/RegularSeasonCompactResults_Prelim2018.csv')
print(df_tour.shape)
print(df_tour.head())


I am working on the file "MNIST_2017/dump_/men_2018_0ld_logistic_script.ipynb".
The first task is:
Here we get our submission info
Can you write Python code for it?

df_sample_sub1 = pd.read_csv('../input/SampleSubmissionStage1.csv')

#prepares sample submission
df_sample_sub2 = pd.read_csv('../input/SampleSubmissionStage2.csv')

df_sample_sub=pd.concat([df_sample_sub1, df_sample_sub2])

print(df_sample_sub.shape)
print(df_sample_sub.head())

df_sample_sub['Season'] = df_sample_sub['ID'].apply(lambda x : int(x.split('_')[0]) )
df_sample_sub['TeamID1'] = df_sample_sub['ID'].apply(lambda x : int(x.split('_')[1]) )
df_sample_sub['TeamID2'] = df_sample_sub['ID'].apply(lambda x : int(x.split('_')[2]) )
print(df_sample_sub.shape)
print(df_sample_sub.head())


I am working on the file "MNIST_2017/dump_/men_2018_0ld_logistic_script.ipynb".
The first task is:
Here we look at the comparable statistics.  For the TeamID2 column, we would consider the inverse of the ratio, and 1 minus the score attempt percentage.
Can you write Python code for it?

def geo_mean( x ):
    return np.exp( np.mean(np.log(x)) )

def harm_mean( x ):
    return np.mean( x ** -1.0 ) ** -1.0

team1d_score_spread = train_test_inner.groupby(['Season', 'TeamID1'])[['Score_Ratio', 'Score_Pct']]\
.agg({ 'Score_Ratio': geo_mean, 'Score_Pct' : harm_mean}).reset_index()\
.set_index('Season').rename(columns = {'Score_Ratio' : 'Score_Ratio1', 'Score_Pct' : 'Score_Pct1'})
team2d_score_spread = train_test_inner.groupby(['Season', 'TeamID2'])[['Score_Ratio', 'Score_Pct']]\
.agg({ 'Score_Ratio': geo_mean, 'Score_Pct' : harm_mean}).reset_index()\
.set_index('Season').rename(columns = {'Score_Ratio' : 'Score_Ratio2', 'Score_Pct' : 'Score_Pct2'})

score_spread = team1d_score_spread.join(team2d_score_spread).reset_index()

#geometric mean of score ratio of team 1 and inverse of team 2
score_spread['Score_Ratio'] = score_spread[['Score_Ratio1', 'Score_Ratio2']].apply(lambda x : ( x[0] * ( x[1] ** -1.0) ), axis = 1 ) ** 0.5

#harmonic mean of score pct
score_spread['Score_Pct'] = score_spread[['Score_Pct1', 'Score_Pct2']].apply(lambda x : 0.5*( x[0] ** -1.0 ) + 0.5*( 1.0 - x[1] ) ** -1.0, axis = 1 ) ** -1.0

score_spread.head()


I am working on the file "MNIST_2017/dump_/men_2018_0ld_logistic_script.ipynb".
The first task is:
Now lets just look at TeamID2, or just the second team info.
Can you write Python code for it?

train_test_inner = pd.merge( train.loc[ train['Season'].isin(years), : ].reset_index(drop = True),
         df_sample_sub.drop(['ID', 'Pred'], axis = 1),
         on = ['Season', 'TeamID1', 'TeamID2'], how = 'inner' )

train_test_inner.head()
train_test_inner.shape


I am working on the file "MNIST_2017/dump_/men_2018_0ld_logistic_script.ipynb".
The first task is:
Now we match the detailed results to the merge dataset above
Can you write Python code for it?

df_concat['DiffSeed'] = df_concat[['LSeed', 'WSeed']].apply(lambda x : 0 if x[0] == x[1] else 1, axis = 1)
print(df_concat.shape)
print(df_concat.head())
print(df_concat.Season.value_counts())


I am working on the file "MNIST_2017/dump_/men_2018_0ld_logistic_script.ipynb".
The first task is:
Training Data Creation
Can you write Python code for it?

winners = df_concat.rename( columns = { 'WTeamID' : 'TeamID1',
                                                       'LTeamID' : 'TeamID2',
                                                      'WScore' : 'Team1_Score',
                                                      'LScore' : 'Team2_Score'}).drop(['WSeed', 'LSeed', 'WLoc'], axis = 1)
winners['Result'] = 1.0

losers = df_concat.rename( columns = { 'WTeamID' : 'TeamID2',
                                                       'LTeamID' : 'TeamID1',
                                                      'WScore' : 'Team2_Score',
                                                      'LScore' : 'Team1_Score'}).drop(['WSeed', 'LSeed', 'WLoc'], axis = 1)

losers['Result'] = 0.0

train = pd.concat( [winners, losers], axis = 0).reset_index(drop = True)

train['Score_Ratio'] = train['Team1_Score'] / train['Team2_Score']
train['Score_Total'] = train['Team1_Score'] + train['Team2_Score']
train['Score_Pct'] = train['Team1_Score'] / train['Score_Total']
print(train.shape)
print(train.head())


I am working on the file "MNIST_2017/dump_/men_2018_0ld_logistic_script.ipynb".
The first task is:
We get the 'average' probability of success for each team
Can you write Python code for it?

team1_probs = sub.groupby('TeamID1')['Pred1'].apply(lambda x : (x ** -1.0).mean() ** -1.0 ).fillna(0.5).to_dict()
team2_probs = sub.groupby('TeamID2')['Pred1'].apply(lambda x : (x ** -1.0).mean() ** -1.0 ).fillna(0.5).to_dict()


I am working on the file "MNIST_2017/dump_/women_2018_gridsearchCV.ipynb".
The first task is:
Any missing value for the prediciton will be imputed with the product of the probabilities calculated above.  We assume these are independent events.
Can you write Python code for it?

sub['Pred'] = sub[['TeamID1', 'TeamID2','Pred1']]\
.apply(lambda x : team1_probs.get(x[0]) * ( 1 - team2_probs.get(x[1]) ) if np.isnan(x[2]) else x[2],
       axis = 1)

sub = sub.drop_duplicates(subset=["ID"], keep='first')

sub[['ID', 'Pred']].to_csv('sub.csv', index = False)

sub[['ID', 'Pred']].head(20)


I am working on the file "MNIST_2017/dump_/women_2018_gridsearchCV.ipynb".
The first task is:
Here we get our submission info
Can you write Python code for it?

#prepares sample submission
df_sample_sub = pd.read_csv('../input/WSampleSubmissionStage2.csv')

df_sample_sub['Season'] = df_sample_sub['ID'].apply(lambda x : int(x.split('_')[0]) )
df_sample_sub['TeamID1'] = df_sample_sub['ID'].apply(lambda x : int(x.split('_')[1]) )
df_sample_sub['TeamID2'] = df_sample_sub['ID'].apply(lambda x : int(x.split('_')[2]) )


I am working on the file "MNIST_2017/dump_/women_2018_gridsearchCV.ipynb".
The first task is:
Here we look at the comparable statistics.  For the TeamID2 column, we would consider the inverse of the ratio, and 1 minus the score attempt percentage.
Can you write Python code for it?

team1d_score_spread = train_test_inner.groupby(['Season', 'TeamID1'])[['Score_Ratio', 'Score_Pct']].median().reset_index()\
.set_index('Season').rename(columns = {'Score_Ratio' : 'Score_Ratio1', 'Score_Pct' : 'Score_Pct1'})
team2d_score_spread = train_test_inner.groupby(['Season', 'TeamID2'])[['Score_Ratio', 'Score_Pct']].median().reset_index()\
.set_index('Season').rename(columns = {'Score_Ratio' : 'Score_Ratio2', 'Score_Pct' : 'Score_Pct2'})

score_spread = team1d_score_spread.join(team2d_score_spread).reset_index()

#geometric mean of score ratio of team 1 and inverse of team 2
score_spread['Score_Ratio'] = score_spread[['Score_Ratio1', 'Score_Ratio2']].apply(lambda x : ( x[0] * ( x[1] ** -1.0) ), axis = 1 ) ** 0.5

#harmonic mean of score pct
score_spread['Score_Pct'] = score_spread[['Score_Pct1', 'Score_Pct2']].apply(lambda x : 0.5*( x[0] ** -1.0 ) + 0.5*( 1.0 - x[1] ) ** -1.0, axis = 1 ) ** -1.0

score_spread.head()


I am working on the file "MNIST_2017/dump_/women_2018_gridsearchCV.ipynb".
The first task is:
Here we store our probabilities
Can you write Python code for it?

train_test_inner['Pred1'] = model.predict_proba(X_train)[:,1]
train_test_missing['Pred1'] = model.predict_proba(X_test)[:,1]


I am working on the file "MNIST_2017/dump_/women_2018_gridsearchCV.ipynb".
The first task is:
Now lets create a model just solely based on the inner group and predict those probabilities.
We will get the teams with the missing result.
Can you write Python code for it?

X_train = train_test_inner.loc[:, ['Season', 'NumOT', 'Score_Ratio', 'Score_Pct']]
train_labels = train_test_inner['Result']

train_test_outer = pd.merge( train.loc[ train['Season'].isin([2014, 2015, 2016, 2017]), : ].reset_index(drop = True),
         df_sample_sub.drop(['ID', 'Pred'], axis = 1),
         on = ['Season', 'TeamID1', 'TeamID2'], how = 'outer' )

train_test_outer = train_test_outer.loc[ train_test_outer['Result'].isnull(),
                                        ['TeamID1', 'TeamID2', 'Season']]

train_test_missing = pd.merge( pd.merge( score_spread.loc[:, ['TeamID1', 'TeamID2', 'Season', 'Score_Ratio', 'Score_Pct']],
                   train_test_outer, on = ['TeamID1', 'TeamID2', 'Season']),
         num_ot.loc[:, ['TeamID1', 'TeamID2', 'Season', 'NumOT']],
         on = ['TeamID1', 'TeamID2', 'Season'])


I am working on the file "MNIST_2017/dump_/women_2018_gridsearchCV.ipynb".
The first task is:
Now we match the detailed results to the merge dataset above
Can you write Python code for it?

df_concat['DiffSeed'] = df_concat[['LSeed', 'WSeed']].apply(lambda x : 0 if x[0] == x[1] else 1, axis = 1)


I am working on the file "MNIST_2017/dump_/women_2018_gridsearchCV.ipynb".
The first task is:
Now we separate the winners from the losers and organize our dataset
Can you write Python code for it?

df_seeds['seed_int'] = df_seeds['Seed'].apply( lambda x : int(x[1:3]) )

df_winseeds = df_seeds.loc[:, ['TeamID', 'Season', 'seed_int']].rename(columns={'TeamID':'WTeamID', 'seed_int':'WSeed'})
df_lossseeds = df_seeds.loc[:, ['TeamID', 'Season', 'seed_int']].rename(columns={'TeamID':'LTeamID', 'seed_int':'LSeed'})
df_dummy = pd.merge(left=df_tour, right=df_winseeds, how='left', on=['Season', 'WTeamID'])
df_concat = pd.merge(left=df_dummy, right=df_lossseeds, on=['Season', 'LTeamID'])


I am working on the file "MNIST_2017/dump_/women_2018_gridsearchCV.ipynb".
The first task is:
We get the 'average' probability of success for each team
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_gridsearchCV.ipynb".
The first task is:
We scale our data for our keras classifier, and make sure our categorical variables are properly processed.
Can you write Python code for it?

X_test = train_test_missing.loc[:, ['Season', 'NumOT', 'Score_Ratio', 'Score_Pct']]

n = X_train.shape[0]

train_test_merge = pd.concat( [X_train, X_test], axis = 0 ).reset_index(drop = True)

train_test_merge = pd.concat( [pd.get_dummies( train_test_merge['Season'].astype(object) ),
            train_test_merge.drop('Season', axis = 1) ], axis = 1 )

train_test_merge = pd.concat( [pd.get_dummies( train_test_merge['NumOT'].astype(object) ),
            train_test_merge.drop('NumOT', axis = 1) ], axis = 1 )

X_train = train_test_merge.loc[:(n - 1), :].reset_index(drop = True)
X_test = train_test_merge.loc[n:, :].reset_index(drop = True)

x_max = X_train.max()
x_min = X_train.min()

X_train = ( X_train - x_min ) / ( x_max - x_min + 1e-14)
X_test = ( X_test - x_min ) / ( x_max - x_min + 1e-14)

train_labels.value_counts()

X_train.head()

from sklearn.linear_model import LogisticRegressionCV
model = LogisticRegressionCV(cv=80,scoring="neg_log_loss",random_state=1
                             #,penalty="l1"
                             #,Cs= Cs_#list(np.arange(1e-7,1e-9,-0.5e-9)) # [0.5,0.1,0.01,0.001] #list(np.power(1, np.arange(-10, 10)))
                             #,max_iter=1000, tol=1e-11
                             #,solver="liblinear"
                             #,n_jobs=4
                            )
model.fit(X_train, train_labels)

#---
Cs = model.Cs_
list(np.power(10.0, np.arange(-10, 10)))
dir(model)
sco = model.scores_[1].mean(axis=0)
#---
import matplotlib.pyplot as plt
plt.plot(Cs
    #np.log10(Cs)
    ,sco)
# plt.ylabel('some numbers')
plt.show()
sco.min()

Cs_= list(np.arange(1.1e-9 - 5e-11
                    ,1.051e-9
                    ,0.2e-13))
len(Cs_)

Cs_= list(np.arange(1e-11
                    ,9.04e-11#1.0508e-9
                    ,0.2e-12))
len(Cs_)

#Cs_= list(np.arange(5.6e-13             - ( (0.01e-13)*1)
#                    ,5.61e-13            - ( (0.01e-13)*1)#1.0508e-9
#                    ,0.2e-15))
#len(Cs_)

Cs_= list(np.arange(1e-11
                   ...


I am working on the file "MNIST_2017/dump_/women_2018_gridsearchCV.ipynb".
The first task is:
We will only consider years relevant to our test submission
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
Any missing value for the prediciton will be imputed with the product of the probabilities calculated above.  We assume these are independent events.
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
First we import some datasets of interest
Can you write Python code for it?

#the seed information
df_seeds = pd.read_csv('../input/WNCAATourneySeeds_SampleTourney2018.csv')

#tour information
df_tour = pd.read_csv('../input/WRegularSeasonCompactResults_PrelimData2018.csv')


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
From the inner join, we will create data per team id to estimate the parameters we are missing that are independent of the year.  Essentially, we are trying to estimate the average behavior of the team across the year.
Can you write Python code for it?

team1d_num_ot = train_test_inner.groupby(['Season', 'TeamID1'])['NumOT'].median().reset_index()\
.set_index('Season').rename(columns = {'NumOT' : 'NumOT1'})
team2d_num_ot = train_test_inner.groupby(['Season', 'TeamID2'])['NumOT'].median().reset_index()\
.set_index('Season').rename(columns = {'NumOT' : 'NumOT2'})

num_ot = team1d_num_ot.join(team2d_num_ot).reset_index()

#sum the number of ot calls and subtract by one to prevent overcounting
num_ot['NumOT'] = num_ot[['NumOT1', 'NumOT2']].apply(lambda x : round( x.sum() ), axis = 1 )

num_ot.head()


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
Here we get our submission info
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
Here we look at the comparable statistics.  For the TeamID2 column, we would consider the inverse of the ratio, and 1 minus the score attempt percentage.
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
Here we store our probabilities
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
Now lets create a model just solely based on the inner group and predict those probabilities.
We will get the teams with the missing result.
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
Now lets just look at TeamID2, or just the second team info.
Can you write Python code for it?

train_test_inner = pd.merge( train.loc[ train['Season'].isin([2018]), : ].reset_index(drop = True),
         df_sample_sub.drop(['ID', 'Pred'], axis = 1),
         on = ['Season', 'TeamID1', 'TeamID2'], how = 'inner' )

train_test_inner.head()


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
Now we match the detailed results to the merge dataset above
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
Now we separate the winners from the losers and organize our dataset
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
Training Data Creation
Can you write Python code for it?

winners = df_concat.rename( columns = { 'WTeamID' : 'TeamID1',
                                                       'LTeamID' : 'TeamID2',
                                                      'WScore' : 'Team1_Score',
                                                      'LScore' : 'Team2_Score'}).drop(['WSeed', 'LSeed', 'WLoc'], axis = 1)
winners['Result'] = 1.0

losers = df_concat.rename( columns = { 'WTeamID' : 'TeamID2',
                                                       'LTeamID' : 'TeamID1',
                                                      'WScore' : 'Team2_Score',
                                                      'LScore' : 'Team1_Score'}).drop(['WSeed', 'LSeed', 'WLoc'], axis = 1)

losers['Result'] = 0.0

train = pd.concat( [winners, losers], axis = 0).reset_index(drop = True)

train['Score_Ratio'] = train['Team1_Score'] / train['Team2_Score']
train['Score_Total'] = train['Team1_Score'] + train['Team2_Score']
train['Score_Pct'] = train['Team1_Score'] / train['Score_Total']


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
We get the 'average' probability of success for each team
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script.ipynb".
The first task is:
We will only consider years relevant to our test submission
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script_with_figures.ipynb".
The first task is:
Any missing value for the prediciton will be imputed with the product of the probabilities calculated above.  We assume these are independent events.
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script_with_figures.ipynb".
The first task is:
First we import some datasets of interest
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script_with_figures.ipynb".
The first task is:
From the inner join, we will create data per team id to estimate the parameters we are missing that are independent of the year.  Essentially, we are trying to estimate the average behavior of the team across the year.
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script_with_figures.ipynb".
The first task is:
Here we get our submission info
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script_with_figures.ipynb".
The first task is:
Here we look at the comparable statistics.  For the TeamID2 column, we would consider the inverse of the ratio, and 1 minus the score attempt percentage.
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script_with_figures.ipynb".
The first task is:
Now lets just look at TeamID2, or just the second team info.
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script_with_figures.ipynb".
The first task is:
Now we match the detailed results to the merge dataset above
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script_with_figures.ipynb".
The first task is:
Now we separate the winners from the losers and organize our dataset
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script_with_figures.ipynb".
The first task is:
We get the 'average' probability of success for each team
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script_with_figures.ipynb".
The first task is:
We scale our data for our keras classifier, and make sure our categorical variables are properly processed.
Can you write Python code for it?


I am working on the file "MNIST_2017/dump_/women_2018_manual_gradiant_script_with_figures.ipynb".
The first task is:
We will only consider years relevant to our test submission
Can you write Python code for it?


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Data import and preprocessing
All data is normalized and serialized into a vector.
Can you write Python code for it?

(x_train, y_train), (x_val, y_val) = mnist.load_data()
y_train

(x_train, y_train), (x_val, y_val) = mnist.load_data()

x_train = x_train.astype('float32') / 255.
x_val = x_val.astype('float32') / 255.

x_train = x_train.reshape((len(x_train), np.prod(x_train.shape[1:])))
x_val = x_val.reshape((len(x_val), np.prod(x_val.shape[1:])))
print(x_train.shape)

y_train = keras.utils.to_categorical(y_train, num_classes)
y_val = keras.utils.to_categorical(y_val, num_classes)


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Decoder Model:
Can you write Python code for it?

encoded_input = Input(shape=(encoding_dim,))
# retrieve the last layer of the autoencoder model
decoder_layer = single_autoencoder.layers[-1]
# create the decoder model
single_decoder = Model(encoded_input, decoder_layer(encoded_input))


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Dropout consists in randomly setting a fraction rate of input units to 0 at each update during training time, which helps prevent overfitting.
Can you write Python code for it?

input_data = Input(shape=(input_dim,), dtype='float32', name='main_input')
x = Dense(hidden1_dim, activation='relu', kernel_initializer='normal')(input_data)
x = Dropout(0.2)(x)
x = Dense(hidden2_dim, activation='relu', kernel_initializer='normal')(x)
x = Dropout(0.2)(x)
output_layer = Dense(num_classes, activation='softmax', kernel_initializer='normal')(x)

model = Model(input_data, output_layer)

model.compile(loss='categorical_crossentropy',
              optimizer=RMSprop(),
              metrics=['accuracy'])

Image("images/mnist_nn1.png")

model.fit(x_train, y_train,
           batch_size=batch_size,
           epochs=20,
           verbose=0,
           validation_split=0.1)

score = model.evaluate(x_val, y_val, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])

fig = plt.figure(figsize=(20,10))
plt.plot(model.history.history['val_acc'])
plt.plot(model.history.history['acc'])
plt.axhline(y=score[1], c="red")
plt.text(0, score[1], "test: " + str(round(score[1], 4)), fontdict=font)
plt.title('model accuracy for neural net with 2 hidden layers')
plt.ylabel('accuracy')
plt.xlabel('epochs')
plt.legend(['valid', 'train'], loc='lower right')
plt.show()


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Encoder Model:
Can you write Python code for it?

# this model maps an input to its encoded representation
single_encoder = Model(input_img, encoded)


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
First, we'll configure our model to use a per-pixel binary crossentropy loss, and the RMSprop optimizer:
Binary Cross Entropy = Binomial Cross Entropy = Special Case of Multinomial Cross Entropy
Can you write Python code for it?


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Global Variables
The number of classes, the input dimension and the batch size are constant.
Can you write Python code for it?

num_classes = 10
input_dim = 784
batch_size = 256


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Save the model
Can you write Python code for it?

# stacked_autoencoder.save('models/stacked_autoencoder.h5')

score = stacked_autoencoder.evaluate(x_val, x_val, verbose=0)
print(score)


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Single autoencoder
Model Definitions
Using keras module with compression to 32 floats.
Can you write Python code for it?

encoding_dim = 32

# input placeholder
input_img = Input(shape=(input_dim,))
encoded = Dense(encoding_dim, activation='relu')(input_img)
decoded = Dense(input_dim, activation='sigmoid')(encoded)

single_autoencoder = Model(input_img, decoded)


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Stacked Autoencoder
Can you write Python code for it?

######## constants for stacked autoencoder ############
encoding_dim1 = 128
encoding_dim2 = 64
encoding_dim3 = 32
decoding_dim1 = 64
decoding_dim2 = 128
decoding_dim3 = input_dim
epochs = 100
batch_size = 256

input_img = Input(shape=(input_dim,))
encoded = Dense(encoding_dim1, activation='relu')(input_img)
encoded = Dense(encoding_dim2, activation='relu')(encoded)
encoded = Dense(encoding_dim3, activation='relu')(encoded)

decoded = Dense(decoding_dim1, activation='relu')(encoded)
decoded = Dense(decoding_dim2, activation='relu')(decoded)
decoded = Dense(decoding_dim3, activation='sigmoid')(decoded)

stacked_autoencoder = Model(input_img, decoded)
stacked_autoencoder.compile(optimizer=RMSprop(), loss='binary_crossentropy')

stacked_autoencoder = keras.models.load_model('models/stacked_autoencoder.h5')
#stacked_autoencoder.fit(x_train, x_train,
#                 epochs=epochs,
#                 batch_size=batch_size,
#                 shuffle=False,
#                 validation_split=0.1)


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Test the Sparse Autoencoder with a simple Neural Net (not for presentation)
Can you write Python code for it?

sparse_ae_test_model = Sequential()
# single_ae_test_model.add(Dense(16, activation='relu', input_shape=(36,)))
# single_ae_test_model.add(Dropout(0.2))
sparse_ae_test_model.add(Dense(num_classes, activation='softmax', input_shape=(encoding_dim,)))

sparse_ae_test_model.compile(loss='categorical_crossentropy',
                             optimizer=RMSprop(),
                             metrics=['accuracy'])

# sparse_ae_test_model = keras.models.load_model('models/sparse_ae_test_model.h5')
sparse_ae_test_model.fit(sparse_encoder.predict(x_train), y_train,
                         batch_size=batch_size,
                         epochs=40,
                         verbose=1,
                         validation_data=(sparse_encoder.predict(x_val), y_val))

sparse_ae_test_model.save('models/sparse_ae_test_model.h5')

score = sparse_ae_test_model.evaluate(sparse_encoder.predict(x_val), y_val, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Test the Vanilla Autoencoder with a simple Neural Net (not for presentation)
Can you write Python code for it?

single_ae_test_model = Sequential()
# single_ae_test_model.add(Dense(16, activation='relu', input_shape=(36,)))
# single_ae_test_model.add(Dropout(0.2))
single_ae_test_model.add(Dense(num_classes, activation='softmax', input_shape=(encoding_dim,)))

single_ae_test_model.compile(loss='categorical_crossentropy',
                             optimizer=RMSprop(),
                             metrics=['accuracy'])

single_ae_test_model = keras.models.load_model('models/single_ae_test_model.h5')
# single_ae_test_model.fit(single_encoder.predict(x_train), y_train,
#                          batch_size=batch_size,
#                          epochs=40,
#                          verbose=1,
#                          validation_data=(single_encoder.predict(x_val), y_val))

single_ae_test_model.save('models/single_ae_test_model.h5')

score = single_ae_test_model.evaluate(single_encoder.predict(x_val), y_val, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
The difference in accuracy is huge. The autoencoder architecture is highly recommendable for denoising signals.
Train with sparsity constraint on hidden layer (not for presentation)
So what does all this mean for sparsity? How does the rectifier activation function allow for sparsity in the hidden units? If the hidden units are exposed to a range of input values it makes sense that the rectifier activation function should lead to more ‘real zeros’ as we sweep across possible inputs. In other words, less neurons in our network would activate because of the limitations imposed by the rectifier activation function. https://www.quora.com/What-does-it-mean-that-activation-functions-like-ReLUs-in-NNs-induce-sparsity-in-the-hidden-units
Can you write Python code for it?

input_img = Input(shape=(input_dim,))
# add a Dense layer with a L1 activity regularizer
encoded = Dense(encoding_dim, activation='relu',
                activity_regularizer=regularizers.l1(10e-5))(input_img)
decoded = Dense(input_dim, activation='sigmoid')(encoded)

sparse_autoencoder = Model(input_img, decoded)

sparse_encoder = Model(input_img, encoded)

encoded_input = Input(shape=(encoding_dim,))
# retrieve the last layer of the autoencoder model
decoder_layer = sparse_autoencoder.layers[-1]
# create the decoder model
sparse_decoder = Model(encoded_input, decoder_layer(encoded_input))


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
The result is slightly better than for single autoencoder.
Can you write Python code for it?

# plot_model(stacked_autoencoder, to_file='images/stacked_autoencoderTD.png', show_shapes=True, show_layer_names=True, rankdir='LR')

Image("images/stacked_autoencoderTD.png")

decoded_imgs = stacked_autoencoder.predict(x_val)

n = 10  # how many digits we will display
plt.figure(figsize=(20, 4))
for i in range(n):
    # display original
    ax = plt.subplot(2, n, i + 1)
    plt.imshow(x_val[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)

    # display reconstruction
    ax = plt.subplot(2, n, i + 1 + n)
    plt.imshow(decoded_imgs[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Train Neural Net to recognize MNIST digits
Can you write Python code for it?

# constants
hidden1_dim = 512
hidden2_dim = 512


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Train or load a stacked denoising autoencoder
Can you write Python code for it?

# denoising_autoencoder = keras.models.load_model('models/denoising_autoencoder.h5')
denoising_autoencoder.fit(x_train_noisy, x_train,
                          epochs=epochs,
                          batch_size=batch_size,
                          shuffle=True,
                          validation_split=0.1,
                          verbose=0,
                          callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])

fig = plt.figure(figsize=(20,10))
plt.plot(denoising_autoencoder.history.history['val_loss'])
plt.plot(denoising_autoencoder.history.history['loss'])
plt.title('model loss for stacked denoising autoencoder')
plt.ylabel('loss')
plt.xlabel('epochs')
plt.legend(['valid', 'train'], loc='lower right')
plt.show()

decoded_imgs = denoising_autoencoder.predict(x_val_noisy)

n = 10  # how many digits we will display
plt.figure(figsize=(20, 4))
for i in range(n):
    # display original
    ax = plt.subplot(2, n, i + 1)
    plt.imshow(x_val[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)

    # display reconstruction
    ax = plt.subplot(2, n, i + 1 + n)
    plt.imshow(decoded_imgs[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()


I am working on the file "MNIST_Autoencoder.ipynb".
The first task is:
Train/load model
Can you write Python code for it?

# single_autoencoder = keras.models.load_model('models/single_autoencoder.h5')
single_autoencoder.fit(x_train, x_train,
                       epochs=50,
                       batch_size=batch_size,
                       shuffle=False,
                       verbose=0,
                       validation_split=0.1,
                       callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])

# single_autoencoder.save('models/single_autoencoder.h5')

score = single_autoencoder.evaluate(x_val, x_val, verbose=0)
print(score)

# plot_model(single_autoencoder, to_file='images/single_autoencoder.png', show_shapes=True, show_layer_names=True, rankdir='LR')

Image("images/single_autoencoder.png")


I am working on the file "MNIST_Autoencoder_dim_reduction.ipynb".
The first task is:
Neural Net Classifier with Single Autoencoder (1 hidden layer)
Can you write Python code for it?

main_input = Input(shape=(input_dim,), dtype='float32', name='main_input')

encoding_layer = Dense(encoder_dim, activation='relu', kernel_initializer='normal')
encoding_layer_output = encoding_layer(main_input)
decoding_layer_output = Dense(input_dim
                              ,activation='sigmoid'
                              ,name='decoder_output'
                              ,kernel_initializer='normal')(encoding_layer_output)

classifier_output = Dense(num_classes
                          ,activation='softmax'
                          ,name='main_output'
                          ,kernel_initializer='normal')(encoding_layer_output)

auto_classifier = Model(inputs=main_input, outputs=[classifier_output, decoding_layer_output])

auto_classifier.compile(optimizer=RMSprop(),
                        loss={'main_output': 'categorical_crossentropy', 'decoder_output': 'mean_squared_error'},
                        loss_weights={'main_output':1., 'decoder_output': 1.},
                        metrics=['accuracy'])

plot_model(auto_classifier, to_file='images/mnist_dim_reduction.png', show_shapes=True, show_layer_names=True, rankdir='LR')

Image("images/mnist_dim_reduction.png")

auto_classifier.fit({'main_input': x_train}
                    ,{'main_output': y_train, 'decoder_output': x_train}
                    ,epochs=epochsize
                    ,batch_size=batchsize
                    ,shuffle=True
                    ,validation_split=0.05
                    ,verbose=0
                    ,callbacks=[TensorBoard(log_dir='./logs')])

score = auto_classifier.evaluate(x=x_val, y=[y_val, x_val], verbose=1)[3]
print(score)

fig = plt.figure(figsize=(20,10))
plt.plot(auto_classifier.history.history['val_main_output_acc'])
plt.plot(auto_classifier.history.history['main_output_acc'])
plt.axhline(y=score, c="red")
plt.text(0, score, "test: " + str(round(score, 4)), fontdict=font)
plt.title('model accuracy for ' + str(round(input_dim / encoder_dim, 2)) + ' x...


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
Compute the accuracy of predictions on the validation and test sets.
Can you write Python code for it?

XV = mnist.validation.images
YV = mnist.validation.labels
labelsV = np.argmax(YV,axis=1)
predsV = Predictions(XV, W, b)
accuracyV = Accuracy(predsV, labelsV)
print("validation accuracy =", accuracyV)

XT = mnist.test.images
YT = mnist.test.labels
labelsT = np.argmax(YT,axis=1)
predsT = Predictions(XT, W, b)
accuracyT = Accuracy(predsT, labelsT)
print("Test accuracy =", accuracyT)


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
InitWeights sets all the weights to zero. Running Predictions with these values should predict everything to
the "0" class. If the images are uniformly distributed over the 10 classes, we should see an accuracy value of 0.1.
Here we get a value of 0.0989 which is pretty close.
Can you write Python code for it?

preds = Predictions(X, W, b)
accuracy = Accuracy(preds, labels)
print("accuracy =", accuracy)
for i in range(10):
    print(preds[10000+i],labels[10000 + i])


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
Run a test to verify that we have the correct shape for W and b and that softmax is being computed correctly.
Each row of softmax needs to add upto 1 and since W,b are initialized to all zeros, all the values should be the same.
Can you write Python code for it?

X, Y, labels = InitExample()

print("X:", X)
print("X.shape:", X.shape)
print("Y:", Y)
print("Y.shape:", Y.shape)
print("labels:", labels)
print("labels.shape:", labels.shape)
m, n = X.shape
C = Y.shape[1]
print("n =", n, "m =", m, "C =", C)

W, b = InitWeights(X, Y)
print("W:", W)
print("W.shape:", W.shape)
print("b:", b)
print("b.shape:", b.shape)
softmax(score(X, W, b))


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
The Accuracy function takes two vectors of class labels and computes the accuracy or in other words how many of them match.
Can you write Python code for it?

def Accuracy(preds, labels):
    accuracy = sum(preds == labels)/(float(len(labels)))
    return accuracy


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
The ComputeCost function takes W,X,Y,b as inputs and returns the current softmax probabilities, cost and gradients for W and b.
Can you write Python code for it?

def ComputeCost(W, X, Y, b):
    m = X.shape[0]
    grads = {}
    prob = softmax(score(X, W, b))
    cost = (-1 / m) * np.sum(Y * np.log(prob))
    dW = (-1 / m) * np.dot(X.T,(Y - prob))
    db = (-1 / m) * np.sum(Y - prob, axis=0)
    grads['dW'] = dW
    grads['db'] = db
    return prob, cost, grads
prob, cost, grads = ComputeCost(W, X, Y, b)
print("cost:", cost)
print("prob:", prob)
print("dW:", grads['dW'])
print("db:", grads['db'])


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
The Predictions function takes X, W and b as inputs and outputs a vector of predicted class labels
Can you write Python code for it?

def Predictions(X, W, b ):
    probs = softmax(score(X, W, b))
    predictions = np.argmax(probs,axis=1)
    return predictions


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
The TrainModels takes as input X, Y, labels, number of iterations and learning rate. Y is a one hot encoding of the labels vector. The function also takes as input a flag "verbose" whic if set to true will print intermediate results.
The function computes the cost and gradients in each iteration and updates W and b.
It outputs W, b, cost history and accuracy. The cost history consists of 50 samples over the iterations and is useful for plotting and to select the optimal learning rate.
Can you write Python code for it?

def TrainModel(X, Y, labels, iterations=500, learningRate=1e-2, verbose=False):
    W, b = InitWeights(X, Y)
    costHistory = []
    start = time.time()
    for i in range(0,iterations):
        prob, cost, grads = ComputeCost(W, X, Y, b)
        W,b = UpdateWeights(W, b, grads, learningRate)
        if (i) % (iterations/50) == 0:
            costHistory.append(cost)
            if verbose:
                print("iteration:", i)
                print("cost =", cost)
                preds = Predictions(X, W, b)
                accuracy = Accuracy(preds, labels)
                print("accuracy =", accuracy)
                print("time elapsed =", time.time() - start)
    preds = Predictions(X, W, b)
    accuracy = Accuracy(preds, labels)
    return W, b, costHistory, accuracy


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
The UpdateWeights functions updates W and b based on the gradients and learning rate.
Can you write Python code for it?

def UpdateWeights(W, b, grads, learningRate):
    W = W - (learningRate * grads['dW'])
    b = b - (learningRate * grads['db'])
    return W,b


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
The following two functions compute the score and softmax given X, W and b.
Can you write Python code for it?

def score(X, W, b):
    return np.dot(X, W) + b

def softmax(scores):
    exp_scores = np.exp(scores)
    s = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
    return s


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
The function InitWeights takes the X and Y matrices as inputs and initializes W and b based on their dimensions.
Can you write Python code for it?

def InitWeights(X, Y):
    m, n = X.shape
    C = Y.shape[1]
    assert m == Y.shape[0], "X and Y do not have the same number of samples"
    W=np.zeros([n, C])
    b=np.zeros([1, C])
    return W, b


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
The plot shows that with a learning rate of 10, the cost does not go down monotonically and keep oscillating which points to it being too high. Learning rates of 0.1 and 1 show similar behaviour with 1 slight better in terms of lower cost. So we will use that for running the model for more epochs till we get good accuracy.
Can you write Python code for it?

for ch in allCostHistory:
    plt.plot(allCostHistory[ch], label=ch)
plt.legend()
plt.show()


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
The tensorflow data loader splits the 60000 images in the MNIST training set into a 55000+5000 examples where the set of 5000 is used as a validation set.
Can you write Python code for it?

print(mnist.train.num_examples)
print(mnist.validation.num_examples)
print(mnist.test.num_examples)

X = mnist.train.images
Y = mnist.train.labels
labels = np.argmax(Y,axis=1)
print(X.shape,Y.shape,labels.shape)

W, b = InitWeights(X, Y)
print(W.shape,b.shape)


I am working on the file "MNIST_NP_Softmax_Basic.ipynb".
The first task is:
Training example 30026 was incorrectly classified as a "4" instead of a "6". Looking at the image shows why the classifier might have gotten it wrong. Example 30027 which was correctly classied as "6" is much easier to classify.
Can you write Python code for it?


I am working on the file "MNIST_TF_CNN_Tutorial.ipynb".
The first task is:
Convolution and Pooling operations
Can you write Python code for it?

def conv2d(x, W):
    return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')

def max_pool_2x2(x):
    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
                        strides=[1, 2, 2, 1], padding='SAME')


I am working on the file "MNIST_TF_CNN_Tutorial.ipynb".
The first task is:
Densely Connected Layer The image size has been reduced to 7x7 and we add a fully-connected layer with 1024 neurons to allow processing on the entire image.
Can you write Python code for it?

W_fc1 = weight_variable([7 * 7 * 64, 1024])
b_fc1 = bias_variable([1024])

h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)


I am working on the file "MNIST_TF_CNN_Tutorial.ipynb".
The first task is:
Dropout
Can you write Python code for it?

keep_prob = tf.placeholder(tf.float32)
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)


I am working on the file "MNIST_TF_CNN_Tutorial.ipynb".
The first task is:
Implement Model
Can you write Python code for it?

cross_entropy = tf.reduce_mean(
    tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y_conv))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))


I am working on the file "MNIST_TF_CNN_Tutorial.ipynb".
The first task is:
Import MNIST Data
Can you write Python code for it?

mnist = input_data.read_data_sets("../datasets/MNIST/", one_hot=True)


I am working on the file "MNIST_TF_CNN_Tutorial.ipynb".
The first task is:
Second Convolutional Layer The second layer will have 64 features for each 5x5 patch.
Can you write Python code for it?

W_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable([64])

h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)


I am working on the file "MNIST_TF_Softmax_Tutorial.ipynb".
The first task is:
Define function that runs the model for given number of batches and returns the training time and accuracy on the training, validation and test data sets
Can you write Python code for it?

def train_and_test_model(batches, verbose=False):
    start = time.time()
    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        for _ in range(batches):
            batch_xs, batch_ys = mnist.train.next_batch(100)
            sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})
        train_time = time.time() - start
        train_accuracy = sess.run(accuracy, feed_dict={x: mnist.train.images, y_: mnist.train.labels})
        validation_accuracy = sess.run(accuracy, feed_dict={x: mnist.validation.images, y_: mnist.validation.labels})
        test_accuracy = sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels})
        if verbose:
            print(batches, train_time, train_accuracy, validation_accuracy, test_accuracy)
        return (train_time, train_accuracy, validation_accuracy, test_accuracy)


I am working on the file "MNIST_TF_Softmax_Tutorial.ipynb".
The first task is:
Import MNIST Data
Can you write Python code for it?


I am working on the file "MNIST_TF_Softmax_Tutorial.ipynb".
The first task is:
Look at sizes of training, validation and test sets
Each image is 28 X 28 pixels
Labels are in one hot encoding for use with softmax
Can you write Python code for it?

print(mnist.train.num_examples)
print(mnist.validation.num_examples)
print(mnist.test.num_examples)
plt.imshow(mnist.train.images[30].reshape(28,28),cmap="Greys")
plt.show()
print (mnist.train.labels[30])


I am working on the file "MNIST_TF_Softmax_Tutorial.ipynb".
The first task is:
The tutorial runs a 1000 batches each of size 100 which is about 2 epochs with 55000 test images. Lets run the model for 1, 2 and 5 epochs and see how accuracy changes with longer training time and also record how long it takes to train.
Can you write Python code for it?

one_epoch = train_and_test_model(550,verbose=True)
two_epoch = train_and_test_model(1100,verbose=True)
five_epoch = train_and_test_model(2750,verbose=True)
ten_epoch = train_and_test_model(5500,verbose=True)
twenty_epoch = train_and_test_model(11000,verbose=True)
twentyfive_epoch = train_and_test_model(13750,verbose=True)
print(one_epoch,two_epoch,five_epoch,ten_epoch,twenty_epoch,twentyfive_epoch)


I am working on the file "MNIST_for_beginners_noNN_noCONV_0.11.0rc2.ipynb".
The first task is:
TensorBoard: Visualizing Learning
Can you write Python code for it?

def variable_summaries(var):
  """Attach a lot of summaries to a Tensor (for TensorBoard visualization)."""
  with tf.name_scope('summaries'):
    mean = tf.reduce_mean(var)
    #tf.summary.scalar('mean', mean)
    tf.scalar_summary(var.name+'_mean', mean)
    with tf.name_scope('stddev'):
      stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))
    #tf.summary.scalar('stddev', stddev)
    tf.scalar_summary(var.name+'_stddev', stddev)
    #tf.summary.scalar('max', tf.reduce_max(var))
    tf.scalar_summary(var.name+'_max', tf.reduce_max(var))
    #tf.summary.scalar('min', tf.reduce_min(var))
    tf.scalar_summary(var.name+'_min', tf.reduce_min(var))
    tf.histogram_summary( var.name, var)

def main2(_):
  # Import data
  mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)

  # Create the model
  x = tf.placeholder(tf.float32, [None, 784])

  with tf.name_scope('W'):
    W = tf.Variable(tf.zeros([784, 10]))
    variable_summaries(W)

  with tf.name_scope('b'):
    b = tf.Variable(tf.zeros([10]))
    variable_summaries(b)

  with tf.name_scope('y'):
    y = tf.matmul(x, W) + b
    variable_summaries(y)

  # Define loss and optimizer
  y_ = tf.placeholder(tf.float32, [None, 10])

  with tf.name_scope('cross_entropy'):
    cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(y, y_))
  #tf.summary.scalar('cross_entropy', cross_entropy)
  tf.scalar_summary('cross_entropy', cross_entropy)

  with tf.name_scope('train_step'):
    train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

  # Test trained model
  with tf.name_scope('accuracy'):
    with tf.name_scope('correct_prediction'):
      correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))

    with tf.name_scope('accuracy'):
      accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
     ...


I am working on the file "MNIST_for_beginners_noNN_noCONV_0.11.0rc2.ipynb".
The first task is:
The current state of the art in classifying these digits can be found here: http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html#4d4e495354
Model
Can you write Python code for it?

def main(_):
  # Import data
  mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)

  # Create the model
  x = tf.placeholder(tf.float32, [None, 784])
  W = tf.Variable(tf.zeros([784, 10]))
  b = tf.Variable(tf.zeros([10]))
  y = tf.matmul(x, W) + b

  # Define loss and optimizer
  y_ = tf.placeholder(tf.float32, [None, 10])

  cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(y, y_))
  train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

  #sess = tf.InteractiveSession()
  with tf.Session() as sess:
      #tf.global_variables_initializer().run()
      init = tf.initialize_all_variables()
      sess.run(init)
      # Train
      for _ in range(1000):
        batch_xs, batch_ys = mnist.train.next_batch(100)
        sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

      # Test trained model
      correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
      accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
      print(">>> Test Accuracy::"+str(sess.run(accuracy, feed_dict={x: mnist.test.images, y_:...


I am working on the file "MNIST_for_beginners_noNN_noCONV_0.12.0-rc1.ipynb".
The first task is:
TensorBoard: Visualizing Learning
Can you write Python code for it?

from tensorflow.contrib.tensorboard.plugins import projector

def variable_summaries(var):
  """Attach a lot of summaries to a Tensor (for TensorBoard visualization)."""
  with tf.name_scope('summaries'):
    mean = tf.reduce_mean(var)
    tf.summary.scalar(var.name+'_mean', mean)
    #tf.scalar_summary(var.name+'_mean', mean)
    stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))
    tf.summary.scalar(var.name+'_stddev', stddev)
    #tf.scalar_summary(var.name+'_stddev', stddev)
    tf.summary.scalar(var.name+'_max', tf.reduce_max(var))
    #tf.scalar_summary(var.name+'_max', tf.reduce_max(var))
    tf.summary.scalar(var.name+'_min', tf.reduce_min(var))
    #tf.histogram_summary( var.name, var)
    tf.summary.histogram( var.name, var)

def main2(_):
  # Import data
  mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)

  #config = tf.contrib.tensorboard.plugins.projector.ProjectorConfig()

  # input images
  with tf.name_scope('input'):
    # None -> batch size can be any size, 784 -> flattened mnist image
    x = tf.placeholder(tf.float32, shape=[None, 784], name="x-input")

    xs = tf.Variable(tf.zeros([batch_size, 784]) , name="x-input-slice1")
    xs = tf.slice(x, [0, 0], [batch_size, 784] , name="x-input-slice2")

    variable_summaries(xs)

    #emb1 = config.embeddings.add()
    #emb1.tensor_name = xs.name
    #emb1.metadata_path = os.path.join(FLAGS.data_dir + '/_logs', 'metadata.tsv')

    # target 10 output classes
    y_ = tf.placeholder(tf.float32, shape=[None, 10], name="y-input")
    #variable_summaries(y_)

  with tf.name_scope('input_image'):
    image_shaped_input = tf.reshape(x, [-1, 28, 28, 1])
    tf.summary.image('image', image_shaped_input, 10)

  with tf.name_scope('W'):
    W = tf.Variable(tf.zeros([784, 10]))
    variable_summaries(W)

  with tf.name_scope('b'):
    b =...


I am working on the file "MNIST_for_beginners_noNN_noCONV_0.12.0-rc1.ipynb".
The first task is:
The current state of the art in classifying these digits can be found here: http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html#4d4e495354
Model
Can you write Python code for it?

def main(_):
  # Import data
  mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)

  # Create the model
  x = tf.placeholder(tf.float32, [None, 784])
  W = tf.Variable(tf.zeros([784, 10]))
  b = tf.Variable(tf.zeros([10]))
  y = tf.matmul(x, W) + b

  # Define loss and optimizer
  y_ = tf.placeholder(tf.float32, [None, 10])

  cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(y, y_))
  train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

  #sess = tf.InteractiveSession()
  with tf.Session() as sess:
      tf.global_variables_initializer().run()
      #init = tf.initialize_all_variables()
      #sess.run(init)
      # Train
      for _ in range(iterations):
        batch_xs, batch_ys = mnist.train.next_batch(batch_size)
        sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

      # Test trained model
      correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
      accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
      print(">>> Test Accuracy::"+str(sess.run(accuracy, feed_dict={x: mnist.test.images, y_:...


I am working on the file "MNIST_for_beginners_noNN_noCONV_0.12.0-rc1.ipynb".
The first task is:
Visualizing data
Can you write Python code for it?

mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
import matplotlib.pyplot as plt
%matplotlib inline

plt.imshow(batch_xs[0].reshape(28, 28))

batch_ys[0]

plt.imshow(batch_xs[10].reshape(28, 28))

batch_ys[10]

plt.imshow(batch_xs[60].reshape(28, 28))

batch_ys[60]


I am working on the file "MNO/proyecto_final/MNO_2017/proyectos/equipos/equipo_6/avance_22_05_2017/code/Clase_SVD_Imagen.ipynb".
The first task is:
Imprimimos resultados de las matrices $U$ $\Sigma$ $Vt$ :
Can you write Python code for it?

print("U:")
print(U)
print("sigma:")
print(sigma)
print("Vt:")
print(Vt)

#TOTAL DE bytes DEL ARREGLO (solo sigma)
sigma.nbytes


I am working on the file "MNO/proyecto_final/MNO_2017/proyectos/equipos/equipo_6/avance_22_05_2017/code/Clase_SVD_Imagen.ipynb".
The first task is:
Reconstruccion de matriz original:
Can you write Python code for it?

np.dot(U, np.dot(S, Vt)) #se usa Vt

imgmatriz


I am working on the file "MNO/proyecto_final/MNO_2017/proyectos/equipos/equipo_6/avance_22_05_2017/code/Clase_SVD_Imagen.ipynb".
The first task is:
Se computa $SVD$ :
Can you write Python code for it?

#SE HACE LA DESCOMPOSICION DE VALORES SINGULARES
U, sigma, Vt = np.linalg.svd(imgmatriz)


I am working on the file "MNO/proyecto_final/MNO_2017/proyectos/equipos/equipo_6/avance_22_05_2017/code/Clase_SVD_Imagen.ipynb".
The first task is:
Visualizamos la $/Sigma$ en una matriz diagonal:
Can you write Python code for it?

S = np.zeros(imgmatriz.shape, "float")
S[:min(imgmatriz.shape), :min(imgmatriz.shape)] = np.diag(sigma)
print(S)


I am working on the file "MODEL.ipynb".
The first task is:
Stage 3: Optimize the model

Apply Transfer Learning method
Can you write Python code for it?

# Test program
%matplotlib inline
%load_ext autoreload
%autoreload 2
from test_program import test_program

test_program()


I am working on the file "MOOC/stanford_cnn_cs231n/assignment1/.ipynb_checkpoints/features-checkpoint.ipynb".
The first task is:
Load data
Similar to previous exercises, we will load CIFAR-10 data from disk.
Can you write Python code for it?

from cs231n.features import color_histogram_hsv, hog_feature

def get_CIFAR10_data(num_training=49000, num_validation=1000, num_test=1000):
    # Load the raw CIFAR-10 data
    cifar10_dir = 'cs231n/datasets/cifar-10-batches-py'
    X_train, y_train, X_test, y_test = load_CIFAR10(cifar10_dir)

    # Subsample the data
    mask = list(range(num_training, num_training + num_validation))
    X_val = X_train[mask]
    y_val = y_train[mask]
    mask = list(range(num_training))
    X_train = X_train[mask]
    y_train = y_train[mask]
    mask = list(range(num_test))
    X_test = X_test[mask]
    y_test = y_test[mask]

    return X_train, y_train, X_val, y_val, X_test, y_test

X_train, y_train, X_val, y_val, X_test, y_test = get_CIFAR10_data()


I am working on the file "MOOC/stanford_cnn_cs231n/assignment1/.ipynb_checkpoints/features-checkpoint.ipynb".
The first task is:
Train SVM on features
Using the multiclass SVM code developed earlier in the assignment, train SVMs on top of the features extracted above; this should achieve better results than training SVMs directly on top of raw pixels.
Can you write Python code for it?

# Use the validation set to tune the learning rate and regularization strength

from cs231n.classifiers.linear_classifier import LinearSVM

learning_rates = [1e-9, 1e-8, 1e-7]
regularization_strengths = [5e4, 5e5, 5e6]

results = {}
best_val = -1
best_svm = None

pass
################################################################################
# TODO:                                                                        #
# Use the validation set to set the learning rate and regularization strength. #
# This should be identical to the validation that you did for the SVM; save    #
# the best trained classifer in best_svm. You might also want to play          #
# with different numbers of bins in the color histogram. If you are careful    #
# you should be able to get accuracy of near 0.44 on the validation set.       #
################################################################################
pass
################################################################################
#                              END OF YOUR CODE                                #
################################################################################

# Print out results.
for lr, reg in sorted(results):
    train_accuracy, val_accuracy = results[(lr, reg)]
    print('lr %e reg %e train accuracy: %f val accuracy: %f' % (
                lr, reg, train_accuracy, val_accuracy))

print('best validation accuracy achieved during cross-validation: %f' % best_val)

# Evaluate your trained SVM on the test set
y_test_pred = best_svm.predict(X_test_feats)
test_accuracy = np.mean(y_test == y_test_pred)
print(test_accuracy)

# An important way to gain intuition about how an algorithm works is to
# visualize the mistakes that it makes. In this visualization, we show examples
# of images that are misclassified by our current system. The first column
# shows images that our system labeled as "plane" but whose true label is
# something other than "plane".

examples_per_class = 8
classes = ['plane', 'car', 'bird', 'cat',...


I am working on the file "MOOC/stanford_cnn_cs231n/assignment1/.ipynb_checkpoints/knn-checkpoint.ipynb".
The first task is:
Cross-validation
We have implemented the k-Nearest Neighbor classifier but we set the value k = 5 arbitrarily. We will now determine the best value of this hyperparameter with cross-validation.
Can you write Python code for it?

num_folds = 5
k_choices = [1, 3, 5, 8, 10, 12, 15, 20, 50, 100]

X_train_folds = []
y_train_folds = []
################################################################################
# TODO:                                                                        #
# Split up the training data into folds. After splitting, X_train_folds and    #
# y_train_folds should each be lists of length num_folds, where                #
# y_train_folds[i] is the label vector for the points in X_train_folds[i].     #
# Hint: Look up the numpy array_split function.                                #
################################################################################
X_train_folds = np.array_split(X_train, num_folds)
y_train_folds = np.array_split(y_train, num_folds)
################################################################################
#                                 END OF YOUR CODE                             #
################################################################################

# A dictionary holding the accuracies for different values of k that we find
# when running cross-validation. After running cross-validation,
# k_to_accuracies[k] should be a list of length num_folds giving the different
# accuracy values that we found when using that value of k.
k_to_accuracies = {}

################################################################################
# TODO:                                                                        #
# Perform k-fold cross validation to find the best value of k. For each        #
# possible value of k, run the k-nearest-neighbor algorithm num_folds times,   #
# where in each case you use all but one of the folds as training data and the #
# last fold as a validation set. Store the accuracies for all fold and all     #
# values of k in the k_to_accuracies dictionary.                               #
################################################################################
for k in k_choices:
    k_to_accuracies[k] = []
    for train_id in range(0,...


I am working on the file "MOOC/stanford_cnn_cs231n/assignment1/.ipynb_checkpoints/knn-checkpoint.ipynb".
The first task is:
Inline Question #1: Notice the structured patterns in the distance matrix, where some rows or columns are visible brighter. (Note that with the default color scheme black indicates low distances while white indicates high distances.)

What in the data is the cause behind the distinctly bright rows?
What causes the columns?

Your Answer: fill this in.
Can you write Python code for it?

# Now implement the function predict_labels and run the code below:
# We use k = 1 (which is Nearest Neighbor).
y_test_pred = classifier.predict_labels(dists, k=1)

# Compute and print the fraction of correctly predicted examples
num_correct = np.sum(y_test_pred == y_test)
accuracy = float(num_correct) / num_test
print('Got %d / %d correct => accuracy: %f' % (num_correct, num_test, accuracy))