I am working on the file "LAB4/recursos/practica4/practica_svm.ipynb".
The first task is:
1.1. Kernel lineal
Completa el código de la función linear_kernel y comprueba tu solución.
Can you write Python code for it?

# Con tu implementación:
K = svm.linear_kernel(x, y)
print "El array K deberia tener dimensiones: (%d, %d)" % (n, m)
print "A ti te sale un array con dimensiones:", K.shape

# Con la implementación de sklearn:
K_ = polynomial_kernel(x, y, degree=1, gamma=1, coef0=1)

# Diferencia entre tu kernel y el de sklearn (deberia salir practicamente 0):
maxdif = np.max(np.abs(K - K_))
print "Maxima diferencia entre tu implementacion y la de sklearn:", maxdif


I am working on the file "LAB4/recursos/practica4/practica_svm.ipynb".
The first task is:
1.2. Kernel polinómico
Completa el código de la función poly_kernel y comprueba tu solución.
Can you write Python code for it?

# Con tu implementación:
K = svm.poly_kernel(x, y, deg=2, b=1)
print "El array K deberia tener dimensiones: (%d, %d)" % (n, m)
print "A ti te sale un array con dimensiones:", K.shape

# Con la implementación de sklearn:
K_ = polynomial_kernel(x, y, degree=2, gamma=1, coef0=1)

# Diferencia entre tu kernel y el de sklearn (deberia salir practicamente 0):
maxdif = np.max(np.abs(K - K_))
print "Maxima diferencia entre tu implementacion y la de sklearn:", maxdif


I am working on the file "LAB4/recursos/practica4/practica_svm.ipynb".
The first task is:
1.3. Kernel gausiano
Completa el código de la función rbf_kernel y comprueba tu solución.
Can you write Python code for it?

s = 1.0

# Con tu implementación:
K = svm.rbf_kernel(x, y, sigma=s)
print "El array K deberia tener dimensiones: (%d, %d)" % (n, m)
print "A ti te sale un array con dimensiones:", K.shape

# Con la implementación de sklearn:
K_ = rbf_kernel(x, y, gamma=1/(2*s**2))

# Diferencia entre tu kernel y el de sklearn (deberia salir practicamente 0):
maxdif = np.max(np.abs(K - K_))
print "Maxima diferencia entre tu implementacion y la de sklearn:", maxdif


I am working on the file "LAB4/recursos/practica4/practica_svm.ipynb".
The first task is:
2. El algoritmo SMO
A continuación vas a completar la clase SVM, que representa un clasificador basado en máquinas de vectores de soporte, y vas a implementar el algoritmo SMO dentro de esta clase.
2.1. Implementa el método evaluate_model
Lo primero que debes hacer es completar el método evaluate_model, que recibe un array con los datos (atributos) del problema $x$ y calcula $f(x)$. Una vez que lo tengas, para comprobar que funciona bien, puedes ejecutar la siguiente celda de código:
Can you write Python code for it?

# Datos de prueba (problema XOR):
x = np.array([[-1, -1], [1, 1], [1, -1], [-1, 1]])
y = np.array([1, 1, -1, -1])

# Alphas y b:
alpha = np.array([0.125, 0.125, 0.125, 0.125])
b = 0

# Clasificador, introducimos la solucion a mano:
svc = svm.SVM(C=1000, kernel="poly", sigma=1, deg=2, b=1)
svc.init_model(alpha, b, x, y)

# Clasificamos los puntos x:
y_ = svc.evaluate_model(x)

# Las predicciones deben ser exactamente iguales que las clases:
print "Predicciones (deberian ser [1, 1, -1, -1]):", y_


I am working on the file "LAB4/recursos/practica4/practica_svm.ipynb".
The first task is:
La siguiente celda realiza las siguientes acciones:

Crea un problema con dos conjuntos de datos (entrenamiento y test) de 50 puntos cada uno, y dos clases (+1 y -1). La frontera que separa las clases es lineal.

Entrena un clasificador SVC para separar las dos clases, con un kernel lineal.

Imprime los vectores de soporte, las alphas y el bias.

Obtiene la tasa de acierto en training y en test.

Y finalmente dibuja el modelo sobre los datos de entrenamiento y test. La línea negra es la frontera de separación, mientras que las líneas azul y roja representan los márgenes para las clases azul y roja respectivamente. Sobre la gráfica de entrenamiento muestra además los vectores de soporte.
Can you write Python code for it?

# Creación del problema, datos de entrenamiento y test:
np.random.seed(300)
n = 50
model = 'linear'
ymargin = 0.5
x, y = createDataSet(n, model, ymargin)
xtest, ytest = createDataSet(n, model, ymargin)

# Construcción del clasificador:
clf = SVC(C=10, kernel='linear', degree=1.0, coef0=1.0, gamma=0.1)
clf.fit(x, y)

# Vectores de soporte:
print("Vectores de soporte:")
for i in clf.support_:
    print("   [%f, %f]  c = %d" % (x[i,0], x[i,1], y[i]))

# Coeficientes a_i y b:
print("Coeficientes a_i:")
print "  ", clf.dual_coef_
print("Coeficiente b:")
print "  ", clf.intercept_[0]

# Calculo del acierto en los conjuntos de entrenamiento y test:
score_train = clf.score(x, y)
print("Score train = %f" % (score_train))
score_test = clf.score(xtest, ytest)
print("Score test = %f" % (score_test))

# Gráficas:
plt.figure(figsize=(12,6))

plt.subplot(121)
plotModel(x[:,0],x[:,1],y,clf,"Training, score = %f" % (score_train))
for i in clf.support_:
    if y[i] == -1:
        plt.plot(x[i,0],x[i,1],'ro',ms=10)
    else:
        plt.plot(x[i,0],x[i,1],'bo',ms=10)

plt.subplot(122)
plotModel(xtest[:,0],xtest[:,1],ytest,clf,"Test, score = %f" % (score_test))


I am working on the file "LAB4/recursos/practica4/practica_svm.ipynb".
The first task is:
La siguiente prueba genera un problema al azar y lo resuelve con tu método y con sklearn. Ambas soluciones deberían ser parecidas, aunque la tuya será mucho más lenta. Prueba con los diferentes tipos de kernels para comprobar tu implementación.
Can you write Python code for it?

# Prueba con otros problemas y comparacion con sklearn:
from sklearn.svm import SVC

# Generacion de los datos:
n = 20
X = np.random.rand(n,2)
y = 2.0*(X[:,0] > X[:,1]) -1

# Uso de SVC:
clf = SVC(C=10.0, kernel='rbf', degree=2.0, coef0=1.0, gamma=0.5)
clf.fit(X, y)
print "Resultados con sklearn:"
print "  -- Num. vectores de soporte =", clf.dual_coef_.shape[1]
print "  -- Bias b =", clf.intercept_[0]

# Uso de tu algoritmo:
svc = svm.SVM(C=10, kernel="rbf", sigma=1.0, deg=2.0, b=1.0)
svc.simple_smo(X, y, maxiter = 500, tol=1.e-15, verb=True, print_every=10)
print "Resultados con tu algoritmo:"
print "  -- Num. vectores de soporte =", svc.num_sv
print "  -- Bias b =", svc.b

# Comparacion entre las alphas:
a1 = clf.dual_coef_
a2 = (svc.alpha * y)[svc.is_sv]

# Maxima diferencia entre tus alphas y las de sklearn:
maxdif = np.max(np.abs(np.sort(a1) - np.sort(a2)))
print "Maxima diferencia entre tus alphas y las de sklearn:", maxdif


I am working on the file "LAB4/src/practica_svm.ipynb".
The first task is:
1.1. Kernel lineal
Completa el código de la función linear_kernel y comprueba tu solución.
Can you write Python code for it?

# Con tu implementación:
K = svm.linear_kernel(x, y)
print ("El array K deberia tener dimensiones: (%d, %d)" % (n, m))
print ("A ti te sale un array con dimensiones:", K.shape)

# Con la implementación de sklearn:
K_ = polynomial_kernel(x, y, degree=1, gamma=1, coef0=1)

# Diferencia entre tu kernel y el de sklearn (deberia salir practicamente 0):
maxdif = np.max(np.abs(K - K_))
print ("Maxima diferencia entre tu implementacion y la de sklearn:", maxdif)


I am working on the file "LAB4/src/practica_svm.ipynb".
The first task is:
1.2. Kernel polinómico
Completa el código de la función poly_kernel y comprueba tu solución.
Can you write Python code for it?

# Con tu implementación:
K = svm.poly_kernel(x, y, deg=2, b=1)
print ("El array K deberia tener dimensiones: (%d, %d)" % (n, m))
print ("A ti te sale un array con dimensiones:", K.shape)

# Con la implementación de sklearn:
K_ = polynomial_kernel(x, y, degree=2, gamma=1, coef0=1)

# Diferencia entre tu kernel y el de sklearn (deberia salir practicamente 0):
maxdif = np.max(np.abs(K - K_))
print ("Maxima diferencia entre tu implementacion y la de sklearn:", maxdif)


I am working on the file "LAB4/src/practica_svm.ipynb".
The first task is:
2. El algoritmo SMO
A continuación vas a completar la clase SVM, que representa un clasificador basado en máquinas de vectores de soporte, y vas a implementar el algoritmo SMO dentro de esta clase.
2.1. Implementa el método evaluate_model
Lo primero que debes hacer es completar el método evaluate_model, que recibe un array con los datos (atributos) del problema $x$ y calcula $f(x)$. Una vez que lo tengas, para comprobar que funciona bien, puedes ejecutar la siguiente celda de código:
Can you write Python code for it?

# Datos de prueba (problema XOR):
x = np.array([[-1, -1], [1, 1], [1, -1], [-1, 1]])
y = np.array([1, 1, -1, -1])

# Alphas y b:
alpha = np.array([0.125, 0.125, 0.125, 0.125])
b = 0

# Clasificador, introducimos la solucion a mano:
svc = svm.SVM(C=1000, kernel="poly", sigma=1, deg=2, b=1)
svc.init_model(alpha, b, x, y)

# Clasificamos los puntos x:
y_ = svc.evaluate_model(x)

# Las predicciones deben ser exactamente iguales que las clases:
print ("Predicciones (deberian ser [1, 1, -1, -1]):", y_)


I am working on the file "LAB4/src/practica_svm.ipynb".
The first task is:
2.2. Completa el resto de métodos de la clase SVM
Completa los métodos select_alphas, calculate_eta, update_alphas y update_b. Cuando los tengas acabados, ejecuta las celdas siguientes para comprobar tu implementación.
La primera prueba consiste en entrenar el problema del XOR:
Can you write Python code for it?

# Prueba con los datos del XOR:
x = np.array([[-1, -1], [1, 1], [1, -1], [-1, 1]])
y = np.array([1, 1, -1, -1])

# Clasificador que entrenamos para resolver el problema:
svc = svm.SVM(C=1000, kernel="poly", sigma=1, deg=2, b=1)
svc.simple_smo(x, y, maxiter = 100, verb=True)

# Imprimimos los alphas y el bias (deberian ser alpha_i = 0.125, b = 0):
print ("alpha =", svc.alpha)
print ("b =", svc.b)

# Clasificamos los puntos x (las predicciones deberian ser iguales a las clases reales):
y_ = svc.evaluate_model(x)
print ("Predicciones =", y_)


I am working on the file "LAB4/src/practica_svm.ipynb".
The first task is:
3. Visualización de modelos sencillos
Finalmente vamos a utilizar la implementación de sklearn para resolver problemas sencillos de clasificación en dos dimensiones. El objetivo es entender cómo funcionan los distintos tipos de kernel (polinómico y RBF) con problemas que se pueden visualizar fácilmente.
Para implementar los modelos utilizaremos la clase SVC del paquete sklearn.svm.
Primero importamos algunos módulos adicionales, establecemos el modo inline para las gráficas de matplotlib e inicializamos la semilla del generador de números aleatorios. El módulo p4_utils contiene funciones para generar datos en 2D y visualizar los modelos.
Can you write Python code for it?

from p4_utils import *
import matplotlib.pyplot as plt
from sklearn.svm import SVC
%matplotlib inline
np.random.seed(19)


I am working on the file "LAB4/src/practica_svm.ipynb".
The first task is:
La siguiente celda realiza las siguientes acciones:

Crea un problema con dos conjuntos de datos (entrenamiento y test) de 50 puntos cada uno, y dos clases (+1 y -1). La frontera que separa las clases es lineal.

Entrena un clasificador SVC para separar las dos clases, con un kernel lineal.

Imprime los vectores de soporte, las alphas y el bias.

Obtiene la tasa de acierto en training y en test.

Y finalmente dibuja el modelo sobre los datos de entrenamiento y test. La línea negra es la frontera de separación, mientras que las líneas azul y roja representan los márgenes para las clases azul y roja respectivamente. Sobre la gráfica de entrenamiento muestra además los vectores de soporte.
Can you write Python code for it?

# Creación del problema, datos de entrenamiento y test:
np.random.seed(300)
n = 50
model = 'rbf'
ymargin = -0.5
x, y = createDataSet(n, model, ymargin)
xtest, ytest = createDataSet(n, model, ymargin)

# Construcción del clasificador:
clf = SVC(C=100, kernel='rbf', degree=1.0, coef0=1.0, gamma=100)
clf.fit(x, y)

# Vectores de soporte:
print("Vectores de soporte: %d" % (len(clf.support_)))
for i in clf.support_:
    print("   [%f, %f]  c = %d" % (x[i,0], x[i,1], y[i]))

# Coeficientes a_i y b:
print("Coeficientes a_i:")
print ("  ", clf.dual_coef_)
print("Coeficiente b:")
print ("  ", clf.intercept_[0])

# Calculo del acierto en los conjuntos de entrenamiento y test:
score_train = clf.score(x, y)
print("Score train = %f" % (score_train))
score_test = clf.score(xtest, ytest)
print("Score test = %f" % (score_test))

# Gráficas:
plt.figure(figsize=(12,6))

plt.subplot(121)
plotModel(x[:,0],x[:,1],y,clf,"Training, score = %f" % (score_train))
for i in clf.support_:
    if y[i] == -1:
        plt.plot(x[i,0],x[i,1],'ro',ms=10)
    else:
        plt.plot(x[i,0],x[i,1],'bo',ms=10)

plt.subplot(122)
plotModel(xtest[:,0],xtest[:,1],ytest,clf,"Test, score = %f" % (score_test))


I am working on the file "LA_Team/Facies_classification_LA_TEAM.ipynb".
The first task is:
Although XGBoost is a very straightforward algorithm to implement, it's difficulty arises in dealing with a big number of hyperparameters. For that reason, we need to develop routines to tune this parameters to optimize the algorithm performance on the data prediction.
We will use a Cross-Validation approach to improve our model by tuning parameters at each step. There are three types of parameters to consider:

General Parameters: Guide the overall functioning
Booster Parameters: Guide the individual booster (tree/regression) at each step
Learning Task Parameters: Guide the optimization performed

We must realize that our training data is quite reduced, therefore to evaluate our model generalization
Can you write Python code for it?

# Cross Validation parameters
cv_folds = 10
rounds = 100

# Proposed Initial Model
xgb1 = xgb.XGBClassifier( learning_rate =0.01, n_estimators=500, max_depth=6,
                          min_child_weight=1, gamma=0, subsample=0.8,
                          colsample_bytree=0.8, objective='multi:softmax',
                          nthread=4, scale_pos_weight=1, seed=27)

xgb_param_1 = xgb1.get_xgb_params()
xgb_param_1['num_class'] = 9

# Perform cross-validation
cvresult = xgb.cv(xgb_param_1, dtrain, num_boost_round=xgb_param_1['n_estimators'],
                  stratified = True, nfold=cv_folds, metrics='merror', early_stopping_rounds=rounds)

print "\nCross Validation Training Report Summary"
print cvresult.tail()

xgb1.set_params(n_estimators=cvresult.shape[0])

#Fit the algorithm on the data
xgb1.fit(X_train, Y_train,eval_metric='merror')

#Predict training set:
predictions = xgb1.predict(X_train)

#Print model report

# Confusion Matrix
conf = confusion_matrix(Y_train, predictions )

# Print Results
print "\nModel Report"
print "-Accuracy: %.6f" % ( accuracy(conf) )
print "-Adjacent Accuracy: %.6f" % ( accuracy_adjacent(conf, adjacent_facies) )

print "\nConfusion Matrix"
display_cm(conf, facies_labels, display_metrics=True, hide_zeros=True)

# Print Feature Importance
feat_imp = pd.Series(xgb1.booster().get_fscore()).sort_values(ascending=False)
feat_imp.plot(kind='bar', title='Feature Importances')
plt.ylabel('Feature Importance Score')


I am working on the file "LA_Team/Facies_classification_LA_TEAM.ipynb".
The first task is:
By performing cross-validation routines we can use the produced results to measure how well the model generalizes and at the same time tune the hyperparameters. In this case, we will explore how the results look like if we vary the learning rate.
Can you write Python code for it?

print("Parameter optimization")
grid_search1 = GridSearchCV(xgb1,{'learning_rate':[0.001,0.01,0.05] , 'n_estimators':[100,200,500]},
                                   scoring='accuracy' , n_jobs = 4)
grid_search1.fit(X_train,Y_train)
print("Best Set of Parameters")
grid_search1.grid_scores_, grid_search1.best_params_, grid_search1.best_score_

# Cross Validation parameters
cv_folds = 10
rounds = 100

# Proposed Initial Model
xgb2 = xgb.XGBClassifier( learning_rate =0.001, n_estimators=500, max_depth=6,
                          min_child_weight=1, gamma=0, subsample=0.8,
                          colsample_bytree=0.8, objective='multi:softmax',
                          nthread=4, scale_pos_weight=1, seed=27)

xgb_param_2 = xgb2.get_xgb_params()
xgb_param_2['num_class'] = 9

# Perform cross-validation
cvresult = xgb.cv(xgb_param_2, dtrain, num_boost_round=xgb_param_2['n_estimators'],
                  nfold=cv_folds, metrics='merror', early_stopping_rounds=rounds)

print "\nCross Validation Training Report Summary"
print cvresult.tail()

xgb2.set_params(n_estimators=cvresult.shape[0])

#Fit the algorithm on the data
xgb2.fit(X_train, Y_train,eval_metric='merror')

#Predict training set:
predictions = xgb2.predict(X_train)

#Print model report

# Confusion Matrix
conf = confusion_matrix(Y_train, predictions )

# Print Results
print "\nModel Report"
print "-Accuracy: %.6f" % ( accuracy(conf) )
print "-Adjacent Accuracy: %.6f" % ( accuracy_adjacent(conf, adjacent_facies) )

# Confusion Matrix
print "\nConfusion Matrix"
display_cm(conf, facies_labels, display_metrics=True, hide_zeros=True)

# Print Feature Importance
feat_imp = pd.Series(xgb2.booster().get_fscore()).sort_values(ascending=False)
feat_imp.plot(kind='bar', title='Feature Importances')
plt.ylabel('Feature Importance Score')

print("Parameter optimization")
grid_search1 = GridSearchCV(xgb2,{'reg_alpha':[1e-5, 1e-2, 0.1, 1, 100] },
                                   scoring='accuracy' , n_jobs =...


I am working on the file "LA_Team/Facies_classification_LA_TEAM.ipynb".
The first task is:
Data Analysis

XGboost will be our weapon of choice. XGBoost (or Extreme Gradient Boosting) is a sophisticated algorithm that corresponds to an advanced implementation of gradient boosting algorithm. Despite it's complexity, it is fairly easy to use and very powerful to deal with many different types of irregularities in data. It has been no surprise then, that is has been extensively applied in many machine learning competitions to obtain very promising results.
For the above reasons, to implement the XGBoost algorithm for our regression problem, we will use the above preprocessed data without modifications.
Can you write Python code for it?

X_train = training_data.drop(['Facies', 'Well Name','Formation','Depth'], axis = 1 )
Y_train = training_data['Facies' ] - 1
dtrain = xgb.DMatrix(X_train, Y_train)
type(Y_train)


I am working on the file "LA_Team/Facies_classification_LA_TEAM.ipynb".
The first task is:
In order to evaluate our classification model accurary we will use the our following defined metrics, based on the confusion matrix once the classification is performed. The first metric only considers misclassification error and the second one takes into account the fact that facies could be misclassified if they belong to a same group with similar geological characteristics.
Can you write Python code for it?

def accuracy(conf):
    total_correct = 0.
    nb_classes = conf.shape[0]
    for i in np.arange(0,nb_classes):
        total_correct += conf[i][i]
    acc = total_correct/sum(sum(conf))
    return acc

adjacent_facies = np.array([[1], [0,2], [1], [4], [3,5], [4,6,7], [5,7], [5,6,8], [6,7]])

def accuracy_adjacent(conf, adjacent_facies):
    nb_classes = conf.shape[0]
    total_correct = 0.
    for i in np.arange(0,nb_classes):
        total_correct += conf[i][i]
        for j in adjacent_facies[i]:
            total_correct += conf[i][j]
    return total_correct / sum(sum(conf))


I am working on the file "LA_Team/Facies_classification_LA_TEAM.ipynb".
The first task is:
Results

We will see the performance of the model by taking the average accuracy and adjacency accuracy in Leaving Out One Well Cross Validation routine.
Can you write Python code for it?

# Import data
filename = '../facies_vectors.csv'
data = pd.read_csv(filename)

data['Well Name'] = data['Well Name'].astype('category')
data['Formation'] = data['Formation'].astype('category')

# Leave out one well for prediction
well_names = data['Well Name'].unique()
f1=[]
for i in range(len(well_names)):

    # Split data
    X_train = data.drop(['Facies', 'Formation','Depth'], axis = 1 )
    Y_train = data['Facies' ] - 1

    train_X = X_train[X_train['Well Name'] != well_names[i] ]
    train_Y = Y_train[X_train['Well Name'] != well_names[i] ]
    test_X  = X_train[X_train['Well Name'] == well_names[i] ]
    test_Y  = Y_train[X_train['Well Name'] == well_names[i] ]

    train_X = train_X.drop(['Well Name'], axis = 1 )
    test_X = test_X.drop(['Well Name'], axis = 1 )

    # Model
    model_final = xgb.XGBClassifier( learning_rate =0.001, n_estimators=500, max_depth=6,
                                   min_child_weight=1, gamma=0, subsample=0.8, reg_alpha = 10,
                                   colsample_bytree=0.8, objective='multi:softmax',
                                   nthread=4, scale_pos_weight=1, seed=27)

    #Fit the algorithm on the data
    model_final.fit( train_X , train_Y , eval_metric = 'merror' )

    # model_final = RandomForestClassifier(n_estimators=1000) # RANDOM FORREST
    #model_final.fit(train_X, train_Y)

    #Predict training set:
    predictions = model_final.predict(test_X)

    #Print model report
    print "\n------------------------------------------------------"
    print "Leaving out well " + well_names[i]
    # Confusion Matrix
    conf = confusion_matrix( test_Y, predictions, labels = np.arange(9) )
    # Print Results
    print "\nModel Report"
    print "-Accuracy: %.6f" % ( accuracy(conf) )
    print "-Adjacent Accuracy: %.6f" % ( accuracy_adjacent(conf, adjacent_facies) )
    print "-F1 Score: %.6f" % ( f1_score ( test_Y , predictions , labels = np.arange(9), average = 'weighted' ) )
    f1.append(f1_score ( test_Y , predictions , labels =...


I am working on the file "LA_Team/Facies_classification_LA_TEAM.ipynb".
The first task is:
We declare the fields "Formation" and "Well Name" as categorical variables and then map them to integer values.
Can you write Python code for it?

training_data['Well Name'] = training_data['Well Name'].astype('category')
training_data['Formation'] = training_data['Formation'].astype('category')
training_data.info()


I am working on the file "LA_Team/Facies_classification_LA_TEAM.ipynb".
The first task is:
We load the training data to start the exploration stage.
Can you write Python code for it?

set_option("display.max_rows", 10)
pd.options.mode.chained_assignment = None

filename = '../facies_vectors.csv'
training_data = pd.read_csv(filename)
training_data


I am working on the file "LA_Team/Facies_classification_LA_TEAM.ipynb".
The first task is:
We must realize the classification problem is highly imbalanced, therefore making it more challenging to approach.
Can you write Python code for it?

facies_colors = ['#F4D03F', '#F5B041','#DC7633','#6E2C00','#1B4F72',
                 '#2E86C1', '#AED6F1', '#A569BD', '#196F3D']

facies_labels = ['SS', 'CSiS', 'FSiS', 'SiSh', 'MS','WS', 'D','PS', 'BS']

facies_counts = training_data['Facies'].value_counts().sort_index()
facies_counts.index = facies_labels
facies_counts.plot(kind='bar',color=facies_colors,title='Distribution of Training Data by Facies')


I am working on the file "LA_Team/Facies_classification_LA_TEAM.ipynb".
The first task is:
We obtain an Average F1 Score from fitting the model to a leaving out one well 0.528226
We import the testing data and apply our trained model to make a prediction.
Can you write Python code for it?

# Load test data
test_data = pd.read_csv('validation_data_nofacies.csv')
test_data['Well Name'] = test_data['Well Name'].astype('category')
X_test = test_data.drop(['Formation', 'Well Name', 'Depth'], axis=1)
# Predict facies of unclassified data
Y_predicted = xgb_final.predict(X_test)
test_data['Facies'] = Y_predicted + 1
# Store the prediction
test_data.to_csv('Prediction.csv')


I am working on the file "LA_Team/Facies_classification_LA_TEAM.ipynb".
The first task is:
We produce correlation plot to observe relationship between variables. The target variable 'Facies' is highly correlated to 'NM_M', 'PE' and 'ILD_log10'.
Can you write Python code for it?

plt.figure(figsize=(10, 10))
sns.heatmap(training_data.corr(), vmax=1.0, square=True)

training_data.describe()


I am working on the file "LA_Team/Facies_classification_LA_TEAM_02.ipynb".
The first task is:
Data Analysis

Our model will be composed by a Deep Neural Network of an input layer, two hidden layers and an output layer.
As input we will consider a set of features extracted by padding a depth interval segment, that way we take into account spatial dependencies.
Can you write Python code for it?

np.random.seed(1337) # For reproducibility
X_train, y_train, X_test = preprocess(data_out)

# Set parameters
input_dim = 77
hidden_dim_1 = 128
hidden_dim_2 = 32
output_dim = 9
batch_size = 32
nb_epoch = 10

def proposed_model():
    # Define the model
    model = Sequential()
    model.add(Dense(128, input_dim=77, init='normal', activation='relu'))
    model.add(Dense(32, input_dim=128, init='normal', activation='relu'))
    model.add(Dropout(0.3))
    model.add(Dense(9, init='normal', activation='softmax'))
    # Compile model
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model


I am working on the file "LA_Team/Facies_classification_LA_TEAM_02.ipynb".
The first task is:
In order to start training stage, it is required to format the data by considering a sliding window over the depth component in order to classify a given set of features at some specific depth for each well in the training set.
Can you write Python code for it?

def preprocess(data_out):
    data = data_out
    well_data = {}
    well_names = list(set(data[:, 0]))
    for name in well_names:
        well_data[name] = [[], []]

    for row in data:
        well_data[row[0]][1].append(row[1])
        well_data[row[0]][0].append(list(row[2::]))

    # Sliding window
    positive_lag = 5
    negative_lag = 5

    chunks = []
    chunks_test = []
    chunk_length = positive_lag+negative_lag+1
    chunks_facies = []
    for name in well_names:
        if name not in ['STUART', 'CRAWFORD']:
            test_well_data = well_data[name]
            log_values = np.array(test_well_data[0])
            log_values_padded = np.lib.pad(log_values, (negative_lag,positive_lag), 'edge')[:, negative_lag:-positive_lag]
            facies_values =  np.array(test_well_data[1])
            for i in range(log_values.shape[0]):
                chunks.append(log_values_padded[i:i+chunk_length, :])
                chunks_facies.append(facies_values[i])
        else:
            test_well_data = well_data[name]
            log_values = np.array(test_well_data[0])
            log_values_padded = np.lib.pad(log_values, (negative_lag,positive_lag), 'edge')[:, negative_lag:-positive_lag]
            facies_values =  np.array(test_well_data[1])
            for i in range(log_values.shape[0]):
                chunks_test.append(log_values_padded[i:i+chunk_length, :])

    chunks_facies = np.array(chunks_facies, dtype=np.int32)-1
    X_ = np.array(chunks)
    X = np.zeros((len(X_),len(X_[0][0]) * len(X_[0])))
    for i in range(len(X_)):
        X[i,:] = X_[i].flatten()

    X_test = np.array(chunks_test)
    X_test_out = np.zeros((len(X_test),len(X_test[0][0]) * len(X_test[0])))
    for i in range(len(X_test)):
        X_test_out[i,:] = X_test[i].flatten()
    y = np_utils.to_categorical(chunks_facies)
    return X, y, X_test_out


I am working on the file "LA_Team/Facies_classification_LA_TEAM_02.ipynb".
The first task is:
Once the set of parameters are fixed, the training stage of our model begins. We perform a Cross Validation routine to evaluate the performance of the model.
Can you write Python code for it?

estimator = KerasClassifier(build_fn=proposed_model, nb_epoch=10, batch_size=50, verbose=0)
# Cross Validation
kfold = KFold(n_splits=5, shuffle=True)
results = cross_val_score(estimator, X_train, y_train, cv=kfold)
print(' Cross Validation Results')
print( results )

plt.hist(results, bins= 5)
plt.title("Model Accuracy")
plt.show()
print ( "Cross Validation Average Accuracy %.3f" %  results.mean() )
print ( "Cross Validation Standard Deviation %.3f" %  results.std() )

# Load the model
model = proposed_model()
#Train model
model.fit(X_train, y_train, nb_epoch=10, verbose=2, shuffle = True)

# Predict Values on Training set
y_predicted = model.predict( X_train , batch_size=32, verbose=0)

# Print Report

# Format output [0 - 8 ]
y_ = np.zeros((len(y_train),1))
for i in range(len(y_train)):
    y_[i] = np.argmax(y_train[i])

y_predicted_ = np.zeros((len(y_predicted), 1))
for i in range(len(y_predicted)):
    y_predicted_[i] = np.argmax( y_predicted[i])

# Confusion Matrix
conf = confusion_matrix(y_, y_predicted_)

# Print Results
print ("\nModel Report")
print ("-Accuracy: %.6f" % ( accuracy(conf) ))
print ("-Adjacent Accuracy: %.6f" % ( accuracy_adjacent(conf, adjacent_facies) ))
print ("\nConfusion Matrix")
display_cm(conf, facies_labels, display_metrics=True, hide_zeros=True)


I am working on the file "LA_Team/Facies_classification_LA_TEAM_02.ipynb".
The first task is:
Prediction

We obtain the predictions for test data.
Can you write Python code for it?

y_test = model.predict( X_test , batch_size=32, verbose=0)
final_predictions = np.zeros((len(y_test),1))
for i in range(len(y_test)):
    final_predictions[i] = np.argmax(y_test[i]) + 1

last_submission = pd.read_csv('Prediction.csv')
ls = last_submission['Facies'].as_matrix()

conf = confusion_matrix(ls, final_predictions)

# Print Results
print ("\nModel Report")
print ("-Accuracy: %.6f" % ( accuracy(conf) ))
print ("-Adjacent Accuracy: %.6f" % ( accuracy_adjacent(conf, adjacent_facies) ))
print ("\nConfusion Matrix")
display_cm(conf, facies_labels, display_metrics=True, hide_zeros=True)

# Store results
test_data = pd.read_csv('../validation_data_nofacies.csv')
test_data['Facies'] = final_predictions
test_data.to_csv('Prediction2.csv')


I am working on the file "LA_Team/Facies_classification_LA_TEAM_02.ipynb".
The first task is:
We fill the missing data values in the PE field with zero and proceed to normalize the data that will be fed into our model.
Can you write Python code for it?

# Set 'Well Name' and 'Formation' fields as categories
training_data['Well Name'] = training_data['Well Name'].astype('category')
training_data['Formation'] = training_data['Formation'].astype('category')

# Fill missing values and normalize for 'PE' field
training_data['PE'] = training_data['PE'].fillna(value=0)
mean_pe = training_data['PE'].mean()
std_pe = training_data['PE'].std()
training_data['PE'] = (training_data['PE']-mean_pe)/std_pe

# Normalize the rest of fields (GR, ILD_log10, DelthaPHI, PHIND,NM_M,RELPOS)
correct_facies_labels = training_data['Facies'].values
feature_vectors = training_data.drop(['Formation', 'Depth'], axis=1)
well_labels = training_data[['Well Name', 'Facies']].values
data_vectors = feature_vectors.drop(['Well Name', 'Facies'], axis=1).values

scaler = preprocessing.StandardScaler().fit(data_vectors)
scaled_features = scaler.transform(data_vectors)
data_out = np.hstack([well_labels, scaled_features])


I am working on the file "LA_Team/Facies_classification_LA_TEAM_02.ipynb".
The first task is:
We load the training and testing data to preprocess it for further analysis.
Can you write Python code for it?

filename = 'train_test_data.csv'
training_data = pd.read_csv(filename)
training_data.head(10)


I am working on the file "LA_Team/Facies_classification_LA_TEAM_03.ipynb".
The first task is:
Data Analysis

We will experiment with an ensemble of classification models to outperform the accuracy at predicting facies. As input we will consider a set of features extracted by padding a depth interval segment, that way we take into account spatial dependencies. As output we obtain a vector filled with values ranging from [0-8] that indicate the presence of each facie with respect to depth.
Can you write Python code for it?

# Reproducibility
np.random.seed(7)
# Load data
X_train, y_train, X_test = preprocess(data_out)
# Obtain labels
y_labels = np.zeros((len(y_train),1))
for i in range(len(y_train)):
    y_labels[i] = np.argmax(y_train[i])
y_labels = y_labels.astype(int)


I am working on the file "LA_Team/Facies_classification_LA_TEAM_03.ipynb".
The first task is:
In order to evaluate our classification model accurary we will use the our following defined metrics, based on the confusion matrix once the classification is performed. The first metric only considers misclassification error and the second one takes into account the fact that facies could be misclassified if they belong to a same group with similar geological characteristics.
Can you write Python code for it?

def accuracy(conf):
    total_correct = 0.
    nb_classes = conf.shape[0]
    for i in np.arange(0,nb_classes):
        total_correct += conf[i][i]
    acc = total_correct/sum(sum(conf))
    return acc

adjacent_facies = np.array([[1], [0,2], [1], [4], [3,5], [4,6,7], [5,7], [5,6,8], [6,7]])
facies_labels = ['SS', 'CSiS', 'FSiS', 'SiSh', 'MS','WS', 'D','PS', 'BS']

def accuracy_adjacent(conf, adjacent_facies):
    nb_classes = conf.shape[0]
    total_correct = 0.
    for i in np.arange(0,nb_classes):
        total_correct += conf[i][i]
        for j in adjacent_facies[i]:
            total_correct += conf[i][j]
    return total_correct / sum(sum(conf))


I am working on the file "LA_Team/Facies_classification_LA_TEAM_03.ipynb".
The first task is:
In order to start training stage, it is required to format the data by considering a sliding window over the depth component in order to classify a given set of features at some specific depth for each well in the training set.
Can you write Python code for it?

def preprocess(data_out):

    data = data_out
    well_data = {}
    well_names = list(set(data[:, 0]))
    for name in well_names:
        well_data[name] = [[], []]

    for row in data:
        well_data[row[0]][1].append(row[1])
        well_data[row[0]][0].append(list(row[2::]))

    # Sliding window
    positive_lag = 0
    negative_lag = 0

    chunks = []
    chunks_test = []
    chunk_length = positive_lag+negative_lag+1
    chunks_facies = []
    for name in well_names:
        if name not in ['STUART', 'CRAWFORD']:
            test_well_data = well_data[name]
            log_values = np.array(test_well_data[0])
            facies_values =  np.array(test_well_data[1])
            for i in range(log_values.shape[0]):
                chunks.append(log_values[i:i+1, :])
                chunks_facies.append(facies_values[i])
        else:
            test_well_data = well_data[name]
            log_values = np.array(test_well_data[0])
            facies_values =  np.array(test_well_data[1])
            for i in range(log_values.shape[0]):
                chunks_test.append(log_values[i:i+1, :])

    chunks_facies = np.array(chunks_facies, dtype=np.int32)-1
    X_ = np.array(chunks)
    X = np.zeros((len(X_),len(X_[0][0]) * len(X_[0])))
    for i in range(len(X_)):
        X[i,:] = X_[i].flatten()

    X_test = np.array(chunks_test)
    X_test_out = np.zeros((len(X_test),len(X_test[0][0]) * len(X_test[0])))
    for i in range(len(X_test)):
        X_test_out[i,:] = X_test[i].flatten()
    y = np_utils.to_categorical(chunks_facies)
    return X, y, X_test_out


I am working on the file "LA_Team/Facies_classification_LA_TEAM_03.ipynb".
The first task is:
Model 1 - Deep Neural Network
Our model will be composed by a Deep Neural Network of an input layer, two hidden layers and an output layer.
Can you write Python code for it?

# Set parameters
input_dim = 77
hidden_dim_1 = 200
hidden_dim_2 = 50
output_dim = 9
batch_size = 32
nb_epoch = 10

def dnn_model():
    # Define the model
    model = Sequential()
    model.add(Dense(200, input_dim=7, init='normal', activation='relu'))
    model.add(Dense(50, input_dim=200, init='normal', activation='relu'))
    model.add(Dense(9, init='normal', activation='softmax'))
    # Compile model
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model


I am working on the file "LA_Team/Facies_classification_LA_TEAM_03.ipynb".
The first task is:
Once the set of parameters are fixed, the training stage of our model begins. We perform a Cross Validation routine to evaluate the performance of the model.
Can you write Python code for it?

# Cross Validation
estimator = KerasClassifier(build_fn=dnn_model, nb_epoch=10, batch_size=1, verbose=0)
skf = StratifiedKFold(n_splits=5, shuffle=True)
results_dnn = cross_val_score(estimator, X_train, y_train, cv= skf.get_n_splits(X_train, y_train))
print(' Cross Validation Results')
print( results_dnn )

# Load the model
model_dnn = dnn_model()

#Train model
model_dnn.fit(X_train, y_train, nb_epoch=10, verbose=2)

# Predict Values on Training set
y_predicted = model_dnn.predict( X_train , batch_size=1, verbose=2)

# Print Report

# Format output [0 - 8 ]
y_ = np.zeros((len(y_train),1))
for i in range(len(y_train)):
    y_[i] = np.argmax(y_train[i])

y_predicted_ = np.zeros((len(y_predicted), 1))
for i in range(len(y_predicted)):
    y_predicted_[i] = np.argmax( y_predicted[i] )

# Confusion Matrix
conf = confusion_matrix(y_, y_predicted_)

# Print Results
print ("\nModel Report")
print ("-Accuracy: %.6f" % ( accuracy(conf) ))
print ("-Adjacent Accuracy: %.6f" % ( accuracy_adjacent(conf, adjacent_facies) ))
print ("\nConfusion Matrix")
display_cm(conf, facies_labels, display_metrics=True, hide_zeros=True)


I am working on the file "LA_Team/Facies_classification_LA_TEAM_03.ipynb".
The first task is:
Prediction

We obtain the predictions for test data.
Can you write Python code for it?

# DNN model Prediction
y_test = model_dnn.predict( X_test , batch_size=100, verbose=0)
predictions_dnn = np.zeros((len(y_test),1))
for i in range(len(y_test)):
    predictions_dnn[i] = np.argmax(y_test[i]) + 1
predictions_dnn = predictions_dnn.astype(int)
# Store results
test_data = pd.read_csv('../validation_data_nofacies.csv')
test_data['Facies'] = predictions_dnn
test_data.to_csv('Prediction_3.csv')


I am working on the file "LA_Team/Facies_classification_LA_TEAM_03.ipynb".
The first task is:
We fill the missing data values in the PE field with zero and proceed to normalize the data that will be fed into our model.
Can you write Python code for it?

# Set 'Well Name' and 'Formation' fields as categories
data['Well Name'] = data['Well Name'].astype('category')
data['Formation'] = data['Formation'].astype('category')

# Fill missing values and normalize for 'PE' field
data['PE'] = data['PE'].fillna(value=0)
mean_pe = data['PE'].mean()
std_pe = data['PE'].std()
data['PE'] = (data['PE']-mean_pe)/std_pe

# Normalize the rest of fields (GR, ILD_log10, DelthaPHI, PHIND,NM_M,RELPOS)
correct_facies_labels = data['Facies'].values
feature_vectors = data.drop(['Formation', 'Depth'], axis=1)
well_labels = data[['Well Name', 'Facies']].values
data_vectors = feature_vectors.drop(['Well Name', 'Facies'], axis=1).values

scaler = preprocessing.StandardScaler().fit(data_vectors)
scaled_features = scaler.transform(data_vectors)
data_out = np.hstack([well_labels, scaled_features])


I am working on the file "LA_Team/Facies_classification_LA_TEAM_05.ipynb".
The first task is:
Data Analysis
In this section we will run a Cross Validation routine
Can you write Python code for it?

from tpot import TPOTClassifier
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = preprocess()

tpot = TPOTClassifier(generations=5, population_size=20,
                      verbosity=2,max_eval_time_mins=20,
                      max_time_mins=100,scoring='f1_micro',
                      random_state = 17)
tpot.fit(X_train, y_train)
print(tpot.score(X_test, y_test))
tpot.export('FinalPipeline.py')

from sklearn.ensemble import  RandomForestClassifier, VotingClassifier
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import BernoulliNB
from sklearn.pipeline import make_pipeline, make_union
from sklearn.preprocessing import FunctionTransformer
import xgboost as xgb
from xgboost.sklearn import  XGBClassifier

# Train and test a classifier
def train_and_test(X_tr, y_tr, X_v, well_v):

    # Feature normalization
    scaler = preprocessing.RobustScaler(quantile_range=(25.0, 75.0)).fit(X_tr)
    X_tr = scaler.transform(X_tr)
    X_v = scaler.transform(X_v)

    # Train classifier
    #clf = make_pipeline(make_union(VotingClassifier([("est", ExtraTreesClassifier(criterion="gini", max_features=1.0, n_estimators=500))]), FunctionTransformer(lambda X: X)), XGBClassifier(learning_rate=0.73, max_depth=10, min_child_weight=10, n_estimators=500, subsample=0.27))
    #clf =  make_pipeline( KNeighborsClassifier(n_neighbors=5, weights="distance") )
    #clf = make_pipeline(MaxAbsScaler(),make_union(VotingClassifier([("est", RandomForestClassifier(n_estimators=500))]), FunctionTransformer(lambda X: X)),ExtraTreesClassifier(criterion="entropy", max_features=0.0001, n_estimators=500))
    # * clf = make_pipeline( make_union(VotingClassifier([("est", BernoulliNB(alpha=60.0, binarize=0.26, fit_prior=True))]), FunctionTransformer(lambda X: X)),RandomForestClassifier(n_estimators=500))
    clf = make_pipeline ( XGBClassifier(learning_rate=0.12, max_depth=3, min_child_weight=10, n_estimators=150, seed = 17, colsample_bytree = 0.9) )
    clf.fit(X_tr,...


I am working on the file "LA_Team/Facies_classification_LA_TEAM_05.ipynb".
The first task is:
Data Preprocessing
Can you write Python code for it?

#Load Data
data = pd.read_csv('../facies_vectors.csv')

# Parameters
feature_names = ['GR', 'ILD_log10', 'DeltaPHI', 'PHIND', 'PE', 'NM_M', 'RELPOS']
facies_names = ['SS', 'CSiS', 'FSiS', 'SiSh', 'MS', 'WS', 'D', 'PS', 'BS']
facies_colors = ['#F4D03F', '#F5B041','#DC7633','#6E2C00', '#1B4F72','#2E86C1', '#AED6F1', '#A569BD', '#196F3D']

# Store features and labels
X = data[feature_names].values
y = data['Facies'].values

# Store well labels and depths
well = data['Well Name'].values
depth = data['Depth'].values

# Fill 'PE' missing values with mean
imp = preprocessing.Imputer(missing_values='NaN', strategy='mean', axis=0)
imp.fit(X)
X = imp.transform(X)


I am working on the file "LA_Team/Facies_classification_LA_TEAM_05.ipynb".
The first task is:
We procceed to run Paolo Bestagini's routine to include a small window of values to acount for the spatial component in the log analysis, as well as the gradient information with respect to depth. This will be our prepared training dataset.
Can you write Python code for it?

# Feature windows concatenation function
def augment_features_window(X, N_neig):

    # Parameters
    N_row = X.shape[0]
    N_feat = X.shape[1]

    # Zero padding
    X = np.vstack((np.zeros((N_neig, N_feat)), X, (np.zeros((N_neig, N_feat)))))

    # Loop over windows
    X_aug = np.zeros((N_row, N_feat*(2*N_neig+1)))
    for r in np.arange(N_row)+N_neig:
        this_row = []
        for c in np.arange(-N_neig,N_neig+1):
            this_row = np.hstack((this_row, X[r+c]))
        X_aug[r-N_neig] = this_row

    return X_aug

# Feature gradient computation function
def augment_features_gradient(X, depth):

    # Compute features gradient
    d_diff = np.diff(depth).reshape((-1, 1))
    d_diff[d_diff==0] = 0.001
    X_diff = np.diff(X, axis=0)
    X_grad = X_diff / d_diff

    # Compensate for last missing value
    X_grad = np.concatenate((X_grad, np.zeros((1, X_grad.shape[1]))))

    return X_grad

# Feature augmentation function
def augment_features(X, well, depth, N_neig=1):

    # Augment features
    X_aug = np.zeros((X.shape[0], X.shape[1]*(N_neig*2+2)))
    for w in np.unique(well):
        w_idx = np.where(well == w)[0]
        X_aug_win = augment_features_window(X[w_idx, :], N_neig)
        X_aug_grad = augment_features_gradient(X[w_idx, :], depth[w_idx])
        X_aug[w_idx, :] = np.concatenate((X_aug_win, X_aug_grad), axis=1)

    # Find padded rows
    padded_rows = np.unique(np.where(X_aug[:, 0:7] == np.zeros((1, 7)))[0])

    return X_aug, padded_rows

X_aug, padded_rows = augment_features(X, well, depth)

# Initialize model selection methods
lpgo = LeavePGroupsOut(2)

# Generate splits
split_list = []
for train, val in lpgo.split(X, y, groups=data['Well Name']):
    hist_tr = np.histogram(y[train], bins=np.arange(len(facies_names)+1)+.5)
    hist_val = np.histogram(y[val], bins=np.arange(len(facies_names)+1)+.5)
    if np.all(hist_tr[0] != 0) & np.all(hist_val[0] != 0):
        split_list.append({'train':train, 'val':val})

def preprocess():
  ...


I am working on the file "LA_Team/Facies_classification_LA_TEAM_05_VALIDATION.ipynb".
The first task is:
Data Analysis
In this section we will run a Cross Validation routine
Can you write Python code for it?

# from tpot import TPOTClassifier
# from sklearn.model_selection import train_test_split
# X_train, X_test, y_train, y_test = preprocess()

# tpot = TPOTClassifier(generations=5, population_size=20,
#                       verbosity=2,max_eval_time_mins=20,
#                       max_time_mins=100,scoring='f1_micro',
#                       random_state = 17)
# tpot.fit(X_train, y_train)
# print(tpot.score(X_test, y_test))
# tpot.export('FinalPipeline.py')

from sklearn.ensemble import  RandomForestClassifier, VotingClassifier
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import BernoulliNB
from sklearn.pipeline import make_pipeline, make_union
from sklearn.preprocessing import FunctionTransformer
import xgboost as xgb
from xgboost.sklearn import  XGBClassifier

# Train and test a classifier

# Pass in the classifier so we can iterate over many seed later.
def train_and_test(X_tr, y_tr, X_v, well_v, clf):

    # Feature normalization
    scaler = preprocessing.RobustScaler(quantile_range=(25.0, 75.0)).fit(X_tr)
    X_tr = scaler.transform(X_tr)
    X_v = scaler.transform(X_v)

    clf.fit(X_tr, y_tr)

    # Test classifier
    y_v_hat = clf.predict(X_v)

    # Clean isolated facies for each well
    for w in np.unique(well_v):
        y_v_hat[well_v==w] = medfilt(y_v_hat[well_v==w], kernel_size=5)

    return y_v_hat


I am working on the file "LA_Team/Facies_classification_LA_TEAM_05_VALIDATION.ipynb".
The first task is:
Data Preprocessing
Can you write Python code for it?


I am working on the file "LA_Team/Facies_classification_LA_TEAM_06.ipynb".
The first task is:
Data Analysis
In this section we will run a Cross Validation routine
Can you write Python code for it?

from tpot import TPOTClassifier
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = preprocess()

tpot = TPOTClassifier(generations=5, population_size=100,
                      verbosity=2, max_eval_time_mins=30,
                      max_time_mins=6*60, scoring='f1_micro',
                      random_state = 17)
tpot.fit(X_train, y_train)
print(tpot.score(X_test, y_test))
#tpot.export('FinalPipeline_LM_long_2.py')

from sklearn.ensemble import  RandomForestClassifier, VotingClassifier
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import BernoulliNB
from sklearn.pipeline import make_pipeline, make_union
from sklearn.preprocessing import FunctionTransformer, StandardScaler
import xgboost as xgb
from xgboost.sklearn import  XGBClassifier

# Train and test a classifier
def train_and_test(X_tr, y_tr, X_v, well_v):

    # Feature normalization
    scaler = preprocessing.RobustScaler(quantile_range=(25.0, 75.0)).fit(X_tr)
    X_tr = scaler.transform(X_tr)
    X_v = scaler.transform(X_v)

    # Train classifier
    #clf = make_pipeline(make_union(VotingClassifier([("est", ExtraTreesClassifier(criterion="gini", max_features=1.0, n_estimators=500))]), FunctionTransformer(lambda X: X)), XGBClassifier(learning_rate=0.73, max_depth=10, min_child_weight=10, n_estimators=500, subsample=0.27))
    #clf =  make_pipeline( KNeighborsClassifier(n_neighbors=5, weights="distance") )
    #clf = make_pipeline(MaxAbsScaler(),make_union(VotingClassifier([("est", RandomForestClassifier(n_estimators=500))]), FunctionTransformer(lambda X: X)),ExtraTreesClassifier(criterion="entropy", max_features=0.0001, n_estimators=500))
    # * clf = make_pipeline( make_union(VotingClassifier([("est", BernoulliNB(alpha=60.0, binarize=0.26, fit_prior=True))]), FunctionTransformer(lambda X: X)),RandomForestClassifier(n_estimators=500))
    clf = make_pipeline(
        make_union(VotingClassifier([("est", BernoulliNB(alpha=0.41000000000000003, binarize=0.43,...


I am working on the file "LA_Team/Facies_classification_LA_TEAM_06.ipynb".
The first task is:
Data Preprocessing
Can you write Python code for it?


I am working on the file "LA_Team/Facies_classification_LA_TEAM_06.ipynb".
The first task is:
Prediction
Can you write Python code for it?

#Load testing data
test_data = pd.read_csv('../validation_data_nofacies.csv')

# Prepare training data
X_tr = X
y_tr = y

# Augment features
X_tr, padded_rows = augment_features(X_tr, well, depth)

# Removed padded rows
X_tr = np.delete(X_tr, padded_rows, axis=0)
y_tr = np.delete(y_tr, padded_rows, axis=0)

# Prepare test data
well_ts = test_data['Well Name'].values
depth_ts = test_data['Depth'].values
X_ts = test_data[feature_names].values

# Augment features
X_ts, padded_rows = augment_features(X_ts, well_ts, depth_ts)

# Predict test labels
y_ts_hat = train_and_test(X_tr, y_tr, X_ts, well_ts)

# Save predicted labels
test_data['Facies'] = y_ts_hat
test_data.to_csv('Prediction_XXI_LM_Final.csv')


I am working on the file "LA_Team/Facies_classification_LA_TEAM_07.ipynb".
The first task is:
Data Preprocessing
Can you write Python code for it?

#Load Data
data = pd.read_csv('../facies_vectors.csv')

# Parameters
feature_names = ['GR', 'ILD_log10', 'DeltaPHI', 'PHIND', 'PE', 'NM_M', 'RELPOS']
facies_names = ['SS', 'CSiS', 'FSiS', 'SiSh', 'MS', 'WS', 'D', 'PS', 'BS']
facies_colors = ['#F4D03F', '#F5B041','#DC7633','#6E2C00', '#1B4F72','#2E86C1', '#AED6F1', '#A569BD', '#196F3D']

print(data.head())
data['PE'] = data.groupby("Facies").PE.transform(lambda x: x.fillna(x.mean()))


I am working on the file "LA_Team/Facies_classification_LA_TEAM_07.ipynb".
The first task is:
Prediction
Can you write Python code for it?

#Load testing data
test_data = pd.read_csv('../validation_data_nofacies.csv')

# Prepare training data
X_tr = X
y_tr = y

# Augment features
X_tr, padded_rows = augment_features(X_tr, well, depth)

# Removed padded rows
X_tr = np.delete(X_tr, padded_rows, axis=0)
y_tr = np.delete(y_tr, padded_rows, axis=0)

# Prepare test data
well_ts = test_data['Well Name'].values
depth_ts = test_data['Depth'].values
X_ts = test_data[feature_names].values

# Augment features
X_ts, padded_rows = augment_features(X_ts, well_ts, depth_ts)

# Predict test labels
y_ts_hat = train_and_test(X_tr, y_tr, X_ts, well_ts)

# Save predicted labels
test_data['Facies'] = y_ts_hat
test_data.to_csv('Prediction_XX_Final.csv')


I am working on the file "LA_Team/Facies_classification_LA_TEAM_07.ipynb".
The first task is:
We procceed to run Paolo Bestagini's routine to include a small window of values to acount for the spatial component in the log analysis, as well as the gradient information with respect to depth. This will be our prepared training dataset.
Can you write Python code for it?

X_train, X_test, y_train, y_test = train_test_split(X_reg, y_reg, train_size=0.75, test_size=0.25)

# Store features and labels
X = data[feature_names].values
y = data['Facies'].values

# Store well labels and depths
well = data['Well Name'].values
depth = data['Depth'].values

# Feature windows concatenation function
def augment_features_window(X, N_neig):

    # Parameters
    N_row = X.shape[0]
    N_feat = X.shape[1]

    # Zero padding
    X = np.vstack((np.zeros((N_neig, N_feat)), X, (np.zeros((N_neig, N_feat)))))

    # Loop over windows
    X_aug = np.zeros((N_row, N_feat*(2*N_neig+1)))
    for r in np.arange(N_row)+N_neig:
        this_row = []
        for c in np.arange(-N_neig,N_neig+1):
            this_row = np.hstack((this_row, X[r+c]))
        X_aug[r-N_neig] = this_row

    return X_aug

# Feature gradient computation function
def augment_features_gradient(X, depth):

    # Compute features gradient
    d_diff = np.diff(depth).reshape((-1, 1))
    d_diff[d_diff==0] = 0.001
    X_diff = np.diff(X, axis=0)
    X_grad = X_diff / d_diff

    # Compensate for last missing value
    X_grad = np.concatenate((X_grad, np.zeros((1, X_grad.shape[1]))))

    return X_grad

# Feature augmentation function
def augment_features(X, well, depth, N_neig=1):

    # Augment features
    X_aug = np.zeros((X.shape[0], X.shape[1]*(N_neig*2+2)))
    for w in np.unique(well):
        w_idx = np.where(well == w)[0]
        X_aug_win = augment_features_window(X[w_idx, :], N_neig)
        X_aug_grad = augment_features_gradient(X[w_idx, :], depth[w_idx])
        X_aug[w_idx, :] = np.concatenate((X_aug_win, X_aug_grad), axis=1)

    # Find padded rows
    padded_rows = np.unique(np.where(X_aug[:, 0:7] == np.zeros((1, 7)))[0])

    return X_aug, padded_rows

X_aug, padded_rows = augment_features(X, well, depth)

# Initialize model selection methods
lpgo = LeavePGroupsOut(2)

# Generate splits
split_list = []
for train, val in lpgo.split(X, y, groups=data['Well Name']):
    hist_tr =...


I am working on the file "LCOS_pattern/LCOS Pattern.ipynb".
The first task is:
and plot it:
Can you write Python code for it?

fig, ax = plt.subplots(figsize=(8, 5))
im = plt.imshow(a, interpolation='none', cmap='magma',
                vmin=0, vmax=255)#, norm=mpl.colors.LogNorm(1, 255))
plt.colorbar()
#plt.tight_layout()

fig, ax = plt.subplots(figsize=(7, 4))
im = plt.imshow(a, interpolation='none', cmap='Greys_r',
                vmin=0, vmax=255)#, norm=mpl.colors.LogNorm(1, 255))
plt.colorbar()
#plt.tight_layout()
plt.xlim(480, 320)
plt.ylim(240, 360)
ax.get_yaxis().set_tick_params(direction='out')
ax.get_xaxis().set_tick_params(direction='out')
plt.savefig('../figures/SI_figure_LCOS_pattern.png', bbox_inches='tight', dpi=300)


I am working on the file "LCOS_pattern/Pattern Roto-Translation Documentation.ipynb".
The first task is:
Apart from the computing the spot regions, the crucial point of computing the multispot
pattern is computing the distance
of each LCOS pixel from the spot center.
Working in the LCOS frame of reference, we can compute the
spot (centers) coordinates as:
Can you write Python code for it?

xc, yc = spotgrid_centers(nrows, ncols, pitch_x=pitch_x, pitch_y=pitch_y,
                          center_x=center_x, center_y=center_y, rotation=rotation)


I am working on the file "LCOS_pattern/Pattern Roto-Translation Documentation.ipynb".
The first task is:
By convention, the LCOS grid is centered on the LCOS pixel (400, 300):
Can you write Python code for it?

LCOS_X_SIZE//2, LCOS_Y_SIZE//2

assert (XL[:, LCOS_X_SIZE//2] == 0).all()
assert (YL[LCOS_Y_SIZE//2] == 0).all()


I am working on the file "LCOS_pattern/Pattern Roto-Translation Documentation.ipynb".
The first task is:
Pattern parameters:
Can you write Python code for it?

# Pattern shape
ncols = 3
nrows = 2

# Coordinates of the LCOS center in the pattern space
center_x = 100
center_y = 200

# Rotation of the LCOS grid versus the pattern space
rotation = 20

# Pitch for the multi-spot pattern
pitch_x = 80
pitch_y = 60


I am working on the file "LCOS_pattern/Pattern Roto-Translation Documentation.ipynb".
The first task is:
The most natural approach is working on the LCOS reference frame and moving
the pattern function with a
rigid transformation
to the desired position and rotation.
The LCOS grid (i.e. pixels) samples this function and the pattern is obtained.
This is conceptually straightforward but, practically computing the rectangular
region for each spot is not trivial when we have a non-zero rotation.
Equivalently, we can think as the LCOS pattern is the result of sampling
a pattern "function" defined in the pattern frame of reference
(centered on its origin and not rotated).
To obtain an LCOS pattern with different translations and rotations,
the LCOS grid is rigidly moved to sample the
(fixed) function at different positions and orientations.
This is mathematically equivalent to the first approach
but makes it trivial to compute
the spot regions simply by using the floor function.
To calculate the rectangular regions assigned to each spot we work
in a Cartesian frame of reference rigidly attached to the centered of the pattern.
In this frame of reference the spot regions can be computed using
a simple floor function if the coordinates are scaled in pitch units.
The function spotmap_func take a pair of coordinate (x, y)
and return the spot-number which contains the point. If (x, y) does not fall
in the region of any spot NaN is returned.
The trick is feeding spotmap_func with the coordinates of a
rigidly transformed LCOS grid.
The (XL, YL) grid is first translated
and then rotated with respect to its center:
Can you write Python code for it?

XLtr, YLtr = rotate((XL - center_x), (YL - center_y), angle=-rotation)
Z_spots = spotmap_func((XLtr, YLtr), (pitch_x, pitch_y), (ncols, nrows))


I am working on the file "LCOS_pattern/Pattern Roto-Translation Documentation.ipynb".
The first task is:
The pattern is computed spot by spot. For each spot we create a mask
of pixels defining the spot region. Then, we compute the distance from
the spot center for each pixel of the region.
The correct grid to use is (XL, YL) in the LCOS frame of reference .
Can you write Python code for it?

pattern = np.zeros_like(Z, dtype=float)
for ispot in range(nrows * ncols):
    mask = Z_spots == ispot
    x0, y0 = xc.ravel()[ispot], yc.ravel()[ispot]
    radius = np.sqrt((XL[mask] - x0)**2 + (YL[mask] - y0)**2)
    pattern[mask] = radius


I am working on the file "LCOS_pattern/Pattern Roto-Translation Documentation.ipynb".
The first task is:
We defines the grid (XL, YL) of LCOS pixel centers in the LCOS frame of reference
with origin in the LCOS center and in pixel units:
Can you write Python code for it?

xl = np.arange(LCOS_X_SIZE) - LCOS_X_SIZE // 2
yl = np.arange(LCOS_Y_SIZE) - LCOS_Y_SIZE // 2
YL, XL = np.mgrid[:LCOS_Y_SIZE, :LCOS_X_SIZE]
XL -= LCOS_X_SIZE // 2
YL -= LCOS_Y_SIZE // 2


I am working on the file "LCOS_pattern/Pattern Roto-Translation Documentation.ipynb".
The first task is:
Z_spots (the 2D array of labeled spot regions) was computed using the inverse rigid transformation of the LCOS grid in the pattern frame of reference. The spot centers, instead, are computed with a direct rigid transformation in the LCOS frame of reference. The two transformations are equivalent, as can bee seen plotting the spot centers and the spot region (on the LCOS frame of reference):
Can you write Python code for it?

Z, x, y = Z_spots, xc, yc
fig, ax = plt.subplots(figsize=(8, 6))
im = ax.imshow(Z, interpolation='none', cmap='viridis', aspect='equal',
               extent=(-LCOS_X_SIZE/2, LCOS_X_SIZE/2, LCOS_Y_SIZE/2, -LCOS_Y_SIZE/2))
ax.scatter(x, y, color='w', edgecolors='k', lw=0.3)


I am working on the file "LC_python/LC_MLtrain.ipynb".
The first task is:
Great! There are the original continuous features, and now a feature for all types of home ownership, purposes for the loan, and different address states!
Shuffle and split data
Specifically, I believe that this data is ordered by the origination date, so it is important to shuffle the data. Luckily this happens anyway with train_test_split.
Can you write Python code for it?

# Import train_test_split
from sklearn.model_selection import train_test_split #sklearn 0.18.1 and up
# Split the 'features' and 'income' data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(feat, labels, test_size = 0.2, random_state = 1)

# Show the results of the split
print ("Training set has {} samples.".format(X_train.shape[0]))
print ("Testing set has {} samples.".format(X_test.shape[0]))


I am working on the file "LC_python/LC_MLtrain.ipynb".
The first task is:
I guess these are pretty understandable, high paying jobs and income.
Lets check the utilizations - having over 100% seems... not possible.
Can you write Python code for it?


I am working on the file "LC_python/LC_MLtrain.ipynb".
The first task is:
Interestingly, this doesn't really improve the precision, there may not be enough bad loan data to give the model something to hang on to, and potentially, it could be that we just don't have the data collected that would be more predictive. Because it is crazy simple, and it actually performs better at reducing false positives than others, at least with this training/test set. It is worth noting that the 0.0095% increase in accuracy over the naive classifier is very small but better than nothing.
Can you write Python code for it?

from sklearn.model_selection import cross_val_score
scores = cross_val_score(best_clf,X_train, y_train, cv=20,scoring=scorer, n_jobs=-1)
scores

difference=scores.mean()-prec
import math
from scipy import stats,special
zscore=difference/(scores.std())
p_values = 1-special.ndtr(zscore)
print(zscore,p_values)

(scores.std()/math.sqrt(10))

scores.std()*100

import joblib
joblib.dump(clf, 'decisiontree.p')

features.columns

# from sklearn import tree
# import pydotplus
# dot_data = tree.export_graphviz(best_clf, out_file=None,feature_names=feat.columns,class_names=['Bad Loan','Fully Paid'])
# graph = pydotplus.graph_from_dot_data(dot_data)
# graph.write_pdf("LC_decisiontree.pdf")

best_clf.feature_importances_

importances = best_clf.feature_importances_

# Plot
import importlib
importlib.reload(vs)
vs.feature_plot(importances, X_train, y_train)

best_clf.tree_.node_count

indices = np.argsort(importances)[::-1]
columns = X_train.columns.values[indices[:10]]
values = importances[indices][:10]
np.cumsum(values)
values
columns


I am working on the file "LC_python/LC_MLtrain.ipynb".
The first task is:
One Hot Encoding
Up to this point, I have only dealt with the "continuous" numerical features, or features where the values have some relation. for example, higher income can be directly compared to lower income. Now I want to deal with discrete features, where values are independent. One example is the address state. Although we all might have a specific way to rank the states, there is no "correct" relationship between any two possible states - they are discrete and independent. These will be one-hot encoded!
Of note is the employment title - this is put in by the person applying to the loan and can be pretty much anything. This will certainly cause an explosion in feature space. I think, at this point, I will just remove that from the feature space instead of encode it.
Can you write Python code for it?

features_raw

features_raw=features_raw.drop(['emp_title','addr_state'],axis=1)

features_raw.dtypes

# One-hot encode the 'features_raw' data using pandas.get_dummies()
feat = pd.get_dummies(features_raw)

#print(income.head(n=10))
# Print the number of features after one-hot encoding
encoded = list(feat.columns)
print ("{} total features after one-hot encoding.".format(len(encoded)))

# Uncomment the following line to see the encoded feature names
print (encoded)


I am working on the file "LC_python/LC_MLtrain.ipynb".
The first task is:
Overfitting with Decision Trees - Success?
Looks like we have classic high variance now (great training scores, terrible test scores)! This makes me more confident that the metric is tenable.
Gridsearch Decision Tree model tuning
Now I'll try to tune the decision tree using gridsearch CV.
Can you write Python code for it?

from sklearn.model_selection import GridSearchCV
from sklearn.metrics import make_scorer
# Initialize the classifier
clf = DecisionTreeRegressor(random_state=2)

# Create the parameters list you wish to tune
parameters = {'max_depth':(4,6,20,None),'min_samples_split':(2,50),'min_samples_leaf':(1,101)} #',,

# Make an R2_score scoring object
scorer = make_scorer(r2_score)

# TODO: Perform grid search on the classifier using 'scorer' as the scoring method
grid_obj = GridSearchCV(clf,parameters,scoring=scorer,n_jobs=-1)

# TODO: Fit the grid search object to the training data and find the optimal parameters
grid_fit = grid_obj.fit(X_train,y_train)

# Get the estimator
best_clf = grid_fit.best_estimator_

# Make predictions using the unoptimized and model
predictions = (clf.fit(X_train, y_train)).predict(X_test)
best_predictions = best_clf.predict(X_test)
# Report the before-and-afterscores
print ("Unoptimized model\n------")
print ("MSE score on testing data: {:.4f}".format(mean_squared_error(y_test, predictions)))
print ("R2-score on testing data: {:.4f}".format(r2_score(y_test, predictions)))
print ("\nOptimized Model\n------")
print ("Final MSE score on the testing data: {:.4f}".format(mean_squared_error(y_test, best_predictions)))
print ("Final R2-score on the testing data: {:.4f}".format(r2_score(y_test, best_predictions)))

best_clf

# from sklearn.model_selection import GridSearchCV
# from sklearn.metrics import make_scorer
# # Initialize the classifier
# clf = GradientBoostingRegressor(random_state=2)

# # Create the parameters list you wish to tune
# parameters = {'learning_rate':(0.01,0.1), 'max_depth':(3,6)}

# # Make an R2_score scoring object
# scorer = make_scorer(r2_score)

# # TODO: Perform grid search on the classifier using 'scorer' as the scoring method
# grid_obj = GridSearchCV(clf,parameters,scoring=scorer,n_jobs=-1)

# # TODO: Fit the grid search object to the training data and find the optimal parameters
# grid_fit = grid_obj.fit(X_train,y_train)

# # Get the estimator
# best_clf =...


I am working on the file "LC_python/LC_MLtrain.ipynb".
The first task is:
Regression Postmortem
Woof, terrible bias, almost no structure is described by these models. I'm actually not sure what to make of such extreme values, but negative R^2 are bad, and large MSE is bad. Likely, they are too simple. I'd like to try and overfit, before I move towards a more modest approach, such as classification. If I can tune a decision tree from overfitting towards something useful, maybe the metric and this regression still have hope.
Attempting to overfit.
For this I will use a standard decision tree, and attempt to overfit with hopes for Post-pruning
Can you write Python code for it?

from sklearn.tree import DecisionTreeRegressor
results2={}
clf_D = DecisionTreeRegressor(random_state=1)
clf_D.fit(X_train,y_train)
predictions_test = clf_D.predict(X_test)
predictions_train = clf_D.predict(X_train[:3000])
end = time() # Get end time

# Calculate the total prediction time

# Compute mean square error on the first 300 training samples
results2['mse_train'] = mean_squared_error(y_train[:3000],predictions_train)

# Compute mean square error on test set
results2['mse_test'] = mean_squared_error(y_test,predictions_test)

# Compute R^2 on the the first 300 training samples
results2['R2_train'] = r2_score(y_train[:3000],predictions_train)

# Compute R^2 on the test set
results2['R2_test'] = r2_score(y_test,predictions_test)

results2


I am working on the file "LC_python/LC_MLtrain.ipynb".
The first task is:
Results
This is promising! The decision tree unsurprisingly shows the best scores on the testing set. I'll try to optimize decision trees next! I will attempt to prune the tree a bit with grid search.
Can you write Python code for it?

# Initialize the classifier
clf = DecisionTreeClassifier(random_state=3)

# Create the parameters list you wish to tune
parameters = {'max_depth':(20,50, None),'max_leaf_nodes':(100,300,None)}

# Make an R2_score scoring object
scorer = make_scorer(precision_score)

# TODO: Perform grid search on the classifier using 'scorer' as the scoring method
grid_obj = GridSearchCV(clf,parameters,scoring=scorer,n_jobs=-1)

# TODO: Fit the grid search object to the training data and find the optimal parameters
grid_fit = grid_obj.fit(X_train,y_train)

# Get the estimator
best_clf = grid_fit.best_estimator_

# Make predictions using the unoptimized and model
predictions = (clf.fit(X_train, y_train)).predict(X_test)
best_predictions = best_clf.predict(X_test)

# Report the before-and-afterscores
print ("Unoptimized model\n------")
print ("ROC AUC score on testing data: {:.4f}".format(roc_auc_score(y_test, predictions)))
print ("Precision on testing data: {:.4f}".format(precision_score(y_test, predictions)))
print ("\nOptimized Model\n------")
print ("Final ROC AUC score on the testing data: {:.4f}".format(roc_auc_score(y_test, best_predictions)))
print ("Final Precision on the testing data: {:.4f}".format(precision_score(y_test, best_predictions)))


I am working on the file "LC_python/LC_MLtrain.ipynb".
The first task is:
Some weird scaling going on, just to check my sanity, check the max values of a few of the features. these may need to be log scaled just to deal with the large values. Income almost always is going to need log scaling anyway.
Can you write Python code for it?

valuespread={'annual_inc':max(features.annual_inc),'delinqmax':max(features.delinq_2yrs),
             'openmax':max(features.open_acc), 'pubmax':max(features.pub_rec),
             'revolbalmax':max(features.revol_bal),'revolutilmax':max(features.revol_util),
             'collectmax':max(features.collections_12_mths_ex_med)}
valuespread


I am working on the file "LC_python/LC_MLtrain.ipynb".
The first task is:
Testing different models
For almost all supervised learning, a version of Decision trees is necessary to test, if only for it's extreme intuitiveness. At the very least, it may be used to shrink the feature space and improve speed for other algorithms. I will use Gradient Boosting regression for this. Secondly, I want to test a kernel based non-linear regression, and I will use KNN for this. Finally I want to test a high order polynomial regression.
defining model tester
Can you write Python code for it?

from sklearn.metrics import r2_score, mean_squared_error
def train_predict(learner, sample_size, X_train, y_train, X_test, y_test):
    '''
    inputs:
       - learner: the learning algorithm to be trained and predicted on
       - sample_size: the size of samples (number) to be drawn from training set
       - X_train: features training set
       - y_train: income training set
       - X_test: features testing set
       - y_test: income testing set
    '''

    results = {}

    # Fit the learner to the training data using slicing with 'sample_size'
    start = time() # Get start time
    learner = clf.fit(X_train.sample(n=sample_size,random_state=1),y_train.sample(n=sample_size,random_state=1)) #df.sample(frac=percent) maybe too?
    end = time() # Get end time

    # Calculate the training time
    results['train_time'] = end-start

    # Get the predictions on the test set,
    #       then get predictions on the first 300 training samples
    start = time() # Get start time
    predictions_test = clf.predict(X_test)
    predictions_train = clf.predict(X_train[:300])
    end = time() # Get end time

    # Calculate the total prediction time
    results['pred_time'] = end-start

    # Compute mean square error on the first 300 training samples
    results['mse_train'] = mean_squared_error(y_train[:300],predictions_train)

    # Compute mean square error on test set
    results['mse_test'] = mean_squared_error(y_test,predictions_test)

    # Compute R^2 on the the first 300 training samples
    results['R2_train'] = r2_score(y_train[:300],predictions_train)

    # Compute R^2 on the test set
    results['R2_test'] = r2_score(y_test,predictions_test)

    # Success
    print ("{} trained on {} samples.".format(learner.__class__.__name__, sample_size))

    # Return the results
    return results

from sklearn.svm import SVR
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.linear_model import LinearRegression,...


I am working on the file "LC_python/LC_MLtrain.ipynb".
The first task is:
There are some crazy revolving balances and utilizations. I need to check those quickly. Everything else looks extreme, but within ranges I'd assume could exist.
Can you write Python code for it?


I am working on the file "LC_python/LC_MLtrain.ipynb".
The first task is:
Utilization looks much more normal!
Now to log transform the skewed distributions.
Log transform Skewed data
Can you write Python code for it?

# Log-transform the skewed features

skewed = ['annual_inc','delinq_2yrs','open_acc', 'pub_rec','revol_bal','total_acc', 'collections_12_mths_ex_med']
features_raw=features.copy()
features_raw[skewed] = features[skewed].apply(lambda x: np.log(x + 1))

# Visualize the new log distributions
imp.reload(vs)
vs.distribution(features_raw, transformed = True)


I am working on the file "LC_python/LC_MLtrain.ipynb".
The first task is:
We see a clear improvement in income, open accounts, and revolving balance. The others are improved but still are dominated by low numbers. That's okay for now. There are extreme outliers but they are important. I am tempted to winsorize, but the values are so heavily weighted toward 0, that the winsorization would affect all values greater than zero.
Can you write Python code for it?

# import scipy.stats
# winsors=['delinq_2yrs','pub_rec','collections_12_mths_ex_med']
# for feature in winsors:
#     winsor[feature]=scipy.stats.mstats.winsorize(features[feature], limits=0.01,axis=1)


I am working on the file "LDA extract topics.ipynb".
The first task is:
efforts with pyLDAvis (visualize the LDA topics)
Can you write Python code for it?

import time

import pyLDAvis.gensim
pyLDAvis.enable_notebook()

#load the LDA results (model, dictionary and corpus)
start_time = time.time()

ldamodel = pickle.load(open('trained models/lda/allstates_10topics_100passes.pickle'))
dictandcorpus = pickle.load(open('trained models/lda/Dictionary.pickle'))
c = dictandcorpus[1]
d = dictandcorpus[0]
del dictandcorpus

elapsed_time = time.time() - start_time
print elapsed_time

data = pyLDAvis.gensim.prepare(ldamodel, c, d)

data


I am working on the file "LDA1.ipynb".
The first task is:
Aufteilen der Daten in Test- und Trainingsdaten
Can you write Python code for it?

test_frac = 0.2
train_set, test_set = get_lda_data(df_prod, test_frac=test_frac)


I am working on the file "LDA1.ipynb".
The first task is:
Auswahl des Produtes durch Schieberegler implementieren
TODO Produkteigenschaften (LG, AD, WD) ausgeben
Can you write Python code for it?

@interact(index=IntSlider(min=0, max=25, value = 11))
def count_per_product(index):
    print("Anzahl der Walzlose: "+str(len(pd.unique(df_all_prod[index]["Header_Walzlos"]))))


I am working on the file "LDA1.ipynb".
The first task is:
Darstellung der Eigenvektoren
Can you write Python code for it?

plt.figure(figsize=(20,10))

for index in range(3):
    ax = plt.subplot(1,3,index+1)
    ax.stem(eigvecs[:, index])
    ax.set_title('Eigenvektor {}'.format(index))
    ax.set_xlabel('Merkmalsindex')
    ax.set_ylabel('Beitrag in Eigenvektor')


I am working on the file "LDA1.ipynb".
The first task is:
Darstellung der Eigenwerte
Can you write Python code for it?

plt.stem(sklearn_LDA.explained_variance_ratio_)
plt.xlabel('Index Eigenwert')
plt.ylabel('Beitrag zur Varianz')
plt.title("Varainzverteilung")


I am working on the file "LDA1.ipynb".
The first task is:
Darstellung der transformierten Trainingsdaten und Klassenzugehörigkeit
Can you write Python code for it?

data = sklearn_LDA.transform(X_train)
plot_lda(data, y_train, 'Transformierte Trainingsdaten')


I am working on the file "LDA1.ipynb".
The first task is:
Dürchführen der LDA auf die Trainingsdaten
Can you write Python code for it?

# extract data and label
X_train, y_train = train_set['data'], train_set['label']
X_test, y_test = test_set['data'], test_set['label']

# number components for transform
n_components = 3

# LDA object
sklearn_LDA = LDA(n_components=n_components, solver='eigen')

# fit with train data
sklearn_LDA = sklearn_LDA.fit(X_train, y_train)


I am working on the file "LDA1.ipynb".
The first task is:
Laden der Merkmalsmatrix
und Vorverarbeitung von Zeilen und Spalten nach Anzahl von NaNs und stark korrelierenden Merkmalen
Can you write Python code for it?


I am working on the file "LDA1.ipynb".
The first task is:
Normalisierung der Daten
Can you write Python code for it?

train_set['data'] = zscore(train_set['data'])
test_set['data'] = zscore(test_set['data'])


I am working on the file "LDA1.ipynb".
The first task is:
Produktweise Sortierung der Daten
minimale Anzahl der Rohre je Walzlos festlegen, um die Schätzung je Walzlos zu verbessern,
und Daten nach Produkten sortieren
Can you write Python code for it?

min_num_walzlos = 300
df_all_prod = [extract_product(df, product_id=product_id, min_num_walzlos=min_num_walzlos) for product_id in range(26)]


I am working on the file "LDA1.ipynb".
The first task is:
Testen der Klassifikation
Can you write Python code for it?

train_pred = sklearn_LDA.predict(X_train)
print('{0:.2f}% Genauigkeit bei der Klassifikation der Trainingsdaten'.format(100*np.mean(train_pred == y_train)))

test_pred = sklearn_LDA.predict(X_test)
print('{0:.2f}% Genauigkeit bei der Klassifikation der Testdaten'.format(100*np.mean(test_pred == y_test)))


I am working on the file "LDA_Final.ipynb".
The first task is:
 Latent Dirichlet Allocation

Theory

Implementation

Inspection

Theory
The raw output of the count vectorizer is too high dimensional (~10,000) to be particularly useful, especially because we find that no individual n-gram has a high signal-to-noise ratio. Therefore, we would like to perform some sort of dimensional reduction. One approach is to find the average sentiment of all words using the SentiWordNet dictionary. It's easy to argue that the sentiment of words in the business section reflects feelings about the economy. Another approach is topic modeling with Latent Dirichlet Allocation (LDA). While topics are not obviously correlated with CCI, it extracts interpretable information from the documents. That the topics generated from LDA are likely not very correlated to SentiWordNet scores makes it even more compelling to use both means of dimensional reduction.
For LDA, we posit that the articles are generated by randomly pulling words from a mixture of topics and that the topics are defined by the unigrams and bigrams that they contain. After random initialization, the algorithm iterates through words and reassigns words to topics based on how often they occur in articles of that topic. Eventually, the allocation becomes more self-consistent as words move to topics where they are more common and articles become more concentrated under a few topics. As opposed to some clustering algorithms, the documents are assigned to a mixture of topics since the same words tend to occur across many articles so the articles are not fully separable.
Implementation
The actual implementation of LDA is fairly complicated because there is a lot of iteration and sampling from distributions. However, it is already nicely implemented in sklearn. Still, it is not immediately clear how many topics one should use in LDA. To get an idea, we can inspect the...
Can you write Python code for it?

def save_sparse_csr(filename,array):
    np.savez(filename,data = array.data ,indices=array.indices,
             indptr =array.indptr, shape=array.shape )

def load_sparse_csr(filename):
    loader = np.load(filename)
    return scipy.sparse.csr_matrix((  loader['data'], loader['indices'], loader['indptr']),
                         shape = loader['shape'])

wordMatrixBigrams = load_sparse_csr('bigramWordMatrix4.npz')

num_topics=8
ldaBigrams = LatentDirichletAllocation(n_topics=num_topics)

ldaDocsBigrams = ldaBigrams.fit_transform(wordMatrixBigrams)

np.save('ldaDocBigramScores4_8', ldaDocsBigrams)


I am working on the file "LDA_Final.ipynb".
The first task is:
As we can see, the topics seem to make some sense. For example, Topic 6 clearly has to do with "healthcare", "pharmaceuticals", and "medicine", but it is also interesting to note that "Canada" comes up a lot in those articles. This is not surprising at all as their system of socialized medicine is often referenced in the US. Also, Topic 8 has to do more with monetary policy as it includes "Euro", "Pound", "money", and "Federal Reserve". Something particularly interesting about Topic 8 is that "bank" does not appear. While I assume that "Federal", "Reserve", and "Bank" all appear frequently in that order, "bank" is more likely to occur elsewhere in the business section, not just regarding monetary policy. This indicates how LDA does not just group words that occur in articles together, it weights against including words that occur frequently across all documents.
Finally, we can group by month and save the topic scores.
Can you write Python code for it?

data = pd.read_csv('allDataWithStems.csv', index_col=0)

grouped = data.groupby('yearmonth')

topicsByMonthBigrams = np.zeros((len(grouped.groups.keys()),ldaDocsBigrams.shape[1]))
for i, month in enumerate(np.sort(grouped.groups.keys())):
    topicsByMonthBigrams[i] = np.mean(ldaDocsBigrams[grouped.get_group(month).index], axis=0)

np.save('topicsByMonthBigrams4_8', topicsByMonthBigrams)


I am working on the file "LDL_transport_model.ipynb".
The first task is:
Assembling and solving the discrete system
The LDL_Sim class is responsible for solve the differential equation.
In the first step the simulation region is discretized by the method discretize. In that function flux continuity between layers and boundary conditions are implemented. It is done in a following way:

the layers are encoded as list layers in discretize, containing indices of boundaries for given discretization

system parameters depending on space ($k,\vec u,\sigma,D_{eff}$) are sampled

the diagonals of matrix are assembled using finite differences in space, neglecting for a moment the space dependence

at regions boudaries the equation is replaced with
  $$J_L = J_R$$
  it takes a form for a boundary at index j:
  $$ c_j v_{j-1} - D_{j-1}\frac{c_j-c_{j-1}}{dx} =  c_j v_{j+1} - D_{j+1}\frac{c_{j+1}-c_{j}}{dx}  $$
  collecting terms with the same $c$:
  $$ c_j \left( v_{j-1}- v_{j+1}  - \frac{D_{j-1}+D_{j+1}}{dx}\right) + c_{j-1}\frac{D_{j-1}}{dx} + c_{j+1}\frac{D_{j+1}}{dx} = 0
    $$

therefore we need to enforce the above equation on the system for each boundary inside the domain

the system matrix is stored as scipy.sparse dia_matrix

The plot_c function is used for plot the concentration profiles.
Can you write Python code for it?

class LDL_Sim(object):

    def __init__(self, pars):
        self.pars = pars
        self.c_st = None
    def discretize(self,N=2000):
        self.N = N
        k = np.ones(N)
        v = np.ones(N)
        Dyf = np.ones(N)

        D,V,sigma,L,k_react = self.pars.get_params()
        l = np.sum(L)
        self.l = l
        self.x=np.linspace(0,l,N)

        layers=[0]+list( np.ceil( (N*(np.cumsum(L)/sum(L)))).astype(np.int32) )

        for i,(l1,l2) in enumerate(zip(layers[:],layers[1:])):
            k[l1:l2] = k_react[i]
            v[l1:l2] = (1.0-sigma[i])*V[i]
            Dyf[l1:l2] = D[i]
        dx2_1 = (N-1)**2/l**2
        dx_1 = (N-1)/l

        diag_l = np.ones(N)*(np.roll(Dyf,-1)*dx2_1)
        diag   = np.ones(N)*(-2.*Dyf*dx2_1 - k + v*dx_1)
        diag_u = np.ones(N)*(np.roll(Dyf,1)*dx2_1 - np.roll(v,1)*dx_1)

        # Layer's junctions
        for j in layers[1:-1]:
            diag[j] = v[j-1]-v[j+1]-(Dyf[j-1]+Dyf[j+1])*dx_1
            diag_l[j-1] = Dyf[j-1]*dx_1
            diag_u[j+1] = Dyf[j+1]*dx_1
        #Boundary Conditions
        diag[0] = 1
        diag[-1] = 1
        diag_u[0+1] = 0
        diag_l[0-2] = 0

        self.L = dia_matrix((np.array([diag_l,diag,diag_u]),np.array([-1,0,1])), shape=(N,N))

    def solve_stationary(self,bc=[1,0]):
        b = np.zeros(self.N)
        b[0],b[-1] = bc
        L = self.L.tocsr()
        self.c_st = sp.sparse.linalg.linsolve.spsolve(L,b)

    def plot_c(self,yrange=(0,0.2),xrange=(0,214),filename=None, color='red', alpha=0.2, style='-'):

        i1,i2 = int(xrange[0]/self.l*self.N),int(xrange[1]/self.l*self.N)
        plt.plot(self.x[i1:i2],self.c_st[i1:i2],color=color,linewidth=2, ls=style)
        plt.ylim( *yrange)
        plt.xlim( *xrange)
        L=self.pars.L
        d=[0]+np.cumsum(self.pars.L).tolist()
        colors=['m','g','b','w']
        for i,(l1,l2) in enumerate(zip(d[:],d[1:])):

            plt.bar([l1,],yrange[1],l2-l1, color=colors[i], linewidth=0.3, ...


I am working on the file "LDL_transport_model.ipynb".
The first task is:
Class that performs simulation
In that class the whole simulation is performed: parameters are initiated and then the equation is solved. As a result we get the object that contains the values of LDL concentration in discretized points and the function plot_c which can plot the concentration profile.
Class can be iitiated with name of one of the parameters sets:

"4L_2012" for S. Chung, K. Vafai, International Journal of Biomechanics 45(2012)
"4L_2006_Ai" for L. Ai, K. Vafai, International Journal of Heat and Mass Transfer 49 (2006) with D(WSS)
"4L_2006_Ai_without_D" for L. Ai, K. Vafai, International Journal of Heat and Mass Transfer 49 (2006) with D from this publication
"2L" for U. Olgac, V. Kurtcuoglu, D. Poulikakos, American Journal of Physiology-Heart and
Circulatory Physiology 294
Can you write Python code for it?

def LDL_simulation(wss=1.79, parameters="2012", bc=[1,0.0047],verbose=True):
    if (parameters=="4L_2012"):
        pars = LDL_Parameters_Vafai2012(WSS=wss)
    elif (parameters=="4L_2006_Ai"):
        pars = LDL_Parameters_Vafai2006_Ai(WSS=wss)
    elif (parameters=="4L_2006_Ai_without_D"):
        pars = LDL_Parameters_Vafai2006_Ai_without_D(WSS=wss)
    elif (parameters=="2L"):
        pars = LDL_Parameters_Olgac_WSS(WSS=wss)
    else:
        print "Parameters error"
        return
    sim = LDL_Sim(pars)
    sim.discretize(130*214)
    sim.solve_stationary(bc=bc)
    if verbose:
        print "The total surfaced LDL concentration:",np.sum(sim.c_st)*(sim.l/(sim.N-1))
    return sim


I am working on the file "LDL_transport_model.ipynb".
The first task is:
Concentration in intima in function of WSS
Can you write Python code for it?

c_endo_wss=[LDL_simulation(wss=x, parameters="2L",verbose=False).c_st[2*130] for x in WSSs]

plot (WSSs,c_endo_wss)
xlim([0.0,1.6])
ylim([0,0.3])
xlabel('WSS [Pa]')
ylabel('$c_{w}end^{*}$')
grid(True)
plt.title("WSS dependence of intima side LDL concentration at the endothelium", fontsize=10)


I am working on the file "LDL_transport_model.ipynb".
The first task is:
Examples
In this section the results presented in publications are calculated
Small WSS=0.02
Can you write Python code for it?

LDL_simulation(wss=0.02, parameters="4L_2012").plot_c(yrange=(0,5.0),xrange=(0,214), color='green', alpha=0.1)
LDL_simulation(wss=0.02, parameters="4L_2006_Ai").plot_c(yrange=(0,5.0),xrange=(0,214), color='blue', alpha=0.1, style='--')
legend(('4LC parametrs','4LA parametrs'));

LDL_simulation(wss=0.02, parameters="4L_2012").plot_c(yrange=(0,5.0),xrange=(0,25), color='green', alpha=0.1)
LDL_simulation(wss=0.02, parameters="4L_2006_Ai").plot_c(yrange=(0,5.0),xrange=(0,25), color='blue', alpha=0.1, style='--')
legend(('4LC parametrs','4LA parametrs'));


I am working on the file "LDL_transport_model.ipynb".
The first task is:
High WSS=2.2
Can you write Python code for it?

sim = LDL_simulation(wss=2.2, parameters="4L_2012")
sim.plot_c(yrange=(0,1.1),xrange=(0,214), color='green', alpha=0.1)
sim = LDL_simulation(wss=2.2, parameters="4L_2006_Ai")
sim.plot_c(yrange=(0,1.1),xrange=(0,214), color='blue', alpha=0.1, style='--')
legend(('4LC parametrs','4LA parametrs'));
# use e.g. xlim(0,10) to zoom

sim = LDL_simulation(wss=2.2, parameters="4L_2012")
sim.plot_c(yrange=(0,1.1),xrange=(0,25), color='green', alpha=0.1)
sim = LDL_simulation(wss=2.2, parameters="4L_2006_Ai_without_D")
sim.plot_c(yrange=(0,1.1),xrange=(0,25), color='orange', alpha=0.1)
sim = LDL_simulation(wss=2.2, parameters="4L_2006_Ai")
sim.plot_c(yrange=(0,1.1),xrange=(0,25), color='blue', alpha=0.1, style='--')

legend(('4LC parametrs','4LA parametrs without D(WSS)','4LA parametrs with D(WSS)'));


I am working on the file "LDL_transport_model.ipynb".
The first task is:
Two layer model
High WSS=2.2
Can you write Python code for it?

sim = LDL_simulation(wss=2.2, parameters="2L")
sim.plot_c(yrange=(0,0.012),xrange=(0,340));


I am working on the file "LDL_transport_model.ipynb".
The first task is:
WSS dependent parameters
The impact of WSS on the transport properties is given by the equations (8-11) developed by Olgac et al.
Can you write Python code for it?


I am working on the file "LFWDMT.ipynb".
The first task is:
Choose LFW category and test images
Next, we pick what category to run and how many images from each category to run on. A smaller number of images will lead to a lower run time and memory requirement, but may degrade image quality. We actually observe we can often get away with fewer than 2000 images. We can also specify a small prefix that is added to the results directory name.
Lastly, we also need to specify which images (corresponding to elements of lfw_ipath and rows of the lfw_attributes file) we want to perform deep manifold traversal for. In this case, we just set test_inds to [0] to run for Aaron Eckhart, but any list will suffice.
Can you write Python code for it?

num_per_cat = 2000
cat = 8 # Do senior, e.g. lfw_categories[8]
prefix = 'Aging'
test_inds = [0]


I am working on the file "LFWDMT.ipynb".
The first task is:
Finally, we run deep manifold traversal. The first time you run on LFW may take a while, because it will extract and save the conv3_1, conv4_1, and conv5_1 VGG features for each image. In subsequent runs, we just load these from disk. Deep manifold traversal places its results in timestamped results directories
Can you write Python code for it?

sorted_inds = np.argsort(lfw_numdata[:,cat])

source_inds = sorted_inds[0:num_per_cat]
target_inds = sorted_inds[-num_per_cat:]
everything_else = sorted_inds[num_per_cat:-num_per_cat]

source_ipath = [lfw_ipaths[i] for i in source_inds]
target_ipath = [lfw_ipaths[i] for i in target_inds]
everything_else_ipath = []

test_ipath = [lfw_ipaths[i] for i in test_inds]

ipath = source_ipath + target_ipath + everything_else_ipath + test_ipath
N = len(source_ipath)
M = len(target_ipath)
L = len(everything_else_ipath)
model = 'vgg'
device_id = 0

max_iter = 2000
hybrid = False
dmt.run(ipath,N,M,L,model,image_dims,device_id,weights,rbf_var,prefix,max_iter,hybrid,True)


I am working on the file "LFWDMT.ipynb".
The first task is:
Load LFW Data
Next, we load in the lfw_attributes file. The following code creates three variables of interest:

lfw_categories -- A list of attributes that we can run DMT on (for example, "senior" will end up being lfw_categories[8])
lfw_numdata -- The numerical attributes for each image in LFW. The rows of this matrix correspond to images, and the columns correspond to categories. For example, Aaron Eckhart's "senior" attribute value is in lfw_numdata[0][8]
lfw_ipaths -- A list of paths to the actual jpg images in lfw, corresponding to the rows of lfw_numdata above.
Can you write Python code for it?

lfw_attr = []
with open('lfw_attributes.txt','rb') as csvfile:
    rdr = csv.reader(csvfile, delimiter='\t')
    for row in rdr:
        lfw_attr.append(row)
lfw_categories = lfw_attr[1][3:]
lfw_numdata = []
lfw_ipaths = []
for i in range(2,13145):
    row = lfw_attr[i]
    name = row[0].replace(' ','_')
    num = row[1].zfill(4)
    path = 'images/lfw/{}/{}_{}.jpg'.format(name,name,num)
    numdata = [float(n) for n in row[2:]]
    lfw_numdata.append(numdata)
    lfw_ipaths.append(path)
lfw_numdata = np.array(lfw_numdata)


I am working on the file "LHCb_Open_Data_Project.ipynb".
The first task is:

Now that you can access the data, you can use a number of functions which can help you analyse it. You can find these functions in the libraries at the top of the page. Try to make a table of some of the information within your data file so that you can get a feel of the typical values for data in the set. Understanding the range of values for different variables will help with plotting graphs.

The data contains information about 'events' that were observed in the detector. An event refers to the particles produced when an interaction took place when two proton are collided at the LHC. The data you have includes information about particles observed in the detector after each collision. If you think of the data as a table, each row of the table is the results from a different collision. The columns of the table are different quantities measured about the particles produced in the collision.
We are interested in analysing the decays of particles called B+ or B- mesons decaying into three other mesons called kaons (K+ or K-). The events you have been given are those in which this process may have occurred. The detector has been used to reconstruct tracks that may have come from the kaons. You are given the measured momenta, charge, and likelihood of the tracks being kaons. You are given information for three tracks in each event, the ones that could be the three kaons that a B+ or B- meson has decayed into. The following information is available about each event: information list
Can you write Python code for it?

# make a table of the data variables here


I am working on the file "LHCb_Open_Data_Project.ipynb".
The first task is:

While drawing the Dalitz plot for the real data, label the axes accordingly. Compare the Dalitz plots of the real data with the one for the simulation.
What are the most striking differences?

Ordering Dalitz variables
You can make a further improvement to allow you to observe the resonances easier. Your resonances R02 and R03 are both composed of the same particle types, K+K-, and hence have the same distributions. It is useful to impose an ordering which distinguishes the resonances. We can call the resonances R0Low and R0High. In each event R0Low is the resonance with the lower mass and the other corresponds to the higher mass combination of kaons. You can now use the mass of these ordered resonances as your Dalitz plot variables, thus effectively "folding" your Dalitz plot so that one axis always has a higher value than the other.
Can you write Python code for it?

# make a new Dalitz plot with a mass ordering of the axes


I am working on the file "LHCb_Open_Data_Project.ipynb".
The first task is:
Adding features of the $B$ meson
In this analysis we are looking for B+ or B- mesons (see B meson) that have decayed into the three charged kaons.
Energy is a conserved quantities. This means that you can use the energy of the three 'daughter' kaons, which you have calculated above, to calculate the energy that the B meson that decayed into them must have.
Momentum is also a conserved quantity. Hence you can also use the momenta of the 'daughter' kaons to calculate the momentum of the B meson. But be careful - momentum is a vector quantity.
Using the Energy of the B meson and the magnitude of the momentum of the B meson you can use the energy-momentum relationship again. This time you are applying it to the B meson. This will allow you to calculate the invariant mass of the B meson.
Can you write Python code for it?

# calculate the energy of the B meson

# calculate the momentum components of the B meson
# and the magnitude of the momentum of the B meson

# calculate the B meson invariant mass
# plot the B meson invariant mass in a histogram


I am working on the file "LHCb_Open_Data_Project.ipynb".
The first task is:
Congratulations! You have performed your first search for a matter anti-matter difference.
Here you have only considered the statistical uncertainty. Your measurement will also have other sources of uncertainty known as systematic uncertainties which you have not considered at this stage.
Dalitz plots and two body resonances
Aims:

Produce Dalitz plots of the simulation and real data sample
Create ordered and binned dalitz plots.
Identify two body resonances in the Dalitz plots

In this stage we introduce you to an important technique for analysing decays of one particle (your charged B meson) into three bodies (the three kaons). This is known as a Dalitz plot.
The decay of the B meson can proceed either directly to the three-body final state or via an intermediate particle. For example, B+ → K+K+K−, could proceed through the decay  B+ → K+R0, where R0 is a neutral particle resonance which can decay R0 → K+K-. Dalitz plots can be used to identify these resonances which are visible as bands on the Dalitz plot.
More information about these plots and why these are used in particle physics research can be found in Dalitz Plot Introduction.
The kinematics of a three-body decay can be fully described using only two variables. The energies and momenta of the three kaons are not independent of each other as they all come from the decay of a B meson and energy and momentum are conserved. The axes of the plots conventionally are the squared invariant masses of two pairs of the decay products. It is a 2D plot, the x and y axes are both squared masses and the density of points in the plot shows the structure.
Consider our decay B+ → K+1K+2K−3, where we have numbered the kaons 1,2,3 to distinguish them. We can calculate the invariant mass of three possible combinations that could correspond to intermediate resonances...
Can you write Python code for it?

# calculate the invariant masses for each possible hadron pair combination

# plot the invariant mass for one of these combinations

# make a Dalitz plot with labelled axes for the simulation data


I am working on the file "LHCb_Open_Data_Project.ipynb".
The first task is:
Hint
Ordered Dalitz plot - You can find the maximum of the mass of R0Low vs R0High elementwise on one axis, and the minimum of on the other. You can use numpy.min(a,b) and numpy.max(a,b) to perform elementwise comparisons between two arrays a and b and return one array filled by either the individual min/max element from the elementwise comparisons.
Binned Dalitz plot
You can improve the representation of your Dalitz plot by binning the data. The hist2d function can be used to make a 2D histogram. The number of bins specification in the hist2d function is the number of bins in one axis.
Can you write Python code for it?

# plot a binned Dalitz Plot
# use colorbar() to make a legend for your plot at the side


I am working on the file "LHCb_Open_Data_Project.ipynb".
The first task is:
Hints
Energy calculation - Use the magnitude of momentum variable you calculated above and the known invariant mass of the kaon to work out the energy of the first hadron. Calculate the energy squared, and then the energy and plot this.
Kaon mass - you can find the kaon mass on wikipedia or in physics textbooks. There is also a reference used by particle physicists: all our knowledge of the properties of the particles are collected together by the particle data group  here.
Calculate the momenta and energies of the second and third kaon candidates also.
Can you write Python code for it?

# calculate variables for the energy of the other two kaons