I am working on the file "04_jit/04_jit.ipynb".
The first task is:
for a full list of the supported native Python features see here.
Numba and NumPy

One objective of Numba is having a seamless integration with NumPy.
Numba excels at generating code that executes on top of NumPy arrays.
Numba understands calls to (some, almost all) NumPy features, and is able to generate equivalent native code for many of them.

To see all the feature of NumPy actually supported by Numba, please see the documentation.
Can you write Python code for it?

#row mean example
@numba.jit('float64[:] (float64[:,:])', nopython=True)
def row_mean(A):
    m,n = A.shape
    mean = np.empty(m)
    for i in range(m):
        mean[i] = np.sum(A[i])/n
    return mean

A = np.random.random((10,10))
print( row_mean(A) )


I am working on the file "04_jit/04_jit.ipynb".
The first task is:
nopython mode
Numba can compile a Python function in two modes:
1. python mode. In Python mode, the compiled code relies on the CPython interpreter. (More flexible, but slow).
2. nopython mode. The code is compiled to standalone 100% machine code that doesn't rely on CPython, i.e, when we call the function, it doesn't pass through the CPython interpreter in any time (Limited, but fast).
Citting the official documentation:
Numba is aware of NumPy arrays as typed memory regions and so can speedup code using NumPy arrays. Other, less well-typed code will be translated to Python C-API calls effectively removing the "interpreter" but not removing the dynamic indirection.

Rule of thumb: In order to successfully use the nopython mode, we need to ensure that no native and unsupported Python features are being used inside the fuction to be jitted.
Can you write Python code for it?

@numba.jit('float64[:] (float64[:,:], float64[:])', nopython=True)
def dot_numba1(A,b):
    m,n = A.shape
    c = np.empty(m)
    for i in range(m):
        c[i] = np.dot(A[i],b)
    return c

A = np.random.random((1000,1000))
b = np.random.random(1000)

%timeit dot_numba1(A,b)

@numba.jit('float64[:] (float64[:,:], float64[:])', nopython=True)
def dot_numba2(A,b):
    m,n = A.shape
    c = []
    for i in range(m):
        c.append( np.dot(A[i],b) )
    return np.array(c)

# A few time ago, it wouldn't work!

%timeit dot_numba2(A,b)


I am working on the file "04_latent_semantic_clustering/svd_clustering.ipynb".
The first task is:
Выберем r=403, т.к.
1) на всей длине кривая имеет дост-но большую производную (по модулю)
2) 403 не очень большое число для семантического пространства
Получим представления документов.
Can you write Python code for it?


I am working on the file "04_latent_semantic_clustering/svd_clustering.ipynb".
The first task is:
Не нашёл библиотеки, которая позволяла бы делать разложение на 1000 сингулярных чисел (>403). Есть имплементации с параметром full_matrices, но для них не хватает оперативной памяти. Поэтому делаю разложение на размер наименьшей оси (кол-во документов).
Can you write Python code for it?

%%time
T, S, Dt = np.linalg.svd(tf_matrix, full_matrices=False)


I am working on the file "04_latent_semantic_clustering/svd_clustering.ipynb".
The first task is:
Откластеризуем их.
Я смотрел на названия работ и на топовые (по нормированному tf) слова. Но попытки интерпретировать кластеры оказались тщетны. Поэтому выбрал число кластеров 10.
Can you write Python code for it?

from utils import STOP_WORDS, get_top_terms
stop_words_ids = [term_to_id[term] for term in STOP_WORDS if term in term_to_id]

top5_terms = get_top_terms(tf_matrix, vocabulary, stop_words_ids, k=5)

def print_clusters(clusters, doc_id_to_path, top_terms = None):
    clusters = np.array(clusters)
    num_clusters = np.unique(clusters).shape[0]
    for c in range(num_clusters):
        print c, "cluster"
        doc_ids = np.where(clusters == c)[0]
        for doc_id in doc_ids:
            doc_path = doc_id_to_path[doc_id]
            if top_terms is None:
                print doc_path[doc_path.find("-") + 1:]
            else:
                print top_terms[doc_id]
        print

NUM_CLUSTERS = 10

kmeans = KMeansClusterer(NUM_CLUSTERS, distance=cosine_distance, repeats=20)
assigned_clusters = kmeans.cluster(D_lsi, assign_clusters=True)
print_clusters(assigned_clusters, doc_id_to_path)


I am working on the file "04_linear_regression_ex4.ipynb".
The first task is:
4.2)
Can you write Python code for it?

with tf.Session(graph=graph) as sess:
    init.run()
    print(y_pred.eval())


I am working on the file "04_linear_regression_ex4.ipynb".
The first task is:
4.5)
Can you write Python code for it?

with graph.as_default():
    m = len(X_train)
    gradients_w = 2/m * tf.matmul(tf.transpose(X), error)
    gradients_b = 2 * tf.reduce_mean(error)


I am working on the file "04_linear_regression_ex4.ipynb".
The first task is:
4.7)
Can you write Python code for it?

n_iterations = 2000

with tf.Session(graph=graph) as sess:
    init.run()
    for iteration in range(n_iterations):
        if iteration % 100 == 0:
            print("Iteration {:5}, MSE: {:.4f}".format(iteration, mse.eval()))
        training_op.run()
    w_val, b_val = sess.run([w, b])

def plot_life_satisfaction_with_linear_model(X_train, y_train, w, b):
    plot_life_satisfaction(X_train, y_train)
    plt.plot([0, 60000], [b, w[0][0] * (60000 / 10000) + b])

plt.figure(figsize=(10, 5))
plot_life_satisfaction_with_linear_model(X_train, y_train, w_val, b_val)
plt.show()


I am working on the file "04_linear_regression_ex4.ipynb".
The first task is:
Building the Linear Regression Model
Exercise 4

In this exercise we will build a linear regression model using TensorFlow. If you are not familiar with the maths behind linear regression models, you can read the explanation below. If you already know this (or if you don't care much about the maths), you can just skip this explanation and simply follow the instructions given in questions 4.1 to 4.3 below.
In a linear regression model, the predictions are a linear combination of the input features. In other words, the predicted value $\hat{y}$ can be computed using the following equation:
$\hat{y} = w_1 x_1 + w_2 x_2 + \dots + w_n x_n + b$
where:
* $x_1, x_2, \dots, x_n $ are the input features,
* $w_1, w_2, \dots, w_n $, are their corresponding weights,
* and $b$ is the bias term (also called the intercept term).
This equation can be expressed in a more compact way using vectors:
$\hat{y} = \langle \mathbf{x}, \mathbf{w} \rangle + b$
where:
* $ \mathbf{x} = \begin{pmatrix}x_1 \ x_2 \ \vdots \ x_n \end{pmatrix}$ is the input feature vector (by convention, vectors are written in bold font),
* $ \mathbf{w} = \begin{pmatrix}w_1 \ w_2 \ \vdots \ w_n \end{pmatrix}$ is the weight vector,
* $\langle \mathbf{x}, \mathbf{w} \rangle$ is the inner product of vectors $\mathbf{x}$ and $\mathbf{w}$, equal to $w_1 x_1 + w_2 x_2 + \dots + w_n x_n$.
It is often more convenient to handle vectors as matrices with a single column (a "column vector"). The inner product $\langle \mathbf{x}, \mathbf{w} \rangle$ is then replaced with the matrix dot product: $\mathbf{x}^T \cdot \mathbf{w}$, where $\mathbf{x}^T$ is the transpose of the column vector $\mathbf{x}$. Transposing a column vector gives you a "row vector" (i.e., a matrix with a single row): $\mathbf{x}^T = \begin{pmatrix} x_1 & x_2 & \dots & x_n \end{pmatrix}$. Once again $\mathbf{x}^T \cdot \mathbf{w} = w_1 x_1 + w_2 x_2 + \dots + w_n x_n$.
Lastly, it is possible to compute predictions for many instances at a time by putting all their input features in a matrix $\mathbf{X}$...
Can you write Python code for it?

graph = tf.Graph()
with graph.as_default():
    X = tf.constant(X_train, name="X")
    y = tf.constant(y_train, name="y")

    b = tf.Variable(0.0, name="b")
    w = tf.Variable(tf.zeros([1, 1]), name="w")
    y_pred = tf.add(tf.matmul(X, w), b, name="y_pred")  # X @ w + b

    init = tf.global_variables_initializer()


I am working on the file "04_linear_regression_ex4.ipynb".
The first task is:
Each row in X_train represents a training instance, in this case a country. In this simple regression example, there is just one feature per instance (i.e., one column), in this case the country's GDP per capita (in tens of thousands of dollars).
Can you write Python code for it?


I am working on the file "04_linear_regression_ex4.ipynb".
The first task is:
Linear Regression with TensorFlow
Loading the training data
Can you write Python code for it?

import numpy as np

data = np.loadtxt("data/life_satisfaction.csv",
                  dtype=np.float32,
                  delimiter=",",
                  skiprows=1,
                  usecols=[1, 2])
X_train = data[:, 0:1] / 10000 # feature scaling
y_train = data[:, 1:2]


I am working on the file "04_model_training.ipynb".
The first task is:
150 observations
4 features (sepal length, sepal width, petal length, petal width)
Response variable is the iris species
Classification problem since response is categorical
More information in the UCI Machine Learning Repository

K-nearest neighbors (KNN) classification

Pick a value for K.
Search for the K observations in the training data that are "nearest" to the measurements of the unknown iris.
Use the most popular response value from the K nearest neighbors as the predicted response value for the unknown iris.

Example training data

KNN classification map (K=1)

KNN classification map (K=5)

Image Credits: Data3classes, Map1NN, Map5NN by Agor153. Licensed under CC BY-SA 3.0
Loading the data
Can you write Python code for it?

# import load_iris function from datasets module
from sklearn.datasets import load_iris

# save "bunch" object containing iris dataset and its attributes
iris = load_iris()

# store feature matrix in "X"
X = iris.data

# store response vector in "y"
y = iris.target

# print the shapes of X and y
print(X.shape)
print(y.shape)


I am working on the file "04_model_training.ipynb".
The first task is:
Name of the object does not matter
Can specify tuning parameters (aka "hyperparameters") during this step
All parameters not specified are set to their defaults
Can you write Python code for it?


I am working on the file "04_model_training.ipynb".
The first task is:
Resources

Nearest Neighbors (user guide), KNeighborsClassifier (class documentation)
Logistic Regression (user guide), LogisticRegression (class documentation)
Videos from An Introduction to Statistical Learning
Classification Problems and K-Nearest Neighbors (Chapter 2)
Introduction to Classification (Chapter 4)
Logistic Regression and Maximum Likelihood (Chapter 4)

Comments or Questions?

Email: kevin@dataschool.io
Website: http://dataschool.io
Twitter: @justmarkham
Can you write Python code for it?

from IPython.core.display import HTML
def css_styling():
    styles = open("styles/custom.css", "r").read()
    return HTML(styles)
css_styling()


I am working on the file "04_model_training.ipynb".
The first task is:
Returns a NumPy array
Can predict for multiple observations at once
Can you write Python code for it?


I am working on the file "04_model_training.ipynb".
The first task is:
Step 2: "Instantiate" the "estimator"

"Estimator" is scikit-learn's term for model
"Instantiate" means "make an instance of"
Can you write Python code for it?


I am working on the file "04_model_training.ipynb".
The first task is:
Step 3: Fit the model with data (aka "model training")

Model is learning the relationship between X and y
Occurs in-place
Can you write Python code for it?


I am working on the file "04_model_training.ipynb".
The first task is:
Step 4: Predict the response for a new observation

New observations are called "out-of-sample" data
Uses the information it learned during the model training process
Can you write Python code for it?


I am working on the file "04_model_training.ipynb".
The first task is:
Using a different classification model
Can you write Python code for it?

# import the class
from sklearn.linear_model import LogisticRegression

# instantiate the model (using the default parameters)
logreg = LogisticRegression()

# fit the model with data
logreg.fit(X, y)

# predict the response for new observations
logreg.predict(X_new)


I am working on the file "04_model_training.ipynb".
The first task is:
Using a different value for K
Can you write Python code for it?

# instantiate the model (using the value K=5)
knn = KNeighborsClassifier(n_neighbors=5)

# fit the model with data
knn.fit(X, y)

# predict the response for new observations
knn.predict(X_new)


I am working on the file "04_numpy.ipynb".
The first task is:
But you can not put a list, tuple, ... directly into it
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
Bytes of all elements
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
Custom dtype, more on that later
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
Define a boolean mask
Can you write Python code for it?

mask = np.array([True, True, False, True, False])


I am working on the file "04_numpy.ipynb".
The first task is:
Dtype
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
Elementwise Operation
Operatiors act elementwise on numpy arrays
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
Find all the spikes in the measurment
Can you write Python code for it?

# Create some random spikes
y = []
for i in range(1000):
    r = np.random.random()
    if  r > 0.95:
        y.append(np.random.randint(-10, 10))
    else:
        y.append(r - 0.5)

y = np.array(y)
x = np.arange(y.size)

# Plot the raw data
plt.figure(figsize=(12,3))
plt.plot(x, y)

# Find all positiv spikes
m_pos = y > np.var(y)
plt.plot(x[m_pos], y[m_pos], ls='', marker='o')

# Find all negatives spikes
m_neg = y < -np.var(y)
plt.plot(x[m_neg], y[m_neg], ls='', marker='o')

plt.ylim(-12, 12)


I am working on the file "04_numpy.ipynb".
The first task is:
If you prefer the stepsize
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
It still handles normal python types
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
Many more ways:
Can you write Python code for it?

np.ones(10)
np.zeros(10)
np.loadtxt('filename.txt', ...)
np.genfromtxt('filename.txt', ...)

...


I am working on the file "04_numpy.ipynb".
The first task is:
Memory
Number of bytes of one element
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
Number fo all elements in array
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
Numpy
Creation
From lists
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
Return True items of the mask
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
Slicing more advanced
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
Slicing the center
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
This does not only work for the numpy universal functions.
Can you write Python code for it?

def parabola(x, a = 1, b=0, c=0):
    return a * (x - b)**2 - c

x = np.linspace(-6, 6, 1001)
plt.plot(x, parabola(x, a=2, b=2, c=3))
plt.plot(x, parabola(x, a=2, b=-2, c=3))
plt.grid()
plt.ylim(-3, 25)


I am working on the file "04_numpy.ipynb".
The first task is:
To hard to remember: Use the np.diff() insted, into also works for higher
Can you write Python code for it?

x = np.linspace(0, 2 * np.pi, 100)
y = np.sin(x)

plt.plot(x, y)
plt.plot(x[1:], np.diff(y) / np.diff(x))


I am working on the file "04_numpy.ipynb".
The first task is:
Ufunc - Universal Functions
Numpy includes a couple of mathematical functions, so called universal functions.

np.sin(), np.cos(), np.tan(), ...
np.sqrt(), np.exp(), np.log(), np.log10(), ...
np.mean(), np.var(),
np.abs(), ...
np.max(), np.min(), ...
np.convolve, ...
np.random.random(), np.random.poission(), ...
... and many many more

Ufunc act elementwise on arrays
Can you write Python code for it?

x = np.linspace(0, 2 * np.pi, 10)
x

np.sin(x)

import matplotlib.pyplot as plt
%matplotlib inline

x = np.linspace(0, 2 * np.pi, 100)
plt.plot(x, np.sin(x))

x = np.linspace(0, 2 * np.pi, 100)
y = np.sin(x)

dx = (x[:-1] - x[1:])
dy = (y[:-1] - y[1:])

plt.plot(x, y, x[1:], dy / dx)
plt.plot(x, np.cos(x), ls='', marker='o', markevery=3)


I am working on the file "04_numpy.ipynb".
The first task is:
Why do i want to do that?
Can you write Python code for it?

import matplotlib.pyplot as plt
%matplotlib inline


I am working on the file "04_numpy.ipynb".
The first task is:
With linspace, always useful for plotting data
Can you write Python code for it?


I am working on the file "04_numpy.ipynb".
The first task is:
You can extent your function behaviour by adding a simpel line
Can you write Python code for it?

def parabola(x, a = 1, b=0, c=0):

    x = np.array(x, copy=False)

    return a * (x - b)**2 - c


I am working on the file "04_numpy.ipynb".
The first task is:
is equal to
Can you write Python code for it?


I am working on the file "04_operators_and_commutators.ipynb".
The first task is:
There is an extensive tutorial that you can refer to. Another useful example is the calculation
of definite and indefinite integrals using SymPy. Consider the following code snippet:
```Python
An indefinite integral
integrate(cos(x), x)
```
This will calculate the primitive function of $\cos(x)$:
\begin{equation}
\int \cos(x)\mathrm{d}x = \sin(x) + C
\end{equation}
Can you write Python code for it?


I am working on the file "04_product/Final_model_and_test.ipynb".
The first task is:
Final Test
Can you write Python code for it?

test_df = pd.read_excel('../resource/test_df.xlsx')
test_sample = test_df.title
test_y = test_df['my_rating'].values
test_real_X = test_df[['avg_rating']].values
test_cat_X = test_df.text
test_watcha_y = test_df['watcha_rating'].values

gnb_test_pred = gnb.predict(test_real_X)
gnb_test_prob = gnb.predict_proba(test_real_X)
mnb_test_pred = mnb.predict(test_cat_X)
mnb_test_prob = mnb.predict_proba(test_cat_X)

mix_test_prob = np.multiply(gnb_test_prob, mnb_test_prob)
mix_test_prob_softmax = np.zeros((12, 5))
for i in range(12):
    mix_test_prob_softmax[i] = softmax(mix_test_prob[i])
mix_test_prob_softmax
np.sum(mix_test_prob_softmax[0])

mix_test_pred = np.zeros(12, )
for i in range(12):
    mix_test_pred[i] = np.argmax(mix_test_prob_softmax[i])
mix_test_pred += 1 # 별점은 1점부터 5점까지이므로(int)
mix_test_pred

test_df['predict'] = mix_test_pred
test_df


I am working on the file "04_product/Final_model_and_test.ipynb".
The first task is:
Gaussian & Multinomial NB fitting

gnb : 'avg_rating' 피쳐 한개만
mnb : alpha는 피쳐가 달라진 관계로(콤마, 띄어쓰기 제거) 다시 cv시행
ngram_range : (1, 2)
tfidf : true
sub_alpha : 0.3
score(mae) : 0.7674
Can you write Python code for it?

from sklearn.naive_bayes import GaussianNB, MultinomialNB
gnb = GaussianNB()
mnb = Pipeline([
        ('vect', CountVectorizer(ngram_range=(1, 2),)),
        ('tfidf', TfidfTransformer()),
        ('clf', MultinomialNB(alpha=0.3)),
    ])

gnb.fit(real_X, y)
gnb_pred = gnb.predict(real_X)
gnb_prob = gnb.predict_proba(real_X)

mnb.fit(cat_X, y)
mnb_pred = mnb.predict(cat_X)
mnb_prob = mnb.predict_proba(cat_X)

mix_prob = np.multiply(gnb_prob, mnb_prob)
mix_prob.shape

def softmax(w, t=1.0):
    """Calculate the softmax of a list of numbers w.

    Parameters
    ----------
    w : list of numbers
    t : float

    Return
    ------
    a list of the same length as w of non-negative numbers

    Examples
    --------
    >>> softmax([0.1, 0.2])
    array([ 0.47502081,  0.52497919])
    >>> softmax([-0.1, 0.2])
    array([ 0.42555748,  0.57444252])
    >>> softmax([0.9, -10])
    array([  9.99981542e-01,   1.84578933e-05])
    >>> softmax([0, 10])
    array([  4.53978687e-05,   9.99954602e-01])
    """
    e = np.exp(np.array(w) / t)
    dist = e / np.sum(e)
    return dist

mix_prob_softmax = np.zeros((544, 5))
for i in range(544):
    mix_prob_softmax[i] = softmax(mix_prob[i])
mix_prob_softmax
np.sum(mix_prob_softmax[0])

mix_pred = np.zeros(544, )
for i in range(544):
    mix_pred[i] = np.argmax(mix_prob_softmax[i])
mix_pred += 1 # 별점은 1점부터 5점까지이므로(int)
mix_pred


I am working on the file "04_product/Final_model_and_test.ipynb".
The first task is:
X_tfidf로 cv했을때

alpha : 0.23
score : 0.791257638511
Can you write Python code for it?

cv = StratifiedKFold(y, n_folds=5, random_state=51)

i_range = []
score_range = []
sigma = []

for a in np.arange(0, 1, 0.01):
    mnb = MultinomialNB(alpha = a)

    scores = np.zeros(5)

    for i, (train_idx, test_idx) in enumerate(cv):
        X_train = X_tfidf[train_idx]
        y_train = y[train_idx]
        X_test = X_tfidf[test_idx]
        y_test = y[test_idx]

        mnb.fit(X_train, y_train)
        y_pred = mnb.predict(X_test)

        scores[i] = mean_absolute_error(y_test, y_pred)

    i_range.append(a)
    score_range.append(np.mean(scores))
    sigma.append(np.std(scores))

best_idx = np.argmin(score_range)
best_alpha = i_range[best_idx]
best_score = score_range[best_idx]
sigma

plt.figure(figsize = (15, 5))
plt.plot(i_range, score_range)
plt.plot(i_range, np.array(score_range) + sigma, 'b--')
plt.plot(i_range, np.array(score_range) - sigma, 'b--')
plt.axhline(best_score + sigma[best_idx], linestyle=':', color='r')
plt.axvline(best_alpha, linestyle=':', color='r')

def find_nearest(array, value):
    idx = (np.abs(array-value)).argmin()
    return idx

sub_alpha = i_range[find_nearest(score_range, best_score+sigma[best_idx])]
sub_score = best_score+sigma[best_idx]

plt.scatter(sub_alpha, sub_score, s=100, c='red')
plt.xlim(0, 1)
plt.ylabel('CV score(mae)')
plt.xlabel('alpha')

print("best alpha : ", best_alpha)
print("best score : ", best_score)
print('   1-sigma : ', round(sigma[best_idx], 4))
print('='*25)
print("sub_opt alpha : ", sub_alpha)
print("sub_opt score : ", sub_score)


I am working on the file "04_product/Final_model_and_test.ipynb".
The first task is:
score
Can you write Python code for it?

mix_score = mean_absolute_error(mix_test_pred, test_y)
watcha_score = mean_absolute_error(test_watcha_y, test_y)

print('mix_score :', mix_score)
print('watcha_score :', watcha_score)

# watcha_rating을 반올림하여 정수로변환하여 스코어 측정해봄
test_watchar_round_y = np.round(test_watcha_y,)
mean_absolute_error(test_watchar_round_y, test_y)


I am working on the file "04_product/Final_model_and_test.ipynb".
The first task is:
training MultiNB & parameter tuning

cat_X => countvec
Can you write Python code for it?

from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer, TfidfTransformer
count_vect = CountVectorizer()
X_counts = count_vect.fit_transform(cat_X)

tfidf_vect = TfidfVectorizer()
X_tfidf = tfidf_vect.fit_transform(cat_X)

from sklearn.cross_validation import StratifiedKFold
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import mean_absolute_error


I am working on the file "04_python_algoritmos_y_funciones/algoritmos_y_funciones.ipynb".
The first task is:
1. Definiciones y conceptos básicos.
Entenderemos por algoritmo a una serie de pasos que persiguen un objetivo específico. Intuitivamente lo podemos relacionar con una receta de cocina: una serie de pasos bien definidos (sin dejar espacio para la confusión del usuario) que deben ser realizados en un orden específico para obtener un determinado resultado.
   En general un buen algoritmo debe poseer las siguientes características:

No debe ser ambiguo en su implementación para cualquier usuario.
Debe definir adecuadamente datos de entrada (inputs).
Debe producir datos de salida (outputs) específicos.
Debe poder realizarse en un número finito de pasos y por ende, en un tiempo finito. ( Ver The Halting Problem ).

Por otra parte, llamaremos código a la materialización, en base a la implementación en la sintaxis adecuada de un determinado lenguaje de programación, de un determinado algoritmo. Entonces, para escribir un buen código y que sea eficiente debe tratar de respetar las ideas anteriores: se debe desarrollar en un número finito de pasos, ocupar adecuadamente las estructuras propias del lenguaje, se debe poder ingresar y manipular adecuadamente los datos de entrada y finalmente entregar el resultado deseado.
A diferencia de lo anterior, una idea un poco menos estructurada es el concepto de pseudo-código. Entenderemos por el anterior a la descripción informal de un determinado algoritmo en un determinado lenguaje de programación. Sin embargo, no debe perder las características esenciales de un algoritmo como claridad en los pasos, inputs y outputs bien definidos, etc. de tal forma que permita la implementación directa de éste en el computador.
Una vez implementado un algoritmo viene el proceso de revisión de éste. Para realizar adecuadamente lo anterior se recomienda contestar las siguentes preguntas:
 1. ¿Mi algoritmo funciona para todos los posibles datos de entrada?
 2. ¿Cuánto tiempo tomará en correr mi algoritmo? ¿Cuánta memoria ocupa en mi computador?
 3. Ya que sé que mi algoritmo funciona: ¿es posible...
Can you write Python code for it?

N = int(raw_input("Ingrese el numero que desea estudiar "))
if N<=1:
    print("Numero N tiene que ser mayor o igual a 2")
elif 2<=N<=3:
    print("{0} es primo".format(N))
else:
    es_primo = True
    for i in range(2, N):
        if N%i==0:
            es_primo = False
    if es_primo:
        print("{0} es primo".format(N))
    else:
        print("{0} es compuesto".format(N))


I am working on the file "04_python_algoritmos_y_funciones/algoritmos_y_funciones.ipynb".
The first task is:
3. Midiendo la complejidad
Como dijimos anteriormente luego de hacer que un algoritmo funcione, una de las preguntas más importantes es la revisión de éste haciendo énfasis en la medición del tiempo que necesita para resolver el problema. Así, la primera interrogante es: ¿cómo podemos medir el tiempo que tarda un algoritmo en relación al tamaño del problema que resuelve? Esto se denomina usualmente como complejidad temporal o, en inglés, como time complexity o escalability.
Sin embargo es importante notar que medir la complejidad temporal de un algoritmo puede resultar un poco complejo puesto que: (a) El tiempo que toma al computador realizar las distintas operaciones en general es heterogeneo, es decir, realizar una suma es mucho más rápido que hacer una división, (b) Computadores distintos puede realizar un determinado experimento en tiempos distintos.
La notación estándar para la complejidad de un algoritmo es mediante la letra mayúscula O, por lo que la complejidad de alguna función la podemos expresar por O("función"), lo que podemos interpretar como que el número de operaciones es proporcional a la función por una determinada constante. Las complejidades más importantes son:

O(1) es un algoritmo de complejidad temporal constante, es decir, el número de operaciones del algoritmo realmente no varía mucho si el tamaño del problema crece.
O(log(n)) es la complejidad logarítmica.
O(n) significa que la complejidad del problema es lineal, es decir, doblar el tamaño del problema dobla el tamaño requerido para su solución.
O($n^2$) significa complejidad cuadrática, es decir, doblar el tamaño del problema cuatriplica el tiempo requerido para su solución.
O($2^n$) y en general O($a^n$), $a>1$, posee complejidad exponencial.

En nuestros algoritmos anteriormente desarrollados:
1. El primer algoritmo tiene complejidad $O(N)$: siempre tarda lo mismo.
2. El segundo algoritmo tiene complejidad variable: si el numero es compuesto tarda en el mejor de los casos O($1$) y O($\sqrt{N}$) en el peor de los casos (como 25, o...
Can you write Python code for it?

def sin_inputs_ni_outputs():
    print "Hola mundo"

def sin_inputs():
    return "42"

def sin_outputs(a,b):
    print a
    print b

def con_input_y_output(a,b):
    return a+b

def con_tuti(a,b,c=2):
    return a+b*c


I am working on the file "04_python_algoritmos_y_funciones/algoritmos_y_funciones.ipynb".
The first task is:
La ejecución de números grandes compuestos se detiene en el primer múltiplo cuando el número es compuesto. Sin embargo, para numeros grandes y primos tarda bastante.
2.3 Tercer Programa
Un último truco que podemos utilizar para verificar más rápidamente si un número es primo es verificar únicamente parte del rango de los múltiplos. Esto se explica mejor con un ejemplo. Consideremos el número 16: los multiplos son 2, 4 y 8. Como el número es compuesto, nuestro algoritmo anterior detecta rápidamente que 2 es un factor, detiene el algoritmo e indica que el número 12 no es primo. Consideremos ahora el numero 17: es un número primo y no tiene factores, por lo que el algoritmo revisa los numeros 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 y 16. Sin embargo, sólo es necesario revisar los números 2, 3, 4, 5 y 6 porque para que exista un factor mayor a 6, tiene que simultáneamente haber un factor menor a 6 tal que la multiplicación sea 17. Esto es, basta revisar los factores más pequeños, donde la cota está dada por el entero más cercano a $\sqrt{17}$ o en general, $\sqrt{N}$.
El algoritmo para verificar si un numero no primo es: verificar si algún numero entero entre $2$ y $\sqrt{N}$ es divisor de $N$.
El pseudo-código es:
1. Ingresar un determinado número $N$ mayor a 1.
2. Si el número es 2 o 3, el numero es primo. Sino, se analizan los restos de la división para cada número entre $2$ y $\sqrt{N-1}$. Si alguno de los restos es cero, entonces el numero $N$ es divisible, y no es primo.
Mientras que el código es el siguiente:
Can you write Python code for it?

N = int(raw_input("Ingrese el numero que desea estudiar "))
if N<=1:
    print("Numero N tiene que ser mayor o igual a 2")
elif 2<=N<=3:
    print("{0} es primo".format(N))
else:
    es_primo = True
    for i in range(2, int(N**.5)):
        if N%i==0:
            es_primo = False
            break
    if es_primo:
        print("{0} es primo".format(N))
    else:
        print("{0} no es primo".format(N))


I am working on the file "04_python_algoritmos_y_funciones/algoritmos_y_funciones.ipynb".
The first task is:
La función con_input_y_output se ejecuta con datos de entrada y produce datos de salida. Cabe destacar que como python no utiliza tipos de datos, la misma función puede aplicarse a distintos tipos de datos mientras la lógica aplicada dentro de la función tenga sentido (y no arroje errores).
Can you write Python code for it?

print con_input_y_output("uno","dos")
print con_input_y_output(1,2)
print con_input_y_output(1.0, 2)
print con_input_y_output(1.0, 2.0)


I am working on the file "04_python_algoritmos_y_funciones/algoritmos_y_funciones.ipynb".
The first task is:
La función sin_inputs_ni_outputs se ejecuta sin recibir datos de entrada ni producir datos de salida (Y no es muy útil).
Can you write Python code for it?


I am working on the file "04_sparse_matrices.ipynb".
The first task is:
Often, once an LIL matrix is created, it is useful to convert it to a CSR format
Note: many scikit-learn algorithms require CSR or CSC format
Can you write Python code for it?


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/06_price_history_varlen-no-outliers-checkpoint.ipynb".
The first task is:
Step 0 - hyperparams
Can you write Python code for it?

num_epochs = 10
series_max_len = 60
num_features = 1  #just one here, the function we are predicting is one-dimensional
state_size = 400
target_len = 30
batch_size = 47


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/06_price_history_varlen-no-outliers-checkpoint.ipynb".
The first task is:
Step 1 - collect data (and/or generate them)
Can you write Python code for it?

csv_in = '../price_history_03a_fixed_width.csv'
npz_path = '../price_history_03_dp_60to30_from_fixed_len.npz'

# XX, YY, sequence_lens, seq_mask = PriceHistoryVarLenDataProvider.createAndSaveDataset(
#     csv_in=csv_in,
#     npz_out=npz_path,
#     input_seq_len=60, target_seq_len=30)
# XX.shape, YY.shape, sequence_lens.shape, seq_mask.shape

dp = PriceHistoryVarLenDataProvider(filteringSeqLens = lambda xx : xx >= target_len,
                                    npz_path=npz_path)
dp.inputs.shape, dp.targets.shape, dp.sequence_lengths.shape, dp.sequence_masks.shape


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/06_price_history_varlen-no-outliers-checkpoint.ipynb".
The first task is:
Step 2 - Build model
Can you write Python code for it?

model = PriceHistoryRnnVarlen(rng=random_state, dtype=dtype, config=config)

graph = model.getGraph(batch_size=batch_size, state_size=state_size,
                       target_len=target_len, series_max_len=series_max_len)

show_graph(graph)


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/06_price_history_varlen-no-outliers-checkpoint.ipynb".
The first task is:
Step 3 training the network
Can you write Python code for it?

num_epochs, state_size, batch_size

def experiment():
    dynStats, predictions_dict = model.run(epochs=num_epochs,
                                        state_size=state_size,
                                         series_max_len=series_max_len,
                                         target_len=target_len,
                                         npz_path=npz_path,
                                         batch_size=batch_size)
    return dynStats, predictions_dict

from os.path import isdir
data_folder = '../../../../Dropbox/data'
assert isdir(data_folder)

dyn_stats, preds_dict = get_or_run_nn(experiment,
                                      filename='001_plain_rnn_60to30', nn_runs_folder= data_folder + '/nn_runs')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

sns.tsplot(data=dp.inputs[ind].flatten())

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/06_price_history_varlen-no-outliers-checkpoint.ipynb".
The first task is:
TODO Co integration
https://en.wikipedia.org/wiki/Cointegration
https://www.quora.com/What-are-some-methods-to-check-similarities-between-two-time-series-data-sets
https://stackoverflow.com/questions/11362943/efficient-cointegration-test-in-python
Mean Squared Error (instead of huber loss)
Can you write Python code for it?

num_epochs, state_size, batch_size
cost_func = PriceHistoryRnnVarlen.COST_FUNCS.MSE

def experiment():
    dynStats, predictions_dict = model.run(epochs=num_epochs,
                                           cost_func= cost_func,
                                           state_size=state_size,
                                           series_max_len=series_max_len,
                                           target_len=target_len,
                                           npz_path=npz_path,
                                           batch_size=batch_size)
    return dynStats, predictions_dict

dyn_stats, preds_dict = get_or_run_nn(experiment,
                                      filename='001_plain_rnn_60to30_mse')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/07_price_history_varlen_rnn_cells-checkpoint.ipynb".
The first task is:
Conclusion
GRU has performed much better than basic RNN
GRU cell - 50 epochs
Can you write Python code for it?

rnn_cell = PriceHistoryRnnVarlen.RNN_CELLS.GRU
num_epochs = 50
state_size, batch_size

def experiment():
    dynStats, predictions_dict = model.run(epochs=num_epochs,
                                           rnn_cell=rnn_cell,
                                           state_size=state_size,
                                           series_max_len=series_max_len,
                                           target_len=target_len,
                                           npz_path=npz_path,
                                           batch_size=batch_size)
    return dynStats, predictions_dict

dyn_stats, preds_dict = get_or_run_nn(experiment,
                                      filename='002_rnn_gru_60to30_50epochs')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/07_price_history_varlen_rnn_cells-checkpoint.ipynb".
The first task is:
Step 1 - collect data (and/or generate them)
Can you write Python code for it?


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/07_price_history_varlen_rnn_cells-checkpoint.ipynb".
The first task is:
Step 2 - Build model
Can you write Python code for it?

model = PriceHistoryRnnVarlen(rng=random_state, dtype=dtype, config=config)

graph = model.getGraph(batch_size=batch_size, state_size=state_size,
                       rnn_cell= PriceHistoryRnnVarlen.RNN_CELLS.GRU,
                       target_len=target_len, series_max_len=series_max_len)

show_graph(graph)


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/07_price_history_varlen_rnn_cells-checkpoint.ipynb".
The first task is:
Step 3 training the network
GRU cell
Can you write Python code for it?

rnn_cell = PriceHistoryRnnVarlen.RNN_CELLS.GRU
num_epochs, state_size, batch_size

def experiment():
    dynStats, predictions_dict = model.run(epochs=num_epochs,
                                           rnn_cell=rnn_cell,
                                           state_size=state_size,
                                           series_max_len=series_max_len,
                                           target_len=target_len,
                                           npz_path=npz_path,
                                           batch_size=batch_size)
    return dynStats, predictions_dict

dyn_stats, preds_dict = get_or_run_nn(experiment,
                                      filename='002_rnn_gru_60to30')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/07_price_history_varlen_rnn_cells-checkpoint.ipynb".
The first task is:
https://r2rt.com/recurrent-neural-networks-in-tensorflow-iii-variable-length-sequences.html
Can you write Python code for it?

from __future__ import division
import tensorflow as tf
from os import path
import numpy as np
import pandas as pd
import csv
from sklearn.model_selection import StratifiedShuffleSplit
from time import time
from matplotlib import pyplot as plt
import seaborn as sns
from mylibs.jupyter_notebook_helper import show_graph
from tensorflow.contrib import rnn
from tensorflow.contrib import learn
import shutil
from tensorflow.contrib.learn.python.learn import learn_runner
from IPython.display import Image
from IPython.core.display import HTML
from mylibs.tf_helper import getDefaultGPUconfig
from data_providers.binary_shifter_varlen_data_provider import \
    BinaryShifterVarLenDataProvider
from data_providers.price_history_varlen_data_provider import PriceHistoryVarLenDataProvider
from models.model_05_price_history_rnn_varlen import PriceHistoryRnnVarlen
from sklearn.metrics import r2_score
from mylibs.py_helper import factors
from fastdtw import fastdtw
from scipy.spatial.distance import euclidean
from statsmodels.tsa.stattools import coint

dtype = tf.float32
seed = 16011984
random_state = np.random.RandomState(seed=seed)
config = getDefaultGPUconfig()
%matplotlib inline

from common import get_or_run_nn


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/13_price_history_seq2seq-raw-checkpoint.ipynb".
The first task is:
Conclusion
???
Basic RNN cell (EOS 0)
Can you write Python code for it?

rnn_cell = PriceHistorySeq2Seq.RNN_CELLS.BASIC_RNN
num_epochs = 10
num_epochs, num_units, batch_size

def experiment():
    return model.run(
        npz_path=npz_path,
        epochs=num_epochs,
        batch_size=batch_size,
        num_units=num_units,
        input_len = input_len,
        target_len = target_len,
        rnn_cell=rnn_cell,
        eos_token = 0
    )

dyn_stats, preds_dict = get_or_run_nn(experiment, filename='006_basic_rnn_seq2seq_zeroEOS_60to30')


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/13_price_history_seq2seq-raw-checkpoint.ipynb".
The first task is:
Conclusion
???
Can you write Python code for it?

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/13_price_history_seq2seq-raw-checkpoint.ipynb".
The first task is:
Conclusion
It is oscillating which is an indication that setting EOS to zero where the real values is not helping the model
Perhaps another idea is to have as first input of the decoder RNN the latest value of the encoding stream
GRU cell - with EOS = 0
Can you write Python code for it?

rnn_cell = PriceHistorySeq2Seq.RNN_CELLS.GRU
num_epochs = 10
num_epochs, num_units, batch_size

def experiment():
    return model.run(
        npz_path=npz_path,
        epochs=num_epochs,
        batch_size=batch_size,
        num_units=num_units,
        input_len = input_len,
        target_len = target_len,
        rnn_cell=rnn_cell,
        eos_token = 0,
    )

#dyn_stats = experiment()
dyn_stats, preds_dict = get_or_run_nn(experiment, filename='006_gru_seq2seq_zeroEOS_60to30')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/13_price_history_seq2seq-raw-checkpoint.ipynb".
The first task is:
GRU cell - with EOS = 1000 - 50 epochs
Can you write Python code for it?

rnn_cell = PriceHistorySeq2Seq.RNN_CELLS.GRU
num_epochs = 50
num_epochs, num_units, batch_size

def experiment():
    return model.run(
        npz_path=npz_path,
        epochs=num_epochs,
        batch_size=batch_size,
        num_units=num_units,
        input_len = input_len,
        target_len = target_len,
        rnn_cell=rnn_cell,
    )

#dyn_stats = experiment()
dyn_stats, preds_dict = get_or_run_nn(experiment, filename='006_gru_seq2seq_EOS1000_50epochs_60to30')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/13_price_history_seq2seq-raw-checkpoint.ipynb".
The first task is:
Step 0 - hyperparams
vocab_size is all the potential words you could have (classification for translation case)
and max sequence length are the SAME thing
decoder RNN hidden units are usually same size as encoder RNN hidden units in translation but for our case it does not seem really to be a relationship there but we can experiment and find out later, not a priority thing right now
Can you write Python code for it?

num_epochs = 10

num_features = 1
num_units = 400 #state size

input_len = 60
target_len = 30

batch_size = 47

#trunc_backprop_len = ??


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/13_price_history_seq2seq-raw-checkpoint.ipynb".
The first task is:
Step 1 - collect data (and/or generate them)
Can you write Python code for it?

npz_path = '../price_history_03_dp_60to30_from_fixed_len.npz'

dp = PriceHistorySeq2SeqDataProvider(npz_path=npz_path, batch_size=batch_size)
dp.inputs.shape, dp.targets.shape

aa, bb = dp.next()
aa.shape, bb.shape


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/13_price_history_seq2seq-raw-checkpoint.ipynb".
The first task is:
Step 2 - Build model
```
PriceHistoryDummySeq2Seq(rng=random_state, dtype=dtype, config=config).getGraph(batch_size=batch_size,
                       num_units=num_units,
                       input_len=input_len,
                       target_len=target_len)
learning rate: 0.001000
60
Tensor("inputs/unstack:0", shape=(47, 1), dtype=float32)
Tensor("encoder_rnn_layer/rnn/basic_rnn_cell_59/Tanh:0", shape=(47, 400), dtype=float32)
decoder inputs series
<type 'list'>
30
Tensor("decoder_rnn_layer/unstack:0", shape=(47, 1), dtype=float32)
30
Tensor("decoder_rnn_layer/rnn/basic_rnn_cell/Tanh:0", shape=(47, 400), dtype=float32)
30
Tensor("readout_layer/add:0", shape=(47, 1), dtype=float32)
Tensor("predictions/Reshape:0", shape=(47, 30), dtype=float32)
Tensor("error/Select:0", shape=(47, 30), dtype=float32)
Tensor("error/Mean:0", shape=(), dtype=float32)
```
Can you write Python code for it?

model = PriceHistorySeq2Seq(rng=random_state, dtype=dtype, config=config)

# graph = model.getGraph(batch_size=batch_size,
#                        num_units=num_units,
#                        input_len=input_len,
#                        target_len=target_len)


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/13_price_history_seq2seq-raw-checkpoint.ipynb".
The first task is:
https://www.youtube.com/watch?v=ElmBrKyMXxs
https://github.com/hans/ipython-notebooks/blob/master/tf/TF%20tutorial.ipynb
https://github.com/ematvey/tensorflow-seq2seq-tutorials
Can you write Python code for it?

from __future__ import division
import tensorflow as tf
from os import path
import numpy as np
import pandas as pd
import csv
from sklearn.model_selection import StratifiedShuffleSplit
from time import time
from matplotlib import pyplot as plt
import seaborn as sns
from mylibs.jupyter_notebook_helper import show_graph
from tensorflow.contrib import rnn
from tensorflow.contrib import learn
import shutil
from tensorflow.contrib.learn.python.learn import learn_runner
from mylibs.tf_helper import getDefaultGPUconfig
from sklearn.metrics import r2_score
from mylibs.py_helper import factors
from fastdtw import fastdtw
from scipy.spatial.distance import euclidean
from statsmodels.tsa.stattools import coint
from common import get_or_run_nn
from data_providers.price_history_seq2seq_data_provider import PriceHistorySeq2SeqDataProvider
from models.price_history_seq2seq import PriceHistorySeq2Seq
from cost_functions.huber_loss import huber_loss
from os.path import isdir

dtype = tf.float32
seed = 16011984
random_state = np.random.RandomState(seed=seed)
config = getDefaultGPUconfig()
%matplotlib inline

data_folder = '../../../../Dropbox/data'
assert isdir(data_folder)


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/13_price_history_seq2seq-raw-checkpoint.ipynb".
The first task is:
learning rate: 0.001000
60
Tensor("inputs/unstack:0", shape=(47, 1), dtype=float32)
Tensor("encoder_rnn_layer/rnn/basic_rnn_cell_59/Tanh:0", shape=(47, 400), dtype=float32)
time: Tensor("decoder_rnn_layer/rnn/while/add:0", shape=(), dtype=int32)
<tensorflow.python.ops.tensor_array_ops.TensorArray object at 0x7fc5424ef090>
Tensor("decoder_rnn_layer/decoder_out_tensor/decoder_out_tensor:0", shape=(?, 47, 400), dtype=float32)
Tensor("readout_layer/readouts:0", shape=(?, 1), dtype=float32)
Tensor("predictions/Reshape:0", shape=(31, 47), dtype=float32)
Tensor("predictions/transpose:0", shape=(47, 31), dtype=float32)
Tensor("error/Select:0", shape=(47, 31), dtype=float32)
Tensor("error/Mean:0", shape=(), dtype=float32)
Can you write Python code for it?


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/15_price_history_seq2seq-native-with-last-input-as-decoder-input-checkpoint.ipynb".
The first task is:
Conclusion
???
GRU cell - without EOS - 800 units
Can you write Python code for it?

rnn_cell = PriceHistorySeq2SeqNative.RNN_CELLS.GRU
num_epochs = 50
num_units = 800
num_epochs, num_units, batch_size

def experiment():
    return model.run(
        npz_path=npz_path,
        epochs=num_epochs,
        batch_size=batch_size,
        num_units=num_units,
        input_len = input_len,
        target_len = target_len,
        rnn_cell=rnn_cell,
    )

#dyn_stats = experiment()
dyn_stats, preds_dict = get_or_run_nn(experiment, filename='008_gru_seq2seq_native_noEOS_60to30_50epochs_800units')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/15_price_history_seq2seq-native-with-last-input-as-decoder-input-checkpoint.ipynb".
The first task is:
Conclusion
The initial price difference of the predictions is still not as good as we would expect, perhaps using an EOS as they do in machine translation models is not the best architecture for our case
GRU cell - without EOS
Can you write Python code for it?

rnn_cell = PriceHistorySeq2SeqNative.RNN_CELLS.GRU
num_epochs = 50
num_epochs, num_units, batch_size

def experiment():
    return model.run(
        npz_path=npz_path,
        epochs=num_epochs,
        batch_size=batch_size,
        num_units=num_units,
        input_len = input_len,
        target_len = target_len,
        rnn_cell=rnn_cell,
    )

#dyn_stats = experiment()
dyn_stats, preds_dict = get_or_run_nn(experiment, filename='008_gru_seq2seq_native_noEOS_60to30_50epochs')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/15_price_history_seq2seq-native-with-last-input-as-decoder-input-checkpoint.ipynb".
The first task is:
Conclusion
There is no way this graph makes much sense but let's give it a try to see how bad really is
Step 3 training the network
RECALL: baseline is around 4 for huber loss for current problem, anything above 4 should be considered as major errors
Basic RNN cell (without EOS)
Can you write Python code for it?

model = PriceHistorySeq2SeqNative(rng=random_state, dtype=dtype, config=config, with_EOS=False)

rnn_cell = PriceHistorySeq2SeqNative.RNN_CELLS.BASIC_RNN
num_epochs = 20
num_epochs, num_units, batch_size

def experiment():
    return model.run(
        npz_path=npz_path,
        epochs=num_epochs,
        batch_size=batch_size,
        num_units=num_units,
        input_len = input_len,
        target_len = target_len,
        rnn_cell=rnn_cell,
    )

dyn_stats, preds_dict = get_or_run_nn(experiment, filename='008_rnn_seq2seq_native_noEOS_60to30_20epochs')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/15_price_history_seq2seq-native-with-last-input-as-decoder-input-checkpoint.ipynb".
The first task is:
Step 1 - collect data (and/or generate them)
Can you write Python code for it?

npz_path = '../price_history_03_dp_60to30_from_fixed_len.npz'

dp = PriceHistorySeq2SeqDataProvider(npz_path=npz_path, batch_size=batch_size, with_EOS=False)
dp.inputs.shape, dp.targets.shape

aa, bb = dp.next()
aa.shape, bb.shape


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/15_price_history_seq2seq-native-with-last-input-as-decoder-input-checkpoint.ipynb".
The first task is:
https://www.youtube.com/watch?v=ElmBrKyMXxs
https://github.com/hans/ipython-notebooks/blob/master/tf/TF%20tutorial.ipynb
https://github.com/ematvey/tensorflow-seq2seq-tutorials
Can you write Python code for it?

from __future__ import division
import tensorflow as tf
from os import path
import numpy as np
import pandas as pd
import csv
from sklearn.model_selection import StratifiedShuffleSplit
from time import time
from matplotlib import pyplot as plt
import seaborn as sns
from mylibs.jupyter_notebook_helper import show_graph
from tensorflow.contrib import rnn
from tensorflow.contrib import learn
import shutil
from tensorflow.contrib.learn.python.learn import learn_runner
from mylibs.tf_helper import getDefaultGPUconfig
from sklearn.metrics import r2_score
from mylibs.py_helper import factors
from fastdtw import fastdtw
from scipy.spatial.distance import euclidean
from statsmodels.tsa.stattools import coint
from common import get_or_run_nn
from data_providers.price_history_seq2seq_data_provider import PriceHistorySeq2SeqDataProvider
from models.price_history_seq2seq_native import PriceHistorySeq2SeqNative

dtype = tf.float32
seed = 16011984
random_state = np.random.RandomState(seed=seed)
config = getDefaultGPUconfig()
%matplotlib inline


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/21_price_history_seq2seq-dyn_dec_ins-checkpoint.ipynb".
The first task is:
Conclusion
???
Trying again with zero as first input to the decoder
Can you write Python code for it?

epochs = 20
num_units = 400
activation = tf.nn.elu
epochs, num_units, batch_size

best_learning_rate = 1e-3 #0.0026945952539362472
batch_norm_enabled = False

def experiment():
    return model.run(npz_path=npz_train,
             epochs=epochs,
             batch_size = batch_size,
             num_units = num_units,
             input_len=input_len,
             target_len=target_len,
             learning_rate = best_learning_rate,
             preds_gather_enabled = True,
             batch_norm_enabled = batch_norm_enabled,
             activation = activation,
            )


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/21_price_history_seq2seq-dyn_dec_ins-checkpoint.ipynb".
The first task is:
Once generate data
Can you write Python code for it?

data_path = '../data/price_history'

#npz_full_train = data_path + '/price_history_03_dp_60to30_train.npz'
#npz_full_train = data_path + '/price_history_60to30_targets_normed_train.npz'

#npz_train = data_path + '/price_history_03_dp_60to30_57980_train.npz'
#npz_train = data_path + '/price_history_03_dp_60to30_6400_train.npz'
npz_train = data_path + '/price_history_60to30_6400_targets_normed_train.npz'

#npz_test = data_path + '/price_history_03_dp_60to30_test.npz'
npz_test = data_path + '/price_history_60to30_targets_normed_test.npz'


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/21_price_history_seq2seq-dyn_dec_ins-checkpoint.ipynb".
The first task is:
Recall that without batch normalization within 10 epochs with num units 400 and batch_size 64 we reached at 4.940
and with having the decoder inputs NOT filled from the outputs
Can you write Python code for it?

%%time
dyn_stats, preds_dict = get_or_run_nn(
  experiment,
  filename='021_seq2seq_60to30_dyn_dec_ins_epochs{}_learning_rate_{:.4f}_batch_norm_{}_num_units{}_elu'.format(
      epochs, best_learning_rate, batch_norm_enabled, num_units
  ))

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


%%time
dyn_stats, preds_dict = get_or_run_nn(
  experiment,
  filename=
    '021_seq2seq_60to30_dyn_dec_ins_epochs{}_learning_rate_{:.4f}_batch_norm_{}_num_units{}_elu_zero_first'.format(
      epochs, best_learning_rate, batch_norm_enabled, num_units
  ))

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


%%time
dyn_stats, preds_dict = get_or_run_nn(
  experiment,
  filename='021_seq2seq_60to30_dyn_dec_ins_epochs{}_learning_rate_{:.4f}_prob_input{}_lamda2_{}'.format(
      epochs, best_learning_rate, keep_prob_input, lamda2
  ))

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/21_price_history_seq2seq-dyn_dec_ins-checkpoint.ipynb".
The first task is:
Step 1 - collect data
Can you write Python code for it?

dp = PriceHistorySeq2SeqDataProvider(npz_path=npz_train, batch_size=batch_size, with_EOS=with_EOS)
dp.inputs.shape, dp.targets.shape

aa, bb = dp.next()
aa.shape, bb.shape


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/21_price_history_seq2seq-dyn_dec_ins-checkpoint.ipynb".
The first task is:
Step 2 - Build model
Can you write Python code for it?

model = PriceHistorySeq2SeqDynDecIns(rng=random_state, dtype=dtype, config=config, with_EOS=with_EOS)

graph = model.getGraph(batch_size=batch_size,
                       num_units=num_units,
                       input_len=input_len,
                       target_len=target_len)

#show_graph(graph)


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/21_price_history_seq2seq-dyn_dec_ins-checkpoint.ipynb".
The first task is:
https://www.youtube.com/watch?v=ElmBrKyMXxs
https://github.com/hans/ipython-notebooks/blob/master/tf/TF%20tutorial.ipynb
https://github.com/ematvey/tensorflow-seq2seq-tutorials
Can you write Python code for it?

from __future__ import division
import tensorflow as tf
from os import path, remove
import numpy as np
import pandas as pd
import csv
from sklearn.model_selection import StratifiedShuffleSplit
from time import time
from matplotlib import pyplot as plt
import seaborn as sns
from mylibs.jupyter_notebook_helper import show_graph, renderStatsList, renderStatsCollection, \
    renderStatsListWithLabels, renderStatsCollectionOfCrossValids
from tensorflow.contrib import rnn
from tensorflow.contrib import learn
import shutil
from tensorflow.contrib.learn.python.learn import learn_runner
from mylibs.tf_helper import getDefaultGPUconfig
from sklearn.metrics import r2_score
from mylibs.py_helper import factors
from fastdtw import fastdtw
from collections import OrderedDict
from scipy.spatial.distance import euclidean
from statsmodels.tsa.stattools import coint
from common import get_or_run_nn
from data_providers.price_history_seq2seq_data_provider import PriceHistorySeq2SeqDataProvider
from data_providers.price_history_dataset_generator import PriceHistoryDatasetGenerator
from skopt.space.space import Integer, Real
from skopt import gp_minimize
from skopt.plots import plot_convergence
import pickle
import inspect
import dill
import sys
from models.price_history_21_seq2seq_dyn_dec_ins import PriceHistorySeq2SeqDynDecIns

dtype = tf.float32
seed = 16011984
random_state = np.random.RandomState(seed=seed)
config = getDefaultGPUconfig()
n_jobs = 1
%matplotlib inline


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/23_price_history_seq2seq-cross-validation-checkpoint.ipynb".
The first task is:
Cross Validating
Can you write Python code for it?

def plotter(stats_list, label_text):
    _ = renderStatsListWithLabels(stats_list=stats_list, label_text=label_text)
    plt.show()

    _ = renderStatsListWithLabels(stats_list=stats_list, label_text=label_text,
                                  title='Validation Error', kk='error(valid)')
    plt.show()

#sorted(factors(6400))

obj = PriceHistory23GpOpt(model=model,
                          stats_npy_filename = 'bayes_opt_23_stats_dic',
                          cv_score_dict_npy_filename = 'bayes_opt_23_cv_scores_dic',
                          random_state=random_state,
                          plotter = plotter,
                          npz_path=npz_train,
                          epochs=15,
                          batch_size=batch_size,
                          input_len=input_len,
                          target_len=target_len,
                          n_splits=5,
                         )

opt_res = obj.run_opt(n_random_starts=2, n_calls=17)

plot_res_gp(opt_res)

opt_res.best_params

filepath = PriceHistory23GpOpt.bayes_opt_dir + '/bayes_opt_23_stats_dic.npy'

renderStatsCollectionOfCrossValids(stats_dic=np.load(filepath)[()], label_texts=[
    'num_units', 'activation', 'lamda2', 'keep_prob_input', 'learning_rate'])
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/23_price_history_seq2seq-cross-validation-checkpoint.ipynb".
The first task is:
Once generate data
Can you write Python code for it?

data_path = '../data/price_history'

#npz_full_train = data_path + '/price_history_03_dp_60to30_train.npz'
#npz_full_train = data_path + '/price_history_60to30_targets_normed_train.npz'
npz_full_train = data_path + '/price_history_03_dp_60to30_global_remove_scale_targets_normed_train.npz'

#npz_train = data_path + '/price_history_03_dp_60to30_57980_train.npz'
#npz_train = data_path + '/price_history_03_dp_60to30_6400_train.npz'
#npz_train = data_path + '/price_history_60to30_6400_targets_normed_train.npz'
npz_train = data_path + '/price_history_03_dp_60to30_6400_global_remove_scale_targets_normed_train.npz'

#npz_test = data_path + '/price_history_03_dp_60to30_test.npz'
#npz_test = data_path + '/price_history_60to30_targets_normed_test.npz'
npz_test = data_path + '/price_history_03_dp_60to30_global_remove_scale_targets_normed_test.npz'


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/23_price_history_seq2seq-cross-validation-checkpoint.ipynb".
The first task is:
Step 0 - hyperparams
vocab_size is all the potential words you could have (classification for translation case)
and max sequence length are the SAME thing
decoder RNN hidden units are usually same size as encoder RNN hidden units in translation but for our case it does not seem really to be a relationship there but we can experiment and find out later, not a priority thing right now
Can you write Python code for it?

num_units = 400 #state size

input_len = 60
target_len = 30

batch_size = 64 #50
with_EOS = False

total_train_size = 57994
train_size = 6400
test_size = 1282


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/23_price_history_seq2seq-cross-validation-checkpoint.ipynb".
The first task is:
Step 1 - collect data
Can you write Python code for it?

# dp = PriceHistorySeq2SeqDataProvider(npz_path=npz_train, batch_size=batch_size, with_EOS=with_EOS)
# dp.inputs.shape, dp.targets.shape

# aa, bb = dp.next()
# aa.shape, bb.shape


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/23_price_history_seq2seq-cross-validation-checkpoint.ipynb".
The first task is:
Step 2 - Build model
Can you write Python code for it?

model = PriceHistorySeq2SeqDynDecIns(rng=random_state, dtype=dtype, config=config, with_EOS=with_EOS)

# graph = model.getGraph(batch_size=batch_size,
#                        num_units=num_units,
#                        input_len=input_len,
#                        target_len=target_len)

#show_graph(graph)


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/23_price_history_seq2seq-cross-validation-checkpoint.ipynb".
The first task is:
https://www.youtube.com/watch?v=ElmBrKyMXxs
https://github.com/hans/ipython-notebooks/blob/master/tf/TF%20tutorial.ipynb
https://github.com/ematvey/tensorflow-seq2seq-tutorials
Can you write Python code for it?

from __future__ import division
import tensorflow as tf
from os import path, remove
import numpy as np
import pandas as pd
import csv
from sklearn.model_selection import StratifiedShuffleSplit
from time import time
from matplotlib import pyplot as plt
import seaborn as sns
from mylibs.jupyter_notebook_helper import show_graph, renderStatsList, renderStatsCollection, \
    renderStatsListWithLabels, renderStatsCollectionOfCrossValids, plot_res_gp, my_plot_convergence
from tensorflow.contrib import rnn
from tensorflow.contrib import learn
import shutil
from tensorflow.contrib.learn.python.learn import learn_runner
from mylibs.tf_helper import getDefaultGPUconfig
from sklearn.metrics import r2_score
from mylibs.py_helper import factors
from fastdtw import fastdtw
from collections import OrderedDict
from scipy.spatial.distance import euclidean
from statsmodels.tsa.stattools import coint
from common import get_or_run_nn
from data_providers.price_history_seq2seq_data_provider import PriceHistorySeq2SeqDataProvider
from data_providers.price_history_dataset_generator import PriceHistoryDatasetGenerator
from skopt.space.space import Integer, Real
from skopt import gp_minimize
from skopt.plots import plot_convergence
import pickle
import inspect
import dill
import sys
from models.price_history_21_seq2seq_dyn_dec_ins import PriceHistorySeq2SeqDynDecIns
from gp_opt.price_history_23_gp_opt import PriceHistory23GpOpt

dtype = tf.float32
seed = 16011984
random_state = np.random.RandomState(seed=seed)
config = getDefaultGPUconfig()
n_jobs = 1
%matplotlib inline


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/30_price_history_dataset_per_mobile_phone-arima-checkpoint.ipynb".
The first task is:
All tests
Can you write Python code for it?

from arima.arima_testing import ArimaTesting

best_params, target_len, npz_full

%%time
keys, scores, preds = ArimaTesting.full_testing(best_params=best_params, target_len=target_len,
                                                    npz_full=npz_full)

# render graphs here

score_arr = np.array(scores)

np.mean(score_arr[np.logical_not(score_arr != score_arr)])


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/30_price_history_dataset_per_mobile_phone-arima-checkpoint.ipynb".
The first task is:
Arima
Can you write Python code for it?

parameters = OrderedDict([
    ('p_auto_regression_order', range(6)), #0-5
    ('d_integration_level', range(3)), #0-2
    ('q_moving_average', range(6)), #0-5
])

cart = cartesian_coord(*parameters.values())
cart.shape

cur_key = dic.keys()[0]
cur_key

cur_sku = dic[cur_key][()]
cur_sku.keys()

train_mat = cur_sku['train']
train_mat.shape

target_len

inputs = train_mat[:, :-target_len]
inputs.shape

targets = train_mat[:, -target_len:]
targets.shape

easy_mode = False

score_dic_filepath = data_path + "/arima/scoredic_easy_mode_{}_{}.npy".format(easy_mode, cur_key)
path.abspath(score_dic_filepath)

%%time
with warnings.catch_warnings():
    warnings.filterwarnings("ignore")
    scoredic = ArimaCV.cross_validate(inputs=inputs, targets=targets, cartesian_combinations=cart,
                                      score_dic_filepath=score_dic_filepath, easy_mode=easy_mode)

#4h 4min 51s / 108 cases => ~= 136 seconds per case !

arr = np.array(list(scoredic.iteritems()))
arr.shape

#np.isnan()
filtered_arr = arr[ np.logical_not(arr[:, 1] != arr[:, 1]) ]
filtered_arr.shape

plt.plot(filtered_arr[:, 1])

minarg = np.argmin(filtered_arr[:, 1])
minarg

best_params = filtered_arr[minarg, 0]
best_params

test_mat = cur_sku['test']
test_ins = test_mat[:-target_len]
test_ins.shape

test_tars = test_mat[-target_len:]
test_tars.shape

test_ins_vals = test_ins.values.reshape(1, -1)
test_ins_vals.shape

test_tars_vals = test_tars.values.reshape(1, -1)
test_tars_vals.shape


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/30_price_history_dataset_per_mobile_phone-arima-checkpoint.ipynb".
The first task is:
Conclusion
If you are training in easy mode then what you get at the end is that the model only cares for the previous value in order to do its predictions and this makes it much easier for everybody but in reality we might not have advantage
Trying
Can you write Python code for it?

args = np.argsort(filtered_arr[:, 1])
args

filtered_arr[args[:10], 0]

%%time
with warnings.catch_warnings():
    warnings.filterwarnings("ignore")
    ae = ArimaEstimator(p_auto_regression_order=4,
                        d_integration_level=1,
                        q_moving_average=3,
                        easy_mode=False)
    print ae.fit(test_ins_vals, test_tars_vals).score(test_ins_vals, test_tars_vals)

plt.figure(figsize=(15,7))
plt.plot(ae.preds.flatten(), label='preds')
test_tars.plot(label='real')
plt.legend()
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/30_price_history_dataset_per_mobile_phone-arima-checkpoint.ipynb".
The first task is:
Step 0 - hyperparams
vocab_size is all the potential words you could have (classification for translation case)
and max sequence length are the SAME thing
decoder RNN hidden units are usually same size as encoder RNN hidden units in translation but for our case it does not seem really to be a relationship there but we can experiment and find out later, not a priority thing right now
Can you write Python code for it?

input_len = 60
target_len = 30
batch_size = 50
with_EOS = False

csv_in = '../price_history_03_seq_start_suddens_trimmed.csv'


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/30_price_history_dataset_per_mobile_phone-arima-checkpoint.ipynb".
The first task is:
Testing with easy mode off
Can you write Python code for it?

%%time
with warnings.catch_warnings():
    warnings.filterwarnings("ignore")
    ae = ArimaEstimator(p_auto_regression_order=best_params[0],
                        d_integration_level=best_params[1],
                        q_moving_average=best_params[2],
                        easy_mode=False)
    score = ae.fit(test_ins_vals, test_tars_vals).score(test_ins_vals, test_tars_vals)

score

plt.figure(figsize=(15,7))
plt.plot(ae.preds.flatten(), label='preds')
test_tars.plot(label='real')
plt.legend()
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/30_price_history_dataset_per_mobile_phone-arima-checkpoint.ipynb".
The first task is:
Testing with easy mode on
Can you write Python code for it?

%%time
with warnings.catch_warnings():
    warnings.filterwarnings("ignore")
    ae = ArimaEstimator(p_auto_regression_order=best_params[0],
                        d_integration_level=best_params[1],
                        q_moving_average=best_params[2],
                        easy_mode=True)
    score = ae.fit(test_ins_vals, test_tars_vals).score(test_ins_vals, test_tars_vals)

score

plt.figure(figsize=(15,7))
plt.plot(ae.preds.flatten(), label='preds')
test_tars.plot(label='real')
plt.legend()
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/31_arima_stationary_price_history-checkpoint.ipynb".
The first task is:

Can you write Python code for it?

parameters = OrderedDict([
    ('p_auto_regression_order', range(6)), #0-5
    ('d_integration_level', range(3)), #0-2
    ('q_moving_average', range(6)), #0-5
])

def cartesian_coord(*arrays):
    grid = np.meshgrid(*arrays)
    coord_list = [entry.ravel() for entry in grid]
    points = np.vstack(coord_list).T
    return points

cart = cartesian_coord(*parameters.values())
cart.shape

108 * 15 / 60 / 2 #in hours!

count_inds=100
random_state = np.random.RandomState(seed=16011984)

random_inds = random_state.choice(range(len(XX)), count_inds, replace=False)
xx = XX[random_inds]
yy = YY[random_inds]

filepath = 'scoredic.npy'

%%time
with warnings.catch_warnings():
    warnings.filterwarnings("ignore")
    for pp, dd, qq in cart:
        scoredic = np.load(filepath)[()] if path.isfile(filepath) else OrderedDict()
        cur_tuple = (pp, dd, qq)
        if cur_tuple in scoredic:
            continue
        else:
            ae = ArimaEstimator(p_auto_regression_order=pp, d_integration_level=dd, q_moving_average=qq,
                                easy_mode=True)
            scoredic[cur_tuple] = ae.fit(inputs=xx, targets=yy).score(xx, yy)
            np.save(filepath, scoredic)

from sklearn.model_selection import GridSearchCV

# Pick hyperparameters
# There is NO separation of training and testing set because there is no transfer of knowledge from the training
# that could be useful for validation
# Run Arima prediction steps for all instances of training dataset and get a fastdtw score from each case
# Get the mean of the fastdtw and this is the CV score
#
# Do this for all possible parameters

fitted = model.fit()

fitted.

model_fit = model.fit(disp=0)

model_fit.summary()

residuals = DataFrame(model_fit.resid)

residuals.plot()
plt.show()

residuals.plot(kind='kde')
plt.show()

residuals.describe()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/31_arima_stationary_price_history-checkpoint.ipynb".
The first task is:
Reverse Moving average
https://stats.stackexchange.com/questions/67907/extract-data-points-from-moving-average
Can you write Python code for it?

orig_seq.plot()

aa = np.zeros((orig_seq.size - window +1, orig_seq.size))
aa.shape

for ii, item in enumerate(aa):
    aa[ii, ii:ii+window] = 1/window

trans1 = aa.dot(orig_seq)
trans1.shape

from numpy.linalg import pinv

trans2 = orig_seq.rolling(window=window).mean()[window-1:].values
trans2.shape

np.allclose(trans1, trans2)

pA = pinv(aa)
pA.shape

reconstructed = pA.dot(trans1)
reconstructed.shape

plt.plot(reconstructed, label='reconstructed')
plt.plot(orig_seq.values, label='original')
plt.legend()
plt.show()

np.allclose(reconstructed, orig_seq.values, rtol=1e-3)

#np.sqrt(np.mean((reconstructed - orig_seq.values)**2)) #RMSE
np.corrcoef(orig_seq.values, reconstructed)


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/31_arima_stationary_price_history-checkpoint.ipynb".
The first task is:
Train - Test
Can you write Python code for it?

XX = series.values
XX.shape

#splitting
train_size = int(len(XX)*2/3)
train_size

train_set = XX[:train_size]
test_set = XX[train_size:]
train_set.shape, test_set.shape

train_set[0]

history = list(train_set)

history[0] = 3333

history = list(train_set)
len(history)

predictions = []

for tt in range(len(test_set)):
    output = ARIMA(history, order=(5,1,0)).fit(disp=0).forecast()
    y_hat = output[0] #0th is the index of the prediction
    predictions.append(y_hat)

    observation = test_set[tt]
    #history.append(observation)
    history.append(y_hat)

    print "predicted: {}, expected: {}".format(y_hat, observation)

error = mean_squared_error(predictions, test_set)
error

plt.figure(figsize=(15,6))
plt.plot(predictions, label='preds')
plt.plot(test_set, label='test set')
plt.legend()
plt.show()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/31_arima_stationary_price_history-checkpoint.ipynb".
The first task is:
http://machinelearningmastery.com/arima-for-time-series-forecasting-with-python/
http://ucanalytics.com/blogs/arima-models-manufacturing-case-study-example-part-3/
https://stats.stackexchange.com/questions/23036/estimating-same-model-over-multiple-time-series
The idea with ARIMA models is that the final residual should look like white noise otherwise, there is juice or information available in the data to extract
Can you write Python code for it?

from __future__ import division
from os import path, remove
import numpy as np
import pandas as pd
import csv
from sklearn.model_selection import StratifiedShuffleSplit
from time import time
from matplotlib import pyplot as plt
import seaborn as sns
from tensorflow.contrib import rnn
from tensorflow.contrib import learn
import shutil
from tensorflow.contrib.learn.python.learn import learn_runner
from sklearn.metrics import r2_score
from fastdtw import fastdtw
from collections import OrderedDict
from scipy.spatial.distance import euclidean
from statsmodels.tsa.stattools import coint
from skopt.space.space import Integer, Real
from skopt import gp_minimize
from skopt.plots import plot_convergence
import pickle
import inspect
import dill
import sys
from pandas import read_csv
from pandas import datetime
from pandas.plotting import autocorrelation_plot
from statsmodels.graphics.tsaplots import plot_pacf
from statsmodels.tsa.arima_model import ARIMA
from pandas import DataFrame
from sklearn.metrics import mean_squared_error
from price_history import PriceHistory
from arima.arima_estimator import ArimaEstimator
import warnings
from collections import OrderedDict

%matplotlib inline


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/33_price_history_autoencoder_dyn_rnn-checkpoint.ipynb".
The first task is:
Quick test run
Can you write Python code for it?

def experiment():
    return model.run(npz_path=npz_path,
                     epochs=100,
                     batch_size = 13,
                     enc_num_units = 400,
                     dec_num_units = 400,
                     ts_len=max_seq_len,
                     learning_rate = 1e-4,
                     preds_gather_enabled = False,
                    )
dyn_stats = experiment()

dyn_stats.plotStats()


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/33_price_history_autoencoder_dyn_rnn-checkpoint.ipynb".
The first task is:
Step 0 - hyperparams
Can you write Python code for it?

factors(689)

max_seq_len = 682

#full_train_size = 55820
#train_size = 55800
#small_train_size = 6000 #just because of performance reasons, no statistics behind this decision
#test_size = 6200

data_path = '../../../../Dropbox/data'

phae_path = data_path + '/price_hist_autoencoder'

csv_in = '../price_history_03_seq_start_suddens_trimmed.csv'
assert path.isfile(csv_in)

npz_unprocessed = phae_path + '/price_history_full_seqs.npz'
assert path.isfile(npz_unprocessed)

npz_dates = phae_path + '/price_history_full_seqs_dates.npz'
assert path.isfile(npz_dates)

npz_train = phae_path + '/price_history_seqs_dates_normed_train.npz'
assert path.isfile(npz_train)

npz_test = phae_path + '/price_history_seqs_dates_normed_test.npz'
assert path.isfile(npz_test)

npz_path = npz_train[:-len('_train.npz')]

for key, val in np.load(npz_train).iteritems():
    print key, ",", val.shape


I am working on the file "04_time_series_prediction/.ipynb_checkpoints/33_price_history_autoencoder_dyn_rnn-checkpoint.ipynb".
The first task is:
targets
Tensor("data/strided_slice:0", shape=(50, 682), dtype=float32)
Tensor("encoder_rnn_layer/rnn/while/Exit_2:0", shape=(?, 10), dtype=float32)
Tensor("encoder_state_out_process/Elu:0", shape=(?, 2), dtype=float32)
Tensor("decoder_state_in_process/Elu:0", shape=(?, 10), dtype=float32)
Tensor("decoder_rnn_layer/rnn/transpose:0", shape=(50, 682, 10), dtype=float32)
Tensor("decoder_outs/Reshape:0", shape=(34100, 10), dtype=float32)
Tensor("readout_affine/Identity:0", shape=(34100, 1), dtype=float32)
Tensor("readout_affine/Reshape:0", shape=(50, 682), dtype=float32)
Tensor("error/mul_1:0", shape=(50, 682), dtype=float32)
Tensor("error/Mean:0", shape=(), dtype=float32)
Tensor("error/Mean:0", shape=(), dtype=float32)
Can you write Python code for it?


I am working on the file "04_time_series_prediction/06_price_history_varlen-no-outliers.ipynb".
The first task is:
Step 0 - hyperparams
Can you write Python code for it?

num_epochs = 10
series_max_len = 60
num_features = 1  #just one here, the function we are predicting is one-dimensional
state_size = 400
target_len = 30
batch_size = 47


I am working on the file "04_time_series_prediction/06_price_history_varlen-no-outliers.ipynb".
The first task is:
Step 1 - collect data (and/or generate them)
Can you write Python code for it?

csv_in = '../price_history_03a_fixed_width.csv'
npz_path = '../price_history_03_dp_60to30_from_fixed_len.npz'

# XX, YY, sequence_lens, seq_mask = PriceHistoryVarLenDataProvider.createAndSaveDataset(
#     csv_in=csv_in,
#     npz_out=npz_path,
#     input_seq_len=60, target_seq_len=30)
# XX.shape, YY.shape, sequence_lens.shape, seq_mask.shape

dp = PriceHistoryVarLenDataProvider(filteringSeqLens = lambda xx : xx >= target_len,
                                    npz_path=npz_path)
dp.inputs.shape, dp.targets.shape, dp.sequence_lengths.shape, dp.sequence_masks.shape


I am working on the file "04_time_series_prediction/06_price_history_varlen-no-outliers.ipynb".
The first task is:
Step 3 training the network
Can you write Python code for it?

num_epochs, state_size, batch_size

def experiment():
    dynStats, predictions_dict = model.run(epochs=num_epochs,
                                        state_size=state_size,
                                         series_max_len=series_max_len,
                                         target_len=target_len,
                                         npz_path=npz_path,
                                         batch_size=batch_size)
    return dynStats, predictions_dict

from os.path import isdir
data_folder = '../../../../Dropbox/data'
assert isdir(data_folder)

dyn_stats, preds_dict = get_or_run_nn(experiment,
                                      filename='001_plain_rnn_60to30', nn_runs_folder= data_folder + '/nn_runs')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

sns.tsplot(data=dp.inputs[ind].flatten())

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

from cost_functions.huber_loss import huber_loss

average_huber_loss = np.mean([np.mean(huber_loss(dp.targets[ind], preds_dict[ind]))
                              for ind in range(len(dp.targets))])
average_huber_loss

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)


I am working on the file "04_time_series_prediction/07_price_history_varlen_rnn_cells.ipynb".
The first task is:
Conclusion
GRU has performed much better than basic RNN
GRU cell - 50 epochs
Can you write Python code for it?

rnn_cell = PriceHistoryRnnVarlen.RNN_CELLS.GRU
num_epochs = 50
state_size, batch_size

def experiment():
    dynStats, predictions_dict = model.run(epochs=num_epochs,
                                           rnn_cell=rnn_cell,
                                           state_size=state_size,
                                           series_max_len=series_max_len,
                                           target_len=target_len,
                                           npz_path=npz_path,
                                           batch_size=batch_size)
    return dynStats, predictions_dict

dyn_stats, preds_dict = get_or_run_nn(experiment,
                                      filename='002_rnn_gru_60to30_50epochs',
                                      nn_runs_folder= data_folder + '/nn_runs')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

average_huber_loss = np.mean([np.mean(huber_loss(dp.targets[ind], preds_dict[ind]))
                              for ind in range(len(dp.targets))])
average_huber_loss


I am working on the file "04_time_series_prediction/07_price_history_varlen_rnn_cells.ipynb".
The first task is:
Step 2 - Build model
Can you write Python code for it?

model = PriceHistoryRnnVarlen(rng=random_state, dtype=dtype, config=config)

graph = model.getGraph(batch_size=batch_size, state_size=state_size,
                       rnn_cell= PriceHistoryRnnVarlen.RNN_CELLS.GRU,
                       target_len=target_len, series_max_len=series_max_len)

show_graph(graph)


I am working on the file "04_time_series_prediction/07_price_history_varlen_rnn_cells.ipynb".
The first task is:
Step 3 training the network
GRU cell
Can you write Python code for it?

rnn_cell = PriceHistoryRnnVarlen.RNN_CELLS.GRU
num_epochs, state_size, batch_size

def experiment():
    dynStats, predictions_dict = model.run(epochs=num_epochs,
                                           rnn_cell=rnn_cell,
                                           state_size=state_size,
                                           series_max_len=series_max_len,
                                           target_len=target_len,
                                           npz_path=npz_path,
                                           batch_size=batch_size)
    return dynStats, predictions_dict

from os.path import isdir
data_folder = '../../../../Dropbox/data'
assert isdir(data_folder)

dyn_stats, preds_dict = get_or_run_nn(experiment,
                                      filename='002_rnn_gru_60to30', nn_runs_folder= data_folder + '/nn_runs')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

average_huber_loss = np.mean([np.mean(huber_loss(dp.targets[ind], preds_dict[ind]))
                              for ind in range(len(dp.targets))])
average_huber_loss


I am working on the file "04_time_series_prediction/07_price_history_varlen_rnn_cells.ipynb".
The first task is:
https://r2rt.com/recurrent-neural-networks-in-tensorflow-iii-variable-length-sequences.html
Can you write Python code for it?

from __future__ import division
import tensorflow as tf
from os import path
import numpy as np
import pandas as pd
import csv
from sklearn.model_selection import StratifiedShuffleSplit
from time import time
from matplotlib import pyplot as plt
import seaborn as sns
from mylibs.jupyter_notebook_helper import show_graph
from tensorflow.contrib import rnn
from tensorflow.contrib import learn
import shutil
from tensorflow.contrib.learn.python.learn import learn_runner
from IPython.display import Image
from IPython.core.display import HTML
from mylibs.tf_helper import getDefaultGPUconfig
from data_providers.binary_shifter_varlen_data_provider import \
    BinaryShifterVarLenDataProvider
from data_providers.price_history_varlen_data_provider import PriceHistoryVarLenDataProvider
from models.model_05_price_history_rnn_varlen import PriceHistoryRnnVarlen
from sklearn.metrics import r2_score
from mylibs.py_helper import factors
from fastdtw import fastdtw
from scipy.spatial.distance import euclidean
from statsmodels.tsa.stattools import coint
from cost_functions.huber_loss import huber_loss

dtype = tf.float32
seed = 16011984
random_state = np.random.RandomState(seed=seed)
config = getDefaultGPUconfig()
%matplotlib inline

from common import get_or_run_nn


I am working on the file "04_time_series_prediction/11_price_history_dummy_seq2seq_EOS_zero.ipynb".
The first task is:
Conclusion
//...
Basic RNN cell - with EOS = 0
Can you write Python code for it?

rnn_cell = PriceHistorySeq2Seq.RNN_CELLS.BASIC_RNN
num_epochs = 50
num_epochs, num_units, batch_size

def experiment():
    return model.run(
        npz_path=npz_path,
        epochs=num_epochs,
        batch_size=batch_size,
        num_units=num_units,
        input_len = input_len,
        target_len = target_len,
        rnn_cell=rnn_cell,
    )

dyn_stats, preds_dict = get_or_run_nn(experiment, filename='004_rnn_dummy_seq2seq_60to30')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


I am working on the file "04_time_series_prediction/11_price_history_dummy_seq2seq_EOS_zero.ipynb".
The first task is:
Conclusion
They are obviously shifted...
GRU cell - with EOS = 0
Can you write Python code for it?

rnn_cell = PriceHistorySeq2Seq.RNN_CELLS.GRU
num_epochs = 20
num_epochs, num_units, batch_size

def experiment():
    return model.run(
        npz_path=npz_path,
        epochs=num_epochs,
        batch_size=batch_size,
        num_units=num_units,
        input_len = input_len,
        target_len = target_len,
        rnn_cell=rnn_cell,
    )

#dyn_stats = experiment()
dyn_stats, preds_dict = get_or_run_nn(experiment, filename='004_rnn_dummy_seq2seq_gru_60to30')

dyn_stats.plotStats()
plt.show()

r2_scores = [r2_score(y_true=dp.targets[ind], y_pred=preds_dict[ind])
            for ind in range(len(dp.targets))]

ind = np.argmin(r2_scores)
ind

reals = dp.targets[ind]
preds = preds_dict[ind]

r2_score(y_true=reals, y_pred=preds)

sns.tsplot(data=dp.inputs[ind].flatten())

fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()

%%time
dtw_scores = [fastdtw(dp.targets[ind], preds_dict[ind])[0]
             for ind in range(len(dp.targets))]

np.mean(dtw_scores)

coint(preds, reals)

cur_ind = np.random.randint(len(dp.targets))
reals = dp.targets[cur_ind]
preds = preds_dict[cur_ind]
fig = plt.figure(figsize=(15,6))
plt.plot(reals, 'b')
plt.plot(preds, 'g')
plt.legend(['reals','preds'])
plt.show()


I am working on the file "04_time_series_prediction/11_price_history_dummy_seq2seq_EOS_zero.ipynb".
The first task is:
Step 0 - hyperparams
vocab_size and max sequence length are the SAME thing
decoder RNN hidden units are usually same size as encoder RNN hidden units in translation but for our case it does not seem really to be a relationship there but we can experiment and find out later, not a priority thing right now
Can you write Python code for it?

num_epochs = 10

num_features = 1
num_units = 400 #state size

input_len = 60
target_len = 30

batch_size = 47

#trunc_backprop_len = ??


I am working on the file "04_time_series_prediction/11_price_history_dummy_seq2seq_EOS_zero.ipynb".
The first task is:
Step 2 - Build model
Can you write Python code for it?

model = PriceHistoryDummySeq2Seq(rng=random_state, dtype=dtype, config=config)

graph = model.getGraph(batch_size=batch_size,
                       num_units=num_units,
                       input_len=input_len,
                       target_len=target_len)

#show_graph(graph)