I am working on the file "Extras/notebooks/.ipynb_checkpoints/Fergus SSL-checkpoint.ipynb".
The first task is:
Good luck telling me where the clusters are! Now the dimensions are highly correlated. This is why, if this was the original data we were given, we need to perform a rotation on the data--decorrelate the axes so they become separable in one dimension (i.e. histograms).
Can you write Python code for it?

from sklearn.decomposition import RandomizedPCA

pca = RandomizedPCA(n_components = 2, random_state = 12)
Xt = pca.fit_transform(Xr)
plt.scatter(Xt[:, 0], Xt[:, 1])

plt.figure(0)
plt.hist(Xt[:, 0], bins = B, normed = True)
plt.title("Dimension 1 ($x$-axis)")
plt.figure(1)
plt.hist(Xt[:, 1], bins = B, normed = True)
plt.title("Dimension 2 ($y$-axis)")


I am working on the file "Extras/notebooks/.ipynb_checkpoints/Fergus SSL-checkpoint.ipynb".
The first task is:
Two extremely well-separated Gaussian clusters. This will be easy--really not even requiring spectral analysis--but it's fine for demo purposes.
Let's explore
First step in any data science project: get a feel for the data. Fergus SSL relies quite on a big on the dimensions of the data being separable, so let's plot the histograms of each individual dimension.
Can you write Python code for it?

# We'll make the number of bins, B
B = 50
plt.figure(0)
plt.hist(X[:, 0], bins = B, normed = True)
plt.title("Dimension 1 ($x$-axis)")
plt.figure(1)
plt.hist(X[:, 1], bins = B, normed = True)
plt.title("Dimension 2 ($y$-axis)")


I am working on the file "Extras/notebooks/.ipynb_checkpoints/Fergus SSL-checkpoint.ipynb".
The first task is:
We can see things happening with the eigenfunctions--and, clearly, the eigenfunctions associated with the smallest eigenvalues are useless, but we knew this from looking at the examples in the Fergus paper itself--but let's continue on to get a feel for things in the interpolation process.
For simplicity, we'll just use the eigenfunctions associated with the 2nd axis ($y$ axis, in this particular case, but in $n$ dimensions this could be any axis).
Can you write Python code for it?

# Assign the eigenfunctions
EF = U2[:, 1:3]  # Take the eigenfunctions, starting at index 1 (inclusive) and ending at 3 (exclusive).
EV = v2[1:3]  # Take the eigenvalues associated with the eigenfunctions

# Create the x-axis from the histogram edges.
x1 = np.array([e1[i:i + 2].mean() for i in range(e1.shape[0] - 1)])  # Finds the midpoints of the histogram bins.
x2 = np.array([e2[i:i + 2].mean() for i in range(e2.shape[0] - 1)])

# Now, use the eigenfunctions (y-axis) to create the interpolators.
f1 = interp1d(x1, EF[:, 0])
f2 = interp1d(x2, EF[:, 1])

# What do they look like?
newdata = np.linspace(x1.min(), x1.max(), 100)
plt.figure(0)
plt.title("$x_1$ dimension")
plt.plot(x1, EF[:, 0], 'o', label = 'Original')
plt.plot(newdata, f1(newdata), '-', label = 'Interpolator')
plt.legend(loc = 0)

newdata = np.linspace(x2.min(), x2.max(), 100)
plt.figure(1)
plt.title("$x_2$ dimension")
plt.plot(x2, EF[:, 1], 'o', label = 'Original')
plt.plot(newdata, f2(newdata), '-', label = 'Interpolator')
plt.legend(loc = 0)

# We now have our eigenfunctions, at least in 1D. Let's see if we can reconstruct that heatmap from Fig 1.

plt.title("Original Data")
_ = plt.hexbin(Xt[:, 0], Xt[:, 1])

x = np.linspace(-2, 2.5, num = 100)  # Our original (rotated) data varies from about (-2, 2.5) on the x-axis.
y = np.linspace(-1, 1.0, num = 100)  # ...and from about (-1, 1) on the y-axis.

# Evaluate the eigenfunctions on each dimension.
fx = f1(x)
fy = f2(y)

# Plot the heatmaps along the grid.
plt.figure(0)
plt.title("Dim $x_1$: $\sigma_1$ = {:.2f}".format(EV[0]))


I am working on the file "Extras/notebooks/.ipynb_checkpoints/Fergus SSL-checkpoint.ipynb".
The first task is:
We've effectively rotated the data about 45 degrees. If we now plot the 1D histograms for the $x$ and $y$ axes again...
Can you write Python code for it?

plt.figure(0)
plt.hist(Xr[:, 0], bins = B, normed = True)
plt.title("Dimension 1 ($x$-axis)")
plt.figure(1)
plt.hist(Xr[:, 1], bins = B, normed = True)
plt.title("Dimension 2 ($y$-axis)")


I am working on the file "Extras/notebooks/.ipynb_checkpoints/Fergus SSL-checkpoint.ipynb".
The first task is:
You see the data are easily separable along both the $x$ and $y$ axes. Put another way--if someone gives you an $x$ value of a data point and asks you what cluster the point comes from, it's easy to make that assessment. Same with just providing the $y$ value of a data point--you could easily predict what cluster it comes from.
Now let's make things a little harder.
Can you write Python code for it?

rng = np.random.RandomState(74)
t = rng.normal(size = (2, 2))
Xr = X @ t
plt.scatter(Xr[:, 0], Xr[:, 1])


I am working on the file "FACLA/HW1.ipynb".
The first task is:
1.

Consider the polynomial $p(x) = (x-2)^9 = x^9 - 18x^8 + 144x^7 - 672x^6 + 2016x^5 - 4032x^4 + 5376x^3 - 4608x^2 + 2304x - 512$

a. Plot $p(x)$ for $x=1.920,\,1.921,\,1.922,\ldots,2.080$ evaluating $p$ via its coefficients $1,\,,-18,\,144,\ldots$
b. Plot the same plot again, now evaluating $p$ via the expression $(x-2)^9$.
c. Explain the difference.
(The numpy method linspace will be useful for this)
Can you write Python code for it?


I am working on the file "FACLA/HW1.ipynb".
The first task is:
WNx: I think my original interpretation of what "evaluate p by its coefficients" meant was wrong, so I'm leaving it out of the final "All" plot, it just drowns everything else out.
c:
WNx: $p(x) = (x-2)^9$ is the 'general' version of the Coefficient interpretation of $p$. It captures the overall trend of $p$ without all the detail. Kind of an average -- gives you the overall picture of what's going on. For instance you'd compresss signal $p$ to its $(x-2)^9$ form, instead of saving its full coeff form.
2.

2. How many different double-precision numbers are there?  Express your answer using powers of 2
WNx: $2^{64} - (2^{53} - 2^0$) for IEEE 754 64-bit Double. See: Quora Link
3.

3. Using the updated Numbers Every Programmer Should Know, how much longer does a main memory reference take than an L1 cache look-up?  How much longer does a disk seek take than a main memory reference?
Can you write Python code for it?


I am working on the file "FACLA/HW1.ipynb".
The first task is:
WNx: gonna have to do SVD to find the singular values of $A$. Then make a matrix $B$ st. $A=QBQ^T$. Then check that A.σ == B.σ. C.Mellon U. page on SVD
From the Lesson 2 notebook, I think I'll start with $B$ and compute $A$ acc. to the eqn, then check σ's of both.
Aha. So σ is s is S. The diagonal matrix of singular values. Everyone's using different names for the same thing. bastards.
Can you write Python code for it?

# setting A & B
B = np.random.randint(-100,100,(3,3))
A = Q@B@Q.T

Ua, sa, Va = np.linalg.svd(A, full_matrices=False)
Ub, sb, Vb = np.linalg.svd(B, full_matrices=False)

# sa & sb are the singular values of A and B
np.isclose(sa, sb)

sa, sb


I am working on the file "FACLA/HW1.ipynb".
The first task is:
WNx: wait, what does it mean to evaluate a function by its coefficients? How is that different than just evaluating it?
--> does she mean to ignore the exponents? Because that would make b. make more sense.. I .. think.*
Can you write Python code for it?

# Signature: np.linspace(start, stop, num=50, endpoint=True, retstep=False, dtype=None)
np.linspace(1.92, 2.08, num=161)
# np.arange(1.92, 2.08, 0.001)

start = 1.92
stop  = 2.08
num   = int((stop-start)/0.001 + 1) # =161

x = np.linspace(start, stop, num)

def p_cœf(x):
    return x - 18*x + 144*x - 672*x + 2016*x - 4032*x + 5376*x - 4608*x + 2304*x - 512
def p_cœf_alt(x):
    return p(x,0)
def p_ex9(x):
    return p(x,1)


I am working on the file "FACLA/L3-notes.ipynb".
The first task is:
Computational Complexity for a MxN matrix in SVD is $M^2N+N^3$, so Randomized (Truncated?) SVD is a massive improvement.

2018/3/7
Write a loop to calculate the error of your decomposition as your vary the # of topics. Plot the results.
Can you write Python code for it?

# 1. how do I calculate decomposition error?:
# I guess I'll use MSE?

# # NumPy:    # https://stackoverflow.com/questions/16774849/mean-squared-error-in-numpy
# def MSEnp(A,B):
#     if type(A) == np.ndarray and type(B) == np.ndarray:
#         return ((A - B) ** 2).mean()
#     else:
#         return np.square((A - B)).mean()

# Scikit-Learn:
from sklearn import metrics
MSE = metrics.mean_squared_error  # usg: mse(A,B)

# 2. Now how to recompose my decomposition?:

%time B = vectors  # original matrix
%time U, S, V = randomized_svd(B, 10) # num_topics = 10

# S is vector of Σ's singular values. Convert back to matrix:
%time Σ = S * np.eye(S.shape[0])

# from SVD formula: A ≈ U@Σ@V.T
%time A = U@Σ@V  ## apparently randomized_svd returns V.T, not V ?

# 3. Finally calculated error I guess:

%time mse_error = MSE(A,B)
print(mse_error)

# Im putting way too much effort into this lol
def fib(n):
    if n <= 1:
        return n
    else:
        f1 = 1
        f2 = 0
        for i in range(n):
            t = f1 + f2
            tmp = f2
            f2 += f1
            f1 = tmp
        return t

for i,e in enumerate(num_topics):
    print(f'Topics: {num_topics[i]:>3}    ',
          f'Time: {num_topics[i]:>3}')

## Setup
import time

B = vectors
num_topics = [fib(i) for i in range(2,14)]
TnE = []  # time & error

## Loop:
for n_topics in num_topics:
    t0 = time.time()
    U, S, Vt = randomized_svd(B, n_topics)
    Σ = S * np.eye(S.shape[0])
    A = U@Σ@Vt
    TnE.append([time.time() - t0, MSE(A,B)])

for i, tne in enumerate(TnE):
    print(f'Topics: {num_topics[i]:>3}    '
          f'Time: {np.round(tne[0],3):>3}    '
          f'Error: {np.round(tne[1],12):>3}')

# https://matplotlib.org/users/pyplot_tutorial.html
plt.plot(num_topics, [tne[1] for tne in TnE])
plt.xlabel('No. Topics')
plt.ylabel('MSE Error')
plt.show()

## R.Thomas' class solution:
step = 20
n = 20
error = np.zeros(n)

for i in range(n):
    U, s, V = randomized_svd(vectors, i * step)
    reconstructed = U @ np.diag(s) @ V
    error[i]...


I am working on the file "FACLA/L3-notes.ipynb".
The first task is:
Randomized SVD method:
Can you write Python code for it?

def randomized_svd(M, n_components, n_oversamples=10, n_iter=4):

    # number of random columns we're going to create is the number of
    # columns we want + number of oversamples (extra buffer)
    n_random = n_components + n_oversamples

    Q = randomized_range_finder(M, n_random, n_iter)

    # project M to the (k + p) dimensional space using basis vectors
    B = Q.T @ M

    # compute SVD on the thin matrix: (k + p) wide
    Uhat, s, V = linalg.svd(B, full_matrices=False)
    del B
    U = Q @ Uhat

    # return the number of components we want from U, s, V
    return U[:, :n_components], s[:n_components], V[:n_components, :]

%time u, s, v = randomized_svd(vectors, 5)

u.shape, s.shape, v.shape

show_topics(v)


I am working on the file "FACLA/L3-notes.ipynb".
The first task is:
So if A is approx equal to Q•Q.T•A .. but not equal.. then Q is not the identity, but is very close to it.
Oh, right. Q: m x r, not m x m...
If both the columns and rows of Q had been orthonormal, then it would have been the Identity, but only the columns (r) are orthonormal.
Q is a tall, skinny matrix.

AW gives range(A). AW has far more rows than columns ==> in practice these columns are approximately orthonormal (v.unlikely to get lin-dep cols when choosing random values).
QR decomposition is foundational to Numerical Linear Algebra.
Q consists of orthonormal columns, R is upper-triangular.
Calculating Truncated-SVD:
1. Compute approximation to range(A). We want Q with r orthonormal columns such that $$A\approx QQ^TA$$
2. Construct $B = Q^T A$, which is small ($r\times n$)
3. Compute the SVD of $B$ by standard methods (fast since $B$ is smaller than $A$): $B = S\, Σ V^T$
4. Since: $$A \approx QQ^TA = Q(S \, ΣV^T)$$ if we set $U = QS$, then we have a low rank approximation $A \approx UΣV^T$.
How to choose $r$?
If we wanted to get 5 cols from a matrix of 100 cols, (5 topics). As a rule of thumb, let's go for 15 instead. You don't want to explicitly pull exactly the amount you want due to the randomized component being present, so you add some buffer.
Since our projection is approximate, we make it a little bigger than we need.
Implementing Randomized SVD:
First we want a randomized range finder.
Can you write Python code for it?

import numpy as np
from sklearn.datasets import fetch_20newsgroups
from sklearn import decomposition
from scipy import linalg
import matplotlib.pyplot as plt

%matplotlib inline
np.set_printoptions(suppress=True)

categories = ['alt.atheism', 'talk.religion.misc', 'comp.graphics', 'sci.space']
remove = ('headers', 'footers', 'quotes')
newsgroups_train = fetch_20newsgroups(subset='train', categories=categories, remove=remove)
# newsgroups_test = fetch_20newsgroups(subset='test', categories=categories, remove=remove)

from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer

vectorizer = CountVectorizer(stop_words='english')
vectors = vectorizer.fit_transform(newsgroups_train.data).todense() # (documents, vocab)

vocab = np.array(vectorizer.get_feature_names())

num_top_words=8

def show_topics(a):
    top_words = lambda t: [vocab[i] for i in np.argsort(t)[:-num_top_words-1:-1]]
    topic_words = ([top_words(t) for t in a])
    return [' '.join(t) for t in topic_words]

# computes an orthonormal matrix whose range approximates the range of A
# power_iteration_normalizer can be safe_sparse_dot (fast but unstable), LU (imbetween), or QR (slow but most accurate)
def randomized_range_finder(A, size, n_iter=5):
    # randomly init our Mat to our size; size: num_cols
    Q = np.random.normal(size=(A.shape[1], size))

    # LU decomp (lower triang * upper triang mat)
    # improves accuracy & normalizes
    for i in range(n_iter):
        Q, _ = linalg.lu(A @ Q, permute_l=True)
        Q, _ = linalg.lu(A.T @ Q, permute_l=True)

    # QR decomp on A & Q
    Q, _ = linalg.qr(A @ Q, mode='economic')
    return Q


I am working on the file "FACLA/L3-pytorch-autograd-example.ipynb".
The first task is:
PyTorch Autograd Introduction
Example from official documentation
Can you write Python code for it?

x = Variable(torch.ones(2,2), requires_grad=True) # requires_grad: calculate gradients
print(x)

print(x.data)

print(x.grad)

y = x + 2
print(y)

z = y * y * 3
out = z.sum()
print(z, out)

out.backward() # backpropagation
print(x.grad)


I am working on the file "FACLA/SVD-NMF-review.ipynb".
The first task is:
The idea of T-SVD is that we want to compute an approximation to the range of $A$. The range of $A$ is the space covered by the column basis.
ie: Range(A) = {y: Ax = y}
that is: all $y$ you can achieve by multiplying $x$ with $A$.
Depending on your space, the bases are vectors that you can take linear combinations of to get any value in your space.
3. Details of Randomized SVD (Truncated)
Our goal is to have an algorithm to perform Truncated SVD using Randomized values from the dataset matrix. We want to use randomization to calculate the topics we're interested in, instead of calculating all of them.
Aha. So.. the way to do that, using randomization, is to have a special kind of randomization. Find a matrix $Q$ with some special properties that will allow us to pull a matrix that is a near match to our dataset matrix $A$ in the ways we want it to be. Ie: It'll have the same singular values, meaning the same importance-ordered topics.
Wow mathematics is really.. somethin.
That process:

Compute an approximation to the range of $A$. ie: we want $Q$ with $r$ orthonormal columns st:

$$A \approx QQ^TA$$

Construct $B = Q^TA,$, which is small $(r \times n)$

Compute the SVD of $B$ by standard methods (fast since $B$ is smaller than $A$), $B = SΣV^T$

Since: $$A \approx QQ^TA = Q(SΣV^T)$$ if we set $U = QS$, then we have a low-rank approximation of $A \approx UΣV^T$.

-- okay so.. confusion here. What is $S$ and $Σ$? Because I see them elsewhere taken to mean the same thing on this subject, but all of a sudden they seem to be totally different things.
-- oh, so apparently $S$ here is actually something different. $Σ$ is what's been interchangeably referred to in Hellenic/Latin letters throughout the notebook.
NOTE that $A: m \times n$ while $Q: m \times r$, so $Q$ is generally a tall, skinny matrix and therefore much smaller & easier to compute with than $A$.
Also, because $S$ & $Q$ are both orthonormal, setting $R = QS$ makes $R$ orthonormal as well.
How do we find Q (in step 1)?
General Idea: we find this...
Can you write Python code for it?

# workflow w NMF is something like this
V = np.random.randint(0, 20, size=(10,10))

m,n = V.shape
d = 5 # num_topics

clsf = decomposition.NMF(n_components=d, random_state=1)

W1 = clsf.fit_transform(V)
H1 = clsf.components_


I am working on the file "FACLA/SVD-NMF-review.ipynb".
The first task is:
Wait but that also gives True for $VV^T$. Hmmm.
2. Truncated SVD
Okay, so SVD is an exact decomposition of a matrix and allows us to pull out distinct topics from data (due to their orthonormality (orthogonality?)).
But doing so for a large data corpus is ... bad. Especially if most of the data's meaning / information relevant to us is captured by a small prominent subset. IE: prevalence of articles like a and the are likely poor indicators of any particular meaning in a piece of text since they're everywhere in English. Likewise for other types of data.
Hmm, so, if I understood correctly, the Σ/S/s/σ matrix is ordered by value max$\rightarrow$min.. but computing the SVD of a large dataset $A$ is exactly what we want to avoid using T-SVD. Okay so how?
$\rightarrow$Full SVD we're calculating the full dimension of topics -- but its handy to limit to the most important ones -- this is how SVD is used in compression.
Aha. This is where I was confused. Truncation is used with Randomization in R-SVD. The Truncated section was just introducing the concept. Got it.
So that's where, in R-SVD, we use a buffer in addition to the portion of the dataset we take for SVD.
And yay scikit-learn has R-SVD built in.
Can you write Python code for it?

from sklearn import decomposition

# ofc this is just dummy data to test it works
datavectors = np.random.randint(-1000,1000,size=(10,50))
U,S,V = decomposition.randomized_svd(datavectors, n_components=5)

U.shape, S.shape, V.shape


I am working on the file "FACLA/SVD-NMF-review.ipynb".
The first task is:
Wait so.. the rows of a matrix $A$ are orthogonal iff $AA^T$ is diagonal? Hmm. Math.StackEx Link
Can you write Python code for it?

np.isclose(np.eye(len(U)), U @ U.T)

np.isclose(np.eye(len(V)), V.T @ V)


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
1.1 Create Datasets
In addition to the provided data, we'll be using external datasets put together by participants in the Kaggle competition. You can download all of them here.
For completeness, the implementation used to put them together is included below.
Can you write Python code for it?

def concat_csvs(dirname):
    path = f'{PATH}{dirname}'
    filenames = glob.glob(f'{path}/*.csv')

    wrote_header = False
    with open(f'{path}.csv', 'w') as outputfle:
        for filename in filenames:
            name = filename.split('.')[0]
            with open(filename) as f:
                line = f.readline()
                if not wrote_header:
                    wrote_header = True
                    outputfile.write('file' + line)
                for line in f:
                    outputfile.write(name + ',' + line)
                outputfile.write('\n')

# concat_csvs('googletrend')
# concat_csvs('weather')


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
1.6.2 All
Can you write Python code for it?

m = md.get_learner(emb_szs, len(df.columns)-len(cat_vars),
                   0.04, 1, [1000,500], [0.001,0.01], y_range=y_range)
λr = 1e-3

m.fit(λr, 1, metrics=[exp_rmspe])

m.fit(λr, 3, metrics=[exp_rmspe])

m.fit(λr, 3, metrics=[exp_rmspe], cycle_len=1)


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
An even better option for picking a validation set is using the exact same length of time period as the test set uses - this is implemented here:
Can you write Python code for it?

val_idx = np.flatnonzero(
    (df.index<=datetime.datetime(2014,9,17)) & (df.index>=datetime.datetime(2014,8,1)))

val_idx=[0]


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
Feature Space:
* train: Training set provided by competition
* store: List of stores
* store_states: Mapping of store to the German state they're in
* List of German state names
* googletrend: Trend of certain google keywords over time, found by users to correlate well w/ given daya
* weather: Weather
* test: Testing set
Can you write Python code for it?

table_names = ['train', 'store', 'store_states', 'state_names',
              'googletrend', 'weather', 'test']


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
In Pandas you can add new columns to a DataFrame by simply definint it. We'll do this for googletrends by extracting dates and state names from the given data an dadding those columns.
We're also going to replace all instances of state name 'NI' to match the usage in the rest of the data: 'HB,NI'. This is a good opportunity to highlight Pandas indexing. We can use .loc[rows, cols] to select a list of rows and a list of columns from the DataFrame. In this case, we're selecting rows w/ statename 'NI' by using a boolean list googletrend.State=='NI' and selecting "State".
Can you write Python code for it?

googletrend['Date'] = googletrend.week.str.split(' - ', expand=True)[0]
googletrend['State']= googletrend.file.str.split('_', expand=True)[2]
googletrend.loc[googletrend.State=='NI', "State"] = 'HB,NI'


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
It's usually a good idea to back up large tables of extracted / wrangled features before you join them onto another one, that way you can go back to it easily if you need to make changes to it.
Can you write Python code for it?

df.to_feather(f'{PATH}df')

# df = pd.read_feather(f'{PATH}df', index_col=0)

df["Date"] = pd.to_datetime(df.Date)

df.columns

joined = join_df(joined, df, ['Store', 'Date'])

joined_test = join_df(joined_test, df, ['Store', 'Date'])


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
Join weather/state names:
Can you write Python code for it?

weather = join_df(weather, state_names, "file", "StateName")


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
NOTE: *you'll get a Buffer dtype mismatch ValueError here unless you have joined or joined_test loaded in from before up above. -- Note for when rerunning this notebook: switching between test and train datasets.
The authors also removed all instances where the store had zero sale / was closed. We speculate that this may have cost them a higher standing in the competition. One reason this may be the case is that a little exploratory data analysis reveals that there are often periods where astores are closed, typically for refurbishment. Before and after these periods, there are naturally spikes in sales that one might expect. By ommitting this data from their training, the authors gave up the ability to leverage information about these periods to predict this otherwise volatile behavior.
Can you write Python code for it?


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
NOTE: Init columns and define df as train, then run all cells below until joined = join_df(.), then Re-init columns and define df as test, then run all cells below, this time with joined_test = join_df(.).

This took me a very long time to learn -- Hakan
Can you write Python code for it?

columns = ["Date", "Store", "Promo", "StateHoliday", "SchoolHoliday"]

df = train[columns]

df = test[columns]


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
NOTE: make sure joined or joined_test is loaded into memory (either reinitialized from above, or loaded from disk if saved as below) when switching between train and test datasets.
Actually you'll probably have to make sure you have them both read in anyway, if you're going to run some of the lines further below in the same cells together.
1.3 Durations
It's common when working with time series data to extract data that explains relationships accross rows as opposed to columns, eg:
* Running averages
* Time until next event
* Time since last event
This is often difficult to do with most table manipulation frameworks, since they're designed to work with relationships across columns. As such, we've created a class to handle this type of data.
We'll define a function get_elapsed for cumulative counting across a sorted DataFrame. Given a particular field fld to monitor, this function will start tracking time since the last occurrence of that field. When the field is seen again, the counter is set to zero.
Upon initialization, this will result in datetime NA's until the field is encountered. This is reset every time a new store is seen. We'll see how to use this shortly.
Can you write Python code for it?

?? proc_df

def get_elapsed(fld, pre):
    day1 = np.timedelta64(1, 'D')
    last_date = np.datetime64()
    last_store = 0
    res = []

    for s,v,d in zip(df.Store.values, df[fld].values, df.Date.values):
        if s != last_store:
            last_date = np.datetime64()
            last_store = s
        if v: last_date = d
        res.append(((d-last_date).astype('timedelta64[D]') / day1).astype(int))
    df[pre+fld] = res


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
Next we want to drop the Store indices grouped together in the window function.
Often in Pandas, there's an option to do this in place. This is time and memory efficient when working with large datasets.
Can you write Python code for it?

bwd.drop('Store', 1, inplace=True)
bwd.reset_index(inplace=True)

fwd.drop('Store', 1, inplace=True)
fwd.reset_index(inplace=True)

df.reset_index(inplace=True)


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
Next we'll demonstrate window functions in Pandas to calculate rolling quantities.
Here we're sorting by date (sort_index()) and counting the number of events of interest (sum() defined in columns in the following week (rolling()), grouped by Store (groupby()). We do the same in the opposite direction.
Can you write Python code for it?

bwd = df[['Store']+columns].sort_index().groupby("Store").rolling(7, min_periods=1).sum()

fwd = df[['Store']+columns].sort_index(ascending=False
                                      ).groupby("Store").rolling(7, min_periods=1).sum()


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
Next we'll extract features "CompetitionOpenSince" and "CompetitionDaysOpen". Note the use of apply() in mapping a function across DataFrame values.
Can you write Python code for it?

for df in (joined, joined_test):
    df["CompetitionOpenSince"] = pd.to_datetime(dict(year=df.CompetitionOpenSinceYear,
                                                     month=df.CompetitionOpenSinceMonth, day=15))
    df["CompetitionDaysOpen"] = df.Date.subtract(df.CompetitionOpenSince).dt.days


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
Now that we've engineered all our features, we need to convert to input compatible with a neural network.
This includes converting categorical variables into contiguous integers or one-hot encodings, normalizing continuous features to standard normal, etc..
Can you write Python code for it?

cat_vars = ['Store', 'DayOfWeek', 'Year', 'Month', 'Day', 'StateHoliday', 'CompetitionMonthsOpen',
    'Promo2Weeks', 'StoreType', 'Assortment', 'PromoInterval', 'CompetitionOpenSinceYear', 'Promo2SinceYear',
    'State', 'Week', 'Events', 'Promo_fw', 'Promo_bw', 'StateHoliday_fw', 'StateHoliday_bw',
    'SchoolHoliday_fw', 'SchoolHoliday_bw']

contin_vars = ['CompetitionDistance', 'Max_TemperatureC', 'Mean_TemperatureC', 'Min_TemperatureC',
   'Max_Humidity', 'Mean_Humidity', 'Min_Humidity', 'Max_Wind_SpeedKm_h',
   'Mean_Wind_SpeedKm_h', 'CloudCover', 'trend', 'trend_DE',
   'AfterStateHoliday', 'BeforeStateHoliday', 'Promo', 'SchoolHoliday']

n = len(joined); n

dep = 'Sales'
joined = joined[cat_vars+contin_vars+[dep, 'Date']].copy()

joined_test[dep] = 0
joined_test = joined_test[cat_vars+contin_vars+[dep, 'Date', 'Id']].copy()

for v in cat_vars: joined[v] = joined[v].astype('category').cat.as_ordered()

apply_cats(joined_test, joined)

for v in contin_vars:
    joined[v] = joined[v].astype('float32')
    joined_test[v] = joined_test[v].astype('float32')


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
Now we can outer join all of our data into a single DataFrame. Recall that in outer joins everytime a value in the joining field on the left table doesn't have a corresponding value on the right table, the corresponding row in the new table has Null values for all right table fields.One way to check that all records are consistent and complete is to check for Null values post-join, as we do here.
Aside Why not just do an inner join? If you're assuming that all records are complete and match on the field you desire, an inner join will do the same thing as an outer join. However, in the event that you're wrong or a mistake is made, an outer join followed by a Null-Check will catch it (Comparing before/after # of rows for inner join is equivalent, but requires keeping track of before/after row #'s. Outer join is easier.).
Can you write Python code for it?

store = join_df(store, store_states, "Store")
len(store[store.State.isnull()])
# NOTE: if you do this multiple times, as per join_df(..)'s code,
# you'll add the last column onto store, with '_y' as its suffix.

joined = join_df(train, store, "Store")
joined_test = join_df(test, store, "Store")
len(joined[joined.StoreType.isnull()]), len(joined_test[joined_test.StoreType.isnull()])

joined = join_df(joined, googletrend, ["State","Year","Week"])
joined_test = join_df(joined_test, googletrend, ["State","Year","Week"])
len(joined[joined.trend.isnull()]), len(joined_test[joined_test.trend.isnull()])

joined = joined.merge(trend_de, 'left', ["Year", "Week"], suffixes=('', '_DE'))
joined_test = joined_test.merge(trend_de, 'left', ["Year", "Week"], suffixes=('', '_DE'))
len(joined[joined.trend_DE.isnull()]), len(joined_test[joined_test.trend_DE.isnull()])

joined = join_df(joined, weather, ["State","Date"])
joined_test = join_df(joined_test, weather, ["State","Date"])
len(joined[joined.Mean_TemperatureC.isnull()]), len(joined_test[joined_test.Mean_TemperatureC.isnull()])

# oh very cool -- I think this function removes extra/accidental column joins
for df in (joined, joined_test):
    for c in df.columns:
        if c.endswith('_y'):
            if c in df.columns: df.drop(c, inplace=True, axis=1)


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
RF
Random Forest
Can you write Python code for it?

from sklearn.ensemble import RandomForestRegressor

((val, trn), (y_val, y_trn)) = split_by_idx(val_idx, df.values, yλ)

m = RandomForestRegressor(n_estimators=40, max_features=0.99, min_samples_leaf=2,
                          n_jobs=-1, oob_score=True)
m.fit(trn, y_trn);

preds = m.predict(val)
m.score(trn, y_trn), m.score(val, y_val), m.oob_score_, exp_rmspe(preds, y_val)


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
Same process for Promo dates.
Can you write Python code for it?

for df in (joined, joined_test):
    df["Promo2Since"] = pd.to_datetime(df.apply(lambda x: Week(
        x.Promo2SinceYear, x.Promo2SinceWeek).monday(), axis=1).astype(pd.datetime))
    df["Promo2Days"] = df.Date.subtract(df["Promo2Since"]).dt.days

for df in (joined, joined_test):
    df.loc[df.Promo2Days<0, "Promo2Days"] = 0
    df.loc[df.Promo2SinceYear<1990, "Promo2Days"] = 0
    df["Promo2Weeks"] = df["Promo2Days"]//7
    df.loc[df.Promo2Weeks<0, "Promo2Weeks"] = 0
    df.loc[df.Promo2Weeks>25, "Promo2Weeks"] = 25
    df.Promo2Weeks.unique()

joined.to_feather(f'{PATH}joined')
joined_test.to_feather(f'{PATH}joined_test')


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
The Google Trends data has a special category for the whole of the US - we'll pull that out so we can use it explicitly. * -- DE?*
Can you write Python code for it?

trend_de = googletrend[googletrend.file == 'Rossmann_DE']


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
The following extracts particular date fields from a complete datetime for the purpose of constructing categoricals.
You should always consider this feature extraction step when working with date-time. Without expanding your date-time into these additional fields, you can't capture any trend/cyclical behavior as a function of time at any of these granularities. We'll add to every table with a date field.
Can you write Python code for it?

add_datepart(weather, "Date", drop=False)
add_datepart(googletrend, "Date", drop=False)
add_datepart(train, "Date", drop=False)
add_datepart(test, "Date", drop=False)


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
To run on the full dataset, use this instead:
Can you write Python code for it?

samp_size = n
joined_samp = joined.set_index("Date")


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
We add the "CompetitionMonthsOpen" field, limiting the maximum to 2 years to limit number of unique categories.
Can you write Python code for it?

for df in (joined, joined_test):
    df["CompetitionMonthsOpen"] = df["CompetitionDaysOpen"]//30
    df.loc[df.CompetitionMonthsOpen>24, "CompetitionMonthsOpen"] = 24

joined.CompetitionMonthsOpen.unique()


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
We can create a ModelData object directly from our DataFrame.
Can you write Python code for it?

md = ColumnarModelData.from_data_frame(PATH, val_idx, df, yλ.astype(np.float32), cat_flds=cat_vars, bs=128,
                                       test_df=df_test)


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
We can use head() to get a quick look at the contents of each table:
* train: Contains store information on a daily basis, tracks things like sales, customers, whether that day was a holiday, etc.
* store: General info about the store including competition, etc.
* store_states: Maps store to state it's in
* state_names: Maps state abbreviations to names
* googletrend: Trend data for particular week/state
* weather: Weather conditions for each state
* test: Same as training table, w/o sales and customers
Can you write Python code for it?


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
We now have our final set of engineered features.
While these steps were explicitly outlined in the paper, these are all fairly typical feature engineering steps for dealing with time series data and are practical in any similar setting.
1.4 Create Features
Can you write Python code for it?

joined = pd.read_feather(f'{PATH}joined')
joined_test = pd.read_feather(f'{PATH}joined_test')

joined.head().T.head(40)


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
We'll back this up as well
Can you write Python code for it?

joined.reset_index(inplace=True)
joined_test.reset_index(inplace=True)

joined.to_feather(f'{PATH}joined')
joined_test.to_feather(f'{PATH}joined_test')


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
We'll be applying this to a subset of columns:
Can you write Python code for it?

# # reading back in joined & joined_test when switching df betwen train/test:
# joined = pd.read_feather(f'{PATH}joined')
# joined_test = pd.read_feather(f'{PATH}joined_test')


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
We'll be using the popular data manipulation framework pandas. Among other things, Pandas allows you to manipulate tables/DataFrames in Python as one would in a database.
We're going to go ahead and load all our csv's as DataFrames into the list tables.
Can you write Python code for it?

tables = [pd.read_csv(f'{PATH}{fname}.csv', low_memory=False) for fname in table_names]

from IPython.display import HTML


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
We'll do this for two more fields:
Can you write Python code for it?

fld = 'StateHoliday'
df = df.sort_values(['Store', 'Date'])
get_elapsed(fld, 'After')
df = df.sort_values(['Store', 'Date'], ascending=[True, False])
get_elapsed(fld, 'Before')

fld = 'Promo'
df = df.sort_values(['Store', 'Date'])
get_elapsed(fld, 'After')
df = df.sort_values(['Store', 'Date'], ascending=[True, False])
get_elapsed(fld, 'Before')


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
We'll replace some erroneous / outlying data
Can you write Python code for it?

for df in (joined, joined_test):
    df.loc[df.CompetitionDaysOpen<0, "CompetitionDaysOpen"] = 0
    df.loc[df.CompetitionOpenSinceYear<1990, "CompetitionDaysOpen"] = 0


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
We're going to run on a sample:
Can you write Python code for it?

idxs = get_cv_idxs(n, val_pct=150000/n)
joined_samp = joined.iloc[idxs].set_index("Date")
samp_size = len(joined_samp); samp_size


I am working on the file "FADL1/L3CA2_rossmann-Copy1.ipynb".
The first task is:
join_df is a function for joining tables on specific fields. By default, we'll be doing a left outer join of right on the left argument using the given fields for each table.
Pandas does joins using the merge method. The suffixes argument doescribes the naming convention for duplicate fields. We've elected to leave the duplicate field names on the left entouched, and append a "_y" to those on the right.
Can you write Python code for it?

def join_df(left, right, left_on, right_on=None, suffix='_y'):
    if right_on is None: right_on = left_on
    return left.merge(right, how='left', left_on=left_on, right_on=right_on,
                      suffixes=("", suffix))


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
1.1 Create Datasets
In addition to the provided data, we'll be using external datasets put together by participants in the Kaggle competition. You can download all of them here.
For completeness, the implementation used to put them together is included below.
Can you write Python code for it?

def concat_csvs(dirname):
    path = f'{PATH}{dirname}'
    filenames = glob.glob(f'{path}/*.csv')

    wrote_header = False
    with open(f'{path}.csv', 'w') as outputfle:
        for filename in filenames:
            name = filename.split('.')[0]
            with open(filename) as f:
                line = f.readline()
                if not wrote_header:
                    wrote_header = True
                    outputfile.write('file' + line)
                for line in f:
                    outputfile.write(name + ',' + line)
                outputfile.write('\n')

# concat_csvs('googletrend')
# concat_csvs('weather')


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
1.2 Data Cleaning / Feature Engineering
As a structuered data problem, we necessarily have to go through all the cleaning and feature engineering, even though we're using a neural network.
Can you write Python code for it?

train, store, store_states, state_names, googletrend, weather, test = tables

len(train), len(test)


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
1.5 DL
We're ready to put together our models.
Root-Mean-Squared Percent Error is the metric Kaggle used for this competition.
Can you write Python code for it?

def inv_y(a): return np.exp(a)

def exp_rmspe(y_pred, targ):
    targ = inv_y(targ)
    pct_var = (targ - inv_y(y_pred))/targ
    return math.sqrt((pct_var**2).mean())

max_log_y = np.max(yλ)
y_range = (0, max_log_y)


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
1.5.1 Sample
Can you write Python code for it?

m = md.get_learner(emb_szs, len(df.columns)-len(cat_vars),
                   0.04, 1, [1000,500], [0.001,0.01], y_range=y_range)
λr = 1e-3


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
1.5.3 Test
Can you write Python code for it?

m = md.get_learner(emb_szs, len(df.columns)-len(cat_vars),
                   0.04, 1, [1000,500], [0.001,0.01], y_range=y_range)
λr = 1e-3

m.fit(λr, 3, metrics=[exp_rmspe])

m.fit(λr, 3, metrics=[exp_rmspe], cycle_len=1)

m.save('val0')

# m.load('val0')

x,y = m.predict_with_targs()

exp_rmspe(x,y)

pred_test = m.predict(True)

pred_test = np.exp(pred_test)

joined_test['Sales'] = pred_test

csv_fn = f'{PATH}tmp/sub.csv'

joined_test[['Id','Sales']].to_csv(csv_fn, index=False)

FileLink(csv_fn)


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
An even better option for picking a validation set is using the exact same length of time period as the test set uses - this is implemented here:
Can you write Python code for it?

val_idx = np.flatnonzero(
    (df.index<=datetime.datetime(2014,9,17)) & (df.index>=datetime.datetime(2014,8,1)))

val_idx=[0]


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
Feature Space:
* train: Training set provided by competition
* store: List of stores
* store_states: Mapping of store to the German state they're in
* List of German state names
* googletrend: Trend of certain google keywords over time, found by users to correlate well w/ given daya
* weather: Weather
* test: Testing set
Can you write Python code for it?

table_names = ['train', 'store', 'store_states', 'state_names',
              'googletrend', 'weather', 'test']


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
In Pandas you can add new columns to a DataFrame by simply definint it. We'll do this for googletrends by extracting dates and state names from the given data an dadding those columns.
We're also going to replace all instances of state name 'NI' to match the usage in the rest of the data: 'HB,NI'. This is a good opportunity to highlight Pandas indexing. We can use .loc[rows, cols] to select a list of rows and a list of columns from the DataFrame. In this case, we're selecting rows w/ statename 'NI' by using a boolean list googletrend.State=='NI' and selecting "State".
Can you write Python code for it?

googletrend['Date'] = googletrend.week.str.split(' - ', expand=True)[0]
googletrend['State']= googletrend.file.str.split('_', expand=True)[2]
googletrend.loc[googletrend.State=='NI', "State"] = 'HB,NI'


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
In time series data, cross-validation is not random. Instead, our holdout data is generally the most recent data, as it would be a in a real application. This issue is discussed in detail in this post on our website.
One approach is to take the last 25% of rows (sorted by date) as our validation set.
Can you write Python code for it?

train_ratio = 0.75
# train_ration = 0.9
train_size = int(samp_size + train_ratio); train_size
val_idx = list(range(train_size, len(df)))


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
Join weather/state names:
Can you write Python code for it?

weather = join_df(weather, state_names, "file", "StateName")


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
NOTE: *you'll get a Buffer dtype mismatch ValueError here unless you have joined or joined_test loaded in from before up above. -- Note for when rerunning this notebook: switching between test and train datasets.
The authors also removed all instances where the store had zero sale / was closed. We speculate that this may have cost them a higher standing in the competition. One reason this may be the case is that a little exploratory data analysis reveals that there are often periods where astores are closed, typically for refurbishment. Before and after these periods, there are naturally spikes in sales that one might expect. By ommitting this data from their training, the authors gave up the ability to leverage information about these periods to predict this otherwise volatile behavior.
Can you write Python code for it?


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
NOTE: Init columns and define df as train, then run all cells below until joined = join_df(.), then Re-init columns and define df as test, then run all cells below, this time with joined_test = join_df(.).

This took me a very long time to learn -- Hakan
Can you write Python code for it?

columns = ["Date", "Store", "Promo", "StateHoliday", "SchoolHoliday"]

df = train[columns]

df = test[columns]


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
NOTE: make sure joined or joined_test is loaded into memory (either reinitialized from above, or loaded from disk if saved as below) when switching between train and test datasets.
Actually you'll probably have to make sure you have them both read in anyway, if you're going to run some of the lines further below in the same cells together.
1.3 Durations
It's common when working with time series data to extract data that explains relationships accross rows as opposed to columns, eg:
* Running averages
* Time until next event
* Time since last event
This is often difficult to do with most table manipulation frameworks, since they're designed to work with relationships across columns. As such, we've created a class to handle this type of data.
We'll define a function get_elapsed for cumulative counting across a sorted DataFrame. Given a particular field fld to monitor, this function will start tracking time since the last occurrence of that field. When the field is seen again, the counter is set to zero.
Upon initialization, this will result in datetime NA's until the field is encountered. This is reset every time a new store is seen. We'll see how to use this shortly.
Can you write Python code for it?

?? proc_df

def get_elapsed(fld, pre):
    day1 = np.timedelta64(1, 'D')
    last_date = np.datetime64()
    last_store = 0
    res = []

    for s,v,d in zip(df.Store.values, df[fld].values, df.Date.values):
        if s != last_store:
            last_date = np.datetime64()
            last_store = s
        if v: last_date = d
        res.append(((d-last_date).astype('timedelta64[D]') / day1).astype(int))
    df[pre+fld] = res


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
Next we want to drop the Store indices grouped together in the window function.
Often in Pandas, there's an option to do this in place. This is time and memory efficient when working with large datasets.
Can you write Python code for it?

bwd.drop('Store', 1, inplace=True)
bwd.reset_index(inplace=True)

fwd.drop('Store', 1, inplace=True)
fwd.reset_index(inplace=True)

df.reset_index(inplace=True)


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
Next we'll extract features "CompetitionOpenSince" and "CompetitionDaysOpen". Note the use of apply() in mapping a function across DataFrame values.
Can you write Python code for it?

for df in (joined, joined_test):
    df["CompetitionOpenSince"] = pd.to_datetime(dict(year=df.CompetitionOpenSinceYear,
                                                     month=df.CompetitionOpenSinceMonth, day=15))
    df["CompetitionDaysOpen"] = df.Date.subtract(df.CompetitionOpenSince).dt.days


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
Now we can outer join all of our data into a single DataFrame. Recall that in outer joins everytime a value in the joining field on the left table doesn't have a corresponding value on the right table, the corresponding row in the new table has Null values for all right table fields.One way to check that all records are consistent and complete is to check for Null values post-join, as we do here.
Aside Why not just do an inner join? If you're assuming that all records are complete and match on the field you desire, an inner join will do the same thing as an outer join. However, in the event that you're wrong or a mistake is made, an outer join followed by a Null-Check will catch it (Comparing before/after # of rows for inner join is equivalent, but requires keeping track of before/after row #'s. Outer join is easier.).
Can you write Python code for it?

store = join_df(store, store_states, "Store")
len(store[store.State.isnull()])
# NOTE: if you do this multiple times, as per join_df(..)'s code,
# you'll add the last column onto store, with '_y' as its suffix.

joined = join_df(train, store, "Store")
joined_test = join_df(test, store, "Store")
len(joined[joined.StoreType.isnull()]), len(joined_test[joined_test.StoreType.isnull()])

joined = join_df(joined, googletrend, ["State","Year","Week"])
joined_test = join_df(joined_test, googletrend, ["State","Year","Week"])
len(joined[joined.trend.isnull()]), len(joined_test[joined_test.trend.isnull()])

joined = joined.merge(trend_de, 'left', ["Year", "Week"], suffixes=('', '_DE'))
joined_test = joined_test.merge(trend_de, 'left', ["Year", "Week"], suffixes=('', '_DE'))
len(joined[joined.trend_DE.isnull()]), len(joined_test[joined_test.trend_DE.isnull()])

joined = join_df(joined, weather, ["State","Date"])
joined_test = join_df(joined_test, weather, ["State","Date"])
len(joined[joined.Mean_TemperatureC.isnull()]), len(joined_test[joined_test.Mean_TemperatureC.isnull()])

# oh very cool -- I think this function removes extra/accidental column joins
for df in (joined, joined_test):
    for c in df.columns:
        if c.endswith('_y'):
            if c in df.columns: df.drop(c, inplace=True, axis=1)


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
Now we'll merge these values onto the df.
Can you write Python code for it?

df = df.merge(bwd, 'left', ['Date', 'Store'], suffixes=['', '_bw'])
df = df.merge(fwd, 'left', ['Date', 'Store'], suffixes=['', '_fw'])

df.drop(columns, 1, inplace=True)

df.head()


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
Same process for Promo dates.
Can you write Python code for it?

for df in (joined, joined_test):
    df["Promo2Since"] = pd.to_datetime(df.apply(lambda x: Week(
        x.Promo2SinceYear, x.Promo2SinceWeek).monday(), axis=1).astype(pd.datetime))
    df["Promo2Days"] = df.Date.subtract(df["Promo2Since"]).dt.days

for df in (joined, joined_test):
    df.loc[df.Promo2Days<0, "Promo2Days"] = 0
    df.loc[df.Promo2SinceYear<1990, "Promo2Days"] = 0
    df["Promo2Weeks"] = df["Promo2Days"]//7
    df.loc[df.Promo2Weeks<0, "Promo2Weeks"] = 0
    df.loc[df.Promo2Weeks>25, "Promo2Weeks"] = 25
    df.Promo2Weeks.unique()

joined.to_feather(f'{PATH}joined')
joined_test.to_feather(f'{PATH}joined_test')


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
Some categorical variables have a lot more levels than others. Store, in particular, has over a thousand!
Can you write Python code for it?

cat_sz = [(c, len(joined_samp[c].cat.categories)+1) for c in cat_vars]

cat_sz


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
This is vert representative of a typical industry dataset.
The following returns summarized aggregate information to each table accross each field.
Can you write Python code for it?

# NOTE .summary() isn't printing, it's returning. So display() is
# needed to print the multiple calls.
for t in tables: display(DataFrameSummary(t).summary())


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
We add the "CompetitionMonthsOpen" field, limiting the maximum to 2 years to limit number of unique categories.
Can you write Python code for it?

for df in (joined, joined_test):
    df["CompetitionMonthsOpen"] = df["CompetitionDaysOpen"]//30
    df.loc[df.CompetitionMonthsOpen>24, "CompetitionMonthsOpen"] = 24

joined.CompetitionMonthsOpen.unique()


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
We now have our final set of engineered features.
While these steps were explicitly outlined in the paper, these are all fairly typical feature engineering steps for dealing with time series data and are practical in any similar setting.
1.4 Create Features
Can you write Python code for it?

joined = pd.read_feather(f'{PATH}joined')
joined_test = pd.read_feather(f'{PATH}joined_test')

joined.head().T.head(40)


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
We'll back this up as well
Can you write Python code for it?

joined.reset_index(inplace=True)
joined_test.reset_index(inplace=True)

joined.to_feather(f'{PATH}joined')
joined_test.to_feather(f'{PATH}joined_test')


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
We'll be applying this to a subset of columns:
Can you write Python code for it?

# # reading back in joined & joined_test when switching df betwen train/test:
# joined = pd.read_feather(f'{PATH}joined')
# joined_test = pd.read_feather(f'{PATH}joined_test')


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
We'll do this for two more fields:
Can you write Python code for it?

fld = 'StateHoliday'
df = df.sort_values(['Store', 'Date'])
get_elapsed(fld, 'After')
df = df.sort_values(['Store', 'Date'], ascending=[True, False])
get_elapsed(fld, 'Before')

fld = 'Promo'
df = df.sort_values(['Store', 'Date'])
get_elapsed(fld, 'After')
df = df.sort_values(['Store', 'Date'], ascending=[True, False])
get_elapsed(fld, 'Before')


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
We'll replace some erroneous / outlying data
Can you write Python code for it?

for df in (joined, joined_test):
    df.loc[df.CompetitionDaysOpen<0, "CompetitionDaysOpen"] = 0
    df.loc[df.CompetitionOpenSinceYear<1990, "CompetitionDaysOpen"] = 0


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
You can pass in custom metrics to learner.fit() that say: print out a number at the end of every epoch by calling this function (in this case: [exp_rmspe] I think it's in an array in case multiple metrics, or consistency.).
The metric used here is Root-Mean-Squared-Percentage
Can you write Python code for it?

m.fit(λr, 3, metrics=[exp_rmspe])

m.fit(λr, 5, metrics=[exp_rmspe], cycle_len=1)

m.fit(λr, 2, metrics=[exp_rmspe], cycle_len=4)


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
You need to tell the FastAI modeldata.get_learner() method how big to make the embedding matrices (emb_szs), and also how many of the columns in the dataframe are continuous variables.
So here: total num. cols minus num categorical vars.
The FastAI/PyTorch Neural Net will then know how to properly handle the data.
The parameters supplied to .get_learner():
* number of continuous variables: n_cont = len(df.columns) - len(cat_vars)
* embedding matrix's dropout (here, emb_drop = 0.04)
* number of outputs of last Lin.Layer (out_sz = 1 for predicting a single number: sales)
* output size: here 1000 activations in the 1st Linear layer, 500 in the 2nd, (szs = [1000,500])
* dropout par for 1st & 2nd Lin.Layers (drops = [0.001, 0.01])
I think the y_range parameter is for ensuring proper Normalization, as DNNs like numbers clustered near zero, and above y_range is defined as a tuple of (0, max(log(y))) -- where max & log are NumPy functions.
Can you write Python code for it?

m = md.get_learner(emb_szs, len(df.columns)-len(cat_vars),
                   0.04, 1, [1000,500], [0.001,0.01], y_range=y_range)
λr = 1e-3

m.lr_find()

m.sched.plot(100)

# learning rate finder run on full dataset:
m.lr_find()
m.sched.plot(100)


I am working on the file "FADL1/L3CA2_rossmann.ipynb".
The first task is:
holy shit that go 30th of 3303 on the Kaggle Rossmann competition. That's top 0.9%. Score: 0.11076 (pvt)
1.6 RF
Random Forest
Can you write Python code for it?

from sklearn.ensemble import RandomForestRegressor

((val, trn), (y_val, y_trn)) = split_by_idx(val_idx, df.values, yλ)

m = RandomForestRegressor(n_estimators=40, max_features=0.99, min_samples_leaf=2,
                          n_jobs=-1, oob_score=True)
m.fit(trn, y_trn);

preds = m.predict(val)
m.score(trn, y_trn), m.score(val, y_val), m.oob_score_, exp_rmspe(preds, y_val)


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
1.2 Data Cleaning / Feature Engineering
As a structuered data problem, we necessarily have to go through all the cleaning and feature engineering, even though we're using a neural network.
Can you write Python code for it?

train, store, store_states, state_names, googletrend, weather, test = tables

len(train), len(test)


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
1.3 Durations
It's common when working with time series data to extract data that explains relationships accross rows as opposed to columns, eg:
* Running averages
* Time until next event
* Time since last event
This is often difficult to do with most table manipulation frameworks, since they're designed to work with relationships across columns. As such, we've created a class to handle this type of data.
We'll define a function get_elapsed for cumulative counting across a sorted DataFrame. Given a particular field fld to monitor, this function will start tracking time since the last occurrence of that field. When the field is seen again, the counter is set to zero.
Upon initialization, this will result in datetime NA's until the field is encountered. This is reset every time a new store is seen. We'll see how to use this shortly.
Can you write Python code for it?

def get_elapsed(fld, pre):
    day1 = np.timedelta64(1, 'D')
    last_date = np.datetime64()
    last_store = 0
    res = []

    for s,v,d in zip(df.Store.values, df[fld].values, df.Date.values):
        if s != last_store:
            last_date = np.datetime64()
            last_store = s
        if v: last_date = d
        res.append(((d - last_date).astype('timedelta64[D]') / day1).astype(int))
    df[pre + fld] = res


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
1.5 RF
Can you write Python code for it?

from sklearn.ensemble import RandomForestRegressor

((val,trn), (y_val,y_trn)) = split_by_idx(val_idx, df.values, yl)

m = RandomForestRegressor(n_estimators=40, max_features=0.99, min_samples_leaf=2,
                          n_jobs=-1, oob_score=True)
m.fit(trn, y_trn)

preds = m.predict(val)
m.score(trn, y_trn), m.score(val, y_val), m.oob_score_, exp_rmspe(preds, y_val)


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
1.6.1 Sample
Can you write Python code for it?

m = md.get_learner(emb_szs, len(df.columns) - len(cat_vars),
                   0.04, 1, [1000,500], [0.001, 0.01], y_range=y_range)
lr = 1e-3

m.fit(lr, 3, metrics=[exp_rmspe])

m.fit(lr, 5, metrics=[exp_rmspe], cycle_len=1)

m.fit(lr, 2, metrics=[exp_rmspe], cycle_len=4)


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
1.6.2 All
Can you write Python code for it?

m = md.get_learner(emb_szs, len(df.columns) - len(cat_vars),
                   0.04, 1, [1000,500], [0.001,0.01], y_range=y_range)
lr = 1e-3

m.fit(lr, 3, metrics=[exp_rmspe])

m.fit(lr, 3, metrics=[exp_rmspe], cycle_len=1)


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
An even better option for picking a validation set is using the exact same length of time period as the test set uses - this is implemented here:
Can you write Python code for it?

val_idx = np.flatnonzero(
    (df.index<=datetime.datetime(2014,9,17)) & (df.index>=datetime.datetime(2014,8,1)))

# val_idx=[0]


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
Feature Space:
* train: Training set provided by competition
* store: List of stores
* store_states: Mapping of store to the German state they're in
* List of German state names
* googletrend: Trend of certain google keywords over time, found by users to correlate well w/ given daya
* weather: Weather
* test: Testing set
Can you write Python code for it?

table_names = ['train', 'store', 'store_states', 'state_names',
               'googletrend', 'weather', 'test']


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
In Pandas you can add new columns to a DataFrame by simply definint it. We'll do this for googletrends by extracting dates and state names from the given data an dadding those columns.
We're also going to replace all instances of state name 'NI' to match the usage in the rest of the data: 'HB,NI'. This is a good opportunity to highlight Pandas indexing. We can use .loc[rows, cols] to select a list of rows and a list of columns from the DataFrame. In this case, we're selecting rows w/ statename 'NI' by using a boolean list googletrend.State=='NI' and selecting "State".
Can you write Python code for it?

googletrend['Date'] = googletrend.week.str.split(' - ', expand=True)[0]
googletrend['State'] = googletrend.file.str.split('_', expand=True)[2]
googletrend.loc[googletrend.State=='NI', 'State'] = 'HB,NI'


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
In time series data, cross-validation is not random. Instead, our holdout data is generally the most recent data, as it would be a in a real application. This issue is discussed in detail in this post on our website.
One approach is to take the last 25% of rows (sorted by date) as our validation set.
Can you write Python code for it?

train_ratio = 0.75
# train_ration = 0.90
train_size = int(samp_size * train_ratio); train_size
val_idx = list(range(train_size, len(df)))


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
It's usually a good idea to back up large tables of extracted / wrangled features before you join them onto another one, that way you can go back to it easily if you need to make changes to it.
Can you write Python code for it?

df.to_feather(f'{PATH}df')

df.to_feather(f'{PATH}df_test')

# df = pd.read_feather(f'{PATH}df', index_col=0)

df["Date"] = pd.to_datetime(df.Date)

df.columns


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
Let's walk through an exmaple:
Say we're looking at School Holiday. We'll first sort by Store, then Date, and then call add_elapsed('SchoolHoliday', 'After'): This'll apply to each row with School Holiday:

Applied to every row of the DataFrame in order of store and date
Will add to the DataFrame the days since seeing a School Holiday
If we sort in the other direction, this'll count the days until another holiday
Can you write Python code for it?

fld = 'SchoolHoliday'
df = df.sort_values(['Store', 'Date'])
get_elapsed(fld, 'After')
df = df.sort_values(['Store', 'Date'], ascending=[True, False])
get_elapsed(fld, 'Before')


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
NOTE: make sure joined or joined_test is loaded into memory (either reinitialized from above, or loaded from disk if saved as below) when switching between train and test datasets.
Actually you'll probably have to make sure you have them both read in anyway, if you're going to run some of the lines further below in the same cells together.
Can you write Python code for it?

joined.to_feather(f'{PATH}joined')
joined_test.to_feather(f'{PATH}joined_test')

joined = pd.read_feather(f'{PATH}joined')
joined_test = pd.read_feather(f'{PATH}joined_test')


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
NOTE: when running on the train-set, do train[columns], when running on the test-set, do test[columns] --- idk yet if you have to run all the cells afterwards, but I think you do.
Can you write Python code for it?


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
Next we'll demonstrate window functions in Pandas to calculate rolling quantities.
Here we're sorting by date (sort_index()) and counting the number of events of interest (sum() defined in columns in the following week (rolling()), grouped by Store (groupby()). We do the same in the opposite direction.
Can you write Python code for it?

bwd = df[['Store']+columns].sort_index().groupby("Store").rolling(7, min_periods=1).sum()

fwd = df[['Store']+columns].sort_index(ascending=False
                                      ).groupby("Store").rolling(7, min_periods=1).sum()


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
Next we'll fill in missing values to avoid complications with NA's. NA (not available) is how Pandas indicates missing values; many models have problems when missing values are present, so it's always important to think about how to deal with them. In these cases, we're picking an aribtirary signal value that doesn't otherwise appear in the data.
Can you write Python code for it?

for df in (joined, joined_test):
    df['CompetitionOpenSinceYear'] = df.CompetitionOpenSinceYear.fillna(1900).astype(np.int32)
    df['CompetitionOpenSinceMonth']= df.CompetitionOpenSinceMonth.fillna(1).astype(np.int32)
    df['Promo2SinceYear'] = df.Promo2SinceYear.fillna(1900).astype(np.int32)
    df['Promo2SinceWeek'] = df.Promo2SinceWeek.fillna(1).astype(np.int32)


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
Now that we've engineered all our features, we need to convert to input compatible with a neural network.
This includes converting categorical variables into contiguous integers or one-hot encodings, normalizing continuous features to standard normal, etc..
Can you write Python code for it?

cat_vars = ['Store', 'DayOfWeek', 'Year', 'Month', 'Day', 'StateHoliday', 'CompetitionMonthsOpen',
    'Promo2Weeks', 'StoreType', 'Assortment', 'PromoInterval', 'CompetitionOpenSinceYear', 'Promo2SinceYear',
    'State', 'Week', 'Events', 'Promo_fw', 'Promo_bw', 'StateHoliday_fw', 'StateHoliday_bw',
    'SchoolHoliday_fw', 'SchoolHoliday_bw']

contin_vars = ['CompetitionDistance', 'Max_TemperatureC', 'Mean_TemperatureC', 'Min_TemperatureC',
   'Max_Humidity', 'Mean_Humidity', 'Min_Humidity', 'Max_Wind_SpeedKm_h',
   'Mean_Wind_SpeedKm_h', 'CloudCover', 'trend', 'trend_DE',
   'AfterStateHoliday', 'BeforeStateHoliday', 'Promo', 'SchoolHoliday']

n = len(joined); n

dep = 'Sales'
joined_test[dep] = 0
joined = joined[cat_vars + contin_vars + [dep, 'Date']].copy()
joined_test = joined_test[cat_vars + contin_vars + [dep, 'Date', 'Id']].copy()

for v in cat_vars: joined[v] = joined[v].astype('category').cat.as_ordered()

apply_cats(joined_test, joined)

for v in contin_vars:
    joined[v] = joined[v].astype('float32')
    joined_test[v] = joined_test[v].astype('float32')


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
The Google Trends data has a special category for the whole of the US - we'll pull that out so we can use it explicitly. * -- DE?*
Can you write Python code for it?

trend_de = googletrend[googletrend.file == 'Rossmann_DE']


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
To run on the full dataset, use this instead:
Can you write Python code for it?

samp_size = n
joined_samp = joined.set_index("Date")


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
We add the "CompetitionMonthsOpen" field, limiting the maximum to 2 years to limit number of unique categories.
Can you write Python code for it?

for df in (joined, joined_test):
    df["CompetitionMonthsOpen"] = df["CompetitionDaysOpen"] // 30
    df.loc[df.CompetitionMonthsOpen > 24, "CompetitionMonthsOpen"] = 24
joined.CompetitionMonthsOpen.unique()


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
We can create a ModelData object directly from our DataFrame.
Can you write Python code for it?

md = ColumnarModelData.from_data_frame(PATH, val_idx, df, yl, cat_flds=cat_vars, bs=128)

# md = ColumnarModelData.from_data_frame(PATH, val_idx, df, yl, cat_flds=cat_vars, bs=128,
#                                        test_df=df_test)


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
We now have our final set of engineered features.
While these steps were explicitly outlined in the paper, these are all fairly typical feature engineering steps for dealing with time series data and are practical in any similar setting.
1.4 Create Features
Can you write Python code for it?

joined = pd.read_feather(f'{PATH}joined')
joined_test = pd.read_feather(f'{PATH}joined_test')

joined.head().T.head(40)

joined_test.columns

joined.columns


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
We'll back this up as well
Can you write Python code for it?

joined.reset_index(inplace=True)
joined_test.reset_index(inplace=True)
# so `` ValueError: cannot insert level_0, already exists `` just means Ive already done this once eh?

joined.to_feather(f'{PATH}joined')
joined_test.to_feather(f'{PATH}joined_test')


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
We'll be using the popular data manipulation framework pandas. Among other things, Pandas allows you to manipulate tables/DataFrames in Python as one would in a database.
We're going to go ahead and load all our csv's as DataFrames into the list tables.
Can you write Python code for it?

tables = [pd.read_csv(f'{PATH}{fname}.csv', low_memory=False) for fname in table_names]

from IPython.display import HTML


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
We're going to run on a sample:
Can you write Python code for it?

idxs = get_cv_idxs(n, val_pct = 150000 / n)
joined_samp = joined.iloc[idxs].set_index("Date")
samp_size = len(joined_samp); samp_size


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
We're going to set the active index to Date
Can you write Python code for it?


I am working on the file "FADL1/L3CA2_rossmann_old.ipynb".
The first task is:
join_df is a function for joining tables on specific fields. By default, we'll be doing a left outer join of right on the left argument using the given fields for each table.
Pandas does joins using the merge method. The suffixes argument doescribes the naming convention for duplicate fields. We've elected to leave the duplicate field names on the left entouched, and append a "_y" to those on the right.
Can you write Python code for it?

def join_df(left, right, left_on, right_on=None, suffix='_y'):
    if right_on is None: right_on = left_on
    return left.merge(right, how='left', left_on=left_on, right_on=right_on,
                      suffixes=("", suffix))


I am working on the file "FADL1/L3CA_lesson2-image-models-backup.ipynb".
The first task is:
In single-label classification each sample belongs to one class. In the previous example, each image is either a dog or a cat.
Can you write Python code for it?

list_paths = [f"{PATH}train-jpg/train_0.jpg", f"{PATH}train-jpg/train_1.jpg"]
titles = ["haze primary", "agriculture clear primary water"]
plots_from_files(list_paths, titles=titles, maintitle="Multi-label Classification")


I am working on the file "FADL1/L3CA_lesson2-image-models-backup.ipynb".
The first task is:
Softmax wouldn't be good because it wants to "pick a thing". Instead, we'll use the Sigmoid. In multi-label clsfn, each sample can belong to one or more classes. In the previous example, the 1st images belong to two classes: haze and primary. The 2nd belongs to four classes: agriculture, clear, primary, and water.
3 Multi-Label Models for Planet Dataset
fastai is baller AF bc it'll look at the labels in the CSV, and if there're > 1 labels ever, for any item, it'll automatically switch to multi-label mode.
Can you write Python code for it?

from planet import f2

metrics = [f2]
f_model = resnet34

label_csv = f'{PATH}train_v2.csv'
n = len(list(open(label_csv)))-1
val_idxs = get_cv_idxs(n)


I am working on the file "FADL1/L3CA_lesson2-image-models-backup.ipynb".
The first task is:
We use a different set of DAs for this dataset -- we also allow vertical flips, since we don't expect the vertical orientation of satellite images to change our classifications.
Can you write Python code for it?

def get_data(sz,bs=64):
    tfms = tfms_from_model(f_model, sz, aug_tfms=transforms_top_down, max_zoom=1.05)
    return ImageClassifierData.from_csv(PATH, 'train-jpg', label_csv, tfms=tfms, bs=bs,
                                        suffix='.jpg', val_idxs=val_idxs, test_name='test-jpg')

data = get_data(256) # the planet images are 256x256

# turning a dataloader into an iterator:
x,y = next(iter(data.val_dl)) # note: .._ds: dataset; .._dl: dataloader | PyTorch concepts
# idea: dataset gives you a single image or object back
#       dataloader gives you back a single (transformed) mini-batch (next mini-batch only)

y    # 64x17: batch size x number of classes

# zip, basically zips two lists together into an iterator
list(zip(data.classes, y[0])) # getting the 0th image's labels -- from validation set


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
A NOTE on the data.resize(int(sz*1.3), 'tmp') line above. The dataloader only resizes images when it outputs them to the model --> the images go into the dataloader at full size. The data.resize(.) method takes a maximum size (here int(64 * 1.3) = 83) and creates a resized copy of the entire dataset to a temporary folder. This technique is purely a speed-up measure, with no actual effect on model performace. ... But very handy in practice when you think about it... good to know..
http://forums.fast.ai/t/dog-breed-identification-challenge/7464/51
http://forums.fast.ai/t/deeplearning-lecnotes3/7866
Can you write Python code for it?

learn = ConvLearner.pretrained(f_model, data, metrics=metrics) # build the model

learn.lr_find()
learn.sched.plot()


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
Double image size again (finally up to original size) & repeat:
Can you write Python code for it?


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
Finally predict with Test Time Augmentation (this is on validation data)
Can you write Python code for it?


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
I made a mistake here. I didn't add the additional test set images. Will do that below. Not sure if there's a cleaner way to add more test data in fastai:
Can you write Python code for it?

# does tfms matter since I'm only adding another test-set ?
tfms = tfms_from_model(f_model, sz=256, aug_tfms=transforms_top_down, max_zoom=1.05)
data = ImageClassifierData.from_csv(PATH, 'train-jpg', label_csv, tfms=tfms, bs=24, suffix='.jpg',
                                    val_idxs=val_idxs, test_name='test-jpg-additional')
learn.set_data(data)

log_preds_adtl = learn.TTA(is_test=True)[0]
preds_adtl = np.exp(log_preds_adtl)

data.test_ds.fnames[0]

predicted_tags_adtl = [[' '.join(classes[np.where(pred >= threshold)])] for pred in preds_adtl]
df_adtl = pd.DataFrame(predicted_tags_adtl, columns = ['tags'])

df_adtl.insert(0, 'image_name', [f.split('/')[1].split('.')[0] for f in data.test_ds.fnames])

df_adtl.head()


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
Images are just matrices of numbers, so we can just multiply them by a number if they're too washed out / hazy:
Can you write Python code for it?

plt.imshow(data.val_ds.denorm(to_np(x))[0]*1.4);
# NOTE: same as: plt.imshow(data.val_ds.get_x(0) * 1.4)


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
Looking at test_preds I'm noticing a base value of 1.0 and predictions that go up from there to, I think, a max of around 2.0. Judging by that and tips in this fastai thread: I'll set a threshold of 1.02 and anything higher than that will be a positive match.
I don't know why my predictions are starting from 1 and not zero, if I did something wrong or if it's just like that.
Can you write Python code for it?

threshold = 1.02
classes = np.array(data.classes)
predicted_tags = [[' '.join(classes[np.where(pred >= threshold)])] for pred in test_preds]

submission = pd.DataFrame(predicted_tags)
submission.columns = ['tags']

submission.insert(0, 'image_name', [f.split('/')[1].split('.')[0] for f in data.test_ds.fnames])
submission.head()

# for single-class; won't work here
# submission = pd.DataFrame(test_preds)
# submission.columns = data.classes
# submission.insert(0, 'id', [f.split('/')[1].split('.')[0] for f in data.test_ds.fnames])

SUBM = PATH + 'subm/'
os.makedirs(SUBM, exist_ok=True)

submission.to_csv(f'{SUBM}FADL1-L3CA-submission-RN34-00.csv.gz', compression='gzip', index=False)

# FileLink(f'{SUBM}FADL1-L3CA-submission-RN34-00.csv.gz')


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
Now we increase (double here) the size of our images and train again:
Can you write Python code for it?

sz = 128

learn.set_data(get_data(sz))
learn.freeze()
learn.fit(lr, 3, cycle_len=1, cycle_mult=2)

learn.unfreeze()
learn.fit(lrs, 3, cycle_len=1, cycle_mult=2)
learn.save(f'{sz}')


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
Oh Shiiit. Those are Logarithmic Predictions?? Aha.. So.. I shouldn't've been exponentiating them in the first place: that explains why I had strange values. Right. So instead of needing a fancy function to iterate & test thresholds (which I can still do), I just need to make a submission of the log predictions. Got it.
Can you write Python code for it?

log_preds_adtl[0]

a = [i for i in range(5)]; b = [-i for i in range(5)];
c = np.concatenate((a,b)); c

def get_data_2(test_name=None):
    label_csv = f'{PATH}train_v2.csv'
    f_model = resnet34
    tfms = tfms_from_model(f_model, sz=256, aug_tfms=transforms_top_down, max_zoom=1.05)
    return ImageClassifierData.from_csv(PATH, 'train-jpg', label_csv, bs=64, tfms=tfms,
                                        suffix='.jpg', val_idxs=[0], test_name=test_name)

def generate_submission(threshold=0.02):
    SUBM = PATH + 'subm/'
    test_log_preds = pd.read_feather(f'{SUBM}test_log_preds.feather').as_matrix()
    test_log_preds_adtl = pd.read_feather(f'{SUBM}test_log_preds_adtl.feather').as_matrix()

    data = get_data_2(test_name='test-jpg')
    fnames = data.test_ds.fnames
    data = get_data_2(test_name='test-jpg-additional')
    fnames = np.concatenate((fnames, data.test_ds.fnames))

    classes = np.array(data.classes)

    preds = np.concatenate((test_log_preds, test_log_preds_adtl))
    names = [f.split('/')[1].split('.')[0] for f in fnames]

    predicted_tags = [[' '.join(classes[np.where(pred >= threshold)])] for pred in preds]

    submission = pd.DataFrame(predicted_tags, columns=['tags'])
    submission.insert(0, 'image_name', names)
    return submission

submission = generate_submission(threshold=0.02)
submission.head()

subm_name = "FADL1-L3CA-submission-RN34-01"
submission.to_csv(f'{SUBM}{subm_name}.csv.gz', compression='gzip', index=False)

FileLink(f'{SUBM}{subm_name}.csv.gz')

submission = generate_submission(threshold=0.03)
subm_name = "FADL1-L3CA-submission-RN34-04"
submission.to_csv(f'{SUBM}{subm_name}.csv.gz', compression='gzip', index=False)

FileLink(f'{SUBM}{subm_name}.csv.gz')

submission = generate_submission(threshold=0.035)
subm_name = "FADL1-L3CA-submission-RN34-05"
submission.to_csv(f'{SUBM}{subm_name}.csv.gz', compression='gzip', index=False)

FileLink(f'{SUBM}{subm_name}.csv.gz')


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
Softmax wouldn't be good because it wants to "pick a thing". Instead, we'll use the Sigmoid. In multi-label clsfn, each sample can belong to one or more classes. In the previous example, the 1st images belong to two classes: haze and primary. The 2nd belongs to four classes: agriculture, clear, primary, and water.
3 Multi-Label Models for Planet Dataset
fastai is baller AF bc it'll look at the labels in the CSV, and if there're > 1 labels ever, for any item, it'll automatically switch to multi-label mode.
Can you write Python code for it?

from planet import f2

metrics = [f2]
f_model = resnet34

label_csv = f'{PATH}train_v2.csv'
n = len(list(open(label_csv)))-1
val_idxs = get_cv_idxs(n, cv_idx=4) # using cv_idx=4 to keep same permutation as orig. notebook


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
Submissino FADL1-L3CA-submission-RN34-00 scored 0.88148 -- 616/938 Private.
Saving raw log predictions incase I try to tweak thresholds later:
Can you write Python code for it?

len(test_log_preds), len(log_preds_adtl)

pd.DataFrame(test_log_preds, columns=data.classes).to_feather(f'{SUBM}test_log_preds.feather')
pd.DataFrame(log_preds_adtl, columns=data.classes).to_feather(f'{SUBM}test_log_preds_adtl.feather')


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
This approach gets up to 50th place on the Kaggle Planet Amazon competition! A competition full of machine-learning / data-scientist wizards. Hail RoboLord. Hail. ..But seriously, better tools are awesome.
3.1 END
Can you write Python code for it?

# reloading everything in one cell after dead kernel:
%reload_ext autoreload
%autoreload 2
%matplotlib inline
from fastai.imports import *
from fastai.transforms import *
from fastai.conv_learner import *
from fastai.model import *
from fastai.dataset import *
from fastai.sgdr import *
PATH = "data/planet/"
from fastai.plots import *
from planet import f2

metrics = [f2]
f_model = resnet34

label_csv = f'{PATH}train_v2.csv'
# n = len(list(open(label_csv)))-1
# val_idxs = get_cv_idxs(n, cv_idx=4)
val_idxs = [0]

def get_data(sz,bs=64):
    tfms = tfms_from_model(f_model, sz, aug_tfms=transforms_top_down, max_zoom=1.05)
    return ImageClassifierData.from_csv(PATH, 'train-jpg', label_csv, tfms=tfms, bs=bs,
                                        suffix='.jpg', val_idxs=val_idxs, test_name='test-jpg')

data = get_data(sz=256)
learn = ConvLearner.pretrained(f_model, data, metrics=metrics)
learn.load(f'256')

learn.summary()

learn.data.sz

# It seems this is still necessary despite similar problems solved in this issue:
# https://github.com/fastai/fastai/issues/23
import resource
rlimit = resource.getrlimit(resource.RLIMIT_NOFILE)
resource.setrlimit(resource.RLIMIT_NOFILE, (4096, rlimit[1]))

test_log_preds = learn.TTA(is_test=True)
# test_preds = np.exp(test_log_preds)

# cut off the unneeded Zero labels
test_log_preds = test_log_preds[0]

test_preds = np.exp(test_log_preds)

data.classes

data.test_ds.fnames[:10]

x = 'test-jpg/test_26652.jpg'

print(x)
print(x.split('/'))
print(x.split('/')[1])
print(x.split('/')[1].split('.')[0])


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
This scores: 0.90181 -- 517/938 Private, when using a threshold of 0.035 and Log predictions.
Can you write Python code for it?

submission = generate_submission(threshold=0.04)
subm_name = "FADL1-L3CA-submission-RN34-06"
submission.to_csv(f'{SUBM}{subm_name}.csv.gz', compression='gzip', index=False)

FileLink(f'{SUBM}{subm_name}.csv.gz')

submission = generate_submission(threshold=0.045)
subm_name = "FADL1-L3CA-submission-RN34-07"
submission.to_csv(f'{SUBM}{subm_name}.csv.gz', compression='gzip', index=False)

FileLink(f'{SUBM}{subm_name}.csv.gz')


I am working on the file "FADL1/L3CA_lesson2-image-models.ipynb".
The first task is:
Tweaking thresholds and seeing if that changes things:
Can you write Python code for it?

temp = pd.read_feather(f'{SUBM}test_log_preds.feather')

temp.head()

temp = temp.as_matrix();
type(temp)

temp[0]


I am working on the file "FADL1/L7CA_lesson7-CAM.ipynb".
The first task is:
2. CAM
Can you write Python code for it?

class SaveFeatures():
    features=None
    def __init__(self, m): self.hook = m.register_forward_hook(self.hook_fn)
    def hook_fn(self, module, input, output): self.features = to_np(output)
    def remove(self): self.hook.remove()

x,y = next(iter(data.val_dl))
x,y = x[None, 1], y[None, 1]

vx = Variable(x.cuda(), requires_grad=True)

dx = data.val_ds.denorm(x)[0]
plt.imshow(dx);

sf = SaveFeatures(m[-4])
py = m(Variable(x.cuda()))
sf.remove()

py = np.exp(to_np(py)[0]); py

feat = np.maximum(0, sf.features[0])
feat.shape


I am working on the file "FADL1/L7CA_lesson7-CAM.ipynb".
The first task is:
The version of resnet that happens to be the best is the preact resnet. They have different internal orderings of conv,pool,res,bn,relu,etc.

We want to take our standard resnet model here and finetune it for dogs and cats. So we need to remove the last layer: (fc): Linear(in_features=512, out_features=1000)
Using ConvLearner.pretrained in FastAI deleted the last two layers:
(avgpool): AvgPool2d(kernel_size=7, stride=1, padding=0, ceil_mode=False, count_include_pad=True)
  (fc): Linear(in_features=512, out_features=1000)
The FastAI library is also the first/only library to replace the penultimate layer with a concatenated Adaptive + Max pooling layer.
For this exercise we'll do a simple version where we grab all the children of the model, delete the last two layers, and add a convolution which just has two outputs. THen do a average pooling, and a softmax.
Can you write Python code for it?

m = nn.Sequential(*children(m)[:-2],
                  nn.Conv2d(512, 2, 3, padding=1),
                  nn.AdaptiveAvgPool2d(1), Flatten(), # 2 layers here
                  nn.LogSoftmax())


I am working on the file "FADL1/L7CA_lesson7-CAM.ipynb".
The first task is:
This matrix is produced to create the heat map. It's just equal to the value of the feature matrix feat times the py vector. Where feat is the relu of the last conv layer's activations, and py is simply eual to the predictions.
Multiplying py with feat zeros-out all dog predictions in the 2nd channel of the 2x7x7 tensor, and retrieves the cat predictions in the 1st.
Put another way, in our model the only thing that happened after the last Conv layer was an Average Pooling layer. That layer took the 7x7 grid and averaged out how much each part is cat-like. So the final prediction was the average cattiness of the entire image. Since it had to average it all; I can just take that input matrix (that it would average), and instead resize it to the image size and overlay it on top.
Can you write Python code for it?

f2 = np.dot(np.rollaxis(feat, 0, 3), py)
f2 -= f2.min()
f2 /= f2.max()
f2


I am working on the file "FADL1/L7CA_lesson7-CAM.ipynb".
The first task is:
nn.Conv2d(512, 2, 3, padding=1) is the 4th from last layer, so we'll freeze up to it.
Can you write Python code for it?

learn.freeze_to(-4)

m[-1].trainable

m[-4].trainable

m[-4]

learn.fit(0.01, 1)

learn.fit(0.01, 1, cycle_len=1)


I am working on the file "FADL1/L7CA_lesson7-cifar10.ipynb".
The first task is:
5. BatchNorm
An issue up above, is that we're having trouble training the ConvNet as we add more layers. If we use larger learningRates, we get NaNs (Infinities), and smaller lrs take forever and doesn't have a chance to explore properly. So it isn't resilient.
To make the model more resilient, we'll use Batched Normalization. BatchNorm is a couple years old now (2018), and makes it much easier to train deep networks.
The network we're going to create will have more layers: 5 Conv layers and 1 Full layer. Back in the day that'd be considered a pretty deep network and would be hard to train. It's very simple now thanks to Batch Norm.
Batch Norm can be used by calling nn.BatchNorm.., we'll write it from scratch to learn about it.
Can you write Python code for it?

class BnLayer(nn.Module):
    def __init__(self, ni, nf, stride=2, kernel_size=3):
        super().__init__()
        self.conv = nn.Conv2d(ni, nf, kernel_size=kernel_size, stride=stride,
                              bias=False, padding=1)
        self.a = nn.Parameter(torch.zeros(nf,1,1))
        self.m = nn.Parameter(torch.ones(nf, 1,1))

    def forward(self, x):
        x = F.relu(self.conv(x))
        x_chan = x.transpose(0,1).contiguous().view(x.size(1), -1)
        if self.training: # true for training set. false for val set.
            self.means = x_chan.mean(1)[:, None, None]
            self.stds  = x_chan.std(1)[:, None, None]
        return (x-self.means) / self.stds * self.m + self.a


I am working on the file "FADL1/L7CA_lesson7-cifar10.ipynb".
The first task is:
And now this model is going to train beautifully just because of that one line: def forward(self, x): return x + super().forward(x). Why is that?
This is called a ResNet block. It says its prediction $$y = x + f(x)$$ in this case the function is a convolution. Which is also saying: $$f(x) = y - x$$, where f(x) is the current layer's prediction, y is the prediction from the previous layer. What it's doing is trying to fit a function f to the difference between y and x. That difference is the residual.
If y is what I'm trying to calculate, and x is the thing I've most recently calculated (input to current layer), then the difference between the two is essentially the error ito what I've calc'd so far. So this is saying  attempt to find a set of convolutional weights that attempts to fill in the amount I was off by.
Lecture at ~ 1:55:00
Can you write Python code for it?

learn = ConvLearner.from_model_data(Resnet([10,20,40,80,160], 10), data)

wd=1e-5

%time learn.fit(1e-2, 2, wds=wd)

%time learn.fit(1e-2, 3, cycle_len=1, cycle_mult=2, wds=wd)

%time learn.fit(1e-2, 8, cycle_len=4, wds=wd)


I am working on the file "FADL1/L7CA_lesson7-cifar10.ipynb".
The first task is:
From this notebook by K.Turgutlu.
Can you write Python code for it?

class SimpleNet(nn.Module):
    def __init__(self, layers):
        super().__init__()
        self.layers = nn.ModuleList([
            nn.Linear(layers[i], layers[i+1]) for i in range(len(layers) - 1)])

    def forward(self, x):
        x = x.view(x.size(0), -1)
        for λ in self.layers:
            λ_x = λ(x)
            x = F.relu(λ_x)
        return F.log_softmax(λ_x, dim=-1)

learn = ConvLearner.from_model_data(SimpleNet([32*32*3, 40, 10]), data)

learn, [o.numel() for o in learn.model.parameters()]

[o for o in learn.model.parameters()]

learn.summary()

learn.lr_find()
learn.sched.plot()

%time learn.fit(lr,2)

%time learn.fit(lr, 2, cycle_len=1)


I am working on the file "FADL1/L7CA_lesson7-cifar10.ipynb".
The first task is:
Look at dem der data
Can you write Python code for it?

data = get_data(32, 4)

x,y = next(iter(data.trn_dl))

plt.imshow(data.trn_ds.denorm(x)[0]);

plt.imshow(data.trn_ds.denorm(x)[3]);


I am working on the file "FADL1/L7CA_lesson7-cifar10.ipynb".
The first task is:
Notice making the model deeper hasn't helped. It's possible to train a standard ConvNet 12 layers deep, but it's hard to do properly. Instead we're going to replace the ConvNet with a ResNet.
7. ResNet
The ResnetLayer class is going to inherit from BnLayer and replace our forward with return x + super().forward(x). And that's it. Everything else is going to be identical -- except that we're now going to make the network 4x deeper.
Can you write Python code for it?

class ResnetLayer(BnLayer):
    def forward(self, x): return x + super().forward(x)

class Resnet(nn.Module):
    def __init__(self, layers, c):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 10, kernel_size=5, stride=1, padding=2)
        self.layers = nn.ModuleList([BnLayer(layers[i], layers[i+1])
            for i in range(len(layers) - 1)])
        self.layers2 = nn.ModuleList([ResnetLayer(layers[i+1], layers[i + 1], 1)
            for i in range(len(layers) - 1)])
        self.layers3 = nn.ModuleList([ResnetLayer(layers[i+1], layers[i + 1], 1)
            for i in range(len(layers) - 1)])
        self.out = nn.Linear(layers[-1], c)

    def forward(self, x):
        x = self.conv1(x)
        for λ,λ2,λ3 in zip(self.layers, self.layers2, self.layers3):
            x = λ3(λ2(λ(x))) # function of a function of a function
        x = F.adaptive_max_pool2d(x, 1)
        x = x.view(x.size(0), -1)
        return F.log_softmax(self.out(x), dim=-1)


I am working on the file "FADL1/L7CA_lesson7-cifar10.ipynb".
The first task is:
The goal is to basically replicated the basic architecture of a ResNet. The simple model above gets an accuracy of around 47%, with 120,000 parameters. Not great. We're deffinitely not using our parameters very well -- they're treating each pixel with a different weight.
Instead we'll find groups of 3x3 pixels with particular patterns, using a ConvNet.
3. CNN
The first step is to replace our FullNet model with a ConvNet model.
Can you write Python code for it?

class ConvNet(nn.Module):
    def __init__(self, layers, c):
        super().__init__()
        self.layers = nn.ModuleList([
            nn.Conv2d(layers[i], layers[i + 1], kernel_size=3, stride=2)
            for i in range(len(layers) - 1)])
        self.pool = nn.AdaptiveMaxPool2d(1)
        self.out = nn.Linear(layers[-1], c)

    def forward(self, x):
        for λ in self.layers: x = F.relu(λ(x))
        x = self.pool(x)
        x = x.view(x.size(0), -1)
        return F.log_softmax(self.out(x), dim=-1)


I am working on the file "FADL1/L7CA_lesson7-cifar10.ipynb".
The first task is:
The idea is, if we have some inputs coming in and a function trying to predict how much the error is, then add on another prediction of error at that new stage, and on and on, then each time we're zooming in closer and closer to the correct answer -- ie: we've gotten to a certain point but there's still an error, a residual, so let's create a model that predicts that residual and add that onto our previous model, and another model that predicts that residual, and adds that on, and etc. If you keep doing that over and over, you should get closer and closer to our answer.
This is based on the theory of Boosting. By specifying return x + super().forward(x) as the thing we're trying to calculate, then we're kind of getting boosting for free.
Note that here, only one convolution is done in the ResNet block. Actual standard ResNet blocks use two convolutions before adding back onto the input.
Note also that the first layer in every block is a standard Conv layer w/ a stride of two, not a Res layer. This is a bottleneck layer. From time to time we change the geometry in a ResNet model. Actual ResNets don't use a standard Conv layer for bottlenecks; but that'll be covered in Part 2 of this course.
Still, this simplified ResNet gets up to 82% accuracy (incl. overfitting).
8. ResNet2
We can make the Resnet even bigger
Can you write Python code for it?

class Resnet2(nn.Module):
    def __init__(self, layers, c, p=0.5):
        super().__init__()
        self.conv1 = BnLayer(3, 16, stride=1, kernel_size=7)
        self.layers = nn.ModuleList([BnLayer(layers[i], layers[i+1])
            for i in range(len(layers) - 1)])
        self.layers2 = nn.ModuleList([ResnetLayer(layers[i+1], layers[i + 1], 1)
            for i in range(len(layers) - 1)])
        self.layers3 = nn.ModuleList([ResnetLayer(layers[i+1], layers[i + 1], 1)
            for i in range(len(layers) - 1)])
        self.out = nn.Linear(layers[-1], c)
        self.drop = nn.Dropout(p) # dropout added

    def forward(self, x):
        x = self.conv1(x)
        for λ,λ2,λ3 in zip(self.layers, self.layers2, self.layers3):
            x = λ3(λ2(λ(x)))
        x = F.adaptive_max_pool2d(x, 1)
        x = x.view(x.size(0), -1)
        x = self.drop(x)
        return F.log_softmax(self.out(x), dim=-1)


I am working on the file "FADL1/L7CA_lesson7-cifar10.ipynb".
The first task is:
We have around 30,000 parameters in the ConvNet, about a quarter that in the simple FullNet, and our accuracy is around 57%, up from 47%.
4. Refactored
We're going to refactor the ConvNet slightly so that we put less stuff in the forward pass. For instance, calling relu each loop isn't ideal. So we'll create a new class called ConvLayer which contains a convolution with a kernel size of 3 and a stride of two, and with padding. Padding becomes especially important when you're down to small layer sizes in later convolutions where throwing away a potential convolution around the edge will lose a significant amount of information.
The relu will be inside the ConvLayer class, making it easier to edit and prevent bugs.
Can you write Python code for it?

class ConvLayer(nn.Module):
    def __init__(self, ni, nf):
        super().__init__()
        self.conv = nn.Conv2d(ni, nf, kernel_size=3, stride=2, padding=1)

    def forward(self, x): return F.relu(self.conv(x))

class ConvNet2(nn.Module):
    def __init__(self, layers, c):
        super().__init__()
        self.layers = nn.ModuleList([ConvLayer(layers[i], layers[i + 1])
                                     for i in range(len(layers) - 1)])
        self.out = nn.Linear(layers[-1], c)

    def forward(self, x):
        for λ in self.layers: x = λ(x)
        x = F.adaptive_max_pool2d(x, 1) # F is nn.Functional
        x = x.view(x.size(0), -1)
        return F.log_softmax(self.out(x), dim=-1)


I am working on the file "FADL1/L7CA_lesson7-cifar10.ipynb".
The first task is:
What's awesome about PyTorch is that a Layer definition and a Neural Network definition are literally identical. They both have a Constructor, and a Forward. Any time you have a layer, you can use it as a neural net, and vice versa.
Also, since AMP has no state (no weights), we don't need to have it as an attribute as in the ConvNet class up above, we can instead call it as a function in the Forward method of ConvNet2.
Can you write Python code for it?

learn = ConvLearner.from_model_data(ConvNet2([3, 20, 40, 80], 10), data)

learn.summary()

%time learn.fit(1e-1, 2)

%time learn.fit(1e-1, 2, cycle_len=1)


I am working on the file "FADL1/L7CA_lesson7-cifar10.ipynb".
The first task is:
learn = ConvLearner.from_model_data(ConvNet([3, 20, 40, 80], 10), data): 3 channels coming in; 1st layer comes out with 20, 2nd with 40, 3rd: 80.
Can you write Python code for it?


I am working on the file "FADL1/L7CA_lesson7-cifar10.ipynb".
The first task is:
nn.Conv2d(layers[i], layers[i + 1], kernel_size=3, stride=2): the first two pars are exactly the same as nn.Linear: num_features_in, num_features_out
Can you write Python code for it?

learn = ConvLearner.from_model_data(ConvNet([3, 20, 40, 80], 10), data)