Muennighoff/xP3x-sample huggingface dataset:
inputs: # On définit une fonction de création de table
def CreateTable(nom_bdd):
    QueryCurs.execute('''CREATE TABLE IF NOT EXISTS ''' + nom_bdd + '''
    (id INTEGER PRIMARY KEY, Name TEXT,City TEXT, Country TEXT, Price REAL)''')

# On définit une fonction qui permet d'ajouter des observations dans la table    
def AddEntry(nom_bdd, Nom,Ville,Pays,Prix):
    QueryCurs.execute('''INSERT INTO ''' + nom_bdd + ''' 
    (Name,City,Country,Price) VALUES (?,?,?,?)''',(Nom,Ville,Pays,Prix))
    
def AddEntries(nom_bdd, data):
    """ data : list with (Name,City,Country,Price) tuples to insert
    """
    QueryCurs.executemany('''INSERT INTO ''' + nom_bdd + ''' 
    (Name,City,Country,Price) VALUES (?,?,?,?)''',data)
    
    
### On va créer la table clients

CreateTable('Clients')

AddEntry('Clients','Toto','Munich','Germany',5.2)
AddEntries('Clients',
    [('Bill','Berlin','Germany',2.3),
    ('Tom','Paris','France',7.8),
    ('Marvin','Miami','USA',15.2),
    ('Anna','Paris','USA',7.8)])

# on va "commit" c'est à dire qu'on va valider la transaction. 
# > on va envoyer ses modifications locales vers le référentiel central - la base de données SQL

CreateDataBase.commit()
Given the above code, generate some markdown instructions for it.

targets: 
La méthode cursor() est un peu particulière : 
Il s'agit d'une sorte de tampon mémoire intermédiaire, destiné à mémoriser temporairement les données en cours de traitement, ainsi que les opérations que vous effectuez sur elles, avant leur transfert définitif dans la base de données. Tant que la  méthode .commit() n'aura pas été appelée, aucun ordre ne sera appliqué à la base de données.

A présent que nous sommes connectés à la base de données, on va créer une table qui contient plusieurs variables de format différents
- ID sera la clé primaire de la base
- Nom, Rue, Ville, Pays seront du text
- Prix sera un réel
inputs: Compute the test statistic of the real data to verify that it works.

targets: T_true = T(y)
print("The test statistic of the real data is T =", T_true)
inputs: I am working on the file "2_Mathematical_Groundwork/2_8_the_discrete_fourier_transform.ipynb".
The first task is:
2.8. The Discrete Fourier Transform (DFT) and the Fast Fourier Transform (FFT)<a id='math:sec:the_discrete_fourier_transform_and_the_fast_fourier_transform'></a>
The continuous version of the Fourier transform can only be computed when the integrals involved can be evaluated analytically, something which is not always possible in real life applications. This is true for a number of reasons, the most relevant of which are:


We don't always have the parametrisation of the signal that we want to find the Fourier transform of.


Signals are measured and recorded at a finite number of points.


Measured signals are contaminated by noise.


In such cases the discrete equivalent of the Fourier transform, called the discrete Fourier transform (DFT), is very useful. In fact, where the scale of the problem necessitates using a computer to perform calculations, the Fourier transform can only be implemented as the discrete equivalent. There are some subtleties we should be aware of when implementing the DFT. These mainly arise because it is very difficult to capture the full information present in a continuous signal with a finite number of samples. In this chapter we review the DFT and extend some of the most useful identities derived in the previous sections to the case where we only have acces to a finite number of samples. The subtleties that arise due to limited sampling will be discussed in the next section.
2.8.1 The discrete time Fourier transform (DTFT): definition<a id='math:sec:the_discrete_time_fourier_transform_definition'></a>
We start by introducing the discrete time Fourier transform (DTFT). The DTFT of a set $\left{y_n \in \mathbb{C}\right}_{n ~ \in ~ \mathbb{Z}}$ results in a Fourier series (see $\S$ 2.3 &#10142;) of the form
<a id='math:eq:8_001'></a><!--\label{math:eq:8_001}-->$$
Y_{2\pi}(\omega) = \sum_{n\,=\,-\infty}^{\infty} y_n\,e^{-\imath \omega n} \quad \mbox{where} \quad n \in \mathbb{Z}.
$$
The resulting function is a periodic function of the frequency variable $\omega$. In the above...
Can you write Python code for it?

targets: 
def loop_DFT(x):
    """
    Implementing the DFT in a double loop
    Input: x = the vector we want to find the DFT of
    """
    #Get the length of the vector (will only work for 1D arrays)
    N = x.size
    #Create vector to store result in
    X = np.zeros(N,dtype=complex)
    for k in range(N):
        for n in range(N):
            X[k] += np.exp(-1j*2.0*np.pi*k*n/N)*x[n]
    return X
inputs: I am working on the file "Code/EIA Bulk Download - extract facility generation.ipynb".
The first task is:
Load emission factors
These are mostly EIA emission factors
Can you write Python code for it?

targets: 
path = os.path.join('Clean data', 'Final emission factors.csv')
ef = pd.read_csv(path, index_col=0)
inputs: # append 

staff.append('Finance')
print(staff)

# pop away the information about his salary

andys_salary = staff.pop(2)
print(andys_salary)
print(staff)

# oops, made a mistake, I want to reinsert information about his salary

staff.insert(3, andys_salary)
print(staff)

contacts = [99993535, "andy@company.com"]

staff = staff+contacts # reassignment of the concatenated list back to staff
print(staff)
Given the above code, generate some markdown instructions for it.

targets: 
4.3 List methods
Right now, let us look at four very useful list methods. Methods are basically operations which modify lists. These are: 

pop which allows us to remove an item in a list. 

So for example if $x_0, x_1, \ldots, x_n$ are items in a list, calling my_list.pop(r) will modify the list so that it contains only $$x_0, \ldots, x_{r-1}, x_{r+1},\ldots, x_n$$ while returning the element $x_r$. 

append which adds items to the end of the list. 

Let's say $x_{n+1}$ is the new object you wish to append to the end of the list. Calling the method my_list.append(x_n+1) will modify the list inplace so that the list will now contain $$x_0, \ldots, x_n, x_{n+1}$$ Note that append does not return any output! 

insert which as the name suggests, allows us to add items to a list in a particular index location

When using this, type my_list.insert(r, x_{n+1}) with  the second argument to the method the object you wish to insert and r the position (still 0 indexed) where this object ought to go in that list. This method modifies the list inplace and does not return any output. After calling the insert method, the list now contains $$x_0,\ldots, x_{r-1}, x_{n+1}, x_{r}, \ldots, x_n$$ This means that my_list[r] = $x_{n+1}$ while my_list[r+1] = $x_{r}$

+ is used to concatenate two lists. If you have two lists and want to join them together producing a union of two (or more lists), use this binary operator. 

This works by returning a union of two lists. So $$[ x_1,\ldots, x_n] + [y_1,\ldots, y_m]$$ is the list containing $$ x_1,\ldots, x_n,y_1, \ldots, y_m$$ This change is not permanent unless you assign the result of the operation to another variable.
inputs: # Solution

psi3=sqrt(0.8)*tensor(H,H) + sqrt(0.2)*tensor(V,V)

out = []
for r in rads:
    out.append(expect(tensor(P(-r),P(r)),psi3))

plt.plot(angles,out,".") # plot in units of pi

# Solution
out = []
for r in rads:
    out.append(expect(tensor(P(r),P(deg2rad(55))),psi3) / expect(tensor(P(r),qeye(2)),psi3))

plt.plot(angles,out,".")
Given the above code, generate some markdown instructions for it.

targets: 
Challenge:
If we change the state to $\big|\psi\big\rangle = \sqrt{0.8} \big|H,H\big\rangle + \sqrt{0.2} \big|V,V\big\rangle$, the two angles that work for this state.
inputs: I am working on the file "Day_2/solutions/17-Pandas-Intro-Solution.ipynb".
The first task is:
As well as just a simple line plot:
Can you write Python code for it?

targets: 
air_quality.O3.plot(grid=True, figsize=(12, 2))
inputs: I am working on the file "2. Recommender system.ipynb".
The first task is:
Crowd Funding Recommender system
Idea

For new crowd funding projects, there are no such classificable factors with old projects (I suppose category is not a key factor)
Users are only interested in a project itself (Funding target money, duration, etc are non of their business)
Doc2vec is a quite logical method to classify documents
Classifying descriptions of projects by using Doc2vec -> Recommending 10 nearest projects with a project users clicked or supported

Outline

Tokenizing description
Run Doc2vec
Calculate distance of each project combinations
Recommend
Can you write Python code for it?

targets: 
cf_df = pd.read_excel('cf_description.xlsx')
inputs: I am working on the file "02-18-17_application_real_data.ipynb".
The first task is:
As you can notice, the punctuation doesn't the word count because the symbols are attached to the previous word. However, the em dash '—' symbol links two words and that will affect the count, so let's replace them with a space so when we split by spaces it will do what we want.
If you don't have the em dash symbol in your keyword you can generate it by doing (Ctrl+Shift+u)+2014.
Can you write Python code for it?

targets: 
#We want to keep first_paragraph without changes, so we create a copy and work with that.
revised_paragraph = first_paragraph 

for element in revised_paragraph:
    if element == '—':
        revised_paragraph = revised_paragraph.replace(element, ' ')

print(revised_paragraph)
inputs: As we can see there is 2.3 times as many sample for the control group. And because of this, we can be picky and select only a part of the samples in the control group that correspond to the samples in the treatment group. To do so, we will match two samples together from each groups corresponding to their propensity score and then only keep and compare the samples matched.
1.3) A propensity score model
Let's calculate the propensity score

targets: from sklearn.linear_model import LogisticRegression

lr = LogisticRegression()
#Select features, that is drop id and treat columns
selectedFeatures = lalonde_data.drop(['id','treat'], axis=1)
#Fit the model
lr.fit(selectedFeatures, lalonde_data['treat']);

#Calculate the propensity scores
propensity_scores = lr.predict_proba(selectedFeatures)

#Only keep the probability of receiving the treatment and store it inside the dataframe
lalonde_data['propensity score'] = [x[1] for x in propensity_scores]
inputs: # separate the calls data for plotting

churnDFs = churnDFs[['Account Length','Day Calls','Eve Calls','CustServ Calls','Churn']]

# Create scatter plot matrix of call data
splom = ff.create_scatterplotmatrix(churnDFs, diag='histogram', index='Churn',  
                                  colormap= dict(
                                      Churn = '#9CBEF1',
                                      Retain = '#04367F'
                                      ),
                                  colormap_type='cat',
                                  height=560, width=650,
                                  size=4, marker=dict(symbol='circle'))
py.iplot(splom)

%%capture

#h2o.connect(ip="35.225.239.147")
h2o.init(nthreads=1, max_mem_size="768m")

%%capture

# Split data into training and testing frames

from sklearn import cross_validation
from sklearn.model_selection import train_test_split

training, testing = train_test_split(churnDF, train_size=0.8, stratify=churnDF["Churn"], random_state=9)
train = h2o.H2OFrame(python_obj=training).drop("State")
test = h2o.H2OFrame(python_obj=testing).drop("State")

# Set predictor and response variables
y = "Churn"
x = train.columns
x.remove(y)
Given the above code, generate some markdown instructions for it.

targets: 
Scatterplot Matrix
inputs: # Use this token pattern to keep single-letter words
vectorizer = CountVectorizer(token_pattern=r'\b\w+\b')
# First, learn vocabulary from the training data and assign columns to words
# Then convert the training data into a sparse matrix
train_matrix = vectorizer.fit_transform(train_data['review_clean'])
# Second, convert the test data into a sparse matrix, using the same word-column mapping
test_matrix = vectorizer.transform(test_data['review_clean'])
Given the above code, generate some markdown instructions for it.

targets: 
Build the word count vector for each review
We will now compute the word count for each word that appears in the reviews. A vector consisting of word counts is often referred to as bag-of-word features. Since most words occur in only a few reviews, word count vectors are sparse. For this reason, scikit-learn and many other tools use sparse matrices to store a collection of word count vectors. Refer to appropriate manuals to produce sparse word count vectors. General steps for extracting word count vectors are as follows:

Learn a vocabulary (set of all words) from the training data. Only the words that show up in the training data will be considered for feature extraction.
Compute the occurrences of the words in each review and collect them into a row vector.
Build a sparse matrix where each row is the word count vector for the corresponding review. Call this matrix train_matrix.
Using the same mapping between words and columns, convert the test data into a sparse matrix test_matrix.

The following cell uses CountVectorizer in scikit-learn. Notice the token_pattern argument in the constructor.
inputs: And you can use vector functions to get the magnitude and angle.

targets: vector_mag(V2), vector_angle(V2)
inputs: %%bash
# after fixing `.at` redundancy - calling it in each apply call
# git checkout 4c978aff110001efdc917ed60cb611139e1b54c9 -b remove-getitem-redundancy
# time gypsy simulate ../private-data/prepped_random_sample_300.csv --output-dir tmp
# rm -rfd tmp

# real	5m36.407s
# user	5m25.740s
# sys	0m2.140s
Given the above code, generate some markdown instructions for it.

targets: 
Hm, this actually got worse, although it is a small sample......... if anything i suspect its because we're calling row.at[] instead of assigning the variable outside the loop. It's ok as the code has less reptition, it's a good tradeoff.
inputs: I am working on the file "04-Project-Fletcher/Phases/Phase_3/Phase_3_Notebook.ipynb".
The first task is:
Step 1: Obtain my tweets!
I will obtain my entire tweet history! Note: For 2nd degree potential followers, I only extract 200 of their most recent tweets!
Can you write Python code for it?

targets: 
gabr_tweets = extract_users_tweets("gabr_ibrahim", 2000)
inputs: # PROMPT 19 TODO Write Test
Given the above code, generate some markdown instructions for it.

targets: 
Step 3c: printBeers() Test
Use the beers and calories from PROMPT6
inputs: <h3>2. <a href="http://www.shogun-toolbox.org/doc/en/latest/classshogun_1_1LinearSolver.html">Linear Solvers</a></h3>
<p> Conjugate Gradient based iterative solvers, that construct the Krylov subspace in their iteration by computing matrix-vector products are most useful for solving sparse linear systems. Here is an overview of CG based solvers that are currently available in Shogun.</p>
<h4> <a href="http://www.shogun-toolbox.org/doc/en/latest/classshogun_1_1CConjugateGradientSolver.html">Conjugate Gradient Solver</a></h4>
This solver solves for system $\mathbf{Qx}=\mathbf{y}$, where $\mathbf{Q}$ is real-valued spd linear operator (e.g. dense/sparse matrix operator), and $\mathbf{y}$ is real vector.

targets: from scipy.sparse import csc_matrix
from scipy.sparse import identity

# creating a random spd matrix
dim = 5
np.random.seed(10)
m = csc_matrix(np.random.randn(dim, dim))
a = m.transpose() * m + csc_matrix(np.identity(dim))
Q = sg.RealSparseMatrixOperator(a)

# creating a random vector
y = np.array(np.random.randn(dim))

# solve the system Qx=y
# the argument is set as True to gather convergence statistics (default is False)
cg = sg.ConjugateGradientSolver(True)
cg.set_iteration_limit(20)
x = cg.solve(Q,y)

print('x:',x)

# verifying the result
print('y:', y)
print('Qx:', Q.apply(x))

residuals = cg.get_residuals()
plt.plot(residuals)
plt.show()
inputs: I am working on the file "Chapman/Ch6-Example_6-05d_gen.ipynb".
The first task is:
First, initialize the values needed in this program.
Can you write Python code for it?

targets: 
r1 =  0.641                # Stator resistance
x1 =  1.106                # Stator reactance
r2 =  0.332                # Rotor resistance
x2 =  0.464                # Rotor reactance
xm = 26.3                  # Magnetization branch reactance
v_phase =  460 / sqrt(3)   # Phase voltage
n_sync  = 1800             # Synchronous speed (r/min)
w_sync  = n_sync * 2*pi/60 # Synchronous speed (rad/s)
inputs: "2. Elemental Mass and Stiffness matrices
The mass and the stiffness matrix are calculated prior time extrapolation, so they are pre-calculated and stored at the beginning of the code. 
The integrals defined in the mass and stiffness matrices are computed using a numerical quadrature, in this cases the GLL quadrature that uses the GLL points and their corresponding weights to approximate the integrals. Hence,
\begin{equation}
M_{ij}^k=\int_{-1}^1 \ell_i^k(\xi)  \ell_j^k(\xi) \ J \ d\xi = \sum_{m=1}^{N_p} w_m \ \ell_i^k (x_m)  \ell_j^k(x_m)\ J =\sum_{m=1}^{N_p} w_m \delta_{im}\ \delta_{jm} \ J= \begin{cases} w_i \ J \ \ \text{ if } i=j \ 0 \ \ \ \ \ \ \ \text{   if } i \neq j\end{cases}
\end{equation}
that is a diagonal mass matrix!. Subsequently, the stiffness matrices is given as 
\begin{equation} 
K_{i,j}= \int_{-1}^1 \ell_i^k(\xi) \cdot \partial x  \ell_j^k(\xi) \ d\xi= \sum{m=1}^{N_p} w_m \ \ell_i^k(x_m)\cdot \partial_x \ell_j^k(x_m)= \sum_{m=1}^{N_p} w_m \delta_{im}\cdot \partial_x\ell_j^k(x_m)= w_i \cdot \partial_x \ell_j^k(x_i) 
\end{equation}
The Lagrange polynomials and their properties have been already used, they determine the integration weights $w_i$ that are returned by the python method "gll". Additionally, the fist derivatives of such basis, $\partial_x \ell_j^k(x_i)$, are needed, the python method "Lagrange1st" returns them."
Please write code following the instructions in jupyter notebook style.

targets: 
# Initialization of system matrices
# -----------------------------------------------------------------
# Elemental Mass matrix
M = np.zeros((N+1, N+1))
for i in range(0, N+1):
    M[i, i] = w[i] * J

# Inverse matrix of M (M is diagonal!)
Minv = np.identity(N+1)
for i in range(0, N+1):
    Minv[i,i] = 1. / M[i,i]

# Elemental Stiffness Matrix
K = np.zeros((N+1, N+1))
for i in range(0, N+1):
    for j in range(0, N+1):
            K[i,j] = w[j] * l1d[i,j] # NxN matrix for every element
inputs: # This Colab requires TF 2.5.
!pip install -U "tensorflow>=2.5"

import os
import pathlib

import matplotlib
import matplotlib.pyplot as plt

import io
import scipy.misc
import numpy as np
from six import BytesIO
from PIL import Image, ImageDraw, ImageFont
from six.moves.urllib.request import urlopen

import tensorflow as tf
import tensorflow_hub as hub

tf.get_logger().setLevel('ERROR')
Given the above code, generate some markdown instructions for it.

targets: 
<table class="tfo-notebook-buttons" align="left">
  <td>     <a target="_blank" href="https://www.tensorflow.org/hub/tutorials/tf2_object_detection"><img src="https://www.tensorflow.org/images/tf_logo_32px.png">TensorFlow.org で表示</a>   </td>
  <td>     <a target="_blank" href="https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/ja/hub/tutorials/tf2_object_detection.ipynb"><img src="https://www.tensorflow.org/images/colab_logo_32px.png">Google Colab で実行</a>   </td>
  <td>     <a target="_blank" href="https://github.com/tensorflow/docs-l10n/blob/master/site/ja/hub/tutorials/tf2_object_detection.ipynb"><img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png">    GitHub でソースを表示</a> </td>
  <td>     <a href="https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/ja/hub/tutorials/tf2_object_detection.ipynb"><img src="https://www.tensorflow.org/images/download_logo_32px.png">ノートブックをダウンロード</a>   </td>
  <td>     <a href="https://tfhub.dev/tensorflow/collections/object_detection/1"><img src="https://www.tensorflow.org/images/hub_logo_32px.png">TF Hub モデルを参照</a>   </td>
</table>

TensorFlow Hub オブジェクト検出 Colab
TensorFlow Hub オブジェクト検出 Colab へようこそ！このノートブックでは、「すぐに使える」画像用オブジェクト検出モデルを実行する手順を説明します。
その他のモデル
このコレクションには、COCO 2017 データセットでトレーニングされた TF 2 オブジェクト検出モデルが含まれています。ここから、現在 tfhub.dev がホストしているすべてのオブジェクト検出モデルを見つけることができます。
インポートとセットアップ
基本のインポートから始めます。
inputs: # Task 1
embed = ...
Given the above code, generate some markdown instructions for it.

targets: 
<h2>TensorFlow Hub Concepts</h2>

TensorFlow Hub is a library for the publication, discovery, and consumption of reusable parts of machine learning models. A module is a self-contained piece of a TensorFlow graph, along with its weights and assets, that can be reused across different tasks in a process known as transfer learning, which we covered as part of the course on Image Models.
To download and use a module, it's as easy as:
However, because modules are self-contained parts of a TensorFlow graph, in order to actually collect values from a module, you'll need to evaluate it in the context of a session.
First, let's explore what hub modules there are. Go to the documentation page and explore a bit.
Note that TensorFlow Hub has modules for Images, Text, and Other. In this case, we're interested in a Text module, so navigate to the Text section.
Within the Text section, there are a number of modules. If you click on a link, you'll be taken to a page that describes the module and links to the original paper where the model was proposed. Click on a model in the Word2Vec section of the page.
Note the details section, which describes what the module expects as input, how it preprocesses data, what it does when it encounters a word it hasn't seen before (OOV means "out of vocabulary") and in this case, how word embeddings can be composed to form sentence embeddings.
Finally, note the URL of the page. This is the URL you can copy to instantiate your module.
<h2>Task 1: Create an embedding using the NNLM model</h2>

To complete this task:
<ol>
    <li>Find the module URL for the NNLM 50 dimensional English model</li>
    <li>Use it to instantiate a module as 'embed'</li>
    <li>Print the embedded representation of "cat"</li>
</ol>

NOTE: downloading hub modules requires downloading a lot of data. Instantiating the module will take a few minutes.
inputs: I am working on the file "Introduction to Data Science in Python/Week 2/.ipynb_checkpoints/Assignment2-checkpoint.ipynb".
The first task is:
Question 1
Which country has won the most gold medals in summer games?
This function should return a single string value.
Can you write Python code for it?

targets: 
def answer_one():
    df_copy = df.copy()
    df_copy.index.names = ["Country"]
    df_copy = df_copy.reset_index()
    df_copy = df_copy.set_index("Gold")
    df_copy = df_copy.sort_index()
    return df_copy.iloc[len(df_copy)-1]["Country"]
answer_one()
inputs: "After the download completes, the blinker logic starts to run inside the FPGA,
converting the 12 MHz clock into a 3 Hz blinking of the LED.

Looks like that's about 3 Hz to me!
<a name="No_Jupyter"></a>But I Don't Like Jupyter!
OK, maybe you're one of those guys, so here's how to do it all from the command line.
First, create a file called blinker.py using your favorite text editor (which is probably vi or Emacs, I guess).
The file just contains the same MyHDL code I showed above.
Here's my version of it:"
Please write code following the instructions in jupyter notebook style.

targets: 
print(open('blinker.py').read())
inputs: "After 60 minutes, you might see that the number of bike at Olin is negative.  We'll fix that problem in the next notebook.
But first, we want to plot the results.
TimeSeries
modsim.py provides an object called a TimeSeries that can contain a sequence of values changing over time.
We can create a new, empty TimeSeries like this:"
Please write code following the instructions in jupyter notebook style.

targets: 
results = TimeSeries()
inputs: I am working on the file "MNIST/0410 -  MNIST Project 6 - The ROC Curve/MNIST.ipynb".
The first task is:
Note the result here may vary from the video as the results from the confusion matrix are different each time you run it.
Can you write Python code for it?

targets: 
precision_score(y_train_0, y_train_pred) # 5618 / (574 + 5618)

5618 / (574 + 5618)
inputs: 6) Create A New Model
Create a new model called second_fashion_model in the cell below.  Make some changes so it is different than fashion_model that you've trained above. The change could be using a different number of layers, different number of convolutions in the layers, etc.
Define the model, compile it and fit it in the cell below.  See how its validation score compares to that of the original model.

targets: # Your code below
____


# Don't remove this line (ensures comptibility with tensorflow 2.0)
second_fashion_model.history.history['val_acc'] = second_fashion_model.history.history['val_accuracy']
# Check your answer
q_6.check()

#_COMMENT_IF(PROD)_
q_6.solution()

#%%RM_IF(PROD)%%
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten, Conv2D

fashion_model = Sequential()
q_1.assert_check_passed()

fashion_model.add(Conv2D(12,
                         activation='relu',
                         kernel_size=3,
                         input_shape = (img_rows, img_cols, 1)))
q_2.assert_check_passed()

fashion_model.add(Conv2D(20, activation='relu', kernel_size=3))
fashion_model.add(Conv2D(20, activation='relu', kernel_size=3))
fashion_model.add(Flatten())
fashion_model.add(Dense(100, activation='relu'))
fashion_model.add(Dense(10, activation='softmax'))

q_3.assert_check_passed()

fashion_model.compile(loss='categorical_crossentropy',
                      optimizer='adam',
                      metrics=['accuracy'])

q_4.assert_check_passed()

# 1 epoch and high val_split to speed up testing
fashion_model.fit(x, y, batch_size=100, epochs=1, validation_split=0.5)

q_5.assert_check_passed()

second_fashion_model = Sequential()
second_fashion_model.add(Conv2D(12,
                         activation='relu',
                         kernel_size=3,
                         input_shape = (img_rows, img_cols, 1)))
# Changed kernel sizes to be 2
second_fashion_model.add(Conv2D(20, activation='relu', kernel_size=2))
second_fashion_model.add(Conv2D(20, activation='relu', kernel_size=2))
# added an addition Conv2D layer
second_fashion_model.add(Conv2D(20, activation='relu', kernel_size=2))
second_fashion_model.add(Flatten())
second_fashion_model.add(Dense(100, activation='relu'))
# It is important not to change the last layer. First argument matches number of classes. Softmax guarantees we get reasonable...
inputs: I am working on the file "2.0/examples/rossiter_mclaughlin.ipynb".
The first task is:
To visualize what is happening, we can plot the radial velocities of each surface element in the mesh at one of these times.
Here just plot on the mesh@model parameterset - the mesh will automatically get coordinates from mesh01 and then we point to rvs@numericalrvs for the facecolors.
Can you write Python code for it?

targets: 
fig = plt.figure(figsize=(12,12))
axs, artists = b['mesh@model'].plot(time=0.03, facecolor='rvs@numericalrvs', edgecolor=None)
inputs: "6.
A veces, es necesario en nuestro problema, tener que eliminar los elementos repetidos de una lista, dejando aquellos que solo aparezcan una sola vez. Es muy común, que muchos usuarios llamen a la función set para esta tarea, haciendo de la lista un conjunto sin elementos repetidos, ordenándolos y luego, el resultado de esto, volverlo a convertir en una lista. Esto, puede no estar mal del todo, pero puede ser que en el caso peor, puede que estemos haciendo un gasto inútil de memoria, tiempo y cálculos, para que, en el caso de que no haya elementos repetidos, sólo obtengamos una lista ordenada.
Es por ello, por lo que existe otra forma de hacerlo. Utilizando lo ya visto, obtén una lista sin elementos repetidos que mantengan el orden de la lista original. Para hacerlo aún más divertido, no uses más de 4 líneas."
Please write code following the instructions in jupyter notebook style.

targets: 
a = [1,1,1,2,5,3,4,8,5,8]
b = []
list(filter(lambda x: b.append(x) if not x in b else False, a))
print("Lista original:\t\t", a)
print("Lista sin repetidos:\t", b)
inputs: I am working on the file "NLP/1-Basic skills/jieba中文处理.ipynb".
The first task is:
关键词提取¶
基于 TF-IDF 算法的关键词抽取
import jieba.analyse
 - jieba.analyse.extract_tags(sentence, topK=20, withWeight=False, allowPOS=())
   - sentence 为待提取的文本
   - topK 为返回几个 TF/IDF 权重最大的关键词，默认值为 20
   - withWeight 为是否一并返回关键词权重值，默认值为 False
   - allowPOS 仅包括指定词性的词，默认值为空，即不筛选
Can you write Python code for it?

targets: 
import jieba.analyse as analyse
lines = open('人工智能火了-人类有些尴尬了.txt').read()
print(type(lines))
seg_list = analyse.extract_tags(lines,topK=15,withWeight=False,allowPOS=())
#print(seg_list)
text = " ".join(seg_list)
print(text)
inputs: Annotating the curve
Wrapping your data (xs and ys) here as a HoloViews element is sufficient to make it visualizable, but there are many other aspects of the data that we can capture to convey more about its meaning to HoloViews. For instance, we might want to specify what the x-axis and y-axis actually correspond to, in the real world. Perhaps this parabola is the trajectory of a ball thrown into the air, in which case we could declare the object as:

targets: trajectory = hv.Curve((xs,ys), kdims=['distance'], vdims=['height'])
trajectory
inputs: # Create a Simulation object by running tardis
from tardis import run_tardis
sim = run_tardis('tardis_example.yml')

# Now use it to create a shell info widget
shell_info = tw.shell_info_from_simulation(sim)

# Call display method of shell_info
shell_info.display()
Given the above code, generate some markdown instructions for it.

targets: 
Shell Info
This widget allows you to get fractional abundances of each shell - all the way from elements to ions to levels - by just clicking on the rows you want to explore!
There are two ways in which you can generate the widget:
Using Simulation object
inputs: %matplotlib inline
import matplotlib.pyplot as plt
temperatures = [4.4,5.1,6.1,6.2,6.1,6.1,5.7,5.2,4.7,4.1,3.9,3.5]
s7 = pd.Series(temperatures, name="Temperature")
s7.plot()
plt.show()
Given the above code, generate some markdown instructions for it.

targets: 
Plotting a Series
Pandas makes it easy to plot Series data using matplotlib (for more details on matplotlib, check out the matplotlib tutorial). Just import matplotlib and call the plot() method:
inputs: "Blocking
We can block a curve. Let's look at a small segment:"
Please write code following the instructions in jupyter notebook style.

targets: 
segment = gr.to_basis(start=600, stop=680)
inputs: # Subtract two matrices
np.subtract(matrix_a, matrix_b)
Given the above code, generate some markdown instructions for it.

targets: 
Subtract Matrices
inputs: <p style="text-align: right; direction: rtl; float: right; clear: both;">
    הפונקציה הזו יכולה לסייע לנו להגיע לתובנות בנוגע לדרך שבה פייתון עובדת בכל מיני מצבים.<br>
    קשה להבין, לדוגמה, מה קורה בקוד הבא:
</p>

targets: collections_of_numbers = [[0, 0, 0]] * 3  # ניצור 3 רשימות של 3 איברים בכל אחת
print(collections_of_numbers)
collections_of_numbers[0].append(100)  # נוסיף את האיבר 100 לרשימה הראשונה בלבד
print(collections_of_numbers)
inputs: I am working on the file "Assignments/ASSIGNMENT-1.ipynb".
The first task is:
Help the cashier
Can you use the modulus operator you just learned about to solve the following task? Imagine you want to help cashiers to return the change in a convenient way. This means you do not want to return hands full of small coins, but rather use bills and as few coins as possible. 
Write code  that classifies a given amount of money into smaller monetary units. Given a specific amout of dollars, your program should report the maximum number of dollar bills,  quarters, dimes, nickels, and pennies. 
Set the amount variable to 11.67. You code should output a report listing the monetary equivalent in dollars, quarters, dimes, nickels, and pennies (one quarter is equivalent to 25 cents; one dime to 10 cents; one nickle to 5 cents and a pennie to 1 cent). Your program should report the maximum number of dollars, then the number of quarters, dimes, nickels, and pennies, in this order, to result in the minimum number of coins. Here are the steps in developing the program:

Convert the amount (11.67) into cents (1167).
First get the amount of cents that you would get after subtracting the maximum amount of dollars (100 cents) using the modulus operator (67 cents).
Then subtract the remainder (67 cents) from the total amount of cents (1167 cents) and divide this by 100 to find the number of dollars.
Use the modulus operator again to find out the remainder after subtracting the maximum amount of quarters (17 cents).
Subtract this remainder (17 cents) from the previous remainder (67 cents) and divide this by 25 to find out the number of quarters.
Follow the same steps for the dimes, nickels and pennies.
Display the result for your cashier! (the amount of dollars, quarters, dimes, nickels and pennies that (s)he would have to give back)`
Can you write Python code for it?

targets: 
# cashier code
inputs: <a id='step4'></a>
4. Download and Load Weather Files
There are various options provided in bifacial_radiance to load weatherfiles. getEPW is useful because you just set the latitude and longitude of the location and it donwloads the meteorologicla data for any location.

targets: # Pull in meteorological data using pyEPW for any global lat/lon
epwfile = demo.getEPW(lat = 37.5, lon = -77.6)  # This location corresponds to Richmond, VA.
inputs: ... and we can draw the decision boundary using pyplot:

targets: x_min, x_max = X_train[:, 0].min() - .5, X_train[:, 0].max() + .5
y_min, y_max = X_train[:, 1].min() - .5, X_train[:, 1].max() + .5
xs = np.arange(x_min, x_max, 0.5)
fig,axes = plt.subplots()
axes.set_aspect('equal')
axes.set_title('Setosa classification')
axes.set_xlabel('Sepal length')
axes.set_ylabel('Sepal width')
axes.set_xlim(x_min, x_max)
axes.set_ylim(y_min, y_max)
plt.sca(axes)
plt.scatter(X_train[:, 0][y_train == 0], X_train[:, 1][y_train == 0], c='red', marker='s')
plt.scatter(X_train[:, 0][y_train == 1], X_train[:, 1][y_train == 1], c='black', marker='x')
ys = (-clf.intercept_[0]- xs * clf.coef_[0, 0]) / clf.coef_[0, 1]
plt.plot(xs, ys, hold=True)
plt.show()
inputs: Achsen kontrollieren

plt.xlim und plt.ylim um der range von den Achsen zu kontrollieren
plt.xscale("log") für ein logaritmischer Achsen

targets: plt.figure()
plt.plot(x, y1)
plt.xlim((1, 5))
plt.ylim((-10, 10))
plt.xscale("log")
plt.plot(x, y2)
plt.xscale("linear")
plt.show()
plt.close()
inputs: "Create Fake Index Data"
Please write code following the instructions in jupyter notebook style.

targets: 
mean = np.array([0.05/252 + 0.02/252, 0.03/252 + 0.02/252])
volatility = np.array([0.2/np.sqrt(252), 0.05/np.sqrt(252)])
variance = np.power(volatility,2)
correlation = np.array(
    [
        [1, 0.25],
        [0.25,1]
    ]
)
covariance = np.zeros((2,2))
for i in range(len(variance)):
    for j in range(len(variance)):
        covariance[i,j] = correlation[i,j]*volatility[i]*volatility[j]

covariance

names = ['foo','bar','rf']
dates = pd.date_range(start='2015-01-01',end='2018-12-31', freq=pd.tseries.offsets.BDay())
n = len(dates)
rdf = pd.DataFrame(
    np.zeros((n, len(names))),
    index = dates,
    columns = names
)

np.random.seed(1)
rdf.loc[:,['foo','bar']] = np.random.multivariate_normal(mean,covariance,size=n)
rdf['rf'] = 0.02/252

pdf = 100*np.cumprod(1+rdf)
pdf.plot()
inputs: Background information on filtering
Here we give some background information on filtering in general,
and how it is done in MNE-Python in particular.
Recommended reading for practical applications of digital
filter design can be found in Parks & Burrus (1987) [1] and
Ifeachor & Jervis (2002) [2], and for filtering in an
M/EEG context we recommend reading Widmann et al. (2015) [7]_.
To see how to use the default filters in MNE-Python on actual data, see
the tut-filter-resample tutorial.
Problem statement
Practical issues with filtering electrophysiological data are covered
in Widmann et al. (2012) [6]_, where they conclude with this statement:
Filtering can result in considerable distortions of the time course
(and amplitude) of a signal as demonstrated by VanRullen (2011) [[3]_].
Thus, filtering should not be used lightly. However, if effects of
filtering are cautiously considered and filter artifacts are minimized,
a valid interpretation of the temporal dynamics of filtered
electrophysiological data is possible and signals missed otherwise
can be detected with filtering.

In other words, filtering can increase signal-to-noise ratio (SNR), but if it
is not used carefully, it can distort data. Here we hope to cover some
filtering basics so users can better understand filtering trade-offs and why
MNE-Python has chosen particular defaults.
Filtering basics
Let's get some of the basic math down. In the frequency domain, digital
filters have a transfer function that is given by:
\begin{align}H(z) &= \frac{b_0 + b_1 z^{-1} + b_2 z^{-2} + \ldots + b_M z^{-M}}
                 {1 + a_1 z^{-1} + a_2 z^{-2} + \ldots + a_N z^{-M}} \
         &= \frac{\sum_{k=0}^Mb_kz^{-k}}{\sum_{k=1}^Na_kz^{-k}}\end{align}
In the time domain, the numerator coefficients $b_k$ and denominator
coefficients $a_k$ can be used to obtain our output data
$y(n)$ in terms of our input data $x(n)$ as:
\begin{align}:label: summations
y(n) &amp;= b_0 x(n) + b_1 x(n-1) + \ldots + b_M x(n-M)
        - a_1 y(n-1) - a_2 y(n - 2) - \ldots - a_N y(n -...

targets: import numpy as np
from scipy import signal, fftpack
import matplotlib.pyplot as plt

from mne.time_frequency.tfr import morlet
from mne.viz import plot_filter, plot_ideal_filter

import mne

sfreq = 1000.
f_p = 40.
flim = (1., sfreq / 2.)  # limits for plotting
inputs: # Choose which one is smaller!
if num_galex1 < num_galex2:
    select_from_galex = idx_galex1
else:
    select_from_galex = idx_galex2

print(len(select_from_galex))
match_galex = data_galex[select_from_galex]
c_selected_galex = c_galex[select_from_galex]

# Select other catalog using GALEX again
# 2MASS
idx_2mass, idx_galex1, d2d, d3d = c_2mass.search_around_sky(c_selected_galex, sep_min)

# WISE
idx_wise, idx_galex2, d2d, d3d = c_wise.search_around_sky(c_selected_galex, sep_min)


print("Number of match in GALEX: ", len(c_selected_galex))
print("Number of match in 2MASS: ", len(idx_2mass))
print("Number of match in WISE : ", len(idx_wise))
Given the above code, generate some markdown instructions for it.

targets: 
Filter all Cats
inputs: 4.2. River Mouth
Is Required: FALSE&nbsp;&nbsp;&nbsp;&nbsp;Type: STRING&nbsp;&nbsp;&nbsp;&nbsp;Cardinality: 0.1
Describe if/how river mouth mixing or estuaries specific treatment is performed

targets: # PROPERTY ID - DO NOT EDIT !  
DOC.set_id('cmip6.ocean.key_properties.nonoceanic_waters.river_mouth')  

# PROPERTY VALUE: 
# Set as follows: DOC.set_value("value")  
DOC.set_value("Kz increase near river mouth (top 20 m)")
inputs: I am working on the file "EllipsesProject/MyEllipsesNotebook.ipynb".
The first task is:
Model training
Can you write Python code for it?

targets: 
# hyperparams
epochs = 50
batch_size = 64
optAlgo = Adam()

# compile model
model.compile(optAlgo,loss=['binary_crossentropy','mse'], metrics=['accuracy'],loss_weights=[1.,.01])

# values to track:
save_to = 'best_model.h5'
checkpoint = ModelCheckpoint(filepath=save_to,verbose=0, save_best_only=True)
callbacks = [checkpoint]

# train model
history = model.fit(X_train, [Y_train_class,Y_train_features], shuffle=True, batch_size=batch_size, 
                             epochs=epochs, verbose=1, validation_split=0.3, callbacks = callbacks)

# Plot training & validation accuracy values
plt.plot(history.history['class_acc'])
plt.plot(history.history['features_acc'])
plt.plot(history.history['val_class_acc'])
plt.plot(history.history['val_features_acc'])
plt.title('Model accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['class_acc','features_acc','val_class_acc', 'val_features_acc'], loc='lower left')
plt.show()

# Plot training & validation loss values (general loss)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model loss (General Loss)')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['loss','val_loss'], loc='upper left')
plt.show()

# Plot training & validation loss values (classification loss)
plt.plot(history.history['class_loss'])
plt.plot(history.history['val_class_loss'])
plt.title('Model loss (Classification Loss)')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['class_loss','val_class_loss'], loc='upper left')
plt.show()

# Plot training & validation loss values (features loss)
plt.plot(history.history['features_loss'])
plt.plot(history.history['val_features_loss'])
plt.title('Model loss (Features Loss)')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['features_loss','val_features_loss'], loc='upper left')
plt.show()
inputs: # Define the model with the best parametrization
...

clf.fit(X_train, y_train)
mean_absolute_error(y_test, clf.predict(X_test))

# Plot the scatterplot
plt.scatter(y_test, clf.predict(X_test))

Given the above code, generate some markdown instructions for it.

targets: 
Fit the best model
inputs: Another way to model the "bag of words" is to evaluate if the tweet has only positive words, only negative words, both positive and negative words or neither positive nor negative words.  Rewrite your code to keep track of all four totals.
Step 5: Count "types" of tweets. Rewrite the code from steps 1 & 3 and determin if each tweet has only positive works, only negative words, both positive and negative words or neither positive nor negative words. Keep total counts the number of each kind of tweet.

targets: only_positive=0;
only_negative=0;
both_pos_and_neg=0;
neither_pos_nor_neg=0;
#Put your code here.

for tweet in tweets_list:
    positive_counter=0
    negative_counter=0
    tweet_processed=tweet.lower()
    for p in punctuation:
        tweet_processed=tweet_processed.replace(p,'')
        words=tweet_processed.split(' ')
        word_count = len(words)
        for word in words:
            if word in positive_words:
                positive_counter=positive_counter+1
            if word in negative_words:
                negative_counter=negative_counter+1
    if(positive_counter > 0):
        if(negative_counter > 0):
            both_pos_and_neg=both_pos_and_neg+1
        else:
            only_positive=only_positive+1;
    else:
        if(negative_counter > 0):
            only_negative=only_negative+1;
        else:
            neither_pos_nor_neg=neither_pos_nor_neg+1;
inputs: I am working on the file "Presentaciones/Notas/06_Versionamiento_git.ipynb".
The first task is:
En caso de ser un repositorio nuevo o desear reinicializar (aceptar nuevas plantillas o mover el directorio), debemos crear el directorio .git y agregar los elementos que teniamos en nuestro directorio. Siempre que se agreguen (o se realiza cualquier actualización de los archivos y directorios, se sugiere realizar un commit para la indicación de la acción en nuestro historial).
Can you write Python code for it?

targets: 
%%bash
cd herramientas
git init
inputs: Confusion Matrix
予測したものと正解の数

targets: %matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import itertools

def plot_confusion_matrix(cm, classes, normalize=False, title='Confusion matrix', cmap=plt.cm.Blues):
    """
    This function prints and plots the confusion matrix.
    Normalization can be applied by setting `normalize=True`.
    """
    if normalize:
        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
        print("Normalized confusion matrix")
    else:
        print('Confusion matrix, without normalization')

    print(cm)

    plt.imshow(cm, interpolation='nearest', cmap=cmap)
    plt.title(title)
    plt.colorbar()
    tick_marks = np.arange(len(classes))
    plt.xticks(tick_marks, classes, rotation=45)
    plt.yticks(tick_marks, classes)

    fmt = '.2f' if normalize else 'd'
    thresh = cm.max() / 2.
    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
        plt.text(j, i, format(cm[i, j], fmt),
                 horizontalalignment="center",
                 color="white" if cm[i, j] > thresh else "black")

    plt.tight_layout()
    plt.ylabel('True label')
    plt.xlabel('Predicted label')


from sklearn.metrics import confusion_matrix

matrix = confusion_matrix(y, ｃｌｆ.predict(X))
plot_confusion_matrix(matrix, ['A', 'B', 'C'], normalize=True)

## クロスバリデーション
for classifier, title in zip(classifiers, ['Decision Tree (depth=2)', 'KNN (k=1)', 'Kernel SVM', 'Iris Classifier']):
    scores = cross_val_score(classifier, X, y, cv=6)
    print("Cross Validation Score of %s" % title)
    print("mean(%s) = %s" % (scores, scores.mean()))
inputs: "13.765.202 lines in train.csv  
8.022.757 lines in test.csv  

Few words about the dataset
Predictions is made in the USA corn growing states (mainly Iowa, Illinois, Indiana) during the season with the highest rainfall (as illustrated by Iowa for the april to august months)
The Kaggle page indicate that the dataset have been shuffled, so working on a subset seems acceptable
The test set is not a extracted from the same data as the training set however, which make the evaluation trickier
Load the dataset"
Please write code following the instructions in jupyter notebook style.

targets: 
%%time
filename = "data/reduced_train_100000.csv"
#filename = "data/reduced_train_100000.csv"
raw = pd.read_csv(filename)
raw = raw.set_index('Id')

raw['Expected'].describe()
inputs: And the survival function.

targets: sf = hf.MakeSurvival()
sf.ts, sf.ss
inputs: # Strings
Given the above code, generate some markdown instructions for it.

targets: 
Strings can be declared with either single or double quotes.
inputs: # This is an example observation from the dataset.

example_observation = []

serialized_example = serialize_example(False, 4, b'goat', 0.9876)
serialized_example
Given the above code, generate some markdown instructions for it.

targets: 
例如，假设您从数据集中获得了一个观测值 [False, 4, bytes('goat'), 0.9876]。您可以使用 create_message() 创建和打印此观测值的 tf.Example 消息。如上所述，每个观测值将被写为一条 Features 消息。请注意，tf.Example 消息只是 Features 消息外围的包装器：
inputs: I am working on the file "4. First steps with NetworkX.ipynb".
The first task is:
Aside: dictionaries!
In the examples above, you may have noticed that we used curly braces {} when defining attributes. The curly braces denote an object called a dictionary (dict). A dictionary is an object that stores key-value pairs. 
For example, I might want to be able to look up a bunch of people by their student IDs. To do that, I could create a dictionary with student IDs as keys, and names as values:
Can you write Python code for it?

targets: 
students = {    # I was born in the 1980s...
    292883094: 'Bob Dole',
    940528892: 'Ross Perot',
    940927000: 'Janet Reno',
}
inputs: # Arg: quality and num_per_dim -> tradeoffs between quality and time spent running 
# quality affects dense=False, and num_per_dim affects dense=True
ckpt_path = './ckpt/exif_final/exif_final.ckpt'
exif_demo = demo.Demo(ckpt_path=ckpt_path, use_gpu=0, quality=3.0, num_per_dim=30)
Given the above code, generate some markdown instructions for it.

targets: 
Initialize Demo Solver