Muennighoff/xP3x-sample huggingface dataset:
inputs: By the end of the workshop you will understand what most of these mean.
But rather than listen through a lesson, you get to try and figure what they mean yourself.
To help you with that, here's a function that prints the information from two arrays side by side:

targets: def info_for_two(one_array, another_array):
    """Prints side-by-side results of running np.info on its inputs."""
    def info_as_ordered_dict(array):
        """Converts return of np.infor into an ordered dict."""
        import collections
        import io
        buffer = io.StringIO()
        np.info(array, output=buffer)
        data = (
            item.split(':') for item in buffer.getvalue().strip().split('\n'))
        return collections.OrderedDict(
            ((key, value.strip()) for key, value in data))
    one_dict = info_as_ordered_dict(one_array)
    another_dict = info_as_ordered_dict(another_array)
    name_w = max(len(name) for name in one_dict.keys())
    one_w = max(len(name) for name in one_dict.values())
    another_w = max(len(name) for name in another_dict.values())
    output =  (
        f'{name:<{name_w}} : {one:>{one_w}} : {another:>{another_w}}'
        for name, one, another in zip(
            one_dict.keys(), one_dict.values(), another_dict.values()))
    print('\n'.join(output))

a = np.arange(5)
b = a[:]
info_for_two(a, b)
inputs: Color Operations
The following spectra.Color methods return new colors:

.blend(other_color, ratio=0.5)
.brighten(amount=10)
.darken(amount=10)
.saturate(amount=10)
.desaturate(amount=10)

The parameter for .brighten/.darken is a positive/negative linear adjustment to the L(ightness) value of the color's Lab representation. (Spectra converts the color to Lab, makes the change, and then converts back to the original color space.)
Likewise, the parameter for .saturate/.desaturate is a positive/negative linear adjustment to the c(hroma) value of the color's Lch representation.

targets: yellow = spectra.html("yellow").to("lab")

tomato.blend(yellow, 0.25).hexcode

swatches([
    tomato,
    tomato.blend(yellow, 0.25),
    tomato.blend(yellow, 0.75),
    yellow
])

swatches([
    tomato.brighten(30),
    tomato,
    tomato.darken(30)
])

swatches([
    tomato.saturate(40),
    tomato,
    tomato.desaturate(40)
])
inputs: "Commit logs
The messages that are added to the commit command are supposed to give a short (often one-line) description of the changes/additions/deletions in the commit. If the -m "message" is omitted when invoking the git commit message an editor will be opened for you to type a commit message (for example useful when a longer commit message is required). 
We can look at the revision log by using the command git log:"
Please write code following the instructions in jupyter notebook style.

targets: 
!git log
inputs: I am working on the file "Dia3/3_Limite_Shockley_Queisser.ipynb".
The first task is:
Tiempo de Graficar
Actividad: Grafica el spectro solar
Ojo: No se te olvide agregar titulos e informacion a la grafica
Compara con el espector visible

3-6 Donde perdemos eficiencia?
Podemos indentificar 5 partes:

Conversion de energia a electricidad util
Energia fotonica debajo del bandgap, esta energia no se absorbe y entonces no se gasta.
Energia fotonica en exceso del bandgap: esto es por que el electron y el hoyo se relajan inmediatamente a borde de las bandas. Por ejemplo, para un semiconductor de 1eV-bandgap, un foton de  3eV crea el mismo par electron-hoyo que uno foton de 1.01eV. Esos 2eV de energia extra que tenia el 3eV se pierden.
Recombination de electron-hoyos: toda la recombinacion en el punto de maxima energia (max-power-point) se disipa como calor;
Voltaje de la celda : el voltaje  max-power-point is less than the bandgap.

Podemos poner esto en una ecuacion:
$(\text{Energia solar entrante}) = V_{MPP} \times I_{MPP}$<br>
$ \qquad + (\text{Energia fotonica debajo del bandgap})$<br>
$ \qquad + (\text{Energia fotonica arriba del bandgap} - \text{Numero de fotones arriba del bandgap} \times \text{Energia Bandgap})$<br>
$ \qquad + ((\text{Numero de fotones arriba del bandgap}) - I_{MPP} / e) \times (\text{Energia Bandgap})$<br>
$ \qquad + I_{MPP} \times (\text{Voltaje del Bandgap} - V_{MPP})$<br>
2 - Luz incidente
Calcularemos una cantidad mas intuitiva
Fotones por unidad de tiempo, area y rango de energia
Haremos este cambio para hacer calculos en funcion de fotones en vez de numero onda.
Para esto definimos FotonesPorTEA (Tiempo, Energia, Area). Convertimos los datos AM1.5 a estas nuevas unidades usando la formula:
$\text{FotonesPerTEA} = \frac{d(\text{numero de fotos, por unidad de tiempo, por unidad de area})}{dE} = \frac{d(\text{Energia foton por unidad de area})}{d\lambda} \; \frac{(\text{numero de fotos, por unidad de tiempo, por unidad de area})}{(\text{Energia foton por unidad de area})} \left| \frac{d\lambda}{dE} \right| = $
$ = (\text{AM1.5 spectro}) \; \frac{1}{\text{energia foton}}...
Can you write Python code for it?

targets: 
def FotonesPorTEA(Ephoton):
    wavelength = nu.hPlanck * nu.c0 / Ephoton
    return AM15interp(wavelength) * (1 / Ephoton) * (nu.hPlanck * nu.c0 / Ephoton**2)
inputs: I am working on the file "HipHopDataExploration/HipHopDataExploration.ipynb".
The first task is:
Building a concordance, gets us to the area of elementary information retrieval (IR)<a href="#fn1" id="ref1">1</a>, think, <i> basic search engine</i>. So why do we even need to “normalize” terms? We want to match <b>U.S.A.</b> and <b>USA</b>. Also when we enter <b>roll</b>, we would like to match <b>Roll</b>, and <b>rolling</b>. One way to do this is to stem the word. That is, reduce it down to its base/stem/root form. As such <b>automate(s)</b>, <b>automatic</b>, <b>automation</b> all reduced to <b>automat</b>. Most stemmers are pretty basic and just chop off standard affixes indicating things like tense (e.g., "-ed") and possessive forms (e.g., "-'s"). Here, we'll use the most popular english language stemmer, the Potter stemmer, which comes with NLTK. 
Once our tokens are stemmed, we can rest easy knowing that roll, Rolling, Rolls will all stem to roll.
<sup id="fn1">1. Some of this content has been adapted from Dan Jurafsky's <a href="https://web.stanford.edu/class/cs124/">Stanford CS124 class</a><a href="#ref1" title="Jump back to footnote 1 in the text."></a></sup>
Can you write Python code for it?

targets: 
porter = nltk.PorterStemmer()

stemmer = nltk.PorterStemmer()
stemmed_tokens = [stemmer.stem(t) for t in AmericanGangster_corpus]

for token in sorted(set(stemmed_tokens))[860:870]:
    print token + ' [' + str(stemmed_tokens.count(token)) + ']'
inputs: # Instantiate a "coarse" 2-group EnergyGroups object
coarse_groups = mgxs.EnergyGroups()
coarse_groups.group_edges = np.array([0., 0.625, 20.0e6])

# Instantiate a "fine" 8-group EnergyGroups object
fine_groups = mgxs.EnergyGroups()
fine_groups.group_edges = np.array([0., 0.058, 0.14, 0.28,
                                    0.625, 4.0, 5.53e3, 821.0e3, 20.0e6])
Given the above code, generate some markdown instructions for it.

targets: 
Now we are finally ready to make use of the openmc.mgxs module to generate multi-group cross sections! First, let's define "coarse" 2-group and "fine" 8-group structures using the built-in EnergyGroups class.
inputs: "<u>Problem 3</u>
Use the reduce function to find the produc of all the entries in the list
[47,11,42,102,13]"
Please write code following the instructions in jupyter notebook style.

targets: 
print reduce(lambda x,y: x*y, [47,11,42,102,13])

# note that you can improve the speed of the calculation using built-in functions
# or better still: using the numpy module
from operator import mul
import numpy as np
a = range(1, 101)

print "\nreduce(lambda x, y: x * y, a)"
%timeit reduce(lambda x, y: x * y, a)   # (1)

print "\nreduce(mul, a)"
%timeit reduce(mul, a)                  # (2)

print "\nnp.prod(a)"
a = np.array(a)
%timeit np.prod(a)                      # (3)
inputs: I am working on the file "Assignment1/Assignment1.ipynb".
The first task is:
Set configurables:
Can you write Python code for it?

targets: 
_DATA_FILEPATH = 'datagovdatasetsviewmetrics.csv'
_ROTATION_DEGREES = 90
_BOTTOM_MARGIN = 0.35
_COLOR_THEME = 'coolwarm'
_LABEL_X = 'Organizations'
_LABEL_Y = 'Views'
_TITLE = 'Organizations with Most Views'
_ORGANIZATION_COUNT = 10
_MAX_LABEL_LENGTH = 20
inputs: "<p style="text-align:right;direction:rtl;">שעון עולמי</p>
<p style="text-align:right;direction:rtl;">בשאלה זו נכתוב גרסה של שעון עולמי התומך ב־4 אזורי זמן:<br>

<ul style="text-align: right; direction: rtl; float: right; clear: both;">
    <li>תל אביב – TLV</li>
    <li>לונדון – LDN</li>
    <li>ניו יורק – NYC</li>
    <li>טוקיו – TYO</li>
</ul>

<p style="text-align: right; direction: rtl; float: right; clear: both;">
אם נקבל תל אביב, נחזיר את השעה בתוספת 3 שעות.<br>
אם נקבל ניו יורק נחזיר את השעה פחות 4.<br>
אם נקבל טוקיו נחזיר את השעה ועוד 9.<br>
בכל מקרה אחר נחזיר את השעה כמו שקיבלנו אותה.
</p>


<p style="text-align:right;direction:rtl;">השעון שלנו בתרגיל זה הוא שעון 24 שעות בפורמט HH:MM.</p>
<p style="text-align:right;direction:rtl;">תחילה, כתבו פונקציה המקבלת את השעה בפורמט HH:MM, ומקבלת את מספר השעות שיש להוסיף או להוריד מהשעה הנתונה, ומחזירה את השעה המעודכנת.</p><br>
<code>time_shift("08:44", 1)</code><br>
<samp style="text-align:left;direction:ltr;">"09:44"</samp><br>
<code>time_shift("07:31", -2)</code><br>
<samp>"05:31"</samp>
<p style="text-align:right;direction:rtl;">כמו כן עליכם לוודא שמוכנסת שעה חוקית, ולהדפיס שגיאה אם לא:</p><br>
<code>time_shift("32:12", 4)</code><br>
<samp>"Invalid time inserted"</samp>
<p style="text-align:right;direction:rtl;">נוסף לכך, עליכם לתמוך במעברים בין יממות. כלומר עליכם לבצע נכון פעולות מסוג זה:</p><br>
<code>time_shift("23:30", 2)</code><br>
<samp>"01:30"</samp><br>
<code>time_shift("04:13", -5)</code><br>
<samp>"23:13"</samp>"
Please write code following the instructions in jupyter notebook style.

targets: 
# כתבו את הפונקציה שלכם כאן
inputs: "12. 1列目をcol1.txtに，2列目をcol2.txtに保存
各行の1列目だけを抜き出したものをcol1.txtに,
2列目だけを抜き出したものをcol2.txtとしてファイルに保存せよ.
確認にはcutコマンドを用いよ."
Please write code following the instructions in jupyter notebook style.

targets: 
col1 = open('col1.txt', 'w')
col2 = open('col2.txt', 'w')
hightemp = [i.replace('\t', ' ').split() for i in open('hightemp.txt', 'r')]

col1.write("\n".join(map(str, [i[0] for i in hightemp])))
col1.close()
col2.write("\n".join(map(str, [i[1] for i in hightemp])))
col2.close()
inputs: "<h1>Logging in to a web server</h1>

<h2>Get username and password</h2>
<li>Best to store in a file for reuse
<li>You will need to set up your own login and password and place them in a file called wikidata.txt
<li>Line one of the file should contain your username
<li>Line two your password"
Please write code following the instructions in jupyter notebook style.

targets: 
with open('wikidata.txt') as f:
    contents = f.read().split('\n')
    username = contents[0]
    password = contents[1]
inputs: %%sql
SELECT STATION FROM CENTRAL_LINE
  WHERE REGEXP_LIKE(STATION,'(?:West)')
Given the above code, generate some markdown instructions for it.

targets: 
The following SQL is equivalent, except that the matched
pattern is not kept for future use in matching.
inputs: I am working on the file "Bipropagation_fashionMNIST.ipynb".
The first task is:
Preprocess the data
The data must be preprocessed before training the network. If you inspect the first image in the training set, you will see that the pixel values fall in the range of 0 to 255:
Can you write Python code for it?

targets: 
plt.figure()
plt.imshow(train_images[0])
plt.colorbar()
plt.grid(False)
inputs: And Bob calculates $A^b \bmod p$:

targets: shared_secret_bob = pow(alice_to_bob, b, p)
print(shared_secret_bob)
inputs: "Create a Baseline DNN Model in Keras
Now let's build the Deep Neural Network (DNN) model in Keras using the functional API. Unlike the sequential API, we will need to specify the input and hidden layers.  Note that we are creating a linear regression baseline model with no feature engineering. Recall that a baseline model is a solution to a problem without applying any machine learning techniques."
Please write code following the instructions in jupyter notebook style.

targets: 
# Build a simple Keras DNN using its Functional API
def rmse(y_true, y_pred):  # Root mean square error
    return tf.sqrt(tf.reduce_mean(tf.square(y_pred - y_true)))


def build_dnn_model():
    # input layer
    inputs = {
        colname: layers.Input(name=colname, shape=(), dtype='float32')
        for colname in NUMERIC_COLS
    }

    # feature_columns
    feature_columns = {
        colname: fc.numeric_column(colname)
        for colname in NUMERIC_COLS
    }

    # Constructor for DenseFeatures takes a list of numeric columns
    dnn_inputs = layers.DenseFeatures(feature_columns.values())(inputs)

    # two hidden layers of [32, 8] just in like the BQML DNN
    h1 = layers.Dense(32, activation='relu', name='h1')(dnn_inputs)
    h2 = layers.Dense(8, activation='relu', name='h2')(h1)

    # final output is a linear activation because this is regression
    output = layers.Dense(1, activation='linear', name='fare')(h2)
    model = models.Model(inputs, output)

    # compile model
    model.compile(optimizer='adam', loss='mse', metrics=[rmse, 'mse'])

    return model
inputs: I am working on the file "0.24/_downloads/c1dad3cf15579a4724c19d6df916b111/mixed_norm_inverse.ipynb".
The first task is:
Compute sparse inverse solution with mixed norm: MxNE and irMxNE
Runs an (ir)MxNE (L1/L2 :footcite:GramfortEtAl2012 or L0.5/L2
:footcite:StrohmeierEtAl2014 mixed norm) inverse solver.
L0.5/L2 is done with irMxNE which allows for sparser source estimates with less
amplitude bias due to the non-convexity of the L0.5/L2 mixed norm penalty.
Can you write Python code for it?

targets: 
# Author: Alexandre Gramfort <alexandre.gramfort@inria.fr>
#         Daniel Strohmeier <daniel.strohmeier@tu-ilmenau.de>
#
# License: BSD-3-Clause

import numpy as np

import mne
from mne.datasets import sample
from mne.inverse_sparse import mixed_norm, make_stc_from_dipoles
from mne.minimum_norm import make_inverse_operator, apply_inverse
from mne.viz import (plot_sparse_source_estimates,
                     plot_dipole_locations, plot_dipole_amplitudes)

print(__doc__)

data_path = sample.data_path()
fwd_fname = data_path + '/MEG/sample/sample_audvis-meg-eeg-oct-6-fwd.fif'
ave_fname = data_path + '/MEG/sample/sample_audvis-ave.fif'
cov_fname = data_path + '/MEG/sample/sample_audvis-shrunk-cov.fif'
subjects_dir = data_path + '/subjects'

# Read noise covariance matrix
cov = mne.read_cov(cov_fname)
# Handling average file
condition = 'Left Auditory'
evoked = mne.read_evokeds(ave_fname, condition=condition, baseline=(None, 0))
evoked.crop(tmin=0, tmax=0.3)
# Handling forward solution
forward = mne.read_forward_solution(fwd_fname)
inputs: As expected, from a certain smoothness (3 x voxel size), the distribution remains the same.
First: the distribution without smoothness!
The RF with smoothness 0 should be just normally distributed values without correlation.  The peaks with smoothness 0 should be just maxima of a sample of normally distributed values.  We know the distribution of a sample with normally distribution values from order statistics:
$f_{max}(x) = nF(x)^{n-1}f(x)$
This is to show that the distribution of the maximum of the sample is the right one.

Simulate n normally distributed values
Take the maximum
Compute for a range of values the distribution of the maximum
Show histogram of simulated maxima with the pdf on top

targets: # random sample

smplm = []

for i in range(100000):
    smpl = np.random.standard_normal((n,))
    smplm.append(max(smpl))

# distribution

xm = np.arange(-1,5,0.001)
ym = n*stats.norm.cdf(xm)**(n-1)*stats.norm.pdf(xm)

# histogram

twocol=cb.qualitative.Paired_10.mpl_colors
plt.figure(figsize=(6,3))
ax=plt.subplot(111)
ax.hist(smplm,100,normed=1,facecolor=twocol[0],alpha=0.75,lw=0)
ax.plot(xm,ym,color=twocol[1],lw=3)
ax.set_xlim([-1,5])
plt.show()
inputs: # Download the data and preview
!gsutil cp gs://ml-design-patterns/fraud_data_kaggle.csv .

fraud_data = pd.read_csv('fraud_data_kaggle.csv')
fraud_data.head()

# Drop a few columns we won't use for this demo
fraud_data = fraud_data.drop(columns=['nameOrig', 'nameDest', 'isFlaggedFraud'])
fraud_data = pd.get_dummies(fraud_data)

# Split into separate dataframes
fraud = fraud_data[fraud_data['isFraud'] == 1]
not_fraud = fraud_data[fraud_data['isFraud'] == 0]

# Take a random sample of non-fraud data
# The .005 frac will give us around an 80/20 split of not-fraud/fraud samples
# You can experiment with this value
not_fraud_sample = not_fraud.sample(random_state=2, frac=.005)

# Put the data back together and shuffle
fraud_data = pd.concat([not_fraud_sample,fraud])
fraud_data = shuffle(fraud_data, random_state=2)

# Look at our data balance after downsampling
fraud_data['isFraud'].value_counts()

train_test_split = int(len(fraud_data) * .8)

train_data = fraud_data[:train_test_split]
test_data = fraud_data[train_test_split:]

train_labels = train_data.pop('isFraud')
test_labels = test_data.pop('isFraud')

model = xgb.XGBRegressor(
    objective='reg:linear'
)

model.fit(train_data.values, train_labels)

# Get some test predictions
y_pred = model.predict(test_data.values)

# To build a confusion matrix using the scikit utility, we'll need the values as ints
y_pred_formatted = []

for i in y_pred:
  y_pred_formatted.append(int(round(i)))

cm = confusion_matrix(test_labels.values, y_pred_formatted)
print(cm)

# This is from the sklearn docs
# https://scikit-learn.org/0.18/auto_examples/model_selection/plot_confusion_matrix.html
def plot_confusion_matrix(cm, classes,
                          normalize=False,
                          title='Confusion matrix',
                          cmap=plt.cm.Blues):
    """
    This function prints and plots the confusion matrix.
    Normalization can be applied by setting `normalize=True`.
    """
    plt.imshow(cm, interpolation='nearest', cmap=cmap)
    plt.title(title)
   ...
Given the above code, generate some markdown instructions for it.

targets: 
Downsampling
To demonstrate downsampling, we'll be using this synthetic fraud detection dataset from Kaggle. We've made a version of it available in a public Cloud Storage bucket.
inputs: %%px --local
def resize_method(filename, method):
    filepath = os.path.join(src_dir, filename)
    im = Image.open(filepath)
    resized_im = im.resize((hsize, vsize), method)
    resized_filepath = os.path.join(trg_dir, prefix + filename)
    resized_im.save(resized_filepath, "JPEG", quality = 100)

def resize_NEAREST(filename):
    resize_method(filename, Image.NEAREST)

def resize_LANCZOS(filename):
    resize_method(filename, Image.LANCZOS)
Given the above code, generate some markdown instructions for it.

targets: 
Let us define functions that do the above in one go for all files:
inputs: 1. Write the simulation which determine if in one round of replication HIV will mutate or not

targets: # Set the states (mutation,no_mutation)

# Set the probabilities

# flip the coin (make the choice)
inputs: # GRADED FUNCTION: normalizeRows

def normalizeRows(x):
    """
    Implement a function that normalizes each row of the matrix x (to have unit length).
    
    Argument:
    x -- A numpy matrix of shape (n, m)
    
    Returns:
    x -- The normalized (by row) numpy matrix. You are allowed to modify x.
    """
    
    ### START CODE HERE ### (≈ 2 lines of code)
    # Compute x_norm as the norm 2 of x. Use np.linalg.norm(..., ord = 2, axis = ..., keepdims = True)
    x_norm = np.linalg.norm(x, ord = 2, axis = 1, keepdims = True)
    
    # Divide x by its norm.
    x = x/x_norm
    ### END CODE HERE ###

    return x

x = np.array([
    [0, 3, 4],
    [1, 6, 4]])
print("normalizeRows(x) = " + str(normalizeRows(x)))
Given the above code, generate some markdown instructions for it.

targets: 
Expected Output: 
<table style="width:100%">
     <tr> 
       <td> **image2vector(image)** </td> 
       <td> [[ 0.67826139]
 [ 0.29380381]
 [ 0.90714982]
 [ 0.52835647]
 [ 0.4215251 ]
 [ 0.45017551]
 [ 0.92814219]
 [ 0.96677647]
 [ 0.85304703]
 [ 0.52351845]
 [ 0.19981397]
 [ 0.27417313]
 [ 0.60659855]
 [ 0.00533165]
 [ 0.10820313]
 [ 0.49978937]
 [ 0.34144279]
 [ 0.94630077]]</td> 
     </tr>


</table>

1.4 - Normalizing rows
Another common technique we use in Machine Learning and Deep Learning is to normalize our data. It often leads to a better performance because gradient descent converges faster after normalization. Here, by normalization we mean changing x to $ \frac{x}{\| x\|} $ (dividing each row vector of x by its norm).
For example, if $$x = 
\begin{bmatrix}
    0 & 3 & 4 \
    2 & 6 & 4 \
\end{bmatrix}\tag{3}$$ then $$\| x\| = np.linalg.norm(x, axis = 1, keepdims = True) = \begin{bmatrix}
    5 \
    \sqrt{56} \
\end{bmatrix}\tag{4} $$and        $$ x_normalized = \frac{x}{\| x\|} = \begin{bmatrix}
    0 & \frac{3}{5} & \frac{4}{5} \
    \frac{2}{\sqrt{56}} & \frac{6}{\sqrt{56}} & \frac{4}{\sqrt{56}} \
\end{bmatrix}\tag{5}$$ Note that you can divide matrices of different sizes and it works fine: this is called broadcasting and you're going to learn about it in part 5.
Exercise: Implement normalizeRows() to normalize the rows of a matrix. After applying this function to an input matrix x, each row of x should be a vector of unit length (meaning length 1).
inputs: <h4>datetime objects can check validity</h4>
<li>A ValueError exception is raised if the object is invalid</li>

targets: some_date=datetime.date(2015,2,29)
#some_date =datetime.date(2016,2,29)
#some_time=datetime.datetime(2015,2,28,23,60,0)
inputs: Character histogram
https://www.careercup.com/question?id=4535655694598144
You have the file with word at a single line. 
input sample file
abactor 
abaculus 
abacus 
Abadite 
. 
. 
Zyrenian
Output
******************************************************************a 
*************b 
**********************************c 
**********************d 
*******************************************************************************e
Character histogram

you have to count the character and create a histogram in alphabetical order. 
now you have to produce a histogram with max 80 character in line in reference to max count 
now same out based histrogram based on the character count

targets: def generate_histogram(texts, width):
    letters = {}
    m = None
    for t in texts:
        for c in t.lower():
            letters[c] = letters.get(c, 0) + 1
            if m is None or letters[c] > m:
                m = letters[c]
    
    for i in range(ord("a"), ord("z") + 1):
        if chr(i) in letters:
            n = int(letters[chr(i)] * width / m)
            print("*" * n + chr(i))
generate_histogram(["aaaaaaaaaabbbbbcccdderrrkkkkk"], 80)
inputs: I am working on the file "DefensiveProgramming_3.ipynb".
The first task is:
And test it...
Can you write Python code for it?

targets: 
test_range_overlap()

test_range_overlap_one_range()
inputs: I am working on the file "00_Python_Crash_Course.ipynb".
The first task is:
Here, the 3rd element of a has been changed to 17.  Now let's check on b.
Can you write Python code for it?

targets: 
b
inputs: # PROMPT 9 - write function


Given the above code, generate some markdown instructions for it.

targets: 
Step 1.c: Refactor into a function getDirections()
Now the we understand the API, let's refactor the code into a useable function 
FUNCTION INPUTS:
PROMPT 5

FUNCTION OUTPUTS:
PROMPT 6

ALGORITHM:
PROMPT 7

TEST (How do we know it's right?)
PROMPT 8
inputs: "Another useful method is the .intersection(...) that returns the records that are equal in both RDDs."
Please write code following the instructions in jupyter notebook style.

targets: 
rdd5 = rdd1.intersection(rdd2)
rdd5.collect()
inputs: "Build the Neural Network
We'll build the components necessary to build a Sequence-to-Sequence model by implementing the following functions below:
- model_inputs
- process_decoding_input
- encoding_layer
- decoding_layer_train
- decoding_layer_infer
- decoding_layer
- seq2seq_model
Input
The model_inputs() function creates TF Placeholders for the Neural Network. It would create the following placeholders:

Input text placeholder named "input" using the TF Placeholder name parameter with rank 2.
Targets placeholder with rank 2.
Learning rate placeholder with rank 0.
Keep probability placeholder named "keep_prob" using the TF Placeholder name parameter with rank 0.

Returns the placeholders in the following the tuple (Input, Targets, Learing Rate, Keep Probability)"
Please write code following the instructions in jupyter notebook style.

targets: 
def model_inputs():
    """
    Create TF Placeholders for input, targets, and learning rate.
    :return: Tuple (input, targets, learning rate, keep probability)
    """

    inputs = tf.placeholder(tf.int32, [None, None], name='input')
    targets = tf.placeholder(tf.int32, [None, None], name='targets')
    learning_rate = tf.placeholder(tf.float32, name='learning_rate')
    keep_prob = tf.placeholder(tf.float32, name='keep_prob')
    return inputs, targets, learning_rate, keep_prob

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_inputs(model_inputs)
inputs: Cool example of using sparklines from Peter Baumgartner
https://twitter.com/pmbaumgartner/status/1084645440224559104

targets: def sparkline_str(x):
    bins=np.histogram(x)[0]
    sl = ''.join(sparklines(bins))
    return sl
sparkline_str.__name__ = "sparkline"

df.groupby('name')['quantity', 'ext price'].agg(['mean', sparkline_str])
inputs: I am working on the file "Heater Calibration Tests.ipynb".
The first task is:
Once $m$ and $c$ have been calculated in the calibrate function, we have to make a preadjustment function, which is given by:
$$ preadjustment(x) := \frac{x - c}{m} $$
Can you write Python code for it?

targets: 
def error_correction_function(heater_current, measurements):
    m, c, r_value, p_value, std_err = calibrate(heater_current, measurements)
    return (lambda x: (x - c) / m)
inputs: - needs to be escaped if we want to include it in the character range:

targets: re.search("[0-9\-]+", "1-2")
inputs: #@title Create fake dataset of adgroups and conversion values
#@markdown We are generating random data: each row is an individual conversion
#@markdown with a given conversion value. \
#@markdown For each conversion, we know the
#@markdown adgroup, which is our only feature here and just consists of 3 letters.

n_consenting_customers = 8000  #@param
n_nonconsenting_customers = 2000  #@param


def simulate_conversion_data_consenting_non_consenting(
    n_consenting_customers: int,
    n_nonconsenting_customers: int) -> typing.Tuple[pd.DataFrame, pd.DataFrame]:
  """Simulates dataframes for consenting and non-consenting customers.

    Args:
      n_consenting_customers: Desired number of consenting customers. Should be
        larger than n_nonconsenting_customers.
      n_nonconsenting_customers: Desired number non non-consenting customers.

    Returns:
      Two dataframes of simulated consenting and non-consenting customers.
  """
  fake_adgroups = np.array(
      ['_'.join(fake_ad) for fake_ad in (combinations('ABCDEFG', 3))])

  data_consenting = pd.DataFrame.from_dict({
      'adgroup':
          fake_adgroups[np.random.randint(
              low=0, high=len(fake_adgroups), size=n_consenting_customers)],
      'conversion_value':
          np.random.lognormal(1, size=n_consenting_customers)
  })

  data_nonconsenting = pd.DataFrame.from_dict({
      'adgroup':
          fake_adgroups[np.random.randint(
              low=0, high=len(fake_adgroups), size=n_nonconsenting_customers)],
      'conversion_value':
          np.random.lognormal(1, size=n_nonconsenting_customers)
  })
  return data_consenting, data_nonconsenting


data_consenting, data_nonconsenting = simulate_conversion_data_consenting_non_consenting(
    n_consenting_customers, n_nonconsenting_customers)
data_consenting.head()
Given the above code, generate some markdown instructions for it.

targets: 
Data Simulation
inputs: %%time

g2 = g.gremlin('g.E().limit(10000)')


print('NODES:')
g2._nodes.info()
g2._nodes.sample(3)

print('EDGES:')
print(g2._edges.info())

g2._edges.sample(3)

%%time

# Enrich nodes dataframe with any available server property data

g3 = g2.fetch_nodes()

print(g3._nodes.info())

g3._nodes.sample(3)

%%time

g3.plot()
Given the above code, generate some markdown instructions for it.

targets: 
Query & plot

PyGraphistry automatically converts gremlin results into node/edge dataframes
Edge queries typically only return node IDs; call fetch_nodes() to enrich your g._nodes dataframe
PyGraphistry plots dataframes
inputs: "And here's the final temperature:"
Please write code following the instructions in jupyter notebook style.

targets: 
coffee.T_final = get_last_value(results.T)
T_final = get_last_value(results.T)
inputs: # Randomly select a record from the loaded test data.
smple = fedata.sample(False, .8).limit(1).select(input_features)

smple.toPandas().head()
Given the above code, generate some markdown instructions for it.

targets: 
Test the functions
We can then test the init() and run() functions right here in the notebook. It's about impossible to debug after publish a web service.
First we get a sample test observation that we can score. For this, we can randomly select a single record from the test data we've loaded from Azure blob.
inputs: I am working on the file "Homework/04 - Applied ML/Homework_04_Referees_teamawesome-Q1 + Bonus.ipynb".
The first task is:
III. Random Forest
Can you write Python code for it?

targets: 
#Initialize
forest = RandomForestClassifier(n_estimators = 100)

# Fit the training data and create the decision trees
forest = forest.fit(trainArr,trainRes_1)

# Take the same decision trees and run it on the test data
Data_Testing = Data_Simple5.sample(frac=0.2)
Input_Data_Testing = Data_Testing.drop(exclude, axis=1)
testArr = Input_Data_Testing.as_matrix()
results = forest.predict(testArr)

Data_Testing['predictions'] = results
Data_Testing.head()

#see percentage of right predictions
correct = list(Data_Testing[Data_Testing['raterScale'] == Data_Testing['predictions']].index)
A = len(correct)
percCorrect = A/Data_Testing['raterScale'].size
percCorrect
inputs: I am working on the file "2.0/tutorials/pblum.ipynb".
The first task is:
Let's see how changing the value of pblum affects the computed light curve.  By default, pblum is set to be 4 pi, giving a total flux for the primary star of ~1.
Since the secondary star in the default binary is identical to the primary star, we'd expect an out-of-eclipse flux of the binary to be ~2.
Can you write Python code for it?

targets: 
b.run_compute()

axs, artists = b.plot(dataset='lc01')
inputs: # remove the BigQuery dataset created for storing the recommendations and all of its tables
! bq rm -r -f -d $PROJECT:$DATASET

# remove the Cloud Storage bucket created and all of its tables
! gsutil rm -r gs://$BUCKET_NAME

# delete the created Dataproc cluster
! gcloud dataproc clusters delete $CLUSTER_NAME --region=$CLUSTER_REGION
Given the above code, generate some markdown instructions for it.

targets: 
Clean up
<a name="section-12"></a>
To clean up all Google Cloud resources used in this project, you can delete the Google Cloud
project you used for the tutorial.
Otherwise, you can delete the individual resources you created in this tutorial:
inputs: "Above, we created a new transform primitive that can be used with Deep Feature Synthesis by deriving a new primitive class using TransformPrimitive as a base and overriding get_function to return a function that calculates the feature. Additionally, we set the input data types that the primitive applies to and the return data type. Input and return data types are defined using a Woodwork ColumnSchema. A full guide on Woodwork logical types and semantic tags can be found in the Woodwork Understanding Types and Tags guide.
Similarly, we can make a new aggregation primitive using AggregationPrimitive."
Please write code following the instructions in jupyter notebook style.

targets: 
class Maximum(AggregationPrimitive):
    name = 'maximum'
    input_types = [ColumnSchema(semantic_tags={'numeric'})]
    return_type = ColumnSchema(semantic_tags={'numeric'})

    def get_function(self):
        def maximum(column):
            return max(column)

        return maximum
inputs: I am working on the file "Prepare analysis data.ipynb".
The first task is:
Same tweet belongs to multiple datasets
Can you write Python code for it?

targets: 
df_merged_meta.t_id.value_counts().head()

df_merged_meta[df_merged_meta.t_id == 700042121877835776][["topic_name"]]

df_merged_meta.t_id.value_counts()[df_merged_meta.t_id.value_counts() > 1]

df_merged_meta[df_merged_meta.t_id == 792354716521009152].T

df_merged_meta["is_controversial"] = df_merged_meta.topic_name.isin(CONTROVERSIAL_TOPICS)
df_merged_meta.is_controversial.value_counts()
inputs: Bias
Here we show that the MLE estimator is unbiased, i.e., that the expected value for the MLE estimate for a parameter is the true value. The expected value is the average over many (-> inf) observations of the same process.

targets: # Now repeat the whole analysis (from data generation on) a certain number of times

# I prepared a convenience function to do that. 
# This function regenerates some data from the same generative process
# used above, then it fits them and returns the list of MLE estimates
# for a (one for each iteration)
# Let's do it 1000 times

many_a_mle = like.generate_and_fit(data_generative_process, 1000)

# Now let's plot the MLEs for a

plt.plot(many_a_mle,'.')
plt.ylabel(r"a$_{MLE}$")
plt.xlabel("iteration")

# Plot the true value
plt.axhline(a_true, color='red',lw=2,linestyle='--')

# We can make an histogram of the MLE estimates
histogram = plt.hist(many_a_mle, 20)
plt.xlabel("a")
# plot the vertical like of the true value
plt.axvline(a_true, color='red', lw=2, linestyle='--', zorder=100)
inputs: # =============================
# change these (both in millis)
delay_stream, slow_emit = 0, 0
# =============================

import rx, threading, random, time
thread = threading.currentThread


def call_observer(obs):
    '''observer functions are invoked, blocking'''
    print_out(obs.__class__.__name__, hash(obs))
    for i in range(2):
        obs.on_next(1)
        if slow_emit:
            time.sleep(slow_emit/1000)
        obs.on_next(1)
        
stream = rx.Observable.create(call_observer).take(10)
if delay_stream:
    stream = stream.delay(delay_stream)

def print_out(*v):
    '''printout of current time, v, and current thread'''
    v_pretty = ' '.join([str(s) for s in v])
    print ('%.8f - %30s - %s\n' % (time.time(), v_pretty, thread().getName()))
    
    
d = stream.subscribe(lambda x: print_out('Observer 1', x))
d = stream.subscribe(lambda x: print_out('Observer 2', x))
Given the above code, generate some markdown instructions for it.

targets: 
Async Operations
It is useful to understand on a high level how RxPY handles asyncronity and when.
E.g. naively you might want to know, when notifying a value to a subscriber, what other subscribers are present.
This makes no sense to ask (I think in general in reactive programming) and it will be clear looking at an example.    
Consider timing and thread outputs in the following:
inputs: !mkdir frames
frame = img
frame_i = 0

h, w = frame.shape[:2]
s = 0.05 # scale coefficient

for i in xrange(1):
    frame = deepdream(net, frame)
    PIL.Image.fromarray(np.uint8(frame)).save("frames/%04d.jpg"%frame_i)
    frame = nd.affine_transform(frame, [1-s,1-s,1], [h*s/2,w*s/2,0], order=1)
    frame_i += 1
Given the above code, generate some markdown instructions for it.

targets: 
What if we feed the deepdream function its own output, after applying a little zoom to it? It turns out that this leads to an endless stream of impressions of the things that the network saw during training. Some patterns fire more often than others, suggestive of basins of attraction.
We will start the process from the same sky image as above, but after some iteration the original image becomes irrelevant; even random noise can be used as the starting point.
inputs: I am working on the file "Datasets/CLEU/.ipynb_checkpoints/CLEU-checkpoint.ipynb".
The first task is:
Limpieza de dataset para P0702: Restricción al paso de automóviles
Can you write Python code for it?

targets: 
# La columna 'Restricción del paso a automóviles' clasifica las manzanas
# que tienen alguna clase de restriccion al paso de automóviles.
P0702_raw = datasets['P0702']
list(P0702_raw['Restricción del paso a automóviles'].unique())

P0702_raw.head()

list(P0702_raw)

# Es conveniente crear un dataset con el total de manzanas por municipio
manzanas = P0702_raw[P0702_raw['Restricción del paso a automóviles'] == 'Total']['Total de manzanas']
manzanas.head()

# Para el parametro, hay que filtrar unicamente las columnas que contienen restricciones a la vialidad
P0702_1 = P0702_raw.loc[P0702_raw['Restricción del paso a automóviles'].isin([
    'Restricción del paso a automóviles en todas las vialidades',
    'Restricción del paso a automóviles en alguna vialidad',
])]
P0702_1.head()

# Una vez filtrados hay que agregar los datos por municipio. También vale la pena quitar las columnas de 
# restriccion al paso de peatones, pues no son de interes para este parametro.
P0702_1 = P0702_1['Total de manzanas'].groupby(level=0).sum().to_frame(name='Restricción del paso a automóviles')
P0702_1.head()

# Unimos ambos datasets para crear el dataset estandar del parámetro
P0702 = manzanas.to_frame().join(P0702_1)
print(len(P0702))
P0702.head()
inputs: $\alpha$-$\beta$ pruning with intelligent memoization takes 17 ms to compute the value of the state gStart for Tic-Tac-Toe. 
Hence, there is no big difference between intelligent memoization and naive memoization in this example.  However, if we implement
a non-trivial game like Connect Four, the situation changes.

targets: %%time
v = evaluate(gStart, maxValue, -1, 1)
v
inputs: I am working on the file "16_reinforcement_learning.ipynb".
The first task is:
Yeah... pretty bad. The neural network will have to learn to do better. First let's see if it is capable of learning the basic policy we used earlier: go left if the pole is tilting left, and go right if it is tilting right. The following code defines the same neural network but we add the target probabilities y, and the training operations (cross_entropy,  optimizer and training_op):
Can you write Python code for it?

targets: 
import tensorflow as tf

reset_graph()

n_inputs = 4
n_hidden = 4
n_outputs = 1

learning_rate = 0.01

initializer = tf.variance_scaling_initializer()

X = tf.placeholder(tf.float32, shape=[None, n_inputs])
y = tf.placeholder(tf.float32, shape=[None, n_outputs])

hidden = tf.layers.dense(X, n_hidden, activation=tf.nn.elu, kernel_initializer=initializer)
logits = tf.layers.dense(hidden, n_outputs)
outputs = tf.nn.sigmoid(logits) # probability of action 0 (left)
p_left_and_right = tf.concat(axis=1, values=[outputs, 1 - outputs])
action = tf.multinomial(tf.log(p_left_and_right), num_samples=1)

cross_entropy = tf.nn.sigmoid_cross_entropy_with_logits(labels=y, logits=logits)
optimizer = tf.train.AdamOptimizer(learning_rate)
training_op = optimizer.minimize(cross_entropy)

init = tf.global_variables_initializer()
saver = tf.train.Saver()
inputs: I am working on the file "8_Calibration/8_1_Calibration_Least_Squares_Problem.ipynb".
The first task is:
The images above contain the real part of the corrupted (green) and uncorrupted (blue)
visibilities as a function of timeslots for baseline 01, 02 and 12 respectively.
8.1.4 Levenberg-Marquardt (create_G_LM) <a id='cal:sec:LM'></a> <!--\label{cal:sec:LM}-->
We are now ready to use least squares to calibrate $\boldsymbol{\mathcal{D}}$ (see <cite data-cite='Yatawatta2012'>GPU accelerated nonlinear optimization in radio interferometric calibration</cite> &#10548;).
We first present a brief review of least squares minimization. Suppose we wish to fit a model $\mathbf{f}\left( \mathbf{m},\breve{\mathbf{g}}\right)$, where $\mathbf{m}$ and $\breve{\mathbf{g}}$ denote
the model input values and a vector of unknown variables respectively, to some data $\left{\mathbf{d}{i},\mathbf{m}{i}\right}$. The vector of unknown variables $\breve{\mathbf{g}}$ parametrize the model. A standard method for determining which parameter vector $\breve{\mathbf{g}}$ best fits the data is to minimize the sum of the squared residuals. This technique is referred to as least squares minimization. The residual vector is denoted by $\mathbf{r}(\mathbf{m},\mathbf{d},\breve{\mathbf{g}}) = \mathbf{d} - \mathbf{f}\left( \mathbf{m},\breve{\mathbf{g}}\right)$. The objective function (the function we wish to minimize) associated with least squares is: $\sum_i \mathbf{r}_i^2$. The function optimize.leastsq is scipy's built-in least squares solver and employs the Levenberg-Marquardt algorithm in the background. The Levenberg-Marquardt algorithm is discussed in more detail in $\S$ 2.11 &#10142;. To use optimize.leastsq one needs a function, here called err_func, that calculates the residual vector $\mathbf{r}$. An initial guess of the parameter vector $\breve{\mathbf{g}}$ is also required.
For calibration the above variables become:
<p class=conclusion>
  <font size=4> <b>Vectorizing</b></font>
  <br>
  <br>
&bull; $\mathbf{d} = [\textrm{vec}(\Re\{\boldsymbol{\mathcal{D}}\}),\textrm{vec}(\Im\{\boldsymbol{\mathcal{D}}\})]$ <br><br>
&bull; $\mathbf{m} =...
Can you write Python code for it?

targets: 
'''Unpolarized direction independent calibration entails finding the G that minimizes ||R-GMG^H||. 
This function evaluates D-GMG^H.
g is a vector containing the real and imaginary components of the antenna gains.
d is a vector containing a vecotrized R (observed visibilities), real and imaginary.
m is a vector containing a vecotrized M (predicted), real and imaginary.
r is a vector containing the residuals.
''' 
def err_func(g,d,m):
    Nm = len(d)/2
    N = len(g)/2
    G = np.diag(g[0:N]+1j*g[N:])
    D = np.reshape(d[0:Nm],(N,N))+np.reshape(d[Nm:],(N,N))*1j #matrization
    M = np.reshape(m[0:Nm],(N,N))+np.reshape(m[Nm:],(N,N))*1j
    T = np.dot(G,M)
    T = np.dot(T,G.conj())
    R = D - T
    r_r = np.ravel(R.real) #vectorization
    r_i = np.ravel(R.imag)
    r = np.hstack([r_r,r_i])
    return r
inputs: Backtesting on real market data
This is all very interesting but not very applied. We next demonstrate how you can create a simple algorithm in zipline -- the open-source backtester that powers Quantopian -- to test this optimization on actual historical stock data.
First, lets load in some historical data using Quantopian's data (if we are running in the Quantopian Research Platform, or the load_bars_from_yahoo() function from zipline.

targets: from zipline.utils.factory import load_bars_from_yahoo
end = pd.Timestamp.utcnow()
start = end - 2500 * pd.tseries.offsets.BDay()

data = load_bars_from_yahoo(stocks=['IBM', 'GLD', 'XOM', 'AAPL', 
                                    'MSFT', 'TLT', 'SHY'],
                            start=start, end=end)

data.loc[:, :, 'price'].plot(figsize=(8,5))
plt.ylabel('price in $')
inputs: I am working on the file "Example/LCCM code example.ipynb".
The first task is:
Modify variables, create dummy variables,and scale variables so the estimated coefficients are of similar magnitudes.
Can you write Python code for it?

targets: 
# Create a dummy variable indicating whether a certain individual in the sample is male or female
df['male'] = (df['Gender']==1).astype(int)

# Create a categorical variable to indicate the income level of the household corresponding to each individual
# in the sample. We will define three categories: low income (income category  < 6) , medium income
# (6 <= income category < 12) and high income (12 <= income category < 16). 
df['low_income'] = (df['HHIncome']<6).astype(int)
df['medium_income'] = ((df['HHIncome']>=6)&(df['HHIncome']<12)).astype(int)
df['high_income'] = ((df['HHIncome']>=12)&(df['HHIncome']<16)).astype(int)
inputs: "Agora só temos um bloco-if com  duas instruções. O último print é executado porque a condição só controla a execução das duas intruções que estão no bloco."
Please write code following the instructions in jupyter notebook style.

targets: 
x=1

if x>2:
    print(10*'.^.')
print('O número é maior que 2.')
print(10*'\'v\'')
inputs: <p style="text-align: right; direction: rtl; float: right; clear: both;">
    ורק בשביל הכיף, נבנה קוד שיאפשר לנו להשתמש במפה כדי להצפין מסרים:
</p>

targets: def encrypt(message, encryption_map):
    encrypted = ''
    for char in message.lower():
        # If we can't find the character, assume it is not encrypted.
        encrypted = encrypted + encryption_map.get(char, char)
    return encrypted

encryption_map = get_ceaser_map()
encrypted_message = encrypt('This is the encrypted message!', encryption_map)
print(encrypted_message)

def create_decryption_map(encryption_map):
    """Actually just flip the keys and the values of the dictionary"""
    decryption_map = {}
    for key, value in encryption_map.items():
        decryption_map[value] = key
    return decryption_map

def decrypt(message, decryption_map):
    decrypted = ''
    for char in message.lower():
        # If we can't find the character, assume it is not encrypted.
        decrypted = decrypted + decryption_map.get(char, char)
    return decrypted

decryption_map = create_decryption_map(encryption_map)
decrypted_message = decrypt(encrypted_message, decryption_map)
print(decrypted_message)
inputs: #truncate to top 5
for tup in top_nz_titles[:5]:
    print 'Title :', tup[0]
    print 'Title :', tup[1]
    try:
        poster = Image(nz_data[quote(tup[0])]['Poster'])
        display(poster)
    except:
        print '''
        |
        | No Poster
        |       
        '''

Given the above code, generate some markdown instructions for it.

targets: 
We could potentially display this visually.
We need to do some trickery as our dictionaries have quoted title names as the keys, so 'The%20Pink%20Panther' instead of 'The Pink Panther', so we need to 'quote' our moviename to get an image.
Top NZ Title Visually
inputs: Create a test value of a directional text

targets: dir_str = 'SOUTH SOUTH EAST'
print(dir_str)
inputs: I am working on the file "Funciones Booleanas/Boole.ipynb".
The first task is:
La formas normales conjuntiva y disyuntivas las podemos calcular como sigue
Can you write Python code for it?

targets: 
to_cnf(p)

to_dnf(p)
inputs: "'Top 10' Contratos de 2017
Agora com a lista de todos os contratos do ano, vamos montar uma tabela e ordenar os dados pelo Valor Principal do Contrato. No Manual da API, aprende-se que esse campo 'valPrincipal'significa o "Valor do contrato sem ocorrência de reajustamentos, ou aditamentos"."
Please write code following the instructions in jupyter notebook style.

targets: 
top10 = df_contratos_empenhados[['txtDescricaoModalidade',
                                 'txtObjetoContrato',
                                 'txtRazaoSocial',
                                 'numCpfCnpj',
                                 'valPrincipal']].sort_values(['valPrincipal'], ascending=False)[:10]

top10
inputs: 2.1 Using plt.plot to plot lines
Before we get to plotting the percentage difference in stock prices. Let's get a feel of how to plot lines in matplotlib. Like a scatterplot, to plot lines we: 
  * Pass an array of x-coordinates and y-coordinates to the first and second arguments of plt.plot. 
  * If we do not pass any other arguments, plt.plot thinks that you want to plot a line graph.

targets: # Use this space to do the following: Plot a simple line graph using plt.plot. Use the data below.

xcoord = [1,2,3,4]
ycoord = [5, -3, 9, -1]

# Answer: