Muennighoff/xP3x-sample huggingface dataset:
inputs: 2.Schreibe eine Funktion, die alle Elemente einer Liste, addiert. Es ist verboten mit "sum" zu arbeiten.

targets: lst = [12, 45, 373, 1028]

def addtntor(mylist):
    total=0
    for elem in mylist:
        total+=elem
    return total

addtntor(lst)
inputs: I am working on the file "2.2/tutorials/settings.ipynb".
The first task is:
dict_set_all
dict_set_all is a BooleanParameter (defaults to False) that controls whether attempting to set a value to a ParameterSet via dictionary access will set all the values in that ParameterSet (if True) or raise an error (if False)
Can you write Python code for it?

targets: 
b['dict_set_all@setting']

b['teff@component']
inputs: # Define file name for binned data. Note "{}" prepared for string formatting
out_bin = '/home/jovyan/DATA/HDF5/K562_B-bulk.binned_{}.hdf5'

res_kb = [100, 20] # Several resolutions in Kb

for res in res_kb:
    print(res)
    
    outmap = out_bin.format(str(res)+'kb') # String formatting

    fragments.saveHeatmap(outmap, res*1000) # Save heatmap

del fragments # delete unwanted object
Given the above code, generate some markdown instructions for it.

targets: 
<a id="binning"></a>
3. Data binning
Go top
The previous analysis involved interactions of restriction fragments, now we would like to work with interactions of genomic bins.
inputs: 7. Optimizer
We implement the optimizer to perform the updates of the weights according to the certain scheme.

targets: class Optimizer(object):
    '''This is a basic class. 
    All other optimizers will inherit it
    '''
    def __init__(self, model, lr=0.01, weight_decay=0.0):
        self.model = model
        self.lr = lr
        self.weight_decay = weight_decay
        
    def update_params(self):
        pass


class SGD(Optimizer):
    '''Stochastic gradient descent optimizer
    https://en.wikipedia.org/wiki/Stochastic_gradient_descent
    '''
        
    def update_params(self):
        weights = self.model.get_params()
        grads = self.model.get_params_gradients()
        for w, dw in zip(weights, grads):
            update = self.lr * dw + self.weight_decay * w
            # it writes the result to the previous variable instead of copying
            np.subtract(w, update, out=w)
inputs: I am working on the file "Miscellaneous/Gesture-Phase-Detection.ipynb".
The first task is:

Can you write Python code for it?

targets: 
pca = PCA(n_components=2)
X = pca.fit_transform(X)

c = []
from matplotlib.pyplot import cm 
n=6
color=iter(cm.rainbow(np.linspace(0,1,n)))
for i in range(n):
    c.append(next(color))

n = 5
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(8,6))

km = KMeans(n_clusters= n , random_state=0)
y_km = km.fit_predict(X)

for i in range(n):
    ax1.scatter(X[y_km==i,0], X[y_km==i,1], c=c[i], marker='o', s=40, label='cluster{}'.format(i))
ax1.set_title('K-means clustering')

ac = AgglomerativeClustering(n_clusters=n, affinity='euclidean', linkage='complete')
y_ac = ac.fit_predict(X)
for i in range(n):
    ax2.scatter(X[y_ac==i,0], X[y_ac==i,1], c=c[i], marker='o', s=40, label='cluster{}'.format(i))
ax2.set_title('Agglomerative clustering')

# Put a legend below current axis
plt.legend()
    
plt.tight_layout()
#plt.savefig('./figures/kmeans_and_ac.png', dpi=300)
plt.show()
inputs: "Choose Word
Implement the pick_word() function to select the next word using probabilities."
Please write code following the instructions in jupyter notebook style.

targets: 
def pick_word(probabilities, int_to_vocab):
    """
    Pick the next word in the generated text
    :param probabilities: Probabilites of the next word
    :param int_to_vocab: Dictionary of word ids as the keys and words as the values
    :return: String of the predicted word
    """

    return int_to_vocab[np.random.choice(len(int_to_vocab), None, False, probabilities)]


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_pick_word(pick_word)
inputs: # # From pytorch source:
# def RNNCell(input, hidden, w_ih, w_hh, b_ih, b_hh):
#     return F.tanh(F.linear(input, w_ih, b_ih) + F.linear(hidden, w_hh, b_hh))

class CharSeqStatefulRNN2(nn.Module):
    def __init__(self, vocab_size, n_fac, bs):
        super().__init__()
        self.vocab_size = vocab_size
        self.e = nn.Embedding(vocab_size, n_fac)
        self.rnn = nn.RNNCell(n_fac, n_hidden)
        self.l_out = nn.Linear(n_hidden, vocab_size)
        self.init_hidden(bs)
        
    def forward(self, cs):
        bs = cs[0].size(0)
        if self.h.size(1) != bs: self.init_hidden(bs)
        outp = []
        o = self.h
        for c in cs:
            o = self.rnn(self.e(c), o)
            outp.append(o)
        outp = self.l_out(torch.stack(outp))
        self.h = repackage_var(o)
        return F.log_softmax(outp, dim=-1).view(-1, self.vocab_size)
    
    def init_hidden(self, bs): self.h = V(torch.zeros(1, bs, n_hidden))

m = CharSeqStatefulRNN2(mdata.nt, n_fac, 512).cuda()
opt = optim.Adam(m.parameters(), 1e-3)

fit(m, mdata, 4, opt, F.nll_loss)
Given the above code, generate some markdown instructions for it.

targets: 
6.3 RNN loop
inputs: # Your code here
Given the above code, generate some markdown instructions for it.

targets: 
Now it's your turn. Can you figure out how to load the rest of the data files into datalist automatically?
Hint: The names of the remaining files are are c2.dat, c3.dat, c4.dat, c5.dat, and c6.dat. What is the only thing that changes among these names? Can you think of a way to generate this part separately and combine it with the rest of the string?
inputs: I am working on the file "2.3/tutorials/distribution_propagation.ipynb".
The first task is:
But we can also use parameters to propagate the distributions through the constraints linking parameters together.  For example, since we have distributions on sma and incl, including asini should combine the two distributions according to the constraint and showing the resulting correlations.
Can you write Python code for it?

targets: 
_ = b.plot_distribution_collection(distribution='mydist', 
                                   parameters=['sma@binary', 'q@binary', 'asini@binary'],
                                   show=True)
inputs: I am working on the file "Generator.ipynb".
The first task is:
Lever 5: PV system cost adjustment
This lever reflects the role of PV system costs on electrification results. All PV based systems will be adjusted by a factor to simulate a higher or lower cost of PV systems (compared to the baseline values entered below). A value lower than 1 means lower investment costs for PV systems compared to baseline, and a value larger than 1 means higher investment cost for PV systems compared to baseline. E.g. 0.75 would mean a cost that is 25% lower compared to baseline costs.
Can you write Python code for it?

targets: 
pv_adjustment_factor = 1
inputs: %%writefile babyweight/trainer/model.py
import datetime
import os
import shutil
import numpy as np
import tensorflow as tf
import hypertune

# Determine CSV, label, and key columns
CSV_COLUMNS = ["weight_pounds",
               "is_male",
               "mother_age",
               "plurality",
               "gestation_weeks"]
LABEL_COLUMN = "weight_pounds"

# Set default values for each CSV column.
# Treat is_male and plurality as strings.
DEFAULTS = [[0.0], ["null"], [0.0], ["null"], [0.0]]


def features_and_labels(row_data):
    """Splits features and labels from feature dictionary.

    Args:
        row_data: Dictionary of CSV column names and tensor values.
    Returns:
        Dictionary of feature tensors and label tensor.
    """
    label = row_data.pop(LABEL_COLUMN)

    return row_data, label  # features, label


def load_dataset(pattern, batch_size=1, mode='eval'):
    """Loads dataset using the tf.data API from CSV files.

    Args:
        pattern: str, file pattern to glob into list of files.
        batch_size: int, the number of examples per batch.
        mode: 'train' | 'eval' to determine if training or evaluating.
    Returns:
        `Dataset` object.
    """
    print("mode = {}".format(mode))
    # Make a CSV dataset
    dataset = tf.data.experimental.make_csv_dataset(
        file_pattern=pattern,
        batch_size=batch_size,
        column_names=CSV_COLUMNS,
        column_defaults=DEFAULTS)

    # Map dataset to features and label
    dataset = dataset.map(map_func=features_and_labels)  # features, label

    # Shuffle and repeat for training
    if mode == 'train':
        dataset = dataset.shuffle(buffer_size=1000).repeat()

    # Take advantage of multi-threading; 1=AUTOTUNE
    dataset = dataset.prefetch(buffer_size=1)

    return dataset


def create_input_layers():
    """Creates dictionary of input layers for each feature.

    Returns:
        Dictionary of `tf.Keras.layers.Input` layers for each feature.
    """
    deep_inputs = {
        colname:...
Given the above code, generate some markdown instructions for it.

targets: 
In the same way we can write to the file model.py the model that we developed in the previous notebooks. 
Create trainer module's model.py to hold Keras model code.
To create our model.py, we'll use the code we wrote for the Wide & Deep model. Look back at your 9_keras_wide_and_deep_babyweight notebook and copy/paste the necessary code from that notebook into its place in the cell below.
inputs: Appendix: Sanity Check Code
I wanted to make sure the resizing didn't cause significant distortion for images which werent' square already. The following code helped with this sanity check.

targets: from scipy import misc
path = testDataPath + "/JayaBhaduri_42.jpg"
img = misc.imread(path)
img = misc.imresize(img, (150, 150))

misc.imsave("test1000.jpg", img)
inputs: "By default, cross_val_score will use StratifiedKFold for classification, which ensures that the class proportions in the dataset are reflected in each fold. If you have a binary classification dataset with 90% of data point belonging to class 0, that would mean that in each fold, 90% of datapoints would belong to class 0.
If you would just use KFold cross-validation, it is likely that you would generate a split that only contains class 0.
It is generally a good idea to use StratifiedKFold whenever you do classification.
StratifiedKFold would also remove our need to shuffle iris.
Let's see what kinds of folds it generates on the unshuffled iris dataset.
Each cross-validation class is a generator of sets of training and test indices:"
Please write code following the instructions in jupyter notebook style.

targets: 
cv = StratifiedKFold(n_splits=5)
for train, test in cv.split(iris.data, iris.target):
    print(test)
inputs: <h3>We can drill down into complaints by Agency by borough</h3>

targets: agency_borough = data.groupby(['Agency', 'Borough']).size().unstack()

agency_borough
inputs: I am working on the file "Nuclear/PoissonML.ipynb".
The first task is:
and some boundaries for the range of parameters:
Can you write Python code for it?

targets: 
bnds = ((-np.infty,np.infty), (-np.infty,np.infty), (0.1,np.infty), (-10., 10.))
inputs: Build Train/Test Set

targets: # build label dataframe
label_df = pd.DataFrame(y.reset_index(drop=True))\
    .reset_index()
label_df.columns = ['example_id', 'label']

# split into training and test 
train_id, test_id = train_test_split(label_df.example_id, random_state=43, test_size=0.2)
train_id = pd.DataFrame(train_id)
test_id  = pd.DataFrame(test_id)

data_train = master_df.merge(train_id, on='example_id')
data_test = master_df.merge(test_id, on='example_id')
print float(data_train.shape[0])/(data_train.shape[0] + data_test.shape[0])

label_train = label_df.merge(train_id, on='example_id')
label_test = label_df.merge(test_id, on='example_id')
inputs: "(IMPLEMENTATION) Train and Validate the Model
Train and validate your model in the code cell below.  Save the final model parameters at filepath 'model_scratch.pt'."
Please write code following the instructions in jupyter notebook style.

targets: 
def train(n_epochs, loaders, model, optimizer, criterion, use_cuda, save_path):
    """returns trained model"""
    # initialize tracker for minimum validation loss
    valid_loss_min = np.Inf 
    
    for epoch in range(1, n_epochs+1):
        # initialize variables to monitor training and validation loss
        train_loss = 0.0
        valid_loss = 0.0
        
        ###################
        # train the model #
        ###################
        model.train()
        for batch_idx, (data, target) in enumerate(loaders['train']):
            # move to GPU
            if use_cuda:
                data, target = data.cuda(), target.cuda()
            ## find the loss and update the model parameters accordingly
            ## record the average training loss, using something like
            ## train_loss = train_loss + ((1 / (batch_idx + 1)) * (loss.data - train_loss))
            
        ######################    
        # validate the model #
        ######################
        model.eval()
        for batch_idx, (data, target) in enumerate(loaders['valid']):
            # move to GPU
            if use_cuda:
                data, target = data.cuda(), target.cuda()
            ## update the average validation loss

            
        # print training/validation statistics 
        print('Epoch: {} \tTraining Loss: {:.6f} \tValidation Loss: {:.6f}'.format(
            epoch, 
            train_loss,
            valid_loss
            ))
        
        ## TODO: save the model if validation loss has decreased
            
    # return trained model
    return model


# train the model
model_scratch = train(100, loaders_scratch, model_scratch, optimizer_scratch, 
                      criterion_scratch, use_cuda, 'model_scratch.pt')

# load the model that got the best validation accuracy
model_scratch.load_state_dict(torch.load('model_scratch.pt'))
inputs: <!--Empty Space for separating topics-->

<h2 id="Makeup_Data">Make Some Data</h2>

Set random seed:

targets: # Set random seed

torch.manual_seed(1)
inputs: I am working on the file "BlogPosts/Decorator_mixing/decorator_mixing.ipynb".
The first task is:
Aha! So the __isabstractmethod__ hasn't disappeared at all! It's just been wrapped by @cachedproperty into the func attribute of the method. But that's not where Python 3 expects it to be. That should be an easy fix.
A Fix!
Can you write Python code for it?

targets: 
class cachedproperty:

    def __init__(self, func):
        self.__doc__ = getattr(func, '__doc__')
        self.__isabstractmethod__ = func.__isabstractmethod__ # The fix!
        self.func = func

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        value = obj.__dict__[self.func.__name__] = self.func(obj)
        return value

    def __repr__(self):
        cn = self.__class__.__name__
        return '<%s func=%s>' % (cn, self.func)
    
    
class AbstractTimeProvider(ABC):
    
    @cachedproperty
    @abstractmethod
    def time(self):
        pass

try:
    AbstractTimeProvider()
except TypeError as e:
    print('TypeError:', e)
inputs: Configure model

targets: model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

modelcheckpoint = ModelCheckpoint(filepath=output_dir+"/weights.{epoch:02d}.hdf5")

if not os.path.exists(output_dir):
    os.makedirs(output_dir)
inputs: # Tuple
# Tuples are read-only collections of items.
a = (1, 2, 3) 
print(a)



# List
# Lists use the square bracket notation and can be index using array notation.
mylist = [1, 2, 3] 
print("Zeroth Value: %d" % mylist[0]) 
mylist.append(4) 
print("List Length: %d" % len(mylist)) 
for value in mylist: 
    print(value)


# Dictionary
# Dictionaries are mappings of names to values, like key-value pairs. Note the use of the curly bracket and colon notations when deﬁning the dictionary.
mydict = {'a': 1, 'b': 2, 'c': 3} 
print("A value: %d" % mydict['a']) 
mydict['a'] = 11 
print("A value: %d" % mydict['a']) 
print("Keys: %s" % mydict.keys()) 
print("Values: %s" % mydict.values()) 
for key in mydict.keys(): 
    print(mydict[key])

Given the above code, generate some markdown instructions for it.

targets: 
Data Structures
There are three data structures in Python that you will ﬁnd the most used and useful. They are tuples, lists and dictionaries.
inputs: # Debug blocker output
dbg = em.debug_blocker(C1, A, B, output_size=200)

# Display first few tuple pairs from the debug_blocker's output
dbg.head()
Given the above code, generate some markdown instructions for it.

targets: 
Debug blocker output
The number of tuple pairs considered for matching is reduced to 10165 (from 176423), but we would want to make sure that the blocker did not drop any potential matches. We could debug the blocker output in py_entitymatching as follows:
inputs: "<br>
Cuando tenemos un caso como este es más seguro utilizar los atributos iloc o loc según sea el caso."
Please write code following the instructions in jupyter notebook style.

targets: 
s[0] #Esta instrucción no llamará s.iloc[0] como esperaríamos y va a generar un error

s.iloc[0]

s.loc[99]
inputs: # Create placeholders
x_train_tensor = tf.placeholder(tf.float32, shape=(None, 784))
x_test_tensor = tf.placeholder(tf.float32, shape=[784])
Given the above code, generate some markdown instructions for it.

targets: 
Step 1: Create placeholders to hold the images. 

Using None for a dimension in shape means it can be any number.
inputs: I am working on the file "01-Intro-to-Computational-Tools-Python.ipynb".
The first task is:
Lists’ values can be replaced by assigning to specific indices.
Can you write Python code for it?

targets: 
country_list[0] = "Iran"
print('Country List is now:', country_list)
inputs: I am working on the file "Data Analyst Nanodegree Project 1.ipynb".
The first task is:
What is our independent variable? What is our dependent variable?
Independent variable: the text color condition (congruent or incongruent).
Dependent variable: the reaction time (the time it takes to name the ink color).
What is an appropriate set of hypotheses for this task? What kind of statistical test do you expect to perform? Justify your choices.
Null hypothesis: the reaction time for the congruent and incongruent conditions is the same.
$$\mu(\text{congruent}) = \mu(\text{incongruent})$$
Alternative hypothesis: the reaction time for the two conditons are different.
$$\mu(\text{congruent})  \neq \mu(\text{incongruent})$$
It's hard to imagine the reverse being true; that the reaction time would be faster for the incongruent condition. On the other hand, it is not a priori completely implausible that the  reaction times would be the same.
We will use the paired t-test to test whether the mean reaction times under the two conditions are equal or not. The t-test allows finding out whether the means of two sets of data are different.
We believe the use of a paired t-test is appropriate since the following conditions are met (see [1], Analysis checklist: Paired t test):
1. Are the differences distributed according to a Gaussian distribution?
As will be shown graphically in a later section, the differences are distributed close to normally.
2. Was the pairing effective?
The pairing was part of the design of the experiment; each pair represents trials from a single participant. This is an effective pairing method.
3. Are the pairs independent?
Each pair corresponds to trials by a different participant; they are independent.
4. Are you comparing exactly two groups?
Yes.
5. If you chose a one-tail P value, did you predict correctly?
We did not choose a one-tail P value (although we could have).
6. Do you care about differences or ratios?
We expect the control values and differences to be of the same order of magnitude, so differences should be fine.
Report some descriptive statistics regarding this dataset....
Can you write Python code for it?

targets: 
IQR_congruent = df['Congruent'].quantile(0.75) - df['Congruent'].quantile(0.25)
IQR_incongruent = df['Incongruent'].quantile(0.75) - df['Incongruent'].quantile(0.25)
print(df['Congruent'].median(), df['Incongruent'].median(), IQR_congruent, IQR_incongruent)
inputs: I am working on the file "FDMS/TME2/TME2FDMS_Florian_Toque.ipynb".
The first task is:
We observe that, as expected, the more we take L1 into account the less features are used.
Can you write Python code for it?

targets: 
eps=0.00001
la = []
cross_sc = []
used_features = []

for lamb in np.arange(0.05,1.05,0.05):
    theta = copy.deepcopy(X[0])
    gd = ElasticNet(alpha = 0.2, l1_ratio=lamb)
    nbIterations = 4000
    gd.fit(X,y)
    scoresSvm = cross_validation.cross_val_score(gd, X, y, cv=5,scoring=scorer)
    print("Lamda: %s | Cross val mean: %.03f | Features: %d"%(str(lamb).ljust(5),np.mean(scoresSvm),np.count_nonzero(gd.coef_)))
    #print("Lamda: %.02f | Cross val mean: %.02f | Features: %d"%(lamb,gd.score(X,y),gd.used_features))
    cross_sc.append(np.mean(scoresSvm))
    la.append(lamb)
    used_features.append(np.count_nonzero(gd.coef_))

fig, ax1 = plt.subplots()

ax2 = ax1.twinx()
ax1.plot(la, cross_sc, '#FF9900')
ax2.plot(la, used_features, '#9933FF')

ax1.set_xlabel('L1 L2 ratio')
ax1.set_ylabel('Cross val score', color='#FF9900')
ax2.set_ylabel('Nb features used', color='#9933FF')

ax1.yaxis.grid(False)
ax2.grid(False)
plt.show()
inputs: "Bounds Testing
UECMResults expose the bounds test of Pesaran, Shin, and Smith (2001).  This test facilitates testing whether there is a level relationship between a set of variables without identifying which variables are I(1). This test provides two sets of critical and p-values.  If the test statistic is below the critical value for the lower bound, then there appears to be no levels relationship irrespective of the order or integration in the $X$ variables.  If it is above the upper bound, then there appears to be a levels relationship again, irrespective of the order of integration of the $X$ variables. There are 5 cases covered in the paper that include different combinations of deterministic regressors in the model or the test.
$$\Delta Y_{t}=\delta_{0} + \delta_{1}t + Z_{t-1}\beta + \sum_{j=0}^{P}\Delta X_{t-j}\Gamma + \epsilon_{t}$$
where $Z_{t-1}$ includes both $Y_{t-1}$ and $X_{t-1}$.
The cases determine which deterministic terms are included in the model and which are tested as part of the test.

No deterministic terms
Constant included in both the model and the test
Constant included in the model but not in the test
Constant and trend included in the model, only trend included in the test
Constant and trend included in the model, neither included in the test

Here we run the test on the Danish money demand data set. Here we see the test statistic is above the 95% critical value for both the lower and upper.
Pesaran, M. H., Shin, Y., & Smith, R. J. (2001). Bounds testing approaches to the analysis of level relationships. Journal of applied econometrics, 16(3), 289-326."
Please write code following the instructions in jupyter notebook style.

targets: 
ecm = UECM(data.lrm, 3, data[["lry", "ibo", "ide"]], 3, trend="c")
ecm_fit = ecm.fit()
bounds_test = ecm_fit.bounds_test(case=4)
bounds_test

bounds_test.crit_vals
inputs: Build a trainable blockwise lower-triangular linear operator. We'll apply it to the standard Normal distribution to implement a (trainable) blockwise matrix transformation and induce the correlation structure of the posterior.
Within the blockwise linear operator, a trainable full-matrix block represents full covariance between two components of the posterior, while a block of zeros (or None) expresses independence. Blocks on the diagonal are either lower-triangular or diagonal matrices, so that the entire block structure represents a lower-triangular matrix.
Applying this bijector to the base distribution results in a multivariate Normal distribution with mean 0 and (Cholesky-factored) covariance equal to the lower-triangular block matrix.

targets: operators = (
    (tf.linalg.LinearOperatorDiag,),  # Variance of uranium weight (scalar).
    (tf.linalg.LinearOperatorFullMatrix,  # Covariance between uranium and floor-by-county weights.
     tf.linalg.LinearOperatorDiag),  # Variance of floor-by-county weight (scalar).
    (None,  # Independence between uranium weight and county effects.
     None,  #  Independence between floor-by-county and county effects.
     tf.linalg.LinearOperatorDiag)  # Independence among the 85 county effects.
    )

block_tril_linop = (
    tfp.experimental.vi.util.build_trainable_linear_operator_block(
        operators, flat_event_size))
scale_bijector = tfb.ScaleMatvecLinearOperatorBlock(block_tril_linop)
inputs: # gv returns both gradients and values of original function
grads, value = gv(x)

print('Gradients:')
for g, var_name in zip(grads, module_vars.keys()):
    print(g, ' w.r.t. ', var_name)
print()
print('Value: ', value)
Given the above code, generate some markdown instructions for it.

targets: 
gv is a module which returns the gradients of loss_fn and the values of loss_fn for the given input:
inputs: "Activations of Conv 2"
Please write code following the instructions in jupyter notebook style.

targets: 
# Conv 2
f, axes = plt.subplots(2, 4, figsize=(16, 8))

for i in range(2):
    for j in range(4):
        ax = axes[i, j]
        ax.imshow(activations['conv_2'][i, j], cmap=plt.cm.gray_r)
        ax.set_xticks([])
        ax.set_yticks([])
        ax.set_title('Channel {}'.format(j + 1))

plt.show()
inputs: "Ajout de contraintes: super(type, obj_ou_type) [TODO]
Une autre syntaxe peut être utilisée pour rendre un peut plus explicite la classe de base à utiliser pour l'appel de la fonction:"
Please write code following the instructions in jupyter notebook style.

targets: 
class A:
    def bonjour(self):
        print("Bonjour de la part de A.")

class B(A):
    def bonjour(self):
        print("Bonjour de la part de B.")

class C(B):
    def bonjour(self):
        print("Bonjour de la part de C.")
        super(C, self).bonjour()

c = C()
c.bonjour()
inputs: "Apply StringIndex to index the entity_id."
Please write code following the instructions in jupyter notebook style.

targets: 
if "id" in documents_entities.columns:
    documents_entities = documents_entities.drop("id")

documents_entities =  StringIndexer(inputCol="entity_id", outputCol="id")\
.fit(documents_entities)\
.transform(documents_entities)

documents_entities.show()

documents_entities_n = documents_entities\
.withColumn("pair", F.struct("id", "confidence_level"))\
.groupBy("document_id")\
.agg(F.collect_list("pair").alias("entities"))

documents_entities_n.printSchema()
inputs: I am working on the file "Generating_permutations_with_Python.ipynb".
The first task is:
However, IPython's %timeit function warns us that itertools.permutation could use caching, and that could bias the result.
One easy way to remove any caching is to test on different input lists, and that can be done, for instance, with random lists.
Can you write Python code for it?

targets: 
from numpy.random import choice

def random_list_of_size_n(n=5, N=1000):
    return list(choice(list(range(1, N + 1)), size=n, replace=False))

random_list_of_size_n(5)

%timeit len(list(itertools_permutations(random_list_of_size_n(5))))
%timeit len(list(third_permutations(random_list_of_size_n(5))))

%timeit len(list(itertools_permutations(random_list_of_size_n(6))))
%timeit len(list(third_permutations(random_list_of_size_n(6))))
inputs: #NBVAL_IGNORE_OUTPUT
op(time=time_range.num-1, dt=dt)
Given the above code, generate some markdown instructions for it.

targets: 
And now execute the operator:
inputs: %matplotlib inline
import pandas as pd
import seaborn as sb
from sklearn.ensemble import RandomForestClassifier
from sklearn.cross_validation import train_test_split
from sklearn.cross_validation import cross_val_score

# We can jump directly to working with the clean data because we saved our cleaned data set
iris_data_clean = pd.read_csv('iris-data-clean.csv')

# Testing our data: Our analysis will stop here if any of these assertions are wrong

# We know that we should only have three classes
assert len(iris_data_clean['class'].unique()) == 3

# We know that sepal lengths for 'Iris-versicolor' should never be below 2.5 cm
assert iris_data_clean.loc[iris_data_clean['class'] == 'Iris-versicolor', 'sepal_length_cm'].min() >= 2.5

# We know that our data set should have no missing measurements
assert len(iris_data_clean.loc[(iris_data_clean['sepal_length_cm'].isnull()) |
                               (iris_data_clean['sepal_width_cm'].isnull()) |
                               (iris_data_clean['petal_length_cm'].isnull()) |
                               (iris_data_clean['petal_width_cm'].isnull())]) == 0

all_inputs = iris_data_clean[['sepal_length_cm', 'sepal_width_cm',
                             'petal_length_cm', 'petal_width_cm']].values

all_classes = iris_data_clean['class'].values

# This is the classifier that came out of Grid Search
random_forest_classifier = RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',
                                max_depth=None, max_features=3, max_leaf_nodes=None,
                                min_samples_leaf=1, min_samples_split=2,
                                min_weight_fraction_leaf=0.0, n_estimators=5, n_jobs=1,
                                oob_score=False, random_state=None, verbose=0, warm_start=True)

# All that's left to do now is plot the cross-validation scores
rf_classifier_scores = cross_val_score(random_forest_classifier, all_inputs, all_classes, cv=10)
sb.boxplot(rf_classifier_scores)
sb.stripplot(rf_classifier_scores,...
Given the above code, generate some markdown instructions for it.

targets: 
Finally, let's extract the core of our work from Steps 1-5 and turn it into a single pipeline.
inputs: "Answer: 

The optimal max_depth for my model is 6, which is close to the expectation that max_depth = 5. The difference may be caused by setting max_features=sqrt as a result of GridSearchCV. 

Question 11
With your parameter-tuned model, what is the best selling price for your client's home? How does this selling price compare to the basic statistics you calculated on the dataset?  
Hint:  Run the code block below to have your parameter-tuned model make a prediction on the client's home."
Please write code following the instructions in jupyter notebook style.

targets: 
sale_price = reg.predict(CLIENT_FEATURES)
print "Predicted value of client's home: {0:.3f}".format(sale_price[0])
print("Compared to mean: %.3f\nCompared to median: %.3f\nStd of boston housing price data: %.3f" 
      % ((sale_price[0]-mean_price)/mean_price,
         (sale_price[0]-median_price)/median_price,
          std_dev))
inputs: "<span id="slip_plat_prod">Choose Platform and Product &#9652;</span>"
Please write code following the instructions in jupyter notebook style.

targets: 
platform = 'LANDSAT_8'
product = 'ls8_usgs_sr_scene'
collection = 'c1'
level = 'l2'
inputs: I am working on the file "3_Plotting_Categorical_Data.ipynb".
The first task is:
You can see that though the category 'Technology' has better sales numbers than others, it is also the one where the most loss making transactions happen. You can drill further down into this.
Can you write Python code for it?

targets: 
# adjust figure size
plt.figure(figsize=(10, 8))

# subplot 1: Sales
plt.subplot(1, 2, 1)
sns.boxplot(x='Product_Category', y='Sales', data=df)
plt.title("Sales")
plt.yscale('log')

# subplot 2: Profit
plt.subplot(1, 2, 2)
sns.boxplot(x='Product_Category', y='Profit', data=df)
plt.title("Profit")

plt.show()
inputs: I am working on the file "2.3/examples/extinction_wd_subdwarf.ipynb".
The first task is:
As always, let's do imports and initialize a logger and a new bundle.
Can you write Python code for it?

targets: 
import matplotlib
matplotlib.rcParams['text.usetex'] = True
matplotlib.rcParams['pdf.fonttype'] = 42
matplotlib.rcParams['ps.fonttype'] = 42
matplotlib.rcParams['mathtext.fontset'] = 'stix'
matplotlib.rcParams['font.family'] = 'STIXGeneral'
from matplotlib import gridspec

%matplotlib inline

import phoebe
from phoebe import u # units
import numpy as np
import matplotlib.pyplot as plt

logger = phoebe.logger('error')

b = phoebe.default_binary()
inputs: I am working on the file "Python for Data Analysis.ipynb".
The first task is:
Sorting and ranking
Can you write Python code for it?

targets: 
obj = Series(range(4), index=['d', 'a', 'b', 'c'])

obj

obj.sort_index()

frame = DataFrame({'b': [4, 7, -3, 2], 'a': [0, 1, 0, 1]})

frame.sort_values('b')
inputs: (1e) Aplicar OHE em uma base de dados 
Finalmente, use a função da parte (1d) para criar atributos OHE para todos os 3 objetos da base de dados artificial.

targets: # EXERCICIO
sampleOHEData = sampleDataRDD.map(lambda x:oneHotEncoding(x, sampleOHEDictManual, numSampleOHEFeats))#<COMPLETAR>
print sampleOHEData.collect()

# TEST Apply OHE to a dataset (1e)
sampleOHEDataValues = sampleOHEData.collect()
Test.assertTrue(len(sampleOHEDataValues) == 3, 'sampleOHEData should have three elements')
Test.assertEquals(sampleOHEDataValues[0], SparseVector(7, {2: 1.0, 3: 1.0}),
                  'incorrect OHE for first sample')
Test.assertEquals(sampleOHEDataValues[1], SparseVector(7, {1: 1.0, 4: 1.0, 5: 1.0}),
                  'incorrect OHE for second sample')
Test.assertEquals(sampleOHEDataValues[2], SparseVector(7, {0: 1.0, 3: 1.0, 6: 1.0}),
                  'incorrect OHE for third sample')
inputs: %octave_pull tempin ALK DIC pH pCO2 fCO2 xCO2 HCO3 CO3 CO2 RF OmegaAr

#print(tempin)
pieces = [ALK, DIC, pH, pCO2, fCO2, xCO2, HCO3, CO3, CO2, RF, OmegaAr]
pieces = np.array(pieces)
pieces = np.squeeze(pieces)
pieces = np.transpose(pieces)
cols = list(['ALK', 'DIC', 'pH', 'pCO2', 'fCO2', 'xCO2', 'HCO3', 'CO3', 'CO2', 'RF', 'OmegaAr'])
#pad = pd.DataFrame(pieces, index=list(tempin), columns=cols)
df = pd.DataFrame(pieces, columns=cols)
df
Given the above code, generate some markdown instructions for it.

targets: 
5. Pull arrays (from octave to python), then use pandas to display a nice table
Pull arrays, then combine into a pandas DataFrame
inputs: # При каждом вызове функции fib создаётся локальная область видимости с переменными n, a, b и i
def fib(n):
    a, b = 0, 1
    for i in range(n):
        a, b = b, a + b
    return a

# Глобальная область видимости содержит переменную k
k = int(input())
print(fib(k))

# При отсутствии переменной в локальной области видимости, используется такая же переменная из глобальной
def f(a):
    return a + b

b = 4
print(f(7))

# Ошибка. Локальная переменная неизвестна после окончания функции 
def f():
    local_var = "Only local"
    print(local_var)

f()
print(local_var)
Given the above code, generate some markdown instructions for it.

targets: 
Области видимости
Область видимости — это изолированная часть программы со своим набором переменных.
Все переменные, объявленные в основной программе находятся в глобальной области видимости.
При каждом вызове функции создаётся новая локальная область видимости.
inputs: I am working on the file "03-loops-control-flow.ipynb".
The first task is:
But it might be that the input is just wrong: there's a typo, and the caller should be warned, and maybe the input completely rejected. We can write a different function to check that.
Can you write Python code for it?

targets: 
from math import pi

def check_angle_normalized(theta_d):
    """
    Check that an angle lies within [0, 360] degrees.
    
    Parameters
    ----------
    
    theta_d : float
        The angle in degrees.
        
    Returns
    -------
    
    normalized : Boolean
        Whether the angle lies within the range
    """
    
    normalized = True
    if theta_d > 360.0:
        normalized = False
        print("Input angle greater than 360 degrees. Did you mean this?")
    if theta_d < 0.0:
        normalized = False
        print("Input angle less than 0 degrees. Did you mean this?")
    return normalized

theta_d = 5134.6
print(check_angle_normalized(theta_d))
theta_d = -52.3
print(check_angle_normalized(theta_d))
inputs: "Create schema file
The deployment requires a schema file to define the incoming data."
Please write code following the instructions in jupyter notebook style.

targets: 
# We'll use the known label, key variables and 
# a few extra columns we won't need. (machineID is required)
key_cols =['label_e','dt_truncated', 'failure','model_encoded','model' ]

# Then get the remaining feature names from the data
input_features = fedata.columns
# Remove the extra stuff if it's in the input_df
input_features = [x for x in input_features if x not in set(key_cols)]

# define the input data frame
inputs = {"input_df": SampleDefinition(DataTypes.SPARK, 
                                       fedata.select(input_features))}

json_schema = generate_schema(run_func=run, inputs=inputs, filepath='service_schema.json')
inputs: %%time
#     generate 8000 pairs
pairs = []
for i in range(8000):
    pairs.append(pcPairGenRand())
#     lin attack
partialKey = linearAttack(pairs)

%%time
def bruteForce(partialKey=partialKey, pairs=pairs):
    print("start brute force")
    for bits20 in range(2 ** 20):
        for bits4 in range(2 ** 4):
            primitiveKeyLocal = \
            (bits20 << 12)| (partialKey[0] << 8) \
            | (bits4 << 4)|(partialKey[1] << 0)
            Kl = {}
            Kl[1] = (primitiveKey >> 16) & 0xFFFF
            Kl[2] = (primitiveKey >> 12) & 0xFFFF
            Kl[3] = (primitiveKey >> 8) & 0xFFFF
            Kl[4] = (primitiveKey >> 4) & 0xFFFF
            Kl[5] = primitiveKey & 0xFFFF
            neq = False
            paircount = len(pairs)
            for index in range(paircount):
                (plain, cipher) = pairs[index]
                if spnFromTextEncryption\
                (x=plain, K=Kl) != cipher:
                    break
                elif index == paircount:
                    print(primitiveKeyLocal)
    print("""done""")
    
Given the above code, generate some markdown instructions for it.

targets: 
实施密码分析
inputs: Also plot the zonally averaged insolation at a few different times of the year:

targets: summer_solstice = 170
winter_solstice = 353
fig, ax = plt.subplots(figsize=(10,8))
ax.plot( lat, Q[:,(summer_solstice, winter_solstice)] );
ax.plot( lat, np.mean(Q, axis=1), linewidth=2 )
ax.set_xbound(-90, 90)
ax.set_xticks( range(-90,100,30) )
ax.set_xlabel('Latitude', fontsize=16 );
ax.set_ylabel('Insolation (W m$^{-2}$)', fontsize=16 );
ax.grid()
inputs: I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
This filter hypothetically achieves zero ripple in the frequency domain,
perfect attenuation, and perfect steepness. However, due to the discontunity
in the frequency response, the filter would require infinite ringing in the
time domain (i.e., infinite order) to be realized. Another way to think of
this is that a rectangular window in frequency is actually sinc_ function
in time, which requires an infinite number of samples, and thus infinite
time, to represent. So although this filter has ideal frequency suppression,
it has poor time-domain characteristics.
Let's try to naïvely make a brick-wall filter of length 0.1 sec, and look
at the filter itself in the time domain and the frequency domain:
Can you write Python code for it?

targets: 
n = int(round(0.1 * sfreq)) + 1
t = np.arange(-n // 2, n // 2) / sfreq  # center our sinc
h = np.sinc(2 * f_p * t) / (4 * np.pi)


def plot_filter(h, title, freq, gain, show=True):
    if h.ndim == 2:  # second-order sections
        sos = h
        n = mne.filter.estimate_ringing_samples(sos)
        h = np.zeros(n)
        h[0] = 1
        h = signal.sosfilt(sos, h)
        H = np.ones(512, np.complex128)
        for section in sos:
            f, this_H = signal.freqz(section[:3], section[3:])
            H *= this_H
    else:
        f, H = signal.freqz(h)
    fig, axs = plt.subplots(2)
    t = np.arange(len(h)) / sfreq
    axs[0].plot(t, h, color=blue)
    axs[0].set(xlim=t[[0, -1]], xlabel='Time (sec)',
               ylabel='Amplitude h(n)', title=title)
    box_off(axs[0])
    f *= sfreq / (2 * np.pi)
    axs[1].semilogx(f, 10 * np.log10((H * H.conj()).real), color=blue,
                    linewidth=2, zorder=4)
    plot_ideal(freq, gain, axs[1])
    mne.viz.tight_layout()
    if show:
        plt.show()

plot_filter(h, 'Sinc (0.1 sec)', freq, gain)
inputs: "<b>Create the figure and set up plotting instances.<br><br>
Create basemaps to use for the horizontal plots.<br><br>
Execute the plotting routines.</b>"
Please write code following the instructions in jupyter notebook style.

targets: 
fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2, figsize=(12, 12))#, sharex='col', sharey='row')

bm1 = create_basemap(corners=corners, proj=proj, resolution='l', area_thresh=1., 
                  lat_spacing=dLat, lon_spacing=dLon, ax=ax1)
bm2 = create_basemap(corners=corners, proj=proj, resolution='l', area_thresh=1., 
                  lat_spacing=dLat, lon_spacing=dLon, ax=ax2)
bm3 = create_basemap(corners=corners, proj=proj, resolution='l', area_thresh=1., 
                  lat_spacing=dLat, lon_spacing=dLon, ax=ax3)
bm4 = create_basemap(corners=corners, proj=proj, resolution='l', area_thresh=1., 
                  lat_spacing=dLat, lon_spacing=dLon, ax=ax4)

# Create a RadarGrid
rgp1 = RadarHorizontalPlot(r1, basemap=bm1)
rgp2 = RadarHorizontalPlot(r2, basemap=bm2)
rgp3 = RadarHorizontalPlot(r3, basemap=bm3)
rgp4 = RadarHorizontalPlot(r4, basemap=bm4)

# Create a Flightlevel instance for the track
flp1 = FlightLevel(fl1, basemap=bm1)
flp2 = FlightLevel(fl2, basemap=bm2)
flp3 = FlightLevel(fl3, basemap=bm3)
flp4 = FlightLevel(fl4, basemap=bm4)

rgp1.plot_cappi('reflectivity', 2., vmin=15., vmax=60., title=' ',
              color_bar=True, cb_pad="10%", cb_loc='bottom', cb_tick_int=4,
              ax=ax1)
rgp2.plot_cappi('Wwind', 2., vmin=-5., vmax=5., title=' ',
              mask_procedure='inside', mask_tuple=('Wwind', -1., 1.),
              cmap='RdBu_r',
              color_bar=True, cb_pad="10%", cb_loc='bottom', 
              ax=ax2)
rgp3.plot_cappi('reflectivity', 5., vmin=15., vmax=60., title=' ',
              color_bar=True, cb_pad="10%", cb_loc='bottom', cb_tick_int=4,
              ax=ax3)
rgp4.plot_cappi('Wwind', 5., vmin=-5., vmax=5., title=' ',
              mask_procedure='inside', mask_tuple=('Wwind', -1., 1.),
              cmap='RdBu_r',
              color_bar=True, cb_pad="10%", cb_loc='bottom', 
              ax=ax4)

flp1.plot_trackmap(
              start_time=start_time, end_time=end_time,
#             color_by_altitude=True, track_cmap='spectral',
#             ...
inputs: I am working on the file "PyConFr2017.ipynb".
The first task is:
import typeguard
Can you write Python code for it?

targets: 
from typeguard import typechecked

def aff(x: int) -> int:
    return x * 2 + 1

aff(2)

aff("2")

taff = typechecked(aff)

taff(2)

taff("2")

taff(2.)

from numbers import Number  # PEP 3141
@typechecked
def taff(x: Number) -> Number:
    return x * 2 + 1

taff(1), taff(1.), taff(1j)

print("** without type checking **")
%timeit aff(2)
print("** with type checking **")
%timeit taff(2)
inputs: "Aki-Richards: 179 chars"
Please write code following the instructions in jupyter notebook style.

targets: 
def a(α,β,ρ,χ,ψ,ω,t):import numpy as n;w=ω-ρ;x=ω+ρ;y=ψ+β;p=n.pi*t/180;s=n.sin(p);return w/x-(y/α)**2*w/x*s**2+(χ-α)/(χ+α)/n.cos((p+n.arcsin(χ/α*s))/2)**2-(y/α)**2*(2*(ψ-β)/y)*s**2

from bruges.reflection import akirichards

# 4-term Aki-Richards equation in 255 characters!
# http://subsurfwiki.org/wiki/Aki–Richards_equation
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

rock1 = (2300, 1200, 2500)  # Vp, Vs, rho for layer 1
rock2 = (2400, 1250, 2450)  # Vp, Vs, rho for layer 2
theta1 = np.arange(40)

result = a(*rock1, *rock2, theta1)

plt.figure(figsize=(10,4))
plt.plot(akirichards(*rock1, *rock2, theta1), 'o')
plt.plot(result)

plt.show()

inspect.getsource(a).strip()

len(inspect.getsource(a).strip())

len(inspect.getsource(akirichards).strip())
inputs: "2.9 Associative Search (Contextual Bandits)

nonassociative task: stationary task. 没有场景
associative task: the goal is to learn a policy: a mapping from situations to the actions that are best in those situations. 有场景的区分
full reinforcement learning problem: action are allowed to effec the next situation as well as the reward. 行为可以影响场景。

Exercise 2.10:

cannot tell which case: 0.5 (random)
can tell: A - 2, B - 1: 0.55

2.10 Summary"
Please write code following the instructions in jupyter notebook style.

targets: 
Image('res/fig2_6.png')
inputs: "Augment features"
Please write code following the instructions in jupyter notebook style.

targets: 
# ## Feature augmentation
# Our guess is that facies do not abrutly change from a given depth layer to the next one. Therefore, we consider features at neighboring layers to be somehow correlated. To possibly exploit this fact, let us perform feature augmentation by:
# - Select features to augment.
# - Aggregating aug_features at neighboring depths.
# - Computing aug_features spatial gradient.
# - Computing aug_features spatial gradient of gradient.


# Feature windows concatenation function
def augment_features_window(X, N_neig, features=-1):
    
    # Parameters
    N_row = X.shape[0]
    if features==-1:
        N_feat = X.shape[1]
        features=np.arange(0,X.shape[1])
    else:
        N_feat = len(features)

    # Zero padding
    X = np.vstack((np.zeros((N_neig, X.shape[1])), X, (np.zeros((N_neig, X.shape[1])))))

    # Loop over windows
    X_aug = np.zeros((N_row, N_feat*(2*N_neig)+X.shape[1]))
    for r in np.arange(N_row)+N_neig:
        this_row = []
        for c in np.arange(-N_neig,N_neig+1):
            if (c==0):
                this_row = np.hstack((this_row, X[r+c,:]))
            else:
                this_row = np.hstack((this_row, X[r+c,features]))
        X_aug[r-N_neig] = this_row

    return X_aug


# Feature gradient computation function
def augment_features_gradient(X, depth, features=-1):
    
    if features==-1:
        features=np.arange(0,X.shape[1])
    # Compute features gradient
    d_diff = np.diff(depth).reshape((-1, 1))
    d_diff[d_diff==0] = 0.001
    X_diff = np.diff(X[:,features], axis=0)
    X_grad = X_diff / d_diff
        
    # Compensate for last missing value
    X_grad = np.concatenate((X_grad, np.zeros((1, X_grad.shape[1]))))
    
    return X_grad



# Feature augmentation function
def augment_features(X, well, depth, N_neig=1, features=-1):
    
    if (features==-1):
        N_Feat=X.shape[1]
    else:
        N_Feat=len(features)
    # Augment features
    X_aug = np.zeros((X.shape[0], X.shape[1] + N_Feat*(N_neig*2+2)))
    for w in np.unique(well):
   ...
inputs: # Run the solver
solver.solve()
Given the above code, generate some markdown instructions for it.

targets: 
Finally, we run the solver.
inputs: ## not all processes have connections, this one does
java_conns = conns[conns.Pid=='3156']
java_conns['Remote Address'].sort_index().value_counts().plot(kind='bar')
Given the above code, generate some markdown instructions for it.

targets: 
We can see that IE was talking to several Internet addresses on web service ports and one local (RFC1918) address on 1337. And Java?
inputs: "Create an Azure File Share
Create an Azure File Share bucket to store our models and other results.
Run the following in your local Bash terminal with Azure CLI or in the Azure Cloud Shell
```bash
export AZ_STORAGE_ACCOUNT_NAME="<your-storage-account-name>"
export AZ_STORAGE_RESOURCE_GROUP="<your-storage-account-resource-group>"
export AZ_SHARE_NAME="mnist"
STORAGE_KEY=$(az storage account keys list --resource-group $AZ_STORAGE_RESOURCE_GROUP --account-name $AZ_STORAGE_ACCOUNT_NAME --query "[0].value" -o tsv)
az storage share create --name $AZ_SHARE_NAME --account-name $AZ_STORAGE_ACCOUNT_NAME --account-key $STORAGE_KEY
echo $STORAGE_KEY
```"
Please write code following the instructions in jupyter notebook style.

targets: 
# Insert the Storage Key in the command below to save the storage access credentials as a Kubernetes secret
%env STORAGE_KEY='<your-storage-account-access-key>'

!kubectl create secret generic azure-share-secret --namespace $TARGET_NAMESPACE --from-literal=azurestorageaccountname=$AZ_STORAGE_ACCOUNT_NAME --from-literal=azurestorageaccountkey=$STORAGE_KEY
inputs: "(a)
Prvo, isprobajte ugrađeni model na linearno odvojivom skupu podataka seven ($N=7$)."
Please write code following the instructions in jupyter notebook style.

targets: 
seven_X = np.array([[2,1], [2,3], [1,2], [3,2], [5,2], [5,4], [6,3]])
seven_y = np.array([1, 1, 1, 1, 0, 0, 0])

# Vaš kôd ovdje
rc = RidgeClassifier()
rc.fit(seven_X, seven_y)

print(accuracy_score(seven_y, rc.predict(seven_X)))