I am working on the file "README.ipynb".
The first task is:
Ilustration of the generative model with weighted / heteroscedastic noises.
Generate a synthetic data with column and row biases only.
Can you write Python code for it?

nrow = 10
ncol = 20
num_factor = 2
signal_level = 1
noise_level = .1
seed = 1
np.random.seed(seed)

w0 = (noise_level * np.random.standard_cauchy(size=(nrow, ncol))) ** -2
r0 = signal_level * np.random.randn(nrow, 1)
c0 = signal_level * np.random.randn(ncol, 1)
e0 = np.random.randn(nrow, ncol) / np.sqrt(w0)
y0 = np.outer(r0, np.ones((ncol, 1))) + np.outer(np.ones((nrow, 1)), c0)
y = y0 + e0


I am working on the file "README.ipynb".
The first task is:
Information about the annotations can be obtained with annotation_summary
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
Installation
Ideally, before installation, create a clean python3 virtual environment to deploy the package, using virtualenvwrapper for example (see http://www.simononsoftware.com/virtualenv-tutorial-part-2/).
Can you write Python code for it?

#### REMOVE in README.md ####
import JGV as package
from IPython.core.display import display, Markdown
if "__install_requires__" in package.__dict__:
    display(Markdown("## Python packages dependencies:\n"))
    for dep in package.__install_requires__:
        display(Markdown("* {}\n".format(dep)))
#############################


I am working on the file "README.ipynb".
The first task is:
Intersections between Bezier curves
Can you write Python code for it?

# Let's find all intersections between redpath and the other
redpath = paths[0]
redpath_attribs = attributes[0]
intersections = []
for path in paths[1:]:
    for (T1, seg1, t1), (T2, seg2, t2) in redpath.intersect(path):
        intersections.append(redpath.point(T1))

disvg(paths, filename='output_intersections.svg', attributes=attributes,
      nodes = intersections, node_radii = [5]*len(intersections))


I am working on the file "README.ipynb".
The first task is:
Invalid options will throw an InvalidOptionError:
Can you write Python code for it?

try:
    ti.sma(DATA, period=-5)
except ti.InvalidOptionError:
    print("Invalid Option!")

print_info(ti.bbands)

ti.bbands(DATA, period=5, stddev=2)


I am working on the file "README.ipynb".
The first task is:
It also has a class attribute referencing the original modeled class:
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
It's a nice piano roll. If the vectorization is repeated, the vectorization will be different due to the enumeration, as sme.enum and sme.canonical is set to True and False, respectively (default settings).
Can you write Python code for it?

print(sme.enumerate, sme.canonical)
vect = sme.transform(smiles)
plt.imshow(vect[0])


I am working on the file "README.ipynb".
The first task is:
LazyLoad then takes a path name, path-like object or dict-like object, which will lazily load each file with a compatible plugin.
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
More Examples
More examples can be found in the tests directory.  Tap the Ⓣ key while in the Github interface to search quickly.
Install
For the meantime:
bash
pip install git+https://github.com/tonyfast/tidy-harness
Background
harness initially responded to the need for scikit-learn models closer to a pandas.DataFrame.  Since a DataFrame is Tidy Data the rows and columns can assist in tracking samples and features over many estimations.  With this knowledge it would be easier to design a testing harness for data science.
The DataFrame has a powerful declarative syntax, consider the groupby and rolling apis.  There is a modern tendency toward declarative and functional syntaxes in scientific computing and visualization.  This is observed in altair, dask, and scikit-learn.
tidy-harness aims to provide a chain interface between pandas.DataFrame objects and other popular scientific computing libraries in the python ecosystem.  The initial harness extensions :

attach a scikit-learn estimator to the dataframe.
attach a shared jinja2 environment to render narratives about the dataframes.
bokeh plotting methods with a contextmanager for interactive visualization development

Development

The development scripts can be run through this notebook.

Jupyter notebooks are used for all Python development in this project.  The key features are:

watchdog file system watcher that converts notebooks to python scripts with nbconvert.  Tests are not converted.
nbconvert with the --execute flag to run notebooks and fill out their output.  _The current goal is for the notebook to be viewable in a Github repo.
pytest-ipynb to run tests directly on the notebooks.

Making the python module
The script below:

Installs a develop copy of harness
Listens for file systems events to convert notebooks to python scripts.
Can you write Python code for it?

%%script bash --bg
python setup.py develop
watchmedo tricks tricks.yaml

# Execute this cell to stop watching the files
%killbgscripts


I am working on the file "README.ipynb".
The first task is:
No errors. Great!
From modeled YANG modules to a NETCONF service
Finally! Time to run a Turing Machine NETCONF server...
First create an instance of the final Turing Machine class
with RPC method definitions:
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
Not the best model until now. However, prolonged training with lowering of the learning rate towards the end will improve the model.
Can you write Python code for it?

#The Enumerator can be used in sampling
i = 0

y_true = y_test[i]
y_pred = model.predict(sme.transform(X_test.iloc[i:i+1]))
print(y_true)
print(y_true - y_pred)

#Enumeration of the SMILES before sampling stabilises the result
smiles_repeat = np.array([X_test.iloc[i:i+1].values[0]]*50)
y_pred = model.predict(sme.transform(smiles_repeat))
print(y_pred.std())
print(y_true - np.median(y_pred))
_ = plt.hist(y_pred)


I am working on the file "README.ipynb".
The first task is:
Notice that in the previous two examples Banzhaf and Shapley-Shubik indices coincides. It doesn't hold in general even in the games of 3 voters:
Can you write Python code for it?

game=px.Game(4,[3, 2, 1])
game.calc() # again it calculates all available indices
print("Banzhaf index:")
print(game.banzhaf)
print("Shapley-Shubik index:")
print(game.shapley_shubik)


I am working on the file "README.ipynb".
The first task is:
Now is a good time to verify if that's really correct YANG.
Just write it to a file:
Can you write Python code for it?

with open('turing-machine.yang', 'w') as f:
    f.write(TM_YANG)


I am working on the file "README.ipynb".
The first task is:
Now lets read it in and also split between train and test sets.
Can you write Python code for it?

from sklearn.cross_validation import train_test_split

# From the names file above...
columns = 'CRIM ZN INDUS CHAS NOX RM AGE DIS RAD TAX PTRATIO B LSTAT MEDV'.split()
data = pd.read_fwf('housing.data', names=columns)
target = data['MEDV']
data = data.drop('MEDV', axis=1)

data_train, data_test, target_train, target_test = train_test_split(data, target, test_size=.2)

print data.describe()
print
print 'MEDV (target) statistics:\n%s' % target.describe()
data.head()


I am working on the file "README.ipynb".
The first task is:
Now run the Turing Machine via RPC:
Can you write Python code for it?

reply = client.send_rpc('<run/>')

tm.state, tm.tape


I am working on the file "README.ipynb".
The first task is:
Now the .to_yang() conversion also includes the rpc definitions,
with descriptions taken from the Python methods' __doc__ strings,
and rpc and input leaf names automatically
created from the Python method and argument names
by replacing underscores with hyphens again:
Can you write Python code for it?

TM_YANG = TM.to_yang(
    prefix='tm', namespace='http://modeled.netconf/turing-machine')
print(TM_YANG)


I am working on the file "README.ipynb".
The first task is:
On voit qu'une image est constituée de 50x50 = 2500 valeurs qui peuvent etre égales à 0 ou à 1.
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
One can also give a list of chromosomes to select in the fasta file
Can you write Python code for it?

j = JGV(fp=example_fasta, verbose=True, ref_list=["I","II","III"])


I am working on the file "README.ipynb".
The first task is:
One can also import the jprint and jhelp function from pycoQC to get a improve the default print and help function in jupyter
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
Optimality of the final policy
The agent effectively either took the waypoint or sat if it was illegal. That, to me, is optimal. Something I also looked into was learning my own waypoint by giving relative headings to the destination [up-left, up, up-right, left, right, down-left, down, down-right]. Obviously the environment is rewarding the wrong rewards for this scenario (tuned to the given waypoint), and I did not want to tamper with the environment so I wasn’t able to test this sufficiently.
To get a formal measure of optimality, for each trial, I counted the number of steps, $t$, as well as the number of suboptimal steps (legal but not following waypoint) $t_s$ and crime (illegal) steps $t_c$. Optimality, $\theta$, on each trial is then 1 minus the ratio of non-optimal steps:
$$\theta = 1 - \frac{t_n + t_c}{t}$$
This is shown below for each trial:
Can you write Python code for it?

def plot_cumulative_optimality(stats, sim_type, ax=None):
    (1. - (stats['suboptimals'] + stats['crimes']) /
     stats['n_turns']).plot(
        ax=ax, kind='area', stacked=False, figsize=(15, 8),
        title='%s Optimality in Each Trial' % (
            sim_type
        )
    )
    pylab.ylabel('% of Optimal Moves')
    pylab.xlabel('Trial#')

train_test_plots(train_stats, test_stats,
                 plot_cumulative_optimality)


I am working on the file "README.ipynb".
The first task is:
Or by directory (loading all .py files):
Can you write Python code for it?

fobj = utils.MockPath('example.py',is_file=True,content="""
class ParserPlugin(object):
    plugin_name = 'example.other'
    plugin_descript = 'a parser for *.example.other files, that outputs (line_number:line)'
    file_regex = '*.example.other'
    def read_file(self, file_obj, **kwargs):
        out_dict = {}
        for i, line in enumerate(file_obj):
            out_dict[i] = line.strip()
        return out_dict
""")
dobj = utils.MockPath(structure=[fobj])
plugins.load_plugins_dir(dobj,'parsers')
plugins.view_plugins()


I am working on the file "README.ipynb".
The first task is:
Other keyword arguments are:

new= Any callable object which will be called instead of member's data type
  if setting a member value which is not an instance of data type.
  Must return a data type instance.
choices= A sequence of allowed member values.
changed= A sequence of callback functions (any callable objects)
  to be triggered after setting member values.
  Get the modeled class instance as first and the value as second argument.

When providing a default value, the data type can be omitted:
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
Plot results
There's a possibility to plot the power distribution as a pie chart:
Can you write Python code for it?

game=px.Game(4,[3, 2, 1])
game.calc()
game.pie_chart()


I am working on the file "README.ipynb".
The first task is:
Plotting the coverage and annotation features of a specific window
interval_plot is undoubtedly the most useful function of the package. It has a large panel of option to customize the plots and will adapt automatically to plot all the annotation and alignment coverage over a defined genomic interval or an entire chromosome
Can you write Python code for it?

j.interval_plot("VI", feature_types=["gene", "transcript", "CDS"])

j.interval_plot("VI", start=220000, end=225000)


I am working on the file "README.ipynb".
The first task is:
Plugins are recognised as classes with a minimal set of attributes matching the plugin category interface:
Can you write Python code for it?

plugins.view_interfaces()

plugins.unload_all_plugins()
plugins.view_plugins()


I am working on the file "README.ipynb".
The first task is:
Plugins can be defined for parsing each file type (see Creating Plugins section):
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
Read in the raw data
The raw data from the UCI Machine Learning Repository contains two files:
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
Reading SVGSs
The svg2paths() function converts an svgfile to a list of Path objects and a separate list of dictionaries containing the attributes of each said path.
Note: Line, Polyline, Polygon, and Path SVG elements can all be converted to Path objects using this function.
Can you write Python code for it?

# Read SVG into a list of path objects and list of dictionaries of attributes
from svgpathtools import svg2paths, wsvg
paths, attributes = svg2paths('test.svg')

# Update: You can now also extract the svg-attributes by setting
# return_svg_attributes=True, or with the convenience function svg2paths2
from svgpathtools import svg2paths2
paths, attributes, svg_attributes = svg2paths2('test.svg')

# Let's print out the first path object and the color it was in the SVG
# We'll see it is composed of two CubicBezier objects and, in the SVG file it
# came from, it was red
redpath = paths[0]
redpath_attribs = attributes[0]
print(redpath)
print(redpath_attribs['stroke'])


I am working on the file "README.ipynb".
The first task is:
Ready for turning on the Turing Machine:
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
Regression
Can you write Python code for it?

from nonconformist.base import RegressorAdapter
from nonconformist.nc import RegressorNc

class MyRegressorAdapter(RegressorAdapter):
    def __init__(self, model, fit_params=None):
        super(MyRegressorAdapter, self).__init__(model, fit_params)

    def fit(self, x, y):
        '''
            x is a numpy.array of shape (n_train, n_features)
            y is a numpy.array of shape (n_train)

            Here, do what is necessary to train the underlying model
            using the supplied training data
        '''
        pass

    def predict(self, x):
        '''
            Obtain predictions from the underlying model

            Make sure this function returns an output that is compatible with
            the nonconformity function used. For default nonconformity functions,
            output from this function should be predicted real values in a
            numpy.array of shape (n_test)
        '''
        pass

my_regressor = None # Initialize an object of your regressor's type
model = MyRegressorAdapter(my_regressor)
nc = RegressorNc(model)


from sklearn.ensemble import RandomForestRegressor
from sklearn.neighbors import KNeighborsRegressor
from nonconformist.nc import NcFactory

nc = NcFactory.create_nc(
    RandomForestRegressor(n_estimators=500),
    normalizer_model=KNeighborsRegressor(n_neighbors=11)
)


from sklearn.neighbors import KNeighborsRegressor
from nonconformist.nc import NcFactory, AbsErrorErrFunc, SignErrorErrFunc

nc_abs = NcFactory.create_nc(
    KNeighborsRegressor(n_neighbors=11),
    AbsErrorErrFunc()
)

nc_sign = NcFactory.create_nc(
    KNeighborsRegressor(n_neighbors=11),
    SignErrorErrFunc()
)


from sklearn.tree import DecisionTreeRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn.neighbors import KNeighborsRegressor

nc_dt = NcFactory.create_nc(DecisionTreeRegressor(min_samples_leaf=5))
nc_rf = NcFactory.create_nc(RandomForestRegressor(n_estimators=500))
nc_knn = NcFactory.create_nc(KNeighborsRegressor(n_neighbors=11))


I am working on the file "README.ipynb".
The first task is:
Regression
Here, we introduce a nonconformity for regression, where
$\alpha_i = ln \left(\left| y_i - \hat{y}_i \right|\right)$
and
$\Gamma_i^{\epsilon} = \hat{y}i \pm e^{\alpha{\epsilon}}$
(This example is provided only for illustration, as it is practically identical to simply taking the absolute error as nonconformity.)
Can you write Python code for it?

from sklearn.svm import SVR
from nonconformist.nc import NcFactory, RegressionErrFunc

class MyRegErrFunc(RegressionErrFunc):
    def __init__(self):
        super(MyRegErrFunc, self).__init__()

    def apply(self, prediction, y):
        '''
            y is a vector of (true) labels
            prediction is a vector of (predicted) labels
        '''
        return np.log(np.abs(prediction - y))

    def apply_inverse(self, nc, significance):
        '''
            apply_inverse() is the (partial) inverse of apply()
            apply_inverse(...)[0] is subtracted from the prediction of the
                underlying model to create the lower boundary of the
                prediction interval
            apply_inverse(...)[1] is added to the prediction of the
                underlying model to create the upper boundary of the
                prediction interval
        '''
        nc = np.sort(nc)[::-1]
        border = int(np.floor(significance * (nc.size + 1))) - 1
        border = min(max(border, 0), nc.size - 1)
        return np.vstack([np.exp(nc[border]), np.exp(nc[border])])

model = SVR()
nc = NcFactory.create_nc(model, MyRegErrFunc())


I am working on the file "README.ipynb".
The first task is:
Rotations and Translations
Can you write Python code for it?

# Let's take a Line and an Arc and make some pictures
top_half = Arc(start=-1, radius=1+2j, rotation=0, large_arc=1, sweep=1, end=1)
midline = Line(-1.5, 1.5)

# First let's make our ellipse whole
bottom_half = top_half.rotated(180)
decorated_ellipse = Path(top_half, bottom_half)

# Now let's add the decorations
for k in range(12):
    decorated_ellipse.append(midline.rotated(30*k))

# Let's move it over so we can see the original Line and Arc object next
# to the final product
decorated_ellipse = decorated_ellipse.translated(4+0j)
wsvg([top_half, midline, decorated_ellipse], filename='decorated_ellipse.svg')


I am working on the file "README.ipynb".
The first task is:
Several annotation can be loaded. Warnings will be thrown if there are chromosomes found in the reference sequence have no feature in the annotation file
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
Similar to annotation, JGV also has an alignment_summary function
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
Single outputs are returned directly. Indicators returning multiple outputs use
a tuple in the order indicated by the outputs property.
Can you write Python code for it?

ti.sqrt(DATA)

print_info(ti.sma)

ti.sma(DATA, period=5)


I am working on the file "README.ipynb".
The first task is:
Take a directory structure, potentially containing multiple file types:
Can you write Python code for it?

datadir = example_mockpaths.directory1
print(datadir.to_string(indentlvl=3,file_content=True,color=True))


I am working on the file "README.ipynb".
The first task is:
The Path class is a mutable sequence, so it behaves much like a list.
So segments can appended, inserted, set by index, deleted, enumerated, sliced out, etc.
Can you write Python code for it?

# Let's append another to the end of it
path.append(CubicBezier(250+350j, 275+350j, 250+225j, 200+100j))
print(path)

# Let's replace the first segment with a Line object
path[0] = Line(200+100j, 200+300j)
print(path)

# You may have noticed that this path is connected and now is also closed (i.e. path.start == path.end)
print("path is continuous? ", path.iscontinuous())
print("path is closed? ", path.isclosed())

# The curve the path follows is not, however, smooth (differentiable)
from svgpathtools import kinks, smoothed_path
print("path contains non-differentiable points? ", len(kinks(path)) > 0)

# If we want, we can smooth these out (Experimental and only for line/cubic paths)
# Note:  smoothing will always works (except on 180 degree turns), but you may want
# to play with the maxjointsize and tightness parameters to get pleasing results
# Note also: smoothing will increase the number of segments in a path
spath = smoothed_path(path)
print("spath contains non-differentiable points? ", len(kinks(spath)) > 0)
print(spath)

# Let's take a quick look at the path and its smoothed relative
# The following commands will open two browser windows to display path and spaths
from svgpathtools import disvg
from time import sleep
disvg(path)
sleep(1)  # needed when not giving the SVGs unique names (or not using timestamp)
disvg(spath)
print("Notice that path contains {} segments and spath contains {} segments."
      "".format(len(path), len(spath)))


I am working on the file "README.ipynb".
The first task is:
The RandomizedSearchCV picked the same model we picked, with a maximum depth of 6. This is the model that maximized the validation score.
Predicting Selling Prices
Finally!
It does a bit worse than I'd hoped (R2 of .82). If I was to use this model for clients, I'd need some measure of the prediction error. Obviously, I can calculate this on the out of sample predictions. However, something worth noting is that it's likely that the standard deviation of home prices increases with the home price itself. Personally, I'd expect standard deviation to be a % of the price, similiar to that observed in stock prices (Hull, 1997). We'll check this next...
Can you write Python code for it?

# Produce a matrix for client data
client_data = data_test

predictions = pd.DataFrame(dict(
    Actual=target_test * 1000.,
    Predicted=reg.predict(client_data) * 1000.,
))

r2 = performance_metric(predictions.Predicted, predictions.Actual)
errors = (predictions.Actual - predictions.Predicted)
error = errors.std()

predictions.plot.scatter(x='Predicted', y='Actual', title='Out of Sample R2 on clients: %.2f. Error Std. Dev +-$%.2f' % (
        r2, error,
))


I am working on the file "README.ipynb".
The first task is:
The ability to convert between Bezier objects to NumPy polynomial objects is very useful.  For starters, we can take turn a list of Bézier segments into a NumPy array
Numpy Array operations on Bézier path segments
Example available here
To further illustrate the power of being able to convert our Bezier curve objects to numpy.poly1d objects and back, lets compute the unit tangent vector of the above CubicBezier object, b, at t=0.5 in four different ways.
Tangent vectors (and more on NumPy polynomials)
Can you write Python code for it?

t = 0.5
### Method 1: the easy way
u1 = b.unit_tangent(t)

### Method 2: another easy way
# Note: This way will fail if it encounters a removable singularity.
u2 = b.derivative(t)/abs(b.derivative(t))

### Method 2: a third easy way
# Note: This way will also fail if it encounters a removable singularity.
dp = p.deriv()
u3 = dp(t)/abs(dp(t))

### Method 4: the removable-singularity-proof numpy.poly1d way
# Note: This is roughly how Method 1 works
from svgpathtools import real, imag, rational_limit
dx, dy = real(dp), imag(dp)  # dp == dx + 1j*dy
p_mag2 = dx**2 + dy**2  # p_mag2(t) = |p(t)|**2
# Note: abs(dp) isn't a polynomial, but abs(dp)**2 is, and,
#  the limit_{t->t0}[f(t) / abs(f(t))] ==
# sqrt(limit_{t->t0}[f(t)**2 / abs(f(t))**2])
from cmath import sqrt
u4 = sqrt(rational_limit(dp**2, p_mag2, t))

print("unit tangent check:", u1 == u2 == u3 == u4)

# Let's do a visual check
mag = b.length()/4  # so it's not hard to see the tangent line
tangent_line = Line(b.point(t), b.point(t) + mag*u1)
disvg([b, tangent_line], 'bg', nodes=[b.point(t)])


I am working on the file "README.ipynb".
The first task is:
The following RPC definitions are again designed
according to the pyang tutorial.
Since those RPC methods are NETCONF/YANG specific,
they are defined after the modeled YANG adaption.
The simplest way is to derive a new class for that purpose:
Can you write Python code for it?

class TM(YANG[TuringMachine]):

    @rpc(argtypes={'tape_content': str})
    # in Python 3 you can also use function annotations
    # and write (self, tape_content: str) below
    # instead of argtypes= above
    def initialize(self, tape_content):
        """Initialize the Turing Machine.
        """
        self.state = 0
        self.head_position = 0
        self.tape = tape_content

    @rpc(argtypes={})
    def run(self):
        """Start the Turing Machine operation.
        """
        TuringMachine.run(self)


I am working on the file "README.ipynb".
The first task is:
The result is a boolean numpy.array with shape (n_test, n_classes), where each row is a boolean vector denoting the class labels included in the prediction region at the specified significance level.
For this particular example, we might obtain, for a given test object, a boolean vector [ True True False ], meaning that the $1-\epsilon$ confidence prediction region contains class labels 0 and 1 (i.e., with 95% probability, one of these two classes will be correct).
Example 1b: Simple TCP (classification)
In this example, we construct a simple transductive conformal predictor for classification, using a support vector classifier as the underlying model.
Can you write Python code for it?

from sklearn.datasets import load_iris
import numpy as np
from sklearn.svm import SVC
from nonconformist.cp import TcpClassifier
from nonconformist.nc import NcFactory

iris = load_iris()
idx = np.random.permutation(iris.target.size)

# Divide the data into training set and test set
idx_train, idx_test = idx[:100], idx[100:]

model = SVC(probability=True)	# Create the underlying model
nc = NcFactory.create_nc(model)	# Create a default nonconformity function
tcp = TcpClassifier(nc)			# Create a transductive conformal classifier

# Fit the TCP using the proper training set
tcp.fit(iris.data[idx_train, :], iris.target[idx_train])

# Produce predictions for the test set, with confidence 95%
prediction = tcp.predict(iris.data[idx_test, :], significance=0.05)

# Print the first 5 predictions
print(prediction[:5, :])


I am working on the file "README.ipynb".
The first task is:
The reverse_transform() function can be used to translate back to a SMILES string, as long as the charset is the same as was used to vectorize.
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
The tape will be set accordingly:
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
There have been added some extra padding to the maximum lenght observed in the smiles array. The SMILES can be transformed to one-hot encoded vectors and showed with matplotlib.
Can you write Python code for it?

import matplotlib.pyplot as plt
%matplotlib inline
vect = sme.transform(smiles)
plt.imshow(vect[0])


I am working on the file "README.ipynb".
The first task is:
There will be many more examples of writing and displaying path data below.
The .point() method and transitioning between path and path segment parameterizations
SVG Path elements and their segments have official parameterizations.
These parameterizations can be accessed using the Path.point(), Line.point(), QuadraticBezier.point(), CubicBezier.point(), and Arc.point() methods.
All these parameterizations are defined over the domain 0 <= t <= 1.
Note: In this document and in inline documentation and doctrings, I use a capital T when referring to the parameterization of a Path object and a lower case t when referring speaking about path segment objects (i.e. Line, QaudraticBezier, CubicBezier, and Arc objects).
Given a T value, the Path.T2t() method can be used to find the corresponding segment index, k, and segment parameter, t, such that path.point(T)=path[k].point(t).
There is also a Path.t2T() method to solve the inverse problem.
Can you write Python code for it?

# Example:

# Let's check that the first segment of redpath starts
# at the same point as redpath
firstseg = redpath[0]
print(redpath.point(0) == firstseg.point(0) == redpath.start == firstseg.start)

# Let's check that the last segment of redpath ends on the same point as redpath
lastseg = redpath[-1]
print(redpath.point(1) == lastseg.point(1) == redpath.end == lastseg.end)

# This next boolean should return False as redpath is composed multiple segments
print(redpath.point(0.5) == firstseg.point(0.5))

# If we want to figure out which segment of redpoint the
# point redpath.point(0.5) lands on, we can use the path.T2t() method
k, t = redpath.T2t(0.5)
print(redpath[k].point(t) == redpath.point(0.5))


I am working on the file "README.ipynb".
The first task is:
This process can be reversed, using encoder plugins:
Can you write Python code for it?

plugins.load_builtin_plugins('encoders')
plugins.view_plugins('encoders')

import json
json.dumps(dct,default=plugins.encode)


I am working on the file "README.ipynb".
The first task is:
To install in development mode:

pip install -e /path/to/repository/

From a MODELED class to a YANG module
MODELED.netconf is based on my
MODELED framework,
which provides tools for defining Python classes
with typed members and methods,
quite similar to Django database models,
but with a more general approach.
Those modeled classes can then automagically be mapped
to data serialization formats, databases,
GUI frameworks, web frameworks, or whatever,
using the integrated modeled.Adapter system.
The MODELED framework is still in a late alpha stage,
needs some internal refactoring, and lacks documentation,
but I am actively working on this.
The basic principles should nevertheless become visible during the following example.
A MODELED Turing Machine
Since MODELED.netconf uses pyang
for auto-generating YANG definitions from modeled classes,
I decided to resemble the Turing Machine example from the
pyang tutorial...
a bit more simplified and with some little structural and naming changes...
however... below is a modeled Turing Machine implementation:
Can you write Python code for it?

import modeled
from modeled import member

class Input(modeled.object):
    """The input part of a Turing Machine program rule.
    """
    state = member[int]()
    symbol = member[str]()

class Output(modeled.object):
    """The output part of a Turing Machine program rule.
    """
    state = member[int]()
    symbol = member[str]()
    head_move = member[str]['L', 'R']()

class Rule(modeled.object):
    """A Turing Machine program rule.
    """
    input = member[Input]()
    output = member[Output]()

    def __init__(self, input, output):
        """Expects both `input` and `output` as mappings.
        """
        self.input = Input(
            # modeled.object.__init__ supports **kwargs
            # for initializing modeled.member values
            **dict(input))
        self.output = Output(**dict(output))

class TuringMachine(modeled.object):
    state = member[int]()
    head_position = member[int]()

    # the list of symbols on the input/output tape
    tape = member.list[str](indexname='cell', itemname='symbol')

    # the machine program as named rules
    program = member.dict[str, Rule](keyname='name')

    def __init__(self, program):
        """Create a Turing Machine with the given `program`.
        """
        program = dict(program)
        for name, (input, output) in program.items():
            self.program[name] = Rule(input, output)

    def run(self):
        """Start the Turing Machine.

        - Runs until no matching input part for current state and tape symbol
          can be found in the program rules.
        """
        self.log = " %s  %d\n" % (''.join(self.tape), self.state)
        while True:
            pos = self.head_position
            if 0 <= pos < len(self.tape):
                symbol = self.tape[pos]
            else:
                symbol = None
            for name, rule in self.program.items():
                if (self.state, symbol) == (rule.input.state, rule.input.symbol):
                    self.log += "%s^%s --> %s\n" % (
                 ...


I am working on the file "README.ipynb".
The first task is:
To try the rendered JSON tree, output in the Jupyter Notebook, go to : https://chrisjsewell.github.io/
Nested Dictionary Manipulation
Can you write Python code for it?

jdict2 = ejson.to_dict(path,['dir1','file1'])
edict.pprint(jdict2,depth=1)

filtered = edict.filter_keys(jdict2,['vol*'],use_wildcards=True)
edict.pprint(filtered)

edict.pprint(edict.flatten(filtered))


I am working on the file "README.ipynb".
The first task is:
To update the package:
Can you write Python code for it?

pip3 install git+https://github.com/a-slide/JupyterGenoViewer.git --upgrade --process-dependency-links


I am working on the file "README.ipynb".
The first task is:
Train the model / get posterior samples using the MatrixFactorization class.
The convergence of MCMC can be slow for some data sets. Alternating least squares works very similarly to MCMC, but gradient descent with momentum could converge faster (not investigated).
Can you write Python code for it?

%%time
from matrix_factorization import MatrixFactorization as MF

# Prior parameters for the model
bias_scale = signal_level
factor_scale = math.sqrt(signal_level)
mf = MF(y_train_coo, num_factor, bias_scale, factor_scale, weight_train)

# Gibbs update of the latent factors can be parallelized, though the overhead
# of multi-processing overwhelms for not very large matrices.
n_burnin = 25 * 10 ** 3
n_mcmc = 1000
y_post_mean, sample_dict = mf.gibbs(n_burnin, n_mcmc, n_update=10, num_process=1)


I am working on the file "README.ipynb".
The first task is:
Units Schema
Can you write Python code for it?

from jsonextended.units import apply_unitschema, split_quantities
withunits = apply_unitschema(filtered,{'volume':'angstrom^3'})
edict.pprint(withunits)

newunits = apply_unitschema(withunits,{'volume':'nm^3'})
edict.pprint(newunits)

edict.pprint(split_quantities(newunits),depth=4)


I am working on the file "README.ipynb".
The first task is:
Updates to the final agent and final performance
Can you write Python code for it?

# Generate training, and test simulations
learned_agent_env, train_stats = generated_sim_stats(
    n_trials=100,
    gamma=0.05,
    initial_alpha=1.0,
    min_alpha=0.05,
    alpha_span=2.0,
)
_, test_stats = generated_sim_stats(
    agent_env=learned_agent_env, n_trials=100)


I am working on the file "README.ipynb".
The first task is:
Usage
The package is meant to be used in a jupyter notebook 4.0.0 +
Notebook setup
Launch the notebook in a terminal
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
Use the generator object for training.
Can you write Python code for it?

model.fit_generator(generator, steps_per_epoch=100, epochs=25, workers=4)

y_pred_train = model.predict(sme.transform(X_train))
y_pred_test = model.predict(sme.transform(X_test))
plt.scatter(y_train, y_pred_train, label="Train")
plt.scatter(y_test, y_pred_test, label="Test")
plt.legend()


I am working on the file "README.ipynb".
The first task is:
Vectorization
Before vectorization SMILES must be stored as strings in an numpy array. The transform takes numpy arrays or pandas series with the SMILES as strings.
Can you write Python code for it?

import numpy as np
smiles = np.array(["CCC(=O)O[C@@]1(CC[NH+](C[C@H]1CC=C)C)c2ccccc2"])
print(smiles.shape)


I am working on the file "README.ipynb".
The first task is:
We obtain a result that is conceptually identical as in the previous example (although the particular output values will differ).
Example 1c: Simple ICP (regression)
In this example, we construct a simple inductive conformal predictor for regression, this time using a random forest regression model as the underlying model.
Can you write Python code for it?

from sklearn.datasets import load_boston
import numpy as np
from sklearn.ensemble import RandomForestRegressor
from nonconformist.cp import IcpRegressor
from nonconformist.nc import NcFactory

boston = load_boston()
idx = np.random.permutation(boston.target.size)

# Divide the data into proper training set, calibration set and test set
idx_train, idx_cal, idx_test = idx[:300], idx[300:399], idx[399:]

model = RandomForestRegressor()	# Create the underlying model
nc = NcFactory.create_nc(model)	# Create a default nonconformity function
icp = IcpRegressor(nc)			# Create an inductive conformal regressor

# Fit the ICP using the proper training set
icp.fit(boston.data[idx_train, :], boston.target[idx_train])

# Calibrate the ICP using the calibration set
icp.calibrate(boston.data[idx_cal, :], boston.target[idx_cal])

# Produce predictions for the test set, with confidence 95%
prediction = icp.predict(boston.data[idx_test, :], significance=0.05)

# Print the first 5 predictions
print(prediction[:5, :])


I am working on the file "README.ipynb".
The first task is:
We'll take a quick look at the provided description: housing.names
Can you write Python code for it?


I am working on the file "README.ipynb".
The first task is:
What I noticed here was that the train performance was very similiar to the test performance in terms of the success rate. My intuition is that this is mainly due to the high gamma, which results in Q values that are slow to converge. Finally my $\alpha$ were decaying fairly slowly due to a span of 100, this caused my temperatures to stay high and randomly sample many suboptimal actions. Combined, this exacerbated bad estimates of Q values, which caused the test run to fail to significantly improve the overall success rate.
However, I did find that the test run was much safer after taking a look at the cumulative trips with crimes, thus it was learning:
Can you write Python code for it?

def plot_cumulative_crimes(stats, sim_type, ax=None):
    (stats['crimes'.split()] > 0).cumsum().plot(
        ax=ax, kind='area', stacked=False, figsize=(15, 8),
        title='Cumulative %s Trials With Any Crimes: %.0f%%' % (
            sim_type, (stats.crimes > 0).mean()*100
        )
    )
    pylab.ylabel('# of Crimes')
    pylab.xlabel('Trial#')

train_test_plots(train_stats, test_stats,
                 plot_cumulative_crimes)


I am working on the file "README.ipynb".
The first task is:
What's driving our model?
For the most part, I've really relied (perhaps too much) on the data to determine our model. I didn't remove any factors. I also didn't do any kinds of factor decompositions or scaling. In terms of scaling, Tree models are effectively invariant so this is ok. Also, while more factors increases the combinations tree models can search, I'm applying some L2 regularization on the model that discourages the model from becoming too complex.
Still, it likely would have been fruitful to do a bit more introspection on the factors before fitting the model, but alas it's too late. We've seen the test set.
Instead, let's take a look at what factors our model determined to be most important.
Can you write Python code for it?

feature_importances = pd.Series(reg.booster().get_fscore())
feature_importances = feature_importances / feature_importances.sum()
feature_importances.sort_values().plot.bar(colormap='Paired', title='Feature importances')


I am working on the file "README.ipynb".
The first task is:
Writing SVGSs (and some geometric functions and methods)
The wsvg() function creates an SVG file from a list of path.  This function can do many things (see docstring in paths2svg.py for more information) and is meant to be quick and easy to use.
Note: Use the convenience function disvg() (or set 'openinbrowser=True') to automatically attempt to open the created svg file in your default SVG viewer.
Can you write Python code for it?

# Let's make a new SVG that's identical to the first
wsvg(paths, attributes=attributes, svg_attributes=svg_attributes, filename='output1.svg')


I am working on the file "README.ipynb".
The first task is:
arc length and inverse arc length
Here we'll create an SVG that shows off the parametric and geometric midpoints of the paths from test.svg.  We'll need to compute use the Path.length(), Line.length(), QuadraticBezier.length(), CubicBezier.length(), and Arc.length() methods, as well as the related inverse arc length methods .ilength() function to do this.
Can you write Python code for it?

# First we'll load the path data from the file test.svg
paths, attributes = svg2paths('test.svg')

# Let's mark the parametric midpoint of each segment
# I say "parametric" midpoint because Bezier curves aren't
# parameterized by arclength
# If they're also the geometric midpoint, let's mark them
# purple and otherwise we'll mark the geometric midpoint green
min_depth = 5
error = 1e-4
dots = []
ncols = []
nradii = []
for path in paths:
    for seg in path:
        parametric_mid = seg.point(0.5)
        seg_length = seg.length()
        if seg.length(0.5)/seg.length() == 1/2:
            dots += [parametric_mid]
            ncols += ['purple']
            nradii += [5]
        else:
            t_mid = seg.ilength(seg_length/2)
            geo_mid = seg.point(t_mid)
            dots += [parametric_mid, geo_mid]
            ncols += ['red', 'green']
            nradii += [5] * 2

# In 'output2.svg' the paths will retain their original attributes
wsvg(paths, nodes=dots, node_colors=ncols, node_radii=nradii,
     attributes=attributes, filename='output2.svg')


I am working on the file "README.ipynb".
The first task is:
nosebook-scrub
A regular expression that will be replaced throughout the expected outputs and generated outputs.
Default: None
Can you write Python code for it?

# you can't fail if you don't try
!nosetests --with-nosebook --nosebook-scrub .+


I am working on the file "README.ipynb".
The first task is:
which can be accessed by the special last result variable _:
Can you write Python code for it?


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Beyond Plain Python.ipynb".
The first task is:
An IPython quick reference card:
Can you write Python code for it?


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Beyond Plain Python.ipynb".
The first task is:
And when your code produces errors, you can control how they are displayed with the %xmode magic:
Can you write Python code for it?

%%writefile mod.py

def f(x):
    return 1.0/(x-1)

def g(y):
    return f(y+1)


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Beyond Plain Python.ipynb".
The first task is:
Another interesting cell magic: create any file you want locally from the notebook:
Can you write Python code for it?

%%writefile test.txt
This is a test file!
It can contain anything I want...

And more...

!cat test.txt


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Beyond Plain Python.ipynb".
The first task is:
Don't foget to exit your debugging session. Raw input can of course be use to ask for user input:
Can you write Python code for it?

enjoy = input('Are you enjoying this tutorial? ')
print('enjoy is:', enjoy)


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Beyond Plain Python.ipynb".
The first task is:
Getting help:
Can you write Python code for it?


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Beyond Plain Python.ipynb".
The first task is:
Line vs cell magics:
Can you write Python code for it?

%timeit list(range(1000))

%%timeit
list(range(10))
list(range(100))


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Beyond Plain Python.ipynb".
The first task is:
Note that all this is available even in multiline blocks:
Can you write Python code for it?

import os
for i,f in enumerate(files):
    if f.endswith('ipynb'):
        !echo {"%02d" % i} - "{os.path.splitext(f)[0]}"
    else:
        print('--')


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Beyond Plain Python.ipynb".
The first task is:
Raw Input in the notebook
Since 1.0 the IPython notebook web application support raw_input which for example allow us to invoke the %debug magic in the notebook:
Can you write Python code for it?


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Beyond Plain Python.ipynb".
The first task is:
The default %xmode is "context", which shows additional context but not all local variables.  Let's restore that one for the rest of our session.
Can you write Python code for it?


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Beyond Plain Python.ipynb".
The first task is:
Typing object_name? will print all sorts of details about any object, including docstrings, function definition lines (for call arguments) and constructor details for classes.
Can you write Python code for it?

import collections
collections.namedtuple?

collections.Counter??

*int*?


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Beyond Plain Python.ipynb".
The first task is:
We can connect automatically a Qt Console to the currently running kernel with the %qtconsole magic, or by typing ipython console --existing <kernel-UUID> in any terminal:
Can you write Python code for it?


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Beyond Plain Python.ipynb".
The first task is:
You can suppress the storage and rendering of output if you append ; to the last cell (this comes in handy when plotting with matplotlib, for example):
Can you write Python code for it?


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Cell Magics.ipynb".
The first task is:
Background Scripts
These scripts can be run in the background, by adding the --bg flag.
When you do this, output is discarded unless you use the --out/err
flags to store output as above.
Can you write Python code for it?

%%ruby --bg --out ruby_lines
for n in 1...10
    sleep 1
    puts "line #{n}"
    STDOUT.flush
end


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Cell Magics.ipynb".
The first task is:
Capturing output
You can also capture stdout/err from these subprocesses into Python variables, instead of letting them go directly to stdout/err
Can you write Python code for it?

%%bash
echo "hi, stdout"
echo "hello, stderr" >&2

%%bash --out output --err error
echo "hi, stdout"
echo "hello, stderr" >&2

print(error)
print(output)


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Cell Magics.ipynb".
The first task is:
Since in the introductory section we already covered the most frequently used line magics, we will focus here on the cell magics, which offer a great amount of power.
Let's load matplotlib and numpy so we can use numerics/plotting at will later on.
Can you write Python code for it?

%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Cell Magics.ipynb".
The first task is:
The %%writefile magic is a very useful tool that writes the cell contents as a named file:
Can you write Python code for it?

%%writefile foo.py
print('Hello world')

%run foo


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Cell Magics.ipynb".
The first task is:
When you do store output of a background thread, these are the stdout/err pipes,
rather than the text of the output.
Can you write Python code for it?


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Third Party Rich Output.ipynb".
The first task is:
Pandas
Pandas is a data analysis library for Python. Its DataFrame objects have an HTML table representation in the Notebook.
Can you write Python code for it?


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Third Party Rich Output.ipynb".
The first task is:
Read this as into a DataFrame:
Can you write Python code for it?


I am working on the file "RES/REF/python_sourcecode/ipython-master/examples/IPython Kernel/Third Party Rich Output.ipynb".
The first task is:
SymPy
SymPy is a symbolic computing library for Python. Its equation objects have LaTeX representations that are rendered in the Notebook.
Can you write Python code for it?

from sympy.interactive.printing import init_printing
init_printing(use_latex='mathjax')

from __future__ import division
import sympy as sym
from sympy import *
x, y, z = symbols("x y z")
k, m, n = symbols("k m n", integer=True)
f, g, h = map(Function, 'fgh')

Rational(3,2)*pi + exp(I*x) / (x**2 + y)

a = 1/x + (x*sin(x) - 1)/x
a

(1/cos(x)).series(x, 0, 6)


I am working on the file "RF_GBM/notebook/Bank Marketing.ipynb".
The first task is:
Ensemble Trees

src: http://www.slideshare.net/hustwj/scaling-up-machine-learning-the-tutorial-kdd-2011-part-iia-tree-ensembles
Random Forest

src: http://www.slideshare.net/0xdata/jan-vitek-distributedrandomforest522013
Can you write Python code for it?

from sklearn.ensemble import RandomForestClassifier

bankModelRF = RandomForestClassifier(n_jobs=-1, oob_score=True)

bankModelRF.fit(inputUpdated[:train.shape[0],:], labels[:train.shape[0]])

bankModelRF.oob_score_


I am working on the file "RF_GBM/notebook/Bank Marketing.ipynb".
The first task is:
Exercise 1
print the number of rows and columns of train and test
Exercise 2
Print the first 10 rows of train
Exercise 3
Print the column types of train and test. Are they the same in both train and test?
Can you write Python code for it?

#train

#test

#Are they the same?

#Combine train and test
frames = [train, test]
input = pd.concat(frames)

#Print first 10 records of input


I am working on the file "RF_GBM/notebook/Bank Marketing.ipynb".
The first task is:
Exercise 10
Convert inputInteger to numpy array
Can you write Python code for it?


I am working on the file "RF_GBM/notebook/Bank Marketing.ipynb".
The first task is:
Exercise 12
Now, change the max_depth = 6 and check the results.
Then, change the max_depth= None and check the results
Can you write Python code for it?

# Prediction
prediction_DT = bankModelDT.predict(inputUpdated[train.shape[0]:,:])

#Compute the error metrics

import sklearn.metrics

sklearn.metrics.auc(labels[train.shape[0]:], prediction_DT)

#What does that tell?

#What's the error AUC for the other Decision Tree Models


I am working on the file "RF_GBM/notebook/Bank Marketing.ipynb".
The first task is:
Exercise 14
Do the following

Predict on test
Find accuracy metrics: AUC, Precision, Recall
How does it compare against Decision Tree

Gradient Boosting Machines
<img src="img/boosting.jpg" style="width:604px;height:428px;">
src: http://www.slideshare.net/hustwj/scaling-up-machine-learning-the-tutorial-kdd-2011-part-iia-tree-ensembles
Can you write Python code for it?

import xgboost as xgb

params = {}
params["min_child_weight"] = 3
params["subsample"] = 0.7
params["colsample_bytree"] = 0.7
params["scale_pos_weight"] = 1
params["silent"] = 0
params["max_depth"] = 4
params["nthread"] = 6
params["gamma"] = 1
params["objective"] = "binary:logistic"
params["eta"] = 0.005
params["base_score"] = 0.1
params["eval_metric"] = "auc"
params["seed"] = 123

plst = list(params.items())
num_rounds = 120

xgtrain_pv = xgb.DMatrix(inputUpdated[:train.shape[0],:], label=labels[:train.shape[0]])
watchlist = [(xgtrain_pv, 'train')]
bankModelXGB = xgb.train(plst, xgtrain_pv, num_rounds)

prediction_XGB = bankModelXGB.predict(xgb.DMatrix(inputUpdated[train.shape[0]:,:]))

sklearn.metrics.auc(labels[train.shape[0]:], prediction_XGB)


I am working on the file "RF_GBM/notebook/Bank Marketing.ipynb".
The first task is:
Exercise 4
Find if any column has missing value
There is a pd.isnull function. How to use that?
Can you write Python code for it?

#Replace deposit with a numeric column
#First, set all labels to be 0
input.at[:, "depositLabel"] = 0
#Now, set depositLabel to 1 whenever deposit is yes
input.at[input.deposit=="yes", "depositLabel"] = 1


I am working on the file "RF_GBM/notebook/Bank Marketing.ipynb".
The first task is:
Exercise 5
Find % of customers in the input dataset who have purchased the term deposit
Can you write Python code for it?

#Create the labels
labels =
labels

#Drop the deposit column
input.drop(["deposit", "depositLabel"], axis=1)


I am working on the file "RF_GBM/notebook/Bank Marketing.ipynb".
The first task is:
Exercise 6
Did it drop? If not, what has to be done?
Exercise 7
Print columnn names of input
Can you write Python code for it?

#Get list of columns that are continuous/integer
continuous_variables = input.dtypes[input.dtypes != "object"].index

continuous_variables

#Get list of columns that are categorical
categorical_variables = input.dtypes[input.dtypes=="object"].index

categorical_variables


I am working on the file "RF_GBM/notebook/Bank Marketing.ipynb".
The first task is:
Exercise 9
Find length of categorical_variables
Can you write Python code for it?

#Load the preprocessing module
from sklearn import preprocessing

for i in range(len(categorical_variables)):
    lbl = preprocessing.LabelEncoder()
    lbl.fit(list(inputCategorical[:,i]))
    inputCategorical[:, i] = lbl.transform(inputCategorical[:, i])

#print inputCategorical


I am working on the file "RF_GBM/notebook/Bank Marketing.ipynb".
The first task is:
Train the model
Model 1: Decision Tree
Can you write Python code for it?

from sklearn import tree
from sklearn.externals.six import StringIO
import pydot

bankModelDT = tree.DecisionTreeClassifier(max_depth=2)

bankModelDT.fit(inputUpdated[:train.shape[0],:], labels[:train.shape[0]])

dot_data = StringIO()
tree.export_graphviz(bankModelDT, out_file=dot_data)
graph = pydot.graph_from_dot_data(dot_data.getvalue())
graph.write_pdf("bankDT.pdf")

#Check the pdf


I am working on the file "RF_Keras_Applications.ipynb".
The first task is:
Bonus stuff: parsing model graph, printing nodes, etc.
This can be helpful, for example, for finding out the exact node names to use.
Can you write Python code for it?

node_info, name_to_node = rf.get_compute_order(graph_def, input_node_name='input_image')

order_to_info = {}
for _, info in node_info.items():
  order_to_info[info.order] = info

print(len(order_to_info.keys()))

for i in range(len(order_to_info.keys())):
  print(order_to_info[i])


I am working on the file "RFray_IS620_WK6.ipynb".
The first task is:
Due to self-loops, all names are connected to themselves.  We can also see that there are 7 women who are most important and 3 who are least important (in the sense of how many other women they possibly know).   Thus, if we wanted to get a message to all of the women, which club should we broadcast to assuming that the women will propagate the idea?
Can you write Python code for it?


I am working on the file "RFray_IS620_WK6.ipynb".
The first task is:
The Women's graph seems to imply that all of the women are connected to each other, that is that all of the women share atleast one group in common.  We can check this below:
Can you write Python code for it?

for i in W.nodes_iter():
    if (len(W.neighbors(i))) != 18:
        print i + " --- " + str(len(W.neighbors(i)))


I am working on the file "RFray_IS620_WK6.ipynb".
The first task is:
With the clubs, we can see that clubs E6-E9 have members in common with all other clubs and thus have the greatest reach
Can you write Python code for it?

def getneigh(W,G):
    for i in W.nodes_iter():
        print i + " --- " + str(len(G.neighbors(i))) + " --- " + str(G.neighbors(i))

getneigh(W,G)

getneigh(C,G)


I am working on the file "RKHS_in_Machine_learning.ipynb".
The first task is:
In the Feature Space (the domain of $\FM$), we can then use linear algebra, such as angles, norms and inner products, inducing nonlinear operations on the Input Space (codomain of $\FM$). The central thing we need, apart from the feature space being a vector space are inner products, as they induce norms and a possibility to measure angles.
Simple classification algorithm using only inner products
Say we are given data points from the mixture of two distributions with densities $p_0,p_1$:
$$x_i \sim w_0 p_0 + w_1 p_1$$ and labels $l_i = 0$ if $x_i$ was actually generated by $p_0$, $l_i = 1$ otherwise. A very simple classification algorithm would be to compute the mean in feature space $\mu_c = \frac{1}{N_c} \sum_{l_i = c} \FM(x_i)$ for $c \in {0,1}$ and assign a test point to the class which is the most similar in terms of inner product. In other words, the decision function
$ f_d:\IS\to{0,1}$ is defined by
$$f_d(x) = \argmax_{c\in{0,1}} \prodDot{\FM(x)}{\mu_c}$$
Can you write Python code for it?

data = np.vstack([stats.multivariate_normal(np.array([-2,2]), np.eye(2)*1.5).rvs(100),
                  stats.multivariate_normal(np.ones(2)*2, np.eye(2)*1.5).rvs(100)])
distr_idx = np.r_[[0]*100, [1]*100]

for (idx, c, marker) in [(0,'r', (0,3,0)), (1, "b", "x")]:
    pl.scatter(*data[distr_idx==idx,:].T, c=c, alpha=0.4, marker=marker)
    pl.arrow(0, 0, *data[distr_idx==idx,:].mean(0), head_width=0.2, head_length=0.2, fc=c, ec=c)
pl.show()


I am working on the file "RKHS_in_Machine_learning.ipynb".
The first task is:
Obviously, the linear kernel might be enough already for this simple dataset. Another interesting observation however is that the Student-t based kernel is more robust to outliers of the datasets and yields a lower variance classification algorithm as compared to using a Gaussian kernel. This is to be expected, given the fatter tails of the Student-t. Now lets look at a dataset that is not linearly separable.
Can you write Python code for it?

data, distr_idx = sklearn.datasets.make_circles(n_samples=400, factor=.3, noise=.05)

for (kern_name, kern) in [("Linear", LinearKernel()),
                          ("Stud", StudentKernel(0.1,10)),
                          ("Gauss1", GaussianKernel(0.1)),
                         ]:
    (sim, dec) = kernel_mean_inner_prod_classification(data[distr_idx==1,:], data[distr_idx==0,:], kern)
    plot_with_contour(data, distr_idx, sim, 'Inner Product classif. '+kern_name, pl = plt)


I am working on the file "RKHS_in_Machine_learning.ipynb".
The first task is:
Remarkably, all positive definite functions are inner products in some feature space.
Theorem Let $\IS$ be a nonempty set and let $\PDK:\IS\times\IS \to \Reals$, called a kernel. The following two conditions are equivalent:
* $\PDK$ is symmetric and positive semi definite (psd), i.e. for all $x_1, \dots, x_m \in \IS$ the matrix $\Gram$ defined by with entries $\Gram_{i,j} = \PDK(x_i, x_j)$ is symmetric psd
$\FM$ is called the Feature Map and $\RKHS_\FM$ the feature space.
* there exists a map $\FM: \IS \to \RKHS_\FM$ to a hilbert space $\RKHS_\FM$ such that $$\PDK(x_i, x_j) = \prodDot{\FM(x_i)}{\FM(x_j)}_\RKHS$$
In other words, $\PDK$ computes the inner product in some $\RKHS_\FM$. We furthermore endow the space with the norm induced by the dot product $\|\cdot\|_\PDK$. From the second condition, it is easy to construct $\PDK$ given $\FM$. A general construction for $\FM$ given $\PDK$ is not as trivial but still elementary.
Construction of the canonical feature map (Aronszajn map)
We give the canonical construction of $\FM$ from $\PDK$, together with a definition of the inner product in the new space. In particular, the feature for each $x \in \IS$ will be a function from $\IS$ to $\Reals$.
$$\FM:\IS \to \Reals^\IS\
\FM(x) = \PDK(\cdot, x)$$
Thus for the linear kernel $\PDK(x,y)=\prodDot{x}{y}$ we have $\FM(x) = \prodDot{\cdot}{x}$ and for the gaussian kernel $\PDK(x,y)=\exp\left(-0.5{\|x-y\|^2}/{\sigma^2}\right)$ we have $\FM(x) = \exp\left(-0.5{\|\cdot -x \|^2}/{\sigma^2}\right)$.
Now $\RKHS$ is the closure of $\FM(\IS)$ wrt. linear combinations of its elements:
$$\RKHS = \left{f: f(\cdot)=\sum_{i=1}^m a_i \PDK(\cdot, x_i) \right} = span(\FM(\IS))$$
where $m \in \Nats, a_i \in \Reals, x \in \IS$. This makes $\RKHS$ a vector space over $\Reals$.
For $f(\cdot)=\sum_{i=1}^m a_i \PDK(\cdot, x_i)$ and $g(\cdot)=\sum_{i=1}^m' b_j \PDK(\cdot, x'j)$ we define the inner product in $\RKHS$ as
$$\prodDot{f}{g} = \sum{i=1}^m \sum_{i=1}^m' b_j a_i \PDK(x'_j, x_i)$$
In particular, for $f(\cdot) = \PDK(\cdot,x), g(\cdot)...
Can you write Python code for it?

class Kernel(object):
    def mean_emb(self, samps):
        return lambda Y: self.k(samps, Y).sum()/len(samps)

    def mean_emb_len(self, samps):
        return self.k(samps, samps).sum()/len(samps**2)

    def k(self, X, Y):
        raise NotImplementedError()

class FeatMapKernel(Kernel):
    def __init__(self, feat_map):
        self.features = feat_map

    def features_mean(self, samps):
        return self.features(samps).mean(0)

    def mean_emb_len(self, samps):
        featue_space_mean = self.features_mean(samps)
        return featue_space_mean.dot(featue_space_mean)

    def mean_emb(self, samps):
        featue_space_mean = self.features(samps).mean(0)
        return lambda Y: self.features(Y).dot(featue_space_mean)

    def k(self, X, Y):
        gram = self.features(X).dot(self.features(Y).T)
        return gram

class LinearKernel(FeatMapKernel):
    def __init__(self):
        FeatMapKernel.__init__(self, lambda x: x)

class GaussianKernel(Kernel):
    def __init__(self, sigma):
        self.width = sigma

    def k(self, X, Y=None):
        assert(len(np.shape(X))==2)

        # if X=Y, use more efficient pdist call which exploits symmetry
        if Y is None:
            sq_dists = squareform(pdist(X, 'sqeuclidean'))
        else:
            assert(len(np.shape(Y))==2)
            assert(np.shape(X)[1]==np.shape(Y)[1])
            sq_dists = cdist(X, Y, 'sqeuclidean')

        K = exp(-0.5 * (sq_dists) / self.width ** 2)
        return K

class StudentKernel(Kernel):
    def __init__(self, s2, df):
        self.dens = dist.mvt(0,s2,df)

    def k(self, X,Y=None):
        if Y is None:
            sq_dists = squareform(pdist(X, 'sqeuclidean'))
        else:
            assert(len(np.shape(Y))==2)
            assert(np.shape(X)[1]==np.shape(Y)[1])
            sq_dists = cdist(X, Y, 'sqeuclidean')
        dists = np.sqrt(sq_dists)
        return exp(self.dens.logpdf(dists.flatten())).reshape(dists.shape)

def...


I am working on the file "RL-Notebooks/Part1_1.ipynb".
The first task is:
The Agent
The code below established our simple neural agent. It consists of a set of values for each of the bandits. Each value is an estimate of the value of the return from choosing the bandit. We use a policy gradient method to update the agent by moving the value for the selected action toward the recieved reward.
Can you write Python code for it?

tf.reset_default_graph()

weights = tf.Variable(tf.ones([num_bandits]))
chosen_action = tf.argmax(weights, 0)

reward_holder = tf.placeholder(shape = [1], dtype = tf.float32)
action_holder = tf.placeholder(shape = [1], dtype = tf.int32)
responsible_weight = tf.slice(weights, action_holder, [1])
loss = -(tf.log(responsible_weight)* reward_holder)
optimizer = tf.train.GradientDescentOptimizer(learning_rate = 0.001)
update = optimizer.minimize(loss)


I am working on the file "RL-Notebooks/Part1_1.ipynb".
The first task is:
Training the Agent
We will train our agent by taking actions in our environment, and recieving rewards. Using the rewards and actions, we can know how to properly update our network in order to more often choose actions that will yield the highest rewards over time.
Can you write Python code for it?

total_episodes = 1000 #Set total number of episodes to train agent on.
total_reward = np.zeros(num_bandits) #Set scoreboard for bandits to 0.
e = 0.1 #Set the chance of taking a random action.

init = tf.initialize_all_variables()

# Launch the tensorflow graph
with tf.Session() as sess:
    sess.run(init)
    i = 0
    while i < total_episodes:

        #Choose either a random action or one from our network.
        if np.random.rand(1) < e:
            action = np.random.randint(num_bandits)
        else:
            action = sess.run(chosen_action)

        reward = pullBandit(bandits[action]) #Get our reward from picking one of the bandits.

        #Update the network.
        _,resp,ww = sess.run([update,responsible_weight,weights], feed_dict={reward_holder:[reward],action_holder:[action]})

        #Update our running tally of scores.
        total_reward[action] += reward
        if i % 50 == 0:
            print("Running reward for the " + str(num_bandits) + " bandits: " + str(total_reward))
        i+=1
print("The agent thinks bandit " + str(np.argmax(ww)+1) + " is the most promising....")
if np.argmax(ww) == np.argmax(-np.array(bandits)):
    print("...and it was right!")
else:
    print("...and it was wrong!")


I am working on the file "RL-Notebooks/Part1_2.ipynb".
The first task is:
The Policy-Based Agent
The code below established our simple neural agent. It takes as input the current state, and returns an action. This allows the agent to take actions which are conditioned on the state of the environment, a critical step toward being able to solve full RL problems. The agent uses a single set of weights, within which each value is an estimate of the value of the return from choosing a particular arm given a bandit. We use a policy gradient method to update the agent by moving the value for the selected action toward the recieved reward.
Can you write Python code for it?

class agent():
    def __init__(self, lr, s_size,a_size):
        #These lines established the feed-forward part of the network. The agent takes a state and produces an action.
        self.state_in= tf.placeholder(shape=[1],dtype=tf.int32)
        state_in_OH = slim.one_hot_encoding(self.state_in,s_size)
        output = slim.fully_connected(state_in_OH,a_size,\
            biases_initializer=None,activation_fn=tf.nn.sigmoid,weights_initializer=tf.ones_initializer())
        self.output = tf.reshape(output,[-1])
        self.chosen_action = tf.argmax(self.output,0)

        #The next six lines establish the training proceedure. We feed the reward and chosen action into the network
        #to compute the loss, and use it to update the network.
        self.reward_holder = tf.placeholder(shape=[1],dtype=tf.float32)
        self.action_holder = tf.placeholder(shape=[1],dtype=tf.int32)
        self.responsible_weight = tf.slice(self.output,self.action_holder,[1])
        self.loss = -(tf.log(self.responsible_weight)*self.reward_holder)
        optimizer = tf.train.GradientDescentOptimizer(learning_rate=lr)
        self.update = optimizer.minimize(self.loss)

tf.reset_default_graph() #Clear the Tensorflow graph.

cBandit = contextual_bandit() #Load the bandits.
myAgent = agent(lr=0.001,s_size=cBandit.num_bandits,a_size=cBandit.num_actions) #Load the agent.
weights = tf.trainable_variables()[0] #The weights we will evaluate to look into the network.

total_episodes = 10000 #Set total number of episodes to train agent on.
total_reward = np.zeros([cBandit.num_bandits,cBandit.num_actions]) #Set scoreboard for bandits to 0.
e = 0.1 #Set the chance of taking a random action.

init = tf.initialize_all_variables()

# Launch the tensorflow graph
with tf.Session() as sess:
    sess.run(init)
    i = 0
    while i < total_episodes:
        s = cBandit.getBandit() #Get a state from the environment.

        #Choose either a random action or one from our network.
        if np.random.rand(1) < e:
            action =...


I am working on the file "RL/CH 10.6_PyTorch_PG.ipynb".
The first task is:
(2) 강화학습에 필요한 공통 기능 구현
Can you write Python code for it?

def list_norm_inplace(buff):
    r_mean = np.mean(buff)
    r_std = np.std(buff)
    for ii in range(len(buff)):
        buff[ii] = (buff[ii] - r_mean) / r_std

def plot_durations(episode_durations):
    plt.figure(2)
    plt.clf()
    durations_t = TC.FloatTensor(episode_durations)
    plt.title('Training...')
    plt.xlabel('Episode')
    plt.ylabel('Duration')
    plt.plot(durations_t.numpy())
    # Take 100 episode averages and plot them too
    if len(durations_t) >= 100:
        means = durations_t.unfold(0, 100, 1).mean(1).view(-1)
        means = TC.cat((TC.zeros(99), means))
        plt.plot(means.numpy())
    plt.show()

def plot_durations_ii(ii, episode_durations, ee, ee_duration=100):
    episode_durations.append(ii+1)
    if (ee + 1) % ee_duration == 0:
        clear_output()
        plot_durations(episode_durations)


I am working on the file "RL/CH 10.6_PyTorch_PG.ipynb".
The first task is:
(3) 정책 경사에 필요한 기반 기능 구현
상태 정보($s_t$)를 입력해 행동에 대한 정책을 결정하는 요소인 정책 확률을 출력하는 함수형 클래스를 구현한다.
$$p_t (a_t=1|s_t) = f_{\mathrm{PGNET}}(s_t)$$
이 정책 확률은 Bernoulli 확률 분포함수를 나타내는 torch.distributions.Bernoulli라는 함수형 클래스에 의해 정책으로 변환된다.
$$\pi_t (a_t=1|s_t) = P_{\mathrm{Bernoulli}}(p_t (a_t=1|s_t))$$
Can you write Python code for it?

class PGNET(nn.Module):
    # 순수하게 Policy gradient로 구성
    def __init__(self, num_state):
        super(PGNET, self).__init__()

        self.fc_in = nn.Linear(num_state, 24)
        self.fc_hidden = nn.Linear(24, 36)
        self.fc_out = nn.Linear(36, 1)

    def forward(self, x):
        x = F.relu(self.fc_in(x))
        x = F.relu(self.fc_hidden(x))
        x = TC.sigmoid(self.fc_out(x))
        return x


I am working on the file "RL/CH 10.6_PyTorch_PG.ipynb".
The first task is:
(4) 정책 경사 방법의 강화학습을 위한 수행 객체
Can you write Python code for it?

class PGNET_AGENT(PGNET):
    # Policy gradient를 실제 사용하기 위해 필요한 변환을 고려함
    def __init__(self, num_state, render_flag=False):
        self.forget_factor = 0.99
        self.learning_rate = 0.01
        self.num_episode = 5000
        self.num_batch = 5
        self.render_flag = render_flag
        self.steps_in_batch = 0
        self.episode_durations = []

        super().__init__(num_state)
        self.optimizer = TC.optim.RMSprop(self.parameters(), lr=self.learning_rate)
        self.init_buff()

    def forward(self, state):
        state_var = Variable(TC.from_numpy(state).float())
        prob = super().forward(state_var)
        # check this function to know what the meaing of log_prob. Is it log(prob)?
        return Bernoulli(prob)

    def push_buff_done(self, reward, state, action, done_flag=False):
        if done_flag: # devide between episodes
            self.reward_buff.append(0)
        else:
            self.reward_buff.append(reward)
        self.state_buff.append(state)
        self.action_buff.append(action)

    def pull_buff(self, ii):
        return self.reward_buff[ii], self.state_buff[ii], self.action_buff[ii]

    def init_buff(self):
        self.reward_buff = []
        self.state_buff = []
        self.action_buff = []

    def transform_discount_reward(self, steps):
        future_reward = 0
        for ii in reversed(range(steps)):
            if self.reward_buff[ii] == 0:
                future_reward = 0
            else:
                future_reward = future_reward * self.forget_factor + self.reward_buff[ii]
                self.reward_buff[ii] = future_reward
        list_norm_inplace(self.reward_buff)

    def train(self, steps):
        self.transform_discount_reward(steps)
        self.optimizer.zero_grad()
        for ii in range(steps):
            reward, state, action = self.pull_buff(ii)
            action_var = Variable(TC.FloatTensor([float(action)]))
            policy =...


I am working on the file "RL/CH 10.6_PyTorch_PG.ipynb".
The first task is:
(5) 수행 객체 호출 함수 및 호출 명령 수행
Can you write Python code for it?

env = gym.make('CartPole-v0')
a_agent = PGNET_AGENT(env.observation_space.shape[0], render_flag=False)
a_agent.run(env)
env.close()


I am working on the file "RL/CH 10.6_PyTorch_PG.ipynb".
The first task is:
(5) 수행 객체 호출 함수 및 호출 명령 수행
먼저 환경을 만든다. 환경은 GYM을 사용한다. GYM은 공개 소프트웨어로 강화학습을 위한 다양한 환경을 제공한다. 여기서는 CartPole이라는 환경을 사용한다.
- GYM을 구할 수 있는 위치: https://gym.openai.com
다음은 환경을 통해 자신을 학습시키는 에이전트의 인스턴스를 학습한다.
그리고 생성한 에이전트를 수행한다. 여기서의 수행은 인스턴스의 학습을 의미한다. 추후로는 학습을 하고 그 후에 태스트를 하는 것까지를 포함할 예정이다.
최종적으로는 만든 환경을 닫는다.
Can you write Python code for it?


I am working on the file "RL/CH 10.6_PyTorch_PG.ipynb".
The first task is:
10.6.2 기반 기능 구현
(2) 강화학습에 필요한 공통 기능 구현
Can you write Python code for it?


I am working on the file "RL/CH 10.6_PyTorch_PG.ipynb".
The first task is:
10.6.3 메인 기능 구현
(4) 정책 경사 방법의 강화학습을 위한 수행 객체
이 코드는 파이썬의 클래스 기능을 이용해 강화학습 에이전트를 만듭니다. 에이전트는 환경과의 시행착오 과정을 통해 자신의 행동에 대한 정책을 점점 학습해 나갑니다. 그리고 여기서는 코드의 나열 순서가 아니라 코드의 동작을 이해하기 쉬운 순서대로 설명합니다. 클래스 멤버 함수가 없거나 함수 내부가 비어 있어도 실제 동작에는 필요한 코드가 있으니 코드 작성시는 전체 코드를 참조하기 바랍니다.
<br>
에이전트 실행을 위한 멤버 함수 run()을 구현한다. run()은 외부에서 부르는 실행을 위한 멤버 함수입니다. 이 함수는 두개의 멤버 함수를 부릅니다.
첫 번째가 한번의 에피소드를 수행하는 self.run_episode()입니다. 강화학습은 학습 과정에서 이 과정을 시행착오라고 합니다. 지도학습은 이미 만들어진 데이터를 통해 학습하는 경우가 일반적이기 때문에 시행착오 과정이 존재하지 않는 것이 대부분입니다. 만약 지도학습에서 데이터를 받아들이는 과정이 있다면 반드시 받아들인 각 데이터에 대해 레이블링 과정이 필요합니다. 이 레이블링은 주로 사람에 의해 이루어지기 때문에 데이터를 받아들이는 과정이 있다면 굉장히 비효율적으로 동작하게 됩니다.
두 번째가 self.train_episode()가 담당하는 학습 과정입니다. 시행착오 과정에서 만들어진 경험 데이터를 통해 학습이 진행됩니다. 강화학습에서는 레이블 정보 대신에 리워드를 사용합니다. 리워드는 에피소드가 수행되는 과정에서 마지막 한번 또는 중간중간에 계속해서 구해질 수 있습니다. 한번의 에피소드안에 여러번의 반복이 시도가 됨으로 각 반복시점에 생성된 데이터는 다음 3가지로 구분할 수 있습니다.
$${s_k, r_k, F^\mathrm{Done}_k |k=1..K_i}$$
여기서 $s_k$는 $k$회에서 상태, $r_k$는 $k$회에서 리워드, 그리고 $F^\mathrm{Done}_k$는 $k$회에서 완료 여부를 나타내는 플래그이다.
Can you write Python code for it?

class PGNET_AGENT(PGNET):
    def run(self, env):
        for ee in range(self.num_episode):
            self.run_episode(env, ee)
            self.train_episode(ee)


I am working on the file "RL/CH 10.6_PyTorch_PG.ipynb".
The first task is:
<br>
기본적인 라이브러리들을 불러온다.아래 중에서 itertools에 있는 count()는 불특정 횟수의 반복을 위해 사전에 정해진 한계가 없이 계속 반복하며 반복 횟수를 돌려준다. 강화학습의 공개 환경 라이브러리는 done이라는 신호를 통해 끝을 알려주기 때문에 count()로 인해 무한번 반복하게 되는 상황을 방지할 수 있다.
Can you write Python code for it?

from IPython.display import clear_output
import random
from itertools import count
import numpy as np
import matplotlib.pyplot as plt


I am working on the file "RL/CH 10.6_PyTorch_PG.ipynb".
The first task is:
<br>
파이토치를 사용하기 위해 필요한 라이브러리들을 불러들이다. 파이토치 라이브러리의 대표 명칭은 torch이다.
Can you write Python code for it?

# Using torch
import torch as TC
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
from torch.distributions import Bernoulli


I am working on the file "RL/CH 10.6_PyTorch_PG.ipynb".
The first task is:
전체 코드
예제 10-5 파이토치 - Cartpole을 다루는 PG 전체 코드
(1) 라이브러리 불러들이기
Can you write Python code for it?

# 그래픽 처리를 위한 확장자 실행
%matplotlib inline

# 기본 라이브러리 불러들이기
from IPython.display import clear_output
import random
from itertools import count
import numpy as np
import matplotlib.pyplot as plt

# 강화학습 환경 관리 라이브러리 불러들이기
import gym

# 파이토치 라이브러리 불러들이기
import torch as TC
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
from torch.distributions import Bernoulli


I am working on the file "RL/PyTorch_PG.ipynb".
The first task is:
III. 메인 코드
정책 경사 방법의 강화학습을 위한 수행 객체
Can you write Python code for it?


I am working on the file "RL/PyTorch_PG.ipynb".
The first task is:
강화학습 환경 관리 라이브러리 불러들이기
Can you write Python code for it?


I am working on the file "RL/PyTorch_PG.ipynb".
The first task is:
정책경사를 위한 기반 기능 구현
상태 정보($s_t$)를 입력해 행동에 대한 정책을 결정하는 요소인 정책 확률을 출력하는 함수형 클래스를 구현한다.
$$p_t (a_t=1|s_t) = f_{\mathrm{PGNET}}(s_t)$$
이 정책 확률은 Bernoulli 확률 분포함수를 나타내는 torch.distributions.Bernoulli라는 함수형 클래스에 의해 정책으로 변환된다.
$$\pi_t (a_t=1|s_t) = P_{\mathrm{Bernoulli}}(p_t (a_t=1|s_t))$$
Can you write Python code for it?


I am working on the file "RL/PyTorch_PG.ipynb".
The first task is:
파이토치 라이브러리 불러들이기
Can you write Python code for it?


I am working on the file "RL_and_RLC_circuit/.ipynb_checkpoints/RLC_circuit_current_v2-checkpoint.ipynb".
The first task is:
RLC Circuit with sinusoidal voltage
Now voltage and its derivative over time (not equal to zero in this case) are given as:
<img src="formula_3.png">
Can you write Python code for it?

ub = [sqrt(2)*u*sin(omega*k + alpha) for k in t]

# definition of the function dp/dt

def di(y,t):
    #x = i, p = di/dt
    x, p = y[0], y[1]

    dx = p
    dp = 1/L*(omega*sqrt(2)*u*cos(omega*t + alpha)-R*p-(1/C)*x)

    return [dx, dp]

#initial state

#initial capacitor voltage
uc0 = 0

y0 = [0.0, 1/L*(ua[0]-uc0)]

I = odeint(di, y0, t)

ib = I[:,0]

#Capacitor voltage derivative
duc2 = ib/C

uc2 = cumtrapz(duc2, dx=0.4/1000, initial=0)

fig, ax = plt.subplots(nrows=2, ncols=1, figsize=(8,8))

ax[0].plot(t,ib, label="Current")
ax[0].set_ylabel("Current [A]")
ax[0].set_xlabel("Time [s]")
ax[0].set_title("Current in R-L-C circuit during switch-on")
ax[0].legend()

ax[1].plot(t,ub, label="Line voltage", color="green")
ax[1].plot(t,uc2, label="Capacitor voltage", color="orange")
ax[1].set_ylabel("Voltage [V]")
ax[1].set_xlabel("Time [s]")
ax[1].set_title("Supply voltage")
ax[1].legend()

fig.tight_layout()

#checking the amplitude value in steady state

Im = sqrt(2)*u/(sqrt(R**2+(XL-XC)**2))
Im


I am working on the file "RL_and_RLC_circuit/.ipynb_checkpoints/RLC_circuit_current_v2-checkpoint.ipynb".
The first task is:
RLC circuit fed with dc voltage
For a dc voltage case it has a constant value, so its derivative over time is equal to zero:

Consequently our equation becomes:

Voltage is given as:
Can you write Python code for it?

ua = [u for k in t]

#definition of the function dp/dt

def di(y,t):
    #x = i, p = di/dt
    x, p = y[0], y[1]

    dx = p
    dp = 1/L*(-R*p-(1/C)*x)

    return [dx, dp]

#initial state

#initial capacitor voltage
uc0 = 0

y0 = [0.0, 1/L*(u-uc0)]

y0

I = odeint(di, y0, t)

ia = I[:,0]

# Capacitor voltage definition:
duc = ia/C

uc = cumtrapz(duc, dx=0.4/1000, initial=0)

# after integration vectors t and uc had different lengths so I need to append one item
np.append(uc, uc[999])

fig, ax = plt.subplots(nrows=2, ncols=1, figsize=(8,8))

ax[0].plot(t,ia, label="Current")
ax[0].set_ylabel("Current [A]")
ax[0].set_xlabel("Time [s]")
ax[0].set_title("Current in R-L-C circuit during switch-on")
ax[0].legend()

ax[1].plot(t,ua, label="Supply voltage", color="green")
ax[1].plot(t,uc, label="Capacitor voltage", color="orange")
ax[1].set_ylabel("Voltage [V]")
ax[1].set_xlabel("Time [s]")
ax[1].set_title("Supply voltage")
ax[1].legend()

fig.tight_layout()

#checking damping factor: if below 1 - underdamped, if above 1 - overdamped
damp = (R/2)*sqrt(C/L)
damp


I am working on the file "RL_and_RLC_circuit/RLC_circuit_current_v2.ipynb".
The first task is:
RLC Circuit with sinusoidal voltage
Now voltage and its derivative over time (not equal to zero in this case) are given as:
<img src="formula_3.png">
Can you write Python code for it?


I am working on the file "RL_and_RLC_circuit/RLC_circuit_current_v2.ipynb".
The first task is:
RLC circuit fed with dc voltage
For a dc voltage case it has a constant value, so its derivative over time is equal to zero:

Consequently our equation becomes:

Voltage is given as:
Can you write Python code for it?


I am working on the file "RL_and_RLC_circuit/RLC_circuit_current_v2.ipynb".
The first task is:
RLC circuit is governed by the following formulas:

To put the last equation in order:

This will be a starting point for the analysis, which includes two cases:

RLC circuit fed with a dc voltage
RLC circuit fed with an ac voltage

First we need to define auxiliary variables
Can you write Python code for it?

# RMS value of voltage
u = 230

#time vector
t = np.linspace(0,0.4, 1000)

#frequency & angular frequency
f = 50
omega = 2 * pi * f

#Resitance (values to consider 5 and 10 Ohms)
R = 5

#Inductance
L = 0.1
XL = 2*pi*f*L

#Capacitance (worth to consider 0.01 - two inertia or 0.001 - oscillator)
C = 0.01
XC = 1/(omega*C)

#Phase angle
phi=atan((XL-XC)/R)

#closing angle [rad]
alpha = 0

XL, XC


I am working on the file "RM.ipynb".
The first task is:
An alternative way to visualize Wannier flow of a periodic system is shown below. The inner circle represent $t/T=0$ and the outer $t/T=1$, the sections of the disc correspond to unitcells.
Can you write Python code for it?

figsize(6,6)
CFwan.polar_w_vs_t('Lower band','ko',ms=10)
CFwan.polar_w_vs_t('Higher band','o',mec='grey',mfc='grey')
legend(numpoints=100,fontsize=15,ncol=2,bbox_to_anchor=(1,0));


I am working on the file "RM.ipynb".
The first task is:
Below we write a generic function that takes the functions $u(t)$,$v(t)$ and $w(t)$ as an argument and then visualizes the pumping process in $d$-space. We will use this function to explore the control freak sequence and the later on also the not so control freak sequence.
Can you write Python code for it?

def seq_and_d(funcs,ti=10):
    '''
    A figure generating function for the Rice Mele model.
    It plots the functions defining the sequence and the d-space structure.
    '''
    figsize(10,5)
    fig=figure()
    func=eval(funcs);
    ax1=fig.add_subplot(121)
    ftsz=20
    # plotting the functions defining the sequence
    plot(tran[:,0],func(tran[:,0])[1],'k-',label=r'$v$',linewidth=3)
    plot(tran[:,0],func(tran[:,0])[2],'g--',label=r'$w$',linewidth=3)
    plot(tran[:,0],func(tran[:,0])[0],'m-',label=r'$u$',linewidth=3)
    plot([tran[ti,0],tran[ti,0]],[-3,3],'r-',linewidth=3)
    # this is just to make things look like in the book
    ylim(-1.5,2.5)
    legend(fontsize=20,loc=3)
    xlabel(r'time $t/T$',fontsize=ftsz)
    xticks(linspace(0,1,5),[r'$0$',r'$0.25$',r'$0.5$',r'$0.75$',r'$1$'],fontsize=ftsz)
    ylabel(r'amplitudes $u,v,w$',fontsize=ftsz)
    yticks([-1,0,1,2],[r'$-1$',r'$0$',r'$1$',r'$2$'],fontsize=ftsz)
    grid(True)

    ax2=fig.add_subplot(122, projection='3d')
    # plotting d space image of the pumping sequence
    plot(*dkt(kran[ti,:],tran[ti,:],func),marker='o',mec='red',mfc='red',ls='-',lw=6,color='red')
    plot(*dkt(kran.flatten(),tran.flatten(),func),color='blue',alpha=0.5)
    # this is just to make things look like in the book
    # basically everything below is just to make things look nice..
    ax2.w_xaxis.set_pane_color((1.0, 1.0, 1.0, 1.0))
    ax2.w_yaxis.set_pane_color((1.0, 1.0, 1.0, 1.0))
    ax2.w_zaxis.set_pane_color((1.0, 1.0, 1.0, 1.0))
    ax2.set_axis_off()
    ax2.grid(False)
    arrprop=dict(mutation_scale=20, lw=1,arrowstyle='-|>,head_length=1.4,head_width=0.6',color="k")
    ax2.add_artist(Arrow3D([-2,4],[0,0],[0,0], **arrprop))
    ax2.add_artist(Arrow3D([0,0],[-2,3.3],[0,0], **arrprop))
    ax2.add_artist(Arrow3D([0,0],[0,0],[-1,2], **arrprop))
    ftsz2=30
    ax2.text(4.4, -1,  0, r'$d_x$', None,fontsize=ftsz2)
    ax2.text(0.3, 3.0, 0, r'$d_y$', None,fontsize=ftsz2)
    ax2.text(0, 0.6, 2.0, r'$d_z$', None,fontsize=ftsz2)
   ...


I am working on the file "RM.ipynb".
The first task is:
Finally we write a simple function to visualize the spectrum and the wavefunctions in a symmilar fashion as we did for the SSH model. We shall now explicitly mark the edge states in the spectrum with red and blue.
Can you write Python code for it?

def enpsi(PD,ti=10,n=10):
    figsize(14,5)
    subplot(121)
    lcol='#53a4d7'
    rcol='#d7191c'
    # Plotting the eigenvalues and
    # a marker showing for which state
    # we are exploring the wavefunction
    plot(tran[:,0],PD.dat,'k-');
    (lambda x:plot(x[:,0],x[:,1],'o',mec=lcol,mfc=lcol,
                   markersize=10))(reshape(PD.lefty,(PD.lefty.size/2,2)))
    (lambda x:plot(x[:,0],x[:,1],'o',mec=rcol,mfc=rcol,
                   markersize=10))(reshape(PD.righty,(PD.righty.size/2,2)))
    plot(tran[ti,0],PD.dat[ti,n],'o',markersize=13,mec='k',mfc='w')

    # Make it look like the book
    xlabel(r'$t/T$',fontsize=25);
    xticks(linspace(0,1,5),fontsize=25)
    ylabel(r'energy $E$',fontsize=25);
    yticks(fontsize=25)
    ylim(-2.99,2.99)
    grid()

    subplot(122)
    # Plotting the sublattice resolved wavefunction
    bar(array(range(0,2*PD.L,2)),  real(array(PD.vecdat[ti][0::2,n].T)),0.9,color='grey',label='A')  # sublattice A
    bar(array(range(0,2*PD.L,2))+1,real(array(PD.vecdat[ti][1::2,n].T)),0.9,color='white',label='B') # sublattice B

    # Make it look like the book
    xticks(2*(array(range(10))),['     '+str(i) for i in array(range(11))[1:]],fontsize=25)
    ylim(-1.2,1.2)
    yticks(linspace(-1,1,5),fontsize=25,x=1.2)
    ylabel('Wavefunction',fontsize=25,labelpad=-460,rotation=-90)
    grid()
    legend(loc='lower right')
    xlabel(r'cell index $m$',fontsize=25);

    tight_layout()


I am working on the file "RM.ipynb".
The first task is:
Next we define a class that we will mainly use to hold data about our pumping sequence. The information in these objects will be used to visualize the spectrum and wavefunctions of bulk and edge localized states.
Can you write Python code for it?

class pumpdata:
    '''
    A class that holds information on spectrum and wavefunctions
    of a pump sequence performed on a finite lattice model.
    Default values are tailored to the control freak sequence.
    '''
    def __init__(self,L=10,numLoc=1,norm_treshold=0.99,func=uvwCF,**kwargs):
        '''
        Initialization function. The default values are set in such a way that they correspond
        to the control freak sequence.
        '''

        self.L=L
        self.dat=[]             # We will collect the data to be
        self.vecdat=[]          # plotted in these arrays.
        self.lefty=[]
        self.righty=[]
        self.lefty=[]
        self.righty=[]

        tlim=kwargs.get('edge_tlim',(0,1)) # We can use this to restrict classification
                                           #  of left and right localized states in time
        for t in tran[:,0]:
            u,v,w=func(t)         # obtain u(t),v(t) and w(t)
            H=H_RM_reals(L,u,v,w) #
            eigdat=eigh(H);       # for a given t here we calculate the eigensystem (values and vectors)
            if tlim[0]<t<tlim[1]:
                # for the interesting time intervall we look for states localized to the edge
                for i in range(2*L):
                    if sum((array(eigdat[1][0::2,i])**2+array(eigdat[1][1::2,i])**2)[0:2*numLoc:2])>norm_treshold:
                        self.lefty=append(self.lefty,[[t,eigdat[0][i]]]);
                    if sum((array(eigdat[1][0::2,i])**2+array(eigdat[1][1::2,i])**2)[:L-2*numLoc:-2])>norm_treshold:
                        self.righty=append(self.righty,[[t,eigdat[0][i]]]);

            self.dat=append(self.dat,eigdat[0]);
            self.vecdat=append(self.vecdat,eigdat[1]);

        self.dat=reshape(self.dat,[len(tran[:,0]),2*L]);          # rewraping the data
        self.vecdat=reshape(self.vecdat,[len(tran[:,0]),2*L,2*L]) # to be more digestable


I am working on the file "RM.ipynb".
The first task is:
Now let us create an instance of the above class with the data of the control freak pump sequence:
Can you write Python code for it?

# Filling up data for the control freak sequence
CFdata=pumpdata(edge_tlim=(0.26,0.74))


I am working on the file "RM.ipynb".
The first task is:
Now let us see what happends as time proceedes!
Can you write Python code for it?


I am working on the file "RM.ipynb".
The first task is:
We can now interact with the above function and see the evolution of the surface states.
Can you write Python code for it?


I am working on the file "RM.ipynb".
The first task is:
We have now done all the heavy lifting with regards of coding. Now we can reuse all the plotting and data generating classes and functions for other sequences.
Moving away from the control freak sequence
Let us now relax the control freak attitude and consider a model which is not strictly localized at all times!
Can you write Python code for it?

def uvwNSCF(t):
    '''
    The u,v and w functions of the not so control freak sequence.
    For the time beeing we assume vbar to be fixed.
    '''
    vbar=1
    return array([sin(t*(2*pi)),vbar+cos(t*(2*pi)),1*t**0])


I am working on the file "RNN - Multilabel.ipynb".
The first task is:
Helper methods
Addition of Dropout and/or other modification to model architecture can be made in LSTM function.
Can you write Python code for it?

def weight_variable(shape):
    initial = tf.truncated_normal(shape, stddev = 0.1)
    return tf.Variable(initial)

def bias_variable(shape):
    initial = tf.constant(0.0, shape = shape)
    return tf.Variable(initial)

def LSTM(x, weight, bias):
    cell = rnn_cell.LSTMCell(n_hidden,state_is_tuple = True)
    multi_layer_cell = tf.nn.rnn_cell.MultiRNNCell([cell] * 2)
    output, state = tf.nn.dynamic_rnn(multi_layer_cell, x, dtype = tf.float32)
    output_flattened = tf.reshape(output, [-1, n_hidden])
    output_logits = tf.add(tf.matmul(output_flattened,weight),bias)
    output_all = tf.nn.sigmoid(output_logits)
    output_reshaped = tf.reshape(output_all,[-1,n_steps,n_classes])
    output_last = tf.gather(tf.transpose(output_reshaped,[1,0,2]), n_steps - 1)
    #output = tf.transpose(output, [1, 0, 2])
    #last = tf.gather(output, int(output.get_shape()[0]) - 1)
    #output_last = tf.nn.sigmoid(tf.matmul(last, weight) + bias)
    return output_last, output_all

weight = weight_variable([n_hidden,n_classes])
bias = bias_variable([n_classes])
y_last, y_all = LSTM(x,weight,bias)


I am working on the file "RNN - Multilabel.ipynb".
The first task is:
Hyperparameters Configuration
Can you write Python code for it?

tf.reset_default_graph()

learning_rate = 0.001
training_epochs = 100
batch_size = 10
total_batches = (train_x.shape[0]//batch_size)

n_input = 1
n_steps = 10
n_hidden = 64
n_classes = 3

alpha = 0.5


I am working on the file "RNN - Multilabel.ipynb".
The first task is:
Input/Output placeholders for Tensorflow graph
Can you write Python code for it?

x = tf.placeholder("float", [None, n_steps, n_input])
y = tf.placeholder("float", [None, n_classes])
y_steps = tf.placeholder("float", [None, n_classes])


I am working on the file "RNN - Multilabel.ipynb".
The first task is:
Loss function: Binary cross entropy and target replication
Loss function used in the paper is a combination of two losses 1) average loss of each time step prediction 2) loss of the prediction calculated at the last time step. Alpha in the combined loss function is a hyper-parameter. See the paper for more information on target replication and loss function.
Can you write Python code for it?

#all_steps_cost=tf.reduce_mean(-tf.reduce_mean((y_steps * tf.log(y_all))+(1 - y_steps) * tf.log(1 - y_all),reduction_indices=1))
all_steps_cost = -tf.reduce_mean((y_steps * tf.log(y_all))  + (1 - y_steps) * tf.log(1 - y_all))
last_step_cost = -tf.reduce_mean((y * tf.log(y_last)) + ((1 - y) * tf.log(1 - y_last)))
loss_function = (alpha * all_steps_cost) + ((1 - alpha) * last_step_cost)

optimizer = tf.train.AdamOptimizer(learning_rate = learning_rate).minimize(loss_function)


I am working on the file "RNN - Multilabel.ipynb".
The first task is:
Training and testing the model
Can you write Python code for it?

with tf.Session() as session:
    tf.global_variables_initializer().run()
    for epoch in range(training_epochs):
        for b in range(total_batches):
            offset = (b * batch_size) % (train_y.shape[0] - batch_size)
            batch_x = train_x[offset:(offset + batch_size), :]
            batch_y = train_y[offset:(offset + batch_size), :]
            batch_y_steps = np.tile(batch_y,((train_x.shape[1]),1))
            _, c = session.run([optimizer, loss_function],feed_dict={x: batch_x, y : batch_y, y_steps: batch_y_steps})
        pred_y = session.run(y_last,feed_dict={x:test_x})
        print("ROC AUC Score: ",roc_auc_score(test_y,pred_y))


I am working on the file "RNN Text Generation - Advanced.ipynb".
The first task is:
Evaluation
Informal evaluation. Use the trained model to generate N new sentences and print them out.
Can you write Python code for it?

#import logging
#logger = logging.getLogger()
#logger.setLevel(logging.DEBUG)

# instantiate generation class on our data
text_gen = TextGenModel(model, index_to_word, word_to_index, sent_max_len=sent_max_len,
                                    temperature=1.0,
                                    use_embeddings=True)
# generate N new sentences
n_sents = 10
print("Epoch {}".format(trained_epochs))
for _ in range(n_sents):
    res = text_gen.pretty_print_sentence(text_gen.get_sentence(15))
    print(res)


I am working on the file "RNN Text Generation - Advanced.ipynb".
The first task is:
Load Word-Embeddings
Notice that with the following approach we are considering only the vocabulary of our training dataset.
An improvements on this side would be to reinstantiate the model for testing including the entirety of the original word-embeddings, such that we can leverage also input words not present in our training vocabulary.
Can you write Python code for it?

embedding_size = 100
embeddings = preprocessing.load_embeddings(join(training_data_folder, "glove", "glove.6B.100d.txt"),
                                          word_to_index.keys())
embeddings_matrix = preprocessing.get_embeddings_matrix(embeddings,
                                                       word_to_index,
                                                        embedding_size)

embeddings_matrix.shape


I am working on the file "RNN Text Generation - Advanced.ipynb".
The first task is:
Model Training
We first consider a basic LSTM with embedding input layer.
Initially we will learn the embedding for scratch, and consider a many-to-many model (that is, sequence-in and sequence-out).
Possible improvements and stuff to try:
* try deep LSTM
* include attention
[TODO] Add considerations about stateful
Can you write Python code for it?

# model parameters
hidden_size = 512
#embedding_size = 128 # already defined when loading word-embeddings
batch_size = 64
stateful = False

# LSTM with embedding layer
model = Sequential()
model.add(Embedding(vocabulary_size, embedding_size,
                    #batch_input_shape=(batch_size, sent_max_len), # needed in case of stateful model
                    mask_zero=True,
                    weights=[embeddings_matrix], # use of pretained embeddings
                    trainable=False))
model.add(BatchNormalization())
#model.add(TimeDistributed(Flatten())) # not needed if proper batch_input_shape specified before
model.add(LSTM(hidden_size,
               return_sequences=True,
               stateful=stateful, # if stateful model, remember to avoid batches shuffling during training
              ))#activation='relu')) ??easily getting loss=nan if using RELU
model.add(TimeDistributed(Dense(vocabulary_size, activation='softmax')))

model.summary()

# compile model
model.compile(loss='sparse_categorical_crossentropy',
              optimizer='adam')

# keep track of trained epoch (for when we rerun a cell)
trained_epochs = 32

model.layers[0].trainable = True

num_epoch = 1
model.fit(X_train[:100000], # for stateful we need
          Y_train[:100000], # number of samples divisible by the batch size
          epochs=num_epoch,
          batch_size=batch_size,
          shuffle=not stateful # don't shuffle if stateful
         )
trained_epochs += num_epoch

model.optimizer.lr=1e-4

num_epoch = 10
model.fit(X_train, Y_train,
          epochs=num_epoch,
          batch_size=batch_size,
          shuffle=not stateful # don't shuffle if stateful
         )
trained_epochs += num_epoch


I am working on the file "RNN Text Generation - Advanced.ipynb".
The first task is:
TF Serving Client
Showcase of a basic client that can use the models served by a tf-serving server.
Can you write Python code for it?

from grpc.beta import implementations

# reference local copy of Tensorflow Serving API Files
sys.path.append(os.path.join(os.getcwd(), os.pardir, 'ext_libs'))
import lib.predict_pb2 as predict_pb2
import lib.prediction_service_pb2 as prediction_service_pb2

host='127.0.0.1'
port=9000
channel = implementations.insecure_channel(host, int(port))
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)

# build request
request = predict_pb2.PredictRequest()
request.model_spec.name = 'rnn' # model name, as given to bazel script
request.model_spec.signature_name = 'predict' # as defined in ModelBuilder

# define inputs
x = X_train[0]
x_tensor = tf.contrib.util.make_tensor_proto(x, dtype=tf.float32, shape=(1, 20,))
request.inputs['inputs'].CopyFrom(x_tensor)

# call prediction on the server
result = stub.Predict(request, timeout=10.0)

# get the output from server response
outputs = result.outputs['outputs']

# extract response-tensor shape
tensor_shape = outputs.tensor_shape
tensor_shape = [dim.size for dim in tensor_shape.dim]

# reshape list of float to given shape
res_tensor = np.array(outputs.float_val).reshape(tensor_shape)

res_tensor.shape


I am working on the file "RNN Text Generation - Advanced.ipynb".
The first task is:
Text Generation
TF Serving Client
We rely on the implementation of the tensorflow serving client as provided in this repository. It is just a refactoring of the previous code for better modularity and reuse.
Can you write Python code for it?

from model.servingClient import ServingClient
from model.textGenerator import TextGenerator

# instantiate text generator object
# with info about which model to use and related configs
text_gen = TextGenerator('/Users/amartinelli/Documents/models/models.ini',
             'darwin_tf',
            'standard_config')

# generate text
text_gen.generate(10)