I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
Hyper-Parameter Optimization
There are several steps required to do hyper-parameter optimization.
Create the Model
We first need a function that takes a set of hyper-parameters and creates the Convolutional Neural Network corresponding to those parameters. We use Keras to build the neural network in TensorFlow, see Tutorial #03-C for more details.
Can you write Python code for it?

def create_model(learning_rate, num_dense_layers,
                 num_dense_nodes, activation):
    """
    Hyper-parameters:
    learning_rate:     Learning-rate for the optimizer.
    num_dense_layers:  Number of dense layers.
    num_dense_nodes:   Number of nodes in each dense layer.
    activation:        Activation function for all layers.
    """

    # Start construction of a Keras Sequential model.
    model = Sequential()

    # Add an input layer which is similar to a feed_dict in TensorFlow.
    # Note that the input-shape must be a tuple containing the image-size.
    model.add(InputLayer(input_shape=(img_size_flat,)))

    # The input from MNIST is a flattened array with 784 elements,
    # but the convolutional layers expect images with shape (28, 28, 1)
    model.add(Reshape(img_shape_full))

    # First convolutional layer.
    # There are many hyper-parameters in this layer, but we only
    # want to optimize the activation-function in this example.
    model.add(Conv2D(kernel_size=5, strides=1, filters=16, padding='same',
                     activation=activation, name='layer_conv1'))
    model.add(MaxPooling2D(pool_size=2, strides=2))

    # Second convolutional layer.
    # Again, we only want to optimize the activation-function here.
    model.add(Conv2D(kernel_size=5, strides=1, filters=36, padding='same',
                     activation=activation, name='layer_conv2'))
    model.add(MaxPooling2D(pool_size=2, strides=2))

    # Flatten the 4-rank output of the convolutional layers
    # to 2-rank that can be input to a fully-connected / dense layer.
    model.add(Flatten())

    # Add fully-connected / dense layers.
    # The number of layers is a hyper-parameter we want to optimize.
    for i in range(num_dense_layers):
        # Name of the layer. This is not really necessary
        # because Keras should give them unique names.
        name = 'layer_dense_{0}'.format(i+1)

        # Add the dense / fully-connected layer to the model.
        # This has two hyper-parameters we...


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
Load Data
The MNIST data-set is about 12 MB and will be downloaded automatically if it is not located in the given dir.
Can you write Python code for it?

from mnist import MNIST
data = MNIST(data_dir="data/MNIST/")


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
NOTE: We will save and load models using Keras so you need to have h5py installed. You also need to have scikit-optimize installed for doing the hyper-parameter optimization.
You should be able to run the following command in a terminal to install them both:
pip install h5py scikit-optimize
NOTE: This Notebook requires plotting functions in scikit-optimize that have not been merged into the official release at the time of this writing. If this Notebook cannot run with the version of scikit-optimize installed by the command above, you may have to install scikit-optimize from a development branch by running the following command instead:
pip install git+git://github.com/Hvass-Labs/scikit-optimize.git@dd7433da068b5a2509ef4ea4e5195458393e6555
Can you write Python code for it?

import skopt
from skopt import gp_minimize, forest_minimize
from skopt.space import Real, Categorical, Integer
from skopt.plots import plot_convergence
from skopt.plots import plot_objective, plot_evaluations
from skopt.plots import plot_histogram, plot_objective_2D
from skopt.utils import use_named_args


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
Plot some of the mis-classified images.
Can you write Python code for it?


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
Plots
There are several plotting functions available in the skopt library. For example, we can plot a histogram for the activation parameter, which shows the distribution of samples during the hyper-parameter optimization.
Can you write Python code for it?

fig, ax = plot_histogram(result=search_result,
                         dimension_name='activation')


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
Predict on New Data
We can also predict the classification for new images. We will just use some images from the test-set but you could load your own images into numpy arrays and use those instead.
Can you write Python code for it?


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
Run the Hyper-Parameter Optimization
Now we are ready to run the actual hyper-parameter optimization using Bayesian optimization from the scikit-optimize package. Note that it first calls fitness() with default_parameters as the starting point we have found by hand-tuning, which should help the optimizer locate better hyper-parameters faster.
There are many more parameters you can experiment with here, including the number of calls to the fitness() function which we have set to 40. But fitness() is very expensive to evaluate so it should not be run too many times, especially for larger neural networks and datasets.
You can also experiment with the so-called acquisition function which determines how to find a new set of hyper-parameters from the internal model of the Bayesian optimizer. You can also try using another Bayesian optimizer such as Random Forests.
Can you write Python code for it?

%%time
search_result = gp_minimize(func=fitness,
                            dimensions=dimensions,
                            acq_func='EI', # Expected Improvement.
                            n_calls=40,
                            x0=default_parameters)


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
Test Run
Before we run the hyper-parameter optimization, let us first check that the various functions above actually work, when we pass the default hyper-parameters.
Can you write Python code for it?


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
The MNIST data-set has now been loaded and consists of 70.000 images and class-numbers for the images. The data-set is split into 3 mutually exclusive sub-sets. We will only use the training and test-sets in this tutorial.
Can you write Python code for it?

print("Size of:")
print("- Training-set:\t\t{}".format(data.num_train))
print("- Validation-set:\t{}".format(data.num_val))
print("- Test-set:\t\t{}".format(data.num_test))


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
Then we convert the predicted class-numbers from One-Hot encoded arrays to integers.
Can you write Python code for it?


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
These are the true class-number for those images. This is only used when plotting the images.
Can you write Python code for it?


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
This is the fitness value associated with these hyper-parameters. This is a negative number because the Bayesian optimizer performs minimization, so we had to negate the classification accuracy which is posed as a maximization problem.
Can you write Python code for it?


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
This is the function that creates and trains a neural network with the given hyper-parameters, and then evaluates its performance on the validation-set. The function then returns the so-called fitness value (aka. objective value), which is the negative classification accuracy on the validation-set. It is negative because skopt performs minimization instead of maximization.
Note the function decorator @use_named_args which wraps the fitness function so that it can be called with all the parameters as a single list, for example: fitness(x=[1e-4, 3, 256, 'relu']). This is the calling-style skopt uses internally.
Can you write Python code for it?

@use_named_args(dimensions=dimensions)
def fitness(learning_rate, num_dense_layers,
            num_dense_nodes, activation):
    """
    Hyper-parameters:
    learning_rate:     Learning-rate for the optimizer.
    num_dense_layers:  Number of dense layers.
    num_dense_nodes:   Number of nodes in each dense layer.
    activation:        Activation function for all layers.
    """

    # Print the hyper-parameters.
    print('learning rate: {0:.1e}'.format(learning_rate))
    print('num_dense_layers:', num_dense_layers)
    print('num_dense_nodes:', num_dense_nodes)
    print('activation:', activation)
    print()

    # Create the neural network with these hyper-parameters.
    model = create_model(learning_rate=learning_rate,
                         num_dense_layers=num_dense_layers,
                         num_dense_nodes=num_dense_nodes,
                         activation=activation)

    # Dir-name for the TensorBoard log-files.
    log_dir = log_dir_name(learning_rate, num_dense_layers,
                           num_dense_nodes, activation)

    # Create a callback-function for Keras which will be
    # run after each epoch has ended during training.
    # This saves the log-files for TensorBoard.
    # Note that there are complications when histogram_freq=1.
    # It might give strange errors and it also does not properly
    # support Keras data-generators for the validation-set.
    callback_log = TensorBoard(
        log_dir=log_dir,
        histogram_freq=0,
        write_graph=True,
        write_grads=False,
        write_images=False)

    # Use Keras to train the model.
    history = model.fit(x=data.x_train,
                        y=data.y_train,
                        epochs=3,
                        batch_size=128,
                        validation_data=validation_data,
                        callbacks=[callback_log])

    # Get the classification accuracy on the validation-set
    # after the last training-epoch.
    accuracy = history.history['val_accuracy'][-1]

   ...


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
This is the search-dimension for the number of dense layers in the neural network. This is an integer and we want at least 1 dense layer and at most 5 dense layers in the neural network.
Can you write Python code for it?

dim_num_dense_layers = Integer(low=1, high=5, name='num_dense_layers')


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
This is the search-dimension for the number of nodes for each dense layer. This is also an integer and we want at least 5 and at most 512 nodes in each layer of the neural network.
Can you write Python code for it?

dim_num_dense_nodes = Integer(low=5, high=512, name='num_dense_nodes')


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
This was developed using Python 3.6 (Anaconda) and package versions:
Can you write Python code for it?

tf.__version__

tf.keras.__version__

skopt.__version__


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
Train and Evaluate the Model
The neural network with the best hyper-parameters is saved to disk so it can be reloaded later. This is the filename for the model.
Can you write Python code for it?


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
We can also make a landscape-plot of the estimated fitness values for two dimensions of the search-space, here taken to be learning_rate and num_dense_layers.
The Bayesian optimizer works by building a surrogate model of the search-space and then searching this model instead of the real search-space, because it is much faster. The plot shows the last surrogate model built by the Bayesian optimizer where yellow regions are better and blue regions are worse. The black dots show where the optimizer has sampled the search-space and the red star shows the best parameters found.
Several things should be noted here. Firstly, this surrogate model of the search-space may not be accurate. It is built from only 40 samples of calls to the fitness() function for training a neural network with a given choice of hyper-parameters. The modelled fitness landscape may differ significantly from its true values especially in regions of the search-space with few samples. Secondly, the plot may change each time the hyper-parameter optimization is run because of random noise in the training process of the neural network. Thirdly, this plot shows the effect of changing these two parameters num_dense_layers and learning_rate when averaged over all other dimensions in the search-space, this is also called a Partial Dependence plot and is a way of visualizing high-dimensional spaces in only 2-dimensions.
Can you write Python code for it?

fig = plot_objective_2D(result=search_result,
                        dimension_name1='learning_rate',
                        dimension_name2='num_dense_layers',
                        levels=50)


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
We can also show another type of matrix-plot. Here the diagonal shows histograms of the sample distributions for each of the hyper-parameters during the Bayesian optimization. The plots below the diagonal show the location of samples in the search-space and the colour-coding shows the order in which the samples were taken. For larger numbers of samples you will likely see that the samples eventually become concentrated in a certain region of the search-space.
Can you write Python code for it?

fig, ax = plot_evaluations(result=search_result, dimension_names=dim_names)


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
We can convert these parameters to a dict with proper names for the search-space dimensions.
First we need a reference to the search-space object.
Can you write Python code for it?


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
We cannot make a landscape plot for the activation hyper-parameter because it is a categorical variable that can be one of two strings relu or sigmoid. How this is encoded depends on the Bayesian optimizer, for example, whether it is using Gaussian Processes or Random Forests. But it cannot currently be plotted using the built-in functions of skopt.
Instead we only want to use the real- and integer-valued dimensions of the search-space which we identify by their names.
Can you write Python code for it?

dim_names = ['learning_rate', 'num_dense_nodes', 'num_dense_layers']


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
We need to import several things from Keras.
Can you write Python code for it?

from tensorflow.keras import backend as K
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import InputLayer, Input
from tensorflow.keras.layers import Reshape, MaxPooling2D
from tensorflow.keras.layers import Conv2D, Dense, Flatten
from tensorflow.keras.callbacks import TensorBoard
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.models import load_model


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
We then combine all these search-dimensions into a list.
Can you write Python code for it?

dimensions = [dim_learning_rate,
              dim_num_dense_layers,
              dim_num_dense_nodes,
              dim_activation]


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
We then evaluate its performance on the test-set.
Can you write Python code for it?

result = model.evaluate(x=data.x_test,
                        y=data.y_test)


I am working on the file "19_Hyper-Parameters.ipynb".
The first task is:
We use the performance on the validation-set as an indication of which choice of hyper-parameters performs the best on previously unseen data. The Keras API needs the validation-set as a tuple.
Can you write Python code for it?

validation_data = (data.x_val, data.y_val)


I am working on the file "1D Numerov Schrodinger Solver.ipynb".
The first task is:
Bandgaps!
For Students: explore the symmetry of these states.
Q: Are there five degenerate states because each state has the particle in only one well?
Q: Why does each cluster of states start to have a slope in the E vs. # graph?
Can you write Python code for it?

for i,state in enumerate(states[0:5]):
    plt.subplot(5,1,i+1)
    plt.plot(x, state**2)

for i,state in enumerate(states[20:25]):
    plt.subplot(5,1,i+1)
    plt.plot(x, state**2)

plt.figure(figsize=(10,3))
plt.plot(x,states[24]**2)
plt.plot(x,states[20]**2)


I am working on the file "1D Numerov Schrodinger Solver.ipynb".
The first task is:
Periodic wells:
Can you write Python code for it?

N = 1000
x, dx = np.linspace(-1,1,N,retstep=True)
V_0 = np.zeros(N)
# periodic wells
V_0[:] = 1000
L = N/12  # width
S = N/10  # s
a = N/4   #
for i in range(5):
    V_0[int(i*S+a):int(i*S+a+L)] = 0

plt.plot(x,V_0)
plt.ylim(-50,3050)
plt.xlim(-1.1,1.1)

Alower = np.diag(np.ones(N)[:-1],k=-1)
Aupper = np.diag(np.ones(N)[:-1],k=1)
Amid = np.diag(-2*np.ones(N),k=0)

A = 1/dx**2 * (Alower + Amid + Aupper)

Blower = np.diag(np.ones(N)[:-1],k=-1)
Bupper = np.diag(np.ones(N)[:-1],k=1)
Bmid = np.diag(10*np.ones(N),k=0)

B = 1/12 * (Blower + Bmid + Bupper)

V = np.diag(V_0)
hbar=1
m=0.5

H = -(hbar**2)/(2*m)*inv(B)*A + V

energy, evecs = eigh(H,eigvals=(0,30))
E0 = energy[0]

states = [evecs[:,i] for i in range(30)]

plt.plot(energy,".")
plt.fill_between(range(31),E0,E0+V_0.max(), color='c', alpha=0.25)

plt.figure(figsize=(16,6))
for i,state in enumerate(states[0:15]):
    # Make these plot at the height for a cool figure!
    plt.plot(x,state**2*3000 + energy[i])

plt.fill_between(x,E0,E0+V_0,color='k',alpha=0.1)
#plt.plot(E0+V_0) TODO
plt.title("Bandgaps in periodic structure")


I am working on the file "1D Numerov Schrodinger Solver.ipynb".
The first task is:
SHO
Can you write Python code for it?

# Finite square well
V_0 = 250*x**2

plt.plot(x,V_0)
plt.ylim(-50,400)
plt.xlim(-1.1,1.1)

Alower = np.diag(np.ones(N)[:-1],k=-1)
Aupper = np.diag(np.ones(N)[:-1],k=1)
Amid = np.diag(-2*np.ones(N),k=0)

A = 1/dx**2 * (Alower + Amid + Aupper)

Blower = np.diag(np.ones(N)[:-1],k=-1)
Bupper = np.diag(np.ones(N)[:-1],k=1)
Bmid = np.diag(10*np.ones(N),k=0)

B = 1/12 * (Blower + Bmid + Bupper)

V = np.diag(V_0)
hbar=1
m=0.5

H = -(hbar**2)/(2*m)*inv(B)*A + V

energy, evecs = eigh(H,eigvals=(0,30))
E0 = energy[0]

states = [evecs[:,i] for i in range(30)]

plt.plot(energy,".")
plt.fill_between(range(31),E0,E0+V_0.max(), color='c', alpha=0.25)


I am working on the file "1D.ipynb".
The first task is:
?
Can you write Python code for it?

ts_reverse['offset'] = ts_reverse['signal'] + 10

ts_reverse[:100].plot(figsize=(4, 4))


I am working on the file "1D.ipynb".
The first task is:
Can you (comfortably) visualize both time scales on the same one plot?
Can you write Python code for it?


I am working on the file "1D.ipynb".
The first task is:
Exercises

Using the above two examples write a a code that explores Fano resonance in a resonant level coupled sideways to a lead!
Using the above examples write a code that explores the Aharonov-Bohm effect!

Some more examples
A simple potential step. The onsite potential in the right lead is shifted by $V_0$. Now in general the number of open chanels in the left and right lead are not the same. If the right channel is closed than we can only have reflection.
Can you write Python code for it?

def Smat_step(E,V0):

    #Definition of the leads
    L1=lead1D()
    L2=lead1D(eps0=V0)
    E=E+0.0001j # In order to make stuff meaningfull for
                # outside of the band we add a tiny
                # imaginary part to the energy

    #Green's function of decoupled leads
    g0= array([[L1.sgf(E=E),0       ],
               [0       ,L2.sgf(E=E)]])

    #Potential coupling the leads
    V= array([[0  ,-1],
              [-1,0]])

    #Dyson's equation
    G=inv(inv(g0)-V)

    #is the channel open?
    isopen=array([[float(imag(L1.kE(E))<0.001)],[float(imag(L2.kE(E))<0.001)]])

    #vector of the sqrt of the velocities
    vs=sqrt(array([[L1.vE(E=E)],[L2.vE(E=E)]]))

    #Scattering matrix from Fisher-Lee relations
    return matrix((1.0j*G*(vs*vs.T)-eye(2))*isopen*isopen.T)

energy_range=linspace(-4,4,1000)
figsize(8,6)
fts=20

@interact(V0=FloatSlider(min=-2,max=2,step=0.1,value=0,description=r'$V_0$'))
def step(V0):
    TR=[]
    REF=[]
    for ene in energy_range:
        SS=Smat_step(ene,V0)
        TR.append(abs(SS[0,1])**2)
        REF.append(abs(SS[0,0])**2)
    TR=array(TR)
    REF=array(REF)
    plot(energy_range,TR,label='T',linewidth=3)
    plot(energy_range,REF,label='R',linewidth=2)
    plot(energy_range,REF+TR,label='T+R',linewidth=1)
    plot(energy_range,zeros_like(energy_range),'k-')
    plot()
    ylim(-0.2,1.8);
    xticks(fontsize=fts)
    yticks(fontsize=fts)
    xlabel('Energy',fontsize=fts);
    legend(fontsize=fts);
    grid();
    title('Transmission for  a potential step',fontsize=fts)


I am working on the file "1D.ipynb".
The first task is:
Introduce data-ink. Touch on the principle of maximizing the data-ink ratio.
Challenges (High-Resolution Data Graphics)
How many time scales can you see (guess)?
What does ts.plot(xlim=[0, 20]) visualize?
ts is a time series, so it's indexed with... time.
Can you write Python code for it?


I am working on the file "1D.ipynb".
The first task is:
Let us investigate the following simple scattering setups:

For each system we shall write a small function that generates the scattering matrix of the problem as a function of the energy $E$ and other relevant parameters. We start by the tunnel junction where we only have $\alpha$ the hopping matrix element coupling the two leads as parameter
Can you write Python code for it?

def Smat_tunnel(E,alpha):

    #Definition of the leads
    L1=lead1D()
    L2=lead1D()
    E=E+0.0000001j # In order to make stuff meaningfull for
                # outside of the band we add a tiny
                # imaginary part to the energy

    #Green's function of decoupled leads
    g0= array([[L1.sgf(E=E),0       ],
               [0       ,L2.sgf(E=E)]])

    #Potential coupling the leads
    V= array([[0  ,alpha],
              [alpha,0]])

    #Dyson's equation
    G=inv(inv(g0)-V)

    #is the channel open?
    #since both sides have the same
    #structure they are open or closed
    #at the same time
    isopen=int(imag(L1.kE(E))<0.001)

    #vector of the sqrt of the velocities
    vs=sqrt(array([[L1.vE(E=E)],[L2.vE(E=E)]]))

    #Scattering matrix from Fisher-Lee relations
    return matrix(1.0j*G*(vs*vs.T)-eye(2))*isopen


I am working on the file "1D.ipynb".
The first task is:
Let's say we are done with exploration, we are now communicating results.
What does the above visualize poorly?
What does it visualize well?
How about the following:
Can you write Python code for it?


I am working on the file "1D.ipynb".
The first task is:
Let's start with 1D data, e.g., a time series. The time-series plot is the most common form of graph. pandas provides a great toolkit for working with time series data (finance, atmospheric and oceanic sciences...).
References:
- Tufte, Edward R. (2001), The Visual Display of Quantitative Information (2nd ed.), Cheshire, CT: Graphics Press
- Time series and date functionality in pandas
Can you write Python code for it?

ts = pd.Series.from_csv('data/coherence_timeseries.csv')

ts.plot()

matplotlib.style.use('ggplot')

ts.plot()


I am working on the file "1D.ipynb".
The first task is:
Let's take a closer look at cyclical data and strong variations.
Can you write Python code for it?

f, (ax1, ax2) = plt.subplots(1, 2, sharey=True)
ax1.plot(ts.loc[:18])
ax2.plot(ts.loc[:6])
ax2.plot(ts.loc[6:12])
ax2.plot(ts.loc[12:18])
ax2.legend(['first', 'second', 'third'], loc=4)
f.set_size_inches(10, 4)


I am working on the file "1D.ipynb".
The first task is:
Now let us write a small class implementing 1D semi-infinite leads. The class below is just a simple collection of the analytic formilae for spectrum, groupvelocity and the surface Green's function. Most functions have some meaningfull standard parameters (e.g.: hopping $\gamma=-1$, on-site potential $\varepsilon_0=0$). The functions are written keeping in mind that for a generic scattering problem energy is the important variable and not the wavenumber.
Can you write Python code for it?

class lead1D:
    'A class for simple 1D leads'
    def __init__(self,eps0=0,gamma=-1,**kwargs):
        'We assume real hopping \gamma and onsite \epsilon_0 parameters!'
        self.eps0=eps0
        self.gamma=gamma
        return

    def Ek(self,k):
        'Spectrum as a function of k'
        return self.eps0+2*self.gamma*cos(k)

    def kE(self,E,**kwargs):
        '''
        Spectrum as a function of E.
        If keyword a=True is given than
        it gives back two k values,
        one positive and one negative.
        '''
        a = kwargs.get('a',False)
        k=arccos((E-self.eps0)/(2*self.gamma))
        if a:
            return array([-k,k])
        else:
            return k

    def vE(self,E=0,**kwargs):
        '''
        Group velocity as a function of E.
        If keyword a=True is given than
        it gives back two v values,
        one positive and one negative.
        '''
        a = kwargs.get('a',False)
        k=self.kE(E)
        v= -2*self.gamma*sin(k)
        if a:
            return array([-v,v])
        else:
            return v

    def sgf(self,E=0):
        '''
        Surgace Green's function of a seminfinite 1D lead.
        '''
        return exp(1.0j *self.kE(E))/self.gamma

    def sgfk(self,k=pi/2):
        '''
        Surgace Green's function of a seminfinite 1D lead in terms of k.
        '''
        return exp(1.0j*k)/self.gamma

    def vk(self,k=pi/2):
        '''
        Group velocity in terms of k
        '''
        return -2*self.gamma*sin(k)


I am working on the file "1D.ipynb".
The first task is:
Now we can write again a script for a nice interactive plot
Can you write Python code for it?

energy_range=linspace(-4,4,1000)
figsize(8,6)
fts=20
@interact(t1=FloatSlider(min=-2,max=2,step=0.1,value=-1,description=r'$\gamma_L$'),
          t2=FloatSlider(min=-2,max=2,step=0.1,value=-1,description=r'$\gamma_R$'),
          eps1=FloatSlider(min=-2,max=2,step=0.1,value=0,description=r'$\varepsilon_1$'))
def BW(t1=-1,t2=-1,eps1=0):
    TR=[]
    REF=[]
    for ene in energy_range:
        SS=Smat_BW(ene,t1,t2,eps1)
        TR.append(abs(SS[0,1])**2)
        REF.append(abs(SS[0,0])**2)
    TR=array(TR)
    REF=array(REF)
    plot(energy_range,TR,label='T',linewidth=3)
    plot(energy_range,REF,label='R',linewidth=2)
    plot(energy_range,REF+TR,label='T+R',linewidth=1)
    plot(energy_range,zeros_like(energy_range),'k-')
    ylim(-0.2,1.8);
    xticks(fontsize=fts)
    yticks(fontsize=fts)
    xlabel('Energy',fontsize=fts);
    legend(fontsize=fts);
    grid();
    title('Transmission for  a resonant level',fontsize=fts)


I am working on the file "1D.ipynb".
The first task is:
Now we write a small script tog generate a figure interactively depending on $\alpha$ so we can explore the parameterspace. We have also included an extra factor on top of the Fisher-Lee relations taking in if a channel is open or not.
Can you write Python code for it?

energy_range=linspace(-4,4,1000) # this will be the plotted energy range
figsize(8,6) # setting the figure size
fts=20       # the default font is a bit too small so we make it larger

#using the interact decorator we can have an interactive plot
@interact(alpha=FloatSlider(min=-2,max=2,step=0.1,value=-1,description=r'$\alpha$'))
def tunnel(alpha=-1):
    '''
    This function draws a picture of the transmission and reflection coefficitents
    of a 1D tunneljunction
    '''
    TR=[] # we shall collect the values to be plotted in these variables
    REF=[]
    for ene in energy_range:                # energy scan
        SS=Smat_tunnel(ene,alpha)           # obtain S-matrix
        TR.append(abs(SS[0,1])**2)          # extract transmission coeff.
        REF.append(abs(SS[0,0])**2)         # extract reflection coeff.
    TR=array(TR)
    REF=array(REF)
    # make a pretty plot
    plot(energy_range,TR,label='T',linewidth=3)
    plot(energy_range,REF,label='R',linewidth=2)
    plot(energy_range,REF+TR,label='T+R',linewidth=2)
    plot(energy_range,zeros_like(energy_range),'k-')
    ylim(-0.2,1.8);
    xticks(fontsize=fts)
    yticks(fontsize=fts)
    xlabel('Energy',fontsize=fts);
    legend(fontsize=fts);
    grid();
    title('Transmission for  a tunnel junction',fontsize=fts)


I am working on the file "1D.ipynb".
The first task is:
Which of the above do you prefer?
Can you write Python code for it?

ts_reverse = ts.sort_index(ascending=False).reset_index()

ts_reverse.columns

del ts_reverse['index']

ts_reverse.columns = ['signal']

ts_reverse.head()

import numpy as np

ts_reverse['variations'] = 5 + ts_reverse['signal'] * 6 + 0.001 * np.arange(len(ts)) ** 2

ts_reverse['variations'].plot()


I am working on the file "1D/1D.ipynb".
The first task is:
As it follows, we are going to define a plot function in order to see how the decision boundary is for a linspace of samples in a defined range. It is easy to think that the results are going to be different in each execution because the learneable parameters are random at the beginning. This parameters are initialized when run the session variable D with the input_node x which is a node that contains the samples mentioned at the beginning of this paragraph.
Can you write Python code for it?

# plot decision surface
def plot_function(D,input_node):
    figure,axis=plots.subplots(1)
    # original distribution
    xaxis=np.linspace(-6,6,1000)
    axis.plot(xaxis, norm.pdf(xaxis,loc=mu,scale=sigma), label='p_distribution')
    # decision boundary
    r=1000 # r is the resolution. It's a good thing to use the same that was used at the beginning with the original distribution
    xaxis=np.linspace(-6,6,r)
    # decision space where the algorithm is going to save initial decisor values
    decisor_beginning=np.zeros((r,1))
    for i in range(int(r/M)):
        x=np.reshape(xaxis[M*i:M*(i+1)],(M,1))
        decisor_beginning[M*i:M*(i+1)]=sess.run(D,{input_node: x})

    axis.plot(xaxis, decisor_beginning, label='decision boundary', color ='black')
    axis.set_ylim(0,1.1)
    plots.legend()

plot_function(D,input_node)
plots.title('Initial Decision Boundary')
print(type(plots.show()))

loss_function_training=np.zeros(1000)
for i in range(1000):
    d=(np.random.random(M)-0.5) * 10.0
    # We do this in order to sample from all the domain and not only from the gaussian distribution
    labelsnew=norm.pdf(d,loc=mu,scale=sigma)
    loss_function_training[i],_=sess.run([lossfunction,optimizer_pre], {input_node: np.reshape(d,(M,1)), labels: np.reshape(labelsnew,(M,1))})

# Loss function with training
plots.plot(loss_function_training, color ='black')
plots.title('Loss Function')
plots.show()

plot_function(D,input_node)
plots.show()

learned_weights = sess.run(output)
# Here we are saving the weiights learned by the pretraining

sess.close()


I am working on the file "1D/1D.ipynb".
The first task is:
At this moment we have both the Multilayer Perceptron and the optimizer function. Now is a good moment to define a pre training process for the discriminator. This will help it to have a base in order to not start knowing nothing and this will reduce the time of convergence. From a math point of view, this process will help the discriminator to have a distribution function more similar to the original one that we will use to generate the real samples.
Can you write Python code for it?

# First of all we are going to define the size of the minibatch that will be used as input in each iteration
M=200

# Now we are going to define a special discriminator different from the original one
with tf.variable_scope("trainedD"):
    # This will take the input in a Mx1 dimension (200x1) which is the batch size
    input_node = tf.placeholder(tf.float32, shape=(M,1))
    # Same size than the batch. As many labels as possible values.
    labels = tf.placeholder(tf.float32, shape=(M,1))
    # The batch as input and only 1 output
    D,output = MultilayerPerceptron(input_node, 1)
    lossfunction = tf.reduce_mean(tf.square(D-labels))

optimizer_pre=optimizer(lossfunction,None)

sess=tf.InteractiveSession()
tf.global_variables_initializer().run()


I am working on the file "1D/1D.ipynb".
The first task is:
In this step we are going to define the function that will allow the learning process to search for better values of the learnable parameters
Can you write Python code for it?

train_iterations=20000
def optimizer(loss, varlist):
    batch = tf.Variable(0)
    rateoflearning = tf.train.exponential_decay(0.001,batch,train_iterations // 4, 0.95, staircase = True)
    optimizer = tf.train.MomentumOptimizer(rateoflearning,0.6).minimize(loss,global_step = batch, var_list = varlist)
    return optimizer


I am working on the file "1D/1D.ipynb".
The first task is:
Now, in this step we are going to define the Multilayer Perceptron. This neural network allow us to map a set of input data onto a set of appropiate outputs. It consist of multiple layers of nodes in a direct graph. Each layer is fully conected with the next one. In this particular case, this neural network is going to have different learnable parameters.
Can you write Python code for it?

# As have been said before, this function allow us to choose the correct number of ouputs we want.
def MultilayerPerceptron(input, output_dimension):
    w1 = tf.get_variable("w0", [input.get_shape()[1], 7], initializer= tf.random_normal_initializer())
    b1 = tf.get_variable("b0", [7], initializer = tf.constant_initializer(0.0))
    w2 = tf.get_variable("w1", [7, 5], initializer = tf.random_normal_initializer())
    b2 = tf.get_variable("b1", [5], initializer = tf.constant_initializer(0.0))
    w3 = tf.get_variable("w2", [5, output_dimension], initializer = tf.random_normal_initializer())
    b3 = tf.get_variable("b2", [output_dimension], initializer = tf.constant_initializer(0.0))
    # Now we will define the operators
    op1=tf.nn.tanh(tf.matmul(input,w1)+b1)
    op2=tf.nn.tanh(tf.matmul(op1,w2)+b2)
    op3=tf.nn.tanh(tf.matmul(op2,w3)+b3)
    return op3, [w1,b1,w2,b2,w3,b3]


I am working on the file "1D/1D.ipynb".
The first task is:
The samples will be token from a normal distribution and they will be used to train the neural network.
Can you write Python code for it?

mu,sigma = 0,1
linespace = np.linspace(-6,6,1000)
plots.plot(linespace, norm.pdf(linespace, loc=mu, scale=sigma))
plots.show()


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
Als nächstes definieren wir die Funktion, mit der die aufsummierten Kräfte und Potentiale für jedes Atom berechnet werden:
Can you write Python code for it?

# Berechnet die Kräfte und das Potential zwischen allen Atomen
@jit(float64(float64[:, :], float64[:,:], int64, float64[:,:],
          float64[:], float64[:]),
     nopython=True, cache=True, nogil=True)
def calc_forces_pot(xy, v, N, f, pot, virial):
    E = 0.0
    f[:] = 0.0
    pot[:] = 0.0
    fij = np.zeros(2)
    potential = 0.0
    virial[:] = 0.0

    for i in range(1,N):
        for j in range(i,N):
            # Distanzvektor:
            rij = xy[:,i] - xy[:,j]
            potential = calc_force_pot_onibyj(rij, fij)
            pot[i] += potential
            f[:,i] += fij
            f[:,j] -= fij
            virial[i] += np.dot(rij, fij)    # Virial-Teil für die Druckberechnung
    E += np.sum(pot)
    return E


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
Da die Temperatur als Mittelwert der kinetischen Teilchenenergie definiert ist und wir verhältnismäßig wenige Atome simuliert haben, schwankt die Temperatur, auch nachdem das thermische Gleichgewicht eingetreten ist, noch deutlich. Wir mitteln deshalb über die zweite Simulation und kommen damit auf einen Wert von 13 K.
Druck
Als nächstes soll der Druck im System berechnet werden. Dieser ist definiert als Kraft pro Fläche und Kraft ist Impulsänderung pro Zeit. Für unsere Box mit festen Wänden müssen wir also nur die beim Stoß mit der Wand hervorgerufene Impulsänderung der Teilchen über eine Zeitspanne aufsummieren und die Summe durch die Zeitspanne teilen:
$$p = \frac{\sum\limits_{i=1}^N 2\cdot m v_{n,i}}{\Delta t \cdot A}$$
Die oben definierte Funktion do_calculation liefert bereits den Druck mal eine  Tiefeneinheit (weil 2D) in natürlichen Einheiten zurück (p_star). Um zum Druck in Pa zu kommen, müssen wir diesen im Fall von Helium mit 8122319 und mit einer Längeneinheit multiplizieren.
Der Druck lässt sich aber auch direkt aus der Molekülbewegung und deren Interaktion aus der Virialgleichung berechnen (vgl. Kapitel reale Arbeitsstoffe in der Thermodynamikvorlesung):
$$p = \underbrace{\frac{N k_B T}{V}}{\text{wie ideales Gas}} + \underbrace{\frac{1}{3}\left<\sum\limits{i<j}^N \overrightarrow{r}_{ij}\cdot \overrightarrow{F}_{ij} \right>}_{\text{Kräfte zw. den Molekülen}}$$
bzw. mit der Definition der Temperatur von oben:
$$p = \frac{N m_\text{Atom}}{3 N V} \left< 2E_{\text{kin}} + \sum\limits_{i<j}^N \overrightarrow{r}_{ij}\cdot \overrightarrow{F}_{ij} \right>$$
Aus der Gleichung wird deutlich, dass sich der Druck im Fluid aus der bereits bekannten Beziehung für ideale Gase $pV=m R_S T=n R_m T = N k_B T$ und einem zusätzlichen Term, der die zwischenmolekularen Kräfte berücksichtigt, zusammensetzt.
Die oben definierte Funktion do_calculation liefert den so berechneten Druck mal eine Tiefeneinheit (weil 2D) in natürlichen Einheiten zurück (p), allerdings noch ohne zeitliche Mittelung.
Can you write Python code for it?

# Neuen Plot einrichten
plt.figure(figsize=(4, 3))
plt.ylabel('$p$ in Pa m')
plt.xlabel('$t$ in ps')
plt.plot(np.array(times2)*t_unit,
         np.array(p2)*p_unit*l_unit*1e-10);

print ("mittlerer Druck berechnet über Wandinteraktion: ",
       p_star2*p_unit*l_unit*1e-10)
print ("mittlerer Druck berechnet über Virialgleichung: ",
       np.mean(p2)*p_unit*l_unit*1e-10)


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
Damit haben wir bereits die wesentliche Physik in Programmcode umgesetzt. Was noch fehlt ist eine Initialisierung der Atompositionen und -geschwindigkeiten. Wir implementieren hierzu eine Funktion, die eine $n \times n$-Anordnung um ein gegebenes Zentrum erzeugt.
Die Anordnung soll so erfolgen, dass alle benachbarten Atome den gleichen Abstand zueinander haben. Der Abstand wird dazu so gewählt, dass sich das Potential im Minimum befindet, also $r=r^*$. Eine mögliche Anordnung, die diese Kriterien erfüllt zeigt die folgende Abbildung:

Die Geschwindigkeit der einzelnen Atome wird zufällig in einem Intervall von 0 bis vmax gewählt. Zusätzlich kann eine Drift-Geschwindigkeit vbias überlagert werden.
Can you write Python code for it?

def init_block(n, x_c, y_c, vmax, vbias):
    delta = 2**(1/6)
    h = math.sqrt(3)*0.5*delta
    xy = np.zeros((2,n*n))
    v = np.zeros((2,n*n))
    for i in range(n):
        for j in range(n):
            if j % 2 == 0:  # gerade Reihe
                x = x_c-0.5*n*delta+i*delta
            else:
                x = x_c-0.5*n*delta+i*delta + delta*0.5
            y = y_c-0.5*n*h+j*h
            xy[0,i*n+j] = x
            xy[1,i*n+j] = y
    v = (np.random.random((2, n*n))-0.5)*vmax
    v[0] += vbias[0]
    v[1] += vbias[1]
    return xy, v, n*n


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
Dichte
Die Dichte $\rho$ kann aus der Atomanzahl, der Atommasse und der Größe des Simulationsgebiets berechnet werden:
$$\rho = \frac{m}{V} = \frac{N\cdot m_{\text{Atom}}}{xydim^3}$$
bzw. in 2D:
$$\rho_{\text{2D}} = \frac{N\cdot m_{\text{Atom}}}{xydim^2} \qquad \text{in }\frac{\text{kg}}{\text{m}^2}$$
Innere Energie, spezifische Wärmekapazität und Isentropenexponent
Die innere Energie des Systems haben wir oben bereits berechnet. Sie setzt sich aus der kinetischen und der potentiellen Energie im Lennard-Jones-Kraftfeld zusammen. Bei einem 1-atomigen Gas wird die innere Energie nur durch die translatorische kinetische Energie der Moleküle bestimmt. D.h. wenn wir einem Behälter mit einem 1-atomigen Gas Energie zuführen, wird diese ausschließlich in die Erhöhung der kinetischen Teilchenenergie gehen.
Bei mehratomigen Gasen kann Energie auch in Form von Rotations- und Schwingungsbewegungen zwischen den Molekülen gespeichert werden. Jeder Freiheitsgrad (FG) eines Moleküls kann genau $\frac{1}{2}k_B T$ Energie speichern. D.h. die innere Energie eines $n$-atomigen idealen Gases lässt sich dann wie folgt berechnen:
$$U = FG \cdot \frac{1}{2} N k_B T$$ bzw. $$u = FG \cdot \frac{1}{2} \frac{k_B T}{m_\text{Atom}} = FG \cdot \frac{1}{2} R T$$
Die spezifische Enthalpie ergibt sich definitionsgemäß für ein ideales Gas ($h=u+pv=u+RT$) zu
$$h = FG \cdot \frac{1}{2} R T + R T = \left(\frac{FG}{2}+1\right) R T$$
Das folgende Bild zeigt einige Beispiele für die Anregung von Freiheitsgraden je nach Form des Moleküls. Bei 2-atomigen Gasen kann auf der Achse zwischen den beiden Molekülen keine Rotationsenergie gespeichert werden, da das Trägheitsmoment um diese Achse vernachlässigbar klein ist. Schwingungsfreiheitsgrade werden bei Gasen erst bei sehr großer Temperatur (> 2000 K) voll angeregt und nehmen bei kleineren Temperaturen keine Energie auf.

Der Isentropenexponent ergibt sich so direkt aus den Freiheitsgraden zu:
$$\kappa = \frac{h}{u} = \frac{FG+2}{FG}$$
|         | FG   | $\kappa$ |
| ------- |:----:| :-------:|
|...
Can you write Python code for it?

# Neuen Plot einrichten
plt.figure(figsize=(4, 3))
plt.ylabel('$p$ in Pa m')
plt.xlabel('$t$ in ps')

traj_np = np.array(traj2[:]).T
traj_x = traj_np[:][50][0]
traj_y = traj_np[:][50][1]

plt.xlim(0,xydim)
plt.ylim(0,xydim)
plt.plot(np.array(traj_x),
         np.array(traj_y), lw=0.5);


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
Die Gesamtenergie im Simulationsgebiet sollte sich während der Simulation nicht ändern. Sie ist ein gutes Maß zur Überprüfung der Genauigkeit einer Simulation. Wir überprüfen also zunächst, ob die Energie im System konstant geblieben ist:
Can you write Python code for it?

# Neuen Plot einrichten
plt.figure(figsize=(4, 3))
plt.ylabel('$E$ in eV')
plt.xlabel('$t$ in ps')
plt.plot(np.array(times+times2)*t_unit,
         (np.array(E_kin+E_kin2)+np.array(E_pot+E_pot2))*E_unit,
         label='$E_{\mathrm{gesamt}}$')
plt.plot(np.array(times+times2)*t_unit,
         np.array(E_kin+E_kin2)*E_unit,
         label = "$E_{\mathrm{kin}}$")
plt.plot(np.array(times+times2)*t_unit,
         np.array(E_pot+E_pot2)*E_unit,
         label = "$E_{\mathrm{pot}}$")
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.);


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
Die mittlere Teilchengeschwindigkeit hat in idealen Gasen eine ähnliche Größenordnung wie die Schallgeschwindigkeit $a=\sqrt{\kappa R_s T}$. In unserem Beispiel mit Helium kommen wir auf eine Geschwindigkeit von:
Can you write Python code for it?

v_mean = np.mean(v2_abs)*v_unit
print ('mittlere Teilchengeschwindigkeit in m/s:', v_mean)


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
Für die Berechnung müssen wir nun noch die Atome im Simulationsgebiet anordnen sowie dessen Größe festlegen und ein paar Variablen initialisieren.
In einem ersten Beispiel definieren wir einen Block aus 400 Atomen, die mit einer zufälligen Geschwindigkeit initialisiert werden:
Can you write Python code for it?

dt = 0.01                     # Zeitschrittweite in natürl. Einh.
dt_store = 0.05               # Zeitschrittweite für's Abspeichern
t_max = 125                   # Simulationsende in natürl. Einh.
xydim = 70                    # Abm. Sim.-gebiet in natürl. Einh.
vmax = 7

# Daten für Helium:
E_unit = 0.00094              # eV
t_unit = 1.76                 # ps
T_unit = 10.9                 # K
m_unit = 4.0                  # u
v_unit = 150.0                # m/s
l_unit = 2.64                 # Angstrom
p_unit = 8122319              # Pa

xy, v, N = init_block(20, xydim/2, xydim/2, vmax, (0.0,0.0))


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
Temperatur
Interessant ist es auch, die Temperatur im Simulationsgebiet über der Zeit darzustellen. Bei einem 1-atomigen Gas ist die kinetische Energie pro Freiheitsgrad des Atoms gerade $\frac{1}{2}k_\text{B} T$.
Die drei translatorischen Freiheitsgrade des 1-atomigen Gases - für jede Raumrichtung einer - führt also zu folgendem Zusammenhang zwischen der mittleren kinetischen Energie aller $N$ Atome und der Temperatur:
$$E_{\text{kin}} = \frac{3}{2}N k_\text{B} T$$
Im Zweidimensionalen ergibt sich dann:
$$E_{\text{kin,2D}} = 2\cdot\frac{1}{2}N k_\text{B} T$$
oder nach der Temperatur aufgelöst:
$$T = \frac{E_{\text{kin,2D}}}{N k_\text{B}}$$
Wenn wir damit die Temperatur über der Zeit auftragen bekommen wir folgendes Bild:
Can you write Python code for it?

# Neuen Plot einrichten
plt.figure(figsize=(4, 3))
plt.ylabel('$T$ in K')
plt.xlabel('$t$ in ps')
plt.plot(np.array(times+times2)*t_unit,
         np.array(E_kin+E_kin2)/N*T_unit);

print ("mittlere Temperatur: ", np.mean(E_kin2)/N*T_unit)


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
Und eine Funktion, welche die gesamte kinetische Energie im System berechnet:
Can you write Python code for it?

# Berechnet die kinetische Energie im System
@jit(float64(float64[:,:]), nopython=True, cache=True, nogil=True)
def calc_kin_energy(v):
    E_kin = 0.0
    for i in range(v[0,:].size):
        E_kin += 0.5 * np.dot(v[:,i],v[:,i])
    return E_kin


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
Verteilung der Teilchengeschwindigkeiten (Maxwell-Boltzmann-Verteilung)
Die Geschwindigkeiten der Atome in einem Gas, einer Flüssigkeit oder einem Festkörper sind nicht alle gleich, sondern über einen größeren Bereich verteilt. Für ein ideales Gas entspricht die Wahrscheinlichkeitsverteilung der Geschwindigkeitsbeträge der sog. Maxwell-Boltzmann-Verteilung. Diese lässt sich herleiten, wenn man davon ausgeht, dass die Teilchen ohne Vorzugsrichtung durch den Raum fliegen und die Geschwindigkeitskomponenten jeweils einer Normalverteilung unterliegen. Im Dreidimensionalen ergibt sich die Wahrscheinlichkeitsdichte-Verteilung (PDF) durch Multiplikation der drei Normalverteilungen für die drei Raumrichtungen zu:
$$p(v) = 4\pi\left(\frac{m_\text{Atom}}{2\pi k_\text{B} T}\right)^\frac{3}{2}v^2\exp\left(-\frac{m_\text{Atom}v^2}{2 k_\text{B} T} \right)$$
Im Zweidimensionalen ergibt sich:
$$p(v) = \frac{m_\text{Atom}}{k_\text{B} T}v \exp\left(-\frac{m_\text{Atom} v^2}{2 k_\text{B} T} \right)$$
Darin ist $k_\text{B} = 1.38064852\cdot 10^{-23}~\text{J/K}~$ die Boltzmann-Konstante.
Es ist erstaunlich, dass die Teilchen unter sich völlig verlustfrei (reibungsfrei) interagieren. Wäre dies nicht so, würde sich das Gas in einem perfekt isolierten Behälter von allein abkühlen.
Im SciPy-Paket ist die Maxwell-Boltzmann-Verteilung bereits implementiert. Mit dem fit-Befehl lassen sich die Parameter der Verteilung aus einer gegebeben Datenreihe abschätzen.
Can you write Python code for it?

import scipy.stats as stats

plt.figure(figsize=(4, 3))

v2_abs = np.sqrt(vel2[-1][0]**2+vel2[-1][1]**2)
for i in range(2,1000):
    v2_abs = np.concatenate((v2_abs,
                             np.sqrt(vel2[-i][0]**2+vel2[-i][1]**2)),
                           axis=0)

n, bins, patches = plt.hist(v2_abs, 50,
                            normed=1, facecolor='blue',
                            alpha=0.75);

maxwell = stats.maxwell
params = maxwell.fit(v2_abs, floc=0)

plt.plot(bins, maxwell.pdf(bins, *params), lw=3, color='red')
plt.xlabel('$v$')
plt.ylabel('$p(v)$');


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
Wir definieren noch eine letzte Funktion, welche die Berechnung über eine vorgegebene Anzahl Zeitschritte durchführt und die berechneten Flugbahnen der Partikel zusammen mit deren Geschwindigkeiten zurückgibt. Die Integration über die Zeit erfolgt mit dem sog. Verlet-Algorithmus, der häufig zur Lösung der Newtonschen Bewegungsgleichung verwendet wird und eine Genauigkeit 2. Ordnung in der Zeit aufweist.
Can you write Python code for it?

def do_calculation(xy, v, xydim, dt=0.02, t_max=1, dt_store=0.2, t0=0):
    traj  = []    # Flugbahnen (Partikelpositionen)
    E_kin = []    # kinetische Energie(t)
    E_pot = []    # potentielle Energie(t)
    vel   = []    # v(Partikel, t)
    times = []    # Zeitpunkte
    p = []        # Druck

    t_store = 0.0
    t = t0
    wall_di = 0.0 # Impulsaenderung an der Wand

    f = np.zeros_like(xy)
    pot = np.zeros_like(v[0,:])
    virial = np.zeros_like(pot)
    N = pot.size

    while t < t_max:
        # Neue Positionen der Atome:
        xy += v*dt + 0.5 * f * dt*dt

        # Änderung der Geschwindigkeit aufgrund der Kräfte
        # (halber Anteil der Änderung)
        v += 0.5 * f * dt

        # Neue Kräfte, Potentiale und Energien berechnen
        pot_e = calc_forces_pot(xy, v, N, f, pot, virial)

        # Zweite Hälfte der Geschwindigkeitsänderung
        v += 0.5 * f * dt

        # Atome markieren, die gerade außerhalb des
        # Simulationsgebiets geflogen sind (=Stoß mit den Wänden)
        mask_l = xy[0,:] <= 0
        mask_r = xy[0,:] >= xydim
        mask_b = xy[1,:] <= 0
        mask_t = xy[1,:] >= xydim

        # Impulsuebertragung an die 4 Wände:
        wall_di += 2*np.sum(np.abs(v[0,mask_l]))
        wall_di += 2*np.sum(np.abs(v[0,mask_r]))
        wall_di += 2*np.sum(np.abs(v[1,mask_b]))
        wall_di += 2*np.sum(np.abs(v[1,mask_t]))

        # vollkommen elasische Stöße an der Wand:
        v[0,mask_l] *= -1
        v[0,mask_r] *= -1
        v[1,mask_b] *= -1
        v[1,mask_t] *= -1

        # Werte speichern
        if (t_store > dt_store):
            kinE = calc_kin_energy(v)
            E_kin.append(kinE)

            p_virial = np.sum(virial)  # Druckanteil durch zwischenmolekulare Kräfte
            p_ideal = 2*kinE           # Druckanteil analog idealem Gas
            p.append((p_virial+p_ideal) / (2*xydim**2))
            traj.append(xy.copy())

...


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
Wir sind jetzt bereit, eine Simulation der Flugbahnen durchzuführen:
Can you write Python code for it?

times, traj, vel, E_pot, E_kin, p_star, p = do_calculation(
                                             xy, v, xydim, dt=dt,
                                             t_max=t_max,
                                             dt_store=dt_store)

times2, traj2, vel2, E_pot2, E_kin2, p_star2, p2 = do_calculation(
                                             np.array(traj[-1]), vel[-1], xydim, dt=dt,
                                             t_max=t_max*2,
                                             dt_store=dt_store, t0=times[-1])


I am working on the file "1_3-Molekulardynamik_Simulation.ipynb".
The first task is:
und dann eine Funktion, mit der wir die Kraft und das Lennard-Jones-Potential zwischen zwei Atomen berechnen können:
Can you write Python code for it?

# Berechnet die Kraft und das Potential zwischen zwei Atomen
# Es wird dabei die Python-Erweiterung Numba verwendet, die
# eine "just-in-time" Kompilierung durchführt, um die Ausführ-
# geschwindigkeit zu erhöhen.
@jit(float64(float64[:], float64[:]),
     nopython=True, cache=True, nogil=True)
def calc_force_pot_onibyj(rij, fij):
    r = math.sqrt(rij[0]*rij[0]+rij[1]*rij[1])
    if r > 0 and r < 3:
        invrsquare = 1/(r*r)
        a = (24 * (2*invrsquare**7 - invrsquare**4))
        #a = (24 * (2*(1/r)**14 - (1/r)**8))
        fij[0] = a * rij[0]
        fij[1] = a * rij[1]
        pot = 4 * (invrsquare**6 - invrsquare**3)
    else:
        fij[0] = 0.0
        fij[1] = 0.0
        pot = 0.0
    return pot


I am working on the file "1_APIs/3_api_workbook.ipynb".
The first task is:
1. Constructing API GET Request

In the first place, we know that every call will require us to provide:

a base URL for the API,
some authorization code or key, and
a format for the response.

So let's put store those in some variables.
Use the following demonstration keys for now, but in the future, get your own!

ef9055ba947dd842effe0ecf5e338af9:15:72340235
25e91a4f7ee4a54813dca78f474e45a0:15:73273810
e15cea455f73cc47d6d971667e09c31c:19:44644296
b931c838cdb745bbab0f213cfc16b7a5:12:44644296
1dc1475b6e7d5ff5a982804cc565cd0b:6:44644296
18046cd15e21e1b9996ddfb6dafbb578:4:44644296
be8992a420bfd16cf65e8757f77a5403:8:44644296
Can you write Python code for it?

# set key
key="be8992a420bfd16cf65e8757f77a5403:8:44644296"

# set base url
base_url="http://api.nytimes.com/svc/search/v2/articlesearch"

# set response format
response_format=".json"


I am working on the file "1_APIs/3_api_workbook.ipynb".
The first task is:
2. Parsing the response text

We can read the content of the server’s response using .text
Can you write Python code for it?

# Inspect the content of the response, parsing the result as text
response_text= r.text
print(response_text[:1000])


I am working on the file "1_APIs/3_api_workbook.ipynb".
The first task is:
4. Formatting

Let's take another look at one of these documents.
Can you write Python code for it?


I am working on the file "1_APIs/3_api_workbook.ipynb".
The first task is:
5. Exporting

We can now export the data to a CSV.
Can you write Python code for it?

keys = all_formatted[1]
# writing the rest
with open('all-formated.csv', 'w') as output_file:
    dict_writer = csv.DictWriter(output_file, keys)
    dict_writer.writeheader()
    dict_writer.writerows(all_formatted)


I am working on the file "1_APIs/3_api_workbook.ipynb".
The first task is:
Capstone Challenge
Using what you learned, tell me if Chris' claim (i.e. that Duke Ellington has gotten more popular lately) holds water.
Can you write Python code for it?


I am working on the file "1_APIs/3_api_workbook.ipynb".
The first task is:
Challenge 3: Make a function
Turn the code above into a function that inputs a search term and a year, and returns all the documents containing that search term in that year.
Can you write Python code for it?

# DEFINE YOUR FUNCTION HERE

# uncomment to test
# get_api_data("Duke Ellington", 2014)


I am working on the file "1_APIs/3_api_workbook.ipynb".
The first task is:
Click on that link to see it returns!
Challenge 1:  Adding a date range
What if we only want to search within a particular date range? The NYT Article Api allows us to specify start and end dates.
Alter the search_params code above so that the request only searches for articles in the year 2015.
You're gonna need to look at the documentation here to see how to do this.
Can you write Python code for it?

# set date parameters here

# Uncomment to test
# r = requests.get(base_url+response_format, params=search_params)
# print(r.url)


I am working on the file "1_APIs/3_api_workbook.ipynb".
The first task is:
Now we're ready to make the request. We use the .get method from the requests library to make an HTTP GET Request.
Can you write Python code for it?

# make request
r = requests.get(base_url+response_format, params=search_params)


I am working on the file "1_APIs/3_api_workbook.ipynb".
The first task is:
That looks intimidating! But it's really just a big dictionary. Let's see what keys we got in there.
Can you write Python code for it?

print(data.keys())

# this is boring
data['status']

# so is this
data['copyright']

# this is what we want!
# data['response']

data['response'].keys()

data['response']['meta']['hits']

# data['response']['docs']
type(data['response']['docs'])


I am working on the file "1_APIs/3_api_workbook.ipynb".
The first task is:
What you see here is JSON text, encoded as unicode text. JSON stands for "Javascript object notation." It has a very similar structure to a python dictionary -- both are built on key/value pairs. This makes it easy to convert JSON response to a python dictionary.
Can you write Python code for it?

# Convert JSON response to a dictionary
data = json.loads(response_text)
# data


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
1. Constructing API GET Request

In the first place, we know that every call will require us to provide:

a base URL for the API,
some authorization code or key, and
a format for the response.

So let's put store those in some variables.
Use the following demonstration keys for now, but in the future, get your own!

ef9055ba947dd842effe0ecf5e338af9:15:72340235
25e91a4f7ee4a54813dca78f474e45a0:15:73273810
e15cea455f73cc47d6d971667e09c31c:19:44644296
b931c838cdb745bbab0f213cfc16b7a5:12:44644296
1dc1475b6e7d5ff5a982804cc565cd0b:6:44644296
18046cd15e21e1b9996ddfb6dafbb578:4:44644296
be8992a420bfd16cf65e8757f77a5403:8:44644296
Can you write Python code for it?


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
4. Formatting

Let's take another look at one of these documents.
Can you write Python code for it?


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
5. Exporting

We can now export the data to a CSV.
Can you write Python code for it?


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
Capstone Challenge
Using what you learned, tell me if Chris' claim (i.e. that Duke Ellington has gotten more popular lately) holds water.
Can you write Python code for it?

# for this challenge, we just need the number of hits.
def get_api_hits(term, year):
    '''
    returns an integer, the number of hits (or articles) mentioning the given term
    in the given year
    '''
    # set base url
    base_url="http://api.nytimes.com/svc/search/v2/articlesearch"

    # set response format
    response_format=".json"

    # set search parameters
    search_params = {"q":term,
                 "api-key":key,
                 "begin_date": str(year) + "0101", # date must be in YYYYMMDD format
                 "end_date":str(year) + "1231"}

    # make request
    r = requests.get(base_url+response_format, params=search_params)

    # convert to a dictionary
    data=json.loads(r.text)

    # get number of hits
    hits = data['response']['meta']['hits']
    return(hits)

get_api_hits("Duke Ellington", 2014)

# collect data
years = range(2005, 2016)
years

all_duke = []
for i in years:
    all_duke.append(get_api_hits("Duke Ellington", i))

all_duke

%matplotlib inline
plt.plot(years, all_duke)
plt.axis([2005, 2015, 0, 200])


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
Challenge 3: Make a function
Turn the code above into a function that inputs a search term and a year, and returns all the documents containing that search term in that year.
Can you write Python code for it?

# DEFINE YOUR FUNCTION HERE

def get_api_data(term, year):
    # set base url
    base_url="http://api.nytimes.com/svc/search/v2/articlesearch"

    # set response format
    response_format=".json"

    # set search parameters
    search_params = {"q":term,
                 "api-key":key,
                 "begin_date": str(year) + "0101", # date must be in YYYYMMDD format
                 "end_date":str(year) + "1231"}

    # make request
    r = requests.get(base_url+response_format, params=search_params)

    # convert to a dictionary
    data=json.loads(r.text)

    # get number of hits
    hits = data['response']['meta']['hits']
    print("number of hits:", str(hits))

    # get number of pages
    pages = int(math.ceil(hits/10))

    # make an empty list where we'll hold all of our docs for every page
    all_docs = []

    # now we're ready to loop through the pages
    for i in range(pages):
        print("collecting page", str(i))

        # set the page parameter
        search_params['page'] = i

        # make request
        r = requests.get(base_url+response_format, params=search_params)

        # get text and convert to a dictionary
        data=json.loads(r.text)

        # get just the docs
        docs = data['response']['docs']

        # add those docs to the big list
        all_docs = all_docs + docs

    return(all_docs)

# uncomment to test
# get_api_data("Duke Ellington", 2014)


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
Challenge 4 Collect more fields
Edit the function above so that we include the lead_paragraph and word_count fields.
HINT: Some articles may not contain a lead_paragraph, in which case, it'll throw an error if you try to address this value (which doesn't exist.) You need to add a conditional statement that takes this into consideraiton. If
Advanced: Add another key that returns a list of keywords associated with the article.
Can you write Python code for it?

def format_articles(unformatted_docs):
    '''
    This function takes in a list of documents returned by the NYT api
    and parses the documents into a list of formated dictionaries,
    with 'id', 'header', and 'date' keys
    '''
    formatted = []
    for i in unformatted_docs:
        dic = {}
        dic['id'] = i['_id']
        dic['headline'] = i['headline']['main']
        dic['date'] = i['pub_date'][0:10] # cutting time of day.
        if i['lead_paragraph']:
            dic['lead_paragraph'] = i['lead_paragraph']
        dic['word_count'] = i['word_count']
        dic['keywords'] = [keyword['value'] for keyword in i['keywords']]
        formatted.append(dic)
    return(formatted)

# uncomment to test
all_formatted = format_articles(all_docs)
# all_formatted[:5]


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
Now we're ready to make the request. We use the .get method from the requests library to make an HTTP GET Request.
Can you write Python code for it?


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
Now, we have a response object called r. We can get all the information we need from this object. For instance, we can see that the URL has been correctly encoded by printing the URL. Click on the link to see what happens.
Can you write Python code for it?


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
That looks intimidating! But it's really just a big dictionary. Let's see what keys we got in there.
Can you write Python code for it?


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
That looks what we want! Let's put that in it's own variable.
Can you write Python code for it?

docs = data['response']['docs']

docs[0]


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
This is all great, but it's pretty messy. What we’d really like to to have, eventually, is a CSV, with each row representing an article, and each column representing something about that article (header, date, etc). As we saw before, the best way to do this is to make a lsit of dictionaries, with each dictionary representing an article and each dictionary representing a field of metadata from that article (e.g. headline, date, etc.) We can do this with a custom function:
Can you write Python code for it?

def format_articles(unformatted_docs):
    '''
    This function takes in a list of documents returned by the NYT api
    and parses the documents into a list of dictionaries,
    with 'id', 'header', and 'date' keys
    '''
    formatted = []
    for i in unformatted_docs:
        dic = {}
        dic['id'] = i['_id']
        dic['headline'] = i['headline']['main']
        dic['date'] = i['pub_date'][0:10] # cutting time of day.
        formatted.append(dic)
    return(formatted)

all_formatted = format_articles(all_docs)

all_formatted[:5]


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
What you see here is JSON text, encoded as unicode text. JSON stands for "Javascript object notation." It has a very similar structure to a python dictionary -- both are built on key/value pairs. This makes it easy to convert JSON response to a python dictionary.
Can you write Python code for it?


I am working on the file "1_APIs/4_api_solutions.ipynb".
The first task is:
You often want to send some sort of data in the URL’s query string. This data tells the API what information you want. In our case, we want articles about Duke Ellington. Requests allows you to provide these arguments as a dictionary, using the params keyword argument. In addition to the search term q, we have to put in the api-key term.
Can you write Python code for it?

# set search parameters
search_params = {"q":"Duke Ellington",
                 "api-key":key}


I am working on the file "1_Basics.ipynb".
The first task is:
2- Branching and Loops
If, Elif, Else
In Python, the formulation for branching is the  if: condition (mind the :) followed by an indentation of one tab that represents what is executed if the condition is true. The indentation is primordial and at the core of Python.
Can you write Python code for it?

statement1 = False
statement2 = False

if statement1:
    print("statement1 is True")
elif statement2:
    print("statement2 is True")
else:
    print("statement1 and statement2 are False")

statement1 = statement2 = True

if statement1:
    if statement2:
        print("both statement1 and statement2 are True")

if statement1:
    if statement2: # Bad indentation!
    #print("both statement1 and statement2 are True") # Uncommenting Would cause an error
        print("here it is ok")
    print("after the previous line, here also")

statement1 = True

if statement1:
    print("printed if statement1 is True")

    print("still inside the if block")

statement1 = False

if statement1:
    print("printed if statement1 is True")

print("outside the if block")


I am working on the file "1_Basics.ipynb".
The first task is:
3- Functions
In Python, a function is defined as  def function_name(function_arguments): followed by an indentation representing what is inside the function. (No return arguments are provided a priori)
Can you write Python code for it?

def fun0():
    print("\"fun0\" just prints")

fun0()


I am working on the file "1_Basics.ipynb".
The first task is:
4- Classes [*]
Classes are at the core of object-oriented programming, they are used to represent an object with related attribues (variables) and methods (functions).
They are defined as functions but with the keyword class class my_class(object): followed by an indentation. The definition of a class usually contains some methods:
* The first argument of a method must be self in auto-reference.
* Some method names have a specific meaning:
   * __init__: method executed at the creation of the object
   * __str__ : method executed to represent the object as a string for instance when the object is passed ot the function print
Can you write Python code for it?

class Point(object):
    """
    Class of a point in the 2D plane.
    """
    def __init__(self, x=0.0, y=0.0):
        """
        Creation of a new point at position (x, y).
        """
        self.x = x
        self.y = y

    def translate(self, dx, dy):
        """
        Translate the point by (dx , dy).
        """
        self.x += dx
        self.y += dy

    def __str__(self):
        return("Point: ({:.2f}, {:.2f})".format(self.x, self.y))

p1 = Point()
print(p1)

p1.translate(3,2)
print(p1)

p2 = Point(1.2,3)
print(p2)


I am working on the file "1_Basics.ipynb".
The first task is:
5- Reading and writing files
open returns a file object, and is most commonly used with two arguments: open(filename, mode).
The first argument is a string containing the filename. The second argument is another string containing a few characters describing the way in which the file will be used (optional, 'r' will be assumed if it’s omitted.):
* 'r' when the file will only be read
* 'w' for only writing (an existing file with the same name will be erased)
* 'a' opens the file for appending; any data written to the file is automatically added to the end
Can you write Python code for it?

f = open('./data/test.txt', 'w')
print(f)


I am working on the file "1_Basics.ipynb".
The first task is:
A useful function is <tt>range</tt> which generated sequences of numbers that can be used in loops.
Can you write Python code for it?

print("Range (from 0) to 4 (excluded) ")
for x in range(4):
    print(x)

print("Range from 2 (included) to 6 (excluded) ")
for x in range(2,6):
    print(x)

print("Range from 1 (included) to 12 (excluded) by steps of 3 ")
for x in range(1,12,3):
    print(x)


I am working on the file "1_Basics.ipynb".
The first task is:
Arguments
It is possible to
* Give the arguments in any order provided that you write the corresponding argument variable name
* Set defaults values to variables so that they become optional
Can you write Python code for it?

def fancy_power(x, p=2, debug=False):
    """
    Here is a fancy version of power that computes the square of the argument or other powers if p is set
    """
    if debug:
        print( "\"fancy_power\" is called with x =", x, " and p =", p)
    return(x**p)

print(fancy_power(5))
print(fancy_power(5,p=3))

res = fancy_power(p=8,x=2,debug=True)
print(res)


I am working on the file "1_Basics.ipynb".
The first task is:
Booleans
Boolean is the type of a variable True or False and thus are extremely useful when coding.
* They can be obtained by comparisons  >, >= (greater, greater or égal), <, <= (smaller, smaller or equal) or membership == , != (equality, different).
* They can be manipulated by the logical operations and, not, or.
Can you write Python code for it?


I am working on the file "1_Basics.ipynb".
The first task is:
Exercise 2:  Fibonacci
The Fibonacci seqence is a sequence of numbers where the next number in the sequence is the sum of the previous two numbers in the sequence. The sequence looks like this: 1, 1, 2, 3, 5, 8, 13. Write a function that generate a given number of elements of the Fibonacci sequence.

Exercise 3: Implement quicksort
The wikipedia page describing this sorting algorithm gives the following pseudocode:

function quicksort('array')
   if length('array') <= 1
        return 'array'
   select and remove a pivot value 'pivot' from 'array'
   create empty lists 'less' and 'greater'
   for each 'x' in 'array'
       if 'x' <= 'pivot' then append 'x' to 'less'
       else append 'x' to 'greater'
   return concatenate(quicksort('less'), 'pivot', quicksort('greater'))

Create a function that sorts a list using quicksort
Can you write Python code for it?

def quicksort(l):
    # ...
    return None

res = quicksort([-2, 3, 5, 1, 3])
print(res)


I am working on the file "1_Basics.ipynb".
The first task is:
If the index is needed along with the value, the function enumerate is useful.
Can you write Python code for it?

for idx, x in enumerate(range(-3,3)):
    print(idx, x)


I am working on the file "1_Basics.ipynb".
The first task is:
Lists
Lists are the base element for sequences of variables in python, they are themselves a variable type.
* The syntax to write them is [ ... , ... ]
* The types of the elements may not be all the same
* The indices begin at $0$ (l[0] is the first element of l)
* Lists can be nested (lists of lists of ...)
Warning: Another type called tuple with the syntax ( ... , ... ) exists in Python. It has almost the same structure than list to the notable exceptions that one cannot add or remove elements from a tuple. We will see them briefly later
Can you write Python code for it?

l = [1, 2, 3, [4,8] , True , 2.3]
print(l, type(l))

print(l[0],type(l[0]))
print(l[3],type(l[3]))
print(l[3][1],type(l[3][1]))

print(l)
print(l[4:]) # l[4:] is l from the position 4 (included)
print(l[:5]) # l[:5] is l up to position 5 (excluded)
print(l[4:5]) # l[4:5] is l between 4 (included) and 5 (excluded) so just 4
print(l[1:6:2])  # l[1:6:2] is l between 1 (included) and 6 (excluded) by steps of 2 thus 1,3,5
print(l[::-1]) # reversed order
print(l[-1]) # last element


I am working on the file "1_Basics.ipynb".
The first task is:
Operation on numbers
The usual operations are
* Multiplication and Division with respecively * and /
* Exponent with **
* Modulo with %
Can you write Python code for it?

print(7 * 3., type(7 * 3.))  # int x float -> float

print(3/2, type(3/2))  # Warning: int in Python 2,  float in Python 3
print(3/2., type(3/2.)) # To be sure

print(2**10, type(2**10))

print(8%2, type(8%2))


I am working on the file "1_Basics.ipynb".
The first task is:
Outputs
return outputs a variable, tuple, dictionary, ...
Can you write Python code for it?

def square(x):
    """
    Return x squared.
    """
    return(x ** 2)

help(square)
res = square(12)
print(res)

def powers(x):
    """
    Return the first powers of x.
    """
    return(x ** 2, x ** 3, x ** 4)

help(powers)

res = powers(12)
print(res, type(res))

two,three,four = powers(3)
print(three,type(three))

def powers_dict(x):
    """
    Return the first powers of x as a dictionary.
    """
    return{"two": x ** 2, "three": x ** 3,  "four": x ** 4}

res = powers_dict(12)
print(res, type(res))
print(res["two"],type(res["two"]))


I am working on the file "1_Basics.ipynb".
The first task is:
Strings and text formatting

Strings are delimited with (double) quotes. They can be handled globally the same way as lists (see above).
<tt>print</tt> displays (tuples of) variables (not necessarily strings).
To include variable into string, it is preferable to use the <tt>format</tt> method.

Warning: text formatting and notably the print method is one of the major differences between Python 2 and Python 3. The method presented here is clean and works in both versions.
Can you write Python code for it?

s = "test"
print(s,type(s))

print(s[0])
print(s + "42")

print(s,42)
print(s+"42")

try:
    print(s+42)
except Exception as error:
    print(error)


I am working on the file "1_Basics.ipynb".
The first task is:
The end of the file has be reached so the command returns ''.
To get to the top, use  f.seek(offset, from_what). The position is computed from adding offset to a reference point; the reference point is selected by the from_what argument. A from_what value of 0 measures from the beginning of the file, 1 uses the current file position, and 2 uses the end of the file as the reference point. from_what can be omitted and defaults to 0, using the beginning of the file as the reference point. Thus f.seek(0) goes to the top.
Can you write Python code for it?


I am working on the file "1_Basics.ipynb".
The first task is:
Try [*]
When a command may fail, you can try to execute it and optionally catch the Exception (i.e. the error).
Can you write Python code for it?

a = [1,2,3]
print(a)

try:
    a[1] = 3
    print("command ok")
except Exception as error:
    print(error)

print(a) # The command went through

try:
    a[6] = 3
    print("command ok")
except Exception as error:
    print(error)

print(a) # The command failed


I am working on the file "1_Basics.ipynb".
The first task is:
Tuples, Dictionaries [*]

Tuples are similar to list but are created with (...,...) or simply comas. They cannot be changed once created.
Can you write Python code for it?

t = (1,'b',876876.908)
print(t,type(t))
print(t[0])

a,b = 12,[987,98987]
u = a,b
print(a,b,u)

try:
    u[1] = 2
except Exception as error:
    print(error)


I am working on the file "1_Basics.ipynb".
The first task is:
Warning: For the file to be actually written and being able to be opened and modified again without mistakes, it is primordial to close the file handle with f.close()
f.read() will read an entire file and put the pointer at the end.
Can you write Python code for it?

f = open('./data/text.txt', 'r')
f.read()

f.read()


I am working on the file "1_Basics.ipynb".
The first task is:
While loop
Similarly to for loops, the syntax iswhile condition: followed by an indentation of one tab which represents what will be executed.
Can you write Python code for it?


I am working on the file "1_Basics.ipynb".
The first task is:
You can have void lists and concatenate list by simply using the + operator, or even repeat them with * .
Can you write Python code for it?


I am working on the file "1_Data_cleaning.ipynb".
The first task is:
Step2
Compute a pandas dataframe with the payloads from the different collections
dataframe columns:
$<is\_malicious> | <injection\_type> | <payload>$
example:
$1 | SQL | ' OR 1=1 LIMIT 1 #$
Can you write Python code for it?

def from_txt_to_dataframe(src_file,is_malicious,injection_type):

    #read file
    payloads_txt = open('data/{}.txt'.format(src_file),'r',encoding='UTF-8').readlines()

    #create dataframe
    payloads = pd.DataFrame(payloads_txt,columns=['payload'])
    payloads['is_malicious'] = [is_malicious]*len(payloads)
    payloads['injection_type'] = [injection_type]*len(payloads)

    print('First 5 lines of ' + injection_type)
    display(payloads.head())

    return payloads

#concatenate all payload dataframes together
payloads = pd.DataFrame(columns=['payload','is_malicious','injection_type'])
payloads = payloads.append(from_txt_to_dataframe('SQLCollection',1,'SQL'))
payloads = payloads.append(from_txt_to_dataframe('XSSCollection',1,'XSS'))
payloads = payloads.append(from_txt_to_dataframe('ShellCollection',1,'SHELL'))
payloads = payloads.append(from_txt_to_dataframe('non-maliciousCollection',0,'LEGAL'))
payloads = payloads.reset_index(drop=True)


I am working on the file "1_Data_cleaning.ipynb".
The first task is:
Step2
clean the data by
- removing ending '\n'
- removing duplicates
- removing empty data points
- removing malicious data points with length 1
- shuffle the data set
Can you write Python code for it?

#Remove ending \n and white spaces
payloads['payload'] = payloads['payload'].str.strip('\n')
payloads['payload'] = payloads['payload'].str.strip()

#Remove any empty data points
rows_before = len(payloads['payload'])
payloads = payloads[payloads['payload'].str.len() != 0]
print('Empty data points removed: ' + str(rows_before - len(payloads)))

#Remove any malicious data points of size 1
rows_before = len(payloads['payload'])
payloads = payloads[(payloads['is_malicious'] == 0) | ((payloads['is_malicious'] == 1) & (payloads['payload'].str.len() > 1))]
print('Malicious data points of size 1 removed: ' + str(rows_before-len(payloads)))

#Remove duplicates
rows_before = len(payloads['payload'])
payloads = payloads.drop_duplicates(subset='payload', keep='last')
print('Duplicate data points removed: ' + str(rows_before-len(payloads)))

#Reformat rows that have the format b'<payload>' into <payload>
payloads['payload'] = [payload[2:-1] if payload.startswith("b'") or payload.startswith('b"')
                        else payload for payload in payloads['payload']]

#Shuffle dataset and reset indices again
payloads = payloads.sample(frac=1).reset_index(drop=True)
payloads.index.name = 'index'

#Remove payloads that cant be saved into .csv using pandas, e.g. they will be null/NA/NaN
payloads.to_csv('data/payloads.csv',encoding='UTF-8')
#reload dataframe from saved .csv. The dataframe will contain a few null values
payloads = pd.read_csv("data/payloads.csv",index_col='index',encoding='UTF-8')
rows_before = len(payloads['payload'])
payloads = payloads[~payloads['payload'].isnull()]
print('null/NaN data points removed: ' + str(rows_before-len(payloads)))

#Lastly, save to .csv
payloads.to_csv('data/payloads.csv',encoding='UTF-8')


I am working on the file "1_Finding_Lane_Lines.ipynb".
The first task is:
At this point, if you were successful you probably have the Hough line segments drawn onto the road, but what about identifying the full extent of the lane and marking it clearly as in the example video (P1_example.mp4)?  Think about defining a line to run the full length of the visible lane based on the line segments you identified with the Hough Transform.  Modify your draw_lines function accordingly and try re-running your pipeline.
Now for the one with the solid yellow lane on the left. This one's more tricky!
Can you write Python code for it?

yellow_output = 'yellow.mp4'
clip2 = VideoFileClip('solidYellowLeft.mp4')
yellow_clip = clip2.fl_image(process_image)
%time yellow_clip.write_videofile(yellow_output, audio=False)

HTML("""
<video width="960" height="540" controls>
  <source src="{0}">
</video>
""".format(yellow_output))


I am working on the file "1_Finding_Lane_Lines.ipynb".
The first task is:
Play the video inline, or if you prefer find the video in your filesystem (should be in the same directory) and play it in your video player of choice.
Can you write Python code for it?

HTML("""
<video width="960" height="540" controls>
  <source src="{0}">
</video>
""".format(white_output))


I am working on the file "1_Finding_Lane_Lines.ipynb".
The first task is:
Reflections
Congratulations on finding the lane lines!  As the final step in this project, we would like you to share your thoughts on your lane finding pipeline... specifically, how could you imagine making your algorithm better / more robust?  Where will your current algorithm be likely to fail?
Please add your thoughts below,  and if you're up for making your pipeline more robust, be sure to scroll down and check out the optional challenge video below!

The pipeline follows the standard process: read the image, Gaussian blur, canny transform, limit the region and then hough transform.

The draw_lines function can be further polished. For example, instead of fitting a linear line, we can fit a polynomial line; And also, for the very short line, the slope might not be stable. Some filters could be added on.

Some parameters in the pipeline could be further optimized.

Submission
If you're satisfied with your video outputs it's time to submit!  Submit this ipython notebook for review.
Optional Challenge - Not complete
Try your lane finding pipeline on the video below.  Does it still work?  Can you figure out a way to make it more robust?  If you're up for the challenge, modify your pipeline so it works with this video and submit it along with the rest of your project!
Can you write Python code for it?

challenge_output = 'extra.mp4'
clip2 = VideoFileClip('challenge.mp4')
challenge_clip = clip2.fl_image(process_image)
%time challenge_clip.write_videofile(challenge_output, audio=False)

HTML("""
<video width="960" height="540" controls>
  <source src="{0}">
</video>
""".format(challenge_output))


I am working on the file "1_Finding_Lane_Lines.ipynb".
The first task is:
Some OpenCV functions (beyond those introduced in the lesson) that might be useful for this project are:
cv2.inRange() for color selection
cv2.fillPoly() for regions selection
cv2.line() to draw lines on an image given endpoints
cv2.addWeighted() to coadd / overlay two images
cv2.cvtColor() to grayscale or change color
cv2.imwrite() to output images to file
cv2.bitwise_and() to apply a mask to an image
Check out the OpenCV documentation to learn about these and discover even more awesome functionality!
Below are some helper functions to help get you started. They should look familiar from the lesson!
Can you write Python code for it?

import math

def grayscale(img):
    """Applies the Grayscale transform
    This will return an image with only one color channel
    but NOTE: to see the returned image as grayscale
    (assuming your grayscaled image is called 'gray')
    you should call plt.imshow(gray, cmap='gray')"""
    return cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
    # Or use BGR2GRAY if you read an image with cv2.imread()
    # return cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

def canny(img, low_threshold, high_threshold):
    """Applies the Canny transform"""
    return cv2.Canny(img, low_threshold, high_threshold)

def gaussian_blur(img, kernel_size):
    """Applies a Gaussian Noise kernel"""
    return cv2.GaussianBlur(img, (kernel_size, kernel_size), 0)

def region_of_interest(img, vertices):
    """
    Applies an image mask.

    Only keeps the region of the image defined by the polygon
    formed from `vertices`. The rest of the image is set to black.
    """
    #defining a blank mask to start with
    mask = np.zeros_like(img)

    #defining a 3 channel or 1 channel color to fill the mask with depending on the input image
    if len(img.shape) > 2:
        channel_count = img.shape[2]  # i.e. 3 or 4 depending on your image
        ignore_mask_color = (255,) * channel_count
    else:
        ignore_mask_color = 255

    #filling pixels inside the polygon defined by "vertices" with the fill color
    cv2.fillPoly(mask, vertices, ignore_mask_color)

    #returning the image only where mask pixels are nonzero
    masked_image = cv2.bitwise_and(img, mask)
    return masked_image

def draw_lines(img, lines, color=[255, 0, 0], thickness=2):
    """
    NOTE: this is the function you might want to use as a starting point once you want to
    average/extrapolate the line segments you detect to map out the full
    extent of the lane (going from the result shown in raw-lines-example.mp4
    to that shown in P1_example.mp4).

    Think about things like separating line segments by their
    slope...


I am working on the file "1_Finding_Lane_Lines.ipynb".
The first task is:
Test on Images
Now you should build your pipeline to work on the images in the directory "test_images"
You should make sure your pipeline works well on these images before you try the videos.
Can you write Python code for it?


I am working on the file "1_Finding_Lane_Lines.ipynb".
The first task is:
run your solution on all test_images and make copies into the test_images directory).
Can you write Python code for it?

#Pipeline
# TODO: Build your pipeline that will draw lane lines on the test_images
# then save them to the test_images directory.
for im_name in os.listdir("test_images/"):

    #This is used for video test
    #if im_name == 'frame220.jpg':
    #    continue

    #read image
    im_name = "test_images/" + im_name
    image = mpimg.imread(im_name)
    gray = grayscale(image)

    #gaussian blur
    k_size = 5
    blur_gray = gaussian_blur(gray, k_size)

    #canny
    low_thres = 50
    high_thres = 150
    edges = canny(blur_gray, low_thres, high_thres)

    #limit region
    ysize = image.shape[0]
    xsize = image.shape[1]
    left_bottom = [100, ysize]
    right_bottom = [(xsize-100), ysize]
    left_top = [xsize/2, 300]
    right_top = [xsize/2, 300]
    vertices = np.array([[left_bottom, right_bottom, right_top, left_top]], dtype = np.int32)
    masked_edges = region_of_interest(edges, vertices)

    #hough transform
    rho = 1
    theta = np.pi/180
    threshold = 50
    min_line_len = 20
    max_line_gap = 5
    hough_img = hough_lines(masked_edges, rho, theta, threshold, min_line_len, max_line_gap)
    combo = weighted_img(hough_img, image)

    out_name = im_name.split('.')[0] + '_output.jpg'
    mpimg.imsave(out_name,combo)

#Partial test

gray = grayscale(image)

k_size = 5
blur_gray = gaussian_blur(gray, k_size)
low_thres = 50
high_thres = 150
edges = canny(blur_gray, low_thres, high_thres)
plt.imshow(edges, cmap='Greys_r')

#Partial test

ysize = image.shape[0]
xsize = image.shape[1]
left_bottom = [100, ysize]
right_bottom = [xsize, ysize]
#left_top = [xsize/2, ysize/2]
#right_top = [xsize/2, ysize/2]
left_top = [xsize/2, 300]
right_top = [xsize/2, 300]

vertices = np.array([[left_bottom, right_bottom, right_top, left_top]], dtype = np.int32)
masked_edges = region_of_interest(edges, vertices)
plt.imshow(masked_edges, cmap='Greys_r')

#Partial test

rho = 1
theta = np.pi/180
threshold = 50
min_line_len = 20
max_line_gap = 5

hough_img = hough_lines(masked_edges, rho,...


I am working on the file "1_In_en_Uitvoer.ipynb".
The first task is:
Filteren
In onze invoer staan dus allemaal overbodige tekens, die moeten we er uit gaan filteren
Can you write Python code for it?

# we gaan nu de inhoud van sudoku_invoer filteren zodat alleen de toegestane tekens overblijven
TEKENS = ['1','2','3','4']
toegestane_invoer = TEKENS + ['.']

gefilterd = [teken for teken in invoer if teken in toegestane_invoer]

# ook veranderen we de sudoku van een 'string' (een serie letters)
# naar een list (een lijst met inhoud)
print(gefilterd)

# van een lijst kunnen we nu bijvoorbeeld alleen het eerste (begin bij nul te tellen):
print('eerste element:', gefilterd[0])
print('tweede tot vierde element:', gefilterd[1:4])


I am working on the file "1_In_en_Uitvoer.ipynb".
The first task is:
Nieuwe regels
Can you write Python code for it?

# alles achter elkaar...
for i, s in enumerate(sudoku):
    print(str(s),end='')

# modulus
KOLOMMEN = 4
for i, s in enumerate(sudoku):
    print(i,i % KOLOMMEN, s)

# alles achter elkaar, maar soms een nieuwe regel
# modulus
KOLOMMEN = 4
for i, s in enumerate(sudoku):
    if ( (i+1) % KOLOMMEN) == 0:
        print(s)
    else:
        print(s,end='')


I am working on the file "1_In_en_Uitvoer.ipynb".
The first task is:
Onbekenden in de sudoku
In onze invoer staan nu punten, dat betekent dat we nog niet weten wat daar moet staan, en dat zou dus 1,2,3 of 4 kunnen zijn. We gaan ieder element in de invoer vervangen voor alle mogelijkheden
Can you write Python code for it?

sudoku = [TEKENS if teken == '.' else teken for teken in gefilterd]
print(sudoku)

## Set
sudoku = [set(letters) for letters in sudoku]
print(sudoku)


# de lijst met tekens in sudoko kunnen we ook weer netjes als een sudoku weergeven

# een voor een de sudoku weergeven (ieder cijfer op nieuwe regel)
for getal in [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]:
    print(getal, sudoku[getal])

for i, s in enumerate(sudoku):
    print(i, s)


I am working on the file "1_Introduction_to_Matplotlib.ipynb".
The first task is:
Clearly, the log scale subplot is far more readable - you can infer that the minimum sales is around 0, the median is approximtely in the middle of 100 and 1000, and the max is reaching 100,000.
Histogram
Histograms are useful for visualising distribution of single variables.
Can you write Python code for it?

# Histograms

plt.hist(df['Sales'])
plt.show()

# The histogram can be made more readable by using
# a log scale

plt.hist(df['Sales'])
plt.yscale('log')
plt.show()


I am working on the file "1_Introduction_to_Matplotlib.ipynb".
The first task is:
Figures and Subplots
You often need to create multiple plots in the same figure, as we'll see in some upcoming examples.
matplotlib has the concept of figures and subplots using which you can create multiple subplots inside the same figure.
To create multiple plots in the same figure, you can use the method plt.subplot(nrows, ncols, nsubplot).
Can you write Python code for it?

x = np.linspace(1, 10, 100)
y = np.log(x)

# initiate a new figure explicitly
plt.figure(1)

# Create a subplot with 1 row, 2 columns

# create the first subplot in figure 1
plt.subplot(121)     # equivalent to plt.subplot(1, 2, 1)
plt.title("y = log(x)")
plt.plot(x, y)

# create the second subplot in figure 1
plt.subplot(122)
plt.title("y = log(x)**2")
plt.plot(x, y**2)
plt.show()


I am working on the file "1_Introduction_to_Matplotlib.ipynb".
The first task is:
Let's see another example - say you want to create 4 subplots in two rows and two columns.
Can you write Python code for it?

# Example: Create a figure having 4 subplots
x = np.linspace(1, 10, 100)

# Optional command, since matplotlib creates a figure by default anyway
plt.figure(1)

# subplot 1
plt.subplot(2, 2, 1)
plt.title("Linear")
plt.plot(x, x)

# subplot 2
plt.subplot(2, 2, 2)
plt.title("Cubic")
plt.plot(x, x**3)

# subplot 3
plt.figure(2)
plt.subplot(2, 2, 1)
plt.title("Log")
plt.plot(x, np.log(x))

# subplot 4
plt.subplot(2, 2, 2)
plt.title("Exponential")
plt.plot(x, x**2)

plt.show()


I am working on the file "1_Introduction_to_Matplotlib.ipynb".
The first task is:
Let's see how to put labels and the x and y axes and the chart title.
Also, you can specify the limits of x and y labels as a range using xlim([xmin, xmax]) and ylim([ymin, ymax]).
Can you write Python code for it?

# Axis labels and title
plt.plot(x, y)

# x and y labels, and title
plt.xlabel("Current")
plt.ylabel("Voltage")
plt.title("Ohm's Law")

# Define the range of labels of the axis
# Arguments: plt.axis(xmin, xmax, ymin, ymax)
plt.xlim([20, 80])
plt.ylim([200, 800])
plt.show()

# Change the colors and line type

# initialising x and y arrays
x = np.linspace(0, 10, 20)
y = x*2

# color blue, line type '+'
plt.plot(x, y, 'b+')

# put x and y labels, and the title
plt.xlabel("Current")
plt.ylabel("Voltage")
plt.title("Ohm's Law")

plt.show()

# Plotting multiple lines on the same plot

x = np.linspace(0, 5, 10)
y = np.linspace(3, 6, 10)

# plot three curves: y, y**2 and y**3 with different line types
plt.plot(x, y, 'r-', x, y**2, 'b+', x, y**3, 'g^')
plt.show()


I am working on the file "1_Introduction_to_Matplotlib.ipynb".
The first task is:
Scatter Plot
Scatter plots are used to visualise two variables, one one each axis.
Can you write Python code for it?

# Scatter plots with two variables: Profit and Sales
plt.scatter(df['Sales'], df['Profit'])
plt.show()


I am working on the file "1_Introduction_to_Matplotlib.ipynb".
The first task is:
The solution to this problem is to change the scale of the axis (in this case, the y axis) so that the range can fit into the size of the plot.
One commonly used technique is to transform an axis into the logarithmic scale. You can transform the scale of an axis using plt.yscale('log').
Can you write Python code for it?

# Usual (linear) scale subplot
plt.subplot(1, 2, 1)
plt.boxplot(df['Sales'])

# log scale subplot
plt.subplot(1, 2, 2)
plt.boxplot(df['Sales'])
plt.yscale('log')
plt.show()


I am working on the file "1_Introduction_to_Matplotlib.ipynb".
The first task is:
Working with Images
matplotlib can also read images using the plt.imread() method. Internally, it reads and stores images as an array. The array can then be used for various data manipulation tasks, just as a normal array.
Let's look at an example.
Can you write Python code for it?

# reading a PNG image
image = plt.imread("number.png")
plt.imshow(image)
plt.show()

# looking at attributes of the image

print(type(image))
print(image.shape)
print(image.dtype)


I am working on the file "1_Linear_Regression.ipynb".
The first task is:
Finally, we fit the model on training set using the LinearRegression() class, and then predict on the test set to check the Root Mean Squared Error (RMSE)
Can you write Python code for it?

lr = LinearRegression(lamda=0)
lr.fit(X_train, y_train)
y_pred = lr.predict(X_test)
print "RMSE is {:.4}".format(np.sqrt(np.mean((y_test.reshape((len(y_test), 1)) - y_pred) ** 2)))


I am working on the file "1_Linear_Regression.ipynb".
The first task is:
We can then randomly split the train and test set
Can you write Python code for it?

from sklearn.cross_validation import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=22)


I am working on the file "1_Model_by_Individual.ipynb".
The first task is:
DATA EXPLORATION
Plot for: Education ~ Age
Can you write Python code for it?

fig = plt.figure(figsize=(16,12))
ax1 = fig.add_subplot(2,2,1)
ax2 = fig.add_subplot(2,2,2)
ax3 = fig.add_subplot(2,2,3)
ax4 = fig.add_subplot(2,2,4)

ax1.plot(dataModel.education,dataModel.P47T,'ro')
ax1.set_ylabel('Ingreso total')
ax1.set_xlabel('Educacion')
ax2.plot(dataModel.age,dataModel.P47T,'ro')
ax2.set_xlabel('Edad')
ax3.plot(dataModel.education,dataModel.P21,'bo')
ax3.set_ylabel('Ingreso Laboral')
ax3.set_xlabel('Educacion')
ax4.plot(dataModel.age,dataModel.P21,'bo')
ax4.set_xlabel('Edad')


I am working on the file "1_Model_by_Individual.ipynb".
The first task is:
MODEL # 2 - ECLAC (Using Log of Individual Income)
Can you write Python code for it?

dataModel2 = functionsForModels.runModel(dataModel, income = 'lnIncome', variables= [
        'primary','secondary','university',
        'male_14to24','male_25to34',
        'female_14to24', 'female_25to34', 'female_35more'])


I am working on the file "1_Model_by_Individual.ipynb".
The first task is:
MODEL # 4 - ECLAC (Using Log of Total Individual Income)
Can you write Python code for it?

dataModel4 = functionsForModels.runModel(dataModel, income = 'lnIncomeT')


I am working on the file "1_Model_by_Individual.ipynb".
The first task is:
Plot for: LnIncome ~ Educ and Age
Can you write Python code for it?

g = sns.JointGrid(x="education", y="lnIncome", data=dataModel)
g.plot_joint(sns.regplot, order=2)
g.plot_marginals(sns.distplot)

g2 = sns.JointGrid(x="age", y="lnIncome", data=dataModel)
g2.plot_joint(sns.regplot, order=2)
g2.plot_marginals(sns.distplot)


I am working on the file "1_Model_by_Individual.ipynb".
The first task is:
REGRESSION MODEL (ALTERNATIVE)
Background:
We tested an alternative model similar to the ECLAC Regression Model using a second polynomial to account for the non-linear relationship between age, education and income.
MODEL # 1 - ALTERNATIVE (Using Log of Total Individual Income)
Can you write Python code for it?

dataModel5 = functionsForModels.runModel(dataModel, income = 'lnIncomeT', variables=['education','education2',
                                'age','age2','female'])


I am working on the file "1_Model_by_Individual.ipynb".
The first task is:
REGRESSION MODEL (ECLAC)
Background:
The ECLAC (Economic Comission for Latin America and the Caribbean) estimates income by using a regression model based on the following variables (education, gender and age):
- x1: primary
- x2: secondary
- x3: university
- x4: male_14to24
- x5: male_25to34
- x6: female_14to24
- x7: female_25to34
- x8: female_35more
MODEL # 1 - ECLAC
Can you write Python code for it?

dataModel1 = functionsForModels.runModel(dataModel, income = 'P21')


I am working on the file "1_Model_by_Individual.ipynb".
The first task is:
Reference:

P21: Refers to individual income by main activity (occupation)
P47T: Refers to total individual income (includes capital gains)
lnIncomeT: Refers to ln of P21
lnIncome: Refers to ln of P47T

Plot for: LnIncome
Can you write Python code for it?

fig = plt.figure(figsize=(16,12))
ax1 = fig.add_subplot(2,2,1)
ax2 = fig.add_subplot(2,2,2)
ax3 = fig.add_subplot(2,2,3)
ax4 = fig.add_subplot(2,2,4)

sns.kdeplot(dataModel.P47T,ax=ax1,color = 'red')
sns.kdeplot(dataModel.lnIncomeT,ax=ax2,color = 'red')
sns.kdeplot(dataModel.P21,ax=ax3)
sns.kdeplot(dataModel.lnIncome,ax=ax4)

print 'mean:', dataModel.lnIncome.mean(), 'std:', dataModel.lnIncome.std()

print 'mean:', dataModel.P21.mean(), 'std:', dataModel.P21.std()

plt.boxplot(list(dataModel.P21), 0, 'gD')


I am working on the file "1_Radio_Science/01_01_a_brief_introduction_to_basic_astrophysics.ipynb".
The first task is:
Import section specific modules:
Can you write Python code for it?


I am working on the file "1_Radio_Science/01_03_radiation_transport.ipynb".
The first task is:
Import section specific modules:
Can you write Python code for it?