I am working on the file "02_Convolutional_Neural_Network.ipynb".
The first task is:
The MNIST data-set has now been loaded and consists of 70,000 images and associated labels (i.e. classifications of the images). The data-set is split into 3 mutually exclusive sub-sets. We will only use the training and test-sets in this tutorial.
Can you write Python code for it?

print("Size of:")
print("- Training-set:\t\t{}".format(len(data.train.labels)))
print("- Test-set:\t\t{}".format(len(data.test.labels)))
print("- Validation-set:\t{}".format(len(data.validation.labels)))


I am working on the file "02_Convolutional_Neural_Network.ipynb".
The first task is:
The MNIST data-set has now been loaded and consists of 70.000 images and class-numbers for the images. The data-set is split into 3 mutually exclusive sub-sets. We will only use the training and test-sets in this tutorial.
Can you write Python code for it?

print("Size of:")
print("- Training-set:\t\t{}".format(data.num_train))
print("- Validation-set:\t{}".format(data.num_val))
print("- Test-set:\t\t{}".format(data.num_test))


I am working on the file "02_Convolutional_Neural_Network.ipynb".
The first task is:
The class-labels are One-Hot encoded, which means that each label is a vector with 10 elements, all of which are zero except for one element. The index of this one element is the class-number, that is, the digit shown in the associated image. We also need the class-numbers as integers for the test-set, so we calculate it now.
Can you write Python code for it?

data.test.cls = np.argmax(data.test.labels, axis=1)


I am working on the file "02_Convolutional_Neural_Network.ipynb".
The first task is:
The convolutional layers expect x to be encoded as a 4-dim tensor so we have to reshape it so its shape is instead [num_images, img_height, img_width, num_channels]. Note that img_height == img_width == img_size and num_images can be inferred automatically by using -1 for the size of the first dimension. So the reshape operation is:
Can you write Python code for it?

x_image = tf.reshape(x, [-1, img_size, img_size, num_channels])


I am working on the file "02_Convolutional_Neural_Network.ipynb".
The first task is:
The following images are the results of applying the convolutional filters to the second image.
Can you write Python code for it?


I am working on the file "02_Convolutional_Neural_Network.ipynb".
The first task is:
The step-size for moving the filter across the input is called the stride. There is a stride for moving the filter horizontally (x-axis) and another stride for moving vertically (y-axis).
In the source-code below, the stride is set to 1 in both directions, which means the filter starts in the upper left corner of the input image and is being moved 1 pixel to the right in each step. When the filter reaches the end of the image to the right, then the filter is moved back to the left side and 1 pixel down the image. This continues until the filter has reached the lower right corner of the input image and the entire output image has been generated.
When the filter reaches the end of the right-side as well as the bottom of the input image, then it can be padded with zeroes (white pixels). This causes the output image to be of the exact same dimension as the input image.
Furthermore, the output of the convolution may be passed through a so-called Rectified Linear Unit (ReLU), which merely ensures that the output is positive because negative values are set to zero. The output may also be down-sampled by so-called max-pooling, which considers small windows of 2x2 pixels and only keeps the largest of those pixels. This halves the resolution of the input image e.g. from 28x28 to 14x14 pixels.
Note that the second convolutional layer is more complicated because it takes 16 input channels. We want a separate filter for each input channel, so we need 16 filters instead of just one. Furthermore, we want 36 output channels from the second convolutional layer, so in total we need 16 x 36 = 576 filters for the second convolutional layer. It can be a bit challenging to understand how this works.
Imports
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
from sklearn.metrics import confusion_matrix
import time
from datetime import timedelta
import math


I am working on the file "02_Convolutional_Neural_Network.ipynb".
The first task is:
This calculates the classification accuracy by first type-casting the vector of booleans to floats, so that False becomes 0 and True becomes 1, and then calculating the average of these numbers.
Can you write Python code for it?

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))


I am working on the file "02_Convolutional_Neural_Network.ipynb".
The first task is:
This was developed using Python 3.6 (Anaconda) and TensorFlow version:
Can you write Python code for it?


I am working on the file "02_Convolutional_Neural_Network.ipynb".
The first task is:
Visualization of Weights and Layers
In trying to understand why the convolutional neural network can recognize handwritten digits, we will now visualize the weights of the convolutional filters and the resulting output images.
Helper-function for plotting convolutional weights
Can you write Python code for it?

def plot_conv_weights(weights, input_channel=0):
    # Assume weights are TensorFlow ops for 4-dim variables
    # e.g. weights_conv1 or weights_conv2.

    # Retrieve the values of the weight-variables from TensorFlow.
    # A feed-dict is not necessary because nothing is calculated.
    w = session.run(weights)

    # Get the lowest and highest values for the weights.
    # This is used to correct the colour intensity across
    # the images so they can be compared with each other.
    w_min = np.min(w)
    w_max = np.max(w)

    # Number of filters used in the conv. layer.
    num_filters = w.shape[3]

    # Number of grids to plot.
    # Rounded-up, square-root of the number of filters.
    num_grids = math.ceil(math.sqrt(num_filters))

    # Create figure with a grid of sub-plots.
    fig, axes = plt.subplots(num_grids, num_grids)

    # Plot all the filter-weights.
    for i, ax in enumerate(axes.flat):
        # Only plot the valid filter-weights.
        if i<num_filters:
            # Get the weights for the i'th filter of the input channel.
            # See new_conv_layer() for details on the format
            # of this 4-dim tensor.
            img = w[:, :, input_channel, i]

            # Plot image.
            ax.imshow(img, vmin=w_min, vmax=w_max,
                      interpolation='nearest', cmap='seismic')

        # Remove ticks from the plot.
        ax.set_xticks([])
        ax.set_yticks([])

    # Ensure the plot is shown correctly with multiple plots
    # in a single Notebook cell.
    plt.show()


I am working on the file "02_Convolutional_Neural_Network.ipynb".
The first task is:
We could also have a placeholder variable for the class-number, but we will instead calculate it using argmax. Note that this is a TensorFlow operator so nothing is calculated at this point.
Can you write Python code for it?


I am working on the file "02_Convolutional_Neural_Network.ipynb".
The first task is:
We have now calculated the cross-entropy for each of the image classifications so we have a measure of how well the model performs on each image individually. But in order to use the cross-entropy to guide the optimization of the model's variables we need a single scalar value, so we simply take the average of the cross-entropy for all the image classifications.
Can you write Python code for it?


I am working on the file "02_DataAcquisition_and_Preparation.ipynb".
The first task is:
2.1 Get the Main Delay Data for 2014 from Downloaded zip Files
First, we want to open and combine the zipped data files for each month of the delay data that has been downloaded according to the process outlined in the data aquisition section above. As we have more than 400,000 recorded flights each month, the dataset is extremely large. We therefore decided to focus on the subset of all flights in 2014 to get the most relevant flight information without missing certain months (this could be important when investigating seasonality effects).
Can you write Python code for it?

# reads all predefined months for a year and merge into one data frame
rawData = pd.DataFrame()
months = ['01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11', '12']
for m in months:
    z = zipfile.ZipFile('cache/{y}{mo}.zip'.format(y=str(2014), mo = m))
    rawData = rawData.append(pd.read_csv(z.open(z.namelist()[0])))
    print "Downloaded", m
# reset index of complete dataset for delays to prepare merging in next step
rawData.reset_index(inplace=True)


I am working on the file "02_DataAcquisition_and_Preparation.ipynb".
The first task is:
A quick check reveals, that filtering was not (really) necessary as all Airlines are still active today.
Can you write Python code for it?

filteredData2014.count()[0], data2014.count()[0]

# save data to cache
filteredData2014.to_csv('cache/linear_model_data.csv')


I am working on the file "02_DataAcquisition_and_Preparation.ipynb".
The first task is:
Filtering the data for active Airlines only
As we want to predict delay times, we throw out any flights that are operated by a shutdown airline.
Can you write Python code for it?

df_active_airlines = pd.read_csv('data/cur_airlines.txt', header=None)
df_active_airlines.columns = [['UNIQUE_CARRIER']];
df_active_airlines.head()

filteredData2014 = data2014.merge(df_active_airlines, on=['UNIQUE_CARRIER', 'UNIQUE_CARRIER'], how='inner')


I am working on the file "02_DataAcquisition_and_Preparation.ipynb".
The first task is:
However, we just need a subset of these columns for our analysis:
Can you write Python code for it?

selectedColumns = [u'index', u'FL_DATE', u'UNIQUE_CARRIER', u'TAIL_NUM', u'FL_NUM',
                   u'ORIGIN', u'DEST', u'CRS_DEP_TIME', u'DEP_TIME', u'DEP_DELAY', u'TAXI_OUT',
                   u'WHEELS_OFF', u'WHEELS_ON', u'TAXI_IN', u'CRS_ARR_TIME', u'ARR_TIME', u'ARR_DELAY',
                   u'CANCELLED', u'DIVERTED', u'CANCELLATION_CODE', u'AIR_TIME', u'DISTANCE',
                   u'CARRIER_DELAY', u'WEATHER_DELAY', u'NAS_DELAY', u'SECURITY_DELAY', u'LATE_AIRCRAFT_DELAY',
                   u'ORIGIN_CITY_NAME', u'DEST_CITY_NAME']
rawData = rawData[selectedColumns]


I am working on the file "02_DataAcquisition_and_Preparation.ipynb".
The first task is:
Just 0.7% of alll flight origins could not be located, so the merge was quite successful.
2.4 Save the Main Final Dataset
The resulting dataframe complete2014Data will be locally stored as csv file.
Can you write Python code for it?


I am working on the file "02_DataAcquisition_and_Preparation.ipynb".
The first task is:
The columns we now have in the dataset are:
Can you write Python code for it?


I am working on the file "02_DataAcquisition_and_Preparation.ipynb".
The first task is:
We can now join these two tables based on their common ID that is saved in the column MFR MDL CODE of the master table and in the index of the aircraft table respectively.
Can you write Python code for it?

master = df_master[['MFR MDL CODE', 'YEAR MFR']].reset_index()
aircrafts = df_aircrafts['MFR'].reset_index()
master.columns = ['TAIL_NUM', 'CODE', 'YEAR']
aircrafts.columns = ['CODE', 'MFR']
joined = pd.merge(master, aircrafts, how='left', on='CODE')


I am working on the file "02_DataAcquisition_and_Preparation.ipynb".
The first task is:
We now join this aircraft information with our delay data and extend the original dataset with the two new features: The year in which the aircraft was built (to determine the age) and the manufacturer.
Can you write Python code for it?

delayFinal = rawData[['TAIL_NUM','UNIQUE_CARRIER']]
delayFinal.TAIL_NUM = delayFinal.TAIL_NUM.str.strip('N')
delaymfr = pd.merge(delayFinal, joined, how='left', on=['TAIL_NUM'])
rawData['AIRCRAFT_YEAR'] = delaymfr.YEAR
rawData['AIRCRAFT_MFR'] = delaymfr.MFR


I am working on the file "02_DataManipulation.ipynb".
The first task is:
A quick-ish introduction to Pandas
based on http://pandas.pydata.org/pandas-docs/stable/10min.html
Can you write Python code for it?

import numpy as np
import pandas as pd

#use a standard dataset of heterogenous data
cars = pd.read_csv('data/mtcars.csv')
cars.head()

#list all columns
cars.columns

#we want to use the car as the "primary key" of a row
cars.index = cars.pop('car')
cars.head()

#describe our dataset
cars.describe()

cars.sort_index(inplace=True)
cars.head()

cars.sort_values('mpg').head(15)

cars.sort_values('hp', ascending=False).head()


I am working on the file "02_DataManipulation.ipynb".
The first task is:
Accessing elements
Can you write Python code for it?

# get a single element
X[0, 0]

# get a row
X[1]

# get a column
X[:, 1]

# Transposing an array
X.T

print(X.shape)
print(X.reshape(5, 3)) #change the layout of the matrix

# indexing by an array of integers (fancy indexing)
indices = np.array([3, 1, 0])
print(indices)
X[:, indices]


I am working on the file "02_DataManipulation.ipynb".
The first task is:
Operations along an axis
Can you write Python code for it?

X

X.shape

np.sum(X, axis=1) # 1...columns

np.max(X, axis=0) # 0...rows


I am working on the file "02_DataManipulation.ipynb".
The first task is:
Selection
Note While many of the NumPy access methods work on DataFrames, use the pandas-specific data access methods, .at, .iat, .loc, .iloc and .ix.
See the Indexing section and below.
Can you write Python code for it?

#single column
cars['mpg']
#depending on the name also cars.mpg works

#or a slice of rows
cars[2:5]

#by label = primary key
cars.loc['Fiat 128':'Lotus Europa']

#selection by position
cars.iloc[3]

cars.iloc[3:5, 0:2]

cars[cars.cyl > 6] # more than 6 cylinders


I am working on the file "02_DataManipulation.ipynb".
The first task is:
Statistics
besides .describe() there are plenty of other staticial measures and aggregation methods in Pandas/Numpy
Can you write Python code for it?

#stats
cars.mean()

cars.mean(axis=1)

#grouping
cars.groupby('cyl').mean()

#grouping different aggregation methods
cars.groupby('cyl').agg({ 'mpg': 'mean', 'qsec': 'min'})


I am working on the file "02_DataManipulation.ipynb".
The first task is:
TASKS

Now there are a series of simple tasks
Can you write Python code for it?

#loading gapminder data (taken from https://github.com/jennybc/gapminder)
# file located at 'data/gapminder-unfiltered.tsv' it uses tabular character as separator
# use the first column as index

#what are the columns of this dataset?

#what is the maximal year contained?

#just select all data of the year 2007

#locate Austria and print it

#list the top 10 countries by life expectancy (lifeExp)

#what is the total population (pop) per continent


I am working on the file "02_DataManipulation_solution.ipynb".
The first task is:
A quick-ish introduction to Pandas
based on http://pandas.pydata.org/pandas-docs/stable/10min.html
Can you write Python code for it?


I am working on the file "02_DataManipulation_solution.ipynb".
The first task is:
Missing Data
Can you write Python code for it?

cars_na = pd.read_csv('data/mtcars_with_nas.csv')

cars_na.isnull().head(4)

#fill with a default value
cars_na.fillna(0).head(4)

#or drop the rows

print(cars_na.shape)
#drop rows with na values
print(cars_na.dropna().shape)
#drop columns with na values
print(cars_na.dropna(axis=1).shape)

#see also http://pandas.pydata.org/pandas-docs/stable/missing_data.html


I am working on the file "02_DataManipulation_solution.ipynb".
The first task is:
Operations along an axis
Can you write Python code for it?


I am working on the file "02_DataManipulation_solution.ipynb".
The first task is:
Selection
Note While many of the NumPy access methods work on DataFrames, use the pandas-specific data access methods, .at, .iat, .loc, .iloc and .ix.
See the Indexing section and below.
Can you write Python code for it?


I am working on the file "02_DataManipulation_solution.ipynb".
The first task is:
Statistics
besides .describe() there are plenty of other staticial measures and aggregation methods in Pandas/Numpy
Can you write Python code for it?


I am working on the file "02_Linear_regression.ipynb".
The first task is:
Tensorflow 코드와 비교
Coding의 Sytle의 차이점이 명확합니다.
pyTorch는 trainning loop안에서

NeuralNetwork의 Graph를 동적으로 생성
loss로 하고 Back Propagation을 수행
NeuralNetwork의 prameter를 update

합니다.
반면에 TensorFlow은 위의 1, 2, 3과정을 static하게 Graph를 만든후,
session을 생성하여 sess.run을 통해서 값을 산출합니다.
하지만, pytorch도 model, loss, optimizer를 trainning loop전에 선언하여 TensorFlow코드랑 거의 동일한 형태입니다.
Can you write Python code for it?

import tensorflow as tf

x_data = [1, 2, 3]
y_data = [1, 2, 3]

w = tf.Variable(tf.random_uniform([1], -1.0, 1.0))
b = tf.Variable(tf.random_uniform([1], -1.0, 1.0))
x = tf.placeholder(tf.float32, name="X")
y = tf.placeholder(tf.float32, name="Y")

# network model, loss function
y_pred = tf.add(tf.mul(w, x), b)
loss = tf.reduce_mean(tf.square(y_pred - y))

# optimizser
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-2)
train_op = optimizer.minimize(loss)

with tf.Session() as sess:
    # initialize network mdoel
    sess.run(tf.global_variables_initializer())

    # trainning
    for step in range(1000):

        # optimizaer step
        sess.run(train_op, feed_dict={x: x_data, y: y_data})

    print ('w = ', sess.run(w), ', b=', sess.run(b))


I am working on the file "02_Linear_regression.ipynb".
The first task is:
optimizer 이용하여 구현
Can you write Python code for it?

import torch
from torch.autograd import Variable

x = Variable(torch.Tensor([[1], [2], [3]]))
y = Variable(torch.Tensor([[1], [2], [3]]))
w = Variable(torch.randn(1, 1), requires_grad = True)
b = Variable(torch.randn(1), requires_grad = True)

optimizer = torch.optim.Adam((w, b), lr=1e-2)

# trainning
for i in range(1000) :
    # network model
    y_pred = torch.mm(x, w)
    y_pred += b.unsqueeze(1).expand_as(y_pred)
    loss = (y_pred - y).pow(2).sum()

    # initialize network mdoel parameter grad
    optimizer.zero_grad()

    # optimizaer step
    loss.backward()
    optimizer.step()

# training result
print ('w = ', w.data[0][0], ', b=', b.data[0])


I am working on the file "02_Linear_regression.ipynb".
The first task is:
torch.nn에서 제공하는 funcion으로 구현

torch.nn.MSELoss()
torch.nn.funcional.Linear(x)

pyTorch는 다음과 정형화된 형태를 사용할 수 있을 것으로 보입니다.

입력변수 설정(생성)

사전 설정

model
loss
opimizer

Trainning loop

(입력 생성)
model 생성
loss 생성
zeroGrad
backpropagation
optimizer step (update model parameter)

Predict
Can you write Python code for it?

import torch
from torch.autograd import Variable

x = Variable(torch.Tensor([[1], [2], [3]]))
y = Variable(torch.Tensor([[1], [2], [3]]))

# define model, loss_function, optimizser
model = torch.nn.Sequential(torch.nn.Linear(1, 1, bias=True))
loss_fn = torch.nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-2)

# trainning
for i in range(1000) :
    # network model
    y_pred = model(x)
    loss = loss_fn(y_pred, y)

    # zero_grad, backward, step(update parameter) in series
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

# training result
print(list(model.parameters()))


I am working on the file "02_Medidas_Localizacion.ipynb".
The first task is:
Ahora veamos que ocurre con la media:
Can you write Python code for it?


I am working on the file "02_Medidas_Localizacion.ipynb".
The first task is:
Caso con menos datos
En un caso con menor cantida dde datos se espera tener una mayor diferencia entre ambas medidas, de ahí si inestabilidad:
Can you write Python code for it?

Serie = np.random.uniform(2.5,10,2e5)
print Serie.mean()
print np.median(Serie)
from scipy import stats as st
print st.skew(Serie)


I am working on the file "02_Medidas_Localizacion.ipynb".
The first task is:
Caso de Introducción de Outliers
QQ plot de las series donde se introducen outliers y la serie en donde no
Can you write Python code for it?

S1 = np.random.normal(0,1,100)
S2 = CreaOutliers(S1,10)
Per1 = np.array([np.percentile(S1,i) for i in range(10,91,10)])
Per2 = np.array([np.percentile(S2,i) for i in range(10,91,10)])

fig = pl.figure(figsize=(9,7))
ax = fig.add_subplot(111)
ax.scatter(Per1,Per2,s=40)
ax.set_xlim(-2,2)
ax.set_ylim(-2,2)
ax.grid(True)
ax.set_xlabel('Deciles Observados',size =14); ax.set_ylabel('Deciles Alterados',size=14)
ax.plot([-2,2],[-2,2],lw=0.5,c='k')
pl.show()


I am working on the file "02_Medidas_Localizacion.ipynb".
The first task is:
Cuantiles
Como una medida no paramétrica de la distribución de los datos se encuentran los cuantiles,
el más conocido es la mediana, sin embargo se pueden obtener cuantiles de cualquier medida.
Que representan : El cuantil del 25% igual a 3.56, indica que el 25% de los datos son iguales o inferiores a 3.56.
Al ser una medida no paramétrica se ve poco afectada por errores en los datos y por datos atípicos.
Can you write Python code for it?

S1 = np.random.normal(0,1,100)
a=pl.boxplot(S1)
a=pl.xlabel('Serie')


I am working on the file "02_Medidas_Localizacion.ipynb".
The first task is:
Ejercicio para observar robustez en ambas medidas
En el siguiente ejercicio generamos 200 veces series aleatorias cada una con 25 entradas,
luego vamos a comprar como son las diferencias entre las medias y las medianas encontradas para cada uno de los casos.
Can you write Python code for it?

medianas = np.zeros(20000)
medias=np.zeros(20000)
for i in range(20000):
    Serie = np.random.normal(0,1,25)
    medias[i] = Serie.mean()
    medianas[i]=np.median(Serie)

def ComparaHistogramas(Vec1,Vec2,bins=15):
    # Genera el histograma de valores
    h1,b1 = np.histogram(Vec1,bins=bins)
    h1 = h1.astype(float); h1 = h1 / h1.sum()
    b1 = (b1[1:]+b1[:-1])/2.0
    h2,b2 = np.histogram(Vec2,bins=bins)
    h2 = h2.astype(float); h2 = h2 / h2.sum()
    b2 = (b2[1:]+b2[:-1])/2.0
    #Genera la figura
    fig=pl.figure(figsize=(10,8))
    ax=fig.add_subplot(111)
    ax.plot(b1,h1,'b',lw=2,label='Vec 1')
    ax.plot(b2,h2,'r',lw=2,label='Vec 2')
    ax.fill_between(b1,h1,color='b',alpha=0.2)
    ax.fill_between(b2,h2,color='r',alpha=0.2)
    ax.set_xlabel('$X$',size=15)
    ax.set_ylabel('$f(x)$',size=15)
    ax.set_xlim(-1,1)
    ax.set_ylim(0,h1.max()+0.05)
    ax.grid(True)
    ax.legend(loc=0)
    # Grafica las localizaciones
    pl.show()
    return h1,h2

HistMedianas, HistMedias = ComparaHistogramas(medianas,medias)


I am working on the file "02_Medidas_Localizacion.ipynb".
The first task is:
Grafica de las medidas de localización paramétricas
Can you write Python code for it?


I am working on the file "02_Medidas_Localizacion.ipynb".
The first task is:
Gráfica de medidas no paramétricas
Can you write Python code for it?


I am working on the file "02_Medidas_Localizacion.ipynb".
The first task is:
Introducción de múltiples Outliers
Que pasa si se introduce una alta cantidad de datos atípicos?, es decir como es la tasa a
la cual la media puede ir pasando a ser cada ves un estimador con un mayor error?.
Can you write Python code for it?

def CreaOutliers(vect,NumOut,Mult=10):
    # Encuentra el rango de oscilacion
    Per = np.array([np.percentile(vect,i) for i in [0.1,99.9]])
    # Genera los aleatorios
    vectOut = np.copy(vect)
    for i in np.random.choice(vect.shape[0],NumOut):
        p = np.random.choice(2,1)[0]
        vectOut[i] = vectOut[i] + Per[p]*Mult*np.random.uniform(2,15,1)[0]
    return vectOut

print Serie3.mean()
print Serie.mean()
print '----------'
print np.median(Serie3)
print np.median(Serie)

# Definición de variables
N = 1000
S1 = np.random.normal(0,1,N)
Medias = []; Std = []
Medianas = []; R25_75 = []
# Introduccion de outliers
for i in np.arange(5,200):
    S2 = CreaOutliers(S1, i)
    Medias.append(S2.mean())
    Medianas.append(np.median(S2))
    Std.append(S2.std())
    R25_75.append(np.percentile(S2,75)-np.percentile(S2,25))
Medias = np.array(Medias)
Medianas = np.array(Medianas)


I am working on the file "02_Medidas_Localizacion.ipynb".
The first task is:
Resultados:
Según lo obtenido la mediana se ve altamente afectada, y la desviación también:
Caso de una distribución Normal
Can you write Python code for it?

# Definición de variables
N = 1000
S1 = np.random.uniform(0,1,N)
Medias = []; Std = []
Medianas = []; R25_75 = []
# Introduccion de outliers
for i in np.arange(5,200):
    S2 = CreaOutliers(S1, i)
    Medias.append(S2.mean())
    Medianas.append(np.median(S2))
    Std.append(S2.std())
    R25_75.append(np.percentile(S2,75)-np.percentile(S2,25))
Medias = np.array(Medias)
Medianas = np.array(Medianas)

fig = pl.figure(figsize=(13,5))
ax = fig.add_subplot(121)
ax.scatter(Medianas,Medias,c=np.arange(5,200))
ax.set_xlabel('Mediana',size=14)
ax.set_ylabel('Media $\mu$',size=14)
ax = fig.add_subplot(122)
ax.scatter(R25_75,Std,c=np.arange(5,200))
#ax.set_xlim(0,1)
ax.set_xlabel('Rango $25%$ - $75\%$',size=14)
ax.set_ylabel('Desviacion $\sigma$',size=14)
pl.show()


I am working on the file "02_Medidas_Localizacion.ipynb".
The first task is:
Suceptibilidad a Datos Atípicos
Un dato atípico se define como aquel dato que se encuentra fuera del rango de oscilación de los datos, o bien que no es coherente con la física del fenómeno que se está sensando, los siguientes son ejemplos de datos atípicos:

Valores exageradamente altos.
Valores negativos en casos de fenómenos sin valores negativos.
Valores fuera de un rango definido.
Secuencia de valores con el mismo valor (no es tanto atípico, pero si es un indicio de problemas)

Una forma de identificarlos es a partir de la media de los valores y la desviación, o los percentiles sobre los que se ubiquen.

$ValAtipico > \mu + N \sigma$, donde $N$ oscila de acuerdo a lo fuerte que se quiera hacer la pregunta
$ValAtipico > P_{99.9}$

Dependiendo de la cantidad de registros en los datos, de la cantidad de valores atípicos y de los valores que estos tengan pueden tener o no consecuencias sobre la serie y sobre posteriores análisis que se realicen sobre la misma.
Ejemplo de robustez ante datos atípicos
Can you write Python code for it?

Serie = np.random.normal(0,1,50)
fig = pl.figure(figsize=(9,7))
pl.plot(Serie)
pl.grid(True)


I am working on the file "02_Medidas_Localizacion.ipynb".
The first task is:
Y que ocurre con la mediana:
Can you write Python code for it?

print np.median(Serie)
print np.median(Serie2)


I am working on the file "02_Mesh_generation/1_Cartesian_Finite_Difference_grid.ipynb".
The first task is:
$1.4x10^{13}$ TB would lead to some serious memory problems when using such a fine mesh, because the NEC-cluster at the computing centre at Christian-Albrechts-University Kiel only has a total of 40 TB of RAM. We have not even considered the required runtime for such a seismic modelling run.
Fortunately, we do not have to discretize the universe down to the atomic or moleculare scale in order to get reasonable modelling results. In a later lecture, we will derive criteria to find the best compromise between accuracy of the modelling results and level of model discretization.
Discretization of a 2D model
Let's start with the 2D acoustic wave equation:
\begin{equation}
\frac{\partial^2 P(x,z,t)}{\partial t^2} - Vp^2(x,z) \biggl(\frac{\partial^2 P(x,z,t)}{\partial x^2} + \frac{\partial^2 P(x,z,t)}{\partial z^2} \biggl)= f \nonumber
\end{equation}
In a first step to solve this problem using finite-differences we have to discretize the pressure wavefield $P(x,z,t)$ and $V_{p0}(x,z)$ at a specific time t on a equidistant 2D Cartesian spatial grid:
<img src="images/2D-grid_cart_ac.png" width="75%">
where we define the positions (x,z) of the discrete points in the model by
\begin{align}
x &= idh,\nonumber \
z &= jdh,\nonumber \
\end{align}
where $dh$ is the spatial grid point distance. We denote the value of the pressure wavefield and P-wave velocity at a discrete point by $P_{j,i}$ and $Vp_{j,i}$, respectively. These are 2D matrices, so we can simply store them in 2D arrays. Let's try to do this in Python.
Can you write Python code for it?

# Import Libraries
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

# Here, I introduce a new library, which is useful
# to define the fonts and size of a figure in a notebook
from pylab import rcParams

# Get rid of a Matplotlib deprecation warning
import warnings
warnings.filterwarnings("ignore")


I am working on the file "02_Mesh_generation/1_Cartesian_Finite_Difference_grid.ipynb".
The first task is:
... or an even more complex layer interface:
Can you write Python code for it?

# Define 2-layer P-wave velocity model with sine-shape layer interface
vp1 = 2.0                   # Vp in half-space [m/s]
depth_layer = 250.0         # average depth of sine layer interface
amp = 100.0                 # amplitude of sine layer undulation
wave = 2.0 * np.pi / 100.0  # angular wavenumber of sine undulation

# define function to create model with undulating sine-shaped interface
def create_sin_model(NXtmp,NZtmp,DH):

    # define homogeneous medium
    model = vp0*np.ones((NZtmp+1,NXtmp+1))

    # Add a layer interface undulating with a sine function to model
    # loop over Cartesian grid
    for j in range(NZtmp+1):
        for i in range(NXtmp+1):

            depth = j*DH
            x = i*DH

            # estimate depth of sine layer interface @ x
            depth_sine = depth_layer + amp * np.sin(wave*x)

            if(depth>depth_sine):
                model[j,i] = vp1

    return model

# Create Vp-model
vp_sine = create_sin_model(NX,NZ,dh)

# Plot Vp-model
cax = plt.imshow(vp_sine,extent=[x[0],x[-1],z[0],z[-1]], origin='lower')
plt.title("Sine layer P-wave velocity model" )
plt.xlabel("x [m]")
plt.ylabel("z [m]")
cbar = plt.colorbar(cax, orientation='vertical', pad=0.02)
cbar.set_label('Vp [m/s]', labelpad=10)
plt.gca().invert_yaxis()
plt.show()


I am working on the file "02_Mesh_generation/1_Cartesian_Finite_Difference_grid.ipynb".
The first task is:
Equidistant Cartesian grid
In lecture 1 we reviewed the forward problems to describe wave propagation in isotropic elastic and acoustic media. Furthermore, analytical solutions for homogeneous 3D, 2D and 1D acoustic media are derived to verify the accuarcy of the finite difference codes we develop later in the course.
Before introducing the basics of finite-difference modelling, I want to give an overview of some concepts to generate discrete subsurface models.
From the continous to discrete world
The partial differential equations, which we developed before describe wave propagation in a continous medium. However, if we want to solve these equations in a computer, we have to discretize the continuum, due to the limitation that a computer can only compute discrete problems. The question is how fine do we have to discretize the continuum in order to get accurately seismic modelling results?
Do we have to discretize down to the molecular scale? For example 12 grams of the carbon isotop C-12 contains $6.022140857x10^{23}$ atoms. Let's assume that we can represent the coordinates of the atoms in each direction by a double precision floating-point number which requires 8 Byte, so in totally we need ...
Can you write Python code for it?

memory_mole_c12 = 6.022140857e23 * 3 * 8 * 1e-12
print ("Memory required = ", memory_mole_c12, " TB")


I am working on the file "02_Mesh_generation/1_Cartesian_Finite_Difference_grid.ipynb".
The first task is:
Nice, next we create and visualize a homogeneous Vp-model discretized on this Cartesian grid.
Can you write Python code for it?

# Define P-wave velocity for homogeneous medium
vp0 = 1.0 # Vp [m/s]
vp = vp0*np.ones((NZ+1,NX+1))

# Plot Vp-model
cax = plt.imshow(vp,extent=[x[0],x[-1],z[0],z[-1]], origin='lower')
plt.title("Homogeneous P-wave velocity model" )
plt.xlabel("x [m]")
plt.ylabel("z [m]")
cbar = plt.colorbar(cax, orientation='vertical', pad=0.02)
cbar.set_label('Vp [m/s]', labelpad=10)
plt.gca().invert_yaxis()
plt.show()


I am working on the file "02_Mesh_generation/1_Cartesian_Finite_Difference_grid.ipynb".
The first task is:
We want to discretize a homogeneous subsurface model with x = 1000 m and z = 500 m and a spatial grid point distance dh = 50 m.
Can you write Python code for it?

# Define FD grid parameters
dh = 50.0
Xmax = 1000.0
Zmax = 500.0

# Define no. of grid points in x- and z-direction and grid point coordinates
def coord_def(Xmax,Zmax,dh):

    # number of grid points in x- and z-direction
    NX = (int)(Xmax / dh)
    NZ = (int)(Zmax / dh)

    print("NX = ",NX)
    print("NZ = ",NZ)

    # x- and z-coordinates of the discrete Cartesian grid points
    x = np.arange(0, (NX+1)*dh, dh)
    z = np.arange(0, (NZ+1)*dh, dh)

    XX, ZZ = np.meshgrid(x, z)

    return NX, NZ, x, z, XX, ZZ

# Define figure size
rcParams['figure.figsize'] = 12, 5

# Calculate coordinates of grid points
NX, NZ, x, z, XX, ZZ = coord_def(Xmax,Zmax,dh)

# Plot Cartesian mesh
plt.plot(XX, ZZ, 'k')
plt.plot(XX.T, ZZ.T, 'k')

# Plot grid points
plt.plot(XX, ZZ, 'bo')

plt.title("Equidistant 2D Cartesian grid" )
plt.xlabel("x [m]")
plt.ylabel("z [m]")
plt.axes().set_aspect('equal')
plt.gca().invert_yaxis()
plt.show()


I am working on the file "02_Mesh_generation/4_Tri_mesh_delaunay_yigma_tepe.ipynb".
The first task is:
Finally, we want a more evenly distribution of the triangle sizes. This can be achieved by imposing a maximum area to the triangles with the option a20.
Can you write Python code for it?

cncfq20adt = triangulate(A,'pq20a20D')
ax = plt.subplot(111, aspect='equal')
tplot.plot(ax,**cncfq20adt)


I am working on the file "02_Mesh_generation/4_Tri_mesh_delaunay_yigma_tepe.ipynb".
The first task is:
Good, now we have defined the model boundary points. Time for some constrained Delaunay triangulation ...
Can you write Python code for it?

# define vertices (no redundant points)
vert = model_bound

# apply Delaunay triangulation to vertices
tri = triangle.delaunay(vert)

# define vertex markers
vertm = np.array(np.zeros((len(vert),1)),dtype='int32')

# define how the vertices are connected, e.g. point 0 is connected to point 1,
# point 1 to point 2 and so on ...
points1 = np.arange(len(vert))
points2 = np.arange(len(vert))+1

# last point is connected to the first point
points2[-1] = 0

# define connectivity of boundary polygon
seg = np.array(np.vstack([points1,points2]).T,dtype='int32')

# define marker for boundary polygon
segm = np.array(np.ones((len(seg),1)),dtype='int32')

# assemble dictionary for triangle optimisation
A = dict(vertices=vert, vertex_markers=vertm, segments=seg, segment_markers=segm,triangles=tri)

# Optimise initial triangulation
cndt = triangle.triangulate(A,'pD')
ax = plt.subplot(111, aspect='equal')
tplot.plot(ax,**cndt)


I am working on the file "02_Mesh_generation/4_Tri_mesh_delaunay_yigma_tepe.ipynb".
The first task is:
In order to use the constrained Delaunay triangulation, we obviously have to define the constraining vertex points lying on the boundaries of our model. In this case it is quite easy, because the TFI mesh is regular.
OK, perhaps not so easy, because we have to be sure that no redundant points are in the final list and the points are sequentially defined in clockwise direction.
Can you write Python code for it?

# Estimate boundary points

# surface topography
surf = np.vstack([X[9,:-2],Z[9,:-2]]).T

# right model boundary
right = np.vstack([X[1:,69],Z[1:,69]]).T

# bottom model boundary
bottom = np.vstack([X[0,1:],Z[0,1:]]).T

# left model boundary
left = np.vstack([X[:-2,0],Z[:-2,0]]).T

# assemble model boundary
model_stack = np.vstack([surf,np.flipud(right)])
model_stack1 = np.vstack([model_stack,np.flipud(bottom)])
model_bound = np.vstack([model_stack1,left])


I am working on the file "02_Mesh_generation/4_Tri_mesh_delaunay_yigma_tepe.ipynb".
The first task is:
Next, we compute the Voronoi diagram for the mesh points. This describes the partitioning of a plane with n points into convex polygons such that each polygon contains exactly one generating point and every point in a given polygon is closer to its generating point than to any other.
Can you write Python code for it?

# calculate and plot Voronoi diagram for mesh points
from scipy.spatial import Voronoi, voronoi_plot_2d

vor = Voronoi(points)

plt.figure(figsize=(12,6))
ax = plt.subplot(111, aspect='equal')
voronoi_plot_2d(vor, ax=ax)

plt.title("Part of Yigma Tepe (Voronoi diagram)" )
plt.xlabel("x [m]")
plt.ylabel("z [m]")
plt.xlim( 25,  75)
plt.ylim(10, 35)
plt.show()


I am working on the file "02_Mesh_generation/4_Tri_mesh_delaunay_yigma_tepe.ipynb".
The first task is:
The above code looks a little bit chaotic, but you can check that the points in the resulting array model_bound are correctly sorted and contains no redundant points.
Can you write Python code for it?

plt.plot(model_bound[:,0],model_bound[:,1],'bo')
plt.title("Yigma Tepe model boundary" )
plt.xlabel("x [m]")
plt.ylabel("z [m]")
plt.axes().set_aspect('equal')
plt.show()


I am working on the file "02_Mesh_generation/4_Tri_mesh_delaunay_yigma_tepe.ipynb".
The first task is:
The regular triangulation within the tumulus looks reasonable. However, the Delaunay triangulation also added unwanted triangles above the topography. To solve this problem we have to use constrained Delaunay triangulation in order to restrict the triangulation to the model below the free-surface topography. Unfortunately, constrained Delaunay triangulation is not available within SciPy.
However, there exists a python wrapper to the mesh generator Triangle by Richard Shewchuck. This wrapper can be installed by

git clone --depth=1 https://github.com/drufat/triangle.git
cd triangle
python setup.py install

A detailed documentation of the Triangle Python wrapper by Dzhelil Rufat can be found here.
Can you write Python code for it?

# import triangulate library
from triangle import triangulate, show_data, plot as tplot
import triangle


I am working on the file "02_Mesh_generation/4_Tri_mesh_delaunay_yigma_tepe.ipynb".
The first task is:
This quad mesh is already able to accurately describe the free-surface topography.
Triangular mesh generation
If we need a triangular mesh, for example for finite element or finite volume modelling,  we could apply Delaunay triangulation to the node point distribution of the Yigma Tepe TFI mesh. For further details related to Delaunay triangulation, I refer to Computational Geometry in Python by Francisco Blanco-Silva.
In a first step, we assemble the x- and z-vectors in a (NX*NZ x 2) matrix.
Can you write Python code for it?

# Reshape X and Z vector
x = X.flatten()
z = Z.flatten()

# Assemble x and z vector into NX*NZ x 2 matrix
points = np.vstack([x,z]).T


I am working on the file "02_Mesh_generation/4_Tri_mesh_delaunay_yigma_tepe.ipynb".
The first task is:
Triangular mesh generation
In the last lesson we learned how to create a quad mesh by Transfinite Interpolation to accurately approximate the strong topography of a sea dike. We can use this mesh for spectral element modelling. But what should we do, if we need a triangular mesh for example for Finite Element modelling?
Yigma Tepe - another problem with strong topography
You might think that the sea dike topography was already quite complex. Well, here is another problem we are currently working on at the "Applied Geophysics" group at Christian-Albrechts University Kiel:
<img src="images/yigmatepe_1.jpg" width="100%">
This is Yigma Tepe, a tumulus which might be a burial of an Attilad King located in Pergamon (Turkey). Extensive geophysical prospecting and archaeological excavations revealed small scale near-surface structures, which makes Yigma Tepe an ideal target to further develop 2D SH and 3D seismic full waveform inversion in the field of archaeological prospection. A critical part is the correct discretization of the complex free-surface topography, crucial for accurate modelling of surface waves.
Let's take a look at a 2D quad mesh created by Transfinite-Interpolation for a SH-profile along the tumulus.
Can you write Python code for it?

# Import Libraries
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

# Here, I introduce a new library, which is useful
# to define the fonts and size of a figure in a notebook
from pylab import rcParams

# Get rid of a Matplotlib deprecation warning
import warnings
warnings.filterwarnings("ignore")

# Load Yigma Tepe quad mesh created by TFI
X = np.loadtxt('data/yigma_tepe_TFI_mesh_X.dat', delimiter=' ', skiprows=0, unpack='True')
Z = np.loadtxt('data/yigma_tepe_TFI_mesh_Z.dat', delimiter=' ', skiprows=0, unpack='True')

# number of grid points in each spatial direction
NZ, NX = np.shape(X)
print("NX = ", NX)
print("NZ = ", NZ)

# Define figure size
rcParams['figure.figsize'] = 10, 7

# Plot Yigma Tepe TFI mesh
plt.plot(X, Z, 'k')
plt.plot(X.T, Z.T, 'k')
plt.plot(X, Z, 'bo', markersize=4)

plt.title("Yigma Tepe TFI mesh" )
plt.xlabel("x [m]")
plt.ylabel("z [m]")
plt.axes().set_aspect('equal')
#plt.savefig('yigma_tepe_TFI.pdf', bbox_inches='tight', format='pdf')
plt.show()


I am working on the file "02_Mesh_generation/4_Tri_mesh_delaunay_yigma_tepe.ipynb".
The first task is:
Very good, compared to the SciPy Delaunay triangulation, no triangles are added above the topography. However, most triangles have very small minimum angles, which would lead to serious numerical issues in later finite element modelling runs. So in the next step we restrict the minimum angle to 20° using the option q20.
Can you write Python code for it?

cncfq20dt = triangulate(A,'pq20D')
ax = plt.subplot(111, aspect='equal')
tplot.plot(ax,**cncfq20dt)


I am working on the file "02_Mesh_generation/lecture_notebooks/1_Cartesian_Finite_Difference_grid.ipynb".
The first task is:
We want to discretize a homogeneous subsurface model with x = 1000 m and z = 500 m and a spatial grid point distance dh = 50 m.
Can you write Python code for it?

# Define FD grid parameters

# Define no. of grid points in x- and z-direction and grid point coordinates


I am working on the file "02_NB_IntroductionNumpy.ipynb".
The first task is:
2. Create a structured array representing a position (x,y) and a color (r,g,b)
Can you write Python code for it?


I am working on the file "02_NB_IntroductionNumpy.ipynb".
The first task is:
2. Vectorization
Can you write Python code for it?

#Apply function to a complete array instead of writing loop to iterate over all elements of the array.
#This is called vectorization. The opposite of vectorization (for loops) is known as the scalar implementation

def f(x):
    return x*np.exp(4)

print(f(a))


I am working on the file "02_NB_IntroductionNumpy.ipynb".
The first task is:
3. Slicing and reshape
Array slicing
x[i:j:s]

picks out the elements starting with index i and stepping s indices at the time up to, but not including, j.
Can you write Python code for it?

x = np.array(range(100))

x[1:-1] # picks out all elements except the first and the last, but contrary to lists, a[1:-1] is not a copy of the data in a.
x[0:-1:2] # picks out every two elements up to, but not including, the last element, while
x[::4] # picks out every four elements in the whole array.


I am working on the file "02_NB_IntroductionNumpy.ipynb".
The first task is:
Array shape manipulation
Can you write Python code for it?

a = np.linspace(-1, 1, 6)
print (a)

a.shape
a.size

# rows, columns
a.shape = (2, 3)
a = a.reshape(2, 3) # alternative

a.shape
print (a)

# len(a) always returns the length of the first dimension of an array. -> no. of rows


I am working on the file "02_NB_IntroductionNumpy.ipynb".
The first task is:
Case 1: a = a+b
The sum is first computed and resulting in a new array and the a is bound to the new array
Can you write Python code for it?


I am working on the file "02_NB_IntroductionNumpy.ipynb".
The first task is:
Case 2: a += b
The elements of b are directly added into the elements of a (in memory) - no intermediate array. These operators implement the so-called "in-place arithmetics" (e.g., +=, *=, /=, -= )
Can you write Python code for it?


I am working on the file "02_NB_IntroductionNumpy.ipynb".
The first task is:
Exercise
1. Create a 10x10 2d array with 1 on the border and 0 inside
Can you write Python code for it?


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
1.5 A Simple Concrete Pyomo Model
It is possible to get nearly the same flexible behavior from models declared to be abstract and models declared to be concrete in Pyomo; however, we will focus on a straightforward concrete example here where the data is hard-wired into the model file. Python programmers will quickly realize that the data could have come from other sources.
We repeat the concrete model already given:
$$\min \quad 2x_1 + 3x_2$$
$$s.t. \quad 3x_1 + 4x_2 \geq 1$$
$$x_1,x_2 \geq 0$$
This is implemented as a concrete model as follows:
Can you write Python code for it?

from pyomo.environ import *

model = ConcreteModel()

model.x = Var([1,2], domain=NonNegativeReals)

model.OBJ = Objective(expr = 2*model.x[1] + 3*model.x[2])

model.Constraint1 = Constraint(expr = 3*model.x[1] + 4*model.x[2] >= 1)


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
Although not required, it is convenient to define index sets. In this example we use the RangeSet function to declare that the sets will be a sequence of integers starting at 1 and ending at a value specified by the the parameters model.m and model.n.
Can you write Python code for it?

model.I = RangeSet(1, model.m)
model.J = RangeSet(1, model.n)


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
Declaration of constraints is similar. A function is declared to deliver the constraint expression. In this case, there can be multiple constraints of the same form because we index the constraints by $i$ in the expression $ \sum_{j=1}^n a_ij x_j \geq b_i \forall i = 1 \ldots m$, which states that we need a constraint for each value of $i$ from one to $m$. In order to parametrize the expression by $i$ we include it as a formal parameter to the function that declares the constraint expression. Technically, we could have used anything for this argument, but that might be confusing. Using an i for an $i$ seems sensible in this situation.
Can you write Python code for it?

def ax_constraint_rule(model, i):
    """return the expression for the constraint for i"""
    return sum(model.a[i,j] * model.x[j] for j in model.J) >= model.b[i]


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
In the object oriented view of all of this, we would say that model object is a class instance of the AbstractModel class, and model.J is a Set object that is contained by this model. Many modeling components in Pyomo can be optionally specified as indexed components: collections of components that are referenced using one or more values. In this example, the parameter model.c is indexed with set model.J.
In order to use this model, data must be given for the values of the parameters. Here is one file that provides data.
Can you write Python code for it?


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
Note:
In Python, and therefore in Pyomo, any text after pound sign is considered to be a comment.

The next line interpreted by Python as part of the model declares the variable $x$. The first argument to the Var function is a set, so it is defined as an index set for the variable. In this case the variable has only one index set, but multiple sets could be used as was the case for the declaration of the parameter model.a. The second argument specifies a domain for the variable. This information is part of the model and will passed to the solver when data is provided and the model is solved. Specification of the NonNegativeReals domain implements the requirement that the variables be greater than or equal to zero.
Can you write Python code for it?

# the next line declares a variable indexed by the set J
model.x = Var(model.J, domain=NonNegativeReals)


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
Note:
In Python, indexes are in square brackets and function arguments are in parentheses.

In order to declare constraints that use this expression, we use the Pyomo Constraint function that takes a variety of arguments. In this case, our model specifies that we can have more than one constraint of the same form and we have created a set, model.I, over which these constraints can be indexed so that is the first argument to the constraint declaration function. The next argument gives the rule that will be used to generate expressions for the constraints. Taken as a whole, this constraint declaration says that a list of constraints indexed by the set model.I will be created and for each member of model.I, the function ax_constraint_rule will be called and it will be passed the model object as well as the member of model.I.
Can you write Python code for it?

# the next line creates one constraint for each member of the set model.I
model.AxbConstraint = Constraint(model.I, rule=ax_constraint_rule)


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
Since glpk is the default solver, there really is no need specify it so the --solver option can be dropped.

Note:
There are two dashes before the command line option names such as solver.

To continue the example, if CPLEX is installed then it can be listed as the solver. The command to solve with CPLEX is
Can you write Python code for it?

!pyomo solve abstract1.py abstract1.dat --solver=cplex


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
The coefficient and right-hand-side data are defined as indexed parameters. When sets are given as arguments to the Param function, they indicate that the set will index the parameter.
Can you write Python code for it?

model.a = Param(model.I, model.J)
model.b = Param(model.I)
model.c = Param(model.J)


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
There are multiple formats that can be used to provide data to a Pyomo model, but the AMPL format works well for our purposes because it contains the names of the data elements together with the data. In AMPL data files, text after a pound sign is treated as a comment. Lines generally do not matter, but statements must be terminated with a semi-colon.
For this particular data file, there is one constraint, so the value of model.m will be one and there are two variables (i.e., the vector model.x is two elements long) so the value of model.n will be two. These two assignments are accomplished with standard assignments. Notice that in AMPL format input, the name of the model is omitted.
Can you write Python code for it?


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
There is only one constraint, so only two values are needed for model.a. When assigning values to arrays and vectors in AMPL format, one way to do it is to give the index(es) and the the value. The line 1 2 4 causes model.a[1,2] to get the value 4. Since model.c has only one index, only one index value is needed so, for example, the line 1 2 causes model.c[1] to get the value 2. Line breaks generally do not matter in AMPL format data files, so the assignment of the value for the single index of model.b is given on one line since that is easy to read.
Can you write Python code for it?


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
This yields the following output on the screen:
The numbers is square brackets indicate how much time was required for each step. Results are written to the file named results.json, which has a special structure that makes it useful for post-processing. To see a summary of results written to the screen, use the --summary option:
Can you write Python code for it?

!pyomo solve abstract1.py abstract1.dat --solver=cplex --summary


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
To declare an objective function, the Pyomo function called Objective is used. The rule argument gives the name of a function that returns the expression to be used. The default sense is minimization. For maximization, the sense=maximize argument must be used. The name that is declared, which is OBJ in this case, appears in some reports and can be almost any name.
Can you write Python code for it?


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
We declare the parameters $m$ and $n$ using the Pyomo Param function. This function can take a variety of arguments; this example illustrates use of the within option that is used by Pyomo to validate the data value that is assigned to the parameter. If this option were not given, then Pyomo would not object to any type of data being assigned to these parameters. As it is, assignment of a value that is not a non-negative integer will result in an error.
Can you write Python code for it?

model.m = Param(within=NonNegativeIntegers)
model.n = Param(within=NonNegativeIntegers)


I am working on the file "02_PyomoOverview.ipynb".
The first task is:
When working with Pyomo (or any other AML), it is convenient to write abstract models in a somewhat more abstract way by using index sets that contain strings rather than index sets that are implied by $1,...,m$ or the summation from 1 to $n$. When this is done, the size of the set is implied by the input, rather than specified directly. Furthermore, the index entries may have no real order. Often, a mixture of integers and indexes and strings as indexes is needed in the same model. To start with an illustration of general indexes, consider a slightly different Pyomo implementation of the model we just presented.
Can you write Python code for it?


I am working on the file "02_Python/2.ipynb".
The first task is:
Charger un module
Le module numpy, installé par défaut avec Anaconda, est une boîte d'outil de calcul numérique populée par de nombreuses foncions mathématiques. Dont la racine carrée.
Can you write Python code for it?

import numpy as np
np.sqrt(9)

from numpy import sqrt
sqrt(9)


I am working on the file "02_Python/2.ipynb".
The first task is:
D'abord, Python ne retourne pas de message d'erreur, mais un nouveau type de donnée: le nombre imaginaire. Puis, 6.123233995736766e-17 n'est pas zéro, mais très proche. La résolution des calculs étant numérique, on obeserve parfois de légères déviations par rapport aux solutions mathématiques.
Si pour un cas particulier, on veut éviter que notre fonction retourne un nombre imaginaire, comment s'y prendre? Avec une condition.
Conditions: if, elif, else

Si la condition 1 est remplie, effectuer une suite d'instruction 1. Si la condition 1 n'est pas remplie, et si la condition 2 est remplie, effectuer la suite d'instruction 2. Sinon, effectuer la suite d'instruction 3.

Voilà comment on exprime une suite de conditions. Pour notre racine d'un nombre négatif, on pourrait procéder comme suit.
Can you write Python code for it?

def racine_positive_nn(x, n=2):
    if x<0:
        raise ValueError("x est négatif")
    elif x==0:
        raise ValueError("x est nul")
    else:
        r = x**(1/n)
        return r


I am working on the file "02_Python/2.ipynb".
The first task is:
En Python, def est le mot-clé pour définir une fonction. Suit ensuite, après un espace, le nom que vous désirez donner à la fonction: racine. Les arguments de la fonction suivent entre les parenthèses. Dans ce cas, x est la valeur de laquelle on veut extraire la racine et n est l'ordre de la racine. L'agument x n'a pas de valeur par défaut: elle doit être spécifiée pour que la fonction fonctionne. La mention n=2 signifie que si la valeur de n n'est pas spécifiée, elle prendra la valeur de 2 (la racine carrée). Pour marquer la fin de la définition et le début de la suite d'instructions, on utilise les deux points :, puis un retour de ligne. Une indentation (ou retrait) de quatre barres d'espacement signifie que l'on se trouve à l'intérieur de la suite d'instructions, où l'on calcule une valeur de r comme l'exposant de l'inverse de l'ordre de la racine. La dernière ligne indique ce que la fonction doit retourner.
Can you write Python code for it?

print(racine(9))
print(racine(x=9))
print(racine(8, 3))
print(racine(x=8, n=3))


I am working on the file "02_Python/2.ipynb".
The first task is:
Le message d'erreur nous dit que A n'est pas défini. Sa version minuscule, a, l'est pourtant. La raison est que Python considère la case dans la définition des objets. Utiliser la mauvaise case mène donc à des erreurs.
Le nom d'une variable doit toujours commencer par une lettre, et ne doit pas contenir de caractères réservés (espaces, +, *, .). Par convention, les objets qui commencent par une lettre majuscules sont utilisés pour définir des classes (modules), utiles pour le développement de logiciels, mais rarement utilisés dans le cadre d'un feuille de calcul scientifique.
Can you write Python code for it?

rendement_arbre = 50 # pomme/arbre
nombre_arbre = 300 # arbre
nombre_pomme = rendement_arbre * nombre_arbre
nombre_pomme


I am working on the file "02_Python/2.ipynb".
The first task is:
Les tuples, définis tuple par Python, différent des listes du fait que ses éléments ne peuvent pas être modifiés. Un tuple est délimité par des parenthèses ( ), et comme chez la liste,  ses éléments sont séparés par des virgules. Les tuples sont moins polyvalents que les listes. Vous les utiliserez probablement rarement, et surtout comme arguments dans certaines fonctions en calcul scientifique.
Can you write Python code for it?

espece = ('Petromyzon marinus', 'Lepisosteus osseus', 'Amia calva', 'Hiodon tergisus')
espece[2] = "Lepomis gibbosus"
espece


I am working on the file "02_Python/2.ipynb".
The first task is:
Message d'erreur: la commande sqrt n'est pas définie.

Quoi, Python n'est pas foutu de calculer une racine carrée?

Par défaut, non.

Mais!
De nombreuses extensions (les modules) permettent de combler ces manques. Nous aborderons ça un peu plus loin dans ce chapitre. Pour l'instant, exerçons-nous à créer notre propre fonction de racine carrée.
Can you write Python code for it?

def racine(x, n=2):
    r = x**(1/n)
    return r


I am working on the file "02_Python/2.ipynb".
The first task is:
Pour accéder aux éléments d'une liste, appelle la liste suivie de la position de l'objet désiré entre crochets. Fait important qui reviendra tout au long du cours: en Python, l'indice du premier élément est zéro.
Can you write Python code for it?

print(espece[0])
print(espece[2])
print(espece[:2])
print(espece[2:])


I am working on the file "02_Python/2.ipynb".
The first task is:
Première observation, si un seul argument est inclus, range retourne une séquence partant de zéro. Seconde observation, la séquence se termine en excluant l'argument. Ainsi, range(2,5) retourne la séquence [2, 3, 4]. En spécifiant la longueur de data comme argument, la séquence range(5) retourne la liste [0, 1, 2, 3, 4], soit les indices dont nous avons besoin pour itérer dans la liste.
Les boucles for vous permettront par exemple de générer en peu de temps 10, 100, 1000 graphiques (autant que vous voulez), chacun issu de simulations obtenues à partir de conidtions initiales différentes, et de les enregistrer dans un répertoire sur votre ordinateur. Un travail qui pourrait prendre des semaines sur Excel peut être effectué en Python en quelques secondes.
Un second outil est disponible pour les itérations: les boucles while. Elles effectue une opération tant qu'un critère n'est pas atteint. Elles sont utiles pour les opérations où l'on cherche une convergence. Je les couvre rapidement puisque'elles sont rarement utilisées dans les flux de travail courrants. En voici un petit exemple.
Can you write Python code for it?

x = 100
while (x > 1.1):
    x=racine(x)
    print(x)


I am working on the file "02_Python/2.ipynb".
The first task is:
Quatorze, c'est bien cela (comptez "L'ours polaire", en incluant l'espace). len, pour lenght (longueur), est une fonction (aussi appelée méthode) incluse par défaut dans l'environnement de travail de Python. La fonction est appelée en écrivant len(). Mais une fonction de quoi? Des arguments qui se trouvent entre les parenthèses. Dans ce cas, il y a un seul argument: c.
En calcul scientifique, il est courrant de lancer des requêtes sur si une résultat est vrai ou faux.
Can you write Python code for it?


I am working on the file "02_Python/2.ipynb".
The first task is:
S'ils ne sont pas spécifiés, Python comprend que les arguments sont entrés dans l'ordre défini dans la fonction. En entrant racine(9), Python comprend que le 9 est attribué à x et donne à n sa valeur par défaut, 2. Ce qui est équivalent à entrer racine(x=9). Les autres entrées sont aussi équivalentes, et extraient la racine cubique. S'il se peut qu'il y ait confusion entre les arguments nommés et ceux qui ne le sont pas, Python vous retournera un message d'erreur. Règle générale, il est préférable pour la lisibilité du code de nommer les arguments plutôt que de les spécifier dans l'ordre.
Supposons maintenant que vous avez une liste de données dont vous voulez extraire la racine.
Can you write Python code for it?


I am working on the file "02_Python/2.ipynb".
The first task is:
Si les données contenues dans une liste sont de même type, cette liste peut être considérée comme un vecteur. En créant une liste de vecteurs de dimensions cohérentes, on crée une matrice. Nous verrons plus tard que pour les vecteurs et les matrices, on utilisera un format offert par un module complémentaire. Pour l'instant, on pourrait définir une matrice comme suit.
Can you write Python code for it?


I am working on the file "02_Python/2.ipynb".
The first task is:
Techniquement, a pointe vers le nombre entier 3. Conséquemment, on peut effectuer des opérations sur a.
Can you write Python code for it?


I am working on the file "02_Python/2.ipynb".
The first task is:
Tout va bien pour l'instant. Remarquez que la dernière opération comporte des espaces entre les nombres et l'opérateur /. Dans ce cas (ce n'est pas toujours le cas), les espaces ne signifient rien - il est même suggéré de les placer pour éclaircir le code, ce qui est utile lorsque les équations sont complexes. Puis, après l'opération 1 / 2, j'ai placé le symbole # suivi d'une note. Le symbole # est interprété par Python comme un ordre de ne pas considérer ce qui le suit. Cela est très utile pour insérer à même le code des commentaires pertinents pour mieux comprendre les opérations.
Assigner des objets à des variables est fondamental en programmation. Par exemple.
Can you write Python code for it?


I am working on the file "02_Python/2.ipynb".
The first task is:
Types de données
Jusqu'à maintenant, nous n'avons utilisé que des nombres entiers (integer ou int) et des nombres réels (float ou float64). Python inclue d'autres types. La chaîne de caractère (string) est un ou plusieurs symboles. Elle est définie entre des double-guillemets " " ou des apostrophes ' '. Il n'existe pas de standard sur l'utilisation de l'un ou de l'autre, mais en règle générale, on utilise les apostrophe pour les experssions courtes, contenant un simple mot ou séquence de lettres, et les guillements pour les phrases. Une raison pour cela: les guillemets sont utiles pour insérer des apostrophes dans une chaîne de caractère.
Can you write Python code for it?

a = "L'ours"
b = "polaire"
a + " " +  b + " ressemble à un faux zèbre."


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
Create a variable
Can you write Python code for it?


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
Datatypes
In computer programming, a data type is a classification identifying one of various types that data
can have.
The most common data type we will see in this class are:

Integers (int): Integers are the classic cardinal numbers: ... -3, -2, -1, 0, 1, 2, 3, 4, ...

Floating Point (float): Floating Point are numbers with a decimal point: 1.2, 34.98, -67,23354435, ...

Booleans (bool): Booleans types can only have one of two values: True or False. In many languages 0 is considered False, and any other value is considered True.

Strings (str): Strings can be composed of one or more characters: ’a’, ’spam’, ’spam spam eggs and spam’. Usually quotes (’) are used to specify a string. For example ’12’ would refer to the string, not the integer.

Collections of Data Types

Scalar: A single value of any data type.

List: A collection of values. May be mixed data types. (1, 2.34, ’Spam’, True) including lists of lists: (1, (1,2,3), (3,4))

Array: A collection of values. Must be same data type. [1,2,3,4] or [1.2, 4.5, 2.6] or [True, False, False] or [’Spam’, ’Eggs’, ’Spam’]

Matrix: A multi-dimensional array: [[1,2], [3,4]] (an array of arrays).
Can you write Python code for it?

a = 1
b = 2.3
c = True
d = "Spam"

type(a), type(b), type(c), type(d)

a + b, type(a + b)

a + c, type(a + c)    # True = 1

a + d

str(a) + d


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
Fancy masking
Can you write Python code for it?

mask3 = np.where(x >= 8)
x[mask3]

# Set all values of x that match mask3 to 0

x[mask3] = 0
x

mask4 = np.where(x != 0)
mask4

#Add 10 to every value of x that matches mask4:

x[mask4] += 100
x


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
Functions
In computer science, a function (also called a procedure, method, subroutine, or routine) is a portion
of code within a larger program that performs a specific task and is relatively independent of the
remaining code. The big advantage of a function is that it breaks a program into smaller, easier
to understand pieces. It also makes debugging easier. A function can also be reused in another
program.
The basic idea of a function is that it will take various values, do something with them, and return a result. The variables in a function are local. That means that they do not affect anything outside the function.
Below is an example of a function that finds the perihelion (closest point to the Sun) for an orbit with a semi-major axis of a and an eccntricity of e
$ \mathrm{Perihelion\ Distance\ (a,ecc)} = a\ (1 - ecc)$
In the example the name of the function is find_peri (you can name functions what ever you want). The function find_peri takes two arguments semi_major and eccentricity, and returns the value of the equation to the main program. In the main program a variable named perihelion_distance is assigned the value returned by find_peri. Notice that in the main program the function find_peri is called using the arguments planet_table[a] and planet_table[ecc]. Since the variables in the function are local, you do not have name them semi_major and eccentricity in the main program.
Can you write Python code for it?

def find_peri(semi_major,eccentricity):

    result = semi_major * (1.0 - eccentricity)          # assign the variable result the value of the function
    return result                                       # return the value of the function to the main program

perihelion_distance = find_peri(planet_table['a'],planet_table['ecc'])

perihelion_distance

planet_table['perihelion'] = perihelion_distance

print(planet_table)


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
Help about a function:
Can you write Python code for it?


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
Indexing
Can you write Python code for it?

x[0]    # The 0th element of the array x

x[-1]    # The last element of the array x


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
NumPy math works over an entire array:
Can you write Python code for it?

y = x * 2
y

sin(x)     # need to Numpy's math functions

np.sin(x)


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
Our basic unit will be the NumPy array
Can you write Python code for it?

np.random.seed(42)                 # set the seed - everyone gets the same random numbers
x = np.random.randint(1,10,20)     # 20 random ints between 1 and 10
x


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
Reading in data - The AstroPy package
Can you write Python code for it?

from astropy.table import QTable

!cat Planets.csv

planet_table = QTable.read('Planets.csv', format='ascii.csv')

planet_table

print(planet_table)


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
Saving a table
Can you write Python code for it?

planet_table.write('NewPlanets.csv', format='ascii.csv')

!cat NewPlanets.csv


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
The results of one function can be used as the input to another function
Can you write Python code for it?

def earth_distance(peri):

    result = peri - 0.98251494
    return result

earth_distance(perihelion_distance)


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
There are lots of different methods that can be applied to a NumPy array
Can you write Python code for it?

x.size                   # Number of elements in x

x.mean()                 # Average of the elements in x

x.sum()                  # Total of the elements in x

x[-5:].sum()              # Total of last 5 elements in x

x.cumsum()                # Cumulative sum

x.cumsum()/x.sum()        # Cumulative percentage


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
UNIX commands can be run by placing a ! before the command:
Can you write Python code for it?


I am working on the file "02_Python_Introduction.ipynb".
The first task is:
or even easier:
Can you write Python code for it?


I am working on the file "02_Pythonic_Image.ipynb".
The first task is:
Amazingly make common trivial tasks really trivial
Can you write Python code for it?

myshow(img>90)

myshow(img>150)

myshow((img>90)+(img>150))


I am working on the file "02_Pythonic_Image.ipynb".
The first task is:
Comparative Operators

Operators

\>

\>=

<

<=

==

These comparative operators follow the same convention as the reset of SimpleITK for binary images. They have the pixel type of ``sitkUInt8`` with values of 0 and 1.
Can you write Python code for it?

img = sitk.ReadImage(fdata("cthead1.png"))
myshow(img)


I am working on the file "02_Pythonic_Image.ipynb".
The first task is:
Cropping
Can you write Python code for it?


I am working on the file "02_Pythonic_Image.ipynb".
The first task is:
Division Operators
All three Python division operators are implemented __floordiv__, __truediv__, and __div__.
The true division's output is a double pixle type.
See PEP 238 to see why Python changed the division operator in Python 3.
Bitwise Logic Operators

Operators

&

|

^

~

Can you write Python code for it?


I am working on the file "02_Pythonic_Image.ipynb".
The first task is:
Flipping
Can you write Python code for it?

img_corner = img[:32,:32]
myshow(img_corner)

myshow(img_corner[::-1,:])

myshow(sitk.Tile(img_corner, img_corner[::-1,::],img_corner[::,::-1],img_corner[::-1,::-1], [2,2]))


I am working on the file "02_Pythonic_Image.ipynb".
The first task is:
Let us begin by developing a convenient method for displaying images in our notebooks.
Can you write Python code for it?

img = sitk.GaussianSource(size=[64]*2)
plt.imshow(sitk.GetArrayFromImage(img))

img = sitk.GaborSource(size=[64]*2, frequency=.03)
plt.imshow(sitk.GetArrayFromImage(img))

def myshow(img):
    nda = sitk.GetArrayFromImage(img)
    plt.imshow(nda)
myshow(img)


I am working on the file "02_Pythonic_Image.ipynb".
The first task is:
Mathematical Operators
Most python mathematical operators are overloaded to call the SimpleITK filter which does that same operation on a per-pixel basis. They can operate on a two images or an image and a scalar.
If two images are used then both must have the same pixel type. The output image type is ussually the same.
As these operators basically call ITK filter, which just use raw C++ operators, care must be taked to prevent overflow, and divide by zero etc.

Operators

+

-

\*

/

//

**

Can you write Python code for it?

img = sitk.ReadImage(fdata("cthead1.png"))
img = sitk.Cast(img,sitk.sitkFloat32)
myshow(img)
img[150,150]

timg = img**2
myshow(timg)
timg[150,150]


I am working on the file "02_Pythonic_Image.ipynb".
The first task is:
Multi-dimension slice indexing
If you are familiar with numpy, sliced index then this should be cake for the SimpleITK image. The Python standard slice interface for 1-D object:

Operation	  Result

d[i]	   ith item of d, starting index 0

d[i:j]	 slice of d from i to j

d[i:j:k]	   slice of d from i to j with step k

With this convient syntax many basic tasks can be easily done.
Can you write Python code for it?


I am working on the file "02_Pythonic_Image.ipynb".
The first task is:
Slice Extraction
A 2D image can be extracted from a 3D one.
Can you write Python code for it?

img = sitk.GaborSource(size=[64]*3, frequency=0.05)

# Why does this produce an error?
myshow(img)

myshow(img[:,:,32])

myshow(img[16,:,:])


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Axial Loads
Can you write Python code for it?


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Define sympy parameters used for geometric description of sections
Can you write Python code for it?

A, A0, t, t0, a, b, h, L = sympy.symbols('A A_0 t t_0 a b h L', positive=True)


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Example 2: twisting moment in z direction
Can you write Python code for it?

S1.set_loads(_Tx=0, _Ty=0, _Nz=0, _Mx=0, _My=0, _Mz=Mz)
S1.compute_stringer_actions()
S1.compute_panel_fluxes();


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Expression of Inertial properties wrt Center of Gravity in with original rotation
Can you write Python code for it?


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Expression of inertial properties in principal reference frame
Can you write Python code for it?

sympy.simplify(S1.Ixx), sympy.simplify(S1.Iyy), sympy.simplify(S1.Ixy), sympy.simplify(S1.θ)


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Import Section class, which contains all calculations
Can you write Python code for it?


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Initialization of sympy symbolic tool and pint for dimension analysis (not really implemented rn as not directly compatible with sympy)
Can you write Python code for it?

ureg = UnitRegistry()
sympy.init_printing()


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Panel Fluxes
Can you write Python code for it?


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Panel Fluxes
evaluated to numerical values
Can you write Python code for it?

{k:sympy.N(S1.q[k].subs(datav)) for k in S1.q }


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Plot of S1 section in inertial reference Frame
Section is plotted wrt center of gravity and rotated (if necessary) so that x and y are principal axes.
Center of Gravity and Shear Center are drawn
Can you write Python code for it?

positions={ii: [float(S1.g.node[ii]['pos'][i].subs(datav)) for i in range(2)] for ii in S1.g.nodes() }

x_ct, y_ct = S1.ct.subs(datav)

plt.figure(figsize=(12,8),dpi=300)
nx.draw(S1.g,with_labels=True, pos=positions)
plt.plot([0],[0],'o',ms=12,label='CG')
plt.plot([x_ct],[y_ct],'^',ms=12, label='SC')
#plt.text(0,0, 'CG', fontsize=24)
#plt.text(x_ct,y_ct, 'SC', fontsize=24)
plt.legend(loc='lower right', shadow=True)
plt.axis('equal')
plt.title("Section in pricipal reference Frame",fontsize=16);


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Plot of S1 section in original reference frame
Define a dictionary of coordinates used by Networkx to plot section as a Directed graph.
Note that arrows are actually just thicker stubs
Can you write Python code for it?

start_pos={ii: [float(S1.g.node[ii]['ip'][i].subs(datav)) for i in range(2)] for ii in S1.g.nodes() }

plt.figure(figsize=(12,8),dpi=300)
nx.draw(S1.g,with_labels=True, arrows= True, pos=start_pos)
plt.arrow(0,0,20,0)
plt.arrow(0,0,0,20)
#plt.text(0,0, 'CG', fontsize=24)
plt.axis('equal')
plt.title("Section in starting reference Frame",fontsize=16);


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Shear center expression
Expressions can be messy, so we evaluate them to numerical values
Can you write Python code for it?


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
Triangular section
Define graph describing the section:
1) stringers are nodes with parameters:
- x coordinate
- y coordinate
- Area
2) panels are oriented edges with parameters:
- thickness
- lenght which is automatically calculated
Can you write Python code for it?

stringers = {1:[(sympy.Integer(0),h),A],
             2:[(sympy.Integer(0),sympy.Integer(0)),A],
             3:[(a,sympy.Integer(0)),A]}

panels = {(1,2):t,
          (2,3):t,
          (3,1):t}


I am working on the file "02_Triangular_Section.ipynb".
The first task is:
We also define numerical values for each symbol in order to plot scaled section and perform calculations
Can you write Python code for it?

values = [(A, 150 * ureg.millimeter**2),(A0, 250  * ureg.millimeter**2),(a, 80 * ureg.millimeter), \
          (b, 20 * ureg.millimeter),(h, 35 * ureg.millimeter),(L, 2000 * ureg.millimeter)]
datav = [(v[0],v[1].magnitude) for v in values]


I am working on the file "02_WP/Admin/example_py3/Winpython_checker.ipynb".
The first task is:
Graphics: Matplotlib, Pandas, Seaborn, Holoviews, Bokeh, bqplot, ipyleaflet, plotnine
Can you write Python code for it?

# Matplotlib
# for more examples, see: http://matplotlib.org/gallery.html
from mpl_toolkits.mplot3d import axes3d
import matplotlib.pyplot as plt
from matplotlib import cm

fig = plt.figure()
ax = fig.gca(projection='3d')
X, Y, Z = axes3d.get_test_data(0.05)
ax.plot_surface(X, Y, Z, rstride=8, cstride=8, alpha=0.3)
cset = ax.contourf(X, Y, Z, zdir='z', offset=-100, cmap=cm.coolwarm)
cset = ax.contourf(X, Y, Z, zdir='x', offset=-40, cmap=cm.coolwarm)
cset = ax.contourf(X, Y, Z, zdir='y', offset=40, cmap=cm.coolwarm)

ax.set_xlabel('X')
ax.set_xlim(-40, 40)
ax.set_ylabel('Y')
ax.set_ylim(-40, 40)
ax.set_zlabel('Z')
ax.set_zlim(-100, 100)

plt.show()

# Seaborn
# for more examples, see http://stanford.edu/~mwaskom/software/seaborn/examples/index.html
import seaborn as sns
sns.set()
df = sns.load_dataset("iris")
sns.pairplot(df, hue="species", size=1.5)

# altair-2.0.0 example
import altair as alt

# Uncomment/run this line to enable Altair in JupyterLab/nteract:
# alt.enable_mime_rendering() # api_v1
#alt.renderers.enable('default') # api_v2
alt.renderers.enable('notebook') # api_v2,if in Notebook
alt.Chart(df).mark_bar().encode(
    x=alt.X('sepal_length', bin=alt.Bin(maxbins=50)),
    y='count(*):Q',
    color='species:N',
    #column='species',
).interactive() # api_v1 .configure_cell(width=200)

# Holoviews
# for more example, see http://holoviews.org/Tutorials/index.html
import numpy as np
import holoviews as hv
hv.extension('matplotlib')
dots = np.linspace(-0.45, 0.45, 11)
fractal = hv.Image(image)

layouts = {y: (fractal * hv.Points(fractal.sample([(i,y) for i in dots])) +
               fractal.sample(y=y) )
            for y in np.linspace(0, 0.45,11)}

hv.HoloMap(layouts, kdims=['Y']).collate().cols(2)

# Bokeh 0.12.5
import numpy as np
from six.moves import zip
from bokeh.plotting import figure, show, output_notebook
N = 4000
x = np.random.random(size=N) * 100
y = np.random.random(size=N) * 100
radii = np.random.random(size=N) * 1.5
colors = ["#%02x%02x%02x" % (int(r), int(g), 150) for r, g in...


I am working on the file "02_WP/Admin/example_py3/Winpython_checker.ipynb".
The first task is:
Ipython Notebook: Interactivity & other
Can you write Python code for it?

import IPython;IPython.__version__

# Audio Example : https://github.com/ipython/ipywidgets/blob/master/examples/Beat%20Frequencies.ipynb
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
from ipywidgets import interactive
from IPython.display import Audio, display
def beat_freq(f1=220.0, f2=224.0):
    max_time = 3
    rate = 8000
    times = np.linspace(0,max_time,rate*max_time)
    signal = np.sin(2*np.pi*f1*times) + np.sin(2*np.pi*f2*times)
    print(f1, f2, abs(f1-f2))
    display(Audio(data=signal, rate=rate))
    try:
         plt.plot(signal); #plt.plot(v.result);
    except:
         pass
    return signal
v = interactive(beat_freq, f1=(200.0,300.0), f2=(200.0,300.0))
display(v)

# Networks graph Example : https://github.com/ipython/ipywidgets/blob/master/examples/Exploring%20Graphs.ipynb
%matplotlib inline
from ipywidgets import interact
import matplotlib.pyplot as plt
import networkx as nx
# wrap a few graph generation functions so they have the same signature

def random_lobster(n, m, k, p):
    return nx.random_lobster(n, p, p / m)

def powerlaw_cluster(n, m, k, p):
    return nx.powerlaw_cluster_graph(n, m, p)

def erdos_renyi(n, m, k, p):
    return nx.erdos_renyi_graph(n, p)

def newman_watts_strogatz(n, m, k, p):
    return nx.newman_watts_strogatz_graph(n, k, p)

@interact(n=(2,30), m=(1,10), k=(1,10), p=(0.0, 1.0, 0.001),
        generator={'lobster': random_lobster,
                   'power law': powerlaw_cluster,
                   'Newman-Watts-Strogatz': newman_watts_strogatz,
                   u'Erdős-Rényi': erdos_renyi,
                   })
def plot_random_graph(n, m, k, p, generator):
    g = generator(n, m, k, p)
    nx.draw(g)
    plt.title(generator.__name__)
    plt.show()


I am working on the file "02_WP/Admin/example_py3/Winpython_checker.ipynb".
The first task is:
Qt libraries Demo
See  Dedicated Qt Libraries Demo
Wrap-up
Can you write Python code for it?

# optional scipy full test (takes up to 10 minutes)
#!cmd /C start cmd /k python.exe  -c "import scipy;scipy.test()"

!pip list


I am working on the file "02_WP/Admin/example_py3/Winpython_checker.ipynb".
The first task is:
Split / Apply / Combine
Split your data into multiple independent groups.
Apply some function to each group.
Combine your groups back into a single data object.
Can you write Python code for it?


I am working on the file "02_WP/Admin/example_py3/Winpython_checker.ipynb".
The first task is:
Web Scraping: Beautifulsoup
Can you write Python code for it?

# checking Web Scraping: beautifulsoup and requests
import requests
from bs4 import BeautifulSoup

URL = 'http://en.wikipedia.org/wiki/Franklin,_Tennessee'

req = requests.get(URL, headers={'User-Agent' : "Mining the Social Web"})
soup = BeautifulSoup(req.text, "lxml")

geoTag = soup.find(True, 'geo')

if geoTag and len(geoTag) > 1:
    lat = geoTag.find(True, 'latitude').string
    lon = geoTag.find(True, 'longitude').string
    print ('Location is at', lat, lon)
elif geoTag and len(geoTag) == 1:
    (lat, lon) = geoTag.string.split(';')
    (lat, lon) = (lat.strip(), lon.strip())
    print ('Location is at', lat, lon)
else:
    print ('No location found')


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
Arrange rows with arrange(), sort()
Can you write Python code for it?

# arrange(flights, year, month, day)
flights.sort_values(['year', 'month', 'day'])

# arrange(flights, desc(arr_delay))
flights.sort_values('arr_delay', ascending=False)


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
But like Hadley mentions, not that useful since it only returns the one column. dplyr and pandas compare well here.
Can you write Python code for it?

# rename(flights, tail_num = tailnum)
flights.rename(columns={'tailnum': 'tail_num'})


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
Data: nycflights13
Can you write Python code for it?

flights = pd.read_csv("flights.csv", index_col=0)

# dim(flights)   <--- The R code
flights.shape  # <--- The python code

# head(flights)
flights.head()


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
FYI this returns a numpy array instead of a Series.
Can you write Python code for it?

# distinct(select(flights, origin, dest))
flights[['origin', 'dest']].drop_duplicates()


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
Grouped operations
Can you write Python code for it?

# planes <- group_by(flights, tailnum)
# delay <- summarise(planes,
#   count = n(),
#   dist = mean(distance, na.rm = TRUE),
#   delay = mean(arr_delay, na.rm = TRUE))
# delay <- filter(delay, count > 20, dist < 2000)

planes = flights.groupby("tailnum")
delay = (planes.agg({"year": "count",
                     "distance": "mean",
                     "arr_delay": "mean"})
               .rename(columns={"distance": "dist",
                                "arr_delay": "delay",
                                "year": "count"})
               .query("count > 20 & dist < 2000"))
delay


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
I'm not sure how dplyr is handling the other columns, like year, in the last example. With pandas, it's clear that we're grouping by them since they're included in the groupby. For the last example, we didn't group by anything, so they aren't included in the result.
Chaining
Any follower of Hadley's twitter account will know how much R users love the %>% (pipe) operator. And for good reason!
Can you write Python code for it?

# flights %>%
#   group_by(year, month, day) %>%
#   select(arr_delay, dep_delay) %>%
#   summarise(
#     arr = mean(arr_delay, na.rm = TRUE),
#     dep = mean(dep_delay, na.rm = TRUE)
#   ) %>%
#   filter(arr > 30 | dep > 30)
(
flights.groupby(['year', 'month', 'day'])
    [['arr_delay', 'dep_delay']]
    .mean()
    .query('arr_delay > 30 | dep_delay > 30')
)


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
It's worth mentioning the other common sorting method for pandas DataFrames, sort_index. Pandas puts much more emphasis on indicies, (or row labels) than R.
This is a design decision that has positives and negatives, which we won't go into here. Suffice to say that when you need to sort a DataFrame by the index, use DataFrame.sort_index.
Select columns with select(), []
Can you write Python code for it?

# select(flights, year, month, day)
flights[['year', 'month', 'day']]

# select(flights, year:day)
flights.loc[:, 'year':'day']

# select(flights, -(year:day))

# No direct equivalent here. I would typically use
# flights.drop(cols_to_drop, axis=1)
# or fligths[flights.columns.difference(pd.Index(cols_to_drop))]
# point to dplyr!

# select(flights, tail_num = tailnum)
flights.rename(columns={'tailnum': 'tail_num'})['tail_num']


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
Or using statsmodels directly for more control over the lowess, with an extremely lazy
"confidence interval".
Can you write Python code for it?

import statsmodels.api as sm

smooth = sm.nonparametric.lowess(delay.delay, delay.dist, frac=1/8)
ax = delay.plot(kind='scatter', x='dist', y = 'delay', figsize=(12, 6),
                color='k', alpha=.5, s=delay['count'] / 10)
ax.plot(smooth[:, 0], smooth[:, 1], linewidth=3);
std = smooth[:, 1].std()
ax.fill_between(smooth[:, 0], smooth[:, 1] - std, smooth[:, 1] + std, alpha=.25);

# destinations <- group_by(flights, dest)
# summarise(destinations,
#   planes = n_distinct(tailnum),
#   flights = n()
# )

destinations = flights.groupby('dest')
destinations.agg({
    'tailnum': lambda x: len(x.unique()),
    'year': 'count'
    }).rename(columns={'tailnum': 'planes',
                       'year': 'flights'})


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
Pandas is more verbose, but the the argument to columns can be any mapping. So it's often used with a function to perform a common task, say df.rename(columns=lambda x: x.replace('-', '_')) to replace any dashes with underscores. Also, rename (the pandas version) can be applied to the Index.
One more note on the differences here.
Pandas could easily include a .select method.
xray, a library that builds on top of NumPy and pandas to offer labeled N-dimensional arrays (along with many other things) does just that.
Pandas chooses the .loc and .iloc accessors because any valid selection is also a valid assignment. This makes it easier to modify the data.
python
flights.loc[:, 'year':'day'] = data
where data is an object that is, or can be broadcast to, the correct shape.
Extract distinct (unique) rows
Can you write Python code for it?

# distinct(select(flights, tailnum))
flights.tailnum.unique()


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
Single table verbs
dplyr has a small set of nicely defined verbs. I've listed their closest pandas verbs.

    dplyr
    pandas

    filter() (and slice())
    query() (and loc[], iloc[])

    arrange()
    sort_values and sort_index()

  select() (and rename())
    __getitem__  (and rename())

  distinct()
    drop_duplicates()

    mutate() (and transmute())
    assign

    summarise()
    None

    sample_n() and sample_frac()
    sample

    %>%
    pipe

Some of the "missing" verbs in pandas are because there are other, different ways of achieving the same goal. For example summarise is spread across mean, std, etc. It's closest analog is actually the .agg method on a GroupBy object, as it reduces a DataFrame to a single row (per group). This isn't quite what .describe does.
I've also included the pipe operator from R (%>%), the pipe method from pandas, even though it isn't quite a verb.
Filter rows with filter(), query()
Can you write Python code for it?

# filter(flights, month == 1, day == 1)
flights.query("month == 1 & day == 1")


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
Summarise values with summarise()
Can you write Python code for it?

# summarise(flights,
#   delay = mean(dep_delay, na.rm = TRUE))
flights.dep_delay.mean()


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
The first example is pretty much identical (aside from the names, mutate vs. assign).
The second example just comes down to language differences. In R, it's possible to implement a function like mutate where you can refer to gain in the line calcuating gain_per_hour, even though gain hasn't actually been calcuated yet.
In Python, you can have arbitrary keyword arguments to functions (which we needed for .assign), but the order of the argumnets is arbitrary since dicts are unsorted and **kwargs* is a dict. So you can't have something like df.assign(x=df.a / df.b, y=x **2), because you don't know whether x or y will come first (you'd also get an error saying x is undefined.
To work around that with pandas, you'll need to split up the assigns, and pass in a callable to the second assign. The callable looks at itself to find a column named gain. Since the line above returns a DataFrame with the gain column added, the pipeline goes through just fine.
Can you write Python code for it?

# transmute(flights,
#   gain = arr_delay - dep_delay,
#   gain_per_hour = gain / (air_time / 60)
# )
(flights.assign(gain=flights.arr_delay - flights.dep_delay)
        .assign(gain_per_hour = lambda df: df.gain / (df.air_time / 60))
        [['gain', 'gain_per_hour']])


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
This is only roughly equivalent.
summarise takes a callable (e.g. mean, sum) and evaluates that on the DataFrame. In pandas these are spread across pd.DataFrame.mean, pd.DataFrame.sum. This will come up again when we look at groupby.
Randomly sample rows with sample_n() and sample_frac()
Can you write Python code for it?

# sample_n(flights, 10)
flights.sample(n=10)

# sample_frac(flights, 0.01)
flights.sample(frac=.01)


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
We see the first big language difference between R and python.
Many python programmers will shun the R code as too magical.
How is the programmer supposed to know that month and day are supposed to represent columns in the DataFrame?
On the other hand, to emulate this very convenient feature of R, python has to write the expression as a string, and evaluate the string in the context of the DataFrame.
The more verbose version:
Can you write Python code for it?

# flights[flights$month == 1 & flights$day == 1, ]
flights[(flights.month == 1) & (flights.day == 1)]

# slice(flights, 1:10)
flights.iloc[:9]


I am working on the file "02_WP/Admin/example_py3/dplyr_pandas.ipynb".
The first task is:
using an internet download to get  flight.qcsv
Can you write Python code for it?

# Downloading and unzipg a file, without R method :
# source= http://stackoverflow.com/a/34863053/3140336
import io
from zipfile import ZipFile
import requests

def get_zip(file_url):
    url = requests.get(file_url)
    zipfile = ZipFile(io.BytesIO(url.content))
    zip_names = zipfile.namelist()
    if len(zip_names) == 1:
        file_name = zip_names.pop()
        extracted_file = zipfile.open(file_name)
        return extracted_file

url=r'https://github.com/winpython/winpython_afterdoc/raw/master/examples/nycflights13_datas/flights.zip'
with io.open("flights.csv", 'wb') as f:
        f.write(get_zip(url).read())

# Some prep work to get the data from R and into pandas
%matplotlib inline
import matplotlib.pyplot as plt
#%load_ext rpy2.ipython

import pandas as pd
import seaborn as sns

pd.set_option("display.max_rows", 5)


I am working on the file "02_WP/Admin/example_py3/installing_R.ipynb".
The first task is:
2 - checking and Installing R binary in the right place
Can you write Python code for it?

# checking it's the official R
import hashlib
def give_hash(of_file, with_this):
    with io.open(r_installer, 'rb') as f:
        return with_this(f.read()).hexdigest()
print (" "*12+"MD5"+" "*(32-12-3)+" "+" "*15+"SHA-1"+" "*(40-15-5)+"\n"+"-"*32+" "+"-"*40)
print ("%s %s %s" % (give_hash(r_installer, hashlib.md5) , give_hash(r_installer, hashlib.sha1),r_installer))
if give_hash(r_installer, hashlib.md5) == hashes[0] and give_hash(r_installer, hashlib.sha1) == hashes[1]:
   print("looks good!")
else:
   print("problem ! please check")
   assert give_hash(r_installer, hashlib.md5) == hashes[0]
   assert give_hash(r_installer, hashlib.sha1) == hashes[1]

# preparing Dos variables
os.environ["R_HOME"] = os.environ["WINPYDIR"]+ "\\..\\tools\\R\\"
os.environ["R_HOMEbin"]=os.environ["R_HOME"] + "bin"

# for installation we need this
os.environ["tmp_Rbase"]=os.path.join(os.path.split(os.environ["WINPYDIR"])[0]  , 'tools','R' )
if 'amd64' in sys.version.lower():
    r_comp ='/COMPONENTS="main,x64,translations'
else:
    r_comp ='/COMPONENTS="main,i386,translations'
os.environ["tmp_R_comp"]=r_comp

# let's install it, if hashes do match
assert give_hash(r_installer, hashlib.md5) == hashes[0]
assert give_hash(r_installer, hashlib.sha1) == hashes[1]
# If you are "USB life style", or multi-winpython
#   ==> CLICK the OPTION "Don't create a StartMenuFolder' <== (when it will show up)

!start cmd /C %r_installer% /DIR=%tmp_Rbase% %tmp_R_comp%


I am working on the file "02_WP/Admin/example_py3/installing_R.ipynb".
The first task is:
4- Install a R package via a IPython  Kernel
Can you write Python code for it?

%load_ext rpy2.ipython

#vitals: 'dplyr', 'R.utils', 'nycflights13'
# installation takes 2 minutes
%R install.packages(c('dplyr','R.utils', 'nycflights13'), repos='http://cran.rstudio.com/')


I am working on the file "02_WP/Admin/example_py3/installing_R.ipynb".
The first task is:
6 - Installing the very best of R pakages (optional, you will start to get a really big directory)
Can you write Python code for it?

# essentials: 'tidyr', 'shiny', 'ggplot2', 'caret' , 'nnet'
# remaining of Hadley Wickahm "stack" (https://github.com/rstudio)
%R install.packages(c('tidyr', 'ggplot2',  'shiny','caret' , 'nnet'), repos='https://cran.rstudio.com/')
%R install.packages(c('knitr', 'purrr', 'readr', 'readxl'), repos='https://cran.rstudio.com/')
%R install.packages(c('rvest', 'lubridate', 'ggvis', 'readr','base64enc'), repos='https://cran.rstudio.com/')

# TRAINING = online training book http://r4ds.had.co.nz/ (or https://github.com/hadley/r4ds)


I am working on the file "02_WP/General/PrintHead_Calculations.ipynb".
The first task is:
Error in Pixel Pitch $\Delta x_{Pitch}$because of substrate speed and clock frequency
$\Delta x_{Pitch} = v_{Substrate}∙\Delta t$
Error is the modulus of the achieved and the ideal pixel pitch.
$e = Rest(\frac{pixel_{Pitch}}{\Delta x_{Raster}})$
Can you write Python code for it?

import numpy as np
import pylab as pl
pl.clf()
frequencies = [50e6, 100e6, 150e6, 200e6] # MHz
speed = np.linspace(0,10,50) # 50 x points from 0m/s to 10m/s

for freq in frequencies:
  delta_x_pitch = (speed*100000) * 1/freq # um/s / s
  #error = numpy.mod(pixel_pitch[1],delta_x_pitch)
  # Plot graph
  pl.plot(speed,delta_x_pitch,label=str(int(freq/1000000))+" MHz")

# Place legend, Axis and Title
pl.legend(loc='best')
pl.xlabel("Speed [m/s]")
pl.ylabel("$\Delta$ x_pitch or Error [um]")
pl.title("Pixel Pitch Errors @ {}dpi".format(resolutions[1]))


I am working on the file "02_WP/General/PrintHead_Calculations.ipynb".
The first task is:
Fire Time
Can you write Python code for it?

def calc_fire_time(speed=10, dpi=1200): # speed = [m/min] dpi = [dpi]
  inch2m      = 0.0254
  pixel_pitch = inch2m/dpi # m
  fire_time = float(pixel_pitch) / (float(speed)/float(60))
  print("Speed       = {} m/min".format(speed))
  print("Resolution  = {} dpi".format(dpi))
  print("Fire Time   = {} s = {} kHz".format(fire_time, 1/fire_time/1000))
  fire_time_act = 0.000131
  dif = fire_time_act - fire_time
  dist = float(speed)/float(60)*dif
  print("Distance    = {} ".format(dist))
  print("")

calc_fire_time(speed=10,dpi=1200)
calc_fire_time(speed=10,dpi=600)
calc_fire_time(speed=10,dpi=150)
calc_fire_time(speed=50,dpi=600)
calc_fire_time(speed=50,dpi=150)