I am working on the file "05_Image_recognition_and_classification/cnn.ipynb".
The first task is:
将要优化的损失函数
To make the model better at classifying the input images, we must somehow change the variables for all the network layers. To do this we first need to know how well the model currently performs by comparing the predicted output of the model y_pred to the desired output y_true.
The cross-entropy is a performance measure used in classification. The cross-entropy is a continuous function that is always positive and if the predicted output of the model exactly matches the desired output then the cross-entropy equals zero. The goal of optimization is therefore to minimize the cross-entropy so it gets as close to zero as possible by changing the variables of the network layers.
TensorFlow has a built-in function for calculating the cross-entropy. Note that the function calculates the softmax internally so we must use the output of layer_fc2 directly rather than y_pred which has already had the softmax applied.
Can you write Python code for it?

cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits=layer_fc2,
                                                        labels=y_true)


I am working on the file "05_Image_recognition_and_classification/cnn.ipynb".
The first task is:
展开层
The convolutional layers output 4-dim tensors. We now wish to use these as input in a fully-connected network, which requires for the tensors to be reshaped or flattened to 2-dim tensors.
Can you write Python code for it?

layer_flat, num_features = flatten_layer(layer_conv3)

layer_flat

num_features


I am working on the file "05_Image_recognition_and_classification/cnn.ipynb".
The first task is:
所预测类
The second fully-connected layer estimates how likely it is that the input image belongs to each of the 2 classes. However, these estimates are a bit rough and difficult to interpret because the numbers may be very small or large, so we want to normalize them so that each element is limited between zero and one and the all the elements sum to one. This is calculated using the so-called softmax function and the result is stored in y_pred.
Can you write Python code for it?


I am working on the file "05_Image_recognition_and_classification/cnn.ipynb".
The first task is:
数据载入
Can you write Python code for it?

data = dataset.read_train_sets(train_path, img_size, classes, validation_size=validation_size)
test_images, test_ids = dataset.read_test_set(test_path, img_size)

print("Size of:")
print("- Training-set:\t\t{}".format(len(data.train.labels)))
print("- Test-set:\t\t{}".format(len(test_images)))
print("- Validation-set:\t{}".format(len(data.valid.labels)))


I am working on the file "05_Image_recognition_and_classification/cnn.ipynb".
The first task is:
评判手段
We need a few more performance measures to display the progress to the user.
This is a vector of booleans whether the predicted class equals the true class of each image.
Can you write Python code for it?

correct_prediction = tf.equal(y_pred_cls, y_true_cls)


I am working on the file "05_Image_recognition_and_classification/cnn.ipynb".
The first task is:
超参配置
Can you write Python code for it?

# Convolutional Layer 1.
filter_size1 = 3
num_filters1 = 32

# Convolutional Layer 2.
filter_size2 = 3
num_filters2 = 32

# Convolutional Layer 3.
filter_size3 = 3
num_filters3 = 64

# Fully-connected layer.
fc_size = 128             # Number of neurons in fully-connected layer.

# Number of color channels for the images: 1 channel for gray-scale.
num_channels = 3

# image dimensions (only squares for now)
img_size = 128

# Size of image when flattened to a single dimension
img_size_flat = img_size * img_size * num_channels

# Tuple with height and width of images used to reshape arrays.
img_shape = (img_size, img_size)

# class info
classes = ['dogs', 'cats']
num_classes = len(classes)

# batch size
batch_size = 32

# validation split
validation_size = .16

# how long to wait after validation loss stops improving before terminating training
early_stopping = None  # use None if you don't want to implement early stoping

train_path = 'data/train/'
test_path = 'data/test/'
checkpoint_dir = "models/"


I am working on the file "05_MNIST.ipynb".
The first task is:
2. 사전 설정
* model
* loss
* opimizer
Can you write Python code for it?

class MnistModel(nn.Module):
    def __init__(self):
        super(MnistModel, self).__init__()
        # input is 28x28
        # padding=2 for same padding
        self.conv1 = nn.Conv2d(1, 32, 5, padding=2)
        # feature map size is 14*14 by pooling
        # padding=2 for same padding
        self.conv2 = nn.Conv2d(32, 64, 5, padding=2)
        # feature map size is 7*7 by pooling
        self.fc1 = nn.Linear(64*7*7, 1024)
        self.fc2 = nn.Linear(1024, 10)

    def forward(self, x):
        x = F.max_pool2d(F.relu(self.conv1(x)), 2)
        x = F.max_pool2d(F.relu(self.conv2(x)), 2)
        x = x.view(-1, 64*7*7)   # reshape Variable
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return F.log_softmax(x)

model = MnistModel()
if is_cuda :  model.cuda()
loss_fn = nn.NLLLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.0001)


I am working on the file "05_MNIST.ipynb".
The first task is:
3. Trainning loop
* (입력 생성)
* model 생성
* loss 생성
* zeroGrad
* backpropagation
* optimizer step (update model parameter)

Summary 추가 및 출력하기
간단하게 loss와 accuracy를 list에 추가하도록 나중에 matplotlib를 이용하여 출력하겠습니다.
이 부분은 train_loss, train_accu라는 list에 loss와 accuracy를 append하면 됩니다.
물론 Tensorboard같은 별도의 전용 tool이 있으면 편리하겠지만, 간단히 확인할 경우 큰 불편사항은 없습니다.
일단 빈 list를 선언한 다음,
python
train_loss = []
train_accu = []
train과정이나 test과정에 loss와 accuracy를 추가하면 됩니다.
```python
pred = output.data.max(1)[1]
accuracy = pred.eq(target.data).sum()/batch_size
train_loss.append(loss.data[0])
train_accu.append(accuracy)
```
Can you write Python code for it?

# trainning
model.train()
train_loss = []
train_accu = []

for epoch in range(3):
    for i, (image, target) in enumerate(train_loader):
        if is_cuda :  image, target = image.cuda(), target.cuda()
        image, target = Variable(image), Variable(target) # 입력image Target 설정
        output = model(image) # model 생성
        loss = loss_fn(output, target) #loss 생성
        optimizer.zero_grad() # zero_grad
        loss.backward() # calc backward grad
        optimizer.step() # update parameter

        pred = output.data.max(1)[1]
        accuracy = pred.eq(target.data).sum()/batch_size

        train_loss.append(loss.data[0])
        train_accu.append(accuracy)

        if i % 300 == 0:
            print(i, loss.data[0])

plt.plot(train_accu)

plt.plot(train_loss)


I am working on the file "05_MNIST.ipynb".
The first task is:
4. Predict & Evaluate
Can you write Python code for it?

model.eval()
correct = 0
for image, target in test_loader:
    if is_cuda :  image, target = image.cuda(), target.cuda()
    image, target = Variable(image, volatile=True), Variable(target)
    output = model(image)
    prediction = output.data.max(1)[1]
    correct += prediction.eq(target.data).sum()

print('\nTest set: Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))


I am working on the file "05_NB_Exercise.ipynb".
The first task is:
An alternative to define colormaps
Can you write Python code for it?

subset = vtk.vtkMaskPoints()
subset.SetOnRatio(10)
subset.RandomModeOn()
subset.SetInputConnection(rectGridReader.GetOutputPort())

#vtk.vtkColorTransferFunction()
#vtk.vtkLookupTable()
lut = vtk.vtkLookupTable()
lut.SetNumberOfColors(256)
lut.SetHueRange(0.667, 0.0)
lut.SetVectorModeToMagnitude()
lut.Build()

hh = vtk.vtkHedgeHog()
hh.SetInputConnection(subset.GetOutputPort())
hh.SetScaleFactor(0.001)

hhm = vtk.vtkPolyDataMapper()
hhm.SetInputConnection(hh.GetOutputPort())
hhm.SetLookupTable(lut)
hhm.SetScalarVisibility(True)
hhm.SetScalarModeToUsePointFieldData()
hhm.SelectColorArray('vectors')
hhm.SetScalarRange((rectGridReader.GetOutput().GetPointData().GetVectors().GetRange(-1)))

hha = vtk.vtkActor()
hha.SetMapper(hhm)


I am working on the file "05_NB_Exercise.ipynb".
The first task is:
Challenge 2: Using the vtkCalulator to compute the vector magnitude
As you should have noticed our data set has only one point data array named vectors. We need now to use this array to calculate the magnitude of the vectors at each point of the grid. We will do this by using the vtk.vtkArrayCalculator().
Can you write Python code for it?

magnitudeCalcFilter = vtk.vtkArrayCalculator()
magnitudeCalcFilter.SetInputConnection(rectGridReader.GetOutputPort())
magnitudeCalcFilter.AddVectorArrayName('vectors')
# Set up here the array that is going to be used for the computation ('vectors')
magnitudeCalcFilter.SetResultArrayName('magnitude')
magnitudeCalcFilter.SetFunction("mag(vectors)")
# Set up here the function that calculates the magnitude of a vector
magnitudeCalcFilter.Update()

#Inspect the output of the calculator using the IPython console to verify the result


I am working on the file "05_NB_Exercise.ipynb".
The first task is:
Challenge 4: Visualize the data set as isosurfaces based on the "magnitude" value
Go to the documentation of vtkContourFilter and explain what does the GenerateValues() do?  Inspect the scalarRange of the magnitude array. What happens when you change these values in the function?
Can you write Python code for it?

scalarRange = ugrid.GetPointData().GetScalars().GetRange()
print(scalarRange)

isoFilter = vtk.vtkContourFilter()
isoFilter.SetInputData(ugrid)
isoFilter.GenerateValues(10, scalarRange)

isoMapper = vtk.vtkPolyDataMapper()
isoMapper.SetInputConnection(isoFilter.GetOutputPort())

isoActor = vtk.vtkActor()
isoActor.SetMapper(isoMapper)
isoActor.GetProperty().SetOpacity(0.5)


I am working on the file "05_NB_Exercise.ipynb".
The first task is:
Renderer, render window, and interactor
Can you write Python code for it?

#Option 1: Default vtk render window
renderer = vtk.vtkRenderer()
renderer.SetBackground(0.5, 0.5, 0.5)
renderer.AddActor(outlineActor)
renderer.ResetCamera()

renderWindow = vtk.vtkRenderWindow()
renderWindow.AddRenderer(renderer)
renderWindow.SetSize(500, 500)
renderWindow.Render()

iren = vtk.vtkRenderWindowInteractor()
iren.SetRenderWindow(renderWindow)
iren.Start()


I am working on the file "05_NB_VTKPython_Scalar.ipynb".
The first task is:
2. Filters

Filter 1: vtkRectilinearGridOutlineFilter() creates wireframe outline for a rectilinear grid.
Can you write Python code for it?

%qtconsole

rectGridOutline = vtk.vtkRectilinearGridOutlineFilter()
rectGridOutline.SetInputData(rectGridReader.GetOutput())


I am working on the file "05_NB_VTKPython_Scalar.ipynb".
The first task is:
3. Mappers

Mapper: vtkPolyDataMapper() maps vtkPolyData to graphics primitives.
Can you write Python code for it?


I am working on the file "05_NB_VTKPython_Scalar.ipynb".
The first task is:
4. Actors
Can you write Python code for it?

outlineActor = vtk.vtkActor()
outlineActor.SetMapper(rectGridOutlineMapper)
outlineActor.GetProperty().SetColor(0, 0, 0)


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Accurate Time - UT1
AstroPy calculates the times of events to a very high accuracy. To do this, is has to account for the fact that  Earth's rotation period is constantly changing due to tidal forces and changes in the Earth's moment of inertia.
To do this, AstroPy uses a time convention called UT1. This system is tied to the rotation of the Earth with repect to the positions of distant quasars. Since the Earth's rotation is constantly changing, the time system UT1 is constanly changing with repect to UTC.
The orientation of the Earth, which must be measured continuously to keep UT1 accurate. This measurement is logged by the International Earth Rotation and Reference Systems Service (IERS). They publish a "bulletin" with the most recent measurements of the Earth's orientation. This bulletin is constantly being updated.
You will run into occasions when you will get a warning that your dates are out of range of the IERS bulletin. To update the bulletin, run the follow block of code:
Can you write Python code for it?

from astroplan import download_IERS_A

download_IERS_A()


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Airmass < 2, you are good to go.
Planning observation is easier with plots
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt

from astroplan.plots import plot_sky, plot_airmass

plot_sky(my_target, mro, midnight_mro);

start_time = astro_set
end_time = astro_rise

delta_t = end_time - start_time

observe_time = start_time + delta_t * np.linspace(0.0, 1.0, 30)

# np.linspace(0, 1, 30) make 30 evenly spaced points from 0.0 to 1.0

plot_sky(my_target, mro, observe_time);


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Astronomical twilight is when the Sun is 18 degrees below the horizon
Can you write Python code for it?

astro_set = mro.twilight_evening_astronomical(now, which='nearest')
astro_rise = mro.twilight_morning_astronomical(now, which='next')

print("Astronomical Evening Twilight starts at {0.iso} UTC".format(astro_set))
print("Astronomical Midnight is at {0.iso} UTC".format(midnight_mro))
print("Astronomical Morning Twilight starts at {0.iso} UTC".format(astro_rise))

observing_length = (astro_rise - astro_set).to(u.h)

print("You can observe for {0:.1f} at MRO tonight".format(observing_length))

# Local Times

print("Astronomical Evening Twilight starts at {0} local time".format(astro_set.to_datetime(mytimezone)))
print("Astronomical Midnight is at {0} local time".format(midnight_mro.to_datetime(mytimezone)))
print("Astronomical Morning Twilight starts at {0} local time".format(astro_rise.to_datetime(mytimezone)))


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Information at your location
Can you write Python code for it?

sunset_here = astrolab.sun_set_time(now, which='nearest')
sunrise_here = astrolab.sun_rise_time(now, which='next')
midnight_here = astrolab.midnight(now, which='next')

print("Sunset will be at {0.iso} UTC".format(sunset_here))
print("Local Midnight will be at {0.iso} UTC".format(midnight_here))
print("Sunrise will be at {0.iso} UTC".format(sunrise_here))

print("Sunset will be at {0} local time".format(sunset_here.to_datetime(mytimezone)))
print("Local Midnight will be at {0} local time".format(midnight_here.to_datetime(mytimezone)))
print("Sunrise will be at {0} local time".format(sunrise_here.to_datetime(mytimezone)))


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Local Siderial Time (LST) will tell you the Right Ascension on the meridian right now.

You can use a star chart to find what constellations are visible now.
Can you write Python code for it?

midnight_mro = mro.midnight(now, which='next')

astrolab.local_sidereal_time(midnight_mro)


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Math with Time and Dates
Can you write Python code for it?

print("In 1 hour and 25 minutes it will be {0} UTC".format(now + 1*u.h + 25*u.min))

Christmas = Time("2016-12-25 00:00:00", format='iso')

dt = Christmas - now

print(dt.to(u.d))           # difference in days

print(dt.to(u.fortnight))   # difference in fortnights

print(dt.to(u.s))           # difference in seconds


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Most targets can be defined by name
Can you write Python code for it?

my_target = FixedTarget.from_name("m31")

my_target.coord

my_target.ra.hms


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Nice high altitude - looking good
Can you write Python code for it?

# You can find the airmass by using the .secz method

altaz_my_target.secz


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Objects in the sky - FixedTarget
You can define targets by coordinates
Can you write Python code for it?

coords = SkyCoord('02h19m00.0s', '+57d07m042s', frame='icrs')
ngc869 = FixedTarget(name='NGC869', coord=coords)

ngc869.ra

ngc869.ra.hms

astrolab.target_is_up(midnight_here, ngc869)

# Altitude and Azimuth of a target at a specific time

aa = astrolab.altaz(midnight_here, ngc869)

aa.alt.degree, aa.az.degree

# You can get the galactice coords of the target

aa.galactic

# You can get the coords at a different epoch (1950)

aa.fk4


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Objects in the sky - Moving Targets (solar system targets)

Astropy used the jplephem package to calculate the positions
The built-in solar system objects are: 'sun', 'mercury', 'venus', 'earth-moon-barycenter', 'earth', 'moon', 'mars', 'jupiter', 'saturn', 'uranus', 'neptune', 'pluto'
Can you write Python code for it?

from astropy.coordinates import get_sun, get_body, get_moon
from astroplan import moon_illumination

get_body('jupiter',now)

moon_midnight = get_moon(midnight_here)

moon_illumination(midnight_here)

my_target.coord.separation(moon_midnight)


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Planning - Observing at MRO
Air Mass is the optical path length through Earth’s atmosphere. At sea-level, the air mass at the zenith is 1. Air mass increases as you move toward the horizon, reaching a value of approximately 38 at the horizon.

The best time to observe a target is at minimum airmass.

When the airmass of your target is getting close to 2, you should be observing another target.
Can you write Python code for it?


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Plot the airmass of the target over the night
Can you write Python code for it?


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Setting your location - Observer
Can you write Python code for it?

astrolab = Observer(longitude = -122.311473 * u.deg,
                    latitude = 47 * u.deg + 39 * u.arcmin + 15 * u.arcsec,
                    elevation = 63.4 * u.m,
                    timezone = 'US/Pacific',
                    name = "Astrolab"
                    )

astrolab


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
The Manastash Ridge Observatory (MRO) is operated by the Astronomy Department of the University of Washington for the training of graduate and undergraduate students as well as for astronomical research.
Can you write Python code for it?

mro = Observer.at_site('mro')

mro

sunset_mro = mro.sun_set_time(now, which='nearest')

print("Sunset at MRO will be at {0} local time".format(sunset_mro.to_datetime(mytimezone)))

(sunset_here - sunset_mro).to(u.min)


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
This is good target for observation at MRO for most of the night
Not all targets can (or should) be observed at all locations
Can you write Python code for it?

mro.target_is_up(astro_set, mars_midnight)

plot_sky(mars_midnight, mro, observe_time);

plot_airmass(mars_midnight, mro, observe_time);


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Time and Dates

All dates and times in are UTC: Coordinated Universal Time

All Time calculation assume that the time is UTC
UTC is related to  Greenwich Mean Time (GMT) but does not change with a change of seasons.
Time will default to 00:00:00 UTC
Can you write Python code for it?


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
Working with timezones (local time)

The python package pytz is used to try to deal with local timezones
Timezone List
Working with tomezones is a quick path to maddness!
Only use timezone conversions for printouts, NEVER calculations!
Can you write Python code for it?

mytimezone = pytz.timezone('US/Pacific')

local_now = now.to_datetime(mytimezone)

print("The current local time is {0}".format(local_now))

# Nepal is in a strange timezone!

everest_timezone = pytz.timezone('Asia/Kathmandu')

everest_local_now = now.to_datetime(everest_timezone)

print("The current local time on Mt. Everest is {0}".format(everest_local_now))


I am working on the file "05_Observing_with_Python.ipynb".
The first task is:
You can turn solar system objects into pseudo FixedTarget objects for observational planning
Can you write Python code for it?

mars_midnight = FixedTarget(name='Mars', coord=get_body('mars',midnight_mro))

mars_midnight


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
Emoji can not be used as variable names (at least not yet ...)
Can you write Python code for it?


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
Formatting works with units
Can you write Python code for it?

NH_D = 34.47 * u.AU

"The New Horizons spacecraft is {0:.1f} from the Sun".format(NH_D)

"The New Horizons spacecraft is at a distance of {0.value:.1f} in the units of {0.unit:s} from the Sun".format(NH_D)


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
Justification and Cleaning
Can you write Python code for it?

line.center(100)

line.ljust(100)

line.rjust(100, "*")

line2 = "            My hovercraft is full of eels      "

line2.strip()

line3 = "*$*$*$*$*$*$*$*$My hovercraft is full of eels*$*$*$*$"

line3.strip('*$')

line3.lstrip('*$'), line3.rstrip('*$')


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
Raw strings begin with a special prefix (r) and signal Python not to interpret backslashes and other special metacharacters in the string, allowing you to pass them through directly to the regular expression engine.
Can you write Python code for it?

regex1 = r"test"

match1 = re.search(regex1, myline)

match1

myline[10:14]

match3 = re.findall(regex1, myline)

match3


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
RegEx Golf!
Can you write Python code for it?

golf_file = open("golf_00").read().splitlines()

golf_file

for i in golf_file:
    print(i)

def regex_test_list(mylist, myregex):

    for line in mylist:

        mytest = re.search(myregex, line)

        if (mytest):
            print(line + " YES")
        else:
            print(line + " NOPE")

regex = r"one"

regex_test_list(golf_file, regex)

regex = r"t|n"

regex_test_list(golf_file, regex)


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
Regular Expression in Python (re)
Can you write Python code for it?

import re

myline = "This is a test, this in only a test"

print(myline)


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
Strings are just arrays of characters
Can you write Python code for it?


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
Use explicit formatting to avoid these errors
Python string formatting has the form:
{Variable Index: Format Type}  .format(Variable)
Can you write Python code for it?

A = 42
B = 1.23456
C = 1.23456e10
D = 'Forty Two'

"I like the number {0:d}".format(A)

"I like the number {0:s}".format(D)

"The number {0:f} is fine, but not a cool as {1:d}".format(B,A)

"The number {0:.3f} is fine, but not a cool as {1:d}".format(C,A)       # 3 places after decimal

"The number {0:.3e} is fine, but not a cool as {1:d}".format(C,A)       # sci notation

"{0:g} and {1:g} are the same format but different results".format(B,C)


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
Watch out for variable types!
Can you write Python code for it?

n = 4

print("I would like " + n + " orders of spam")

print("I would like " + str(n) + " orders of spam")


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
Working with Files and Directories (OS agnostic)

The os package allows you to do operating system stuff without         worrying about what system you are using
Can you write Python code for it?

import os

os.chdir("./MyData")

my_data_dir = os.listdir()

my_data_dir

for file in my_data_dir:

    if file.endswith(".txt"):
        print(file)

for file in my_data_dir:

    if file.endswith(".txt"):
        print(os.path.abspath(file))


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
Working with strings
Can you write Python code for it?


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
You can also find files with glob
Can you write Python code for it?

import glob

my_files = glob.glob('02_*.fits')

my_files

for file in my_files:

    file_size = os.stat(file).st_size
    out_string = "The file {0} as a size of {1} bytes".format(file,file_size)

    print(out_string)


I am working on the file "05_Python_StringsAndStuff.ipynb".
The first task is:
You can compare strings
Can you write Python code for it?

"spam" == "good"

"spam" != "good"

"spam" == "spam"

"sp" < "spam"

"spam" < "eggs"


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Accessing elements by indexing:
Can you write Python code for it?

print(s1[0])
print(s2[4])

print(t1[1])

print(l2[2])


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Apply It!

Write a small program that finds your Chineze Zodiac and its characteristics using tuples and datetime module, resulting output:

This program will display your Chinese Zodiac Sign and Associated personal characteristics.

Enter your year of birth (yyyy): 1984
Your Chinese Zodiac sign is the Rat

Your personal charactersitics...
Forthright, industrious, sensitive, intellectual, sociable

Would you like to enter another year? (y/n): n

Here are the characteristics:

rat = 'Forthright, industrious, sensitive, intellectual, sociable'

ox = 'Dependable, methodical, modest, born leader, patient'

tiger = 'Unpredictable, rebellious, passionate, daring, impulsive'

rabbit = 'Good friend, kind, soft-spoken, cautious, artistic'

dragon = 'Strong, self-assured, proud. decisive, loyal'

snake = 'Deep thinker, creative, responsible, calm, purposeful'

horse = 'Cheerful, quick-witted, perceptive, talkative, open-minded'

goat = 'Sincere, sympathetic, shy, generous, mothering'

monkey = 'Motivator, inquisitive, flexible, innovative, problem solver'

rooster = 'Organizer, self-assured, decisive, perfectionist, zealous'

dog = 'Honest, unpretentious, idealistic, moralistic, easy going'

pig = 'Peace-loving, hard-working, trusting, understanding, thoughtful'

Test Time
Question1: Which of the following sequence types is a mutable type?
a) Strings
b) Lists
c) Tuples

Question2: What is the result of the following snippet:
lst = [4,2,9,1]
lst.insert(2,3)

a) [4,2,3,9,1]
b) [4,3,2,9,1]
c) [4,2,9,2,1]

Question3: Which of the following set of operations can be applied to any sequence?
a) len(s), s[i], s+w (concatenation)
b) max(s), s[i], sum(s)
c) len(s), s[i], s.sort()

Iterating Over Sequences
We can iterate over sequences using while loop, the one that we learned last lecture. However there is a better and more Pythonic way, it is called For loops

For loops are used to construct definite loops.

Syntax:
for k in...
Can you write Python code for it?


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Checking Membership with in :
Can you write Python code for it?


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Concatenation:
Can you write Python code for it?


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Finding Indexes from elements:
Can you write Python code for it?

# c = s1.count("l") # 2
word = "This is a very nice day in Houston!"
user_input = input("Please enter a letter that I can find index for you: ")
i = 0
lst = []
c = word.count(user_input)
while i < len(word):
    if c > 1:
        if word[i] == user_input:
            lst.append(i)
    elif c == 0:
        print(0)
        break
    else:
        if word[i] == user_input:
            print(i)
            break
    i = i + 1
print(lst)

s1.index('l')

l1.index('peach')

t1.index(3)


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Finding length with len():
Can you write Python code for it?

# Finding length for string
print(len(s1))
print(len(s2))

# Finding length for tuples
print(len(t1))
print(len(t2))

# Finding length for lists
print(len(l1))
print(len(l2))


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Finding maximum:
Can you write Python code for it?


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Finding minimum:
Can you write Python code for it?


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
For statement can be applied to all sequence types, including strings. Let's see how:
Can you write Python code for it?


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
List Comprehension
The range function allows for the generation of sequences of integers in fixed increments.
List Comprehensions in Python can be used to generate more varied sequences.
Can you write Python code for it?

[x**2 for x in [1,2,3]]

[x**2 for x in range(10)]

nums = [-1,1,-2,2,-3,3,-4,4,-5,5]
[x for x in nums if x >= 0]

[ord(ch) for ch in 'Hello World']

vowels = ('a', 'e', 'i','o', 'u')
w = 'Hello World!'
[ch for ch in w if ch in vowels]


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Lists and Tuples can used with more dimensions. Nested lists and tuples are giving this flexibility.
Can you write Python code for it?


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Now since we know about the beautiful for loops, it's time to learn a built-in function most commonly used with for loops: a built-in range() function
Can you write Python code for it?

list(range(2))

list(range(2,11))

list(range(1, 11, 2))


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Slicing:
Can you write Python code for it?

s1[1:4]

s2[3:]

l1[1:]

t1

t1[::-1] # Special Slicing feature


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Test Time
Question1: For nums=[10,30,20,40] , what does the following for loop output?
for k in nums:
    print(k)

Questions2: For fruit='strawberry' , what does the following for loop output?
for k in range(0,len(fruit),2):
    print(fruit[k], end='')

Question3: For nums=[12, 4, 11, 23, 18, 41, 27] , what is the value of k when the while loop terminates?
k = 0
while k < len(nums) and nums[k] != 18:
    k += 1
Can you write Python code for it?

nums=[12, 4, 11, 23, 18, 41, 27]
k = 0
while k < len(nums) and nums[k] != 18:
    k += 1
print(k)

fruit = "Strawberry"
for k in range(0, len(fruit),2):
    print(fruit[k], end='')


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
Tuples in Python
In contrast to lists, tuple is defined and cannot be altered. Otherwise, lists and tuples are essentially same. To denote tuples we use ( )
Can you write Python code for it?

nums = (10,20,30)

type(nums)

print(nums[2])

nums.insert(1,15) # Non alterable

del nums[2]

nums.append(40)

y = 75
x = 45

(x,y) = (y,x)

x

y


I am working on the file "05_Sequences_Lecture.ipynb".
The first task is:
When we trying to get the specific elements in nested lists or tuples, we should do something different:
Can you write Python code for it?

class_grades[0] # Prints the first sub-list

# We can get the specific item like this in a long way
student1_grades = class_grades[0]
student1_exam1 = student1_grades[0]
student1_exam1

# In short this is more conventional
class_grades[0][0]

# Let's write a small script that calculates the class average.
k = 0
exam_avg = []
while k < len(class_grades):
    avg = (class_grades[k][0] + class_grades[k][1] + class_grades[k][2]) / 3.0
    exam_avg.append(avg)
    k += 1

format((sum(exam_avg) / 3.0), '.2f')


I am working on the file "05_Stats/5_statistics.ipynb".
The first task is:
Donc, la probabilité d'obtenir une valeur entre -1 et 2 lors d'un tirage aléatoire chez une distribution normale à moyenne de 0 et à écart-type de 1 est d'environ 82%.
La courbe normale peut être utile pour évaluer la distribution d'une population. Par exemple, on peut calculer les limites de région sur la courbe normale qui contient 95% des valeurs possibles en tranchant 2.5% de part et d'autre de la moyenne. Il s'agit ainsi de l'intervalle de confiance sur la déviation de la distribution.
Can you write Python code for it?

fill_condition = (x > stats.norm(loc=0, scale=1).ppf(0.025)) & (x<stats.norm(loc=0, scale=1).ppf(0.975))
plt.plot(x, y1)
plt.fill_between(x[fill_condition], y1[fill_condition], color='pink')
plt.text(x=stats.norm(loc=0, scale=1).ppf(0.025), y=0.1,
         s=str(np.round(stats.norm(loc=0, scale=1).ppf(0.025), 2)), ha='center')
plt.text(x=stats.norm(loc=0, scale=1).ppf(0.975), y=0.1,
         s=str(np.round(stats.norm(loc=0, scale=1).ppf(0.975), 2)), ha='center')


I am working on the file "05_Stats/5_statistics.ipynb".
The first task is:
La probabilité que vous venez de calculer est une p-value.
Une p-value est une probabilité que le résultat soit obtenu de manière aléatoire si l'hypothèse nulle est vraie.
Dans le cas qui nous concerne, il s'agit de la probabilité d'obtenir 19 succès ou plus dans un jeu non biaisé. La convention veut que le seuil de significativité d'un test en biostatistique soit de 5%. Dans ce cas particulier, la probabilité de 0.7% est plus faible que 5% et nous pouvons conclure à ce seuil que la pièce est biaisée.
Nous y reviendrons, mais retenez pour l'instant que la p-value est dans la littérature scientifique une source importante de mauvaise interprétation et que le seuil purement arbitraire de 5% n'est pas un seuil particulièrement prudent.
Puisque vous maîtrisez bien Python, vous désirez éviter les formulations statistiques. Vous modélisez alors un grand nombre de jeux de 25 lancés que vous savez non-biaisés (avec numpy), puis vous calculez la proportion de ces lancés qui donnent 19 piles ou plus. Comme ceci.
Can you write Python code for it?

more_than_19 = 0 # initialisation du décompte des jeux où vous gagnez avec 19 et plus
n_sim = 100000 # nombre de simulations
for i in range(n_sim):
    pf_result = np.random.binomial(n=25, p=0.5) # tirage
    if pf_result >= 19: # si le tirage donne 19 ou plus, ajouter 1 au décompte
        more_than_19 = more_than_19 + 1

print("La proportion des", n_sim,
      "simulations ayant obtenu 19 piles ou plus est de",
      more_than_19/n_sim * 100, "%")


I am working on the file "05_Stats/5_statistics.ipynb".
The first task is:
Le résultat est semblable. Pour une personne habituée à réfléchir en terme de code, les tenants et aboutissants du processus par modélisation sont bien plus intuitif qu'un développement analytique. La formulation analytique reste néanmoins la référence en science.
La probabilité que la pièce ne soit pas biaisée est faible. Étant donné que nous avons nous-même écrit le code pour générer le tirage, nous pouvons auditer notre pièce de monnaie virtuelle. Or nous vions bien écrit pf_result = np.random.binomial(n=25, p=0.5), où p=0.5 indique que la pièce n'était pas biaisée. Bien que la probabilité que la pièce ne soit pas biaisée est très faible, cette probabilité n'est pas nulle. Trancher l'enjeu au seuil 0.05 nous mène vers une erreur de diagnostique de type 1, où nous rejetons l'hypothèse nulle alors qu'elle est vraie.
Les variables
Nous avons abordé au chapitre 3 la notion de variable par l'intermédiaire d'une donnée. Une variable est l'observation d'une caractéristique décrivant un échantillon et qui est susceptible de varier d'un échantillon à un autre. Si les observations varient en effet d'un échantillon à un autre, on parlera de variable aléatoire. Même le hasard est régit par certaines loi: ce qui est aléatoire dans une variable peut être décrit par des lois de probabilité, que nous verrons plus bas.
Mais restons aux variables pour l'instant. Par convention, on peut attribuer aux variables un symbole mathématique. Par exemple, on peut donner à la masse volumique d'un sol (qui est le résultat d'une méthodologie précise) le sympole $\rho$. Lorsque l'on attribue une valeur à $\rho$, on parle d'une donnée. Chaque donnée d'une observation a un indice qui lui est propre, que l'on désigne souvent par $i$, que l'on place en indice $\rho_i$. Pour la première donnée, on a $i=1$, donc $\rho_1$. Pour un nombre $n$ d'échantillons, on aura $\rho_1$, $\rho_2$, $\rho_3$, ..., $\rho_n$, formant le vecteur $\rho = \left[\rho_1, \rho_2, \rho_3, ..., \rho_n \right]$.
En Python, une variable est associée à une liste, une...
Can you write Python code for it?

ρ = np.array([1.34, 1.52, 1.26, 1.43, 1.39]) # matrice 1D
data = pd.DataFrame({'ρ': ρ}) # tableau
data


I am working on the file "05_Stats/5_statistics.ipynb".
The first task is:
Mise en situation: Vous jouez à pile ou face avec Schtroumpfette. Ce sera elle ou vous qui devrez payer un sac de chip au gagnant au jeu de pile ou face où la pèce est lancée 25 fois. Vous choisissez pile, elle choisi face. La fonction np.random.binomial retournera le nombre de 1 (pile). Quant à la fonction np.random.seed, elle assume que la sortie de la fonction aléatoire sera toujours la même à chaque fois que la cellule sera exécutée (pour les fins de l'exemple).
Can you write Python code for it?

np.random.seed(seed=184008)
pf_result = np.random.binomial(n=25, p=0.5)
pf_result


I am working on the file "05_Stats/5_statistics.ipynb".
The first task is:
Normale / Student
La plus répendue de ces lois est probablement la loi normale, parfois nommée loi gaussienne et plus rarement loi laplacienne. Il s'agit de la distribution classique en forme de cloche. Lorsque l'on a peut d'observations, en général moins de 30, la loi normale sera avantageusement substituée par la loi de Student. Au delà de 30, la loi de Student et la loi normale sont équivalentes.
La loi normale est décrite par une moyenne, qui désigne la tendance centrale, et une variance, qui désigne l'étalement des probabilités autours de la moyenne. La racine carrée de la variance est l'écart-type.
En analyse multivariée, des données distribuées de manière multinormale ont des distributions normales sous toutes les perspectives. Elles sont désignées par un vecteur de moyennes et une matrice de covariance.
Pour les distributions mesures exclusivement positives (comme le poids ou la taille) sont parfois avantageusement approximées par une loi log-normale, ce qui est une loi normale sur le logarithme des valeurs: la moyenne d'une loi log-normale est la moyenne géométrique.
Les modules scipy.stats permettent de tracer des courbes normales.
Can you write Python code for it?

x = np.arange(-10, 10, 0.001) # créer un vecteur
y1 = stats.norm.pdf(x, 0, 1) # la fonction de probabilité normale pour x, moyenne de 0, écart-type de 1
y2 = stats.norm.pdf(x, 0, 2)
y3 = stats.norm.pdf(x, 0, 3)

plt.plot(x, y1, label="Écart-type = 1")
plt.plot(x, y2, label="Écart-type = 2")
plt.plot(x, y3, label="Écart-type = 3")
plt.xlabel('Variable')
plt.ylabel('Densité')
plt.legend()


I am working on the file "05_Stats/5_statistics.ipynb".
The first task is:
On pourrait aussi être intéressé à l'intervalle de confiance sur la moyenne. En effet, la moyenne suit aussi une distribution normale, dont la tendance centrale est la moyenne de la distribution, et dont l'écart-type est noté erreur standard. On calcule cette erreur en divisant la variance par le nombre d'observation. Ainsi, pour 5 échantillons:
Can you write Python code for it?

standard_error = stats.norm(loc=0, scale=1/5).ppf(0.975) # égal à l'inverse de 0.025. notez l'échelle divisée par 5

fill_condition = (x > -standard_error) & (x < standard_error)
plt.plot(x, y1)
plt.fill_between(x[fill_condition], y1[fill_condition], color='pink')
plt.text(x=-standard_error, y=0.1, s=str(np.round(-standard_error, 2)), ha='center')
plt.text(x=standard_error, y=0.1, s=str(np.round(standard_error, 2)), ha='center')


I am working on the file "05_Stats/5_statistics.ipynb".
The first task is:
Poisson
La loi de Poisson (avec un P majuscule, introduite par le mathématicien français Siméon Denis Poisson) décrit des distributions discrètes de probabilité d'un nombre d'événements se produisant dans l'espace ou dans le temps. Les distributions de Poisson décrive ce qui tient du décompte. Il peut s'agir du nombre de grenouilles traversant une rue quotidiennement, du nombre de plants d'asclépiades se trouvant sur une terre cultivée, ou du nombre d'événemnts de précipitation au mois de juin, etc. La distribution de poisson n'a qu'un seul paramètre, $\lambda$, qui décrit tant la moyenne du décompte.
Par exemple, en un mois de 30 jours, et une moyenne de 8 événements de précipitation pour ce mois, on obtient la distribution suivante.
Can you write Python code for it?

x = np.arange(31)
y = stats.poisson(8).pmf(x)
print('La somme des probabilités est de', y.sum())

plt.plot(x, y, 'o', markersize=10)
plt.vlines(x, 0, y, color='blue')
plt.xlabel('Nombre de tirages gagnants')
plt.ylabel('Nombre de tirages gagnants');


I am working on the file "05_Stats/5_statistics.ipynb".
The first task is:
Pour certains modèles, vous devrez vous assurer vous-même de l'encodage. Pour d'autre, en particulier avec l'interface par formule de statsmodels, ce sera fait automatiquement. Nous verrons cela plus loin.
Les probabilités

« Nous sommes si éloignés de connaître tous les agens de la nature, et leurs divers modes d'action ; qu'il ne serait pas philosophique de nier les phénomènes, uniquement parce qu'ils sont inexplicables dans l'état actuel de nos connaissances. Seulement, nous devons les examiner avec une attention d'autant plus scrupuleuse, qu'il paraît plus difficile de les admettre ; et c'est ici que le calcul des probabilités devient indispensable, pour déterminer jusqu'à quel point il faut multiplier les observations ou les expériences, afin d'obtenir en faveur des agens qu'elles indiquent, une probabilité supérieure aux raisons que l'on peut avoir d'ailleurs, de ne pas les admettre. » — Pierre-Simon de Laplace

Une probabilité est la vraissemblance qu'un événements se réalise chez un échantillon. Les probabilités forment le cadre des systèmes stochastiques, c'est-à-dire des systèmes trop complexes pour en connaître exactement les aboutissants, auxquels ont attribue une part de hasard. Ces systèmes sont prédominants dans les processus vivants.
On peut dégager deux perspectives sur les probabilités: l'une passe par une interprétations fréquentistes, l'autre bayésiennes. L'interprétation fréquentiste représente la fréquence des occurences après un nombre infini d'événements. Par exemple, si vous jouez à pile ou face un grand nombre de fois, le nombre de pile sera égal à la moitié du nombre de lancés. Il s'agit de l'interprétation communément utilisée.
L'interprétation bayésienne vise à quantifier l'incertitude des phénomènes. Dans cette perspective, plus l'information s'accumule, plus l'incertitude diminue. Cette approche gagne en notoriété notamment parce qu'elle permet de décrire des phénomènes qui, intrinsèquement, ne peuvent être répétés infiniments (absence d'asymptote), comme celles qui sont bien...
Can you write Python code for it?

x = np.arange(26)
y = stats.binom(25, 0.5).pmf(x)
print('La somme des probabilités est de', y.sum())

plt.plot(x, y, 'o', markersize=10)
plt.vlines(x, 0, y, color='blue')
plt.xlabel('Nombre de tirages gagnants')
plt.ylabel('Nombre de tirages gagnants');


I am working on the file "05_Stats/5_statistics.ipynb".
The first task is:
Uniforme
La distribution la plus simple est probablement la distribution uniforme. Si la variable est discrète, chaque catégorie est associé à une probabilité égale. Si la variable est continue, la probabilité est directement proportionnelle à la largeur de l'intervalle.  On utilise rarement la distribution uniforme en biostatistiques, sinon pour décrire des a prioris vagues pour l'analyse bayésienne (ce sujet est traité dans le document 5.1_bayes.ipynb).
Can you write Python code for it?

x = np.arange(-4, 4, 0.01) # créer un vecteur
y1 = stats.uniform.pdf(x, -3, 6) # la fonction de probabilité uniforme pour x, partant de -3, d'une largeur de 6
y2 = stats.uniform.pdf(x, -2, 4)
y3 = stats.uniform.pdf(x, -1, 2)

plt.plot(x, y1, label="Uniforme entre -3 et 3")
plt.plot(x, y2, label="Uniforme entre -2 et 2")
plt.plot(x, y3, label="Uniforme entre -1 et 1")
plt.xlabel('Variable')
plt.ylabel('Densité')
plt.legend()


I am working on the file "05_Stats/5_statistics.ipynb".
The first task is:
Vous calculez les probabilités d'obtenir des résultats de 19 à 25 inclusivement, puis vous en faites la somme.
Can you write Python code for it?

probs = []
for i in range(19, 26): # avec range, la limite supérieure est exclue
    probs.append(my_binomial(25, i, 0.5))

print("La probabilité d'obtenir 19 piles ou plus est de",
       np.sum(probs) * 100, "%")


I am working on the file "05_Stats/5_statistics.ipynb".
The first task is:
Vous pouvez aussi bien utiliser le sous-module stats de scipy.
Can you write Python code for it?

from scipy import stats
stats.binom(25, 0.5).pmf(range(19, 26)).sum()


I am working on the file "05_autodiff_ex5.ipynb".
The first task is:
5.2)
Can you write Python code for it?

with graph.as_default():
    [fp] = tf.gradients(f, [x])

with tf.Session(graph=graph):
    print(fp.eval(feed_dict={x: 5.0}))


I am working on the file "05_autodiff_ex5.ipynb".
The first task is:
Exercise 5

5.1) Create a simple graph that computes the function $f(x) = x^2 - 3x + 1$. Define $x$ as a placeholder for a simple scalar value of type float32 value (i.e., shape=[], dtype=tf.float32). Create a session and evaluate $f(5)$. You should find 11.0.
5.2) Add an operation that computes the derivative of $f(x)$ with regards to $x$, noted $f'(x)$. Create a session and evaluate $f'(5)$. You should find 7.0.
Hint: use tf.gradients().
5.3) Using a MomentumOptimizer, find the value of $x$ that minimizes $f(x)$. You should find $\hat{x}=1.5$.
Hint: you need to change x into a Variable. Moreover, the MomentumOptimizer has its own variables that need to be initialized, so don't forget to create an init operation using a tf.global_variables_initializer(), and call it at the start of the session.
Try not to peek at the solution below before you have done the exercise! :)

Exercise 5 - Solution
5.1)
Can you write Python code for it?

graph = tf.Graph()
with graph.as_default():
    x = tf.placeholder(tf.float32, shape=[], name="x")
    f = tf.square(x) - 3 * x + 1

with tf.Session(graph=graph):
    print(f.eval(feed_dict={x: 5.0}))


I am working on the file "05_autodiff_ex5.ipynb".
The first task is:
Faster Optimizers
Can you write Python code for it?

learning_rate = 0.01
momentum = 0.8

graph = tf.Graph()
with graph.as_default():
    X = tf.constant(X_train, dtype=tf.float32, name="X")
    y = tf.constant(y_train, dtype=tf.float32, name="y")

    b = tf.Variable(0.0, name="b")
    w = tf.Variable(tf.zeros([1, 1]), name="w")
    y_pred = tf.add(tf.matmul(X, w), b, name="y_pred")  # X @ w + b

    mse = tf.reduce_mean(tf.square(y_pred - y), name="mse")

    optimizer = tf.train.MomentumOptimizer(learning_rate, momentum)
    training_op = optimizer.minimize(mse)

    init = tf.global_variables_initializer()

n_iterations = 500

with tf.Session(graph=graph) as sess:
    init.run()
    for iteration in range(n_iterations):
        if iteration % 100 == 0:
            print("Iteration {:5}, MSE: {:.4f}".format(iteration, mse.eval()))
        training_op.run()
    w_val, b_val = sess.run([w, b])

plt.figure(figsize=(10, 5))
plot_life_satisfaction_with_linear_model(X_train, y_train, w_val, b_val)
plt.show()


I am working on the file "05_autodiff_ex5.ipynb".
The first task is:
How does the optimizer know which variables to tweak? Answer: the TRAINABLE_VARIABLES collection.
Can you write Python code for it?

coll = graph.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES)
[var.op.name for var in coll]


I am working on the file "05_autodiff_ex5.ipynb".
The first task is:
Making Predictions Outside of TensorFlow
Can you write Python code for it?

cyprus_gdp_per_capita = 22000
cyprus_life_satisfaction = w_val[0][0] * cyprus_gdp_per_capita / 10000 + b_val
cyprus_life_satisfaction


I am working on the file "05_autodiff_ex5.ipynb".
The first task is:
Note that it's possible to replace the output value of any operation, not just placeholders. So, for example, even though x is now a Variable, you can use a feed_dict to use any value you want, for example to compute f(5.0). Important: this does not affect the variable!
Can you write Python code for it?

with tf.Session(graph=graph):
    init.run()
    print(x.eval()) # x == 0.0
    print(f.eval()) # f(0) == 1.0
    print(f.eval(feed_dict={x: 5.0})) # use 5.0 instead of the value of x, to compute f(5)
    print(x.eval()) # x is still 0.0
    print(f.eval()) # f(0) is still 1.0


I am working on the file "05_autodiff_ex5.ipynb".
The first task is:
Saving and Restoring a Model
Can you write Python code for it?

graph = tf.Graph()
with graph.as_default():
    X = tf.placeholder(tf.float32, shape=[None, 1], name="X")
    y = tf.placeholder(tf.float32, shape=[None, 1], name="y")

    b = tf.Variable(0.0, name="b")
    w = tf.Variable(tf.zeros([1, 1]), name="w")
    y_pred = tf.add(tf.matmul(X, w), b, name="y_pred")  # X @ w + b

    mse = tf.reduce_mean(tf.square(y_pred - y), name="mse")

    optimizer = tf.train.MomentumOptimizer(learning_rate, momentum)
    training_op = optimizer.minimize(mse)

    init = tf.global_variables_initializer()
    saver = tf.train.Saver() # <= At the very end of the construction phase

n_iterations = 500

with tf.Session(graph=graph) as sess:
    init.run()
    for iteration in range(n_iterations):
        if iteration % 100 == 0:
            print("Iteration {:5}, MSE: {:.4f}".format(
                iteration,
                mse.eval(feed_dict={X: X_train, y: y_train})))
        training_op.run(feed_dict={X: X_train, y: y_train}) # <= FEED THE DICT
    saver.save(sess, "./my_life_satisfaction_model")

with tf.Session(graph=graph) as sess:
    saver.restore(sess, "./my_life_satisfaction_model")
    # make the prediction:
    y_pred_val = y_pred.eval(feed_dict={X: X_test})

y_pred_val


I am working on the file "05_autodiff_ex5.ipynb".
The first task is:
Using autodiff Instead
Can you write Python code for it?

graph = tf.Graph()
with graph.as_default():
    X = tf.constant(X_train, dtype=tf.float32, name="X")
    y = tf.constant(y_train, dtype=tf.float32, name="y")

    b = tf.Variable(0.0, name="b")
    w = tf.Variable(tf.zeros([1, 1]), name="w")
    y_pred = tf.add(tf.matmul(X, w), b, name="y_pred")  # X @ w + b

    mse = tf.reduce_mean(tf.square(y_pred - y), name="mse")

    gradients_w, gradients_b = tf.gradients(mse, [w, b])  # <= IT'S AUTODIFF MAGIC!

    tweak_w_op = tf.assign(w, w - learning_rate * gradients_w)
    tweak_b_op = tf.assign(b, b - learning_rate * gradients_b)
    training_op = tf.group(tweak_w_op, tweak_b_op)

    init = tf.global_variables_initializer()

n_iterations = 2000

with tf.Session(graph=graph) as sess:
    init.run()
    for iteration in range(n_iterations):
        if iteration % 100 == 0:
            print("Iteration {:5}, MSE: {:.4f}".format(iteration, mse.eval()))
        training_op.run()
    w_val, b_val = sess.run([w, b])

plt.figure(figsize=(10, 5))
plot_life_satisfaction_with_linear_model(X_train, y_train, w_val, b_val)
plt.show()


I am working on the file "05_autodiff_ex5.ipynb".
The first task is:
Using placeholders
Can you write Python code for it?

graph = tf.Graph()
with graph.as_default():
    X = tf.placeholder(tf.float32, shape=[None, 1], name="X") # <= None allows for any
    y = tf.placeholder(tf.float32, shape=[None, 1], name="y") #    training batch size

    b = tf.Variable(0.0, name="b")
    w = tf.Variable(tf.zeros([1, 1]), name="w")
    y_pred = tf.add(tf.matmul(X, w), b, name="y_pred")  # X @ w + b

    mse = tf.reduce_mean(tf.square(y_pred - y), name="mse")

    optimizer = tf.train.MomentumOptimizer(learning_rate, momentum)
    training_op = optimizer.minimize(mse)

    init = tf.global_variables_initializer()

n_iterations = 500

X_test = np.array([[22000]], dtype=np.float32) / 10000

with tf.Session(graph=graph) as sess:
    init.run()
    for iteration in range(n_iterations):
        feed_dict = {X: X_train, y: y_train}
        if iteration % 100 == 0:
            print("Iteration {:5}, MSE: {:.4f}".format(
                iteration,
                mse.eval(feed_dict))) # <= FEED TRAINING DATA
        training_op.run(feed_dict)    # <= FEED TRAINING DATA
    # make the prediction:
    y_pred_val = y_pred.eval(feed_dict={X: X_test}) # <= FEED TEST DATA

y_pred_val


I am working on the file "05_bqnotebook/exploration.ipynb".
The first task is:
Arrival delay conditioned on departure delay
Can you write Python code for it?

%%bigquery depdelay
SELECT
    DEP_DELAY,
    AVG(ARR_DELAY) AS arrival_delay,
    STDDEV(ARR_DELAY) AS stddev_arrival_delay,
    COUNT(ARR_DELAY) AS numflights
FROM
    dsongcp.flights_tzcorr
GROUP BY
    DEP_DELAY
HAVING numflights > 370
ORDER BY DEP_DELAY

depdelay[:5]

ax = depdelay.plot(kind='line', x='DEP_DELAY',
              y='arrival_delay', yerr='stddev_arrival_delay')

Z_30 = 0.52
depdelay['arr_delay_30'] = (Z_30 * depdelay['stddev_arrival_delay']) \
             + depdelay['arrival_delay']

ax = plt.axes()
depdelay.plot(kind='line', x='DEP_DELAY', y='arr_delay_30',
              ax=ax, ylim=(0,30), xlim=(0,30), legend=False)
ax.set_xlabel('Departure Delay (minutes)')
ax.set_ylabel('> 30% prob of this\n Arrival Delay (minutes)');

x = np.arange(0, 30)
y = np.ones_like(x) * 15
ax.plot(x, y, 'r.');

y = np.arange(0, 30)
x = np.ones_like(y) * 13
ax.plot(x, y, 'g.');

%%bigquery depdelay
SELECT
    DEP_DELAY,
    APPROX_QUANTILES(ARR_DELAY, 101)[OFFSET(70)] AS arrival_delay,
    COUNT(ARR_DELAY) AS numflights
FROM
    dsongcp.flights_tzcorr
GROUP BY
    DEP_DELAY
HAVING numflights > 370
ORDER BY DEP_DELAY

ax = plt.axes()
depdelay.plot(kind='line', x='DEP_DELAY', y='arrival_delay',
              ax=ax, ylim=(0,30), xlim=(0,30), legend=False)
ax.set_xlabel('Departure Delay (minutes)')
ax.set_ylabel('> 30% prob of this\n Arrival Delay (minutes)');

x = np.arange(0, 30)
y = np.ones_like(x) * 15
ax.plot(x, y, 'r.');

y = np.arange(0, 30)
x = np.ones_like(y) * 16
ax.plot(x, y, 'g.');


I am working on the file "05_bqnotebook/exploration.ipynb".
The first task is:
Calls to BigQuery
We can also directly query BigQuery with the Python library:
Can you write Python code for it?

bq = bigquery.Client()

sql = """
SELECT
  COUNTIF(arr_delay >= 15)/COUNT(arr_delay) AS frac_delayed
FROM dsongcp.flights_tzcorr
"""
bq.query(sql).to_dataframe()


I am working on the file "05_bqnotebook/exploration.ipynb".
The first task is:
Installing dependencies
Regular Python dependencies can be installed using pip
Can you write Python code for it?


I am working on the file "05_bqnotebook/exploration.ipynb".
The first task is:
Juypter magic
Can you write Python code for it?

%%html
This cell will print out a <b> HTML </b> string.


I am working on the file "05_bqnotebook/exploration.ipynb".
The first task is:
Let's draw a Probability Distribution Function (PDF) of different arrival delays. In a Notebook we can assign the output of a cell magic query to a variable, in this case df:
Can you write Python code for it?

%%bigquery df
SELECT ARR_DELAY, DEP_DELAY
FROM dsongcp.flights_tzcorr
WHERE DEP_DELAY >= 10

type(df)

df.describe()

sns.set_style("whitegrid")
# sns.set(font_scale = 1.5)
ax = sns.violinplot(data=df, x='ARR_DELAY', inner='box', orient='h')
ax.axes.set_xlim(-50, 300);


I am working on the file "05_bqnotebook/exploration.ipynb".
The first task is:
Plotting distributions
Can you write Python code for it?

%%bigquery df
SELECT ARR_DELAY, DEP_DELAY
FROM dsongcp.flights_tzcorr

df.describe()

df['ontime'] = df['DEP_DELAY'] < 10
df[df['ARR_DELAY'] > 0].head()

sns.set_style("whitegrid")
ax = sns.violinplot(data=df, x='ARR_DELAY', y='ontime', inner='box', orient='h')
ax.set_xlim(-50, 200);

ax = sns.violinplot(data=df, x='ARR_DELAY', y='ontime',
                    inner='box', orient='h', gridsize=1000)
ax.set_xlim(-50, 200)


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
Adjectives and Adverbs
skipping
Unsimplified Tags
skipping
Exploring Tagged Corpora
Find words that follow 'often'
Can you write Python code for it?

brown_learned_text = brn.words(categories='learned')
sorted(set(b for a, b in nltk.bigrams(brown_learned_text) if a == 'often'))


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
Default taggers are still useful because after processing several thousand words of English text, most new words will be nouns

The Regular Expression Tagger
parts of speech expressed as regular expressions
Can you write Python code for it?

patterns = [
    (r'.*ing$', 'VBG'),               # gerunds
    (r'.*ed$', 'VBD'),                # simple past
    (r'.*es$', 'VBZ'),                # 3rd singular present
    (r'.*ould$', 'MD'),               # modals
    (r'.*\'s$', 'NN$'),               # possessive nouns
    (r'.*s$', 'NNS'),                 # plural nouns
    (r'^-?[0-9]+(.[0-9]+)?$', 'CD'),  # cardinal numbers
    (r'.*', 'NN')                     # nouns (default)
]

brown_sents[3]

regexp_tagger = nltk.RegexpTagger(patterns)
regexp_tagger.tag(brown_sents[3])

regexp_tagger.evaluate(brown_tagged_sents)


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
Example: create a default tagger that tags everything as NN
default taggers assign a tag to every single word
Can you write Python code for it?

raw = 'I do not like green eggs and ham, I do not like them Sam I am!'
tokens = word_tokenize(raw)
default_tagger = nltk.DefaultTagger('NN')
default_tagger.tag(tokens)


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
Identifying words that have ambiguous POS tags
Can you write Python code for it?

brown_news_tagged = brn.tagged_words(categories='news', tagset='universal')
data = nltk.ConditionalFreqDist((word.lower(), tag)
                                for word, tag in brown_news_tagged)
for word in sorted(data.conditions()):
    if len(data[word]) > 3:
        tags = [tag for tag, _ in data[word].most_common()]
        print(word, ' '.join(tags))


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
Lots of None tags! The BigramTagger never saw a lot of word pairs during training
Accuracy score will be very poor:
Can you write Python code for it?


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
N-Gram Tagging
Unigram Tagging
A unigram tagger behaves just like a lookup tagger, except there is a more convenient technique for setting it up, called training
Training a UnigramTagger by specifying tagged sentence data as a parameter
Can you write Python code for it?

from nltk.corpus import brown
brown_tagged_sents = brown.tagged_sents(categories='news')
brown_sents = brown.sents(categories='news')
unigram_tagger = nltk.UnigramTagger(brown_tagged_sents)
unigram_tagger.tag(brown_sents[2007])


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
Nouns
what parts of speech appear before nouns?
Can you write Python code for it?

word_tag_pairs = nltk.bigrams(brown_news_tagged)
list(word_tag_pairs)[:20]

word_tag_pairs = nltk.bigrams(brown_news_tagged) # generator needs to be redefined
noun_preceders = [a[1] for a, b in word_tag_pairs if b[1] == 'NOUN']
noun_preceders[:20]

fdist = nltk.FreqDist(noun_preceders)
[tag for tag, _ in fdist.most_common()]

fdist.plot(cumulative=True)


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
Reading Tagged Corpora
The/at Fulton/np-tl County/nn-tl Grand/jj-tl Jury/nn-tl said/vbd Friday/nr an/at investigation/nn of/in Atlanta's/np$ recent/jj primary/nn election/nn produced/vbd / no/at evidence/nn ''/'' that/cs any/dti irregularities/nns took/vbd place/nn ./.

if a corpus contains tagged text, the corpus interface will have a tagged_words() method
Can you write Python code for it?

nltk.corpus.brown.tagged_words()

nltk.corpus.brown.tagged_words(tagset='universal')

print(nltk.corpus.nps_chat.tagged_words())

nltk.corpus.conll2000.tagged_words()

nltk.corpus.treebank.tagged_words()


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
Separating Training and Testing Data -- train/test split
Can you write Python code for it?

split_size = int(len(brown_tagged_sents) * 0.9)
split_size

train_sents = brown_tagged_sents[:split_size]
test_sents = brown_tagged_sents[split_size:]
unigram_tagger = nltk.UnigramTagger(train_sents)
unigram_tagger.evaluate(test_sents)


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
Tagged Corpora
Representing Tagged Tokens
typically represented as a tuple: (token, tag)
Can be generated using str2tuple...
Can you write Python code for it?

tagged_token = nltk.tag.str2tuple('fly/NN')
tagged_token

sent = '''
The/AT grand/JJ jury/NN commented/VBD on/IN a/AT number/NN of/IN
other/AP topics/NNS ,/, AMONG/IN them/PPO the/AT Atlanta/NP and/CC
Fulton/NP-tl County/NN-tl purchasing/VBG departments/NNS which/WDT it/PPS
said/VBD ``/`` ARE/BER well/QL operated/VBN and/CC follow/VB generally/RB
accepted/VBN practices/NNS which/WDT inure/VB to/IN the/AT best/JJT
interest/NN of/IN both/ABX governments/NNS ''/'' ./.
'''
[nltk.tag.str2tuple(t) for t in sent.split()]


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
Tagged corpora in other languages...
Can you write Python code for it?

nltk.corpus.sinica_treebank.tagged_words()

nltk.corpus.indian.tagged_words()

nltk.corpus.mac_morpho.tagged_words()

nltk.corpus.conll2002.tagged_words()

nltk.corpus.cess_cat.tagged_words()


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
The Lookup Tagger
find 100 most frequent words and store their most-likely tag; evaluate its performance against previously tagged data brown_tagged_sents
Can you write Python code for it?

fd = nltk.FreqDist(brown.words(categories='news'))
cfd = nltk.ConditionalFreqDist(brown.tagged_words(categories='news'))
most_freq_words = fd.most_common(100)
likely_tags = dict((word, cfd[word].max()) for word, _ in most_freq_words)
baseline_tagger = nltk.UnigramTagger(model=likely_tags)
baseline_tagger.evaluate(brown_tagged_sents)


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
not all corpora have the same tag sets, so can force mapping to universal tagset
Can you write Python code for it?


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
reverse the order... conditions: tags, events: words
see the likely words, given a tag
Can you write Python code for it?

wsh = nltk.corpus.treebank.tagged_words()
cfd2 = nltk.ConditionalFreqDist((tag, word) for (word, tag) in wsj)
list(cfd2['ADJ'])[:25]


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
text.similar(w) finds words that appear in similar contexts to w
from book: The text.similar() method takes a word w, finds all contexts w1w w2, then finds all words w' that appear in the same context, i.e. w1w'w2
words similar to 'woman' -- mostly nouns
Can you write Python code for it?

text = nltk.Text(word.lower() for word in nltk.corpus.brown.words())
text.similar('woman')


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
words and tags are paired. treat the word as a condition and the tag as an event and create a cfd
lists the most common tags, given a word
Can you write Python code for it?

cfd1 = nltk.ConditionalFreqDist(wsj)
cfd1['yield'].most_common()

cfd1['cut'].most_common()


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
words similar to 'bought' -- mostly verbs
Can you write Python code for it?


I am working on the file "05_categorizing_and_tagging_words.ipynb".
The first task is:
~5% of trigrams are ambiguous
Confusion Matrix -- charting gold standard tags against tagger-assigned tags
Can you write Python code for it?

test_tags = [tag for sent in brown.sents(categories='editorial')
                 for word, tag in t2.tag(sent)]
gold_tags = [tag for word, tag in brown.tagged_words(categories='editorial')]
nltk.ConfusionMatrix(gold_tags, test_tags)

print(nltk.ConfusionMatrix(gold_tags, test_tags))


I am working on the file "05_create_dataset/05_audio.ipynb".
The first task is:
Text
We can break down a paragraph into sentences.
And we can do sentence-embedding to get a numeric representation of each sentence.
A paragraph now becomes an image!
Can you write Python code for it?

import tensorflow_hub as hub
paragraph = """
Siddhartha gave his clothes to a poor Brahmin on the road and
only retained his loincloth andearth-colored unstitched cloak.
He only ate once a day and never cooked food. He fasted fourteen
days. He fasted twenty-eight days. The flesh disappeared from
his legs and cheeks. Strange dreams were reflected in his enlarged
eyes. The nails grew long on his thin fingers and a dry, bristly
beard appeared on his chin. His glance became icy when he
encountered women; his lips curled with contempt when he passed
through a town of well-dressed people. He saw businessmen trading,
princes going to the hunt, mourners weeping over their dead,
prostitutes offering themselves, doctors attending the sick,
priests deciding the day for sowing, lovers making love, mothers
soothing their children -and all were not worth a passing glance,
everything lied, stank of lies; they were all illusions of sense,
happiness and beauty. All were doomed to decay. The world tasted
bitter. Life was pain.
"""

print(paragraph.split('.'))

embed = hub.load("https://tfhub.dev/google/universal-sentence-encoder/4")
embeddings = embed(paragraph.split('.'))

import matplotlib.pyplot as plt

plt.figure(figsize=(5,10))
plt.imshow(embeddings.numpy(), aspect=25.0, cmap='gray');


I am working on the file "05_create_dataset/05_audio.ipynb".
The first task is:
Vision ML on Audio, Video, Text, etc.
This notebook shows you how to use the spectrogram
of an audio file as a grayscale image input to an
ML model.
Can you write Python code for it?

!gsutil cp gs://ml-design-patterns/audio_train/00353774.wav cello.wav
!gsutil cp gs://ml-design-patterns/audio_train/001ca53d.wav sax.wav

import matplotlib.pyplot as plt
from scipy import signal
from scipy.io import wavfile
import numpy as np

fig, ax = plt.subplots(2, 2, figsize=(15, 10))
for idx, instr in enumerate(['sax', 'cello']):
  sample_rate, samples = wavfile.read(instr + '.wav')
  ax[idx][0].plot(samples)
  _, _, spectro = signal.spectrogram(samples, sample_rate)
  img = np.log(spectro)
  ax[idx][1].imshow(img, cmap='gray', aspect='auto');
  ax[idx][1].set_title(instr)
  print(img.shape)


I am working on the file "05_create_dataset/05_split_tfrecord.ipynb".
The first task is:
Running on Dataflow
Apache Beam code can be executed in a serverless way using Cloud Dataflow.
The key thing is to:

Replace beam.Pipeline() by:

options = {
      'staging_location': os.path.join(OUTPUT_DIR, 'tmp', 'staging'),
      'temp_location': os.path.join(OUTPUT_DIR, 'tmp'),
      'job_name': JOBNAME,
      'project': PROJECT,
      'teardown_policy': 'TEARDOWN_ALWAYS',
      'save_main_session': True
  }
opts = beam.pipeline.PipelineOptions(flags=[], **options)
with beam.Pipeline(RUNNER, options=opts) as p:

Can you write Python code for it?

%%bash
PROJECT=$(gcloud config get-value project)
BUCKET=${PROJECT}

python3 -m jpeg_to_tfrecord \
       --all_data gs://practical-ml-vision-book/flowers_5_jpeg/flower_photos/all_data.csv \
       --labels_file gs://practical-ml-vision-book/flowers_5_jpeg/flower_photos/dict.txt \
       --project_id $PROJECT \
       --output_dir gs://${BUCKET}/data/flower_tfrecords


I am working on the file "05_cython/05_actividad.ipynb".
The first task is:
La distancia de Hausdorff nuevamente...
En esta actividad volveremos a implementar la distancia/métrica de Hausdorff, pero ahora utilizando Cython.
La métrica de Hausdorff corresponde a un métrica o distancia ocupada para medir cuán disímiles son dos subconjuntos dados.
Esta tiene muchas aplicaciones, en particular para comparar el parecido entre imágenes. En el caso en donde los conjuntos son arreglos bidimensionales, la definición es la siguiente:
Sean $X \in \mathbb{R}^{m \times 3}$ e  $Y \in \mathbb{R}^{n \times 3}$ dos matrices, la métrica/distancia de Hausdorff sobre sobre estas como:
$$
d_H(X,Y) = \max \left(\ \max_{i\leq m} \min_{j \leq n} d(X[i],Y[j]), \ \max_{j\leq n} \min_{i \leq m} d(Y[j],X[i]) \ \right)
$$
donde $d$ es la distancia Euclideana clásica. ($X[i]$ indíca la i-ésima fila de X).
Ilustración unidimensional: Distancia entre funciones.
<img src='data/hausdorff.png' style="width: 600px;">
A continuación se le proveen 3 funciones que implementan tal métrica, usando Numba.
Can you write Python code for it?

@numba.jit('float64 (float64[:], float64[:])')
def metric_numba(x, y):
    """
    standard Euclidean distance
    """
    ret = x-y
    ret *= ret
    return np.sqrt(ret).sum()

@numba.jit('float64 (float64[:], float64[:,:])', nopython=True)
def inf_dist_numba(x, Y):
    """
    inf distance between row x and array Y
    """
    m = Y.shape[0]
    inf = np.inf

    for i in range(m):
        dist = metric_numba(x, Y[i])
        if dist < inf:
            inf = dist
    return inf

@numba.jit('float64 (float64[:,:], float64[:,:])', nopython=True)
def hausdorff_numba(X, Y):
    """
    Hausdorff distance between arrays X and Y
    """
    m = X.shape[0]
    n = Y.shape[0]
    sup1 = -1.
    sup2 = -1.

    for i in range(m):
        inf1 = inf_dist_numba(X[i], Y)
        if inf1 > sup1:
            sup1 = inf1
    for i in range(n):
        inf2 = inf_dist_numba(Y[i], X)
        if inf2 > sup2:
            sup2 = inf2

    return max(sup1, sup2)


I am working on the file "05_cython/05_cython.ipynb".
The first task is:
4.- Other advanced things you can do with Cython
We have seen that with Cython we can implement our algorithms achieving C performance. Moreover it is very versatile and we can do some other advanced thing with it:
Object-oriented programming: Classes and methods
To support object-oriented programming, Cython supports writing normal Python classes exactly as in Python.
Can you write Python code for it?

%%cython -a -c=-fPIC -c=-fwrapv -c=-O3 -c=-fno-strict-aliasing
#!python
#cython: cdivision=True, boundscheck=False, nonecheck=False, wraparound=False, initializedcheck=False

cdef class A(object):
    def   d(self): return 0
    cdef  int c(self): return 0
    cpdef int p(self): return 0

    def test_def(self, long num):
        while num > 0:
            self.d()
            num -= 1

    def test_cdef(self, long num):
        while num > 0:
            self.c()
            num -= 1

    def test_cpdef(self, long num):
        while num > 0:
            self.p()
            num -= 1

%%timeit n = 1000000
a1 = A()
a1.test_def(n)

%%timeit n = 1000000
a1 = A()
a1.test_cdef(n)

%%timeit n = 1000000
a1 = A()
a1.test_cpdef(n)


I am working on the file "05_cython/05_cython.ipynb".
The first task is:
Alternative usage of Cython: Outside the notebook
If you want to use Cython outside the notebook (the way it was thought...), you have to do the work of the magic:
1. Write the function into a .pyx file.
2. Cythonize it with cython filename.pyx generating the filename.c file.
3. Compile it with GCC:
gcc -shared -fPIC -fwrapv -O3 -fno-strict-aliasing -I/home/mavillan/anaconda3/include/python3.6m -o primes.so primes.c
<div id='cython++' />
2.- Advanced usage
In this section we will consider the example of computing a distance matrix: Given the matrices $A_{m,3}$ and $B_{n,3}$ (each row is a 3D-position), the distance matrix has entries $D_{i,j} = d(A[i],B[j])$.
NumPy Arrays
You can use NumPy from Cython exactly the same as in regular Python, but by doing so you are losing potentially high speedups because Cython has support for fast access to NumPy arrays.
Can you write Python code for it?

# Matrices to use
A = np.random.random((1000,3))
B = np.random.random((500,3))

def dist(a, b):
    return np.sqrt(np.sum((a-b)**2))

def distance_matrix_python(A, B):
    m = A.shape[0]
    n = B.shape[0]
    D = np.empty((m,n))
    for i in range(m):
        for j in range(n):
            D[i,j] = dist(A[i],B[j])
    return D

%timeit distance_matrix_python(A,B)

%%cython -a
import numpy as np

def dist(a, b):
    return np.sqrt(np.sum((a-b)**2))

def distance_matrix_cython0(A, B):
    m = A.shape[0]
    n = B.shape[0]
    D = np.empty((m,n))
    for i in range(m):
        for j in range(n):
            D[i,j] = dist(A[i],B[j])
    return D

%timeit distance_matrix_cython0(A,B)


I am working on the file "05_cython/05_cython.ipynb".
The first task is:
Compiler optimization
With the -c option we can pass the compiler (gcc) optimization options. Below we use the most common of them:
Can you write Python code for it?

%%cython -a -c=-fPIC -c=-fwrapv -c=-O3 -c=-fno-strict-aliasing
import numpy as np
cimport numpy as cnp

ctypedef cnp.float64_t float64_t
from libc.math cimport sqrt

def dist(float64_t[::1] a, float64_t[::1] b):
    cdef:
        int i = 0
        int n = a.shape[0]
        float ret = 0
    for i in range(n):
        ret += (a[i]-b[i])**2
    return sqrt(ret)

def distance_matrix_cython4(float64_t[:,::1] A, float64_t[:,::1] B):
    cdef:
        int m = A.shape[0]
        int n = B.shape[0]
        int i,j
        float64_t[:,::1] D = np.empty((m,n))
    for i in range(m):
        for j in range(n):
            D[i,j] = dist(A[i], B[j])
    return D

%timeit -n 10 distance_matrix_cython4(A,B)


I am working on the file "05_cython/05_cython.ipynb".
The first task is:
Function inlining
In computing, inline expansion, or inlining, is a manual or compiler optimization that replaces a function call site with the body of the called function.
As a rule of thumb: Some inlining will improve speed at very minor cost of space, but excess inlining will hurt speed, due to inlined code consuming too much of the instruction cache, and also cost significant space.
Can you write Python code for it?

%%cython -a -c=-fPIC -c=-fwrapv -c=-O3 -c=-fno-strict-aliasing
#!python
#cython: cdivision=True, boundscheck=False, nonecheck=False, wraparound=False, initializedcheck=False

import numpy as np
cimport numpy as cnp

ctypedef cnp.float64_t float64_t
from libc.math cimport sqrt

cdef inline float64_t dist(float64_t[::1] a, float64_t[::1] b):
    cdef:
        int i = 0
        int n = a.shape[0]
        float ret = 0
    for i in range(n):
        ret += (a[i]-b[i])**2
    return sqrt(ret)

def distance_matrix_cython7(float64_t[:,::1] A, float64_t[:,::1] B):
    cdef:
        int m = A.shape[0]
        int n = B.shape[0]
        int i,j
        float64_t[:,::1] D = np.empty((m,n))
    for i in range(m):
        for j in range(n):
            D[i,j] = dist(A[i,:], B[j,:])
    return D

%timeit -n 10 distance_matrix_cython7(A,B)


I am working on the file "05_cython/05_cython.ipynb".
The first task is:
Numba wins this time! but: This is not the final form of Cython...
Inspecting Cython bottlenecks with annotations
We can inspect the C code generated by Cython with the -a argument. Let's inspect the code used above.
The non-optimized lines will be shown in a gradient of yellow (white lines are faster, yellow lines are slower), telling you which lines are the least efficiently compiled to C. By clicking on a line, you can see the generated C code corresponding to that line.
Can you write Python code for it?

%%cython -a
def primes_cython1(n):
    primes = [False, False] + [True] * (n - 2)
    i= 2
    while i < n:
        # We do not deal with composite numbers.
        if not primes[i]:
            i += 1
            continue
        k= i+i
        # We mark multiples of i as composite numbers.
        while k < n:
            primes[k] = False
            k += i
        i += 1
    # We return all numbers marked with True.
    return [i for i in range(2, n) if primes[i]]

%%cython -a
def primes_cython2(int n):
    # Note the type declarations below
    cdef list primes = [False, False] + [True] * (n - 2)
    cdef int i = 2
    cdef int k = 0
    # The rest of the functions is unchanged
    while i < n:
        # We do not deal with composite numbers.
        if not primes[i]:
            i += 1
            continue
        k= i+i
        # We mark multiples of i as composite numbers.
        while k < n:
            primes[k] = False
            k += i
        i += 1
    # We return all numbers marked with True.
    return [i for i in range(2, n) if primes[i]]


I am working on the file "05_cython/05_cython.ipynb".
The first task is:
Pure C functions
With the cdef keywork we can realy define C function, as we shown below. In such functions all variable types should be defined and should have a return type, and can't be called directly in Python, i.e, only can be called by functions defined in the same module.
There is a midpoint between def and cdef which automatically creates a Python function with the same name, so the function can be called directly.
Can you write Python code for it?

%%cython -a -c=-fPIC -c=-fwrapv -c=-O3 -c=-fno-strict-aliasing
#!python
#cython: cdivision=True, boundscheck=False, nonecheck=False, wraparound=False, initializedcheck=False

import numpy as np
cimport numpy as cnp

ctypedef cnp.float64_t float64_t
from libc.math cimport sqrt

cdef float64_t dist(float64_t[::1] a, float64_t[::1] b):
    cdef:
        int i = 0
        int n = a.shape[0]
        float ret = 0
    for i in range(n):
        ret += (a[i]-b[i])**2
    return sqrt(ret)

def distance_matrix_cython6(float64_t[:,::1] A, float64_t[:,::1] B):
    cdef:
        int m = A.shape[0]
        int n = B.shape[0]
        int i,j
        float64_t[:,::1] D = np.empty((m,n))
    for i in range(m):
        for j in range(n):
            D[i,j] = dist(A[i,:], B[j,:])
    return D

%timeit -n 10 distance_matrix_cython6(A,B)


I am working on the file "05_cython/05_cython.ipynb".
The first task is:
Then: In general, Cython will be the most efficient when it can compile data structures and operations directly to C by __making as few CPython API calls as possible__. Specifying the types of the variables often leads to greater speed improvements.
Just for curiosity let's see the performance Numba's JIT achieves:
Can you write Python code for it?

@numba.jit(nopython=True)
def primes_numba(n):
    primes = [False, False] + [True] * (n - 2)
    i= 2
    while i < n:
        # We do not deal with composite numbers.
        if not primes[i]:
            i += 1
            continue
        k= i+i
        # We mark multiples of i as composite numbers.
        while k < n:
            primes[k] = False
            k += i
        i += 1
    # We return all numbers marked with True.
    res = []
    for i in range(2,n):
        if primes[i]: res.append(i)
    return res

tn = %timeit -o primes_numba(10000)


I am working on the file "05_cython/05_cython.ipynb".
The first task is:
We achieve x2 speed improvement doing (practically) nothing!.
When we add %%cython at the beginning of the cell, the code gets compiled by Cython into a C extension. Then, this extension is loaded, and the compiled function is readily available in the interactive namespace.
Lets help the compiler by explicitly defining the type of the variables with the cdef macro/keyword:
Can you write Python code for it?

%%cython
def primes_cython2(int n):
    # Note the type declarations below
    cdef list primes = [False, False] + [True] * (n - 2)
    cdef int i = 2
    cdef int k = 0
    # The rest of the functions is unchanged
    while i < n:
        # We do not deal with composite numbers.
        if not primes[i]:
            i += 1
            continue
        k= i+i
        # We mark multiples of i as composite numbers.
        while k < n:
            primes[k] = False
            k += i
        i += 1
    # We return all numbers marked with True.
    return [i for i in range(2, n) if primes[i]]

tc2 = %timeit -o primes_cython2(10000)

print("Cython version 1 speedup: {0}".format(tp.best/tc1.best))
print("Cython version 2 speedup: {0}".format(tp.best/tc2.best))


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
A similar yet distint representation is provided by pcolormesh().
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Above commands enable pylab environment => direct access to numpy, scipy and matplotlib. The option 'inline' results in plot outputs to be directly embedded in the Notebook. If this causes problems, remove the option 'inline'.
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Alternatively, Matplotlib understands the MATLAB syntax, e.g., for the above command (does not work with 'inline' enabled):
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Anybody bar charts?
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
As you will have seen, we retrieved handles to the individual pie slices. Let's do something with them:
Can you write Python code for it?

figure(figsize=(5,5))
handles=pie([1,2,3,4], explode=[0.2,0,0,0], shadow=True, labels=['A','B','C','D'])
setp(handles[0] [0], color='y')
setp(handles[1] [0], text='Blubber')


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
By using Numpy arrays, Matplotlib can conveniently be used as a function ploting program:
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Compare the two figures, spot the similarities and differences.
Matplotlib sports an add-on module for 3D graphics as detailed below. First, we need to import this module.
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Contour plots are well suited for visualization of three-dimensional data sets:
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Especially for figures with multiple subpanels it may be advisable to increase the figure size somewhat. Do this by using function arguments in the figure() call:
Can you write Python code for it?

fig2=figure(figsize=(10,10))
subplot(2,1,1)
plot(xv,yv,'b-')
subplot(2,1,2)
plot(yv,xv,'ro')


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Exercises for Numpy and Matplotlib
Exercise 1
Plot the following functions into individual subplots:
$f(x) = e^{-\alpha x^2},\quad g(x) = \alpha x^3-5x^2, \quad h(x) = \mathrm{erf}(x)$
with $\alpha \in {0.1,0.5,1}$
hint: use scipy for the definition of $\mathrm{erf}(x)$
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Figures can be saved in a number of output formats such as Postscript:
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
For horizontal bar charts, you would use barh().
Where there are bars, there should be pies:
Can you write Python code for it?

close('all')
figure(figsize=(5,5))
handles=pie([1,2,3,4], explode=[0.2,0,0,0], shadow=True, labels=['A','B','C','D'])
handles


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Get a handle to current figure and close it:
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
In addition to the above 3D scatter plot, other plot types are supported, such as 3D surface plots:
Can you write Python code for it?

close('all'); fig=figure()
ax=Axes3D(fig)
xv=linspace(-10,10,100); yv=linspace(-10,10,100)
cx,cy=meshgrid(xv,yv)
cz=0.5*cx+exp(-cy**2)
tilt=ax.plot_surface(cx,cy,cz,linewidth=0, cmap=cm.jet)


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Let's add grid lines:
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Let's display the same data in contour representation:
Can you write Python code for it?

close('all'); fig=figure()
ax=Axes3D(fig)
ax.contour(cx,cy,cz,cstride=2,rstride=2, cmap=cm.jet)


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Let's do a figure with several subpanels:
Can you write Python code for it?

fig2=figure()
subplot(2,1,1)
plot(xv,yv,'b-')
subplot(2,1,2)
plot(yv,xv,'ro')

close(fig2)


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Let's pimp the plot a little:
Can you write Python code for it?

mybar=bar(xv, yv, width=1, yerr=0.5)
xticks(xv, ['A','B','C','D'])
setp(mybar, color='r',
edgecolor='k')


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Let's switch to the Qt backend for 3D plotting
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Matplotlib also offers quiver() plots which are illustrated in the following example (taken from http://www.loria.fr/~rougier/teaching/matplotlib/):
Can you write Python code for it?

close('all')
n=8; X,Y=np.mgrid[0:n,0:n]
T=np.arctan2(Y-n/2.0,X-n/2.0)
R=10+np.sqrt((Y-n/2.0)**2+(X-n/2.0)**2)
U,V=R*np.cos(T),R*np.sin(T)
axes([0.025,0.025,0.95,0.95])
quiver(X,Y,U,V,R,alpha=.5)
quiver(X,Y,U,V, edgecolor='k', facecolor= 'None', linewidth=.5)
show()


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Or in MATLAB style, close all open figures:
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Polar plots are also nicely illustrated on the very same homepage:
Can you write Python code for it?

close('all')
ax=axes([0.025,0.025,0.95,0.95],polar=True)
N=20; theta=np.arange(0.0,2*np.pi,2*np.pi/N)
radii=10*np.random.rand(N)
width=np.pi/4*np.random.rand(N)
bars=bar(theta,radii,width=width,bottom=0.0)
for r,bar in zip(radii,bars):
    bar.set_facecolor( cm.jet(r/10.))
    bar.set_alpha(0.5)
show()


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Some more commands for plot formatting:
Can you write Python code for it?

axis()

axis([0.5,4.5,-0.5,5.5])

ti=title('Very important data')
xl=xlabel('time'); yl=ylabel('value')
setp(xl,fontweight='bold')


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Style sheets
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Then, we can play around.
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
There are several predefinded style sheets for matplotlib. You can show all availible styles by typing
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Try moving and rotating the (so far empty) plot in three dimensions. Once you have calmed down, let's populate the plot with some data:
Can you write Python code for it?

close('all')
fig=figure(); ax=Axes3D(fig)
import random as rn
xv=[]; yv=[]; zv=[]
for c in range(100):
   xv.append(rn.random()); yv.append(rn.random())
   zv.append(rn.random())
ax.scatter(xv,yv,zv)


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
Try some other colormaps such as cm.bone, cm.spring or cm.cool (once more, these are the MATLAB color schemes).
We do one more example that shows the use of 3D surface and 3D contour plots (as opposed to the 2D contour plots above).
Can you write Python code for it?

close('all'); fig=figure()
ax=Axes3D(fig)
xv=linspace(-10,10,100); yv=linspace(-10,10,100)
cx,cy=meshgrid(xv,yv)
cz=0*cx

def gauss2D(x0,y0,sigx=1,sigy=1,height=1):
   z=height*exp(-((cx-x0)**2/2/sigx**2)-((cy-y0)**2/2/sigy**2))
   return z

cz=cz+gauss2D(-2,3)
cz=cz+gauss2D(2,4,2,3)
ax.plot_surface(cx,cy,cz,linewidth=0,cstride=2, rstride=2,cmap=cm.jet)


I am working on the file "05_dataio_matplotlib/.ipynb_checkpoints/Matplotlib_2015-checkpoint.ipynb".
The first task is:
and your plots will look similar to those created with ggplot2 in R
Can you write Python code for it?

x = np.linspace(0,10,100)
y = np.sin(x)
plot(x,y)


I am working on the file "05_dataio_matplotlib/FileHandling.ipynb".
The first task is:
JavaScript Object Notation (JSON)
JSON (/ˈdʒeɪsən/ JAY-sən),[1] or JavaScript Object Notation, is an open standard format that uses human-readable text to transmit data objects consisting of attribute–value pairs. It is used primarily to transmit data between a server and web application, as an alternative to XML.
Although originally derived from the JavaScript scripting language, JSON is a language-independent data format. Code for parsing and generating JSON data is readily available in many programming languages.
https://en.wikipedia.org/wiki/JSON
Can you write Python code for it?

cat employees.json

import json

d = json.load( open('employees.json') )

d['employees'][1]


I am working on the file "05_dataio_matplotlib/FileHandling.ipynb".
The first task is:
Object serialization using pickle
Can you write Python code for it?

# the pickle class is used to serialize python variables (convert them to bytestrings)
import pickle


I am working on the file "05_dataio_matplotlib/FileHandling.ipynb".
The first task is:
Organizing files in folders
Can you write Python code for it?

# create a folder for the data files
import os

# get current working directory
work_path = os.getcwd()
print(work_path)

# define path for data files
data_path = os.path.join(work_path, 'data/')

# check if folder exists already
if not os.path.exists(data_path):
    os.mkdir(data_path)


I am working on the file "05_dataio_matplotlib/FileHandling.ipynb".
The first task is:
Parsing data
Can you write Python code for it?

with open('data.txt', 'r') as f:
    lines = f.readlines()
#print(lines)

data = {}

# iterate over all lines in the file
for line in lines:
    if line.startswith('#'): # skip comments
        continue
    left, right = line.split(':') # split splits a string at the occurence of the keyword
    data[ left.strip() ] = float(right) # strip removes leading and tailing spaces
print(data)


I am working on the file "05_dataio_matplotlib/FileHandling.ipynb".
The first task is:
how to save objects to a file
Can you write Python code for it?

with open('save.p', 'wb') as f:
    pickle.dump(d, f)

with open('save.p', 'wb') as f:
    pickle.dump(d, f)

with open('save.p', 'rb') as f:
    loaded_data = pickle.load(f)
print(loaded_data)


I am working on the file "05_dataio_matplotlib/FileHandling.ipynb".
The first task is:
reading from files
Can you write Python code for it?

# usage of with to open files is recommended in python
with open('new_file.txt', 'r') as f: # open file for reading
    content = f.read() # get the whole content of a file into a string

print(content)

with open('new_file.txt', 'r') as f: # open file for reading
    lines = f.readlines()
print(lines)
f.close()


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
A similar yet distint representation is provided by pcolormesh().
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
A simple plotting example. Maybe some variations:
Can you write Python code for it?

plot(xv,yv,'ro')

myplot=plot(xv,yv,'k--')
setp(myplot,linewidth=3.0,marker='+',markersize=30)


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Above commands enable pylab environment => direct access to numpy, scipy and matplotlib. The option 'inline' results in plot outputs to be directly embedded in the Notebook. If this causes problems, remove the option 'inline'.
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Analogously, you can use semilogy() and loglog() for plots with log y-axis and loglog plots.
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Anybody bar charts?
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Available settings for a plot can be found this way (does not work with 'inline' enabled):
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Check whether they are correlated using a scatter plot
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Contour plots are well suited for visualization of three-dimensional data sets:
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Especially for figures with multiple subpanels it may be advisable to increase the figure size somewhat. Do this by using function arguments in the figure() call:
Can you write Python code for it?

fig2=figure(figsize=(10,10))
subplot(2,1,1)
plot(xv,yv,'b-')
subplot(2,1,2)
plot(yv,xv,'ro')


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Exercise 2
generate two 1D-arrays $A$ and $B$ of size $N$ containing Gaussian random numbers
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Exercises for Numpy and Matplotlib
Exercise 1
Plot the following functions into individual subplots:
$f(x) = e^{-\alpha x^2},\quad g(x) = \alpha x^3-5x^2, \quad h(x) = \mathrm{erf}(x)$
with $\alpha \in {0.1,0.5,1}$
hint: use scipy for the definition of $\mathrm{erf}(x)$
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Figures can be saved in a number of output formats such as Postscript:
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Get a handle to current figure and close it:
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
In addition to the above 3D scatter plot, other plot types are supported, such as 3D surface plots:
Can you write Python code for it?

close('all'); fig=figure()
ax=Axes3D(fig)
xv=linspace(-10,10,100); yv=linspace(-10,10,100)
cx,cy=meshgrid(xv,yv)
cz=0.5*cx+exp(-cy**2)
tilt=ax.plot_surface(cx,cy,cz,linewidth=0, cmap=cm.jet)


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Let's add grid lines:
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Let's display the same data in contour representation:
Can you write Python code for it?

close('all'); fig=figure()
ax=Axes3D(fig)
ax.contour(cx,cy,cz,cstride=2,rstride=2, cmap=cm.jet)


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Let's do a figure with several subpanels:
Can you write Python code for it?

fig2=figure()
subplot(2,1,1)
plot(xv,yv,'b-')
subplot(2,1,2)
plot(yv,xv,'ro')

close(fig2)


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Let's switch to the Qt backend for 3D plotting
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Matplotlib also offers quiver() plots which are illustrated in the following example (taken from http://www.loria.fr/~rougier/teaching/matplotlib/):
Can you write Python code for it?

close('all')
n=8; X,Y=np.mgrid[0:n,0:n]
T=np.arctan2(Y-n/2.0,X-n/2.0)
R=10+np.sqrt((Y-n/2.0)**2+(X-n/2.0)**2)
U,V=R*np.cos(T),R*np.sin(T)
axes([0.025,0.025,0.95,0.95])
quiver(X,Y,U,V,R,alpha=.5)
quiver(X,Y,U,V, edgecolor='k', facecolor= 'None', linewidth=.5)
show()


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Plot their 2D density as a contour plot (hint: the density can be obtained from a histogram)
Plot the density as 3D surface plot
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Polar plots are also nicely illustrated on the very same homepage:
Can you write Python code for it?

close('all')
ax=axes([0.025,0.025,0.95,0.95],polar=True)
N=20; theta=np.arange(0.0,2*np.pi,2*np.pi/N)
radii=10*np.random.rand(N)
width=np.pi/4*np.random.rand(N)
bars=bar(theta,radii,width=width,bottom=0.0)
for r,bar in zip(radii,bars):
    bar.set_facecolor( cm.jet(r/10.))
    bar.set_alpha(0.5)
show()


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Style sheets
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
Try moving and rotating the (so far empty) plot in three dimensions. Once you have calmed down, let's populate the plot with some data:
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2015.ipynb".
The first task is:
to pick one of them, type e.g.
Can you write Python code for it?


I am working on the file "05_dataio_matplotlib/Matplotlib_2016.ipynb".
The first task is:
Above commands enable pylab environment => direct access to numpy, scipy and matplotlib. The option 'inline' results in plot outputs to be directly embedded in the Notebook. If this causes problems, remove the option 'inline'.
Can you write Python code for it?