I am working on the file "01_numpy_basics.ipynb".
The first task is:
Similarly, in numpy there is the arange function which instead generates a numpy.ndarray
Can you write Python code for it?

ra = np.arange(10)
print(ra)

print(type(ra))


I am working on the file "01_numpy_basics.ipynb".
The first task is:
So, why is it useful then?
So far the numpy.ndarray looks awefully much like a Python list (or nested list).
Why not simply use Python lists for computations instead of creating a new array type?
There are several reasons:

Python lists are very general.

They can contain any kind of object.
They are dynamically typed.
They do not support mathematical functions such as matrix and dot multiplications, etc.
Implementing such functions for Python lists would not be very efficient because of the dynamic typing.

Numpy arrays are statically typed and homogeneous.

The type of the elements is determined when array is created.

Numpy arrays are memory efficient.

Because of the static typing, fast implementation of mathematical functions such as multiplication and addition of numpy arrays can be implemented in a compiled language (C and Fortran is used).
Can you write Python code for it?

L = range(100000)

%timeit [i**2 for i in L]

a = np.arange(100000)

%timeit a**2  # This operation is called Broadcasting - more on this later!

%timeit [element**2 for element in a]


I am working on the file "01_numpy_basics.ipynb".
The first task is:
Spoiler Alert: very simple example of indexing in NumPy
Well...it works as expected, doesn't it?
Can you write Python code for it?


I am working on the file "01_numpy_basics.ipynb".
The first task is:
Terminology Assumption
In the numpy package the terminology used for vectors, matrices and higher-dimensional data sets is array.
Reference Documentation

On the web: http://docs.scipy.org/

Interactive help:
Can you write Python code for it?


I am working on the file "01_openlibs/tensorflow/00_resource/tf_examples/notebooks/0_Prerequisite/mnist_dataset_intro.ipynb".
The first task is:
A next_batch function that can iterate over the whole dataset and return only the desired fraction of the dataset samples (in order to save memory and avoid to load the entire dataset).
Can you write Python code for it?

# Get the next 64 images array and labels
batch_X, batch_Y = mnist.train.next_batch(64)


I am working on the file "01_openlibs/tensorflow/01_examples/0_prerequisite/mnist_dataset_intro.ipynb".
The first task is:
通过'next_batch'方法遍历整个数据集，只返回需要的部分数据（为了节省内存，避免加载整个数据集）
Can you write Python code for it?

# 获取下一组64张图像数组与分类列表
batch_X, batch_Y = mnist.train.next_batch(64)

print(batch_X.shape)
print(batch_Y.shape)


I am working on the file "01_openlibs/tensorflow/03_segmentfault/00_introduction.ipynb".
The first task is:
Build Graph阶段的代码，只是在Python内定义了Graph的结构，并不会真正执行。在Launch Graph阶段，所有的变量要先进行初始化。每个变量可以单独初始化，但这样做有些繁琐，所以TensorFlow提供了一个方便的函数global_variables_initializer()可以在graph中添加一个初始化所有变量的op。
When you launch the graph, variables have to be explicitly initialized before you can run Ops that use their value. All variables are automatically collected in the graph where they are created. By default, the constructor adds the new variable to the graph collection GraphKeys.GLOBAL_VARIABLES. The convenience function global_variables() returns the contents of that collection. The most common initialization pattern is to use the convenience function global_variables_initializer() to add an Op to the graph that initializes all the variables.
Launch Graph
Can you write Python code for it?

# Launch the graph.
sess = tf.Session()
sess.run(init)

for step in range(201):
    sess.run(train)


I am working on the file "01_openlibs/tensorflow/03_segmentfault/00_introduction.ipynb".
The first task is:
loss是最小二乘法需要的目标函数，是一个Tensor，具体的op不再赘述。
Can you write Python code for it?

optimizer = tf.train.GradientDescentOptimizer(0.5)
train = optimizer.minimize(loss)


I am working on the file "01_openlibs/tensorflow/03_segmentfault/00_introduction.ipynb".
The first task is:
tf.Variable是TensorFlow的一个类，是取值可变的Tensor，构造函数的第一个参数是初始值initial_value。
initial_value: A Tensor, or Python object convertible to a Tensor, which is the initial value for the Variable.
tf.zeros(shape, dtype=tf.float32, name=None)是一个op，用于生成取值全是0的Constant Value Tensor。
tf.random_uniform(shape, minval=0, maxval=None, dtype=tf.float32, seed=None, name=None)是一个op，用于生成服从uniform distribution的Random Tensor。
Can you write Python code for it?


I am working on the file "01_openlibs/tensorflow/03_segmentfault/00_introduction.ipynb".
The first task is:
下面我们来剖析一下关键代码。TensorFlow的代码往往由两个部分组成：

A construction phase, that assembles a graph
An execution phase that uses a session to execute ops in the graph.

Session是一个类，作用是把graph ops部署到Devices（CPUs/GPUs），并提供具体执行这些op的方法。
为什么要这么设计呢？考虑到Python运行性能较低，我们在执行numerical computing的时候，都会尽量使用非python语言编写的代码，比如使用NumPy这种预编译好的C代码来做矩阵运算。
在Python内部计算环境和外部计算环境（如NumPy）切换需要花费的时间称为overhead cost。对于一个简单运算，比如矩阵运算，从Python环境切换到Numpy，Numpy运算得到结果，再从Numpy切回Python，这个成本，比纯粹在Python内部做同类运算的成本要低很多。但是，一个复杂数值运算由多个基本运算组合而成，如果每个基本运算来一次这种环境切换，overhead cost就不可忽视了。为了减少来回的环境切换，TensorFlow的做法是，先在Python内定义好整个Graph，然后在Python外运行整个完整的Graph。因此TensorFlow的代码结构也就对应为两个阶段了。
Build Graph
Can you write Python code for it?

W = tf.Variable(tf.random_uniform([1], -1.0, 1.0))
b = tf.Variable(tf.zeros([1]))


I am working on the file "01_openlibs/tensorflow/03_segmentfault/00_introduction.ipynb".
The first task is:
这一步指定求解器，并设定求解器的最小化目标为损失。train代表了求解器执行一次的输出Tensor。这里我们使用了梯度下降求解器，每一步会对输入loss求一次梯度，然后将loss里Variable类型的Tensor按照梯度更新取值。
Can you write Python code for it?


I am working on the file "01_parsing.ipynb".
The first task is:
CODING TIME: Put parsing and indexing together into a single function..
Can you write Python code for it?


I am working on the file "01_parsing.ipynb".
The first task is:
DataFrames
People spend a lot of time reading code, especially their own code.
Lets do two things in using DataFrames: make our code more readable
and not reinvent the wheel (i.e. parsers). We have pride in the
code we write!
First an example of using DataFrames...
Can you write Python code for it?

np.random.seed = 1
df = pd.DataFrame(np.random.randint(0, 10, size=(6, 4)), columns=['A', 'B', 'C', 'D'])
df

df += 1
df

df.loc[:, 'A'] = [0, 0, 1, 1, 2, 2]
df

df.groupby('A')[['B', 'C', 'D']].apply(lambda f: f.sum())


I am working on the file "01_parsing.ipynb".
The first task is:
First approach
Write a function that reads an xyz trajectory file in. We are going to need to be able to separate numbers from atomic symbols; an XYZ trajectory file looks like:
nat [unit]
[first frame]
symbol1 x11 y11 z11
symbol2 x21 y21 z21
nat [unit]
[second frame]
symbol1 x12 y12 z12
symbol2 x22 y22 z22
Stuff in [ ] are optional (if units are absent, angstroms are assumed; a blank is included if no comments are present).
Here is an example file parser. All it does is read line by line and return a list of these lines.
Can you write Python code for it?

def skeleton_naive_xyz_parser(path):
    '''
    Simple xyz parser.
    '''
    # Read in file
    lines = None
    with open(path) as f:
        lines = f.readlines()
    # Process lines
    # ...
    # Return processed lines
    # ...
    return lines

lines = skeleton_naive_xyz_parser(xyz_path)
lines


I am working on the file "01_parsing.ipynb".
The first task is:
Lets attach a meaningful index
This is easy since we know the number of atoms and number of frames...
Can you write Python code for it?

df = pandas_xyz_parser(xyz_path)
df.index = pd.MultiIndex.from_product((range(nframe), range(nat)), names=['frame', 'atom'])
df


I am working on the file "01_parsing.ipynb".
The first task is:
Saving your work!
We did all of this work parsing our data, but this Python kernel won't be alive eternally so lets save
our data so that we can load it later (i.e. in the next notebook!).
We are going to create an HDF5 store to save our DataFrame(s) to disk.
HDF is a high performance, portable, binary data storage format designed with scientific data exchange in mind. Use it!
Also note that pandas has extensive IO functionality.
Can you write Python code for it?

xyz = parse(xyz_path, nframe, nat)
store = pd.HDFStore('xyz.hdf5', mode='w')
store.put('xyz', xyz)
store.close()


I am working on the file "01_parsing.ipynb".
The first task is:
Testing your functions is key
A couple of quick tests should suffice...though these barely make the cut...
Can you write Python code for it?

print(len(df) == nframe * nat)    # Make sure that we have the correct number of rows
print(df.dtypes)                  # Make sure that each column's type is correct


I am working on the file "01_particle_counting.ipynb".
The first task is:
Finally, we have gamma radiation - photons that set off single pixels, or two, three of four when they hit the boundaries between pixels:
Can you write Python code for it?

# An example of a tripixel gamma candidate.
gamma_cluster_1 = KlusterFinder(pixel_dict, 256, 256).getListOfKlusters()[25]

# Make the image!
make_kluster_image(gamma_cluster_1)


I am working on the file "01_particle_counting.ipynb".
The first task is:
It's fun to make the pictures of the clusters, but we can actually use code to do some important - but, dare we say, tedious - work for us. For example, how did you get on with the particle counting in Exercise 1? Our code makes tasks like this a lot easier...
Can you write Python code for it?

print("Number of clusters (particle) in the data frame: %d" % (len(klusters)))


I am working on the file "01_particle_counting.ipynb".
The first task is:
Notice too how all of the energy is centred in the middle of the cluster.
Secondly, we have beta radiation - electrons and positrons - that scatters through the silicon of the detector leaving curly tracks of pixels like in the following cluster:
Can you write Python code for it?

# An example of a beta cluster from the data.
beta_cluster_1 = KlusterFinder(pixel_dict, 256, 256).getListOfKlusters()[11]

# Make the image!
make_kluster_image(beta_cluster_1)


I am working on the file "01_particle_counting.ipynb".
The first task is:
Now, you should be able to make out three sorts of clusters - groups of pixels adjacent pixels - in the image, corresponding to the three types of radiation we'd expect to see from this measurement.
Firstly, there are the alphas. Alpha radiation consists of helium nuclei - two protons and two neutrons with an overall charge of 2+. These deposit large amounts of energy in the detector. To cut a long story short, this results in many pixels being activated over a large area in the image, resulting in "blobs"  like these:
Can you write Python code for it?

# Find the clusters - groups of adjacent pixels - in the data.
from cernatschool.kluster import KlusterFinder

# Get one of the alpha clusters.
alpha_cluster_1 = KlusterFinder(pixel_dict, 256, 256).getListOfKlusters()[4]

# Import a function for making an image of just one cluster.
from cernatschool.visualisation import make_kluster_image

# Make the image.
make_kluster_image(alpha_cluster_1)


I am working on the file "01_particle_counting.ipynb".
The first task is:
Of course, what we're really interested in is what the pixels look like as a whole so we can see the patterns and start to identify what has been detected. We've written some code to do this for you:
Can you write Python code for it?

# Import a function that extracts a pixel dictionary from the data frame.
from cernatschool.helpers import get_pixel_dictionary_from_dataframe

# Import a function that makes an image from a pixel dictionary.
from cernatschool.visualisation import make_frame_image

# (This line just allows us to show images as we go...)
%matplotlib inline

## The pixel dictionary - contains the pixel information in the format {X:C}.
pixel_dict = get_pixel_dictionary_from_dataframe(df000)

# Make the frame image from the pixel dictionary.
make_frame_image(pixel_dict)


I am working on the file "01_particle_counting.ipynb".
The first task is:
The DataFrame object provides a convenient wrapper for the pixel data read from the raw text file the detector produced. For example, we can find out how many pixels there are in total in the frame:
Can you write Python code for it?

print("There are %d pixels in the data frame." % (len(df000)))


I am working on the file "01_particle_counting.ipynb".
The first task is:
We can also combine our search criteria by using boolean operators in the loc method like so:
Can you write Python code for it?

## The minimum particle size.
min_size = 25

## The minimum density.
min_density = 0.5

# Find clusters that aren't on the edge, have a minimum size of 25 pixels and
# a minimum density of 0.8 using multiple conditions linked by AND & operators.
kdf.loc[(kdf['isedgekluster']==False ) & ((kdf['size'] >= min_size) & (kdf['density_uw'] >= min_density))]

# Exercise 3: Particle spotting
#
# Use the make_kluster_image function to examine these particles more closely.
#
# What do you notice?


I am working on the file "01_particle_counting.ipynb".
The first task is:
Which particles are on the edge of the frame?
Can you write Python code for it?


I am working on the file "01_python.ipynb".
The first task is:
Access
How do we access data in an array?
Can you write Python code for it?

my_array[0, 1]

my_array[:,0]

my_vec = numpy.array([[1], [2]])
print my_vec

numpy.dot(my_array, my_vec)
numpy.cross?

my_array * my_vec


I am working on the file "01_python.ipynb".
The first task is:
Constructors
Ways to make arrays in NumPy.
Can you write Python code for it?

my_array = numpy.array([[1, 2], [3, 4]])
print my_array

numpy.linspace(-1, 1, 10)

numpy.zeros([3, 3])

numpy.ones([2, 3, 2])

numpy.empty([2,3])


I am working on the file "01_python.ipynb".
The first task is:
Control Flow
if statements are the most basic unit of logic and allows us to conditionally operate on things.
Can you write Python code for it?

x = 4
if x > 5:
    print "x is greater than 5"
elif x < 5:
    print "x is less than 5"
else:
    print "x is equal to 5"


I am working on the file "01_python.ipynb".
The first task is:
In compound statements it can become more difficult to figure out where possible rounding might occur so be careful when you evaluate statements.
Can you write Python code for it?


I am working on the file "01_python.ipynb".
The first task is:
Note that we do not get what we expect from this expression as we expected from above.  What would we have to change to get this to work?
We could go back to change our initializations but we could also use the function float to force these values to be of float type:
Can you write Python code for it?

float(room_capacity - num_students) / float(room_capacity) * 100.0


I am working on the file "01_python.ipynb".
The first task is:
NumPy
The most important part of NumPy is the specification of an array object called an ndarray.  This object as its name suggests stores array like information in multiple dimensions.  These objects allow a programmer to access the data in a multitude of different ways as well as create common types of arrays and operate on these arrays easily.
Can you write Python code for it?


I am working on the file "01_python.ipynb".
The first task is:
Python also understands imaginary numbers:
Can you write Python code for it?


I am working on the file "01_python.ipynb".
The first task is:
Python returns the floor of the 1 / 2 because we gave it integers to divide.  It then interprets the result as also needing to be an integer.  If one of the numbers was a decimal number we would have a decimal number as a result (really these are floating point numbers float).
Can you write Python code for it?


I am working on the file "01_python.ipynb".
The first task is:
Some of the more advanced mathematical functions are stored in modules.  In order to use these functions we must first import them into our notebook and then use them.
Can you write Python code for it?

import math

math?

math.sqrt(2.0)

math.sin(math.pi / 2.0)

from math import *
sin(pi)


I am working on the file "01_python.ipynb".
The first task is:
Variables
Variables are defined and assigned to like many other languages.
Can you write Python code for it?

num_students = 80
room_capacity = 85
(room_capacity - num_students) / room_capacity * 100.0


I am working on the file "01_python.ipynb".
The first task is:
for allows us to repeat tasks over a range of values or objects.
Can you write Python code for it?

for i in range(5):
    print i

for i in range(3,7):
    print i

for animal in ['cat', 'dog', 'chinchilla']:
    print animal

for n in range(2, 10):
    is_prime = True
    for x in range(2, n):
        if n % x == 0:
            print n, 'equals', x, '*', n / x
            is_prime = False
            break
    if is_prime:
        print "%s is a prime number" % (n)


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
A list is defined with square brackets and list elements are separated by commas.
Whenever we want to access an element of a list we can do so by typing the list name and the index of the element in square brackets.
Can you write Python code for it?

inputs = [5, 9, 3, 2, 7, 8]

print(inputs[3])
print(inputs[0])
print(inputs[-1])  # returns the last element of the list


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
Built-in functions
Python already comes with many built-in functions. You can find a complete list here. One very useful function is the print function. It allows you to print stuff to the output. For example:
Can you write Python code for it?

calculation_result = 13 * 5

print(calculation_result)

# or maybe even nicer:
print('the calculation result is:', calculation_result)


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
Comments
Strings are not the only type of text in our code. Sometimes you want to add additional information to a human reader of your code but you don't want python to evaluate it. We can use the # sign to tell python that everything behind this sign in the same line is a comment and should not be evaluated. For example:
Can you write Python code for it?

5 + 7 # here we add 5 and 7
# 3 * 5 this calulation is not evaluated because it is inside the comment.


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
Here, we took the value of x, added 1 to it and then assigned the result back to the same name x. As a shortcut for such an operation we can use the += operator:
Can you write Python code for it?


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
I hope you aggree that using a function in this example makes the code clearer and easier to read. Now it is your turn!
Define a function called gravitational_force wich takes an input argument mass and returns the gravitational force.
Can you write Python code for it?

# Define your function here:

# If you defined the function correctly the following statment should
# return the gravitational force acting on a mass of 4 kg
gravitational_force(4)


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
If you fixed the code correctly, it should take the ball roughly 0.45 s to fall from a height of 1 meter before it reaches y = 0. Then, the ball continues to fall to negative y values. Again, this is not a very realistic behaviour of our ball. At y = 0 there is the floor of our room. It should perform an elastic collision and bounce up again. We somehow have to adjust our behaviour when our ball hits the floor.
Conditions
First, we have to get to know one more data type: a boolean. A boolean can only have two different states. It can either be True or False. We for example create a boolean if we compare things. for example:
Can you write Python code for it?


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
It is also possible to get slices of the complete list. The general notation is [begin:end:step]. For example:
Can you write Python code for it?

inputs = [5, 9, 3, 2, 7, 8]

print(inputs[:4]) # the first four list elements
print(inputs[1:4]) # elements from index 1 to (not including) index 4
print(inputs[::2]) # every second element


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
Let's say you want to calculate the gravitational force of some object with a mass of 4 kg.
Assign 9.81 to a variable called acceleration!
Assign 4 to a variable called mass!
Multiply both variables to calculate the gravitational force!
Note, that the = is similar to the = in math, but it is not quite the same. Python always evaluates code on the right of the equal sign and assigns it to the variable on the left. So we can do:
Can you write Python code for it?


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
Luckily we already banned this complicated calculation into the body of a function, so we don't have to deal with it each time we do the calculation. Still, the code is pretty long and contains lots of repetitions.
Lists can help use to make this more efficient. Instead of creating several variables input_1, input_2, input_3 ... it is more convenient to store all the values in a list:
Can you write Python code for it?


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
Obviously, this is not a finished game yet but hopefully it's clear how objects can help to write code for such an example.
Now your task is to define a class yourself.
* Define a class named Ball!
* Each ball should have the properties position, velocity, and radius that are set when calling the __init__ function. (Note, that there are two underscores in the front and at the end of the name)
* Create an instance of the class Ball called tennis_ball with a position [0, 1] a velocity [0, 0] and a radius 0.033
Can you write Python code for it?

# Define your class here

# Create an instance called tennis_ball here

# If you have done everything correctly the following should print
# the position, velocity and radius of the tennis ball.
print('the position of the tennis ball is:', tennis_ball.position)
print('the velocity of the tennis ball is:', tennis_ball.velocity)
print('the radius of the tennis ball is:' , tennis_ball.radius)


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
Ok. Now we have conveniently stored all our input values in a list. But how do we perform our "complicated calculation" with all these inputs? The easiest solution is to use a for loop. have a look at the following example:
Can you write Python code for it?

inputs = [5, 9, 3, 2, 7, 8]

for number in inputs:
    print(number)


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
Okay, now the properties of the ball are set. It has a position represented by a list with an x and y coordinate. You can also think of this list as a vector. So far, you have set the x value to 0 and the y value to 1. This means, the ball is floating in a height of 1 (meter). The ball's velocity is also a vector which, so far, only contains zeros.
A ball floating in a height of 1 meter is not very realistic. The ball should be able to move! We assume that there is no friction. The only thing that changes the velocity of the ball is the gravitational force. To make the ball move you have to:
1. Update the position of the ball. We will do that be adding the velocity multiplied with time step. This is not absolutely correct because the velocity is not constant during a step. However, if we keep the step size reasonably small we should be fine.
2. Update the velocity of the ball.
Fix the code below to make the ball move!
Can you write Python code for it?

class Ball:
    def __init__(self, start_position, start_velocity, radius):
        self.position = start_position
        self.velocity = start_velocity
        self.radius = radius

    # so for the move function doesn't change anything. Fix it!
    def move(self, time_step):
        self.position[0] += 0
        self.position[1] += 0

        self.velocity[0] += 0
        self.velocity[1] += 0

tennis_ball = Ball([0, 1], [0, 0], 0.033)
time_step = 0.04

for step in range(1, 15): # range is a built-in function which returns all number is a given range
    tennis_ball.move(time_step)
    print('time:', step*time_step, 'position:', tennis_ball.position)


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
So far, our functions had exactly one input argument and returned one output. It is also possible to have zero or multiple inputs and or outputs. For example:
Can you write Python code for it?

def add(a, b):
    return a + b

def return_5_anyway(): # note that you still need brackets even when your function takes no input
    return 5

return_5_anyway() # also wenn we call a function we allways have to use the brackets


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
Such booleans can be used to adjust the behaviour of our code in certain conditions by using the if-statement. All code in the body of an if-statement is only executed when the condition of the if statement evaluates to True. Have look at the following example:
Can you write Python code for it?

if 5 > 1:
    # this is executed because 5 > 1 is True
    print('5 > 1 is True')

if 5 < 3:
    # this is not executed because 5 < 3 is False
    print('I can write what I want. It will not be printed')


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
The same also works for +=, -=, *= and /=.
Assign 3 to a variable called x!
Multiply x with 2 and assign the result to x!
Return the result by typing x!
Variables can also hold data types other than numbers. For example they can store text. In programming a text piece is called a string. To tell python that something is a string and for example not a variable name, it has to be enclosed in quotes.
Execute the following example!
Can you write Python code for it?

my_first_string = 'Kartoffe'
my_second_string = 'lauf'

my_first_string + my_second_string * 2


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
This code is perfectly valid. However, if you code like this your code will soon be very messy. Whenever you start copying parts of your code, there is always a better way to structure your code. In the example above we could clean up the code by defining a function which does this complicated calculation for us. A function is typically something that: takes some input, does something with the input, and then returns something. For example:
Can you write Python code for it?

def square(number):
    squared = number * number
    return squared


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
We can also check if things are equal == or not equal !=
Can you write Python code for it?

print(5 == 5)  # note that we use == as = is already used for assignments in python
print(5 != 3)


I am working on the file "01_python_intro/python_intro.ipynb".
The first task is:
We can get access to even more useful functions by importing modules. For example, we can import the math module containing all kinds of mathematical functionality. After the import we can access functions in the module by typing name_of_the_module.function . For example:
Can you write Python code for it?


I am working on the file "01_python_tipos_basico/tipos_basicos.ipynb".
The first task is:
2.2 Números reales o de punto flotante.
La operatoria con números flotantes nos permite utilizar las operaciones matemáticas que bien conocemos sin problema. Para ingresar un número de punto flotante se debe ingresar su parte entera seguida por su parte decimal. Sin embargo, si queremos operar como número de punto flotante a un número entero se debe agregar un 0 luego del punto.
Veamos que nos entrega el computador en el caso de la división anterior con números de punto flotante:
Can you write Python code for it?


I am working on the file "01_python_tipos_basico/tipos_basicos.ipynb".
The first task is:
2.3 Números complejos
Python nos permite operar con números complejos a través de la sintaxis a+bj, donde j denota al número imaginario en el sistema. Las operaciones posibles sobre los numeros complejos son las mismas de los numeros reales. Veamos algunos ejemplos:
Can you write Python code for it?

(3 + 2j) + (- 2j) # Sigue siendo numero complejo aunque se cancele la parte imaginaria

(3 + 2j) - (12 + 5j)

(3-2j)*(-2j)

(12+5j)/(4-3j)


I am working on the file "01_python_tipos_basico/tipos_basicos.ipynb".
The first task is:
2.5 Texto o Strings
Los strings son las cadenas de texto que nos permiten, por ejemplo, imprimir información por pantalla. Al igual que los ejemplos de datos anteriores poseen características y operatorias propias que nos facilitan tremendamente la utilización de éstos. La definición de strings se puede realizarse con comillas simples ('), comillas dobles ("), triple comilla simple (''') o triple comilla doble ("""). El uso de strings en python es tan importante que hemos definido una sección especial para explicarlos y practicarlos: string en python.
Can you write Python code for it?

string_1 = "Ex Umbra in Solem"
print string_1

string_2 = 'Ble ble'
print string_2

string_3 = 'Ble ble'
print string_3

string_4 = 'Ble ble'
print string_4


I am working on the file "01_python_tipos_basico/tipos_basicos.ipynb".
The first task is:
Cabe destacar que python soporta el uso encadenado de expresiones lógicas de comparación:
Can you write Python code for it?


I am working on the file "01_python_tipos_basico/tipos_basicos.ipynb".
The first task is:
Desafío 3
Evalúe la siguiente expresión
$$ 1 + \sqrt{2} + \sqrt[3]{3} + \sqrt[4]{4} + \sqrt[5]{5}  + \sqrt[6]{6}$$
Pista: Recuerde que $\sqrt[n]{p} = p^{\frac{1}{n}}$.
Can you write Python code for it?


I am working on the file "01_python_tipos_basico/tipos_basicos.ipynb".
The first task is:
Desafío 4
Dado un cierto numero $x$, evalue si el número es múltiplo de 6 y 7 pero no de 8.
PISTA: Un número $n$ es múltiplo de $p$ si el modulo (resto de división entera) es cero. En python, esto es n%p==0
Can you write Python code for it?

x = 426
respuesta = True and True or False
print respuesta


I am working on the file "01_python_tipos_basico/tipos_basicos.ipynb".
The first task is:
Los numeros reales (o flotantes) tienen las mismas operaciones que los numeros enteros.
Los números reales no pueden representarse con precisión infinita en el computador, por lo que resulta necesario considerar que siempre existe un error de redondeo.
Desafío 2
Evalúe la siguiente expresión utilizando las operaciones de enteros:
$$ \frac{1 + \frac{2}{4+\frac{8}{16}}}{1 - \frac{1}{1+\frac{1}{2}}}$$
Pista: El resultado debería ser $4.33333333333$
Can you write Python code for it?

# INSERTAR CODIGO AQUI #
num = 1
den = 1
print num/den


I am working on the file "01_python_tipos_basico/tipos_basicos.ipynb".
The first task is:
Observación: En python 3 la división "tradicional" de enteros da un flotante. Esto si bien no es conceptualmente coherente (la operacion de 2 tipos deberia originar el mismo tipo de datos), es en la práctica lo que se busca y es "pragmaticamente" más coherente. Si se busca la división entera, es necesario utilizar //.
Desafío 1
Calcular el siguiente valor utilizando las operaciones anteriormente descritas:
$$ 1 + 2^2 - 3^3 + 4^4 - 5^5 + 6^6 -7^7 +8^8 -9^9 $$
Pista: El resultado deberia ser $-371423051$
Can you write Python code for it?


I am working on the file "01_python_tipos_basico/tipos_basicos.ipynb".
The first task is:
Si definimos un un número complejo $a$ podemos encontrar su parte real e imaginaria mediante la sintaxis "a.real" y "a.imag", mientras que su módulo puede ser determinado a través de la función "abs". Veamos un ejemplo de estas operatorias especiales:
Can you write Python code for it?

a=3+4j
print a.real
print a.imag
print abs(a)


I am working on the file "01a - Example of Phase-Plane Analysis.ipynb".
The first task is:
Analysis of the System: Numpy Meshgrid
Can you write Python code for it?

import numpy as np
from matplotlib import pyplot as plt
x1, x2 = np.meshgrid(np.linspace(-.5, .5, 10), np.linspace(-.5, .5, 10))
x1dot = x2
x2dot = -3 * x1 - x1 ** 2 - x2

plt.figure()
plt.quiver(x1, x2, x1dot, x2dot)
plt.show()

@widgets.interact (
    x_start=(-10.0, 10.0, 0.1),
    x_stop=(-10.0, 10.0, 0.1),
    y_start=(-10.0, 10.0, 0.1),
    y_stop=(-10.0, 10.0, 0.1),
    space=(10, 50)
)
def inter_plot(x_start, x_stop, y_start, y_stop, space):
    x1, x2 = np.meshgrid(np.linspace(x_start, x_stop, space), np.linspace(y_start, y_stop, space))
    x1dot = x2
    x2dot = -3 * x1 - x1 ** 2 - x2
    plt.figure()
    plt.quiver(x1, x2, x1dot, x2dot)
    plt.show()


I am working on the file "01a - Example of Phase-Plane Analysis.ipynb".
The first task is:
Next we find the determinate of $A_1 - \lambda I$
Can you write Python code for it?

λ_fun = sp.det(A_1 - λ * sp.eye(2))
λ_fun

sp.solve(λ_fun)


I am working on the file "01a - Example of Phase-Plane Analysis.ipynb".
The first task is:
Now we let $\dot{\underline{x}} = A_1 \, \underline{x}$.  Therefore A_1 would be:
Can you write Python code for it?


I am working on the file "01a_enacbe.ipynb".
The first task is:
Grafična rešitev
Rešitve enačbe si lahko predstavljamo grafično. Iščemo vrednosti $x$, pri katerih je leva stran enaka desni. Če narišemo graf leve in desne strani na isto sliko, so rešitve enačbe ravno x-koordinate presečišča obeh grafov. Za risanje grafov uporabimo knjižnico matplotlib. Graf funkcije narišemo tako, da funkcijo tabeliramo v veliko točkah. Da lažje računamo s tabelami, uporabimo tudi knjižnico numpy, ki je namenjena delu z vektorji in matrikami.
Can you write Python code for it?

import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
t = np.arange(-1,3,0.01) # zaporedje x-ov, v katerih bomo tabelirali funkcijo
leva_f = sym.lambdify(x,enacba.lhs) # lambdify iz leve strani enačbe naredi python funkcijo, ki jo uporabimo na t
desna_f = sym.lambdify(x,enacba.rhs) # podobno za desno stran (rhs - right hand side, lhs - left hand side)
plt.plot(t,leva_f(t)) # leva stran /funkcija leva_f deluje po komponentah seznama t
plt.plot(t,[desna_f(ti) for ti in t]) # funkcija desna_t je konstanta (število 3) in zato ne vrne seznama iste dolžine kot t
plt.ylim(0,5)
plt.plot(resitve,[leva_f(r) for r in resitve],'or')
plt.show()


I am working on the file "01a_enacbe.ipynb".
The first task is:
Od tu naprej postane precej komplicirano, kako rešitve programsko izluščiti iz zadnjega rezultata. Če nas zanimajo le rešitve, lahko zgornji postopek izpustimo in preprosto uporabimo funkcijo solve.
Can you write Python code for it?

# rešitve enačbe najlažje dobimo s funkcijo solve
resitve = sym.solve(enacba)
resitve


I am working on the file "01a_enacbe.ipynb".
The first task is:
Za začetek povsem sledimo korakom, ki smo jih naredili „na roke“. Povzamimo „algoritem“

vse člene damo na levo stran
enačbo pomnožimo z $x$
levo stran faktoriziramo
iz faktorjev preberemo rešitev
Can you write Python code for it?


I am working on the file "01b_neenacbe.ipynb".
The first task is:
Naloga
Simbolično in grafično rešite neenačbe:
$$ x^3-1>x;\quad\left|x+1\right| \le \left|2x-1\right|;\quad \sin(x)<\frac{\sqrt{3}}{2}\text{ na intervalu } [0,2\pi)$$
<< nazaj: enačbe ... naprej: kompleksna števila >>
Can you write Python code for it?

import disqus
%reload_ext disqus
%disqus matpy


I am working on the file "01b_neenacbe.ipynb".
The first task is:
Neenačba s trigonometričnimi funkcijami
Poskusimo rešiti še enačbe s trigonometričnimi funkcijami
\begin{equation}
\cos(x)\le\frac{1}{2}
\end{equation}
Can you write Python code for it?

nen3 = sym.cos(x)<=1/2
nen3

solve_univariate_inequality(nen3,x,relational=False)


I am working on the file "01b_neenacbe.ipynb".
The first task is:
Neenačba z absolutno vrednostjo
Funkcija solve_univariate_inequality nima težav niti z enačbami z absolutno vrednostjo. Poglejmo si rešitev enačbe
\begin{equation}
\left|x^2-2\right|\ge x
\end{equation}
Can you write Python code for it?

nen2 = abs(x**2 -2) >= x
solve_univariate_inequality(nen2, x, relational=False)

# grafična rešitev
leva_f = sym.lambdify(x,nen2.lhs)
t = np.linspace(-3,3) # zaporedje x-ov, v katerih bomo tabelirali funkcijo
plt.plot(t,[leva_f(ti) for ti in t])
plt.plot(t,sym.lambdify(x,nen2.rhs)(t))
plt.title("Rešitev neenačbe sta intervala, kjer je modra črta nad zeleno")
plt.plot([-3,1],[-3.9,-3.9],'-or')
plt.plot([2,3],[-3.9,-3.9],'-or')
plt.show()

# Prav tako ni težav, če je več absolutnih vrednosti
solve_univariate_inequality(abs(x-1) <= 2*abs(x)-2, x, relational=False)

# težave se pojavijo, če so absolutne vrednosti gnezdene
solve_univariate_inequality(abs(x-1) <= abs(2*abs(x)-2), x, relational=False)


I am working on the file "01b_neenacbe.ipynb".
The first task is:
Primeri
Reši neenačbo
\begin{equation}
x^2-2\ge x
\end{equation}
Za neenačbe z eno neznanko uporabimo splošno funkcijo solve_univariate_inequality, ki prepozna tip neeenačbe in uporabi primerno metodo. Ko definiramo spremenljivko x, posebej označimo, da gre za realno število.
Can you write Python code for it?

from sympy.solvers.inequalities import solve_univariate_inequality
from sympy import Symbol

x = Symbol('x', real=True) #  x simbol, ki je realno število
neenacba = x**2 -2 >= x
solve_univariate_inequality(neenacba,x)


I am working on the file "01b_neenacbe.ipynb".
The first task is:
Če želimo rešitev v obliki intervalov, ukazu solve_univariate_inequality dodamo parameter relational=False.
Can you write Python code for it?

solve_univariate_inequality(neenacba, x, relational=False) # rešitev neenačbe v obliki intervala

# grafična rešitev
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
t = np.linspace(-3,3) # zaporedje x-ov, v katerih bomo tabelirali funkcijo
leva_f = sym.lambdify(x,neenacba.lhs)
desna_f = sym.lambdify(x,neenacba.rhs)
plt.plot(t,leva_f(t))
plt.plot(t,desna_f(t))
plt.title("Rešitev neenačbe sta intervala, kjer je modra črta nad zeleno")
plt.plot([-3,-1],[-3.9,-3.9],'ro-')
plt.plot([2,3],[-3.9,-3.9],'ro-')
plt.show()


I am working on the file "01c_kompleksna.ipynb".
The first task is:
<< nazaj: neenačbe
Can you write Python code for it?


I am working on the file "01c_kompleksna.ipynb".
The first task is:
Enačbe s kompleksnimi števili
Reši enačbo
$$iz^2-\bar{z}=0$$
Uporabimo ukaz solve iz knjižnice sympy.
Can you write Python code for it?

from sympy import Symbol,Eq,conjugate,solve,im,re
z = Symbol('z')
solve(Eq(I*z**2-conjugate(z),0),z)


I am working on the file "01c_kompleksna.ipynb".
The first task is:
Faktorizacija polinoma
Reši enačbo
$$z^4 + 4 = 0,$$ nato pa razstavi polinom $z^4 + 4$ na dva kvadratna
faktorja z realnimi koeficienti.
Can you write Python code for it?

z = Symbol('z')
resitve = solve(z**4+4,z)
resitve

sym.prod([z-z0 for z0 in resitve])


I am working on the file "01c_kompleksna.ipynb".
The first task is:
Množica rešitev je podana implicitno z enačbo. Zato uporabimo contour iz knjižnice matplotlib.
Can you write Python code for it?

import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
# generiramo mrežo točk (x,y) v kateri bomo izračunali vrednosti leve strani enačbe
xi = np.linspace(-1,1)
X,Y = np.meshgrid(xi,xi) # pripravimo tabele, ki jih lahko direktno vstavimo v funkcijo
fun = sym.lambdify((x,y),leva) # izraz spremenimo v Python funkcijo
plt.contour(X,Y,fun(X,Y),levels=[0]) # narišemo množico točk, pri katerih je leva stran enaka 0 (levels=[0])
plt.axis("equal")
plt.title("množica je krožnica")


I am working on the file "01c_kompleksna.ipynb".
The first task is:
Polarni zapis
Kompleksno število lahko zapišemo tudi v polarni obliki
$$ z =r e^{i\phi} = r(\cos(\phi) + i\sin(\phi)).$$
Polarna oblika je primerna za množenje, potenciranje in korene.
Primer
Poišči vse korene
$$\sqrt[3]{-27+27i}.$$
Iščemo rešitve enačbe
$$ z^3 = -27+27i.$$
Can you write Python code for it?

r,f = sym.symbols("r,f",real=True)
z = r*sym.exp(I*f)
z**3

sym.expand_complex(z**3)

z0 = -27+27*I
abs(z0)

sym.arg(z0)


I am working on the file "01c_kompleksna.ipynb".
The first task is:
Polinom $z^4+4$ lahko zapišemo kot produkt linearnih faktorjev $(z-z_i)$, kjer so $z_i$ ničle polinoma. Rešitve nastopajo kot konjugirani pari $-1\pm i$ in $1\pm i$. Če skupaj zmnožimo linearne faktorje, ki pripadajo konjugiranemu paru, dobimo razcep na kvadratne faktorje z linearnimi koeficienti.
Can you write Python code for it?

izraz = sym.expand((z-resitve[0])*(z-resitve[1]))*sym.expand((z-resitve[2])*(z-resitve[3]))
izraz

# če kvadratna faktorja zmnožimo, dobimo prvotni polinom
sym.expand(izraz)

# razcep na kvadratne faktorje lahko izračunamo tudi s funkcijo factor
sym.factor(z**4+4)


I am working on the file "01c_kompleksna.ipynb".
The first task is:
Posebej izenačimo realni del in imaginarni del enačbe.
Can you write Python code for it?

enacbe = [Eq(re(lhs),0),Eq(im(lhs),0)]
enacbe

resitve = solve(enacbe,[x,y])
resitve # pari (x,y), ki rešijo sistem enačb

[r[0]+I*r[1] for r in resitve] # rešitve prvotne enačbe kot kompleksna števila


I am working on the file "01c_kompleksna.ipynb".
The first task is:
Primer
Skiciraj množico rešitev neenačbe
  $$ |z-3i+4|\le 4$$
v kompleksni ravnini.
Can you write Python code for it?

leva = abs(z-3*I+4)-4
leva

# generiramo mrežo točk (x,y) v kateri bomo izračunali vrednosti leve strani enačbe
xi = np.linspace(-8,0)
yi = np.linspace(-2,8)
X,Y = np.meshgrid(xi,yi) # pripravimo tabele, ki jih lahko direktno vstavimo v funkcijo
fun = sym.lambdify((x,y),leva) # izraz spremenimo v Python funkcijo
fun = np.vectorize(fun)        # Poskrbimo, da funkcija sprejme tudi tabele
plt.contourf(X,Y,fun(X,Y),levels=[0,-4]) # narišemo množico točk, pri katerih je leva stran med 0 in -4 (levels=[0,-4])
plt.axis("equal")
plt.title("množica je poln krog")


I am working on the file "01c_kompleksna.ipynb".
The first task is:
Primer
Zapiši število $$z=\frac{1-2i}{1+i}$$ v obliki $x+iy$ in izračunaj $|z|$.
Can you write Python code for it?

z = (1-2j)/(1+1j) #Python zlahka opravi z deljenjem kompleksnih števil, a računa približno s števili s plavajočo vejico
z

abs(z) # rezultat je podan v decimalnem zapisu


I am working on the file "01c_kompleksna.ipynb".
The first task is:
Če želimo, da Python pusti rezultat v obliki korenov in ulomkov, potrebujemo knjižnico sympy.
Can you write Python code for it?

import sympy as sym
from sympy import I #uvozimo imaginarno enoto
sym.init_printing() # lepši izpis formul

a = 1+2*I
a

(type(I),I**2)

z = (1-2*I)/(1+I)
z

abs(z)

from sympy import S,I
z = 1+I
type(z)

abs(z)


I am working on the file "01sepEvents.ipynb".
The first task is:
Let's start getting some more detailed data from the trips as well
Can you write Python code for it?

# df['favorite_count'] = df['favorite_count'].astype(np.int64)
# df['retweet_count'] = df['retweet_count'].astype(np.int64)
# df['text'] = df['text'].astype(str)
# df['id'] = df['id'].astype(np.int64)
# df['day_of_week'] = df['day_of_week'].astype(np.int64)
# df['day_of_month'] = df['day_of_month'].astype(np.int64)
# df['month'] = df['month'].astype(np.int64)
# df['time_of_day'] = df['time_of_day'].astype(np.int64)
df.loc[df["hashtags"]=='on',"hashtags"] = np.nan
df.convert_objects(convert_numeric=True)
df.dtypes
len(df)

# Pull out potential trains from both hashtags and text
df["topic_train"] = df["text"].apply(lambda x: check_train_id(x))
df["topic_train"] = df["topic_train"].apply(lambda x: str(x)[1:-1])
df["topic_train"].fillna(np.nan)
df.head(2)


I am working on the file "02 - Character Building.ipynb".
The first task is:
All that remains to be done is to get them into the usual form that bookworm expects (a list of tuples). We'll call this our list of characters.
Can you write Python code for it?

characters = [tuple([character + ' ']) for character in set(candidates)]


I am working on the file "02 - Character Building.ipynb".
The first task is:
I've also noticed that when dealing with characters in so many contexts, we'll sometimes get characters appearing as themselves and as posessives (ie Frodo and Frodo's). After stripping out punctuation we'll see Frodo and Frodos, so we just have to get rid of those words which end in s but are otherwise duplicates of other words already in the list.
Can you write Python code for it?

candidates = [c for c in candidates if not (c[-1] == 's' and c[:-1] in candidates)]
len(candidates)


I am working on the file "02 - Character Building.ipynb".
The first task is:
I've noticed that Spacy will occasionally tag non-title-case words as proper nouns, which is odd. Character names will typically be capitalised so we'll get rid of anything which isn't. We're relying on authors' 'proper' use of English language conventions here, which aren't always going to be perfect, but that's part of the joy of this project; we can still make a decent approximation of the truth despite the messiness of the initial data.
Can you write Python code for it?

candidates = [c for c in candidates if c.istitle()]
len(candidates)


I am working on the file "02 - Character Building.ipynb".
The first task is:
It makes more sense to start with a stripped down list of the unique words which appear in the text. That way, we only have to tag each word once. We should also get strip out all punctuation from the words before we start.
Can you write Python code for it?

remove_punctuation = lambda s: s.translate(str.maketrans('', '', string.punctuation+'’'))
words = [remove_punctuation(p) for p in book.split()]

unique_words = list(set(words))
len(unique_words)


I am working on the file "02 - Character Building.ipynb".
The first task is:
Name Extraction
The solution uses a little bit of creativity and a whole load of brute-force.
A sensible place to start when looking for character names is to go and find proper nouns. Part-of-speech tagging is a pretty well known NLP technique which does just that - it assigns a tag to each word in a string according to its type - noun, verb, adverb etc (you can read more about part-of-speech tagging here). Spacy has a pretty good POS tagger - having messed around with both Spacy and NLTK, I can say that with confidence that Spacy's is currently the more adept tagger. If we run this over the whole book and grab everything tagged as 'PROPN' we'll return a list of plausible proper nouns.
This process could take a while, given that we have this many individual words to interpret and tag:
Can you write Python code for it?


I am working on the file "02 - Character Building.ipynb".
The first task is:
Obviously this is a significantly longer list than what we usually deal with when we list characters manually, and it's a lot less precise (fore- and surnames won't be connected, nicknames won't be attached to their characters etc), but it's a pretty good trade-off for the lack of required contextual knowledge before starting the analysis. You'll also notice that a few 'names' aren't actually names - they're normal words that have somehow managed to slip through the gauntlet. As we move into the typical bookworm analysis process below, we'll see that this doesn't significantly affect the final results.
Standard Analysis & Comparisons to Manual Character Assignment
We can now throw our list of automatically collected characters and our usual sequences into the usual bookworm analysis process: finding connections between characters and sequences and then connections between characters and characters. Finally we'll set up a dataframe of interactions which can be passed to NetworkX to visualise (we'll go through the last bit of this process in much more detail in the next notebook.
Because we're now dealing with a much longer list of characters, the computation will take longer (the cooccurence matrix calculation scales quadratically with the number of characters). It should still run comfortably on a decent laptop.
Can you write Python code for it?

df = find_connections(sequences, characters)
cooccurence = calculate_cooccurence(df)


I am working on the file "02 - Character Building.ipynb".
The first task is:
We can import a few things to help visualise the graph that we've put together
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (12,9)

G = nx.from_pandas_dataframe(interaction_df,
                             source='source',
                             target='target')

nx.draw(G, with_labels=True)


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
Above, a few things are interesting to notice:
- When comparing the same corresponding data for each website, we can clearly distinguish a strong correlation which is something we can expect.
- There is a negative correlation between the rankings and every other feature (except the # Students) which is logical since the rankings are the only caracteristics where lower is better.
- There is a higher (absolute) correlation between the ranking of TopUniversity and the number of students than for Times Higher Education. We could guess from that that TopUniversity's criterias for the ranking give more importance to the size of the university than Times Higher Education's criterias.
- There is an (absolute) correlation between the ratios and the rankings (higher ratios would have better ranks on average).
- The number of faculty members is correlated with the number of students. Which is logic because it is not desirable to have a university with a lot of faculty members but very few students and inversly.
- It is good to note that both rankings seem not to always agree on the number of students, faculty members and international students as the correlation is not of 1.
Note that by 'absolute' correlation we mean the absolute value of the correlation
Best university
Basic method
To find the best university according to both ranking, we could use the following method: We take the mean of both ranking and the university with the smallest mean is the best university. With this method we try to find the university closest to the top in both rankings.
Can you write Python code for it?

df_all['Rank mean'] = (df_all['Rank times'] + df_all['Rank topuni'])/2.0
best = df_all.sort_values(by=['Rank mean'], ascending=True).iloc[0]

print("The best university according to both ranking is " + best.name + " with a mean rank score of {}".format(best["Rank mean"]))

del df_all['Rank mean']


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
Alternative method
What we could do, is extract the principal components of the data (A PCA, basically) and project the points on the main component.
Can you write Python code for it?

pca = PCA(n_components=2)
# Creates pairs of ranks for each university
ranks = np.array([df_all[['Rank times']].values.flatten(), df_all[['Rank topuni']].values.flatten()]).T
# Tells the PCA what the data is and then get the data in its new basis
ranks_new_basis = pca.fit(ranks).transform(ranks)
# Get the ratio of importance for each component
pca_score = pca.explained_variance_ratio_
# Get the components
V = pca.components_
# Multiplies the components by their ratio of importance compared to the least important component
ranks_pca_axis_x, ranks_pca_axis_y = V.T * pca_score / pca_score.min()


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
And then we use regex on every university name.
Can you write Python code for it?

regex1 = r'\([^()]*\)'
regex2 = r"(the)|(university)|(of)|(de)|(-)|( )|(’)|(')|\."
p = re.compile('('+regex1+'|'+regex2+')')

for df in [df1, df2]:
    df['University_regex'] = df.index.values
    # We first take care of the case
    df['University_regex'] = df['University_regex'].apply(lambda x: x.casefold())
    # Then we remove useless words, spaces, parentheses, their content and so on
    df['University_regex'] = df['University_regex'].apply(lambda x: str(re.sub(p, '', x)))
    # We then normalize the text to remove accents
    df['University_regex'] = df['University_regex'].apply(lambda x: unicodedata.normalize("NFKD", x))
    # And swith the encoding to utf-8
    df['University_regex'] = df['University_regex'].apply(lambda x: x.rstrip().encode('ascii', errors='ignore')
                                                                     .decode('utf-8'))


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
Best university according to both ranking :
Can you write Python code for it?

print("Best university according to PCA is : " + df_all.sort_values(by='Rank total').iloc[0].name)


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
English talking countries are more international (in term of international students ratio) in general and also small european countries like Luxembourg (really impressive how high), Austria and Switzerland.
Let's find the best region according to 1) ratio between faculty members and students and 2) ratio between local and international students. Previous assumptions about what is the best ratio still hold.
Per region ratio
Ratio between faculty members and students
Can you write Python code for it?

df2_region = df2.groupby(['Region']).agg(np.mean)

region_ratio_fac_stu(df2_region)


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
In Denmark there are around 6 students for a staff member. It's clearly above the average.
Ratio between local and international students
Can you write Python code for it?


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
In part 1 (top universities) we will create functions to compute and plot the different required ratios that we will reuse in part 2.
To scrap the datas (for both website) we used postman and saw there were a json string we could scrap and parse to get the ranking. For www.topuniversities.com we had to scrap the university specific page to get some more complete informations such as the number of students.
www.topuniversities.com
The ranking is broken and contains only the top 199 universities even though the last university has rank 200, if we want to have 200 universities we should include the university ranked 201, as we want the top 200, we will take only the 199 first universities.
Can you write Python code for it?

r = requests.get("https://www.topuniversities.com/sites/default/files/qs-rankings-data/357051.txt")
json_dict = r.json()
json_dict = json_dict['data']
json_dict = json_dict[:199]

for e in json_dict:
    #Delete useless informations
    del e['nid']
    del e['logo']
    del e['core_id']
    del e['score']
    del e['cc']
    del e['stars']
    del e['guide']

    #Get the university specific url
    r = requests.get("https://www.topuniversities.com/" + e['url'])
    soup = BeautifulSoup(r.text, 'html.parser')

    #Get faculty members numbers
    scrap_list = soup.find_all('div', class_='faculty-main wrapper col-md-4')
    if(len(scrap_list) == 0): #If no informations about the faculty members is given
        e['# Faculty members'] = 0
        e['# Int. faculty members'] = 0
    else:
        n_list = scrap_list[0].find_all(class_="number")
        if(len(n_list) == 2):
            e['# Faculty members'] = int(re.sub(',', '', n_list[0].text.strip()))
            e['# Int. faculty members'] = int(re.sub(',', '', n_list[1].text.strip()))
        else: #If they don't have international faculty member
            e['# Faculty members'] = int(re.sub(',', '', n_list[0].text.strip()))
            e['# Int. faculty members'] = 0

    #Get students numbers
    n_list = soup.find_all('div', class_='students-main wrapper col-md-4')
    if(len(n_list) != 0):
        e['# Students'] = int(re.sub(',', '', n_list[0].find_all(class_="number")[0].text.strip()))
    else:
        e['# Students'] = 0

    n_list = soup.find_all('div', class_='int-students-main wrapper col-md-4')
    if(len(n_list) != 0):
        e['# Int. students'] = int(re.sub(',', '', n_list[0].find_all(class_="number")[0].text.strip()))
    else:
        e['# Int. students'] = 0

    #We don't need the url anymore, we can delete it
    del e['url']

df1 = pd.DataFrame(json_dict)

df1=df1.rename(columns = {'rank_display':'Rank', 'title' : 'University', 'country' : 'Country', 'region' : 'Region'})
df1["Rank"] =...


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
Oceania is clearly the leader here.
www.timeshighereducation.com
Scraping
Using Postman when browsing the page with the ranking, I saw that a request for a json file was made. It actually contains all the information we need in a single file.
Helper to get the region way more easily
Can you write Python code for it?

countries = dict(df1[['Country', 'Region']].drop_duplicates().values.tolist())
countries['Luxembourg'] = 'Europe'


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
Once we have the principle component, we can use it to transform our data and change its basis from (rank times, rank topuni) to (main component, second component)
Can you write Python code for it?

fig = plt.figure(figsize=(10, 10))
ax = fig.add_subplot(111)
plt.scatter(ranks_new_basis.T[0], ranks_new_basis.T[1])

# Creates main component arrow
plt.plot([-125, 150], [0, 0], c='g', label='Main component')
plt.plot([140, 150], [4, 0], c='g')
plt.plot([140, 150], [-4, 0], c='g')

plt.xlabel('Main component')
plt.ylabel('Second component')
plt.title('Universities ranking in new basis')
plt.legend()
plt.show()


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
Once we know that the data is ready to be merged, we apply an inner merge. We chose to use an inner merge instead of an outer merge because it makes more sense to only keep universities present in both rankings.
Can you write Python code for it?

df_merged = df2.reset_index().merge(df1, how='inner', on='University_regex').set_index('University')


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
Per university ratio
Ratio between faculty members and students
Best university according to ratio between faculty members and students. We assume that a bigger ratio is better. Even though it might not be desirable, we assume it is the case for this exercise.
We plot the bar charts only for the first 30 top universites (according to the ratio) to get an idea of the trend.
Can you write Python code for it?

def uni_ratio_fac_stu(df):
    #Compute ratio
    df['Ratio Faculty members'] = df['# Faculty members']/df['# Students']

    #Sort the rows according to ratio and locate the top 30 ones
    df_sorted_ratios = df.sort_values(by=['Ratio Faculty members'], ascending=False).head(30)

    #Plot the bar chart and print the best one
    fig, ax = plt.subplots(figsize=(20,10))

    plot = sns.barplot(y=df_sorted_ratios['Ratio Faculty members'], x=df_sorted_ratios.index, ax=ax)
    plot.set_xticklabels(plot.get_xticklabels(), rotation=90)
    print("Best university according to ratio between faculty members and students is : " + df_sorted_ratios.iloc[0].name)

    #Delete the ratio column of the dataframe
    del df_sorted_ratios

uni_ratio_fac_stu(df1)


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
Plot of the correlation
Can you write Python code for it?

fig, ax = plt.subplots(figsize=(10,10))
sns.heatmap(df_all.corr(), ax=ax, square=True)


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
Ratio between local and international students
Let's find the best region according to the ratio between local and international students. Previous assumptions about what is the best ratio still hold.
Can you write Python code for it?

def region_ratio_int_stu(df):
    #Compute the ratios
    df['Ratio'] = df['# Int. students']/df['# Students']

    #Sort the rows according to ratios
    sorted_df = df.sort_values(by=['Ratio'], ascending=False)

    #Plot the bar chart and print the best one
    fig, ax = plt.subplots(figsize=(20,10))

    plot = sns.barplot(y=sorted_df.Ratio, x=sorted_df.index, ax=ax)
    plot.set_xticklabels(plot.get_xticklabels(), rotation=50)

    print("Best region according to ratio between local and international students is : " + sorted_df.iloc[0].name)

    #Delete datas we don't need anymore
    del df
    del sorted_df

region_ratio_int_stu(df1_region)


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
Same observations as for the previous bar charts. It is good to note that the universities and their order are not the same as the previous ones.
Per country ratio
Ratio between faculty members and students
Let's find the best country according to the ratio between faculty members and students. Previous assumptions about what is the best ratio still hold.
Can you write Python code for it?

def country_ratio_fac_stu(df):
    #Group by country and compute the ratios
    df['Ratio'] = df['# Faculty members']/df['# Students']

    #Sort the rows according to ratios
    sorted_df = df.sort_values(by=['Ratio'], ascending=False)

    #Plot the bar chart and print the best one
    fig, ax = plt.subplots(figsize=(20,10))

    plot = sns.barplot(y=sorted_df.Ratio, x=sorted_df.index, ax=ax)
    plot.set_xticklabels(plot.get_xticklabels(), rotation=50)
    print("Best country according to ratio between faculty members and students is : " + sorted_df.iloc[0].name)

    #Delete datas we don't need anymore
    del df['Ratio']
    del sorted_df

df1_country = df1.groupby(['Country']).agg(np.mean)

country_ratio_fac_stu(df1_country)


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
This bar chart is a bit different than the previous ones. Indeed the first ratios decrease way faster than the last ones but we could still use a combintion of two linear functions to approximate the decrease.
Per region ratio
Ratio between faculty members and students
Let's find the best region according to the ratio between faculty members and students. Previous assumptions about what is the best ratio still hold.
Can you write Python code for it?

def region_ratio_fac_stu(df):
    #Group by region and compute the ratios
    df['Ratio'] = df['# Faculty members']/df['# Students']

    #Sort the rows according to ratios
    sorted_df = df.sort_values(by=['Ratio'], ascending=False)

    #Plot the bar chart and print the best one
    fig, ax = plt.subplots(figsize=(20,10))

    df = df.groupby(['Region']).agg(sum)
    plot = sns.barplot(y=sorted_df.Ratio, x=sorted_df.index, ax=ax)
    plot.set_xticklabels(plot.get_xticklabels(), rotation=50)

    print("Best region according to ratio between faculty members and students is : " + sorted_df.iloc[0].name)

    #Delete datas we don't need anymore
    del df['Ratio']

df1_region = df1.groupby(['Region']).agg(np.mean)

region_ratio_fac_stu(df1_region)


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
URL containing the requested data in json
Can you write Python code for it?

timeshighereducation_url = "https://www.timeshighereducation.com/sites/default/files/the_data_rankings/world_university_rankings_2018_limit0_369a9045a203e176392b9fb8f8c1cb2a.json"

ranking_brute = requests.get(timeshighereducation_url).json()['data']

infos_to_keep = ["rank", "name", "location", "stats_number_students", "stats_pc_intl_students", "stats_student_staff_ratio"]
column_names =  ["Rank", "University", "Country", "# Students", "% Int. students", "% Faculty members"]

# creating dataframe
df2 = pd.DataFrame(ranking_brute[:200], index=range(1, 201), columns=infos_to_keep)

# more expressive column names
df2.columns = column_names

df2["Rank"] = df2["Rank"].apply(lambda x: int(re.sub("[^0-9]", "", x)))

# string to numerical values
df2["# Students"] = pd.to_numeric(df2["# Students"].map(lambda x: x.replace(",", "")))
df2["% Int. students"] = pd.to_numeric(df2["% Int. students"].map(lambda x: x.replace("%", ".")))
df2["% Faculty members"] = pd.to_numeric(df2["% Faculty members"])

# adding number of international students
df2["# Int. students"] = (df2["# Students"] * df2["% Int. students"] / 100).map(round)

# adding number of faculty members
df2["# Faculty members"] = (df2["# Students"] / df2["% Faculty members"]).map(round)

# Changing Russian federation to Russia
df2['Country'].replace('Russian Federation', 'Russia', inplace=True)

# adding regions
df2["Region"] = df2["Country"].map(lambda c: countries[c])

# Usign meaningful index
df2.index = df2['University']
del df2['University']

# Deleting % Int. students because it is not needed anymore
del df2['% Int. students']
del df2["% Faculty members"]


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
We can see that LSE school has 70% of international students which is really impressive.
Per country ratio
Ratio between faculty members and students
Let's find the best country according to 1) ratio between faculty members and students and 2) ratio between local and international students. Previous assumptions about what is the best ratio still hold.
Can you write Python code for it?

df2_country = df2.groupby(['Country']).agg(np.mean)

country_ratio_fac_stu(df2_country)


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
We now add this new rank in the DataFrame and show the 20 first universities in our new ranking system.
Can you write Python code for it?


I am working on the file "02 - Data from the Web/homework02.ipynb".
The first task is:
We then plotted the different Universities according to their respective ranking and showed the first two main component of the data (green and red).
Can you write Python code for it?

starting_point = (100, 100)

fig = plt.figure(figsize=(10, 10))
ax = fig.add_subplot(111)
plt.title('Universities ranking')
plt.plot(df_all[['Rank times']].values.flatten(), df_all[['Rank topuni']].values.flatten(), '.')
plt.plot([starting_point[0], starting_point[0] + ranks_pca_axis_x[0]*20], [starting_point[1], starting_point[1] + ranks_pca_axis_y[0]*20], label='Main component')
plt.plot([starting_point[0], starting_point[0] + ranks_pca_axis_x[1]*20], [starting_point[1], starting_point[1] + ranks_pca_axis_y[1]*20], label='Second component')
plt.xlabel('Ranking according to Times')
plt.ylabel('Ranking according to Topuni')
plt.legend()
plt.show()


I am working on the file "02 - Data structures.ipynb".
The first task is:

    EXERCISE: Calculate the population numbers relative to Belgium

    EXERCISE: Calculate the population density for each country and add this as a new column to the dataframe.

Some other useful methods
Sorting the rows of the DataFrame according to the values in a column:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
Alignment!
Only, pay attention to alignment: operations between series will align on the index:
Can you write Python code for it?

s1 = population[['Belgium', 'France']]
s2 = population[['France', 'Germany']]

s1

s2

s1 + s2


I am working on the file "02 - Data structures.ipynb".
The first task is:
Alignment! (unlike numpy)
Only, pay attention to alignment: operations between series will align on the index:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
An overview of that information can be given with the info() method:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
And we can do things like sorting the items in the array, and indexing to take the first two rows:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
Are young people more likely to survive?
Can you write Python code for it?

df['Survived'].sum() / df['Survived'].count()

df25 = df[df['Age'] <= 25]
df25['Survived'].sum() / len(df25['Survived'])


I am working on the file "02 - Data structures.ipynb".
The first task is:
As you play around with DataFrames, you'll notice that many operations which work on NumPy arrays will also work on dataframes.
For example there's arithmetic. Let's compute density of each country:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
Attributes of a Series: index and values
The series has a built-in concept of an index, which by default is the numbers 0 through N - 1
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
Attributes of the DataFrame
A DataFrame has besides a index attribute, also a columns attribute:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
Data structures
Pandas does this through two fundamental object types, both built upon NumPy arrays: the Series object, and the DataFrame object.
Series
A Series is a basic holder for one-dimensional labeled data. It can be created much as a NumPy array is created:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
DataFrames: Multi-dimensional Data
A DataFrame is a tablular data structure (multi-dimensional object to hold labeled data) comprised of rows and columns, akin to a spreadsheet, database table, or R's data.frame object. You can think of it as multiple Series object which share the same index.
<img src="img/dataframe.png" width=110%>
One of the most common ways of creating a dataframe is from a dictionary of arrays or lists.
Note that in the IPython notebook, the dataframe will display in a rich HTML view:
Can you write Python code for it?

data = {'country': ['Belgium', 'France', 'Germany', 'Netherlands', 'United Kingdom'],
        'population': [11.3, 64.3, 81.3, 16.9, 64.9],
        'area': [30510, 671308, 357050, 41526, 244820],
        'capital': ['Brussels', 'Paris', 'Berlin', 'Amsterdam', 'London']}
countries = pd.DataFrame(data)
countries


I am working on the file "02 - Data structures.ipynb".
The first task is:
Fancy indexing, like indexing with a list or boolean indexing:
Can you write Python code for it?

population[['France', 'Netherlands']]

population[population > 20]


I am working on the file "02 - Data structures.ipynb".
The first task is:
How does the survival rate of the passengers differ between sexes?
Can you write Python code for it?

df.groupby('Sex')[['Survived']].aggregate(lambda x: x.sum() / len(x))


I am working on the file "02 - Data structures.ipynb".
The first task is:
However, for this dataset, it does not say that much:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
If we don't like what the index looks like, we can reset it and set one of our columns:
Can you write Python code for it?

countries = countries.set_index('country')
countries


I am working on the file "02 - Data structures.ipynb".
The first task is:
In this way, a Series object can be thought of as similar to an ordered dictionary mapping one typed value to another typed value.
In fact, it's possible to construct a series directly from a Python dictionary:
Can you write Python code for it?

pop_dict = {'Germany': 81.3,
            'Belgium': 11.3,
            'France': 64.3,
            'United Kingdom': 64.9,
            'Netherlands': 16.9}
population = pd.Series(pop_dict)
population


I am working on the file "02 - Data structures.ipynb".
The first task is:
Many things we have seen for numpy, can also be used with pandas objects.
Slicing:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
One useful method to use is the describe method, which computes summary statistics for each column:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
Or how does it differ between the different classes?
Can you write Python code for it?

df.groupby('Pclass')['Survived'].aggregate(lambda x: x.sum() / len(x)).plot(kind='bar')


I am working on the file "02 - Data structures.ipynb".
The first task is:
Starting from reading this dataset, to answering questions about this data in a few lines of code:
What is the age distribution of the passengers?
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
Tabular data
Can you write Python code for it?

df = pd.read_csv("data/titanic.csv")

df.head()


I am working on the file "02 - Data structures.ipynb".
The first task is:
The plot method can be used to quickly visualize the data in different ways:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
To access a Series representing a column in the data, use typical indexing syntax:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
To check the data types of the different columns:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
Unlike the NumPy array, though, this index can be something other than integers:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
We can index the populations like a dict as expected:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
You can access the underlying numpy array representation with the .values attribute:
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
You can play with the kind keyword: 'line', 'bar', 'hist', 'density', 'area', 'pie', 'scatter', 'hexbin'
Importing and exporting data
A wide range of input/output formats are natively supported by pandas:

CSV, text
SQL database
Excel
HDF5
json
html
pickle
...
Can you write Python code for it?


I am working on the file "02 - Data structures.ipynb".
The first task is:
but with the power of numpy arrays:
Can you write Python code for it?


I am working on the file "02 - Input Output (IO) - Numpy, Scipy, h5py.ipynb".
The first task is:
After saving this data, check the file structure:
Can you write Python code for it?

def print_attrs(name, obj): # Function that prints the name and object
    print(name)
    print(obj)

with h5py.File("01-data_write.hdf5", 'r') as file_obj:
    file_obj.visititems(print_attrs) # Use .visititems to get info

with h5py.File("01-data_write.hdf5", 'r') as file_obj:
    print(file_obj["/Numbers/CSV"].name)
    print(file_obj["/Numbers/CSV"].value)

    print(file_obj["/Numbers/MAT"].name)
    print(file_obj["/Numbers/MAT"].value)

    print(file_obj["/Text/lorem"].name)
    print(file_obj["/Text/lorem"].value)


I am working on the file "02 - Input Output (IO) - Numpy, Scipy, h5py.ipynb".
The first task is:
Binary Files:
Binary files store the same information as text or csv, but do so directly in bytes, rather than using ascii to encode. They have the advantage of being faster to read and smaller in size, but are not readily readable by a typical text editor (notepad, vim, sublime, etc).
Note: be careful to avoid numpy.fromfile and numpy.tofile as they are not platform independent!
Can you write Python code for it?

# Read in the csv from the previous step:
file_contents = np.loadtxt(open('01-simpledata_write.csv'), delimiter=",")
print(file_contents)

# Save as a binary file:
np.savetxt('01-simpledata_write.bin', file_contents_write*2) # Note the lack of demiliter
file_contents = np.loadtxt('01-simpledata_write.bin')

# The following is not recommended, as it is platform dependent:
#np.ndarray.tofile(file_contents_write, '01-simpledata_write.bin')
#file_contents = np.fromfile('01-simpledata_write.bin')

print(file_contents)


I am working on the file "02 - Input Output (IO) - Numpy, Scipy, h5py.ipynb".
The first task is:
For coinvenience, it's possible to print all of the information using .visititems:
Can you write Python code for it?

def print_attrs(name, obj):
    print(name)
    if isinstance(obj, h5py.Group):
        print(obj)
    if isinstance(obj, h5py.Dataset):
        print(obj.value)

with h5py.File("01-data_write.hdf5", 'r') as file_obj:
    file_obj.visititems(print_attrs)


I am working on the file "02 - Input Output (IO) - Numpy, Scipy, h5py.ipynb".
The first task is:
From standard input/keyboard:
The import of simple text files can be performed directly in python via:
Can you write Python code for it?

kb_contents = 'Lorem ipsum dolor sit amet, consectetur adipiscing elit.'
print(kb_contents)


I am working on the file "02 - Input Output (IO) - Numpy, Scipy, h5py.ipynb".
The first task is:
HDF5 files
HDF5 or Hierarchical Data Format provides a file format that has a much greater amount of flexibility at the cost of a bit more complexity. HDF5 is ideal when there would otherwise have been many small files. There are two main objects:
- Groups: folder-like containers that work like Python dictionaries
- Datasets: NumPy-like arrays
Can you write Python code for it?

import h5py

# Load csv data:
data_csv = np.loadtxt(open('01-simpledata_write.csv'), delimiter=",")

# Load mat data:
data_mat = sio.loadmat('01-simplemat_write.mat')['testvar_write']

# Load text data:
with open('01-simpletext.txt','r') as file_obj:
    data_txt = file_obj.read()

# Create a h5py file object:
with h5py.File("01-data_write.hdf5", "w") as file_obj:
    # Use file_obj to create data sets

    # Create a dataset object and assign the values from data:
    dataset1 = file_obj.create_dataset("data", data = data_csv)


I am working on the file "02 - Input Output (IO) - Numpy, Scipy, h5py.ipynb".
The first task is:
The "Hierarchical" part of the HDF5 file format provides groups, which act like Python dictionaries or 'folders' for the various Datasets.
Can you write Python code for it?

# Open the same h5py file object:
with h5py.File("01-data_write.hdf5", "w") as file_obj:

    # Create a group object, and create datasets underneath it:
    grp_nums = file_obj.create_group("Numbers")
    dataset_csv = grp_nums.create_dataset("CSV", data=data_csv)
    dataset_mat = grp_nums.create_dataset("MAT", data=data_mat)

    # Create a second group object, and create datasets underneath it:
    grp_txt = file_obj.create_group("Text")
    txt_hf5 = np.asarray(data_txt, dtype="S") # Convert to NumPy S dtype:
    dataset_txt = grp_txt.create_dataset("lorem", data=txt_hf5)


I am working on the file "02 - Interactivity.ipynb".
The first task is:
At this point, the UI controls work just like they would if interact had been used. You can manipulate them interactively and the function will be called. However, the widget instance returned by interactive also give you access to the current keyword arguments and return value of the underlying Python function.
Here are the current keyword arguments. If you rerun this cell after manipulating the sliders, the values will have changed.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
Basic interact
At the most basic level, interact autogenerates UI controls for function arguments, and then calls the function with those arguments when you manipulate the controls interactively. To use interact, you need to define a function that you want to explore. Here is a function that prints its only argument x.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
Dropdown menus can be produced by passing a tuple of strings. In this case, the strings are both used as the names in the dropdown menu UI and passed to the underlying Python function.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
Exercises
String sorting

Write a sort_string function that takes a string as its input and prints a new string consisting of the original one, sorted.
Add a reverse keyword argument with a default of False to allow for the sorting to be done in reverse.
Then, use interact to create a user interface for exploring your sort_string function.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
Fixing arguments using fixed
There are times when you may want to explore a function using interact, but fix one or more of its arguments to specific values. This can be accomplished by wrapping values with the fixed function.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
If a 3-tuple of integers is passed (min,max,step) the step size can also be set.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
If you pass a string, interact will generate a text area.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
If you want a dropdown menu that passes non-string values to the Python function, you can pass a dictionary. The keys in the dictionary are used for the names in the dropdown menu UI and the values are the arguments that are passed to the underlying Python function.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
Interactive Plots
Interact can also be used with almost any function with supported parameter types, including those that output matplotlib plots.  In the following example, a function that plots the sum of two sine waves is interacted with.
Can you write Python code for it?

@interact(frequency1=20., frequency2=21.)
def plot_waves(frequency1, frequency2):
    x = np.linspace(0, 3*np.pi, 500)
    plt.plot(x, np.sin(x*frequency1) + np.sin(x*frequency2))


I am working on the file "02 - Interactivity.ipynb".
The first task is:
Loading Matplotlib demos with %load
IPython's %load magic can be used to load any Matplotlib demo by its URL:
Can you write Python code for it?

%load http://matplotlib.org/mpl_examples/showcase/integral_demo.py

"""
Plot demonstrating the integral as the area under a curve.

Although this is a simple example, it demonstrates some important tweaks:

    * A simple line plot with custom color and line width.
    * A shaded region created using a Polygon patch.
    * A text label with mathtext rendering.
    * figtext calls to label the x- and y-axes.
    * Use of axis spines to hide the top and right spines.
    * Custom tick placement and labels.
"""
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.patches import Polygon

def func(x):
    return (x - 3) * (x - 5) * (x - 7) + 85

a, b = 2, 9 # integral limits
x = np.linspace(0, 10)
y = func(x)

fig, ax = plt.subplots()
plt.plot(x, y, 'r', linewidth=2)
plt.ylim(ymin=0)

# Make the shaded region
ix = np.linspace(a, b)
iy = func(ix)
verts = [(a, 0)] + list(zip(ix, iy)) + [(b, 0)]
poly = Polygon(verts, facecolor='0.9', edgecolor='0.5')
ax.add_patch(poly)

plt.text(0.5 * (a + b), 30, r"$\int_a^b f(x)\mathrm{d}x$",
         horizontalalignment='center', fontsize=20)

plt.figtext(0.9, 0.05, '$x$')
plt.figtext(0.1, 0.9, '$y$')

ax.spines['right'].set_visible(False)
ax.spines['top'].set_visible(False)
ax.xaxis.set_ticks_position('bottom')

ax.set_xticks((a, b))
ax.set_xticklabels(('$a$', '$b$'))
ax.set_yticks([])

plt.show()


I am working on the file "02 - Interactivity.ipynb".
The first task is:
Notice that a slider is only produced for p as the value of q is fixed.
Widget abbreviations
When you pass an integer valued keyword argument (x=10) to interact, it generates an integer valued slider control with a range of $[-10,+3\times10]$. In this case 10 is an abbreviation for an actual slider widget:
python
IntSliderWidget(min=-10,max=30,step=1,value=10)
In fact, we can get the same result if we pass this IntSliderWidget as the keyword argument for x:
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
Or return the value
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
Plotting with Matplotlib
IPython works with the Matplotlib plotting library, which integrates Matplotlib with IPython's display system and event loop handling.
matplotlib mode
To make plots using Matplotlib, you must first enable IPython's matplotlib mode.
To do this, run the %matplotlib magic command to enable plotting in the current Notebook.
This magic takes an optional argument that specifies which Matplotlib backend should be used.  Most of the time, in the Notebook, you will want to use the inline backend, which will embed plots inside the Notebook:
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
The children of the ContainerWidget are two integer valued sliders produced by the widget abbreviations above.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
The step size can be changed by passing a 3rd element in the tuple.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
This examples clarifies how interact proceses its keyword arguments:

If the keyword argument is Widget instance with a value attribute, that widget is used. Any widget with a value attribute can be used, even custom ones.
Otherwise, the value is treated as a widget abbreviation that is converted to a widget before it is used.

The following table gives an overview of different widget abbreviations:

Keyword argument	Widget

`True` or `False`	CheckboxWidget

`'Hi there'`	TextareaWidget

`value` or `(min,max)` or `(min,max,step)` if integers are passed	IntSliderWidget

`value` or `(min,max)` or `(min,max,step)` if floats are passed	FloatSliderWidget

`('orange','apple')` or `{'one':1,'two':2}`	Dropdown

You have seen how the checkbox and textarea widgets work above. Here, more details about the different abbreviations for sliders and dropdowns are given.
If a 2-tuple of integers is passed (min,max) a integer valued slider is produced with those minimum and maximum (inclusive) values. In this case, the default step size of 1 is used.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
To actually display the widgets, you can use IPython's display function.
Can you write Python code for it?

from IPython.display import display
display(w)


I am working on the file "02 - Interactivity.ipynb".
The first task is:
When we call interact, we pass fixed(20) for q to hold it fixed at a value of 20.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
When you move the slider, the function is called and the current value of x is printed.
If you pass True or False, interact will generate a checkbox:
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
When you pass this function as the first argument to interact along with an integer keyword argument (x=10), a slider is generated and bound to the function.
Can you write Python code for it?


I am working on the file "02 - Interactivity.ipynb".
The first task is:
You can also use Matplotlib GUI backends in the Notebook, such as the Qt backend (%matplotlib qt). This will use Matplotlib's interactive Qt UI in a floating window to the side of your browser. Of course, this only works if your browser is running on the same system as the Notebook Server.  You can always call the display function to paste figures into the Notebook document.
Making a simple plot
With matplotlib enabled, plotting should just work.
Can you write Python code for it?

import matplotlib.pyplot as plt
import numpy as np

x = np.linspace(0, 3*np.pi, 500)
plt.plot(x, np.sin(x**2))
plt.title('A simple chirp');


I am working on the file "02 - Interactivity.ipynb".
The first task is:
interact can also be used as a decorator. This allows you to define a function and interact with it in a single shot. As this example shows, interact also works with functions that have multiple arguments.
Can you write Python code for it?

@interact(x=True, y=1.0)
def g(x, y):
    print(x, y)


I am working on the file "02 Compute Items SIFT Features.ipynb".
The first task is:
Compute and Plot All Items
Can you write Python code for it?

for item in items:
    for view in views:
        print(item + '_' + view)
        plot(item,view)
        plt.show()


I am working on the file "02 Compute Items SIFT Features.ipynb".
The first task is:
Compute and Save
Issues:
* too many features (e.g. composition_book)
* too few features (e.g. avery_binder)
Can you write Python code for it?

import cv2, glob
from utils import compute_sift, imread_rgb, imread_gray, pack_keypoint, write_features_to_file

def worker(item):
    folder = ITEM_FOLDER + '/' + item + '/'
    files = glob.glob(folder + '*.png')
    for filename in files:
        image_RGB = imread_rgb(filename)
        if not image_RGB is None:
            image_RGB = cv2.resize(image_RGB,(RESIZE_X,RESIZE_Y))
            file_mask = filename[:-4] + '_mask.pgm'
            mask = imread_gray(file_mask)
            (kp, des) = compute_sift(image_RGB, mask)
            kp, des = pack_keypoint(kp, des)
            write_features_to_file(filename[:-4] + '_sift.npy', kp, des)

%%time
from multiprocessing import Pool

print('Computing SIFTs of images resized to %d x %d' % (RESIZE_X,RESIZE_Y))
pool_size = 6
pool = Pool(pool_size)
result = []
for item in items:
    result.append( pool.apply_async(worker, (item,)) )
pool.close()
pool.join()
for r in result:
     r.get()


I am working on the file "02 Compute Items SIFT Features.ipynb".
The first task is:
Computing Test
Can you write Python code for it?

from matplotlib import pyplot as plt
%matplotlib inline

import cv2, numpy as np
from ipywidgets import interact
from utils import compute_sift, imread_rgb, imread_gray

def compute_and_plot(item,view):
    prefix = ITEM_FOLDER + '/' + item + '/' + item + '_' + view
    filename = prefix + '.png'
    image_RGB = imread_rgb(filename)
    if not image_RGB is None:
        image_RGB = cv2.resize(image_RGB,(RESIZE_X,RESIZE_Y))
        filename = prefix + '_mask.pgm'
        mask = imread_gray(filename)
        (kp, des) = compute_sift(image_RGB, mask)
        kp_img = cv2.drawKeypoints(image_RGB,kp,color=(0,255,0))
        print('%d features detected' % len(kp))
        plt.imshow(kp_img); plt.axis('off');

views = ['top_01','top-side_01','top-side_02','bottom_01','bottom-side_01','bottom-side_02']
interact(compute_and_plot,item=items,view=views);


I am working on the file "02 Compute Items SIFT Features.ipynb".
The first task is:
Plot All Items
Can you write Python code for it?

for item in items:
    for view in views:
        print(item + '_' + view)
        load_and_plot(item,view)
        plt.show()


I am working on the file "02 Compute Items SIFT Features.ipynb".
The first task is:
Statistics
Can you write Python code for it?

import glob
from utils import read_features_from_file, unpack_keypoint

item_view = []
num_sift = []
for item in items:
    folder = ITEM_FOLDER + '/' + item + '/'
    files = glob.glob(folder + '*_sift.npy')
    for filename in files:
        kp, des = read_features_from_file(filename)
        kp, des = unpack_keypoint(kp, des)
        item_view.append(filename)
        num_sift.append(len(des))

from matplotlib import pyplot as plt
%matplotlib inline
plt.hist(num_sift,bins=60);

[(ns, str(iv.split('/')[-1][:-9])) for ns, iv in sorted(zip(num_sift,item_view), reverse=True) if ns>2000]

[(ns, str(iv.split('/')[-1][:-9])) for ns, iv in sorted(zip(num_sift,item_view), reverse=True) if ns<50]


I am working on the file "02 Introduction to Strings.ipynb".
The first task is:
After you run the cell above to assign your message to message2, enter a Python command below to print the value of message2 and then execute it.
Quoting Quotes
Why does Python allow both double and single quotes?  Execute the following cell to see what can go wrong with using single quotes.
Can you write Python code for it?

message3 = 'I'm looking for someone to share in an adventure.'


I am working on the file "02 Introduction to Strings.ipynb".
The first task is:
Note that the print command above doesn't use quotes. What do you think the cell below will print?  Run it to see what happens.  There will be more about this later, but the print function accepts variables and literals.
Can you write Python code for it?


I am working on the file "02 Introduction to Strings.ipynb".
The first task is:
Quotes in Quotes
But, thanks to the fact that Python also allows double quotes, the more elegant way to include apostrophes in your string is to enclose the whole string in double quotes.  Try using double quotes to fix the line below.
Jupyter Note: Jupyter will automatically close your quotes for you as you type, so you will need to delete any extra quotes that you don't want.  Autoclose is a Jupyter feature that can be turned off, but we'll just have to deal with it now!
Can you write Python code for it?

message3 = 'I'm looking for someone to share in an adventure.'

print(message3)


I am working on the file "02 Introduction to Strings.ipynb".
The first task is:
The Python interpreter thinks the apostrophe in I'm is the end quote, and it can't understand the rest of the line, so it gives a syntax error starting right after the second quote.
#### Escape character \
One way you can fix the error by adding a backslash \ before the apostrophe you want to print. The backslash is called an "escape character" and tells Python to accept the next character as part of the string. Add a backslash before the apostrophe in I'm to fix the code below, then check it with the print statement.
Can you write Python code for it?

message3 = 'I\'m looking for someone to share in an adventure.'

print(message3)


I am working on the file "02 Introduction to Strings.ipynb".
The first task is:
Triple Quotes
More complex quoting requirements can be handled with triple quotes.  You can quote single and double quotes as well as line feeds with triple quotes.  Here's an example...
Can you write Python code for it?

movie_quote = """One of my favorite lines from The Godfather is:
"I'm gong to make him an offer he can't refuse."
Do you know who said this?"""

print(movie_quote)

print("socratica".upper())  # Just playing around - strings have methods


I am working on the file "02 Jupyter Notebook & Python Intro.ipynb".
The first task is:
Aktionen
Can you write Python code for it?

type("4.56") #mit type kann ich überprüfen, wie es der computer versteht, ob es float, integer und dates

print('Hallo' + 'wie' + 'geht' + 'es') #reine addition

print('Hallo','wie','geht','es') #mit kommas ist es etwas leserlicher

#Alle anderen gängigen:
#minus -
#Mal *
#geteilt durch /
#Spezial: Modulo. %, geteilt durch und der Rest, der übrigbleibt

13 % 5


I am working on the file "02 Jupyter Notebook & Python Intro.ipynb".
The first task is:
Datentypen
Can you write Python code for it?

#Strings
'Hallo wie geht es Dir'
"12345" #versteht der computer als text

str(124) #soll es als text verstehen

#Integer
567

int('1234') #hier versteht es der computer als zahl

#Floats
4.542323 #hier steht es der computer als zahlen mit kommastellen

float(12)

#Dates, eigentlich Strings
'15-11-2019'


I am working on the file "02 Jupyter Notebook & Python Intro.ipynb".
The first task is:
Dictionaries
Verwende hier die geschwungene Klammern
Can you write Python code for it?

#Komische, geschwungene Klammern #dictionary sind mächtigere listen
{'Tier'#schlüssel: 'Hund', 'Grösse': 124, 'Alter': 10} #geschwungene klammern alt 8 + 9 eckige klammern alt 5 + 6

dct = {'Tier': 'Hund', 'Grösse': 124, 'Alter': 10} #dct ist die variable

dct['Tier']

#List of Dictionaires #sind
dct_lst = [{'Tier': 'Hund', 'Grösse': 124, 'Alter': 10}, {'Tier': 'Katze', 'Grösse': 130, 'Alter': 8}]

dct_lst[0]

dct_lst[0]['Alter']


I am working on the file "02 Jupyter Notebook & Python Intro.ipynb".
The first task is:
For loop with list of dictionaries
Can you write Python code for it?

dic_lst = [{'Animal': 'Dog', 'Size': 45},
          {'Animal': 'Cat', 'Size': 23},
          {'Animal': 'Bird', 'Size': 121212}]

for dic in dic_lst:
    print(dic)

for dic in dic_lst:
    print(dic['Animal'])

for dic in dic_lst:
    print(dic['Animal'] + ': '+  str(dic['Size']))


I am working on the file "02 Jupyter Notebook & Python Intro.ipynb".
The first task is:
Variablen, Vergleiche und Zuordnungen von Variablen
Can you write Python code for it?

#Grösser und kleiner als:
#< >

#Gleich == (wichtig, doppelte Gleichzeichen) #doppeltes Gleichzeichen -> ist der Vergleich
#Denn das einfache Gleichzeichen definiert eine Variable #einfaches Gleichzeichen -> ist die Zuordnung (Variable)

'Schweiz' == 'reich'

Schweiz = 'reich'

Schweiz == 'reich'

'Schweiz' = 'reich'

1 = 6

a = b

a = 'b' #hier ordne ich zu

a == "b" #hier vergleiche ich a mit b und er sagt, es sei richtig, weil ich vorher zugeordnet habe

a


I am working on the file "02 Jupyter Notebook & Python Intro.ipynb".
The first task is:
if - else - (elif)
Can you write Python code for it?

elem = int(input('Wie alt bist Du?')) #elem ist die variable

if elem < 0:
    print('Das ist unmöglich')
else:
    print('Du bist aber alt')

elem = int(input('Wie alt bist Du?'))

if elem < 0:
    print('Das ist unmöglich')
elif elem < 25: #ich kann unendlich elif einfügen, dann werden beide parameter verwendet / ansonsten if, ausschliessend
    print('Du bist aber jung')
else:
    print('Du bist aber alt')


I am working on the file "02 Python II Hausaugaben.ipynb".
The first task is:
3.Entwickle einen Code, der den Nutzer nach der Länge seinem Namen fragt, und ihm dann sagt, wieviele Zeichen sein Name hat.
Can you write Python code for it?

name = input("How many letters does your name contain? ")
print("Your name has " + str(len(name)) +" letters.")


I am working on the file "02 Python II Hausaugaben.ipynb".
The first task is:
4.Entwickle eine Funktion mit dem Namen km_rechner, der für die untenaufgeführten automatisch die Umrechung von Meilen in km durchführt und gerundet auf eine Kommastelle anzeigt.
Can you write Python code for it?

def km_rechner(Distanz):
    result = round(Distanz*0.621371,1)
    return print(result)

km_rechner(5)
km_rechner(123)
km_rechner(53)