I am working on the file "Introduction_to_Algorithms/24_Single-Source_Shortest_Paths/note.ipynb".
The first task is:
interesting application: to determine critical paths in PERT chart analysis.
24.3 Dijkstra's algorithm (greedy strategy)
weighted, directed graph $G = (V, E)$ and all $w(u, v) \geq 0$.
Dijkstra's algorithm maintains a set $S$ of vertices whose final shortest-path weights from the source $s$ have already been determined.
we use a min-priority queue $Q$ of vertices, keyed by their $d$ values.
Can you write Python code for it?


I am working on the file "Introduction_to_Algorithms/25_All-Pairs_Shortest_Paths/note.ipynb".
The first task is:
25.2 The Floyd-Warshall algorithm
use a different dynamic-programming formulation to solve. $O(V^3)$.
The structure of a shortest path
intermediate vertex of a simple path $p = <v_1, v_2, \dotsc, v_n>$ is any vertex of $p$ other than $v_1$ or $v_n$.
Consider all paths from $i$ to $j$ whose intermediate vertices are all drawn from ${1, 2, \dotsc, k}$, and let $p$ be a minimum-weight path from among them.

If $k$ is not an intermediate vertex of path $p$, then all intermediate vertices of path $p$ are in the set ${1, 2, \dotsc, k-1}$.

If $k$ is an intermediate vertex of path $p$, then we decompose $p$ into $i \overset{p_1}{\to} k \overset{p_2}{\to} j$, so all intermediate vertices of $p_1$ are in the set ${1, 2, \dotsc, k-1}$, same with $p_2$.
Can you write Python code for it?


I am working on the file "Introduction_to_Algorithms/25_All-Pairs_Shortest_Paths/note.ipynb".
The first task is:
Constructing a shortest path
Transitive closure of a directed graph
to determine whether $G$ contains a path from $i$ to $j$ for all vetex pairs $i, j \in V$.
This method subsitutes the logical OR and logical AND for the arithmetic operations $\min$ and $+$ in the Floyd_Warshall algorithm.
Can you write Python code for it?


I am working on the file "Introduction_to_Algorithms/25_All-Pairs_Shortest_Paths/note.ipynb".
The first task is:
If we replace $\infty$ by 0 in EXTEND-SHORTEST-PATHS, it can be converted to matrix muplication: $L^{(n-1)} = W^{n-1}$.
Improving the running time
We can compute $L^{(n-1)}$ by divide-and-conquer methods: $O(n) \to $O(\lg n)$.
Total running time: $O(n^3 n) \to O(n^3 \lg n)$
Can you write Python code for it?


I am working on the file "Introduction_to_Algorithms/26_Maximum_Flow/note.ipynb".
The first task is:
26 Maximum Flow
each directed edge in a flow network like a conduit for the material: Each conduit has a stated capacity, vertices are conduit junctions.
In the maximum-flow problem, we wish to compute the greatest rate at which we can ship material from the source to the sink without violating any capacity constraints.
26.1 Flow networks
Let $G = (V, E)$ be a flow network with a capacity function $c$.
Let $s$ be the source of the network, and let $t$ be the sink.
A flow in $G$ is a real-valued function $f : V \times V \to \mathcal{R}$ that satisfies the following two properties:

Capacity constraint: $0 \geq f(u, v) \geq c(u, v) : \forall u, v \in V$

Flow conservation: $$\displaystyle \sum_{v \in V} f(v, u) = \sum_{v \in V} f(u, v)$$

real-world flow problem may violates our assumption:
modeling problems with antiparalled edges.

networks with multiple sources and sinks:
add a supersourse and a supersink.
Can you write Python code for it?

plt.imshow(plt.imread('./res/fig26_1.png'))

# Exercise


I am working on the file "Introduction_to_Algorithms/26_Maximum_Flow/note.ipynb".
The first task is:
26.2 The Ford-Fulkerson method
The Ford-Fulkerson method depends on three important ideas: residual networks, augmenting paths, and cuts.
```c
FORD-FULKERSON-METHOD(G, s, t)
initialize flow f to 0
while there exists an augmenting path p in the residual network G_f
    augment flow f along p
return f
```
Residual networks
the residual network $G_f$ consists of edges with capacitites that represent how we can change the flow on edges of $G$.
we define the residual capacity $c_f(u, v)$ by
\begin{equation}
    c_f(u, v) = \begin{cases}
        c(u, v) - f(u, v) \, & \text{ if } (u, v) \in E \
        f(v, u) \, & \text{ if } (v, u) \in E \
        0 \, & \text{ otherwise}
    \end{cases}
\end{equation}
the residual network of $G$ induced by $f$ is $G_f = (V, E_f)$, where
$$E_f = { (u, v) \in V \times V : c_f(u, v) > 0 }$$
If $f$ is a flow in $G$ and $f'$ is a flow in the corresponding residual network $G_f$, we define$f \uparrow f'$, the augmentation of flow $f$ by $f'$:
\begin{equation}
    (f \uparrow f')(u, v) = \begin{cases}
        f(u, v) + f'(u, v) - f'(v, u) \, & \text{ if } (u, v) \in E \
        0 \, & \text{ otherwise}
    \end{cases}
\end{equation}
Can you write Python code for it?


I am working on the file "Introduction_to_Algorithms/26_Maximum_Flow/note.ipynb".
The first task is:
26.3 Maximum bipartite matching
a maximum matching in a bipartite graph $G$ corresponds to a maximum flow in its corresponding flow network $G'$, and we can therefore compute a maximum matching in $G$ by running a maximum-flow algorithm on $G'$.
Can you write Python code for it?


I am working on the file "Introduction_to_Algorithms/26_Maximum_Flow/note.ipynb".
The first task is:
Augmenting paths
an augmenting path $p$ is a simple path from $s$ to $t$ in the residual network $G_f$.
$$c_f(p) = \min { c_f(u, v): \text{ $(u, v)$ is on $p$} }$$
Cuts of flow networks
The Ford-Fulkerson method repeatedly augments the flow along augmenting paths until it has found a maximum flow.
The max-flow min-cut theorem tells us that a flow is maximum if and only if its residual network contains no augmenting path.
The basic Ford-Fulkerson algorithm
Can you write Python code for it?


I am working on the file "Introduction_to_Pandas.ipynb".
The first task is:
Before we explore the package pandas, let's import pandas package. We often use pd to refer to pandas in convention.
Can you write Python code for it?

%matplotlib inline
import pandas as pd
import numpy as np


I am working on the file "Introduction_to_Pandas.ipynb".
The first task is:
Filtering out Missing Data
You have a number of options for filtering out missing data.
Can you write Python code for it?

df = pd.DataFrame([[1, 6.5, 3.], [1., np.nan, np.nan],
                [np.nan, np.nan, np.nan], [np.nan, 6.5, 3.]])

df

cleaned = df.dropna()  # delete rows with Nan value
cleaned

df.dropna(how='all')  # delete rows with all Nan value

df.dropna(thresh=2)  # keep the rows with at least thresh non-Nan value

df.fillna(0)  # fill Nan with a constant


I am working on the file "Introduction_to_Pandas.ipynb".
The first task is:
Grouped and apply
Can you write Python code for it?

df

df.groupby(df['state']).mean()

df.groupby(df['state'])[['pop', 'rain']].apply(lambda x: x.max() - x.min())

grouped = df.groupby(df['state'])
group_list = []
for name, group in grouped:
    print(name)
    print(group)
    print('\n')


I am working on the file "Introduction_to_Pandas.ipynb".
The first task is:
Plotting in DataFrame
Can you write Python code for it?

variables = pd.DataFrame({'normal': np.random.normal(size=100),
                       'gamma': np.random.gamma(1, size=100),
                       'poisson': np.random.poisson(size=100)})
variables.head()

variables.shape

variables.cumsum().plot()

variables.cumsum().plot(subplots=True)


I am working on the file "Introduction_to_Pandas.ipynb".
The first task is:
Series
A Series is a single vector of data (like a Numpy array) with an index that labels each element in the vector.
Can you write Python code for it?

counts = pd.Series([223, 43, 53, 24, 43])
counts

type(counts)


I am working on the file "Introduction_to_Pandas.ipynb".
The first task is:
Set Hierarchical indexing
Can you write Python code for it?

df

df_h = df.set_index(['state', 'year'])
df_h

df_h.index.is_unique

df_h.loc['Ohio', :].max() - df_h.loc['Ohio', :].min()


I am working on the file "Introduction_to_Pandas.ipynb".
The first task is:
Sorting index
Can you write Python code for it?

df.sort_index(ascending=False)

df.sort_index(axis=1, ascending=False)


I am working on the file "Introduction_to_Pandas.ipynb".
The first task is:
Summarizing and Computing Descriptive Statistics
Built in functions to calculate the values over row or columns.
Can you write Python code for it?

df

df.loc[:, ['pop', 'rain']].sum()

df.loc[:,['pop', 'rain']].mean()

df.loc[:, ['pop', 'rain']].var()

df.loc[:, ['pop', 'rain']].cumsum()


I am working on the file "Introduction_to_Pandas.ipynb".
The first task is:
We can give both the array of values and the index meaningful labels themselves:
Can you write Python code for it?

fruit.name = 'counts'
fruit.index.name = 'fruit'
fruit


I am working on the file "Introduction_to_Pandas.ipynb".
The first task is:
You can get the array representation and index object of the Series via its values and index atrributes, respectively.
Can you write Python code for it?


I am working on the file "Introduction_to_Pandas.ipynb".
The first task is:
data filtering
Can you write Python code for it?


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
Conditional Statement

Conditional statement are used to execute code fragments based on a condition.
The most commonly used construct is if-else and while statement
Can you write Python code for it?

N = 10
if N%2 == 0:
    print('Even')
else:
    print('Odd')

N = 9
if N%2 == 0:
    print('Even')
else:
    print('Odd')

N = 9
if N%2 == 0 and N > 10:
    print('A Even number greater than 10')
else:
    print('smaller than 10 or odd number')

N = 0
while N <= 10:
    if N%2 == 0:
        print(N, 'is Even.')
    else:
        print(N, ' is Odd.')
    N = N+1


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
Dict

Dict is another very important data structure in python
Comparing to list, it is a unordered sets or objects
Moreover, the item are accessed via keys and not via their position
Can you write Python code for it?

empty = {}
empty

type(empty)

food = {'ham': 'yes',
       'egg': 'yes',
       'spam': 'no'}
food

food['ham']

food['spam'] = 'yes'
food

len(food)

# Accessing non Existing keys
food['apple']

# Merge dictionary
food2 = {'beef': 'yes',
        'shoe': 'no'}
food.update(food2)
food

# Iterating over Dictionary
for key in food:
    print(key)
    print(food[key])
    print('\n')


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
Exercises 1
Please construct a function named productEven which imput a list of integers and return the product of the even numbers in the list.
For example:
pruductEven([2, 3, 8, 7]) gives 16
Can you write Python code for it?

def productEven(l):
    ret = 1
    for item in l:
        if item % 2 == 0:
            ret = ret * item
    return ret

productEven([2, 3, 8, 7])


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
Exercises 3
Write a python program to count the number 4 in a given list.
Can you write Python code for it?

l = [4, 2, 5, 6, 4]
count = 0
for item in l:
    if item == 4:
        count = count + 1
count


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
Exercises 4
Write a Python program to compute the greatest common divisor (GCD) of two positive integers.
Can you write Python code for it?

def gcd(a, b):
    for i in range(max(a, b)+1, 0, -1):
        if a%i == 0 and b%i == 0:
            return i

gcd(4, 6)


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
Function
Can you write Python code for it?

# basic function
def add(x, y):
    return x+y

add(2, 3)

# function with optional parameters
def add(x, y=5):
    return x + y

add(3)

add(3, 9)

# Keyword Paramters
def add(x, y=5):
    return x + y

add(3, y=10)

def isEven(N):
    if N%2 == 0:
        return True
    else:
        return False

isEven(10)

isEven(9)


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
In addition to indexing, slicing is also supported. While indexing is used to obtain individual item, slicing allows you to obtain sub-list:
Format of slicing: begin:end:step_size
Can you write Python code for it?

squares = [1, 2, 3, 4, 5]
squares[1:4]  # Get sub-list from the 1 to 4 with step size 1

squares[1:4:2]  # Get sub-list from 1 to 4 with step size 2

squares[:4]  # from begining of the list to 4 with step size 1

squares[1:]  # from 1 to the end of the list with step size 1

squares[::2]  # from begining to the end with step size 2


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
List
Python knows a number of compound data types, used to group together other values. The most versatile is the list, which can be written as a list of comma-separated values (items) between square brackets. Lists might contain items of different types, but usually the items all have the same type.
Can you write Python code for it?

empty_list = []
empty_list
type(empty_list)

squares = [1, 4, 9, 16, 25]
squares


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
List can be indexed, with the first item having index 0.
Can you write Python code for it?

squares[0]

squares[1]

len(squares)

squares[len(squares) - 1]

squares[-1]  # Last item

squares[-2]  # Second-last item


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
List support the + operation.
Can you write Python code for it?

list1 = [1, 2]
list2 = [3, 4]

list3 = list1 + list2
list3

list1 = [1, 2, 3]
list2 = [x**2 for x in list1]
list2


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
Python Iteration
Can you write Python code for it?

squares = [1, 4, 9, 16, 25]
for s in squares:
    print(s)

for i in range(10):
    print(i)

for i in range(1, 10):
    print(i)

for i in range(1, 10, 2):
    print(i)

N = 8
fact = 1
for i in range(1, N+1):
    fact = fact * i
fact

# For any i, we list the integer less than i
for i in range(5):
    print("i=", i)
    l = []
    for j in range(0,i):
        l.append(j)
    print(l)


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
String also support indexing and slicing like list. You can see the string as a list of charaters.
Can you write Python code for it?

s[1:]

s[-1]

s1 = 'Hello '
s2 = 'World!'

s3 = s1 + s2
s3

s3.isdigit()

s3.isupper()

s3.lower()


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
Strings
Besides numbers, Python can also manipulate strings, which can be expressed in several ways. They can be enclosed in single quotes ('...') or double quotes ("...") with the same result. \ can be used to escape quotes:
Can you write Python code for it?

'Hello World!'

type('hello')

'doesn\'t'

s = 'Hello World'
s


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
Use bool operators to create bool type
Can you write Python code for it?


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
Use equal sign(=) to assign a value to variable like math variable.
Can you write Python code for it?

width = 30
width

width = 30
height = 2
width * height


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
With Python, use ** operator to calculate powers.
Can you write Python code for it?


I am working on the file "Introduction_to_Python.ipynb".
The first task is:
You can replace the item in the list and add new items at the end of the list.
Can you write Python code for it?

names = []

names.append('John')

names

names.append('Paul')
names

names[1] = 'Alice'
names

names2 = ['Ryan', 'Tang']
names2

names = names + names2
names

names.append(['Fang', 'HE'])
names

squares.append(11)
squares

squares[1:3] = [12, 13, 14]
squares

len(squares)  # calculate the number of item in the list


I am working on the file "Introduction_to_Python3.ipynb".
The first task is:
Calculus with Sympy
Can you write Python code for it?

# wolfman is better at symblic computation
from sympy import init_printing
from sympy import init_session

init_session(quiet=True)
Integral(sqrt(1/x), x)

x, y, z = symbols('x y z')

# differation
diff(exp(x ** 2), x)

# partial derivative
# https://mzucker.github.io/2018/04/11/sympy-case-studies-part-2-derivatives.html
import sympy
from sympy import init_printing
# from sympy import init_session

init_session(quiet=True)
x = Symbol('x', real=True)
f = exp(-x**2 / 2) * cos(sympy.pi*x)

dfdx = f.diff(x)

print("f(x) =", sympy.simplify(dfdx))

diff(exp(-x**2 / 2) * cos(sympy.pi*x), x)

integrate(exp(-x**2 - y**2), (x, -oo, oo), (y, -oo, oo))

integ = Integral((x**4 + x**2*exp(x) - x**2 - 2*x*exp(x) - 2*x -
                  exp(x))*exp(x)/((x - 1)**2*(x + 1)**2*(exp(x) + 1)), x)

integ

integ.doit()

# limit
limit(x**x, x, 0)

# SymPy is able to solve algebraic equations, in one and several variables
solve(x**4 - 1, x)

solve([x + 5*y - 2, -3*x + 6*y - 15], [x, y])


I am working on the file "Introduction_to_Python3.ipynb".
The first task is:
Control flow tools
Can you write Python code for it?

# if statements
# if true:
#     do something
# else:
#     do something else
x = int(input("Please enter an integer: "))

if x < 0:
    print("x is less than zero.")
elif x == 0:
    print("x is just zero.")
else:
    print("x is greater than zero.")

# another example
# evaluate whether you give the right number
import random
x = random.randint(1, 99)
# input your code here to print whether you have guesses the right number
# if right, print: Wow, you got the right number!
# if not, print: This is not the right one. Keep trying!

input_num = int(input("Please enter your guess ingeger:"))
if input_num == x:
    print("Wow, you got the right number!")
else:
    print("This is not the right one. Keep trying!")

# for statements
fruits = ['apple', 'orange', 'banana', 'grape', 'blueberry', 'mango']
for fruit in fruits:
    print(fruit, len(fruit))

# combine if and for statements
fruits = ['apple', 'orange', 'banana', 'grape']
# print all the fruit that with length greater than or equal than 6

for fruit in fruits:
    if len(fruit) >= 6:
        print(fruit, len(fruit))

# iterate over a sequence of numbers, the built-in function range()
for i in range(15):
    if i % 2 == 0:
        print(i)

# while statement
# do something as long as the condition is met
# the sum of  1+2+3+...+100
tol_sum = 0
step = 0

while step < 101:
    tol_sum += step
    step += 1

print(tol_sum)

# break, pass and continue statements
# http://www.cse.unsw.edu.au/~en1811/python-docs/python-3.6.4-docs-pdf/tutorial.pdf
# https://docs.python.org/3/tutorial/index.html


I am working on the file "Introduction_to_Python3.ipynb".
The first task is:
Evaluate the $3 x^2 + 7x -1$ with respect of $x = 2$
Can you write Python code for it?

from sympy import *
from sympy import init_printing

init_session(quiet=True)

x = symbols('x')
f = Function('f')(x)

fprime = diff(3*(x ** 2) + 7*x -1, x)
fprime

fprime.evalf(subs={x:2})

# for more details, please refer https://docs.sympy.org/latest/index.html


I am working on the file "Introduction_to_Python3.ipynb".
The first task is:
How about $x = (5^2 + 9) / 10$ and $y = exp({x^2}/2)$?
Can you write Python code for it?

x = (5 ** 2 + 9) / 10
import math
y = math.exp(x **2 /2)
print('x=', x)
print('y=', y)
# https://docs.python.org/3.7/library/math.html

x = (5 ** 2 + 9) / 10
import math
y = math.exp(math.pow(x, 2) / 2)
print('x=', x)
print('y=', y)

## comparision
5 == 9

# Logical operators: and, or, not
x = 5 == 9 and 4 == 4
y = (5 == 9 or 4 == 4)
print(x)
print(y)

# not
not 6 == 5

6 != 5

6 is not 5


I am working on the file "Introduction_to_Python3.ipynb".
The first task is:
Modules

code reuse
Implementing shared service or data
Can you write Python code for it?

# from module import function
# function()

# import module
# module.function()

# create a file called fibo.py
# Fibonacci numbers module

def fib(n):    # write Fibonacci series up to n
    a, b = 0, 1
    while a < n:
        print(a, end=' ')
        a, b = b, a+b
    print()

def fib2(n):   # return Fibonacci series up to n
    result = []
    a, b = 0, 1
    while a < n:
        result.append(a)
        a, b = b, a+b
    return result

import fibo
fibo.fib(1000)

fibo.fib2(100)

fib = fibo.fib
fib(500)

import os

os.listdir('.')


I am working on the file "Introduction_to_Python3.ipynb".
The first task is:
Practical example
the molar heat capacities at constant pressure (in $J K^{-1} mol^{-1}$) for $N_2$ gases, in the 273 to 3800 K range, are approximated by
\begin{equation}
    C_{P, N_2} = 27.38 + 6.23 \times 10^{-3} T - 9.5 \times 10^{-7} T^2
\end{equation}
where temperature is in $K$. Please show expressions for $H(T)$ and $S(T)$.
Can you write Python code for it?

# H(T) is the integrals of Cp(T)
from sympy import *
from sympy import init_printing

init_session(quiet=True)

T = symbols('T')
heat_capacity = 27.38 + 6.23*1.e-3*T - 9.5*1.e-7* (T**2)
H = Integral(heat_capacity, T)
H

# first method: get the integral expression and compute the values
H.doit()

# second method: do the integration directly
H_value = integrate(heat_capacity, (T, 273, 3800))

H_value


I am working on the file "Introduction_to_Python3.ipynb".
The first task is:
Whitespace is meaningful to python, espically indentation and placement of newlines.
* \ suggests continuting the codes.
* # comment
* """""" docstring
Can you write Python code for it?

# list
x = [1, 2, 3]
# mutable
x[0] = 9
print(x)

# methods on lists
x = [1,22, 30]

x.append(7)
x

x.extend([8, 5.4])
# x + [8, 5.4]
x

# + creates a fresh list (with a new memory reference)
# extend operates on list li in place

x.remove(5.4)
x

# sort in place
x.sort(reverse=False)
x

x.reverse()
x

# list slicing
# list[begin: end]
x = [10, 15, 24, 30, 8.9]
print(x[2:4])
print(x[:3])
print(x[2:])
print(x[-1])
print(x[:])

5 in x

len(x)

# tuples, inmutable
t = 12345, 54321, 'hello!'
type(t)

t2 = t, (1, 2, 3)
t2

t3 = t, [0, 0.8, 7]
t3

# tuple slicing is the same as list
t3[-1]

# sets
basket = {'apple', 'orange', 'apple', 'pear', 'orange', 'banana'}
print(basket)

# set operations
a = set('abracadabra')
b = set('alacazam')

print(a - b)

print(b - a)

a | b

a & b

a ^ b

# dictionary
iphone_year = {
    "iphone" : 2007,
    "iphone 3G" : 2008,
    "iphone 3GS" : 2009,
    "iphone 4" : 2010,
    "iphone 4S" : 2011,
    "iphone 5" : 2012}

iphone_year.keys()

iphone_year.values()

iphone_year['iphone 4S']

'iphone 6' in iphone_year

iphone_year['iphone x'] = 2018
# iphone_year.update({'iphone x': 2018})

print(iphone_year)

# convertion
list((2, 4, 'a'))

set({'b', 'd', 'a', 'a', 'z'})

dict([(1,'apple'), (2,'ball')])


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
But also is very important that we can substitute a variable with a new expression! This way we can build sequences...e.g. $x^{x^{x^{x}}}$
Can you write Python code for it?


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
For multiple symbols one can do
Can you write Python code for it?

y, z = sym.symbols('y z')
y+z+1

x='abc'

expr = x + 'def' ; expr


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
Lambdify
lambdify() is used to convert a SymPy expression to another library (usually NumPy). lambdify() acts as a lambda function but also converting the SymPy expressions to a another numerical library.
For example if we want to use the subs flag of the evalf() function to evaluate the function at 1000 points, then we can use numpy and lambdify...
Can you write Python code for it?

import numpy as np

a = np.arange(100)

expr = sin(x)

f = lambdify(x, expr, "numpy")  # takes an argument x=x0 and returns the expr(x0)

f(a)


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
N.B.
subs() returns a new expression, i.e. the expr is not changed!

subs is immutable! To perform multiple substitution we use a list of tuples:
Can you write Python code for it?


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
Powers
There are identities for powers that hold under constraints. For example:

$x^a x^b = x^{a+b}$   -> holds always

x^a y^a = (xy)^a  -> holds when $x,y>0$ and $a\in R$

(x^a)^b = x^{ab}   -> holds when b is integer

Symbols and requirements
For this reason when defining symbols we can put specific requirements as their arguments.
For example we define for this section:
Can you write Python code for it?

x, y = symbols('x y', positive=True)

a,b = symbols('a b', real=True)

z, t, c = symbols('z t c ')


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
Simplification
Functions that are used to simplify expressions
simplify()
Can you write Python code for it?

simplify(cos(x)**2 + sin(x)**2)

expr = (x**3 + x**2 - x - 1)/(x**2 + 2*x + 1);
expr

simplify(expr)

expr = gamma(x)/gamma(x-2); expr   ## gamma(x) := Gamma functions

simplify (expr)


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
Strings -> Sympy Expressions
We can use the sympify() ( NOT SIMPLIFY ) to convert string to sympy expressions
Can you write Python code for it?

str_expr = "x**2+3*x+2"

expr = sym.sympify(str_expr)

expr

expr.subs(x,2)


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
Substitute
Can you write Python code for it?

x = sym.symbols('x')
expr = x+1
expr

expr.subs(x,1) # should be 2

expr.subs(x,z)


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
The importance of substition is the following

We can simply evaluate an expression at a point. e.g. for the expression cos(x)+1 we want to know its value when x=0:
Can you write Python code for it?


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
There are roundoff errors smaller than the desired precision. Such numbers can be cut using the chop flag in eval():
Can you write Python code for it?

one = cos(1)**2 + sin(1)**2

(one-1).evalf()

(one-1).evalf(chop=True)


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
This is very usefull because we can use list comprehension for the substitutions!
Can you write Python code for it?

expr = x**4 - 4*x**3 + 4*x**2 - 2*x + 3
expr

replacements = [(x**i, y**i) for i in range(5) if i%2==0]  #replace all instances of x that
                                                           # have even powers with y

expr.subs(replacements)


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
To evaluate an expression with a symbol to a point we can use the subs flag in evalf() using a dictionary : Symbol: point
Can you write Python code for it?

expr = cos(2*x)
expr

expr.evalf(subs={x:0.2})

cos(2*0.2) #it works


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
To force a simplification without messing with the assumptions, while
Can you write Python code for it?


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
Trigonometric Simplification
N.B. Remember that the Python's convention for inverse trig equations is acos(), asin() etc..
Can you write Python code for it?


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
Very usefull when combined with the coeff() function that fives the coefficients of a symbol in the expression:
Can you write Python code for it?

expr

expr.coeff(x,2)

expr.coeff(y, 1)

cl.coeff(x,3)

cl.coeff(x,2)


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
apart() - Partial Fraction Decomposition
apart() simply performs a partial fraction decomposition on a rational function.
Can you write Python code for it?


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
cancel()
cancel() is used to take any rational function and put it into standard canonical form.
Can you write Python code for it?


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
collect() and coeff()
collect() is used to collect common powers of a term
Can you write Python code for it?

expr = x*y +x-3+2*x**2 - z*x**2 + x**3; expr

cl = collect(expr, x)

cl


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
expand_power_exp() and expand_power_base()
expand_power_exp() and expand_power_base() applies the identities 1,2 from right to left, expanding the exponent or the base of an expression
Can you write Python code for it?

expr = x**(a+b); expr

expand_power_exp(expr)

expr = (x*y)**a; expr

expand_power_base(expr)


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
factor() & factor_list()
factor is used on polynomials with rational coefficients and factors it inot irreducible factors over the rational numbers
Can you write Python code for it?

expr = x**3 - x**2 + x- 1;
expr

factor(expr)

expr = x**2*z + 4*x*y*z + 4*y**2*z; expr

factor (expr)

factor_list(expr)


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
forcing it with an argument
Can you write Python code for it?


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
powdenest()
This is to apply identity 3 on nested powers
Can you write Python code for it?

expr = (x**a)**b; expr  # automatically is simplified

powdenest(expr)


I am working on the file "Introduction_to_SymPy.ipynb".
The first task is:
trigsimp()
trigsimp() is used to simplify expressions based on trigonometric identities.
Can you write Python code for it?

expr = sin(x)**4 - 2*cos(x)**2*sin(x)**2 + cos(x)**4; expr

trigsimp(expr)

expr = sin(x)*tan(x)/sec(x); expr

trigsimp(expr)

expr = cosh(x)**2 + sinh(x)**2 ; expr

trigsimp(expr)


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.0 - Introduction-checkpoint.ipynb".
The first task is:
Basic example of iteration:
Python
for data in iterable:
    operation
Can you write Python code for it?

years = [2010, 2011, 2016, 2020]
for year in years:
    if is_leap_year(year):
        print("{} is a leap year!".format(year))


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.0 - Introduction-checkpoint.ipynb".
The first task is:
Basic example of selection:
Python
if (condition):
    operation
else
    operation

Q. Is the year 2000 a, gregorian calendar, leap year?
Leap years:

The year is evenly divisible by 4
If the year can be evenly divided by 100, it is NOT a leap year, unless the year is also evenly divisible by 400.
Can you write Python code for it?

def is_leap_year(year):
    """
    Returns if the year is a leap year.
    """
    if year % 100 == 0:
        return year % 400 == 0
    else:
        return year % 4 == 0


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.0 - Introduction-checkpoint.ipynb".
The first task is:
Context managers, can take care of handling the opening and closing of files.
Can you write Python code for it?

with open('test.file', 'w') as file_handle:
    file_handle.write("Hello")


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.0 - Introduction-checkpoint.ipynb".
The first task is:
For loops are also fine but more verbose:
Can you write Python code for it?

total = 0
for x in data:
    if x > 2:
        total += x
total


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.0 - Introduction-checkpoint.ipynb".
The first task is:
Influences from functional programming make code easier to read and understand - list comprehensions.
Can you write Python code for it?


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.0 - Introduction-checkpoint.ipynb".
The first task is:
Q. Mutate a string.
Can you write Python code for it?


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.0 - Introduction-checkpoint.ipynb".
The first task is:
Q. Sorting tabular datasets. Lists of Tuples and the sorted built-in.
Can you write Python code for it?

table = [("Nathan", 182), ("Michael", 175), ("Sam", 190)]
table

sorted(table)

sorted(table, key=lambda x:x[1], reverse=True)


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.0 - Introduction-checkpoint.ipynb".
The first task is:
Sets are also appropriate if you don't want order to be preserved.
Can you write Python code for it?

container = set(["nathan", 182])
container

182 in container


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.0 - Introduction-checkpoint.ipynb".
The first task is:
There are also dictionary comprehensions.
Can you write Python code for it?


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.0 - Introduction-checkpoint.ipynb".
The first task is:
Tip: Python allows you to use both functional and imperative styles and lets the user determine which approach is best.
What makes Python different from other languages?
Can you write Python code for it?


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.5 - Scikit-Learn-checkpoint.ipynb".
The first task is:
Form the feature matrix, which is the collection of measurements or observations that are associated with a particular state or label.
Can you write Python code for it?

X = pd.DataFrame(iris.data, columns=iris.feature_names)
X.head()


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.5 - Scikit-Learn-checkpoint.ipynb".
The first task is:
Form the set of labels, the classification of the state based on the measurements.
Can you write Python code for it?

Y = pd.DataFrame(np.vstack([iris.target, np.choose(iris.target,
                                                   iris.target_names)]).T,
                 columns=["label id", "label name"])
y = Y["label id"]

Y.head()


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.5 - Scikit-Learn-checkpoint.ipynb".
The first task is:
Measuring accuracy alone ignores other important features of a classifier:
http://scikit-learn.org/stable/modules/model_evaluation.html
Can you write Python code for it?

from sklearn.metrics import classification_report

print(classification_report(y_test, clf.predict(X_test), target_names=iris.target_names))


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.5 - Scikit-Learn-checkpoint.ipynb".
The first task is:
Q. How many measurements and how many classes of flower?
Can you write Python code for it?

print("There are {} measurements and {} classes of flower.".format(X.size, Y["label name"].unique().size))


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.5 - Scikit-Learn-checkpoint.ipynb".
The first task is:
Using a simple linear classifier, lets train the model and evaluate the performance on the test set.
Can you write Python code for it?

from sklearn.svm import LinearSVC

clf = LinearSVC()

clf.fit(X_train, y_train)


I am working on the file "Introductions/.ipynb_checkpoints/Tutorial - 1.5 - Scikit-Learn-checkpoint.ipynb".
The first task is:
We can look at how the classifier performed on the test set using the classifiers "score" method. This reports the mean accuracy of the classifier.
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
Basic boolean expressions, selection and iteration.
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
Basic example of iteration:
Python
for data in iterable:
    operation
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
Basic example of selection:
Python
if (condition):
    operation
else
    operation

Q. Is the year 2000 a, gregorian calendar, leap year?
Leap years:

The year is evenly divisible by 4
If the year can be evenly divided by 100, it is NOT a leap year, unless the year is also evenly divisible by 400.
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
Context managers, can take care of handling the opening and closing of files.
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
For simple things the R language looks similar, especially math.
R
data <- c(1,2,3,4)
sum(data)
Q. Sum an array of values only if the values are greater than 2.
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
Functions and Classes
Q. Write a greeting function that takes two arguments and returns a string.
Can you write Python code for it?

def greet(first_name, last_name):
    """Returns a string"""
    return "Hello {} {}.".format(first_name, last_name)

greet("Nathan","Faggian")


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
Influences from functional programming make code easier to read and understand - list comprehensions.
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
Q. An infinite fibonacci sequence. A. Using generators.
Can you write Python code for it?

def fib():
   a, b = 0, 1
   while True:
      yield b
      a, b = b, a + b

fib_generator = fib()

value = next(fib_generator)
value


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
Q. Assign a value to a variable (references, most things are mutable)
Can you write Python code for it?

name = "Nathan Faggian"
name

yourname = name

id(name), id(yourname)


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
Q. Form an array of values and sum then together using the sum built-in.
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
Q. Who is the tallest person?
Can you write Python code for it?

import operator

sorted(heights.items(), key=operator.itemgetter(0), reverse=True)


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
Q. Write a basic class that encapsulates some data.
Can you write Python code for it?

class Wallet(object):
    """
    A Wallet contains integers of dollars and cents.
    """

    def __init__(self, dollars, cents):
        self.dollars = dollars
        self.cents = cents

    def __repr__(self):
        return "Wallet({},{})".format(self.dollars, self.cents)

    def __del__(self):
        pass

pouch = Wallet(1000,0)
print("A wallet: {}, with {} dollars and {} cents.".format(pouch,
                                                           pouch.dollars,
                                                           pouch.cents))


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
There are also dictionary comprehensions.
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.0 - Introduction.ipynb".
The first task is:
Tip: Python allows you to use both functional and imperative styles and lets the user determine which approach is best.
What makes Python different from other languages?
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.2 - Matplotlib.ipynb".
The first task is:
Q. Go to the matplotlib gallery and paste in an example.
Can you write Python code for it?

"""
Demo of scatter plot on a polar axis.

Size increases radially in this example and color increases with angle (just to
verify the symbols are being scattered correctly).
"""

import numpy as np
import matplotlib.pyplot as plt

N = 150
r = 2 * np.random.rand(N)
theta = 2 * np.pi * np.random.rand(N)
area = 200 * r**2 * np.random.rand(N)
colors = theta

ax = plt.subplot(111, polar=True)
c = plt.scatter(theta, r, c=colors, s=area, cmap=plt.cm.hsv)
c.set_alpha(0.75)

plt.show()


I am working on the file "Introductions/Tutorial - 1.2 - Matplotlib.ipynb".
The first task is:
The matplotlib API is both rich and deep, a good way to learn is to work through examples of the plots in the gallery.
Can you write Python code for it?

data = np.random.randint(0, 10, (25))

figure, axis = plt.subplots(figsize=(8, 5))
axis.plot(data);
axis.grid()


I am working on the file "Introductions/Tutorial - 1.2 - Matplotlib.ipynb".
The first task is:
We can change the line properties, color and axis properties.
Can you write Python code for it?

figure, axis = plt.subplots(figsize=(8, 5))

axis.plot(data, color='r', marker='o', linestyle='--')

axis.grid()
axis.set_xlabel("Random variable")
axis.set_ylabel("Rangom value")
axis.set_title("Random figure")

axis.set_ylim(-1, data.max() + 1.)


I am working on the file "Introductions/Tutorial - 1.3 - Scipy.ipynb".
The first task is:
Plot the distribution of the data, using a histogram. Using the hist method.
Can you write Python code for it?

fig, ax = plt.subplots(figsize=(10,5))
ax.hist(samp, 100, normed=1)
ax.set_xlim(-5, 5);
ax.grid()


I am working on the file "Introductions/Tutorial - 1.3 - Scipy.ipynb".
The first task is:
Plot the found minimum.
Can you write Python code for it?

%matplotlib inline

x = np.arange(-10, 10, 0.1)
fig, ax = plt.subplots(figsize=(10,5))
ax.plot(x, f(x));
ax.hlines(0,-10, 10, alpha=0.1);
ax.plot(initial_x, f(initial_x), 'or')
ax.plot(minimum_x, f(minimum_x), 'og')


I am working on the file "Introductions/Tutorial - 1.4 - Pandas.ipynb".
The first task is:
A table behave similarly to a numpy array (in fact underneath it is a nd-array), meaning we can use fancy indexing:
Can you write Python code for it?

missing_data = table[table > 0.6]
missing_data


I am working on the file "Introductions/Tutorial - 1.4 - Pandas.ipynb".
The first task is:
Matrix representation of data, provided by numpy, often isn't enough and this is where Pandas can help, through the introduction of a "data-frame", which is a tabular representation.
Can you write Python code for it?

pd.DataFrame(data)

table = pd.DataFrame(data, columns=["a","b","c","d","e"])
table


I am working on the file "Introductions/Tutorial - 1.4 - Pandas.ipynb".
The first task is:
Note the automatic selection of an index (left hand column). This can be used to pull out rows of interest.
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.4 - Pandas.ipynb".
The first task is:
Q. Plot the monthly statistics for the "a" category.
Can you write Python code for it?

data = table[["date", "a"]].set_index("date")
data.head()

ax = data.plot(figsize=(10,5), color='g')
data.resample('w').plot(color='m', ax=ax)
data.resample('w', how="min").plot(color='b', ax=ax)
data.resample('w', how="max").plot(color='r', ax=ax)
ax.legend(['a', 'mean(a)', 'min(a)', 'max(a)']);

ax = data.plot(figsize=(10,5), color='g')
data.resample("w", how=["mean", "min", "max"]).plot(ax=ax, color=["m","b","r"])
ax.legend(['a', 'mean(a)', 'min(a)', 'max(a)']);


I am working on the file "Introductions/Tutorial - 1.4 - Pandas.ipynb".
The first task is:
Q. What are the summary statistics for December?
Can you write Python code for it?

table[table["month"] == "December"].describe()

groups = table.groupby("month").describe().loc["December"]


I am working on the file "Introductions/Tutorial - 1.4 - Pandas.ipynb".
The first task is:
Q. What are the summary statistics of the "missing_data" table?
Can you write Python code for it?

missing_data.describe()

missing_data.count()

missing_data.min()


I am working on the file "Introductions/Tutorial - 1.4 - Pandas.ipynb".
The first task is:
Q. What is the 95 quantile for each month.
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.4 - Pandas.ipynb".
The first task is:
Similarly we can select data from columns.
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - 1.4 - Pandas.ipynb".
The first task is:
Split - Apply - Combine
A powerful concept that will help you to perform complex operations on datasets.
http://pandas.pydata.org/pandas-docs/stable/groupby.html
Can you write Python code for it?

dates = pd.date_range(start="1/1/14", end="31/12/14")

categories = ["a", "b", "c", "d"]
noisey_signal = np.c_[np.sin(np.linspace(0, 2*np.pi, 365)) * np.random.rand(365)]
data = np.hstack([0.5 * noisey_signal, 3.5 * noisey_signal, 0.01 * noisey_signal, noisey_signal])
table = pd.DataFrame(data=data, columns=categories)

table["date"] = dates
table["month"] = [x.strftime("%B") for x in dates]

table.head()

table.set_index("date")[["a","b","c","d"]].plot(figsize=(10,5))


I am working on the file "Introductions/Tutorial - 1.5 - Scikit-Learn.ipynb".
The first task is:
Best practice suggests that we should always partition our data into train, test and validation sets. It is very common to do this using randomized partitioning.
http://scikit-learn.org/stable/modules/cross_validation.html
Can you write Python code for it?

from sklearn import cross_validation

X_train, X_test, y_train, y_test = cross_validation.train_test_split(X, y, test_size=0.4, random_state=0)


I am working on the file "Introductions/Tutorial - 1.5 - Scikit-Learn.ipynb".
The first task is:
Measuring accuracy alone ignores other important features of a classifier:
http://scikit-learn.org/stable/modules/model_evaluation.html
Can you write Python code for it?

from sklearn.metrics import classification_report

print(classification_report(y_test, clf.predict(X_test), target_names=iris.target_names))


I am working on the file "Introductions/Tutorial - 1.5 - Scikit-Learn.ipynb".
The first task is:
Using a simple linear classifier, lets train the model and evaluate the performance on the test set.
Can you write Python code for it?

from sklearn.svm import LinearSVC

clf = LinearSVC()

clf.fit(X_train, y_train)


I am working on the file "Introductions/Tutorial - 1.5 - Scikit-Learn.ipynb".
The first task is:
We can look at how the classifier performed on the test set using the classifiers "score" method. This reports the mean accuracy of the classifier.
Can you write Python code for it?


I am working on the file "Introductions/Tutorial - Example - 1.3 - Maximize return on investment.ipynb".
The first task is:
Q. Using all the historical closing prices of MSFT shares calculate which 60 day periods yielded a return on investment greater than 25%.
Can you write Python code for it?

ls *.csv

stock = pd.read_csv('msft.csv', parse_dates=['Date']).set_index('Date').sort()
stock.tail()

stock["Open"].plot(figsize=(15, 5));

def ROI(series):
    """Return on investiment"""
    return (series[-1] - series[0]) / series[0]

days = 60
stock["ROI"] = pd.rolling_apply(stock["Close"], days, ROI)
stock.tail()

good_times = stock[stock.ROI > 0.25]
good_times.tail()

ax = stock["Close"].plot(figsize=(15, 5));
ax.plot(good_times.index, good_times["Close"], 'or', alpha=0.3);


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Append
Append is a really useful function with a list - allows values to be added to the list, which is something you'll do all the time in physics coding.
Can you write Python code for it?


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Arithmetic with arrays
Can you write Python code for it?

from numpy import array
a = array([1,2,3,4],int)
print(a+1)

a = array([1,2,3,4],int)
b = array([2,4,6,8],int)
print(a*b)


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Arrays

Lists can be added to with append, arrays have fixed lengths.
Lists can have mixed types, arrays are all the same type (good for debugging).
Arrays can be 2D (e.g. images).
Arrays behave like vectors or matrices (not true with lists).
Can you write Python code for it?

from numpy import zeros
a = zeros(4,float)
print(a)

a=zeros([3,4],float)
print(a)

from numpy import empty
a=empty(4,float)
print(a)

from numpy import zeros
a=zeros([2,2],int)
a[0,1]=1
a[1,0]=-1
print(a)


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Basic Maths
Can you write Python code for it?


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Dropping a Ball from a Tower
Can you write Python code for it?

h = float(input("Enter the height of the tower: "))
g=9.81
t=(2.0*h/g)**0.5
print("The ball will hit the ground after",t,"seconds")


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Example of Using Math Functions
Can you write Python code for it?

from math import sin,cos,pi

print("This program will convert from r,theta to x,y co-ordinates")
r = float(input("Enter r: "))
d = float(input("Enter theta in degrees: "))

theta = d*pi/180
x = r*cos(theta)
y = r*sin(theta)
print("x =",x," y =",y)


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Examples of If and While Statements
Can you write Python code for it?

x = int(input("Enter a whole number no greater than ten: "))
while x>10:
            print("This is greater than ten.  Please try again.")
            x = int(input("Enter a whole number no greater than ten: "))
print("Your number is",x)

x = int(input("Enter a whole number no greater than ten: "))
if x>10:
       print("You entered a number greater than ten.")
       print("Let me fix that for you.")
       x = 10
print("Your number is",x)

n = int(input("Enter an Integer: "))
if n%2==0:
        print("even")
else:
        print("odd")


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Finding the size of an array
Can you write Python code for it?

a = array([[1,2,3],[4,5,6]],int)
print(a.size)
print(a.shape)


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Inputting stuff
Can you write Python code for it?

y = input("Enter the value of y: ")
print(y)

x = input("Enter the value of x: ")
print(x)


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Making an array with arange:
Can you write Python code for it?

from numpy import arange
x=arange(1,11,1)
print(x)

x=arange(1.0,11.0,2.0)
print(x)


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Now check two intergers
Can you write Python code for it?

print("Enter two integers, one even, one odd.")
m=int(input("Enter the first integer: "))
n=int(input("Enter the second integer: "))
while (m+n)%2==0:
        print("One must be even the other odd.")
        m=int(input("Enter the first integer: "))
        n=int(input("Enter the second integer: "))
print("The numbers you chose are", m, "and", n)


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Now linspace - which is similar, but not identical
Can you write Python code for it?

from numpy import linspace
x=linspace(1,100.0,10)
print(x)


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Python Modifiers
Can you write Python code for it?

x,y = 1,2.5
print(x+y)

x +=1
print(x)

x +=1
print(x)

x,y=y,x
print(x,y)


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Quicker method using range
Can you write Python code for it?

R = 1.097e-2
for m in range(1,4):
    print("Series for m = ",m)
    for n in range(m+1,m+6):
        invlambda = R*(1/m**2 - 1/n**2)
        print("   ",1/invlambda, " nm")


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Reading from a File
(Something you will do all the time in physics)
Can you write Python code for it?

from numpy import loadtxt
a = loadtxt("values.txt",float)
print(a)


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Slicing
If r is a list, then r[m:n] is a subset of the list starting with element m, and going up to but not including element n.
Can you write Python code for it?


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Specify the type of the input variable
Can you write Python code for it?

x = float(input("Enter the value of x: "))
print("The value of x is",x)


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
To remove a value from the end of a list:
Can you write Python code for it?


I am working on the file "IntroductiontoPython/Introduction to Python Notes.ipynb".
The first task is:
Variable Types
Can you write Python code for it?


I am working on the file "IntroductiontoPython/UserDefinedFunction.ipynb".
The first task is:
Finding distance to the origin in cylindrical co-ordinates:
Can you write Python code for it?

from math import sqrt, cos, sin

def distance(r,theta,z):
    x = r*cos(theta)
    y = r*sin(theta)
    d = sqrt(x**2+y**2+z**2)
    return d

D = distance(2.0,0.1,1.5)
print(D)


I am working on the file "IntroductiontoPython/UserDefinedFunction.ipynb".
The first task is:
Link to What's in Scipy.constants: https://docs.scipy.org/doc/scipy/reference/constants.html
Library Functions in Maths
(and numpy)
Can you write Python code for it?

x = 4**0.5
print(x)
x = np.sqrt(4)
print(x)


I am working on the file "IntroductiontoPython/UserDefinedFunction.ipynb".
The first task is:
The reason these are useful are for things like the below, where you want to make the same calculation many times. This finds all the prime numbers (only divided by 1 and themselves) from 2 to 100.
Can you write Python code for it?

for n in range(2,100):
    if len(factors(n))==1:
        print(n)


I am working on the file "IntroductiontoPython/UserDefinedFunction.ipynb".
The first task is:
User Defined Functions
Here we'll practice writing our own functions.
Functions start with
python
def name(input):
and must end with a statement to return the value calculated
python
return x
To run a function your code would look like this:
```python
import numpy as np
def name(input)
```

FUNCTION CODE HERE

```python
 return D
y=int(input("Enter y:"))
D = name(y)
print(D)
```
First - write a function to calculate n factorial. Reminder:
$n! = \pi^n_{k=1} k$
~
~
~
~
~
~
~
~
~
~
~
~
~
~
Can you write Python code for it?

def factorial(n):
    f = 1.0
    for k in range(1,n+1):
        f *= k
    return f

print("This programme calculates n!")
n = int(input("Enter n:"))
a = factorial(10)
print("n! = ", a)


I am working on the file "Introductory statistical data analysis with pandas PyData Berlin 2017.ipynb".
The first task is:
All credit for the data, and our many thanks, go to the principal investigators who collected this data
and made it available:

Hungarian Institute of Cardiology. Budapest: Andras Janosi, M.D.
University Hospital, Zurich, Switzerland: William Steinbrunn, M.D.
University Hospital, Basel, Switzerland: Matthias Pfisterer, M.D.
V.A. Medical Center, Long Beach and Cleveland Clinic Foundation: Robert Detrano, M.D., Ph.D.

Online repositories for data used and for this notebook:
http://archive.ics.uci.edu/ml/machine-learning-databases/heart-disease/
https://github.com/AlexandruAgachi/introductory_statistics_tutorial/blob/master/Introductory%20statistical%20data%20analysis%20with%20pandas%20PyData%20Berlin%202017.ipynb
here specify your own directory for where you stored the datasets on your computer:
For me: cd Downloads/Heart disease dataset
note: when downloading the datasets, some people had to rename them manually with a 'txt' ending
the four data files that interest us should be named:
processed.cleveland.data.txt
processed.hungarian.data.txt
processed.switzerland.data.txt
processed.va.data.txt
Data management is an integral part of your research process
Choices you make here will influence your entire study
Can you write Python code for it?

#let's load the first dataset in pandas with the read_csv function. This will create pandas dataframe objects per below.
cleveland = pd.read_csv('processed.cleveland.data.txt', header=None)

cleveland.head(10)

hungary = pd.read_csv('processed.hungarian.data.txt', header=None)

hungary.head(10)

switzerland = pd.read_csv('processed.switzerland.data.txt', header=None)

switzerland.head(10)

va = pd.read_csv('processed.va.data.txt', header=None)

va.head(10)

df = pd.concat([cleveland, hungary, va, switzerland])

df

#we rename all columns to make them more legible
df.columns = ['age', 'sex', 'chest_pain', 'rest_bp', 'cholesterol', 'fasting_bs', 'rest_ecg', 'max_heart_rate', 'exercise_angina', 'st_depression', 'slope', 'fluoroscopy',
             'defect', 'diagnosis']

#let's look at our dataframe. the head(10) option tells pandas to return only the first ten rows of our dataframe.
df.head(10)


I am working on the file "Introductory statistical data analysis with pandas PyData Berlin 2017.ipynb".
The first task is:
Machine learning for data exploration
For me machine learning is simply the branch of statistics that has traditionally focused on working with large,
heterogenous datasets, characterized by numerous variables that interact in nonlinear ways.
It is a collection of tools, just like statistics in general.
An amazing article on traditional parametric statistics vs machine learning is:
"Statistical Modeling: The Two Cultures," Leo Breiman, 2001.
Machine learning can be used for:
1. regression
2. classification
3. feature engineering and/or selection
Accuracy = test error rate. The rate at which an algorithm correctly classifies or estimates.
Goal is to minimize test error rate.
(in linear regression, which we saw before, accuracy was the mean squared error)
(in logistic regression: accuracy = how well the model classifies observations)
Supervised vs unsupervised learning
In supervised learning we work with labeled data
In unsupervised learning we aim to find patterns in unlabeled data
In machine learning we will regularly face the bias-variance trade off:
Variance = change in parameter estimates across different data sets
Bias = how far off model estimated values are from true values
ideally we want low variance and low bias, but they are negatively associated. As one decreases, the other increases.
Generally, as complexity of model increases, this leads to higher variance and lower bias
Simple models will have lower variance, but also be more biased.
We will briefly apply three machine learning algorithms here, which we could use in our exploratory data analysis:
Decision trees
Random forests
Support vector machines
I intend to expand this section in the future. It is difficult today to not take advantage of some of the
machine learning tools available, including when it comes to data exploration
We often forget that some key applications of machine learning tools are to help us gain insights faster than with
more manual methods, and in feature engineering and/or selection
In the data exploration phase, you...
Can you write Python code for it?

df_clean_copy.columns

#although decision trees and random forests are particularly suitable for dealing with categorical variables, and in
#many statistical packages we can input the categorical variables directly, because of how they are implemented in
#scikit learn, in this package we cannot input categorical variables directly. We first have to encode them in a
#process called one hot encoding. This creates a separate column (variable) for each value of our explanatory
#categorical variable. For this we use pandas' get_dummies function

df_sex = pd.get_dummies(df_clean_copy['sex'], prefix = 'sex')

df_chest_pain = pd.get_dummies(df_clean_copy['chest_pain'], prefix = 'chest_pain')
df_fasting_bs = pd.get_dummies(df_clean_copy['fasting_bs'], prefix = 'fasting_bs')
df_rest_ecg = pd.get_dummies(df_clean_copy['rest_ecg'], prefix = 'rest_ecg')
df_exercise_angina = pd.get_dummies(df_clean_copy['exercise_angina'], prefix='exercise_angina')

df_merged = pd.concat([df_clean_copy, df_sex, df_chest_pain, df_fasting_bs, df_rest_ecg, df_exercise_angina], axis=1)

df_merged.columns

df_dt = df_merged[['age', 'sex_0.0', 'sex_1.0', 'chest_pain_1.0', 'chest_pain_2.0', 'chest_pain_3.0', \
                       'chest_pain_4.0', 'rest_bp', 'cholesterol', 'fasting_bs_0.0', 'fasting_bs_1.0', \
                      'rest_ecg_1.0', 'rest_ecg_2.0', 'max_heart_rate', 'exercise_angina_0.0', 'exercise_angina_1.0',\
                       'st_depression', 'diagnosis_binary']]

#here we select as predictors all variables in our dataset expect for the response one
predictors = df_dt.ix[:, df_dt.columns != 'diagnosis_binary']

predictors.head(5)

#we select as the target our response variable
target = df_dt['diagnosis_binary']

#we create our training and testing datasets. Each will have its predictors and its target (response variable).
pred_train, pred_test, tar_train, tar_test = train_test_split(predictors, target, test_size = 0.4)

print(pred_train.shape, pred_test.shape, tar_train.shape, tar_test.shape)

classifier =...


I am working on the file "Introductory statistical data analysis with pandas PyData Berlin 2017.ipynb".
The first task is:
Now we can move to inferential statistics
We call them inferential statistics because we try to infer the population's paramters from our sample
The process is always the same and involves hypothesis testing:
1. define null hypothesis and alternate hypothesis
2. analyze evidence
3. interpret results
Typical H0: there is no relationship between the explanatory and response variable
Typical H1: there is a statistically significant relationship
Bivariate statistical tools:
Three main tools: ANOVA; chi-square; correlation coefficient
Type 1 vs Type 2 errors
Type 1 error: the incorrect rejection of a true null hypothesis
Type 2 error: retaining a false null hypothesis
we will test the null hypothesis that age and diagnosis are not related.
the type of variables we have (explanatory/response and categorical/quantitative for each) determines the type of
statistical tools we will use:
Explanatory categorical and response quantitative: ANOVA
Explanatory categorical and response categorical: Chi Square test
Explanatory quantitative and response categorical: classify/bin explanatory variable and use chi square test
Explanatory quantitative and response quantitative: pearson correlation
a result is statistically significant if it is unlikely to be the result of chance
before performing these analyses, one needs to use the .dropna() function to include only valid data
Can you write Python code for it?

#Going the wrong way?
#In regression analysis you can change your predictor and response variables. This is because they may be correlated,
#in which case X is correlated to Y and by definition Y is correlated with X. There is no directionality implied, which
#is also why you cannot talk of causation, but only of correlation.
test1 = smf.ols(formula = 'age ~ C(diagnosis)', data = df_clean_copy).fit()
print(test1.summary())

#*Whenever we have an explanatory categorical variable with more than two levels, we need to explicitly state this for
#statsmodels. Here we state it by adding 'C'and the variable name in parentheses. By definition, statsmodels then
#converts the first level of the variable into the reference group, therefore showing only n-1 levels in the results.
#each level is in brief a comparison to the reference group. Here the reference group becomes a diagnosis of 0.


I am working on the file "Introductory statistical data analysis with pandas PyData Berlin 2017.ipynb".
The first task is:
Random forests
Random forests, an ensemble learning method, are a more sophisticated method of using trees
Pioneered by Tin Kam Ho, Leo Breiman (who also created 'bagging') and Adele Cutler
For a full discussion, please see
https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm
http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html
Can you write Python code for it?

classifier2 = RandomForestClassifier(n_estimators = 25)
classifier2 = classifier2.fit(pred_train, tar_train)

predictions2 = classifier2.predict(pred_test)

sklearn.metrics.confusion_matrix(tar_test, predictions2)

#let's look at which of our variables the random forest considers most important:
#(these follow the order in which we input them to the random forest i.e. our dataset)
print(classifier2.feature_importances_)

sklearn.metrics.accuracy_score(tar_test, predictions2)


I am working on the file "Introductory statistical data analysis with pandas PyData Berlin 2017.ipynb".
The first task is:
We identified some explanatory variables that are associated with age, but our model overall barely explains 27%
of the variation in the response variable
let's run some diagnostics
Can you write Python code for it?

#Q-Q plot for normality
fig1 = sm.qqplot(model3.resid, line='r')
#red line represents residuals we would expect if model residuals were normally distributed
#our residuals below deviate somewhat from red line, especially at lower and higher quantiles, meaning they do not
#follow a normal distribution. This means the curvilinear association in our model is not fully explained by our model.
#We could add more explanatory variables in this case to try to better explain any curvilinear association.

#simple plot of residuals
stdres = pd.DataFrame(model3.resid_pearson)
plt.plot(stdres, 'o', ls='None')
l = plt.axhline(y=0, color='r')
plt.ylabel('Standardized Residual')
plt.xlabel('Observation Number')
#resid_pearson normalizes our model's residuals
#ls='none' means points will not be connected

#we expect most residuals to fall within 2sd of the mean. More than 2 indicate there are outliers, and more than 3
#extreme outliers.
#if more than 1% of our observations have standardized residuals with an absolute value greater than 2.5, or more than 5%
#have one greater than or equal to 2, there is evidence that the fit of the model is poor. top cause of this is
#ommission of important explanatory variables in our model.

#standardized residuals in linear regression will always be linear, and the line will be horizontal

#normalizing or standardizing residuals amounts to making them have a mean of 0 and sd of 1 so as to fit a normal
#standard distribution

#if residuals show a strong pattern (up, down, polynomial) then it is a good indication of nonlinearity
#in the underlying relationship

#leverage plot
fig4 = sm.graphics.influence_plot(model3, size=8)
print(fig4)

#leverage, always between 0 and 1. At 0, a standardized residual has no influence on our model.
#leverage is a measure of how much influence a specific residual (and therefore observation) has on our model.
#we see that we have extreme outliers, but they are low leverage, meaning they do not have an undue influence on our
#estimation of the regression...


I am working on the file "Introductory statistical data analysis with pandas PyData Berlin 2017.ipynb".
The first task is:
What sort of study was this?
A. Observational or experimental
Observational and experimental data
to know if your data is obersevational or experimental, you ask if the explanatory variable was manipulated or not
In a true experiment, 3 conditions:
1. only one variable is manipulated
2. we have a control group
3. random* assignment (analysis stage in randomized trials: check clss imbalances, and if any found consider incl.
in model as explanatory variables, for statistical control). In theory, in this case one could determine causality.
Quasi experiment:
1. only one variable is manipulated
2. control group
3. no random assignment; groups pre selected. i.e. drug users study.
To improve a quasi experimental design: add confounding variables; have a control group; use a pre-test/post-test
design confounder=control variable=covariate=third variable=lurking variable
in an observational study the regression line only describes data you see. it cannot be used to predict result of
intervention
*randomization works best as your sample size approaches infinity. for small sizes, imbalances in the groups
can occur.
B. Longitudinal or cross sectional
Then data analysis? Wrong!
Background research. Always starts with background research. Domain knowledge.
We quickly find three studies using this dataset:
Detrano et al.: http://www.ajconline.org/article/0002-9149(89)90524-9/pdf
Sundaram and Kakade: http://www2.rmcil.edu/dataanalytics/v2015/papers/Clinical_Decision_Support_For_Heart_Disease.pdf
Soni et al.: http://www.enggjournals.com/ijcse/doc/IJCSE11-03-06-120.pdf
Value the time of domain experts.
Dataset we'll focus on today. look at txt file AND at explanation
First stage in a study is always exploratory data analysis
We typically receive the data without context, in a raw file that we cannot easily interpret
The five steps of exploratory data analysis
1. ?
2. organizing and summarizing the data
3. Looking for important features and patterns
4. Looking for exceptions
5. Interpreting these findings in the context of...
Can you write Python code for it?


I am working on the file "Introductory statistical data analysis with pandas PyData Berlin 2017.ipynb".
The first task is:
number of observations to variables largely exceeds heuristic rule of 5 to 1
see Victoria Stodden (2006) for a discussion of why this is important
https://web.stanford.edu/~vcs/thesis.pdf
convert all variables to numeric ones
Can you write Python code for it?

#mark all variables as numeric data, and signify, for the relevant ones, that they are categorical rather than
#quantitative variables
#errors='coerce' tells pandas to return invalid values as NaN rather than as the input values themselves
#crucial to do this step otherwise subsequent analyses will not work properly. i.e. pandas would interpret missing
#values as strings
df['age'] = pd.to_numeric(df['age'], errors='coerce')
df['sex'] = pd.to_numeric(df['sex'], errors='coerce').astype('category')
df['chest_pain'] = pd.to_numeric(df['chest_pain'], errors='coerce').astype('category')
df['rest_bp'] = pd.to_numeric(df['rest_bp'], errors='coerce')
df['cholesterol'] = pd.to_numeric(df['cholesterol'], errors='coerce')
df['fasting_bs'] = pd.to_numeric(df['fasting_bs'], errors='coerce').astype('category')
df['rest_ecg'] = pd.to_numeric(df['rest_ecg'], errors='coerce').astype('category')
df['max_heart_rate'] = pd.to_numeric(df['max_heart_rate'], errors='coerce')
df['exercise_angina'] = pd.to_numeric(df['exercise_angina'], errors='coerce').astype('category')
df['st_depression'] = pd.to_numeric(df['st_depression'], errors='coerce')
df['slope'] = pd.to_numeric(df['slope'], errors='coerce').astype('category')
df['fluoroscopy'] = pd.to_numeric(df['fluoroscopy'], errors='coerce').astype('category')
df['defect'] = pd.to_numeric(df['defect'], errors='coerce').astype('category')
df['diagnosis'] = pd.to_numeric(df['diagnosis'], errors='coerce').astype('category')


I am working on the file "Introductory statistical data analysis with pandas PyData Berlin 2017.ipynb".
The first task is:
rules for visualizing two variables (recap).
Their order does not matter here, hence Q-C and C-Q use the same visualization rule
C-Q: bivariate bar graph with sns factorplot, categories on x axis, and mean of response variable on y axis
Q-Q: scatterplot with sns regplot
C-C: you can plot them one a time. problem with a bivariate graph is that mean has no meaning in context of a
categorical variable
Can you write Python code for it?

#kind = bar asks for a bar graph and ci=None suppresses error bars
sns.factorplot(x='sex', y='age', kind='bar', data=df_clean, ci=None)
plt.suptitle('Gender vs age')

#categorical explanatory variable 'sex' and quantitative response variable 'rest_bp'
sns.factorplot(x='sex', y='rest_bp', data=df_clean, kind='bar')

df_clean['chest_pain'].dtype

#We use a regplot to plot two quantitative variables, age and cholesterol, while also having a regression line suggesting
#any association present
#we always plot the explanatory variable on the x axis, and the response variable on the y axis
sns.regplot(x='age', y='cholesterol', data=df_clean)

#We see that several observations have a cholesterol of "0". This will need to be investigated subsequently and treated
#as missing values

#how can we gain a better idea of how two categorical variables interact?
df_clean.groupby('sex')['diagnosis'].value_counts()/len(df)

#common statistical boxplot visualization
sns.boxplot(x='cholesterol', y = 'sex', data=df_clean)
plt.suptitle('Cholesterol levels by gender')

#describing the dataset/the variables of the dataset, after we group the values in the dataset by diagnostic category
#the describe function gives us the key statistics for each quantitative variable in the dataset
df_clean.groupby('diagnosis').describe()

#before starting to manipulate the dataset itself, we make a copy, and will work on the copy rather than the original
df_clean_copy = df_clean.copy()


I am working on the file "Introductory statistical data analysis with pandas PyData Berlin 2017.ipynb".
The first task is:
the relationship between chest pain and the diagnosis holds for both levels of the sex variables, hence it is not a
moderator.
we would test for moderator variables in the case of a quantitative response variable the same way:
1. divide the population into the sublevels of the third variables
2. conduct an smf.ols test for each to see if the relationship is statistically significant for each level
identifying a confounding variable does not allow to establish causation, just to get closer to a causal connection.
due to infinite number of possible lurking variables, observational studies cannot rly establish causation
a lurking of confounding variable is a third variable that is associated with both the explanatory and response
variables.
i.e. x=firefighters; y=damage caused by a fire. plot would suggest more firefighters causes more fire damage.
in reality there is a third confounding variable that influences both, seriousness of the fire.
In a study we want to demonstrate that our statistical relationship is valid even after controlling for confounders.
now we will test whether there is a relationship between two quantitative variables, age and cholesterol
for this we use the pearson correlation test
r, going from -1 to 1 only tells us whether the two variables are linearly related. they may be related in nonlinear ways
therefore it's always important to look at r in parallel with a scatterplot of the two variables
r squared is a measure of how much variability in one variable can be explained by the other variable
to calculate the pearson coefficient we need to remove all missing values
Please remember that when two variables are correlated it is possible that:
X causes Y or Y causes X
Z causes both X and Y
X and Y are correlated by chance - a spurious correlation
Can you write Python code for it?

df_clean_copy.columns

scat1 = sns.regplot(x='age', y = 'cholesterol', fit_reg=True, data = df_clean_copy)
plt.xlabel('Age')
plt.ylabel('Cholesterol')
plt.suptitle('Relationship between age and cholesterol')
scat1

#the r coefficient is a measure of association, of how closely points are clustered around a line
#correlations are always between -1 and 1
#r measures solely linear association
#association, not causation
#it is a number without units
#it can mislead in presence of outliers or non linear association -> always draw a scatter plot as well and check visually
#when you look at a scatter plot you look at direction form and strength of relationship
#if you identify a nonlinear association, with one or more curves, then you know you ought to add nonlinear explanatory
#variables to your regression model, to aim to capture this nonlinear association as well.
#ecological correlations based on averages can be misleading and overstate strength of associations for individual
#units
print('association between age and cholesterol')
print(scipy.stats.pearsonr(df_clean_copy['age'], df_clean_copy['cholesterol']))


I am working on the file "Introductory statistical data analysis with pandas PyData Berlin 2017.ipynb".
The first task is:
the results definitely suggest a discussion about class imbalance is in order here.
Can you write Python code for it?

#Diagnosis
#we check the counts per each value of the variable. sort=False tells pandas not to sort the results by values.
#normalize = True tells it to return the relative frequencies rather than the absolute counts
#if we had not cleaned the data, we could add parameter dropna=False so that value_counts does not drop null values
df_clean['diagnosis'].value_counts(sort=False, normalize=True)

sns.countplot(x='diagnosis', data=df_clean)
plt.suptitle('Frequency distribution of diagnosis state')

#Let's look at age now
#the describe request gives us the count, mean, std, min, max, as well as the quartiles for the
#respective value distribution
df_clean['age'].describe()

#sns distplot function combines matplotlib hist() function with sns kdeplot() function.
sns.distplot(df_clean['age'])
plt.suptitle('Distribution of age')

df_clean['max_heart_rate'].describe()

sns.distplot(df_clean['max_heart_rate'])
plt.suptitle('Distribution of maximal heart rate')

sns.swarmplot('diagnosis', data=df_clean)


I am working on the file "Introduzione a Python - Object Oriented Programming.ipynb".
The first task is:
Classes vs. Closures
Can you write Python code for it?

# Definizione di una classe che implementa un "adder"
class Adder(object):
    def __init__(self, n=0):
        self.n = n   # Stato mutabile della classe (DANGER ZONE!!!)
    def __call__(self, m):
        return self.n + m

add5_istanza = Adder(5)
print(add5_istanza(1), add5_istanza(7))

# Definizione di una closure che implementa un "adder"
def make_adder(n=0):
    def adder(m):
        return n + m
    return adder
add5_function = make_adder(5)
print(add5_function(1), add5_function(7))

add5_istanza.n = 1
print(add5_istanza(1), add5_istanza(7))
print(add5_function(1), add5_function(7))


I am working on the file "Introduzione a Python - Object Oriented Programming.ipynb".
The first task is:
Danger ZONE !!!
Can you write Python code for it?


I am working on the file "Introduzione a Python - Object Oriented Programming.ipynb".
The first task is:
NOTA: Con gli oggetti per poter capire l'esecuzione del codice bisogna capire qual'è stata la storia di esecuzione precedente...
Can you write Python code for it?

# Definizione di una classe che implementa un "counter"
class Counter(object):
    def __init__(self, n=0):
        self.n = n
    def __call__(self):
        self.n += 1
        return self.n

counter_istanza = Counter(5)
print(counter_istanza(), counter_istanza())

# Definizione di una closure che implementa un "counter"
def make_counter(n=0):
    def state():
        c = n
        while True:
            c += 1
            yield c

    def counter():
        return next(f)

    f = state()

    return counter

counter_function = make_counter(5)
print(counter_function(), counter_function())

# faccio una modifica apparentemente "innocua"
counter_istanza.n = 'ciao'

print(counter_function(), counter_function())

# ... dopo un po` riutilizzo il counter_istanza
print(counter_istanza())


I am working on the file "Introduzione a Python - Parte seconda.ipynb".
The first task is:
A questo punto possiamo rappresentare la funzione $f(x)=x^3$ nel modo seguente:
Can you write Python code for it?

#%matplotlib inline # Da usare sono una volta
X = range(-10, 11)
Y = H(X)
plot(X,Y)
title("Esempio di $f(x)=x^3$, con $-10 \leq x \leq 10$")


I am working on the file "Introduzione a Python - Parte seconda.ipynb".
The first task is:
Altri tipi di plot utili
Per maggiori informazioni guardare le demo di Matplotlib.
Can you write Python code for it?

# ESEMPIO PRESO DA: http://matplotlib.org/examples/statistics/histogram_demo_cumulative.html
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import mlab

np.random.seed(0)
mu = 200
sigma = 25
n_bins = 50
x = np.random.normal(mu, sigma, size=100)

fig, ax = plt.subplots(figsize=(8, 4))

# plot the cumulative histogram
n, bins, patches = ax.hist(x, n_bins, normed=1, histtype='step',
                           cumulative=True, label='Empirica')

# Add a line showing the expected distribution.
y = mlab.normpdf(bins, mu, sigma).cumsum()
y /= y[-1]

ax.plot(bins, y, 'k--', linewidth=1.5, label='Teorica')

# tidy up the figure
ax.grid(True)
ax.legend(loc='right')
ax.set_title('Cumulative step histograms')
ax.set_xlabel('Annual rainfall (mm)')
ax.set_ylabel('Likelihood of occurrence')

plt.show()

# ESEMPIO PRESO DA: http://matplotlib.org/examples/statistics/histogram_demo_features.html
import numpy as np
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt

np.random.seed(0)

# example data
mu = 100  # mean of distribution
sigma = 15  # standard deviation of distribution
x = mu + sigma * np.random.randn(437)

num_bins = 50

fig, ax = plt.subplots()

# the histogram of the data
n, bins, patches = ax.hist(x, num_bins, normed=1)

# add a 'best fit' line
y = mlab.normpdf(bins, mu, sigma)
ax.plot(bins, y, '--')
ax.set_xlabel('Smarts')
ax.set_ylabel('Probability density')
ax.set_title(r'Histogram of IQ: $\mu=100$, $\sigma=15$')

# Tweak spacing to prevent clipping of ylabel
fig.tight_layout()
plt.show()


I am working on the file "Introduzione a Python - Parte seconda.ipynb".
The first task is:
Importando la libreria math si hanno a disposizione tutta una serie di funzioni standard, tra cui:
pow(x,y), sin(x), cos(x), tan(x), exp(x), log(x, base)
che possono essere disegnate come segue:
Can you write Python code for it?

import math
X = [0.05*x for x in range(-62,62)]
Y = [math.sin(x) for x in X]
plot(X, Y)


I am working on the file "Introduzione a Python - Parte seconda.ipynb".
The first task is:
In Python 3.x la creazione di liste viene eseguita in maniera lazy, ovvero la lista non viene effettivamente creata sino a quando non si itera sui suoi elementi. Questo può essere fatto in due modi:
Can you write Python code for it?

[i for i in Ls]

list(Ls)

# La seconda funzione la chiamiamo MinMaxElement
def MinMaxElement(Xs):
    minimo = Xs[0]
    massimo = minimo
    for x in Xs:
        if x < minimo:
            minimo = x
        if x > massimo:
            massimo = x
    return [minimo, massimo]

print("As =",As)
print("Esempio di chiamata: MinMaxElement(As) =", MinMaxElement(As))
r = MinMaxElement(As)
print("minimo =", r[0], "| massimo =", r[1])

def Potenza(base, esponente=2):
    """
    Calcola la potenza (base)^(esponente).
    Se il secondo parametro non viene passato,
    calcola la potenza quadrata, ovvero (esponente=2).
    """

    return base**esponente

# La terza funziona la chiamiamo semplicemene H,
# ed usiamo la funzione Potenza(base, esponente) per calcolare il cubo
def H(Xs):
    Ls = []
    for x in Xs:
        Ls += [Potenza(x, 3)]
    return Ls

print("Esempio di chiamata: H([2,3,1]) =", H([2,3,1]))


I am working on the file "Introduzione a Python - Parte seconda.ipynb".
The first task is:
NOTA: Python ha un sistema di tipi dinamico. Cosa succede quindi se utilizzo una funzione passandogli dei parametri non previsti?
Can you write Python code for it?

LunghezzaLista(3)

# La prima funzione la chiamiamo `LunghezzaLista`
def LunghezzaLista(Xs):
    """
    Calcola e restituisce il numero di elementi nella lista Xs.
    """
    # Controlla il tipo dell'argomento
    if type(Xs) != list:
        raise TypeError('Ueh, pirla! LunghezzaLista(Xs): La variabile di input Xs non e` una lista.')

    # Itera la lista e conta il numero di suoi elementi
    n = 0
    for x in Xs:
        n += 1
    return n

LunghezzaLista(3)


I am working on the file "Introduzione a Python - Parte seconda.ipynb".
The first task is:
Overloading di un operatore
Un operatore può assumere un significativo diverso in base agli operandi a cui viene applicato. Per esempio:
L'operatore + :
1. tra interi, floats, e numeri complessi esegue l'addizione
2. tra liste, effettua la concatenazione
L'operatore * :
1. tra interi, floats, e numeri complessi esegue la moltiplicazione
2. tra un intero e una stringa: PROVARE (viene chiamato operatore di "repetition")
Can you write Python code for it?


I am working on the file "Introduzione a Python - Parte seconda.ipynb".
The first task is:
Rappresentazione grafica di funzioni scalari
Con i notebook di python si possono facilmente rappresentare graficamente le funzioni scalari.
Per poterlo fare si utilizza la libreria Matplotlib, che si ispira al plot di Matlab.
Per potere usare quella libreria la si deve "importare" con il comando seguente:
Can you write Python code for it?

from matplotlib.pyplot import *
%matplotlib inline

%matplotlib inline
import matplotlib.pyplot as pyplot
# Per disegnare un singolo punto
pyplot.plot([1],[1], 'ro')

# Per disegnare un singolo punto
pyplot.plot([1,2],[2,1], 'b')
pyplot.plot([1],[2], 'ro')
pyplot.plot([2],[1], 'ro')

Xs = range(-100,100)
Ys = list(map(lambda x: math.sin(0.1*x), Xs))
pyplot.plot(Xs, Ys, 'r')


I am working on the file "Introduzione a Python - Parte seconda.ipynb".
The first task is:
È possibile inizializzare un generatore di numeri pseudo casuali con la funzione seed della libreria random.
Can you write Python code for it?

from random import seed
for _ in range(10):
    seed(13)
    print(randint(1,10))


I am working on the file "Introduzione a Python - Parte terza.ipynb".
The first task is:
Classi e Oggetti
Il piu' semplice esempio di built-in oggetto in python e' il tipo complex:
x = 1+3j
In questo caso una semplice implementazione in python potrebbe essere:
Can you write Python code for it?

class MyComplex:

    def __init__(self, Real, Img):
        self.real = Real
        self.img = Img

    def __str__(self):
        s = str(self.real)
        if self.img >= 0:
            s += '+'
        s += str(self.img)+'j'
        return s

    def coniugate(self):
        return MyComplex(self.real, -self.img)

c = MyComplex(1,4)

print(c)

print(c.coniugate())

c.img

type(c)

c.img = 5
print(c.coniugate())


I am working on the file "Introduzione a Python - Parte terza.ipynb".
The first task is:
ESERCIZIO 1: Scrivere una funzione che prende in input due tuple e restituisce una tupla che contiene gli elementi che sono sia nella prima che nella seconda tupla. Scrivere anche una funzione di test che comprenda qualche caso significativo.
Can you write Python code for it?

def Intersect(As, Bs):
    # COMPLETARE
    # COMPLETARE
    return As

def UnitTest():
    # COMPLETARE
    # COMPLETARE
    pass


I am working on the file "Introduzione a Python - Parte terza.ipynb".
The first task is:
ESERCIZIO 2: (obiettivo: intuire il vantaggio di poter avere anche oggi non mutabili)
Si scriva una funzione che prenda in input due liste e che rimuove dalla prima lista ogni elemento che compare nella seconda lista. La funzione non ritorna nulla, ma modifica la prima lista data in input.
Si può usare il metodo L.remove(e) che rimuove dalla lista L il primo elemento uguale a e. Per testare se un elemento appartiene ad una lista basta eseguire il controllo e in L:
if 3 in [1,2,3,4,5]:
    print('ok')
else:
    print('failed')
Can you write Python code for it?

def RimuoviDuplicati(As, Bs):
    # DA COMPLETARE
    # DA COMPLETARE
    pass

L1 = [2, 4, 2, 5, 6, 6, 3, 2, 9, 4]
L2 = [2, 4, 7]
RimuoviDuplicati(L1, L2)
print('L1 =', L1)


I am working on the file "Introduzione a Python - Parte terza.ipynb".
The first task is:
ESERCIZIO 4: Si scriva una funzione simile alla precedente, chiamata ReadAndPlotAge, ma che restituisca una lista contenente l'èta di ciascun passegero (il sesto campo di ciascuna riga). Si usi tale lista per fare due plot (si vedano i notebook della seconda lezione e relativi esempi di uso di matplotlib con la funzione hist):

Si rappresenti l'istogramma dell'età dei passegeri usando 8 bin
Si rappresenti la distribuzione cumulatà dell'età dei passengeri, usando 100 bins

Si testi la funzione usando il train.csv.
Can you write Python code for it?

def ReadAndPlotAge(filename):
    # DA COMPLETARE
    # DA COMPLETARE
    # DA COMPLETARE
    pass


I am working on the file "Introduzione a Python - Parte terza.ipynb".
The first task is:
Lettura di file di testo
Per leggere un file di testo chiamato filename, bisogna prima "aprirlo" con il comando open(filename, mode) usando la modalità mode='r'. Si usi invece mode='w' se si vuole scrivere un nuovo file oppure mode='a' se vuole aggiungere righe in coda ad un file esistente. Il metodo open restituisce un file handle (una sorta di puntatore) per poter accedere al file e manipolarne il contenuto.
ESEMPIO: Il seguente codice, apre il file di testo train-small.csv (il suffisso ".csv" sta per "comma separated values") e ne legge una riga alla volta stampandola a video. Il file contiene le informazioni di 10 passeggeri del Titanic.
Can you write Python code for it?

doc = open('data/train-small.csv', 'r')
for row in doc:
    print(row)
doc.close()


I am working on the file "Introduzione a Python - Parte terza.ipynb".
The first task is:
Metodi utili sui dizionari
I seguenti metodi sono molto utili per usare i dizionari:

len(d): restituisce il numero di elementi nel dizionario d
d.keys(): restituisce una vista delle chiavi del dizionario d
d.values(): restituisce una vista dei valori del dizionario d
key in d: restituisce True se la chiave key è nel dizionario d
d.get(key, value): restituisce d[key] se key è in d, altrimenti restituisce il valore value
d[key] = value: associa il valore value alla chiave key nel dizionario d
del d[key]: rimuove la chiave k dal dizionario d
for key in d: itera sulle chiavi del dizionario d

Insiemi
È possibile definire degli insiemi di elementi usando oggetti di tipo set: questi sono delle collezioni non ordinate di oggetti mutabili (più precisamente su cui è definita una funzione di hash). Sono molto utili quando si deve controllare l'appartenenza di un elemento ad un dato insieme, avere delle collezioni di oggetti senza duplicati, oppure se si devono effetturare delle operazioni su insiemi come unione, intersezione e differenza tra insiemi.
Per maggiori dettagli, si rimanda alla documentazione ufficiale.
ESEMPIO:
Can you write Python code for it?

A = set([2,3,4,2,4,6,7,8])
print(A)
print(3 in A)
B = set([4,4,2,1])
A = A.intersection(B)
print(A)
A = A.difference(B)
print(A)


I am working on the file "Introduzione a Python - Parte terza.ipynb".
The first task is:
Metodi utili sulle stringhe
I seguenti metodi sono tutti molto utili e restituiscono delle nuove stringhe lasciando la stringa iniziale immutata:

s.count(s1): conta qualche volte la stringa s1 è contenuta in s
s.find(s1): restituisce l'indice della stringa s in cui ha trovato per la prima volta la stringa s1; altrimenti restituisce -1
s.rfind(s1): come sopra, ma inizia dalla fine di s (la r sta per reversed)
s.replace(old,new): sostituisce tutte le sottostringhe uguali a old in s con la stringa new
s.strip(): rimuove tutti i caratteri blanks dalla stringa s
s.split(d): suddivide la stringa in sottostringhe usando il carattere d come separatore

Dizionari
I dizionari sono una struttura dati molto utilizzati in python. Sono delle strutture dati che corrispondono a delle liste di coppie (key, value). I dizionari sono degli oggetti mutabili come le liste, ma non sono una sequenza ordinata, e quindi non possono essere indicizzati con dei numeri interi.
La key viene utilizzata come chiave per indicizzare un value. Per esempio:
Dict["hello"] = "ciao"
Abbiamo la chiave "hello" utilizzata per indicizzare l'elemento "ciao" nel dizionario Dict. Per semplicita', potete pensare ai dizionari a come dei vettori di elementi indicizzati da altri oggetti immutabili, come ad esempio delle stringhe.
ESEMPIO:
Can you write Python code for it?

# Creo un dizionario vuoto
Vocabolario = dict()
print(type(Vocabolario))

Vocabolario["keep"] = "stai"
Vocabolario["calm"] = "sereno"
print("Enrico,", Vocabolario["keep"], Vocabolario["calm"])

# E` possibile enumerare i dizionari con un ciclo for
Vocabolario["hello"] = "ciao"
for key in Vocabolario:
    print("chiave:", key, "- valore:", Vocabolario[key])


I am working on the file "Introduzione a Python - Parte terza.ipynb".
The first task is:
NOTA: Per definire una tuple di lunghezza pari a uno, ovvero di un singolo elemento, bisogna usare la strana sintassi (1,): si noti la virgola dopo l'uno.
Can you write Python code for it?


I am working on the file "Introduzione a Python - Parte terza.ipynb".
The first task is:
Operazioni su tuple
Con le tuple è possibile effettuare operazioni di concatenamento, indicizzazione, slicing e repetition.
ESEMPIO:
Can you write Python code for it?

t1 = (2, 'ciao', 5)
t2 = (3.14, 987)
# Concatenazione
print('concatenazione:', t1+t2)
# Indicizzazione
print('indicizzazione:',t1[1])
# Slicing
print('slicing:',(t1+t2)[2:])
# Repetition
print('repetition:',t2*3)


I am working on the file "Introduzione a Python - Parte terza.ipynb".
The first task is:
Per vedere alcuni metodi utili sulle funzioni, ricordarsi di usare il tab destro per l'autocompletion dopo il nome+punto di una variabile:
commento. '+' <tab>
Esempio:
Can you write Python code for it?

#commento.

# INDICIZZAZIONE E SLICING
print(commento[3], commento[-1], commento[2:5])

print(commento.lower())

print(commento.upper())


I am working on the file "Introduzione a Python - Parte terza.ipynb".
The first task is:
Si noti che in effetti ogni riga del file contiene una serie di valori separati da una virgola e che è come se ogni riga contennesse già il carattere \n che stampa delle righe vuote.
Metodi utili per manipolare files
I seguenti metodi sono molto utili per manipolare dei files:

open(filename, mode): apre il file chiamato "filename" in modalità mode (lettura, scrittura, aggiunta)
pf.read(): restituisce un'unica stringa contenente l'intero contenuto del file
pf.readline(): restituisce la prossima riga associata al file handler pf
pf.readlines(): restituisce una lista di stringhe, una per ciascuna riga contenuta nel file
pf.write(s): scrive la stringa s nel file pf
pf.writeLines(S): scrive ciascun elemento della collezione di stringhe contenute in S in una riga diversa del file pf
pf.close(): chiude il file handler pf

ESERCIZIO 3: Si scriva una funzione che prenda in input un nome di un file, apra il file in lettura, e su ogni riga:

rimuove i caratteri \n. Suggerimento: usare la funzione replace()
rimuove le doppie virgolette "
divide la riga in sottostringhe. Suggerimento: usare la funzione split()

Infine la funzione deve restituire il numero di persone che nell'elenco hanno il secondo campo pari a 1. Si testi la funzione usando il file train.csv che si trova nella cartella data.
Can you write Python code for it?

def ReadAndCount(filename):
    # DA COMPLETARE
    return 0

print(ReadAndCount('data/train.csv'))


I am working on the file "Introduzione a Python - Prima parte.ipynb".
The first task is:
ATTENZIONE: La divisione tra numeri interi non considera il resto
Can you write Python code for it?


I am working on the file "Introduzione a Python - Prima parte.ipynb".
The first task is:
Comandi utili nel workspace
Per controllare le variabili in memoria (nel workspace):
Can you write Python code for it?


I am working on the file "Introduzione a Python - Prima parte.ipynb".
The first task is:
IMPORTANTE: Abituarsi ad usare il tasto tab per avere il completamento automatico
Can you write Python code for it?

# Aggiungere un elemento ad una lista
As.append(27)
print(As)

As.remove(7)
print(As)

As.insert(4, 12)
print(As)

# Invertire l'ordine di una lista
As.reverse()
print(As)

# Ordinamento come funzione
Bs = sorted(As)
print(As, Bs)

# Ordinamento "IN_PLACE"
As.sort()
print(As)


I am working on the file "Introduzione a Python - Prima parte.ipynb".
The first task is:
NOTA: In Python 2.7 questa espressione viene valutata a 0.666666 in quanto z vienve valutata con una divisione tra interi
Can you write Python code for it?


I am working on the file "Introduzione a Python - Prima parte.ipynb".
The first task is:
Numeri Reali (?)
Can you write Python code for it?

y=1.0

print(y)

type(x), type(y)

type(x+y)


I am working on the file "Introduzione a Python - Prima parte.ipynb".
The first task is:
Python supporta la possibilità di usare i nomi di variabili usati nella definizione di una funzione per passare gli argomenti (chiamati keyword arguments). Questo si differenza dalla tipica notazione posizionale di molti linguaggi di programmazione, tipo il C, e permette  di rendere il codice più leggibile e di evitare ambiguità nel passare gli argomenti ad una funzione. Si noti infine che nella definizione della funzione Potenza2 è stato definito un parametro di default (default parameter values) per definire il valore dell'esponente.
Liste
Can you write Python code for it?

As = [5,3,2,8,7,13]
print(As)
print(type(As))

# help(list)


I am working on the file "Introduzione a Python - Prima parte.ipynb".
The first task is:
Type conversion o type cast
Sono usati per convertire un tipo in un altro, in maniera dinamica. Per convertire un oggetto in un dato tipo, basta usare il nome del tipo come se fosse una funzione. Esempi:
Can you write Python code for it?


I am working on the file "Introduzione a Python - Prima parte.ipynb".
The first task is:
Vogliamo ora scrivere la funzione:
$$f : \mathbb{C} \times \mathbb{C} \rightarrow \mathbb{C}$$
con
$$f(x,y) = x^y$$
In python scriviamo:
Can you write Python code for it?

def Potenza(base, esponente):
    """
    Calcola la potenza (base)^(esponente).
    """

    return base**esponente

Potenza(2, 3)

def Potenza2(base, esponente=2):
    """
    Calcola la potenza (base)^(esponente).
    Se il secondo parametro non viene passato,
    calcola la potenza quadrata, ovvero (esponente=2).
    """

    return base**esponente

print(Potenza2(14))
print(Potenza2(14,3))

Potenza2(esponente=4, base=10)