I am working on the file "November 28, 2016/Covered in class 11-28-2016.ipynb".
The first task is:
List comprehensions
List comphrensions take the form:
[expression for variable in sequence if condition]
Can you write Python code for it?

"""Create a list from a list comprehension."""
lst = [i for i in range(1, 11) if i <= 3]
print(lst)


I am working on the file "November 28, 2016/Covered in class 11-28-2016.ipynb".
The first task is:
Notice especially the "punctuation" separating the various parts of the comprehension:
[... for ... in ... if ...]
Without the if clause:
[... for ... in ...]
These patterns are functionally equivalent to the more familiar list accumulation pattern.
Can you write Python code for it?

"""Create a list by accumulating values."""
lst = []
for i in range(1, 10):
    if i <= 3:
        lst.append(i)
print(lst)

"""Create a list by accumulating values without if condition."""
lst = []
for i in range(1, 11):
    lst.append(i)
print(lst)


I am working on the file "November 28, 2016/Covered in class 11-28-2016.ipynb".
The first task is:
You can translate many for loops into list comprehensions:

lst = []
for i in range(1, 10):          becomes          lst = [i for i in range(1, 10) if i <= 3]
    if i <= 3:
        lst.append(i)

lst = []
for i in range(1, 10):          becomes          lst = [i for i in range(1, 10)]
    lst.append(i)

doubled = []
for i in range(1, 10):          becomes          doubled = [i * 2 for i in range(1, 10)]
    doubled.append(i * 2)

Generally:

lst = []
for variable in sequence:       becomes          lst = [expression for variable in sequence if condition]
    if condition:
        lst.append(expression)

list traversal
The sequential accessing of each element in a list.
Can you write Python code for it?

"""List traversal using a for loop."""
lst = [1, 2, 3]
for i in lst:
    print(i)

"""List traversal using a for loop and an index."""
lst = [1, 2, 3]
for i in range(len(lst)):
    print(lst[i])


I am working on the file "November 28, 2016/Covered in class 11-28-2016.ipynb".
The first task is:
clone
To create a new object that has the same value as an existing object. Copying a reference to an object creates an alias but doesn’t clone the object.
Can you write Python code for it?

"""Clone a list to get a new object with the same values."""
lst = [1, 2, 3]
alias = lst     # create an alias to lst
clone = lst[:]  # clone lst
print("Is alias an alias of lst?", alias is lst)
print("Is clone an alias of lst?", clone is lst)
print("Do lst, alias, and clone all have the same values?", lst == alias == clone)


I am working on the file "November 28, 2016/Covered in class 11-28-2016.ipynb".
The first task is:
element
One of the values in a list (or other sequence). The bracket operator selects elements of a list.
Can you write Python code for it?

"""Access an element of a list by using square brackets with an index."""
lst = [1, 2, 3]
print("The third element of lst:", lst[2])


I am working on the file "November 28, 2016/Covered in class 11-28-2016.ipynb".
The first task is:
index
An integer variable or value that indicates an element of a list.
Can you write Python code for it?

"""By varying the index, you can access different elements of a list."""
lst = [1, 2, 3]
i = 0    # i holds the index value
print("The element of lst at index 0 is", lst[i])
i = 1
print("The element of lst at index 1 is", lst[i])


I am working on the file "November 28, 2016/Covered in class 11-28-2016.ipynb".
The first task is:
list
A collection of objects, where each object is identified by an index. Like other types str, int, float, etc. there is also a list type-converter function that tries to turn its argument into a list.
Can you write Python code for it?

"""Create a list from a list literal."""
lst = [1, 2, 3]
print(lst)

"""Create a list using list()."""
lst = list(range(1, 4))
print(lst, type(lst))


I am working on the file "November 28, 2016/Covered in class 11-28-2016.ipynb".
The first task is:
modifier
A function which changes its arguments inside the function body. Only mutable types can be changed by modifiers.
Can you write Python code for it?

import time
delay = 3

def title_modifier(lst):
    """This function will change the list passed to it."""
    for i in range(len(lst)):
        lst[i] = ' '.join([word[0].upper() + word[1:].lower() for word in lst[i].split()])

places = ["new york", "kansas city", "los angeles", "seattle"]

print("Places before being modified:", places, "\n")
time.sleep(delay)

print("Calling title_modifier(places)...", "\n")
return_value = title_modifier(places)
time.sleep(delay)

print("The return value of title_modifier(places) is", return_value, "\n")
time.sleep(delay)

print("Places after being modified:", places, "\n")

import time
delay = 3

def title_modifier(lst):
    """This function will change the list passed to it."""
    for i in range(len(lst)):
        place = lst[i]
        words = place.split()
        title_cased_words = []
        for word in words:
            first_letter = word[0]
            rest_of_word = word[1:]
            title_cased_words.append(first_letter.upper() + rest_of_word.lower())
        lst[i] = ' '.join(title_cased_words)

places = ["new york", "kansas city", "los angeles", "seattle"]

print("Places before being modified:", places, "\n")
time.sleep(delay)

print("Calling title_modifier(places)...", "\n")
return_value = title_modifier(places)
time.sleep(delay)

print("The return value of title_modifier(places) is", return_value, "\n")
time.sleep(delay)

print("Places after being modified:", places, "\n")


I am working on the file "November 28, 2016/Covered in class 11-28-2016.ipynb".
The first task is:
mutable data type
A data type in which the elements can be modified. All mutable types are compound types. Lists are mutable data types; strings are not.
Can you write Python code for it?

"""Lists are mutable."""
cities = ["Albuquerque", "Chicago", "Paris"]

print("Cities before changing cities[1]:", cities)

cities[1] = "Tokyo"

print("Cities after changing cities[1]:", cities)

"""Strings are not mutable"""
cities = "Albuquerque Chicago Paris"

print("Cities as a string:", cities)

cities[1] = "Tokyo"

print("This will never print because it is an error to try to change part of a string")


I am working on the file "November 28, 2016/Covered in class 11-28-2016.ipynb".
The first task is:
nested list
A list that is an element of another list.
Can you write Python code for it?

"""Lists of lists (nested lists).

The format of hourly_forecasts is a list of days:

hourly_forecasts = [day1, day2, ..., dayN]

Each day has a name and a list of conditions associated with that day for the hours of 1-3 pm:

day = [name, [conditions_at_1pm, conditions_at_2pm, conditions_at_3pm]]

Therefore, hourly_forecasts is, in part, a list of lists of lists.
"""
hourly_forecasts = [["Monday", ["Sunny", "Sunny", "Partly Cloudy"]],
                   ["Tuesday", ["Cloudy", "Cloudy", "Cloudy"]],
                   ["Wednesday", ["Partly Cloudy", "Sunny", "Sunny"]]]
for day in hourly_forecasts:
    name = day[0]
    hourly_conditions = day[1]
    i = 1
    for conditions in hourly_conditions:
        print("The conditions at " + str(i) + "pm on " + name + " are forecast to be " + conditions)
        i += 1


I am working on the file "November 28, 2016/Covered in class 11-28-2016.ipynb".
The first task is:
sequence
Any of the data types that consist of an ordered collection of elements, with each element identified by an index.
Can you write Python code for it?

"""Lists are sequences."""
lst = [1, 2, 3]

print("lst:    ", lst)
print("lst[0]: ", lst[0])

"""Tuples are sequences."""
t = (1, 2, 3)

print("t:    ", t)
print("t[0]: ", t[0])


I am working on the file "November 28, 2016/Covered in class 11-28-2016.ipynb".
The first task is:
side effect
A change in the state of a program made by calling a function that is not a result of reading the return value from the function. Side effects can only be produced by modifiers.
Can you write Python code for it?

def change_forecast(forecast):
    """This modifier function changes the value of the
    forecast without returning anything.
    """
    for i in range(len(forecast)):
        forecast[i] = "Cloudy"

forecast = ["Sunny", "Sunny", "Sunny"]
print("The forecast is", forecast)

return_value = change_forecast(forecast)

print("The return_value from calling change_forecast(forecast) is", return_value)

print("The forecast is", forecast)


I am working on the file "Nuclear/PoissonML.ipynb".
The first task is:
I'm modelling the spectral lines as skewed Gaussians. I can't be bothered to code up a skewed Gaussian myself, so I'm using the predefined function from the lmfit library: https://lmfit.github.io/lmfit-py/builtin_models.html#skewedgaussianmodel
$$
\frac{A}{\sigma \sqrt{2\pi}}{\rm e}^{-(x-\mu)^2/2\sigma^2}\left{1+{\rm erf}\left[\frac{\gamma(x-\mu)}{\sigma\sqrt{2}}\right]\right}
$$
This has four free parameters: the amplitude, $A$; the mean position, $\mu$; the width, $\sigma$; and the skew, $\gamma$. It's the values of those parameters that we'll be fitting for.
Can you write Python code for it?

def gauss_fn(p0, x):

    # extract parameters from p0 array:
    amp,mu,sigma,gamma = p0

    # initiate model:
    model = SkewedGaussianModel()

    # set initial parameter values
    params = model.make_params(amplitude=amp, center=mu, sigma=sigma, gamma=gamma)

    # evaluate model at x-positions:
    ymod = model.eval(params=params,x=x)

    return ymod


I am working on the file "Nuclear/PoissonML.ipynb".
The first task is:
Let's plot it up and see how it looks:
Can you write Python code for it?

pl.subplot(111)
pl.scatter(xvals,ymeas,c='b')
pl.plot(xvals,yvals,c='r')
pl.show()


I am working on the file "Nuclear/PoissonML.ipynb".
The first task is:
Now I'll simulate the line at a range of x-positions:
Can you write Python code for it?

xvals = np.arange(0,500,1)
yvals = gauss_fn(p0,xvals)


I am working on the file "Nuclear/PoissonML.ipynb".
The first task is:
Now let's try and fit the pretend data to recover the input parameters. The data have Poisson noise, so we need a Poisson loglikelihood.
Note: the $\chi^2$ function is a Gaussian loglikelihood
The Poisson probability density function looks like this:
$$
\frac{\lambda^x {\rm e}^{-\lambda}}{x!}
$$
so the loglikelihood looks like this:
$$
\log \left( \frac{\lambda^x {\rm e}^{-\lambda}}{x!} \right)
$$
$$
= \log \lambda^x + \log {\rm e}^{-\lambda} - \log x!
$$
$$
= x\log \lambda - \lambda - \left{x\log (x) - x  \right}
$$
where I have used the Stirling approximation for the factorial in the final term.
You can also find this here: https://www.statlect.com/fundamentals-of-statistics/Poisson-distribution-maximum-likelihood
Can you write Python code for it?

def lnlike(p0, x, y):

    # get model for these parameters:
    ymod = gauss_fn(p0,x)

    # Poisson loglikelihood:
    ll = np.sum(ymod[np.where(ymeas!=0)]*np.log(ymeas[np.where(ymeas!=0)])) - np.sum(ymeas) - np.sum(ymod[np.where(ymod!=0)]*np.log(ymod[np.where(ymod!=0)]) - ymod[np.where(ymod!=0)])

    return ll


I am working on the file "Nuclear/PoissonMLRealData.ipynb".
The first task is:
https://lmfit.github.io/lmfit-py/builtin_models.html#skewedgaussianmodel
Can you write Python code for it?

def gauss_fn(p0, x):

    amp,mu,sigma,gamma = p0
    model = SkewedGaussianModel()

    #amp*=sigma*np.sqrt(2*np.pi)

    # set initial parameter values
    params = model.make_params(amplitude=amp, center=mu, sigma=sigma, gamma=gamma)
    ymod = model.eval(params=params,x=x)

    return ymod

def lnlike(p0, x, y):

    # get model for these parameters:
    ymod = gauss_fn(p0,x)

    # Poisson loglikelihood:
    ll = np.sum(ymod[np.where(ymeas!=0)]*np.log(ymeas[np.where(ymeas!=0)])) - np.sum(ymeas) - np.sum(ymod[np.where(ymod!=0)]*np.log(ymod[np.where(ymod!=0)]) - ymod[np.where(ymod!=0)])

    return ll

# use maxvalue to guess amplitude:
a0 = np.max(ymeas)

# use position of maxvalue to guess mean:
m0 = xvals[np.argmax(ymeas)]

# just guess width:
s0 = 10.

# just guess skew:
g0 = -2.

# adjust the amplitude for the normalisation factor:
a0*=s0*np.sqrt(2*np.pi)

print a0,m0,s0,g0
p0 = np.array([a0,m0,s0,g0])

bnds = ((-np.infty,np.infty), (-np.infty,np.infty), (0.1,np.infty), (-10., 10.))

nll = lambda *args: -lnlike(*args)

result = op.minimize(nll, p0, bounds=bnds, args=(xvals, ymeas))
p1 = result["x"]

print p1

yfit = gauss_fn(p1,xvals)
pl.subplot(111)
pl.scatter(xvals,ymeas)
pl.plot(xvals,yfit,c='r')
pl.show()

print lnlike(p1, xvals, ymeas)

res = (yfit - ymeas)/np.sqrt(ymeas)
pl.plot(res)
pl.ylabel(r"$\sigma$")


I am working on the file "Nuclear_test_seismic_data.ipynb".
The first task is:
Clearly a substantially more powerful explosion.
Frequency analysis
Let's get a little more data, 15 minutes' worth, and we'll query the high-resolution channel, HHZ.
Can you write Python code for it?

st = client.get_waveforms("IC", "MDJ", "*", "HHZ", t, t + 15*60)
st.plot()

z = st.traces[0]
NFFT = 2048
Fs = st.traces[0].meta.sampling_rate  # Sample rate in Hz

plt.figure(figsize=(15, 5))
Pxx, freqs, bins, im = plt.specgram(z, NFFT=NFFT, Fs=Fs, noverlap=int(0.9*NFFT), cmap='viridis', vmin=-50, vmax=50)
plt.show()


I am working on the file "Nuclear_test_seismic_data.ipynb".
The first task is:
How does the test compare to previous tests?
Let's compare to the test in September 2016:
Can you write Python code for it?


I am working on the file "Nuclear_test_seismic_data.ipynb".
The first task is:
This st thing is the ObsPy stream.
Can you write Python code for it?


I am working on the file "Nuclear_test_seismic_data.ipynb".
The first task is:
We can plot the tests together:
Can you write Python code for it?

label = "{network}.{station}.{location}.{channel}".format(**st.traces[0].meta)

plt.figure(figsize=(15,5))
plt.plot(st.traces[0], label="3 Sept 2017")
plt.plot(st1.traces[0], label="9 Sept 2016")
plt.text(-200, 2.5e6, label, size=20)
plt.legend()
plt.grid()
plt.show()


I am working on the file "Nuclear_test_seismic_data.ipynb".
The first task is:
We requested the broadband channel, but some stations have lots of others:

LHZ: long period ~1 Hz, with Z for vertical, or N or E for horizontal.
BHZ: broadband 10–80 Hz, with Z for vertical, or N or E for horizontal.
HHZ: broadband 80–250 Hz, with Z for vertical, or N or E for horizontal.
Some stations have loads of others, see Appendix A in the SEED Manual.

Where is the station?
To do this you need basemap and cartopy in your environment:
conda install basemap
conda install cartopy
Can you write Python code for it?

inventory = client.get_stations(network="IC", station="MDJ")
inventory.plot()
plt.show()

inventory


I am working on the file "Nuclear_test_seismic_data.ipynb".
The first task is:
We want station MDJ in eastern China in the IC network.
Can you write Python code for it?

from obspy import UTCDateTime

t = UTCDateTime("2017-09-03_03:30:00")
st = client.get_waveforms("IC", "MDJ", "00", "BHZ", t, t + 5*60)
st.plot()


I am working on the file "Nuclear_test_seismic_data.ipynb".
The first task is:
We'll use the awesome ObsPy library. Following the instructions in the ObsPy documentation, and using IRIS as the data source (per Steve's tweets)...
Can you write Python code for it?

from obspy.clients.fdsn import Client

client = Client("IRIS")


I am working on the file "NumPy.ipynb".
The first task is:
Comparand timpul mediu de executie a sumei listelor si respectiv array-urilor 1D in numpy realizam
performanta acestei operatii in numpy.
Operatiile cu vectori si matrici in numpy sunt vectorizate.

Produsul Pr dintre un scalar a si un array A se calculeaza invocand Pr=a*A;

Daca doua array-uri A, B sunt de tip  (m,n), atunci  P=A*B este array-ul de tip (m,n)
si elemente $p_{ij}=a_{ij}*b_{ij}$ (inmultirea se efectueaza element cu element):
Can you write Python code for it?

a=2.5
v=np.arange(1,5)
print a*v

w=np.linspace(2,6,4)
print v*w


I am working on the file "NumPy.ipynb".
The first task is:
Conversia listelor si tuple-urilor in array-uri
Can you write Python code for it?

t=(3,4,7)# acesta este un tuple
ta=np.asarray(t)

print t
print ta

L=[5,6,8,11]

print type(L), L
La=np.asarray(L, float)
print type(La), La


I am working on the file "NumPy.ipynb".
The first task is:
Daca $A\in\mathbb{R}^{m\times p}, B\in\mathbb{R}^{p\times n}$, atunci produsul lor se calculeaza
apeland, la fel np.dot(A,B):
Can you write Python code for it?

A=np.arange(12).reshape((4,3))
B=np.arange(1,10).reshape((3,3))
C=np.dot(A,B)
print A
print B
print C


I am working on the file "NumPy.ipynb".
The first task is:
Daca dorim sa generam o matrice unitate cu elemente de tip int atunci se apeleeaza np.eye astfel:
Can you write Python code for it?


I am working on the file "NumPy.ipynb".
The first task is:
Daca n este un numar intreg pozitiv, atunci x=np.arange(n) este vectorul de elemente
0,1, ..., n-1.

xx=np,arange(m,n), cu m<n, este vectorul de elemente m, m+1, ..., n-1

v=np.arrange(a, b, h), cu  a<b si h>0,  este vectorul de elemente v[i]= a+i*h, unde i ia valori de la 0 pana la
valoarea maxima, imax, pentru care a+imax*h<b.

a este valoarea de start, b valoarea de stop, iar h este pasul cu care se divide intervalul
    [a,b] prin puncte echidistante.

In definitia lui y avem  a=3, b=12, h=2

In definitia lui z, a=0, b=1, h=0.3

Pasul h poate fi si negativ. In acest caz, x=np.arange(b,a,h), cu b>a,  genereaza vectorul
de elemente x[i]=b+i*h, unde i ia valori de la 0 pana la valoarea maxima imax pt care b+imax*h>a.
Can you write Python code for it?

x=np.arange(10,4,-1)
print x

x=np.linspace(np.pi/6, 2*np.pi/3, 10)
print x


I am working on the file "NumPy.ipynb".
The first task is:
Daca v este un array 1D, deci un vector linie, apeland v.transpose() nu se genereaza un vector coloana:
Can you write Python code for it?


I am working on the file "NumPy.ipynb".
The first task is:
Determinantul si inversa unui array 2D:
Can you write Python code for it?

A=np.array([[2, -5], [1, -7]], float)
print A
d=np.linalg.det(A)
print 'Determinantul lui A este d=', np.linalg.det(A)
B=np.linalg.inv(A)
print  B


I am working on the file "NumPy.ipynb".
The first task is:
Functiile matematice din modulul math nu sunt insa vectorizate:
Can you write Python code for it?

X=[0, 0.5, 1,1.5, 2]
Y=math.sin(X)

from IPython.core.display import HTML
def  css_styling():
    styles = open("./custom.css", "r").read()
    return HTML(styles)
css_styling()


I am working on the file "NumPy.ipynb".
The first task is:
Functiile matematice implementate in modulul math sunt reimplementate in numpy, dar in timp ce cele din math au ca argument
un scalar, cele din numpy admit ca argument array-uri.
Mai precis, daca $x=[x_0, x_1, \ldots, x_{n-1}]$, $n\geq 1$, este un array 1D, atunci $f(x)$ este array-ul
$f(x)=[f(x_0), f(x_1), \ldots, f(x_{n-1})]$ si analog, daca $A$ este un array 2D  de dimensiuni (m,n), atunci  $B=f(A)$ este un array 2D de aceleasi dimensiuni si elementele sale sunt $b_{ij}=f(a_{ij})$.
Mai precis, sa experimentam evaluarea functiei $f(x)=\sin(x)$ apeland math.sin si np.sin, transmitand argumente scalare si respectiv vectoriale:
Can you write Python code for it?

print math.sin(math.pi/3)
print np.sin(np.pi/3)

x=np.arange(0,np.pi, 0.5)
y=np.sin(x)
print x
print y.round(3)# afiseaza elementele lui y  cu 3 zecimale


I am working on the file "NumPy.ipynb".
The first task is:
In declararea si initializarea array-urilor in acest fel se poate omite  dtype (data type) scriind doar tipul:
Can you write Python code for it?

u=np.array([-3,0,2], int)
B=np.array([[1,5],[-3,4]], float)
print u
print B


I am working on the file "NumPy.ipynb".
The first task is:
In fisierul  vector.txt  fiecare numar este scris pe o alta linie.
Evident ca ele se pot scrie pe aceeasi linie separate de spatiu.
Citind un astfel de fisier se genereaza un vector, care apoi se poate redimensiona la un array 2D, folosind metoda reshape ( a se vedea mai jos).
Can you write Python code for it?

A=np.loadtxt("Myfiles/matrice.txt")
print A.shape
print  A[:,1]


I am working on the file "NumPy.ipynb".
The first task is:
In fisierul matrice.txt datele sunt inregistrate matricial, adica pe  5 linii si in fiecare linie sunt 5 valori.
Functia numpy.fromfile construieste un array dintr-un fisier de date text sau binar, continand date de tip cunoscut.
Exemple de utilizare a acestei functii vom da mai  tarziu.
Metode asociate unui  array
O metoda asociata unui array se apeleaza astfel: NumeArray.metoda.
NumeArray.shape returneaza un tuple cu un element pentru un array 1D, respectiv cu doua elemente pentru un array 2D. Din punctul de vedere al Algebrei liniare, elementul/elementele tuple-ului indica spatiul
 $\mathbb{R}^n$, respectiv $\mathbb{R}^{m\times n}$, caruia apartine  array-ul respectiv.
Can you write Python code for it?

v=np.arange(2,21,5)
t=v.shape
print v, ',t=', t

A=np.array([[-2,3,5], [1,4,0]], int)
print 'A are dimensiunile', A.shape


I am working on the file "NumPy.ipynb".
The first task is:
Lista functiilor din modulul math se gaseste aici:
http://docs.python.org/2/library/math.html
Modulul cmath contine functii de argument complex. Functiile din acest modul le vom folosi in semestrul 2.
O colectie de module se numeste pachet.
Pachetele de baza pentru calcul stiintific in Python sunt numpy si scipy.
Numele numpy este derivat de la numerical Python, iar scipy de la scientific Python.
Pachetul numpy contine printre altele:

definitia array-ului n-dimensional

functii care actioneaza asupra array-urilor

modulul linalg  ce contine functii specifice algebrei liniare, dar si alte module

instrumente pentru a integra cod C/C++

Cursul de Algebra liniara opereaza cu vectori si matrici.

Un vector din $\mathbb{R}^n$ este reprezentat
in numpy printr-un array de dimensiune 1 (il vom referi in continuare ca array 1D).

O matrice $A\in\mathbb{R}^{m\times n}$ este un array 2-dimensional (2D)

Modalitati de a crea array-uri
Pentru a crea si  manipula array-uri intr-un program Python sau in IPython Notebook, se importa pachetul numpy:
Can you write Python code for it?

import numpy as np

v=np.array([-2, 1, 4], dtype=int)
w=np.array([3.14, -1, 0, 5.2], dtype=float)
print v, w


I am working on the file "NumPy.ipynb".
The first task is:
Norma unui vector:
Can you write Python code for it?

v=np.array([-1,2,5], float)
no=np.linalg.norm(v) # norm este o functie din modulul linalg al pachetului numpy
print no, ', Versorul lui v=', v/no


I am working on the file "NumPy.ipynb".
The first task is:
NumPy
NumPy is the fundamental package for scientific computing with Python. It contains among other things:

a powerful N-dimensional array object
sophisticated (broadcasting) functions
useful linear algebra, Fourier transform, and random number capabilities

Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container of generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily integrate with a wide variety of databases.
Library documentation: http://www.numpy.org/
NumPy’s main object is the homogeneous multidimensional array. It is a table of elements (usually numbers), all of the same type, indexed by a tuple of positive integers. In NumPy dimensions are called axes. The number of axes is rank.
For example, the coordinates of a point in 3D space [1, 2, 1] is an array of rank 1, because it has one axis. That axis has a length of 3. In the example pictured below, the array has rank 2 (it is 2-dimensional). The first dimension (axis) has a length of 2, the second dimension has a length of 3.
NumPy’s array class is called ndarray. It is also known by the alias array. Note that numpy.array is not the same as the Standard Python Library class array.array, which only handles one-dimensional arrays and offers less functionality. The more important attributes of an ndarray object are:
np.array
list as argument to array
Can you write Python code for it?


I am working on the file "NumPy.ipynb".
The first task is:
Pentru a estima timpul de rulare al functiei sumaL am folosit comanda magica %timeit
din IPython. %timeit calculeaza media timpului de rulare a functiei. Media este calculata pentru
mai multe executii ale functiei.
In numpy suma a a doua array-uri este implementata element cu element si operatia de adunare se noteaza simplu v+w:
Can you write Python code for it?

n=1000
v=np.empty(n, int)
w=np.empty(n, int)
for i in range(n):
    v[i]=2*i
    w[i]=3*i+1

%timeit  s=v+w


I am working on the file "NumPy.ipynb".
The first task is:
Redimensionarea unui array
Can you write Python code for it?


I am working on the file "NumPy.ipynb".
The first task is:
Remarcam ca NumeArray.reshape(t) returneaza un array de dimensiunile setate de tuple-ul t, iar array-ul initial ramane neschimbat.
Metoda reshape este foarte utila atunci cand un array este construit din date citite dintr-un fisier.
Transpusa unui array  2D se poate genera apeland metoda NumeArray.transpose()
Can you write Python code for it?

A=np.arange(10).reshape((2,5))
print A

print A.transpose()


I am working on the file "NumPy.ipynb".
The first task is:
With noise: damped least squares
Can you write Python code for it?

# Add noise.
dc = G.dot(m)  # Python < 3.5
dc = G @ m     # Python 3.5

# Add to the data.
s = 1
d = dc + s * np.random.random(dc.shape)

# Use the second form.
I = np.eye(N)
µ = 2.5  # We can use Unicode symbols in Python 3, just be careful
m_est = G.T.dot(la.inv(G.dot(G.T) + µ * I)).dot(d)
d_pred = G.dot(m_est)

# Or, in Python 3.5
m_est = G.T @ la.inv(G @ G.T + µ * I) @ d
d_pred = G @ m_est

plot_all(m, d, m_est, d_pred)


I am working on the file "NumPy.ipynb".
The first task is:
With noise: damped least squares with first derivative regularization
Can you write Python code for it?

convmtx([1, -1], 5)

W = convmtx([1,-1], M)[:,:-1]  # Skip last column


I am working on the file "NumPy.ipynb".
The first task is:
deoarece in Python un vector este unic afisat,  pe linie.
Pentru a  a afisa/manipula coordonatele lui v pe coloana se defineste v ca un array 2D, cu o singura linie,
astfel:
Can you write Python code for it?

v=np.array([[1,2,4]], float)
print v
print v.transpose()
print v.shape, v.transpose().shape


I am working on the file "NumPy.ipynb".
The first task is:
np.arange
Can you write Python code for it?

# To create sequences of numbers, NumPy provides a function analogous to range that returns arrays instead of lists.


I am working on the file "NumPy.ipynb".
The first task is:
np.asarray(Ob) converteste obiectul Ob (in cazul de mai sus un tuple si respectiv o lista de date numerice) la un array.
Can you write Python code for it?

A=np.array([[1,2,3],[4,5,6]], float)
B=np.copy(A)
print A
print B

A[1][2]=-7
print A
print B


I am working on the file "NumPy.ipynb".
The first task is:
np.copy(A) creaza un nou array cu aceleasi elemente ca si A. Observam ca modificand un element in A si afisand din nou pe A si B,
B nu s-a schimbat, deci este un array pentru care s-a alocat o zona separata de memorie.
De interes deosebit sunt functiile np.arange si np.linspace, care creaza array-uri speciale:
Can you write Python code for it?

n=6
x=np.arange(6)
print x
xx=np.arange(2,9)
print xx

y=np.arange(3,12,2)
print y

z=np.arange(0,1, 0.3)
print z


I am working on the file "NumPy.ipynb".
The first task is:
np.empty
Can you write Python code for it?

#function empty creates an array whose initial content is random and depends on the state of the memory


I am working on the file "NumPy.ipynb".
The first task is:
np.ones creaza un array cu toate elementele egale cu 1.
Can you write Python code for it?

v=np.array([1,4,6,-3], int)
z=np.zeros_like(v, float)
print v, z


I am working on the file "NumPy.ipynb".
The first task is:
np.zeros(n)  si respectiv np.zeros((m,n)) creaza array-uri si initializeaza elementele lor cu zero, implicit de tip float.
Can you write Python code for it?

u=np.ones(4, int)
M=np.ones((3,3), int)
print u
print M


I am working on the file "NumPy.ipynb".
The first task is:
np.zeros_like(A) creaza un array de aceleasi dimensiuni ca A, dar de elemente 0.
np.eye(n) genereaza matricea unitate $I_n$, avand elementele de tip implicit float:
Can you write Python code for it?


I am working on the file "NumPy.ipynb".
The first task is:
shape
np.zeros
Can you write Python code for it?

#The function zeros creates an array full of zeros


I am working on the file "NumPy.ipynb".
The first task is:
v si w sunt vectori carora li s-au precizat coordonatele si tipul elementelor, respectiv int
si float
Can you write Python code for it?


I am working on the file "NumPy.ipynb".
The first task is:
x este un vector din $\mathbb{R}^{4}$, care eventual se va initializa cu elemente de tip float, iar B este un array de 2 linii si 3 coloane (vezi tuple-ul (2,3) in definitie).
Can you write Python code for it?

y=np.zeros(3)
C=np.zeros((4,4))
print y
print C


I am working on the file "NumPy.ipynb".
The first task is:
x=np.linspace(a, b, n) este vectorul ale carui elemente se obtin divizand intervalul [a,b]
prin n puncte echidistante. In exemplul precedent, intervalul $[\pi/6, 2\pi/3]$ a fost divizat
prin 10 puncte echidistante.
np.arange(a, b, h) si np.linspace(a,b,n) se vor folosi pentru a genera grafice de functii, pentru a trasa curbe.
Crearea unui array dintr-un fisier
Exemplele de mai sus au fost ilustrate prin array-uri cu numar redus de elemente. In problemele practice
se lucreaza cu array-uri 1D sau 2D cu numar mare de elemente. In acest caz acestea se citesc din fisiere.
Functia numpy.loadtxt citeste un fisier text ce contine
date din care se pot construi array-uri.
Can you write Python code for it?

v=np.loadtxt("Myfiles/vector.txt")
print v
print v.size


I am working on the file "NumPy/(5)convenience_function.ipynb".
The first task is:
1. 股票相关性分析
本例子中， 我们使用2个示例数据集提供收盘价数据，第一家公司是BHP Billiton(BHP)，其主要业务是石油、金属和钻石开采；第二家公司是Vale(VALE)，也是一家金属开采业公司，他们有部分业务是重合的。我们来分析一下他们的股票相关性。
Can you write Python code for it?

# 首先读入两只股票的收盘价，并计算收益率
bhp_cp = np.loadtxt('BHP.csv', delimiter=',', usecols=(6,), unpack=True)
vale_cp = np.loadtxt('VALE.csv', delimiter=',', usecols=(6,), unpack=True)
bhp_returns = np.diff(bhp_cp) / bhp_cp[:-1]
vale_returns = np.diff(vale_cp) / vale_cp[:-1]


I am working on the file "NumPy/(5)convenience_function.ipynb".
The first task is:
3. 计算净额成交量
成交量表示价格波动的大小，净额成交量（On-Balance Volume)是由当日收盘价、前一天的收盘价以及当日成交量计算得出的。
以前一日为基期计算当日的OBV的值（可认为基期的OBV的值为0）。若当日收盘价高于前一日收盘价，则本日OBV等于基期OBV加上当日成交量，否则减去当日成交量。
我们需要在成交量前面乘上一个有收盘价变化决定的正负号。
Can you write Python code for it?

cp, volume = np.loadtxt('BHP.csv', delimiter=',', usecols=(6,7), unpack=True)
change = np.diff(cp)
print "Change:", change


I am working on the file "NumPy/(5)convenience_function.ipynb".
The first task is:
4. 模拟交易过程
使用vectorize函数可以减少你的程序中使用循环的次数，NumPy中的vectorize函数相当于Python中的map函数。我们用它来计算单个交易日的利润。
Can you write Python code for it?

# 读入数据
# op is opening price,hp is the highest price
# lp is the lowest price, cp is closing price
op, hp, lp, cp = np.loadtxt('BHP.csv', delimiter=',', usecols=(3,4,5,6), unpack=True)


I am working on the file "NumPy/(5)convenience_function.ipynb".
The first task is:
使用NumPy的sign函数返回每个元素的正负号。
Can you write Python code for it?

signs = np.sign(change)
print "Signs:\n", signs


I am working on the file "NumPy/(5)convenience_function.ipynb".
The first task is:
我们尝试以比开盘价稍低一点的价格买入股票。如果这个价格不在当日的股价范围内，则尝试买入失败，没有获利，也没有亏损，我们返回0。否则，我们将以当日收盘价卖出，所获得的利润即买入卖出的差价。
Can you write Python code for it?

def calc_profit(op, high, low, close):
    # 以开盘价买入,这里不考虑买入多少股
    buy = op

    if low < buy < high:
        return (close-buy) / buy
    else:
        return 0

# 矢量化一个函数，这样可以避免使用循环
func = np.vectorize(calc_profit)
profits = func(op, hp, lp, cp)
print 'Profits:\n', profits

# 我们选择非零利润的交易日并计算平均值
real_trades = profits[profits != 0]
print 'Number of trades:\n', len(real_trades), round(100.0 * len(real_trades)/len(cp), 2),"%"
print "Average profit/loss % :", round(np.mean(real_trades) * 100, 2)

# 选择正盈利的交易日并计算平均利润
winning_trades = profits[profits > 0]
print "Number of winning trades", len(winning_trades), round(100.0*len(winning_trades)/len(cp),2),"%"
print "Average profit %", round(np.mean(winning_trades) *100, 2)

# 选择负盈利的交易日并计算平均损失
losing_trades = profits[profits < 0]
print "Number of winning trades", len(losing_trades), round(100.0*len(losing_trades)/len(cp),2),"%"
print "Average profit %", round(np.mean(losing_trades) *100, 2)


I am working on the file "NumPy/(5)convenience_function.ipynb".
The first task is:
求极值
Can you write Python code for it?

# 极值位于导数为0的点
der = np.polyder(poly)
print "Dervative:\n", der
# 得到多项式导函数的系数

# 求出导函数的根，即找出原多项式函数的极值点
print "Extremas: ", np.roots(der)

# 通过argmax和argmin函数找到最大最小值点来检查结果
vals = np.polyval(poly, t)
print "Maximum index: ", np.argmax(vals)
print "Minimum index: ", np.argmin(vals)


I am working on the file "NumPy/(5)convenience_function.ipynb".
The first task is:
理想情况下，BHP和VALE股票收盘价的差价越小越好。在极限情况下，差值可以在某个点为0。用roots函数找到拟合多项式函数在什么时候达到0。
Can you write Python code for it?


I am working on the file "NumPy/(5)convenience_function.ipynb".
The first task is:
用相关系数来度量两只股票的相关程度。相关系数的取值范围在-1到1之间，一组数据域自身的相关系数为1.使用corrcoef函数计算相关系数。
Can you write Python code for it?

# 使用corrcoef计算更加精确
print 'Correlation coefficient:\n', np.corrcoef(bhp_returns, vale_returns)


I am working on the file "NumPy/(5)convenience_function.ipynb".
The first task is:
相关系数矩阵关于对角线对称，BHP与VALE的相关系数等于VALE和BHP的相关系数。看起来0.68的相关系数表示他们的相关程度似乎不是很强。
判断两只股票的价格走势是否同步
如果它们的差值偏离了平均差值2倍于标准差的距离，则认为他们走势不同步
Can you write Python code for it?

difference = bhp_cp - vale_cp
avg = np.mean(difference)
dev = np.std(difference)
# 检查最后一次收盘价是否在同步状态
print "Out of sync : ", np.abs(difference[-1] - avg) > 2*dev


I am working on the file "NumPy/(5)convenience_function.ipynb".
The first task is:
绘制拟合曲线
Can you write Python code for it?

plot(t, bhp_cp-vale_cp)
plot(t, vals)
show()


I am working on the file "NumPy/.ipynb_checkpoints/(1)numpy_array_basis1-checkpoint.ipynb".
The first task is:
2. 数组的索引和切片
Can you write Python code for it?

a = arange(9)
#下标0-7， 以2为步长
print a[:7:2]

#以负数下标翻转数组
print a[::-1]
print a[::-2]


I am working on the file "NumPy/.ipynb_checkpoints/(1)numpy_array_basis1-checkpoint.ipynb".
The first task is:
dtype类的属性
Can you write Python code for it?

t = dtype('float64')
print t.char
print t.type
print t.str


I am working on the file "NumPy/.ipynb_checkpoints/(1)numpy_array_basis1-checkpoint.ipynb".
The first task is:
flatten 也是展平
flatten函数会请求分配内存来保存结果，而ravel函数只是返回数组的一个视图（view）
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(1)numpy_array_basis1-checkpoint.ipynb".
The first task is:
resize和reshape函数功能一样
但resize会直接改变所操作的数组
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(1)numpy_array_basis1-checkpoint.ipynb".
The first task is:
str属性可以给出数据类型的字符串表示，该字符串的首个字符表示字节序，然后是字符编码，然后是所占字节数
字节序是指位长为32和64的字（word）存储的顺序，包括大端序(big-endian)和小端序(little-endian)。
大端序是将最高位字节存储在最低的内存地址处，用>表示；与之相反，小端序是将最低位字节存储在最低的内存地址处，用<表示。
创建自定义数据类型
自定义数据类型是一种异构数据类型，可以当做用来记录电子表格或数据库中一行数据的结构。
下面我们创建一种自定义的异构数据类型，该数据类型包括一个用字符串记录的名字、一个用整数记录的数字以及一个用浮点数记录的价格。
Can you write Python code for it?

t = dtype([('name', str_, 40), ('numitems', int32), ('price', float32)])

t

t['name']

itemz = array([('Meaning of life DVD', 32, 3.14), ('Butter', 13, 2.72)], dtype=t)

itemz[1]


I am working on the file "NumPy/.ipynb_checkpoints/(1)numpy_array_basis1-checkpoint.ipynb".
The first task is:
transpose转置矩阵
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(1)numpy_array_basis1-checkpoint.ipynb".
The first task is:
复数不能转换成整数和浮点数
Numpy数组中每一个元素均为相同的数据类型，现在给出单个元素所占字节
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(1)numpy_array_basis1-checkpoint.ipynb".
The first task is:
数组的shape属性返回一个元祖(tuple)，元组中的元素即NumPy数组每一个维度的大小。
1. 创建多维数组
array函数可以依据给定的对象生成数组。
给定的对象应是类数组，如python的列表、numpy的arange函数
Can you write Python code for it?

m = array([arange(2), arange(2)])

print m
print m.shape
print type(m)
print type(m.shape)


I am working on the file "NumPy/.ipynb_checkpoints/(1)numpy_array_basis1-checkpoint.ipynb".
The first task is:
用元组设置维度
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(1)numpy_array_basis1-checkpoint.ipynb".
The first task is:
选取元素
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(10)assert_function-checkpoint.ipynb".
The first task is:
2. assert_approx_equal断言近似相等
如果两个数字的近似程度没有达到指定的有效数字要求，assert_approx_equal函数就抛出异常。
触发条件为：abs(actual-expected) >= 10**(significant-1)
Can you write Python code for it?

# 指定8位有效数字
print "Significance 8", np.testing.assert_approx_equal(0.123456789,0.123456780, significant=8)

# 指定9位有效数字
# 抛出异常
print "Significance 9", np.testing.assert_approx_equal(0.123456789,0.123456780, significant=9)


I am working on the file "NumPy/.ipynb_checkpoints/(10)assert_function-checkpoint.ipynb".
The first task is:
3. assert_array_almost_equal断言数组近似相等
如果两个数组中元素的近似程度没有达到指定的精度要求，assert_array_almost_equal函数将抛出异常。
触发条件为： |expected - actual| < 0.5 * 10^(-decimal)
Can you write Python code for it?

print "Decimal 8", np.testing.assert_array_almost_equal([0, 0.123456789], [0, 0.123456780], decimal=8)

print "Decimal 9", np.testing.assert_array_almost_equal([0, 0.123456789], [0, 0.123456780], decimal=9)


I am working on the file "NumPy/.ipynb_checkpoints/(10)assert_function-checkpoint.ipynb".
The first task is:
4. assert_array_equal断言数组相等
如果两个数组对象不相同，assert_array_equal函数将抛出异常。两个数组相等必须形状一致且元素也严格相等，允许数组存在NaN元素。
比较数组也可以使用assert_allclose函数。该函数有参数atol(absolute tolerance,绝对容差限)和rtol(relative tolerance，相对容差限)。对于两个数组a和b，将测试是否满足以下条件：| a - b | <= (atol+rtol*b)
Can you write Python code for it?

print "Pass", np.testing.assert_allclose([0,0.123456789,np.nan],[0,0.123456780,np.nan], rtol=1e-7,atol=0)

print "Fail", np.testing.assert_array_equal([0,0.123456789,np.nan],[0,0.123456780,np.nan])


I am working on the file "NumPy/.ipynb_checkpoints/(10)assert_function-checkpoint.ipynb".
The first task is:
多ULP浮点数的比较
assert_array_max_ulp函数可以指定ULP的数量作为允许的误差上界。参数maxulp接受整数作为ULP数量的上限，默认值为1。
Can you write Python code for it?

print "one eps", np.testing.assert_array_max_ulp(1.0, 1.0+eps)

print "two eps", np.testing.assert_array_max_ulp(1.0, 1.0+eps*2,maxulp=2)


I am working on the file "NumPy/.ipynb_checkpoints/(2)numpy_array_basis2-checkpoint.ipynb".
The first task is:
1. 数组的组合
Numpy数组有水平组合、垂直组合和深度组合等多种组合方式，我们将使用vstack、dstack、hstack、column_stack、row_stack以及concatenate函数完成数组的组合
Can you write Python code for it?

a = arange(9).reshape(3,3)
b = 2*a
print 'a: \n',a
print 'b: \n',b


I am working on the file "NumPy/.ipynb_checkpoints/(2)numpy_array_basis2-checkpoint.ipynb".
The first task is:
2. 数组的分割
Numpy数组可以进行水平、垂直和深度分割，相关的函数有hsplit、vsplit、dsplit和split。我们可以将数组分割成相同大小的子数组，也可以指定原数组中需要分割的位置。
（1）水平分割
Can you write Python code for it?

a

#将数组沿水平方向分割为3个相同大小的子数组
hsplit(a, 3)

#调用split函数，指定参数axis=1
split(a, 3, axis=1)


I am working on the file "NumPy/.ipynb_checkpoints/(2)numpy_array_basis2-checkpoint.ipynb".
The first task is:
4. 数组的转换
tolist函数将numpy数组转换成python列表
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(2)numpy_array_basis2-checkpoint.ipynb".
The first task is:
astype函数可以在转换数组时指定数据类型
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(2)numpy_array_basis2-checkpoint.ipynb".
The first task is:
（1）水平组合
将ndarray对象构成的元组作为参数，传给hstack函数
或者使用concatenate函数实现该功能
Can you write Python code for it?

hstack((a, b))

concatenate((a,b), axis=1)


I am working on the file "NumPy/.ipynb_checkpoints/(2)numpy_array_basis2-checkpoint.ipynb".
The first task is:
（2）垂直分割
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(2)numpy_array_basis2-checkpoint.ipynb".
The first task is:
（3）深度组合
深度组合，就是将一系列数组沿着纵轴(深度)方向进行层叠组合。
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(2)numpy_array_basis2-checkpoint.ipynb".
The first task is:
（4）列组合
column_stack函数。
对于一维数组将按列方向进行组合。
对于二维数组，效果和hstack一样。
Can you write Python code for it?

oned = arange(2)
twice_oned = 2 * oned

column_stack((oned, twice_oned))

column_stack((a,b))

column_stack((a,b)) == hstack((a,b))


I am working on the file "NumPy/.ipynb_checkpoints/(2)numpy_array_basis2-checkpoint.ipynb".
The first task is:
（5）行组合
按行方向进行组合。对于两个一维数组，将直接层叠起来合成一个二维数组；对于二维数组，row_stack和vstack的效果一样。
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(3)common_functions1-checkpoint.ipynb".
The first task is:
2.2 算术平均
使用mean函数计算算术平均
Can you write Python code for it?

mean = np.mean(c)
print "mean = ",mean

print "mean = ", c.mean()


I am working on the file "NumPy/.ipynb_checkpoints/(3)common_functions1-checkpoint.ipynb".
The first task is:
3. 求最大最小值和取值范围
步骤：读入最高价和最低价，使用max和min函数得到最大最小值。
Can you write Python code for it?

h,l = np.loadtxt('data.csv', delimiter=',', usecols=(4,5), unpack=True)
print 'hightest = ', np.max(h)
print 'lowest = ', np.min(l)


I am working on the file "NumPy/.ipynb_checkpoints/(3)common_functions1-checkpoint.ipynb".
The first task is:
4. 计算中位数
计算收盘价的中位数
Can you write Python code for it?

closing_price = np.loadtxt('data.csv', delimiter=',', usecols=(6,), unpack=True)
print 'median = ', np.median(closing_price)


I am working on the file "NumPy/.ipynb_checkpoints/(3)common_functions1-checkpoint.ipynb".
The first task is:
5. 计算方差
方差体现变量变化的程度，股价变动过于剧烈的股票一定会给持有者制造麻烦。
Can you write Python code for it?

print "variance = ", np.var(closing_price)

#手动求方差
print 'variance from definition = ', np.mean( (closing_price-c.mean())**2 )


I am working on the file "NumPy/.ipynb_checkpoints/(3)common_functions1-checkpoint.ipynb".
The first task is:
TWAP是Time0Weighted Average Price，时间加权平均价格，其基本思想是最近的价格重要性大一些，所以我们应该对近期的价格给以较高的权重。
我们使用arange函数创建从0递增的自然数序列，自然数的个数即为收盘价的个数。
Can you write Python code for it?

t = np.arange(len(c))
print "twap = ",np.average(c, weights=t)


I am working on the file "NumPy/.ipynb_checkpoints/(3)common_functions1-checkpoint.ipynb".
The first task is:
data.csv文件是苹果公司的历史股价数据。第一列为股票代码，第二列为dd-mm-yyyy格式的日期，第三列为空，随后各列依次是开盘价(4)、最高价(5)、最低价(6)和收盘价(7)，最后一列为当日的成交量(8)。
loadtxt函数中，usecols参数为一个元组，以获得第7字段至第8字段的数据，也就是股票的收盘价和成交量数据。
unpack参数设置为True，意思是分拆存储不同列的数据，即分别将收盘价和成交量的数组赋值给变量c和v。
用usecols中的参数指定我们感兴趣的数据列，并将unpack参数设置为True使得不同列的数据分别存储。
Can you write Python code for it?

c

v

#选择第4列，开盘价
opening_price = np.loadtxt('data.csv', delimiter=',', usecols=(3,), unpack=True)

print opening_price


I am working on the file "NumPy/.ipynb_checkpoints/(3)common_functions1-checkpoint.ipynb".
The first task is:
numpy中ptp函数可以计算数组的取值范围。该函数返回的是数组元素最大值和最小值的差值，即max(array)-min(array)。
Can you write Python code for it?

print 'Spread high price : ', np.ptp(h)
print 'Spread low price : ', np.ptp(l)


I am working on the file "NumPy/.ipynb_checkpoints/(3)common_functions1-checkpoint.ipynb".
The first task is:
对数组进行排序，之后再去中位数
Can you write Python code for it?

sorted_closing = np.msort(closing_price)
print "sorted_closing_price = ", sorted_closing

#先判断数组的个数是奇数还是偶数
N = len(closing_price)
median_ind = (N-1)/2
if N & 0x1 :
    print "median = ", sorted_closing[median_ind]
else:
    print "median = ", (sorted_closing[median_ind]+sorted_closing[median_ind+1])/2


I am working on the file "NumPy/.ipynb_checkpoints/(4)common_functions2——stock_analysis-checkpoint.ipynb".
The first task is:
1. 股票收益率
简单收益率是指相邻两个价格之间的变化率，而对数收益率是指所有价格取对数后两两之间的差值。由于对数收益率是两两价格相除再去对数，所以其可以用来衡量价格收益率。
投资者最感兴趣的是收益率的方差或标准差，这代表了投资风险的大小。
1.1 计算简单收益率
diff函数计算离散差分，计算收益率，需要用差值除以前一天的价格。
Can you write Python code for it?

#cp means closing price
cp = np.loadtxt('data.csv', delimiter=',', usecols=(6,), unpack=True)
cp_diff = np.diff(cp)
rate_of_returns = cp_diff / cp[:-1]

cp

#去掉最后一个
#从开始到倒数第二个
cp[:-1]

rate_of_returns

print "standard deviation = ", np.std(rate_of_returns)


I am working on the file "NumPy/.ipynb_checkpoints/(4)common_functions2——stock_analysis-checkpoint.ipynb".
The first task is:
1.2 对数收益率
取对数要注意事先检查数据中不包含0和负值，确保输入满足定义域条件
Can you write Python code for it?

log_rate_of_returns = np.diff(np.log(cp))

log_rate_of_returns


I am working on the file "NumPy/.ipynb_checkpoints/(4)common_functions2——stock_analysis-checkpoint.ipynb".
The first task is:
1.3 计算波动率
波动率volatility是对价格变动的一种度量。年波动率等于对数收益率的标准差除以其均值，在除以交易日倒数的平方根，通常交易日去252天。
Can you write Python code for it?

annual_volatility = np.std(log_rate_of_returns) / np.mean(log_rate_of_returns)
annual_volatility = annual_volatility / np.sqrt(1. / 252.)
print annual_volatility


I am working on the file "NumPy/.ipynb_checkpoints/(4)common_functions2——stock_analysis-checkpoint.ipynb".
The first task is:
2. 日期分析
我们读入收盘价数据，根据星期几来切分数据，计算平均价格，最后找出一周的那一天的平均收盘价最高
Numpy是面向浮点数运算的，对日期处理需要专门的方法
loadtxt函数中有一个特定的参数converters，这是数据列到转换函数之间映射的字典。
Can you write Python code for it?

#convert funtion
import datetime
def datestr2num(s):
    return datetime.datetime.strptime(s, "%d-%m-%Y").date().weekday()

#字符串首先会按照指定形式"%d-%m-%Y"转换成一个datetime对象，随后datetime对象被转换成date对象，最后调用weekday方法返回一个数字
#0代表星期一，6代表星期天

#cp means closing price
dates, cp = np.loadtxt('data.csv', delimiter=',', usecols=(1,6), converters={1:datestr2num}, unpack=True)
print "Dates = ", dates

#保存各工作日的平均收盘价
averages = np.zeros(5)


I am working on the file "NumPy/.ipynb_checkpoints/(4)common_functions2——stock_analysis-checkpoint.ipynb".
The first task is:
5. 绘制趋势线
趋势线，是根据股价走势图上很多所谓的枢轴点绘成的曲线，即描绘的是价格变化的趋势。
Can you write Python code for it?

%matplotlib inline
from matplotlib.pyplot import plot
from matplotlib.pyplot import show

#我们假设枢轴点的位置是最高价、最低价、收盘价的均值
hp, lp, cp = np.loadtxt('data.csv', delimiter=',', usecols=(4,5,6), unpack=True)
pivots = (hp+lp+cp) / 3
print "Pivots:\n", pivots


I am working on the file "NumPy/.ipynb_checkpoints/(4)common_functions2——stock_analysis-checkpoint.ipynb".
The first task is:
apply_along_axis函数会调用一个自定义函数，作用于每一个数组元素上。在调用apply_along_axis时提供我们自定义的函数summarize，并指定要作用的轴或维度的编号以及函数的参数
Can you write Python code for it?

#这里indices是三个array
def summarize(indices, open, high, low, close):
    monday_open = open[indices[0]]
    week_high = np.max(np.take(high, indices))
    week_low = np.min(np.take(low, indices))
    friday_close = close[indices[-1]]
    return ("APPL", monday_open, week_high, week_low, friday_close)

#这里参数1，代表作用于每一行，一共三行，得到三行结果
#相当于axis为1的时候进行计算
week_summary = np.apply_along_axis(summarize, 1, weeks_indices, op, hp, lp, cp)
print "Week summary", week_summary


I am working on the file "NumPy/.ipynb_checkpoints/(4)common_functions2——stock_analysis-checkpoint.ipynb".
The first task is:
sqrt函数中的除法运算必须使用浮点数才能得到正确结果。
Can you write Python code for it?

print "Monthly volatility", annual_volatility * np.sqrt(1. / 12.)


I am working on the file "NumPy/.ipynb_checkpoints/(4)common_functions2——stock_analysis-checkpoint.ipynb".
The first task is:
where函数会根据指定的条件返回所有满足条件的数组元素的索引值；take函数根据这些索引值从数组中取出相应的元素。
Can you write Python code for it?

for i in xrange(5):
    indices = np.where(dates == i)
    prices = np.take(cp, indices)
    avg = prices.mean()
    print "Day", i, "prices", prices, "Average", avg
    averages[i] = avg

#找出哪个工作日的平均收盘价最高，哪个最低
top = averages.max()
bottom = averages.min()
print "Highest average", top
print "Top day of the week", np.argmax(averages)
print "Lowest average", bottom
print "Bottom day of the week", np.argmin(averages)


I am working on the file "NumPy/.ipynb_checkpoints/(4)common_functions2——stock_analysis-checkpoint.ipynb".
The first task is:
得到收益率为正的情况，where函数可以根据制定条件返回数组的索引值
Can you write Python code for it?

pos_return_indices = np.where(log_rate_of_returns > 0)
print 'Indices with positive returns', pos_return_indices


I am working on the file "NumPy/.ipynb_checkpoints/(4)common_functions2——stock_analysis-checkpoint.ipynb".
The first task is:
补充： ndarray对象的几种方法

clip，修剪
compress，压缩
prod，相乘
cumprod，累积乘积
Can you write Python code for it?

#clip方法
#将所有比给定最大值还大的元素全部设为给定最大值
#将所有比给定最小值还小的元素全部设为最小值
a = np.arange(5)
print "a = ",a
print "Clipped:", a.clip(1,3)

#compress方法
#返回一个根据给定条件筛选后的数组
a = np.arange(5)
print a
print "Compressed: ", a.compress(a > 2)

# prod方法计算数组中所有元素的乘积
# 这种方法可以计算阶乘 factorial
b = np.arange(1,6)
print  "b = ", b
print "factorial: ", b.prod()

# cumprod方法，计算数组元素的累积乘积
print "factorials array: ", b.cumprod()


I am working on the file "NumPy/.ipynb_checkpoints/(5)convenience_function-checkpoint.ipynb".
The first task is:
1. 股票相关性分析
本例子中， 我们使用2个示例数据集提供收盘价数据，第一家公司是BHP Billiton(BHP)，其主要业务是石油、金属和钻石开采；第二家公司是Vale(VALE)，也是一家金属开采业公司，他们有部分业务是重合的。我们来分析一下他们的股票相关性。
Can you write Python code for it?

# 首先读入两只股票的收盘价，并计算收益率
bhp_cp = np.loadtxt('BHP.csv', delimiter=',', usecols=(6,), unpack=True)
vale_cp = np.loadtxt('VALE.csv', delimiter=',', usecols=(6,), unpack=True)
bhp_returns = np.diff(bhp_cp) / bhp_cp[:-1]
vale_returns = np.diff(vale_cp) / vale_cp[:-1]


I am working on the file "NumPy/.ipynb_checkpoints/(5)convenience_function-checkpoint.ipynb".
The first task is:
2. 多项式拟合
NumPy中的plotfit函数可以用多项式去拟合一系列数据点，无论这些数据点是否来自连续函数都适用。
Can you write Python code for it?

# 用三次多项式去拟合两只股票收盘价的差价
t = np.arange(len(bhp_cp))
poly = np.polyfit(t, bhp_cp-vale_cp, 3)
print "Polynomial fit\n", poly

# 用刚才得到的多项式对象，推断下一个值
print "Next value: ", np.polyval(poly, t[-1]+1)


I am working on the file "NumPy/.ipynb_checkpoints/(5)convenience_function-checkpoint.ipynb".
The first task is:
4. 模拟交易过程
使用vectorize函数可以减少你的程序中使用循环的次数，NumPy中的vectorize函数相当于Python中的map函数。我们用它来计算单个交易日的利润。
Can you write Python code for it?

# 读入数据
# op is opening price,hp is the highest price
# lp is the lowest price, cp is closing price
op, hp, lp, cp = np.loadtxt('BHP.csv', delimiter=',', usecols=(3,4,5,6), unpack=True)


I am working on the file "NumPy/.ipynb_checkpoints/(5)convenience_function-checkpoint.ipynb".
The first task is:
使用NumPy的sign函数返回每个元素的正负号。
Can you write Python code for it?

signs = np.sign(change)
print "Signs:\n", signs


I am working on the file "NumPy/.ipynb_checkpoints/(5)convenience_function-checkpoint.ipynb".
The first task is:
使用Numpy的piecewise函数获取数组元素的正负。piecewise(分段)，可以根据给定取值，得到分段。
Can you write Python code for it?

pieces = np.piecewise(change, [change<0, change>0], [-1,1])
print "Pieces:\n", pieces

# 检查两次输出是否一致
print "Arrays equal?", np.array_equal(signs, pieces)

# OBV值的计算依赖于前一日的收盘价
print "On balance volume: \n", volume[1:]*signs


I am working on the file "NumPy/.ipynb_checkpoints/(5)convenience_function-checkpoint.ipynb".
The first task is:
我们尝试以比开盘价稍低一点的价格买入股票。如果这个价格不在当日的股价范围内，则尝试买入失败，没有获利，也没有亏损，我们返回0。否则，我们将以当日收盘价卖出，所获得的利润即买入卖出的差价。
Can you write Python code for it?

def calc_profit(op, high, low, close):
    # 以开盘价买入,这里不考虑买入多少股
    buy = op

    if low < buy < high:
        return (close-buy) / buy
    else:
        return 0

# 矢量化一个函数，这样可以避免使用循环
func = np.vectorize(calc_profit)
profits = func(op, hp, lp, cp)
print 'Profits:\n', profits

# 我们选择非零利润的交易日并计算平均值
real_trades = profits[profits != 0]
print 'Number of trades:\n', len(real_trades), round(100.0 * len(real_trades)/len(cp), 2),"%"
print "Average profit/loss % :", round(np.mean(real_trades) * 100, 2)


I am working on the file "NumPy/.ipynb_checkpoints/(5)convenience_function-checkpoint.ipynb".
The first task is:
求极值
Can you write Python code for it?

# 极值位于导数为0的点
der = np.polyder(poly)
print "Dervative:\n", der
# 得到多项式导函数的系数

# 求出导函数的根，即找出原多项式函数的极值点
print "Extremas: ", np.roots(der)

# 通过argmax和argmin函数找到最大最小值点来检查结果
vals = np.polyval(poly, t)
print "Maximum index: ", np.argmax(vals)
print "Minimum index: ", np.argmin(vals)


I am working on the file "NumPy/.ipynb_checkpoints/(5)convenience_function-checkpoint.ipynb".
The first task is:
理想情况下，BHP和VALE股票收盘价的差价越小越好。在极限情况下，差值可以在某个点为0。用roots函数找到拟合多项式函数在什么时候达到0。
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(5)convenience_function-checkpoint.ipynb".
The first task is:
用相关系数来度量两只股票的相关程度。相关系数的取值范围在-1到1之间，一组数据域自身的相关系数为1.使用corrcoef函数计算相关系数。
Can you write Python code for it?

# 使用corrcoef计算更加精确
print 'Correlation coefficient:\n', np.corrcoef(bhp_returns, vale_returns)


I am working on the file "NumPy/.ipynb_checkpoints/(5)convenience_function-checkpoint.ipynb".
The first task is:
相关系数矩阵关于对角线对称，BHP与VALE的相关系数等于VALE和BHP的相关系数。看起来0.68的相关系数表示他们的相关程度似乎不是很强。
判断两只股票的价格走势是否同步
如果它们的差值偏离了平均差值2倍于标准差的距离，则认为他们走势不同步
Can you write Python code for it?

difference = bhp_cp - vale_cp
avg = np.mean(difference)
dev = np.std(difference)
# 检查最后一次收盘价是否在同步状态
print "Out of sync : ", np.abs(difference[-1] - avg) > 2*dev


I am working on the file "NumPy/.ipynb_checkpoints/(5)convenience_function-checkpoint.ipynb".
The first task is:
绘制拟合曲线
Can you write Python code for it?

plot(t, bhp_cp-vale_cp)
plot(t, vals)
show()


I am working on the file "NumPy/.ipynb_checkpoints/(5)convenience_function-checkpoint.ipynb".
The first task is:
这说明，最后一次收盘价不再同步状态，我们暂时不能进行交易
Can you write Python code for it?

# 绘制收益率曲线
t = np.arange(len(bhp_returns))
plot(t, bhp_returns, lw=1)
plot(t, vale_returns, lw=2)
show()


I am working on the file "NumPy/.ipynb_checkpoints/(6)linear_algebra-checkpoint.ipynb".
The first task is:
*号表示共轭转置
Can you write Python code for it?

A = np.mat("4 11 14;8 7 -2")
print "A:\n", A

U, Sigma, V = np.linalg.svd(A, full_matrices=False)
print "U:\n", U
print "Sigma:\n", Sigma
print "V:\n", V

# Sigma矩阵是奇异值矩阵对角线上的值
np.diag(Sigma)

# check
M = U*np.diag(Sigma)*V
print "Product:\n", M


I am working on the file "NumPy/.ipynb_checkpoints/(6)linear_algebra-checkpoint.ipynb".
The first task is:
1. 矩阵
在NumP中，矩阵是ndarray的子类，可以由专用的字符串格式来创建。我们可以使用mat、matrix、以及bmat函数来创建矩阵。
1.1 创建矩阵
mat函数创建矩阵时，若输入已经为matrix或ndarray对象，则不会为它们创建副本。因此，调用mat函数和调用matrix(data, copy=False)等价。
在创建矩阵的专用字符串中，矩阵的行与行之间用分号隔开，行内的元素之间用空格隔开。
Can you write Python code for it?

A = np.mat('1 2 3; 4 5 6; 7 8 9')
print "Creation from string:\n", A

# 转置
print "Transpose A :\n", A.T

# 逆矩阵
print "Inverse A :\n", A.I

# 通过NumPy数组创建矩阵
print "Creation from array: \n", np.mat(np.arange(9).reshape(3,3))


I am working on the file "NumPy/.ipynb_checkpoints/(6)linear_algebra-checkpoint.ipynb".
The first task is:
2. 线性代数
线性代数是数学的一个重要分支。numpy.linalg模块包含线性代数的函数。使用这个模块，我们可以计算逆矩阵、求特征值、解线性方程组以及求解行列式。
2.1 计算逆矩阵
使用inv函数计算逆矩阵。
Can you write Python code for it?

A = np.mat("0 1 2; 1 0 3; 4 -3 8")
print "A:\n", A
inverse = np.linalg.inv(A)
print "inverse of A:\n", inverse

print "check inverse:\n", inverse * A


I am working on the file "NumPy/.ipynb_checkpoints/(6)linear_algebra-checkpoint.ipynb".
The first task is:
2.2 行列式
行列式是与方阵相关的一个标量值。对于一个nn的实数矩阵，行列式描述的是一个线性变换对“有向体积”所造成的影响。行列式的值为正，表示保持了空间的定向(顺时针或逆时针)，为负表示颠倒空间的定向。*numpy.linalg模块中的det函数可以计算矩阵的行列式。
Can you write Python code for it?

A = np.mat("3 4; 5 6")
print "A:\n", A

print "Determinant:\n", np.linalg.det(A)


I am working on the file "NumPy/.ipynb_checkpoints/(6)linear_algebra-checkpoint.ipynb".
The first task is:
2.3 求解线性方程组
矩阵可以对向量进行线性变换，这对应于数学中的线性方程组。solve函数可以求解形如Ax = b的线性方程组，其中A是矩阵，b是一维或二维的数组，x是未知变量。
Can you write Python code for it?

A = np.mat("1 -2 1; 0 2 -8; -4 5 9")
print "A:\n", A
b = np.array([0,8,-9])
print "b:\n", b

x = np.linalg.solve(A, b)
print "Solution:\n", x

# check
print "Check:\n",b == np.dot(A, x)
print np.dot(A, x)


I am working on the file "NumPy/.ipynb_checkpoints/(6)linear_algebra-checkpoint.ipynb".
The first task is:
2.5 奇异值分解
SVD(Singular Value Decomposition，奇异值分解)是一种因子分解运算，将一个矩阵分解为3个矩阵的乘积。奇异值分解是特征值分解的一种推广。
在numpy.linalg模块中的svd函数可以对矩阵进行奇异值分解。该函数返回3个矩阵——U、Sigma和V，其中U和V是正交矩阵，Sigma包含输入矩阵的奇异值。
Can you write Python code for it?

from IPython.display import Latex
Latex(r"$M=U \Sigma V^*$")


I am working on the file "NumPy/.ipynb_checkpoints/(6)linear_algebra-checkpoint.ipynb".
The first task is:
2.6 广义逆矩阵
广义逆矩阵可以使用numpy.linalg模块中的pinv函数进行求解。inv函数只接受方阵作为输入矩阵，而pinv函数没有这个限制。
Can you write Python code for it?

A = np.mat("4 11 14; 8 7 -2")
print "A:\n", A

pseudoinv = np.linalg.pinv(A)
print "Pseudo inverse:\n", pseudoinv

# check
print "Check pseudo inverse:\n", A*pseudoinv


I am working on the file "NumPy/.ipynb_checkpoints/(6)linear_algebra-checkpoint.ipynb".
The first task is:
得到的结果并非严格意义上的单位矩阵，但是非常近似。
Can you write Python code for it?

A = np.mat("0 1 2; 1 0 3; 4 -3 8")
print "A:\n", A
inverse = np.linalg.inv(A)
print "inverse of A:\n", inverse
print "check inverse:\n", inverse * A

pseudoinv = np.linalg.pinv(A)
print "Pseudo inverse:\n", pseudoinv
print "Check pseudo inverse:\n", A*pseudoinv


I am working on the file "NumPy/.ipynb_checkpoints/(7)universal_functions-checkpoint.ipynb".
The first task is:
(2) accumulate方法：可以递归作用于输入数组，与reduce不同的是，它将存储运算的中间结果并返回。add函数上调用accumulate方法，等价于直接调用cumsum函数。
Can you write Python code for it?

print "Accumulate:\n", np.add.accumulate(a)

print "cumsum:\n", np.cumsum(a)


I am working on the file "NumPy/.ipynb_checkpoints/(7)universal_functions-checkpoint.ipynb".
The first task is:
(3) floor_divide函数总是返回整数结果，相当于先调用divide函数再调用floor函数。
floor函数对浮点数进行向下取整并返回整数。
Can you write Python code for it?

print "Floor Divide:\n", np.floor_divide(a, b), np.floor_divide(b, a)


I am working on the file "NumPy/.ipynb_checkpoints/(7)universal_functions-checkpoint.ipynb".
The first task is:
(3) reduceat方法有点复杂，它需要输入一个数组以及一个索引值列表作为参数
Can you write Python code for it?

print "Reduceat:\n", np.add.reduceat(a, [0,5,2,7])


I am working on the file "NumPy/.ipynb_checkpoints/(7)universal_functions-checkpoint.ipynb".
The first task is:
(4) outer方法：返回一个数组，它的秩(rank)等于两个输入数组的秩的和。它会作用于两个输入数组之间存在的所有元素对。
Can you write Python code for it?

print "Outer:\n", np.add.outer(np.arange(3), a)


I am working on the file "NumPy/.ipynb_checkpoints/(7)universal_functions-checkpoint.ipynb".
The first task is:
1. 通用函数
通用函数的输入时一组标量、输出也是一组标量，他们通常可以对应于基本数学运算。如加减乘除。
通用函数的文档
通用函数是对普通的python函数进行矢量化，它是对ndarray对象的逐个元素的操作。
1.1 创建通用函数
使用NumPy中的frompyfunc函数，通过一个Python函数来创建通用函数。
Can you write Python code for it?

# 定义一个Python函数
def pyFunc(a):
    result = np.zeros_like(a)
    # 这里可以看出来对逐个元素的操作
    result = 42
    return result


I am working on the file "NumPy/.ipynb_checkpoints/(7)universal_functions-checkpoint.ipynb".
The first task is:
1.2 通用函数的方法
通用函数并不是真正的函数，而是能够表示函数的numpy.ufunc的对象。frompyfunc是一个构造ufunc类对象的工厂函数。
通用函数类有4个方法：reduce、accumulate、reduceat、outer。这些方法只对输入两个参数、输出一个参数的ufunc对象有效。
1.3 在add函数上分别调用4个方法
(1) reduce方法：沿着指定的轴，在连续的数组元素之间递归调用通用函数，即可得到输入数组的规约(reduce)计算结果。对于add函数，其对数组的reduce计算结果等价于对数组元素求和。
Can you write Python code for it?

a = np.arange(9)
print "a:\n", a
print "Reduce:\n", np.add.reduce(a)


I am working on the file "NumPy/.ipynb_checkpoints/(7)universal_functions-checkpoint.ipynb".
The first task is:
2. 数组的除法运算
在NumPy中，计算算术运算符+、-、* 隐式关联着通用函数add、subtrack和multiply。也就是说，当你对NumPy数组使用这些运算符时，对应的通用函数将自动被调用。
除法包含的过程比较复杂，在数组的除法运算中射击三个通用函数divide、true_divide和floor_division，以及两个对应的运算符/和//。
(1) divide函数在整数除法中均只保留整数部分
Can you write Python code for it?

a = np.array([2, 6, 5])
b = np.array([1, 2, 3])
print "Divide:\n", np.divide(a, b), np.divide(b, a)


I am working on the file "NumPy/.ipynb_checkpoints/(7)universal_functions-checkpoint.ipynb".
The first task is:
4.3 计算一个数被2的幂数整除后的余数
计算余数的技巧只在模为2的幂数时有效。二进制的位左移一位，数值翻倍。
上一个例子看到，将2的幂数减去1，得到一串1组成的二进制数，这为我们提供了掩码，与这样的掩码做位与操作，即可得到以2的幂数作为模的余数。
在NumPy中，<<操作符对应于left_shift函数。
Can you write Python code for it?

print "Modulus 4:\n", x & ((1<<2) - 1)

def mod_2_pow(x, n):
    mod = x & ((1<<n) - 1)
    return mod

mod_2_pow(x,2)

mod_2_pow(x, 3)


I am working on the file "NumPy/.ipynb_checkpoints/(7)universal_functions-checkpoint.ipynb".
The first task is:
mod函数与remainder函数的功能完全一致，%操作符仅仅是remainder函数的简写
(2) fmod函数处理负数的方式和remainder不同。所得余数的正负由被除数决定，与除数的正负无关
Can you write Python code for it?

print "Fmod:\n", np.fmod(a, 2)

print np.fmod(a, -2)


I am working on the file "NumPy/.ipynb_checkpoints/(7)universal_functions-checkpoint.ipynb".
The first task is:
使用zeros_like函数创建一个和a形状相同的、元素全部为0的数组result。flat属性提供了一个扁平迭代器，可以逐个设置数组元素的值。
使用frompyfunc创建通用函数，制定输入参数为1，输出参数为1。
Can you write Python code for it?

ufunc1 = np.frompyfunc(pyFunc, 1, 1)
ret = ufunc1(np.arange(4))
print "The answer:\n", ret

ret = ufunc1(np.arange(4).reshape(2,2))
print "The answer:\n", ret


I am working on the file "NumPy/.ipynb_checkpoints/(8)random_module-checkpoint.ipynb".
The first task is:
1. 随机数
随机数在蒙特卡罗方法(Monto Carlo method)、随机积分等很多方面都有应用。真随机数的产生很困难，因此在实际应用中我们通常使用伪随机数。
有关随机数的函数可以在NumPy的random模块中找到。随机数发生器的核心算法是基于马特赛特旋转演算法(Mersenne Twister algorithm)。随机数可以从离散分布或连续分布中产生。
分布函数有一个可选的参数size，用于指定需要产生的随机数的数量。该参数允许设置为一个整数或元组，生成的随机数将填满指定形状的数组。支持的离散分布包括几何分布、超几何分布和二项分布等。
硬币赌博游戏
二项分布是n个独立重复的是非试验中成功次数的离散概率分布，这些概率是固定不变的，与试验结果无关。

现在对一个硬币赌博游戏下8份赌注
在硬币赌博游戏中，每一轮抛9枚硬币，如果少于5枚硬币正面朝上，你将损失8份赌注中的1份；否则，你将赢得1份赌注
初始资本为1000份赌注
Can you write Python code for it?

# 初始化一个全0的数组来存放剩余资本
# 以参数10000调用binomial函数，进行10000轮硬币赌博游戏
cash = np.zeros(10000)
cash[0] = 1000
outcome = np.random.binomial(9, 0.5, size=len(cash))

# 模拟每一轮抛硬币的结果，更新cash数组
# 打印出outcome的最大最小值，检查输出中是否有异常
for i in xrange(1, len(cash)):
    if outcome[i] < 5:
        cash[i] = cash[i-1] - 1
    elif outcome[i] < 10:
        cash[i] = cash[i-1] + 1
    else:
        raise AssertionError("Unexpected outcome" + outcome)

print outcome.min(), outcome.max()

plot(np.arange(len(cash)), cash)
show


I am working on the file "NumPy/.ipynb_checkpoints/(8)random_module-checkpoint.ipynb".
The first task is:
2. 超几何分布
超几何分布（hypergeometric distribution）是一种离散概率分布，它描述的是一个罐子有两种物件，无放回地从中抽取指定数量的物件后，抽出指定种类物件的数量。
NumPy的random模块中的hypergeometric函数可以模拟这种分布。
模拟游戏秀节目
游戏规则：
- 每当参赛者回答对一个问题，他们可以从一个罐子里摸出3个球并放回
- 罐子里有一个“倒霉球”，一旦摸到这个球，参赛者会被扣去6分
- 如果摸出3个球全部来自其余的25个普通球，可以得1分
- 100道问题被正确回答，其得分情况如何？
Can you write Python code for it?

points = np.zeros(100)
# 第一个参数是罐中普通球的个数
# 第二个参数是倒霉球的个数
# 第三个参数是每次摸球的个数（采样数）
outcomes = np.random.hypergeometric(25, 1, 3, size=len(points))

for i in xrange(len(points)):
    if outcomes[i] == 3:
        points[i] = points[i-1] + 1
    elif outcomes[i] == 2:
        points[i] = points[i-1] - 6
    else:
        print outcomes[i]

plot(points)
show()


I am working on the file "NumPy/.ipynb_checkpoints/(8)random_module-checkpoint.ipynb".
The first task is:
3. 连续分布
连续分布可以用概率密度函数(Probability Density Function, PDF)来描述。随机变量落在某一区间内的概率等于概率密度函数在该区间的曲线下方的面积。
NumPy的random模块中有一系列连续分布的函数——beta、chisquare、exponential、f、gamma、gumbel、laplace、lognormal、logistic、multivariate_normal、nonchetral_chisquare、noncentral_f、normal等。
3.1 绘制正态分布
Can you write Python code for it?

# 产生指定数量的随机数
N = 10000
normal_values = np.random.normal(size=N)

# 绘制分布直方图
dummy, bins, dummy = plt.hist(normal_values, np.sqrt(N), normed=True, lw=1)
sigma = 1
mu = 0
plot(bins, 1/(sigma*np.sqrt(2*np.pi)) * np.exp(-(bins-mu)**2 / (2*sigma**2)), lw=2)
show()

import matplotlib.gridspec as gridspec
gs = gridspec.GridSpec(2,2)
fig = plt.figure(figsize=(10,8))
ax = []

N = 10000
sigma = 1
mu = 0

for a in xrange(2):
    for b in xrange(2):
        ax.append(fig.add_subplot(gs[a,b]))
        normal_values = np.random.normal(size=N)
        dummy, bins, dummy = plt.hist(normal_values, np.sqrt(N), normed=True, lw=1)
        ax[-1].plot(bins, 1/(sigma*np.sqrt(2*np.pi)) * np.exp(-(bins-mu)**2 / (2*sigma**2)), lw=2)

# 使得子图适应figure的间距
fig.tight_layout()
show()


I am working on the file "NumPy/.ipynb_checkpoints/(8)random_module-checkpoint.ipynb".
The first task is:
3.2 对数正态分布
对数正态分布(lognormal distribution)是自然对数服从正态分布的任意随机变量的概率分布。
Can you write Python code for it?

N = 10000
lognormal_values = np.random.lognormal(size=N)

dummy, bins, dummy = plt.hist(lognormal_values, np.sqrt(N), normed=True, lw=1)
sigma = 1
mu = 0

x = np.linspace(min(bins), max(bins), len(bins))
pdf = np.exp(-(np.log(x)-mu)**2 / (2*sigma**2)) / (x*sigma*np.sqrt(2*np.pi))
plot(x, pdf, lw=3)
show()


I am working on the file "NumPy/.ipynb_checkpoints/(9)sort_and_search-checkpoint.ipynb".
The first task is:
3. 搜索
NumPy有多个函数可以在数据中进行搜索。

argmax函数返回数组中最大值对应的下标
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(9)sort_and_search-checkpoint.ipynb".
The first task is:
argmin和nanargmin函数功能类似，只是换成最小值

argwhere函数根据天骄搜索非零的元素，并分组返回对应的下标
Can you write Python code for it?


I am working on the file "NumPy/.ipynb_checkpoints/(9)sort_and_search-checkpoint.ipynb".
The first task is:
nanargmax函数提供相同的功能，但忽略NaN值
Can you write Python code for it?

b = np.array([np.nan, 2, 4])
np.nanargmax(b)


I am working on the file "NumPy/.ipynb_checkpoints/(9)sort_and_search-checkpoint.ipynb".
The first task is:
searchsorted函数为7和-2返回索引5和0.用这些索引作为插入位置，我们生成了数组[-2  0  1  2  3  4  7]，这样就维持了数组的顺序。
4. 数组元素抽取
NumPy的extract函数可以根据某个条件从数组中抽取元素。这与where函数相似，nonzero函数专门用来抽取非零的数组元素。
Can you write Python code for it?

a = np.arange(7)

# 生成选择偶数元素的条件变量
condition = (a%2) == 0

# 使用extract函数基于生成的条件从数组中抽取元素
print "Even numbers:\n", np.extract(condition, a)

print "Non zero:\n", np.nonzero(a)


I am working on the file "NumPy/.ipynb_checkpoints/(9)sort_and_search-checkpoint.ipynb".
The first task is:
排序

sort函数返回排序后的数组
lexsort函数根据键值的字典序进行排序
argsort函数返回输入数组排序后的下标
ndarray类sort方法可对数组进行原地排序
msort函数沿着第一个周排序
sort_complex函数对复数按照先实数后虚数的顺序进行排序

1. 按字典序排序
NumPy中的lexsort函数返回输入数组按字典序排序的下标。我们需要给lexsort函数提供排序所依据的键值数组或元组。
Can you write Python code for it?

# 我们使用转换函数，载入收盘价和日期数据
import datetime
def datestr2num(s):
    return datetime.datetime.strptime(s, "%d-%m-%Y").toordinal()

dates, cp = np.loadtxt('AAPL.csv', delimiter=',', usecols=(1,6), converters={1:datestr2num}, unpack=True)
dates = dates.astype(int)

# 数据本身已经按照日期顺序排好，不过我们现在优先按照收盘价排序
# lexsort返回排序后的数组下标
indices = np.lexsort((dates, cp))
print "Indices:\n", indices

print ["%s %s" % (datetime.date.fromordinal(dates[i]), cp[i]) for i in indices]


I am working on the file "NumPyTutorial.ipynb".
The first task is:
Arrays can be accessed like lists (index, slicing).
Slices can be written to!
Can only contain objects of the same type.
Can not change the shape.

Array Math
Can you write Python code for it?


I am working on the file "NumPyTutorial.ipynb".
The first task is:
Create arrays with array, ones, zeros, empty.
Create 1D-arrays with arange, linspace, logspace.
Create arrays/matrices with the same shape as another with zeros_like, ones_like and empty_like.
Get the shape of an array with a.shape (not a function!)

Access
Can you write Python code for it?


I am working on the file "NumPyTutorial.ipynb".
The first task is:
Math operations and functions in NumPy all work elementwise with arrays.
Own functions can be written to support both numbers and numpy arrays!

More NumPy functions

NumPy contains various functions to work with arrays (sorting, FFT, ...)

Matplotlib
Simple Plots
Can you write Python code for it?

xs = linspace(0.0, 2*pi, 100)
plot(sin(xs))

figure()
plot(xs, sin(xs))

# define size of the figure
#rcParams['figure.figsize'] = 10, 7

plot(xs, sin(xs), 'o-', label="sin(x)")
plot(xs, cos(xs), 'rx--', label="cos(x)")
xlabel("x")
ylabel("f(x)")
legend()


I am working on the file "NumPyTutorial.ipynb".
The first task is:
NumPy Arrays
Creation
Can you write Python code for it?

a1 = array([1.0, 2.0, 3.0])
a2 = arange(1.0, 5.0, 0.5)
a3 = linspace(1.0, 10.0, 17)
print(a1)
print(a2)
print(a3)

m1 = array([[1.0, 2.0],
            [3.0, 4.0]])
print(m1)

a4 = ones(10)
m2 = zeros((5,5))
a5 = empty(10)
print(a4)
print(m2)
print(a5)

a7 = ones_like(a1)
m2 = zeros_like(m1)
# zeros_like, empty_like
print(a7)
print(m2)

print(a7.shape)
print(m1.shape)


I am working on the file "NumPyTutorial.ipynb".
The first task is:
Subplots can be created.
The number gives the plot grid and the number of the active subplot.

Log and Linear Scales
Can you write Python code for it?

xs = linspace(1.0, 10.0, 100)
subplot(221)
plot(xs)
plot(xs**2)
plot(0.001*exp(xs))

subplot(222)
loglog(xs)
loglog(xs**2)
loglog(exp(xs))

subplot(223)
semilogy(xs)
semilogy(xs**2)
semilogy(0.001*exp(xs))

subplot(224)
semilogx(xs)
semilogx(xs**2)
semilogx(0.001*exp(xs))


I am working on the file "NumPyTutorial.ipynb".
The first task is:
plot to plot numpy arrays.
Have a look at the matplotlib gallery to see how to style the plots.

Subplots
Can you write Python code for it?

subplot(221)
plot(xs, sin(xs))
subplot(222)
plot(xs, cos(xs))
subplot(223)
plot(xs, exp(xs))
subplot(224)
plot(xs, log(xs))


I am working on the file "NumPy_2D.ipynb".
The first task is:
Compute the data; this is the forward problem.
Can you write Python code for it?

d = G @ m

plt.imshow(d, cmap='viridis')


I am working on the file "NumPy_2D.ipynb".
The first task is:
Form the discrete kernel, G.
Can you write Python code for it?

N = 20
L = 100
alpha = 0.08

x = np.arange(0, M, 1) * L/(M-1)
dx = L/(M-1)
r = np.arange(0, N, 1) * L/(N-1)

G = np.zeros((N, M))
for j in range(M):
    for k in range(N):
        G[k,j] = dx * np.exp(-alpha * np.abs(r[k] - x[j])**2)


I am working on the file "NumPy_2D.ipynb".
The first task is:
I changed the shape of m so we have to make a new G.
Can you write Python code for it?

M = m.shape[0]
N = 20
L = 100
alpha = 0.08

x = np.arange(0, M, 1) * L/(M-1)
dx = L/(M-1)
r = np.arange(0, N, 1) * L/(N-1)

G = np.zeros((N, M))
for j in range(M):
    for k in range(N):
        G[k,j] = dx * np.exp(-alpha * np.abs(r[k] - x[j])**2)


I am working on the file "NumPy_2D.ipynb".
The first task is:
Mauricio's 1D problem in 2D
We first define the model.
Can you write Python code for it?

M = 50
m = np.zeros((M+1, 1))  # Have to do +1 because we're going to lose one in computing RC series.
m[10:15,:] = 1.0
m[15:27,:] = -0.3
m[27:35,:] = 2.1

m = (m[1:] - m[:-1]) / (m[1:] + m[:-1] + 1e-9)  # Small number avoid division by zero.

m = np.repeat(m, 50, axis=-1)
plt.imshow(m, cmap='viridis')


I am working on the file "NumPy_2D.ipynb".
The first task is:
Minimum least squares solution:
Can you write Python code for it?

m_est = G.T @ la.inv(G @ G.T) @ d
d_pred = G @ m_est

plot_all(m, d, m_est, d_pred)


I am working on the file "NumPy_2D.ipynb".
The first task is:
More complex models
Can you write Python code for it?

from io import BytesIO
import requests
from urllib.parse import quote

text = '+ ='
url = "https://chart.googleapis.com/chart"
params = {'chst': 'd_text_outline',
          'chld': '000000|36|h|000000|_|{}'.format(text),
         }

r = requests.get(url, params)
b = BytesIO(r.content)

img = mpimg.imread(b)

m = np.pad(img[...,3], 20, 'constant')

plt.imshow(m, cmap='viridis')

m = (m[1:] - m[:-1]) / (m[1:] + m[:-1] + 1e-9)  # Small number avoid division by zero.


I am working on the file "NumPy_2D.ipynb".
The first task is:
Noise-free: minimum norm
Can you write Python code for it?


I am working on the file "NumPy_2D.ipynb".
The first task is:
Now we solve:
$$ \hat{\mathbf{m}} = (\mathbf{G}^\mathrm{T} \mathbf{G} + \mu \mathbf{W}^\mathrm{T} \mathbf{W})^{-1} \mathbf{G}^\mathrm{T} \mathbf{d} \ \ $$
Can you write Python code for it?

m_est = la.inv(G.T @ G + µ * W.T @ W) @ G.T @ d
d_pred = G @ m_est

plot_all(m, d, m_est, d_pred)


I am working on the file "NumPy_2D.ipynb".
The first task is:
Solve with LAPACK
Can you write Python code for it?

m_est = la.lstsq(G, d)[0]
d_pred = G @ m_est

plot_all(m, d, m_est, d_pred)


I am working on the file "NumPy_2D.ipynb".
The first task is:
With noise: damped least squares
Can you write Python code for it?

s = 1
d += s * np.random.random(d.shape)

I = np.eye(N)
µ = 2.5
m_est = G.T @ la.inv(G @ G.T + µ * I) @ d
d_pred = G @ m_est

plot_all(m, d, m_est, d_pred)


I am working on the file "NumPy_2D.ipynb".
The first task is:
With noise: damped least squares with first derivative regularization
Can you write Python code for it?

W = convmtx([1,-1], M)[:,:-1]  # Skip last column


I am working on the file "NumPy_reflectivity.ipynb".
The first task is:
But I really want to use the reflectivity, so let's compute that:
Can you write Python code for it?

D = convmtx([-1, 1], imp.size)[:, :-1]

D

r = D @ imp

plt.plot(r[:-1])


I am working on the file "NumPy_reflectivity.ipynb".
The first task is:
Construct the model m
Can you write Python code for it?


I am working on the file "NumPy_reflectivity.ipynb".
The first task is:
Forward model the data d
Now we can perform the forward problem: computing the data.
Can you write Python code for it?


I am working on the file "NumPy_reflectivity.ipynb".
The first task is:
Forward operator: convolution with wavelet
Now we make the kernel matrix G, which represents convolution.
Can you write Python code for it?

from scipy.signal import ricker
wavelet = ricker(40, 2)

plt.plot(wavelet)

# Downsampling: set to 1 to use every sample.
s = 2

# Make G.
G = convmtx(wavelet, m.size)[::s, 20:70]
plt.imshow(G, cmap='viridis', interpolation='none')

# Or we can use bruges (pip install bruges)
# from bruges.filters import ricker
# wavelet = ricker(duration=0.04, dt=0.001, f=100)
# G = convmtx(wavelet, m.size)[::s, 21:71]

# f, (ax0, ax1) = plt.subplots(1, 2)
# ax0.plot(wavelet)
# ax1.imshow(G, cmap='viridis', interpolation='none', aspect='auto')


I am working on the file "NumPy_reflectivity.ipynb".
The first task is:
I don't know how best to control the magnitude of the coefficients or how to combine this matrix with G, so for now we'll stick to the model m being the reflectivity, calculated the normal way.
Can you write Python code for it?


I am working on the file "NumPy_reflectivity.ipynb".
The first task is:
Let's visualize these components for fun...
Can you write Python code for it?

def add_subplot_axes(ax, rect, axisbg='w'):
    """
    Facilitates the addition of a small subplot within another plot.

    From: http://stackoverflow.com/questions/17458580/
    embedding-small-plots-inside-subplots-in-matplotlib

    License: CC-BY-SA

    Args:
        ax (axis): A matplotlib axis.
        rect (list): A rect specifying [left pos, bot pos, width, height]
    Returns:
        axis: The sub-axis in the specified position.
    """
    def axis_to_fig(axis):
        fig = axis.figure

        def transform(coord):
            a = axis.transAxes.transform(coord)
            return fig.transFigure.inverted().transform(a)

        return transform

    fig = plt.gcf()
    left, bottom, width, height = rect
    trans = axis_to_fig(ax)
    x1, y1 = trans((left, bottom))
    x2, y2 = trans((left + width, bottom + height))
    subax = fig.add_axes([x1, y1, x2 - x1, y2 - y1])
    x_labelsize = subax.get_xticklabels()[0].get_size()
    y_labelsize = subax.get_yticklabels()[0].get_size()
    x_labelsize *= rect[2] ** 0.5
    y_labelsize *= rect[3] ** 0.5
    subax.xaxis.set_tick_params(labelsize=x_labelsize)
    subax.yaxis.set_tick_params(labelsize=y_labelsize)
    return subax

from matplotlib import gridspec, spines

fig = plt.figure(figsize=(12, 6))

gs = gridspec.GridSpec(5, 8)

# Set up axes.
axw = plt.subplot(gs[0, :5])   # Wavelet.
axg = plt.subplot(gs[1:4, :5]) # G
axm = plt.subplot(gs[:, 5])    # m
axe = plt.subplot(gs[:, 6])    # =
axd = plt.subplot(gs[1:4, 7])    # d

cax = add_subplot_axes(axg, [-0.08, 0.05, 0.03, 0.5])

params = {'ha': 'center',
          'va': 'bottom',
          'size': 40,
          'weight': 'bold',
         }

axw.plot(G[5], 'o', c='r', mew=0)
axw.plot(G[5], 'r', alpha=0.4)
axw.locator_params(axis='y', nbins=3)
axw.text(1, 0.6, "wavelet", color='k')

im = axg.imshow(G, cmap='viridis', aspect='1', interpolation='none')
axg.text(45, G.shape[0]//2, "G", color='w', **params)
axg.axhline(5, color='r')
plt.colorbar(im, cax=cax)

y = np.arange(m.size)
axm.plot(m, y, 'o',...


I am working on the file "NumPy_wavelets_no_noise.ipynb".
The first task is:
But I really want to use the reflectivity, so let's compute that:
Can you write Python code for it?

m = (imp[1:] - imp[:-1]) / (imp[1:] + imp[:-1])

plt.plot(m)

# We can plot the impedance too, scaled to fit on this plot, so we
# can see how the interface property relates to the rock property.
plt.plot(imp/1e8, alpha=0.5)
plt.show()


I am working on the file "NumPy_wavelets_no_noise.ipynb".
The first task is:
Construct the model m
We start with the slightly odd-seeming number of samples 51. This is because when we calculate the impedance contrasts (reflectivities), we'll lose a sample. Since I'd like 50 samples in the final reflectivity model m, we have to start with 50 + 1 samples in the impedance model.
Can you write Python code for it?


I am working on the file "NumPy_wavelets_no_noise.ipynb".
The first task is:
Forward model the data d
Now we can perform the forward problem: computing the data. We use the formulation
$$ \mathbf{d} = \mathbf{Gm} $$
Note that this equation is often rearranged and written $$ \mathbf{Ax} = \mathbf{b} $$ in linear algebra textbooks and in other tutorials, etc. Just in case you're looking for more examples. In that case, A is analogous to G, and b to d, and we're solving for x not m. All just notation — we can of course call things whatever we want.
Can you write Python code for it?


I am working on the file "NumPy_wavelets_no_noise.ipynb".
The first task is:
It's interesting to look at these intermediate arrays:
Can you write Python code for it?

fig = plt.figure(figsize=(12,6))
ax0 = fig.add_subplot(121)
_ = ax0.imshow(la.inv(G @ G.T), cmap='viridis', interpolation='none')
ax1 = fig.add_subplot(122)
_ = ax1.imshow(G.T @ la.inv(G @ G.T), cmap='viridis', interpolation='none')


I am working on the file "NumPy_wavelets_no_noise.ipynb".
The first task is:
LU factorization
Again, G must be square.
Can you write Python code for it?

lu, piv = spla.lu_factor(G)
plt.imshow(lu, cmap='viridis', interpolation='none')
plt.show()

# NB This block will fail if G is not square.

m_est = spla.lu_solve((lu, piv), d)
d_pred = G @ m_est
plot_all(m, d, m_est, d_pred, equalize=False)


I am working on the file "NumPy_wavelets_no_noise.ipynb".
The first task is:
Let's normalize the wavelet amplitude to 1 so that the amplitude relates directly to the reflectivity.
Can you write Python code for it?

wavelet /= np.amax(wavelet)

# Downsampling: set to 1 to use every sample.
s = 2

# Make G.
G = convmtx(wavelet, m.size)[::s, 10:60]
plt.imshow(G, cmap='viridis', interpolation='none')
plt.show()

# Or we can use bruges (pip install bruges)
# from bruges.filters import ricker
# wavelet = ricker(duration=0.04, dt=0.001, f=100)
# G = convmtx(wavelet, m.size)[::s, 21:71]

# f, (ax0, ax1) = plt.subplots(1, 2)
# ax0.plot(wavelet)
# ax1.imshow(G, cmap='viridis', interpolation='none', aspect='auto')


I am working on the file "NumPy_wavelets_no_noise.ipynb".
The first task is:
Let's visualize these components for fun. This is Figure 1 in the printed manuscript. I've hidden all the plotting in the utils.py file. You could do the basics with a lot less code, but I wanted it to look nice for the paper.
Can you write Python code for it?

from utils import plot_gmd

plot_gmd(G, m, d)


I am working on the file "NumPy_wavelets_no_noise.ipynb".
The first task is:
Make a consistent x space for the model to live in. This might be 'depth'.
Can you write Python code for it?


I am working on the file "NumPy_wavelets_no_noise.ipynb".
The first task is:
Some helpful functions
These are some functions we will use later. I'm defining them up-front so that we can talk about the maths without too much code getting in the way. It's probably best to ignore these for now, but when we get to them, you might want to come back here to understand what they do.
Can you write Python code for it?

def norm(m):
    return m.T @ m

def misfit(d, d_pred):
    misfit = (d_pred - d).T @ (d_pred - d)
    return np.asscalar(misfit)

from scipy import linalg as spla

def convmtx(h, n):
    """
    Equivalent of MATLAB's convmtx function, http://www.mathworks.com/help/signal/ref/convmtx.html.

    Makes the convolution matrix, C. The product C.x is the convolution of h and x.

    Args
        h (ndarray): a 1D array, the kernel.
        n (int): the number of rows to make.

    Returns
        ndarray. Size m+n-1
    """
    col_1 = np.r_[h[0], np.zeros(n-1)]
    row_1 = np.r_[h, np.zeros(n-1)]
    return spla.toeplitz(col_1, row_1)

convmtx([1, -1], 5)


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
<hr />

Aside: minimum norm without inverting G
"Don't invert that matrix!" — says John Cook, and we should listen to John because he is much smarter than us.
OK, no problem... We can use other solvers, like np.linalg.solve, or scipy.linalg.lu_solve. These are implemented via LAPACK gesv and are much faster than inverting matrices with NumPy.
linalg.solve or linalg.lstsq
Can you write Python code for it?

try:
    """Exact solution, matrix must be square and full-rank."""
    m_est = la.solve(G, d)
    print("Exact solution.")
except la.LinAlgError:
    """The problem is underdetermined, find optimal solution."""
    m_est = la.lstsq(G, d)[0]
    print("Best solution by least squares.")

d_pred = G @ m_est

from utils import plot_all

plot_all(m, d, m_est, d_pred)


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
<hr />

With noise: minimum weighted norm, or damped least squares
Now let's add noise! It's going to make things harder.
First, add noise:
Can you write Python code for it?

s = 0.025
d = G @ m + s * (np.random.random(d.shape) - 0.5)


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Construct the model m
We start with the slightly odd-seeming number of samples 51. This is because when we calculate the impedance contrasts (reflectivities), we'll lose a sample. Since I'd like 50 samples in the final reflectivity model m, we have to start with 50 + 1 samples in the impedance model.
Can you write Python code for it?


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Forward model the data d
Now we can perform the forward problem: computing the data. We use the formulation
$$ \mathbf{d} = \mathbf{Gm} $$
Note that this equation is often rearranged and written $$ \mathbf{Ax} = \mathbf{b} $$ in linear algebra textbooks and in other tutorials, etc. Just in case you're looking for more examples. In that case, A is analogous to G, and b to d, and we're solving for x not m. All just notation — we can of course call things whatever we want.
Can you write Python code for it?


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
It's interesting to look at these intermediate arrays:
Can you write Python code for it?


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Lasso
The docs
Linear Model trained with L1 prior as regularizer (aka the Lasso).
For reasons I don't understand, I cannot get the ordinary Lasso mode to fit.
Can you write Python code for it?

reg = linear_model.LassoCV(fit_intercept=False)
reg.fit(G, d)
m_est = reg.coef_
d_pred = G @ m_est
plot_all(m, d, m_est, d_pred)


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Let's normalize the wavelet amplitude to 1 so that the amplitude relates directly to the reflectivity.
Can you write Python code for it?


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Make a consistent x space for the model to live in. This might be 'depth'.
Can you write Python code for it?


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Minimum weighted norm with first derivative regularization
Can you write Python code for it?

W = convmtx([1, -1], m.size)[:, :-1]  # Skip last column


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Minimum weighted norm with second derivative regularization
Can you write Python code for it?

W = convmtx([1, -2, 1], m.size)[:, 1:-1]  # Skip first and last columns.

Q = la.inv(W @ W.T)

m_est = Q @ G.T @ la.inv(G @ Q @ G.T) @ d
d_pred = G @ m_est

plot_all(m, d, m_est, d_pred, equalize=False)


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Note that G * m gives us exactly the same result as np.convolve(w_, m). This is just another way of implementing convolution that lets us use linear algebra to perform the operation, and its inverse.
Can you write Python code for it?

plt.plot(np.convolve(wavelet, m, mode='same')[::s], c='#462f7c', lw=3)
plt.plot(G @ m, 'red')


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Notice that we lost a sample: now there are only 50 samples. This is what we wanted.
Forward operator: convolution with wavelet
Now we make the kernel matrix G, which represents convolution.
Can you write Python code for it?

from scipy.signal import ricker
wavelet = ricker(points=20, a=2)

plt.plot(wavelet)
plt.show()


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Now for the rest of the preliminaries.
Can you write Python code for it?

import numpy.linalg as la
import matplotlib.pyplot as plt
%matplotlib inline

from ipywidgets import interact, interactive, fixed
import ipywidgets as widgets


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Now the minimum norm doesn't work so well:
Can you write Python code for it?

m_est = G.T @ la.inv(G @ G.T) @ d
d_pred = G @ m_est

plot_all(m, d, m_est, d_pred, equalize=False)


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Now use Mauricio's second form:
Can you write Python code for it?

I = np.eye(G.shape[0])
µ = 2.5  # We can use Unicode symbols in Python 3, just be careful
m_est = G.T @ la.inv(G @ G.T + µ * I) @ d
d_pred = G @ m_est

plot_all(m, d, m_est, d_pred)


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Now we solve:
$$ \hat{\mathbf{m}} = (\mathbf{G}^\mathrm{T} \mathbf{G} + \mu \mathbf{W}^\mathrm{T} \mathbf{W})^{-1} \mathbf{G}^\mathrm{T} \mathbf{d} \ \ $$
Can you write Python code for it?

m_est = la.inv(G.T @ G + µ * W.T @ W) @ G.T @ d
d_pred = G @ m_est

plot_all(m, d, m_est, d_pred)


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Orthogonal matching pursuit
The docs
Seems like a sort of sparse spike inversion.
Can provide number of spikes, or tolerance. Try setting n_nonzero_coefs or tol hyperparameters.
Can you write Python code for it?

reg = linear_model.OrthogonalMatchingPursuit()
reg.fit(G, d)
m_est = reg.coef_
d_pred = G @ m_est
plot_all(m, d, m_est, d_pred)


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Ridge regression
Linear Model trained with L2 regularization.
sklearn.linear_model.Ridge
Also known as Tikhonov regularization.
Can you write Python code for it?

reg = linear_model.Ridge(alpha=1, fit_intercept=False)
reg.fit(G, d)
m_est = reg.coef_
d_pred = G @ m_est
plot_all(m, d, m_est, d_pred)


I am working on the file "NumPy_wavelets_with_noise.ipynb".
The first task is:
Scikit Learn
Scikit-Learn is a machine learning package that has a large number of linear models, to do different kinds of regression with regularization. These will perform much better in the face of noise.
Can you write Python code for it?


I am working on the file "Numba inside PostgreSQL.ipynb".
The first task is:
Compare Performance of the Numba Compiled Version with the Pure-Python Function
The messages from plpy.log() will end up in the postgres logfile.  The output from running this function will be the relative speedup of using the Numba version of the function.  Speaking more generally, the amount of speedup will, of course, vary depending upon the amount and character of data being consumed, the nature of the algorithm, how data is consumed from a table (presumably what you'd likely be doing inside postgres), et cetera, et cetera.
Can you write Python code for it?

with psycopg2.connect(database='radon_fingerprints',
                       host='localhost',
                       port=5437) as conn:
    with conn.cursor() as cursor:
        cursor.execute("SELECT prep_sum2d();")

        cursor.execute("SELECT speedtest_sum2d();")

        rows = cursor.fetchall()
        conn.commit()

rows


I am working on the file "Numba inside PostgreSQL.ipynb".
The first task is:
Create Compute-Intensive Python Function
We'll create two functions in PL/Python.  The first does the work of defining the function and even compiling it with Numba.  The second performs a speedtest to compare the pure-Python version of the function with the Numba compiled version of it.
Imagining a more real-world use case, the first might be a function we want to make fast and the second is perhaps the code that uses that function (which could have been conventional SQL, no Python needed).
Can you write Python code for it?

with psycopg2.connect(database='radon_fingerprints',
                      host='localhost',
                      port=5437) as conn:
    with conn.cursor() as cursor:
        cursor.execute("CREATE EXTENSION IF NOT EXISTS plpythonu;")
        cursor.execute("""
            CREATE OR REPLACE FUNCTION prep_sum2d ()
                RETURNS integer
            AS $$

                # GD is a so-called "global dictionary" made available
                # to us by PL/Python.  It allows us to share information
                # across functions/code within a single session.  We
                # will use it to "share" our Numba-compiled function
                # with a different Python function defined later in
                # this cell.
                if 'numba' in GD and 'numpy' in GD:
                    numpy = GD['numpy']
                    numba = GD['numba']
                else:
                    import numpy
                    import numba
                    GD['numpy'] = numpy
                    GD['numba'] = numba

                # Define our compute-intensive function to play with.
                # (This is the example offered on the main Numba webpage.)
                def sum2d(arr):
                    M, N = arr.shape
                    result = 0.0
                    for i in range(M):
                        for j in range(N):
                            result += arr[i,j]
                    return result

                # Store it in PL/Python's special 'GD' dict for ease of later use.
                GD['sum2d'] = sum2d

                # Compile a version of sum2d using Numba, and store it for later use.
                jitsum2d = numba.jit(sum2d, target='gpu')
                csum2d = jitsum2d.compile(numba.double(numba.double[:,::1]))
                GD['jitsum2d'] = jitsum2d
                GD['csum2d'] = csum2d

                return 1

            $$ LANGUAGE plpythonu;
        """)
        #cursor.execute("DROP FUNCTION speedtest_sum2d();")
       ...


I am working on the file "Numba_Demo_001.ipynb".
The first task is:
About Cython:
- generates C code from a Python code.
- allows to define low level C-types.
- in this example we use C-type double.
- C code is generated, compiled and executed.
Benchmarks JIT vs Cython
Can you write Python code for it?

run_benchmarks(functions=[sum_j, sum_c],
               call_parameters=(1000000000.,),
               num_times=10)


I am working on the file "Numba_Demo_001.ipynb".
The first task is:
Benchmark results
Lets measure them:
Can you write Python code for it?

run_benchmarks(functions=[sum_p, sum_j],
               call_parameters=(10000000,),
               num_times=5,
               logy=True) # Logarithmic scale


I am working on the file "Numba_Demo_001.ipynb".
The first task is:
Numba JIT functionality works in the following way:
- At each call of a function $f(x)$, numba looks at the type $T$ of $x$.
- If it is the first time that type have been used generates a native implementation $f_T(x)$.
- If it is not  the first time that type have been used generates fetches the native implementation from a cache.
- Numba executes $f_T(x)$ that is orders of magnitudes faster than a pure Python implementation.
And what about adding Cython to the game?
Lets define the same functon, but tuned to operate with floats:
$$sum(x) = \sum\limits_{x=0}^X x$$
We redefine it using numba and cython, this time using float numbers.
Can you write Python code for it?

from numba import jit

@jit
def sum_j(x):
    y = 0.
    x_i = 0.
    while x_i < x:
        y += x_i
        x_i += 1.
    return y

%load_ext Cython

%%cython
def sum_c(double x):
    cdef double y = 0.
    cdef double x_i = 0.
    while x_i < x:
        y += x_i
        x_i += 1.
    return y


I am working on the file "Numba_Demo_001.ipynb".
The first task is:
Numba caching
A second run to study numba caching mechanism
Can you write Python code for it?

run_benchmarks(functions=[sum_j],
               call_parameters=(1000000000000000.,),
               num_times=5,
               logy=True) # Logarithmic scale


I am working on the file "Numba_Demo_001.ipynb".
The first task is:
The numba jitted function is comparable with the cythonized one, lets check what was the C code cython used, just to give us an idea of the efficiency of the code generated.
Can you write Python code for it?

%%cython --annotate
def sum_c(double x):
    cdef double y = 0.
    cdef double x_i = 0.
    while x_i < x:
        y += x_i
        x_i += 1.
    return y


I am working on the file "Numba_Demo_002.ipynb".
The first task is:
But the recursive implementation is not efficient, it is recursive, and at each recursion call itself twice:
$$O(n) = 2^{n}$$
Lets convert it to an iteractive function by memoizing former results at each iteration:
Can you write Python code for it?

def fibonacci_i(x):
    assert x >= 0, 'x must be a positive integer'

    if x <= 1:  # First 2 cases.
        return x

    y_2 = 0
    y_1 = 1
    y_0 = 0
    for n in range(x - 1):
        y_0 = y_1 + y_2
        y_1, y_2 = y_0, y_1

    return y_0

X = [x for x in range(10)]
print('X = ' + repr(X))

Y = [fibonacci_i(x) for x in X]
print('Y = ' + repr(Y))


I am working on the file "Numba_Demo_002.ipynb".
The first task is:
Iterative approach have a better CPU complexity:
$$O(n) = n$$
Benchmark
Lets define a benchmark and use it to compare the two functions:
Can you write Python code for it?

from IPython.display import display
from itertools import combinations
import time
import matplotlib
import os
import pandas as pd

import matplotlib.pyplot as plt
matplotlib.style.use('ggplot')
%matplotlib inline

# Different platforms require different functions to properly measure current timestamp:
if os.name == 'nt':
    now = time.clock
else:
    now = time.time

def run_benchmarks(minX, maxX, step=1, num_runs=10, functions=None):
    assert functions, 'Please pass a sequence of functions to be tested'
    assert num_runs > 0, 'Number of runs must be strictly positive'

    def _name(function):
        return '${%s(x)}$' % function.__name__

    def _measure(x, function):
        T_0 = now()
        for i in range(num_runs):
            function(x)
        return (now() - T_0) / num_runs

    df = pd.DataFrame()
    for function in functions:
        function(minX)  # This is necessary to let JIT produce the native function.
        X = [x
             for x in range(minX, maxX + 1, step)]
        Y = [_measure(x, function) for x in X]
        df[_name(function)] = pd.Series(data=Y, index=X)

    plt.figure()
    df.plot(figsize=(10,5),
            title='$y=Log_{10}(T[f(x)])$',
            style='o-',
            logy=True)

    if len(functions) >= 2:
        comb = combinations(((_name(function), df[_name(function)].values)
                             for function in functions), 2)
        for (nameA, timesA), (nameB, timesB) in comb:
            title = '$y=\\frac{T[%s]}{T[%s]}$' % (nameA[1:-1], nameB[1:-1])
            plt.figure()
            (df[nameA] / df[nameB]).plot(figsize=(10,3.5),
                                         title=title,
                                         style='o-')

run_benchmarks(0, 30, functions=[fibonacci_r, fibonacci_i])


I am working on the file "Numba_Demo_002.ipynb".
The first task is:
The iterative aproach, being linear, is already hundred thousand times faster for $x=30$
Lets jit them up!
Lets try to apply JIT optimizations on both implementations:
Can you write Python code for it?

from numba import jit

@jit
def fibonacciJit_r(x):
    assert x >= 0, 'x must be a positive integer'

    if x <= 1:  # First 2 cases.
        return x

    return fibonacciJit_r(x - 1) + fibonacciJit_r(x - 2)

@jit
def fibonacciJit_i(x):
    assert x >= 0, 'x must be a positive integer'

    if x <= 1:  # First 2 cases.
        return x

    y_2 = 0
    y_1 = 1
    y_0 = 0
    for n in range(x - 1):
        y_0 = y_1 + y_2
        y_1, y_2 = y_0, y_1

    return y_0

run_benchmarks(0, 30, functions=[fibonacciJit_r, fibonacciJit_i])


I am working on the file "NumberLineModel.ipynb".
The first task is:
Compare two ordinal numbers
The easiest way to use the model to compare any two given numerosity is via gaussian subtraction, where e.g. for numerosity $5,6$, $mu = 6-5$ and $sigma = \sqrt{\sigma^2_5 + \sigma^2_6}$
This corresponds to the degree of overlapping regions of two normal distributions.
The more overlap there is between the two Gaussian's representations being compared, the less accurately they can be discriminated.
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import matplotlib.mlab as mlab
import math
from matplotlib import style
#style.use('fivethirtyeight')
style.use('ggplot')

mu = 1
variance = 1
weber = .125
sigma = math.sqrt(variance)
x = np.linspace(0, 14, 1000)
maximum_activation = mlab.normpdf(1, 1, 1*.125)

for mu in range(5,7):
    sigma = mu * weber
    #normalized plot, maximum activation one
    #sigma = math.sqrt(variance)
    plt.plot(x,mlab.normpdf(x, mu, sigma)/maximum_activation)

plt.xlabel('Number')
plt.ylabel('Activation')
plt.title('Number Line Model')
plt.xticks(np.arange(min(x), max(x)+1, 1.0))
plt.show()

from math import sqrt
sigma_5 = 5 * weber
sigma_6 = 6 * weber
new_mu = 6-5
difference_sigma = sqrt(sigma_5**2+sigma_6**2)

x = np.linspace(-3, 3, 1000)
plt.plot(x,mlab.normpdf(x, new_mu, difference_sigma)/maximum_activation, color='#096c99')
plt.axvline(x=0, ls = '-.', color='#99a2a1')
plt.show()


I am working on the file "NumberLineModel.ipynb".
The first task is:
Just Noticiable Difference (JND)
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import matplotlib.mlab as mlab
import math
from matplotlib import style
#style.use('fivethirtyeight')
style.use('ggplot')

mu = 1
variance = 1
weber = .125
sigma = math.sqrt(variance)
x = np.linspace(5, 30, 1000)
maximum_activation = mlab.normpdf(1, 1, 1*.125)

for mu in range(16,19):
    sigma = mu * weber
    #normalized plot, maximum activation one
    #sigma = math.sqrt(variance)
    plt.plot(x,mlab.normpdf(x, mu, sigma)/maximum_activation)

plt.xlabel('Number')
plt.ylabel('Activation')
plt.title('Number Line Model')
plt.xticks(np.arange(min(x), max(x)+1, 1.0))
plt.show()

sigma_a = 16 * weber
sigma_b = 17 * weber
difference_sigma = sqrt(sigma_a**2+sigma_b**2)
test =  np.random.normal(1, difference_sigma, 1000)
100-len(test[test<0])/10 #/1000*100


I am working on the file "NumberLineModel.ipynb".
The first task is:
Plot Magnitude/Size error
Can you write Python code for it?

keys_1 = ['1-2','2-3', '3-4', '4-5', '5-6', '6-7', '7-8', '8-9']
result_magnitude_1 = [experiment['distance1'][keys]/1000 for keys in keys_1]

result_magnitude_1

x = np.arange(1, 9)
plt.plot(x,np.array(result_magnitude_1),color='#096c99')

plt.xlabel('Magnitude')
plt.ylabel('Error-Rate in %')
plt.title('Magnitude Effect')
plt.xticks(np.arange(min(x), max(x)+1, 1.0))
plt.savefig('magnitudeEffect.png')
plt.show()