I am working on the file "Intro_to_Neural_Networks/Python+Basics+With+Numpy+v3.ipynb".
The first task is:
Expected output:
test: Hello World

What you need to remember:
- Run your cells using SHIFT+ENTER (or "Run cell")
- Write code in the designated areas using Python 3 only
- Do not modify the code outside of the designated areas
1 - Building basic functions with numpy
Numpy is the main package for scientific computing in Python. It is maintained by a large community (www.numpy.org). In this exercise you will learn several key numpy functions such as np.exp, np.log, and np.reshape. You will need to know how to use these functions for future assignments.
1.1 - sigmoid function, np.exp()
Before using np.exp(), you will use math.exp() to implement the sigmoid function. You will then see why np.exp() is preferable to math.exp().
Exercise: Build a function that returns the sigmoid of a real number x. Use math.exp(x) for the exponential function.
Reminder:
$sigmoid(x) = \frac{1}{1+e^{-x}}$ is sometimes also known as the logistic function. It is a non-linear function used not only in Machine Learning (Logistic Regression), but also in Deep Learning.

To refer to a function belonging to a specific package you could call it using package_name.function(). Run the code below to see an example with math.exp().
Can you write Python code for it?

# GRADED FUNCTION: basic_sigmoid

import math

def basic_sigmoid(x):
    """
    Compute sigmoid of x.

    Arguments:
    x -- A scalar

    Return:
    s -- sigmoid(x)
    """

    ### START CODE HERE ### (≈ 1 line of code)
    s = 1/(1+math.exp(-x))
    ### END CODE HERE ###

    return s

basic_sigmoid(3)


I am working on the file "Intro_to_Neural_Networks/Python+Basics+With+Numpy+v3.ipynb".
The first task is:
Furthermore, if x is a vector, then a Python operation such as $s = x + 3$ or $s = \frac{1}{x}$ will output s as a vector of the same size as x.
Can you write Python code for it?

# example of vector operation
x = np.array([1, 2, 3])
print (x + 3)


I am working on the file "Intro_to_Neural_Networks/Python+Basics+With+Numpy+v3.ipynb".
The first task is:
In fact, if $ x = (x_1, x_2, ..., x_n)$ is a row vector then $np.exp(x)$ will apply the exponential function to every element of x. The output will thus be: $np.exp(x) = (e^{x_1}, e^{x_2}, ..., e^{x_n})$
Can you write Python code for it?

import numpy as np

# example of np.exp
x = np.array([1, 2, 3])
print(np.exp(x)) # result is (exp(1), exp(2), exp(3))


I am working on the file "Intro_to_OO.ipynb".
The first task is:
As you modify your Class all of the methods within the Class know about the modifications.
For example: Let us add some more attributes to our Asteroid data.
Can you write Python code for it?

class SpaceRock(object):

    def __init__(self, name = None, ab_mag = None, albedo = None,
                 semi_major= None, ecc = None):

        self.name = name
        self.ab_mag = ab_mag
        self.albedo = albedo
        self.semi_major = semi_major
        self.ecc = ecc

    def diameter(self):
        result = (1329.0 / np.sqrt(self.albedo)) * (10 ** (-0.2 * self.ab_mag))
        return result * u.km

    def two_diameter(self):
        result = 2.0 * self.diameter()
        return result

    def find_perihelion(self):
        result = self.semi_major * ( 1.0 - self.ecc )
        return result * u.AU

another_rock_table = QTable.read('PHA.csv', format='ascii.csv')

print(another_rock_table)

name = another_rock_table['Name']
ab_mag = another_rock_table['H']
albedo = another_rock_table['A']
semi_major = another_rock_table['a']
ecc = another_rock_table['ecc']

more_rocks = SpaceRock(name=name, ecc = ecc, semi_major=semi_major, ab_mag=ab_mag, albedo=albedo)

more_rocks.diameter()

more_rocks.find_perihelion()

for Idx,Value in enumerate(more_rocks.find_perihelion()):

    rock_name = more_rocks.name[Idx]

    my_string = "The Asteroid {0} has a perihelion distance of {1:.2f}".format(rock_name, Value)

    print(my_string)


I am working on the file "Intro_to_OO.ipynb".
The first task is:
Documentation
If you want to share your code with collaborators or with your future self, you should include documentation. We've neglected that above, so let's add in some docstrings!
Can you write Python code for it?

class SpaceRock(object):

    """Container for Asteroids"""

    def __init__(self, name = None, ab_mag = None, albedo = None,
                 semi_major= None, ecc = None):

        """
        Parameters
        ----------
        name : string
            Name of the target
        ab_mag : array-like
            Absolute Magnitude of each Asteroid
        albedo : array-like
            Albedo of each Asteroid
        semi_major : array-like
            Semi Major Axis of each Asteroid in AU
        ecc : array-like
            Eccentricity of each Asteroid
        """

        self.name = name
        self.ab_mag = ab_mag
        self.albedo = albedo
        self.semi_major = semi_major
        self.ecc = ecc

    def diameter(self):
        """
        Determine the diameter (in km) of the Asteroids
        """
        result = (1329.0 / np.sqrt(self.albedo)) * (10 ** (-0.2 * self.ab_mag))
        return result * u.km

    def two_diameter(self):
        """
        Determine twice the diameter (in km) of the Asteroids
        """
        result = 2.0 * self.diameter()
        return result

    def find_perihelion(self):
        """
        Determine the perihelion distance of the Asteroids in AU
        """
        result = self.semi_major * ( 1.0 - self.ecc )
        return result * u.AU

rocks = SpaceRock(name=name, ab_mag=ab_mag, albedo=albedo)


I am working on the file "Intro_to_OO.ipynb".
The first task is:
Methods
So far this just looks like another way to store your data. It becomes more powerful when you write methods for your object. Methods can be thought of as functions associated with an object.
You can now access the attributes of the object within methods by calling self.<attribute name>.
Let's make a simple method for the SpaceRock object, which determines the size of the asteroid.
Can you write Python code for it?

class SpaceRock(object):

    def __init__(self, name=None, ab_mag=None, albedo=None):
        self.name = name
        self.ab_mag = ab_mag
        self.albedo = albedo

    def diameter(self):
        result = (1329.0 / np.sqrt(self.albedo)) * (10 ** (-0.2 * self.ab_mag))
        return result * u.km

new_asteroid = SpaceRock(name=name, ab_mag=ab_mag, albedo=albedo)

new_asteroid.diameter()


I am working on the file "Intro_to_OO.ipynb".
The first task is:
Real data
Lets use some real data. A short version of the MainBelt.csv dataset from last week
Can you write Python code for it?

from astropy.table import QTable

rock_table = QTable.read('MainBelt_small.csv', format='ascii.csv')

print(rock_table)

name = rock_table['Name']
ab_mag = rock_table['H']
albedo = rock_table['A']

rocks = SpaceRock(name=name, ab_mag=ab_mag, albedo=albedo)

rocks.diameter()


I am working on the file "Intro_to_PP_edward.ipynb".
The first task is:
MCMC: Gibbs
Can you write Python code for it?

# BACKWARD MODEL
T = 10000
q_pheads = Empirical(params=tf.Variable(tf.ones([T])*0.5))

# INFERENCE
inference = ed.Gibbs(latent_vars={pheads: q_pheads},
                    data={c: c_train})
inference.run()

# CRITICISM
mean, stddev = sess.run([q_pheads.mean(),q_pheads.stddev()])
print('Inferred posterior mean:')
print(mean)
print('Inferred posterior std:')
print(stddev)

plt.plot(q_pheads.params.eval());
plt.axhline(y=pheads_true)

x = tf.range(0.0,1.0,0.001)
plt.plot(*sess.run([x,pheads.prob(x)]));
plt.plot(*sess.run([x,pheads_cond.prob(x)],
                  {c:c_train}));
plt.hist(q_pheads.params.eval(),
        bins=100,range=(0.0,1.0),
        normed = True);
plt.axvline(x=pheads_true);

plt.plot(lags(q_pheads.params).eval());


I am working on the file "Intro_to_PP_edward.ipynb".
The first task is:
MCMC: Hamiltonian monte carlo
Metropolis hasting and Gibbs will fail in high_dimensional paramter space while HMC will not.
HMC unitlises the geometry of the important regions of the posterior (where the probability mass is concentrated) to guide the jump proposal.
For Gibbs you need to be able to derive the complete conditional algebraically. Most of the times this is not the case, so Gibbs cannot be used. Hamiltonian monte carlo does not have this restriction. However, it only works for continuous variables. This is because it works with derivatives (i.e. we need smooth functions).
Can you write Python code for it?

# BACKWARD MODEL
T = 10000 # number of empirical samples
q_pheads = Empirical(params=tf.Variable(tf.ones([T])*.5))

# INFERENCE
inference = ed.HMC(latent_vars={pheads: q_pheads},
                  data = {c: c_train})
inference.run(step_size=1.0 / N, n_steps=20)

# criticism
mean, stddev = sess.run([q_pheads.mean(), q_pheads.stddev()])
print("Inferred posterior mean:")
print(mean)
print("Inferred posterior stddev")
print(stddev)

plt.plot(q_pheads.params.eval())
plt.axhline(y=pheads_true)

plt.plot(lags(q_pheads.params).eval())

x = tf.range(0.0, 1.0, 0.001)
plt.plot(*sess.run([x, pheads.prob(x)]))
plt.plot(*sess.run([x, pheads_cond.prob(x)],
                  {c:c_train}))
plt.hist(q_pheads.params.eval(),
        bins=100, range=(0.0, 1.0),
        normed=True)
plt.axvline(x=pheads_true)


I am working on the file "Intro_to_PP_edward.ipynb".
The first task is:
approximate inference :MCMC: metroplis hastings:
Can you write Python code for it?

# BACKWARD MODEL
T = 10000
q_pheads = Empirical(params=tf.Variable(tf.ones([T])*0.5))

# INFERENCE
proposal_pheads = Beta(concentration1=1.0,
                       concentration0=1.0)
inference = ed.MetropolisHastings(latent_vars = {pheads: q_pheads},
                                 proposal_vars={pheads: proposal_pheads},
                                 data={c:c_train})
inference.run()

# criticism
mean, stddev = sess.run([q_pheads.mean(),q_pheads.stddev()])
print("inferred posterior mean:")
print(mean)
print("inferred posterior std:")
print(stddev)

plt.plot(q_pheads.params.eval());
plt.axhline(y=pheads_true)

def lags(x):
    mean = tf.reduce_mean(x)
    var = tf.cast(tf.size(x) - 1, tf.float32) *tf.reduce_mean(tf.square(x-mean))
    ret = tf.map_fn(lambda k:tf.cond(tf.equal(k,0),
                                     lambda:var,
                                     lambda:tf.reduce_sum((x[:-k] - mean) * (x[k:]-mean))),
                                     tf.range(0,tf.size(x)),
                                     dtype=tf.float32)
    return ret / var

plt.plot(lags(q_pheads.params).eval());

x = tf.range(0.0,1.0,0.001)
plt.plot(*sess.run([x,pheads.prob(x)]));
plt.plot(*sess.run([x,pheads_cond.prob(x)],
                  {c:c_train}));
plt.hist(q_pheads.params.eval(),
        bins=100,range=(0.0,1.0),
        normed = True);
plt.axvline(x=pheads_true);


I am working on the file "Intro_to_PP_edward.ipynb".
The first task is:
exact solution
Can you write Python code for it?

# inference
pheads_cond = ed.complete_conditional(pheads)
pheads_post = ed.copy(pheads_cond, {c: c_train})

sess.run({key:val for key, val in six.iteritems(pheads_post.parameters) if isinstance(val,tf.Tensor)})

# criticism
mean, stddev = sess.run([pheads_post.mean(),pheads_post.stddev()])
print(" exact posterior mean: " + str(mean))
print("exact posterior std" + str(stddev))

x = tf.range(0.0,1.0,0.001)
plt.plot(*sess.run([x,pheads.prob(x)]));
plt.plot(*sess.run([x,pheads_post.prob(x)]));
plt.axvline(x=pheads_true);


I am working on the file "Introduccion Machine Learning.ipynb".
The first task is:

Clasificacion
La meta es predecir una categoria a traves del las variables de entrada:
* Spam
* Clasificacion de imagenes
* Diagnosticos medicos
* Analisis de sentimiento
Can you write Python code for it?


I am working on the file "Introduccion Machine Learning.ipynb".
The first task is:

Clustering
Si antes hemos realizado una clasificacion por medio de variables conociendo de antemano la solucion(sistemas supervisados). Los metodos de clustering nos ayudan a identificar grupos naturales sin categorizar o estructuras ocultas (sistemas sin supervisar)
- Grupos de paciencies similares
- Clasificacion de productos
Can you write Python code for it?

from sklearn.datasets import load_digits
digits = load_digits()
digits.data[2,:]


I am working on the file "Introduccion Machine Learning.ipynb".
The first task is:

Reduccion dimensional
La tecnica de reduccion dimensional nos ayudan a explorar los datos con gran cantidad de variables
- Ayudar a dibujar el problema
- Reducir las variables de entrada para un algoritmo, evitando sobreajuste, por ejemplo en los arboles de decisiones
- Reducen la cantidad de datos a guardar
Entrenamiento
Can you write Python code for it?

from sklearn.decomposition import PCA

numero_de_variables=2
X = PCA(numero_de_variables).fit_transform(digits.data)

datos=pd.DataFrame(data={'variable1':X[:, 0],'variable2':X[:, 1],'target':digits.target,'output':labels})
datos.head()

sns.lmplot('variable1', 'variable2',
           data=datos, col='target',
           fit_reg=False )


I am working on the file "Introduccion Machine Learning.ipynb".
The first task is:
<img src='imagenes/arbol_decision.png'>
Prediccion
Can you write Python code for it?

prestamos.head(1)

print ("Probalidad de ser rechazado para un prestamo: {}".format(24295/28288))

decision_tree_model.predict_proba(prestamos_procesado[features].iloc[0])


I am working on the file "Introduccion Machine Learning.ipynb".
The first task is:
Agrupacion
Como se ha observado, el centro de cada clusters parece correcto: cada uno de ellos se asemeja a un numero. Ahora vamos a ver si realmente lo que se muestra son los numeros que tienen a su alrededor
Can you write Python code for it?

from scipy.stats import mode
import numpy as np

labels = np.zeros_like(clusters)
prediccion_por_cluster=[]
for i in range(10):
    mask = (clusters == i)
    numero=mode(digits.target[mask])[0]
    prediccion_por_cluster.append(numero[0])
    labels[mask] = numero

utiles.mostrar_imagenes(est.cluster_centers_,prediccion=prediccion_por_cluster)


I am working on the file "Introduccion Machine Learning.ipynb".
The first task is:
Entrenamiento
Can you write Python code for it?

from sklearn.cluster import KMeans

est = KMeans(n_clusters=10)
clusters = est.fit_predict(digits.data)

utiles.mostrar_imagenes(est.cluster_centers_,prediccion=clusters)


from sklearn.tree import DecisionTreeClassifier

target='prestamo_basura'
features=prestamos_procesado.columns
features=features.drop(target)

decision_tree_model=DecisionTreeClassifier(max_depth=3,criterion='entropy')
decision_tree_model.fit(prestamos_procesado[features],prestamos_procesado[target])


I am working on the file "Introduccion Machine Learning.ipynb".
The first task is:
Introducción A Machine learning
Tabla de contenidos

Introduccion a Machine Learning
Regresion
Clasificacion
Clustering
Reduccion dimensional

Introduccion a Machine Learning
Machine Learning: Crear algoritmo capaces de identificar patrones complejos automaticamente.

Regresiones
Uno de los usos mas importantes y extendidos en ML, nos permite hacer predicciones a partir de las relaciones entre las variables observadas.
* Predicciones de consumo energetico
* Esperanza de vida

Can you write Python code for it?

import pandas as pd

house_price = pd.read_csv('BD/home_data.csv')
house_price.head(3)


I am working on the file "Introduccion Machine Learning.ipynb".
The first task is:
Preprocesamiento de datos

Nuestro arbol de decision solamente entiende que cada variable categorica sea binaria, vamos a binarizar cada variable que tengamos
Para que el algorimo entienda que es clasificacion binaria, los grupos deben de ser [-1,1]
Can you write Python code for it?

variables_categoricas=['plazo','grado','sub_grado','propiedad_vivienda','proposito']
prestamos_procesado=utiles.to_hot_encoding(prestamos,variables_categoricas)

prestamos_procesado.prestamo_basura=prestamos_procesado.prestamo_basura.apply(lambda x:1 if x else -1)
prestamos_procesado.head(2)


I am working on the file "Introduccion Machine Learning.ipynb".
The first task is:
Vamos a inspeccionar cuantas clases tiene cada variable categorica
Can you write Python code for it?

for columna in prestamos.columns.values:
    print (columna,pd.np.unique(prestamos[columna].values))


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
Además de esto, podemos acceder a un elemento del array, podemos acceder de la misma manera que se hace en las listas de Python, usando el operador [].
Can you write Python code for it?

array_1d = np.arange(-5,5,1)
array_1d[1]


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
Ahora que ya sabemos cómo manejarnos con los arrays de NumPy, cómo crearlos y acceder a la información de ellos, ya podemos empezar a realizar operaciones matemáticas con ellos.
Ejercicio
Gracias a las distintas formas de indexar un array que nos permite NumPy, podemos hacer operaciones de forma vectorizada, evitando los bucles. Esto supone un incremento en la eficiencia del código y tener un código más corto y legible. Para ello, vamos a realizar el siguiente ejercicio.
Genera una matriz aleatoria cuadrada de tamaño 1000. Una vez creada, genera una nueva matriz donde las filas y columnas 0 y $n-1$ estén repetidas 500 veces y el centro de la matriz quede exactamente igual a la original. Un ejemplo de esto lo podemos ver a continuación: $$\left(\begin{matrix} 1 & 2 & 3 \ 2 & 3 & 4 \ 3 & 4 & 5\end{matrix}\right) \Longrightarrow
\left(\begin{matrix}
1 & 1 & 1 & 2 & 3 & 3 & 3 \
1 & 1 & 1 & 2 & 3 & 3 & 3 \
1 & 1 & 1 & 2 & 3 & 3 & 3 \
2 & 2 & 2 & 3 & 4 & 4 & 4 \
3 & 3 & 3 & 4 & 5 & 5 & 5 \
3 & 3 & 3 & 4 & 5 & 5 & 5 \
3 & 3 & 3 & 4 & 5 & 5 & 5
\end{matrix}\right) $$
Impleméntalo usando un bucle for y vectorizando el cálculo usando lo anteriormente visto para ver la diferencias de tiempos usando ambas variantes. Para medir el tiempo, puedes usar el módulo time.
Operaciones con Arrays
Ahora que ya hemos visto cómo crear y manejar arrays, podemos pasar a ver cómo realizar operaciones aritméticas con ellos. NumPy tiene funciones como np.add, np.substract, np.multiply y np.divide para sumar, restar, multiplicar y dividir arrays. También podemos calcular el módulo entre dos arrays usando np.remainder. Pero, no es necesario usar estas funciones como tal, sino que podemos usar nuestros operadores aritméticos de siempre: +, -, *, / y %.
Can you write Python code for it?

a = np.array([1, 2, 3])
b = np.array([4, 5, 6])

print("Suma usando el operador +:\n\t", a + b)
print("Suma usando np.add:\n\t", np.add(a,b))


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
Calcula la media y la desviación típica de la matriz.
Obtén el elemento mínimo y máximo de la matriz.
Calcula el determinante, la traza y la traspuesta de la matriz.
Calcula la descomposición en valores singulares de la matriz.
Calcula el valor de la suma de los elementos de la diagonal principal de la matriz.

Salvar arrays de NumPy en ficheros
Hay veces que en nuestro problema podemos generar matrices de un tamaño considerable, y que el tiempo de cómputo que hemos necesitado para obtener esa matriz es demasiado grande como para permitirnos el lujo de generarlo de nuevo, o simplemente no es posible reproducir los cálculos.
Es por ello que NumPy ofrece una solución, y es que nos permite el poder salvar estos arrays en ficheros gracias a las funciones save, savetxt, savez o savez_compressed.
* save: guarda el array en un fichero binario de NumPy. Este fichero tiene como formato .npy.
* savetxt: guarda el array en un fichero de texto.
* savez: guarda varios arrays en un mismo fichero sin comprimir. Este fichero tiene formato .npz.
* savez_compressed: similar al anterior pero en este caso el fichero estará comprimido. Su formato también es .npz.
Para más infomración y ejemplos de uso, siempre podréis consultar la documentación oficial.
Quiero más ejercicios para seguir entrenando con NumPy
Existe un repositorio en GitHub lleno de ejercicios que cubren un gran número de módulos y funciones de NumPy, con las soluciones a dichos ejercicios. Estos ejercicios, al igual que esta breve introducción, se encuentran en jupyter-notebooks que podemos descargar y manipular. Este repositorio se encuentra aquí. Y recordar que siempre que useis un repositorio en GitHub y os ha sido de utilidad, dadle star al repositorio.
Funciones lambda ($\lambda$)
Las funciones lambda se utilizan para declarar pequeñas funciones anónimas con un objetivo claro. Su comportamiento es parecido a las funciones que declaramos con def, aunque no son exactamente iguales.
Estas funciones lambda se asocian a una única expresión, por lo que elementos...
Can you write Python code for it?

# Definimos la función lambda de esta manera
suma_l = lambda x, y: x + y

print("Suma con lambda: ", suma_l(4,1))

def suma(x,y):
    return x+y

print("Suma con def: ", suma(4,1))


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
Con esta línea, ya tendremos en nuestro pequeño script el paquete listo para ser usado. Para acceder a los módulos tendremos que hacer numpy.nombre_de_la_función. Esto no es que este mal, pero por así decirlo, el estandar es importar NumPy como se ve a continuación:
Can you write Python code for it?


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
Con esto, ya podemos utilizar la función reduce. Al igual que las anteriores, recibe como primer argumento una función que aplicar sobre la lista que recibe como segundo argumento. Vamos a ver un ejemplo de cómo calcular el factorial de un número en una sola línea.
Can you write Python code for it?

n = 10
fact = reduce(lambda x,y:x*y, range(1,n+1))
print("Factorial de un 10: ", fact)


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
De esta forma, vuestro código pasa a ser más legible, el resto de gente y mas cómodo y rápido de escribir. Además de que por convenio, debes importar NumPy como np.
Arrays de NumPy
Numpy puede trabajar con arrays de $n$ dimensiones, pero los más comunes son los arrays de una, dos o tres dimensiones (1D, 2D, 3D). Además de esto, podemos establecer el tipo de los elementos del array a la hora de crearlo. Esto es muy importante, ya que si el array del tipo está determinado antes de ejecutar el script, Python no tiene que pararse a inferir el tipo de estos elementos, por lo que ganamos mucha velocidad.
Además de esto, ciertas operaciones en NumPy, o librerías que requieren el uso de NumPy, necesitan que los operandos sean de un determinado tipo.
A continuación vamos a crear una serie de arrays de distintas dimensiones a partir de una lista propia de Python:
Can you write Python code for it?

lista1D = [1, 5, 6, 79]
lista2D = [[2, 5.3, 0, -1.99], [14.5, 5, -5., 1]]
lista3D = [[[1, 5, 6, 79], [5, 7, 9, 0]], [[1, 5, 6, 79], [5, 7, 9, 0]]]

uni_dimensional = np.array(lista1D, dtype=np.int32) # Los elementos de este array serán enteros de 32 bits
bi_dimensional  = np.array(lista2D, dtype=np.float64) # En este caso, serán float de 64 bits
tri_dimensional = np.array(lista3D) # Y en este caso?

print("Array 1D:\n", uni_dimensional)
print("Array 2D:\n", bi_dimensional)
print("Array 3D:\n", tri_dimensional)


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
Función filter
La función filter es la manera elegante de filtrar nuestras listas y eliminar aquellos elementos que no cumplan cierta condición. Al igual que map, recibe como parámetros una función y una sequence. Por ejemplo, vamos a extraer los elementos pares e impares de la sucesión de Fibonacci en listas separadas usando filter:
Can you write Python code for it?

def Fibonacci(n):
    a, b = 0, 1
    yield a
    yield b
    i = 0
    while i < n-2:
        a, b = b, a + b
        yield b
        i+=1

pybonacci = list(Fibonacci(1000))
print(pybonacci[0:50], '...', pybonacci[-10:])
# Obtenemos los pares
even = list(filter(lambda x: x%2==0, pybonacci))
# Y los impares
odd = list(filter(lambda x: x%2!=0, pybonacci))

print("Impares:\n", odd[:50], "...", odd[-3])

print("Pares:\n", even[:50], "...", even[-3])


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
Otro uso muy conocido de las funciones lambda es el uso de estas en las funciones filter o map, ya que podemos filtrar elementos que cumplan una serie de condiciones, haciendo uso de la Programación Funcional.
La programación funcional es un paradigma de programación que evalua las expresiones al igual que se hace en matemáticas, es decir, se evalua la expresión, se devuelve un resultado y los operandos quedan inmutables. Esto hace que dada una función $f$ que realiza una tarea determinada, siempre devuelva el mismo resultado si recibe como parámetro el elemento $x$. Esto es conocido como transparencia referencial. Esta programación funcional se basa mucho en el $\lambda$-calculus.
Python, aunque sea un lenguaje por naturaleza imperativo, tiene la posibilidad de emular esta programación funcional y emular también la transparencia referencial, usando las funciones lambda que hemos visto, y funciones como filter, map o reduce, que gracias a la programación funcional, nos permite hacer en una sola línea de código lo que haríamos en bastante más escribiendo de forma "clásica", y de forma más eficiente y elegante.
Función map
La función map recibe como parámetros una función y una lista o sequence. map aplica la función que recibe como entrada sobre la lista y devuelve un generador (en Python 2 devuelve una lista). Un generador es una estructura de Python que actúa como un iterador. Por ejemplo, al usar la función range(5), obtenemos un generador de números enteros de 0 a 5. Los elementos de este generador los podemos obtener todos del tirón haciendo list(range(5)) o bien, sacarlos de uno en uno en un bucle for i in range(5). Una vez generados estos elementos, desaparecen del generador (obviamente).
Vamos a ver un ejemplo a continuación, en el que usaremos la función map sin y con funciones lambda.
Can you write Python code for it?

from numpy import pi

# Definimos nuestra lista que queremos transformar
degrees = [180, 250, 0, 18, 214, ]

# Pasar de grados a radianes
def deg2rad(deg):
    return deg * pi/180

rads_generator = map(deg2rad, degrees)

print("Lista transformada: ", list(rads_generator))

# En caso de que queramos extraerlos de uno en uno:
# for rads in rads_generator:
#     print(rads)

rads_generator_lambda = map(lambda x: x*pi/180, degrees)
print("Lista transformada con lambda: ", list(rads_generator_lambda))


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
Pero, en el caso de que queramos comparar si dos arrays son completamente iguales, devolviendo un único valor que sea True o False, debemos usar la función np.array_equal(array_1, array_2).
Can you write Python code for it?

print("¿Son iguales a y b?", np.array_equal(a, b))


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
También están disponibles los operadores lógicos & y | para realizar operaciones lógicas con ellas. Sin embargo, NumPy también ofrece las funciones np.logical_and(), np.logical_or() para realizar estas operaciones y np.logical_not para negar el array.
Otras funciones a las que tenemos acceso son a funciones que nos ofrecen datos estadísticos como la media o la desviación típica, o el valor de la suma de una fila de un array, etc. A continuación, podemos ver algunas de las que ofrece:
* array.sum(): devuelve la suma total de los componentes del array.
* array.min(): devuelve el mínimo valor del array.
* array.max(): devuelve el valor máximo de la fila o la columna, dependiendo del valor que tenga el parámetro axis.
* array.cumsum(): nos devuelve un nuevo array donde cada elemento es la suma acumulativa de los elementos del array. Al igual que antes, es dependiente del parámetro axis.
* array.mean(): para obtener la media.
* array.median(): para obtener la mediana.
* np.std(array): para obtener la desviación típica del array.
Ejercicio
Dada la siguiente matriz, realiza los siguientes apartados:
Can you write Python code for it?


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
También podemos hacer que una función lambda actúe como una clausura léxica. Pero, ¿qué es una clausura? Una clausura léxica es un nombre para una función que recuerda los valores que se encuentran en un cierto espacio de nombres, incluso cuando el flujo de ejecución del programa no se encuentra en ese espacio de nombres. Vamos a ver un ejemplo de esto.
Can you write Python code for it?

def suma_n(n):
    return lambda x: x+n

suma5 = suma_n(5)
suma10 = suma_n(10)

print("Suma 5: ", suma5(5))
print("Suma 10: ",suma10(5))


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
También son capaces de realizar bucles internos y llamar a funciones como map, usar las expresiones de comprensión de listas, etc.
Can you write Python code for it?

import sys

pinta = lambda x: list(map(sys.stdout.write, x))

lista = pinta(['Un\n', 'Dos\n', 'Tres\n'])

bucle = lambda x: [i for i in range(0,10) if i%x==0]

bucle(2)


I am working on the file "Introducción a Python - nivel medio.ipynb".
The first task is:
Y usando estos dos mismos operadores, por ejemplo, en los arrays bidimensionales podemos acceder a todos los elementos de una fila o una columna de la siguiente forma:
Can you write Python code for it?

array_random2d = np.random.random((4,6))
print("Matriz aleatoria:\n", array_random2d)

# Para acceder a todos los elementos de la segunda fila
print("Fila: \n", array_random2d[1])

# Para acceder a todos los elementos de la cuarta columna
print("Columna: \n", array_random2d[:,3])

# Filas de la segunda fila en adelante incluida esta
print("Filas: \n", array_random2d[1:])

# Columnas de la tercera columna en adelante
print("Columnas: \n", array_random2d[:,2:])

# Subconjuntos en filas y comunas
print("Subconjunto de columnas: \n", array_random2d[0:2,2:5])


I am working on the file "Introducción a Python.ipynb".
The first task is:
Además, podemos hacer las operaciones típicas sobre conjuntos.
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
Ahora bien, en Python existe una forma mucho más cómoda de iterar sobre los valores de una lista sin tener que estar pendiente de un índice i:
Can you write Python code for it?

for amigo in mis_amigos:
    print(amigo)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Ahora bien, si sólo vamos a usar unas operaciones concretas de una librería, podemos especificar cuáles son y así no tener que usar el nombre de la librería para poder utilizarlas.
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
Ahora imagina que quieres hacer un programa para saludar a tu amiga Marta, ¿cómo lo harías? De primeras, podrías hacer lo siguiente:
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
Así, tendríamos un programa para saludar a cualquier Marta del mundo, pero, ¿y si quisiéramos un programa para poder saludar a cualquier persona? Para poder llevar esto a cabo necesitamos introducir un concepto de programación básico: las variables.
Variables
Una variable en programación describe un lugar donde guardar información, que puede ser diferente en cada ejecución. En nuestro caso, queremos saludar al usuario que ejecute nuestro programa, pero no sabemos su nombre a priori y no lo sabremos hasta la ejecución, por tanto, necesitamos reservar un espacio para poder guardar ahí el nombre del usuario durante la ejecución de nuestro programa.
Hay muchos tipos de variables:

números enteros (int)
números reales (float o double)
caracteres (char) y cadenas de caracteres (string)
booleanos (bool), estos últimos sólo pueden tomar dos valores: verdadero (True) o falso (False).

En otros lenguajes de programación, como C, necesitamos indicar el tipo que tiene cada variable que declaramos. En cambio, en python no es necesario ya que el propio intérprete lo infiere por sí mismo.
Una vez hemos aprendido el concepto de variable, ya sí podemos hacer nuestro programa para saludar a cualquier persona.
Can you write Python code for it?

nombre = input("¿Cómo te llamas? ")
print("Hola ", nombre,"!")


I am working on the file "Introducción a Python.ipynb".
The first task is:
Aunque también es posible iterar sobre diccionarios usando bucles for. Para ello, usamos la función items:
Can you write Python code for it?

for nombre, edad in mis_amigos.items():
    print(nombre, edad)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Cuando $n \% x = 0$, dejamos de hacer módulos con $n$ pues ya sabemos que no es primo. Por tanto, nos salimos del bucle con un break. Al salirnos con el break, no entramos en el else sino que volvemos al bucle inicial. En cambio, no hemos encontrado ningún $x$ tal que $n \% x = 0$, ejecutamos la condición else para decir que $n$ es primo.
Ejercicio
¿Cuál es la diferencia entre la sentencia break y la sentencia continue?
Pista: consúltalo en la documentación de Python.
Estructuras de datos
Por ahora sólo hemos estudiado variables en las que podemos guardar un único valor: un número, una letra, una frase... ¿No te da la impresión de que esto se queda algo corto? Sí, y probablemente no eres la única persona que lo ha pensado. Las estructuras de datos son variables compuestas, esto quiere decir que en ellas podemos almacenar muchos datos. Hay estructuras de datos de todo tipo, en python tenemos las siguientes:

Listas
Tuplas
Conjuntos
Diccionarios

Listas
Imagina que quieres guardar en una variable los nombres de tus mejores amigos. Una muy buena opción para hacerlo es una lista. Al igual que en la vida real hacemos listas como lista de cosas por hacer, lista para la compra, lista de propósitos de año nuevo, etc. en Python también podemos hacerlas usando esta estructura de datos.
Con las listas, podemos guardar en un mismo sitio variables relacionadas entre sí. Esto nos permite poder aplicar operaciones sobre todas ellas sin tener que repetir código.
Para declarar una lista, usaremos los [].
Can you write Python code for it?

mis_amigos = ['Paloma', 'Paula', 'Teresa', 'Marina', 'Braulio']
print(mis_amigos)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Cuando iteramos sobre una lista o un conjunto, podemos usar la función enumerate para obtener la posición y el elemento de la misma forma:
Can you write Python code for it?

for posicion, elemento in enumerate(['tic', 'tac', 'toe']):
    print(posicion, elemento)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Cuidado! Si por casualidad un amigo cumpliese un año más no podríamos ponerlo en la tupla, debido a que las tuplas no pueden modificarse.
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
Ejercicio
Crea la siguiente matriz en una línea:
$$ M_{2 \times 3} = \left ( \begin{matrix}
1 & 2 & 3 \
4 & 5 & 6
\end{matrix} \right)$$
Tuplas
Imagina que deseas guardar tanto los nombres de tus mejores amigos como su edad. De primeras podrías pensar en hacer dos listas, de la siguiente forma:
| índice | 0 | 1 | 2 | 3 | 4 |
|:------:|:-:|:-:|:-:|:-:|:-:|
| nombres| Paloma | Paula | Teresa | Marina | Braulio |
| edades | 25 | 20 | 19 | 19 | 21 |
De tal forma que para saber la edad de Paloma (primer elemento de la lista nombres) tendríamos que mirar el primer elemento de la lista edades. Pero, ¿y si te dijera que en Python podríamos guardar en una misma variable la edad y el nombre de una persona? Se puede! Con las llamadas tuplas.
Una tupla es una serie de valores separados por comas.
Can you write Python code for it?

tupla_ejemplo = 5, 'perro', 3.6
print(tupla_ejemplo)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Ejercicio
Crea una lista con todos los números pares del 0 al 10 en una única línea.
Listas anidadas
Seguramente te habrás preguntado si se puede hacer una lista cuyos elementos sean listas, y la respuesta es ¡sí!. Esta representación de listas anidadas se suele usar para representar matrices. Por ejemplo, si queremos representar la siguiente matriz en Python:
$$ M_{2 \times 3} = \left ( \begin{matrix}
1 & 2 & 3 \
2 & 3 & 4
\end{matrix} \right)$$
Lo haríamos de la siguiente forma:
Can you write Python code for it?

M = [[j for j in range(i, i+3)] for i in range(1,3)]
M


I am working on the file "Introducción a Python.ipynb".
The first task is:
Ejercicio
Haz un programa que le pida al usuario un número (de ninjas). Si dicho número es menor que 50 y es par, el programa imprimirá "puedo con ellos!", en caso contrario imprimirá "no me vendría mal una ayudita..."
Nota: para saber si un número es par o no debes usar el operador $\%$ y para saber si dos condiciones se cuplen a la vez, el operador lógico and
while
En el ejemplo anterior le decíamos al usuario que no podíamos hacer la raíz negativa de un número pero, ¿cómo haríamos para, en vez de darle este mensaje, volver a pedirle un número una y otra vez hasta que fuese negativo? Necesitamos ejecutar el mismo código hasta que se dé la condición que buscamos: que el usuario introduzca un número positivo.
Esto podemos hacerlo usando un bucle while!
Can you write Python code for it?

num = int(input("Introduce un número: "))

while (num < 0):
    num = int(input("Introduce un número: "))

raiz = sqrt(num)
print(raiz)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Ejercicio
¿Es buena idea usar la función set para eliminar los elementos repetidos de una lista?
Un último detalle sobre conjuntos: para crear un conjunto vacío usamos set() ya que usar {} creará un diccionario vacío.
Diccionarios
A diferencia de las estructuras de datos que hemos visto hasta ahora, los diccionarios no se indexan por números sino por claves (keys). Cada entrada de nuestro diccionario está formada por dos valores distintos: la clave y el valor. La clave nos sirve para acceder al elemento (valor) de forma rápida. Debido a que las claves sirven para identificar a cada elemento,deben ser únicas: si introduces un nuevo elemento en el diccionario con clave repetida, se sobreescribirá el elemento anterior así que ¡mucho cuidado!
Vamos a ver un pequeño ejemplo de uso de los diccionarios para que nos quede más claro. Volvamos al ejemplo anterior sobre nuestros amigos y su edad.
Can you write Python code for it?

mis_amigos = {'Paloma':25, 'Paula':20, 'Teresa':19, 'Marina':19,
              'Braulio':21}
print(mis_amigos)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Ejercicio:
Haz una lista de la compra e imprime los siguientes elementos:

Penúltimo elemento
Del segundo al cuarto elemento
Los tres últimos
Todos!

Por último, elimina el tercer elemento de la lista usando la sentencia del
Iterando sobre una lista
Con los bucles podemos iterar sobre los valores de una lista. Típicamente, un programador C iteraría sobre una lista de la siguiente forma:
Can you write Python code for it?

for i in range(len(mis_amigos)):
    print(mis_amigos[i])


I am working on the file "Introducción a Python.ipynb".
The first task is:
Ejercicio:
Vuelve a hacer la lista de la compra que hiciste en el último ejercicio, pero esta vez guarda cada elemento de la lista de la compra junto con su precio. Después, imprime los siguientes elementos:

El precio del tercer elemento.
El nombre del último elemento.
Tanto el nombre como el precio del primer elemento.

Cojuntos
Un conjunto es una lista de elementos ordenados y en la que no hay elementos repetidos. Se definen con el operador {. Sus operadores básicos son eliminar elementos repetidos, consultar si un elemento está en el conjunto o no y, por supuesto, operaciones típicas de los conjuntos como la unión, la intersección, la diferencia...
¿Qué ventajas puede darte usar un conjunto en lugar de una lista? Al estar ordenados, es mucho más rápido encontrar un elemento aunque esto también hace que insertar nuevos elementos sea más costoso.
Podemos crear un conjunto a partir de una lista:
Can you write Python code for it?

mi_lista = [5,4,6,3,7,5,1,9,3]
print(mi_lista)
mi_conjunto = set(mi_lista)
print(mi_conjunto)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Elementos de A que no están en B (diferencia):
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
Elementos que están en A o en B pero no en ambos (diferencia simétrica):
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
Elementos que están o bien en A o bien en B (unión):
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
En Python también existe lo que se llama list comprehesions, que son una forma mucho más sencilla y fácil de leer para crear listas. Por ejemplo, si queremos hacer una lista con las potencias de 2, podríamos hacerlo de la siguiente forma:
Can you write Python code for it?

potencias = []
for x in range(10):
    potencias.append(2**x)

print(potencias)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Entonces, uno podría pensar que está obligado a conocer la longitud de la lista si quiere acceder al último elemento pero nada más lejos de la realidad! En python, también existen los índices inversos que nos permiten acceder a los elementos de la lista al revés:
| Elemento | Posición |
|:--------:|:--------:|
|Paloma    | -5       |
|Paula     | -4       |
|Teresa    | -3       |
|Marina    | -2       |
|Braulio   | -1       |
Por lo que para acceder al último elemento de nuestra lista sólo tendríamos que usar el índice -1:
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
Esta librería tiene muchas más operaciones que puedes consultar en la documentación oficial
Estructuras de control
Hasta ahora, nuestros programas se basan en una serie de instrucciones que se ejecutan una detrás de otra. Esto limita mucho los programas que podemos hacer, ya que no nos permite controlar el flujo de ejecución (control flow) de nuestro programa. A continuación vamos a ver una serie de instrucciones especiales que nos permiten hacer precisamente eso.
if
Imagina que estás operando con raíces cuadradas, como sabrás la raíz cuadrada de un número es negativa, y quieres evitar hacer la raíz cuadrada si el número introducido por el usuario es negativo.
Can you write Python code for it?

num = int(input("Introduce un número: "))
raiz = sqrt(num)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Iterando sobre diccionarios
Es posible crear diccionarios en una sola línea, usando las dict comprehesions. En el siguiente ejemplo, cada clave almacena su valor al cuadrado:
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
La función dict nos permite hacer diccionarios directamente desde Tuplas de la siguiente forma:
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
Los valores de las tuplas también tienen índices, por tanto, en nuestro caso el nombre tendría el índice 0 y la edad el índice 1. Por tanto, si queremos acceder a la edad de Paloma tenemos que usar el operador [] dos veces: una para acceder al elemento de la lista que queremos y otra para acceder al elemento de la tupla que queremos.
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
Operando con variables
En Python podemos hacer operaciones sobre las variables que tenemos de forma muy sencilla. A modo de resumen, las principales operaciones en python son:
| Símbolo | Operación     |
|:-------:|:-------------:|
|+        | suma          |
|-        | resta         |
|*        | multiplicación|
|/        | división      |
Estos operadores se pueden usar con cualquier tipo de variable, tanto números como letras. A continuación, te muestro varios ejemplos.
Podemos concatenar dos palabras usando el operador +.
Can you write Python code for it?

nombre1 = "Marta"
nombre2 = "María"
suma_nombres = nombre1 + " y " + nombre2
print(suma_nombres)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Pero si queremos quedarnos con los tres primeros, podemos simplemente hacerlo de la siguiente forma:
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
Por último, si queremos iterar sobre dos listas o conjuntos a la vez (del mismo tamaño), podemos hacerlo usando la función zip:
Can you write Python code for it?

questions = ['name', 'quest', 'favourite color']
answers = ['lancelot', 'the holy grail', 'blue']

for q, a in zip(questions, answers):
    print("What is your {}? It is {}.".format(q,a))


I am working on the file "Introducción a Python.ipynb".
The first task is:
Si queremos saber los nombres de cada uno de nuestros amigos podemos listar las claves:
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
También podemos multiplicar una palabra por un número usando el operador *.
Can you write Python code for it?

num1 = 5
mult_letra_num = nombre1 * 5
print(mult_letra_num)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Y cuando las keys son simples string también es posible definir un diccionario de la siguiente forma:
Can you write Python code for it?

dict(Paloma=25, Paula=20, Teresa=19, Marina=19, Braulio=21)


I am working on the file "Introducción a Python.ipynb".
The first task is:
Y también podemos ver si hemos incluido a un amigo o no en nuestro diccionario:
Can you write Python code for it?


I am working on the file "Introducción a Python.ipynb".
The first task is:
¡No! Esto es debido a que los elementos de los conjuntos deben ser hashables. Tal y como se explica en la documentación, un objeto es hashable si su clase define la función __hash__(), que calcula un entero para caracterizar a un objeto, por lo que objetos con el mismo valor deben tener el mismo entero hash, y la función __eq__(), que sirve para comparar objetos.
Las estructuras de datos que pueden cambiar su valor, como las listas o los diccionarios, no son hashables y, por lo tanto, no pueden ser elementos de un conjunto.
Iterando sobre conjuntos
Al igual que con las listas, también es posible iterar sobre conjuntos y hacer set comprehesions:
Can you write Python code for it?

{x for x in 'abracadabra' if x not in 'abc'}

for c in mi_conjunto:
    print(c)


I am working on the file "Introducción a Python.ipynb".
The first task is:
¿Qué podemos hacer para que no ocurra esto? Controlar con un if la condición de que el número sea positivo para hacer la raíz cuadrada y avisar al usuario en caso contrario:
Can you write Python code for it?

num = int(input("Introduce un número: "))
if num > 0:
    raiz = sqrt(num)
else:
    print("No puedo hacer la raíz de un número negativo!")


I am working on the file "Introducing Random Variables.ipynb".
The first task is:
A Function on the Outcome Space
Suppose you roll a die five times and add up the number of spots you see. If that seems artificial, be patient for a moment and you'll soon see why it's interesting.
The sum of the rolls is a numerical function on the outcome space $\Omega$  of five rolls. The sum is thus a random variable. Let's call it  $S$ . Then, formally,
$S: \Omega \rightarrow { 5, 6, \ldots, 30 }$
The range of  $S$  is the integers 5 through 30, because each die shows at least one and at most six spots. We can also use the equivalent notation
$\Omega \stackrel{S}{\rightarrow} { 5, 6, \ldots, 30 }$
From a computational perspective, the elements of  $\Omega$  are in the column omega of five_roll_space. Let's apply this function and create a larger table.
Can you write Python code for it?

five_rolls_sum = pd.DataFrame({
    'Omega':five_rolls,
    'S(omega)':five_roll_space['Omega'].map(lambda val: sum(val)),
    'P(omega)':five_roll_probs
})
five_rolls_sum


I am working on the file "Introducing Random Variables.ipynb".
The first task is:
As you can see above, Product spaces(Probability spaces) get large very quickly.
If we are tossing 10 times, the outcome space would consist of the $2^{10}$  sequences of 10 elements where each element is H or T.

The outcomes are a pain to list by hand, but computers are good at saving us that kind of pain.
Lets take example of rolling die,

If we roll a die 5 times, there are almost 8,000 possible outcomes:
Can you write Python code for it?

die = np.arange(1, 7, 1)
five_rolls = list(product(die, repeat=5))
# five_rolls = [list(i) for i in product(die, repeat=5)]
five_roll_probs = (1/6**5)**np.ones(6**5)
five_roll_space = pd.DataFrame({
   'Omega':five_rolls,
    'P(omega)':five_roll_probs
})
five_roll_space


I am working on the file "Introducing Random Variables.ipynb".
The first task is:
Functions of Random Variables,
A random variable is a numerical function on  $\Omega$ . Therefore by composition, a numerical function of a random variable is also a random variable.
For example,  $S^2$  is a random variable, calculated as follows:
$S^2(\omega) = \big{(} S(\omega)\big{)}^2$
Thus for example  $S^2(\text{[6 6 6 6 6]}) = 30^2 = 900$.
Events Determined by $S$
From the table five_rolls_sum it is hard to tell how many rows show a sum of 6, or 10, or any other value. To better understand the properties of $S$, we have to organize the information in five_rolls_sum.
For any subset $A$ of the range of $S$, define the event ${S \in A}$ as
$$
{S \in A } = {\omega: S(\omega) \in A }
$$
That is, ${ S \in A}$ is the collection of all $\omega$ for which $S(\omega)$ is in $A$.
If that definition looks unfriendly, try it out in a special case. Take $A = {5, 30}$. Then ${S \in A}$ if and only if either all the rolls show 1 spot or all the rolls show 6 spots. So
$$
{S \in A} = {\text{[1 1 1 1 1], [6 6 6 6 6]}}
$$
It is natural to ask about the chance the sum is a particular value, say 10. That's not easy to read off the table, but we can access the corresponding rows:
Can you write Python code for it?


I am working on the file "Introducing Random Variables.ipynb".
The first task is:
That's 1 in a computing environment, and it is true in general for the distribution of any random variable.
Probabilities in a distribution are non-negative and sum to 1.
Visualising Distribution
Can you write Python code for it?

dist_S.ix[:,0], dist_S.ix[:,1]

s = dist_S.ix[:,0]
p_s = dist_S.ix[:,1]

dist_S = pd.concat([s,p_s],axis=1)
dist_S

dist_S.plot(x="S(omega)",y="P(omega)", kind="bar")

from prob140 import Plot

!pip install sympy


I am working on the file "Introducing Random Variables.ipynb".
The first task is:
The contents of the table – all the possible values of the random variable, along with all their probabilities – are called the probability distribution of  $S$ , or just distribution of  $S$  for short. The distribution shows how the total probability of 100% is distributed over all the possible values of  $S$ .
Let's check this, to make sure that all the  $\omega$ 's in the outcome space have been accounted for in the column of probabilities.
Can you write Python code for it?


I am working on the file "Introducing Random Variables.ipynb".
The first task is:
There are 126 values of $\omega$ for which $S(\omega) = 10$. Since all the $\omega$ are equally likely, the chance that $S$ has the value 10 is 126/7776.
We are informal with notation and write ${ S = 10 }$ instead of ${ S \in {10} }$:
$$
P(S = 10) = \frac{126}{7776} = 1.62\%
$$
This is how Random Variables help us quantify the results of experiments for the purpose of analysis.
i.e., Random variables provide numerical summaries of the experiment in question. - Stats110 harvard (also below paragraph)

This definition is abstract but fundamental; one of the most important skills to
develop when studying probability and statistics is the ability to go back and forth
between abstract ideas and concrete examples. Relatedly, it is important to work
on recognizing the essential pattern or structure of a problem and how it connectsto problems you have studied previously. We will often discuss stories that involve
tossing coins or drawing balls from urns because they are simple, convenient sce-
narios to work with, but many other problems are isomorphic: they have the same
essential structure, but in a different guise.

we can use mathematical opeartion on these variables since they are real valued function nowto problems you have studied previously. We will often discuss stories that involve
tossing coins or drawing balls from urns because they are simple, convenient sce-
narios to work with, but many other problems are isomorphic: they have the same
essential structure, but in a di↵erent guise.
Looking at Distributions
The table below shows all the possible values of  $S$  along with all their probabilities. It is called a "Probability Distribution Table" for  $S$ .
Can you write Python code for it?

dist_S = five_rolls_sum.drop('Omega', axis=1).groupby('S(omega)', as_index=False).sum()
dist_S


I am working on the file "Introduction to Cython.ipynb".
The first task is:
4X speedup with so little effort is pretty nice. What else can we do?
Cython has a nice "-a" flag (for annotation) that can provide clues about why your code is slow.
Can you write Python code for it?

%%cython -a

def f4(double x):
    y = x**4 - 3*x
    return y

def integrate_f4(double a, double b, int n):
    cdef:
        double dx = (b - a) / n
        double dx2 = dx / 2
        double s = f4(a) * dx2
        int i = 0
    for i in range(1, n):
        s += f4(a + i * dx) * dx
    s += f4(b) * dx2
    return s


I am working on the file "Introduction to Cython.ipynb".
The first task is:
Example: C++ int graph
Can you write Python code for it?

%%cython -a
#distutils: language=c++
from cython.operator cimport dereference as deref, preincrement as inc

from libcpp.vector cimport vector
from libcpp.map cimport map as cppmap

cdef class Graph:
    cdef cppmap[int, vector[int]] _adj

    cpdef int has_node(self, int node):
        return self._adj.find(node) != self._adj.end()

    cdef void add_node(self, int new_node):
        cdef vector[int] out
        if not self.has_node(new_node):
            self._adj[new_node] = out

    def add_edge(self, int u, int v):
        self.add_node(u)
        self.add_node(v)
        self._adj[u].push_back(v)
        self._adj[v].push_back(u)

    def __getitem__(self, int u):
        return self._adj[u]

    cdef vector[int] _degrees(self):
        cdef vector[int] deg
        cdef int first = 0
        cdef vector[int] edges
        cdef cppmap[int, vector[int]].iterator it = self._adj.begin()
        while it != self._adj.end():
            deg.push_back(deref(it).second.size())
            it = inc(it)
        return deg

    def degrees(self):
        return self._degrees()

g0 = Graph()

g0.add_edge(1, 5)
g0.add_edge(1, 6)

g0[1]

g0.has_node(1)

g0.degrees()

import networkx as nx
g = nx.barabasi_albert_graph(100000, 6)
with open('graph.txt', 'w') as fout:
    for u, v in g.edges_iter():
        fout.write('%i,%i\n' % (u, v))

%timeit list(g.degree())

myg = Graph()
def line2edges(line):
    u, v = map(int, line.rstrip().split(','))
    return u, v

edges = map(line2edges, open('graph.txt'))

for u, v in edges:
    myg.add_edge(u, v)

%timeit mydeg = myg.degrees()


I am working on the file "Introduction to Cython.ipynb".
The first task is:
Exercise (if time)
Write a parallel matrix multiplication routine.
Part 4: interact with C/C++ code
Can you write Python code for it?

%%cython -a
# distutils: language=c++
import cython
from libcpp.vector cimport vector

@cython.boundscheck(False)
def build_list_with_vector(double[::1] in_arr):
    cdef vector[double] out
    cdef int i
    for i in range(in_arr.shape[0]):
        out.push_back(in_arr[i])
    return out

build_list_with_vector(np.random.rand(10))


I am working on the file "Introduction to Cython.ipynb".
The first task is:
Not too bad, but this can add up. Let's see if Cython can do better:
Can you write Python code for it?

%load_ext cython

%%cython

def f2(x):
    y = x**4 - 3*x
    return y

def integrate_f2(a, b, n):
    dx = (b - a) / n
    dx2 = dx / 2
    s = f2(a) * dx2
    for i in range(1, n):
        s += f2(a + i * dx) * dx
    s += f2(b) * dx2
    return s

%timeit integrate_f2(-100, 100, int(1e5))


I am working on the file "Introduction to Cython.ipynb".
The first task is:
Rubbish! How do we fix this?
Exercise: use %%cython -a to speed up the code
Part 3: write parallel code
Warning:: Dragons afoot.
Can you write Python code for it?

%%cython -a
import cython
from cython.parallel import prange
import numpy as np

@cython.boundscheck(False)
def mean3filter3(double[::1] arr, double[::1] out):
    cdef int i, j, k = arr.shape[0]-1
    with nogil:
        for i in prange(1, k-1, schedule='static',
                        chunksize=(k-2) // 2, num_threads=2):
            for j in range(i-1, i+1):
                out[i] += arr[j]
            out[i] /= 3
    out[0] = (arr[0] + arr[1]) / 2
    out[-1] = (arr[-1] + arr[-2]) / 2
    return np.asarray(out)

rin = np.random.rand(1e7)
rout = np.empty_like(rin)

%timeit mean3filter2(rin, rout)

%timeit mean3filter3(rin, rout)


I am working on the file "Introduction to Cython.ipynb".
The first task is:
That's a lot of yellow still! How do we reduce this?
Exercise: change the f4 declaration to C
Part 2: work with NumPy arrays
This is a very small subset of Python. Most scientific application deal not with single values, but with arrays of data.
Can you write Python code for it?

import numpy as np

def mean3filter(arr):
    arr_out = np.empty_like(arr)
    for i in range(1, arr.shape[0] - 1):
        arr_out[i] = np.sum(arr[i-1 : i+1]) / 3
    arr_out[0] = (arr[0] + arr[1]) / 2
    arr_out[-1] = (arr[-1] + arr[-2]) / 2
    return arr_out

%timeit mean3filter(np.random.rand(1e5))

%%cython
import cython
import numpy as np

@cython.boundscheck(False)
def mean3filter2(double[::1] arr):
    cdef double[::1] arr_out = np.empty_like(arr)
    cdef int i
    for i in range(1, arr.shape[0]-1):
        arr_out[i] = np.sum(arr[i-1 : i+1]) / 3
    arr_out[0] = (arr[0] + arr[1]) / 2
    arr_out[-1] = (arr[-1] + arr[-2]) / 2
    return np.asarray(arr_out)

%timeit mean3filter2(np.random.rand(1e5))


I am working on the file "Introduction to Cython.ipynb".
The first task is:
The final bit of "easy" Cython optimization is "declaring" the variables inside the function:
Can you write Python code for it?

%%cython

def f4(double x):
    y = x**4 - 3*x
    return y

def integrate_f4(double a, double b, int n):
    cdef:
        double dx = (b - a) / n
        double dx2 = dx / 2
        double s = f4(a) * dx2
        int i = 0
    for i in range(1, n):
        s += f4(a + i * dx) * dx
    s += f4(b) * dx2
    return s

%timeit integrate_f4(-100, 100, int(1e5))


I am working on the file "Introduction to Cython.ipynb".
The first task is:
Using Cython in production code
Use setup.py to build your Cython files.
```python
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy as np
setup(
  cmdclass = {'build_ext': build_ext},
  ext_modules = [
    Extension("prange_demo", ["prange_demo.pyx"],
              include_dirs=[np.get_include()],
              extra_compile_args=['-fopenmp'],
              extra_link_args=['-fopenmp', '-lgomp']),
  ]
)
```
Exercise
Write a Cython module with a setup.py to run the mean-3 filter, then import from the notebook.
Can you write Python code for it?

import numpy as np
from mean3 import mean3filter
mean3filter(np.random.rand(10))


from mean3 import mean3filter
mean3filter(np.random.rand(10))


I am working on the file "Introduction to Data Science in Python/Week 2/.ipynb_checkpoints/Assignment2-checkpoint.ipynb".
The first task is:
Part 2
For the next set of questions, we will be using census data from the United States Census Bureau. Counties are political and geographic subdivisions of states in the United States. This dataset contains population data for counties and states in the US from 2010 to 2015. See this document for a description of the variable names.
The census dataset (census.csv) should be loaded as census_df. Answer questions using this as appropriate.
Question 5
Which state has the most counties in it? (hint: consider the sumlevel key carefully! You'll need this for future questions too...)
This function should return a single string value.
Can you write Python code for it?

census_df = pd.read_csv('census.csv')
census_df.head()

def answer_five():
    census_copy = census_df.copy()
    census_copy = census_copy.reset_index()
    census_copy = census_copy.groupby("STNAME")["CTYNAME"].count().sort_values()
    return census_copy.keys()[-1:][0]
answer_five()


I am working on the file "Introduction to Data Science in Python/Week 2/.ipynb_checkpoints/Assignment2-checkpoint.ipynb".
The first task is:
Question 0 (Example)
What is the first country in df?
This function should return a Series.
Can you write Python code for it?

# You should write your whole answer within the function provided. The autograder will call
# this function and compare the return value against the correct solution value
def answer_zero():
    # This function returns the row for Afghanistan, which is a Series object. The assignment
    # question description will tell you the general format the autograder is expecting
    return df.iloc[0]

# You can examine what your function returns by calling it in the cell. If you have questions
# about the assignment formats, check out the discussion forums for any FAQs
answer_zero()


I am working on the file "Introduction to Data Science in Python/Week 2/.ipynb_checkpoints/Assignment2-checkpoint.ipynb".
The first task is:
Question 2
Which country had the biggest difference between their summer and winter gold medal counts?
This function should return a single string value.
Can you write Python code for it?

def answer_two():
    df_copy = df.copy()
    df_copy.index.names = ["Country"]
    df_copy = df_copy.reset_index()
    df_copy.loc[:,'diff'] = pd.Series(df_copy["Gold"] - df_copy["Gold.1"], index=df_copy.index)
    df_copy = df_copy.set_index("diff")
    df_copy = df_copy.sort_index()
    return df_copy.iloc[len(df_copy)-1]["Country"]
answer_two()


I am working on the file "Introduction to Data Science in Python/Week 2/.ipynb_checkpoints/Assignment2-checkpoint.ipynb".
The first task is:
Question 3
Which country has the biggest difference between their summer gold medal counts and winter gold medal counts relative to their total gold medal count?
$$\frac{Summer~Gold - Winter~Gold}{Total~Gold}$$
Only include countries that have won at least 1 gold in both summer and winter.
This function should return a single string value.
Can you write Python code for it?

def answer_three():
    df_copy = df.copy()
    only_gold = df[(df["Gold"]>0) & (df["Gold.1"]>0)]
    only_gold.index.names = ["Country"]
    only_gold = only_gold.reset_index()
    only_gold.loc[:,"avg"] = pd.Series((only_gold["Gold"] - only_gold["Gold.1"])/only_gold["Gold.2"], index=only_gold.index)
    only_gold = only_gold.set_index("avg")
    only_gold = only_gold.sort_index(ascending=False)
    return only_gold.iloc[0]["Country"]
answer_three()


I am working on the file "Introduction to Data Science in Python/Week 2/.ipynb_checkpoints/Assignment2-checkpoint.ipynb".
The first task is:
Question 7
Which county has had the largest absolute change in population within the period 2010-2015? (Hint: population values are stored in columns POPESTIMATE2010 through POPESTIMATE2015, you need to consider all six columns.)
e.g. If County Population in the 5 year period is 100, 120, 80, 105, 100, 130, then its largest change in the period would be |130-80| = 50.
This function should return a single string value.
Can you write Python code for it?

import numpy as np
def answer_seven():
    census_df = pd.read_csv("census.csv", encoding="ISO-8859-1")
    census = census_df[census_df["SUMLEV"]==50]
    columns_to_keep = ["CTYNAME","POPESTIMATE2010","POPESTIMATE2011","POPESTIMATE2012","POPESTIMATE2013","POPESTIMATE2014","POPESTIMATE2015"]
    census = census[columns_to_keep]
    census = census.set_index("CTYNAME")
    census = census.apply(lambda x:np.max(x) - np.min(x), axis=1)
    census.sort(ascending=False)
    return census.keys().tolist()[0]
answer_seven()


I am working on the file "Introduction to Data Science in Python/Week 2/.ipynb_checkpoints/Assignment2-checkpoint.ipynb".
The first task is:
Question 8
In this datafile, the United States is broken up into four regions using the "REGION" column.
Create a query that finds the counties that belong to regions 1 or 2, whose name starts with 'Washington', and whose POPESTIMATE2015 was greater than their POPESTIMATE 2014.
This function should return a 5x2 DataFrame with the columns = ['STNAME', 'CTYNAME'] and the same index ID as the census_df (sorted ascending by index).
Can you write Python code for it?

def answer_eight():
    census_df = pd.read_csv("census.csv", encoding="ISO-8859-1")
    census = census_df[census_df["SUMLEV"]==50]
    columns_to_keep = ['REGION', 'STNAME', 'CTYNAME', 'POPESTIMATE2014', 'POPESTIMATE2015']
    census = census[columns_to_keep]
    census = census[((census['REGION'] == 1) | (census['REGION'] == 2)) & (census["CTYNAME"].str.contains("Washington")) & (census['POPESTIMATE2015'] > census['POPESTIMATE2014'])]
    census = census.drop(census.columns[[0,3,4]], axis=1)
    return census
answer_eight()


I am working on the file "Introduction to Data Science in Python/Week 2/Assignment2.ipynb".
The first task is:
Part 2
For the next set of questions, we will be using census data from the United States Census Bureau. Counties are political and geographic subdivisions of states in the United States. This dataset contains population data for counties and states in the US from 2010 to 2015. See this document for a description of the variable names.
The census dataset (census.csv) should be loaded as census_df. Answer questions using this as appropriate.
Question 5
Which state has the most counties in it? (hint: consider the sumlevel key carefully! You'll need this for future questions too...)
This function should return a single string value.
Can you write Python code for it?


I am working on the file "Introduction to Data Science in Python/Week 2/Assignment2.ipynb".
The first task is:
Question 2
Which country had the biggest difference between their summer and winter gold medal counts?
This function should return a single string value.
Can you write Python code for it?


I am working on the file "Introduction to Data Science in Python/Week 2/Assignment2.ipynb".
The first task is:
Question 3
Which country has the biggest difference between their summer gold medal counts and winter gold medal counts relative to their total gold medal count?
$$\frac{Summer~Gold - Winter~Gold}{Total~Gold}$$
Only include countries that have won at least 1 gold in both summer and winter.
This function should return a single string value.
Can you write Python code for it?


I am working on the file "Introduction to Data Science in Python/Week 2/Assignment2.ipynb".
The first task is:
Question 4
Write a function that creates a Series called "Points" which is a weighted value where each gold medal (Gold.2) counts for 3 points, silver medals (Silver.2) for 2 points, and bronze medals (Bronze.2) for 1 point. The function should return only the column (a Series object) which you created.
This function should return a Series named Points of length 146
Can you write Python code for it?

def answer_four():
    df_copy = df.copy()
    Points = pd.Series(df_copy["Gold.2"]*3 + df_copy["Silver.2"]*2 + df_copy["Bronze.2"]*1)
    return Points
answer_four()


I am working on the file "Introduction to Data Science in Python/Week 2/Assignment2.ipynb".
The first task is:
Question 6
Only looking at the three most populous counties for each state, what are the three most populous states (in order of highest population to lowest population)? Use CENSUS2010POP.
This function should return a list of string values.
Can you write Python code for it?

def answer_six():
    census_df = pd.read_csv("census.csv", encoding="ISO-8859-1")
    census = census_df[census_df["SUMLEV"]==50]
    columns_to_keep = ["STNAME","CTYNAME","CENSUS2010POP"]
    census = census[columns_to_keep]
    census = census.reset_index()
    census = census.groupby('STNAME')
    census = census.apply(lambda x: x.sort_values('CENSUS2010POP', ascending=False).head(3))
    census = census.groupby('STNAME')
    census = census["CENSUS2010POP"].apply(lambda x: x.sum())
    census.sort_values(ascending=False)
    census = census.head(3)
    return census.keys().tolist()
answer_six()


I am working on the file "Introduction to Data Science in Python/Week 2/Assignment2.ipynb".
The first task is:
Question 8
In this datafile, the United States is broken up into four regions using the "REGION" column.
Create a query that finds the counties that belong to regions 1 or 2, whose name starts with 'Washington', and whose POPESTIMATE2015 was greater than their POPESTIMATE 2014.
This function should return a 5x2 DataFrame with the columns = ['STNAME', 'CTYNAME'] and the same index ID as the census_df (sorted ascending by index).
Can you write Python code for it?


I am working on the file "Introduction to Data Science in Python/Week 2/Assignment2.ipynb".
The first task is:
You are currently looking at version 1.2 of this notebook. To download notebooks and datafiles, as well as get help on Jupyter notebooks in the Coursera platform, visit the Jupyter Notebook FAQ course resource.

Assignment 2 - Pandas Introduction
All questions are weighted the same in this assignment.
Part 1
The following code loads the olympics dataset (olympics.csv), which was derrived from the Wikipedia entry on All Time Olympic Games Medals, and does some basic data cleaning.
The columns are organized as # of Summer games, Summer medals, # of Winter games, Winter medals, total # number of games, total # of medals. Use this dataset to answer the questions below.
Can you write Python code for it?

import pandas as pd

df = pd.read_csv('olympics.csv', index_col=0, skiprows=1)

for col in df.columns:
    if col[:2]=='01':
        df.rename(columns={col:'Gold'+col[4:]}, inplace=True)
    if col[:2]=='02':
        df.rename(columns={col:'Silver'+col[4:]}, inplace=True)
    if col[:2]=='03':
        df.rename(columns={col:'Bronze'+col[4:]}, inplace=True)
    if col[:1]=='№':
        df.rename(columns={col:'#'+col[1:]}, inplace=True)

names_ids = df.index.str.split('\s\(') # split the index by '('

df.index = names_ids.str[0] # the [0] element is the country name (new index)
df['ID'] = names_ids.str[1].str[:3] # the [1] element is the abbreviation or ID (take first 3 characters from that)

df = df.drop('Totals')
df.head()


I am working on the file "Introduction to Data Science in Python/Week 3/.ipynb_checkpoints/Assignment3-checkpoint.ipynb".
The first task is:
<br>
Answer the following questions in the context of only the top 15 countries by Scimagojr Rank (aka the DataFrame returned by answer_one())
Question 3 (6.6%)
What is the average GDP over the last 10 years for each country? (exclude missing values from this calculation.)
This function should return a Series named avgGDP with 15 countries and their average GDP sorted in descending order.
Can you write Python code for it?


I am working on the file "Introduction to Data Science in Python/Week 3/.ipynb_checkpoints/Assignment3-checkpoint.ipynb".
The first task is:
Question 10 (6.6%)
Create a new column with a 1 if the country's % Renewable value is at or above the median for all countries in the top 15, and a 0 if the country's % Renewable value is below the median.
This function should return a series named HighRenew whose index is the country name sorted in ascending order of rank.
Can you write Python code for it?

def answer_ten():
    Top15 = answer_one()
    median = Top15['% Renewable'].median()
    Top15['HighRenew'] = 1
    Top15['HighRenew'] = Top15.where(Top15['% Renewable'] >= median, 0)
    Top15['HighRenew'] = Top15['HighRenew'].map(lambda x:0 if x == 0 else 1)
    return Top15['HighRenew']
answer_ten()


I am working on the file "Introduction to Data Science in Python/Week 3/.ipynb_checkpoints/Assignment3-checkpoint.ipynb".
The first task is:
Question 11 (6.6%)
Use the following dictionary to group the Countries by Continent, then create a dateframe that displays the sample size (the number of countries in each continent bin), and the sum, mean, and std deviation for the estimated population of each country.
python
ContinentDict  = {'China':'Asia',
                  'United States':'North America',
                  'Japan':'Asia',
                  'United Kingdom':'Europe',
                  'Russian Federation':'Europe',
                  'Canada':'North America',
                  'Germany':'Europe',
                  'India':'Asia',
                  'France':'Europe',
                  'South Korea':'Asia',
                  'Italy':'Europe',
                  'Spain':'Europe',
                  'Iran':'Asia',
                  'Australia':'Australia',
                  'Brazil':'South America'}
This function should return a DataFrame with index named Continent ['Asia', 'Australia', 'Europe', 'North America', 'South America'] and columns ['size', 'sum', 'mean', 'std']
Can you write Python code for it?

def answer_eleven():
    Top15 = answer_one()
    ContinentDict  = {'China':'Asia',
                  'United States':'North America',
                  'Japan':'Asia',
                  'United Kingdom':'Europe',
                  'Russian Federation':'Europe',
                  'Canada':'North America',
                  'Germany':'Europe',
                  'India':'Asia',
                  'France':'Europe',
                  'South Korea':'Asia',
                  'Italy':'Europe',
                  'Spain':'Europe',
                  'Iran':'Asia',
                  'Australia':'Australia',
                  'Brazil':'South America'}
    Top15['Continent'] = None
    Top15 = Top15.reset_index()
    Top15['Continent'] = Top15['Country'].map(lambda x: ContinentDict[x] if x in ContinentDict else None)
    new_df = pd.DataFrame(Top15.groupby(by='Continent').size(), columns=['size'])
    Top15 = Top15.set_index('Country')
    Top15['pop'] = pd.Series(Top15['Energy Supply']/Top15['Energy Supply per Capita'])
    new_df['sum'] = Top15.groupby(by='Continent')['pop'].apply(np.sum).astype(float)
    new_df['mean'] = Top15.groupby(by='Continent')['pop'].apply(np.mean)
    new_df['std'] = Top15.groupby(by='Continent')['pop'].apply(np.std)
    return new_df
answer_eleven()


I am working on the file "Introduction to Data Science in Python/Week 3/.ipynb_checkpoints/Assignment3-checkpoint.ipynb".
The first task is:
Question 13 (6.6%)
Convert the Population Estimate series to a string with thousands separator (using commas). Do not round the results.
e.g. 317615384.61538464 -> 317,615,384.61538464
This function should return a Series PopEst whose index is the country name and whose values are the population estimate string.
Can you write Python code for it?

import locale
def answer_thirteen():
    Top15 = answer_one()
    PopEst = pd.Series(Top15['Energy Supply']/Top15['Energy Supply per Capita'])
    PopEst = PopEst.map(lambda x: str(format(int(str(x).split(".")[0]),",d")) + "." + str(x).split(".")[1])
    return PopEst
answer_thirteen()


I am working on the file "Introduction to Data Science in Python/Week 3/.ipynb_checkpoints/Assignment3-checkpoint.ipynb".
The first task is:
Question 2 (6.6%)
The previous question joined three datasets then reduced this to just the top 15 entries. When you joined the datasets, but before you reduced this to the top 15 items, how many entries did you lose?
This function should return a single number.
Can you write Python code for it?

%%HTML
<svg width="800" height="300">
  <circle cx="150" cy="180" r="80" fill-opacity="0.2" stroke="black" stroke-width="2" fill="blue" />
  <circle cx="200" cy="100" r="80" fill-opacity="0.2" stroke="black" stroke-width="2" fill="red" />
  <circle cx="100" cy="100" r="80" fill-opacity="0.2" stroke="black" stroke-width="2" fill="green" />
  <line x1="150" y1="125" x2="300" y2="150" stroke="black" stroke-width="2" fill="black" stroke-dasharray="5,3"/>
  <text  x="300" y="165" font-family="Verdana" font-size="35">Everything but this!</text>
</svg>

def answer_two():
    energy = pd.read_excel("Energy Indicators.xls", skiprows=17, skip_footer=38, names=['#1', '#2', 'Country', 'Energy Supply', 'Energy Supply per Capita', '% Renewable'], na_values="...").drop(['#1', '#2'], axis=1)
    energy['Energy Supply'] = energy['Energy Supply'] * 1000000
    energy['Country'] = energy['Country'].map(lambda x: x.rstrip('0123456789'))
    energy['Country'] = energy['Country'].map(lambda x: x[:x.find("(")-1] if x.find("(") != -1  else x)
    columns={"Republic of Korea": "South Korea", "United States of America": "United States", "United Kingdom of Great Britain and Northern Ireland": "United Kingdom", "China, Hong Kong Special Administrative Region": "Hong Kong"}
    energy['Country'] = energy['Country'].map(lambda x: columns[x] if x in columns else x)
    energy = energy.set_index('Country')
    GDP = pd.read_csv("world_bank.csv", header=None, skiprows=5)
    columns_to_keep = [num for num in range(GDP.columns.__len__()-10,GDP.columns.__len__())]
    columns_to_keep.insert(0, 0)
    GDP = GDP[columns_to_keep]
    column_names = ["Country","2006","2007","2008","2009","2010","2011","2012","2013","2014","2015"]
    GDP.columns = column_names
    columns = {"Korea, Rep.": "South Korea", "Iran, Islamic Rep.": "Iran", "Hong Kong SAR, China": "Hong Kong"}
    GDP['Country'] = GDP['Country'].map(lambda x: columns[x] if x in columns else x)
    GDP = GDP.rename(columns={'Country':'Country'})
    GDP = GDP.set_index('Country')
   ...


I am working on the file "Introduction to Data Science in Python/Week 3/.ipynb_checkpoints/Assignment3-checkpoint.ipynb".
The first task is:
Question 5 (6.6%)
What is the mean Energy Supply per Capita?
This function should return a single number.
Can you write Python code for it?

def answer_five():
    Top15 = answer_one()
    return Top15['Energy Supply per Capita'].mean()
answer_five()


I am working on the file "Introduction to Data Science in Python/Week 3/.ipynb_checkpoints/Assignment3-checkpoint.ipynb".
The first task is:
Question 9 (6.6%)
Create a column that estimates the number of citable documents per person.
What is the correlation between the number of citable documents per capita and the energy supply per capita? Use the .corr() method, (Pearson's correlation).
This function should return a single number.
(Optional: Use the built-in function plot9() to visualize the relationship between Energy Supply per Capita vs. Citable docs per Capita)
Can you write Python code for it?

def answer_nine():
    Top15 = answer_one()
    Top15['PopEst'] = Top15['Energy Supply'] / Top15['Energy Supply per Capita']
    Top15['Citable docs per Capita'] = Top15['Citable documents'] / Top15['PopEst']
    answer = Top15.corr().loc['Energy Supply per Capita']['Citable docs per Capita']
    return answer
answer_nine()

#plot9() # Be sure to comment out plot9() before submitting the assignment!


I am working on the file "Introduction to Data Science in Python/Week 3/Assignment3.ipynb".
The first task is:
<br>
Answer the following questions in the context of only the top 15 countries by Scimagojr Rank (aka the DataFrame returned by answer_one())
Question 3 (6.6%)
What is the average GDP over the last 10 years for each country? (exclude missing values from this calculation.)
This function should return a Series named avgGDP with 15 countries and their average GDP sorted in descending order.
Can you write Python code for it?


I am working on the file "Introduction to Data Science in Python/Week 3/Assignment3.ipynb".
The first task is:
Optional
Use the built in function plot_optional() to see an example visualization.
Can you write Python code for it?

#plot_optional() # Be sure to comment out plot_optional() before submitting the assignment!


I am working on the file "Introduction to Data Science in Python/Week 3/Assignment3.ipynb".
The first task is:
Question 10 (6.6%)
Create a new column with a 1 if the country's % Renewable value is at or above the median for all countries in the top 15, and a 0 if the country's % Renewable value is below the median.
This function should return a series named HighRenew whose index is the country name sorted in ascending order of rank.
Can you write Python code for it?


I am working on the file "Introduction to Data Science in Python/Week 3/Assignment3.ipynb".
The first task is:
Question 11 (6.6%)
Use the following dictionary to group the Countries by Continent, then create a dateframe that displays the sample size (the number of countries in each continent bin), and the sum, mean, and std deviation for the estimated population of each country.
python
ContinentDict  = {'China':'Asia',
                  'United States':'North America',
                  'Japan':'Asia',
                  'United Kingdom':'Europe',
                  'Russian Federation':'Europe',
                  'Canada':'North America',
                  'Germany':'Europe',
                  'India':'Asia',
                  'France':'Europe',
                  'South Korea':'Asia',
                  'Italy':'Europe',
                  'Spain':'Europe',
                  'Iran':'Asia',
                  'Australia':'Australia',
                  'Brazil':'South America'}
This function should return a DataFrame with index named Continent ['Asia', 'Australia', 'Europe', 'North America', 'South America'] and columns ['size', 'sum', 'mean', 'std']
Can you write Python code for it?


I am working on the file "Introduction to Data Science in Python/Week 3/Assignment3.ipynb".
The first task is:
Question 12 (6.6%)
Cut % Renewable into 5 bins. Group Top15 by the Continent, as well as these new % Renewable bins. How many countries are in each of these groups?
This function should return a Series with a MultiIndex of Continent, then the bins for % Renewable. Do not include groups with no countries.
Can you write Python code for it?

def answer_twelve():
    Top15 = answer_one()
    ContinentDict  = {'China':'Asia',
                  'United States':'North America',
                  'Japan':'Asia',
                  'United Kingdom':'Europe',
                  'Russian Federation':'Europe',
                  'Canada':'North America',
                  'Germany':'Europe',
                  'India':'Asia',
                  'France':'Europe',
                  'South Korea':'Asia',
                  'Italy':'Europe',
                  'Spain':'Europe',
                  'Iran':'Asia',
                  'Australia':'Australia',
                  'Brazil':'South America'}
    Top15 = Top15.reset_index()
    Top15['bins'] = pd.cut(Top15['% Renewable'], bins=5)
    Top15['Continent'] = Top15['Country'].map(lambda x: ContinentDict[x] if x in ContinentDict else None)
    Top15 = Top15.groupby(by=['Continent','bins']).size()
    return Top15
answer_twelve()


I am working on the file "Introduction to Data Science in Python/Week 3/Assignment3.ipynb".
The first task is:
Question 2 (6.6%)
The previous question joined three datasets then reduced this to just the top 15 entries. When you joined the datasets, but before you reduced this to the top 15 items, how many entries did you lose?
This function should return a single number.
Can you write Python code for it?


I am working on the file "Introduction to Data Science in Python/Week 3/Assignment3.ipynb".
The first task is:
Question 4 (6.6%)
By how much had the GDP changed over the 10 year span for the country with the 6th largest average GDP?
This function should return a single number.
Can you write Python code for it?

def answer_four():
    Top15 = answer_one()
    series = Top15[['2006', '2007', '2008','2009', '2010', '2011', '2012', '2013', '2014', '2015']].mean(axis=1).sort_values(ascending=False)
    country = series.keys()[5]
    min_val = Top15.loc[country]['2006']
    max_val = Top15.loc[country]['2015']
    return max_val - min_val
answer_four()


I am working on the file "Introduction to Data Science in Python/Week 3/Assignment3.ipynb".
The first task is:
Question 6 (6.6%)
What country has the maximum % Renewable and what is the percentage?
This function should return a tuple with the name of the country and the percentage.
Can you write Python code for it?

def answer_six():
    Top15 = answer_one()
    answer = (Top15['% Renewable'].sort_values()[-1:].keys()[0], Top15['% Renewable'].max())
    return answer
answer_six()


I am working on the file "Introduction to Data Science in Python/Week 3/Assignment3.ipynb".
The first task is:
Question 8 (6.6%)
Create a column that estimates the population using Energy Supply and Energy Supply per capita.
What is the third most populous country according to this estimate?
This function should return a single string value.
Can you write Python code for it?

def answer_eight():
    Top15 = answer_one()
    col = pd.Series(Top15['Energy Supply']/Top15['Energy Supply per Capita'])
    col = col.sort_values(ascending=False)
    answer = col.keys()[2]
    return answer
answer_eight()


I am working on the file "Introduction to Probability.ipynb".
The first task is:
Exercise 1:
Do the same simulation for die experiment. You should get stable line at 1/6 when number of throws is higher(~n=10K).
Can you write Python code for it?

from IPython.core.pylabtools import figsize
import numpy as np
from matplotlib import pyplot as plt
figsize(11, 9)

import scipy.stats as stats

dist = stats.beta
n_trials = [0, 1, 2, 3, 4, 5, 8, 15, 50, 500]
data = stats.bernoulli.rvs(0.5, size=n_trials[-1])
x = np.linspace(0, 1, 100)

# For the already prepared, I'm using Binomial's conj. prior.
for k, N in enumerate(n_trials):
    sx = plt.subplot(len(n_trials)/2, 2, k+1)
    plt.xlabel("$p$, probability of heads") \
        if k in [0, len(n_trials)-1] else None
    plt.setp(sx.get_yticklabels(), visible=False)
    heads = data[:N].sum()
    y = dist.pdf(x, 1 + heads, 1 + N - heads)
    plt.plot(x, y, label="observe %d tosses,\n %d heads" % (N, heads))
    plt.fill_between(x, 0, y, color="#348ABD", alpha=0.4)
    plt.vlines(0.5, 0, 4, color="k", linestyles="--", lw=1)

    leg = plt.legend()
    leg.get_frame().set_alpha(0.4)
    plt.autoscale(tight=True)

plt.suptitle("Bayesian updating of posterior probabilities",
             y=1.02,
             fontsize=14)

plt.tight_layout()


I am working on the file "Introduction to Probability.ipynb".
The first task is:
For n=100000
Can you write Python code for it?

# TODO: make this plot more beautiful(with grids and color) and should be able to show value on hover
flips = comp_prob_inference.flip_fair_coins(100000)
comp_prob_inference.plot_discrete_histogram(flips, frequency=True)

n = 100000
heads_so_far = 0
fraction_of_heads = []
for i in range(n):
    if comp_prob_inference.flip_fair_coin() == 'heads':
        heads_so_far += 1
    fraction_of_heads.append(heads_so_far / (i+1))

plt.figure(figsize=(8, 4))
plt.plot(range(1, n+1), fraction_of_heads)
plt.xlabel('Number of flips')
plt.ylabel('Fraction of heads')


I am working on the file "Introduction to Probability.ipynb".
The first task is:
Seems like it will take endless time click everyone and record it to prove as we increase n then probability of Heads is going to be: $$\frac{1}{2}$$

and same for Tails.
 Lets try with a different way - another virtual coin flipping and comparing it with theoretical probability of Heads and Tails $$\frac{1}{2}$$
Can you write Python code for it?

from IPython.display import HTML
HTML('<iframe src="http://localhost/Seeing-Theory/basic-probability/index.html#first" width="100%" height="700px" scrolling="no" style="margin-top: -70px;" frameborder="0"></iframe>')


I am working on the file "Introduction to Probability.ipynb".
The first task is:
Still taking lot of time to get 0.5 probability. Lets try it with Python
Simulating coin-toss experiment with Python
Can you write Python code for it?

%matplotlib inline
from utils import comp_prob_inference
import matplotlib.pyplot as plt

comp_prob_inference.flip_fair_coin()

flips = comp_prob_inference.flip_fair_coins(100)

comp_prob_inference.plot_discrete_histogram(flips)

comp_prob_inference.plot_discrete_histogram(flips, frequency=True)


I am working on the file "Introduction to Python for Data Analysis.ipynb".
The first task is:
Exercise: early data exploration and pandas syntax
Practice the following steps with these 4 data sets located at http://teachingdatascience.github.io/Rdatasets/datasets.html.
Copy the csv link and use that with read_csv to import it.
DATA SETS

chickwts
infert
msleep
Cars93

INSTRUCTIONS

import the data into python using pd.read_csv()
call the dtypes variable to identify the columns and their data types.
call the describe function to collect statistics about numerical data types.
for each other column (object columns):use
dataframe['column'].unique()  and
dataframe.groupby('column')['column'].count()

Write a short markdown that describes the data set and its columns.

Data Munging basics with pandas
What interesting data point do we learn when we run the following code?
Can you write Python code for it?


I am working on the file "Introduction to Python for Data Analysis.ipynb".
The first task is:
Like with everything else in python, the DataFrame is an object. We'll use a function called type to identify the object name:
Can you write Python code for it?


I am working on the file "Introduction to Python for Data Analysis.ipynb".
The first task is:
Pivot and Aggregate
Earlier we saw use of the function groupby. A pandas groupby object has other functionality, such as returning statistics about a column, or all columns, or aggregating only what's defined.
Similarly, pivot tables can be used to explore groups side by side.
Can you write Python code for it?

# import numpy as np
nyt['column_of_ones'] = 1
nyt['Click_Thru'] = nyt['Clicks'] / nyt['Impressions']
nyt['Click_Thru'] = nyt['Click_Thru'].apply(lambda x: 0 if x == np.inf else x)
nyt_signedin_only = nyt[nyt.Signed_In == 1]
nyt_group = nyt.groupby('Age')

print nyt_group.mean()

print nyt_group['Click_Thru'].mean()

print nyt_group.agg({
    'Click_Thru': ['mean', 'max'],
    'Gender': 'mean',
})

print nyt[nyt.Signed_In == 1].pivot_table(
    values='column_of_ones',
    index='Gender',
    columns='Age',
    aggfunc='count'
)


I am working on the file "Introduction to Python for Data Analysis.ipynb".
The first task is:
Practice filtering for the following:

rows where age is less than 30
rows where gender is 0 but signed_in is 1
rows where age is less than 30 or more than 60.

Selecting Rows, Columns, and Finding Uniques
Earlier we selected columns (or a list of columns) using key notation (like with dictionaries). Pandas also accepts dot notation (akin to an object mapper, or javascript object notation).
To select rows based on the index, we can use .ix[], which accepts either keys (when the index is a string) or a range (when the index is integer based). (pandas does allow more specific typing, loc and iloc)
Otherwise, passing a range in will do fine.
Can you write Python code for it?

# Finding the first ten rows
print nyt[0:10]

# This will still return the first ten rows
print nyt.set_index('Age')[0:10]

# Since the index is positional, this will also return the first 10 rows.
print nyt.ix[0:10]

# This first one will work fine, as it returns back all rows where Age == 40.
# The second will fail, as .ix does not treat the index as positional, when the index does not represent the row order.
print nyt.set_index('Age').ix[40]
try:
    print nyt.set_index('Age').ix[40:45]
except Exception as e:
    print e.message

# Using dot notation and returning uniques of that column
print nyt.Age.head()
print nyt.Age.unique()


I am working on the file "Introduction to Python for Data Analysis.ipynb".
The first task is:
This translates to:

Found in the pandas library...
... in the core module...
... in the frame module...
...as a class, DataFrame

Objects in python are filled with variables and functions, and we use dot notation to access them.
# object.variable
nyt.dtypes
# object.function()
nyt.describe()

One great advantage with iPython is it has tab completion which means we can type nyt., press tab, and it'll show us what variables and functions exist.
Practice this with the following code cell with the following steps:

wrap the code with the type function: what data type does it return?
if it returns an "instancemethod" object, try the wrapper with a () to excute the function. What data type does the function return?
Can you write Python code for it?

# example:
print nyt.dtypes
nyt.dtypes?
print type(nyt.dtypes)

# Try wrapping type with the following:
print nyt.describe
print nyt.describe()
print nyt.shape
print nyt.index
print nyt.columns
print nyt.groupby
print nyt.groupby('Age')
print nyt.Age


I am working on the file "Introduction to Python for Data Analysis.ipynb".
The first task is:
Use these new tools in order to visualize some of this New York Times ad performance data.
Can you write Python code for it?

nyt[nyt.Signed_In == 1].Age.hist()

print nyt[nyt.Signed_In == 1].Clicks.hist()

nyt.describe()

nyt['Click_Thru'] = nyt['Clicks'] / nyt['Impressions']
nyt[nyt.Signed_In == 1].plot(kind='scatter', x='Age', y='Click_Thru')


I am working on the file "Introduction to Python for Data Analysis.ipynb".
The first task is:
Visually from creating the data frame you can tell the data looks different, yet in the .describe() call, the data shares very similar features. The two primary plotting tools we uses from matplotlib are histograms and scatterplots, which help us understand the shape of data.
Can you write Python code for it?

%matplotlib inline
# above magic allows plots to show in our notebook.

for y in ['y1', 'y2', 'y3', 'y4']:
    if y != 'y4':
        print anscombe.plot(kind='scatter', x='x', y=y)
    else:
        print anscombe.plot(kind='scatter', x='x4', y=y)


I am working on the file "Introduction to Python for Data Analysis_Outputs.ipynb".
The first task is:
Basic Data Types: Strings, Lists, Tuples
Try the same thing on your own to learn about the following python objects: strings, lists, and tuples.
Can you write Python code for it?

some_string1 = 'apples'
some_string2 = 'and'
some_string3 = 'bananas'
print some_string1, some_string2, some_string3
print some_string1 + some_string2 + some_string3

mutable_list = ["apple", "apple", "banana", "kiwi", "bear", "strawberry", "strawberry"]
immutable_tuple = ("apple", "apple", "banana", "kiwi", "bear", "strawberry", "strawberry")

print len(some_string1)
print len(mutable_list)
print len(immutable_tuple)

print some_string1[0:5]
print mutable_list[0:4]
print immutable_tuple[5:6]

try:
    some_string1[5] = 'd'
except TypeError as e:
    print e

mutable_list[5] = 'mango'
try:
    immutable_tuple[5] = 'not going to work'
except TypeError as e:
    print e

a = [3 for i in range(10)]
print a


I am working on the file "Introduction to Python for Data Analysis_Outputs.ipynb".
The first task is:
Libraries for Data Science¶
Data Scientists use a wide variety of libraries in Python that make working with data significantly easier. Those libraries primarily consist of:
1.numpy
2.scipy
3.pandas
4.matplotlib
5.statsmodels
6.scikit-learn
7.nltk

Though there are countless others available.
For today, we'll primarily focus ourselves around the library that is 99% of our work: pandas
pandas and object oriented programming¶
pandas is a library built on top of numpy, which allows us to use excel-like matrices in the python programming space. These special matrices are called DataFrames, the primary object in pandas.
Earlier we loaded up the csv file from our computer; pandas can also parse from a URL.
Can you write Python code for it?


I am working on the file "Introduction to Python for Data Analysis_Outputs.ipynb".
The first task is:
Mutate, Transform (apply)
There are three most common techniques in mutating or transforming the data.
One approach would be to directly use values from other columns to derive new columns. The other would be to apply functions to columns using a named or lambda function.
You can generate the new columns by defining their name as a key.
Can you write Python code for it?

nyt['column_of_ones'] = 1

nyt['saw_ad_many_times'] = nyt['Impressions'].apply(lambda x: 1 if x > 5 else 0)

def saw_add_func(x):
    if x > 5:
        return 1
    else:
        return 0

nyt['saw_ad_many_times'] = nyt['Impressions'].apply(saw_add_func)

# practice on one column:
# a common approach to missing data, let's either set the values to null, or -1:
# import numpy as np
# np.nan

# by default apply works with a single column, however you can use it to iterate by row as well (though it is slower)
nyt['not_signed_in'] = nyt.apply(lambda row: 0 if row['Signed_In'] else 1, axis=1)


I am working on the file "Introduction to Python for Data Analysis_Outputs.ipynb".
The first task is:
Pivot and Aggregate
Earlier we saw use of the function groupby. A pandas groupby object has other functionality, such as returning statistics about a column, or all columns, or aggregating only what's defined.
Similarly, pivot tables can be used to explore groups side by side.
Can you write Python code for it?


I am working on the file "Introduction to Python for Data Analysis_Outputs.ipynb".
The first task is:
This translates to:

Found in the pandas library...
... in the core module...
... in the frame module...
...as a class, DataFrame

Objects in python are filled with variables and functions, and we use dot notation to access them.
# object.variable
nyt.dtypes
# object.function()
nyt.describe()

One great advantage with iPython is it has tab completion which means we can type nyt., press tab, and it'll show us what variables and functions exist.
Practice this with the following code cell with the following steps:

wrap the code with the type function: what data type does it return?
if it returns an "instancemethod" object, try the wrapper with a () to excute the function. What data type does the function return?
Can you write Python code for it?


I am working on the file "Introduction to Python for Data Analysis_Outputs.ipynb".
The first task is:
Use these new tools in order to visualize some of this New York Times ad performance data.
Can you write Python code for it?