I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- Cuál será el valor futuro de \$100 en 1, 2, 3 y 4 años a una tasa de interés de 3% anual?
Can you write Python code for it?

cf.pvfv(pval  =  100, # una de las variables puede ser un vector
        nper  =    [1, 2, 3, 4],
        nrate =    3.0)


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- Cuál será el valor futuro de \$100 si se invierte a un año al 1%, a 2 años al 1.5% y a 3 años al 1.7%, con una capitalización anual?
Can you write Python code for it?

cf.tvmm(pval  =  100,               # si varias variables son listas
        pmt   =    0,               # todas ellas deben tener la misma
        nper  =    [1,   2,   3],   # longitud
        nrate =    [1.0, 1.5, 1.7]) #


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- Cuál será el valor futuro de \$100, \$ 200, \$ 300 y \$ 400 en 5 años a una tasa de interés de 3% anual?
Can you write Python code for it?

cf.pvfv(pval  =  [100, 200, 300, 400], # una de las variables puede ser un vector
        nper  =  5,
        nrate =  3.0)


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- [2, 62] Se compra una propiedad por \$ 32000. Si se presenta una depreciación del 2% por año, cuál será el valor de la propiedad al final de 6 años? R/ \$ 28346.96
Can you write Python code for it?

cf.pvfv(pval  = -32000,
        nrate =     -2.0,
        nper  =      6)


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- [2, pag. 43] Realice el mismo ejemplo anterior sobre una base de 365 días.
Can you write Python code for it?


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- [2, pág. 53] Se abre una cuenta hoy con un depósito de \$ 775. La tasa de interés es de (6.25%/24). Si se siguen realizando depósitos de \$ 50, cuánto tiempo se requiere para alcanzar un saldo de \$ 4000?
Can you write Python code for it?

cf.tvmm(nrate =  6.25/24, # tasa de interes
        pval   = -775,    # deposito inicial
        pmt    =  -50,    # depositos periodicos
        fval   = 4000,    # saldo final
        due    = 0)       # depositos al final del periodo

# saldo al final de 58 periodos
cf.tvmm(nrate  =    6.25/24, # tasa de interes
        nper   =   58,       # numero de periodos
        pval   = -775,       # deposito inicial
        pmt    =  -50,       # depositos periodicos
        due    = 0)          # depositos al final del periodo


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- [2, pág. 55] Qué tasa de interés debe obtenerse para acumular \$ 10000 en 32 periodos si se hace una inversión de \$ 6000? R/ 1.61%
Can you write Python code for it?

cf.pvfv(pval = -6000,         # deposito inicial
        nper =    32,         # numero de periodos
        fval = 10000)         # saldo final

cf.tvmm(pval = -6000,     # deposito inicial
        nper =    32,     # numero de periodos
        pmt  = 0,         # pago periodico
        fval = 10000)     # saldo final


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- [2, pág. 58] Cuánto se puede pagar por una propiedad que generará un flujo neto anual de \$ 17500 durante 5 años, si al final la propiedad se puede vender en \$ 540.000? (la tasa de interés es del 12%)
Can you write Python code for it?

cf.tvmm(pmt   =  17500,   # pago periodico anual
        fval  = 540000,   # valor de venta
        nrate =     12.0, # tasa de interés
        nper  =      5)   # numero de periodos


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- [2, pág. 59] Calcule el pago mensual de una hipoteca por \$ 243400 pagada en 348 meses a una tasa del 5.25%/12.
Can you write Python code for it?

cf.pvpmt(pval  = 243400,       # monto
         nrate =      5.25/12, # tasa de interés
         nper  =    348)       # numero de periodos


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- [2, pág. 61] Si se consignan \$ 50 al principio de cada mes en una nueva cuenta que paga 6.25%/12, cuál es el saldo al final de 24 meses?
Can you write Python code for it?

cf.tvmm(pval  =   0,        # monto
        nrate =   6.25/12,  # tasa de interés
        pmt   = -50.0,      # pago mensual
        due   =   1,        # pago al principio del periodo
        nper  =  24)        # numero de periodos


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- [3, pág. 81] Se está financiando la compra de un carro nuevo con un leasing a tres años a una tasa de interés del 10.5%/12. El precio del carro es de \$ 7250. Se debe realizar un pago inicial de \$ 1500. Cuánto es el pago mensual si los pagos se hacen al final del mes?
Can you write Python code for it?

cf.pvpmt(pval  = 7250-1500,
         nrate =   10.5/12,
         nper  =   36)

# si se desea reducir el pago periodico en $ 10, cual tasa de interés debería obtenerse.
cf.pvpmt(pval = 7250-1500,
         pmt  = -176.89,
         nper =   36)


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- [3, pág. 82] El pago máximo mensual por una hipoteca a 30 años es de \$ 630. Si usted puede realizar un pago inicial de \$ 12.000 y la tasa es de 11.5%/12, cuál es el precio máximo de la propiedad?
Can you write Python code for it?


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Ejemplo.-- [3, pág. 83] Se constituye una hipoteca a 25 años por un monto de \$ 75250 y una tasa de interés mensual de 13.8%/12. (a) cuál es el pago mensual; (b) Si se anticipa la venta de la propiedad al final de 4 años, cuánto debe pagarse para cancelar la deuda?
Can you write Python code for it?


I am working on the file "2016-03/IE-01-calculos-basicos.ipynb".
The first task is:
Se se hacen únicamente 327 pagos, cuál es el valor del pago final para cancelar completamente la deuda?
Can you write Python code for it?

m = cf.tvmm(nrate =    10.5/12, # tasa de interes
             pval = 35000,      # valor presente
             pmt  =  -325,
             nper =   327,
             due  =     0)      # 'end'
m

-325 + m


I am working on the file "2016-03/IE-02-representacion-flujos-y-tasas.ipynb".
The first task is:
Conversión de tasas de interés
Interés anticipado e interés vencido
Contenido
<img src="images/antxven.png" width=500>
Interés vencido: se paga al final del periodo.
$$F=P(1+r)$$
Interés anticipado: se paga al inicio del periodo (antes de su causación). En este caso surge una paradoja, que el interés se puede reinvertir a la misma tasa de interés (anticipadamente):
$$F = P + Pr_  + Pr_^2 + ...$$
La suma infinita anterior puede reescribirse como:
$$F=P \frac{1}{1 - r_*}$$
Igualando las dos ecuaciones anteriores:
$$\frac{1}{1 - r_}=1+r ~,~~ r=\frac{r_}{1 - r_} ~,~~ r_=\frac{r}{1 + r}$$
Interés efectivo anual:
$$r_a=\left[1+\left (\frac{r_}{1-r_}\right)\right]^n-1$$
cashflow tiene las funciones nom2eff() y eff2nom() para realizar las conversiones entre nominal (anticipado y vencido) y efectivo respectivamente.
Ejemplo.-- Se solicita un prestamo a un año con un interés anticipado del 20%. ¿Determine el interés efectivo pagado por el dinero?
Can you write Python code for it?


I am working on the file "2016-03/IE-02-representacion-flujos-y-tasas.ipynb".
The first task is:
Ejemplo.-- Se va a tomar un crédito a 48 meses. La tasa inicial es del 3% y aumenta un punto cada año. Represente la tasa de interés.
Can you write Python code for it?

cf.nominal_rate(const_value = 3,
                start = (2000, 0),
                nper = 48,
                pyr = 12,
                spec= [(12, 4),      # tasa para el año 2
                       (24, 5),      # tasa para el año 3
                       (36, 6)])     # tasa para el año 4)

x = cf.nominal_rate(const_value = 3,
                    start = (2000, 0),
                    nper = 48,
                    pyr = 12,
                    spec= [(12, 4),      # tasa para el año 2
                           (24, 5),      # tasa para el año 3
                           (36, 6)])     # tasa para el año 4)
x[5] = 100
x


I am working on the file "2016-03/IE-02-representacion-flujos-y-tasas.ipynb".
The first task is:
Ejemplo.-- Sea un interés nominal del 12% capitalizado mensualmente. Calcule:

Tasa efectiva mensual
Tasa efectiva trimestral
Tasa efectiva anual

<img src="images/tasa-nominal-efectiva.png" width=600>
Can you write Python code for it?

## tasa efectiva mensual
0.12 / 12

## tasa efectiva trimestral
erate, _ = cf.iconv(nrate = 3 * 0.12 / 12, pyr = 3)
erate

## tasa efectiva anual
erate, _ = cf.iconv(nrate = 12.0, pyr = 12)
erate


I am working on the file "2016-03/IE-02-representacion-flujos-y-tasas.ipynb".
The first task is:
Ejemplo.-- Si se desea obtener una tasa efectiva anual del 36%, ¿cuánto se deberá cobrar en forma anticipada anual para obenerla?
Can you write Python code for it?


I am working on the file "2016-03/IE-02-representacion-flujos-y-tasas.ipynb".
The first task is:
En algunos casos es necesario introducir patrones de flujo más complejos.
Can you write Python code for it?

cf.cashflow(const_value=[0, 1, 2, 2, 4, 5, 6, 7, 8])

## para  5 <= t < 10, el valor es $ 100, y 0 en el resto de los casos
cf.cashflow(const_value=0, nper=15, pyr=1, spec=[(t,100) for t in range(5,10)])

## un flujo escalonado
a = [(t, 100) for t in range( 1,  5)]
b = [(t, 150) for t in range( 6, 10)]
c = [(t, 200) for t in range(11, 13)]
cf.cashflow(const_value=0, nper=20, pyr=1, spec=a + b + c)

## flujo con gradiente geométrico (aumento del 5% por periodo)
cf.cashflow(const_value=0, nper=20, pyr=1, spec=[(t, 100 * 1.05 ** (t-5)) for t in range(5,10)])


I am working on the file "2016-03/IE-02-representacion-flujos-y-tasas.ipynb".
The first task is:
Nomenclatura
Contenido
<img src="images/nomenclatura.png" width=600>

Ejercicios
Ejercicio.-- Cuál es la tasa efectiva anual equivalente a 15% N.A.M.V. (nominal anual mes vencido)?
Ejercicio.-- Cuál es la tasa efectiva anual equivalente a 23% N.A.T.A. (nominal anual trimestre anticipado)?
Ejercicio.-- Sea un interés nominal del 39.29% capitalizado mensualmente a cuánto equivale en términos semestrales? (R/ 6.15%)
Ejercicio.-- ¿Cuál es el valor futuro de \$ 609 dentro de 2 años a una tasa del 2% NATV?
Ejercicio.-- ¿Cuál es el valor presente de un pago único de \$ 890 recibido dentro de 6 años a una tasa de 2.7% NATA?
Ejercicio.-- ¿Qué cantidad de dinero se poseerá después de prestar \$ 2300 al 27% NAMA durante 3 años?
Ejercicio.-- ¿Cuál es la tasa efectiva semestral equivalente al 14% NMTA?
Ejercicio.-- ¿Cuánto dinero mensual se debe empezar a abonar hoy si se desea reunir \$ 28700 al final de 5 años y los ahorros rentan el 16%?
Ejercicio.--  Se decide ahorrar mensualmente \$ 900 los cuales depositará al principio de cada mes en una entidad financiera que paga un interés del 30%. ¿Cuánto habrá acumulado al cabo de 2 años?

Representación de tasas de interés usando cashflow
En la modelación financiera es común tener que representar tasas de interés que cambian en el tiempo. La librería cashflow permite realizar esta tarea. En la tabla que se presenta a continuación, la columna n  indica para que periodos se aplica el valor correspondiente de la tasa (columna rate). Por definición, la tasa para el periodo 0 siempre es 0.
Can you write Python code for it?

cf.nominal_rate(const_value=10, start=(2000, 0), nper=8, pyr=4)

cf.nominal_rate(const_value=10, start=(2000, 0), nper=8, pyr=6)

spec = ((2000, 3), 10)
cf.nominal_rate(const_value=1, start=(2000, 0), nper=8, pyr=4, spec=spec)

spec = [(3, 10), (6, 20)]
cf.nominal_rate(const_value=1, start=(2000, 0), nper=8, pyr=4, spec=spec)

cf.nominal_rate(const_value=[10, 20]*10, pyr=4)


I am working on the file "2016-03/IE-02-representacion-flujos-y-tasas.ipynb".
The first task is:
Representación de flujos genéricos de caja
cashflow también permite la representación de flujos de efectivo en forma similar (pero no igual) a las tasas de interés, pero en este caso las tuplas (time, value) representan valores puntuales en el tiempo.
Can you write Python code for it?

cf.cashflow(const_value=1,   # valor constante
            start=(2000, 0), # (periodo mayor, periodo menor)
            nper=8,          # número total de periodos
            pyr=4)           # número de periodos por año

## un valor puntual puede ser introducido mediante una tupla
spec = ((2000, 3), 10)  # ((major, minor), value)
cf.cashflow(const_value=1, start=(2000, 0), nper=8, pyr=4, spec=spec)

cf.cashflow(const_value=1, start=(2000, 0), nper=8, pyr=4, spec=((2000, 3), 10))

spec = [((2000, 3), 10), ((2001, 3), 10)]
cf.cashflow(const_value=1, start=(2000, 0), nper=8, pyr=4, spec=spec)

spec = (3, 10)
cf.cashflow(const_value=1, start=(2000, 0), nper=8, pyr=4, spec=spec)

cf.cashflow(const_value=1, start=(2000, 0), nper=8, pyr=4, spec=(3, 10))

spec = [(3, 10), (7, 10)]
cf.cashflow(const_value=1, start=(2000, 0), nper=8, pyr=4, spec=spec)

cf.cashflow(const_value=[10]*10, pyr=4)

cf.cashflow(const_value=[-10]*4)

## un flujo de caja es un objeto que puede guardarse
## en una variable para usarse después
x = cf.cashflow(const_value=[0, 1, 2, 3], pyr=4)
x[3] = 10
x

## es posible alterar y acceder a valores individuales
## para cada periodo de tiempo usando []
x[3]

x[(0, 3)] = 0

x

x[(0,2)]

abs(cf.cashflow(const_value=[-10]*4, pyr=4))

cf.cashflow(const_value=[1]*4, pyr=4) + cf.cashflow(const_value=[2]*4, pyr=4)

cf.cashflow(const_value=[6]*4, pyr=4) // cf.cashflow(const_value=[4]*4, pyr=4)

x = cf.cashflow( const_value=[2]*4, pyr=4)
x += cf.cashflow( const_value=[3]*4, pyr=4)
x

x = cf.cashflow( const_value=[6]*4, pyr=4)
x //= cf.cashflow( const_value=[4]*4, pyr=4)
x

x = cf.cashflow( const_value=[2]*4, pyr=4)
x *= cf.cashflow( const_value=[3]*4, pyr=4)
x

x = cf.cashflow( const_value=[6]*4, pyr=4)
x -= cf.cashflow( const_value=[4]*4, pyr=4)
x

cf.cashflow( const_value=[2]*4, pyr=4) * cf.cashflow( const_value=[3]*4, pyr=4)

cf.cashflow( const_value=[6]*4, pyr=4) - cf.cashflow( const_value=[4]*4, pyr=4)

cf.cashflow( const_value=[6]*4, pyr=4).tolist()

cflo =...


I am working on the file "2016-03/IE-03-calculos-sobre-flujos.ipynb".
The first task is:
Devaluación
Contenido
<img src="images/devaluacion.png" width=600>
$$d=\frac{f-f_e}{1+f_e},~~~~1+d=\frac{1+f}{1+f_e}$$
Cálculos con moneda extranjera
Contenido

$i$ -- tasa equivalente de interés en moneda local

$i_e$ -- tasa de interés en moneda extranjera

$d$ -- tasa de devaluación de la moneda local frente a la moneda extranjera.

$f$ -- tasa de inflación interna

$f_e$ -- tasa de inflación externa

Interés equivalente para un préstamo en moneda extranjera:
$$1+i=(1+i_e)(1+d)$$
Ejemplo.-- ¿Cuál es la rentabilidad en pesos de un proyecto extranjero que ofrece una rentabilidad del 1.5% mensual?, se estima una devaluación mensual de 2.0%
Can you write Python code for it?


I am working on the file "2016-03/IE-03-calculos-sobre-flujos.ipynb".
The first task is:
Ejemplo.-- Considere un flujo de caja de \$ 100, y una duración de 10 periodos. Calcule el impuesto de renta si la tasa es del 30% para los periodos 1 a 5 y del 35% para los periodos restantes.
Can you write Python code for it?

cflo = cf.cashflow(const_value=[100]*10)
tax_rate = cf.nominal_rate(const_value=[30] * 10,spec=(5,35))
x=cf.after_tax_cashflow(cflo,           # flujo de efectivo
                    tax_rate = tax_rate) # impuesto de renta
cf.cfloplot(x)


I am working on the file "2016-03/IE-03-calculos-sobre-flujos.ipynb".
The first task is:
Ejemplo.-- Considere un flujo de efectivo constante de \$ 1000 durante 10 periodos. Conviértalo a moneda extranjera si la tasa de cambio es de \$ 2.
Can you write Python code for it?

x = cf.cashflow([1000]*10)
cf.currency_conversion(cflo=x, exchange_rate = 2)


I am working on the file "2016-03/IE-03-calculos-sobre-flujos.ipynb".
The first task is:
Impuestos corporativos
Contenido
Tipos de Impuestos.

Impuestos sobre activos tangibles o intangibles.

Lincencias para establecimiento de negocios.

Impuestos al consumo sobre productos básicos.

Impuestos a las ventas.

Impuestos a la renta.

Deducciones.
El impuesto de renta es calculado como el ingreso bruto menos las deducciones permitidas:

Salarios.

Rentas.

Intereses pagados.

Publicidad.

Planes de pensión.

Gastos de investigación.

Depreciación, amortización y agotamiento.

Ejemplo.-- Sea un flujo constante de \$ 1000 para los periodos 1 a 5 y -90 para los periodos 6 a 10. Calcule el impuesto de renta para una tasa impositiva del 30%.
Can you write Python code for it?

cflo=cf.cashflow(const_value=1000,  nper=10, spec=[(t,-90) for t in range(5,10)])
cf.cfloplot(cflo)

tax_rate = cf.nominal_rate(const_value=[30] * 10)
x=cf.after_tax_cashflow(cflo,           # flujo de efectivo
                    tax_rate = tax_rate) # impuesto de renta
cf.cfloplot(x)


I am working on the file "2016-03/IE-03-calculos-sobre-flujos.ipynb".
The first task is:
Inflación
Contenido
Se define como la pérdida de poder adquisitivo de la moneda. Existen diferentes medidas:

Indice de precios del consumidos (IPC)

Indice de precios del productor (IPP)

Medidas derivadas a partir del producto interno bruto

Tasa promedio de inflación: tasa porcentual anual que representa el incremento de los precios sobre un periodo de un año respecto al año anterior.
Tratamiento de la inflación
Contenido

Precios constantes.

Precios corrientes.

Tasa de interés del mercado $(i)$. Incluye una componente de utilidad económica y una de inflación.

Tasa de interés libre de inflación $(i')$. Representa únicamente la componente de utilidad económica

Tasa de inflación general $(f)$. Incremento promedio anual en los precios de los bienes y servicios

Relaciones de equivalencia
Contenido

$F_n'$: valor en moneda constante (en precios del periodo 0) que ocurren en el flujo de caja al final del periodo $n$.

$F_n$: valor en moneda corriente que ocurren en el flujo de caja al final del periodo $n$.

Conversión entre moneda constante y corriente:
$$F_n=F_n^{'}~(1+f)^n$$
Valor presente:
$$P=F_n^{'}~(1+i^{'})^{-n} = F_n^{'}~(1+i)^{-n}$$
Relación entre tasas:
$$i^{'}=\frac{i-f}{1+f}$$
$$(1+i^{'})(1+f)=1+i$$
Ejemplo.-- Considere un flujo de caja constante de \$ 1000. Conviertalo a dinero del periodo 0 usando una inflación del 5% por periodo.
Can you write Python code for it?

x = cf.cashflow(const_value=[1000]*10)
x

cf.const2curr(cflo=x,inflation=cf.nominal_rate(const_value=[5]*10))


I am working on the file "2016-03/IE-04-inversiones.ipynb".
The first task is:
Caso 2: depreciación en linea recta.
Can you write Python code for it?

## considere un activo depreciable
d = cf.depreciation_sl(costs=cf.cashflow(const_value=0, nper=5, spec=(0, 200)),
                       life=cf.cashflow(const_value=0, nper=5, spec=(0, 5)),
                       noprint=True)
d

# crea el flujo de caja
i = cf.cashflow(const_value=[500]*5)          # ingresos brutos: $ 500 por periodo
g = cf.cashflow(const_value=[250]*5)                         # gastos operativos: $ 250 por periodo
x = i-g                                      # ingreso antes de impuestos
x

x = x - d                     # depreciacion
x                            # ingreso antes de impuestos

x = x - cf.after_tax_cashflow(x, tax_rate = cf.nominal_rate(const_value=[35] * 5)) # impuesto de renta del 35%
x                                              # ingreso despues de impuestos

x + d  # flujo de caja


I am working on the file "2016-03/IE-04-inversiones.ipynb".
The first task is:
Cuentas de Ahorros
Contenido
Ejercicio.-- Construya el balance para 12 meses, si habre una cuenta de ahorro que paga el 2% mensual, y hace 12 depósitos de \$ 100 (puede usar Microsoft Excel).
Ejercicio.-- Para el ejercicio anterior, cuánto es la suma de los intereses aritmética de los intereses recibidos?
Ejercicio.-- Construya el balance para 12 meses, si usted tiene una cuenta de ahorro que paga el 2% mensual con un capital inicial de \$ 1000, y hace retiros mensuales de \$ 100 iniciando hoy (puede usar Microsoft Excel).
Ejercicio.-- Para el ejercicio anterior, cuánto es la suma de los intereses aritmética de los intereses recibidos?
Modelado de cuentas de ahorro usando cashflow
Contenido
cashflows permite el modelado de cuentas de ahorro (e intrumentos financieros similares).
Ejemplo.-- Construya el balance para 12 meses, si habre una cuenta de ahorro que paga el 2% mensual, y hace 13 depósitos de \$ 100 (con el primer depósito se abre la cuenta).
Can you write Python code for it?

cflo = cf.cashflow(const_value=[100] * 13)
cflo

nrate = cf.nominal_rate([2] * 13)
nrate

cf.savings(deposits =  cflo,      # deposito periodico
           initbal  =     0,      # balance inicial
           rate     = nrate,      # tasa de interes mensual
           noprint  = False)      # imprime la tabla

x, y = cf.savings(deposits = cflo,      # deposito periodico
                  initbal  = 0,         # balance inicial
                  rate     = nrate,     # tasa de interes mensual
                  noprint  = True)      # no imprime la tabla

## intereses como lista
x

y # balance final


I am working on the file "2016-03/IE-04-inversiones.ipynb".
The first task is:
Depreciación
Contenido
Permite la deducción de los costos de las inversiones (pérdida de valor de ellas) con el fin de reducir el impuesto de renta.
Depreciación en linea recta
Contenido
La depreciación en todos los años es igual.

$P$ -- valor de compra en libros.

$N$ -- número de años en que ocurre la depreciación (vida útil del activo).

$D_n$ -- depreciación en el año $n$.

$$D_n = \frac{P-F}{N}$$

$F$ -- valor de salvamento al final de la vida útil.

Ejemplo.-- Un activo cuesta  \$ 200 . Si la depreciación ocurre durante 5 años, cuál es su depreciación anual?.
Can you write Python code for it?

#
# los datos del activo se ingresan como una dupla
# conformada asi: (costo, vida util)

costs = cf.cashflow(const_value=0, nper=15, spec=[(1, 200), (7, 200)]) # Costo del
life = cf.cashflow(const_value=0, nper=15, spec=[(1, 5), (7,3)])
#
# 'SL' indica la depreciación en linea recta
#                        (cost, life)
cf.depreciation_sl(costs=costs, life=life,noprint=False)

#
# el resultado se imprime en forma de tabla:
#
#  * Beg. Book Value -- representa el valor total de los activos al inicio del periodo
#  * Cost -- Costo en activos en el periodo
#  * Depre -- depreciación total para el periodo t
#  * Accum. Depre --- depreciación acumulada por periodo
#  * End.Book Value -- valor final de los activos al final del periodo


I am working on the file "2016-03/IE-04-inversiones.ipynb".
The first task is:
Depreciación por suma de los dígitos de los años (o depreciación acelerada)
Contenido
En este método de depreciación se hace mayor al principio y más pequeña al final. Se basa en la progresión desde $N$ hasta 1.
$$S = \sum_{i=1}^N i = \frac{1}{2} N(N + 1)$$
$$D_n = (P - F) * \frac{(N - n + 1)}{S} = 2 * (P - F) * \frac{N-n+1}{N(N+1)}$$
Ejemplo.-- Calcule la depreciación por el método de la suma de los dígitos de los años para el activo del ejemplo anterior.
Can you write Python code for it?

## 'SOYD' -- sum of year's digits
cf.depreciation_soyd(costs=costs, life=life,noprint=False) # (cost, life)
# noprint es para imprimir la tabla cuando se indica False


I am working on the file "2016-03/IE-04-inversiones.ipynb".
The first task is:
Efecto de la depreciación sobre el impuesto de renta y el flujo de caja
Contenido
Caso 1: sin depreciación.
Can you write Python code for it?

# crea el flujo de caja
i = cf.cashflow(const_value=[500]*5)          # ingresos brutos: $ 500 por periodo
g = cf.cashflow(const_value=[250]*5)                         # gastos operativos: $ 250 por periodo
x = i-g                                      # ingreso antes de impuestos
x

x = x - cf.after_tax_cashflow(x, tax_rate = cf.nominal_rate(const_value=[35] * 5)) # impuesto de renta del 35%
x                                              # ingreso despues de impuestos


I am working on the file "2016-03/IE-04-inversiones.ipynb".
The first task is:
Ejemplo.-- Realice el mismo ejemplo anterior, pero considerando que la tasa de interes es del 4% a partir de $t$ = 5.
Can you write Python code for it?

nrate= cf.nominal_rate(const_value=[2] * 13,spec=(5,4))
cflo = cf.cashflow(const_value=[100] * 13)
x = cf.savings(deposits = cflo,       # deposito periodico
               initbal  =    0,      # balance inicial
               rate     =    nrate, # tasa de interes mensual
               noprint = False)


I am working on the file "2016-03/IE-04-inversiones.ipynb".
The first task is:
Ejemplo.-- Se tiene una cuenta con un saldo inicial de \$ 100. Se hacen depósitos  al final de cada trimestre por \$ 50 (se hará el primer depósito en 3 meses). La tasa de interés es del 2% mensual y cambiará a 1% a partir del 13avo mes (incluido). Cuál será el saldo al final de mes 24?.
Can you write Python code for it?

nrate= cf.nominal_rate(const_value=[2] * 25,spec=(13,1))

cflo = cf.cashflow(const_value=0, nper=25, spec=[(t*3,50) for t in range(1,9)] )

x = cf.savings(deposits = cflo,       # deposito periodico
               initbal  = 100,      # balance inicial
               rate     = nrate, # tasa de interes mensual
               noprint = False)


I am working on the file "2016-03/IE-04-inversiones.ipynb".
The first task is:
Fixed loan
Can you write Python code for it?

##
## Este tipo es similar a las anteriores, pero siempre
## se abona la misma cantidad a capital. El pago periodico
## es igual al abono a capital más el interés sobre saldos.
##
nrate = cf.nominal_rate(const_value=10, nper=11)
cf.fixed_ppal_loan(amount=1000,
                   nrate=nrate,
                   grace=0,
                   dispoints=0,
                   orgpoints=0,
                   prepmt=None,
                   balloonpmt=None)

##
## periodo de gracia de 3
##
nrate = cf.nominal_rate(const_value=10, nper=11)
cf.fixed_ppal_loan(amount=1000,
                   nrate=nrate,
                   grace=3,
                   dispoints=0,
                   orgpoints=0,
                   prepmt=None,
                   balloonpmt=None)

##
## este tipo de crédito admite una tasa de interés variable
##
nrate = cf.nominal_rate(const_value=10, nper=11, spec=(6,8)) # cambia en el periodo 6
cf.fixed_ppal_loan(amount=1000,
                   nrate=nrate,
                   grace=0,
                   dispoints=0,
                   orgpoints=0,
                   prepmt=None,
                   balloonpmt=None)

##
## este tipo de crédito admite pagos adicionales programados
## como abono a la deuda
##
nrate = cf.nominal_rate(const_value=10, nper=11)
cf.fixed_ppal_loan(amount=1000,
                   nrate=nrate,
                   grace=0,
                   dispoints=0,
                   orgpoints=0,
                   prepmt=None,
                   balloonpmt=cf.cashflow(const_value=0, nper = 11, spec=(5, 500)))

##
## este tipo de crédito admite una tasa de interés variable
##
nrate = cf.nominal_rate(const_value=10, nper=11,spec=(6,8)) # cambia en el periodo 6
cf.fixed_ppal_loan(amount=1000,
                   nrate=nrate,
                   grace=0,
                   dispoints=0,
                   orgpoints=0,
                   prepmt=None,
                   balloonpmt=cf.cashflow(const_value=0, nper = 11, spec=(5, 500)))

##
## ejemplo anterior con periodo...


I am working on the file "2016-03/IE-04-inversiones.ipynb".
The first task is:
Note que en el resultado anterior el ingreso después de impuestos es mayor que en el caso 1.
Caso 2: depreciación acelerada.
Can you write Python code for it?

## activo depreciable
costs=cf.cashflow(const_value=0, nper=5, spec=(0, 200))
life=cf.cashflow(const_value=0, nper=5, spec=(0, 5))
d =cf.depreciation_soyd(costs=costs, life=life)
d

# crea el flujo de caja
i = cf.cashflow(const_value=[500]*5)          # ingresos brutos: $ 500 por periodo
g = cf.cashflow(const_value=[250]*5)                         # gastos operativos: $ 250 por periodo
x = i-g                                      # ingreso antes de impuestos
x

x = x - d                     # depreciacion
x                             # ingreso antes de impuestos

x = x - cf.after_tax_cashflow(x, tax_rate = cf.nominal_rate(const_value=[35] * 5)) # impuesto de renta del 35%
x                                              # ingreso despues de impuestos

x + d  # flujo de caja


I am working on the file "2016-03/IE-04-inversiones.ipynb".
The first task is:
Préstamos
Bullet Loan
Este es un tipo de préstamo en que se pagan únicamente intereses durante la vida del crédito; con la última cuota se paga el capital.
En el siguiente ejemplo se usa cashflow para modelar un préstamo de \$ 1000 a una tasa de interés del 10% y una duración de 10 periodos de tiempo.
Can you write Python code for it?

nrate = cf.nominal_rate(const_value=10, nper=11)
x=cf.bullet_loan(amount=1000,
                 nrate=nrate,
                 dispoints=0,
                 orgpoints=0,
                 prepmt=None)
x

##
## todos los tipos de créditos tiene funciones
## para obtener las columnas de la tabla de
## amortizacion como un flujo de efectivo
##
x.intpmt

sum(x.intpmt)

x.ppalpmt  ## pagos a capital

x.begppalbal ## balance inicial del periodo

x.endppalbal ## balance final del periodo

##
## se admiten el prepago de la deuda sin penalizacion
## a continuación se modela un prepago de deuda de $ 500 en t = 5
##
nrate = cf.nominal_rate(const_value=10, nper=11)
x=cf.bullet_loan(amount=1000,
                 nrate=nrate,
                 dispoints=0,
                 orgpoints=0,
                 prepmt=cf.cashflow(const_value=0, nper=11,spec=(5,500)))
x

##
## Si el prepago de deuda es superior al monto,
## se ajusta al saldo
##
nrate = cf.nominal_rate(const_value=10, nper=11)
x=cf.bullet_loan(amount=1000,
                 nrate=nrate,
                 dispoints=0,
                 orgpoints=0,
                 prepmt=cf.cashflow(const_value=0, nper=11,spec=(5,5000)))
x

##
## este tipo de crédito admite una tasa de interés variable
##
nrate = cf.nominal_rate(const_value=10, nper=11, spec=(6,8)) # cambia en el periodo 6
x=cf.bullet_loan(amount=1000,
                 nrate=nrate,
                 dispoints=0,
                 orgpoints=0,
                 prepmt=None)
x


I am working on the file "2016-03/IE-05-marr.ipynb".
The first task is:
En la modelación de créditos con cashflow se consideran dos tipos de costos:

Los puntos de descuento (dispoints) como porcentaje sobre el monto de la deuda. Estos son una forma de pago de intereses por anticipado con el fin de bajar la tasa de interés del crédito.

Los puntos de originación (orgpoints) como porcentaje del monto de deuda. Son los costos de constitución del crédito y no son considerados como intereses.

Ya que los intereses de los créditos pueden descontarse como costos financieros, estos disminuyen el pago del impuesto de renta. Por consiguiente, en el análisis de los créditos debe tenerse en cuenta el beneficio por pago de intereses el cual equivale a los impuestos pagados por periodo multiplicados por la tasa del impuesto de renta. Ya que los puntos de descuento son intereses, estos se tienen en cuenta en el cálculo de este beneficio.
Can you write Python code for it?

## flujo de caja para el crédito antes de impuestos
credito1.to_cashflow(tax_rate = 30.0)

## la tasa efectiva pagada por el crédito es
## aquella que hace el valor presente cero para
## el flujo de caja anterior (antes o después de
## impuestos)
credito1.true_rate(tax_rate = 30.0)

## se almacenna los datos para este credito
monto[1]   = 2000
interes[1] = credito1.true_rate(tax_rate = 30.0)

## préstamo 2.
## -------------------------------------------------------
##
credito2 = cf.fixed_rate_loan(amount    = 1000,     # monto
                              nrate     =   20,  # tasa de interés
                              start     = None,
                              grace     =    0,
                              life      =    4,     # número de cuotas
                              dispoints =    0.24)  # costos de originación
credito2

credito2.to_cashflow(tax_rate = 30)

credito2.true_rate(tax_rate = 30)

## se almacenna los datos para este credito
monto[2]   = 1000
interes[2] = credito2.true_rate(tax_rate = 30)

## préstamo 3.
## -------------------------------------------------------
##
nrate = cf.nominal_rate(const_value=7, nper=5)
credito3 = cf.bullet_loan(amount     = 5000,     # monto
                          nrate      =   nrate,  # tasa de interés
                          orgpoints  =    0.01,  # costos de originación
                          dispoints  =    0.20)  # puntos de descuento
credito3

credito3.to_cashflow(tax_rate = 30.0) ### malo

credito3.true_rate(tax_rate = 30.0)

## se almacenan los datos de este crédito
monto[3]   = 5000
interes[3] = credito3.true_rate(tax_rate = 30.0)

## montos
monto

## tasas
interes

## Costo ponderado del capital (WACC)
## -------------------------------------------------------------
## es el promdio ponderado de las tasas por
## el porcentaje de capital correspondiente a cada fuente
##
s    = sum(monto)           # capital total
wacc = sum([x*r/s for x, r in zip(monto, interes)])
wacc


I am working on the file "2016-03/IE-06-analisis.ipynb".
The first task is:
Análisis de sensibilidad
Se realiza variando una de las variables del problema para determinar el efecto en el indicador utilizado.
Ejemplo.-- Se tiene un proyecto con la siguiente información:

Años de operación: 10

Años de construcción: 1

Producción: 1000 unidades por año

Precio de venta: \$ 10 por unidad

Costos operativos: \$ 220 por año

Costos de inversión: \$ 2000 depreciables a 10 años.

MARR = 10%

Impuesto de renta: 30%

Construya la gráfica de sensibilidad del npv a la producción, el precio de venta y la inversión.
Can you write Python code for it?

## se construye una función que recibe la información relevante y retorn el npv

def project(marr,
            produccion,
            precio,
            costo,
            inversion):

    ingre = cf.cashflow(const_value=0,
                        nper=11,
                        spec = [(t, precio * produccion) if t > 0 else (0,0) for t in range(11)])

    opera = cf.cashflow(const_value=0,
                        nper=11,
                        spec = [(t, costo)               if t > 0 else (0,0) for t in range(11)])

    inver = cf.cashflow(const_value=0,
                        nper=11,
                        spec = (0, inversion))

    dep   = cf.depreciation_sl(costs = cf.cashflow(const_value=0, nper=11, spec=(0, inversion)),
                               life  = cf.cashflow(const_value=0, nper=11, spec=(0, 10)),
                               noprint = True)

    antes = ingre - opera - inver - dep

    desp  = cf.after_tax_cashflow(antes,
                                  tax_rate = cf.nominal_rate(const_value=[30] * 11))

    neto  = antes + desp

    npv   = cf.timevalue(cflo  = neto,
                          marr = cf.nominal_rate([marr]*11))

    return npv

project(10, 100, 10, 220, 2000)

## resultados para diferentes valores de la MARR
x=[]
for i in [8, 10, 12]:
    x.append(project( i, 100, 10, 220, 2000))
x

## resultados para diferentes valores de la inversión
[project(10, 100, 10, 220, x) for x in [1600, 1800, 2000, 2200, 2400]]

## resultados para diferentes valores del precio
[project(10, 100, x, 220, 2000) for x in [8, 9, 10, 11, 12]]

import matplotlib.pyplot as plt
%matplotlib inline

precio = [8, 9, 10, 11, 12]
y = [project(0.10, 100, x, 220, 2000) for x in precio]

plt.plot(precio, y)


I am working on the file "2016-03/IE-06-analisis.ipynb".
The first task is:
Criterio de la tasa interna de retorno
Tasa Interna de Retorno (IRR).
$r^*$ -- tasa interna de retorno (irr). Es la tasa que hace el valor presente igual a cero.
$$PV(r^) = \sum_{t=0}^N \frac{F_t}{(1+r^)^t} ~=~0$$
Ejemplo.-- Calcule la irr para el ejemplo anterior.
Can you write Python code for it?


I am working on the file "2016-03/IE-06-analisis.ipynb".
The first task is:
Tasa Interna de Retorno Modificada.
<img src="images/mirr.png" width=650>
$$(1+MIRR)^N = \frac{\sum_{t=0}^N \max(F_t,0) * (1+r_1)^{N-t} }{ \sum_{t=0}^N \min(F_t,0) * (1+r_2)^{-t} }$$
$r_1$ -- tasa de reinversión.
$r_2$ -- tasa de financiamiento.
Can you write Python code for it?

cf.mirr(cflo=cflo, finance_rate=0, reinvest_rate=0)

## la función puede recibir varios flujos de fondos simulataneamente.
cf.irr([cflo, cflo, cflo])


I am working on the file "2016-04-01-hackathon-CodeImpot/reforme_revenu_de_base_nouvelles_simulations_rdb_adultes.ipynb".
The first task is:
Calculs de référence (revenu disponible, IR, RSA, CSG imposable/deductible, impot total)
Can you write Python code for it?

simulation_ref_individu_seul.calculate('revdisp')

simulation_ref_individu_seul.calculate('irpp')

simulation_ref_individu_seul.calculate_add('rsa')

simulation_ref_individu_seul.calculate('csg_imposable_salaire')

simulation_ref_individu_seul.calculate('csg_deductible_salaire')

simulation_ref_individu_seul.calculate('tot_impot')


I am working on the file "2016-04-01-hackathon-CodeImpot/reforme_revenu_de_base_nouvelles_simulations_rdb_adultes.ipynb".
The first task is:
Couples sans enfants
On fait varier le salaire d'un des deux, l'autre est fixé à 0.
Can you write Python code for it?

def make_two_parents_scenario(nombre_enfants = 0, year = None, tax_benefit_system = tax_benefit_system,
                                    axes_variable = 'salaire_de_base',  ax_variable_max = 60000, count = 13):
    enfant = [dict(
        birth = date(2005, 1, 1),
        )]
    enfants = enfant * nombre_enfants
    scenario = tax_benefit_system.new_scenario().init_single_entity(
        axes = [[
            dict(
                count = nb_palier,
                min = min_salaire_de_base,
                max = max_salaire_de_base,
                name = 'salaire_de_base',
                period = year,
                ),
            dict(
                count = nb_palier,
                min = min_salaire_de_base,
                max = max_salaire_de_base,
                name = 'salaire_de_base',
                period = year-1,
                ),
            dict(
                count = nb_palier,
                min = min_salaire_de_base,
                max = max_salaire_de_base,
                name = 'salaire_de_base',
                period = year-2,
                )
            ]],
        period = year,
        parent1 = dict(
            birth = date(1980, 1, 1),
            ),
        parent2 = dict(
            birth = date(1980, 1, 1),
            ),
        enfants = enfants,
        menage = dict(
            loyer = 1000,
            statut_occupation = 4,
            ),
        )
    return scenario

scenario_alone_ref = make_two_parents_scenario(0, 2014)
simulation_alone_ref = scenario_alone_ref.new_simulation()

scenario_alone_reform = make_two_parents_scenario(0, 2014, reform)
simulation_alone_reform = scenario_alone_reform.new_simulation()

revenu_disponible_couple_ref = simulation_alone_ref.calculate_add("revdisp")
print revenu_disponible_couple_ref
revenu_disponible_couple_rdb = simulation_alone_reform.calculate_add("revdisp")
salaire_de_base_couple_ref = simulation_alone_ref.calculate("salaire_de_base")
salaire_de_base_couple =...


I am working on the file "2016-04-01-hackathon-CodeImpot/reforme_revenu_de_base_nouvelles_simulations_rdb_adultes.ipynb".
The first task is:
Graphiques: scenario variant selon salaire_de_base (0 => 60k par palier de 5k)
Can you write Python code for it?

min_salaire_de_base = 0
max_salaire_de_base = 60000
nb_palier = max_salaire_de_base / 5000

import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline

# REFERENCE
year = 2014
scenario_ref_individu_seul = tax_benefit_system.new_scenario().init_single_entity(
    period = year,
    parent1 = dict(
        birth = date(1980, 1, 1),
        statmarit = u'Célibataire',
    ),
    foyer_fiscal = dict(
        caseT = True,
    ),
    enfants = [],
    axes = [[
            dict(
                count = nb_palier,
                min = min_salaire_de_base,
                max = max_salaire_de_base,
                name = 'salaire_de_base',
                period = year,
                ),
            dict(
                count = nb_palier,
                min = min_salaire_de_base,
                max = max_salaire_de_base,
                name = 'salaire_de_base',
                period = year-1,
                ),
            dict(
                count = nb_palier,
                min = min_salaire_de_base,
                max = max_salaire_de_base,
                name = 'salaire_de_base',
                period = year-2,
                )
            ]],
)
simulation_ref_individu_seul = scenario_ref_individu_seul.new_simulation(debug = True)

# RDB
scenario_rdb_individu_seul = reform.new_scenario().init_single_entity(
    period = year,
    parent1 = dict(
        birth = date(1980, 1, 1),
        statmarit = u'Célibataire',
    ),
    foyer_fiscal = dict(
        caseT = True,
    ),
    enfants = [],
    axes = [[
            dict(
                count = nb_palier,
                min = min_salaire_de_base,
                max = max_salaire_de_base,
                name = 'salaire_de_base',
                period = year,
                ),
            dict(
                count = nb_palier,
                min = min_salaire_de_base,
                max = max_salaire_de_base,
                name = 'salaire_de_base',
                period = year-1,...


I am working on the file "2016-04-01-hackathon-CodeImpot/reforme_revenu_de_base_nouvelles_simulations_rdb_adultes.ipynb".
The first task is:
Réforme : 1. Revenu de base
Can you write Python code for it?

from numpy import logical_not as not_, minimum as min_, maximum as max_, logical_and as and_, logical_or as or_
#- Hausse de la CSG déductible au taux de 25%
#- Mise en place d'un revenu de base adulte égal à RSA socle célibataire - forfait logement célibataire
#- Intégrer le revenu de base au revenu disponible
#- Mise en place d'un crédit d'impot familles monoparentales montant ??? (50€)
#- Supprimer le RSA

#-Visualisation graphique en abscisse salaire brut et en ordonnée variation du revenu disponible
# pour un célibataire sans enfant
# pour un couple sans enfant
# une famille monoparentale

#(- Nouveau calcul de l'IRPP)

#- Hausse de la CSG déductible au taux de 20%
#montant_csg_crds calcul à partir law_node.taux-plein law_node.taux_réduit et law_node.taux
import json
#- Hausse de la CSG déductible au taux de 20%
#montant_csg_crds calcul à partir law_node.taux-plein law_node.taux_réduit et law_node.taux
def modify_legislation_json(reference_legislation_json_copy):
    for value_json in reference_legislation_json_copy['children']['csg']['children']['activite']['children']['deductible']['children']['taux']['values']:
        value_json['value'] = 0.20
    return reference_legislation_json_copy

#- Mise en place d'un revenu de base adulte égal à RSA socle célibataire - forfait logement célibataire

class rdb(ReformeRevenuDeBase.Variable):
    column = FloatCol
    entity_class = Individus
    label = u"Revenu de base"

    def function(self, simulation, period):
        period = period.start.offset('first-of', 'month').period('month')
        age = simulation.calculate('age')
        rmi = simulation.legislation_at(period.start).minim.rmi

        return period, ((age >= 18) * rmi.rmi * ( 1 -rmi.forfait_logement.taux1) + not_(age >= 18) * 0)

#- Intégrer le revenu de base au revenu disponible
class revdisp(ReformeRevenuDeBase.Variable):
    reference = Menages.column_by_name['revdisp']

    def function(self, simulation, period):
        '''
        Revenu disponible -...


I am working on the file "2016-04-01-hackathon-CodeImpot/reforme_revenu_de_base_nouvelles_simulations_rdb_adultes.ipynb".
The first task is:
Système socio-fiscal
Can you write Python code for it?

TaxBenefitSystem = init_country()
tax_benefit_system = TaxBenefitSystem()

from openfisca_core import reforms

ReformeRevenuDeBase = reforms.make_reform(
    key = 'reforme_rdb',
    name = u"Réforme Revenu de base",
    reference = tax_benefit_system,
    )


I am working on the file "2016-04-01-hackathon-CodeImpot/reforme_revenu_de_base_nouvelles_simulations_rdb_adultes.ipynb".
The first task is:
Tests
Can you write Python code for it?

reform = ReformeRevenuDeBase()
reform.modify_legislation_json(modifier_function = modify_legislation_json)


I am working on the file "2016-04-04-ANOVA-as-sumofsquares-decomposition.ipynb".
The first task is:
Anderson's Iris Data revisited
Can you write Python code for it?

irisurl = "https://raw.githubusercontent.com/Bio204-class/bio204-datasets/master/iris.csv"
iris = pd.read_csv(irisurl)

iris.head()

sbn.distplot(iris["Sepal.Length"])
pass

sbn.violinplot(x="Species", y="Sepal.Length", data=iris)
pass

setosa = iris[iris.Species =='setosa']
versicolor = iris[iris.Species=='versicolor']
virginica = iris[iris.Species == 'virginica']

ANOVA_oneway([setosa['Sepal.Length'], versicolor['Sepal.Length'],
            virginica['Sepal.Length']])


I am working on the file "2016-04-04-ANOVA-as-sumofsquares-decomposition.ipynb".
The first task is:
Calculate the sums of squares
Can you write Python code for it?

SSbtw = sum_squares_between(samples)
SSwin = sum_squares_within(samples)
SStot = sum_squares_total(samples)

print("SS between:", SSbtw)
print("SS within:", SSwin)
print("SS total:", SStot)


I am working on the file "2016-04-04-ANOVA-as-sumofsquares-decomposition.ipynb".
The first task is:
Calculate the value of $R^2$ for our ANOVA model:
Can you write Python code for it?


I am working on the file "2016-04-04-ANOVA-as-sumofsquares-decomposition.ipynb".
The first task is:
Draw a figure to illustrate the within group distributions and the total distribution.
Can you write Python code for it?

sbn.set_palette("deep")

bins = np.linspace(-3, 3, 10)
fig, axes = plt.subplots(1, 5, figsize=(20,5), sharex=True, sharey=True)

for i, sample in enumerate(samples):
    axes[i].hist(sample, bins=bins, histtype='stepfilled',
                 linewidth=1.5, label='sample {}'.format(i+1))
    axes[i].legend()

axes[-1].hist(allobs, bins=bins, histtype='stepfilled', linewidth=2, label='all data')
axes[-1].legend()
axes[-1].set_title("All data combined", fontsize=20)

axes[0].set_ylabel("Frequency", fontsize=20)
axes[2].set_xlabel("X", fontsize=20)

fig.tight_layout()
pass


I am working on the file "2016-04-04-ANOVA-as-sumofsquares-decomposition.ipynb".
The first task is:
Generate ANOVA table:
Can you write Python code for it?


I am working on the file "2016-04-04-ANOVA-as-sumofsquares-decomposition.ipynb".
The first task is:
Simulate ANOVA under the null hypothesis of no difference in group means
Can you write Python code for it?

## simulate one way ANOVA under the null hypothesis of no
## difference in group means

groupmeans = [0, 0, 0, 0]
k = len(groupmeans) # number of groups
groupstds = [1] * k  # standard deviations equal across groups
n = 25 # sample size

# generate samples
samples = [stats.norm.rvs(loc=i, scale=j, size = n) for (i,j) in zip(groupmeans,groupstds)]
allobs = np.concatenate(samples)


I am working on the file "2016-04-04-ANOVA-as-sumofsquares-decomposition.ipynb".
The first task is:
Sums of squares functions
Let's start by writing a set of functions for calculating sums of squared deviations from the mean (also called "sums of squared differences"), or "sums-of-squares" for short.
Can you write Python code for it?

def sum_squares_total(groups):
    """Calculate total sum of squares ignoring groups.

    groups should be a sequence of np.arrays representing the samples asssigned
    to their respective groups.
    """
    allobs = np.ravel(groups)  # np.ravel collapses arrays or lists into a single list
    grandmean = np.mean(allobs)
    return np.sum((allobs - grandmean)**2)

def sum_squares_between(groups):
    """Between group sum of squares"""
    ns = np.array([len(g) for g in groups])
    grandmean = np.mean(np.ravel(groups))
    groupmeans = np.array([np.mean(g) for g in groups])
    return np.sum(ns * (groupmeans - grandmean)**2)

def sum_squares_within(groups):
    """Within group sum of squares"""
    groupmeans = np.array([np.mean(g) for g in groups])
    group_sumsquares = []
    for i in range(len(groups)):
        groupi = np.asarray(groups[i])
        groupmeani = groupmeans[i]
        group_sumsquares.append(np.sum((groupi - groupmeani)**2))
    return np.sum(group_sumsquares)

def degrees_freedom(groups):
    """Calculate the """
    N = len(np.ravel(groups))
    k = len(groups)
    return (k-1, N - k, N - 1)

def ANOVA_oneway(groups):
    index = ['BtwGroup', 'WithinGroup', 'Total']
    cols = ['df', 'SumSquares','MS','F','pval']

    df   = degrees_freedom(groups)
    ss   = sum_squares_between(groups), sum_squares_within(groups), sum_squares_total(groups)
    ms   = ss[0]/df[0], ss[1]/df[1], ""
    F    = ms[0]/ms[1], "", ""
    pval = stats.f.sf(F[0], df[0], df[1]), "", ""

    tbl = pd.DataFrame(index=index, columns=cols)
    tbl.index.name = 'Source'
    tbl.df = df
    tbl.SumSquares = ss
    tbl.MS = ms
    tbl.F = F
    tbl.pval = pval

    return tbl

def ANOVA_R2(anovatbl):
    SSwin = anovatbl.SumSquares[1]
    SStot = anovatbl.SumSquares[2]
    return (1.0 - (SSwin/SStot))


I am working on the file "2016-04-04-ANOVA-as-sumofsquares-decomposition.ipynb".
The first task is:
scipy.stats has an f_oneway function for calculating the f statistic and assocated p-value for a one-way ANOVA.  Let's compare oure result above to the implementation in scipy.stats
Can you write Python code for it?

# f_oneway expects the samples to be passed in as individual arguments
# this *samples notation "unpacks" the list of samples, treating each as
# an argument to the function.
# see: https://docs.python.org/3/tutorial/controlflow.html#unpacking-argument-lists

stats.f_oneway(*samples)


I am working on the file "2016-04-11-Sampling-Distribution-Correlation-Coefficient.ipynb".
The first task is:
$X^2$-square test of independence using scipy.stats
Can you write Python code for it?

# Same analysis with scipy.stats fxns
observed_nomargin = pd.crosstab(bumpus.survived, bumpus.sex, margins=False)
Chi2, Pval, Dof, Expected = stats.chi2_contingency(observed_nomargin.values,
                                                   correction=False)

Chi2, Pval, Dof

bumpus.survived.unique()


I am working on the file "2016-04-11-Sampling-Distribution-Correlation-Coefficient.ipynb".
The first task is:
Association Between Categorical Variables
A standard approach for testing the independence/dependence of a pair of categorical variables is to use a $\chi^2$ (Chi-square) test of independence.
The null and alternative hypotheses for the $\chi^2$ test are as follows:

$H_0$: the two categorical variables are independent
$H_A$: the two categorical variables are dependent

Contingency tables
We typically depict the relationship between categorical variables using a "contingency table".
|       | B1                  | B2                  | Total                         |
|-------|---------------------|---------------------|-------------------------------|
| A1    | $c_{11}$            | $c_{12}$            | $c_{11}$+$c_{12}$               |
| A2    | $c_{21}$            | $c_{22}$            | $c_{12}$+$c_{22}$               |
| Total | $c_{11}$+$c_{21}$ | $c_{12}$+$c_{22}$ | $c_{11}$+$c_{12}$+$c_{12}$+$c_{22}$ |
The rows and columns indicate the different categories for variables A and B respectively, and the cells, $c_{ij}$ give the counts of the number of observations for the corresponding combination of A and B.  For example, the cell $c_{11}$ gives the number of observations that that belong to both the category A1 and B1, while $c_{12}$ gives the number that are both A1 and B2, etc.
Can you write Python code for it?

# construct a contingency table for the sex and survival categorical variables
# from the bumpus data set
dataurl = "https://github.com/Bio204-class/bio204-datasets/raw/master/bumpus-data.txt"
bumpus = pd.read_table(dataurl)

observed = pd.crosstab(bumpus.survived, bumpus.sex, margins=True)
observed


I am working on the file "2016-04-11-Sampling-Distribution-Correlation-Coefficient.ipynb".
The first task is:
Confidence intervals in the space of the transformed variables are:
$$
100(1 - \alpha)\%\text{CI}: \operatorname{arctanh}(\rho) \in [\operatorname{arctanh}(r) \pm z_{\alpha/2}SE]
$$
To put this back in terms of untransformed correlations:
$$
100(1 - \alpha)\%\text{CI}: \rho \in [\operatorname{tanh}(\operatorname{arctanh}(r) - z_{\alpha/2}SE), \operatorname{tanh}(\operatorname{arctanh}(r) + z_{\alpha/2}SE)]
$$
Let's write a Python function to calculate the confidence intervals for correlations:
Can you write Python code for it?

def correlationCI(r, n, alpha=0.05):
    mu = np.arctanh(r)
    sigma = 1.0/np.sqrt(n-3)
    z = stats.norm.ppf(alpha/2)
    left = np.tanh(mu) - z*sigma
    right = np.tanh(mu) + z*sigma
    return (left, right)

correlationCI(0, 30)

ssizes = np.arange(10,250,step=10)
cis = []
for i in ssizes:
    cis.append(correlationCI(0, i)[0])

plt.plot(ssizes, cis, '-o')
plt.xlabel("Sample size")
plt.ylabel("Half-width of CI")
plt.title(r"""Half-width of CIs for $\rho=0$
as as function of sample size""")

pass

stats.pearsonr(sample[:,0], sample[:,1])

?stats.pearsonr


I am working on the file "2016-04-11-Sampling-Distribution-Correlation-Coefficient.ipynb".
The first task is:
Expected counts
If the two categorical variables were independent than we would expect that the count in each cell of the contigency table to equal the product of the marginal probalities times the total number of observations.
Can you write Python code for it?

nobs = observed.ix['All','All']

prob_female = observed.ix['All','f']/nobs
prob_male = observed.ix['All', 'm']/nobs

prob_surv = observed.ix['T', 'All']/nobs
prob_died = observed.ix['F', 'All']/nobs

expected_counts = []
for i in (prob_died, prob_surv):
    row = []
    for j in (prob_female, prob_male):
        row.append(i * j * nobs)
    expected_counts.append(row + [np.sum(row)])
expected_counts.append(np.sum(expected_counts,axis=0).tolist())

expected = pd.DataFrame(expected_counts, index=observed.index, columns=observed.columns)

print("Table of Expected Counts")
expected

Z2 = (observed-expected)**2/expected
Z2

chi2 = np.sum(Z2.values)
chi2

df = (len(bumpus.survived.unique()) - 1) * (len(bumpus.sex.unique()) - 1)
print("chi2 = {}, df = {}, pval = {}".format(chi2, df, stats.chi2.sf(chi2, df=df)))


I am working on the file "2016-04-11-Sampling-Distribution-Correlation-Coefficient.ipynb".
The first task is:
Sampling Distribution of Pearson Correlation
As with all the other statistics we've looked at over the course of the semester, the sample correlation coefficient may differ substantially from the underly population correlation coefficient, depending on the vagaries of sampling.
First we'll generate a single sample, drawn from an underlying bivariate normal distribution with uncorrelated variables.
Can you write Python code for it?

# generate bivariate normal data for uncorrelated variables
# See the docs on scipy.stats.multivariate_normal

# bivariate mean
mean = [0,0]

# covariance matrix
cov = np.array([[1,0],
                [0,1]])

sample = stats.multivariate_normal.rvs(mean=mean, cov=cov, size=30)

sbn.jointplot(sample[:,0], sample[:,1])
print("Correlation matrix:\n", np.corrcoef(sample, rowvar=False, ddof=1))


I am working on the file "2016-04-11-Sampling-Distribution-Correlation-Coefficient.ipynb".
The first task is:
Simulate the sampling distribution of correlation coefficient, uncorrelated variables
First we'll simulate the sampling distribution of $r_{xy}$ when the true population correlation $\rho_{xy} = 0$ and the sample size $n = 30$.
Can you write Python code for it?

mean = [0,0]
cov = [[1,0],
       [0,1]]
ssize = 30
nsims = 2500

cors = []
for n in range(nsims):
    sample = stats.multivariate_normal.rvs(mean=mean, cov=cov, size=ssize)
    r = np.corrcoef(sample, rowvar=False, ddof=1)[0,1]
    cors.append(r)
sbn.distplot(cors)
pass


I am working on the file "2016-04-13-association-categorical-variables.ipynb".
The first task is:
Difference between Observed and Expected counts
Since the  margins of the table (the row and column labeled "All") are the same for both the observed and expected counts, let's work with just the inner cells of the table.
Can you write Python code for it?

Observed = observed.ix[:-1, :-1]  # take all but the last row and colum
Expected = expected.ix[:-1, :-1]  # as above

# This code cell just shows how you can do some fancy HTML output
# from Jupyter code cells.  Here I wanted to output both observed and expected
# count tables in the same cell with some nicely formatted headers.
# This is purely for presentation, we're not doing an analysis here.

from IPython.display import display, HTML

def fmt(x):
    return "{:04.2f}".format(x)

Obs_as_html = Observed.to_html(float_format=fmt)
Exp_as_html = Expected.to_html(float_format=fmt)

HTML("
Observed Counts
\n\n" + Obs_as_html + "\n\n" +
     "
Expected Counts
\n\n" + Exp_as_html)


I am working on the file "2016-04-13-association-categorical-variables.ipynb".
The first task is:
For our bumpus example, the degrees of freedom is df=1. We can calculate the corresponding p-value for our sample chi2 value of ~3.13.
Can you write Python code for it?


I am working on the file "2016-04-13-association-categorical-variables.ipynb".
The first task is:
We can quantify the difference between the observed counts and expected counts on a per cell basis using something which is essentially a squared deviation (but for counts rather than values of a random variate) normalized by the expectation.
Can you write Python code for it?

diff_squared = (Observed-Expected)**2/Expected
diff_squared


I am working on the file "2016-04-13-association-categorical-variables.ipynb".
The first task is:
We can think of the $\chi^2$-statistic as the sum of the normalized squared differences between observed and expected counts (where each cell is normalized by the expected counts).
The $\chi^2$-distribution
Under the null hypothesis that the categorical variables are independent the $\chi^2$-statistic has a sampling distribution called the $\chi^2$-distribution.  The $\chi^2$-distribution takes a single parameter, degree of freedom (df), which for the $\chi^2$ test of independence is equal to $(m-1)(n-1)$ where $m$ and $n$ are the number of categories in the two variables.
Can you write Python code for it?

x = np.linspace(0,8,300)

chi2_df1_pdf = stats.chi2.pdf(x, df=1)
chi2_df2_pdf = stats.chi2.pdf(x, df=2)
chi2_df3_pdf = stats.chi2.pdf(x, df=3)
chi2_df4_pdf = stats.chi2.pdf(x, df=4)
chi2_df5_pdf = stats.chi2.pdf(x, df=5)

plt.plot(x, chi2_df1_pdf, label="df = 1")
plt.plot(x, chi2_df2_pdf, label="df = 2")
plt.plot(x, chi2_df3_pdf, label="df = 3")
plt.plot(x, chi2_df4_pdf, label="df = 4")
plt.plot(x, chi2_df5_pdf, label="df = 5")

plt.xlabel('$\chi^2$')
plt.ylabel('Density')
plt.ylim(0,1)  # note this will cut off the curve for df=1
plt.legend()
pass


I am working on the file "2016-04-18-Linear-Regression.ipynb".
The first task is:
Illustration of regression with Iris data
Can you write Python code for it?

url = "http://roybatty.org/iris.csv"
iris = pd.read_csv(url)
iris.columns = iris.columns.str.replace('.',"_")

iris.head()

setosa = iris.query('Species == "setosa"')
setosa.shape

g = sbn.jointplot(setosa.Sepal_Length, setosa.Sepal_Width, space=0.15,)
g.set_axis_labels("Sepal Length", "Sepal Width")
plt.subplots_adjust(top=0.85)  # adjust top of subplots relative to figure so there is room for title
g.fig.suptitle("Bivariate Distribution of\nSepal Length and Sepal Width\nIris setosa specimens", fontsize=14)
pass

rgr = stats.linregress(setosa.Sepal_Length, setosa.Sepal_Width)
print("Regression slope:", rgr.slope)
print("Regression intercept:", rgr.intercept)
print("Regression R:", rgr.rvalue)

sbn.regplot(setosa.Sepal_Length, setosa.Sepal_Width)
plt.xlabel("Sepal Length")
plt.ylabel("Sepal Width")
plt.title("Linear Least-Squares Regression\nminimizes the sum of squared deviation")
pass

sbn.jointplot(setosa.Sepal_Length, setosa.Sepal_Width, kind="reg")
pass


I am working on the file "2016-04-18-Linear-Regression.ipynb".
The first task is:
Linear Regression -- finding the line of "best fit"
What if we wanted to estimate the linear function that relates $Y$ to $X$?
Linear functions are those whose graphs are straight lines. A linear function of a variable $x$ is usually written as:
$$
 \hat{y} = f(x)  = bx + a
$$
where $a$ and $b$ are constants.  In  geometric terms $b$ is the slope of the line and $a$ is the value of the function when $x$ is zero (usually the referred to as the "y-intercept").
There are infinitely many such linear functions of $x$ we could define. Which line should we use if we want to be able to predict $y$?
Regression Terminology

Predictors, explanatory, or independent variable -- the variables upon which we want to make our prediction.

Outcomes, dependent, or response variable -- the variable we are trying to predict/explain in our regression.

The optimality criterion for least-squares regression
Find the linear function, $f(x)$, that minimizes
$$
\sum (y_i - f(x))^2
$$
i.e. find the linear function that minimizes the squared deviations in the $y$ direction.
Can you write Python code for it?

# calculate regression using built-in scipy.stats.linregress
# we'll show the underlying calculations later in the notebook

rgr = stats.linregress(X,Y)
b = rgr.slope
a = rgr.intercept

# plot scatter
plt.scatter(X, Y, color='steelblue', s=30)

# plot regression line
plt.plot(X, b*X + a, color='indianred', alpha=0.5)

# plot lines from regression to actual value
for (x,y) in zip(X, Y):
    plt.vlines(x, y, b*x + a, linestyle='dashed', color='indianred')

plt.xlabel("X")
plt.ylabel("Y")
plt.gcf().set_size_inches(6,6)

plt.title("Linear Least-Squares Regression\nminimizes the sum of squared deviates",fontsize=14)
pass


I am working on the file "2016-04-18-Linear-Regression.ipynb".
The first task is:
Regression as sum-of-squares decomposition
Like ANOVA, regression can be viewed as a decomposition of the sum-of-squares deviances.
$$
ss(y) = ss(\hat{y}) + ss(residuals)
$$
Can you write Python code for it?

ssy = np.sum((Y - np.mean(Y))**2)
ssyhat = np.sum((Yhat - np.mean(Yhat))**2)
ssresiduals = np.sum((residuals - np.mean(residuals))**2)

print("SSTotal:", ssy)
print("SSYhat:", ssyhat)
print("SSResiduals:", ssresiduals)
print("SSYhat + SSresiduals:", ssyhat + ssresiduals)


I am working on the file "2016-04-18-Linear-Regression.ipynb".
The first task is:
Regression confidence intervals
To understand regression confidence intervals, let's simulate the sampling distribution of the slope and intercept.  For this simulation we will hold $X$ fixed, and repeatedly generate samples of $Y$.
Can you write Python code for it?

npts = 25
x = np.linspace(0,5,npts) + stats.norm.rvs(loc=0, scale=2, size=npts)

A = 1
B = 1.5

slopes = []
intercepts = []
yhat = []

nsims = 1000
for i in range(nsims):
    y = B*x + A + stats.norm.rvs(loc=0, scale=2, size=npts)
    b = np.corrcoef(x,y,ddof=1)[0,1] * (np.std(y,ddof=1)/np.std(x,ddof=1))
    a = np.mean(y) - b * np.mean(x)
    yhat.append(b * x + a)
    slopes.append(b)
    intercepts.append(a)

fig, (ax1, ax2) = plt.subplots(1,2, figsize=(10,4))
sbn.distplot(slopes, ax=ax1)
sbn.distplot(intercepts, ax=ax2)
ax1.set_title('Sampling Distribution\nof Regression Slope, b')
ax2.set_title('Sampling Distribution\nof Regression Intercept, a')
pass

for r in yhat:
    plt.plot(x, r, alpha=0.0075, color='steelblue', marker=None, linewidth=1)
plt.plot(x, B*x + A, color='indianred', marker=None, linewidth=3)
pass

b_ci95low = np.percentile(slopes, 5)
b_ci95hi = np.percentile(slopes, 95)

a_ci95low = np.percentile(intercepts, 5)
a_ci95hi = np.percentile(intercepts, 95)

print("95% CI for slope of regression:", (b_ci95low, b_ci95hi))
print("95% CI for intercepts of regression:", (a_ci95low, a_ci95hi))


I am working on the file "2016-04-18-Linear-Regression.ipynb".
The first task is:
Seaborn will automatically draw CIs for you
The seaborn.regplot function will draw the bivariate scatter, the corresponding linear regression, and confidence interval for you.
Can you write Python code for it?


I am working on the file "2016-04-18-Linear-Regression.ipynb".
The first task is:
Simulate a bivariate relationship
Let's simulate two variables, $X$ and $Y$, where $Y$ is a function of $X$ plus some independent noise.
Can you write Python code for it?

npts = 25
X = np.linspace(0, 5, npts) + stats.norm.rvs(loc=0, scale=1, size=npts)

a = 1.0
b = 1.5
Y = b*X + a + stats.norm.rvs(loc=0, scale=2, size=npts)

g = sbn.jointplot(X, Y)
g.set_axis_labels("X", "Y")
pass


I am working on the file "2016-04-18-Linear-Regression.ipynb".
The first task is:
Solution for the least-squares criterion
The slope, $b$, and intercept, $a$, that minimize this quantity are:
\begin{align}
b &= \frac{s_{xy}}{s^2_x} = r_{xy}\frac{s_y}{s_x}\
\
a &= \overline{y} - b\overline{x}
\end{align}
Can you write Python code for it?

# here's a simple function to calculate the least squares regression of Y on X

def lsqr_regression(X, Y):
    covxy = np.cov(X, Y, ddof=1)[0,1]
    varx = np.var(X, ddof=1)
    b = covxy/varx
    a = np.mean(Y) - b * np.mean(X)
    return b, a

def plot_regression_line(X, Y, b, a, axis, **kw):
    minx = min(X)
    maxx = max(X)
    yhatmin = b*minx + a
    yhatmax = b*maxx + a
    axis.plot((minx,maxx), (yhatmin, yhatmax), marker=None, **kw)

b, a = lsqr_regression(X, Y)
fig, ax = plt.subplots(figsize=(6,6))
ax.scatter(X, Y, color='steelblue')
plot_regression_line(X, Y, b, a, ax, color='indianred', alpha=0.9)
ax.set_xlabel("X")
ax.set_ylabel("Y")
ax.set_title("Regression of Y on X",fontsize=14)

print("Estimated slope, b:", b)
print("Estimated intercept, a:", a)
pass


I am working on the file "2016-04-20-MultipleRegression.ipynb".
The first task is:
3D Scatter Plot
Maplotlib 3D plotting functions lives in a module called mpl_toolkits.mplot3d.  Here's how to create a 3D scatter plot.
Can you write Python code for it?

# drawing a 3D scatter plot
from mpl_toolkits.mplot3d import Axes3D

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter3D(data2.x1, data2.x2, data2.y, c='red', s=30)

# this sets the viewpoint in the 3D space from which we are viewing
# the plot
ax.view_init(elev=10., azim=30)

ax.set_xlabel("$X_1$", fontsize=12)
ax.set_ylabel("$X_2$", fontsize=12)
ax.set_zlabel("$Y$", fontsize=12)
fig.tight_layout()

pass


I am working on the file "2016-04-20-MultipleRegression.ipynb".
The first task is:
Fitting a multiple regression using StatsModels
The formula based method for specifying models extends to the multiple regression case, as illustrated below.
Can you write Python code for it?

# read the model as y is a function of x1 and x2
model2 = smf.ols('y ~ x1 + x2', data2).fit()
print(model2.summary())


I am working on the file "2016-04-20-MultipleRegression.ipynb".
The first task is:
For a full list of attributes and functions associated with the RegressionResults object see the StatsModel documentation here.
Multiple regression is conceptually similar to bivariate regression
The idea behind multiple regression is almost exactly the same as bivariate regression, except now we try and fit a linear model for $Y$ using multiple explanatory variables, $X_1, X_2,\ldots, X_m$. That is we're looking for a linear function, $f(X_1, X_2,\ldots,X_m)$ that minimizes:
$$
\sum_{i=1}^n(y_i - f(x_1, x_2,\ldots, x_m))^2
$$
A linear function of more than one variable is written as:
$$
f(X_1, X_2,\ldots,X_m) = a + b_1X_1 + b_2X_2 + \cdots + b_mX_m
$$
The values, $b_1, b_2,\ldots,b_m$ are the regression coefficients.  Geometrically they have the same interpretation as in the bivariate case -- slopes with respect to the corresponding variable.
Solving the multiple regression problem
Mathematically, we simultaneously find the best fitting regression coefficients, $b_1, b_2,\ldots,b_m$, using linear algebra.  However, since we haven't covered linear algebra in this course, I will omit the details.
Lines, planes, and hyperplanes
A linear function of one variable is a line, a linear function of two variables is a plane, and a linear function of more than two variables is generally called a hyperplane.
Example: Modeling $Y$ as a function of $X_1$ and $X_2$
Let's fit a multiple regression to a synthetic data set that involves two explanatory variables, $X_1$ and $X_2$. We'll also take this as an opportunity to introduce 3D plots in matplotlib.
Can you write Python code for it?

# generate a synthetic data set
npts = 30
x1 = np.linspace(1,5,npts) + np.random.normal(0, 2, size=npts)
x2 = np.linspace(5,10,npts) + np.random.normal(0, 2, size=npts)

# "true "model parameters
a = 1
b1, b2 = 2, -1

y = a + b1*x1 + b2*x2 + np.random.normal(0,3, size=npts)

data2 = pd.DataFrame(dict(x1=x1, x2=x2, y=y))


I am working on the file "2016-04-20-MultipleRegression.ipynb".
The first task is:
For example, we can print summary tables using the summary() method:
Can you write Python code for it?


I am working on the file "2016-04-20-MultipleRegression.ipynb".
The first task is:
OLS regression with the StatsModels library
Let's introduce the StatsModels library.  StatsModels implements a wide range of statistical methods that complement the functions defined in the scipy.stats module. First we'll revisit simple bivariate regression using StatsModels as then we'll turn to multiple regression.
Can you write Python code for it?

# import the formula based module for statsmodels
import statsmodels.formula.api as smf


I am working on the file "2016-04-20-MultipleRegression.ipynb".
The first task is:
Review of bivariate regression
Recall the model for bivariate least-squares regression.
When we regress $Y$ and $X$ we're looking for a linear function, $f(X)$, for which the following sum-of-squared deviations is minimized:
$$
\sum_{i=1}^n (y_i - f(x_i))^2
$$
The general form a linear function of one variable is a line,
$$
f(x) = a + bX
$$
where $b$ is the slope of the line and $a$ is the intercept.
In pictures, what we're looking for is the line that minimizes the length of the gray lines in the figure below.
Can you write Python code for it?

# generate example data
np.random.seed(20140420)
npts = 30
x = np.linspace(0,10,npts) + np.random.normal(0,2,size=npts)
a = 1
b = 2
y = b*x + a + np.random.normal(0,4,size=npts)

# fit regression
regr = stats.linregress(x,y)
regr_line = regr.slope*x + regr.intercept

# draw scatter
plt.plot(x,y, 'o')

# draw regression line
plt.plot(x, regr_line, marker=None, color='firebrick', alpha=0.5)

# draw residuals
plt.vlines(x, y, regr_line, color='gray')

plt.title("Regression of Y on X")
plt.xlabel("X")
plt.ylabel("Y")
pass


I am working on the file "2016-04-20-MultipleRegression.ipynb".
The first task is:
The ols function returns an object of type RegressionResultsWrapper.  This is a Python object that bundles together lots of useful information about the model we just fit.
Can you write Python code for it?


I am working on the file "2016-04-20-MultipleRegression.ipynb".
The first task is:
We can get individual properties such as the paramaeters of the model (i.e. slope and intercept), coefficient of determination, the degrees of freedom for the model, etc.
Can you write Python code for it?

fitmodel.params

fitmodel.rsquared

fitmodel.df_model, fitmodel.df_resid


I am working on the file "2016-04-20-MultipleRegression.ipynb".
The first task is:
We first put the x and y data into a DataFrame, in order to facilitate the use of R-style formulas for specifying statistical models (R is another programming environment that is popular for statistical analysis).
Can you write Python code for it?

# put the x and y data with column names x and y
data = pd.DataFrame(dict(x=x, y=y))


I am working on the file "2016-04-25-Parallels-Regression-and-ANOVA.ipynb".
The first task is:
ANOVA like tables for each regression
Can you write Python code for it?

Ftable_ind = bivariate_regression_table(fit_ind)
Ftable_ind

Ftable_dep = bivariate_regression_table(fit_dep)
Ftable_dep


I am working on the file "2016-04-25-Parallels-Regression-and-ANOVA.ipynb".
The first task is:
Compare regression F-statistic and corresponding p-value to that from ANOVA
Can you write Python code for it?

fit_data.fvalue, fit_data.f_pvalue

stats.f_oneway(data.Y[data.group == -1], data.Y[data.group == 1])


I am working on the file "2016-04-25-Parallels-Regression-and-ANOVA.ipynb".
The first task is:
Fit regression model
Can you write Python code for it?

fit_data = smf.ols('Y ~ group', data).fit()


I am working on the file "2016-04-25-Parallels-Regression-and-ANOVA.ipynb".
The first task is:
Fit the regressions with statsmodels
Can you write Python code for it?

fit_ind = smf.ols('y ~ x', data_ind).fit()
fit_dep = smf.ols('y ~ x', data_dep).fit()


I am working on the file "2016-04-25-Parallels-Regression-and-ANOVA.ipynb".
The first task is:
Functions to decompose sums of squares
Can you write Python code for it?

def sum_squares(x):
    x = np.asarray(x)
    return np.sum((x - np.mean(x))**2)

def bivariate_regression_table(fit):
    """ A function to create an ANOVA-like table for a bivariate regression"""
    df_model = fit.df_model
    df_resid = fit.df_resid
    df_total = df_model + df_resid

    SStotal = sum_squares(fit.model.endog)
    SSmodel = sum_squares(fit.predict())
    SSresid = sum_squares(fit.resid)

    MSmodel = SSmodel / df_model
    MSresid = SSresid / df_resid

    Fstat = MSmodel / MSresid

    Pval = stats.f.sf(Fstat, df_model, df_resid)

    Ftable = pd.DataFrame(index=["Model","Residuals","Total"],
                          columns=["df", "SS", "MS", "F", "Pval"],
                          data = dict(df = [df_model, df_resid, df_total],
                                      SS = [SSmodel, SSresid, SStotal],
                                      MS = [MSmodel, MSresid, ""],
                                      F = [Fstat, "", ""],
                                      Pval = [Pval, "", ""]))
    return Ftable


I am working on the file "2016-04-25-Parallels-Regression-and-ANOVA.ipynb".
The first task is:
If using statsmodels you don't need to explicit create the effect coding variables like we did above. You can specify that a variable is a categorical variables in the formula itself.
Can you write Python code for it?

iris_fit2 = smf.ols('SepalLength ~ C(Species)', iris).fit()


I am working on the file "2016-04-25-Parallels-Regression-and-ANOVA.ipynb".
The first task is:
Plot the regressions using info return by statsmodels
Can you write Python code for it?

fig, (ax1, ax2) = plt.subplots(1,2, figsize=(10,4), sharex=True, sharey=True)

ax1.scatter(x, yind, s=60, alpha=0.75, color='steelblue')
ax1.plot(x, fit_ind.predict(), color='firebrick', alpha=0.5)

ax2.scatter(x, ydep, s=60, alpha=0.75, color='steelblue')
ax2.plot(x, fit_dep.predict(), color='firebrick', alpha=0.5)

ax1.set_xlabel("X",fontsize=15)
ax1.set_ylabel("Y",fontsize=15)
ax1.set_title("Y independent of X", fontsize=18)

ax2.set_xlabel("X",fontsize=15)
ax2.set_title("Y dependent on X", fontsize=18)

pass


I am working on the file "2016-04-25-Parallels-Regression-and-ANOVA.ipynb".
The first task is:
Regression as sum-of-squares decomposition
Example data sets
We setup two synthetic data sets -- one where $Y$ is independent of $X$, and a second where $Y$ is depedendent on $X$.
Can you write Python code for it?

n = 25
x = np.linspace(-5, 5, n) + stats.norm.rvs(loc=0, scale=1, size=n)

a, b = 1, 0.75

# I've chosen values to make yind and ydep have about the same variance
yind = a + stats.norm.rvs(loc=0, scale=np.sqrt(8), size=n)
ydep = a + b*x + stats.norm.rvs(loc=0, scale=1, size=n)

# create two different data frames for ease of use with statsmodels
data_ind = pd.DataFrame(dict(x = x, y = yind))
data_dep = pd.DataFrame(dict(x = x, y = ydep))


I am working on the file "2016-04-25-Parallels-Regression-and-ANOVA.ipynb".
The first task is:
Two-group one-way ANOVA as a bivariate regression
To setup ANOVA for two-groups as a regression problem, we use "dummy coding" where we incorporate group information into a predictor variable.
Can you write Python code for it?

Y1 = stats.norm.rvs(loc=0, scale=1, size=10)
Y2 = stats.norm.rvs(loc=1, scale=1, size=10)
Y = np.concatenate([Y1,Y2])
groups = [-1]*10 + [1]*10  # setup dummy variable to represent grouping
data = pd.DataFrame(dict(Y = Y,
                        group = groups))

data.head(3)

data.tail(3)

data.corr()

sbn.stripplot(x="group", y="Y", hue="group", data=data,s=10)
pass


I am working on the file "2016-04-SRON-NumPy.ipynb".
The first task is:
... have a look at y and predict what the next expression will output ...
Can you write Python code for it?


I am working on the file "2016-04-SRON-NumPy.ipynb".
The first task is:
Array functions
Can you write Python code for it?

a

a.sum()

a.sum(axis=1)

a.sum(axis=0)

a = np.random.random(24).reshape(2,3,4)
a

a.shape

a.sum(axis=2)

a.sum()

a *= 7
a, a.sum()

a += 3
a, a.sum()

a.sum(axis=(1,2))

a.sum(axis=(0,1))

a.sum(axis=(0,1)).sum()


I am working on the file "2016-04-SRON-NumPy.ipynb".
The first task is:
Dimensions
Arrays in Python are not truely n-dimensional: they can be a 1d-array of 1d-arrays (of ...)
NumPy arrays are truely n-dimensional.
Can you write Python code for it?

a = np.random.random(15).reshape(3,5)
a

a.size

a.shape


I am working on the file "2016-04-SRON-NumPy.ipynb".
The first task is:
Indexing
Can you write Python code for it?


I am working on the file "2016-04-SRON-NumPy.ipynb".
The first task is:
Linear algebra
Can you write Python code for it?

a = np.arange(15).reshape(3,5)
a

a*a

a.transpose()

a.dot(a.transpose())

a.transpose().dot(a)


I am working on the file "2016-04-SRON-NumPy.ipynb".
The first task is:
Other linear algebra functions: see https://docs.scipy.org/doc/numpy-dev/reference/routines.linalg.html
Solve linear equation
Can you write Python code for it?

# from numpy.org
a = np.array([[1, 2, 3], [3, 4, 6.7], [5, 9.0, 5]])
b =  np.array([3, 2, 1])
np.linalg.solve(a, b)  # solve the equation ax = b


I am working on the file "2016-04-SRON-NumPy.ipynb".
The first task is:
Play
with indexing and slicing...
Predict this:
Can you write Python code for it?


I am working on the file "2016-1/w04/.ipynb_checkpoints/sistemas_lineales-checkpoint.ipynb".
The first task is:
Ejercicio 1
¿Qué pasa si los puntos no se encuentran sobre una parábola?
Ejercicio 2
Tomen las mediciones de una cantidad $y$ a diferentes tiempos $t$: $(t_0,y_0)=(0,3)$, $(t_1,y_1)=(0.25,1)$, $(t_2,y_2)=(0.5,-3)$, $(t_3,y_3)=(0.75,1)$. Estas medidas son parte de una función periódica que se puede escribir como
$y = a\cos(\pi t) + b\cos(2\pi t) + c\cos(3\pi t) + d\cos(4\pi t)$
donde $a$, $b$, $c$, and $d$ son parámetros. Construya un sistema de ecuaciones lineales y encuentre el valor de estos parámetros. Verifique su respuesta haciendo una gráfica.
Mínimos cuadrados
Volvamos por un momento al ejercicio de la parábola. Que pasaría si en realidad tuviéramos 10 mediciones? En ese caso la matriz $A$ sería de 10 por 3 y no podríamos encontrar una inversa. Aún así es interesante el problema de encontrar los parámetros de la parábola a partir de las mediciones. Aunque en este caso tenemos que olvidarnos de que la parábola pase por todos los puntos experimentales porque en general no lo va a hacer.
Para este caso tenemos que definir un criterio para decir que los parámetros son los mejores. Un posible criterio es que la suma de los cuadrados entre la curva teórica y los datos sea mínima. ¿Cómo podemos entonces encontrar una solución para este caso?
Cambiando un poco la notación pensemos que tenemos un vector $d$ de datos, un vector $m$ con los parámetros del modelo que queremos encontrar y una matriz $G$ que resume la información sobre la teoría físca que queremos utilizar para explicar los datos. De esta manera el problema se podría escribir como
$G m = d$
Donde $G$ en general no es invertible. Pero usando el criterio de mínimos cuadrados vamos a tener que el vector $m$ en realidad puede ser estimado por un vector $\hat{m}$ que cumple la siguiente condición
$G^T G \hat{m} = G^{T}d$
donde $T$ indica la transpuesta. Si ahora escribimos $G^{T}G=A$, $\hat{m}=x$ y $G^{T}d=b$ volvemos al problema del principio y podemos encontrar fácilmente a $\hat{m}$
Ejercicio 3
Los datos...
Can you write Python code for it?

data = np.loadtxt("movimiento.dat")
plt.scatter(data[:,0], data[:,1])


I am working on the file "2016-1/w04/sistemas_lineales.ipynb".
The first task is:
Construyendo un sistemas de ecuaciones lineales
Tenemos ahora el ejemplo siguiente. Tenemos tres puntos en el plano (x,y) y queremos encontrar la parábola que pasa por esos tres puntos.
La ecuación de la parábola es $y=ax^2+bx+c$, si tenemos tres puntos $(x_1,y_1)$, $(x_2,y_2)$, $(x_3,y_3)$ podemos definir el siguiente sistema de ecuaciones lineales.
$
\begin{split}
x_1^2a+x_1b+c&=y_1 \
x_2^2a+x_2b+c&=y_2 \
x_3^2a+x_3b+c&=y_3 \
\end{split}
$
Que en notación matricial se ven así
$
\left(
\begin{array}{ccc}
x_1^2 & x_1 & 1 \
x_2^2 & x_2 & 1 \
x_3^2 & x_3 & 1 \
\end{array}
\right)
\left(
\begin{array}{c}
a \b \c \
\end{array}
\right)
=
\left(
\begin{array}{c}
y_1 \
y_2 \
y_3 \
\end{array}
\right)
$
Vamos a resolver este sistema lineal, asumiendo que los tres puntos son: $(x_1,y_1)=(-2,2)$, $(x_2,y_2)=(1,-1)$, $(x_3,y_3)=(4,4)$
Can you write Python code for it?

#primero construimos las matrices A y b
xp = np.array([-2, 1,4])
yp = np.array([ 2,-1,4])
A = np.zeros((3,3))
b = np.zeros(3)
for i in range(3):
    A[i] = xp[i]**2, xp[i], 1  # Store one row at a time
    b[i] = yp[i]
print 'Array A: '
print A
print 'b: ',b

#ahora resolvemos el sistema lineal y graficamos la solucion
sol = np.linalg.solve(A,b)
print 'solution is: ', sol
print 'A dot sol: ', np.dot(A,sol)
plt.plot([-2,1,4], [2,-1,4], 'ro')
x = np.linspace(-3,5,100)
y = sol[0]*x**2 + sol[1]*x + sol[2]
plt.plot(x,y,'b')


I am working on the file "2016-2/ej1/ej1SOL.ipynb".
The first task is:
Ahora sólo queda llamar esta función para cada $m$.
Can you write Python code for it?


I am working on the file "2016-2/ej1/ej1SOL.ipynb".
The first task is:
La siguiente función realiza la integral entre $a$ y $b$ de la función $f$ para un número de pasos $N$ utilizando la regla de Simpson. Los parámetros $m$ y $x$ son los correspondientes a $J_m(x)$.
Can you write Python code for it?

def simpson(f, a, b, N, m, x):
    if N%2 == 1:
        N += 1
    h = (b-a)/N
    result = f(m,x,a) + f(m,x,b)
    for i in range(1, N, 2):
        result += 4*f(m,x,a + i*h)
    for i in range(2, N-1, 2):
        result += 2*f(m,x,a + i*h)
    result *= h/3.0
    return result


I am working on the file "2016-2/ej1/ej1SOL.ipynb".
The first task is:
Note que una integral corresponde a un punto de las gráficas que se quieren obtener. Entonces, la siguiente función realiza el procedimiento varias veces, generando arreglos para el eje $X$ y el eje $Y$ de las gráficas
Can you write Python code for it?

def plot_bessel(m):
    X = np.linspace(0,20,41)
    Y =np.zeros(41)
    for i in list(range(len(X))):
        Y[i]=simpson(bessel_integrand, 0.0, np.pi, 1000, m, X[i])

    plot(X,Y)
    plt.xlabel("$x$", fontsize = "15")
    plt.ylabel("$J_m(x)$", fontsize = "15")
    plt.title("$m = $"+str(m), fontsize="15")
    plt.show()
    plt.close()


I am working on the file "2016-2/ej1/ej1SOL.ipynb".
The first task is:
Podemos verificar esto utilizando las funciones de Bessel de scipy. De nuevo se grafican en su orden para $m=0,1,2$.
Can you write Python code for it?

import scipy.special
X = np.linspace(0,20,41)

plot(X,scipy.special.jn(0,X))
plt.xlabel("$x$", fontsize = "15")
plt.ylabel("$J_m(x)$", fontsize = "15")
plt.title("$m = 0$", fontsize="18")
plt.show()
plt.close()

plot(X,scipy.special.jn(1,X))
plt.xlabel("$x$", fontsize = "15")
plt.ylabel("$J_m(x)$", fontsize = "15")
plt.title("$m = 1$", fontsize="18")
plt.show()
plt.close()

plot(X,scipy.special.jn(2,X))
plt.xlabel("$x$", fontsize = "15")
plt.ylabel("$J_m(x)$", fontsize = "15")
plt.title("$m = 2$", fontsize="18")
plt.show()
plt.close()


I am working on the file "2016-2017.Meetings/fall/01.pca-mnist.ipynb".
The first task is:
How can we visualize the distribution of these points in $\mathbb{R}^{64}$? We need to approximate the relative positions of the points in 1, 2 or 3 dimensions in order to get a sense for how they are distributed. We can do this with principal components analysis (PCA). The overarching idea here is that our points approximate some high dimensional manifold (a manifold with dimensionality less than or equal to 64), and we can approximate this approximation with dimensionality reduction techniques such as PCA. If this is all very confusing to you, don't worry about it! I'm purposely being vague in my description - a more thorough explanation would require a more rigorous treatment of the math behind PCA.
Let's try to approximate the relative positions of our points in $\mathbb{R}^3$, then:
Can you write Python code for it?

from sklearn.decomposition import PCA
both = [X[i] for i in range(len(y)) if y[i] == 0 or y[i] == 1]
labels = [y_ for y_ in y if y_ == 0 or y_ == 1]
pca = PCA(n_components=3)
Xproj3d = pca.fit_transform(both)


I am working on the file "2016-2017.Meetings/fall/01.pca-mnist.ipynb".
The first task is:
The 0s are an amber color and the 1s are turquoise in the above plot. It looks like they form two (three?) distinct clusters! How does the two-dimensional projection look?
Can you write Python code for it?

pca = PCA(n_components=2)
Xproj2d = pca.fit_transform(both)
plt.scatter(Xproj2d.T[0], Xproj2d.T[1], c=colors, s=30, alpha=0.7)


I am working on the file "2016-2017.Meetings/fall/02.hello_machine2.ipynb".
The first task is:
A lot just happened in those three short lines. Let's step through it line by line:
from sklearn.linear_model import LogisticRegression
From the sklearn (scikit-learn) linear_model module, we import a classifier called Logistic Regression. Linear models are models (or predictors) that attempt to seperate vector inputs of different classes using a linear function. Geometrically, this means our model tries to draw a seperating hyperplane between classes, as opposed to a curved (non-linear) manifold. Logistic regression is a classifier, meaning it can only predict categorical labels, but it is not limited to  binary classification, as we'll see later.
clf = LogisticRegression()
LogisticRegression is just a Python object, so we instantiate it and assign it to the variable clf.
clf.fit(both, labels)
fit in sklearn is the name of the method call that trains a model on a dataset $X$ given the correct labels $y$.
Both LogisticRegression and fit have additional parameters for fine tuning the training process, but the above calls demonstrate model training at its simplest.
So now what? We have a classifier that we can pass in an unlabled input vector to, and have it predict whether that input represents a one or a zero - but in doing so we have run into a big problem.
A Really Big Problem
A natural question to ask about our predictor is "how accurate is it?". We could pass in each $x \in X$ to our predictor, have it predict the label, and compare its prediction to the answer in labels. But this would give us a false sense of confidence in how accurate our predictor actually is. Because we trained our predictor on $X$, we have effectively already given it the answer key to the test. This is not a good way to test how well our predictor can predict never before seen data.
To get around this problem, we split our dataset into a training set and a test set before training our model. This way we can train the model on the training set, then test how well it extrapolates to never before seen data on the test set. This is...
Can you write Python code for it?

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(both, labels, test_size=0.3)

clf = LogisticRegression()
clf.fit(X_train, y_train)
clf.score(X_test, y_test)


I am working on the file "2016-2017.Meetings/fall/02.hello_machine2.ipynb".
The first task is:
Here's a 2D projection of the entire digits dataset using PCA, yikes! By the way, PCA is a linear dimensionality reduction technique, so it gives us a rough idea of what a linear classifier like logistic regression has to deal with. There also exist non-linear dimensionality reduction techniques, which let you project on non-linear manifolds like spheres, instead of linear manifolds like hyperplanes.
Can you write Python code for it?

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
clf = LogisticRegression()
clf.fit(X_train, y_train)
clf.score(X_test, y_test)


I am working on the file "2016-2017.Meetings/fall/02.hello_machine2.ipynb".
The first task is:
Not so easy now, is it? But is 94.8% accuracy good "enough"? Depends on your application.
Can you write Python code for it?


I am working on the file "2016-2017.Meetings/fall/02.hello_machine2.ipynb".
The first task is:
Supervised learning is an area of study within machine learning that entails passing an input vector into a model and outputting a label. Supervised learning is further broken down into classification tasks, in which the label $y$ is taken from some finite set of objects like {red, green blue} or {0, 1, 2, ..., 9} and regression tasks, in which the label $y$ is taken from an infinite set, usually the set of real numbers $\mathbb{R}$. We do this by training our model on $X$, given the correct labels $y$. When we train our model, our model is learning a function that maps from input vectors $x$ to output labels $y$ - hence the name machine learning.
Let's train a binary classifier that is able to correctly predict the label of the vectors in our two-label dataset both, using the class labels in labels. A binary classifier is to be contrasted with a multiclass classifier, which predicts a label within a set of two or more classes.
Can you write Python code for it?

from sklearn.linear_model import LogisticRegression
clf = LogisticRegression() # clf is code speak for 'classifier'
clf.fit(X=both, y=labels)


I am working on the file "2016-2017.Meetings/fall/02.hello_machine2.ipynb".
The first task is:
clf.predict_proba tells us how confident our predictor is for each label that that is the correct label for the input. The above table, along with the score, tells us that this was a very easy classification task for our predictor.
How effective do you think logistic regression will be on the entire digits dataset?
Can you write Python code for it?

from sklearn.decomposition import PCA
pca = PCA(2)
Xproj = pca.fit_transform(X)

plt.scatter(Xproj.T[0], Xproj.T[1], c=y, alpha=0.5)


I am working on the file "2016-2017.Meetings/fall/05.DIY_naive_bayes/naive_bayes_classifier_solution/naive_bayes_tweet_classifier.ipynb".
The first task is:
Analyzing Outcomes
We see that we predicted "hillary" correctly 755 times, and predicted "trump" correctly 648 times. However, we made a fair amount of errors as well; confusing "trump" for "hillary" 142 times and vice versa 44 times.
Let's calculate our accuracy
Can you write Python code for it?

accuracy = (outcomes[("hillary", "hillary")] + outcomes[("trump", "trump")]) / sum(outcomes.values())

print("accuracy: {}".format(accuracy))


I am working on the file "2016-2017.Meetings/fall/05.DIY_naive_bayes/naive_bayes_classifier_solution/naive_bayes_tweet_classifier.ipynb".
The first task is:
Applying the Classifier
In this problem, the posterior probability of a tweet, represented as a set of words $(x_1, \dots, x_n)$, being written by Hillary is defined with Bayes' rule as: $$\mathbb{P}(H|x_1, \dots, x_n) \approx \frac{\mathbb{P}(H)\prod\limits_{i=1}^n \mathbb{P}(x_i | H)}{\mathbb{P}(H)\prod\limits_{i=1}^n \mathbb{P}(x_i | H) + \mathbb{P}(T)\prod\limits_{i=1}^n \mathbb{P}(x_i | T)}$$
We now have all the tools we need to calculate this posterior probability of a tweet being written by Hillary Clinton or Donald Trump. We'll walk through what everything represents step by step.
$\mathbb{P}(H)$ is the prior probability of a tweet being written by Hillary, which is calculated with $\frac{# of Hillary tweets}{# of total tweets}$. This information is in the total_tweet_count dictionary. Similarly, we can calculate $\mathbb{P}(T)$, the prior probability of a tweet being written by Trump, which is $\frac{# of Trump tweets}{# of total tweets}$.
$\prod\limits_{i=1}^n \mathbb{P}(x_i | H)$ is the product of the conditional probabilities for each word $\mathbb{P}(x_i | H)$. We've calculated $\mathbb{P}(x_i | H)$ for each word in the dictionary word_probabilities_per_candidate, so we can simply multiply them together to get their product. We take a similar approach with $\prod\limits_{i=1}^n \mathbb{P}(x_i | T)$.
Log-Probabilities
Since we're multiplying many small values together, we run the risk of floating-point underflow. We solve this, in short, by turning $a \times b$ into $log(a) + log(b)$. Since converting the probabilities to log values does nothing to their ordering (if $\mathbb{P}(A) > \mathbb{P}(B)$, then $log(\mathbb{P}(A)) > log(\mathbb{P}(B)$)). In this way, we avoid arithmetic underflow while keeping the well-ordering of the probabilities, which is all we need for classification.
Can you write Python code for it?

# create a new counter to keep track of how many we classified correctly
# and incorrectly. tuples are arranged (our guess, correct label)
outcomes = {("trump", "trump"): 0.0,
            ("trump", "hillary"): 0.0,
            ("hillary", "hillary"): 0.0,
            ("hillary", "trump"): 0.0
           }

# calculate our priors and take the log
prior_trump = total_tweet_count['trump'] / sum(total_tweet_count.values())
prior_hillary = total_tweet_count['hillary'] / sum(total_tweet_count.values())
log_prior_trump = log(prior_trump)
log_prior_hillary = log(prior_hillary)

# open the test data and test labels and refer to them as "test_data" and "test_labels", respectively
with open(TEST_DATA_PATH, 'r') as test_data, open(TEST_LABELS_PATH, 'r') as test_labels:
    # zip `test_data` and `test_labels` together to create a list of tuples of (tweet,label),
    # which we then iterate over. the `tweet` variable refers to an individual tweet
    # and the `label` variable refers to an individual label
    for test_tweet, test_label in zip(test_data, test_labels):
        # turn the test tweet into tokens
        test_tokens = tokenize(test_tweet)
        log_p_tweet_given_trump = 0.0
        log_p_tweet_given_hillary = 0.0
        for token in test_tokens:
            if token in vocabulary:
                p_word_given_trump = word_probabilities_per_candidate['trump'][token]
                log_p_word_given_trump = log(p_word_given_trump)
                # remember adding two logs is like multiplying their raw values
                log_p_tweet_given_trump += log_p_word_given_trump

                p_word_given_hillary = word_probabilities_per_candidate['hillary'][token]
                log_p_word_given_hillary = log(p_word_given_hillary)
                log_p_tweet_given_hillary += log_p_word_given_hillary
            else:
                # note that the token isn't in word_probabilities (and thus
                # isn't seen in our train set), then we just ignore it
                # this works fine for this...


I am working on the file "2016-2017.Meetings/fall/05.DIY_naive_bayes/naive_bayes_classifier_solution/naive_bayes_tweet_classifier.ipynb".
The first task is:
From the output of total_tweet_count, we know that there are 2404 "hillary" tweets in our train set and 2362 "trump" tweets in our train set.
Can you write Python code for it?

first_10_vocabulary = {k: vocabulary[k] for k in vocabulary.keys()[:10]}
first_10_vocabulary


I am working on the file "2016-2017.Meetings/fall/05.DIY_naive_bayes/naive_bayes_classifier_solution/naive_bayes_tweet_classifier.ipynb".
The first task is:
Inspecting the first 10 items of the inner dictionary of word_counts_per_candidate referring to "hillary" (word_counts_per_candidate[hillary]) shows that "electricity" was used 1 time across all tweets, "elegible" 2 times, "foul" 1 time, "four" 11 times, etc.
Can you write Python code for it?

first_10_word_counts_per_candidate_T = {k: word_counts_per_candidate["trump"][k] for k in word_counts_per_candidate["trump"].keys()[:10]}
first_10_word_counts_per_candidate_T


I am working on the file "2016-2017.Meetings/fall/05.DIY_naive_bayes/naive_bayes_classifier_solution/naive_bayes_tweet_classifier.ipynb".
The first task is:
Inspecting the first 10 items of the vocabulary dictionary shows that "electricity" was used 3 times across all tweets, "elegible" and "foul" 3 times as well, "four" 27 times, etc.
Can you write Python code for it?

first_10_word_counts_per_candidate_H = {k: word_counts_per_candidate["hillary"][k] for k in word_counts_per_candidate["hillary"].keys()[:10]}
first_10_word_counts_per_candidate_H


I am working on the file "2016-2017.Meetings/fall/05.DIY_naive_bayes/naive_bayes_classifier_solution/naive_bayes_tweet_classifier.ipynb".
The first task is:
Let's take a look at our output / a bit of our output for the long dictionaries
Can you write Python code for it?