I am working on the file "09 - Ordinary Differential Equation (ODE) Examples.ipynb".
The first task is:
However, for reasons that will become clear for higher order ODEs, it makes sense to use a change of variables:
$ \frac{dy}{dx} = Sin(x) -\theta $
where:
$ \theta = y $
For odeint, the variable $z$ can be used to represent the variable of integration:
$z = [\theta]$
Can you write Python code for it?

# Define the ODE as function, use odeint to determine numerical solution
def dy_dx(z, x):
    theta = z
    return np.sin(x) - theta

# Use the ODE function odeint
Y_calc = odeint(dy_dx, 0, X)
Y_calc = np.array(Y_calc).flatten()

# Plot
plt.plot(X, Y_hand, label="y by hand")
plt.plot(X, Y_calc, linestyle="dashed", color='red',linewidth=3, \
         label="Python odeint")
plt.legend(loc='best')
plt.show()


I am working on the file "09 - Ordinary Differential Equation (ODE) Examples.ipynb".
The first task is:
Numerical Derivatives:
By hand:
$\frac{d}{dx} Sin(x) = Cos(x)$
We can try at two different points:
$\frac{d}{dx} Sin(\pi)  = -1$
$\frac{d}{dx} Sin(2\pi) = 1$
Can you write Python code for it?

result = derivative(np.sin,np.pi,dx=1e-6) #Specifying the spacing helps get a more accurate answer
print("The derivative of sin(x) at x = pi is {0}".format(result))

result = derivative(np.sin,2*np.pi,dx=1e-6)
print("The derivative of sin(x) at x = 2pi is {0}".format(result))


I am working on the file "09 - Ordinary Differential Equation (ODE) Examples.ipynb".
The first task is:
Starting with the ODE:
$ \frac{dy}{dx} + y = Sin(x) $
and
$ y(0) = 0 $
The solution is:
$ y = \frac{1}{2}(Sin(x) - Cos(x) + e^{-x}) $
To see how this solution was determined by hand, please see Appendix I.
Can you write Python code for it?

#Perform the calculation by hand:
X = np.linspace(0,2*np.pi,1000000)

Y_hand = 1/2.*(np.sin(X) - np.cos(X) + np.exp(-X))

plt.plot(X, Y_hand, label="y by hand" )
plt.show()


I am working on the file "09 - Ordinary Differential Equation (ODE) Examples.ipynb".
The first task is:
To leverage SciPy's odeint function, a python function is used to represent the ODE:
$ \frac{dy}{dx} = Sin(x) - y $
Can you write Python code for it?

# Define the ODE as function, use odeint to determine numerical solution
def dy_dx(y, x):
    return np.sin(x) - y


I am working on the file "09 Exceptions.ipynb".
The first task is:
try/finally Statement
The other flavor of the try statement is a specialization that has to do with finalization (a.k.a. termination) actions. If a finally clause is included in a try, Python will always run its block of statements “on the way out” of the try statement, whether an exception occurred while the try block was running or not.
In it's general form, it is:
try:
    statements # Run this action first
finally:
    statements # Always run this code on the way out

with/as Context Managers
Python 2.6 and 3.0 introduced a new exception-related statement—the with, and its optional as clause. This statement is designed to work with context manager objects, which support a new method-based protocol, similar in spirit to the way that iteration tools work with methods of the iteration protocol.
Context Manager Intro
Basic Usage:
with expression [as variable]:
    with-block
Classical Usage
```python
with open(r'C:\misc\data') as myfile:
    for line in myfile:
        print(line)
    # ...more code here...
```
... even using multiple context managers:
python
with open('script1.py') as f1, open('script2.py') as f2:
    for (linenum, (line1, line2)) in enumerate(zip(f1, f2)):
        if line1 != line2:
            print('%s\n%r\n%r' % (linenum, line1, line2))
How it works

The expression is evaluated,resulting in an object known as a context manager that must have __enter__ and __exit__ methods

The context manager’s __enter__ method is called. The value it returns is assigned to the variable in the as clause if present, or simply discarded otherwise

The code in the nested with block is executed.

If the with block raises an exception, the __exit__(type,value,traceback) method is called with the exception details. These are the same three values returned by sys.exc_info (Python function). If this method returns a false value, the exception is re-raised; otherwise, the exception is terminated. The exception should normally be reraised so that it is propagated outside the with...
Can you write Python code for it?

class TraceBlock:
    def message(self, arg):
        print('running ' + arg)

    def __enter__(self):
        print('starting with block')
        return self

    def __exit__(self, exc_type, exc_value, exc_tb):
        if exc_type is None:
            print('exited normally\n')
        else:
            print('raise an exception! ' + str(exc_type))
            return False # Propagate

with TraceBlock() as action:
    action.message('test 1')
    print('reached')

with TraceBlock() as action:
    action.message('test 2')
    raise TypeError()
    print('not reached')


I am working on the file "09--sqlite3.ipynb".
The first task is:
An alternative to fetchone() is fetchall(), which will return a list of tuples for all rows, starting at the cursor.

Since the preceding code has already fetched the first two results, calling fetchall() at this point will return all remaining results.
Can you write Python code for it?


I am working on the file "09--sqlite3.ipynb".
The first task is:
Exercise. Compute the GPA of every student, but report the name (rather than GT ID) and GPA.
Cleanup
As one final bit of information, it's good practice to shutdown the cursor and connection, the same way you close files.
Can you write Python code for it?


I am working on the file "09--sqlite3.ipynb".
The first task is:
Exercise. Suppose we wish to maintain a second table, called Takes, which records classes that students have taken and the grades they earn.
In particular, each row of Takes stores a student by his/her GT ID, the course he/she took, and the grade he/she earned. More formally, suppose this table is defined as follows:
Can you write Python code for it?

c.execute ('create table Takes (gtid integer, course text, grade real)')


I am working on the file "09--sqlite3.ipynb".
The first task is:
Exercise. Write a query to select only the names and grades of students who took CSE 6040.
Aggregations
Another common style of query is an aggregation, which is a summary of information across multiple records, rather than the raw records themselves.
For instance, suppose we want to compute the GPA for each unique GT ID from the Takes table. Here is a query that does it:
Can you write Python code for it?

query = '''
    select Students.name, avg (Takes.grade)
        from Takes, Students
        where Students.gtid=Takes.gtid
        group by Takes.gtid
'''

for match in c.execute (query):
    print (match)


I am working on the file "09--sqlite3.ipynb".
The first task is:
Given a table, the most common operation is a query. The simplest kind of query is called a select.
The following example selects all rows (items) from the Students table.
Can you write Python code for it?


I am working on the file "09--sqlite3.ipynb".
The first task is:
Note: This command will fail if the table already exists. If you are trying to carry out these exercises from scratch, you may need to remove any existing example.db first.

To populate the table with items, you can use the command, insert into.
Can you write Python code for it?

c.execute ("insert into Students values (123, 'Vuduc')")
c.execute ("insert into Students values (456, 'Chau')")
c.execute ("insert into Students values (381, 'Bader')")
c.execute ("insert into Students values (991, 'Sokol')")


I am working on the file "09--sqlite3.ipynb".
The first task is:
Write a command to insert the following records into the Takes table.

Vuduc: CSE 6040 - A (4.0), ISYE 6644 - B (3.0), MGMT 8803 - D (1.0)
Sokol: CSE 6040 - A (4.0), ISYE 6740 - A (4.0)
Chau: CSE 6040 - C (2.0), CSE 6740 - C (2.0), MGMT 8803 - B (3.0)
Can you write Python code for it?

# Insert your solution here; use the next cell to test the output
taken_spring2015 = [(991, "CSE 6040", 4.0),
                    (456, "CSE 6040", 4.0),
                    (123, "CSE 6040", 2.0),
                    (123, "ISYE 6644", 3.0),
                    (123, "MGMT 8803", 1.0),
                    (991, "ISYE 6740", 4.0),
                    (456, "CSE 6740", 2.0),
                    (456, "MGMT 8803", 3.0)]

c.executemany ('insert into Takes values (?, ?, ?)', taken_spring2015)

# Displays the results of your code
for row in c.execute ('select * from Takes'):
    print (row)


I am working on the file "09--sqlite3.ipynb".
The first task is:
sqlite maintains databases as files; in this example, the name of that file is example.db.

If the named file does not yet exist, connecting to it in this way will create it.

To issue commands to the database, you also need to create a cursor.
Can you write Python code for it?

# Create a 'cursor' for executing commands
c = conn.cursor ()


I am working on the file "09-Errors-and-Exceptions.ipynb".
The first task is:
Here we see that when the error was raised in the try statement (in this case, a ZeroDivisionError), the error was caught, and the except statement was executed.
One way this is often used is to check user input within a function or another piece of code.
For example, we might wish to have a function that catches zero-division and returns some other value, perhaps a suitably large number like $10^{100}$:
Can you write Python code for it?

def safe_divide(a, b):
    try:
        return a / b
    except:
        return 1E100

safe_divide(1, 2)

safe_divide(2, 0)


I am working on the file "09-Errors-and-Exceptions.ipynb".
The first task is:
Note that the second block here did not get executed: this is because the first block did not return an error.
Let's put a problematic statement in the try block and see what happens:
Can you write Python code for it?

try:
    print("let's try something:")
    x = 1 / 0 # ZeroDivisionError
except:
    print("something bad happened!")


I am working on the file "09-Errors-and-Exceptions.ipynb".
The first task is:
Now the user knows exactly why the input is invalid, and could even use a try...except block to handle it!
Can you write Python code for it?

N = -10
try:
    print("trying this...")
    print(fibonacci(N))
except ValueError:
    print("Bad value: need to do something else")


I am working on the file "09-Errors-and-Exceptions.ipynb".
The first task is:
One potential problem here is that the input value could be negative.
This will not currently cause any error in our function, but we might want to let the user know that a negative N is not supported.
Errors stemming from invalid parameter values, by convention, lead to a ValueError being raised:
Can you write Python code for it?

def fibonacci(N):
    if N < 0:
        raise ValueError("N must be non-negative")
    L = []
    a, b = 0, 1
    while len(L) < N:
        a, b = b, a + b
        L.append(a)
    return L

fibonacci(10)

fibonacci(-10)


I am working on the file "09-Errors-and-Exceptions.ipynb".
The first task is:
Or if you try an operation that's not defined:
Can you write Python code for it?


I am working on the file "09-Errors-and-Exceptions.ipynb".
The first task is:
Or maybe you're trying to access a sequence element that doesn't exist:
Can you write Python code for it?


I am working on the file "09-Errors-and-Exceptions.ipynb".
The first task is:
Or you might be trying to compute a mathematically ill-defined result:
Can you write Python code for it?


I am working on the file "09-Errors-and-Exceptions.ipynb".
The first task is:
There is a subtle problem with this code, though: what happens when another type of exception comes up? For example, this is probably not what we intended:
Can you write Python code for it?


I am working on the file "09-Errors-and-Exceptions.ipynb".
The first task is:
With this pattern, you can further customize the exception handling of your function.
Defining custom exceptions
In addition to built-in exceptions, it is possible to define custom exceptions through class inheritance.
For instance, if you want a special kind of ValueError, you can do this:
Can you write Python code for it?

class MySpecialError(ValueError):
    pass

raise MySpecialError("here's the message")


I am working on the file "09-Errors-and-Exceptions.ipynb".
The first task is:
You might find this useful as you develop more customized code.
try...except...else...finally
In addition to try and except, you can use the else and finally keywords to further tune your code's handling of exceptions.
The basic structure is this:
Can you write Python code for it?

try:
    print("try something here")
except:
    print("this happens only if it fails")
else:
    print("this happens only if it succeeds")
finally:
    print("this happens no matter what")


I am working on the file "09-Machine-Learning-II/HW09/README_orig.ipynb".
The first task is:
Question 1
Write a function called "variable_helper" which takes one argument:

df, which is a pandas data frame

and returns:

d, a dictionary where keys are the column names of df and values are one of "numeric", "categorical", "ordinal", "date/time", or "text", corresponding to the feature type of each column.
Can you write Python code for it?


I am working on the file "09-Machine-Learning-II/HW09/README_orig.ipynb".
The first task is:
Sample output:
In [1]: get_categories(df[['offense_id','beat','UC2 Literal']])
Out[1]: {'UC2 Literal': array(['AGG ASSAULT', 'AUTO THEFT', 'BURGLARY-NONRES',
                'BURGLARY-RESIDENCE', 'HOMICIDE', 'LARCENY-FROM VEHICLE',
                'LARCENY-NON VEHICLE', 'RAPE', 'ROBBERY-COMMERCIAL',
                'ROBBERY-PEDESTRIAN', 'ROBBERY-RESIDENCE'], dtype=object),
         'beat': array([101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113,
                114, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212,
                213, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312,
                313, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412,
                413, 414, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511,
                512, 601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612,
                701, 702, 703, 704, 705, 706, 707, 708, 710])}
Short explanation: UC2 Literal and beat are the only categorical variables in the data frame df[['offense_id','beat','UC2 Literal']].
Question 3
Write a function called "code_shift" which takes one argument:

df, which is a pandas data frame

and returns:

a pandas data frame with columns "offense_id", "Shift", "ShiftID", where ShiftID is 0 if "Shift" is "Unk", 1 if "Morn", 2 if "Day", and 3 if "Eve".
Can you write Python code for it?


I am working on the file "09-Machine-Learning-II/HW09/README_orig.ipynb".
The first task is:
Sample output:
In [1]: rating_confusion(cer[:20])
Out[1]: ratingID     0  1
predicted_ratingID
                0   15  0
                1   3   2
Question 5
Write a function called "prediction_metrics" which takes one argument:

cer, which is a pandas data frame

and returns:

metrics_dict, a python dictionary object where the keys are 'precision', 'recall', 'F1' and the values are the numeric values for precision, recall, and F1 score, where ratingID is the prediction target and predicted_ratingID is a model output.
Can you write Python code for it?


I am working on the file "09-Machine-Learning-II/HW09/README_orig.ipynb".
The first task is:
Sample output:
In [1]: variable_helper(df[['offense_id','beat','x','y']])
Out[1]: {'beat': 'categorical',
         'offense_id': 'ordinal',
         'x': 'numeric',
         'y': 'numeric'}
Short explanation: offense_id is a number assigned to each offense. There is a natural ordering implied in the id number (based on order of occurrence). Because of this, offense_id is an ordinal feature. The beat uses a numeric label, but refers to a geographic location. There is no natural ordering, so beat is a categorical feature. The location variables (x and y) are numeric position coordinates.
Question 2
Write a function called "get_categories" which takes one argument:

df, which is a pandas data frame

and returns:

cat, a dictionary where keys are names of columns of df corresponding to categorical features, and values are arrays of all the unique values that the feature can take.
Can you write Python code for it?


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
Analysing Lightcurve data
We can asses the degree to which the lyra curves correlate with each other:
Can you write Python code for it?

cross_correlation = np.correlate(lyra_lightcurve.data['CHANNEL3'],
                                 lyra_lightcurve.data['CHANNEL4'])
print(cross_correlation)


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
Handling Data over time
There's a widespread trend in solar physics at the moment for correlation over actual science, so being able to handle data over time spans is a skill we all need to have. Python has ample support for this so lets have a look at what we can use.

 Learning Objectives

 Understand and use SunPy lightcurve.

 Create a pandas dataframe.

 Utilise the datetime package

 Use the pandas dataframe to plot the data within it.

SunPy Lightcurve
SunPy provides a lightcurve object to handle this type of time series data. The module has a number of instruments associated with it, including:

GOES XRS LightCurve
SDO EVE LightCurve for level 0CS data
Proba-2 LYRA LightCurve
NOAA Solar Cycle monthly indices.
Nobeyama Radioheliograph Correlation LightCurve.
RHESSI X-ray Summary LightCurve.

We're going to examine the lightcurves created by a solar flare on June 7th 2011.
Lets begin with the import statements:
Can you write Python code for it?

from __future__ import print_function, division
import numpy as np
import sunpy
from sunpy import lightcurve as lc
import matplotlib.pyplot as plt
%matplotlib inline

import warnings
warnings.filterwarnings('ignore')


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
Looking back at when we discussed the first element of data, and the format of the time index was awkward to use so lets do something about that.
Can you write Python code for it?


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
Much easier. Note that the format of table has now changed and are pandas specific datetime objects, and looks like this:
Can you write Python code for it?


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
Note that the date on the x-axis is a little messed up we can fix with fig.autofmt_xdate()

 How do the lifetimes change?

Now that we have the plot for the maximum length, now make a bar graph of the lifetimes of the features.

 Exoplanet Data

Now, to all the astronomers out there, let us process some real data. We have some txt files containing the lightcurve from a recent paper. Can you process the data and show us the planet?

HINT: You'll need to treat this data slightly differently. The date here is in Julian Day so you will need to use [these](http://docs.astropy.org/en/v1.1.1/api/astropy.time.Time.html) docs to convert it to a sensible datetime object, before you make the DataFrame.

Can you write Python code for it?

import numpy as np
ep_data = np.loadtxt('data/XO1_wl_transit_FLUX.txt')

ep_dict = {'flux':ep_data[:, 1],
           'err_flux':ep_data[:, 2]}

ep_df = pd.DataFrame(data=ep_dict, index=ep_data[:,0])
ep_df.index = pd.to_datetime(ep_df.index)

from astropy.time import Time
t = Time(ep_data[:, 0], format='jd')
UTC = t.datetime

ep_df.index = UTC

ep_df


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
Notice that the meta data information is stored in something called OrderedDict

 On Dictionaries

We can create keyword-data pairs to form a dictionary (shock horror) of values. In this case we have defined some strings and number to represent temperatures across europe

temps = {'Brussles': 9, 'London': 3, 'Barcelona': 13, 'Rome': 16}
temps['Rome']
16

We can also find out what keywords are associated with a given dictionary, In this case:

temps.keys()
dict_keys(['London', 'Barcelona', 'Rome', 'Brussles'])

We can use these keys to plot specific parameters from the lightcurve, Aluminium is 'CHANNEL_3' and Zirconium is 'CHANNEL_4. These measurements are taken on a instuments which detect events at different energy levels.
Can you write Python code for it?

plt.figure(1, figsize=(10,5))
plt.plot(lyra_lightcurve.data.index, lyra_lightcurve.data['CHANNEL3'], color='blue', label='Al filter')
plt.plot(lyra_lightcurve.data.index, lyra_lightcurve.data['CHANNEL4'], color='red', label='Zr filter')
plt.ylabel('Flux (Wm^2)')
plt.show()


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
Now lets create some lightcurves
Can you write Python code for it?


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
Now the next logical step would be to make a for loop and iterate over the index and reassign it. HOWEVER there is almost always a better way. And Pandas has a to_dateime() method that we can feed the index:
Can you write Python code for it?


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
Now, the above line imports information on some solar features over a sample time period. Specifically we have, maximum length, lifetime and time at which they occured. Now if we type data[0] what will happen?
Can you write Python code for it?


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
Pandas
In its own words Pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive. Pandas has two forms of structures, 1D series and 2D dataframe. It also has its own functions associated with it.
It is also amazing.
Lightcurve uses these in built Pandas functions, so we can find out things like the maximum of curves:
Can you write Python code for it?

# max time argument taken from long and short GOES channels
max_t_goes_long = goes_lightcurve.data['xrsb'].idxmax()
max_t_goes_short = goes_lightcurve.data['xrsa'].idxmax()

# max_time argument taken from channel 3 & 4 LYRA channe;s
max_t_lyra_al = lyra_lightcurve.data['CHANNEL3'].idxmax()
max_t_lyra_zr = lyra_lightcurve.data['CHANNEL4'].idxmax()

print('GOES long: ', max_t_goes_long)
print('GOES short: ', max_t_goes_short)
print('LYRA Al: ', max_t_lyra_al)
print('LYRA Zr: ', max_t_lyra_zr)


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
Part of the advantage of using these inbuilt functions we can get a quicklook at our data using short commands:
Can you write Python code for it?

fig = goes_lightcurve.peek()
fig = lyra_lightcurve.peek()


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
So lets plot them on the graph
Can you write Python code for it?

# create figure with raw curves
plt.figure(1, figsize=(10,5))
plt.plot(lyra_lightcurve.data.index, lyra_lightcurve.data['CHANNEL3'], color='blue', label='Al filter')
plt.plot(lyra_lightcurve.data.index, lyra_lightcurve.data['CHANNEL4'], color='red', label='Zr filter')
plt.ylabel('Flux (Wm^2)')

# max lines
plt.axvline(max_t_lyra_al,color='blue',linestyle='dashed', linewidth=2)
plt.axvline(max_t_lyra_zr,color='red',linestyle='dashed')
plt.axvline(max_t_goes_long,color='green',linestyle='dashed',linewidth=2)


I am working on the file "09-Time-Series-Data.ipynb".
The first task is:
This is the first row of the array, containing the first element of our three properties. This particular example is a stuctured array, so the columns and rows can have properties and assign properties to the header. We can ask what the title of these columns is by using a dtype command:
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
As with most machine learning approaches, to validate your model you need training and testing partitions. Since we don't have any labels (topic modeling is unsupervised machine learning), we just need to do this for the novel strings:
Can you write Python code for it?

train = novel_list[:120]
test = novel_list[120:]


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Before we fit the model, we need to remember that with a lot of these probabilistic models random number generators are used to star the algorithm. If we want our results to be reproducible, we need to set the random seed of the math library we use, in this case numpy:
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Before we go anywhere, let's randomly shuffle the rows so that we don't have them ordered by dates or anything else:
Can you write Python code for it?

np.random.seed(0)
metadata_tb = Table.from_df(metadata_tb.to_df().sample(frac=1))
metadata_tb.show(5)


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Challenge
Add these topic assignments back to our Table metadata_tb
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Document Term Matrix
Now we need to make a document term matrix, just as we have in the past two classes. We can pull in our CountVectorizer from sklearn again to create our dtm:
Can you write Python code for it?

from sklearn.feature_extraction.text import CountVectorizer


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
I've retrained the model for 13 topics and exported as below (note the max_iter=1000 takes a long time, so I've pickled the model again):

```python
lda = LatentDirichletAllocation(n_components=13, max_iter=1000)
lda_model = lda.fit(dtm)
pickle.dump((lda, lda_model, dtm, cv), open('13-topics.pkl', 'wb'))
```

We can load in the pre-trained model from the pickle:
Can you write Python code for it?

lda, lda_model, dtm, cv = pickle.load(open('scripts/13-topics.pkl', 'rb'))


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Interpreting the Model
There are many strategies that can be used to interpret the output of a topic model. In this case, we will look for any correlations between the topic distributions and metadata.
We'll first grab all the topic distributions similar to what we did above. Remember, the order of the novels is still the same!
Can you write Python code for it?

list_of_doctopics = [doc_topic[n] for n in range(len(doc_topic))]
list_of_doctopics[0]


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Let's check the doc string:
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Let's double check they all came through:
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Let's get a list of documents as strings just like we did with the novels, and then we'll randomly shuffle them in case they're ordered by category already:
Can you write Python code for it?

documents_train = train_subset.data
np.random.shuffle(documents_train)

print(documents_train[0])


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Let's plot these results:
Can you write Python code for it?

plt.plot(try_topic_n, [x for x in scores])
plt.xlabel('number of topics')
plt.ylabel('coherence score')


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Many papers in the social sciences still don't use a quantitative evaluation metric. Many use the library pyLDAvis to simply visualize the topic distributions, looking for the right size and little overlap in topics as markers of a well-chosen number of topics:
Can you write Python code for it?

pyLDAvis.enable_notebook()
pyLDAvis.sklearn.prepare(lda_model, dtm, cv)


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Now let's print the top 10 words of the 20 topics for the model we trained, using our display_topics function. Have a look through the output and see what topics you can spot:
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Now we just fit the model, as we've done with all sklearn models! This may take a while, a lot is going on:
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Now we'll do the same for the test set:
Can you write Python code for it?

test_subset = pickle.load(open('scripts/20-news-test.pkl', 'rb'))
documents_test = test_subset.data
np.random.shuffle(documents_test)
print(documents_test[0])


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Since we're topic modeling, we don't care about what they've been labeled, but it'll be interesting to see how our topics line up with these!
How many documents are there?
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
The corr() method will give us a correlation matrix:
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Then we can just index our topic numbers to get the corresponding number of topics with the highest coherency:
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
To clarify:

`filename`: Name of file on disk

`id`: Unique ID in Piper corpus

`language`: Language of novel

`date`: Initial publication date

`author`: Author's name

`title`: Title of novel

`gender`: Authorial gender

`person`: Textual perspective

`length`: Number of tokens in novel

We see a list of filenames in the table, these map into a folder we have called txtlab_Novel150_English:
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
To get our words back out we'll use the method get_feature_names()
Can you write Python code for it?

dtm_feature_names = cv.get_feature_names()
dtm_feature_names[:10]


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
To get the probabilities for each topic for a given book we can print the whole probability list for a given novel:
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Topic Modeling
Latent Dirichlet Allocation (LDA) Models
LDA reflects an intuition that words in a text are not merely chosen at random but are drawn from underlying concepts (the so-called "latent variables"). The goal of LDA is to look across many texts in order to reverse engineer these concepts by finding words that tend to cluster with one another. For this reason, LDA has been referred to as "the mother of all word collocation techniques."
sklearn has the LatentDirichletAllocation function:
Can you write Python code for it?

from sklearn.decomposition import LatentDirichletAllocation


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Topic Modeling in Python
In Lisa Rhody's article, "Topic Modeling and Figurative Language", she uses LDA topic modeling to look at ekphrasis poetry. She argues that ekphrasis poetry is particulary well-suited to an LDA analysis because of the assumption of a previously existing set of topics. She's able to extract a number of topics, each constituted of a set of words and probabilities. While we don't have Rhody's corpus, we can use this technique on any large text corpus. We'll use a corpus of novels curated by Andrew Piper.

Corpus Description
We'll look at an English-language subset of Andrew Piper's novel corpus, totaling 150 novels by British and American authors spanning the years 1771-1930. These texts are each in a separate plaintext file in our data folder. Metadata is contained in a spreadsheet distributed with the novel files by the txtLAB at McGill.
The metadata provided describes the corpus that exists as .txt files. So let's first read in the metadata:
Can you write Python code for it?

metadata_tb = Table.read_table('data/txtlab_Novel150_English.csv')
metadata_tb.show(5)


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Topics
To print the topics, we can write a function. display_topics will print the most probable words to show up in each topic.
Can you write Python code for it?

def display_topics(model, feature_names, num_top_words):
    for topic_idx, topic in enumerate(model.components_):
        print(topic_idx, " ".join([feature_names[i] for i in topic.argsort()[:-num_top_words - 1:-1]]))


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
We can add these columns to our metadata_tb Table:
Can you write Python code for it?

meta = metadata_tb.to_df()
meta[df.columns] = df
meta.head()


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
We can double check that our feature limit was enforced by calling len on the dtm_feature_names:
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
We can print which topic each novel is closest to by indexing the topic probabilities and using the argmax function:
Can you write Python code for it?

doc_topic = lda.transform(dtm)

for n in range(doc_topic.shape[0]):
    topic_most_pr = doc_topic[n].argmax()
    print(metadata_tb['author'][n], metadata_tb['title'][n])
    print("doc: {} topic: {}\n".format(n,topic_most_pr))


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
We can see the column variables we have in the metadata with the .labels attribute:
Can you write Python code for it?


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
We see some strong correlations of topics with date, recall:
Can you write Python code for it?

display_topics(lda, dtm_feature_names, 10)

meta.plot.scatter(x='date', y=1)

meta.plot.scatter(x='date', y=5)

meta.plot.scatter(x='date', y=6)

meta.plot.scatter(x='date', y=12)


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
We'll make a DataFrame, which is similar to a Table, with the probabilities for the topics (columns) and documents (rows):
Can you write Python code for it?

df = pd.DataFrame(list_of_doctopics)
df.head()


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
While you may not have seen the importance of max_features, max_df and min_df before, for topic modeling this is extremely important, because otherwise your topics will not be super coherent.
Let's start out with this:

max_features = 5000  (i.e. only include 5000 tokens in our dtm)
max_df = .8  (i.e. don't keep any tokens that appear in > 80% of the documents)
min_df = 5  (i.e. only keep the token if it appears in > 5 documents)

We'll add in a stop_words='english' too, which automatically uses its own stopwords list to remove from our dtm:
Can you write Python code for it?

cv = CountVectorizer(max_features=5000, stop_words='english', max_df=0.80, min_df=5)


I am working on the file "09-Topic-Modeling/01-Topic-Modeling.ipynb".
The first task is:
Why do you think we see this?
Homework
We're going to download the 20 Newsgroups, a widely used corpus for demos of general texts:

The 20 Newsgroups data set is a collection of approximately 20,000 newsgroup documents, partitioned (nearly) evenly across 20 different newsgroups. To the best of my knowledge, it was originally collected by Ken Lang, probably for his Newsweeder: Learning to filter netnews paper, though he does not explicitly mention this collection. The 20 newsgroups collection has become a popular data set for experiments in text applications of machine learning techniques, such as text classification and text clustering.

Let's read in the training data:
Can you write Python code for it?

train_subset = pickle.load(open('scripts/20-news-train.pkl', 'rb'))


I am working on the file "09-conversion-monedas.ipynb".
The first task is:
Devaluación
<img src="images/devaluacion.png" width=600>
$$d=\frac{f-f_e}{1+f_e},~~~~1+d=\frac{1+f}{1+f_e}$$
Cálculos con moneda extranjera

$i$ -- tasa equivalente de interés en moneda local.
$i_e$ -- tasa de interés en moneda extranjera.
$d$ -- tasa de devaluación de la moneda local frente a la moneda extranjera.
$f$ -- tasa de inflación interna.
$f_e$ -- tasa de inflación externa.

Interés equivalente para un préstamo en moneda extranjera:
$$1+i=(1+i_e)(1+d)$$
Ejemplo.-- ¿Cuál es la rentabilidad en pesos de un proyecto extranjero que ofrece una rentabilidad del 1.5% mensual?, se estima una devaluación mensual de 2.0%
Can you write Python code for it?


I am working on the file "09-conversion-monedas.ipynb".
The first task is:
Ejemplo.-- Considere el mismo caso anterior con una devaluación por período del 1%.
Can you write Python code for it?

cf.currency_conversion(cflo=cf.cashflow([1000]*10, start=2016),
                       exchange_rate=2,
                       devaluation=cf.interest_rate(const_value=[1]*10, start=2016))


I am working on the file "09-conversion-monedas.ipynb".
The first task is:
Ejemplo.-- Considere el mismo caso anterior pero usando el período 10 como base.
Can you write Python code for it?

cf.currency_conversion(cflo=cf.cashflow([1000]*10, start=2016),
                       exchange_rate=2,
                       devaluation=cf.interest_rate(const_value=[1]*10, start=2016),
                       base_date=9)

cf.currency_conversion(cflo=cf.cashflow([1000]*10, start=2016),
                       exchange_rate=2,
                       devaluation=cf.interest_rate(const_value=[1]*10, start=2016),
                       base_date='2025')


I am working on the file "09-conversion-monedas.ipynb".
The first task is:
currency_conversion(cflo, exchange_rate=1,
  devaluation=None, base_date=0)
Convierte un flujo de efectivo de moneda extranjera a moneda local usando la tasa de cambio especificada.

cflo-- flujo de efectivo en moneda extranjera.
exchange_rate -- tasa de cambio.
devaluation -- devaluación de la moneda local frente a la extranjera expresada como una tasa nominal.
base_date -- período base para realizar los cálculos de devaluación.

Ejemplo.-- Considere un flujo de efectivo constante de $ 1000 durante 10 períodos. Conviértalo a moneda extranjera si la tasa de cambio es de $ 2.
Can you write Python code for it?

cf.currency_conversion(cflo=cf.cashflow([1000]*10, start=2016),
                       exchange_rate=2)


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
And we now must re-run all our tests again, as the code has changed once more:
Can you write Python code for it?

print(real_quadratic_roots(1, 0, 1))
print(real_quadratic_roots(1, 0, 0))
print(real_quadratic_roots(1, 0, -1))
print(real_quadratic_roots(0, 1, 1))


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
As a final test, we check what happens if the equation degenerates to a linear equation where $a=0$, using $x + 1 = 0$ with solution $-1$:
Can you write Python code for it?


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
As we wanted, it has returned None. We also check what happens if the roots are zero, using $x^2 = 0$:
Can you write Python code for it?


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
But what happens if we try something really stupid?
Can you write Python code for it?


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
Formalizing tests
This small set of tests covers most of the cases we are concerned with. However, by this point it's getting hard to remember

what each line is actually testing, and
what the correct value is meant to be.

To formalize this, we write each test as a small function that contains this information for us. Let's start with the $x^2 - 1 = 0$ case where the roots are $\pm 1$:
Can you write Python code for it?

from numpy.testing import assert_equal, assert_allclose

def test_real_distinct():
    """
    Test that the roots of x^2 - 1 = 0 are \pm 1.
    """

    roots = (1.0, -1.0)
    assert_equal(real_quadratic_roots(1, 0, -1), roots,
                 err_msg="Testing x^2-1=0; roots should be 1 and -1.")

test_real_distinct()


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
Here we have caught two possible types of error within the tuple (which must, in this case, have parantheses) and captured the specific error in the variable exception. This variable can then be used: here we just print it out.
Normally best practise is to be as specific as possible on the error you are trying to catch.
Extending the logic
Sometimes you may want to perform an action only if an error did not occur. For example, let's suppose we wanted to store the result of dividing 4 by a divisor, and also store the divisor, but only if the divisor is valid.
One way of doing this would be the following:
Can you write Python code for it?

denominators = [1.0, 0.0, 3.0, "zero", 5.0]
results = []
divisors = []
for denominator in denominators:
    try:
        result = divide(4.0, denominator)
    except (ZeroDivisionError, TypeError) as exception:
        print("Error of type {} for denominator {}".format(exception, denominator))
    else:
        results.append(result)
        divisors.append(denominator)
print(results)
print(divisors)


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
In this case we get an exception, which we don't want. We fix this problem:
Can you write Python code for it?

from math import sqrt

def real_quadratic_roots(a, b, c):
    """
    Find the real roots of the quadratic equation a x^2 + b x + c = 0, if they exist.

    Parameters
    ----------

    a : float
        Coefficient of x^2
    b : float
        Coefficient of x^1
    c : float
        Coefficient of x^0

    Returns
    -------

    roots : tuple or float or None
        The root(s) (two if a genuine quadratic, one if linear, None otherwise)

    Raises
    ------

    NotImplementedError
        If the equation has trivial a and b coefficients, so isn't solvable.
    """

    discriminant = b**2 - 4.0*a*c
    if discriminant < 0.0:
        return None

    if a == 0:
        if b == 0:
            raise NotImplementedError("Cannot solve quadratic with both a"
                                      " and b coefficients equal to 0.")
        else:
            return -c / b

    x_plus = (-b + sqrt(discriminant)) / (2.0*a)
    x_minus = (-b - sqrt(discriminant)) / (2.0*a)

    return x_plus, x_minus


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
It should be obvious to the code that this is going to go wrong. Rather than letting the code hit the ZeroDivisionError exception automatically, we can raise it ourselves, with a more meaningful error message:
Can you write Python code for it?

def divide_sum(numerator, denominator1, denominator2):
    """
    Divide a number by a sum.

    Parameters
    ----------

    numerator: float
        numerator
    denominator1: float
        Part of the denominator
    denominator2: float
        Part of the denominator

    Returns
    -------

    fraction: float
        numerator / (denominator1 + denominator2)
    """

    if (denominator1 + denominator2) == 0:
        raise ZeroDivisionError("The sum of denominator1 and denominator2 is zero!")

    return numerator / (denominator1 + denominator2)

divide_sum(1, 1, -1)


def divide_sum(numerator, denominator1, denominator2):
    """
    Divide a number by a sum.

    Parameters
    ----------

    numerator: float
        numerator
    denominator1: float
        Part of the denominator
    denominator2: float
        Part of the denominator

    Returns
    -------

    fraction: float
        numerator / (denominator1 + denominator2)
    """

    if (denominator1 + denominator2) == 0:
        raise ZeroDivisionError("The sum of denominator1 and denominator 2 is zero!")

    return numerator / (denominator1 + denominator2)

divide_sum(1, 1, -1)


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
So, the code works fine until we pass in input that we shouldn't. When we do, this causes the code to stop. To show how this can be a problem, consider the loop:
Can you write Python code for it?

denominators = [1.0, 0.0, 3.0, 5.0]
for denominator in denominators:
    print(divide(4.0, denominator))


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
Something has gone wrong. Looking at the code, we see that the x_minus line has been copied and pasted from the x_plus line, without changing the sign correctly. So we fix that error:
Can you write Python code for it?

from math import sqrt

def real_quadratic_roots(a, b, c):
    """
    Find the real roots of the quadratic equation a x^2 + b x + c = 0, if they exist.

    Parameters
    ----------

    a : float
        Coefficient of x^2
    b : float
        Coefficient of x^1
    c : float
        Coefficient of x^0

    Returns
    -------

    roots : tuple or None
        The roots
    """

    discriminant = b**2 - 4.0*a*c
    if discriminant < 0.0:
        return None

    x_plus = (-b + sqrt(discriminant)) / (2.0*a)
    x_minus = (-b - sqrt(discriminant)) / (2.0*a)

    return x_plus, x_minus


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
Testing
How do we know if our code is working correctly? It is not when the code runs and returns some value: as seen above, there may be times where it makes sense to stop the code even when it is correct, as it is being used incorrectly. We need to test the code to check that it works.
Unit testing is the idea of writing many small tests that check if simple cases are behaving correctly. Rather than trying to prove that the code is correct in all cases (which could be very hard), we check that it is correct in a number of tightly controlled cases (which should be more straightforward). If we later find a problem with the code, we add a test to cover that case.
Consider a function solving for the real roots of the quadratic equation again. This time, if there are no real roots we shall return None (to say there are no roots) instead of raising an exception.
Can you write Python code for it?

from math import sqrt

def real_quadratic_roots(a, b, c):
    """
    Find the real roots of the quadratic equation a x^2 + b x + c = 0, if they exist.

    Parameters
    ----------

    a : float
        Coefficient of x^2
    b : float
        Coefficient of x^1
    c : float
        Coefficient of x^0

    Returns
    -------

    roots : tuple or None
        The roots
    """

    discriminant = b**2 - 4.0*a*c
    if discriminant < 0.0:
        return None

    x_plus = (-b + sqrt(discriminant)) / (2.0*a)
    x_minus = (-b + sqrt(discriminant)) / (2.0*a)

    return x_plus, x_minus


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
Testing that one floating point number equals another can be dangerous. Consider $x^2 - 2 x + (1 - 10^{-10}) = 0$ with roots $1.1 \pm 10^{-5} )$:
Can you write Python code for it?

from math import sqrt

def test_real_distinct_irrational():
    """
    Test that the roots of x^2 - 2 x + (1 - 10**(-10)) = 0 are 1 \pm 1e-5.
    """

    roots = (1 + 1e-5, 1 - 1e-5)
    assert_equal(real_quadratic_roots(1, -2.0, 1.0 - 1e-10), roots,
                 err_msg="Testing x^2-2x+(1-1e-10)=0; roots should be 1 +- 1e-5.")

test_real_distinct_irrational()


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
The assert_allclose statement takes options controlling the precision of our test.
We can now write out all our tests:
Can you write Python code for it?

from math import sqrt
from numpy.testing import assert_equal, assert_allclose

def test_no_roots():
    """
    Test that the roots of x^2 + 1 = 0 are not real.
    """

    roots = None
    assert_equal(real_quadratic_roots(1, 0, 1), roots,
                 err_msg="Testing x^2+1=0; no real roots.")

def test_zero_roots():
    """
    Test that the roots of x^2 = 0 are both zero.
    """

    roots = (0, 0)
    assert_equal(real_quadratic_roots(1, 0, 0), roots,
                 err_msg="Testing x^2=0; should both be zero.")

def test_real_distinct():
    """
    Test that the roots of x^2 - 1 = 0 are \pm 1.
    """

    roots = (1.0, -1.0)
    assert_equal(real_quadratic_roots(1, 0, -1), roots,
                 err_msg="Testing x^2-1=0; roots should be 1 and -1.")

def test_real_distinct_irrational():
    """
    Test that the roots of x^2 - 2 x + (1 - 10**(-10)) = 0 are 1 \pm 1e-5.
    """

    roots = (1 + 1e-5, 1 - 1e-5)
    assert_allclose(real_quadratic_roots(1, -2.0, 1.0 - 1e-10), roots,
                 err_msg="Testing x^2-2x+(1-1e-10)=0; roots should be 1 +- 1e-5.")

def test_real_linear_degeneracy():
    """
    Test that the root of x + 1 = 0 is -1.
    """

    root = -1.0
    assert_equal(real_quadratic_roots(0, 1, 1), root,
                 err_msg="Testing x+1=0; root should be -1.")

test_no_roots()
test_zero_roots()
test_real_distinct()
test_real_distinct_irrational()
test_real_linear_degeneracy()


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
The idea here is given by the names. Python will try to execute the code inside the try block. This is just like an if or a for block: each command that is indented in that block will be executed in order.
If, and only if, an error arises then the except block will be checked. If the error that is produced matches the one listed then instead of stopping, the code inside the except block will be run instead.
To show how this works with different errors, consider a different silly error:
Can you write Python code for it?

try:
    print(divide(4.0, "zero"))
except ZeroDivisionError:
    print("Dividing by zero is a silly thing to do!")


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
The statements in the else block are only run if the try block succeeds. If it doesn't - if the statements in the try block raise an exception - then the statements in the else block are not run.
Exceptions in your own code
Sometimes you don't want to wait for the code to break at a low level, but instead stop when you know things are going to go wrong. This is usually because you can be more informative about what's going wrong. Here's a slightly artificial example:
Can you write Python code for it?

def divide_sum(numerator, denominator1, denominator2):
    """
    Divide a number by a sum.

    Parameters
    ----------

    numerator: float
        numerator
    denominator1: float
        Part of the denominator
    denominator2: float
        Part of the denominator

    Returns
    -------

    fraction: float
        numerator / (denominator1 + denominator2)
    """

    return numerator / (denominator1 + denominator2)

divide_sum(1, 1, -1)


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
There are a large number of standard exceptions in Python, and most of the time you should use one of those, combined with a meaningful error message. One is particularly useful: NotImplementedError.
This exception is used when the behaviour the code is about to attempt makes no sense, is not defined, or similar. For example, consider computing the roots of the quadratic equation, but restricting to only real solutions. Using the standard formula
$$  x_{\pm} = \frac{-b \pm \sqrt{b^2 - 4ac}}{2a} $$
we know that this only makes sense if $b^2 \ge 4ac$. We put this in code as:
Can you write Python code for it?

from math import sqrt

def real_quadratic_roots(a, b, c):
    """
    Find the real roots of the quadratic equation a x^2 + b x + c = 0, if they exist.

    Parameters
    ----------

    a : float
        Coefficient of x^2
    b : float
        Coefficient of x^1
    c : float
        Coefficient of x^0

    Returns
    -------

    roots : tuple
        The roots

    Raises
    ------

    NotImplementedError
        If the roots are not real.
    """

    discriminant = b**2 - 4.0*a*c
    if discriminant < 0.0:
        raise NotImplementedError("The discriminant is {} < 0. "
                                  "No real roots exist.".format(discriminant))

    x_plus = (-b + sqrt(discriminant)) / (2.0*a)
    x_minus = (-b - sqrt(discriminant)) / (2.0*a)

    return x_plus, x_minus

print(real_quadratic_roots(1.0, 5.0, 6.0))

real_quadratic_roots(1.0, 1.0, 5.0)


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
There are a large number of standard exceptions in Python, and most of the time you should use one of those, combined with a meaningful error message. One is particularly useful: NotImplementedError.
This exception is used when the behaviour the code is about to attempt makes no sense, is not defined, or similar. For example, consider computing the roots of the quadratic equation, but restricting to only real solutions. Using the standard formula
\begin{equation}
  x_{\pm} = \frac{-b \pm \sqrt{b^2 - 4ac}}{2a}
\end{equation}
we know that this only makes sense if $b^2 \ge 4ac$. We put this in code as:
Can you write Python code for it?

from math import sqrt

def real_quadratic_roots(a, b, c):
    """
    Find the real roots of the quadratic equation a x^2 + b x + c = 0, if they exist.

    Parameters
    ----------

    a : float
        Coefficient of x^2
    b : float
        Coefficient of x^1
    c : float
        Coefficient of x^0

    Returns
    -------

    roots : tuple
        The roots

    Raises
    ------

    NotImplementedError
        If the roots are not real.
    """

    discriminant = b**2 - 4.0*a*c
    if discriminant < 0.0:
        raise NotImplementedError("The discriminant is {}<0. "
                                  "No real roots exist.".format(discriminant))

    x_plus = (-b + sqrt(discriminant)) / (2.0*a)
    x_minus = (-b - sqrt(discriminant)) / (2.0*a)

    return x_plus, x_minus

print(real_quadratic_roots(1.0, 5.0, 6.0))

real_quadratic_roots(1.0, 1.0, 5.0)


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
There are three sensible results, but we only get the first.
There are many more complex, real cases where it's not obvious that we're doing something wrong ahead of time. In this case, we want to be able to try running the code and catch errors without stopping the code. This can be done in Python:
Can you write Python code for it?

try:
    print(divide(4.0, 0.0))
except ZeroDivisionError:
    print("Dividing by zero is a silly thing to do!")

denominators = [1.0, 0.0, 3.0, 5.0]
for denominator in denominators:
    try:
        print(divide(4.0, denominator))
    except ZeroDivisionError:
        print("Dividing by zero is a silly thing to do!")


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
This doesn't give us much information, and may lose information that we need in order to handle the error. We can capture the exception to a variable, and then use that variable:
Can you write Python code for it?

try:
    print(divide(4.0, "zero"))
except (ZeroDivisionError, TypeError) as exception:
    print("Some error occured: {}".format(exception))


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
We could catch any error:
Can you write Python code for it?

try:
    print(divide(4.0, "zero"))
except:
    print("Some error occured")


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
We get the expected behaviour. We also check what happens if the roots are real, using $x^2 - 1 = 0$ which has roots $\pm 1$:
Can you write Python code for it?


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
We have changed the code, so now have to re-run all our tests, in case our change broke something else:
Can you write Python code for it?

print(real_quadratic_roots(1, 0, 1))
print(real_quadratic_roots(1, 0, 0))
print(real_quadratic_roots(1, 0, -1))


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
We see that the solutions match to the first 14 or so digits, but this isn't enough for them to be exactly the same. In this case, and in most cases using floating point numbers, we want the result to be "close enough": to match the expected precision. There is an assertion for this as well:
Can you write Python code for it?

from math import sqrt

def test_real_distinct_irrational():
    """
    Test that the roots of x^2 - 2 x + (1 - 10**(-10)) = 0 are 1 \pm 1e-5.
    """

    roots = (1 + 1e-5, 1 - 1e-5)
    assert_allclose(real_quadratic_roots(1, -2.0, 1.0 - 1e-10), roots,
                 err_msg="Testing x^2-2x+(1-1e-10)=0; roots should be 1 +- 1e-5.")

test_real_distinct_irrational()


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
We see that, as it makes no sense to divide by a string, we get a TypeError instead of a ZeroDivisionError. We could catch both errors:
Can you write Python code for it?

try:
    print(divide(4.0, "zero"))
except ZeroDivisionError:
    print("Dividing by zero is a silly thing to do!")
except TypeError:
    print("Dividing by a string is a silly thing to do!")


I am working on the file "09-exceptions-testing.ipynb".
The first task is:
What this function does is checks that the results of the function call match the expected value, here stored in roots. If it didn't match the expected value, it would raise an exception:
Can you write Python code for it?

def test_should_fail():
    """
    Comparing the roots of x^2 - 1 = 0 to (1, 1), which should fail.
    """

    roots = (1.0, 1.0)
    assert_equal(real_quadratic_roots(1, 0, -1), roots,
                 err_msg="Testing x^2-1=0; roots should be 1 and 1."
                 " So this test should fail")

test_should_fail()


I am working on the file "09-notebook-after-class.ipynb".
The first task is:
Aufgabe 4     Weniger Redundanz dank besonderer Merkmale
Beseitigen Sie die Redundanz in den lexikalischen Regeln (Zeilen 8 - 32) der folgenden Grammatik durch eine Typhierarchie (wo dies nötig ist). Achten Sie darauf, die Menge der akzeptierten Sätze weder zu verkleinern noch zu vergrößern!
Anzugeben sind die neuen Grammatikregeln, sowie Ihre Typhierarchie (z. B. in graphischer Form).
Can you write Python code for it?

redundant_grammar = """
S -> NP[KAS=nom] VP

NP[KAS=?y] -> DET[GEN=?x,KAS=?y] NOM[GEN=?x,KAS=?y]
NOM[GEN=?x,KAS=?y] -> N[GEN=?x,KAS=?y] NP[KAS=gen]
NOM[GEN=?x,KAS=?y] -> N[GEN=?x,KAS=?y]

DET[GEN=mask,KAS=nom] -> "der"
DET[GEN=mask,KAS=gen] -> "des"
DET[GEN=mask,KAS=dat] -> "dem"
DET[GEN=mask,KAS=akk] -> "den"
DET[GEN=fem,KAS=nom] -> "die"
DET[GEN=fem,KAS=gen] -> "der"
DET[GEN=fem,KAS=dat] -> "der"
DET[GEN=fem,KAS=akk] -> "die"
DET[GEN=neut,KAS=nom] -> "das"
DET[GEN=neut,KAS=gen] -> "des"
DET[GEN=neut,KAS=dat] -> "dem"
DET[GEN=neut,KAS=akk] -> "das"

N[GEN=mask,KAS=nom] -> "Mann"
N[GEN=mask,KAS=gen] -> "Mannes"
N[GEN=mask,KAS=dat] -> "Mann"
N[GEN=mask,KAS=akk] -> "Mann"
N[GEN=fem,KAS=nom] -> "Frau"
N[GEN=fem,KAS=gen] -> "Frau"
N[GEN=fem,KAS=dat] -> "Frau"
N[GEN=fem,KAS=akk] -> "Frau"
N[GEN=neut,KAS=nom] -> "Buch"
N[GEN=neut,KAS=gen] -> "Buches"
N[GEN=neut,KAS=dat] -> "Buch"
N[GEN=neut,KAS=akk] -> "Buch"

VP -> V NP[KAS=dat] NP[KAS=akk]
V -> "gibt" | "schenkt"
"""

pos_sentences = [
    "der Mann gibt der Frau das Buch",
    "die Frau des Mannes gibt dem Mann der Frau das Buch des Buches"
]


I am working on the file "09-notebook-after-class.ipynb".
The first task is:
Folgendes sollte leer sein:
Can you write Python code for it?


I am working on the file "09-notebook-after-class.ipynb".
The first task is:
Folgendes sollte wieder funktionieren. Betrachten Sie aufmerksam die Merkmale im Syntaxbaum.
Can you write Python code for it?

for t in parser.parse("der Mann der Frau schläft".split()):
    display(t)


I am working on the file "09-notebook-after-class.ipynb".
The first task is:
Gegeben seien folgende Merkmalstrukturen:
Can you write Python code for it?

f1 = FeatStruct(
    '[Vorname=Max, Nachname=Mustermann,' +
    'Privat=[Strasse=Hauptstrasse, Ort=[Muenchen]]]'
)
f2 = FeatStruct(
    '[Arbeit=[Strasse="Oettingenstrasse", Ort=(1)["Muenchen"]],' +
    'Privat=[Ort->(1)]]')
f3 = FeatStruct(
    '[Strasse="Hauptstrasse"]'
)
f4 = FeatStruct(
    '[Privat=[Strasse="Hauptstrasse", Ort=["Passau"]]]'
)


I am working on the file "09-notebook-after-class.ipynb".
The first task is:
Hausaufgaben
Aufgabe 3     Unifikation II
Es seien wieder die Merkmalstrukturen aus Aufgabe 1 gegeben.
Unifizieren Sie:
- f1 mit f4
Can you write Python code for it?


I am working on the file "09-notebook-after-class.ipynb".
The first task is:
Hier muss die Typhierarchie in Form eines Dictionary definiert werden:
Can you write Python code for it?

type_hierarchy = {
    "gen": [],
    "nongen": ["nomakk", "dat"],
    "nomakk": ["nom", "akk"],
    "nom": [],
    "dat": [],
    "akk": []
}

CASE = HierarchicalFeature("CASE", type_hierarchy)
compiled_grammar = nltk.grammar.FeatureGrammar.fromstring(
    grammar, features=(CASE, TYPE)
)
parser = nltk.FeatureEarleyChartParser(compiled_grammar)


I am working on the file "09-notebook-after-class.ipynb".
The first task is:
Testen Sie mit Ihren eigenen Negativbeispielen!
Can you write Python code for it?

neg_sentences = [

]

def test_grammar(grammar, sentences):
    cfg = nltk.grammar.FeatureGrammar.fromstring(grammar)
    parser = nltk.parse.FeatureEarleyChartParser(cfg)

    for i, sent in enumerate(sentences, 1):
        print("Satz {}: {}".format(i, sent))
        results = parser.parse(sent.split())
        analyzed = False
        for tree in results:
            print(tree)  # oder display(tree)
            analyzed = True
        if not analyzed:
            print("Keine Analyse möglich", file=sys.stderr)

test_grammar(redundant_grammar, pos_sentences)


I am working on the file "09-notebook-after-class.ipynb".
The first task is:
Unifizieren Sie:
- f1 mit f2
Can you write Python code for it?


I am working on the file "09-notebook-solution.ipynb".
The first task is:
Folgendes sollte leer sein:
Can you write Python code for it?


I am working on the file "09-notebook-solution.ipynb".
The first task is:
Gegeben seien folgende Merkmalstrukturen:
Can you write Python code for it?


I am working on the file "09-notebook-solution.ipynb".
The first task is:
Hier muss die Typhierarchie in Form eines Dictionary definiert werden:
Can you write Python code for it?


I am working on the file "09-notebook-solution.ipynb".
The first task is:
Testen Sie mit Ihren eigenen Negativbeispielen!
Can you write Python code for it?

neg_sentences = [
    "des Mannes gibt der Frau das Buch",
    "Mann gibt der Frau das Buch",
    "der Mann gibt der Frau Buch",
    "der Frau gibt dem Buch den Mann",
    "das Buch der Mann gibt der Frau das Buch"
]

import sys

def test_grammar(parser, sentences):
    for i, sent in enumerate(sentences, 1):
        print("Satz {}: {}".format(i, sent))
        sys.stdout.flush()
        results = parser.parse(sent.split())
        analyzed = False
        for tree in results:
            display(tree)
            analyzed = True
        if not analyzed:
            print("Keine Analyse möglich", file=sys.stderr)
            sys.stderr.flush()

test_grammar(parser, pos_sentences)

test_grammar(parser, neg_sentences)


I am working on the file "09-notebook-solution.ipynb".
The first task is:
f2 mit f4
Can you write Python code for it?


I am working on the file "09-notebook.ipynb".
The first task is:
Aufgabe 2     Typhierarchie im NLTK
Gegeben sei folgende Typhierarchie:
$$\bot \sqsubseteq \text{Genitiv}$$
$$\bot \sqsubseteq \text{nicht-Genitiv}$$
$$\text{nicht-Genitiv} \sqsubseteq \text{Nominativ-Akkusativ}$$
$$\text{nicht-Genitiv} \sqsubseteq \text{Dativ}$$
$$\text{Nominativ-Akkusativ} \sqsubseteq \text{Nominativ}$$
$$\text{Nominativ-Akkusativ} \sqsubseteq \text{Akkusativ}$$
Implementieren Sie mithilfe der Klasse HierarchicalFeature,
die Sie sich von der Kurs-Website herunterladen können, ein Feature CASE, das der vorgegebenen Typhierarchie entspricht.
Nutzen Sie dieses Feature dann, um Übergenerierung in folgender Grammatik zu vermeiden:
Can you write Python code for it?

grammar = """
S -> NP VP
NP -> DET NOM
NOM -> N NP | N
VP -> V

V -> "schläft"
DET[GEN=mask] -> "der" | "des"
DET[GEN=fem] -> "die" | "der"
DET[GEN=neut] -> "das" | "des"
N[GEN=mask] -> "Mann" | "Mannes"
N[GEN=fem] -> "Frau"
N[GEN=neut] -> "Kind" | "Kindes"
"""

import nltk
from typed_features import HierarchicalFeature, TYPE


I am working on the file "09-notebook.ipynb".
The first task is:
Aufgabe 4     Weniger Redundanz dank besonderer Merkmale
Beseitigen Sie die Redundanz in den lexikalischen Regeln (Zeilen 8 - 32) der folgenden Grammatik durch eine Typhierarchie (wo dies nötig ist). Achten Sie darauf, die Menge der akzeptierten Sätze weder zu verkleinern noch zu vergrößern!
Anzugeben sind die neuen Grammatikregeln, sowie Ihre Typhierarchie (z. B. in graphischer Form).
Can you write Python code for it?


I am working on the file "09-notebook.ipynb".
The first task is:
Folgendes sollte funktionieren:
Can you write Python code for it?

for t in parser.parse("das Kind des Mannes schläft".split()):
    print(t)


I am working on the file "09-notebook.ipynb".
The first task is:
Folgendes sollte leer sein:
Can you write Python code for it?


I am working on the file "09-notebook.ipynb".
The first task is:
Hier muss die Typhierarchie in Form eines Dictionary definiert werden:
Can you write Python code for it?

type_hierarchy = {

}

CASE = HierarchicalFeature("CASE", type_hierarchy)
compiled_grammar = nltk.grammar.FeatureGrammar.fromstring(
    grammar, features=(CASE, TYPE)
)
parser = nltk.FeatureEarleyChartParser(compiled_grammar)


I am working on the file "09-notebook.ipynb".
The first task is:
Testen Sie mit Ihren eigenen Negativbeispielen!
Can you write Python code for it?

neg_sentences = [

]

from IPython.display import display

def test_grammar(grammar, sentences):
    cfg = nltk.grammar.FeatureGrammar.fromstring(grammar)
    parser = nltk.parse.FeatureEarleyChartParser(cfg)

    for i, sent in enumerate(sentences, 1):
        print("Satz {}: {}".format(i, sent))
        results = parser.parse(sent.split())
        analyzed = False
        for tree in results:
            print(tree)  # oder display(tree)
            analyzed = True
        if not analyzed:
            print("Keine Analyse möglich", file=sys.stderr)

test_grammar(redundant_grammar, pos_sentences)


I am working on the file "09-notebook.ipynb".
The first task is:
f2 mit f3
Can you write Python code for it?


I am working on the file "09. 기초 확률론2 - 확률 변수/01. NumPy를 사용한 난수 발생.ipynb".
The first task is:
그러나 unique 값은 데이터에 존재하는 값에 대해서만 갯수를 세므로 나올 수 있음에도 불구하고 데이터가 하나도 없는 값에 대한 정보를 주지 않는다.
따라서 데이터가 주사위를 던졌을 때 나오는 수처럼 특정 범위안의 수인 경우에는 bincount에 minlength 인수를 설정하여 쓰는 것이 더 편리하다. 이 때는 0 부터 minlength - 1 까지의 숫자에 대해 각각 카운트를 한다. 데이터가 없을 경우에는 카운트 값이 0이 된다.
Can you write Python code for it?

np.bincount([1, 1, 2, 2, 2, 3], minlength=6)


I am working on the file "09. 기초 확률론2 - 확률 변수/01. NumPy를 사용한 난수 발생.ipynb".
The first task is:
기존의 데이터에서 샘플링하기
이미 있는 데이터 집합에서 일부를 선택하는 것을 샘플링(sampling)이라고 한다. 샘플링에는 choice 명령을 사용한다. choice 명령은 다음과 같은 인수를 가질 수 있다.
numpy.random.choice(a, size=None, replace=True, p=None)

a : 배열이면 원래의 데이터, 정수이면 range(a) 명령으로 데이터 생성
size : 정수. 샘플 숫자
replace : 불리언. True이면 한번 선택한 데이터를 다시 선택 가능
p : 배열. 각 데이터가 선택될 수 있는 확률
Can you write Python code for it?

np.random.choice(5, 5, replace=False)  # shuffle 명령과 같다.

np.random.choice(5, 3, replace=False)  # 3개만 선택

np.random.choice(5, 10) # 반복해서 10개 선택

np.random.choice(5, 10, p=[0.1, 0, 0.3, 0.6, 0])  # 선택 확률을 다르게 해서 10개 선택


I am working on the file "09. 기초 확률론2 - 확률 변수/01. NumPy를 사용한 난수 발생.ipynb".
The first task is:
아까와 같은 숫자가 나오는 것을 확인할 수 있다.
기존의 데이터의 순서 바꾸기
기존에 있는 데이터의 순서를 바꾸려면 shuffle 명령을 사용한다.
Can you write Python code for it?

x = np.arange(10)
np.random.shuffle(x)
x


I am working on the file "09. 기초 확률론2 - 확률 변수/01. NumPy를 사용한 난수 발생.ipynb".
The first task is:
이제 시드를 0으로 재설정하고 다시 난수를 발생시켜 본다.
Can you write Python code for it?

np.random.seed(0)

np.random.rand(5)

np.random.rand(10)

np.random.rand(10)


I am working on the file "09. 기초 확률론2 - 확률 변수/01. NumPy를 사용한 난수 발생.ipynb".
The first task is:
정수 난수 통계
이렇게 발생시킨 난수가 실수값이면 히스토그램 등을 사용하여 분석하면 된다.
만약 난수가 정수값이면 unique 명령이나 bincount 명령으로 데이터 값을 분석할 수 있다. 이 명렬들은 random 서브패키지가 아니라 NumPy 바로 아래에 포함된 명령이다.
unique 명령은 데이터에서 중복된 값을 제거하고 중복되지 않는 값의 리스트를 출력한다. return_counts 인수를 True 로 설정하면 각 값을 가진 데이터 갯수도 출력한다.
Can you write Python code for it?

np.unique([11, 11, 2, 2, 34, 34])

a = np.array(['a', 'b', 'b', 'c', 'a'])
index, count = np.unique(a, return_counts=True)

index

count


I am working on the file "09. 기초 확률론2 - 확률 변수/03. 확률 모형이란.ipynb".
The first task is:
이 데이터는 사실 진짜로 컴퓨터 주사위를 던져 만든 숫자이다. 이 숫자를 만드는 코드는 아래와 같다.
Can you write Python code for it?

np.random.seed(0)
np.random.randint(1, 7, 92)


I am working on the file "09. 기초 확률론2 - 확률 변수/03. 확률 모형이란.ipynb".
The first task is:
이 히스토그램에서 -0.143394 부터 0.437156 사이의 값이 전체의 약 24%를 차지하고 있음을 알 수 있다. 그럼 만약 -0.01  부터 0.01 사이의 구간에 대한 정보를 얻고 싶다면? 더 세부적인 구간에 대해 정보를 구하고 싶다면 히스토그램의 구간을 더 작게 나누어야 한다.
Can you write Python code for it?

ns, bins, ps = plt.hist(x, bins=100)
plt.show()

pd.DataFrame([bins, ns/1000])


I am working on the file "09. 기초 확률론2 - 확률 변수/03. 확률 모형이란.ipynb".
The first task is:
정확한 묘사를 위해 구간의 수를 증가시키면 몇 가지 문제가 발생한다.
우선 구간의 간격이 작아지면서 하나의 구간에 있는 자료의 수가 점점 적어진다. 만약 구간 수가 무한대에 가깝다면 하나의 구간 폭은 0으로 수렴하고 해당 구간의 자료 수도 0으로 수렴할 것이다. 따라서 분포의 상대적인 모양을 살펴보기 힘들어진다.
더 큰 문제는 분포를 묘사하기 위한 정보가 증가한다는 점이다. 데이터의 분포를 묘사하는 이유는 적은 갯수의 숫자를 통해 데이터의 전반적인 모습을 빠르게 파악하기 위한 것인데 묘사를 위한 정보의 양이 증가하면 원래의 목적을 잃어버린다.
확률 모형
이러한 문제를 해결하기 위해 만들어진 것이 확률 모형(probability model)이다. 확률 모형은 추후 설명할 확률 변수(random variable)라는 것을 이용하여 데이터 분포를 수학적으로 정의하는 방법을 말한다.
보통 확률 분포 함수(probability distribution function) 또는 확률 밀도 함수(probability density function)라고 불리우는 미리 정해진 함수의 수식을 사용한다. 이 때 이 함수들의 계수를 분포의 모수(parameter)라고 부른다.
예를 들어 가장 널리 쓰이는 확률 모형의 하나인 가우시안 정규 분포(Gaussian normal distribution)는 다음과 같은 수식으로 확률 밀도 함수를 정의한다.
$$ N(x; \mu, \sigma) = \frac{1}{\sigma\sqrt{2\pi}}\, e^{-\frac{(x - \mu)^2}{2 \sigma^2}} $$
이 함수의 독립 변수는 자료의 값을 의미하는 변수 $x$이다. 식에서 사용된 문자 $\mu$와 $\sigma$는 평균(mean)과 표준편차(standard deviation)를 뜻하는 모수이다. 함수 표기에서 세미콜론(;)은 독립 변수와 모수를 구분하기 위해 사용하였다.
이 함수식이 실제 분포와 어떤 관계를 가지는지는 분포 함수에 대한 다음 노트북을 참조한다.
[[school_notebook:4d74d1b5651245a7903583f30ae44608]]
다음 그림은 scipy를 사용하여 평균 0, 표준편차 1인 표준 정규 분포(standard normal distribution)의 모양을 그린것이다.
Can you write Python code for it?

x = np.linspace(-3, 3, 100)
y = sp.stats.norm.pdf(x)
plt.plot(x, y)
plt.show()


I am working on the file "09. 기초 확률론2 - 확률 변수/03. 확률 모형이란.ipynb".
The first task is:
히스토그램을 그리면 다음과 같다.
Can you write Python code for it?

ns, bins, ps = plt.hist(x, bins=10)
plt.show()

pd.DataFrame([bins, ns/1000])


I am working on the file "09. 기초 확률론2 - 확률 변수/05. 누적 분포 함수와 확률 밀도 함수.ipynb".
The first task is:
누적 밀도 함수 즉 cdf는 다음과 같은 특징을 가진다.

$F(-\infty) = 0$
$F(+\infty) = 1$
$F(x) \geq F(y) \;\; \text{ if } \;\; x > y $

확률 밀도 함수
누적 분포 함수는 확률 분포를 함수라는 편리한 상태로 바꾸어 주었다. 누적 분포 함수는 확률이 어느 사건(event)에 어느 정도 분포되어 있는지 수학적으로 명확하게 표현해 준다.
그러나 누적 분포 함수가 표현하는 사건이 음수 무한대를 시작점으로 하고 변수 $x$를 끝점으로 하는 구간이다보니 분포의 형상을 직관적으로 이해하기는 힘든 단점이 있다. 다시 말해서 어떤 확률 변수 값이 더 자주 나오는지에 대한 정보를 알기 힘들다는 점이다.
이를 알기 위해서는 확률 변수가 나올 수 있는 전체 구간 ($-\infty$ ~ $\infty$)을 아주 작은 폭을 가지는 구간들로 나눈 다음 각 구간의 확률을 살펴보는 것이 편리하다. 다만 이렇게 되면 구간의 폭(width)을 어느 정도로 정의해야 하는지에 대한 추가적인 약속이 필요하기 때문에 실효성이 떨어진다.
이러한 단점을 보완하기 위해 생각한 것이 절대적인 확률이 아닌 상대적인 확률 분포 형태만을 보기 위한 확률 밀도 함수(probability density function)이다.
누적 확률 분포 그래프의 x축의 오른쪽으로 이동하면서 크기의 변화를 살펴보자.만약 특정한 $x$값 근처의 구간에 확률이 배정되지 않았다면 누적 분포 함수는 그 구간을 지나도 증가하지 않는다. 즉, 기울기가 0이다. 왜냐하면 $x$ 값이 커졌다(x축의 오른쪽으로 이동하였다)는 것은 앞의 구간을 포함하는 더 큰 구간(사건)에 대한 확률을 묘사하고 있는 것인데 추가적으로 포함된 신규 구간에 확률이 없다면 그 신규 구간을 포함한 구간이나 포함하지 않은 구간이나 배정된 확률이 같기 때문이다.
누적 분포 함수의 기울기가 0이 아닌 경우는 추가적으로 포함된 구간에 0이 아닌 확률이 할당되어 있는 경우이다. 만약 더 많은 확률이 할당되었다면 누적 분포 함수는 그 구간을 지나면서 더 빠른 속도로 증가할 것이다. 다시 말해서 함수의 기울기가 커진다. 이러한 방식으로 누적 분포의 기울기의 크기를 보면 각 위치에 배정된 확률의 상대적 크기를 알 수 있다.
기울기를 구하는 수학적 연산이 미분(differentiation)이므로 확률 밀도 함수는 누적 분포 함수의 미분으로 정의한다.
$$ \dfrac{dF(x)}{dx} = f(x) $$
이를 적분으로 나타내면 다음과 같다.
$$ F(x) = \int_{-\infty}^{x} f(u) du $$
확률 밀도 함수는 특정 확률 변수 구간의 확률이 다른 구간에 비해 상대적으로 얼마나 높은가를 나타내는 것이며 그 값 자체가 확률은 아니다라는 점을 명심해야 한다.
확률 밀도 함수는 다음과 같은 특징을 가진다.

$-\infty$ 부터 $\infty$ 까지 적분하면 그 값은 1이 된다.

$$ \int_{-\infty}^{\infty} f(u)du = 1$$

확률 밀도 함수는 0보다 같거나 크다.

$$ f(x) \geq 0 $$
앞서 보인 시계 바늘 문제에서 확률 밀도함수를 구하면 다음과 같다.
Can you write Python code for it?

t = np.linspace(-100, 500, 1000)
F = t / 360
F[t < 0] = 0
F[t > 360] = 1
f = np.gradient(F)  # 수치미분
plt.plot(t, f)
plt.ylim(-0.0001, f.max()*1.1)
plt.xticks([0, 180, 360]);
plt.title("Probability Density Function");
plt.xlabel("$x$ (deg.)");
plt.ylabel("$f(x)$");


I am working on the file "09. 기초 확률론2 - 확률 변수/05. 누적 분포 함수와 확률 밀도 함수.ipynb".
The first task is:
시계 바늘 확률 문제의 경우를 예로 들어보자. 이 경우에는 각도가 0도부터 360까지이지만 음의 무한대를 시작점으로 해도 상관없다.
$$ F(0) = P({ -\infty {}^{\circ} \leq \theta < 0 {}^{\circ} }) = 0 $$
$$ F(10) = P({ -\infty {}^{\circ} \leq \theta < 10 {}^{\circ} }) = \dfrac{1}{36} $$
$$ F(20) = P({ -\infty {}^{\circ} \leq \theta < 20 {}^{\circ} }) = \dfrac{2}{36} $$
$$ \vdots $$
$$ F(350) = P({ -\infty {}^{\circ} \leq \theta < 350 {}^{\circ} }) = \dfrac{35}{36} $$
$$ F(360) = P({ -\infty {}^{\circ} \leq \theta < 360 {}^{\circ} }) = 1 $$
$$ F(370) = P({ -\infty {}^{\circ} \leq \theta < 370 {}^{\circ} }) = 1 $$
$$ F(380) = P({ -\infty {}^{\circ} \leq \theta < 380 {}^{\circ} }) = 1 $$
$$ \vdots $$
이를 NumPy와 matplotlib를 사용하여 그래프로 그래면 다음과 같다.
Can you write Python code for it?

t = np.linspace(-100, 500, 100)
F = t / 360
F[t < 0] = 0
F[t > 360] = 1
plt.plot(t, F)
plt.ylim(-0.1, 1.1)
plt.xticks([0, 180, 360]);
plt.title("Cumulative Distribution Function");
plt.xlabel("$x$ (deg.)");
plt.ylabel("$F(x)$");


I am working on the file "09. 기초 확률론2 - 확률 변수/05. 누적 분포 함수와 확률 밀도 함수.ipynb".
The first task is:
확률 질량 함수
이산 확률 분포는 확률 밀도 함수를 정의할 수 없는 대신 확률 질량 함수가 존재한다. 확률 질량 함수(probability mass funtion)는 이산 확률 변수의 가능한 값 하나 하나에 대해 확률을 정의한 함수이다. 예를 들어 6면체인 주사위를 던져서 나올 수 있는 값은 1부터 6까지의 이산적인 값을 가지는데 이러한 이산 확률 변수는 예를 들어 다음과 같은 확률 질량 함수를 가질 수 있다. 이 경우에는 공정하지 않은(unfair) 주사위의 확률 분포를 보이고 있다.
Can you write Python code for it?


I am working on the file "09.Reading_writing_files.ipynb".
The first task is:
And finally - time to be a....
<img src="images/sani.jpg">
Can you write Python code for it?


I am working on the file "09.Reading_writing_files.ipynb".
The first task is:
As you can see, the contents of the file are now assigned to a variable, f2_contents. f2_contents is a string.
Can you write Python code for it?


I am working on the file "09.Reading_writing_files.ipynb".
The first task is:
Exercise
All our theory has to lead to something. And here we have a hands on challenge. And just so you're not completely stumped by it, this is a problem you have partially worked on before.
As a data scientist, about 70 to 80% of your job will actually be exploring data sets, cleaning it, and running exploratory tests again on cleaned data sets. Modeling and predictions are trivial once the heavy lifting has been done, and I'm not exagerrating. Find any data scientist, and ask him or her what the most time consuming part of their job is.
Let's go to https://fakenumber.org/ and generate a few fake phone numbers. Then let's change it to look like a real world txt file with copied texts from multiple sources, words, names, extension numbers etc. Make it as difficult as possible for you. You have to keep challenging yourself to get to a higher and higher level. Think of it as playing COD/Super Mario/Contra/your favourite game. I have posted the solution a few boxes below, but this is for you to challenge yourself. No one's looking, and you can't cheat yourself!
Can you write Python code for it?

# This will create the phonenumbers.txt file. Unix FTW again!
%%file Data/phonenumbers.txt
202-555-0116
202-555-0181 Jill
202-555-0142
202-(555)-0173 Bryan
+1-202-555-0116
+1-202-555-0181
+1-202-555-0142 Raj
+1-202-555-0173
+1-(202)-555-0137 Jonah Lomu
Not really free 800-555-1231x1234
800-555-1212 ext. 1234
work 1-(800) 555.1212 #1234

ls

def clean_up(phone_num):
    number = ""
    digits = {"0","1","2","3","4","5","6","7","8","9"}

    for char in phone_num:
        if char in digits:
            number = result + c
    return number

f = open("Data/phonenumbers.txt", "r")
content = f.read()
f.close()

# Let's have a peek at the file
content

# Time to split the lines
lines = content.split("\n")

"""
I am only doing this since we know there are a couple of lines
Else, please don't print the whole file. There are other techniques
to look at snippets of data from within a file, and we will go over
them in later lessons.
"""
lines

for line in lines:
    print(clean(line))


I am working on the file "09.Reading_writing_files.ipynb".
The first task is:
Processing File Contents
Now that we have read from the file, we need to analyse it. Let's begin!
Can you write Python code for it?

lines = f2_contents.split("\n")
lines

len(lines)

for line in lines:
    #print("Length of line '{}' is {}".format(line, len(line)))
    print (len(line))


I am working on the file "09.Reading_writing_files.ipynb".
The first task is:
Reading from the disk
Can you write Python code for it?

f2 = open("example.txt", "r")
f2_contents = f2.read()
f2.close()

f2_contents


I am working on the file "09/.ipynb_checkpoints/09 - Functions-checkpoint.ipynb".
The first task is:
1b. Driving Speed Part II
Now write a function called to_mpm that, when given miles per hour, computes the meters per minute.
Can you write Python code for it?

#magic numbers -- unique values with
#def to_mpm(speed):
#return speed * 26.8
#return to_kmh(speed) * 1000 / 60

def to_mpm(speed):
    return round(speed * 26.8224)

mph = 40
print("You are driving", mph, "in mph")
print("You are driving", to_kmh(mph), "in kmh")
print("You are driving", to_mpm(mph), "in meters/minute")


I am working on the file "09/.ipynb_checkpoints/09 - Functions-checkpoint.ipynb".
The first task is:
2. Broken Function
The code below won't work. Why not?
Can you write Python code for it?

# You have to wash ten cars on every street, along with the cars in your driveway.
# With the following list of streets, how many cars do we have?

def total(n):
    return n * 10

# Here are the streets
streets = ['10th Ave', '11th Street', '45th Ave']

# Let's count them up
total = len(streets)

# And add one
count = total + 1

# And see how many we have
print(total(count))


I am working on the file "09/.ipynb_checkpoints/09 - Functions-checkpoint.ipynb".
The first task is:
3. Data converter
We have a bunch of data in different formats, and we need to normalize it! The data looks like this:
python
var first = { 'measurement': 3.4, 'scale': 'kilometer' }
var second = { 'measurement': 9.1, 'scale': 'mile' }
var third = { 'measurement': 2.0, 'scale': 'meter' }
var fourth = { 'measurement': 9.0, 'scale': 'inches' }
Write a function called to_meters(...). When you send it a dictionary, have it examine the measurement and scale and return the adjusted value. For the values above, 3.4 kilometers should be 3400.0 meters, 9.1 miles should be around 14600, and 9 inches should be apprxoimately 0.23.
Can you write Python code for it?

first = { 'measurement': 3.4, 'scale': 'kilometer' }
second = { 'measurement': 9.1, 'scale': 'mile' }
third = { 'measurement': 2.0, 'scale': 'meter' }
fourth = { 'measurement': 9.0, 'scale': 'inches' }

def to_meters(measurement):
    if measurement['scale'] == 'kilometer':
        return measurement['measurement'] * 1000
    if measurement['scale'] == 'meter':
        return measurement['measurement']
    if measurement['scale'] == 'miles':
        return measurement['measurement'] * 1.6 == 1000
    return 99

print(to_meters(first))
print(to_meters(second))


I am working on the file "09/.ipynb_checkpoints/09 - Functions-checkpoint.ipynb".
The first task is:
Almost everything useful is a function. Python has a ton of other built-in functions!
Along with len,  a couple you might have seen are:

abs(...) takes a number and returns the absolute value of the number
int(...) takes a string or float and returns it as an integer
round(...) takes a float and returns a rounded version of it
sum(...) takes a list and returns the sum of all of its elements
max(...) takes a list and returns the largest of all of its selements
print(...) takes whatever you want to give it and displays it on the screen

Functions can also come from packages and libraries. The .get part of requests.get is a function, too!
And here, to prove it to you?
Can you write Python code for it?

max

print

import requests
requests.get

# And if we just wanted to use them, for some reason
n = -34
print(n, "in absolute value is", abs(n))

print("We can add after casting to int:", 55 + int("55"))

n = 4.4847
print(n, "can be rounded to", round(n))
print(n, "can also be rounded to 2 decimal points", round(n, 2))

numbers = [4, 22, 40, 54]
print("The total of the list is", sum(numbers))


I am working on the file "09/.ipynb_checkpoints/09 - Functions-checkpoint.ipynb".
The first task is:
Function Naming
Your function name has to be unique, otherwise Python will get confused. No other functions or variabels can share its name!
For example, if you call it len it'll forget about the built-in len function, and if you give one of your variables the name print suddenly Python won't understand how print(...) works anymore.
If you end up doing this, you'll get errors like the one below
Can you write Python code for it?

def greet(name):
    return "Hello " + name

# This one works
print(greet("Soma"))

# Overwrite the function greet with a string
greet = "blah"

# Trying the function again breaks
print(greet("Soma"))


I am working on the file "09/.ipynb_checkpoints/09 - Functions-checkpoint.ipynb".
The first task is:
Horrifying, right? Thank goodness for functions.
Writing your own functions
I've always been kind of jealous of len(...) and its crowd. It seemed unfair that Python made a list of cool, important functions, and neither me nor you had any say in the matter. What if I want a function that turns all of the periods in a sentence into exclamation points, or prints out a word a hundred million times?
Well, turns out that isn't a problem. We can do that. Easily! And we will. If you can type def and use a colon, you can write a function.
A function that you write yourself looks like this:
Can you write Python code for it?

# A function to multiply a number by two
def double(number):
    bigger = number * 2
    return bigger
#what happens inside the function STAYS inside the function
#unless you use return, you don't know what happens within the function


I am working on the file "09/.ipynb_checkpoints/09 - Functions-checkpoint.ipynb".
The first task is:
Parameters
In our function double, we have a parameter called number.
py
def double(number):
    bigger = number * 2
    return bigger
Notice in the last example up above, though, we called double(age). Those don't match!!!
The thing is, your function doesn't care what the variable you send it is called. Whatever you send it, it will rename. It's like if someone adopted my cat Smushface, they might think calling her Petunia would be a little bit nicer (it wouldn't be, but I wouldn't do anything about it).
Here's an example with my favorite variable name potato_soup
Can you write Python code for it?

def exclaim(potato_soup):
    return potato_soup + "!!!!!!!!!!"

invitation = "I hope you can come to my wedding"
print(exclaim(invitation))

line = "I am sorry to hear you have the flu"
print(exclaim(line))


I am working on the file "09/.ipynb_checkpoints/09 - Functions-checkpoint.ipynb".
The first task is:
See? Functions make the world run.
One useful role they play is functions hide code that you wouldn't want to type a thousand times. For example, you might have used urlretrieve from urllib to download files from around the internet. If you didn't use urlretrieve you'd have to type all of this:
Can you write Python code for it?

def urlretrieve(url, filename=None, reporthook=None, data=None):
    url_type, path = splittype(url)

    with contextlib.closing(urlopen(url, data)) as fp:
        headers = fp.info()

        # Just return the local path and the "headers" for file://
        # URLs. No sense in performing a copy unless requested.
        if url_type == "file" and not filename:
            return os.path.normpath(path), headers

        # Handle temporary file setup.
        if filename:
            tfp = open(filename, 'wb')
        else:
            tfp = tempfile.NamedTemporaryFile(delete=False)
            filename = tfp.name
            _url_tempfiles.append(filename)

        with tfp:
            result = filename, headers
            bs = 1024*8
            size = -1
            read = 0
            blocknum = 0
            if "content-length" in headers:
                size = int(headers["Content-Length"])

            if reporthook:
                reporthook(blocknum, bs, size)

            while True:
                block = fp.read(bs)
                if not block:
                    break
                read += len(block)
                tfp.write(block)
                blocknum += 1
                if reporthook:
                    reporthook(blocknum, bs, size)

    if size >= 0 and read < size:
        raise ContentTooShortError(
            "retrieval incomplete: got only %i out of %i bytes"
            % (read, size), result)

    return result


I am working on the file "09/.ipynb_checkpoints/09 - Functions-checkpoint.ipynb".
The first task is:
invitation and line both get renamed to potato_soup inside of the function, so you can reuse the function with any variable of any name.
Let's say I have a function that does some intense calculations:
py
def sum_times_two(a, b):
    added = a + b
    return added * 2
To reiterate: a and b have nothing to do with the values outside of the function. You don't have to make variables called a and b and then send them to the function, the function takes care of that by itself. For example, the below examples are perfectly fine.
py
sum_times_two(2, 3)
r = 4
y = 7
sum_times_two(r, y)
When you're outside of the function, you almost never have to think about what's inside the function. You don't care about what variabels are called or anything. It's a magic box. Think about how you don't know what len looks like inside, or print, but you use them all of the time!
Why functions?
Two reasons to use functions, since maybe you'll ask:
Don't Repeat Yourself - If you find yourself writing the same code again and again, it's a good time to put that code into a function. len(...) is a function because Python people decided that you shouldn't have to write length-calculating code every time you wanted to see how many characters were in a string.
Code Modularity - sometimes it's just nice to organize your code. All of your parts that deal with counting dog names can go over here, and all of the stuff that has to do with boroughs goes over there. In the end it can make for more readable and maintanable code. (Maintainable code = code you can edit in the future without thinking real hard)
Those reasons probably don't mean much to you right now, and I sure don't blame you. Abstract programming concepts are just dumb abstract things until you actually start using them.
Let's say I wanted to greet someone and then tell them how long their name is, because I'm pedantic.
Can you write Python code for it?

name = "Nancy"
name_length = len(name)
print("Hello", name, "your name is", name_length, "letters long")

name = "Brick"
name_length = len(name)
print("Hello", name, "your name is", name_length, "letters long")

name = "Saint Augustine"
name_length = len(name)
print("Hello", name, "your name is", name_length, "letters long")


I am working on the file "09/.ipynb_checkpoints/09 - Functions-checkpoint.ipynb".
The first task is:
return
The role of a function is generally to do something and then send the result back to us. len sends us back the length of the string, requests.get sends us back the web page we requested.
py
def double(a):
    return a * 2
This is called the return statement. You don't have to send something back (print doesn't) but you usually want to.
Writing a custom function
Let's say we have some code that compares the number of boats you have to the number of cars you have.
python
if boat_count > car_count:
    print "Larger"
else:
    print "Smaller"
Simple, right? But unfortunately we're at a rich people convention where they're always comparing the number of boats to the number of cars to the number of planes etc etc etc. If we have to check again and again and again and again for all of those people and always print Larger or Smaller I'm sure we'd get bored of typing all that. So let's convert it to a function!
Let's give our function a name of size_comparison. Remember: We can name our functions whatever we want, as long as it's unique.
Our function will take two parameters. they're boat_coat and car_count above, but we want generic, re-usable names, so maybe like, uh, a and b?
For our function's return value, let's have it send back "Larger" or "Smaller".
Can you write Python code for it?

# Our cool function
def size_comparison(a, b):
    if a > b:
        return "Larger"
    else:
        return "Smaller"

print(size_comparison(4, 5.5))
print(size_comparison(65, 2))
print(size_comparison(34.2, 33))


I am working on the file "09/09 - Functions.ipynb".
The first task is:
1b. Driving Speed Part II
Now write a function called to_mpm that, when given miles per hour, computes the meters per minute.
Can you write Python code for it?


I am working on the file "09/09 - Functions.ipynb".
The first task is:
1c. Driving Speed Part III
Rewrite to_mpm to use the to_kmh function. D.R.Y.!
Can you write Python code for it?

def to_mpm(speeed):
    mpm = to_kmh * 16.6667
    return round(mpm)


I am working on the file "09/09 - Functions.ipynb".
The first task is:
2. Broken Function
The code below won't work. Why not?
Can you write Python code for it?


I am working on the file "09/09 - Functions.ipynb".
The first task is:
3. Data converter
We have a bunch of data in different formats, and we need to normalize it! The data looks like this:
python
var first = { 'measurement': 3.4, 'scale': 'kilometer' }
var second = { 'measurement': 9.1, 'scale': 'mile' }
var third = { 'measurement': 2.0, 'scale': 'meter' }
var fourth = { 'measurement': 9.0, 'scale': 'inches' }
Write a function called to_meters(...). When you send it a dictionary, have it examine the measurement and scale and return the adjusted value. For the values above, 3.4 kilometers should be 3400.0 meters, 9.1 miles should be around 14600, and 9 inches should be apprxoimately 0.23.
Can you write Python code for it?


I am working on the file "09/09 - Functions.ipynb".
The first task is:
Almost everything useful is a function. Python has a ton of other built-in functions!
Along with len,  a couple you might have seen are:

abs(...) takes a number and returns the absolute value of the number
int(...) takes a string or float and returns it as an integer
round(...) takes a float and returns a rounded version of it
sum(...) takes a list and returns the sum of all of its elements
max(...) takes a list and returns the largest of all of its selements
print(...) takes whatever you want to give it and displays it on the screen

Functions can also come from packages and libraries. The .get part of requests.get is a function, too!
And here, to prove it to you?
Can you write Python code for it?


I am working on the file "09/09 - Functions.ipynb".
The first task is:
Do you know how exhausted I got typing all of that out? And how it makes no sense at all? Luckily, functions save us: all of our code goes into one place so we don't have to repeat ourselves, and we can give it a descriptive name.
Can you write Python code for it?

def weird_greeting(name):
    name_length = len(name)
    print("Hello", name, "your name is", name_length, "letters long")

weird_greeting("Nancy")
weird_greeting("Brick")
weird_greeting("Saint Augustine")


I am working on the file "09/09 - Functions.ipynb".
The first task is:
See? Functions make the world run.
One useful role they play is functions hide code that you wouldn't want to type a thousand times. For example, you might have used urlretrieve from urllib to download files from around the internet. If you didn't use urlretrieve you'd have to type all of this:
Can you write Python code for it?


I am working on the file "09/09 - Functions.ipynb".
The first task is:
invitation and line both get renamed to potato_soup inside of the function, so you can reuse the function with any variable of any name.
Let's say I have a function that does some intense calculations:
py
def sum_times_two(a, b):
    added = a + b
    return added * 2
To reiterate: a and b have nothing to do with the values outside of the function. You don't have to make variables called a and b and then send them to the function, the function takes care of that by itself. For example, the below examples are perfectly fine.
py
sum_times_two(2, 3)
r = 4
y = 7
sum_times_two(r, y)
When you're outside of the function, you almost never have to think about what's inside the function. You don't care about what variabels are called or anything. It's a magic box. Think about how you don't know what len looks like inside, or print, but you use them all of the time!
Why functions?
Two reasons to use functions, since maybe you'll ask:
Don't Repeat Yourself - If you find yourself writing the same code again and again, it's a good time to put that code into a function. len(...) is a function because Python people decided that you shouldn't have to write length-calculating code every time you wanted to see how many characters were in a string.
Code Modularity - sometimes it's just nice to organize your code. All of your parts that deal with counting dog names can go over here, and all of the stuff that has to do with boroughs goes over there. In the end it can make for more readable and maintanable code. (Maintainable code = code you can edit in the future without thinking real hard)
Those reasons probably don't mean much to you right now, and I sure don't blame you. Abstract programming concepts are just dumb abstract things until you actually start using them.
Let's say I wanted to greet someone and then tell them how long their name is, because I'm pedantic.
Can you write Python code for it?


I am working on the file "09/09 - Functions.ipynb".
The first task is:
return
The role of a function is generally to do something and then send the result back to us. len sends us back the length of the string, requests.get sends us back the web page we requested.
py
def double(a):
    return a * 2
This is called the return statement. You don't have to send something back (print doesn't) but you usually want to.
Writing a custom function
Let's say we have some code that compares the number of boats you have to the number of cars you have.
python
if boat_count > car_count:
    print "Larger"
else:
    print "Smaller"
Simple, right? But unfortunately we're at a rich people convention where they're always comparing the number of boats to the number of cars to the number of planes etc etc etc. If we have to check again and again and again and again for all of those people and always print Larger or Smaller I'm sure we'd get bored of typing all that. So let's convert it to a function!
Let's give our function a name of size_comparison. Remember: We can name our functions whatever we want, as long as it's unique.
Our function will take two parameters. they're boat_coat and car_count above, but we want generic, re-usable names, so maybe like, uh, a and b?
For our function's return value, let's have it send back "Larger" or "Smaller".
Can you write Python code for it?


I am working on the file "09/Homework_9_Skinner_Functions_and_Earthquakes.ipynb".
The first task is:
PART FOUR: The other bits
Can you write Python code for it?

def other_mag_to_words(quake):
    mag_other = str(quake['mag'])
    return mag_other

def type_of_event(quake):
    return quake['type']

def date_other_events(quake):
    datestring = quake['time']
    date = dateutil.parser.parse(datestring)
    return date.strftime("%B %d")

def location_other_events(quake):
    location_string = str(quake['place'])
    return location_string

def other_events_to_sentence(quake):
    return "There was also a magnitude " + other_mag_to_words(quake) + \
    " " + type_of_event(quake) + " on " + date_other_events(quake) + " " \
    + location_other_events(quake) + "."

for x in earthquakes:
        if x['type'] != 'earthquake':
            print(other_events_to_sentence(x))


I am working on the file "09/Homework_9_Skinner_Functions_and_Earthquakes.ipynb".
The first task is:
PART TWO: Write the eq_to_sentence function
Can you write Python code for it?

def eq_to_sentence(quake):
    str_depth_to_words = str(depth_to_words(quake))
    str_mag_to_words = str(mag_to_words(quake))
    str_mag_numbers = str(mag_numbers(quake))
    return 'A ' + str_depth_to_words + " " + str_mag_to_words + " " + 'Magnitude' \
    + " " + str_mag_numbers + " earthquake was reported " + day(quake) + " " \
    + time(quake) + " on " + date(quake) + " " + location(quake) + "."
print(eq_to_sentence(earthquake))


I am working on the file "09/Homework_9_Skinner_Functions_and_Earthquakes_Graded.ipynb".
The first task is:
PART THREE: Doing it in bulk
Can you write Python code for it?

#from urllib.request import urlopen
#many_quakes = urlopen("http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/1.0_month.csv").read()

import pandas as pd
#quakes = pd.read_csv('http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/1.0_month.csv')

quakes = pd.read_csv('http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/1.0_month.csv')
earthquakes = quakes.to_dict('records')

#earthquakes

for x in earthquakes:
        if x['mag'] > 4.0:
            print(eq_to_sentence(x))


I am working on the file "09/Homework_9_Skinner_Functions_and_Earthquakes_Graded.ipynb".
The first task is:
PART TWO: Write the eq_to_sentence function
Can you write Python code for it?


I am working on the file "09/homework-9-schuetz.ipynb".
The first task is:
(If you really want to do it with pandas, it's for index, row in earthquakes_df.iterrows():)
<p>Loop through each earthquake, printing sentence descriptions for the ones that are above or equal to 4.0 on the Richter scale.
Can you write Python code for it?

for earthquake in earthquakes:
    if earthquake['mag'] > 4:
        print(eq_to_sentence(earthquake))