I am working on the file "0/Pandas Intro.ipynb".
The first task is:
Create Data
The data set will consist of 5 baby names and the number of births recorded for that year (1880).
Can you write Python code for it?

# The inital set of baby names and bith rates
names = ['Bob','Jessica','Mary','John','Mel']
births = [968, 155, 77, 578, 973]


I am working on the file "0/Pandas Intro.ipynb".
The first task is:
Even though this functions has many parameters, we will simply pass it the location of the text file.
Location = C:\Users\ENTER_USER_NAME.xy\startups\births1880.csv
Note: Depending on where you save your notebooks, you may need to modify the location above.
Can you write Python code for it?

Location = r'C:\Users\david\notebooks\update\births1880.csv'
df = pd.read_csv(Location)


I am working on the file "0/Pandas Intro.ipynb".
The first task is:
Get Data
To pull in the csv file, we will use the pandas function read_csv. Let us take a look at this function and what inputs it takes.
Can you write Python code for it?


I am working on the file "0/Pandas Intro.ipynb".
The first task is:
If we wanted to give the columns specific names, we would have to pass another paramter called names. We can also omit the header parameter.
Can you write Python code for it?

df = pd.read_csv(Location, names=['Names','Births'])
df


I am working on the file "0/Pandas Intro.ipynb".
The first task is:
Present Data
Here we can plot the Births column and label the graph to show the end user the highest point on the graph. In conjunction with the table, the end user has a clear picture that Mel is the most popular baby name in the data set.
plot() is a convinient attribute where pandas lets you painlessly plot the data in your dataframe. We learned how to find the maximum value of the Births column in the previous section. Now to find the actual baby name of the 973 value looks a bit tricky, so lets go over it.
Explain the pieces:
df['Names'] - This is the entire list of baby names, the entire Names column
df['Births'] - This is the entire list of Births in the year 1880, the entire Births column
df['Births'].max() - This is the maximum value found in the Births column
[df['Births'] == df['Births'].max()] IS EQUAL TO [Find all of the records in the Births column where it is equal to 973]
df['Names'][df['Births'] == df['Births'].max()] IS EQUAL TO Select all of the records in the Names column WHERE [The Births column is equal to 973]
An alternative way could have been to use the Sorted dataframe:
Sorted['Names'].head(1).value
The str() function simply converts an object into a string.
Can you write Python code for it?

# Create graph
df['Births'].plot()

# Maximum value in the data set
MaxValue = df['Births'].max()

# Name associated with the maximum value
MaxName = df['Names'][df['Births'] == df['Births'].max()].values

# Text to display on graph
Text = str(MaxValue) + " - " + MaxName

# Add text to graph
plt.annotate(Text, xy=(1, MaxValue), xytext=(8, 0),
                 xycoords=('axes fraction', 'data'), textcoords='offset points')

print("The most popular name")
df[df['Births'] == df['Births'].max()]


I am working on the file "0/Pandas Intro.ipynb".
The first task is:
The only parameters we will use is index and header. Setting these parameters to True will prevent the index and header names from being exported. Change the values of these parameters to get a better understanding of their use.
Can you write Python code for it?


I am working on the file "0/Pandas Intro.ipynb".
The first task is:
To merge these two lists together we will use the zip function.
Can you write Python code for it?

zip?

BabyDataSet = list(zip(names,births))
BabyDataSet


I am working on the file "0/Pandas Intro.ipynb".
The first task is:
We are basically done creating the data set. We now will use the pandas library to export this data set into a csv file.
df will be a DataFrame object. You can think of this object holding the contents of the BabyDataSet in a format similar to a sql table or an excel spreadsheet. Lets take a look below at the contents inside df.
Can you write Python code for it?

df = pd.DataFrame(data = BabyDataSet, columns=['Names', 'Births'])
df


I am working on the file "0/Pandas Intro.ipynb".
The first task is:
You can think of the numbers [0,1,2,3,4] as the row numbers in an Excel file. In pandas these are part of the index of the dataframe. You can think of the index as the primary key of a sql table with the exception that an index is allowed to have duplicates.
[Names, Births] can be though of as column headers similar to the ones found in an Excel spreadsheet or sql database.

Delete the csv file now that we are done using it.
Can you write Python code for it?


I am working on the file "0/Pandas.ipynb".
The first task is:
Finally, let's look at how health violations are distributed across the different score description bins. We'll use the same discretizing process used earlier to create the score descriptions.
Can you write Python code for it?

# Let us bin health violations using the cities quantizations
descriptions = ['Poor', 'Needs Improvement', 'Adequate', 'Good']
bins = [-1, 70, 85, 90, 100]

# copy the scores from our grouped DataFrame, DataFrames manipulate
# in place if we do not explicitly copy them.
scores = violation_table['Score'].copy()
violation_transform = violation_table.copy()

# built-in pandas funcion which assigns each data point in
# 'scores' to an interval in 'bins' with labels of 'descriptions'
discretized_scores = pd.cut(scores, bins ,labels=descriptions)
violation_transform["Scores"] = discretized_scores

grouped = violation_transform.groupby('Scores')


I am working on the file "0/Pandas.ipynb".
The first task is:
First, load the data into pandas dataframes
Can you write Python code for it?

# import pandas library which provides an R like environment for python.
# if you do not have it installed: sudo easy_install pandas.
import pandas as pd
import scipy as sp
from scipy import stats

# store relevant file paths in variables since we may use them frequently
businesses = 'businesses.csv'
inspections = 'inspections.csv'

# load each file into a Pandas DataFrame, pandas automatically converts the first line into a header for the columns
df_business = pd.read_csv(businesses)
df_inspection = pd.read_csv(inspections)


I am working on the file "0/Pandas.ipynb".
The first task is:
Let's plot a histogram for the violations data to see how the violations are distributed
Can you write Python code for it?

figure(figsize=[18,7])

violation_hist = violation_table['description'].value_counts().plot(kind="bar")


I am working on the file "0/Pandas.ipynb".
The first task is:
Now that we have a basic idea of the distribution, let us look at some more interesting statistics
Can you write Python code for it?

scores =  most_recent_inspection_results['Score']
mean = scores.mean()
median = scores.median()

# compute descriptive summary statistics of the inspection scores
summary = scores.describe()

mode = sp.stats.mode(scores)
skew = scores.skew()

# compute quantiles
ninety = scores.quantile(0.9)
eighty = scores.quantile(0.8)
seventy = scores.quantile(0.7)
sixty = scores.quantile(0.6)

print "Skew: " + str(skew)
print "90%: " + str(ninety)
print "80%: " + str(eighty)
print "70%: " + str(seventy)
print "60%: " + str(sixty)
print summary


I am working on the file "0/Pandas.ipynb".
The first task is:
Now that we have our joined data, we can start exploring it
We can start by grouping our data by business in order to find its most recent score for the inspections
Can you write Python code for it?


I am working on the file "0/Pandas.ipynb".
The first task is:
Now, we can inspect the first 10 rows of the DataFrame to ensure that the data is loaded properly and get an idea of the schema
Can you write Python code for it?


I am working on the file "0/Pandas.ipynb".
The first task is:
Propose Solutions
Since we have explored our data and have a better idea of its nature, we can begin to devise a plan to answer our questions. This is usually the most iterative part of the entire process: as we learn more about our data we modify our approach, and as modify our solutions we must re-examine our data.
Goals:

How does an individual restaurants' score compare to the whole/aggregate of SF?
Are SF's inspections better or worse than other cities?
If a restaurant has not yet been inspected, can we approximate/predict what score it will receive?

Solutions:

Collect summary statistics (mean, median, standard deviation, etc.) about distribution of scores.
Acquire data on inspection scores for other cities, compare distribution of cities.
Perform a linear regression on historic data on past inspections combined with scores from other 'similar' restaurants.

Collect Metrics
This is the step where derivative values are often calculated, including summary statistics, transformations on the data, and correlations.  There also is a bit of traditional data mining involved as most machine learning occurs in the solutions and metrics stages (in our formulation).  We could even go so far as to say that the results of predictive models are simply additional metrics: the probability of defaulting on a loan, the cluster a new product belongs in, or the score of a restaurant that hasn't been inspected yet.
The purpose of this part of the process is to calculate the information you need to begin evaluating and testing you solutions and hypotheses.
Given a mapping from score ranges to descriptions, we want to assign a discription to each business in our grouped DataFrame. We will use create description bins using the description ranges, then use a built-in Pandas function to assign each business to a bin.
Can you write Python code for it?

# first we need to discretize the numerical values, this can be
# thought of as converting a continuous variable into a categorical one.
descriptions = ['Poor', 'Needs Improvement', 'Adequate', 'Good']
bins = [-1, 70, 85, 90, 100]

# copy the scores from our grouped DataFrame, DataFrames manipulate
# in place if we do not explicitly copy them.
scores = most_recent_inspection_results['Score'].copy()
score_transform = most_recent_inspection_results.copy()

# built-in pandas function which assigns each data point in
# 'scores' to an interval in 'bins' with labels of 'descriptions'
discretized_scores = pd.cut(scores, bins ,labels=descriptions)


I am working on the file "0/Pandas.ipynb".
The first task is:
Split-Apply-Combine
A visual representation of how group-by, aggregate, and apply semantics work
We can bin the restaurants by scores to understand the distribution of inspections better.  Here we create a histogram to understand the distribution of scores better
Can you write Python code for it?

from scipy.stats import expon

# create a matplotlib figure with size [15,7]
figure(figsize=[15,7])

# pandas built-in histogram function automatically distributes and counts bin values
h = most_recent_inspection_results['Score'].hist(bins=100)

# create x-axis ticks of even numbers 0-100
xticks(np.arange(40, 100, 2))

# add a title to the current figure, our histogram
h.set_title("Histogram of Inspection Scores")


I am working on the file "0/Pandas.ipynb".
The first task is:
Then we can define a new DataFrame by applying a function to the old DataFrame.
Can you write Python code for it?

# a funtion that takes a DataFrame and returns the row with the newest date
def most_recent(df, column='date'):
    return df.sort_values(by=column)[-1:]

# input to most_recent is the DataFrame of each group, in this case
# all of the rows and columns for each business (grouped on business_id).
most_recent_inspection_results = grouped_business.apply(most_recent)

# We applied the most_recent function to extract the row
# of the DataFrame with the most recent inspection.
most_recent_inspection_results.head()


I am working on the file "0/Pandas.ipynb".
The first task is:
Then we'll plot a new violation histogram based on the filtered dataset.
Can you write Python code for it?


I am working on the file "0/Pandas.ipynb".
The first task is:
We can 'join' DataFrames just as we would database tables pandas uses a left-outer join by default, meaning that all
the records from the businesses will be present even if there
is not a corresponding row in the inspections.
Can you write Python code for it?

# join the two DataFrames on the 'business_id' column
big_table = df_business.merge(df_inspection, on='business_id')

# the joined DataFrame columns: frame1 columns + frame2 columns
# in our case it is the concatenation of df_business and df_inspection columns
print 'Business:\t' + str(df_business.columns) + '\n'
print 'Inspection:\t' + str(df_inspection.columns) + '\n'
print 'Big Table:\t' + str(big_table.columns)

# allows for row and column indexing succinctly
big_table.iloc[:10, :4]


I am working on the file "0/Pandas.ipynb".
The first task is:
We can now find the most common violations for each group by applying a function that counts and normalizes the offense types across the description groups.
Can you write Python code for it?

# a funtion that takes a DataFrame and returns the top violations
def common_offenses(df):
    return pd.DataFrame(df['description'].value_counts(normalize=True) * 100).head(10)

top_offenses = grouped.apply(common_offenses)
top_offenses


I am working on the file "0/Pandas.ipynb".
The first task is:
We can see that a majority of restaurants are adequate or good, according to the quantiles only 25% have scores less than 88.  While the histogram of the numeric scores gives us a more granular look at the data, it can be quite difficult to derive value from it.  Is an 86 a filthy/unhealthy restaurant or did it simply forget a few nuanced inspection rules?  The Score Legend provides us a mapping from a raw score to a meaningful value, similar to the scaling of standardized test raw scores.
If we are not satisfied with our evaluation, we need to iterate on our approach:

Do I need more/better data?
Do I need to try a different proposed solution?
Do I need to calculate different metrics?

We'll plot the histogram once again, this time marking the quantiles to provide a more meaningful representation of the scores.
Can you write Python code for it?

# create a matplotlib figure with size [15,7]
figure(figsize=[15,7])

# pandas built-in histogram function automatically distributes and counts bin values
h = most_recent_inspection_results['Score'].hist(bins=100)

# summary statistics vertical lines
axvline(x=mean,color='red',ls='solid', lw="3", label="mean")
axvline(x=median,color='green',ls='solid', lw="3", label="median")
axvline(x=mode[0][0],color='orange',ls='solid', lw="3", label="mode")

# 25th quantile
axvline(x=summary['25%'],color='maroon',ls='dashed', lw="3", label="25th")
axvspan(40, summary['25%'], facecolor="maroon", alpha=0.3)

# 75th quantile
axvline(x=summary['75%'],color='black',ls='dashed', lw="3", label="75th")
axvspan(40, summary['75%'], facecolor="yellow", alpha=0.3 )

# create x-axis ticks of even numbers 0-100
xticks(np.arange(40, 104, 2))

# add legend to graph
legend(loc=2)

# add a title to the current figure, our histogram
h.set_title("Histogram of Inspection Scores with Quantiles")

savefig('quantiles.png', bbox_inches=0, transparent=True)

print summary


I am working on the file "00 - Data Loading.ipynb".
The first task is:
Data is always a numpy array (or sparse matrix) of shape (n_samples, n_features)
Split the data to get going
Can you write Python code for it?

from sklearn.cross_validation import train_test_split
X_train, X_test, y_train, y_test = train_test_split(digits.data,
                                                    digits.target)


I am working on the file "00 Compute Items Mask.ipynb".
The first task is:
Compute and Plot All Items
Can you write Python code for it?

for item in items:
    for view in views:
        print(item + '_' + view)
        compute_and_plot(item,view)
        plt.show()


I am working on the file "00 Compute Items Mask.ipynb".
The first task is:
Compute and Save
Issues:
* make a version for items with holes, e.g.:
  * fiskars_scissors
  * duct_tape
  * ...
Can you write Python code for it?

import cv2, glob
from utils import imread_rgb, item_mask

def worker(item):
        folder = ITEM_FOLDER + '/' + item + '/'
        files = glob.glob(folder + '*.png')
        for filename in files:
            image_RGB = imread_rgb(filename)
            if not image_RGB is None:
                image_RGB = cv2.resize(image_RGB,(RESIZE_X,RESIZE_Y))
                mask = item_mask(image_RGB, BACKGROUND_THRESHOLD)
                cv2.imwrite(filename[:-4]+'_mask.pgm', mask)

%%time
from multiprocessing import Pool

print('Computing mask of images')
print('* resized to %d x %d' % (RESIZE_X,RESIZE_Y))
print('* background threshold %d' % (BACKGROUND_THRESHOLD))
pool_size = 6
pool = Pool(pool_size)
result = []
for item in items:
    result.append( pool.apply_async(worker, (item,)) )
pool.close()
pool.join()
for r in result:
     r.get()


I am working on the file "00 Compute Items Mask.ipynb".
The first task is:
Plot All Items
Can you write Python code for it?

for item in items:
    for view in views:
        print(item + '_' + view)
        load_and_plot(item,view)
        plt.show()


I am working on the file "00 Compute Items Mask.ipynb".
The first task is:
Top
Computing Test
Can you write Python code for it?

from matplotlib import pyplot as plt
%matplotlib inline

import cv2, numpy as np
from ipywidgets import interact
from utils import imread_rgb, item_mask

def compute_and_plot(item,view):
    prefix = ITEM_FOLDER + '/' + item + '/' + item + '_' + view
    filename = prefix + '.png'
    image_RGB = imread_rgb(filename)
    if not image_RGB is None:
        image_RGB = cv2.resize(image_RGB,(RESIZE_X,RESIZE_Y))
        mask = item_mask(image_RGB, BACKGROUND_THRESHOLD)
        plt.subplot(121); plt.imshow(image_RGB); plt.axis('off');
        plt.subplot(122); plt.imshow(mask,cmap='gray'); plt.axis('off')
        area = np.sum(mask==255)
        print('Area: %d pixels' % area)

views = ['top_01','top-side_01','top-side_02','bottom_01','bottom-side_01','bottom-side_02']
interact(compute_and_plot,item=items,view=views);


I am working on the file "00 Compute Items Mask.ipynb".
The first task is:
Top
Plot Item Files
Can you write Python code for it?

from matplotlib import pyplot as plt
%matplotlib inline

import cv2, numpy as np
from ipywidgets import interact
from utils import imread_rgb, imread_gray

def load_and_plot(item,view):
    prefix = ITEM_FOLDER + '/' + item + '/' + item + '_' + view
    filename = prefix + '.png'
    image_RGB = imread_rgb(filename)
    if not image_RGB is None:
        image_RGB = cv2.resize(image_RGB,(RESIZE_X,RESIZE_Y))
        plt.subplot(121); plt.imshow(image_RGB); plt.axis('off');
        filename = prefix + '_mask.pgm'
        mask = imread_gray(filename)
        if not mask is None:
            area = np.sum(mask==255)
            print('Area: %d' % area)
            plt.subplot(122); plt.imshow( mask,cmap='gray'); plt.axis('off');

views = ['top_01','top-side_01','top-side_02','bottom_01','bottom-side_01','bottom-side_02']
interact(load_and_plot,item=items,view=views);


I am working on the file "00 Compute Items Mask.ipynb".
The first task is:
Top
Statistics
Can you write Python code for it?

import cv2, glob, numpy as np
from utils import imread_gray

item_view = []
area = []
for item in items:
    folder = ITEM_FOLDER + '/' + item + '/'
    files = glob.glob(folder + '*_mask.pgm')
    for filename in files:
        mask = imread_gray(filename)
        if not mask is None:
            item_view.append(filename)
            area.append( np.sum(mask==255) )

from matplotlib import pyplot as plt
%matplotlib inline
plt.hist(area,bins=30); plt.ticklabel_format(style='sci', axis='x', scilimits=(0,0));

[(a, str(iv.split('/')[-1][:-9])) for a, iv in sorted(zip(area,item_view), reverse=True) if a>160000]

[(a, str(iv.split('/')[-1][:-9])) for a, iv in sorted(zip(area,item_view), reverse=True) if a<30000]


I am working on the file "00 Javascript In Notebooks.ipynb".
The first task is:
A better approach is to use require.js to load external modules, which is supported starting in IPython 2.0.  First, define the paths to the different modules:
Can you write Python code for it?

%%javascript
require.config({
    paths: {
        'three': "https://cdnjs.cloudflare.com/ajax/libs/three.js/r74/three",
    }
});


I am working on the file "00 Javascript In Notebooks.ipynb".
The first task is:
A lower-level approach is to implement a _repr_javascript_() object method that returns Javascript.  This lets you add Javascript rendering to any object:
Can you write Python code for it?

class HelloWorld():
    def _repr_javascript_(self):
        return "element.append('HelloWorld class');"

hw = HelloWorld()
hw


I am working on the file "00 Javascript In Notebooks.ipynb".
The first task is:
Because of this, the animation event loop will stop when the library is reloaded.  For example, executing the following cell will cause the rotating cube to stop:
Can you write Python code for it?

libs = ["https://cdnjs.cloudflare.com/ajax/libs/three.js/r74/three.js"]
Javascript("""console.log(THREE);""", lib=libs)


I am working on the file "00 Javascript In Notebooks.ipynb".
The first task is:
For example:
Can you write Python code for it?

code = JSCode("""
    function test() {
        console.log("test!");
    }

    function output(string) {
        element.append(string);
    }
""")


I am working on the file "00 Javascript In Notebooks.ipynb".
The first task is:
Generating Javascript from Python
You often want to generate Javascript from Python.
Here's a hack that takes a fragment of Javascript, finds all the functions that are declared, and generates bound Python methods for each function:
Can you write Python code for it?

import types
from functools import partial
import json

class JSCode(object):
    def __init__(self, code):
        self.code = code
        self.invokes = ""

        # Find all the function name(  lines in the Javasript code
        for line in code.split("\n"):
            line = line.strip().split()
            if line and line[0] == 'function':
                funcname = line[1].split("(")[0]

                # For each create a bound method to add to the list of invocations
                def invoke(self, *args):
                    argstr = ','.join([json.dumps(arg) for arg in args])
                    self.invokes += "%s(%s);\n" % (funcname, argstr)

                setattr(self, funcname, types.MethodType(invoke, self))

    def _repr_javascript_(self):
        """Return the code definitions and all invocations"""
        return self.code + self.invokes


I am working on the file "00 Javascript In Notebooks.ipynb".
The first task is:
Global Variables
It's sometimes handy to define a global variable or function that can be used in later Javascript cells.  Attributes added to the windows object are available as global variables:
Can you write Python code for it?

%%javascript
window.foovar = 42;

%%javascript
element.append(foovar);


I am working on the file "00 Javascript In Notebooks.ipynb".
The first task is:
Loading External Javascript Libraries
You often need to load other Javascript libraries before your code is run.  The lib argument to the Javascript() display object specifies required libraries:
Can you write Python code for it?

libs = ["https://cdnjs.cloudflare.com/ajax/libs/three.js/r74/three.js"]
Javascript("""element.append(THREE.REVISION);""", lib=libs)


I am working on the file "00 Javascript In Notebooks.ipynb".
The first task is:
Now, the code object has test() and output() methods corresponding to Javascript methods:
Can you write Python code for it?


I am working on the file "00-BestPractices/Decorator.ipynb".
The first task is:
Decorating function that accept Arguments
Can you write Python code for it?

# Template
def proxy(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

def trace(func):
    def wrapper(*args, **kwargs):
        print(f'TRACE: calling {func.__name__}() '
              f'with {args}, {kwargs}')

        original_result = func(*args, **kwargs)

        print(f'TRACE: {func.__name__}() '
              f'returned {original_result!r}')

        return original_result
    return wrapper

@trace
def say(name, line):
    return f'{name}: {line}'

say('Jane', 'Hello, World')


I am working on the file "00-BestPractices/Decorator.ipynb".
The first task is:
Decorator
Decorators are “wrappers”, which means that they let you execute code before and after the function they decorate without modifying the function itself.
Decorators allow you to define reusable building blocks that can change or extend the behavior of other functions. And they let you do that without permanently modifying the wrapped function itself. The function’s behavior changes only when it’s decorated.
In basic terms, a decorator is a callable that takes a callable as input and returns another callable.
The following function has that property and could be considered the simplest decorator one could possibly write:
Can you write Python code for it?

def null_decorator(func):
    print("I'm decorating")
    return func

def greet():
    return 'Hello!'

greeting= null_decorator(greet)
greeting()


I am working on the file "00-BestPractices/Decorator.ipynb".
The first task is:
Example HTML
Can you write Python code for it?

# The decorator to make it bold
def makebold(fn):
    # The new function the decorator returns
    def wrapper():
        # Insertion of some code before and after
        return "" + fn() + ""
    return wrapper

# The decorator to make it italic
def makeitalic(fn):
    # The new function the decorator returns
    def wrapper():
        # Insertion of some code before and after
        return "" + fn() + ""
    return wrapper

@makebold
@makeitalic
def say():
    return "hello"

say()


I am working on the file "00-BestPractices/Decorator.ipynb".
The first task is:
Example Make a sandwich
Can you write Python code for it?

def bread(func):
    def wrapper():
        print("</''''''\>")
        func()
        print("<\______/>")
    return wrapper

def ingredients(func):
    def wrapper():
        print("#tomatoes#")
        func()
        print("~salad~")
    return wrapper

@bread
@ingredients
def sandwich(food="--ham--"):
    print(food)

sandwich()


I am working on the file "00-BestPractices/Decorator.ipynb".
The first task is:
Function references
So a function can be:
1. assigned to a variable
2. defined in another function
That means that a function can return another function
Can you write Python code for it?

def getTalk(kind="shout"):

    # We define functions on the fly
    def shout(word="yes"):
        return word.capitalize()+"!"

    def whisper(word="yes") :
        return word.lower()+"...";

    # Then we return one of them
    if kind == "shout":
        # We don't use "()", we are not calling the function,
        # we are returning the function object
        return shout
    else:
        return whisper

getTalk()

talk = getTalk()

talk

talk()

getTalk()("HellO")


I am working on the file "00-BestPractices/Decorator.ipynb".
The first task is:
Functions inside another functions
Can you write Python code for it?

def talk():
    def whisper(word="yes"):
        return word.lower() + '...'

    print(whisper())

talk()

# But "whisper" DOES NOT EXIST outside "talk":
whisper()


I am working on the file "00-BestPractices/Decorator.ipynb".
The first task is:
More complex decorator
The decorator converts the result of the decoreted functin to uppercase letters
Can you write Python code for it?

def uppercase(func):
    def wrapper():
        original_result = func()
        modified_result = original_result.upper()
        return modified_result
    return wrapper

@uppercase
def greet():
    return 'Hello!'

greet()


I am working on the file "00-BestPractices/Decorator.ipynb".
The first task is:
Other Example
Can you write Python code for it?

### Hard coding
# A decorator is a function that expects ANOTHER function as parameter
def my_shiny_new_decorator(a_function_to_decorate):

    # Inside, the decorator defines a function on the fly: the wrapper.
    # This function is going to be wrapped around the original function
    # so it can execute code before and after it.
    def the_wrapper_around_the_original_function():

        # Put here the code you want to be executed BEFORE the original function is called
        print("Before the function runs")

        # Call the function here (using parentheses)
        a_function_to_decorate()

        # Put here the code you want to be executed AFTER the original function is called
        print("After the function runs")

    # At this point, "a_function_to_decorate" HAS NEVER BEEN EXECUTED.
    # We return the wrapper function we have just created.
    # The wrapper contains the function and the code to execute before and after. It’s ready to use!
    return the_wrapper_around_the_original_function

# Now imagine you create a function you don't want to ever touch again.
def a_stand_alone_function():
    print("I am a stand alone function, don't you dare modify me")

a_stand_alone_function()
#outputs: I am a stand alone function, don't you dare modify me

# Well, you can decorate it to extend its behavior.
# Just pass it to the decorator, it will wrap it dynamically in
# any code you want and return you a new function ready to be used:

a_stand_alone_function_decorated = my_shiny_new_decorator(a_stand_alone_function)
a_stand_alone_function_decorated()
#outputs:
#Before the function runs
#I am a stand alone function, don't you dare modify me
#After the function runs

@my_shiny_new_decorator
def another_stand_alone_function():
    print("Leave me alone")

another_stand_alone_function()


I am working on the file "00-BestPractices/Decorator.ipynb".
The first task is:
To understand decorators, you must first understand that:
1. functions are objects in Python
2. functions can be defined inside another function
Functions are objects
Can you write Python code for it?

def shout(word="yes"):
    return word.capitalize()+"!"

shout()

# As an object, you can assign the function to a variable like any other object
scream = shout

# Notice we don't use parentheses: we are not calling the function,
# we are putting the function "shout" into the variable "scream".
# It means you can then call "shout" from "scream"
scream

scream()

#You can also remove the old name 'shout', and the function
# will still be accessible from 'scream'
del shout
print(shout())

scream()


I am working on the file "00-BestPractices/Decorator.ipynb".
The first task is:
With getTalk() we return the shout function and then with ("HellO"), we pass the argument at shout function
If you can return a function, you can pass one as a parameter
Can you write Python code for it?

def doSomethingBefore(func):
    print("I do something before then I call the function you gave me")
    print(func())

doSomethingBefore(scream)


I am working on the file "00-BestPractices/Decorator.ipynb".
The first task is:
null_decorator is a callable (it’s a function), it takes another callable as its input, and it returns the same input callable without modifying it.
Can you write Python code for it?

@null_decorator
def greet():
    return 'Hello!'

greet()


I am working on the file "00-BestPractices/Iterator.ipynb".
The first task is:
iterable is an object that it can be used in iteration (with a for loop) by using iterator
iterator is an object that define how actually do the iteration: specifically what is the next element. That's why it must have __next__ method
To support iteration an object needs to implement the iterator protocol by providing the iter and next dunder methods.
Class-based iterators are only one way to write iterable objects in Python. Also consider generators and generator expressions.
Can you write Python code for it?

numbers = [1, 2, 3]
for n in numbers:
    print(n)

iterator=iter(numbers)

next(iterator)

next(iterator)

next(iterator)

next(iterator)


I am working on the file "00-IntroPython.ipynb".
The first task is:
Alguns métodos de listas:

.append
.extend
.sort
.remove
.reverse
.insert
Can you write Python code for it?

lista = list(range(10))
print(lista)

lista.append(10)
print(lista)

lista.insert(4, 11)
print(lista)

lista.sort()
print(lista)

lista.extend(['elemento extra'])
print(lista)


I am working on the file "00-IntroPython.ipynb".
The first task is:
Dicionários
Can you write Python code for it?


I am working on the file "00-IntroPython.ipynb".
The first task is:
Exemplo prático de ler dados ASCII em Python

Nossos dados estão organizados na pasta data/CTD.
São arquivos ASCII que podem ser abertos em qualquer editor de texto.
Leia o arquivo "leia_me.txt" para maiores descrições do formato dos dados.
Can you write Python code for it?

fname = './data/CTD/estacao08.dat'

with open(fname, 'r') as f:
    lines = f.readlines()
    data = lines[0].strip().split()[3:]
    depth = float(data[0])
    lat = float(data[1])
    lon = float(data[2])
    print("Prof.: %s m\nLongitude: %2.4f deg\nLatitude: %2.4f deg" %
          (depth, lon, lat))

import os
from glob import glob

lista = glob(os.path.join('data', 'CTD', '*.dat'))
lista

def break_lines(line):
    return [float(num) for num in line.strip().split()]

def get_topo(url='./data/etopo5.nc'):
    etopodata = Dataset(url)
    topoin = etopodata.variables['ROSE'][:]
    lons = etopodata.variables['ETOPO05_X'][:]
    lats = etopodata.variables['ETOPO05_Y'][:]
    topoin, lons = shiftgrid(180., topoin, lons, start=False)
    return topoin, lons, lats

depth, lat, lon = [], [], []

for fname in lista:
    with open(fname, 'r') as f:
        lines = f.readlines()
        d, la, lo = break_lines(lines[0])[3:]
        lon.append(lo)
        lat.append(la)
        depth.append(d)

import seawater as sw

dist, angle = sw.dist(lon, lat)

dist = np.r_[0, dist.cumsum()]

%matplotlib inline
import matplotlib.pyplot as plt

plt.plot(dist, depth)
plt.gca().invert_yaxis()


I am working on the file "00-IntroPython.ipynb".
The first task is:
Sets
Can you write Python code for it?

set(['joseph', 'maria', 'pedro', 'maria'])


I am working on the file "00-IntroPython.ipynb".
The first task is:
Slicing
Can you write Python code for it?


I am working on the file "00-IntroPython.ipynb".
The first task is:
String Methods
Can you write Python code for it?

text = "Um bom aluno sempre..."

text.lower(), text.split(), text.replace('aluno', 'estudante')

print('O valor de pi é {:.2f}'.format(np.pi))
print('O valor de pi é {:.6f}'.format(np.pi))
print('O valor de 10 x pi é {:0.2e}'.format(10*np.pi))


I am working on the file "00-IntroPython.ipynb".
The first task is:
Strings
Can you write Python code for it?

print('Texto simples com "aspas" duplas.\n')

print("'Texto simples com 'aspas' simples.\n")

print("""Texto complexo com quebra
de linha, aspas "duplas" e 'simples'.\n""")

print(r"Texto \n 'cru', ideal para Latex $^\circ$.")


I am working on the file "00-Introduction/Installation Check.ipynb".
The first task is:
NLP Models
In order to fully use the NLTK package for Natural Language Processing, we need to download a couple of language models that give Python extra instructions. For example, the 'punkt' model below tells Python how to break strings of text into individual words or sentences.
Running this cell will require a stable internet connection and perhaps a little patience. If it completes successfully, then it will print the word "True" at the bottom.
Can you write Python code for it?

nltk_data = ["punkt", "words", "stopwords", "averaged_perceptron_tagger", "maxent_ne_chunker", 'wordnet']
nltk.download(nltk_data)


I am working on the file "00-Introduction/Installation Check.ipynb".
The first task is:
Required Packages
If you installed Python through the Anaconda platform, then the packages below should already be installed on your hard drive and Python should be able to find them.
Can you write Python code for it?

import os
import numpy
import matplotlib
import pandas
import sklearn
import nltk
import gensim

print("Success!")


I am working on the file "00-Introduction/Installation_Check.ipynb".
The first task is:
As a quick, opening toy example to see Python in action, let's find all the present participles used in Jane Austen's Pride and Prejudice. There is a plain text file containing this book in this folder.
Part of the reason why people use Python to do work on human-language texts (natural language processing) is because it makes tasks like this relatively simple.
Can you write Python code for it?

# every line that starts with a hash is a comment
# the computer ignores these lines, they are meant to address a human reader
# here is some starter code to make sure everything is set up (don't worry about understanding everything here)
for line in open('../Data/Austen_PrideAndPrejudice.txt', encoding='utf-8'):
    for word in line.split():
        if word.endswith('ing'):
            print(word)


I am working on the file "00-Introduction/Installation_Check.ipynb".
The first task is:
Visualization¶
The code in the cell below is not Python but a direct instruction to the Jupyter Notebook. Any visualizations that we produce will appear within the notebook itself (as opposed to an external image processor).
Can you write Python code for it?


I am working on the file "00-initial-analysis.ipynb".
The first task is:
Artists
Can you write Python code for it?

artist_count = %sql SELECT * FROM filtered_tags WHERE category = 'a' LIMIT 100

list_count(artist_count)

plot_count(artist_count, 10, logx=False)


I am working on the file "00-initial-analysis.ipynb".
The first task is:
Filter out some more tags
There are some tags than can not be easily inferred from just looking on the images, that do not correspont with the use cases I have in mind or that train on useless features. Better to either remove such posts or just remove these tags.
I looked through the first 500 tags from the list above and took note of the following ones:

monochrome
comic
greyscale
alternate_costume
translated
artist_request
sketch
character_name
copyright_request
artist_name
cosplay
dated
signature
parody
twitter_username
copyright_name
alternate_hairstyle
no_humans
translation_request
english
gradient
crossover
doujinshi
genderswap
remodel_(kantai_collection)
game_cg
cover_page
official_art
scan
text
4koma
traditional_media
3d
photo
lowres

It might also be a good idea to remove the tags for series (category y) because I don't know why this would be useful. The same for artists but since these are not in the TOP 1000 it should be no problem either way.
Some more filtering
And now get a list of all the images without the bold marked tags from the last section.
Furthermore we need information about the distribution of the remaining tags etc. and we need to write them into some easily parsable format for further processing with our neural networks.
Can you write Python code for it?

%%sql
CREATE OR REPLACE TEMPORARY VIEW final_posts AS
SELECT *
FROM filtered_images
WHERE id NOT IN
    (SELECT DISTINCT post_id
     FROM tagged
     WHERE tag_id
     IN
        (SELECT id as tag_id FROM tags
         WHERE name = 'comic'
         OR name = 'no_humans'
         OR name = '4koma'
         OR name = 'photo'
         OR name = '3d'))
ORDER BY id ASC

final_posts = %sql SELECT id, rating, file_ext FROM final_posts

%%sql
SELECT tags.id, tags.name, tag_count.count
FROM
(
    SELECT tagged.tag_id, COUNT(tagged.post_id)
    FROM
    final_posts
    INNER JOIN
    tagged
    ON final_posts.id = tagged.post_id
    GROUP BY tagged.tag_id
    HAVING COUNT(tagged.post_id) >= 10000
) tag_count
INNER JOIN
tags
ON tag_count.tag_id = tags.id
WHERE tags.category = 'g'
ORDER BY tag_count.count DESC

final_tag_count = _

final_posts = np.array(list(map(tuple, final_posts)),
                       dtype=[("id", np.int32), ("rating", "<U1"), ("file_ext", "<U4")])
final_tag_count = np.array(list(map(tuple, final_tag_count)),
                           dtype=[("id", np.int32), ("name", '<U29'), ("count", np.int32)])

pd.DataFrame(final_posts).to_hdf("metadata.h5", "posts", mode="a", complevel=9, complib="bzip2")
pd.DataFrame(final_tag_count).to_hdf("metadata.h5", "tag_count", mode="a", complevel=9, complib="bzip2")


I am working on the file "00-initial-analysis.ipynb".
The first task is:
How much R18 content is there on danbooru?
There are three ratings:
* s:safe
* q:questionable
* e:explicit
Can you write Python code for it?

rating_distribution = %sql SELECT date_trunc('month', created_at) AS month, rating, COUNT(*) FROM posts GROUP BY month, rating ORDER BY month ASC, rating DESC
rating_distribution = np.array(rating_distribution).reshape((150,3,3))
rating_distribution = pd.DataFrame(rating_distribution[:,:,2], columns=rating_distribution[0,:,1], index=rating_distribution[:,0,0], dtype=np.int)

rating_distribution.plot.area()
plt.xlim(rating_distribution.index[0], rating_distribution.index[-1])
plt.legend(["safe", "questionable", "explicit"], title="Rating")
plt.xlabel("Date of upload")
plt.ylabel("Uploads per month")
plt.title("Distribution of uploads over time, grouped by rating")


I am working on the file "00-initial-analysis.ipynb".
The first task is:
How much tags
This is the first interesting thing.
Can you write Python code for it?

%%sql
CREATE OR REPLACE TEMPORARY VIEW filtered_tags AS
    (SELECT tags.name, tag_count.count, tags.category
    FROM
        (SELECT tag_id, COUNT(post_id)
         FROM tagged INNER JOIN filtered_images ON filtered_images.id = tagged.post_id
         GROUP BY tag_id) tag_count
    INNER JOIN
        tags ON tags.id = tag_count.tag_id
    ORDER BY tag_count.count DESC)


I am working on the file "00-initial-analysis.ipynb".
The first task is:
Popular anime characters
Can you write Python code for it?

character_count = %sql SELECT * FROM filtered_tags WHERE category = 'c' LIMIT 100

list_count(character_count)

plot_count(character_count, 10, logx=False)


I am working on the file "00-initial-analysis.ipynb".
The first task is:
Scores
First let's take a look at the ratings users can give to images. This is just for warming up since this is a feature that's not so much used on Danbooru.
Can you write Python code for it?

scores = %sql SELECT score, COUNT(*) FROM posts GROUP BY score ORDER BY score DESC

worst_post = %sql SELECT id FROM posts WHERE score = (SELECT MIN(score) FROM posts)

best_post = %sql SELECT id FROM posts WHERE score = (SELECT MAX(score) FROM posts)

pd_count = scores.DataFrame()["count"]
pd_count.index = scores.DataFrame()["score"]
pd_count.plot(logy=True)
plt.ylabel("count")
plt.title("Post score distribution")


I am working on the file "00-initial-analysis.ipynb".
The first task is:
Status
Now we take a look at the status of the posts since we'll most likely not use deleted or banned posts. We have the following categories:
is_pending, is_flagged, is_deleted, is_banned
Can you write Python code for it?

pending_posts = %sql SELECT id FROM posts WHERE is_pending = true
flagged_posts = %sql SELECT id FROM posts WHERE is_flagged = true
deleted_posts = %sql SELECT id FROM posts WHERE is_deleted = true
banned_posts = %sql SELECT id FROM posts WHERE is_banned = true

def single_column_to_set(column):
    return set(idx[0] for idx in column)
posts_with_status = map(single_column_to_set, (pending_posts, flagged_posts, deleted_posts, banned_posts))
posts_with_status = list(posts_with_status)

post_flag_count = np.array(list(map(len, posts_with_status)))
post_flag_count_log = post_flag_count.copy()
post_flag_count_log[post_flag_count == 0] = 1
post_flag_count_log = np.log10(post_flag_count_log)

rects = plt.bar(range(4), post_flag_count_log)
plt.xticks(range(4), ["pending", "flagged", "deleted", "banned"])
for idx, rect in enumerate(rects):
    height = rect.get_height()
    width = rect.get_width()
    plt.text(rect.get_x() + width / 2, height, post_flag_count[idx], va="bottom", ha="center")
plt.ylabel("count")
yticks = plt.yticks()[0][1:-1]
plt.yticks(yticks, map(lambda x: "$10^{:d}$".format(int(x)), yticks))
plt.title("Post status distribution")

print("There is a small intersection of deleted and banned posts: %d" % len(posts_with_status[2] & posts_with_status[3]))
print("What is even the difference between these two?")


I am working on the file "00-initial-analysis.ipynb".
The first task is:
Wow, there are a lot of deleted posts. I think I need to check them out manually and their reason for deletion.
But first create a view filtered_posts where flagged posts are removed. Furthermore remove children and remove posts where the aspect ration is not in 1:2 to 2:1.
Can you write Python code for it?

%%sql
CREATE OR REPLACE TEMPORARY VIEW filtered_posts AS
    (SELECT * FROM posts
     WHERE is_pending = false
     AND is_flagged = false
     AND is_deleted = false
     AND is_banned = false
     AND parent_id = 0
     AND (image_width >= 512 OR image_height >= 512)
     AND cast(image_width as double precision) / cast(image_height as double precision) BETWEEN 0.5 AND 2.0)

filtered_count = %sql SELECT COUNT(*) FROM filtered_posts

filtered_ext = %sql SELECT DISTINCT file_ext FROM filtered_posts

print("Now after filtering we have %d posts left with the following file extensions:" % filtered_count[0][0])
print(reduce(list_to_string, (x[0] for x in filtered_ext)))

%%sql
CREATE OR REPLACE TEMPORARY VIEW filtered_images AS
    (SELECT * FROM filtered_posts
     WHERE file_ext = 'jpeg'
     OR file_ext = 'jpg'
     OR file_ext = 'png')

%sql SELECT COUNT(*) FROM filtered_images


I am working on the file "00-tidyData/CleanDataHAR.ipynb".
The first task is:
As you see, the train set has 7352 observations, spread in 3 files
Merge the training and the test datasets
Can you write Python code for it?

allSub = pd.concat([trainSub, testSub], ignore_index=True)

allSub.shape


I am working on the file "00-tidyData/CleanDataHAR.ipynb".
The first task is:
But instead, from the data set creates a second, independent tidy data set with the average of each variable for each activity and each subject.
Can you write Python code for it?

grouped = all.groupby (['SubjectID', 'Activity'])


I am working on the file "00-tidyData/CleanDataHAR.ipynb".
The first task is:
It's also possible to add a column name after creation:
Can you write Python code for it?

testY.columns = ['ActivityID']

testY.head()


I am working on the file "00-tidyData/CleanDataHAR.ipynb".
The first task is:
Merge Subjects and X data frames by columns
Can you write Python code for it?

all = pd.concat([allX, allSub], axis=1)

all.shape

all.head()


I am working on the file "00-tidyData/CleanDataHAR.ipynb".
The first task is:
Now add allY to the new all dataframe
Can you write Python code for it?

allY.shape

all = pd.concat([all, allY], axis=1)

all.shape


I am working on the file "00-tidyData/CleanDataHAR.ipynb".
The first task is:
Now let's move to the train folder'
Can you write Python code for it?

trainSub = pd.read_table('UCI HAR Dataset/train/subject_train.txt', header=None, names=['SubjectID'])

trainSub.shape

trainX = pd.read_table('UCI HAR Dataset/train/X_train.txt', sep='\s+', header=None)

trainX.shape

trainY = pd.read_table('UCI HAR Dataset/train/y_train.txt', sep=' ', header=None, names=['ActivityID'])

trainY.shape


I am working on the file "00-tidyData/CleanDataHAR.ipynb".
The first task is:
Now the new data frame has 562 columns, ad the last column is the Subject ID.
same for allY: add it to main data frame as extra column but first map activity label to activity code
Map activity label to code
Can you write Python code for it?

allY.head()

act

allY.tail()

for i in act['ID']:
    activity = act[act['ID'] == i]['Activity']  # get activity cell given ID
    allY = allY.replace({i: activity.iloc[0]})  # replace this ID with activity string

allY.columns = ['Activity']

allY.head()

allY.tail()


I am working on the file "00-tidyData/CleanDataHAR.ipynb".
The first task is:
Read the files and load the datasets
Pre-requisite: the dataset archive has been downloaded and un-compressed in the same directory
Can you write Python code for it?

cat UCI\ HAR\ Dataset/activity_labels.txt

act = pd.read_table('UCI HAR Dataset/activity_labels.txt', header=None, sep=' ', names=('ID','Activity'))

act

type(act)

act.columns


I am working on the file "00-tidyData/CleanDataHAR.ipynb".
The first task is:
The file X_test requires to use as a separator a regular expression, because sometimes more blanks are used
Can you write Python code for it?


I am working on the file "00-tidyData/CleanDataHAR.ipynb".
The first task is:
features table has 561 observations of 2 variables: ID and sensor's name
Can you write Python code for it?

testSub = pd.read_table('UCI HAR Dataset/test/subject_test.txt', header=None, names=['SubjectID'])

testSub.shape

testSub.head()


I am working on the file "00.Setup and Primers.ipynb".
The first task is:
A NumPy primer
NumPy array dtypes and shapes
Can you write Python code for it?

import numpy as np

a = np.array([1, 2, 3])

a

b = np.array([[0, 2, 4], [1, 3, 5]])

b

b.shape

b.dtype

a.shape

a.dtype

np.zeros(5)

np.ones(shape=(3, 4), dtype=np.int32)


I am working on the file "00.Setup and Primers.ipynb".
The first task is:
Also see this fun "100 numpy exercises" site
A Matplotlib primer
Can you write Python code for it?

%matplotlib inline

import matplotlib.pyplot as plt

x = np.linspace(0, 2, 10)

x

plt.plot(x, 'o-');

plt.plot(x, x, 'o-', label='linear')
plt.plot(x, x ** 2, 'x-', label='quadratic')

plt.legend(loc='best')
plt.title('Linear vs Quadratic progression')
plt.xlabel('Input')
plt.ylabel('Output');

samples = np.random.normal(loc=1.0, scale=0.5, size=1000)

samples.shape

samples.dtype

samples[:30]

plt.hist(samples, bins=50);

samples_1 = np.random.normal(loc=1, scale=.5, size=10000)
samples_2 = np.random.standard_t(df=10, size=10000)

bins = np.linspace(-3, 3, 50)
_ = plt.hist(samples_1, bins=bins, alpha=0.5, label='samples 1')
_ = plt.hist(samples_2, bins=bins, alpha=0.5, label='samples 2')
plt.legend(loc='upper left');

plt.scatter(samples_1, samples_2, alpha=0.1)

samples_3 = np.random.normal(loc=2, scale=.5, size=10000)

fig = plt.figure()
ax1 = fig.add_subplot(111)
ax1.scatter(samples_1, samples_2, alpha=0.1, c='b', marker="s", label='first')
ax1.scatter(samples_3, samples_2, alpha=0.1, c='r', marker="o", label='second')
plt.show()


I am working on the file "00.Setup and Primers.ipynb".
The first task is:
Combining arrays
Can you write Python code for it?

a

b

d

np.concatenate([a, a, a])

np.vstack([a, b, d])

np.hstack([b, d])


I am working on the file "00.Setup and Primers.ipynb".
The first task is:
Finding the location of an installed package and its version:
Can you write Python code for it?


I am working on the file "00.Setup and Primers.ipynb".
The first task is:
Or check it all at once:  pip install version_information and check versions with a magic command.
Can you write Python code for it?

!pip install version_information

%load_ext version_information
%version_information numpy, scipy, matplotlib, pandas, tensorflow, sklearn, skflow


I am working on the file "00.Setup and Primers.ipynb".
The first task is:
Reshaping and inplace update
Can you write Python code for it?

e = np.arange(12)

e

f = e.reshape(3, 4)

f

e

e[5:] = 0

e

f


I am working on the file "001_dict_for_statistic_text.ipynb".
The first task is:
使用另外的一个defaultdict类，需要引入collections包
Can you write Python code for it?

from collections import defaultdict

sentence = "The ability to analyze data with Python is critical in data science Learn the basics and move on to create stunning visualizations"

word_dict = defaultdict(int)

for word in sentence.split():
    word_dict[word] +=1

print word_dict


I am working on the file "006 - Frank Underwood/schema-presentation/Schema presentation.ipynb".
The first task is:
How do we validate it?
Can you write Python code for it?

if data.get("operation") not in ["upload", "delete"]:
    raise SomeError("Operation not valid.")

try:
    timeout = int(data.get("timeout"))
except ValueError:
    raise SomeError("Timeout not a number.")

if not 0 < timeout <= 3600:
    raise SomeError("Timeout not up to one hour in the future.")

if data.get("md5") and not isinstance(data["md5"], str):
    raise SomeError("md5 is not a valid MD5 hash.")

if not isinstance(data.get("files"), dict):
    raise SomeError("files must be a dictionary.")

# etc


I am working on the file "006 - Frank Underwood/schema-presentation/Schema presentation.ipynb".
The first task is:
Tricks
Can you write Python code for it?

schema = Schema(And(int, fetch_user_by_id))

data = "/tmp/pythess"

try:
    print schema.validate(data)
except SchemaError as e:
    print(e)

schema = Schema(range(10))

data = [2, 4, 6, 2, 2, 2, 20]

try:
    print schema.validate(data)
except SchemaError as e:
    print(e)

schema = Schema({
    "shred_date": And(
        basestring,
        Use(ciso8601.parse_datetime_unaware),
        datetime.datetime,
        Use(lambda d: (d - datetime.datetime.now()).days),
        lambda d: 0 < d < 120,
        error="shred_date must be a valid future date string up to 120 days from now.")
})

data = {
    "shred_date": "2016-10-10T00:00:00Z",
}

try:
    print schema.validate(data)
except SchemaError as e:
    print(e)

operations = {"upload": "PUT", "delete": "DELETE", "replace": "POST"}
schema = Schema(And(Use(json.loads, error="Invalid JSON"), {
        "operation": And(lambda s: s in operations.keys(), Use(operations.get), error="Valid operations are: %s" % ", ".join(operations.keys())),
        "files": {And(basestring, lambda s: len(s) > 5, error="Filename must be a string longer than 5 characters."): {
            Optional("size"): And(int, lambda i: i > 0, error="Size must be a positive integer."),
            Optional("shred_date"): And(
                basestring,  # Make sure it's a string.
                Use(ciso8601.parse_datetime_unaware),  # Parse it into a date.
                datetime.datetime,  # Make sure it's a date now.
                lambda d: 0 < (d - datetime.datetime.now()).days < 120,  # Make sure it's in the future, up to 120 days.
                error="shred_date must be a valid future date string up to 120 days from now.")
            }}}))
data = """{
    "operation": "repklace",
    "files": {
        "file.nam": {
            "size": 100,
            "shred_date": "2016-01-01T00:00:00Z"
        }}}}"""
try:
    print schema.validate(data)
except SchemaError as e:
    print e


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
A potentially more frequent source of confusion is the fact that the algorithm used by fmod for computing modulo is also different from that used by %, so the sign of the result is different. mixed-sign inputs.
Exponents and Logarithms
Exponential growth curves appear in economics, physics, and other sciences. Python has a built-in exponentiation operator (“**”), but pow() can be useful when you need to pass a callable function as an argument.
Can you write Python code for it?


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
Alternate Representations
modf() takes a single floating point number and returns a tuple containing the fractional and whole number parts of the input value.
Can you write Python code for it?

import math

for i in range(6):
    print('{}/2 = {}'.format(i, math.modf(i/2.0)))


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
Both values are limited in precision only by the platform’s floating point C library.
Testing for exceptional values
Floating point calculations can result in two types of exceptional values. INF (“infinity”) appears when the double used to hold a floating point value overflows from a value with a large absolute value.
There are several reserved bit patterns, mostly those with all ones in the exponent field. These allow for tagging special cases as Not A Number—NaN. If there are all ones and the fraction is zero, the number is Infinite.
The IEEE standard specifies:
Inf = Inf, 7ff0000000000000 // -Inf = -Inf, fff0000000000000
NaN = NaN, fff8000000000000 // -NaN = NaN, 7ff8000000000000
Can you write Python code for it?


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
Comparing
Comparisons for floating point values can be error prone, with each step of the computation potentially introducing errors due to the numerical representation. The isclose() function uses a stable algorithm to minimize these errors and provide a way for relative as well as absolute comparisons. The formula used is equivalent to
abs(a-b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)
By default, isclose() uses relative comparison with the tolerance set to 1e-09, meaning that the difference between the values must be less than or equal to 1e-09 times the larger absolute value between a and b. Passing a keyword argument rel_tol to isclose() changes the tolerance. In this example, the values must be within 10% of each other.
The comparison between 0.1 and 0.09 fails because of the error representing 0.1.
Can you write Python code for it?


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
Computing the square roots of negative numbers requires complex numbers, which are not handled by math. Any attempt to calculate a square root of a negative value results in a ValueError.
There are two variations of log(). Given floating point representation and rounding errors the computed value produced by log(x, b) has limited accuracy, especially for some bases. log10() computes log(x, 10), using a more accurate algorithm than log().
Can you write Python code for it?


import math

print '{:2}  {:^12}  {:^20}  {:^20}  {:8}'.format('i', 'x', 'accurate', 'inaccurate', 'mismatch')
print '{:-^2}  {:-^12}  {:-^20}  {:-^20}  {:-^8}'.format('', '', '', '', '')

for i in range(0, 10):
    x = math.pow(10, i)
    accurate = math.log10(x)
    inaccurate = math.log(x, 10)
    match = '' if int(inaccurate) == i else '*'
    print '{:2d}  {:12.1f}  {:20.18f}  {:20.18f}  {:^5}'.format(i, x, accurate, inaccurate, match)


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
Converting to Integers
The math module includes three functions for converting floating point values to whole numbers. Each takes a different approach, and will be useful in different circumstances.
The simplest is trunc(), which truncates the digits following the decimal, leaving only the significant digits making up the whole number portion of the value. floor() converts its input to the largest preceding integer, and ceil() (ceiling) produces the largest integer following sequentially after the input value.
Can you write Python code for it?


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
From here we can learn, for instance:
Can you write Python code for it?


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
Positive and Negative Signs
The absolute value of number is its value without a sign. Use fabs() to calculate the absolute value of a floating point number.
Can you write Python code for it?

import math

print math.fabs(-1.1)
print math.fabs(-0.0)
print math.fabs(0.0)
print math.fabs(1.1)


import math

print(math.fabs(-1.1))
print(math.fabs(-0.0))
print(math.fabs(0.0))
print(math.fabs(1.1))


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
Raising 1 to any power always returns 1.0, as does raising any value to a power of 0.0. Most operations on the not-a-number value nan return nan. If the exponent is less than 1, pow() computes a root.
Since square roots (exponent of 1/2) are used so frequently, there is a separate function for computing them.
Can you write Python code for it?

import math

print(math.sqrt(9.0))
print(math.sqrt(3))
try:
    print(math.sqrt(-1))
except ValueError as err:
    print('Cannot compute sqrt(-1):', err)


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
Similarly, we can learn the limits of the IEEE 754 standard
Largest Real = 1.79769e+308, 7fefffffffffffff // -Largest Real = -1.79769e+308, ffefffffffffffff
Smallest Real = 2.22507e-308, 0010000000000000 // -Smallest Real = -2.22507e-308, 8010000000000000
Zero = 0, 0000000000000000 // -Zero = -0, 8000000000000000
eps = 2.22045e-16, 3cb0000000000000 // -eps = -2.22045e-16, bcb0000000000000
Interestingly, one could define an even larger constant (more about this below)
Can you write Python code for it?

infinity = float("inf")
infinity

infinity/10000


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
Special constants
Many math operations depend on special constants. math includes values for $\pi$ and $e$.
Can you write Python code for it?

import math

print 'π: %.30f' % math.pi
print 'e: %.30f' % math.e


import math

print ('π: %.30f' % math.pi)
print ('e: %.30f' % math.e)
print('nan: {:.30f}'.format(math.nan))
print('inf: {:.30f}'.format(math.inf))


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
The lines in the output with trailing * highlight the inaccurate values.
As with other special-case functions, the function exp() uses an algorithm that produces more accurate results than the general-purpose equivalent math.pow(math.e, x).
Can you write Python code for it?

import math

x = 2

fmt = '%.20f'
print(fmt % (math.e ** 2))
print(fmt % math.pow(math.e, 2))
print(fmt % math.exp(2))


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
The modulo operator (%) computes the remainder of a division expression (i.e., 5 % 2 = 1). The operator built into the language works well with integers but, as with so many other floating point operations, intermediate calculations cause representational issues that result in a loss of data. fmod() provides a more accurate implementation for floating point values.
Can you write Python code for it?


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
This discrepancy is caused by an implementation difference in the library used by C Python.
Division operations using infinite values are undefined. The result of dividing a number by infinity is NaN (“not a number”).
Can you write Python code for it?

import math

x = (10.0 ** 200) * (10.0 ** 200)
y = x/x

print('x =', x)
print('isnan(x) =', math.isnan(x))
print('y = x / x =', x/x)
print('y == nan =', y == float('nan'))
print('isnan(y) =', math.isnan(y))


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
To determine the sign of a value, either to give a set of values the same sign or simply for comparison, use copysign() to set the sign of a known good value. An extra function like copysign() is needed because comparing NaN and -NaN directly with other values does not work.
Can you write Python code for it?


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
To use a fixed or "absolute" tolerance, pass abs_tol instead of rel_tol.
For an absolute tolerance, the difference between the input values must be less than the tolerance given.
Can you write Python code for it?


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
When the exponent in this example grows large enough, the square of x no longer fits inside a double, and the value is recorded as infinite. Not all floating point overflows result in INF values, however. Calculating an exponent with floating point values, in particular, raises OverflowError instead of preserving the INF result.
Can you write Python code for it?

x = 10.0 ** 200

print 'x    =', x
print 'x*x  =', x*x
try:
    print 'x**2 =', x**2
except OverflowError, err:
    print err


I am working on the file "00_02_numbers_in_python.ipynb".
The first task is:
ldexp() is the inverse of frexp(). Using the same formula as frexp(), ldexp() takes the mantissa and exponent values as arguments and returns a floating point number.
Can you write Python code for it?


I am working on the file "00_Admin/CheatSheet/Jupyter Features CheatSheets.ipynb".
The first task is:
%load
Can you write Python code for it?

# loads content of a file into the cell
# %load ../sample_files/hello_world.py
if __name__ == "__main__":
  print("Hello World!")


I am working on the file "00_Admin/CheatSheet/Jupyter Features CheatSheets.ipynb".
The first task is:
%time & timesit
Can you write Python code for it?

# %time will give you information about a single run of the code in your cell.
%time
import time
for _ in range(1000):
  time.sleep(0.01)# sleep for 0.01 seconds

# %timeit uses the Python timeit module which runs a statement 100,000 times (by default) and then provides the mean of the fastest three times.
import numpy
%timeit numpy.random.normal(size=100)


I am working on the file "00_Admin/CheatSheet/Jupyter Features CheatSheets.ipynb".
The first task is:
Available Cell Magics
Can you write Python code for it?

%%!
%%HTML
%%SVG
%%bash
%%capture
%%debug
%%file
%%html
%%javascript
%%js
%%latex
%%perl
%%prun
%%pypy
%%python
%%python2
%%python3
%%ruby
%%script
%%sh
%%svg
%%sx
%%system
%%time
%%timeit
%%writefile

from IPython.display import Image
Image(filename='../sample_files/xkcd_estimation.png')

from IPython.display import Image
Image(url='http://python.org/images/python-logo.gif')

from IPython.display import YouTubeVideo
YouTubeVideo('1j_HxD4iLn8')

import antigravity


I am working on the file "00_Admin/CheatSheet/Jupyter Features CheatSheets.ipynb".
The first task is:
Avaliable line Magics
python
%alias
%alias_magic
%autocall
%automagic
%autosave
%bookmark
%cat
%cd
%clear
%colors
%config
%connect_info
%cp
%debug
%dhist
%dirs
%doctest_mode
%ed
%edit
%env            # %env: Set Environment Variables
%gui
%hist
%history
%killbgscripts
%ldir
%less
%lf
%lk
%ll
%load
%load_ext
%loadpy
%logoff
%logon
%logstart
%logstate
%logstop
%ls
%lsmagic
%lx
%macro
%magic
%man
%matplotlib
%mkdir
%more
%mv
%notebook
%page
%pastebin
%pdb
%pdef
%pdoc
%pfile
%pinfo
%pinfo2
%popd
%pprint
%precision
%profile
%prun
%psearch
%psource
%pushd
%pwd
%pycat
%pylab
%qtconsole
%quickref
%recall
%rehashx
%reload_ext
%rep
%rerun
%reset
%reset_selective
%rm
%rmdir
%run
%save
%sc
%set_env
%store
%sx
%system
%tb
%time        # time code execution
%timeit      # time code execution
%unalias
%unload_ext
%who
%who_ls
%whos
%xdel
%xmode
%env
Can you write Python code for it?

# lists all environment variables
%env
# Set environment variable NUM_THREADS
%env NUM_THREADS=4


I am working on the file "00_Admin/CheatSheet/Jupyter Features CheatSheets.ipynb".
The first task is:
Magic Commands
List all Magics
Can you write Python code for it?


I am working on the file "00_Admin/CheatSheet/Python Conversion CheatSheet.ipynb".
The first task is:
Format Mini Language
format_spec ::=  [[fill]align][sign][#][0][width][,][.precision][type]
fill        ::=  <any character>
align       ::=  "<" | ">" | "=" | "^"
sign        ::=  "+" | "-" | " "
width       ::=  integer
precision   ::=  integer
type        ::=  "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"
"{" [field_name] ["!" conversion] [":" format_spec] "}"
          /                  "r"|"s"                   \
         /               (r)epr   (s)tr                 \
arg_name                                                 \
| ("." attribute_name | "[" element_index "]")*           \
|        |                       |                         \
|     identifier         integer | index_string            |
|                                   (quotes                |
[identifier                          not required)         |
 |integer]                                                 |
                                                           |
 _________________________________________________________/ \________
/                                                                    \
      ":"
         [[fill]align][sign][#][0][width][,][.precision][type]
  [default]--> < left    +   |  |  (int)       (int)    b base 2
  [default --> > right  [-]  |  |                       c character
   for         ^ center " "  |  \                       d base 10
   numbers]    =             |   `zero padding          e exponent (e)
                             |                          E exponent (E)
                            use 0b,0o,0x                f fixed point
                             for 2  8 16                F ^^(same)^^
  b base 2     c character                 [default]--> g general (???)
  o base 8     s string                                 G general 2 (?)
  d base 10                                             n number (general 3)
  x base 16                                             o base 8
  X base 16                  ...
Can you write Python code for it?

var = 1234.56789
print("{:>6.5}".format(var))        # right aligned to 6 digits, precision 5 (5 numbers including point e.g. 0.123)

print("{:020}".format(var))         # filled up with 0 to 20 digits, 00000000001234.56789

print('{:{width}.{prec}f}'.format(var, width=5, prec=2))

print('{:_<10}'.format('test'))
print('{:_>10}'.format('test'))
print('{:_^10}'.format('test'))

print("0x{:0x}".format(int(var)))   # in hex small case, 0x4d2
print("0x{:0X}".format(int(var)))   # in hex small case, 0x4D2

print("{:04x}".format(int(var)))    # in hex small case, 0x04d2
print("{:04X}".format(int(var)))    # in hex small case, 0x04D2


I am working on the file "00_Admin/CheatSheet/Python Conversion CheatSheet.ipynb".
The first task is:
My hex2bin()
Can you write Python code for it?


I am working on the file "00_Admin/Features/Jupyter Debug.ipynb".
The first task is:
Interactive Python Console
When this line is hit a python console will be invoked
Can you write Python code for it?


I am working on the file "00_Admin/Features/Jupyter Debug.ipynb".
The first task is:
Pixie Debugger
Doesn't work in Jupyterlab because of unsupported Javascript output.
Can you write Python code for it?

import pixiedust

%%pixie_debugger

import random
def find_max (values):
    max = 0
    for val in values:
        if val > max:
            max = val
    return max
find_max(random.sample(range(100), 10))


I am working on the file "00_Admin/Features/Jupyter Debug.ipynb".
The first task is:
Tracer
Can you write Python code for it?

def test_debug(y):
    x = 10
    # One-liner to start the debugger here.
    from IPython.core.debugger import Tracer; Tracer()()
    x = x + y

    for i in range(10):
        x = x+i

    return x

test_debug(10)


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Compared to Matlab, the last element is not inclusive.
Computation with vectors is quite easy
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Edit mode and Command mode
Once you click on a notebook cell to select it, you may interact with it in two ways, which are called modes. Later on, when you are reviewing this material again, read more about this in Reference 1.
Edit mode:

We enter edit mode by pressing Enter or double-clicking on the cell.

We know we are in this mode when we see a green cell border and a prompt in the cell area.

When we are in edit mode, we can type into the cell, like a normal text editor.

Command mode:

We enter in command mode by pressing Esc or clicking outside the cell area.

We know we are in this mode when we see a grey cell border with a left blue margin.

In this mode, certain keys are mapped to shortcuts to help with
  common actions.

You can find a list of the shortcuts by selecting Help->Keyboard Shortcuts
from the notebook menu bar. You may want to leave this for later, and come back to it, but it becomes more helpful the more you use Jupyter.
How to shut down the kernel and exit
Closing the browser tab where you've been working on a notebook does not immediately "shut down" the compute kernel. So you sometimes need to do a little housekeeping.
Once you close a notebook, you will see in the main Jupyter app that your
notebook file has a green book symbol next to it. You should click in the box at the left of that symbol, and then click where it says Shutdown. You don't need to do this all the time, but if you have a lot of notebooks running, they will use resources in your machine.
Similarly, Jupyter is still running even after you close the tab that has the Jupyter dashboard open. To exit the Jupyter app, you should go to the terminal that you used to open Jupyter, and type [Ctrl] + [c] to exit.
Nbviewer
Nbviewer is a free web service that allows you to share static versions of hosted notebook files, as if they were a web page. If a notebook file is publicly available on the web, you can view it by entering its URL in the nbviewer web page, and hitting the Go! button. The notebook will be...
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Error! Why? Let's inspect what Python has to say and explore what is happening.
Python is a dynamic language, which means that you don't need to specify a type to invoke an existing object. The humorous nickname for this is "duck typing":
"If it looks like a duck, and quacks like a duck, then it's probably a duck."
In other words, a variable has a type, but we don't need to specify it. It will just behave like it's supposed to when we operate with it (it'll quack and walk like nature intended it to).
But sometimes you need to make sure you know the type of a variable. Thankfully, Python offers a function to find out the type of a variable: type().
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Functions
It is good coding practice to avoid repeating ourselves: we want to write code that is reusable, not only because it leads to less typing but also because it reduces errors. If you find yourself doing the same calculation multiple times, it's better to encapsulate it into a function.
A function is a compact collection of code that executes some action on its arguments.
Once defined, you can call a function as many times as you want. When we call a function, we execute all the code inside the function. The result of the execution depends on the definition of the function and on the values that are passed into it as arguments. Functions might or might not return values in their last operation.
The syntax for defining custom Python functions is:
python
def function_name(arg_1, arg_2, ...):
    '''
    docstring: description of the function
    '''
    <body of the function>
The docstring of a function is a message from the programmer documenting what he or she built. Docstrings should be descriptive and concise. They are important because they explain (or remind) the intended use of the function to the users. You can later access the docstring of a function using the function help() and passing the name of the function. If you are in a notebook, you can also prepend a question mark '?' before the name of the function and run the cell to display the information of a function.
Let 's try it!
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Let's do some arithmetic operations with our new variables:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Let's see an interesting case:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Matlab users should be cautioned, because the index of the first vector element is not 1 but 0, so you access the first element by
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Matrices
Similar to vectors we can access matrix elements. Again, I emphasize that the first index is 0 and the last element is not inclusive
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Note that the last element is not inclusive! Another significant difference compared to Matlab, so keep this in mind, when creating vectors:
Can you write Python code for it?

f = numpy.arange(0,11)
print(f)

g = numpy.arange(10,21,2)
print(g)


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Notice what we did above: we used the print() function with a string message, followed by a variable, and Python printed a useful combination of the message and the variable value. This is a pro tip! You want to print for humans. Let's now check the type of the new variables we just created above:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Notice, that the underlying NumPy functions use angles in radians instead of degrees as input parameters. However, if you prefer degrees, NumPy can also solve this problem
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Now, we can control the flow inside the FOR-loop with if-statements
Can you write Python code for it?

for i in range(1,4):
    if(i>=2 and i<3):
        print("i = ", i)


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Similar to Matlab you can access matrix rows via :, the second row of matrix M
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
So, let 's try some functions, like sine and cosine
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
String variables
In addition to name and value, Python variables have a type: the type of the value it refers to. For example, an integer value has type int, and a real number has type float. A string is a variable consisting of a sequence of characters marked by two quotes, and it has type str.
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
The operation above is called concatenation: chaining two strings together into one. Insteresting, eh? But look at this:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Time to Plot
You will love the Python library Matplotlib! You'll learn here about its module pyplot, which makes line plots.
We need some data to plot. Let's define a NumPy array, compute derived data using its square, cube and square root (element-wise), and plot these values with the original array in the x-axis.
Can you write Python code for it?

xarray = numpy.linspace(0, 2, 41)
print(xarray)

pow2 = xarray**2
pow3 = xarray**3
pow_half = numpy.sqrt(xarray)


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
To plot the resulting arrays as a function of the orginal one (xarray) in the x-axis, we need to import the module pyplot from Matplotlib.
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Vectors
We can define vectors in many different ways, e.g. as a NumPy array:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
We can also add and multiply complex number
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
We'll use the pyplot.plot() function, specifying the line color ('k' for black) and line style ('-', '--' and ':' for continuous, dashed and dotted line), and giving each line a label. Note that the values for color, linestyle and label are given in quotes.
Can you write Python code for it?

#Plot x^2
pyplot.plot(xarray, pow2, color='k', linestyle='-', label='square')
#Plot x^3
pyplot.plot(xarray, pow3, color='k', linestyle='--', label='cube')
#Plot sqrt(x)
pyplot.plot(xarray, pow_half, color='k', linestyle=':', label='square root')
#Plot the legends in the best location
pyplot.legend(loc='best')


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
What about complex numbers? With the imaginary number
$j^2 = -1$ you can define complex numbers
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
What element is accessed by the index -2
As in Matlab you can also access different ranges of vector elements
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
What happened? Isn't $9^{1/2} = 3$? (Raising to the power $1/2$ is the same as taking the square root.) Did Python get this wrong?
Compare with this:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
What if you try to "add" two strings?
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
Yes! The order of operations matters!
If you don't remember what we are talking about, review the Arithmetics/Order of operations. A frequent situation that exposes this is the following:
Variables and their type
Variables consist of two parts: a name and a value. When we want to give a variable its name and value, we use the equal sign: name = value. This is called an assignment. The name of the variable goes on the left and the value on the right.
The first thing to get used to is that the equal sign in an assignment has a different meaning than it has in Algebra! Think of it as an arrow pointing from name to value.
<img src="images/variables.png" style="width: 400px;"/>
We have many possibilities for variable names: they can be made up of upper and lowercase letters, underscores and digits… although digits cannot go on the front of the name. For example, valid variable names are:
python
    x
    x1
    X_2
    name_3
    NameLastname
Keep in mind, there are reserved words that you can't use; they are the special Python keywords.
OK. Let's assign some values to variables and do some operations with them:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
or new matrices
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/0_Jupyter_Python_short_intro.ipynb".
The first task is:
or the inverse
Can you write Python code for it?

A = numpy.matrix('1 2 3; 5 7 6; 1 4 6')
print(A)

A.I

A @ A.I


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
(3) If-elif-else statement:
Can you write Python code for it?

a = 3
b = 5

if a > b:
    print('a is bigger than b')
elif a < b:
    print('a is smaller than b')
else:
    print('a is equal to b')


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
A helpful way to visualize slices is to imagine that the indices point to the spaces between characters in the string. That way, when you write my_string[i], you would be referring to the "character to the right of i" (Reference 2).
Check out the diagram below. We start counting at zero; the letter 'g' is to the right of index 2. So if we want to grab the sub-string 'gin' from 'engineer', we need [start:end]=[2:5].
<img src="images/slicing.png" style="width: 400px;"/>
Try it yourself!
Can you write Python code for it?

# Define your string
eng_string = 'engineer'

# Grab 'gin'slice
eng_string[2:5]


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Another useful method is startswith(), to find out if a string starts with a certain character.
Later on in this lesson we'll see a more interesting example; but for now, let's just "check" if our string starts with the word 'great'.
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Challenge question:
— What is the value of the variable name after executing the for statement above? Discuss with your neighbor. (Confirm your guess in a code cell.)
Exercise:
Say we have a list of lists (a.k.a., a nested list), as follows:
Python
fullnames = [['sam','jones'], ['zoe','smith'],['joe','cheek'],['tom','perez'] ]
Write some code that creates two simple lists: one with the first names, another with the last names from the nested list above, but capitalized.
To start, you need to create two empty lists using the square brackets with nothing inside. We've done that for you below. Hint: Use the append() list method!
Can you write Python code for it?

fullnames = [ ['sam','jones'], ['zoe','smith'],['joe','cheek'],['tom','perez'] ]
firstnames = []
lastnames = []

# Write your code here


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Check the string variable to confirm that it didn't change (it still has the period at the end):
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Each element of new_list is, of course, another list.
As with strings, we access list elements with indices and slicing notation. The first element of new_list is the list of integers from 1 to 5, while the second element is the list of three fruit names.
Can you write Python code for it?

new_list[0]

new_list[1]

# Accessing the first two elements of the list fruits
fruits[0:2]


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Edit mode and Command mode
Once you click on a notebook cell to select it, you may interact with it in two ways, which are called modes. Later on, when you are reviewing this material again, read more about this in Reference 1.
Edit mode:

We enter edit mode by pressing Enter or double-clicking on the cell.

We know we are in this mode when we see a green cell border and a prompt in the cell area.

When we are in edit mode, we can type into the cell, like a normal text editor.

Command mode:

We enter in command mode by pressing Esc or clicking outside the cell area.

We know we are in this mode when we see a grey cell border with a left blue margin.

In this mode, certain keys are mapped to shortcuts to help with
  common actions.

You can find a list of the shortcuts by selecting Help->Keyboard Shortcuts
from the notebook menu bar. You may want to leave this for later, and come back to it, but it becomes more helpful the more you use Jupyter.
How to shut down the kernel and exit
Closing the browser tab where you've been working on a notebook does not immediately "shut down" the compute kernel. So you sometimes need to do a little housekeeping.
Once you close a notebook, you will see in the main Jupyter app that your
notebook file has a green book symbol next to it. You should click in the box at the left of that symbol, and then click where it says Shutdown. You don't need to do this all the time, but if you have a lot of notebooks running, they will use resources in your machine.
Similarly, Jupyter is still running even after you close the tab that has the Jupyter dashboard open. To exit the Jupyter app, you should go to the terminal that you used to open Jupyter, and type [Ctrl] + [c] to exit.
Nbviewer
Nbviewer is a free web service that allows you to share static versions of hosted notebook files, as if they were a web page. If a notebook file is publicly available on the web, you can view it by entering its URL in the nbviewer web page, and hitting the Go! button. The notebook will be...
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Exercise
Replace the last element of alist with something different.
Being able to modify elements in a list is a "property"  of Python lists; other Python objects we'll see later in the course also behave like this, but not all Python objects do. For example, you cannot modify elements in a a string. If we try, Python will complain.
Fine! Let's try it:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Exercises:

Define a string called 'banana' and print out the first and last 'a'.
Using the same string, grab the 2 possible slices that correspond to the word 'ana' and print them out.
Create your own slicing exercise and ask your classmates to give it a try (work in groups of 3).

The following lines contain the solutions; to reveal the answer, select the lines with the mouse:
Solution Exercise 1:
 b = 'banana'
 print(b[1])
 print(b[-1])
Solution Exercise 2:
 print(b[1:4])
 print(b[3:])
What else we can do with strings?
Python has many useful built-in functions for strings. You'll learn a few of them in this section. A technical detail: in Python, some functions are associated with a particular class of objects (e.g., strings). The word method is used in this case, and we have a new way to call them: the dot operator. It is a bit counter-intuitive in that the name of the method comes after the dot, while the name of the particular object it acts on comes first. Like this: mystring.method().
If you are curious about the many available methods for strings, go to the section "Built-in String Methods" in this tutorial.
Let's use a quote by Albert Einstein as a string and apply some useful string methods.
Can you write Python code for it?

AE_quote = "Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid."


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Exercises:

From the integers list, grab the slice [2, 3, 4] and then [4, 5].
Create your own list and design an exercise for grabbing slices, working with your classmates.

Adding elements to a list
We can add elements to a list using the append() method: it appends the object we pass into the existing list. For example, to add the element 6 to our integers list, we can do:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Exercises:

Use the count() method to count how many letters 'a' are in AE_quote?
Using the same method, how many isolated letters 'a' are in AE_quote?
Use the index() method to find the position of the words  'genius', 'judge' and 'tree' in AE_quote.
Using slice syntax, extract the words in exercise 3 from AE_quote.

Two more string methods turn out to be useful when you are working with texts and you need to clean, separate or categorize parts of the text.
Let's work with a different string, a quote by Eleanor Roosevelt:
Can you write Python code for it?

ER_quote = "   Great minds discuss ideas; average minds discuss events; small minds discuss people.  "


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Great! Now we know that my_string is eleven characters long. What happens if we enter this number as an index?
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
How do we know the index of the last element in the string?
Python has a built-in function called len() that gives the information about length of an object. Let's try it:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
If we want to know how many of those 'e' charachters are in the range [0:20], we do:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
In the example above, we used the len() function to calculate the length of the string 'fish', and we used the result to calculate the ending index. However, if the string is too long, having a line that calculates the length might be inconvenient or may make your code look messy. To avoid this, we can use the find() or index() methods to calculate the end position. In the 'fish' example, we could look for the index of the word 'by' (the word that follows 'fish') and subtract 1 from that index to get the index that corresponds to the space right after 'fish'. There are many ways to slice strings, only limited by your imagination!
Note:
Remember that the ending index is not inclusive, which is why we want the index of the space that follows the string 'fish'.
Can you write Python code for it?

idx_start = AE_quote.index('fish')
idx_end = AE_quote.index('by') - 1 # -1 to get the index off the space after 'fish'

AE_quote[idx_start:idx_end]


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
In the last list example, the last element of the list is actually another list. Yes! we can totally do that.
We can also assign lists to variable names, for example:
Can you write Python code for it?

integers = [1, 2, 3, 4, 5]
fruits = ['apple', 'banana', 'orange']

print(integers)

print(fruits)

new_list = [integers, fruits]

print(new_list)


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
It returns -1… but careful, that doesn't mean that the position is at the end of the original string! If we read the documentation, we confirm that a returned value of -1 indicates that the sub-string we are looking for is not in the string we are searching in.
A similar method is index(): it works like the find() method, but throws an error if the string we are searching for is not found.
Syntax:
str.index(substr, start, end)
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
It's important to mention that we don't need to match the character until we hit the white space.
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Let's check that the integer list now has a 6 at the end:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Let's replace it with the integer value 4:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Let's see what happens when we try to look for a string that is not in the quote.
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Let's split the ER_quote by a different character, a semicolon:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Let's supose you want to strip the period at the end; you could do the following:
ER_quote = ER_quote.strip('.')
But if we don't want to keep the changes in our string variable, we don't overwrite the variable as we did above. Let's just see how it looks:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
List membership
Checking for list membership in Python looks pretty close to plain English!
Syntax
To check if an element is in a list:
element in list
To check if an element is not in a list:
element not in list
Can you write Python code for it?

'strawberry' in fruits

'strawberry' not in fruits


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Note: The % represents a modulo operation: it gives the remainder from division of the first argument by the second
Tip: You can uncomment this following cell, and learn a good trick to ask the user to insert a number. You can use this instead of assigning a specific value to x above.
Can you write Python code for it?

#x = float(input('Insert your number: '))


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Notice that the string we defined above contains extra white spaces at the beginning and at the end. In this case, we did it on purpose, but bothersome extra spaces are often present when reading text from a file (perhaps due to paragraph indentation).
Strings have a method that allows us to get rid of those extra white spaces.
The strip() method returns a copy of the string in which all characters given as argument are stripped from the beginning and the end of the string.
Syntax:
str.strip([chars])
The default argument is the space character. For example, if we want to remove the white spaces in the ER_quote, and save the result back in ER_quote, we can do:
Can you write Python code for it?


I am working on the file "00_Intro_Python_Jupyter_notebooks/2_Jupyter_strings_and_lists.ipynb".
The first task is:
Oops. We have an error: why? We know that the length of my_string is eleven. But the integer 11 doesn't work as an index. If you expected to get the last element, it's because you forgot that Python starts counting at zero. Don't worry: it takes some getting used to.
The error message says that the index is out of range: this is because the index of the last element will always be: len(string) - 1. In our case, that number is 10. Let's try it out.
Can you write Python code for it?