I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
Data Types in Python
Building Blocks
To analyze data in Python, data has to be stored as some kind of data type. These data types form the structure of the data and make data easily accessible. Python's basic building blocks are:
* Numbers (integer, floating point, and complex)
* Booleans (true/false)
* Strings
* Lists
* Dictionaries
* Tuples
We will discuss the first four data types above as these are relevant for us.
Python is a dynamically typed language, meaning that - unlike in static languages such as VBA, C, Java etc. - you do not explicitly need to assign a data type to a variable. Python will do that for you. A few examples will explain this best:
Can you write Python code for it?

a = 42       # In VBA you would first have to define data type, only then the value: Dim a as integer; a = 42
b = 10.3     # VBA: Dim b as Double; b = 10.3
c = 'hello'  # VBA: Dim c as String; c = "hello"
d = True     # VBA: Dim d as Boolean; d = True

print('a: ', (a))
print('b: ', type(b))
print('c: ', type(c))
print('d: ', type(d))


I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
IMPORTANT NOTE:
When slicing and assigning part of an existing array to a new variable, the new variable will only hold a "view" but not a copy. This means, that if you change a value in the new array, the original array will also be changed. The idea behind this is to save memory. But fear not: with the ".copy()" method, you still can get a true copy.
Here a few corresponding examples for better understanding:
Can you write Python code for it?

ySub = y[:2, :2]
print(ySub)

ySub[0, 0] = 99
print(ySub, '\n')
print(y)

ySubCopy = y[:2, :2].copy()
ySubCopy[0, 0] = 33
print(ySubCopy, '\n')
print(y)


I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
Index: How to Access Elements
What might be a bit counterintuitive at the beginning is that Python's indexing starts at 0. Other than that, accessing the $i$'th element (starting at 0) of a list or a array is straight forward.
Can you write Python code for it?

print(e, '\n')  # List from above
print(x, '\n')  # One dimensional np array from above
print(y, '\n')  # Two dimensional np array from above

e[2]

e[2] * 2

x[5]

y[2, 0]  # Note again that [m, n] starts counting for both rows (m) as well as columns (n) from 0


I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
It is also possible to have more than one condition as the next example shows.
Can you write Python code for it?

x = 20

if x > 0:
    print(x, 'is positive')
elif x < 0:
    print(x, 'is negative')
else:
    print(x, 'is neither strictly positive nor strictly negative')


I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
Knowing the index we can also replace elements of an array:
Can you write Python code for it?


I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
Lists
Now let's look at lists. Lists are capable of combining multiple data types.
Can you write Python code for it?

e = ['Calynn', 'Dillon', '10.3', c, d]
print('e: ', type(e))
print([type(item) for item in e])


I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
Loops
"For" Loops
"For" loops iterate over a given sequence. They are very easy to implement as the following example shows. We start with an example and give some explanations afterwards.
For our example, let's assume you ought to sum up the integer values of a sequence from 10 to 1 with a loop. There are obviously more efficient ways of doing this but this serves well as an introductory example. From primary school we know the result is easily calculated as
$$
\begin{equation}
    \sum_{i=1}^n x_i = \dfrac{n (n+1)}{2} \qquad -> \qquad \dfrac{10 \cdot 11}{2} = 55
\end{equation}
$$
Can you write Python code for it?

seq = np.arange(start=10, stop=0, step=-1)
seq

seqSum = 0

for value in seq:
    seqSum = seqSum + value

seqSum


I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
Note that I do not need to use np.ones(shape=(2, 3), dtype='float32') to define the shape. It's ok to go with the short version as long as the order is correct, Python will understand. However, going with the explicit version helps to make your code more readable and I encourage students to follow this advice.
Each predefined function is documented in a help page. One can search for it by calling ?np.ones, np.ones? or help(np.ones). If it is not clear how the function is precisely called, it is best to use * (wildcard character) - as in np.*ne*?. This will list all functions in the NumPy library which contain 'ne' in their name. In our example np.ones? shows the following details:
Can you write Python code for it?


I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
Note that the third element in list e is set in quotation marks and thus Python interprets this as string.
NumPy Arrays
NumPy Arrays from Lists
For as useful list appear, its flexibility come at a high cost. Because each element contains not only the value itself but also information about the data type, storing data in list consumes a lot of memory. For this reason the Python community introduced NumPy (short for Numerical Python). Among other things, this package provides fixed-type arrays which are more efficient to store and operate on dense data than simple lists.
Fixed-type arrays are dense arrays of uniform type. Uniform here means that all entries of the array have the same data type, e.g. all are floating point numbers. We start by importing the NumPy package (following general convention we import this package under the alias name np) and create some simple NumPy arrays form Python lists.
Can you write Python code for it?

import numpy as np

# Integer array
np.array([3, 18, 12])

# Floating point array
np.array([3., 18, 12])


I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
Running Code in Jupyter
How did I execute this code section? If I only want to run a code cell, I select the cell and hit ctrl + enter. If you wish to run the entire notebook, go to Kernel (dropdown menu) and select "Restart & Run All". There are lots of shortcuts that let you handle a Jupyter notebook just from the keyboard. Press H on your keyboard (or go to Help/Keyboard Shortcuts) to see all of the shortcuts.
Simple Arithmetics
Simple arithmetic operations are straight forward:
Can you write Python code for it?

a = 2 + 4 - 8      # Addition & Subtraction
b = 6 * 7 / 3 - 2  # Multiplication & Division
c = 2**(1/2)       # Exponents & Square root
d = 10 % 3         # Modulus
e = 10 // 3        # Floor division

print(' a =', a, '\n',
      'b =', b, '\n',
      'c =', c, '\n',
      'd =', d, '\n',
      'e =', e)


I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
Similarly, we can explicitly set the data type:
Can you write Python code for it?

np.array([3, 18, 12], dtype='float32')

# Multidimensional arrays
np.array([range(i, i + 3) for i in [1, 2, 3]])


I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
To access the end of an array, you can also use negative indices:
Can you write Python code for it?


I am working on the file "0101_GettingStartedWithPython.ipynb".
The first task is:
We've seen above that we can define the data type for NumPy arrays. A list of available data types can be found in the NumPy documentation.
NumPy Arrays from Scratch
Sometimes it is helpful to create arrays from scratch. Here are some examples:
Can you write Python code for it?

# Integer array with 8 zeros
np.zeros(shape=8, dtype='int')

# 2x3 floating-point array filled with 1s
np.ones((2, 3), 'float32')


I am working on the file "0102_Pandas.ipynb".
The first task is:
.dropna() can not drop single values, but it can drop full columns or rows. For this, the method takes the parameter axis='rows' or axis='columns'.
Can you write Python code for it?

df.dropna()  # Similar to df.dropna(axis=0) or df.dropna(axis='rows')

df.dropna(axis='columns')  # similar to df.dropna(axis=1)


I am working on the file "0102_Pandas.ipynb".
The first task is:
As became obvious from above examples, pd.date_range by default generates daily timestamps. If you wish another frequency - such as monthly, annual, etc. - you add the freq= argument to your command.
Can you write Python code for it?


I am working on the file "0102_Pandas.ipynb".
The first task is:
Assume we wish to investigate ABB's stock a bit further. For this we need to slice the multiindex objcet df. Slicing multiindex objects is a bit trickier than doing the same on a simple data frame with only a single index. Below is an example how you slice the DataFrame based on a date range (here we take all dates) and on the ticker 'ABBN'. For further examples on how to slice multiindex objects, see here.
Can you write Python code for it?

# Assign ABB data to variable abb
idx = pd.IndexSlice
abb = df.loc[idx[:, ['ABBN']], idx[:]].copy()

# Add column indicating the quarter (but excl. year)
abb['Quarter'] = pd.PeriodIndex(abb.index.levels[0], freq='Q').strftime('Q%q')

# Add rolling 252d mean
abb['Rol252dMean'] = abb['Close'].rolling(window=252).mean()

# Add (annualized) historical rolling volatility
abb['Rol252dVol'] = abb['Return'].rolling(window=252).std() * np.sqrt(252)

# Drop Ticker Index as it is all ABB data now
abb = abb.reset_index(level=1, drop=True)

abb.tail(3)


I am working on the file "0102_Pandas.ipynb".
The first task is:
Because this was recognized as a source of confusion, Pandas introduced the loc and iloc attribute. loc allows indexing and slicing with the explicit index, iloc allows slicing that always references the implicit index.
Can you write Python code for it?

data.loc[1]

data.loc[1:3]

data.iloc[1]

data.iloc[1:3]


I am working on the file "0102_Pandas.ipynb".
The first task is:
Beyond the axis you can specify the parameter how and thresh.
For parameter how, default is set to how='any' which means that any row or column (depending on your selection) with NaN values will be dropped. Alternatively you could set it to how='all' to remove only those rows/columns where all entries are of sort NaN.
For parameter thresh, default is set to thresh=None. For example setting a thresh=3 will drop rows/columns with less than 3 non-null values.
Can you write Python code for it?

df['D'] = np.nan
df

df.dropna(axis='columns', how='all')

df.dropna(axis='rows', thresh=3)


I am working on the file "0102_Pandas.ipynb".
The first task is:
Combining Datasets
Concat and Append
Concatenating, appending, merging or joining data sets is a deep (and some say 'dull') topic. Anyone who had the pleasure of learning relational algebra can tell. Pandas has four functions that will do the job for you and of which you should have heard:

pd.append()
pd.concat()
pd.merge()
pd.join()

From time to time one of these functions will appear in this course. However, we will not properly discuss these functions in any detail. Unfortunately, doing it would consume too much time and would be beyond the purpose of this course. Nonetheless, I recommend to spend 15min in learning the basics by reading through Pandas' intuitive introduction which can be found here. It is kept fairly brief and with all the examples and visual representations, the functions are explained in a much better way than this tutorial could ever do.
Another valuable resource is of course again Jake VanderPlas' Data Science Handbook. You might find his explanations and examples very helpful and since it's freely available on GitHub, why not give it a shot. Here's the link (Combining Datasets).
Pandas Time Series
Timestamps and Periods
Pandas was developed with a focus on financial data. It thus does not surprise that Pandas has incorporated an easy and reliable way of handling datetime formats. Pandas handling of date/time objects improves on the existing package datetime and NumPy's numpy.datetime64 object and provides the necessary tools to efficiently handle datetimes.
The most basic kind of time series object in Pandas are pd.Series or pd.DataFrame objects indexed with timestamps.
Can you write Python code for it?

# Simple Time Stamps
print(pd.Timestamp(day=31, year=2021, month=12))
print(pd.Timestamp(2021, 12, 31, 13, 8))
print(pd.Timestamp('10.03.2021 17:32:15'))
print(pd.Timestamp('31.03.2021 17:32:15'))
print(pd.Timestamp('2021-03-31'))
print(pd.Timestamp('2000-07'))
print(pd.to_datetime("1st of August, 1992"))


I am working on the file "0102_Pandas.ipynb".
The first task is:
Date Ranges
The command pd.date_range generates an index with indicated length according to a particular frequency.
Can you write Python code for it?


I am working on the file "0102_Pandas.ipynb".
The first task is:
For further details on how to load/import data to Python check Pandas' tutorial on the topic.
Example: Working with Stock Data
In what follows it is shown how DataFrames are helpful in analyzing data. For that we will make use of the previously loaded stock data. The functions run below will not be introduced individually. But based on the annotated code, the comments in class and the output, the functions should easily be understood.
Can you write Python code for it?

# Sort df for dates (ascending)
df = df.sort_index(ascending=True)


I am working on the file "0102_Pandas.ipynb".
The first task is:
Having the data set up helps run further analysis. Note that plots will be discussed in a separate chapter and thus we will not get into it here.
Can you write Python code for it?

# Setup for plotting
%matplotlib inline
import matplotlib.pyplot as plt
plt.style.use('seaborn-whitegrid')
import seaborn as sns
import scipy as sp
import statsmodels.api as sm

abb[['Close', 'Rol252dMean']].plot(figsize=(16, 10));

abb[['Close', 'Rol252dVol', 'Return']].plot(subplots=True, figsize=(16, 10));


I am working on the file "0102_Pandas.ipynb".
The first task is:
Here's an overview of Pandas frequency codes:
| Code | Description        || Code  |  Description |
|:--------:|------------------------||:---------:|------------------|
| D        | Calendar day           ||  A        | Year end |
| B        | Business day           ||  AS       | Year start |
| W        | Weekly                 ||  BA       | Business year end |
| M        | Month end              ||  BAS      | Business year start |
| MS       | Month start            ||  H        | Hours |
| BM       | Business month end     ||  BH       | Business hours |
| BMS      | Business month start   ||  T        | Minutes |
| Q        | Quarter end            ||  S        | Seconds |
| QS       | Quarter start          ||  L        | Miliseconds |
| BQ       | Business quarter end   ||  U        | Microseconds |
| BQS      | Business quarter start ||  N        | Nanoseconds |
Beyond the above frequencies, Pandas has one more useful further option: "week of month". This enables you to
get dates like the third Friday of each month. Anyone dealing with options will recoginze these dates as the standard dates of monthly expiry.
Can you write Python code for it?

# Third Friday of each month between Jan-18 and Sep-18
pd.date_range('2021-01-01', '2021-09-01', freq='WOM-3FRI')


I am working on the file "0102_Pandas.ipynb".
The first task is:
IMPORTANT:
Notice that when using the explicit index, the final index is included. On the other hand, when you use the implicit index (i.e. data[0:2]), the final index is excluded.
Let's consider an example where the Series object has an explicit integer index.
Can you write Python code for it?

data = pd.Series(['a', 'b', 'c'], index=[1, 3, 5])
data

# Explicit index when indexing
data[1]

# Implicit index when slicing
data[1:3]


I am working on the file "0102_Pandas.ipynb".
The first task is:
If we wish to fill the blanks with another value than NaN, we can do so by using the add() method and specify the fill_value.
Can you write Python code for it?


I am working on the file "0102_Pandas.ipynb".
The first task is:
Importing Data
File Path
For most of this course we will use data stored in csv format which we will have to import. For this we can make use of Panda's read_csv() function. If you check the function's help page, you might be overwhelmed by all the possible parameter. Below follows an example which loads Swiss stock market data for the four companies Schindler, ABB, Georg Fischer, and Sulzer from a csv. To load it we necessarily need to specify the file name and its path.
Pandas will start looking from where your current python file or notebook is located. Python's working directory is set to where your current .py or .ipynb file is stored. If you have stored your file in a subfolder, one can simply preced the file name with the path: pd.read_csv('dataSubFolder/anotherSubFolder/data.csv). Given your file is located in another folder, one could either use an explicit path as in pd.read_csv('C:/Users/Username/Path/To/Your/Folder/data.csv') or you can move from your current directory to where your data is located with '..' For example pd.read_csv('../../../dataFolder/data.csv') will go 3 levels up and then into a dataFolder. If you wish to check the path of your current working directory, use !cd (Windows) or !pwd (Mac) to find out.
Can you write Python code for it?

# Print working directory (uncomment to run)
#!cd

pd.read_csv('Data/ShareData.csv', sep=',').head(3)

fl = pd.read_csv('Data/ShareData.csv', sep=',')
fl.dtypes


I am working on the file "0102_Pandas.ipynb".
The first task is:
Importing from Web Link
When data is updated on a regular basis, it is certainly more convenient to directly load a related file from an existing (static) url than to manually download it time and time again before running a script. Since Pandas version 0.19.2, pd.read_csv() is able to handle that. A simple example is provided below, where the csv file with historical closing prices of the 30 day volatility index on the SMI (VSMI) is downloaded.
Can you write Python code for it?

url = 'https://www.six-group.com/exchanges/downloads/indexdata/h_vsmi_30.csv'
data = pd.read_csv(url, sep=';', parse_dates=['Date'],
                   dayfirst=True, index_col='Date')
data.tail()


I am working on the file "0102_Pandas.ipynb".
The first task is:
Indexing and Selection
In the previous chapter we discussed how to access elements of NumPy arrays. In general, the same patterns are applicable to Pandas objects. However, there are a few quirks that we'll discuss to prevent confusion.
Can you write Python code for it?

data = pd.Series([0, 1, 2], index=['a', 'b', 'c'])

# Adding a float
data['d'] = 2.5
data

# Slicing by explicit index
data['a':'c']

# Slicing by implicit index
data[0:2]


I am working on the file "0102_Pandas.ipynb".
The first task is:
Indexing for DataFrame works similar to what we discussed thus far.
Can you write Python code for it?


I am working on the file "0102_Pandas.ipynb".
The first task is:
Let us say we want to have a statistical summary of the closing prices per share. We can use the .groupby() method to first split the values, select the closing prices, and then apply the .describe() method to have the desired summary.
Can you write Python code for it?

df.groupby(['Ticker'])['Close'].describe()

# Add a column with the returns
shft = len(df.index.levels[1])
df['Return'] = np.log(df['Close'] / df['Close'].shift(shft))

# Check for NA values
df.isnull().sum()


I am working on the file "0102_Pandas.ipynb".
The first task is:
Notice that pd.Timestamp('10.03.2020') is interpreted as 3rd of October 2018 while pd.Timestamp('31.03.2020') as 31st of March. Here it is important to realize that the default format is the American way of writing a date: 'mm.dd.yyyy'.
Besides pd.Timestamp Pandas does also have a function for periods: pd.Period. The difference is subtle: The former is for a specific point in time, the latter represents a bounded interval.
Can you write Python code for it?

# Time Periods
Q3 = pd.Period('2021-09', freq='Q')
Q3

M9 = pd.Period('2021-09', freq='M')
M9


I am working on the file "0102_Pandas.ipynb".
The first task is:
NumPy provides special functions which can deal with NaN values.
Can you write Python code for it?

print(val.sum(), val.min(), val.max())
print(np.nansum(val), np.nanmin(val), np.nanmax(val))


I am working on the file "0102_Pandas.ipynb".
The first task is:
Often the Shapiro Wilk test is used to check if values follow a normal distribution. The function sp.stats.shapiro()  tests the null hypothesis that the data was drawn from a normal distribution. If the p-value is very small, it means it is unlikely that the data came from a normal distribution.
Can you write Python code for it?

# Apply Shapiro-Wilk test
print('Shapiro Wilk Test:')
print('Test Statistic: ', sp.stats.shapiro(rets)[0])
print('p-Value:        ', sp.stats.shapiro(rets)[1])

# Plot the log-returns with a normal distribution
plt.hist(rets, bins=50, density=True, label='frequency')
plt.xlabel('log-returns')
plt.ylabel('frequency')
x = np.linspace(np.min(rets), np.max(rets))
plt.plot(x, sp.stats.norm.pdf(x, loc=np.mean(rets), scale=np.std(rets)),
         'r', lw=2.0, label='pdf')
plt.legend();


I am working on the file "0102_Pandas.ipynb".
The first task is:
One slicing option you might come across when studying python scripts is the .ix indexer. It is a hybrid of the two functions .loc and .iloc. However, the .ix indexer is deprecated in favor of the more strict .iloc and .loc and thus we won't discuss it here.
Index Alignment
Pandas will align indices in the process of performing operations for both Series as well as DataFrames. This proves to be very convenient when dealing with incomplete data.
Can you write Python code for it?

np.random.seed(1234)
A = pd.DataFrame(np.random.randint(low=10, high=99, size=(2,2)),
                 columns=['A', 'C'])
A

B = pd.DataFrame(np.random.randint(low=0, high=10, size=(3,3)),
                 columns=list('BAC'))
B

A + B


I am working on the file "0102_Pandas.ipynb".
The first task is:
Pandas Series
While a NumPy array has an implicitly defined index, Pandas allows for an explicitly defined index. That could mean strings or nonsequential indices. We first look at Pandas Series objects. These are one-dimensional arrays of indexed data. We can use lists or arrays to create it.
Can you write Python code for it?

data = pd.Series([0.25, 0.5, 0.25, 1])
data

data = pd.Series([0.25, 0.5, 0.25, 1], index=['w', 'x', 'y', 'z'])
data

# Item access works as we would expect
data['x']

# Another example
data = pd.Series([0.25, 0.5, 0.75, 1],
                 index=[2, 7, 4, 1])
data

data[2]

# Pandas series from NumPy array
vec = np.linspace(start=0.2, stop=1, num=5)
pd.Series(vec)

type(pd.Series(vec))


I am working on the file "0102_Pandas.ipynb".
The first task is:
The pd.Period function has specific properties such as start_time and end_time.
Can you write Python code for it?


I am working on the file "0102_Pandas.ipynb".
The first task is:
To select a subset, we can apply the same logic as shown before.
Can you write Python code for it?

# Fancy indexing
rng = pd.date_range(start='2021-02-28', end='2021-03-01')
ts[rng]

# Indexing by string
print(ts['20211231'])
print(ts['2021-06-30'])


I am working on the file "0102_Pandas.ipynb".
The first task is:
Ultimately, there are many more ways to create a DataFrame but we'll leave it with the above two examples. Reason being, as we will see later on, that data for a DataFrame is usually imported from a txt, csv or xls file. This process is fairly simple and will be discussed later on. Regarding DataFrames, check the excellent overview by Chris Moffit or see the help page pd.DataFrame? for more examples on how to create a DataFrame.
As we would expect, each DataFrame has some callable attributes.
Can you write Python code for it?

print(comps.index)
print(comps.size, comps.shape, comps.ndim)


I am working on the file "0102_Pandas.ipynb".
The first task is:
We can also use dot notation to access a column.
Can you write Python code for it?


I am working on the file "0102_Pandas.ipynb".
The first task is:
While NumPy only accepts np.nan, Pandas is also able do handle None as input. Yet internally, Pandas will convert None values to NaN.
Can you write Python code for it?

seq = pd.Series([1, np.nan, 2, None])
seq


I am working on the file "0103_Plotting.ipynb".
The first task is:
An alternative approach emerging is the plt.subplot_mosaic function. It offers a suprisingly simple way of arranging plots.
Can you write Python code for it?

figure_mosaic = """
AABC
AADE
"""

fig, axes = plt.subplot_mosaic(mosaic=figure_mosaic, figsize=(11, 5))
axes["B"].plot(x, y1)
axes["C"].plot(x, y2)
axes["D"].scatter(x, y1)
axes["E"].hist(y1, bins=50, density=True, alpha=0.5, edgecolor='blue')

axes["A"].text(0.1, 0.9, "A", color='blue')
axes["A"].text(0.5, 0.5, "Easy, not!?", ha="center", va="center", size=38)
axes["B"].text(1, 0.35, "B", color='blue')
axes["C"].text(1, 0.9, "C", color='blue')
axes["D"].text(1, 0.35, "D", color='blue')
axes["E"].text(0.35, 35, "E", color='blue');


I am working on the file "0103_Plotting.ipynb".
The first task is:
As shown in the above example, the figure can again be adjusted to the user's liking. Available arguments are listed here. alpha=0.5 adjusts the density of the bar color. With density=True the counts will be normalized to form a probability density. These two arguments are helpful if one wishes to combine two histograms as shown in below example where the histogram displays a standard normal pdf combined with its cdf.
Can you write Python code for it?

# RV from a normal distribution
norm = np.random.normal(size=5000)

plt.hist(norm, bins=50, histtype='stepfilled', alpha=0.3, density=True, cumulative=True)
plt.hist(norm, bins=50, histtype='stepfilled', alpha=0.3, density=True);


I am working on the file "0103_Plotting.ipynb".
The first task is:
Axes, Grids and Labels
Per default, matplotlib sets the axis based on the plotted data. However, sometimes it is desirable to have better control of the axis and not let matplotlib's default settings rule the axis. Adjusting axis limits is done with plt.xlim() and plt.ylim() methods. The two input values to these methods will set limits to the axis. Alternatively, the plt.axis() method allows for further control. For example plt.axis('tight') will automatically tighten the bounds around the current plot (which is sometimes desireable when you want to use your plot in a report/article).
By setting the style to plt.style.use('seaborn-whitegrid') each plot automatically adds a grid. Python's default style does not come with a grid. In case you wish to add/adjust a grid to the default style, you do this with the plt.grid() method. Details on the valid arguments for the plt.grid() can be found here.
Title, axis and labels are added with plt.xlabel(), plt.ylabel(), plt.title().
Can you write Python code for it?

plt.figure(figsize=(8, 4))  # Adjusting figure size (width, height)
plt.plot(x, y)
plt.title("Random Walk with $X \sim N(0, 1)$")
plt.xlabel('step')
plt.ylabel('value')
plt.xlim(-1, 11)
plt.ylim(-2, 2);


I am working on the file "0103_Plotting.ipynb".
The first task is:
Below is an example one could use if one wishes to add a Kernel Density Estimation (KDE) to a histogram. Further information regarding the problem of bin-size when using histograms, and KDE estimation you can find here or here.
Can you write Python code for it?

kde = sps.gaussian_kde(norm)
xx = np.linspace(-4, 4, 1000)

fig, ax = plt.subplots(figsize=(8, 6))
ax.hist(norm, bins=50, histtype='stepfilled', alpha=0.3, density=True)
ax.plot(xx, kde(xx));


I am working on the file "0103_Plotting.ipynb".
The first task is:
Histogram
Histograms are often a great first step in understanding the distribution of a given dataset. To plot a histogram Matplotlib offers the plt.hist() command. Let's show a simple example:
Can you write Python code for it?

# RV from a gamma distribution
gamma = np.random.gamma(size=5000, shape=3, scale=2)

plt.hist(gamma, bins=50, density=True,
         alpha=0.5, edgecolor='blue');


I am working on the file "0103_Plotting.ipynb".
The first task is:
Histogram: Share Returns
Literature often assumes returns to be normally distributed. In a hypothetical example, we plot the annual returns of the 5'000 simulated portfolios in a histogram and combine it with a kernel density estimate and a normal distribution (given mean, sd of data at hand).
Can you write Python code for it?

# Import Seaborn package
import seaborn as sns

# Plot histogram with kde and normal fit given data
sns.distplot(pfrets, bins=50, norm_hist=True, fit=sps.norm,
             hist_kws={'label':'frequency', 'alpha':0.8, 'edgecolor':'k'},
             kde_kws={'label':'kde', 'color':'b'},
             fit_kws={'label':'Normal pdf', 'alpha':0.5, 'color':'r'})
plt.legend(fontsize=16);


I am working on the file "0103_Plotting.ipynb".
The first task is:
In order to change the figure and axes you could use the plt.gcf() (get current figure) and plt.gca() (get current axes) routines.
Object-Oriented Interface
And here's the same plot using this style of plotting:
Can you write Python code for it?

# Create grid of plots
# ax will be array of two Axes objects
fig, ax = plt.subplots(2)
ax[0].plot(x, np.sin(x))
ax[1].plot(x, np.cos(x));


I am working on the file "0103_Plotting.ipynb".
The first task is:
In the object-oriented style you directly work on the figure and axes objects. Sometimes, if you are asked to draw complicated figures, this approach might be advisable as it provides easier control over the figure and axes objects. But in general, these two interface styles provide the same functionalities - the differences are of technical nature. Chosing one approach over the other boils down to preference.
In this notebook we will primarily use the MATLAB style but you should be aware of both, especially when searching the web for an answer to a specific question you are facing in plotting a figure.
Line Plots
Simple Plot
The most basic plot is the visualization of some function $f(x)$. For simplicity and as a toy example, let the function be the periodic $\sin(x)$. We wish to plot $f(x)$ for interval $x \in [0, 10]$.
Can you write Python code for it?

x = np.linspace(0, 10, 100)

plt.plot(x, np.sin(x));


I am working on the file "0103_Plotting.ipynb".
The first task is:
Line Colors and Styles
The plt.plot() function takes additional arguments which can be used to adjust the plot style. The color keyword takes a string argument and can be specified in a variety of ways:
* Specify color by name (red, green, blue, cyan, magenta, yellow, black; see here)
* Short color code (r, g, b, c, m, y, k)
* Grayscale between 0 and 1
* Hex code (RRGGBB from 00 to FF)
* RGB tuple, values $\in [0, 1]$
* all HTML color names (see here)
Can you write Python code for it?

plt.plot(x, x + 0, color='blue')
plt.plot(x, x + 1, color='g')
plt.plot(x, x + 2, color='0.75')
plt.plot(x, x + 3, color='#FFDD44')
plt.plot(x, x + 4, color=(1.0,0.2,0.3))
plt.plot(x, x + 5, color='chartreuse');


I am working on the file "0103_Plotting.ipynb".
The first task is:
Matplotlib will automatically cycle through a set of default colors if no color is specified. By defining a plot style such as plt.style.use('seaborn-whitegrid') the default colors are taken from the 'seaborn-whitegrid'-style file.
To adjust the linestyle, specify the respective style either with the code or the string. Available are:
| Code      | keyword         |
|:-------------:|---------------------|
| '-'           | 'solid' (default) |
| '--'          | 'dashed'            |
| '-.'          | 'dashdot'           |
| ':'           | 'dotted'            |
If you wish to keep your code concise you can combine the linestyle code with the color in a single string.
Can you write Python code for it?

plt.plot(x, x + 0, '-g')   # solid green
plt.plot(x, x + 1, '--c')  # dashed cyan
plt.plot(x, x + 2, linestyle='-.')  # dashdot
plt.plot(x, x + 3, ls='dotted');  # dotted red


I am working on the file "0103_Plotting.ipynb".
The first task is:
Matplotlib's Two Interfaces
What might be confusing to some is that matplotlib is built to handle two different plotting interfaces: one which follows the MATLAB-style and a more powerful object-oriented interface. Two examples will show the differences.
MATLAB-Style Interface
Can you write Python code for it?

# x-axis range
x = np.linspace(0, 10, 100)

# Create first of two panels
plt.subplot(2,1,1)  # (row, col, panel no.)
plt.plot(x, np.sin(x))

# Create second of two panels
plt.subplot(2,1,2)
plt.plot(x, np.cos(x));


I am working on the file "0103_Plotting.ipynb".
The first task is:
Now that we have the data in a proper format, we use Seaborn's lmplot() function to plot debt against gdp growth. The lmplot() adds a linear regression to the data, incl. confidence bands (here set to 95%; based on 100 bootstrap resamples).
Can you write Python code for it?

# Draw scatterplot incl. linear regression
# ci=95 and n_boot=100 are default values, thus
# these are just shown for better understanding
fig = sns.lmplot(x='GDP growth', y='Debt', hue='Country', data=df,
                 truncate=True, height=10, ci=95, n_boot=100)

# Add more informative axis labels
fig.set_axis_labels('GDP Growth (in %)', 'Central Gvmt. Debt (total, in % of GDP)');


I am working on the file "0103_Plotting.ipynb".
The first task is:
Scatter Plot
It is often said that central government debt is positively correlated with GDP growth. We could investigate this (on a superficial scale, of course) by plotting the data against each other in a scatter plot. In below chart we use data downloaded from the world bank for Switzerland, the United Kingdom, and the USA.
In order to download the data, we need to know the indicators in the World Bank's database. The best way to find these is to go to the World Bank's data catalogue and use their search engine. Once you've found what you were looking for, you will find the indicator in the url. For GDP growth it is "NY.GDP.MKTP.KD.ZG"  and "GC.DOD.TOTL.GD.ZS" for central government debt (total, in % of GDP).
Can you write Python code for it?

# Import worldbank functions as wb
from pandas_datareader import wb

# Set parameter and download data
ctry = ['US', 'GBR', 'CH']
ind = ['NY.GDP.MKTP.KD.ZG', 'GC.DOD.TOTL.GD.ZS']
df = wb.download(indicator=ind, country=ctry, start=1990, end=2018)

# Drop (multi)index and format columns
df.reset_index(inplace=True)
df.columns = ['Country', 'Year', 'GDP growth', 'Debt']
df.head()


I am working on the file "0103_Plotting.ipynb".
The first task is:
Scatter Plots
Above we discussed the plt.plot() method to draw line plots. The same function is capable of drawing simple scatter plots too. It is highly efficient, especially for larger data sets (> 2'000 data points), but its efficiency comes at the cost of flexibility. The additional plt.scattter() function is more flexible and therefore provides a powerful alternative for smaller to medium sized scatter plots.
Can you write Python code for it?

# Generate std. normal rv
rv = np.random.normal(size=(200, 2))

# Draw scatter plot
plt.scatter(rv[:, 0], rv[:, 1]);


I am working on the file "0103_Plotting.ipynb".
The first task is:
Subplots
Sometimes it is more expressive to plot mutliple figures aligned on columns or rows. Matplotlib's two main routines for doing this are plt.subplot() and plt.subplots() (note the s at the end of the second command). The former serves well for simple plots with a limited number of subplots:
Can you write Python code for it?

for i in range(1, 7):
    plt.subplot(2, 3, i)
    plt.text(0.5, 0.5, str((2, 3, i)),
             fontsize=18, ha='center')

# Adjust amount of height (hspace) and width (wspace)
# reserved for white space between subplots
plt.subplots_adjust(hspace=0.3, wspace=0.5)


I am working on the file "0103_Plotting.ipynb".
The first task is:
The plt.subplots() function provides better control over elements and axes. Rather than creating subplot by subplot, plt.subplots() creates a full grid of subplots in a single line. Accessing them works similar to a NumPy array. Sharing x-axis and y-axis is easily done with the sharex= and sharey= command, respectively. Below is an example on how to create a 2x3 grid of subplots, where all plots in the same row share a common y-axis scale and all axes in the same column have their x-axis scale in common. We will use the object-oriented approach.
Can you write Python code for it?

markers = ['.', '+', 'x', '1', '2', '3']

# Draw two rows of 3 plots each with shared axes
fig, ax = plt.subplots(2, 3, sharex='col', sharey='row')

# Add main title above subplots
fig.suptitle('Main Title', fontsize=18)

# Access each plot and add text
for i in range(2):
    for j in range(3):
        # Create some random data
        data = np.random.uniform(size=(30, 2))

        # Scatterplot with text label
        ax[i, j].scatter(data[:, 0], data[:, 1], marker=markers[2*j+i])
        ax[i, j].text(0.5, 0.5, str((i, j)), fontsize=18,
                      ha='center', weight='bold');


I am working on the file "0103_Plotting.ipynb".
The first task is:
There are many more arguments to further adjust your plot to your liking. A toy example is given below, where the linewidth is manually set and circled markers are added (marker='o'). Regarding available markers a limited overview is shown in section 'Scatterplots' below. The full list of available markers can be found here. For a furhter description on adjustable properties see the Matplotlib documentation.
Can you write Python code for it?

# Random walk data
np.random.seed(1234)
y = np.hstack([0, np.random.standard_normal(10).cumsum()])
x = range(len(y))

plt.plot(x, y, color='m', linewidth=2.0,
         marker='o', markeredgecolor='b', markersize=20,
         markerfacecolor='g', fillstyle='bottom');


I am working on the file "0103_Plotting.ipynb".
The first task is:
There are other options such as the plotting functionalities provided through Plotly, Bokeh or Altair, which are much more accessible in plotting barplots than Matplotlib.
Pie Chart
What is written above about matplotlib's bar chart functionalities is true for pie charts as well, although to a lesser extent. Compared to bar charts, the coding of pie charts is less cumbersome and the available options are ok. Nevertheless, pie chart feel awkwardly old fashioned and personally I do not encourage students to use them. Waffle charts, as an equivalent comparable, provide a much better overview. And here again R (combined with the 'ggplo2' and 'waffle' package) simply outdo Matplotlib. Still, for reference I provide below an example of a pie chart code (based on matplotlib's example).
Can you write Python code for it?

# Pie chart with slices ordered/plotted counter-clockwise:
labels = ['Electrification Products', 'Robotics and Motion',
         'Industrial Automation', 'Power Grids']
sizes = [1528, 1195, 824, 1021]
explode = (0, 0.1, 0, 0)  # only "explode" 2nd slice (i.e. 'Robotics')

plt.pie(sizes, explode=explode, labels=labels, autopct='%1.1f%%',
        shadow=True, startangle=90)
plt.axis('equal');  # Equal aspect ratio ensures that pie is drawn as a circle.


I am working on the file "011. Word to Vector/.ipynb_checkpoints/word2vec-checkpoint.ipynb".
The first task is:
1. Let's Prepare the Data Set
1.1 Download the Data
Can you write Python code for it?

url = 'http://mattmahoney.net/dc/'

def try_download(filename, expected_bytes):
    # Download the file if it is not already present
    # Check the file is the correct size
    if not os.path.exists(filename):
        filename, _ = urlretrieve(url + filename, filename)
    statinfo = os.stat(filename)
    # Grab the stats about the file

    if statinfo.st_size == expected_bytes:
        print('Found and verified file: {}' .format(filename))
    else:
        print(statinfo.st_size)
        raise Exception('Failed to verify the file: {}!' .format(filename))

    return filename

filename = try_download('text8.zip', 31344016)


I am working on the file "011. Word to Vector/.ipynb_checkpoints/word2vec-checkpoint.ipynb".
The first task is:
1.3 Replace rare words with UNK token
Can you write Python code for it?

vocabulary_size = 50000

def build_dataset(words):
    count=[['UNK', -1]]
    # Get the most frequently used words and their frequency
    count.extend(collections.Counter(words).most_common(vocabulary_size -1))

    dictionary = dict()
    for word, _ in count:
        dictionary[word] = len(dictionary)

    data = list()
    unk_count = 0
    for word in words:
        if word in dictionary:
            index = dictionary[word]
        else:
            index = 0
            unk_count = unk_count + 1
        data.append(index)
    count[0][1] = unk_count
    reverse_dictionary = dict(zip(dictionary.values(), dictionary.keys()))

    return data, count, reverse_dictionary, dictionary

data, count, reverse_dictionary, dictionary = build_dataset(words)

print('Most common words (+UNK)', count[:5])
print('Sample data', data[:10])
del words  # Hint to reduce memory.


I am working on the file "011. Word to Vector/.ipynb_checkpoints/word2vec-checkpoint.ipynb".
The first task is:
1.4 Function to generate a training batch for the skip-gram model
Can you write Python code for it?

data_index = 0

def generate_batch(batch_size, num_skips, skip_window):
    global data_index
    assert batch_size % num_skips == 0
    assert num_skips <= 2 * skip_window

    batch = np.ndarray(shape=(batch_size), dtype = np.int32)
    labels = np.ndarray(shape=(batch_size, 1), dtype = np.int32)
    span = 2 * skip_window + 1 #[skip_window, target, skip_window ]
    buffer = collections.deque(maxlen=span)

    for _ in range(span):
        buffer.append(data[data_index])
        data_index = (data_index+1) % len(data)
    for i in range(batch_size // num_skips):
        target = skip_window # target label at the center of the buffer
        targets_to_avoid = [skip_window]
        for j in range(num_skips):
            while target in targets_to_avoid:
                target = random.randint(0, span-1)
            targets_to_avoid.append(target)
            batch[i*num_skips +j] = buffer[skip_window]
            labels[i * num_skips + j, 0] = buffer[target]
        buffer.append(data[data_index])
        data_index = (data_index + 1) % len(data)
    return batch, labels

print('data:', [reverse_dictionary[di] for di in data[:8]])

for num_skips, skip_window in [(2, 1), (4, 2)]:
    data_index = 0
    batch, labels = generate_batch(batch_size=8, num_skips=num_skips, skip_window=skip_window)
    print('\nwith num_skips = %d and skip_window = %d:' % (num_skips, skip_window))
    print('    batch:', [reverse_dictionary[bi] for bi in batch])
    print('    labels:', [reverse_dictionary[li] for li in labels.reshape(8)])


I am working on the file "011. Word to Vector/.ipynb_checkpoints/word2vec-checkpoint.ipynb".
The first task is:
2. Modelling
2.1 Define the Graph
Can you write Python code for it?

batch_size = 128
embedding_size = 128 # Dimension of the embedding vector.
skip_window = 1 # How many words to consider left and right.
num_skips = 2 # How many times to reuse an input to generate a label.
# We pick a random validation set to sample nearest neighbors. here we limit the
# validation samples to the words that have a low numeric ID, which by
# construction are also the most frequent.
valid_size = 16 # Random set of words to evaluate similarity on.
valid_window = 100 # Only pick dev samples in the head of the distribution.
valid_examples = np.array(random.sample(range(valid_window), valid_size))
num_sampled = 64 # Number of negative examples to sample.

graph = tf.Graph()

with graph.as_default():

  # Input data.
  train_dataset = tf.placeholder(tf.int32, shape=[batch_size])
  train_labels = tf.placeholder(tf.int32, shape=[batch_size, 1])
  valid_dataset = tf.constant(valid_examples, dtype=tf.int32)

  # Variables.
  embeddings = tf.Variable(
    tf.random_uniform([vocabulary_size, embedding_size], -1.0, 1.0))
  softmax_weights = tf.Variable(
    tf.truncated_normal([vocabulary_size, embedding_size],
                         stddev=1.0 / math.sqrt(embedding_size)))
  softmax_biases = tf.Variable(tf.zeros([vocabulary_size]))

  # Model.
  # Look up embeddings for inputs.
  embed = tf.nn.embedding_lookup(embeddings, train_dataset)
  # Compute the softmax loss, using a sample of the negative labels each time.
  loss = tf.reduce_mean(
    tf.nn.sampled_softmax_loss(weights=softmax_weights, biases=softmax_biases, inputs=embed,
                               labels=train_labels, num_sampled=num_sampled, num_classes=vocabulary_size))

  # Optimizer.
  # Note: The optimizer will optimize the softmax_weights AND the embeddings.
  # This is because the embeddings are defined as a variable quantity and the
  # optimizer's `minimize` method will by default modify all variable quantities
  # that contribute to the tensor it is passed.
  # See docs on `tf.train.Optimizer.minimize()` for more details.
 ...


I am working on the file "011. Word to Vector/.ipynb_checkpoints/word2vec-checkpoint.ipynb".
The first task is:
2.2 Training the Model
Can you write Python code for it?

num_steps = 100001

with tf.Session(graph=graph) as session:
  tf.global_variables_initializer().run()
  print('Initialized')
  average_loss = 0
  for step in range(num_steps):
    batch_data, batch_labels = generate_batch(
      batch_size, num_skips, skip_window)
    feed_dict = {train_dataset : batch_data, train_labels : batch_labels}
    _, l = session.run([optimizer, loss], feed_dict=feed_dict)
    average_loss += l
    if step % 2000 == 0:
      if step > 0:
        average_loss = average_loss / 2000
      # The average loss is an estimate of the loss over the last 2000 batches.
      print('Average loss at step %d: %f' % (step, average_loss))
      average_loss = 0
    '''
    # note that this is expensive (~20% slowdown if computed every 500 steps)
    if step % 10000 == 0:
      sim = similarity.eval()
      for i in range(valid_size):
        valid_word = reverse_dictionary[valid_examples[i]]
        top_k = 8 # number of nearest neighbors
        nearest = (-sim[i, :]).argsort()[1:top_k+1]
        log = 'Nearest to %s:' % valid_word
        for k in range(top_k):
          close_word = reverse_dictionary[nearest[k]]
          log = '%s %s,' % (log, close_word)
        print(log)
    '''
  final_embeddings = normalized_embeddings.eval()

num_points = 400

tsne = TSNE(perplexity=30, n_components=2, init='pca', n_iter=5000)
two_d_embeddings = tsne.fit_transform(final_embeddings[1:num_points+1, :])

def plot(embeddings, labels):
  assert embeddings.shape[0] >= len(labels), 'More labels than embeddings'
  plt.figure(figsize=(15,15))  # in inches
  for i, label in enumerate(labels):
    x, y = embeddings[i,:]
    plt.scatter(x, y)
    plt.annotate(label, xy=(x, y), xytext=(5, 2), textcoords='offset points',
                   ha='right', va='bottom')
  plt.show()

words = [reverse_dictionary[i] for i in range(1, num_points+1)]
plot(two_d_embeddings, words)


I am working on the file "011. Word to Vector/word2vec.ipynb".
The first task is:
1. Let's Prepare the Data Set
1.1 Download the Data
Can you write Python code for it?


I am working on the file "011. Word to Vector/word2vec.ipynb".
The first task is:
1.2 Unzip the Data and read it into RAM
Can you write Python code for it?

def read_data(filename):
    # Extrac the first file enclosed in the zip as a list of words
    with zipfile.ZipFile(filename) as f:
        data = tf.compat.as_str(f.read(f.namelist()[0])).split()
    return data

words = read_data(filename)
print('Data Size: {}' .format(len(words)))
print(words[:20])


I am working on the file "011. Word to Vector/word2vec.ipynb".
The first task is:
1.4 Function to generate a training batch for the skip-gram model
Can you write Python code for it?


I am working on the file "011. Word to Vector/word2vec.ipynb".
The first task is:
2. Modelling
2.1 Define the Graph
Can you write Python code for it?


I am working on the file "011. Word to Vector/word2vec.ipynb".
The first task is:
2.2 Training the Model
Can you write Python code for it?


I am working on the file "014 - Lorde/tao_mro/tao_of_python.ipynb".
The first task is:

Can you write Python code for it?

from IPython.display import Image, display
display(Image(url='figures/mind_blown.gif', width=400))


I am working on the file "014 - Lorde/tao_mro/tao_of_python.ipynb".
The first task is:
Rule 1: Everything is an object
And I mean everything. Even things that are "primitive types" in other languages.
* You can store them in variables
* You can pass them as parameters to functions
* You can return them as the result of functions
* You can construct them at runtime
And more importantly: You can treat every programming construct in a uniform and consistent way
Functions as objects

When you use the keyword def, Python creates a function object.
Functions can be passed around as arguments to other functions.
These functions that take other functions as arguments are called higher order functions.
e.g. the map function takes a function and an iterable and applies the function to each item in the iterable.

Classes as objects

When you use the keyword class, Python executes it and creates an object.
This object (the class) is itself capable of creating objects (the instances), and this is why it's a class.
Since classes are objects, they must be generated by something, this is metaclasses.
Since metaclasses objects, they must also be generated by something, this is again metaclasses.

Therefore: Objects are instances of classes, classes are instances of metaclasses and metaclasses are instances of themselves.
Metaclasses (these are objects too!)

Metaclasses are the "stuff" that creates classes.
You define classes in order to create objects, right?
We learned that Python classes are objects.
Well, metaclasses are what creates these objects. They are the classes' classes, you can picture them this way:
MyClass = MetaClass()
MyObject = MyClass()

Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t (the people who actually need them know with certainty that they need them, and don’t need an explanation about why). Tim Peters

What is type?

Remember the function type? The good old function that lets you know what type an object is.
type can also create classes on the fly. type can take the description of a class as parameters, and...
Can you write Python code for it?

class A:
    pass
a = A()
print(type(a))
print(type(A))
print(A.__bases__)

A = type('A', (), {})
a = A()
print(type(a))
print(type(A))
print(A.__bases__)
print(isinstance(a, A), isinstance(a, object), issubclass(A, object))

def f():
    """My name is f."""
    pass
print(type(f))
print(type(type(f)))
print(type(f).__bases__)
print(f.__doc__)


I am working on the file "014. RNN for Sin2/main.ipynb".
The first task is:
2. Let's define a our models
Can you write Python code for it?

def my_rnn(x, W, b):
    x = tf.transpose(x, [1, 0, 2])
    x = tf.reshape(x, [-1, n_input])
    x = tf.split(x, n_step, axis=0)

    lstm_cell = rnn.BasicLSTMCell(n_hidden, forget_bias=1.0)
    outputs, states = rnn.static_rnn(lstm_cell, x, dtype=tf.float32)
    return tf.nn.bias_add(tf.matmul(outputs[-1], W), b)

learning_rate = 1e-3

n_hidden = 10
n_step = 10
n_input = 1
n_output = 10

n_epoch = 10
epoch_size = 100
n_iter = n_epoch * epoch_size

tf.reset_default_graph()

x = tf.placeholder(tf.float32, [None, n_step, n_input])
y = tf.placeholder(tf.float32, [None, n_output])

W = tf.Variable(tf.truncated_normal([n_hidden, n_output]))
b = tf.Variable(tf.truncated_normal([n_output]))

h = my_rnn(x, W, b)

individual_losses = tf.reduce_sum(tf.squared_difference(h,y), 1)
loss = tf.reduce_mean(individual_losses)
optimiser = tf.train.AdamOptimizer(learning_rate).minimize(loss)

%%time
batch_size = 50
sess = tf.InteractiveSession()
tf.global_variables_initializer().run()

for iter in range(n_iter+1):
    # No mini Batch yet
    _, p, f = generate_sin(batch_size)
    optimiser.run(feed_dict={x:p, y:f})

    if iter% epoch_size ==0:
        print('Epoch: {} Loss: {}' .format(int(iter/epoch_size), loss.eval(feed_dict={x:p, y:f})))

t, p, f = generate_sin(2000, T = 50, t_offset=10)

l = loss.eval(feed_dict={x:p, y:f})
print(l)
fcast = h.eval(feed_dict={x:p})

plt.plot(t, f[:,0], label='Actual')
plt.plot(t, fcast[:,0], label='fcast')
plt.legend()


I am working on the file "015 - Breaking Benjamin/meta_alltheway/meta_alltheway.ipynb".
The first task is:
<img src="figures/class_instance_creation.png" style="display:block;margin:auto;width:60%;"/>
Our example with a class as metaclass
Can you write Python code for it?

class CamelToSnake(type):
    def __new__(mcs, name, bases, attrs):
        # pick up any attribute that doesn't start with '__' and snakecase it
        snake_attrs = {}
        for attr_name, attr_val in attrs.items():
            if not name.startswith('__'):
                snake_attrs[convert(attr_name)] = attr_val
            else:
                snake_attrs[attr_name] = attr_val
        return super().__new__(mcs, name, bases, snake_attrs)

class MyVector(metaclass=CamelToSnake):
    def addToVector(self, other): pass
    def subtractFromVector(self, other): pass
    def calculateDotProduct(self, other): pass
    def calculateCrossProduct(self, other): pass
    def calculateTripleProduct(self, other): pass

print([a for a in dir(MyVector) if not a.startswith('__')])


I am working on the file "015 - Breaking Benjamin/meta_alltheway/meta_alltheway.ipynb".
The first task is:
Creating classes dynamically

Remember the function type? The good old function that lets you know what type an object is.
type can also create classes on the fly. type can take the description of a class as parameters, and return a class as type(name, bases, attrs).
name is a string giving the name of the class to be constructed.
bases is a tuple giving the parent classes of the class to be constructed.
attrs is a dictionary of the attributes and methods of the class to be constructed.

For example:
python
class Foo:
    bar = True
is equivalent with:
python
Foo = type('Foo', (), {'bar':True})
What are metaclasses

Metaclasses are the "stuff" that creates classes. You define classes in order to create objects, right?
We learned that Python classes are objects.
Well, metaclasses are what creates these objects. They are the classes' classes, you can picture them this way:

MyClass = MetaClass(), MyObject = MyClass()
<img src="figures/oo_relationships_revisited.png" style="display:block;margin:auto;width:80%;"/>
What can act as a metaclass?

Something that can create a class, i.e. anything that subclasses or uses type.
Any python callable. A callable is an object that can be invoked with the function operator ().
Any Python class. MyClass() calls the __init__() function of the class
Any Python function or instance whose class implements __call__(). foo() is equivalent to foo.__call__(). This is true in both cases (i.e. all functions objects implement __call__).

i.e. a metaclass is something that we call with some parameters and returns a class object

When you write:
python
class A:
    ...
you are actually not creating the class, you describe the class and type creates this class object for you.

Each class can have one metaclass, however this metaclass can subclass multiple metaclasses.

Our example with a function as metaclass
Can you write Python code for it?

import re

def convert(name):
    s1 = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', name)
    return re.sub('([a-z0-9])([A-Z])', r'\1_\2', s1).lower()

# the metaclass will automatically get passed the same arguments that we pass to `type`
def camel_to_snake_case(name, bases, attrs):
    """Return a class object, with its attributes from camelCase to snake_case."""
    print("Calling the metaclass camel_to_snake_case to construct class: {}".format(name))

    # pick up any attribute that doesn't start with '__' and snakecase it
    snake_attrs = {}
    for attr_name, attr_val in attrs.items():
        if not name.startswith('__'):
            snake_attrs[convert(attr_name)] = attr_val
        else:
            snake_attrs[attr_name] = attr_val
    return type(name, bases, snake_attrs) # let `type` do the class creation

class MyVector(metaclass=camel_to_snake_case):
    def addToVector(self, other): pass
    def subtractFromVector(self, other): pass
    def calculateDotProduct(self, other): pass
    def calculateCrossProduct(self, other): pass
    def calculateTripleProduct(self, other): pass

print([a for a in dir(MyVector) if not a.startswith('__')])


I am working on the file "015 - Breaking Benjamin/meta_alltheway/meta_alltheway.ipynb".
The first task is:
Make our class a Singleton
Can you write Python code for it?

class Singleton(type):
    _instances = {}
    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super().__call__(*args, **kwargs)
        return cls._instances[cls]

class SnakeSingleton(CamelToSnake, Singleton):
    pass

class MyVector(metaclass=SnakeSingleton):
    def addToVector(self, other): pass
    def subtractFromVector(self, other): pass
    def calculateDotProduct(self, other): pass
    def calculateCrossProduct(self, other): pass
    def calculateTripleProduct(self, other): pass

print([a for a in dir(MyVector) if not a.startswith('__')])
v1 = MyVector(); v2 = MyVector()
print(v1 is v2)


I am working on the file "015 - Breaking Benjamin/meta_alltheway/meta_alltheway.ipynb".
The first task is:
You are using them already

You are using metaclasses even if you are not aware of using them.
They make things possible in Python that wouldn't be otherwise or would require substantial changes to the language.
You are a programmer willing to go out of his comfort zone, learn new things and challenge common practices.

but but but... I want to make my own metaclasses!
OK, I got a simple problem for you

You are friends with some guys, they are quite good programmers.
The learned Python but all have a Javascript background (you know, from that other meetup).
They wrote a Python library that you really want to use.
They used camelCase for all their methods instead of underscore_method_names.
You really want to use their library but love get_first_name and hate getFirstName

Possible solutions

Override __getattr__:
Renaming must happen everytime you access the method.
When trying to get attribute/method of an object, the object's superclasses are looked up before __getattr__. Problems with inherited methods.

Introspection. Use inspect.getmembers(MyClass, inspect.isfunction) and rename functions. This works but:
You need to run this everytime you load each class.
Is too specific. What if you want to add extra functionality in how this classes operate (e.g. make a class Singleton or make each class register itself somewhere when it's being used)?

Class decorator:
Need to redefine a new nested class for every class you want to alter.
You will run into problems with inheritance.

Use metaclasses.

Warning! Metaclasses ahead!
<img src="figures/abandon_hope.jpg" style="display:block;margin:auto;width:100%;"/>
Classes as objects

In most languages, classes are just pieces of code that describe how to produce an object.
But classes are more than that in Python. Classes are objects too.
This object (the class) is itself capable of creating objects (the instances), and this is why it's a class. But still, it's an object, and therefore:
You can assign it to a variable.
You can copy it.
You can add attributes to...
Can you write Python code for it?

class ObjectCreator:
    pass
print(ObjectCreator)  # you can pass a class as a parameter because it's an object

print(hasattr(ObjectCreator, 'new_attribute'))
ObjectCreator.new_attribute = 'foo' # you can add attributes to a class
print(hasattr(ObjectCreator, 'new_attribute'))
print(ObjectCreator.new_attribute)

ObjectCreatorMirror = ObjectCreator # you can assign a class to a variable
print(ObjectCreatorMirror.new_attribute)
print(ObjectCreatorMirror())


I am working on the file "016 - Yak Shaving/func_py/func_py.ipynb".
The first task is:
Generators
A generator is a factory object that lazily produces values (i.e. generates values on demand). A generator is either:
1. A function that contains the yield keyword (yield expression).
    - When this function is called, it does not execute, but returns a generator object.
    - The first time that next() is called, when this function encounters the yield keyword it suspends execution at that point, saves its context and returns to the caller along with any value in expression.
    - When the caller invokes next() again, execution of the function continues till another yield or return is encountered or end of function is reached.
2. A generator expression, i.e. a syntactic construct for creating an anonymous generator object, following the form of mathematical set-builder notation. These are like list comprehensions but enclosed in () instead of [].
    - The semantics of a generator expression are equivalent to creating an anonymous generator function and calling it.
A generator is always an iterator, but not vice versa (iterator is a more general concept).
Can you write Python code for it?

# generator expression
g = (x ** 2 for x in range(10))
print(next(g))
print(next(g))
print([i for i in g])

# generator function
def _gen(exp):
    for x in exp:
        yield x ** 2
g = _gen(range(10))
print(next(g))
print(next(g))
print([i for i in g])

# generator
def it_gen(text):
    for ch in text:
        yield ch.upper()

# generator expression
def it_genexp(text):
    return (ch.upper() for ch in text)

# iterator protocol (__iter__)
class ItIter():
    def __init__(self, text):
        self.text = text
        self.index = 0
    def __iter__(self):
        return self
    def __next__(self):
        try:
            result = self.text[self.index].upper()
        except IndexError:
            raise StopIteration
        self.index += 1
        return result

# iterator protocol (__getitem__)
class ItGetItem():
    def __init__(self, text):
        self.text = text
    def __getitem__(self, index):
        result = self.text[index].upper()
        return result

# an iterable of iterables (see what i did there? :P)
for iterator in (it_gen, it_genexp, ItIter, ItGetItem):
    for ch in iterator('abcde'):
        print(ch, end='')
    print('\n')

import collections

def flatten(container):
    """Flatten an iterable of arbitrary depth."""
    for item in container:
        if isinstance(item, collections.Iterable) and not isinstance(item, (str, bytes)):
            for element in flatten(item):
                yield element
        else:
            yield item

d3 = [[[i for i in range(2)] for j in range(3)] for k in range(4)]
print(d3)
print([k for i in d3 for j in i for k in j])
print([i for i in flatten(d3)])


I am working on the file "016 - Yak Shaving/func_py/func_py.ipynb".
The first task is:
Here, x is the iterable, while y and z are two individual instances of an iterator, producing values from the iterable x. Both y and z hold state, as you can see from the example. In this example, x is a data structure (a list), but that is not a requirement.
Often, for pragmatic reasons, iterable classes will implement both __iter__() and __next__() in the same class, and have __iter__() return self, which makes the class both an iterable and its own iterator. It is perfectly fine to return a different object as the iterator, though.

When you write:
python
x = [1, 2, 3]
for elem in x:
    ...
This is what actually happens:
<img src="figures/iterable_vs_iterator.png" style="display:block;margin:auto;width:80%;"/>
Can you write Python code for it?

s = 'cat'  # s is an ITERABLE
           # s is a str object that is immutable
           # s has no state
           # s has a __getitem__() method

t = iter(s)    # t is an ITERATOR
               # t has state (it starts by pointing at the "c")
               # t has a __next__() method and an __iter__() method
try:
    print(next(t))        # the next() function returns the next value and advances the state
    print(next(t))        # the next() function returns the next value and advances
    print(next(t))        # the next() function returns the next value and advances
    print(next(t))        # next() raises StopIteration to signal that iteration is complete
except StopIteration as e:
    print("StopIteration raised")

class FibonacciIterator:
    """
    Produces an arbitrary number of the Fibonacci numbers.
    Is an both an iterable and an iterator.
    """
    def __init__(self):
        self.prev = 0
        self.curr = 1

    def __iter__(self):
        return self

    def __next__(self):
        self.prev, self.curr = self.curr, self.prev + self.curr
        return self.prev

class Countdown:
    """A simple iterable, NOT an iterator"""
    def __iter__(self):
        return iter([5, 4, 3, 2, 1, 'launch'])

f = FibonacciIterator()
print([next(f) for _i in range(10)])
c = Countdown()
print([i for i in c])


I am working on the file "016 - Yak Shaving/func_py/func_py.ipynb".
The first task is:
Higher order functions and decorators

Python functions are objects.
Can be defined in functions.
Can be assigned to variables.
Can be used as function parameters or returned from functions.
Decorators are syntactic sugar for higher order functions.

<img src="figures/brace_yourselves.jpg" style="display:block;margin:auto;width:80%;"/>
Can you write Python code for it?

# Higher order functions

def makebold(fn):
    def wrapped():
        return "" + fn() + ""
    return wrapped

def hello():
    return "hello world"

print(hello())
hello = makebold(hello)
print(hello())

# Decorated function with *args and **kewargs

def makebold(fn):
    def wrapped(*args, **kwargs):
        return "" + fn(*args, **kwargs) + ""
    return wrapped

@makebold  # hello = makebold(hello)
def hello(*args, **kwargs):
    return "Hello. args: {}, kwargs: {}".format(args, kwargs)

print(hello('world', 'pythess', where='soho'))

# Decorators can be combined

def makeitalic(fn):
    def wrapped(*args, **kwargs):
        return "" + fn(*args, **kwargs) + ""
    return wrapped

def makebold(fn):
    def wrapped(*args, **kwargs):
        return "" + fn(*args, **kwargs) + ""
    return wrapped

@makeitalic
@makebold  # hello = makeitalic(makebold(hello))
def hello(*args, **kwargs):
    return "Hello. args: {}, kwargs: {}".format(args, kwargs)

print(hello('world', 'pythess', where='soho'))

# Decorators can be instances of callable classes

class BoldMaker:
    def __init__(self, fn):
        self.fn = fn
    def __call__(self, *args, **kwargs):
        return "" + self.fn(*args, **kwargs) + ""

@BoldMaker  # hello = Bookmaker(hello)
def hello(*args, **kwargs):
    return "Hello. args: {}, kwargs: {}".format(args, kwargs)

# hello.__call__(*args, **kwargs)
print(hello('world', 'pythess', where='soho'))

# Decorators can take arguments

def enclose_in_tags(opening_tag, closing_tag):  # returns a decorator
    def make_with_tags(fn):  # returns a decorated function
        def wrapped():  # the function to be decorated (modified)
            return opening_tag + fn() + closing_tag
        return wrapped
    return make_with_tags

# decorator function make_with_tags with the arguments in closure
heading_decorator = enclose_in_tags('
', '
')
paragraph_decorator = enclose_in_tags('
', '
')

def hello():
    return "hello world"

h1_hello =...


I am working on the file "016 - Yak Shaving/func_py/func_py.ipynb".
The first task is:
Iterables and Iterators

Iteration is a general term for taking each item of something, one after another. Any time you use a loop, explicit or implicit, to go over a group of items, that is iteration.
An iterable is an anything that can be looped over. It either:
Has an __iter__ method which returns an iterator for that object when you call iter() on it, or implicitly in a for loop.
Defines a __getitem__ method that can take sequential indexes starting from zero (and raises an IndexError when the indexes are no longer valid).

An iterator is:
A stateful helper object which defines a  __next__ method and will produce the next value when you call next() on it. If there are no further items, it raises the StopIteration exception.
An object that is self-iterable (meaning that it has an __iter__ method that returns self).

Therefore: An iterable is an object from which we can get an iterator. An iterator is always an iterable. An iterable is not always an iterator but will always return an iterator.
<img src="figures/always_iterators.jpg" style="display:block;margin:auto;width:65%;"/>
Can you write Python code for it?

x = [1, 2, 3]  # this is an iterable
y = iter(x)  # an iterator of this iterable
z = iter(x)  # another iterator of this iterable
print(next(y))
print(next(y))
print(next(z))
print(type(x))
print(type(y))


I am working on the file "016 - Yak Shaving/func_py/func_py.ipynb".
The first task is:
What can act as a function in Python?

A function object, created with the def statement.
A lambda anonymous function, restricted to an expression in single line.
An instance of a class implementing the __call__ function.
Can you write Python code for it?

def min_def(x, y):
    return x if x < y else y

min_lambda = lambda x, y: x if x < y else y

class MinClass:
    def __call__(self, x, y):
        return x if x < y else y

min_class = MinClass()
print(min_def(2,3) == min_lambda(2, 3) == min_class(2,3))


I am working on the file "01A_TensorFlow_Basics.ipynb".
The first task is:
Basic TensorFlow test from the installation instructions
Can you write Python code for it?

import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))


I am working on the file "01A_TensorFlow_Basics.ipynb".
The first task is:
Define a simple TensorFlow graph
The example from the TensorFlow Introduction slides:
Can you write Python code for it?

a = tf.constant(3.0, dtype=tf.float32)
b = tf.constant(4.0, dtype=tf.float32)
sum_a_b = tf.add(a, b)

# Using the Python "with" as a context manager
with tf.Session() as sess:
    print(sess.run(sum_a_b)) # Prints “7.0” to the screen


I am working on the file "01A_TensorFlow_Basics.ipynb".
The first task is:
Let's take a look at how the data is organized:
Can you write Python code for it?

# Dataset statistics
print('Training image data: {0}'.format(mnist.train.images.shape))
print('Validation image data: {0}'.format(mnist.validation.images.shape))
print('Testing image data: {0}'.format(mnist.test.images.shape))
print('28 x 28 = {0}'.format(28*28))

print('\nTest Labels: {0}'.format(mnist.test.labels.shape))
labels = np.arange(10)
num_labels = np.sum(mnist.test.labels, axis=0, dtype=np.int)
print('Label distribution:{0}'.format(list(zip(labels, num_labels))))

# Example image
print('\nTrain image 1 is labelled one-hot as {0}'.format(mnist.train.labels[1,:]))
image = np.reshape(mnist.train.images[1,:],[28,28])
plt.imshow(image, cmap='gray')


I am working on the file "01A_TensorFlow_Basics.ipynb".
The first task is:
Logistic Regression Model
Define the graph input: this is where we feed in our training images into the model. Since MNIST digits are pretty small and the model we're using is very simple, we'll feed them in as flat vectors.
Can you write Python code for it?

# Define input placeholder
x = tf.placeholder(tf.float32, [None, 784])


I am working on the file "01A_TensorFlow_Basics.ipynb".
The first task is:
Not bad for a simple model and a few lines of code.  Before we close the session, there's one more interesting thing we can do. Normally, it can be difficult to inspect exactly what the filters in a model are doing, but since this model is so simple, and the weights transform the data directly to their logits, we can actually visualize what the model's learning by simply plotting the weights. The results look pretty reasonable:
Can you write Python code for it?

# Get weights
weights = sess.run(W)

fig, ax = plt.subplots(1, 10, figsize=(20, 2))

for digit in range(10):
    ax[digit].imshow(weights[:,digit].reshape(28,28), cmap='gray')

# Close session to finish
sess.close()


I am working on the file "01A_TensorFlow_Basics.ipynb".
The first task is:
Notice, because of the way the dependency links are connected in our graph, running the optimizer requires an input to both the training image placeholder x and the training label placeholder y_ (as it should). The values of all variables (W and b) are updated in place automatically by the optimizer.
Now let's see how we did! For every image in our test set, we run the data through the model, and take the digit in which we have the highest confidence as our answer. We then compute an accuracy by seeing how many we got correct:
Can you write Python code for it?

# Test trained model
correct_prediction = tf.equal(tf.argmax(py, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print('Test accuracy: {0}'.format(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels})))


I am working on the file "01A_TensorFlow_Basics.ipynb".
The first task is:
That defines our forward pass of our model! We now have a graph that performs a forward pass: given an input image, the graph returns the probabilities the model thinks the input is each of the 10 classes. Are we done?
Not quite. We don't know the values of $W$ and $b$ yet. We're going to learn those by defining a loss and using gradient descent to do backpropagation. Essentially, we'll be taking the derivative with respect to each of the elements in $W$ and $b$ and wiggling them in a direction that reduces our loss.
The loss we commonly use in classification is cross-entropy. Cross-entropy is a concept from information theory:
\begin{align}
H_{y'}(y)=-\sum_i y'_i \text{log}(y_i)
\end{align}
Cross-entropy not only captures how correct (max probability corresponds to the right answer) the model's answers are, it also accounts for how confident (high confidence in correct answers) they are. This encourages the model to produce very high probabilities for correct answers while driving down the probabilities for the wrong answers, instead of merely be satisfied with it being the argmax.
In supervised models, we need labels to learn, so we create a placeholder for the labels in our training data:
Can you write Python code for it?

# Define labels placeholder
y_ = tf.placeholder(tf.float32, [None, 10])


I am working on the file "01A_TensorFlow_Basics.ipynb".
The first task is:
The entire model, with the complete model definition, training, and evaluation (but minus the weights visualization), is below. Note the slight difference when calculating the softmax; this is done for numerical stability purposes.
Can you write Python code for it?

import tensorflow as tf
from tqdm import trange
from tensorflow.examples.tutorials.mnist import input_data

# Import data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# Create the model
x = tf.placeholder(tf.float32, [None, 784])
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
y = tf.matmul(x, W) + b

# Define loss and optimizer
y_ = tf.placeholder(tf.float32, [None, 10])
cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

# Create a Session object, initialize all variables
sess = tf.Session()
sess.run(tf.global_variables_initializer())

# Train
for _ in trange(1000):
    batch_xs, batch_ys = mnist.train.next_batch(100)
    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

# Test trained model
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print('Test accuracy: {0}'.format(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels})))

sess.close()


I am working on the file "01A_TensorFlow_Basics.ipynb".
The first task is:
The previous graph only produces a constant output. Not particularly interesting. To create a similar graph that can acccept variable inputs:
Can you write Python code for it?

a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)
sum_a_b = tf.add(a, b)

with tf.Session() as sess:
    feed_dict = {a: 3.0, b: 4.0}
    print('1st Result: {0}'.format(sess.run(sum_a_b, feed_dict=feed_dict)))

    feed_dict = {a: 2015, b: 2020}
    print('2nd Result: {0}'.format(sess.run(sum_a_b, feed_dict=feed_dict)))


I am working on the file "01A_TensorFlow_Basics.ipynb".
The first task is:
There are much cleverer ways to design a training regimen that stop training once the model is converged and before it starts overfitting, but for this demo, we'll keep it simple:
Can you write Python code for it?

# Train the model
# trange is a tqdm function. It's the same as range, but adds a pretty progress bar
for _ in trange(1000):
    batch_xs, batch_ys = mnist.train.next_batch(100)
    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})


I am working on the file "01A_TensorFlow_Basics.ipynb".
The first task is:
To get our predicted probabilities of each digit, let's first start with the probability of a digit being a 3 like the image above. For our simple model, we start by applying a linear transformation. That is, we multiply each value of the input vector by a weight, sum them all together, and then add a bias. In equation form:
\begin{align}
y_3 = \sum_i w_{i,3} x_i + b_3
\end{align}
The magnitude of this result $y_3$, we'll take as being correlated to our belief in how likely we think the input digit was a 3. The higher the value of $y_3$, the more likely we think the input image $x$ was a 3 (ie, we'd hope we'd get a relatively large value for $y_3$ for the above image). Remember though, our original goal was to identify all 10 digits, so we also have:
\begin{align}
y_0 =& \sum_i w_{i,0} x_i + b_0 \
&\vdots \
y_9 =& \sum_i w_{i,9} x_i + b_9
\end{align}
We can express this in matrix form as:
\begin{align}
y = W x + b
\end{align}
To put this into our graph in TensorFlow, we need to define some Variables to hold the weights and biases:
Can you write Python code for it?

# Define linear transformation
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
y = tf.matmul(x, W) + b


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
"Attempting to use uninitialized value Variable." In order to initialize the variable, we actually have to run the assign op that was created for us. Recall that this was the third operation in the graph.
If we want to run an operation just for its side effect, not to fetch its output, sess.run does support passing in an op directly anywhere you would normally pass a tensor (as part of the fetches data structure). Here, we'll just pass it standalone:
Can you write Python code for it?


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
A Tensor is a lightweight Python object that represents a piece of data flowing along the edges of our graph. That data can be a scalar, a vector, a matrix, or a higher-dimensional array. (The dimensionality of a tensor t is accessible as an attribute: t.shape. Here, a shape of (), Python's empty tuple, means that t is a scalar.) The tensor's data is not actually stored inside the Tensor object, and in fact does not exist in Python at all. t does not know it will refer to the value 3.14. A Tensor is just a lightweight way to reference a piece of data that will be computed by TensorFlow's execution engine.
All tensors are the outputs of some operation already in the graph. We use Tensors for two things:
1. When we eventually tell TensorFlow to run our computation graph, we need to let it know which of the many intermediate values we actually want it to report back to us. To do this, we pass in a list of tensors, references to specific values it will compute, that we wish it to "fetch."
2. When we create operations with inputs, we need to tell them which other operations' outputs to consume. To do this, we specify input tensors.
To see this in action, let's create another constant operation, then an addition operation that consumes the output tensors of each constant and sums them. First, we call tf.constant to create our second constant. As a convenience, the tf.constant function actually returns the output tensor of the constant operation it creates:
Can you write Python code for it?

another_const_tensor = tf.constant(1.414)
another_const_tensor

g.get_operations()


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
A few things to note:
1. We were able to fetch the same tensor multiple times but with different feed_dict arguments.
2. The placeholder's dimension is dynamic. (You can optionally specify a static dimension when defining a placeholder, by passing a shape argument to tf.placeholder.)
3. The add operation works much like numpy's add operation, in that it can add (element-wise) two multidimensional arrays. It also supports broadcasting, if you're familiar with that from the numpy world.
4. When asked to fetch an array (single- or multi-dimensional), TensorFlow returns a numpy array.
As an aside, the Python arithmetic operators ($+$, $-$, $/$, $*$) are overridden for the Tensor type. Evaluating the expression $a + b$, where $a$ and $b$ are tensors, has the side effect of adding an Add operation to the default graph with $a$ and $b$ as inputs. The result of evaluating $a + b$ is the output tensor of that add operation. This makes it easy to add many nodes quickly. For instance,
Can you write Python code for it?

distance_from_origin = tf.sqrt((a * a) + (b * b))
g.get_operations()

distance_from_origin

sess = tf.Session(config=config)
sess.run(distance_from_origin, feed_dict={a: 3., b: 4.})

sess.close()


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
As expected, we get 2, the partial derivative of eqn with respect to b at the point (a=5, b=7) (at any point, actually, but TensorFlow is computing it at this specific point).
Let's look at a couple variations. If we pass multiple xs, we get a gradient back instead of just a partial derivative -- this is a list of partial derivatives.
Can you write Python code for it?

gradient_wrt_both = tf.gradients(eqn, [a, b])
sess.run(gradient_wrt_both)


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
As you can see, for our extremely simple problem, Adam hurts more than it helps. Its momentum feature means that the big derivatives from early on in training still have an effect even as we get close to the valley, causing us to overshoot the minimum. Running this for more than 100 iterations would eventually bring us back to 0. (An image that is sometimes used to explain momentum-based optimization approaches vs. typical gradient descent is that if normal gradient descent can be thought of as a man slowly and cautiously walking downhill, momentum-based approaches are better understood as a heavy ball rolling its way downhill. The ball will likely overshoot the bottom then have to roll back down, taking a while to settle. The benefit is that the ball can roll straight past small local minima, and stay immune to certain types of pathological terrains.)
Machine learning with TensorFlow
So, how do we use all this for machine learning?
At this point, you're likely ready to move onto any other TensorFlow tutorial you find on the Internet (or in this Github repo). But in case you want to stay here, we'll work our way through a simple example: we'll create a fake dataset of temperatures and number of hospital visits in a fictional city over a number of months. (We assume that colder temperatures lead to more hospital visits.) We'll then fit a linear model to this data.
The goal of this exercise is for you to see how placeholders, variables, and gradient descent come together to enable fitting a model to some data. Although we'll use a linear model here -- i.e., we assume that the data y can be understood as a linear function of the input $x$ -- the process is the exact same when using a nonlinear model, like a neural network.
Creating the fake data
Let's begin by creating some fake data for us to train on. We'll make 1000 samples. We generate temperatures according to a normal distribution using numpy, then we generate hospital visit numbers according to the formula, 1000 - 5*temps, plus some random noise.
Can you write Python code for it?

import numpy as np
temps = np.random.normal(55, 20, 1000)
random_noise = np.random.normal(0, 100, 1000)
hosp_visits = 1000 - 5 * temps + random_noise
plt.plot(temps[:200], hosp_visits[:200], "o")


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
As you can see, the value of loss gets closer and closer to 0 as time goes on.
Now let's replace our manual derivative calculation with automatic differentiation, and convince ourselves that nothing changes:
Can you write Python code for it?

tf.reset_default_graph()
g = tf.get_default_graph()

x = tf.Variable(5.0)
loss = x * x

# tf.gradients is called "gradients" and not "derivative" for a reason: it
# returns a _list_ of partial derivatives, even if you only pass in one x.
# Pull out first element (in our case, the list only has one element).
dloss_dx = tf.gradients(loss, x)[0]
new_x_value = x - dloss_dx * step_size
assign_new_x = tf.assign(x, new_x_value)

with tf.Session(config=config) as sess:
    # Initialize x:
    sess.run(x.initializer)

    # We will fetch the following tensors each iteration
    to_fetch = {'x': x, 'loss': loss, 'derivative': assign_new_x}
    for i in range(100):
        fetched = sess.run(to_fetch)

        # every tenth step, print our progress
        if i % 10 == 0 or i+1==100:
            print("Iter {}: {}".format(i, fetched))


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
Computing gradients
In machine learning, we typically compute some loss function (or error) that quantifies how poorly our model is capturing the patterns in the data. In many models, this loss function is differentiable, which means we can compute partial derivatives of our loss function with respect to the parameters of our model. These derivatives essentially tell us: as each parameter changes, how does it affect the loss? Given this information, we have an easy way to make the model better: perturb the parameters slightly in the direction that causes the loss to go down, according to the partial derivatives. We can do this repeatedly, computing new partial derivatives after each step, until we get to a local minimum. This technique is called "gradient descent."
One benefit of using a computation graph is that Tensorflow can automatically calculate these derivatives for us. More accurately, it can augment our computation graph with operations that compute any derivatives we'd like. To do this, we use the tf.gradients function. It takes in two arguments: ys and xs. ys is a tensor or list of tensors we'd like to calculate the derivatives of. xs is a tensor or list of tensors we'd like to calculate those derivatives with respect to. When called, tf.gradients traverses the computation graph backward from ys to xs, adding for each operation along that path one or more supplemental operations for computing the gradient. These individual-operation gradients can be composed using the chain rule (which tf.gradients also takes care of). The return value of tf.gradients is a handle to a tensor that represents the answer, $\frac{dy}{dx}$, at the end of that chain.
This will hopefully become clearer with a simple example. Let's create a small graph:
Can you write Python code for it?

a = tf.constant(5, name='a')
b = tf.constant(7, name='b')
eqn = a + 2 * b
g.get_operations()


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
Computing the loss
To compute the loss -- a quantity measuring how bad our model is -- we use sum-of-squares formula: for each data point in the current batch of data, we subtract the real y from our predicted_y and square the difference; then we take the average of all these. (Taking their sum would work just as well, but by taking the average, we get a number that doesn't depend on the amount of data in a batch, which can be useful for human interpretation and for comparing models with different batch sizes.)
Can you write Python code for it?

avg_loss = tf.reduce_mean(tf.squared_difference(predicted_y, y))


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
Creating an optimizer
Now it's time to actually create our optimization op. We'll use the basic GradientDescentOptimizer here, with a learning rate of 0.0005. (How did we come by this number? Trying different numbers and seeing how the losses looked on the validation set. Feel free to play with this a bit more.)
Can you write Python code for it?


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
Even before you call your first TensorFlow function, a lot is going on behind the scenes. For example, an empty default graph object is created. (In order to make it easier to hit the ground running with TensorFlow, and to make using the library less verbose, Google has laced the system with global state. In order to fully understand what's happening in your program, you need to know that these state variables, like "current graph" and "current scope" and "current session", exist. As we'll soon see, most of the functions you'll call in TensorFlow operate by quietly accessing this hidden state.)
We can access the default graph explicitly using tf.get_default_graph:
Can you write Python code for it?


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
Here, [8, 8] == [7, 6] + [1, 2].
Can you write Python code for it?

sess.close()

tf.reset_default_graph()
g = tf.get_default_graph()


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
If we attempt to get the value of the variable, we will get an error:
Can you write Python code for it?


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
In fact, let's start a second session as well. One of the roles sessions play is to keep track of variable values across executions of the graph, and this will help us visualize that.
Can you write Python code for it?


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
In the last example, TensorFlow created a copy of the fetches data structure, but with each tensor replaced by its actual value, computed during execution.
When you're done with a session, you should close it, to free any resources it's keeping track of:
Can you write Python code for it?


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
Initializing variables and training
Finally, it's time for training. Let's add an op to the graph that initializes all variables, then start a session and run the training code.
Can you write Python code for it?

init_all_vars = tf.global_variables_initializer()

with tf.Session(config=config) as sess:
    sess.run(init_all_vars)
    for i in range(5000):
        X_batch, y_batch = next(training_generator)
        feed_dict = {X: X_batch, y: y_batch}
        _, loss, m_pred, b_pred = sess.run([train_one_step, avg_loss, m, b], feed_dict=feed_dict)
        if i % 500 == 0:
            validation_feed_dict = {X: valid_X, y: valid_y}
            valid_loss = sess.run(avg_loss, feed_dict=validation_feed_dict)
            print("Iter {}: training loss = {}, validation loss = {}, m={}, b={}".format(i, loss, valid_loss, m_pred, b_pred))

    test_feed_dict = {X: test_X, y: test_y}
    m_pred, b_pred, loss = sess.run([m, b, avg_loss], test_feed_dict)
    print("m: {}, b: {}, test loss: {}".format(m_pred, b_pred, loss))


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
It should make sense that add_op has two inputs and one output.
Running computations using tf.Session
In order to execute your computation graph, you need to create a Session object. A Session object stores a reference to the graph to execute, some configuration information (determining, for instance, how much memory TensorFlow should allocate on your GPU), and storage for stateful components of your computation graph (which must be kept around across multiple executions of the graph). We haven't talked about any "stateful components" yet, so it's okay if that piece doesn't make sense yet. We'll see Variables, the simplest stateful component, very soon.
You can create a session using the tf.Session constructor. It can take in a graph parameter, but if you omit it, the session will use the default graph.
Can you write Python code for it?

# This piece is only necessary so as not to use up an ungodly amount of GPU memory:
config = tf.ConfigProto()
config.gpu_options.allow_growth = True

# This is the actual code creating the session. You can omit the config arg
# if you have no configuration to do.
sess = tf.Session(config=config)

sess.graph == g


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
It's common to perform some data cleaning operations on our input data before attempting to use a machine learning algorithm. We'll do that here, subtracting the (empirical) mean and dividing by the standard deviation of the temperature:
Can you write Python code for it?

normalized_temps = (temps - np.mean(temps)) / np.std(temps)


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
Let's fix that.
Can you write Python code for it?


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
Let's start adding some operations to g. An operation is a node of the computation graph. It contains only some light metadata, like "I am an addition operation, and my inputs come from these two other operations." Although Python Operation objects don't actually do anything, we usually think of an operation in terms of what it will cause the execution engine to do after the graph has been completely built and is handed over to TensorFlow to run.
Every operation takes in some number of inputs (0 or more), and produces 0 or more outputs. Its outputs can become the inputs to other operations. Executing an operation can also have side effects, like printing something to the console, recording data to a file, or modifying a variable in memory. Again, all this computation happens after the graph has been completely built. The Operation object itself is simply a description of computation that will take place.
Perhaps the simplest operation we can create is constant. It has zero inputs and one output. When we create a constant operation, we define what that constant output will be (this is stored as metadata on the Operation object we create). TensorFlow's tf.constant function creates a constant operation and adds it to the default graph:
Can you write Python code for it?


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
Notice that Tensorflow has added a constant node, mul/x, to represent the constant 2. Other than that, this should look as expected: an op for a and for b, a multiplication, and an addition. (Remember, Python's operations + and * have been overridden for the tensor type, and they now have side effects of adding operations to the default graph!)
Let's now calculate the derivative of eqn with respect to b.
Can you write Python code for it?

grad = tf.gradients(eqn, b)
g.get_operations()


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
Notice that some of the operations Adam has added are Variables! This is because Adam keeps track of certain statistics across iterations, to implement a sort of "momentum." Because of this, before we can run our minimize_loss operation, we need to make sure Adam's variables are initialized. Rather than painstakingly initialize each one, we can use tf.global_variables_initializer() to add an op to the graph that initializes all variables created so far.
Can you write Python code for it?

initialize_all = tf.global_variables_initializer()

with tf.Session(config=config) as sess:
    # Initialize ALL variables
    sess.run(initialize_all)

    # We will fetch the following tensors each iteration
    to_fetch = {'x': x, 'loss': loss, 'train_op': minimize_loss}
    for i in range(100):
        fetched = sess.run(to_fetch)

        # every tenth step, print our progress
        if i % 10 == 0 or i+1==100:
            print("Iter {}: {}".format(i, fetched))


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
Notice:
1. Variable objects have an initializer property that refers to the Assign operation that gives them their initial value.
2. Fetching an operation instead of a tensor performs its side effect but returns None -- even if the operation has an output.
Let's return to a blank slate.
Can you write Python code for it?

sess.close()
sess2.close()

tf.reset_default_graph()
g = tf.get_default_graph()


I am working on the file "01B_TensorFlow_Fundamentals.ipynb".
The first task is:
Now, there are two operations in the graph. TensorFlow has named them 'Const' and 'Const_1', but you can also give them names yourself by passing a name keyword argument to the tf.constant function. Tensors are named following the formula op_name:output_number, so Const_1:0 means "the 0th output of the Const_1 operation."
Let's create an addition operation to add the two tensors together, using tf.add. Again, this creates an operation in the default graph, and returns a reference to the operation's output -- a tensor.
Can you write Python code for it?

sum_tensor = tf.add(const_tensor, another_const_tensor)
sum_tensor

g.get_operations()

add_op = g.get_operations()[2]
len(add_op.inputs), len(add_op.outputs)