I am working on the file "PythonNotebooks/PlatformPlots/Plot_TimeSeries1.ipynb".
The first task is:
Read variables and units
We assume the data file is present in the following directory:
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Plot_TimeSeries1.ipynb".
The first task is:
The units for the temperature are also changed:
Can you write Python code for it?

temperature_units2 = '($^{\circ}$C)'

fig = plt.figure(figsize=(15, 8))
ax = fig.add_subplot(111)
ax.xaxis.set_major_locator(dates.YearLocator(base=2))
ax.xaxis.set_minor_locator(dates.YearLocator())
plt.plot(time2, temperature_values, linewidth=0.5)
plt.ylabel(temperature_units2, rotation=0., horizontalalignment='right')
plt.title(figure_title)
plt.xlabel('Year')
fig.autofmt_xdate()
plt.grid()


I am working on the file "PythonNotebooks/PlatformPlots/Plot_TimeSeries1.ipynb".
The first task is:
We also mask the temperature values that have quality flag not equal to 1.
Can you write Python code for it?

temperature_values = np.ma.masked_where(temperatureQC != 1, temperature_values)


I am working on the file "PythonNotebooks/PlatformPlots/Plot_TimeSeries1.ipynb".
The first task is:
We use the os mudule to extend the ~.
Can you write Python code for it?

import os
datafile = os.path.expanduser(datafile)

with netCDF4.Dataset(datafile, 'r') as ds:
    time_values = ds.variables['TIME'][:]
    temperature_values = ds.variables['TEMP'][:]
    temperatureQC = ds.variables['TEMP_QC'][:]
    time_units = ds.variables['TIME'].units
    temperature_units = ds.variables['TEMP'].units
    time2 = netCDF4.num2date(time_values, time_units)


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Before making the plot, it is necessary to transform the (lon, lat) coordinates into the new projection that we defined.
Can you write Python code for it?

lon_p, lat_p = m(lon_masked, lat_masked)


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Colored histogram
We start by defining the colormap and the limits for the histogram.
Can you write Python code for it?

cmap = plt.cm.get_cmap('RdYlBu_r')
Y, X = np.histogram(salinity_atdepth_masked, np.arange(32., 40., 0.2))


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Histogram
We start with a simple histogram showing the distribution, using 0.2 unit bins.
Can you write Python code for it?

fig = plt.figure(figsize=(8,6))
plt.hist(salinity_atdepth_masked, np.arange(32., 40., 0.2))
plt.xlabel('Salinity')
plt.ylabel('Number of data points')
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Let's examine the shape of the variables:
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Most of the measurements have a QF=1 (good).
All the profiles have had a quality control applied (0 profiles with QF=0).
4332 values have a QF=9 (not_used). They correspond to the profiles that don't reach the considered depth (count returns the number of non-masked elements of an array):
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Now we take the salinity and its corresponding flags at that level.
Can you write Python code for it?

salinity_atdepth = salinity[:, depthindex]
salinity_atdepth_QC = salinityQC[:, depthindex]


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Operations on data
Extreme values
We can check what the minimal and maximal salinity values are, and their positions. Later we will represent them on a plot.
Can you write Python code for it?

minsalinity, maxsalinity = np.min(salinity_atdepth), np.max(salinity_atdepth)
minsalinityindex, maxsalinityindex = np.argmin(salinity_atdepth), np.argmax(salinity_atdepth)
lonmin, latmin = lon[minsalinityindex], lat[minsalinityindex]
lonmax, latmax = lon[maxsalinityindex], lat[maxsalinityindex]

print 'Minimum salinity = %s at %sE, %sN' % (minsalinity, lonmin, latmin)
print 'Maximum salinity = %s at %sE, %sN' % (maxsalinity, lonmax, latmax)


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Plotting
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib as mpl
from matplotlib import colors


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Python modules
We import modules to read the netCDF file and to perform basic operations on the data.
Can you write Python code for it?

import cf
from netCDF4 import Dataset
import numpy as np
import os


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Quality flags
A quality flag (QF) is attributed to each measurement. We can easily count the measurements for each QF, using a loop from 0 to 9.
Can you write Python code for it?

for qf in np.arange(0, 10):
        print 'QF=%s: %s profiles' % (qf, np.sum(salinity_atdepth_QC == qf))


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Read data
As most of the file content is compliant with the CF conventions, we can use the cf package to read the variable we need:
* Longitude and latitude
* Time
* Depth
* Salinity
Variable and coordinates
Can you write Python code for it?

datafile = os.path.expanduser(datafile)

f = cf.read(datafile)
salinity = f.select('sea_water_salinity')[0].array
ftime = f.select('time')[0].array
lon = f.select('lon')[0].array
lat = f.select('lat')[0].array
depth = f.select('depth')[0].array
f.close()


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Scatter plot
We keep the same colormap as for the histogram and we define the range of values for the scatter plot.
Can you write Python code for it?

norm = colors.Normalize(vmin=32., vmax=39.)


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Select one depth level
We can take the data on a given depth level to perform basic operations and a plot. Let's try with a depth of 100 meters.
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
The plot is created with the same command as before, except that we use m.scatter() instead of plt.scatter().
In addition we will:
* add a coastline,
* shade the continent.
Finally, the colorbar will be placed below the map.
Can you write Python code for it?

fig = plt.figure(figsize=(10,8))
m.scatter(lon_p, lat_p, s=10, c=salinity_atdepth_masked, edgecolor='None', cmap=cmap, norm=norm)
plt.colorbar(scat, extend='both', orientation='horizontal', pad=0.05)
m.fillcontinents(color='gray', lake_color='white')
m.drawcoastlines(linewidth=0.5)
plt.title('Salinity at ' + str(mydepth) + ' meters \n' + str(goodmeasurements) + ' measurements')

plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
The salinity data are organised in profiles, each of them with a time, a longitude and a latitude. We can exmaine the dimension of the coordinates:
Can you write Python code for it?

print str(len(lon)) + ' available profiles'
print 'on ' + str(len(depth)) + ' depth levels.'


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
Then we make the plot.
Can you write Python code for it?

x_span = X.max() - X.min()
C = [cmap(((x - X.min()) / x_span)) for x in X]
fig = plt.figure(figsize=(8, 6))
plt.bar(X[:-1], Y, color=C, width=X[1]-X[0])
plt.xlabel('Salinity')
plt.ylabel('Number of data points')
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
This would mean that we have 26693 measurements available at that depth. However, some values can be masked, because the profile doesn't reach that depth. So to compute the real number of observations, we can use count which is an in-place method of cf.
Can you write Python code for it?

goodmeasurements = salinity_atdepth.count()
print str(goodmeasurements)  + ' data points available'


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
We can do better by including coastline, scale and other labels.
Scatter on a map
We first need to import the Basemap module to plot data on a map, then the projection has to be defined. We select a Mollweide projection centered at 0º longitude, with a crude resolution for the coastline.
Can you write Python code for it?

from mpl_toolkits.basemap import Basemap
m = Basemap(projection='moll', lon_0=0, resolution='c')


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
We create a scatter plot with colors scaled 32 and 39 and indicate the positions of the minimum and maximum salinity (squares).
Can you write Python code for it?

fig = plt.figure(figsize=(10,8))
scat = plt.scatter(lon, lat, s=10, c=salinity_atdepth_masked, edgecolor='None', cmap=cmap, norm=norm)
plt.plot(lonmax, latmax, 'rs', ms=10)
plt.plot(lonmin, latmin, 'bs', ms=10)
plt.colorbar(scat, extend='both')
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
We set the fontsize to 20 pts.
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Read_CORA_dataset.ipynb".
The first task is:
We will only consider the observations with QF=1. To do this, we could try selecting only the values of the array that satisfy the condition:
Can you write Python code for it?

salinity_plot = salinity_atdepth[salinity_atdepth_QC == 1]
print salinity_plot.shape
print lon.shape


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_1.ipynb".
The first task is:
Data
Now we want to load some of the variables: we use the ds.variables
Can you write Python code for it?

time = ds.variables['TIME']
temperature = ds.variables['TEMP']


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_1.ipynb".
The first task is:
It seems that we have not taken into accound the quality flags of the data. We can load the corresponding variable TEMP_QC.
Can you write Python code for it?

temperatureQC = ds.variables['TEMP_QC']
plt.plot(temperatureQC[:])
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_1.ipynb".
The first task is:
Let's examine the variable temperature
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_1.ipynb".
The first task is:
Now we will generate a new plot of the time series using only data with QF = 1.
Can you write Python code for it?

plt.plot(temperature[(temperatureQC[:, 0] == 1), 0])
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_1.ipynb".
The first task is:
Quality flags
Just a quick plot to see everything is fine. More details about the plots will be given later.
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt
plt.plot(temperature)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_1.ipynb".
The first task is:
The meaning of the quality flags is also stored in the file.
Can you write Python code for it?

print 'Flag values: ' + str(temperatureQC.flag_values)
print 'Flag meanings: ' + temperatureQC.flag_meanings


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_1.ipynb".
The first task is:
The resulting plot now seems correct, with values ranging roughly between 10 and 28ºC.
Last thing to remember: close the netCDF file!
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_1.ipynb".
The first task is:
To get the variable attributes:
Can you write Python code for it?

print 'Time units: ' + time.units
print 'Temperature units: ' + temperature.units


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_1.ipynb".
The first task is:
To read the file we need the netCDF4 interface for python.
Can you write Python code for it?

import netCDF4
ds = netCDF4.Dataset(datafile, 'r')


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_1.ipynb".
The first task is:
We can access the global attributes individually:
Can you write Python code for it?

print 'Institution: ' + ds.institution
print 'Reference: ' + ds.institution_references


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_2.ipynb".
The first task is:
Read the file
The method is exactly the same as the previous example, except the file has not been downloaded on your machine. In the present case, the file is updated several times a day, meaning that running the same code at different time of the day could produce different results, as more data will be included.
To be sure the file is closed, we will use the context manager with ... as ....
Can you write Python code for it?

with netCDF4.Dataset(dataurl) as ds:
    temperature_values = ds.variables['WTR_TEM_SBE37'][:]
    time_values = ds.variables['time'][:]
    temperature_units = ds.variables['WTR_TEM_SBE37'].units
    time_units = ds.variables['time'].units


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_2.ipynb".
The first task is:
The variable storing the temperature is not called temp or TEMP, but WTR_TEM_SBE37. The variable names can be obtained by checking the link on the thredds server or by running:
Can you write Python code for it?

with netCDF4.Dataset(dataurl) as ds:
    print ds.variables.keys()


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_2.ipynb".
The first task is:
We will use data from a mooring located in the Ibiza Channel.<br/>
To get the file name, browse the directories in the thredds daa server until you find the file you want (http://thredds.socib.es/thredds/dodsC/mooring/conductivity_and_temperature_recorder/buoy_canaldeibiza-scb_sbe37006/L1/dep0003_buoy-canaldeibiza_scb-sbe37006_L1_latest.nc.html). Then copy the field called Data URL.
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_2.ipynb".
The first task is:
With the time units in seconds since 1st January 1970, it is not easy to see which period the curve represents.<br/>
The solution is to change the time format using the units attribute of the variable and the num2date function.
Can you write Python code for it?

time2 = netCDF4.num2date(time_values, time_units)
print time2[0:5]

plt.plot(time2, temperature_values)
plt.ylabel(temperature_units, fontsize=20)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_3.ipynb".
The first task is:
A simple plot
Can you write Python code for it?

time2 = netCDF4.num2date(time_values, time_units)
plt.plot(time2, temperature_values)
plt.ylabel(temperature_units, fontsize=20)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_3.ipynb".
The first task is:
Read the file
We use the function read. Doing so, we easily obtain a nice summary of the file content.
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_3.ipynb".
The first task is:
To extract the time variable:
Can you write Python code for it?

time = temperature[0].coord('time')
time


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_3.ipynb".
The first task is:
We inspect the corresponding coordinates:
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_3.ipynb".
The first task is:
We see that the file contains 4 variables:
1. temperature
2. salinity
3. conductivity.
Each of them has 4 dimensions: longitude, latitude, time and depth.
Read variable, coordinates and units
From the previous commands we cannot know the name of the variables within the file. But that's not necessary. Temperature can be retrived using its standard name:
Can you write Python code for it?

temperature = f.select('sea_water_temperature')
temperature


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_3.ipynb".
The first task is:
and to get the values:
Can you write Python code for it?

time_values = temperature[0].coord('time').array
time_units = temperature[0].coord('time').units
print time_values[0:20]
print ' '
print 'Time units: ' + time_units


I am working on the file "PythonNotebooks/PlatformPlots/Read_TimeSeries_3.ipynb".
The first task is:
but in other cases (ex: different sensors measuring temperature with data in a common file), one can obtain more than one variable.
To get the temperature values, we select the first element (index = 0 in python, not 1) and convert it into an array.
Can you write Python code for it?

temperature_values = temperature[0].array
temperature_units = temperature[0].units
print temperature_values[0:20]
print 'Temperature units: ' + temperature_units


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_1.ipynb".
The first task is:
2.3 Masking data
Within numpy, there is a specific module to deal with masked array. It offers many possibilities to mask an array, for instance mask an array where greater than a given value masked_greater. Here we will use masked_where, which masks an array where a condition is met. The condition will be stated as POSITION_QC != 1, i.e., values not equal to 1.
Can you write Python code for it?

lon_masked = np.ma.masked_where(POSITION_QC != 1, lon)
lat_masked = np.ma.masked_where(POSITION_QC != 1, lat)


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_1.ipynb".
The first task is:
3. Plotting the data values
As we have the coordinates and the temperature, it is easy to produce a scatter plot.
Can you write Python code for it?

fig = plt.figure()
plt.scatter(lon_masked, lat_masked, s=20, c=temperature, edgecolor='None', cmap=plt.cm.afmhot_r)
plt.colorbar()
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_1.ipynb".
The first task is:
After that we retry the same plot:
Can you write Python code for it?

fig = plt.figure()
plt.plot(lon_masked, lat_masked)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_1.ipynb".
The first task is:
Now we have several possibilities to take the QF into account:
* Remove the bad data.
* Mask the data considered as bad.
2.2 Removing bad data
The idea is to generate new arrays storing the coordinates, only with the values that are considered as good.
Can you write Python code for it?

lon_clean = lon[POSITION_QC == 1]
lat_clean = lat[POSITION_QC == 1]


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_1.ipynb".
The first task is:
That seems ok. Note: with this method, the new arrays have a different size:
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_1.ipynb".
The first task is:
The file path should be adapted to the computer you are working with.
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_1.ipynb".
The first task is:
We load the coordinates (longitude, latitude and depth), and the temperature if it exists. To this end, we use the try...except statement. If the temperature variable were not present in the file, we would get a KeyError.
Can you write Python code for it?

with netCDF4.Dataset(drifterfile) as nc:
    lon = nc.variables['LONGITUDE'][:]
    lat = nc.variables['LATITUDE'][:]
    depth = nc.variables['DEPH'][:]

    try:
        temperature = nc.variables['TEMP'][:, 0]
        print 'Variable temperature is in the file'
    except KeyError:
        print 'No variable temperature in this file'


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_1.ipynb".
The first task is:
With this file it seems that there is a problem with some of the data points. However we considered all the positions, whatever their quality. Now we will only take the good data points, with QF = 1. To do so, we read the variable relative to the quality of the position.
Can you write Python code for it?

with netCDF4.Dataset(drifterfile) as nc:
    POSITION_QC = nc.variables['POSITION_QC'][:]


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_1.ipynb".
The first task is:
With this file we see that indeed, temperature is available.
2. Basic plotting
2.1 First try
Just to visualize what the data look like, we plot the successive positions.
Can you write Python code for it?

fig = plt.figure()
plt.plot(lon, lat)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_2.ipynb".
The first task is:
Loop on the files
We create a loop on the netCDF files located in our directory. Longitude, latitude and dept are read from every file, while the temperature is not always available. For the plot, we only take the data with have a depth dimension equal to 1.
Can you write Python code for it?

fig = plt.figure(figsize=(12, 8))

nfiles_notemp = 0
filelist = sorted(glob.glob(basedir+'*.nc'))
for datafiles in filelist:

    with netCDF4.Dataset(datafiles) as nc:
        lon = nc.variables['LONGITUDE'][:]
        lat = nc.variables['LATITUDE'][:]
        depth = nc.variables['DEPH'][:]

        try:
            temperature = nc.variables['TEMP'][:,0]

            if(depth.shape[1] == 1):

                scat = plt.scatter(lon.mean(), lat.mean(), s=15., c=temperature.mean(), edgecolor='None',
                                   cmap=cmaptemp, norm=normtemp)

        except KeyError:
            # print 'No variable temperature in this file'
            #temperature = np.nan*np.ones_like(lat)
            nfiles_notemp+=1

# Add colorbar and title
cbar = plt.colorbar(scat, extend='both')
cbar.set_label('$^{\circ}$C', rotation=0, ha='left')
plt.title('Temperature from surface drifters\n' + str(year) + '-' + str(month).zfill(2))
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_2.ipynb".
The first task is:
Simple plot
Configuration
We start by defining some options for the scatter plot:
* the range of temperature that will be shown;
* the colormap;
* the ticks to be put on the colorbar.
Can you write Python code for it?

tempmin, tempmax = 5., 30.
cmaptemp = plt.cm.RdYlBu_r
normtemp = colors.Normalize(vmin=tempmin, vmax=tempmax)
tempticks = np.arange(tempmin, tempmax+0.1,2.5)


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_2.ipynb".
The first task is:
The rest of the configuration of the plot can be kept as it was.
Loop on the files
We can copy the part of the code used before. We need to add a line for the projection of the coordinates: lon, lat = m(lon, lat). After the loop we can add the coastline and the continents.
Can you write Python code for it?

fig = plt.figure(figsize=(12, 8))

nfiles_notemp = 0
filelist = sorted(glob.glob(basedir+'*.nc'))
for datafiles in filelist:

    with netCDF4.Dataset(datafiles) as nc:
        lon = nc.variables['LONGITUDE'][:]
        lat = nc.variables['LATITUDE'][:]
        depth = nc.variables['DEPH'][:]

        lon, lat = m(lon, lat)
        try:
            temperature = nc.variables['TEMP'][:,0]

            if(depth.shape[1] == 1):

                scat = m.scatter(lon.mean(), lat.mean(), s=15., c=temperature.mean(), edgecolor='None',
                                   cmap=cmaptemp, norm=normtemp)

        except KeyError:
            # print 'No variable temperature in this file'
            #temperature = np.nan*np.ones_like(lat)
            nfiles_notemp+=1

# Add colorbar and title
cbar = plt.colorbar(scat, extend='both', shrink=0.7)
cbar.set_label('$^{\circ}$C', rotation=0, ha='left')
m.drawcoastlines(linewidth=0.2)
m.fillcontinents(color = 'gray')
plt.title('Temperature from surface drifters\n' + str(year) + '-' + str(month).zfill(2))
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_2.ipynb".
The first task is:
We also counted how many files don't have the temperature variable:
Can you write Python code for it?

print 'Number of files: ' + str(len(filelist))
print 'Number of files without temperature: ' + str(nfiles_notemp)


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_3.ipynb".
The first task is:
After the interpolation and prior to the plot, it is necessary to mask the NaN values that could have been generated.
Can you write Python code for it?

temperature_interp = np.ma.masked_where(np.isnan(temperature_interp), temperature_interp)


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_3.ipynb".
The first task is:
Creation of a gridded field
There are many ways of getting a gridded field from sparsely distributed observations. We will show two simple applications.
Linear interpolation
With scipy, the module interpolate provide many functions to perform interpolations.<br />
In particular, griddata aims to interpolate unstructured D-dimensional data.
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_3.ipynb".
The first task is:
Interpolation in a specific region
If we want to focus on a given area and increase the resolution, we just have to change the interpolation grid:
Can you write Python code for it?

lonmin, lonmax, latmin, latmax = -25., 35., 20.0, 60.
deltalon, deltalat = 2., 2.

lon_interp, lat_interp = np.meshgrid(np.arange(lonmin, lonmax, deltalon), np.arange(latmin, latmax, deltalat))
temperature_interp = griddata((lon_checked, lat_checked), temperature_checked, (lon_interp, lat_interp), method='linear')
temperature_interp = np.ma.masked_where(np.isnan(temperature_interp), temperature_interp)


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_3.ipynb".
The first task is:
Now we have the coordinates and the temperature for all the files, we can save them in a file that we can re-use later.<br/>
To do so, we will use the numpy savetxt function.
The function requires:
* the name of the file where the data will be written,
* the data to be saved, in the form of an array.
To create a unique array from the different arrays (coordinates and temperature), we use the function c_.
Can you write Python code for it?

datafile = './lon_lat_temperature_' + str(year) + '_' + str(month).zfill(2) + '.txt'
np.savetxt(datafile, np.c_[lon_checked, lat_checked, temperature_checked], fmt='%3.5f %3.5f %2.2f')

print 'Data saved in file ' + datafile


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_3.ipynb".
The first task is:
On the map we will also add the data locations.
Can you write Python code for it?

fig = plt.figure(figsize=(12, 8))
m = Basemap(llcrnrlon=lonmin, llcrnrlat=latmin,
            urcrnrlon=lonmax, urcrnrlat=latmax, resolution='i')

lon_interp_map, lat_interp_map = m(lon_interp, lat_interp)
lon_checked_map, lat_checked_map = m(lon_checked, lat_checked)
pcm = m.pcolormesh(lon_interp_map, lat_interp_map, temperature_interp, cmap=cmaptemp, norm=normtemp)
plt.plot(lon_checked_map, lat_checked, 'ko', ms=0.1)

cbar = plt.colorbar(pcm, extend='both', shrink=0.9)
cbar.set_label('$^{\circ}$C', rotation=0, ha='left')
m.drawcoastlines(linewidth=0.2)
m.fillcontinents(color = 'gray')
plt.title('Gridded temperature from surface drifters\n' + str(year) + '-' + str(month).zfill(2), fontsize=20)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_3.ipynb".
The first task is:
Plotting
Same code as in the previous examples. pcolormesh (pseudocolor plot of a 2-D array) is used for the representation of the gridded field.
Can you write Python code for it?

fig = plt.figure(figsize=(12, 8))
tempmin, tempmax = 0., 30.
cmaptemp = plt.cm.RdYlBu_r
normtemp = colors.Normalize(vmin=tempmin, vmax=tempmax)
tempticks = np.arange(tempmin, tempmax+0.1,2.5)

m = Basemap(projection='moll', lon_0=0, resolution='c')
lon_interp_map, lat_interp_map = m(lon_interp, lat_interp)

pcm = m.pcolormesh(lon_interp_map, lat_interp_map, temperature_interp, cmap=cmaptemp, norm=normtemp)

cbar = plt.colorbar(pcm, extend='both', shrink=0.7)
cbar.set_label('$^{\circ}$C', rotation=0, ha='left')
m.drawcoastlines(linewidth=0.2)
m.fillcontinents(color = 'gray')
plt.title('Gridded temperature from surface drifters\n' + str(year) + '-' + str(month).zfill(2))
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_3.ipynb".
The first task is:
Reading all the data
The loop is the same as the 2 previous examples, except that now we will keep the data in arrays that we create empty:
Can you write Python code for it?

lon_checked, lat_checked, temperature_checked = np.array([]), np.array([]), np.array([])

filelist = sorted(glob.glob(basedir+'*.nc'))
k = 0
for datafiles in filelist:

    # print datafiles
    with netCDF4.Dataset(datafiles) as nc:

        lon = nc.variables['LONGITUDE'][:]
        lat = nc.variables['LATITUDE'][:]
        depth = nc.variables['DEPH'][:]
        POSITION_QC = nc.variables['POSITION_QC'][:]

        if depth.shape[1] == 1:

            try:
                TEMP_QC = nc.variables['TEMP_QC'][:, 0]
                temperature = nc.variables['TEMP'][:]
                gooddata = np.where(np.logical_and((TEMP_QC == 1), (POSITION_QC == 1)))

                temperature = temperature[gooddata]

                temperature_checked = np.append(temperature_checked, temperature)
                lon_checked = np.append(lon_checked, lon[gooddata])
                lat_checked = np.append(lat_checked, lat[gooddata])

            except KeyError:
                k += 1
                # print 'No variable temperature in this file'


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_3.ipynb".
The first task is:
The directory where we store the data files:
Can you write Python code for it?

basedir = "~/DataOceano/MyOcean/INSITU_GLO_NRT_OBSERVATIONS_013_030/monthly/" + str(year) + str(month).zfill(2) + '/'
basedir = os.path.expanduser(basedir)


I am working on the file "PythonNotebooks/PlatformPlots/Read_drifter_data_3.ipynb".
The first task is:
The interpolated field is obtained as:
Can you write Python code for it?

temperature_interp = griddata((lon_checked, lat_checked), temperature_checked, (lon_interp, lat_interp), method='linear')


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_drifter.ipynb".
The first task is:
Basic plot
Before starting the plots, we set the font size to 16:
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_drifter.ipynb".
The first task is:
For the plot, we will keep the same projection.<br/>
We also have to set the colormap.
Can you write Python code for it?

cmap = plt.cm.Spectral_r
norm = colors.Normalize(vmin=26, vmax=29)


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_drifter.ipynb".
The first task is:
Scatter plot
If the platforms also measures the temperature, we can create a scatter plot (the color of the dot on the map depends on the measured temperature).<br/>
The temperature variable is loaded the same way as for the coordinates.
Can you write Python code for it?

with netCDF4.Dataset(datadir + datafile) as nc:
    temperature = nc.variables['TEMP'][:,1]
    print temperature.shape


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_drifter.ipynb".
The first task is:
The figure can be improved by adding the landmask and coastline.
Plot on a map
The first thing to do is to create a projection using basemap. We can obtain the bounding box from the previous plot:
Can you write Python code for it?

lonmin, lonmax = ax.get_xlim()
latmin, latmax = ax.get_ylim()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_drifter.ipynb".
The first task is:
The new figure is generated similarly to the previous one:
Can you write Python code for it?

lon2, lat2 = m(lon, lat)
fig = plt.figure(figsize=(8,8))
m.plot(lon2, lat2, 'ko', ms=1)

m.drawcoastlines(linewidth=0.5, zorder=3)
m.fillcontinents(zorder=2)

m.drawparallels(np.arange(-90.,91.,5.), labels=[1,0,0,0], zorder=1)
m.drawmeridians(np.arange(-180.,181.,5.), labels=[0,1,0,0], zorder=1)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_drifter.ipynb".
The first task is:
We extract only the spatial coordinates:
Can you write Python code for it?

with netCDF4.Dataset(datadir + datafile) as nc:
    lon = nc.variables['LONGITUDE'][:]
    lat = nc.variables['LATITUDE'][:]


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_drifter.ipynb".
The first task is:
We will also indicate Start and End labels at the corresponding locations.
Can you write Python code for it?

fig = plt.figure(figsize=(8,8))
ax = plt.subplot(111)
plt.plot(lon, lat, 'ko', ms=1)
plt.text(lon[0], lat[0], 'Start', fontsize=16)
plt.text(lon[-1], lat[-1], 'End', fontsize=16)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_drifter.ipynb".
The first task is:
then we create the projection, slightly enlarging the longitude extension:
Can you write Python code for it?

m = Basemap(projection='merc', llcrnrlat=latmin, urcrnrlat=latmax,
            llcrnrlon=lonmin-5, urcrnrlon=lonmax+5, lat_ts=20, resolution='i')


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring.ipynb".
The first task is:
As the platform is fixed, we will work on time series.<br/>
We will read the time and the sea water temperature variables, as well as their respective units.
Can you write Python code for it?

with netCDF4.Dataset(datadir + datafile) as nc:
    time0 = nc.variables['TIME'][:]
    time0_units = nc.variables['TIME'].units
    temperature = nc.variables['TEMP'][:]
    temperature_units = nc.variables['TEMP'].units

print ('Temperature units = %s' %temperature_units)


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring.ipynb".
The first task is:
Basic plot
For a time series, we simply use the plot function of matplotlib.<br/>
Also, we set the font size to 16:
Can you write Python code for it?

fig = plt.figure(figsize=(8,8))
ax = plt.subplot(111)
plt.plot(time0, temperature, 'k-')
plt.xlabel(time0_units)
plt.ylabel(temperature_units)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring.ipynb".
The first task is:
Data reading
The data file is located in the datafiles directory.
Can you write Python code for it?

datadir = './datafiles/'
datafile = 'GL_TS_MO_62164.nc'


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring.ipynb".
The first task is:
Finally, to avoid to have the overlap of the date ticklabels, we use the autofmt_xdate function.<br/>
Everything is in place to create the improved plot.
Can you write Python code for it?

fig = plt.figure(figsize=(8,8))
ax = plt.subplot(111)
plt.plot(dates, temperature, 'k-')
plt.ylabel(temperature_units)
fig.autofmt_xdate()
plt.title('Temperature at ' + platform_name)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring.ipynb".
The first task is:
The dates contains datetime objects.
We also extract the platform name from the file:
Can you write Python code for it?

with netCDF4.Dataset(datadir + datafile) as nc:
    platform_name = nc.platform_name


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring_NorthWestShelf.ipynb".
The first task is:
As the platform is fixed, we will work on time series.<br/>
We will read the time and the sea water temperature variables, as well as their respective units.
Can you write Python code for it?

with netCDF4.Dataset(datadir + datafile) as nc:
    time0 = nc.variables['TIME'][:]
    time0_units = nc.variables['TIME'].units
    sealevel = nc.variables['SLEV'][:]
    sealevel_units = nc.variables['SLEV'].units


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring_NorthWestShelf.ipynb".
The first task is:
As we plotted all the values, regardless of the quality flags, the result is not meaningful.
Select data according to Quality Flag
We have to load the corresponding variables:
Can you write Python code for it?

with netCDF4.Dataset(datadir + datafile) as nc:
    sealevel_QC = nc.variables['SLEV_QC'][:]


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring_NorthWestShelf.ipynb".
The first task is:
Data reading
The data file is located in the datafiles directory.
Can you write Python code for it?

datadir = './datafiles/'
datafile = 'NO_TS_MO_HoekVanHollandTG.nc'


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring_NorthWestShelf.ipynb".
The first task is:
Finally, to avoid to have the overlap of the date ticklabels, we use the autofmt_xdate function.<br/>
Everything is in place to create the improved plot.
Can you write Python code for it?

fig = plt.figure(figsize=(15,8))
ax = plt.subplot(111)
plt.plot(dates, sealevel, 'ko-', lw=0.2, ms=1)
plt.ylabel(sealevel_units)
plt.title('Sea level at station HoekVanHolland')
fig.autofmt_xdate()
plt.grid()
plt.savefig('NO_TS_MO_HoekVanHollandTG.png', dpi=300)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring_NorthWestShelf.ipynb".
The first task is:
Let's check the plot again:
Can you write Python code for it?

fig = plt.figure(figsize=(8,8))
ax = plt.subplot(111)
plt.plot(time0, sealevel, 'k-')
plt.xlabel(time0_units)
plt.ylabel(sealevel_units)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring_NorthWestShelf.ipynb".
The first task is:
Let's have a look at the dimension of the array
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring_NorthWestShelf.ipynb".
The first task is:
Still bad. It seems the QF don't allow us to filter out the data.<br>
Let's have a closer look at it:
Can you write Python code for it?

fig = plt.figure(figsize=(8,8))
ax = plt.subplot(111)
plt.plot(time0, sealevel_QC, 'ko')
plt.xlabel(time0_units)
plt.ylabel('Quality flags')
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring_NorthWestShelf.ipynb".
The first task is:
The first number corresponds to the time and the second to the depth.
Basic plot
For a time series, we simply use the plot function of matplotlib.<br>
The 1st line change the font size to 16 (see matplotlib.RcParams).
Can you write Python code for it?

rcParams.update({'font.size': 16})
fig = plt.figure(figsize=(8,8))
ax = plt.subplot(111)
plt.plot(time0, sealevel, 'k-')
plt.xlabel(time0_units)
plt.ylabel(sealevel_units)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring_NorthWestShelf.ipynb".
The first task is:
The units set for the time is maybe not the easiest to read.

However the netCDF4 module offers easy solutions to properly convert the time.
Converting time units
NetCDF4 provides the function num2date to convert the time vector into dates.

http://unidata.github.io/netcdf4-python/#section7
Can you write Python code for it?

from netCDF4 import num2date
dates = num2date(time0, units=time0_units)
print dates[:5]


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring_NorthWestShelf.ipynb".
The first task is:
The values are either 1 (good data) or 9 (missing values), never a value indicating suspect or bad data.
A possible solution is to keep only sea level measurements lower than, let's say, 3 meters.
Can you write Python code for it?

sealevel = np.ma.masked_outside(sealevel, -3., 3.)

fig = plt.figure(figsize=(15,8))
ax = plt.subplot(111)
plt.plot(time0, sealevel, 'ko-', lw=0.2, ms=1)
plt.xlabel(time0_units)
plt.ylabel(sealevel_units)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring_NorthWestShelf.ipynb".
The first task is:
Variable units and dimension
Can you write Python code for it?

print('Sea level units = %s' %sealevel_units)


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_mooring_NorthWestShelf.ipynb".
The first task is:
and we keep only the sea level values with a flag equal to 1.<br>
To do so, we use the masked arrays module.
Can you write Python code for it?

sealevel = np.ma.masked_where(sealevel_QC!=1, sealevel)


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_profiler.ipynb".
The first task is:
3-D plot
We illustrate with a simple example how to have a 3-dimensional representation of the profiles.<br/>
First we import the required modules.
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_profiler.ipynb".
The first task is:
Adjusted variables
The profiler files also contain adjusted variables (pressure, temperature, salinity) which correspond to the variable after the application of a correction.
We will repeat the T-S diagram with the adjusted variables.
Can you write Python code for it?

with netCDF4.Dataset(datadir + datafile) as nc:
    temperature_adj = nc.variables['TEMP_ADJUSTED'][:]
    temperature_adj_name = nc.variables['TEMP_ADJUSTED'].long_name
    salinity_adj = nc.variables['PSAL_ADJUSTED'][:]
    salinity_adj_name = nc.variables['PSAL_ADJUSTED'].long_name

fig = plt.figure(figsize=(8,8))
ax = plt.subplot(111)
plt.plot(temperature, salinity, 'r.', markersize=2)
plt.plot(temperature_adj, salinity_adj, 'k.', markersize=2)
plt.xlabel("%s (%s)" % (temperature_adj_name, temperature_units))
plt.ylabel("%s (%s)" % (salinity_adj_name, salinity_units))
plt.ylim(22.5, 42)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_profiler.ipynb".
The first task is:
Location of the profiles
In this first plot we want to see the location of the profiles obtained with the profiler.
Can you write Python code for it?

mpl.rcParams.update({'font.size': 16})
fig = plt.figure(figsize=(8,8))
ax = plt.subplot(111)
plt.plot(lon, lat, 'ko', ms=1)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_profiler.ipynb".
The first task is:
Profile plot
We extract the salinity and the vertical coordinate used in the profile, the pressure in this case.<br/>
For the x-dimension, we will use the time.
Can you write Python code for it?

with netCDF4.Dataset(datadir + datafile) as nc:
    pressure = nc.variables['PRES'][:]
    pressure_name = nc.variables['PRES'].long_name
    pressure_units = nc.variables['PRES'].units
    salinity = nc.variables['PSAL'][:]
    salinity_name = nc.variables['PSAL'].long_name
    salinity_units = nc.variables['PSAL'].units
    time = nc.variables['TIME'][:]
    time_units = nc.variables['TIME'].units
    time_name = nc.variables['TIME'].long_name
    dates = num2date(time, units=time_units)


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_profiler.ipynb".
The first task is:
The new figure is generated similarly to the previous one:
Can you write Python code for it?

lon2, lat2 = m(lon, lat)
fig = plt.figure(figsize=(8,8))
m.plot(lon2, lat2, 'ko', ms=3)

m.drawcoastlines(linewidth=0.5, zorder=3)
m.fillcontinents(zorder=2)

m.drawparallels(np.arange(-90.,91.,2.), labels=[1,0,0,0], zorder=1)
m.drawmeridians(np.arange(-180.,181.,3.), labels=[0,0,1,0], zorder=1)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_profiler.ipynb".
The first task is:
The salinity has it's highest values near surface, but we can also see an increase between 800 and 1000 m depth.
T-S diagram
For the temperature-salinity (T-S) diagram we need to load the temperature variable.
Can you write Python code for it?

with netCDF4.Dataset(datadir + datafile) as nc:
    temperature = nc.variables['TEMP'][:]
    temperature_name = nc.variables['TEMP'].long_name
    temperature_units = nc.variables['TEMP'].units


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_profiler.ipynb".
The first task is:
The x and y labels for the plot are directly taken from the netCDF variable attributes.
Can you write Python code for it?

fig = plt.figure(figsize=(8,8))
ax = plt.subplot(111)
plt.plot(temperature, salinity, 'k.', markersize=2)
plt.xlabel("%s (%s)" % (temperature_name, temperature_units))
plt.ylabel("%s (%s)" % (salinity_name, salinity_units))
plt.ylim(22.5, 42)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_profiler.ipynb".
The first task is:
Then the plot is easily obtained by specifying the coordinates (x, y, z) and the variables (salinity) to be plotted.
Can you write Python code for it?

fig = plt.figure(figsize=(8,8))
ax = fig.add_subplot(111, projection='3d')
for ntime in range(0, ntimes):
    plt.scatter(lon[ntime]*np.ones(ndepths), lat[ntime]*np.ones(ndepths), zs=-pressure[ntime,:], zdir='z',
                s=20, c=salinity[ntime,:], edgecolor='None', cmap=cmap, norm=norm)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_profiler.ipynb".
The first task is:
We also have to set the colormap and the limits for the salinity.
Can you write Python code for it?

cmap = plt.cm.Spectral_r
norm = colors.Normalize(vmin=34, vmax=36)


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_profiler.ipynb".
The first task is:
We extract only the spatial coordinates:
Can you write Python code for it?

with netCDF4.Dataset(datadir + datafile) as nc:
    lon = nc.variables['LONGITUDE'][:]
    lat = nc.variables['LATITUDE'][:]


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_profiler.ipynb".
The first task is:
We specify the coordinates (time and pressure) and the salinity as the arguments of the scatter plot:
* s=10 indicate the size of the dots,
* c=salinity indicates which variable is used as the z-dimension (color)
* edgecolor='None' means that no color is applied around the edge of the marker
* cmap=cmap sets the colormap to cmap, defined before and
* norm=norm sets the limits for the color scale.
Can you write Python code for it?

fig = plt.figure(figsize=(8,8))
ax = plt.subplot(111)
for ntime in range(0, len(time)):
    plt.scatter(time[ntime]*np.ones_like(pressure[ntime,]), pressure[ntime,:],
                s=20, c=salinity[0,:], edgecolor='None', cmap=cmap, norm=norm)
plt.ylim(0.0, 1200)
plt.gca().invert_yaxis()
plt.colorbar(extend='both')
fig.autofmt_xdate()
plt.ylabel("%s (%s)" % (pressure_name, pressure_units))
plt.xlabel("%s (%s)" % (time_name, time_units))
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_vessel.ipynb".
The first task is:
3-D plot
We illustrate with a simple example how to have a 3-dimensional representation of the profiles.<br/>
First we import the required modules.
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_vessel.ipynb".
The first task is:
Data reading
The data file is located in the datafiles directory.
Can you write Python code for it?

datadir = './datafiles/'
datafile = 'GL_PR_ML_EXRE0065_2010.nc'


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_vessel.ipynb".
The first task is:
Once we have the projection, the coordinates have to be changed into this projection:
Can you write Python code for it?


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_vessel.ipynb".
The first task is:
Profile plot
We read the temperature, salinity and depth variables.
Can you write Python code for it?

with netCDF4.Dataset(datadir + datafile) as nc:
    depth = nc.variables['DEPH'][:]
    temperature = nc.variables['TEMP'][:]
    temperature_name = nc.variables['TEMP'].long_name
    temperature_units = nc.variables['TEMP'].units
    salinity = nc.variables['PSAL'][:]
    salinity_name = nc.variables['PSAL'].long_name
    salinity_units = nc.variables['PSAL'].units
    time = nc.variables['TIME'][:]
    time_units = nc.variables['TIME'].units

print depth.shape
print temperature.shape

nprofiles, ndepths = depth.shape

fig = plt.figure(figsize=(8,8))
ax = plt.subplot(111)
for nn in range(0, nprofiles):
    plt.plot(temperature[nn,:], depth[nn,:], 'k-', linewidth=0.5)
plt.gca().invert_yaxis()
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_vessel.ipynb".
The first task is:
The graph confirms that we have data obtained during different periods:
* July-August 2010,
* December 2010.
T-S diagram
The x and y labels for the plot are directly taken from the netCDF variable attributes.
Can you write Python code for it?

fig = plt.figure(figsize=(8,8))
ax = plt.subplot(111)
plt.plot(temperature, salinity, 'ko', markersize=2)
plt.xlabel("%s (%s)" % (temperature_name, temperature_units))
plt.ylabel("%s (%s)" % (salinity_name, salinity_units))
plt.ylim(32, 36)
plt.grid()
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_vessel.ipynb".
The first task is:
The locations of the vessel stations are added on a map with the coastline and the land mask.
Can you write Python code for it?

mpl.rcParams.update({'font.size': 16})
fig = plt.figure(figsize=(8,8))
m.plot(lon2, lat2, 'ko', ms=2)

m.drawcoastlines(linewidth=0.5, zorder=3)
m.fillcontinents(zorder=2)

m.drawparallels(np.arange(-90.,91.,0.5), labels=[1,0,0,0], zorder=1)
m.drawmeridians(np.arange(-180.,181.,0.5), labels=[0,0,1,0], zorder=1)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_vessel.ipynb".
The first task is:
Then the plot is easily obtained by specifying the coordinates (x, y, z) and the variables (salinity) to be plotted.
Can you write Python code for it?

cmap = plt.cm.Spectral_r
norm = colors.Normalize(vmin=32, vmax=36)

fig = plt.figure(figsize=(8,8))
ax = fig.add_subplot(111, projection='3d')
for ntime in range(0, nprofiles):
    plt.scatter(lon[ntime]*np.ones(ndepths), lat[ntime]*np.ones(ndepths), zs=-depth[ntime,:], zdir='z',
                s=20, c=salinity[ntime,:], edgecolor='None', cmap=cmap, norm=norm)
plt.colorbar(cmap=cmap, norm=norm)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_CMEMS_vessel.ipynb".
The first task is:
We extract only the spatial coordinates:
Can you write Python code for it?

with netCDF4.Dataset(datadir + datafile) as nc:
    lon = nc.variables['LONGITUDE'][:]
    lat = nc.variables['LATITUDE'][:]
print lon.shape


I am working on the file "PythonNotebooks/PlatformPlots/plot_positions_latest_global.ipynb".
The first task is:
Basic plot
We read the coordinate variables from the file and plot them on the map.
Then we loop on the files:
Can you write Python code for it?

lon = np.zeros(nfiles)
lat = np.zeros(nfiles)
for count, datafiles in enumerate(filelist):
    with netCDF4.Dataset(datafiles) as nc:
        lon[count] = np.nanmean(nc.variables['LONGITUDE'][:])
        lat[count] = np.nanmean(nc.variables['LATITUDE'][:])


I am working on the file "PythonNotebooks/PlatformPlots/plot_positions_latest_global.ipynb".
The first task is:
File reading
We create a list of the files available in the directory:
Can you write Python code for it?

datadir = os.path.expanduser(datadir)
filelist = sorted(glob.glob(datadir + '*nc'))
nfiles = len(filelist)
print("Number of files = %u" % (nfiles))


I am working on the file "PythonNotebooks/PlatformPlots/plot_positions_latest_global.ipynb".
The first task is:
Improved plot
A projection is created, so we have access to a coastline, land mask, ...
Can you write Python code for it?

from mpl_toolkits.basemap import Basemap
m = Basemap(projection='moll', lon_0=0, resolution='c')
lon_p, lat_p = m(lon, lat)

import matplotlib
font = {'family' : 'serif',
        'size'   : 16}

matplotlib.rc('font', **font)

fig = plt.figure(figsize=(10,8))
m.plot(lon_p, lat_p, 'ko', markersize=2)
m.drawparallels(np.arange(-80, 80., 20.),labels=[True,False,False,True], zorder=2)
m.drawmeridians(np.arange(-180, 180., 30.), zorder=2)
m.fillcontinents(color='gray', zorder=3)
m.drawcoastlines(linewidth=0.5)
plt.title('Platform locations on\n December 1st, 2015', fontsize=20)

plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_positions_latest_global.ipynb".
The first task is:
Now we can loop on the files (just the first 10) to check if it's okay:
Can you write Python code for it?

for datafiles in filelist[0:10]:
    print os.path.basename(datafiles)


I am working on the file "PythonNotebooks/PlatformPlots/plot_positions_latest_global.ipynb".
The first task is:
Pie chart
It is also useful to have a pie chart showing the relative importance of each data type.
Can you write Python code for it?

fig2 = plt.figure(figsize=(10, 8))
plt.pie(nfiles, labels=datatypename.values(), colors=colorlist,
        autopct='%1.1f%%', startangle=90)
plt.savefig('./figures/platform_piechart_20151201.png', dpi=300)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_positions_latest_global.ipynb".
The first task is:
We also mask the bad values of coordinates:
Can you write Python code for it?

lon = np.ma.masked_outside(lon, -180., 360.)
lat = np.ma.masked_outside(lat, -90., 90.)

fig = plt.figure(figsize=(8, 8))
plt.plot(lon, lat, 'ko', markersize=2)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_positions_latest_global.ipynb".
The first task is:
We create a list of colors for the plot:
Can you write Python code for it?

colorlist = ['red', 'yellowgreen', 'lightskyblue', 'gold', 'violet', 'lightgray']

m = Basemap(projection='robin', lat_0=0, lon_0=0., resolution='l')

fig = plt.figure(figsize=(10,8))
ax = plt.subplot(111)
nfiles = np.zeros(len(datatypename.keys()))
for ntype, datatype in enumerate(datatypename.keys()):
    filelist = sorted(glob.glob(datadir + '*LATEST_*_' + datatype + '*nc'))
    nfiles[ntype] = len(filelist)
    lon = np.zeros(nfiles[ntype])
    lat = np.zeros(nfiles[ntype])
    for count, datafiles in enumerate(filelist):
        with netCDF4.Dataset(datafiles) as nc:
            lon[count] = np.nanmean(nc.variables['LONGITUDE'][:])
            lat[count] = np.nanmean(nc.variables['LATITUDE'][:])
    lon_p, lat_p = m(lon, lat)
    m.plot(lon_p, lat_p, 'o', markerfacecolor=colorlist[ntype], markeredgecolor=colorlist[ntype],
           markersize=3, label=datatypename[datatype] + ': ' + str(int(nfiles[ntype])) + ' files')
box = ax.get_position()
ax.set_position([box.x0, box.y0, box.width * 0.8, box.height])
ax.legend(loc='center left', bbox_to_anchor=(1, 0.5),fontsize=14)

m.drawparallels(np.arange(-80, 80., 30.),labels=[True,False,False,True], zorder=2)
m.drawmeridians(np.arange(-180, 180., 90.),labels=[True,False,False,True], zorder=2)
m.fillcontinents(color='gray', zorder=3)
m.drawcoastlines(linewidth=0.5)
plt.title('Platform locations on\n December 1st, 2015', fontsize=20)
plt.savefig('./figures/platform_types_20151201.png', dpi=300)
plt.show()


I am working on the file "PythonNotebooks/PlatformPlots/plot_positions_latest_global.ipynb".
The first task is:
We see that some data types are not often present in the list, so we will only take those with at least 50 data files.<br>
We create a dictionnary with the abbreviation and the corresponding data types:
Can you write Python code for it?

datatypename = {'DB': 'Drifting buoys', 'DC': 'Drifting buoy reporting calculated sea water current',
                'MO': 'Fixed buoys or mooring time series', 'PF': 'Profiling floats vertical profiles',
                'RF': 'River flows', 'TE': 'TESAC messages on GTS'}
datatypename.keys()


I am working on the file "PythonNotebooks/ROC_and_CI/Comp_Vision_Ishan_Handa_ROC_and_CI_Beaver.ipynb".
The first task is:
Now calculating TPR and FNR for the first positive test
Can you write Python code for it?

sample_size = min(positive_length, negative_length)
# all_TPRs and all_FPRs will be used later to evalute confidence intervals for each threshold level
all_TPRs = [[None for _ in range(sample_size)] for _ in range(len(thresholds))]
all_FPRs = [[None for _ in range(sample_size)] for _ in range(len(thresholds))]

for j in range(0, sample_size):
    current_positive_sample = positive_data[j]
    TPRs = [None] * len(thresholds)

    current_negative_sample = negative_data[j]
    FPRs = [None] * len(thresholds)

    for i in range(0, len(thresholds)):
        test_positive = current_positive_sample[current_positive_sample >= thresholds[i]]
        tpr = len(test_positive) / len(current_positive_sample)
        TPRs[i] = tpr # This is the calculated TPR value for threshold level i in sample j
        all_TPRs[i][j] = tpr # The calculated TPR value is also added to all_TPR values for this threshold.(Used later to calculate confidence intervals)

        test_negative = current_negative_sample[current_negative_sample >= thresholds[i]]
        fpr = len(test_negative) / len(current_negative_sample)
        FPRs[i] = fpr # This is the calculated FPR value for threshold level i in sample j
        all_FPRs[i][j] = fpr

    print('\n\nPLOTTING ROC FOR CASE: {}'.format(j))
    plt.scatter(FPRs, TPRs, color='red')
    plt.show()

import scipy as sp
import scipy.stats

# Function to calculate confidence interval. By default it calculated 80%.
def mean_confidence_interval(data, confidence=0.8):
    a = 1.0*numpy.array(data)
    n = len(a)
    m, se = numpy.mean(a), scipy.stats.sem(a)
    h = se * sp.stats.t._ppf((1+confidence)/2., n-1)
    return m, max(0.0, m-h), min(1.0 ,m+h)

# Calculating and printing Confidence Intervals for all threshold values.

thresh_s = []
ci_lower_TPR = []
ci_lower_FPR = []
ci_TPR_diff = []
ci_upper_TPR = []
ci_upper_FPR = []
ci_FPR_diff = []

print("\n\nConfidence Intervals for TPRs:")
for i in range(0, len(thresholds)):
    mean_tpr, lower_tpr, upper_tpr =...


I am working on the file "PythonNotebooks/ROC_and_CI/Comp_Vision_Ishan_Handa_ROC_and_CI_Beaver.ipynb".
The first task is:
Now we need to define the threshold points over which the ROC will be plotted.
Can you write Python code for it?

# Here we are defining preset threshold levels for which TPR and FPR values will be calculated
thresholds = numpy.arange(0.0,1.0,0.05)
print('Thresholds: {}'.format(thresholds))


I am working on the file "PythonNotebooks/ROC_and_CI/Comp_Vision_Ishan_Handa_ROC_and_CI_Hedgehog.ipynb".
The first task is:
Now calculating TPR and FNR for the first positive test
Can you write Python code for it?


I am working on the file "PythonNotebooks/ROC_and_CI/Comp_Vision_Ishan_Handa_ROC_and_CI_Hedgehog.ipynb".
The first task is:
Now we need to define the threshold points over which the ROC will be plotted.
Can you write Python code for it?


I am working on the file "PythonNotebooks/ROC_and_CI/Comp_Vision_Ishan_Handa_ROC_and_CI_Quail.ipynb".
The first task is:
Now calculating TPR and FNR for the first positive test
Can you write Python code for it?


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_download.ipynb".
The first task is:
1. index history example (NRT & REP products)
Can you write Python code for it?

ftp=ftplib.FTP(distribution_unit,user,password)
ftp.cwd("Core")
ftp.cwd(product_name)
aimedFileName = 'index_history.txt'
local_filename = aimedFileName
local_file = open(local_filename, 'wb')
ftp.retrbinary('RETR ' + aimedFileName, local_file.write)
local_file.close()
ftp.quit()
#ready when 221 Goodbye.!


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_download.ipynb".
The first task is:
3. index latest example (NRT products)
Can you write Python code for it?

ftp=ftplib.FTP(distribution_unit,user,password)
ftp.cwd("Core")
ftp.cwd(product_name)
aimedFileName = 'index_latest.txt'
local_filename = aimedFileName
local_file = open(local_filename, 'wb')
ftp.retrbinary('RETR ' + aimedFileName, local_file.write)
local_file.close()
ftp.quit()
#ready when 221 Goodbye.!


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_download.ipynb".
The first task is:

QUICK VIEW

Can you write Python code for it?

import numpy as np
import pandas as pd
from random import randint

index_file = 'index_history.txt' #choose index file to look at a ramdom line

index = np.genfromtxt(index_file, skip_header=6, unpack=False, delimiter=',', dtype=None,
           names=['catalog_id', 'file_name', 'geospatial_lat_min', 'geospatial_lat_max',
                     'geospatial_lon_min', 'geospatial_lon_max',
                     'time_coverage_start', 'time_coverage_end',
                     'provider', 'date_update', 'data_mode', 'parameters'])

dataset = randint(0,len(index)) #ramdom line of the index file

values = [index[dataset]['catalog_id'], ''+index[dataset]['file_name']+'', index[dataset]['geospatial_lat_min'], index[dataset]['geospatial_lat_max'],
                 index[dataset]['geospatial_lon_min'], index[dataset]['geospatial_lon_max'], index[dataset]['time_coverage_start'],
                 index[dataset]['time_coverage_end'], index[dataset]['provider'], index[dataset]['date_update'], index[dataset]['data_mode'],
                 index[dataset]['parameters']]
headers = ['catalog_id', 'file_name', 'geospatial_lat_min', 'geospatial_lat_max',
                     'geospatial_lon_min', 'geospatial_lon_max',
                     'time_coverage_start', 'time_coverage_end',
                     'provider', 'date_update', 'data_mode', 'parameters']
df = pd.DataFrame(values, index=headers, columns=[dataset])
df.style


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_boundingbox.ipynb".
The first task is:
2. netCDF filtering/selection
Can you write Python code for it?

selected_netCDFs = [];
for netCDF in index:
        file_name = netCDF['file_name']

        geospatial_lat_min = float(netCDF['geospatial_lat_min'])
        geospatial_lat_max = float(netCDF['geospatial_lat_max'])
        geospatial_lon_min = float(netCDF['geospatial_lon_min'])
        geospatial_lon_max = float(netCDF['geospatial_lon_max'])

        bounding_box = shapely.geometry.box(geospatial_lon_min, geospatial_lat_min, geospatial_lon_max, geospatial_lat_max)
        bounding_box_centroid = bounding_box.centroid

        if (targeted_bounding_box.contains(bounding_box_centroid)):
            selected_netCDFs.append(file_name)

print("total: " +str(len(selected_netCDFs)))


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_boundingbox.ipynb".
The first task is:

DOWNLOAD

Index file download
Can you write Python code for it?

import ftplib

ftp=ftplib.FTP(distribution_unit,user,password)
ftp.cwd("Core")
ftp.cwd(product_name)
remote_filename= index_file
local_filename = remote_filename
local_file = open(local_filename, 'wb')
ftp.retrbinary('RETR ' + remote_filename, local_file.write)
local_file.close()
ftp.quit()
#ready when 221 Goodbye.!


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_boundingbox.ipynb".
The first task is:

FILTERING CRITERIA

Can you write Python code for it?

from shapely.geometry import box, multipoint
import shapely


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_boundingbox.ipynb".
The first task is:

QUICK VIEW

Can you write Python code for it?

import numpy as np
import pandas as pd
from random import randint

index = np.genfromtxt(index_file, skip_header=6, unpack=False, delimiter=',', dtype=None,
           names=['catalog_id', 'file_name', 'geospatial_lat_min', 'geospatial_lat_max',
                     'geospatial_lon_min', 'geospatial_lon_max',
                     'time_coverage_start', 'time_coverage_end',
                     'provider', 'date_update', 'data_mode', 'parameters'])

dataset = randint(0,len(index)) #ramdom line of the index file

values = [index[dataset]['catalog_id'], ''+index[dataset]['file_name']+'', index[dataset]['geospatial_lat_min'], index[dataset]['geospatial_lat_max'],
                 index[dataset]['geospatial_lon_min'], index[dataset]['geospatial_lon_max'], index[dataset]['time_coverage_start'],
                 index[dataset]['time_coverage_end'], index[dataset]['provider'], index[dataset]['date_update'], index[dataset]['data_mode'],
                 index[dataset]['parameters']]
headers = ['catalog_id', 'file_name', 'geospatial_lat_min', 'geospatial_lat_max',
                     'geospatial_lon_min', 'geospatial_lon_max',
                     'time_coverage_start', 'time_coverage_end',
                     'provider', 'date_update', 'data_mode', 'parameters']
df = pd.DataFrame(values, index=headers, columns=[dataset])
df.style


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_boundingbox.ipynb".
The first task is:
Regarding the above glimpse, it is posible to filter by 12 criteria. As example we will setup next a filter to only download those files that contains data within a defined boundingbox.
1. Aimed boundingbox
Can you write Python code for it?

targeted_geospatial_lat_min = 55.0   # enter min latitude of your bounding box
targeted_geospatial_lat_max =  70.0   # enter max latitude of your bounding box
targeted_geospatial_lon_min = 12.0  # enter min longitude of your bounding box
targeted_geospatial_lon_max =  26.00  # enter max longitude of your bounding box

targeted_bounding_box = box(targeted_geospatial_lon_min, targeted_geospatial_lat_min, targeted_geospatial_lon_max, targeted_geospatial_lat_max)


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_by_datatype.ipynb".
The first task is:
2. netCDF filtering/selection

Remember that depending on the folder we are, the data_type tag in the file_name is at an specific place: for history directory is at position 2 and for monthly and latest directory it would be at position 3. See more at naming convention or PUM
Can you write Python code for it?

#read file lines (iterate over them)
selected_netCDFs = [];
for netCDF in index:
    file_name = netCDF['file_name'].decode('utf-8')
    last_idx_slash = file_name.rfind('/')
    ncdf_file_name = file_name[last_idx_slash+1:]

    #set up a selection criteria: i.e specific data_type
    #history: position 2
    #monthly and latest: position 3
    position = 3
    data_type = ncdf_file_name.split('_')[position] #index_latest (above)
    aimed_data_type = 'MO'#choose a data_type: i.e mooring

    if data_type == aimed_data_type :
        selected_netCDFs.append(file_name)
print("total: " +str(len(selected_netCDFs)))


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_by_datatype.ipynb".
The first task is:

FILTERING CRITERIA

Regarding the above glimpse, it is posible to filter by 12 criteria. As example we will setup next a filter to only download those files that contains an data_type.
1. Aimed data_type
Can you write Python code for it?


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_fileUpdate.ipynb".
The first task is:
1. Number of minutes
Can you write Python code for it?

time_lapse = 10 #all files updated within the last 10 hours will be downloaded
end_date = datetime.datetime.today()
ini_date = end_date - datetime.timedelta(hours=time_lapse)


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_fileUpdate.ipynb".
The first task is:
<h3> SELECTION DOWNLOAD </h3>
Can you write Python code for it?

for nc in selected_netCDFs:
    last_idx_slash = nc.rfind('/')
    ncdf_file_name = nc[last_idx_slash+1:]
    folders = nc.split('/')[3:len(file_name.split('/'))-1]
    host = nc.split('/')[2] #distribution unit

    ftp=ftplib.FTP(host,user,password)
    for folder in folders:
        ftp.cwd(folder)

    local_file = open(ncdf_file_name, 'wb')
    ftp.retrbinary('RETR '+ncdf_file_name, local_file.write)
    local_file.close()
    ftp.quit()


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_fileUpdate.ipynb".
The first task is:

DOWNLOAD

Index file download
Can you write Python code for it?


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_parameter.ipynb".
The first task is:
2. netCDF filtering/selection
Can you write Python code for it?

#read file lines (iterate over them)
selected_netCDFs = [];
for netCDF in index:
    file_name = netCDF['file_name'].decode('utf-8')
    parameters = netCDF['parameters'].decode('utf-8')
    param_list = parameters.split(' ');
    #set up a selection criteria: i.e specific parameter
    if 'PSAL' in param_list:
        selected_netCDFs.append(file_name)
print("total: " +str(len(selected_netCDFs)))


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_parameter.ipynb".
The first task is:
<h3> SELECTION DOWNLOAD </h3>
Can you write Python code for it?

for nc in selected_netCDFs:
    last_idx_slash = file_name.rfind('/')
    ncdf_file_name = file_name[last_idx_slash+1:]
    folders = file_name.split('/')[3:len(file_name.split('/'))-1]
    host = file_name.split('/')[2] #distribution unit

    ftp=ftplib.FTP(host,user,password)
    for folder in folders:
    ftp.cwd(folder)

    local_file = open(ncdf_file_name, 'wb')
    ftp.retrbinary('RETR '+ncdf_file_name, local_file.write)
    local_file.close()
    ftp.quit()


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_parameter.ipynb".
The first task is:

FILTERING CRITERIA

Regarding the above glimpse, it is posible to filter by 12 criteria. As example we will setup next a filter to only download those files that contains a specific variable (PSAL). More variables at Insitu Tac parameters list.
1. Aimed parameter
Can you write Python code for it?


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_parameter.ipynb".
The first task is:

QUICK VIEW

Reading a random line of the index file to know more about the information it contains.
Can you write Python code for it?


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_platformCategory.ipynb".
The first task is:
2. netCDF filtering/selection
Can you write Python code for it?

selected_netCDFs = [];

for netCDF in index:
    file_name = netCDF['file_name']

    folders = file_name.split('/')[3:len(file_name.split('/'))-1]
    category = file_name.split('/')[3:len(file_name.split('/'))-1][len(file_name.split('/')[3:len(file_name.split('/'))-1])-1]

    if (category == targeted_category):
        selected_netCDFs.append(file_name)

print("total: " +str(len(selected_netCDFs)))


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_platformCategory.ipynb".
The first task is:

DOWNLOAD

Index file download
Can you write Python code for it?

import ftplib

ftp=ftplib.FTP(distribution_unit,user,password)
ftp.cwd("Core")
ftp.cwd(product_name)
local_file = open(index_file, 'wb')
ftp.retrbinary('RETR ' + index_file, local_file.write)
local_file.close()
ftp.quit()
#ready when 221 Goodbye.!


I am working on the file "PythonNotebooks/indexFileNavigation/index_file_navigation_platformCategory.ipynb".
The first task is:

FILTERING CRITERIA

Regarding the above glimpse, it is posible to filter by 12 criteria. As example we will setup next a filter to only download those files that contains data within a defined boundingbox.
1. Aimed category
Can you write Python code for it?


I am working on the file "PythonQnA_10_more_comprehensions.ipynb".
The first task is:
Exercise 0 (students_test: 1 point). Write some code that computes a new list named students[:], which holds the names of the students as they from "top to bottom" in the table.
Can you write Python code for it?

### BEGIN SOLUTION (LIST COMPREHENSION)
students = [L[0] for L in grades[1:]]
### END SOLUTION

### BEGIN SOLUTION (FOR LOOP)
students = []
for x in range (1, len(grades)):
    students.append(grades[x][0])
### END SOLUTION

# `students_test`: Test cell
print(students)
assert type(students) is list
assert students == ['Thorny', 'Mac', 'Farva', 'Rabbit', 'Ursula', 'Foster']
print("\n(Passed!)")


I am working on the file "PythonQnA_10_more_comprehensions.ipynb".
The first task is:
Exercise 2 (grade_lists_test: 1 point). Write some code to compute a new dictionary, named grade_lists, that maps names of students to lists of their exam grades. The grades should be converted from strings to integers. For instance, grade_lists['Thorny'] == [100, 90, 80].
Can you write Python code for it?

# Create a dict mapping names to lists of grades.
### BEGIN SOLUTION (LIST COMPREHENSIONS)
grade_lists = {L[0]: [int(g) for g in L[1:]] for L in grades[1:]}
### END SOLUTION

### BEGIN SOLUTION (FOR LOOP)
grade_lists = {}
for x in range (1, len(grades)):
    stu_grades = []
    for y in range(1, len(grades[x])):
        stu_grades.append(int(grades[x][y]))
    grade_lists[grades[x][0]] = stu_grades
### END SOLUTION

# `grade_lists_test`: Test cell
print(grade_lists)
assert type(grade_lists) is dict, "Did not create a dictionary."
assert len(grade_lists) == len(grades)-1, "Dictionary has the wrong number of entries."
assert {'Thorny', 'Mac', 'Farva', 'Rabbit', 'Ursula', 'Foster'} == set(grade_lists.keys()), "Dictionary has the wrong keys."
assert grade_lists['Thorny'] == [100, 90, 80], 'Wrong grades for: Thorny'
assert grade_lists['Mac'] == [88, 99, 111], 'Wrong grades for: Mac'
assert grade_lists['Farva'] == [45, 56, 67], 'Wrong grades for: Farva'
assert grade_lists['Rabbit'] == [59, 61, 67], 'Wrong grades for: Rabbit'
assert grade_lists['Ursula'] == [73, 79, 83], 'Wrong grades for: Ursula'
assert grade_lists['Foster'] == [89, 97, 101], 'Wrong grades for: Foster'
print("\n(Passed!)")


I am working on the file "PythonQnA_10_more_comprehensions.ipynb".
The first task is:
Exercise 3 (grade_dicts_test: 2 points). Write some code to compute a new dictionary, grade_dicts, that maps names of students to dictionaries containing their scores. Each entry of this scores dictionary should be keyed on assignment name and hold the corresponding grade as an integer. For instance, grade_dicts['Thorny']['Exam 1'] == 100.
Can you write Python code for it?

# Create a dict mapping names to dictionaries of grades.
### BEGIN SOLUTION (LIST COMPREHENSION)
grade_dicts = {L[0]: dict(zip(assignments, [int(g) for g in L[1:]])) for L in grades[1:]}
### END SOLUTION

# Create a dict mapping names to dictionaries of grades.
### BEGIN SOLUTION (FOR LOOP)
grade_dicts = {}
for x in range(1, len(grades)):
    stu_grades = []
    for y in range(1, len(grades[x])):
        stu_grades.append(int(grades[x][y]))
    grade_dicts[grades[x][0]] = dict(zip(assignments, stu_grades))
### END SOLUTION

# `grade_dicts_test`: Test cell
print(grade_dicts)
assert type(grade_dicts) is dict, "Did not create a dictionary."
assert len(grade_dicts) == len(grades)-1, "Dictionary has the wrong number of entries."
assert {'Thorny', 'Mac', 'Farva', 'Rabbit', 'Ursula', 'Foster'} == set(grade_dicts.keys()), "Dictionary has the wrong keys."
assert grade_dicts['Foster']['Exam 1'] == 89, 'Wrong score'
assert grade_dicts['Foster']['Exam 3'] == 101, 'Wrong score'
assert grade_dicts['Foster']['Exam 2'] == 97, 'Wrong score'
assert grade_dicts['Ursula']['Exam 1'] == 73, 'Wrong score'
assert grade_dicts['Ursula']['Exam 3'] == 83, 'Wrong score'
assert grade_dicts['Ursula']['Exam 2'] == 79, 'Wrong score'
assert grade_dicts['Rabbit']['Exam 1'] == 59, 'Wrong score'
assert grade_dicts['Rabbit']['Exam 3'] == 67, 'Wrong score'
assert grade_dicts['Rabbit']['Exam 2'] == 61, 'Wrong score'
assert grade_dicts['Mac']['Exam 1'] == 88, 'Wrong score'
assert grade_dicts['Mac']['Exam 3'] == 111, 'Wrong score'
assert grade_dicts['Mac']['Exam 2'] == 99, 'Wrong score'
assert grade_dicts['Farva']['Exam 1'] == 45, 'Wrong score'
assert grade_dicts['Farva']['Exam 3'] == 67, 'Wrong score'
assert grade_dicts['Farva']['Exam 2'] == 56, 'Wrong score'
assert grade_dicts['Thorny']['Exam 1'] == 100, 'Wrong score'
assert grade_dicts['Thorny']['Exam 3'] == 80, 'Wrong score'
assert grade_dicts['Thorny']['Exam 2'] == 90, 'Wrong score'
print("\n(Passed!)")


I am working on the file "PythonQnA_10_more_comprehensions.ipynb".
The first task is:
Exercise 5 (grades_by_assignment_test: 2 points). Write some code to compute a dictionary named grades_by_assignment, whose keys are assignment (exam) names and whose values are lists of scores over all students on that assignment. For instance, grades_by_assignment['Exam 1'] == [100, 88, 45, 59, 73, 89].
Can you write Python code for it?

### BEGIN SOLUTION (LIST COMPREHENSION)
grades_by_assignment = {a: [int(L[k]) for L in grades[1:]] for k, a in zip(range(1, 4), assignments)}
### END SOLUTION

### BEGIN SOLUTION (FOR LOOP)
grades_by_assignment = {}
for k in range(0, len(assignments)): #1,2,3
    stu_assignment = []
    for m in range(1,len(grades)):
        stu_assignment.append(int(grades[m][k+1]))
    grades_by_assignment[assignments[k]] = stu_assignment
### END SOLUTION

# `grades_by_assignment_test`: Test cell
print(grades_by_assignment)
assert type(grades_by_assignment) is dict, "Output is not a dictionary."
assert len(grades_by_assignment) == 3, "Wrong number of assignments."
assert grades_by_assignment['Exam 1'] == [100, 88, 45, 59, 73, 89], 'Wrong grades list'
assert grades_by_assignment['Exam 3'] == [80, 111, 67, 67, 83, 101], 'Wrong grades list'
assert grades_by_assignment['Exam 2'] == [90, 99, 56, 61, 79, 97], 'Wrong grades list'
print("\n(Passed!)")


I am working on the file "PythonQnA_1_vars_and_types.ipynb".
The first task is:
Exercise. Now let's use our knowledge to practise and play with a function that takes a list and returns exactly the same list if every element of the list is a string. Otherwise, it returns a new list with all non-string elements converted to string. To avoid confusion, the function also returns a flag variable showing whether the list has been modified. Try to add some print statements to investigate the types of elements in the list, how the elements are printed out, and how the whole array looks like before and after type conversion.
Can you write Python code for it?

def list_converter(l):
    """
    l - input list

    Returns a list where all elements have been stringified
    as well a flag to indicate if the list has been modified
    """
    assert (type(l) == list)
    flag = False
    for el in l:
        # print the type of el
        if type(el) != str:
            flag = True
    if flag:
        new_list = []
        for el in l:
            # how would be each element printed out? what's the element type?
            new_list.append(str(el))
            # print how the new list looks like
        return new_list, flag
    else:
        return l, flag

# `list_converter_test`: Test cell
l = ['4', 8, 15, '16', 23, 42]
l_true = ['4', '8', '15', '16', '23', '42']
new_l, flag = list_converter(l)

print ("list_converter({}) -> {} [True: {}], new list flag is {}".format(l, new_l, l_true, flag))
assert new_l == l_true
assert flag

new_l, flag = list_converter(l_true)
print ("list_converter({}) -> {} [True: {}], new list flag is {}".format(l, new_l, l_true, flag))
assert new_l == l_true
assert not flag


I am working on the file "PythonQnA_1_vars_and_types.ipynb".
The first task is:
Here, a variable x stores number 42 as an integer. However, we can store the same number as a different type or within another data structure - as float, string, part of a list or a tuple. Depending on the type of variable, Python will print it slightly differently.
Can you write Python code for it?

x_float = float(42)
x_scientific = 42e0
x_str = '42'

print ('42 as a float', x_float)
print ('42 as a float in scientific notation', x_scientific)
print ('42 as a string', x_str)


I am working on the file "PythonQnA_1_vars_and_types.ipynb".
The first task is:
So far it looks pretty normal. float adds floating point to the integer. Scientific notation is typically a float (serious scientists don't work with integers!). 42 as string looks exactly like we expected and similar to the integer, but their behaviors are different. The difference in behavior isn't obvious until we make it a part of a collection, for example a list.
Can you write Python code for it?

x_list = [x, x_str, x_float, x_scientific]
print ("All 42 in a list:", x_list)


I am working on the file "PythonQnA_1_vars_and_types.ipynb".
The first task is:
We can store it in a variable and print it out. It doesn't sound confusing at all!
Can you write Python code for it?


I am working on the file "PythonQnA_2_scoping.ipynb".
The first task is:
However, a function cannot simultaneously reference global and local variables with the same name. That means if you have a global a within a function, none of the parameters may be named a.
Consider the following scenario.
Can you write Python code for it?

a = 42

def my_print():
    try:
        print("Printing GLOBAL variable from inside function", a)
    except:
        print ("UnboundLocalError: local variable 'a' referenced before assignment")
        return
    a = 3
    print("Printing from inside function", a)

my_print()
print("Printing from outside function", a)


I am working on the file "PythonQnA_2_scoping.ipynb".
The first task is:
However, if we want to change that and access the a variable from global scope, we still can do that using global keyword.
Can you write Python code for it?

a = 42

def my_print():
    global a
    print("Printing from inside function", a)

my_print()
print("Printing from outside function", a)


I am working on the file "PythonQnA_2_scoping.ipynb".
The first task is:
This error is very common case case in Jupyter Notebooks and hence it's important to understand. Quite often when you restart your notebook and run a cell in the middle of the notebook, you'll see a similar type of error whose name is not definied. That happens because when you restart your notebook, it's context is reset; and if you don't have the variable defined in the same cell, the variable does not exist in the scope of the notebook. All the cells have the same scope, but the way variables are populated is determined by the running order of notebook cells.
Exercise. A terrific tool for visualizing the concept of scopes is Phil Guo's Python Tutor. Visit that website and click on the "Start visualizing your code now" link. Enter any of the above programs into the code box you see, and then do a line-by-line execution to see how variables are created and come to exist in different scopes.
Then try visualizing this program. See if you can predict what it will output before you run it.
Can you write Python code for it?

x = 1

def foo(x):
    # Q: Can you use `global x` at this part of the program?
    def bar(y):
        global x
        print("bar: x == {}, y == {}".format(x, y))

    print("foo: x == {}".format(x))
    x -= 3
    bar(2*x)

print("x == {}".format(x))
foo(5)
print("x == {}".format(x))


I am working on the file "PythonQnA_3_funcs_returns.ipynb".
The first task is:
Now let's do something useful. Here's a function that should return an integer part of half of a number. So, half(5) == 2, as well as half(4) == 2. The problem is, the function is faulty and it doesn't do what it should (clearly, someone with knowledge of Python 2, not Python 3 wrote it). Add some print statements to debug and modify the function so it passes testcases.
Can you write Python code for it?

def half(num):
    h = num/2
    return h

h_true = 2

h = half(4)
print("half(4) -> {} [True: {}]".format(h, h_true))
assert h == h_true
assert type(h) == int

h = half(5)
print("half(5) -> {} [True: {}]".format(h, h_true))
assert h == h_true
assert type(h) == int

print("\n(Passed!)")


I am working on the file "PythonQnA_4_mutability.ipynb".
The first task is:
Aliasing. Another important concept is aliasing. Let's start with example:
Can you write Python code for it?

a = list(range(5))
print ("The list we created:", a)

b = a
print ("The second list we created:", b)

a[3] = 10
print ("The first list after changing it:", a)
print ("And the second list:", b)


I am working on the file "PythonQnA_4_mutability.ipynb".
The first task is:
Almost all list methods in Python do not return a new list, but modify (or mutate) it. For that reason, if you have several aliases, all of them reflect the changes after list mutation.
Can you write Python code for it?

a = list(range(5))
print ("The list we created:", a)

b = a
print ("The second list we created:", b)

b.append(11)
a.extend([12, 13])
print ("The first list after mutation:", a)
print ("The second list after mutation:", b)

a = list(range(5))
b = a + [11, 12]

print ("The list we created:", a)
print ("The second list we created:", b)

b.append(21)
a.extend([22, 23])
print ("The first list after mutation:", a)
print ("The second list after mutation:", b)


I am working on the file "PythonQnA_4_mutability.ipynb".
The first task is:
Exercise. Aliasing also happens at function call boundaries. Try to predict what the following code will do before you run it.
Can you write Python code for it?

def rem_sublist(L, i, j):
    L[i:j] = []

a = list(range(10))
print(a)
rem_sublist(a, 2, 5)
print(a)


I am working on the file "PythonQnA_4_mutability.ipynb".
The first task is:
Interesting... We wanted to change two elements (line 7), but added four instead! Rather, we mutated list a with list b. "Mutability" means, simply, that we can change the structure of the object.
It's easy to change, or mutate, a list. Virtually any action on the list changes its structure, length, the order of its elements, or something else that is likely important. After mutation, it's the same list with the same variable name, but different content.
Hashing. Besides being mutable, objects may or may not be hashable.
As you should know by know, Python provides built-in collections, like dict and set, which provide "high-speed" access to their elements. The way they do that is through the __hash__() method. A hash function is one that takes an object as input and returns an (arbitrary) number. You can read more about hash functions on Wikipedia. If an object has a __hash__() method then it is hashable. We can check how it works trying to call hash() function on different data types:
Can you write Python code for it?

print ("hash of int(42) is", hash(42))
print ("hash of float(42.001) is", hash(42.001))
print ("hash of str('42') is", hash('42'))
try:
    print ("hash of list(42) is", hash([42]))
except TypeError:
    print("TypeError: unhashable type: 'list'")


I am working on the file "PythonQnA_4_mutability.ipynb".
The first task is:
We see that we can get a hash from int (exact int value), from float, and string (although hash for them is not as obvious as for int), but not for a list. You can see that trying to call hash() function on a list returns a
TypeError. That happens because list doesn't implement __hash__() method, which is a consequence of list mutability. Why? Because if a list can mutate, there isn't a way to create a unique mapping of a given list to a consistent value, because the list could change at any time. So, when you have some collection of elements in a list and you need to make it hashable (for example, to use a key for dict), turn it into tuple:
Can you write Python code for it?


I am working on the file "PythonQnA_5_pythonic_code.ipynb".
The first task is:
And now everything is the same, in a more Pythonic way: simultaneously iterating over two collections where there is a one-to-one correspondence is a pattern referred to as a "zipper iteration," which Python handles nicely via its zip() function.
Can you write Python code for it?

first_names = ['Leonard', 'Sheldon', 'Howard', 'Rajesh']
last_names = ['Hofstadter', 'Cooper', 'Wolowitz', 'Koothrappali']

name_dict = dict(zip(last_names, first_names))
print ("Name dictionary is", name_dict)


I am working on the file "PythonQnA_5_pythonic_code.ipynb".
The first task is:
But we can compute at least the mean with less code using the built-in library function, sum().
Can you write Python code for it?

array = [1, 2, 3, 4, 5, 6]

mean = sum(array) / len(array)

dist = []
for i in range(len(array)):
    dist += [array[i] - mean]

print ("The mean of the array", array, "is", mean, "distances are", dist)


I am working on the file "PythonQnA_5_pythonic_code.ipynb".
The first task is:
Exercise. Now, enlightened by all this knowledge, let's try to write a function that takes a string, drops all the words that contain letter 'o', and return a new string without these words. In Python, it's truly a single line function :)
Can you write Python code for it?

def pick_o(s):
    pass

s = 'the quick brown fox jumped over the lazy dog'
true_string = 'brown fox over dog'
new_string = pick_o(s)
print("pick_o('{}') -> '{}' [True: '{}']".format(s, new_string, true_string))
assert new_string == true_string


I am working on the file "PythonQnA_5_pythonic_code.ipynb".
The first task is:
The counter always increases "in sync" with iteration over the list of words. The enumerate() function captures this pattern succinctly.
Pretty good, but we can do even better.
Can you write Python code for it?

list_of_words = ['the', 'quick', 'brown', 'fox', 'jumped', 'over', 'the', 'lazy', 'dog']
new_string = ' '.join(list_of_words)
print ("The resulting string is '" + new_string + "'.")


I am working on the file "PythonQnA_5_pythonic_code.ipynb".
The first task is:
This "pattern" of code is sometimes referred to as a classic "procedural" approach.
Now let's consider a more Pythonic approach.
Can you write Python code for it?

list_of_words = ['the', 'quick', 'brown', 'fox', 'jumped', 'over', 'the', 'lazy', 'dog']

i = 0 # Current position in the list
new_string = '' # String to hold the output

for word in list_of_words: # Iterate over words
    new_string += word
    i += 1
    # Add a space to join the words together if it's not the last word
    if i < len(list_of_words):
        new_string += ' '

print ("The resulting string is '" + new_string + "'.")


I am working on the file "PythonQnA_6_sorting.ipynb".
The first task is:
As you can see, sorted() not only sorts the list, it returns a new list where the elements are already sorted. On the other hand, sorted() does not change the state of original list. So here is a good rule of thumb: if you need to sort a list forever and you don't care about its previous state, use list.sort(). If you need a new list with sorted elements, use sorted().
Sorted is a very powerful function, that can sort virtually anything, if it's iterable and comparable (or in simple English, if it can be sorted).
Among those thing, you can sort tuples, because by using sorted() you don't change it. But as a result you're going to see a list:
Can you write Python code for it?

a = (2, 6, 3, 4, 1, 9)
print ("Tuple before sorting", a)

b = sorted(a)
print ("That's what 'sorted(tuple)' returns:", b)
print ("Tuple after sorting", a)


I am working on the file "PythonQnA_6_sorting.ipynb".
The first task is:
But that's not the only way of sorting stuff in python. There is a built-in function, sorted(), that among other things can sort lists. In a slightly different manner:
Can you write Python code for it?

a = [2, 6, 3, 4, 1, 9]
print ("List before sorting", a)

b = sorted(a)
print ("That's what 'sorted(list)' returns:", b)
print ("List after sorting", a)


I am working on the file "PythonQnA_6_sorting.ipynb".
The first task is:
Exercise:
Now let's play around with this helpful function a little bit more: let's sort a dictionary. Can you figure out what each line of code is doing before running it?
Can you write Python code for it?

from collections import defaultdict
a = "how many characters are in this sentence, and which one is the most common one?"
def find_most_common_character(s):
    chars = defaultdict(int)
    words = s.split(" ")
    for word in words:
        for c in word:
            chars[c] += 1
    # Q: How about sorting dictionary by key?
    return sorted(chars.items(), key=lambda item: item[1], reverse=True)[0]

find_most_common_character(a)


I am working on the file "PythonQnA_7_sets.ipynb".
The first task is:
Exercise. One curious thing about sets and frozensets is that you can mix them together in binary operations. What could be the result of such operations? Well, it's not very obvious. Try to think about what you think the operations below should return and what they actually return. If you're interested in diving deeper inside Python, you can look here to get some insights.
Can you write Python code for it?

print ("Set intersection between {} and {} is {}".format(a, d, a & d))
print ("Set intersection between {} and {} is {}".format(c, b, c & b))

print ("Set difference between {} and {} is {}".format(a, d, a - d))
print ("Set difference between {} and {} is {}".format(d, a, d - a))


I am working on the file "PythonQnA_7_sets.ipynb".
The first task is:
Read-only sets: frozenset. Beside the "normal" set, we have another helpful friend in the set family, the frozenset. A frozenset shares same operations with normal set, except for that fact that they return frozensets. (Huh?)
Can you write Python code for it?

c = frozenset ([1, 2, 3])
d = frozenset ([2, 3, 4])

print ("Set c is", c)
print ("Set d is", d)

print ("Set intersection is", c & d)
print ("Set union is", c | d)
print ("Set symmetric difference is", c ^ d)
print ("Set difference 'c - d' is", c - d)
print ("Set difference 'd - c' is", d - c)


I am working on the file "PythonQnA_8_comprehensions.ipynb".
The first task is:
As you can see, the translation from the math to code is natural.
In a procedural or iterative style, an equivalent program might look like the following.
Can you write Python code for it?

y2 = []
for x in S:
    y2.append(f(x))

print("3. All (x, f(x)) pairs: {}".format(list(zip(S, y2))))

assert y1 == y2


I am working on the file "PythonQnA_8_comprehensions.ipynb".
The first task is:
Example 2 also has an equivalent in the two variable case.
Example 5
For example,
\begin{align}
C &= \begin{cases}
g(i,j) & i \neq j \
0 & i = j
\end{cases} \
i &\in 0\cdots2, j \in 0\cdots2
\end{align}
Can you write Python code for it?

C1 = [g(i,j) if i !=j else 0 for i in range(0,3) for j in range(0,3)]

print(C1)
print(np.array(C1).reshape(3,3))


I am working on the file "PythonQnA_8_comprehensions.ipynb".
The first task is:
List comprehensions
List comprehensions are a special syntax for compactly generating lists. Historically, they come from a programming style referred to as functional programming.
A list comprehensions can always be expanded into procedural statements using loops. Although comprehensions have a slight advantage in performance when compared to loops, this doesn't mean that you should always prefer comprehensions over procedural code. Too much syntactic sugar can be hazardous to your (program's) health, in the sense of making it hard to read.
Common patterns with list comprehensions in single variable
The following sections contain examples of common patterns where list comprehensions are useful. The patterns described here are by no means exhaustive. Rather, they are meant to act as a solution template for common problems.
A typical use of a list comprehension in a single variable is to expand statements in mathematics known as (universal) quantifiers, or "for-all" statements.
Example 1
The following mathematical statement,
$$
y = f(x) \ \ \forall x \in S,
$$
is usually read as, "for all elements $x$ from a collection $S$, (let) $y$ equal $f(x)$." Statements of this form have natural translations into Python using list comprehensions. To wit:
Can you write Python code for it?

print("1. S == {}".format(S))
y1 = [f(x) for x in S]
print("2. All (x, f(x)) pairs: {}".format(list(zip(S, y1))))
plt.scatter(S, y1)


I am working on the file "PythonQnA_8_comprehensions.ipynb".
The first task is:
NOTE:
This is not different from the first pattern in syntactic terms. This is a trick based on the ternary expressions in Python.
The procedural equivalent of this code is shown below.
Can you write Python code for it?

y2 = []
for x in S:
    if x <= 5:
        y2.append(0)
    else:
        y2.append(f(x))
print(*zip(S, y2))
assert y1 == y2
print("Passed!")


I am working on the file "PythonQnA_8_comprehensions.ipynb".
The first task is:
The pattern can be generalized as
python
output_list = [expr(i,j) for i in iterable1 for j in iterable2 if condition(i,j)]
Comprehensions can be used with even more variables but readability takes a serious hit with more than two variables.
See PEP 202 (https://www.python.org/dev/peps/pep-0202/) for more details about list comprehensions. I highly encourage reading PEP documents since you often get the rationale behind a feature in the language straight from the horse's mouth.
Dictionary comprehensions
Some other built-in collections in Python have "comprehensive" analogues. One example is the dictionary comprehension, which is described in PEP 274 (https://www.python.org/dev/peps/pep-0274/).
You can use dictionary comprehensions in ways very similar to list comprehensions, except that the output of a dictionary comprehension is, well, a dictionary instead of a list.
Mathematically, dictionary comprehensions are suited to representing functions.
$$
\begin{align}
    x \rightarrow f(x), x \in S
\end{align}
$$
can be translated as
Can you write Python code for it?

dict_comp = {x: f(x) for x in S}
print(dict_comp)


I am working on the file "PythonQnA_8_comprehensions.ipynb".
The first task is:
The patterns discussed in the previous section also apply here. There are mainly two kinds of patterns in the single variable case.
python
dict_comp1 = {x: expr(x) for x in iterable}
dict_comp2 = {x: expr(x) for x in iterable if condition(x)}
What not to do with comprehensions
1. Do not use side effects within comprehensions
Can you write Python code for it?

# Bad code
[print(i) for i in range(3)]

# you know you can do better than that
for i in range(3):
    print(i)
# that's better


I am working on the file "PythonQnA_8_comprehensions.ipynb".
The first task is:
The procedural code is more readable compared to the comprehension
Can you write Python code for it?

# procedural code is more readable in this case here
x2 = []
for i in range(100):
    # optimus primes are beyond our reach, https://oeis.org/A217090
    if i not in (5, 7, 11, 13, 17, 19, 29, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97):
        if i <= 10:
            x2.append(i)
        elif 10 < i <= 20:
            x2.append(i**2)
        elif 20 < i <= 50:
            x2.append(i**4)
        else:
            x2.append(1.0 / i)
assert x2 == x1


I am working on the file "PythonQnA_8_comprehensions.ipynb".
The first task is:
The procedural equivalent of this code is shown below.
Can you write Python code for it?

C2 = []
for i in range(3):
    for j in range(3):
        C2.append(g(i, j))
print(C2)
assert C1 == C2
print("Passed!")


I am working on the file "PythonQnA_9_generators.ipynb".
The first task is:
All the rules discussed in the previous sections about conditionals also apply to generator expressions
Can you write Python code for it?

import numpy as np
print(list(i**2 for i in range(11) if i <=5))
print(list(i**2 if i <=5 else 1 for i in range(11)))
mat = list(i**2 + j**2 if i < j else i + j for i in range(3) for j in range(3))
print(np.array(mat).reshape(3,3))