I am working on the file "EQTransformerviaGoogleColab.ipynb".
The first task is:
Downloading Continuous Data
The following will download the information on the stations that are available based on your search criteria:
Can you write Python code for it?

import os
json_basepath = os.path.join(os.getcwd(),"json/station_list.json")

from EQTransformer.utils.downloader import makeStationList

makeStationList(json_path=json_basepath, client_list=["SCEDC"], min_lat=35.50, max_lat=35.60, min_lon=-117.80, max_lon=-117.40, start_time="2019-09-01 00:00:00.00", end_time="2019-09-03 00:00:00.00", channel_list=["HH[ZNE]", "HH[Z21]", "BH[ZNE]"], filter_network=["SY"], filter_station=[])


I am working on the file "EQTransformerviaGoogleColab.ipynb".
The first task is:
Once you have a copy of the source, you can cd to EQTransformer directory
Can you write Python code for it?


I am working on the file "EQTransformerviaGoogleColab.ipynb".
The first task is:
Option (II) directly from mseed files:
You can perform the detection & phase picking directly on downloaded MiniSeed files. This saves both preprocessing time and the extra space needed for the hdf5 file and is recommended for larger (longer) datasets. However, it can be more memory intensive. So it is better to have your MiniSeed fils being shorter than one month or so.
This option also does not allow you to estimate the uncertainties, save the prediction probabilities, or use the advantages of having hdf5 files which makes it easy to access the raw event waveforms based on detection results.
Can you write Python code for it?

from EQTransformer.core.mseed_predictor import mseed_predictor

mseed_predictor(input_dir='downloads_mseeds',
                input_model='EQTransformer/ModelsAndSampleData/EqT_model.h5',
                stations_json='jason/station_list.json',
                output_dir='detection_results',
                detection_threshold=0.2,
                P_threshold=0.1,
                S_threshold=0.1,
                number_of_plots=10,
                plot_mode='time_frequency',
                batch_size=500,
                overlap=0.3)


I am working on the file "EQTransformerviaGoogleColab.ipynb".
The first task is:
Rewrite setup.py  :
'numpy==1.20.3' -> 'numpy==1.19.2'
Install
Need Restart Runtime again when this cell running is done.
Menu -> Runtime -> Restart Runtime
Can you write Python code for it?


I am working on the file "EQTransformerviaGoogleColab.ipynb".
The first task is:
This will generate one station_name.hdf5 and one station_name.csv file for each of your station’s data and put them into a directory named mseed_dir+_hdfs. Then you need to pass the name of this directory (which contains all of your hdf5 & CSV files) and a model to the following command:
Can you write Python code for it?

from EQTransformer.core.mseed_predictor import mseed_predictor

mseed_predictor(input_dir='downloads_mseeds',
                input_model='EQTransformer/ModelsAndSampleData/EqT_model.h5',
                stations_json='json/station_list.json',
                output_dir='detection_results',
                detection_threshold=0.2,
                P_threshold=0.1,
                S_threshold=0.1,
                number_of_plots=10,
                plot_mode='time_frequency',
                batch_size=500,
                overlap=0.3)


I am working on the file "ERA5_atmospheric_rivers.ipynb".
The first task is:
Determine hysteresis settings
Can you write Python code for it?

signal = 1/sb[3]

### set thresholds

perc_upper=95
perc_lower=90

upper_threshold=np.percentile(signal, perc_upper)
lower_threshold=np.percentile(signal, perc_lower)

del signal

# use directions of pixel based sobel transform and magnitudes from calibrated physical sobel.
dat = pixel_sb.transpose([3,2,1,0]).astype('float32')
del pixel_sb
dat[:,:,:,3] = sb[3].transpose([2,1,0])

mask = cp_edge_thinning(dat)
#thinned = mask.transpose([2, 1, 0])
dat = sb.transpose([3,2,1,0]).copy().astype('float32')

edges = cp_double_threshold(data=dat, mask=mask, a=1/upper_threshold, b=1/lower_threshold)
m = edges.transpose([2, 1, 0])

del mask

#plot_signal_histogram(box, signal, lower_threshold, upper_threshold);

cmap_rev = cm.get_cmap('gray_r')
fig = plt.figure(figsize=(20, 10))
ax = fig.add_subplot(111, projection=ccrs.Mercator())
pcm = ax.pcolormesh(
lons, lats, m[timeind], cmap=cmap_rev)
ax.coastlines()
fig.colorbar(pcm)

## remove spurious boundary effects (resulting from imposed periodicity)
m[:,0:1,:]=0
m[:,:,0:1]=0
#m[:,np.size(m, axis=1)-1,:]=0
#m[:,:,np.size(m, axis=2)-1]=0

m[:,-2:-1,:]=0
m[:,:,-2:-1]=0

fig = plt.figure(figsize=(20, 10))
ax = fig.add_subplot(111, projection=ccrs.Mercator())
pcm = ax.pcolormesh(
lons, lats, m[timeind], cmap=cmap_rev)
ax.coastlines()
fig.colorbar(pcm)

# load lsm

lsmfile = netCDF4.Dataset(data_folder / "ERA5_lsm_Pacific.nc", "r", format="NETCDF4")
lsm = lsmfile.variables['lsm'][:,:,:]

# show edges with coastlines

#fig = plt.figure(figsize=(20, 10))
#ax = fig.add_subplot(111, projection=ccrs.Mercator())
#pcm = ax.pcolormesh(
#lons, lats, m[timeind,:,:])
#ax.coastlines()
#fig.colorbar(pcm)

fig = plt.figure(figsize=(20, 10))
ax = fig.add_subplot(111, projection=ccrs.PlateCarree())
pcm = ax.pcolormesh(
lons, lats, m[timeind,:,:], cmap=cmap_rev)
ax.coastlines()
#plt.show()
fig.colorbar(pcm)

### the next lines cut parts of land according to criteria in Dettinger et al., 2011 (Table 1)

## all cells with little bit of land become 1
lsm[lsm>0]=1

## remove Western parts...


I am working on the file "ERA5_atmospheric_rivers.ipynb".
The first task is:
Enter your settings here
Can you write Python code for it?

#data_folder = Path("/home/bathiany/Sebastian/datamining/edges/Abrupt/hypercc/evaluation/AtmosphericRivers")
data_folder = Path("/media/bathiany/Elements/obsdata/qvi")

year=1998
months='04'
#months='01-04'
#months='05-08'
#months='09-12'

## smoothing scales
sigma_d = unit('100 km')    # space
sigma_t = unit('1 hour')    # time

### aspect ratio: all weight on space
gamma = 1e10

## date choice for illustration: 25 April 1998
timeind=26*24 + 12 #hourly data


I am working on the file "ERA5_atmospheric_rivers.ipynb".
The first task is:
Load and inspect the data
Next we define a box. The box contains all information on the geometry of the data. It loads the lattitudes and longitudes of the grid points from the NetCDF file and computes quantities like resolution.
Can you write Python code for it?

from datetime import date, timedelta

box = data_set.box

print("({:.6~P}, {:.6~P}, {:.6~P}) per pixel".format(*box.resolution))
for t in box.time[:3]:
    print(box.date(t), end=', ')
print(" ...")

dt = box.time[1:] - box.time[:-1]
print("time steps: max", dt.max(), "min", dt.min())

data = data_set.data

lons = box.lon.copy()
lats = box.lat.copy()

# a look at the event

fig = plt.figure(figsize=(20, 10))
ax = fig.add_subplot(111, projection=ccrs.Mercator())
pcm = ax.pcolormesh(
lons, lats, data_set.data[timeind,:,:])
cbar = fig.colorbar(pcm)
cbar.ax.tick_params(labelsize=16)
#ax.coastlines()
#fig.colorbar(pcm, labelsize=10)

# Smoothing
smooth_data = gaussian_filter(box, data, [sigma_t, sigma_d, sigma_d])

del data


I am working on the file "ERDDAP/ERDDAP_timing.ipynb".
The first task is:
Try JSON response -- might want to use this in a web app
Can you write Python code for it?

url = 'http://www.neracoos.org/erddap/tabledap/B01_accelerometer_all.json?time,significant_wave_height&time>"now-7days"'
print(url)

%%timeit
response = urllib.urlopen(url)
data = json.loads(response.read())

response = urllib.urlopen(url)
data = json.loads(response.read())

data


I am working on the file "ERDDAP/GliderDAC_Search.ipynb".
The first task is:
Define a function that returns a Pandas DataFrame based on the dataset ID.  The ERDDAP request variables (e.g. pressure, temperature) are hard-coded here, so this routine should be modified for other ERDDAP endpoints or datasets
Can you write Python code for it?

def download_df(glider_id):
    from pandas import DataFrame, read_csv
#    from urllib.error import HTTPError
    uri = ('https://data.ioos.us/gliders/erddap/tabledap/{}.csv'
           '?trajectory,wmo_id,time,latitude,longitude,depth,pressure,temperature'
           '&time>={}'
           '&time<={}'
           '&latitude>={}'
           '&latitude<={}'
           '&longitude>={}'
           '&longitude<={}').format
    url = uri(glider_id,start,stop,lat_min,lat_max,lon_min,lon_max)
    print(url)
    # Not sure if returning an empty df is the best idea.
    try:
        df = read_csv(url, index_col='time', parse_dates=True, skiprows=[1])
    except:
        df = pd.DataFrame()
    return df

# concatenate the dataframes for each dataset into one single dataframe
df = pd.concat(list(map(download_df, dft['Dataset ID'].values)))

print('Total Data Values Found: {}'.format(len(df)))

df.head()

df.tail()


I am working on the file "ERDDAP/GliderDAC_Search.ipynb".
The first task is:
Now search for all temperature data in specified bounding box and temporal extent
Can you write Python code for it?

start = '2000-01-01T00:00:00Z'
stop  = '2017-02-22T00:00:00Z'
lat_min =  39.
lat_max =  41.5
lon_min = -72.
lon_max = -69.
standard_name = 'sea_water_temperature'
endpoint = 'https://data.ioos.us/gliders/erddap/search/advanced.csv'

import pandas as pd

base = (
    '{}'
    '?page=1'
    '&itemsPerPage=1000'
    '&searchFor='
    '&protocol=(ANY)'
    '&cdm_data_type=(ANY)'
    '&institution=(ANY)'
    '&ioos_category=(ANY)'
    '&keywords=(ANY)'
    '&long_name=(ANY)'
    '&standard_name={}'
    '&variableName=(ANY)'
    '&maxLat={}'
    '&minLon={}'
    '&maxLon={}'
    '&minLat={}'
    '&minTime={}'
    '&maxTime={}').format

url = base(
    endpoint,
    standard_name,
    lat_max,
    lon_min,
    lon_max,
    lat_min,
    start,
    stop
)

print(url)

dft = pd.read_csv(url, usecols=['Title', 'Summary', 'Institution', 'Dataset ID'])
print('Glider Datasets Found = {}'.format(len(dft)))
dft


I am working on the file "ERDDAP/OOI-ERDDAP_Search.ipynb".
The first task is:
Define a function that returns a Pandas DataFrame based on the dataset ID.  The ERDDAP request variables (e.g. "ctdpf_ckl_wfp_instrument_ctdpf_ckl_seawater_temperature") are hard-coded here, so this routine should be modified for other ERDDAP endpoints or datasets.
Since we didn't actually find any glider data, we just request the last temperature value from each dataset, using the ERDDAP orderByMax("time") constraint.  This way we can see when the data ends, and if the mooring locations look correct
Can you write Python code for it?

def download_df(glider_id):
    from pandas import DataFrame, read_csv
#    from urllib.error import HTTPError
    uri = ('http://ooi-data.marine.rutgers.edu/erddap/tabledap/{}.csv'
           '?trajectory,'
           'time,latitude,longitude,'
           'ctdpf_ckl_wfp_instrument_ctdpf_ckl_seawater_temperature'
           '&orderByMax("time")'
           '&time>={}'
           '&time<={}'
           '&latitude>={}'
           '&latitude<={}'
           '&longitude>={}'
           '&longitude<={}').format
    url = uri(glider_id,start,stop,lat_min,lat_max,lon_min,lon_max)
    print(url)
    # Not sure if returning an empty df is the best idea.
    try:
        df = read_csv(url, index_col='time', parse_dates=True, skiprows=[1])
    except:
        df = pd.DataFrame()
    return df

df = pd.concat(list(map(download_df, dft['Dataset ID'].values)))

print('Total Data Values Found: {}'.format(len(df)))

df

%matplotlib inline
import matplotlib.pyplot as plt
import cartopy.crs as ccrs
from cartopy.feature import NaturalEarthFeature

bathym_1000 = NaturalEarthFeature(name='bathymetry_J_1000',
                                  scale='10m', category='physical')

fig, ax = plt.subplots(
    figsize=(9, 9),
    subplot_kw=dict(projection=ccrs.PlateCarree())
)
ax.coastlines(resolution='10m')
ax.add_feature(bathym_1000, facecolor=[0.9, 0.9, 0.9], edgecolor='none')

dx = dy = 0.5
ax.set_extent([lon_min-dx, lon_max+dx, lat_min-dy, lat_max+dy])

g = df.groupby('trajectory')
for glider in g.groups:
    traj = df[df['trajectory'] == glider]
    ax.plot(traj['longitude'], traj['latitude'], 'o', label=glider)

gl = ax.gridlines(crs=ccrs.PlateCarree(), draw_labels=True,
                  linewidth=2, color='gray', alpha=0.5, linestyle='--')

ax.legend();


I am working on the file "ERDDAP/OOI-ERDDAP_Search.ipynb".
The first task is:
First try just searching for "glider"
Can you write Python code for it?

url = 'http://ooi-data.marine.rutgers.edu/erddap/search/advanced.csv?page=1&itemsPerPage=1000&searchFor=glider'
dft = pd.read_csv(url, usecols=['Title', 'Summary', 'Institution', 'Dataset ID'])
dft.head()


I am working on the file "ESA_NEOCC_Riddles/Riddle 1.ipynb".
The first task is:
Below we plot the eccentricity and aphelion of those satellites in relation to the curve. We see that most objects have eccentricities around 0.2-0.3, away from the maximum at 0.55.
Can you write Python code for it?

plt.figure(figsize = (10,6), facecolor = 'w')
plt.plot(eps, f)
plt.xlim((0,1))
plt.ylim((1,1.25))
plt.xlabel('$\\varepsilon$')
plt.ylabel('$f(\\varepsilon, E_0)$');
plt.plot(good_objects['Eccentricity'], good_objects['Aphelion']/r0, '.')
plt.title('Objects that spend approximately half the time inside 1.3au');


I am working on the file "ESA_NEOCC_Riddles/Riddle 1.ipynb".
The first task is:
For the bonus part of the question there is some ambiguity about what "similar cases" means. If we understand this as objects that spend approximately half the time inside 1.3au and have an aphelion as large as possible given this condition, then a search in the database with the criteria that the eccentricity must be between 0.5 and 0.6 and the aphelion between 1.5 and 1.6 shows the following objects. This yields the objects we want, because the property that the object spends half the time inside 1.3au is completely determined by the relation between the eccentricity and the aphelion.
| Designator |  Eccentricity |  Aphelion (au) |
|------------|---------------|----------------|
|2001XX4 |0.556838 |1.5669|
|2006EK53 |0.517586 |1.5559|
|2007DA |0.573595 |1.5894|
|2007US |0.577134 |1.5121|
|2008LC2 |0.515329 |1.5285|
|2009UM1 |0.59611 |1.5344|
|2011CK22 |0.565387 |1.5757|
|2012VE37 |0.536756 |1.5985|
|2013AH76 |0.546486 |1.5228|
|2016GG216 |0.515344 |1.528|
|2016VU2 |0.547716 |1.5271|
|2017SK21 |0.506605 |1.5429|
|2018CC1 |0.500576 |1.5917|
|2018BB5 |0.592965 |1.5233|
|2019QZ4 |0.560951 |1.5978|
|2019SM2 |0.5503 |1.5186|
|2019EO |0.531979 |1.523|
|2019GD20 |0.536367 |1.5772|
|2019SC8 |0.530063 |1.5955|
|2019YR6 |0.516992 |1.5625|
|2020AD3 |0.594735 |1.5674|
|3753 Cruithne |0.514844 |1.5114|
|88213 2001AF2 |0.595312 |1.5219|
|141531 2002GB |0.528936 |1.5168|
|250577 2005AC |0.517868 |1.5936|
|439437 2013NK4 |0.549901 |1.5856|
|481025 2004VA1 |0.51237 |1.59|
|511684 2015BN509 |0.568469 |1.5787|
|512234 2015VO66 |0.599967 |1.5234|
|512244 2015YE18 |0.540916 |1.5985|
|539856 2017EV2 |0.55758 |1.5735|
If we understand "similar objects" just as objects that spend half the time inside 1.3au, then the search is more complex. We need to pull out from the database the eccentricity and aphelion for all the objects and select those whose eccentricity and aphelion divided by 1.3au lies close to the curve depicted below.
This curve shows the relation between the eccentricity $\varepsilon$ and the quotient between the...
Can you write Python code for it?

plt.figure(figsize = (10,6), facecolor = 'w')
plt.plot(eps, f)
plt.xlim((0,1))
plt.ylim((1,1.25))
plt.xlabel('$\\varepsilon$')
plt.ylabel('$f(\\varepsilon, E_0)$')
plt.title('Aphelion / critical radius ratio in terms of eccentricity');


I am working on the file "ESA_NEOCC_Riddles/Riddle 1.ipynb".
The first task is:
Now that we have $\varepsilon$ in terms of $E_0$, we can compute and plot $f$ in terms of $E_0$. Below we plot $f$ and its maximum.
Can you write Python code for it?

f = (1 + eps)/(1 - eps * np.cos(E0))
plt.figure(figsize = (10,6), facecolor = 'w')
plt.plot(E0, f)

idx = np.argmax(f)
plt.plot(E0[idx], f[idx], 'o', color = 'red')

plt.xlabel('$E_0$')
plt.ylabel('$f(\\varepsilon, E_0)$')
plt.title('Aphelion / critical radius ratio in terms of eccentric anomaly at $M = \\pi/2$');


I am working on the file "ESA_NEOCC_Riddles/Riddle 1.ipynb".
The first task is:
We assume that the NEO orbit is described by a Keplerian elliptical orbit. If we write $r_0 = 1.3\ \mathrm{au}$, the condition that the object spends exactly half of the time inside $r_0$ (and half the time outside $r_0$) can be rewritten by saying that when the object's mean anomaly $M$ equals $M_0 = \pi/2$, then the distance between the object and the Sun $r$ equals $r_0$. Indeed, since the mean anomaly grows uniformly in time, an object satisfying such condition would spend half of the orbital period having mean anomaly $M$ with $|M| < \pi/2$ and inside $r_0$, and half of the orbital period having mean anomaly $M$ with $|M| > \pi/2$ and outside $r_0$ (here we assume the mean anomaly to range between $-\pi$ and $\pi$).
By using Kepler's formula for the eccentric anomaly $E$, we see that when $M = M_0$, the eccentric anomaly $E_0$ is given as the solution of the equation
$$M_0 = E_0 - \varepsilon \sin E_0,$$
where $\varepsilon$ is the eccentricity.
The formula for the heliocentric distance then yields the condition
$$r_0 = a(1 - \varepsilon \cos E_0),$$
where $a$ is the semi-major axis. Now, the aphelion radius for such an orbit is
$$a(1 + \varepsilon) = r_0 \frac{1+\varepsilon}{1 - \varepsilon \cos E_0}.$$
Since $r_0$ is fixed, we want to maximise
$$f(\varepsilon, E_0) = \frac{1+\varepsilon}{1 - \varepsilon \cos E_0}$$
subject to the condition
$$E_0 - \varepsilon \sin E_0 = M_0.$$
I haven't been able to find an analytic formula for the solution to this problem, but the solution can be approximated numerically, as we do below.
First note that $\varepsilon$ can be computed in terms of $E_0$ by
$$\varepsilon = \frac{E_0 - M_0}{\sin E_0},$$
since $M_0 = \pi/2$ is fixed. As long as $\sin E_0 \neq 0$ this formula makes sense, but not all the values for $E_0$ are acceptable, since we need $0 \leq \varepsilon < 1$. Below we plot $\varepsilon$ in terms of $E_0$ for the acceptable range of parameters.
Can you write Python code for it?

M0 = np.pi/2
E0 = np.linspace(1e-3, np.pi, 10000)
eps = (E0 - M0)/np.sin(E0)

plt.figure(figsize = (10,6), facecolor = 'w')
plt.plot(E0, eps)
plt.xlabel('$E_0$')
plt.ylabel('$\\varepsilon$')
plt.ylim((0,1))
plt.title('Eccentricity in terms of eccentric anomaly at $M = \\pi/2$');


I am working on the file "ESEO.ipynb".
The first task is:
An example packet from ESEO.
Can you write Python code for it?


I am working on the file "ESEO.ipynb".
The first task is:
Here we have 18 3d instead of 1839 near the end.
Can you write Python code for it?

after_unstuffing.size/8

after_derandom = nrzi_decode(descramble(after_unstuffing))
hexprint(reflect_bytes(after_derandom))

after_derandom.size


I am working on the file "ESEO.ipynb".
The first task is:
Perform G3RUH descrambling.
Can you write Python code for it?

def descramble(x):
    y = np.concatenate((np.zeros(17, dtype='uint8'), x))
    z = y[:-17] ^ y[5:-12] ^ y[17:]
    return z

def nrzi_decode(x):
    return x ^ np.concatenate((np.zeros(1, dtype = 'uint8'), x[:-1])) ^ 1

data_descrambled = descramble(data_rev_destuff)
hexprint(data_descrambled)


I am working on the file "ESEO.ipynb".
The first task is:
Perform NRZ-I decoding.
Can you write Python code for it?

data_nrz = nrzi_decode(data_descrambled)
hexprint(data_nrz)


I am working on the file "ESEO.ipynb".
The first task is:
Perform bit de-stuffing.
Can you write Python code for it?

def destuff(x):
    y = list()
    run = 0
    for i, bit in enumerate(x):
        if run == 5:
            if bit == 1:
                print('Long run found at bit', i)
                break
            else:
                run = 0
        elif bit == 0:
            run = 0
            y.append(bit)
        elif bit == 1:
            run += 1
            y.append(bit)
    return np.array(y, dtype = 'uint8')

data_rev_destuff = destuff(data_rev)


I am working on the file "ESEO.ipynb".
The first task is:
Reverse the bytes in the data.
Can you write Python code for it?

def reflect_bytes(x):
    return np.fliplr(x[:x.size//8*8].reshape((-1,8))).ravel()

data_rev = reflect_bytes(data)
hexprint(data_rev)


I am working on the file "ESEO.ipynb".
The first task is:
That it interesting, we have found a run of ones longer than 5 inside the data. We wouln't expect such a run due to byte stuffing. This happens during byte of a total of 161 data bytes.
Can you write Python code for it?


I am working on the file "ESEO.ipynb".
The first task is:
The CRC is CRC16_CCITT_ZERO following the notation of this online calculator.
Data from SITAEL
Can you write Python code for it?


I am working on the file "ESEO.ipynb".
The first task is:
The long sequences of zeros are a good indicator, but still we don't have the expected 8A A6 8A 9E 40 40 60 92 AE 68 88 AA 98 61 AX.25 header.
Reflect the bytes again.
Can you write Python code for it?

data_nrz_rev = reflect_bytes(data_nrz)
hexprint(data_nrz_rev)


I am working on the file "ESEO.ipynb".
The first task is:
Trim the data between the 0x7e7e flags. We skip Reed-Solomon decoding, since we are confident that there are no bit errors. We remove the 16 Reed-Solomon parity check bytes.
Can you write Python code for it?


I am working on the file "ETVL-IPy-09-Taller-Energia.ipynb".
The first task is:
1.-- Cuántos registros tiene el archivo?
Can you write Python code for it?

x=pd.read_csv('AportesDiario_2015.csv', sep=';',decimal=',',thousands='.',skiprows=2)
len(x)

x.head()


I am working on the file "ETVL-IPy-09-Taller-Energia.ipynb".
The first task is:
2.-- Cuántas regiones hidrológicas diferentes hay?
Can you write Python code for it?


I am working on the file "ETVL-IPy-09-Taller-Energia.ipynb".
The first task is:
4.-- Cuántos registros hay por región hidrológica?
Can you write Python code for it?

y = x.groupby('Region Hidrologica')
y.size()


I am working on the file "ETVL-IPy-09-Taller-Energia.ipynb".
The first task is:
5.-- Cuál es el promedio de aportes en energía kWh por región?
Can you write Python code for it?


I am working on the file "ETVL-IPy-09-Taller-Energia.ipynb".
The first task is:
6.-- Cuáles registros no tienen datos?
Can you write Python code for it?

Caudal=len(x[x['Aportes Caudal m3/s'].isnull()])
Aportes=len(x[x['Aportes Energia kWh'].isnull()])
Aport =len(x[x['Aportes %'].isnull()])

print (Caudal)
print (Aportes)
print (Aport)

# x.dropna() borra los registros na
len(x) - len(x.dropna())


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
A add_numbers también pordemos agregarle un parámetro opcional llamado flag.
Can you write Python code for it?

def add_numbers(x, y, z=None, flag=False):
    if (flag):
        print('¡Flag es verdad, y es muy útil!')
    if (z==None):
        return x + y
    else:
        return x + y + z

print(add_numbers(1, 2, 20, flag=True))


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Esta instrucción devuelve el último elemento de la lista o de la cadena.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Este es un ejemplo de cómo se puede hacer un ciclo para recorrer los elementos de la lista.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Este es un pedazo de la lista o cadena desde el inicio y se detiene antes del tercer elemento.
Can you write Python code for it?

print(y)
print(y[:3])
print(x)
print(x[:3])


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
La función min devuelve el elemento más pequeño en un iterable o el más pequeño de dos o más argumentos.
Can you write Python code for it?

x = (1,3,5, -6)
minimum = min(x)
print(minimum)


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
La función pow(x,y) devuelve x a la potencia de y.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
La función x%y devuelve el resto de la división entre x y y.
Can you write Python code for it?

mod1 = 4%2
print(mod1)
mod2 = 5%2
print(mod2)


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
La función x//y devuelve la parte entera del cociente entre x y y.
Can you write Python code for it?

div = 5/2
print(div)
divint = 5//2
print(divint)


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Las dimensiones del arreglo zt son (4,2) después de la transposición.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:

Leer y escribir archivos CSV

Vamos a importar un archivo txt como
uno CSV que contiene como columnas las etiquetas:

Nombre,Departamento,Mes de Cumpleaños

Y como filas:

Adelis Nieves, Matemáticas, Junio
Miguel Astor, Redes, Septiembre
Francisco Sans, Computación Gráfica, Diciembre
Antonio Escalante, Letras, Diciembre
Can you write Python code for it?

import csv

with open('empleados_cumple.txt') as csv_file:
    csv_reader = csv.reader(csv_file, delimiter=',')
    line_count = 0
    for row in csv_reader:
        if line_count == 0:
            print(f'Los nombres de las columnas son: {", ".join(row)}')
            line_count += 1
        else:
            print(f'\t{row[0]} trabaja en el departamento de {row[1]}, y nació en el mes de {row[2]}.')
            line_count += 1
    print(f'Líneas procesadas: {line_count}.')


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Lenguaje de programación Python: Python Numérico (NumPy)
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
O podemos utilizar el operador de índices:
Can you write Python code for it?

i=0
while( i != len(y) ):
    print(y[i])
    i = i + 1


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Podemos pasarle una lista de listas para crear un arreglo multidimensional.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Se puede usar un segundo : para indicar el tamaño del paso. array [empieza: termina: paso]
Aquí estamos comenzando el quinto elemento desde el final, y contando hacia atrás cada 2 hasta que se alcanza el comienzo de la matriz.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Use * para repetir listas.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Use + para concatenar listas.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Use : para indicar un rango. array[empieza:termina]
Dejando empieza or termina vacío quedará predeterminado al principio / final de la matriz.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Use contadores negativos desde el final.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Use el operador in para chequear si algo está dentro de la lista.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Use hstack para apilar arreglos en secuencia horizontalmente (por columna).
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Use la notación bracket [] para picar o rebanar listas o cadenas de caracteres
Can you write Python code for it?

y =[2, 'c', 34, -2, 'f']
print(y[1]) #primer elemento de la lista
print(y[0:1]) #primer elemento devuelto como lista
print(y[0:2]) #dos primeros elementos devueltos como lista

x = 'Esto es una cadena de caracteres'
print(x[0]) #primer caracter
print(x[0:1]) #primer caracter, especificando último caracter
print(x[0:2]) #dos primeros caracteres


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Vamos a mirar la transposición de matrices. La transposición permuta las dimensiones de la matriz.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:

Veamos algunas modificaciones en la estructura de datos lista.

Usemos append para gregar un elemento a la lista.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
Y este es un pedazo de la lista o cadena empezando desde el tercer elemento y siguiendo hasta el final.
Can you write Python code for it?

print(y)
print(y[3:])
print(x)
print(x[3:])


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
add_numbers puede actualizarse agregando un tercer parámetro. Usando print se pueden imprimir múltiples opciones en una sola celda de Jupyter.
Can you write Python code for it?

def add_numbers(x,y,z=None):
    if (z==None):
        return x+y
    else:
        return x+y+z

print(add_numbers(1, 2))
print(add_numbers(1, 2, 3))


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
arange devuelve valores espaciados uniformemente dentro de un intervalo dado.
Can you write Python code for it?

n = np.arange(0, 30, 2) # empieza en 0 cuenta de 2 en 2, se detiene antes de 30
n


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
<br>
reshape devuelve una matriz con los mismos datos con una nueva forma.
Can you write Python code for it?

n = n.reshape(3, 5) #cambia la forma de manera que se obtenga una matriz 3x5
n


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
Combinando arreglos
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
Creando arreglos
<br>
Podemos generar una lista y convertirla en un arreglo NumPy.
Can you write Python code for it?

milista = [1, 25, 31, 5]
x = np.array(milista)
x


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
Indexando/Rebanando
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
Operaciones
Use +, -, *, / and ** para realizar operaciones de suma , resta, multiplicación, división y potencia elemento a elemento.
Can you write Python code for it?

print(x)
print(y)
print(x + y) # suma elemento a elemento     [1 2 3] + [4 5 6] = [5  7  9]
print(x - y) # resta elemento a elemento  [1 2 3] - [4 5 6] = [-3 -3 -3]

print(x)
print(y)
print(x * y) # multiplicación elemento a elemento  [1 2 3] * [4 5 6] = [4  10  18]
print(x / y) # división elemento a elemento         [1 2 3] / [4 5 6] = [0.25  0.4  0.5]

print(x)
print(x**2) # potencia elemento a elemento   [1 2 3] ^2 =  [1 4 9]


I am working on the file "EVI - 2018/EVI 04/Modulo1.ipynb".
The first task is:
Producto punto o escalar:
$ \begin{bmatrix}x_1 \ x_2 \ x_3\end{bmatrix}
\cdot
\begin{bmatrix}y_1 \ y_2 \ y_3\end{bmatrix}
= x_1 y_1 + x_2 y_2 + x_3 y_3$
Can you write Python code for it?

print(x)
print(y)
x.dot(y) #producto escalar

print(y)
z = np.array([y, y**2])
print(z)
print(len(z)) #número de filas del arreglo


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Aquí podemos ignorar la primera fila del DataFrame para dejar más limpia la tabla de información no relevante.
Can you write Python code for it?

df = pd.read_csv('olympics.csv', index_col = 0, skiprows=1)
df.head()


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Aquí tenemos un ejemplo de un elemento nuevo en la lista de índices que no tiene un valor asignado, no existe un país asociado al índice Natación y Pandas representa este valor faltante con NaN.
Can you write Python code for it?

deportes = {'Capoeira': 'Brasil',
          'Rayuela': 'Chile',
          'Pelota Vasca': 'País Vasco',
          'Béisbol': 'Cuba',
           'Rugby': 'Gales',
           'Golf': 'Escocia',
           'Corrida de Toros': 'España',
           'Sumo': 'Japón'}
s = pd.Series(deportes, index=['Capoeira', 'Sumo', 'Pelota Vasca',  'Natación'])
s


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Buscar qué países han ganado sólo medallas de oro en Invierno y nunca en Verano.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
El atributo .columns nos permite ver el nombre de las comlumnas del DataFrame y el atributo .rename modificar el nombre.
Can you write Python code for it?

df.columns

for col in df.columns:
    if col[:2]=='01':
        df.rename(columns={col:'Gold' + col[4:]}, inplace=True)
    if col[:2]=='02':
        df.rename(columns={col:'Silver' + col[4:]}, inplace=True)
    if col[:2]=='03':
        df.rename(columns={col:'Bronze' + col[4:]}, inplace=True)
    if col[:1]=='№':
        df.rename(columns={col:'#' + col[1:]}, inplace=True)

df.head()


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Es importante saber como NumPy y Pandas manejan los datos faltantes. En Python tenemos el tipo None para indicar un dato faltante. Si tenemos una lista de números, Pandas automáticamente convierte este valor None en un valor designado como NaN, que significa Not a Number.
Can you write Python code for it?

numeros = [1, 2, None]
pd.Series(numeros)


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Este es un ejemplo de una serie donde los valores del conjunto de índices no son únicos. Esto hace que las tablas de datos funcionen diferente y es por ello que agregar nuevos elementos debe hacerse con el método append, que en primera instancia, no modificará la serie sino que devuelve una nueva serie con los elementos que se agregaron.
Can you write Python code for it?

deportes_originales = pd.Series({'Capoeira': 'Brasil',
          'Rayuela': 'Chile',
          'Pelota Vasca': 'País Vasco',
          'Béisbol': 'Cuba',
           'Rugby': 'Gales',
           'Golf': 'Escocia',
           'Corrida de Toros': 'España',
           'Sumo': 'Japón'})
paises_que_aman_el_beisbol = pd.Series(['Venezuela',
                                      'USA',
                                      'Cuba',
                                      'Puerto Rico',
                                       'Dominicana'],
                                   index=['Béisbol',
                                          'Béisbol',
                                          'Béisbol',
                                          'Béisbol',
                                          'Béisbol'])
todos_los_paises = deportes_originales.append(paises_que_aman_el_beisbol)

deportes_originales

paises_que_aman_el_beisbol

todos_los_paises

todos_los_paises.loc['Béisbol']


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Finalmente, es muy sencillo agregar una columna al DataFrame.
Can you write Python code for it?

df['Ubicación'] = ['Venezuela', 'Chile', 'Argentina']
df


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Importamos la biblioteca NumPy. También es importante saber que NaN no es None. Cuando hacemos un test para saber si NaN es NaN tambien obtendremos False.
Can you write Python code for it?

import numpy as np
np.nan == None

np.nan == np.nan


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Los cuadernos de Jupyter tienen funciones mágicas que pueden ser útiles. Una de ellas es %%timeit que nos servirá para ver cuál de los dos métodos para sumar elementos de una serie es más rápido.
Basta con tipear el símbolo % y la tecla Tab para obtener una lista de las funciones mágicas de Jupyter.
Can you write Python code for it?

%%timeit -n 100
sumar = 0
for elemento in s:
    sumar+=elemento

%%timeit -n 100
sumar = np.sum(s)


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
NumPy y Pandas tienen el broadcasting, se puede aplicar una operación a cada valor de la serie y modificarla.
Can you write Python code for it?

s+=2 #Suma 2 a cada elemento de la serie usando broadcasting
s.head()


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Pero ... no hay que preocuparse mucho por eso! Podemos leer este archivo en formato CSV en un DataFrame usando la función read_csv.
Can you write Python code for it?

import pandas as pd
df = pd.read_csv('olympics.csv')
df.head()


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Podemos agregar elementos a una serie de la siguiente forma:
Can you write Python code for it?

import pandas as pd
s = pd.Series([1, 2, 3])
s.loc['Animal'] = 'Oso'
s


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Podemos eliminar una columna de manera sencilla, usando simplemente la palabra clave del y el índice o nombre de la comuna.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Podemos establecer otro tipo de condiciones para hacer búsquedas más complejas. Por ejemplo, buscar la cantidad de países que han ganado medalla de oro alguna vez.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Podemos hacer búsquedas en las series por posición de índices o por etiqueta de índices. Si queremos hacer búsqueda por ubicación numérica (empezando desde 0) utilizamos el atributo iloc. Si por otra parte, hacemos búqueda por etiqueta de índice entonces usamos el atributo loc.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Se necesita la función especial isnan de NumPy para chequear la presencia de un no número en nuestros datos.
Can you write Python code for it?

print(np.isnan(np.nan))
print(None is None)
print(np.nan is np.nan)


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
También podemos chequear el tipo de dato usando la función type de Python.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
También podemos hacer una copia del DataFrame con la función copy() para guardar la extracción de información.
Can you write Python code for it?

copiar_df = df.copy()
copiar_df = copiar_df.drop('Tienda 1')
copiar_df

copiar_df.drop?


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
También podemos seleccionar columnas agregando un parámetro extra al atributo loc.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Una forma de trabajar es iterar sobre un conjunto de datos e invocar una operación de interés
Can you write Python code for it?

total = 0
for elemento in s:
    total+=elemento
print(total)


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Una manera poco eficiente de hacer esto es iterar sobre cada elemento de la serie para hacer la suma. El método .iteritems() devuelve un iterador sobre los pares (key, value) (clave, valor) de un diccionario, en este caso de nuestra serie s.
Can you write Python code for it?

for etiqueta, valor in s.iteritems():
    s.set_value(etiqueta, valor+2)
s.head()

%%timeit -n 10
s = pd.Series(np.random.randint(0,1000,10000))
for etiqueta, valor in s.iteritems():
    s.loc[etiqueta]= valor+2

%%timeit -n 10
s = pd.Series(np.random.randint(0,1000,10000))
s+=2


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
Usando .T.loc[] se puede seleccionar una columna usando como parámetro la etiqueta de su nombre.
Can you write Python code for it?

df.T.loc['Costo']

df['Costo']

df.loc['Tienda 1']['Costo']


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
loc también tiene soporte para rebanar o seleccionar del DataFrame con la notación []
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
<br>
¿Qué pasa cuando tenemos una lista de índices que son enteros?
Can you write Python code for it?

deportes = {99: 'Brasil',
          100: 'Chile',
          101: 'País Vasco',
          102: 'Cuba',
           103: 'Gales',
           104: 'Escocia',
           105: 'España',
           106: 'Japón'}
s = pd.Series(deportes)
s


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
Haciendo búsquedas en un DataFrame
<br>
Podemos buscar en el DataFrame con una máscara Booleana qué países tienen (True) o no (False) una medalla de oro.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo2.ipynb".
The first task is:
Lectura de un DataFrame
<br>
Usemos !cat para leer un archivo de formato CSV. Nota: !cat funciona para Linux y Mac pero puede no funcionar para Windows :(
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
<br>
Ahora usamos la función replace()
Can you write Python code for it?

datos1['A'].replace(np.nan, media, inplace = True)
datos1


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
<br>
Correlación entre variables
Tomememos las dos variables del ejemplo anterior...
Can you write Python code for it?

import matplotlib.pyplot as plt
from scipy import stats
x=df['engine-size'] #Variable predictora
y= df['price'] #Variable objetivo o que deseamos predecir
slope, intercept, r_value, p_value, std_err = stats.linregress(x,y)
line = slope*x+intercept
plt.plot(x,y,'o', x, line)
ax = plt.gca()
fig = plt.gcf()
plt.xlabel('Tamaño del motor', fontsize=9)#Etiquetal del eje-x
plt.ylabel('Precio', fontsize=9)#Etiqueta del eje-y
plt.title('Gráfico de dispersión de Tamaño del motor Vs. Precio', fontsize=13)#Nombre del gráfico


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
<br>
En dropna() el argumento subset considera la etiqueta para seleccionar el conjunto a descartar, axis=0 descarta filas (axis=1 columnas) y inplace= True hace que los cambios se ejecuten directamente en DataFrame.
Can you write Python code for it?

datos1.dropna(subset=['A'], axis= 0, inplace= True)
datos1


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:

Estadística descriptiva

Tabla de resumen estadístico
Can you write Python code for it?

import numpy as np
df = pd.read_csv('Automobile_data.csv')
df.head()

df.describe()


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:

Este gráfico de barras nos indica que hay un número alto de automóviles con precio menor a 10000, entre otras cosas .... ¿Qué cosas? ;)

Gráfico de dispersión

Este gráfico de dispersión muestra la relación entre las variables tamaño del motor y precio.
Can you write Python code for it?

import matplotlib.pyplot as plt
x= df['engine-size'] #Variable predictora
y= df['price'] #Variable objetivo o que deseamos predecir
plt.scatter(x, y) #Gráfico de dispersión en Matplotlib
plt.title('Gráfico de dispersión de Tamaño del motor Vs. Precio', fontsize=13)#Nombre del gráfico
plt.xlabel('Tamaño del motor', fontsize=13)#Etiquetal del eje-x
plt.ylabel('Precio', fontsize=13)#Etiqueta del eje-y


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:

Gráficos de barras (o histogramas)

Vamos a generar datos aleatoriamente y hacer un gráfico de barras.
Can you write Python code for it?

np.random.seed(14000) #Generación de números aleatorios
pdhist = pd.Series(np.random.randn(1000)) #Serie de números aleatorios
pdhist.hist(normed=True) # Muestra las barras
pdhist.plot(fontsize=13, kind='kde') #Gráfico de barras (kde = Kernel Density Estimation plot. Haga la prueba con 'hist')


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:

Gráficos de cajas (o Boxplots)

Vamos a generar datos aleatoriamente y hacer un gráfico de caja.
Can you write Python code for it?

np.random.seed(1500) #generación aleatoria números
dfb = pd.DataFrame(np.random.randn(10,5)) #DataFrame de dimensiones 10x5
dfb.boxplot(return_type='axes') #Grafico de caja de cada categoría.
dfb.head()


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
<br>
Hay mucho más que aprender! Por ejemplo: ¿Qué sucede si axis=0? R: pues posiblemente el resultado sea que Pandas pegue todos los valores y sus índices. Como se muestra a continuación:
Can you write Python code for it?

pd.concat([df_info_empleados, fecha_ingreso_df, art_vendidos_df], axis=0)


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
<br>
Método Mínimo - Máximo: toma cada valor, $x_{viejo}$ le resta el mínimo
valor de esa característica y luego se divide por el rango de esa característica, es decir, $x_{nuevo} = \frac{x_{viejo} - x_{mínimo}}{x_{máximo} - x_{mínimo}}$
Can you write Python code for it?

dimensiones['Largo'] = (dimensiones['Largo']-dimensiones['Largo'].min())/(dimensiones['Largo'].max() - dimensiones['Largo'].min())
dimensiones['Ancho'] = (dimensiones['Ancho']-dimensiones['Ancho'].min())/(dimensiones['Ancho'].max() - dimensiones['Ancho'].min())
dimensiones['Alto'] = (dimensiones['Alto']-dimensiones['Alto'].min())/(dimensiones['Alto'].max() - dimensiones['Alto'].min())
dimensiones


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
<br>
Método Puntaje estándar:
Can you write Python code for it?

dimensiones['Largo'] = (dimensiones['Largo']-dimensiones['Largo'].mean())/(dimensiones['Largo'].std())
dimensiones['Ancho'] = (dimensiones['Ancho']-dimensiones['Ancho'].mean())/(dimensiones['Ancho'].std())
dimensiones['Alto'] = (dimensiones['Alto']-dimensiones['Alto'].mean())/(dimensiones['Alto'].std())
dimensiones


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
<br>
Método de "Escala de característica simple": se divide cada valor por el
valor máximo para esa característica, $x_{nuevo} = \frac{x_{viejo}}{x_{máximo}}$
Can you write Python code for it?

dimensiones['Largo'] = dimensiones['Largo']/dimensiones['Largo'].max()
dimensiones['Ancho'] = dimensiones['Ancho']/dimensiones['Ancho'].max()
dimensiones['Alto'] = dimensiones['Alto']/dimensiones['Alto'].max()
dimensiones


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:

Otros ejemplos de cómo variar el parámetro how se pueden encontrar en el libro Python for Data Analysis - McKinney.

Supongamos que tenemos ahora un nuevo DataFrame que coincide en número de filas con el anterior. Por ejemplo:
Can you write Python code for it?

fecha_ingreso_df = pd.DataFrame([{'Nombre': 'Adriana', 'Fecha de Ingreso': '20/06/2013'},
                         {'Nombre': 'Andrés', 'Fecha de Ingreso': '10/01/2018'},
                         {'Nombre': 'Cristóbal', 'Fecha de Ingreso': '20/03/2011'}])
fecha_ingreso_df = fecha_ingreso_df.set_index('Nombre')
art_vendidos_df = pd.DataFrame([{'Nombre': 'Adriana', 'Art.Vendidos/Total Art.': 123/10000},
                         {'Nombre': 'Andrés', 'Art.Vendidos/Total Art.': 1450/10000},
                         {'Nombre': 'Cristóbal', 'Art.Vendidos/Total Art.': 5000/10000}])
art_vendidos_df = art_vendidos_df.set_index('Nombre')

print(fecha_ingreso_df.head())
print(art_vendidos_df.head())


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
<br>
Pandas reset_index () es un método para restablecer el índice de un DataFrame. El establece como índices una lista de enteros que van desde 0 hasta la longitud de los datos.
Can you write Python code for it?


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
<br>
Podemos tener un par de tablas de datos que nos interese unir o combinar en un mismo DataFrame.
Can you write Python code for it?

empleados_df = pd.DataFrame([{'Nombre': 'Adriana', 'Función': 'Gerente de ventas'},
                         {'Nombre': 'Andrés', 'Función': 'Vendedor 1'},
                         {'Nombre': 'Cristóbal', 'Función': 'Gerente de departamento'}])
empleados_df = empleados_df.set_index('Nombre')
grado_df = pd.DataFrame([{'Nombre': 'Andrés', 'Grado': 'Nivel 3'},
                           {'Nombre': 'Cristóbal', 'Grado': 'Nivel 1'},
                           {'Nombre': 'Adriana', 'Grado': 'Nivel 2'}])
grado_df = grado_df.set_index('Nombre')
print(empleados_df.head())
print()
print(grado_df.head())


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
<br>
Tomemos los datos del archivo Automobile_data.csv para crear un gráfico de caja de 3 variables que definen las dimensiones de los automóviles.
Can you write Python code for it?

x = df['length'] #Variable Largo
y = df['width'] #Variable Ancho
z =df['height'] #Variable Alto
dfbp = pd.DataFrame([x,y,z]).T #Creando un DataFrame con las dimensiones de los autosmóviles
dfbp.boxplot(fontsize=13, return_type='axes') #Gráfico de caja de las 3 variables
#Tarea!!!!! Normalice estos datos y haga el nuevo gráfico de caja


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
<br>
La función replace() permite reemplazar valores faltantes en el DataFrame por valores nuevos. En nuestro ejemplo, reemplazaremos con el promedio, que se calcula con la función mean().
Can you write Python code for it?

datos1 = pd.DataFrame([24, np.nan, np.nan, 23,np.nan, 12, np.nan, 17, np.nan, 2 ,5], columns = list('A'))
media = datos1['A'].mean()
media


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
Ahora calculemos el coeficiente de correlación y el p-valor entre las variables 'Caballos de Fuerza' y 'Precio' usando 'stats.pearson()'
Can you write Python code for it?

from scipy import stats
stats.pearsonr(df['horsepower'], df['price'])


I am working on the file "EVI - 2018/EVI 04/Modulo3.ipynb".
The first task is:
El gráfico de dispersión anterior revela que hay una relación lineal positiva entre el tamaño del motor y el precio del auto. Es decir, a medida que aumenta el tamaño del motor aumenta el precio.
Este gráfico de dispersión revela que hay una relación lineal negativa entre las millas que recorre el auto por combustible que usa y el precio del mismo. Es decir, mientras más millas por galón el auto es más económico.
Can you write Python code for it?

import matplotlib.pyplot as plt
from scipy import stats
x=df['highway-mpg'] #Variable predictora
y= df['price'] #Variable objetivo o que deseamos predecir
slope, intercept, r_value, p_value, std_err = stats.linregress(x,y)
line = slope*x+intercept
plt.plot(x,y,'o', x, line)
ax = plt.gca()
fig = plt.gcf()
plt.xlabel('Millas por galón en autopista', fontsize=9)#Etiquetal del eje-x
plt.ylabel('Precio', fontsize=9)#Etiqueta del eje-y
plt.title('Gráfico de dispersión de Millas por galón en autopista Vs. Precio', fontsize=13)#Nombre del gráfico


I am working on the file "EXAMPLES/2018_NL_Central_Analysis.ipynb".
The first task is:
Graphing Games Back
First we create a modified column to numerically represent 'Games Back'
Can you write Python code for it?

# Insert new column called 'int_GB' that represents GB as either negative (meaning behind) or 0.0 meaning in first or tied for first
def modify_gb(gb):
    if gb == 'Tied':
        gb = 0.0
    elif 'up' in gb:
        gb = 0.0 # float(gb.replace('up ', ''))
    else:
        gb = -float(gb)

    return gb

brewers['int_GB'] = brewers['GB'].dropna().apply(modify_gb)
cardinals['int_GB'] = cardinals['GB'].dropna().apply(modify_gb)
cubs['int_GB'] = cubs['GB'].dropna().apply(modify_gb)
pirates['int_GB'] = pirates['GB'].dropna().apply(modify_gb)
reds['int_GB'] = reds['GB'].dropna().apply(modify_gb)


I am working on the file "EXAMPLES/2018_NL_Central_Analysis.ipynb".
The first task is:
Graphing Total Runs and Runs Allowed
First we create new columns for Total Runs, and Total Runs Allowed
Can you write Python code for it?

# Get total runs-to-date and total runs_allowed-to-date for each team
brewers['Total_Runs'] = brewers['R'].cumsum()
brewers['Total_RA'] = brewers['RA'].cumsum()

cubs['Total_Runs'] = cubs['R'].cumsum()
cubs['Total_RA'] = cubs['RA'].cumsum()

cardinals['Total_Runs'] = cardinals['R'].cumsum()
cardinals['Total_RA'] = cardinals['RA'].cumsum()

pirates['Total_Runs'] = pirates['R'].cumsum()
pirates['Total_RA'] = pirates['RA'].cumsum()

reds['Total_Runs'] = reds['R'].cumsum()
reds['Total_RA'] = reds['RA'].cumsum()


I am working on the file "EXAMPLES/2018_NL_Central_Analysis.ipynb".
The first task is:
Graphing Wins and Win Percentages
Here we can see that the Brewers and Cubs were in a tightly contested race all year long. At about game 120 the brewers went into a small slump while the Cardinals ascended, leading to a short period where the Brewers dropped to 3rd place. This didn't last long however, as the Brewers got back into a groove and found their way to 2nd and ultimately 1st place.
Can you write Python code for it?

# Graph Wins Comparison #
plt.rcParams["figure.figsize"] = (8,6)

plt.plot(brewers['Wins'], label='Brewers', c='navy')
plt.plot(cardinals['Wins'], label='Cardinals', c='#b30000')
plt.plot(cubs['Wins'], label='Cubs', c='blue')
plt.plot(pirates['Wins'], label='Pirates', c='gold')
plt.plot(reds['Wins'], label='Reds', c='red')

plt.xticks(np.arange(0, len(brewers.index), step=10))
plt.xlabel('Game')
plt.ylabel('Wins')

plt.legend(loc='lower right')

plt.title("NL Central Wins Comparison ({})".format(year))

# Graph Win Percentage Comparison #
plt.rcParams["figure.figsize"] = (8,6)

plt.plot(brewers['Win_Percentage'], label='Brewers', c='navy')
plt.plot(cardinals['Win_Percentage'], label='Cardinals', c='#b30000')
plt.plot(cubs['Win_Percentage'], label='Cubs', c='blue')
plt.plot(pirates['Win_Percentage'], label='Pirates', c='gold')
plt.plot(reds['Win_Percentage'], label='Reds', c='red')

plt.xticks(np.arange(0, len(brewers.index), step=10))
plt.xlabel('Game')
plt.ylabel('Win Percentage')

plt.legend(loc='lower right')

plt.title("NL Central Win Percentage Comparison ({})".format(year))


I am working on the file "EXAMPLES/2018_NL_Central_Analysis.ipynb".
The first task is:
Summary Statistics
A quick glance at the summary statistics for each team in order of their ending division rank.
Looking at these statistics, you can see the similarity in overall performance. Mean Runs are very similar for the top three teams with the #3 Cardinals slightly edging out the Brewers and Cubs. However, in Runs Allowed, the #2 Cubs come out in front, with the Brewers second, and the Cardinals third.
Interestingly, the hottest streak belongs to the Pirates, who ended the season ranked 4th in the division.
Can you write Python code for it?

brewers.describe()

cubs.describe()

cardinals.describe()

pirates.describe()

reds.describe()


I am working on the file "EXAMPLES/2018_NL_Central_Analysis.ipynb".
The first task is:
What the Graph Shows
Graphing 'Games Back' gives an even clearer image of how the NL Central was contended throughout the year.
The Reds fell to an early 10 Games Back, and never recovered. They ended the year at a season low of 28 Games Back.
The pirates, while contending early on, fell to ~13 Games Back mid-season. They showed signs of life in games 90-110, even surpassing the Cardinals for 3rd place. However, they would soon fall to a season low of ~15 Games Back.
The Brewers and Cubs were consistely within ~4 Games of each other for most of the season. While the Brewers had a rough patch in games 110-135 -- dropping to a season low of 6 Games Back, and falling behind the Cardinals for 2nd place -- they quickly recovered in the final games of the season. They would go on to defeat the Cubs in a Game 163 Divisional Tie-Breaker, becoming the NL Central Champions.
Can you write Python code for it?

# Graph NL Central Games Back Comparison #
plt.rcParams["figure.figsize"] = (8,6)

plt.plot(brewers['int_GB'], label='Brewers', c='navy')
plt.plot(cardinals['int_GB'], label='Cardinals', c='#b30000')
plt.plot(cubs['int_GB'], label='Cubs', c='blue')
plt.plot(pirates['int_GB'], label='Pirates', c='gold')
plt.plot(reds['int_GB'], label='Reds', c='red')

plt.xticks(np.arange(0, len(brewers.index), step=10))
plt.yticks(np.arange(0, -30, step=-2))
plt.xlabel('Game')
plt.ylabel('Games Back (in negative)')

plt.title("NL Central Games Back Comparison ({})".format(year))
plt.legend(loc='lower left')


I am working on the file "EXAMPLES/Example1_plotMetricsOverTime.ipynb".
The first task is:
At this point, we have created a query to retrieve the metrics from the SQLite database, used sqlite3 to connect to the database, retreieved the metrics, closed the connection, and then ensured that the start times are in a datetime format for plotting purposes.
This is what the dataframe looks like:
Can you write Python code for it?


I am working on the file "EXAMPLES/Example1_plotMetricsOverTime.ipynb".
The first task is:
Create a connection to the database and run the query, loading it into a pandas dataframe
Can you write Python code for it?

try:
   conn = sqlite3.connect(db_name)
   DF = pd.read_sql_query(SQLcommand, conn, parse_dates=['start','end'])
   conn.close
except Exception as e:
    print(f"Unable to connect to or find the {metric} table in the database {db_name}:\n{e}")


I am working on the file "EXAMPLES/Example1_plotMetricsOverTime.ipynb".
The first task is:
For plotting purposes, we will create a new dataframe where each column (only one column in this case) is the metric and the associated values, and the index is the date of that value.
Can you write Python code for it?

plotDF = pd.DataFrame()
plotDF[metric] = DF['value']
plotDF.index=DF['start']

print(plotDF)


I am working on the file "EXAMPLES/Example1_plotMetricsOverTime.ipynb".
The first task is:
Now we use that dataframe to produce a plot.
Can you write Python code for it?

ax = plotDF.plot(style='.', color='k', title=metric)
ax.xaxis.set_major_locator(mdates.DayLocator(interval=2))
plt.minorticks_off()
date_form = DateFormatter("%m-%d")
ax.xaxis.set_major_formatter(date_form)
plt.gcf().autofmt_xdate()


I am working on the file "EXAMPLES/Example1_plotMetricsOverTime.ipynb".
The first task is:
The first step is to create a query that will be used to retrieve the sample_rms (or whatever metric you are using in the code block above).
Can you write Python code for it?

SQLcommand = f"SELECT * FROM {metric} WHERE start >= '{startDate}' " \
             f"and start < '{endDate }' and (target like '{target}');"

print("\nCommand used to retrieve metrics from the sqlite database:")
print(SQLcommand)


I am working on the file "EXAMPLES/Example2_plotPSDs.ipynb".
The first task is:
At this point, we have created a query to retrieve the metrics from the SQLite database, used sqlite3 to connect to the database, retreieved the metrics, closed the connection, and then ensured that the start times are in a datetime format for plotting purposes.
This is what the dataframe looks like:
Can you write Python code for it?


I am working on the file "EXAMPLES/Example2_plotPSDs.ipynb".
The first task is:
Create a connection to the database and run the query, loading it into a pandas dataframe
Can you write Python code for it?

try:
    conn = sqlite3.connect(db_name)
    DF = pd.read_sql_query(SQLcommand, conn, parse_dates=['start','end'])
    conn.close
except:
    print(f"Unable to connect to or find the {metric} table in the database {db_name}")


I am working on the file "EXAMPLES/Example2_plotPSDs.ipynb".
The first task is:
Now that we have the dataframe in this arrangement, we can start plotting it up.
First, a colorful version of the plot:
Can you write Python code for it?

ax = plotDF.plot(legend=False, alpha=1, title=f'{target}\n{startDate} through {endDate}')
ax.set_xscale('log')
ax.invert_xaxis()
ax.set_ylabel('power')
plt.grid(True)
plt.savefig(filename)


I am working on the file "EXAMPLES/Example2_plotPSDs.ipynb".
The first task is:
Now we need to set some variables.
Can you write Python code for it?

filename = 'PSD.png'
db_name = '../ispaq_example.db'
metric = 'psd_corrected'
startDate = '2020-10-01'
endDate = '2020-10-15'
target = 'IU.ANMO.00.BH1.M'
filename = f'example2_{target}_{startDate}_{endDate}_PSD.png'
filename2 = f'example2_{target}_{startDate}_{endDate}_PSD_bw.png'


I am working on the file "EXAMPLES/Example2_plotPSDs.ipynb".
The first task is:
The first step is to create a query that will be used to retrieve the psds.
Can you write Python code for it?

SQLcommand = "SELECT * FROM " + metric + \
                     " WHERE start >= '" + startDate + "' " \
                     "and start < '" + endDate + "' " \
                     "and (target like '" + target + "');"

print('\nThis is the query used to retrieve the PSDs from the database:')
print(SQLcommand)


I am working on the file "EXAMPLES/Example2_plotPSDs.ipynb".
The first task is:
We will take the information from the dataframe that we loaded and rearrange it for plotting.
Can you write Python code for it?

plotDF = pd.DataFrame()
for name, group in DFgrouped:
    tmpDF = pd.DataFrame()
    tmpDF[name] = group['power']
    tmpDF.set_axis(group['frequency'], axis='index', inplace=True)

    plotDF = pd.concat([plotDF, tmpDF], axis=1, sort=False)

print("\nThis is the dataframe that will be used to plot:")
print(plotDF)


I am working on the file "EXAMPLES/Example3_plotPDFs.ipynb".
The first task is:
Create a connection to the database and run the query, loading it into a pandas dataframe
Can you write Python code for it?

try:
    conn = sqlite3.connect(db_name)
    DF = pd.read_sql_query(SQLcommand, conn, parse_dates=['start','end'])
    conn.close
except:
    print(f"Unable to connect to or find the {metric} table in the database {db_name}")

if DF.empty:
    print("Empty return: there are no PDFs that were retrieved")

print(DF)


I am working on the file "EXAMPLES/Example3_plotPDFs.ipynb".
The first task is:
Create a new dataframe for plotting: rows are powers, columns are periods, value is percent of hits
Can you write Python code for it?

plotDF = pd.DataFrame(0,index=powers,columns=freqs)
nonZeroFreqs=[]
for power in powers:
    for freq in freqs:
        value = DF[(DF['frequency']==freq) & (DF['power']== power)]['percent'].values
        try:
            plotDF.loc[power,freq] = value[0]
            if value[0] != 0:
                # Keep track of the frequencies that have hits, for axes limits
                nonZeroFreqs.append(freq)
        except:
            continue


I am working on the file "EXAMPLES/Example3_plotPDFs.ipynb".
The first task is:
For each frequency-power bin, calculate what percentage of the total hits for that frequency are at that power.
Can you write Python code for it?

DF['percent'] = DF['hits'] / DF['total'] * 100


I am working on the file "EXAMPLES/Example4_calculatePDFs.ipynb".
The first task is:
Now that we've handled that, we can calculate the PDFs. First, we move into the directory that contains the database, since that's where the ISPAQ code expects us to be. Then we call on the calculate_PDF method, which will return dataframes that contain: PDF values, modes, maximums, and minimums. This may take a few minutes.
Can you write Python code for it?

os.chdir(path_parent)
[pdfDF,modesDF, maxDF, minDF] = PDF_aggregator.calculate_PDF(pd.DataFrame(), args.stations, args.starttime, args.endtime, concierge)
print(pdfDF)


I am working on the file "EXAMPLES/Example4_calculatePDFs.ipynb".
The first task is:
Those will now be used to create a userRequest, which will then be used to create a concierge object. The concierge object will need to later be passed into the method that actually calculates the PDFs.
Can you write Python code for it?

user_request = UserRequest(args, logger=logger)
concierge = concierge.Concierge(user_request, logger=logger)
print(concierge, logger)
print(concierge.logger)


I am working on the file "EXAMPLES/Example4_calculatePDFs.ipynb".
The first task is:
With the modules imported, we now need to set up some variables that will be required to run the ISPAQ code. This includes a logger and an arguments class that contains the fields from the preference file.
Can you write Python code for it?

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s', datefmt='%Y-%m-%d %H:%M:%S')

class args:
    db_name = 'ispaq_example.db'
    starttime = UTCDateTime('2020-10-01')
    endtime = UTCDateTime('2021-10-15')
    metrics = 'pdf'
    stations = 'IU.ANMO.00.BHZ.M'  # The "stations" must refer to a single target, including the quality code (N.S.L.C.Q)
    preferences_file = f'{path_parent}/preference_files/default.txt'
    station_url = 'IRIS'
    dataselect_url = 'IRIS'
    event_url = 'IRIS'
    resp_dir = ''
    output = 'db'
    csv_dir = f'{path_parent}/csv/'
    sncl_format = 'N.S.L.C.'
    sigfigs = 6
    pdf_type = 'plot'
    pdf_interval = 'aggregated'
    plot_include = ''
    pdf_dir = f'{path_parent}/pdfs/'
    psd_dir = f'{path_parent}/psds/'


I am working on the file "EXAMPLES/comparing_legendary_teams.ipynb".
The first task is:
Last let's take a look at score differential
Who had the largest win margins? The plot below shows cumulative run differential throughout the season. The Yankees are the clear winners of this one, which is saying a lot considering they are being compared against two of the best teams of all time! Point Murderers Row.
Can you write Python code for it?

mariners['scorediff'] = (mariners['R'] - mariners['RA']).cumsum()
yankees['scorediff'] = (yankees['R'] - yankees['RA']).cumsum()
reds['scorediff'] = (reds['R'] - reds['RA']).cumsum()

plt.plot(mariners['scorediff'],label=" '01 Mariners")
plt.plot(yankees['scorediff'],label=" '27 Yankees")
plt.plot(reds['scorediff'],label=" '76 Reds")
plt.legend(loc=4)
plt.xlabel('Games into Season')
plt.ylabel('Runs Scored - Runs Against')
plt.title('Cumulative Run Differential');


I am working on the file "EXAMPLES/comparing_legendary_teams.ipynb".
The first task is:
Next let's take a look at their records and runs.
Here the Yankees and Mariners are similar, while the Big Red Machine lags behind. Point Mariners and Yankees.
Can you write Python code for it?

# create a numeric indicator of a win and take cumulative sum of wins-to-date at each point in the season
mariners['win-count'] = np.where(mariners['W/L']=='W', 1, 0).cumsum()
yankees['win-count'] = np.where(yankees['W/L']=='W', 1, 0).cumsum()
reds['win-count'] = np.where(reds['W/L']=='W', 1, 0).cumsum()

plt.plot(mariners['win-count'],label=" '01 Mariners")
plt.plot(yankees['win-count'],label=" '27 Yankees")
plt.plot(reds['win-count'],label=" '76 Reds")
plt.legend(loc=4)
plt.xlabel('Games into Season')
plt.ylabel('Win Count')
plt.title('Record Throughout Season');


I am working on the file "EXAMPLES/comparing_legendary_teams.ipynb".
The first task is:
Summary Statistics
Let's start by looking at some summary stats on each team's season. The '01 Mainers had the best hot streak of all, winning 15 straight at one point. The '27 Yankees, however, had the highest scoring offense with an impressive 6.3 runs per game.
Can you write Python code for it?

mariners.describe()

reds.describe()

yankees.describe()


I am working on the file "EXAMPLES/fivethirtyeight_new_science_of_hitting.ipynb".
The first task is:
As you can see, the "sweet spot" where these two metrics are just right for producing a hit is not a simple blob on the graph as one might expect! In fact, there seem to be two distinct patterns happening here. Let's take a look at this same chart for home runs only and see if those are responsible for the differing patterns.
Can you write Python code for it?

data2['hr'] = data2.events=='home_run'

fig, ax = plt.subplots(figsize=(8, 8))
sns.despine(fig, left=True, bottom=True)
sns.scatterplot(x="launch_speed", y="launch_angle",
                hue="hr",
                palette='binary',
                data=data2, ax=ax)
ax.set_title("Home Runs by Launch Angle and Exit Velocity");


I am working on the file "EXAMPLES/fivethirtyeight_new_science_of_hitting.ipynb".
The first task is:
First, collect two months worth of pitch-level data from Statcast
Can you write Python code for it?

# collect Statcast data on all pitches from the months of May and June
data = statcast('2017-05-01', '2017-06-30')
print(data.shape)


I am working on the file "EXAMPLES/fivethirtyeight_new_science_of_hitting.ipynb".
The first task is:
Now for the fun part: plot a batted ball's probability of becoming a hit as a function of its launch angle and exit velocity.
Can you write Python code for it?

fig, ax = plt.subplots(figsize=(8, 8))
sns.despine(fig, left=True, bottom=True)
sns.scatterplot(x="launch_speed", y="launch_angle",
                hue="estimated_ba_using_speedangle",
                palette='viridis',
                data=data2, ax=ax)
ax.set_title("Hit probability by Launch Angle and Exit Velocity");


I am working on the file "EXAMPLES/fivethirtyeight_new_science_of_hitting.ipynb".
The first task is:
So there you have it. From the two patterns observed when plotting hit probability against exit velocity and launch angle, the round cluster around 115mph speed and 30 degrees launch angle represents mostly home runs, while the other pattern observed is within-the-park hits.
Another question we can ask: is it always better to hit the ball harder?
Here we will look at a few metrics, but most importantly expected wOBA (weighted on base average), as they vary by hit speed. First, let's break hit speed into five evenly spaced bins and see how a few variables look when broken down by exit velocity.
Can you write Python code for it?


I am working on the file "EXAMPLES/fivethirtyeight_new_science_of_hitting.ipynb".
The first task is:
There are definitely some patterns there. To take a closer look at the metric of interest, let's use a few more bins and graph the expected weighted on base average value of a hit against its exit velocity. This should give a rough answer as to whether it's always better to hit the ball harder.
Can you write Python code for it?

groups = data2.groupby(pd.cut(data2.launch_speed, 30))
ax = groups['estimated_woba_using_speedangle'].mean().plot()
ax.set_xlabel('Launch Speed', fontsize=14)
ax.set_ylabel('Expected wOBA Value', fontsize=14);


I am working on the file "EXAMPLES/imputed_derivation.ipynb".
The first task is:
Group over years to see the cases where the threshold is passed
Can you write Python code for it?

# Print out the results
summary.groupby(["launch_angle", "launch_speed","bb_type"]).size().reset_index(name="Years above threshold")


I am working on the file "EXAMPLES/imputed_derivation.ipynb".
The first task is:
Output distributions look clean, so this set looks like a good start.
Also want to output top handful for each bb_type for each year. This repeats some code above - not the cleanest, but I don't want the key parts of the notebook buried under a huge number of tables.
Can you write Python code for it?

for year,df in zip(range(2015, 2021),dfs):
    for bb_type in dfs[0].bb_type.dropna().unique():
        # Isolate each bb_type
        i = df[df["bb_type"] == bb_type]
        # Sort by pairs of launch angle and speed
        i = i.groupby(["launch_angle", "launch_speed"]).size().reset_index(name="count").sort_values("count", ascending=False)
        # Derive fraction of total
        i["fraction"] = i["count"] / i["count"].sum()
        print(f"bb_type: {bb_type}, year: {year}")
        print(i.head(5))


I am working on the file "EXAMPLES/imputed_derivation.ipynb".
The first task is:
Since there's a bit of variance year-to-year and especially difference in 2020 with Hawkeye, grab a month from each year
Can you write Python code for it?

# Grab 1 month per year
dfs = []
for year in range(2015, 2021):
    print(f"Starting year {year}")
    dfs.append(statcast(start_dt=f'{year}-08-01', end_dt=f'{year}-09-01',verbose=False))


I am working on the file "EXAMPLES/imputed_derivation.ipynb".
The first task is:
Validate Results:
Can you write Python code for it?

for i,year in enumerate(range(2015, 2021)):
    plot_bb_profile(dfs[i])
    plt.title(f'{year}')
    plt.xlabel("Launch Angle")
    plt.show()

for i,year in enumerate(range(2015, 2021)):
    skimmed_df = dfs[i].merge(summary.drop_duplicates(), how="left", on=["launch_angle","launch_speed","bb_type"],indicator=True)
    plot_bb_profile(skimmed_df[skimmed_df['_merge'] == 'left_only'])
    plt.title(f'{year}')
    plt.xlabel("Launch Angle")
    plt.show()


I am working on the file "EXAMPLES_diffusion1D.ipynb".
The first task is:
1D diffusion
Simple analytic solutions assuming diffusivity is independent of the diffusing species.
Error functions (the default) are usually preferred for the early stages of diffusion, and the infinite sum (erf_or_sum='infsum') more accurately represents diffusion at later stages (Crank 1975, Section 2.1)
Can you write Python code for it?

fig, ax, x_data, y_data = models.diffusion1D(length_microns=500., log10D_m2s=-12., time_seconds=3600.)

fig, ax, x_data, y_data = models.diffusion1D(length_microns=500, log10D_m2s=-12, time_seconds=2*3600,
                                             init=0.2, fin=1., symmetric=False, centered=False)


I am working on the file "EXAMPLES_diffusion1D.ipynb".
The first task is:
1D diffusion in a thin slab
See Eq 4.18 in Crank, 1975. C is the concentration
Can you write Python code for it?

from pynams.diffusion import models
time_hours, c_over_c0 = models.diffusionThinSlab(log10D_m2s=-14., thickness_microns=300.,
                                                 max_time_hours=2000, timesteps=300)
plt.plot(time_hours, c_over_c0)
plt.xlabel('Time in hours')
plt.ylabel('Concentration / Initial concentration\nmeasured in the center of the slab')
plt.title('1-dimensional diffusion in a thin slab');


I am working on the file "EXAMPLES_diffusion1D.ipynb".
The first task is:
Fit 1D diffusion data
Default solves for time while holding constant
* the initial constant = the maximum concentration (vary_initial=True)
* the log10 of the diffusivity in m2/s constant at -12
* the final concentration = zero
Change these setting using keywords log10Dm2s, vary_initial, vary_final, varyD, vary_time, initial_unit_value, final_unit_value
Can you write Python code for it?

initial, final, log10D, minutes = profile.fitD(symmetric=False)


I am working on the file "EXAMPLES_diffusion1D.ipynb".
The first task is:
Plot 1D diffusion profiles on top of your data
The plot_diffusion method combines plot_area_profile() with diffusion1D() and includes most of the same keywords.
Can you write Python code for it?

fig, ax, ax_water = profile.plot_diffusion(time_seconds=60*5, log10D_m2s=-13, centered=False)

fig, ax, ax_water = profile.plot_diffusion(time_seconds=60*5, log10D_m2s=-13, centered=True,
                                           fin=0.6, symmetric=False)


I am working on the file "EXAMPLES_diffusion3D.ipynb".
The first task is:
3-dimensional diffusion models
non-path integrated (3Dnpi) models
3-dimensional diffusion in a rectangular parallelepiped can be determined analytically simply by multiplying the 1D unit solutions together (Carslaw & Jaeger 1959 Section 6.4; Crank 1975 Section 2.5.1).
'Non-path integrated' indicates that these are diffusion profiles assumed to go through the center of the sample, e.g., for a sample that has been polished into the center such that each measurement is assumed to have a constant concentration.
Required input:
* A list of the 3 lengths in each direction (if only 1, assumes a cube shape)
* A list of the 3 diffusivities in each direction (if only 1, assumes isotropic)
* Time
Output:
* figure and axes handles if plot3=True (default) and axes=None (default)
* the 3D matrix of concentrations throughout the crystal, v
* 2 lists of the x and y values across each profile, as plotted
Change the length of the x and y lists with keyword points (default=50)
Can you write Python code for it?

fig, axes, v, x, y = models.diffusion3Dnpi(lengths_microns=my_sample.lengths_microns,
                                           log10Ds_m2s=-11, time_seconds=3600.)

models.diffusion3Dnpi(lengths_microns=1000., log10Ds_m2s=[-13., -12., -11.],
                      time_seconds=30*60, initial=0, final=0.8, ytop=0.8,
                      centered=False, show_line_at_initial=False);

v, x, y = models.diffusion3Dnpi(lengths_microns=my_sample.lengths_microns, final=0.5,
                                log10Ds_m2s=-10.5, time_seconds=3600., axes=axes,
                                styles3=[styles.style_1]*3)

print('shape of 3D matrix of concentration data')
print(np.shape(v))
print('\ndata shown in plots above is returned as x and y')


I am working on the file "EXAMPLES_diffusion3D.ipynb".
The first task is:
Group the three profiles together into a single Block
Can you write Python code for it?

my_first_block = Block(name = 'Kunlun diopside initial',
                       profiles = [profile1, profile2, profile3])


I am working on the file "EXAMPLES_diffusion3D.ipynb".
The first task is:
Plot areas for each profile in Block
Be sure to run the make_areas() method first
Can you write Python code for it?

my_first_block.get_baselines(print_confirmation=False)
my_first_block.make_areas()
fig, list_of_3_axes = my_first_block.plot_areas_3panels()


I am working on the file "EXAMPLES_diffusion3D.ipynb".
The first task is:
Plot whole-block areas for each profile in Block
Can you write Python code for it?

fig, list_of_3_axes = my_first_block.plot_areas_3panels(wholeblock=True)


I am working on the file "EXAMPLES_diffusion3D.ipynb".
The first task is:
Specify the initial profiles
If you want to treat these data as whole-block data, you will need to specify the initial profile for each profile either directly (see below) or using the initial_profile keyword when creating the profile.
Can you write Python code for it?

profile1.initial_profile = profile1 # this is the default assumed above


I am working on the file "EXAMPLES_diffusion3D.ipynb".
The first task is:
whole-block (3Dwb) models
'Whole-block' here indicates that the measurements include changes along the direction of the infrared ray path, i.e., the thickness direction. The calculations start out like 3Dnpi above and then account for the path integration.
Required input:
* A list of the 3 lengths in each direction (if only 1, assumes a cube shape)
* A list of the 3 diffusivities in each direction (if only 1, assumes isotropic)
* Time
* A list of the raypaths for each traverse
Output:
* figure and axes handles if plot3=True (default) and axes=None (default)
* 2 lists of the x and y values across each profile, as plotted
Other keywords and options are similar to 3Dnpi above.
Can you write Python code for it?

raypaths = ['c', 'c', 'b']
fig, ax, x, y = models.diffusion3Dwb(lengths_microns=my_sample.lengths_microns,
                                     log10Ds_m2s=-11, time_seconds=3600.,
                                     raypaths=raypaths)
for idx, axis in enumerate(ax):
    axis.text(0, 0.2, ' '.join(('ray path ||', raypaths[idx])), ha='center')


I am working on the file "EXAMPLES_experimentation.ipynb".
The first task is:
What does that fO2 correspond to in mV reported by an O2 sensor?
Can you write Python code for it?

from pynams import V_from_log10fO2
V_from_log10fO2(celsius=1000, log10fO2=fO2)


I am working on the file "EXAMPLES_experimentation.ipynb".
The first task is:
What is the log base 10 of the fO2 in bars for a given temperature and buffer?
Can you write Python code for it?

from pynams import fO2
fO2 = fO2(celsius=1000, buffer_curve='NNO')
print(fO2)


I am working on the file "EXERCICIO_tocantins_focos_queimadas_set_2016.ipynb".
The first task is:

Agrupando os municípios e os focos:
Can you write Python code for it?

focos_municipios = pd.concat([municipios_tocantins, focos_mes], axis=1)
focos_municipios.head()


I am working on the file "EXERCICIO_tocantins_focos_queimadas_set_2016.ipynb".
The first task is:

Importando o Shapefile de municípios do IBGE:
Can you write Python code for it?

municipios = gpd.read_file(r"C:\Users\dougl\Desktop\programacao\focos_2016\municipios\municipios.shp")

#17 é o código IBGE para o Estado de Tocantins
municipios_to = municipios[municipios["GEOCODIGO"].str[0:2] == "17"]

focos_setembro.crs = {'init': 'epsg:4674'}
municipios_to.crs = {'init': 'epsg:4674'}


I am working on the file "EXERCICIO_tocantins_focos_queimadas_set_2016.ipynb".
The first task is:

Juntando os dados em um mesmo grupo:
Can you write Python code for it?

tocantins_dados = gpd.sjoin(focos_setembro, municipios_to, how="right", op='intersects')

tocantins_dados.head(4)

municipios_tocantins = municipios_to.set_index('NOME')

focos_mes = tocantins_dados.groupby('NOME').Municipi.count()

focos_mes.index.name = "Municipios"
focos_mes.name = "focos"
focos_mes.head()


I am working on the file "EXERCICIO_tocantins_focos_queimadas_set_2016.ipynb".
The first task is:

Testando o arquivo Shape
Can you write Python code for it?


I am working on the file "EXERCISE 3 - pathways.ipynb".
The first task is:
Filtering results on p-value
Back to Python....we can access top_fit variable because it was passed above as an "output" parameter to the R code. We use it to select the probes below a specific p-value threshold.
Can you write Python code for it?

top_probes = top_fit[top_fit.p_value < 0.05] \
              .set_index(['probe']).sort(['p_value'], ascending = True)
top_probes

def get_gene_name(probe):
    return probe.split('\\')[0]

top_probes['gene_name'] = top_probes.index.map(get_gene_name)
top_probes


I am working on the file "EXERCISE 3 - pathways.ipynb".
The first task is:
Here we reorganize the data such that each line represents a subject and the columns represent the different clinical attributes.
Can you write Python code for it?

obs_df2 = obs_df1.pivot(index = 'subject.inTrialId', columns = 'label', values = 'value')
obs_df3 = obs_df2.convert_objects()
obs_df4 = obs_df3.rename(
    columns = lambda c: c.replace('\Public Studies\COPD_Bhattacharya_GSE8581\\', '')[:-1],
    inplace = False)
obs_df4


I am working on the file "EXERCISE 3 - pathways.ipynb".
The first task is:
Pandas is useful for converting the JSON format, received by the REST API, into an easy to use Python Pandas DataFrame object.
Can you write Python code for it?

studies = api.get_studies()
json_normalize(studies['studies'])

study_id = 'GSE8581'
obs = api.get_observations(study = study_id)
obs_df1 = json_normalize(obs)

obs_df1


I am working on the file "EXERCISE 3 - pathways.ipynb".
The first task is:
Part 2 - Retrieving molecular (aka "high dimensional") data

Downloading the expression data for our chosen study
This can take a while (~1 minute)
Can you write Python code for it?

(hdHeader, hdRows) = api.get_hd_node_data(study = study_id,
                                          node_name = 'Lung',
                                          projection='log_intensity')


I am working on the file "EXERCISE 3 - pathways.ipynb".
The first task is:
Select only the data rows where the probes are mapped to a gene (aka bioMarker). Gene ID\probe ID will form the row names while the patientId values are used for the columns.
Can you write Python code for it?

rows = [row.value[0].doubleValue._values for row in hdRows if row.bioMarker != 'null']
row_names = [row.bioMarker + "\\" + row.label for row in hdRows if row.bioMarker != 'null']
col_names = [assay.patientId for assay in hdHeader.assay]


I am working on the file "EXERCISE 3 - pathways.ipynb".
The first task is:
Using the control and treatment sets made a few cells above, we build the "design" data frame showing which subjects are part of which set.
Can you write Python code for it?

design_df = DataFrame({'control': control[col_names], 'treatment': treatment[col_names]})
design_df


I am working on the file "EXERCISE TranSMART REST API V2 (2017).ipynb".
The first task is:
And do the same for all observations for this patient set.
Can you write Python code for it?

obs = api.get_observations(study = study_id, patientSet = patient_set_id)
obsDataframe = json_normalize(api.format_observations(obs))
obsDataframe


I am working on the file "EXERCISE TranSMART REST API V2 (2017).ipynb".
The first task is:
Next we ask for the full list of observations for this study. This list will include one row per observation, with information from all their dimensions. The columns will have headers like <dimension name>.<field name> and numericValue or stringValue for the actual observation value.
Can you write Python code for it?

obs = api.get_observations(study = study_id)
obsDataframe = json_normalize(api.format_observations(obs))
obsDataframe

#DO STUFF WITH THE TRAINING STUDY HERE


I am working on the file "EXERCISE TranSMART REST API V2 (2017).ipynb".
The first task is:
Next we import and configure Pandas, a Python library to work with data.
Can you write Python code for it?

import pandas as pd
from pandas.io.json import json_normalize

pd.set_option('max_colwidth', 1000)
pd.set_option("display.max_rows",100)
pd.set_option("display.max_columns",100)


I am working on the file "EXERCISE TranSMART REST API V2 (2017).ipynb".
The first task is:
Now let's return all patients for the patient set we made!
Can you write Python code for it?

patients = api.get_patients(patientSet = patient_set_id)
json_normalize(patients['patients'])


I am working on the file "EXERCISE TranSMART REST API V2 (2017).ipynb".
The first task is:
Part 2: Combining Glowing Bear and the Python client
For the second part we will work with the Glowing Bear user interface that was developed at The Hyve, funded by IMI Translocation and BBMRI.
An API is great to extract exactly the data you need and analyze that. But it is harder to get a nice overview of all data that is available and define the exact set to extract. That is where the Glowing Bear was built for.
Please go to http://glowingbear2-head.thehyve.net and create a Patient Set on the Data Selection tab (under Select patients). Once you have saved your patient set, copy the patient set identifier and paste that below.
Can you write Python code for it?


I am working on the file "EXERCISE TranSMART REST API V2 (2017).ipynb".
The first task is:
We choose the TRAINING study and ask for all patients in this study. You will get a list with their patient details and patient identifier.
Can you write Python code for it?

study_id = 'TRAINING'
patients = api.get_patients(study = study_id)
json_normalize(patients['patients'])


I am working on the file "EZGAN.ipynb".
The first task is:
And here's the generator. When it's called, it starts by creating a batch of random noise from the latente space $z$, then passes it through a handful of convolutions to produce a 28 x 28 image.
This structure is borrowed from Tim O'Shea.
Can you write Python code for it?

def generator(batch_size, z_dim):
    z = tf.truncated_normal([batch_size, z_dim], mean=0, stddev=1, name='z')
    g_w1 = tf.get_variable('g_w1', [z_dim, 3136], dtype=tf.float32, initializer=tf.truncated_normal_initializer(stddev=0.02))
    g_b1 = tf.get_variable('g_b1', [3136], initializer=tf.truncated_normal_initializer(stddev=0.02))
    g1 = tf.matmul(z, g_w1) + g_b1
    g1 = tf.reshape(g1, [-1, 56, 56, 1])
    g1 = tf.contrib.layers.batch_norm(g1, epsilon=1e-5, scope='bn1')
    g1 = tf.nn.relu(g1)

    # Generate 50 features
    g_w2 = tf.get_variable('g_w2', [3, 3, 1, z_dim/2], dtype=tf.float32, initializer=tf.truncated_normal_initializer(stddev=0.02))
    g_b2 = tf.get_variable('g_b2', [z_dim/2], initializer=tf.truncated_normal_initializer(stddev=0.02))
    g2 = tf.nn.conv2d(g1, g_w2, strides=[1, 2, 2, 1], padding='SAME')
    g2 = g2 + g_b2
    g2 = tf.contrib.layers.batch_norm(g2, epsilon=1e-5, scope='bn2')
    g2 = tf.nn.relu(g2)
    g2 = tf.image.resize_images(g2, [56, 56])

    # Generate 25 features
    g_w3 = tf.get_variable('g_w3', [3, 3, z_dim/2, z_dim/4], dtype=tf.float32, initializer=tf.truncated_normal_initializer(stddev=0.02))
    g_b3 = tf.get_variable('g_b3', [z_dim/4], initializer=tf.truncated_normal_initializer(stddev=0.02))
    g3 = tf.nn.conv2d(g2, g_w3, strides=[1, 2, 2, 1], padding='SAME')
    g3 = g3 + g_b3
    g3 = tf.contrib.layers.batch_norm(g3, epsilon=1e-5, scope='bn3')
    g3 = tf.nn.relu(g3)
    g3 = tf.image.resize_images(g3, [56, 56])

    # Final convolution with one output channel
    g_w4 = tf.get_variable('g_w4', [1, 1, z_dim/4, 1], dtype=tf.float32, initializer=tf.truncated_normal_initializer(stddev=0.02))
    g_b4 = tf.get_variable('g_b4', [1], initializer=tf.truncated_normal_initializer(stddev=0.02))
    g4 = tf.nn.conv2d(g3, g_w4, strides=[1, 2, 2, 1], padding='SAME')
    g4 = g4 + g_b4
    g4 = tf.sigmoid(g4)

    # No batch normalization at the final layer, but we do add
    # a sigmoid activator to make the generated images crisper.
    # Dimensions of g4:...


I am working on the file "EZGAN.ipynb".
The first task is:
Here we set up our losses and optimizers.
Can you write Python code for it?

sess = tf.Session()

batch_size = 50
z_dimensions = 100

x_placeholder = tf.placeholder("float", shape = [None,28,28,1], name='x_placeholder')
# x_placeholder is for feeding input images to the discriminator

Gz = generator(batch_size, z_dimensions)
# Gz holds the generated images

Dx = discriminator(x_placeholder)
# Dx hold the discriminator's prediction probabilities
# for real MNIST images

Dg = discriminator(Gz, reuse=True)
# Dg holds discriminator prediction probabilities for generated images

g_loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=Dg, labels=tf.ones_like(Dg)))

d_loss_real = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=Dx, labels=tf.fill([batch_size, 1], 0.9)))
d_loss_fake = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=Dg, labels=tf.zeros_like(Dg)))
d_loss = d_loss_real + d_loss_fake

tvars = tf.trainable_variables()

d_vars = [var for var in tvars if 'd_' in var.name]
g_vars = [var for var in tvars if 'g_' in var.name]

# Train the discriminator
# Increasing from 0.001 in GitHub version
with tf.variable_scope(tf.get_variable_scope(), reuse=False) as scope:
    d_trainer_fake = tf.train.AdamOptimizer(0.0001).minimize(d_loss_fake, var_list=d_vars)
    d_trainer_real = tf.train.AdamOptimizer(0.0001).minimize(d_loss_real, var_list=d_vars)

    # Train the generator
    # Decreasing from 0.004 in GitHub version
    g_trainer = tf.train.AdamOptimizer(0.0001).minimize(g_loss, var_list=g_vars)


I am working on the file "EZGAN.ipynb".
The first task is:
Here's where we pass helpful summary scalars and sample images to TensorBoard.
Can you write Python code for it?

tf.summary.scalar('Generator_loss', g_loss)
tf.summary.scalar('Discriminator_loss_real', d_loss_real)
tf.summary.scalar('Discriminator_loss_fake', d_loss_fake)

d_real_count_ph = tf.placeholder(tf.float32)
d_fake_count_ph = tf.placeholder(tf.float32)
g_count_ph = tf.placeholder(tf.float32)

tf.summary.scalar('d_real_count', d_real_count_ph)
tf.summary.scalar('d_fake_count', d_fake_count_ph)
tf.summary.scalar('g_count', g_count_ph)

# Sanity check to see how the discriminator evaluates
# generated and real MNIST images
d_on_generated = tf.reduce_mean(discriminator(generator(batch_size, z_dimensions)))
d_on_real = tf.reduce_mean(discriminator(x_placeholder))

tf.summary.scalar('d_on_generated_eval', d_on_generated)
tf.summary.scalar('d_on_real_eval', d_on_real)

images_for_tensorboard = generator(batch_size, z_dimensions)
tf.summary.image('Generated_images', images_for_tensorboard, 10)
merged = tf.summary.merge_all()
logdir = "tensorboard/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S") + "/"
writer = tf.summary.FileWriter(logdir, sess.graph)


I am working on the file "EZGAN.ipynb".
The first task is:
In a clever structure suggested by Adit Deshpande, the discriminator and generator networks will sit inside separate functions that we can call as needed.
Here's the discriminator network. It takes x_image and returns a real/fake classification. As you'll see below, we can either feed x_image through a placeholder, or from another tensor—for instance, the output of the generator.
This network structure is taken directly from TensorFlow's Deep MNIST for Experts tutorial.
Can you write Python code for it?

def discriminator(x_image, reuse=False):
    if (reuse):
        tf.get_variable_scope().reuse_variables()

    # First convolutional and pool layers
    # These search for 32 different 5 x 5 pixel features
    d_w1 = tf.get_variable('d_w1', [5, 5, 1, 32], initializer=tf.truncated_normal_initializer(stddev=0.02))
    d_b1 = tf.get_variable('d_b1', [32], initializer=tf.constant_initializer(0))
    d1 = tf.nn.conv2d(input=x_image, filter=d_w1, strides=[1, 1, 1, 1], padding='SAME')
    d1 = d1 + d_b1
    d1 = tf.nn.relu(d1)
    d1 = tf.nn.avg_pool(d1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

    # Second convolutional and pool layers
    # These search for 64 different 5 x 5 pixel features
    d_w2 = tf.get_variable('d_w2', [5, 5, 32, 64], initializer=tf.truncated_normal_initializer(stddev=0.02))
    d_b2 = tf.get_variable('d_b2', [64], initializer=tf.constant_initializer(0))
    d2 = tf.nn.conv2d(input=d1, filter=d_w2, strides=[1, 1, 1, 1], padding='SAME')
    d2 = d2 + d_b2
    d2 = tf.nn.relu(d2)
    d2 = tf.nn.avg_pool(d2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

    # First fully connected layer
    d_w3 = tf.get_variable('d_w3', [7 * 7 * 64, 1024], initializer=tf.truncated_normal_initializer(stddev=0.02))
    d_b3 = tf.get_variable('d_b3', [1024], initializer=tf.constant_initializer(0))
    d3 = tf.reshape(d2, [-1, 7 * 7 * 64])
    d3 = tf.matmul(d3, d_w3)
    d3 = d3 + d_b3
    d3 = tf.nn.relu(d3)

    # Second fully connected layer
    d_w4 = tf.get_variable('d_w4', [1024, 1], initializer=tf.truncated_normal_initializer(stddev=0.02))
    d_b4 = tf.get_variable('d_b4', [1], initializer=tf.constant_initializer(0))

    # Final layer
    d4 = tf.matmul(d3, d_w4) + d_b4
    # d4 dimensions: batch_size x 1

    return d4


I am working on the file "EZGAN.ipynb".
The first task is:
Now let's see some of the images produced by the generator. (The generator has also been sending its images to TensorBoard regularly; click the "images" tab in TensorBoard to see them as this runs.)
And, as a sanity check, let's look at some real MNIST images and make sure that the discriminator correctly classifies them as real MINST images.
Can you write Python code for it?

test_images = sess.run(generator(10, 100))
test_eval = sess.run(discriminator(x_placeholder), {x_placeholder: test_images})

real_images = mnist.validation.next_batch(10)[0].reshape([10, 28, 28, 1])
real_eval = sess.run(discriminator(x_placeholder), {x_placeholder: real_images})

# Show discriminator's probabilities for the generated images,
# and display the images
for i in range(10):
    print(test_eval[i])
    plt.imshow(test_images[i, :, :, 0], cmap='Greys')
    plt.show()

# Now do the same for real MNIST images
for i in range(10):
    print(real_eval[i])
    plt.imshow(real_images[i, :, :, 0], cmap='Greys')
    plt.show()