I am working on the file "DSMCER_Hw/dsmcer-hw4-danielfather7/HW4 skel.ipynb".
The first task is:
Part 1: Short answer questions
Explain in your own words the following concepts:

Residual squared error : Residual means comparing with estimated value (not true value), so it means the square of the diviation between observed value and the estimated value. Considering there is a model $y_i = \alpha{x_i} + \beta$, then residual would be $\epsilon =  y_{i, obs} - y_{i, est} = y_{i, obs} - (\alpha{x_i} + \beta)$, and residual square would be $\epsilon^2$.
Sum of squared residuals : Add all $\epsilon^2$ from i = 1 to i = n. The least squares approach in simple linear regression chooses $\beta_0$ and $\beta_1$ to minimize the RSS.
$R^2$ statistic : $R^2$ is a kind of fraction and it always takes on a value between 0 to 1. It tells how fit a regression is.

note: I expect you to not simply read these definitions in ISL and modestly change the words. Please take a few minutes to read multiple definitions and come up with your own definition
Part 2: Logistic Regression

Look up the 'Default' dataset features in the ISL textbook.  See page  130 of the book and online resources for more information.  This dataset should be easy to find

Implement both the linear and logistic regression models using sklearn, calculate the coefficients $\beta_0$ and $\beta_1$ and exactly (to the best of your ability) reproduce Figure 4.2 in the book.
Can you write Python code for it?

data = pd.read_csv('Default.csv')
data = data.drop('Unnamed: 0',axis = 1)
#change Yes, No to 1, 0.
data['def_chg'] = data.default.factorize()[0]
data.head()

#linear regression
regr = linear_model.LinearRegression()
regr.fit(data.balance.reshape(-1,1),data.def_chg)
#logistic regression
log_org = linear_model.LogisticRegression(solver='newton-cg')
log_org.fit(data.balance.reshape(-1,1),data.def_chg)
x_array = np.arange(data.balance.min(),data.balance.max())

plt.figure(figsize=(10,5))

#left
plt.subplot(121)
plt.scatter(data.balance,data.def_chg)
plt.plot(data.balance,regr.predict(data.balance.reshape(-1,1)),color = 'g')
plt.plot([-100,3000],[0,0],linestyle = 'dashed', color = 'k')
plt.plot([-100,3000],[1,1],linestyle = 'dashed', color = 'k')
plt.xlim([-140,3000])
plt.xlabel('Balance')
plt.ylabel('Probability of default')

#right
plt.subplot(122)
plt.scatter(data.balance,data.def_chg)
plt.plot(x_array, log_org.predict_proba(x_array.reshape(-1,1))[:,1], color = 'g')
plt.plot([-100,3000],[0,0],linestyle = 'dashed', color = 'k')
plt.plot([-100,3000],[1,1],linestyle = 'dashed', color = 'k')
plt.xlim([-140,3000])
plt.xlabel('Balance')
plt.ylabel('Probability of default')

print('B0, B1 for linear: ', regr.intercept_, regr.coef_[0])
print('B0, B1 for logistic: ', log_org.intercept_[0], log_org.coef_[0][0])


I am working on the file "DSMCER_Hw/dsmcer-hw4-danielfather7/HW4 skel.ipynb".
The first task is:
Part 3: Simple Linear Regression

Make sure you understand all the components of the SLR model Jim showed in class and code the same thing yourself

Generate a separate set of training data (25 points) and validation data (15 poins).  Each data set should have the same amount of irreducible error (random noise term) applied to it

Make a plot of the error in the testing data as a function of the random strength. There is no "one answer" for this part, open ended graded based on creativity and quality of results

For the exact example I showed in class, calculate the RSE, SSR, $R^2$, 95% confidence intervals for both $\beta_i$, and evaluate the P-value at 0.05 significance level for $\beta_1$

Explain in plain language the meaning of the P-value test
Can you write Python code for it?

#model: Y = 3X + 4
#Generate a separate set of training data (25 points) and validation data (15 poins).
pts=25
x_tra = np.linspace(-50,50,num = pts)
x_val = np.linspace(-50,50,num = 15)
B0=4
B1=3
y_act = B0 + B1*x_tra
y_val = B0 + B1*x_val
np.random.seed(123)

#add noise scaled to 25% of range to data
yrand = y_act + .25*(y_act.max()-y_act.min())*np.random.normal(size = pts)
yrand_val = y_val + .25*(y_val.max()-y_val.min())*np.random.normal(size = 15)

plt.figure(figsize=(5,15))
plt.subplot(311)
plt.scatter(x_tra,yrand)
plt.title('Training data (25 points)')
plt.ylabel('y')
plt.xlabel('x')
plt.grid(linestyle = 'dashed')

plt.subplot(312)
plt.scatter(x_val,yrand_val)
plt.title('Validation data (15 points)')
plt.ylabel('y')
plt.xlabel('x')
plt.grid(linestyle = 'dashed')

plt.subplot(313)
#Use a for loop to test adding different percentage strength of error
for i in range(100):
    yrand_val = y_val + (i/100)*(y_val.max()-y_val.min())*np.random.normal(size = 15)
#calculate RSE
    plt.plot(i/100,math.sqrt((((y_val-yrand_val)**2).sum())/(15-2)),marker='o',color = 'b')
plt.title('Error in testing data as a function of random strength')
plt.ylabel('RSE')
plt.xlabel('Percent Error in Testing Data')
plt.grid(linestyle = 'dashed')

#exact example showed in class
#model: Y = 3X + 4
#size of training data and scale of random noise
pts=25
noisescale=.25
x=np.linspace(-50,50,num=pts)
B0=4
B1=3
yactual=B0+B1*x
np.random.seed(123)
#add noise scaled to 25% of range to data
yrand=yactual+noisescale*(yactual.max()-yactual.min())*np.random.normal(size=pts)
#SLR
regr=linear_model.LinearRegression()
regr.fit(x.reshape(-1,1),yrand)
print('B0, B1: ',regr.intercept_, regr.coef_[0])

ypred = regr.predict(x.reshape(-1,1))
RSS = ((yrand - ypred)**2).sum()
print('RSS =', RSS)
RSE = math.sqrt(RSS/(pts - 2))
print ('RSE =', RSE)
TSS = ((yrand - yrand.mean())**2).sum()
Rsqr = (TSS-RSS)/TSS
print ('R^2 =', Rsqr)
SE_beta0 = math.sqrt((RSE**2)*(1/pts + x.mean()/((x - x.mean())**2).sum()))
SE_beta1 = math.sqrt(RSE**2 / ((x -...


I am working on the file "DSMCER_Hw/dsmcer-hw4-danielfather7/HW4 skel.ipynb".
The first task is:
Part 5: Resampling methods
I would like you to write a clearly labeled function that performs bootstrapping using a subset of the HCEPD_100K.csv data. Your function should be versatile in terms of how much of the data is selected for the bootstrap and and how many iterations are run.

Prepare a plot of MSE (using the same 3-parameter PCE fit (using mass/VOC/lumo features). The x-axis should be the number of bootstrap samples and the y-axis should be the MSE. You should show a line for boostrap samples sizes of 100, 1000, and 5000.

For the same sampling you should show a plot the estimate of the three relevant beta coefficients
Can you write Python code for it?

#Create a bootstrap sampling fuction and return average MSE and beta
def bootstrap(data, num, iter_):
    MSE_avg = 0
    B1_avg = 0
    B2_avg = 0
    B3_avg = 0
    for i in range(iter_):
        #choose randomly with replacement
        df = data.sample(num, replace = True)
        #MLR
        regr = linear_model.LinearRegression()
        regr.fit(df[['mass','voc','e_lumo_alpha']],df.pce)
        pred = regr.predict(df[['mass','voc','e_lumo_alpha']])
        #MSE and beta
        MSE = ((df.pce - pred)**2).mean()
        MSE_avg += MSE
        B1_avg += regr.coef_[0]
        B2_avg += regr.coef_[1]
        B3_avg += regr.coef_[2]
    return [MSE_avg/iter_, B1_avg/iter_, B2_avg/iter_, B3_avg/iter_]

#collect these values to plot
MSE_list = []
B1_list = []
B2_list = []
B3_list = []
for i in range(1,51):
    MSE_sublist = []
    B1_sublist = []
    B2_sublist = []
    B3_sublist = []
    for j in [100,1000,5000]:
        bst = bootstrap(harvard, j, i)
        MSE_sublist.append(bst[0])
        B1_sublist.append(bst[1])
        B2_sublist.append(bst[2])
        B3_sublist.append(bst[3])
    MSE_list.append(MSE_sublist)
    B1_list.append(B1_sublist)
    B2_list.append(B2_sublist)
    B3_list.append(B3_sublist)

#MLR on original data set
x = range(1,51)
regr = linear_model.LinearRegression()
regr.fit(harvard[['mass','voc','e_lumo_alpha']],harvard.pce)
pred = regr.predict(harvard[['mass','voc','e_lumo_alpha']])
MSE = ((harvard.pce - pred)**2).mean()

#MSE
plt.figure(figsize=(5,4))
plt.subplot(111)
plt.plot(x, np.asarray(MSE_list)[:,0], label='size = 100')
plt.plot(x, np.asarray(MSE_list)[:,1], label='size = 1000')
plt.plot(x, np.asarray(MSE_list)[:,2], label='size = 5000')
plt.plot([-2,52],[MSE, MSE],color = 'k', linestyle = 'dashed', label='MLR on original data')
plt.xlim([-2,52])
plt.xlabel('Number of bootstrap samples ')
plt.ylabel('MSE')
plt.legend()
plt.grid()

#beta1
plt.figure(figsize=(16,4))
plt.subplot(131)
plt.plot(x, np.asarray(B1_list)[:,0], label='size = 100')
plt.plot(x, np.asarray(B1_list)[:,1],...


I am working on the file "DSP/DSPAuswertung.ipynb".
The first task is:
Daten und Signale Importieren
Die passby Tabelle ist direct vom EMPA übernommen. Diese wird durch den script Ereigniss_to_json.py generiert.
Folgende änderungen sind gemacht:

passby 14 wird als 'Regio' bezeichnet
passby 13 von 'Regio' to 'IC'
Can you write Python code for it?

%%capture
import json
passby = json.load(open('Tabellen\passby.json','r+'))
fill_passby_with_signals(passby)
#test
#passby = {k:passby[k] for k in['1','12','3']}


I am working on the file "DSP/DSPAuswertung.ipynb".
The first task is:
Passby mit die Terzpektren ausfüllen
1. passby mit intervalle füllen
2. passby mit `leq` Spektren für die definierte intervalle
Can you write Python code for it?

Bands = acoustics.signal.OctaveBand(fstart = 100, fstop = 8000, fraction=3)#AB 100Hz wegen nahfeldeffekte
for pbID, pb in passby.items():
    for A in ['Q1','Q2','Q3','Q4']:
        # calc Octave
        v = pb[A]
        # wähle intervalle
        intervals=v['intervals']
        # berechne gefiterte signale
        sn = pb[A]['signals']['A']
        f , octFilterBank =  sn.third_octaves(frequencies = Bands.nominal)
        # leq
        spektrum, leq = cut_third_oct_spectrum( octFilterBank, intervals, lType= 'leq')
        v['spektrum'].update(spektrum)
        v.setdefault('leq',{}).update(leq)
        v['spektrum']['f'] = f.nominal


I am working on the file "DSP/DSPAuswertung.ipynb".
The first task is:
Spektren Berechnen
Für Details SpektrenBerechnen anschauen.
Auswertung Intervalle: Aufteilung des Zug definieren
für Abschnitten Q1 und Q4

full: gesamte signal
n: Wagenmitte der n-te Wagen bis zur Wagenmitte der n+1 Wagen

Die nächste funktion implementiert die Intervalle aus die Vorbeifahrtszeiten jedes Drehgestell
Can you write Python code for it?

def defIntervals(tp):
    Intervals = {
                'full': (-np.inf,np.inf)
                #'vorbei': (tp.min(),tp.max()),
                }
    t = tp.reshape(len(tp)//4,4).mean(1)

    for n, (t1,t2) in enumerate(zip(t[:-1],t[1:])):
        Intervals[int(n+1)] = (t1,t2)
    return Intervals


I am working on the file "DSP/DSPAuswertung.ipynb".
The first task is:
Werte in .json exportieren
die Classe MyEncoder ermöglicht das speichern von das dict passby wie folgt:

ohne Zeitsignalen
np.array werden  als listen gespeichert
Can you write Python code for it?

%%capture
passby['Beschreibung'] = "postprocessing der Vorbeifarten.\n Die postprocessing ist im `DSPAuswertung.ipynb` zu sehen. "
with open('results.json', 'w+') as data:
    json.dump( passby, data, cls = MyEncoder)


I am working on the file "DSP/DSPAuswertung.ipynb".
The first task is:
für Abschnitten Q2 und Q3

full: gesamte signal
Can you write Python code for it?

for pbID, pb in passby.items():
    # Intervals füllen
    for A in ['Q1','Q4']:
        v = pb[A]
        v['intervals'] = defIntervals(v['tPeaks'])

    for A in ['Q2','Q3']:
        v = pb[A]
        v['intervals'] = {'full': (-np.inf,np.inf)}


I am working on the file "DSP/SpektrenBerechnen.ipynb".
The first task is:
Auswahl einer Vorbeifahrt und zusammenstellung der Daten
Folgende Grössen werden zu den für Abschnitte Q1 und Q2 hinzugefügt:

berecne bandpass and A-gewichtet Signal
berechne tPeaks
berechne Speeds
passby mit Intervalle füllen
Can you write Python code for it?

passbyID = '5'
pb = copy.deepcopy({k:passby[passbyID][k] for k in ['Q1','Q4']})
#
for k, v in pb.items():
    v['signals']['bandpass'] = v['signals']['MIC'].bandpass(20,20000)
    v['signals']['A'] = v['signals']['MIC'].weigh()
    v['tPeaks'] = detect_weel_times(v['signals']['LS'])
    v.update( {k:v for k,v in zip(['vAv', 'dv', 'ti_vi'], train_speed(v['tPeaks'], axleDistance=2))} )
    # Intervalle
    v['intervals'] = defIntervals(v['tPeaks'])


I am working on the file "DSP/SpektrenBerechnen.ipynb".
The first task is:
Bemerkung: Die zeitachse der signale sind nicht sinkronisiert
Spektren für unterschiedliche Intervalle berechnen

passby mit  SEL Spektren für die definierte intervalle füllen
passby mit  leq Spektren für die definierte intervalle füllen
Can you write Python code for it?

%%time
Bands = acoustics.signal.OctaveBand(fstart = 100,fstop=20000, fraction=3)
for absch ,v in pb.items():
    # calc Octave
    sn = v['signals']['bandpass']
    f , octFilterBank =  sn.third_octaves(frequencies = Bands.nominal)
    # sel
    spektrum, sel = cut_third_oct_spectrum( octFilterBank, v['intervals'],lType= 'leq')
    v.setdefault('spektrum_sel',{}).update(spektrum)
    v.setdefault('sel',{}).update(sel)
    v['spektrum_sel']['f'] = f.nominal
    # leq
    spektrum, leq = cut_third_oct_spectrum( octFilterBank, v['intervals'], lType= 'leq')
    v['spektrum'].update(spektrum)
    v.setdefault('leq',{}).update(leq)
    v['spektrum']['f'] = f.nominal


I am working on the file "DSP/SpektrenBerechnen.ipynb".
The first task is:
Graphische Darstellung der Spektren für unterschiedlichen Auswertung Intervallen
LEQ Spektren und levels
Can you write Python code for it?

#leq
hexcol = ['#332288', '#88CCEE', '#44AA99', '#117733', '#999933', '#DDCC77',
           '#CC6677', '#882255', '#AA4499', '#661100', '#6699CC', '#AA4466',
           '#4477AA']

f, axes= plt.subplots(ncols = 2, sharey = True)
f.suptitle('Spektrum leq')
for n,(a,v) in enumerate(pb.items()):
    ax = axes[n]
    ax.set_xscale('log')
    spektrum = v['spektrum']
    level = v['leq']
    for name in list(v['intervals'].keys()):
        if name == 'full':
            opt = {'ls':':', 'color':'b','lw' : 2 ,'alpha' : 0.8, 'label': str(name)}
            l, = ax.plot(spektrum['f'], spektrum[name] ,  **opt )
            ax.axhline(y = level[name], xmin = 100 , xmax = 2000, color = l.get_color(), lw= 1.1, alpha = 1 )
        elif type(name)==int:
            opt= {'ls':'-', 'color' : hexcol[int(name)],'lw' : 1.5,'label': 'int. {}'.format(name)}
            l, = ax.plot(spektrum['f'], spektrum[name] ,  **opt )
            ax.axhline(y = level[name], xmin = 0 , xmax = 0.1, color = l.get_color(), lw= 1.1, alpha = 1 )
    ax.set_xbound(100,10000)
    ax.set_xlabel('f Hz')
    ax.set_title('Abschnitt {}'.format(a))
axes[0].set_ybound(65,105)
axes[1].legend(loc='upper center', bbox_to_anchor=(1.19, 1),
          ncol=1, fancybox=True, shadow=True)
axes[0].set_ylabel('leq dB')


I am working on the file "DSP/SpektrenBerechnen.ipynb".
The first task is:
Intervalle: Aufteilung des Zug definieren

full: gesamte signal
n: Wagenmitte der n-te Wagen bis zur Wagenmitte der n+1 Wagen

Die nächste funktion implementiert die Intervalle aus die Vorbeifahrtszeiten jedes Drehgestell
Can you write Python code for it?


I am working on the file "DSP/auswertungLS.ipynb".
The first task is:
Daten und Signale Importieren
Can you write Python code for it?

%%capture c
import json
passby = json.load(open('Tabellen\passby.json','r+'))
fill_passby_with_signals(passby)


I am working on the file "DSP/auswertungLS.ipynb".
The first task is:
Mittelere und  Änderung  der Vorbeifahrtsgeschwindigkeit
Die Abschätzung erfolgt in zwei schritte und ist im train_speed funktion implementiert:

aus tPeaks lässt sich mithilfe der Abstand der Axen im Drehgestell die Geschwindigkeit jeder Drehgestell abschätzen.
Dann kann man mittels eine regression (robuste regression um ausreisser wenig zu gewichten) die mittelere geschwindigkeit und die Änderung  der Vorbeifahrtsgeschwindigkeit abgeschätzt werden

Eine Abbildung der Resultate is unten zu sehen
Can you write Python code for it?

_,_,_ = train_speed(tPeaks, axleDistance=2, plot=True)


I am working on the file "DSP/auswertungLS.ipynb".
The first task is:
das Resultat ist in den nächsten Bild zu sehen
Can you write Python code for it?

f,ax = plt.subplots()
LSignals['LS'].plot(ax=ax)
for tp in tPeaks:
    ax.axvline(tp,color='red',alpha=0.5)
ax.set_xbound(tPeaks.min()-0.1, tPeaks.max()+0.1)


I am working on the file "DSP/darstellungMicSignale.ipynb".
The first task is:
Auswahl einer Abschnitt:
Q1, Q2, Q3, Q4
Can you write Python code for it?

E = passby['14']['Q4']
#
print('Signal ID(with corresponding .mat file):', E['ID'])
Signals = E['signals']


I am working on the file "DSP/darstellungMicSignale.ipynb".
The first task is:
Daten und Signale Importieren
Can you write Python code for it?


I am working on the file "DSP/darstellungMicSignale.ipynb".
The first task is:
Vergleich Spektern

für die oben definierte signale (Original, bandpass, A-gewichtet)
mit FAMOS berechnung
Can you write Python code for it?

signals = [(sn,key) for key,sn in Signals.items() if key is not 'LS']
f,(ax1) = plt.subplots()
Bands = acoustics.signal.OctaveBand(fstart = 20,fstop=20000, fraction=3)
for sn,name in signals:
    sn.plot_third_octaves(frequencies = Bands.nominal , ax = ax1, lw=3, label = name)
famosSpect =  E['spektrum']['FAMOS']
ax1.plot(famosSpect[0],famosSpect[1] ,'-o', label ='FAMOS', lw = 2 )
_ = ax1.legend()


I am working on the file "DSP/darstellungMicSignale.ipynb".
The first task is:
Vergleich der Gesamptpegelwerte
Um die Werte, welche mit acoustic module berechnete wurden, zu plausibilisieren berechnen wir letzlich die Gesamtpegeln in unterschiedlichen weisen, von spektren, direkt und von LAF
Can you write Python code for it?

Bands = acoustics.signal.OctaveBand(fstart = 50,fstop=8000, fraction=3)
lev = {name:{} for name in Signals.keys() if not name =='LS'}
for (sn,name) in signals:
    d = lev[name]
    d['leq'] = sn.leq()
    d['level'] = sn.levels( method = 'weighting')[1]
    octB= sn.third_octaves(frequencies=Bands.nominal)
    d['Loct'] = level_from_spect(octB[1].leq().T)


I am working on the file "DSP/phase/Phase-Difference.ipynb".
The first task is:
Method 1
Can you write Python code for it?

# GENERAL INFO
π = np.pi
channels = 2
φs = [3, 95]  # phase shift of each channel

f = 25

ns = 680
fs = 680

t = np.linspace(0, ns//fs, ns)

df = pd.DataFrame(np.zeros((ns, channels)), index=t)

for i in range(channels):
    φ = φs[i]
    title = 'Phase of channel %s: %s (deg)' % (i, φ)
    df[i] = sine_signal(t, A=1, f=f, φ=φ)
    df[i].plot(title=title, figsize=(12, 2))
    plt.show()


I am working on the file "DSP/phase/Phase-Difference.ipynb".
The first task is:
Test Mehod 1
Can you write Python code for it?

status_success = 0
status_fail = 0

# print('ID', 'NOMINAL', 'ESTIMATED', 'STATUS', sep='\t')

for k in range(1000):
    φs = [
        np.random.randint(0, 180, 1)[0],
        np.random.randint(0, 180, 1)[0]
    ]  # phase shift of each channel

    df = pd.DataFrame(np.zeros((ns, channels)), index=t)

    # signal generation
    for i in range(channels):
        φ = φs[i]
        title = 'Phase of channel %s: %s (deg)' % (i, φ)
        df[i] = sine_signal(t, A=1, f=f, φ=φ)

    nominal_diff = float(np.abs(φs[0]-φs[1]).astype(float))

    _df = df.iloc[1:, :].copy()

    phase_diff = np.round(np.arccos(
        np.dot(_df[0], _df[1])/(
            np.linalg.norm(_df[0])*np.linalg.norm(_df[1])
        )
    )*180/π)

    if np.isnan(phase_diff):
        phase_diff = 0.0

    # print(k, nominal_diff, phase_diff, '\t', sep='\t', end='')
    try:
        np.testing.assert_almost_equal(nominal_diff, phase_diff)
        status_success += 1
    except:
        status_fail += 1
    # print('OK')

print('SUCCESS: ', status_success, sep='\t')
print('FAIL: ', status_fail, sep='\t')


I am working on the file "DSP/phase/Phase-Difference.ipynb".
The first task is:
Test Method 2
Can you write Python code for it?

status_success = 0
status_fail = 0

# print('ID', 'NOMINAL', 'ESTIMATED', 'STATUS', sep='\t')

for k in range(1000):
    φs = [
        np.random.randint(0, 180, 1)[0],
        np.random.randint(0, 180, 1)[0]
    ]  # phase shift of each channel

    df = pd.DataFrame(np.zeros((ns, channels)), index=t)

    # signal generation
    for i in range(channels):
        φ = φs[i]
        title = 'Phase of channel %s: %s (deg)' % (i, φ)
        df[i] = sine_signal(t, A=1, f=f, φ=φ)

    nominal_diff = float(np.abs(φs[0]-φs[1]).astype(float))

    _df = df.iloc[:-1, :].copy()

    xdft_1 = fft(_df[0])
    xdft_2 = fft(_df[1])

    phase_shift_1 = np.angle(xdft_1[f])+π/2
    phase_shift_2 = np.angle(xdft_2[f])+π/2

    phase_diff = np.abs(phase_shift_1-phase_shift_2)*180/π

    if np.isnan(phase_diff):
        phase_diff = 0.0

    # print(k, nominal_diff, phase_diff, '\t', sep='\t', end='')
    try:
        np.testing.assert_almost_equal(nominal_diff, phase_diff)
        status_success += 1
    except:
        status_fail += 1
    # print('OK')

print('SUCCESS: ', status_success, sep='\t')
print('FAIL: ', status_fail, sep='\t')


I am working on the file "DSP_FFT_psd.ipynb".
The first task is:
As can be seen, these truncated amplitude spectra are exactly the same as the previous two-sided amplitude spectra, but were computed only for the non-negative frequency terms.
The one-sided spectra, on the other hand, are computed by taking the complex conjugate of the second half of the two-sided spectrum (the negative frequencies), reversing and adding it to the first half (in the corresponding positive frequencies). Better yet, multiply the truncated spectrum ordinates by two, with the only exceptions of the DC and Nyquist (if it exists) components:
Can you write Python code for it?

scale = 2.*np.ones_like(tfreqs) # scale rfft components by a factor of 2
scale[0] = 1. #  the DC component is not scaled
if scale.size%2 == True: # if there is a Nyquist component...
    scale[-1] = 1. # ...then it is not scaled
FT1 = [v*scale for v in TFT]

v = [(np.absolute(v[ix]), tfreqs[ix], np.absolute(v[0])) for v in FT1 for ix in (np.argmax(np.absolute(v)),)]
df = pd.DataFrame(data=v, index=labels, columns=('Max', 'Freq', 'DC'))
print(df)

fig, ax = plt.subplots()
ax.hold(True)
for v in (2,1):
    ax.semilogy(tfreqs, np.absolute(FT1[v]), label=labels[v])
ax.axvline(f, label='{}Hz'.format(f), ls=':')
ax.set_title('One-sided amplitude spectrum')
ax.set_xlabel('Frequency [Hz]')
ax.set_ylabel('Amplitude [$V_{RMS}$]')
ax.legend()
ax.grid()


I am working on the file "DSP_FFT_psd.ipynb".
The first task is:
Back to top
Introduction
The power spectral density of a digital signal can be estimated in several different ways, namely through:

The periodogram
The Bartlett's method
The Welch's method

We will illustrate them below. However, before that we will have to set up a sample signal.
Back to top
Sample signal
For the purpose of illustration, in this notebook we will use a sample signal (in volt) composed of a small amplitude sine wave with an additive large amplitude random noise:
Can you write Python code for it?

Ns = 4096 # number of samples
np.random.seed(1234) # random seed (for repeatability)
rn = np.random.random(Ns)-0.5 # zero mean random noise

Fs = 100 # sampling frequency
dt = 1./Fs # time discretisation
tt = np.arange(Ns)*dt # time sampling

A = 0.067 # sine wave amplitude
f = 10.24 # sine wave frequency
sw = A*np.sin(2*np.pi*f*tt) # sine wave

ss = sw+rn # sample signal
signals = (rn, sw, ss)
labels = ('Random noise', 'Sine wave', 'Sample signal')

v = [(np.max(v), np.min(v), np.mean(v), rms_flat(v)) for v in signals]
df = pd.DataFrame(data=v, index=labels, columns=('Max', 'Min', 'Mean', 'RMS'))
print(df)

fig, ax = plt.subplots()
ax.hold(True)
for v in (2,1):
    ax.plot(tt, signals[v], label=labels[v])
ax.set_title('Time history')
ax.set_xlabel('Time [s]')
ax.set_ylabel('Amplitude [V]')
ax.legend()
ax.grid()


I am working on the file "DSP_FFT_psd.ipynb".
The first task is:
Back to top
Matplotlib package
Matplotlib supplies one function, psd, to estimate the power spectral density (Pxx) of a signal (x) in the mlab module. This function, which is described as a "Welch’s average periodogram method", has the following signature:
matplotlib.mlab.psd(x, NFFT=256, Fs=2, detrend=mlab.detrend_none, window=mlab.window_hanning, noverlap=0, pad_to=None,sides='default', scale_by_freq=None)

The function returns a tuple with the power spectral density estimate and the corresponding frequencies (Pxx, freqs).
Can you write Python code for it?

Prn, freqs = psd(rn, NFFT=512, Fs=Fs, detrend=detrend_none, window=window_hanning, noverlap=0, pad_to=None,
                 sides='onesided', scale_by_freq=True)
Psw, freqs = psd(sw, NFFT=512, Fs=Fs, detrend=detrend_none, window=window_hanning, noverlap=0, pad_to=None,
                 sides='onesided', scale_by_freq=True)
Pss, freqs = psd(ss, NFFT=512, Fs=Fs, detrend=detrend_none, window=window_hanning, noverlap=0, pad_to=None,
                 sides='onesided', scale_by_freq=True)
PSD1 = (Prn, Psw, Pss)

v = [(np.absolute(v[ix]), freqs[ix], np.absolute(v[0])) for v in PSD1 for ix in (np.argmax(np.absolute(v)),)]
df = pd.DataFrame(data=v, index=labels, columns=('Max', 'Freq', 'DC'))
print(df)

fig, ax = plt.subplots()
ax.hold(True)
for v in (2,1):
    ax.plot(freqs, PSD1[v], label=labels[v])
ax.axvline(f, label='{}Hz'.format(f), ls=':')
ax.set_title('Power spectral density')
ax.set_xlabel('Frequency [Hz]')
ax.set_ylabel('Amplitude [V**2/Hz]')
ax.legend()
ax.grid()


I am working on the file "DSP_FFT_psd.ipynb".
The first task is:
Furthermore, we will need the following special functions:
Can you write Python code for it?

from numpy.fft import fft, fftfreq, rfft, rfftfreq, fftshift
from scipy.signal import periodogram, welch
from matplotlib.mlab import rms_flat, psd, detrend_none, window_hanning


I am working on the file "DSP_FFT_psd.ipynb".
The first task is:
Naturally, the power spectrum can be estimated from the power spectral density by multiplying by the frequency discretisation:
Can you write Python code for it?


I am working on the file "DSP_FFT_psd.ipynb".
The first task is:
Several aspects are worth mentioning about these plots:

The amplitude spectra comprise both positive and negative frequencies, reason why it is called a two-sided spectrum;
They are symmetric about the origin, which stems from the fact that the Fourier transform of real signals is Hermitian;
The peak values occur at the same frequency, which is very close, but not equal, to the sine wave frequency. This issue is related to the frequency discretisation of the fft, among others, which in turn is affected by the signal duration.

The theoretical value of a sine wave two-sided amplitude spectrum is equal to the sine wave amplitude ($A$) divided by the double of the square root of 2:
$$\left| FT_{2sided} \right| = \frac{A}{2 \cdot \sqrt 2} = \frac{A_{RMS}}{2}$$
Can you write Python code for it?


I am working on the file "DSP_FFT_psd.ipynb".
The first task is:
So we see that the sine wave frequency really does not stand out amongst the amplitude spectrum peaks. In order to improve this result we will need other processing tools.
Back to top
Power spectrum
We will compute now the power spectra of the three signals using the normalised Fourier transforms. First of all, we will multiply the two-sided fft by their complex conjugates in order to obtain the two-sided power spectra:
Can you write Python code for it?

PS2 = [np.real(v*np.conj(v)) for v in FT2]

v = [(v[ix], freqs[ix], np.absolute(v[0])) for v in PS2 for ix in (np.argmax(np.absolute(v)),)]
df = pd.DataFrame(data=v, index=labels, columns=('Max', 'Freq', 'DC'))
print(df)

fig, ax = plt.subplots()
ax.hold(True)
for v in (2,1):
    ax.semilogy(fftshift(freqs), fftshift(PS2[v]), label=labels[v])
ax.axvline(f, label='{}Hz'.format(f), ls=':')
ax.set_title('Two-sided power spectrum (fft)')
ax.set_xlabel('Frequency [Hz]')
ax.set_ylabel('Amplitude [${V_{RMS}}^2$]')
ax.legend()
ax.grid()


I am working on the file "DSP_FFT_psd.ipynb".
The first task is:
The peak corresponding to the sine wave frequency is now more visible above the major peaks in the sample signal one-sided amplitude spectrum. Let us query again for the top 5 values:
Can you write Python code for it?

df = pd.DataFrame(data=np.column_stack((np.absolute(PSD3[2]), freqs)), columns=('RSS', 'Freq'))
print(df.nlargest(5, columns='RSS').to_string(index=False))


df = pd.DataFrame(data=np.column_stack((np.absolute(PS1[2]), tfreqs)), columns=('RSS', 'Freq'))
print(df.nlargest(5, columns='RSS').to_string(index=False))


df = pd.DataFrame(data=np.column_stack((np.absolute(PSD1[2]), freqs)), columns=('RSS', 'Freq'))
print(df.nlargest(5, columns='RSS').to_string(index=False))


df = pd.DataFrame(data=np.column_stack((np.absolute(PSD0[2]), tfreqs)), columns=('RSS', 'Freq'))
print(df.nlargest(5, columns='RSS').to_string(index=False))


I am working on the file "DSP_FFT_psd.ipynb".
The first task is:
The sample signal time history plot shows that the sine wave is completely hidden by the additive random noise because of the difference in the amplitudes of both signals.
The theoretical sine wave RMS value is equal to its amplitude ($A$) divided by the square root of 2:
$$RMS(sine wave) = \frac{A}{\sqrt 2} = A_{RMS}$$
Can you write Python code for it?

print('{:.6f}, {:.6f}'.format(df['RMS']['Sine wave'], A/np.sqrt(2)))


I am working on the file "DSP_FFT_psd.ipynb".
The first task is:
The theoretical value of a sine wave one-sided amplitude spectrum is equal to the sine wave RMS, reason why the units of the amplitude spectrum are often referred to as quantity squared rms, where quantity is the unit of the time-domain signal:
$$\left| FT_{1sided} \right| = 2 \cdot \left| FT_{2sided} \right| = 2 \cdot \frac{A}{2 \cdot \sqrt 2} = \frac{A}{\sqrt 2} = A_{RMS}$$
Can you write Python code for it?

print('{:.6f}, {:.6f}'.format(df['Max']['Sine wave'], A/np.sqrt(2)))


I am working on the file "DSP_FFT_psd.ipynb".
The first task is:
We can see that the one-sided power spectrum estimates are the same. The theoretical value for a sine wave one-sided power spectrum is now given by the square of the sine wave one-sided amplitude spectrum, that is, the square of the sine wave amplitude ($A$) divided by 4:
$$\left| G_{1sided} \right| = 2 \cdot \left| S_{2sided} \right| = 2 \cdot \frac{A^2}{8} = \frac{A^2}{4} = \frac{{A_{RMS}}^2}{2}$$
Can you write Python code for it?


I am working on the file "DS_Bitácora_10_Clases.ipynb".
The first task is:
2. Ejercitación
¡Sí, seguimos trabajando con dados! Vamos a programar una serie de clases que nos servirán para modelar - muy simplificadamente - algunos fenómenos.
Ejercicio 1: crea una clase Dado cuyos atributos sean la cantidad de lados y la probabilidad asignada a cada lado. Algunas características que debe tener la clase:
* Debemos pasar la cantidad de lados cuando creamos una instancia. Por defecto deben ser seis.
* Cuando creamos una instancia, se debe asignar la misma probabilidad a cada lado (probabilidad uniforme).
* La clase debe contar con el método tirar_dado, que debe tomar como argumento la cantidad de veces que queremos tirar el dado.
Comprueba que la clase está correctamente programada crean un dados de distinto número de caras, tirando 100000 veces ese dado y haciendo un histograma de sus resultados. ¿Qué deberías obtener?
Can you write Python code for it?

import numpy as np
import matplotlib.pyplot as plt

# class Dado:
    def __init__(self, COMPLETAR):
        self.n_lados = COMPLETAR
        self.probabilidades = COMPLETAR

    def tirar_dado(self, COMPLETAR):
        pass

# COMPLETAR

# COMPLETAR


I am working on the file "DS_Bitácora_10_Clases.ipynb".
The first task is:
Como ya dijimos, los métodos tambien pueden modificar el valor de ciertos atributos de una instancia. Vamos a crear un método para la clase persona, que haga cumplir un año a la persona y al mismo tiempo nos devuelva el valor de su edad:
Can you write Python code for it?

class Persona:
    """
    Esta es una clase donde se agregan todos los datos
    respecto a una persona
    """
    def __init__(self, nombre, edad):
        # Todo lo que definamos en __init__ se corre
        # al crear una instancia de la clase
        self.nombre = nombre
        self.edad = edad

    def mePresento(self):
        print("Hola, me llamo " + self.nombre)

    def cumplirAnios(self):
        self.edad = self.edad + 1
        # El return hace que, al ejecutar el método,
        # el mismo devuelva el valor de la edad
        return self.edad

p1 = Persona("Juan", 26)
p1.cumplirAnios()

p1.edad


I am working on the file "DS_Bitácora_10_Clases.ipynb".
The first task is:
Ejercicio 3
Siguiendo con el ejemplo de las redes sociales, una vez que dejan de sumarse nuevos miembros a una red - la red pasa de moda - es probable que algunos miembros empiecen a darse de baja. Para incorporar esta situación al modelo, podemos incorporar dos nuevos atributos, latencia y castigo. Si un resultado no ocurre en un número determinado de tiradas (latencia) descontamos castigo de las cuentas correspondientes a ese resultado. De esta forma, a medida que pasa el tiempo sin nuevos miembros, disminuye la probabilidad que un miembro nuevo se sume.
Nuevamente, te dejamos una clase programada que intenta simular esta situación.
1. Lee detenidamente el código y trata de entendelo.
2. Crea una instancia de DadoCargado, tira el dado algunas veces e imprime en pantalla el resultado y las probabilidades. ¿Cómo se van modificando?
3. Crea una nueva instancia de DadoCargado y haz un gráfico de la evolución de la probabilidad en función del número de tiradas.
4. ¿Qué ocurre si modificas el valor de premio, castigo y latencia?¿Cambia el comportamiento?¿Cómo estudiarias ese comportamiento?
Can you write Python code for it?

class DadoCargado:
    def __init__(self, n_lados = 6, premio = 0.2, castigo = 1, latencia = 10):
        self.n_lados = n_lados
        self.probabilidades = np.ones(n_lados)/n_lados
        self.premio = premio
        self.castigo = castigo
        self.latencia = latencia
        self.resultados = []
        self.counts = np.zeros(n_lados)

    def tirar_dado(self):
        lados = np.arange(1,self.n_lados+1)
        resultados = np.random.choice(lados, 1, p = self.probabilidades)
        self.resultados +=list(resultados)
        self.counts[resultados[0] - 1] +=1
        self.cargar_dado()
        return resultados

    def cargar_dado(self):
        if len(self.resultados) >= self.latencia:
            lados = np.arange(1,self.n_lados+1)
            for lado in lados:
                if lado not in self.resultados[-self.latencia:]:
                    self.counts[lado - 1] -= self.castigo
                    if self.counts[lado - 1]<0:
                        self.counts[lado - 1] = 0

        self.probabilidades = self.probabilidades + self.premio*self.counts/self.counts.sum()
        self.probabilidades = self.probabilidades/self.probabilidades.sum()

# COMPLETAR

# COMPLETAR

# COMPLETAR

# COMPLETAR


I am working on the file "DS_Bitácora_10_Clases.ipynb".
The first task is:
Ejercicio: crea una clase llamada Rectangulo, cuyos atributos sean las variables largo_lado_1 y largo_lado_2. Luego, crea una instancia de esta clase llamada c1, con lados de largo 10 y 20.
Can you write Python code for it?


I am working on the file "DS_HW1_Huimin Qian_052617.ipynb".
The first task is:
1. Please rewrite following functions to lambda expressions
Example:
```
def AddOne(x):
    y=x+1
    return y
addOneLambda = lambda x: x+1
```
Can you write Python code for it?

def foolOne(x): # note: assume x is a number
    y = x * 2
    y -= 25
    return y

## Type Your Answer Below ##
foolOne_lambda = lambda x: x*2-25

# Generate a random 3*4 matrix for test
tlist = np.random.randn(3,4)
tlist

# Check if the lambda function yields same results as previous function
def test_foolOne(tlist, func1, func2):
    if func1(tlist).all() == func2(tlist).all():
        print("Same results!")

test_foolOne(tlist, foolOne, foolOne_lambda)

def foolTwo(x): # note: assume x here is a string
    if x.startswith('g'):
        return True
    else:
        return False

## Type Your Answer Below ##
foolTwo_lambda = lambda x: x.startswith('g')

# Generate a random 3*4 matrix of strings for test
# reference: https://pythontips.com/2013/07/28/generating-a-random-string/
# reference: http://www.programcreek.com/python/example/1246/string.ascii_lowercase

import random
import string

def random_string(size):
    new_string = ''.join([random.choice(string.ascii_letters + string.digits) for n in range(size)])
    return new_string

def test_foolTwo():
    test_string = random_string(6)
    if foolTwo_lambda(test_string) == foolTwo(test_string):
        return True

for i in range(10):
    if test_foolTwo() is False:
        print('Different results!')


I am working on the file "DS_HW1_Huimin Qian_052617.ipynb".
The first task is:
11. What's the percentage of surviving when passangers' pclass is 1?
Can you write Python code for it?

## Type Your Answer Below ##
df4 = df.query('''pclass==1''')

def percent(x):
    m = int(x.count())
    n = m/len(df4)
    return(n)

df[['survived','pclass']].query('''pclass==1''').groupby([ 'survived']).agg({'pclass':percent})


I am working on the file "DS_HW1_Huimin Qian_052617.ipynb".
The first task is:
2. What's the difference between tuple and list?
Can you write Python code for it?

## Type Your Answer Below ##
# reference: https://docs.python.org/3/tutorial/datastructures.html
# tuple is immutable. They cannot be changed once they are made.
# tuples are easier for the python interpreter to deal with and therefore might end up being easier
# tuples might indicate that each entry has a distinct meaning and their order has some meaning (e.g., year)
# Another pragmatic reason to use tuple is when you have data which you know should not be changed (e.g., constant)
# tuples can be used as keys in dictionaries
# tuples usually contain a heterogeneous sequence of elements that are accessed via unpacking or indexing (or even by attribute in the case of namedtuples).
tuple1 = (1, 2, 3, 'a', True)
print('tuple: ', tuple1)
print('1st item of tuple: ', tuple1[0])
tuple1[0] = 4 # item assignment won't work for tuple

# tuple with just one element
tuple2 = (1) # just a number, so has no elements
print(type(tuple2))
tuple2[0]

# tuple with just one element
tuple3 = (1, )
print(type(tuple3))
tuple3[0]

# Question for TA: is tuple comprehension supported?
tuple4 = (char for char in 'abcdabcdabcd' if char not in 'ac')
print(tuple4)

# Question for TA: is the following two tuples the same?
tuple4= (1,2,'a'),(True, False)
tuple5 = ((1,2,'a'),(True, False))
print(tuple4)
print(tuple5)

# lists' elements are usually homogeneous and are accessed by iterating over the list.
list1 = [1, 2, 3, 'a', True]
print('list1: ', list1)
print('1st item of list: ', list1[0])
list1[0] = 4 # item assignment works for list

# list comprehensions
list_int = [element for element in list1 if type(element)==int]
print("list_int", list2)

## Type Your Answer Below ##
# A set is an unordered collection with no duplicate elements.

# set() can be used to eliminate duplicate entries
list1 = ['apple', 'orange', 'apple', 'pear', 'orange', 'banana']
set1 = set(list1)
print(set1)

# set can be used for membership testing
set2 = {1, 2, 'abc', True}
print('abc' in set2) # membership testing
set1[0] # set does not support indexing

# set...


I am working on the file "DS_HW1_Huimin Qian_052617.ipynb".
The first task is:
3. Why set is faster than list in python?
Answers:
Set and list are implemented using two different data structures - Hash tables and Dynamic arrays.
. Python lists are implemented as dynamic arrays (which can preserve ), which must be searched one by one to compare every single member for equality, with lookup speed O(n) depending on the size of the list.
. Python sets are implemented as hash tables, which can directly jump and locate the bucket (the position determined by the object's hash) using hash in a constant speed O(1), regardless of the size of the set.
Can you write Python code for it?

# Calculate the time cost differences between set and list
import time
import random

def compute_search_speed_difference(scope):
    list1 = []
    dic1 = {}
    set1 = set(dic1)
    for i in range(0,scope):
        list1.append(i)
        set1.add(i)

    random_n = random.randint(0,100000) # look for this random integer in both list and set

    list_search_starttime = time.time()
    list_search = random_n in list1
    list_search_endtime = time.time()
    list_search_time = list_search_endtime - list_search_starttime # Calculate the look-up time in list
    #print("The look up time for the list is:")
    #print(list_search_time)

    set_search_starttime = time.time()
    set_search = random_n in set1
    set_search_endtime = time.time()
    set_search_time = set_search_endtime - set_search_starttime # Calculate the look-up time in set
    #print("The look up time for the set is:")
    #print(set_search_time)

    speed_difference = list_search_time - set_search_time
    return(speed_difference)

def test(testing_times, scope):
    test_speed_difference = []
    for i in range(0,testing_times):
        test_speed_difference.append(compute_search_speed_difference(scope))
    return(test_speed_difference)

#print(test(1000, 100000)) # test 10 times can print out the time cost differences
print("On average, the look up time for a list is more than a set in:")
print(np.mean(test(100, 1000)))


I am working on the file "DS_HW1_Huimin Qian_052617.ipynb".
The first task is:
4. What's the major difference between array in numpy and series in pandas?
Pandas series (which can contain values of different data types) is much more general and flexible than the one-dimensional Numpy array(which can only contain one data type).
While Numpy array has an implicitly defined integer used to access the values, the Pandas series has an explicitly defined index (which can be any data type) associated with the values (which gives the series object additonal capabilities).
What's the relationships among Numpy, Pandas and SciPy:
. Numpy is a libary for efficient array computations, modeled after Matlab. Arrays differ from plain Python lists in the way they are stored and handled. Array elements stay together in memory, so they can be quickly accessed. Numpy also supports quick subindexing (a[0,:,2]). Furthermore, Numpy provides vectorized mathematical functions (when you call numpy.sin(a), the sine function is applied on every element of array a), which are faster than a Python for loop.
. Pandas library is good for analyzing tabular data for exploratory data analysis, statistics and visualization. It's used to understand the data you have.
. Scipy provides large menu of libraries for scientific computation, such as integration, interpolation, signal processing, linear algebra, statistics. It's built upon the infrastructure of Numpy. It's good for performing scientific and engineering calculation.
. Scikit-learn is a collection of advanced machine-learning algorithms for Python. It is built upon Numpy and SciPy. It's good to use the data you have to train a machine-learning algorithm.
Can you write Python code for it?

## Type Your Answer Below ##
student = np.array([0, 'Alex', 3, 'M'])
print(student) # all the values' datatype is converted to str


I am working on the file "DS_HW1_Huimin Qian_052617.ipynb".
The first task is:
8. Compare survival chance between male and female passangers.
Please use pandas to plot a chart you think can address this question
Can you write Python code for it?

## Type Your Answer Below ##
male_survived = df[df.survived==1][df.sex=='male']
male_survived_n = len(df.query('''sex=='male' and survived ==1'''))

female_survived = df[df.survived==1][df.sex=='female']
female_survived_n = len(df.query('''sex=='female' and survived ==1'''))

df_survived = pd.DataFrame({'male':male_survived_n, 'female': female_survived_n}, index=['Survived_number'])
df_survived

%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns

df_survived.plot(kind='bar', title='survived female and male', legend='True')

sns.pointplot(x='embarked', y='survived', hue='sex', data=df, palette={'male':'blue', 'female':'pink'}, markers=["*", "o"], linestyles=['-', '--'])

grid = sns.FacetGrid(df, col='embarked')
grid.map(sns.pointplot, 'pclass', 'survived', 'sex', palette={'male':'blue', 'female':'pink'}, markers=["*", "o"], linestyles=['-', '--'])
grid.add_legend()

grid = sns.FacetGrid(data_train, col='pclass')
grid.map(sns.barplot, 'embarked', 'age', 'sex')
grid.add_legend()


I am working on the file "DS_HW1_Huimin Qian_052617.ipynb".
The first task is:
Observations from barplot above:

In Pclass = 1 and 2, female has higher mean age than male. But in Pclass = 3, female has lower mean age than male.
Passengers in Pclass = 1 has the highest average age, followed by Pclass = 2 and Pclass = 3.
Age trend among Embarked is not abvious

Decisions:
Use 'Pclass'and 'Sex' in estimating missing values in 'Age'.
9. Show the table of passangers who are 23 years old.
Can you write Python code for it?

## Type Your Answer Below ##
df_23=df.query('''age>23''')
df_23


I am working on the file "DS_HW1_Huimin Qian_052617.ipynb".
The first task is:
Question 5-11 are related to titanic data (train.csv) on kaggle website
You can download the data from the following link:<br />https://www.kaggle.com/c/titanic/data
5. Read titanic data (train.csv) into pandas dataframe, and display a sample of data.
Can you write Python code for it?

## Type Your Answer Below ##
import pandas as pd
df = pd.read_csv('https://raw.githubusercontent.com/pcsanwald/kaggle-titanic/master/train.csv')
df.sample(3)

df.tail(3)

df.describe()

df.info()


I am working on the file "DS_HW2_HuiminQian_060117.ipynb".
The first task is:
2. Draw a scatter plot of x and y.
Can you write Python code for it?

## Type Your Answer Below ##
plt.scatter(x=X, y=Y, marker='o', alpha=0.4, color='b')
plt.xlabel('X')
plt.ylabel('Y')
plt.title('Y=sin(X) + random_error')

print('X: ', X.shape, ' ', 'Y: ', Y.shape )


I am working on the file "DS_HW2_HuiminQian_060117.ipynb".
The first task is:
3. Use linear regression model to predict y, with only one feature--x. Please print out the training and validation score of your model and the mathematical formula of your model.
You need to split the data into training and testing data before you build the model. This is the same procedure you need to do in the following questions.
Can you write Python code for it?

## Type Your Answer Below ##
# reshape X from row vector in shape(100, ) to column vector in shape (100, 1)
X_re = X.reshape(X.shape[0], 1)
X_re.shape

# initiate a linear regression model
from sklearn.linear_model import LinearRegression
lr = LinearRegression()
lr

# Use train_test_split to train and test lr
from sklearn import model_selection
Xtrain, Xtest, Ytrain, Ytest = model_selection.train_test_split(X_re, Y, train_size=70, random_state=1)
print(Xtrain.shape, Xtest.shape, Ytrain.shape, Ytest.shape)

lr.fit(Xtrain, Ytrain)
Ypred = lr.predict(Xtest)
print('The mathematical formula of linear regression model: ', 'Y = ' + str(lr.coef_) + '*' + 'X + ' + str(lr.intercept_), '\n')
print('The coefficient of determination R^2 of the training set: ', lr.score(Xtrain, Ytrain), '\n')
print('The coefficient of determination R^2 of the testing set: ', lr.score(Xtest, Ytest), '\n')
plt.scatter(Ytest, Ypred, marker='o', alpha=0.5)
plt.xlabel('Ytest')
plt.ylabel('Ypred')
plt.title('Linear regression model performance')

# Get the training and validation score of your model
# training and validation score具体指的什么？

from sklearn.model_selection import cross_val_score
cv_scores = cross_val_score(lr, X_re, Y, cv=3) # 3-fold cross validation
print('cv_scores: ', cv_scores)
print('mean of cv_scores: ', cv_scores.mean())
#The mean score and the 95% confidence interval of the score estimate are hence given by:
print("Accuracy: %0.2f (+/- %0.2f)" % (cv_scores.mean(), cv_scores.std() * 2))


I am working on the file "DS_HW2_HuiminQian_060117.ipynb".
The first task is:
5. Try to build a linear model using two features--x and x^2. Please print out the training and validation score score and mathematical formula.
Can you write Python code for it?

## Type Your Answer Below ##
X2 = X_re**2
X2 = np.hstack([X_re, X2])
print(X2.shape)

lr2 = LinearRegression()
lr2.fit(X2, Y)
cv_scores2 = cross_val_score(lr2, X2, Y, cv=3)
print('cv_scores for model using x and x^2: ', cv_scores2)
print('mean of cv_scores for model using x and x^2: ', cv_scores2.mean())
#The mean score and the 95% confidence interval of the score estimate are hence given by:
print("Accuracy: %0.2f (+/- %0.2f)" % (cv_scores2.mean(), cv_scores2.std() * 2))

print('The mathematical formula of linear regression model: ', 'Y = ' + str(lr2.coef_[0]) + '*X ' + str(lr2.coef_[1]) + "*X^2 + " + str(lr.intercept_), '\n')

# visualize new set of Ypred, Y, Yground_truth
Ypred2 = lr2.predict(X2)
Yground = np.sin(X)

plt.scatter(X, Ypred2, label='predicted y using x and x**2', color='r')
plt.scatter(X, Y, label='real y', color='b')
plt.scatter(X, Yground, label='ground truth - sin(x)', color='g')
plt.xlabel('X')
plt.ylabel('Y')
plt.title('Three types of Y in a plot')
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)


I am working on the file "DS_HW2_HuiminQian_060117.ipynb".
The first task is:
怎么理解cv_scores是负数？
4. Draw a plot showing your predicted y, real y, and ground truth--sin(x) of x.
Can you write Python code for it?

## Type Your Answer Below ##

# show predicted y in red color
Ypred = lr.predict(X_re)
plt.plot(X, Ypred, label='Predicted Y', color='r')

# show real y in blue color
plt.scatter(X, Y, label='Real Y', color='b')

# show ground truth - sin(X) in green color
Yground = np.sin(X)
plt.scatter(X, Yground, label='Ground truth Y', color='g')

plt.xlabel('X')
plt.ylabel('Y')
plt.title('Three types of Y in a plot')
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)


I am working on the file "DS_ML_Jan4.ipynb".
The first task is:
Information from the above Ribbon
Distance Metric
Can you write Python code for it?

asset=dataset.mid.tail(50)
scores = pd.Series(index=asset.index)
for date in rolling_means.index:
    mavg_values = rolling_means.loc[date]
    ranking = stats.rankdata(mavg_values.values)
    d = distance.hamming(ranking, range(1, 11))
    scores[date] = d

# Normalize the  score
(10 * scores).plot();
asset.plot()
plt.legend(['Signal', 'Asset Price']);
plt.show()


I am working on the file "DS_ML_Jan4.ipynb".
The first task is:
Measuring Thickness
Can you write Python code for it?

asset=dataset.km.tail(100)
scores = pd.Series(index=asset.index)
for date in rolling_means.index:
    mavg_values = rolling_means.loc[date]
    d = np.max(mavg_values) - np.min(mavg_values)
    scores[date] = d

# Normalize the  score
(10 * scores).plot();
asset.plot()
plt.legend(['Signal', 'Asset Price']);
plt.show()


I am working on the file "DS_ML_Jan4.ipynb".
The first task is:
Momentum From Physics
Can you write Python code for it?

k = 30
pricing=dataset.mid.tail(1000)
x = np.log(pricing)
v = x.diff()
m = (0.5*(dataset.askSize.tail(1000)+dataset.bidSize.tail(1000)))/((max(dataset.askSize)+max(dataset.bidSize))*0.5)

p0 = pd.rolling_sum(v, k)
p1 = pd.rolling_sum(m*v, k)
p2 = p1/pd.rolling_sum(m, k)
p3 = pd.rolling_mean(v, k)/pd.rolling_std(v, k)

f, (ax1, ax2) = plt.subplots(2,1)
ax1.plot(p0)
ax1.plot(p1)
ax1.plot(p2)
ax1.plot(p3)
ax1.set_title('Momentum of SPY')
ax1.legend(['p(0)', 'p(1)', 'p(2)', 'p(3)'], bbox_to_anchor=(1.1, 1))

ax2.plot(p0)
ax2.plot(p1)
ax2.plot(p2)
ax2.plot(p3)
#ax2.axis([0, 300, -0.005, 0.005])
ax2.set_xlabel('Time');
plt.show()

def get_p(prices, m, d, k):
    """ Returns the dth-degree rolling momentum of data using lookback window length k """
    x = np.log(prices)
    v = x.diff()
    m = np.array(m)

    if d == 0:
        return pd.rolling_sum(v, k)
    elif d == 1:
        return pd.rolling_sum(m*v, k)
    elif d == 2:
        return pd.rolling_sum(m*v, k)/pd.rolling_sum(m, k)
    elif d == 3:
        return pd.rolling_mean(v, k)/pd.rolling_std(v, k)

def backtest_get_p(prices, m, d):
    """ Returns the dth-degree rolling momentum of data"""
    v = np.diff(np.log(prices))
    m = np.array(m)

    if d == 0:
        return np.sum(v)
    elif d == 1:
        return np.sum(m*v)
    elif d == 2:
        return np.sum(m*v)/np.sum(m)
    elif d == 3:
        return np.mean(v)/np.std(v)

k = 30
d=3
prices=dataset.mid.tail(1000)
x = np.log(pricing)
v = x.diff()
m = (0.5*(dataset.askSize.tail(1000)+dataset.bidSize.tail(1000)))/((max(dataset.askSize)+max(dataset.bidSize))*0.5)

p0 = pd.rolling_sum(v, k)
p1 = pd.rolling_sum(m*v, k)
p2 = p1/pd.rolling_sum(m, k)
p3 = pd.rolling_mean(v, k)/pd.rolling_std(v, k)


I am working on the file "DS_ML_Jan4.ipynb".
The first task is:
Moving Average Crossover Ribbons
Can you write Python code for it?

asset=dataset.mid.tail(200)
asset.plot(alpha = 0.5)

rolling_means = {}

for i in np.linspace(10, 100, 10):
    X = pd.rolling_mean(asset,int(i))
    rolling_means[i] = X
    X.plot(alpha = 0.7)

rolling_means = pd.DataFrame(rolling_means).dropna()
plt.show()


I am working on the file "DS_ML_Jan4.ipynb".
The first task is:
https://www.quantopian.com/research/notebooks/Cloned%20from%20%22Quantopian%20Lecture%20Series%3A%20Measuring%20Momentum%22.ipynb
Can you write Python code for it?

from statsmodels import regression
import statsmodels.api as sm
import scipy.stats as stats
import scipy.spatial.distance as distance
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

short_mavg = pd.rolling_mean(dataset.mid,5)
long_mavg = pd.rolling_mean(dataset.mid,15)

dataset.mid.tail(100).plot(alpha = 0.5)
dataset.high.tail(100).plot()
dataset.low.tail(100).plot()
dataset.vwap.tail(100).plot()
dataset.km.tail(100).plot()
short_mavg.tail(100).plot()
long_mavg.tail(100).plot(alpha = 0.5)

plt.ylabel('Price')
plt.show()


I am working on the file "DUD-E.ipynb".
The first task is:
Let's see which target has the most actives and decoys.
Can you write Python code for it?

for target in dude_database:
    actives = list(target.actives)
    decoys = list(target.decoys)
    print('Target: ' + target.dude_id,
          'Number of actives: ' + str(len(actives)),
          'Number of decoys: ' + str(len(decoys)),
          sep='\t\t')


I am working on the file "DUD-E.ipynb".
The first task is:
Now we can get one target or iterate over all targets in our directory.
Let's choose one target.
Can you write Python code for it?


I am working on the file "DUD-E.ipynb".
The first task is:
target has four properties: protein, ligand, actives and decoys:

protein - protein molecule

ligand - ligand molecule

actives - generator containing actives

decoys - generator containing decoys
Can you write Python code for it?


I am working on the file "Daily/20150724_Kepler_452.ipynb".
The first task is:
Note that there is not an $0.95 M_{\odot}$ mass track plotted. The right-most track for both model sets is a $0.90 M_{\odot}$ track. Assuming the star is burning hydrogen in the core and is not on the pre-main-sequence, then we can estimate a mass of approximately $1.03\pm0.03 M_{\odot}$ from the Dartmouth 2008 models (dashed lines). If we instead look at the Dartmouth 2015 models, we find the mass is approximately $1.08\pm0.03 M_{\odot}$, consistent with the first esimate, within $2\sigma$. Once one builds in the metallicity uncertainty, the errors increase further, providing a greater consistency between the two measurements. Jenkins et al. quote a mass of $1.04\pm0.05 M_{\odot}$, in agreement with the aforementioned values.
One small factor that was not accounted for is that the observed metallicity provides the present day metallicity, which is not necessarily equivalent to the quote metallicity for model mass tracks. Due to gravitational settling and multiple diffusive processes, one may need to use models with a higher proto-stellar (re: initial) surface metal abundance to acheive a present day value of [Fe/H] $= +0.2$ dex. At most, we might expect an 0.1 dex reduction in the surface abundance of heavy element over time. Although this is very rough, it should provide an upper limit to the uncertainty one expects heavy element diffusion to inflict on model properties.
Can you write Python code for it?

fig, ax = plt.subplots(1, 1, figsize=(10, 10))

# configure axes
ax.set_xlabel('Effective Temperature (K)', fontsize=22.)
ax.set_xlim(6000., 5000.)
ax.set_ylabel('$\log_{10}(g)$', fontsize=22.)
ax.set_ylim(4.6, 4.1)
ax.tick_params(axis='both', which='major', length=20., labelsize=20.)

# directories for metallicity of +0.2, +0.3 dex
d08_directory_03 = '../../evolve/dsep08/trk/fehp03afep0'

# plot mass tracks for GS98 and GAS07 composition
for mass in masses:
    d08_file = '{:s}/m{:03.0f}fehp02afep0.jc2mass'.format(d08_directory, mass*100.)
    d08_03_file = '{:s}/m{:03.0f}fehp03afep0.jc2mass'.format(d08_directory_03, mass*100.)

    try:
        d08_trk = np.genfromtxt(d08_file)
        d08_03_trk = np.genfromtxt(d08_03_file)
    except IOError:
        continue

    ax.plot(10**d08_trk[:, 1], d08_trk[:, 2], '--', lw=2, color='#444444')
    ax.plot(10**d08_03_trk[:, 1], d08_03_trk[:, 2], '-', lw=2, color='#333333')

# add Kepler 452 point
ax.errorbar([5757.], [4.32], xerr=85., yerr=0.09, fmt='-o', lw=3, markersize=14., color='#4682B4')


I am working on the file "Daily/20150724_Kepler_452.ipynb".
The first task is:
Solar Abundance Distribution
Can you write Python code for it?

fig, ax = plt.subplots(1, 1, figsize=(10, 10))

# configure axes
ax.set_xlabel('Effective Temperature (K)', fontsize=22.)
ax.set_xlim(6000., 5000.)
ax.set_ylabel('$\log_{10}(g)$', fontsize=22.)
ax.set_ylim(4.6, 4.1)
ax.tick_params(axis='both', which='major', length=20., labelsize=20.)

# approximate mass range from measured temperature
masses = np.arange(0.90, 1.3, 0.05)

# directories for metallicity of +0.2 dex
f15_directory = '../../evolve/dmestar/trk/gas07/p020/a0/amlt2202'
d08_directory = '../../evolve/dsep08/trk/fehp02afep0'

# plot mass tracks for GS98 and GAS07 composition
for mass in masses:
    f15_file = '{:s}/m{:04.0f}_GAS07_p020_p0_y27_mlt2.202.trk'.format(f15_directory, mass*1000.)
    d08_file = '{:s}/m{:03.0f}fehp02afep0.jc2mass'.format(d08_directory, mass*100.)

    try:
        f15_trk = np.genfromtxt(f15_file)
        d08_trk = np.genfromtxt(d08_file)
    except IOError:
        continue

    ax.plot(10**d08_trk[:, 1], d08_trk[:, 2], '--', lw=2, color='#444444')
    ax.plot(10**f15_trk[:, 1], f15_trk[:, 2], '-', lw=2, color='#333333')

# add Kepler 452 point
ax.errorbar([5757.], [4.32], xerr=85., yerr=0.09, fmt='-o', lw=3, markersize=14., color='#4682B4')


I am working on the file "Daily/20150728_peak_magnetic_field.ipynb".
The first task is:
Differences in the effeictive temperature frame are considerable; nearly 1000 K cooler in the case where magnetic fields are implemented! We can now push the models further with the new boundary definition. For example,
Can you write Python code for it?

filename = 'm0900_GAS07_p000_p0_y26_mlt2.040_mag25kG.trk'
std_iso  = np.genfromtxt('../../evolve/dmestar/trk/gas07/p000/a0/amlt2202/m0900_GAS07_p000_p0_y26_mlt2.202.trk')
new_iso  = np.genfromtxt('../../evolve/models/tmp/{0}'.format(filename))

fig , ax = plt.subplots(3, 1, figsize=(8, 12), sharex=True)

ax[0].set_title('$M = 0.90 M_{\\odot}$ with $\\langle{\\rm B}f\\rangle$ = 2.5 kG', family='serif', fontsize=20.)
ax[2].set_xlabel('Age (Myr)', fontsize=20.)
ax[0].set_ylabel('Effective Temperature (K)', fontsize=20.)
ax[1].set_ylabel('Luminosity ($L_{\\odot}$)', fontsize=20.)
ax[2].set_ylabel('Radius ($R_{\\odot}$)', fontsize=20.)
ax[1].set_ylim(0.0, 7.0)
ax[2].set_ylim(1.0, 7.0)
for axis in ax:
    axis.set_xlim(1.0e-1, 1.0e1)
    axis.tick_params(which='major', axis='both', length=10.0, labelsize=16.)

# Temperature
ax[0].semilogx(new_iso[:,0]/1.0e6, 10**new_iso[:,1], '-', lw=2, color='#1e90ff')
ax[0].semilogx(std_iso[:,0]/1.0e6, 10**std_iso[:,1], '-', lw=4, color='#555555')

# Luminosity
ax[1].semilogx(new_iso[:,0]/1.0e6, 10**new_iso[:,3], '-', lw=2, color='#1e90ff')
ax[1].semilogx(std_iso[:,0]/1.0e6, 10**std_iso[:,3], '-', lw=4, color='#555555')

# Radius
ax[2].semilogx(new_iso[:,0]/1.0e6, 10**new_iso[:,4], '-', lw=2, color='#1e90ff')
ax[2].semilogx(std_iso[:,0]/1.0e6, 10**std_iso[:,4], '-', lw=4, color='#555555')


I am working on the file "Daily/20150728_peak_magnetic_field.ipynb".
The first task is:
In the figure above, the standard track (dark grey) is slightly cooler than the unperturbed magnetic tracks due to a difference in the depth at which the surface boundary conditions are defined. The above track is used only for illustration, as $\tau_{\rm ross} = 10$ standard models have not been computed, yet.
Now, at a higher mass where magnetic models have not previously been computed.
Can you write Python code for it?

filename = 'm0600_GAS07_p000_p0_y26_mlt2.040_mag25kG.trk'
std_iso  = np.genfromtxt('../../evolve/dmestar/trk/gas07/p000/a0/amlt2202/m0600_GAS07_p000_p0_y26_mlt2.202.trk')
new_iso  = np.genfromtxt('../../evolve/models/tmp/{0}'.format(filename))

fig , ax = plt.subplots(3, 1, figsize=(8, 12), sharex=True)

ax[0].set_title('$M = 0.60 M_{\\odot}$ with $\\langle{\\rm B}f\\rangle$ = 2.5 kG', family='serif', fontsize=20.)
ax[2].set_xlabel('Age (Myr)', fontsize=20.)
ax[0].set_ylabel('Effective Temperature (K)', fontsize=20.)
ax[1].set_ylabel('Luminosity ($L_{\\odot}$)', fontsize=20.)
ax[2].set_ylabel('Radius ($R_{\\odot}$)', fontsize=20.)
ax[1].set_ylim(0.0, 5.0)
ax[2].set_ylim(0.0, 5.0)
for axis in ax:
    axis.set_xlim(1.0e-1, 1.0e1)
    axis.tick_params(which='major', axis='both', length=10.0, labelsize=16.)

# Temperature
ax[0].semilogx(new_iso[:,0]/1.0e6, 10**new_iso[:,1], '-', lw=2, color='#1e90ff')
ax[0].semilogx(std_iso[:,0]/1.0e6, 10**std_iso[:,1], '-', lw=4, color='#555555')

# Luminosity
ax[1].semilogx(new_iso[:,0]/1.0e6, 10**new_iso[:,3], '-', lw=2, color='#1e90ff')
ax[1].semilogx(std_iso[:,0]/1.0e6, 10**std_iso[:,3], '-', lw=4, color='#555555')

# Radius
ax[2].semilogx(new_iso[:,0]/1.0e6, 10**new_iso[:,4], '-', lw=2, color='#1e90ff')
ax[2].semilogx(std_iso[:,0]/1.0e6, 10**std_iso[:,4], '-', lw=4, color='#555555')


I am working on the file "Daily/20150728_peak_magnetic_field.ipynb".
The first task is:
It is clear that interior magnetic field strengths remain below 100 kG for all values of the surface field strength when the peak magnetic field strength is assigned to a radial fraction greater than approximately $0.40 R_{\star}$. These values for magnetic field strengths ($< 100 \textrm{ kG}$) permit convergence as they produce values of the magnetic pressure and magnetic energy density that are well below values for the gas pressure and the internal energy of the ambient plasma.
This is important to understand once we introduce magnetic fields in young stars that later develop a radiative core during their pre-main-sequence evolution. Since the code was initially designed to treat the effects of magnetic fields in main-sequence stars, the development of a radiative core was not explicitly considered. Instead, radiative cores existed prior to the inclusion of a magnetic field. Since maost stars have radiative cores that extend no lower than $0.40 R_{\star}$, there was no need to be concerned about convergence outside of strong surface magnetic fields. As a result, we defined the peak magnetic field strength to occur at the base of the tachocline (radiative-convection zone interface).
Fully convective stars were assigned peak magnetic field strengths at $0.30 R_{\star}$, which is just beyond the maximum depth observed for the convection zone in partially convective stars and is also where the peak magnetic field strength appeared (a very slight peak) in 3D MHD models of fully convective dynamos (Browning et al. 2008).
Pre-main-sequence stars therefore pose an interesting problem: star that eventually become partially convective on the main sequence start off as fully convective stars along the Hyashi track. However, once a radiative core begins developing at the beginning of the Henyey track, the peak magnetic field strength jumps from $0.50 R_{\star}$ to $0.12 R_{\star}$, which causes an immense increase in the peak magnetic field strength. Such a deep convection zone is not seen in main-sequence low-mass...
Can you write Python code for it?

filename = 'm0400_GAS07_p000_p0_y26_mlt2.040_mag25kG.trk'
std_iso  = np.genfromtxt('../../evolve/dmestar/trk/gas07/p000/a0/amlt2202/m0400_GAS07_p000_p0_y26_mlt2.202.trk')
old_iso  = np.genfromtxt('../../evolve/dmestar/trk/gas07/p000/a0/amlt2040/mag25kG/{0}'.format(filename))
new_iso  = np.genfromtxt('../../evolve/models/tmp/{0}'.format(filename))

fig , ax = plt.subplots(3, 1, figsize=(8, 12), sharex=True)

ax[0].set_title('$M = 0.40 M_{\\odot}$ with $\\langle{\\rm B}f\\rangle$ = 2.5 kG', family='serif', fontsize=20.)
ax[2].set_xlabel('Age (Myr)', fontsize=20.)
ax[0].set_ylabel('Effective Temperature (K)', fontsize=20.)
ax[1].set_ylabel('Luminosity ($L_{\\odot}$)', fontsize=20.)
ax[2].set_ylabel('Radius ($R_{\\odot}$)', fontsize=20.)
for axis in ax:
    axis.set_xlim(1.0e-1, 1.0e1)
    axis.tick_params(which='major', axis='both', length=10.0, labelsize=16.)

# Temperature
ax[0].semilogx(old_iso[:,0]/1.0e6, 10**old_iso[:,1], '-', lw=3, color='#800000')
ax[0].semilogx(new_iso[:,0]/1.0e6, 10**new_iso[:,1], '-', lw=2, color='#1e90ff')
ax[0].semilogx(std_iso[:,0]/1.0e6, 10**std_iso[:,1], '-', lw=4, color='#555555')

# Luminosity
ax[1].semilogx(old_iso[:,0]/1.0e6, 10**old_iso[:,3], '-', lw=3, color='#800000')
ax[1].semilogx(new_iso[:,0]/1.0e6, 10**new_iso[:,3], '-', lw=2, color='#1e90ff')
ax[1].semilogx(std_iso[:,0]/1.0e6, 10**std_iso[:,3], '-', lw=4, color='#555555')

# Radius
ax[2].semilogx(old_iso[:,0]/1.0e6, 10**old_iso[:,4], '-', lw=3, color='#800000')
ax[2].semilogx(new_iso[:,0]/1.0e6, 10**new_iso[:,4], '-', lw=2, color='#1e90ff')
ax[2].semilogx(std_iso[:,0]/1.0e6, 10**std_iso[:,4], '-', lw=4, color='#555555')


I am working on the file "Daily/20150728_peak_magnetic_field.ipynb".
The first task is:
which leads to the following figure,
Can you write Python code for it?

fig, ax = plt.subplots(1, 1, figsize=(8.0, 8.0))

# configure axis labels and limits
ax.set_xlabel('Radius Fraction, $R/R_{\\star}$', fontsize=20.)
ax.set_ylabel('Magnetic Field Strength (kG)', fontsize=20.)
ax.set_ylim(0.1, 1.0e3)
ax.tick_params(which='major', axis='both', length=10.0, labelsize=16.)
ax.grid(True)

for bf in bfield_surface:
    ax.semilogy(radial_points, bf*bfield_scaling, '-', lw=3)

fig.tight_layout()


I am working on the file "Daily/20150729_small_magnetic_grid.ipynb".
The first task is:
Dartmouth & MARCS; Solar abundance: Grevesse, Asplund, & Sauval 2007
Can you write Python code for it?

# open output file objects
output_files = [open('files/dmestar_{:07.1f}myr_gas07_z+0.00_a+0.00_mag25kG.iso'.format(age/1.0e6), 'w')
                for age in ages]

trk_directory = '../../evolve/dmestar/trk/gas07/p000/a0/amlt2040/mag25kG'
for mass in masses:
    trk_filename = 'm{:04.0f}_GAS07_p000_p0_y26_mlt2.040_mag25kG.trk'.format(mass*1000.)
    try:
        gas07_trk = np.genfromtxt('{0}/{1}'.format(trk_directory, trk_filename), usecols=(0, 1, 2, 3, 4, 8))
    except IOError:
        continue

    # extract only relevant age chunk for easier interpolation
    gas07_trk = np.array([time_step for time_step in gas07_trk if 1.0e6 <= time_step[0] <= 5.0e7])

    # generate linear interpolation curve as a function of age
    try:
        icurve = interp1d(gas07_trk[:, 0], gas07_trk[:, 1:], kind='linear', axis=0)
    except IndexError:
        continue

    # extract properties at the requested age
    trk_props = icurve(ages)

    i = 0
    for props in trk_props:
        s = '{:6.3f}'.format(mass)
        for prop in props:
            if np.isnan(prop) or prop < -12.0:
                prop = -12.0
            s += '{:14.6f}'.format(prop)
        s += '\n'
        output_files[i].write(s)
        i += 1
    #print "{:4.2f} Mo Track Processed.".format(mass)

# close output files
for f in output_files:
    f.close()


I am working on the file "Daily/20150729_small_magnetic_grid.ipynb".
The first task is:
Interpolate onto a finer mass grid,
Can you write Python code for it?

fine_mass_grid = np.arange(0.1, 0.85, 0.02)
for age in ages:
    iso_filename = 'files/dmestar_{:07.1f}myr_gs98_z+0.00_a+0.00_mag25kG.iso'.format(age/1.0e6)
    isochrone = np.genfromtxt(iso_filename)

    # generate interpolation curves
    icurve = interp1d(isochrone[:,0], isochrone[:,1:], axis=0, kind='slinear')

    # interpolate onto a finer mass grid
    fine_isochrone = icurve(fine_mass_grid)
    fine_isochrone = np.column_stack((fine_mass_grid, fine_isochrone))

    # write header
    header  = 'Dartmouth Stellar Evolution Model: Quick Isochrone \n\n'
    header += 'Age = {:7.1f} Myr   [Fe/H] = {:+5.2f}   [a/Fe] = {:+5.2f} \n\n'.format(age/1.e6, 0.0, 0.0)
    header += '{:^14} {:^14} {:^14} {:^14} {:^14} {:^14}'.format('Mass', 'log(Teff)', 'log(g)', 'log(L/Lo)',
                                                                 'log(R/Ro)', 'A(Li)')

    # overwrite original file
    np.savetxt(iso_filename, fine_isochrone, fmt='%14.6f', header=header)


I am working on the file "Daily/20150729_small_magnetic_grid.ipynb".
The first task is:
Magnetic Isochrones
Process the magnetic mass tracks into isochrones. Since mass tracks are computed with a relatively course mass resolution ($0.05 M_{\odot}$), spline interpolation is used to smooth the resulting isochrones with a finer mass resolution.
Below, a grid of isochrones is computed from 5 to 30 Myr in steps of 1 Myr.
Can you write Python code for it?

from scipy.interpolate import interp1d

ages = np.arange(5.0e6, 3.1e7, 1.0e6)  # ages requested


I am working on the file "Daily/20150729_young_magnetic_models.ipynb".
The first task is:
Line styles and colors represent the same model combinations, as before.
Lithium abundance as a function of mass, temperature, and luminosity:
Can you write Python code for it?


I am working on the file "Daily/20150729_young_magnetic_models.ipynb".
The first task is:
Magnetic isochrones were computed earlier. Details can be found in this notebook entry on a small magnetic stellar grid. I'll focus on those computed with the Grevesse, Asplund, & Sauval (2007; henceforth GAS07) solar abundance distribution. Three ages will be examined: 5 Myr, 12 Myr, and 30 Myr, in line with the previous magnetic model diagnostic figures.
Can you write Python code for it?

std_iso_05 = np.genfromtxt('files/dmestar_00005.0myr_z+0.00_a+0.00_gas07_t010.iso')
std_iso_12 = np.genfromtxt('files/dmestar_00012.0myr_z+0.00_a+0.00_gas07_t010.iso')
std_iso_30 = np.genfromtxt('files/dmestar_00030.0myr_z+0.00_a+0.00_gas07_t010.iso')

mag_iso_05 = np.genfromtxt('files/dmestar_00005.0myr_gas07_z+0.00_a+0.00_mag25kG.iso')
mag_iso_12 = np.genfromtxt('files/dmestar_00012.0myr_gas07_z+0.00_a+0.00_mag25kG.iso')
mag_iso_30 = np.genfromtxt('files/dmestar_00030.0myr_gas07_z+0.00_a+0.00_mag25kG.iso')


I am working on the file "Daily/20150803_pressure_reduc_spots.ipynb".
The first task is:
M Dwarfs
Shulyak et al. (2010, 2014) have measured distribution of magnetic fields on the surfaces of M dwarfs by modeling FeH bands in high resolution, high S/N Stokes I sectra. They find M dwarf spectra are typically best fit by a uniform 1 kG magnetic field everywhere on the star, but with the addition of local concentrations across the surface. These local concentrations can reach upward of 7 – 8 kG. We now ask, do the stars that possess these field lie close to the locus defined by models?
Can you write Python code for it?

Shulyak_max_B = np.array([[3400., 100., 6.5, 0.5],
                          [3400., 100., 6.0, 0.5],
                          [3300., 100., 7.5, 0.5],
                          [3100., 100., 6.5, 0.5],
                          [3100.,  50., 1.0, 0.5]])

fig, ax = plt.subplots(1, 1, figsize=(8,4))

ax.set_xlabel('Photospheric Temperature (K)', fontsize=20.)
ax.set_ylabel('Equipartition Magnetic Field (kG)', fontsize=20.)
ax.tick_params(which='major', axis='both', length=10., labelsize=16.)
ax.grid(True)

ax.fill_between(np.arange(3240., 5880., 20.), B_field_Peq_smooth, B_field_Eeq_smooth, facecolor='#555555', alpha=0.3, edgecolor='#eeeeee')
ax.errorbar(Shulyak_max_B[:,0], Shulyak_max_B[:,2], xerr=Shulyak_max_B[:,1], yerr=Shulyak_max_B[:,3], lw=2, fmt='o', c='#555555')


I am working on the file "Daily/20150803_pressure_reduc_spots.ipynb".
The first task is:
We find that the data from Shulyak et al. (2014) for the maximum mangetic field strengths (in local concentrations) of active stars are on the order of those expected from either energy of pressure equipartition. The fact that two have weaker magnetic fields only indicates that the there were no strong local concentrations, similar to the surface of the quiet Sun.
Note, however, that there is some uncertainty in this comparison. Shulyak et al. (2014) quote the effective temperature, which for M dwarfs is not necessarily equal to the photospheric temperature. Model atmospheres predict that the "effective temperature" for an M dwarf occurs in the optically thin layers above the opaque photosphere. Thus, it is more likely that the photospheric temperature for the Shulyak sample is greater than those quoted above.
If we instead convert the quoted effective temperatures to photospheric temperatures using stellar model atmospheres, we find
Can you write Python code for it?

Shulyak_max_B = np.array([[3685., 100., 6.5, 0.5],
                          [3685., 100., 6.0, 0.5],
                          [3578., 100., 7.5, 0.5],
                          [3379., 100., 6.5, 0.5],
                          [3379.,  50., 1.0, 0.5]])

fig, ax = plt.subplots(1, 1, figsize=(8,4))

ax.set_xlabel('Photospheric Temperature (K)', fontsize=20.)
ax.set_ylabel('Equipartition Magnetic Field (kG)', fontsize=20.)
ax.tick_params(which='major', axis='both', length=10., labelsize=16.)
ax.grid(True)

ax.fill_between(np.arange(3240., 5880., 20.), B_field_Peq_smooth, B_field_Eeq_smooth, facecolor='#555555', alpha=0.3, edgecolor='#eeeeee')
ax.errorbar(Shulyak_max_B[:,0] + 100., Shulyak_max_B[:,2], xerr=Shulyak_max_B[:,1], yerr=Shulyak_max_B[:,3], lw=2, fmt='o', c='#555555')


I am working on the file "Daily/20150803_pressure_reduc_spots.ipynb".
The first task is:
While the temperature ratio appears fixed in Equations (7) and (8), variation in the derived values can be estimated from the variation in $\gamma = c_p / c_v$. There will also be variation in the specific relation between the gas internal energy and the gas pressure, but this is more difficult to quantify from stellar evolution model output.
To address this problem, a small grid of models was run with masses in the range $0.1$ — $0.9 M_{\odot}$ with a mass resolution of $0.1 M_{\odot}$. Output from the code was modified to yield $\gamma$ directly as a function of radius, temperature, pressure, and density.
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt

Teffs = np.arange(3000., 6100., 250.)
Tspot_fixed_gamma = Teffs*(0.4)**0.4

# gamma where tau ~ 1 (note, no 0.2 Msun point)
Gammas = np.array([1.22, 1.29, 1.30, 1.29, 1.36, 1.55, 1.63, 1.65])
ModelT = np.array([3.51, 3.57, 3.59, 3.61, 3.65, 3.71, 3.74, 3.77]) # tau = 1
ModelT = 10**ModelT
ModelTeff = 10**np.array([3.47, 3.53, 3.55, 3.57, 3.60, 3.65, 3.69, 3.73]) # T = Teff
Tratio = 0.4**((Gammas - 1.0)/Gammas)
Tspot_physical_gamma = np.array([ModelT[i]*0.4**((Gammas[i] - 1.0)/Gammas[i]) for i in range(len(ModelT))])

# smoothed curve
from scipy.interpolate import interp1d
icurve = interp1d(ModelT, Tspot_physical_gamma, kind='cubic')
Tphot_smoothed = np.arange(3240., 5880., 20.)
Tspot_smoothed = icurve(Tphot_smoothed)

# approximate Berdyugina data
DeltaT = np.array([ 350.,  450.,  700., 1000., 1300., 1650., 1850.])
BerdyT = np.array([3300., 3500., 4000., 4500., 5000., 5500., 5800.])
BSpotT = BerdyT - DeltaT

print Tratio

fig, ax = plt.subplots(1, 1, figsize=(8,4))

ax.set_xlabel('Photospheric Temperature (K)', fontsize=20.)
ax.set_ylabel('Spot Temperature (K)', fontsize=20.)
ax.tick_params(which='major', axis='both', length=10., labelsize=16.)
ax.grid(True)

ax.plot(Teffs, Tspot_fixed_gamma, '--', lw=2, dashes=(10., 10.), markersize=9.0, c='#1e90ff')
ax.plot(BerdyT, BSpotT, '-', lw=2 , dashes=(25., 15.), c='#000080')
ax.fill_between(BerdyT, BSpotT - 200., BSpotT + 200., facecolor='#000080', alpha=0.1, edgecolor='#eeeeee')
ax.plot(ModelT, Tspot_physical_gamma, 'o', lw=2, markersize=9.0, c='#800000')
ax.plot(Tphot_smoothed, Tspot_smoothed, '-', lw=3, c='#800000')

fig, ax = plt.subplots(1, 1, figsize=(8,4))

ax.set_xlabel('Photospheric Temperature (K)', fontsize=20.)
ax.set_ylabel('T(phot) - T(spot) (K)', fontsize=20.)
ax.tick_params(which='major', axis='both', length=10., labelsize=16.)
ax.grid(True)

ax.plot(Teffs, Teffs - Tspot_fixed_gamma, '--', lw=2, dashes=(10., 10.), markersize=9.0, c='#1e90ff')
ax.plot(BerdyT, DeltaT, '-', lw=2 , dashes=(25., 15.),...


I am working on the file "Daily/20150811_young_model_bcs.ipynb".
The first task is:
Grey vs non-grey
Load tracks where the only difference is the adoption of the grey vs non-grey boundary conditions.
Can you write Python code for it?


I am working on the file "Daily/20150811_young_model_bcs.ipynb".
The first task is:
It's clear from the above figures that altering the solar abundance distribution affects the morphology of single star stellar evolution mass tracks. The divergence of mass tracks computed with the GAS07 and AGSS09 solar abundances is curious. Both have helium and heavy element abundances lower than is present in the GS98 track and solar-calibrated mixing lengths that are larger than in the case of the GS98 abundances. In general, the AGSS09 mass tracks mirror the GS98 mass tracks, with the exception of slight offsets that are present throughout fully convective evolution. Some discrepancies occur near where models develop a radiative core below 0.80 Msun.
The figures show that, at young ages, the GAS07 mass tracks are cooler than the GS98 and AGSS09 tracks, often by about 50 K, which is approaching a significant fraction of typical observational uncertainties (~ 70 – 100 K). Why it is that the GAS07 tracks and AGSS09 tracks show different offsets from the GS98 models is unclear. One does not that, at least in the case of lower mass stars, the GAS07 and AGSS09 tracks possess similar morphologies (notably at 0.5 Msun). It may be that the GS98 model atmospheres are not producing appropriate thermal structures at depth. There is some evidence for this based on comparing thermal structures from PHOENIX BT-Settl, NextGen, and MARCS atmospheres to the custom NextGen atmospheres computed for the original Dartmouth model release.
Boundary Condition Fit Depth
Using MARCS model atmospheres with GAS07 solar abundances.
Can you write Python code for it?


I am working on the file "Daily/20150811_young_model_bcs.ipynb".
The first task is:
Now when solar calibration is taken into account
Can you write Python code for it?


I am working on the file "Daily/20150820_A_star_models.ipynb".
The first task is:
All three Dartmouth models exhibit similar morphological trends along the sub-giant branch with subtle differences owing to small differences in adopted chemical compositions, convective mixing lengths, and locations where surface boundary conditions are attached. The Padova model, on the other hand, shows distinct morphological differences compared to the Dartmouth tracks and exhibits an overall cooler effective throughout its evolution by approximately 500 K. I shifted the Parsec track so to match the Dartmouth models near the MSTO around 7800 K by adding 500 K to the effective temperature and 0.02 dex to the logarithm of the bolometric luminosity.
The Padova model shows a morphology that is somewhat mixed between the Dartmouth models with GS98 and GAS07 solar compositions. In general, this does not appear to be a signifiacnt source of discrepancy between the models, with the MS morphology being similar up toward the end of the blue loop. However, the Padova model enters the sub-giant branch with a shallower slope. This leads to the Padova model showing a larger relative difference between temperatures at the turn-off and bRGB. The luminosity at the bRGB is comparable to our track with the GAS07 solar abundance, whereas the GS98 models (with higher overall Z) extend to lower luminosities. The extend of the sub-giant branch in effective temperature among the Padova models suggests that the internal structure of these particular models are distinct from those of the Dartmouth models.
These types of structural differences along the sub-giant branch could lead to a variety of asteroseismic signatures among the models. Whether a higher mass Padova model would yield a similar sub-giant branch extend is hard to gauge with their course grid resolution (see below). We can also ask how radii (here, log(g)s) differ between the models as a function of age to probe how the asteroseismic mean density will be affected by potential structural differences in the evolution models.
Can you write Python code for it?

fig, ax = plt.subplots(1, 1, figsize=(10., 6.))

ax.set_xlim(0.5, 0.7)
ax.set_ylim(4.5, 0.5)
ax.set_xlabel('${\\rm Age\\ (Gyr)}$', fontsize=20.)
ax.set_ylabel('$\\log (g)$', fontsize=20.)
ax.tick_params(which='major', axis='both', length=15., labelsize=16.)

ax.plot(trk_atlas[:,0]/1.0e9, trk_atlas[:,2], lw=2, c='#0094b2', label='GS98, Phx tau=10')
ax.plot(trk_dsep[:,0]/1.0e9, trk_dsep[:,2], lw=2, dashes=(15., 5.), c='#B22222', label='GS98, Phx T(tau)=Teff')
ax.plot(trk_marcs[:,0]/1.0e9, trk_marcs[:,2], lw=2, dashes=(2.0, 2.0), c='#555555', label='GAS07, Mrc tau=50')
ax.plot(trk_parsec[:,1]/1.0e9, trk_parsec[:,4], lw=2, dashes=(10., 10.), c='#9400D2', label='Parsec 2008')

ax.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)


I am working on the file "Daily/20150820_A_star_models.ipynb".
The first task is:
Link to directory where model tracks exist.
Can you write Python code for it?

dsep15_directory = '/Users/grefe950/evolve/dmestar/trk'
dsep08_directory = '/Users/grefe950/evolve/dsep08/trk/fehp00afep0/'
parsec_directory = '/Users/grefe950/evolve/padova/trk/2008/z017y26'


I am working on the file "Daily/20150820_A_star_models.ipynb".
The first task is:
Routine to load and trim model mass track.
Can you write Python code for it?

def loadTrack(filename):
    trk = np.genfromtxt(filename, usecols=(0,1,2,3,4,5))
    bools = [x[0] > 1.0e8 for x in trk]
    return np.compress(bools, trk, axis=0)


I am working on the file "Daily/20150821_mass_track_compositions.ipynb".
The first task is:
Preliminary definitions, including masses and file extensions.
Can you write Python code for it?

masses = [0.1, 0.5, 1.0, 1.5]

# directory extensions
gs98_dir   = 'gs98/p000/a0/amlt1884'
gas07_dir  = 'gas07/p000/a0/amlt2202'
agss09_dir = 'agss09/p000/a0/amlt1991'

# file name extensions
gs98_ext   = '_GS98_p000_p0_y28_mlt1.884.trk'
gas07_ext  = '_GAS07_p000_p0_y26_mlt2.202.trk'
agss09_ext = '_AGSS09_p000_p0_y27_mlt1.991.trk'

colors = {2:'#0094b2', 1:'#B22222', 0:'#56b4ea', 3:'#555555'}
fig, ax = plt.subplots(2, 2, figsize=(12., 12.))

for i in range(len(masses)):
    mass = masses[i]
    row  = i/2
    col  = i%2

    # set axis properties
    axis = ax[row, col]
    axis.tick_params(which='major', axis='both', length=15., labelsize=16.)
    axis.set_xlabel('effective temperature (K)', fontsize=18.)
    axis.set_ylabel('log(L/Lo)', fontsize=18.)
    axis.invert_xaxis()

    # load mass tracks
    gs98 = loadTrack('{:s}/m{:04.0f}{:s}'.format(gs98_dir, mass*1000., gs98_ext))
    gas07 = loadTrack('{:s}/m{:04.0f}{:s}'.format(gas07_dir, mass*1000., gas07_ext))
    agss09 = loadTrack('{:s}/m{:04.0f}{:s}'.format(agss09_dir, mass*1000., agss09_ext))

    axis.plot(10**gs98[:,1], gs98[:,3], lw=3, c=colors[3], label='GS98')
    axis.plot(10**gas07[:,1], gas07[:,3], dashes=(2.0, 2.0), lw=3, c=colors[1], label='GAS07')
    axis.plot(10**agss09[:,1], agss09[:,3], dashes=(20., 5.), lw=3, c=colors[2], label='AGSS09')

    axis.legend(loc=2, fontsize=14.)
fig.tight_layout()

colors = {2:'#0094b2', 1:'#B22222', 0:'#56b4ea', 3:'#555555'}
fig, ax = plt.subplots(4, 1, figsize=(8., 16.))

for i in range(len(masses)):
    mass = masses[i]

    # set axis properties
    axis = ax[i]
    axis.tick_params(which='major', axis='both', length=15., labelsize=16.)
    axis.set_ylabel('effective temperature (K)', fontsize=18.)
    axis.set_xlabel('age (Gyr)', fontsize=18.)

    # load mass tracks
    gs98 = loadTrack('{:s}/m{:04.0f}{:s}'.format(gs98_dir, mass*1000., gs98_ext))
    gas07 = loadTrack('{:s}/m{:04.0f}{:s}'.format(gas07_dir, mass*1000., gas07_ext))
    agss09 =...


I am working on the file "Daily/20150821_mass_track_compositions.ipynb".
The first task is:
Quick mass track loader
Can you write Python code for it?

def loadTrack(filename):
    return np.genfromtxt(filename, usecols=(0, 1, 2, 3, 4, 5))


I am working on the file "Daily/20150826_agb_mtg_preparation.ipynb".
The first task is:
Energy equipartition dictates that the temperature ratio of a spot to its surroundings is roughly equal to $0.4^{(\gamma - 1)/\gamma}$. This leads to the following spot temperatures,
Can you write Python code for it?

fig, ax = plt.subplots(1, 1, figsize=(8,4))

ax.set_xlabel('Effective Temperature (K)', fontsize=20.)
ax.set_ylabel('Spot Temperature (K)', fontsize=20.)
ax.tick_params(which='major', axis='both', length=10., labelsize=16.)
ax.grid(True)

ax.plot(Teff, Ttau*0.4**((gammas - 1)/gammas), '--o', c="#b22222", markersize=10.)


I am working on the file "Daily/20150826_agb_mtg_preparation.ipynb".
The first task is:
Now CMDs against $M_J$
Can you write Python code for it?


I am working on the file "Daily/20150826_agb_mtg_preparation.ipynb".
The first task is:
Now we'll need to load appropriate stellar evolution isochrones with bolometric corrections derived from MARCS model atmospheres. MARCS uses Grevesse et al. (2007), but differences between colors derived from GAS07, AGSS09, and CIFIST11 are negligible.
Can you write Python code for it?

iso_gs98  = np.genfromtxt('../Projects/pleiades_colors/data/dmestar_00120.0myr_z+0.00_a+0.00_phx.iso')
iso_gas07 = np.genfromtxt('../Projects/pleiades_colors/data/dmestar_00120.0myr_z+0.00_a+0.00_marcs.iso')


I am working on the file "Daily/20150826_agb_mtg_preparation.ipynb".
The first task is:
Solar Abundances and NIR Broadband Photometric Colors
We'll begin by loading necessary data, here NIR photometric data for the Pleiades (and M67?).
Can you write Python code for it?

Pleiades_NIR = np.genfromtxt('../Projects/pleiades_colors/data/Stauffer_Pleiades_nir.dat',
                             usecols=(2, 3, 5, 6, 8, 9))  # J, errJ, H, errH, K, errK

dist_mod = 5.62 # distance modulus for the Pleiades


I am working on the file "Daily/20150826_agb_mtg_preparation.ipynb".
The first task is:
Spot Temperatures on AGB Stars
Using fairly simple energy equipartition arguments with a polytropic equation of state, I was able to estimate spot temperature contrasts for low mass stars with convective envelopes (G – M) with reasonble accuracy. Can we do this for AGB stars?
We'll use MARCS model hydrostatic model atmospheres to derive approximations for atmospheric properties where $\tau \approx 1$.
Can you write Python code for it?


I am working on the file "Daily/20150826_agb_mtg_preparation.ipynb".
The first task is:
Temperature contrasts are on the order of 86 – 87%, meaning spots are only slightly cooler than the surrounding photosphere.
Estimating magnetic field strengths required to generate spots is more difficult to assess. One could compare the local Alfven velocity to the convective velocity. Alternatively, following an analogy with the Sun, umbral magnetic field strengths often reach equipartition with the gas pressure outside of a spot. Therefore,
Can you write Python code for it?

fig, ax = plt.subplots(1, 1, figsize=(8,4))

ax.set_xlabel('Effective Temperature (K)', fontsize=20.)
ax.set_ylabel('Equipartition Strength (G)', fontsize=20.)
ax.tick_params(which='major', axis='both', length=10., labelsize=16.)
ax.grid(True)

ax.plot(Teff, np.sqrt(8.0*np.pi*Ptau), '--o', c="#b22222", markersize=10.)


I am working on the file "Daily/20150826_agb_mtg_preparation.ipynb".
The first task is:
We've loaded isochrones with both Grevesse & Sauval (1998) and Grevesse et al. (2007) solar abundance distributions. The former is similar to Anders & Grevesse (1989) and Grevesse & Noels (1993) distributions in that the solar surface $(Z/X) = 0.023$ – $0.025$. On the other hand, the Grevesse et al. (2007) distribution has a characteristic $(Z/X) = 0.014$.
Starting with CMDs against $M_K$.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
A standard isochrone would only have 6 columns, so 11 indicates this isochrone does have photometric magnitudes computed, likely BV(Ic) (JK)2MASS.
Can you write Python code for it?

test_bcs = test_surface(10**iso[:,1], iso[:, 2], 0.0)

test_bcs.shape


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
Again, we have a good match to tabulated values,
Teff      logg   [Fe/H]  [a/Fe]         B           V           R           I
3000.00    5.00    0.00    0.00        -6.603      -5.641      -4.566      -3.273
However, since we are using a tabulated metallicity, the interpolation may proceed without too much trouble. If we select a metallicity between grid points, how do we fare?
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
Change to directory containing bolometric correction files.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
Compute magnitudes for a Dartmouth isochrone.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
Convert surface magnitudes to absolute magnitudes using the distance modulus and the radius of the star.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
Export a new isochrone with colors from AGSS09 (PHX)
Can you write Python code for it?

new_isochrone = np.column_stack((iso[:, :6], phx_mags))
np.savetxt('/Users/grefe950/Notebook/Projects/pleiades_colors/data/dmestar_00120.0myr_z+0.00_a+0.00_mixed.iso',
            new_isochrone, fmt='%16.8f')


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
For each Teff and logg combination we now have BCs for BV(RI)c from BT-Settl models. Now we need to convert the bolometric corrections to absolute magnitudes.
Can you write Python code for it?

bol_mags = 4.74 - 2.5*iso[:, 3]
for i in range(test_bcs.shape[1]):
    bcs = -1.0*np.log10(10**iso[:, 1]/5777.) + test_bcs[:, i] - 5.0*iso[:, 4]
    if i == 0:
        test_mags = bol_mags - bcs
    else:
        test_mags = np.column_stack((test_mags, bol_mags - bcs))
iso[50, 0:4], iso[50, 6:], test_mags[50]


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
For reference, at [Fe/H] = $-0.5$ dex, we have
Teff      logg   [Fe/H]  [a/Fe]         B           V           R           I
3000.00    5.00   -0.50    0.20        -6.533      -5.496      -4.424      -3.154
The interpolation routine has seemingly handled the non-monotonic nature of the metallicity column, as all interpolate values lie between values at the two respective nodes.

Now let's import an isochrone and calcuate colors for stellar models for comparison against MARCS bolometric corrections.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
Load a bolometric correction table, say for the Cousins AB photometric system.
Can you write Python code for it?

bc_table = np.genfromtxt('colmag.BT-Settl.server.JOHNSON.AB.bolcor', comments='!')


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
Load an isochrone from the Lyon-Phoenix series.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
Now compare against MARCS values.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
Now, the structure of the file is quite irregular. The grid is not rectangular, which is not an immediate problem. The table is strucutred such that column 0 contains Teff in increasing order, followed by logg in column 1 in increasing order. However, metallicities in column 2 appear to be in decreasing order, which may be a problem for simple interpolation routines. Alpha abundances follow and are in increasing order, but since this is a "standard" grid, whereby alpha enrichment is a function of metallicity, we can ignore it for the moment.
Let's take a first swing at the problem by using the LinearND Interpolator from SciPy.
Can you write Python code for it?

test_surface = scint.LinearNDInterpolator(bc_table[:, :3], bc_table[:, 4:])


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
Separate Test Case
These are clearly not correct and are between 1 and 2 magnitudes off from expected values. Need to reproduce the Phoenix group's results, first.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
The surface compiled, but that is not a guarantee that the interpolation will work successfully. Some tests are required to confirm this is the case. Let's try a few Teffs at logg = 5 with solar metallicity.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
This agrees with data in the bolometric correciton table.
Teff      logg   [Fe/H]  [a/Fe]         B           V           R           I
1500.00    5.00    0.00    0.00       -15.557     -16.084     -11.560      -9.291
Now, let's raise the temperature.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_bol_corrs.ipynb".
The first task is:
This appears consistent. What about progressing to lower metallicity values?
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_cifist_bcs.ipynb".
The first task is:
Again, we have a good match to tabulated values,
Teff      logg   [Fe/H]  [a/Fe]         B           V           R           I
3000.00    5.00    0.00    0.00        -6.603      -5.641      -4.566      -3.273
However, since we are using a tabulated metallicity, the interpolation may proceed without too much trouble. If we select a metallicity between grid points, how do we fare?
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_cifist_bcs.ipynb".
The first task is:
Change to directory containing bolometric correction files.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_cifist_bcs.ipynb".
The first task is:
For each Teff and logg combination we now have BCs for BV(RI)c from BT-Settl models. Now we need to convert the bolometric corrections to absolute magnitudes.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_cifist_bcs.ipynb".
The first task is:
For reference, at [Fe/H] = $-0.5$ dex, we have
Teff      logg   [Fe/H]  [a/Fe]         B           V           R           I
3000.00    5.00   -0.50    0.20        -6.533      -5.496      -4.424      -3.154
The interpolation routine has seemingly handled the non-monotonic nature of the metallicity column, as all interpolate values lie between values at the two respective nodes.

Now let's import an isochrone and calcuate colors for stellar models for comparison against MARCS bolometric corrections.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_cifist_bcs.ipynb".
The first task is:
Let's try something different: using the color tables provided by the Phoenix group, from which the bolometric corrections are calculated.
Can you write Python code for it?

col_table = np.genfromtxt('colmag.BT-Settl.server.COUSINS.Vega', comments='!')


I am working on the file "Daily/20150902_phoenix_cifist_bcs.ipynb".
The first task is:
Load a bolometric correction table, say for the Cousins AB photometric system.
Can you write Python code for it?

bc_table = np.genfromtxt('colmag.BT-Settl.server.JOHNSON.Vega', comments='!')


I am working on the file "Daily/20150902_phoenix_cifist_bcs.ipynb".
The first task is:
Load an isochrone from the Lyon-Phoenix series.
Can you write Python code for it?


I am working on the file "Daily/20150902_phoenix_cifist_bcs.ipynb".
The first task is:
Now, the structure of the file is quite irregular. The grid is not rectangular, which is not an immediate problem. The table is strucutred such that column 0 contains Teff in increasing order, followed by logg in column 1 in increasing order. However, metallicities in column 2 appear to be in decreasing order, which may be a problem for simple interpolation routines. Alpha abundances follow and are in increasing order, but since this is a "standard" grid, whereby alpha enrichment is a function of metallicity, we can ignore it for the moment.
Let's take a first swing at the problem by using the LinearND Interpolator from SciPy.
Can you write Python code for it?

test_surface = scint.LinearNDInterpolator(bc_table[:, :2], bc_table[:, 4:])


I am working on the file "Daily/20150902_phoenix_cifist_bcs.ipynb".
The first task is:
This agrees with data in the bolometric correciton table.
Teff      logg   [Fe/H]  [a/Fe]         B           V           R           I
1500.00    5.00    0.00    0.00       -15.557     -16.084     -11.560      -9.291
Now, let's raise the temperature.
Can you write Python code for it?


I am working on the file "Daily/20150912_mag_profiles.ipynb".
The first task is:
For particular types of instabilities, we are concerned with the radial variation of $B/\rho$.
Can you write Python code for it?


I am working on the file "Daily/20150912_mag_profiles.ipynb".
The first task is:
Now we can load other files containing the density structures.
Can you write Python code for it?

g_grad = np.genfromtxt('files/gauss.grad')
d_grad = np.genfromtxt('files/dipole.grad')


I am working on the file "Daily/20150912_mag_profiles.ipynb".
The first task is:
Plot as a function of depth in the star, based on magnetic field profiles.
Can you write Python code for it?

fig, ax = plt.subplots(1, 1, figsize=(7., 5.))

ax.tick_params(which='major', axis='both', labelsize=16., length=14.)
ax.set_xlabel('Fractional Radius', fontsize=16.)
ax.set_ylabel('Linear Growth Rate [day]', fontsize=16.)

ax.semilogy(10**(g_grad[:,0] - 10.26), Hp/va_g/86400., lw=3, c='#555555', label='Gaussian')
ax.semilogy(10**(d_grad[:,0] - 10.26), Hp/va_d/86400., lw=3, c='#b22222', label='Dipole')
ax.legend(loc=1)