I am working on the file "Projects/mlt_calib/resampling_tests.ipynb".
The first task is:
Of course, we should perhaps only look at the limited range of our sample that overlaps with the calibration range of Bonaca et al. (2012).
Can you write Python code for it?

low_sub = np.array([star for star in low_data if star[1] <= 4.6])

our_alphas = trifit_low.predict(low_sub)
b12_alphas = (c0 + c1*low_sub[:, 1] + c2*low_sub[:, 3] + c3*low_sub[:, 2])*1.884/1.69

fig, ax = plt.subplots(1, 1, figsize=(5., 5.))

ax.set_xlim(0.9, 2.0)
ax.set_ylim(0.9, 2.0)

ax.plot(our_solar*1.884, b12_solar*1.884, '*', markersize=10., c='y')
ax.plot([0.0, 2.5], [0.0, 2.5], dashes=(20., 5.), lw=2, c='#b22222')
ax.plot(our_alphas, b12_alphas, 'o', markersize=6.0, c='#555555', alpha=0.6)


I am working on the file "Projects/mlt_calib/resampling_tests.ipynb".
The first task is:
On the left, we have residuals and partial resdiuals when all data points are used in the trilinear analysis. On the right, the same for when points affected by the grid boundaries have been removed. The top two panels show total residuals of the fit to mixing length parameters as a function of inferred stellar mass. There are issues at the lowest masses in both instances, either due to the influence of low mass stars with high values for the inferred mixing length parameters, or because there is a genuine change of slope in the relation that is not adequately reproduced by a linear model. This change of slope may be the result of not treating atmospheric physics self-consistently (fixed $\alpha = 1.5$), which may affect cool stars where convective is occuring in the outer, optically thin layers.
Partial residuals are shown in the bottom six panels. These isolate the impact of the given independent variables on the dependent variable (here $\alpha$). Therefore, we see that in each case, there is a direct correlation between the dependent parameter (stellar properties) and $\alpha$, with the exception of the metallicity on the left-hand side for which we cannot rule out the null hypothesis that the correlation is the result of random scatter. For the rest of the cases, the correlations are readily apparent.
We can thus conclude that, for our sample of stars under the model conditions/assumptions present in the Dartmouth models, the mixing length parameters is: (1) directly correlated with log(g) and log(Teff) and (2) plausibly correlated with metallicity.
Would like to construct a figure like Figure 5 from Ludwig, Freytag, & Steffen (1999). In lieu of that (for the moment), we can compute our value for the mixing length at some of their grid points.
Can you write Python code for it?

logg = 4.44
logT = np.log10(4500.)
Fe_H = 0.0

print("Our alpha: {:5.3f}; LFS alpha: 1.7".format(b0 + b1*logg + b2*Fe_H + b3*logT))

logg = 4.44
logT = np.log10(5770.)
Fe_H = 0.0

print("Our alpha: {:5.3f}; LFS alpha: 1.6".format(b0 + b1*logg + b2*Fe_H + b3*logT))


I am working on the file "Projects/mlt_calib/resampling_tests.ipynb".
The first task is:
Panel 3 in the top right shows estimates for the helium abundance, which was not constrained in this run, and my be safely ignored for comparing the validity of the two methods. Both methods yields similar results with some difference in the inferred age and mixing length. However, they do not appear to significntly affect the results. It is therefore seems most reasonable to use the KDEs computed using bootstrap resampled posteriors from the full MCMC simulations (probability weighted).
How does this compare to results where we compute the KDE from the resulting posterior distributions of the MCMC simluation, without weighting results by their probability (a second time)?
Can you write Python code for it?

fig, ax = plt.subplots(3, 3, figsize=(12., 12.))

for i in range(9):
    row = i/3
    col = i%3
    axis = ax[row, col]

    axis.plot([min(kde_fbr[:, i]), max(kde_fbr[:, i])], [min(kde_fbr[:, i]), max(kde_fbr[:, i])],
                  '-', lw=3, c='#b22222', alpha=0.8)
    axis.plot(kde_fbr[:, i], kde_pdf[:, i], 'o', markersize=5.0, c='#555555', alpha=0.6)
fig.tight_layout()


I am working on the file "Projects/mlt_calib/resampling_tests.ipynb".
The first task is:
Perform a trilinear regression to the data.
Can you write Python code for it?

trifit_all = sm.regression.linear_model.GLS(fit_data_all[:, 5], all_data, sigma=fit_data_all[:, 14]).fit()
trifit_low = sm.regression.linear_model.GLS(fit_data_low[:, 5], low_data, sigma=fit_data_low[:, 14]).fit()

print(trifit_all.summary())  # All data points included


I am working on the file "Projects/mlt_calib/resampling_tests.ipynb".
The first task is:
Results from this comparison are nearly identical to those from the previous comparison. Data from the previous run are shown as light gray points above.
Boundary Condition Fit Depth
Here, we investigate whether our fitting of the surface boundary conditions below the nominal photosphere ($\tau = 10$) produces different results as the mixing length parameter is changed. Such a dependence would not be too surprising, as outer radiative layers are typically more sensitive to variations in the mixing length parameter. Given that we are not capturing variation in those layers (fixed atmospheric boundary conditions), this may emphasize the role of deeper convection for setting the stellar radii in our models. The latter is more difficult to perturb, so lower values of the mixing length may be requied to provide accurate models.
Let's now look at how the stellar radius is affected by our choice of boundary conditions for three separate masses that span the range of stellar masses probed in our study.
Can you write Python code for it?


I am working on the file "Projects/mlt_calib/resampling_tests.ipynb".
The first task is:
Results in this case are quite striking. Ages, mixing lengths, and metallicities all appear quite different between the two modal estimates from their respective KDEs. With metallicities, we find that the KDE applied directly to the posterior distribution function from the MCMC simluation produces metallicities that are up to 0.2 dex higher than when we resample the posterior. We also find that ages tend to be older when using the raw posterior distributions. Similarly, there is a propensity for the raw posteriors to produce higher mixing length parameters as compared to the resampled posterior.

How do these differences affect the resulting relationships that we derive for the mixing length parameter as a funciton of stellar properties? Let's look at the same two final sets of inferred parameters as the previous figure.
Can you write Python code for it?


I am working on the file "Projects/mlt_calib/resampling_tests.ipynb".
The first task is:
The results can largely be inferred by visiual inspection of the two sets of data. In the case of bootstrap resampled inferences, the two dominant correlations are between temperature and mass, while there is not clear correlation with metallicity. For this data, the Pearson $\rho$ correlation coefficient is not necessarily a reasonable test, given that there is not necessarily a linear relationship between the different parameters. However, both Kendall $\tau$ and Spearman $r$ correlation coefficients, which do not assume linearity, show what could be significant trends with temperature and mass. These trends are far more significant when we use a bootstrapped resampled KDE as opposed to the raw MCMC simluation results.
This is quite concerning, since one wishes to have a single unique modal value that is relatively independent of the method used for estimating it. Is there something behind the differences? I postulate that the differences may be due to the adopted metallicity uncertainties and whether one wishes to place more confidence in the observed values. As a test, I recomputed the bootstrap resampled estimates using a larger uncertainty on the metallicity (0.2 dex instead of 0.05 dex) to see if that produces a significant differences in the results.
Can you write Python code for it?


I am working on the file "Projects/mlt_calib/resampling_tests.ipynb".
The first task is:
There is an approximately 4% difference between the extrapolated solar mixing length and the true solar calibrated value. For comparison, the mixing length trilinear fit presented by Bonaca et al. (2012) yields a solar mixing length parameters 6% lower than their solar calibrated value -- again, quite good agreement given that this did not necessarily need to be the case.

BF noted that our relations appear to be significantly steeper than those predicted by both Bonaca et al. (2012) and 3D RHD models. The latter is certainly true, but how do our relations compare to Bonaca et al. (2012), suitably re-scaled to our solar calibration point? First we need to define the coefficients in their fit.
Can you write Python code for it?


I am working on the file "Projects/mlt_calib/resampling_tests.ipynb".
The first task is:
We recover the metallicity correlation. In fact, the recovered rank-correlation coefficients are fully consistent with modal values computed from the raw posterior distributions. This result highlights the sensitivity of the resulting correlation between metallicity and mixing length parameter with our confidence in the observed metallicities (weak or strong prior).
Critically, the strength of the metallicity prior does not alter the inferred correlations between mass and $T_{\rm eff}$ with $\alpha_{\rm MLT}$. These two trends appear robust against our prior assumptions regarding the metallicity measurement uncertianty.

Tri-linear analysis
To compare with other works in the literature, a tri-linear analysis should be performed so that we can simultaneously extract how $\alpha$ varies as a function of multiple parameters.
Can you write Python code for it?


I am working on the file "Projects/newsrobot-moviereview.ipynb".
The first task is:
Scrape Internet Movie Database
Here we'll extract information from the HTML code. This is the hard part. Luckily, we already have a lot of information here.
Can you write Python code for it?

# Scrape movie Interstellar (2014).
soup = makesoup('http://www.imdb.com/title/tt0816692/')

# Movie ID.
movieid = soup.find(property="pageId").get("content")

# Movie URL.
movielink = 'http://www.imdb.com/title/' + movieid + '/'

# Title.
title = soup.find(itemprop="name").get_text()
title = title.strip() # Remove white spaces before and after title.

# Year.
year = soup.find(id="titleYear").get_text()
year = year[1:5] # Remove parentheses, so that (2014) becomes 2014.
year = int(year) # Year is a string now, int() will convert it to an integer (number).

# Remove year from title.
titleclean = title.replace("(" + str(year) + ")", "").strip()

# Length of movie.
duration = soup.find(itemprop="duration").get_text()
duration = duration.strip() # Remove white spaces before and after.

# Movie director.
director = soup.find(itemprop="director").find(itemprop="name").get_text()
director = director.strip() # Remove white spaces before and after.

# Movie rating.
rating = soup.find(itemprop="ratingValue").get_text()
rating = float(rating) # Float makes rating into a decimal number, like 6.8.

# Create a list of main actors.
actors = []
for castlist in soup.find_all("table", "cast_list"):
	for actor in castlist.find_all(itemprop="name"):
		actors.append(actor.get_text().strip())


I am working on the file "Projects/ngc2516_spots/.ipynb_checkpoints/bolometric_corrections-checkpoint.ipynb".
The first task is:
Curiously, it returns values that do not appear to be out of line. It's quite possible that the code is somehow attempting to extrapolate since we use a 4-point Lagrange inteprolation, which may unknowingly provide an extrapolation beyond the defined grid space. Comparing to Phoenix model atmospheres with the Caffau et al. (2011) solar abundances for the Johnson-Cousins and 2MASS systems, our optical $BV(RI)_C$ magnitudes are systematically 1.0 mag fainter than Phoenix at 120 Myr, and our $JHK$ magnitudes are fainter by approximately 0.04 mag at the same age.
Finally, safely deallocate memory and close the log file.
Can you write Python code for it?


I am working on the file "Projects/ngc2516_spots/.ipynb_checkpoints/bolometric_corrections-checkpoint.ipynb".
The first task is:
Next, we need to load the appropriate tables.
Can you write Python code for it?

FeH = 0.0  # dex; atmospheric [Fe/H]
aFe = 0.0  # dex; atmospheric [alpha/Fe]

brand = 'marcs'  # use theoretical corrections from MARCS atmospheres
phot_filters = ['U', 'B', 'V', 'R', 'I', 'J', 'H', 'K']  # select a subset of filters

bc.bolcorrection.bc_init(FeH, aFe, brand, phot_filters)  # initialize tables


I am working on the file "Projects/ngc2516_spots/bolometric_corrections.ipynb".
The first task is:
Curiously, it returns values that do not appear to be out of line. It's quite possible that the code is somehow attempting to extrapolate since we use a 4-point Lagrange inteprolation, which may unknowingly provide an extrapolation beyond the defined grid space. Comparing to Phoenix model atmospheres with the Caffau et al. (2011) solar abundances for the Johnson-Cousins and 2MASS systems, our optical $BV(RI)_C$ magnitudes are systematically 1.0 mag fainter than Phoenix at 120 Myr, and our $JHK$ magnitudes are fainter by approximately 0.04 mag at the same age.
Finally, safely deallocate memory and close the log file.
Can you write Python code for it?


I am working on the file "Projects/ngc2516_spots/bolometric_corrections.ipynb".
The first task is:
Next, we need to load the appropriate tables.
Can you write Python code for it?


I am working on the file "Projects/ngc2516_spots/bolometric_corrections.ipynb".
The first task is:
The extremely large (or small) magnitudes for the 5300 K star is very strange. These issues do not occur for the same command execution in the terminal shell.
Now, what happens when we happen to request a temperature for a value outside the grid?
Can you write Python code for it?

bc.bolcorrection.bc_eval(2204.0, 4.83, -3.47, len(phot_filters))  # outside of grid -- should return garbage.


I am working on the file "Projects/ngc2516_spots/cmd_age_composition.ipynb".
The first task is:
Broad band optical photometry is availble for NGC 2516 from Jeffries, Thurston, & Hambly (2001, A&A, 375, 863).
Can you write Python code for it?

ngc2516 = np.genfromtxt('data/jeff_2001.tsv', delimiter=';', comments='#')
ngc2516 = np.array([star for star in ngc2516 if star[9] == 1])  # extract candidate members

fig, ax = plt.subplots(1, 2, figsize=(16, 8), sharey=True)

for axis in ax:
    axis.grid(True)
    axis.set_ylim(22., 8.)
    axis.tick_params(which='major', axis='both', length=10., labelsize=18.)
ax[0].set_ylabel('$V\\ {\\rm(mag)}$', fontsize=20.)
ax[0].set_xlabel('$(B - V)_0$', fontsize=20.)
ax[1].set_xlabel('$(V - I_C)_0$', fontsize=20.)

ax[0].plot(ngc2516[:,4], ngc2516[:,3], 'o', markersize=7.0, c='#999999')
ax[1].plot(ngc2516[:,5], ngc2516[:,3], 'o', markersize=7.0, c='#999999')

fig.tight_layout()


I am working on the file "Projects/ngc2516_spots/cmd_age_composition.ipynb".
The first task is:
In the above CMDs, candidate binary members are shown as light grey points, while candidate single-star members are shown as dark grey points. Note that in the $(B-V)_0/V$ CMD there are numerous binary candidates lying behind the single-star candidate points.
We can now use the single-star sequence to estimate the distance modulus, reddening and extinction coefficients, the cluster age, and chemical composition. The latter depends on which set of model isochrones that we adopt, with explicit regard for the solar abundance distribution adopted by the model sets. Helioseismic evidence lends favor to the Grevesse & Sauval (1998) solar abundance distribution, as does the morphology of the solar-age open cluster M67, where the development of a convective core appears at too high a mass when adopting the Grevesse, Asplund, & Sauval (2007) solar composition (VandenBerg et al. 2008).
However, while stellar interior structure favors the adoption of solar abundance distributions with a higher metallicity, results from studies of broad band colors of low-mass stars indicate that more modern values of the solar metallicity are preferred, particularly with repsect to the revised oxygen abundance. This appears to not just be the result of revised molecular line lists (e.g., TiO, H$_2$O), but is related more to the lower abundance of oxygen (Baraffe et al. 2015; Feiden, Eriksson, & Christophe in prep). Therefore, we fit CMDs for NGC 2516 with isochrones with both the GS98 and GAS07 solar abundance distributions.
Note that more recent measurements of the solar abundance distribution have increased the overall metallicity from that presented by GAS07, but this has only a small impact on model interior structure and derived spectra / bolometric corrections (B. Edvardsson, priv. comm.).
We can adopt prior estimates of the distance modulus and reddening coefficients as a starting point for our analysis. Terndrup et al. estimated that $E(B-V) = 0.05 \pm 0.01$ and $(m - M) = 7.96 \pm 0.05$. We'll need to use only a subset of...
Can you write Python code for it?


I am working on the file "Projects/ngc2516_spots/cmd_age_composition.ipynb".
The first task is:
Notably, the $(B-V)/V$ CMD has sources along the sub-giant and giant branches, and seemingly stars near the base of the RGB. WEBDA flags stars extending from the MS between $V = 6 \rightarrow 8$ as blue stragglers. A more positive identification is required to ensure this is not an extension of the MS itself.
Current Phoenix color-Teff transformations provided by Dotter do not extend to hot enough temperatures to permit a fully CMD analysis. The turn-off for NGC 2516 appears to occur somewhere in the vicinity of $4.0 \pm 0.5 M_{\odot}$, at least based on a quick analysis. Therefore, we adopt the semi-empirical bolometric corrections from VandenBerg & Clem (2003), which provide reasonble agreement with purely synthetic transformations at hotter temperatures. From this, we can see that a 120 Myr isochrone is too young, based on the MS turnoff. However, it should be noted that we must use a more rigorous isochrone computation to reliably determine the CMD morphology from the turn-off to the base of the RGB.
This is compatible with estimates from Silaj & Landstreet (2014), who estimated an age of approximately 160 Myr from CMD fitting, but a younger age of 125 Myr from HRD fitting. Their analysis did not include a complete CMD or HRD, but only from a select few star (although they were located at critical junctures). Nevertheless, the few apparent sub-giant branch stars are crucial to identifying an appropriate isochrone, as they provide an estimate of the morphology between the MSTO and bRGB.
M Dwarf Knee
Here, we zoom in on this feature and arbitrarly shift the GS98 isochrone in distance so that it is conincident with the empirical sequence. Similarly, we provide an arbitrary shift in reddening to the GAS07 isochrone to provide conicidence with the M-dwarf knee.
Can you write Python code for it?

fig, ax = plt.subplots(1, 1, figsize=(8, 8))

ax.grid(True)
ax.set_xlim(1.0, 1.8)
ax.set_ylim(21., 15.)
ax.tick_params(which='major', axis='both', length=10., labelsize=18.)
ax.set_ylabel('$V\\ {\\rm(mag)}$', fontsize=20.)
ax.set_xlabel('$(B - V)_0$', fontsize=20.)

# (B-V)
ax.plot(ngc2516_bin[:,4], ngc2516_bin[:,3], 'o', markersize=7.0, c='#eeeeee')
ax.plot(ngc2516_sin[:,4], ngc2516_sin[:,3], 'o', markersize=7.0, c='#999999')

# arbitrary (re: unphysical) shift to reddening
ax.plot((iso_gs98[:,6] - iso_gs98[:,7]), iso_gs98[:,7] + 7.95, '-', lw=3, c='#A21616')

# arbitrary (re: unphysical) shift to reddening
ax.plot((iso_gas07[:,6] - iso_gas07[:,7]) + 0.05, iso_gas07[:,7] + 7.95, '-', lw=4, c='#1F7676')

fig.tight_layout()


I am working on the file "Projects/ngc2516_spots/cmd_age_composition.ipynb".
The first task is:
Take note that the red isochrone is computed with the GS98 solar abundance distribution and the blue isochrone is computed with the GAS07 abundance distribution.
Without much effort, the isochrones reproduce the properties of upper main sequence in the CMDs. There are clear issues at the reddest $(B-V)_0$ colors, but as we noted earlier, the M-dwarf knee occurs at approximately the same location in both the observations and the models. There is a slow decline in the isochrone's $V$-band magnitude prior to the onset of the M-dwarf knee in the GAS07 isochrone, but the dominant knee feature is inline with a similar feature in the data. Similarly, although the GS98 isochrone is clearly too luminous [actually, since the other CMD is so well reproduced, it's likely that the $(B-V)$ color of the GS98 isochrone is incorrect], the morphology at and below the M-dwarf knee is well reproduced.
To extract an age, composition, reddening, etc., we need to apply a quantitative metric to assign goodness of fit. A typical $\chi^2$ metric might be useful as a first pass, cycling through several ages (CMDs are likely insensitive to age), metallicities, and reddening/extinction coefficients. Since there is no turn-off or sub-giant-branch present in the data, a more sophisticated approach examining number counts at various evolutionary stages would be uninformative. That is, unless there is more data available on the open cluster archive, WEBDA.
Naturally, WEBDA has a collection of data on NGC 2516. At least one reference contains target sources on the sub-giant and red-giant branches. These should be incorporated into our fitting to permit a reasonable age estimate. The archive indicates there are at least 9 Ap stars and 1 Am stars. A recent article proposed using Ap stars to estimate ages of clusters. In fact, Silaj & Landstreet (2014, A&A, 566, A132) have used magnetic Ap stars to estimate the age of NGC 2516.
Loading WEBDA $UBV$ photometric data.
Can you write Python code for it?


I am working on the file "Projects/ngc2516_spots/cmd_age_composition.ipynb".
The first task is:
The difficult part of using this feature is defining it quantitatively. There is clearly a rounded bend to the knee followed by a nearly vertical drop before (potentially) recovering to a more shallower grade. If we can quantitatively define at which magnitude the slope becomes nearly vertical, it can be used a "standard candle" of sorts, with that feature corresponding to an absolute magnitude in $V$, or even in other passbands, like $K$, which is better determined by stellar models.
Unfortunately for NGC 2516, there is no reliable $K$-band photometry across the entire CMD, seemingly only at lower masses. We may attempt $I_C$-band for this data set,
Can you write Python code for it?

fig, ax = plt.subplots(1, 1, figsize=(8, 8))

ax.grid(True)
ax.set_xlim(1.0, 1.8)
ax.set_ylim(19., 13.)
ax.tick_params(which='major', axis='both', length=10., labelsize=18.)
ax.set_ylabel('$I_C\\ {\\rm(mag)}$', fontsize=20.)
ax.set_xlabel('$(B - V)_0$', fontsize=20.)

# (B-V)
ax.plot(ngc2516_bin[:,4], ngc2516_bin[:,3]-ngc2516_bin[:,4], 'o', markersize=7.0, c='#eeeeee')
ax.plot(ngc2516_sin[:,4], ngc2516_sin[:,3]-ngc2516_sin[:,4], 'o', markersize=7.0, c='#999999')

# arbitrary (re: unphysical) shift to reddening
ax.plot((iso_gs98[:,6] - iso_gs98[:,7]) +0.05, iso_gs98[:,9] + 7.95, '-', lw=3, c='#A21616')

# arbitrary (re: unphysical) shift to reddening
ax.plot((iso_gas07[:,6] - iso_gas07[:,7]) + 0.05, iso_gas07[:,8] + 7.95, '-', lw=4, c='#1F7676')

fig.tight_layout()


I am working on the file "Projects/ngc2516_spots/cmd_age_composition.ipynb".
The first task is:
When fitting CMDs with isochrones, a question arises of whether we should fit the CMD with or without candidate binary stars and also whether the we should fit the median of the sequence, the lower edge, or upper edge of the sequence. For now, we'll fit the isochrone to the median value of the single star sequence. That is, we're going to remove candidate binaries from the membership sample.
Can you write Python code for it?

ngc2516_sin = np.array([star for star in ngc2516 if star[10] == 0])  # extract candidate non-binary members
ngc2516_bin = np.array([star for star in ngc2516 if star[10] == 1])  # extract candidate binary members

fig, ax = plt.subplots(1, 2, figsize=(16, 8), sharey=True)

for axis in ax:
    axis.grid(True)
    axis.set_ylim(22., 8.)
    axis.tick_params(which='major', axis='both', length=10., labelsize=18.)
ax[0].set_ylabel('$V\\ {\\rm(mag)}$', fontsize=20.)
ax[0].set_xlabel('$(B - V)_0$', fontsize=20.)
ax[1].set_xlabel('$(V - I_C)_0$', fontsize=20.)

ax[0].plot(ngc2516_bin[:,4], ngc2516_bin[:,3], 'o', markersize=7.0, c='#eeeeee')
ax[1].plot(ngc2516_bin[:,5], ngc2516_bin[:,3], 'o', markersize=7.0, c='#eeeeee')
ax[0].plot(ngc2516_sin[:,4], ngc2516_sin[:,3], 'o', markersize=7.0, c='#999999')
ax[1].plot(ngc2516_sin[:,5], ngc2516_sin[:,3], 'o', markersize=7.0, c='#999999')

fig.tight_layout()


I am working on the file "Projects/ngc2516_spots/cool_spot_colors.ipynb".
The first task is:
Compute colors for photospheric and spot components.
Can you write Python code for it?

phx_opt_phot = opt_surface(10**spot_prop[:, 1], spot_mags[:, 2])
phx_opt_spot = opt_surface(10**spot_prop[:, 2], spot_mags[:, 2])
phx_nir_phot = nir_surface(10**spot_prop[:, 1], spot_mags[:, 2])
phx_nir_spot = nir_surface(10**spot_prop[:, 2], spot_mags[:, 2])


I am working on the file "Projects/ngc2516_spots/cool_spot_colors.ipynb".
The first task is:
Compute colors using Phoenix BT-Settl models using CIFIST 2015 color tables. Colors were shown to be compatible with MARCS colors in another note.
Can you write Python code for it?

phx_opt_mags = opt_surface(10.0**iso[:, 1], iso[:, 2])
phx_nir_mags = nir_surface(10.0**iso[:, 1], iso[:, 2])


I am working on the file "Projects/ngc2516_spots/cool_spot_colors.ipynb".
The first task is:
Compute luminosity fractions for spots and photosphere for use in combining the two contributions.
Can you write Python code for it?


I am working on the file "Projects/ngc2516_spots/cool_spot_colors.ipynb".
The first task is:
Isochrone Comparisons
We may now compare morphologies of spotted isochrones computed using Phoenix and MARCS color tables.
Can you write Python code for it?


I am working on the file "Projects/ngc2516_spots/cool_spot_colors.ipynb".
The first task is:
Load Spotted Isochrone(s)
There are two types of spotted isochrones, one with magnitudes and colors with average surface properties, the other has more detailed information about spot temperatures and luminosities.
Can you write Python code for it?


I am working on the file "Projects/ngc2516_spots/cool_spot_colors.ipynb".
The first task is:
Now combine spot properties with the photospheric properties to derive properties for spotted stars.
Can you write Python code for it?

phx_opt_spot_mags = np.empty(phx_opt_phot.shape)
phx_nir_spot_mags = np.empty(phx_nir_phot.shape)

for i in range(phx_opt_phot.shape[1]):
    phx_opt_spot_mags[:,i] = -2.5*np.log10(0.6*10**(-phx_opt_phot[:,i]/2.5)
                                         + 0.4*10**(-phx_opt_spot[:,i]/2.5))

for i in range(phx_nir_phot.shape[1]):
    phx_nir_spot_mags[:,i] = -2.5*np.log10(0.6*10**(-phx_nir_phot[:,i]/2.5)
                                         + 0.4*10**(-phx_nir_spot[:,i]/2.5))


I am working on the file "Projects/ngc2516_spots/cool_spot_colors.ipynb".
The first task is:
Setting up Phoenix Grid Interpolation
First, load required color tables (1 optical, 1 NIR).
Can you write Python code for it?

phx_col_dir = '/Users/grefe950/Projects/starspot/starspot/color/tab/phx/CIFIST15'
opt_table = np.genfromtxt('{0}/colmag.BT-Settl.server.COUSINS.Vega'.format(phx_col_dir), comments='!')
nir_table = np.genfromtxt('{0}/colmag.BT-Settl.server.2MASS.Vega'.format(phx_col_dir), comments='!')


I am working on the file "Projects/ngc2516_spots/cool_spot_colors.ipynb".
The first task is:
Stack colors with stellar properties to form a new isochrone.
Can you write Python code for it?

phx_iso = np.column_stack((iso[:, :6], phx_opt_mags))  # stack props with BVRI
phx_iso = np.column_stack((phx_iso, phx_nir_mags))     # stack props/BVRI with JHK


I am working on the file "Projects/ngc2516_spots/cool_spot_colors.ipynb".
The first task is:
Stack with average surface properties to form a spotted isochrone.
Can you write Python code for it?

spt_iso = np.column_stack((spot_mags[:, :6], phx_opt_spot_mags))
spt_iso = np.column_stack((spt_iso, phx_nir_spot_mags))


I am working on the file "Projects/ngc2516_spots/ngc2516_vs_pleiades.ipynb".
The first task is:
Adopt literature values for reddening, neglecting differential reddening across the Pleiades.
Can you write Python code for it?

pl_dis = 5.61
pl_ebv = 0.034
pl_evi = 1.25*pl_ebv
pl_evk = 2.78*pl_ebv
pl_eik = pl_evk - pl_evi
pl_av  = 3.12*pl_ebv

ng_dis = 7.95
ng_ebv = 0.12
ng_evi = 1.25*ng_ebv
ng_evk = 2.78*ng_ebv
ng_eik = ng_evk - ng_evi
ng_av  = 3.12*ng_ebv


I am working on the file "Projects/ngc2516_spots/ngc2516_vs_pleiades.ipynb".
The first task is:
Import NGC 2516 low-mass star data.
Can you write Python code for it?

ngc2516 = np.genfromtxt('data/ngc2516_Christophe_v3.dat')                   # data for this study from J&J (2012)
irwin07 = np.genfromtxt('data/irwin2007.phot')                              # data from Irwin+ (2007)
jeffr01 = np.genfromtxt('data/jeff_2001.tsv', delimiter=';', comments='#')  # data from Jeffries+ (2001)
jeffr01 = np.array([star for star in jeffr01 if star[9] == 1])              # extract candidate members


I am working on the file "Projects/ngc2516_spots/ngc2516_vs_pleiades.ipynb".
The first task is:
Jackson et al. (2009) recommend a small correction to I-band magnitudes from Irwin et al. (2007) to place them on the same photometric scale as Jeffries et al. (2001), which they deem to be "better calibrated." Jackson & Jeffries (2012) suggest that the tabulated data (on Vizier) has been transformed to the "better calibrated" system. Key to understanding their results, however, is to also transform $(V-I_C)$ and then calculate a correction to $V$-band magnitudes, as well.
Can you write Python code for it?


I am working on the file "Projects/ngc2516_spots/ngc2516_vs_pleiades.ipynb".
The first task is:
Overlay the CMDs for each cluster, corrected for reddening and distance.
Can you write Python code for it?

fig, ax = plt.subplots(1, 2, figsize=(12., 8.), sharex=True, sharey=True)

for axis in ax:
    axis.grid(True)
    axis.tick_params(which='major', axis='both', length=15., labelsize=16.)
    axis.set_ylim(12., 5.)
    axis.set_xlim(0.5, 3.0)
    axis.set_xlabel('$(V - I_C)$', fontsize=20.)

ax[0].set_ylabel('$M_V$', fontsize=20.)

ax[0].plot(jeffr01[:,5] - ng_evi, jeffr01[:,3] - ng_av - ng_dis,
            'o', markersize=4.0, c='#555555', alpha=0.2)
ax[0].plot(irwin07[:, 7] - irwin07[:, 8] - ng_evi, irwin07[:, 7] - ng_av - ng_dis,
            'o', c='#1e90ff', markersize=4.0, alpha=0.6)
ax[0].plot(ngc2516[:, 1] - ngc2516[:, 2] - ng_evi, ngc2516[:, 1] - ng_av - ng_dis,
            'o', c='#555555', markersize=4.0, alpha=0.8)
ax[0].plot(iso_emp_k14[:, 2] - pl_evi, iso_emp_k14[:, 0] - pl_av - pl_dis,
               dashes=(20., 5.), lw=3, c='#b22222')

ax[1].plot(irwin07[:, 7] - irwin07[:, 8] - ng_evi, irwin07[:, 7] - ng_av - ng_dis,
            'o', c='#1e90ff', markersize=4.0, alpha=0.6)
ax[1].plot(ngc2516[:, 1] - ngc2516[:, 2] - ng_evi, ngc2516[:, 1] - ng_av - ng_dis,
            'o', c='#555555', markersize=4.0, alpha=0.6)
ax[1].plot(iso_emp_k14[:, 2] - pl_evi, iso_emp_k14[:, 0] - pl_av - pl_dis,
               dashes=(20., 5.), lw=3, c='#b22222')


I am working on the file "Projects/ngc2516_spots/ngc2516_vs_pleiades.ipynb".
The first task is:
The empirical isochrone from Kamai et al. (2014) agrees well with photometric data for NGC 2516, with both corrected for differential extinction. There may be some small disagreements at various locations along the sequence, but the morphology of the empirical isochrone is broadly consistent with NGC 2516. However, stars in NGC 2516 appear to have bluer $(B-V)$ colors for $(V-I_C) > 1.8$.

Exploring a transformation of the Irwin+ data from Johnson $(V-I)$ to Cousins $(V-I_C)$ from Bessell (1979). This is ignoring issues related to photometric calibrations, that Jackson & Jeffries posit is important. I'm not claiming JJ are wrong, just wondering how using a more standard photometric transformation would affect the resulting CMDs.
WAIT: The Irwin et al. (2007) data file suggests the I band photometry is quoted in terms of Cousins I, not Johnson... unlike what is stated in their paper.
Can you write Python code for it?

tmp_data = np.genfromtxt('data/irwin2007.phot')  # re-load Irwin et al. (2007) data into new array


I am working on the file "Projects/ngc2516_spots/ngc2516_vs_pleiades.ipynb".
The first task is:
While data in the $M_V/(V-I_C)$ CMD appears to be bluer for early M-dwarf stars and redder for later M-dwarf stars, we find that M-dwarfs in NGC 2516 appear to be generally bluer than low-mass stars in the Pleiades.
An interesting implication is that empirical isochornes based on the Pleiades or NGC 2516 may not reliably fit other clusters. Something is different between the two. Is it magnetic activity, or perhaps chemical composition?

$(B-V)/(V-I_C)$ color-color diagram using data from Jeffries et al. (2001) for NGC 2516.
Can you write Python code for it?

fig, ax = plt.subplots(1, 1, figsize=(6., 8.))

ax.grid(True)
ax.tick_params(which='major', axis='both', length=15., labelsize=16.)
ax.set_xlim(-0.5, 2.0)
ax.set_ylim( 0.0, 2.5)
ax.set_ylabel('$(V - I_C)$', fontsize=20.)
ax.set_xlabel('$(B - V)$', fontsize=20.)

ax.plot(jeffr01[:,4] - ng_ebv, jeffr01[:,5] - ng_evi,
            'o', markersize=4.0, c='#555555', alpha=0.2)
ax.plot(iso_emp_k14[:, 1] - pl_ebv, iso_emp_k14[:, 2] - pl_evi,
               dashes=(20., 5.), lw=3, c='#b22222')


I am working on the file "Projects/ngc2516_spots/ngc2516_vs_pleiades.ipynb".
The first task is:
While the Stauffer et al (2007) and Jackson et al. (2009) samples lie a bit redward of the median sequence in the Jeffries et al. (2001), the former two samples compare well against the empirical cluster sequence (shown as a red dashed line; Kamai et al. 2014) from the Pleiades in a $M_V/(V-I_C)$ CMD.
What about $M_V/(V-K)$ and $M_V/(I_C-K)$ CMDs?
Can you write Python code for it?

fig, ax = plt.subplots(1, 2, figsize=(12., 8.), sharey=True)

for axis in ax:
    axis.grid(True)
    axis.tick_params(which='major', axis='both', length=15., labelsize=16.)
    axis.set_ylim(12., 5.)

ax[0].set_xlim(1.0, 6.0)
ax[0].set_xlabel('$(V - K)$', fontsize=20.)
ax[0].set_ylabel('$M_V$', fontsize=20.)

# include K_CIT --> K_2mass correction for NGC 2516
ax[0].plot(ngc2516[:, 1] - ngc2516[:, 3] - 0.024 - ng_evk, ngc2516[:, 1] - ng_av - ng_dis,
            'o', c='#555555', markersize=4.0, alpha=0.6)
ax[0].plot(iso_emp_k14[:, 3] - pl_evk, iso_emp_k14[:, 0] - pl_av - pl_dis,
               dashes=(20., 5.), lw=3, c='#b22222')

ax[1].set_xlim(0.5, 3.0)
ax[1].set_xlabel('$(I_C - K)$', fontsize=20.)

ax[1].plot(ngc2516[:, 2] - ngc2516[:, 3] - 0.024 - ng_eik, ngc2516[:, 1] - ng_av - ng_dis,
            'o', c='#555555', markersize=4.0, alpha=0.6)
ax[1].plot(iso_emp_k14[:, 3] - iso_emp_k14[:, 2] - pl_eik, iso_emp_k14[:, 0] - pl_av - pl_dis,
               dashes=(20., 5.), lw=3, c='#b22222')


I am working on the file "Projects/ngc2516_spots/possible_binaries.ipynb".
The first task is:
A selection criterion of 0.2 mag above the zero point corresponds to, approximately, a $3\sigma$ threshold. We could be conservative (in the sense of testing models) and remove stars beginning below 0.1 mag. The impact on the resulting mean radii can be probed later. ~~For now, we'll stick with our initial 0.2 mag criterion.~~ There may be better reason to adopt 0.0 as the threshold. Need to review the binary sequence definition later in Irwin+ (2007).
How does this affect the appearance of color magnitude diagrams?
Can you write Python code for it?


I am working on the file "Projects/ngc2516_spots/possible_binaries.ipynb".
The first task is:
This has a strong affect on the CMDs of NGC 2516. Both display tighter sequences, as expected. Notably, removing binaries based on $V$-band magnitude removes suspected binaries flagged from visual inspection of the $I_C$-band magnitude.
How, then, are the radii (and mean radii) affected by the removal of binaries? First, radius-color figures.
Can you write Python code for it?

# load data with *OLD* mean values
mean_radii_VI = np.genfromtxt('data/ngc2516_avg_radii_V-I_v2.dat')
mean_radii_IK = np.genfromtxt('data/ngc2516_avg_radii_I-K_v2.dat')

fig, ax = plt.subplots(1, 2, figsize=(16.0, 8.0))

for axis in ax:
    axis.set_ylabel('Radius ($R_{\\odot}$)', fontsize=20.)
    axis.tick_params(which='major', axis='both', length=10., labelsize=16.)
ax[0].set_xlabel('$(V - I_C)$', fontsize=20.)
ax[1].set_xlabel('$(I_C - K)$', fontsize=20.)

ax[0].plot((ngc2516_bin[:,1] - ngc2516_bin[:,2]), ngc2516_bin[:,10], '*', lw=2, ms=12.0, c='#800000')
ax[0].plot((ngc2516_new[:,1] - ngc2516_new[:,2]), ngc2516_new[:,10], 'o', lw=2, ms=8.0,  c='#008B8B')
ax[0].plot(mean_radii_VI[:,0], mean_radii_VI[:,1], '-', lw=3, c = '#444444')
ax[0].fill_between(mean_radii_VI[:,0], mean_radii_VI[:,2], mean_radii_VI[:,3], edgecolor='#eeeeee', facecolor='#eeeeee')
ax[1].plot((ngc2516_bin[:,2] - ngc2516_bin[:,3]), ngc2516_bin[:,10], '*', lw=2, ms=12.0, c='#800000')
ax[1].plot((ngc2516_new[:,2] - ngc2516_new[:,3]), ngc2516_new[:,10], 'o', lw=2, ms=8.0,  c='#008B8B')
ax[1].plot(mean_radii_IK[:,0], mean_radii_IK[:,1], '-', lw=3, c = '#444444')
ax[1].fill_between(mean_radii_IK[:,0], mean_radii_IK[:,2], mean_radii_IK[:,3], edgecolor='#eeeeee', facecolor='#eeeeee')


I am working on the file "Projects/ngc2516_spots/possible_binaries.ipynb".
The first task is:
This is quite interesting. Binaries appear to have estimated radii consistent with those from the single star population. It's therefore possible that removing binaries will increase the mean radii as a function of both magnitude, since the stars with the largest radii are considered to be among the single star population.
Does it make sense that binaries lie on the smaller side? Recall,
\begin{equation}
    \frac{R\sin i}{R_{\odot}} = 0.124 \frac{P}{2\pi} v\sin i,
\end{equation}
so the radius is proportional to both the rotational velocity and the rotational period. At a given magnitude, one would expect the average measured $v\sin i$ of a binary to be greater than for single stars. That would tend to increase the measurement of $R\sin i$ for a fixed orbital period. But, if the star has a brighter magnitude than it would if the two stars are single, then the adopted template spectrum for the binary system may be too early of a spectral type, leading to intrinsically broader spectral lines. This would tend to produce lower $v\sin i$ values.
Can you write Python code for it?


I am working on the file "Projects/ngc2516_spots/sta_spots_lithium.ipynb".
The first task is:
And now stellar bolometric luminosity.
Can you write Python code for it?

fig, ax = plt.subplots(3, 2, figsize=(10., 12.), sharex=True, sharey=True)

# set x-axis labels
for i in range(2):
    ax[2, i].set_xlim(1., -1.5)
    ax[2, i].set_ylim(0.1, 3.6)
    ax[2, i].set_xlabel('$\\log (L/L_{\\odot})$', fontsize=20.)

# load standard isochrone
iso = loadIsochrone('../isochrone_120.0myr_z+0.00_a+0.00_marcs.iso')

# initial Li abundance
Li_init = np.array([[1., 3.3], [-3., 3.3]])

# left column, vary rho
for i in range(3):
    ax[i, 0].tick_params(which='major', axis='both', length=15., labelsize=16.)
    ax[i, 0].set_ylabel('A(Li)', fontsize=20.)

    ax[i, 0].plot(iso[:,3], iso[:,5], lw=3, dashes=(15., 5.), c=colors[3])
    ax[i, 0].plot(Li_init[:,0], Li_init[:,1], lw=2, dashes=(5.0, 5.0), c=colors[3])

    # compute zeta for (varpi, varrho); plot spotted isochrone
    zeta   = 1.0 - varrho*(1.0 - varpi[2-i]**4)
    for j in range(len(zeta)):
        filename = 'mag_zet{:+4.2f}_eps+1.00_rho{:+4.2f}_pi{:+4.2f}.dat'.format(zeta[j], varrho[j], varpi[2-i])
        line = '$\\varrho = ${:4.2f}, $\\varpi = ${:4.2f}'.format(varrho[j], varpi[2-i])
        spot_iso = loadIsochrone(filename)
        ax[i, 0].plot(spot_iso[:, 3], spot_iso[:, 5], lw=3, c=colors[2-j], label=line)

    ax[i, 0].legend(loc=3, fontsize=14.)
    ax[i, 0].grid(True)

# left column, vary varpi
for i in range(3):
    ax[i, 1].tick_params(which='major', axis='both', length=15., labelsize=16.)
    ax[i, 1].plot(iso[:,3], iso[:,5], lw=3, dashes=(15., 5.), c=colors[3])
    ax[i, 1].plot(Li_init[:,0], Li_init[:,1], lw=3, dashes=(5.0, 5.0), c=colors[3])

    # compute zeta for (varpi, varrho); plot spotted isochrone
    zeta   = 1.0 - varrho[i]*(1.0 - varpi**4)
    for j in range(len(zeta)):
        filename = 'mag_zet{:+4.2f}_eps+1.00_rho{:+4.2f}_pi{:+4.2f}.dat'.format(zeta[j], varrho[i], varpi[j])
        line = '$\\varrho = ${:4.2f}, $\\varpi = ${:4.2f}'.format(varrho[i], varpi[j])
        spot_iso = loadIsochrone(filename)
        ax[i, 1].plot(spot_iso[:, 3], spot_iso[:, 5], lw=3, c=colors[j],...


I am working on the file "Projects/ngc2516_spots/sta_spots_lithium.ipynb".
The first task is:
Lithium abundances are typically compared against either inferred stellar "effective" temperatures or against stellar bolometric luminosities. We'll begin with the "effective" temperature, which is taken to be the average surface temperature (as would presumably be derived from colors or SED fitting).
Can you write Python code for it?

#colors = {0: '#1F7676', 1:'#0473B3', 2: '#A21616', 3:'#555555'}
colors = {2:'#0094b2', 1:'#B22222', 0:'#56b4ea', 3:'#555555'}
varpi  = np.array([0.7, 0.8, 0.9])
varrho = np.array([0.3, 0.5, 0.7])

fig, ax = plt.subplots(3, 2, figsize=(10., 12.), sharex=True, sharey=True)

# set x-axis labels
for i in range(2):
    ax[2, i].set_xlim(6600., 4000.)
    ax[2, i].set_ylim(0.1, 3.6)
    ax[2, i].set_xlabel('Average Temperature', fontsize=20.)

# load standard isochrone
iso = loadIsochrone('../isochrone_120.0myr_z+0.00_a+0.00_marcs.iso')

# initial Li abundance
Li_init = np.array([[3000., 3.3], [8000., 3.3]])

# left column, vary rho
for i in range(3):
    ax[i, 0].tick_params(which='major', axis='both', length=15., labelsize=16.)
    ax[i, 0].set_ylabel('A(Li)', fontsize=20.)

    ax[i, 0].plot(10**iso[:,1], iso[:,5], lw=3, dashes=(15., 5.), c=colors[3])
    ax[i, 0].plot(Li_init[:,0], Li_init[:,1], lw=2, dashes=(5.0, 5.0), c=colors[3])

    # compute zeta for (varpi, varrho); plot spotted isochrone
    zeta   = 1.0 - varrho*(1.0 - varpi[2-i]**4)
    for j in range(len(zeta)):
        filename = 'mag_zet{:+4.2f}_eps+1.00_rho{:+4.2f}_pi{:+4.2f}.dat'.format(zeta[j], varrho[j], varpi[2-i])
        line = '$\\varrho = ${:4.2f}, $\\varpi = ${:4.2f}'.format(varrho[j], varpi[2-i])
        spot_iso = loadIsochrone(filename)
        ax[i, 0].plot(10**spot_iso[:, 1], spot_iso[:, 5], lw=3, c=colors[2-j], label=line)

    ax[i, 0].legend(loc=3, fontsize=14.)
    ax[i, 0].grid(True)

# left column, vary varpi
for i in range(3):
    ax[i, 1].tick_params(which='major', axis='both', length=15., labelsize=16.)
    ax[i, 1].plot(10**iso[:,1], iso[:,5], lw=3, dashes=(15., 5.), c=colors[3])
    ax[i, 1].plot(Li_init[:,0], Li_init[:,1], lw=2, dashes=(5.0, 5.0), c=colors[3])

    # compute zeta for (varpi, varrho); plot spotted isochrone
    zeta   = 1.0 - varrho[i]*(1.0 - varpi**4)
    for j in range(len(zeta)):
        filename = 'mag_zet{:+4.2f}_eps+1.00_rho{:+4.2f}_pi{:+4.2f}.dat'.format(zeta[j],...


I am working on the file "Projects/ngc2516_spots/sta_spots_simple_relations.ipynb".
The first task is:
First we can look at combinations of spot areal coverage and spot temperature contrast.
Can you write Python code for it?

def arealCoverage(zeta, varpis):
    return (1. - zeta)/(1. - varpis**4)

fig, ax = plt.subplots(1, 1, figsize=(6., 6.))

for zeta in zetas:
    ax.plot(varpi, arealCoverage(zeta, varpi), '-', lw=3, c='#0094b2')

ax.grid(True)
ax.set_xlim(0.0, 1.0)
ax.set_ylim(0.0, 1.0)
ax.set_xlabel('Temperature Contrast $\\varpi$', fontsize=16., family='serif')
ax.set_ylabel('Areal Coverage $\\varrho$', fontsize=16., family='serif')
ax.tick_params(which='major', axis='both', length=15., labelsize=16.)


I am working on the file "Projects/ngc2516_spots/sta_spots_simple_relations.ipynb".
The first task is:
For a range of stellar effective temperatures, this results in an average surface temperature anomaly (in K).
Can you write Python code for it?

Teffs = np.arange(3000., 7000.)

fig, ax = plt.subplots(1, 1, figsize=(8., 4.))

for zeta in zetas:
    ax.plot(Teffs, Teffs*(zeta**0.25 - 1.), '-', lw=3, c='#0094b2')

ax.grid(True)
#ax.set_xlim(0.0, 1.0)
#ax.set_ylim(0.0, 1.0)
ax.set_xlabel('Original Effective Temperature (K)', fontsize=16., family='serif')
ax.set_ylabel('Mean Temperature Anomaly (K)', fontsize=16., family='serif')
ax.tick_params(which='major', axis='both', length=15., labelsize=16.)


I am working on the file "Projects/ngc2516_spots/sta_spots_simple_relations.ipynb".
The first task is:
Moving on to the average surface temperature,
Can you write Python code for it?

zetas  = np.arange(0.0, 1.0, 0.01)

fig, ax = plt.subplots(1, 1, figsize=(6., 6.))

ax.plot(zetas, zetas**0.25, '-', lw=3, c='#0094b2')

ax.grid(True)
ax.set_xlim(0.0, 1.0)
ax.set_ylim(0.0, 1.0)
ax.set_xlabel('Luminosity Ratio $\\zeta$', fontsize=16., family='serif')
ax.set_ylabel('Average Surface Temperature Ratio', fontsize=16., family='serif')
ax.tick_params(which='major', axis='both', length=15., labelsize=16.)

zetas = np.arange(0.1, 1.0, 0.1)
def arealCoverage(zeta, varpis):
    return (1. - zeta**0.25)/(1. - varpis**4)

fig, ax = plt.subplots(1, 1, figsize=(6., 6.))

for zeta in zetas:
    ax.plot(varpi, arealCoverage(zeta, varpi), '-', lw=3, c='#0094b2')

ax.grid(True)
ax.set_xlim(0.0, 1.0)
ax.set_ylim(0.0, 1.0)
ax.set_xlabel('Temperature Contrast $\\varpi$', fontsize=16., family='serif')
ax.set_ylabel('Areal Coverage $\\varrho$', fontsize=16., family='serif')
ax.tick_params(which='major', axis='both', length=15., labelsize=16.)


I am working on the file "Projects/pleiades_colors/isochrone_interp.ipynb".
The first task is:
Again, check to confirm lengths are equal.
Can you write Python code for it?


I am working on the file "Projects/pleiades_colors/isochrone_interp.ipynb".
The first task is:
Check to confirm that isochrones are equal in length.
Can you write Python code for it?


I am working on the file "Projects/pleiades_colors/isochrone_interp.ipynb".
The first task is:
The tracks shift as they should, so the interpolation worked! Now, we can save the new isochrones and compute colors for them separately.
Can you write Python code for it?

np.savetxt('data/dmestar_00600.0myr_z+0.10_a+0.00_marcs.iso', iso_zp10, fmt='%14.8f')
np.savetxt('data/dmestar_00600.0myr_z+0.15_a+0.00_marcs.iso', iso_zp15, fmt='%14.8f')


I am working on the file "Projects/pleiades_colors/isochrone_interp.ipynb".
The first task is:
These should now be intermediate between the original two isochrones. Note that we've only interpolate fundamental properties, and not photometric magnitudes. These can be added separately as interpolating magnitudes in the isochrone may not be as accurate as interpolating bolometric corrections from the original data tables.
Can you write Python code for it?


I am working on the file "Projects/pleiades_colors/isochrone_interp.ipynb".
The first task is:
and then interpolating at two intermediate points.
Can you write Python code for it?

iso_zp10 = isoLinInterp(0.10)
iso_zp15 = isoLinInterp(0.15)


I am working on the file "Projects/pleiades_colors/pleiades_praesepe_comp.ipynb".
The first task is:
Already we see a hint that Pleiades stars are as blue as one might expect (on average) given the properties of the cluster. Similarly, the stars in Praesepe are just as red as expected. However, issues arise once one begins to directly compare K-dwarf stars in the two clusters. The empirical isochrone from Kamai et al. (2014) reproduces well the Pleiades locus and is barely visible under the model isochrone, lending credence to the quality of the model color predictions above $M_V \sim 8$. However, if we compare the Kamai empirical isochrone to stars in Praesepe, the Kamai isochrone is noticeably bluer between $6 < M_V < 8$, or the magnitude range covered largely by K-dwarf stars. Yet, in both cases the model isochrones reproduce the morpohology of the CMDs, differing only by their ages (insignificant) and their metallicity.
Since this analysis is relatively sensitive to reddening and distance modulii, we should demonstrate how CMDs look in other passbands and also, perhaps, in color-color diagrams.
Can you write Python code for it?

praesepe_a02 = np.genfromtxt('data/Adams_Praesepe.txt', usecols=(3, 4, 5, 6, 7, 8))  # 2MASS NIR photometry
praesepe_h99 = np.genfromtxt('data/Hodgkin_Praesepe.txt') # JHK (UKIRT?) photometry
praesepe_t08 = np.genfromtxt('data/Taylor_Praesepe.txt')  # VRI (Cousins) photometry

fig, ax = plt.subplots(1, 2, figsize=(12., 8.), sharex=True, sharey=True)

for axis in ax:
    axis.grid(True)
    axis.tick_params(which='major', axis='both', length=15., labelsize=16.)
    axis.set_ylim(10., 1.)
    axis.set_xlim(0.0, 1.5)
    axis.set_xlabel('$(J - K)$', fontsize=20.)

# PLEIADES
ax[0].set_title('Pleiades', fontsize=20., family='serif')
ax[0].set_ylabel('$M_K$', fontsize=20.)
ax[0].plot(pleiades_s07[:, 0] - pleiades_s07[:, 4] - pl_ejk, pleiades_s07[:, 4] - pl_ak - pl_dis,
            'o', c='#555555', markersize=4.0, alpha=0.2)
ax[0].plot(iso_120_gas07[:, 11] - iso_120_gas07[:, 13], iso_120_gas07[:, 13], lw=3, c='#0094b2')

# PRAESEPE
ax[1].set_title('Praesepe', fontsize=20., family='serif')
ax[1].plot(praesepe_a02[:, 0] - praesepe_a02[:, 4] - pr_ejk, praesepe_a02[:, 4] - pr_ak - pr_dis,
            'o', c='#555555', markersize=4.0, alpha=0.2)
ax[1].plot(praesepe_h99[:, 7] - pr_ejk, praesepe_h99[:, 4] - pr_ak - pr_dis,
            'o', c='#555555', markersize=4.0, alpha=0.2)
ax[1].plot(iso_600_gas07[:, 11] - iso_600_gas07[:, 13], iso_600_gas07[:, 13], lw=3, c='#0094b2')


I am working on the file "Projects/pleiades_colors/pleiades_praesepe_comp.ipynb".
The first task is:
Unfortunately, for Praesepe, there is not enough NIR data G and K dwarfs beyond the MgH bump. We can see from the Pleiades cluster that there are some issues with the observed K band magnitudes in the small region between the MgH bump and the formation of strong water absorption.
Can you write Python code for it?

fig, ax = plt.subplots(1, 2, figsize=(12., 8.), sharex=True, sharey=True)

for axis in ax:
    axis.grid(True)
    axis.tick_params(which='major', axis='both', length=15., labelsize=16.)
    axis.set_ylim(10., 1.)
    axis.set_xlim(0.0, 2.5)
    axis.set_xlabel('$(V - I_C)$', fontsize=20.)

# PLEIADES
ax[0].set_title('Pleiades', fontsize=20., family='serif')
ax[0].set_ylabel('$M_V$', fontsize=20.)
ax[0].plot(pleiades_s07[:, 7] - pleiades_s07[:, 8] - pl_evi, pleiades_s07[:, 7] - pl_av - pl_dis,
            'o', c='#555555', markersize=4.0, alpha=0.2)
ax[0].plot(pleiades_k14[:, 4] - pl_evi, pleiades_k14[:, 0] - pl_av - pl_dis,
            'o', c='#555555', markersize=4.0, alpha=0.2)
ax[0].plot(iso_emp_k14[:, 2] - pl_ebv, iso_emp_k14[:, 0] - pl_av - pl_dis,
               dashes=(20., 5.), lw=3, c='#B22222', alpha=0.8)
ax[0].plot(iso_120_gas07[:, 8] - iso_120_gas07[:, 10], iso_120_gas07[:, 8], lw=3, c='#0094b2')
ax[0].plot(iso_120_mixed[:, 7] - iso_120_mixed[:,  9], iso_120_mixed[:, 7], lw=3, c='#555555')

# PRAESEPE
ax[1].set_title('Praesepe', fontsize=20., family='serif')
ax[1].plot(praesepe_t08[:, 5] + praesepe_t08[:, 7] - pr_evi, praesepe_t08[:, 3] - pr_av - pr_dis,
            'o', c='#555555', markersize=4.0, alpha=0.2)
ax[1].plot(praesepe_j11[:, -1] - pr_evi, praesepe_j11[:, 0] - pr_av - pr_dis,
            'o', c='#555555', markersize=4.0, alpha=0.2)
ax[1].plot(iso_600_gas07[:, 8] - iso_600_gas07[:, 10], iso_600_gas07[:, 8], lw=3, c='#0094b2')


I am working on the file "Projects/project_pi_calculation_monte_carlo.ipynb".
The first task is:
Monte Carlo Simulation (with LCG where multiplier = 13, offset = 0 and modulus = 31)
Can you write Python code for it?

def monte_carlo_process_customized(toss):
    x0 = time.time()
    args = (x0, 13, 0, 31)
    hit = 0
    for i in range(toss):
        u1, *args = linear_congruential_generator(*args)
        u2, *args = linear_congruential_generator(*args)
        if pow(u1, 2) + pow(u2, 2) < 1.0:
            hit += 1
    return hit * 4.0 / toss

pi = monte_carlo_process_customized(2000000)
print('pi = %.10f, err = %.10f' %(pi, abs(pi - np.pi)))


I am working on the file "Projects/project_pi_calculation_monte_carlo.ipynb".
The first task is:
Monte Carlo Simulation (with Quasi-random numbers)¶
Can you write Python code for it?

def monte_carlo_process_quasi(toss):
    hit = 0
    px = halton(2, toss)
    py = halton(3, toss)
    for i in range(toss):
        u1 = px[i]
        u2 = py[i]
        if pow(u1, 2) + pow(u2, 2) < 1.0:
            hit += 1
    return hit * 4.0 / toss

pi = monte_carlo_process_quasi(2000000)
print('pi = %.10f, err = %.10f' %(pi, abs(pi - np.pi)))


I am working on the file "Projects/project_pi_calculation_monte_carlo.ipynb".
The first task is:
Necesaary Function For Monte Carlo Simulation
Can you write Python code for it?

def linear_congruential_generator(x, a, b, m):
    x = (a * x + b) % m
    u = x / m
    return u, x, a, b, m

def stdrand(x):
    return linear_congruential_generator(x, pow(7, 5), 0, pow(2, 31) - 1)[:2]

def halton(p, n):
    b = np.zeros(math.ceil(math.log(n + 1) / math.log(p)))
    u = np.zeros(n)
    for j in range(n):
        i = 0
        b[0] = b[0] + 1
        while b[i] > p - 1 + np.finfo(float).eps:
            b[i] = 0
            i += 1
            b[i] += 1
        u[j] = 0
        for k in range(1, b.size + 1):
            u[j] = u[j] + b[k-1] * pow(p, -k)
    return u


I am working on the file "Projects/senap/common_blocks.ipynb".
The first task is:
Check to make sure we've acheived what we set out to do.
Can you write Python code for it?


I am working on the file "Projects/senap/common_blocks.ipynb".
The first task is:
Check whether we've isolated common block names.
Can you write Python code for it?


I am working on the file "Projects/senap/common_blocks.ipynb".
The first task is:
That seems to be quite reasonable. Now for all keys,
Can you write Python code for it?

for key in common_block_names:
    print key.upper()
    for entry in marcs_common_blocks:
        if entry[2].lower().find(key) != -1:
            print "\t {:16s} on line: {:4s}".format(entry[0], entry[1])
        else:
            pass


I am working on the file "Projects/senap/common_blocks.ipynb".
The first task is:
Thre are clearly some issues related to programming styles. Most repeated occurrences are the result of the user "closing" the common block or by including multiple common blocks on a single line. Let's remove those with some brute force tactics.
Can you write Python code for it?

second_round_names = [entry[entry.rfind('/') + 1:] for entry in common_block_names if entry.rfind('/') != -1]

second_round_names


I am working on the file "Projects/senap/real_variables.ipynb".
The first task is:
An initial attempt to extract all instances of hard coded single precision real variables was done using grep and a regular expression,
bash
grep -i "[0-9].e.[0-9]" *.f > marcs_single_precision.txt
where -i flags the result to be case insensitive. Let's read in the result.
Can you write Python code for it?

single_precision_vars = [line.rstrip('\n') for line in fi.input('marcs_single_precision.txt')]


I am working on the file "Projects/senap/real_variables.ipynb".
The first task is:
Changing these can be done by a single person, but would perhaps benefit from multiple editors working on a subset of the declarations.
Can you write Python code for it?

print "There are {:.0f} lines that need editing.".format(len(single_precision_vars))


I am working on the file "Projects/senap/real_variables.ipynb".
The first task is:
Everything looks reasonble, but I know for a fact that scientific notation can also be declared in FORMAT statements. Therefore, it's necessary to remove these from the entry list.
Can you write Python code for it?

single_precision_vars = [line for line in single_precision_vars if line.lower().find('format') == -1]


I am working on the file "Projects/senap/real_variables.ipynb".
The first task is:
If divided equally, that means each of 3 people will need to check 26 separate subroutines. However, these should we weighted so that each person searches roughly the same number of routines and the same number of lines.
Can you write Python code for it?

person_1 = []
person_2 = []
person_3 = []
person_4 = []
for i, routine in enumerate(total_lines):
    if i % 4 == 0:
        person_1.append(routine)
    elif i % 4 == 1:
        person_2.append(routine)
    elif i % 4 == 2:
        person_3.append(routine)
    elif i % 4 == 3:
        person_4.append(routine)
    else:
        print "Whoops, misplaced", routine


I am working on the file "Projects/senap/real_variables.ipynb".
The first task is:
Make sure we haven't removed valid entries, as compared to the initial output.
Can you write Python code for it?


I am working on the file "Projects/senap/real_variables.ipynb".
The first task is:
Now, let's confirm that everything looks about equal.
Can you write Python code for it?

print len(person_1), len(person_2), len(person_3), len(person_4)


I am working on the file "Projects/senap/real_variables.ipynb".
The first task is:
So that didn't work out quite as well as I had anticipated. Certainly person 2 will have considerably more work cut out for them, of order 1400 lines of code.
Looking at the routines in each set, it's possible to even up the 1st and 3rd sets in terms of lines of code by switching one entry with approx. 150 lines of code.
Can you write Python code for it?

print person_1
print
print person_2
print
print person_3
print
print person_4

person_2.append(person_4.pop(-5))


I am working on the file "Projects/senap/real_variables.ipynb".
The first task is:
Take a quick look at the first few lines,
Can you write Python code for it?


I am working on the file "Projects/senap/real_variables.ipynb".
The first task is:
We can now organize the data by subroutine and output the lines on which a single precision real is expcted to occur. There will be false positives, but those can be checked by eye.
Can you write Python code for it?

file_stream = open('single_precision_index.txt', 'w')
routine = ''
for line in single_precision_vars:
    line = line.split(':')
    if line[0] == routine:
        file_stream.write('\t{:4s} :: {:50s}\n'.format(line[1].strip(), line[2].strip()))
    else:
        routine = line[0]
        file_stream.write('\n\n{:s}\n\n'.format(line[0].rstrip('.f').upper()))
        file_stream.write('\t{:4s} :: {:70s}\n'.format(line[1].strip(), line[2].strip()))
file_stream.close()


I am working on the file "Projects/senap/real_variables.ipynb".
The first task is:
We've now combined the total number of lines in each file with the number of edits to fixed single precision constants that need to be made. Of course, there are instances where function calls need to be verified, but one expects this is somewhat proportional to the number of lines in the entire file.
Can you write Python code for it?

print "\t There are {:.0f} subroutines that need checking.".format(len(total_lines))


I am working on the file "Projects/senap/real_variables.ipynb".
The first task is:
Which means that, split among 3 people, it's approximately 96 lines of code per person. There are some routines that contain significantly more lines that require editing, as compared to others. We can create a histogram showing number of lines per subroutine.
Can you write Python code for it?

N_lines = []
routine = ''
n = 0
for line in single_precision_vars:
    line = line.split(':')
    if line[0] == routine:
        n += 1
    else:
        # output previous routine count
        N_lines.append([routine, n])
        n = 0

        # start a new routine
        routine = line[0]
        n += 1
N_lines.pop(0)


I am working on the file "Projects/senap/real_variables.ipynb".
The first task is:
and the number of lines of code
Can you write Python code for it?

for person in [person_1, person_2, person_3, person_4]:
    print sum([x[1] for x in person])


I am working on the file "Projects/upper_sco_age/apsidal_motion_age.ipynb".
The first task is:
Now, compute the $c_2$ coefficients for each age.
Can you write Python code for it?

mean_trk_B = np.empty((len(ages), 3))
for i, age in enumerate(ages):
    c2s = c2(masses, [10**new_trk_Ba[i, 4], 10**new_trk_Bb[i, 4]], e, a,
             rotation='synchronized')
    avg_k2 = (c2s[0]*new_trk_Ba[i, 10] + c2s[1]*new_trk_Bb[i, 10])/(sum(c2s))

    mean_trk_B[i] = np.array([age, 10**new_trk_Ba[i, 4] + 10**new_trk_Bb[i, 4], avg_k2])


I am working on the file "Projects/upper_sco_age/apsidal_motion_age.ipynb".
The first task is:
One can see from this that the general relativistic component is a very small contribution to the total apsidal motion of the system.
Let's look at the evolution of the apsidal motion for the A/B binary system.
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt

fig, ax = plt.subplots(1, 1, figsize=(8., 8.), sharex=True)

ax.grid(True)
ax.tick_params(which='major', axis='both', length=15., labelsize=18.)
ax.set_xlabel('Age (yr)', fontsize=20., family='serif')
ax.set_ylabel('Apsidal Motion Rate (deg / cycle)', fontsize=20., family='serif')
ax.plot([1.0e6, 1.0e8], [0.0215, 0.0215], '--', lw=1, c='#555555')
ax.plot([1.0e6, 1.0e8], [0.0255, 0.0255], '--', lw=1, c='#555555')
ax.semilogx(cl_apsidal_motion_rate[:, 0], cl_apsidal_motion_rate[:, 1], '-', lw=2, c='#b22222')


I am working on the file "Projects/upper_sco_age/apsidal_motion_age.ipynb".
The first task is:
We are now in a position to compute the classical apsidal motion rate from the combined A/B tracks.
Can you write Python code for it?

cl_apsidal_motion_rate = np.empty((len(ages), 2))
for i, age in enumerate(ages):
    c2_AB = c2(masses_2, [10**new_trk_A[i, 4], a + 0.5*mean_trk_B[i, 1]], e2, a2)
    cl_apsidal_motion_rate[i] = np.array([age, 360.0*(c2_AB[0]*new_trk_A[i, 10] + c2_AB[1]*mean_trk_B[i, 2])])

GR_apsidal_motion_rate = 5.45e-4*(sum(masses)/33.945)**(2./3.) / (1.0 - e2**2)  # Giménez (1985)

GR_apsidal_motion_rate


I am working on the file "Projects/upper_sco_age/apsidal_motion_age.ipynb".
The first task is:
What complicates the issue is that the interior structure constants for the B components also vary as a function of age, so we need to compute a mean mass track using the $c_2$ coefficients and the individual $k_2$ values.
Can you write Python code for it?

import numpy as np

trk_Ba = np.genfromtxt('/Users/grefe950/evolve/dmestar/trk/gs98/p000/a0/amlt1884/m0980_GS98_p000_p0_y28_mlt1.884.trk')
trk_Bb = np.genfromtxt('/Users/grefe950/evolve/dmestar/trk/gs98/p000/a0/amlt1884/m0940_GS98_p000_p0_y28_mlt1.884.trk')


I am working on the file "Projects/upper_sco_age/apsidal_motion_age.ipynb".
The first task is:
With that, we have an estimate for the mean B component properties as a function of age. One complicating factor is the "radius" of the average B component. If we are modeling the potential created by the Ba/Bb components as that of a single star, we need to assume that the A component never enters into any region of the combined potential that is dominated by either component.
Unfortunately, it is very likely that the ratio of the Ba/Bb binary "radius" to the semi-major axis of the A/B orbit is going to be a dominant contributor to the apsidal motion.
Attempt 1: Semi-major axis + radius of B component
Let's define orbtial properties of the (A, B) system.
Can you write Python code for it?


I am working on the file "Projects/upper_sco_age/isochrone_comparison.ipynb".
The first task is:
Now we need to place each onto a fixed grid so that we can analyze the relative change of the magnetic track to the standard track.
Can you write Python code for it?

from scipy.interpolate import interp1d
log10_ages = np.arange(6.0, 8.01, 0.01)

def setEqualAgeGrid(iso):
    icurve = interp1d(iso[:,0], iso, axis=0, kind='linear')
    return icurve(10**np.arange(6.0, 8.01, 0.01))

# Compare relative radii as a function of age
fig, ax = plt.subplots(3, 1, figsize=(6.0, 12.0))

ages = 10**log10_ages

std_trk = setEqualAgeGrid(std_m0900)
mag_trk = setEqualAgeGrid(mag_m0900)
ax[0].semilogx(ages, 10**mag_trk[:,4]/10**std_trk[:,4] - 1.0, '-',  lw=3)
ax[1].semilogx(ages, 10**mag_trk[:,1]/10**std_trk[:,1] - 1.0, '-',  lw=3)
ax[2].semilogx(ages, 10**mag_trk[:,3]/10**std_trk[:,3] - 1.0, '-',  lw=3)

std_trk = setEqualAgeGrid(std_m1200)
mag_trk = setEqualAgeGrid(mag_m1200)
ax[0].semilogx(ages, 10**mag_trk[:,4]/10**std_trk[:,4] - 1.0, '--', lw=3)
ax[1].semilogx(ages, 10**mag_trk[:,1]/10**std_trk[:,1] - 1.0, '--',  lw=3)
ax[2].semilogx(ages, 10**mag_trk[:,3]/10**std_trk[:,3] - 1.0, '--',  lw=3)

std_trk = setEqualAgeGrid(std_m1500)
mag_trk = setEqualAgeGrid(mag_m1500)
ax[0].semilogx(ages, 10**mag_trk[:,4]/10**std_trk[:,4] - 1.0, '-.', lw=3)
ax[1].semilogx(ages, 10**mag_trk[:,1]/10**std_trk[:,1] - 1.0, '-.',  lw=3)
ax[2].semilogx(ages, 10**mag_trk[:,3]/10**std_trk[:,3] - 1.0, '-.',  lw=3)

# Compare evolution
fig, ax = plt.subplots(3, 1, figsize=(6.0, 12.0))

ages = 10**log10_ages

std_trk = setEqualAgeGrid(std_m0900)
mag_trk = setEqualAgeGrid(mag_m0900)
ax[0].semilogx(ages, 10**mag_trk[:,4], '-',  lw=3, c='b')
ax[0].semilogx(ages, 10**std_trk[:,4], '--', lw=3, c='b')
ax[1].semilogx(ages, 10**mag_trk[:,1], '-',  lw=3, c='b')
ax[1].semilogx(ages, 10**std_trk[:,1], '--', lw=3, c='b')
ax[2].semilogx(ages, 10**mag_trk[:,3], '-',  lw=3, c='b')
ax[2].semilogx(ages, 10**std_trk[:,3], '--', lw=3, c='b')

std_trk = setEqualAgeGrid(std_m1200)
mag_trk = setEqualAgeGrid(mag_m1200)
ax[0].semilogx(ages, 10**mag_trk[:,4], '-',  lw=3, c='r')
ax[0].semilogx(ages, 10**std_trk[:,4], '--', lw=3, c='r')
ax[1].semilogx(ages, 10**mag_trk[:,1], '-',  lw=3, c='r')
ax[1].semilogx(ages, 10**std_trk[:,1], '--',...


I am working on the file "Projekt_knjige.ipynb".
The first task is:
Analiza podatkov:
Ocene knjig po stoletjih:
Najprej bomo analizirali podatke o knjigah po stoletjih. Ogledali si bomo povprečne ocene glasovanja za najboljšo knjigo stoletja. Najprej dodamo v našo tabelo še en stolpec, v katerem dodamo stoletje v katerem je bila knjiga izdana.
Spodaj vidimo prikaz te tabele.
Poleg tega izračunamo še povprečne ocene knjig po stoletjih. Najprej jih prikažemo, potem pa s pomočjo njih narišemo še graf.
Can you write Python code for it?

stoletja['Stoletje'] = (stoletja.Leto_izdaje // 100) + 1
stoletja

povprecje = stoletja.groupby('Stoletje').mean()
povprecje['Ocena']

%matplotlib inline
povprecje['Ocena'].plot(title = 'Povprečna ocena knjig posameznega stoletja:').set_ylabel('Ocena')


I am working on the file "Projekt_knjige.ipynb".
The first task is:
Sedaj si bomo ogledali še ocene knjig, ki so prejele nagrado in se uvrstile na top lestvico stoletja.
Can you write Python code for it?

skupni['Ocena'].plot(kind='bar',title = 'Ocene zmagovalnih knjig:').set_ylabel('Ocena')


I am working on the file "Projekt_knjige.ipynb".
The first task is:
Večina knjig, ki je dosegla prvo mesto na podelitvi nagrad, ni prišla v izbor prvih 100 knjig stoletja. Tu se vidi različna merila med sodniki na podelitvi nagrad in med vsakdanjimi bralci, ki glasujejo na spletni strani.
V naslednji tabeli vidimo, da je le 8 knjig, ki imajo visoko uvrstitev na glasovanju in so dosegle prvo mesto na podelitvi nagrad Choice award.
Can you write Python code for it?

vsi = pd.merge(nagrade, stoletja,left_index='Naslov', right_index='Naslov', how ='outer')
skupni = vsi.dropna(axis=0, how='any')
skupni


I am working on the file "Projeto 1 - CD.ipynb".
The first task is:
ANÁLISE INICIAL E NOVA FILTRAGEM:
COM A PRECISÃO DOS VALORES MOSTRADOS ACIMA, PODE-SE OBSERVAR ONDE HÁ MAIOR CONCENTRAÇÃO EM CADA UMA DAS REGIÕES DE INTERESSE DE ACORDO COM A DISPARIDADE ANALISADA ANTERIORAMENTE NOS GRÁFICOS. DESSA FORMA A PARTIR DE AGORA A ANÁLISE SE CENTRARÁ APENAS ÀQUELES QUE PASSARAM POR SITUACÃO DE INSEGURANÇA ABRINDO PARA UMA NOVA VARIÁVEL, CHAMADA DE PROBLEMA ALIMENTAR E PAUTADA EM PERGUNTAS QUE DEMONSTRAM FALTA DE COMIDA OU ALIMENTAÇÃO RESTRITA POR CONTA DE FALTA DE DINHEIRO.
Can you write Python code for it?


I am working on the file "Projeto 1 - CD.ipynb".
The first task is:
APROFUNDAMENTO NAS REGIÕES:
GRÁFICO DE FREQUÊNCIA SEGUIDO DE UMA TABELA QUE POTENCIALIZA A ANÁLISE DOS VALORES, JÁ QUE MOSTRA OS VALORES ABSOLUTOS E VISA BUSCAR MAIOR COMPREENSÃO E COERÊNCIA DOS VALORES.
Can you write Python code for it?

tb1= (pd.crosstab(base.REGIAO,base.Insegurança_Alimentar,margins=True,rownames=["REGIÃO"],colnames=["Insegurança Alimentar"],normalize='index')*100).round(decimals=1)
plot = tb1.plot(kind="bar",title="Distribuição Regional de Insegurança Alimentar 2013 (G3)")

abs1=pd.crosstab(base.REGIAO,base.Insegurança_Alimentar, margins=True, rownames=['REGIÃO'], colnames=['INSEGURANÇA ALIMENTAR'])
abs1=abs1.loc[['NORTE','NORDESTE','SUDESTE','SUL','CENTRO-OESTE']]
abs1


I am working on the file "Projeto 1 - CD.ipynb".
The first task is:
CRIACÃO DA VARIÁVEL INSEGURANÇA ALIMENTAR:
A SEGUIR MODIFICA-SE AS VARIÁVEIS (PERGUNTAS SOBRE INSEGURANÇA ALIMENTAR) CRIANDO UMA ÚNICA CHAMADA "INSEGURANÇA ALIMENTAR". O MOTIVO PARA ISSO É QUE AS 4 PERGUNTAS FEITAS REPRESENTAM SITUAÇÕES DE DIFICULDADE PARA SE ALIMENTAR, PORTANTO PARA SE CONSIDERAR UMA PESSOA QUE PASSOU POR SITUAÇÃO DE DIFICULDADE ALIMENTAR DEVE SE TER PELO MENOS UMA PERGUNTA RESPONDIDA COM "SIM". HÁ AINDA A CARACTERIZACAO PARA CATEGORIA DAS 4 PERGUNTAS.
Can you write Python code for it?

base.loc[(base.V2103==1) | (base.V2105==1) | (base.V2107==1) | (base.V2109==1),'Insegurança_Alimentar'] = 'Sim'
base.loc[(base.V2103==3) & (base.V2105==3) & (base.V2107==3) & (base.V2109==3),'Insegurança_Alimentar'] = 'Não'
base.V2103=base.V2103.astype("category")
base.V2105=base.V2105.astype("category")
base.V2107=base.V2107.astype("category")
base.V2109=base.V2109.astype("category")

base9.loc[(base9.V2103==1) | (base9.V2105==1) | (base9.V2107==1) | (base9.V2109==1),'Insegurança_Alimentar'] = 'Sim'
base9.loc[(base9.V2103==3) & (base9.V2105==3) & (base9.V2107==3) & (base9.V2109==3),'Insegurança_Alimentar'] = 'Não'
base9.V2103=base9.V2103.astype("category")
base9.V2105=base9.V2105.astype("category")
base9.V2107=base9.V2107.astype("category")
base9.V2109=base9.V2109.astype("category")


I am working on the file "Projeto 1 - CD.ipynb".
The first task is:
Caracterização dos problemas alimentares:
Os próximos gráficos tem como objetivo avaliar, além do comportamento da variável "problema alimentar" de acordo com a renda mensal familiar comparar com a distribuição de "insegurança alimentar" ou seja se a distribuição analisada anteriormente se mantém de certa maneira nessa variável que por sinal é dependente da inicial, "insegurança alimentar".
Can you write Python code for it?

frenda3 = pd.cut(base.RENDA[(base.Problema_Alimentar=='Sim')&(base.REGIAO=="NORTE")], bins=faixa, right=False)
t3 = (frenda3.value_counts(sort=False, normalize=True)*100).round(decimals=1)
print(t3,"\n")

plot = base.RENDA[(base.Problema_Alimentar=='Sim')&(base.REGIAO=="NORTE")].plot.hist(bins=faixa,title="Problema Alimentar - NORTE - 2013 (H5)", weights=zeros_like(base.RENDA[(base.Problema_Alimentar=='Sim')&(base.REGIAO=="NORTE")])+1./base.RENDA[(base.Problema_Alimentar=='Sim')&(base.REGIAO=="NORTE")].size*100, figsize=(6, 6), alpha=0.5,color="purple")
plt.ylabel('Frequência relativa (em %)')
plt.xlabel('Renda (em reais)')
plt.show()

frenda4 = pd.cut(base.RENDA[(base.Problema_Alimentar=='Sim')&(base.REGIAO=="NORDESTE")], bins=faixa, right=False)
t4 = (frenda4.value_counts(sort=False, normalize=True)*100).round(decimals=1)
print(t4,"\n")

plot = base.RENDA[(base.Problema_Alimentar=='Sim')&(base.REGIAO=="NORDESTE")].plot.hist(bins=faixa,title="Problema Alimentar - NORDESTE - 2013(H6)", weights=zeros_like(base.RENDA[(base.Problema_Alimentar=='Sim')&(base.REGIAO=="NORDESTE")])+1./base.RENDA[(base.Problema_Alimentar=='Sim')&(base.REGIAO=="NORDESTE")].size*100, figsize=(6, 6), alpha=0.5,color="darkgreen")
plt.ylabel('Frequência relativa (em %)')
plt.xlabel('Renda (em reais)')
plt.show()

frenda39 = pd.cut(base9.RENDA[(base9.Problema_Alimentar=='Sim')&(base.REGIAO=="CENTRO-OESTE")], bins=faixa, right=False)
t39 = (frenda39.value_counts(sort=False, normalize=True)*100).round(decimals=1)
print(t39,"\n")

plot = base9.RENDA[(base9.Problema_Alimentar=='Sim')&(base9.REGIAO=="CENTRO-OESTE")].plot.hist(bins=faixa,title="Problema Alimentar - CENTRO-OESTE - 2009(H7)", weights=zeros_like(base9.RENDA[(base9.Problema_Alimentar=='Sim')&(base9.REGIAO=="CENTRO-OESTE")])+1./base9.RENDA[(base9.Problema_Alimentar=='Sim')&(base9.REGIAO=="CENTRO-OESTE")].size*100, figsize=(6, 6), alpha=0.5,color="black")
plt.ylabel('Frequência relativa (em %)')
plt.xlabel('Renda (em reais)')
plt.show()

frenda49 =...


I am working on the file "Projeto 1 - CD.ipynb".
The first task is:
DEFINIÇÃO DAS REGIÕES E TRANSFORMAÇÃO EM UMA CATEGORIA;
Can you write Python code for it?


I am working on the file "Projeto 1 - CD.ipynb".
The first task is:
DIVISÃO EM ZONA RURAL E URBANA, A SEGUNDA VARIÁVEL DE ANÁLISE
Can you write Python code for it?


I am working on the file "Projeto 1 - CD.ipynb".
The first task is:
MUDANÇA DA VARIÁVEL INICIAL QUE MOSTRA O ANO DE PESQUISA.
Can you write Python code for it?


I am working on the file "Projeto 1 - CD.ipynb".
The first task is:
PRIMEIRA OBSERVAÇÃO:
OCORRÊNCIA DE PESSOAS QUE JÁ PASSARAM POR SITUAÇÕES DE INSEGURANÇA ALIMENTAR ("Sim") PARA POSTERIORMENTE ANALISAR AINDA A DIFERENÇA ENTRE AS REGIÕES E ZONAS.
Can you write Python code for it?

g1 = (base.Insegurança_Alimentar.value_counts(sort=False, normalize=True)*100).round(decimals=1)
plot = g1.plot(kind='bar',title='DIFICULDADE ALIMENTAR 2013 (G1)',figsize=(5, 5),color=('b','g'))
print(g1,"\n")

g2 = (base9.Insegurança_Alimentar.value_counts(sort=False, normalize=True)*100).round(decimals=1)
plot = g2.plot(kind='bar',title='DIFICULDADE ALIMENTAR 2009 (G2)',figsize=(5, 5),color=('b','g'))
print(g2,"\n")


I am working on the file "Projeto 1 - CD.ipynb".
The first task is:
TABELA 2 - 2013
Can you write Python code for it?

writer = pd.ExcelWriter('Tabela2-2013.xlsx',engine='xlsxwriter')
base.to_excel(writer,sheet_name="Projeto_1")
writer.save()

base9=base9[(base9.Insegurança_Alimentar=="Sim")]
base9


I am working on the file "Projeto.ipynb".
The first task is:
Excluding rows that are not needed
Can you write Python code for it?

excludes = [
 't1_ban1',
 't1_ban2',
 't1_ban3',
 't1_ban4',
 't1_ban5',
 't1_champ1_sum1',
 't1_champ1_sum2',
 't1_champ1id',
 't1_champ2_sum1',
 't1_champ2_sum2',
 't1_champ2id',
 't1_champ3_sum1',
 't1_champ3_sum2',
 't1_champ3id',
 't1_champ4_sum1',
 't1_champ4_sum2',
 't1_champ4id',
 't1_champ5_sum1',
 't1_champ5_sum2',
 't1_champ5id',
 't2_ban1',
 't2_ban2',
 't2_ban3',
 't2_ban4',
 't2_ban5',
 't2_champ1_sum1',
 't2_champ1_sum2',
 't2_champ1id',
 't2_champ2_sum1',
 't2_champ2_sum2',
 't2_champ2id',
 't2_champ3_sum1',
 't2_champ3_sum2',
 't2_champ3id',
 't2_champ4_sum1',
 't2_champ4_sum2',
 't2_champ4id',
 't2_champ5_sum1',
 't2_champ5_sum2',
 't2_champ5id']

for exclude in excludes:
    df = df.drop(exclude)

print(df.columns)

df.printSchema()

df.dtypes

df.select('gameId','t1_inhibitorKills','t2_towerKills','winner').show(15)


I am working on the file "Projeto.ipynb".
The first task is:
Inserting necessary fields insed features
Can you write Python code for it?

feat_fields = ['gameDuration',
              'seasonId',
              'firstBlood',
              'firstTower',
              'firstInhibitor',
              'firstBaron',
              'firstDragon',
              'firstRiftHerald',
              't1_towerKills',
              't1_inhibitorKills',
              't2_baronKills',
              't1_dragonKills',
              't1_riftHeraldKills',
              't2_towerKills',
              't2_inhibitorKills',
              't2_baronKills',
              't2_dragonKills',
              't2_riftHeraldKills']

assembler = VectorAssembler(inputCols=feat_fields, outputCol="features")
output = assembler.transform(df)

# The df will contain all the old Coluns and a new one features
# which will contain features we want
output.select('gameDuration','seasonId', 'features').show(20)


I am working on the file "Projeto.ipynb".
The first task is:
Separating features from target

The LogisticRegression function needs two Colums in df - label and features. (https://stackoverflow.com/questions/44475917/how-does-an-mllib-estimator-know-what-are-the-features-and-target-columns)
Features are the variables we are gonna use to predict the target variable
First, winner is gonna be the label as we want to know the winner
After, all the other info are gonna be the target
Can you write Python code for it?

# Renaming winner to label
df = df.withColumnRenamed("winner","label")
df.printSchema()


I am working on the file "Projeto.ipynb".
The first task is:
Split our data into training set and test set
Can you write Python code for it?

(trainingData, testData) = output.randomSplit([0.8, 0.2], seed = 1234)
print("Training Dataset Count: " + str(trainingData.count()))
print("Test Dataset Count: " + str(testData.count()))


I am working on the file "Projeto/.ipynb_checkpoints/pagerank-webgoogle-checkpoint.ipynb".
The first task is:
Aplicação do PageRank na Base da Dados Real
Após o teste, vamos aplicar o algoritmo de PageRank na base de dados de referências de páginas da web (arquivo web-Google.txt) encontrada no site da Stanford.
Can you write Python code for it?

diretorio_base = os.path.join('data')
caminho = os.path.join('web-Google.txt')
arquivo = os.path.join(diretorio_base, caminho)

linksGoogle = sc.textFile(arquivo).filter(lambda x: "#" not in x).map(lambda x: x.split("\t"))

linksAgrupados = linksGoogle.groupByKey().cache()

ranks = linksAgrupados.map(lambda url_agrupados: (url_agrupados[0], 1.0))

for x in range(1,8):
    # Adiciona ranks inicializados com 1.0 na posição [1][1] da matriz
    agrupaIdLinkComRank = linksAgrupados.join(ranks)\
                .flatMap(lambda url_rank: atualizaRank(url_rank[1][0], url_rank[1][1]))
    # Soma os valores com o mesmo id e adiciona o fator de normalização
    ranks = agrupaIdLinkComRank.reduceByKey(add)\
                                .mapValues(lambda rankFatorD: (rankFatorD * 0.85) + 0.15)


I am working on the file "Projeto/.ipynb_checkpoints/pagerank-webgoogle-checkpoint.ipynb".
The first task is:
Resultado do Teste:
Can you write Python code for it?

for (link, rank) in ranks_teste.sortBy(lambda x:-x[1]).take(3):
    print("ID: %s  Ranking: %s." % (link, rank))


I am working on the file "Projeto/pagerank-webgoogle.ipynb".
The first task is:
Base de Teste
Can you write Python code for it?

example = pd.read_csv("data/web-Google-test.txt",sep = "\t", names=['from','to'])
example

G = nx.from_pandas_dataframe(example, 'from', 'to',create_using=nx.DiGraph())

import matplotlib as mp
%matplotlib inline


I am working on the file "Projeto/pagerank-webgoogle.ipynb".
The first task is:
Grafo (Relacionamento dos Vértices)
Can you write Python code for it?

import matplotlib.pyplot as plt
nx.draw_networkx(G, node_color = 'lightgreen', node_size = 1000,arrows=True)
plt.savefig('pictures/graph_example.png')

from operator import add
sc = SparkContext.getOrCreate()

diretorio_base = os.path.join('data')
caminho_teste = os.path.join('web-Google-test.txt')
arquivo_teste = os.path.join(diretorio_base, caminho_teste)

def atualizaRank(listaUrls, rank):
    num_urls = len(listaUrls)
    rankAtualizado = []
    for x in listaUrls:
        rankAtualizado.append((x, (rank / num_urls)))
    return rankAtualizado

linksGoogle_teste = sc.textFile(arquivo_teste).filter(lambda x: "#" not in x).map(lambda x: x.split("\t"))

linksAgrupados_teste = linksGoogle_teste.groupByKey().cache()
#print(linksAgrupados.take(1))
#for it in linksAgrupados.take(1)[0][1]:
#  print(it)

ranks_teste = linksAgrupados_teste.map(lambda url_agrupados: (url_agrupados[0], 1.0))

for x in range(1,2):
    # Adiciona ranks inicializados com 1.0 na posição [1][1] da matriz
    agrupaIdLinkComRank_teste = linksAgrupados_teste .join(ranks_teste)\
                .flatMap(lambda url_rank: atualizaRank(url_rank[1][0], url_rank[1][1]))
    # Soma os valores com o mesmo id e adiciona o fator de normalização
    ranks_teste = agrupaIdLinkComRank_teste.reduceByKey(add)\
                                .mapValues(lambda rankFatorD: (rankFatorD * 0.85) + 0.15)


I am working on the file "Projeto/pagerank-webgoogle.ipynb".
The first task is:
Resultado das 10 primeias páginas mais relevantes:
Can you write Python code for it?

for (link, rank) in ranks.sortBy(lambda x:-x[1]).take(10):
    print("ID: %s  Ranking: %s." % (link, rank))


I am working on the file "Projeto02.ipynb".
The first task is:
Calculando as Probabilidade da Relevância dos Tweets:
Can you write Python code for it?

probSIM = []
l = 1
for i in range(len(excelte.Teste)):
    clinha = []
    for k in range(len(excelte.Teste[i])):
        chance = 0
        for j in range(len(LS)):
            if LS[j][0] == excelte.Teste[i][k]:
                chance = ((LS[j][1]+1)/(len(LS)+(len(LS)+len(LN))))
                break
        if chance > 0:
            clinha.append(chance)
        elif chance == 0:
            clinha.append(1/(len(LS)+(len(LS)+len(LN))))
    l = 1
    for x in clinha:
        l *= x
    probSIM.append(l)

probNAO = []
l = 1
for i in range(len(excelte.Teste)):
    clinha = []
    for k in range(len(excelte.Teste[i])):
        chance = 0
        for j in range(len(LN)):
            if LN[j][0] == excelte.Teste[i][k]:
                chance = ((LN[j][1]+1)/(len(LN)+(len(LS)+len(LN))))
                break
        if chance > 0:
            clinha.append(chance)
        elif chance == 0:
            clinha.append(1/(len(LN)+(len(LS)+len(LN))))
    l = 1
    for x in clinha:
        l *= x
    probNAO.append(l)


I am working on the file "Projeto02.ipynb".
The first task is:
Coletando Dados
Agora vamos coletar os dados. Tenha em mente que dependendo do produto escolhido, não haverá uma quantidade significativa de mensagens, ou ainda poder haver muitos retweets.

Configurando:
Can you write Python code for it?

#Produto escolhido:
produto = 'itau'

#Quantidade mínima de mensagens capturadas:
n = 500
#Quantidade mínima de mensagens para a base de treinamento:
t = 300

#Filtro de língua, escolha uma na tabela ISO 639-1.
lang = 'pt'


I am working on the file "Projeto02.ipynb".
The first task is:
Cálculos e contagem de palavras:
Can you write Python code for it?

for i in exceltr.Relevancia:
    if i == 'sim':
        listaprobs.append(i)
    if i == 'não':
        listaprobs.append(i)
QY = 0
QN = 0
for a in listaprobs:
    if a == 'sim':
        QY += 1
    if a == 'não':
        QN += 1

#Conta cada palavra da lista
LS = [[x,SIMs.count(x)] for x in set(SIMs)]
LN = [[y,NAOs.count(y)] for y in set(NAOs)]

#Calcula quantas palavras existem no espaço amostral
palav = 0
sins = 0
naos = 0
for a in range(len(LS)):
    palav = palav + LS[a][1]
    sins = sins + LS[a][1]
for a in range(len(LN)):
    palav = palav + LN[a][1]
    naos = naos + LN[a][1]

print("Quantidade de sim", QY)
print("Quantidade de sim", QN)
print('Total de palavras', len(LS)+len(LN))
print('Total relevantes', len(LS))
print('Total não relevantes', len(LN))

#Limpando a nova planilha
for a in range(len(excelte.Teste)):
    coluna11 = excelte.Teste[a].lower().split()
    for b in range(len(coluna11)):
        for punctuation in string.punctuation:
            coluna11[b] = coluna11[b].replace(punctuation, '')
        coluna11[b] = coluna11[b].replace('—', '')
        coluna11[b] = coluna11[b].replace('rt', '')
    while '' in coluna11:
        coluna11.remove('')
    excelte.Teste[i] = coluna11


I am working on the file "Projeto02.ipynb".
The first task is:
Importando as Bibliotecas que serão utilizadas. Esteja livre para adicionar outras.
Can you write Python code for it?

import tweepy
import math
import os.path
import pandas as pd
import json
from random import shuffle
import string


I am working on the file "Projeto02.ipynb".
The first task is:
Por fim, podemos comparar as probabilidades:
Can you write Python code for it?

L2 = []
for a in range(len(probSIM)):
    if probSIM[a]>probNAO[a]:
        L2.append('sim')
    else:
        L2.append('não')


I am working on the file "Projeto02.ipynb".
The first task is:
Salvando os dados em uma planilha Excel:
Can you write Python code for it?

#Verifica se o arquivo não existe para não substituir um conjunto pronto
if not os.path.isfile('./{0}.xlsx'.format(produto)):

    #Abre o arquivo para escrita
    writer = pd.ExcelWriter('{0}.xlsx'.format(produto))

    #divide o conjunto de mensagens em duas planilhas
    dft = pd.DataFrame({'Treinamento' : pd.Series(msgs[:t])})
    dft.to_excel(excel_writer = writer, sheet_name = 'Treinamento', index = False)

    dfc = pd.DataFrame({'Teste' : pd.Series(msgs[t:])})
    dfc.to_excel(excel_writer = writer, sheet_name = 'Teste', index = False)

    #fecha o arquivo
    writer.save()

exceltr=pd.read_excel("itau.xlsx")
excelte = pd.read_excel("itauteste.xlsx")
SIMs = []
NAOs = []
listaprobs = []


I am working on the file "Projeto02.ipynb".
The first task is:
Verificando a performance
Agora você deve testar o seu Classificador com a base de Testes.

Você deve extrair as seguintes medidas:
* Porcentagem de positivos falsos (marcados como relevante mas não são relevantes)
* Porcentagem de positivos verdadeiros (marcado como relevante e são relevantes)
* Porcentagem de negativos verdadeiros (marcado como não relevante e não são relevantes)
* Porcentagem de negativos falsos (marcado como não relevante e são relevantes)
Opcionalmente:
* Criar categorias intermediárias de relevância baseado na diferença de probabilidades. Exemplo: muito relevante, relevante, neutro, irrelevante e muito irrelevante.
Can you write Python code for it?

print("Positivos Falsos", (20/161))
print("Positivos verdadeiros", (16/39))
print("Irrelevantes verdadeiros", (141/161))
print("Irrelevantes Falsos", (23/39))


I am working on the file "Projeto1 Layout.ipynb".
The first task is:
Na célula a cima foi representada a quantidade absoluta e quantidade em porcentagem com o número de moradores por domicílio. Esta parte foi crucial para a formulação dos 'crosstabs' que virão a seguir, assim como para os gráficos e conclusões.
Can you write Python code for it?

#2009
#Cruzando variaveis (quantidade de moradores & perda de peso)
ct1 = pd.crosstab(novabase09.V0106, novabase09.V2120, margins=True, normalize = 'index')
#Cruzando variaveis (renda per capita & descarte de lixo)
ct2 = pd.crosstab(novabase09.V4622, novabase09.V0218, margins=True, normalize = 'index')

#Cruzando variaveis (renda per capita & quantidade de moradores)
ct3 = pd.crosstab(novabase09.V0106, novabase09.V4622, margins=True, normalize = 'index')

#Cruzando variaveis (estado & descarte de lixo )
ct4 = pd.crosstab(novabase09.UF, novabase09.V0218,  margins=True, normalize = 'index')

#Cruzando variaveis (estado & perda de peso)
ct5 =  pd.crosstab(novabase09.UF, novabase09.V2120,  margins=True, normalize = 'index')

#2013
#Cruzando variaveis (quantidade de moradores & perda de peso)
d1 = pd.crosstab(novabase13.V0105, novabase13.V2134, margins=True, normalize = 'index')

#Cruzando variaveis (renda per capita & quantidade de moradores)
d3 = pd.crosstab(novabase13.V0105, V4621_cat, margins=True, normalize = 'index')


I am working on the file "Projeto1 Layout.ipynb".
The first task is:
Na célula a cima foram escolhidas as variáveis que serão utilizadas. Em sequência, temos: Estado; quantidade de moradores por domicílio; descarte de lixo; perda de peso e renda per capita. Obs.: foram utilizados dois anos: 2009 e 2013.
Can you write Python code for it?

#Tabela de 2009
novabase09.V4622 = novabase09.V4622.astype('category')
novabase09.V4622.cat.categories = ('nenhuma', 'ate 1/4', 'de 1/4 ate 1/2', 'de 1/2 ate 1', 'de 1 ate 2', 'de 2 ate 3', 'de 3 ate 5', 'mais de 5','sem declaração')

novabase09.UF = novabase09.UF.astype('category')
novabase09.UF.cat.categories = ('rondonia', 'acre', 'amazonas', 'roraima', 'para', 'amapa', 'tocantins', 'maranhao', 'piaui', 'ceara', 'rio grande do norte', 'paraiba', 'pernambuco', 'alagoas','sergipe','bahia', 'minas gerais', 'espirito santo', 'rio de janeiro', 'sao paulo', 'parana','santa catarina','rio grande do sul', 'mato grosso do sul', 'mato grosso', 'goias', 'DF')

novabase09.V2120 = novabase09.V2120.astype('category')
novabase09.V2120.cat.categories = ('muito', 'medio', 'pouco')

novabase09.V0218 = novabase09.V0218.astype('category')
novabase09.V0218.cat.categories = ('coletado diretamente', 'coletado indiretamente', 'queimado', 'descarte em terreno baldio', 'descarte em rio ou mar', 'outro destino' )

#Tabela de 2013
V4621_cat = pd.cut(novabase13.V4621, bins=9)
faixa = (0,197,394, 788, 1576, 2364, 3152, 3940, 50000)
V4621_cat = pd.cut(novabase13.V4621, bins=faixa)
V4621_cat.value_counts(sort=False, normalize=True)*100

novabase13.UF = novabase13.UF.astype('category')
novabase13.UF.cat.categories = ('rondonia', 'acre', 'amazonas', 'roraima', 'para', 'amapa', 'tocantins', 'maranhao', 'piaui', 'ceara', 'rio grande do norte', 'paraiba', 'pernambuco', 'alagoas','sergipe','bahia', 'minas gerais', 'espirito santo', 'rio de janeiro', 'sao paulo', 'parana','santa catarina','rio grande do sul', 'mato grosso do sul', 'mato grosso', 'goias', 'DF')

novabase13.V2134 = novabase13.V2134.astype('category')
novabase13.V2134.cat.categories = ('perdeu peso', 'não perdeu peso')


I am working on the file "Projeto1 Layout.ipynb".
The first task is:
No código a cima foi feita a categorização das variáveis como quantitativas ou qualitativas.
Can you write Python code for it?

# Tabelas 2009
#print('Quantidade de moradores por domicilios:')
#a = novabase09.V0106.value_counts(sort=True)
#ap = novabase09.V0106.value_counts(True) * 100
#print(a,'\n')
#print("Quantidade em porcentagem")
#print(ap,'\n')

#print('Salarios per capita (frequencia):')
#b = novabase09.V4622.value_counts(sort=True)
#bp = novabase09.V4622.value_counts(True) * 100
#print(b,'\n')
#print("Quantidade em porcentagem")
#print(bp,'\n')

#print('A quantidade de peso perdido:')
#c = novabase09.V2120.value_counts(sort=True)
#cp = novabase09.V2120.value_counts(True) * 100
#print(c,'\n')
#print("Quantidade em porcentagem")
#print(cp,'\n')

#print('Forma de descarte do lixo:')
#d = novabase09.V0218.value_counts(sort=True)
#dp = novabase09.V0218.value_counts(True) * 100
#print(d,'\n')
#print("Quantidade em porcentagem")
#print(dp,'\n')


I am working on the file "Projeto1 Layout.ipynb".
The first task is:
No código a cima foram definidas as crosstabs (cruzamento de variáveis). Foram utilizadas apenas as crosstabs CT1; CT2; D1 e D3.
Can you write Python code for it?

#2009
#Gráficos sobre variáveis cruzadas:
ct1.plot(kind='bar', color=('red', 'orange','yellow'), stacked = True)
plt.title('CT1')
plt.xlabel('Quantidade de moradores no domicílio')
plt.ylabel('Quantidade de pessoas que perderam peso (%)')
plt.tight_layout()
plt.show()

#ct2.plot(kind='bar', color=('green', 'yellow','red', 'brown', 'blue', 'black'), stacked = True)
#plt.title('CT2')
#plt.xlabel('Renda per capita')
#plt.ylabel('Descarte de lixo (%)')
#plt.tight_layout()

ct3.plot(kind='bar', color=('gray', 'red','orange', 'yellow', 'green', 'blue', 'purple', 'pink', 'black'), stacked = True)
plt.title('CT3')
plt.xlabel('Quantidade de moradores')
plt.ylabel('Renda per capita (%)')
plt.tight_layout()
plt.show()

d1.plot(kind='bar', color=('red','green'), stacked = True)
plt.title('D1')
plt.xlabel('Quantidade de moradores no domicílio')
plt.ylabel('Quantidade de pessoas que perderam peso (%)')
plt.tight_layout()
plt.show()

d3.plot(kind='bar', color=('red','orange', 'yellow', 'green', 'blue', 'purple', 'pink', 'black'), stacked = True)
plt.title('D3')
plt.xlabel('Quantidade de moradores')
plt.ylabel('Renda per capita (%)')
plt.tight_layout()
plt.show()

#c3.plot(kind='bar', color=('grey', 'red','orange', 'yellow', 'green', 'blue', 'purple', 'pink', 'black'), stacked = True)
#plt.title('C3')
#plt.xlabel('Quantidade de moradores')
#plt.ylabel('Renda per capita (%)')
#plt.tight_layout()
#plt.show()

#ct4.plot(kind='bar', color=('green', 'yellow','red', 'brown', 'blue', 'black'), stacked = True)
#plt.title('CT4')
#plt.xlabel('Estado')
#plt.ylabel('Descarte de lixo (%)')
#plt.tight_layout()
#plt.show()

#ct5.plot(kind='bar', color=('red', 'yellow', 'green'), stacked = True)
#plt.title('CT5')
#plt.xlabel('Estado')
#plt.ylabel('Quantidade de pessoas que perderam peso (%)')
#plt.tight_layout()
#plt.show()

ct1.plot(kind='hist', alpha=0.5)
plt.title('HISTOGRAMA CT1')

ct1.plot(kind='box')
plt.title('QUANTIS CT1')

ct3.plot(kind='hist', alpha=0.5)
plt.title('HISTOGRAMA CT3')

ct3.plot(kind='box')
plt.title('QUANTIS...


I am working on the file "Projeto3Final.ipynb".
The first task is:
Embaralha e divide a planilha em 2. Uma contendo 15000 jogadores (train) e outra com aproximadamente 3000 jogadores (test)
Can you write Python code for it?

fifa = shuffle(fifa)

train = fifa.iloc[:15000]
test = fifa.iloc[15000:]


I am working on the file "Projeto3Final.ipynb".
The first task is:
Faremos, agora, a correlação entre as variáveis, com o alvo em Value. Agora o gráfico será plotado em barras e possuirá qual o valor da correlação entre as variáveis
Can you write Python code for it?

train.drop(["Photo", "Flag","Club Logo","Name"],1,inplace=True)

train.drop("ID",1,inplace=True)

l_encode = LabelEncoder()
obj_feat = ["Club", "Nationality","Preferred Positions"]
for var in obj_feat:
    train[var] = l_encode.fit_transform(train[var].astype(str))
train.shape

def clean_values(x):
    try:
        if len(x)>2:
            y = x[:2]
            return y
        else:
            return x
    except TypeError:
        return x
columns_to_clean = [col for col in train.columns if col not in ["Age","Nationality",
                                                                  "Overall","Potential",
                                                                 "Club","Value","Wage",
                                                                  "Special"]]
for col in columns_to_clean:
    train[col] = train[col].apply(lambda x : clean_values(x))

train = train.dropna(axis=1, how="any")

def modelfit(alg, dtrain, features, performCV=True, printFeatureImportance=True, cv_folds=10):

    alg.fit(dtrain[features],dtrain["Value"] )

    dtrain_predictions = alg.predict(dtrain[features])

    cv_score = cross_validation.cross_val_score(alg, dtrain[features], dtrain["Value"], cv=cv_folds,
                                                scoring='neg_mean_squared_error')
    cv_score = np.sqrt(np.abs(cv_score))

    print ("\nModel Report")
    print ("RMSE : %.4g" % np.sqrt(metrics.mean_squared_error(dtrain["Value"], dtrain_predictions)))
    print ("CV Score : Mean - %.4g | Std - %.4g | Min - %.4g | Max - %.4g" % (np.mean(cv_score),
                                                                             np.std(cv_score),np.min(cv_score),
                                                                              np.max(cv_score)))
    if printFeatureImportance:
        feat_imp = pd.Series(alg.feature_importances_, features).sort_values(ascending=False)
        feat_imp.plot(kind='bar', title='Feature Importances')
        plt.ylabel('Feature Importance...


I am working on the file "Projeto3Final.ipynb".
The first task is:
Importa a planilha que contem os dados utilizados. Esta planilha foi importada de www.kaggle.com
Can you write Python code for it?


I am working on the file "Projeto3Final.ipynb".
The first task is:
Normaliza a coluna 'Value' (valor do jogador) e a coluna 'Wage' (salário do jogador por semana)
Can you write Python code for it?

def extrai(value):
    out = value.replace('€', '')
    if 'M' in out:
        out = float(out.replace('M', ''))*1000000
    elif 'K' in value:
        out = float(out.replace('K', ''))*1000
    return float(out)

fifa['Value'] = fifa['Value'].apply(lambda x: extrai(x))
fifa['Wage'] = fifa['Wage'].apply(lambda x: extrai(x))


I am working on the file "Projeto3Final.ipynb".
The first task is:
Regressão de Ridge na planilha de teste
Can you write Python code for it?

ridgetest = ridge_regression(test, x, 1e-20)

plt.plot(test['Value'],ridgetest,'.', color="red")
plt.axis([0, 130000000, 0, 130000000])
plt.xlabel("Valor real")
plt.ylabel("Valor premeditado")
plt.show()


I am working on the file "Projeto3Final.ipynb".
The first task is:
Verifica, através da cor azul, quais as variáveis que mais influenciam a variável 'Value'. Quanto mais próximo de azul escuro for o quadrado da intersecção, maior é a correlação entre as duas variáveis.
Can you write Python code for it?

corr = fifa.drop('ID', axis = 1).corr()

fig = plt.figure(figsize=(20,16))

ax = sns.heatmap(corr,
                 xticklabels=corr.columns.values,
                 yticklabels=corr.columns.values,
                 linewidths=0.25,
                 vmax=1.0,
                 square=True,
                 cmap = 'PuBu',
                 linecolor='black',
                 annot=False
                )


I am working on the file "Projeto3Final.ipynb".
The first task is:
Verificando o quanto Overall influencia no Valor do jogador
Can you write Python code for it?

var = 'Value'
data = pd.concat([train['Overall'], train[var]], axis=1)
data.plot.scatter(x=var, y='Overall', ylim=(60,100), xlim=(0,150000000));


I am working on the file "Prometheus.ipynb".
The first task is:
Advanced usage
Can you write Python code for it?

# Import the module and set figure rendering to be inline. (You will not see any output yet.)
# If you already used the GUI, this cell does not need to be run.
%matplotlib notebook
from pyrho import *